annotate flys-backend/README @ 5779:ebec12def170

Datacage: Add a pool of builders to make it multi threadable. XML DOM is not thread safe. Therefore the old implementation only allowed one thread to use the builder at a time. As the complexity of the configuration has increased over time this has become a bottleneck of the whole application because it took quiet some time to build a result. Furthermore the builder code path is visited very frequent. So many concurrent requests were piled up resulting in long waits for the users. To mitigate this problem a round robin pool of builders is used now. Each of the pooled builders has an independent copy of the XML template and can be run in parallel. The number of builders is determined by the system property 'flys.datacage.pool.size'. It defaults to 4.
author Sascha L. Teichmann <teichmann@intevation.de>
date Sun, 21 Apr 2013 12:48:09 +0200
parents 95af0f3aabfb
children
rev   line source
164
324e12ab7a05 Converted schema to be PostgreSQL compatible.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
1 For the artifact database
324e12ab7a05 Converted schema to be PostgreSQL compatible.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
2 # su - postgres
324e12ab7a05 Converted schema to be PostgreSQL compatible.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
3 $ createuser --no-createrole --no-superuser --pwprompt --no-createdb artifacts
324e12ab7a05 Converted schema to be PostgreSQL compatible.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
4 $ createdb --encoding=UTF-8 --owner artifacts artifactsdb
324e12ab7a05 Converted schema to be PostgreSQL compatible.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
5
324e12ab7a05 Converted schema to be PostgreSQL compatible.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
6 For the flys database
324e12ab7a05 Converted schema to be PostgreSQL compatible.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
7
324e12ab7a05 Converted schema to be PostgreSQL compatible.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
8 # su - postgres
324e12ab7a05 Converted schema to be PostgreSQL compatible.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
9 $ createuser --no-createrole --no-superuser --pwprompt --no-createdb flys
324e12ab7a05 Converted schema to be PostgreSQL compatible.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
10 $ createdb --encoding=UTF-8 --owner flys flystest1
4765
90142c839fe5 README: Add instruction to do a simple standalone DateGuesser execution.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 164
diff changeset
11
5054
790353fadf30 Add minimal documenation how to build the single jar
Andre Heinecke <aheinecke@intevation.de>
parents: 4765
diff changeset
12
790353fadf30 Add minimal documenation how to build the single jar
Andre Heinecke <aheinecke@intevation.de>
parents: 4765
diff changeset
13 Build an importer package:
790353fadf30 Add minimal documenation how to build the single jar
Andre Heinecke <aheinecke@intevation.de>
parents: 4765
diff changeset
14 mvn -f pom.xml clean compile assembly:single
790353fadf30 Add minimal documenation how to build the single jar
Andre Heinecke <aheinecke@intevation.de>
parents: 4765
diff changeset
15 Alternatively with oracle:
790353fadf30 Add minimal documenation how to build the single jar
Andre Heinecke <aheinecke@intevation.de>
parents: 4765
diff changeset
16 mvn -f pom-oracle.xml clean compile assembly:single
790353fadf30 Add minimal documenation how to build the single jar
Andre Heinecke <aheinecke@intevation.de>
parents: 4765
diff changeset
17
4765
90142c839fe5 README: Add instruction to do a simple standalone DateGuesser execution.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 164
diff changeset
18 Standalone DateGuesser testing example:
90142c839fe5 README: Add instruction to do a simple standalone DateGuesser execution.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 164
diff changeset
19
90142c839fe5 README: Add instruction to do a simple standalone DateGuesser execution.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 164
diff changeset
20 mvn -e -Dexec.mainClass=de.intevation.flys.utils.DateGuesser -Dexec.args="110803" exec:java <<EOF
90142c839fe5 README: Add instruction to do a simple standalone DateGuesser execution.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 164
diff changeset
21 110803
90142c839fe5 README: Add instruction to do a simple standalone DateGuesser execution.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 164
diff changeset
22 EOF
90142c839fe5 README: Add instruction to do a simple standalone DateGuesser execution.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 164
diff changeset
23
5331
95af0f3aabfb AtFileParser, README: Added main-method for AT-File-Parsing debugging, doc.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5054
diff changeset
24 Some importer allow standalone usage:
95af0f3aabfb AtFileParser, README: Added main-method for AT-File-Parsing debugging, doc.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5054
diff changeset
25 mvn -e -Dexec.mainClass=de.intevation.flys.importer.parsers.AtFileParser -Dexec.args=/home/felix/.bashrc exec:java

http://dive4elements.wald.intevation.org