Mercurial > dive4elements > river
annotate flys-backend/contrib/run_geo.sh @ 5779:ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
XML DOM is not thread safe. Therefore the old implementation only allowed one thread
to use the builder at a time. As the complexity of the configuration
has increased over time this has become a bottleneck of the whole application
because it took quiet some time to build a result. Furthermore the builder code path
is visited very frequent. So many concurrent requests were piled up
resulting in long waits for the users.
To mitigate this problem a round robin pool of builders is used now.
Each of the pooled builders has an independent copy of the XML template
and can be run in parallel.
The number of builders is determined by the system property
'flys.datacage.pool.size'. It defaults to 4.
author | Sascha L. Teichmann <teichmann@intevation.de> |
---|---|
date | Sun, 21 Apr 2013 12:48:09 +0200 |
parents | 673f68bb19a4 |
children |
rev | line source |
---|---|
3683
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
1 #!/bin/sh |
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
2 |
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
3 # Required |
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
4 RIVER_PATH="/vol1/projects/Geospatial/flys-3.0/testdaten/Gewaesser/Saar" |
4971
fc7ebf104779
Update documentation and example run script
Andre Heinecke <aheinecke@intevation.de>
parents:
4873
diff
changeset
|
5 RIVER_NAME="Saar" |
4873
2b371e42a9af
Add HWS Lines parameters and ogr_connection example
Andre Heinecke <aheinecke@intevation.de>
parents:
3683
diff
changeset
|
6 |
2b371e42a9af
Add HWS Lines parameters and ogr_connection example
Andre Heinecke <aheinecke@intevation.de>
parents:
3683
diff
changeset
|
7 # Set this to your target database for Oracle |
3683
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
8 HOST=localhost |
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
9 USER=flys28 |
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
10 PASS=flys28 |
4873
2b371e42a9af
Add HWS Lines parameters and ogr_connection example
Andre Heinecke <aheinecke@intevation.de>
parents:
3683
diff
changeset
|
11 # Alternatively you can provide a direct connection string: |
2b371e42a9af
Add HWS Lines parameters and ogr_connection example
Andre Heinecke <aheinecke@intevation.de>
parents:
3683
diff
changeset
|
12 # OGR_CONNECTION="PG:dbname=flys host=localhost port=5432 user=flys password=flys" |
3683
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
13 |
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
14 # Optional |
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
15 VERBOSE=1 |
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
16 SKIP_AXIS=0 |
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
17 SKIP_KMS=0 |
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
18 SKIP_CROSSSECTIONS=0 |
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
19 SKIP_FIXPOINTS=0 |
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
20 SKIP_BUILDINGS=0 |
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
21 SKIP_FLOODPLAINS=0 |
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
22 SKIP_HYDR_BOUNDARIES=0 |
4873
2b371e42a9af
Add HWS Lines parameters and ogr_connection example
Andre Heinecke <aheinecke@intevation.de>
parents:
3683
diff
changeset
|
23 SKIP_HWS_LINES=0 |
2b371e42a9af
Add HWS Lines parameters and ogr_connection example
Andre Heinecke <aheinecke@intevation.de>
parents:
3683
diff
changeset
|
24 SKIP_HWS_POINTS=0 |
3683
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
25 SKIP_GAUGE_LOCATION=0 |
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
26 SKIP_CATCHMENTS=0 |
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
27 SKIP_UESG=0 |
5035
1f33012cc02a
Add skip-dgm option to example run script
Andre Heinecke <aheinecke@intevation.de>
parents:
4996
diff
changeset
|
28 SKIP_DGM=0 |
5353
d086ce6c13d3
Add jetties importer
Andre Heinecke <aheinecke@intevation.de>
parents:
5074
diff
changeset
|
29 SKIP_JETTIES=0 |
5545
ed149d5d7fb7
Add floodmarks importer
Andre Heinecke <aheinecke@intevation.de>
parents:
5353
diff
changeset
|
30 SKIP_FLOODMARKS=0 |
3683
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
31 |
5066
8208df10df79
Update run example scripts to work with the new packaging
Andre Heinecke <aheinecke@intevation.de>
parents:
5035
diff
changeset
|
32 # There should be no need to change anything below this line |
8208df10df79
Update run example scripts to work with the new packaging
Andre Heinecke <aheinecke@intevation.de>
parents:
5035
diff
changeset
|
33 |
3683
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
34 DIR=`dirname $0` |
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
35 DIR=`readlink -f "$DIR"` |
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
36 |
5074
90b9b80ffbbf
Fix example run script to work with the new packaging
Andre Heinecke <aheinecke@intevation.de>
parents:
5066
diff
changeset
|
37 OPTIONAL_LIBS="${DIR}"/opt |
5066
8208df10df79
Update run example scripts to work with the new packaging
Andre Heinecke <aheinecke@intevation.de>
parents:
5035
diff
changeset
|
38 if [ -d "$OPTIONAL_LIBS" ]; then |
8208df10df79
Update run example scripts to work with the new packaging
Andre Heinecke <aheinecke@intevation.de>
parents:
5035
diff
changeset
|
39 export PATH="$OPTIONAL_LIBS/bin:$PATH" |
8208df10df79
Update run example scripts to work with the new packaging
Andre Heinecke <aheinecke@intevation.de>
parents:
5035
diff
changeset
|
40 export LD_LIBRARY_PATH="$OPTIONAL_LIBS/lib:$LD_LIBRARY_PATH" |
8208df10df79
Update run example scripts to work with the new packaging
Andre Heinecke <aheinecke@intevation.de>
parents:
5035
diff
changeset
|
41 export LD_LIBRARY_PATH="$OPTIONAL_LIBS/lib64:$LD_LIBRARY_PATH" |
8208df10df79
Update run example scripts to work with the new packaging
Andre Heinecke <aheinecke@intevation.de>
parents:
5035
diff
changeset
|
42 export PYTHONPATH="$OPTIONAL_LIBS/lib/python2.6/site-packages:$PYTHONPATH" |
5074
90b9b80ffbbf
Fix example run script to work with the new packaging
Andre Heinecke <aheinecke@intevation.de>
parents:
5066
diff
changeset
|
43 export PYTHONPATH="$OPTIONAL_LIBS/lib64/python2.6/site-packages:$PYTHONPATH" |
90b9b80ffbbf
Fix example run script to work with the new packaging
Andre Heinecke <aheinecke@intevation.de>
parents:
5066
diff
changeset
|
44 export GDAL_DATA="$OPTIONAL_LIBS/share/gdal" |
5066
8208df10df79
Update run example scripts to work with the new packaging
Andre Heinecke <aheinecke@intevation.de>
parents:
5035
diff
changeset
|
45 fi |
8208df10df79
Update run example scripts to work with the new packaging
Andre Heinecke <aheinecke@intevation.de>
parents:
5035
diff
changeset
|
46 |
3683
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
47 exec python $DIR/shpimporter/shpimporter.py \ |
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
48 --directory $RIVER_PATH \ |
5563
673f68bb19a4
Geo-importer: accept river names with white space
Tom Gottfried <tom@intevation.de>
parents:
5545
diff
changeset
|
49 --river_name "$RIVER_NAME" \ |
4873
2b371e42a9af
Add HWS Lines parameters and ogr_connection example
Andre Heinecke <aheinecke@intevation.de>
parents:
3683
diff
changeset
|
50 --ogr_connection "$OGR_CONNECTION" \ |
3683
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
51 --host $HOST \ |
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
52 --user $USER \ |
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
53 --password $PASS \ |
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
54 --verbose $VERBOSE \ |
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
55 --skip_axis $SKIP_AXIS \ |
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
56 --skip_kms $SKIP_KMS \ |
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
57 --skip_crosssections $SKIP_CROSSSECTIONS \ |
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
58 --skip_fixpoints $SKIP_FIXPOINTS \ |
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
59 --skip_buildings $SKIP_BUILDINGS \ |
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
60 --skip_floodplains $SKIP_FLOODPLAINS \ |
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
61 --skip_hydr_boundaries $SKIP_HYDR_BOUNDARIES \ |
4298ef9bd4a6
Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff
changeset
|
62 --skip_gauge_locations $SKIP_GAUGE_LOCATION \ |
4873
2b371e42a9af
Add HWS Lines parameters and ogr_connection example
Andre Heinecke <aheinecke@intevation.de>
parents:
3683
diff
changeset
|
63 --skip_uesgs $SKIP_UESG \ |
2b371e42a9af
Add HWS Lines parameters and ogr_connection example
Andre Heinecke <aheinecke@intevation.de>
parents:
3683
diff
changeset
|
64 --skip_hws_lines $SKIP_HWS_LINES \ |
5035
1f33012cc02a
Add skip-dgm option to example run script
Andre Heinecke <aheinecke@intevation.de>
parents:
4996
diff
changeset
|
65 --skip_hws_points $SKIP_HWS_POINTS \ |
5353
d086ce6c13d3
Add jetties importer
Andre Heinecke <aheinecke@intevation.de>
parents:
5074
diff
changeset
|
66 --skip_dgm $SKIP_DGM \ |
5545
ed149d5d7fb7
Add floodmarks importer
Andre Heinecke <aheinecke@intevation.de>
parents:
5353
diff
changeset
|
67 --skip_jetties $SKIP_JETTIES \ |
ed149d5d7fb7
Add floodmarks importer
Andre Heinecke <aheinecke@intevation.de>
parents:
5353
diff
changeset
|
68 --skip_floodmarks $SKIP_FLOODMARKS |