Mercurial > dive4elements > river

annotate flys-backend/contrib/run_hydr_morph.sh @ 5097:415ac9bc5a56 dami

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Add importer packaging script and some doc about it
author Andre Heinecke <aheinecke@intevation.de>
date Mon, 25 Feb 2013 15:40:15 +0100
parents 4f65d833680f
children b9ae676a9afe
rev   line source
3683
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
1 #!/bin/bash
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
2
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
3 ######################### CONFIG OPTIONS ############################
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
4 INFO_GEW="/vol1/projects/Geospatial/flys-3.0/testdaten/saar.gew"
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
5 BACKEND_USER="flys28"
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
6 BACKEND_PASS="flys28"
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
7 BACKEND_HOST="czech-republic.atlas.intevation.de"
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
8 BACKEND_PORT="1521"
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
9 BACKEND_NAME="XE"
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
10 LOG4J_CONFIG="conf/log4j.properties"
5066
8208df10df79 Update run example scripts to work with the new packaging
Andre Heinecke <aheinecke@intevation.de>
parents: 4766
diff changeset
11 JAR="hydr_morph/importer.jar"
3683
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
12 #####################################################################
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
13
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
14
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
15 ########################## Oracle Settings ##########################
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
16 BACKEND_DB_PREFIX="jdbc:oracle:thin:@"
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
17 BACKEND_DB_DRIVER="oracle.jdbc.OracleDriver"
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
18 BACKEND_DB_DIALECT="org.hibernate.dialect.OracleDialect"
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
19 BACKEND_URL=$BACKEND_DB_PREFIX//$BACKEND_HOST:$BACKEND_PORT/$BACKEND_NAME
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
20 #####################################################################
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
21
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
22
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
23 ######################## Custom Importer Settings ###################
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
24 IMPORTER_DRY_RUN=false
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
25 IMPORTER_MAINVALUE_TYPES=QWTD
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
26 IMPORTER_ANNOTATION_TYPES="conf/annotation-types.xml"
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
27
5043
63ad79657453 importer: Sort options alphabetically.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5042
diff changeset
28 IMPORTER_SKIP_ANNOTATIONS=false
63ad79657453 importer: Sort options alphabetically.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5042
diff changeset
29 IMPORTER_SKIP_BWASTR=false
63ad79657453 importer: Sort options alphabetically.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5042
diff changeset
30 IMPORTER_SKIP_DA50S=false
63ad79657453 importer: Sort options alphabetically.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5042
diff changeset
31 IMPORTER_SKIP_DA66S=false
63ad79657453 importer: Sort options alphabetically.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5042
diff changeset
32 IMPORTER_SKIP_EXTRA_WST=false
63ad79657453 importer: Sort options alphabetically.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5042
diff changeset
33 IMPORTER_SKIP_FIXATIONS=false
63ad79657453 importer: Sort options alphabetically.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5042
diff changeset
34 IMPORTER_SKIP_FLOOD_WATER=false
63ad79657453 importer: Sort options alphabetically.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5042
diff changeset
35 IMPORTER_SKIP_FLOOD_PROTECTION=false
3683
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
36 IMPORTER_SKIP_GAUGES=false
4135
ac93f397018d Importer: Ignore historical discharge tables.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 3683
diff changeset
37 IMPORTER_SKIP_HISTORICAL_DISCHARGE_GAUGES=false
5043
63ad79657453 importer: Sort options alphabetically.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5042
diff changeset
38 IMPORTER_SKIP_HYKS=false
63ad79657453 importer: Sort options alphabetically.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5042
diff changeset
39 IMPORTER_SKIP_OFFICIAL_LINES=false
3683
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
40 IMPORTER_SKIP_PRFS=false
4766
655e86f025c1 run_hydr_morph.sh: Add SKIP_DA50 and SKIP_W80 flags.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 4711
diff changeset
41 IMPORTER_SKIP_W80S=false
5043
63ad79657453 importer: Sort options alphabetically.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5042
diff changeset
42 IMPORTER_SKIP_WST=false
3683
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
43
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
44 IMPORTER_SKIP_BED_HEIGHT_SINGLE=false
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
45 IMPORTER_SKIP_BED_HEIGHT_EPOCH=false
5043
63ad79657453 importer: Sort options alphabetically.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5042
diff changeset
46 IMPORTER_SKIP_FLOW_VELOCITY=false
3683
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
47 IMPORTER_SKIP_MORPHOLOGICAL_WIDTH=false
5043
63ad79657453 importer: Sort options alphabetically.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5042
diff changeset
48 IMPORTER_SKIP_SEDIMENT_DENSITY=false
3683
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
49 IMPORTER_SKIP_SEDIMENT_YIELD=false
5043
63ad79657453 importer: Sort options alphabetically.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5042
diff changeset
50 IMPORTER_SKIP_SQ_RELATION=false
3683
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
51 IMPORTER_SKIP_WATERLEVELS=false
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
52 IMPORTER_SKIP_WATERLEVEL_DIFFERENCES=false
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
53 #####################################################################
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
54
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
55 #MIN_MEMORY="8192m"
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
56 MIN_MEMORY="1024m"
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
57
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
58
5066
8208df10df79 Update run example scripts to work with the new packaging
Andre Heinecke <aheinecke@intevation.de>
parents: 4766
diff changeset
59 ######################### Run Importer ##############################
8208df10df79 Update run example scripts to work with the new packaging
Andre Heinecke <aheinecke@intevation.de>
parents: 4766
diff changeset
60 OPTIONAL_LIBS="${DIR}"/../opt
8208df10df79 Update run example scripts to work with the new packaging
Andre Heinecke <aheinecke@intevation.de>
parents: 4766
diff changeset
61 if [ -d "$OPTIONAL_LIBS" ]; then
8208df10df79 Update run example scripts to work with the new packaging
Andre Heinecke <aheinecke@intevation.de>
parents: 4766
diff changeset
62 export PATH="$OPTIONAL_LIBS/bin:$PATH"
8208df10df79 Update run example scripts to work with the new packaging
Andre Heinecke <aheinecke@intevation.de>
parents: 4766
diff changeset
63 export LD_LIBRARY_PATH="$OPTIONAL_LIBS/lib:$LD_LIBRARY_PATH"
8208df10df79 Update run example scripts to work with the new packaging
Andre Heinecke <aheinecke@intevation.de>
parents: 4766
diff changeset
64 export LD_LIBRARY_PATH="$OPTIONAL_LIBS/lib64:$LD_LIBRARY_PATH"
8208df10df79 Update run example scripts to work with the new packaging
Andre Heinecke <aheinecke@intevation.de>
parents: 4766
diff changeset
65 fi
3683
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
66
5066
8208df10df79 Update run example scripts to work with the new packaging
Andre Heinecke <aheinecke@intevation.de>
parents: 4766
diff changeset
67 export LC_ALL=de_DE@euro # Workaround encoding problem
3683
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
68
5066
8208df10df79 Update run example scripts to work with the new packaging
Andre Heinecke <aheinecke@intevation.de>
parents: 4766
diff changeset
69 exec java -jar \
3683
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
70 -Xmx$MIN_MEMORY \
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
71 -server \
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
72 -Dlog4j.configuration=file://`readlink -f $LOG4J_CONFIG` \
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
73 -Dflys.backend.importer.infogew.file=$INFO_GEW \
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
74 -Dflys.backend.main.value.types=$IMPORTER_MAINVALUE_TYPES \
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
75 -Dflys.backend.importer.annotation.types=$IMPORTER_ANNOTATION_TYPES \
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
76 -Dflys.backend.importer.dry.run=$IMPORTER_DRY_RUN \
5042
70650981a54d importer: Sort options alphabetically.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5018
diff changeset
77 -Dflys.backend.importer.skip.annotations=$IMPORTER_SKIP_ANNOTATIONS \
70650981a54d importer: Sort options alphabetically.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5018
diff changeset
78 -Dflys.backend.importer.skip.bed.height.single=$IMPORTER_SKIP_BED_HEIGHT_SINGLE \
70650981a54d importer: Sort options alphabetically.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5018
diff changeset
79 -Dflys.backend.importer.skip.bed.height.epoch=$IMPORTER_SKIP_BED_HEIGHT_EPOCH \
5018
e939098ee272 run_hydr_morph.sh,Config: Add flags for importer to skip bwastr_id.csv parsing.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 4766
diff changeset
80 -Dflys.backend.importer.skip.bwastr=$IMPORTER_SKIP_BWASTR \
4766
655e86f025c1 run_hydr_morph.sh: Add SKIP_DA50 and SKIP_W80 flags.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 4711
diff changeset
81 -Dflys.backend.importer.skip.da50s=$IMPORTER_SKIP_DA50S \
4711
35dd03e04e38 Added new parameter SKIP_DA66S to importer.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 4135
diff changeset
82 -Dflys.backend.importer.skip.da66s=$IMPORTER_SKIP_DA66S \
3683
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
83 -Dflys.backend.importer.skip.extra.wsts=$IMPORTER_SKIP_EXTRA_WST \
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
84 -Dflys.backend.importer.skip.fixations=$IMPORTER_SKIP_FIXATIONS \
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
85 -Dflys.backend.importer.skip.flood.water=$IMPORTER_SKIP_FLOOD_WATER \
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
86 -Dflys.backend.importer.skip.flood.protection=$IMPORTER_SKIP_FLOOD_PROTECTION \
5042
70650981a54d importer: Sort options alphabetically.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5018
diff changeset
87 -Dflys.backend.importer.skip.flow.velocity=$IMPORTER_SKIP_FLOW_VELOCITY \
70650981a54d importer: Sort options alphabetically.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5018
diff changeset
88 -Dflys.backend.importer.skip.gauges=$IMPORTER_SKIP_GAUGES \
70650981a54d importer: Sort options alphabetically.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5018
diff changeset
89 -Dflys.backend.importer.skip.historical.discharge.tables=$IMPORTER_SKIP_HISTORICAL_DISCHARGE_GAUGES \
70650981a54d importer: Sort options alphabetically.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5018
diff changeset
90 -Dflys.backend.importer.skip.hyks=$IMPORTER_SKIP_HYKS \
70650981a54d importer: Sort options alphabetically.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5018
diff changeset
91 -Dflys.backend.importer.skip.morphological.width=$IMPORTER_SKIP_MORPHOLOGICAL_WIDTH \
70650981a54d importer: Sort options alphabetically.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5018
diff changeset
92 -Dflys.backend.importer.skip.official.lines=$IMPORTER_SKIP_OFFICIAL_LINES \
70650981a54d importer: Sort options alphabetically.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5018
diff changeset
93 -Dflys.backend.importer.skip.prfs=$IMPORTER_SKIP_PRFS \
3683
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
94 -Dflys.backend.importer.skip.sediment.density=$IMPORTER_SKIP_SEDIMENT_DENSITY \
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
95 -Dflys.backend.importer.skip.sediment.yield=$IMPORTER_SKIP_SEDIMENT_YIELD \
5042
70650981a54d importer: Sort options alphabetically.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5018
diff changeset
96 -Dflys.backend.importer.skip.sq.relation=$IMPORTER_SKIP_SQ_RELATION \
70650981a54d importer: Sort options alphabetically.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5018
diff changeset
97 -Dflys.backend.importer.skip.w80s=$IMPORTER_SKIP_W80S \
3683
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
98 -Dflys.backend.importer.skip.waterlevels=$IMPORTER_SKIP_WATERLEVELS \
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
99 -Dflys.backend.importer.skip.waterlevel.differences=$IMPORTER_SKIP_WATERLEVEL_DIFFERENCES \
5042
70650981a54d importer: Sort options alphabetically.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5018
diff changeset
100 -Dflys.backend.importer.skip.wst=$IMPORTER_SKIP_WST \
3683
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
101 -Dflys.backend.user=$BACKEND_USER \
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
102 -Dflys.backend.password=$BACKEND_PASS \
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
103 -Dflys.backend.url=$BACKEND_URL \
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
104 -Dflys.backend.driver=$BACKEND_DB_DRIVER \
4298ef9bd4a6 Added run scripts for both importers.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
105 -Dflys.backend.dialect=$BACKEND_DB_DIALECT \
5066
8208df10df79 Update run example scripts to work with the new packaging
Andre Heinecke <aheinecke@intevation.de>
parents: 4766
diff changeset
106 $JAR
http://dive4elements.wald.intevation.org