dive4elements/river: backend/contrib/import

annotate backend/contrib/import_river.sh @ 8472:3f505fba522f

(issue1772) Use 0.001km tolarance instead of 0.1 to find matching km. There is no sense to use a define here. I will not write static final double NULLPOINTNULLNULLONE=0.001 if i just want to use that value and not any other value which may make sense in some other place. Using hardcoded values can have its merits and makes the code easier to read.

author	Andre Heinecke <andre.heinecke@intevation.de>
date	Tue, 18 Nov 2014 15:24:40 +0100
parents	d86cc6a17b7a
children	44ba71b0b75a

rev	line source
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	1 #!/bin/bash
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	2 # Import script for rivers
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	3 #
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	4 # Authors:
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	5 # Andre Heinecke <aheinecke@intevation.de>
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	6 #
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	7 # Copyright:
5168 650f94ec3122 Fix copyright on import_river.sh Andre Heinecke <aheinecke@intevation.de> parents: 5167 diff changeset	8 # Copyright (C) 2013 Intevation GmbH
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	9 #
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	10 # This program is free software; you can redistribute it and/or
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	11 # modify it under the terms of the GNU General Public License
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	12 # as published by the Free Software Foundation; either version 2
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	13 # of the License, or (at your option) any later version.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	14 #
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	15 # This program is distributed in the hope that it will be useful,
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	16 # but WITHOUT ANY WARRANTY; without even the implied warranty of
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	17 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	18 # GNU General Public License for more details.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	19 #
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	20 # You should have received a copy of the GNU General Public License
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	21 # along with this program; if not, write to the Free Software
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	22 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	23
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	24 set -e
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	25
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	26 # Default settings
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	27 DEFAULT_HOST=localhost
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	28 DEFAULT_PORT=1521
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	29 DEFAULT_USER=flys_dami
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	30 DEFAULT_PASS=flys_dami
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	31 DEFAULT_LOG=$PWD/logs
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	32 DEFAULT_BACKEND_NAME="XE"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	33 JAR="hydr_morph/importer.jar"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	34 IMPORTER_DRY_RUN=false
6449 25912df0facf Import main values of type '-' by default Tom Gottfried <tom.gottfried@intevation.de> parents: 6417 diff changeset	35 IMPORTER_MAINVALUE_TYPES=QWTD-
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	36 IMPORTER_ANNOTATION_TYPES="conf/annotation-types.xml"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	37
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	38
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	39 MIN_MEMORY="8024m"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	40
6651 8782ba5cb222 Importer scripts: add default encoding to correctly read gew-files Tom Gottfried <tom@intevation.de> parents: 6449 diff changeset	41 # Default encoding. Change here if necessary
8782ba5cb222 Importer scripts: add default encoding to correctly read gew-files Tom Gottfried <tom@intevation.de> parents: 6449 diff changeset	42 export LC_ALL=de_DE@euro
8782ba5cb222 Importer scripts: add default encoding to correctly read gew-files Tom Gottfried <tom@intevation.de> parents: 6449 diff changeset	43
5169 db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	44 if [ -z "$OPTIONAL_LIBS" ]; then
5212 14186e924534 Fix opt path if OPTIONAL_LIBS is not set Andre Heinecke <aheinecke@intevation.de> parents: 5192 diff changeset	45 OPTIONAL_LIBS="$(dirname $0)/opt"
5169 db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	46 fi
db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	47
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	48 if [ -d "$OPTIONAL_LIBS" ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	49 export PATH="$OPTIONAL_LIBS/bin:$PATH"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	50 export LD_LIBRARY_PATH="$OPTIONAL_LIBS/lib:$LD_LIBRARY_PATH"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	51 export LD_LIBRARY_PATH="$OPTIONAL_LIBS/lib64:$LD_LIBRARY_PATH"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	52 export PYTHONPATH="$OPTIONAL_LIBS/lib/python2.6/site-packages:$PYTHONPATH"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	53 export PYTHONPATH="$OPTIONAL_LIBS/lib64/python2.6/site-packages:$PYTHONPATH"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	54 export GDAL_DATA="$OPTIONAL_LIBS/share/gdal"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	55 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	56
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	57 usage(){
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	58 cat << EOF
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	59
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	60 usage: $0 [options] gew_file
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	61
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	62 Import a river described by the gew_file
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	63
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	64 OPTIONS:
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	65 -?, --help Show this message
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	66 -u, --username=<username> Database username. Default: $DEFAULT_USER
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	67 -w, --password=<password> Database password. Default: $DEFAULT_PASS
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	68 -h, --host=<host> Connect to database on host <host>.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	69 Default: $DEFAULT_HOST
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	70 -p, --port=<number> Use port number <number>. Default: $DEFAULT_PORT
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	71 -d, --db-name=<database_name> Name of the database / backend. Default: $DEFAULT_BACKEND_NAME
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	72 -l, --log-dir=<directory> Directory in which to create the log files.
7728 c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	73 Default: $PWD/logs
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	74 --postgres Database is PostgreSQL
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	75 --skip-hydro Skip import of hydrological data
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	76 --skip-morpho Skip import of morphological data
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	77 --skip-geo Skip import of geographic data
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	78 --skip-prf Skip import of cross section data
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	79 EOF
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	80 exit 0
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	81 }
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	82
7728 c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	83 OPTS=`getopt -o ?u:w:h:p:d:l: \
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	84 -l help,username:,password:,host:,port:,db-name:,log-dir:,skip-hydro,skip-morpho,skip-geo,skip-prf,postgres \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	85 -n $0 -- "$@"`
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	86 if [ $? != 0 ] ; then usage; fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	87 eval set -- "$OPTS"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	88 while true ; do
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	89 case "$1" in
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	90 "-?"\|"--help")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	91 usage;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	92 "--")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	93 shift
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	94 break;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	95 "-u"\|"--username")
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	96 DBUSER=$2
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	97 shift 2;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	98 "-w"\|"--password")
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	99 DBPASS=$2
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	100 shift 2;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	101 "-h"\|"--host")
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	102 DBHOST=$2
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	103 shift 2;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	104 "-p"\|"--port")
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	105 DBPORT=$2
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	106 shift 2;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	107 "-l"\|"--log-dir")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	108 LOG=$2
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	109 shift 2;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	110 "-d"\|"--db-name")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	111 BACKEND_NAME=$2
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	112 shift 2;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	113 "--skip-hydro")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	114 SKIP_HYDRO="TRUE"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	115 shift;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	116 "--skip-morpho")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	117 SKIP_MORPHO="TRUE"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	118 shift;;
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	119 "--skip-prf")
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	120 SKIP_PRF="TRUE"
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	121 shift;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	122 "--skip-geo")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	123 SKIP_GEO="TRUE"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	124 shift;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	125 "--postgres")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	126 POSTGRES="TRUE"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	127 shift;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	128 *)
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	129 echo "Unknown Option $1"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	130 usage;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	131 esac
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	132 done
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	133
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	134 if [ -z $DBUSER ]; then
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	135 DBUSER=$DEFAULT_USER
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	136 fi
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	137 if [ -z $DBPASS ]; then
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	138 DBPASS=$DEFAULT_PASS
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	139 fi
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	140 if [ -z $DBPORT ]; then
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	141 DBPORT=$DEFAULT_PORT
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	142 fi
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	143 if [ -z $DBHOST ]; then
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	144 DBHOST=$DEFAULT_HOST
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	145 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	146 if [ -z $BACKEND_NAME ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	147 BACKEND_NAME=$DEFAULT_BACKEND_NAME
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	148 fi
7728 c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	149 if [ -z $LOG ]; then
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	150 LOG=$DEFAULT_LOG
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	151 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	152
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	153 if [ $# != 1 ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	154 usage
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	155 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	156
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	157 if [ ! -r $1 ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	158 echo "Could not open $1 please ensure it exists and is readable"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	159 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	160
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	161 GEW_FILE="$1"
5459 7c646d4d5103 be more flexible with river names (e.g. containing white space) in import script Tom Gottfried <tom@intevation.de> parents: 5355 diff changeset	162 RIVER_NAME=$(grep "Gew.sser" "$1" \| sed 's/Gew.sser: //')
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	163 DATE=$(date +%Y.%m.%d_%H%M)
5459 7c646d4d5103 be more flexible with river names (e.g. containing white space) in import script Tom Gottfried <tom@intevation.de> parents: 5355 diff changeset	164 LOG_DIR=${LOG}/`basename $GEW_FILE .gew`-$DATE
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	165 mkdir -p ${LOG_DIR}
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	166
7728 c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	167 cat > "$LOG_DIR/log4j.properties" << "EOF"
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	168 log4j.rootLogger=DEBUG, IMPORTER
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	169 log4j.appender.IMPORTER.layout=org.apache.log4j.PatternLayout
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	170 log4j.appender.IMPORTER.layout.ConversionPattern=%d [%t] %-5p %c - %m%n
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	171 log4j.appender.IMPORTER=org.apache.log4j.RollingFileAppender
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	172 log4j.appender.IMPORTER.File=./import.log
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	173 log4j.appender.IMPORTER.MaxFileSize=100000KB
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	174 log4j.appender.IMPORTER.MaxBackupIndex=10
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	175 EOF
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	176
5355 480d2f1f61cc Fix import_rivers script for postgres Andre Heinecke <aheinecke@intevation.de> parents: 5212 diff changeset	177 if [ "$POSTGRES" = "TRUE" ]; then
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	178 JAR=$(echo "$JAR" \| sed 's/importer/importer_psql/')
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	179 if [ ! -r "$JAR" ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	180 echo "Could not find Postgres importer $JAR"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	181 exit 1
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	182 fi
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	183 OGR_CONNECTION="PG:dbname=$BACKEND_NAME host=$DBHOST port=$DBPORT \
4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	184 user=$DBUSER password=$DBPASS"
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	185 BACKEND_DB_PREFIX="jdbc:postgresql:"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	186 BACKEND_DB_DRIVER="org.postgresql.Driver"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	187 BACKEND_DB_DIALECT="org.hibernate.dialect.PostgreSQLDialect"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	188 else
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	189 BACKEND_DB_PREFIX="jdbc:oracle:thin:@"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	190 BACKEND_DB_DRIVER="oracle.jdbc.OracleDriver"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	191 BACKEND_DB_DIALECT="org.hibernate.dialect.OracleDialect"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	192 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	193
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	194 BACKEND_URL=$BACKEND_DB_PREFIX//$DBHOST:$DBPORT/$BACKEND_NAME
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	195
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	196 echo "Importing $RIVER_NAME into $BACKEND_URL."
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	197
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	198 import_hydro(){
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	199 LOG_FILE=${LOG_DIR}/hydro.log
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	200 echo Importing Hydrological data.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	201 echo Logging into: $LOG_FILE
7728 c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	202 sed -i 's!$log4j.appender.IMPORTER.File=$.*!\1'"$LOG_FILE"'!' \
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	203 $LOG_DIR/log4j.properties
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	204 java -jar \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	205 -Xmx$MIN_MEMORY \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	206 -server \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	207 -Dlog4j.configuration=file://$LOG_DIR/log4j.properties \
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	208 -Dflys.backend.user=$DBUSER \
4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	209 -Dflys.backend.password=$DBPASS \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	210 -Dflys.backend.url=$BACKEND_URL \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	211 -Dflys.backend.driver=$BACKEND_DB_DRIVER \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	212 -Dflys.backend.dialect=$BACKEND_DB_DIALECT \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	213 -Dflys.backend.importer.infogew.file="$GEW_FILE" \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	214 -Dflys.backend.main.value.types=$IMPORTER_MAINVALUE_TYPES \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	215 -Dflys.backend.importer.annotation.types=$IMPORTER_ANNOTATION_TYPES \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	216 -Dflys.backend.importer.dry.run=$IMPORTER_DRY_RUN \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	217 -Dflys.backend.importer.skip.annotations=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	218 -Dflys.backend.importer.skip.bwastr=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	219 -Dflys.backend.importer.skip.extra.wsts=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	220 -Dflys.backend.importer.skip.fixations=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	221 -Dflys.backend.importer.skip.flood.water=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	222 -Dflys.backend.importer.skip.flood.protection=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	223 -Dflys.backend.importer.skip.gauges=false \
7326 5e621410db05 Importer: skip historical discharge tables per default as they come via ETL. Tom Gottfried <tom@intevation.de> parents: 6825 diff changeset	224 -Dflys.backend.importer.skip.historical.discharge.tables=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	225 -Dflys.backend.importer.skip.hyks=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	226 -Dflys.backend.importer.skip.official.lines=false \
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	227 -Dflys.backend.importer.skip.prfs=true \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	228 -Dflys.backend.importer.skip.w80s=true \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	229 -Dflys.backend.importer.skip.w80.csvs=true \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	230 -Dflys.backend.importer.skip.da50s=true \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	231 -Dflys.backend.importer.skip.da66s=true \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	232 -Dflys.backend.importer.skip.wst=false \
6417 14b5674e57a4 Importer: add skip flag for measurement stations to scripts Tom Gottfried <tom@intevation.de> parents: 5838 diff changeset	233 -Dflys.backend.importer.skip.measurement.stations=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	234 -Dflys.backend.importer.skip.waterlevel.differences=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	235 -Dflys.backend.importer.skip.waterlevels=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	236 -Dflys.backend.importer.skip.sq.relation=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	237 -Dflys.backend.importer.skip.sediment.density=true \
8056 d86cc6a17b7a Importer: Import sediment load at measurement stations. Tom Gottfried <tom@intevation.de> parents: 8032 diff changeset	238 -Dflys.backend.importer.skip.sediment.load=true \
8032 fd3a24336e6a SCHEMA CHANGE and Importer (only longitudinal section data so far): only grain fractions given now in schema are valid. We are handling sediment loads, not yields. "Tom Gottfried <tom@intevation.de>" parents: 7925 diff changeset	239 -Dflys.backend.importer.skip.sediment.load.ls=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	240 -Dflys.backend.importer.skip.morphological.width=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	241 -Dflys.backend.importer.skip.flow.velocity=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	242 -Dflys.backend.importer.skip.bed.height.single=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	243 $JAR
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	244 }
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	245
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	246 import_morpho(){
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	247 LOG_FILE=${LOG_DIR}/morpho.log
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	248 echo Importing Morphological data.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	249 echo Logging into: $LOG_FILE
7728 c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	250 sed -i 's!$log4j.appender.IMPORTER.File=$.*!\1'"$LOG_FILE"'!' \
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	251 $LOG_DIR/log4j.properties
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	252 java -jar \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	253 -Xmx$MIN_MEMORY \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	254 -server \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	255 -Dlog4j.configuration=file://$LOG_DIR/log4j.properties \
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	256 -Dflys.backend.user=$DBUSER \
4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	257 -Dflys.backend.password=$DBPASS \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	258 -Dflys.backend.url=$BACKEND_URL \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	259 -Dflys.backend.driver=$BACKEND_DB_DRIVER \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	260 -Dflys.backend.dialect=$BACKEND_DB_DIALECT \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	261 -Dflys.backend.importer.infogew.file="$GEW_FILE" \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	262 -Dflys.backend.main.value.types=$IMPORTER_MAINVALUE_TYPES \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	263 -Dflys.backend.importer.annotation.types=$IMPORTER_ANNOTATION_TYPES \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	264 -Dflys.backend.importer.dry.run=$IMPORTER_DRY_RUN \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	265 -Dflys.backend.importer.skip.annotations=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	266 -Dflys.backend.importer.skip.bwastr=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	267 -Dflys.backend.importer.skip.extra.wsts=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	268 -Dflys.backend.importer.skip.fixations=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	269 -Dflys.backend.importer.skip.flood.water=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	270 -Dflys.backend.importer.skip.flood.protection=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	271 -Dflys.backend.importer.skip.gauges=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	272 -Dflys.backend.importer.skip.historical.discharge.tables=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	273 -Dflys.backend.importer.skip.hyks=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	274 -Dflys.backend.importer.skip.official.lines=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	275 -Dflys.backend.importer.skip.prfs=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	276 -Dflys.backend.importer.skip.w80s=true \
6706 c3e38ea87964 Importer scripts: add skip flag for W80-CSV to import_river.sh Tom Gottfried <tom@intevation.de> parents: 6658 diff changeset	277 -Dflys.backend.importer.skip.w80.csvs=true \
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	278 -Dflys.backend.importer.skip.da50s=true \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	279 -Dflys.backend.importer.skip.da66s=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	280 -Dflys.backend.importer.skip.wst=true \
6417 14b5674e57a4 Importer: add skip flag for measurement stations to scripts Tom Gottfried <tom@intevation.de> parents: 5838 diff changeset	281 -Dflys.backend.importer.skip.measurement.stations=false \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	282 -Dflys.backend.importer.skip.waterlevel.differences=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	283 -Dflys.backend.importer.skip.waterlevels=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	284 -Dflys.backend.importer.skip.sq.relation=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	285 -Dflys.backend.importer.skip.sediment.density=false \
8056 d86cc6a17b7a Importer: Import sediment load at measurement stations. Tom Gottfried <tom@intevation.de> parents: 8032 diff changeset	286 -Dflys.backend.importer.skip.sediment.load=false \
8032 fd3a24336e6a SCHEMA CHANGE and Importer (only longitudinal section data so far): only grain fractions given now in schema are valid. We are handling sediment loads, not yields. "Tom Gottfried <tom@intevation.de>" parents: 7925 diff changeset	287 -Dflys.backend.importer.skip.sediment.load.ls=false \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	288 -Dflys.backend.importer.skip.morphological.width=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	289 -Dflys.backend.importer.skip.flow.velocity=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	290 -Dflys.backend.importer.skip.bed.height.single=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	291 $JAR
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	292 }
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	293
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	294 import_prf(){
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	295 LOG_FILE=${LOG_DIR}/prf.log
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	296 echo Importing cross section data.
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	297 echo Logging into: $LOG_FILE
7728 c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	298 sed -i 's!$log4j.appender.IMPORTER.File=$.*!\1'"$LOG_FILE"'!' \
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	299 $LOG_DIR/log4j.properties
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	300 java -jar \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	301 -Xmx$MIN_MEMORY \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	302 -server \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	303 -Dlog4j.configuration=file://$LOG_DIR/log4j.properties \
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	304 -Dflys.backend.user=$DBUSER \
4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	305 -Dflys.backend.password=$DBPASS \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	306 -Dflys.backend.url=$BACKEND_URL \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	307 -Dflys.backend.driver=$BACKEND_DB_DRIVER \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	308 -Dflys.backend.dialect=$BACKEND_DB_DIALECT \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	309 -Dflys.backend.importer.infogew.file="$GEW_FILE" \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	310 -Dflys.backend.main.value.types=$IMPORTER_MAINVALUE_TYPES \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	311 -Dflys.backend.importer.annotation.types=$IMPORTER_ANNOTATION_TYPES \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	312 -Dflys.backend.importer.dry.run=$IMPORTER_DRY_RUN \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	313 -Dflys.backend.importer.skip.annotations=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	314 -Dflys.backend.importer.skip.bwastr=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	315 -Dflys.backend.importer.skip.extra.wsts=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	316 -Dflys.backend.importer.skip.fixations=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	317 -Dflys.backend.importer.skip.flood.water=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	318 -Dflys.backend.importer.skip.flood.protection=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	319 -Dflys.backend.importer.skip.gauges=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	320 -Dflys.backend.importer.skip.historical.discharge.tables=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	321 -Dflys.backend.importer.skip.hyks=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	322 -Dflys.backend.importer.skip.official.lines=true \
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	323 -Dflys.backend.importer.skip.prfs=false \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	324 -Dflys.backend.importer.skip.w80s=false \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	325 -Dflys.backend.importer.skip.w80.csvs=false \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	326 -Dflys.backend.importer.skip.da50s=false \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	327 -Dflys.backend.importer.skip.da66s=false \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	328 -Dflys.backend.importer.skip.wst=true \
6417 14b5674e57a4 Importer: add skip flag for measurement stations to scripts Tom Gottfried <tom@intevation.de> parents: 5838 diff changeset	329 -Dflys.backend.importer.skip.measurement.stations=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	330 -Dflys.backend.importer.skip.waterlevel.differences=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	331 -Dflys.backend.importer.skip.waterlevels=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	332 -Dflys.backend.importer.skip.sq.relation=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	333 -Dflys.backend.importer.skip.sediment.density=true \
8056 d86cc6a17b7a Importer: Import sediment load at measurement stations. Tom Gottfried <tom@intevation.de> parents: 8032 diff changeset	334 -Dflys.backend.importer.skip.sediment.load=true \
8032 fd3a24336e6a SCHEMA CHANGE and Importer (only longitudinal section data so far): only grain fractions given now in schema are valid. We are handling sediment loads, not yields. "Tom Gottfried <tom@intevation.de>" parents: 7925 diff changeset	335 -Dflys.backend.importer.skip.sediment.load.ls=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	336 -Dflys.backend.importer.skip.morphological.width=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	337 -Dflys.backend.importer.skip.flow.velocity=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	338 -Dflys.backend.importer.skip.bed.height.single=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	339 $JAR
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	340 }
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	341
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	342 import_geo(){
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	343 LOG_FILE=${LOG_DIR}/geo.log
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	344 echo Importing Geographic data.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	345 echo Logging into: $LOG_FILE
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	346
5169 db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	347 RIVER_PATH=$(grep "WSTDatei:" "$GEW_FILE" \| awk '{print $2}')
db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	348 RIVER_PATH=$(dirname "$RIVER_PATH")/../..
db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	349 RIVER_PATH=$(readlink -f "$RIVER_PATH")
db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	350
db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	351 exec python $(dirname $0)/geodaesie/shpimporter.py \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	352 --directory $RIVER_PATH \
5563 673f68bb19a4 Geo-importer: accept river names with white space Tom Gottfried <tom@intevation.de> parents: 5459 diff changeset	353 --river_name "$RIVER_NAME" \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	354 --ogr_connection "$OGR_CONNECTION" \
6658 89f856e24d27 Importer scripts: shpimporter needs backend name to play with Oracle. Tom Gottfried <tom@intevation.de> parents: 6651 diff changeset	355 --host $DBHOST/$BACKEND_NAME \
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	356 --user $DBUSER \
4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	357 --password $DBPASS \
5169 db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	358 --verbose 1 \
db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	359 > "$LOG_FILE" 2>&1
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	360 }
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	361
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	362
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	363 if [ "$SKIP_HYDRO" != "TRUE" ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	364 import_hydro
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	365 fi
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	366 if [ "$SKIP_PRF" != "TRUE" ]; then
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	367 import_prf
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	368 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	369 if [ "$SKIP_MORPHO" != "TRUE" ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	370 import_morpho
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	371 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	372 if [ "$SKIP_GEO" != "TRUE" ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	373 import_geo
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	374 fi

Mercurial > dive4elements > river

annotate backend/contrib/import_river.sh @ 8472:3f505fba522f