dive4elements/river: backend/contrib/import

annotate backend/contrib/import_river.sh @ 7786:61a9e7df0728

INSTALL_DE: small corrections and (hopefully) improvements.

author	Tom Gottfried <tom@intevation.de>
date	Wed, 05 Mar 2014 17:52:19 +0100
parents	c842ebb0db86
children	06a4071143b0

rev	line source
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	1 #!/bin/bash
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	2 # Import script for rivers
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	3 #
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	4 # Authors:
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	5 # Andre Heinecke <aheinecke@intevation.de>
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	6 #
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	7 # Copyright:
5168 650f94ec3122 Fix copyright on import_river.sh Andre Heinecke <aheinecke@intevation.de> parents: 5167 diff changeset	8 # Copyright (C) 2013 Intevation GmbH
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	9 #
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	10 # This program is free software; you can redistribute it and/or
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	11 # modify it under the terms of the GNU General Public License
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	12 # as published by the Free Software Foundation; either version 2
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	13 # of the License, or (at your option) any later version.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	14 #
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	15 # This program is distributed in the hope that it will be useful,
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	16 # but WITHOUT ANY WARRANTY; without even the implied warranty of
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	17 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	18 # GNU General Public License for more details.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	19 #
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	20 # You should have received a copy of the GNU General Public License
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	21 # along with this program; if not, write to the Free Software
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	22 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	23
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	24 set -e
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	25
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	26 # Default settings
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	27 DEFAULT_HOST=localhost
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	28 DEFAULT_PORT=1521
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	29 DEFAULT_USER=flys_dami
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	30 DEFAULT_PASS=flys_dami
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	31 DEFAULT_LOG=$PWD/logs
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	32 DEFAULT_BACKEND_NAME="XE"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	33 JAR="hydr_morph/importer.jar"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	34 IMPORTER_DRY_RUN=false
6449 25912df0facf Import main values of type '-' by default Tom Gottfried <tom.gottfried@intevation.de> parents: 6417 diff changeset	35 IMPORTER_MAINVALUE_TYPES=QWTD-
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	36 IMPORTER_ANNOTATION_TYPES="conf/annotation-types.xml"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	37
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	38
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	39 MIN_MEMORY="8024m"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	40
6651 8782ba5cb222 Importer scripts: add default encoding to correctly read gew-files Tom Gottfried <tom@intevation.de> parents: 6449 diff changeset	41 # Default encoding. Change here if necessary
8782ba5cb222 Importer scripts: add default encoding to correctly read gew-files Tom Gottfried <tom@intevation.de> parents: 6449 diff changeset	42 export LC_ALL=de_DE@euro
8782ba5cb222 Importer scripts: add default encoding to correctly read gew-files Tom Gottfried <tom@intevation.de> parents: 6449 diff changeset	43
5169 db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	44 if [ -z "$OPTIONAL_LIBS" ]; then
5212 14186e924534 Fix opt path if OPTIONAL_LIBS is not set Andre Heinecke <aheinecke@intevation.de> parents: 5192 diff changeset	45 OPTIONAL_LIBS="$(dirname $0)/opt"
5169 db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	46 fi
db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	47
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	48 if [ -d "$OPTIONAL_LIBS" ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	49 export PATH="$OPTIONAL_LIBS/bin:$PATH"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	50 export LD_LIBRARY_PATH="$OPTIONAL_LIBS/lib:$LD_LIBRARY_PATH"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	51 export LD_LIBRARY_PATH="$OPTIONAL_LIBS/lib64:$LD_LIBRARY_PATH"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	52 export PYTHONPATH="$OPTIONAL_LIBS/lib/python2.6/site-packages:$PYTHONPATH"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	53 export PYTHONPATH="$OPTIONAL_LIBS/lib64/python2.6/site-packages:$PYTHONPATH"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	54 export GDAL_DATA="$OPTIONAL_LIBS/share/gdal"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	55 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	56
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	57 usage(){
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	58 cat << EOF
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	59
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	60 usage: $0 [options] gew_file
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	61
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	62 Import a river described by the gew_file
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	63
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	64 OPTIONS:
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	65 -?, --help Show this message
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	66 -u, --username=<username> Database username. Default: $DEFAULT_USER
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	67 -w, --password=<password> Database password. Default: $DEFAULT_PASS
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	68 -h, --host=<host> Connect to database on host <host>.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	69 Default: $DEFAULT_HOST
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	70 -p, --port=<number> Use port number <number>. Default: $DEFAULT_PORT
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	71 -d, --db-name=<database_name> Name of the database / backend. Default: $DEFAULT_BACKEND_NAME
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	72 -l, --log-dir=<directory> Directory in which to create the log files.
7728 c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	73 Default: $PWD/logs
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	74 --postgres Database is PostgreSQL
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	75 --skip-hydro Skip import of hydrological data
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	76 --skip-morpho Skip import of morphological data
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	77 --skip-geo Skip import of geographic data
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	78 --skip-prf Skip import of cross section data
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	79 EOF
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	80 exit 0
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	81 }
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	82
7728 c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	83 OPTS=`getopt -o ?u:w:h:p:d:l: \
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	84 -l help,username:,password:,host:,port:,db-name:,log-dir:,skip-hydro,skip-morpho,skip-geo,skip-prf,postgres \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	85 -n $0 -- "$@"`
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	86 if [ $? != 0 ] ; then usage; fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	87 eval set -- "$OPTS"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	88 while true ; do
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	89 case "$1" in
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	90 "-?"\|"--help")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	91 usage;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	92 "--")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	93 shift
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	94 break;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	95 "-u"\|"--username")
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	96 DBUSER=$2
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	97 shift 2;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	98 "-w"\|"--password")
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	99 DBPASS=$2
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	100 shift 2;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	101 "-h"\|"--host")
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	102 DBHOST=$2
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	103 shift 2;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	104 "-p"\|"--port")
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	105 DBPORT=$2
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	106 shift 2;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	107 "-l"\|"--log-dir")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	108 LOG=$2
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	109 shift 2;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	110 "-d"\|"--db-name")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	111 BACKEND_NAME=$2
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	112 shift 2;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	113 "--skip-hydro")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	114 SKIP_HYDRO="TRUE"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	115 shift;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	116 "--skip-morpho")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	117 SKIP_MORPHO="TRUE"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	118 shift;;
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	119 "--skip-prf")
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	120 SKIP_PRF="TRUE"
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	121 shift;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	122 "--skip-geo")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	123 SKIP_GEO="TRUE"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	124 shift;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	125 "--postgres")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	126 POSTGRES="TRUE"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	127 shift;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	128 *)
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	129 echo "Unknown Option $1"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	130 usage;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	131 esac
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	132 done
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	133
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	134 if [ -z $DBUSER ]; then
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	135 DBUSER=$DEFAULT_USER
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	136 fi
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	137 if [ -z $DBPASS ]; then
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	138 DBPASS=$DEFAULT_PASS
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	139 fi
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	140 if [ -z $DBPORT ]; then
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	141 DBPORT=$DEFAULT_PORT
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	142 fi
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	143 if [ -z $DBHOST ]; then
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	144 DBHOST=$DEFAULT_HOST
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	145 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	146 if [ -z $BACKEND_NAME ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	147 BACKEND_NAME=$DEFAULT_BACKEND_NAME
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	148 fi
7728 c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	149 if [ -z $LOG ]; then
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	150 LOG=$DEFAULT_LOG
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	151 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	152
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	153 if [ $# != 1 ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	154 usage
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	155 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	156
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	157 if [ ! -r $1 ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	158 echo "Could not open $1 please ensure it exists and is readable"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	159 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	160
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	161 GEW_FILE="$1"
5459 7c646d4d5103 be more flexible with river names (e.g. containing white space) in import script Tom Gottfried <tom@intevation.de> parents: 5355 diff changeset	162 RIVER_NAME=$(grep "Gew.sser" "$1" \| sed 's/Gew.sser: //')
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	163 DATE=$(date +%Y.%m.%d_%H%M)
5459 7c646d4d5103 be more flexible with river names (e.g. containing white space) in import script Tom Gottfried <tom@intevation.de> parents: 5355 diff changeset	164 LOG_DIR=${LOG}/`basename $GEW_FILE .gew`-$DATE
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	165 mkdir -p ${LOG_DIR}
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	166
7728 c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	167 cat > "$LOG_DIR/log4j.properties" << "EOF"
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	168 log4j.rootLogger=DEBUG, IMPORTER
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	169 log4j.appender.IMPORTER.layout=org.apache.log4j.PatternLayout
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	170 log4j.appender.IMPORTER.layout.ConversionPattern=%d [%t] %-5p %c - %m%n
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	171 log4j.appender.IMPORTER=org.apache.log4j.RollingFileAppender
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	172 log4j.appender.IMPORTER.File=./import.log
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	173 log4j.appender.IMPORTER.MaxFileSize=100000KB
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	174 log4j.appender.IMPORTER.MaxBackupIndex=10
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	175 EOF
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	176
5355 480d2f1f61cc Fix import_rivers script for postgres Andre Heinecke <aheinecke@intevation.de> parents: 5212 diff changeset	177 if [ "$POSTGRES" = "TRUE" ]; then
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	178 JAR=$(echo "$JAR" \| sed 's/importer/importer_psql/')
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	179 if [ ! -r "$JAR" ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	180 echo "Could not find Postgres importer $JAR"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	181 exit 1
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	182 fi
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	183 OGR_CONNECTION="PG:dbname=$BACKEND_NAME host=$DBHOST port=$DBPORT \
4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	184 user=$DBUSER password=$DBPASS"
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	185 BACKEND_DB_PREFIX="jdbc:postgresql:"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	186 BACKEND_DB_DRIVER="org.postgresql.Driver"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	187 BACKEND_DB_DIALECT="org.hibernate.dialect.PostgreSQLDialect"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	188 else
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	189 BACKEND_DB_PREFIX="jdbc:oracle:thin:@"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	190 BACKEND_DB_DRIVER="oracle.jdbc.OracleDriver"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	191 BACKEND_DB_DIALECT="org.hibernate.dialect.OracleDialect"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	192 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	193
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	194 BACKEND_URL=$BACKEND_DB_PREFIX//$DBHOST:$DBPORT/$BACKEND_NAME
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	195
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	196 echo "Importing $RIVER_NAME into $BACKEND_URL."
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	197
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	198 import_hydro(){
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	199 LOG_FILE=${LOG_DIR}/hydro.log
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	200 echo Importing Hydrological data.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	201 echo Logging into: $LOG_FILE
7728 c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	202 sed -i 's!$log4j.appender.IMPORTER.File=$.*!\1'"$LOG_FILE"'!' \
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	203 $LOG_DIR/log4j.properties
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	204 java -jar \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	205 -Xmx$MIN_MEMORY \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	206 -server \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	207 -Dlog4j.configuration=file://$LOG_DIR/log4j.properties \
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	208 -Dflys.backend.user=$DBUSER \
4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	209 -Dflys.backend.password=$DBPASS \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	210 -Dflys.backend.url=$BACKEND_URL \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	211 -Dflys.backend.driver=$BACKEND_DB_DRIVER \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	212 -Dflys.backend.dialect=$BACKEND_DB_DIALECT \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	213 -Dflys.backend.importer.infogew.file="$GEW_FILE" \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	214 -Dflys.backend.main.value.types=$IMPORTER_MAINVALUE_TYPES \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	215 -Dflys.backend.importer.annotation.types=$IMPORTER_ANNOTATION_TYPES \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	216 -Dflys.backend.importer.dry.run=$IMPORTER_DRY_RUN \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	217 -Dflys.backend.importer.skip.annotations=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	218 -Dflys.backend.importer.skip.bwastr=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	219 -Dflys.backend.importer.skip.extra.wsts=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	220 -Dflys.backend.importer.skip.fixations=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	221 -Dflys.backend.importer.skip.flood.water=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	222 -Dflys.backend.importer.skip.flood.protection=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	223 -Dflys.backend.importer.skip.gauges=false \
7326 5e621410db05 Importer: skip historical discharge tables per default as they come via ETL. Tom Gottfried <tom@intevation.de> parents: 6825 diff changeset	224 -Dflys.backend.importer.skip.historical.discharge.tables=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	225 -Dflys.backend.importer.skip.hyks=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	226 -Dflys.backend.importer.skip.official.lines=false \
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	227 -Dflys.backend.importer.skip.prfs=true \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	228 -Dflys.backend.importer.skip.w80s=true \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	229 -Dflys.backend.importer.skip.w80.csvs=true \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	230 -Dflys.backend.importer.skip.da50s=true \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	231 -Dflys.backend.importer.skip.da66s=true \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	232 -Dflys.backend.importer.skip.wst=false \
6417 14b5674e57a4 Importer: add skip flag for measurement stations to scripts Tom Gottfried <tom@intevation.de> parents: 5838 diff changeset	233 -Dflys.backend.importer.skip.measurement.stations=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	234 -Dflys.backend.importer.skip.waterlevel.differences=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	235 -Dflys.backend.importer.skip.waterlevels=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	236 -Dflys.backend.importer.skip.sq.relation=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	237 -Dflys.backend.importer.skip.sediment.density=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	238 -Dflys.backend.importer.skip.sediment.yield=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	239 -Dflys.backend.importer.skip.morphological.width=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	240 -Dflys.backend.importer.skip.flow.velocity=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	241 -Dflys.backend.importer.skip.bed.height.single=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	242 -Dflys.backend.importer.skip.bed.height.epoch=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	243 $JAR
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	244 }
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	245
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	246 import_morpho(){
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	247 LOG_FILE=${LOG_DIR}/morpho.log
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	248 echo Importing Morphological data.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	249 echo Logging into: $LOG_FILE
7728 c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	250 sed -i 's!$log4j.appender.IMPORTER.File=$.*!\1'"$LOG_FILE"'!' \
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	251 $LOG_DIR/log4j.properties
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	252 java -jar \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	253 -Xmx$MIN_MEMORY \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	254 -server \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	255 -Dlog4j.configuration=file://$LOG_DIR/log4j.properties \
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	256 -Dflys.backend.user=$DBUSER \
4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	257 -Dflys.backend.password=$DBPASS \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	258 -Dflys.backend.url=$BACKEND_URL \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	259 -Dflys.backend.driver=$BACKEND_DB_DRIVER \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	260 -Dflys.backend.dialect=$BACKEND_DB_DIALECT \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	261 -Dflys.backend.importer.infogew.file="$GEW_FILE" \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	262 -Dflys.backend.main.value.types=$IMPORTER_MAINVALUE_TYPES \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	263 -Dflys.backend.importer.annotation.types=$IMPORTER_ANNOTATION_TYPES \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	264 -Dflys.backend.importer.dry.run=$IMPORTER_DRY_RUN \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	265 -Dflys.backend.importer.skip.annotations=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	266 -Dflys.backend.importer.skip.bwastr=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	267 -Dflys.backend.importer.skip.extra.wsts=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	268 -Dflys.backend.importer.skip.fixations=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	269 -Dflys.backend.importer.skip.flood.water=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	270 -Dflys.backend.importer.skip.flood.protection=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	271 -Dflys.backend.importer.skip.gauges=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	272 -Dflys.backend.importer.skip.historical.discharge.tables=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	273 -Dflys.backend.importer.skip.hyks=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	274 -Dflys.backend.importer.skip.official.lines=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	275 -Dflys.backend.importer.skip.prfs=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	276 -Dflys.backend.importer.skip.w80s=true \
6706 c3e38ea87964 Importer scripts: add skip flag for W80-CSV to import_river.sh Tom Gottfried <tom@intevation.de> parents: 6658 diff changeset	277 -Dflys.backend.importer.skip.w80.csvs=true \
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	278 -Dflys.backend.importer.skip.da50s=true \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	279 -Dflys.backend.importer.skip.da66s=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	280 -Dflys.backend.importer.skip.wst=true \
6417 14b5674e57a4 Importer: add skip flag for measurement stations to scripts Tom Gottfried <tom@intevation.de> parents: 5838 diff changeset	281 -Dflys.backend.importer.skip.measurement.stations=false \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	282 -Dflys.backend.importer.skip.waterlevel.differences=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	283 -Dflys.backend.importer.skip.waterlevels=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	284 -Dflys.backend.importer.skip.sq.relation=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	285 -Dflys.backend.importer.skip.sediment.density=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	286 -Dflys.backend.importer.skip.sediment.yield=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	287 -Dflys.backend.importer.skip.morphological.width=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	288 -Dflys.backend.importer.skip.flow.velocity=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	289 -Dflys.backend.importer.skip.bed.height.single=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	290 -Dflys.backend.importer.skip.bed.height.epoch=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	291 $JAR
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	292 }
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	293
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	294 import_prf(){
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	295 LOG_FILE=${LOG_DIR}/prf.log
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	296 echo Importing cross section data.
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	297 echo Logging into: $LOG_FILE
7728 c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	298 sed -i 's!$log4j.appender.IMPORTER.File=$.*!\1'"$LOG_FILE"'!' \
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	299 $LOG_DIR/log4j.properties
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	300 java -jar \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	301 -Xmx$MIN_MEMORY \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	302 -server \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	303 -Dlog4j.configuration=file://$LOG_DIR/log4j.properties \
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	304 -Dflys.backend.user=$DBUSER \
4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	305 -Dflys.backend.password=$DBPASS \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	306 -Dflys.backend.url=$BACKEND_URL \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	307 -Dflys.backend.driver=$BACKEND_DB_DRIVER \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	308 -Dflys.backend.dialect=$BACKEND_DB_DIALECT \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	309 -Dflys.backend.importer.infogew.file="$GEW_FILE" \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	310 -Dflys.backend.main.value.types=$IMPORTER_MAINVALUE_TYPES \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	311 -Dflys.backend.importer.annotation.types=$IMPORTER_ANNOTATION_TYPES \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	312 -Dflys.backend.importer.dry.run=$IMPORTER_DRY_RUN \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	313 -Dflys.backend.importer.skip.annotations=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	314 -Dflys.backend.importer.skip.bwastr=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	315 -Dflys.backend.importer.skip.extra.wsts=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	316 -Dflys.backend.importer.skip.fixations=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	317 -Dflys.backend.importer.skip.flood.water=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	318 -Dflys.backend.importer.skip.flood.protection=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	319 -Dflys.backend.importer.skip.gauges=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	320 -Dflys.backend.importer.skip.historical.discharge.tables=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	321 -Dflys.backend.importer.skip.hyks=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	322 -Dflys.backend.importer.skip.official.lines=true \
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	323 -Dflys.backend.importer.skip.prfs=false \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	324 -Dflys.backend.importer.skip.w80s=false \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	325 -Dflys.backend.importer.skip.w80.csvs=false \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	326 -Dflys.backend.importer.skip.da50s=false \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	327 -Dflys.backend.importer.skip.da66s=false \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	328 -Dflys.backend.importer.skip.wst=true \
6417 14b5674e57a4 Importer: add skip flag for measurement stations to scripts Tom Gottfried <tom@intevation.de> parents: 5838 diff changeset	329 -Dflys.backend.importer.skip.measurement.stations=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	330 -Dflys.backend.importer.skip.waterlevel.differences=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	331 -Dflys.backend.importer.skip.waterlevels=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	332 -Dflys.backend.importer.skip.sq.relation=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	333 -Dflys.backend.importer.skip.sediment.density=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	334 -Dflys.backend.importer.skip.sediment.yield=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	335 -Dflys.backend.importer.skip.morphological.width=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	336 -Dflys.backend.importer.skip.flow.velocity=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	337 -Dflys.backend.importer.skip.bed.height.single=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	338 -Dflys.backend.importer.skip.bed.height.epoch=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	339 $JAR
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	340 }
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	341
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	342 import_geo(){
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	343 LOG_FILE=${LOG_DIR}/geo.log
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	344 echo Importing Geographic data.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	345 echo Logging into: $LOG_FILE
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	346
5169 db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	347 RIVER_PATH=$(grep "WSTDatei:" "$GEW_FILE" \| awk '{print $2}')
db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	348 RIVER_PATH=$(dirname "$RIVER_PATH")/../..
db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	349 RIVER_PATH=$(readlink -f "$RIVER_PATH")
db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	350
db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	351 exec python $(dirname $0)/geodaesie/shpimporter.py \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	352 --directory $RIVER_PATH \
5563 673f68bb19a4 Geo-importer: accept river names with white space Tom Gottfried <tom@intevation.de> parents: 5459 diff changeset	353 --river_name "$RIVER_NAME" \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	354 --ogr_connection "$OGR_CONNECTION" \
6658 89f856e24d27 Importer scripts: shpimporter needs backend name to play with Oracle. Tom Gottfried <tom@intevation.de> parents: 6651 diff changeset	355 --host $DBHOST/$BACKEND_NAME \
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	356 --user $DBUSER \
4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	357 --password $DBPASS \
5169 db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	358 --verbose 1 \
db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	359 > "$LOG_FILE" 2>&1
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	360 }
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	361
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	362
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	363 if [ "$SKIP_HYDRO" != "TRUE" ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	364 import_hydro
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	365 fi
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	366 if [ "$SKIP_PRF" != "TRUE" ]; then
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	367 import_prf
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	368 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	369 if [ "$SKIP_MORPHO" != "TRUE" ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	370 import_morpho
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	371 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	372 if [ "$SKIP_GEO" != "TRUE" ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	373 import_geo
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	374 fi

Mercurial > dive4elements > river

annotate backend/contrib/import_river.sh @ 7786:61a9e7df0728