dive4elements/river: backend/contrib/import

annotate backend/contrib/import_river.sh @ 8870:c26fb37899ca

Introduced groups for modules. Modules marked with the same group-id, will be put together in the ui. Also using now the localization info from the server instead of localizing the modules again on the client side.

author	gernotbelger
date	Wed, 07 Feb 2018 11:59:13 +0100
parents	f7c49975cc15
children	1fec0a06d833 0a5239a1e46e

rev	line source
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	1 #!/bin/bash
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	2 # Import script for rivers
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	3 #
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	4 # Authors:
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	5 # Andre Heinecke <aheinecke@intevation.de>
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	6 #
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	7 # Copyright:
5168 650f94ec3122 Fix copyright on import_river.sh Andre Heinecke <aheinecke@intevation.de> parents: 5167 diff changeset	8 # Copyright (C) 2013 Intevation GmbH
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	9 #
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	10 # This program is free software; you can redistribute it and/or
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	11 # modify it under the terms of the GNU General Public License
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	12 # as published by the Free Software Foundation; either version 2
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	13 # of the License, or (at your option) any later version.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	14 #
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	15 # This program is distributed in the hope that it will be useful,
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	16 # but WITHOUT ANY WARRANTY; without even the implied warranty of
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	17 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	18 # GNU General Public License for more details.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	19 #
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	20 # You should have received a copy of the GNU General Public License
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	21 # along with this program; if not, write to the Free Software
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	22 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	23
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	24 set -e
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	25
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	26 # Default settings
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	27 DEFAULT_HOST=localhost
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	28 DEFAULT_PORT=1521
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	29 DEFAULT_USER=flys_dami
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	30 DEFAULT_PASS=flys_dami
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	31 DEFAULT_LOG=$PWD/logs
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	32 DEFAULT_BACKEND_NAME="XE"
8675 ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems. Tom Gottfried <tom@intevation.de> parents: 8674 diff changeset	33 JAR="bin/river-backend-1.0-SNAPSHOT*.jar"
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	34 IMPORTER_DRY_RUN=false
6449 25912df0facf Import main values of type '-' by default Tom Gottfried <tom.gottfried@intevation.de> parents: 6417 diff changeset	35 IMPORTER_MAINVALUE_TYPES=QWTD-
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	36 IMPORTER_ANNOTATION_TYPES="conf/annotation-types.xml"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	37
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	38
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	39 MIN_MEMORY="8024m"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	40
6651 8782ba5cb222 Importer scripts: add default encoding to correctly read gew-files Tom Gottfried <tom@intevation.de> parents: 6449 diff changeset	41 # Default encoding. Change here if necessary
8782ba5cb222 Importer scripts: add default encoding to correctly read gew-files Tom Gottfried <tom@intevation.de> parents: 6449 diff changeset	42 export LC_ALL=de_DE@euro
8782ba5cb222 Importer scripts: add default encoding to correctly read gew-files Tom Gottfried <tom@intevation.de> parents: 6449 diff changeset	43
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	44 usage(){
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	45 cat << EOF
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	46
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	47 usage: $0 [options] gew_file
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	48
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	49 Import a river described by the gew_file
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	50
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	51 OPTIONS:
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	52 -?, --help Show this message
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	53 -u, --username=<username> Database username. Default: $DEFAULT_USER
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	54 -w, --password=<password> Database password. Default: $DEFAULT_PASS
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	55 -h, --host=<host> Connect to database on host <host>.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	56 Default: $DEFAULT_HOST
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	57 -p, --port=<number> Use port number <number>. Default: $DEFAULT_PORT
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	58 -d, --db-name=<database_name> Name of the database / backend. Default: $DEFAULT_BACKEND_NAME
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	59 -l, --log-dir=<directory> Directory in which to create the log files.
7728 c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	60 Default: $PWD/logs
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	61 --postgres Database is PostgreSQL
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	62 --skip-hydro Skip import of hydrological data
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	63 --skip-morpho Skip import of morphological data
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	64 --skip-geo Skip import of geographic data
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	65 --skip-prf Skip import of cross section data
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	66 EOF
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	67 exit 0
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	68 }
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	69
7728 c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	70 OPTS=`getopt -o ?u:w:h:p:d:l: \
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	71 -l help,username:,password:,host:,port:,db-name:,log-dir:,skip-hydro,skip-morpho,skip-geo,skip-prf,postgres \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	72 -n $0 -- "$@"`
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	73 if [ $? != 0 ] ; then usage; fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	74 eval set -- "$OPTS"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	75 while true ; do
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	76 case "$1" in
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	77 "-?"\|"--help")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	78 usage;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	79 "--")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	80 shift
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	81 break;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	82 "-u"\|"--username")
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	83 DBUSER=$2
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	84 shift 2;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	85 "-w"\|"--password")
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	86 DBPASS=$2
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	87 shift 2;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	88 "-h"\|"--host")
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	89 DBHOST=$2
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	90 shift 2;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	91 "-p"\|"--port")
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	92 DBPORT=$2
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	93 shift 2;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	94 "-l"\|"--log-dir")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	95 LOG=$2
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	96 shift 2;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	97 "-d"\|"--db-name")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	98 BACKEND_NAME=$2
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	99 shift 2;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	100 "--skip-hydro")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	101 SKIP_HYDRO="TRUE"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	102 shift;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	103 "--skip-morpho")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	104 SKIP_MORPHO="TRUE"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	105 shift;;
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	106 "--skip-prf")
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	107 SKIP_PRF="TRUE"
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	108 shift;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	109 "--skip-geo")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	110 SKIP_GEO="TRUE"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	111 shift;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	112 "--postgres")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	113 POSTGRES="TRUE"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	114 shift;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	115 *)
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	116 echo "Unknown Option $1"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	117 usage;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	118 esac
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	119 done
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	120
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	121 if [ -z $DBUSER ]; then
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	122 DBUSER=$DEFAULT_USER
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	123 fi
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	124 if [ -z $DBPASS ]; then
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	125 DBPASS=$DEFAULT_PASS
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	126 fi
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	127 if [ -z $DBPORT ]; then
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	128 DBPORT=$DEFAULT_PORT
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	129 fi
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	130 if [ -z $DBHOST ]; then
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	131 DBHOST=$DEFAULT_HOST
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	132 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	133 if [ -z $BACKEND_NAME ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	134 BACKEND_NAME=$DEFAULT_BACKEND_NAME
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	135 fi
7728 c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	136 if [ -z $LOG ]; then
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	137 LOG=$DEFAULT_LOG
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	138 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	139
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	140 if [ $# != 1 ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	141 usage
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	142 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	143
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	144 if [ ! -r $1 ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	145 echo "Could not open $1 please ensure it exists and is readable"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	146 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	147
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	148 GEW_FILE="$1"
5459 7c646d4d5103 be more flexible with river names (e.g. containing white space) in import script Tom Gottfried <tom@intevation.de> parents: 5355 diff changeset	149 RIVER_NAME=$(grep "Gew.sser" "$1" \| sed 's/Gew.sser: //')
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	150 DATE=$(date +%Y.%m.%d_%H%M)
5459 7c646d4d5103 be more flexible with river names (e.g. containing white space) in import script Tom Gottfried <tom@intevation.de> parents: 5355 diff changeset	151 LOG_DIR=${LOG}/`basename $GEW_FILE .gew`-$DATE
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	152 mkdir -p ${LOG_DIR}
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	153
7728 c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	154 cat > "$LOG_DIR/log4j.properties" << "EOF"
8820 6d25d8f81168 import_river.sh: log less verbose. Tom Gottfried <tom@intevation.de> parents: 8819 diff changeset	155 log4j.rootLogger=INFO, IMPORTER
7728 c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	156 log4j.appender.IMPORTER.layout=org.apache.log4j.PatternLayout
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	157 log4j.appender.IMPORTER.layout.ConversionPattern=%d [%t] %-5p %c - %m%n
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	158 log4j.appender.IMPORTER=org.apache.log4j.RollingFileAppender
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	159 log4j.appender.IMPORTER.File=./import.log
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	160 log4j.appender.IMPORTER.MaxFileSize=100000KB
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	161 log4j.appender.IMPORTER.MaxBackupIndex=10
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	162 EOF
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	163
5355 480d2f1f61cc Fix import_rivers script for postgres Andre Heinecke <aheinecke@intevation.de> parents: 5212 diff changeset	164 if [ "$POSTGRES" = "TRUE" ]; then
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	165 JAR=$(echo "$JAR" \| sed 's/importer/importer_psql/')
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	166 if [ ! -r "$JAR" ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	167 echo "Could not find Postgres importer $JAR"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	168 exit 1
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	169 fi
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	170 OGR_CONNECTION="PG:dbname=$BACKEND_NAME host=$DBHOST port=$DBPORT \
4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	171 user=$DBUSER password=$DBPASS"
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	172 BACKEND_DB_PREFIX="jdbc:postgresql:"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	173 BACKEND_DB_DRIVER="org.postgresql.Driver"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	174 BACKEND_DB_DIALECT="org.hibernate.dialect.PostgreSQLDialect"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	175 else
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	176 BACKEND_DB_PREFIX="jdbc:oracle:thin:@"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	177 BACKEND_DB_DRIVER="oracle.jdbc.OracleDriver"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	178 BACKEND_DB_DIALECT="org.hibernate.dialect.OracleDialect"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	179 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	180
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	181 BACKEND_URL=$BACKEND_DB_PREFIX//$DBHOST:$DBPORT/$BACKEND_NAME
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	182
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	183 echo "Importing $RIVER_NAME into $BACKEND_URL."
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	184
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	185 import_hydro(){
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	186 LOG_FILE=${LOG_DIR}/hydro.log
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	187 echo Importing Hydrological data.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	188 echo Logging into: $LOG_FILE
7728 c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	189 sed -i 's!$log4j.appender.IMPORTER.File=$.*!\1'"$LOG_FILE"'!' \
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	190 $LOG_DIR/log4j.properties
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	191 java -jar \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	192 -Xmx$MIN_MEMORY \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	193 -server \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	194 -Dlog4j.configuration=file://$LOG_DIR/log4j.properties \
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	195 -Dflys.backend.user=$DBUSER \
4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	196 -Dflys.backend.password=$DBPASS \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	197 -Dflys.backend.url=$BACKEND_URL \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	198 -Dflys.backend.driver=$BACKEND_DB_DRIVER \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	199 -Dflys.backend.dialect=$BACKEND_DB_DIALECT \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	200 -Dflys.backend.importer.infogew.file="$GEW_FILE" \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	201 -Dflys.backend.main.value.types=$IMPORTER_MAINVALUE_TYPES \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	202 -Dflys.backend.importer.annotation.types=$IMPORTER_ANNOTATION_TYPES \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	203 -Dflys.backend.importer.dry.run=$IMPORTER_DRY_RUN \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	204 -Dflys.backend.importer.skip.annotations=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	205 -Dflys.backend.importer.skip.bwastr=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	206 -Dflys.backend.importer.skip.extra.wsts=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	207 -Dflys.backend.importer.skip.fixations=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	208 -Dflys.backend.importer.skip.flood.water=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	209 -Dflys.backend.importer.skip.flood.protection=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	210 -Dflys.backend.importer.skip.gauges=false \
7326 5e621410db05 Importer: skip historical discharge tables per default as they come via ETL. Tom Gottfried <tom@intevation.de> parents: 6825 diff changeset	211 -Dflys.backend.importer.skip.historical.discharge.tables=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	212 -Dflys.backend.importer.skip.hyks=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	213 -Dflys.backend.importer.skip.official.lines=false \
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	214 -Dflys.backend.importer.skip.prfs=true \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	215 -Dflys.backend.importer.skip.w80s=true \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	216 -Dflys.backend.importer.skip.w80.csvs=true \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	217 -Dflys.backend.importer.skip.da50s=true \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	218 -Dflys.backend.importer.skip.da66s=true \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	219 -Dflys.backend.importer.skip.wst=false \
6417 14b5674e57a4 Importer: add skip flag for measurement stations to scripts Tom Gottfried <tom@intevation.de> parents: 5838 diff changeset	220 -Dflys.backend.importer.skip.measurement.stations=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	221 -Dflys.backend.importer.skip.waterlevel.differences=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	222 -Dflys.backend.importer.skip.waterlevels=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	223 -Dflys.backend.importer.skip.sq.relation=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	224 -Dflys.backend.importer.skip.sediment.density=true \
8056 d86cc6a17b7a Importer: Import sediment load at measurement stations. Tom Gottfried <tom@intevation.de> parents: 8032 diff changeset	225 -Dflys.backend.importer.skip.sediment.load=true \
8032 fd3a24336e6a SCHEMA CHANGE and Importer (only longitudinal section data so far): only grain fractions given now in schema are valid. We are handling sediment loads, not yields. "Tom Gottfried <tom@intevation.de>" parents: 7925 diff changeset	226 -Dflys.backend.importer.skip.sediment.load.ls=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	227 -Dflys.backend.importer.skip.morphological.width=true \
8819 79f9d837c5cc Add missing skip-option in import script. Tom Gottfried <tom@intevation.de> parents: 8675 diff changeset	228 -Dflys.backend.importer.skip.porosity=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	229 -Dflys.backend.importer.skip.flow.velocity=true \
8674 44ba71b0b75a (issue1763) Correct properties in importer scripts. Tom Gottfried <tom@intevation.de> parents: 8056 diff changeset	230 -Dflys.backend.importer.skip.bed.height=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	231 $JAR
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	232 }
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	233
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	234 import_morpho(){
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	235 LOG_FILE=${LOG_DIR}/morpho.log
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	236 echo Importing Morphological data.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	237 echo Logging into: $LOG_FILE
7728 c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	238 sed -i 's!$log4j.appender.IMPORTER.File=$.*!\1'"$LOG_FILE"'!' \
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	239 $LOG_DIR/log4j.properties
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	240 java -jar \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	241 -Xmx$MIN_MEMORY \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	242 -server \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	243 -Dlog4j.configuration=file://$LOG_DIR/log4j.properties \
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	244 -Dflys.backend.user=$DBUSER \
4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	245 -Dflys.backend.password=$DBPASS \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	246 -Dflys.backend.url=$BACKEND_URL \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	247 -Dflys.backend.driver=$BACKEND_DB_DRIVER \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	248 -Dflys.backend.dialect=$BACKEND_DB_DIALECT \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	249 -Dflys.backend.importer.infogew.file="$GEW_FILE" \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	250 -Dflys.backend.main.value.types=$IMPORTER_MAINVALUE_TYPES \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	251 -Dflys.backend.importer.annotation.types=$IMPORTER_ANNOTATION_TYPES \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	252 -Dflys.backend.importer.dry.run=$IMPORTER_DRY_RUN \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	253 -Dflys.backend.importer.skip.annotations=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	254 -Dflys.backend.importer.skip.bwastr=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	255 -Dflys.backend.importer.skip.extra.wsts=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	256 -Dflys.backend.importer.skip.fixations=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	257 -Dflys.backend.importer.skip.flood.water=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	258 -Dflys.backend.importer.skip.flood.protection=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	259 -Dflys.backend.importer.skip.gauges=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	260 -Dflys.backend.importer.skip.historical.discharge.tables=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	261 -Dflys.backend.importer.skip.hyks=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	262 -Dflys.backend.importer.skip.official.lines=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	263 -Dflys.backend.importer.skip.prfs=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	264 -Dflys.backend.importer.skip.w80s=true \
6706 c3e38ea87964 Importer scripts: add skip flag for W80-CSV to import_river.sh Tom Gottfried <tom@intevation.de> parents: 6658 diff changeset	265 -Dflys.backend.importer.skip.w80.csvs=true \
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	266 -Dflys.backend.importer.skip.da50s=true \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	267 -Dflys.backend.importer.skip.da66s=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	268 -Dflys.backend.importer.skip.wst=true \
6417 14b5674e57a4 Importer: add skip flag for measurement stations to scripts Tom Gottfried <tom@intevation.de> parents: 5838 diff changeset	269 -Dflys.backend.importer.skip.measurement.stations=false \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	270 -Dflys.backend.importer.skip.waterlevel.differences=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	271 -Dflys.backend.importer.skip.waterlevels=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	272 -Dflys.backend.importer.skip.sq.relation=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	273 -Dflys.backend.importer.skip.sediment.density=false \
8056 d86cc6a17b7a Importer: Import sediment load at measurement stations. Tom Gottfried <tom@intevation.de> parents: 8032 diff changeset	274 -Dflys.backend.importer.skip.sediment.load=false \
8032 fd3a24336e6a SCHEMA CHANGE and Importer (only longitudinal section data so far): only grain fractions given now in schema are valid. We are handling sediment loads, not yields. "Tom Gottfried <tom@intevation.de>" parents: 7925 diff changeset	275 -Dflys.backend.importer.skip.sediment.load.ls=false \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	276 -Dflys.backend.importer.skip.morphological.width=false \
8819 79f9d837c5cc Add missing skip-option in import script. Tom Gottfried <tom@intevation.de> parents: 8675 diff changeset	277 -Dflys.backend.importer.skip.porosity=false \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	278 -Dflys.backend.importer.skip.flow.velocity=false \
8674 44ba71b0b75a (issue1763) Correct properties in importer scripts. Tom Gottfried <tom@intevation.de> parents: 8056 diff changeset	279 -Dflys.backend.importer.skip.bed.height=false \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	280 $JAR
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	281 }
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	282
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	283 import_prf(){
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	284 LOG_FILE=${LOG_DIR}/prf.log
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	285 echo Importing cross section data.
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	286 echo Logging into: $LOG_FILE
7728 c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	287 sed -i 's!$log4j.appender.IMPORTER.File=$.*!\1'"$LOG_FILE"'!' \
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	288 $LOG_DIR/log4j.properties
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	289 java -jar \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	290 -Xmx$MIN_MEMORY \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	291 -server \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	292 -Dlog4j.configuration=file://$LOG_DIR/log4j.properties \
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	293 -Dflys.backend.user=$DBUSER \
4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	294 -Dflys.backend.password=$DBPASS \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	295 -Dflys.backend.url=$BACKEND_URL \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	296 -Dflys.backend.driver=$BACKEND_DB_DRIVER \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	297 -Dflys.backend.dialect=$BACKEND_DB_DIALECT \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	298 -Dflys.backend.importer.infogew.file="$GEW_FILE" \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	299 -Dflys.backend.main.value.types=$IMPORTER_MAINVALUE_TYPES \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	300 -Dflys.backend.importer.annotation.types=$IMPORTER_ANNOTATION_TYPES \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	301 -Dflys.backend.importer.dry.run=$IMPORTER_DRY_RUN \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	302 -Dflys.backend.importer.skip.annotations=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	303 -Dflys.backend.importer.skip.bwastr=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	304 -Dflys.backend.importer.skip.extra.wsts=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	305 -Dflys.backend.importer.skip.fixations=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	306 -Dflys.backend.importer.skip.flood.water=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	307 -Dflys.backend.importer.skip.flood.protection=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	308 -Dflys.backend.importer.skip.gauges=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	309 -Dflys.backend.importer.skip.historical.discharge.tables=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	310 -Dflys.backend.importer.skip.hyks=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	311 -Dflys.backend.importer.skip.official.lines=true \
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	312 -Dflys.backend.importer.skip.prfs=false \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	313 -Dflys.backend.importer.skip.w80s=false \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	314 -Dflys.backend.importer.skip.w80.csvs=false \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	315 -Dflys.backend.importer.skip.da50s=false \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	316 -Dflys.backend.importer.skip.da66s=false \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	317 -Dflys.backend.importer.skip.wst=true \
6417 14b5674e57a4 Importer: add skip flag for measurement stations to scripts Tom Gottfried <tom@intevation.de> parents: 5838 diff changeset	318 -Dflys.backend.importer.skip.measurement.stations=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	319 -Dflys.backend.importer.skip.waterlevel.differences=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	320 -Dflys.backend.importer.skip.waterlevels=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	321 -Dflys.backend.importer.skip.sq.relation=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	322 -Dflys.backend.importer.skip.sediment.density=true \
8056 d86cc6a17b7a Importer: Import sediment load at measurement stations. Tom Gottfried <tom@intevation.de> parents: 8032 diff changeset	323 -Dflys.backend.importer.skip.sediment.load=true \
8032 fd3a24336e6a SCHEMA CHANGE and Importer (only longitudinal section data so far): only grain fractions given now in schema are valid. We are handling sediment loads, not yields. "Tom Gottfried <tom@intevation.de>" parents: 7925 diff changeset	324 -Dflys.backend.importer.skip.sediment.load.ls=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	325 -Dflys.backend.importer.skip.morphological.width=true \
8819 79f9d837c5cc Add missing skip-option in import script. Tom Gottfried <tom@intevation.de> parents: 8675 diff changeset	326 -Dflys.backend.importer.skip.porosity=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	327 -Dflys.backend.importer.skip.flow.velocity=true \
8674 44ba71b0b75a (issue1763) Correct properties in importer scripts. Tom Gottfried <tom@intevation.de> parents: 8056 diff changeset	328 -Dflys.backend.importer.skip.bed.height=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	329 $JAR
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	330 }
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	331
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	332 import_geo(){
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	333 LOG_FILE=${LOG_DIR}/geo.log
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	334 echo Importing Geographic data.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	335 echo Logging into: $LOG_FILE
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	336
5169 db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	337 RIVER_PATH=$(grep "WSTDatei:" "$GEW_FILE" \| awk '{print $2}')
db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	338 RIVER_PATH=$(dirname "$RIVER_PATH")/../..
db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	339 RIVER_PATH=$(readlink -f "$RIVER_PATH")
db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	340
8823 efa52271786b Fix path to python importer. Tom Gottfried <tom@intevation.de> parents: 8820 diff changeset	341 exec python $(dirname $0)/shpimporter/shpimporter.py \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	342 --directory $RIVER_PATH \
5563 673f68bb19a4 Geo-importer: accept river names with white space Tom Gottfried <tom@intevation.de> parents: 5459 diff changeset	343 --river_name "$RIVER_NAME" \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	344 --ogr_connection "$OGR_CONNECTION" \
6658 89f856e24d27 Importer scripts: shpimporter needs backend name to play with Oracle. Tom Gottfried <tom@intevation.de> parents: 6651 diff changeset	345 --host $DBHOST/$BACKEND_NAME \
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	346 --user $DBUSER \
4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	347 --password $DBPASS \
5169 db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	348 --verbose 1 \
db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	349 > "$LOG_FILE" 2>&1
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	350 }
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	351
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	352
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	353 if [ "$SKIP_HYDRO" != "TRUE" ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	354 import_hydro
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	355 fi
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	356 if [ "$SKIP_PRF" != "TRUE" ]; then
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	357 import_prf
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	358 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	359 if [ "$SKIP_MORPHO" != "TRUE" ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	360 import_morpho
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	361 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	362 if [ "$SKIP_GEO" != "TRUE" ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	363 import_geo
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	364 fi

Mercurial > dive4elements > river

annotate backend/contrib/import_river.sh @ 8870:c26fb37899ca