dive4elements/river: backend/contrib/import

annotate backend/contrib/import_river.sh @ 9744:b1b48fa7bd80 3.2.x

Include both database drivers in one pom.xml This makes it unnecessary to package for a specific database and makes it possible to connect to both database backends from the same installation (e.g. in a Docker container).

author	Tom Gottfried <tom@intevation.de>
date	Tue, 30 Aug 2022 14:33:20 +0200
parents	0a5239a1e46e
children

rev	line source
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	1 #!/bin/bash
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	2 # Import script for rivers
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	3 #
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	4 # Authors:
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	5 # Andre Heinecke <aheinecke@intevation.de>
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	6 #
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	7 # Copyright:
5168 650f94ec3122 Fix copyright on import_river.sh Andre Heinecke <aheinecke@intevation.de> parents: 5167 diff changeset	8 # Copyright (C) 2013 Intevation GmbH
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	9 #
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	10 # This program is free software; you can redistribute it and/or
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	11 # modify it under the terms of the GNU General Public License
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	12 # as published by the Free Software Foundation; either version 2
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	13 # of the License, or (at your option) any later version.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	14 #
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	15 # This program is distributed in the hope that it will be useful,
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	16 # but WITHOUT ANY WARRANTY; without even the implied warranty of
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	17 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	18 # GNU General Public License for more details.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	19 #
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	20 # You should have received a copy of the GNU General Public License
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	21 # along with this program; if not, write to the Free Software
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	22 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	23
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	24 set -e
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	25
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	26 # Default settings
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	27 DEFAULT_HOST=localhost
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	28 DEFAULT_PORT=1521
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	29 DEFAULT_USER=flys_dami
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	30 DEFAULT_PASS=flys_dami
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	31 DEFAULT_LOG=$PWD/logs
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	32 DEFAULT_BACKEND_NAME="XE"
8675 ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems. Tom Gottfried <tom@intevation.de> parents: 8674 diff changeset	33 JAR="bin/river-backend-1.0-SNAPSHOT*.jar"
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	34 IMPORTER_DRY_RUN=false
6449 25912df0facf Import main values of type '-' by default Tom Gottfried <tom.gottfried@intevation.de> parents: 6417 diff changeset	35 IMPORTER_MAINVALUE_TYPES=QWTD-
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	36 IMPORTER_ANNOTATION_TYPES="conf/annotation-types.xml"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	37
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	38
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	39 MIN_MEMORY="8024m"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	40
6651 8782ba5cb222 Importer scripts: add default encoding to correctly read gew-files Tom Gottfried <tom@intevation.de> parents: 6449 diff changeset	41 # Default encoding. Change here if necessary
8782ba5cb222 Importer scripts: add default encoding to correctly read gew-files Tom Gottfried <tom@intevation.de> parents: 6449 diff changeset	42 export LC_ALL=de_DE@euro
8782ba5cb222 Importer scripts: add default encoding to correctly read gew-files Tom Gottfried <tom@intevation.de> parents: 6449 diff changeset	43
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	44 usage(){
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	45 cat << EOF
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	46
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	47 usage: $0 [options] gew_file
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	48
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	49 Import a river described by the gew_file
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	50
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	51 OPTIONS:
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	52 -?, --help Show this message
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	53 -u, --username=<username> Database username. Default: $DEFAULT_USER
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	54 -w, --password=<password> Database password. Default: $DEFAULT_PASS
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	55 -h, --host=<host> Connect to database on host <host>.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	56 Default: $DEFAULT_HOST
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	57 -p, --port=<number> Use port number <number>. Default: $DEFAULT_PORT
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	58 -d, --db-name=<database_name> Name of the database / backend. Default: $DEFAULT_BACKEND_NAME
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	59 -l, --log-dir=<directory> Directory in which to create the log files.
7728 c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	60 Default: $PWD/logs
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	61 --postgres Database is PostgreSQL
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	62 --skip-hydro Skip import of hydrological data
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	63 --skip-morpho Skip import of morphological data
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	64 --skip-geo Skip import of geographic data
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	65 --skip-prf Skip import of cross section data
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	66 EOF
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	67 exit 0
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	68 }
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	69
7728 c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	70 OPTS=`getopt -o ?u:w:h:p:d:l: \
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	71 -l help,username:,password:,host:,port:,db-name:,log-dir:,skip-hydro,skip-morpho,skip-geo,skip-prf,postgres \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	72 -n $0 -- "$@"`
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	73 if [ $? != 0 ] ; then usage; fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	74 eval set -- "$OPTS"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	75 while true ; do
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	76 case "$1" in
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	77 "-?"\|"--help")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	78 usage;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	79 "--")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	80 shift
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	81 break;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	82 "-u"\|"--username")
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	83 DBUSER=$2
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	84 shift 2;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	85 "-w"\|"--password")
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	86 DBPASS=$2
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	87 shift 2;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	88 "-h"\|"--host")
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	89 DBHOST=$2
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	90 shift 2;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	91 "-p"\|"--port")
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	92 DBPORT=$2
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	93 shift 2;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	94 "-l"\|"--log-dir")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	95 LOG=$2
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	96 shift 2;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	97 "-d"\|"--db-name")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	98 BACKEND_NAME=$2
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	99 shift 2;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	100 "--skip-hydro")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	101 SKIP_HYDRO="TRUE"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	102 shift;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	103 "--skip-morpho")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	104 SKIP_MORPHO="TRUE"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	105 shift;;
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	106 "--skip-prf")
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	107 SKIP_PRF="TRUE"
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	108 shift;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	109 "--skip-geo")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	110 SKIP_GEO="TRUE"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	111 shift;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	112 "--postgres")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	113 POSTGRES="TRUE"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	114 shift;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	115 *)
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	116 echo "Unknown Option $1"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	117 usage;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	118 esac
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	119 done
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	120
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	121 if [ -z $DBUSER ]; then
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	122 DBUSER=$DEFAULT_USER
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	123 fi
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	124 if [ -z $DBPASS ]; then
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	125 DBPASS=$DEFAULT_PASS
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	126 fi
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	127 if [ -z $DBPORT ]; then
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	128 DBPORT=$DEFAULT_PORT
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	129 fi
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	130 if [ -z $DBHOST ]; then
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	131 DBHOST=$DEFAULT_HOST
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	132 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	133 if [ -z $BACKEND_NAME ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	134 BACKEND_NAME=$DEFAULT_BACKEND_NAME
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	135 fi
7728 c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	136 if [ -z $LOG ]; then
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	137 LOG=$DEFAULT_LOG
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	138 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	139
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	140 if [ $# != 1 ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	141 usage
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	142 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	143
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	144 if [ ! -r $1 ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	145 echo "Could not open $1 please ensure it exists and is readable"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	146 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	147
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	148 GEW_FILE="$1"
5459 7c646d4d5103 be more flexible with river names (e.g. containing white space) in import script Tom Gottfried <tom@intevation.de> parents: 5355 diff changeset	149 RIVER_NAME=$(grep "Gew.sser" "$1" \| sed 's/Gew.sser: //')
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	150 DATE=$(date +%Y.%m.%d_%H%M)
5459 7c646d4d5103 be more flexible with river names (e.g. containing white space) in import script Tom Gottfried <tom@intevation.de> parents: 5355 diff changeset	151 LOG_DIR=${LOG}/`basename $GEW_FILE .gew`-$DATE
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	152 mkdir -p ${LOG_DIR}
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	153
9726 0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	154 cat > "$LOG_DIR/log4j2.xml" << "EOF"
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	155 <?xml version="1.0" encoding="UTF-8"?>
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	156 <Configuration>
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	157 <Appenders>
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	158 <RollingFile
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	159 name="RollingFile"
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	160 fileName="import.log"
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	161 filePattern="import.log.%i">
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	162 <PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5p %c - %m%n"/>
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	163 <SizeBasedTriggeringPolicy size="100000 KB"/>
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	164 <DefaultRolloverStrategy max="10"/>
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	165 </RollingFile>
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	166 </Appenders>
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	167 <Loggers>
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	168 <Root level="DEBUG">
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	169 <AppenderRef ref="RollingFile"/>
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	170 </Root>
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	171 </Loggers>
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	172 </Configuration>
7728 c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	173 EOF
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	174
5355 480d2f1f61cc Fix import_rivers script for postgres Andre Heinecke <aheinecke@intevation.de> parents: 5212 diff changeset	175 if [ "$POSTGRES" = "TRUE" ]; then
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	176 JAR=$(echo "$JAR" \| sed 's/importer/importer_psql/')
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	177 if [ ! -r "$JAR" ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	178 echo "Could not find Postgres importer $JAR"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	179 exit 1
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	180 fi
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	181 OGR_CONNECTION="PG:dbname=$BACKEND_NAME host=$DBHOST port=$DBPORT \
4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	182 user=$DBUSER password=$DBPASS"
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	183 BACKEND_DB_PREFIX="jdbc:postgresql:"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	184 BACKEND_DB_DRIVER="org.postgresql.Driver"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	185 BACKEND_DB_DIALECT="org.hibernate.dialect.PostgreSQLDialect"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	186 else
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	187 BACKEND_DB_PREFIX="jdbc:oracle:thin:@"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	188 BACKEND_DB_DRIVER="oracle.jdbc.OracleDriver"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	189 BACKEND_DB_DIALECT="org.hibernate.dialect.OracleDialect"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	190 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	191
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	192 BACKEND_URL=$BACKEND_DB_PREFIX//$DBHOST:$DBPORT/$BACKEND_NAME
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	193
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	194 echo "Importing $RIVER_NAME into $BACKEND_URL."
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	195
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	196 import_hydro(){
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	197 LOG_FILE=${LOG_DIR}/hydro.log
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	198 echo Importing Hydrological data.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	199 echo Logging into: $LOG_FILE
9726 0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	200 sed -i "s/import.log/$LOG_FILE/" $LOG_DIR/log4j2.xml
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	201 java -jar \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	202 -Xmx$MIN_MEMORY \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	203 -server \
9726 0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	204 -Dlog4j2.configurationFile=file://$LOG_DIR/log4j2.xml \
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	205 -Dflys.backend.user=$DBUSER \
4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	206 -Dflys.backend.password=$DBPASS \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	207 -Dflys.backend.url=$BACKEND_URL \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	208 -Dflys.backend.driver=$BACKEND_DB_DRIVER \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	209 -Dflys.backend.dialect=$BACKEND_DB_DIALECT \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	210 -Dflys.backend.importer.infogew.file="$GEW_FILE" \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	211 -Dflys.backend.main.value.types=$IMPORTER_MAINVALUE_TYPES \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	212 -Dflys.backend.importer.annotation.types=$IMPORTER_ANNOTATION_TYPES \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	213 -Dflys.backend.importer.dry.run=$IMPORTER_DRY_RUN \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	214 -Dflys.backend.importer.skip.annotations=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	215 -Dflys.backend.importer.skip.bwastr=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	216 -Dflys.backend.importer.skip.extra.wsts=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	217 -Dflys.backend.importer.skip.fixations=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	218 -Dflys.backend.importer.skip.flood.water=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	219 -Dflys.backend.importer.skip.flood.protection=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	220 -Dflys.backend.importer.skip.gauges=false \
7326 5e621410db05 Importer: skip historical discharge tables per default as they come via ETL. Tom Gottfried <tom@intevation.de> parents: 6825 diff changeset	221 -Dflys.backend.importer.skip.historical.discharge.tables=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	222 -Dflys.backend.importer.skip.hyks=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	223 -Dflys.backend.importer.skip.official.lines=false \
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	224 -Dflys.backend.importer.skip.prfs=true \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	225 -Dflys.backend.importer.skip.w80s=true \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	226 -Dflys.backend.importer.skip.w80.csvs=true \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	227 -Dflys.backend.importer.skip.da50s=true \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	228 -Dflys.backend.importer.skip.da66s=true \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	229 -Dflys.backend.importer.skip.wst=false \
6417 14b5674e57a4 Importer: add skip flag for measurement stations to scripts Tom Gottfried <tom@intevation.de> parents: 5838 diff changeset	230 -Dflys.backend.importer.skip.measurement.stations=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	231 -Dflys.backend.importer.skip.waterlevel.differences=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	232 -Dflys.backend.importer.skip.waterlevels=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	233 -Dflys.backend.importer.skip.sq.relation=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	234 -Dflys.backend.importer.skip.sediment.density=true \
8056 d86cc6a17b7a Importer: Import sediment load at measurement stations. Tom Gottfried <tom@intevation.de> parents: 8032 diff changeset	235 -Dflys.backend.importer.skip.sediment.load=true \
8032 fd3a24336e6a SCHEMA CHANGE and Importer (only longitudinal section data so far): only grain fractions given now in schema are valid. We are handling sediment loads, not yields. "Tom Gottfried <tom@intevation.de>" parents: 7925 diff changeset	236 -Dflys.backend.importer.skip.sediment.load.ls=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	237 -Dflys.backend.importer.skip.morphological.width=true \
8819 79f9d837c5cc Add missing skip-option in import script. Tom Gottfried <tom@intevation.de> parents: 8675 diff changeset	238 -Dflys.backend.importer.skip.porosity=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	239 -Dflys.backend.importer.skip.flow.velocity=true \
8674 44ba71b0b75a (issue1763) Correct properties in importer scripts. Tom Gottfried <tom@intevation.de> parents: 8056 diff changeset	240 -Dflys.backend.importer.skip.bed.height=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	241 $JAR
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	242 }
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	243
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	244 import_morpho(){
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	245 LOG_FILE=${LOG_DIR}/morpho.log
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	246 echo Importing Morphological data.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	247 echo Logging into: $LOG_FILE
9726 0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	248 sed -i "s/import.log/$LOG_FILE/" $LOG_DIR/log4j2.xml
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	249 java -jar \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	250 -Xmx$MIN_MEMORY \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	251 -server \
9726 0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	252 -Dlog4j2.configurationFile=file://$LOG_DIR/log4j2.xml \
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	253 -Dflys.backend.user=$DBUSER \
4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	254 -Dflys.backend.password=$DBPASS \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	255 -Dflys.backend.url=$BACKEND_URL \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	256 -Dflys.backend.driver=$BACKEND_DB_DRIVER \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	257 -Dflys.backend.dialect=$BACKEND_DB_DIALECT \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	258 -Dflys.backend.importer.infogew.file="$GEW_FILE" \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	259 -Dflys.backend.main.value.types=$IMPORTER_MAINVALUE_TYPES \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	260 -Dflys.backend.importer.annotation.types=$IMPORTER_ANNOTATION_TYPES \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	261 -Dflys.backend.importer.dry.run=$IMPORTER_DRY_RUN \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	262 -Dflys.backend.importer.skip.annotations=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	263 -Dflys.backend.importer.skip.bwastr=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	264 -Dflys.backend.importer.skip.extra.wsts=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	265 -Dflys.backend.importer.skip.fixations=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	266 -Dflys.backend.importer.skip.flood.water=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	267 -Dflys.backend.importer.skip.flood.protection=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	268 -Dflys.backend.importer.skip.gauges=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	269 -Dflys.backend.importer.skip.historical.discharge.tables=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	270 -Dflys.backend.importer.skip.hyks=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	271 -Dflys.backend.importer.skip.official.lines=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	272 -Dflys.backend.importer.skip.prfs=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	273 -Dflys.backend.importer.skip.w80s=true \
6706 c3e38ea87964 Importer scripts: add skip flag for W80-CSV to import_river.sh Tom Gottfried <tom@intevation.de> parents: 6658 diff changeset	274 -Dflys.backend.importer.skip.w80.csvs=true \
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	275 -Dflys.backend.importer.skip.da50s=true \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	276 -Dflys.backend.importer.skip.da66s=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	277 -Dflys.backend.importer.skip.wst=true \
6417 14b5674e57a4 Importer: add skip flag for measurement stations to scripts Tom Gottfried <tom@intevation.de> parents: 5838 diff changeset	278 -Dflys.backend.importer.skip.measurement.stations=false \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	279 -Dflys.backend.importer.skip.waterlevel.differences=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	280 -Dflys.backend.importer.skip.waterlevels=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	281 -Dflys.backend.importer.skip.sq.relation=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	282 -Dflys.backend.importer.skip.sediment.density=false \
8056 d86cc6a17b7a Importer: Import sediment load at measurement stations. Tom Gottfried <tom@intevation.de> parents: 8032 diff changeset	283 -Dflys.backend.importer.skip.sediment.load=false \
8032 fd3a24336e6a SCHEMA CHANGE and Importer (only longitudinal section data so far): only grain fractions given now in schema are valid. We are handling sediment loads, not yields. "Tom Gottfried <tom@intevation.de>" parents: 7925 diff changeset	284 -Dflys.backend.importer.skip.sediment.load.ls=false \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	285 -Dflys.backend.importer.skip.morphological.width=false \
8819 79f9d837c5cc Add missing skip-option in import script. Tom Gottfried <tom@intevation.de> parents: 8675 diff changeset	286 -Dflys.backend.importer.skip.porosity=false \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	287 -Dflys.backend.importer.skip.flow.velocity=false \
8674 44ba71b0b75a (issue1763) Correct properties in importer scripts. Tom Gottfried <tom@intevation.de> parents: 8056 diff changeset	288 -Dflys.backend.importer.skip.bed.height=false \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	289 $JAR
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	290 }
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	291
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	292 import_prf(){
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	293 LOG_FILE=${LOG_DIR}/prf.log
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	294 echo Importing cross section data.
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	295 echo Logging into: $LOG_FILE
9726 0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	296 sed -i "s/import.log/$LOG_FILE/" $LOG_DIR/log4j2.xml
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	297 java -jar \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	298 -Xmx$MIN_MEMORY \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	299 -server \
9726 0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	300 -Dlog4j2.configurationFile=file://$LOG_DIR/log4j2.xml \
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	301 -Dflys.backend.user=$DBUSER \
4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	302 -Dflys.backend.password=$DBPASS \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	303 -Dflys.backend.url=$BACKEND_URL \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	304 -Dflys.backend.driver=$BACKEND_DB_DRIVER \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	305 -Dflys.backend.dialect=$BACKEND_DB_DIALECT \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	306 -Dflys.backend.importer.infogew.file="$GEW_FILE" \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	307 -Dflys.backend.main.value.types=$IMPORTER_MAINVALUE_TYPES \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	308 -Dflys.backend.importer.annotation.types=$IMPORTER_ANNOTATION_TYPES \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	309 -Dflys.backend.importer.dry.run=$IMPORTER_DRY_RUN \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	310 -Dflys.backend.importer.skip.annotations=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	311 -Dflys.backend.importer.skip.bwastr=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	312 -Dflys.backend.importer.skip.extra.wsts=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	313 -Dflys.backend.importer.skip.fixations=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	314 -Dflys.backend.importer.skip.flood.water=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	315 -Dflys.backend.importer.skip.flood.protection=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	316 -Dflys.backend.importer.skip.gauges=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	317 -Dflys.backend.importer.skip.historical.discharge.tables=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	318 -Dflys.backend.importer.skip.hyks=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	319 -Dflys.backend.importer.skip.official.lines=true \
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	320 -Dflys.backend.importer.skip.prfs=false \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	321 -Dflys.backend.importer.skip.w80s=false \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	322 -Dflys.backend.importer.skip.w80.csvs=false \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	323 -Dflys.backend.importer.skip.da50s=false \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	324 -Dflys.backend.importer.skip.da66s=false \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	325 -Dflys.backend.importer.skip.wst=true \
6417 14b5674e57a4 Importer: add skip flag for measurement stations to scripts Tom Gottfried <tom@intevation.de> parents: 5838 diff changeset	326 -Dflys.backend.importer.skip.measurement.stations=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	327 -Dflys.backend.importer.skip.waterlevel.differences=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	328 -Dflys.backend.importer.skip.waterlevels=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	329 -Dflys.backend.importer.skip.sq.relation=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	330 -Dflys.backend.importer.skip.sediment.density=true \
8056 d86cc6a17b7a Importer: Import sediment load at measurement stations. Tom Gottfried <tom@intevation.de> parents: 8032 diff changeset	331 -Dflys.backend.importer.skip.sediment.load=true \
8032 fd3a24336e6a SCHEMA CHANGE and Importer (only longitudinal section data so far): only grain fractions given now in schema are valid. We are handling sediment loads, not yields. "Tom Gottfried <tom@intevation.de>" parents: 7925 diff changeset	332 -Dflys.backend.importer.skip.sediment.load.ls=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	333 -Dflys.backend.importer.skip.morphological.width=true \
8819 79f9d837c5cc Add missing skip-option in import script. Tom Gottfried <tom@intevation.de> parents: 8675 diff changeset	334 -Dflys.backend.importer.skip.porosity=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	335 -Dflys.backend.importer.skip.flow.velocity=true \
8674 44ba71b0b75a (issue1763) Correct properties in importer scripts. Tom Gottfried <tom@intevation.de> parents: 8056 diff changeset	336 -Dflys.backend.importer.skip.bed.height=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	337 $JAR
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	338 }
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	339
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	340 import_geo(){
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	341 LOG_FILE=${LOG_DIR}/geo.log
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	342 echo Importing Geographic data.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	343 echo Logging into: $LOG_FILE
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	344
5169 db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	345 RIVER_PATH=$(grep "WSTDatei:" "$GEW_FILE" \| awk '{print $2}')
db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	346 RIVER_PATH=$(dirname "$RIVER_PATH")/../..
db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	347 RIVER_PATH=$(readlink -f "$RIVER_PATH")
db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	348
8823 efa52271786b Fix path to python importer. Tom Gottfried <tom@intevation.de> parents: 8820 diff changeset	349 exec python $(dirname $0)/shpimporter/shpimporter.py \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	350 --directory $RIVER_PATH \
5563 673f68bb19a4 Geo-importer: accept river names with white space Tom Gottfried <tom@intevation.de> parents: 5459 diff changeset	351 --river_name "$RIVER_NAME" \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	352 --ogr_connection "$OGR_CONNECTION" \
6658 89f856e24d27 Importer scripts: shpimporter needs backend name to play with Oracle. Tom Gottfried <tom@intevation.de> parents: 6651 diff changeset	353 --host $DBHOST/$BACKEND_NAME \
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	354 --user $DBUSER \
4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	355 --password $DBPASS \
5169 db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	356 --verbose 1 \
db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	357 > "$LOG_FILE" 2>&1
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	358 }
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	359
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	360
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	361 if [ "$SKIP_HYDRO" != "TRUE" ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	362 import_hydro
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	363 fi
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	364 if [ "$SKIP_PRF" != "TRUE" ]; then
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	365 import_prf
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	366 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	367 if [ "$SKIP_MORPHO" != "TRUE" ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	368 import_morpho
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	369 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	370 if [ "$SKIP_GEO" != "TRUE" ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	371 import_geo
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	372 fi

Mercurial > dive4elements > river

annotate backend/contrib/import_river.sh @ 9744:b1b48fa7bd80 3.2.x