dive4elements/river: backend/contrib/import

annotate backend/contrib/import_river.sh @ 9751:308a0d822d18 3.2.x

Keep configuration and data in distinct directories This allows having distinct volumes for configuration and data (artifact database, generated mapfiles and shapefiles, etc.). While at it, cleanup MapServer configuration a little bit.

author	Tom Gottfried <tom@intevation.de>
date	Tue, 11 Oct 2022 11:42:09 +0200
parents	0a5239a1e46e
children

rev	line source
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	1 #!/bin/bash
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	2 # Import script for rivers
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	3 #
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	4 # Authors:
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	5 # Andre Heinecke <aheinecke@intevation.de>
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	6 #
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	7 # Copyright:
5168 650f94ec3122 Fix copyright on import_river.sh Andre Heinecke <aheinecke@intevation.de> parents: 5167 diff changeset	8 # Copyright (C) 2013 Intevation GmbH
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	9 #
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	10 # This program is free software; you can redistribute it and/or
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	11 # modify it under the terms of the GNU General Public License
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	12 # as published by the Free Software Foundation; either version 2
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	13 # of the License, or (at your option) any later version.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	14 #
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	15 # This program is distributed in the hope that it will be useful,
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	16 # but WITHOUT ANY WARRANTY; without even the implied warranty of
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	17 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	18 # GNU General Public License for more details.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	19 #
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	20 # You should have received a copy of the GNU General Public License
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	21 # along with this program; if not, write to the Free Software
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	22 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	23
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	24 set -e
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	25
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	26 # Default settings
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	27 DEFAULT_HOST=localhost
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	28 DEFAULT_PORT=1521
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	29 DEFAULT_USER=flys_dami
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	30 DEFAULT_PASS=flys_dami
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	31 DEFAULT_LOG=$PWD/logs
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	32 DEFAULT_BACKEND_NAME="XE"
8675 ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems. Tom Gottfried <tom@intevation.de> parents: 8674 diff changeset	33 JAR="bin/river-backend-1.0-SNAPSHOT*.jar"
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	34 IMPORTER_DRY_RUN=false
6449 25912df0facf Import main values of type '-' by default Tom Gottfried <tom.gottfried@intevation.de> parents: 6417 diff changeset	35 IMPORTER_MAINVALUE_TYPES=QWTD-
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	36 IMPORTER_ANNOTATION_TYPES="conf/annotation-types.xml"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	37
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	38
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	39 MIN_MEMORY="8024m"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	40
6651 8782ba5cb222 Importer scripts: add default encoding to correctly read gew-files Tom Gottfried <tom@intevation.de> parents: 6449 diff changeset	41 # Default encoding. Change here if necessary
8782ba5cb222 Importer scripts: add default encoding to correctly read gew-files Tom Gottfried <tom@intevation.de> parents: 6449 diff changeset	42 export LC_ALL=de_DE@euro
8782ba5cb222 Importer scripts: add default encoding to correctly read gew-files Tom Gottfried <tom@intevation.de> parents: 6449 diff changeset	43
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	44 usage(){
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	45 cat << EOF
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	46
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	47 usage: $0 [options] gew_file
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	48
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	49 Import a river described by the gew_file
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	50
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	51 OPTIONS:
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	52 -?, --help Show this message
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	53 -u, --username=<username> Database username. Default: $DEFAULT_USER
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	54 -w, --password=<password> Database password. Default: $DEFAULT_PASS
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	55 -h, --host=<host> Connect to database on host <host>.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	56 Default: $DEFAULT_HOST
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	57 -p, --port=<number> Use port number <number>. Default: $DEFAULT_PORT
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	58 -d, --db-name=<database_name> Name of the database / backend. Default: $DEFAULT_BACKEND_NAME
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	59 -l, --log-dir=<directory> Directory in which to create the log files.
7728 c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	60 Default: $PWD/logs
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	61 --postgres Database is PostgreSQL
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	62 --skip-hydro Skip import of hydrological data
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	63 --skip-morpho Skip import of morphological data
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	64 --skip-geo Skip import of geographic data
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	65 --skip-prf Skip import of cross section data
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	66 EOF
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	67 exit 0
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	68 }
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	69
7728 c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	70 OPTS=`getopt -o ?u:w:h:p:d:l: \
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	71 -l help,username:,password:,host:,port:,db-name:,log-dir:,skip-hydro,skip-morpho,skip-geo,skip-prf,postgres \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	72 -n $0 -- "$@"`
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	73 if [ $? != 0 ] ; then usage; fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	74 eval set -- "$OPTS"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	75 while true ; do
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	76 case "$1" in
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	77 "-?"\|"--help")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	78 usage;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	79 "--")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	80 shift
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	81 break;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	82 "-u"\|"--username")
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	83 DBUSER=$2
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	84 shift 2;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	85 "-w"\|"--password")
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	86 DBPASS=$2
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	87 shift 2;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	88 "-h"\|"--host")
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	89 DBHOST=$2
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	90 shift 2;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	91 "-p"\|"--port")
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	92 DBPORT=$2
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	93 shift 2;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	94 "-l"\|"--log-dir")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	95 LOG=$2
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	96 shift 2;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	97 "-d"\|"--db-name")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	98 BACKEND_NAME=$2
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	99 shift 2;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	100 "--skip-hydro")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	101 SKIP_HYDRO="TRUE"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	102 shift;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	103 "--skip-morpho")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	104 SKIP_MORPHO="TRUE"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	105 shift;;
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	106 "--skip-prf")
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	107 SKIP_PRF="TRUE"
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	108 shift;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	109 "--skip-geo")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	110 SKIP_GEO="TRUE"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	111 shift;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	112 "--postgres")
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	113 POSTGRES="TRUE"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	114 shift;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	115 *)
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	116 echo "Unknown Option $1"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	117 usage;;
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	118 esac
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	119 done
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	120
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	121 if [ -z $DBUSER ]; then
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	122 DBUSER=$DEFAULT_USER
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	123 fi
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	124 if [ -z $DBPASS ]; then
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	125 DBPASS=$DEFAULT_PASS
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	126 fi
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	127 if [ -z $DBPORT ]; then
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	128 DBPORT=$DEFAULT_PORT
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	129 fi
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	130 if [ -z $DBHOST ]; then
5192 1b6d4edeeb4c Fix variable assignments after rename Andre Heinecke <aheinecke@intevation.de> parents: 5191 diff changeset	131 DBHOST=$DEFAULT_HOST
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	132 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	133 if [ -z $BACKEND_NAME ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	134 BACKEND_NAME=$DEFAULT_BACKEND_NAME
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	135 fi
7728 c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	136 if [ -z $LOG ]; then
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	137 LOG=$DEFAULT_LOG
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	138 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	139
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	140 if [ $# != 1 ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	141 usage
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	142 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	143
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	144 if [ ! -r $1 ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	145 echo "Could not open $1 please ensure it exists and is readable"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	146 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	147
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	148 GEW_FILE="$1"
5459 7c646d4d5103 be more flexible with river names (e.g. containing white space) in import script Tom Gottfried <tom@intevation.de> parents: 5355 diff changeset	149 RIVER_NAME=$(grep "Gew.sser" "$1" \| sed 's/Gew.sser: //')
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	150 DATE=$(date +%Y.%m.%d_%H%M)
5459 7c646d4d5103 be more flexible with river names (e.g. containing white space) in import script Tom Gottfried <tom@intevation.de> parents: 5355 diff changeset	151 LOG_DIR=${LOG}/`basename $GEW_FILE .gew`-$DATE
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	152 mkdir -p ${LOG_DIR}
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	153
9726 0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	154 cat > "$LOG_DIR/log4j2.xml" << "EOF"
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	155 <?xml version="1.0" encoding="UTF-8"?>
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	156 <Configuration>
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	157 <Appenders>
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	158 <RollingFile
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	159 name="RollingFile"
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	160 fileName="import.log"
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	161 filePattern="import.log.%i">
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	162 <PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5p %c - %m%n"/>
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	163 <SizeBasedTriggeringPolicy size="100000 KB"/>
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	164 <DefaultRolloverStrategy max="10"/>
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	165 </RollingFile>
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	166 </Appenders>
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	167 <Loggers>
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	168 <Root level="DEBUG">
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	169 <AppenderRef ref="RollingFile"/>
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	170 </Root>
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	171 </Loggers>
0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	172 </Configuration>
7728 c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	173 EOF
c842ebb0db86 Importer: fix logging in import_river.sh. Tom Gottfried <tom@intevation.de> parents: 7326 diff changeset	174
5355 480d2f1f61cc Fix import_rivers script for postgres Andre Heinecke <aheinecke@intevation.de> parents: 5212 diff changeset	175 if [ "$POSTGRES" = "TRUE" ]; then
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	176 JAR=$(echo "$JAR" \| sed 's/importer/importer_psql/')
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	177 if [ ! -r "$JAR" ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	178 echo "Could not find Postgres importer $JAR"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	179 exit 1
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	180 fi
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	181 OGR_CONNECTION="PG:dbname=$BACKEND_NAME host=$DBHOST port=$DBPORT \
4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	182 user=$DBUSER password=$DBPASS"
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	183 BACKEND_DB_PREFIX="jdbc:postgresql:"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	184 BACKEND_DB_DRIVER="org.postgresql.Driver"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	185 BACKEND_DB_DIALECT="org.hibernate.dialect.PostgreSQLDialect"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	186 else
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	187 BACKEND_DB_PREFIX="jdbc:oracle:thin:@"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	188 BACKEND_DB_DRIVER="oracle.jdbc.OracleDriver"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	189 BACKEND_DB_DIALECT="org.hibernate.dialect.OracleDialect"
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	190 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	191
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	192 BACKEND_URL=$BACKEND_DB_PREFIX//$DBHOST:$DBPORT/$BACKEND_NAME
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	193
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	194 echo "Importing $RIVER_NAME into $BACKEND_URL."
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	195
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	196 import_hydro(){
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	197 LOG_FILE=${LOG_DIR}/hydro.log
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	198 echo Importing Hydrological data.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	199 echo Logging into: $LOG_FILE
9726 0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	200 sed -i "s/import.log/$LOG_FILE/" $LOG_DIR/log4j2.xml
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	201 java -jar \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	202 -Xmx$MIN_MEMORY \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	203 -server \
9726 0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	204 -Dlog4j2.configurationFile=file://$LOG_DIR/log4j2.xml \
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	205 -Dflys.backend.user=$DBUSER \
4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	206 -Dflys.backend.password=$DBPASS \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	207 -Dflys.backend.url=$BACKEND_URL \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	208 -Dflys.backend.driver=$BACKEND_DB_DRIVER \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	209 -Dflys.backend.dialect=$BACKEND_DB_DIALECT \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	210 -Dflys.backend.importer.infogew.file="$GEW_FILE" \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	211 -Dflys.backend.main.value.types=$IMPORTER_MAINVALUE_TYPES \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	212 -Dflys.backend.importer.annotation.types=$IMPORTER_ANNOTATION_TYPES \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	213 -Dflys.backend.importer.dry.run=$IMPORTER_DRY_RUN \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	214 -Dflys.backend.importer.skip.annotations=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	215 -Dflys.backend.importer.skip.bwastr=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	216 -Dflys.backend.importer.skip.extra.wsts=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	217 -Dflys.backend.importer.skip.fixations=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	218 -Dflys.backend.importer.skip.flood.water=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	219 -Dflys.backend.importer.skip.flood.protection=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	220 -Dflys.backend.importer.skip.gauges=false \
7326 5e621410db05 Importer: skip historical discharge tables per default as they come via ETL. Tom Gottfried <tom@intevation.de> parents: 6825 diff changeset	221 -Dflys.backend.importer.skip.historical.discharge.tables=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	222 -Dflys.backend.importer.skip.hyks=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	223 -Dflys.backend.importer.skip.official.lines=false \
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	224 -Dflys.backend.importer.skip.prfs=true \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	225 -Dflys.backend.importer.skip.w80s=true \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	226 -Dflys.backend.importer.skip.w80.csvs=true \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	227 -Dflys.backend.importer.skip.da50s=true \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	228 -Dflys.backend.importer.skip.da66s=true \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	229 -Dflys.backend.importer.skip.wst=false \
6417 14b5674e57a4 Importer: add skip flag for measurement stations to scripts Tom Gottfried <tom@intevation.de> parents: 5838 diff changeset	230 -Dflys.backend.importer.skip.measurement.stations=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	231 -Dflys.backend.importer.skip.waterlevel.differences=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	232 -Dflys.backend.importer.skip.waterlevels=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	233 -Dflys.backend.importer.skip.sq.relation=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	234 -Dflys.backend.importer.skip.sediment.density=true \
8056 d86cc6a17b7a Importer: Import sediment load at measurement stations. Tom Gottfried <tom@intevation.de> parents: 8032 diff changeset	235 -Dflys.backend.importer.skip.sediment.load=true \
8032 fd3a24336e6a SCHEMA CHANGE and Importer (only longitudinal section data so far): only grain fractions given now in schema are valid. We are handling sediment loads, not yields. "Tom Gottfried <tom@intevation.de>" parents: 7925 diff changeset	236 -Dflys.backend.importer.skip.sediment.load.ls=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	237 -Dflys.backend.importer.skip.morphological.width=true \
8819 79f9d837c5cc Add missing skip-option in import script. Tom Gottfried <tom@intevation.de> parents: 8675 diff changeset	238 -Dflys.backend.importer.skip.porosity=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	239 -Dflys.backend.importer.skip.flow.velocity=true \
8674 44ba71b0b75a (issue1763) Correct properties in importer scripts. Tom Gottfried <tom@intevation.de> parents: 8056 diff changeset	240 -Dflys.backend.importer.skip.bed.height=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	241 $JAR
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	242 }
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	243
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	244 import_morpho(){
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	245 LOG_FILE=${LOG_DIR}/morpho.log
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	246 echo Importing Morphological data.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	247 echo Logging into: $LOG_FILE
9726 0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	248 sed -i "s/import.log/$LOG_FILE/" $LOG_DIR/log4j2.xml
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	249 java -jar \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	250 -Xmx$MIN_MEMORY \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	251 -server \
9726 0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	252 -Dlog4j2.configurationFile=file://$LOG_DIR/log4j2.xml \
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	253 -Dflys.backend.user=$DBUSER \
4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	254 -Dflys.backend.password=$DBPASS \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	255 -Dflys.backend.url=$BACKEND_URL \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	256 -Dflys.backend.driver=$BACKEND_DB_DRIVER \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	257 -Dflys.backend.dialect=$BACKEND_DB_DIALECT \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	258 -Dflys.backend.importer.infogew.file="$GEW_FILE" \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	259 -Dflys.backend.main.value.types=$IMPORTER_MAINVALUE_TYPES \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	260 -Dflys.backend.importer.annotation.types=$IMPORTER_ANNOTATION_TYPES \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	261 -Dflys.backend.importer.dry.run=$IMPORTER_DRY_RUN \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	262 -Dflys.backend.importer.skip.annotations=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	263 -Dflys.backend.importer.skip.bwastr=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	264 -Dflys.backend.importer.skip.extra.wsts=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	265 -Dflys.backend.importer.skip.fixations=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	266 -Dflys.backend.importer.skip.flood.water=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	267 -Dflys.backend.importer.skip.flood.protection=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	268 -Dflys.backend.importer.skip.gauges=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	269 -Dflys.backend.importer.skip.historical.discharge.tables=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	270 -Dflys.backend.importer.skip.hyks=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	271 -Dflys.backend.importer.skip.official.lines=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	272 -Dflys.backend.importer.skip.prfs=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	273 -Dflys.backend.importer.skip.w80s=true \
6706 c3e38ea87964 Importer scripts: add skip flag for W80-CSV to import_river.sh Tom Gottfried <tom@intevation.de> parents: 6658 diff changeset	274 -Dflys.backend.importer.skip.w80.csvs=true \
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	275 -Dflys.backend.importer.skip.da50s=true \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	276 -Dflys.backend.importer.skip.da66s=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	277 -Dflys.backend.importer.skip.wst=true \
6417 14b5674e57a4 Importer: add skip flag for measurement stations to scripts Tom Gottfried <tom@intevation.de> parents: 5838 diff changeset	278 -Dflys.backend.importer.skip.measurement.stations=false \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	279 -Dflys.backend.importer.skip.waterlevel.differences=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	280 -Dflys.backend.importer.skip.waterlevels=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	281 -Dflys.backend.importer.skip.sq.relation=false \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	282 -Dflys.backend.importer.skip.sediment.density=false \
8056 d86cc6a17b7a Importer: Import sediment load at measurement stations. Tom Gottfried <tom@intevation.de> parents: 8032 diff changeset	283 -Dflys.backend.importer.skip.sediment.load=false \
8032 fd3a24336e6a SCHEMA CHANGE and Importer (only longitudinal section data so far): only grain fractions given now in schema are valid. We are handling sediment loads, not yields. "Tom Gottfried <tom@intevation.de>" parents: 7925 diff changeset	284 -Dflys.backend.importer.skip.sediment.load.ls=false \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	285 -Dflys.backend.importer.skip.morphological.width=false \
8819 79f9d837c5cc Add missing skip-option in import script. Tom Gottfried <tom@intevation.de> parents: 8675 diff changeset	286 -Dflys.backend.importer.skip.porosity=false \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	287 -Dflys.backend.importer.skip.flow.velocity=false \
8674 44ba71b0b75a (issue1763) Correct properties in importer scripts. Tom Gottfried <tom@intevation.de> parents: 8056 diff changeset	288 -Dflys.backend.importer.skip.bed.height=false \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	289 $JAR
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	290 }
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	291
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	292 import_prf(){
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	293 LOG_FILE=${LOG_DIR}/prf.log
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	294 echo Importing cross section data.
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	295 echo Logging into: $LOG_FILE
9726 0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	296 sed -i "s/import.log/$LOG_FILE/" $LOG_DIR/log4j2.xml
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	297 java -jar \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	298 -Xmx$MIN_MEMORY \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	299 -server \
9726 0a5239a1e46e Upgrade to Log4j 2 Tom Gottfried <tom@intevation.de> parents: 8829 diff changeset	300 -Dlog4j2.configurationFile=file://$LOG_DIR/log4j2.xml \
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	301 -Dflys.backend.user=$DBUSER \
4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	302 -Dflys.backend.password=$DBPASS \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	303 -Dflys.backend.url=$BACKEND_URL \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	304 -Dflys.backend.driver=$BACKEND_DB_DRIVER \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	305 -Dflys.backend.dialect=$BACKEND_DB_DIALECT \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	306 -Dflys.backend.importer.infogew.file="$GEW_FILE" \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	307 -Dflys.backend.main.value.types=$IMPORTER_MAINVALUE_TYPES \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	308 -Dflys.backend.importer.annotation.types=$IMPORTER_ANNOTATION_TYPES \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	309 -Dflys.backend.importer.dry.run=$IMPORTER_DRY_RUN \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	310 -Dflys.backend.importer.skip.annotations=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	311 -Dflys.backend.importer.skip.bwastr=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	312 -Dflys.backend.importer.skip.extra.wsts=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	313 -Dflys.backend.importer.skip.fixations=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	314 -Dflys.backend.importer.skip.flood.water=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	315 -Dflys.backend.importer.skip.flood.protection=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	316 -Dflys.backend.importer.skip.gauges=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	317 -Dflys.backend.importer.skip.historical.discharge.tables=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	318 -Dflys.backend.importer.skip.hyks=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	319 -Dflys.backend.importer.skip.official.lines=true \
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	320 -Dflys.backend.importer.skip.prfs=false \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	321 -Dflys.backend.importer.skip.w80s=false \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	322 -Dflys.backend.importer.skip.w80.csvs=false \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	323 -Dflys.backend.importer.skip.da50s=false \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	324 -Dflys.backend.importer.skip.da66s=false \
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	325 -Dflys.backend.importer.skip.wst=true \
6417 14b5674e57a4 Importer: add skip flag for measurement stations to scripts Tom Gottfried <tom@intevation.de> parents: 5838 diff changeset	326 -Dflys.backend.importer.skip.measurement.stations=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	327 -Dflys.backend.importer.skip.waterlevel.differences=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	328 -Dflys.backend.importer.skip.waterlevels=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	329 -Dflys.backend.importer.skip.sq.relation=true \
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	330 -Dflys.backend.importer.skip.sediment.density=true \
8056 d86cc6a17b7a Importer: Import sediment load at measurement stations. Tom Gottfried <tom@intevation.de> parents: 8032 diff changeset	331 -Dflys.backend.importer.skip.sediment.load=true \
8032 fd3a24336e6a SCHEMA CHANGE and Importer (only longitudinal section data so far): only grain fractions given now in schema are valid. We are handling sediment loads, not yields. "Tom Gottfried <tom@intevation.de>" parents: 7925 diff changeset	332 -Dflys.backend.importer.skip.sediment.load.ls=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	333 -Dflys.backend.importer.skip.morphological.width=true \
8819 79f9d837c5cc Add missing skip-option in import script. Tom Gottfried <tom@intevation.de> parents: 8675 diff changeset	334 -Dflys.backend.importer.skip.porosity=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	335 -Dflys.backend.importer.skip.flow.velocity=true \
8674 44ba71b0b75a (issue1763) Correct properties in importer scripts. Tom Gottfried <tom@intevation.de> parents: 8056 diff changeset	336 -Dflys.backend.importer.skip.bed.height=true \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	337 $JAR
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	338 }
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	339
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	340 import_geo(){
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	341 LOG_FILE=${LOG_DIR}/geo.log
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	342 echo Importing Geographic data.
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	343 echo Logging into: $LOG_FILE
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	344
5169 db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	345 RIVER_PATH=$(grep "WSTDatei:" "$GEW_FILE" \| awk '{print $2}')
db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	346 RIVER_PATH=$(dirname "$RIVER_PATH")/../..
db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	347 RIVER_PATH=$(readlink -f "$RIVER_PATH")
db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	348
8823 efa52271786b Fix path to python importer. Tom Gottfried <tom@intevation.de> parents: 8820 diff changeset	349 exec python $(dirname $0)/shpimporter/shpimporter.py \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	350 --directory $RIVER_PATH \
5563 673f68bb19a4 Geo-importer: accept river names with white space Tom Gottfried <tom@intevation.de> parents: 5459 diff changeset	351 --river_name "$RIVER_NAME" \
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	352 --ogr_connection "$OGR_CONNECTION" \
6658 89f856e24d27 Importer scripts: shpimporter needs backend name to play with Oracle. Tom Gottfried <tom@intevation.de> parents: 6651 diff changeset	353 --host $DBHOST/$BACKEND_NAME \
5191 4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	354 --user $DBUSER \
4aa748db8be3 Use less ambigious names for variables Andre Heinecke <aheinecke@intevation.de> parents: 5176 diff changeset	355 --password $DBPASS \
5169 db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	356 --verbose 1 \
db1a000a21a9 Make OPTIONAL_LIBS overwritable and fix geo importer call Andre Heinecke <aheinecke@intevation.de> parents: 5168 diff changeset	357 > "$LOG_FILE" 2>&1
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	358 }
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	359
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	360
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	361 if [ "$SKIP_HYDRO" != "TRUE" ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	362 import_hydro
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	363 fi
6825 3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	364 if [ "$SKIP_PRF" != "TRUE" ]; then
3aff1b00618a import_rivers.sh: skip cross sections instead of WST-file in first run. Tom Gottfried <tom.gottfried@intevation.de> parents: 6706 diff changeset	365 import_prf
5167 a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	366 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	367 if [ "$SKIP_MORPHO" != "TRUE" ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	368 import_morpho
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	369 fi
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	370 if [ "$SKIP_GEO" != "TRUE" ]; then
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	371 import_geo
a0abb6787ab1 Add first version of import_river script Andre Heinecke <aheinecke@intevation.de> parents: diff changeset	372 fi

Mercurial > dive4elements > river

annotate backend/contrib/import_river.sh @ 9751:308a0d822d18 3.2.x