Mercurial > dive4elements > river

annotate contrib/make_flys_release/bin/make-importer-package.sh @ 9650:a2a42a6bac6b

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Importer (s/u-info) extensions: outer try/catch for parse and log of line no, catching parsing exception if not enough value fields, parsing error and warning log messages with line number, detecting and rejecting duplicate data series, better differentiation between error and warning log messages
author mschaefer
date Mon, 23 Mar 2020 14:57:03 +0100
parents f7007b1aaa3f
children f3aa7a12c86a
rev   line source
5097
415ac9bc5a56 Add importer packaging script and some doc about it
Andre Heinecke <aheinecke@intevation.de>
parents:
diff changeset
1 #!/bin/bash
8675
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
2 # Packaging-script for Dive4Elements River-Importer
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
3 #
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
4 # Copyright (C) 2011 - 2014 by Bundesanstalt für Gewässerkunde
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
5 # Software engineering by Intevation GmbH
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
6 #
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
7 # This file is Free Software under the GNU AGPL (>=v3)
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
8 # and comes with ABSOLUTELY NO WARRANTY! Check out the
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
9 # documentation coming with Dive4Elements River for details.
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
10
5097
415ac9bc5a56 Add importer packaging script and some doc about it
Andre Heinecke <aheinecke@intevation.de>
parents:
diff changeset
11 set -e
415ac9bc5a56 Add importer packaging script and some doc about it
Andre Heinecke <aheinecke@intevation.de>
parents:
diff changeset
12
8675
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
13 # See ../README for more information
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
14 # The working directory. Resulting tarball will be placed here.
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
15 PKG_DIR=/tmp/flys-importer
5097
415ac9bc5a56 Add importer packaging script and some doc about it
Andre Heinecke <aheinecke@intevation.de>
parents:
diff changeset
16 # Path to the flys checkout
8675
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
17 SOURCE_DIR=$(readlink -f `dirname $0`)/../../..
5097
415ac9bc5a56 Add importer packaging script and some doc about it
Andre Heinecke <aheinecke@intevation.de>
parents:
diff changeset
18
8675
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
19 usage(){
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
20 cat << EOF
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
21
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
22 usage: $0 [options] VERSION [EXTRAS]
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
23
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
24 Create a D4E River Importer-package
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
25
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
26 OPTIONS:
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
27 -?, --help Show this message
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
28 -o, --oracle Package is for Oracle.
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
29 Suitable jar-file for JDBC driver has to be given with
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
30 the environment variable ORACLE_JDBC
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
31 VERSION must specify a tag (usually MAYOR.MINOR.PATCH) or a branch name.
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
32 With EXTRAS, a tarball with dependencies can be given.
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
33 EOF
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
34 exit 0
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
35 }
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
36
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
37 OPTS=`getopt -o ?,o -l help,oracle -n $0 -- "$@"`
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
38
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
39 if [ $? != 0 ] ; then usage; fi
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
40 eval set -- "$OPTS"
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
41 while true ; do
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
42 case "$1" in
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
43 "-?"|"--help")
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
44 usage;;
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
45 "--")
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
46 shift
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
47 break;;
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
48 "-o"|"--oracle")
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
49 BUILD_ORACLE="TRUE"
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
50 shift;;
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
51 *)
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
52 echo "Unknown Option $1"
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
53 usage;;
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
54 esac
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
55 done
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
56
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
57 if [ $# != 1 ]; then
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
58 usage
5179
4355b4cecef8 Improve importer packaging by just expecting a tarball with
Andre Heinecke <aheinecke@intevation.de>
parents: 5172
diff changeset
59 fi
5097
415ac9bc5a56 Add importer packaging script and some doc about it
Andre Heinecke <aheinecke@intevation.de>
parents:
diff changeset
60
8675
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
61 VERSION=$1
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
62 EXTRAS=$2
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
63
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
64 # Update to VERSION
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
65 echo "WARNING: any local changes in $SOURCE_DIR will be packaged."
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
66 cd ${SOURCE_DIR}
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
67 if [ -z "`hg tags | sed -n "/$VERSION/p"`" -a -z "`hg branches | sed -n "/$VERSION/p"`" ]
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
68 then
8742
f7007b1aaa3f Importer-packaging: just a warning would end up with a package that really looks like VERSION.
Tom Gottfried <tom@intevation.de>
parents: 8675
diff changeset
69 echo "ERROR: No tag or branch $VERSION found in repository of $repo!"
f7007b1aaa3f Importer-packaging: just a warning would end up with a package that really looks like VERSION.
Tom Gottfried <tom@intevation.de>
parents: 8675
diff changeset
70 exit 1
8675
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
71 else
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
72 hg up "$VERSION"
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
73 fi
5520
d5e1e4d9559e compile manual from within importer packaging script
Tom Gottfried <tom.gottfried@intevation.de>
parents: 5491
diff changeset
74
6814
e41059fb4073 Importer Doc: Update title page automatically.
Tom Gottfried <tom.gottfried@intevation.de>
parents: 5855
diff changeset
75 # create PDF of manual
7334
f7d3fc619976 Importer manual: use unique revision identifier.
Tom Gottfried <tom@intevation.de>
parents: 7331
diff changeset
76 REV=`hg parent | sed -n '1s/[[:alnum:]]*: *\([0-9]*:.*\)/\1/;1p'`
8675
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
77 cd ${SOURCE_DIR}/backend/doc/documentation/de/
7334
f7d3fc619976 Importer manual: use unique revision identifier.
Tom Gottfried <tom@intevation.de>
parents: 7331
diff changeset
78 sed -i "s/documentrevision..rev.*/documentrevision}{rev$REV}/" \
6814
e41059fb4073 Importer Doc: Update title page automatically.
Tom Gottfried <tom.gottfried@intevation.de>
parents: 5855
diff changeset
79 importer-manual.tex
7331
bb9735fbb680 Importer packaging: run pdflatex three times to get references and page numbering right.
Tom Gottfried <tom@intevation.de>
parents: 6814
diff changeset
80 # run pdflatex three times to get references and page numbering right
bb9735fbb680 Importer packaging: run pdflatex three times to get references and page numbering right.
Tom Gottfried <tom@intevation.de>
parents: 6814
diff changeset
81 pdflatex importer-manual.tex
bb9735fbb680 Importer packaging: run pdflatex three times to get references and page numbering right.
Tom Gottfried <tom@intevation.de>
parents: 6814
diff changeset
82 pdflatex importer-manual.tex
6814
e41059fb4073 Importer Doc: Update title page automatically.
Tom Gottfried <tom.gottfried@intevation.de>
parents: 5855
diff changeset
83 pdflatex importer-manual.tex
5097
415ac9bc5a56 Add importer packaging script and some doc about it
Andre Heinecke <aheinecke@intevation.de>
parents:
diff changeset
84
8675
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
85 # package importer
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
86 if [ "$BUILD_ORACLE" = "TRUE" ]; then
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
87 if [ ! -f $ORACLE_JDBC ]; then
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
88 echo "Could not find oracle jdbc in $ORACLE_JDBC"
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
89 echo "Please make sure that the ORACLE_JDBC variable is set correctly"
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
90 exit 1
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
91 fi
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
92 mvn install:install-file -DgroupId=ojdbc -DartifactId=ojdbc \
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
93 -Dversion=0 -Dpackaging=jar -Dfile=$ORACLE_JDBC -DgeneratePom=true
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
94 mvn -f $SOURCE_DIR/backend/pom-oracle.xml clean package assembly:single
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
95 else
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
96 mvn -f $SOURCE_DIR/backend/pom.xml clean package assembly:single
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
97 fi
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
98
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
99 echo "INFO: create tarball"
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
100 rm -fr $PKG_DIR
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
101 mkdir $PKG_DIR
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
102 cd $PKG_DIR
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
103
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
104 mv $SOURCE_DIR/backend/target/river-backend-1.0-SNAPSHOT*-flys-importer.tar \
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
105 flys-importer-$VERSION.tar
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
106 if [ -f "$EXTRAS" ]; then
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
107 tar -xzf "$EXTRAS"
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
108 tar -rf flys-importer-$VERSION.tar opt
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
109 fi
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
110
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
111 gzip flys-importer-$VERSION.tar
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
112 sha1sum flys-importer-$VERSION.tar.gz > flys-importer-$VERSION.tar.gz.sha1
ae3df0f96e15 Mavenize importer packaging and allow packaging for specific versions and database systems.
Tom Gottfried <tom@intevation.de>
parents: 8670
diff changeset
113 echo Package is at: `readlink -f flys-importer-$VERSION.tar.gz`
http://dive4elements.wald.intevation.org