Mercurial > dive4elements > river
view contrib/make_flys_release/bin/make-importer-package.sh @ 5779:ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
XML DOM is not thread safe. Therefore the old implementation only allowed one thread
to use the builder at a time. As the complexity of the configuration
has increased over time this has become a bottleneck of the whole application
because it took quiet some time to build a result. Furthermore the builder code path
is visited very frequent. So many concurrent requests were piled up
resulting in long waits for the users.
To mitigate this problem a round robin pool of builders is used now.
Each of the pooled builders has an independent copy of the XML template
and can be run in parallel.
The number of builders is determined by the system property
'flys.datacage.pool.size'. It defaults to 4.
| author | Sascha L. Teichmann <teichmann@intevation.de> |
|---|---|
| date | Sun, 21 Apr 2013 12:48:09 +0200 |
| parents | d5e1e4d9559e |
| children | cff1eea9854f |
line wrap: on
line source
#!/bin/bash set -e # See README for more information # The working directory. Resulting tarball will be placed in the directory above. PKG_DIR=~/tmp/flys-importer # Path to the flys checkout FLYS_DIR=~/flys/flys/ # Tarball that will be extracted into flys-imprter/opt EXTRAS=$1 rm -fr $PKG_DIR mkdir -p $PKG_DIR/hydr_morph mkdir -p $PKG_DIR/geodaesie mkdir -p $PKG_DIR/opt/lib64 mkdir -p $PKG_DIR/schema mkdir -p $PKG_DIR/conf cat > "$PKG_DIR/conf/log4j.properties" << "EOF" log4j.rootLogger=DEBUG, IMPORTER log4j.appender.IMPORTER.layout=org.apache.log4j.PatternLayout log4j.appender.IMPORTER.layout.ConversionPattern=%d [%t] %-5p %c - %m%n log4j.appender.IMPORTER=org.apache.log4j.RollingFileAppender log4j.appender.IMPORTER.File=./import.log log4j.appender.IMPORTER.MaxFileSize=100000KB log4j.appender.IMPORTER.MaxBackupIndex=10 EOF cd ${FLYS_DIR}/flys-backend mvn -f pom-oracle.xml clean compile assembly:single cp target/flys-backend-1.0-SNAPSHOT-jar-with-dependencies.jar \ $PKG_DIR/hydr_morph/importer.jar mvn -f pom.xml clean compile assembly:single cp target/flys-backend-1.0-SNAPSHOT-jar-with-dependencies.jar \ $PKG_DIR/hydr_morph/importer_psql.jar cp ${FLYS_DIR}/flys-backend/contrib/shpimporter/*.py $PKG_DIR/geodaesie cp ${FLYS_DIR}/flys-backend/contrib/run_geo.sh \ ${FLYS_DIR}/flys-backend/contrib/run_hydr_morph.sh \ ${FLYS_DIR}/flys-backend/contrib/import_river.sh \ $PKG_DIR cp ${FLYS_DIR}/flys-backend/doc/annotation-types.xml $PKG_DIR/conf if [ -f "$EXTRAS" ]; then cd $PKG_DIR tar -xzf "$EXTRAS" fi cp ${FLYS_DIR}/flys-backend/doc/schema/*.sql $PKG_DIR/schema cd ${FLYS_DIR}/flys-backend/doc/documentation/de/ make importer-manual.pdf cp importer-manual.pdf $PKG_DIR sed -i 's/shpimporter\/shp/geodaesie\/shp/' $PKG_DIR/run_geo.sh cd $PKG_DIR/.. DATE=$(date +%Y%m%d%H%M) tar -czf flys-importer_${DATE}.tar.gz flys-importer sha1sum flys-importer_${DATE}.tar.gz > flys-importer_${DATE}.tar.gz.sha1 echo Package is at: `readlink -f flys-importer_${DATE}.tar.gz`