Mercurial > dive4elements > river

annotate backend/src/main/java/org/dive4elements/river/importer/Importer.java @ 8187:3bb1c62ad732

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Moved package org.dive4elements.river.utils to org.dive4elements.river.backend.utils.
author Sascha L. Teichmann <teichmann@intevation.de>
date Thu, 04 Sep 2014 15:03:25 +0200
parents 8ca5d0e3c438
children 9d2e69f971f5
rev   line source
5844
4dd33b86dc61 Added header to river backend.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
1 /* Copyright (C) 2011, 2012, 2013 by Bundesanstalt für Gewässerkunde
4dd33b86dc61 Added header to river backend.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
2 * Software engineering by Intevation GmbH
4dd33b86dc61 Added header to river backend.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
3 *
5992
4c3ccf2b0304 Removed trailing whitespace.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5844
diff changeset
4 * This file is Free Software under the GNU AGPL (>=v3)
5844
4dd33b86dc61 Added header to river backend.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
5 * and comes with ABSOLUTELY NO WARRANTY! Check out the
5992
4c3ccf2b0304 Removed trailing whitespace.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5844
diff changeset
6 * documentation coming with Dive4Elements River for details.
5844
4dd33b86dc61 Added header to river backend.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
7 */
4dd33b86dc61 Added header to river backend.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
8
5829
18619c1e7c2a Repaired internal references.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5828
diff changeset
9 package org.dive4elements.river.importer;
177
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
10
5829
18619c1e7c2a Repaired internal references.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5828
diff changeset
11 import org.dive4elements.artifacts.common.utils.XMLUtils;
765
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
12
5829
18619c1e7c2a Repaired internal references.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5828
diff changeset
13 import org.dive4elements.river.importer.parsers.AnnotationClassifier;
18619c1e7c2a Repaired internal references.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5828
diff changeset
14 import org.dive4elements.river.importer.parsers.BundesWasserStrassenParser;
18619c1e7c2a Repaired internal references.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5828
diff changeset
15 import org.dive4elements.river.importer.parsers.InfoGewParser;
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 766
diff changeset
16
177
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
17 import java.io.File;
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
18 import java.io.IOException;
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
19
5027
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
20 import java.util.HashMap;
180
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
21 import java.util.List;
5027
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
22 import java.util.Map;
180
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
23
188
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
24 import java.sql.SQLException;
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
25
177
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
26 import org.apache.log4j.Logger;
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
27
180
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
28 import org.hibernate.Transaction;
188
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
29 import org.hibernate.HibernateException;
180
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
30
765
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
31 import org.w3c.dom.Document;
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
32
8187
3bb1c62ad732 Moved package org.dive4elements.river.utils to org.dive4elements.river.backend.utils.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6890
diff changeset
33 import org.dive4elements.river.backend.utils.StringUtil;
4707
2b0426b79a92 Documentation.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 3659
diff changeset
34
5016
255898799ed9 Minimal whitespace and doc cosmetics.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 4708
diff changeset
35 /** Data Importer. Further processing happens per-river. */
177
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
36 public class Importer
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
37 {
5016
255898799ed9 Minimal whitespace and doc cosmetics.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 4708
diff changeset
38 /** Private logger. */
177
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
39 private static Logger log = Logger.getLogger(Importer.class);
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
40
5027
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
41 private static String BWASTR_ID_CSV_FILE = "BWASTR_ID.csv";
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
42
180
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
43 protected List<ImportRiver> rivers;
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
44
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
45 public Importer() {
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
46 }
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
47
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
48 public Importer(List<ImportRiver> rivers) {
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
49 this.rivers = rivers;
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
50 }
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
51
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
52 public List<ImportRiver> getRivers() {
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
53 return rivers;
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
54 }
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
55
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
56 public void setRivers(List<ImportRiver> rivers) {
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
57 this.rivers = rivers;
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
58 }
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
59
5027
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
60 /** Write rivers and their dependencies/dependants to db. */
180
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
61 public void writeRivers() {
182
1c0afb01bc93 Importer: Fixed error in HQL statement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 180
diff changeset
62 log.debug("write rivers started");
188
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
63
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
64 for (ImportRiver river: rivers) {
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
65 log.debug("writing river '" + river.getName() + "'");
190
d40da430d2fe Added stub code to write gauges
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 189
diff changeset
66 river.storeDependencies();
497
67fd63e4ef66 Importer: centralized caching
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 199
diff changeset
67 ImporterSession.getInstance().getDatabaseSession().flush();
180
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
68 }
188
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
69
182
1c0afb01bc93 Importer: Fixed error in HQL statement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 180
diff changeset
70 log.debug("write rivers finished");
180
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
71 }
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
72
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
73 public void writeToDatabase() {
183
222f4db3430a Importer: Used thread local pattern to make sharing of session easier.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 182
diff changeset
74
180
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
75 Transaction tx = null;
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
76
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
77 try {
497
67fd63e4ef66 Importer: centralized caching
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 199
diff changeset
78 tx = ImporterSession.getInstance()
67fd63e4ef66 Importer: centralized caching
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 199
diff changeset
79 .getDatabaseSession().beginTransaction();
180
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
80
188
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
81 try {
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
82 writeRivers();
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
83 }
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
84 catch (HibernateException he) {
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
85 Throwable t = he.getCause();
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
86 while (t instanceof SQLException) {
5238
2fceb5511dc0 Tiny whitespace cosmetic.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5106
diff changeset
87 SQLException sqle = (SQLException) t;
188
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
88 log.error("SQL exeception chain:", sqle);
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
89 t = sqle.getNextException();
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
90 }
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
91 throw he;
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
92 }
180
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
93
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
94 tx.commit();
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
95 }
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
96 catch (RuntimeException re) {
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
97 if (tx != null) {
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
98 tx.rollback();
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
99 }
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
100 throw re;
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
101 }
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
102 }
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
103
765
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
104 public static AnnotationClassifier getAnnotationClassifier() {
1223
268f8da412e3 Importer: Added a central configuration to allow skipping of parsing/storing individual sub systems.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1211
diff changeset
105 String annotationTypes = Config.INSTANCE.getAnnotationTypes();
765
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
106
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
107 if (annotationTypes == null) {
766
aa9e3da95c31 Importer: Fix import of annotation type classification.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 765
diff changeset
108 log.info("no annotation types file configured.");
765
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
109 return null;
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
110 }
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
111
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
112 File file = new File(annotationTypes);
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
113
766
aa9e3da95c31 Importer: Fix import of annotation type classification.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 765
diff changeset
114 log.info("use annotation types file '" + file + "'");
aa9e3da95c31 Importer: Fix import of annotation type classification.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 765
diff changeset
115
765
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
116 if (!(file.isFile() && file.canRead())) {
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
117 log.warn("annotation type file '" + file + "' is not readable.");
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
118 return null;
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
119 }
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
120
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
121 Document rules = XMLUtils.parseDocument(file);
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
122
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
123 if (rules == null) {
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
124 log.warn("cannot parse annotation types file.");
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
125 return null;
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
126 }
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
127
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
128 return new AnnotationClassifier(rules);
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
129 }
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
130
4707
2b0426b79a92 Documentation.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 3659
diff changeset
131
2b0426b79a92 Documentation.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 3659
diff changeset
132 /** Starting point for importing river data. */
177
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
133 public static void main(String [] args) {
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
134
765
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
135 InfoGewParser infoGewParser = new InfoGewParser(
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
136 getAnnotationClassifier());
177
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
137
2806
33f40b23edd8 Initial checkin for parsing MINFO bed heights.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2804
diff changeset
138 log.info("Start parsing rivers...");
33f40b23edd8 Initial checkin for parsing MINFO bed heights.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2804
diff changeset
139
5027
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
140 File bwastrFile = null;
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
141
177
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
142 for (String gew: args) {
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
143 log.info("parsing info gew file: " + gew);
5027
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
144 File gewFile = new File(gew);
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
145 if (bwastrFile == null) {
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
146 bwastrFile = new File(gewFile.getParentFile(), BWASTR_ID_CSV_FILE);
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
147 }
177
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
148 try {
5027
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
149 infoGewParser.parse(gewFile);
177
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
150 }
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
151 catch (IOException ioe) {
5106
d16398d65a59 Importer: Print a decent error when failing to parse a gew and exit.
Andre Heinecke <aheinecke@intevation.de>
parents: 5080
diff changeset
152 log.error("error while parsing gew: " + gew, ioe);
d16398d65a59 Importer: Print a decent error when failing to parse a gew and exit.
Andre Heinecke <aheinecke@intevation.de>
parents: 5080
diff changeset
153 System.exit(1);
177
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
154 }
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
155 }
180
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
156
2804
68d1661a57cf Added a new config option to specify a INFO.gew file by system property.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 1223
diff changeset
157 String gew = Config.INSTANCE.getInfoGewFile();
68d1661a57cf Added a new config option to specify a INFO.gew file by system property.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 1223
diff changeset
158 if (gew != null && gew.length() > 0) {
68d1661a57cf Added a new config option to specify a INFO.gew file by system property.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 1223
diff changeset
159 log.info("parsing info gew file: " + gew);
5027
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
160 File gewFile = new File(gew);
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
161 if (bwastrFile == null) {
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
162 bwastrFile = new File(gewFile.getParentFile(), BWASTR_ID_CSV_FILE);
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
163 }
2804
68d1661a57cf Added a new config option to specify a INFO.gew file by system property.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 1223
diff changeset
164 try {
5027
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
165 infoGewParser.parse(gewFile);
2804
68d1661a57cf Added a new config option to specify a INFO.gew file by system property.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 1223
diff changeset
166 }
68d1661a57cf Added a new config option to specify a INFO.gew file by system property.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 1223
diff changeset
167 catch (IOException ioe) {
5106
d16398d65a59 Importer: Print a decent error when failing to parse a gew and exit.
Andre Heinecke <aheinecke@intevation.de>
parents: 5080
diff changeset
168 log.error("error while parsing gew: " + gew, ioe);
d16398d65a59 Importer: Print a decent error when failing to parse a gew and exit.
Andre Heinecke <aheinecke@intevation.de>
parents: 5080
diff changeset
169 System.exit(1);
2804
68d1661a57cf Added a new config option to specify a INFO.gew file by system property.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 1223
diff changeset
170 }
68d1661a57cf Added a new config option to specify a INFO.gew file by system property.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 1223
diff changeset
171 }
68d1661a57cf Added a new config option to specify a INFO.gew file by system property.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 1223
diff changeset
172
5027
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
173 // Look for official numbers.
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
174 BundesWasserStrassenParser bwastrIdParser =
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
175 new BundesWasserStrassenParser();
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
176
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
177 // Read bwastFile (river-dir + BWASTR_ID_CSV_FILE).
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
178 if (!Config.INSTANCE.skipBWASTR()) {
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
179 try{
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
180 bwastrIdParser.parse(bwastrFile);
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
181 HashMap<String,Long> map = bwastrIdParser.getMap();
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
182
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
183 // Now link rivers with official numbers.
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
184 for(ImportRiver river: infoGewParser.getRivers()) {
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
185 for(Map.Entry<String, Long> entry: map.entrySet()) {
6890
8ca5d0e3c438 BWASTR-Import: name of tributary contains name of river, not the other way round.
Tom Gottfried <tom.gottfried@intevation.de>
parents: 5992
diff changeset
186 if (StringUtil.containsIgnoreCase(river.getName(), entry.getKey())) {
5027
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
187 river.setOfficialNumber(entry.getValue());
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
188 log.debug(river.getName() + " is mapped to bwastr " + entry.getValue());
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
189 }
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
190 }
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
191 }
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
192 } catch (IOException ioe) {
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
193 log.warn("BWASTR-file could not be loaded.");
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
194 }
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
195 }
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
196 else {
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
197 log.debug("skip reading BWASTR_ID.csv");
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
198 }
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
199
1223
268f8da412e3 Importer: Added a central configuration to allow skipping of parsing/storing individual sub systems.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1211
diff changeset
200 if (!Config.INSTANCE.dryRun()) {
199
ed38839a6b08 Ported over some WST parsing stuff from desktop flys
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 190
diff changeset
201 new Importer(infoGewParser.getRivers()).writeToDatabase();
ed38839a6b08 Ported over some WST parsing stuff from desktop flys
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 190
diff changeset
202 }
4708
45d05402b2a6 Importer: Emit log message when a dry run was run.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 4707
diff changeset
203 else {
45d05402b2a6 Importer: Emit log message when a dry run was run.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 4707
diff changeset
204 log.info("Dry run, not writing to database.");
45d05402b2a6 Importer: Emit log message when a dry run was run.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 4707
diff changeset
205 }
177
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
206 }
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
207 }
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
208 // vim:set ts=4 sw=4 si et sta sts=4 fenc=utf8 :
http://dive4elements.wald.intevation.org