Mercurial > dive4elements > river

annotate flys-backend/src/main/java/org/dive4elements/river/importer/Importer.java @ 5828:dfb26b03b179

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Moved directories to org.dive4elements.river
author Sascha L. Teichmann <teichmann@intevation.de>
date Thu, 25 Apr 2013 11:53:11 +0200
parents flys-backend/src/main/java/de/intevation/flys/importer/Importer.java@2fceb5511dc0
children 18619c1e7c2a
rev   line source
177
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
1 package de.intevation.flys.importer;
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
2
765
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
3 import de.intevation.artifacts.common.utils.XMLUtils;
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
4
5027
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
5 import de.intevation.flys.importer.parsers.AnnotationClassifier;
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
6 import de.intevation.flys.importer.parsers.BundesWasserStrassenParser;
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 766
diff changeset
7 import de.intevation.flys.importer.parsers.InfoGewParser;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 766
diff changeset
8
177
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
9 import java.io.File;
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
10 import java.io.IOException;
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
11
5027
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
12 import java.util.HashMap;
180
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
13 import java.util.List;
5027
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
14 import java.util.Map;
180
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
15
188
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
16 import java.sql.SQLException;
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
17
177
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
18 import org.apache.log4j.Logger;
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
19
180
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
20 import org.hibernate.Transaction;
188
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
21 import org.hibernate.HibernateException;
180
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
22
765
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
23 import org.w3c.dom.Document;
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
24
5027
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
25 import de.intevation.flys.utils.StringUtil;
4707
2b0426b79a92 Documentation.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 3659
diff changeset
26
5016
255898799ed9 Minimal whitespace and doc cosmetics.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 4708
diff changeset
27 /** Data Importer. Further processing happens per-river. */
177
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
28 public class Importer
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
29 {
5016
255898799ed9 Minimal whitespace and doc cosmetics.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 4708
diff changeset
30 /** Private logger. */
177
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
31 private static Logger log = Logger.getLogger(Importer.class);
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
32
5027
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
33 private static String BWASTR_ID_CSV_FILE = "BWASTR_ID.csv";
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
34
180
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
35 protected List<ImportRiver> rivers;
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
36
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
37 public Importer() {
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
38 }
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
39
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
40 public Importer(List<ImportRiver> rivers) {
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
41 this.rivers = rivers;
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
42 }
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
43
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
44 public List<ImportRiver> getRivers() {
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
45 return rivers;
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
46 }
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
47
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
48 public void setRivers(List<ImportRiver> rivers) {
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
49 this.rivers = rivers;
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
50 }
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
51
5027
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
52 /** Write rivers and their dependencies/dependants to db. */
180
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
53 public void writeRivers() {
182
1c0afb01bc93 Importer: Fixed error in HQL statement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 180
diff changeset
54 log.debug("write rivers started");
188
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
55
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
56 for (ImportRiver river: rivers) {
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
57 log.debug("writing river '" + river.getName() + "'");
190
d40da430d2fe Added stub code to write gauges
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 189
diff changeset
58 river.storeDependencies();
497
67fd63e4ef66 Importer: centralized caching
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 199
diff changeset
59 ImporterSession.getInstance().getDatabaseSession().flush();
180
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
60 }
188
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
61
182
1c0afb01bc93 Importer: Fixed error in HQL statement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 180
diff changeset
62 log.debug("write rivers finished");
180
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
63 }
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
64
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
65 public void writeToDatabase() {
183
222f4db3430a Importer: Used thread local pattern to make sharing of session easier.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 182
diff changeset
66
180
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
67 Transaction tx = null;
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
68
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
69 try {
497
67fd63e4ef66 Importer: centralized caching
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 199
diff changeset
70 tx = ImporterSession.getInstance()
67fd63e4ef66 Importer: centralized caching
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 199
diff changeset
71 .getDatabaseSession().beginTransaction();
180
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
72
188
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
73 try {
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
74 writeRivers();
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
75 }
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
76 catch (HibernateException he) {
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
77 Throwable t = he.getCause();
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
78 while (t instanceof SQLException) {
5238
2fceb5511dc0 Tiny whitespace cosmetic.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5106
diff changeset
79 SQLException sqle = (SQLException) t;
188
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
80 log.error("SQL exeception chain:", sqle);
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
81 t = sqle.getNextException();
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
82 }
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
83 throw he;
003ac16812dd Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 183
diff changeset
84 }
180
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
85
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
86 tx.commit();
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
87 }
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
88 catch (RuntimeException re) {
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
89 if (tx != null) {
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
90 tx.rollback();
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
91 }
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
92 throw re;
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
93 }
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
94 }
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
95
765
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
96 public static AnnotationClassifier getAnnotationClassifier() {
1223
268f8da412e3 Importer: Added a central configuration to allow skipping of parsing/storing individual sub systems.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1211
diff changeset
97 String annotationTypes = Config.INSTANCE.getAnnotationTypes();
765
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
98
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
99 if (annotationTypes == null) {
766
aa9e3da95c31 Importer: Fix import of annotation type classification.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 765
diff changeset
100 log.info("no annotation types file configured.");
765
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
101 return null;
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
102 }
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
103
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
104 File file = new File(annotationTypes);
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
105
766
aa9e3da95c31 Importer: Fix import of annotation type classification.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 765
diff changeset
106 log.info("use annotation types file '" + file + "'");
aa9e3da95c31 Importer: Fix import of annotation type classification.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 765
diff changeset
107
765
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
108 if (!(file.isFile() && file.canRead())) {
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
109 log.warn("annotation type file '" + file + "' is not readable.");
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
110 return null;
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
111 }
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
112
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
113 Document rules = XMLUtils.parseDocument(file);
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
114
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
115 if (rules == null) {
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
116 log.warn("cannot parse annotation types file.");
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
117 return null;
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
118 }
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
119
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
120 return new AnnotationClassifier(rules);
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
121 }
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
122
4707
2b0426b79a92 Documentation.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 3659
diff changeset
123
2b0426b79a92 Documentation.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 3659
diff changeset
124 /** Starting point for importing river data. */
177
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
125 public static void main(String [] args) {
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
126
765
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
127 InfoGewParser infoGewParser = new InfoGewParser(
763c4137d6e1 Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 497
diff changeset
128 getAnnotationClassifier());
177
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
129
2806
33f40b23edd8 Initial checkin for parsing MINFO bed heights.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2804
diff changeset
130 log.info("Start parsing rivers...");
33f40b23edd8 Initial checkin for parsing MINFO bed heights.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2804
diff changeset
131
5027
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
132 File bwastrFile = null;
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
133
177
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
134 for (String gew: args) {
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
135 log.info("parsing info gew file: " + gew);
5027
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
136 File gewFile = new File(gew);
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
137 if (bwastrFile == null) {
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
138 bwastrFile = new File(gewFile.getParentFile(), BWASTR_ID_CSV_FILE);
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
139 }
177
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
140 try {
5027
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
141 infoGewParser.parse(gewFile);
177
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
142 }
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
143 catch (IOException ioe) {
5106
d16398d65a59 Importer: Print a decent error when failing to parse a gew and exit.
Andre Heinecke <aheinecke@intevation.de>
parents: 5080
diff changeset
144 log.error("error while parsing gew: " + gew, ioe);
d16398d65a59 Importer: Print a decent error when failing to parse a gew and exit.
Andre Heinecke <aheinecke@intevation.de>
parents: 5080
diff changeset
145 System.exit(1);
177
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
146 }
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
147 }
180
3908bfb2aa43 Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 177
diff changeset
148
2804
68d1661a57cf Added a new config option to specify a INFO.gew file by system property.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 1223
diff changeset
149 String gew = Config.INSTANCE.getInfoGewFile();
68d1661a57cf Added a new config option to specify a INFO.gew file by system property.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 1223
diff changeset
150 if (gew != null && gew.length() > 0) {
68d1661a57cf Added a new config option to specify a INFO.gew file by system property.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 1223
diff changeset
151 log.info("parsing info gew file: " + gew);
5027
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
152 File gewFile = new File(gew);
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
153 if (bwastrFile == null) {
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
154 bwastrFile = new File(gewFile.getParentFile(), BWASTR_ID_CSV_FILE);
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
155 }
2804
68d1661a57cf Added a new config option to specify a INFO.gew file by system property.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 1223
diff changeset
156 try {
5027
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
157 infoGewParser.parse(gewFile);
2804
68d1661a57cf Added a new config option to specify a INFO.gew file by system property.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 1223
diff changeset
158 }
68d1661a57cf Added a new config option to specify a INFO.gew file by system property.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 1223
diff changeset
159 catch (IOException ioe) {
5106
d16398d65a59 Importer: Print a decent error when failing to parse a gew and exit.
Andre Heinecke <aheinecke@intevation.de>
parents: 5080
diff changeset
160 log.error("error while parsing gew: " + gew, ioe);
d16398d65a59 Importer: Print a decent error when failing to parse a gew and exit.
Andre Heinecke <aheinecke@intevation.de>
parents: 5080
diff changeset
161 System.exit(1);
2804
68d1661a57cf Added a new config option to specify a INFO.gew file by system property.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 1223
diff changeset
162 }
68d1661a57cf Added a new config option to specify a INFO.gew file by system property.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 1223
diff changeset
163 }
68d1661a57cf Added a new config option to specify a INFO.gew file by system property.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 1223
diff changeset
164
5027
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
165 // Look for official numbers.
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
166 BundesWasserStrassenParser bwastrIdParser =
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
167 new BundesWasserStrassenParser();
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
168
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
169 // Read bwastFile (river-dir + BWASTR_ID_CSV_FILE).
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
170 if (!Config.INSTANCE.skipBWASTR()) {
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
171 try{
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
172 bwastrIdParser.parse(bwastrFile);
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
173 HashMap<String,Long> map = bwastrIdParser.getMap();
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
174
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
175 // Now link rivers with official numbers.
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
176 for(ImportRiver river: infoGewParser.getRivers()) {
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
177 for(Map.Entry<String, Long> entry: map.entrySet()) {
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
178 if (StringUtil.containsIgnoreCase(entry.getKey(), river.getName())) {
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
179 river.setOfficialNumber(entry.getValue());
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
180 log.debug(river.getName() + " is mapped to bwastr " + entry.getValue());
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
181 }
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
182 }
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
183 }
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
184 } catch (IOException ioe) {
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
185 log.warn("BWASTR-file could not be loaded.");
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
186 }
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
187 }
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
188 else {
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
189 log.debug("skip reading BWASTR_ID.csv");
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
190 }
25c180fa6162 Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
191
1223
268f8da412e3 Importer: Added a central configuration to allow skipping of parsing/storing individual sub systems.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1211
diff changeset
192 if (!Config.INSTANCE.dryRun()) {
199
ed38839a6b08 Ported over some WST parsing stuff from desktop flys
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 190
diff changeset
193 new Importer(infoGewParser.getRivers()).writeToDatabase();
ed38839a6b08 Ported over some WST parsing stuff from desktop flys
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 190
diff changeset
194 }
4708
45d05402b2a6 Importer: Emit log message when a dry run was run.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 4707
diff changeset
195 else {
45d05402b2a6 Importer: Emit log message when a dry run was run.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 4707
diff changeset
196 log.info("Dry run, not writing to database.");
45d05402b2a6 Importer: Emit log message when a dry run was run.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 4707
diff changeset
197 }
177
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
198 }
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
199 }
31895d24387e Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
200 // vim:set ts=4 sw=4 si et sta sts=4 fenc=utf8 :
http://dive4elements.wald.intevation.org