Mercurial > dive4elements > river
annotate flys-backend/src/main/java/de/intevation/flys/importer/Importer.java @ 5811:67e07c29dd29
ImportTimeInterval: Verbosity increased and fixed bug
in getPeer when stoptime is null.
author | Felix Wolfsteller <felix.wolfsteller@intevation.de> |
---|---|
date | Wed, 24 Apr 2013 14:49:00 +0200 |
parents | 2fceb5511dc0 |
children |
rev | line source |
---|---|
177
31895d24387e
Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
1 package de.intevation.flys.importer; |
31895d24387e
Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
2 |
765
763c4137d6e1
Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
497
diff
changeset
|
3 import de.intevation.artifacts.common.utils.XMLUtils; |
763c4137d6e1
Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
497
diff
changeset
|
4 |
5027
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
5 import de.intevation.flys.importer.parsers.AnnotationClassifier; |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
6 import de.intevation.flys.importer.parsers.BundesWasserStrassenParser; |
1211
f08fe480092c
Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
766
diff
changeset
|
7 import de.intevation.flys.importer.parsers.InfoGewParser; |
f08fe480092c
Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
766
diff
changeset
|
8 |
177
31895d24387e
Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
9 import java.io.File; |
31895d24387e
Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
10 import java.io.IOException; |
31895d24387e
Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
11 |
5027
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
12 import java.util.HashMap; |
180
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
13 import java.util.List; |
5027
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
14 import java.util.Map; |
180
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
15 |
188
003ac16812dd
Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
183
diff
changeset
|
16 import java.sql.SQLException; |
003ac16812dd
Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
183
diff
changeset
|
17 |
177
31895d24387e
Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
18 import org.apache.log4j.Logger; |
31895d24387e
Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
19 |
180
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
20 import org.hibernate.Transaction; |
188
003ac16812dd
Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
183
diff
changeset
|
21 import org.hibernate.HibernateException; |
180
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
22 |
765
763c4137d6e1
Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
497
diff
changeset
|
23 import org.w3c.dom.Document; |
763c4137d6e1
Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
497
diff
changeset
|
24 |
5027
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
25 import de.intevation.flys.utils.StringUtil; |
4707
2b0426b79a92
Documentation.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
3659
diff
changeset
|
26 |
5016
255898799ed9
Minimal whitespace and doc cosmetics.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
4708
diff
changeset
|
27 /** Data Importer. Further processing happens per-river. */ |
177
31895d24387e
Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
28 public class Importer |
31895d24387e
Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
29 { |
5016
255898799ed9
Minimal whitespace and doc cosmetics.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
4708
diff
changeset
|
30 /** Private logger. */ |
177
31895d24387e
Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
31 private static Logger log = Logger.getLogger(Importer.class); |
31895d24387e
Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
32 |
5027
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
33 private static String BWASTR_ID_CSV_FILE = "BWASTR_ID.csv"; |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
34 |
180
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
35 protected List<ImportRiver> rivers; |
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
36 |
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
37 public Importer() { |
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
38 } |
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
39 |
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
40 public Importer(List<ImportRiver> rivers) { |
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
41 this.rivers = rivers; |
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
42 } |
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
43 |
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
44 public List<ImportRiver> getRivers() { |
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
45 return rivers; |
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
46 } |
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
47 |
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
48 public void setRivers(List<ImportRiver> rivers) { |
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
49 this.rivers = rivers; |
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
50 } |
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
51 |
5027
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
52 /** Write rivers and their dependencies/dependants to db. */ |
180
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
53 public void writeRivers() { |
182
1c0afb01bc93
Importer: Fixed error in HQL statement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
180
diff
changeset
|
54 log.debug("write rivers started"); |
188
003ac16812dd
Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
183
diff
changeset
|
55 |
003ac16812dd
Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
183
diff
changeset
|
56 for (ImportRiver river: rivers) { |
003ac16812dd
Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
183
diff
changeset
|
57 log.debug("writing river '" + river.getName() + "'"); |
190
d40da430d2fe
Added stub code to write gauges
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
189
diff
changeset
|
58 river.storeDependencies(); |
497
67fd63e4ef66
Importer: centralized caching
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
199
diff
changeset
|
59 ImporterSession.getInstance().getDatabaseSession().flush(); |
180
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
60 } |
188
003ac16812dd
Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
183
diff
changeset
|
61 |
182
1c0afb01bc93
Importer: Fixed error in HQL statement.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
180
diff
changeset
|
62 log.debug("write rivers finished"); |
180
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
63 } |
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
64 |
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
65 public void writeToDatabase() { |
183
222f4db3430a
Importer: Used thread local pattern to make sharing of session easier.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
182
diff
changeset
|
66 |
180
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
67 Transaction tx = null; |
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
68 |
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
69 try { |
497
67fd63e4ef66
Importer: centralized caching
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
199
diff
changeset
|
70 tx = ImporterSession.getInstance() |
67fd63e4ef66
Importer: centralized caching
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
199
diff
changeset
|
71 .getDatabaseSession().beginTransaction(); |
180
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
72 |
188
003ac16812dd
Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
183
diff
changeset
|
73 try { |
003ac16812dd
Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
183
diff
changeset
|
74 writeRivers(); |
003ac16812dd
Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
183
diff
changeset
|
75 } |
003ac16812dd
Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
183
diff
changeset
|
76 catch (HibernateException he) { |
003ac16812dd
Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
183
diff
changeset
|
77 Throwable t = he.getCause(); |
003ac16812dd
Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
183
diff
changeset
|
78 while (t instanceof SQLException) { |
5238
2fceb5511dc0
Tiny whitespace cosmetic.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5106
diff
changeset
|
79 SQLException sqle = (SQLException) t; |
188
003ac16812dd
Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
183
diff
changeset
|
80 log.error("SQL exeception chain:", sqle); |
003ac16812dd
Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
183
diff
changeset
|
81 t = sqle.getNextException(); |
003ac16812dd
Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
183
diff
changeset
|
82 } |
003ac16812dd
Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
183
diff
changeset
|
83 throw he; |
003ac16812dd
Store annotations in backend.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
183
diff
changeset
|
84 } |
180
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
85 |
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
86 tx.commit(); |
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
87 } |
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
88 catch (RuntimeException re) { |
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
89 if (tx != null) { |
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
90 tx.rollback(); |
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
91 } |
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
92 throw re; |
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
93 } |
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
94 } |
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
95 |
765
763c4137d6e1
Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
497
diff
changeset
|
96 public static AnnotationClassifier getAnnotationClassifier() { |
1223
268f8da412e3
Importer: Added a central configuration to allow skipping of parsing/storing individual sub systems.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1211
diff
changeset
|
97 String annotationTypes = Config.INSTANCE.getAnnotationTypes(); |
765
763c4137d6e1
Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
497
diff
changeset
|
98 |
763c4137d6e1
Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
497
diff
changeset
|
99 if (annotationTypes == null) { |
766
aa9e3da95c31
Importer: Fix import of annotation type classification.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
765
diff
changeset
|
100 log.info("no annotation types file configured."); |
765
763c4137d6e1
Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
497
diff
changeset
|
101 return null; |
763c4137d6e1
Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
497
diff
changeset
|
102 } |
763c4137d6e1
Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
497
diff
changeset
|
103 |
763c4137d6e1
Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
497
diff
changeset
|
104 File file = new File(annotationTypes); |
763c4137d6e1
Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
497
diff
changeset
|
105 |
766
aa9e3da95c31
Importer: Fix import of annotation type classification.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
765
diff
changeset
|
106 log.info("use annotation types file '" + file + "'"); |
aa9e3da95c31
Importer: Fix import of annotation type classification.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
765
diff
changeset
|
107 |
765
763c4137d6e1
Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
497
diff
changeset
|
108 if (!(file.isFile() && file.canRead())) { |
763c4137d6e1
Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
497
diff
changeset
|
109 log.warn("annotation type file '" + file + "' is not readable."); |
763c4137d6e1
Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
497
diff
changeset
|
110 return null; |
763c4137d6e1
Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
497
diff
changeset
|
111 } |
763c4137d6e1
Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
497
diff
changeset
|
112 |
763c4137d6e1
Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
497
diff
changeset
|
113 Document rules = XMLUtils.parseDocument(file); |
763c4137d6e1
Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
497
diff
changeset
|
114 |
763c4137d6e1
Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
497
diff
changeset
|
115 if (rules == null) { |
763c4137d6e1
Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
497
diff
changeset
|
116 log.warn("cannot parse annotation types file."); |
763c4137d6e1
Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
497
diff
changeset
|
117 return null; |
763c4137d6e1
Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
497
diff
changeset
|
118 } |
763c4137d6e1
Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
497
diff
changeset
|
119 |
763c4137d6e1
Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
497
diff
changeset
|
120 return new AnnotationClassifier(rules); |
763c4137d6e1
Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
497
diff
changeset
|
121 } |
763c4137d6e1
Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
497
diff
changeset
|
122 |
4707
2b0426b79a92
Documentation.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
3659
diff
changeset
|
123 |
2b0426b79a92
Documentation.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
3659
diff
changeset
|
124 /** Starting point for importing river data. */ |
177
31895d24387e
Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
125 public static void main(String [] args) { |
31895d24387e
Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
126 |
765
763c4137d6e1
Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
497
diff
changeset
|
127 InfoGewParser infoGewParser = new InfoGewParser( |
763c4137d6e1
Added classification of annotation types. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
497
diff
changeset
|
128 getAnnotationClassifier()); |
177
31895d24387e
Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
129 |
2806
33f40b23edd8
Initial checkin for parsing MINFO bed heights.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2804
diff
changeset
|
130 log.info("Start parsing rivers..."); |
33f40b23edd8
Initial checkin for parsing MINFO bed heights.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
2804
diff
changeset
|
131 |
5027
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
132 File bwastrFile = null; |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
133 |
177
31895d24387e
Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
134 for (String gew: args) { |
31895d24387e
Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
135 log.info("parsing info gew file: " + gew); |
5027
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
136 File gewFile = new File(gew); |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
137 if (bwastrFile == null) { |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
138 bwastrFile = new File(gewFile.getParentFile(), BWASTR_ID_CSV_FILE); |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
139 } |
177
31895d24387e
Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
140 try { |
5027
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
141 infoGewParser.parse(gewFile); |
177
31895d24387e
Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
142 } |
31895d24387e
Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
143 catch (IOException ioe) { |
5106
d16398d65a59
Importer: Print a decent error when failing to parse a gew and exit.
Andre Heinecke <aheinecke@intevation.de>
parents:
5080
diff
changeset
|
144 log.error("error while parsing gew: " + gew, ioe); |
d16398d65a59
Importer: Print a decent error when failing to parse a gew and exit.
Andre Heinecke <aheinecke@intevation.de>
parents:
5080
diff
changeset
|
145 System.exit(1); |
177
31895d24387e
Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
146 } |
31895d24387e
Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
147 } |
180
3908bfb2aa43
Store imported rivers into database. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
177
diff
changeset
|
148 |
2804
68d1661a57cf
Added a new config option to specify a INFO.gew file by system property.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
1223
diff
changeset
|
149 String gew = Config.INSTANCE.getInfoGewFile(); |
68d1661a57cf
Added a new config option to specify a INFO.gew file by system property.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
1223
diff
changeset
|
150 if (gew != null && gew.length() > 0) { |
68d1661a57cf
Added a new config option to specify a INFO.gew file by system property.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
1223
diff
changeset
|
151 log.info("parsing info gew file: " + gew); |
5027
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
152 File gewFile = new File(gew); |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
153 if (bwastrFile == null) { |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
154 bwastrFile = new File(gewFile.getParentFile(), BWASTR_ID_CSV_FILE); |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
155 } |
2804
68d1661a57cf
Added a new config option to specify a INFO.gew file by system property.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
1223
diff
changeset
|
156 try { |
5027
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
157 infoGewParser.parse(gewFile); |
2804
68d1661a57cf
Added a new config option to specify a INFO.gew file by system property.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
1223
diff
changeset
|
158 } |
68d1661a57cf
Added a new config option to specify a INFO.gew file by system property.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
1223
diff
changeset
|
159 catch (IOException ioe) { |
5106
d16398d65a59
Importer: Print a decent error when failing to parse a gew and exit.
Andre Heinecke <aheinecke@intevation.de>
parents:
5080
diff
changeset
|
160 log.error("error while parsing gew: " + gew, ioe); |
d16398d65a59
Importer: Print a decent error when failing to parse a gew and exit.
Andre Heinecke <aheinecke@intevation.de>
parents:
5080
diff
changeset
|
161 System.exit(1); |
2804
68d1661a57cf
Added a new config option to specify a INFO.gew file by system property.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
1223
diff
changeset
|
162 } |
68d1661a57cf
Added a new config option to specify a INFO.gew file by system property.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
1223
diff
changeset
|
163 } |
68d1661a57cf
Added a new config option to specify a INFO.gew file by system property.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
1223
diff
changeset
|
164 |
5027
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
165 // Look for official numbers. |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
166 BundesWasserStrassenParser bwastrIdParser = |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
167 new BundesWasserStrassenParser(); |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
168 |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
169 // Read bwastFile (river-dir + BWASTR_ID_CSV_FILE). |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
170 if (!Config.INSTANCE.skipBWASTR()) { |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
171 try{ |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
172 bwastrIdParser.parse(bwastrFile); |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
173 HashMap<String,Long> map = bwastrIdParser.getMap(); |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
174 |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
175 // Now link rivers with official numbers. |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
176 for(ImportRiver river: infoGewParser.getRivers()) { |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
177 for(Map.Entry<String, Long> entry: map.entrySet()) { |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
178 if (StringUtil.containsIgnoreCase(entry.getKey(), river.getName())) { |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
179 river.setOfficialNumber(entry.getValue()); |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
180 log.debug(river.getName() + " is mapped to bwastr " + entry.getValue()); |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
181 } |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
182 } |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
183 } |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
184 } catch (IOException ioe) { |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
185 log.warn("BWASTR-file could not be loaded."); |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
186 } |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
187 } |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
188 else { |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
189 log.debug("skip reading BWASTR_ID.csv"); |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
190 } |
25c180fa6162
Use BWASTRParser to set official numbers on rivers.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
5016
diff
changeset
|
191 |
1223
268f8da412e3
Importer: Added a central configuration to allow skipping of parsing/storing individual sub systems.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
1211
diff
changeset
|
192 if (!Config.INSTANCE.dryRun()) { |
199
ed38839a6b08
Ported over some WST parsing stuff from desktop flys
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
190
diff
changeset
|
193 new Importer(infoGewParser.getRivers()).writeToDatabase(); |
ed38839a6b08
Ported over some WST parsing stuff from desktop flys
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
190
diff
changeset
|
194 } |
4708
45d05402b2a6
Importer: Emit log message when a dry run was run.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
4707
diff
changeset
|
195 else { |
45d05402b2a6
Importer: Emit log message when a dry run was run.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
4707
diff
changeset
|
196 log.info("Dry run, not writing to database."); |
45d05402b2a6
Importer: Emit log message when a dry run was run.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents:
4707
diff
changeset
|
197 } |
177
31895d24387e
Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
198 } |
31895d24387e
Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
199 } |
31895d24387e
Importer: Added info gew parser.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff
changeset
|
200 // vim:set ts=4 sw=4 si et sta sts=4 fenc=utf8 : |