Mercurial > dive4elements > river

annotate backend/src/main/java/org/dive4elements/river/importer/ImportCrossSection.java @ 9650:a2a42a6bac6b

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Importer (s/u-info) extensions: outer try/catch for parse and log of line no, catching parsing exception if not enough value fields, parsing error and warning log messages with line number, detecting and rejecting duplicate data series, better differentiation between error and warning log messages
author mschaefer
date Mon, 23 Mar 2020 14:57:03 +0100
parents 392bbcd8a88b
children
rev   line source
5844
4dd33b86dc61 Added header to river backend.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
1 /* Copyright (C) 2011, 2012, 2013 by Bundesanstalt für Gewässerkunde
4dd33b86dc61 Added header to river backend.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
2 * Software engineering by Intevation GmbH
4dd33b86dc61 Added header to river backend.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
3 *
5992
4c3ccf2b0304 Removed trailing whitespace.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5844
diff changeset
4 * This file is Free Software under the GNU AGPL (>=v3)
5844
4dd33b86dc61 Added header to river backend.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
5 * and comes with ABSOLUTELY NO WARRANTY! Check out the
5992
4c3ccf2b0304 Removed trailing whitespace.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5844
diff changeset
6 * documentation coming with Dive4Elements River for details.
5844
4dd33b86dc61 Added header to river backend.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
7 */
4dd33b86dc61 Added header to river backend.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
8
5829
18619c1e7c2a Repaired internal references.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5828
diff changeset
9 package org.dive4elements.river.importer;
1204
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
10
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
11 import java.util.List;
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
12
1205
5f1506fc7636 Made import of cross sections work.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1204
diff changeset
13 import org.apache.log4j.Logger;
8986
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
14 import org.dive4elements.river.importer.common.StoreMode;
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
15 import org.dive4elements.river.model.CrossSection;
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
16 import org.dive4elements.river.model.River;
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
17 import org.dive4elements.river.model.TimeInterval;
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
18 import org.hibernate.Query;
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
19 import org.hibernate.Session;
1205
5f1506fc7636 Made import of cross sections work.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1204
diff changeset
20
4705
3199dbc20e22 Documentation added.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 1205
diff changeset
21 /** CrossSection to be imported, holds list of ImportCrossSectionLines. */
1204
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
22 public class ImportCrossSection
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
23 {
7540
9344aa0fb021 Fixed logger instantiations that were done with wrong class.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5992
diff changeset
24 private static Logger log = Logger.getLogger(ImportCrossSection.class);
1205
5f1506fc7636 Made import of cross sections work.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1204
diff changeset
25
1204
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
26 protected ImportRiver river;
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
27 protected String description;
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
28 protected ImportTimeInterval timeInterval;
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
29 protected List<ImportCrossSectionLine> lines;
8986
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
30 protected StoreMode storeMode;
1204
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
31
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
32 protected CrossSection peer;
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
33
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
34 public ImportCrossSection() {
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
35 }
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
36
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
37 public ImportCrossSection(
8986
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
38 final ImportRiver river,
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
39 final String description,
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
40 final ImportTimeInterval timeInterval,
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
41 final List<ImportCrossSectionLine> lines
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
42 ) {
1204
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
43 this.river = river;
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
44 this.description = description;
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
45 this.timeInterval = timeInterval;
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
46 this.lines = lines;
8986
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
47 this.storeMode = StoreMode.NONE;
1204
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
48 wireWithLines();
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
49 }
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
50
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
51 public void wireWithLines() {
8986
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
52 for (final ImportCrossSectionLine line: this.lines) {
1204
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
53 line.setCrossSection(this);
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
54 }
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
55 }
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
56
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
57 public ImportRiver getRiver() {
8986
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
58 return this.river;
1204
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
59 }
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
60
8986
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
61 public void setRiver(final ImportRiver river) {
1204
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
62 this.river = river;
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
63 }
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
64
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
65 public String getDescription() {
8986
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
66 return this.description;
1204
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
67 }
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
68
8986
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
69 public void setDescription(final String description) {
1204
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
70 this.description = description;
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
71 }
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
72
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
73 public ImportTimeInterval getTimeInterval() {
8986
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
74 return this.timeInterval;
1204
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
75 }
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
76
8986
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
77 public void setTimeInterval(final ImportTimeInterval timeInterval) {
1204
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
78 this.timeInterval = timeInterval;
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
79 }
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
80
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
81 public void storeDependencies() {
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
82
8986
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
83 log.info("store cross section '" + this.description + "'");
1205
5f1506fc7636 Made import of cross sections work.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1204
diff changeset
84
1204
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
85 getPeer();
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
86
8986
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
87 // int i = 1;
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
88 // final int N = this.lines.size();
1205
5f1506fc7636 Made import of cross sections work.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 1204
diff changeset
89
8986
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
90 for (final ImportCrossSectionLine line: this.lines) {
1204
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
91 line.storeDependencies();
8986
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
92 // log.info(" stored " + i + " lines. remaining: " + (N-i));
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
93 // ++i;
1204
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
94 }
8986
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
95 log.info(" lines stored: " + this.lines.size());
1204
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
96 }
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
97
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
98 public CrossSection getPeer() {
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
99
8986
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
100 if (this.peer == null) {
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
101 final River r = this.river.getPeer();
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
102 final TimeInterval t = (this.timeInterval != null) ? this.timeInterval.getPeer() : null;
1204
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
103
8986
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
104 final Session session =
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
105 ImporterSession.getInstance().getDatabaseSession();
1204
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
106
8986
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
107 final Query query = session.createQuery(
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
108 "from CrossSection where " +
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
109 "river=:r and " +
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
110 "timeInterval=:t and " +
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
111 "description=:d");
1204
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
112
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
113 query.setParameter("r", r);
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
114 query.setParameter("t", t);
8986
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
115 query.setParameter("d", this.description);
1204
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
116
8986
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
117 final List<CrossSection> crossSections = query.list();
1204
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
118 if (crossSections.isEmpty()) {
8986
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
119 this.peer = new CrossSection(r, t, this.description);
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
120 session.save(this.peer);
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
121 this.storeMode = StoreMode.INSERT;
1204
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
122 }
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
123 else {
8986
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
124 this.peer = crossSections.get(0);
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
125 this.storeMode = StoreMode.UPDATE;
1204
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
126 }
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
127 }
8986
392bbcd8a88b Database inserts accelerated by suppressing unnecessary database queries for new data series
mschaefer
parents: 7540
diff changeset
128 return this.peer;
1204
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
129 }
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
130 }
22858e7cca79 Integrated PRF parsing into importer. Needs testing!
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
131 // vim:set ts=4 sw=4 si et sta sts=4 fenc=utf8 :
http://dive4elements.wald.intevation.org