Mercurial > dive4elements > river

annotate backend/src/main/java/org/dive4elements/river/importer/parsers/MeasurementStationsParser.java @ 9650:a2a42a6bac6b

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Importer (s/u-info) extensions: outer try/catch for parse and log of line no, catching parsing exception if not enough value fields, parsing error and warning log messages with line number, detecting and rejecting duplicate data series, better differentiation between error and warning log messages
author mschaefer
date Mon, 23 Mar 2020 14:57:03 +0100
parents c43d8c1a4455
children
rev   line source
5844
4dd33b86dc61 Added header to river backend.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
1 /* Copyright (C) 2011, 2012, 2013 by Bundesanstalt für Gewässerkunde
4dd33b86dc61 Added header to river backend.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
2 * Software engineering by Intevation GmbH
4dd33b86dc61 Added header to river backend.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
3 *
5992
4c3ccf2b0304 Removed trailing whitespace.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5844
diff changeset
4 * This file is Free Software under the GNU AGPL (>=v3)
5844
4dd33b86dc61 Added header to river backend.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
5 * and comes with ABSOLUTELY NO WARRANTY! Check out the
5992
4c3ccf2b0304 Removed trailing whitespace.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5844
diff changeset
6 * documentation coming with Dive4Elements River for details.
5844
4dd33b86dc61 Added header to river backend.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
7 */
4dd33b86dc61 Added header to river backend.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
8
5829
18619c1e7c2a Repaired internal references.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5828
diff changeset
9 package org.dive4elements.river.importer.parsers;
4193
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
10
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
11 import java.text.ParseException;
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
12 import java.util.ArrayList;
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
13 import java.util.Date;
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
14 import java.util.List;
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
15
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
16 import org.apache.log4j.Logger;
5829
18619c1e7c2a Repaired internal references.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5828
diff changeset
17 import org.dive4elements.river.importer.ImportMeasurementStation;
18619c1e7c2a Repaired internal references.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5828
diff changeset
18 import org.dive4elements.river.importer.ImportRange;
18619c1e7c2a Repaired internal references.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5828
diff changeset
19 import org.dive4elements.river.importer.ImportTimeInterval;
8989
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
20 import org.dive4elements.river.importer.common.AbstractParser;
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
21 import org.dive4elements.river.model.MeasurementStation;
4193
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
22
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
23
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
24 public class MeasurementStationsParser extends LineParser {
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
25
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
26 public static class MeasurementStationParserException extends Exception {
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
27
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
28 private static final long serialVersionUID = 1L;
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
29
8989
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
30 public MeasurementStationParserException(final String msg) {
4193
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
31 super(msg);
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
32 }
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
33 }
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
34
8412
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
35 public static final int MIN_COLUMNS = 9;
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
36
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
37 public static final int MAX_COMMENT_LENGTH = 512;
4193
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
38
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
39 private static final Logger log = Logger
8989
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
40 .getLogger(MeasurementStationsParser.class);
4193
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
41
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
42 private List<ImportMeasurementStation> measurementStations;
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
43 private ImportMeasurementStation current;
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
44
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
45 @Override
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
46 protected void reset() {
8989
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
47 this.measurementStations = new ArrayList<>();
4193
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
48 }
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
49
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
50 @Override
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
51 protected void finish() {
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
52 }
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
53
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
54 @Override
8989
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
55 protected void handleLine(final int lineNum, final String line) {
4193
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
56 if (line == null || line.startsWith(START_META_CHAR)) {
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
57 log.info("skip meta information at line " + lineNum);
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
58 return;
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
59 }
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
60
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
61 try {
8989
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
62 this.current = new ImportMeasurementStation();
8412
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
63 handleDataLine(lineNum, line);
8989
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
64 this.measurementStations.add(this.current);
4193
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
65 }
8989
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
66 catch (final MeasurementStationParserException e) {
4193
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
67 log.warn("Problem in line " + lineNum + ": " + e.getMessage());
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
68 }
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
69 }
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
70
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
71 public List<ImportMeasurementStation> getMeasurementStations() {
8989
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
72 return this.measurementStations;
4193
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
73 }
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
74
8989
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
75 protected void handleDataLine(final int lineNum, final String line)
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
76 throws MeasurementStationParserException {
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
77 final String[] cols = line.split(SEPERATOR_CHAR);
4193
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
78
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
79 if (cols == null || cols.length < MIN_COLUMNS) {
8989
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
80 final int num = cols != null ? cols.length : 0;
4193
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
81 throw new MeasurementStationParserException("Not enough columns: "
8989
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
82 + num);
4193
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
83 }
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
84
8989
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
85 this.current.name = getName(cols, lineNum);
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
86 this.current.range = getRange(cols, lineNum);
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
87 this.current.measurementType = getMeasurementType(cols, lineNum);
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
88 this.current.riverside = getRiverside(cols, lineNum);
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
89 this.current.gauge = getGauge(cols, lineNum);
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
90 this.current.observationTimerange = getObservationTimerange(cols, lineNum);
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
91 this.current.operator = getOperator(cols, lineNum);
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
92 this.current.comment = getComment(cols, lineNum);
4193
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
93 }
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
94
8989
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
95 protected String getName(final String[] cols, final int lineNum)
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
96 throws MeasurementStationParserException {
4193
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
97 if (cols[0] == null || cols[0].length() == 0) {
8412
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
98 throw new MeasurementStationParserException("invalid name in line "
8989
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
99 + lineNum);
4193
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
100 }
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
101
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
102 return cols[0];
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
103 }
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
104
8989
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
105 protected ImportRange getRange(final String[] cols, final int lineNum) {
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
106 final String from = cols[1];
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
107 final String to = cols[4];
8412
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
108 if (from == null || from.length() == 0) {
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
109 log.error("No station found in line" + lineNum);
5451
278d8759c92b Allow null values in measurement_station columns without 'not null' constraints.
Raimund Renkert <rrenkert@intevation.de>
parents: 4193
diff changeset
110 return null;
4193
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
111 }
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
112
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
113 try {
8412
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
114 if (to == null || to.length() == 0) {
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
115 log.warn("No end km found in line " + lineNum);
8989
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
116 return new ImportRange(AbstractParser.parseDecimal(from));
8412
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
117 }
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
118
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
119 try {
8989
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
120 return new ImportRange(AbstractParser.parseDecimal(from), AbstractParser.parseDecimal(to));
8412
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
121 }
8989
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
122 catch (final NumberFormatException e) {
8412
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
123 log.warn("Unparseable end km in line " + lineNum +
8989
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
124 ". Error: " + e.getMessage());
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
125 return new ImportRange(AbstractParser.parseDecimal(from));
8412
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
126 }
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
127
4193
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
128 }
8991
c43d8c1a4455 Parse via decimalformat instead of bigdecimal constructor (to cope with strings with chars after the actual number)
mschaefer
parents: 8989
diff changeset
129 catch (final ParseException e) {
8412
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
130 log.error("Unparseable station in line " + lineNum +
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
131 ". Error: " + e.getMessage());
5451
278d8759c92b Allow null values in measurement_station columns without 'not null' constraints.
Raimund Renkert <rrenkert@intevation.de>
parents: 4193
diff changeset
132 return null;
4193
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
133 }
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
134 }
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
135
8989
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
136 protected String getMeasurementType(final String[] cols, final int lineNum)
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
137 throws MeasurementStationParserException {
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
138 final String mtype = cols[2].trim();
8665
8fbc0649da13 Move static measurement types to their natural home.
Tom Gottfried <tom@intevation.de>
parents: 8642
diff changeset
139 if (!(MeasurementStation.MEASUREMENT_TYPE_BEDLOAD.equals(mtype) ||
8fbc0649da13 Move static measurement types to their natural home.
Tom Gottfried <tom@intevation.de>
parents: 8642
diff changeset
140 MeasurementStation.MEASUREMENT_TYPE_SUSP.equals(mtype))) {
4193
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
141 throw new MeasurementStationParserException(
8989
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
142 "invalid measurement type in line " + lineNum);
4193
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
143 }
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
144
8642
9db1f48bfea9 Enforce strings we rely on.
Tom Gottfried <tom@intevation.de>
parents: 8412
diff changeset
145 return mtype;
4193
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
146 }
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
147
8989
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
148 protected String getRiverside(final String[] cols, final int lineNum) {
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
149 final String col = cols[3];
8412
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
150 if (col == null || col.length() == 0) {
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
151 log.warn("No river side given in line " + lineNum);
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
152 }
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
153 return col;
4193
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
154 }
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
155
8989
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
156 protected String getGauge(final String[] cols, final int lineNum) {
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
157 final String col = cols[5];
8412
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
158 if (col == null || col.length() == 0) {
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
159 log.warn("Invalid gauge found in line " + lineNum);
4193
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
160 }
8412
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
161 return col;
4193
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
162 }
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
163
8412
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
164 protected ImportTimeInterval getObservationTimerange(
8989
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
165 final String[] cols,
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
166 final int lineNum
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
167 ) {
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
168 final String col = cols[7];
8412
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
169 if (col == null || col.length() == 0) {
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
170 log.warn("Observation time invalid in line " + lineNum);
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
171 return null;
4193
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
172 }
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
173
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
174 try {
8989
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
175 final Date date = getDate(col);
4193
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
176
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
177 if (date != null) {
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
178 return new ImportTimeInterval(date);
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
179 }
8412
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
180 log.warn("Observation time invalid in line " + lineNum);
4193
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
181 }
8989
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
182 catch (final ParseException pe) {
8412
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
183 log.warn("Unparseable observation time '" + col +
8989
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
184 "' in line " + lineNum);
4193
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
185 }
5451
278d8759c92b Allow null values in measurement_station columns without 'not null' constraints.
Raimund Renkert <rrenkert@intevation.de>
parents: 4193
diff changeset
186 return null;
4193
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
187 }
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
188
8989
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
189 protected String getOperator(final String[] cols, final int lineNum) {
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
190 final String col = cols[8];
8412
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
191 if (col == null || col.length() == 0) {
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
192 log.warn("No operator given in line " + lineNum);
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
193 }
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
194 return col;
4193
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
195 }
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
196
8989
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
197 protected String getComment(final String[] cols, final int lineNum) {
8412
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
198 if (cols.length > MIN_COLUMNS) {
8989
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
199 final String col = cols[9];
8412
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
200 if (col.length() > MAX_COMMENT_LENGTH) {
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
201 log.warn("Comment in line " + lineNum +
8989
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
202 " longer than allowed " + MAX_COMMENT_LENGTH +
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits
mschaefer
parents: 8665
diff changeset
203 " characters. Truncated.");
8412
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
204 return col.substring(0, MAX_COMMENT_LENGTH);
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
205 }
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
206 return col;
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
207 }
17db08570637 SCHEMA CHANGE: removed superfluous columns station and river_id from measurement_stations and adapted other components accordingly.
Tom Gottfried <tom@intevation.de>
parents: 5992
diff changeset
208 return null;
4193
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
209 }
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
210 }
http://dive4elements.wald.intevation.org