Mercurial > dive4elements > river

annotate backend/src/main/java/org/dive4elements/river/importer/parsers/WstParser.java @ 8187:3bb1c62ad732

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Moved package org.dive4elements.river.utils to org.dive4elements.river.backend.utils.
author Sascha L. Teichmann <teichmann@intevation.de>
date Thu, 04 Sep 2014 15:03:25 +0200
parents fc8d5b5f2c99
children 5e38e2924c07
rev   line source
5844
4dd33b86dc61 Added header to river backend.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
1 /* Copyright (C) 2011, 2012, 2013 by Bundesanstalt für Gewässerkunde
4dd33b86dc61 Added header to river backend.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
2 * Software engineering by Intevation GmbH
4dd33b86dc61 Added header to river backend.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
3 *
5992
4c3ccf2b0304 Removed trailing whitespace.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5844
diff changeset
4 * This file is Free Software under the GNU AGPL (>=v3)
5844
4dd33b86dc61 Added header to river backend.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
5 * and comes with ABSOLUTELY NO WARRANTY! Check out the
5992
4c3ccf2b0304 Removed trailing whitespace.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5844
diff changeset
6 * documentation coming with Dive4Elements River for details.
5844
4dd33b86dc61 Added header to river backend.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
7 */
4dd33b86dc61 Added header to river backend.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
8
5829
18619c1e7c2a Repaired internal references.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5828
diff changeset
9 package org.dive4elements.river.importer.parsers;
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
10
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
11 import java.util.ArrayList;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
12 import java.util.HashSet;
6506
1c68d4b2a1c0 Fix import of source from wst-files
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6504
diff changeset
13 import java.util.Arrays;
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
14
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
15 import java.io.File;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
16 import java.io.IOException;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
17 import java.io.LineNumberReader;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
18 import java.io.InputStreamReader;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
19 import java.io.FileInputStream;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
20
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
21 import java.text.NumberFormat;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
22
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
23 import org.apache.log4j.Logger;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
24
8187
3bb1c62ad732 Moved package org.dive4elements.river.utils to org.dive4elements.river.backend.utils.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7883
diff changeset
25 import org.dive4elements.river.backend.utils.StringUtil;
3bb1c62ad732 Moved package org.dive4elements.river.utils to org.dive4elements.river.backend.utils.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7883
diff changeset
26 import org.dive4elements.river.backend.utils.DateGuesser;
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
27
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
28 import java.util.regex.Pattern;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
29 import java.util.regex.Matcher;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
30
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
31 import java.math.BigDecimal;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
32
5829
18619c1e7c2a Repaired internal references.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5828
diff changeset
33 import org.dive4elements.river.importer.ImportWstQRange;
18619c1e7c2a Repaired internal references.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5828
diff changeset
34 import org.dive4elements.river.importer.ImportWstColumn;
18619c1e7c2a Repaired internal references.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5828
diff changeset
35 import org.dive4elements.river.importer.ImportTimeInterval;
18619c1e7c2a Repaired internal references.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5828
diff changeset
36 import org.dive4elements.river.importer.ImportRange;
18619c1e7c2a Repaired internal references.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5828
diff changeset
37 import org.dive4elements.river.importer.ImportUnit;
18619c1e7c2a Repaired internal references.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5828
diff changeset
38 import org.dive4elements.river.importer.ImportWst;
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
39
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
40 public class WstParser
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
41 {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
42 private static Logger log = Logger.getLogger(WstParser.class);
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
43
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
44 public static final String COLUMN_BEZ_TEXT = "column-bez-text";
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
45 public static final String COLUMN_BEZ_BREITE = "column-bez-breite";
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
46 public static final String COLUMN_QUELLE = "column-quelle";
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
47 public static final String COLUMN_DATUM = "column-datum";
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
48
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
49 public static final BigDecimal UNDEFINED_ZERO =
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
50 new BigDecimal(0.0);
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
51 public static final BigDecimal MIN_RANGE =
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
52 new BigDecimal(-Double.MAX_VALUE);
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
53 public static final BigDecimal MAX_RANGE =
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
54 new BigDecimal(Double.MAX_VALUE);
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
55
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
56 public static final String ENCODING = "ISO-8859-1";
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
57
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
58 public static final Pattern UNIT_COMMENT =
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
59 Pattern.compile("\\*\\s*[kK][mM]\\s+(.+)");
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
60
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
61 public static final Pattern UNIT =
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
62 Pattern.compile("[^\\[]*\\[([^]]+)\\].*");
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
63
6508
b2eab164a441 Backend: WST parser. Detect ranges of years in column-datum
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6506
diff changeset
64 public static final Pattern YEAR_INTERVAL =
b2eab164a441 Backend: WST parser. Detect ranges of years in column-datum
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6506
diff changeset
65 Pattern.compile("(\\d{4})\\s*[-/]\\s*(\\d{4})");
b2eab164a441 Backend: WST parser. Detect ranges of years in column-datum
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6506
diff changeset
66
5541
b09c095a0b7e WST Importer: Potential fix for tighten gaps between Q ranges.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 3662
diff changeset
67 public static final double INTERVAL_GAP = 0.00001d;
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
68
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
69 protected ImportWst wst;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
70
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
71 protected ImportRange lastRange;
7222
c5214e04b9b8 WST-parser: making ranges tightly fit was broken since a<b was enforced. Re-enabled.
Tom Gottfried <tom@intevation.de>
parents: 7003
diff changeset
72 protected Double lastA;
c5214e04b9b8 WST-parser: making ranges tightly fit was broken since a<b was enforced. Re-enabled.
Tom Gottfried <tom@intevation.de>
parents: 7003
diff changeset
73 protected Double lastB;
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
74
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
75 public WstParser() {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
76 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
77
6337
224d7c4b5291 Backend: Create ImportWsts for official lines expicitly. They are going to be somewhat special. Removed some cruft from the WstParser.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6328
diff changeset
78 public WstParser(ImportWst wst) {
224d7c4b5291 Backend: Create ImportWsts for official lines expicitly. They are going to be somewhat special. Removed some cruft from the WstParser.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6328
diff changeset
79 this.wst = wst;
224d7c4b5291 Backend: Create ImportWsts for official lines expicitly. They are going to be somewhat special. Removed some cruft from the WstParser.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6328
diff changeset
80 }
224d7c4b5291 Backend: Create ImportWsts for official lines expicitly. They are going to be somewhat special. Removed some cruft from the WstParser.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6328
diff changeset
81
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
82 public ImportWst getWst() {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
83 return wst;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
84 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
85
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
86 public void setWst(ImportWst wst) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
87 this.wst = wst;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
88 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
89
7339
40e5ad76103c WST Import: Added parse error exception to WST parser.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7252
diff changeset
90 public static final class ParseException extends Exception {
40e5ad76103c WST Import: Added parse error exception to WST parser.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7252
diff changeset
91 public ParseException() {
40e5ad76103c WST Import: Added parse error exception to WST parser.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7252
diff changeset
92 }
40e5ad76103c WST Import: Added parse error exception to WST parser.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7252
diff changeset
93
40e5ad76103c WST Import: Added parse error exception to WST parser.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7252
diff changeset
94 public ParseException(String msg) {
40e5ad76103c WST Import: Added parse error exception to WST parser.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7252
diff changeset
95 super(msg);
40e5ad76103c WST Import: Added parse error exception to WST parser.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7252
diff changeset
96 }
40e5ad76103c WST Import: Added parse error exception to WST parser.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7252
diff changeset
97 } // class ParseException
40e5ad76103c WST Import: Added parse error exception to WST parser.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7252
diff changeset
98
6727
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 6508
diff changeset
99 /** Returns a new ImportTimeInterval with a date guessed from string. */
6328
53d08f33d094 Backend: Moved guessing of main values and there time intervals out of the STA parser. Same come will be useful to extend the WST parser to better handle official lines.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6029
diff changeset
100 public static ImportTimeInterval guessDate(String string) {
2357
2e18e12fac11 Added date guesser for WST columns.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2346
diff changeset
101 try {
6508
b2eab164a441 Backend: WST parser. Detect ranges of years in column-datum
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6506
diff changeset
102 Matcher m = YEAR_INTERVAL.matcher(string);
b2eab164a441 Backend: WST parser. Detect ranges of years in column-datum
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6506
diff changeset
103 if (m.matches()) {
b2eab164a441 Backend: WST parser. Detect ranges of years in column-datum
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6506
diff changeset
104 return new ImportTimeInterval(
b2eab164a441 Backend: WST parser. Detect ranges of years in column-datum
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6506
diff changeset
105 DateGuesser.guessDate(m.group(1)),
b2eab164a441 Backend: WST parser. Detect ranges of years in column-datum
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6506
diff changeset
106 DateGuesser.guessDate(m.group(2)));
b2eab164a441 Backend: WST parser. Detect ranges of years in column-datum
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6506
diff changeset
107 }
b2eab164a441 Backend: WST parser. Detect ranges of years in column-datum
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6506
diff changeset
108
2357
2e18e12fac11 Added date guesser for WST columns.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2346
diff changeset
109 return new ImportTimeInterval(
2e18e12fac11 Added date guesser for WST columns.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2346
diff changeset
110 DateGuesser.guessDate(string));
2e18e12fac11 Added date guesser for WST columns.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2346
diff changeset
111 }
2e18e12fac11 Added date guesser for WST columns.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2346
diff changeset
112 catch (IllegalArgumentException iae) {
6028
25d927cb4b25 WstParser: warn if timestamp can not be set, cosmetics.
Tom Gottfried <tom.gottfried@intevation.de>
parents: 5992
diff changeset
113 log.warn("WST: String '" + string +
25d927cb4b25 WstParser: warn if timestamp can not be set, cosmetics.
Tom Gottfried <tom.gottfried@intevation.de>
parents: 5992
diff changeset
114 "' could not be interpreted as valid timestamp");
2357
2e18e12fac11 Added date guesser for WST columns.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2346
diff changeset
115 }
2e18e12fac11 Added date guesser for WST columns.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2346
diff changeset
116 return null;
2e18e12fac11 Added date guesser for WST columns.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2346
diff changeset
117 }
2e18e12fac11 Added date guesser for WST columns.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2346
diff changeset
118
7339
40e5ad76103c WST Import: Added parse error exception to WST parser.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7252
diff changeset
119 public void parse(File file) throws IOException, ParseException {
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
120
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
121 log.info("Parsing WST file '" + file + "'");
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
122
7375
450582bcad75 Replaced tabs with 4 spaces.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7374
diff changeset
123 if (wst == null) {
450582bcad75 Replaced tabs with 4 spaces.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7374
diff changeset
124 wst = new ImportWst(file.getName());
450582bcad75 Replaced tabs with 4 spaces.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7374
diff changeset
125 }
450582bcad75 Replaced tabs with 4 spaces.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7374
diff changeset
126 else {
450582bcad75 Replaced tabs with 4 spaces.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7374
diff changeset
127 wst.setDescription(file.getName());
450582bcad75 Replaced tabs with 4 spaces.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7374
diff changeset
128 }
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
129
6337
224d7c4b5291 Backend: Create ImportWsts for official lines expicitly. They are going to be somewhat special. Removed some cruft from the WstParser.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6328
diff changeset
130 LineNumberReader in =
224d7c4b5291 Backend: Create ImportWsts for official lines expicitly. They are going to be somewhat special. Removed some cruft from the WstParser.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6328
diff changeset
131 new LineNumberReader(
224d7c4b5291 Backend: Create ImportWsts for official lines expicitly. They are going to be somewhat special. Removed some cruft from the WstParser.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6328
diff changeset
132 new InputStreamReader(
224d7c4b5291 Backend: Create ImportWsts for official lines expicitly. They are going to be somewhat special. Removed some cruft from the WstParser.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6328
diff changeset
133 new FileInputStream(file), ENCODING));
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
134 try {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
135 String input;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
136 boolean first = true;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
137 int columnCount = 0;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
138
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
139 String [] lsBezeichner = null;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
140 String [] langBezeichner = null;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
141 int [] colNaWidths = null;
6504
7664ab97b4c7 Backend: Store source of a wst column if available.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6337
diff changeset
142 String [] quellen = null;
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
143 String [] daten = null;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
144
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
145 BigDecimal [] aktAbfluesse = null;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
146 BigDecimal [] firstAbfluesse = null;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
147
5544
12f4d088f088 WST importer: Cosmetics.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5543
diff changeset
148 BigDecimal minKm = MAX_RANGE;
12f4d088f088 WST importer: Cosmetics.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5543
diff changeset
149 BigDecimal maxKm = MIN_RANGE;
12f4d088f088 WST importer: Cosmetics.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5543
diff changeset
150 BigDecimal kmHist1 = null;
12f4d088f088 WST importer: Cosmetics.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5543
diff changeset
151 BigDecimal kmHist2 = null;
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
152
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
153 boolean columnHeaderChecked = false;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
154
7883
fc8d5b5f2c99 Removed trailing whitespace.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7488
diff changeset
155 /* Default string for altitude reference
6819
1d60614c1059 WstParser: warn only once if no unit comment is found
Tom Gottfried <tom.gottfried@intevation.de>
parents: 6809
diff changeset
156 * if none is found in WST-file.
1d60614c1059 WstParser: warn only once if no unit comment is found
Tom Gottfried <tom.gottfried@intevation.de>
parents: 6809
diff changeset
157 * Use in case no unit comment is found in file */
6809
4ff177f48952 WstParser and Importer doc: More sensible default unit and warn if it is used.
Tom Gottfried <tom.gottfried@intevation.de>
parents: 6730
diff changeset
158 String einheit = "m ü. unbekannte Referenz";
6819
1d60614c1059 WstParser: warn only once if no unit comment is found
Tom Gottfried <tom.gottfried@intevation.de>
parents: 6809
diff changeset
159 boolean unitFound = false;
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
160
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
161 HashSet<BigDecimal> kms = new HashSet<BigDecimal>();
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
162
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
163 while ((input = in.readLine()) != null) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
164 String line = input;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
165 if (first) { // fetch number of columns
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
166 if ((line = line.trim()).length() == 0) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
167 continue;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
168 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
169 try {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
170 columnCount = Integer.parseInt(line);
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
171 if (columnCount <= 0) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
172 throw new NumberFormatException(
6028
25d927cb4b25 WstParser: warn if timestamp can not be set, cosmetics.
Tom Gottfried <tom.gottfried@intevation.de>
parents: 5992
diff changeset
173 "number of columns <= 0");
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
174 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
175 log.debug("Number of columns: " + columnCount);
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
176 wst.setNumberColumns(columnCount);
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
177 lsBezeichner = new String[columnCount];
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
178 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
179 catch (NumberFormatException nfe) {
3662
0d27d02b1208 backend: Completed the error messages of the importer.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2357
diff changeset
180 log.warn("WST: invalid number.", nfe);
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
181 continue;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
182 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
183 first = false;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
184 continue;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
185 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
186
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
187 line = line.replace(',', '.');
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
188
7374
82191d111130 WstParser: Do not add Q-range for trailing Q-line.
Tom Gottfried <tom@intevation.de>
parents: 7342
diff changeset
189 // handle Q-lines
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
190 if (line.startsWith("*\u001f")) {
7488
1890bd524d27 WST-Parser: revert the assumption of only complete columns.
Tom Gottfried <tom@intevation.de>
parents: 7487
diff changeset
191 BigDecimal [] data = parseLineAsDouble(line, columnCount, false, true);
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
192
7374
82191d111130 WstParser: Do not add Q-range for trailing Q-line.
Tom Gottfried <tom@intevation.de>
parents: 7342
diff changeset
193 if (aktAbfluesse != null) { // add Q-ranges obtained from previous lines
5544
12f4d088f088 WST importer: Cosmetics.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5543
diff changeset
194 if (kmHist1 != null && kmHist2 != null
7341
588b02a4078a Comments.
Tom Gottfried <tom@intevation.de>
parents: 7340
diff changeset
195 && kmHist1.compareTo(kmHist2) < 0) { // stations descending in file
5544
12f4d088f088 WST importer: Cosmetics.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5543
diff changeset
196 BigDecimal t = minKm; minKm = maxKm; maxKm = t;
12f4d088f088 WST importer: Cosmetics.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5543
diff changeset
197 }
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
198 addInterval(minKm, maxKm, aktAbfluesse);
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
199 minKm = MAX_RANGE;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
200 maxKm = MIN_RANGE;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
201 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
202
7374
82191d111130 WstParser: Do not add Q-range for trailing Q-line.
Tom Gottfried <tom@intevation.de>
parents: 7342
diff changeset
203 // obtain Q-values from current line
7488
1890bd524d27 WST-Parser: revert the assumption of only complete columns.
Tom Gottfried <tom@intevation.de>
parents: 7487
diff changeset
204 aktAbfluesse = new BigDecimal[data.length];
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
205 log.debug("new q range: " + columnCount);
7488
1890bd524d27 WST-Parser: revert the assumption of only complete columns.
Tom Gottfried <tom@intevation.de>
parents: 7487
diff changeset
206 for (int i = 0; i < data.length; ++i) {
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
207 if (data[i] != null) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
208 log.debug(" column: " + data[i]);
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
209 aktAbfluesse[i] = data[i];
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
210 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
211 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
212
7488
1890bd524d27 WST-Parser: revert the assumption of only complete columns.
Tom Gottfried <tom@intevation.de>
parents: 7487
diff changeset
213 // remember Q-values from first Q-line for header generation
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
214 if (firstAbfluesse == null) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
215 firstAbfluesse = (BigDecimal [])aktAbfluesse.clone();
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
216 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
217 continue;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
218 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
219
7374
82191d111130 WstParser: Do not add Q-range for trailing Q-line.
Tom Gottfried <tom@intevation.de>
parents: 7342
diff changeset
220 // handle special column identifiers
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
221 if (line.startsWith("*!")) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
222 String spezial = line.substring(2).trim();
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
223
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
224 if (spezial.length() == 0) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
225 continue;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
226 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
227
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
228 if (spezial.startsWith(COLUMN_BEZ_TEXT)) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
229 spezial = spezial.substring(COLUMN_BEZ_TEXT.length()).trim();
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
230 if (spezial.length() == 0) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
231 continue;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
232 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
233 langBezeichner = StringUtil.splitQuoted(spezial, '"');
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
234 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
235 else if (spezial.startsWith(COLUMN_BEZ_BREITE)) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
236 spezial = spezial.substring(COLUMN_BEZ_BREITE.length()).trim();
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
237
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
238 if (spezial.length() == 0) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
239 continue;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
240 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
241
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
242 String[] split = spezial.split("\\s+");
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
243
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
244 colNaWidths = new int[split.length];
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
245 for (int i=0; i < split.length; i++) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
246 colNaWidths[i] = Integer.parseInt(split[i]);
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
247 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
248 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
249 else if (spezial.startsWith(COLUMN_QUELLE)) {
6506
1c68d4b2a1c0 Fix import of source from wst-files
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6504
diff changeset
250 spezial = spezial.substring(COLUMN_QUELLE.length()).trim();
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
251 if (spezial.length() == 0) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
252 continue;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
253 }
6504
7664ab97b4c7 Backend: Store source of a wst column if available.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6337
diff changeset
254 quellen = StringUtil.splitQuoted(spezial, '"');
7252
c894b7b45c4c Replaced some tabs by spaces. Guys, please configure your editors not to insert tabs!
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6727
diff changeset
255 log.debug("sources: " + Arrays.toString(quellen));
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
256 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
257 else if (spezial.startsWith(COLUMN_DATUM)) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
258 spezial = spezial.substring(COLUMN_DATUM.length()).trim();
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
259 if (spezial.length() == 0) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
260 continue;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
261 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
262 daten = StringUtil.splitQuoted(spezial, '"');
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
263 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
264 continue;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
265 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
266
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
267 if (line.length() < 11) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
268 continue;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
269 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
270
7374
82191d111130 WstParser: Do not add Q-range for trailing Q-line.
Tom Gottfried <tom@intevation.de>
parents: 7342
diff changeset
271 // handle comment lines to fetch unit
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
272 if (line.startsWith("*")) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
273 Matcher m = UNIT_COMMENT.matcher(line);
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
274 if (m.matches()) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
275 log.debug("unit comment found");
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
276 // XXX: This hack is needed because desktop
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
277 // FLYS is broken figuring out the unit
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
278 String [] units = m.group(1).split("\\s{2,}");
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
279 m = UNIT.matcher(units[0]);
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
280 einheit = m.matches() ? m.group(1) : units[0];
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
281 log.debug("unit: " + einheit);
6819
1d60614c1059 WstParser: warn only once if no unit comment is found
Tom Gottfried <tom.gottfried@intevation.de>
parents: 6809
diff changeset
282 unitFound = true;
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
283 }
6819
1d60614c1059 WstParser: warn only once if no unit comment is found
Tom Gottfried <tom.gottfried@intevation.de>
parents: 6809
diff changeset
284
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
285 continue;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
286 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
287
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
288 if (firstAbfluesse != null) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
289 if (!columnHeaderChecked) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
290 int unknownCount = 0;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
291 HashSet<String> uniqueColumnNames =
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
292 new HashSet<String>();
6029
f9c5e1a8032d WstParser: set column names from specified line also if unspecified line does not exist
Tom Gottfried <tom.gottfried@intevation.de>
parents: 6028
diff changeset
293 if (langBezeichner != null) {
f9c5e1a8032d WstParser: set column names from specified line also if unspecified line does not exist
Tom Gottfried <tom.gottfried@intevation.de>
parents: 6028
diff changeset
294 // use column name from '*!column-bez-text'-line
f9c5e1a8032d WstParser: set column names from specified line also if unspecified line does not exist
Tom Gottfried <tom.gottfried@intevation.de>
parents: 6028
diff changeset
295 lsBezeichner = StringUtil.fitArray(
f9c5e1a8032d WstParser: set column names from specified line also if unspecified line does not exist
Tom Gottfried <tom.gottfried@intevation.de>
parents: 6028
diff changeset
296 langBezeichner, lsBezeichner);
f9c5e1a8032d WstParser: set column names from specified line also if unspecified line does not exist
Tom Gottfried <tom.gottfried@intevation.de>
parents: 6028
diff changeset
297 }
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
298 for (int i = 0; i < lsBezeichner.length; ++i) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
299 if (lsBezeichner[i] == null
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
300 || lsBezeichner[i].length() == 0) {
6029
f9c5e1a8032d WstParser: set column names from specified line also if unspecified line does not exist
Tom Gottfried <tom.gottfried@intevation.de>
parents: 6028
diff changeset
301 // generate alternative column names
7488
1890bd524d27 WST-Parser: revert the assumption of only complete columns.
Tom Gottfried <tom@intevation.de>
parents: 7487
diff changeset
302 double q = firstAbfluesse.length > i ?
1890bd524d27 WST-Parser: revert the assumption of only complete columns.
Tom Gottfried <tom@intevation.de>
parents: 7487
diff changeset
303 firstAbfluesse[i].doubleValue() : 0d;
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
304 if (q < 0.001) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
305 lsBezeichner[i] =
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
306 "<unbekannt #" + unknownCount + ">";
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
307 ++unknownCount;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
308 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
309 else {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
310 lsBezeichner[i] = "Q="+format(q);
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
311 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
312 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
313 String candidate = lsBezeichner[i];
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
314 int collision = 1;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
315 while (!uniqueColumnNames.add(candidate)) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
316 candidate = lsBezeichner[i] +
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
317 " (" + collision + ")";
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
318 ++collision;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
319 }
2357
2e18e12fac11 Added date guesser for WST columns.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2346
diff changeset
320 ImportWstColumn iwc = wst.getColumn(i);
2e18e12fac11 Added date guesser for WST columns.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2346
diff changeset
321 iwc.setName(candidate);
6504
7664ab97b4c7 Backend: Store source of a wst column if available.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6337
diff changeset
322 if (quellen != null && i < quellen.length) {
7664ab97b4c7 Backend: Store source of a wst column if available.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6337
diff changeset
323 iwc.setSource(quellen[i]);
7664ab97b4c7 Backend: Store source of a wst column if available.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6337
diff changeset
324 }
5558
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5544
diff changeset
325 String potentialDate = daten != null && i < daten.length
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5544
diff changeset
326 ? daten[i]
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5544
diff changeset
327 : candidate;
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5544
diff changeset
328 iwc.setTimeInterval(guessDate(potentialDate));
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
329 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
330 columnHeaderChecked = true;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
331 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
332
7488
1890bd524d27 WST-Parser: revert the assumption of only complete columns.
Tom Gottfried <tom@intevation.de>
parents: 7487
diff changeset
333 BigDecimal [] data = parseLineAsDouble(line, columnCount, true, false);
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
334
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
335 BigDecimal kaem = data[0];
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
336
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
337 if (!kms.add(kaem)) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
338 log.warn(
3662
0d27d02b1208 backend: Completed the error messages of the importer.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 2357
diff changeset
339 "WST: km " + kaem +
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
340 " (line " + in.getLineNumber() +
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
341 ") found more than once. -> ignored");
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
342 continue;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
343 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
344
7342
c9b9176b0530 WST-Parser: check consistence of station ordering in file.
Tom Gottfried <tom@intevation.de>
parents: 7341
diff changeset
345 // check consistence of station ordering in file
7883
fc8d5b5f2c99 Removed trailing whitespace.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7488
diff changeset
346 if (kmHist2 != null &&
7342
c9b9176b0530 WST-Parser: check consistence of station ordering in file.
Tom Gottfried <tom@intevation.de>
parents: 7341
diff changeset
347 kmHist2.compareTo(kmHist1) != kmHist1.compareTo(kaem)) {
7883
fc8d5b5f2c99 Removed trailing whitespace.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7488
diff changeset
348 throw new ParseException("WST: Stations in " + file +
fc8d5b5f2c99 Removed trailing whitespace.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7488
diff changeset
349 " near line " + in.getLineNumber() +
7342
c9b9176b0530 WST-Parser: check consistence of station ordering in file.
Tom Gottfried <tom@intevation.de>
parents: 7341
diff changeset
350 " not ordered. File rejected.");
7375
450582bcad75 Replaced tabs with 4 spaces.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7374
diff changeset
351 }
7342
c9b9176b0530 WST-Parser: check consistence of station ordering in file.
Tom Gottfried <tom@intevation.de>
parents: 7341
diff changeset
352
c9b9176b0530 WST-Parser: check consistence of station ordering in file.
Tom Gottfried <tom@intevation.de>
parents: 7341
diff changeset
353 // remember stations in two previous lines
5544
12f4d088f088 WST importer: Cosmetics.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5543
diff changeset
354 kmHist2 = kmHist1;
12f4d088f088 WST importer: Cosmetics.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5543
diff changeset
355 kmHist1 = kaem;
5543
732a8bd44650 Wst Parser: Yet another attempt to fix the order of the Q ranges (made by S. Teichmann)
Tom Gottfried <tom.gottfried@intevation.de>
parents: 5542
diff changeset
356
7341
588b02a4078a Comments.
Tom Gottfried <tom@intevation.de>
parents: 7340
diff changeset
357 // iteratively determine actual km-range
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
358 if (kaem.compareTo(minKm) < 0) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
359 minKm = kaem;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
360 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
361 if (kaem.compareTo(maxKm) > 0) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
362 maxKm = kaem;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
363 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
364
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
365 // extract values
7488
1890bd524d27 WST-Parser: revert the assumption of only complete columns.
Tom Gottfried <tom@intevation.de>
parents: 7487
diff changeset
366 for (int i = 0; i < data.length - 1; ++i) {
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
367 addValue(kaem, data[i+1], i);
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
368 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
369
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
370 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
371 else { // firstAbfluesse == null
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
372 if (langBezeichner != null) {
6029
f9c5e1a8032d WstParser: set column names from specified line also if unspecified line does not exist
Tom Gottfried <tom.gottfried@intevation.de>
parents: 6028
diff changeset
373 // nothing to do
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
374 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
375 else if (colNaWidths != null) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
376 for (int j = 0, i = 0, N = input.length();
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
377 j < colNaWidths.length && i < N;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
378 i += colNaWidths[j++]
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
379 ) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
380 lsBezeichner[j] = input.substring(
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
381 i, i+colNaWidths[j]).trim();
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
382 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
383 }
7883
fc8d5b5f2c99 Removed trailing whitespace.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7488
diff changeset
384 else { // fetch column names from non-comment header line
7480
2a2e89c01588 WST-Parser: Improve handling of line-endings.
Tom Gottfried <tom@intevation.de>
parents: 7375
diff changeset
385 // (above first Qs)
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
386 // first column begins at position 8 in line
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
387 for (int i = 8, col = 0; i < input.length(); i += 9) {
7480
2a2e89c01588 WST-Parser: Improve handling of line-endings.
Tom Gottfried <tom@intevation.de>
parents: 7375
diff changeset
388 // one column header is 9 chars wide
2a2e89c01588 WST-Parser: Improve handling of line-endings.
Tom Gottfried <tom@intevation.de>
parents: 7375
diff changeset
389 // but the last one may be shorter
2a2e89c01588 WST-Parser: Improve handling of line-endings.
Tom Gottfried <tom@intevation.de>
parents: 7375
diff changeset
390 if (col < lsBezeichner.length) {
2a2e89c01588 WST-Parser: Improve handling of line-endings.
Tom Gottfried <tom@intevation.de>
parents: 7375
diff changeset
391 lsBezeichner[col++] =
7883
fc8d5b5f2c99 Removed trailing whitespace.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7488
diff changeset
392 input.substring(i,
7480
2a2e89c01588 WST-Parser: Improve handling of line-endings.
Tom Gottfried <tom@intevation.de>
parents: 7375
diff changeset
393 Math.min(i + 9, input.length())).trim();
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
394 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
395 if (col == lsBezeichner.length) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
396 break;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
397 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
398 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
399 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
400 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
401
5542
f3b270e5462e Wst Parser: Another attempt to fix the order of the Q ranges.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5541
diff changeset
402 } // for all lines in WST file
2346
f834b411ca57 Added db table, model class and importer stuff for units.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 1211
diff changeset
403
6819
1d60614c1059 WstParser: warn only once if no unit comment is found
Tom Gottfried <tom.gottfried@intevation.de>
parents: 6809
diff changeset
404 if (!unitFound) {
1d60614c1059 WstParser: warn only once if no unit comment is found
Tom Gottfried <tom.gottfried@intevation.de>
parents: 6809
diff changeset
405 log.warn("no unit and height reference found. Using default.");
1d60614c1059 WstParser: warn only once if no unit comment is found
Tom Gottfried <tom.gottfried@intevation.de>
parents: 6809
diff changeset
406 }
2346
f834b411ca57 Added db table, model class and importer stuff for units.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 1211
diff changeset
407 wst.setUnit(new ImportUnit(einheit));
5544
12f4d088f088 WST importer: Cosmetics.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5543
diff changeset
408
7883
fc8d5b5f2c99 Removed trailing whitespace.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7488
diff changeset
409 // add Q-ranges obtained from previous lines
7374
82191d111130 WstParser: Do not add Q-range for trailing Q-line.
Tom Gottfried <tom@intevation.de>
parents: 7342
diff changeset
410 // in case there was no further Q-line
82191d111130 WstParser: Do not add Q-range for trailing Q-line.
Tom Gottfried <tom@intevation.de>
parents: 7342
diff changeset
411 // but only if there were values following the last Q-line
82191d111130 WstParser: Do not add Q-range for trailing Q-line.
Tom Gottfried <tom@intevation.de>
parents: 7342
diff changeset
412 if (minKm != MAX_RANGE && maxKm != MIN_RANGE) {
7375
450582bcad75 Replaced tabs with 4 spaces.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7374
diff changeset
413 if (kmHist1 != null && kmHist2 != null
450582bcad75 Replaced tabs with 4 spaces.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7374
diff changeset
414 && kmHist1.compareTo(kmHist2) < 0) { // stations descending in file
450582bcad75 Replaced tabs with 4 spaces.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7374
diff changeset
415 BigDecimal t = minKm; minKm = maxKm; maxKm = t;
450582bcad75 Replaced tabs with 4 spaces.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7374
diff changeset
416 }
450582bcad75 Replaced tabs with 4 spaces.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7374
diff changeset
417 addInterval(minKm, maxKm, aktAbfluesse);
450582bcad75 Replaced tabs with 4 spaces.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7374
diff changeset
418 }
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
419 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
420 finally {
6337
224d7c4b5291 Backend: Create ImportWsts for official lines expicitly. They are going to be somewhat special. Removed some cruft from the WstParser.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6328
diff changeset
421 in.close();
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
422 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
423 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
424
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
425 protected void addValue(BigDecimal km, BigDecimal w, int index) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
426 if (w != null) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
427 ImportWstColumn column = wst.getColumn(index);
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
428 column.addColumnValue(km, w);
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
429 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
430 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
431
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
432 private static final NumberFormat NF = getNumberFormat();
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
433
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
434 private static final NumberFormat getNumberFormat() {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
435 NumberFormat nf = NumberFormat.getInstance();
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
436 nf.setMinimumFractionDigits(2);
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
437 nf.setMaximumFractionDigits(2);
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
438 return nf;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
439 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
440
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
441 protected static String format(double value) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
442 return NF.format(value);
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
443 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
444
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
445 protected void addInterval(
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
446 BigDecimal from,
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
447 BigDecimal to,
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
448 BigDecimal [] values
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
449 ) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
450 log.debug("addInterval: " + from + " " + to);
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
451
5795
202a73ce6704 WST-Parser: Avoid trying to set MIN_RANGE when actually no stations are given
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5558
diff changeset
452 if (values == null || from == MAX_RANGE || from == MIN_RANGE) {
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
453 return;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
454 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
455
7375
450582bcad75 Replaced tabs with 4 spaces.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7374
diff changeset
456 // expand single-line i.e. 0-lenght Q-range to minimal length
450582bcad75 Replaced tabs with 4 spaces.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7374
diff changeset
457 if (from == to) {
450582bcad75 Replaced tabs with 4 spaces.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7374
diff changeset
458 if (lastRange != null && lastA > lastB) {
450582bcad75 Replaced tabs with 4 spaces.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7374
diff changeset
459 to = new BigDecimal(from.doubleValue() - INTERVAL_GAP);
450582bcad75 Replaced tabs with 4 spaces.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7374
diff changeset
460 }
450582bcad75 Replaced tabs with 4 spaces.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7374
diff changeset
461 else {
450582bcad75 Replaced tabs with 4 spaces.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7374
diff changeset
462 to = new BigDecimal(from.doubleValue() + INTERVAL_GAP);
450582bcad75 Replaced tabs with 4 spaces.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7374
diff changeset
463 }
450582bcad75 Replaced tabs with 4 spaces.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7374
diff changeset
464 }
7002
0850a4983fcd Issue1333: expand single-line i.e. 0-lenght Q-range to minimal length.
Tom Gottfried <tom.gottfried@intevation.de>
parents: 6819
diff changeset
465
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
466 ImportRange range = new ImportRange(from, to);
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
467
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
468 // little workaround to make the q ranges tightly fit.
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
469 // Leave a very small gap to ensure that the range queries
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
470 // still work.
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
471
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
472 if (lastRange != null) {
5541
b09c095a0b7e WST Importer: Potential fix for tighten gaps between Q ranges.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 3662
diff changeset
473 double a2 = range.getA().doubleValue();
7222
c5214e04b9b8 WST-parser: making ranges tightly fit was broken since a<b was enforced. Re-enabled.
Tom Gottfried <tom@intevation.de>
parents: 7003
diff changeset
474 double b2 = range.getB().doubleValue();
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
475
7222
c5214e04b9b8 WST-parser: making ranges tightly fit was broken since a<b was enforced. Re-enabled.
Tom Gottfried <tom@intevation.de>
parents: 7003
diff changeset
476 if (lastA < lastB) {
5541
b09c095a0b7e WST Importer: Potential fix for tighten gaps between Q ranges.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 3662
diff changeset
477 lastRange.setB(new BigDecimal(a2 - INTERVAL_GAP));
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
478 }
7222
c5214e04b9b8 WST-parser: making ranges tightly fit was broken since a<b was enforced. Re-enabled.
Tom Gottfried <tom@intevation.de>
parents: 7003
diff changeset
479 else { // lastA >= lastB
c5214e04b9b8 WST-parser: making ranges tightly fit was broken since a<b was enforced. Re-enabled.
Tom Gottfried <tom@intevation.de>
parents: 7003
diff changeset
480 lastRange.setA(new BigDecimal(b2 + INTERVAL_GAP));
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
481 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
482 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
483
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
484 for (int i = 0; i < values.length; ++i) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
485 ImportWstColumn column = wst.getColumn(i);
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
486 ImportWstQRange wstQRange = new ImportWstQRange(range, values[i]);
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
487 column.addColumnQRange(wstQRange);
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
488 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
489
7222
c5214e04b9b8 WST-parser: making ranges tightly fit was broken since a<b was enforced. Re-enabled.
Tom Gottfried <tom@intevation.de>
parents: 7003
diff changeset
490 lastA = from.doubleValue();
c5214e04b9b8 WST-parser: making ranges tightly fit was broken since a<b was enforced. Re-enabled.
Tom Gottfried <tom@intevation.de>
parents: 7003
diff changeset
491 lastB = to.doubleValue();
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
492 lastRange = range;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
493 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
494
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
495 private static final BigDecimal [] parseLineAsDouble(
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
496 String line,
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
497 int count,
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
498 boolean bStation,
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
499 boolean bParseEmptyAsZero
7487
e327492ba9d0 Correct r7481: use ParseException instead of IllegalArgument that was catched at wrong place.
Tom Gottfried <tom@intevation.de>
parents: 7481
diff changeset
500 ) throws ParseException {
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
501 String [] tokens = parseLine(line, count, bStation);
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
502
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
503 BigDecimal [] doubles = new BigDecimal[tokens.length];
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
504
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
505 for (int i = 0; i < doubles.length; ++i) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
506 String token = tokens[i].trim();
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
507 if (token.length() != 0) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
508 doubles[i] = new BigDecimal(token);
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
509 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
510 else if (bParseEmptyAsZero) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
511 doubles[i] = UNDEFINED_ZERO;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
512 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
513 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
514
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
515 return doubles;
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
516 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
517
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
518 private static String [] parseLine(
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
519 String line,
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
520 int tokenCount,
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
521 boolean bParseStation
7487
e327492ba9d0 Correct r7481: use ParseException instead of IllegalArgument that was catched at wrong place.
Tom Gottfried <tom@intevation.de>
parents: 7481
diff changeset
522 ) throws ParseException {
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
523 ArrayList<String> strings = new ArrayList<String>();
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
524
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
525 if (bParseStation) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
526 if (line.length() < 8) {
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
527 throw new IllegalArgumentException("station too short");
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
528 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
529 strings.add(line.substring(0, 8));
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
530 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
531
7481
bdb12632c5f5 WST-Parser and doc: reject files with wrong number of columns.
Tom Gottfried <tom@intevation.de>
parents: 7480
diff changeset
532 int pos = 0;
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
533 for (int i = 0; i < tokenCount; ++i) {
7481
bdb12632c5f5 WST-Parser and doc: reject files with wrong number of columns.
Tom Gottfried <tom@intevation.de>
parents: 7480
diff changeset
534 pos += 9;
bdb12632c5f5 WST-Parser and doc: reject files with wrong number of columns.
Tom Gottfried <tom@intevation.de>
parents: 7480
diff changeset
535 if (pos >= line.length()) {
7488
1890bd524d27 WST-Parser: revert the assumption of only complete columns.
Tom Gottfried <tom@intevation.de>
parents: 7487
diff changeset
536 break;
7481
bdb12632c5f5 WST-Parser and doc: reject files with wrong number of columns.
Tom Gottfried <tom@intevation.de>
parents: 7480
diff changeset
537 }
7883
fc8d5b5f2c99 Removed trailing whitespace.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7488
diff changeset
538 strings.add(line.substring(pos,
7480
2a2e89c01588 WST-Parser: Improve handling of line-endings.
Tom Gottfried <tom@intevation.de>
parents: 7375
diff changeset
539 Math.min(pos + 8, line.length())));
1211
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
540 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
541
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
542 return strings.toArray(new String[strings.size()]);
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
543 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
544 }
f08fe480092c Moved file parsers to separate package.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
545 // vim:set ts=4 sw=4 si et sta sts=4 fenc=utf8 :
http://dive4elements.wald.intevation.org