dive4elements/river: backend/src/main/java/org/dive4elements/river/importer/parsers/WstParser.java annotate

annotate backend/src/main/java/org/dive4elements/river/importer/parsers/WstParser.java @ 9650:a2a42a6bac6b

Importer (s/u-info) extensions: outer try/catch for parse and log of line no, catching parsing exception if not enough value fields, parsing error and warning log messages with line number, detecting and rejecting duplicate data series, better differentiation between error and warning log messages

author	mschaefer
date	Mon, 23 Mar 2020 14:57:03 +0100
parents	5ff8ce9a2e06
children

rev	line source
5844 4dd33b86dc61 Added header to river backend. Sascha L. Teichmann <teichmann@intevation.de> parents: 5838 diff changeset	1 /* Copyright (C) 2011, 2012, 2013 by Bundesanstalt für Gewässerkunde
4dd33b86dc61 Added header to river backend. Sascha L. Teichmann <teichmann@intevation.de> parents: 5838 diff changeset	2 * Software engineering by Intevation GmbH
4dd33b86dc61 Added header to river backend. Sascha L. Teichmann <teichmann@intevation.de> parents: 5838 diff changeset	3 *
5992 4c3ccf2b0304 Removed trailing whitespace. Sascha L. Teichmann <teichmann@intevation.de> parents: 5844 diff changeset	4 * This file is Free Software under the GNU AGPL (>=v3)
5844 4dd33b86dc61 Added header to river backend. Sascha L. Teichmann <teichmann@intevation.de> parents: 5838 diff changeset	5 * and comes with ABSOLUTELY NO WARRANTY! Check out the
5992 4c3ccf2b0304 Removed trailing whitespace. Sascha L. Teichmann <teichmann@intevation.de> parents: 5844 diff changeset	6 * documentation coming with Dive4Elements River for details.
5844 4dd33b86dc61 Added header to river backend. Sascha L. Teichmann <teichmann@intevation.de> parents: 5838 diff changeset	7 */
4dd33b86dc61 Added header to river backend. Sascha L. Teichmann <teichmann@intevation.de> parents: 5838 diff changeset	8
5829 18619c1e7c2a Repaired internal references. Sascha L. Teichmann <teichmann@intevation.de> parents: 5828 diff changeset	9 package org.dive4elements.river.importer.parsers;
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	10
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	11 import java.io.File;
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	12 import java.io.FileInputStream;
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	13 import java.io.IOException;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	14 import java.io.InputStreamReader;
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	15 import java.io.LineNumberReader;
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	16 import java.math.BigDecimal;
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	17 import java.text.NumberFormat;
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	18 import java.util.ArrayList;
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	19 import java.util.Arrays;
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	20 import java.util.HashSet;
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	21 import java.util.regex.Matcher;
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	22 import java.util.regex.Pattern;
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	23
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	24 import org.apache.log4j.Logger;
8187 3bb1c62ad732 Moved package org.dive4elements.river.utils to org.dive4elements.river.backend.utils. Sascha L. Teichmann <teichmann@intevation.de> parents: 7883 diff changeset	25 import org.dive4elements.river.backend.utils.DateGuesser;
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	26 import org.dive4elements.river.backend.utils.StringUtil;
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	27 import org.dive4elements.river.importer.ImportRange;
5829 18619c1e7c2a Repaired internal references. Sascha L. Teichmann <teichmann@intevation.de> parents: 5828 diff changeset	28 import org.dive4elements.river.importer.ImportTimeInterval;
18619c1e7c2a Repaired internal references. Sascha L. Teichmann <teichmann@intevation.de> parents: 5828 diff changeset	29 import org.dive4elements.river.importer.ImportUnit;
18619c1e7c2a Repaired internal references. Sascha L. Teichmann <teichmann@intevation.de> parents: 5828 diff changeset	30 import org.dive4elements.river.importer.ImportWst;
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	31 import org.dive4elements.river.importer.ImportWstColumn;
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	32 import org.dive4elements.river.importer.ImportWstQRange;
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	33
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	34 public class WstParser
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	35 {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	36 private static Logger log = Logger.getLogger(WstParser.class);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	37
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	38 public static final String COLUMN_BEZ_TEXT = "column-bez-text";
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	39 public static final String COLUMN_BEZ_BREITE = "column-bez-breite";
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	40 public static final String COLUMN_QUELLE = "column-quelle";
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	41 public static final String COLUMN_DATUM = "column-datum";
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	42
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	43 public static final BigDecimal UNDEFINED_ZERO =
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	44 new BigDecimal(0.0);
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	45 public static final BigDecimal MIN_RANGE =
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	46 new BigDecimal(-Double.MAX_VALUE);
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	47 public static final BigDecimal MAX_RANGE =
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	48 new BigDecimal(Double.MAX_VALUE);
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	49
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	50 public static final String ENCODING = "ISO-8859-1";
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	51
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	52 public static final Pattern UNIT_COMMENT =
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	53 Pattern.compile("\\\\s[kK][mM]\\s+(.+)");
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	54
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	55 public static final Pattern UNIT =
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	56 Pattern.compile("[^\\[]\\[([^]]+)\\].");
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	57
6508 b2eab164a441 Backend: WST parser. Detect ranges of years in column-datum Sascha L. Teichmann <teichmann@intevation.de> parents: 6506 diff changeset	58 public static final Pattern YEAR_INTERVAL =
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	59 Pattern.compile("(\\d{4})\\s[-/]\\s(\\d{4})");
6508 b2eab164a441 Backend: WST parser. Detect ranges of years in column-datum Sascha L. Teichmann <teichmann@intevation.de> parents: 6506 diff changeset	60
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	61 public static final BigDecimal INTERVAL_GAP = new BigDecimal("0.00001");
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	62
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	63 protected ImportWst wst;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	64
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	65 protected ImportRange lastRange;
8987 5ff8ce9a2e06 Modification of the range adding fixed mschaefer parents: 8976 diff changeset	66 protected Double lastA;
5ff8ce9a2e06 Modification of the range adding fixed mschaefer parents: 8976 diff changeset	67 protected Double lastB;
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	68
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	69 public WstParser() {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	70 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	71
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	72 public WstParser(final ImportWst wst) {
6337 224d7c4b5291 Backend: Create ImportWsts for official lines expicitly. They are going to be somewhat special. Removed some cruft from the WstParser. Sascha L. Teichmann <teichmann@intevation.de> parents: 6328 diff changeset	73 this.wst = wst;
224d7c4b5291 Backend: Create ImportWsts for official lines expicitly. They are going to be somewhat special. Removed some cruft from the WstParser. Sascha L. Teichmann <teichmann@intevation.de> parents: 6328 diff changeset	74 }
224d7c4b5291 Backend: Create ImportWsts for official lines expicitly. They are going to be somewhat special. Removed some cruft from the WstParser. Sascha L. Teichmann <teichmann@intevation.de> parents: 6328 diff changeset	75
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	76 public ImportWst getWst() {
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	77 return this.wst;
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	78 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	79
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	80 public void setWst(final ImportWst wst) {
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	81 this.wst = wst;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	82 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	83
7339 40e5ad76103c WST Import: Added parse error exception to WST parser. Sascha L. Teichmann <teichmann@intevation.de> parents: 7252 diff changeset	84 public static final class ParseException extends Exception {
40e5ad76103c WST Import: Added parse error exception to WST parser. Sascha L. Teichmann <teichmann@intevation.de> parents: 7252 diff changeset	85 public ParseException() {
40e5ad76103c WST Import: Added parse error exception to WST parser. Sascha L. Teichmann <teichmann@intevation.de> parents: 7252 diff changeset	86 }
40e5ad76103c WST Import: Added parse error exception to WST parser. Sascha L. Teichmann <teichmann@intevation.de> parents: 7252 diff changeset	87
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	88 public ParseException(final String msg) {
7339 40e5ad76103c WST Import: Added parse error exception to WST parser. Sascha L. Teichmann <teichmann@intevation.de> parents: 7252 diff changeset	89 super(msg);
40e5ad76103c WST Import: Added parse error exception to WST parser. Sascha L. Teichmann <teichmann@intevation.de> parents: 7252 diff changeset	90 }
40e5ad76103c WST Import: Added parse error exception to WST parser. Sascha L. Teichmann <teichmann@intevation.de> parents: 7252 diff changeset	91 } // class ParseException
40e5ad76103c WST Import: Added parse error exception to WST parser. Sascha L. Teichmann <teichmann@intevation.de> parents: 7252 diff changeset	92
6727 af70e3ab2e83 Doc. Felix Wolfsteller <felix.wolfsteller@intevation.de> parents: 6508 diff changeset	93 /** Returns a new ImportTimeInterval with a date guessed from string. */
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	94 public static ImportTimeInterval guessDate(final String string) {
2357 2e18e12fac11 Added date guesser for WST columns. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2346 diff changeset	95 try {
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	96 final Matcher m = YEAR_INTERVAL.matcher(string);
6508 b2eab164a441 Backend: WST parser. Detect ranges of years in column-datum Sascha L. Teichmann <teichmann@intevation.de> parents: 6506 diff changeset	97 if (m.matches()) {
b2eab164a441 Backend: WST parser. Detect ranges of years in column-datum Sascha L. Teichmann <teichmann@intevation.de> parents: 6506 diff changeset	98 return new ImportTimeInterval(
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	99 DateGuesser.guessDate(m.group(1)),
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	100 DateGuesser.guessDate(m.group(2)));
6508 b2eab164a441 Backend: WST parser. Detect ranges of years in column-datum Sascha L. Teichmann <teichmann@intevation.de> parents: 6506 diff changeset	101 }
b2eab164a441 Backend: WST parser. Detect ranges of years in column-datum Sascha L. Teichmann <teichmann@intevation.de> parents: 6506 diff changeset	102
2357 2e18e12fac11 Added date guesser for WST columns. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2346 diff changeset	103 return new ImportTimeInterval(
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	104 DateGuesser.guessDate(string));
2357 2e18e12fac11 Added date guesser for WST columns. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2346 diff changeset	105 }
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	106 catch (final IllegalArgumentException iae) {
6028 25d927cb4b25 WstParser: warn if timestamp can not be set, cosmetics. Tom Gottfried <tom.gottfried@intevation.de> parents: 5992 diff changeset	107 log.warn("WST: String '" + string +
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	108 "' could not be interpreted as valid timestamp");
2357 2e18e12fac11 Added date guesser for WST columns. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2346 diff changeset	109 }
2e18e12fac11 Added date guesser for WST columns. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2346 diff changeset	110 return null;
2e18e12fac11 Added date guesser for WST columns. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2346 diff changeset	111 }
2e18e12fac11 Added date guesser for WST columns. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2346 diff changeset	112
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	113 public void parse(final File file) throws IOException, ParseException {
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	114
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	115 log.info("Parsing WST file '" + file + "'");
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	116
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	117 if (this.wst == null) {
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	118 this.wst = new ImportWst(file.getName());
7375 450582bcad75 Replaced tabs with 4 spaces. Sascha L. Teichmann <teichmann@intevation.de> parents: 7374 diff changeset	119 }
450582bcad75 Replaced tabs with 4 spaces. Sascha L. Teichmann <teichmann@intevation.de> parents: 7374 diff changeset	120 else {
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	121 this.wst.setDescription(file.getName());
7375 450582bcad75 Replaced tabs with 4 spaces. Sascha L. Teichmann <teichmann@intevation.de> parents: 7374 diff changeset	122 }
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	123
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	124 final LineNumberReader in =
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	125 new LineNumberReader(
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	126 new InputStreamReader(
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	127 new FileInputStream(file), ENCODING));
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	128 try {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	129 String input;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	130 boolean first = true;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	131 int columnCount = 0;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	132
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	133 String [] lsBezeichner = null;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	134 String [] langBezeichner = null;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	135 int [] colNaWidths = null;
6504 7664ab97b4c7 Backend: Store source of a wst column if available. Sascha L. Teichmann <teichmann@intevation.de> parents: 6337 diff changeset	136 String [] quellen = null;
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	137 String [] daten = null;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	138
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	139 BigDecimal [] aktAbfluesse = null;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	140 BigDecimal [] firstAbfluesse = null;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	141
5544 12f4d088f088 WST importer: Cosmetics. Sascha L. Teichmann <teichmann@intevation.de> parents: 5543 diff changeset	142 BigDecimal minKm = MAX_RANGE;
12f4d088f088 WST importer: Cosmetics. Sascha L. Teichmann <teichmann@intevation.de> parents: 5543 diff changeset	143 BigDecimal maxKm = MIN_RANGE;
12f4d088f088 WST importer: Cosmetics. Sascha L. Teichmann <teichmann@intevation.de> parents: 5543 diff changeset	144 BigDecimal kmHist1 = null;
12f4d088f088 WST importer: Cosmetics. Sascha L. Teichmann <teichmann@intevation.de> parents: 5543 diff changeset	145 BigDecimal kmHist2 = null;
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	146
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	147 boolean columnHeaderChecked = false;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	148
7883 fc8d5b5f2c99 Removed trailing whitespace. Sascha L. Teichmann <teichmann@intevation.de> parents: 7488 diff changeset	149 /* Default string for altitude reference
6819 1d60614c1059 WstParser: warn only once if no unit comment is found Tom Gottfried <tom.gottfried@intevation.de> parents: 6809 diff changeset	150 * if none is found in WST-file.
1d60614c1059 WstParser: warn only once if no unit comment is found Tom Gottfried <tom.gottfried@intevation.de> parents: 6809 diff changeset	151 * Use in case no unit comment is found in file */
6809 4ff177f48952 WstParser and Importer doc: More sensible default unit and warn if it is used. Tom Gottfried <tom.gottfried@intevation.de> parents: 6730 diff changeset	152 String einheit = "m ü. unbekannte Referenz";
6819 1d60614c1059 WstParser: warn only once if no unit comment is found Tom Gottfried <tom.gottfried@intevation.de> parents: 6809 diff changeset	153 boolean unitFound = false;
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	154
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	155 final HashSet<BigDecimal> kms = new HashSet<>();
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	156
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	157 while ((input = in.readLine()) != null) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	158 String line = input;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	159 if (first) { // fetch number of columns
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	160 if ((line = line.trim()).length() == 0) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	161 continue;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	162 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	163 try {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	164 columnCount = Integer.parseInt(line);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	165 if (columnCount <= 0) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	166 throw new NumberFormatException(
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	167 "number of columns <= 0");
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	168 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	169 log.debug("Number of columns: " + columnCount);
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	170 this.wst.setNumberColumns(columnCount);
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	171 lsBezeichner = new String[columnCount];
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	172 }
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	173 catch (final NumberFormatException nfe) {
3662 0d27d02b1208 backend: Completed the error messages of the importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2357 diff changeset	174 log.warn("WST: invalid number.", nfe);
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	175 continue;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	176 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	177 first = false;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	178 continue;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	179 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	180
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	181 line = line.replace(',', '.');
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	182
7374 82191d111130 WstParser: Do not add Q-range for trailing Q-line. Tom Gottfried <tom@intevation.de> parents: 7342 diff changeset	183 // handle Q-lines
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	184 if (line.startsWith("*\u001f")) {
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	185 final BigDecimal [] data = parseLineAsDouble(
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	186 line, columnCount, false, true);
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	187
8856 5e38e2924c07 Fix code style. Tom Gottfried <tom@intevation.de> parents: 8187 diff changeset	188 if (aktAbfluesse != null) {
5e38e2924c07 Fix code style. Tom Gottfried <tom@intevation.de> parents: 8187 diff changeset	189 // add Q-ranges obtained from previous lines
5544 12f4d088f088 WST importer: Cosmetics. Sascha L. Teichmann <teichmann@intevation.de> parents: 5543 diff changeset	190 if (kmHist1 != null && kmHist2 != null
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	191 && kmHist1.compareTo(kmHist2) < 0) {
8856 5e38e2924c07 Fix code style. Tom Gottfried <tom@intevation.de> parents: 8187 diff changeset	192 // stations descending in file
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	193 final BigDecimal t = minKm; minKm = maxKm; maxKm = t;
5544 12f4d088f088 WST importer: Cosmetics. Sascha L. Teichmann <teichmann@intevation.de> parents: 5543 diff changeset	194 }
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	195 addInterval(minKm, maxKm, aktAbfluesse);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	196 minKm = MAX_RANGE;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	197 maxKm = MIN_RANGE;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	198 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	199
7374 82191d111130 WstParser: Do not add Q-range for trailing Q-line. Tom Gottfried <tom@intevation.de> parents: 7342 diff changeset	200 // obtain Q-values from current line
7488 1890bd524d27 WST-Parser: revert the assumption of only complete columns. Tom Gottfried <tom@intevation.de> parents: 7487 diff changeset	201 aktAbfluesse = new BigDecimal[data.length];
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	202 log.debug("new q range: " + columnCount);
7488 1890bd524d27 WST-Parser: revert the assumption of only complete columns. Tom Gottfried <tom@intevation.de> parents: 7487 diff changeset	203 for (int i = 0; i < data.length; ++i) {
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	204 if (data[i] != null) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	205 log.debug(" column: " + data[i]);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	206 aktAbfluesse[i] = data[i];
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	207 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	208 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	209
8856 5e38e2924c07 Fix code style. Tom Gottfried <tom@intevation.de> parents: 8187 diff changeset	210 // remember Q-values from first Q-line
5e38e2924c07 Fix code style. Tom Gottfried <tom@intevation.de> parents: 8187 diff changeset	211 // for header generation
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	212 if (firstAbfluesse == null) {
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	213 firstAbfluesse = aktAbfluesse.clone();
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	214 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	215 continue;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	216 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	217
7374 82191d111130 WstParser: Do not add Q-range for trailing Q-line. Tom Gottfried <tom@intevation.de> parents: 7342 diff changeset	218 // handle special column identifiers
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	219 if (line.startsWith("*!")) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	220 String spezial = line.substring(2).trim();
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	221
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	222 if (spezial.length() == 0) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	223 continue;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	224 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	225
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	226 if (spezial.startsWith(COLUMN_BEZ_TEXT)) {
8856 5e38e2924c07 Fix code style. Tom Gottfried <tom@intevation.de> parents: 8187 diff changeset	227 spezial = spezial.substring(
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	228 COLUMN_BEZ_TEXT.length()).trim();
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	229 if (spezial.length() == 0) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	230 continue;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	231 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	232 langBezeichner = StringUtil.splitQuoted(spezial, '"');
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	233 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	234 else if (spezial.startsWith(COLUMN_BEZ_BREITE)) {
8856 5e38e2924c07 Fix code style. Tom Gottfried <tom@intevation.de> parents: 8187 diff changeset	235 spezial = spezial.substring(
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	236 COLUMN_BEZ_BREITE.length()).trim();
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	237
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	238 if (spezial.length() == 0) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	239 continue;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	240 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	241
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	242 final String[] split = spezial.split("\\s+");
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	243
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	244 colNaWidths = new int[split.length];
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	245 for (int i=0; i < split.length; i++) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	246 colNaWidths[i] = Integer.parseInt(split[i]);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	247 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	248 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	249 else if (spezial.startsWith(COLUMN_QUELLE)) {
8856 5e38e2924c07 Fix code style. Tom Gottfried <tom@intevation.de> parents: 8187 diff changeset	250 spezial = spezial.substring(
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	251 COLUMN_QUELLE.length()).trim();
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	252 if (spezial.length() == 0) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	253 continue;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	254 }
6504 7664ab97b4c7 Backend: Store source of a wst column if available. Sascha L. Teichmann <teichmann@intevation.de> parents: 6337 diff changeset	255 quellen = StringUtil.splitQuoted(spezial, '"');
7252 c894b7b45c4c Replaced some tabs by spaces. Guys, please configure your editors not to insert tabs! Sascha L. Teichmann <teichmann@intevation.de> parents: 6727 diff changeset	256 log.debug("sources: " + Arrays.toString(quellen));
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	257 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	258 else if (spezial.startsWith(COLUMN_DATUM)) {
8856 5e38e2924c07 Fix code style. Tom Gottfried <tom@intevation.de> parents: 8187 diff changeset	259 spezial = spezial.substring(
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	260 COLUMN_DATUM.length()).trim();
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	261 if (spezial.length() == 0) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	262 continue;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	263 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	264 daten = StringUtil.splitQuoted(spezial, '"');
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	265 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	266 continue;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	267 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	268
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	269 if (line.length() < 11) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	270 continue;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	271 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	272
7374 82191d111130 WstParser: Do not add Q-range for trailing Q-line. Tom Gottfried <tom@intevation.de> parents: 7342 diff changeset	273 // handle comment lines to fetch unit
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	274 if (line.startsWith("*")) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	275 Matcher m = UNIT_COMMENT.matcher(line);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	276 if (m.matches()) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	277 log.debug("unit comment found");
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	278 // XXX: This hack is needed because desktop
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	279 // FLYS is broken figuring out the unit
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	280 final String [] units = m.group(1).split("\\s{2,}");
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	281 m = UNIT.matcher(units[0]);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	282 einheit = m.matches() ? m.group(1) : units[0];
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	283 log.debug("unit: " + einheit);
6819 1d60614c1059 WstParser: warn only once if no unit comment is found Tom Gottfried <tom.gottfried@intevation.de> parents: 6809 diff changeset	284 unitFound = true;
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	285 }
6819 1d60614c1059 WstParser: warn only once if no unit comment is found Tom Gottfried <tom.gottfried@intevation.de> parents: 6809 diff changeset	286
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	287 continue;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	288 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	289
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	290 if (firstAbfluesse != null) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	291 if (!columnHeaderChecked) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	292 int unknownCount = 0;
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	293 final HashSet<String> uniqueColumnNames =
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	294 new HashSet<>();
6029 f9c5e1a8032d WstParser: set column names from specified line also if unspecified line does not exist Tom Gottfried <tom.gottfried@intevation.de> parents: 6028 diff changeset	295 if (langBezeichner != null) {
f9c5e1a8032d WstParser: set column names from specified line also if unspecified line does not exist Tom Gottfried <tom.gottfried@intevation.de> parents: 6028 diff changeset	296 // use column name from '*!column-bez-text'-line
f9c5e1a8032d WstParser: set column names from specified line also if unspecified line does not exist Tom Gottfried <tom.gottfried@intevation.de> parents: 6028 diff changeset	297 lsBezeichner = StringUtil.fitArray(
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	298 langBezeichner, lsBezeichner);
6029 f9c5e1a8032d WstParser: set column names from specified line also if unspecified line does not exist Tom Gottfried <tom.gottfried@intevation.de> parents: 6028 diff changeset	299 }
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	300 for (int i = 0; i < lsBezeichner.length; ++i) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	301 if (lsBezeichner[i] == null
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	302 \|\| lsBezeichner[i].length() == 0) {
6029 f9c5e1a8032d WstParser: set column names from specified line also if unspecified line does not exist Tom Gottfried <tom.gottfried@intevation.de> parents: 6028 diff changeset	303 // generate alternative column names
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	304 final double q = firstAbfluesse.length > i ?
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	305 firstAbfluesse[i].doubleValue() : 0d;
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	306 if (q < 0.001) {
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	307 lsBezeichner[i] =
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	308 "<unbekannt #" + unknownCount + ">";
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	309 ++unknownCount;
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	310 }
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	311 else {
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	312 lsBezeichner[i] = "Q="+format(q);
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	313 }
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	314 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	315 String candidate = lsBezeichner[i];
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	316 int collision = 1;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	317 while (!uniqueColumnNames.add(candidate)) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	318 candidate = lsBezeichner[i] +
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	319 " (" + collision + ")";
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	320 ++collision;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	321 }
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	322 final ImportWstColumn iwc = this.wst.getColumn(i);
2357 2e18e12fac11 Added date guesser for WST columns. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2346 diff changeset	323 iwc.setName(candidate);
6504 7664ab97b4c7 Backend: Store source of a wst column if available. Sascha L. Teichmann <teichmann@intevation.de> parents: 6337 diff changeset	324 if (quellen != null && i < quellen.length) {
7664ab97b4c7 Backend: Store source of a wst column if available. Sascha L. Teichmann <teichmann@intevation.de> parents: 6337 diff changeset	325 iwc.setSource(quellen[i]);
7664ab97b4c7 Backend: Store source of a wst column if available. Sascha L. Teichmann <teichmann@intevation.de> parents: 6337 diff changeset	326 }
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	327 final String potentialDate =
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	328 daten != null && i < daten.length
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	329 ? daten[i]
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	330 : candidate;
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	331 iwc.setTimeInterval(guessDate(potentialDate));
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	332 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	333 columnHeaderChecked = true;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	334 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	335
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	336 final BigDecimal [] data = parseLineAsDouble(
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	337 line, columnCount, true, false);
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	338
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	339 final BigDecimal kaem = data[0];
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	340
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	341 if (!kms.add(kaem)) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	342 log.warn(
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	343 "WST: km " + kaem +
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	344 " (line " + in.getLineNumber() +
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	345 ") found more than once. -> ignored");
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	346 continue;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	347 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	348
7342 c9b9176b0530 WST-Parser: check consistence of station ordering in file. Tom Gottfried <tom@intevation.de> parents: 7341 diff changeset	349 // check consistence of station ordering in file
7883 fc8d5b5f2c99 Removed trailing whitespace. Sascha L. Teichmann <teichmann@intevation.de> parents: 7488 diff changeset	350 if (kmHist2 != null &&
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	351 kmHist2.compareTo(kmHist1) != kmHist1.compareTo(kaem)
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	352 ) {
7883 fc8d5b5f2c99 Removed trailing whitespace. Sascha L. Teichmann <teichmann@intevation.de> parents: 7488 diff changeset	353 throw new ParseException("WST: Stations in " + file +
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	354 " near line " + in.getLineNumber() +
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	355 " not ordered. File rejected.");
7375 450582bcad75 Replaced tabs with 4 spaces. Sascha L. Teichmann <teichmann@intevation.de> parents: 7374 diff changeset	356 }
7342 c9b9176b0530 WST-Parser: check consistence of station ordering in file. Tom Gottfried <tom@intevation.de> parents: 7341 diff changeset	357
c9b9176b0530 WST-Parser: check consistence of station ordering in file. Tom Gottfried <tom@intevation.de> parents: 7341 diff changeset	358 // remember stations in two previous lines
5544 12f4d088f088 WST importer: Cosmetics. Sascha L. Teichmann <teichmann@intevation.de> parents: 5543 diff changeset	359 kmHist2 = kmHist1;
12f4d088f088 WST importer: Cosmetics. Sascha L. Teichmann <teichmann@intevation.de> parents: 5543 diff changeset	360 kmHist1 = kaem;
5543 732a8bd44650 Wst Parser: Yet another attempt to fix the order of the Q ranges (made by S. Teichmann) Tom Gottfried <tom.gottfried@intevation.de> parents: 5542 diff changeset	361
7341 588b02a4078a Comments. Tom Gottfried <tom@intevation.de> parents: 7340 diff changeset	362 // iteratively determine actual km-range
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	363 if (kaem.compareTo(minKm) < 0) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	364 minKm = kaem;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	365 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	366 if (kaem.compareTo(maxKm) > 0) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	367 maxKm = kaem;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	368 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	369
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	370 // extract values
7488 1890bd524d27 WST-Parser: revert the assumption of only complete columns. Tom Gottfried <tom@intevation.de> parents: 7487 diff changeset	371 for (int i = 0; i < data.length - 1; ++i) {
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	372 addValue(kaem, data[i+1], i);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	373 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	374
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	375 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	376 else { // firstAbfluesse == null
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	377 if (langBezeichner != null) {
6029 f9c5e1a8032d WstParser: set column names from specified line also if unspecified line does not exist Tom Gottfried <tom.gottfried@intevation.de> parents: 6028 diff changeset	378 // nothing to do
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	379 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	380 else if (colNaWidths != null) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	381 for (int j = 0, i = 0, N = input.length();
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	382 j < colNaWidths.length && i < N;
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	383 i += colNaWidths[j++]
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	384 ) {
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	385 lsBezeichner[j] = input.substring(
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	386 i, i+colNaWidths[j]).trim();
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	387 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	388 }
7883 fc8d5b5f2c99 Removed trailing whitespace. Sascha L. Teichmann <teichmann@intevation.de> parents: 7488 diff changeset	389 else { // fetch column names from non-comment header line
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	390 // (above first Qs)
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	391 // first column begins at position 8 in line
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	392 for (int i = 8, col = 0; i < input.length(); i += 9) {
7480 2a2e89c01588 WST-Parser: Improve handling of line-endings. Tom Gottfried <tom@intevation.de> parents: 7375 diff changeset	393 // one column header is 9 chars wide
2a2e89c01588 WST-Parser: Improve handling of line-endings. Tom Gottfried <tom@intevation.de> parents: 7375 diff changeset	394 // but the last one may be shorter
2a2e89c01588 WST-Parser: Improve handling of line-endings. Tom Gottfried <tom@intevation.de> parents: 7375 diff changeset	395 if (col < lsBezeichner.length) {
2a2e89c01588 WST-Parser: Improve handling of line-endings. Tom Gottfried <tom@intevation.de> parents: 7375 diff changeset	396 lsBezeichner[col++] =
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	397 input.substring(
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	398 i,
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	399 Math.min(i + 9, input.length())
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	400 ).trim();
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	401 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	402 if (col == lsBezeichner.length) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	403 break;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	404 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	405 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	406 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	407 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	408
5542 f3b270e5462e Wst Parser: Another attempt to fix the order of the Q ranges. Sascha L. Teichmann <teichmann@intevation.de> parents: 5541 diff changeset	409 } // for all lines in WST file
2346 f834b411ca57 Added db table, model class and importer stuff for units. Ingo Weinzierl <ingo.weinzierl@intevation.de> parents: 1211 diff changeset	410
6819 1d60614c1059 WstParser: warn only once if no unit comment is found Tom Gottfried <tom.gottfried@intevation.de> parents: 6809 diff changeset	411 if (!unitFound) {
1d60614c1059 WstParser: warn only once if no unit comment is found Tom Gottfried <tom.gottfried@intevation.de> parents: 6809 diff changeset	412 log.warn("no unit and height reference found. Using default.");
1d60614c1059 WstParser: warn only once if no unit comment is found Tom Gottfried <tom.gottfried@intevation.de> parents: 6809 diff changeset	413 }
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	414 this.wst.setUnit(new ImportUnit(einheit));
5544 12f4d088f088 WST importer: Cosmetics. Sascha L. Teichmann <teichmann@intevation.de> parents: 5543 diff changeset	415
7883 fc8d5b5f2c99 Removed trailing whitespace. Sascha L. Teichmann <teichmann@intevation.de> parents: 7488 diff changeset	416 // add Q-ranges obtained from previous lines
7374 82191d111130 WstParser: Do not add Q-range for trailing Q-line. Tom Gottfried <tom@intevation.de> parents: 7342 diff changeset	417 // in case there was no further Q-line
82191d111130 WstParser: Do not add Q-range for trailing Q-line. Tom Gottfried <tom@intevation.de> parents: 7342 diff changeset	418 // but only if there were values following the last Q-line
82191d111130 WstParser: Do not add Q-range for trailing Q-line. Tom Gottfried <tom@intevation.de> parents: 7342 diff changeset	419 if (minKm != MAX_RANGE && maxKm != MIN_RANGE) {
7375 450582bcad75 Replaced tabs with 4 spaces. Sascha L. Teichmann <teichmann@intevation.de> parents: 7374 diff changeset	420 if (kmHist1 != null && kmHist2 != null
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	421 && kmHist1.compareTo(kmHist2) < 0) {
8856 5e38e2924c07 Fix code style. Tom Gottfried <tom@intevation.de> parents: 8187 diff changeset	422 // stations descending in file
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	423 final BigDecimal t = minKm; minKm = maxKm; maxKm = t;
7375 450582bcad75 Replaced tabs with 4 spaces. Sascha L. Teichmann <teichmann@intevation.de> parents: 7374 diff changeset	424 }
450582bcad75 Replaced tabs with 4 spaces. Sascha L. Teichmann <teichmann@intevation.de> parents: 7374 diff changeset	425 addInterval(minKm, maxKm, aktAbfluesse);
450582bcad75 Replaced tabs with 4 spaces. Sascha L. Teichmann <teichmann@intevation.de> parents: 7374 diff changeset	426 }
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	427 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	428 finally {
6337 224d7c4b5291 Backend: Create ImportWsts for official lines expicitly. They are going to be somewhat special. Removed some cruft from the WstParser. Sascha L. Teichmann <teichmann@intevation.de> parents: 6328 diff changeset	429 in.close();
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	430 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	431 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	432
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	433 protected void addValue(final BigDecimal km, final BigDecimal w, final int index) {
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	434 if (w != null) {
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	435 final ImportWstColumn column = this.wst.getColumn(index);
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	436 column.addColumnValue(km, w);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	437 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	438 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	439
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	440 private static final NumberFormat NF = getNumberFormat();
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	441
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	442 private static final NumberFormat getNumberFormat() {
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	443 final NumberFormat nf = NumberFormat.getInstance();
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	444 nf.setMinimumFractionDigits(2);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	445 nf.setMaximumFractionDigits(2);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	446 return nf;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	447 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	448
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	449 protected static String format(final double value) {
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	450 return NF.format(value);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	451 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	452
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	453 protected void addInterval(
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	454 final BigDecimal from,
8987 5ff8ce9a2e06 Modification of the range adding fixed mschaefer parents: 8976 diff changeset	455 BigDecimal to,
5ff8ce9a2e06 Modification of the range adding fixed mschaefer parents: 8976 diff changeset	456 final BigDecimal[] values) {
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	457 log.debug("addInterval: " + from + " " + to);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	458
5795 202a73ce6704 WST-Parser: Avoid trying to set MIN_RANGE when actually no stations are given Felix Wolfsteller <felix.wolfsteller@intevation.de> parents: 5558 diff changeset	459 if (values == null \|\| from == MAX_RANGE \|\| from == MIN_RANGE) {
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	460 return;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	461 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	462
7375 450582bcad75 Replaced tabs with 4 spaces. Sascha L. Teichmann <teichmann@intevation.de> parents: 7374 diff changeset	463 // expand single-line i.e. 0-lenght Q-range to minimal length
450582bcad75 Replaced tabs with 4 spaces. Sascha L. Teichmann <teichmann@intevation.de> parents: 7374 diff changeset	464 if (from == to) {
8987 5ff8ce9a2e06 Modification of the range adding fixed mschaefer parents: 8976 diff changeset	465 if (this.lastRange != null && this.lastA > this.lastB) {
5ff8ce9a2e06 Modification of the range adding fixed mschaefer parents: 8976 diff changeset	466 to = from.subtract(INTERVAL_GAP);
5ff8ce9a2e06 Modification of the range adding fixed mschaefer parents: 8976 diff changeset	467 }
5ff8ce9a2e06 Modification of the range adding fixed mschaefer parents: 8976 diff changeset	468 else {
5ff8ce9a2e06 Modification of the range adding fixed mschaefer parents: 8976 diff changeset	469 to = from.add(INTERVAL_GAP);
5ff8ce9a2e06 Modification of the range adding fixed mschaefer parents: 8976 diff changeset	470 }
7375 450582bcad75 Replaced tabs with 4 spaces. Sascha L. Teichmann <teichmann@intevation.de> parents: 7374 diff changeset	471 }
7002 0850a4983fcd Issue1333: expand single-line i.e. 0-lenght Q-range to minimal length. Tom Gottfried <tom.gottfried@intevation.de> parents: 6819 diff changeset	472
8987 5ff8ce9a2e06 Modification of the range adding fixed mschaefer parents: 8976 diff changeset	473 final ImportRange range = new ImportRange(from, to);
5ff8ce9a2e06 Modification of the range adding fixed mschaefer parents: 8976 diff changeset	474
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	475 // little workaround to make the q ranges tightly fit.
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	476 // Leave a very small gap to ensure that the range queries
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	477 // still work.
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	478
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	479 if (this.lastRange != null) {
8987 5ff8ce9a2e06 Modification of the range adding fixed mschaefer parents: 8976 diff changeset	480 if (this.lastA < this.lastB) {
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	481 this.lastRange.setB(range.getA().subtract(INTERVAL_GAP));
8987 5ff8ce9a2e06 Modification of the range adding fixed mschaefer parents: 8976 diff changeset	482 }
5ff8ce9a2e06 Modification of the range adding fixed mschaefer parents: 8976 diff changeset	483 else { // lastA >= lastB
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	484 this.lastRange.setA(range.getB().add(INTERVAL_GAP));
8987 5ff8ce9a2e06 Modification of the range adding fixed mschaefer parents: 8976 diff changeset	485 }
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	486 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	487
8987 5ff8ce9a2e06 Modification of the range adding fixed mschaefer parents: 8976 diff changeset	488 for (int i = 0; i < values.length; ++i) {
5ff8ce9a2e06 Modification of the range adding fixed mschaefer parents: 8976 diff changeset	489 final ImportWstColumn column = this.wst.getColumn(i);
5ff8ce9a2e06 Modification of the range adding fixed mschaefer parents: 8976 diff changeset	490 final ImportWstQRange wstQRange = new ImportWstQRange(range, values[i]);
5ff8ce9a2e06 Modification of the range adding fixed mschaefer parents: 8976 diff changeset	491 column.addColumnQRange(wstQRange);
5ff8ce9a2e06 Modification of the range adding fixed mschaefer parents: 8976 diff changeset	492 }
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	493
8987 5ff8ce9a2e06 Modification of the range adding fixed mschaefer parents: 8976 diff changeset	494 this.lastA = from.doubleValue();
5ff8ce9a2e06 Modification of the range adding fixed mschaefer parents: 8976 diff changeset	495 this.lastB = to.doubleValue();
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	496 this.lastRange = range;
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	497 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	498
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	499 private static final BigDecimal [] parseLineAsDouble(
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	500 final String line,
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	501 final int count,
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	502 final boolean bStation,
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	503 final boolean bParseEmptyAsZero
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	504 ) throws ParseException {
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	505 final String [] tokens = parseLine(line, count, bStation);
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	506
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	507 final BigDecimal [] doubles = new BigDecimal[tokens.length];
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	508
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	509 for (int i = 0; i < doubles.length; ++i) {
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	510 final String token = tokens[i].trim();
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	511 if (token.length() != 0) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	512 doubles[i] = new BigDecimal(token);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	513 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	514 else if (bParseEmptyAsZero) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	515 doubles[i] = UNDEFINED_ZERO;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	516 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	517 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	518
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	519 return doubles;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	520 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	521
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	522 private static String [] parseLine(
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	523 final String line,
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	524 final int tokenCount,
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	525 final boolean bParseStation
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	526 ) throws ParseException {
e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	527 final ArrayList<String> strings = new ArrayList<>();
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	528
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	529 if (bParseStation) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	530 if (line.length() < 8) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	531 throw new IllegalArgumentException("station too short");
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	532 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	533 strings.add(line.substring(0, 8));
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	534 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	535
7481 bdb12632c5f5 WST-Parser and doc: reject files with wrong number of columns. Tom Gottfried <tom@intevation.de> parents: 7480 diff changeset	536 int pos = 0;
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	537 for (int i = 0; i < tokenCount; ++i) {
7481 bdb12632c5f5 WST-Parser and doc: reject files with wrong number of columns. Tom Gottfried <tom@intevation.de> parents: 7480 diff changeset	538 pos += 9;
bdb12632c5f5 WST-Parser and doc: reject files with wrong number of columns. Tom Gottfried <tom@intevation.de> parents: 7480 diff changeset	539 if (pos >= line.length()) {
7488 1890bd524d27 WST-Parser: revert the assumption of only complete columns. Tom Gottfried <tom@intevation.de> parents: 7487 diff changeset	540 break;
7481 bdb12632c5f5 WST-Parser and doc: reject files with wrong number of columns. Tom Gottfried <tom@intevation.de> parents: 7480 diff changeset	541 }
7883 fc8d5b5f2c99 Removed trailing whitespace. Sascha L. Teichmann <teichmann@intevation.de> parents: 7488 diff changeset	542 strings.add(line.substring(pos,
8976 e541938dd3ab Range data handled consistently as BigDecimal to minimize the fractional part of a and b mschaefer parents: 8856 diff changeset	543 Math.min(pos + 8, line.length())));
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	544 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	545
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	546 return strings.toArray(new String[strings.size()]);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	547 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	548 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	549 // vim:set ts=4 sw=4 si et sta sts=4 fenc=utf8 :

Mercurial > dive4elements > river

annotate backend/src/main/java/org/dive4elements/river/importer/parsers/WstParser.java @ 9650:a2a42a6bac6b