dive4elements/river: backend/src/main/java/org/dive4elements/river/importer/parsers/WstParser.java annotate

annotate backend/src/main/java/org/dive4elements/river/importer/parsers/WstParser.java @ 7481:bdb12632c5f5

WST-Parser and doc: reject files with wrong number of columns.

author	Tom Gottfried <tom@intevation.de>
date	Thu, 31 Oct 2013 15:25:31 +0100
parents	2a2e89c01588
children	e327492ba9d0

rev	line source
5844 4dd33b86dc61 Added header to river backend. Sascha L. Teichmann <teichmann@intevation.de> parents: 5838 diff changeset	1 /* Copyright (C) 2011, 2012, 2013 by Bundesanstalt für Gewässerkunde
4dd33b86dc61 Added header to river backend. Sascha L. Teichmann <teichmann@intevation.de> parents: 5838 diff changeset	2 * Software engineering by Intevation GmbH
4dd33b86dc61 Added header to river backend. Sascha L. Teichmann <teichmann@intevation.de> parents: 5838 diff changeset	3 *
5992 4c3ccf2b0304 Removed trailing whitespace. Sascha L. Teichmann <teichmann@intevation.de> parents: 5844 diff changeset	4 * This file is Free Software under the GNU AGPL (>=v3)
5844 4dd33b86dc61 Added header to river backend. Sascha L. Teichmann <teichmann@intevation.de> parents: 5838 diff changeset	5 * and comes with ABSOLUTELY NO WARRANTY! Check out the
5992 4c3ccf2b0304 Removed trailing whitespace. Sascha L. Teichmann <teichmann@intevation.de> parents: 5844 diff changeset	6 * documentation coming with Dive4Elements River for details.
5844 4dd33b86dc61 Added header to river backend. Sascha L. Teichmann <teichmann@intevation.de> parents: 5838 diff changeset	7 */
4dd33b86dc61 Added header to river backend. Sascha L. Teichmann <teichmann@intevation.de> parents: 5838 diff changeset	8
5829 18619c1e7c2a Repaired internal references. Sascha L. Teichmann <teichmann@intevation.de> parents: 5828 diff changeset	9 package org.dive4elements.river.importer.parsers;
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	10
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	11 import java.util.ArrayList;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	12 import java.util.HashSet;
6506 1c68d4b2a1c0 Fix import of source from wst-files Sascha L. Teichmann <teichmann@intevation.de> parents: 6504 diff changeset	13 import java.util.Arrays;
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	14
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	15 import java.io.File;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	16 import java.io.IOException;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	17 import java.io.LineNumberReader;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	18 import java.io.InputStreamReader;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	19 import java.io.FileInputStream;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	20
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	21 import java.text.NumberFormat;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	22
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	23 import org.apache.log4j.Logger;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	24
5829 18619c1e7c2a Repaired internal references. Sascha L. Teichmann <teichmann@intevation.de> parents: 5828 diff changeset	25 import org.dive4elements.river.utils.StringUtil;
18619c1e7c2a Repaired internal references. Sascha L. Teichmann <teichmann@intevation.de> parents: 5828 diff changeset	26 import org.dive4elements.river.utils.DateGuesser;
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	27
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	28 import java.util.regex.Pattern;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	29 import java.util.regex.Matcher;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	30
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	31 import java.math.BigDecimal;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	32
5829 18619c1e7c2a Repaired internal references. Sascha L. Teichmann <teichmann@intevation.de> parents: 5828 diff changeset	33 import org.dive4elements.river.importer.ImportWstQRange;
18619c1e7c2a Repaired internal references. Sascha L. Teichmann <teichmann@intevation.de> parents: 5828 diff changeset	34 import org.dive4elements.river.importer.ImportWstColumn;
18619c1e7c2a Repaired internal references. Sascha L. Teichmann <teichmann@intevation.de> parents: 5828 diff changeset	35 import org.dive4elements.river.importer.ImportTimeInterval;
18619c1e7c2a Repaired internal references. Sascha L. Teichmann <teichmann@intevation.de> parents: 5828 diff changeset	36 import org.dive4elements.river.importer.ImportRange;
18619c1e7c2a Repaired internal references. Sascha L. Teichmann <teichmann@intevation.de> parents: 5828 diff changeset	37 import org.dive4elements.river.importer.ImportUnit;
18619c1e7c2a Repaired internal references. Sascha L. Teichmann <teichmann@intevation.de> parents: 5828 diff changeset	38 import org.dive4elements.river.importer.ImportWst;
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	39
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	40 public class WstParser
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	41 {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	42 private static Logger log = Logger.getLogger(WstParser.class);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	43
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	44 public static final String COLUMN_BEZ_TEXT = "column-bez-text";
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	45 public static final String COLUMN_BEZ_BREITE = "column-bez-breite";
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	46 public static final String COLUMN_QUELLE = "column-quelle";
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	47 public static final String COLUMN_DATUM = "column-datum";
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	48
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	49 public static final BigDecimal UNDEFINED_ZERO =
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	50 new BigDecimal(0.0);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	51 public static final BigDecimal MIN_RANGE =
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	52 new BigDecimal(-Double.MAX_VALUE);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	53 public static final BigDecimal MAX_RANGE =
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	54 new BigDecimal(Double.MAX_VALUE);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	55
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	56 public static final String ENCODING = "ISO-8859-1";
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	57
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	58 public static final Pattern UNIT_COMMENT =
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	59 Pattern.compile("\\\\s[kK][mM]\\s+(.+)");
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	60
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	61 public static final Pattern UNIT =
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	62 Pattern.compile("[^\\[]\\[([^]]+)\\].");
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	63
6508 b2eab164a441 Backend: WST parser. Detect ranges of years in column-datum Sascha L. Teichmann <teichmann@intevation.de> parents: 6506 diff changeset	64 public static final Pattern YEAR_INTERVAL =
b2eab164a441 Backend: WST parser. Detect ranges of years in column-datum Sascha L. Teichmann <teichmann@intevation.de> parents: 6506 diff changeset	65 Pattern.compile("(\\d{4})\\s[-/]\\s(\\d{4})");
b2eab164a441 Backend: WST parser. Detect ranges of years in column-datum Sascha L. Teichmann <teichmann@intevation.de> parents: 6506 diff changeset	66
5541 b09c095a0b7e WST Importer: Potential fix for tighten gaps between Q ranges. Sascha L. Teichmann <teichmann@intevation.de> parents: 3662 diff changeset	67 public static final double INTERVAL_GAP = 0.00001d;
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	68
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	69 protected ImportWst wst;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	70
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	71 protected ImportRange lastRange;
7222 c5214e04b9b8 WST-parser: making ranges tightly fit was broken since a<b was enforced. Re-enabled. Tom Gottfried <tom@intevation.de> parents: 7003 diff changeset	72 protected Double lastA;
c5214e04b9b8 WST-parser: making ranges tightly fit was broken since a<b was enforced. Re-enabled. Tom Gottfried <tom@intevation.de> parents: 7003 diff changeset	73 protected Double lastB;
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	74
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	75 public WstParser() {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	76 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	77
6337 224d7c4b5291 Backend: Create ImportWsts for official lines expicitly. They are going to be somewhat special. Removed some cruft from the WstParser. Sascha L. Teichmann <teichmann@intevation.de> parents: 6328 diff changeset	78 public WstParser(ImportWst wst) {
224d7c4b5291 Backend: Create ImportWsts for official lines expicitly. They are going to be somewhat special. Removed some cruft from the WstParser. Sascha L. Teichmann <teichmann@intevation.de> parents: 6328 diff changeset	79 this.wst = wst;
224d7c4b5291 Backend: Create ImportWsts for official lines expicitly. They are going to be somewhat special. Removed some cruft from the WstParser. Sascha L. Teichmann <teichmann@intevation.de> parents: 6328 diff changeset	80 }
224d7c4b5291 Backend: Create ImportWsts for official lines expicitly. They are going to be somewhat special. Removed some cruft from the WstParser. Sascha L. Teichmann <teichmann@intevation.de> parents: 6328 diff changeset	81
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	82 public ImportWst getWst() {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	83 return wst;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	84 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	85
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	86 public void setWst(ImportWst wst) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	87 this.wst = wst;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	88 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	89
7339 40e5ad76103c WST Import: Added parse error exception to WST parser. Sascha L. Teichmann <teichmann@intevation.de> parents: 7252 diff changeset	90 public static final class ParseException extends Exception {
40e5ad76103c WST Import: Added parse error exception to WST parser. Sascha L. Teichmann <teichmann@intevation.de> parents: 7252 diff changeset	91 public ParseException() {
40e5ad76103c WST Import: Added parse error exception to WST parser. Sascha L. Teichmann <teichmann@intevation.de> parents: 7252 diff changeset	92 }
40e5ad76103c WST Import: Added parse error exception to WST parser. Sascha L. Teichmann <teichmann@intevation.de> parents: 7252 diff changeset	93
40e5ad76103c WST Import: Added parse error exception to WST parser. Sascha L. Teichmann <teichmann@intevation.de> parents: 7252 diff changeset	94 public ParseException(String msg) {
40e5ad76103c WST Import: Added parse error exception to WST parser. Sascha L. Teichmann <teichmann@intevation.de> parents: 7252 diff changeset	95 super(msg);
40e5ad76103c WST Import: Added parse error exception to WST parser. Sascha L. Teichmann <teichmann@intevation.de> parents: 7252 diff changeset	96 }
40e5ad76103c WST Import: Added parse error exception to WST parser. Sascha L. Teichmann <teichmann@intevation.de> parents: 7252 diff changeset	97 } // class ParseException
40e5ad76103c WST Import: Added parse error exception to WST parser. Sascha L. Teichmann <teichmann@intevation.de> parents: 7252 diff changeset	98
6727 af70e3ab2e83 Doc. Felix Wolfsteller <felix.wolfsteller@intevation.de> parents: 6508 diff changeset	99 /** Returns a new ImportTimeInterval with a date guessed from string. */
6328 53d08f33d094 Backend: Moved guessing of main values and there time intervals out of the STA parser. Same come will be useful to extend the WST parser to better handle official lines. Sascha L. Teichmann <teichmann@intevation.de> parents: 6029 diff changeset	100 public static ImportTimeInterval guessDate(String string) {
2357 2e18e12fac11 Added date guesser for WST columns. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2346 diff changeset	101 try {
6508 b2eab164a441 Backend: WST parser. Detect ranges of years in column-datum Sascha L. Teichmann <teichmann@intevation.de> parents: 6506 diff changeset	102 Matcher m = YEAR_INTERVAL.matcher(string);
b2eab164a441 Backend: WST parser. Detect ranges of years in column-datum Sascha L. Teichmann <teichmann@intevation.de> parents: 6506 diff changeset	103 if (m.matches()) {
b2eab164a441 Backend: WST parser. Detect ranges of years in column-datum Sascha L. Teichmann <teichmann@intevation.de> parents: 6506 diff changeset	104 return new ImportTimeInterval(
b2eab164a441 Backend: WST parser. Detect ranges of years in column-datum Sascha L. Teichmann <teichmann@intevation.de> parents: 6506 diff changeset	105 DateGuesser.guessDate(m.group(1)),
b2eab164a441 Backend: WST parser. Detect ranges of years in column-datum Sascha L. Teichmann <teichmann@intevation.de> parents: 6506 diff changeset	106 DateGuesser.guessDate(m.group(2)));
b2eab164a441 Backend: WST parser. Detect ranges of years in column-datum Sascha L. Teichmann <teichmann@intevation.de> parents: 6506 diff changeset	107 }
b2eab164a441 Backend: WST parser. Detect ranges of years in column-datum Sascha L. Teichmann <teichmann@intevation.de> parents: 6506 diff changeset	108
2357 2e18e12fac11 Added date guesser for WST columns. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2346 diff changeset	109 return new ImportTimeInterval(
2e18e12fac11 Added date guesser for WST columns. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2346 diff changeset	110 DateGuesser.guessDate(string));
2e18e12fac11 Added date guesser for WST columns. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2346 diff changeset	111 }
2e18e12fac11 Added date guesser for WST columns. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2346 diff changeset	112 catch (IllegalArgumentException iae) {
6028 25d927cb4b25 WstParser: warn if timestamp can not be set, cosmetics. Tom Gottfried <tom.gottfried@intevation.de> parents: 5992 diff changeset	113 log.warn("WST: String '" + string +
25d927cb4b25 WstParser: warn if timestamp can not be set, cosmetics. Tom Gottfried <tom.gottfried@intevation.de> parents: 5992 diff changeset	114 "' could not be interpreted as valid timestamp");
2357 2e18e12fac11 Added date guesser for WST columns. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2346 diff changeset	115 }
2e18e12fac11 Added date guesser for WST columns. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2346 diff changeset	116 return null;
2e18e12fac11 Added date guesser for WST columns. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2346 diff changeset	117 }
2e18e12fac11 Added date guesser for WST columns. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2346 diff changeset	118
7339 40e5ad76103c WST Import: Added parse error exception to WST parser. Sascha L. Teichmann <teichmann@intevation.de> parents: 7252 diff changeset	119 public void parse(File file) throws IOException, ParseException {
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	120
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	121 log.info("Parsing WST file '" + file + "'");
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	122
7375 450582bcad75 Replaced tabs with 4 spaces. Sascha L. Teichmann <teichmann@intevation.de> parents: 7374 diff changeset	123 if (wst == null) {
450582bcad75 Replaced tabs with 4 spaces. Sascha L. Teichmann <teichmann@intevation.de> parents: 7374 diff changeset	124 wst = new ImportWst(file.getName());
450582bcad75 Replaced tabs with 4 spaces. Sascha L. Teichmann <teichmann@intevation.de> parents: 7374 diff changeset	125 }
450582bcad75 Replaced tabs with 4 spaces. Sascha L. Teichmann <teichmann@intevation.de> parents: 7374 diff changeset	126 else {
450582bcad75 Replaced tabs with 4 spaces. Sascha L. Teichmann <teichmann@intevation.de> parents: 7374 diff changeset	127 wst.setDescription(file.getName());
450582bcad75 Replaced tabs with 4 spaces. Sascha L. Teichmann <teichmann@intevation.de> parents: 7374 diff changeset	128 }
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	129
6337 224d7c4b5291 Backend: Create ImportWsts for official lines expicitly. They are going to be somewhat special. Removed some cruft from the WstParser. Sascha L. Teichmann <teichmann@intevation.de> parents: 6328 diff changeset	130 LineNumberReader in =
224d7c4b5291 Backend: Create ImportWsts for official lines expicitly. They are going to be somewhat special. Removed some cruft from the WstParser. Sascha L. Teichmann <teichmann@intevation.de> parents: 6328 diff changeset	131 new LineNumberReader(
224d7c4b5291 Backend: Create ImportWsts for official lines expicitly. They are going to be somewhat special. Removed some cruft from the WstParser. Sascha L. Teichmann <teichmann@intevation.de> parents: 6328 diff changeset	132 new InputStreamReader(
224d7c4b5291 Backend: Create ImportWsts for official lines expicitly. They are going to be somewhat special. Removed some cruft from the WstParser. Sascha L. Teichmann <teichmann@intevation.de> parents: 6328 diff changeset	133 new FileInputStream(file), ENCODING));
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	134 try {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	135 String input;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	136 boolean first = true;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	137 int columnCount = 0;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	138
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	139 String [] lsBezeichner = null;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	140 String [] langBezeichner = null;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	141 int [] colNaWidths = null;
6504 7664ab97b4c7 Backend: Store source of a wst column if available. Sascha L. Teichmann <teichmann@intevation.de> parents: 6337 diff changeset	142 String [] quellen = null;
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	143 String [] daten = null;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	144
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	145 BigDecimal [] aktAbfluesse = null;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	146 BigDecimal [] firstAbfluesse = null;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	147
5544 12f4d088f088 WST importer: Cosmetics. Sascha L. Teichmann <teichmann@intevation.de> parents: 5543 diff changeset	148 BigDecimal minKm = MAX_RANGE;
12f4d088f088 WST importer: Cosmetics. Sascha L. Teichmann <teichmann@intevation.de> parents: 5543 diff changeset	149 BigDecimal maxKm = MIN_RANGE;
12f4d088f088 WST importer: Cosmetics. Sascha L. Teichmann <teichmann@intevation.de> parents: 5543 diff changeset	150 BigDecimal kmHist1 = null;
12f4d088f088 WST importer: Cosmetics. Sascha L. Teichmann <teichmann@intevation.de> parents: 5543 diff changeset	151 BigDecimal kmHist2 = null;
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	152
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	153 boolean columnHeaderChecked = false;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	154
6819 1d60614c1059 WstParser: warn only once if no unit comment is found Tom Gottfried <tom.gottfried@intevation.de> parents: 6809 diff changeset	155 /* Default string for altitude reference
1d60614c1059 WstParser: warn only once if no unit comment is found Tom Gottfried <tom.gottfried@intevation.de> parents: 6809 diff changeset	156 * if none is found in WST-file.
1d60614c1059 WstParser: warn only once if no unit comment is found Tom Gottfried <tom.gottfried@intevation.de> parents: 6809 diff changeset	157 * Use in case no unit comment is found in file */
6809 4ff177f48952 WstParser and Importer doc: More sensible default unit and warn if it is used. Tom Gottfried <tom.gottfried@intevation.de> parents: 6730 diff changeset	158 String einheit = "m ü. unbekannte Referenz";
6819 1d60614c1059 WstParser: warn only once if no unit comment is found Tom Gottfried <tom.gottfried@intevation.de> parents: 6809 diff changeset	159 boolean unitFound = false;
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	160
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	161 HashSet<BigDecimal> kms = new HashSet<BigDecimal>();
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	162
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	163 while ((input = in.readLine()) != null) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	164 String line = input;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	165 if (first) { // fetch number of columns
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	166 if ((line = line.trim()).length() == 0) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	167 continue;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	168 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	169 try {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	170 columnCount = Integer.parseInt(line);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	171 if (columnCount <= 0) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	172 throw new NumberFormatException(
6028 25d927cb4b25 WstParser: warn if timestamp can not be set, cosmetics. Tom Gottfried <tom.gottfried@intevation.de> parents: 5992 diff changeset	173 "number of columns <= 0");
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	174 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	175 log.debug("Number of columns: " + columnCount);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	176 wst.setNumberColumns(columnCount);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	177 lsBezeichner = new String[columnCount];
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	178 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	179 catch (NumberFormatException nfe) {
3662 0d27d02b1208 backend: Completed the error messages of the importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2357 diff changeset	180 log.warn("WST: invalid number.", nfe);
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	181 continue;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	182 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	183 first = false;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	184 continue;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	185 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	186
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	187 line = line.replace(',', '.');
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	188
7374 82191d111130 WstParser: Do not add Q-range for trailing Q-line. Tom Gottfried <tom@intevation.de> parents: 7342 diff changeset	189 // handle Q-lines
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	190 if (line.startsWith("*\u001f")) {
7481 bdb12632c5f5 WST-Parser and doc: reject files with wrong number of columns. Tom Gottfried <tom@intevation.de> parents: 7480 diff changeset	191 BigDecimal [] data = null;
bdb12632c5f5 WST-Parser and doc: reject files with wrong number of columns. Tom Gottfried <tom@intevation.de> parents: 7480 diff changeset	192 try {
bdb12632c5f5 WST-Parser and doc: reject files with wrong number of columns. Tom Gottfried <tom@intevation.de> parents: 7480 diff changeset	193 data = parseLineAsDouble(line, columnCount, false, true);
bdb12632c5f5 WST-Parser and doc: reject files with wrong number of columns. Tom Gottfried <tom@intevation.de> parents: 7480 diff changeset	194 }
bdb12632c5f5 WST-Parser and doc: reject files with wrong number of columns. Tom Gottfried <tom@intevation.de> parents: 7480 diff changeset	195 catch(IllegalArgumentException iae) {
bdb12632c5f5 WST-Parser and doc: reject files with wrong number of columns. Tom Gottfried <tom@intevation.de> parents: 7480 diff changeset	196 log.error(iae.getMessage());
bdb12632c5f5 WST-Parser and doc: reject files with wrong number of columns. Tom Gottfried <tom@intevation.de> parents: 7480 diff changeset	197 return;
bdb12632c5f5 WST-Parser and doc: reject files with wrong number of columns. Tom Gottfried <tom@intevation.de> parents: 7480 diff changeset	198 }
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	199
7374 82191d111130 WstParser: Do not add Q-range for trailing Q-line. Tom Gottfried <tom@intevation.de> parents: 7342 diff changeset	200 if (aktAbfluesse != null) { // add Q-ranges obtained from previous lines
5544 12f4d088f088 WST importer: Cosmetics. Sascha L. Teichmann <teichmann@intevation.de> parents: 5543 diff changeset	201 if (kmHist1 != null && kmHist2 != null
7341 588b02a4078a Comments. Tom Gottfried <tom@intevation.de> parents: 7340 diff changeset	202 && kmHist1.compareTo(kmHist2) < 0) { // stations descending in file
5544 12f4d088f088 WST importer: Cosmetics. Sascha L. Teichmann <teichmann@intevation.de> parents: 5543 diff changeset	203 BigDecimal t = minKm; minKm = maxKm; maxKm = t;
12f4d088f088 WST importer: Cosmetics. Sascha L. Teichmann <teichmann@intevation.de> parents: 5543 diff changeset	204 }
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	205 addInterval(minKm, maxKm, aktAbfluesse);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	206 minKm = MAX_RANGE;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	207 maxKm = MIN_RANGE;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	208 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	209
7374 82191d111130 WstParser: Do not add Q-range for trailing Q-line. Tom Gottfried <tom@intevation.de> parents: 7342 diff changeset	210 // obtain Q-values from current line
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	211 aktAbfluesse = new BigDecimal[columnCount];
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	212 log.debug("new q range: " + columnCount);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	213 for (int i = 0; i < Math.min(columnCount, data.length); ++i) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	214 if (data[i] != null) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	215 log.debug(" column: " + data[i]);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	216 aktAbfluesse[i] = data[i];
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	217 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	218 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	219
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	220 if (firstAbfluesse == null) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	221 firstAbfluesse = (BigDecimal [])aktAbfluesse.clone();
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	222 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	223 continue;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	224 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	225
7374 82191d111130 WstParser: Do not add Q-range for trailing Q-line. Tom Gottfried <tom@intevation.de> parents: 7342 diff changeset	226 // handle special column identifiers
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	227 if (line.startsWith("*!")) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	228 String spezial = line.substring(2).trim();
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	229
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	230 if (spezial.length() == 0) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	231 continue;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	232 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	233
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	234 if (spezial.startsWith(COLUMN_BEZ_TEXT)) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	235 spezial = spezial.substring(COLUMN_BEZ_TEXT.length()).trim();
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	236 if (spezial.length() == 0) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	237 continue;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	238 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	239 langBezeichner = StringUtil.splitQuoted(spezial, '"');
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	240 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	241 else if (spezial.startsWith(COLUMN_BEZ_BREITE)) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	242 spezial = spezial.substring(COLUMN_BEZ_BREITE.length()).trim();
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	243
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	244 if (spezial.length() == 0) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	245 continue;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	246 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	247
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	248 String[] split = spezial.split("\\s+");
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	249
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	250 colNaWidths = new int[split.length];
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	251 for (int i=0; i < split.length; i++) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	252 colNaWidths[i] = Integer.parseInt(split[i]);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	253 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	254 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	255 else if (spezial.startsWith(COLUMN_QUELLE)) {
6506 1c68d4b2a1c0 Fix import of source from wst-files Sascha L. Teichmann <teichmann@intevation.de> parents: 6504 diff changeset	256 spezial = spezial.substring(COLUMN_QUELLE.length()).trim();
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	257 if (spezial.length() == 0) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	258 continue;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	259 }
6504 7664ab97b4c7 Backend: Store source of a wst column if available. Sascha L. Teichmann <teichmann@intevation.de> parents: 6337 diff changeset	260 quellen = StringUtil.splitQuoted(spezial, '"');
7252 c894b7b45c4c Replaced some tabs by spaces. Guys, please configure your editors not to insert tabs! Sascha L. Teichmann <teichmann@intevation.de> parents: 6727 diff changeset	261 log.debug("sources: " + Arrays.toString(quellen));
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	262 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	263 else if (spezial.startsWith(COLUMN_DATUM)) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	264 spezial = spezial.substring(COLUMN_DATUM.length()).trim();
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	265 if (spezial.length() == 0) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	266 continue;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	267 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	268 daten = StringUtil.splitQuoted(spezial, '"');
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	269 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	270 continue;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	271 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	272
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	273 if (line.length() < 11) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	274 continue;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	275 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	276
7374 82191d111130 WstParser: Do not add Q-range for trailing Q-line. Tom Gottfried <tom@intevation.de> parents: 7342 diff changeset	277 // handle comment lines to fetch unit
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	278 if (line.startsWith("*")) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	279 Matcher m = UNIT_COMMENT.matcher(line);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	280 if (m.matches()) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	281 log.debug("unit comment found");
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	282 // XXX: This hack is needed because desktop
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	283 // FLYS is broken figuring out the unit
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	284 String [] units = m.group(1).split("\\s{2,}");
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	285 m = UNIT.matcher(units[0]);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	286 einheit = m.matches() ? m.group(1) : units[0];
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	287 log.debug("unit: " + einheit);
6819 1d60614c1059 WstParser: warn only once if no unit comment is found Tom Gottfried <tom.gottfried@intevation.de> parents: 6809 diff changeset	288 unitFound = true;
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	289 }
6819 1d60614c1059 WstParser: warn only once if no unit comment is found Tom Gottfried <tom.gottfried@intevation.de> parents: 6809 diff changeset	290
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	291 continue;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	292 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	293
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	294 if (firstAbfluesse != null) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	295 if (!columnHeaderChecked) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	296 int unknownCount = 0;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	297 HashSet<String> uniqueColumnNames =
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	298 new HashSet<String>();
6029 f9c5e1a8032d WstParser: set column names from specified line also if unspecified line does not exist Tom Gottfried <tom.gottfried@intevation.de> parents: 6028 diff changeset	299 if (langBezeichner != null) {
f9c5e1a8032d WstParser: set column names from specified line also if unspecified line does not exist Tom Gottfried <tom.gottfried@intevation.de> parents: 6028 diff changeset	300 // use column name from '*!column-bez-text'-line
f9c5e1a8032d WstParser: set column names from specified line also if unspecified line does not exist Tom Gottfried <tom.gottfried@intevation.de> parents: 6028 diff changeset	301 lsBezeichner = StringUtil.fitArray(
f9c5e1a8032d WstParser: set column names from specified line also if unspecified line does not exist Tom Gottfried <tom.gottfried@intevation.de> parents: 6028 diff changeset	302 langBezeichner, lsBezeichner);
f9c5e1a8032d WstParser: set column names from specified line also if unspecified line does not exist Tom Gottfried <tom.gottfried@intevation.de> parents: 6028 diff changeset	303 }
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	304 for (int i = 0; i < lsBezeichner.length; ++i) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	305 if (lsBezeichner[i] == null
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	306 \|\| lsBezeichner[i].length() == 0) {
6029 f9c5e1a8032d WstParser: set column names from specified line also if unspecified line does not exist Tom Gottfried <tom.gottfried@intevation.de> parents: 6028 diff changeset	307 // generate alternative column names
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	308 double q = firstAbfluesse[i].doubleValue();
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	309 if (q < 0.001) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	310 lsBezeichner[i] =
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	311 "<unbekannt #" + unknownCount + ">";
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	312 ++unknownCount;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	313 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	314 else {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	315 lsBezeichner[i] = "Q="+format(q);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	316 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	317 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	318 String candidate = lsBezeichner[i];
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	319 int collision = 1;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	320 while (!uniqueColumnNames.add(candidate)) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	321 candidate = lsBezeichner[i] +
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	322 " (" + collision + ")";
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	323 ++collision;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	324 }
2357 2e18e12fac11 Added date guesser for WST columns. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2346 diff changeset	325 ImportWstColumn iwc = wst.getColumn(i);
2e18e12fac11 Added date guesser for WST columns. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2346 diff changeset	326 iwc.setName(candidate);
6504 7664ab97b4c7 Backend: Store source of a wst column if available. Sascha L. Teichmann <teichmann@intevation.de> parents: 6337 diff changeset	327 if (quellen != null && i < quellen.length) {
7664ab97b4c7 Backend: Store source of a wst column if available. Sascha L. Teichmann <teichmann@intevation.de> parents: 6337 diff changeset	328 iwc.setSource(quellen[i]);
7664ab97b4c7 Backend: Store source of a wst column if available. Sascha L. Teichmann <teichmann@intevation.de> parents: 6337 diff changeset	329 }
5558 b223be96758c Merged Sascha L. Teichmann <teichmann@intevation.de> parents: 5544 diff changeset	330 String potentialDate = daten != null && i < daten.length
b223be96758c Merged Sascha L. Teichmann <teichmann@intevation.de> parents: 5544 diff changeset	331 ? daten[i]
b223be96758c Merged Sascha L. Teichmann <teichmann@intevation.de> parents: 5544 diff changeset	332 : candidate;
b223be96758c Merged Sascha L. Teichmann <teichmann@intevation.de> parents: 5544 diff changeset	333 iwc.setTimeInterval(guessDate(potentialDate));
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	334 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	335 columnHeaderChecked = true;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	336 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	337
7481 bdb12632c5f5 WST-Parser and doc: reject files with wrong number of columns. Tom Gottfried <tom@intevation.de> parents: 7480 diff changeset	338 BigDecimal [] data = null;
bdb12632c5f5 WST-Parser and doc: reject files with wrong number of columns. Tom Gottfried <tom@intevation.de> parents: 7480 diff changeset	339 try {
bdb12632c5f5 WST-Parser and doc: reject files with wrong number of columns. Tom Gottfried <tom@intevation.de> parents: 7480 diff changeset	340 data = parseLineAsDouble(line, columnCount, true, false);
bdb12632c5f5 WST-Parser and doc: reject files with wrong number of columns. Tom Gottfried <tom@intevation.de> parents: 7480 diff changeset	341 }
bdb12632c5f5 WST-Parser and doc: reject files with wrong number of columns. Tom Gottfried <tom@intevation.de> parents: 7480 diff changeset	342 catch(IllegalArgumentException iae) {
bdb12632c5f5 WST-Parser and doc: reject files with wrong number of columns. Tom Gottfried <tom@intevation.de> parents: 7480 diff changeset	343 log.error(iae.getMessage());
bdb12632c5f5 WST-Parser and doc: reject files with wrong number of columns. Tom Gottfried <tom@intevation.de> parents: 7480 diff changeset	344 return;
bdb12632c5f5 WST-Parser and doc: reject files with wrong number of columns. Tom Gottfried <tom@intevation.de> parents: 7480 diff changeset	345 }
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	346
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	347 BigDecimal kaem = data[0];
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	348
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	349 if (!kms.add(kaem)) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	350 log.warn(
3662 0d27d02b1208 backend: Completed the error messages of the importer. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: 2357 diff changeset	351 "WST: km " + kaem +
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	352 " (line " + in.getLineNumber() +
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	353 ") found more than once. -> ignored");
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	354 continue;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	355 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	356
7342 c9b9176b0530 WST-Parser: check consistence of station ordering in file. Tom Gottfried <tom@intevation.de> parents: 7341 diff changeset	357 // check consistence of station ordering in file
c9b9176b0530 WST-Parser: check consistence of station ordering in file. Tom Gottfried <tom@intevation.de> parents: 7341 diff changeset	358 if (kmHist2 != null &&
c9b9176b0530 WST-Parser: check consistence of station ordering in file. Tom Gottfried <tom@intevation.de> parents: 7341 diff changeset	359 kmHist2.compareTo(kmHist1) != kmHist1.compareTo(kaem)) {
7481 bdb12632c5f5 WST-Parser and doc: reject files with wrong number of columns. Tom Gottfried <tom@intevation.de> parents: 7480 diff changeset	360 throw new ParseException("WST: Stations in " + file +
7342 c9b9176b0530 WST-Parser: check consistence of station ordering in file. Tom Gottfried <tom@intevation.de> parents: 7341 diff changeset	361 " near line " + in.getLineNumber() +
c9b9176b0530 WST-Parser: check consistence of station ordering in file. Tom Gottfried <tom@intevation.de> parents: 7341 diff changeset	362 " not ordered. File rejected.");
7375 450582bcad75 Replaced tabs with 4 spaces. Sascha L. Teichmann <teichmann@intevation.de> parents: 7374 diff changeset	363 }
7342 c9b9176b0530 WST-Parser: check consistence of station ordering in file. Tom Gottfried <tom@intevation.de> parents: 7341 diff changeset	364
c9b9176b0530 WST-Parser: check consistence of station ordering in file. Tom Gottfried <tom@intevation.de> parents: 7341 diff changeset	365 // remember stations in two previous lines
5544 12f4d088f088 WST importer: Cosmetics. Sascha L. Teichmann <teichmann@intevation.de> parents: 5543 diff changeset	366 kmHist2 = kmHist1;
12f4d088f088 WST importer: Cosmetics. Sascha L. Teichmann <teichmann@intevation.de> parents: 5543 diff changeset	367 kmHist1 = kaem;
5543 732a8bd44650 Wst Parser: Yet another attempt to fix the order of the Q ranges (made by S. Teichmann) Tom Gottfried <tom.gottfried@intevation.de> parents: 5542 diff changeset	368
7341 588b02a4078a Comments. Tom Gottfried <tom@intevation.de> parents: 7340 diff changeset	369 // iteratively determine actual km-range
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	370 if (kaem.compareTo(minKm) < 0) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	371 minKm = kaem;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	372 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	373 if (kaem.compareTo(maxKm) > 0) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	374 maxKm = kaem;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	375 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	376
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	377 // extract values
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	378 for (int i = 0; i < columnCount; ++i) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	379 addValue(kaem, data[i+1], i);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	380 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	381
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	382 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	383 else { // firstAbfluesse == null
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	384 if (langBezeichner != null) {
6029 f9c5e1a8032d WstParser: set column names from specified line also if unspecified line does not exist Tom Gottfried <tom.gottfried@intevation.de> parents: 6028 diff changeset	385 // nothing to do
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	386 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	387 else if (colNaWidths != null) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	388 for (int j = 0, i = 0, N = input.length();
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	389 j < colNaWidths.length && i < N;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	390 i += colNaWidths[j++]
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	391 ) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	392 lsBezeichner[j] = input.substring(
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	393 i, i+colNaWidths[j]).trim();
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	394 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	395 }
7480 2a2e89c01588 WST-Parser: Improve handling of line-endings. Tom Gottfried <tom@intevation.de> parents: 7375 diff changeset	396 else { // fetch column names from non-comment header line
2a2e89c01588 WST-Parser: Improve handling of line-endings. Tom Gottfried <tom@intevation.de> parents: 7375 diff changeset	397 // (above first Qs)
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	398 // first column begins at position 8 in line
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	399 for (int i = 8, col = 0; i < input.length(); i += 9) {
7480 2a2e89c01588 WST-Parser: Improve handling of line-endings. Tom Gottfried <tom@intevation.de> parents: 7375 diff changeset	400 // one column header is 9 chars wide
2a2e89c01588 WST-Parser: Improve handling of line-endings. Tom Gottfried <tom@intevation.de> parents: 7375 diff changeset	401 // but the last one may be shorter
2a2e89c01588 WST-Parser: Improve handling of line-endings. Tom Gottfried <tom@intevation.de> parents: 7375 diff changeset	402 if (col < lsBezeichner.length) {
2a2e89c01588 WST-Parser: Improve handling of line-endings. Tom Gottfried <tom@intevation.de> parents: 7375 diff changeset	403 lsBezeichner[col++] =
2a2e89c01588 WST-Parser: Improve handling of line-endings. Tom Gottfried <tom@intevation.de> parents: 7375 diff changeset	404 input.substring(i,
2a2e89c01588 WST-Parser: Improve handling of line-endings. Tom Gottfried <tom@intevation.de> parents: 7375 diff changeset	405 Math.min(i + 9, input.length())).trim();
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	406 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	407 if (col == lsBezeichner.length) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	408 break;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	409 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	410 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	411 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	412 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	413
5542 f3b270e5462e Wst Parser: Another attempt to fix the order of the Q ranges. Sascha L. Teichmann <teichmann@intevation.de> parents: 5541 diff changeset	414 } // for all lines in WST file
2346 f834b411ca57 Added db table, model class and importer stuff for units. Ingo Weinzierl <ingo.weinzierl@intevation.de> parents: 1211 diff changeset	415
6819 1d60614c1059 WstParser: warn only once if no unit comment is found Tom Gottfried <tom.gottfried@intevation.de> parents: 6809 diff changeset	416 if (!unitFound) {
1d60614c1059 WstParser: warn only once if no unit comment is found Tom Gottfried <tom.gottfried@intevation.de> parents: 6809 diff changeset	417 log.warn("no unit and height reference found. Using default.");
1d60614c1059 WstParser: warn only once if no unit comment is found Tom Gottfried <tom.gottfried@intevation.de> parents: 6809 diff changeset	418 }
2346 f834b411ca57 Added db table, model class and importer stuff for units. Ingo Weinzierl <ingo.weinzierl@intevation.de> parents: 1211 diff changeset	419 wst.setUnit(new ImportUnit(einheit));
5544 12f4d088f088 WST importer: Cosmetics. Sascha L. Teichmann <teichmann@intevation.de> parents: 5543 diff changeset	420
7374 82191d111130 WstParser: Do not add Q-range for trailing Q-line. Tom Gottfried <tom@intevation.de> parents: 7342 diff changeset	421 // add Q-ranges obtained from previous lines
82191d111130 WstParser: Do not add Q-range for trailing Q-line. Tom Gottfried <tom@intevation.de> parents: 7342 diff changeset	422 // in case there was no further Q-line
82191d111130 WstParser: Do not add Q-range for trailing Q-line. Tom Gottfried <tom@intevation.de> parents: 7342 diff changeset	423 // but only if there were values following the last Q-line
82191d111130 WstParser: Do not add Q-range for trailing Q-line. Tom Gottfried <tom@intevation.de> parents: 7342 diff changeset	424 if (minKm != MAX_RANGE && maxKm != MIN_RANGE) {
7375 450582bcad75 Replaced tabs with 4 spaces. Sascha L. Teichmann <teichmann@intevation.de> parents: 7374 diff changeset	425 if (kmHist1 != null && kmHist2 != null
450582bcad75 Replaced tabs with 4 spaces. Sascha L. Teichmann <teichmann@intevation.de> parents: 7374 diff changeset	426 && kmHist1.compareTo(kmHist2) < 0) { // stations descending in file
450582bcad75 Replaced tabs with 4 spaces. Sascha L. Teichmann <teichmann@intevation.de> parents: 7374 diff changeset	427 BigDecimal t = minKm; minKm = maxKm; maxKm = t;
450582bcad75 Replaced tabs with 4 spaces. Sascha L. Teichmann <teichmann@intevation.de> parents: 7374 diff changeset	428 }
450582bcad75 Replaced tabs with 4 spaces. Sascha L. Teichmann <teichmann@intevation.de> parents: 7374 diff changeset	429 addInterval(minKm, maxKm, aktAbfluesse);
450582bcad75 Replaced tabs with 4 spaces. Sascha L. Teichmann <teichmann@intevation.de> parents: 7374 diff changeset	430 }
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	431 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	432 finally {
6337 224d7c4b5291 Backend: Create ImportWsts for official lines expicitly. They are going to be somewhat special. Removed some cruft from the WstParser. Sascha L. Teichmann <teichmann@intevation.de> parents: 6328 diff changeset	433 in.close();
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	434 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	435 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	436
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	437 protected void addValue(BigDecimal km, BigDecimal w, int index) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	438 if (w != null) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	439 ImportWstColumn column = wst.getColumn(index);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	440 column.addColumnValue(km, w);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	441 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	442 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	443
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	444 private static final NumberFormat NF = getNumberFormat();
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	445
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	446 private static final NumberFormat getNumberFormat() {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	447 NumberFormat nf = NumberFormat.getInstance();
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	448 nf.setMinimumFractionDigits(2);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	449 nf.setMaximumFractionDigits(2);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	450 return nf;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	451 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	452
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	453 protected static String format(double value) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	454 return NF.format(value);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	455 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	456
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	457 protected void addInterval(
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	458 BigDecimal from,
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	459 BigDecimal to,
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	460 BigDecimal [] values
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	461 ) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	462 log.debug("addInterval: " + from + " " + to);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	463
5795 202a73ce6704 WST-Parser: Avoid trying to set MIN_RANGE when actually no stations are given Felix Wolfsteller <felix.wolfsteller@intevation.de> parents: 5558 diff changeset	464 if (values == null \|\| from == MAX_RANGE \|\| from == MIN_RANGE) {
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	465 return;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	466 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	467
7375 450582bcad75 Replaced tabs with 4 spaces. Sascha L. Teichmann <teichmann@intevation.de> parents: 7374 diff changeset	468 // expand single-line i.e. 0-lenght Q-range to minimal length
450582bcad75 Replaced tabs with 4 spaces. Sascha L. Teichmann <teichmann@intevation.de> parents: 7374 diff changeset	469 if (from == to) {
450582bcad75 Replaced tabs with 4 spaces. Sascha L. Teichmann <teichmann@intevation.de> parents: 7374 diff changeset	470 if (lastRange != null && lastA > lastB) {
450582bcad75 Replaced tabs with 4 spaces. Sascha L. Teichmann <teichmann@intevation.de> parents: 7374 diff changeset	471 to = new BigDecimal(from.doubleValue() - INTERVAL_GAP);
450582bcad75 Replaced tabs with 4 spaces. Sascha L. Teichmann <teichmann@intevation.de> parents: 7374 diff changeset	472 }
450582bcad75 Replaced tabs with 4 spaces. Sascha L. Teichmann <teichmann@intevation.de> parents: 7374 diff changeset	473 else {
450582bcad75 Replaced tabs with 4 spaces. Sascha L. Teichmann <teichmann@intevation.de> parents: 7374 diff changeset	474 to = new BigDecimal(from.doubleValue() + INTERVAL_GAP);
450582bcad75 Replaced tabs with 4 spaces. Sascha L. Teichmann <teichmann@intevation.de> parents: 7374 diff changeset	475 }
450582bcad75 Replaced tabs with 4 spaces. Sascha L. Teichmann <teichmann@intevation.de> parents: 7374 diff changeset	476 }
7002 0850a4983fcd Issue1333: expand single-line i.e. 0-lenght Q-range to minimal length. Tom Gottfried <tom.gottfried@intevation.de> parents: 6819 diff changeset	477
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	478 ImportRange range = new ImportRange(from, to);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	479
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	480 // little workaround to make the q ranges tightly fit.
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	481 // Leave a very small gap to ensure that the range queries
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	482 // still work.
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	483
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	484 if (lastRange != null) {
5541 b09c095a0b7e WST Importer: Potential fix for tighten gaps between Q ranges. Sascha L. Teichmann <teichmann@intevation.de> parents: 3662 diff changeset	485 double a2 = range.getA().doubleValue();
7222 c5214e04b9b8 WST-parser: making ranges tightly fit was broken since a<b was enforced. Re-enabled. Tom Gottfried <tom@intevation.de> parents: 7003 diff changeset	486 double b2 = range.getB().doubleValue();
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	487
7222 c5214e04b9b8 WST-parser: making ranges tightly fit was broken since a<b was enforced. Re-enabled. Tom Gottfried <tom@intevation.de> parents: 7003 diff changeset	488 if (lastA < lastB) {
5541 b09c095a0b7e WST Importer: Potential fix for tighten gaps between Q ranges. Sascha L. Teichmann <teichmann@intevation.de> parents: 3662 diff changeset	489 lastRange.setB(new BigDecimal(a2 - INTERVAL_GAP));
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	490 }
7222 c5214e04b9b8 WST-parser: making ranges tightly fit was broken since a<b was enforced. Re-enabled. Tom Gottfried <tom@intevation.de> parents: 7003 diff changeset	491 else { // lastA >= lastB
c5214e04b9b8 WST-parser: making ranges tightly fit was broken since a<b was enforced. Re-enabled. Tom Gottfried <tom@intevation.de> parents: 7003 diff changeset	492 lastRange.setA(new BigDecimal(b2 + INTERVAL_GAP));
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	493 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	494 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	495
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	496 for (int i = 0; i < values.length; ++i) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	497 ImportWstColumn column = wst.getColumn(i);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	498 ImportWstQRange wstQRange = new ImportWstQRange(range, values[i]);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	499 column.addColumnQRange(wstQRange);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	500 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	501
7222 c5214e04b9b8 WST-parser: making ranges tightly fit was broken since a<b was enforced. Re-enabled. Tom Gottfried <tom@intevation.de> parents: 7003 diff changeset	502 lastA = from.doubleValue();
c5214e04b9b8 WST-parser: making ranges tightly fit was broken since a<b was enforced. Re-enabled. Tom Gottfried <tom@intevation.de> parents: 7003 diff changeset	503 lastB = to.doubleValue();
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	504 lastRange = range;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	505 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	506
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	507 private static final BigDecimal [] parseLineAsDouble(
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	508 String line,
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	509 int count,
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	510 boolean bStation,
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	511 boolean bParseEmptyAsZero
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	512 ) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	513 String [] tokens = parseLine(line, count, bStation);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	514
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	515 BigDecimal [] doubles = new BigDecimal[tokens.length];
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	516
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	517 for (int i = 0; i < doubles.length; ++i) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	518 String token = tokens[i].trim();
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	519 if (token.length() != 0) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	520 doubles[i] = new BigDecimal(token);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	521 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	522 else if (bParseEmptyAsZero) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	523 doubles[i] = UNDEFINED_ZERO;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	524 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	525 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	526
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	527 return doubles;
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	528 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	529
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	530 private static String [] parseLine(
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	531 String line,
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	532 int tokenCount,
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	533 boolean bParseStation
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	534 ) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	535 ArrayList<String> strings = new ArrayList<String>();
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	536
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	537 if (bParseStation) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	538 if (line.length() < 8) {
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	539 throw new IllegalArgumentException("station too short");
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	540 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	541 strings.add(line.substring(0, 8));
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	542 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	543
7481 bdb12632c5f5 WST-Parser and doc: reject files with wrong number of columns. Tom Gottfried <tom@intevation.de> parents: 7480 diff changeset	544 int pos = 0;
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	545 for (int i = 0; i < tokenCount; ++i) {
7481 bdb12632c5f5 WST-Parser and doc: reject files with wrong number of columns. Tom Gottfried <tom@intevation.de> parents: 7480 diff changeset	546 pos += 9;
bdb12632c5f5 WST-Parser and doc: reject files with wrong number of columns. Tom Gottfried <tom@intevation.de> parents: 7480 diff changeset	547 if (pos >= line.length()) {
bdb12632c5f5 WST-Parser and doc: reject files with wrong number of columns. Tom Gottfried <tom@intevation.de> parents: 7480 diff changeset	548 throw new IllegalArgumentException(
bdb12632c5f5 WST-Parser and doc: reject files with wrong number of columns. Tom Gottfried <tom@intevation.de> parents: 7480 diff changeset	549 "WST: number of columns is less than expected. File rejected.");
bdb12632c5f5 WST-Parser and doc: reject files with wrong number of columns. Tom Gottfried <tom@intevation.de> parents: 7480 diff changeset	550 }
7480 2a2e89c01588 WST-Parser: Improve handling of line-endings. Tom Gottfried <tom@intevation.de> parents: 7375 diff changeset	551 strings.add(line.substring(pos,
2a2e89c01588 WST-Parser: Improve handling of line-endings. Tom Gottfried <tom@intevation.de> parents: 7375 diff changeset	552 Math.min(pos + 8, line.length())));
1211 f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	553 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	554
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	555 return strings.toArray(new String[strings.size()]);
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	556 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	557 }
f08fe480092c Moved file parsers to separate package. Sascha L. Teichmann <sascha.teichmann@intevation.de> parents: diff changeset	558 // vim:set ts=4 sw=4 si et sta sts=4 fenc=utf8 :

Mercurial > dive4elements > river

annotate backend/src/main/java/org/dive4elements/river/importer/parsers/WstParser.java @ 7481:bdb12632c5f5