dive4elements/river: backend/src/main/java/org/dive4elements/river/importer/common/AbstractParser.java annotate

annotate backend/src/main/java/org/dive4elements/river/importer/common/AbstractParser.java @ 9650:a2a42a6bac6b

Importer (s/u-info) extensions: outer try/catch for parse and log of line no, catching parsing exception if not enough value fields, parsing error and warning log messages with line number, detecting and rejecting duplicate data series, better differentiation between error and warning log messages

author	mschaefer
date	Mon, 23 Mar 2020 14:57:03 +0100
parents	ddebd4c2fe93
children

rev	line source
8971 50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	1 /* Copyright (C) 2017 by Bundesanstalt für Gewässerkunde
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	2 * Software engineering by
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	3 * Björnsen Beratende Ingenieure GmbH
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	4 * Dr. Schumacher Ingenieurbüro für Wasser und Umwelt
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	5 *
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	6 * This file is Free Software under the GNU AGPL (>=v3)
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	7 * and comes with ABSOLUTELY NO WARRANTY! Check out the
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	8 * documentation coming with Dive4Elements River for details.
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	9 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	10
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	11 package org.dive4elements.river.importer.common;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	12
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	13 import java.io.File;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	14 import java.io.FileInputStream;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	15 import java.io.FilenameFilter;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	16 import java.io.InputStreamReader;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	17 import java.io.LineNumberReader;
8989 2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits mschaefer parents: 8971 diff changeset	18 import java.math.BigDecimal;
8991 c43d8c1a4455 Parse via decimalformat instead of bigdecimal constructor (to cope with strings with chars after the actual number) mschaefer parents: 8989 diff changeset	19 import java.text.DecimalFormat;
8971 50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	20 import java.text.NumberFormat;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	21 import java.text.ParseException;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	22 import java.util.ArrayList;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	23 import java.util.List;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	24 import java.util.Locale;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	25 import java.util.TreeSet;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	26 import java.util.regex.Matcher;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	27 import java.util.regex.Pattern;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	28
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	29 import org.apache.log4j.Logger;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	30 import org.dive4elements.river.backend.utils.EpsilonComparator;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	31 import org.dive4elements.river.importer.ImportRiver;
9650 a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	32 import org.dive4elements.river.importer.ImporterSession;
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	33 import org.hibernate.Session;
8971 50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	34
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	35 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	36 * Abstract base class for a parser of one FLYS csv data file.<br />
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	37 * The {@link parse} method creates a SERIES object for the meta data
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	38 * and a list of KMLINE objects for the km value lines read from the file.<br />
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	39 * The {@link store} method gets or creates the corresponding database objects
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	40 * by the hibernate binding classes DB_SERIES and DB_KMTUPLE,
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	41 * and updates or inserts them in the database.
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	42 * DB_SERIES has a one-to-many relationship with DB_KMTUPLE.<br />
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	43 * <br />
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	44 * The structure of the file is as follows:<br />
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	45 * <ul>
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	46 * <li>one or more comment lines (#) with the meta info of the data series</li>
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	47 * <li>the comment line with the column titles of values table, starting with the km column</li>
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	48 * <li>the rows of the values table, each one on its own line</li>
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	49 * </ul>
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	50 *
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	51 * @author Matthias Schäfer
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	52 *
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	53 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	54 public abstract class AbstractParser<DB_SERIES, DB_KMTUPLE, KMLINE extends AbstractKmLineImport<DB_SERIES, DB_KMTUPLE>, HEADER extends AbstractSeriesImport<DB_SERIES, DB_KMTUPLE, KMLINE>> implements ImportParser {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	55
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	56 /*** FIELDS ***/
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	57
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	58 public static final String ENCODING = "ISO-8859-1";
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	59
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	60 protected static final Locale DEFAULT_LOCALE = Locale.GERMAN;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	61
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	62 public static final String START_META_CHAR = "#";
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	63
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	64 protected static final String SEPARATOR_CHAR = ";";
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	65
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	66 protected static final Pattern META_RIVERNAME = Pattern.compile("^#\\s((Gew.sser)\|(Gewaesser)):\\s(\\S[^;]).", Pattern.CASE_INSENSITIVE);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	67
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	68 protected static final Pattern META_KMRANGE_INFO = Pattern.compile("^#\\sStrecke:\\s(\\S[^;]).", Pattern.CASE_INSENSITIVE);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	69
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	70 protected static final Pattern META_COMMENTS = Pattern.compile("^#\\sweitere Bemerkungen:\\s(\\S[^;]).", Pattern.CASE_INSENSITIVE);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	71
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	72 private static final Pattern META_COLUMNTITLES = Pattern.compile("^#\\sFluss.km\\s*;.+", Pattern.CASE_INSENSITIVE);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	73
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	74 private static final Pattern META_SUBGROUP = Pattern.compile("^##.*", Pattern.CASE_INSENSITIVE);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	75
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	76 private static NumberFormat numberFormat = NumberFormat.getInstance(Locale.ROOT);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	77
8991 c43d8c1a4455 Parse via decimalformat instead of bigdecimal constructor (to cope with strings with chars after the actual number) mschaefer parents: 8989 diff changeset	78 private static DecimalFormat bigDecimalFormat;
c43d8c1a4455 Parse via decimalformat instead of bigdecimal constructor (to cope with strings with chars after the actual number) mschaefer parents: 8989 diff changeset	79
9650 a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	80 protected static final String INVALID_VALUE_ERROR_FORMAT = "Invalid or missing %s value";
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	81
8991 c43d8c1a4455 Parse via decimalformat instead of bigdecimal constructor (to cope with strings with chars after the actual number) mschaefer parents: 8989 diff changeset	82 static {
c43d8c1a4455 Parse via decimalformat instead of bigdecimal constructor (to cope with strings with chars after the actual number) mschaefer parents: 8989 diff changeset	83 bigDecimalFormat = (DecimalFormat) NumberFormat.getInstance(Locale.ROOT);
c43d8c1a4455 Parse via decimalformat instead of bigdecimal constructor (to cope with strings with chars after the actual number) mschaefer parents: 8989 diff changeset	84 bigDecimalFormat.setParseBigDecimal(true);
c43d8c1a4455 Parse via decimalformat instead of bigdecimal constructor (to cope with strings with chars after the actual number) mschaefer parents: 8989 diff changeset	85 }
c43d8c1a4455 Parse via decimalformat instead of bigdecimal constructor (to cope with strings with chars after the actual number) mschaefer parents: 8989 diff changeset	86
8971 50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	87 /**
9012 abe069eb180c Import of vegetation zones added mschaefer parents: 8991 diff changeset	88 * How the km column and its content are expected
abe069eb180c Import of vegetation zones added mschaefer parents: 8991 diff changeset	89 */
abe069eb180c Import of vegetation zones added mschaefer parents: 8991 diff changeset	90 protected enum KmMode {
abe069eb180c Import of vegetation zones added mschaefer parents: 8991 diff changeset	91 NONE, UNIQUE, DUPLICATES
abe069eb180c Import of vegetation zones added mschaefer parents: 8991 diff changeset	92 }
abe069eb180c Import of vegetation zones added mschaefer parents: 8991 diff changeset	93
abe069eb180c Import of vegetation zones added mschaefer parents: 8991 diff changeset	94 /**
8971 50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	95 * Path of the file or directory to import from
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	96 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	97 protected final File importPath;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	98
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	99 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	100 * Part of {@link importPath} without the river root dir
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	101 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	102 protected final File rootRelativePath;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	103
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	104 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	105 * River for which the import runs
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	106 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	107 protected final ImportRiver river;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	108
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	109 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	110 * Reader during parse
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	111 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	112 protected LineNumberReader in;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	113
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	114 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	115 * Last line read from in
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	116 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	117 protected String currentLine;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	118
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	119 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	120 * State of the header lines parse loop
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	121 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	122 protected ParsingState headerParsingState;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	123
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	124 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	125 * Series header of the stations table, with the imported meta info.
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	126 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	127 protected HEADER seriesHeader;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	128
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	129 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	130 * List of meta info Pattern matched during {@link handleMetaLine}
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	131 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	132 protected final List<Pattern> metaPatternsMatched;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	133
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	134 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	135 * Column titles of the stations table, starting with the km column.
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	136 * All strings have been trimmed.
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	137 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	138 protected final List<String> columnTitles;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	139
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	140 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	141 * List of the km value tuples imported, no duplicate km
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	142 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	143 protected final List<KMLINE> values;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	144
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	145 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	146 * Ordered list with the imported km to check for duplicates.
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	147 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	148 protected final TreeSet<Double> kmExists;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	149
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	150
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	151 /*** CONSTRUCTORS ***/
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	152
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	153 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	154 * Constructs a parser for an import file
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	155 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	156 public AbstractParser(final File importPath, final File rootRelativePath, final ImportRiver river) {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	157 this.importPath = importPath;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	158 this.rootRelativePath = rootRelativePath;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	159 this.river = river;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	160 this.metaPatternsMatched = new ArrayList<>();
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	161 this.kmExists = new TreeSet<>(EpsilonComparator.CMP);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	162 this.columnTitles = new ArrayList<>();
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	163 this.values = new ArrayList<>();
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	164 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	165
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	166
9650 a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	167 /*** FILE-METHODS ***/
8971 50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	168
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	169 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	170 * Lists all files from a directory having a type extension (starting with dot)
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	171 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	172 protected static List<File> listFiles(final File importDir, final String extension) {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	173 final File[] files = importDir.listFiles(new FilenameFilter() {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	174 @Override
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	175 public boolean accept(final File dir, final String name) {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	176 return name.toLowerCase().endsWith(extension);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	177 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	178 });
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	179 final List<File> fl = new ArrayList<>();
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	180 if (files != null)
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	181 for (final File file : files)
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	182 fl.add(file);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	183 return fl;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	184 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	185
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	186 /**
9650 a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	187 * Lists all files from a directory matching a file name pattern
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	188 */
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	189 protected static List<File> listFiles(final File importDir, final Pattern filenamePattern) {
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	190 final File[] files = importDir.listFiles(new FilenameFilter() {
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	191 @Override
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	192 public boolean accept(final File dir, final String name) {
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	193 return filenamePattern.matcher(name).matches();
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	194 }
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	195 });
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	196 final List<File> fl = new ArrayList<>();
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	197 if (files != null)
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	198 for (final File file : files)
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	199 fl.add(file);
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	200 return fl;
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	201 }
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	202
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	203 /*** PARSE-METHODS ***/
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	204
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	205 /**
8971 50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	206 * Parses a file and adds series and values to the parser's collection
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	207 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	208 @Override
9650 a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	209 public void parse() throws Exception {
8971 50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	210 logStartInfo();
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	211 this.seriesHeader = createSeriesImport(this.importPath.getName().replaceAll("\\.csv", ""));
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	212 this.metaPatternsMatched.clear();
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	213 this.kmExists.clear();
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	214 this.headerParsingState = ParsingState.CONTINUE;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	215 try {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	216 try {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	217 this.in = new LineNumberReader(new InputStreamReader(new FileInputStream(this.importPath), ENCODING));
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	218 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	219 catch (final Exception e) {
9650 a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	220 logError("Could not open (%s)", e.getMessage());
8971 50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	221 this.headerParsingState = ParsingState.STOP;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	222 }
9650 a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	223 try {
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	224 this.currentLine = null;
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	225 while (this.headerParsingState != ParsingState.STOP) {
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	226 this.currentLine = this.in.readLine();
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	227 if (this.currentLine == null)
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	228 break;
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	229 this.currentLine = this.currentLine.trim();
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	230 if (this.currentLine.isEmpty())
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	231 continue;
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	232 if (this.headerParsingState == ParsingState.CONTINUE) {
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	233 handleMetaLine();
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	234 if (this.headerParsingState == ParsingState.DONE)
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	235 checkMetaData();
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	236 }
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	237 else
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	238 handleDataLine();
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	239 }
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	240 if (this.headerParsingState != ParsingState.STOP)
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	241 getLog().info(String.format("Number of values found: %d", this.seriesHeader.getValueCount()));
8971 50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	242 }
9650 a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	243 catch (final Exception e) {
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	244 throw new Exception(String.format("Parsing error (last read line: %d)", this.in.getLineNumber() + 1), e);
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	245 }
8971 50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	246 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	247 finally {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	248 if (this.in != null) {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	249 this.in.close();
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	250 this.in = null;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	251 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	252 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	253 if (this.headerParsingState == ParsingState.STOP)
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	254 logError("Parsing of the file stopped due to a severe error");
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	255 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	256
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	257 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	258 * Writes the parse start info to the log
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	259 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	260 protected void logStartInfo() {
9650 a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	261 getLog().info(String.format("Start parsing:;'%s'", this.rootRelativePath));
8971 50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	262 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	263
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	264 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	265 * Strips separator chars from a meta info text, and trims leading and trailing whitespace
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	266 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	267 public static String parseMetaInfo(final String text) {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	268 return text.replace(SEPARATOR_CHAR, "").trim();
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	269 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	270
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	271 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	272 * Parses a number string with dot or comma as decimal char, and returning null in case of an error
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	273 */
9650 a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	274 public static Number parseDoubleCheckNull(final String[] values, final int index) {
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	275 if (index > values.length - 1)
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	276 return null;
8971 50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	277 try {
9650 a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	278 return parseDouble(values[index]);
8971 50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	279 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	280 catch (final Exception e) {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	281 return null;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	282 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	283 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	284
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	285 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	286 * Parses a number string with dot or comma as decimal char
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	287 *
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	288 * @throws ParseException
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	289 */
9650 a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	290 private static Number parseDouble(final String text) throws ParseException {
8971 50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	291 return numberFormat.parse(text.replace(',', '.'));
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	292 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	293
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	294 /**
9650 a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	295 * Parses an integer number string , and returning null in case of an error
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	296 */
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	297 public static Integer parseIntegerCheckNull(final String[] values, final int index) {
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	298 if (index > values.length - 1)
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	299 return null;
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	300 try {
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	301 return Integer.valueOf((values[index]));
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	302 }
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	303 catch (final Exception e) {
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	304 return null;
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	305 }
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	306 }
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	307
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	308 /**
8989 2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits mschaefer parents: 8971 diff changeset	309 * Parses a number string as a BigDecimal, replacing a comma with a dot first
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits mschaefer parents: 8971 diff changeset	310 */
8991 c43d8c1a4455 Parse via decimalformat instead of bigdecimal constructor (to cope with strings with chars after the actual number) mschaefer parents: 8989 diff changeset	311 public static BigDecimal parseDecimal(final String text) throws ParseException {
c43d8c1a4455 Parse via decimalformat instead of bigdecimal constructor (to cope with strings with chars after the actual number) mschaefer parents: 8989 diff changeset	312 return (BigDecimal) bigDecimalFormat.parse(text.replace(',', '.'));
8989 2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits mschaefer parents: 8971 diff changeset	313 }
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits mschaefer parents: 8971 diff changeset	314
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits mschaefer parents: 8971 diff changeset	315 /**
8971 50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	316 * Creates a new series import object
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	317 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	318 protected abstract HEADER createSeriesImport(final String filename);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	319
9650 a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	320
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	321 /*** METAHEADER-PARSE-METHODS ***/
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	322
8971 50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	323 protected void handleMetaLine() {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	324 if (META_SUBGROUP.matcher(this.currentLine).matches())
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	325 return;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	326 else if (handleMetaRivername())
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	327 return;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	328 else if (handleMetaKmrange_info())
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	329 return;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	330 else if (handleMetaComment())
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	331 return;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	332 else if (handleMetaOther())
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	333 return;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	334 else if (handleMetaColumnTitles()) {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	335 if (this.headerParsingState != ParsingState.STOP)
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	336 this.headerParsingState = ParsingState.DONE;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	337 return;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	338 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	339 else {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	340 if (this.currentLine.startsWith(START_META_CHAR)) {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	341 if (this.headerParsingState != ParsingState.IGNORE)
9650 a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	342 logLineWarning("Not matching any known meta type");
8971 50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	343 else
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	344 this.headerParsingState = ParsingState.CONTINUE;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	345 }
9650 a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	346 else
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	347 this.headerParsingState = ParsingState.DONE; // no more meta data expected, if neither meta line nor empty line
8971 50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	348 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	349 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	350
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	351 private boolean handleMetaRivername() {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	352 if (META_RIVERNAME.matcher(this.currentLine).matches()) {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	353 this.metaPatternsMatched.add(META_RIVERNAME);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	354 return true;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	355 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	356 else
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	357 return false;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	358 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	359
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	360 private boolean handleMetaKmrange_info() {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	361 final Matcher m = META_KMRANGE_INFO.matcher(this.currentLine);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	362 if (m.matches()) {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	363 this.metaPatternsMatched.add(META_KMRANGE_INFO);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	364 this.seriesHeader.setKmrange_info(parseMetaInfo(m.group(1)));
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	365 return true;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	366 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	367 return false;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	368 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	369
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	370 private boolean handleMetaComment() {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	371 final Matcher m = META_COMMENTS.matcher(this.currentLine);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	372 if (m.matches()) {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	373 this.metaPatternsMatched.add(META_COMMENTS);
9038 4c5eeaff554c Database column "comment" renamed to "notes" (restrictions in Oracle) mschaefer parents: 9012 diff changeset	374 this.seriesHeader.setNotes(parseMetaInfo(m.group(1)));
8971 50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	375 return true;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	376 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	377 return false;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	378 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	379
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	380 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	381 * Parses currentLine for non-default meta info
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	382 *
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	383 * @return Whether the line has been handled
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	384 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	385 protected boolean handleMetaOther() {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	386 return false;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	387 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	388
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	389 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	390 * Parses a header line for the km table column header line
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	391 *
9650 a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	392 * @return Whether the line has been handled (also in case of State=STOP),<br>
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	393 * and we are ready for reading the km values lines (or cancel parsing)
8971 50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	394 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	395 protected boolean handleMetaColumnTitles() {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	396 if (META_COLUMNTITLES.matcher(this.currentLine).matches()) {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	397 this.metaPatternsMatched.add(META_COLUMNTITLES);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	398 this.columnTitles.clear();
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	399 final String[] titles = this.currentLine.split(SEPARATOR_CHAR, 0);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	400 for (int i = 0; i <= titles.length - 1; i++)
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	401 this.columnTitles.add(titles[i].trim());
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	402 return true;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	403 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	404 return false;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	405 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	406
9012 abe069eb180c Import of vegetation zones added mschaefer parents: 8991 diff changeset	407 /**
9650 a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	408 * Check meta data after all meta data lines (#) have been read
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	409 */
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	410 protected boolean checkMetaData() {
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	411 if (this.columnTitles.size() <= 1) {
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	412 logError("No valid header line with column titles found");
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	413 this.headerParsingState = ParsingState.STOP;
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	414 return false;
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	415 }
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	416 if (checkSeriesExistsAlready()) {
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	417 logError("Data series/filename exists already in the database");
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	418 this.headerParsingState = ParsingState.STOP;
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	419 return false;
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	420 }
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	421 return true;
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	422 }
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	423
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	424 /**
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	425 * Checks the existence of the active series in the database
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	426 */
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	427 protected boolean checkSeriesExistsAlready() {
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	428 if (!checkRiverExists())
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	429 return false;
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	430 final Session session = ImporterSession.getInstance().getDatabaseSession();
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	431 final List<DB_SERIES> rows = this.seriesHeader.querySeriesItem(session, this.river.getPeer(), true);
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	432 return !rows.isEmpty();
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	433 }
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	434
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	435 /**
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	436 * Checks the existence of the active river in the database
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	437 */
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	438 protected boolean checkRiverExists() {
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	439 return (this.river.getPeer(false) != null);
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	440 }
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	441
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	442
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	443 /*** VALUELINE-PARSE-METHODS ***/
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	444
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	445 /**
9012 abe069eb180c Import of vegetation zones added mschaefer parents: 8991 diff changeset	446 * Parses a values line and adds the values record
abe069eb180c Import of vegetation zones added mschaefer parents: 8991 diff changeset	447 */
abe069eb180c Import of vegetation zones added mschaefer parents: 8991 diff changeset	448 protected void handleDataLine() {
8971 50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	449 final String[] values = this.currentLine.split(SEPARATOR_CHAR, 0);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	450 // Skip import line without data or only km
9650 a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	451 if (values.length < 2) {
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	452 logLineWarning("Too few data");
8971 50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	453 return;
9650 a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	454 }
9012 abe069eb180c Import of vegetation zones added mschaefer parents: 8991 diff changeset	455 Double km = Double.NaN;
abe069eb180c Import of vegetation zones added mschaefer parents: 8991 diff changeset	456 if (kmMode() != KmMode.NONE) {
abe069eb180c Import of vegetation zones added mschaefer parents: 8991 diff changeset	457 try {
abe069eb180c Import of vegetation zones added mschaefer parents: 8991 diff changeset	458 km = Double.valueOf(parseDouble(values[0]).doubleValue());
abe069eb180c Import of vegetation zones added mschaefer parents: 8991 diff changeset	459 if (kmMode() == KmMode.UNIQUE) {
abe069eb180c Import of vegetation zones added mschaefer parents: 8991 diff changeset	460 if (this.kmExists.contains(km)) {
9650 a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	461 logLineWarning("Duplicate km '%s'", values[0]);
9012 abe069eb180c Import of vegetation zones added mschaefer parents: 8991 diff changeset	462 return;
abe069eb180c Import of vegetation zones added mschaefer parents: 8991 diff changeset	463 }
abe069eb180c Import of vegetation zones added mschaefer parents: 8991 diff changeset	464 this.kmExists.add(km);
8971 50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	465 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	466 }
9012 abe069eb180c Import of vegetation zones added mschaefer parents: 8991 diff changeset	467 catch (final Exception e) {
9650 a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	468 logLineWarning("Invalid km: %s", e.getMessage());
9012 abe069eb180c Import of vegetation zones added mschaefer parents: 8991 diff changeset	469 return;
abe069eb180c Import of vegetation zones added mschaefer parents: 8991 diff changeset	470 }
8971 50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	471 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	472 final KMLINE value = createKmLineImport(km, values);
9650 a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	473 if (value != null) {
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	474 final boolean added = this.seriesHeader.addValue(value);
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	475 if (!added)
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	476 logLineWarning("Duplicate data line");
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	477 }
8971 50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	478 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	479
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	480 /**
9012 abe069eb180c Import of vegetation zones added mschaefer parents: 8991 diff changeset	481 * How {@link handleDataLine} shall handle the km column (if any)
8971 50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	482 */
9012 abe069eb180c Import of vegetation zones added mschaefer parents: 8991 diff changeset	483 protected KmMode kmMode() {
abe069eb180c Import of vegetation zones added mschaefer parents: 8991 diff changeset	484 return KmMode.UNIQUE;
8971 50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	485 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	486
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	487 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	488 * Creates a value import item with the km and other fields of the current line;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	489 * the km has been validated
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	490 *
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	491 * @return value item, or null if parse error
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	492 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	493 protected abstract KMLINE createKmLineImport(final Double km, final String[] values);
9650 a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	494
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	495
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	496 /*** STORE-METHODS ***/
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	497
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	498 /**
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	499 * Stores the parsed series and values in the database
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	500 */
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	501 @Override
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	502 public void store() {
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	503 if (this.headerParsingState != ParsingState.STOP) {
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	504 this.seriesHeader.store(this.river.getPeer());
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	505 final String counts = String.format("parse=%d, insert=%d, update/ignore=%d", this.seriesHeader.getValueCount(),
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	506 this.seriesHeader.getValueStoreCount(StoreMode.INSERT), this.seriesHeader.getValueStoreCount(StoreMode.UPDATE));
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	507 if (this.seriesHeader.getValueCount() > this.seriesHeader.getValueStoreCount(StoreMode.INSERT))
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	508 logWarning("Number of value inserts less than number parsed: %s", counts);
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	509 else
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	510 getLog().info("Number of values records: " + counts);
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	511 }
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	512 else
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	513 logWarning("Severe parsing errors, not storing series '%s'", this.seriesHeader.getFilename());
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	514 }
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	515
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	516
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	517 /*** LOG-METHODS ***/
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	518
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	519 /**
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	520 * Gets the class's logger
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	521 */
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	522 protected abstract Logger getLog();
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	523
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	524 /**
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	525 * Logs an error message, appending the relative file path
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	526 */
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	527 protected void logError(final String message) {
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	528 getLog().error(buildLogMessage(message));
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	529 }
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	530
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	531 /**
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	532 * Logs an error message, appending the relative file path
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	533 */
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	534 protected void logError(final String format, final Object... args) {
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	535 getLog().error(buildLogMessage(String.format(format, args)));
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	536 }
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	537
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	538 /**
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	539 * Logs an error message with current line number, appending the relative file path
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	540 */
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	541 protected void logLineError(final String message) {
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	542 getLog().error(buildLineLogMessage(message));
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	543 }
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	544
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	545 /**
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	546 * Logs an error message with current line number, appending the relative file path
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	547 */
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	548 protected void logLineError(final String format, final Object... args) {
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	549 getLog().error(buildLineLogMessage(String.format(format, args)));
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	550 }
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	551
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	552 /**
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	553 * Logs a warning message, appending the relative file path
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	554 */
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	555 protected void logWarning(final String message) {
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	556 getLog().warn(buildLogMessage(message));
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	557 }
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	558
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	559 /**
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	560 * Logs a warning message, appending the relative file path
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	561 */
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	562 protected void logWarning(final String format, final Object... args) {
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	563 getLog().warn(buildLogMessage(String.format(format, args)));
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	564 }
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	565
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	566 /**
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	567 * Logs a warning message, appending the line number and the relative file path
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	568 */
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	569 protected void logLineWarning(final String message) {
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	570 getLog().warn(buildLineLogMessage(message));
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	571 }
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	572
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	573 /**
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	574 * Logs a warning message, appending the line number and the relative file path
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	575 */
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	576 protected void logLineWarning(final String format, final Object... args) {
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	577 getLog().warn(buildLineLogMessage(String.format(format, args)));
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	578 }
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	579
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	580 /**
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	581 * Logs an info message, appending the relative file path
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	582 */
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	583 protected void logInfo(final String message) {
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	584 getLog().info(buildLogMessage(message));
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	585 }
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	586
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	587 /**
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	588 * Logs a debug message, appending the relative file path
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	589 */
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	590 protected void logDebug(final String message) {
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	591 getLog().debug(buildLogMessage(message));
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	592 }
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	593
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	594 /**
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	595 * Logs a trace message, appending the relative file path
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	596 */
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	597 protected void logTrace(final String message) {
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	598 getLog().trace(buildLogMessage(message));
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	599 }
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	600
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	601 private String buildLogMessage(final String message) {
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	602 return String.format("%s;%s", message, this.rootRelativePath);
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	603 }
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	604
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	605 private String buildLineLogMessage(final String message) {
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	606 return String.format("Line %d: %s;%s", this.in.getLineNumber(), message, this.rootRelativePath);
a2a42a6bac6b Importer (s/u-info) extensions: mschaefer parents: 9056 diff changeset	607 }
8971 50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	608 }

Mercurial > dive4elements > river

annotate backend/src/main/java/org/dive4elements/river/importer/common/AbstractParser.java @ 9650:a2a42a6bac6b