dive4elements/river: backend/src/main/java/org/dive4elements/river/importer/common/AbstractParser.java annotate

annotate backend/src/main/java/org/dive4elements/river/importer/common/AbstractParser.java @ 8991:c43d8c1a4455

Parse via decimalformat instead of bigdecimal constructor (to cope with strings with chars after the actual number)

author	mschaefer
date	Tue, 10 Apr 2018 16:43:32 +0200
parents	2693bfaf503d
children	abe069eb180c

rev	line source
8971 50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	1 /* Copyright (C) 2017 by Bundesanstalt für Gewässerkunde
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	2 * Software engineering by
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	3 * Björnsen Beratende Ingenieure GmbH
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	4 * Dr. Schumacher Ingenieurbüro für Wasser und Umwelt
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	5 *
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	6 * This file is Free Software under the GNU AGPL (>=v3)
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	7 * and comes with ABSOLUTELY NO WARRANTY! Check out the
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	8 * documentation coming with Dive4Elements River for details.
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	9 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	10
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	11 package org.dive4elements.river.importer.common;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	12
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	13 import java.io.File;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	14 import java.io.FileInputStream;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	15 import java.io.FilenameFilter;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	16 import java.io.IOException;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	17 import java.io.InputStreamReader;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	18 import java.io.LineNumberReader;
8989 2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits mschaefer parents: 8971 diff changeset	19 import java.math.BigDecimal;
8991 c43d8c1a4455 Parse via decimalformat instead of bigdecimal constructor (to cope with strings with chars after the actual number) mschaefer parents: 8989 diff changeset	20 import java.text.DecimalFormat;
8971 50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	21 import java.text.NumberFormat;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	22 import java.text.ParseException;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	23 import java.util.ArrayList;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	24 import java.util.List;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	25 import java.util.Locale;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	26 import java.util.TreeSet;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	27 import java.util.regex.Matcher;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	28 import java.util.regex.Pattern;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	29
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	30 import org.apache.log4j.Logger;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	31 import org.dive4elements.river.backend.utils.EpsilonComparator;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	32 import org.dive4elements.river.importer.ImportRiver;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	33
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	34 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	35 * Abstract base class for a parser of one FLYS csv data file.<br />
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	36 * The {@link parse} method creates a SERIES object for the meta data
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	37 * and a list of KMLINE objects for the km value lines read from the file.<br />
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	38 * The {@link store} method gets or creates the corresponding database objects
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	39 * by the hibernate binding classes DB_SERIES and DB_KMTUPLE,
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	40 * and updates or inserts them in the database.
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	41 * DB_SERIES has a one-to-many relationship with DB_KMTUPLE.<br />
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	42 * <br />
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	43 * The structure of the file is as follows:<br />
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	44 * <ul>
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	45 * <li>one or more comment lines (#) with the meta info of the data series</li>
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	46 * <li>the comment line with the column titles of values table, starting with the km column</li>
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	47 * <li>the rows of the values table, each one on its own line</li>
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	48 * </ul>
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	49 *
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	50 * @author Matthias Schäfer
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	51 *
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	52 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	53 public abstract class AbstractParser<DB_SERIES, DB_KMTUPLE, KMLINE extends AbstractKmLineImport<DB_SERIES, DB_KMTUPLE>, HEADER extends AbstractSeriesImport<DB_SERIES, DB_KMTUPLE, KMLINE>> implements ImportParser {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	54
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	55 /*** FIELDS ***/
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	56
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	57 public static final String ENCODING = "ISO-8859-1";
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	58
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	59 protected static final Locale DEFAULT_LOCALE = Locale.GERMAN;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	60
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	61 public static final String START_META_CHAR = "#";
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	62
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	63 protected static final String SEPARATOR_CHAR = ";";
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	64
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	65 protected static final Pattern META_RIVERNAME = Pattern.compile("^#\\s((Gew.sser)\|(Gewaesser)):\\s(\\S[^;]).", Pattern.CASE_INSENSITIVE);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	66
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	67 protected static final Pattern META_KMRANGE_INFO = Pattern.compile("^#\\sStrecke:\\s(\\S[^;]).", Pattern.CASE_INSENSITIVE);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	68
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	69 protected static final Pattern META_COMMENTS = Pattern.compile("^#\\sweitere Bemerkungen:\\s(\\S[^;]).", Pattern.CASE_INSENSITIVE);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	70
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	71 private static final Pattern META_COLUMNTITLES = Pattern.compile("^#\\sFluss.km\\s*;.+", Pattern.CASE_INSENSITIVE);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	72
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	73 private static final Pattern META_SUBGROUP = Pattern.compile("^##.*", Pattern.CASE_INSENSITIVE);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	74
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	75 private static NumberFormat numberFormat = NumberFormat.getInstance(Locale.ROOT);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	76
8991 c43d8c1a4455 Parse via decimalformat instead of bigdecimal constructor (to cope with strings with chars after the actual number) mschaefer parents: 8989 diff changeset	77 private static DecimalFormat bigDecimalFormat;
c43d8c1a4455 Parse via decimalformat instead of bigdecimal constructor (to cope with strings with chars after the actual number) mschaefer parents: 8989 diff changeset	78
c43d8c1a4455 Parse via decimalformat instead of bigdecimal constructor (to cope with strings with chars after the actual number) mschaefer parents: 8989 diff changeset	79 static {
c43d8c1a4455 Parse via decimalformat instead of bigdecimal constructor (to cope with strings with chars after the actual number) mschaefer parents: 8989 diff changeset	80 bigDecimalFormat = (DecimalFormat) NumberFormat.getInstance(Locale.ROOT);
c43d8c1a4455 Parse via decimalformat instead of bigdecimal constructor (to cope with strings with chars after the actual number) mschaefer parents: 8989 diff changeset	81 bigDecimalFormat.setParseBigDecimal(true);
c43d8c1a4455 Parse via decimalformat instead of bigdecimal constructor (to cope with strings with chars after the actual number) mschaefer parents: 8989 diff changeset	82 }
c43d8c1a4455 Parse via decimalformat instead of bigdecimal constructor (to cope with strings with chars after the actual number) mschaefer parents: 8989 diff changeset	83
8971 50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	84 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	85 * Path of the file or directory to import from
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	86 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	87 protected final File importPath;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	88
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	89 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	90 * Part of {@link importPath} without the river root dir
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	91 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	92 protected final File rootRelativePath;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	93
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	94 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	95 * River for which the import runs
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	96 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	97 protected final ImportRiver river;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	98
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	99 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	100 * Reader during parse
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	101 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	102 protected LineNumberReader in;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	103
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	104 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	105 * Last line read from in
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	106 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	107 protected String currentLine;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	108
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	109 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	110 * State of the header lines parse loop
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	111 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	112 protected ParsingState headerParsingState;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	113
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	114 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	115 * Series header of the stations table, with the imported meta info.
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	116 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	117 protected HEADER seriesHeader;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	118
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	119 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	120 * List of meta info Pattern matched during {@link handleMetaLine}
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	121 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	122 protected final List<Pattern> metaPatternsMatched;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	123
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	124 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	125 * Column titles of the stations table, starting with the km column.
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	126 * All strings have been trimmed.
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	127 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	128 protected final List<String> columnTitles;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	129
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	130 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	131 * List of the km value tuples imported, no duplicate km
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	132 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	133 protected final List<KMLINE> values;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	134
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	135 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	136 * Ordered list with the imported km to check for duplicates.
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	137 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	138 protected final TreeSet<Double> kmExists;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	139
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	140
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	141 /*** CONSTRUCTORS ***/
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	142
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	143 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	144 * Constructs a parser for an import file
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	145 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	146 public AbstractParser(final File importPath, final File rootRelativePath, final ImportRiver river) {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	147 this.importPath = importPath;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	148 this.rootRelativePath = rootRelativePath;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	149 this.river = river;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	150 this.metaPatternsMatched = new ArrayList<>();
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	151 this.kmExists = new TreeSet<>(EpsilonComparator.CMP);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	152 this.columnTitles = new ArrayList<>();
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	153 this.values = new ArrayList<>();
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	154 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	155
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	156
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	157 /*** METHODS ***/
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	158
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	159 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	160 * Lists all files from a directory having a type extension (starting with dot)
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	161 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	162 protected static List<File> listFiles(final File importDir, final String extension) {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	163 final File[] files = importDir.listFiles(new FilenameFilter() {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	164 @Override
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	165 public boolean accept(final File dir, final String name) {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	166 return name.toLowerCase().endsWith(extension);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	167 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	168 });
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	169 final List<File> fl = new ArrayList<>();
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	170 if (files != null)
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	171 for (final File file : files)
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	172 fl.add(file);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	173 return fl;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	174 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	175
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	176 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	177 * Parses a file and adds series and values to the parser's collection
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	178 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	179 @Override
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	180 public void parse() throws IOException {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	181 logStartInfo();
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	182 this.seriesHeader = createSeriesImport(this.importPath.getName().replaceAll("\\.csv", ""));
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	183 this.metaPatternsMatched.clear();
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	184 this.kmExists.clear();
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	185 this.headerParsingState = ParsingState.CONTINUE;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	186 try {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	187 try {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	188 this.in = new LineNumberReader(new InputStreamReader(new FileInputStream(this.importPath), ENCODING));
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	189 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	190 catch (final Exception e) {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	191 logError("Could not open (" + e.getMessage() + ")");
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	192 this.headerParsingState = ParsingState.STOP;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	193 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	194 this.currentLine = null;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	195 while (this.headerParsingState != ParsingState.STOP) {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	196 this.currentLine = this.in.readLine();
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	197 if (this.currentLine == null)
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	198 break;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	199 this.currentLine = this.currentLine.trim();
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	200 if (this.currentLine.isEmpty())
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	201 continue;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	202 if (this.headerParsingState == ParsingState.CONTINUE)
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	203 handleMetaLine();
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	204 else
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	205 handleDataLine();
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	206 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	207 if (this.headerParsingState != ParsingState.STOP)
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	208 getLog().info("Number of values found: " + this.seriesHeader.getValueCount());
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	209 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	210 finally {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	211 if (this.in != null) {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	212 this.in.close();
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	213 this.in = null;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	214 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	215 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	216 if (this.headerParsingState == ParsingState.STOP)
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	217 logError("Parsing of the file stopped due to a severe error");
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	218 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	219
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	220 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	221 * Writes the parse start info to the log
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	222 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	223 protected void logStartInfo() {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	224 getLog().info("Start parsing:;'" + this.rootRelativePath + "'");
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	225 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	226
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	227 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	228 * Stores the parsed series and values in the database
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	229 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	230 @Override
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	231 public void store() {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	232 if (this.headerParsingState != ParsingState.STOP) {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	233 this.seriesHeader.store(this.river.getPeer());
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	234 final String counts = String.format("parse=%d, insert=%d, update/ignore=%d", this.seriesHeader.getValueCount(),
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	235 this.seriesHeader.getValueStoreCount(StoreMode.INSERT), this.seriesHeader.getValueStoreCount(StoreMode.UPDATE));
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	236 if (this.seriesHeader.getValueCount() > this.seriesHeader.getValueStoreCount(StoreMode.INSERT))
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	237 logWarning("Number of value inserts less than number parsed: " + counts);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	238 else
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	239 getLog().info("Number of values records: " + counts);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	240 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	241 else
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	242 logWarning("Severe parsing errors, not storing series '" + this.seriesHeader.getFilename() + "'");
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	243 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	244
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	245 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	246 * Strips separator chars from a meta info text, and trims leading and trailing whitespace
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	247 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	248 public static String parseMetaInfo(final String text) {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	249 return text.replace(SEPARATOR_CHAR, "").trim();
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	250 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	251
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	252 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	253 * Parses a number string with dot or comma as decimal char, and returning null in case of an error
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	254 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	255 public static Number parseDoubleWithNull(final String text) {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	256 try {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	257 return parseDouble(text);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	258 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	259 catch (final Exception e) {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	260 return null;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	261 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	262 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	263
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	264 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	265 * Parses a number string with dot or comma as decimal char
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	266 *
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	267 * @throws ParseException
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	268 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	269 public static Number parseDouble(final String text) throws ParseException {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	270 return numberFormat.parse(text.replace(',', '.'));
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	271 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	272
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	273 /**
8989 2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits mschaefer parents: 8971 diff changeset	274 * Parses a number string as a BigDecimal, replacing a comma with a dot first
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits mschaefer parents: 8971 diff changeset	275 */
8991 c43d8c1a4455 Parse via decimalformat instead of bigdecimal constructor (to cope with strings with chars after the actual number) mschaefer parents: 8989 diff changeset	276 public static BigDecimal parseDecimal(final String text) throws ParseException {
c43d8c1a4455 Parse via decimalformat instead of bigdecimal constructor (to cope with strings with chars after the actual number) mschaefer parents: 8989 diff changeset	277 return (BigDecimal) bigDecimalFormat.parse(text.replace(',', '.'));
8989 2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits mschaefer parents: 8971 diff changeset	278 }
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits mschaefer parents: 8971 diff changeset	279
2693bfaf503d Fixed several BigDecimal(double) creations by BigDecimal(String) parsing to avoid unnecessary decimal digits mschaefer parents: 8971 diff changeset	280 /**
8971 50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	281 * Gets the class's logger
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	282 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	283 protected abstract Logger getLog();
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	284
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	285 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	286 * Logs an error message, appending the relative file path
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	287 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	288 protected void logError(final String message) {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	289 getLog().error(message + ";" + this.rootRelativePath);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	290 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	291
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	292 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	293 * Logs a warning message, appending the relative file path
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	294 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	295 protected void logWarning(final String message) {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	296 getLog().warn(message + ";" + this.rootRelativePath);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	297 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	298
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	299 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	300 * Creates a new series import object
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	301 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	302 protected abstract HEADER createSeriesImport(final String filename);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	303
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	304 protected void handleMetaLine() {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	305 if (META_SUBGROUP.matcher(this.currentLine).matches())
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	306 return;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	307 else if (handleMetaRivername())
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	308 return;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	309 else if (handleMetaKmrange_info())
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	310 return;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	311 else if (handleMetaComment())
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	312 return;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	313 else if (handleMetaOther())
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	314 return;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	315 else if (handleMetaColumnTitles()) {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	316 if (this.headerParsingState != ParsingState.STOP)
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	317 this.headerParsingState = ParsingState.DONE;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	318 return;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	319 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	320 else {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	321 if (this.currentLine.startsWith(START_META_CHAR)) {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	322 if (this.headerParsingState != ParsingState.IGNORE)
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	323 logWarning("Not matching any known meta type in line " + this.in.getLineNumber() + ", ignored");
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	324 else
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	325 this.headerParsingState = ParsingState.CONTINUE;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	326 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	327 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	328 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	329
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	330 private boolean handleMetaRivername() {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	331 if (META_RIVERNAME.matcher(this.currentLine).matches()) {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	332 this.metaPatternsMatched.add(META_RIVERNAME);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	333 return true;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	334 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	335 else
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	336 return false;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	337 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	338
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	339 private boolean handleMetaKmrange_info() {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	340 final Matcher m = META_KMRANGE_INFO.matcher(this.currentLine);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	341 if (m.matches()) {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	342 this.metaPatternsMatched.add(META_KMRANGE_INFO);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	343 this.seriesHeader.setKmrange_info(parseMetaInfo(m.group(1)));
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	344 return true;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	345 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	346 return false;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	347 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	348
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	349 private boolean handleMetaComment() {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	350 final Matcher m = META_COMMENTS.matcher(this.currentLine);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	351 if (m.matches()) {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	352 this.metaPatternsMatched.add(META_COMMENTS);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	353 this.seriesHeader.setComment(parseMetaInfo(m.group(1)));
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	354 return true;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	355 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	356 return false;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	357 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	358
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	359 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	360 * Parses currentLine for non-default meta info
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	361 *
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	362 * @return Whether the line has been handled
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	363 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	364 protected boolean handleMetaOther() {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	365 return false;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	366 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	367
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	368 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	369 * Parses a header line for the km table column header line
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	370 *
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	371 * @return Whether the line has been handled and we are ready for reading the km values lines
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	372 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	373 protected boolean handleMetaColumnTitles() {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	374 if (META_COLUMNTITLES.matcher(this.currentLine).matches()) {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	375 this.metaPatternsMatched.add(META_COLUMNTITLES);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	376 this.columnTitles.clear();
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	377 final String[] titles = this.currentLine.split(SEPARATOR_CHAR, 0);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	378 for (int i = 0; i <= titles.length - 1; i++)
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	379 this.columnTitles.add(titles[i].trim());
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	380 return true;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	381 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	382 return false;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	383 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	384
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	385 private void handleDataLine() {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	386 final String[] values = this.currentLine.split(SEPARATOR_CHAR, 0);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	387 // Skip import line without data or only km
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	388 if (values.length < 2)
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	389 return;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	390 Double km;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	391 try {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	392 km = Double.valueOf(parseDouble(values[0]).doubleValue());
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	393 if (kmMustBeUnique()) {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	394 if (this.kmExists.contains(km)) {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	395 logWarning("Ignoring duplicate station '" + values[0] + "' in line " + this.in.getLineNumber());
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	396 return;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	397 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	398 this.kmExists.add(km);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	399 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	400 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	401 catch (final Exception e) {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	402 logError("Not parseable km in line " + this.in.getLineNumber() + ": " + e.getMessage());
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	403 return;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	404 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	405 final KMLINE value = createKmLineImport(km, values);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	406 if (value != null)
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	407 this.seriesHeader.addValue(value);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	408 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	409
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	410 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	411 * Whether {@link handleDataLine} shall check for and reject km duplicates
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	412 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	413 protected boolean kmMustBeUnique() {
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	414 return true;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	415 }
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	416
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	417 /**
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	418 * Creates a value import item with the km and other fields of the current line;
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	419 * the km has been validated
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	420 *
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	421 * @return value item, or null if parse error
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	422 */
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	423 protected abstract KMLINE createKmLineImport(final Double km, final String[] values);
50416a0df385 Importer for the Schifffahrt (S-INFO) and Oekologie (U-INFO) files mschaefer parents: diff changeset	424 }

Mercurial > dive4elements > river

annotate backend/src/main/java/org/dive4elements/river/importer/common/AbstractParser.java @ 8991:c43d8c1a4455