Mercurial > dive4elements > river

annotate backend/src/main/java/org/dive4elements/river/importer/parsers/WaterlevelParser.java @ 8187:3bb1c62ad732

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Moved package org.dive4elements.river.utils to org.dive4elements.river.backend.utils.
author Sascha L. Teichmann <teichmann@intevation.de>
date Thu, 04 Sep 2014 15:03:25 +0200
parents 07cc4cd9233e
children 5e38e2924c07
rev   line source
5844
4dd33b86dc61 Added header to river backend.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
1 /* Copyright (C) 2011, 2012, 2013 by Bundesanstalt für Gewässerkunde
4dd33b86dc61 Added header to river backend.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
2 * Software engineering by Intevation GmbH
4dd33b86dc61 Added header to river backend.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
3 *
5992
4c3ccf2b0304 Removed trailing whitespace.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5844
diff changeset
4 * This file is Free Software under the GNU AGPL (>=v3)
5844
4dd33b86dc61 Added header to river backend.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
5 * and comes with ABSOLUTELY NO WARRANTY! Check out the
5992
4c3ccf2b0304 Removed trailing whitespace.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5844
diff changeset
6 * documentation coming with Dive4Elements River for details.
5844
4dd33b86dc61 Added header to river backend.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
7 */
4dd33b86dc61 Added header to river backend.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
8
5829
18619c1e7c2a Repaired internal references.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5828
diff changeset
9 package org.dive4elements.river.importer.parsers;
2844
c4db0f75a94a Prepared the importer to import MINFO specific waterlevel values and added a stub for a parser.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
10
2845
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
11 import java.io.File;
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
12 import java.io.IOException;
5049
b5e3df1feeb4 WaterlevelParser: Add Wst-Converting functionality.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
13 import java.math.BigDecimal;
2845
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
14 import java.text.NumberFormat;
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
15 import java.text.ParseException;
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
16 import java.util.ArrayList;
2844
c4db0f75a94a Prepared the importer to import MINFO specific waterlevel values and added a stub for a parser.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
17 import java.util.List;
2845
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
18 import java.util.regex.Matcher;
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
19 import java.util.regex.Pattern;
2844
c4db0f75a94a Prepared the importer to import MINFO specific waterlevel values and added a stub for a parser.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
20
2845
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
21 import org.apache.log4j.Logger;
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
22
6319
550cb41d37d9 issue1354: TimeIntervals in Waterlevels from csv files.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 6282
diff changeset
23 import org.dive4elements.river.importer.ImportTimeInterval;
5829
18619c1e7c2a Repaired internal references.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5828
diff changeset
24 import org.dive4elements.river.importer.ImportUnit;
2844
c4db0f75a94a Prepared the importer to import MINFO specific waterlevel values and added a stub for a parser.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
25
5829
18619c1e7c2a Repaired internal references.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5828
diff changeset
26 import org.dive4elements.river.importer.ImportRange;
18619c1e7c2a Repaired internal references.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5828
diff changeset
27 import org.dive4elements.river.importer.ImportWst;
18619c1e7c2a Repaired internal references.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5828
diff changeset
28 import org.dive4elements.river.importer.ImportWstColumn;
18619c1e7c2a Repaired internal references.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5828
diff changeset
29 import org.dive4elements.river.importer.ImportWstColumnValue;
18619c1e7c2a Repaired internal references.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5828
diff changeset
30 import org.dive4elements.river.importer.ImportWstQRange;
8187
3bb1c62ad732 Moved package org.dive4elements.river.utils to org.dive4elements.river.backend.utils.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7927
diff changeset
31 import org.dive4elements.river.backend.utils.DateUtil;
2844
c4db0f75a94a Prepared the importer to import MINFO specific waterlevel values and added a stub for a parser.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
32
5049
b5e3df1feeb4 WaterlevelParser: Add Wst-Converting functionality.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
33
b5e3df1feeb4 WaterlevelParser: Add Wst-Converting functionality.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
34 /**
b5e3df1feeb4 WaterlevelParser: Add Wst-Converting functionality.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
35 * Parse CSV Waterlevel files.
b5e3df1feeb4 WaterlevelParser: Add Wst-Converting functionality.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
36 * As these waterlevels are probably used in fixation analysis
b5e3df1feeb4 WaterlevelParser: Add Wst-Converting functionality.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
37 * only, functionality to export them to "fixation"-wsts
b5e3df1feeb4 WaterlevelParser: Add Wst-Converting functionality.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
38 * has been added (the ImportWaterlevel*- stuff is actually
b5e3df1feeb4 WaterlevelParser: Add Wst-Converting functionality.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
39 * not needed to do so.)
b5e3df1feeb4 WaterlevelParser: Add Wst-Converting functionality.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5016
diff changeset
40 */
2844
c4db0f75a94a Prepared the importer to import MINFO specific waterlevel values and added a stub for a parser.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
41 public class WaterlevelParser extends LineParser {
c4db0f75a94a Prepared the importer to import MINFO specific waterlevel values and added a stub for a parser.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
42
2845
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
43 private static final Logger log = Logger.getLogger(WaterlevelParser.class);
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
44
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
45 private static final NumberFormat nf =
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
46 NumberFormat.getInstance(DEFAULT_LOCALE);
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
47
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
48 private static final Pattern META_Q_RANGE =
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
49 Pattern.compile("Abfluss\\s\\[(.*)\\];(.*)");
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
50
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
51 public static final Pattern META_UNIT =
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
52 Pattern.compile("^Einheit: \\[(.*)\\].*");
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
53
7377
ad2fdc34910a Importer: Avoid 0-lenght Q-ranges in WST-CSV-parsers.
Tom Gottfried <tom@intevation.de>
parents: 6319
diff changeset
54 public static final double INTERVAL_GAP = 0.00001d;
ad2fdc34910a Importer: Avoid 0-lenght Q-ranges in WST-CSV-parsers.
Tom Gottfried <tom@intevation.de>
parents: 6319
diff changeset
55
5240
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
56 private List<ImportWst> waterlevels;
2844
c4db0f75a94a Prepared the importer to import MINFO specific waterlevel values and added a stub for a parser.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
57
5240
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
58 private ImportWst current;
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
59
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
60 /** The Waterlevel-Wst s will always have but one column. */
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
61 private ImportWstColumn column;
2845
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
62
5240
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
63 /** The current (incomplete) Q Range. */
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
64 private ImportWstQRange currentQRange;
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
65
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
66 /** The current (incomplete) km range for Q Range. */
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
67 private ImportRange currentRange;
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
68
2845
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
69 private String currentDescription;
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
70
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
71
2844
c4db0f75a94a Prepared the importer to import MINFO specific waterlevel values and added a stub for a parser.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
72 public WaterlevelParser() {
5240
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
73 waterlevels = new ArrayList<ImportWst>();
2844
c4db0f75a94a Prepared the importer to import MINFO specific waterlevel values and added a stub for a parser.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
74 }
c4db0f75a94a Prepared the importer to import MINFO specific waterlevel values and added a stub for a parser.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
75
c4db0f75a94a Prepared the importer to import MINFO specific waterlevel values and added a stub for a parser.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
76
5240
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
77 public List<ImportWst> getWaterlevels() {
2844
c4db0f75a94a Prepared the importer to import MINFO specific waterlevel values and added a stub for a parser.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
78 return waterlevels;
c4db0f75a94a Prepared the importer to import MINFO specific waterlevel values and added a stub for a parser.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
79 }
c4db0f75a94a Prepared the importer to import MINFO specific waterlevel values and added a stub for a parser.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
80
c4db0f75a94a Prepared the importer to import MINFO specific waterlevel values and added a stub for a parser.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
81
c4db0f75a94a Prepared the importer to import MINFO specific waterlevel values and added a stub for a parser.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
82 @Override
2845
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
83 public void parse(File file) throws IOException {
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
84 currentDescription = file.getName();
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
85
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
86 super.parse(file);
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
87 }
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
88
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
89
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
90 @Override
2844
c4db0f75a94a Prepared the importer to import MINFO specific waterlevel values and added a stub for a parser.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
91 protected void reset() {
5240
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
92 currentQRange = null;
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
93 current = new ImportWst(currentDescription);
5264
731030b69c7e Fix Waterlevel*-Parser to correctly register the wst-columns.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5247
diff changeset
94 current.setNumberColumns(1);
731030b69c7e Fix Waterlevel*-Parser to correctly register the wst-columns.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5247
diff changeset
95 column = current.getColumn(0);
731030b69c7e Fix Waterlevel*-Parser to correctly register the wst-columns.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5247
diff changeset
96 column.setName(currentDescription);
731030b69c7e Fix Waterlevel*-Parser to correctly register the wst-columns.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5247
diff changeset
97 column.setDescription(currentDescription);
6319
550cb41d37d9 issue1354: TimeIntervals in Waterlevels from csv files.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 6282
diff changeset
98
550cb41d37d9 issue1354: TimeIntervals in Waterlevels from csv files.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 6282
diff changeset
99 // Try to extract and set the TimeInterval.
550cb41d37d9 issue1354: TimeIntervals in Waterlevels from csv files.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 6282
diff changeset
100 Matcher m = WaterlevelDifferencesParser.YEARS_IN_COLUMN.matcher(currentDescription);
550cb41d37d9 issue1354: TimeIntervals in Waterlevels from csv files.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 6282
diff changeset
101
550cb41d37d9 issue1354: TimeIntervals in Waterlevels from csv files.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 6282
diff changeset
102 if (m.matches()) {
550cb41d37d9 issue1354: TimeIntervals in Waterlevels from csv files.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 6282
diff changeset
103 int startYear = Integer.parseInt(m.group(1));
550cb41d37d9 issue1354: TimeIntervals in Waterlevels from csv files.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 6282
diff changeset
104 int endYear = Integer.parseInt(m.group(2));
550cb41d37d9 issue1354: TimeIntervals in Waterlevels from csv files.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 6282
diff changeset
105 ImportTimeInterval time = new ImportTimeInterval(
7927
07cc4cd9233e Add new DateUtil class.
"Tom Gottfried <tom@intevation.de>"
parents: 7883
diff changeset
106 DateUtil.getStartDateFromYear(startYear),
07cc4cd9233e Add new DateUtil class.
"Tom Gottfried <tom@intevation.de>"
parents: 7883
diff changeset
107 DateUtil.getEndDateFromYear(endYear)
6319
550cb41d37d9 issue1354: TimeIntervals in Waterlevels from csv files.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 6282
diff changeset
108 );
550cb41d37d9 issue1354: TimeIntervals in Waterlevels from csv files.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 6282
diff changeset
109 column.setTimeInterval(time);
550cb41d37d9 issue1354: TimeIntervals in Waterlevels from csv files.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 6282
diff changeset
110 } else {
550cb41d37d9 issue1354: TimeIntervals in Waterlevels from csv files.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 6282
diff changeset
111 log.debug("No time interval in column header found: " + currentDescription);
550cb41d37d9 issue1354: TimeIntervals in Waterlevels from csv files.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 6282
diff changeset
112 }
550cb41d37d9 issue1354: TimeIntervals in Waterlevels from csv files.
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 6282
diff changeset
113
6282
62dca91ca229 Switch wst-kind of waterlevel/-differences (7 and 6).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5992
diff changeset
114 current.setKind(7);
2844
c4db0f75a94a Prepared the importer to import MINFO specific waterlevel values and added a stub for a parser.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
115 }
c4db0f75a94a Prepared the importer to import MINFO specific waterlevel values and added a stub for a parser.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
116
c4db0f75a94a Prepared the importer to import MINFO specific waterlevel values and added a stub for a parser.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
117
c4db0f75a94a Prepared the importer to import MINFO specific waterlevel values and added a stub for a parser.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
118 @Override
c4db0f75a94a Prepared the importer to import MINFO specific waterlevel values and added a stub for a parser.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
119 protected void finish() {
2845
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
120 if (current != null) {
5240
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
121 if (currentQRange != null) {
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
122 List<ImportWstColumnValue> cValues = column.getColumnValues();
7377
ad2fdc34910a Importer: Avoid 0-lenght Q-ranges in WST-CSV-parsers.
Tom Gottfried <tom@intevation.de>
parents: 6319
diff changeset
123 // Set end of range to last station
ad2fdc34910a Importer: Avoid 0-lenght Q-ranges in WST-CSV-parsers.
Tom Gottfried <tom@intevation.de>
parents: 6319
diff changeset
124 // or expand range to minimal length in case it would be 0
7883
fc8d5b5f2c99 Removed trailing whitespace.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7377
diff changeset
125 // TODO: should otherwise be extended to
fc8d5b5f2c99 Removed trailing whitespace.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7377
diff changeset
126 // (first station of next range - INTERVAL_GAP),
7377
ad2fdc34910a Importer: Avoid 0-lenght Q-ranges in WST-CSV-parsers.
Tom Gottfried <tom@intevation.de>
parents: 6319
diff changeset
127 // assuming always ascending stations
ad2fdc34910a Importer: Avoid 0-lenght Q-ranges in WST-CSV-parsers.
Tom Gottfried <tom@intevation.de>
parents: 6319
diff changeset
128 BigDecimal lastStation = cValues.get(cValues.size() -1).getPosition();
ad2fdc34910a Importer: Avoid 0-lenght Q-ranges in WST-CSV-parsers.
Tom Gottfried <tom@intevation.de>
parents: 6319
diff changeset
129 if (lastStation.compareTo(currentRange.getA()) == 0) {
7883
fc8d5b5f2c99 Removed trailing whitespace.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 7377
diff changeset
130 currentRange.setB(new BigDecimal(lastStation.doubleValue()
7377
ad2fdc34910a Importer: Avoid 0-lenght Q-ranges in WST-CSV-parsers.
Tom Gottfried <tom@intevation.de>
parents: 6319
diff changeset
131 + INTERVAL_GAP));
ad2fdc34910a Importer: Avoid 0-lenght Q-ranges in WST-CSV-parsers.
Tom Gottfried <tom@intevation.de>
parents: 6319
diff changeset
132 }
ad2fdc34910a Importer: Avoid 0-lenght Q-ranges in WST-CSV-parsers.
Tom Gottfried <tom@intevation.de>
parents: 6319
diff changeset
133 else {
ad2fdc34910a Importer: Avoid 0-lenght Q-ranges in WST-CSV-parsers.
Tom Gottfried <tom@intevation.de>
parents: 6319
diff changeset
134 currentRange.setB(lastStation);
ad2fdc34910a Importer: Avoid 0-lenght Q-ranges in WST-CSV-parsers.
Tom Gottfried <tom@intevation.de>
parents: 6319
diff changeset
135 }
ad2fdc34910a Importer: Avoid 0-lenght Q-ranges in WST-CSV-parsers.
Tom Gottfried <tom@intevation.de>
parents: 6319
diff changeset
136
5240
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
137 currentQRange.setRange(currentRange);
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
138 column.addColumnQRange(currentQRange);
2845
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
139 }
2844
c4db0f75a94a Prepared the importer to import MINFO specific waterlevel values and added a stub for a parser.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
140
2845
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
141 waterlevels.add(current);
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
142 }
2844
c4db0f75a94a Prepared the importer to import MINFO specific waterlevel values and added a stub for a parser.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
143 }
c4db0f75a94a Prepared the importer to import MINFO specific waterlevel values and added a stub for a parser.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
144
c4db0f75a94a Prepared the importer to import MINFO specific waterlevel values and added a stub for a parser.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
145 @Override
4193
f63b39799d2d Adapted DB schema (added relation measurement_station); improved importer to read files from filesystem with measurement stations.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2845
diff changeset
146 protected void handleLine(int lineNum, String line) {
2845
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
147 if (line.startsWith(START_META_CHAR)) {
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
148 handleMetaLine(stripMetaLine(line));
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
149 return;
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
150 }
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
151 else if (handleQRange(line)) {
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
152 return;
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
153 }
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
154 else {
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
155 handleDataLine(line);
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
156 return;
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
157 }
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
158 }
2844
c4db0f75a94a Prepared the importer to import MINFO specific waterlevel values and added a stub for a parser.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
159
2845
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
160
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
161 private void handleMetaLine(String meta) {
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
162 Matcher m = META_UNIT.matcher(meta);
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
163
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
164 if (m.matches()) {
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
165 String unit = m.group(1);
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
166 log.debug("Found unit: '" + unit + "'");
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
167
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
168 current.setUnit(new ImportUnit(unit));
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
169 }
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
170 }
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
171
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
172
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
173 private boolean handleQRange(String line) {
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
174 Matcher m = META_Q_RANGE.matcher(line);
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
175
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
176 if (m.matches()) {
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
177 String unitStr = m.group(1);
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
178 String valueStr = m.group(2);
5240
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
179 try {
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
180 if (currentQRange != null) {
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
181 // Finish off the last one.
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
182 List<ImportWstColumnValue> cValues = column.getColumnValues();
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
183 // Set end of range to last station.
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
184 currentRange.setB(cValues.get(cValues.size() -1).getPosition());
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
185 currentQRange.setRange(currentRange);
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
186 column.addColumnQRange(currentQRange);
2845
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
187 }
5240
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
188 currentQRange = new ImportWstQRange(null,
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
189 new BigDecimal(nf.parse(valueStr).doubleValue()));
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
190 currentRange = new ImportRange();
2845
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
191
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
192 log.debug("Found new Q range: Q=" + valueStr);
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
193
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
194 return true;
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
195 }
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
196 catch (ParseException pe) {
5490
b3dd14fc13a6 Parsers: do not call warnings error
Tom Gottfried <tom.gottfried@intevation.de>
parents: 5264
diff changeset
197 log.warn("Unparseable Q range: '" + line + "'");
2845
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
198 }
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
199 }
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
200
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
201 return false;
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
202 }
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
203
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
204
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
205 private void handleDataLine(String line) {
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
206 String[] cols = line.split(SEPERATOR_CHAR);
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
207
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
208 if (cols == null || cols.length < 2) {
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
209 log.warn("skip invalid waterlevel line: '" + line + "'");
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
210 return;
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
211 }
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
212
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
213 try {
5240
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
214 // Store the value and remember the position for QRange, if needed.
2845
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
215 Double station = nf.parse(cols[0]).doubleValue();
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
216 Double value = nf.parse(cols[1]).doubleValue();
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
217
5240
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
218 BigDecimal stationBD = new BigDecimal(station);
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
219
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
220 column.addColumnValue(stationBD, new BigDecimal(value));
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
221
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
222 if (currentRange.getA() == null) {
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
223 currentRange.setA(stationBD);
9c024718bffb WaterlevelParser: Use Wst-structures directly (instead of Waterlevel*).
Felix Wolfsteller <felix.wolfsteller@intevation.de>
parents: 5231
diff changeset
224 }
2845
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
225 }
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
226 catch (ParseException pe) {
5490
b3dd14fc13a6 Parsers: do not call warnings error
Tom Gottfried <tom.gottfried@intevation.de>
parents: 5264
diff changeset
227 log.warn("Unparseable number in data row: " + line);
2845
a65b5b0ade6f Completed the MINFO waterlevel import.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 2844
diff changeset
228 }
2844
c4db0f75a94a Prepared the importer to import MINFO specific waterlevel values and added a stub for a parser.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
229 }
c4db0f75a94a Prepared the importer to import MINFO specific waterlevel values and added a stub for a parser.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
230 }
c4db0f75a94a Prepared the importer to import MINFO specific waterlevel values and added a stub for a parser.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents:
diff changeset
231 // vim:set ts=4 sw=4 si et sta sts=4 fenc=utf8 :
http://dive4elements.wald.intevation.org