Mercurial > dive4elements > river
view flys-backend/ChangeLog @ 769:321b9e480f72
Added a method to river to find the gauge with the max overlap with a given interval.
flys-backend/trunk@2235 c6561f87-3c4e-4783-a992-168aeb5c3f6f
| author | Sascha L. Teichmann <sascha.teichmann@intevation.de> |
|---|---|
| date | Sun, 26 Jun 2011 16:13:08 +0000 |
| parents | 87ac543c7ab9 |
| children | 33a79496ba9e |
line wrap: on
line source
2011-06-26 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/model/River.java: Added a method maxOverlap to determine the gauge which has the max common length to a given interval. This is for numerical stability in slightly overlapping gauge ranges. 2011-06-26 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/model/River.java: When looking up a gauge by km containment use precision of 1e-6. 2011-06-22 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/model/River.java: Added method to find gauge by its name. 2011-06-19 Sascha L. Teichmann <sascha.teichmann@intevation.de> * doc/annotation-types.xml: Fixed some rules. * src/main/java/de/intevation/flys/importer/Importer.java, src/main/java/de/intevation/flys/importer/InfoGewParser.java, src/main/java/de/intevation/flys/importer/ImportRiver.java: Call the right constructors now. * src/main/java/de/intevation/flys/importer/ImportAnnotationType.java: Added getter/setter for name property. * src/main/java/de/intevation/flys/importer/AnnotationsParser.java: Print duplicated at WARN level not as DEBUG. * src/main/java/de/intevation/flys/importer/AnnotationClassifier.java: Fixed XPath expression to build the internal lookup structures correctly. 2011-06-19 Sascha L. Teichmann <sascha.teichmann@intevation.de> Added classification of annotation types. Needs testing! * doc/annotation-types.xml: New. Rules to classify the different types of annotations. The classification works like this: There are unique types like 'Bruecke', 'Pegel' and so on. They are defined in the /annotation/type section and identified by their name. One of the types can be set as the default type if no rule applies. In the /annotation/pattern section are two types of pattern. 1 - file pattern: If a KM file is opened its filename is matched against the regular expressions of these patterns. If a match is found the corresponding type is used as the default type in the open file. If no match is found the global default type is used as the default type. 2 - line patterns: For each line of an open KM file these patterns are applied to find a match. If a match is found the corresponding type is used as the type of the annotation. If no match is found the default file default is assumed to be the right type. For the file default see 1. * src/main/java/de/intevation/flys/importer/Importer.java: To activate the annotation type classification set the system property 'flys.backend.importer.annotation.types' to the path of a XML looking like the annotation-types.xml file. If the system property is not set no classification is done. * src/main/java/de/intevation/flys/importer/AnnotationClassifier.java: New. Implements the classification. * src/main/java/de/intevation/flys/importer/AnnotationsParser.java, src/main/java/de/intevation/flys/importer/InfoGewParser.java, src/main/java/de/intevation/flys/importer/ImportRiver.java: Looped through the annotation type classification. 2011-06-18 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/model/River.java: Added method to find gauge by a position lying in its range. 2011-06-14 Sascha L. Teichmann <sascha.teichmann@intevation.de> First part of flys/issue18 * doc/schema/postgresql.sql: Add new table 'annotation_types'. To update existing databases: BEGIN; CREATE SEQUENCE ANNOTATION_TYPES_ID_SEQ; CREATE TABLE annotation_types ( id int PRIMARY KEY NOT NULL, name VARCHAR(256) NOT NULL UNIQUE ); ALTER TABLE annotations ADD COLUMN type_id int REFERENCES annotation_types(id); COMMIT; * doc/schema/postgresql-cleanup.sql: Removed. Hopeless out-dated. * src/main/java/de/intevation/flys/model/AnnotationType.java: New. Hibernate model to access the type. * src/main/java/de/intevation/flys/backend/SessionFactoryProvider.java: Register the new backend type. * src/main/java/de/intevation/flys/model/Annotation.java: References the annotation type. * src/main/java/de/intevation/flys/importer/ImportAnnotationType.java: New. Model to help import the annotation type. * src/main/java/de/intevation/flys/importer/ImportAnnotation.java: Uses the import type. * src/main/java/de/intevation/flys/importer/AnnotationsParser.java: Set the annotation type to 'null'. TODO: Do the classification! 2011-06-14 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/AtFileParser.java: Fix for flys/issue110. start index was shifted by one. 2011-06-14 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/ImportAnnotation.java: Forgot to store reference to edge. 2011-06-14 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/backend/SessionFactoryProvider.java: Register backend model. * src/main/java/de/intevation/flys/importer/ImportEdge.java: New. Model for importing the edges. * src/main/java/de/intevation/flys/importer/AnnotationsParser.java: Parses the edges of an annotation, too. 2011-06-14 Sascha L. Teichmann <sascha.teichmann@intevation.de> * doc/schema/postgresql.sql: Added table edges to model 'Kanten' of an annotation. To update existing databases: BEGIN; CREATE SEQUENCE EDGES_ID_SEQ; CREATE TABLE edges ( id int PRIMARY KEY NOT NULL, top NUMERIC, bottom NUMERIC); ALTER TABLE annotations ADD COLUMN edge_id int REFERENCES edges(id); COMMIT; * src/main/java/de/intevation/flys/model/Edge.java: New. A 'Kanten'-Model. * src/main/java/de/intevation/flys/model/Annotation.java: References the edges. 2011-06-08 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/model/River.java: Added method to find gauge only by station position. 2011-05-24 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/model/River.java: Make search for gauges independent of from/to order. 2011-05-24 Sascha L. Teichmann <sascha.teichmann@intevation.de> * doc/schema/postgresql.sql: Added a new view to select qs of a WST. To update existing databases: CREATE VIEW wst_q_values AS SELECT wc.position AS column_pos, wqr.q AS q, r.a AS a, r.b AS b, wc.wst_id AS wst_id FROM wst_column_q_ranges wcqr JOIN wst_q_ranges wqr ON wcqr.wst_q_range_id = wqr.id JOIN ranges r ON wqr.range_id = r.id JOIN wst_columns wc ON wcqr.wst_column_id = wc.id ORDER BY wc.position, wcqr.wst_column_id, r.a; 2011-05-24 Sascha L. Teichmann <sascha.teichmann@intevation.de> * doc/schema/postgresql.sql: Added a new view to select ws of a WST. To update existing databases: CREATE VIEW wst_w_values AS SELECT wcv."position" AS km, wcv.w AS w, wc."position" AS column_pos, w.id AS wst_id FROM wst_column_values wcv JOIN wst_columns wc ON wcv.wst_column_id = wc.id JOIN wsts w ON wc.wst_id = w.id ORDER BY wcv."position", wc."position"; 2011-05-23 Sascha L. Teichmann <sascha.teichmann@intevation.de> flys/issue76 * src/main/java/de/intevation/flys/importer/WstParser.java: Close gaps between q ranges. 2011-05-20 Ingo Weinzierl <ingo@intevation.de> Tagged RELEASE 2.3.1 2011-05-13 Ingo Weinzierl <ingo@intevation.de> Tagged RELEASE 2.3 2011-05-13 Ingo Weinzierl <ingo@intevation.de> * Changes: Changes for release 2.3 - see Changes file to get to know about the version numbers of this software. 2011-05-10 Sascha L. Teichmann <sascha.teichmann@intevation.de> * doc/schema/postgresql.sql: Added a column which tells if a river counts its km up or downwards. To update existing databases: ALTER TABLE rivers ADD COLUMN km_up BOOLEAN NOT NULL DEFAULT true; * src/main/java/de/intevation/flys/model/River.java: Adjust Hibernate mapping of new column. 2011-05-10 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/AtFileParser.java: Fixed flys/issue11 and flys51. 2011-05-09 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/ValueKey.java: Add some unsharp comparison (eps = 1e-6) to avoid numerical problems. * src/main/java/de/intevation/flys/importer/IdValueKey.java: Removed. Not need any longer. * src/main/java/de/intevation/flys/importer/ImporterSession.java: Use unsharp lookup. * src/main/java/de/intevation/flys/importer/ImportWst.java, src/main/java/de/intevation/flys/importer/ImportRiver.java: Flush more often. Hopefully this reduces hibernate sync problems?! 2011-05-09 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/ImporterSession.java: Fixed silly bug. Set flush mode back to auto because manual flushing causes some undetermined problems. 2011-05-09 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/ImporterSession.java: Do not load _all_ values from discharge tables and wst columns. This is extremly slow and will lead to OOM if more rivers are imported. Now only the last 20 columns und discharge tables are cached. * src/main/java/de/intevation/flys/importer/ValueKey.java: New. Key for caching discharge table values and wst column values. * src/main/java/de/intevation/flys/importer/IdValueKey.java: Fixed bug in equals(). * src/main/java/de/intevation/flys/importer/ImportWstColumn.java: Removed too eloquent debug output. 2011-05-09 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/IdValueKey.java: Use BigDecimals as representation for the numeric components which prevents running into unique constraint problems caused by imprecision. * src/main/java/de/intevation/flys/importer/ImportRange.java, src/main/java/de/intevation/flys/importer/ImporterSession.java: Ranges are now cached globally, too. * src/main/java/de/intevation/flys/importer/ImportWstColumn.java: Improved logging. * src/main/java/de/intevation/flys/importer/ImportDischargeTableValue.java: Removed superfluous imports. 2011-05-08 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/StaFileParser.java: Only accept main value types 'Q', 'W', 'D' and 'T' by default. '-' is not handled any more because it causes too much constraint problems. You can overwrite the imported type with the system property flys.backend.main.value.types" (default "QWTD") * src/main/java/de/intevation/flys/importer/ImporterSession.java: Set session flush mode to manual. Hopefully this improves the performance a bit. * src/main/java/de/intevation/flys/importer/ImportWst.java, src/main/java/de/intevation/flys/importer/ImportGauge.java, src/main/java/de/intevation/flys/importer/ImportWstColumn.java, src/main/java/de/intevation/flys/importer/ImportRange.java, src/main/java/de/intevation/flys/importer/ImportDischargeTable.java: Improved logging. 2011-05-08 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/WstColumnValueKey.java: Deleted * src/main/java/de/intevation/flys/importer/IdValueKey.java: Reinserted here in a more generalized form. * src/main/java/de/intevation/flys/importer/ImporterSession.java: Cache the discharge table value, too. * src/main/java/de/intevation/flys/importer/ImportDischargeTableValue.java: Use the global cache. 2011-05-08 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/ImporterSession.java: New. Centralized caching in the thread local context. Importing the Elbe leads to OOM because the column values of the WST files where loaded separately for every file. * src/main/java/de/intevation/flys/importer/ImportPosition.java, src/main/java/de/intevation/flys/importer/Importer.java, src/main/java/de/intevation/flys/importer/ImportAnnotation.java, src/main/java/de/intevation/flys/importer/ImportWstQRange.java, src/main/java/de/intevation/flys/importer/ImportWst.java, src/main/java/de/intevation/flys/importer/ImportMainValue.java, src/main/java/de/intevation/flys/importer/ImportMainValueType.java, src/main/java/de/intevation/flys/importer/ImportNamedMainValue.java, src/main/java/de/intevation/flys/importer/ImportRiver.java, src/main/java/de/intevation/flys/importer/ImportGauge.java, src/main/java/de/intevation/flys/importer/ImportWstColumnValue.java, src/main/java/de/intevation/flys/importer/ImportWstColumnQRange.java, src/main/java/de/intevation/flys/importer/ImportWstColumn.java, src/main/java/de/intevation/flys/importer/ImportRange.java, src/main/java/de/intevation/flys/importer/ImportDischargeTableValue.java, src/main/java/de/intevation/flys/importer/ImportDischargeTable.java, src/main/java/de/intevation/flys/importer/ImportAttribute.java, src/main/java/de/intevation/flys/importer/ImportTimeInterval.java: Adjusted to use the new global context. 2011-05-08 Sascha L. Teichmann <sascha.teichmann@intevation.de> * doc/schema/postgresql.sql: Dropped constraint which forces discharge tables to have a unique time interval for a given gauge and kind. There are AT files (historical Mosel/Perl/Perl/1967-1981.at and Mosel/Perl/1967-1981-1.at) which violate this constraint. Its a technical question to the customer how to handle these cases. To adjust existing databases: ALTER TABLE discharge_tables DROP CONSTRAINT discharge_tables_gauge_id_key; 2011-05-08 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/WstParser.java: There are wst files where column names are not unique. Make them unique by appending (1), (2) and so on. 2011-05-05 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/ImportRiver.java: Import the "HW-Schutzanlagen", too. 2011-05-05 Sascha L. Teichmann <sascha.teichmann@intevation.de> Make import of historical discharge tables work. * doc/schema/postgresql.sql: Added forgotten column 'description'. !!! You have to drop your database !!! * src/main/java/de/intevation/flys/importer/ImportDischargeTable.java: src/main/java/de/intevation/flys/model/DischargeTable.java: Add the forgotten description property. * src/main/java/de/intevation/flys/importer/AtFileParser.java: Fixed problems with date recognition. * src/main/java/de/intevation/flys/importer/ImportGauge.java: Prefix the description of the historical discharge tables with "Histor.Abflusstafeln". * src/main/java/de/intevation/flys/importer/ImportTimeInterval.java: Fixed silly programming error. 2011-05-05 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/AtFileParser.java: Inject 'kind' attribute. * src/main/java/de/intevation/flys/importer/ImportGauge.java: Traverse the "Histor.Abflusstafeln" for the historical discharge tables. too. TODO: Store them in the database. * src/main/java/de/intevation/flys/importer/ImportDischargeTable.java: Added convinience constructor to set the kind of the discharge table. 2011-05-05 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/ImportRiver.java: Prefix "Zusätzliche Längsschnitte" with "Zus.Längsschnitte" in description. 2011-05-05 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/ImportRiver.java: Import 'Hochwasser-Marken', too. * src/main/java/de/intevation/flys/importer/WstParser.java: Removed superfluous import. 2011-05-05 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/WstParser.java: Ignore lines that contain km positions which were found before in the same file. 2011-05-05 Ingo Weinzierl <ingo@intevation.de> * src/main/java/de/intevation/flys/model/River.java: Added a method that returns all gauges of the river intersected by a given start and end point. 2011-05-03 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/ImportTimeInterval.java: Forgot to fetch peer from result set. 2011-05-03 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/ImportTimeInterval.java: New. Importer model help fetching the database peer. * src/main/java/de/intevation/flys/model/TimeInterval.java: Add convinience constructor with start and stop time. * src/main/java/de/intevation/flys/importer/AtFileParser.java: Attach a time interval to a discharge table if we find one. * src/main/java/de/intevation/flys/importer/ImportDischargeTable.java: Store the reference to the importer model of the time interval of the discharge table. 2011-05-03 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/AtFileParser.java: Try to extract time ranges from at files. 2011-05-03 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/ImportRiver.java: Parse and store "amtliche Linien" wst files. 2011-05-03 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/ImportRiver.java: Parse and store fixation wst files as well. 2011-05-03 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/ImportRiver.java: Parse the "zusaetzliche Laengsschnitte", too. * src/main/java/de/intevation/flys/importer/ImportWst.java: Add getter/setter for column 'kind'. 2011-05-02 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/WstParser.java: Fixed flys/issue19: Do not take km column in wst file as a water level. 2011-05-02 Ingo Weinzierl <ingo@intevation.de> * src/main/java/de/intevation/flys/model/Gauge.java: Introduced a 'scale' that is used to adjust the range of min/max W values. 2011-05-01 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/WstParser.java: Removed superfluous import. 2011-04-20 Ingo Weinzierl <ingo@intevation.de> * src/main/java/de/intevation/flys/importer/WstColumnValueKey.java: New. This class is used as distinct key of a WstColumnValue - e.g. as key in a map. * src/main/java/de/intevation/flys/importer/ImportWst.java: A WstColumnValue cache is build up while initialization. This cache contains all WstColumnValues that exist in the database. * src/main/java/de/intevation/flys/importer/ImportWstColumn.java: New constructor that takes the WstColumnValues cache. It is used to initialize new ImportWstColumnValue objects. * src/main/java/de/intevation/flys/importer/ImportWstColumnValue.java: Speedup: A ImportWstColumnValue has a WstColumnValues cache that contains all WstColumnValues existing in the database. This makes it unnecessary to call an sql statement for each WstColumnValue to determine its existence in the database. 2011-04-18 Sascha L. Teichmann <sascha.teichmann@intevation.de> * doc/schema/postgresql.sql, doc/schema/postgresql-cleanup.sql: Added a view 'wst_value_table' which aggregates the data to build w/q value tables. To update existing databases: BEGIN; CREATE VIEW wst_value_table AS SELECT wcv.position AS position, w, (SELECT q FROM wst_column_q_ranges wcqr JOIN wst_q_ranges wqr ON wcqr.wst_q_range_id = wqr.id JOIN ranges r ON r.id = wqr.range_id WHERE wcqr.wst_column_id = wc.id AND wcv.position BETWEEN r.a AND r.b) AS q, wc.position AS column_pos, w.id AS wst_id FROM wst_column_values wcv JOIN wst_columns wc ON wcv.wst_column_id = wc.id JOIN wsts w ON wc.wst_id = w.id ORDER BY wcv.position ASC, wc.position DESC; COMMIT; 2011-04-18 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/model/Wst.java: Add forgotten one to many relation Wst -> WstColumn. 2011-04-18 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/ImportWst.java, src/main/java/de/intevation/flys/importer/WstParser.java: Import of q ranges of wst files was totally broken. :-/ You have to reimport all your data. 2011-04-18 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/ImportDischargeTable.java: Forgot kind parameter in peer fetching query. 2011-04-18 Ingo Weinzierl <ingo@intevation.de> * src/main/java/de/intevation/flys/model/River.java:: Added a new method to determine the gauge based on a given start and end point of the river. 2011-04-15 Sascha L. Teichmann <sascha.teichmann@intevation.de> * doc/schema/postgresql.sql: Added 'position' column to wst_columns to allow order them by there column position in the original wst file. Update existing database with: BEGIN; ALTER TABLE wst_columns ADD COLUMN position int; UPDATE wst_columns w SET position = id - (SELECT min(id) FROM wst_columns WHERE wst_id = w.wst_id); ALTER TABLE wst_columns ADD CONSTRAINT wst_columns_wst_id_position_key UNIQUE (wst_id, position); COMMIT; * src/main/java/de/intevation/flys/model/Wst.java, src/main/java/de/intevation/flys/model/WstColumn.java: Adjusted models. * src/main/java/de/intevation/flys/importer/ImportWst.java, src/main/java/de/intevation/flys/importer/ImportWstColumn.java: Adjusted importer. 2011-04-15 Sascha L. Teichmann <sascha.teichmann@intevation.de> * doc/schema/postgresql.sql: Forget ',' in schema. 2011-04-15 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/model/Wst.java (determineMinMaxQ): Fixed index problem when an empty list is returned. 2011-04-15 Ingo Weinzierl <ingo@intevation.de> * src/main/java/de/intevation/flys/model/Wst.java: A Wst is now able to return its min and max Q values. * src/main/java/de/intevation/flys/model/Gauge.java: A Gauge is now able to return its min and max W values. 2011-04-15 Sascha L. Teichmann <sascha.teichmann@intevation.de> * doc/schema/postgresql.sql: Added new column 'kind' in discharge tables and wst to distinguish between different kinds like 'Haupt-WST', 'zusaetzliche Laengsschnitte', 'amtliche Daten' and so on. Update existing databases with: BEGIN; ALTER TABLE discharge_tables ADD COLUMN kind int NOT NULL DEFAULT 0; ALTER TABLE wsts ADD COLUMN kind int NOT NULL DEFAULT 0; COMMIT; * src/main/java/de/intevation/flys/model/DischargeTable.java src/main/java/de/intevation/flys/model/Wst.java, src/main/java/de/intevation/flys/importer/ImportWst.java, src/main/java/de/intevation/flys/importer/ImportDischargeTable.java: Adjusted the models. 2011-04-15 Ingo Weinzierl <ingo@intevation.de> * src/main/java/de/intevation/flys/backend/SessionHolder.java: Moved to this module from flys-artifacts. This is necessary to get access to the current session in this module as well. 2011-04-14 Ingo Weinzierl <ingo@intevation.de> * src/main/java/de/intevation/flys/model/River.java: Added a method that returns the min and max distance of a river. 2011-04-03 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/**/*.java: Removed trailing whitespace. 2011-03-30 Ingo Weinzierl <ingo@intevation.de> Tagged RELEASE 0.1 2011-03-28 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/model/Range.java: Forgot to save the last change before commit. 2011-03-28 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/model/Range.java: Added methods to find out if two ranges intersects. 2011-03-24 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/model/Gauge.java: Added an one to many relation to the discharge tables of a gauge. 2011-03-22 Sascha L. Teichmann <sascha.teichmann@intevation.de> Finished import of WSTs. TODO 1: Speed it up! It takes on a high end machine over 7(!) minutes only for the data of the Saar. TODO 2: Double precision floating point representations produced by the the parsers leed to unique constraint violations in the backend on a second run. So the import is currently only working on freshly initialized data bases. More consequent working with BigDecimal and some rounding may be of help here. * src/main/java/de/intevation/flys/model/WstColumnValue.java: Added convinience constructors. * src/main/java/de/intevation/flys/importer/ImportWstColumnValue.java: Added getPeer() method. * src/main/java/de/intevation/flys/importer/ImportWstColumn.java: Add a list of the ImportWstColumnValues produced by the WST parser. * src/main/java/de/intevation/flys/importer/WstParser.java: Add the (km, w) values to the ImportWstColumns. 2011-03-22 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/WstParser.java: Build models for wsts, wst columns and q ranges and store them in the backend. TODO: store the w values. * src/main/java/de/intevation/flys/model/WstQRange.java src/main/java/de/intevation/flys/model/Wst.java, src/main/java/de/intevation/flys/model/Range.java, src/main/java/de/intevation/flys/model/WstColumnQRange.java, src/main/java/de/intevation/flys/model/WstColumn.java: Added convinience constructors. * src/main/java/de/intevation/flys/importer/ImportWstQRange.java, src/main/java/de/intevation/flys/importer/ImportWst.java, src/main/java/de/intevation/flys/importer/ImportRiver.java, src/main/java/de/intevation/flys/importer/ImportWstColumnQRange.java, src/main/java/de/intevation/flys/importer/ImportWstColumn.java, src/main/java/de/intevation/flys/importer/ImportRange.java: Added getPeer() methods. 2011-03-22 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/WstParser.java: The unit extraction in the WST parser of desktop FLYS is broken! Add a hack here to repair this for our importer. Desktop FLYS needs a fix, too! 2011-03-22 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/WstParser.java: Ported some stuff ver from WSTSource.java of desktop flys to parse WST files. TODO: create instances of the import models. * src/main/java/de/intevation/flys/utils/StringUtil.java: Copied from desktop flys. Used for some string operations in WST parser. * src/main/java/de/intevation/flys/importer/Importer.java: Added system property 'flys.backend.importer.dry.run'. Set to true only the parsing is done and no writing to the backend. Default: false. * src/main/java/de/intevation/flys/App.java, src/main/java/de/intevation/flys/model/MainValueType.java: Removed needless imports. 2011-03-22 Ingo Weinzierl <ingo@intevation.de> * src/main/java/de/intevation/flys/model/DischargeTableValue.java, src/main/java/de/intevation/flys/model/DischargeTable.java: Added new constructors. * src/main/java/de/intevation/flys/importer/AtFileParser.java: New. This parser is used to '*.at' files. * src/main/java/de/intevation/flys/importer/ImportGauge.java: Added code to import discharge tables. * src/main/java/de/intevation/flys/importer/ImportDischargeTableValue.java, src/main/java/de/intevation/flys/importer/ImportDischargeTable.java: New. Helper models for import discharge tables. 2011-03-22 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/WstParser.java, src/main/java/de/intevation/flys/importer/ImportRiver.java: Added stub for WST parser. 2011-03-22 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/ImportWstQRange.java, src/main/java/de/intevation/flys/importer/ImportWst.java, src/main/java/de/intevation/flys/importer/ImportWstColumnValue.java, src/main/java/de/intevation/flys/importer/ImportWstColumnQRange.java, src/main/java/de/intevation/flys/importer/ImportWstColumn.java: Added importer helper model stubs for WST imports. 2011-03-21 Sascha L. Teichmann <sascha.teichmann@intevation.de> Second part of parsing/storing main values. Should be finished now. * src/main/java/de/intevation/flys/importer/ImportNamedMainValue.java, src/main/java/de/intevation/flys/importer/ImportMainValue.java: New. Helper models for import main values, * src/main/java/de/intevation/flys/model/MainValue.java, src/main/java/de/intevation/flys/model/NamedMainValue.java: Added convinience constructors. * src/main/java/de/intevation/flys/importer/ImportGauge.java: Write main values to backend, too. * src/main/java/de/intevation/flys/importer/StaFileParser.java: Build importer models for main values. * src/main/java/de/intevation/flys/importer/ImportMainValueType.java: Data was called 'value'. Now it is 'name' to fit the schema. 2011-03-21 Sascha L. Teichmann <sascha.teichmann@intevation.de> * doc/schema/postgresql.sql: Fixed wrong unique constraint. * src/main/java/de/intevation/flys/importer/ImportRiver.java: Added some logging when storing gauges. 2011-03-21 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/model/Gauge.java: Add forgotten column river_id. * src/main/java/de/intevation/flys/importer/ImportGauge.java: Small HQL fix. 2011-03-21 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/model/Gauge.java: Added convinience contructor. * src/main/java/de/intevation/flys/importer/ImportGauge.java: Fixed getPeer() method. * src/main/java/de/intevation/flys/importer/StaFileParser.java: Fixed parsing of STA files. 2011-03-21 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/ImportRiver.java, src/main/java/de/intevation/flys/importer/ImportGauge.java: Propagate river into storing of gauges. 2011-03-21 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/Importer.java: Added code to store rivers not only annotations. * src/main/java/de/intevation/flys/importer/ImportRiver.java: Added stub code to write gauges. 2011-03-17 Sascha L. Teichmann <sascha.teichmann@intevation.de> First part of parsing main values. * src/main/java/de/intevation/flys/App.java: Commented out creation of dummy rivers. * src/main/java/de/intevation/flys/model/NamedMainValues.java: Moved to NamedMainValue. * src/main/java/de/intevation/flys/model/NamedMainValue.java: New. Formerly NamedMainValues. * src/main/java/de/intevation/flys/model/MainValue.java: New. Forgotten part of the model. * src/main/java/de/intevation/flys/model/MainValueType.java: Data is String not BigDecimal * src/main/java/de/intevation/flys/model/Range.java: Removed contructor with double arguments. Using BigDecimal now. * src/main/java/de/intevation/flys/importer/PegelGltParser.java: Propagate BigDecimal usage. * src/main/java/de/intevation/flys/importer/Importer.java: Removed needless import. Added TODO * src/main/java/de/intevation/flys/importer/ImportRiver.java: Parse the dependencies of the gauges, too. * src/main/java/de/intevation/flys/importer/StaFileParser.java: New. Parser for STA files. * src/main/java/de/intevation/flys/importer/ImportGauge.java: Call STA file parser. * src/main/java/de/intevation/flys/importer/AnnotationsParser.java, src/main/java/de/intevation/flys/importer/ImportRange.java: Uses BigDecimal now. * src/main/java/de/intevation/flys/importer/ImportAttribute.java: Fixed wrong type cast in equals. * src/main/java/de/intevation/flys/importer/ImportMainValueType.java: New. Helper model for importing main value types. * src/main/java/de/intevation/flys/backend/SessionFactoryProvider.java: Register forgotten MainValue model. 2011-03-17 Sascha L. Teichmann <sascha.teichmann@intevation.de> Store annotations in backend. * src/main/java/de/intevation/flys/model/Annotation.java: New convinience constructor. * src/main/java/de/intevation/flys/model/River.java: Added toString() method. * src/main/java/de/intevation/flys/model/Range.java: Fixed nasty mistake in @OneToOne annotatation. New convinience constructors. * src/main/java/de/intevation/flys/importer/ImportPosition.java src/main/java/de/intevation/flys/importer/ImportAnnotation.java, src/main/java/de/intevation/flys/importer/ImportRange.java src/main/java/de/intevation/flys/importer/ImportAttribute.java: Make storing to backend work. It's a bit too slow. :-/ * src/main/java/de/intevation/flys/importer/ImportRiver.java: Fetch peer from backend. Added method to store annotations. * src/main/java/de/intevation/flys/importer/Importer.java: Stored annotations into backend. More eloquent SQL exception handling. 2011-03-17 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/model/Attribute.java, src/main/java/de/intevation/flys/model/Position.java: Added convinience constructors. * src/main/java/de/intevation/flys/importer/ImportPosition.java, src/main/java/de/intevation/flys/importer/ImportAttribute.java: Bound them to there backend peers. 2011-03-17 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/AnnotationsParser.java: New. Added parser to read *.KM files. * src/main/java/de/intevation/flys/importer/ImportPosition.java, src/main/java/de/intevation/flys/importer/PegelGltParser.java, src/main/java/de/intevation/flys/importer/ImportRiver.java, src/main/java/de/intevation/flys/importer/ImportAnnotation.java, src/main/java/de/intevation/flys/importer/ImportRange.java, src/main/java/de/intevation/flys/importer/InfoGewParser.java, src/main/java/de/intevation/flys/importer/ImportAttribute.java: Adjusted to load the annotations from *.KM files. 2011-03-17 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/ImportPosition.java, src/main/java/de/intevation/flys/importer/ImportRange.java, src/main/java/de/intevation/flys/importer/ImportAttribute.java, src/main/java/de/intevation/flys/importer/ImportAnnotation.java: New helper models for import. * src/main/java/de/intevation/flys/importer/PegelGltParser.java, src/main/java/de/intevation/flys/importer/ImportGauge.java: Use new models. 2011-03-17 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/PegelGltParser.java: New. Parser for PEGEL.GLT files. * src/main/java/de/intevation/flys/importer/ImportGauge.java: New. Import model for gauges. * src/main/java/de/intevation/flys/utils/DBCPConnectionProvider.java: Removed needless imports. * src/main/java/de/intevation/flys/importer/ImportRiver.java: Added method to parse the gauges. * src/main/java/de/intevation/flys/importer/InfoGewParser.java: Trigger pegel glt file parsing. 2011-03-17 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/Importer.java: Used thread local pattern to make sharing of session easier. 2011-03-17 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/Importer.java: Fixed error in HQL statement. 2011-03-17 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/utils/DBCPConnectionProvider.java: Commented out a debug block because it leaks the db password. 2011-03-16 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/importer/InfoGewParser.java: Expose imported rivers. * src/main/java/de/intevation/flys/importer/InfoGewParser.java: Store imported rivers into database. Needs testing! 2011-03-16 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/backend/SessionFactoryProvider.java: Default connection parameters are now overwritable with system properties (defaults in brackets): - user name: flys.backend.user (flys) - user password: flys.backend.password (flys) - db dialect: flys.backend.dialect (org.hibernate.dialect.PostgreSQLDialect) - db driver: flys.backend.driver (org.postgresql.Driver) - db url: flys.backend.url (jdbc:postgresql://localhost:5432/flys) 2011-03-16 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/backend/SessionFactoryProvider.java: Expose createSessionFactory() as public to be usable without a artifact database running. 2011-03-16 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/utils/FileTools.java: Tools for handling with filenames. Currently there is a repair(File) method with repairs letter case errors which is useful when reading windows filenames on a un*x platform. * src/main/java/de/intevation/flys/importer/Importer.java: Standalone app to read data from the file system and store it in a database. Currently it does not store anything. It only loads info gew files. * src/main/java/de/intevation/flys/importer/InfoGewParser.java: Info gew parser. * src/main/java/de/intevation/flys/importer/ImportRiver.java: Helper model of a river used produced by parsing the info gew files. 2011-03-15 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/backend/SessionFactoryProvider.java: New. SessionFactoryProvider.getSessionFactory() provides a SessionFactory to use the Hibernate O/R mapper for the FLYS backend. 2011-03-15 Sascha L. Teichmann <sascha.teichmann@intevation.de> * pom.xml: Added dependency to artifacts-commons to be able to use the global configuration of the artifact database. 2011-03-15 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/App.java: Wire all POJOs to corresponding factory. * src/main/java/de/intevation/flys/model/*.java: Generate all foreign key constraints. TODO: name them correctly because the machine generated names are ugly and do not fit the PostgreSQL names. * doc/schema/postgresql.sql: Small quantifier fix in descriptions of wst columns. 2011-03-14 Sascha L. Teichmann <sascha.teichmann@intevation.de> * doc/schema/postgresql.sql: Fixed wrong spelled column references in foreign keys introduces with last change. 2011-03-14 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/flys/model/*.java: Added column annotations for simple fields. TODO: foreign keys. 2011-03-14 Sascha L. Teichmann <sascha.teichmann@intevation.de> * doc/schema/postgresql.sql, doc/schema/postgresql-cleanup.sql: Fixed inconsistent table names. * src/main/java/de/intevation/flys/model/*.java: Added entity and id annotations. 2011-03-14 Sascha L. Teichmann <sascha.teichmann@intevation.de> * doc/schema/postgresql.sql, doc/schema/postgresql-cleanup.sql: Added missing sequences. * doc/schema/sqlite.sql: Deleted. Not longer supported. 2011-03-11 Sascha L. Teichmann <sascha.teichmann@intevation.de> * doc/schema/postgresql-cleanup.sql: Forgot to add. * src/main/java/de/intevation/flys/App.java: Use Apache Commons DBCP as Hibernate connection provide. * src/main/java/de/intevation/flys/model/River.java: Added a constructor with string argument. Set the sequence increment to 1 (eat up 100 at a time before). * pom.xml: Added PostgreSQL 8.4 driver as runtime dependency. 2011-03-11 Sascha L. Teichmann <sascha.teichmann@intevation.de> * doc/schema/postgresql-cleanup.sql: New. Tear down schema for a postgres database. * doc/schema/postgresql.sql: Added squence for auto generating ids in river table. Cleaned up schema. * src/main/java/de/intevation/flys/App.java: Simple test app to interact with hibernate. Needs to be removed because its only a toy. * src/main/java/de/intevation/flys/utils/DBCPConnectionProvider.java: New. Binds Apache Commons to Hibernate. * pom.xml: Added dependencies to log4j, commons dbcp, JPA of hibernate. * src/main/java/de/intevation/flys/model/River.java: Added JPA annotations. * src/main/java/de/intevation/flys/model/*.java: Replaced Long with Integer because column ids are only four bytes wide. 2011-03-11 Sascha L. Teichmann <sascha.teichmann@intevation.de> * doc/schema/sqlite.sql, doc/schema/postgresql.sql: Fixed smaller issues in ddl. * src/main/java/de/intevation/flys/model/*.java: Added POJOs of to be mapped. TODO: Map them! * pom.xml: Added plugin config for hibernate. 2011-03-09 Sascha L. Teichmann <sascha.teichmann@intevation.de> * pom.xml: Added dependency (and corresponding repository) to Hibernate Core 3.6.1 Final 2011-03-09 Sascha L. Teichmann <sascha.teichmann@intevation.de> * pom.xml, src/**: Created a new empty maven project: $ mvn archetype:create \ -DgroupId=de.intevation.flys \ -DartifactId=flys-backend 2011-03-09 Sascha L. Teichmann <sascha.teichmann@intevation.de> * README: New. Some setup instructions. * doc/schema/postgresql.sql: New. Schema converted to PostgreSQL * doc/schema/sqlite.sql: Fixed defect foreign key constraints. 2011-03-09 Sascha L. Teichmann <sascha.teichmann@intevation.de> * doc/schema/sqlite.sql: Factorized time intervals out into a separated table. 2011-01-22 Sascha L. Teichmann <sascha.teichmann@intevation.de> * contrib/import-kms.py, contrib/import-gew.py: Initial scripts to import data into SQLite database. They still need some work. 2011-02-10 Sascha L. Teichmann <sascha.teichmann@intevation.de>: * doc/schema/sqlite.sql: Added initial schema for FLYS database.