Mercurial > dive4elements > river

comparison etl/ChangeLog @ 5838:5aa05a7a34b7

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Rename modules to more fitting names.
author Sascha L. Teichmann <teichmann@intevation.de>
date Thu, 25 Apr 2013 15:23:37 +0200
parents flys-aft/ChangeLog@9c78264109ad
children
comparison
equal deleted inserted replaced
5837:d9901a08d0a6 5838:5aa05a7a34b7
1 2012-09-15 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
2
3 * ../.hgignore: Add ignores for flys-aft, too.
4
5 * src/main/java/de/intevation/aft/DischargeTable.java: Ignore
6 tables with unset GUELTIG_VON or GUELTIG_BIS values.
7
8 2012-09-11 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
9
10 * README.txt: Added infos how to build.
11
12 * doc/conf-oracle.xml: Demo config for Oracle.
13
14 * bin/run.sh: New start script.
15 * bin/log4j.properties: Demo log4j config.
16
17 * pom.xml: Added config for Maven assembly plugin.
18 * pom-oracle.xml: New. Has extra dependency to Oracle JDBC.
19
20 2012-09-11 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
21
22 * pom.xml: Java 1.5 -> 1.6
23 * README.txt: Removed new line.
24
25 2012-02-16 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
26
27 * README.txt: Describe configuration and function. TODO:
28 Write about running.
29
30 2012-02-16 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
31
32 * README.txt: New. Contains error messages by now. TODO: Write
33 more about the whole process.
34
35 * src/main/java/de/intevation/aft/SyncContext.java,
36 src/main/java/de/intevation/aft/DischargeTable.java,
37 src/main/java/de/intevation/aft/Notification.java,
38 src/main/java/de/intevation/aft/River.java,
39 src/main/java/de/intevation/aft/Sync.java:
40 Adjusted and improved error messages.
41
42 2012-02-08 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
43
44 * src/main/resources/sql/flys-common.properties: Insert
45 new discharge tables as 'Historische Abflusstafel' kind.
46
47 2012-01-11 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
48
49 * src/main/resources/sql/flys-oracle-jdbc-oracledriver.properties:
50 Added 'FROM DUAL' clause when selecting new ids from sequences.
51 Sync process between AFT(Oracle) and FLYS(Oracle) is working now!
52
53 2012-01-10 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
54
55 * src/main/resources/sql/flys-oracle-jdbc-oracledriver.properties: New.
56 Statements to make the FLYS database connection Oracle compatible.
57 Untested!
58
59 2012-01-10 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
60
61 * src/main/resources/sql/aft-oracle-jdbc-oracledriver.properties: New.
62 Statements to make the AFT database connection Oracle compatible.
63
64 2012-01-09 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
65
66 * src/main/java/de/intevation/aft/DischargeTable.java: Write
67 warning if there are discharge tables with same descriptions
68 in FLYS or AFT and ignore the redundant ones. This led
69 to an ever growing FLYS database.
70
71 2012-01-09 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
72
73 * src/main/java/de/intevation/aft/Sync.java: Log if modifications
74 are found or not.
75
76 * src/main/java/de/intevation/aft/River.java: Commit/rollback
77 changes on gauge if a gauge is updated.
78
79 2012-01-09 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
80
81 * src/main/java/de/intevation/aft/DischargeTable.java: Store
82 the W/Q values in sets to prevent value duplications leading
83 to unique constraint violations in FLYS. Log a warning
84 when loading a W/Q value duplication.
85
86 This have the nice side effect that the W/Q values are
87 written sorted by Q/W which is of benefit for FLYS.
88
89 2012-01-09 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
90
91 * src/main/java/de/intevation/aft/River.java: Fixed logic bug
92 when writing discharge tables of an gauge existing in both dbs.
93
94 * src/main/java/de/intevation/aft/DischargeTable.java: Moved
95 some SQL code from River here to simplify the persistence.
96
97 2012-01-09 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
98
99 * src/main/java/de/intevation/aft/TimeInterval.java(toString): Added
100 toString() method.
101
102 * src/main/java/de/intevation/aft/SyncContext.java: Added debug
103 logging when creating a new time inteval.
104
105 * src/main/java/de/intevation/aft/DischargeTable.java: Added
106 warning when start and end of a time interval from AFT
107 are ordered start > end.
108
109 2012-01-07 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
110
111 * src/main/java/de/intevation/db/SymbolicStatement.java:
112 Added setLong() method. Used when setting the official number
113 of a gauge.
114
115 * src/main/java/de/intevation/aft/River.java: Store the new
116 discharge tables in FLYS when gauges exist in both
117 FLYS and AFT and there are discharge tables that are only in AFT.
118 Store official number as long.
119
120 2012-01-07 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
121
122 * src/main/java/de/intevation/aft/River.java: Store
123 the W/Q differences of existing discharge tables
124 to the FLYS database.
125
126 * src/main/java/de/intevation/aft/DischargeTable.java: Added
127 getter/setter for W/Q values.
128
129 2012-01-06 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
130
131 * src/main/java/de/intevation/aft/River.java: Do the pairing
132 of discharge table of a gauge that needs updates. TODO:
133 Build the W/Q difference of found FLYS/AFT matches and
134 create the discharge tables in FLYS that are found in AFT.
135
136 2012-01-06 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
137
138 * src/main/resources/sql/flys-common.properties: Added statement
139 to load all discharge tables of a given gauge.
140
141 * src/main/java/de/intevation/aft/DischargeTable.java:
142
143 * src/main/java/de/intevation/aft/DIPSGauge.java: Store
144 the official number, too.
145
146 * src/main/java/de/intevation/aft/River.java: In case of
147 updating a gauge load all discharge tables of that gauge
148 from FLYS and AFT. TODO: Do pairing based on the descriptions.
149
150 * src/main/java/de/intevation/aft/DischargeTable.java: Code
151 to load the discharge table from FLYS and AFT.
152
153 2012-01-06 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
154
155 * src/main/java/de/intevation/aft/WQDiff.java: New.
156 Calculates the difference of two W/Q value table of a
157 discharge table. This can be used to write an optimized
158 change set in terms of executed SQL to the FLYS database.
159
160 * src/main/java/de/intevation/aft/WQ.java: Changed the EPS_CMP
161 comparator to first sort by Q and then by W because the Qs
162 are more distinct and the dominant component.
163
164 * src/main/resources/sql/flys-common.properties: Added statement
165 to delete W/Q values.
166
167 2012-01-06 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
168
169 * src/main/java/de/intevation/utils/XML.java: Added code
170 to send/receive documents from streams.
171
172 * src/main/java/de/intevation/aft/Notification.java: New.
173 Sends XML documents via HTTP POST to given URLs.
174
175 * src/main/java/de/intevation/aft/Sync.java: Send notifications
176 if the FLYS database was modified. Useful to invalidate caches
177 in the artifact server.
178
179 2012-01-05 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
180
181 * doc/conf.xml: Added demo notification url.
182
183 * src/main/java/de/intevation/aft/River.java,
184 src/main/java/de/intevation/aft/Rivers.java,
185 src/main/java/de/intevation/aft/Sync.java: Modifications
186 are bubbled up to main() to send notifactions.
187
188 2012-01-05 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
189
190 * src/main/java/de/intevation/aft/DischargeTable.java: Store
191 W/Q values to FLYS.
192
193 * src/main/resources/sql/flys-common.properties: Added statements
194 to store W/Q values into FLYS database.
195
196 2012-01-05 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
197
198 * src/main/java/de/intevation/aft/WQ.java: New. W/Q model used
199 for AFT and FLYS.
200
201 * src/main/java/de/intevation/aft/DischargeTable.java: Holds
202 a list of its W/Q values now. Values are loadable from AFT
203 and FLYS.
204
205 * src/main/resources/sql/aft-common.properties,
206 src/main/resources/sql/flys-common.properties: Added statements
207 to load W/Q values for a given discharge table.
208
209 2012-01-04 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
210
211 * src/main/java/de/intevation/aft/SyncContext.java(fetchOrCreateFLYSTimeInterval):
212 Create FLYS time intervals if they are not in the database.
213
214 * src/main/java/de/intevation/aft/DischargeTable.java: New. Model
215 for discharge tables.
216
217 * src/main/java/de/intevation/aft/TimeInterval.java: Added
218 convinience constructors.
219
220 * src/main/java/de/intevation/aft/River.java: Store discharge tables.
221
222 * src/main/java/de/intevation/aft/Sync.java: Exit with errorcode
223 if syncing fails.
224
225 * src/main/resources/sql/aft-common.properties: Fetch the
226 description of a discharge table, too.
227
228 * src/main/resources/sql/flys-common.properties: Added statements
229 to create time intevals and discharge tables.
230
231 2012-01-03 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
232
233 * src/main/java/de/intevation/aft/TimeInterval.java: New.
234 Model for FLYS time intervals.
235
236 * src/main/java/de/intevation/aft/SyncContext.java: Preload
237 existing time intervals from FLYS.
238
239 * src/main/java/de/intevation/aft/Sync.java: Call init()
240 after construction to ensure that the db connections are
241 closed properly.
242
243 * src/main/resources/sql/flys-common.properties: Added statement
244 to fetch the time intervals from FLYS.
245
246 2012-01-03 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
247
248 * src/main/java/de/intevation/aft/River.java: Fetch discharge table
249 infos from AFT.
250
251 * src/main/resources/sql/aft-common.properties: Added statement to fetch
252 infos from ABFLUSSTAFEL.
253
254 2012-01-02 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
255
256 * src/main/resources/sql/flys-common.properties: Added statements
257 to create gauges in FLYS.
258
259 * src/main/java/de/intevation/aft/DIPSGauge.java: Make more fields
260 accessible for gauge creation in FLYS.
261
262 * src/main/java/de/intevation/aft/River.java: Store new gauges
263 in FLYS.
264
265 * src/main/java/de/intevation/db/ConnectedStatements.java:
266 Added logging, make methods of transaction handling public.
267
268 * src/main/java/de/intevation/db/SymbolicStatement.java(setDouble):
269 Fixed argument type problem.
270
271 2012-01-02 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
272
273 * src/main/java/de/intevation/db/ConnectionBuilder.java:
274 Set auto commit of new connection to false to enable transaction.
275
276 * src/main/java/de/intevation/db/ConnectedStatements.java:
277 Added methods to begin, commit and rollback transactions.
278 Relies on savepoint support which is check by database metadata.
279
280 2011-12-22 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
281
282 * src/main/java/de/intevation/aft/River.java: Figure out
283 which gauges must be updated, which must be created.
284
285 * src/main/java/de/intevation/aft/DIPSGauge.java: Store
286 info from AFT and FLYS, too.
287
288 * src/main/resources/sql/flys-common.properties: Fetch the
289 official number, too.
290 2011-12-20 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
291
292 * src/main/java/de/intevation/aft/River.java: Removed
293 index DIPS gauge number -> DIPS gauge.
294
295 * src/main/java/de/intevation/aft/SyncContext.java: Readded
296 here, because the index can be shared by all rivers.
297
298 2011-12-20 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
299
300 * doc/repair.xsl: Repair XSL transform which brings the
301 DIPS gauge numbers of the 15 FLYS gauges to the same
302 numbers as they are used in "Pegel Online".
303
304 !!! The purpose of this script is to do more repairing !!!
305
306 * doc/pegelstationen.xml: Sub document of repair. Used
307 for lookup the correct pegel numbers.
308
309 * doc/conf.xml: Changed to optionally load the repair XSLT.
310
311 * src/main/java/de/intevation/aft/Sync.java: Load the
312 repair XSL transformation if configured.
313
314 * src/main/java/de/intevation/utils/XML.java: Added code
315 to make XSL transforms possible.
316
317 * src/main/java/de/intevation/aft/River.java,
318 src/main/java/de/intevation/aft/Rivers.java: Fixed logging.
319
320 2011-12-20 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
321
322 * src/main/java/de/intevation/aft/River.java,
323 src/main/java/de/intevation/aft/DIPSGauge.java: Make DIPS check
324 more verbose.
325
326 2011-12-16 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
327
328 * src/main/java/de/intevation/db/SymbolicStatement.java:
329 Made the setX() methods cascadable.
330
331 * src/main/java/de/intevation/aft/River.java: Fetches
332 the gauges from the database.
333
334 * src/main/resources/sql/aft-common.properties,
335 src/main/resources/sql/flys-common.properties: Added gauges
336 statements.
337
338 2011-12-14 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
339
340 * src/main/java/de/intevation/db/SymbolicStatement.java:
341 Added execute(), executeQuery() & Co.
342
343 * src/main/java/de/intevation/aft/IdPair.java: New. Base class
344 for id pairs to identify same object in both databases.
345
346 * src/main/java/de/intevation/aft/River.java: New. To sync
347 the objects of one river.
348
349 * src/main/java/de/intevation/aft/Rivers.java: Figure out
350 only the rivers which are in both databases and sync them.
351
352 * src/main/java/de/intevation/aft/Sync.java: Only pass the
353 connected statements to the sync.
354
355 * src/main/resources/sql/flys-common.properties: Fixed SQL for
356 fetching the rivers.
357
358 * pom.xml: Added dependency to PostgreSQL.
359
360 * doc/conf.xml: SQLite needs a driver class.
361
362 2011-12-13 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
363
364 * src/main/java/de/intevation/db/ConnectionBuilder.java: Removed
365 statements here.
366
367 * src/main/java/de/intevation/db/Statements.java: Added method
368 to access the hole map of statements.
369
370 * src/main/java/de/intevation/db/ConnectedStatements.java: New.
371 A cache that binds prepared statements to a connection.
372
373 2011-12-13 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
374
375 * src/main/java/de/intevation/db/ConnectionBuilder.java: Added
376 access to Statements.
377
378 * src/main/java/de/intevation/db/SymbolicStatement.java: New.
379 Made top level from inner class of Statements.
380
381 * src/main/java/de/intevation/db/Statements.java: Moved SymbolicStatement
382 out to top level class.
383
384 * src/main/java/de/intevation/aft/Rivers.java: Syncing beginns at
385 river level.
386
387 * src/main/java/de/intevation/aft/Sync.java: Start the syncing with
388 the rivers of both dbs.
389
390 2011-12-13 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
391
392 * doc/conf.xml: Adjusted structure to be more generic.
393
394 * src/main/java/de/intevation/utils/XML.java: Allow namespace aware
395 file parsing.
396
397 * src/main/java/de/intevation/db/ConnectionBuilder.java: New. Evaluate
398 config and builds a new db connection.
399
400 * src/main/java/de/intevation/aft/Sync.java: Load config file.
401
402 * pom.xml: Added dependency to SQLite JDBC driver.
403
404 2011-12-13 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
405
406 * doc/conf.xml: New. Configuration file.
407
408 2011-12-13 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
409
410 * ChangeLog: New. Argh! Forgot to check it in before.
411
412 * src/main/java/de/intevation/utils/XML.java: New. XML/XPath support.
413 Mainly a stripped down version of
414 de.intevation.artifacts.common.utils.XMLUtils
415
416 2011-12-12 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
417
418 * src/main/java/de/intevation/db/Statements.java: Added support
419 for symbolic prepared statements.
420
421 2011-12-12 Sascha L. Teichmann <sascha.teichmann@inteavtion.de>
422
423 * src/main/java/de/intevation/db/Statements.java: New. Load statements
424 from ressources.
425
426 * src/main/resources/sql/aft-common.properties: New. Common statements
427 for the AFT side of the sync.
428
429 * src/main/resources/sql/flys-common.properties: New. Common statements
430 for the FLYS side of the sync.
431
432 * pom.xml: Added dependency to log4j
http://dive4elements.wald.intevation.org