comparison geo-backend/ChangeLog @ 557:05fcb3c553fd 0.4

merged geo-backend/0.4
author Thomas Arendsen Hein <thomas@intevation.de>
date Fri, 28 Sep 2012 12:13:50 +0200
parents 48121fa4dc95
children 23a0f2c5e3b8
comparison
equal deleted inserted replaced
540:80630520e25a 557:05fcb3c553fd
1 2010-01-27 Hans Plum <hans@intevation.de>
2
3 RELEASE 0.4
4
5 * Changes, NEWS, ChangeLog: Summarized changes.
6
7 2009-01-25 Sascha L. Teichmann <sascha.teichmann@intevation.de>
8
9 Fix for gnv/issue107
10
11 * src/main/java/de/intevation/gnv/geobackend/util/RedundancyRemover.java:
12 New. Used to reduce the redundancy of data in SQL result sets. Column
13 values are very distinct so the degree or redundancy is very high. E.g
14 the SDE produces for each date value a java.util.GregorianCalendar object
15 which are very heavy weighted. We only use them r/o so we can hash each date
16 value to one unique representative. Same logic applys to integer values
17 and so on.
18
19 With this technique we are able to reduce the memory consumption by
20 over 90%. Because it is not very feasible to store the whole history
21 of column we only have a limited number of cached values per column.
22 This is controlled via the system property
23 "de.intevation.gnv.geobackend.util.RedundancyRemover.lookback" which
24 defaults to 1029.
25
26 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java:
27 Use RedundancyRemovers to filter the incoming traffic from SDE.
28
29 2010-01-26 Tim Englich <tim.englich@intevation.de>
30
31 * src/test/java/de/intevation/gnv/geobackend/base/query/ToCharSample.java :
32 Added Sample for bug in ArcSDE 9.3 using to_char-Function with SpatialQuery
33 * src/test/java/de/intevation/gnv/geobackend/base/query/GroupBySample.java :
34 Added Sample for Bug in ArcSDE using group by with SpatialQuery
35
36 2009-01-25 Sascha L. Teichmann <sascha.teichmann@intevation.de>
37
38 Another attempt to fix gnv/issue34
39
40 * src/main/java/de/intevation/gnv/geobackend/base/DefaultResultDescriptor.java:
41 Add a toString() method.
42
43 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEConnection.java:
44 Added an idle time detection mechanism. You can call touch() on
45 a connection to refresh the internal timestamp and ask with isActive()
46 if a constructor given time interval in milliseconds is exceeded.
47
48 * src/main/java/de/intevation/gnv/geobackend/sde/connectionpool/ArcSDEPoolableObjectFactory.java:
49 Added a check of inactivity to validateObject() before relying on SDE logic.
50 This is the last line of defence if the testServer() call is going not to respond.
51 This duplicates the eviction policy of Apache Commons pool a bit but I found
52 this way to be more trustworthy.
53
54 The idle time is configure with the property 'serverInactiveInterval'.
55 units: seconds. Defaults to 5 minutes.
56
57 The unit of 'serverRoundtripInterval' (parameter of testServer()) is now
58 in seconds, too. Default: 5 seconds. This is the value used in GeoTools.
59
60 2009-01-09 Sascha L. Teichmann <sascha.teichmann@intevation.de>
61
62 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/SDEQuery.java,
63 src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java,
64 src/main/java/de/intevation/gnv/geobackend/sde/datasources/RasterObject.java:
65 Cleanup imports.
66
67 2010-01-15 Tim Englich <tim.englich@intevation.de>
68
69 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEConnection.java (isValid):
70 Implemented the isValid-method of the ArcSDEConnection for
71 getting a propper Validation of the used SEConnection.
72 For the Validation the Method testServer of the SEConnection is used.
73
74 * src/main/java/de/intevation/gnv/geobackend/sde/connectionpool/ArcSDEPoolableObjectFactory.java (ArcSDEPoolableObjectFactory):
75 Added Member serverRoundtripInterval to represent the Time in
76 Milliseconds which is allowed to reused a returned Connection.
77 Also used Connection.isValid Method instead of Connection.isClosed
78 to validate the Connection befor it could be borrowed from the Pool.
79
80 2009-01-12 Sascha L. Teichmann <sascha.teichmann@intevation.de>
81
82 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/RasterObject.java:
83 Added bicubic spline interpolation on raster tiles. This
84 should increase the 'virtual' resolution more than the
85 bilinear interpolation. To enable it write 'bicubic' into
86 the configuration file.
87
88 * pom.xml: Added dependency to Apache Common Math 2.0 for
89 the cubic spline interpolation.
90
91 2009-01-09 Sascha L. Teichmann <sascha.teichmann@intevation.de>
92
93 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/RasterObject.java:
94 Added static method to fetch matching interpolation type for
95 given string. Defaults to nearest neighbor if no string is matching.
96
97 2009-01-09 Sascha L. Teichmann <sascha.teichmann@intevation.de>
98
99 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/RasterObject.java:
100 Implemented bilinear interpolation.
101
102 2009-01-09 Sascha L. Teichmann <sascha.teichmann@intevation.de>
103
104 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java:
105 Fixed transformations to figure out the right tile in the SDE.
106 Taking the tile origin into account now and does not do wrong rounding
107 of column and row indices. Simplified the code a lot.
108
109 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/RasterObject.java:
110 Fixed access to right right pixel now. Removed the envelope code because
111 its not needed any longer. You can ask a RasterObject directly now if
112 a given coordinate is on the tile now. The tile space is now correctly
113 spanned between (0, 0) and (tile width - 1, tile height - 1) now which
114 allows other access patterns like bilinear interpolation. TODO: implement
115 bilinear interpolation.
116
117 2010-01-08 Tim Englich <tim.englich@intevation.de>
118
119 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/SDEQuery.java,
120 src/main/java/de/intevation/gnv/geobackend/sde/datasources/Row.java,
121 src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java,
122 src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEConnection.java,
123 src/main/java/de/intevation/gnv/geobackend/base/connectionpool/ConnectionPoolFactory.java (initializeConnectionPool):
124 Removed Encodingerrors from listed Files. All Files are now UTF-8 compliant.
125
126 2010-01-05 Tim Englich <tim.englich@intevation.de>
127
128 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/RasterObject.java:
129 Made it serializable to be cachable beyond reboot.
130
131 2010-01-05 Tim Englich <tim.englich@intevation.de>
132
133 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (handleResultSet):
134 Switched to return the RasterObject instead of the depth in an Rasterquery.
135 This should help to get a better performance.
136 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/RasterObject.java (RasterObject):
137 Added new Object for Caching Rastertiles and all Informations that are
138 required for calculating the Depth to a given Coordinate.
139
140 2010-01-04 Tim Englich <tim.englich@intevation.de>
141
142 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (handleResultSet):
143 Added Rasterdatasupport to the ARCSDESTATEMENT.
144 Now it is possible to Query the Value of an Pixel like that:
145 SELECT ST_ASTEXT(RASTER) FROM MEDIAN.TOPO_BS_D_6X10SEC where INTERSECTS(RASTER, "POINT( 12.084721594184593 54.35583351483295)")
146
147 2009-12-19 Sascha L. Teichmann <sascha.teichmann@intevation.de>
148
149 * contrib/sql-cache.xml: Configured disk storage to survive restarts.
150
151 * src/main/java/de/intevation/gnv/geobackend/base/query/QueryExecutorFactory.java:
152 Added explicit shutdown hook.
153
154 * src/main/java/de/intevation/gnv/geobackend/base/query/CachingQueryExecutorFactory.java:
155 Fixed spelling of cacheResults() which prevented results to be stored in cache.
156 Added some logging.
157
158 2009-12-18 Sascha L. Teichmann <sascha.teichmann@intevation.de>
159
160 Experimental caching of SQL results via Ehcache
161
162 * pom.xml: Added dependency to Ehcache
163
164 * src/main/java/de/intevation/gnv/geobackend/base/query/QueryExecutor.java:
165 Extended interface to support caching by SQL statements as Strings.
166
167 * src/main/java/de/intevation/gnv/geobackend/base/query/QueryExecutorBase.java:
168 Implements new interface with no caching
169
170 * src/main/java/de/intevation/gnv/geobackend/base/query/container/QueryContainerFactory.java:
171 synchronized access to singleton.
172
173 * src/main/java/de/intevation/gnv/geobackend/base/query/DefaultQueryExceutor.java:
174 calls caching methods of base class.
175
176 * src/main/java/de/intevation/gnv/geobackend/base/query/QueryExecutorFactory.java:
177 When system property 'query.executor.factory' is given the value is used as
178 a class name to be instantiated as a sub class of QueryExecutorFactory.
179 This happens if the singleton of QueryExecutorFactory is created. With the
180 mechanism it is possible to replace the standard behavior of creating QueryExecutors.
181
182 * src/main/java/de/intevation/gnv/geobackend/base/query/CachingQueryExecutorFactory.java:
183 New: This class extends QueryExecutorFactory and can be used as replacement for
184 the standard QueryExecutorFactory. Usage:
185
186 -Dquery.executor.factory=de.intevation.gnv.geobackend.base.query.CachingQueryExecutorFactory
187
188 If you pass in another system property 'caching.query.executor.config' you can
189 customize the caching. The value is used as a file name to an XML file with
190 Ehcache configuration.
191
192 * contrib/sql-cache.xml: Demo configuration for sql caching. Usage:
193
194 -Dquery.executor.factory=de.intevation.gnv.geobackend.base.query.CachingQueryExecutorFactory
195 -Dcaching.query.executor.config=contrib/sql-cache.xml
196
197 2009-12-17 Sascha L. Teichmann <sascha.teichmann@intevation.de>
198
199 RELEASE 0.3
200
201 * Changes, NEWS, ChangeLog: Summarized changes.
202
203 2009-12-17 Ingo Weinzierl <ingo.weinzierl@intevation.de>
204
205 * src/main/java/de/intevation/gnv/geobackend/base/DefaultResult.java:
206 Fixed bug in DefaultResult when querying an index out of bounds.
207
208 2009-12-15 Tim Englich <tim.englich@intevation.de>
209
210 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (executeQuery):
211 We have to use a different SpatialFilter for Point in Polygon Queries as
212 in AreaIntersection Statements.
213 So we have to include a switch which will do that.
214 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEUtils.java (createPoints):
215 Added Support of Point-Objects for the possibility to execute a Point in Pologon
216 Statement using the ArcSDE.
217
218 2009-12-11 Ingo Weinzierl <ingo.weinzierl@intevation.de>
219
220 * src/main/java/de/intevation/gnv/geobackend/base/DefaultResult.java:
221 DefaultResult will return null if it doesn't contain the specified column
222 name in method getString(String columnName), instead of throwing an
223 exception.
224
225 2009-12-11 Sascha L. Teichmann <sascha.teichmann@intevation.de>
226
227 * src/main/java/de/intevation/gnv/geobackend/base/DefaultResultDescriptor.java,
228 src/main/java/de/intevation/gnv/geobackend/base/query/container/QueryContainer.java,
229 src/main/java/de/intevation/gnv/geobackend/base/query/container/DefaultQueryContainer.java:
230 Removed needless imports.
231
232 2009-11-30 Tim Englich <tim.englich@intevation.de>
233
234 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/SDEResultSet.java,
235 src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java,
236 src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEResultSetMetaData.java,
237 src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEConnection.java:
238 Added missing Methodstubs for Java 1.6 compability.
239 ###################################################
240 ###### ATTENTION: ONLY USE JAVA 1.6 FOR THIS ######
241 ###### AND ALL USING MODULES. ######
242 ###################################################
243
244 2009-11-27 Tim Englich <tim.englich@intevation.de>
245
246 * src/test/ressources/QueryExecutorTestCase.properties,
247 src/test/java/de/intevation/gnv/geobackend/base/query/QueryExecutorTestCase.java:
248 Revert Changes of the commit of Revision 377 which was unintended
249 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (executeQuery):
250 Changed Splitalgorithm of Statements so that it is Possible to add to_date Colums into a Statement
251 with Spatial-Columns or Spatial-Filters
252
253 2009-11-13 Hans Plum <hans@intevation.de>
254
255 RELEASE 0.2
256
257 * Changes, NEWS: Summarized changes.
258
259 2009-11-13 Hans Plum <hans@intevation.de>
260
261 * Made some ChangeLog entries a bit clearer.
262
263 2009-11-10 Tim Englich <tim.englich@intevation.de>
264
265 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (executeQuery):
266 Changed the splitting of the Statements for ARCSDE-Transformation.
267 Now it is possible to use a Select-Statement as an Table.
268
269 2009-11-06 Ingo Weinzierl <ingo.weinzierl@intevation.de>
270
271 * target/**: Removed. Should not be in version control.
272
273 2009-11-05 Tim Englich <tim.englich@intevation.de>
274
275 * src/test/ressources/QueryExecutorTestCase.properties,
276 src/test/java/de/intevation/gnv/geobackend/base/query/QueryExecutorTestCase.java (testSpatialQueryWithInnerSelect):
277 Added an Tescase to test the InnerSelect Statments in Spatial-Queries.
278 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/Row.java (getPosValue):
279 Bugfix. An Point could not have an Z-Coordinate-Value
280 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (executeQuery):
281 Added the possibility to execute Spatial-Queries using InnerStatements instead
282 of WKT in Intersects-where-clauses.
283
284 2009-11-02 Tim Englich <tim.englich@intevation.de>
285
286 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEConnection.java (createStatement):
287 Changed TODO to a question
288 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (executeQuery):
289 CodeCleanup: Removed deprecated TODO
290 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/SDEResultSet.java (getColumnIndex):
291 Removed Typo.
292 * src/main/java/de/intevation/gnv/geobackend/sde/connectionpool/ArcSDEPoolableObjectFactory.java:
293 CodeCleanup: Removed deprecated TODOS and replace Tabs with whitespaces.
294 * src/main/java/de/intevation/gnv/geobackend/base/query/container/DefaultQueryContainer.java:,
295 src/main/java/de/intevation/gnv/geobackend/base/query/container/QueryContainer.java:
296 CodeCleanup: Removed unused Method getQueryIds
297 * src/main/java/de/intevation/gnv/geobackend/sde/connectionpool/ArcSDEConnectionPool.java (closeConnection):
298 CodeCleanup: Inspect TODO and ansewerd the Question that the Connection has not to
299 be closed at that position but perhaps insite the ConnectionPool.
300
301 2009-10-30 Tim Englich <tim.englich@intevation.de>
302
303 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/Row.java (getPosValue):
304 Added Support for Polygons without interior Rings. This is required for generating
305 Horizontal Cross-Section Outputs.
306 The Z-Value of an Point was also added to the WKT-Represantation.
307
308 2009-10-29 Tim Englich <tim.englich@intevation.de>
309
310 Issue 45: Defintion of gaps in temporal and spatial data
311
312 * src/main/java/de/intevation/gnv/geobackend/base/DefaultResult.java (getInteger):
313 Added Integer-Support for DoubleValues because the ArcSDE does
314 not read values like -1 as Integer-values. issue45
315
316 2009-10-20 Tim Englich <tim.englich@intevation.de>
317
318 Issue 57: Improving memory usage of data objects
319
320 * src/main/java/de/intevation/gnv/geobackend/base/query/QueryExecutorBase.java (QueryExecutorBase),
321 src/main/java/de/intevation/gnv/geobackend/base/ResultDescriptor.java (getColumnClassName),
322 src/main/java/de/intevation/gnv/geobackend/base/Result.java,
323 src/main/java/de/intevation/gnv/geobackend/base/DefaultResultDescriptor.java (serialVersionUID),
324 src/main/java/de/intevation/gnv/geobackend/base/DefaultResult.java (getDate):
325 Integrated Patch of issue57 to get some Memoryusage-improvements
326
327 2009-10-19 Sascha L. Teichmann <sascha.teichmann@intevation.de>
328
329 * src/main/java/de/intevation/gnv/geobackend/base/DefaultResult.java:
330 Extends from HashMap directly to avoid one level of indirection and
331 do not create so much needless objects.
332
333 TODO: The representation in form of HashMaps is not very efficent!
334 Use array with index instead.
335
336 2009-10-14 Tim Englich <tim.englich@intevation.de>
337
338 * src/test/java/de/intevation/gnv/geobackend/base/query/QueryExecutorTestCase.java (testSpatialQueryWithoutIntersects),
339 src/test/ressources/QueryExecutorTestCase.properties:
340 Added further Queries for Testing Spatial-Query-Support
341
342 2009-10-14 Tim Englich <tim.englich@intevation.de>
343
344 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/Row.java (getPosValue):
345 Add WKT-Syntaxt to Shape ReturnValue
346 * src/test/ressources/QueryExecutorTestCase.properties:
347 Changed Test-Query-Statement for Spatial-Queries with INNERJOINS and ORDER BY
348 * src/test/java/de/intevation/gnv/geobackend/base/query/QueryExecutorTestCase.java (testSpatialQueryWithoutIntersects):
349 Changed expected Result-Size because of Changes in the QueryStatement which is used.
350 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (executeQuery):
351 Added INNERJOIN, ORDER BY and GROUP BY support to Spatial Queries
352
353 2009-10-13 Tim Englich <tim.englich@intevation.de>
354
355 * src/test/java/de/intevation/gnv/geobackend/base/query/QueryExecutorTestCase.java (testSpatialQueryWithoutIntersects):
356 Added an TaesCase for Querying spatial Data without an spatial restriction
357 * src/test/ressources/QueryExecutorTestCase.properties:
358 Added TestQuery with no spatial restriction
359 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (executeQuery):
360 Now it is possible to send spatial Quieries without an spatial restriction in the where clause.
361
362 2009-10-13 Tim Englich <tim.englich@intevation.de>
363
364 Issue 34: SQL Exception in REST Server when accessing geo-backend
365
366 * src/main/java/de/intevation/gnv/geobackend/sde/connectionpool/ArcSDEPoolableObjectFactory.java (validateObject) Edited:
367 Added Tests if the Object is Closed. And add propper instance Tests
368 * src/main/java/de/intevation/gnv/geobackend/sde/connectionpool/ArcSDEConnectionPool.java (initialize):
369 Added further ConfigurationParameters that could be set in the *.properties-File.
370 maxWait, maxIdle, minIdle, testOnBorrow, testOnReturn, testWhileIdle, numTestsPerEvictionRun,
371 timeBetweenEvictionRunsMillis, testWhileIdle
372 * src/test/ressources/ArcSDEConnectionPoolTestCase.properties:
373 Added som Configurationproperties for testing.
374
375
376 2009-10-13 Tim Englich <tim.englich@intevation.de>
377 * src/test/java/de/intevation/gnv/geobackend/base/query/QueryExecutorTestCase.java (QueryExecutorTestCase):
378 Changed expected Result of Spatial-Query-TestCase because of Changes to the used Query
379 * src/test/ressources/QueryExecutorTestCase.properties:
380 Added SFS-Conformal Query
381 * pom.xml:
382 Added dependency to JTS 1.9
383 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (executeQuery):
384 Added Spatial-Query-Support for ArcSDE
385 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEUtils.java (createPoints):
386 Switched Method Signature to JTS-Geometry-Support
387 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/Row.java (getStringValue):
388 Added String-Support for GeometryObjects
389
390 2009-10-07 Sascha L. Teichmann <sascha.teichmann@intevation.de>
391
392 RELEASE 0.1
393
394 * Changes, NEWS: Summarized changes.
395
396 2009-10-06 Sascha L. Teichmann <sascha.teichmann@intevation.de>
397
398 * Changelog -> ChangeLog: Renamed to make eclipse happy. Unified changelog style.
399
400 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/SDEQuery.java,
401 src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java:
402 Removed needless imports.
403
404 * pom.xml: xmllint-ed
405
406 2009-09-29 Tim Englich <tim.englich@intevation.de>
407
408 * src/main/java/de/intevation/gnv/geobackend/util/DateUtils.java (DATE_PATTERN) Edited:
409 Make the Constants assessible for other Parts of the Project to reduce the TouchPoints for
410 Dateformats.
411
412 2009-09-24 Tim Englich <tim.englich@intevation.de>
413
414 * src/main/java/de/intevation/gnv/geobackend/base/ResultDescriptor.java Edited,
415 src/main/java/de/intevation/gnv/geobackend/base/Result.java Edited,
416 src/main/java/de/intevation/gnv/geobackend/base/DefaultResultDescriptor.java Edited,
417 src/main/java/de/intevation/gnv/geobackend/base/DefaultResult.java Edited:
418 Make Classes Serializable for Storing in ArtifactDatabase
419
420 2009-09-21 Tim Englich <tim.englich@intevation.de>
421
422 * src/main/java/de/intevation/gnv/geobackend/base/DefaultResult.java:
423 Added formatted Date Handling in getString Method
424 * src/test/ressources/QueryExecutorTestCase.properties Edited,
425 src/test/java/de/intevation/gnv/geobackend/base/query/QueryExecutorTestCase.java Edited:
426 Some tests for Spatialqueries added
427 * src/main/java/de/intevation/gnv/geobackend/base/query/DefaultQueryExceutor.java Edited:
428 Put the final Querystring into a Logmessage
429
430 2009-09-09 Tim Englich <tim.englich@intevation.de>
431
432 * src/main/java/de/intevation/gnv/geobackend/base/DefaultResult.java (getDate):
433 . Intance Check for DateValues
434 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/SDEResultSet.java (next):
435 Logging removed because the performance slowdown fetching many ResultSets
436
437 2009-09-04 Tim Englich <tim.englich@intevation.de>
438
439 * pom.xml:

http://dive4elements.wald.intevation.org