Mercurial > dive4elements > gnv-client
diff geo-backend/ChangeLog @ 557:05fcb3c553fd 0.4
merged geo-backend/0.4
| author | Thomas Arendsen Hein <thomas@intevation.de> |
|---|---|
| date | Fri, 28 Sep 2012 12:13:50 +0200 |
| parents | 48121fa4dc95 |
| children | 23a0f2c5e3b8 |
line wrap: on
line diff
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/geo-backend/ChangeLog Fri Sep 28 12:13:50 2012 +0200 @@ -0,0 +1,439 @@ +2010-01-27 Hans Plum <hans@intevation.de> + + RELEASE 0.4 + + * Changes, NEWS, ChangeLog: Summarized changes. + +2009-01-25 Sascha L. Teichmann <sascha.teichmann@intevation.de> + + Fix for gnv/issue107 + + * src/main/java/de/intevation/gnv/geobackend/util/RedundancyRemover.java: + New. Used to reduce the redundancy of data in SQL result sets. Column + values are very distinct so the degree or redundancy is very high. E.g + the SDE produces for each date value a java.util.GregorianCalendar object + which are very heavy weighted. We only use them r/o so we can hash each date + value to one unique representative. Same logic applys to integer values + and so on. + + With this technique we are able to reduce the memory consumption by + over 90%. Because it is not very feasible to store the whole history + of column we only have a limited number of cached values per column. + This is controlled via the system property + "de.intevation.gnv.geobackend.util.RedundancyRemover.lookback" which + defaults to 1029. + + * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java: + Use RedundancyRemovers to filter the incoming traffic from SDE. + +2010-01-26 Tim Englich <tim.englich@intevation.de> + + * src/test/java/de/intevation/gnv/geobackend/base/query/ToCharSample.java : + Added Sample for bug in ArcSDE 9.3 using to_char-Function with SpatialQuery + * src/test/java/de/intevation/gnv/geobackend/base/query/GroupBySample.java : + Added Sample for Bug in ArcSDE using group by with SpatialQuery + +2009-01-25 Sascha L. Teichmann <sascha.teichmann@intevation.de> + + Another attempt to fix gnv/issue34 + + * src/main/java/de/intevation/gnv/geobackend/base/DefaultResultDescriptor.java: + Add a toString() method. + + * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEConnection.java: + Added an idle time detection mechanism. You can call touch() on + a connection to refresh the internal timestamp and ask with isActive() + if a constructor given time interval in milliseconds is exceeded. + + * src/main/java/de/intevation/gnv/geobackend/sde/connectionpool/ArcSDEPoolableObjectFactory.java: + Added a check of inactivity to validateObject() before relying on SDE logic. + This is the last line of defence if the testServer() call is going not to respond. + This duplicates the eviction policy of Apache Commons pool a bit but I found + this way to be more trustworthy. + + The idle time is configure with the property 'serverInactiveInterval'. + units: seconds. Defaults to 5 minutes. + + The unit of 'serverRoundtripInterval' (parameter of testServer()) is now + in seconds, too. Default: 5 seconds. This is the value used in GeoTools. + +2009-01-09 Sascha L. Teichmann <sascha.teichmann@intevation.de> + + * src/main/java/de/intevation/gnv/geobackend/sde/datasources/SDEQuery.java, + src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java, + src/main/java/de/intevation/gnv/geobackend/sde/datasources/RasterObject.java: + Cleanup imports. + +2010-01-15 Tim Englich <tim.englich@intevation.de> + + * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEConnection.java (isValid): + Implemented the isValid-method of the ArcSDEConnection for + getting a propper Validation of the used SEConnection. + For the Validation the Method testServer of the SEConnection is used. + + * src/main/java/de/intevation/gnv/geobackend/sde/connectionpool/ArcSDEPoolableObjectFactory.java (ArcSDEPoolableObjectFactory): + Added Member serverRoundtripInterval to represent the Time in + Milliseconds which is allowed to reused a returned Connection. + Also used Connection.isValid Method instead of Connection.isClosed + to validate the Connection befor it could be borrowed from the Pool. + +2009-01-12 Sascha L. Teichmann <sascha.teichmann@intevation.de> + + * src/main/java/de/intevation/gnv/geobackend/sde/datasources/RasterObject.java: + Added bicubic spline interpolation on raster tiles. This + should increase the 'virtual' resolution more than the + bilinear interpolation. To enable it write 'bicubic' into + the configuration file. + + * pom.xml: Added dependency to Apache Common Math 2.0 for + the cubic spline interpolation. + +2009-01-09 Sascha L. Teichmann <sascha.teichmann@intevation.de> + + * src/main/java/de/intevation/gnv/geobackend/sde/datasources/RasterObject.java: + Added static method to fetch matching interpolation type for + given string. Defaults to nearest neighbor if no string is matching. + +2009-01-09 Sascha L. Teichmann <sascha.teichmann@intevation.de> + + * src/main/java/de/intevation/gnv/geobackend/sde/datasources/RasterObject.java: + Implemented bilinear interpolation. + +2009-01-09 Sascha L. Teichmann <sascha.teichmann@intevation.de> + + * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java: + Fixed transformations to figure out the right tile in the SDE. + Taking the tile origin into account now and does not do wrong rounding + of column and row indices. Simplified the code a lot. + + * src/main/java/de/intevation/gnv/geobackend/sde/datasources/RasterObject.java: + Fixed access to right right pixel now. Removed the envelope code because + its not needed any longer. You can ask a RasterObject directly now if + a given coordinate is on the tile now. The tile space is now correctly + spanned between (0, 0) and (tile width - 1, tile height - 1) now which + allows other access patterns like bilinear interpolation. TODO: implement + bilinear interpolation. + +2010-01-08 Tim Englich <tim.englich@intevation.de> + + * src/main/java/de/intevation/gnv/geobackend/sde/datasources/SDEQuery.java, + src/main/java/de/intevation/gnv/geobackend/sde/datasources/Row.java, + src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java, + src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEConnection.java, + src/main/java/de/intevation/gnv/geobackend/base/connectionpool/ConnectionPoolFactory.java (initializeConnectionPool): + Removed Encodingerrors from listed Files. All Files are now UTF-8 compliant. + +2010-01-05 Tim Englich <tim.englich@intevation.de> + + * src/main/java/de/intevation/gnv/geobackend/sde/datasources/RasterObject.java: + Made it serializable to be cachable beyond reboot. + +2010-01-05 Tim Englich <tim.englich@intevation.de> + + * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (handleResultSet): + Switched to return the RasterObject instead of the depth in an Rasterquery. + This should help to get a better performance. + * src/main/java/de/intevation/gnv/geobackend/sde/datasources/RasterObject.java (RasterObject): + Added new Object for Caching Rastertiles and all Informations that are + required for calculating the Depth to a given Coordinate. + +2010-01-04 Tim Englich <tim.englich@intevation.de> + + * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (handleResultSet): + Added Rasterdatasupport to the ARCSDESTATEMENT. + Now it is possible to Query the Value of an Pixel like that: + SELECT ST_ASTEXT(RASTER) FROM MEDIAN.TOPO_BS_D_6X10SEC where INTERSECTS(RASTER, "POINT( 12.084721594184593 54.35583351483295)") + +2009-12-19 Sascha L. Teichmann <sascha.teichmann@intevation.de> + + * contrib/sql-cache.xml: Configured disk storage to survive restarts. + + * src/main/java/de/intevation/gnv/geobackend/base/query/QueryExecutorFactory.java: + Added explicit shutdown hook. + + * src/main/java/de/intevation/gnv/geobackend/base/query/CachingQueryExecutorFactory.java: + Fixed spelling of cacheResults() which prevented results to be stored in cache. + Added some logging. + +2009-12-18 Sascha L. Teichmann <sascha.teichmann@intevation.de> + + Experimental caching of SQL results via Ehcache + + * pom.xml: Added dependency to Ehcache + + * src/main/java/de/intevation/gnv/geobackend/base/query/QueryExecutor.java: + Extended interface to support caching by SQL statements as Strings. + + * src/main/java/de/intevation/gnv/geobackend/base/query/QueryExecutorBase.java: + Implements new interface with no caching + + * src/main/java/de/intevation/gnv/geobackend/base/query/container/QueryContainerFactory.java: + synchronized access to singleton. + + * src/main/java/de/intevation/gnv/geobackend/base/query/DefaultQueryExceutor.java: + calls caching methods of base class. + + * src/main/java/de/intevation/gnv/geobackend/base/query/QueryExecutorFactory.java: + When system property 'query.executor.factory' is given the value is used as + a class name to be instantiated as a sub class of QueryExecutorFactory. + This happens if the singleton of QueryExecutorFactory is created. With the + mechanism it is possible to replace the standard behavior of creating QueryExecutors. + + * src/main/java/de/intevation/gnv/geobackend/base/query/CachingQueryExecutorFactory.java: + New: This class extends QueryExecutorFactory and can be used as replacement for + the standard QueryExecutorFactory. Usage: + + -Dquery.executor.factory=de.intevation.gnv.geobackend.base.query.CachingQueryExecutorFactory + + If you pass in another system property 'caching.query.executor.config' you can + customize the caching. The value is used as a file name to an XML file with + Ehcache configuration. + + * contrib/sql-cache.xml: Demo configuration for sql caching. Usage: + + -Dquery.executor.factory=de.intevation.gnv.geobackend.base.query.CachingQueryExecutorFactory + -Dcaching.query.executor.config=contrib/sql-cache.xml + +2009-12-17 Sascha L. Teichmann <sascha.teichmann@intevation.de> + + RELEASE 0.3 + + * Changes, NEWS, ChangeLog: Summarized changes. + +2009-12-17 Ingo Weinzierl <ingo.weinzierl@intevation.de> + + * src/main/java/de/intevation/gnv/geobackend/base/DefaultResult.java: + Fixed bug in DefaultResult when querying an index out of bounds. + +2009-12-15 Tim Englich <tim.englich@intevation.de> + + * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (executeQuery): + We have to use a different SpatialFilter for Point in Polygon Queries as + in AreaIntersection Statements. + So we have to include a switch which will do that. + * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEUtils.java (createPoints): + Added Support of Point-Objects for the possibility to execute a Point in Pologon + Statement using the ArcSDE. + +2009-12-11 Ingo Weinzierl <ingo.weinzierl@intevation.de> + + * src/main/java/de/intevation/gnv/geobackend/base/DefaultResult.java: + DefaultResult will return null if it doesn't contain the specified column + name in method getString(String columnName), instead of throwing an + exception. + +2009-12-11 Sascha L. Teichmann <sascha.teichmann@intevation.de> + + * src/main/java/de/intevation/gnv/geobackend/base/DefaultResultDescriptor.java, + src/main/java/de/intevation/gnv/geobackend/base/query/container/QueryContainer.java, + src/main/java/de/intevation/gnv/geobackend/base/query/container/DefaultQueryContainer.java: + Removed needless imports. + +2009-11-30 Tim Englich <tim.englich@intevation.de> + + * src/main/java/de/intevation/gnv/geobackend/sde/datasources/SDEResultSet.java, + src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java, + src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEResultSetMetaData.java, + src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEConnection.java: + Added missing Methodstubs for Java 1.6 compability. + ################################################### + ###### ATTENTION: ONLY USE JAVA 1.6 FOR THIS ###### + ###### AND ALL USING MODULES. ###### + ################################################### + +2009-11-27 Tim Englich <tim.englich@intevation.de> + + * src/test/ressources/QueryExecutorTestCase.properties, + src/test/java/de/intevation/gnv/geobackend/base/query/QueryExecutorTestCase.java: + Revert Changes of the commit of Revision 377 which was unintended + * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (executeQuery): + Changed Splitalgorithm of Statements so that it is Possible to add to_date Colums into a Statement + with Spatial-Columns or Spatial-Filters + +2009-11-13 Hans Plum <hans@intevation.de> + + RELEASE 0.2 + + * Changes, NEWS: Summarized changes. + +2009-11-13 Hans Plum <hans@intevation.de> + + * Made some ChangeLog entries a bit clearer. + +2009-11-10 Tim Englich <tim.englich@intevation.de> + + * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (executeQuery): + Changed the splitting of the Statements for ARCSDE-Transformation. + Now it is possible to use a Select-Statement as an Table. + +2009-11-06 Ingo Weinzierl <ingo.weinzierl@intevation.de> + + * target/**: Removed. Should not be in version control. + +2009-11-05 Tim Englich <tim.englich@intevation.de> + + * src/test/ressources/QueryExecutorTestCase.properties, + src/test/java/de/intevation/gnv/geobackend/base/query/QueryExecutorTestCase.java (testSpatialQueryWithInnerSelect): + Added an Tescase to test the InnerSelect Statments in Spatial-Queries. + * src/main/java/de/intevation/gnv/geobackend/sde/datasources/Row.java (getPosValue): + Bugfix. An Point could not have an Z-Coordinate-Value + * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (executeQuery): + Added the possibility to execute Spatial-Queries using InnerStatements instead + of WKT in Intersects-where-clauses. + +2009-11-02 Tim Englich <tim.englich@intevation.de> + + * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEConnection.java (createStatement): + Changed TODO to a question + * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (executeQuery): + CodeCleanup: Removed deprecated TODO + * src/main/java/de/intevation/gnv/geobackend/sde/datasources/SDEResultSet.java (getColumnIndex): + Removed Typo. + * src/main/java/de/intevation/gnv/geobackend/sde/connectionpool/ArcSDEPoolableObjectFactory.java: + CodeCleanup: Removed deprecated TODOS and replace Tabs with whitespaces. + * src/main/java/de/intevation/gnv/geobackend/base/query/container/DefaultQueryContainer.java:, + src/main/java/de/intevation/gnv/geobackend/base/query/container/QueryContainer.java: + CodeCleanup: Removed unused Method getQueryIds + * src/main/java/de/intevation/gnv/geobackend/sde/connectionpool/ArcSDEConnectionPool.java (closeConnection): + CodeCleanup: Inspect TODO and ansewerd the Question that the Connection has not to + be closed at that position but perhaps insite the ConnectionPool. + +2009-10-30 Tim Englich <tim.englich@intevation.de> + + * src/main/java/de/intevation/gnv/geobackend/sde/datasources/Row.java (getPosValue): + Added Support for Polygons without interior Rings. This is required for generating + Horizontal Cross-Section Outputs. + The Z-Value of an Point was also added to the WKT-Represantation. + +2009-10-29 Tim Englich <tim.englich@intevation.de> + + Issue 45: Defintion of gaps in temporal and spatial data + + * src/main/java/de/intevation/gnv/geobackend/base/DefaultResult.java (getInteger): + Added Integer-Support for DoubleValues because the ArcSDE does + not read values like -1 as Integer-values. issue45 + +2009-10-20 Tim Englich <tim.englich@intevation.de> + + Issue 57: Improving memory usage of data objects + + * src/main/java/de/intevation/gnv/geobackend/base/query/QueryExecutorBase.java (QueryExecutorBase), + src/main/java/de/intevation/gnv/geobackend/base/ResultDescriptor.java (getColumnClassName), + src/main/java/de/intevation/gnv/geobackend/base/Result.java, + src/main/java/de/intevation/gnv/geobackend/base/DefaultResultDescriptor.java (serialVersionUID), + src/main/java/de/intevation/gnv/geobackend/base/DefaultResult.java (getDate): + Integrated Patch of issue57 to get some Memoryusage-improvements + +2009-10-19 Sascha L. Teichmann <sascha.teichmann@intevation.de> + + * src/main/java/de/intevation/gnv/geobackend/base/DefaultResult.java: + Extends from HashMap directly to avoid one level of indirection and + do not create so much needless objects. + + TODO: The representation in form of HashMaps is not very efficent! + Use array with index instead. + +2009-10-14 Tim Englich <tim.englich@intevation.de> + + * src/test/java/de/intevation/gnv/geobackend/base/query/QueryExecutorTestCase.java (testSpatialQueryWithoutIntersects), + src/test/ressources/QueryExecutorTestCase.properties: + Added further Queries for Testing Spatial-Query-Support + +2009-10-14 Tim Englich <tim.englich@intevation.de> + + * src/main/java/de/intevation/gnv/geobackend/sde/datasources/Row.java (getPosValue): + Add WKT-Syntaxt to Shape ReturnValue + * src/test/ressources/QueryExecutorTestCase.properties: + Changed Test-Query-Statement for Spatial-Queries with INNERJOINS and ORDER BY + * src/test/java/de/intevation/gnv/geobackend/base/query/QueryExecutorTestCase.java (testSpatialQueryWithoutIntersects): + Changed expected Result-Size because of Changes in the QueryStatement which is used. + * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (executeQuery): + Added INNERJOIN, ORDER BY and GROUP BY support to Spatial Queries + +2009-10-13 Tim Englich <tim.englich@intevation.de> + + * src/test/java/de/intevation/gnv/geobackend/base/query/QueryExecutorTestCase.java (testSpatialQueryWithoutIntersects): + Added an TaesCase for Querying spatial Data without an spatial restriction + * src/test/ressources/QueryExecutorTestCase.properties: + Added TestQuery with no spatial restriction + * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (executeQuery): + Now it is possible to send spatial Quieries without an spatial restriction in the where clause. + +2009-10-13 Tim Englich <tim.englich@intevation.de> + + Issue 34: SQL Exception in REST Server when accessing geo-backend + + * src/main/java/de/intevation/gnv/geobackend/sde/connectionpool/ArcSDEPoolableObjectFactory.java (validateObject) Edited: + Added Tests if the Object is Closed. And add propper instance Tests + * src/main/java/de/intevation/gnv/geobackend/sde/connectionpool/ArcSDEConnectionPool.java (initialize): + Added further ConfigurationParameters that could be set in the *.properties-File. + maxWait, maxIdle, minIdle, testOnBorrow, testOnReturn, testWhileIdle, numTestsPerEvictionRun, + timeBetweenEvictionRunsMillis, testWhileIdle + * src/test/ressources/ArcSDEConnectionPoolTestCase.properties: + Added som Configurationproperties for testing. + + +2009-10-13 Tim Englich <tim.englich@intevation.de> + * src/test/java/de/intevation/gnv/geobackend/base/query/QueryExecutorTestCase.java (QueryExecutorTestCase): + Changed expected Result of Spatial-Query-TestCase because of Changes to the used Query + * src/test/ressources/QueryExecutorTestCase.properties: + Added SFS-Conformal Query + * pom.xml: + Added dependency to JTS 1.9 + * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (executeQuery): + Added Spatial-Query-Support for ArcSDE + * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEUtils.java (createPoints): + Switched Method Signature to JTS-Geometry-Support + * src/main/java/de/intevation/gnv/geobackend/sde/datasources/Row.java (getStringValue): + Added String-Support for GeometryObjects + +2009-10-07 Sascha L. Teichmann <sascha.teichmann@intevation.de> + + RELEASE 0.1 + + * Changes, NEWS: Summarized changes. + +2009-10-06 Sascha L. Teichmann <sascha.teichmann@intevation.de> + + * Changelog -> ChangeLog: Renamed to make eclipse happy. Unified changelog style. + + * src/main/java/de/intevation/gnv/geobackend/sde/datasources/SDEQuery.java, + src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java: + Removed needless imports. + + * pom.xml: xmllint-ed + +2009-09-29 Tim Englich <tim.englich@intevation.de> + + * src/main/java/de/intevation/gnv/geobackend/util/DateUtils.java (DATE_PATTERN) Edited: + Make the Constants assessible for other Parts of the Project to reduce the TouchPoints for + Dateformats. + +2009-09-24 Tim Englich <tim.englich@intevation.de> + + * src/main/java/de/intevation/gnv/geobackend/base/ResultDescriptor.java Edited, + src/main/java/de/intevation/gnv/geobackend/base/Result.java Edited, + src/main/java/de/intevation/gnv/geobackend/base/DefaultResultDescriptor.java Edited, + src/main/java/de/intevation/gnv/geobackend/base/DefaultResult.java Edited: + Make Classes Serializable for Storing in ArtifactDatabase + +2009-09-21 Tim Englich <tim.englich@intevation.de> + + * src/main/java/de/intevation/gnv/geobackend/base/DefaultResult.java: + Added formatted Date Handling in getString Method + * src/test/ressources/QueryExecutorTestCase.properties Edited, + src/test/java/de/intevation/gnv/geobackend/base/query/QueryExecutorTestCase.java Edited: + Some tests for Spatialqueries added + * src/main/java/de/intevation/gnv/geobackend/base/query/DefaultQueryExceutor.java Edited: + Put the final Querystring into a Logmessage + +2009-09-09 Tim Englich <tim.englich@intevation.de> + + * src/main/java/de/intevation/gnv/geobackend/base/DefaultResult.java (getDate): + . Intance Check for DateValues + * src/main/java/de/intevation/gnv/geobackend/sde/datasources/SDEResultSet.java (next): + Logging removed because the performance slowdown fetching many ResultSets + +2009-09-04 Tim Englich <tim.englich@intevation.de> + + * pom.xml: