Mercurial > dive4elements > gnv-client
view geo-backend/ChangeLog @ 557:05fcb3c553fd 0.4
merged geo-backend/0.4
author | Thomas Arendsen Hein <thomas@intevation.de> |
---|---|
date | Fri, 28 Sep 2012 12:13:50 +0200 |
parents | 48121fa4dc95 |
children | 23a0f2c5e3b8 |
line wrap: on
line source
2010-01-27 Hans Plum <hans@intevation.de> RELEASE 0.4 * Changes, NEWS, ChangeLog: Summarized changes. 2009-01-25 Sascha L. Teichmann <sascha.teichmann@intevation.de> Fix for gnv/issue107 * src/main/java/de/intevation/gnv/geobackend/util/RedundancyRemover.java: New. Used to reduce the redundancy of data in SQL result sets. Column values are very distinct so the degree or redundancy is very high. E.g the SDE produces for each date value a java.util.GregorianCalendar object which are very heavy weighted. We only use them r/o so we can hash each date value to one unique representative. Same logic applys to integer values and so on. With this technique we are able to reduce the memory consumption by over 90%. Because it is not very feasible to store the whole history of column we only have a limited number of cached values per column. This is controlled via the system property "de.intevation.gnv.geobackend.util.RedundancyRemover.lookback" which defaults to 1029. * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java: Use RedundancyRemovers to filter the incoming traffic from SDE. 2010-01-26 Tim Englich <tim.englich@intevation.de> * src/test/java/de/intevation/gnv/geobackend/base/query/ToCharSample.java : Added Sample for bug in ArcSDE 9.3 using to_char-Function with SpatialQuery * src/test/java/de/intevation/gnv/geobackend/base/query/GroupBySample.java : Added Sample for Bug in ArcSDE using group by with SpatialQuery 2009-01-25 Sascha L. Teichmann <sascha.teichmann@intevation.de> Another attempt to fix gnv/issue34 * src/main/java/de/intevation/gnv/geobackend/base/DefaultResultDescriptor.java: Add a toString() method. * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEConnection.java: Added an idle time detection mechanism. You can call touch() on a connection to refresh the internal timestamp and ask with isActive() if a constructor given time interval in milliseconds is exceeded. * src/main/java/de/intevation/gnv/geobackend/sde/connectionpool/ArcSDEPoolableObjectFactory.java: Added a check of inactivity to validateObject() before relying on SDE logic. This is the last line of defence if the testServer() call is going not to respond. This duplicates the eviction policy of Apache Commons pool a bit but I found this way to be more trustworthy. The idle time is configure with the property 'serverInactiveInterval'. units: seconds. Defaults to 5 minutes. The unit of 'serverRoundtripInterval' (parameter of testServer()) is now in seconds, too. Default: 5 seconds. This is the value used in GeoTools. 2009-01-09 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/gnv/geobackend/sde/datasources/SDEQuery.java, src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java, src/main/java/de/intevation/gnv/geobackend/sde/datasources/RasterObject.java: Cleanup imports. 2010-01-15 Tim Englich <tim.englich@intevation.de> * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEConnection.java (isValid): Implemented the isValid-method of the ArcSDEConnection for getting a propper Validation of the used SEConnection. For the Validation the Method testServer of the SEConnection is used. * src/main/java/de/intevation/gnv/geobackend/sde/connectionpool/ArcSDEPoolableObjectFactory.java (ArcSDEPoolableObjectFactory): Added Member serverRoundtripInterval to represent the Time in Milliseconds which is allowed to reused a returned Connection. Also used Connection.isValid Method instead of Connection.isClosed to validate the Connection befor it could be borrowed from the Pool. 2009-01-12 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/gnv/geobackend/sde/datasources/RasterObject.java: Added bicubic spline interpolation on raster tiles. This should increase the 'virtual' resolution more than the bilinear interpolation. To enable it write 'bicubic' into the configuration file. * pom.xml: Added dependency to Apache Common Math 2.0 for the cubic spline interpolation. 2009-01-09 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/gnv/geobackend/sde/datasources/RasterObject.java: Added static method to fetch matching interpolation type for given string. Defaults to nearest neighbor if no string is matching. 2009-01-09 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/gnv/geobackend/sde/datasources/RasterObject.java: Implemented bilinear interpolation. 2009-01-09 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java: Fixed transformations to figure out the right tile in the SDE. Taking the tile origin into account now and does not do wrong rounding of column and row indices. Simplified the code a lot. * src/main/java/de/intevation/gnv/geobackend/sde/datasources/RasterObject.java: Fixed access to right right pixel now. Removed the envelope code because its not needed any longer. You can ask a RasterObject directly now if a given coordinate is on the tile now. The tile space is now correctly spanned between (0, 0) and (tile width - 1, tile height - 1) now which allows other access patterns like bilinear interpolation. TODO: implement bilinear interpolation. 2010-01-08 Tim Englich <tim.englich@intevation.de> * src/main/java/de/intevation/gnv/geobackend/sde/datasources/SDEQuery.java, src/main/java/de/intevation/gnv/geobackend/sde/datasources/Row.java, src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java, src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEConnection.java, src/main/java/de/intevation/gnv/geobackend/base/connectionpool/ConnectionPoolFactory.java (initializeConnectionPool): Removed Encodingerrors from listed Files. All Files are now UTF-8 compliant. 2010-01-05 Tim Englich <tim.englich@intevation.de> * src/main/java/de/intevation/gnv/geobackend/sde/datasources/RasterObject.java: Made it serializable to be cachable beyond reboot. 2010-01-05 Tim Englich <tim.englich@intevation.de> * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (handleResultSet): Switched to return the RasterObject instead of the depth in an Rasterquery. This should help to get a better performance. * src/main/java/de/intevation/gnv/geobackend/sde/datasources/RasterObject.java (RasterObject): Added new Object for Caching Rastertiles and all Informations that are required for calculating the Depth to a given Coordinate. 2010-01-04 Tim Englich <tim.englich@intevation.de> * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (handleResultSet): Added Rasterdatasupport to the ARCSDESTATEMENT. Now it is possible to Query the Value of an Pixel like that: SELECT ST_ASTEXT(RASTER) FROM MEDIAN.TOPO_BS_D_6X10SEC where INTERSECTS(RASTER, "POINT( 12.084721594184593 54.35583351483295)") 2009-12-19 Sascha L. Teichmann <sascha.teichmann@intevation.de> * contrib/sql-cache.xml: Configured disk storage to survive restarts. * src/main/java/de/intevation/gnv/geobackend/base/query/QueryExecutorFactory.java: Added explicit shutdown hook. * src/main/java/de/intevation/gnv/geobackend/base/query/CachingQueryExecutorFactory.java: Fixed spelling of cacheResults() which prevented results to be stored in cache. Added some logging. 2009-12-18 Sascha L. Teichmann <sascha.teichmann@intevation.de> Experimental caching of SQL results via Ehcache * pom.xml: Added dependency to Ehcache * src/main/java/de/intevation/gnv/geobackend/base/query/QueryExecutor.java: Extended interface to support caching by SQL statements as Strings. * src/main/java/de/intevation/gnv/geobackend/base/query/QueryExecutorBase.java: Implements new interface with no caching * src/main/java/de/intevation/gnv/geobackend/base/query/container/QueryContainerFactory.java: synchronized access to singleton. * src/main/java/de/intevation/gnv/geobackend/base/query/DefaultQueryExceutor.java: calls caching methods of base class. * src/main/java/de/intevation/gnv/geobackend/base/query/QueryExecutorFactory.java: When system property 'query.executor.factory' is given the value is used as a class name to be instantiated as a sub class of QueryExecutorFactory. This happens if the singleton of QueryExecutorFactory is created. With the mechanism it is possible to replace the standard behavior of creating QueryExecutors. * src/main/java/de/intevation/gnv/geobackend/base/query/CachingQueryExecutorFactory.java: New: This class extends QueryExecutorFactory and can be used as replacement for the standard QueryExecutorFactory. Usage: -Dquery.executor.factory=de.intevation.gnv.geobackend.base.query.CachingQueryExecutorFactory If you pass in another system property 'caching.query.executor.config' you can customize the caching. The value is used as a file name to an XML file with Ehcache configuration. * contrib/sql-cache.xml: Demo configuration for sql caching. Usage: -Dquery.executor.factory=de.intevation.gnv.geobackend.base.query.CachingQueryExecutorFactory -Dcaching.query.executor.config=contrib/sql-cache.xml 2009-12-17 Sascha L. Teichmann <sascha.teichmann@intevation.de> RELEASE 0.3 * Changes, NEWS, ChangeLog: Summarized changes. 2009-12-17 Ingo Weinzierl <ingo.weinzierl@intevation.de> * src/main/java/de/intevation/gnv/geobackend/base/DefaultResult.java: Fixed bug in DefaultResult when querying an index out of bounds. 2009-12-15 Tim Englich <tim.englich@intevation.de> * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (executeQuery): We have to use a different SpatialFilter for Point in Polygon Queries as in AreaIntersection Statements. So we have to include a switch which will do that. * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEUtils.java (createPoints): Added Support of Point-Objects for the possibility to execute a Point in Pologon Statement using the ArcSDE. 2009-12-11 Ingo Weinzierl <ingo.weinzierl@intevation.de> * src/main/java/de/intevation/gnv/geobackend/base/DefaultResult.java: DefaultResult will return null if it doesn't contain the specified column name in method getString(String columnName), instead of throwing an exception. 2009-12-11 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/gnv/geobackend/base/DefaultResultDescriptor.java, src/main/java/de/intevation/gnv/geobackend/base/query/container/QueryContainer.java, src/main/java/de/intevation/gnv/geobackend/base/query/container/DefaultQueryContainer.java: Removed needless imports. 2009-11-30 Tim Englich <tim.englich@intevation.de> * src/main/java/de/intevation/gnv/geobackend/sde/datasources/SDEResultSet.java, src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java, src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEResultSetMetaData.java, src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEConnection.java: Added missing Methodstubs for Java 1.6 compability. ################################################### ###### ATTENTION: ONLY USE JAVA 1.6 FOR THIS ###### ###### AND ALL USING MODULES. ###### ################################################### 2009-11-27 Tim Englich <tim.englich@intevation.de> * src/test/ressources/QueryExecutorTestCase.properties, src/test/java/de/intevation/gnv/geobackend/base/query/QueryExecutorTestCase.java: Revert Changes of the commit of Revision 377 which was unintended * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (executeQuery): Changed Splitalgorithm of Statements so that it is Possible to add to_date Colums into a Statement with Spatial-Columns or Spatial-Filters 2009-11-13 Hans Plum <hans@intevation.de> RELEASE 0.2 * Changes, NEWS: Summarized changes. 2009-11-13 Hans Plum <hans@intevation.de> * Made some ChangeLog entries a bit clearer. 2009-11-10 Tim Englich <tim.englich@intevation.de> * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (executeQuery): Changed the splitting of the Statements for ARCSDE-Transformation. Now it is possible to use a Select-Statement as an Table. 2009-11-06 Ingo Weinzierl <ingo.weinzierl@intevation.de> * target/**: Removed. Should not be in version control. 2009-11-05 Tim Englich <tim.englich@intevation.de> * src/test/ressources/QueryExecutorTestCase.properties, src/test/java/de/intevation/gnv/geobackend/base/query/QueryExecutorTestCase.java (testSpatialQueryWithInnerSelect): Added an Tescase to test the InnerSelect Statments in Spatial-Queries. * src/main/java/de/intevation/gnv/geobackend/sde/datasources/Row.java (getPosValue): Bugfix. An Point could not have an Z-Coordinate-Value * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (executeQuery): Added the possibility to execute Spatial-Queries using InnerStatements instead of WKT in Intersects-where-clauses. 2009-11-02 Tim Englich <tim.englich@intevation.de> * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEConnection.java (createStatement): Changed TODO to a question * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (executeQuery): CodeCleanup: Removed deprecated TODO * src/main/java/de/intevation/gnv/geobackend/sde/datasources/SDEResultSet.java (getColumnIndex): Removed Typo. * src/main/java/de/intevation/gnv/geobackend/sde/connectionpool/ArcSDEPoolableObjectFactory.java: CodeCleanup: Removed deprecated TODOS and replace Tabs with whitespaces. * src/main/java/de/intevation/gnv/geobackend/base/query/container/DefaultQueryContainer.java:, src/main/java/de/intevation/gnv/geobackend/base/query/container/QueryContainer.java: CodeCleanup: Removed unused Method getQueryIds * src/main/java/de/intevation/gnv/geobackend/sde/connectionpool/ArcSDEConnectionPool.java (closeConnection): CodeCleanup: Inspect TODO and ansewerd the Question that the Connection has not to be closed at that position but perhaps insite the ConnectionPool. 2009-10-30 Tim Englich <tim.englich@intevation.de> * src/main/java/de/intevation/gnv/geobackend/sde/datasources/Row.java (getPosValue): Added Support for Polygons without interior Rings. This is required for generating Horizontal Cross-Section Outputs. The Z-Value of an Point was also added to the WKT-Represantation. 2009-10-29 Tim Englich <tim.englich@intevation.de> Issue 45: Defintion of gaps in temporal and spatial data * src/main/java/de/intevation/gnv/geobackend/base/DefaultResult.java (getInteger): Added Integer-Support for DoubleValues because the ArcSDE does not read values like -1 as Integer-values. issue45 2009-10-20 Tim Englich <tim.englich@intevation.de> Issue 57: Improving memory usage of data objects * src/main/java/de/intevation/gnv/geobackend/base/query/QueryExecutorBase.java (QueryExecutorBase), src/main/java/de/intevation/gnv/geobackend/base/ResultDescriptor.java (getColumnClassName), src/main/java/de/intevation/gnv/geobackend/base/Result.java, src/main/java/de/intevation/gnv/geobackend/base/DefaultResultDescriptor.java (serialVersionUID), src/main/java/de/intevation/gnv/geobackend/base/DefaultResult.java (getDate): Integrated Patch of issue57 to get some Memoryusage-improvements 2009-10-19 Sascha L. Teichmann <sascha.teichmann@intevation.de> * src/main/java/de/intevation/gnv/geobackend/base/DefaultResult.java: Extends from HashMap directly to avoid one level of indirection and do not create so much needless objects. TODO: The representation in form of HashMaps is not very efficent! Use array with index instead. 2009-10-14 Tim Englich <tim.englich@intevation.de> * src/test/java/de/intevation/gnv/geobackend/base/query/QueryExecutorTestCase.java (testSpatialQueryWithoutIntersects), src/test/ressources/QueryExecutorTestCase.properties: Added further Queries for Testing Spatial-Query-Support 2009-10-14 Tim Englich <tim.englich@intevation.de> * src/main/java/de/intevation/gnv/geobackend/sde/datasources/Row.java (getPosValue): Add WKT-Syntaxt to Shape ReturnValue * src/test/ressources/QueryExecutorTestCase.properties: Changed Test-Query-Statement for Spatial-Queries with INNERJOINS and ORDER BY * src/test/java/de/intevation/gnv/geobackend/base/query/QueryExecutorTestCase.java (testSpatialQueryWithoutIntersects): Changed expected Result-Size because of Changes in the QueryStatement which is used. * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (executeQuery): Added INNERJOIN, ORDER BY and GROUP BY support to Spatial Queries 2009-10-13 Tim Englich <tim.englich@intevation.de> * src/test/java/de/intevation/gnv/geobackend/base/query/QueryExecutorTestCase.java (testSpatialQueryWithoutIntersects): Added an TaesCase for Querying spatial Data without an spatial restriction * src/test/ressources/QueryExecutorTestCase.properties: Added TestQuery with no spatial restriction * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (executeQuery): Now it is possible to send spatial Quieries without an spatial restriction in the where clause. 2009-10-13 Tim Englich <tim.englich@intevation.de> Issue 34: SQL Exception in REST Server when accessing geo-backend * src/main/java/de/intevation/gnv/geobackend/sde/connectionpool/ArcSDEPoolableObjectFactory.java (validateObject) Edited: Added Tests if the Object is Closed. And add propper instance Tests * src/main/java/de/intevation/gnv/geobackend/sde/connectionpool/ArcSDEConnectionPool.java (initialize): Added further ConfigurationParameters that could be set in the *.properties-File. maxWait, maxIdle, minIdle, testOnBorrow, testOnReturn, testWhileIdle, numTestsPerEvictionRun, timeBetweenEvictionRunsMillis, testWhileIdle * src/test/ressources/ArcSDEConnectionPoolTestCase.properties: Added som Configurationproperties for testing. 2009-10-13 Tim Englich <tim.englich@intevation.de> * src/test/java/de/intevation/gnv/geobackend/base/query/QueryExecutorTestCase.java (QueryExecutorTestCase): Changed expected Result of Spatial-Query-TestCase because of Changes to the used Query * src/test/ressources/QueryExecutorTestCase.properties: Added SFS-Conformal Query * pom.xml: Added dependency to JTS 1.9 * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (executeQuery): Added Spatial-Query-Support for ArcSDE * src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEUtils.java (createPoints): Switched Method Signature to JTS-Geometry-Support * src/main/java/de/intevation/gnv/geobackend/sde/datasources/Row.java (getStringValue): Added String-Support for GeometryObjects 2009-10-07 Sascha L. Teichmann <sascha.teichmann@intevation.de> RELEASE 0.1 * Changes, NEWS: Summarized changes. 2009-10-06 Sascha L. Teichmann <sascha.teichmann@intevation.de> * Changelog -> ChangeLog: Renamed to make eclipse happy. Unified changelog style. * src/main/java/de/intevation/gnv/geobackend/sde/datasources/SDEQuery.java, src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java: Removed needless imports. * pom.xml: xmllint-ed 2009-09-29 Tim Englich <tim.englich@intevation.de> * src/main/java/de/intevation/gnv/geobackend/util/DateUtils.java (DATE_PATTERN) Edited: Make the Constants assessible for other Parts of the Project to reduce the TouchPoints for Dateformats. 2009-09-24 Tim Englich <tim.englich@intevation.de> * src/main/java/de/intevation/gnv/geobackend/base/ResultDescriptor.java Edited, src/main/java/de/intevation/gnv/geobackend/base/Result.java Edited, src/main/java/de/intevation/gnv/geobackend/base/DefaultResultDescriptor.java Edited, src/main/java/de/intevation/gnv/geobackend/base/DefaultResult.java Edited: Make Classes Serializable for Storing in ArtifactDatabase 2009-09-21 Tim Englich <tim.englich@intevation.de> * src/main/java/de/intevation/gnv/geobackend/base/DefaultResult.java: Added formatted Date Handling in getString Method * src/test/ressources/QueryExecutorTestCase.properties Edited, src/test/java/de/intevation/gnv/geobackend/base/query/QueryExecutorTestCase.java Edited: Some tests for Spatialqueries added * src/main/java/de/intevation/gnv/geobackend/base/query/DefaultQueryExceutor.java Edited: Put the final Querystring into a Logmessage 2009-09-09 Tim Englich <tim.englich@intevation.de> * src/main/java/de/intevation/gnv/geobackend/base/DefaultResult.java (getDate): . Intance Check for DateValues * src/main/java/de/intevation/gnv/geobackend/sde/datasources/SDEResultSet.java (next): Logging removed because the performance slowdown fetching many ResultSets 2009-09-04 Tim Englich <tim.englich@intevation.de> * pom.xml: