view geo-backend/ChangeLog @ 557:05fcb3c553fd 0.4

merged geo-backend/0.4
author Thomas Arendsen Hein <thomas@intevation.de>
date Fri, 28 Sep 2012 12:13:50 +0200
parents 48121fa4dc95
children 23a0f2c5e3b8
line wrap: on
line source
2010-01-27  Hans Plum <hans@intevation.de>

	RELEASE 0.4

	* Changes, NEWS, ChangeLog: Summarized changes.

2009-01-25	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	Fix for gnv/issue107

	* src/main/java/de/intevation/gnv/geobackend/util/RedundancyRemover.java:
	  New. Used to reduce the redundancy of data in SQL result sets. Column
	  values are very distinct so the degree or redundancy is very high. E.g 
	  the SDE produces for each date value a java.util.GregorianCalendar object
	  which are very heavy weighted. We only use them r/o so we can hash each date 
	  value to one unique representative. Same logic applys to integer values 
	  and so on.
	  
	  With this technique we are able to reduce the memory consumption by
	  over 90%. Because it is not very feasible to store the whole history
	  of column we only have a limited number of cached values per column.
	  This is controlled via the system property
	  "de.intevation.gnv.geobackend.util.RedundancyRemover.lookback" which
	  defaults to 1029.

	* src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java:
	  Use RedundancyRemovers to filter the incoming traffic from SDE.

2010-01-26  Tim Englich  <tim.englich@intevation.de>

	* src/test/java/de/intevation/gnv/geobackend/base/query/ToCharSample.java : 
	  Added Sample for bug in ArcSDE 9.3 using to_char-Function with SpatialQuery
	* src/test/java/de/intevation/gnv/geobackend/base/query/GroupBySample.java : 
	  Added Sample for Bug in ArcSDE using group by with SpatialQuery

2009-01-25	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	Another attempt to fix gnv/issue34

	* src/main/java/de/intevation/gnv/geobackend/base/DefaultResultDescriptor.java:
	  Add a toString() method.

	* src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEConnection.java:
	  Added an idle time detection mechanism. You can call touch() on
	  a connection to refresh the internal timestamp and ask with isActive()
	  if a constructor given time interval in milliseconds is exceeded.

	* src/main/java/de/intevation/gnv/geobackend/sde/connectionpool/ArcSDEPoolableObjectFactory.java:
	  Added a check of inactivity to validateObject() before relying on SDE logic.
	  This is the last line of defence if the testServer() call is going not to respond.
	  This duplicates the eviction policy of Apache Commons pool a bit but I found
	  this way to be more trustworthy.

	  The idle time is configure with the property 'serverInactiveInterval'.
	  units: seconds. Defaults to 5 minutes.

	  The unit of 'serverRoundtripInterval' (parameter of testServer()) is now
	  in seconds, too. Default: 5 seconds. This is the value used in GeoTools.

2009-01-09	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* src/main/java/de/intevation/gnv/geobackend/sde/datasources/SDEQuery.java,
	  src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java,
	  src/main/java/de/intevation/gnv/geobackend/sde/datasources/RasterObject.java:
	  Cleanup imports.

2010-01-15  Tim Englich  <tim.englich@intevation.de>

	* src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEConnection.java (isValid): 
	  Implemented the isValid-method of the ArcSDEConnection for 
	  getting a propper Validation of the used SEConnection.
	  For the Validation the Method testServer of the SEConnection is used.
	
	* src/main/java/de/intevation/gnv/geobackend/sde/connectionpool/ArcSDEPoolableObjectFactory.java (ArcSDEPoolableObjectFactory): 
	  Added Member serverRoundtripInterval to represent the Time in 
	  Milliseconds which is allowed to reused a returned Connection.
	  Also used Connection.isValid Method instead of Connection.isClosed 
	  to validate the Connection befor it could be borrowed from the Pool.

2009-01-12	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* src/main/java/de/intevation/gnv/geobackend/sde/datasources/RasterObject.java:
	  Added bicubic spline interpolation on raster tiles. This
	  should increase the 'virtual' resolution more than the
	  bilinear interpolation. To enable it write 'bicubic' into
	  the configuration file.

	* pom.xml: Added dependency to Apache Common Math 2.0 for
	  the cubic spline interpolation.

2009-01-09	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* src/main/java/de/intevation/gnv/geobackend/sde/datasources/RasterObject.java:
	  Added static method to fetch matching interpolation type for
	  given string. Defaults to nearest neighbor if no string is matching.

2009-01-09	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* src/main/java/de/intevation/gnv/geobackend/sde/datasources/RasterObject.java:
	  Implemented bilinear interpolation.

2009-01-09	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java:
	  Fixed transformations to figure out the right tile in the SDE.
	  Taking the tile origin into account now and does not do wrong rounding
	  of column and row indices. Simplified the code a lot.

	* src/main/java/de/intevation/gnv/geobackend/sde/datasources/RasterObject.java:
	  Fixed access to right right pixel now. Removed the envelope code because
	  its not needed any longer. You can ask a RasterObject directly now if
	  a given coordinate is on the tile now. The tile space is now correctly
	  spanned between (0, 0) and (tile width - 1, tile height - 1) now which
	  allows other access patterns like bilinear interpolation. TODO: implement
	  bilinear interpolation.

2010-01-08  Tim Englich  <tim.englich@intevation.de>

	* src/main/java/de/intevation/gnv/geobackend/sde/datasources/SDEQuery.java,
	  src/main/java/de/intevation/gnv/geobackend/sde/datasources/Row.java,
	  src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java, 
	  src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEConnection.java, 
	  src/main/java/de/intevation/gnv/geobackend/base/connectionpool/ConnectionPoolFactory.java (initializeConnectionPool): 
	  Removed Encodingerrors from listed Files. All Files are now UTF-8 compliant.

2010-01-05  Tim Englich  <tim.englich@intevation.de>

	* src/main/java/de/intevation/gnv/geobackend/sde/datasources/RasterObject.java:
	  Made it serializable to be cachable beyond reboot.

2010-01-05  Tim Englich  <tim.englich@intevation.de>

	* src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (handleResultSet): 
	  Switched to return the RasterObject instead of the depth in an Rasterquery.
	  This should help to get a better performance.
	* src/main/java/de/intevation/gnv/geobackend/sde/datasources/RasterObject.java (RasterObject): 
	  Added new Object for Caching Rastertiles and all Informations that are
	  required for calculating the Depth to a given Coordinate.

2010-01-04  Tim Englich  <tim.englich@intevation.de>

	* src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (handleResultSet): 
	  Added Rasterdatasupport to the ARCSDESTATEMENT.
	  Now it is possible to Query the Value of an Pixel like that:
	  SELECT ST_ASTEXT(RASTER) FROM MEDIAN.TOPO_BS_D_6X10SEC where INTERSECTS(RASTER, "POINT( 12.084721594184593 54.35583351483295)")

2009-12-19	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* contrib/sql-cache.xml: Configured disk storage to survive restarts.

	* src/main/java/de/intevation/gnv/geobackend/base/query/QueryExecutorFactory.java:
	  Added explicit shutdown hook.

	* src/main/java/de/intevation/gnv/geobackend/base/query/CachingQueryExecutorFactory.java:
	  Fixed spelling of cacheResults() which prevented results to be stored in cache.
	  Added some logging.

2009-12-18	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	Experimental caching of SQL results via Ehcache

	* pom.xml: Added dependency to Ehcache

	* src/main/java/de/intevation/gnv/geobackend/base/query/QueryExecutor.java:
	  Extended interface to support caching by SQL statements as Strings.

	* src/main/java/de/intevation/gnv/geobackend/base/query/QueryExecutorBase.java:
	  Implements new interface with no caching
	
	* src/main/java/de/intevation/gnv/geobackend/base/query/container/QueryContainerFactory.java:
	  synchronized access to singleton.

	* src/main/java/de/intevation/gnv/geobackend/base/query/DefaultQueryExceutor.java:
	  calls caching methods of base class.

	* src/main/java/de/intevation/gnv/geobackend/base/query/QueryExecutorFactory.java:
	  When system property 'query.executor.factory' is given the value is used as
	  a class name to be instantiated as a sub class of QueryExecutorFactory.
	  This happens if the singleton of QueryExecutorFactory is created. With the
	  mechanism it is possible to replace the standard behavior of creating QueryExecutors.

	* src/main/java/de/intevation/gnv/geobackend/base/query/CachingQueryExecutorFactory.java:
	  New: This class extends QueryExecutorFactory and can be used as replacement for
	  the standard QueryExecutorFactory. Usage: 

	  -Dquery.executor.factory=de.intevation.gnv.geobackend.base.query.CachingQueryExecutorFactory

	  If you pass in another system property 'caching.query.executor.config' you can
	  customize the caching. The value is used as a file name to an XML file with 
	  Ehcache configuration.

	* contrib/sql-cache.xml: Demo configuration for sql caching. Usage:

	  -Dquery.executor.factory=de.intevation.gnv.geobackend.base.query.CachingQueryExecutorFactory
	  -Dcaching.query.executor.config=contrib/sql-cache.xml

2009-12-17	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	RELEASE 0.3

	* Changes, NEWS, ChangeLog: Summarized changes.

2009-12-17  Ingo Weinzierl <ingo.weinzierl@intevation.de>

	* src/main/java/de/intevation/gnv/geobackend/base/DefaultResult.java:
	  Fixed bug in DefaultResult when querying an index out of bounds.

2009-12-15  Tim Englich  <tim.englich@intevation.de>

	* src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (executeQuery): 
	  We have to use a different SpatialFilter for Point in Polygon Queries as 
	  in AreaIntersection Statements.
	  So we have to include a switch which will do that.
	* src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEUtils.java (createPoints): 
	  Added Support of Point-Objects for the possibility to execute a Point in Pologon 
	  Statement using the ArcSDE.

2009-12-11  Ingo Weinzierl <ingo.weinzierl@intevation.de>

	* src/main/java/de/intevation/gnv/geobackend/base/DefaultResult.java:
	  DefaultResult will return null if it doesn't contain the specified column
	  name in method getString(String columnName), instead of throwing an
	  exception.

2009-12-11	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* src/main/java/de/intevation/gnv/geobackend/base/DefaultResultDescriptor.java,
	src/main/java/de/intevation/gnv/geobackend/base/query/container/QueryContainer.java,
	src/main/java/de/intevation/gnv/geobackend/base/query/container/DefaultQueryContainer.java:
	Removed needless imports.

2009-11-30  Tim Englich  <tim.englich@intevation.de>

	* src/main/java/de/intevation/gnv/geobackend/sde/datasources/SDEResultSet.java,
	  src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java, 
	  src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEResultSetMetaData.java, 
	  src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEConnection.java: 
	  Added missing Methodstubs for Java 1.6 compability.
	  ###################################################
	  ###### ATTENTION: ONLY USE JAVA 1.6 FOR THIS ######
	  ######         AND ALL USING MODULES.        ######
	  ###################################################

2009-11-27  Tim Englich  <tim.englich@intevation.de>

	* src/test/ressources/QueryExecutorTestCase.properties,
	  src/test/java/de/intevation/gnv/geobackend/base/query/QueryExecutorTestCase.java: 
	  Revert Changes of the commit of Revision 377 which was unintended
	* src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (executeQuery): 
	  Changed Splitalgorithm of Statements so that it is Possible to add to_date Colums into a Statement
	  with Spatial-Columns or Spatial-Filters

2009-11-13  Hans Plum <hans@intevation.de>

	RELEASE 0.2

	* Changes, NEWS: Summarized changes.

2009-11-13  Hans Plum <hans@intevation.de>

	* Made some ChangeLog entries a bit clearer.

2009-11-10  Tim Englich  <tim.englich@intevation.de>

	* src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (executeQuery): 
	  Changed the splitting of the Statements for ARCSDE-Transformation.
	  Now it is possible to use a Select-Statement as an Table.

2009-11-06  Ingo Weinzierl <ingo.weinzierl@intevation.de>

	* target/**: Removed. Should not be in version control.

2009-11-05  Tim Englich  <tim.englich@intevation.de>

	* src/test/ressources/QueryExecutorTestCase.properties,
	  src/test/java/de/intevation/gnv/geobackend/base/query/QueryExecutorTestCase.java (testSpatialQueryWithInnerSelect):
	  Added an Tescase to test the InnerSelect Statments in Spatial-Queries.
	* src/main/java/de/intevation/gnv/geobackend/sde/datasources/Row.java (getPosValue): 
	  Bugfix. An Point could not have an Z-Coordinate-Value
	* src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (executeQuery): 
	  Added the possibility to execute Spatial-Queries using InnerStatements instead
	  of WKT in Intersects-where-clauses.

2009-11-02  Tim Englich  <tim.englich@intevation.de>

	* src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEConnection.java (createStatement): 
	  Changed TODO to a question
	* src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (executeQuery): 
	  CodeCleanup: Removed deprecated TODO
	* src/main/java/de/intevation/gnv/geobackend/sde/datasources/SDEResultSet.java (getColumnIndex): 
	  Removed Typo.
	* src/main/java/de/intevation/gnv/geobackend/sde/connectionpool/ArcSDEPoolableObjectFactory.java: 
	  CodeCleanup: Removed deprecated TODOS and replace Tabs with whitespaces.
	* src/main/java/de/intevation/gnv/geobackend/base/query/container/DefaultQueryContainer.java:,
	  src/main/java/de/intevation/gnv/geobackend/base/query/container/QueryContainer.java:
	  CodeCleanup: Removed unused Method getQueryIds
	* src/main/java/de/intevation/gnv/geobackend/sde/connectionpool/ArcSDEConnectionPool.java (closeConnection): 
	  CodeCleanup: Inspect TODO and ansewerd the Question that the Connection has not to
	  be closed at that position but perhaps insite the ConnectionPool. 

2009-10-30  Tim Englich  <tim.englich@intevation.de>

	* src/main/java/de/intevation/gnv/geobackend/sde/datasources/Row.java (getPosValue): 
	  Added Support for Polygons without interior Rings. This is required for generating
	  Horizontal Cross-Section Outputs.
	  The Z-Value of an Point was also added to the WKT-Represantation.

2009-10-29  Tim Englich  <tim.englich@intevation.de>

	Issue 45: Defintion of gaps in temporal and spatial data

	* src/main/java/de/intevation/gnv/geobackend/base/DefaultResult.java (getInteger): 
	  Added Integer-Support for DoubleValues because the ArcSDE does 
	  not read values like -1 as Integer-values. issue45

2009-10-20  Tim Englich  <tim.englich@intevation.de>

	Issue 57: Improving memory usage of data objects

	* src/main/java/de/intevation/gnv/geobackend/base/query/QueryExecutorBase.java (QueryExecutorBase),
	  src/main/java/de/intevation/gnv/geobackend/base/ResultDescriptor.java (getColumnClassName),
	  src/main/java/de/intevation/gnv/geobackend/base/Result.java, 
	  src/main/java/de/intevation/gnv/geobackend/base/DefaultResultDescriptor.java (serialVersionUID), 
	  src/main/java/de/intevation/gnv/geobackend/base/DefaultResult.java (getDate):
	  Integrated Patch of issue57 to get some Memoryusage-improvements  

2009-10-19	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* src/main/java/de/intevation/gnv/geobackend/base/DefaultResult.java:
	Extends from HashMap directly to avoid one level of indirection and
	do not create so much needless objects.

	TODO: The representation in form of HashMaps is not very efficent!
	Use array with index instead.

2009-10-14  Tim Englich  <tim.englich@intevation.de>

	* src/test/java/de/intevation/gnv/geobackend/base/query/QueryExecutorTestCase.java (testSpatialQueryWithoutIntersects), 
	  src/test/ressources/QueryExecutorTestCase.properties: 
	  Added further Queries for Testing Spatial-Query-Support
	  
2009-10-14  Tim Englich  <tim.englich@intevation.de>

	* src/main/java/de/intevation/gnv/geobackend/sde/datasources/Row.java (getPosValue): 
	  Add WKT-Syntaxt to Shape ReturnValue
	* src/test/ressources/QueryExecutorTestCase.properties: 
	  Changed Test-Query-Statement for Spatial-Queries with INNERJOINS and ORDER BY
	* src/test/java/de/intevation/gnv/geobackend/base/query/QueryExecutorTestCase.java (testSpatialQueryWithoutIntersects): 
	  Changed expected Result-Size because of Changes in the QueryStatement which is used.
	* src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (executeQuery): 
	  Added INNERJOIN, ORDER BY and GROUP BY support to Spatial Queries

2009-10-13  Tim Englich  <tim.englich@intevation.de>

	* src/test/java/de/intevation/gnv/geobackend/base/query/QueryExecutorTestCase.java (testSpatialQueryWithoutIntersects): 
	  Added an TaesCase for Querying spatial Data without an spatial restriction
	* src/test/ressources/QueryExecutorTestCase.properties: 
	  Added TestQuery with no spatial restriction
	* src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (executeQuery): 
	  Now it is possible to send spatial Quieries without an spatial restriction in the where clause.
	
2009-10-13  Tim Englich  <tim.englich@intevation.de>

	Issue 34: SQL Exception in REST Server when accessing geo-backend

	* src/main/java/de/intevation/gnv/geobackend/sde/connectionpool/ArcSDEPoolableObjectFactory.java (validateObject) Edited:
	  Added Tests if the Object is Closed. And add propper instance Tests 
	* src/main/java/de/intevation/gnv/geobackend/sde/connectionpool/ArcSDEConnectionPool.java (initialize):
	  Added further ConfigurationParameters that could be  set in the *.properties-File.
	  maxWait, maxIdle, minIdle, testOnBorrow, testOnReturn, testWhileIdle, numTestsPerEvictionRun, 
	  timeBetweenEvictionRunsMillis, testWhileIdle 
	* src/test/ressources/ArcSDEConnectionPoolTestCase.properties: 
	  Added som Configurationproperties for testing.
	  
	  
2009-10-13  Tim Englich  <tim.englich@intevation.de>
	* src/test/java/de/intevation/gnv/geobackend/base/query/QueryExecutorTestCase.java (QueryExecutorTestCase):
	  Changed expected Result of Spatial-Query-TestCase because of Changes to the used Query 
	* src/test/ressources/QueryExecutorTestCase.properties: 
	  Added SFS-Conformal Query
	* pom.xml:
	  Added dependency to JTS 1.9
	* src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java (executeQuery): 
	  Added Spatial-Query-Support for ArcSDE
	* src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEUtils.java (createPoints): 
	  Switched Method Signature to JTS-Geometry-Support
	* src/main/java/de/intevation/gnv/geobackend/sde/datasources/Row.java (getStringValue): 
	  Added String-Support for GeometryObjects

2009-10-07	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	RELEASE 0.1

	* Changes, NEWS: Summarized changes.

2009-10-06	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* Changelog -> ChangeLog: Renamed to make eclipse happy. Unified changelog style.

	* src/main/java/de/intevation/gnv/geobackend/sde/datasources/SDEQuery.java,
	  src/main/java/de/intevation/gnv/geobackend/sde/datasources/ArcSDEStatement.java:
	  Removed needless imports.

	* pom.xml: xmllint-ed

2009-09-29	Tim Englich	<tim.englich@intevation.de>

	* src/main/java/de/intevation/gnv/geobackend/util/DateUtils.java (DATE_PATTERN) Edited:
	  Make the Constants assessible for other Parts of the Project to reduce the TouchPoints for
	  Dateformats.

2009-09-24  Tim Englich  <tim.englich@intevation.de>

	* src/main/java/de/intevation/gnv/geobackend/base/ResultDescriptor.java Edited, 
	  src/main/java/de/intevation/gnv/geobackend/base/Result.java Edited, 
	  src/main/java/de/intevation/gnv/geobackend/base/DefaultResultDescriptor.java Edited,  
	  src/main/java/de/intevation/gnv/geobackend/base/DefaultResult.java Edited:
	  Make Classes Serializable for Storing in ArtifactDatabase 

2009-09-21  Tim Englich  <tim.englich@intevation.de>

	* src/main/java/de/intevation/gnv/geobackend/base/DefaultResult.java: 
	  Added formatted Date Handling in getString Method
	* src/test/ressources/QueryExecutorTestCase.properties Edited, 
	  src/test/java/de/intevation/gnv/geobackend/base/query/QueryExecutorTestCase.java Edited:
	  Some tests for Spatialqueries added 
	* src/main/java/de/intevation/gnv/geobackend/base/query/DefaultQueryExceutor.java Edited: 
	  Put the final Querystring into a Logmessage

2009-09-09  Tim Englich  <tim.englich@intevation.de>

	* src/main/java/de/intevation/gnv/geobackend/base/DefaultResult.java (getDate):
	 . Intance Check for DateValues
	* src/main/java/de/intevation/gnv/geobackend/sde/datasources/SDEResultSet.java (next):
	  Logging removed because the performance slowdown fetching many ResultSets 

2009-09-04  Tim Englich  <tim.englich@intevation.de>

	* pom.xml: 

http://dive4elements.wald.intevation.org