view Changelog @ 32:c2d53bd30ab8

Re-factored artifact API for better integration of background processing. artifacts/trunk@78 c6561f87-3c4e-4783-a992-168aeb5c3f6f
author Sascha L. Teichmann <sascha.teichmann@intevation.de>
date Sun, 13 Sep 2009 14:50:53 +0000
parents c4d85a8532d1
children 251e8904d6c2
line wrap: on
line source
2009-09-13	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	Refactored the persistent handling of the artifacts in the 
	artifact database. When an artifact is called (describe, feed,
	advance, out) a new CallContext is given instead of the
	old Object context. CallContext.globalContext() returns the
	old Object context now. CallContext.afterCall() may be
	called from inside describe, feed, advance and out to tell
	the artifact database what to do after the processing of
	the specific call.
	
	The accepted values are:
		- NOTHING    for doing nothing
		- TOUCH      for just update the last access time
		- STORE      for persisting the artifact.
		- BACKGROUND to signal that the artifact has started
		             a background operation.

	After the background operation the artifact has to call
	CallContext.fromBackground() to signal the artifact 
	database that the artifact needs to be touched or stored.

	!!! Without this call the artifact database assumes there
	!!! is still some background operation on this artifact
	!!! which prevents reclaiming resources

	* artifacts/src/main/java/de/intevation/artifacts/CallContext.java:
	New. New API to cope with background processing.

	* artifacts/src/main/java/de/intevation/artifacts/Artifact.java:
	Replaced old Object context with CallContext context.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/DefaultArtifact.java:
	Adjusted.

	* artifacts/src/main/java/de/intevation/artifacts/ArtifactDatabase.java:
	For a better encapsulation of the persistence operations of the
	artifacts the database does not return artifacts any more. It delegates
	the describe, feed, advance and out call to internal structures.

	* artifacts/src/main/java/de/intevation/artifacts/ArtifactDatabaseException.java:
	New. Simplifies handling of error conditions inside of artifact database.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/ArtifactDatabaseImpl.java:
	Adjusted to new interface. Internally it handles the background API.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/rest/ArtifactResource.java,
	artifact-database/src/main/java/de/intevation/artifactdatabase/rest/FactoriesResource.java,
	artifact-database/src/main/java/de/intevation/artifactdatabase/rest/OutRepresentation.java,
	artifact-database/src/main/java/de/intevation/artifactdatabase/rest/CreateResource.java,
	artifact-database/src/main/java/de/intevation/artifactdatabase/rest/ArtifactOutResource.java:
	Adjusted to new artifact database interface (which greatly simplifies things)

	* artifact-database/src/main/java/de/intevation/artifactdatabase/DatabaseCleaner.java:
	Artifacts in background are not removed from database any longer.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/App.java: Little
	re-wirering need for new cleanup.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/Backend.java: 
	Simplified to do the SQL stuff only. The most infrastructure stuff is now done
	in ArtifactDatabaseImpl.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/Id.java:
	New. Helper base class to enable filtering of background artifacts by there
	database id.

	* artifact-database/src/main/resources/sql/org-h2-driver.properties:
	Modified the INSERT statement to write the serialized artifact, too.

2009-09-12	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* artifacts/src/main/java/de/intevation/artifacts/Artifact.java(out):
	out() is now called with an java.io.Outputstream to place
	the output.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/DefaultArtifact.java,
	artifact-database/src/main/java/de/intevation/artifactdatabase/Backend.java,
	artifact-database/src/main/java/de/intevation/artifactdatabase/rest/OutRepresentation.java:
	Adjusted.

2009-09-11	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* artifact-database/doc/example-conf/conf.xml: Added
	sleep time for cleanup thread.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/App.java:
	Start cleanup thread at startup.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/Backend.java:
	Make artifact restoring bit more robust, added ref to
	cleanup thread to do an immediate cleanup when dead artifact
	is found by lookup.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/DatabaseCleaner.java:
	New. The cleanup thread. It cleans up the artifact database
	perodically and calls the endOfLife() methods of the dead
	artifacts. Default cleanup interval is 5 minutes.

	* artifact-database/src/main/resources/sql/org-h2-driver.properties:
	Tuned SQL a bit to fetch only a 50 artifacts in search for
	outdated artifacts to avoid too much memory consumption.

	* contrib/run.sh: Little tweak to work with more than one
	commons-pool version installed.

	* TODO: Removed remarks about the now implemented killer 
	thread and the moves XML stuff.

	* README: Changed text how to create a H2 database from
	command line only. XXX: This is still broken. The best
	way to set up the database by now is to start the Console 
	client, connect to the database, c&p the schema.sql 
	into the SQL text area and execute it.

2009-09-10	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* artifact-database/src/main/java/de/intevation/artifactdatabase/rest/ArtifactOutResource.java:
	New. Handler for 'out'. Reachable via HTTP GET '/artifact/{uuid}/{type}'.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/rest/OutRepresentation.java:
	New. Special representation for out. TODO: use streaming API.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/rest/ArtifactResource.java:
	Regrouped imports.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/rest/RestApp.java: Added
	handler for 'out' to routing table.

2009-09-10	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* artifact-database/src/main/java/de/intevation/artifactdatabase/rest/ArtifactResource.java:
	Added handler for 'advance' and 'feed' to be called by HTTP POST '/artifact/{uuid}'.

2009-09-10	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* artifact-database/src/main/java/de/intevation/artifactdatabase/rest/ArtifactResource.java:
	Added handler for 'describe' to be fetched by HTTP GET '/artifact/{uuid}'.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/rest/RestApp.java:
	Add route to new describe handler.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/Backend.java: Fixed
	problems with serialization of artifacts to backing database.

2009-09-10	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* artifact-database/src/main/java/de/intevation/artifactdatabase/XMLUtils.java:
	Added method to get strings from XML documents via XPath.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/DefaultArtifact.java:
	Used XMLUtils to create new XML documents.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/ArtifactDatabaseImpl.java:
	Forgot to store backend in instance variable.

	* artifacts/src/main/java/de/intevation/artifacts/ArtifactNamespaceContext.java:
	Added static instance of this class.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/rest/CreateResource.java:
	Forward call to ArtifactDatabase, do some error handling.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/Backend.java:
	Treat problems with database errors more gracefully.

	* contrib/run.sh: Added dependencies to common-pool and common-dbcp.

	* artifact-database/doc/data/create-artifact.xml: New. example data set to
	create an artifact. Usage

	$ curl --data-binary @doc/data/create-artifact.xml -D - http://localhost:8181/create

2009-09-10	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* artifact-database/src/main/java/de/intevation/artifactdatabase/rest/FactoriesResource.java:
	Removed needless import.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/Config.java,
	artifact-database/src/main/java/de/intevation/artifactdatabase/XMLUtils.java:
	Moved some XPath stuff from Config into XMLUtils.

2009-09-10	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* artifacts/src/main/java/de/intevation/artifacts/Artifact.java(setup): Added
	callback parameter in form of the creating factory.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/Backend.java,
	artifact-database/src/main/java/de/intevation/artifactdatabase/DefaultArtifact.java:
	Adjusted to match new Artifact interface.
	
	* artifact-database/src/main/java/de/intevation/artifactdatabase/DefaultArtifactFactory.java:
	Call artifact.setup() with this.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/DefaultArtifact.java:
	Removed namespace stuff.

	* artifacts/src/main/java/de/intevation/artifacts/ArtifactNamespaceContext.java:
	Created namespace stuff, need for XPath for e.g.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/rest/FactoriesResource.java:
	Use new namespace stuff.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/rest/Standalone.java: 
	Added author.

2009-09-09	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* artifact-database/src/main/java/de/intevation/artifactdatabase/XMLUtils.java:
	Added method newXPath() to create XPaths with namespace binding.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/Config.java:
	Uses new method from XMLUtils.

2009-09-09	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* trunk/artifact-database/src/main/java/de/intevation/artifactdatabase/rest/FactoriesResource.java,
	trunk/artifact-database/src/main/java/de/intevation/artifactdatabase/rest/CreateResource.java:
	Removed meta data from resources. They prevented serving the right results.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/Backend.java:
	Removed needless import.

2009-09-09	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* artifact-database/src/main/java/de/intevation/artifactdatabase/rest/CreateResource.java:
	New. Resource to create an artifact. Not working on artifact database yet.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/rest/RestApp.java:
	Attached create to web server.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/rest/FactoriesResource.java:
	Added meta data to Get annotation.

2009-09-09	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* artifact-database/pom.xml: Added dependency to restlet.ext.xml
	for XML handling in representations.

	* contrib/run.sh: Adjusted to fit new dependency.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/XMLUtils.java:
	New. Handle some boiler plate XML stuff. Move more stuff of this kind
	into this file.

	* TODO: Added remark to move more XML stuff into XMLUtils.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/rest/Standalone.java:
	Tell at startup on which port we are listing.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/rest/FactoriesResource.java:
	Representation to list the available factories.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/rest/RestApp.java:
	Attach new factory listing to web app.

2009-09-09	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* artifact-database/doc/artifactdb-example-conf.xml: removed
	* artifact-database/doc/example-conf/conf.xml: Re-added here.

	The configuration is now found in a subdirectory set by the
	system property 'artifact.database.dir'. If this property is
	not set '~/.artitactdb' is used. This directory should contain
	a file 'conf.xml' with the configuration details.
	  If the artifact database is started as a standalone the config
	directory is searched for 'log4j.properties' to configure the
	logging.
	  The H2 database is also search in this directory with the
	name 'artifacts.db' if the connection url in config.xml is
	not overwritten.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/Config.java:
	Refactored a bit to follow the new configuration policy.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/DBConnection.java:
	Build default connection url as described.

	* artifact-database/doc/example-conf/log4j.properties: New.
	example logging config

	* artifact-database/src/main/java/de/intevation/artifactdatabase/FactoryBootstrap.java:
	Added some more logging infos.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/App.java:
	Load the log4j configuration at startup and starts a REST standalone web server.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/rest: New.
	Package for REST binding.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/rest/Standalone.java:
	Convenience wrapper to start a standalone REST web server.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/rest/RestApp.java:
	Concrete REST application. TODO: attach restlets to interact with artifact database.

	* contrib/run.sh: Little start script to launch artifact database for test
	purpose from classes without the need to build packages. 
	Background: The maven exec:exec goal does not work consistently for me (slt). 
	At home it does, at work it doe not. Anyway. Usage:

	$ ./contrib/run.sh \
	      -Dartifact.database.dir=artifact-database/doc/example-conf/ \
	      de.intevation.artifactdatabase.App

2009-09-08	Tim Englich	<tim.englich@intevation.de>

	* src/main/java/de/intevation/artifactdatabase/DefaultArtifact.java : Edited
	Constants for Namespace-URL and Namespace-Prefix added 

2009-09-08	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* artifact-database/src/main/java/de/intevation/artifactdatabase/SQL.java,
	artifact-database/src/main/java/de/intevation/artifactdatabase/DefaultArtifact.java,
	artifact-database/src/main/java/de/intevation/artifactdatabase/Config.java,
	artifact-database/src/main/java/de/intevation/artifactdatabase/FactoryBootstrap.java,
	artifact-database/src/main/java/de/intevation/artifactdatabase/Backend.java,
	artifact-database/src/main/java/de/intevation/artifactdatabase/DefaultArtifactFactory.java:
	Integrated log4j

	* artifact-database/pom.xml: Added dependency to log4j

	* TODO: Removed remark to integrate logging.

2009-09-07  Tim Englich  <tim.englich@intevation.de>

	* src/main/java/de/intevation/artifactdatabase/FactoryBootstrap.java: edited
	Bug fixed in XPath Expression. Now the Nodes and not the Text of the Nodes will
	be used instantiating the ArtifactFactory. 
	
2009-09-07	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* artifact-database/src/main/java/de/intevation/artifactdatabase/Backend.java:
	Implemented loading of artifacts from database. If the last access of an 
	artifact is too long ago for its time to life, null is returned.

	* TODO: Added remark to implement a "killer" thread which periodically cleans the
	database from outdated artifacts and calls the endOfLife() method on those.

2009-09-07	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* artifact-database/src/main/java/de/intevation/artifactdatabase/SQL.java:
	Properties wrapper for SQL statements. The properties are looked up
	in resorces /sql/DRIVER.properties (DRIVER is the name of the db driver class,
	lowercased and '.' replaced by '-'. 'org.h2.Driver' turns to
	'org-h2-driver.properties', e.g.

	* artifact-database/src/main/resources/sql/org-h2-driver.properties:
	SQL statements for H2 database.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/Backend.java:
	Artifacts are proxies/wrapped by an special Artifact implementation
	to bound db activities transparentely to the Artifact. XXX: It has
	to be evaluated if a more direct concept like extending the Artifact API
	to store itself would be more robust for future implementations.
	TODO: Loading Artifact from db is not implemented yet.

2009-09-06	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* artifact-database/src/main/java/de/intevation/artifactdatabase/Backend.java:
	New. Used to wrap database presentations of artifacts into artifacts.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/ArtifactDatabaseImpl.java:
	New. Implements a ArtifactDatabase backed by Backend.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/Config.java:
	Removed empty last line.
	
	* artifact-database/src/main/java/de/intevation/artifactdatabase/App.java:
	Wired together Backend, ArtifactDatabaseImpl and FactoryBootstrap.

2009-09-06	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* artifact-database/src/main/java/de/intevation/artifactdatabase/DBConnection.java:
	Typo in XPath expression.

2009-09-06	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* artifacts/src/main/java/de/intevation/artifacts/Artifact.java(setup):
	Added the setup() method to have symmetric counter part to endOfLife().

	* artifacts/src/main/java/de/intevation/artifacts/ArtifactFactory.java(timeToLiveUntouched):
	Added this method to let the factory decide how long an artifact should live in ms.
	This is not a part of the Artifact itself because this is only evaluated once when
	the artifact is created.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/DefaultArtifact.java:
	New. Simple base class implementation of the Artifact interface.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/DefaultArtifactFactory.java:
	New. Simple base class implementation of the ArtifactFactory interface. When setup()
	on instances of this class is called, it pull ttl, name, description and
	the artifact class name from the node given. See artifactdb-example-conf.xml
	for examples.

	* artifact-database/pom.xml: Cleaned up XML. 
	Introduced dependency to apache commons dbcp, used for pooling of the 
	database connections to artifact db. 
	Added parameters for the exec:exec goal to make the project
	startable without building packages.

	* artifact-database/doc/schema.sql: Removed AUTO_INCREMENT from primary key
	to avoid compatibility issues with other non-H2 databases (PostgreSQL, Oracle, ...)
	which have no or limited support for generated keys in the JDBC driver. Now
	using an explicit sequence. TTL ist now big int to bring the resolution to ms.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/Config.java:
	Refactored a bit to make the XPath access function usable on arbitrary XML
	documents and parts of.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/DBConnection.java:
	New. Exposes DataSource from a apache dbcp connection pool configured by the
	global configuration file. See artifactdb-example-conf.xml for examples.
	TODO: Write some documentation about this.

	* artifact-database/doc/artifactdb-example-conf.xml: Adjusted to be a more realistic
	example config file. Added references to DefaultArtifact/DefaultArtifactFactory and
	demonstrate how to configure the connection pool.

	* TODO: Add remark to document the connection file.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/FactoryBootstrap.java:
	Do not crash when config does not contain any factories.

2009-09-04	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* artifact-database/src/main/java/de/intevation/artifactdatabase/Config.java (getNodeXPath):
	New method to get a single node from configuration.

	* artifacts/src/main/java/de/intevation/artifacts/Artifact.java (endOfLife):
	New method called when artifact is going to be removed from artifact data base.

2009-09-04	Tim Englich	<tim.englich@intevation.de>

	* src/main/java/de/intevation/artifactdatabase/FactoryBootstrap.java (loa,
	* src/main/java/de/intevation/artifacts/ArtifactFactory.java (setup):
	New Parameter for Setop to put the ArtifactFactoryNode into the setup routine.

2009-09-04	Tim Englich	<tim.englich@intevation.de>

	* pom.xml: Edited
	Buildplugins for Maven-Compiler and Maven-Eclipse Plugin integrated.

2009-09-04	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* artifact-database/doc/artifactdb-example-conf.xml: Example for artifact db
	configuration.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/DefaultArtifactContextFactory.java:
	New. Default implementation of ArtifactContextFactory. Creates a DefaultArtifactContext.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/DefaultArtifactContext.java:
	New. Default implementation of ArtifactContext.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/FactoryBootstrap.java:
	New. Class to bootstrap the context and the artifact factories.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/App.java: Bootstrap
	factories.

	* artifact-database/src/main/java/de/intevation/artifactdatabase/Config.java: New.
	Loads configuration as XML from a file. The file is specified by the system
	property "artifact.database.config" and defaults to "artifactdb-conf.xml".
	The configuration can be searched via XPath.

	* artifact-database/pom.xml: Made de.intevation.artifactdatabase.App the main
	class for start up.

	* TODO: Added remark to add logging to artifact database.

2009-09-04	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* artifacts/src/main/java/de/intevation/artifacts/ArtifactContextFactory.java:
	New. Factory for a global context in the artifact data base. Useful to
	create shared ressources for artifacts like caches et al.

	* artifacts/src/main/java/de/intevation/artifacts/ArtifactDatabase.java,
	artifacts/src/main/java/de/intevation/artifacts/ArtifactFactory.java,
	artifacts/src/main/java/de/intevation/artifacts/ArtifactContextFactory.java,
	artifacts/src/main/java/de/intevation/artifacts/Artifact.java: Uses the
	global context more consistent.

	* artifact-database/doc/schema.sql: Using BINARY instead of BLOB to
	avoid external files for each artifact blob. See H2 documentation
	for details.

2009-09-03	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* artifact-database/doc/schema.sql: Schema to store artifacts
	in H2 databases.

	* artifact-database/pom.xml: Added dependencies to H2 and
	restlet.

	* pom.xml: Added repository to fetch restlet.

	* README: Describe how to create a H2 database to store artifacts.
	TODO: Make it more general because description assumes usage of
	a unixoid system.

2009-09-03	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* artifact-database/**: created new sub project for the artefact 
	database server.

	* pom.xml: Registered new sub project and made artifacts a
	sub project, too.

	* artifacts/pom.xml: Adjusted to make a artifacts a sub project.

2009-09-02	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* artifacts/src/main/java/de/intevation/artifacts/Artifact.java:
	Interface of the central component of the system.

	* artifacts/src/main/java/de/intevation/artifacts/ArtifactDatabase.java:
	Central place to store artifacts in.

	* artifacts/src/main/java/de/intevation/artifacts/ArtifactFactory.java:
	Factory to build artifacts. Works together with ArtifactDatabase.

	* artifacts/pom.xml: Simple maven file to compile the project.

2009-09-01	Sascha L. Teichmann	<sascha.teichmann@intevation.de>

	* README, Changelog, Changes, NEWS, TODO: New. Initial setup

http://dive4elements.wald.intevation.org