Mercurial > dive4elements > framework
view Changelog @ 32:c2d53bd30ab8
Re-factored artifact API for better integration of background processing.
artifacts/trunk@78 c6561f87-3c4e-4783-a992-168aeb5c3f6f
author | Sascha L. Teichmann <sascha.teichmann@intevation.de> |
---|---|
date | Sun, 13 Sep 2009 14:50:53 +0000 |
parents | c4d85a8532d1 |
children | 251e8904d6c2 |
line wrap: on
line source
2009-09-13 Sascha L. Teichmann <sascha.teichmann@intevation.de> Refactored the persistent handling of the artifacts in the artifact database. When an artifact is called (describe, feed, advance, out) a new CallContext is given instead of the old Object context. CallContext.globalContext() returns the old Object context now. CallContext.afterCall() may be called from inside describe, feed, advance and out to tell the artifact database what to do after the processing of the specific call. The accepted values are: - NOTHING for doing nothing - TOUCH for just update the last access time - STORE for persisting the artifact. - BACKGROUND to signal that the artifact has started a background operation. After the background operation the artifact has to call CallContext.fromBackground() to signal the artifact database that the artifact needs to be touched or stored. !!! Without this call the artifact database assumes there !!! is still some background operation on this artifact !!! which prevents reclaiming resources * artifacts/src/main/java/de/intevation/artifacts/CallContext.java: New. New API to cope with background processing. * artifacts/src/main/java/de/intevation/artifacts/Artifact.java: Replaced old Object context with CallContext context. * artifact-database/src/main/java/de/intevation/artifactdatabase/DefaultArtifact.java: Adjusted. * artifacts/src/main/java/de/intevation/artifacts/ArtifactDatabase.java: For a better encapsulation of the persistence operations of the artifacts the database does not return artifacts any more. It delegates the describe, feed, advance and out call to internal structures. * artifacts/src/main/java/de/intevation/artifacts/ArtifactDatabaseException.java: New. Simplifies handling of error conditions inside of artifact database. * artifact-database/src/main/java/de/intevation/artifactdatabase/ArtifactDatabaseImpl.java: Adjusted to new interface. Internally it handles the background API. * artifact-database/src/main/java/de/intevation/artifactdatabase/rest/ArtifactResource.java, artifact-database/src/main/java/de/intevation/artifactdatabase/rest/FactoriesResource.java, artifact-database/src/main/java/de/intevation/artifactdatabase/rest/OutRepresentation.java, artifact-database/src/main/java/de/intevation/artifactdatabase/rest/CreateResource.java, artifact-database/src/main/java/de/intevation/artifactdatabase/rest/ArtifactOutResource.java: Adjusted to new artifact database interface (which greatly simplifies things) * artifact-database/src/main/java/de/intevation/artifactdatabase/DatabaseCleaner.java: Artifacts in background are not removed from database any longer. * artifact-database/src/main/java/de/intevation/artifactdatabase/App.java: Little re-wirering need for new cleanup. * artifact-database/src/main/java/de/intevation/artifactdatabase/Backend.java: Simplified to do the SQL stuff only. The most infrastructure stuff is now done in ArtifactDatabaseImpl. * artifact-database/src/main/java/de/intevation/artifactdatabase/Id.java: New. Helper base class to enable filtering of background artifacts by there database id. * artifact-database/src/main/resources/sql/org-h2-driver.properties: Modified the INSERT statement to write the serialized artifact, too. 2009-09-12 Sascha L. Teichmann <sascha.teichmann@intevation.de> * artifacts/src/main/java/de/intevation/artifacts/Artifact.java(out): out() is now called with an java.io.Outputstream to place the output. * artifact-database/src/main/java/de/intevation/artifactdatabase/DefaultArtifact.java, artifact-database/src/main/java/de/intevation/artifactdatabase/Backend.java, artifact-database/src/main/java/de/intevation/artifactdatabase/rest/OutRepresentation.java: Adjusted. 2009-09-11 Sascha L. Teichmann <sascha.teichmann@intevation.de> * artifact-database/doc/example-conf/conf.xml: Added sleep time for cleanup thread. * artifact-database/src/main/java/de/intevation/artifactdatabase/App.java: Start cleanup thread at startup. * artifact-database/src/main/java/de/intevation/artifactdatabase/Backend.java: Make artifact restoring bit more robust, added ref to cleanup thread to do an immediate cleanup when dead artifact is found by lookup. * artifact-database/src/main/java/de/intevation/artifactdatabase/DatabaseCleaner.java: New. The cleanup thread. It cleans up the artifact database perodically and calls the endOfLife() methods of the dead artifacts. Default cleanup interval is 5 minutes. * artifact-database/src/main/resources/sql/org-h2-driver.properties: Tuned SQL a bit to fetch only a 50 artifacts in search for outdated artifacts to avoid too much memory consumption. * contrib/run.sh: Little tweak to work with more than one commons-pool version installed. * TODO: Removed remarks about the now implemented killer thread and the moves XML stuff. * README: Changed text how to create a H2 database from command line only. XXX: This is still broken. The best way to set up the database by now is to start the Console client, connect to the database, c&p the schema.sql into the SQL text area and execute it. 2009-09-10 Sascha L. Teichmann <sascha.teichmann@intevation.de> * artifact-database/src/main/java/de/intevation/artifactdatabase/rest/ArtifactOutResource.java: New. Handler for 'out'. Reachable via HTTP GET '/artifact/{uuid}/{type}'. * artifact-database/src/main/java/de/intevation/artifactdatabase/rest/OutRepresentation.java: New. Special representation for out. TODO: use streaming API. * artifact-database/src/main/java/de/intevation/artifactdatabase/rest/ArtifactResource.java: Regrouped imports. * artifact-database/src/main/java/de/intevation/artifactdatabase/rest/RestApp.java: Added handler for 'out' to routing table. 2009-09-10 Sascha L. Teichmann <sascha.teichmann@intevation.de> * artifact-database/src/main/java/de/intevation/artifactdatabase/rest/ArtifactResource.java: Added handler for 'advance' and 'feed' to be called by HTTP POST '/artifact/{uuid}'. 2009-09-10 Sascha L. Teichmann <sascha.teichmann@intevation.de> * artifact-database/src/main/java/de/intevation/artifactdatabase/rest/ArtifactResource.java: Added handler for 'describe' to be fetched by HTTP GET '/artifact/{uuid}'. * artifact-database/src/main/java/de/intevation/artifactdatabase/rest/RestApp.java: Add route to new describe handler. * artifact-database/src/main/java/de/intevation/artifactdatabase/Backend.java: Fixed problems with serialization of artifacts to backing database. 2009-09-10 Sascha L. Teichmann <sascha.teichmann@intevation.de> * artifact-database/src/main/java/de/intevation/artifactdatabase/XMLUtils.java: Added method to get strings from XML documents via XPath. * artifact-database/src/main/java/de/intevation/artifactdatabase/DefaultArtifact.java: Used XMLUtils to create new XML documents. * artifact-database/src/main/java/de/intevation/artifactdatabase/ArtifactDatabaseImpl.java: Forgot to store backend in instance variable. * artifacts/src/main/java/de/intevation/artifacts/ArtifactNamespaceContext.java: Added static instance of this class. * artifact-database/src/main/java/de/intevation/artifactdatabase/rest/CreateResource.java: Forward call to ArtifactDatabase, do some error handling. * artifact-database/src/main/java/de/intevation/artifactdatabase/Backend.java: Treat problems with database errors more gracefully. * contrib/run.sh: Added dependencies to common-pool and common-dbcp. * artifact-database/doc/data/create-artifact.xml: New. example data set to create an artifact. Usage $ curl --data-binary @doc/data/create-artifact.xml -D - http://localhost:8181/create 2009-09-10 Sascha L. Teichmann <sascha.teichmann@intevation.de> * artifact-database/src/main/java/de/intevation/artifactdatabase/rest/FactoriesResource.java: Removed needless import. * artifact-database/src/main/java/de/intevation/artifactdatabase/Config.java, artifact-database/src/main/java/de/intevation/artifactdatabase/XMLUtils.java: Moved some XPath stuff from Config into XMLUtils. 2009-09-10 Sascha L. Teichmann <sascha.teichmann@intevation.de> * artifacts/src/main/java/de/intevation/artifacts/Artifact.java(setup): Added callback parameter in form of the creating factory. * artifact-database/src/main/java/de/intevation/artifactdatabase/Backend.java, artifact-database/src/main/java/de/intevation/artifactdatabase/DefaultArtifact.java: Adjusted to match new Artifact interface. * artifact-database/src/main/java/de/intevation/artifactdatabase/DefaultArtifactFactory.java: Call artifact.setup() with this. * artifact-database/src/main/java/de/intevation/artifactdatabase/DefaultArtifact.java: Removed namespace stuff. * artifacts/src/main/java/de/intevation/artifacts/ArtifactNamespaceContext.java: Created namespace stuff, need for XPath for e.g. * artifact-database/src/main/java/de/intevation/artifactdatabase/rest/FactoriesResource.java: Use new namespace stuff. * artifact-database/src/main/java/de/intevation/artifactdatabase/rest/Standalone.java: Added author. 2009-09-09 Sascha L. Teichmann <sascha.teichmann@intevation.de> * artifact-database/src/main/java/de/intevation/artifactdatabase/XMLUtils.java: Added method newXPath() to create XPaths with namespace binding. * artifact-database/src/main/java/de/intevation/artifactdatabase/Config.java: Uses new method from XMLUtils. 2009-09-09 Sascha L. Teichmann <sascha.teichmann@intevation.de> * trunk/artifact-database/src/main/java/de/intevation/artifactdatabase/rest/FactoriesResource.java, trunk/artifact-database/src/main/java/de/intevation/artifactdatabase/rest/CreateResource.java: Removed meta data from resources. They prevented serving the right results. * artifact-database/src/main/java/de/intevation/artifactdatabase/Backend.java: Removed needless import. 2009-09-09 Sascha L. Teichmann <sascha.teichmann@intevation.de> * artifact-database/src/main/java/de/intevation/artifactdatabase/rest/CreateResource.java: New. Resource to create an artifact. Not working on artifact database yet. * artifact-database/src/main/java/de/intevation/artifactdatabase/rest/RestApp.java: Attached create to web server. * artifact-database/src/main/java/de/intevation/artifactdatabase/rest/FactoriesResource.java: Added meta data to Get annotation. 2009-09-09 Sascha L. Teichmann <sascha.teichmann@intevation.de> * artifact-database/pom.xml: Added dependency to restlet.ext.xml for XML handling in representations. * contrib/run.sh: Adjusted to fit new dependency. * artifact-database/src/main/java/de/intevation/artifactdatabase/XMLUtils.java: New. Handle some boiler plate XML stuff. Move more stuff of this kind into this file. * TODO: Added remark to move more XML stuff into XMLUtils. * artifact-database/src/main/java/de/intevation/artifactdatabase/rest/Standalone.java: Tell at startup on which port we are listing. * artifact-database/src/main/java/de/intevation/artifactdatabase/rest/FactoriesResource.java: Representation to list the available factories. * artifact-database/src/main/java/de/intevation/artifactdatabase/rest/RestApp.java: Attach new factory listing to web app. 2009-09-09 Sascha L. Teichmann <sascha.teichmann@intevation.de> * artifact-database/doc/artifactdb-example-conf.xml: removed * artifact-database/doc/example-conf/conf.xml: Re-added here. The configuration is now found in a subdirectory set by the system property 'artifact.database.dir'. If this property is not set '~/.artitactdb' is used. This directory should contain a file 'conf.xml' with the configuration details. If the artifact database is started as a standalone the config directory is searched for 'log4j.properties' to configure the logging. The H2 database is also search in this directory with the name 'artifacts.db' if the connection url in config.xml is not overwritten. * artifact-database/src/main/java/de/intevation/artifactdatabase/Config.java: Refactored a bit to follow the new configuration policy. * artifact-database/src/main/java/de/intevation/artifactdatabase/DBConnection.java: Build default connection url as described. * artifact-database/doc/example-conf/log4j.properties: New. example logging config * artifact-database/src/main/java/de/intevation/artifactdatabase/FactoryBootstrap.java: Added some more logging infos. * artifact-database/src/main/java/de/intevation/artifactdatabase/App.java: Load the log4j configuration at startup and starts a REST standalone web server. * artifact-database/src/main/java/de/intevation/artifactdatabase/rest: New. Package for REST binding. * artifact-database/src/main/java/de/intevation/artifactdatabase/rest/Standalone.java: Convenience wrapper to start a standalone REST web server. * artifact-database/src/main/java/de/intevation/artifactdatabase/rest/RestApp.java: Concrete REST application. TODO: attach restlets to interact with artifact database. * contrib/run.sh: Little start script to launch artifact database for test purpose from classes without the need to build packages. Background: The maven exec:exec goal does not work consistently for me (slt). At home it does, at work it doe not. Anyway. Usage: $ ./contrib/run.sh \ -Dartifact.database.dir=artifact-database/doc/example-conf/ \ de.intevation.artifactdatabase.App 2009-09-08 Tim Englich <tim.englich@intevation.de> * src/main/java/de/intevation/artifactdatabase/DefaultArtifact.java : Edited Constants for Namespace-URL and Namespace-Prefix added 2009-09-08 Sascha L. Teichmann <sascha.teichmann@intevation.de> * artifact-database/src/main/java/de/intevation/artifactdatabase/SQL.java, artifact-database/src/main/java/de/intevation/artifactdatabase/DefaultArtifact.java, artifact-database/src/main/java/de/intevation/artifactdatabase/Config.java, artifact-database/src/main/java/de/intevation/artifactdatabase/FactoryBootstrap.java, artifact-database/src/main/java/de/intevation/artifactdatabase/Backend.java, artifact-database/src/main/java/de/intevation/artifactdatabase/DefaultArtifactFactory.java: Integrated log4j * artifact-database/pom.xml: Added dependency to log4j * TODO: Removed remark to integrate logging. 2009-09-07 Tim Englich <tim.englich@intevation.de> * src/main/java/de/intevation/artifactdatabase/FactoryBootstrap.java: edited Bug fixed in XPath Expression. Now the Nodes and not the Text of the Nodes will be used instantiating the ArtifactFactory. 2009-09-07 Sascha L. Teichmann <sascha.teichmann@intevation.de> * artifact-database/src/main/java/de/intevation/artifactdatabase/Backend.java: Implemented loading of artifacts from database. If the last access of an artifact is too long ago for its time to life, null is returned. * TODO: Added remark to implement a "killer" thread which periodically cleans the database from outdated artifacts and calls the endOfLife() method on those. 2009-09-07 Sascha L. Teichmann <sascha.teichmann@intevation.de> * artifact-database/src/main/java/de/intevation/artifactdatabase/SQL.java: Properties wrapper for SQL statements. The properties are looked up in resorces /sql/DRIVER.properties (DRIVER is the name of the db driver class, lowercased and '.' replaced by '-'. 'org.h2.Driver' turns to 'org-h2-driver.properties', e.g. * artifact-database/src/main/resources/sql/org-h2-driver.properties: SQL statements for H2 database. * artifact-database/src/main/java/de/intevation/artifactdatabase/Backend.java: Artifacts are proxies/wrapped by an special Artifact implementation to bound db activities transparentely to the Artifact. XXX: It has to be evaluated if a more direct concept like extending the Artifact API to store itself would be more robust for future implementations. TODO: Loading Artifact from db is not implemented yet. 2009-09-06 Sascha L. Teichmann <sascha.teichmann@intevation.de> * artifact-database/src/main/java/de/intevation/artifactdatabase/Backend.java: New. Used to wrap database presentations of artifacts into artifacts. * artifact-database/src/main/java/de/intevation/artifactdatabase/ArtifactDatabaseImpl.java: New. Implements a ArtifactDatabase backed by Backend. * artifact-database/src/main/java/de/intevation/artifactdatabase/Config.java: Removed empty last line. * artifact-database/src/main/java/de/intevation/artifactdatabase/App.java: Wired together Backend, ArtifactDatabaseImpl and FactoryBootstrap. 2009-09-06 Sascha L. Teichmann <sascha.teichmann@intevation.de> * artifact-database/src/main/java/de/intevation/artifactdatabase/DBConnection.java: Typo in XPath expression. 2009-09-06 Sascha L. Teichmann <sascha.teichmann@intevation.de> * artifacts/src/main/java/de/intevation/artifacts/Artifact.java(setup): Added the setup() method to have symmetric counter part to endOfLife(). * artifacts/src/main/java/de/intevation/artifacts/ArtifactFactory.java(timeToLiveUntouched): Added this method to let the factory decide how long an artifact should live in ms. This is not a part of the Artifact itself because this is only evaluated once when the artifact is created. * artifact-database/src/main/java/de/intevation/artifactdatabase/DefaultArtifact.java: New. Simple base class implementation of the Artifact interface. * artifact-database/src/main/java/de/intevation/artifactdatabase/DefaultArtifactFactory.java: New. Simple base class implementation of the ArtifactFactory interface. When setup() on instances of this class is called, it pull ttl, name, description and the artifact class name from the node given. See artifactdb-example-conf.xml for examples. * artifact-database/pom.xml: Cleaned up XML. Introduced dependency to apache commons dbcp, used for pooling of the database connections to artifact db. Added parameters for the exec:exec goal to make the project startable without building packages. * artifact-database/doc/schema.sql: Removed AUTO_INCREMENT from primary key to avoid compatibility issues with other non-H2 databases (PostgreSQL, Oracle, ...) which have no or limited support for generated keys in the JDBC driver. Now using an explicit sequence. TTL ist now big int to bring the resolution to ms. * artifact-database/src/main/java/de/intevation/artifactdatabase/Config.java: Refactored a bit to make the XPath access function usable on arbitrary XML documents and parts of. * artifact-database/src/main/java/de/intevation/artifactdatabase/DBConnection.java: New. Exposes DataSource from a apache dbcp connection pool configured by the global configuration file. See artifactdb-example-conf.xml for examples. TODO: Write some documentation about this. * artifact-database/doc/artifactdb-example-conf.xml: Adjusted to be a more realistic example config file. Added references to DefaultArtifact/DefaultArtifactFactory and demonstrate how to configure the connection pool. * TODO: Add remark to document the connection file. * artifact-database/src/main/java/de/intevation/artifactdatabase/FactoryBootstrap.java: Do not crash when config does not contain any factories. 2009-09-04 Sascha L. Teichmann <sascha.teichmann@intevation.de> * artifact-database/src/main/java/de/intevation/artifactdatabase/Config.java (getNodeXPath): New method to get a single node from configuration. * artifacts/src/main/java/de/intevation/artifacts/Artifact.java (endOfLife): New method called when artifact is going to be removed from artifact data base. 2009-09-04 Tim Englich <tim.englich@intevation.de> * src/main/java/de/intevation/artifactdatabase/FactoryBootstrap.java (loa, * src/main/java/de/intevation/artifacts/ArtifactFactory.java (setup): New Parameter for Setop to put the ArtifactFactoryNode into the setup routine. 2009-09-04 Tim Englich <tim.englich@intevation.de> * pom.xml: Edited Buildplugins for Maven-Compiler and Maven-Eclipse Plugin integrated. 2009-09-04 Sascha L. Teichmann <sascha.teichmann@intevation.de> * artifact-database/doc/artifactdb-example-conf.xml: Example for artifact db configuration. * artifact-database/src/main/java/de/intevation/artifactdatabase/DefaultArtifactContextFactory.java: New. Default implementation of ArtifactContextFactory. Creates a DefaultArtifactContext. * artifact-database/src/main/java/de/intevation/artifactdatabase/DefaultArtifactContext.java: New. Default implementation of ArtifactContext. * artifact-database/src/main/java/de/intevation/artifactdatabase/FactoryBootstrap.java: New. Class to bootstrap the context and the artifact factories. * artifact-database/src/main/java/de/intevation/artifactdatabase/App.java: Bootstrap factories. * artifact-database/src/main/java/de/intevation/artifactdatabase/Config.java: New. Loads configuration as XML from a file. The file is specified by the system property "artifact.database.config" and defaults to "artifactdb-conf.xml". The configuration can be searched via XPath. * artifact-database/pom.xml: Made de.intevation.artifactdatabase.App the main class for start up. * TODO: Added remark to add logging to artifact database. 2009-09-04 Sascha L. Teichmann <sascha.teichmann@intevation.de> * artifacts/src/main/java/de/intevation/artifacts/ArtifactContextFactory.java: New. Factory for a global context in the artifact data base. Useful to create shared ressources for artifacts like caches et al. * artifacts/src/main/java/de/intevation/artifacts/ArtifactDatabase.java, artifacts/src/main/java/de/intevation/artifacts/ArtifactFactory.java, artifacts/src/main/java/de/intevation/artifacts/ArtifactContextFactory.java, artifacts/src/main/java/de/intevation/artifacts/Artifact.java: Uses the global context more consistent. * artifact-database/doc/schema.sql: Using BINARY instead of BLOB to avoid external files for each artifact blob. See H2 documentation for details. 2009-09-03 Sascha L. Teichmann <sascha.teichmann@intevation.de> * artifact-database/doc/schema.sql: Schema to store artifacts in H2 databases. * artifact-database/pom.xml: Added dependencies to H2 and restlet. * pom.xml: Added repository to fetch restlet. * README: Describe how to create a H2 database to store artifacts. TODO: Make it more general because description assumes usage of a unixoid system. 2009-09-03 Sascha L. Teichmann <sascha.teichmann@intevation.de> * artifact-database/**: created new sub project for the artefact database server. * pom.xml: Registered new sub project and made artifacts a sub project, too. * artifacts/pom.xml: Adjusted to make a artifacts a sub project. 2009-09-02 Sascha L. Teichmann <sascha.teichmann@intevation.de> * artifacts/src/main/java/de/intevation/artifacts/Artifact.java: Interface of the central component of the system. * artifacts/src/main/java/de/intevation/artifacts/ArtifactDatabase.java: Central place to store artifacts in. * artifacts/src/main/java/de/intevation/artifacts/ArtifactFactory.java: Factory to build artifacts. Works together with ArtifactDatabase. * artifacts/pom.xml: Simple maven file to compile the project. 2009-09-01 Sascha L. Teichmann <sascha.teichmann@intevation.de> * README, Changelog, Changes, NEWS, TODO: New. Initial setup