Mercurial > dive4elements > river
annotate flys-artifacts/src/main/java/de/intevation/flys/artifacts/datacage/templating/BuilderPool.java @ 5779:ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
XML DOM is not thread safe. Therefore the old implementation only allowed one thread
to use the builder at a time. As the complexity of the configuration
has increased over time this has become a bottleneck of the whole application
because it took quiet some time to build a result. Furthermore the builder code path
is visited very frequent. So many concurrent requests were piled up
resulting in long waits for the users.
To mitigate this problem a round robin pool of builders is used now.
Each of the pooled builders has an independent copy of the XML template
and can be run in parallel.
The number of builders is determined by the system property
'flys.datacage.pool.size'. It defaults to 4.
author | Sascha L. Teichmann <teichmann@intevation.de> |
---|---|
date | Sun, 21 Apr 2013 12:48:09 +0200 |
parents | |
children | d38004f0c52f |
rev | line source |
---|---|
5779
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
1 package de.intevation.flys.artifacts.datacage.templating; |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
2 |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
3 import java.util.ArrayDeque; |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
4 import java.util.Deque; |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
5 import java.util.List; |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
6 import java.util.Map; |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
7 |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
8 import java.sql.SQLException; |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
9 |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
10 import javax.xml.transform.Transformer; |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
11 import javax.xml.transform.TransformerConfigurationException; |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
12 import javax.xml.transform.TransformerException; |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
13 import javax.xml.transform.TransformerFactory; |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
14 |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
15 import javax.xml.transform.dom.DOMResult; |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
16 import javax.xml.transform.dom.DOMSource; |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
17 |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
18 import org.apache.log4j.Logger; |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
19 |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
20 import org.w3c.dom.Document; |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
21 import org.w3c.dom.Node; |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
22 |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
23 /** A little round robin pool of builders to mitigate |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
24 * the fact the XML DOM documents are not thread safe. |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
25 */ |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
26 public class BuilderPool |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
27 { |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
28 private static Logger log = Logger.getLogger(BuilderPool.class); |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
29 |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
30 private static final int DEFAULT_POOL_SIZE = 4; |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
31 |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
32 private static final int POOL_SIZE = Math.max( |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
33 Integer.getInteger("flys.datacage.pool.size", DEFAULT_POOL_SIZE), 1); |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
34 |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
35 private Deque<Builder> pool; |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
36 |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
37 public BuilderPool(Document document) { |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
38 this(document, POOL_SIZE); |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
39 } |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
40 |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
41 public BuilderPool(Document document, int poolSize) { |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
42 |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
43 if (log.isDebugEnabled()) { |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
44 log.debug("Create build pool with " + poolSize + " elements."); |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
45 } |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
46 |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
47 pool = new ArrayDeque<Builder>(poolSize); |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
48 for (int i = 0; i < poolSize; ++i) { |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
49 Document doc = i > 0 // Clone all but the first. |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
50 ? cloneDocument(document) |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
51 : document; |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
52 pool.add(new Builder(doc)); |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
53 } |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
54 } |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
55 |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
56 private final static Document cloneDocument(Document document) { |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
57 try { |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
58 TransformerFactory tfactory = TransformerFactory.newInstance(); |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
59 Transformer xformer = tfactory.newTransformer(); |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
60 DOMSource src = new DOMSource(document); |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
61 DOMResult dst = new DOMResult(); |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
62 xformer.transform(src, dst); |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
63 return (Document)dst.getNode(); |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
64 } |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
65 catch (TransformerConfigurationException tce) { |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
66 log.error(tce); |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
67 } |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
68 catch (TransformerException te) { |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
69 log.error(te); |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
70 } |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
71 |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
72 log.error( |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
73 "Returning original DOM document. " + |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
74 "This will result in threading errors!"); |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
75 |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
76 return document; |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
77 } |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
78 |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
79 public void build( |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
80 List<Builder.NamedConnection> connections, |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
81 Node output, |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
82 Map<String, Object> parameters |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
83 ) |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
84 throws SQLException |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
85 { |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
86 Builder builder; |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
87 synchronized (pool) { |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
88 try { |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
89 while ((builder = pool.poll()) == null) { |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
90 pool.wait(); |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
91 } |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
92 } |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
93 catch (InterruptedException ie) { |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
94 log.debug("Waiting for builder interrupted. Build canceled."); |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
95 return; |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
96 } |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
97 } |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
98 try { |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
99 builder.build(connections, output, parameters); |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
100 } |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
101 finally { |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
102 synchronized (pool) { |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
103 pool.add(builder); |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
104 pool.notify(); |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
105 } |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
106 } |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
107 } |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
108 } |
ebec12def170
Datacage: Add a pool of builders to make it multi threadable.
Sascha L. Teichmann <teichmann@intevation.de>
parents:
diff
changeset
|
109 // vim:set ts=4 sw=4 si et sta sts=4 fenc=utf-8 : |