Mercurial > dive4elements > river

annotate flys-artifacts/src/main/java/de/intevation/flys/artifacts/model/sq/Outlier.java @ 5396:aaf810d4ec82

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Use new state 'outlier mehtod selection' in SQ-relation. The outlier method is no more configurable in conf.xml. The user can now select the method in SQ-relation parameters.
author Raimund Renkert <rrenkert@intevation.de>
date Mon, 25 Mar 2013 12:40:26 +0100
parents 846b0441f905
children 2fe120e1e4df
rev   line source
3187
1e2733f749b5 SQ: Added specialized outlier test for the S/Q relation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
1 package de.intevation.flys.artifacts.model.sq;
1e2733f749b5 SQ: Added specialized outlier test for the S/Q relation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
2
4795
8ee270a3ef25 Small code cleanups in S/Q outlier tests.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
3 import de.intevation.artifacts.common.utils.Config;
8ee270a3ef25 Small code cleanups in S/Q outlier tests.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
4
8ee270a3ef25 Small code cleanups in S/Q outlier tests.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
5 import de.intevation.flys.artifacts.math.GrubbsOutlier;
8ee270a3ef25 Small code cleanups in S/Q outlier tests.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
6 import de.intevation.flys.artifacts.math.StdDevOutlier;
8ee270a3ef25 Small code cleanups in S/Q outlier tests.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
7
3187
1e2733f749b5 SQ: Added specialized outlier test for the S/Q relation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
8 import java.util.ArrayList;
1e2733f749b5 SQ: Added specialized outlier test for the S/Q relation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
9 import java.util.List;
1e2733f749b5 SQ: Added specialized outlier test for the S/Q relation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
10
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3187
diff changeset
11 import org.apache.commons.math.MathException;
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3187
diff changeset
12
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
13 import org.apache.log4j.Logger;
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
14
3187
1e2733f749b5 SQ: Added specialized outlier test for the S/Q relation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
15 public class Outlier
1e2733f749b5 SQ: Added specialized outlier test for the S/Q relation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
16 {
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
17 private static Logger log = Logger.getLogger(Outlier.class);
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
18
4794
a7d080347ac3 MINFO: Allow two methods for outlier test in SQ relation.
Raimund Renkert <rrenkert@intevation.de>
parents: 3572
diff changeset
19 private static final String OUTLIER_METHOD =
a7d080347ac3 MINFO: Allow two methods for outlier test in SQ relation.
Raimund Renkert <rrenkert@intevation.de>
parents: 3572
diff changeset
20 "/artifact-database/options/minfo-sq/outlier-method/@name";
a7d080347ac3 MINFO: Allow two methods for outlier test in SQ relation.
Raimund Renkert <rrenkert@intevation.de>
parents: 3572
diff changeset
21
5396
aaf810d4ec82 Use new state 'outlier mehtod selection' in SQ-relation.
Raimund Renkert <rrenkert@intevation.de>
parents: 4816
diff changeset
22 private static final String GRUBBS = "outlier.method.grubbs";
4794
a7d080347ac3 MINFO: Allow two methods for outlier test in SQ relation.
Raimund Renkert <rrenkert@intevation.de>
parents: 3572
diff changeset
23
4795
8ee270a3ef25 Small code cleanups in S/Q outlier tests.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
24 //private static final String STD_DEV = "std-dev";
4794
a7d080347ac3 MINFO: Allow two methods for outlier test in SQ relation.
Raimund Renkert <rrenkert@intevation.de>
parents: 3572
diff changeset
25
3187
1e2733f749b5 SQ: Added specialized outlier test for the S/Q relation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
26 public interface Callback {
1e2733f749b5 SQ: Added specialized outlier test for the S/Q relation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
27
3566
8d0f06b76e09 S/Q relation: Fixed flys/issue748
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3552
diff changeset
28 void initialize(List<SQ> sqs) throws MathException;
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
29
3187
1e2733f749b5 SQ: Added specialized outlier test for the S/Q relation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
30 double eval(SQ sq);
1e2733f749b5 SQ: Added specialized outlier test for the S/Q relation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
31
3566
8d0f06b76e09 S/Q relation: Fixed flys/issue748
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3552
diff changeset
32 void iterationFinished(
8d0f06b76e09 S/Q relation: Fixed flys/issue748
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3552
diff changeset
33 double stdDev,
8d0f06b76e09 S/Q relation: Fixed flys/issue748
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3552
diff changeset
34 SQ outlier,
8d0f06b76e09 S/Q relation: Fixed flys/issue748
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3552
diff changeset
35 List<SQ> remaining);
3187
1e2733f749b5 SQ: Added specialized outlier test for the S/Q relation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
36
1e2733f749b5 SQ: Added specialized outlier test for the S/Q relation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
37 } // interface Callback
1e2733f749b5 SQ: Added specialized outlier test for the S/Q relation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
38
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
39 public static void detectOutliers(
3187
1e2733f749b5 SQ: Added specialized outlier test for the S/Q relation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
40 Callback callback,
1e2733f749b5 SQ: Added specialized outlier test for the S/Q relation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
41 List<SQ> sqs,
5396
aaf810d4ec82 Use new state 'outlier mehtod selection' in SQ-relation.
Raimund Renkert <rrenkert@intevation.de>
parents: 4816
diff changeset
42 double stdDevFactor,
aaf810d4ec82 Use new state 'outlier mehtod selection' in SQ-relation.
Raimund Renkert <rrenkert@intevation.de>
parents: 4816
diff changeset
43 String method
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3187
diff changeset
44 )
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3187
diff changeset
45 throws MathException
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3187
diff changeset
46 {
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
47 boolean debug = log.isDebugEnabled();
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
48
4794
a7d080347ac3 MINFO: Allow two methods for outlier test in SQ relation.
Raimund Renkert <rrenkert@intevation.de>
parents: 3572
diff changeset
49 if (method == null) {
a7d080347ac3 MINFO: Allow two methods for outlier test in SQ relation.
Raimund Renkert <rrenkert@intevation.de>
parents: 3572
diff changeset
50 method = "std-dev";
a7d080347ac3 MINFO: Allow two methods for outlier test in SQ relation.
Raimund Renkert <rrenkert@intevation.de>
parents: 3572
diff changeset
51 }
4795
8ee270a3ef25 Small code cleanups in S/Q outlier tests.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
52
8ee270a3ef25 Small code cleanups in S/Q outlier tests.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
53 if (debug) {
8ee270a3ef25 Small code cleanups in S/Q outlier tests.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
54 log.debug("stdDevFactor: " + stdDevFactor);
8ee270a3ef25 Small code cleanups in S/Q outlier tests.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
55 log.debug("method: " + method);
8ee270a3ef25 Small code cleanups in S/Q outlier tests.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
56 }
8ee270a3ef25 Small code cleanups in S/Q outlier tests.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
57
3566
8d0f06b76e09 S/Q relation: Fixed flys/issue748
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3552
diff changeset
58 List<SQ> data = new ArrayList<SQ>(sqs);
3187
1e2733f749b5 SQ: Added specialized outlier test for the S/Q relation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
59
4795
8ee270a3ef25 Small code cleanups in S/Q outlier tests.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
60 double [] stdDev = new double[1];
8ee270a3ef25 Small code cleanups in S/Q outlier tests.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
61
8ee270a3ef25 Small code cleanups in S/Q outlier tests.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
62 boolean useGrubbs = method.equals(GRUBBS);
8ee270a3ef25 Small code cleanups in S/Q outlier tests.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
63
8ee270a3ef25 Small code cleanups in S/Q outlier tests.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
64 if (useGrubbs) {
8ee270a3ef25 Small code cleanups in S/Q outlier tests.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
65 stdDevFactor = Math.max(0d, Math.min(stdDevFactor/100d, 1d));
8ee270a3ef25 Small code cleanups in S/Q outlier tests.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
66 }
8ee270a3ef25 Small code cleanups in S/Q outlier tests.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
67
8ee270a3ef25 Small code cleanups in S/Q outlier tests.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
68 List<Double> values = new ArrayList<Double>(data.size());
8ee270a3ef25 Small code cleanups in S/Q outlier tests.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
69
3566
8d0f06b76e09 S/Q relation: Fixed flys/issue748
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3552
diff changeset
70 while (data.size() > 2) {
3187
1e2733f749b5 SQ: Added specialized outlier test for the S/Q relation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
71
3566
8d0f06b76e09 S/Q relation: Fixed flys/issue748
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3552
diff changeset
72 callback.initialize(data);
3187
1e2733f749b5 SQ: Added specialized outlier test for the S/Q relation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
73
4794
a7d080347ac3 MINFO: Allow two methods for outlier test in SQ relation.
Raimund Renkert <rrenkert@intevation.de>
parents: 3572
diff changeset
74 for (SQ sq: data) {
a7d080347ac3 MINFO: Allow two methods for outlier test in SQ relation.
Raimund Renkert <rrenkert@intevation.de>
parents: 3572
diff changeset
75 values.add(callback.eval(sq));
3187
1e2733f749b5 SQ: Added specialized outlier test for the S/Q relation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
76 }
1e2733f749b5 SQ: Added specialized outlier test for the S/Q relation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
77
4795
8ee270a3ef25 Small code cleanups in S/Q outlier tests.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
78 Integer ndx = useGrubbs
8ee270a3ef25 Small code cleanups in S/Q outlier tests.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
79 ? GrubbsOutlier.findOutlier(values, stdDevFactor, stdDev)
8ee270a3ef25 Small code cleanups in S/Q outlier tests.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
80 : StdDevOutlier.findOutlier(values, stdDevFactor, stdDev);
8ee270a3ef25 Small code cleanups in S/Q outlier tests.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
81
4794
a7d080347ac3 MINFO: Allow two methods for outlier test in SQ relation.
Raimund Renkert <rrenkert@intevation.de>
parents: 3572
diff changeset
82 if (ndx == null) {
a7d080347ac3 MINFO: Allow two methods for outlier test in SQ relation.
Raimund Renkert <rrenkert@intevation.de>
parents: 3572
diff changeset
83 callback.iterationFinished(stdDev[0], null, data);
a7d080347ac3 MINFO: Allow two methods for outlier test in SQ relation.
Raimund Renkert <rrenkert@intevation.de>
parents: 3572
diff changeset
84 break;
3187
1e2733f749b5 SQ: Added specialized outlier test for the S/Q relation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
85 }
1e2733f749b5 SQ: Added specialized outlier test for the S/Q relation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
86
4795
8ee270a3ef25 Small code cleanups in S/Q outlier tests.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
87 SQ outlier = data.remove(ndx.intValue());
8ee270a3ef25 Small code cleanups in S/Q outlier tests.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
88 if (debug) {
8ee270a3ef25 Small code cleanups in S/Q outlier tests.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
89 log.debug("stdDev: " + stdDev[0]);
4816
846b0441f905 Removed trailing whitespace.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4795
diff changeset
90 log.debug("removed " + ndx +
4795
8ee270a3ef25 Small code cleanups in S/Q outlier tests.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
91 "; S: " + outlier.getS() + " Q: " + outlier.getQ());
8ee270a3ef25 Small code cleanups in S/Q outlier tests.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
92 }
4794
a7d080347ac3 MINFO: Allow two methods for outlier test in SQ relation.
Raimund Renkert <rrenkert@intevation.de>
parents: 3572
diff changeset
93 callback.iterationFinished(stdDev[0], outlier, data);
4795
8ee270a3ef25 Small code cleanups in S/Q outlier tests.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
94 values.clear();
3566
8d0f06b76e09 S/Q relation: Fixed flys/issue748
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3552
diff changeset
95 }
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3187
diff changeset
96 }
3187
1e2733f749b5 SQ: Added specialized outlier test for the S/Q relation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
97 }
1e2733f749b5 SQ: Added specialized outlier test for the S/Q relation.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
98 // vim:set ts=4 sw=4 si et sta sts=4 fenc=utf8 :
http://dive4elements.wald.intevation.org