Mercurial > dive4elements > river

annotate artifacts/src/main/java/org/dive4elements/river/artifacts/model/fixings/Fitting.java @ 9348:a3f318347707

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Show wq outliers within same thems with different symbol: not ready yet
author gernotbelger
date Tue, 31 Jul 2018 11:25:38 +0200
parents 83bb52fa0c32
children ddcd52d239cd
rev   line source
5863
4897a58c8746 River artifacts: Added new copyright headers.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
1 /* Copyright (C) 2011, 2012, 2013 by Bundesanstalt für Gewässerkunde
4897a58c8746 River artifacts: Added new copyright headers.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
2 * Software engineering by Intevation GmbH
4897a58c8746 River artifacts: Added new copyright headers.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
3 *
5994
af13ceeba52a Removed trailing whitespace.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5863
diff changeset
4 * This file is Free Software under the GNU AGPL (>=v3)
5863
4897a58c8746 River artifacts: Added new copyright headers.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
5 * and comes with ABSOLUTELY NO WARRANTY! Check out the
5994
af13ceeba52a Removed trailing whitespace.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5863
diff changeset
6 * documentation coming with Dive4Elements River for details.
5863
4897a58c8746 River artifacts: Added new copyright headers.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
7 */
4897a58c8746 River artifacts: Added new copyright headers.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
8
5831
bd047b71ab37 Repaired internal references
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
9 package org.dive4elements.river.artifacts.model.fixings;
3011
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
10
3107
cbf308f5c41b FixA: Store the standard deviation of delta Ws of reference points into parameters, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3096
diff changeset
11 import gnu.trove.TDoubleArrayList;
3011
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
12
3107
cbf308f5c41b FixA: Store the standard deviation of delta Ws of reference points into parameters, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3096
diff changeset
13 import java.util.ArrayList;
cbf308f5c41b FixA: Store the standard deviation of delta Ws of reference points into parameters, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3096
diff changeset
14 import java.util.List;
3011
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
15
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
16 import org.apache.commons.math.MathException;
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
17 import org.apache.commons.math.optimization.fitting.CurveFitter;
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
18 import org.apache.commons.math.optimization.general.LevenbergMarquardtOptimizer;
3107
cbf308f5c41b FixA: Store the standard deviation of delta Ws of reference points into parameters, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3096
diff changeset
19 import org.apache.commons.math.stat.descriptive.moment.StandardDeviation;
4794
a7d080347ac3 MINFO: Allow two methods for outlier test in SQ relation.
Raimund Renkert <rrenkert@intevation.de>
parents: 3729
diff changeset
20 import org.apache.log4j.Logger;
3011
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
21
5831
bd047b71ab37 Repaired internal references
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
22 import org.dive4elements.river.artifacts.math.GrubbsOutlier;
bd047b71ab37 Repaired internal references
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
23 import org.dive4elements.river.artifacts.math.fitting.Function;
3011
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
24
9348
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
25 public class Fitting {
3011
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
26 private static Logger log = Logger.getLogger(Fitting.class);
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
27
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
28 /** Use instance of this factory to find meta infos for outliers. */
3096
d7b0f52d6d04 FixA: Calculate Delta W(t) for reference points, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3073
diff changeset
29 public interface QWDFactory {
9348
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
30 QWD create(double q, double w, boolean isOutlier);
3011
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
31 }
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
32
9348
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
33 private final boolean checkOutliers;
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
34
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
35 private final Function function;
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
36
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
37 private final QWDFactory qwdFactory;
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
38
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
39 private double chiSqr;
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
40
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
41 private double[] parameters;
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
42
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
43 private final List<QWD> removed = new ArrayList<>();
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
44
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
45 private QWD[] referenced;
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
46
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
47 private double standardDeviation;
3022
705d2058b682 FixA: Store the referenced QW for each km, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3011
diff changeset
48
9348
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
49 public Fitting(Function function, QWDFactory qwdFactory, boolean checkOutliers) {
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
50 this.function = function;
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
51 this.qwdFactory = qwdFactory;
3022
705d2058b682 FixA: Store the referenced QW for each km, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3011
diff changeset
52 this.checkOutliers = checkOutliers;
3011
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
53 }
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
54
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
55 public Function getFunction() {
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
56 return function;
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
57 }
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
58
3022
705d2058b682 FixA: Store the referenced QW for each km, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3011
diff changeset
59 public boolean getCheckOutliers() {
705d2058b682 FixA: Store the referenced QW for each km, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3011
diff changeset
60 return checkOutliers;
705d2058b682 FixA: Store the referenced QW for each km, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3011
diff changeset
61 }
705d2058b682 FixA: Store the referenced QW for each km, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3011
diff changeset
62
3011
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
63 public double getChiSquare() {
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
64 return chiSqr;
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
65 }
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
66
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
67 public void reset() {
9348
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
68 chiSqr = 0.0;
3011
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
69 parameters = null;
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
70 removed.clear();
3022
705d2058b682 FixA: Store the referenced QW for each km, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3011
diff changeset
71 referenced = null;
3107
cbf308f5c41b FixA: Store the standard deviation of delta Ws of reference points into parameters, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3096
diff changeset
72 standardDeviation = 0.0;
3011
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
73 }
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
74
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
75 public boolean hasOutliers() {
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
76 return !removed.isEmpty();
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
77 }
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
78
9348
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
79 public QWD[] outliersToArray() {
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
80 return removed.toArray(new QWD[removed.size()]);
3011
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
81 }
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
82
9348
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
83 public QWD[] referencedToArray() {
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
84 return referenced != null ? (QWD[]) referenced.clone() : null;
3022
705d2058b682 FixA: Store the referenced QW for each km, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3011
diff changeset
85 }
705d2058b682 FixA: Store the referenced QW for each km, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3011
diff changeset
86
3065
ee653971fa84 FixA: Figure out max Q during fitting.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3030
diff changeset
87 public double getMaxQ() {
ee653971fa84 FixA: Figure out max Q during fitting.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3030
diff changeset
88 double maxQ = -Double.MAX_VALUE;
ee653971fa84 FixA: Figure out max Q during fitting.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3030
diff changeset
89 if (referenced != null) {
9348
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
90 for (QWI qw : referenced) {
6868
08e3c22500f3 Fix Analysis: Code simplification in facets.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5994
diff changeset
91 double q = qw.getQ();
08e3c22500f3 Fix Analysis: Code simplification in facets.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5994
diff changeset
92 if (q > maxQ) {
08e3c22500f3 Fix Analysis: Code simplification in facets.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5994
diff changeset
93 maxQ = q;
3065
ee653971fa84 FixA: Figure out max Q during fitting.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3030
diff changeset
94 }
ee653971fa84 FixA: Figure out max Q during fitting.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3030
diff changeset
95 }
ee653971fa84 FixA: Figure out max Q during fitting.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3030
diff changeset
96 }
ee653971fa84 FixA: Figure out max Q during fitting.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3030
diff changeset
97 return maxQ;
ee653971fa84 FixA: Figure out max Q during fitting.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3030
diff changeset
98 }
ee653971fa84 FixA: Figure out max Q during fitting.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3030
diff changeset
99
9348
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
100 public double[] getParameters() {
3011
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
101 return parameters;
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
102 }
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
103
3107
cbf308f5c41b FixA: Store the standard deviation of delta Ws of reference points into parameters, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3096
diff changeset
104 public double getStandardDeviation() {
cbf308f5c41b FixA: Store the standard deviation of delta Ws of reference points into parameters, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3096
diff changeset
105 return standardDeviation;
cbf308f5c41b FixA: Store the standard deviation of delta Ws of reference points into parameters, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3096
diff changeset
106 }
cbf308f5c41b FixA: Store the standard deviation of delta Ws of reference points into parameters, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3096
diff changeset
107
9348
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
108 public boolean fit(double[] qs, double[] ws) {
3011
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
109
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
110 TDoubleArrayList xs = new TDoubleArrayList(qs.length);
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
111 TDoubleArrayList ys = new TDoubleArrayList(ws.length);
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
112
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
113 for (int i = 0; i < qs.length; ++i) {
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
114 if (!Double.isNaN(qs[i]) && !Double.isNaN(ws[i])) {
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
115 xs.add(qs[i]);
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
116 ys.add(ws[i]);
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
117 }
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
118 }
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
119
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
120 if (xs.size() < 2) {
3073
0ace00c0c12a FixA: Improved W/Q facet code
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3066
diff changeset
121 log.warn("Too less points.");
3011
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
122 return false;
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
123 }
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
124
3565
b136113dad53 FixA: Only evict only one(!) data point as outlier before recalculating the function.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3203
diff changeset
125 List<Double> inputs = new ArrayList<Double>(xs.size());
3011
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
126
5831
bd047b71ab37 Repaired internal references
Sascha L. Teichmann <teichmann@intevation.de>
parents: 4794
diff changeset
127 org.dive4elements.river.artifacts.math.Function instance = null;
3096
d7b0f52d6d04 FixA: Calculate Delta W(t) for reference points, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3073
diff changeset
128
3202
307842cf8d9e FixA: Make fitting more tolerant.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3127
diff changeset
129 LevenbergMarquardtOptimizer lmo = null;
307842cf8d9e FixA: Make fitting more tolerant.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3127
diff changeset
130
3011
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
131 for (;;) {
3202
307842cf8d9e FixA: Make fitting more tolerant.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3127
diff changeset
132 parameters = null;
7300
83bb52fa0c32 (issue1529) Be more tolerant in the fitting.
Andre Heinecke <aheinecke@intevation.de>
parents: 6868
diff changeset
133 for (double tolerance = 1e-10; tolerance < 1e-1; tolerance *= 10d) {
3011
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
134
3202
307842cf8d9e FixA: Make fitting more tolerant.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3127
diff changeset
135 lmo = new LevenbergMarquardtOptimizer();
307842cf8d9e FixA: Make fitting more tolerant.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3127
diff changeset
136 lmo.setCostRelativeTolerance(tolerance);
307842cf8d9e FixA: Make fitting more tolerant.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3127
diff changeset
137 lmo.setOrthoTolerance(tolerance);
307842cf8d9e FixA: Make fitting more tolerant.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3127
diff changeset
138 lmo.setParRelativeTolerance(tolerance);
307842cf8d9e FixA: Make fitting more tolerant.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3127
diff changeset
139
307842cf8d9e FixA: Make fitting more tolerant.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3127
diff changeset
140 CurveFitter cf = new CurveFitter(lmo);
307842cf8d9e FixA: Make fitting more tolerant.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3127
diff changeset
141
307842cf8d9e FixA: Make fitting more tolerant.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3127
diff changeset
142 for (int i = 0, N = xs.size(); i < N; ++i) {
307842cf8d9e FixA: Make fitting more tolerant.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3127
diff changeset
143 cf.addObservedPoint(xs.getQuick(i), ys.getQuick(i));
307842cf8d9e FixA: Make fitting more tolerant.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3127
diff changeset
144 }
307842cf8d9e FixA: Make fitting more tolerant.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3127
diff changeset
145
307842cf8d9e FixA: Make fitting more tolerant.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3127
diff changeset
146 try {
307842cf8d9e FixA: Make fitting more tolerant.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3127
diff changeset
147 parameters = cf.fit(function, function.getInitialGuess());
307842cf8d9e FixA: Make fitting more tolerant.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3127
diff changeset
148 break;
307842cf8d9e FixA: Make fitting more tolerant.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3127
diff changeset
149 }
307842cf8d9e FixA: Make fitting more tolerant.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3127
diff changeset
150 catch (MathException me) {
307842cf8d9e FixA: Make fitting more tolerant.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3127
diff changeset
151 if (log.isDebugEnabled()) {
307842cf8d9e FixA: Make fitting more tolerant.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3127
diff changeset
152 log.debug("tolerance " + tolerance + " + failed.");
307842cf8d9e FixA: Make fitting more tolerant.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3127
diff changeset
153 }
307842cf8d9e FixA: Make fitting more tolerant.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3127
diff changeset
154 }
3011
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
155 }
3202
307842cf8d9e FixA: Make fitting more tolerant.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3127
diff changeset
156 if (parameters == null) {
7300
83bb52fa0c32 (issue1529) Be more tolerant in the fitting.
Andre Heinecke <aheinecke@intevation.de>
parents: 6868
diff changeset
157 /*
9348
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
158 * log.debug("Parameters is null");
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
159 * for (int i = 0, N = xs.size(); i < N; ++i) {
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
160 * log.debug("DATA: " + xs.getQuick(i) + " " + ys.getQuick(i));
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
161 * }
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
162 */
3011
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
163 return false;
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
164 }
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
165
3127
71484036b6ae FixA: Moved function instantiation to always have a valid function instance.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 3107
diff changeset
166 // This is the paraterized function for a given km.
71484036b6ae FixA: Moved function instantiation to always have a valid function instance.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 3107
diff changeset
167 instance = function.instantiate(parameters);
71484036b6ae FixA: Moved function instantiation to always have a valid function instance.
Raimund Renkert <raimund.renkert@intevation.de>
parents: 3107
diff changeset
168
3022
705d2058b682 FixA: Store the referenced QW for each km, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3011
diff changeset
169 if (!checkOutliers) {
3011
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
170 break;
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
171 }
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
172
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
173 inputs.clear();
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
174
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
175 for (int i = 0, N = xs.size(); i < N; ++i) {
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
176 double y = instance.value(xs.getQuick(i));
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
177 if (Double.isNaN(y)) {
3565
b136113dad53 FixA: Only evict only one(!) data point as outlier before recalculating the function.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3203
diff changeset
178 y = Double.MAX_VALUE;
3011
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
179 }
3565
b136113dad53 FixA: Only evict only one(!) data point as outlier before recalculating the function.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3203
diff changeset
180 inputs.add(Double.valueOf(ys.getQuick(i) - y));
3011
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
181 }
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
182
4794
a7d080347ac3 MINFO: Allow two methods for outlier test in SQ relation.
Raimund Renkert <rrenkert@intevation.de>
parents: 3729
diff changeset
183 Integer outlier = GrubbsOutlier.findOutlier(inputs);
3011
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
184
3565
b136113dad53 FixA: Only evict only one(!) data point as outlier before recalculating the function.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3203
diff changeset
185 if (outlier == null) {
3011
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
186 break;
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
187 }
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
188
3565
b136113dad53 FixA: Only evict only one(!) data point as outlier before recalculating the function.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3203
diff changeset
189 int idx = outlier.intValue();
9348
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
190 removed.add(qwdFactory.create(xs.getQuick(idx), ys.getQuick(idx), true));
3565
b136113dad53 FixA: Only evict only one(!) data point as outlier before recalculating the function.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3203
diff changeset
191 xs.remove(idx);
b136113dad53 FixA: Only evict only one(!) data point as outlier before recalculating the function.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3203
diff changeset
192 ys.remove(idx);
3011
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
193 }
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
194
3107
cbf308f5c41b FixA: Store the standard deviation of delta Ws of reference points into parameters, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3096
diff changeset
195 StandardDeviation stdDev = new StandardDeviation();
cbf308f5c41b FixA: Store the standard deviation of delta Ws of reference points into parameters, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3096
diff changeset
196
3096
d7b0f52d6d04 FixA: Calculate Delta W(t) for reference points, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3073
diff changeset
197 referenced = new QWD[xs.size()];
3022
705d2058b682 FixA: Store the referenced QW for each km, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3011
diff changeset
198 for (int i = 0; i < referenced.length; ++i) {
9348
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
199 QWD qwd = qwdFactory.create(xs.getQuick(i), ys.getQuick(i), false);
3066
4c22194b733a Some minor bugfixes in fix analysis calculation and some more debug output.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 3065
diff changeset
200
3096
d7b0f52d6d04 FixA: Calculate Delta W(t) for reference points, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3073
diff changeset
201 if (qwd == null) {
3066
4c22194b733a Some minor bugfixes in fix analysis calculation and some more debug output.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 3065
diff changeset
202 log.warn("QW creation failed!");
9348
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
203 } else {
3096
d7b0f52d6d04 FixA: Calculate Delta W(t) for reference points, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3073
diff changeset
204 referenced[i] = qwd;
9348
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
205 double dw = (qwd.getW() - instance.value(qwd.getQ())) * 100.0;
3096
d7b0f52d6d04 FixA: Calculate Delta W(t) for reference points, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3073
diff changeset
206 qwd.setDeltaW(dw);
3107
cbf308f5c41b FixA: Store the standard deviation of delta Ws of reference points into parameters, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3096
diff changeset
207 stdDev.increment(dw);
3066
4c22194b733a Some minor bugfixes in fix analysis calculation and some more debug output.
Ingo Weinzierl <ingo.weinzierl@intevation.de>
parents: 3065
diff changeset
208 }
3022
705d2058b682 FixA: Store the referenced QW for each km, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3011
diff changeset
209 }
705d2058b682 FixA: Store the referenced QW for each km, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3011
diff changeset
210
3107
cbf308f5c41b FixA: Store the standard deviation of delta Ws of reference points into parameters, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3096
diff changeset
211 standardDeviation = stdDev.getResult();
cbf308f5c41b FixA: Store the standard deviation of delta Ws of reference points into parameters, too.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3096
diff changeset
212
3011
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
213 chiSqr = lmo.getChiSquare();
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
214
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
215 return true;
ab81ffd1343e FixA: Reactivated rewrite of the outlier checks.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
216 }
9348
a3f318347707 Show wq outliers within same thems with different symbol: not ready yet
gernotbelger
parents: 7300
diff changeset
217 }
http://dive4elements.wald.intevation.org