Mercurial > dive4elements > river

annotate artifacts/src/main/java/org/dive4elements/river/artifacts/model/sq/Fitting.java @ 6787:51eb6491c537

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
author Sascha L. Teichmann <teichmann@intevation.de>
date Thu, 08 Aug 2013 18:14:38 +0200
parents b8f94e865875
children 0a5239a1e46e
rev   line source
5863
4897a58c8746 River artifacts: Added new copyright headers.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
1 /* Copyright (C) 2011, 2012, 2013 by Bundesanstalt für Gewässerkunde
4897a58c8746 River artifacts: Added new copyright headers.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
2 * Software engineering by Intevation GmbH
4897a58c8746 River artifacts: Added new copyright headers.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
3 *
5994
af13ceeba52a Removed trailing whitespace.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5863
diff changeset
4 * This file is Free Software under the GNU AGPL (>=v3)
5863
4897a58c8746 River artifacts: Added new copyright headers.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
5 * and comes with ABSOLUTELY NO WARRANTY! Check out the
5994
af13ceeba52a Removed trailing whitespace.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5863
diff changeset
6 * documentation coming with Dive4Elements River for details.
5863
4897a58c8746 River artifacts: Added new copyright headers.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
7 */
4897a58c8746 River artifacts: Added new copyright headers.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
8
5831
bd047b71ab37 Repaired internal references
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5396
diff changeset
9 package org.dive4elements.river.artifacts.model.sq;
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
10
6787
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
11 import org.dive4elements.artifacts.common.utils.StringUtils;
5831
bd047b71ab37 Repaired internal references
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5396
diff changeset
12 import org.dive4elements.river.artifacts.math.fitting.Function;
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
13
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
14 import java.util.ArrayList;
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
15 import java.util.List;
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
16
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
17 import org.apache.commons.math.MathException;
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
18
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
19 import org.apache.commons.math.optimization.fitting.CurveFitter;
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
20
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
21 import org.apache.commons.math.optimization.general.LevenbergMarquardtOptimizer;
6777
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
22 import org.apache.commons.math.stat.regression.SimpleRegression;
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
23
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
24 import org.apache.log4j.Logger;
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
25
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
26 public class Fitting
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
27 implements Outlier.Callback
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
28 {
6761
9479cb7c8cd5 flys/issue748: Force linear curve fitting. This is a real hack! Set the system property "minfo.sq.fitting.nonlinear" to re-enable the old behavior.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5994
diff changeset
29 // XXX: Hack to force linear fitting!
9479cb7c8cd5 flys/issue748: Force linear curve fitting. This is a real hack! Set the system property "minfo.sq.fitting.nonlinear" to re-enable the old behavior.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5994
diff changeset
30 private static final boolean USE_NON_LINEAR_FITTING =
9479cb7c8cd5 flys/issue748: Force linear curve fitting. This is a real hack! Set the system property "minfo.sq.fitting.nonlinear" to re-enable the old behavior.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5994
diff changeset
31 Boolean.getBoolean("minfo.sq.fitting.nonlinear");
9479cb7c8cd5 flys/issue748: Force linear curve fitting. This is a real hack! Set the system property "minfo.sq.fitting.nonlinear" to re-enable the old behavior.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5994
diff changeset
32
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
33 private static Logger log = Logger.getLogger(Fitting.class);
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
34
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
35 public interface Callback {
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
36
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
37 void afterIteration(
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
38 double [] parameters,
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
39 SQ [] measurements,
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
40 SQ [] outliers,
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
41 double standardDeviation,
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
42 double chiSqr);
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
43 } // interfacte
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
44
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
45 protected Function function;
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
46
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
47 protected double [] coeffs;
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
48
5831
bd047b71ab37 Repaired internal references
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5396
diff changeset
49 protected org.dive4elements.river.artifacts.math.Function instance;
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
50
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
51 protected double stdDevFactor;
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
52 protected double chiSqr;
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
53
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
54 protected Callback callback;
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
55
6780
b8f94e865875 S/Q relation: Part I of 'faking' fitting linear data. S/Qs can now be subclassed and processed through views. TODO: Add knob to setup a linear data path.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6777
diff changeset
56 protected SQ.View sqView;
b8f94e865875 S/Q relation: Part I of 'faking' fitting linear data. S/Qs can now be subclassed and processed through views. TODO: Add knob to setup a linear data path.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6777
diff changeset
57
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
58 public Fitting() {
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
59 }
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
60
6780
b8f94e865875 S/Q relation: Part I of 'faking' fitting linear data. S/Qs can now be subclassed and processed through views. TODO: Add knob to setup a linear data path.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6777
diff changeset
61 public Fitting(Function function, double stdDevFactor, SQ.View sqView) {
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
62 this.function = function;
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
63 this.stdDevFactor = stdDevFactor;
6780
b8f94e865875 S/Q relation: Part I of 'faking' fitting linear data. S/Qs can now be subclassed and processed through views. TODO: Add knob to setup a linear data path.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6777
diff changeset
64 this.sqView = sqView;
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
65 }
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
66
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
67 public Function getFunction() {
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
68 return function;
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
69 }
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
70
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
71 public void setFunction(Function function) {
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
72 this.function = function;
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
73 }
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
74
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
75 public double getStdDevFactor() {
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
76 return stdDevFactor;
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
77 }
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
78
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
79 public void setStdDevFactor(double stdDevFactor) {
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
80 this.stdDevFactor = stdDevFactor;
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
81 }
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
82
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
83 @Override
3566
8d0f06b76e09 S/Q relation: Fixed flys/issue748
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3552
diff changeset
84 public void initialize(List<SQ> sqs) throws MathException {
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
85
6777
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
86 if (USE_NON_LINEAR_FITTING
6787
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
87 || function.getParameterNames().length != 2) {
6777
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
88 nonLinearFitting(sqs);
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
89 }
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
90 else {
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
91 linearFitting(sqs);
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
92 }
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
93 }
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
94
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
95 protected void linearFitting(List<SQ> sqs) {
6787
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
96 coeffs = linearRegression(sqs);
6777
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
97 instance = function.instantiate(coeffs);
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
98 }
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
99
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
100 protected double [] linearRegression(List<SQ> sqs) {
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
101
6787
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
102 String [] pns = function.getParameterNames();
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
103 double [] result = new double[pns.length];
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
104
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
105 if (sqs.size() < 2) {
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
106 log.debug("not enough points");
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
107 return result;
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
108 }
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
109
6777
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
110 SimpleRegression reg = new SimpleRegression();
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
111
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
112 for (SQ sq: sqs) {
6787
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
113 double s = sqView.getS(sq);
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
114 double q = sqView.getQ(sq);
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
115 reg.addData(q, s);
6777
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
116 }
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
117
6787
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
118 double m = reg.getIntercept();
6777
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
119 double b = reg.getSlope();
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
120
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
121 if (log.isDebugEnabled()) {
6787
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
122 log.debug("m: " + m);
6777
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
123 log.debug("b: " + b);
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
124 }
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
125
6787
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
126 int mIdx = StringUtils.indexOf("m", pns);
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
127 int bIdx = StringUtils.indexOf("b", pns);
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
128
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
129 if (mIdx == -1 || bIdx == -1) {
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
130 log.error("index not found: " + mIdx + " " + bIdx);
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
131 return result;
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
132 }
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
133
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
134 result[bIdx] = m;
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
135 result[mIdx] = b;
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
136
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
137 return result;
6777
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
138 }
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
139
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
140
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
141 protected void nonLinearFitting(List<SQ> sqs) throws MathException {
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
142
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
143 LevenbergMarquardtOptimizer optimizer =
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
144 new LevenbergMarquardtOptimizer();
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
145
6761
9479cb7c8cd5 flys/issue748: Force linear curve fitting. This is a real hack! Set the system property "minfo.sq.fitting.nonlinear" to re-enable the old behavior.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5994
diff changeset
146 CurveFitter cf = new CurveFitter(optimizer);
6777
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
147
3566
8d0f06b76e09 S/Q relation: Fixed flys/issue748
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3552
diff changeset
148 for (SQ sq: sqs) {
6787
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
149 cf.addObservedPoint(sqView.getQ(sq), sqView.getS(sq));
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
150 }
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
151
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
152 coeffs = cf.fit(
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
153 function, function.getInitialGuess());
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
154
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
155 instance = function.instantiate(coeffs);
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
156
6761
9479cb7c8cd5 flys/issue748: Force linear curve fitting. This is a real hack! Set the system property "minfo.sq.fitting.nonlinear" to re-enable the old behavior.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5994
diff changeset
157 chiSqr = optimizer.getChiSquare();
9479cb7c8cd5 flys/issue748: Force linear curve fitting. This is a real hack! Set the system property "minfo.sq.fitting.nonlinear" to re-enable the old behavior.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5994
diff changeset
158 }
9479cb7c8cd5 flys/issue748: Force linear curve fitting. This is a real hack! Set the system property "minfo.sq.fitting.nonlinear" to re-enable the old behavior.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5994
diff changeset
159
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
160 @Override
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
161 public double eval(SQ sq) {
6780
b8f94e865875 S/Q relation: Part I of 'faking' fitting linear data. S/Qs can now be subclassed and processed through views. TODO: Add knob to setup a linear data path.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6777
diff changeset
162 double s = instance.value(sqView.getQ(sq));
b8f94e865875 S/Q relation: Part I of 'faking' fitting linear data. S/Qs can now be subclassed and processed through views. TODO: Add knob to setup a linear data path.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6777
diff changeset
163 return sqView.getS(sq) - s;
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
164 }
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
165
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
166 @Override
3566
8d0f06b76e09 S/Q relation: Fixed flys/issue748
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3552
diff changeset
167 public void iterationFinished(
8d0f06b76e09 S/Q relation: Fixed flys/issue748
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3552
diff changeset
168 double standardDeviation,
8d0f06b76e09 S/Q relation: Fixed flys/issue748
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3552
diff changeset
169 SQ outlier,
8d0f06b76e09 S/Q relation: Fixed flys/issue748
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3552
diff changeset
170 List<SQ> remainings
8d0f06b76e09 S/Q relation: Fixed flys/issue748
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3552
diff changeset
171 ) {
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
172 if (log.isDebugEnabled()) {
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
173 log.debug("iterationFinished ----");
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
174 log.debug(" num remainings: " + remainings.size());
3566
8d0f06b76e09 S/Q relation: Fixed flys/issue748
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3552
diff changeset
175 log.debug(" has outlier: " + outlier != null);
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
176 log.debug(" standardDeviation: " + standardDeviation);
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
177 log.debug(" Chi^2: " + chiSqr);
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
178 log.debug("---- iterationFinished");
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
179 }
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
180 callback.afterIteration(
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
181 coeffs,
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
182 remainings.toArray(new SQ[remainings.size()]),
3566
8d0f06b76e09 S/Q relation: Fixed flys/issue748
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3552
diff changeset
183 outlier != null ? new SQ [] { outlier} : new SQ [] {},
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
184 standardDeviation,
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
185 chiSqr);
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
186 }
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
187
6787
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
188 public boolean fit(List<SQ> sqs, String method, Callback callback) {
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
189
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
190 if (sqs.size() < 2) {
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
191 log.warn("Too less points for fitting.");
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
192 return false;
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
193 }
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
194
6780
b8f94e865875 S/Q relation: Part I of 'faking' fitting linear data. S/Qs can now be subclassed and processed through views. TODO: Add knob to setup a linear data path.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6777
diff changeset
195 sqs = new ArrayList<SQ>(sqs);
b8f94e865875 S/Q relation: Part I of 'faking' fitting linear data. S/Qs can now be subclassed and processed through views. TODO: Add knob to setup a linear data path.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6777
diff changeset
196
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
197 this.callback = callback;
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
198
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
199 try {
5396
aaf810d4ec82 Use new state 'outlier mehtod selection' in SQ-relation.
Raimund Renkert <rrenkert@intevation.de>
parents: 3572
diff changeset
200 Outlier.detectOutliers(this, sqs, stdDevFactor, method);
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
201 }
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
202 catch (MathException me) {
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
203 log.warn(me);
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
204 return false;
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
205 }
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
206
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
207 return true;
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
208 }
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
209 }
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
210 // vim:set ts=4 sw=4 si et sta sts=4 fenc=utf8 :
http://dive4elements.wald.intevation.org