Mercurial > dive4elements > river

annotate artifacts/src/main/java/org/dive4elements/river/artifacts/model/sq/Fitting.java @ 9726:0a5239a1e46e 3.2.x

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Upgrade to Log4j 2
author Tom Gottfried <tom@intevation.de>
date Wed, 02 Mar 2022 10:26:50 +0100
parents 51eb6491c537
children
rev   line source
5863
4897a58c8746 River artifacts: Added new copyright headers.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
1 /* Copyright (C) 2011, 2012, 2013 by Bundesanstalt für Gewässerkunde
4897a58c8746 River artifacts: Added new copyright headers.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
2 * Software engineering by Intevation GmbH
4897a58c8746 River artifacts: Added new copyright headers.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
3 *
5994
af13ceeba52a Removed trailing whitespace.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5863
diff changeset
4 * This file is Free Software under the GNU AGPL (>=v3)
5863
4897a58c8746 River artifacts: Added new copyright headers.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
5 * and comes with ABSOLUTELY NO WARRANTY! Check out the
5994
af13ceeba52a Removed trailing whitespace.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5863
diff changeset
6 * documentation coming with Dive4Elements River for details.
5863
4897a58c8746 River artifacts: Added new copyright headers.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
7 */
4897a58c8746 River artifacts: Added new copyright headers.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5838
diff changeset
8
5831
bd047b71ab37 Repaired internal references
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5396
diff changeset
9 package org.dive4elements.river.artifacts.model.sq;
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
10
6787
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
11 import org.dive4elements.artifacts.common.utils.StringUtils;
5831
bd047b71ab37 Repaired internal references
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5396
diff changeset
12 import org.dive4elements.river.artifacts.math.fitting.Function;
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
13
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
14 import java.util.ArrayList;
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
15 import java.util.List;
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
16
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
17 import org.apache.commons.math.MathException;
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
18
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
19 import org.apache.commons.math.optimization.fitting.CurveFitter;
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
20
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
21 import org.apache.commons.math.optimization.general.LevenbergMarquardtOptimizer;
6777
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
22 import org.apache.commons.math.stat.regression.SimpleRegression;
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
23
9726
0a5239a1e46e Upgrade to Log4j 2
Tom Gottfried <tom@intevation.de>
parents: 6787
diff changeset
24 import org.apache.logging.log4j.Logger;
0a5239a1e46e Upgrade to Log4j 2
Tom Gottfried <tom@intevation.de>
parents: 6787
diff changeset
25 import org.apache.logging.log4j.LogManager;
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
26
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
27 public class Fitting
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
28 implements Outlier.Callback
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
29 {
6761
9479cb7c8cd5 flys/issue748: Force linear curve fitting. This is a real hack! Set the system property "minfo.sq.fitting.nonlinear" to re-enable the old behavior.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5994
diff changeset
30 // XXX: Hack to force linear fitting!
9479cb7c8cd5 flys/issue748: Force linear curve fitting. This is a real hack! Set the system property "minfo.sq.fitting.nonlinear" to re-enable the old behavior.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5994
diff changeset
31 private static final boolean USE_NON_LINEAR_FITTING =
9479cb7c8cd5 flys/issue748: Force linear curve fitting. This is a real hack! Set the system property "minfo.sq.fitting.nonlinear" to re-enable the old behavior.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5994
diff changeset
32 Boolean.getBoolean("minfo.sq.fitting.nonlinear");
9479cb7c8cd5 flys/issue748: Force linear curve fitting. This is a real hack! Set the system property "minfo.sq.fitting.nonlinear" to re-enable the old behavior.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5994
diff changeset
33
9726
0a5239a1e46e Upgrade to Log4j 2
Tom Gottfried <tom@intevation.de>
parents: 6787
diff changeset
34 private static Logger log = LogManager.getLogger(Fitting.class);
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
35
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
36 public interface Callback {
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
37
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
38 void afterIteration(
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
39 double [] parameters,
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
40 SQ [] measurements,
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
41 SQ [] outliers,
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
42 double standardDeviation,
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
43 double chiSqr);
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
44 } // interfacte
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
45
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
46 protected Function function;
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
47
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
48 protected double [] coeffs;
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
49
5831
bd047b71ab37 Repaired internal references
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5396
diff changeset
50 protected org.dive4elements.river.artifacts.math.Function instance;
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
51
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
52 protected double stdDevFactor;
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
53 protected double chiSqr;
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
54
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
55 protected Callback callback;
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
56
6780
b8f94e865875 S/Q relation: Part I of 'faking' fitting linear data. S/Qs can now be subclassed and processed through views. TODO: Add knob to setup a linear data path.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6777
diff changeset
57 protected SQ.View sqView;
b8f94e865875 S/Q relation: Part I of 'faking' fitting linear data. S/Qs can now be subclassed and processed through views. TODO: Add knob to setup a linear data path.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6777
diff changeset
58
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
59 public Fitting() {
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
60 }
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
61
6780
b8f94e865875 S/Q relation: Part I of 'faking' fitting linear data. S/Qs can now be subclassed and processed through views. TODO: Add knob to setup a linear data path.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6777
diff changeset
62 public Fitting(Function function, double stdDevFactor, SQ.View sqView) {
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
63 this.function = function;
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
64 this.stdDevFactor = stdDevFactor;
6780
b8f94e865875 S/Q relation: Part I of 'faking' fitting linear data. S/Qs can now be subclassed and processed through views. TODO: Add knob to setup a linear data path.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6777
diff changeset
65 this.sqView = sqView;
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
66 }
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
67
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
68 public Function getFunction() {
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
69 return function;
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
70 }
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
71
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
72 public void setFunction(Function function) {
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
73 this.function = function;
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
74 }
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
75
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
76 public double getStdDevFactor() {
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
77 return stdDevFactor;
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
78 }
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
79
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
80 public void setStdDevFactor(double stdDevFactor) {
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
81 this.stdDevFactor = stdDevFactor;
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
82 }
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
83
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
84 @Override
3566
8d0f06b76e09 S/Q relation: Fixed flys/issue748
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3552
diff changeset
85 public void initialize(List<SQ> sqs) throws MathException {
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
86
6777
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
87 if (USE_NON_LINEAR_FITTING
6787
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
88 || function.getParameterNames().length != 2) {
6777
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
89 nonLinearFitting(sqs);
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
90 }
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
91 else {
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
92 linearFitting(sqs);
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
93 }
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
94 }
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
95
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
96 protected void linearFitting(List<SQ> sqs) {
6787
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
97 coeffs = linearRegression(sqs);
6777
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
98 instance = function.instantiate(coeffs);
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
99 }
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
100
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
101 protected double [] linearRegression(List<SQ> sqs) {
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
102
6787
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
103 String [] pns = function.getParameterNames();
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
104 double [] result = new double[pns.length];
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
105
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
106 if (sqs.size() < 2) {
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
107 log.debug("not enough points");
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
108 return result;
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
109 }
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
110
6777
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
111 SimpleRegression reg = new SimpleRegression();
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
112
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
113 for (SQ sq: sqs) {
6787
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
114 double s = sqView.getS(sq);
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
115 double q = sqView.getQ(sq);
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
116 reg.addData(q, s);
6777
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
117 }
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
118
6787
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
119 double m = reg.getIntercept();
6777
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
120 double b = reg.getSlope();
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
121
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
122 if (log.isDebugEnabled()) {
6787
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
123 log.debug("m: " + m);
6777
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
124 log.debug("b: " + b);
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
125 }
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
126
6787
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
127 int mIdx = StringUtils.indexOf("m", pns);
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
128 int bIdx = StringUtils.indexOf("b", pns);
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
129
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
130 if (mIdx == -1 || bIdx == -1) {
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
131 log.error("index not found: " + mIdx + " " + bIdx);
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
132 return result;
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
133 }
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
134
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
135 result[bIdx] = m;
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
136 result[mIdx] = b;
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
137
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
138 return result;
6777
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
139 }
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
140
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
141
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
142 protected void nonLinearFitting(List<SQ> sqs) throws MathException {
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
143
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
144 LevenbergMarquardtOptimizer optimizer =
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
145 new LevenbergMarquardtOptimizer();
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
146
6761
9479cb7c8cd5 flys/issue748: Force linear curve fitting. This is a real hack! Set the system property "minfo.sq.fitting.nonlinear" to re-enable the old behavior.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5994
diff changeset
147 CurveFitter cf = new CurveFitter(optimizer);
6777
48f6780c372d S/Q relation: More Excel compat.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6761
diff changeset
148
3566
8d0f06b76e09 S/Q relation: Fixed flys/issue748
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3552
diff changeset
149 for (SQ sq: sqs) {
6787
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
150 cf.addObservedPoint(sqView.getQ(sq), sqView.getS(sq));
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
151 }
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
152
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
153 coeffs = cf.fit(
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
154 function, function.getInitialGuess());
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
155
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
156 instance = function.instantiate(coeffs);
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
157
6761
9479cb7c8cd5 flys/issue748: Force linear curve fitting. This is a real hack! Set the system property "minfo.sq.fitting.nonlinear" to re-enable the old behavior.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5994
diff changeset
158 chiSqr = optimizer.getChiSquare();
9479cb7c8cd5 flys/issue748: Force linear curve fitting. This is a real hack! Set the system property "minfo.sq.fitting.nonlinear" to re-enable the old behavior.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5994
diff changeset
159 }
9479cb7c8cd5 flys/issue748: Force linear curve fitting. This is a real hack! Set the system property "minfo.sq.fitting.nonlinear" to re-enable the old behavior.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 5994
diff changeset
160
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
161 @Override
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
162 public double eval(SQ sq) {
6780
b8f94e865875 S/Q relation: Part I of 'faking' fitting linear data. S/Qs can now be subclassed and processed through views. TODO: Add knob to setup a linear data path.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6777
diff changeset
163 double s = instance.value(sqView.getQ(sq));
b8f94e865875 S/Q relation: Part I of 'faking' fitting linear data. S/Qs can now be subclassed and processed through views. TODO: Add knob to setup a linear data path.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6777
diff changeset
164 return sqView.getS(sq) - s;
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
165 }
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
166
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
167 @Override
3566
8d0f06b76e09 S/Q relation: Fixed flys/issue748
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3552
diff changeset
168 public void iterationFinished(
8d0f06b76e09 S/Q relation: Fixed flys/issue748
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3552
diff changeset
169 double standardDeviation,
8d0f06b76e09 S/Q relation: Fixed flys/issue748
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3552
diff changeset
170 SQ outlier,
8d0f06b76e09 S/Q relation: Fixed flys/issue748
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3552
diff changeset
171 List<SQ> remainings
8d0f06b76e09 S/Q relation: Fixed flys/issue748
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3552
diff changeset
172 ) {
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
173 if (log.isDebugEnabled()) {
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
174 log.debug("iterationFinished ----");
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
175 log.debug(" num remainings: " + remainings.size());
3566
8d0f06b76e09 S/Q relation: Fixed flys/issue748
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3552
diff changeset
176 log.debug(" has outlier: " + outlier != null);
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
177 log.debug(" standardDeviation: " + standardDeviation);
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
178 log.debug(" Chi^2: " + chiSqr);
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
179 log.debug("---- iterationFinished");
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
180 }
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
181 callback.afterIteration(
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
182 coeffs,
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
183 remainings.toArray(new SQ[remainings.size()]),
3566
8d0f06b76e09 S/Q relation: Fixed flys/issue748
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3552
diff changeset
184 outlier != null ? new SQ [] { outlier} : new SQ [] {},
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
185 standardDeviation,
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
186 chiSqr);
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
187 }
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
188
6787
51eb6491c537 S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property S/Q: Excel compat completed: Now the data is linearized before fitting. This can be prevented by setting the system property "minfo.sq.calcution.non.linear.fitting" to true.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6780
diff changeset
189 public boolean fit(List<SQ> sqs, String method, Callback callback) {
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
190
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
191 if (sqs.size() < 2) {
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
192 log.warn("Too less points for fitting.");
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
193 return false;
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
194 }
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
195
6780
b8f94e865875 S/Q relation: Part I of 'faking' fitting linear data. S/Qs can now be subclassed and processed through views. TODO: Add knob to setup a linear data path.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6777
diff changeset
196 sqs = new ArrayList<SQ>(sqs);
b8f94e865875 S/Q relation: Part I of 'faking' fitting linear data. S/Qs can now be subclassed and processed through views. TODO: Add knob to setup a linear data path.
Sascha L. Teichmann <teichmann@intevation.de>
parents: 6777
diff changeset
197
3552
1df6984628c3 S/Q: Extented the result data model of the S/Q calculation to
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents: 3190
diff changeset
198 this.callback = callback;
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
199
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
200 try {
5396
aaf810d4ec82 Use new state 'outlier mehtod selection' in SQ-relation.
Raimund Renkert <rrenkert@intevation.de>
parents: 3572
diff changeset
201 Outlier.detectOutliers(this, sqs, stdDevFactor, method);
3188
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
202 }
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
203 catch (MathException me) {
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
204 log.warn(me);
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
205 return false;
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
206 }
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
207
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
208 return true;
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
209 }
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
210 }
1e46ced2bb57 SQ: Added fitting shell for SQ curves.
Sascha L. Teichmann <sascha.teichmann@intevation.de>
parents:
diff changeset
211 // vim:set ts=4 sw=4 si et sta sts=4 fenc=utf8 :
http://dive4elements.wald.intevation.org