Hi Guys. I am struggling to use the least squares optimizer to fit a 2 parameter non-linear function to a curve of observed data points. I am pretty sure I am doing something stupid because I don't know the maths. My MultivariateJacobianFunction moves away from my starting values on the first iteration but then converges back to the starting values. My parameter are CP and W' and this is what happens:
cp 250.0 W' 24000.0 (starting values) cp 197.17292724155843 W' 5627.212534968825 cp 233.7927945744999 W' 18662.916420529345 cp 245.72618039512386 W' 22592.92114117481 cp 248.91747806252718 W' 23643.613738664597 cp 249.99999974080222 W' 23999.9999146684 ... cp 249.99999993520052 W' 23999.99997866709 optimum {250; 24,000} Any ideas? Here is the code: public class LeastSquaresExample { private static final double DELTA = 0.000001; // for calculating derivatives public static void main(String[] args) { Vector2D[] observedPoints = new Vector2D[3]; observedPoints[0] = new Vector2D(388, 250); // maps power in watts to time in seconds observedPoints[1] = new Vector2D(368, 450); observedPoints[2] = new Vector2D(321, 780); int pmax = 961; MultivariateJacobianFunction fn = point -> { double cp = point.getEntry(0); double wPrime = point.getEntry(1); System.out.println("cp " + cp + " W' " + wPrime); RealVector value = new ArrayRealVector(observedPoints.length); RealMatrix jacobian = new Array2DRowRealMatrix(observedPoints.length, 2); for (int i = 0; i < observedPoints.length; i++) { double p = observedPoints[i].getX(); value.setEntry(i, morton3pTime(cp, wPrime, pmax, p)); // each row in the jacobian is a measurement and cols are partial derivatives wrt cp(0) and wPrime(1) double a, b; a = morton3pTime(cp - DELTA, wPrime, pmax, p); b = morton3pTime(cp + DELTA, wPrime, pmax, p); jacobian.setEntry(i, 0, (a - b) / (DELTA * 2)); a = morton3pTime(cp, wPrime - DELTA, pmax, p); b = morton3pTime(cp, wPrime + DELTA, pmax, p); jacobian.setEntry(i, 1, (a - b) / (DELTA * 2)); } return new Pair<>(value, jacobian); }; double[] target = new double[observedPoints.length]; for (int i = 0; i < observedPoints.length; i++) target[i] = observedPoints[i].getY(); LeastSquaresProblem problem = new LeastSquaresBuilder() .start(new double[]{250.0, 24000.0}) .model(fn) .target(target) .maxEvaluations(1000) .maxIterations(1000) .build(); LeastSquaresOptimizer.Optimum optimum = new LevenbergMarquardtOptimizer().optimize(problem); RealVector pt = optimum.getPoint(); System.out.println("optimum " + pt); } private static double morton3pTime(double cp, double wPrime, double pmax, double p) { return wPrime / (p - cp) + wPrime / (cp - pmax); } }