Re: Use non-linear least squares to fit a function

[Yaqiang Wang]

I found the solution to calculate partial gradients of the parameters
unsing FiniteDifferencesDifferentiator and DerivativeStructure:
https://github.com/meteoinfo/MeteoInfo/blob/master/meteoinfo-math/src/main/java/org/meteoinfo/math/optimize/MyParametricUnivariateFunction.java.
A test case was also added:
https://github.com/meteoinfo/MeteoInfo/blob/master/meteoinfo-math/src/test/java/org/meteoinfo/math/optimize/FiniteDifferencesDifferentiatorTest.java
.

Hope this will be helpful on this issue.

Regards
Yaqiang

On Tue, Sep 20, 2022 at 3:44 PM Yaqiang Wang  wrote:

> Unfortunately, I can not figure out how to calculate partial gradients
> using DerivativeStructure, so I used a simple way to calculate the
> gradients of the parameters (
> https://github.com/meteoinfo/MeteoInfo/blob/9586b1a850721e418370a8a3d90af1e8fea03674/meteoinfo-math/src/main/java/org/meteoinfo/math/optimize/MyParametricUnivariateFunction.java).
> The result looks reasonable:
> http://www.meteothink.org/downloads/temp/curve_fit.png.
>
> Regards
> Yaqiang
>
> On Tue, Aug 2, 2022 at 7:34 PM Gilles Sadowski 
> wrote:
>
>> Hello.
>>
>> Le mar. 2 août 2022 à 10:42, Yaqiang Wang  a
>> écrit :
>> >
>> > Gilles,
>> >
>> > What I gather from the documentation is that the intended purpose is
>> > > to track the values of some function and all its derivatives when the
>> > > function is defined programmatically (using the usual arithmetical
>> > > operators, and generalizations of the functions defined in the "Math"
>> > > JDK class).  IIUC, one gains automatic access to the derivatives
>> > > without defining them analytically (only the function need be
>> defined).
>> > >
>> >
>> > Yes, the function is defined programmatically.
>>
>> Perhaps have a look at
>>
>> https://commons.apache.org/proper/commons-math/commons-math4-legacy/apidocs/org/apache/commons/math4/legacy/analysis/differentiation/GradientFunction.html
>> The associated unit test could also help figure out how to use
>> "DerivativeStructure":
>>
>> https://commons.apache.org/proper/commons-math/commons-math4-legacy/xref-test/org/apache/commons/math4/legacy/analysis/differentiation/GradientFunctionTest.html
>>
>> >
>> > How is "function" defined here?
>> > >
>> >
>> > "function" is a ParameUnivariateFunction instance which implements
>> > UnivariateFunction interface with parameters (
>> >
>> https://github.com/meteoinfo/MeteoInfo/blob/master/meteoinfo-math/src/main/java/org/meteoinfo/math/optimize/ParamUnivariateFunction.java
>> ).
>> > Its is used to wrap the Jython function (
>> >
>> https://github.com/meteoinfo/MeteoInfo/blob/master/meteoinfo-lab/pylib/mipylib/numeric/optimize/minpack.py#L78-L135
>> > ).
>> >
>> > The test Jython script and result figure from current curve_fit function
>> > can be found here:
>> http://www.meteothink.org/downloads/temp/curve_fit-1.png
>> > , http://www.meteothink.org/downloads/temp/curve_fit-2.png,
>> > http://www.meteothink.org/downloads/temp/curve_fit-3.png.
>> >
>> > Your code of gradient method (as below) was also tested with good
>> result (
>> > http://www.meteothink.org/downloads/temp/curve_fit-4.png). But of
>> course it
>> > can only work for that specific function.
>>
>> Sure, but I still can't figure out where the problem is:  Do you want to
>> implement the fitting using numerical derivatives because the gradient
>> cannot be expressed simply, or just to save one input (the equivalent
>> of the code below)?
>> In the latter case, I believe that "DerivativeStructure" could do it.  But
>> so should it, with a package that performs symbolic differentiation (?).
>>
>> > public double[] gradient(double x, double ... parameters) {
>> > > final double a = parameters[0];
>> > > final double b = parameters[1];
>> > > final double c = parameters[2];
>> > > final double[] grad = new double[3];
>> > > grad[0] = Math.exp(-b * x);
>> > > grad[1] = -a * x * grad[0];
>> > > grad[2] = 1;
>> > > return grad;
>> > > }
>> > >
>>
>> Regards,
>> Gilles
>>
>> P.S. If you succeed in framing the solution to your problem with
>> "DrivativeStructure",
>>we are interested in documenting it in "SimpleCurveFitter".
>>
>> >>> [...]
>>
>> -
>> To unsubscribe, e-mail: user-unsubscr...@commons.apache.org
>> For additional commands, e-mail: user-h...@commons.apache.org
>>
>>
>
> --
> *
> Dr. Yaqiang Wang
> Chinese Academy of Meteorological Sciences (CAMS)
> 46, Zhong-Guan-Cun South Avenue
> Beijing, 100081
> China
>
> yaqiang.w...@gmail.com
>
> www.meteothink.org
> **
>


-- 
*
Dr. Yaqiang Wang
Chinese Academy of Meteorological Sciences (CAMS)
46, Zhong-Guan-Cun South Avenue
Beijing, 100081
China

yaqiang.w...@gmail.com

www.meteothink.org

Re: Use non-linear least squares to fit a function

[Yaqiang Wang]

Unfortunately, I can not figure out how to calculate partial gradients
using DerivativeStructure, so I used a simple way to calculate the
gradients of the parameters (
https://github.com/meteoinfo/MeteoInfo/blob/9586b1a850721e418370a8a3d90af1e8fea03674/meteoinfo-math/src/main/java/org/meteoinfo/math/optimize/MyParametricUnivariateFunction.java).
The result looks reasonable:
http://www.meteothink.org/downloads/temp/curve_fit.png.

Regards
Yaqiang

On Tue, Aug 2, 2022 at 7:34 PM Gilles Sadowski  wrote:

> Hello.
>
> Le mar. 2 août 2022 à 10:42, Yaqiang Wang  a
> écrit :
> >
> > Gilles,
> >
> > What I gather from the documentation is that the intended purpose is
> > > to track the values of some function and all its derivatives when the
> > > function is defined programmatically (using the usual arithmetical
> > > operators, and generalizations of the functions defined in the "Math"
> > > JDK class).  IIUC, one gains automatic access to the derivatives
> > > without defining them analytically (only the function need be defined).
> > >
> >
> > Yes, the function is defined programmatically.
>
> Perhaps have a look at
>
> https://commons.apache.org/proper/commons-math/commons-math4-legacy/apidocs/org/apache/commons/math4/legacy/analysis/differentiation/GradientFunction.html
> The associated unit test could also help figure out how to use
> "DerivativeStructure":
>
> https://commons.apache.org/proper/commons-math/commons-math4-legacy/xref-test/org/apache/commons/math4/legacy/analysis/differentiation/GradientFunctionTest.html
>
> >
> > How is "function" defined here?
> > >
> >
> > "function" is a ParameUnivariateFunction instance which implements
> > UnivariateFunction interface with parameters (
> >
> https://github.com/meteoinfo/MeteoInfo/blob/master/meteoinfo-math/src/main/java/org/meteoinfo/math/optimize/ParamUnivariateFunction.java
> ).
> > Its is used to wrap the Jython function (
> >
> https://github.com/meteoinfo/MeteoInfo/blob/master/meteoinfo-lab/pylib/mipylib/numeric/optimize/minpack.py#L78-L135
> > ).
> >
> > The test Jython script and result figure from current curve_fit function
> > can be found here:
> http://www.meteothink.org/downloads/temp/curve_fit-1.png
> > , http://www.meteothink.org/downloads/temp/curve_fit-2.png,
> > http://www.meteothink.org/downloads/temp/curve_fit-3.png.
> >
> > Your code of gradient method (as below) was also tested with good result
> (
> > http://www.meteothink.org/downloads/temp/curve_fit-4.png). But of
> course it
> > can only work for that specific function.
>
> Sure, but I still can't figure out where the problem is:  Do you want to
> implement the fitting using numerical derivatives because the gradient
> cannot be expressed simply, or just to save one input (the equivalent
> of the code below)?
> In the latter case, I believe that "DerivativeStructure" could do it.  But
> so should it, with a package that performs symbolic differentiation (?).
>
> > public double[] gradient(double x, double ... parameters) {
> > > final double a = parameters[0];
> > > final double b = parameters[1];
> > > final double c = parameters[2];
> > > final double[] grad = new double[3];
> > > grad[0] = Math.exp(-b * x);
> > > grad[1] = -a * x * grad[0];
> > > grad[2] = 1;
> > > return grad;
> > > }
> > >
>
> Regards,
> Gilles
>
> P.S. If you succeed in framing the solution to your problem with
> "DrivativeStructure",
>we are interested in documenting it in "SimpleCurveFitter".
>
> >>> [...]
>
> -
> To unsubscribe, e-mail: user-unsubscr...@commons.apache.org
> For additional commands, e-mail: user-h...@commons.apache.org
>
>

-- 
*
Dr. Yaqiang Wang
Chinese Academy of Meteorological Sciences (CAMS)
46, Zhong-Guan-Cun South Avenue
Beijing, 100081
China

yaqiang.w...@gmail.com

www.meteothink.org
**

Re: Use non-linear least squares to fit a function

[Gilles Sadowski]

Hello.

Le mar. 2 août 2022 à 10:42, Yaqiang Wang  a écrit :
>
> Gilles,
>
> What I gather from the documentation is that the intended purpose is
> > to track the values of some function and all its derivatives when the
> > function is defined programmatically (using the usual arithmetical
> > operators, and generalizations of the functions defined in the "Math"
> > JDK class).  IIUC, one gains automatic access to the derivatives
> > without defining them analytically (only the function need be defined).
> >
>
> Yes, the function is defined programmatically.

Perhaps have a look at
  
https://commons.apache.org/proper/commons-math/commons-math4-legacy/apidocs/org/apache/commons/math4/legacy/analysis/differentiation/GradientFunction.html
The associated unit test could also help figure out how to use
"DerivativeStructure":

https://commons.apache.org/proper/commons-math/commons-math4-legacy/xref-test/org/apache/commons/math4/legacy/analysis/differentiation/GradientFunctionTest.html

>
> How is "function" defined here?
> >
>
> "function" is a ParameUnivariateFunction instance which implements
> UnivariateFunction interface with parameters (
> https://github.com/meteoinfo/MeteoInfo/blob/master/meteoinfo-math/src/main/java/org/meteoinfo/math/optimize/ParamUnivariateFunction.java).
> Its is used to wrap the Jython function (
> https://github.com/meteoinfo/MeteoInfo/blob/master/meteoinfo-lab/pylib/mipylib/numeric/optimize/minpack.py#L78-L135
> ).
>
> The test Jython script and result figure from current curve_fit function
> can be found here: http://www.meteothink.org/downloads/temp/curve_fit-1.png
> , http://www.meteothink.org/downloads/temp/curve_fit-2.png,
> http://www.meteothink.org/downloads/temp/curve_fit-3.png.
>
> Your code of gradient method (as below) was also tested with good result (
> http://www.meteothink.org/downloads/temp/curve_fit-4.png). But of course it
> can only work for that specific function.

Sure, but I still can't figure out where the problem is:  Do you want to
implement the fitting using numerical derivatives because the gradient
cannot be expressed simply, or just to save one input (the equivalent
of the code below)?
In the latter case, I believe that "DerivativeStructure" could do it.  But
so should it, with a package that performs symbolic differentiation (?).

> public double[] gradient(double x, double ... parameters) {
> > final double a = parameters[0];
> > final double b = parameters[1];
> > final double c = parameters[2];
> > final double[] grad = new double[3];
> > grad[0] = Math.exp(-b * x);
> > grad[1] = -a * x * grad[0];
> > grad[2] = 1;
> > return grad;
> > }
> >

Regards,
Gilles

P.S. If you succeed in framing the solution to your problem with
"DrivativeStructure",
   we are interested in documenting it in "SimpleCurveFitter".

>>> [...]

-
To unsubscribe, e-mail: user-unsubscr...@commons.apache.org
For additional commands, e-mail: user-h...@commons.apache.org

Re: Use non-linear least squares to fit a function

[Yaqiang Wang]

Gilles,

What I gather from the documentation is that the intended purpose is
> to track the values of some function and all its derivatives when the
> function is defined programmatically (using the usual arithmetical
> operators, and generalizations of the functions defined in the "Math"
> JDK class).  IIUC, one gains automatic access to the derivatives
> without defining them analytically (only the function need be defined).
>

Yes, the function is defined programmatically.

How is "function" defined here?
>

"function" is a ParameUnivariateFunction instance which implements
UnivariateFunction interface with parameters (
https://github.com/meteoinfo/MeteoInfo/blob/master/meteoinfo-math/src/main/java/org/meteoinfo/math/optimize/ParamUnivariateFunction.java).
Its is used to wrap the Jython function (
https://github.com/meteoinfo/MeteoInfo/blob/master/meteoinfo-lab/pylib/mipylib/numeric/optimize/minpack.py#L78-L135
).

The test Jython script and result figure from current curve_fit function
can be found here: http://www.meteothink.org/downloads/temp/curve_fit-1.png
, http://www.meteothink.org/downloads/temp/curve_fit-2.png,
http://www.meteothink.org/downloads/temp/curve_fit-3.png.

Your code of gradient method (as below) was also tested with good result (
http://www.meteothink.org/downloads/temp/curve_fit-4.png). But of course it
can only work for that specific function.

public double[] gradient(double x, double ... parameters) {
> final double a = parameters[0];
> final double b = parameters[1];
> final double c = parameters[2];
> final double[] grad = new double[3];
> grad[0] = Math.exp(-b * x);
> grad[1] = -a * x * grad[0];
> grad[2] = 1;
> return grad;
> }
>

Regards
Yaqiang

On Tue, Aug 2, 2022 at 1:21 AM Gilles Sadowski  wrote:

> Hello.
>
> Le lun. 1 août 2022 à 16:03, Yaqiang Wang  a
> écrit :
> >
> > Gilles,
> >
> > Thanks so much for your patiently response! I know I can write a gradient
> > method for a specific function, but my purpose is to make the gradient
> > method suitable for any function of yi = f(xi, p1, p2, p3, ...). That
> means
> > the users don't need to override a new fixed gradient method for a new
> > function, just like the SciPy's curve_fit (
> >
> https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.curve_fit.html
> ).
> > So I tried to calculate gradient array through numerical differentiation
> (
> >
> https://github.com/meteoinfo/MeteoInfo/blob/master/meteoinfo-math/src/main/java/org/meteoinfo/math/optimize/MyParametricUnivariateFunction.java#L33-L56
> ,
> > the code also is attache below), and please let me know whether the code
> is
> > correct for my purpose? Thanks!
>
> I'm still confused about your use case (maybe because I did not see
> the figures of what you expect vs what you got).
>
> I've never used the "DerivativeStructure" (and I've just noticed that the
> link to the reference document is not accessible anymore).
>
> What I gather from the documentation is that the intended purpose is
> to track the values of some function and all its derivatives when the
> function is defined programmatically (using the usual arithmetical
> operators, and generalizations of the functions defined in the "Math"
> JDK class).  IIUC, one gains automatic access to the derivatives
> without defining them analytically (only the function need be defined).
>
> >
> > @Override
> > public double[] gradient(double v, double... parameters) {
> > function.setParameters(parameters);
>
> How is "function" defined here?
>
> >
> > // create a differentiator
> > FiniteDifferencesDifferentiator differentiator =
> > new FiniteDifferencesDifferentiator(nbPoints, stepSize);
>
> If you assume that "function" is defined analytically, you don't need to
> use "FiniteDifferentiator" (moreover, its use is not recomended IIUC
> the documentation).
> However, if the derivatives cannot be expressed analytically, it seems
> that "DerivativeStructure" is an overly complex utility if in the end, it's
> just replacing the finite differences formulae[1] which you can write in
> about the same number lines as your code below.
>
> Regards,
> Gilles
>
> [1] https://en.wikipedia.org/wiki/Numerical_differentiation
>
> >
> > // create a new function that computes both the value and the
> derivatives
> > // using DerivativeStructure
> > UnivariateDifferentiableFunction diffFunc =
> > differentiator.differentiate(function);
> >
> > double y = function.value(v);
> > int n = parameters.length;
> > double[] gradients = new double[n];
> > for (int i = 0; i < n; i++) {
> > DerivativeStructure xDS = new DerivativeStructure(n, 1, i,
> > parameters[i]);
> > DerivativeStructure yDS = diffFunc.value(xDS);
> > int[] idx = new int[n];
> > idx[i] = 1;
> > gradients[i] = yDS.getPartialDerivative(idx);
> > }
> >
> > return gradients;
> > }
> >
> >

Re: Use non-linear least squares to fit a function

[Gilles Sadowski]

Hello.

Le lun. 1 août 2022 à 16:03, Yaqiang Wang  a écrit :
>
> Gilles,
>
> Thanks so much for your patiently response! I know I can write a gradient
> method for a specific function, but my purpose is to make the gradient
> method suitable for any function of yi = f(xi, p1, p2, p3, ...). That means
> the users don't need to override a new fixed gradient method for a new
> function, just like the SciPy's curve_fit (
> https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.curve_fit.html).
> So I tried to calculate gradient array through numerical differentiation (
> https://github.com/meteoinfo/MeteoInfo/blob/master/meteoinfo-math/src/main/java/org/meteoinfo/math/optimize/MyParametricUnivariateFunction.java#L33-L56,
> the code also is attache below), and please let me know whether the code is
> correct for my purpose? Thanks!

I'm still confused about your use case (maybe because I did not see
the figures of what you expect vs what you got).

I've never used the "DerivativeStructure" (and I've just noticed that the
link to the reference document is not accessible anymore).

What I gather from the documentation is that the intended purpose is
to track the values of some function and all its derivatives when the
function is defined programmatically (using the usual arithmetical
operators, and generalizations of the functions defined in the "Math"
JDK class).  IIUC, one gains automatic access to the derivatives
without defining them analytically (only the function need be defined).

>
> @Override
> public double[] gradient(double v, double... parameters) {
> function.setParameters(parameters);

How is "function" defined here?

>
> // create a differentiator
> FiniteDifferencesDifferentiator differentiator =
> new FiniteDifferencesDifferentiator(nbPoints, stepSize);

If you assume that "function" is defined analytically, you don't need to
use "FiniteDifferentiator" (moreover, its use is not recomended IIUC
the documentation).
However, if the derivatives cannot be expressed analytically, it seems
that "DerivativeStructure" is an overly complex utility if in the end, it's
just replacing the finite differences formulae[1] which you can write in
about the same number lines as your code below.

Regards,
Gilles

[1] https://en.wikipedia.org/wiki/Numerical_differentiation

>
> // create a new function that computes both the value and the derivatives
> // using DerivativeStructure
> UnivariateDifferentiableFunction diffFunc =
> differentiator.differentiate(function);
>
> double y = function.value(v);
> int n = parameters.length;
> double[] gradients = new double[n];
> for (int i = 0; i < n; i++) {
> DerivativeStructure xDS = new DerivativeStructure(n, 1, i,
> parameters[i]);
> DerivativeStructure yDS = diffFunc.value(xDS);
> int[] idx = new int[n];
> idx[i] = 1;
> gradients[i] = yDS.getPartialDerivative(idx);
> }
>
> return gradients;
> }
>
>
> By the way, I am using Apache commons math 3.6.1 at present. Today I also
> tried the 4.0-SNAPSHOT version but the result is the same.
>
> Regards
> Yaqiang
>
>>> [...]

-
To unsubscribe, e-mail: user-unsubscr...@commons.apache.org
For additional commands, e-mail: user-h...@commons.apache.org

Re: Use non-linear least squares to fit a function

[Yaqiang Wang]

Gilles,

Thanks so much for your patiently response! I know I can write a gradient
method for a specific function, but my purpose is to make the gradient
method suitable for any function of yi = f(xi, p1, p2, p3, ...). That means
the users don't need to override a new fixed gradient method for a new
function, just like the SciPy's curve_fit (
https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.curve_fit.html).
So I tried to calculate gradient array through numerical differentiation (
https://github.com/meteoinfo/MeteoInfo/blob/master/meteoinfo-math/src/main/java/org/meteoinfo/math/optimize/MyParametricUnivariateFunction.java#L33-L56,
the code also is attache below), and please let me know whether the code is
correct for my purpose? Thanks!

@Override
public double[] gradient(double v, double... parameters) {
function.setParameters(parameters);

// create a differentiator
FiniteDifferencesDifferentiator differentiator =
new FiniteDifferencesDifferentiator(nbPoints, stepSize);

// create a new function that computes both the value and the derivatives
// using DerivativeStructure
UnivariateDifferentiableFunction diffFunc =
differentiator.differentiate(function);

double y = function.value(v);
int n = parameters.length;
double[] gradients = new double[n];
for (int i = 0; i < n; i++) {
DerivativeStructure xDS = new DerivativeStructure(n, 1, i,
parameters[i]);
DerivativeStructure yDS = diffFunc.value(xDS);
int[] idx = new int[n];
idx[i] = 1;
gradients[i] = yDS.getPartialDerivative(idx);
}

return gradients;
}


By the way, I am using Apache commons math 3.6.1 at present. Today I also
tried the 4.0-SNAPSHOT version but the result is the same.

Regards
Yaqiang

On Mon, Aug 1, 2022 at 8:33 PM Gilles Sadowski  wrote:

> Hi.
>
> Le dim. 31 juil. 2022 à 18:05, Yaqiang Wang  a
> écrit :
> >
> > Currently I just want to fit univariate function such as the following
> Python function:
> >
> > def func(x, a, b, c):
> > return a * exp(-b * x) + c
> >
> >
> > I also tried using SimpleCurveFitter to do it.
> MyParametricUnivariateFunction implements ParametricUnivariateFunction and
> overrides value and gradient methods (
> https://github.com/meteoinfo/MeteoInfo/blob/master/meteoinfo-math/src/main/java/org/meteoinfo/math/optimize/MyParametricUnivariateFunction.java),
> and the gradient array is calculated through numerical differentiation.
>
> Given a list of points, { xi } (i = 0, 1, ... , N), and their associated
> values,
> { y_i }, "SimpleCurveFitter" aims at finding the best fit of an assumed
> function "f"
>   yi = f(xi, p1, p2, p3, ...)
> whose parameters p1, p2, p3, ... can be adjusted.
> Internally, "SimpleCurveFitter" computes the Jacobian matrix
>   df(x0)/dp1 df(x0)/dp2 df(x0)/dp3 ...
>   df(x1)/dp1 df(x1)/dp2 df(x1)/dp3 ...
>  
>   df(xN)/dp1 df(xN)/dp2 df(xN)/dp3 ...
> needed by the "Levenberg-Marquardt" least-squares optimizer.
> The gradient of "f" is an array where each slot contains the partial
> derivative of the univariate function (at the given "x") wrt to each
> _parameter_.
> For your function above, the code would be (untested):
> ---CUT---
> public class MyFunction implements ParametricUnivariateFunction {
>
> public double value(double x, double ... parameters) {
> final double a = parameters[0];
> final double b = parameters[1];
> final double c = parameters[2];
> return a * Math.exp(-b * x) + c
> }
>
> public double[] gradient(double x, double ... parameters) {
> final double a = parameters[0];
> final double b = parameters[1];
> final double c = parameters[2];
> final double[] grad = new double[3];
> grad[0] = Math.exp(-b * x);
> grad[1] = -a * x * grad[0];
> grad[2] = 1;
> return grad;
> }
> }
> ---CUT---
>
> HTH,
> Gilles
>
> > [...]
>
> -
> To unsubscribe, e-mail: user-unsubscr...@commons.apache.org
> For additional commands, e-mail: user-h...@commons.apache.org
>
>

-- 
*
Dr. Yaqiang Wang
Chinese Academy of Meteorological Sciences (CAMS)
46, Zhong-Guan-Cun South Avenue
Beijing, 100081
China

yaqiang.w...@gmail.com

www.meteothink.org
**

Re: Use non-linear least squares to fit a function

[Gilles Sadowski]

Hi.

Le dim. 31 juil. 2022 à 18:05, Yaqiang Wang  a écrit :
>
> Currently I just want to fit univariate function such as the following Python 
> function:
>
> def func(x, a, b, c):
> return a * exp(-b * x) + c
>
>
> I also tried using SimpleCurveFitter to do it. MyParametricUnivariateFunction 
> implements ParametricUnivariateFunction and overrides value and gradient 
> methods 
> (https://github.com/meteoinfo/MeteoInfo/blob/master/meteoinfo-math/src/main/java/org/meteoinfo/math/optimize/MyParametricUnivariateFunction.java),
>  and the gradient array is calculated through numerical differentiation.

Given a list of points, { xi } (i = 0, 1, ... , N), and their associated values,
{ y_i }, "SimpleCurveFitter" aims at finding the best fit of an assumed
function "f"
  yi = f(xi, p1, p2, p3, ...)
whose parameters p1, p2, p3, ... can be adjusted.
Internally, "SimpleCurveFitter" computes the Jacobian matrix
  df(x0)/dp1 df(x0)/dp2 df(x0)/dp3 ...
  df(x1)/dp1 df(x1)/dp2 df(x1)/dp3 ...
 
  df(xN)/dp1 df(xN)/dp2 df(xN)/dp3 ...
needed by the "Levenberg-Marquardt" least-squares optimizer.
The gradient of "f" is an array where each slot contains the partial
derivative of the univariate function (at the given "x") wrt to each
_parameter_.
For your function above, the code would be (untested):
---CUT---
public class MyFunction implements ParametricUnivariateFunction {

public double value(double x, double ... parameters) {
final double a = parameters[0];
final double b = parameters[1];
final double c = parameters[2];
return a * Math.exp(-b * x) + c
}

public double[] gradient(double x, double ... parameters) {
final double a = parameters[0];
final double b = parameters[1];
final double c = parameters[2];
final double[] grad = new double[3];
grad[0] = Math.exp(-b * x);
grad[1] = -a * x * grad[0];
grad[2] = 1;
return grad;
}
}
---CUT---

HTH,
Gilles

> [...]

-
To unsubscribe, e-mail: user-unsubscr...@commons.apache.org
For additional commands, e-mail: user-h...@commons.apache.org

Re: Use non-linear least squares to fit a function

[Gilles Sadowski]

Hi.

Le dim. 31 juil. 2022 à 18:05, Yaqiang Wang  a écrit :
>
> Currently I just want to fit univariate function such as the following Python 
> function:
>
> def func(x, a, b, c):
> return a * exp(-b * x) + c
>
>
> I also tried using SimpleCurveFitter to do it. MyParametricUnivariateFunction 
> implements ParametricUnivariateFunction and overrides value and gradient 
> methods 
> (https://github.com/meteoinfo/MeteoInfo/blob/master/meteoinfo-math/src/main/java/org/meteoinfo/math/optimize/MyParametricUnivariateFunction.java),
>  and the gradient array is calculated through numerical differentiation. The 
> curveFit function in OptimizeUtil class was developed for curve fitting using 
> SimpleCurveFitter 
> (https://github.com/meteoinfo/MeteoInfo/blob/master/meteoinfo-math/src/main/java/org/meteoinfo/math/optimize/OptimizeUtil.java#L121).
>  Then the Jython curve_fit function was implemented 
> (https://github.com/meteoinfo/MeteoInfo/blob/master/meteoinfo-lab/pylib/mipylib/numeric/optimize/minpack.py#L97).
>
> I tested curve_fit with a simple function and the result looks ok.
>
>
> But for the function I mentioned at the beginning, curve_fit can not return 
> the fitting result.

If you had attached figures to this message, the ML manager
quite possibly discarded them.
Perhaps better file a JIRA report:
   https://issues.apache.org/jira/projects/MATH

Regards,
Gilles

>
>
> I have no idea how to correct it.
>
> Regards
> Yaqiang
>
> On Sun, Jul 31, 2022 at 7:49 PM Gilles Sadowski  wrote:
>>
>> Le dim. 31 juil. 2022 à 03:46, Yaqiang Wang  a écrit 
>> :
>> >
>> > Gilles,
>> >
>> > Thanks for your reply! Concretely to the point, I want to implement SciPy
>> > curve_fit function using Java and Jython through Apache commons math
>> > library. Any idea of it?
>>
>> I guess that your function is multivariate (?); otherwise you'd use the
>> "SimpleCurveFitter" class (from package "o.a.c.math4.legacy.fitting".[1]
>>
>> The non-linear least-squares utility is defined in package
>> "o.a.c.math4.legacy.fitting.leastsquares".[2]
>> This functionality lets the user provide
>>  * the (multivariate) function
>>  * the Jacobian matrix
>> Do I understand correctly that you want to implement a utility that
>> automatically generates the Jacobian through numerical differentiation
>> of the function?
>>
>> Regards,
>> Gilles
>>
>> [1] 
>> https://commons.apache.org/proper/commons-math/commons-math4-legacy/apidocs/org/apache/commons/math4/legacy/fitting/SimpleCurveFitter.html
>> [2] 
>> https://commons.apache.org/proper/commons-math/commons-math4-legacy/apidocs/org/apache/commons/math4/legacy/fitting/leastsquares/package-summary.html
>>
>> > > [...]

-
To unsubscribe, e-mail: user-unsubscr...@commons.apache.org
For additional commands, e-mail: user-h...@commons.apache.org

Re: Use non-linear least squares to fit a function

[Yaqiang Wang]

Currently I just want to fit univariate function such as the following
Python function:

def func(x, a, b, c):
return a * exp(-b * x) + c


I also tried using SimpleCurveFitter to do
it. MyParametricUnivariateFunction implements ParametricUnivariateFunction
and overrides value and gradient methods (
https://github.com/meteoinfo/MeteoInfo/blob/master/meteoinfo-math/src/main/java/org/meteoinfo/math/optimize/MyParametricUnivariateFunction.java),
and the gradient array is calculated through numerical differentiation. The
curveFit function in OptimizeUtil class was developed for curve fitting
using SimpleCurveFitter (
https://github.com/meteoinfo/MeteoInfo/blob/master/meteoinfo-math/src/main/java/org/meteoinfo/math/optimize/OptimizeUtil.java#L121).
Then the Jython curve_fit function was implemented (
https://github.com/meteoinfo/MeteoInfo/blob/master/meteoinfo-lab/pylib/mipylib/numeric/optimize/minpack.py#L97
).

I tested curve_fit with a simple function and the result looks ok.
[image: cuve_fit_1.png]

But for the function I mentioned at the beginning, curve_fit can not return
the fitting result.
[image: curve_fit_2.png]

I have no idea how to correct it.

Regards
Yaqiang

On Sun, Jul 31, 2022 at 7:49 PM Gilles Sadowski 
wrote:

> Le dim. 31 juil. 2022 à 03:46, Yaqiang Wang  a
> écrit :
> >
> > Gilles,
> >
> > Thanks for your reply! Concretely to the point, I want to implement SciPy
> > curve_fit function using Java and Jython through Apache commons math
> > library. Any idea of it?
>
> I guess that your function is multivariate (?); otherwise you'd use the
> "SimpleCurveFitter" class (from package "o.a.c.math4.legacy.fitting".[1]
>
> The non-linear least-squares utility is defined in package
> "o.a.c.math4.legacy.fitting.leastsquares".[2]
> This functionality lets the user provide
>  * the (multivariate) function
>  * the Jacobian matrix
> Do I understand correctly that you want to implement a utility that
> automatically generates the Jacobian through numerical differentiation
> of the function?
>
> Regards,
> Gilles
>
> [1]
> https://commons.apache.org/proper/commons-math/commons-math4-legacy/apidocs/org/apache/commons/math4/legacy/fitting/SimpleCurveFitter.html
> [2]
> https://commons.apache.org/proper/commons-math/commons-math4-legacy/apidocs/org/apache/commons/math4/legacy/fitting/leastsquares/package-summary.html
>
> > > [...]
>
> -
> To unsubscribe, e-mail: user-unsubscr...@commons.apache.org
> For additional commands, e-mail: user-h...@commons.apache.org
>
>

-- 
*
Dr. Yaqiang Wang
Chinese Academy of Meteorological Sciences (CAMS)
46, Zhong-Guan-Cun South Avenue
Beijing, 100081
China

yaqiang.w...@gmail.com

www.meteothink.org
**

Re: Use non-linear least squares to fit a function

[Gilles Sadowski]

Le dim. 31 juil. 2022 à 03:46, Yaqiang Wang  a écrit :
>
> Gilles,
>
> Thanks for your reply! Concretely to the point, I want to implement SciPy
> curve_fit function using Java and Jython through Apache commons math
> library. Any idea of it?

I guess that your function is multivariate (?); otherwise you'd use the
"SimpleCurveFitter" class (from package "o.a.c.math4.legacy.fitting".[1]

The non-linear least-squares utility is defined in package
"o.a.c.math4.legacy.fitting.leastsquares".[2]
This functionality lets the user provide
 * the (multivariate) function
 * the Jacobian matrix
Do I understand correctly that you want to implement a utility that
automatically generates the Jacobian through numerical differentiation
of the function?

Regards,
Gilles

[1] 
https://commons.apache.org/proper/commons-math/commons-math4-legacy/apidocs/org/apache/commons/math4/legacy/fitting/SimpleCurveFitter.html
[2] 
https://commons.apache.org/proper/commons-math/commons-math4-legacy/apidocs/org/apache/commons/math4/legacy/fitting/leastsquares/package-summary.html

> > [...]

-
To unsubscribe, e-mail: user-unsubscr...@commons.apache.org
For additional commands, e-mail: user-h...@commons.apache.org

Re: Use non-linear least squares to fit a function

[Yaqiang Wang]

Gilles,

Thanks for your reply! Concretely to the point, I want to implement SciPy
curve_fit function using Java and Jython through Apache commons math
library. Any idea of it?

Regards
Yaqiang

On Sat, Jul 30, 2022 at 11:17 PM Gilles Sadowski 
wrote:

> Hello.
>
> Le sam. 30 juil. 2022 à 10:58, Yaqiang Wang  a
> écrit :
> >
> > I know implementing the ParametricUnivariateFunction
> > <
> https://commons.apache.org/proper/commons-math/apidocs/org/apache/commons/math4/analysis/ParametricUnivariateFunction.html
> >
> > interface
> > and overriding value and gradient methods can fit the custom curve
> > function. But the gradient array has to be calculated for each special
> > function.
> > I want to know is it possible to calculate gradient array without
> > the information of the function formula,
>
> I don't follow; what kind of expression do you want to minimize?
> A minimal example would be useful to understand what the issue is.
>
> > just like SciPy curve_fit function
> > (
> >
> https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.curve_fit.html
> )?
> > I have tried it in the MeteoInfo project using  getJacobianFunction which
> > calculates gradient array using UnivariateDifferentiableFunction and
> > DerivativeStructure.
> > And Jython was used to mimic the curve_fit function (
> >
> https://github.com/meteoinfo/MeteoInfo/blob/master/meteoinfo-lab/pylib/mipylib/numeric/optimize/minpack.py
> ).
> >
> >
> > But I can not get the same result as SciPy curve_fit function. I nee your
> > help, Thanks!
>
> What you want to achieve is not clear to me.
>
> Sorry,
> Gilles
>
> -
> To unsubscribe, e-mail: user-unsubscr...@commons.apache.org
> For additional commands, e-mail: user-h...@commons.apache.org
>
>

-- 
*
Dr. Yaqiang Wang
Chinese Academy of Meteorological Sciences (CAMS)
46, Zhong-Guan-Cun South Avenue
Beijing, 100081
China

yaqiang.w...@gmail.com

www.meteothink.org
**

Re: Use non-linear least squares to fit a function

[Gilles Sadowski]

Hello.

Le sam. 30 juil. 2022 à 10:58, Yaqiang Wang  a écrit :
>
> I know implementing the ParametricUnivariateFunction
> 
> interface
> and overriding value and gradient methods can fit the custom curve
> function. But the gradient array has to be calculated for each special
> function.
> I want to know is it possible to calculate gradient array without
> the information of the function formula,

I don't follow; what kind of expression do you want to minimize?
A minimal example would be useful to understand what the issue is.

> just like SciPy curve_fit function
> (
> https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.curve_fit.html)?
> I have tried it in the MeteoInfo project using  getJacobianFunction which
> calculates gradient array using UnivariateDifferentiableFunction and
> DerivativeStructure.
> And Jython was used to mimic the curve_fit function (
> https://github.com/meteoinfo/MeteoInfo/blob/master/meteoinfo-lab/pylib/mipylib/numeric/optimize/minpack.py).
>
>
> But I can not get the same result as SciPy curve_fit function. I nee your
> help, Thanks!

What you want to achieve is not clear to me.

Sorry,
Gilles

-
To unsubscribe, e-mail: user-unsubscr...@commons.apache.org
For additional commands, e-mail: user-h...@commons.apache.org

Re: Use non-linear least squares to fit a function

[Yaqiang Wang]

I forgot to add the linkage of  getJacobianFunction (
https://github.com/meteoinfo/MeteoInfo/blob/master/meteoinfo-math/src/main/java/org/meteoinfo/math/optimize/OptimizeUtil.java
 ).

Regards
Yaqiang

On Sat, Jul 30, 2022 at 4:58 PM Yaqiang Wang  wrote:

> I know implementing the ParametricUnivariateFunction
> 
>  interface
> and overriding value and gradient methods can fit the custom curve
> function. But the gradient array has to be calculated for each special
> function. I want to know is it possible to calculate gradient array without
> the information of the function formula, just like SciPy curve_fit function
> (
> https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.curve_fit.html)?
> I have tried it in the MeteoInfo project using  getJacobianFunction which
> calculates gradient array using UnivariateDifferentiableFunction and 
> DerivativeStructure.
> And Jython was used to mimic the curve_fit function (
> https://github.com/meteoinfo/MeteoInfo/blob/master/meteoinfo-lab/pylib/mipylib/numeric/optimize/minpack.py).
>
>
> But I can not get the same result as SciPy curve_fit function. I nee your
> help, Thanks!
>
> Regards
> Yaqiang
>
> --
> *
> Dr. Yaqiang Wang
> Chinese Academy of Meteorological Sciences (CAMS)
> 46, Zhong-Guan-Cun South Avenue
> Beijing, 100081
> China
>
> yaqiang.w...@gmail.com
>
> www.meteothink.org
> **
>


-- 
*
Dr. Yaqiang Wang
Chinese Academy of Meteorological Sciences (CAMS)
46, Zhong-Guan-Cun South Avenue
Beijing, 100081
China

yaqiang.w...@gmail.com

www.meteothink.org
**