Hi Edwin,
Interestingly, that Brown test problem is part of the test suite for
nlinear (see multifit_nlinear/test_brown3.c). It looks like I set gtol =
GSL_DBL_EPSILON^(0.9) for the test suite problems, so it is much
stricter than the recommended value in the manual.
As a general rule, determining convergence for optimization problems
is more of an art than a science, which is why GSL offers 2 seperate
rules (based on the step size and the gradient). I had tried to add a
third rule based on the change in ||f|| from one iteration to the next,
but I found this didn't work so well so I disabled it.
If the gradient test isn't working well, you could disable it by
setting gtol = 0, and then only the xtol test will be used.
But in any case I will take a look at this Brown test problem to try
to understand what is going on.
Thanks,
Patrick
On 03/22/2018 08:02 AM, Sarkissian, Edwin (398E) wrote:
Hello Patrick,
After I studied the gradient-test algorithm implemented in
gsl_multifit_nlinear_test, I realized that the algorithm avoids some problems
by scaling the gradient; however, the scaling method results in another
problem. Adding a constant to a cost function does not change the shape (ups
and downs, minima and maxima, ...) of the cost function, and minima will stay
where they are. However, the gradient-base convergence test given at
https://www.gnu.org/software/gsl/doc/html/nls.html#testing-for-convergence
could produce false results if one simple adds a large constant to the cost
function. For example, for the same x and the same gtol, the test may indicate
no-convergence for
( ||f(x)||^2 )/2
but indicate convergence for
( ||f(x)||^2 + ||C||^2 )/2
where C is constant.
I encountered such a problem when I was trying to find the minimum of a test
nonlinear least square problem due to Brown. It is the number 4 problem in
Section 3 of the attached document. For the gtol suggested at
https://www.gnu.org/software/gsl/doc/html/nls.html#testing-for-convergence, the
gradient-based convergence test gives a false positive (converged) result.
Regards,
Edwin
On 3/21/18, 12:26 PM, "Patrick Alken" <al...@colorado.edu> wrote:
Hello,
Can you provide a minimal working example program which illustrates
the issue?
Thanks,
Patrick
On 03/21/2018 08:02 AM, Sarkissian, Edwin (398E) wrote:
> Dear Madam/Sir,
>
> Sometimes, the function
> gsl_multifit_nlinear_test
> sets the argument info to 2 (converged based on gradient check) when the
gradient is very large. For example, in one case the gradient is [-999999,
0.179997].
>
> I have noticed the same issue with the function
> gsl_multifit_fdfsolver_test
>
> Thank you,
> Edwin Sarkissian