On 11/07/2010 5:00 AM, Matthew Killeya wrote:
>Thanks. Seems to me the easiest sensible fix might be to change the
>default abs.tol=0 in R and add a warning in the help files?
>
That was exactly my suggestion in the message to which Ravi was
replying, but he apparently has doubts.
Duncan Murdoch
>Matt
>
>On 11 July 2010 01:41, Duncan Murdoch <murdoch.dun...@gmail.com> wrote:
>
>
>>On 10/07/2010 7:32 PM, Ravi Varadhan wrote:
>>
>>
>>>Hi,
>>>
>>>The best solution would be to identify where the problem is in the
FORTRAN
>>>code and correct it. However, this problem of premature
termination due to
>>>absolute function convergence is highly unlikely to occur in
practice. As
>>>John Nash noted, this is going to be highly unlikely for multi-dimensional
>>>parameters (it is also unlikely for one-dimensional problem). However,
>>>unless we understand the source of the problem, we cannot feel comfortable
>>>in saying with absolute certainty that this will not occur for n >
1.
>>> Therefore, I would suggest that either we fix the problem at its
source or
>>>we set abs.tol=0, since there is little harm in doing so.
>>>
>>>
>>>
>>>
>>Just for future reference: that's not the kind of answer that
leads to
>>anything getting done. So I'll leave it to the authors of nlminb.
>>
>>Duncan Murdoch
>>
>> Ravi.
>>
>>>____________________________________________________________________
>>>
>>>Ravi Varadhan, Ph.D.
>>>Assistant Professor,
>>>Division of Geriatric Medicine and Gerontology
>>>School of Medicine
>>>Johns Hopkins University
>>>
>>>Ph. (410) 502-2619
>>>email: rvarad...@jhmi.edu
>>>
>>>
>>>----- Original Message -----
>>>From: Duncan Murdoch <murdoch.dun...@gmail.com>
>>>Date: Saturday, July 10, 2010 7:32 am
>>>Subject: Re: [R] Not nice behaviour of nlminb (windows 32 bit, version
>>>2.11.1)
>>>To: Ravi Varadhan <rvarad...@jhmi.edu>
>>>Cc: Matthew Killeya <matthewkill...@googlemail.com>, Peter Ehlers
<
>>>ehl...@ucalgary.ca>, r-help@r-project.org, ba...@stat.wisc.edu
>>>
>>>
>>>
>>>
>>>
>>>>Ravi Varadhan wrote:
>>>> >Hi,
>>>> >
>>>> >The absolute function stopping criterion is not meant for any positive
>>>>objective function. It is meant for functions whose minimum is
0. Here is
>>>>what David Gay's documentation from PORT says:
>>>> >
>>>> >"6 - absolute function convergence: |f (x)| < V(AFCTOL) =
V(31). This
>>>>test is only of interest in
>>>> >problems where f (x) = 0 means a ‘‘perfect fit’’, such as nonlinear
>>>>least-squares problems."
>>>> > Okay, I've taken a more careful look at the docs, and they
do say
>>>>that the return code 6 does not necessarily indicate convergence:
"The
>>>>desirable return codes are 3, 4, 5, and sometimes 6". So we
shouldn't by
>>>>default terminate on it, we should allow users to choose that if
they want
>>>>faster convergence to perfect fits.
>>>> Would changing the default for abs.tol to zero be a reasonable solution?
>>>> Duncan Murdoch
>>>> >For example, let us try a positive objective function:
>>>> >
>>>> > >>nlminb( obj = function(x) x^2 + 1, start=1, lower=-Inf, upper=Inf,
>>>>control=list(trace=TRUE)) > 0: 2.0000000: 1.00000
>>>> > 1: 1.0000000: 0.00000
>>>> > 2: 1.0000000: 0.00000
>>>> >$par
>>>> >[1] 0
>>>> >
>>>> >$objective
>>>> >[1] 1
>>>> >
>>>> >$convergence
>>>> >[1] 0
>>>> >
>>>> >$message
>>>> >[1] "relative convergence (4)"
>>>> >
>>>> >$iterations
>>>> >[1] 2
>>>> >
>>>> >$evaluations
>>>> >function gradient 3 2 >
>>>> >Here the absolute function criterion does not kicks in. >
>>>> >Now let us try a function whose minimum value is 0.
>>>> >
>>>> > >>nlminb( obj = function(x) x^2, start=6, grad=function(x) 2*x,
>>>>lower=-Inf, upper=Inf, control=list(trace=TRUE) )
>>>> >> > 0: 36.000000: 6.00000
>>>> > 1: 4.0000000: 2.00000
>>>> > 2: 4.9303807e-32: 2.22045e-16
>>>> >$par
>>>> >[1] 2.220446e-16
>>>> >
>>>> >$objective
>>>> >[1] 4.930381e-32
>>>> >
>>>> >$convergence
>>>> >[1] 0
>>>> >
>>>> >$message
>>>> >[1] "absolute function convergence (6)"
>>>> >
>>>> >$iterations
>>>> >[1] 2
>>>> >
>>>> >$evaluations
>>>> >function gradient 4 3 >We see that convergence is
>>>>attained and that the stoppage is due to absolute function criterion.
>>>>
>>>>>Suppose, we now set abs.tol=0:
>>>>>
>>>> >
>>>> > >>nlminb( obj = function(x) x^2, start=6, grad=function(x) 2*x,
>>>>lower=-Inf, upper=Inf, control=list(trace=TRUE, abs.tol=0) )
>>>> >> > 0: 36.000000: 6.00000
>>>> > 1: 4.0000000: 2.00000
>>>> > 2: 4.9303807e-32: 2.22045e-16
>>>> > 3: 2.4308653e-63: -4.93038e-32
>>>> > 4: 2.9962729e-95: -5.47382e-48
>>>> > 5:1.4772766e-126: 1.21543e-63
>>>> > 6:1.8208840e-158: 1.34940e-79
>>>> > 7:8.9776511e-190: -2.99627e-95
>>>> > 8:1.1065809e-221: -3.32653e-111
>>>> > 9:5.4558652e-253: 7.38638e-127
>>>> > 10:6.7248731e-285: 8.20053e-143
>>>> > 11:3.3156184e-316: -1.82088e-158
>>>> > 12: 0.0000000: -2.02159e-174
>>>> > 13: 0.0000000: -2.02159e-174
>>>> >$par
>>>> >[1] -2.021587e-174
>>>> >
>>>> >$objective
>>>> >[1] 0
>>>> >
>>>> >$convergence
>>>> >[1] 0
>>>> >
>>>> >$message
>>>> >[1] "X-convergence (3)"
>>>> >
>>>> >$iterations
>>>> >[1] 13
>>>> >
>>>> >$evaluations
>>>> >function gradient 15 13 > Now, we see that it
takes a
>>>>while to stop, eventhough it is clear that convergence has been attained
>>>>after 2 iterations. This demonstrates the need for the absolute
function
>>>>criterion for obj functions whose minimum is exactly 0.
Although, there is
>>>>nothing wrong with setting abs.tol=0, except for some loss of computational
>>>>efficiency. >Now, let us get back to Matthew' example:
>>>> >
>>>> > >>nlminb( obj = function(x) x, start=1, lower=-2, upper=2,
>>>>control=list(trace=TRUE)) > 0: 1.0000000: 1.00000
>>>> > 1: 0.0000000: 0.00000
>>>> >$par
>>>> >[1] 0
>>>> >
>>>> >$objective
>>>> >[1] 0
>>>> >
>>>> >$convergence
>>>> >[1] 0
>>>> >
>>>> >$message
>>>> >[1] "absolute function convergence (6)"
>>>> >
>>>> >$iterations
>>>> >[1] 1
>>>> >
>>>> >$evaluations
>>>> >function gradient 2 2 > >>nlminb( obj =
function(x) x,
>>>>start=1, lower=-2, upper=2, control=list(trace=TRUE, abs.tol=0))
> 0:
>>>> 1.0000000: 1.00000
>>>> > 1: 0.0000000: 0.00000
>>>> > 2: -2.0000000: -2.00000
>>>> > 3: -2.0000000: -2.00000
>>>> >$par
>>>> >[1] -2
>>>> >
>>>> >$objective
>>>> >[1] -2
>>>> >
>>>> >$convergence
>>>> >[1] 0
>>>> >
>>>> >$message
>>>> >[1] "both X-convergence and relative convergence (5)"
>>>> >
>>>> >$iterations
>>>> >[1] 3
>>>> >
>>>> >$evaluations
>>>> >function gradient 3 3 >
>>>> >Thus it is evident that setting abs.tol=0 is a reasonable, general
>>>>solution for functions whose minimum value is non-zero, because
it protects
>>>>against premature termination at iteration `n' whenever |f(x_n)|
< abs.tol.
>>>> The only limitation being that of loss of efficiency in problems
where
>>>>f(x*) = 0. where x* is the local minimum.
>>>> >
>>>> >Ravi.
>>>> >____________________________________________________________________
>>>> >
>>>> >Ravi Varadhan, Ph.D.
>>>> >Assistant Professor,
>>>> >Division of Geriatric Medicine and Gerontology
>>>> >School of Medicine
>>>> >Johns Hopkins University
>>>> >
>>>> >Ph. (410) 502-2619
>>>> >email: rvarad...@jhmi.edu
>>>> >
>>>> >
>>>> >----- Original Message -----
>>>> >From: Duncan Murdoch <murdoch.dun...@gmail.com>
>>>> >Date: Friday, July 9, 2010 6:54 pm
>>>> >Subject: Re: [R] Not nice behaviour of nlminb (windows 32 bit,
version
>>>>2.11.1)
>>>> >To: Matthew Killeya <matthewkill...@googlemail.com>
>>>> >Cc: Peter Ehlers <ehl...@ucalgary.ca>, Ravi Varadhan <
>>>>rvarad...@jhmi.edu>, r-help@r-project.org, ba...@stat.wisc.edu
>>>> >
>>>> >
>>>> > >>On 09/07/2010 6:09 PM, Matthew Killeya wrote:
>>>> >> >Yes clearly a bug... there are numerous variations ...
problem seems
>>>>to be
>>>> >> >for a linear function whenever the first function valuation
is 1.
>>>> >> > Not at all. You can get the same problem on a quadratic
that
>>>>happens to have a zero at an inconvenient place, e.g.
>>>> >> nlminb( obj = function(x) x^2-25, start=6, lower=-Inf,
upper=Inf )
>>>> >> Ravi's workaround of setting the abs.tol to zero fixes this
example,
>>>>but I think it's pretty clear from the documentation that the
whole thing
>>>>was designed for positive objective functions, so I wouldn't
count on his
>>>>workaround solving all the problems.
>>>> >> Duncan Murdoch
>>>> >> >e.g. two more examples:
>>>> >> > nlminb( obj = function(x) x+1, start=0, lower=-Inf,
upper=Inf )
>>>> >> > nlminb( obj = function(x) x+2, start=-1, lower=-Inf,
upper=Inf )
>>>> >> >
>>>> >> >(I wasn't sure where best to report a bug, so emailed the
help list)
>>>> >> >
>>>> >> >On 9 July 2010 22:10, Peter Ehlers <ehl...@ucalgary.ca> wrote:
>>>> >> >
>>>> >> > >>Actually, it looks like any value other than 1.0
>>>> >> >>(and in (lower, upper)) for start will work.
>>>> >> >>
>>>> >> >> -Peter Ehlers
>>>> >> >>
>>>> >> >>
>>>> >> >>On 2010-07-09 14:45, Ravi Varadhan wrote:
>>>> >> >>
>>>> >> >> >>>Setting abs.tol = 0 works! This turns-off the absolute
>>>>function
>>>> >> >>>convergence
>>>> >> >>>criterion.
>>>> >> >>>
>>>> >> >>>
>>>> >> >>> nlminb( objective=function(x) x, start=1, lower=-2, upper=2,
>>>> >> >>> control=list(abs.tol=0))
>>>> >> >>>$par
>>>> >> >>>[1] -2
>>>> >> >>>
>>>> >> >>>$objective
>>>> >> >>>[1] -2
>>>> >> >>>
>>>> >> >>>$convergence
>>>> >> >>>[1] 0
>>>> >> >>>
>>>> >> >>>$message
>>>> >> >>>[1] "both X-convergence and relative convergence (5)"
>>>> >> >>>
>>>> >> >>>$iterations
>>>> >> >>>[1] 3
>>>> >> >>>
>>>> >> >>>$evaluations
>>>> >> >>>function gradient
>>>> >> >>> 3 3
>>>> >> >>>
>>>> >> >>>
>>>> >> >>>This is clearly a bug.
>>>> >> >>>
>>>> >> >>>
>>>> >> >>>Ravi.
>>>> >> >>>
>>>> >> >>>-----Original Message-----
>>>> >> >>>From: r-help-boun...@r-project.org [
>>>> >> >>>On
>>>> >> >>>Behalf Of Ravi Varadhan
>>>> >> >>>Sent: Friday, July 09, 2010 4:42 PM
>>>> >> >>>To: 'Duncan Murdoch'; 'Matthew Killeya'
>>>> >> >>>Cc: r-help@r-project.org; ba...@stat.wisc.edu
>>>> >> >>>Subject: Re: [R] Not nice behaviour of nlminb (windows 32
bit,
>>>>version
>>>> >> >>>2.11.1)
>>>> >> >>>
>>>> >> >>>Duncan, `nlminb' is not intended for non-negative
functions only.
>>>> There
>>>> >> >>>is
>>>> >> >>>indeed something strange happening in the algorithm!
>>>> >> >>>
>>>> >> >>>start<- 1.0 # converges to wrong minimum
>>>> >> >>>
>>>> >> >>>startp<- 1.0 + .Machine$double.eps # correct
>>>> >> >>>
>>>> >> >>>startm<- 1.0 - .Machine$double.eps # correct
>>>> >> >>>
>>>> >> >>> nlminb( objective=obj, start=start, lower=-2, upper=2)
>>>> >> >>> $par
>>>> >> >>>[1] 0
>>>> >> >>>
>>>> >> >>>$objective
>>>> >> >>>[1] 0
>>>> >> >>>
>>>> >> >>>$convergence
>>>> >> >>>[1] 0
>>>> >> >>>
>>>> >> >>>$message
>>>> >> >>>[1] "absolute function convergence (6)"
>>>> >> >>>
>>>> >> >>>$iterations
>>>> >> >>>[1] 1
>>>> >> >>>
>>>> >> >>>$evaluations
>>>> >> >>>function gradient
>>>> >> >>> 2 2
>>>> >> >>>
>>>> >> >>>
>>>> >> >>> >>>>nlminb( objective=obj, start=startp, lower=-2,
upper=2)
>>>> >> >>>>
>>>> >> >>>> >>>$par
>>>> >> >>>[1] -2
>>>> >> >>>
>>>> >> >>>$objective
>>>> >> >>>[1] -2
>>>> >> >>>
>>>> >> >>>$convergence
>>>> >> >>>[1] 0
>>>> >> >>>
>>>> >> >>>$message
>>>> >> >>>[1] "both X-convergence and relative convergence (5)"
>>>> >> >>>
>>>> >> >>>$iterations
>>>> >> >>>[1] 3
>>>> >> >>>
>>>> >> >>>$evaluations
>>>> >> >>>function gradient
>>>> >> >>> 3 3
>>>> >> >>>
>>>> >> >>>
>>>> >> >>> >>>>nlminb( objective=obj, start=startm, lower=-2,
upper=2)
>>>> >> >>>>
>>>> >> >>>> >>>$par
>>>> >> >>>[1] -2
>>>> >> >>>
>>>> >> >>>$objective
>>>> >> >>>[1] -2
>>>> >> >>>
>>>> >> >>>$convergence
>>>> >> >>>[1] 0
>>>> >> >>>
>>>> >> >>>$message
>>>> >> >>>[1] "both X-convergence and relative convergence (5)"
>>>> >> >>>
>>>> >> >>>$iterations
>>>> >> >>>[1] 3
>>>> >> >>>
>>>> >> >>>$evaluations
>>>> >> >>>function gradient
>>>> >> >>> 3 3
>>>> >> >>>
>>>> >> >>>
>>>> >> >>> From the convergence message the `absolute function convergence'
>>>>seems to
>>>> >> >>> be
>>>> >> >>>the culprit, although I do not understand why that stopping
>>>>criterion is
>>>> >> >>>becoming effective, when the algorithm is started at x=1,
but not
>>>>at any
>>>> >> >>>other values. The documentation in IPORT makes it clear
that this
>>>> >> >>>criterion
>>>> >> >>>is effective only for functions where f(x*) = 0, where x*
is a
>>>>local
>>>> >> >>>minimum. In this example, x=0 is not a local minimum for
f(x), so
>>>>that
>>>> >> >>>criterion should not apply.
>>>> >> >>>
>>>> >> >>>
>>>> >> >>>Ravi.
>>>> >> >>>
>>>> >> >>>
>>>> >> >>>-----Original Message-----
>>>> >> >>>From: r-help-boun...@r-project.org [
>>>> >> >>>On
>>>> >> >>>Behalf Of Duncan Murdoch
>>>> >> >>>Sent: Friday, July 09, 2010 3:45 PM
>>>> >> >>>To: Matthew Killeya
>>>> >> >>>Cc: r-help@r-project.org; ba...@stat.wisc.edu
>>>> >> >>>Subject: Re: [R] Not nice behaviour of nlminb (windows 32
bit,
>>>>version
>>>> >> >>>2.11.1)
>>>> >> >>>
>>>> >> >>>On 09/07/2010 10:37 AM, Matthew Killeya wrote:
>>>> >> >>>
>>>> >> >>> >>>> nlminb( obj = function(x) x, start=1, lower=-Inf,
>>>>upper=Inf )
>>>> >> >>>>
>>>> >> >>>>
>>>> >> >>>> >>>If you read the PORT documentation carefully,
you'll
>>>>see that their
>>>> >> >>>convergence criteria are aimed at minimizing positive functions.
>>>> (They
>>>> >> >>>never state this explicitly, as far as I can see.) So one
>>>>stopping
>>>> >> >>>criterion is that |f(x)|< abs.tol, and that's what it
found for
>>>>you. I
>>>> >> >>>don't know if there's a way to turn this off.
>>>> >> >>>
>>>> >> >>>Doug or Deepayan, do you know if nlminb can be made to
work on
>>>>functions
>>>> >> >>>that go negative?
>>>> >> >>>
>>>> >> >>>Duncan Murdoch
>>>> >> >>>
>>>> >> >>> $par
>>>> >> >>> >>>>[1] 0
>>>> >> >>>>
>>>> >> >>>>$objective
>>>> >> >>>>[1] 0
>>>> >> >>>>
>>>> >> >>>>$convergence
>>>> >> >>>>[1] 0
>>>> >> >>>>
>>>> >> >>>>$message
>>>> >> >>>>[1] "absolute function convergence (6)"
>>>> >> >>>>
>>>> >> >>>>$iterations
>>>> >> >>>>[1] 1
>>>> >> >>>>
>>>> >> >>>>$evaluations
>>>> >> >>>>function gradient
>>>> >> >>>> 2 2
>>>> >> >>>>
>>>> >> >>>> [[alternative HTML version deleted]]
>>>> >> >>>>
>>>> >> >>>>
>>>> >> >>>> >
>>>> >> >
>>>>
>>>>
>
>
