Re: [R] Can I monitor the iterative/convergence process while using Optim or MaxLik?
Bill,
I am not able to get this to work for tracking inside the gradient function:
trackFn <- function (fn) {
# return function like fn that stashes its
# inputs and outputs in local datasets.
X <- NULL
VALUE <- NULL
force(fn)
function(x) {
X <<- rbind(X, x) # stash arguments
val <- fn(x)
VALUE <<- rbind(VALUE, val) # stash value
val
}
}
fr <- function(x) { ## Rosenbrock Banana function
x1 <- x[1]
x2 <- x[2]
100 * (x2 - x1 * x1)^2 + (1 - x1)^2
}
grr <- function(x) { ## Gradient of 'fr'
x1 <- x[1]
x2 <- x[2]
c(-400 * x1 * (x2 - x1 * x1) - 2 * (1 - x1), 200 * (x2 - x1 * x1))
}
o <- optim(c(-1.2, 1), fr, tfr <- trackFn(grr))
#with(environment(tfr), matplot(X, VALUE, type="l"))
with(environment(tfr), plot(X, type="l"))
What is wrong here?
Ravi.
Ravi Varadhan, Ph.D.
Assistant Professor,
Division of Geriatric Medicine and Gerontology
School of Medicine
Johns Hopkins University
Ph. (410) 502-2619
email: rvarad...@jhmi.edu
- Original Message -----
From: William Dunlap
Date: Tuesday, September 14, 2010 5:38 pm
Subject: RE: [R] Can I monitor the iterative/convergence process while using
Optim or MaxLik?
To: Ravi Varadhan , Greg Snow
Cc: r-help@r-project.org
> You can also save the objective (or gradient) function
> arguments and values in datasets so you can later plot
> or print them. E.g., the following lets you avoid hand
> editing the objective function to do this. If you wanted
> to track the gradient, replace th c(VALUE,val) with
> rbind(VALUE,val). (Either would be ok with the objective.)
>
> trackFn <- function (fn) {
> # return function like fn that stashes its
> # inputs and outputs in local datasets.
> X <- NULL
> VALUE <- NULL
> force(fn)
> function(x) {
> X <<- rbind(X, x) # stash arguments
> val <- fn(x)
> VALUE <<- c(VALUE, val) # stash value
> val
> }
> }
>
> Typical usage would be
>
> > fr <- function(x) { ## Rosenbrock Banana function
> + x1 <- x[1]
> + x2 <- x[2]
> + 100 * (x2 - x1 * x1)^2 + (1 - x1)^2
> + }
> > o <- optim(c(-1.2, 1), tfr <- trackFn(fr))
> > with(environment(tfr), matplot(X, VALUE, type="l"))
> > with(environment(tfr), plot(X, type="l"))
>
> Bill Dunlap
> Spotfire, TIBCO Software
> wdunlap tibco.com
>
> > -----Original Message-----
> > From: r-help-boun...@r-project.org
> > [ On Behalf Of Ravi Varadhan
> > Sent: Tuesday, September 14, 2010 2:19 PM
> > To: Greg Snow
> > Cc: r-help@r-project.org
> > Subject: Re: [R] Can I monitor the iterative/convergence
> > process while using Optim or MaxLik?
> >
> > This is generally not the best way to do it for the following
> > reasons.
> >
> > The objective function may be evaluated multiple times within
> > a single iteration, for various reasons including (i)
> > gradient evaluation when analytic gradient is not specified,
> > and (ii) during line-search stretgy for steplength
> > determination. Therefore, embedding a `cat' or `print'
> > statement inside the objective function may generate a lot of
> > noise. You can run your suggested `optim' example to verify this!
> >
> > A better approach is to embed the `cat' statement inside the
> > gradient function, in addition to using the `trace' and
> > `REPORT' arguments, which are integers in `optim'. Here is
> > the `optim' example that you suggested:
> >
> > fr <- function(x) { ## Rosenbrock Banana function
> > x1 <- x[1]
> > x2 <- x[2]
> > 100 * (x2 - x1 * x1)^2 + (1 - x1)^2
> > }
> >
> > grr.print <- function(x) { ## Gradient of 'fr'
> > cat("pars: ", x, '\n')
> > flush.console()
> > x1 <- x[1]
> > x2 <- x[2]
> > c(-400 * x1 * (x2 - x1 * x1) - 2 * (1 - x1), 200 * (x2 - x1 * x1))
>
> > }
> >
> > > optim(c(-1.2,1), fr, grr.print, method = "BFGS",
> > control=list(trace=1, REPORT=1))
> >
> > This prints out both parameter values and the function value
> > at each iteration.
> >
> > This approach cannot, however, be used in `optim' when
> > analytic gradient is not available (or for derivative free
> > methods, e.g. Nelder-Mead).
> >
> >
> > Hope this helps,
> > Ravi.
> >
> >
Re: [R] Can I monitor the iterative/convergence process while using Optim or MaxLik?
You can also save the objective (or gradient) function
arguments and values in datasets so you can later plot
or print them. E.g., the following lets you avoid hand
editing the objective function to do this. If you wanted
to track the gradient, replace th c(VALUE,val) with
rbind(VALUE,val). (Either would be ok with the objective.)
trackFn <- function (fn) {
# return function like fn that stashes its
# inputs and outputs in local datasets.
X <- NULL
VALUE <- NULL
force(fn)
function(x) {
X <<- rbind(X, x) # stash arguments
val <- fn(x)
VALUE <<- c(VALUE, val) # stash value
val
}
}
Typical usage would be
> fr <- function(x) { ## Rosenbrock Banana function
+ x1 <- x[1]
+ x2 <- x[2]
+ 100 * (x2 - x1 * x1)^2 + (1 - x1)^2
+ }
> o <- optim(c(-1.2, 1), tfr <- trackFn(fr))
> with(environment(tfr), matplot(X, VALUE, type="l"))
> with(environment(tfr), plot(X, type="l"))
Bill Dunlap
Spotfire, TIBCO Software
wdunlap tibco.com
> -Original Message-
> From: r-help-boun...@r-project.org
> [mailto:r-help-boun...@r-project.org] On Behalf Of Ravi Varadhan
> Sent: Tuesday, September 14, 2010 2:19 PM
> To: Greg Snow
> Cc: r-help@r-project.org
> Subject: Re: [R] Can I monitor the iterative/convergence
> process while using Optim or MaxLik?
>
> This is generally not the best way to do it for the following
> reasons.
>
> The objective function may be evaluated multiple times within
> a single iteration, for various reasons including (i)
> gradient evaluation when analytic gradient is not specified,
> and (ii) during line-search stretgy for steplength
> determination. Therefore, embedding a `cat' or `print'
> statement inside the objective function may generate a lot of
> noise. You can run your suggested `optim' example to verify this!
>
> A better approach is to embed the `cat' statement inside the
> gradient function, in addition to using the `trace' and
> `REPORT' arguments, which are integers in `optim'. Here is
> the `optim' example that you suggested:
>
> fr <- function(x) { ## Rosenbrock Banana function
> x1 <- x[1]
> x2 <- x[2]
> 100 * (x2 - x1 * x1)^2 + (1 - x1)^2
> }
>
> grr.print <- function(x) { ## Gradient of 'fr'
> cat("pars: ", x, '\n')
> flush.console()
> x1 <- x[1]
> x2 <- x[2]
> c(-400 * x1 * (x2 - x1 * x1) - 2 * (1 - x1), 200 * (x2 - x1 * x1))
> }
>
> > optim(c(-1.2,1), fr, grr.print, method = "BFGS",
> control=list(trace=1, REPORT=1))
>
> This prints out both parameter values and the function value
> at each iteration.
>
> This approach cannot, however, be used in `optim' when
> analytic gradient is not available (or for derivative free
> methods, e.g. Nelder-Mead).
>
>
> Hope this helps,
> Ravi.
>
> ____________________________
>
> Ravi Varadhan, Ph.D.
> Assistant Professor,
> Division of Geriatric Medicine and Gerontology
> School of Medicine
> Johns Hopkins University
>
> Ph. (410) 502-2619
> email: rvarad...@jhmi.edu
>
>
> - Original Message -
> From: Greg Snow
> Date: Tuesday, September 14, 2010 4:27 pm
> Subject: Re: [R] Can I monitor the iterative/convergence
> process while using Optim or MaxLik?
> To: Sally Luo , "r-help@r-project.org"
>
>
>
> > The simple way to do this is just to rewrite your function to be
> > optimized to print out the current values that it was called with,
> > this way you will see where it is and what it is doing.
> Look at the
> > cat function for how to print the values, also look at the
> > flush.console function.
> >
> > For example you could take the 1st example on the optim
> help page and
> > insert the following 2 lines as the 1st 2 lines of the f function:
> >
> > cat(x, '\n')
> > flush.console()
> >
> > and then at each iteration you can see the values tried.
> >
> > --
> > Gregory (Greg) L. Snow Ph.D.
> > Statistical Data Center
> > Intermountain Healthcare
> > greg.s...@imail.org
> > 801.408.8111
> >
> >
> > > -Original Message-
> > > From: r-help-boun...@r-project.org [
> > > project.org] On Behalf Of Sally Luo
> > > Sent: Tuesday, September 14, 2010 1:50 PM
> > > To: r-help@r-project.org
> > > Subject: [R] Can I monitor the iterative/convergence
> process while
> > &
Re: [R] Can I monitor the iterative/convergence process while using Optim or MaxLik?
This is generally not the best way to do it for the following reasons.
The objective function may be evaluated multiple times within a single
iteration, for various reasons including (i) gradient evaluation when analytic
gradient is not specified, and (ii) during line-search stretgy for steplength
determination. Therefore, embedding a `cat' or `print' statement inside the
objective function may generate a lot of noise. You can run your suggested
`optim' example to verify this!
A better approach is to embed the `cat' statement inside the gradient function,
in addition to using the `trace' and `REPORT' arguments, which are integers in
`optim'. Here is the `optim' example that you suggested:
fr <- function(x) { ## Rosenbrock Banana function
x1 <- x[1]
x2 <- x[2]
100 * (x2 - x1 * x1)^2 + (1 - x1)^2
}
grr.print <- function(x) { ## Gradient of 'fr'
cat("pars: ", x, '\n')
flush.console()
x1 <- x[1]
x2 <- x[2]
c(-400 * x1 * (x2 - x1 * x1) - 2 * (1 - x1), 200 * (x2 - x1 * x1))
}
> optim(c(-1.2,1), fr, grr.print, method = "BFGS", control=list(trace=1,
> REPORT=1))
This prints out both parameter values and the function value at each iteration.
This approach cannot, however, be used in `optim' when analytic gradient is not
available (or for derivative free methods, e.g. Nelder-Mead).
Hope this helps,
Ravi.
Ravi Varadhan, Ph.D.
Assistant Professor,
Division of Geriatric Medicine and Gerontology
School of Medicine
Johns Hopkins University
Ph. (410) 502-2619
email: rvarad...@jhmi.edu
----- Original Message -
From: Greg Snow
Date: Tuesday, September 14, 2010 4:27 pm
Subject: Re: [R] Can I monitor the iterative/convergence process while using
Optim or MaxLik?
To: Sally Luo , "r-help@r-project.org"
> The simple way to do this is just to rewrite your function to be
> optimized to print out the current values that it was called with,
> this way you will see where it is and what it is doing. Look at the
> cat function for how to print the values, also look at the
> flush.console function.
>
> For example you could take the 1st example on the optim help page and
> insert the following 2 lines as the 1st 2 lines of the f function:
>
> cat(x, '\n')
> flush.console()
>
> and then at each iteration you can see the values tried.
>
> --
> Gregory (Greg) L. Snow Ph.D.
> Statistical Data Center
> Intermountain Healthcare
> greg.s...@imail.org
> 801.408.8111
>
>
> > -Original Message-
> > From: r-help-boun...@r-project.org [
> > project.org] On Behalf Of Sally Luo
> > Sent: Tuesday, September 14, 2010 1:50 PM
> > To: r-help@r-project.org
> > Subject: [R] Can I monitor the iterative/convergence process while
> > using Optim or MaxLik?
> >
> > Hi R-helpers,
> >
> > Is it possible that I have the estimates from each step/iteration shown
> > on
> > the computer screen in order to monitor the process while I am using
> > Optim
> > or MaxLik?
> >
> > Thanks for your help.
> >
> > Maomao
> >
> >[[alternative HTML version deleted]]
> >
> > __
> > R-help@r-project.org mailing list
> >
> > PLEASE do read the posting guide
> > guide.html
> > and provide commented, minimal, self-contained, reproducible code.
>
> __
> R-help@r-project.org mailing list
>
> PLEASE do read the posting guide
> and provide commented, minimal, self-contained, reproducible code.
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Can I monitor the iterative/convergence process while using Optim or MaxLik?
On 14 September 2010 21:49, Sally Luo wrote: > Is it possible that I have the estimates from each step/iteration shown on > the computer screen in order to monitor the process while I am using Optim > or MaxLik? > You can call maxLik() with argument "print.level" set to a value larger than 0. /Arne -- Arne Henningsen http://www.arne-henningsen.name [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Can I monitor the iterative/convergence process while using Optim or MaxLik?
The simple way to do this is just to rewrite your function to be optimized to print out the current values that it was called with, this way you will see where it is and what it is doing. Look at the cat function for how to print the values, also look at the flush.console function. For example you could take the 1st example on the optim help page and insert the following 2 lines as the 1st 2 lines of the f function: cat(x, '\n') flush.console() and then at each iteration you can see the values tried. -- Gregory (Greg) L. Snow Ph.D. Statistical Data Center Intermountain Healthcare greg.s...@imail.org 801.408.8111 > -Original Message- > From: r-help-boun...@r-project.org [mailto:r-help-boun...@r- > project.org] On Behalf Of Sally Luo > Sent: Tuesday, September 14, 2010 1:50 PM > To: r-help@r-project.org > Subject: [R] Can I monitor the iterative/convergence process while > using Optim or MaxLik? > > Hi R-helpers, > > Is it possible that I have the estimates from each step/iteration shown > on > the computer screen in order to monitor the process while I am using > Optim > or MaxLik? > > Thanks for your help. > > Maomao > > [[alternative HTML version deleted]] > > __ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting- > guide.html > and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
