Yes, of course! The Nelder-Mead is derivative-free. Thanks, Ravi.
____________________________________________________________________ Ravi Varadhan, Ph.D. Assistant Professor, Division of Geriatric Medicine and Gerontology School of Medicine Johns Hopkins University Ph. (410) 502-2619 email: rvarad...@jhmi.edu ----- Original Message ----- From: William Dunlap <wdun...@tibco.com> Date: Tuesday, September 14, 2010 6:15 pm Subject: RE: RE: [R] Can I monitor the iterative/convergence process whileusing Optim or MaxLik? To: Ravi Varadhan <rvarad...@jhmi.edu> Cc: Greg Snow <greg.s...@imail.org>, r-help@r-project.org > I think the default method for optim never evaluates > the gradient function: VALUE and X are both NULL in > your example. Also o$counts shows NA for the gradient > entry. Use method="BGFS": > > > o <- optim(c(-1.2, 1), fr, tfr <- trackFn(grr), method="BFGS") > > with(environment(tfr), str(VALUE)) > num [1:43, 1:2] -215.6 136.42 56.44 10.27 9.35 ... > - attr(*, "dimnames")=List of 2 > ..$ : chr [1:43] "val" "val" "val" "val" ... > ..$ : NULL > > > Bill Dunlap > Spotfire, TIBCO Software > wdunlap tibco.com > > > -----Original Message----- > > From: Ravi Varadhan [ > > Sent: Tuesday, September 14, 2010 3:01 PM > > To: William Dunlap > > Cc: Greg Snow; r-help@r-project.org > > Subject: Re: RE: [R] Can I monitor the iterative/convergence > > process whileusing Optim or MaxLik? > > > > Bill, > > > > I am not able to get this to work for tracking inside the > > gradient function: > > > > trackFn <- function (fn) { > > # return function like fn that stashes its > > # inputs and outputs in local datasets. > > X <- NULL > > VALUE <- NULL > > force(fn) > > function(x) { > > X <<- rbind(X, x) # stash arguments > > val <- fn(x) > > VALUE <<- rbind(VALUE, val) # stash value > > val > > } > > } > > > > fr <- function(x) { ## Rosenbrock Banana function > > x1 <- x[1] > > x2 <- x[2] > > 100 * (x2 - x1 * x1)^2 + (1 - x1)^2 > > } > > > > grr <- function(x) { ## Gradient of 'fr' > > x1 <- x[1] > > x2 <- x[2] > > c(-400 * x1 * (x2 - x1 * x1) - 2 * (1 - x1), 200 * (x2 - x1 * x1)) > > > } > > > > o <- optim(c(-1.2, 1), fr, tfr <- trackFn(grr)) > > #with(environment(tfr), matplot(X, VALUE, type="l")) > > with(environment(tfr), plot(X, type="l")) > > > > What is wrong here? > > > > Ravi. > > > > ____________________________________________________________________ > > > > Ravi Varadhan, Ph.D. > > Assistant Professor, > > Division of Geriatric Medicine and Gerontology > > School of Medicine > > Johns Hopkins University > > > > Ph. (410) 502-2619 > > email: rvarad...@jhmi.edu > > > > > > ----- Original Message ----- > > From: William Dunlap <wdun...@tibco.com> > > Date: Tuesday, September 14, 2010 5:38 pm > > Subject: RE: [R] Can I monitor the iterative/convergence > > process while using Optim or MaxLik? > > To: Ravi Varadhan <rvarad...@jhmi.edu>, Greg Snow > > <greg.s...@imail.org> > > Cc: r-help@r-project.org > > > > > > > You can also save the objective (or gradient) function > > > arguments and values in datasets so you can later plot > > > or print them. E.g., the following lets you avoid hand > > > editing the objective function to do this. If you wanted > > > to track the gradient, replace th c(VALUE,val) with > > > rbind(VALUE,val). (Either would be ok with the objective.) > > > > > > trackFn <- function (fn) { > > > # return function like fn that stashes its > > > # inputs and outputs in local datasets. > > > X <- NULL > > > VALUE <- NULL > > > force(fn) > > > function(x) { > > > X <<- rbind(X, x) # stash arguments > > > val <- fn(x) > > > VALUE <<- c(VALUE, val) # stash value > > > val > > > } > > > } > > > > > > Typical usage would be > > > > > > > fr <- function(x) { ## Rosenbrock Banana function > > > + x1 <- x[1] > > > + x2 <- x[2] > > > + 100 * (x2 - x1 * x1)^2 + (1 - x1)^2 > > > + } > > > > o <- optim(c(-1.2, 1), tfr <- trackFn(fr)) > > > > with(environment(tfr), matplot(X, VALUE, type="l")) > > > > with(environment(tfr), plot(X, type="l")) > > > > > > Bill Dunlap > > > Spotfire, TIBCO Software > > > wdunlap tibco.com > > > > > > > -----Original Message----- > > > > From: r-help-boun...@r-project.org > > > > [ On Behalf Of Ravi Varadhan > > > > Sent: Tuesday, September 14, 2010 2:19 PM > > > > To: Greg Snow > > > > Cc: r-help@r-project.org > > > > Subject: Re: [R] Can I monitor the iterative/convergence > > > > process while using Optim or MaxLik? > > > > > > > > This is generally not the best way to do it for the following > > > > > reasons. > > > > > > > > The objective function may be evaluated multiple times within > > > > > a single iteration, for various reasons including (i) > > > > gradient evaluation when analytic gradient is not specified, > > > > and (ii) during line-search stretgy for steplength > > > > determination. Therefore, embedding a `cat' or `print' > > > > statement inside the objective function may generate a lot of > > > > > noise. You can run your suggested `optim' example to > > verify this! > > > > > > > > A better approach is to embed the `cat' statement inside the > > > > gradient function, in addition to using the `trace' and > > > > `REPORT' arguments, which are integers in `optim'. Here is > > > > the `optim' example that you suggested: > > > > > > > > fr <- function(x) { ## Rosenbrock Banana function > > > > x1 <- x[1] > > > > x2 <- x[2] > > > > 100 * (x2 - x1 * x1)^2 + (1 - x1)^2 > > > > } > > > > > > > > grr.print <- function(x) { ## Gradient of 'fr' > > > > cat("pars: ", x, '\n') > > > > flush.console() > > > > x1 <- x[1] > > > > x2 <- x[2] > > > > c(-400 * x1 * (x2 - x1 * x1) - 2 * (1 - x1), 200 * (x2 - > > x1 * x1)) > > > > > > > } > > > > > > > > > optim(c(-1.2,1), fr, grr.print, method = "BFGS", > > > > control=list(trace=1, REPORT=1)) > > > > > > > > This prints out both parameter values and the function value > > > > at each iteration. > > > > > > > > This approach cannot, however, be used in `optim' when > > > > analytic gradient is not available (or for derivative free > > > > methods, e.g. Nelder-Mead). > > > > > > > > > > > > Hope this helps, > > > > Ravi. > > > > > > > > > > ____________________________________________________________________ > > > > > > > > Ravi Varadhan, Ph.D. > > > > Assistant Professor, > > > > Division of Geriatric Medicine and Gerontology > > > > School of Medicine > > > > Johns Hopkins University > > > > > > > > Ph. (410) 502-2619 > > > > email: rvarad...@jhmi.edu > > > > > > > > > > > > ----- Original Message ----- > > > > From: Greg Snow <greg.s...@imail.org> > > > > Date: Tuesday, September 14, 2010 4:27 pm > > > > Subject: Re: [R] Can I monitor the iterative/convergence > > > > process while using Optim or MaxLik? > > > > To: Sally Luo <shali...@gmail.com>, "r-help@r-project.org" > > > > <r-help@r-project.org> > > > > > > > > > > > > > The simple way to do this is just to rewrite your > > function to be > > > > > > > > optimized to print out the current values that it was called > > > > with, > > > > > this way you will see where it is and what it is doing. > > > > Look at the > > > > > cat function for how to print the values, also look at the > > > > > flush.console function. > > > > > > > > > > For example you could take the 1st example on the optim > > > > help page and > > > > > insert the following 2 lines as the 1st 2 lines of the > > f function: > > > > > > > > > > cat(x, '\n') > > > > > flush.console() > > > > > > > > > > and then at each iteration you can see the values tried. > > > > > > > > > > -- > > > > > Gregory (Greg) L. Snow Ph.D. > > > > > Statistical Data Center > > > > > Intermountain Healthcare > > > > > greg.s...@imail.org > > > > > 801.408.8111 > > > > > > > > > > > > > > > > -----Original Message----- > > > > > > From: r-help-boun...@r-project.org [ > > > > > > project.org] On Behalf Of Sally Luo > > > > > > Sent: Tuesday, September 14, 2010 1:50 PM > > > > > > To: r-help@r-project.org > > > > > > Subject: [R] Can I monitor the iterative/convergence > > > > process while > > > > > > using Optim or MaxLik? > > > > > > > > > > > > Hi R-helpers, > > > > > > > > > > > > Is it possible that I have the estimates from each > > > > step/iteration shown > > > > > > on > > > > > > the computer screen in order to monitor the process > > > > while I am using > > > > > > Optim > > > > > > or MaxLik? > > > > > > > > > > > > Thanks for your help. > > > > > > > > > > > > Maomao > > > > > > > > > > > > [[alternative HTML version deleted]] > > > > > > > > > > > > ______________________________________________ > > > > > > R-help@r-project.org mailing list > > > > > > > > > > > > PLEASE do read the posting guide > > > > > > guide.html > > > > > > and provide commented, minimal, self-contained, > > > > reproducible code. > > > > > > > > > > ______________________________________________ > > > > > R-help@r-project.org mailing list > > > > > > > > > > PLEASE do read the posting guide > > > > > and provide commented, minimal, self-contained, > > reproducible code. > > > > > > > > ______________________________________________ > > > > R-help@r-project.org mailing list > > > > > > > > PLEASE do read the posting guide > > > > > > > > and provide commented, minimal, self-contained, > > reproducible code. > > > > > > ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.