[R] Can I monitor the iterative/convergence process while using Optim or MaxLik?
Hi R-helpers, Is it possible that I have the estimates from each step/iteration shown on the computer screen in order to monitor the process while I am using Optim or MaxLik? Thanks for your help. Maomao [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Can I monitor the iterative/convergence process while using Optim or MaxLik?
The simple way to do this is just to rewrite your function to be optimized to print out the current values that it was called with, this way you will see where it is and what it is doing. Look at the cat function for how to print the values, also look at the flush.console function. For example you could take the 1st example on the optim help page and insert the following 2 lines as the 1st 2 lines of the f function: cat(x, '\n') flush.console() and then at each iteration you can see the values tried. -- Gregory (Greg) L. Snow Ph.D. Statistical Data Center Intermountain Healthcare greg.s...@imail.org 801.408.8111 -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r- project.org] On Behalf Of Sally Luo Sent: Tuesday, September 14, 2010 1:50 PM To: r-help@r-project.org Subject: [R] Can I monitor the iterative/convergence process while using Optim or MaxLik? Hi R-helpers, Is it possible that I have the estimates from each step/iteration shown on the computer screen in order to monitor the process while I am using Optim or MaxLik? Thanks for your help. Maomao [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting- guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Can I monitor the iterative/convergence process while using Optim or MaxLik?
On 14 September 2010 21:49, Sally Luo shali...@gmail.com wrote: Is it possible that I have the estimates from each step/iteration shown on the computer screen in order to monitor the process while I am using Optim or MaxLik? You can call maxLik() with argument print.level set to a value larger than 0. /Arne -- Arne Henningsen http://www.arne-henningsen.name [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Can I monitor the iterative/convergence process while using Optim or MaxLik?
This is generally not the best way to do it for the following reasons. The objective function may be evaluated multiple times within a single iteration, for various reasons including (i) gradient evaluation when analytic gradient is not specified, and (ii) during line-search stretgy for steplength determination. Therefore, embedding a `cat' or `print' statement inside the objective function may generate a lot of noise. You can run your suggested `optim' example to verify this! A better approach is to embed the `cat' statement inside the gradient function, in addition to using the `trace' and `REPORT' arguments, which are integers in `optim'. Here is the `optim' example that you suggested: fr - function(x) { ## Rosenbrock Banana function x1 - x[1] x2 - x[2] 100 * (x2 - x1 * x1)^2 + (1 - x1)^2 } grr.print - function(x) { ## Gradient of 'fr' cat(pars: , x, '\n') flush.console() x1 - x[1] x2 - x[2] c(-400 * x1 * (x2 - x1 * x1) - 2 * (1 - x1), 200 * (x2 - x1 * x1)) } optim(c(-1.2,1), fr, grr.print, method = BFGS, control=list(trace=1, REPORT=1)) This prints out both parameter values and the function value at each iteration. This approach cannot, however, be used in `optim' when analytic gradient is not available (or for derivative free methods, e.g. Nelder-Mead). Hope this helps, Ravi. Ravi Varadhan, Ph.D. Assistant Professor, Division of Geriatric Medicine and Gerontology School of Medicine Johns Hopkins University Ph. (410) 502-2619 email: rvarad...@jhmi.edu - Original Message - From: Greg Snow greg.s...@imail.org Date: Tuesday, September 14, 2010 4:27 pm Subject: Re: [R] Can I monitor the iterative/convergence process while using Optim or MaxLik? To: Sally Luo shali...@gmail.com, r-help@r-project.org r-help@r-project.org The simple way to do this is just to rewrite your function to be optimized to print out the current values that it was called with, this way you will see where it is and what it is doing. Look at the cat function for how to print the values, also look at the flush.console function. For example you could take the 1st example on the optim help page and insert the following 2 lines as the 1st 2 lines of the f function: cat(x, '\n') flush.console() and then at each iteration you can see the values tried. -- Gregory (Greg) L. Snow Ph.D. Statistical Data Center Intermountain Healthcare greg.s...@imail.org 801.408.8111 -Original Message- From: r-help-boun...@r-project.org [ project.org] On Behalf Of Sally Luo Sent: Tuesday, September 14, 2010 1:50 PM To: r-help@r-project.org Subject: [R] Can I monitor the iterative/convergence process while using Optim or MaxLik? Hi R-helpers, Is it possible that I have the estimates from each step/iteration shown on the computer screen in order to monitor the process while I am using Optim or MaxLik? Thanks for your help. Maomao [[alternative HTML version deleted]] __ R-help@r-project.org mailing list PLEASE do read the posting guide guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list PLEASE do read the posting guide and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Can I monitor the iterative/convergence process while using Optim or MaxLik?
You can also save the objective (or gradient) function arguments and values in datasets so you can later plot or print them. E.g., the following lets you avoid hand editing the objective function to do this. If you wanted to track the gradient, replace th c(VALUE,val) with rbind(VALUE,val). (Either would be ok with the objective.) trackFn - function (fn) { # return function like fn that stashes its # inputs and outputs in local datasets. X - NULL VALUE - NULL force(fn) function(x) { X - rbind(X, x) # stash arguments val - fn(x) VALUE - c(VALUE, val) # stash value val } } Typical usage would be fr - function(x) { ## Rosenbrock Banana function + x1 - x[1] + x2 - x[2] + 100 * (x2 - x1 * x1)^2 + (1 - x1)^2 + } o - optim(c(-1.2, 1), tfr - trackFn(fr)) with(environment(tfr), matplot(X, VALUE, type=l)) with(environment(tfr), plot(X, type=l)) Bill Dunlap Spotfire, TIBCO Software wdunlap tibco.com -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of Ravi Varadhan Sent: Tuesday, September 14, 2010 2:19 PM To: Greg Snow Cc: r-help@r-project.org Subject: Re: [R] Can I monitor the iterative/convergence process while using Optim or MaxLik? This is generally not the best way to do it for the following reasons. The objective function may be evaluated multiple times within a single iteration, for various reasons including (i) gradient evaluation when analytic gradient is not specified, and (ii) during line-search stretgy for steplength determination. Therefore, embedding a `cat' or `print' statement inside the objective function may generate a lot of noise. You can run your suggested `optim' example to verify this! A better approach is to embed the `cat' statement inside the gradient function, in addition to using the `trace' and `REPORT' arguments, which are integers in `optim'. Here is the `optim' example that you suggested: fr - function(x) { ## Rosenbrock Banana function x1 - x[1] x2 - x[2] 100 * (x2 - x1 * x1)^2 + (1 - x1)^2 } grr.print - function(x) { ## Gradient of 'fr' cat(pars: , x, '\n') flush.console() x1 - x[1] x2 - x[2] c(-400 * x1 * (x2 - x1 * x1) - 2 * (1 - x1), 200 * (x2 - x1 * x1)) } optim(c(-1.2,1), fr, grr.print, method = BFGS, control=list(trace=1, REPORT=1)) This prints out both parameter values and the function value at each iteration. This approach cannot, however, be used in `optim' when analytic gradient is not available (or for derivative free methods, e.g. Nelder-Mead). Hope this helps, Ravi. Ravi Varadhan, Ph.D. Assistant Professor, Division of Geriatric Medicine and Gerontology School of Medicine Johns Hopkins University Ph. (410) 502-2619 email: rvarad...@jhmi.edu - Original Message - From: Greg Snow greg.s...@imail.org Date: Tuesday, September 14, 2010 4:27 pm Subject: Re: [R] Can I monitor the iterative/convergence process while using Optim or MaxLik? To: Sally Luo shali...@gmail.com, r-help@r-project.org r-help@r-project.org The simple way to do this is just to rewrite your function to be optimized to print out the current values that it was called with, this way you will see where it is and what it is doing. Look at the cat function for how to print the values, also look at the flush.console function. For example you could take the 1st example on the optim help page and insert the following 2 lines as the 1st 2 lines of the f function: cat(x, '\n') flush.console() and then at each iteration you can see the values tried. -- Gregory (Greg) L. Snow Ph.D. Statistical Data Center Intermountain Healthcare greg.s...@imail.org 801.408.8111 -Original Message- From: r-help-boun...@r-project.org [ project.org] On Behalf Of Sally Luo Sent: Tuesday, September 14, 2010 1:50 PM To: r-help@r-project.org Subject: [R] Can I monitor the iterative/convergence process while using Optim or MaxLik? Hi R-helpers, Is it possible that I have the estimates from each step/iteration shown on the computer screen in order to monitor the process while I am using Optim or MaxLik? Thanks for your help. Maomao [[alternative HTML version deleted]] __ R-help@r-project.org mailing list PLEASE do read the posting guide guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list PLEASE do read the posting guide and provide commented, minimal, self-contained, reproducible code
Re: [R] Can I monitor the iterative/convergence process while using Optim or MaxLik?
Bill, I am not able to get this to work for tracking inside the gradient function: trackFn - function (fn) { # return function like fn that stashes its # inputs and outputs in local datasets. X - NULL VALUE - NULL force(fn) function(x) { X - rbind(X, x) # stash arguments val - fn(x) VALUE - rbind(VALUE, val) # stash value val } } fr - function(x) { ## Rosenbrock Banana function x1 - x[1] x2 - x[2] 100 * (x2 - x1 * x1)^2 + (1 - x1)^2 } grr - function(x) { ## Gradient of 'fr' x1 - x[1] x2 - x[2] c(-400 * x1 * (x2 - x1 * x1) - 2 * (1 - x1), 200 * (x2 - x1 * x1)) } o - optim(c(-1.2, 1), fr, tfr - trackFn(grr)) #with(environment(tfr), matplot(X, VALUE, type=l)) with(environment(tfr), plot(X, type=l)) What is wrong here? Ravi. Ravi Varadhan, Ph.D. Assistant Professor, Division of Geriatric Medicine and Gerontology School of Medicine Johns Hopkins University Ph. (410) 502-2619 email: rvarad...@jhmi.edu - Original Message - From: William Dunlap wdun...@tibco.com Date: Tuesday, September 14, 2010 5:38 pm Subject: RE: [R] Can I monitor the iterative/convergence process while using Optim or MaxLik? To: Ravi Varadhan rvarad...@jhmi.edu, Greg Snow greg.s...@imail.org Cc: r-help@r-project.org You can also save the objective (or gradient) function arguments and values in datasets so you can later plot or print them. E.g., the following lets you avoid hand editing the objective function to do this. If you wanted to track the gradient, replace th c(VALUE,val) with rbind(VALUE,val). (Either would be ok with the objective.) trackFn - function (fn) { # return function like fn that stashes its # inputs and outputs in local datasets. X - NULL VALUE - NULL force(fn) function(x) { X - rbind(X, x) # stash arguments val - fn(x) VALUE - c(VALUE, val) # stash value val } } Typical usage would be fr - function(x) { ## Rosenbrock Banana function + x1 - x[1] + x2 - x[2] + 100 * (x2 - x1 * x1)^2 + (1 - x1)^2 + } o - optim(c(-1.2, 1), tfr - trackFn(fr)) with(environment(tfr), matplot(X, VALUE, type=l)) with(environment(tfr), plot(X, type=l)) Bill Dunlap Spotfire, TIBCO Software wdunlap tibco.com -Original Message- From: r-help-boun...@r-project.org [ On Behalf Of Ravi Varadhan Sent: Tuesday, September 14, 2010 2:19 PM To: Greg Snow Cc: r-help@r-project.org Subject: Re: [R] Can I monitor the iterative/convergence process while using Optim or MaxLik? This is generally not the best way to do it for the following reasons. The objective function may be evaluated multiple times within a single iteration, for various reasons including (i) gradient evaluation when analytic gradient is not specified, and (ii) during line-search stretgy for steplength determination. Therefore, embedding a `cat' or `print' statement inside the objective function may generate a lot of noise. You can run your suggested `optim' example to verify this! A better approach is to embed the `cat' statement inside the gradient function, in addition to using the `trace' and `REPORT' arguments, which are integers in `optim'. Here is the `optim' example that you suggested: fr - function(x) { ## Rosenbrock Banana function x1 - x[1] x2 - x[2] 100 * (x2 - x1 * x1)^2 + (1 - x1)^2 } grr.print - function(x) { ## Gradient of 'fr' cat(pars: , x, '\n') flush.console() x1 - x[1] x2 - x[2] c(-400 * x1 * (x2 - x1 * x1) - 2 * (1 - x1), 200 * (x2 - x1 * x1)) } optim(c(-1.2,1), fr, grr.print, method = BFGS, control=list(trace=1, REPORT=1)) This prints out both parameter values and the function value at each iteration. This approach cannot, however, be used in `optim' when analytic gradient is not available (or for derivative free methods, e.g. Nelder-Mead). Hope this helps, Ravi. Ravi Varadhan, Ph.D. Assistant Professor, Division of Geriatric Medicine and Gerontology School of Medicine Johns Hopkins University Ph. (410) 502-2619 email: rvarad...@jhmi.edu - Original Message - From: Greg Snow greg.s...@imail.org Date: Tuesday, September 14, 2010 4:27 pm Subject: Re: [R] Can I monitor the iterative/convergence process while using Optim or MaxLik? To: Sally Luo shali...@gmail.com, r-help@r-project.org r-help@r-project.org The simple way to do this is just to rewrite your function to be optimized to print out the current values that it was called with, this way you will see where it is and what it is doing. Look at the cat function for how to print