Re: [R] Tracing gradient during optimization
Rob Steele suggested the same thing but I'm not sure I understand how to implement this exactly. Is there any documentation that you could suggest? This might be something that could be useful for the future. Thanks, Shimrit On Tue, Feb 24, 2009 at 5:09 PM, Greg Snow wrote: > It looks like you found a solution, but if you find yourself in this > situation again using optim, then one approach is to modify your function > that you are optimizing (or write a wrapper for it) to produce the tracing > information for you. > > -- > Gregory (Greg) L. Snow Ph.D. > Statistical Data Center > Intermountain Healthcare > greg.s...@imail.org > 801.408.8111 > > > > -Original Message- > > From: r-help-boun...@r-project.org [mailto:r-help-boun...@r- > > project.org] On Behalf Of Shimrit Abraham > > Sent: Tuesday, February 24, 2009 7:00 AM > > To: r-help@r-project.org > > Subject: [R] Tracing gradient during optimization > > > > Hi everyone, > > > > I am currently using the function optim() to maximize/minimize > > functions and > > I would like to see more output of the optimization procedure, in > > particular > > the numerical gradient of the parameter vector during each iteration. > > The documentation of optim() describes that the trace parameter should > > allow > > one to trace the progress of the optimization. > > I use the following command: > > > > optim(par = vPar, > > fn = calcLogLik, > > method = "BFGS", > > control = list(trace = TRUE, fnscale = -1, maxit = 2000)); > > > > which gives very little information: > > > > initial value 3.056998 > > final value 2.978351 > > converged > > > > Specifying trace >1, for instance trace = 20, does not result in more > > information. Is there a way to view more details of the progress > > perhaps by > > using another optimizer? > > > > Thanks, > > > > Shimrit Abraham > > > > [[alternative HTML version deleted]] > > > > __ > > R-help@r-project.org mailing list > > https://stat.ethz.ch/mailman/listinfo/r-help > > PLEASE do read the posting guide http://www.R-project.org/posting- > > guide.html > > and provide commented, minimal, self-contained, reproducible code. > [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Tracing gradient during optimization
Yes, initially, it didn't work and thanks to one of the examples in the help
file, I found out that I need to set maximize = T...but thanks for your
suggestion anyway.
I mainly work with state space models and I'm currently dealing with a case
where the estimation time is halved (!!!) by spg().
Shimrit
On Tue, Feb 24, 2009 at 3:25 PM, Ravi Varadhan wrote:
> Hi Shimrit,
>
> Make sure that you set maximize=TRUE in the control settings (since you
> have
> fnscale = -1 in your optim() call).
>
> A nice feature of spg() is that the entire code is in R, and can be readily
> seen by just typing the function name at the R prompt. On smaller problems
> (with only a few parameters), it is usually slower than optim() or
> nlminb(),
> since much of the computing is performed in C and/or Fortran. But the
> difference in speed is not that important in small problems, anyway.
> However, it is faster on large-scale problems.
>
> Best,
> Ravi.
>
>
>
>
> ---
>
> Ravi Varadhan, Ph.D.
>
> Assistant Professor, The Center on Aging and Health
>
> Division of Geriatric Medicine and Gerontology
>
> Johns Hopkins University
>
> Ph: (410) 502-2619
>
> Fax: (410) 614-9625
>
> Email: rvarad...@jhmi.edu
>
> Webpage: http://www.jhsph.edu/agingandhealth/People/Faculty/Varadhan.html
>
>
>
>
>
> ----
>
>
> -Original Message-
> From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org]
> On
> Behalf Of Shimrit Abraham
> Sent: Tuesday, February 24, 2009 10:15 AM
> To: Ravi Varadhan
> Cc: r-help@r-project.org
> Subject: Re: [R] Tracing gradient during optimization
>
> Hi Ravi,
>
> Thanks for your great suggestion, it does exactly what I need as it
> provides
> more insight into what is going on in the 'black box'. In addition, it's
> much faster than optim(). I will use this function in the future.
>
> Kind Regards,
>
> Shimrit
>
>
>
> On Tue, Feb 24, 2009 at 2:33 PM, Ravi Varadhan wrote:
>
> > Hi,
> >
> > If you look at the source code for optim() in the optim.c file, you
> > will see the following lines for "BFGS":
> >
> >if (trace && (iter % nREPORT == 0))
> >Rprintf("iter%4d value %f\n", iter, f);
> >
> > This means that "BFGS" does not output gradient values when you
> > "trace" the iterations. Let us look at the code for "L-BFGS-B":
> >
> >if(trace == 1 && (iter % nREPORT == 0)) {
> >Rprintf("iter %4d value %f\n", iter, f);
> >
> > So, it seems like even "L-BFGS-B" algorithm is also not going to be
> > useful to you.
> >
> >
> > You can use the spg() function in the "BB" package. Its usage is very
> > similar to that of optim(). When you specify trace=TRUE, it will give
> > you both function and (projected) gradient information. You can use
> > the "triter" parameter to control the frequency of output, i.e. settig
> > triter=1, will give you the fn and gr values at each iteration.
> >
> >library(BB)
> >?spg
> >
> > Hope this helps,
> > Ravi.
> >
> >
> >
> >
> > --
> > --
> > ---
> >
> > Ravi Varadhan, Ph.D.
> >
> > Assistant Professor, The Center on Aging and Health
> >
> > Division of Geriatric Medicine and Gerontology
> >
> > Johns Hopkins University
> >
> > Ph: (410) 502-2619
> >
> > Fax: (410) 614-9625
> >
> > Email: rvarad...@jhmi.edu
> >
> > Webpage:
> > http://www.jhsph.edu/agingandhealth/People/Faculty/Varadhan.html
> >
> >
> >
> >
> > --
> > --
> >
> >
> >
> > -Original Message-
> > From: r-help-boun...@r-project.org
> > [mailto:r-help-boun...@r-project.org]
> > On
> > Behalf Of Shimrit Abraham
> > Sent: Tuesday, February 24, 2009 9:00 AM
> > To: r-help@r-project.org
> > Subject: [R] Tracing gradient during optimization
> >
> > Hi everyone,
> >
> > I am currently using the function optim() to maximize/minimize
> > functions and I would like to see more output of the optimization
> > procedure, in parti
Re: [R] Tracing gradient during optimization
Hi Ravi,
Thanks for your great suggestion, it does exactly what I need as it provides
more insight into what is going on in the 'black box'. In addition, it's
much faster than optim(). I will use this function in the future.
Kind Regards,
Shimrit
On Tue, Feb 24, 2009 at 2:33 PM, Ravi Varadhan wrote:
> Hi,
>
> If you look at the source code for optim() in the optim.c file, you will
> see
> the following lines for "BFGS":
>
>if (trace && (iter % nREPORT == 0))
>Rprintf("iter%4d value %f\n", iter, f);
>
> This means that "BFGS" does not output gradient values when you "trace" the
> iterations. Let us look at the code for "L-BFGS-B":
>
>if(trace == 1 && (iter % nREPORT == 0)) {
>Rprintf("iter %4d value %f\n", iter, f);
>
> So, it seems like even "L-BFGS-B" algorithm is also not going to be useful
> to you.
>
>
> You can use the spg() function in the "BB" package. Its usage is very
> similar to that of optim(). When you specify trace=TRUE, it will give you
> both function and (projected) gradient information. You can use the
> "triter" parameter to control the frequency of output, i.e. settig
> triter=1,
> will give you the fn and gr values at each iteration.
>
>library(BB)
>?spg
>
> Hope this helps,
> Ravi.
>
>
>
>
>
> ---
>
> Ravi Varadhan, Ph.D.
>
> Assistant Professor, The Center on Aging and Health
>
> Division of Geriatric Medicine and Gerontology
>
> Johns Hopkins University
>
> Ph: (410) 502-2619
>
> Fax: (410) 614-9625
>
> Email: rvarad...@jhmi.edu
>
> Webpage: http://www.jhsph.edu/agingandhealth/People/Faculty/Varadhan.html
>
>
>
>
>
>
>
>
> -Original Message-
> From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org]
> On
> Behalf Of Shimrit Abraham
> Sent: Tuesday, February 24, 2009 9:00 AM
> To: r-help@r-project.org
> Subject: [R] Tracing gradient during optimization
>
> Hi everyone,
>
> I am currently using the function optim() to maximize/minimize functions
> and
> I would like to see more output of the optimization procedure, in
> particular
> the numerical gradient of the parameter vector during each iteration.
> The documentation of optim() describes that the trace parameter should
> allow
> one to trace the progress of the optimization.
> I use the following command:
>
> optim(par = vPar,
> fn = calcLogLik,
> method = "BFGS",
> control = list(trace = TRUE, fnscale = -1, maxit = 2000));
>
> which gives very little information:
>
> initial value 3.056998
> final value 2.978351
> converged
>
> Specifying trace >1, for instance trace = 20, does not result in more
> information. Is there a way to view more details of the progress perhaps by
> using another optimizer?
>
> Thanks,
>
> Shimrit Abraham
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
>
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[R] Tracing gradient during optimization
Hi everyone, I am currently using the function optim() to maximize/minimize functions and I would like to see more output of the optimization procedure, in particular the numerical gradient of the parameter vector during each iteration. The documentation of optim() describes that the trace parameter should allow one to trace the progress of the optimization. I use the following command: optim(par = vPar, fn = calcLogLik, method = "BFGS", control = list(trace = TRUE, fnscale = -1, maxit = 2000)); which gives very little information: initial value 3.056998 final value 2.978351 converged Specifying trace >1, for instance trace = 20, does not result in more information. Is there a way to view more details of the progress perhaps by using another optimizer? Thanks, Shimrit Abraham [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Efficient matrix computations
Thanks for your suggestions. I'll try to implement what you suggested. Perhaps the following information can help you to think of alternative ways to speed up computations: I am coding the Kalman Filter in R because I have certain requirements that are not provided by the packages regarding State Space models (dlm, sspir, dse, are there any more? ) . Since I don't see a way to avoid a for loop, I am trying to make the matrix computations as efficent as possible by using the properties that some of these matrices possess, hence my questions. The Kalman Filter requires sandwich matrix products, say ZPZ', where P is a covariance matrix, hence positive definite and symmetric (Z is not necessarily a square matrix). To compute the likelihood, I also need an efficient way to calculate the determinant of a covariance matrix, which is again positive definite and symmetric. Please let me know if this information gives you some new ideas on how to solve my problem. Thanks, S.A. On Tue, Feb 17, 2009 at 11:09 AM, Dimitris Rizopoulos < d.rizopou...@erasmusmc.nl> wrote: > sorry, in my previous e-mail it should be > > tcrossprod() > # and > prod(eigen(mat, symmetric = TRUE, only.values = TRUE)$values) > > > Best, > Dimitris > > Shimrit Abraham wrote: > >> Hi, >> >> I am looking for two ways to speed up my computations: >> >> 1. Is there a function that efficiently computes the 'sandwich product' of >> three matrices, say, ZPZ' >> 2. Is there a function that efficiently computes the determinant of a >> positive definite symmetric matrix? >> >> Thanks, >> >> S.A. >> >>[[alternative HTML version deleted]] >> >> __ >> R-help@r-project.org mailing list >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide >> http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. >> >> > -- > Dimitris Rizopoulos > Assistant Professor > Department of Biostatistics > Erasmus Medical Center > > Address: PO Box 2040, 3000 CA Rotterdam, the Netherlands > Tel: +31/(0)10/7043478 > Fax: +31/(0)10/7043014 > > [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Efficient matrix computations
Hi, I am looking for two ways to speed up my computations: 1. Is there a function that efficiently computes the 'sandwich product' of three matrices, say, ZPZ' 2. Is there a function that efficiently computes the determinant of a positive definite symmetric matrix? Thanks, S.A. [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] sandwich matrix multiplication and efficient determinant
Hi, I am looking for two ways to speed up my computations: 1. Is there a function that efficiently computes the 'sandwich product' of three matrices, say, ZPZ' 2. Is there a function that efficiently computes the determinant of a positive definite symmetric matrix? Thanks, S.A. [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Suppressing output in Garch estimation
HI, I'm calling the garch() function several times within a for-loop and I would like to suppress the output that the function generates. Setting garch.control(trace = F) does not seem to help. Any suggestions on how to solve this problem? Thanks, S.A. [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] plot multiple time series
Hi, I have a dataframe containing a date object in the first column and numeric data in two other columns, for a total of three columns. I would like to plot the 2 numeric data columns against the dates in one window. How do I do this? It is easy to do if only one data series is to be plotted against a set of dates, but two or more datasets seems to be harder. Note: I have daily data where weekends and holidays are left out, e.g. stock returns. Therefore, I prefer not to construct a new 'dates' vector. Thanks, S.A. [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
