Re: [R] optim seems to be finding a local minimum
One more thing: trying to defend R's honor, I've run optimx instead of optim (after dividing the IV by its max - same as for optim). I did not use L-BFGS-B with lower bounds anymore. Instead, I've used Nelder-Mead (no bounds). First, it was faster: for a loop across 10 different IVs BFGS took 6.14 sec and Nelder-Mead took just 3.9 sec. Second, the solution was better - Nelder-Mead fits were ALL better than L-BFGS-B fits and ALL better than Excel solver's solutions. Of course, those were small improvements, but still, it's nice! Dimitri On Mon, Nov 14, 2011 at 5:26 PM, Dimitri Liakhovitski dimitri.liakhovit...@gmail.com wrote: Just to provide some closure: I ended up dividing the IV by its max so that the input vector (IV) is now between zero and one. I still used optim: myopt - optim(fn=myfunc, par=c(1,1), method=L-BFGS-B, lower=c(0,0)) I was able to get great fit, in 3 cases out of 10 I've beaten Excel Solver, but in 7 cases I lost to Excel - but again, by really tiny margins (generally less than 1% of Excel's fit value). Thank you everybody! Dimitri On Fri, Nov 11, 2011 at 10:28 AM, John C Nash nas...@uottawa.ca wrote: Some tips: 1) Excel did not, as far as I can determine, find a solution. No point seems to satisfy the KKT conditions (there is a function kktc in optfntools on R-forge project optimizer. It is called by optimx). 2) Scaling of the input vector is a good idea given the seeming wide range of values. That is, assuming this can be done. If the function depends on the relative values in the input vector rather than magnitude, this may explain the trouble with your function. That is, if the function depends on the relative change in the input vector and not its scale, then optimizers will have a lot of trouble if the scale factor for this vector is implicitly one of the optimization parameters. 3) If you can get the gradient function you will almost certainly be able to do better, especially in finding whether you have a minimum i.e., null gradient, positive definite Hessian. When you have gradient function, kktc uses Jacobian(gradient) to get the Hessian, avoiding one level of digit cancellation. JN On 11/11/2011 10:20 AM, Dimitri Liakhovitski wrote: Thank you very much to everyone who replied! As I mentioned - I am not a mathematician, so sorry for stupid comments/questions. I intuitively understand what you mean by scaling. While the solution space for the first parameter (.alpha) is relatively compact (probably between 0 and 2), the second one (.beta) is all over the place - because it is a function of IV (input vector). And that's, probably, my main challenge - that I am trying to write a routine for different possible IVs that I might be facing (they may be in hundreds, in thousands, in millions). Should I be rescaling the IV somehow (e.g., by dividing it by its max) - or should I do something with the parameter .beta inside my function? So far, I've written a loop over many different starting points for both parameters. Then, I take the betas around the best solution so far, split it into smaller steps for beta (as starting points) and optimize again for those starting points. What disappoints me is that even when I found a decent solution (the minimized value of 336) it was still worse than the Solver solution! And I am trying to prove to everyone here that we should do R, not Excel :-) Thanks again for your help, guys! Dimitri On Fri, Nov 11, 2011 at 9:10 AM, John C Nash nas...@uottawa.ca wrote: I won't requote all the other msgs, but the latest (and possibly a bit glitchy) version of optimx on R-forge 1) finds that some methods wander into domains where the user function fails try() (new optimx runs try() around all function calls). This includes L-BFGS-B 2) reports that the scaling is such that you really might not expect to get a good solution then 3) Actually gets a better result than the xlf-myfunc(c(0.888452533990788,94812732.0897449)) xlf [1] 334.607 with Kelley's variant of Nelder Mead (from dfoptim package), with myoptx method par fvalues fns grs itns conv KKT1 4 LBFGSB NA, NA 8.988466e+307 NA NULL NULL NA 2 Rvmmin 0.1, 200186870.6 25593.83 20 1 NULL 0 FALSE 3 bobyqa 6.987875e-01, 2.001869e+08 1933.229 44 NA NULL 0 FALSE 1 nmkb 8.897590e-01, 9.470163e+07 334.1901 204 NA NULL 0 FALSE KKT2 xtimes meths 4 NA 0.01 LBFGSB 2 FALSE 0.11 Rvmmin 3 FALSE 0.24 bobyqa 1 FALSE 1.08 nmkb But do note the terrible scaling. Hardly surprising that this function does not work. I'll have to delve deeper to see what the scaling setup should be because of the nature of the function setup involving some of the data. (optimx includes parscale on all methods). However, original poster DID include code, so it was easy to do a quick check. Good for
Re: [R] optim seems to be finding a local minimum
Just to provide some closure: I ended up dividing the IV by its max so that the input vector (IV) is now between zero and one. I still used optim: myopt - optim(fn=myfunc, par=c(1,1), method=L-BFGS-B, lower=c(0,0)) I was able to get great fit, in 3 cases out of 10 I've beaten Excel Solver, but in 7 cases I lost to Excel - but again, by really tiny margins (generally less than 1% of Excel's fit value). Thank you everybody! Dimitri On Fri, Nov 11, 2011 at 10:28 AM, John C Nash nas...@uottawa.ca wrote: Some tips: 1) Excel did not, as far as I can determine, find a solution. No point seems to satisfy the KKT conditions (there is a function kktc in optfntools on R-forge project optimizer. It is called by optimx). 2) Scaling of the input vector is a good idea given the seeming wide range of values. That is, assuming this can be done. If the function depends on the relative values in the input vector rather than magnitude, this may explain the trouble with your function. That is, if the function depends on the relative change in the input vector and not its scale, then optimizers will have a lot of trouble if the scale factor for this vector is implicitly one of the optimization parameters. 3) If you can get the gradient function you will almost certainly be able to do better, especially in finding whether you have a minimum i.e., null gradient, positive definite Hessian. When you have gradient function, kktc uses Jacobian(gradient) to get the Hessian, avoiding one level of digit cancellation. JN On 11/11/2011 10:20 AM, Dimitri Liakhovitski wrote: Thank you very much to everyone who replied! As I mentioned - I am not a mathematician, so sorry for stupid comments/questions. I intuitively understand what you mean by scaling. While the solution space for the first parameter (.alpha) is relatively compact (probably between 0 and 2), the second one (.beta) is all over the place - because it is a function of IV (input vector). And that's, probably, my main challenge - that I am trying to write a routine for different possible IVs that I might be facing (they may be in hundreds, in thousands, in millions). Should I be rescaling the IV somehow (e.g., by dividing it by its max) - or should I do something with the parameter .beta inside my function? So far, I've written a loop over many different starting points for both parameters. Then, I take the betas around the best solution so far, split it into smaller steps for beta (as starting points) and optimize again for those starting points. What disappoints me is that even when I found a decent solution (the minimized value of 336) it was still worse than the Solver solution! And I am trying to prove to everyone here that we should do R, not Excel :-) Thanks again for your help, guys! Dimitri On Fri, Nov 11, 2011 at 9:10 AM, John C Nash nas...@uottawa.ca wrote: I won't requote all the other msgs, but the latest (and possibly a bit glitchy) version of optimx on R-forge 1) finds that some methods wander into domains where the user function fails try() (new optimx runs try() around all function calls). This includes L-BFGS-B 2) reports that the scaling is such that you really might not expect to get a good solution then 3) Actually gets a better result than the xlf-myfunc(c(0.888452533990788,94812732.0897449)) xlf [1] 334.607 with Kelley's variant of Nelder Mead (from dfoptim package), with myoptx method par fvalues fns grs itns conv KKT1 4 LBFGSB NA, NA 8.988466e+307 NA NULL NULL NA 2 Rvmmin 0.1, 200186870.6 25593.83 20 1 NULL 0 FALSE 3 bobyqa 6.987875e-01, 2.001869e+08 1933.229 44 NA NULL 0 FALSE 1 nmkb 8.897590e-01, 9.470163e+07 334.1901 204 NA NULL 0 FALSE KKT2 xtimes meths 4 NA 0.01 LBFGSB 2 FALSE 0.11 Rvmmin 3 FALSE 0.24 bobyqa 1 FALSE 1.08 nmkb But do note the terrible scaling. Hardly surprising that this function does not work. I'll have to delve deeper to see what the scaling setup should be because of the nature of the function setup involving some of the data. (optimx includes parscale on all methods). However, original poster DID include code, so it was easy to do a quick check. Good for him. JN ## Comparing this solution to Excel Solver solution: myfunc(c(0.888452533990788,94812732.0897449)) -- Dimitri Liakhovitski marketfusionanalytics.com __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Dimitri Liakhovitski marketfusionanalytics.com __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the
Re: [R] optim seems to be finding a local minimum
Actually, Interval Analysis can be used to find _all_ optima (including the global optimum) within a starting box. It's not particularly well-known in statistical circles. See this (for example): http://bib.tiera.ru/ShiZ/math/other/Global%20Optimization%20Using%20Interval%20Analysis%20-%20E.%20Hansen%20(CRC).pdfhttp://bib.tiera.ru/ShiZ/math/other/Global%20Optimization%20Using%20Interval%20Analysis%20-%20E.%20Hansen%20%28CRC%29.pdf Kevin On Thu, Nov 10, 2011 at 3:20 PM, Rolf Turner rolf.tur...@xtra.co.nz wrote: On 11/11/11 08:55, Dimitri Liakhovitski wrote: Bert, that's exactly where I started. I found optim in the first paragraph under General Purpose Continuous Solvers and used bounded BFGS for a constrained optimization for a situation with more than 1 parameters. Again, not being an engineer / mathematician - would greatly appreciate any pointers. I'm no expert on numerical optimisation (though I've done quite a bit of it in my own ham-fisted way :-) ). Anyway, it seems to me that the only strategy that anyone can use in seeking a global optimum when there are multiple local optima is to use a wide variety of starting values. Possibly placed on a (relatively coarse) grid. It can be a tough problem. Good luck. cheers, Rolf Turner __** R-help@r-project.org mailing list https://stat.ethz.ch/mailman/**listinfo/r-helphttps://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/** posting-guide.html http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Kevin Wright [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] optim seems to be finding a local minimum
Ben Bolker bbolker at gmail.com writes: Simulated annealing and other stochastic global optimization methods are also possible solutions, although they may or may not work better than the many-starting-points solution -- it depends on the problem, and pretty much everything has to be tuned. Tabu search http://en.wikipedia.org/wiki/Tabu_search is another possibility, although I don't know much about it ... It is known that the Excel Solver has much improved during recent years. Still there are slightly better points such as myfunc(c(0.889764228112319, 94701144.5712312)) # 334.18844 restricting the domain to [0, 1] x [0, 10^9] for an evolutionary approach, for instance DEoptim::DEoptim(). Finding a global optimum in 2 dimensions is not so difficult. Here the scale of the second variable could pose a problem as small local minima might be overlooked easily. Hans Werner __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] optim seems to be finding a local minimum
Hans W Borchers hwborchers at googlemail.com writes: Ben Bolker bbolker at gmail.com writes: Simulated annealing and other stochastic global optimization methods are also possible solutions, although they may or may not work better than the many-starting-points solution -- it depends on the problem, and pretty much everything has to be tuned. Tabu search http://en.wikipedia.org/wiki/Tabu_search is another possibility, although I don't know much about it ... It is known that the Excel Solver has much improved during recent years. Still there are slightly better points such as myfunc(c(0.889764228112319, 94701144.5712312)) # 334.18844 restricting the domain to [0, 1] x [0, 10^9] for an evolutionary approach, for instance DEoptim::DEoptim(). Finding a global optimum in 2 dimensions is not so difficult. Here the scale of the second variable could pose a problem as small local minima might be overlooked easily. Have taken a (quick) second look at the problem, I agree that scaling and centering are more likely to be useful solutions than stochastic global optimization stuff. Even using 'parscale' (see the optim help page) may clear up the problem. Ben Bolker __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] optim seems to be finding a local minimum
I won't requote all the other msgs, but the latest (and possibly a bit glitchy) version of optimx on R-forge 1) finds that some methods wander into domains where the user function fails try() (new optimx runs try() around all function calls). This includes L-BFGS-B 2) reports that the scaling is such that you really might not expect to get a good solution then 3) Actually gets a better result than the xlf-myfunc(c(0.888452533990788,94812732.0897449)) xlf [1] 334.607 with Kelley's variant of Nelder Mead (from dfoptim package), with myoptx methodpar fvalues fns grs itns conv KKT1 4 LBFGSB NA, NA 8.988466e+307 NA NULL NULL NA 2 Rvmmin 0.1, 200186870.6 25593.83 201 NULL0 FALSE 3 bobyqa 6.987875e-01, 2.001869e+08 1933.229 44 NA NULL0 FALSE 1 nmkb 8.897590e-01, 9.470163e+07 334.1901 204 NA NULL0 FALSE KKT2 xtimes meths 4NA 0.01 LBFGSB 2 FALSE 0.11 Rvmmin 3 FALSE 0.24 bobyqa 1 FALSE 1.08 nmkb But do note the terrible scaling. Hardly surprising that this function does not work. I'll have to delve deeper to see what the scaling setup should be because of the nature of the function setup involving some of the data. (optimx includes parscale on all methods). However, original poster DID include code, so it was easy to do a quick check. Good for him. JN ## Comparing this solution to Excel Solver solution: myfunc(c(0.888452533990788,94812732.0897449)) -- Dimitri Liakhovitski marketfusionanalytics.com __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] optim seems to be finding a local minimum
Hi Dimitri, Your problem has little to do with local versus global optimum. You can convince yourself that the solution you got is not even a local optimum by checking the gradient at the solution. The main issue is that your objective function is not differentiable everywhere. So, you have 2 options: either you use a smooth objective function (e.g. squared residuals) or you use an optimization algorithm than can handle non-smooth objective function. Here I show that your problem is well solved by the `nmkb' function (a bound-constraints version of Nelder-Mead simplex method) from the dfoptim package. library(dfoptim) myopt2 - nmkb(fn=myfunc, par=c(0.1,max(IV)), lower=0) myopt2 $par [1] 8.897590e-01 9.470163e+07 $value [1] 334.1901 $feval [1] 204 $restarts [1] 0 $convergence [1] 0 $message [1] Successful convergence Then, there is also the issue of properly scaling your function, because it is poorly scaled. Look how different the 2 parameters are - they are 7 orders of magnitude apart. You are really asking for trouble here. Hope this is helpful, Ravi. --- Ravi Varadhan, Ph.D. Assistant Professor, Division of Geriatric Medicine and Gerontology School of Medicine Johns Hopkins University Ph. (410) 502-2619 email: rvarad...@jhmi.edumailto:rvarad...@jhmi.edu [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] optim seems to be finding a local minimum
Thank you very much to everyone who replied! As I mentioned - I am not a mathematician, so sorry for stupid comments/questions. I intuitively understand what you mean by scaling. While the solution space for the first parameter (.alpha) is relatively compact (probably between 0 and 2), the second one (.beta) is all over the place - because it is a function of IV (input vector). And that's, probably, my main challenge - that I am trying to write a routine for different possible IVs that I might be facing (they may be in hundreds, in thousands, in millions). Should I be rescaling the IV somehow (e.g., by dividing it by its max) - or should I do something with the parameter .beta inside my function? So far, I've written a loop over many different starting points for both parameters. Then, I take the betas around the best solution so far, split it into smaller steps for beta (as starting points) and optimize again for those starting points. What disappoints me is that even when I found a decent solution (the minimized value of 336) it was still worse than the Solver solution! And I am trying to prove to everyone here that we should do R, not Excel :-) Thanks again for your help, guys! Dimitri On Fri, Nov 11, 2011 at 9:10 AM, John C Nash nas...@uottawa.ca wrote: I won't requote all the other msgs, but the latest (and possibly a bit glitchy) version of optimx on R-forge 1) finds that some methods wander into domains where the user function fails try() (new optimx runs try() around all function calls). This includes L-BFGS-B 2) reports that the scaling is such that you really might not expect to get a good solution then 3) Actually gets a better result than the xlf-myfunc(c(0.888452533990788,94812732.0897449)) xlf [1] 334.607 with Kelley's variant of Nelder Mead (from dfoptim package), with myoptx method par fvalues fns grs itns conv KKT1 4 LBFGSB NA, NA 8.988466e+307 NA NULL NULL NA 2 Rvmmin 0.1, 200186870.6 25593.83 20 1 NULL 0 FALSE 3 bobyqa 6.987875e-01, 2.001869e+08 1933.229 44 NA NULL 0 FALSE 1 nmkb 8.897590e-01, 9.470163e+07 334.1901 204 NA NULL 0 FALSE KKT2 xtimes meths 4 NA 0.01 LBFGSB 2 FALSE 0.11 Rvmmin 3 FALSE 0.24 bobyqa 1 FALSE 1.08 nmkb But do note the terrible scaling. Hardly surprising that this function does not work. I'll have to delve deeper to see what the scaling setup should be because of the nature of the function setup involving some of the data. (optimx includes parscale on all methods). However, original poster DID include code, so it was easy to do a quick check. Good for him. JN ## Comparing this solution to Excel Solver solution: myfunc(c(0.888452533990788,94812732.0897449)) -- Dimitri Liakhovitski marketfusionanalytics.com __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Dimitri Liakhovitski marketfusionanalytics.com __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] optim seems to be finding a local minimum
Some tips: 1) Excel did not, as far as I can determine, find a solution. No point seems to satisfy the KKT conditions (there is a function kktc in optfntools on R-forge project optimizer. It is called by optimx). 2) Scaling of the input vector is a good idea given the seeming wide range of values. That is, assuming this can be done. If the function depends on the relative values in the input vector rather than magnitude, this may explain the trouble with your function. That is, if the function depends on the relative change in the input vector and not its scale, then optimizers will have a lot of trouble if the scale factor for this vector is implicitly one of the optimization parameters. 3) If you can get the gradient function you will almost certainly be able to do better, especially in finding whether you have a minimum i.e., null gradient, positive definite Hessian. When you have gradient function, kktc uses Jacobian(gradient) to get the Hessian, avoiding one level of digit cancellation. JN On 11/11/2011 10:20 AM, Dimitri Liakhovitski wrote: Thank you very much to everyone who replied! As I mentioned - I am not a mathematician, so sorry for stupid comments/questions. I intuitively understand what you mean by scaling. While the solution space for the first parameter (.alpha) is relatively compact (probably between 0 and 2), the second one (.beta) is all over the place - because it is a function of IV (input vector). And that's, probably, my main challenge - that I am trying to write a routine for different possible IVs that I might be facing (they may be in hundreds, in thousands, in millions). Should I be rescaling the IV somehow (e.g., by dividing it by its max) - or should I do something with the parameter .beta inside my function? So far, I've written a loop over many different starting points for both parameters. Then, I take the betas around the best solution so far, split it into smaller steps for beta (as starting points) and optimize again for those starting points. What disappoints me is that even when I found a decent solution (the minimized value of 336) it was still worse than the Solver solution! And I am trying to prove to everyone here that we should do R, not Excel :-) Thanks again for your help, guys! Dimitri On Fri, Nov 11, 2011 at 9:10 AM, John C Nash nas...@uottawa.ca wrote: I won't requote all the other msgs, but the latest (and possibly a bit glitchy) version of optimx on R-forge 1) finds that some methods wander into domains where the user function fails try() (new optimx runs try() around all function calls). This includes L-BFGS-B 2) reports that the scaling is such that you really might not expect to get a good solution then 3) Actually gets a better result than the xlf-myfunc(c(0.888452533990788,94812732.0897449)) xlf [1] 334.607 with Kelley's variant of Nelder Mead (from dfoptim package), with myoptx methodpar fvalues fns grs itns conv KKT1 4 LBFGSB NA, NA 8.988466e+307 NA NULL NULL NA 2 Rvmmin 0.1, 200186870.6 25593.83 201 NULL0 FALSE 3 bobyqa 6.987875e-01, 2.001869e+08 1933.229 44 NA NULL0 FALSE 1 nmkb 8.897590e-01, 9.470163e+07 334.1901 204 NA NULL0 FALSE KKT2 xtimes meths 4NA 0.01 LBFGSB 2 FALSE 0.11 Rvmmin 3 FALSE 0.24 bobyqa 1 FALSE 1.08 nmkb But do note the terrible scaling. Hardly surprising that this function does not work. I'll have to delve deeper to see what the scaling setup should be because of the nature of the function setup involving some of the data. (optimx includes parscale on all methods). However, original poster DID include code, so it was easy to do a quick check. Good for him. JN ## Comparing this solution to Excel Solver solution: myfunc(c(0.888452533990788,94812732.0897449)) -- Dimitri Liakhovitski marketfusionanalytics.com __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] optim seems to be finding a local minimum
Hello!
I am trying to create an R optimization routine for a task that's
currently being done using Excel (lots of tables, formulas, and
Solver).
However, otpim seems to be finding a local minimum.
Example data, functions, and comparison with the solution found in
Excel are below.
I am not experienced in optimizations so thanks a lot for your advice!
Dimitri
### 2 Inputs:
IV-data.frame(IV=c(0,6672895.687,13345791.37,20018687.06,26691582.75,33364478.44,40037374.12,46710269.81,53383165.5,60056061.18,66728956.87,73401852.56,80074748.24,86747643.93,93420539.62,100093435.3,106766331,113439226.7,120112122.4,126785018.1,133457913.7,140130809.4,146803705.1,153476600.8,160149496.5,166822392.2,173495287.9,180168183.5,186841079.2,193513974.9,200186870.6))
DV-data.frame(DV=c(0,439.8839775,829.7360945,1176.968757,1487.732038,1767.147276,2019.49499,2248.366401,2456.78592,2647.310413,2822.109854,2983.033036,3131.661246,3269.352233,3397.276321,3516.446162,3627.741311,3731.928591,3829.679009,3921.581866,4008.156537,4089.862363,4167.106955,4240.253215,4309.625263,4375.513474,4438.178766,4497.856259,4554.75841,4609.077705,4660.988983))
## Function transformIV transforms a data frame column IV using
parameters .alpha .beta:
## It returns a data frame column IV_transf:
transformIV = function(.alpha,.beta) {
IV_transf - as.data.frame(1 - (1/exp((IV/.beta)^.alpha)))
return(IV_transf)
}
### Function mysum calculates the sum of absolute residuals after a
regression with a single predictor:
mysum- function(myIV,myDV){
regr-lm(myDV[[1]] ~ 0 + myIV[[1]])
mysum-sum(abs(regr$resid))
return(mysum)
}
### Function to be optimized;
### param is a vector of 2 values (.alpha and .beta)
myfunc - function(param){
myalpha-param[1]
mybeta-param[2]
IVtransf-transformIV(myalpha, mybeta)
sumofdevs-mysum(myIV=IVtransf,myDV=DV)
return(sumofdevs)
}
# Optimizing using optim:
myopt - optim(fn=myfunc, par=c(0.1,max(IV)), method=L-BFGS-B, lower=0)
(myopt)
myfunc(myopt$par)
## Comparing this solution to Excel Solver solution:
myfunc(c(0.888452533990788,94812732.0897449))
--
Dimitri Liakhovitski
marketfusionanalytics.com
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] optim seems to be finding a local minimum
Just to add:
I also experimented with the starting parameters (par) under optim,
especially with the second one. I tried 1, 10, 100, 1000, etc.
When I tried 100,000,000 then I got a somewhat better solution (but
still not as good as in Excel). However, under message it said:
ERROR: ABNORMAL_TERMINATION_IN_LNSRCH
Dimitri
On Thu, Nov 10, 2011 at 1:50 PM, Dimitri Liakhovitski
dimitri.liakhovit...@gmail.com wrote:
Hello!
I am trying to create an R optimization routine for a task that's
currently being done using Excel (lots of tables, formulas, and
Solver).
However, otpim seems to be finding a local minimum.
Example data, functions, and comparison with the solution found in
Excel are below.
I am not experienced in optimizations so thanks a lot for your advice!
Dimitri
### 2 Inputs:
IV-data.frame(IV=c(0,6672895.687,13345791.37,20018687.06,26691582.75,33364478.44,40037374.12,46710269.81,53383165.5,60056061.18,66728956.87,73401852.56,80074748.24,86747643.93,93420539.62,100093435.3,106766331,113439226.7,120112122.4,126785018.1,133457913.7,140130809.4,146803705.1,153476600.8,160149496.5,166822392.2,173495287.9,180168183.5,186841079.2,193513974.9,200186870.6))
DV-data.frame(DV=c(0,439.8839775,829.7360945,1176.968757,1487.732038,1767.147276,2019.49499,2248.366401,2456.78592,2647.310413,2822.109854,2983.033036,3131.661246,3269.352233,3397.276321,3516.446162,3627.741311,3731.928591,3829.679009,3921.581866,4008.156537,4089.862363,4167.106955,4240.253215,4309.625263,4375.513474,4438.178766,4497.856259,4554.75841,4609.077705,4660.988983))
## Function transformIV transforms a data frame column IV using
parameters .alpha .beta:
## It returns a data frame column IV_transf:
transformIV = function(.alpha,.beta) {
IV_transf - as.data.frame(1 - (1/exp((IV/.beta)^.alpha)))
return(IV_transf)
}
### Function mysum calculates the sum of absolute residuals after a
regression with a single predictor:
mysum- function(myIV,myDV){
regr-lm(myDV[[1]] ~ 0 + myIV[[1]])
mysum-sum(abs(regr$resid))
return(mysum)
}
### Function to be optimized;
### param is a vector of 2 values (.alpha and .beta)
myfunc - function(param){
myalpha-param[1]
mybeta-param[2]
IVtransf-transformIV(myalpha, mybeta)
sumofdevs-mysum(myIV=IVtransf,myDV=DV)
return(sumofdevs)
}
# Optimizing using optim:
myopt - optim(fn=myfunc, par=c(0.1,max(IV)), method=L-BFGS-B, lower=0)
(myopt)
myfunc(myopt$par)
## Comparing this solution to Excel Solver solution:
myfunc(c(0.888452533990788,94812732.0897449))
--
Dimitri Liakhovitski
marketfusionanalytics.com
--
Dimitri Liakhovitski
marketfusionanalytics.com
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] optim seems to be finding a local minimum
Refer to the CRAN Optimization task view, please. That is a much more
appropriate place to begin than posting a query here.
All numerical optimizers only produce local optima.
-- Bert
On Thu, Nov 10, 2011 at 11:24 AM, Dimitri Liakhovitski
dimitri.liakhovit...@gmail.com wrote:
Just to add:
I also experimented with the starting parameters (par) under optim,
especially with the second one. I tried 1, 10, 100, 1000, etc.
When I tried 100,000,000 then I got a somewhat better solution (but
still not as good as in Excel). However, under message it said:
ERROR: ABNORMAL_TERMINATION_IN_LNSRCH
Dimitri
On Thu, Nov 10, 2011 at 1:50 PM, Dimitri Liakhovitski
dimitri.liakhovit...@gmail.com wrote:
Hello!
I am trying to create an R optimization routine for a task that's
currently being done using Excel (lots of tables, formulas, and
Solver).
However, otpim seems to be finding a local minimum.
Example data, functions, and comparison with the solution found in
Excel are below.
I am not experienced in optimizations so thanks a lot for your advice!
Dimitri
### 2 Inputs:
IV-data.frame(IV=c(0,6672895.687,13345791.37,20018687.06,26691582.75,33364478.44,40037374.12,46710269.81,53383165.5,
60056061.18,66728956.87,73401852.56,80074748.24,86747643.93
,93420539.62,100093435.3,106766331,113439226.7,120112122.4,126785018.1,133457913.7,140130809.4,146803705.1,153476600.8,160149496.5,166822392.2,173495287.9,180168183.5,186841079.2,193513974.9,200186870.6))
DV-data.frame(DV=c(0,439.8839775,829.7360945
,1176.968757,1487.732038,1767.147276,2019.49499,2248.366401
,2456.78592,2647.310413,2822.109854,2983.033036,3131.661246,3269.352233,
3397.276321,3516.446162
,3627.741311,3731.928591,3829.679009,3921.581866,4008.156537,4089.862363,
4167.106955,4240.253215,4309.625263,4375.513474,4438.178766
,4497.856259,4554.75841,4609.077705,4660.988983))
## Function transformIV transforms a data frame column IV using
parameters .alpha .beta:
## It returns a data frame column IV_transf:
transformIV = function(.alpha,.beta) {
IV_transf - as.data.frame(1 - (1/exp((IV/.beta)^.alpha)))
return(IV_transf)
}
### Function mysum calculates the sum of absolute residuals after a
regression with a single predictor:
mysum- function(myIV,myDV){
regr-lm(myDV[[1]] ~ 0 + myIV[[1]])
mysum-sum(abs(regr$resid))
return(mysum)
}
### Function to be optimized;
### param is a vector of 2 values (.alpha and .beta)
myfunc - function(param){
myalpha-param[1]
mybeta-param[2]
IVtransf-transformIV(myalpha, mybeta)
sumofdevs-mysum(myIV=IVtransf,myDV=DV)
return(sumofdevs)
}
# Optimizing using optim:
myopt - optim(fn=myfunc, par=c(0.1,max(IV)), method=L-BFGS-B, lower=0)
(myopt)
myfunc(myopt$par)
## Comparing this solution to Excel Solver solution:
myfunc(c(0.888452533990788,94812732.0897449))
--
Dimitri Liakhovitski
marketfusionanalytics.com
--
Dimitri Liakhovitski
marketfusionanalytics.com
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
--
Bert Gunter
Genentech Nonclinical Biostatistics
Internal Contact Info:
Phone: 467-7374
Website:
http://pharmadevelopment.roche.com/index/pdb/pdb-functional-groups/pdb-biostatistics/pdb-ncb-home.htm
[[alternative HTML version deleted]]
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] optim seems to be finding a local minimum
Bert,
that's exactly where I started. I found optim in the first paragraph
under General Purpose Continuous Solvers and used bounded BFGS for a
constrained optimization for a situation with more than 1 parameters.
Again, not being an engineer / mathematician - would greatly
appreciate any pointers.
Thank you, everyone!
Dimitri
On Thu, Nov 10, 2011 at 2:39 PM, Bert Gunter gunter.ber...@gene.com wrote:
Refer to the CRAN Optimization task view, please. That is a much more
appropriate place to begin than posting a query here.
All numerical optimizers only produce local optima.
-- Bert
On Thu, Nov 10, 2011 at 11:24 AM, Dimitri Liakhovitski
dimitri.liakhovit...@gmail.com wrote:
Just to add:
I also experimented with the starting parameters (par) under optim,
especially with the second one. I tried 1, 10, 100, 1000, etc.
When I tried 100,000,000 then I got a somewhat better solution (but
still not as good as in Excel). However, under message it said:
ERROR: ABNORMAL_TERMINATION_IN_LNSRCH
Dimitri
On Thu, Nov 10, 2011 at 1:50 PM, Dimitri Liakhovitski
dimitri.liakhovit...@gmail.com wrote:
Hello!
I am trying to create an R optimization routine for a task that's
currently being done using Excel (lots of tables, formulas, and
Solver).
However, otpim seems to be finding a local minimum.
Example data, functions, and comparison with the solution found in
Excel are below.
I am not experienced in optimizations so thanks a lot for your advice!
Dimitri
### 2 Inputs:
IV-data.frame(IV=c(0,6672895.687,13345791.37,20018687.06,26691582.75,33364478.44,40037374.12,46710269.81,53383165.5,60056061.18,66728956.87,73401852.56,80074748.24,86747643.93,93420539.62,100093435.3,106766331,113439226.7,120112122.4,126785018.1,133457913.7,140130809.4,146803705.1,153476600.8,160149496.5,166822392.2,173495287.9,180168183.5,186841079.2,193513974.9,200186870.6))
DV-data.frame(DV=c(0,439.8839775,829.7360945,1176.968757,1487.732038,1767.147276,2019.49499,2248.366401,2456.78592,2647.310413,2822.109854,2983.033036,3131.661246,3269.352233,3397.276321,3516.446162,3627.741311,3731.928591,3829.679009,3921.581866,4008.156537,4089.862363,4167.106955,4240.253215,4309.625263,4375.513474,4438.178766,4497.856259,4554.75841,4609.077705,4660.988983))
## Function transformIV transforms a data frame column IV using
parameters .alpha .beta:
## It returns a data frame column IV_transf:
transformIV = function(.alpha,.beta) {
IV_transf - as.data.frame(1 - (1/exp((IV/.beta)^.alpha)))
return(IV_transf)
}
### Function mysum calculates the sum of absolute residuals after a
regression with a single predictor:
mysum- function(myIV,myDV){
regr-lm(myDV[[1]] ~ 0 + myIV[[1]])
mysum-sum(abs(regr$resid))
return(mysum)
}
### Function to be optimized;
### param is a vector of 2 values (.alpha and .beta)
myfunc - function(param){
myalpha-param[1]
mybeta-param[2]
IVtransf-transformIV(myalpha, mybeta)
sumofdevs-mysum(myIV=IVtransf,myDV=DV)
return(sumofdevs)
}
# Optimizing using optim:
myopt - optim(fn=myfunc, par=c(0.1,max(IV)), method=L-BFGS-B,
lower=0)
(myopt)
myfunc(myopt$par)
## Comparing this solution to Excel Solver solution:
myfunc(c(0.888452533990788,94812732.0897449))
--
Dimitri Liakhovitski
marketfusionanalytics.com
--
Dimitri Liakhovitski
marketfusionanalytics.com
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
--
Bert Gunter
Genentech Nonclinical Biostatistics
Internal Contact Info:
Phone: 467-7374
Website:
http://pharmadevelopment.roche.com/index/pdb/pdb-functional-groups/pdb-biostatistics/pdb-ncb-home.htm
--
Dimitri Liakhovitski
marketfusionanalytics.com
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] optim seems to be finding a local minimum
On 11/11/11 08:55, Dimitri Liakhovitski wrote: Bert, that's exactly where I started. I found optim in the first paragraph under General Purpose Continuous Solvers and used bounded BFGS for a constrained optimization for a situation with more than 1 parameters. Again, not being an engineer / mathematician - would greatly appreciate any pointers. I'm no expert on numerical optimisation (though I've done quite a bit of it in my own ham-fisted way :-) ). Anyway, it seems to me that the only strategy that anyone can use in seeking a global optimum when there are multiple local optima is to use a wide variety of starting values. Possibly placed on a (relatively coarse) grid. It can be a tough problem. Good luck. cheers, Rolf Turner __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] optim seems to be finding a local minimum
Rolf Turner rolf.turner at xtra.co.nz writes: On 11/11/11 08:55, Dimitri Liakhovitski wrote: Bert, that's exactly where I started. I found optim in the first paragraph under General Purpose Continuous Solvers and used bounded BFGS for a constrained optimization for a situation with more than 1 parameters. Again, not being an engineer / mathematician - would greatly appreciate any pointers. I'm no expert on numerical optimisation (though I've done quite a bit of it in my own ham-fisted way ). Anyway, it seems to me that the only strategy that anyone can use in seeking a global optimum when there are multiple local optima is to use a wide variety of starting values. Possibly placed on a (relatively coarse) grid. Simulated annealing and other stochastic global optimization methods are also possible solutions, although they may or may not work better than the many-starting-points solution -- it depends on the problem, and pretty much everything has to be tuned. Tabu search http://en.wikipedia.org/wiki/Tabu_search is another possibility, although I don't know much about it ... __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
