On Fri, Nov 11, 2011 at 04:13:23AM +0900, Mathieu Blondel wrote: > Somewhat related to the coarse to fine search, there would also be the > idea of binary search. As Gael said, there may be many local optima > but if you're optimizing only one parameter, there may be times in > which a binary search can make sense.
This is the general setting of simplex optimizers, in particular the Nelder-Mead method, which is reasonnably robust to noise if the underlying function is bell-shaped. In 1D, it gives the so-called dichotomy method. I gave it a try a while ago, in our private codebase, implementing what I called a 'DichotomyCV' estimator. To make this somewhat work, I had to modify a bit the nelder-mead from scipy to have more control on the behavior with noise. I wanted to test it on real applications, and contribute it to the scikit in the long run. I gave up: on real applications it kept failing, and I realized that I didn't trust it, and would run a grid search if I didn't get good scores. The number one challenge was to deal with plateaus with additional noise. Simplex methods get stuck there. The code is available if people want it, but it will need significant rework to be usable on real problems, IMHO. Gaël ------------------------------------------------------------------------------ RSA(R) Conference 2012 Save $700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1 _______________________________________________ Scikit-learn-general mailing list Scikit-learn-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
