That work is under submission at an academic conference and will be made available if/when the paper is published.
In terms of algorithms for hyperparameter tuning, we consider Grid Search, Random Search, a couple of older derivative-free optimization methods, and a few newer methods - TPE (aka HyperOpt from James Bergstra), SMAC (from Frank Hutter's group), and Spearmint (Jasper Snoek's method) - the short answer is that in our hands Random Search works surprisingly well for the low-dimensional problems we looked at, but TPE and SMAC perform slightly better. I've got a private branch with TPE (as well as random and grid search) integrated with MLI, but the code is research quality right now and not extremely general. We're actively working on bringing these things up to snuff for a proper open source release. On Fri, Apr 4, 2014 at 11:28 AM, Yi Zou <yi.zou.li...@gmail.com> wrote: > Hi, Evan, > > Just noticed this thread, do you mind sharing more details regarding > algorithms targetted at hyperparameter tuning/model selection? or a link > to dev git repo for that work. > > thanks, > yi > > > On Wed, Apr 2, 2014 at 6:03 PM, Evan R. Sparks <evan.spa...@gmail.com>wrote: > >> Targeting 0.9.0 should work out of the box (just a change to the >> build.sbt) - I'll push some changes I've been sitting on to the public repo >> in the next couple of days. >> >> >> On Wed, Apr 2, 2014 at 4:05 AM, Krakna H <shankark+...@gmail.com> wrote: >> >>> Thanks for the update Evan! In terms of using MLI, I see that the Github >>> code is linked to Spark 0.8; will it not work with 0.9 (which is what I >>> have set up) or higher versions? >>> >>> >>> On Wed, Apr 2, 2014 at 1:44 AM, Evan R. Sparks [via Apache Spark User >>> List] <[hidden email]<http://user/SendEmail.jtp?type=node&node=3632&i=0> >>> > wrote: >>> >>>> Hi there, >>>> >>>> MLlib is the first component of MLbase - MLI and the higher levels of >>>> the stack are still being developed. Look for updates in terms of our >>>> progress on the hyperparameter tuning/model selection problem in the next >>>> month or so! >>>> >>>> - Evan >>>> >>>> >>>> On Tue, Apr 1, 2014 at 8:05 PM, Krakna H <[hidden >>>> email]<http://user/SendEmail.jtp?type=node&node=3615&i=0> >>>> > wrote: >>>> >>>>> Hi Nan, >>>>> >>>>> I was actually referring to MLI/MLBase (http://www.mlbase.org); is >>>>> this being actively developed? >>>>> >>>>> I'm familiar with mllib and have been looking at its documentation. >>>>> >>>>> Thanks! >>>>> >>>>> >>>>> On Tue, Apr 1, 2014 at 10:44 PM, Nan Zhu [via Apache Spark User List] >>>>> <[hidden email] <http://user/SendEmail.jtp?type=node&node=3612&i=0>>wrote: >>>>> >>>>>> mllib has been part of Spark distribution (under mllib directory), >>>>>> also check http://spark.apache.org/docs/latest/mllib-guide.html >>>>>> >>>>>> and for JIRA, because of the recent migration to apache JIRA, I think >>>>>> all mllib-related issues should be under the Spark umbrella, >>>>>> https://issues.apache.org/jira/browse/SPARK/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel >>>>>> >>>>>> -- >>>>>> Nan Zhu >>>>>> >>>>>> On Tuesday, April 1, 2014 at 10:38 PM, Krakna H wrote: >>>>>> >>>>>> What is the current development status of MLI/MLBase? I see that the >>>>>> github repo is lying dormant (https://github.com/amplab/MLI) and >>>>>> JIRA has had no activity in the last 30 days ( >>>>>> https://spark-project.atlassian.net/browse/MLI/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel). >>>>>> Is the plan to add a lot of this into mllib itself without needing a >>>>>> separate API? >>>>>> >>>>>> Thanks! >>>>>> >>>>>> ------------------------------ >>>>>> View this message in context: Status of >>>>>> MLI?<http://apache-spark-user-list.1001560.n3.nabble.com/Status-of-MLI-tp3610.html> >>>>>> Sent from the Apache Spark User List mailing list >>>>>> archive<http://apache-spark-user-list.1001560.n3.nabble.com/>at >>>>>> Nabble.com. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> ------------------------------ >>>>>> If you reply to this email, your message will be added to the >>>>>> discussion below: >>>>>> >>>>>> http://apache-spark-user-list.1001560.n3.nabble.com/Status-of-MLI-tp3610p3611.html >>>>>> To start a new topic under Apache Spark User List, email [hidden >>>>>> email] <http://user/SendEmail.jtp?type=node&node=3612&i=1> >>>>>> To unsubscribe from Apache Spark User List, click here. >>>>>> NAML<http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> >>>>>> >>>>> >>>>> >>>>> ------------------------------ >>>>> View this message in context: Re: Status of >>>>> MLI?<http://apache-spark-user-list.1001560.n3.nabble.com/Status-of-MLI-tp3610p3612.html> >>>>> >>>>> Sent from the Apache Spark User List mailing list >>>>> archive<http://apache-spark-user-list.1001560.n3.nabble.com/>at >>>>> Nabble.com. >>>>> >>>> >>>> >>>> >>>> ------------------------------ >>>> If you reply to this email, your message will be added to the >>>> discussion below: >>>> >>>> http://apache-spark-user-list.1001560.n3.nabble.com/Status-of-MLI-tp3610p3615.html >>>> To start a new topic under Apache Spark User List, email [hidden >>>> email] <http://user/SendEmail.jtp?type=node&node=3632&i=1> >>>> To unsubscribe from Apache Spark User List, click here. >>>> NAML<http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> >>>> >>> >>> >>> ------------------------------ >>> View this message in context: Re: Status of >>> MLI?<http://apache-spark-user-list.1001560.n3.nabble.com/Status-of-MLI-tp3610p3632.html> >>> Sent from the Apache Spark User List mailing list >>> archive<http://apache-spark-user-list.1001560.n3.nabble.com/>at Nabble.com. >>> >> >> >
