[ https://issues.apache.org/jira/browse/SPARK-12098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-12098: ------------------------------------ Assignee: (was: Apache Spark) > Cross validator with multi-arm bandit search > -------------------------------------------- > > Key: SPARK-12098 > URL: https://issues.apache.org/jira/browse/SPARK-12098 > Project: Spark > Issue Type: New Feature > Components: ML, MLlib > Reporter: Xusen Yin > > The classic cross-validation requires all inner classifiers iterate to a > fixed number of iterations, or until convergence states. It is costly > especially in the massive data scenario. According to the paper > Non-stochastic Best Arm Identification and Hyperparameter Optimization > (http://arxiv.org/pdf/1502.07943v1.pdf), we can see a promising way to reduce > the amount of total iterations of cross-validation with multi-armed bandit > search. > The multi-armed bandit search for cross-validation (bandit search for short) > requires warm-start of ml algorithms, and fine-grained control of the inner > behavior of the corss validator. > Since there are bunch of algorithms of bandit search to find the best > parameter set, we intent to provide only a few of them in the beginning to > reduce the test/perf-test work and make it more stable. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org