[
https://issues.apache.org/jira/browse/SPARK-21027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16045228#comment-16045228
]
Joseph K. Bradley commented on SPARK-21027:
-------------------------------------------
One other comment: The Scala implementation can be faster for some tasks but
can be slower for others (b/c of conflicting Spark jobs). It'd be nice to
permit the user to tune the parallelism, with some reasonable default such as 4
parallel training jobs.
> Parallel One vs. Rest Classifier
> --------------------------------
>
> Key: SPARK-21027
> URL: https://issues.apache.org/jira/browse/SPARK-21027
> Project: Spark
> Issue Type: New Feature
> Components: PySpark
> Affects Versions: 2.2.0, 2.2.1
> Reporter: Ajay Saini
>
> Currently, the Scala implementation of OneVsRest allows the user to run a
> parallel implementation in which each class is evaluated in a different
> thread. This implementation allows up to a 2X speedup as determined by
> experiments but is not currently not tunable. Furthermore, the python
> implementation of OneVsRest does not parallelize at all. It would be useful
> to add a parallel, tunable implementation of OneVsRest to the python library
> in order to speed up the algorithm.
> A ticket for the Scala implementation of this classifier is here:
> https://issues.apache.org/jira/browse/SPARK-21028
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
