[
https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15658055#comment-15658055
]
Saikat Kanjilal commented on SPARK-9487:
----------------------------------------
[~srowen] given this is my first patch, I wanted to understand a few things, I
was looking at this:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68529/consoleFull
and its not clear that these unit tests are in any way related to my changes,
any insight on this, are these tests that happen to fail due to other
dependencies missing, if so is someone else working to fix these?
I will move onto working on python unit tests under the same PR next.
> Use the same num. worker threads in Scala/Python unit tests
> -----------------------------------------------------------
>
> Key: SPARK-9487
> URL: https://issues.apache.org/jira/browse/SPARK-9487
> Project: Spark
> Issue Type: Improvement
> Components: PySpark, Spark Core, SQL, Tests
> Affects Versions: 1.5.0
> Reporter: Xiangrui Meng
> Labels: starter
> Attachments: ContextCleanerSuiteResults, HeartbeatReceiverSuiteResults
>
>
> In Python we use `local[4]` for unit tests, while in Scala/Java we use
> `local[2]` and `local` for some unit tests in SQL, MLLib, and other
> components. If the operation depends on partition IDs, e.g., random number
> generator, this will lead to different result in Python and Scala/Java. It
> would be nice to use the same number in all unit tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
