[
https://issues.apache.org/jira/browse/SPARK-9487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15658479#comment-15658479
]
Saikat Kanjilal edited comment on SPARK-9487 at 11/11/16 11:20 PM:
-------------------------------------------------------------------
ok so I've spent the last hour or so doing deeper investigations into the
failures, I used this
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68531/ as a
point of reference, listed below is what I found
LogisticRegressionSuite (passed successfully on both my master and feature
branches)
OneVsRestSuite (passed successfully on both my master and feature branches)
DataFrameStatSuite (passed successfully on both my master and feature
branches)
DataFrameSuite (passed successfully on both my master and feature branches)
SQLQueryTestSuite (passed successfully on both my master and feature branches)
ForeachSinkSuite (passed successfully on both my master and feature branches)
JavaAPISuite (failed on both my master and feature branches)
The master branch does not have any code changes from me and the feature branch
of course does
I am running individual tests by issuing commands like the following from the
root directory based on the documentation: ./build/mvn test -P...
-DwildcardSuites=none -Dtest=org.apache.spark.streaming.JavaAPISuite
Therefore my conclusion so far based on the above jenkins report is that my
changes have not introduced any new failures that were not already there,
[~srowen] please let me know if my methodology is off anywhere
was (Author: kanjilal):
ok so I've spent the last hour or so doing deeper investigations into the
failures, I used this
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68531/ as a
point of reference, listed below is what I found
java/scala test my master
branch
my feature branch
LogisticRegressionSuite success
success
OneVsRestSuite success
success
DataFrameStatSuite success
success
DataFrameSuite success
success
SQLQueryTestSuite success
success
ForeachSinkSuite success
success
JavaAPISuite
failure
failure
The master branch does not have any code changes from me and the feature branch
of course does
I am running individual tests by issuing commands like the following from the
root directory based on the documentation: ./build/mvn test -P...
-DwildcardSuites=none -Dtest=org.apache.spark.streaming.JavaAPISuite
Therefore my conclusion so far based on the above jenkins report is that my
changes have not introduced any new failures that were not already there,
[~srowen] please let me know if my methodology is off anywhere
> Use the same num. worker threads in Scala/Python unit tests
> -----------------------------------------------------------
>
> Key: SPARK-9487
> URL: https://issues.apache.org/jira/browse/SPARK-9487
> Project: Spark
> Issue Type: Improvement
> Components: PySpark, Spark Core, SQL, Tests
> Affects Versions: 1.5.0
> Reporter: Xiangrui Meng
> Labels: starter
> Attachments: ContextCleanerSuiteResults, HeartbeatReceiverSuiteResults
>
>
> In Python we use `local[4]` for unit tests, while in Scala/Java we use
> `local[2]` and `local` for some unit tests in SQL, MLLib, and other
> components. If the operation depends on partition IDs, e.g., random number
> generator, this will lead to different result in Python and Scala/Java. It
> would be nice to use the same number in all unit tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
