jenkins uses SBT, so you need to do the test run there. They are different, and have different test runners in particular.
On 30 Nov 2016, at 04:14, Saikat Kanjilal <sxk1...@hotmail.com<mailto:sxk1...@hotmail.com>> wrote: Hello Spark dev community, I took this the following jira item (https://github.com/apache/spark/pull/15848) and am looking for some general pointers, it seems that I am running into issues where things work successfully doing local development on my macbook pro but fail on jenkins for a multitiude of reasons and errors, here's an example, if you see this build output report: https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69297/ you will see the DataFrameStatSuite, now locally I am running these individual tests with this command: ./build/mvn test -P... -DwildcardSuites=none -Dtest=org.apache.spark.sql.DataFrameStatSuite. It seems that I need to emulate a jenkins like environment locally, this seems sort of like an untenable hurdle, granted that my changes involve changing the total number of workers in the sparkcontext and if so should I be testing my changes in an environment that more closely resembles jenkins. I really want to work on/complete this PR but I keep getting hamstrung by a dev environment that is not equivalent to our CI environment. There's always the option of creating a linux VM/container with jenkins in it; there's a nice trick there in which you can have it watch a git branch, and have it kick off a run whenever you push up to it. That way, you have your own personal jenkins to do the full regression tests, while you yourself work on a small bit. I'm guessing/hoping I'm not the first one to run into this so some insights. pointers to get past this would be very appreciated , would love to keep contributing and hoping this is a hurdle that's overcomeable with some tweaks to my dev environment. Thanks in advance.