GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/20128
[SPARK-21893][SPARK-22142][BUILD][FOLLOWUP] Enables PySpark tests for Flume and Kafka in Jenkins ## What changes were proposed in this pull request? This PR proposes to enable PySpark tests for Flume and Kafka in Jenkins by explicitly setting the environment variables in `modules.py`. Seems we are not taking the dependencies into account when calculating environment variables: https://github.com/apache/spark/blob/3a07eff5af601511e97a05e6fea0e3d48f74c4f0/dev/run-tests.py#L554-L561 ## How was this patch tested? Manual tests with Jenkins in https://github.com/apache/spark/pull/20126. **Before** - https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85559/consoleFull ``` [info] Setup the following environment variables for tests: ... ``` **After** - https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85560/consoleFull ``` [info] Setup the following environment variables for tests: ENABLE_KAFKA_0_8_TESTS=1 ENABLE_FLUME_TESTS=1 ... ``` You can merge this pull request into a Git repository by running: $ git pull https://github.com/HyukjinKwon/spark SPARK-21893 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/20128.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #20128 ---- commit 3b1533a400ecf2eeee77cf0e18b8cb76a14305b0 Author: hyukjinkwon <gurwls223@...> Date: 2017-12-31T15:35:54Z Enables PySpark tests for Flume and Kafka in Jenkins ---- --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org