Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/23117#discussion_r236431733
--- Diff: dev/run-tests.py ---
@@ -434,6 +434,63 @@ def run_python_tests(test_modules, parallelism):
run_cmd(command)
+def run_python_tests_with_coverage(test_modules, parallelism):
+ set_title_and_block("Running PySpark tests with coverage report",
"BLOCK_PYSPARK_UNIT_TESTS")
+
+ command = [os.path.join(SPARK_HOME, "python",
"run-tests-with-coverage")]
+ if test_modules != [modules.root]:
+ command.append("--modules=%s" % ','.join(m.name for m in
test_modules))
+ command.append("--parallelism=%i" % parallelism)
+ run_cmd(command)
+ post_python_tests_results()
+
+
+def post_python_tests_results():
+ if "SPARK_TEST_KEY" not in os.environ:
+ print("[error] 'SPARK_TEST_KEY' environment variable was not set.
Unable to post"
+ "PySpark coverage results.")
+ sys.exit(1)
--- End diff --
hmm, this will be a headache for us in our internal builds, as we also run
these tests, and also set AMPLAB_JENKINS as its sort of used as a catch-all for
making builds quiet etc., but we won't have this key obviously.
you dont' need to cater to our internal builds, of course, but I'm
wondering if this will cause a headache for more users that want to run tests
themselves but won't have the key?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
