Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/21437#discussion_r192192843
--- Diff: python/pyspark/tests.py ---
@@ -543,6 +543,20 @@ def test_tc_on_driver(self):
tc = TaskContext.get()
self.assertTrue(tc is None)
+ def test_get_local_property(self):
+ """Verify that local properties set on the driver are available in
TaskContext."""
+ key = "testkey"
+ value = "testvalue"
+ self.sc.setLocalProperty(key, value)
+ try:
+ rdd = self.sc.parallelize(range(1), 1)
+ prop1 = rdd.map(lambda x:
TaskContext.get().getLocalProperty(key)).collect()[0]
--- End diff --
I can address that in a follow-up PR that I am working on for SPARK-24396
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
