HyukjinKwon commented on issue #24946: [SPARK-27234][SS][PYTHON] Use
InheritableThreadLocal for current epoch in EpochTracker (to support Python
UDFs)
URL: https://github.com/apache/spark/pull/24946#issuecomment-521464400
Yup. BTW `IntegratedUDFTestUtils` doesn't exist in master branch. I
HyukjinKwon commented on issue #24946: [SPARK-27234][SS][PYTHON] Use
InheritableThreadLocal for current epoch in EpochTracker (to support Python
UDFs)
URL: https://github.com/apache/spark/pull/24946#issuecomment-521426162
Am fine with backporting but @zsxwing WDYT?
---
HyukjinKwon commented on issue #24946: [SPARK-27234][SS][PYTHON] Use
InheritableThreadLocal for current epoch in EpochTracker (to support Python
UDFs)
URL: https://github.com/apache/spark/pull/24946#issuecomment-520685272
This fix will be included in Apache Spark 3.0. I think you should up
HyukjinKwon commented on issue #24946: [SPARK-27234][SS][PYTHON] Use
InheritableThreadLocal for current epoch in EpochTracker (to support Python
UDFs)
URL: https://github.com/apache/spark/pull/24946#issuecomment-514437195
Merged to master.
Thanks all.
--
HyukjinKwon commented on issue #24946: [SPARK-27234][SS][PYTHON] Use
InheritableThreadLocal for current epoch in EpochTracker (to support Python
UDFs)
URL: https://github.com/apache/spark/pull/24946#issuecomment-514013132
Thanks for review and approval, @zsxwing.
Just FYI, there's s
HyukjinKwon commented on issue #24946: [SPARK-27234][SS][PYTHON] Use
InheritableThreadLocal for current epoch in EpochTracker (to support Python
UDFs)
URL: https://github.com/apache/spark/pull/24946#issuecomment-514012160
Thanks for review and approval, @zsxwing.
BTW, I understand t
HyukjinKwon commented on issue #24946: [SPARK-27234][SS][PYTHON] Use
InheritableThreadLocal for current epoch in EpochTracker (to support Python
UDFs)
URL: https://github.com/apache/spark/pull/24946#issuecomment-513676624
retest this please
HyukjinKwon commented on issue #24946: [SPARK-27234][SS][PYTHON] Use
InheritableThreadLocal for current epoch in EpochTracker (to support Python
UDFs)
URL: https://github.com/apache/spark/pull/24946#issuecomment-507101455
gentle ping .. :-) ..
-
HyukjinKwon commented on issue #24946: [SPARK-27234][SS][PYTHON] Use
InheritableThreadLocal for current epoch in EpochTracker (to support Python
UDFs)
URL: https://github.com/apache/spark/pull/24946#issuecomment-505662431
In case of `org.apache.spark.TaskContext`, looks it's manually set f
HyukjinKwon commented on issue #24946: [SPARK-27234][SS][PYTHON] Use
InheritableThreadLocal for current epoch in EpochTracker (to support Python
UDFs)
URL: https://github.com/apache/spark/pull/24946#issuecomment-505260782
#24945 is merged. Let me rebase.
-
HyukjinKwon commented on issue #24946: [SPARK-27234][SS][PYTHON] Use
InheritableThreadLocal for current epoch in EpochTracker (to support Python
UDFs)
URL: https://github.com/apache/spark/pull/24946#issuecomment-505227175
It works after this PR as below:
```python
from pyspark.sq
HyukjinKwon commented on issue #24946: [SPARK-27234][SS][PYTHON] Use
InheritableThreadLocal for current epoch in EpochTracker (to support Python
UDFs)
URL: https://github.com/apache/spark/pull/24946#issuecomment-504873202
Ah .. `ContinuousCoalesceRDD` is dependent on the thread local not b
12 matches
Mail list logo