Yicong Huang created SPARK-60280:
------------------------------------
Summary: Replace _is_iter_based with an EvalTypeHandler class
property
Key: SPARK-60280
URL: https://issues.apache.org/jira/browse/SPARK-60280
Project: Spark
Issue Type: Sub-task
Components: PySpark
Affects Versions: 5.0.0
Reporter: Yicong Huang
Replace the hardcoded _is_iter_based(eval_type) set in python/pyspark/worker.py
with an iter_based_udf ClassVar on EvalTypeHandler, set on the handlers of the
iterator-based eval types. The profiler wrappers take the flag from the
resolved handler class. No behavior change.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
