Hyukjin Kwon created SPARK-61537:
------------------------------------
Summary: Copy the Observations of a deep copy of a DataFrame in
the Python client, as PySpark does
Key: SPARK-61537
URL: https://issues.apache.org/jira/browse/SPARK-61537
Project: Spark
Issue Type: Bug
Components: Connect
Affects Versions: connect-rust-4.3.0
Reporter: Hyukjin Kwon
Follow-up of SPARK-61426. After copy.deepcopy(df) of a DataFrame observed with
an Observation, an action on the copy sets the original Observation, as the
copy is rebuilt from the relation of the plan and takes the Observations of the
original plan. PySpark's deep copy copies the plan objects, and its Observation
with them, so the original Observation stays unset (copy.copy shares the plan,
and the Observation, in both).
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]