Hyukjin Kwon created SPARK-61537:
------------------------------------

             Summary: Copy the Observations of a deep copy of a DataFrame in 
the Python client, as PySpark does
                 Key: SPARK-61537
                 URL: https://issues.apache.org/jira/browse/SPARK-61537
             Project: Spark
          Issue Type: Bug
          Components: Connect
    Affects Versions: connect-rust-4.3.0
            Reporter: Hyukjin Kwon


Follow-up of SPARK-61426. After copy.deepcopy(df) of a DataFrame observed with 
an Observation, an action on the copy sets the original Observation, as the 
copy is rebuilt from the relation of the plan and takes the Observations of the 
original plan. PySpark's deep copy copies the plan objects, and its Observation 
with them, so the original Observation stays unset (copy.copy shares the plan, 
and the Observation, in both).



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to