zhoubin created SPARK-51704:
-------------------------------
Summary: Eliminate unnecessary collect operation
Key: SPARK-51704
URL: https://issues.apache.org/jira/browse/SPARK-51704
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 3.5.5, 4.0.0, 4.1.0
Reporter: zhoubin
Change the `TreeNode` operation from `{{{}collect`{}}} to
`{{{}collectFirst`,{}}} when it is supposed to find the first qualified node
rather than apply function through out all the node and it's child, this
function should not take any effect or make any change to the node which
applied for.
Any scenarios satisfy below factors:
# Apply certain operation recursively on the `TreeNode` including itself
# The final purpose is to find the first qualified node in a pre order way
# The final purpose is to verify that there is at least one node which satisfy
the requirement
# Partial function applied should not effect the node or other related
Especially, the last factor should be reviewed carefully.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
