Timothy Miller created SPARK-42776:
--------------------------------------
Summary: BroadcastHashJoinExec.requiredChildDistribution called
before columnar replacement rules
Key: SPARK-42776
URL: https://issues.apache.org/jira/browse/SPARK-42776
Project: Spark
Issue Type: Bug
Components: Optimizer
Affects Versions: 3.3.1
Environment: I'm prototyping on a Mac, but that's not really relevant.
Reporter: Timothy Miller
I am trying to replace BroadcastHashJoinExec with a columnar equivalent.
However, I noticed that BroadcastHashJoinExec.requiredChildDistribution gets
called BEFORE the columnar replacement rules. As a result, the object that gets
broadcast is the plain old hashmap created from row data. By the time the
columnar replacement rules are applied, it's too late to get Spark to broadcast
any other kind of object.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
