[
https://issues.apache.org/jira/browse/SPARK-36630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
gaoyajun02 updated SPARK-36630:
-------------------------------
Description: Currently when AQE's queryStage is not materialized, it uses
the stats of the logical plan to estimate whether the plan can be converted to
BHJ, and in some scenarios the estimated value is several orders of magnitude
smaller than the actual broadcast data, which can lead to large tables being
broadcast (was: Currently when AQE's queryStage is not materialized, it uses
the stats of the logical plan to estimate whether the plan can be converted to
BHJ, and in some scenarios the estimated value is several orders of magnitude
larger than the actual broadcast data, which can lead to large tables being
broadcast)
> Add the option to use physical statistics to avoid large tables being
> broadcast
> -------------------------------------------------------------------------------
>
> Key: SPARK-36630
> URL: https://issues.apache.org/jira/browse/SPARK-36630
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.2.0
> Reporter: gaoyajun02
> Priority: Major
>
> Currently when AQE's queryStage is not materialized, it uses the stats of the
> logical plan to estimate whether the plan can be converted to BHJ, and in
> some scenarios the estimated value is several orders of magnitude smaller
> than the actual broadcast data, which can lead to large tables being broadcast
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
