[
https://issues.apache.org/jira/browse/SPARK-27602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16833194#comment-16833194
]
angerszhu commented on SPARK-27602:
-----------------------------------
[~hyukjin.kwon]
Extract the framework of CBO how to calculate the statistic cost, change some
node to inject my code to get accuracy statistic of partition table. Take the
CBO's algorithm。 Also change the way to process the LogicalPlan
> SparkSQL CBO can't get true size of partition table after partition pruning
> ---------------------------------------------------------------------------
>
> Key: SPARK-27602
> URL: https://issues.apache.org/jira/browse/SPARK-27602
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 2.2.0, 2.3.0, 2.4.0
> Reporter: angerszhu
> Priority: Major
>
> When I want to do extract a cost of one sql for myself's cost framework, I
> found that CBO can't get true size of partition table since when partition
> pruning is true. we just need corresponding partition's size. It just use the
> tables's statistic.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
