[
https://issues.apache.org/jira/browse/SPARK-6586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14386167#comment-14386167
]
Michael Armbrust commented on SPARK-6586:
-----------------------------------------
I can see the utility for seeing the original plan in any given run of the
optimizer. However, providing it for any arbitrarily assembly of query
plans feels like unnecessary complexity to me. I think its only reasonable
to add such instrumentation when it was actually useful to solve an issue.
Doing so speculatively only leads to code complexity. If you have a
concrete example where this information would be useful we can continue to
discuss, but otherwise this issue should be closed.
Additionally, PRs to add new features should *always* have tests.
Otherwise these features will be broken almost immediately.
> Add the capability of retrieving original logical plan of DataFrame
> -------------------------------------------------------------------
>
> Key: SPARK-6586
> URL: https://issues.apache.org/jira/browse/SPARK-6586
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Reporter: Liang-Chi Hsieh
> Priority: Minor
>
> In order to solve a bug, since #5217, {{DataFrame}} now uses analyzed plan
> instead of logical plan. However, by doing that we can't know the logical
> plan of a {{DataFrame}}. But it might be still useful and important to
> retrieve the original logical plan in some use cases.
> In this pr, we introduce the capability of retrieving original logical plan
> of {{DataFrame}}.
> The approach is that we add an {{analyzed}} variable to {{LogicalPlan}}. Once
> {{Analyzer}} finishes analysis, it sets {{analyzed}} of {{LogicalPlan}} as
> {{true}}. In {{QueryExecution}}, we keep the original logical plan in the
> analyzed plan. In {{LogicalPlan}}, a method {{originalPlan}} is added to
> recursively replace the analyzed logical plan with original logical plan and
> retrieve it.
> Besides the capability of retrieving original logical plan, this modification
> also can avoid do plan analysis if it is already analyzed.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
