[ https://issues.apache.org/jira/browse/SPARK-19290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15830031#comment-15830031 ]
Wenchen Fan edited comment on SPARK-19290 at 1/19/17 2:29 PM: -------------------------------------------------------------- sorry my modification to the title failed because of network problems, now I've updated the title and added the description. was (Author: cloud_fan): sorry my modification to the title failed because of network problems, not I've updated the title and added the description. > add a new extending interface in Analyzer for post-hoc resolution > ----------------------------------------------------------------- > > Key: SPARK-19290 > URL: https://issues.apache.org/jira/browse/SPARK-19290 > Project: Spark > Issue Type: Improvement > Components: SQL > Reporter: Wenchen Fan > Assignee: Wenchen Fan > > To implement DDL commands, we added several analyzer rules in sql/hive module > to analyze DDL related plans. However, our Analyzer currently only have one > extending interface: extendedResolutionRules, which defines extra rules that > will be run together with other rules in the resolution batch, and doesn't > fit DDL rules well, because: > 1. DDL rules may do some checking and normalization, but we may do it many > times as the resolution batch will run rules again and again, until fixed > point, and it's hard to tell if a DDL rule has already done its checking and > normalization. It's fine because DDL rules are idempotent, but it's bad for > analysis performance > 2. some DDL rules may depend on others, and it's pretty hard to write if > conditions to guarantee the dependencies. It will be good if we have a batch > which run rules in one pass, so that we can guarantee the dependencies by > rules order. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org