Maryann Xue created SPARK-24802: ----------------------------------- Summary: Optimization Rule Exclusion Key: SPARK-24802 URL: https://issues.apache.org/jira/browse/SPARK-24802 Project: Spark Issue Type: New Feature Components: SQL Affects Versions: 2.3.0 Reporter: Maryann Xue
Since Spark has provided fairly clear interfaces for adding user-defined optimization rules, it would be nice to have an easy-to-use interface for excluding an optimization rule from the Spark query optimizer as well. This would make customizing Spark optimizer easier and sometimes could debugging issues too. # Add a new config {{spark.sql.optimizer.excludedRules}}, with the value being a list of rule names separated by comma. # Modify the current {{batches}} method to remove the excluded rules from the default batches. Log the rules that have been excluded. # Split the existing default batches into "post-analysis batches" and "optimization batches" so that only rules in the "optimization batches" can be excluded. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org