GitHub user maryannxue opened a pull request: https://github.com/apache/spark/pull/21876
[SPARK-24802][SQL][FOLLOW-UP] Add a new config for Optimization Rule Exclusion ## What changes were proposed in this pull request? This is an extension to the original PR, in which rule exclusion did not work for classes derived from Optimizer, e.g., SparkOptimizer. To solve this issue, Optimizer and its derived classes will define/override `defaultBatches` and `nonExcludableRules` in order to define its default rule set as well as rules that cannot be excluded by the SQL config. In the meantime, Optimizer's `batches` method is dedicated to the rule exclusion logic and is defined "final". ## How was this patch tested? Added UT. You can merge this pull request into a Git repository by running: $ git pull https://github.com/maryannxue/spark rule-exclusion Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/21876.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #21876 ---- commit eaec2f5f2b4e3193de41655b84a1dc936b0e50a3 Author: maryannxue <maryannxue@...> Date: 2018-07-13T21:32:01Z [SPARK-24802] Optimization Rule Exclusion commit 84f1a6b5cba08df8684179e9d7195545be655e76 Author: maryannxue <maryannxue@...> Date: 2018-07-18T06:13:50Z Address review comments commit ff23edf81a4a78d1589ed582a1802b94a8ebf4c6 Author: maryannxue <maryannxue@...> Date: 2018-07-21T02:37:54Z Address review comments commit b154979236e211dc7185ca8e450493f0c6b0f469 Author: maryannxue <maryannxue@...> Date: 2018-07-21T02:41:21Z change test name commit 87afe4fbcaf71d303b07612f9ceb9ad25dd3dcda Author: maryannxue <maryannxue@...> Date: 2018-07-23T00:35:01Z address review comments commit 39b6ce9548c99363e81cb246b4cbe5534d710f3e Author: maryannxue <maryannxue@...> Date: 2018-07-23T04:28:00Z address review comments commit a2161ef1f333f2cc039df0ecc8c96e5ec27e00ff Author: maryannxue <maryannxue@...> Date: 2018-07-25T18:52:13Z Merge remote-tracking branch 'origin/master' into rule-exclusion commit 3730053d7386188042b2f2d4bd6784c3de722df6 Author: maryannxue <maryannxue@...> Date: 2018-07-25T20:08:19Z Extend rule-exclusion to Optimizer sub-classes, esp. SparkOptimizer ---- --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org