GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/20448
[SPARK-23203][SQL] make DataSourceV2Relation immutable ## What changes were proposed in this pull request? This is inspired by https://github.com/apache/spark/pull/20387, but only focus on making the plan immutable. The idea is simple, instead of keeping the mutable `DataSourceReader` in the plan, we should keep `DataSourceV2`, and create the reader when needed. The pushdown information will be stored in the plan, instead of relying on the mutable reader. This can also help us removing 2 unnecessary APIs from `SupportsPushDownCatalystFilters` and `SupportsPushDownFilters`. ## How was this patch tested? I improved the test in `DataSourceV2Suite`, to make sure this new change doesn't break the column pruning and filter push down. You can merge this pull request into a Git repository by running: $ git pull https://github.com/cloud-fan/spark immutable-plan Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/20448.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #20448 ---- commit 7441334e210944e1419be4059d12c06385c586cb Author: Wenchen Fan <wenchen@...> Date: 2018-01-31T03:12:00Z make DataSourceV2Relation immutable ---- --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org