[GitHub] [spark] mingjialiu commented on pull request #29564: [WIP][SPARK-32708] Query optimization fails to reuse exchange with DataSourceV2

2020-09-09 Thread GitBox
mingjialiu commented on pull request #29564: URL: https://github.com/apache/spark/pull/29564#issuecomment-689963482 > If exchange reuse is broken, it means plan equality is broken somewhere. I think `Seq[Expression]` is OK as long as we canonicalize it before comparing it. `FileSourceScanE

[GitHub] [spark] mingjialiu commented on pull request #29564: [WIP][SPARK-32708] Query optimization fails to reuse exchange with DataSourceV2

2020-09-09 Thread GitBox
mingjialiu commented on pull request #29564: URL: https://github.com/apache/spark/pull/29564#issuecomment-689963685 > In Branch 3.0, there is a mixed-in trait `SupportsPushDownFilters` which is introduced by #19136 and #19424 . > > However, if we are going to cherry-pick the PRs ment

[GitHub] [spark] mingjialiu commented on pull request #29564: [WIP][SPARK-32708] Query optimization fails to reuse exchange with DataSourceV2

2020-09-10 Thread GitBox
mingjialiu commented on pull request #29564: URL: https://github.com/apache/spark/pull/29564#issuecomment-690597364 > The fix LGTM, can you add a test? Hi, it's a bit tricky to repro in unit test. Can I get some pointers on populating different expression ids for the same column?

[GitHub] [spark] mingjialiu commented on pull request #29564: [WIP][SPARK-32708] Query optimization fails to reuse exchange with DataSourceV2

2020-09-10 Thread GitBox
mingjialiu commented on pull request #29564: URL: https://github.com/apache/spark/pull/29564#issuecomment-690599004 Hi, Regarding test coverage, it's a bit tricky to repro in a unit test. Can I get some pointers on populating different expression ids for the same column? Or tes

[GitHub] [spark] mingjialiu commented on pull request #29564: [WIP][SPARK-32708] Query optimization fails to reuse exchange with DataSourceV2

2020-09-10 Thread GitBox
mingjialiu commented on pull request #29564: URL: https://github.com/apache/spark/pull/29564#issuecomment-690810458 > > The fix LGTM, can you add a test? > > Regarding test coverage,  it's a bit tricky to repro in a unit test. Can I get some pointers on populating different expressio

[GitHub] [spark] mingjialiu commented on pull request #29564: [WIP][SPARK-32708] Query optimization fails to reuse exchange with DataSourceV2

2020-09-10 Thread GitBox
mingjialiu commented on pull request #29564: URL: https://github.com/apache/spark/pull/29564#issuecomment-690811936 > > The fix LGTM, can you add a test? > > Regarding test coverage,  it's a bit tricky to repro in a unit test. Can I get some pointers on populating different expressio

[GitHub] [spark] mingjialiu commented on pull request #29564: [WIP][SPARK-32708] Query optimization fails to reuse exchange with DataSourceV2

2020-09-12 Thread GitBox
mingjialiu commented on pull request #29564: URL: https://github.com/apache/spark/pull/29564#issuecomment-690811936 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a