[GitHub] spark issue #22518: [SPARK-25482][SQL] ReuseSubquery can be useless when the...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22518 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98681/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22518: [SPARK-25482][SQL] ReuseSubquery can be useless when the...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22518 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22518: [SPARK-25482][SQL] ReuseSubquery can be useless when the...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22518 **[Test build #98681 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98681/testReport)** for PR 22518 at commit [`144091f`](https://github.com/apache/spark/commit/144091fed2a05d0d0a9d8ed3d592be61473d814b). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22518: [SPARK-25482][SQL] ReuseSubquery can be useless when the...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22518 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/4913/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22518: [SPARK-25482][SQL] ReuseSubquery can be useless when the...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22518 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22518: [SPARK-25482][SQL] ReuseSubquery can be useless when the...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22518 **[Test build #98681 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98681/testReport)** for PR 22518 at commit [`144091f`](https://github.com/apache/spark/commit/144091fed2a05d0d0a9d8ed3d592be61473d814b). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22518: [SPARK-25482][SQL] ReuseSubquery can be useless when the...
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/22518 @mgaido91 I see, thanks for the explanation! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22518: [SPARK-25482][SQL] ReuseSubquery can be useless when the...
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/22518 @gengliangwang no, let me cite and explain the PR description. I am not sure how to improve it, but if you have suggestions I am happy to. The main point of the PR is to address an issue which arise when: > When a `ExecSubqueryExpression` is copied Now the point is, can this condition happen? The answer is yes, and one situation in which this happens (as reported in the JIRA) is > when a filter containing a scalar subquery is pushed to a DataSource. So in the plan we have two `ExecSubqueryExpression` each with a copy of the same `SubqueryExec`. The problem which arises in this condition is that: > `ReuseSubquery` becomes useless, as replacing the `SubqueryExec` is ignored since the new plan is equal to the previous one. So this result in the subquery being executed twice (as the two `SubqueryExec` are distinct, despite they are the same). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22518: [SPARK-25482][SQL] ReuseSubquery can be useless when the...
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/22518 @mgaido91 Sorry but can you have a more detailed explanation in the PR description? With the code changes, the predicate with subquery result can be push down into data source. Is this the main point of the PR? And why is that creating a new expr ID can fix it? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22518: [SPARK-25482][SQL] ReuseSubquery can be useless when the...
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/22518 > hmm, how can this happen? you can check the UT which reproduces the issue. The scalar subquery is pushed down as part of the filter `GreaterThen` --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22518: [SPARK-25482][SQL] ReuseSubquery can be useless when the...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22518 > This can happen for instance when a filter containing a scalar subquery is pushed to a DataSource hmm, how can this happen? I don't think a data source can handle a filter of subquery... --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22518: [SPARK-25482][SQL] ReuseSubquery can be useless when the...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22518 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22518: [SPARK-25482][SQL] ReuseSubquery can be useless when the...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22518 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96472/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22518: [SPARK-25482][SQL] ReuseSubquery can be useless when the...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22518 **[Test build #96472 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96472/testReport)** for PR 22518 at commit [`ec72458`](https://github.com/apache/spark/commit/ec724581d4bee264c83dc0824fe6d8a7be43e2ef). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22518: [SPARK-25482][SQL] ReuseSubquery can be useless when the...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22518 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22518: [SPARK-25482][SQL] ReuseSubquery can be useless when the...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22518 **[Test build #96472 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96472/testReport)** for PR 22518 at commit [`ec72458`](https://github.com/apache/spark/commit/ec724581d4bee264c83dc0824fe6d8a7be43e2ef). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22518: [SPARK-25482][SQL] ReuseSubquery can be useless when the...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22518 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3379/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22518: [SPARK-25482][SQL] ReuseSubquery can be useless when the...
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/22518 cc @cloud-fan @dongjoon-hyun @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22518: [SPARK-25482][SQL] ReuseSubquery can be useless when the...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22518 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22518: [SPARK-25482][SQL] ReuseSubquery can be useless when the...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22518 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96427/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22518: [SPARK-25482][SQL] ReuseSubquery can be useless when the...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22518 **[Test build #96427 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96427/testReport)** for PR 22518 at commit [`36fa664`](https://github.com/apache/spark/commit/36fa664c6d251901270984115ff2ebfd1b665fca). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22518: [SPARK-25482][SQL] ReuseSubquery can be useless when the...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22518 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22518: [SPARK-25482][SQL] ReuseSubquery can be useless when the...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22518 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96428/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22518: [SPARK-25482][SQL] ReuseSubquery can be useless when the...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22518 **[Test build #96428 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96428/testReport)** for PR 22518 at commit [`7c75067`](https://github.com/apache/spark/commit/7c75067767ed6935960d09c7915da86fea3553fa). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22518: [SPARK-25482][SQL] ReuseSubquery can be useless when the...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22518 **[Test build #96428 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96428/testReport)** for PR 22518 at commit [`7c75067`](https://github.com/apache/spark/commit/7c75067767ed6935960d09c7915da86fea3553fa). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22518: [SPARK-25482][SQL] ReuseSubquery can be useless when the...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22518 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3345/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22518: [SPARK-25482][SQL] ReuseSubquery can be useless when the...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22518 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22518: [SPARK-25482][SQL] ReuseSubquery can be useless when the...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22518 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22518: [SPARK-25482][SQL] ReuseSubquery can be useless when the...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22518 **[Test build #96427 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96427/testReport)** for PR 22518 at commit [`36fa664`](https://github.com/apache/spark/commit/36fa664c6d251901270984115ff2ebfd1b665fca). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22518: [SPARK-25482][SQL] ReuseSubquery can be useless when the...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22518 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3344/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
