[GitHub] spark issue #13570: [SPARK-15832][SQL] Embedded IN/EXISTS predicate subquery...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/13570 @hvanhovell @jkbradley Could you add @ioana-delaney to the whitelist? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13570: [SPARK-15832][SQL] Embedded IN/EXISTS predicate subquery...
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/13570 Did this break the build? https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Compile/job/spark-master-compile-sbt-scala-2.10/1707/console It looks like the last commit may not have been tested. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13570: [SPARK-15832][SQL] Embedded IN/EXISTS predicate subquery...
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13570 LGTM - merging to master/2.0 thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13570: [SPARK-15832][SQL] Embedded IN/EXISTS predicate subquery...
Github user ioana-delaney commented on the issue: https://github.com/apache/spark/pull/13570 @hvanhovell The EXISTS/NOT EXISTS predicates will have an empty condition. e.g. select c1 from t1 where EXISTS (select c2 from t2) == Optimized Logical Plan == Project [_1#224 AS c1#227] +- Join LeftSemi :- LocalRelation [_1#224, _2#225] +- LocalRelation [c2#239] But the other subquery predicates are quaranteed to have at least one condition. Regarding the rewriteExistentialExpr interface, I think that I need to pass an expression instead of a sequence of conditions since the last case in the main rewrite rule does not have conditions. It's just an expression. e.g. where (case when c2 IN (select 1 as one) then 1 else 2) = c1 Please let me know. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13570: [SPARK-15832][SQL] Embedded IN/EXISTS predicate subquery...
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13570 @ioana-delaney no worries. I think the approach you have taken is the correct one. I have left one smallish comment. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13570: [SPARK-15832][SQL] Embedded IN/EXISTS predicate subquery...
Github user ioana-delaney commented on the issue: https://github.com/apache/spark/pull/13570 @hvanhovell Thank you for reviewing the changes and I apologize for the delay in replying. I simplified the code. However, I don't think this is what you suggested. What you suggested I believe, was to pull out the rewrite of the inner expressions into an outer loop. I made a few attempts but I could not find a way to decouple the expressions' generation from the new plans' generation. When rewriting the expression, I am also building the plans bottom-up. Please take a look at my new changes and advise. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13570: [SPARK-15832][SQL] Embedded IN/EXISTS predicate subquery...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13570 **[Test build #3070 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3070/consoleFull)** for PR 13570 at commit [`eea703a`](https://github.com/apache/spark/commit/eea703aa673aab5f56d6a97ad86860422cd563a3). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13570: [SPARK-15832][SQL] Embedded IN/EXISTS predicate subquery...
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/13570 @ioana-delaney great catch! The overall PR seems pretty solid. I left one smallish code organization related comment. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13570: [SPARK-15832][SQL] Embedded IN/EXISTS predicate subquery...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13570 **[Test build #3070 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3070/consoleFull)** for PR 13570 at commit [`eea703a`](https://github.com/apache/spark/commit/eea703aa673aab5f56d6a97ad86860422cd563a3). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #13570: [SPARK-15832][SQL] Embedded IN/EXISTS predicate subquery...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13570 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
