[GitHub] spark issue #16998: [SPARK-19665][SQL][WIP] Improve constraint propagation

2017-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16998 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73316/ Test PASSed. ---

[GitHub] spark issue #16998: [SPARK-19665][SQL][WIP] Improve constraint propagation

2017-02-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16998 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16998: [SPARK-19665][SQL][WIP] Improve constraint propagation

2017-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16998 **[Test build #73316 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73316/testReport)** for PR 16998 at commit [`5be21b3`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #16998: [SPARK-19665][SQL][WIP] Improve constraint propagation

2017-02-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16998 **[Test build #73316 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73316/testReport)** for PR 16998 at commit [`5be21b3`](https://github.com/apache/spark/commit/5b

[GitHub] spark issue #16998: [SPARK-19665][SQL][WIP] Improve constraint propagation

2017-02-21 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16998 @sameeragarwal That's correct. > By the way, as an aside we should probably allow constraint inference/propagation to be turned off via a conf flag to provide a quick work around against the

[GitHub] spark issue #16998: [SPARK-19665][SQL][WIP] Improve constraint propagation

2017-02-21 Thread sameeragarwal
Github user sameeragarwal commented on the issue: https://github.com/apache/spark/pull/16998 By the way, as an aside we should probably allow constraint inference/propagation to be turned off via a conf flag to provide a quick work around against these kind of problems. --- If your

[GitHub] spark issue #16998: [SPARK-19665][SQL][WIP] Improve constraint propagation

2017-02-21 Thread sameeragarwal
Github user sameeragarwal commented on the issue: https://github.com/apache/spark/pull/16998 @viirya please correct me if I'm wrong but scanning through this patch, it appears that the underlying problem is that duplicating and tracking aliased constraints using a `Set` tends to blow

[GitHub] spark issue #16998: [SPARK-19665][SQL][WIP] Improve constraint propagation

2017-02-20 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16998 > I do not get your points. What does this mean? Constraint propagation is a bottom up mechanism for inferring the constraints. Can you elaborate your idea in the more formal way. We fully e

[GitHub] spark issue #16998: [SPARK-19665][SQL][WIP] Improve constraint propagation

2017-02-20 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16998 > Another issue is we actually don't need the additional constraints at most of time. For example, if there is a constraint "a > b", and "a" is aliased to "c" and "d". When we use this constraint

[GitHub] spark issue #16998: [SPARK-19665][SQL][WIP] Improve constraint propagation

2017-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16998 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16998: [SPARK-19665][SQL][WIP] Improve constraint propagation

2017-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16998 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73174/ Test PASSed. ---

[GitHub] spark issue #16998: [SPARK-19665][SQL][WIP] Improve constraint propagation

2017-02-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16998 **[Test build #73174 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73174/testReport)** for PR 16998 at commit [`6cb896f`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #16998: [SPARK-19665][SQL][WIP] Improve constraint propagation

2017-02-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16998 **[Test build #73174 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73174/testReport)** for PR 16998 at commit [`6cb896f`](https://github.com/apache/spark/commit/6c

[GitHub] spark issue #16998: [SPARK-19665][SQL][WIP] Improve constraint propagation

2017-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16998 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73163/ Test FAILed. ---

[GitHub] spark issue #16998: [SPARK-19665][SQL][WIP] Improve constraint propagation

2017-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16998 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16998: [SPARK-19665][SQL][WIP] Improve constraint propagation

2017-02-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16998 **[Test build #73163 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73163/testReport)** for PR 16998 at commit [`d691c66`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #16998: [SPARK-19665][SQL][WIP] Improve constraint propagation

2017-02-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16998 **[Test build #73163 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73163/testReport)** for PR 16998 at commit [`d691c66`](https://github.com/apache/spark/commit/d6

[GitHub] spark issue #16998: [SPARK-19665][SQL][WIP] Improve constraint propagation

2017-02-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16998 **[Test build #73158 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73158/testReport)** for PR 16998 at commit [`24fb723`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #16998: [SPARK-19665][SQL][WIP] Improve constraint propagation

2017-02-20 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16998 @hvanhovell Yes. #16785 only does a limited improvement. Both #16785 and this are non-parallel approach. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark issue #16998: [SPARK-19665][SQL][WIP] Improve constraint propagation

2017-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16998 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16998: [SPARK-19665][SQL][WIP] Improve constraint propagation

2017-02-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16998 **[Test build #73159 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73159/testReport)** for PR 16998 at commit [`917de74`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #16998: [SPARK-19665][SQL][WIP] Improve constraint propagation

2017-02-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16998 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73159/ Test FAILed. ---

[GitHub] spark issue #16998: [SPARK-19665][SQL][WIP] Improve constraint propagation

2017-02-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16998 **[Test build #73159 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73159/testReport)** for PR 16998 at commit [`917de74`](https://github.com/apache/spark/commit/91

[GitHub] spark issue #16998: [SPARK-19665][SQL][WIP] Improve constraint propagation

2017-02-20 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16998 @viirya does this PR supersede #16785? I do like the non-parallel approach. I will try to take a more in-depth look at the end of the week (beginning of the next sprint). --- If your project

[GitHub] spark issue #16998: [SPARK-19665][SQL][WIP] Improve constraint propagation

2017-02-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16998 **[Test build #73158 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73158/testReport)** for PR 16998 at commit [`24fb723`](https://github.com/apache/spark/commit/24

[GitHub] spark issue #16998: [SPARK-19665][SQL][WIP] Improve constraint propagation

2017-02-20 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16998 cc @cloud-fan @hvanhovell @sameeragarwal --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature ena