Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/13876
Thank you for review and merging, @cloud-fan and @rxin .
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/13876
thanks, merging to master!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/13876
At this time, it passed as expected.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13876
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13876
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61901/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13876
**[Test build #61901 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61901/consoleFull)**
for PR 13876 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13876
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61897/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13876
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13876
**[Test build #61897 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61897/consoleFull)**
for PR 13876 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13876
**[Test build #61901 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61901/consoleFull)**
for PR 13876 at commit
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/13876
There is one failure in `HiveSparkSubmitSuite`. It seems to be irrelevant.
I'll retry the test.
```
- SPARK-8020: set sql conf in spark conf *** FAILED *** (31 seconds, 981
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/13876
Retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13876
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61898/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13876
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13876
**[Test build #61898 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61898/consoleFull)**
for PR 13876 at commit
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/13876
Thank you for review, @cloud-fan .
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/13876
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13876
**[Test build #61898 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61898/consoleFull)**
for PR 13876 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13876
**[Test build #61897 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61897/consoleFull)**
for PR 13876 at commit
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/13876
Looks pretty good.
cc @cloud-fan for another look.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13876
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13876
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61888/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13876
**[Test build #61888 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61888/consoleFull)**
for PR 13876 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13876
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13876
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61887/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13876
**[Test build #61887 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61887/consoleFull)**
for PR 13876 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13876
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61878/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13876
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13876
**[Test build #61878 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61878/consoleFull)**
for PR 13876 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13876
**[Test build #61888 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61888/consoleFull)**
for PR 13876 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13876
**[Test build #61887 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61887/consoleFull)**
for PR 13876 at commit
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/13876
Now, the scope of PR is reduced a lot. But, I hope this PR still covers
majority of real queries.
Thank you for many advice.
---
If your project is set up for it, you can reply to this
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13876
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13876
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61875/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13876
**[Test build #61875 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61875/consoleFull)**
for PR 13876 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13876
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61871/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13876
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13876
**[Test build #61871 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61871/consoleFull)**
for PR 13876 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13876
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13876
**[Test build #61869 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61869/consoleFull)**
for PR 13876 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13876
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61869/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13876
**[Test build #61878 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61878/consoleFull)**
for PR 13876 at commit
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/13876
Hmm. I will rename it according to the advice. Sorry for the delay!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/13876
Although the name is different from your advice, now it has a simple lazy
val, `optimizable`, for that.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13876
**[Test build #61875 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61875/consoleFull)**
for PR 13876 at commit
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/13876
Oh, I see. Sorry for misunderstanding that.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/13876
It's ok. Just have it for list.forall(_.deterministic). We can do the check
(ExpressionSet(list) < conf_min_value || ExpressionSet(list) < list.size) later.
---
If your project is set up for it, you
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/13876
Hm, not `OptimizeCodegen` case. It's about one PR about maxBranchCases?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/13876
For this `OptimizeIn` example, we need `list.forall(_.deterministic) &&
(ExpressionSet(list) < conf_min_value || ExpressionSet(list) < list.size)`
would be one candidate for `lazy val`. It
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13876
**[Test build #61871 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61871/consoleFull)**
for PR 13876 at commit
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/13876
Why do we need the var?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/13876
No problem. I'll remove the trait, but we need still `var`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/13876
hm let's not generalize this yet. It is also possible that an expression is
optimizable by one rule but not optimizable by another rule.
I'm not trying to prevent every case from happening - I
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/13876
To show as an example, I added `trait Optimizable` having `optimizable`
flag.
This is similar with `foldable` or `nullable`. But, it's not a general
concept for all `Expression`, so I
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13876
**[Test build #61869 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61869/consoleFull)**
for PR 13876 at commit
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/13876
Ah, in order to optimize once, `list.forall(_.deterministic)` seems not
sufficient.
It passes many normal cases again and again. For `OptimizeIn` optimizer
case, we need `SQLConf`, too.
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/13876
Hi, @rxin .
Now, it's simplified for the all deterministic cases and passed the Jenkins
again.
Thank you for advice.
---
If your project is set up for it, you can reply to this email
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13876
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61860/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13876
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13876
**[Test build #61860 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61860/consoleFull)**
for PR 13876 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13876
**[Test build #61860 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61860/consoleFull)**
for PR 13876 at commit
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/13876
Hi, @rxin .
Could you review this PR again?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13876
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13876
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61760/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13876
**[Test build #61760 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61760/consoleFull)**
for PR 13876 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13876
**[Test build #61760 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61760/consoleFull)**
for PR 13876 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13876
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13876
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61662/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13876
**[Test build #61662 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61662/consoleFull)**
for PR 13876 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13876
**[Test build #61662 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61662/consoleFull)**
for PR 13876 at commit
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/13876
Rebased to the master.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/13876
Hi, @rxin .
Could you reivew this `OptimizeIn` again when you have some time?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13876
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61563/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13876
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13876
**[Test build #61563 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61563/consoleFull)**
for PR 13876 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13876
**[Test build #61563 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61563/consoleFull)**
for PR 13876 at commit
76 matches
Mail list logo
