Github user liancheng commented on the issue:
https://github.com/apache/spark/pull/15703
Thanks everyone for the review!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/15703
LGTM. Merging to master.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
Github user yhuai commented on the issue:
https://github.com/apache/spark/pull/15703
Code changes looks good to me. Let's also do a benchmark to sanity check
our implementation.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15703
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15703
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68734/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15703
**[Test build #68734 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68734/consoleFull)**
for PR 15703 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15703
**[Test build #68734 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68734/consoleFull)**
for PR 15703 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15703
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15703
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68493/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15703
**[Test build #68493 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68493/consoleFull)**
for PR 15703 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15703
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68491/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15703
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15703
**[Test build #68491 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68491/consoleFull)**
for PR 15703 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15703
**[Test build #68493 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68493/consoleFull)**
for PR 15703 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15703
**[Test build #68491 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68491/consoleFull)**
for PR 15703 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15703
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68488/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15703
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15703
**[Test build #68488 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68488/consoleFull)**
for PR 15703 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15703
**[Test build #68488 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68488/consoleFull)**
for PR 15703 at commit
Github user liancheng commented on the issue:
https://github.com/apache/spark/pull/15703
The last build failure was because of a logical conflict between this PR
and the master branch. Resolving it.
---
If your project is set up for it, you can reply to this email and have your
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15703
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/68433/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15703
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15703
**[Test build #68433 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68433/consoleFull)**
for PR 15703 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15703
**[Test build #68433 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68433/consoleFull)**
for PR 15703 at commit
Github user liancheng commented on the issue:
https://github.com/apache/spark/pull/15703
OK, now it's ready for review and merge.
cc @yhuai @JoshRosen @cloud-fan
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15703
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67938/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15703
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15703
**[Test build #67938 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67938/consoleFull)**
for PR 15703 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15703
**[Test build #67938 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67938/consoleFull)**
for PR 15703 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15703
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67930/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15703
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15703
**[Test build #67930 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67930/consoleFull)**
for PR 15703 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15703
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67927/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15703
**[Test build #67927 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67927/consoleFull)**
for PR 15703 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15703
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15703
**[Test build #67930 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67930/consoleFull)**
for PR 15703 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15703
**[Test build #67927 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67927/consoleFull)**
for PR 15703 at commit
Github user liancheng commented on the issue:
https://github.com/apache/spark/pull/15703
It turned out that I didn't initialize Hive UDAF evaluators properly.
Quoted from commit message of my previous commit:
> Hive UDAFs are sensitive to aggregation mode, and must be
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15703
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67910/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15703
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15703
**[Test build #67910 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67910/consoleFull)**
for PR 15703 at commit
Github user tejasapatil commented on the issue:
https://github.com/apache/spark/pull/15703
There is no doubt that this PR is not going to make things better. I'm
already sold :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user liancheng commented on the issue:
https://github.com/apache/spark/pull/15703
@tejasapatil Another point that I'd like to add is that even if the
performance for a single UDAF like `GenericUDAFCollectList` regresses, you
still have performance gains if such UDAFs are used
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15703
**[Test build #67910 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67910/consoleFull)**
for PR 15703 at commit
Github user liancheng commented on the issue:
https://github.com/apache/spark/pull/15703
I found that I'm handling bridged UDAFs properly, which caused a few test
failures. Working on it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user liancheng commented on the issue:
https://github.com/apache/spark/pull/15703
@tejasapatil For `collect_set` and `collect_list`, we'll simply migrate
them to `TypedImperativeAggregate` and so that they become Spark native
aggregate functions. We can also handle other
Github user tejasapatil commented on the issue:
https://github.com/apache/spark/pull/15703
This will surely improve performance for the UDAFs where the data shrinks
(eg. `max` as you pointed out). I am not sure if it would be better for UDAFs
like `GenericUDAFCollectSet`,
Github user liancheng commented on the issue:
https://github.com/apache/spark/pull/15703
I can't reproduce those test failures when executing failed test cases
individually. Seems that it's related to execution order. Still investigating.
---
If your project is set up for it, you
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/15703
cc @tejasapatil
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15703
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67844/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15703
**[Test build #67844 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67844/consoleFull)**
for PR 15703 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15703
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15703
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15703
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67842/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15703
**[Test build #67842 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67842/consoleFull)**
for PR 15703 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15703
**[Test build #67844 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67844/consoleFull)**
for PR 15703 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15703
**[Test build #67842 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67842/consoleFull)**
for PR 15703 at commit
Github user liancheng commented on the issue:
https://github.com/apache/spark/pull/15703
Will add more details in the PR description soon.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
58 matches
Mail list logo
