Github user icexelloss commented on the issue:
https://github.com/apache/spark/pull/20295
@HyukjinKwon Thanks for the comment. I will continue with the current
approach unless objection raises. I will work on comments and refinements in
the next day or two.
---
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20295
For https://github.com/apache/spark/pull/20295#issuecomment-360297123, I am
fine without new serialization protocol actually. I didn't have a strong
preference there because I wasn't sure if
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20295
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86651/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20295
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20295
**[Test build #86651 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86651/testReport)**
for PR 20295 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20295
**[Test build #86651 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86651/testReport)**
for PR 20295 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20295
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/241/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20295
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20295
Build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20295
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86604/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20295
**[Test build #86604 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86604/testReport)**
for PR 20295 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20295
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86606/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20295
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20295
**[Test build #86606 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86606/testReport)**
for PR 20295 at commit
Github user icexelloss commented on the issue:
https://github.com/apache/spark/pull/20295
Hi all,
I did some digging and I think adding a serialization form that serialize a
key object along with a Arrow record batch is quite complicated because we are
using
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20295
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/206/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20295
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20295
**[Test build #86606 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86606/testReport)**
for PR 20295 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20295
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/205/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20295
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20295
**[Test build #86604 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86604/testReport)**
for PR 20295 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20295
Build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20295
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/203/
Github user icexelloss commented on the issue:
https://github.com/apache/spark/pull/20295
Let me experiment with new serialization approach. Will update here.
---
-
To unsubscribe, e-mail:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20295
To me, seems roughly fine.
> Alternatively, we can implement a new serialization protocol for
GROUP_MAP eval type, i.e, instead of sending an arrow batch, we could send a
group row and
Github user icexelloss commented on the issue:
https://github.com/apache/spark/pull/20295
Yep, that's correct.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20295
How do we turn a single group column to a series? just repeat the group
column?
---
-
To unsubscribe, e-mail:
Github user icexelloss commented on the issue:
https://github.com/apache/spark/pull/20295
@cloud-fan Currently I sent group columns along with the extra data column.
For example, if the original DataFrame has `id, v` and group column is `id`,
the current implementation in this PR
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20295
How are you going to send the group columns? For a group we have only one
group row and a bunch of data rows.
---
-
To
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20295
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20295
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86290/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20295
**[Test build #86290 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86290/testReport)**
for PR 20295 at commit
Github user icexelloss commented on the issue:
https://github.com/apache/spark/pull/20295
cc @ueshin @HyukjinKwon @cloud-fan @viirya
This PR implements discussion here
https://github.com/apache/spark/pull/20211#pullrequestreview-87657832. There
are more refinement needs to
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20295
**[Test build #86290 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86290/testReport)**
for PR 20295 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20295
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86286/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20295
**[Test build #86286 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86286/testReport)**
for PR 20295 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20295
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20295
**[Test build #86286 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86286/testReport)**
for PR 20295 at commit
38 matches
Mail list logo
