Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/20386
Close this PR now. Resolve the problem with #20454.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20386
There is a lesson I learned from streaming data source v1: even it's
totally internal, there are people already using it and ask us to not remove
the API.
I think it's also true for the
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/20386
> I assume this API is necessary . . . it sounds reasonable to provide a
callback for task commit.
I agree it sounds reasonable, but we shouldn't add methods to a new API
blindly and
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/20386
@rdblue @cloud-fan @jose-torres thanks for the comments!
I was trying to make the API compatible with `onTaskCommit(taskCommit:
TaskCommitMessage)` in `FileCommitProtocol` possible.
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20386
`FileCommitProtocol.onTaskCommit` is called in `FileFormatWriter.write`, so
this PR is required to migrate file-based data sources.
By a quick look, it seems
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/20386
@gengliangwang, what is the use case supported by this? In other words, how
is `onTaskCommit(taskCommit: TaskCommitMessage)` currently used that requires
this change?
In general, I'm more
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/20386
@rdblue The target is 2.3 release. Thanks for your time!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20386
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86829/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20386
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20386
**[Test build #86829 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86829/testReport)**
for PR 20386 at commit
Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/20386
@cloud-fan, is the intent to get this into 2.3.0? If so, I'll make time to
review it today.
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20386
Build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20386
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86822/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20386
**[Test build #86822 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86822/testReport)**
for PR 20386 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20386
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20386
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86826/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20386
**[Test build #86826 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86826/testReport)**
for PR 20386 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20386
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86823/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20386
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20386
**[Test build #86823 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86823/testReport)**
for PR 20386 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20386
**[Test build #86829 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86829/testReport)**
for PR 20386 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20386
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/389/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20386
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20386
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20386
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/387/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20386
**[Test build #86826 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86826/testReport)**
for PR 20386 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20386
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/384/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20386
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20386
**[Test build #86823 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86823/testReport)**
for PR 20386 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20386
**[Test build #86822 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86822/testReport)**
for PR 20386 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20386
Build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20386
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/383/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20386
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20386
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86809/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20386
**[Test build #86809 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86809/testReport)**
for PR 20386 at commit
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20386
I like this change! It adds a missing feature which is required for
migrating the file-based data source(which use `FileCommitProtocol` and has a
callback for task commit), and also make it
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20386
**[Test build #86809 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86809/testReport)**
for PR 20386 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20386
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20386
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/373/
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/20386
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20386
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20386
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86801/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20386
**[Test build #86801 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86801/testReport)**
for PR 20386 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20386
**[Test build #86801 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86801/testReport)**
for PR 20386 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20386
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20386
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/365/
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20386
CC @rdblue @zsxwing @jose-torres @sameeragarwal
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20386
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86788/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20386
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20386
**[Test build #86788 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86788/testReport)**
for PR 20386 at commit
50 matches
Mail list logo
