Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/14691
Based on the JIRA discussion and then the resulting mailing list discussion
I'm closing this PR while we investigate options for an improved Sink API in
general.
---
If your project is set up for
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/14691
Agreed. Let's continue to discuss on [JIRA](SPARK-16407) for more
visibility.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user jodersky commented on the issue:
https://github.com/apache/spark/pull/14691
I totally agree! This PR LGTM, but I think a discussion on enabling truly
pluggable sources through providers will require more discussion on a more
visible channel. @zsxwing, what do you think?
Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/14691
It also lets you do things with dynamically created sink providers with
custom functions which the string based API really doesn't have good support
for (beside the string based param API for config
Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/14691
@jodersky I <3 more compile time type safety (although to keep it similar
to the regular write API I think we need to provide both APIs).
---
If your project is set up for it, you can reply to this
Github user jodersky commented on the issue:
https://github.com/apache/spark/pull/14691
I like the idea! This is might not be the best place to start a discussion,
but I reckon that the sink provider api could also eventually be used to
provision builtin sinks. It would make the curre
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14691
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14691
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65118/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14691
**[Test build #65118 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65118/consoleFull)**
for PR 14691 at commit
[`7bb643c`](https://github.com/apache/spark/commit/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14691
**[Test build #65118 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65118/consoleFull)**
for PR 14691 at commit
[`7bb643c`](https://github.com/apache/spark/commit/7
Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/14691
@zsxwing I'm not sure I understand - the current API allows users to
specify the class name and uses reflection to create it - this allows users to
specify a specific instance of the class but the t
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/14691
In general, we don't want to use any source or sink classes to an API of
DataStreamWriter because they are not as stable as DataStreamWriter. This is
similar to DataFrameWriter.
---
If your projec
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14691
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64568/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14691
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14691
**[Test build #64568 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64568/consoleFull)**
for PR 14691 at commit
[`c7bbffc`](https://github.com/apache/spark/commit/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14691
**[Test build #64568 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64568/consoleFull)**
for PR 14691 at commit
[`c7bbffc`](https://github.com/apache/spark/commit/c
Github user shaneknapp commented on the issue:
https://github.com/apache/spark/pull/14691
jenkins, test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wi
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14691
**[Test build #64551 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64551/consoleFull)**
for PR 14691 at commit
[`c7bbffc`](https://github.com/apache/spark/commit/c
Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/14691
cc @tdas let me know your thoughts on exposing this API to the users - I
think the use case is pretty clear but we could expose it in another way if you
don't like this API.
---
If your project is
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14691
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14691
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63934/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14691
**[Test build #63934 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63934/consoleFull)**
for PR 14691 at commit
[`70e663c`](https://github.com/apache/spark/commit/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/14691
**[Test build #63934 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63934/consoleFull)**
for PR 14691 at commit
[`70e663c`](https://github.com/apache/spark/commit/7
23 matches
Mail list logo
