Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22688
Thanks all!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/22688
Thank you, @HyukjinKwon and @cloud-fan and @viirya .
Merged to master.
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22688
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97246/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22688
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22688
**[Test build #97246 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97246/testReport)**
for PR 22688 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22688
**[Test build #97246 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97246/testReport)**
for PR 22688 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22688
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22688
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/22688
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/22688
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22688
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22688
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97239/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22688
**[Test build #97239 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97239/testReport)**
for PR 22688 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22688
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22688
**[Test build #97239 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97239/testReport)**
for PR 22688 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22688
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/22688
Retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/22688
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22688
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22688
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97238/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22688
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22688
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97234/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22688
**[Test build #97234 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97234/testReport)**
for PR 22688 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22688
**[Test build #97238 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97238/testReport)**
for PR 22688 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22688
**[Test build #97238 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97238/testReport)**
for PR 22688 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22688
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22688
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22688
**[Test build #97234 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97234/testReport)**
for PR 22688 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22688
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22688
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22688
I have no idea why it passes in my local. I fixed the test.
---
-
To unsubscribe, e-mail:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22688
Hm, yea, this was passed in my local so I expected this was flaky but seems
I should fix.
---
-
To unsubscribe, e-mail:
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/22688
Seems the same test failed?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22688
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22688
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97229/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22688
**[Test build #97229 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97229/testReport)**
for PR 22688 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22688
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22688
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22688
**[Test build #97229 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97229/testReport)**
for PR 22688 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22688
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22688
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22688
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97224/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22688
**[Test build #97224 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97224/testReport)**
for PR 22688 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22688
**[Test build #97224 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97224/testReport)**
for PR 22688 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22688
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22688
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/22688
Retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22688
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97207/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22688
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22688
**[Test build #97207 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97207/testReport)**
for PR 22688 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22688
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22688
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22688
**[Test build #97207 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97207/testReport)**
for PR 22688 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22688
+1 to deal with this as non blocker. I understand data source v2 is under
heavy development and unstable but strongly think we should backport .. it
breaks a basic operation ..
---
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22688
ah good point. I think the original design of append operator assumes the
table already exists, so a schema should be provided. If we treat file path as
a table, then append should fail for your
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22688
Another concern is, it doesn't sound to me straightforward that readsupport
is created and executed in write path.
So, at least, we should just read the schema when it's needed (when
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22688
Other datasources such as Parquet, ORC and JDBC won't be adoptable to the
current design since we can't read the schema from the information given the
code I provided above.
---
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22688
The point is not write only datasource, @cloud-fan. For instance,
`spark.read.range(1).format("source").write("non-existent-path")`. There's no
way to read the schema. There's no complaints so
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22688
Do we have users complain about it? In the new write API design, a data
source must provide a schema.
I don't think it's practical that people need a write-only data source
which can
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22688
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22688
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97196/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22688
**[Test build #97196 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97196/testReport)**
for PR 22688 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22688
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22688
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22688
cc @cloud-fan and @rdblue, this is more conservative but I would prefer to
revert it at https://github.com/apache/spark/pull/22686 rather then exposing
append mode in 2.4. I don't think it's a
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22688
**[Test build #97196 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97196/testReport)**
for PR 22688 at commit
66 matches
Mail list logo
