Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22119
Close this one and open #22133
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22119
@tgravescs @dongjoon-hyun Thanks for the explanation. We should add a
configuration instead of reverting.
---
-
To unsubsc
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/22119
+1 for @tgravescs 's comments. In terms of usability, the mapping and
configuration will be easier for the most customers.
For the following @gengliangwang 's comment, technically ther
Github user tgravescs commented on the issue:
https://github.com/apache/spark/pull/22119
How do users manually migrate and keep compatibility?That is the
problem I have, I am all for reverting, if we have an easy way for users to
migrate to the internal one.
Note that
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22119
But it seems that creating a configuration makes everyone happy...
I will wait for another day to get more thoughts.
---
--
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22119
@tgravescs I saw your comments. Just feel that we can make it simpler by
reverting it.
For hive tables that used Databricks spark-avro, the tables can still using
the Databricks repo(sinc
Github user tgravescs commented on the issue:
https://github.com/apache/spark/pull/22119
Sorry I'm a bit confused by what is going on here. It looks like you just
reverted the change. I thought we were simply adding a config so its
configurable as to whether its in the mapping table
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22119
I am not sure how useful the configuration for AVRO is.
For the hive table example @dongjoon-hyun mentioned in
https://issues.apache.org/jira/browse/SPARK-24924?focusedCommentId=165707
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22119
This particular inconsistency could confuse users because CSV's one has
existed for a long time. I think configuration makes this safer since both
sides make sense I believe.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22119
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22119
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94841/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22119
**[Test build #94841 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94841/testReport)**
for PR 22119 at commit
[`656790e`](https://github.com/apache/spark/commit/6
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22119
CSV is loaded by default, while AVRO is not. So having a backward
compatibility mapping in CSV only still makes sense.
Let's remove the mapping for CSV in 3.0.
---
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/22119
For details, see the discussion in the JIRA
https://issues.apache.org/jira/browse/SPARK-24924
---
-
To unsubscribe, e-mail: r
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/22119
If we all agree this databricks mapping is not reasonable, I think it's ok
to have this inconsistency and remove the mapping for CSV in 3.0.
It's weird to make the same mistake just to mak
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22119
Sorry if I missed some comments somewhere but just for clarification,
should we do it for CSV in 3.0.0? Inconsistency should also be taken into
account. Actually configuration sounds making mor
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22119
**[Test build #94841 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94841/testReport)**
for PR 22119 at commit
[`656790e`](https://github.com/apache/spark/commit/65
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22119
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional comma
Github user gengliangwang commented on the issue:
https://github.com/apache/spark/pull/22119
@tgravescs @dongjoon-hyun @HyukjinKwon @cloud-fan
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For a
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22119
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/2239/
20 matches
Mail list logo
