[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17303213#comment-17303213
]
Yu Xiang commented on SPARK-24924:
--
[~tgraves], [~Gengliang.Wang] [~dongjoon],
Hi, I am struggling
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16583868#comment-16583868
]
Dongjoon Hyun commented on SPARK-24924:
---
Hi, All.
I created SPARK-25143 as a more general and
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16583483#comment-16583483
]
Gengliang Wang commented on SPARK-24924:
[~dongjoon] I see. I am now +1 with adding new
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582948#comment-16582948
]
Dongjoon Hyun commented on SPARK-24924:
---
[~Gengliang.Wang] . Ur, the latest consensus isn't
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16581929#comment-16581929
]
Gengliang Wang commented on SPARK-24924:
As package "org.apache.spark.sql.avro" is external
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16581608#comment-16581608
]
Thomas Graves commented on SPARK-24924:
---
I'd be ok with that but CSV has been that way already for
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16581546#comment-16581546
]
Dongjoon Hyun commented on SPARK-24924:
---
[~tgraves] . In that case, for consistency, we had better
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16581117#comment-16581117
]
Thomas Graves commented on SPARK-24924:
---
[~cloud_fan] [~hyukjin.kwon] seems no one else has a
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573221#comment-16573221
]
Thomas Graves commented on SPARK-24924:
---
| There was a discussion about why we shouldn't support
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16572564#comment-16572564
]
Hyukjin Kwon commented on SPARK-24924:
--
[~cloud_fan], Yea, adding them as implicit sounds not a
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16572556#comment-16572556
]
Wenchen Fan commented on SPARK-24924:
-
> I assume we could theoretically also support the
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16571908#comment-16571908
]
Thomas Graves commented on SPARK-24924:
---
so originally when I started on this I didn't know about
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16571852#comment-16571852
]
Thomas Graves commented on SPARK-24924:
---
thanks, I missed it in the output for spark as I was just
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570880#comment-16570880
]
Dongjoon Hyun commented on SPARK-24924:
---
Yep. It will work if those 3rd-party packages are rebuilt
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570840#comment-16570840
]
Thomas Graves commented on SPARK-24924:
---
so officially the spark api compatibility is only at the
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570756#comment-16570756
]
Dongjoon Hyun commented on SPARK-24924:
---
1. Theoretically, Spark 2.4 should handle both Hive
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570736#comment-16570736
]
Thomas Graves commented on SPARK-24924:
---
so if the user includes the databricks jar and they
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570702#comment-16570702
]
Dongjoon Hyun commented on SPARK-24924:
---
For Hive tables, the format name is stored as a table
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570638#comment-16570638
]
Thomas Graves commented on SPARK-24924:
---
So something I just thought of that I want to clarify, is
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570617#comment-16570617
]
Dongjoon Hyun commented on SPARK-24924:
---
Thank you for confirming and giving the right direction
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570586#comment-16570586
]
Thomas Graves commented on SPARK-24924:
---
For compatibility we can't remove it unless major
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570562#comment-16570562
]
Dongjoon Hyun commented on SPARK-24924:
---
Sorry for the late responses, [~tgraves] and guys. I was
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570220#comment-16570220
]
Thomas Graves commented on SPARK-24924:
---
{quote}I have followed the changes in Avro and I don't
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569109#comment-16569109
]
Felix Cheung commented on SPARK-24924:
--
I tend to agree that we shouldn't "magically" remap
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569104#comment-16569104
]
Hyukjin Kwon commented on SPARK-24924:
--
For fully qualifed path, we already could specify like
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569085#comment-16569085
]
Wenchen Fan commented on SPARK-24924:
-
when the short name conflicts, I feel it's better to pick the
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569072#comment-16569072
]
Hyukjin Kwon commented on SPARK-24924:
--
Also, for clarification, we already issue warnings:
{code}
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16569071#comment-16569071
]
Hyukjin Kwon commented on SPARK-24924:
--
If it already throws an error for CSV case too, I would
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16568454#comment-16568454
]
Reynold Xin commented on SPARK-24924:
-
I like the improved error message (I didn't read the earlier
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16568393#comment-16568393
]
Thomas Graves commented on SPARK-24924:
---
| It wouldn't be very different for 2.4.0. It could be
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16568352#comment-16568352
]
Hyukjin Kwon commented on SPARK-24924:
--
cc [~cloud_fan] since we talked about this for CSV, and
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16568348#comment-16568348
]
Hyukjin Kwon commented on SPARK-24924:
--
{quote}
but at the same time we aren't adding the
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16568204#comment-16568204
]
Thomas Graves commented on SPARK-24924:
---
[~felixcheung] did your discussion on the same thing with
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16568199#comment-16568199
]
Thomas Graves commented on SPARK-24924:
---
Hmm, so we are adding this for ease of upgrading I guess
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567708#comment-16567708
]
Hyukjin Kwon commented on SPARK-24924:
--
Similar discussion was made in SPARK-20590 when we port
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16567045#comment-16567045
]
Thomas Graves commented on SPARK-24924:
---
why are we doing this? If a user ships the spark-avro
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560624#comment-16560624
]
Apache Spark commented on SPARK-24924:
--
User 'gatorsmile' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16556427#comment-16556427
]
Apache Spark commented on SPARK-24924:
--
User 'dongjoon-hyun' has created a pull request for this
38 matches
Mail list logo
