[
https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808388#comment-16808388
]
vaquar khan commented on SPARK-21797:
-
{quote}Issue is related to AWS storage class not Apache Spark
[
https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16201917#comment-16201917
]
Steve Loughran commented on SPARK-21797:
Update, in HADOOP-14874 I've noted we could use the
[
https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146193#comment-16146193
]
Steve Loughran commented on SPARK-21797:
No> That's a shame. I only came across the option when I
[
https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145716#comment-16145716
]
Boris Clémençon commented on SPARK-21797:
--
FYI, the flag
[
https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141483#comment-16141483
]
Steve Loughran commented on SPARK-21797:
bq. According to our test, it is 20% slower maximum to
[
https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141358#comment-16141358
]
Boris Clémençon commented on SPARK-21797:
--
That's very good news indeed. Easiest way to fix! A
[
https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140408#comment-16140408
]
Steve Loughran commented on SPARK-21797:
This is happening deep the Amazon EMR team's closed
[
https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140261#comment-16140261
]
Boris Clémençon commented on SPARK-21797:
--
Steve,
This is the stacks:
{noformat}
WARN
[
https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140147#comment-16140147
]
Steve Loughran commented on SPARK-21797:
Note that if it is just during spark partition
[
https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140138#comment-16140138
]
Steve Loughran commented on SPARK-21797:
I was talking about the cost and time of getting data
[
https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139986#comment-16139986
]
Sean Owen commented on SPARK-21797:
---
Sure, but in all events, this is an operation that is fine with
[
https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139978#comment-16139978
]
Boris Clémençon commented on SPARK-21797:
--
Hi Steve,
to be sure we understand each other, *I
[
https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139919#comment-16139919
]
Steve Loughran commented on SPARK-21797:
If you are using S3// URLs then its the AWS team's
[
https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16136669#comment-16136669
]
Boris Clémençon commented on SPARK-21797:
--
OK, I see. I will have a look at the aws-sdk-java
[
https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16135236#comment-16135236
]
Sean Owen commented on SPARK-21797:
---
Sure, but this would not be a change in Spark, but in the AWS SDK
[
https://issues.apache.org/jira/browse/SPARK-21797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16135233#comment-16135233
]
Boris Clémençon commented on SPARK-21797:
--
Hi Sean,
Thanks for the quick answer. I understand
16 matches
Mail list logo
