[jira] [Commented] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2017-09-08 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16158488#comment-16158488 ] Hudson commented on HADOOP-13421: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12822 (See

[jira] [Commented] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2017-09-07 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16157073#comment-16157073 ] Steve Loughran commented on HADOOP-13421: - oh, one thing: docs? > Switch to v2 of the S3 List

[jira] [Commented] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2017-09-07 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16157072#comment-16157072 ] Steve Loughran commented on HADOOP-13421: - LGTM +1 > Switch to v2 of the S3 List Objects API

[jira] [Commented] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2017-09-07 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16156974#comment-16156974 ] Steve Loughran commented on HADOOP-13421: - I'd say "fall back to the default version". Why? it

[jira] [Commented] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2017-09-06 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16155746#comment-16155746 ] Aaron Fabbri commented on HADOOP-13421: --- {quote} better fs.s3a.list.version="1" line us up for a

[jira] [Commented] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2017-09-06 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16155261#comment-16155261 ] Steve Loughran commented on HADOOP-13421: - {code} fs.s3a.use.list.v1 {code} better {code}

[jira] [Commented] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2017-09-05 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154759#comment-16154759 ] Hadoop QA commented on HADOOP-13421: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2017-09-05 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154687#comment-16154687 ] Aaron Fabbri commented on HADOOP-13421: --- [~ste...@apache.org] thanks, I was thinking the same

[jira] [Commented] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2017-09-05 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153342#comment-16153342 ] Steve Loughran commented on HADOOP-13421: - w.r.t regression testing, you could subclass some

[jira] [Commented] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2017-09-04 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16152669#comment-16152669 ] Steve Loughran commented on HADOOP-13421: - Nothing wrong with getting things utterly

[jira] [Commented] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2017-09-01 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151302#comment-16151302 ] Aaron Fabbri commented on HADOOP-13421: --- Whoever added the forced list response paging to

[jira] [Commented] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2017-08-31 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16149268#comment-16149268 ] Aaron Fabbri commented on HADOOP-13421: --- Agreed. > Switch to v2 of the S3 List Objects API in S3A

[jira] [Commented] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2017-08-31 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148808#comment-16148808 ] Steve Loughran commented on HADOOP-13421: - OK, let's get the 13345 merge rounded off & in & then

[jira] [Commented] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2017-08-30 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148373#comment-16148373 ] Aaron Fabbri commented on HADOOP-13421: --- Got this working, with v1 compatibility config knob (off

[jira] [Commented] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2017-08-29 Thread Aaron Fabbri (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146398#comment-16146398 ] Aaron Fabbri commented on HADOOP-13421: --- I'd be happy to work on this.. If anyone else has a patch

[jira] [Commented] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2017-08-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145096#comment-16145096 ] Steve Loughran commented on HADOOP-13421: - Anyone got a patch for this we could get done in time

[jira] [Commented] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2016-11-29 Thread Thomas Demoor (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15705631#comment-15705631 ] Thomas Demoor commented on HADOOP-13421: Thanks for reaching out [~ste...@apache.org]

[jira] [Commented] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2016-11-25 Thread Pieter Reuse (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15695725#comment-15695725 ] Pieter Reuse commented on HADOOP-13421: --- Thank you for putting this on my radar,

[jira] [Commented] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2016-11-24 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15693398#comment-15693398 ] Steve Loughran commented on HADOOP-13421: - Listing v2 will also improve LIST availability for

[jira] [Commented] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2016-11-22 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15688049#comment-15688049 ] Steve Loughran commented on HADOOP-13421: - We may need to make this optional, to support other

[jira] [Commented] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2016-07-26 Thread Steven K. Wong (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15394405#comment-15394405 ] Steven K. Wong commented on HADOOP-13421: - Listing 100 keys, time taken is 15% shorter when

[jira] [Commented] (HADOOP-13421) Switch to v2 of the S3 List Objects API in S3A

2016-07-26 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15393853#comment-15393853 ] Steve Loughran commented on HADOOP-13421: - this could be useful. How tangible are the speedups