steveloughran commented on issue #1442: HADOOP-16570. S3A committers encounter
scale issues
URL: https://github.com/apache/hadoop/pull/1442#issuecomment-538499572
ok, merged in. thank you for the reviews.
This is an automate
steveloughran commented on issue #1442: HADOOP-16570. S3A committers encounter
scale issues
URL: https://github.com/apache/hadoop/pull/1442#issuecomment-538461283
Also filed: https://issues.apache.org/jira/browse/HADOOP-16632
The failed assertion was caused by a speculative task writ
steveloughran commented on issue #1442: HADOOP-16570. S3A committers encounter
scale issues
URL: https://github.com/apache/hadoop/pull/1442#issuecomment-538430235
rebasing against trunk HADOOP-16207 I'm getting some failures which I'm
blaming on test setup code, such as the fact that temp
steveloughran commented on issue #1442: HADOOP-16570. S3A committers encounter
scale issues
URL: https://github.com/apache/hadoop/pull/1442#issuecomment-537213459
just pushed up a rebased update; tries to address all of Ryans comments,
plus I use Tasks to parallelise the partition deletion
steveloughran commented on issue #1442: HADOOP-16570. S3A committers encounter
scale issues
URL: https://github.com/apache/hadoop/pull/1442#issuecomment-537138482
> The only problem I saw was loading all of the committed files for the
_SUCCESS marker.
let me review that. thanks for
steveloughran commented on issue #1442: HADOOP-16570. S3A committers encounter
scale issues
URL: https://github.com/apache/hadoop/pull/1442#issuecomment-536533462
checkstyle warnings
```
./hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/commit/files/SuccessData.java:5
steveloughran commented on issue #1442: HADOOP-16570. S3A committers encounter
scale issues
URL: https://github.com/apache/hadoop/pull/1442#issuecomment-535658748
latest test run -s3 ireland. There's a new unit test which with the current
values takes 1 min; plan to cut the numbers back, j
