[GitHub] [hadoop] steveloughran commented on issue #1160: HADOOP-16458 LocatedFileStatusFetcher.getFileStatuses failing intermittently with s3
steveloughran commented on issue #1160: HADOOP-16458 LocatedFileStatusFetcher.getFileStatuses failing intermittently with s3 URL: https://github.com/apache/hadoop/pull/1160#issuecomment-537135148 finally merged to trunk. Thx for review This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #1160: HADOOP-16458 LocatedFileStatusFetcher.getFileStatuses failing intermittently with s3
steveloughran commented on issue #1160: HADOOP-16458 LocatedFileStatusFetcher.getFileStatuses failing intermittently with s3 URL: https://github.com/apache/hadoop/pull/1160#issuecomment-531802859 Did a full test run `-Ds3guard -Ddynamodb -Dscale -Dauth`; all failures DDB table delete timeouts and the (known) prune one. All good otherwise. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #1160: HADOOP-16458 LocatedFileStatusFetcher.getFileStatuses failing intermittently with s3
steveloughran commented on issue #1160: HADOOP-16458
LocatedFileStatusFetcher.getFileStatuses failing intermittently with s3
URL: https://github.com/apache/hadoop/pull/1160#issuecomment-527513846
checkstyle is about an 81 char line where splitting it would make things
less readable
```
./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileInputFormat.java:255:
new InterruptedIOException("Interrupted while getting file
statuses"): Line is longer than 80 characters (found 81). [LineLength]
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #1160: HADOOP-16458 LocatedFileStatusFetcher.getFileStatuses failing intermittently with s3
steveloughran commented on issue #1160: HADOOP-16458 LocatedFileStatusFetcher.getFileStatuses failing intermittently with s3 URL: https://github.com/apache/hadoop/pull/1160#issuecomment-525782579 thanks. Think I've addressed your comments; pushing up for yetus then I'll commit. Tested: S3 ireland This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #1160: HADOOP-16458 LocatedFileStatusFetcher.getFileStatuses failing intermittently with s3
steveloughran commented on issue #1160: HADOOP-16458 LocatedFileStatusFetcher.getFileStatuses failing intermittently with s3 URL: https://github.com/apache/hadoop/pull/1160#issuecomment-516959483 Unless this test is confusing DDB so there's no cleanup, I'd point to those test failures less as a sign of a problem here and more that our test runs still seem somehow to cause confusion, even after the tombstone fixes. `ITestS3GuardDDBRootOperations` will save the results of its scans to somewhere under target/ which you can use to see where a discrepancy has crept in. Usually its something in s3 which isn't in s3guard: an OOB event of some kind This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #1160: HADOOP-16458 LocatedFileStatusFetcher.getFileStatuses failing intermittently with s3
steveloughran commented on issue #1160: HADOOP-16458 LocatedFileStatusFetcher.getFileStatuses failing intermittently with s3 URL: https://github.com/apache/hadoop/pull/1160#issuecomment-516865161 Updated PR *remove unused test suite * rebased to trunk * split the too-long test case in I`ITestRestrictedReadAccess` into separate pieces, using fields to propagate paths between them. It will still fail fast on the first assert failure, but as the operations and assertions move up the set of methods used in a LocatedFileStatusFetcher scan, that's OK. If an early assert fails, so will the final one * added some specific intercept wrappers to improve legibility. * javadocs No changes to production code. Tested, S3 Ireland. All well except for HADOOP-16477, which surface as I'd set the fs.s3a.encryption.key for the bucket. Unrelated This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on issue #1160: HADOOP-16458 LocatedFileStatusFetcher.getFileStatuses failing intermittently with s3
steveloughran commented on issue #1160: HADOOP-16458 LocatedFileStatusFetcher.getFileStatuses failing intermittently with s3 URL: https://github.com/apache/hadoop/pull/1160#issuecomment-516108978 tested: s3 ireland with `-Dparallel-tests -DtestsThreadCount=12 -Ds3guard -Ddynamodb -Dnonauth -Dscale` Also, check out [cloudstore Release: 2019-07-29](https://github.com/steveloughran/cloudstore/releases/tag/tag_2019-07-29_release), whose `locatefiles` API call invokes this codepath and lets you see what the debugging is up to, including printing out the statistics of operations called. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
