[GitHub] [hadoop] cbevard1 commented on pull request #5563: HADOOP-18706: Improve S3ABlockOutputStream recovery

2023-05-04 Thread via GitHub
cbevard1 commented on PR #5563: URL: https://github.com/apache/hadoop/pull/5563#issuecomment-1535323312 Shoot. That was in the last checkstyle, and I thought I bumped it out to 113. My IDE says it was indented to 113, but it wasn't. I should have run check style before committing. Anyways,

[GitHub] [hadoop] cbevard1 commented on pull request #5563: HADOOP-18706: Improve S3ABlockOutputStream recovery

2023-05-04 Thread via GitHub
cbevard1 commented on PR #5563: URL: https://github.com/apache/hadoop/pull/5563#issuecomment-1534755287 Done. Integration tests were run in us-east-2 with the following options: `-Dparallel-tests -DtestsThreadCount=16 -Dscale` -- This is an automated message from the Apache Git

[GitHub] [hadoop] cbevard1 commented on pull request #5563: HADOOP-18706: Improve S3ABlockOutputStream recovery

2023-05-03 Thread via GitHub
cbevard1 commented on PR #5563: URL: https://github.com/apache/hadoop/pull/5563#issuecomment-1533531643 Sorry about that. I won't rebase anymore. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hadoop] cbevard1 commented on pull request #5563: HADOOP-18706: Improve S3ABlockOutputStream recovery

2023-04-21 Thread via GitHub
cbevard1 commented on PR #5563: URL: https://github.com/apache/hadoop/pull/5563#issuecomment-1517915320 > minor: checkstyle unhappy about line length...please keep at 100 chars or less No problem, will do. > How about adding a new test case or modifying testRegularUpload() to

[GitHub] [hadoop] cbevard1 commented on pull request #5563: HADOOP-18706: Improve S3ABlockOutputStream recovery

2023-04-20 Thread via GitHub
cbevard1 commented on PR #5563: URL: https://github.com/apache/hadoop/pull/5563#issuecomment-1516625886 Nevermind. I was running verify from a new terminal window where JAVA_HOME was set to version 18. That was causing issues with the com.google.inject:guice dependency in

[GitHub] [hadoop] cbevard1 commented on pull request #5563: HADOOP-18706: Improve S3ABlockOutputStream recovery

2023-04-20 Thread via GitHub
cbevard1 commented on PR #5563: URL: https://github.com/apache/hadoop/pull/5563#issuecomment-1516391716 Correction: after rebasing, all of the tests in `org.apache.hadoop.fs.s3a.commit.terasort.ITestTerasortOnS3A` are failing with errors. It looks like these tests are also failing in

[GitHub] [hadoop] cbevard1 commented on pull request #5563: HADOOP-18706: Improve S3ABlockOutputStream recovery

2023-04-20 Thread via GitHub
cbevard1 commented on PR #5563: URL: https://github.com/apache/hadoop/pull/5563#issuecomment-1516289775 I did see intermittent failures once with one of the tests in `org.apache.hadoop.fs.contract.s3a.ITestS3AContractRootDir`. I can't remember which test it was, but when I ran just that

[GitHub] [hadoop] cbevard1 commented on pull request #5563: HADOOP-18706: Improve S3ABlockOutputStream recovery

2023-04-19 Thread via GitHub
cbevard1 commented on PR #5563: URL: https://github.com/apache/hadoop/pull/5563#issuecomment-1515127752 @steveloughran I'm running the integration tests in us-east-2 and the only option I'm setting is the parallel test and thread count (`mvn verify -Dparallel-tests -DtestsThreadCount=8`)

[GitHub] [hadoop] cbevard1 commented on pull request #5563: HADOOP-18706: Improve S3ABlockOutputStream recovery

2023-04-18 Thread via GitHub
cbevard1 commented on PR #5563: URL: https://github.com/apache/hadoop/pull/5563#issuecomment-1513699708 @steveloughran thanks for your feedback. I've added the span ID to the file name as you suggested for better debugging. > If you really want upload to be recoverable then you need