[ https://issues.apache.org/jira/browse/HADOOP-15837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16645031#comment-16645031 ]
Steve Loughran commented on HADOOP-15837: ----------------------------------------- Patch 002; fix checkstyle. Tested S3 ireland with " -Ds3guard -Ddynamodb"; failures in unrelated issues as discussed and covered elsewhere * this patch is ready for review * I plan to backport to 2.10-3.9 the change to the case statement as that addresses the key issue of capacity change while S3Guard is in use, which autoscale will implicitly do in high-load situations (i.e. use S3Guard heavily, DDB triggers scale up, S3Guard fails) + [~mackrorysd] : can you look @ this. It's a serious issue which will surface in the wild Errors {code} [ERROR] Errors: [ERROR] ITestS3GuardConcurrentOps.testConcurrentTableCreations:166->deleteTable:77 » ... [ERROR] ITestS3GuardToolDynamoDB.testDynamoDBInitDestroyCycle:315 » ResourceInUse Atte... [ERROR] ITestS3GuardToolDynamoDB>AbstractS3GuardToolTestBase.testSetCapacityFailFastIfNotGuarded:330->AbstractS3GuardToolTestBase.lambda$testSetCapacityFailFastIfNotGuarded$2:331->AbstractS3GuardToolTestBase.run:115 » FileNotFound {code} > DynamoDB table Update can fail S3A FS init > ------------------------------------------ > > Key: HADOOP-15837 > URL: https://issues.apache.org/jira/browse/HADOOP-15837 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 > Affects Versions: 3.2.0 > Environment: s3guard test with small capacity (10) but autoscale > enabled & multiple consecutive parallel test runs executed...this seems to > have been enough load to trigger the state change > Reporter: Steve Loughran > Assignee: Steve Loughran > Priority: Major > Attachments: HADOOP-15837-001.patch, HADOOP-15837-002.patch > > > When DDB autoscales a table, it goes into an UPDATING state. The > waitForTableActive operation in the AWS SDK doesn't seem to wait long enough > for this to recover. We need to catch & retry -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org