[ https://issues.apache.org/jira/browse/HADOOP-16667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16957018#comment-16957018 ]
Steve Loughran commented on HADOOP-16667: ----------------------------------------- {code} [ERROR] testConcurrentTableCreations(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardConcurrentOps) Time elapsed: 131.092 s <<< FAILURE! java.lang.AssertionError: 16/16 threads threw exceptions while initializing on iteration 0 at org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardConcurrentOps.testConcurrentTableCreations(ITestS3GuardConcurrentOps.java:175) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.hadoop.fs.s3a.AWSServiceThrottledException: initTable on stevel-test-table-deleteme.testConcurrentTableCreations1192961737: com.amazonaws.services.dynamodbv2.model.LimitExceededException: Subscriber limit exceeded: There is a limit of 256 tables per subscriber (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: LimitExceededException; Request ID: D7114912B7V15ALBCFQSESHKTJVV4KQNSO5AEMVJF66Q9ASUAAJG): Subscriber limit exceeded: There is a limit of 256 tables per subscriber (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: LimitExceededException; Request ID: D7114912B7V15ALBCFQSESHKTJVV4KQNSO5AEMVJF66Q9ASUAAJG) at org.apache.hadoop.fs.s3a.S3AUtils.translateDynamoDBException(S3AUtils.java:422) at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:207) at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStoreTableManager.initTable(DynamoDBMetadataStoreTableManager.java:225) at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.initialize(DynamoDBMetadataStore.java:513) at org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardConcurrentOps$2.call(ITestS3GuardConcurrentOps.java:148) at org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardConcurrentOps$2.call(ITestS3GuardConcurrentOps.java:139) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ... 1 more Caused by: com.amazonaws.services.dynamodbv2.model.LimitExceededException: Subscriber limit exceeded: There is a limit of 256 tables per subscriber (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: LimitExceededException; Request ID: D7114912B7V15ALBCFQSESHKTJVV4KQNSO5AEMVJF66Q9ASUAAJG) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1712) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1367) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1113) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:770) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:744) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:726) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:686) at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:668) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:532) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:512) at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.doInvoke(AmazonDynamoDBClient.java:4279) at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.invoke(AmazonDynamoDBClient.java:4246) at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.executeCreateTable(AmazonDynamoDBClient.java:1059) at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.createTable(AmazonDynamoDBClient.java:1025) at com.amazonaws.services.dynamodbv2.document.DynamoDB.createTable(DynamoDB.java:96) at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStoreTableManager.createTable(DynamoDBMetadataStoreTableManager.java:309) at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStoreTableManager.initTable(DynamoDBMetadataStoreTableManager.java:214) ... 7 more [INFO] Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 59.996 s - in org.apache.hadoop.fs.s3a.ITestS3AContractGetFileStatusV1List [INFO] Tests run: 50, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 120.666 s - in org.apache.hadoop.fs.s3a.ITestS3AFSMainOperations [INFO] Tests run: 45, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 111.175 s - in org.apache.hadoop.fs.s3a.select.ITestS3Select [INFO] Tests run: 153, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 444.552 s - in org.apache.hadoop.fs.s3a.ITestS3ARemoteFileChanged [INFO] Tests run: 24, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 551.98 s - in org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations [INFO] [INFO] Results: [INFO] [ERROR] Failures: [ERROR] ITestS3GuardConcurrentOps.testConcurrentTableCreations:175 16/16 threads threw exceptions while initializing on iteration 0 [ERROR] Errors: [ERROR] ITestS3GuardToolDynamoDB.testDynamoDBInitDestroyCycle:201->AbstractS3GuardToolTestBase.expectSuccess:116 » AWSServiceThrottled [ERROR] ITestS3GuardToolDynamoDB.testDynamoTableTagging:153 » AWSServiceThrottled init... [INFO] [ERROR] Tests run: 1088, Failures: 1, Errors: 2, Skipped: 74 [INFO] {code} > s3guard LimitExceededException -too many tables > ----------------------------------------------- > > Key: HADOOP-16667 > URL: https://issues.apache.org/jira/browse/HADOOP-16667 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3, test > Reporter: Steve Loughran > Priority: Major > > Failure in {{ITestS3GuardConcurrentOps.testConcurrentTableCreations}} > {code} > Caused by: com.amazonaws.services.dynamodbv2.model.LimitExceededException: > Subscriber limit exceeded: > There is a limit of 256 tables per subscriber (Service: AmazonDynamoDBv2; > Status Code: 400; Error Code: LimitExceededException; Request ID: > RLKFNENN2AG5ML87U3HIGLU94NVV4KQNSO5AEMVJF66Q9ASUAAJG) > {code} > Probably caused by table delete leakage. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org