[ https://issues.apache.org/jira/browse/HADOOP-14576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16236463#comment-16236463 ]
Aaron Fabbri commented on HADOOP-14576: --------------------------------------- I've seen similar issues intermittently just running S3A integration tests. I will paste a fresh stack trace next time I hit it. For now, sharing another stack trace our Hive folks provided a while back. Caused by: org.apache.hadoop.fs.s3a.AWSServiceIOException: get on s3a:///1000_unpartitioned_parquet_cdh_ip-10-0-0-158/catalog_sales/.hive-staging_hive_2017-06-21_00-27-07_895_5004400669443203568-24/_tmp.-ext-10000/000530_0: com.amazonaws.services.dynamodbv2.model.ResourceNotFoundException: Requested resource not found (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ResourceNotFoundException; Request ID: L93OFC0JOT2N6FT45BIPRQVP1FVV4KQNSO5AEMVJF66Q9ASUAAJG): Requested resource not found (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ResourceNotFoundException; Request ID: L93OFC0JOT2N6FT45BIPRQVP1FVV4KQNSO5AEMVJF66Q9ASUAAJG) at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:178) at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:101) at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.get(DynamoDBMetadataStore.java:395) at org.apache.hadoop.fs.s3a.S3AFileSystem.innerGetFileStatus(S3AFileSystem.java:1775) at org.apache.hadoop.fs.s3a.S3AFileSystem.innerRename(S3AFileSystem.java:776) at org.apache.hadoop.fs.s3a.S3AFileSystem.rename(S3AFileSystem.java:734) at org.apache.hadoop.hive.ql.util.ParallelDirectoryRenamer$1.call(ParallelDirectoryRenamer.java:105) at org.apache.hadoop.hive.ql.util.ParallelDirectoryRenamer$1.call(ParallelDirectoryRenamer.java:101) ... 4 more Caused by: com.amazonaws.services.dynamodbv2.model.ResourceNotFoundException: Requested resource not found (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ResourceNotFoundException; Request ID: L93OF...snip...) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1588) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1258) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1030) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:742) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:716) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667) > s3guard DynamoDB resource not found: tables not ACTIVE state after initial > connection > ------------------------------------------------------------------------------------- > > Key: HADOOP-14576 > URL: https://issues.apache.org/jira/browse/HADOOP-14576 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 > Affects Versions: 3.0.0-beta1 > Reporter: Sean Mackrory > Priority: Major > > We currently only anticipate tables not being in the ACTIVE state when first > connecting. It is possible for a table to be in the ACTIVE state and move to > an UPDATING state during partitioning events. Attempts to read or write > during that time will result in an AmazonServerException getting thrown. We > should try to handle that better... -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org