tsolanki95 opened a new issue #1867: URL: https://github.com/apache/hudi/issues/1867
Received the following error using the default installation of Hudi in EMR 5.29.0 (Hudi version 5.0.0): `RetryInvocationHandler: Exception while invoking ConsistencyCheckerS3FileSystem.open over numm. Retrying after sleeping for 35000ms. com.amazon.ws.emr.hadoop.fs.consistency.exception.ConsistencyException: eTag in metadata for File '<s3 path>/.hoodie_partition_metadata' does not match eTag from S3!` This is typically happening due to eTag verification in emrfs consistent view, which verifies that for a file on s3, we are using the latest version of the file (based on the eTag stored in dynamoDB table. We posed this question on [stack overflow](https://stackoverflow.com/questions/63052142/error-while-emrfs-consistency-view-enabled-along-with-hudi) and saw someone commented that this happens when you are writing files without using emrfs, but rather with standard AWS-SDK. Is current hudi implementation working on emrfs consistent view (a solution we put in earlier to overcome S3 eventual consistency issues in spark)? If so, do we need to disable `fs.s3.consistent.metadata.etag.verification.enabled`? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
