[GitHub] [hudi] tsolanki95 edited a comment on issue #1867: [SUPPORT] hudi is incurring emrfs eTag inconsistency issue with s3 and emrfs consistent view
tsolanki95 edited a comment on issue #1867: URL: https://github.com/apache/hudi/issues/1867#issuecomment-663688633 @luffyd We put in consistent view as a solution earlier, based on AWS support, to solve issues with using spark with S3 eventual consistency model causing duplicates in our data. We are now looking towards changing some of our datasets to utilize hudi but our compute resources still utilize EMRFS consistent view. As part of the transition, when some of our datasets utilize hudi and some do not, it would be good to be able to run spark with hudi on EMRFS consistent view. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] tsolanki95 edited a comment on issue #1867: [SUPPORT] hudi is incurring emrfs eTag inconsistency issue with s3 and emrfs consistent view
tsolanki95 edited a comment on issue #1867: URL: https://github.com/apache/hudi/issues/1867#issuecomment-663688633 @luffyd We put in consistent view as a solution earlier, based on AWS support, to solve issues with using spark with S3 eventual consistency model. We are now looking towards changing some of our datasets to utilize hudi but our compute resources still utilize EMRFS consistent view. As part of the transition, when some of our datasets utilize hudi and some do not, it would be good to be able to run spark with hudi on EMRFS consistent view. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
