nicolas paris created HUDI-6350: ----------------------------------- Summary: AWS Hive sync: allow to enable/disable MDT on athena Key: HUDI-6350 URL: https://issues.apache.org/jira/browse/HUDI-6350 Project: Apache Hudi Issue Type: New Feature Reporter: nicolas paris
athena has a nice (but hidden) feature to leverage the hudi metadata table instead of listing files on s3. This in theorry reduce the s3 slow down trouble (too much listing), speeds-up query planning. THis can be easily achieved by adding table property: hudi.metadata-listing-enabled'='TRUE" While on athena v2, this feature really helps, on athena v3 at the time of writing this, something is going very wrong and the query can be x100 slower. see https://docs.aws.amazon.com/athena/latest/ug/querying-hudi.html -- This message was sent by Atlassian Jira (v8.20.10#820010)