Hello Syed, We're working on adding support for Hudi with Athena. Also, whenever we have support for Hudi with Athena - you should be able to use the Athena Data Connector for External Hive Metastore (https://docs.aws.amazon.com/athena/latest/ug/connect-to-data-source-hive.html).
If you can drop me an email ([email protected]), I would be happy to provide you with more details. On 2019/12/31 19:16:07, Vinoth Chandar <[email protected]> wrote: > Can one of the aws folks please chime in here? IIRC I saw some tweets > mentioning Hudi/Athena support is in the works. > Not sure myself. > > On Sun, Dec 29, 2019 at 11:33 PM Syed Abdul Kather <[email protected]> > wrote: > > > Hi Team, > > > > We have built the "CDC pipeline with apache hudi and debezium" . It > > works very well in our production. > > > > But we have inhouse Ambari Cluster with Hive metastore for all the ETL > > purpose and Athena for all analytics purposes. To make hudi table we work > > on the athena we have preserved only the latest version and create the > > table in parquet format . > > > > Right now hive metastore get update using hudi itself . But to keep the > > athena metastore in sync we have wrote a separate script to manage. But > > that looks like not right approach . As only the required the affected > > partition needs to be updated in athena side. > > > > Please suggest as right approach here . > > > > Thanks and Regards, > > S SYED ABDUL KATHER > > >
