Great !!!. We are heavy user of Athena. Right now make it work with hudi we
have done workaround (maintaining only the latest version of the record.
Thanks, Rahul for your updates.
Thanks and Regards,
S SYED ABDUL KATHER
On Thu, Jan 9, 2020 at 3:39 AM Rahul Bhartia <[email protected]>
wrote:
> Hello Syed,
>
> We're working on adding support for Hudi with Athena. Also, whenever we
> have support for Hudi with Athena - you should be able to use the Athena
> Data Connector for External Hive Metastore (
> https://docs.aws.amazon.com/athena/latest/ug/connect-to-data-source-hive.html
> ).
>
> If you can drop me an email ([email protected]), I would be happy to
> provide you with more details.
>
> On 2019/12/31 19:16:07, Vinoth Chandar <[email protected]> wrote:
> > Can one of the aws folks please chime in here? IIRC I saw some tweets
> > mentioning Hudi/Athena support is in the works.
> > Not sure myself.
> >
> > On Sun, Dec 29, 2019 at 11:33 PM Syed Abdul Kather <[email protected]>
> > wrote:
> >
> > > Hi Team,
> > >
> > > We have built the "CDC pipeline with apache hudi and debezium" . It
> > > works very well in our production.
> > >
> > > But we have inhouse Ambari Cluster with Hive metastore for all the ETL
> > > purpose and Athena for all analytics purposes. To make hudi table we
> work
> > > on the athena we have preserved only the latest version and create the
> > > table in parquet format .
> > >
> > > Right now hive metastore get update using hudi itself . But to keep the
> > > athena metastore in sync we have wrote a separate script to manage. But
> > > that looks like not right approach . As only the required the affected
> > > partition needs to be updated in athena side.
> > >
> > > Please suggest as right approach here .
> > >
> > > Thanks and Regards,
> > > S SYED ABDUL KATHER
> > >
> >
>
