I have heard of a tool called Alluxio used between Hudi and Spark/Presto. ( https://www.alluxio.io/blog/building-high-performance-data-lake-using-apache-hudi-and-alluxio-at-t3go/ ) I was wondering if Apache Ignite could serve the same purpose, allowing queries to be processed faster.
On Thu, Sep 15, 2022 at 10:29 AM Jeremy McMillan < [email protected]> wrote: > I just read this, about hudi, and I can't see a use case for putting hudi > behind an Ignite write-through cache. > > https://www.xenonstack.com/insights/what-is-hudi > > Hudi seems to be a write accelerator for Spark on HDFS, primarily. > > What would the expected outcome be if we assume the magic integration was > present and working as you intend? What's the difference between that and > not using Ignite with Hudi? > > On Wed, Sep 14, 2022, 22:50 Tecno Brain <[email protected]> > wrote: > >> In particular I am looking if anyone has used Apache Ignite as a >> write-through cache to Hudi. >> Does that make sense? >> >> On Wed, Sep 14, 2022 at 10:50 PM Tecno Brain < >> [email protected]> wrote: >> >>> I was wondering if anybody has used Hudi + Ignite? >>> Any references to articles, conferences are greatly appreciated. >>> >>> Thanks >>> >>> >>> >>>
