Brandon, This sounds great! We've also thought around similar lines of hot % vs cold % all accessible through a unified query interface. Let us know how your POC turns out.
-Nishith On Mon, Mar 18, 2019 at 1:04 PM Brandon Geise <[email protected]> wrote: > Will do on the JIRA. I need to do a POC to see if it's something to > pursue on our side. > > Our use case would be similar using Spark + Presto and then having a 30 > day hot storage and cold being on S3 and was thinking of having Alluxio be > cache/memory portion. > > On 3/18/19, 3:49 PM, "Vinoth Chandar" <[email protected]> wrote: > > Great! I can definitely see use-cases for refreshing an alluxio/ignite > cache incrementally.. > > i.e Hive => ETL => Hudi on DFS => Incremental Pull + upsert => Hudi on > in > memory FS > > > if you want to pursue a JIRA, please let me know. I will add you as a > contributor > > On Mon, Mar 18, 2019 at 12:41 PM Brandon Geise <[email protected] > > > wrote: > > > Thanks, Vinoth. I'll take a look. It seems to be a great starting > point! > > > > On 3/18/19, 3:35 PM, "Vinoth Chandar" <[email protected]> wrote: > > > > I have actually played around with Apache Ignite integration > which > > supports > > append(). > > > > > https://github.com/vinothchandar/incubator-hudi/commit/dd578947ec1db9388038f0a1863a90b3761cd571 > > > > > > Alluxio would work as well I believe > > > > Something like Kafka => DeltaStreamer => Hudi/igfs could give > you a > > mutable > > in-memory near real time analytics > > (sorry for bundling up so many buzzwords :P) > > > > > > > > > > > > > > > > > > > > On Mon, Mar 18, 2019 at 12:11 PM Brandon Geise < > [email protected] > > > > > wrote: > > > > > Hi, > > > > > > > > > > > > Has anyone used Hudi in combination with Alluxio? Based on my > > > understanding of each solution, it seems that at a file level > this > > > could/should all work together, but if someone has direct > experience > > I’d > > > love to hear about it. > > > > > > > > > > > > Thanks, > > > > > > Brandon > > > > > > > > > > > > > > > > > >
