Hi Mich, I only tried Alluxio so I can’t give you a comparison. In my experience, I use Alluxio for the big data set (50GB - 100GB) which is the input of the pipelines jobs so you can reuse the result from previous job.
> On Oct 27, 2016, at 5:39 PM, Mich Talebzadeh <mich.talebza...@gmail.com> > wrote: > > Thanks Chanh, > > Can it share RDDs. > > Personally I have not used either Alluxio or Ignite. > > Are there major differences between these two > Have you tried Alluxio for sharing Spark RDDs and if so do you have any > experience you can kindly share > Regards > > > Dr Mich Talebzadeh > > LinkedIn > https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > > <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw> > > http://talebzadehmich.wordpress.com <http://talebzadehmich.wordpress.com/> > > Disclaimer: Use it at your own risk. Any and all responsibility for any loss, > damage or destruction of data or any other property which may arise from > relying on this email's technical content is explicitly disclaimed. The > author will in no case be liable for any monetary damages arising from such > loss, damage or destruction. > > > On 27 October 2016 at 11:29, Chanh Le <giaosu...@gmail.com > <mailto:giaosu...@gmail.com>> wrote: > Hi Mich, > Alluxio is the good option to go. > > Regards, > Chanh > >> On Oct 27, 2016, at 5:28 PM, Mich Talebzadeh <mich.talebza...@gmail.com >> <mailto:mich.talebza...@gmail.com>> wrote: >> >> >> There was a mention of using Zeppelin to share RDDs with many users. From >> the notes on Zeppelin it appears that this is sharing UI and I am not sure >> how easy it is going to be changing the result set with different users >> modifying say sql queries. >> >> There is also the idea of caching RDDs with something like Apache Ignite. >> Has anyone really tried this. Will that work with multiple applications? >> >> It looks feasible as RDDs are immutable and so are registered tempTables etc. >> >> Thanks >> >> >> Dr Mich Talebzadeh >> >> LinkedIn >> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw >> >> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw> >> >> http://talebzadehmich.wordpress.com <http://talebzadehmich.wordpress.com/> >> >> Disclaimer: Use it at your own risk. Any and all responsibility for any >> loss, damage or destruction of data or any other property which may arise >> from relying on this email's technical content is explicitly disclaimed. The >> author will in no case be liable for any monetary damages arising from such >> loss, damage or destruction. >> > >
