Hi, Take a look at Hadoop Accelerator [1]. I think it fits your use case very well. It will accelerate your map-reduce jobs even before you start moving data in memory because it replaces Hadoop's map-reduce engine. On top of that you can use IGFS which is an in-memory file system and which can be used as a caching layer for the data. Persistence to HDFS is supported as well.
[1] https://apacheignite-fs.readme.io/docs/hadoop-accelerator -Val -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Apache-Ignite-vs-alluxio-tp9140p9159.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
