Hi Ingiters Spark has an RDD interface which lets you plug in various back-ends like HDFS and JDBC etc. I think it would be a great idea if the community could leverage Ignites in-memory file system to provide blazing fast analytics using both ingite and spark.
Here is the interface : https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/rdd/RDD.scala The idea would be to compute splits and partitions from ignitefs and use these in an RDD, an example is: https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/rdd/NewHadoopRDD.scala Would like to know whether this is feasible, as it would be a useful addition to both communities and help complement other implementations for tachyon for example. Cheers Devl
