I completely agree with you. I discuss many time with our team that this integration have not any gain about speed up and i have your idea about caching in ignite because in deep learning we have nothing to share in job because every job independently works on it's portion of data. In your opinion one idea can be ignite+ignite ml+ dl4j +spark? What benefit we can achive in this integration. ?ignite ml cant used for deep independently?
On Wednesday, January 9, 2019, zaleslaw <zaleslaw....@gmail.com> wrote: > Dear Mehdi Sey > > Yes, both platforms are used for in-memory computing, but they have > different APIs and history of feature creation and different ways of > integration with famous DL frameworks (like DL4j and TensorFlow). > > From my point of view, you have no speed up in Ignite + Spark + DL4j > integration. > > Caching data in Ignite as a backend for RDD and dataframes first of all is > acceleration of business logic based on SQL queries. Not the same for ML > frameworks. > > We have no proof, that usage Ignite as a backend could speed up DL4j or > MLlib algorithms. > > Moreover, to avoid this, we wrote own ML library which is more better than > MLlib and runs natively on Ignite. > > In my opinon, you should choose Ignite + Ignite ML + TF integration or > Spark > + DL4j to solve your Data Science task (where you need neural networks). > > > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
