Re: Loading data from Spark Cluster to Ignite Cache to perform Ignite ML

2019-01-02 Thread Stephen Darlington
Where does the data in your Spark DataFrame come from? As I understand it, that would all be in Spark’s memory anyway? Anyway, I didn’t test this exact scenario, but it seems that it writing directly to an Ignite DataFrame should work — why did you think it wouldn’t? I can’t say whether it

Loading data from Spark Cluster to Ignite Cache to perform Ignite ML

2018-12-26 Thread zaleslaw
Hi, Igniters! I am looking for a possibility to load data from Spark RDD or DataFrame to Ignite cache with next declaration IgniteCache dataCache to perform Ignite ML algorithms. As I understand the current mechanism of Ignite-Spark integration helps to store RDD/DF from Spark in Ignite to