Re: Fail to cache rdd: java.lang.NoSuchMethodError: scala.Predef$.$conforms()Lscala/Predef$$less$colon$less;

2016-10-12 Thread vkulichenko
Hi Tracy, Here are my answers: 1. Correct. IgniteContext is actually a wrapper around SparkContext which is used to provide connectivity with the Ignite cluster. 2. Please refer to [1] for information about how to load the data into the cache. You can implement your own CacheStore or use IgniteDa

Re: Fail to cache rdd: java.lang.NoSuchMethodError: scala.Predef$.$conforms()Lscala/Predef$$less$colon$less;

2016-10-06 Thread Tracyl
Hi Denis, This is really helpful. Yes, I need the original dataframe for other API. Now I am using RDD[String, Row] as type and caching dataframe using: val rdd = df.map(row => (row.getAs[String]("KEY"), row)) igniteRDD.savePairs(rdd) It works perfectly fine. Also I was able to reconstruct the d

Re: Fail to cache rdd: java.lang.NoSuchMethodError: scala.Predef$.$conforms()Lscala/Predef$$less$colon$less;

2016-10-06 Thread Denis Magda
Presently, You need to covert your data frame into key-value tuples and insert them in an IgniteRDD https://apacheignite-fs.readme.io/docs/ignitecontext-igniterdd#section-saving-values-to-ignite T

Re: Fail to cache rdd: java.lang.NoSuchMethodError: scala.Predef$.$conforms()Lscala/Predef$$less$colon$less;

2016-10-06 Thread Tracyl
Thanks for prompt reply. So if I want to cache dataframe in IgniteCache, I have to do define a custom data model class(e.g. https://github.com/apache/ignite/blob/master/examples/src/main/java/org/apache/ignite/examples/model/Person.java ) as a schema of dataframe, then construct objects and declare

Re: Fail to cache rdd: java.lang.NoSuchMethodError: scala.Predef$.$conforms()Lscala/Predef$$less$colon$less;

2016-10-06 Thread Denis Magda
Duplicate message. Replied you in a similar one http://apache-ignite-users.70518.x6.nabble.com/Does-IgniteCache-could-be-access-thr-Spark-JDBC-data-source-api-tp8117.html

Re: Fail to cache rdd: java.lang.NoSuchMethodError: scala.Predef$.$conforms()Lscala/Predef$$less$colon$less;

2016-10-06 Thread Denis Magda
Hi, Presently Ignite integration for Spark is not intended to be used with Spark Data Frames. There is the ticket you can track on that was created in order to fill this gap. https://issues.apache.org/jira/browse/IGNITE-3084 As for NoSuchMeth

Fail to cache rdd: java.lang.NoSuchMethodError: scala.Predef$.$conforms()Lscala/Predef$$less$colon$less;

2016-10-06 Thread Tracyl
Hi team, I was trying to cache dataframe in Ignite Cache. I was able to cache generic type data elements(RDD). However each time when I use igniteRDDF.saveValues() to cache a non-generic data type(e.g. RDD), it will trigger the noSuchMethod for saveValues as following shows. I am using scala 2.10

Fail to cache rdd: java.lang.NoSuchMethodError: scala.Predef$.$conforms()Lscala/Predef$$less$colon$less;

2016-10-06 Thread Tracyl
Hi team, I was trying to cache dataframe in Ignite Cache. I was able to cache generic type data elements(RDD). However each time when I use igniteRDDF.saveValues() to cache a non-generic data type(e.g. RDD), it will trigger the noSuchMethod for saveValues as following shows. I am using scala 2.10,