Re: compute ignite data with spark

2018-02-27 Thread shawn.du
Hi Denis,Thanks, that's cool! looking forward 2.4.0 release. ThanksShawn On 2/28/2018 12:11,Denis Magda wrote: Hi Shawn,In addition to RDDs, you'll be able to use Data Frames APIs soon with Ignite as stora

Re: compute ignite data with spark

2018-02-27 Thread Denis Magda
Hi Shawn, In addition to RDDs, you'll be able to use Data Frames APIs soon with Ignite as storage for Spark. They will be released within nearest weeks in Ignite 2.4. As for your question on how Ignite compares to Spark. The fist is not just a computational engine. It's a distributed database (or

Re: compute ignite data with spark

2018-02-26 Thread shawn.du
Hi, stan,Thanks. I am evaluating igniteRDD. it is cool!  I am new to spark, I can get sqlContext by ds.sqlContext.If I want to query all using select, what's the table name?see below in spark-shell,  I want to do something like > sc.sql("select * from ").  Where is the table name?===

RE: compute ignite data with spark

2018-02-26 Thread Stanislav Lukyanov
Hi Shawn, You can use Ignite standalone and you can also use it together with Spark. Please take a look at these SO question and an article: https://stackoverflow.com/questions/36036910/apache-spark-vs-apache-ignite https://insidebigdata.com/2016/06/20/apache-ignite-and-apache-spark-complementary-