Re: What / Where / When / How questions in Spark 2.0 ?

2016-05-21 Thread Amit Sela
; https://docs.google.com/document/d/1y4qlQinjjrusGWlgq-mYmbxRW2z7-_X5Xax-GG0YsC0/edit?usp=sharing > > From: "Sela, Amit" > Date: Saturday, May 21, 2016 at 11:52 PM > To: Ovidiu-Cristian MARCU , "user @spark" > > Cc: Ovidiu Cristian Marcu > Subject:

Re: What / Where / When / How questions in Spark 2.0 ?

2016-05-21 Thread Ovidiu-Cristian MARCU
-Cristian MARCU <mailto:ovidiu-cristian.ma...@inria.fr>>, "user @spark" > mailto:user@spark.apache.org>> > Cc: Ovidiu Cristian Marcu <mailto:ovidiu21ma...@gmail.com>> > Subject: Re: What / Where / When / How questions in Spark 2.0 ? > > This is a “Tech

Re: What / Where / When / How questions in Spark 2.0 ?

2016-05-21 Thread Sela, Amit
u-Cristian MARCU mailto:ovidiu-cristian.ma...@inria.fr>>, "user @spark" mailto:user@spark.apache.org>> Cc: Ovidiu Cristian Marcu mailto:ovidiu21ma...@gmail.com>> Subject: Re: What / Where / When / How questions in Spark 2.0 ? This is a “Technical Vision” paper

Re: What / Where / When / How questions in Spark 2.0 ?

2016-05-21 Thread Sela, Amit
Ovidiu Cristian Marcu mailto:ovidiu21ma...@gmail.com>> Subject: Re: What / Where / When / How questions in Spark 2.0 ? Could you please consider a short answer regarding the Apache Beam Capability Matrix todo’s for future Spark 2.0 release [4]? (some related references belo

Re: What / Where / When / How questions in Spark 2.0 ?

2016-05-16 Thread Ovidiu-Cristian MARCU
Could you please consider a short answer regarding the Apache Beam Capability Matrix todo’s for future Spark 2.0 release [4]? (some related references below [5][6]) Thanks [4] http://beam.incubator.apache.org/capability-matrix/#cap-full-what

What / Where / When / How questions in Spark 2.0 ?

2016-05-16 Thread Ovidiu-Cristian MARCU
Hi, We can see in [2] many interesting (and expected!) improvements (promises) like extended SQL support, unified API (DataFrames, DataSets), improved engine (Tungsten relates to ideas from modern compilers and MPP databases - similar to Flink [3]), structured streaming etc. It seems we somehow