Hi, We can see in [2] many interesting (and expected!) improvements (promises) like extended SQL support, unified API (DataFrames, DataSets), improved engine (Tungsten relates to ideas from modern compilers and MPP databases - similar to Flink [3]), structured streaming etc. It seems we somehow assist at a smart unification of Big Data analytics (Spark, Flink - best of two worlds)!
How does Spark respond to the missing What/Where/When/How questions (capabilities) highlighted in the unified model Beam [1] ? Best, Ovidiu [1] https://cloud.google.com/blog/big-data/2016/05/why-apache-beam-a-google-perspective <https://cloud.google.com/blog/big-data/2016/05/why-apache-beam-a-google-perspective> [2] https://databricks.com/blog/2016/05/11/spark-2-0-technical-preview-easier-faster-and-smarter.html <https://databricks.com/blog/2016/05/11/spark-2-0-technical-preview-easier-faster-and-smarter.html> [3] http://stratosphere.eu/project/publications/ <http://stratosphere.eu/project/publications/>
