Are there any alternatives to Hive "stored by" clause as Spark 2.0 does not support it

2018-02-07 Thread Pralabh Kumar
Hi Spark 2.0 doesn't support stored by . Is there any alternative to achieve the same.

[CFP] DataWorks Summit, San Jose, 2018

2018-02-07 Thread Yanbo Liang
Hi All, DataWorks Summit, San Jose, 2018 is a good place to share your experience of advanced analytics, data science, machine learning and deep learning. We have Artificial Intelligence and Data Science session, to cover technologies such as: Apache Spark, Sciki-learn, TensorFlow, Keras,

Re: Difficulties building spark-master with sbt

2018-02-07 Thread Sean Owen
The master SBT builds seem OK, like: https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/spark-master-test-sbt-hadoop-2.7/ It looks like an issue between Windows, SBT, and your env I think. On Wed, Feb 7, 2018 at 5:12 PM ds wrote: > After

Difficulties building spark-master with sbt

2018-02-07 Thread ds
After cloning today's version of spark-master, I run the following command: S:\spark-master>sbt ./build/sbt -Pyarn -Phadoop-2.7 -Dhadoop.version=2.7.0 -Phive -Phive-thriftserver clean package with the intention of building both the source and test projects and generating the corresponding .jar

`convertMetastoreOrc/Parquet` issue

2018-02-07 Thread Dongjoon Hyun
Hi, All. SPARK-22279 turned on `convertMetastoreOrc` by default for `Feature Parity with Parquet`. Unfortunately, it will be turned off back via https://github.com/apache/spark/pull/20536 in Apache Spark 2.3 RC3 because a well-known