Spark sample submitted with cluster deploy-mode does not work in Standalone

2017-08-07 Thread ctang
Spark sample submitted with cluster deploy-mode (./bin/run-example --master spark://localhost:6066 --deploy-mode=cluster SparkPi 10) throw out following error, any one knows what the problem is? == 17/08/07 16:27:39 ERROR RestSubmissionClient: Exception from the cluster:

How to insert a dataframe as a static partition to a partitioned table

2017-07-19 Thread ctang
I wonder if there are any easy ways (or APIs) to insert a dataframe (or DataSet), which does not contain the partition columns, as a static partition to the table. For example, The DataSet with columns (col1, col2) will be inserted into a table (col1, col2) partitioned by column partcol as a