Re: [Spark SQL 1.3.1] data frame saveAsTable returns exception

2015-05-21 Thread Grega Kešpret
*Cc:* Ishwardeep Singh; user *Subject:* Re: [Spark SQL 1.3.1] data frame saveAsTable returns exception I think this is a bug in our date handling that should be fixed in Spark 1.4. On Wed, May 13, 2015 at 8:23 AM, ayan guha guha.a...@gmail.com wrote: Your stack trace says it can't convert

Re: [Spark SQL 1.3.1] data frame saveAsTable returns exception

2015-05-14 Thread Michael Armbrust
date for Spark version 1.4? Regards, Ishwardeep *From:* Michael Armbrust [mailto:mich...@databricks.com] *Sent:* Wednesday, May 13, 2015 10:54 PM *To:* ayan guha *Cc:* Ishwardeep Singh; user *Subject:* Re: [Spark SQL 1.3.1] data frame saveAsTable returns exception I think

RE: [Spark SQL 1.3.1] data frame saveAsTable returns exception

2015-05-14 Thread Ishwardeep Singh
: Re: [Spark SQL 1.3.1] data frame saveAsTable returns exception I think this is a bug in our date handling that should be fixed in Spark 1.4. On Wed, May 13, 2015 at 8:23 AM, ayan guha guha.a...@gmail.commailto:guha.a...@gmail.com wrote: Your stack trace says it can't convert date to integer. You

[Spark SQL 1.3.1] data frame saveAsTable returns exception

2015-05-13 Thread Ishwardeep Singh
Hi , I am using Spark SQL 1.3.1. I have created a dataFrame using jdbc data source and am using saveAsTable() method but got the following 2 exceptions: java.lang.RuntimeException: Unsupported datatype DecimalType() at scala.sys.package$.error(package.scala:27) at

Re: [Spark SQL 1.3.1] data frame saveAsTable returns exception

2015-05-13 Thread Ishwardeep Singh
Hi, I am using spark-shell and the steps using which I can reproduce the issue are as follows: scala val dateDimDF= sqlContext.load(jdbc,Map(url-jdbc:teradata://192.168.145.58/DBS_PORT=1025,DATABASE=BENCHQADS,LOB_SUPPORT=OFF,USER= BENCHQADS,PASSWORD=abc,dbtable - date_dim)) scala

Re: [Spark SQL 1.3.1] data frame saveAsTable returns exception

2015-05-13 Thread ayan guha
Your stack trace says it can't convert date to integer. You sure about column positions? On 13 May 2015 21:32, Ishwardeep Singh ishwardeep.si...@impetus.co.in wrote: Hi , I am using Spark SQL 1.3.1. I have created a dataFrame using jdbc data source and am using saveAsTable() method but got