Re: StructField Translation Error with Spark SQL

You meant for fields which are nullable. Can you pastebin the complete stack trace ? Try 1.6.1 when you have a chance. Thanks On Wed, Apr 20, 2016 at 10:20 PM, Charles Nnamdi Akalugwu < cprenzb...@gmail.com> wrote: > I get the same error for fields which are not null unfortunately. > > Can't

Re: StructField Translation Error with Spark SQL

I get the same error for fields which are not null unfortunately. Can't translate null value for field StructField(density,DecimalType(4,2),true) On Apr 21, 2016 1:37 AM, "Ted Yu" wrote: > The weight field is not nullable. > > Looks like your source table had null value for

Re: StructField Translation Error with Spark SQL

The weight field is not nullable. Looks like your source table had null value for this field. On Wed, Apr 20, 2016 at 4:11 PM, Charles Nnamdi Akalugwu < cprenzb...@gmail.com> wrote: > Hi, > > I am using spark 1.4.1 and trying to copy all rows from a table in one > MySQL Database to a Amazon RDS

StructField Translation Error with Spark SQL

Hi, I am using spark 1.4.1 and trying to copy all rows from a table in one MySQL Database to a Amazon RDS table using spark SQL. Some columns in the source table are defined as DECIMAL type and are nullable. Others are not. When I run my spark job, val writeData =