Re: Making withColumn nullable

[Koert Kuipers]

it should be by default nullable except for certain primitives where it
defaults to non-nullable

you can use Option for your return value to indicate nullability.

On Fri, Jan 27, 2017 at 10:32 AM, Ninad Shringarpure 
wrote:

> HI Team,
>
> When I add a column to my data frame using withColumn and assign some
> value, it automatically creates the schema with this column to be not
> nullable.
> My final Hive table schema where I want to insert it has this column to be
> nullable and hence throws an error when I try to save.
>
> Is there a way of making the column I add with withColumn method to be set
> to nullable?
>
> Thanks,
> Ninad
>

Making withColumn nullable

[Ninad Shringarpure]

HI Team,

When I add a column to my data frame using withColumn and assign some
value, it automatically creates the schema with this column to be not
nullable.
My final Hive table schema where I want to insert it has this column to be
nullable and hence throws an error when I try to save.

Is there a way of making the column I add with withColumn method to be set
to nullable?

Thanks,
Ninad