i all,
>> >>
>> >> Consider the following case:
>> >>
>> >> import java.sql.Timestamp
>> >> case class MyProduct(t: Timestamp, a: Float)
>> >> val rdd = sc.parallelize(List(MyProduct(new Timestamp(0), 10))).toDF()
>> >&
> >> rdd.printSchema()
> >>
> >> The output is:
> >> root
> >> |-- t: timestamp (nullable = true)
> >> |-- a: float (nullable = false)
> >>
> >> How can I set the timestamp column to be NOT nullable?
> >>
> >> R
>
>> The output is:
>> root
>> |-- t: timestamp (nullable = true)
>> |-- a: float (nullable = false)
>>
>> How can I set the timestamp column to be NOT nullable?
>>
>> Regards,
>> Luis
>>
>>
>>
>> --
>> View this me
Hi Jacek,
I have not used Encoders before. Definitely this works! Thank you!
Luis
On 4 August 2016 at 18:23, Jacek Laskowski wrote:
> On Thu, Aug 4, 2016 at 11:56 PM, luismattor wrote:
>
> > import java.sql.Timestamp
> > case class MyProduct(t: Timestamp, a: Float)
> > val rdd = sc.paralleli
e)
> |-- a: float (nullable = false)
>
> How can I set the timestamp column to be NOT nullable?
>
> Regards,
> Luis
>
>
>
> --
> View this message in context: http://apache-spark-user-list.
> 1001560.n3.nabble.com/How-to-set-nullable-field-when-
> create-DataFrame-u
On Thu, Aug 4, 2016 at 11:56 PM, luismattor wrote:
> import java.sql.Timestamp
> case class MyProduct(t: Timestamp, a: Float)
> val rdd = sc.parallelize(List(MyProduct(new Timestamp(0), 10))).toDF()
> rdd.printSchema()
>
> The output is:
> root
> |-- t: timestamp (nullable = true)
> |-- a: floa
On Thu, Aug 4, 2016 at 11:56 PM, luismattor wrote:
> How can I set the timestamp column to be NOT nullable?
Hi,
Given [1] it's not possible without defining your own Encoder for
Dataset (that you use implicitly).
It'd be something as follows:
implicit def myEncoder: Encoder[MyProduct] = ???
s
I set the timestamp column to be NOT nullable?
Regards,
Luis
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/How-to-set-nullable-field-when-create-DataFrame-using-case-class-tp27479.html
Sent from the Apache Spark User List mailing list archive at Nabble.com