Re: How to set nullable field when create DataFrame using case class

rdd = sc.parallelize(List(MyProduct(new Timestamp(0), 10))).toDF() >> >> rdd.printSchema() >> >> >> >> The output is: >> >> root >> >> |-- t: timestamp (nullable = true) >> >> |-- a: float (nullable = false) >> >>

Re: How to set nullable field when create DataFrame using case class

st(MyProduct(new Timestamp(0), 10))).toDF() > >> rdd.printSchema() > >> > >> The output is: > >> root > >> |-- t: timestamp (nullable = true) > >> |-- a: float (nullable = false) > >> > >> How can I set the timestamp column

Re: How to set nullable field when create DataFrame using case class

))).toDF() >> rdd.printSchema() >> >> The output is: >> root >> |-- t: timestamp (nullable = true) >> |-- a: float (nullable = false) >> >> How can I set the timestamp column to be NOT nullable? >> >> Regards, >> Luis >> >> &g

Re: How to set nullable field when create DataFrame using case class

Hi Jacek, I have not used Encoders before. Definitely this works! Thank you! Luis On 4 August 2016 at 18:23, Jacek Laskowski wrote: > On Thu, Aug 4, 2016 at 11:56 PM, luismattor wrote: > > > import java.sql.Timestamp > > case class MyProduct(t:

Re: How to set nullable field when create DataFrame using case class

imestamp (nullable = true) > |-- a: float (nullable = false) > > How can I set the timestamp column to be NOT nullable? > > Regards, > Luis > > > > -- > View this message in context: http://apache-spark-user-list. > 1001560.n3.nabble.com/How-to-set-nullable-field

Re: How to set nullable field when create DataFrame using case class

On Thu, Aug 4, 2016 at 11:56 PM, luismattor wrote: > import java.sql.Timestamp > case class MyProduct(t: Timestamp, a: Float) > val rdd = sc.parallelize(List(MyProduct(new Timestamp(0), 10))).toDF() > rdd.printSchema() > > The output is: > root > |-- t: timestamp (nullable

Re: How to set nullable field when create DataFrame using case class

On Thu, Aug 4, 2016 at 11:56 PM, luismattor wrote: > How can I set the timestamp column to be NOT nullable? Hi, Given [1] it's not possible without defining your own Encoder for Dataset (that you use implicitly). It'd be something as follows: implicit def myEncoder:

How to set nullable field when create DataFrame using case class

I set the timestamp column to be NOT nullable? Regards, Luis -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-set-nullable-field-when-create-DataFrame-using-case-class-tp27479.html Sent from the Apache Spark User List mailing list archive at Nabble.com