I see some nice answers at https://stackoverflow.com/questions/46072411/can-i-change-the-nullability-of-a-column-in-my-spark-dataframe
On Thu, 14 Oct 2021 at 5:21 PM, [email protected] <[email protected]> wrote: > Gurus, > > I have an RDD in PySpark that I can convert to DF through > > df = rdd.toDF() > > However, when I do > > df.printSchema() > > I see the columns as nullable. = true by default > > root > |-- COL-1: long (nullable = true) > |-- COl-2: double (nullable = true) > |-- COl-3: string (nullable = true) > > What would be the easiest way to make COL-1 NOT NULLABLE > > Thanking you > -- Cheers, Sonal https://github.com/zinggAI/zingg
