Hi, There's a config option for this. Try setting this to false in your spark conf.
spark.sql.jsonGenerator.ignoreNullFields On Tuesday, October 4, 2022, Karthick Nk <kcekarth...@gmail.com> wrote: > Hi all, > > I need to convert pyspark dataframe into json . > > While converting , if all rows values are null/None for that particular > column that column is getting removed from data. > > Could you suggest a way to do this. I need to convert dataframe into json > with columns. > > Thanks >
