Hi All, I'm trying to insert the content of a dataframe to a partitioned parquet-formatted hive table using
*df.write.mode(SaveMode.Append).insertInto(myTable)* with *hive.exec.dynamic.partition = 'true' * and *hive.exec.dynamic.partition.mode = 'nonstrict'.* I keep getting an *parquet.io.ParquetEncodingException* saying that *"empty fields are illegal, the field should be ommited completely instead"* . The schema includes arrays, and the df do contain some empty entries for these fields. However, when I insert the df content into a *non*-partitioned table, I do not get an error. Can anyone give me an explanation about what is going on here ? Why does this error appear only when the hive table is partitioned ? Thanks, Best Regards, Jens
