Configure spark.kryoserializer.buffer.max at runtime does not take effect

Hi all, I would like to configure the following setting during runtime as below: spark = (SparkSession .builder .appName("ElasticSearchIndex") .config("spark.kryoserializer.buffer.max", "1g") .getOrCreate()) But I still hit error, Caused by: org.apache.spark.SparkException:

Drop partition with PURGE fail

Hi all, I am using spark 2.0 and would like execute the command below to drop the partition with PURGE. sql = "ALTER TABLE db.table1 DROP IF EXISTS PARTITION (pkey='150') PURGE" spark.sql(sql) It throws exception: /Py4JJavaError: An error occurred while calling o45.sql. :