Hello, I would love to have hive merge the small files in my managed hive context after every query. Right now, I am setting the hive configuration in my Spark Job configuration but hive is not managing the files. Do I need to set the hive fields in around place? How do you set Hive configurations in Spark?
Here is what I'd like to set hive.merge.mapfilestrue hive.merge.mapredfilestrue hive.merge.size.per.task256000000 hive.merge.smallfiles.avgsize16000000
