Dear Hudi Users, We noticed an issue with 0.6.0 release and would like to notify you all. https://issues.apache.org/jira/browse/HUDI-1230 This affects Spark Datasource batch writes on MOR tables only. This problem will NOT be seen when inline compaction is enabled. Spark Structured Streaming Writes on MOR table is also unaffected. The implication of this bug is that spark-submit jobs running DataSource batch writes on MOR table will not shut down after the job is complete.
As a work around, please set this redundant hoodie config in your spark-submit job when running batch writes in Spark. hoodie.datasource.compaction.async.enable=false As mentioned above, if inline compaction is enabled, the above workaround is not needed at all. Udit has already opened a PR to address this issue : https://github.com/apache/hudi/pull/2046 Thanks,Balaji.V
