bhasudha commented on issue #1828: URL: https://github.com/apache/hudi/issues/1828#issuecomment-658282320
> Hi Guys, > > Is it possible to retain only last commit? When I put 'hoodie.cleaner.commits.retained': 1 in hudi_options I still have two last commits. One that is being written and the previous one. What I want to achieve is to have only last change and last parquet file. @kirkuz providing some context. Cleaning and compaction happen in the background (asynchronous to ingestion itself). When the cleaner kicks in it would get rid of the older commit. If there is an ongoing write, generally there could be two possibilities - 1. the write would succeed. in which case based on `hoodie.cleaner.commits.retained` the cleaner would get rid of the old version when it triggers. 2. the write would fail for some reason - in this case the cleaner would later get rid of the failed commit and retain the other version (which is the last succeeded one) This is why you are seeing two commits. This should not affect the queries. Can you please elaborate on what you were looking for in terms of use case/ performance concern etc to help us understand better ? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
