[
https://issues.apache.org/jira/browse/LUCENE-8310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Simon Willnauer updated LUCENE-8310:
------------------------------------
Attachment: LUCENE-8310.patch
> Relax IWs check on pending deletes
> ----------------------------------
>
> Key: LUCENE-8310
> URL: https://issues.apache.org/jira/browse/LUCENE-8310
> Project: Lucene - Core
> Issue Type: Improvement
> Affects Versions: 7.4, master (8.0)
> Reporter: Simon Willnauer
> Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: LUCENE-8310.patch
>
>
> I recently fixed the check in IW to fail if there are pending deletes. After
> upgrading to a snapshot I realized the consequences of this check. It made
> most of our usage of IW to for instance prepare commit metadata, rollback to
> safe commit-points etc. impossible since we have to now busy wait on top of
> directory util all deletes are actually gone even though that we can
> guarantee that our history always goes forward. ie we are truly append-only
> in the sense of never reusing segment generations. The fix that I made was
> basically return false from a _checkPendingDeletions_ in a directory wrapper
> to work around it.
> I do expect this to happen to a lot of lucene users even if they use IW
> correctly. My proposal is to make the check in IW a bit more sophisticated
> and only fail if there are pending deletes that are in the future from a
> generation perspective. We really don't care about files from the past. My
> patch checks the segment generation of each pending file which is safe since
> that is the same procedure we apply in IndexFileDeleter to inc reference etc.
> and only fail if the pending delete is in the future.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
