Hi all, I've been exploring whether Lucene could support indexes with more than the current ~2B document limit, and would appreciate an early read on the direction before we invest further.
The approach keeps doc IDs as ints within each segment and uses longs for doc IDs across the whole index. It appears to avoid file-format changes, but requires API changes above the leaf readers, including in the top-docs path. That would be a major-version change and affect callers that assume doc IDs are ints. I have a draft PR with the implementation and the rough edges I know about: https://github.com/apache/lucene/pull/16847 I'm not proposing this for Lucene 11 or asking for a release target. At this stage, I'd mainly appreciate feedback on whether the leaf-int/global-long split is a direction worth pursuing and whether there are fundamental issues I may be missing. Thanks, Jim
