Jackie-Jiang commented on PR #20042:
URL: https://github.com/apache/pinot/pull/20042#issuecomment-6699270793
I'd prefer to put the bounding in the reader API so immutable inverted
indexes don't pay the per-posting `isEmpty()` / `last()` checks.
Could we add an overload to `InvertedIndexReader<T>`:
```java
default T getDocIds(int dictId, int numDocs) {
return getDocIds(dictId);
}
```
Here, `numDocs` is the query's planning-time segment document count.
Immutable indexes already fit that bound; mutable readers and adapters over
them must override the method. For `RealtimeInvertedIndex`:
```java
@Override
public MutableRoaringBitmap getDocIds(int dictId, int numDocs) {
MutableRoaringBitmap docIds = getDocIds(dictId); // Existing fresh clone
docIds.remove(numDocs, 1L << 32);
return docIds;
}
```
Then have every inverted-index lookup in `InvertedIndexFilterOperator` pass
`_numDocs`, including the count and `getBitmaps()` paths. The bound stays fixed
for the query, but is applied to each newly read bitmap.
Please also cover `MutableOpenStructDataSource.DefaultFoldingInvertedIndex`:
it wraps live postings without implementing `MutableInvertedIndex`, so an
`instanceof MutableInvertedIndex` check would miss it. Its overload should trim
the final result after folding in absent documents.
This keeps the existing one-argument API, avoids an additional bitmap copy
or per-query reader-wrapper allocation, and leaves ownership handling with the
reader. Immutable readers only have the default forwarding call, which the JIT
can inline; there is no per-bitmap bounding work on that path. The existing
consuming-segment regressions should remain, with coverage for the folded
open-struct reader as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
