zihanx opened a new pull request, #16496:
URL: https://github.com/apache/lucene/pull/16496
…LeafWithOrdinals for scatter/gather
[Disclaimer: I work at Amazon customer facing product search]
### Description
Redesigns the recently-added `@lucene.internal` `ReaderUtil.partitionByLeaf`
and adds an
ordinals-tracking variant. This is the first building block for a future
doc-values retriever
(scatter/gather); the retriever itself is a separate, later PR and is
**not** included here.
Tracking issue: #15905
#### What changed
- **`partitionByLeaf(ScoreDoc[], leaves)` → `partitionByLeaf(int[],
leaves)`.** Partitioning only
ever needs the doc id; `ScoreDoc` carries score/shardIndex the operation
ignores. Callers holding
`ScoreDoc[]` do a trivial extraction into an `int[]`. This also avoids a
four-method explosion
(`ScoreDoc[]` vs `int[]` × ordinals vs no-ordinals).
- **Added `partitionByLeafWithOrdinals(int[], leaves)`** returning a
`PartitionedHits(int[][] docIdsByLeaf, int[][] ordinalsByLeaf)` record,
where
`ordinalsByLeaf[k][i]` is the index in the original input array of the doc
id at
`docIdsByLeaf[k][i]`, so callers can reassemble per-leaf results back into
input order. The
round-trip invariant is `docIdsByLeaf[k][i] ==
globalDocIds[ordinalsByLeaf[k][i]]`.
#### Design notes
- **Two methods, not a boolean flag or a single always-ordinals method.**
`partitionByLeaf` uses the
`Arrays.sort(int[])` fast path; `partitionByLeafWithOrdinals` uses an
`IntroSorter` over parallel
`int[]` arrays (doc ids + ordinals swapped in lockstep) to avoid the
boxing/lambda overhead of a
comparator-based sort. The no-ordinals method's javadoc positions it as an
optimized subset of the
ordinals variant.
- **Ordinals are returned per-leaf (`int[][]`), not flat.** The future
retriever runs one task per
leaf, so per-leaf ordinals make each leaf a self-contained unit of work
with no prefix-sum
bookkeeping. Same shape as `docIdsByLeaf`.
- **Unsorted input, no mutation.** Both methods accept input in any order
and take a defensive
`clone()` before sorting, so caller arrays are never mutated.
#### Breaking change
This changes the signature of `partitionByLeaf(ScoreDoc[], leaves)`, added
recently in #15803 and
marked `@lucene.internal`. Since it's internal and new, this is a straight
replacement rather than a
deprecation.
#### Tests
`TestReaderUtil` covers both methods: empty / single-segment / multi-segment
/ skipped-empty-segments
/ unsorted-input / no-mutation cases, plus randomized runs verifying leaf
membership, ascending
in-leaf order, and the ordinal round-trip.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
