raghavyadav01 opened a new pull request, #19604:
URL: https://github.com/apache/pinot/pull/19604
### Problem
A key absent from a segment entirely already reads as the child field spec's
default, through `OpenStructDataSource.getValueFieldSpec`. A key absent from
*one document* read as the standard type default instead, so the same key
answered two different values depending on why it was missing.
It isn't only cosmetic. `MapFilterOperator` refuses the sparse JSON-index
fast path when a predicate names the declared default, because the index cannot
see the documents that lack the key:
```java
// Docs without the key read as the key's default null value, which the JSON
index cannot see
if (values.contains((String)
sparseKeyDs.getDataSourceMetadata().getFieldSpec().getDefaultNullValue())) {
return null;
}
```
That guard reads the value from the field spec while the reader returned the
type default, so the two disagreed. A `NOT_IN` over a declared default took the
fast path and missed exactly the documents the guard exists to protect.
### Change
`SparseKeyForwardIndexReader` takes the child field spec and reads its
default once, using it for the scalar accessors when the declared default
matches that type and falling back to the standard type default otherwise.
Behaviour is unchanged for a key with no declared default.
Resolves the TODO referencing #19466.
### Testing
`SparseKeyDataSourceTest` gains a case asserting a declared default is
returned for a document missing the key, and one asserting the type default
still applies when none is declared. The existing OPEN_STRUCT suite passes (134
tests in `pinot-segment-local`).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
