Jackie-Jiang opened a new issue, #19466: URL: https://github.com/apache/pinot/issues/19466
An OPEN_STRUCT child field spec can declare a custom `defaultNullValue`, but only one of the four per-key storage layouts honors it. With null handling disabled, the same query can therefore return different values for the same key depending only on which segment answers it. ### Current behavior A document that does not carry the key reads as: | Layout | Effective default | Resolved by | |---|---|---| | Key absent from the whole segment | the declared child spec's `defaultNullValue` | `OpenStructDataSource#getValueFieldSpec` | | Materialized (dense) key | standard dimension default of the stored type | `OpenStructColumnSplitter#writeDenseKeyColumn` | | Sparse key | standard dimension default of the stored type | `SparseKeyDataSource` | | Consuming (mutable) key | standard dimension default of the stored type | `MutableOpenStructIndex#allocateKeyColumn` | So for a declared STRING child with `"defaultNullValue": "N/A"`, a segment in which the key never occurs reads `N/A` for every row, while a segment in which the key occurs at least once reads `null` for every other row. ### Desired Pick one effective-default rule and apply it across all four layouts. The natural choice is the declared child field spec, which `OpenStructDataSource#getValueFieldSpec` already returns, since it is the only rule that reflects what was configured. The sealed and consuming paths are coupled: `MutableOpenStructIndex` deliberately mirrors `OpenStructColumnSplitter` so that a document's resolved value does not change when a consuming segment is sealed. They have to move together. `MapFilterOperator#trySparseJsonIndex` already refuses the sparse JSON fast path for the key's declared default, so it needs no change. ### Code sites - `OpenStructColumnSplitter#writeDenseKeyColumn` - `MutableOpenStructIndex#allocateKeyColumn` - `SparseKeyDataSource` Each carries a TODO pointing at this issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
