siddharthteotia opened a new pull request, #18569:
URL: https://github.com/apache/pinot/pull/18569
Fixes #18568.
## Summary
`JsonExtractScalarTransformFunction` and `JsonExtractIndexTransformFunction`
both have `_nullHandlingEnabled` plumbed through `init()` but their six typed
SV methods (`transformTo{Int,Long,Float,Double,BigDecimal,String}ValuesSV`)
ignore it and throw unconditionally when the JSON path doesn't resolve and no
default literal is supplied. This PR makes both transforms honor
`_nullHandlingEnabled`: when query-level null handling is on, unresolved rows
surface as SQL `NULL` (typed null placeholder + a bit in `getNullBitmap()`)
instead of throwing.
See the issue for the full behavioral matrix (before/after, NH on/off, with
and without default value) and the worked country/clicks example.
## Scope
**This commit:** `JsonExtractScalarTransformFunction` only.
**Next commit on this branch:** apply the same fix to
`JsonExtractIndexTransformFunction` (PR description will be updated once that
commit lands so reviewers can eyeball each half independently).
MV transforms are unchanged. The legacy throw is preserved when null
handling is off, so callers that haven't opted in see no behavior change.
## Tests
Added a country/click fixture in `JsonExtractScalarTransformFunctionTest`
covering:
- **4 JSON structural shapes** — flat scalar, nested object, array element,
array of objects
- **Unresolved variants** — missing key, explicit JSON null, empty doc,
empty array, out-of-bounds index, null parent
- **All 6 SV typed paths** — INT / LONG / FLOAT / DOUBLE / BIG_DECIMAL /
STRING
- **3 query shapes** — projection, GROUP BY with COUNT, DISTINCT
- **Both null-handling modes** — ON (verifies SQL `NULL` surfaces correctly)
and OFF (verifies legacy throw preserved)
63 new parameterized test cases. All existing tests in the class continue to
pass.
## Test plan
- [x] `./mvnw -pl pinot-core -am
-Dtest=JsonExtractScalarTransformFunctionTest test` — 174 tests pass (109
pre-existing + 65 new from the country/click suite, including expansions to
existing parameterized tests)
- [x] `./mvnw -pl pinot-core -am -Dtest=JsonExtractScalarTest test` — 23
tests pass
- [x] `./mvnw -pl pinot-core -am spotless:apply checkstyle:check
license:check` clean
- [ ] Same suite re-run after the `JsonExtractIndexTransformFunction` commit
lands
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
