xiangfu0 opened a new pull request, #19127:
URL: https://github.com/apache/pinot/pull/19127
## Summary
- stream top-level JSON values directly into `GenericRow` for the default
JSON record extractor
- skip materializing unselected containers while preserving Pinot's existing
value conversion and missing-field behavior
- retain the map-based path for custom record extractors and parsers without
direct-row support
- add decoder-focused JMH coverage for all-field and selected-field workloads
## Motivation
The existing decoder first materializes every top-level value in a per-row
`Map`, then copies selected values into `GenericRow`. For ingestion
configurations that select only a subset of a wide JSON payload, this creates
avoidable parsing work and allocations.
## Performance
Focused JMH results for selected fields (`2` warmup iterations, `5` measured
iterations, `5s` each, one fork, JDK 25):
| Payload | Throughput | Allocation |
| --- | ---: | ---: |
| Small | +54.7% | -47.6% |
| Medium | +63.5% | -63.3% |
| Large | +149.7% | -91.1% |
| Nested selected object | +13.4% | -5.5% |
These numbers measure decoder work rather than end-to-end Pinot ingestion.
To reproduce after packaging `pinot-perf`:
```bash
java -Xms2G -Xmx4G \
-cp 'pinot-perf/target/pinot-perf-pkg/lib/*' \
org.openjdk.jmh.Main \
'org.apache.pinot.perf.BenchmarkJsonParsing.(mapThenExtractSelectedFields|directToGenericRowSelectedFields)'
\
-wi 2 -i 5 -f 1 -w 5s -r 5s -prof gc
```
## Compatibility
- Custom `RecordExtractor` implementations continue through the existing
map-based path.
- Text JSON, Smile, CBOR, PostgreSQL JSONB, and auto-detected supported
formats use the direct path.
- Parsers without direct-row support return `false` and retain the existing
fallback.
## Validation
- `JSONMessageDecoderTest`
- `JSONMessageDecoderBinaryTest`
- `JSONRecordExtractorTest`
- 41 tests passed
- Spotless, Checkstyle, and license format/check passed for `pinot-json` and
`pinot-perf`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
