xiangfu0 opened a new pull request, #18722: URL: https://github.com/apache/pinot/pull/18722
## Motivation A common ingestion pattern extracts many fields from a single JSON-string column — e.g. several `JSONPATHSTRING(message, '$.x')` transforms on one record. Each `jsonPath*` call re-parses the **whole** document from scratch, so extracting N fields parses the document N times. On a JSON-heavy realtime log table this re-parsing dominated transform CPU. [#18711](https://github.com/apache/pinot/pull/18711) added an *explicit, opt-in* way to avoid this — `jsonExtractObject(col)` materializes a parsed intermediate column once. This PR adds the *automatic, transparent* equivalent so existing configs benefit without restructuring. ## Change A tiny **per-thread, identity-keyed** cache of parsed documents in `JsonFunctions`. Every `jsonPath`/`jsonPathArray` call for a given record receives the *same* `String` instance, so an identity (`==`) keyed cache collapses the N parses to one. Properties: - **Per-thread** (`ThreadLocal`) — no races, no visibility concerns; retention bounded to `DOCUMENT_CACHE_SIZE` (4) documents per thread. - **Identity-keyed** — a different value is a different `String` instance, so a hit always implies identical content; there is no stale/wrong-result risk and no `equals`/`hashCode` over a large JSON string. - **Parse failures are not cached** — invalid JSON re-throws (unchanged), no poisoned slot. - **Read-only contract** — like the `jsonExtractObject` parse-once path, a path landing on a container returns a live node of the shared parsed model, so `jsonPath`/`jsonPathArray` results must be treated as read-only. The `jsonPathString`/`Long`/`Double` overloads copy out a scalar or a re-serialized string and are unaffected (this is the overwhelmingly common ingestion path). The aliasing is identical to the already-merged `jsonExtractObject` mechanism. ## Performance Micro-benchmark — extracting 8 fields from a ~1 KB message: **3.4× faster** than re-parsing per field (1 parse + 8 reads vs 8 parses + 8 reads). The miss path adds only a `ThreadLocal.get()` + a 4-element reference scan, negligible against a parse. ## Testing `JsonFunctionsTest#testJsonPathParseCache` covers transparency (results identical to the uncached path), round-robin eviction (12 documents × 3 rounds), identity-vs-`equals` keying, read-only container reuse, and that parse failures are not cached. Full `JsonFunctionsTest` passes (40/40). ## Backward compatibility None required. Results are identical to the uncached path; only the number of parses changes. No config / SPI / wire-format surface touched. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
