ganeshashree opened a new pull request, #59828:
URL: https://github.com/apache/spark/pull/59828
### What changes were proposed in this pull request?
Follow-up to SPARK-59144. When a nested SQL/JSON constructor is reached
through a
*routed* or *qualified* `JSON_ARRAY` / `JSON_OBJECT` call, splice the inner
constructor's raw JSON output into the outer array/object instead of quoting
it as
a string.
Three splice paths are now handled, all collation-robust:
- Unqualified nested `JSON_ARRAY` routes through a new `UnresolvedJsonArray`
+
`ResolveJsonConstructors` analyzer rule (placed just before
`ApplyDefaultCollation`),
carrying the parse-time-frozen `formatJson` flags.
- Unqualified `JSON_OBJECT` comma form splices via a
`JsonImplicitFormatCarrier`.
- Routed/qualified calls (both constructors) re-derive the splice from the
resolved
argument in the expression builders.
It also fixes a collation bug surfaced by this work: under a non-default
object/view
`DEFAULT COLLATION`, `ApplyDefaultCollation` wraps the nested producer in a
collation
`Cast` before the routed/qualified builder runs, so the builder failed to
recognize
it as implicitly-JSON and silently quoted the value (e.g. `["[1]"]` instead
of
`[[1]]`). A localized `JsonArray.splicesThroughCollationCast` (used only by
the two
builders) sees through an analyzer-inserted collation `Cast` while leaving
user-
specified casts and the `.sql` detach cast untouched, so rendering and parse
behavior
are unaffected.
### Why are the changes needed?
SPARK-59144 made unqualified JSON constructors route through routine
resolution, but
nested constructors reached via routed or qualified calls were still
serialized as
quoted strings, producing incorrect JSON. This completes that work so the
splice
behaves consistently regardless of how the constructor is invoked.
### Does this PR introduce _any_ user-facing change?
Yes. Nested JSON constructors reached through routed or qualified
`JSON_ARRAY` /
`JSON_OBJECT` calls now splice their raw JSON output rather than embedding a
quoted
string. The value was previously incorrect; this only affects unreleased
master.
### How was this patch tested?
New and extended cases in `JsonArraySuite` and `JsonObjectSuite` covering
the splice,
round-trip, nested-shadow, bare-star-in-nested, and non-default-collation
splice
scenarios, plus `SparkSessionExtensionSuite` and `ResolverGuardSuite`
coverage for the
new unresolved node. Catalyst compile and lint (100-col / non-ASCII) are
clean.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Isaac
This pull request and its description were written by Isaac.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
