ganeshashree commented on code in PR #58034:
URL: https://github.com/apache/spark/pull/58034#discussion_r4257264324
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala:
##########
@@ -2272,3 +2331,424 @@ case class JsonTypeof(child: Expression)
override protected def withNewChildInternal(newChild: Expression):
JsonTypeof =
copy(child = newChild)
}
+
+/**
+ * The SQL:2016 `JSON_OBJECT` constructor function (feature T811): constructs
a JSON object from
+ * key-value pairs, written `key VALUE value`, `KEY key VALUE value`, `key :
value`, or the
+ * MySQL-style `key, value`.
+ *
+ * Keys must be non-null strings; a null key is an error. The `ON NULL` clause
controls whether
+ * null-valued pairs are included (NULL ON NULL, the standard default) or
omitted (ABSENT ON NULL).
+ *
+ * A value is spliced in raw (unquoted) when it carries `FORMAT JSON` (tracked
per member by
+ * `rawJson`): the explicit clause (e.g. `JSON_OBJECT('a' VALUE '{"b":1}'
FORMAT JSON)`), or the
+ * implicit form carried by a nested JSON constructor / `JSON_QUERY` under
KEEP QUOTES (see
+ * `rawJsonValue`). Explicit `FORMAT JSON` on a non-constructor is arbitrary
user text, so
+ * `needsValidation` marks it for JSON validation at eval; a nested
constructor is trusted.
+ *
+ * Examples:
+ * JSON_OBJECT('id' VALUE 7, 'name' VALUE 'Ada') ->
'{"id":7,"name":"Ada"}'
+ * JSON_OBJECT('id': 7, 'v': NULL) -> '{"id":7,"v":null}'
+ * JSON_OBJECT('id': 7, 'v': NULL ABSENT ON NULL) -> '{"id":7}'
+ * JSON_OBJECT() -> '{}'
+ *
+ * A clause-free call that is not itself lexically nested in an enclosing
constructor's *value*
+ * position routes through function resolution and is rebuilt by
`JsonObjectExpressionBuilder`, so a
+ * same-named routine can shadow the built-in `json_object`: always for the
comma form (a nested
+ * producer value routes too, carrying its raw-splice eligibility via
`JsonImplicitFormatCarrier`),
+ * but for the `VALUE` / `:` forms only when no member carries a raw value
(`rawJson.contains(true)`
+ * -- an explicit `FORMAT JSON` or a nested producer in *value* position --
keeps it direct).
+ * Otherwise it is built directly from the grammar (clause-bearing, or
value-nested); a nested
+ * *key*-position call stays routable (see `AstBuilder.visitJsonObject`).
+ * The user-facing reference lives in
`docs/sql-ref-syntax-qry-select-json-object.md`.
+ */
+case class JsonObjectExpr(
+ members: Seq[(Expression, Expression)],
+ rawJson: Seq[Boolean],
+ needsValidation: Seq[Boolean],
+ nullBehavior: JsonConstructorNullBehavior =
JsonConstructorNullBehavior.Null,
+ returning: DataType = StringType,
+ timeZoneId: Option[String] = None)
+ extends Expression
+ with TimeZoneAwareExpression
+ with CodegenFallback
Review Comment:
Sounds good, I'll keep it in this PR.
Before I start, I want to check on scope. To write the whole object through
a single lifecycle, I'll need to add new public API to `JacksonGenerator`.
Right now it only exposes the whole-container `write(...)` methods and keeps
all the member-level writing private, which is the reason each member currently
goes through its own one-element array. That generator is also used by
`to_json`, the JSON file-source writer, and `Dataset.toJSON`, so changing it is
a bit broader than just the JSON_OBJECT code.
So two questions:
1. Do you want the full single-evaluator rewrite (including the
`JacksonGenerator` change) here, or is it enough for this PR to restore
whole-stage codegen and cut the per-row allocations, and leave the
`JacksonGenerator` member-write API as a separate change?
2. Should I convert `JSON_ARRAY` to the same path in this PR too so the two
stay consistent, or leave it for later?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
