chenhao-db commented on code in PR #46011: URL: https://github.com/apache/spark/pull/46011#discussion_r1561844116
########## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/variant/VariantExpressionEvalUtils.scala: ########## @@ -41,4 +41,15 @@ object VariantExpressionEvalUtils { input.toString, BadRecordException(() => input, cause = e)) } } + + def isVariantNull(input: VariantVal): Boolean = { + if (input == null) { + // This is a SQL NULL, not a Variant NULL + false + } else { + val valMeta = input.getValue Review Comment: I think `valMeta` is a confusing name. You are really just checking the `value`. ########## sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/variant/VariantExpressionEvalUtilsSuite.scala: ########## @@ -122,4 +122,42 @@ class VariantExpressionEvalUtilsSuite extends SparkFunSuite { Map("sizeLimit" -> "16.0 MiB", "functionName" -> "`parse_json`")) } } + + test("isVariantNull") { Review Comment: Recommend to also add SQL tests in `VariantEndToEndSuite`. ########## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala: ########## @@ -822,6 +822,7 @@ object FunctionRegistry { // Variant expression[ParseJson]("parse_json"), + expression[IsVariantNull]("is_variant_null"), Review Comment: I guess you need to look at `ExpressionsSchemaSuite`. It will fail if you only register here but haven't modified `sql-expression-schema.md`. ########## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/variant/variantExpressions.scala: ########## @@ -73,6 +73,45 @@ case class ParseJson(child: Expression) copy(child = newChild) } +@ExpressionDescription( + usage = "_FUNC_(expr) - Check if a variant value is a variant NULL.", Review Comment: We'd better be explicit in the doc: it returns true if and only if the input is a variant null (conventionally, we use lower case to refer to a variant null). For all other cases, including the input is a SQL NULL, we return false. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org