JacksonKjar commented on code in PR #59215:
URL: https://github.com/apache/spark/pull/59215#discussion_r4172587887
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/DeserializerBuildHelper.scala:
##########
@@ -448,6 +448,25 @@ object DeserializerBuildHelper {
exprs.Literal.create(null, dt),
NewInstance(cls, arguments, Nil, propagateNull = false, dt,
outerPointerGetter))
+ case AgnosticEncoders.JavaRecordEncoder(tag, fields) =>
+ val cls = tag.runtimeClass
+ val dt = ObjectType(cls)
+ val arguments = fields.zipWithIndex.map {
+ case (field, i) =>
+ val newTypePath = walkedTypePath.recordField(
+ field.enc.clsTag.runtimeClass.getName,
+ field.name)
+ val getter = addToPath(path, field.name, field.enc.dataType,
newTypePath)
+ expressionWithNullSafety(
+ createDeserializer(field.enc, getter, newTypePath),
+ field.enc.nullable,
Review Comment:
Good catch, thanks. Fixed in 4672fc15bb2: the record deserializer now uses
`field.nullable` like the JavaBean arm, so a null in a `@Nonnull` component
fails with `NOT_NULL_ASSERT_VIOLATION`.
Added a decode test in `JavaRecordEncoderSuite` (codegen and interpreted)
and your as(...) example to `JavaRecordDatasetSuite`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
