hudi-agent commented on code in PR #19056:
URL: https://github.com/apache/hudi/pull/19056#discussion_r3526792047
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSchemaUtils.scala:
##########
@@ -306,4 +308,34 @@ object HoodieSchemaUtils {
s" tableSchemaFields: $tableSchemaPartitionFields, partitionFields:
$partitionFields.")
}
}
+
+ /**
+ * Checks whether a Spark [[StructField]] represents a Hudi BLOB column.
BLOB is a logical
+ * type stored as a struct, identified by the
[[HoodieSchema.TYPE_METADATA_FIELD]] metadata
+ * attached to the field. Applies to both INLINE and OUT_OF_LINE (EXTERNAL)
blobs.
+ */
Review Comment:
🤖 nit: `StructField.metadata` in Spark is always non-null — it's initialised
to `Metadata.empty` by the constructor when not explicitly provided. The `md !=
null` guard is always `true` and could mislead a future reader into thinking
`null` metadata is a real case to defend against. Could you drop it and just
call `field.metadata.contains(...)`?
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag
quality.</i></sub>
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/HoodieOptionConfig.scala:
##########
@@ -232,13 +233,23 @@ object HoodieOptionConfig {
ValidationUtils.checkArgument(schema.exists(f => resolver(f.name,
getRootLevelFieldName(primaryKey))),
s"Can't find primaryKey `$primaryKey` in ${schema.treeString}.")
}
+ val pkBlobs = HoodieSchemaUtils.findBlobFields(schema,
primaryKeys.get.mkString(","), resolver)
+ ValidationUtils.checkArgument(pkBlobs.isEmpty,
+ s"BLOB type column(s) ${pkBlobs.mkString("[", ", ", "]")} cannot be
used as primaryKey (record key). " +
+ "BLOB columns hold large binary payloads (INLINE or EXTERNAL) and
are not supported as " +
Review Comment:
🤖 nit: the sentence `"BLOB columns hold large binary payloads (INLINE or
EXTERNAL) and are not supported as record key, ordering/preCombine, or
partition path fields."` is copy-pasted 4 times across 3 files
(`HoodieOptionConfig`, `HoodieCatalogTable`, `HoodieSparkSqlWriter`). It might
be worth extracting it as a private constant in `HoodieSchemaUtils` (e.g.
`BLOB_KEY_USAGE_ERROR_HINT`) so a future wording change only needs to happen in
one place.
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag
quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
