cshuo commented on code in PR #19831:
URL: https://github.com/apache/hudi/pull/19831#discussion_r3930671014
##########
hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/io/storage/row/lance/HoodieFlinkLanceArrowUtils.java:
##########
@@ -180,6 +197,36 @@ private static Field toArrowField(String name, LogicalType
type) {
return new Field(name, new FieldType(type.isNullable(), toArrowType(type),
null), children);
}
+ private static Field toArrowField(String name, LogicalType type,
HoodieSchema hoodieSchema) {
+ if (hoodieSchema.getType() != HoodieSchemaType.VECTOR) {
+ return toArrowField(name, type);
+ }
+
+ HoodieSchema.Vector vectorSchema = (HoodieSchema.Vector) hoodieSchema;
+ validateLanceVector(name, vectorSchema);
+ ValidationUtils.checkArgument(type instanceof ArrayType,
+ "VECTOR column '" + name + "' must map to a Flink ARRAY type");
+ ArrayType arrayType = (ArrayType) type;
+ Field elementField = toArrowField("element", arrayType.getElementType());
+ return new Field(
+ name,
+ new FieldType(
+ type.isNullable(),
+ new ArrowType.FixedSizeList(vectorSchema.getDimension()),
Review Comment:
VECTOR field reading support will be added in a separate follow-up PR,
tracked by #19821. This PR is scoped to the writer changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
