wombatu-kun commented on code in PR #20760:
URL: https://github.com/apache/hudi/pull/20760#discussion_r4652233298


##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/io/storage/row/HoodieRowParquetWriteSupport.java:
##########
@@ -789,9 +791,11 @@ private Type convertField(HoodieSchema fieldSchema, 
StructField structField, Typ
     } else if (dataType == DataTypes.DateType) {
       return Types.primitive(INT32, repetition)
           .as(LogicalTypeAnnotation.dateType()).named(structField.name());
-    } else if (dataType == DataTypes.ShortType) {
-      return Types.primitive(INT32, repetition)
-          .as(LogicalTypeAnnotation.intType(16, 
true)).named(structField.name());
+    } else if (dataType == DataTypes.ByteType || dataType == 
DataTypes.ShortType) {
+      // Plain INT32 without an INT(8) / INT(16) annotation: the table schema 
has no 8- or 16-bit
+      // integer type and records these columns as int, which is what every 
other write path writes

Review Comment:
   The claim that every other write path emits plain INT32 is too broad because 
Flink's `ParquetSchemaConverter` still writes TINYINT/SMALLINT with 
INT(8)/INT(16). Scope this and the matching test comment to other Spark write 
paths.



##########
hudi-common/src/test/java/org/apache/hudi/metadata/stats/TestValueType.java:
##########
@@ -252,6 +253,26 @@ public void testFromParquetPrimitiveType() {
     assertEquals(ValueType.DOUBLE, 
valueTypeFor(PrimitiveType.PrimitiveTypeName.DOUBLE));
   }
 
+  @Test
+  public void testFromParquetIntegerAnnotations() {

Review Comment:
   The new `OriginalTypeParquetAdapter` integer mappings are not exercised 
because these tests go through `ValueType`, which selects 
`LogicalTypeParquetAdapter` with the current Parquet dependency. Add direct 
assertions against `OriginalTypeParquetAdapter` so the pre-1.11 compatibility 
path is covered.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to