dan-s1 commented on code in PR #10850:
URL: https://github.com/apache/nifi/pull/10850#discussion_r2764926834
##########
nifi-extension-bundles/nifi-parquet-bundle/nifi-parquet-processors/src/test/java/org/apache/nifi/parquet/ParquetTestUtils.java:
##########
@@ -89,5 +93,247 @@ private static ParquetWriter<GenericRecord>
createParquetWriter(final Schema sch
.build();
}
+ /**
+ * Creates a parquet file with timestamp_millis logical type for testing
NIFI-15548.
+ *
+ * @param numRecords Number of records to create
+ * @return The created parquet file
+ * @throws IOException if file creation fails
+ */
+ public static File createTimestampParquetFile(int numRecords) throws
IOException {
+ // Create schema with timestamp_millis logical type
+ final Schema timestampSchema =
LogicalTypes.timestampMillis().addToSchema(Schema.create(Schema.Type.LONG));
+
+ final Schema recordSchema = Schema.createRecord("TimestampRecord",
null, "test", false);
+ recordSchema.setFields(java.util.List.of(
Review Comment:
Include import for ` java.util.List` and shorten the code here and in the
other places below this is used.
```suggestion
recordSchema.setFields(List.of(
```
##########
nifi-extension-bundles/nifi-extension-utils/nifi-record-utils/nifi-avro-record-utils/src/main/java/org/apache/nifi/avro/AvroTypeUtil.java:
##########
@@ -1155,9 +1155,18 @@ private static Object normalizeValue(final Object value,
final Schema avroSchema
final String logicalName = logicalType.getName();
if (LOGICAL_TYPE_DATE.equals(logicalName)) {
// date logical name means that the value is number of
days since Jan 1, 1970
+ // Handle both Integer (legacy) and LocalDate (newer Avro
libraries)
+ if (value instanceof LocalDate) {
+ return java.sql.Date.valueOf((LocalDate) value);
+ }
return java.sql.Date.valueOf(LocalDate.ofEpochDay((int)
value));
} else if (LOGICAL_TYPE_TIME_MILLIS.equals(logicalName)) {
// time-millis logical name means that the value is number
of milliseconds since midnight.
+ // Handle both Integer (legacy) and LocalTime (newer Avro
libraries)
+ if (value instanceof LocalTime) {
+ final LocalTime localTime = (LocalTime) value;
+ return new Time(localTime.toNanoOfDay() / 1_000_000);
+ }
Review Comment:
Intellij suggests replacing with pattern variable
```suggestion
if (value instanceof LocalTime localTime) {
return new Time(localTime.toNanoOfDay() / 1_000_000);
}
```
##########
nifi-extension-bundles/nifi-extension-utils/nifi-record-utils/nifi-avro-record-utils/src/main/java/org/apache/nifi/avro/AvroTypeUtil.java:
##########
@@ -1171,10 +1180,23 @@ private static Object normalizeValue(final Object
value, final Schema avroSchema
final String logicalName = logicalType.getName();
if (LOGICAL_TYPE_TIME_MICROS.equals(logicalName)) {
+ // Handle both Long (legacy) and LocalTime (newer Avro
libraries)
+ if (value instanceof LocalTime) {
+ final LocalTime localTime = (LocalTime) value;
+ return new Time(localTime.toNanoOfDay() / 1_000_000);
+ }
Review Comment:
Intellij suggests using a pattern variable.
```suggestion
if (value instanceof LocalTime localTime) {
return new Time(localTime.toNanoOfDay() / 1_000_000);
}
```
##########
nifi-extension-bundles/nifi-parquet-bundle/nifi-parquet-processors/src/test/java/org/apache/nifi/parquet/TestParquetReader.java:
##########
@@ -217,6 +218,217 @@ public void testReaderWithAddListElementRecordsDisabled()
throws IOException, In
"MapRecord[{name=Bob, favorite_number=1,
favorite_colors=[blue, red, yellow]}]");
}
+ /**
+ * Test for NIFI-15548: ParquetReader fails on timestamps.
+ * This test programmatically creates a parquet file with timestamp_millis
logical type
+ * and verifies that the timestamp values can be read without
ClassCastException.
+ */
+ @Test
+ public void testReadParquetWithTimestampMillis() throws IOException,
MalformedRecordException {
+ final int numRecords = 5;
+ final File parquetFile =
ParquetTestUtils.createTimestampParquetFile(numRecords);
Review Comment:
Could you pass a File object annotated with a JUnit5 `TempDir` annotation so
it would handle the cleanup of any files (hence not needing the finally
statements) to delete the created file?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
