Sönke Liebau created NIFI-15866:
-----------------------------------
Summary: nserting Date values in Iceberg tables
Key: NIFI-15866
URL: https://issues.apache.org/jira/browse/NIFI-15866
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 2.9.0
Reporter: Sönke Liebau
The PutIcebergRecord processor currently does not support columns with type
Date.
When inserting in a table with a Date column, each time a ClassCastException
between java.sql.Date (from the Flowfile) and java.time.LocalDate (required
from PutIcebergRecord) is thrown.
This behavior occurs with Avro as well as Parquet and their specific readers
configured in PutIcebergRecord, with a hardcorded Avro Schema referencing the
Date column.
This bug is very similar to an already fixed bug regarding
Datetimes/Timestamps: https://issues.apache.org/jira/browse/NIFI-15568.
For reproducing the bug in NiFi:
* Create an Iceberg table with one Date column
* Generate a record flowfile with GenerateFlowFile containing a value for this
column, for example as CSV or JSON. Give an explicit Avro Schema as attribute
in the GenerateFlowfile:
+CSV:+
dateCol
234234
+Avro Schema:+
{
"type": "record",
"name": "Document",
"namespace": "com.example",
"fields": [
{
"name": "dateCol",
"type": ["null", \{"type": "int", "logicalType": "date"} ]
}
]
}
* Use a ConvertRecord to transform the CSV to an Avro flowfile, using a
AvroRecordSetWriter with setting „Use ‚Schema Text‘ Property“
* Write into Iceberg with PutIcebergRecord and a default AvroReader
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
