henrisaksi opened a new pull request, #11749: URL: https://github.com/apache/nifi/pull/11749
<!-- Licensed to the Apache Software Foundation (ASF) under one or more --> <!-- contributor license agreements. See the NOTICE file distributed with --> <!-- this work for additional information regarding copyright ownership. --> <!-- The ASF licenses this file to You under the Apache License, Version 2.0 --> <!-- (the "License"); you may not use this file except in compliance with --> <!-- the License. You may obtain a copy of the License at --> <!-- http://www.apache.org/licenses/LICENSE-2.0 --> <!-- Unless required by applicable law or agreed to in writing, software --> <!-- distributed under the License is distributed on an "AS IS" BASIS, --> <!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. --> <!-- See the License for the specific language governing permissions and --> <!-- limitations under the License. --> # Summary [NIFI-16074](https://issues.apache.org/jira/browse/NIFI-16074) `PutIcebergRecord` passes NiFi byte values (`byte[]` or `Object[]` of `Byte`) to Iceberg unconverted, so writes to `binary`, `fixed`, `geometry` and `geography` columns fail with a `ClassCastException`, such as `[Ljava.lang.Object; cannot be cast to java.nio.ByteBuffer`. String values for `uuid` columns fail the same way with `String cannot be cast to UUID`. This change converts these values in `RecordConverter` by the target Iceberg column type, including values nested in structs, lists and maps: | Iceberg type | Converted from | Converted to | |---|---|---| | `binary`, `geometry`, `geography` | `byte[]`, `Object[]` of `Byte` | `ByteBuffer` | | `fixed` | `Object[]` of `Byte`, `ByteBuffer` | `byte[]` | | `uuid` | `String`, `byte[]`, `Object[]` of `Byte` | `UUID` | Values that already have the target type, and null values, are unchanged. Records are now also converted when the table has a `uuid` column, because UUIDs are often read as Strings, which do not otherwise trigger Record conversion. Iceberg 1.12 ([apache/iceberg#16982](https://github.com/apache/iceberg/pull/16982), upgraded in NIFI-16407) writes `geometry` and `geography` values from Well-Known Binary in a `ByteBuffer`, so this change also allows `PutIcebergRecord` to write native geospatial columns in format-version 3 tables. This completes the bytes and UUID parts of NIFI-16074, after the timestamp, list and struct parts were resolved in NIFI-16069. Unlike #11423, the change is limited to these value conversions and leaves the existing timestamp, date, time, list, struct and map handling unchanged. ### Tests - `RecordConverterTest`: conversion of each type from `byte[]`, `Object[]`, `ByteBuffer`, `String` and null values, values nested in lists, maps and structs, and String UUIDs in Records without other fields requiring conversion - `DelegatedRecordTest`: `binary`, `fixed` and `uuid` values read through `DelegatedRecord` - `ParquetIcebergWriterTest`: writing `binary`, `fixed` and `uuid` values with `ParquetIcebergWriter`, and `geometry` and `geography` values to a format-version 3 table, checking the column bounds that Iceberg records # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-00000` - Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-00000` - Pull request contains [commits signed](https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits) with a registered key indicating `Verified` status ### Pull Request Formatting - Pull Request based on current revision of the `main` branch - Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [x] Build completed using `./mvnw clean install -P contrib-check` for the changed modules (`-pl nifi-extension-bundles/nifi-iceberg-bundle/nifi-iceberg-processors,nifi-extension-bundles/nifi-iceberg-bundle/nifi-iceberg-parquet-writer`) - [x] JDK 21 - [ ] JDK 25 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files No new dependencies. ### Documentation - [ ] Documentation formatting appears as expected in rendered files 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
