[ https://issues.apache.org/jira/browse/NIFI-11449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17712091#comment-17712091 ]
Abdelrahim Ahmad commented on NIFI-11449: ----------------------------------------- Improving this process will allow Nifi to be the main tool to ingest data into these new Technologies. So we don't have to deal with another way to ingest data. > add autocommit property to control commit in PutDatabaseRecord processor > ------------------------------------------------------------------------ > > Key: NIFI-11449 > URL: https://issues.apache.org/jira/browse/NIFI-11449 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions > Affects Versions: 1.21.0 > Environment: Any Nifi Deployment > Reporter: Abdelrahim Ahmad > Priority: Blocker > Labels: Trino, autocommit, database, iceberg, putdatabaserecord > > The issue is with the {{PutDatabaseRecord}} processor in Apache NiFi. When > using the processor with the Trino-JDBC-Driver to write to an Iceberg > catalog, it disables the autocommit feature. This leads to errors such as > "{*}Catalog only supports writes using autocommit: iceberg{*}". > To fix this issue, the autocommit feature needs to be added in the processor > to be enabled/disabled. > enabling auto-commit in the Nifi PutDatabaseRecord processor is important for > Deltalake, Iceberg, and Hudi as it ensures data consistency and integrity by > allowing atomic writes to be performed in the underlying database. This will > allow the process to be widely used with bigger range of databases. > P.S.: using PutSQL is not a good option at all due to the sensitivity of > these tables when dealing with small inserts. > Thanks and best regards > Abdelrahim Ahmad -- This message was sent by Atlassian Jira (v8.20.10#820010)