[nifi] branch main updated: NIFI-11271: This closes #7795. Removed deprecated Kerberos-related properties and updated to make use of KerberosUserService; some code cleanup to bring up-to-date with Jav
This is an automated email from the ASF dual-hosted git repository.
joewitt pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/nifi.git
The following commit(s) were added to refs/heads/main by this push:
new 16c527271b NIFI-11271: This closes #7795. Removed deprecated
Kerberos-related properties and updated to make use of KerberosUserService;
some code cleanup to bring up-to-date with Java 21 recommendations such as
Stream.toList() instead of Stream.collect(Collectors.toList()) and using
enhanced switch statements
16c527271b is described below
commit 16c527271bbfc2f83b251383880dcb78972ce7e4
Author: Mark Payne
AuthorDate: Tue Sep 26 11:42:22 2023 -0400
NIFI-11271: This closes #7795. Removed deprecated Kerberos-related
properties and updated to make use of KerberosUserService; some code cleanup to
bring up-to-date with Java 21 recommendations such as Stream.toList() instead
of Stream.collect(Collectors.toList()) and using enhanced switch statements
Signed-off-by: Joseph Witt
---
.../nifi-kudu-controller-service/pom.xml | 11 +-
.../nifi/controller/kudu/KuduLookupService.java| 164 +---
.../processors/kudu/AbstractKuduProcessor.java | 296 +
.../org/apache/nifi/processors/kudu/PutKudu.java | 88 +++---
.../apache/nifi/processors/kudu/MockPutKudu.java | 146 ++
.../apache/nifi/processors/kudu/TestPutKudu.java | 134 ++
6 files changed, 203 insertions(+), 636 deletions(-)
diff --git
a/nifi-nar-bundles/nifi-kudu-bundle/nifi-kudu-controller-service/pom.xml
b/nifi-nar-bundles/nifi-kudu-bundle/nifi-kudu-controller-service/pom.xml
index 1d23041702..36c48656be 100644
--- a/nifi-nar-bundles/nifi-kudu-bundle/nifi-kudu-controller-service/pom.xml
+++ b/nifi-nar-bundles/nifi-kudu-bundle/nifi-kudu-controller-service/pom.xml
@@ -63,11 +63,6 @@
2.0.0-SNAPSHOT
provided
-
-org.apache.nifi
-nifi-kerberos-credentials-service-api
-provided
-
org.apache.nifi
nifi-record
@@ -88,6 +83,12 @@
kudu-client
${kudu.version}
+
+org.apache.nifi
+nifi-kerberos-user-service-api
+2.0.0-SNAPSHOT
+provided
+
diff --git
a/nifi-nar-bundles/nifi-kudu-bundle/nifi-kudu-controller-service/src/main/java/org/apache/nifi/controller/kudu/KuduLookupService.java
b/nifi-nar-bundles/nifi-kudu-bundle/nifi-kudu-controller-service/src/main/java/org/apache/nifi/controller/kudu/KuduLookupService.java
index b208186fac..a53444340d 100644
---
a/nifi-nar-bundles/nifi-kudu-bundle/nifi-kudu-controller-service/src/main/java/org/apache/nifi/controller/kudu/KuduLookupService.java
+++
b/nifi-nar-bundles/nifi-kudu-bundle/nifi-kudu-controller-service/src/main/java/org/apache/nifi/controller/kudu/KuduLookupService.java
@@ -16,6 +16,18 @@
*/
package org.apache.nifi.controller.kudu;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Base64;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
+import javax.security.auth.login.LoginException;
import org.apache.kudu.ColumnSchema;
import org.apache.kudu.ColumnTypeAttributes;
import org.apache.kudu.Schema;
@@ -38,12 +50,11 @@ import org.apache.nifi.controller.AbstractControllerService;
import org.apache.nifi.controller.ConfigurationContext;
import org.apache.nifi.controller.ControllerServiceInitializationContext;
import org.apache.nifi.expression.ExpressionLanguageScope;
-import org.apache.nifi.kerberos.KerberosCredentialsService;
+import org.apache.nifi.kerberos.KerberosUserService;
import org.apache.nifi.lookup.RecordLookupService;
import org.apache.nifi.processor.util.StandardValidators;
import org.apache.nifi.reporting.InitializationException;
import org.apache.nifi.security.krb.KerberosAction;
-import org.apache.nifi.security.krb.KerberosKeytabUser;
import org.apache.nifi.security.krb.KerberosUser;
import org.apache.nifi.serialization.SimpleRecordSchema;
import org.apache.nifi.serialization.record.MapRecord;
@@ -52,20 +63,6 @@ import org.apache.nifi.serialization.record.RecordField;
import org.apache.nifi.serialization.record.RecordFieldType;
import org.apache.nifi.serialization.record.RecordSchema;
-import javax.security.auth.login.LoginException;
-import java.util.ArrayList;
-import java.util.Arrays;
-import java.util.Base64;
-import java.util.Collections;
-import java.util.HashMap;
-import java.util.HashSet;
-import java.util.List;
-import java.util.Map;
-import java.util.Optional;
-import java.util.Set;
-import java.util.concurrent.TimeUnit;
-import java.util.stream.Collectors;
-
@CapabilityDescription("Lookup a record
[nifi] branch main updated: NIFI-12074 updated images to base on Java 21 and added Python3 (#7785)
This is an automated email from the ASF dual-hosted git repository.
markap14 pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/nifi.git
The following commit(s) were added to refs/heads/main by this push:
new ff05a5d158 NIFI-12074 updated images to base on Java 21 and added
Python3 (#7785)
ff05a5d158 is described below
commit ff05a5d158137c83034368a031b251ab066e9397
Author: Joe Witt
AuthorDate: Tue Sep 26 07:08:32 2023 -0700
NIFI-12074 updated images to base on Java 21 and added Python3 (#7785)
* NIFI-12074 updated images to base on Java 21 and added Python3
* NIFI-12074 ensure python env included, enable python extension mount
---
nifi-docker/dockerhub/DockerRun.sh | 2 +-
nifi-docker/dockerhub/Dockerfile| 11 +++---
nifi-docker/dockerhub/README.md | 5 -
nifi-docker/dockerhub/sh/start.sh | 32 ++---
nifi-docker/dockermaven/Dockerfile | 17 +--
nifi-docker/dockermaven/integration-test.sh | 6 --
nifi-docker/dockermaven/pom.xml | 1 +
nifi-docker/pom.xml | 4 ++--
8 files changed, 37 insertions(+), 41 deletions(-)
diff --git a/nifi-docker/dockerhub/DockerRun.sh
b/nifi-docker/dockerhub/DockerRun.sh
index 97d0092cfc..7554bfaa78 100755
--- a/nifi-docker/dockerhub/DockerRun.sh
+++ b/nifi-docker/dockerhub/DockerRun.sh
@@ -22,4 +22,4 @@ DOCKER_IMAGE="$(grep -Ev '(^#|^\s*$|^\s*\t*#)'
DockerImage.txt)"
NIFI_IMAGE_VERSION="$(echo "${DOCKER_IMAGE}" | cut -d : -f 2)"
echo "Running Docker Image: ${DOCKER_IMAGE}"
-docker run -d --name "nifi-${NIFI_IMAGE_VERSION}" -p 8080:8080 -p 8443:8443 -p
1:1 -p 8000:8000 -p 8181:8181 "${DOCKER_IMAGE}"
+docker run -d --name "nifi-${NIFI_IMAGE_VERSION}" -p 8443:8443 -p 1:1
-p 8000:8000 "${DOCKER_IMAGE}"
diff --git a/nifi-docker/dockerhub/Dockerfile b/nifi-docker/dockerhub/Dockerfile
index 61db651783..d77d57d349 100644
--- a/nifi-docker/dockerhub/Dockerfile
+++ b/nifi-docker/dockerhub/Dockerfile
@@ -1,3 +1,4 @@
+# syntax=docker/dockerfile:1
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
@@ -16,8 +17,8 @@
# under the License.
#
-ARG IMAGE_NAME=eclipse-temurin
-ARG IMAGE_TAG=11-jre
+ARG IMAGE_NAME=bellsoft/liberica-openjdk-alpine
+ARG IMAGE_TAG=21
FROM ${IMAGE_NAME}:${IMAGE_TAG}
ARG MAINTAINER="Apache NiFi "
LABEL maintainer="${MAINTAINER}"
@@ -48,6 +49,8 @@ RUN groupadd -g ${GID} nifi || groupmod -n nifi `getent group
${GID} | cut -d: -
&& chown -R nifi:nifi ${NIFI_BASE_DIR} \
&& apt-get update \
&& apt-get install -y jq xmlstarlet procps unzip \
+&& apt-get install -y python3 \
+&& apt-get install -y python3-pip \
&& apt-get -y autoremove \
&& apt-get clean autoclean \
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
@@ -73,6 +76,7 @@ RUN curl -fSL ${MIRROR_BASE_URL}/${NIFI_BINARY_PATH} -o
${NIFI_BASE_DIR}/nifi-${
&& mkdir -p ${NIFI_HOME}/flowfile_repository \
&& mkdir -p ${NIFI_HOME}/content_repository \
&& mkdir -p ${NIFI_HOME}/provenance_repository \
+&& mkdir -p ${NIFI_HOME}/python_extensions \
&& mkdir -p ${NIFI_HOME}/state \
&& mkdir -p ${NIFI_LOG_DIR} \
&& ln -s ${NIFI_HOME} ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}
@@ -83,13 +87,14 @@ VOLUME ${NIFI_LOG_DIR} \
${NIFI_HOME}/flowfile_repository \
${NIFI_HOME}/content_repository \
${NIFI_HOME}/provenance_repository \
+ ${NIFI_HOME}/python_extensions \
${NIFI_HOME}/state
# Clear nifi-env.sh in favour of configuring all environment variables in the
Dockerfile
RUN echo "#!/bin/sh\n" > $NIFI_HOME/bin/nifi-env.sh
# Web HTTP(s) & Socket Site-to-Site Ports
-EXPOSE 8080 8443 1 8000
+EXPOSE 8443/tcp 1/tcp 8000/tcp
WORKDIR ${NIFI_HOME}
diff --git a/nifi-docker/dockerhub/README.md b/nifi-docker/dockerhub/README.md
index 6a4c75e9cc..f7113a0340 100644
--- a/nifi-docker/dockerhub/README.md
+++ b/nifi-docker/dockerhub/README.md
@@ -15,6 +15,10 @@
## Latest changes
+### 2.0.0
+
+- Changed base image to bellsoft/liberica-openjdk-debian:21 as NiFi 2.0.0
requires Java 21
+
### 1.19.0
- Changed base image to eclipse-temurin:11-jre as openjdk:8-jre is no longer
maintained
@@ -215,7 +219,6 @@ can be published to the host.
| Function | Property | Port |
|--|---|---|
-| HTTP Port| nifi.web.http.port| 8080 |
| HTTPS Port | nifi.web.https.port | 8443 |
| Remote Input Socket Port | nifi.remote.input.socket.port | 1 |
| JVM Debugger | java.arg.debug| 8000 |
diff --git a/nifi-docker/dockerhub/sh/start.sh
b/nifi-docker/dockerhub/sh/start.sh
index ea4319ebc9..b343945513
[nifi] branch support/nifi-1.x updated: NIFI-12117 Allow configuring Jackson's max string length in JoltTransformJSON
This is an automated email from the ASF dual-hosted git repository.
pvillard pushed a commit to branch support/nifi-1.x
in repository https://gitbox.apache.org/repos/asf/nifi.git
The following commit(s) were added to refs/heads/support/nifi-1.x by this push:
new 31850d10fa NIFI-12117 Allow configuring Jackson's max string length
in JoltTransformJSON
31850d10fa is described below
commit 31850d10fafbd016c23c704d897b50a120bf79c9
Author: Bryan Bende
AuthorDate: Mon Sep 25 11:44:33 2023 -0400
NIFI-12117 Allow configuring Jackson's max string length in
JoltTransformJSON
Signed-off-by: Pierre Villard
This closes #7788.
---
.../processors/standard/JoltTransformJSON.java | 30 +++---
1 file changed, 26 insertions(+), 4 deletions(-)
diff --git
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/JoltTransformJSON.java
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/JoltTransformJSON.java
index 2b66f8937f..bfeb2556db 100644
---
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/JoltTransformJSON.java
+++
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/JoltTransformJSON.java
@@ -17,15 +17,18 @@
package org.apache.nifi.processors.standard;
import com.bazaarvoice.jolt.JoltTransform;
+import com.bazaarvoice.jolt.JsonUtil;
import com.bazaarvoice.jolt.JsonUtils;
+import com.fasterxml.jackson.core.StreamReadConstraints;
+import com.fasterxml.jackson.databind.ObjectMapper;
import com.github.benmanes.caffeine.cache.Cache;
import com.github.benmanes.caffeine.cache.Caffeine;
import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.RequiresInstanceClassLoading;
import org.apache.nifi.annotation.behavior.SideEffectFree;
import org.apache.nifi.annotation.behavior.SupportsBatching;
-import org.apache.nifi.annotation.behavior.InputRequirement;
import org.apache.nifi.annotation.behavior.WritesAttribute;
-import org.apache.nifi.annotation.behavior.RequiresInstanceClassLoading;
import org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.annotation.documentation.Tags;
import org.apache.nifi.annotation.lifecycle.OnScheduled;
@@ -42,6 +45,7 @@ import org.apache.nifi.flowfile.FlowFile;
import org.apache.nifi.flowfile.attributes.CoreAttributes;
import org.apache.nifi.logging.ComponentLog;
import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.DataUnit;
import org.apache.nifi.processor.ProcessContext;
import org.apache.nifi.processor.ProcessSession;
import org.apache.nifi.processor.Relationship;
@@ -153,6 +157,15 @@ public class JoltTransformJSON extends AbstractProcessor {
.defaultValue("false")
.build();
+public static final PropertyDescriptor MAX_STRING_LENGTH = new
PropertyDescriptor.Builder()
+.name("max-string-length")
+.displayName("Max String Length")
+.description("The maximum allowed length of a string value when
parsing the JSON document")
+.required(true)
+.defaultValue("20 MB")
+.addValidator(StandardValidators.DATA_SIZE_VALIDATOR)
+.build();
+
public static final Relationship REL_SUCCESS = new Relationship.Builder()
.name("success")
.description("The FlowFile with transformed content will be routed
to this relationship")
@@ -165,6 +178,7 @@ public class JoltTransformJSON extends AbstractProcessor {
private final static List properties;
private final static Set relationships;
private volatile ClassLoader customClassLoader;
+private volatile JsonUtil jsonUtil;
private final static String DEFAULT_CHARSET = "UTF-8";
/**
@@ -182,6 +196,7 @@ public class JoltTransformJSON extends AbstractProcessor {
_properties.add(JOLT_SPEC);
_properties.add(TRANSFORM_CACHE_SIZE);
_properties.add(PRETTY_PRINT);
+_properties.add(MAX_STRING_LENGTH);
properties = Collections.unmodifiableList(_properties);
final Set _relationships = new HashSet<>();
@@ -287,7 +302,7 @@ public class JoltTransformJSON extends AbstractProcessor {
final Object inputJson;
try (final InputStream in = session.read(original)) {
-inputJson = JsonUtils.jsonToObject(in);
+inputJson = jsonUtil.jsonToObject(in);
} catch (final Exception e) {
logger.error("JSON parsing failed for {}", original, e);
session.transfer(original, REL_FAILURE);
@@ -303,7 +318,7 @@ public class JoltTransformJSON extends AbstractProcessor {
}
final Object
[nifi] branch main updated: NIFI-12117 Allow configuring Jackson's max string length in JoltTransformJSON
This is an automated email from the ASF dual-hosted git repository.
pvillard pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/nifi.git
The following commit(s) were added to refs/heads/main by this push:
new 04e54d5ea7 NIFI-12117 Allow configuring Jackson's max string length
in JoltTransformJSON
04e54d5ea7 is described below
commit 04e54d5ea7f24b470c52fd38e18ae3e0eb2de62d
Author: Bryan Bende
AuthorDate: Mon Sep 25 11:44:33 2023 -0400
NIFI-12117 Allow configuring Jackson's max string length in
JoltTransformJSON
Signed-off-by: Pierre Villard
This closes #7788.
---
.../processors/standard/JoltTransformJSON.java | 30 +++---
1 file changed, 26 insertions(+), 4 deletions(-)
diff --git
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/JoltTransformJSON.java
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/JoltTransformJSON.java
index 0f7f960b55..5a3ae9fca8 100644
---
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/JoltTransformJSON.java
+++
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/JoltTransformJSON.java
@@ -17,14 +17,17 @@
package org.apache.nifi.processors.standard;
import com.bazaarvoice.jolt.JoltTransform;
+import com.bazaarvoice.jolt.JsonUtil;
import com.bazaarvoice.jolt.JsonUtils;
+import com.fasterxml.jackson.core.StreamReadConstraints;
+import com.fasterxml.jackson.databind.ObjectMapper;
import com.github.benmanes.caffeine.cache.Cache;
import com.github.benmanes.caffeine.cache.Caffeine;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.RequiresInstanceClassLoading;
import org.apache.nifi.annotation.behavior.SideEffectFree;
import org.apache.nifi.annotation.behavior.SupportsBatching;
-import org.apache.nifi.annotation.behavior.InputRequirement;
import org.apache.nifi.annotation.behavior.WritesAttribute;
-import org.apache.nifi.annotation.behavior.RequiresInstanceClassLoading;
import org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.annotation.documentation.Tags;
import org.apache.nifi.annotation.lifecycle.OnScheduled;
@@ -41,6 +44,7 @@ import org.apache.nifi.flowfile.FlowFile;
import org.apache.nifi.flowfile.attributes.CoreAttributes;
import org.apache.nifi.logging.ComponentLog;
import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.DataUnit;
import org.apache.nifi.processor.ProcessContext;
import org.apache.nifi.processor.ProcessSession;
import org.apache.nifi.processor.Relationship;
@@ -151,6 +155,15 @@ public class JoltTransformJSON extends AbstractProcessor {
.defaultValue("false")
.build();
+public static final PropertyDescriptor MAX_STRING_LENGTH = new
PropertyDescriptor.Builder()
+.name("max-string-length")
+.displayName("Max String Length")
+.description("The maximum allowed length of a string value when
parsing the JSON document")
+.required(true)
+.defaultValue("20 MB")
+.addValidator(StandardValidators.DATA_SIZE_VALIDATOR)
+.build();
+
public static final Relationship REL_SUCCESS = new Relationship.Builder()
.name("success")
.description("The FlowFile with transformed content will be routed
to this relationship")
@@ -163,6 +176,7 @@ public class JoltTransformJSON extends AbstractProcessor {
private final static List properties;
private final static Set relationships;
private volatile ClassLoader customClassLoader;
+private volatile JsonUtil jsonUtil;
private final static String DEFAULT_CHARSET = "UTF-8";
/**
@@ -180,6 +194,7 @@ public class JoltTransformJSON extends AbstractProcessor {
_properties.add(JOLT_SPEC);
_properties.add(TRANSFORM_CACHE_SIZE);
_properties.add(PRETTY_PRINT);
+_properties.add(MAX_STRING_LENGTH);
properties = Collections.unmodifiableList(_properties);
final Set _relationships = new HashSet<>();
@@ -285,7 +300,7 @@ public class JoltTransformJSON extends AbstractProcessor {
final Object inputJson;
try (final InputStream in = session.read(original)) {
-inputJson = JsonUtils.jsonToObject(in);
+inputJson = jsonUtil.jsonToObject(in);
} catch (final Exception e) {
logger.error("JSON parsing failed for {}", original, e);
session.transfer(original, REL_FAILURE);
@@ -301,7 +316,7 @@ public class JoltTransformJSON extends AbstractProcessor {
}
final Object transformedJson = TransformUtils.transform(transform,
inputJson);
-jsonString
[nifi] branch support/nifi-1.x updated: NIFI-12128 updated cassandra, jgit, and netty
This is an automated email from the ASF dual-hosted git repository. pvillard pushed a commit to branch support/nifi-1.x in repository https://gitbox.apache.org/repos/asf/nifi.git The following commit(s) were added to refs/heads/support/nifi-1.x by this push: new efc07f69fc NIFI-12128 updated cassandra, jgit, and netty efc07f69fc is described below commit efc07f69fc3289bc12b972ab3e4aaf218bf8aadf Author: Joseph Witt AuthorDate: Mon Sep 25 16:02:01 2023 -0700 NIFI-12128 updated cassandra, jgit, and netty Signed-off-by: Pierre Villard This closes #7793. --- nifi-nar-bundles/nifi-cassandra-bundle/pom.xml | 2 +- nifi-registry/pom.xml | 2 +- pom.xml| 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/nifi-nar-bundles/nifi-cassandra-bundle/pom.xml b/nifi-nar-bundles/nifi-cassandra-bundle/pom.xml index 5cee9149d5..13d63e17ee 100644 --- a/nifi-nar-bundles/nifi-cassandra-bundle/pom.xml +++ b/nifi-nar-bundles/nifi-cassandra-bundle/pom.xml @@ -23,7 +23,7 @@ -3.11.4 +3.11.5 19.0 diff --git a/nifi-registry/pom.xml b/nifi-registry/pom.xml index 21f2e0648b..bca4bf2f41 100644 --- a/nifi-registry/pom.xml +++ b/nifi-registry/pom.xml @@ -42,7 +42,7 @@ 3.12.0 3.7.0 2.3.2 -5.13.1.202206130422-r +5.13.2.202306221912-r 2.9.2 diff --git a/pom.xml b/pom.xml index aa23ea1b1b..253801eec5 100644 --- a/pom.xml +++ b/pom.xml @@ -143,7 +143,7 @@ 1.3.11 3.10.6.Final 2.2 -4.1.97.Final +4.1.98.Final 5.3.30 5.8.7 1.6.11
[nifi] branch main updated: NIFI-12128 updated cassandra, jgit, and netty
This is an automated email from the ASF dual-hosted git repository. pvillard pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/nifi.git The following commit(s) were added to refs/heads/main by this push: new 81dc36a473 NIFI-12128 updated cassandra, jgit, and netty 81dc36a473 is described below commit 81dc36a473c3a2441e28cc93cc97ba5e0d7bbae0 Author: Joseph Witt AuthorDate: Mon Sep 25 16:02:01 2023 -0700 NIFI-12128 updated cassandra, jgit, and netty Signed-off-by: Pierre Villard This closes #7793. --- nifi-nar-bundles/nifi-cassandra-bundle/pom.xml | 2 +- nifi-registry/pom.xml | 2 +- pom.xml| 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/nifi-nar-bundles/nifi-cassandra-bundle/pom.xml b/nifi-nar-bundles/nifi-cassandra-bundle/pom.xml index 1d665ba0bf..c1d47932a4 100644 --- a/nifi-nar-bundles/nifi-cassandra-bundle/pom.xml +++ b/nifi-nar-bundles/nifi-cassandra-bundle/pom.xml @@ -23,7 +23,7 @@ -3.11.4 +3.11.5 19.0 diff --git a/nifi-registry/pom.xml b/nifi-registry/pom.xml index 23a9db16b8..70277e3952 100644 --- a/nifi-registry/pom.xml +++ b/nifi-registry/pom.xml @@ -41,7 +41,7 @@ 7.0.0 3.12.0 2.3.2 -6.5.0.202303070854-r +6.7.0.202309050840-r 2.10.0 diff --git a/pom.xml b/pom.xml index 1e8b1b6872..6c6e28cdc5 100644 --- a/pom.xml +++ b/pom.xml @@ -142,7 +142,7 @@ 5.5.0 3.10.6.Final 2.2 -4.1.97.Final +4.1.98.Final 5.3.30 5.8.7 1.6.11
[nifi] branch support/nifi-1.x updated: NIFI-12129 - Bump org.xerial.snappy:snappy-java from 1.1.10.3 to 1.1.10.4
This is an automated email from the ASF dual-hosted git repository. pvillard pushed a commit to branch support/nifi-1.x in repository https://gitbox.apache.org/repos/asf/nifi.git The following commit(s) were added to refs/heads/support/nifi-1.x by this push: new 0083f9ef11 NIFI-12129 - Bump org.xerial.snappy:snappy-java from 1.1.10.3 to 1.1.10.4 0083f9ef11 is described below commit 0083f9ef1114827ce04ce3ce1867fd7519dd915a Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> AuthorDate: Mon Sep 25 18:45:42 2023 + NIFI-12129 - Bump org.xerial.snappy:snappy-java from 1.1.10.3 to 1.1.10.4 Bumps [org.xerial.snappy:snappy-java](https://github.com/xerial/snappy-java) from 1.1.10.3 to 1.1.10.4. - [Release notes](https://github.com/xerial/snappy-java/releases) - [Commits](https://github.com/xerial/snappy-java/compare/v1.1.10.3...v1.1.10.4) --- updated-dependencies: - dependency-name: org.xerial.snappy:snappy-java dependency-type: direct:production ... Signed-off-by: dependabot[bot] Signed-off-by: Pierre Villard This closes #7792. --- pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pom.xml b/pom.xml index a5d282e28f..aa23ea1b1b 100644 --- a/pom.xml +++ b/pom.xml @@ -618,7 +618,7 @@ org.xerial.snappy snappy-java -1.1.10.3 +1.1.10.4
[nifi] branch dependabot/maven/org.xerial.snappy-snappy-java-1.1.10.4 deleted (was 88693924b7)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/maven/org.xerial.snappy-snappy-java-1.1.10.4 in repository https://gitbox.apache.org/repos/asf/nifi.git was 88693924b7 Bump org.xerial.snappy:snappy-java from 1.1.10.3 to 1.1.10.4 The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
[nifi] branch main updated: NIFI-12129 - Bump org.xerial.snappy:snappy-java from 1.1.10.3 to 1.1.10.4
This is an automated email from the ASF dual-hosted git repository. pvillard pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/nifi.git The following commit(s) were added to refs/heads/main by this push: new b76074a6c9 NIFI-12129 - Bump org.xerial.snappy:snappy-java from 1.1.10.3 to 1.1.10.4 b76074a6c9 is described below commit b76074a6c9427f588182942dd04d51ab04cf3184 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> AuthorDate: Mon Sep 25 18:45:42 2023 + NIFI-12129 - Bump org.xerial.snappy:snappy-java from 1.1.10.3 to 1.1.10.4 Bumps [org.xerial.snappy:snappy-java](https://github.com/xerial/snappy-java) from 1.1.10.3 to 1.1.10.4. - [Release notes](https://github.com/xerial/snappy-java/releases) - [Commits](https://github.com/xerial/snappy-java/compare/v1.1.10.3...v1.1.10.4) --- updated-dependencies: - dependency-name: org.xerial.snappy:snappy-java dependency-type: direct:production ... Signed-off-by: dependabot[bot] Signed-off-by: Pierre Villard This closes #7792. --- pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pom.xml b/pom.xml index 5b4361ffdd..1e8b1b6872 100644 --- a/pom.xml +++ b/pom.xml @@ -628,7 +628,7 @@ org.xerial.snappy snappy-java -1.1.10.3 +1.1.10.4
[nifi] branch support/nifi-1.x updated: NIFI-11177: Add defensive code for null values for Iceberg
This is an automated email from the ASF dual-hosted git repository.
pvillard pushed a commit to branch support/nifi-1.x
in repository https://gitbox.apache.org/repos/asf/nifi.git
The following commit(s) were added to refs/heads/support/nifi-1.x by this push:
new e98a2ef5fd NIFI-11177: Add defensive code for null values for Iceberg
e98a2ef5fd is described below
commit e98a2ef5fd7b0c4229545130f6c5aebb670e8d27
Author: Matt Burgess
AuthorDate: Thu Sep 21 23:56:58 2023 -0400
NIFI-11177: Add defensive code for null values for Iceberg
Signed-off-by: Pierre Villard
This closes #.
---
.../serialization/record/util/DataTypeUtils.java | 2 +-
.../iceberg/converter/GenericDataConverters.java | 29 +++---
.../iceberg/TestIcebergRecordConverter.java| 28 +
3 files changed, 55 insertions(+), 4 deletions(-)
diff --git
a/nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/util/DataTypeUtils.java
b/nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/util/DataTypeUtils.java
index 8752d85b19..7464b28ee9 100644
---
a/nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/util/DataTypeUtils.java
+++
b/nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/util/DataTypeUtils.java
@@ -238,7 +238,7 @@ public class DataTypeUtils {
public static UUID toUUID(Object value) {
if (value == null) {
-throw new IllegalTypeConversionException("Null values cannot be
converted to a UUID");
+return null;
}
if (value instanceof UUID) {
diff --git
a/nifi-nar-bundles/nifi-iceberg-bundle/nifi-iceberg-common/src/main/java/org/apache/nifi/processors/iceberg/converter/GenericDataConverters.java
b/nifi-nar-bundles/nifi-iceberg-bundle/nifi-iceberg-common/src/main/java/org/apache/nifi/processors/iceberg/converter/GenericDataConverters.java
index 8854bf6e0c..c8ee7bd171 100644
---
a/nifi-nar-bundles/nifi-iceberg-bundle/nifi-iceberg-common/src/main/java/org/apache/nifi/processors/iceberg/converter/GenericDataConverters.java
+++
b/nifi-nar-bundles/nifi-iceberg-bundle/nifi-iceberg-common/src/main/java/org/apache/nifi/processors/iceberg/converter/GenericDataConverters.java
@@ -29,6 +29,7 @@ import
org.apache.nifi.serialization.record.util.DataTypeUtils;
import java.math.BigDecimal;
import java.nio.ByteBuffer;
+import java.sql.Time;
import java.sql.Timestamp;
import java.time.LocalDateTime;
import java.time.LocalTime;
@@ -91,7 +92,8 @@ public class GenericDataConverters {
@Override
public LocalTime convert(Object data) {
-return DataTypeUtils.toTime(data, () ->
DataTypeUtils.getDateFormat(timeFormat), null).toLocalTime();
+Time time = DataTypeUtils.toTime(data, () ->
DataTypeUtils.getDateFormat(timeFormat), null);
+return time == null ? null : time.toLocalTime();
}
}
@@ -106,7 +108,7 @@ public class GenericDataConverters {
@Override
public LocalDateTime convert(Object data) {
final Timestamp convertedTimestamp =
DataTypeUtils.toTimestamp(data, () ->
DataTypeUtils.getDateFormat(dataType.getFormat()), null);
-return convertedTimestamp.toLocalDateTime();
+return convertedTimestamp == null ? null :
convertedTimestamp.toLocalDateTime();
}
}
@@ -121,7 +123,7 @@ public class GenericDataConverters {
@Override
public OffsetDateTime convert(Object data) {
final Timestamp convertedTimestamp =
DataTypeUtils.toTimestamp(data, () ->
DataTypeUtils.getDateFormat(dataType.getFormat()), null);
-return OffsetDateTime.ofInstant(convertedTimestamp.toInstant(),
ZoneId.of("UTC"));
+return convertedTimestamp == null ? null :
OffsetDateTime.ofInstant(convertedTimestamp.toInstant(), ZoneId.of("UTC"));
}
}
@@ -129,6 +131,9 @@ public class GenericDataConverters {
@Override
public byte[] convert(Object data) {
+if (data == null) {
+return null;
+}
final UUID uuid = DataTypeUtils.toUUID(data);
ByteBuffer byteBuffer = ByteBuffer.wrap(new byte[16]);
byteBuffer.putLong(uuid.getMostSignificantBits());
@@ -147,6 +152,9 @@ public class GenericDataConverters {
@Override
public byte[] convert(Byte[] data) {
+if (data == null) {
+return null;
+}
Validate.isTrue(data.length == length, String.format("Cannot write
byte array of length %s as fixed[%s]", data.length, length));
return ArrayUtils.toPrimitive(data);
}
@@ -156,6 +164,9 @@ public class GenericDataConverters {
@Override
public ByteBuffer convert(Byte[] data) {
+if (data == null) {
+return null;
+}
return
[nifi] branch main updated: NIFI-11177: Add defensive code for null values for Iceberg
This is an automated email from the ASF dual-hosted git repository.
pvillard pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/nifi.git
The following commit(s) were added to refs/heads/main by this push:
new e7da2ab5b8 NIFI-11177: Add defensive code for null values for Iceberg
e7da2ab5b8 is described below
commit e7da2ab5b81ee96493ba2c20203fb247411f45eb
Author: Matt Burgess
AuthorDate: Thu Sep 21 23:56:58 2023 -0400
NIFI-11177: Add defensive code for null values for Iceberg
Signed-off-by: Pierre Villard
This closes #.
---
.../serialization/record/util/DataTypeUtils.java | 2 +-
.../iceberg/converter/GenericDataConverters.java | 29 +++---
.../iceberg/TestIcebergRecordConverter.java| 28 +
3 files changed, 55 insertions(+), 4 deletions(-)
diff --git
a/nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/util/DataTypeUtils.java
b/nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/util/DataTypeUtils.java
index 8752d85b19..7464b28ee9 100644
---
a/nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/util/DataTypeUtils.java
+++
b/nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/util/DataTypeUtils.java
@@ -238,7 +238,7 @@ public class DataTypeUtils {
public static UUID toUUID(Object value) {
if (value == null) {
-throw new IllegalTypeConversionException("Null values cannot be
converted to a UUID");
+return null;
}
if (value instanceof UUID) {
diff --git
a/nifi-nar-bundles/nifi-iceberg-bundle/nifi-iceberg-common/src/main/java/org/apache/nifi/processors/iceberg/converter/GenericDataConverters.java
b/nifi-nar-bundles/nifi-iceberg-bundle/nifi-iceberg-common/src/main/java/org/apache/nifi/processors/iceberg/converter/GenericDataConverters.java
index 8854bf6e0c..c8ee7bd171 100644
---
a/nifi-nar-bundles/nifi-iceberg-bundle/nifi-iceberg-common/src/main/java/org/apache/nifi/processors/iceberg/converter/GenericDataConverters.java
+++
b/nifi-nar-bundles/nifi-iceberg-bundle/nifi-iceberg-common/src/main/java/org/apache/nifi/processors/iceberg/converter/GenericDataConverters.java
@@ -29,6 +29,7 @@ import
org.apache.nifi.serialization.record.util.DataTypeUtils;
import java.math.BigDecimal;
import java.nio.ByteBuffer;
+import java.sql.Time;
import java.sql.Timestamp;
import java.time.LocalDateTime;
import java.time.LocalTime;
@@ -91,7 +92,8 @@ public class GenericDataConverters {
@Override
public LocalTime convert(Object data) {
-return DataTypeUtils.toTime(data, () ->
DataTypeUtils.getDateFormat(timeFormat), null).toLocalTime();
+Time time = DataTypeUtils.toTime(data, () ->
DataTypeUtils.getDateFormat(timeFormat), null);
+return time == null ? null : time.toLocalTime();
}
}
@@ -106,7 +108,7 @@ public class GenericDataConverters {
@Override
public LocalDateTime convert(Object data) {
final Timestamp convertedTimestamp =
DataTypeUtils.toTimestamp(data, () ->
DataTypeUtils.getDateFormat(dataType.getFormat()), null);
-return convertedTimestamp.toLocalDateTime();
+return convertedTimestamp == null ? null :
convertedTimestamp.toLocalDateTime();
}
}
@@ -121,7 +123,7 @@ public class GenericDataConverters {
@Override
public OffsetDateTime convert(Object data) {
final Timestamp convertedTimestamp =
DataTypeUtils.toTimestamp(data, () ->
DataTypeUtils.getDateFormat(dataType.getFormat()), null);
-return OffsetDateTime.ofInstant(convertedTimestamp.toInstant(),
ZoneId.of("UTC"));
+return convertedTimestamp == null ? null :
OffsetDateTime.ofInstant(convertedTimestamp.toInstant(), ZoneId.of("UTC"));
}
}
@@ -129,6 +131,9 @@ public class GenericDataConverters {
@Override
public byte[] convert(Object data) {
+if (data == null) {
+return null;
+}
final UUID uuid = DataTypeUtils.toUUID(data);
ByteBuffer byteBuffer = ByteBuffer.wrap(new byte[16]);
byteBuffer.putLong(uuid.getMostSignificantBits());
@@ -147,6 +152,9 @@ public class GenericDataConverters {
@Override
public byte[] convert(Byte[] data) {
+if (data == null) {
+return null;
+}
Validate.isTrue(data.length == length, String.format("Cannot write
byte array of length %s as fixed[%s]", data.length, length));
return ArrayUtils.toPrimitive(data);
}
@@ -156,6 +164,9 @@ public class GenericDataConverters {
@Override
public ByteBuffer convert(Byte[] data) {
+if (data == null) {
+return null;
+}
return
[nifi] branch main updated: NIFI-12024: Add CSV Writer property to CSVRecordSetWriter and a FastCSV writer implementation
This is an automated email from the ASF dual-hosted git repository.
pvillard pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/nifi.git
The following commit(s) were added to refs/heads/main by this push:
new 3f5661232e NIFI-12024: Add CSV Writer property to CSVRecordSetWriter
and a FastCSV writer implementation
3f5661232e is described below
commit 3f5661232e2e229e97e3ae32139e379de6c755eb
Author: Matt Burgess
AuthorDate: Tue Sep 12 22:09:49 2023 -0400
NIFI-12024: Add CSV Writer property to CSVRecordSetWriter and a FastCSV
writer implementation
Signed-off-by: Pierre Villard
This closes #7686.
---
.../org/apache/nifi/csv/CSVRecordSetWriter.java| 38 +-
.../org/apache/nifi/csv/WriteFastCSVResult.java| 229 +
.../apache/nifi/csv/TestWriteFastCSVResult.java| 381 +
3 files changed, 646 insertions(+), 2 deletions(-)
diff --git
a/nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/csv/CSVRecordSetWriter.java
b/nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/csv/CSVRecordSetWriter.java
index ca934a7c9c..7f7b5f26ee 100644
---
a/nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/csv/CSVRecordSetWriter.java
+++
b/nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/csv/CSVRecordSetWriter.java
@@ -21,8 +21,10 @@ import org.apache.commons.csv.CSVFormat;
import org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.annotation.documentation.Tags;
import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.components.AllowableValue;
import org.apache.nifi.components.PropertyDescriptor;
import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.expression.ExpressionLanguageScope;
import org.apache.nifi.logging.ComponentLog;
import org.apache.nifi.schema.access.SchemaNotFoundException;
import org.apache.nifi.serialization.DateTimeTextRecordSetWriter;
@@ -43,9 +45,31 @@ import java.util.Map;
+ "corresponding to the record fields.")
public class CSVRecordSetWriter extends DateTimeTextRecordSetWriter implements
RecordSetWriterFactory {
+// CSV writer implementations
+public static final AllowableValue APACHE_COMMONS_CSV = new
AllowableValue("commons-csv", "Apache Commons CSV",
+"The CSV writer implementation from the Apache Commons CSV
library.");
+
+public static final AllowableValue FAST_CSV = new
AllowableValue("fast-csv", "FastCSV",
+"The CSV writer implementation from the FastCSV library. NOTE:
This writer only officially supports RFC-4180, so it recommended to "
++ "set the 'CSV Format' property to 'RFC 4180'. It does
handle some non-compliant CSV data, for that case set the 'CSV Format' property
to "
++ "'CUSTOM' and the other custom format properties (such
as 'Trim Fields', 'Trim double quote', etc.) as appropriate. Be aware that this
"
++ "may cause errors if FastCSV doesn't handle the property
settings correctly (such as 'Quote Mode'), but otherwise may process the output
as expected even "
++ "if the data is not fully RFC-4180 compliant.");
+public static final PropertyDescriptor CSV_WRITER = new
PropertyDescriptor.Builder()
+.name("csv-writer")
+.displayName("CSV Writer")
+.description("Specifies which writer implementation to use to
write CSV records. NOTE: Different writers may support different subsets of
functionality "
++ "and may also exhibit different levels of performance.")
+.expressionLanguageSupported(ExpressionLanguageScope.NONE)
+.allowableValues(APACHE_COMMONS_CSV, FAST_CSV)
+.defaultValue(APACHE_COMMONS_CSV.getValue())
+.required(true)
+.build();
+
private volatile ConfigurationContext context;
private volatile boolean includeHeader;
+private volatile String csvWriter;
private volatile String charSet;
// it will be initialized only if there are no dynamic csv formatting
properties
@@ -55,6 +79,7 @@ public class CSVRecordSetWriter extends
DateTimeTextRecordSetWriter implements R
protected List getSupportedPropertyDescriptors() {
final List properties = new
ArrayList<>(super.getSupportedPropertyDescriptors());
properties.add(CSVUtils.CSV_FORMAT);
+properties.add(CSV_WRITER);
properties.add(CSVUtils.VALUE_SEPARATOR);
properties.add(CSVUtils.INCLUDE_HEADER_LINE);
[nifi] branch support/nifi-1.x updated: NIFI-12126: Downgrade snowflake-jdbc to 3.13.33 snowflake-ingest-sdk:2.0.3 is not compatible with snowflake-jdbc:3.14.x
This is an automated email from the ASF dual-hosted git repository. pvillard pushed a commit to branch support/nifi-1.x in repository https://gitbox.apache.org/repos/asf/nifi.git The following commit(s) were added to refs/heads/support/nifi-1.x by this push: new 24994a53c3 NIFI-12126: Downgrade snowflake-jdbc to 3.13.33 snowflake-ingest-sdk:2.0.3 is not compatible with snowflake-jdbc:3.14.x 24994a53c3 is described below commit 24994a53c355f52679d22aae8a96db5f06450f10 Author: Peter Turcsanyi AuthorDate: Mon Sep 25 19:14:03 2023 +0200 NIFI-12126: Downgrade snowflake-jdbc to 3.13.33 snowflake-ingest-sdk:2.0.3 is not compatible with snowflake-jdbc:3.14.x Signed-off-by: Pierre Villard This closes #7790. --- nifi-nar-bundles/nifi-snowflake-bundle/pom.xml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/nifi-nar-bundles/nifi-snowflake-bundle/pom.xml b/nifi-nar-bundles/nifi-snowflake-bundle/pom.xml index 8dfd026f71..2485ade3b8 100644 --- a/nifi-nar-bundles/nifi-snowflake-bundle/pom.xml +++ b/nifi-nar-bundles/nifi-snowflake-bundle/pom.xml @@ -40,7 +40,8 @@ net.snowflake snowflake-jdbc -3.14.1 + +3.13.33
[nifi] branch main updated: NIFI-12126: Downgrade snowflake-jdbc to 3.13.33 snowflake-ingest-sdk:2.0.3 is not compatible with snowflake-jdbc:3.14.x
This is an automated email from the ASF dual-hosted git repository. pvillard pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/nifi.git The following commit(s) were added to refs/heads/main by this push: new 5644fdd725 NIFI-12126: Downgrade snowflake-jdbc to 3.13.33 snowflake-ingest-sdk:2.0.3 is not compatible with snowflake-jdbc:3.14.x 5644fdd725 is described below commit 5644fdd72575f9e8e45d8b057e61f9d59e1e9078 Author: Peter Turcsanyi AuthorDate: Mon Sep 25 19:14:03 2023 +0200 NIFI-12126: Downgrade snowflake-jdbc to 3.13.33 snowflake-ingest-sdk:2.0.3 is not compatible with snowflake-jdbc:3.14.x Signed-off-by: Pierre Villard This closes #7790. --- nifi-nar-bundles/nifi-snowflake-bundle/pom.xml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/nifi-nar-bundles/nifi-snowflake-bundle/pom.xml b/nifi-nar-bundles/nifi-snowflake-bundle/pom.xml index 3f1391307a..e2da9899e7 100644 --- a/nifi-nar-bundles/nifi-snowflake-bundle/pom.xml +++ b/nifi-nar-bundles/nifi-snowflake-bundle/pom.xml @@ -40,7 +40,8 @@ net.snowflake snowflake-jdbc -3.14.1 + +3.13.33
