[jira] [Commented] (NIFI-11453) Update ignite-core to 2.14.0
[ https://issues.apache.org/jira/browse/NIFI-11453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17722987#comment-17722987 ] Jeyassri Balachandran commented on NIFI-11453: -- [~exceptionfactory] I have not worked with Ignite client library until now. But I'll go through their documentation to understand the code changes to be done along with the version upgrade. I'll refer: [https://ignite.apache.org/releases/latest/javadoc/] [https://ignite.apache.org/docs/latest/quick-start/java] [https://ignite.apache.org/docs/latest/thin-clients/java-thin-client] > Update ignite-core to 2.14.0 > - > > Key: NIFI-11453 > URL: https://issues.apache.org/jira/browse/NIFI-11453 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.21.0 >Reporter: Mike R >Priority: Major > > Update ignite-core to 2.14.0 from 1.6.0. The dependency has CVEs in 1.6.0 > resolved in 2.14.0: > [CVE-2020-1963|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-1963] > [CVE-2018-8018|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-8018] > [CVE-2018-1295|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-1295] > [CVE-2017-7686|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-7686] > [CVE-2016-6805|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-6805] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #7250: [NIFI-11550] Refactored Groovy tests to Java.
exceptionfactory commented on code in PR #7250:
URL: https://github.com/apache/nifi/pull/7250#discussion_r1194544215
##
nifi-commons/nifi-expression-language/src/test/java/org/apache/nifi/attribute/expression/language/TestQuery.java:
##
@@ -1177,6 +1179,99 @@ public void testReplaceAllWithMatchingGroup() {
verifyEquals("${attr:replaceAll('.*?(l+).*', '$1')}", attributes,
"ll");
}
+@Test
+public void testReplaceShouldReplaceAllLiteralMatches() {
+int n = 3;
+final String ORIGINAL_VALUE = "Hello World";
+final Map attributes = Map.of("single", ORIGINAL_VALUE,
+"repeating", StringUtils.repeat(ORIGINAL_VALUE, " ", n));
+final String REPLACEMENT_VALUE = "Goodbye Planet";
+final String EXPECTED_REPEATING_RESULT =
StringUtils.repeat(REPLACEMENT_VALUE, " ", n);
+final String REPLACE_SINGLE_EXPRESSION = "${single:replace('" +
ORIGINAL_VALUE + "', '" + REPLACEMENT_VALUE + "')}";
+final String REPLACE_REPEATING_EXPRESSION = "${repeating:replace('" +
ORIGINAL_VALUE + "', '" + REPLACEMENT_VALUE + "')}";
+Query replaceSingleQuery = Query.compile(REPLACE_SINGLE_EXPRESSION);
+Query replaceRepeatingQuery =
Query.compile(REPLACE_REPEATING_EXPRESSION);
+
+QueryResult replaceSingleResult = replaceSingleQuery.evaluate(new
StandardEvaluationContext(attributes));
+QueryResult replaceRepeatingResult =
replaceRepeatingQuery.evaluate(new StandardEvaluationContext(attributes));
+
+assertEquals(REPLACEMENT_VALUE, replaceSingleResult.getValue());
+assertEquals(AttributeExpression.ResultType.STRING,
replaceSingleResult.getResultType());
+assertEquals(EXPECTED_REPEATING_RESULT,
replaceRepeatingResult.getValue());
+assertEquals(AttributeExpression.ResultType.STRING,
replaceRepeatingResult.getResultType());
+}
+
+@Test
+public void testReplaceFirstShouldOnlyReplaceFirstRegexMatch() {
+// Arrange
Review Comment:
```suggestion
```
##
nifi-commons/nifi-expression-language/src/test/java/org/apache/nifi/attribute/expression/language/TestQuery.java:
##
@@ -1177,6 +1179,99 @@ public void testReplaceAllWithMatchingGroup() {
verifyEquals("${attr:replaceAll('.*?(l+).*', '$1')}", attributes,
"ll");
}
+@Test
+public void testReplaceShouldReplaceAllLiteralMatches() {
+int n = 3;
+final String ORIGINAL_VALUE = "Hello World";
Review Comment:
Understanding that this is a migration, can you correct the style of these
methods variables to use camelCase instead of all capital letters?
```suggestion
final String originalValue = "Hello World";
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-11554) Upgrade to OpenSAML 4.3.0
[ https://issues.apache.org/jira/browse/NIFI-11554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11554: Status: Patch Available (was: Open) > Upgrade to OpenSAML 4.3.0 > - > > Key: NIFI-11554 > URL: https://issues.apache.org/jira/browse/NIFI-11554 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Fix For: 2.0.0 > > > [OpenSAML 3|https://shibboleth.atlassian.net/wiki/spaces/OSAML/overview] has > reached end of life and Spring Security 5 deprecated support for OpenSAML 3 > in recent releases. > OpenSAML 4 requires Java 11 and provides the current supported version, > integrated with Spring Security 5. [Spring Security SAML2 > Migrations|https://docs.spring.io/spring-security/reference/5.8/migration/servlet/saml2.html] > highlights the dependency management settings necessary to use OpenSAML 4, > along with the related component changes required. > OpenSAML does not publish to Maven Central according to project document on > [use of Maven > Central|https://shibboleth.atlassian.net/wiki/spaces/DEV/pages/1123844333/Use+of+Maven+Central]. > The Shibboleth repository must be added to resolve the latest OpenSAML > versions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory opened a new pull request, #7251: NIFI-11554 Upgrade OpenSAML from 3.4.6 to 4.3.0
exceptionfactory opened a new pull request, #7251: URL: https://github.com/apache/nifi/pull/7251 # Summary [NIFI-11554](https://issues.apache.org/jira/browse/NIFI-11554) Upgrades [OpenSAML](https://shibboleth.atlassian.net/wiki/spaces/OSAML/overview) from 3.4.6 to 4.3.0 and replaces deprecated Spring Security components with current supported versions. OpenSAML 4 requires Java 11 and replaces OpenSAML 3, which has reached end of life according to project documentation. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-11554) Upgrade to OpenSAML 4.3.0
David Handermann created NIFI-11554: --- Summary: Upgrade to OpenSAML 4.3.0 Key: NIFI-11554 URL: https://issues.apache.org/jira/browse/NIFI-11554 Project: Apache NiFi Issue Type: Improvement Components: Core Framework Reporter: David Handermann Assignee: David Handermann Fix For: 2.0.0 [OpenSAML 3|https://shibboleth.atlassian.net/wiki/spaces/OSAML/overview] has reached end of life and Spring Security 5 deprecated support for OpenSAML 3 in recent releases. OpenSAML 4 requires Java 11 and provides the current supported version, integrated with Spring Security 5. [Spring Security SAML2 Migrations|https://docs.spring.io/spring-security/reference/5.8/migration/servlet/saml2.html] highlights the dependency management settings necessary to use OpenSAML 4, along with the related component changes required. OpenSAML does not publish to Maven Central according to project document on [use of Maven Central|https://shibboleth.atlassian.net/wiki/spaces/DEV/pages/1123844333/Use+of+Maven+Central]. The Shibboleth repository must be added to resolve the latest OpenSAML versions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #7194: NIFI-11167 - Add Excel Record Reader
exceptionfactory commented on code in PR #7194:
URL: https://github.com/apache/nifi/pull/7194#discussion_r1194517265
##
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/excel/ExcelReader.java:
##
@@ -0,0 +1,193 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.excel;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.context.PropertyContext;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.schema.access.SchemaAccessStrategy;
+import org.apache.nifi.schema.access.SchemaNotFoundException;
+import org.apache.nifi.schema.inference.InferSchemaAccessStrategy;
+import org.apache.nifi.schema.inference.RecordSourceFactory;
+import org.apache.nifi.schema.inference.SchemaInferenceEngine;
+import org.apache.nifi.schema.inference.SchemaInferenceUtil;
+import org.apache.nifi.schema.inference.TimeValueInference;
+import org.apache.nifi.schemaregistry.services.SchemaRegistry;
+import org.apache.nifi.serialization.DateTimeUtils;
+import org.apache.nifi.serialization.MalformedRecordException;
+import org.apache.nifi.serialization.RecordReader;
+import org.apache.nifi.serialization.RecordReaderFactory;
+import org.apache.nifi.serialization.SchemaRegistryService;
+import org.apache.nifi.serialization.record.RecordSchema;
+import org.apache.nifi.stream.io.NonCloseableInputStream;
+import org.apache.poi.ss.usermodel.Row;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.atomic.AtomicReferenceArray;
+import java.util.stream.IntStream;
+
+@Tags({"excel", "spreadsheet", "xlsx", "parse", "record", "row", "reader",
"values", "cell"})
+@CapabilityDescription("Parses a Microsoft Excel document returning each row
in each sheet as a separate record. "
++ "This reader allows for inferring a schema either based on the first
line of an Excel sheet if a 'header line' is "
++ "present or from all the desired sheets, or providing an explicit
schema "
++ "for interpreting the values. See Controller Service's Usage for
further documentation. "
++ "This reader is currently only capable of processing .xlsx "
++ "(XSSF 2007 OOXML file format) Excel documents and not older .xls
(HSSF '97(-2007) file format) documents.)")
+public class ExcelReader extends SchemaRegistryService implements
RecordReaderFactory {
+
+private static final AllowableValue HEADER_DERIVED = new
AllowableValue("excel-header-derived", "Use fields From Header",
+"The first chosen row of the Excel sheet is a header row that
contains the columns representative of all the rows " +
+"in the desired sheets. The schema will be derived by
using those columns in the header.");
+public static final PropertyDescriptor DESIRED_SHEETS = new
PropertyDescriptor
+.Builder().name("extract-sheets")
+.displayName("Sheets to Extract")
+.description("Comma separated list of Excel document sheet names
whose rows should be extracted from the excel document. If this property" +
+" is left blank then all the rows from all the sheets will
be extracted from the Excel document. The list of names is case in-sensitive.
Any sheets not" +
+" specified in this value will be ignored. A bulletin will
be generated if a specified sheet(s) are not found.")
+.required(false)
+
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+
[GitHub] [nifi] nathluu commented on a diff in pull request #7245: NIFI-8831 Support using MSSQL as nifi registry relation database engine
nathluu commented on code in PR #7245:
URL: https://github.com/apache/nifi/pull/7245#discussion_r1194519284
##
nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/java/org/apache/nifi/registry/security/authorization/database/DatabaseUserGroupProvider.java:
##
@@ -121,7 +129,9 @@ public void checkInheritability(final String
proposedFingerprint) throws Authori
@Override
public User addUser(final User user) throws AuthorizationAccessException {
Validate.notNull(user);
-final String sql = "INSERT INTO UGP_USER(IDENTIFIER, IDENTITY) VALUES
(?, ?)";
+final String sql = (databaseType instanceof SQLServerDatabaseType)
+? "INSERT INTO UGP_USER(IDENTIFIER, [IDENTITY]) VALUES (?, ?)"
+: "INSERT INTO UGP_USER(IDENTIFIER, IDENTITY) VALUES (?, ?)";
Review Comment:
Because IDENTITY is a keyword in MSSQL but not on H2, mysql, mariaDB and
postgresql so for MSSQL it must be enclosed by [].
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (NIFI-11553) additional configurability for GCP pubsub processors
Paul Grey created NIFI-11553: Summary: additional configurability for GCP pubsub processors Key: NIFI-11553 URL: https://issues.apache.org/jira/browse/NIFI-11553 Project: Apache NiFi Issue Type: Improvement Reporter: Paul Grey Assignee: Paul Grey (1) Allow specification of endpoint via: https://cloud.google.com/java/docs/reference/google-cloud-pubsub/latest/com.google.cloud.pubsub.v1.Publisher.Builder#com_google_cloud_pubsub_v1_Publisher_Builder_setEndpoint_java_lang_String_ (2) Allow uniform configurability of batch settings via: https://cloud.google.com/java/docs/reference/gax/latest/com.google.api.gax.batching.BatchingSettings.Builder Implement in the context of AbstractGCPubSubProcessor if possible, such that all implementing processors are able to leverage this configurability. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] dan-s1 opened a new pull request, #7250: [NIFI-11550] Refactored Groovy tests to Java.
dan-s1 opened a new pull request, #7250: URL: https://github.com/apache/nifi/pull/7250 # Summary [NIFI-11550](https://issues.apache.org/jira/browse/NIFI-11550) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] emiliosetiadarma commented on a diff in pull request #7238: NIFI-11531: refactored tests in nifi-security-utils
emiliosetiadarma commented on code in PR #7238:
URL: https://github.com/apache/nifi/pull/7238#discussion_r1194441114
##
nifi-commons/nifi-security-utils/src/test/java/org/apache/nifi/security/util/crypto/CipherUtilityTest.java:
##
@@ -0,0 +1,322 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.security.util.crypto;
+
+import org.apache.nifi.security.util.EncryptionMethod;
+import org.apache.nifi.security.util.KeyDerivationFunction;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.Test;
+
+import java.security.Security;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.function.Function;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+
+import static org.junit.jupiter.api.Assertions.assertArrayEquals;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+public class CipherUtilityTest {
+private static final Pattern KEY_LENGTH_PATTERN =
Pattern.compile("([\\d]+)BIT");
+// TripleDES must precede DES for automatic grouping precedence
+private static final List CIPHERS = Arrays.asList("AES",
"TRIPLEDES", "DES", "RC2", "RC4", "RC5", "TWOFISH");
+private static final List SYMMETRIC_ALGORITHMS =
Arrays.stream(EncryptionMethod.values())
+.map(it -> it.getAlgorithm())
+.filter(algorithm -> algorithm.startsWith("PBE") ||
algorithm.startsWith("AES"))
+.collect(Collectors.toList());
+private static final Map> ALGORITHMS_MAPPED_BY_CIPHER
= SYMMETRIC_ALGORITHMS
+.stream()
+.collect(Collectors.groupingBy(algorithm ->
CIPHERS.stream().filter(cipher ->
algorithm.contains(cipher)).findFirst().get()));
+
+// Manually mapped as of 03/21/21 1.13.0
+private static final Map>
ALGORITHMS_MAPPED_BY_KEY_LENGTH = new HashMap<>();
+static {
+ALGORITHMS_MAPPED_BY_KEY_LENGTH.put(40,
Arrays.asList("PBEWITHSHAAND40BITRC2-CBC",
+"PBEWITHSHAAND40BITRC4"));
+ALGORITHMS_MAPPED_BY_KEY_LENGTH.put(64,
Arrays.asList("PBEWITHMD5ANDDES",
+"PBEWITHSHA1ANDDES"));
+ALGORITHMS_MAPPED_BY_KEY_LENGTH.put(112,
Arrays.asList("PBEWITHSHAAND2-KEYTRIPLEDES-CBC",
+"PBEWITHSHAAND3-KEYTRIPLEDES-CBC"));
+ALGORITHMS_MAPPED_BY_KEY_LENGTH.put(128,
Arrays.asList("PBEWITHMD5AND128BITAES-CBC-OPENSSL",
+"PBEWITHMD5ANDRC2",
+"PBEWITHSHA1ANDRC2",
+"PBEWITHSHA256AND128BITAES-CBC-BC",
+"PBEWITHSHAAND128BITAES-CBC-BC",
+"PBEWITHSHAAND128BITRC2-CBC",
+"PBEWITHSHAAND128BITRC4",
+"PBEWITHSHAANDTWOFISH-CBC",
+"AES/CBC/NoPadding",
+"AES/CBC/PKCS7Padding",
+"AES/CTR/NoPadding",
+"AES/GCM/NoPadding"));
+ALGORITHMS_MAPPED_BY_KEY_LENGTH.put(192,
Arrays.asList("PBEWITHMD5AND192BITAES-CBC-OPENSSL",
+"PBEWITHSHA256AND192BITAES-CBC-BC",
+"PBEWITHSHAAND192BITAES-CBC-BC",
+"AES/CBC/NoPadding",
+"AES/CBC/PKCS7Padding",
+"AES/CTR/NoPadding",
+"AES/GCM/NoPadding"));
+ALGORITHMS_MAPPED_BY_KEY_LENGTH.put(256,
Arrays.asList("PBEWITHMD5AND256BITAES-CBC-OPENSSL",
+"PBEWITHSHA256AND256BITAES-CBC-BC",
+"PBEWITHSHAAND256BITAES-CBC-BC",
+"AES/CBC/NoPadding",
+"AES/CBC/PKCS7Padding",
+"AES/CTR/NoPadding",
+"AES/GCM/NoPadding"));
+}
+
+@BeforeAll
+static void setUpOnce() {
+Security.addProvider(new BouncyCastleProvider());
Review Comment:
Will check the tests one by one
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
[GitHub] [nifi] emiliosetiadarma commented on a diff in pull request #7238: NIFI-11531: refactored tests in nifi-security-utils
emiliosetiadarma commented on code in PR #7238:
URL: https://github.com/apache/nifi/pull/7238#discussion_r1194438329
##
nifi-commons/nifi-security-utils/src/test/java/org/apache/nifi/security/util/crypto/BcryptSecureHasherTest.java:
##
@@ -0,0 +1,334 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.security.util.crypto;
+
+import at.favre.lib.crypto.bcrypt.Radix64Encoder;
+import org.bouncycastle.util.encoders.Hex;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.condition.EnabledIfSystemProperty;
+
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Base64;
+import java.util.Collections;
+import java.util.List;
+import java.util.stream.Collectors;
+
+import static org.junit.jupiter.api.Assertions.assertArrayEquals;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertNotEquals;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+public class BcryptSecureHasherTest {
+
+@Test
+void testShouldBeDeterministicWithStaticSalt() {
+// Arrange
+int cost = 4;
+
+int testIterations = 10;
+byte[] inputBytes = "This is a sensitive value".getBytes();
+
+final String EXPECTED_HASH_HEX =
"24326124303424526b6a4559512f526245447959554b6553304471622e596b4c5331655a2e6c61586550484c69464d783937564c566d47354250454f";
+
+BcryptSecureHasher bcryptSH = new BcryptSecureHasher(cost);
+
+final List results = new ArrayList<>();
+
+// Act
+for (int i = 0; i < testIterations; i++) {
+byte[] hash = bcryptSH.hashRaw(inputBytes);
+String hashHex = new String(Hex.encode(hash));
+results.add(hashHex);
+}
+
+// Assert
+results.forEach(result -> assertEquals(EXPECTED_HASH_HEX, result));
+}
+
+@Test
+void testShouldBeDifferentWithRandomSalt() {
+// Arrange
+int cost = 4;
+int saltLength = 16;
+
+int testIterations = 10;
+byte[] inputBytes = "This is a sensitive value".getBytes();
+
+final String EXPECTED_HASH_HEX =
"24326124303424546d6c47615342546447463061574d6755324673642e38675a347a6149356d6b4d50594c542e344e68337962455a4678384b676a75";
+
+BcryptSecureHasher bcryptSH = new BcryptSecureHasher(cost, saltLength);
+
+final List results = new ArrayList<>();
+
+// Act
+for (int i = 0; i < testIterations; i++) {
+byte[] hash = bcryptSH.hashRaw(inputBytes);
+String hashHex = new String(Hex.encode(hash));
+results.add(hashHex);
+}
+
+// Assert
+assertEquals(results.size(),
results.stream().distinct().collect(Collectors.toList()).size());
+results.forEach(result -> assertNotEquals(EXPECTED_HASH_HEX, result));
+}
+
+@Test
+void testShouldHandleArbitrarySalt() {
+// Arrange
+int cost = 4;
+
+final String input = "This is a sensitive value";
+byte[] inputBytes = input.getBytes();
+
+final String EXPECTED_HASH_HEX =
"24326124303424526b6a4559512f526245447959554b6553304471622e596b4c5331655a2e6c61586550484c69464d783937564c566d47354250454f";
+final String EXPECTED_HASH_BASE64 =
"JDJhJDA0JFJrakVZUS9SYkVEeVlVS2VTMERxYi5Za0xTMWVaLmxhWGVQSExpRk14OTdWTFZtRzVCUEVP";
+final byte[] EXPECTED_HASH_BYTES = Hex.decode(EXPECTED_HASH_HEX);
+
+// Static salt instance
+BcryptSecureHasher staticSaltHasher = new BcryptSecureHasher(cost);
+BcryptSecureHasher arbitrarySaltHasher = new BcryptSecureHasher(cost,
16);
+
+final byte[] STATIC_SALT = "NiFi Static
Salt".getBytes(StandardCharsets.UTF_8);
+final String DIFFERENT_STATIC_SALT = "Diff Static Salt";
+
+// Act
+byte[] staticSaltHash = staticSaltHasher.hashRaw(inputBytes);
+byte[] arbitrarySaltHash = arbitrarySaltHasher.hashRaw(inputBytes,
STATIC_SALT);
+byte[] differentArbitrarySaltHash =
[GitHub] [nifi] emiliosetiadarma commented on a diff in pull request #7238: NIFI-11531: refactored tests in nifi-security-utils
emiliosetiadarma commented on code in PR #7238:
URL: https://github.com/apache/nifi/pull/7238#discussion_r1194434867
##
nifi-commons/nifi-security-utils/src/test/java/org/apache/nifi/security/util/crypto/Argon2SecureHasherTest.java:
##
@@ -0,0 +1,424 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.security.util.crypto;
+
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+import org.bouncycastle.util.encoders.Hex;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.condition.EnabledIfSystemProperty;
+
+import java.nio.charset.StandardCharsets;
+import java.security.Security;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.function.Function;
+import java.util.stream.Collectors;
+
+import static org.junit.jupiter.api.Assertions.assertArrayEquals;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertNotEquals;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+public class Argon2SecureHasherTest {
+@BeforeAll
+static void setUpOnce() throws Exception {
+Security.addProvider(new BouncyCastleProvider());
+}
Review Comment:
Removing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] emiliosetiadarma commented on a diff in pull request #7238: NIFI-11531: refactored tests in nifi-security-utils
emiliosetiadarma commented on code in PR #7238:
URL: https://github.com/apache/nifi/pull/7238#discussion_r1194434579
##
nifi-commons/nifi-security-utils/src/test/java/org/apache/nifi/security/util/crypto/Argon2CipherProviderTest.java:
##
@@ -0,0 +1,418 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.security.util.crypto;
+
+import org.apache.commons.codec.binary.Base64;
+import org.apache.commons.codec.binary.Hex;
+import org.apache.nifi.security.util.EncryptionMethod;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+import javax.crypto.Cipher;
+import javax.crypto.spec.IvParameterSpec;
+import javax.crypto.spec.SecretKeySpec;
+import java.nio.charset.StandardCharsets;
+import java.security.Security;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+import static org.junit.jupiter.api.Assertions.assertArrayEquals;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertNotNull;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+public class Argon2CipherProviderTest {
+private static final String PLAINTEXT = "ExactBlockSizeRequiredForProcess";
+
+private static List strongKDFEncryptionMethods;
+
+private static final int DEFAULT_KEY_LENGTH = 128;
+private final String SALT_HEX = "0123456789ABCDEFFEDCBA9876543210";
+private static List VALID_KEY_LENGTHS;
+private RandomIVPBECipherProvider cipherProvider;
+private final List FULL_SALT_LENGTH_RANGE = Arrays.asList(49, 50,
51, 52, 53);
+
+@BeforeAll
+static void setUpOnce() throws Exception {
+Security.addProvider(new BouncyCastleProvider());
+
+strongKDFEncryptionMethods = Arrays.stream(EncryptionMethod.values())
+.filter(EncryptionMethod::isCompatibleWithStrongKDFs)
+.collect(Collectors.toList());
+
+VALID_KEY_LENGTHS = Arrays.asList(128, 192, 256);
+}
+
+@BeforeEach
+void setUp() {
+// Very fast parameters to test for correctness rather than production
values
+cipherProvider = new Argon2CipherProvider(1024, 1, 3);
+}
+
+@Test
+void testGetCipherShouldBeInternallyConsistent() throws Exception {
+// Arrange
+final String PASSWORD = "shortPassword";
+final byte[] SALT = cipherProvider.generateSalt();
+
+// Act
+for (EncryptionMethod em : strongKDFEncryptionMethods) {
+// Initialize a cipher for encryption
+Cipher cipher = cipherProvider.getCipher(em, PASSWORD, SALT,
DEFAULT_KEY_LENGTH, true);
+byte[] iv = cipher.getIV();
+
+byte[] cipherBytes = cipher.doFinal(PLAINTEXT.getBytes("UTF-8"));
+
+cipher = cipherProvider.getCipher(em, PASSWORD, SALT, iv,
DEFAULT_KEY_LENGTH, false);
+byte[] recoveredBytes = cipher.doFinal(cipherBytes);
+String recovered = new String(recoveredBytes, "UTF-8");
+
+// Assert
+assertEquals(PLAINTEXT, recovered);
+}
+}
+
+@Test
+void testArgon2ShouldSupportExternalCompatibility() throws Exception {
+// Arrange
+
+// Default values are hashLength = 32, memory = 1024, parallelism = 1,
iterations = 3, but the provided salt will contain the parameters used
+cipherProvider = new Argon2CipherProvider();
+
+final String PLAINTEXT = "This is a plaintext message.";
+final String PASSWORD = "thisIsABadPassword";
+final int hashLength = 256;
+
+// These values can be generated by running `$ ./openssl_argon2.rb` in
the terminal
+final byte[] SALT =
Hex.decodeHex("68d29a1d8021f45954333767358a2492".toCharArray());
+final byte[] IV =
Hex.decodeHex("808590f35f9fba14dbda9c2bb2b76a79".toCharArray());
+
+final String
[GitHub] [nifi] emiliosetiadarma commented on a diff in pull request #7238: NIFI-11531: refactored tests in nifi-security-utils
emiliosetiadarma commented on code in PR #7238:
URL: https://github.com/apache/nifi/pull/7238#discussion_r1194421748
##
nifi-commons/nifi-security-utils/src/test/java/org/apache/nifi/security/util/crypto/AESKeyedCipherProviderTest.java:
##
@@ -14,311 +14,298 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.nifi.security.util.crypto
+package org.apache.nifi.security.util.crypto;
-import org.apache.commons.codec.binary.Hex
-import org.apache.nifi.security.util.EncryptionMethod
-import org.bouncycastle.jce.provider.BouncyCastleProvider
-import org.junit.jupiter.api.BeforeAll
-import org.junit.jupiter.api.Test
-import org.slf4j.Logger
-import org.slf4j.LoggerFactory
+import org.apache.commons.codec.DecoderException;
+import org.apache.commons.codec.binary.Hex;
+import org.apache.nifi.security.util.EncryptionMethod;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.Test;
-import javax.crypto.Cipher
-import javax.crypto.SecretKey
-import javax.crypto.spec.SecretKeySpec
-import java.security.SecureRandom
-import java.security.Security
+import javax.crypto.Cipher;
+import javax.crypto.SecretKey;
+import javax.crypto.spec.SecretKeySpec;
+import java.security.NoSuchAlgorithmException;
+import java.security.SecureRandom;
+import java.security.Security;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+import java.util.stream.Collectors;
-import static org.junit.jupiter.api.Assertions.assertEquals
-import static org.junit.jupiter.api.Assertions.assertFalse
-import static org.junit.jupiter.api.Assertions.assertThrows
-import static org.junit.jupiter.api.Assertions.assertTrue
-import static org.junit.jupiter.api.Assumptions.assumeTrue
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+import static org.junit.jupiter.api.Assumptions.assumeTrue;
-class AESKeyedCipherProviderGroovyTest {
-private static final Logger logger =
LoggerFactory.getLogger(AESKeyedCipherProviderGroovyTest.class)
+public class AESKeyedCipherProviderTest {
+private static final String KEY_HEX = "0123456789ABCDEFFEDCBA9876543210";
-private static final String KEY_HEX = "0123456789ABCDEFFEDCBA9876543210"
+private static final String PLAINTEXT = "ExactBlockSizeRequiredForProcess";
-private static final String PLAINTEXT = "ExactBlockSizeRequiredForProcess"
+private static final List keyedEncryptionMethods =
Arrays.stream(EncryptionMethod.values())
+.filter(EncryptionMethod::isKeyedCipher)
+.collect(Collectors.toList());
-private static final List keyedEncryptionMethods =
EncryptionMethod.values().findAll { it.keyedCipher }
+private static SecretKey key;
-private static final SecretKey key = new
SecretKeySpec(Hex.decodeHex(KEY_HEX as char[]), "AES")
@BeforeAll
static void setUpOnce() throws Exception {
-Security.addProvider(new BouncyCastleProvider())
+Security.addProvider(new BouncyCastleProvider());
-logger.metaClass.methodMissing = { String name, args ->
-logger.info("[${name?.toUpperCase()}] ${(args as List).join(" ")}")
+try {
+key = new SecretKeySpec(Hex.decodeHex(KEY_HEX.toCharArray()),
"AES");
+} catch (final DecoderException e) {
+throw new RuntimeException(e);
}
}
private static boolean isUnlimitedStrengthCryptoAvailable() {
-Cipher.getMaxAllowedKeyLength("AES") > 128
+try {
+return Cipher.getMaxAllowedKeyLength("AES") > 128;
+} catch (final NoSuchAlgorithmException e) {
+throw new RuntimeException(e);
+}
}
@Test
void testGetCipherShouldBeInternallyConsistent() throws Exception {
// Arrange
-KeyedCipherProvider cipherProvider = new AESKeyedCipherProvider()
+KeyedCipherProvider cipherProvider = new AESKeyedCipherProvider();
// Act
for (EncryptionMethod em : keyedEncryptionMethods) {
-logger.info("Using algorithm: ${em.getAlgorithm()}")
-
// Initialize a cipher for encryption
-Cipher cipher = cipherProvider.getCipher(em, key, true)
-byte[] iv = cipher.getIV()
-logger.info("IV: ${Hex.encodeHexString(iv)}")
+Cipher cipher = cipherProvider.getCipher(em, key, true);
+byte[] iv = cipher.getIV();
-byte[] cipherBytes = cipher.doFinal(PLAINTEXT.getBytes("UTF-8"))
-logger.info("Cipher text: ${Hex.encodeHexString(cipherBytes)}
${cipherBytes.length}")
+byte[] cipherBytes = cipher.doFinal(PLAINTEXT.getBytes("UTF-8"));
-
[GitHub] [nifi] emiliosetiadarma commented on a diff in pull request #7238: NIFI-11531: refactored tests in nifi-security-utils
emiliosetiadarma commented on code in PR #7238:
URL: https://github.com/apache/nifi/pull/7238#discussion_r1194421375
##
nifi-commons/nifi-security-utils/src/test/java/org/apache/nifi/security/util/crypto/AESKeyedCipherProviderTest.java:
##
@@ -14,311 +14,298 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.nifi.security.util.crypto
+package org.apache.nifi.security.util.crypto;
-import org.apache.commons.codec.binary.Hex
-import org.apache.nifi.security.util.EncryptionMethod
-import org.bouncycastle.jce.provider.BouncyCastleProvider
-import org.junit.jupiter.api.BeforeAll
-import org.junit.jupiter.api.Test
-import org.slf4j.Logger
-import org.slf4j.LoggerFactory
+import org.apache.commons.codec.DecoderException;
+import org.apache.commons.codec.binary.Hex;
+import org.apache.nifi.security.util.EncryptionMethod;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.Test;
-import javax.crypto.Cipher
-import javax.crypto.SecretKey
-import javax.crypto.spec.SecretKeySpec
-import java.security.SecureRandom
-import java.security.Security
+import javax.crypto.Cipher;
+import javax.crypto.SecretKey;
+import javax.crypto.spec.SecretKeySpec;
+import java.security.NoSuchAlgorithmException;
+import java.security.SecureRandom;
+import java.security.Security;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+import java.util.stream.Collectors;
-import static org.junit.jupiter.api.Assertions.assertEquals
-import static org.junit.jupiter.api.Assertions.assertFalse
-import static org.junit.jupiter.api.Assertions.assertThrows
-import static org.junit.jupiter.api.Assertions.assertTrue
-import static org.junit.jupiter.api.Assumptions.assumeTrue
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+import static org.junit.jupiter.api.Assumptions.assumeTrue;
-class AESKeyedCipherProviderGroovyTest {
-private static final Logger logger =
LoggerFactory.getLogger(AESKeyedCipherProviderGroovyTest.class)
+public class AESKeyedCipherProviderTest {
+private static final String KEY_HEX = "0123456789ABCDEFFEDCBA9876543210";
-private static final String KEY_HEX = "0123456789ABCDEFFEDCBA9876543210"
+private static final String PLAINTEXT = "ExactBlockSizeRequiredForProcess";
-private static final String PLAINTEXT = "ExactBlockSizeRequiredForProcess"
+private static final List keyedEncryptionMethods =
Arrays.stream(EncryptionMethod.values())
+.filter(EncryptionMethod::isKeyedCipher)
+.collect(Collectors.toList());
-private static final List keyedEncryptionMethods =
EncryptionMethod.values().findAll { it.keyedCipher }
+private static SecretKey key;
-private static final SecretKey key = new
SecretKeySpec(Hex.decodeHex(KEY_HEX as char[]), "AES")
@BeforeAll
static void setUpOnce() throws Exception {
-Security.addProvider(new BouncyCastleProvider())
+Security.addProvider(new BouncyCastleProvider());
-logger.metaClass.methodMissing = { String name, args ->
-logger.info("[${name?.toUpperCase()}] ${(args as List).join(" ")}")
+try {
+key = new SecretKeySpec(Hex.decodeHex(KEY_HEX.toCharArray()),
"AES");
+} catch (final DecoderException e) {
+throw new RuntimeException(e);
}
}
private static boolean isUnlimitedStrengthCryptoAvailable() {
-Cipher.getMaxAllowedKeyLength("AES") > 128
+try {
+return Cipher.getMaxAllowedKeyLength("AES") > 128;
+} catch (final NoSuchAlgorithmException e) {
+throw new RuntimeException(e);
+}
}
Review Comment:
Will remove it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] emiliosetiadarma commented on a diff in pull request #7238: NIFI-11531: refactored tests in nifi-security-utils
emiliosetiadarma commented on code in PR #7238:
URL: https://github.com/apache/nifi/pull/7238#discussion_r1194416953
##
nifi-commons/nifi-security-utils/src/test/java/org/apache/nifi/security/util/crypto/AESKeyedCipherProviderTest.java:
##
@@ -14,311 +14,298 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.nifi.security.util.crypto
+package org.apache.nifi.security.util.crypto;
-import org.apache.commons.codec.binary.Hex
-import org.apache.nifi.security.util.EncryptionMethod
-import org.bouncycastle.jce.provider.BouncyCastleProvider
-import org.junit.jupiter.api.BeforeAll
-import org.junit.jupiter.api.Test
-import org.slf4j.Logger
-import org.slf4j.LoggerFactory
+import org.apache.commons.codec.DecoderException;
+import org.apache.commons.codec.binary.Hex;
+import org.apache.nifi.security.util.EncryptionMethod;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.Test;
-import javax.crypto.Cipher
-import javax.crypto.SecretKey
-import javax.crypto.spec.SecretKeySpec
-import java.security.SecureRandom
-import java.security.Security
+import javax.crypto.Cipher;
+import javax.crypto.SecretKey;
+import javax.crypto.spec.SecretKeySpec;
+import java.security.NoSuchAlgorithmException;
+import java.security.SecureRandom;
+import java.security.Security;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+import java.util.stream.Collectors;
-import static org.junit.jupiter.api.Assertions.assertEquals
-import static org.junit.jupiter.api.Assertions.assertFalse
-import static org.junit.jupiter.api.Assertions.assertThrows
-import static org.junit.jupiter.api.Assertions.assertTrue
-import static org.junit.jupiter.api.Assumptions.assumeTrue
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+import static org.junit.jupiter.api.Assumptions.assumeTrue;
-class AESKeyedCipherProviderGroovyTest {
-private static final Logger logger =
LoggerFactory.getLogger(AESKeyedCipherProviderGroovyTest.class)
+public class AESKeyedCipherProviderTest {
+private static final String KEY_HEX = "0123456789ABCDEFFEDCBA9876543210";
-private static final String KEY_HEX = "0123456789ABCDEFFEDCBA9876543210"
+private static final String PLAINTEXT = "ExactBlockSizeRequiredForProcess";
-private static final String PLAINTEXT = "ExactBlockSizeRequiredForProcess"
+private static final List keyedEncryptionMethods =
Arrays.stream(EncryptionMethod.values())
+.filter(EncryptionMethod::isKeyedCipher)
+.collect(Collectors.toList());
-private static final List keyedEncryptionMethods =
EncryptionMethod.values().findAll { it.keyedCipher }
+private static SecretKey key;
-private static final SecretKey key = new
SecretKeySpec(Hex.decodeHex(KEY_HEX as char[]), "AES")
@BeforeAll
static void setUpOnce() throws Exception {
-Security.addProvider(new BouncyCastleProvider())
+Security.addProvider(new BouncyCastleProvider());
Review Comment:
I wasn't able to get the tests to complete without it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-11471) Component Timeouts are configurable in Stateless Nifi
[
https://issues.apache.org/jira/browse/NIFI-11471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Young updated NIFI-11471:
---
Status: Patch Available (was: Open)
https://github.com/apache/nifi/pull/7242
> Component Timeouts are configurable in Stateless Nifi
> -
>
> Key: NIFI-11471
> URL: https://issues.apache.org/jira/browse/NIFI-11471
> Project: Apache NiFi
> Issue Type: Improvement
> Components: NiFi Stateless
>Reporter: Dye357
>Assignee: David Young
>Priority: Major
>
> Currently in Stateless Nifi each component is required to start within a
> statically defined timeout. In some cases (IE controller services which pull
> down web resources) this timeout is too short and causes the entire Stateless
> Nifi instance to fail starting.
> {code:java}
> private static final long COMPONENT_ENABLE_TIMEOUT_MILLIS =
> TimeUnit.SECONDS.toMillis(10); {code}
> Suggested improvement is to make this timeout configurable.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #7249: NIFI-11547: Fix DBCPConnectionPool verification
exceptionfactory commented on code in PR #7249:
URL: https://github.com/apache/nifi/pull/7249#discussion_r1194371642
##
nifi-nar-bundles/nifi-extension-utils/nifi-dbcp-base/src/main/java/org/apache/nifi/dbcp/AbstractDBCPConnectionPool.java:
##
@@ -157,7 +157,7 @@ private void loginKerberos(KerberosUser kerberosUser)
throws InitializationExcep
protected abstract DataSourceConfiguration
getDataSourceConfiguration(final ConfigurationContext context);
-protected void configureDataSource(final ConfigurationContext context,
final DataSourceConfiguration configuration) {
+protected void configureDataSource(final ConfigurationContext context,
final BasicDataSource dataSource, final DataSourceConfiguration configuration) {
Review Comment:
Naming this variable `dataSource` shadows the protected member variable, so
the property name should be changed for clarity.
##
nifi-nar-bundles/nifi-extension-utils/nifi-dbcp-base/src/main/java/org/apache/nifi/dbcp/AbstractDBCPConnectionPool.java:
##
@@ -79,7 +79,7 @@ public List verify(final
ConfigurationContext context,
final BasicDataSource dataSource = new BasicDataSource();
Review Comment:
This variable name shadows the protected member variable name, so it would
be helpful to rename the variable in this method.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-11547) DBCP Verification Error - Failes to configure data source
[
https://issues.apache.org/jira/browse/NIFI-11547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17722929#comment-17722929
]
Lehel Boér commented on NIFI-11547:
---
Created PR:
https://github.com/apache/nifi/pull/7249
> DBCP Verification Error - Failes to configure data source
> --
>
> Key: NIFI-11547
> URL: https://issues.apache.org/jira/browse/NIFI-11547
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.21.0
>Reporter: Roman
>Assignee: Lehel Boér
>Priority: Critical
> Attachments: image.png
>
>
> After upgrading Nifi from 1.19.1 to 1.21.0 an Verification Error appears for
> DBCP Connection Pool Controller Service in Verify process.
> Nothing changed in configuration and happens for all DBCP Controller Services
> doesn't matter which database is configured.
> DB-Connection, Username, PW and Driver is everything fine and working.
> Also the enabled processor service is working in and configured flow
> You can see the error in screenshot and stacktrace as attachment.
>
> {code:java}
> 2023-05-15 14:53:12,346 ERROR [Verify Controller Service Config Thread-1]
> org.apache.nifi.dbcp.DBCPConnectionPool
> DBCPConnectionPool[id=1f437b6b-0188-1000--ff2d84f1] Failed to
> configure Data Source.
> java.lang.NullPointerException: null
> at
> org.apache.nifi.dbcp.AbstractDBCPConnectionPool.configureDataSource(AbstractDBCPConnectionPool.java:163)
> at
> org.apache.nifi.dbcp.AbstractDBCPConnectionPool.verify(AbstractDBCPConnectionPool.java:82)
> at
> org.apache.nifi.controller.service.StandardControllerServiceNode.verifyConfiguration(StandardControllerServiceNode.java:492)
> at
> org.apache.nifi.web.dao.impl.StandardControllerServiceDAO.verifyConfiguration(StandardControllerServiceDAO.java:424)
> at
> org.apache.nifi.web.dao.impl.StandardControllerServiceDAO$$FastClassBySpringCGLIB$$6b77954.invoke()
> at
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:708)
> at
> org.apache.nifi.web.dao.impl.StandardControllerServiceDAO$$EnhancerBySpringCGLIB$$2ebb6749.verifyConfiguration()
> at
> org.apache.nifi.web.StandardNiFiServiceFacade.performControllerServiceConfigVerification(StandardNiFiServiceFacade.java:2933)
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
> at
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:708)
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$$EnhancerBySpringCGLIB$$4af10f85.performControllerServiceConfigVerification()
> at
> org.apache.nifi.web.api.ControllerServiceResource.lambda$performAsyncConfigVerification$28(ControllerServiceResource.java:1110)
> at
> org.apache.nifi.web.api.concurrent.AsyncRequestManager$2.run(AsyncRequestManager.java:117)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
> at
>
[jira] [Updated] (NIFI-11547) DBCP Verification Error - Failes to configure data source
[
https://issues.apache.org/jira/browse/NIFI-11547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lehel Boér updated NIFI-11547:
--
Status: Patch Available (was: Open)
> DBCP Verification Error - Failes to configure data source
> --
>
> Key: NIFI-11547
> URL: https://issues.apache.org/jira/browse/NIFI-11547
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.21.0
>Reporter: Roman
>Assignee: Lehel Boér
>Priority: Critical
> Attachments: image.png
>
>
> After upgrading Nifi from 1.19.1 to 1.21.0 an Verification Error appears for
> DBCP Connection Pool Controller Service in Verify process.
> Nothing changed in configuration and happens for all DBCP Controller Services
> doesn't matter which database is configured.
> DB-Connection, Username, PW and Driver is everything fine and working.
> Also the enabled processor service is working in and configured flow
> You can see the error in screenshot and stacktrace as attachment.
>
> {code:java}
> 2023-05-15 14:53:12,346 ERROR [Verify Controller Service Config Thread-1]
> org.apache.nifi.dbcp.DBCPConnectionPool
> DBCPConnectionPool[id=1f437b6b-0188-1000--ff2d84f1] Failed to
> configure Data Source.
> java.lang.NullPointerException: null
> at
> org.apache.nifi.dbcp.AbstractDBCPConnectionPool.configureDataSource(AbstractDBCPConnectionPool.java:163)
> at
> org.apache.nifi.dbcp.AbstractDBCPConnectionPool.verify(AbstractDBCPConnectionPool.java:82)
> at
> org.apache.nifi.controller.service.StandardControllerServiceNode.verifyConfiguration(StandardControllerServiceNode.java:492)
> at
> org.apache.nifi.web.dao.impl.StandardControllerServiceDAO.verifyConfiguration(StandardControllerServiceDAO.java:424)
> at
> org.apache.nifi.web.dao.impl.StandardControllerServiceDAO$$FastClassBySpringCGLIB$$6b77954.invoke()
> at
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:708)
> at
> org.apache.nifi.web.dao.impl.StandardControllerServiceDAO$$EnhancerBySpringCGLIB$$2ebb6749.verifyConfiguration()
> at
> org.apache.nifi.web.StandardNiFiServiceFacade.performControllerServiceConfigVerification(StandardNiFiServiceFacade.java:2933)
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
> at
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:708)
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$$EnhancerBySpringCGLIB$$4af10f85.performControllerServiceConfigVerification()
> at
> org.apache.nifi.web.api.ControllerServiceResource.lambda$performAsyncConfigVerification$28(ControllerServiceResource.java:1110)
> at
> org.apache.nifi.web.api.concurrent.AsyncRequestManager$2.run(AsyncRequestManager.java:117)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
> at
>
[GitHub] [nifi] Lehel44 opened a new pull request, #7249: NIFI-11547: Fix DBCPConnectionPool verification
Lehel44 opened a new pull request, #7249: URL: https://github.com/apache/nifi/pull/7249 # Summary [NIFI-11547](https://issues.apache.org/jira/browse/NIFI-11547) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-11552) Support FlowFile attributes in PutIceberg's Table Name property
Matt Burgess created NIFI-11552: --- Summary: Support FlowFile attributes in PutIceberg's Table Name property Key: NIFI-11552 URL: https://issues.apache.org/jira/browse/NIFI-11552 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: Matt Burgess Fix For: 1.latest, 2.latest The documentation for PutIceberg's Table Name property says it doesn’t support any Expression Language but the code calls the evaluate method on the property without passing in a FlowFile, so at the very least it supports Variable Registry. this Jira proposes to add EL support including the FlowFile attributes for the Table Name property and update the documentation to reflect the new behavior. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-11551) Default Use Avro Logical Types to 'true'
Matt Burgess created NIFI-11551: --- Summary: Default Use Avro Logical Types to 'true' Key: NIFI-11551 URL: https://issues.apache.org/jira/browse/NIFI-11551 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: Matt Burgess Fix For: 2.latest Starting with Apache NiFi 2.0, we should default any Use Avro Logical Types properties to 'true'. NiFi supports most/all Avro logical types as native NiFi record field types, so to get better datatype resolution downstream, we should be using Avro logical types wherever possible. Historically, the reason it defaulted to false was because NiFi originally used Avro before logical types were implemented, so when they were added to Avro, NiFi kept the existing behavior of not using logical types by default. At this point we should use logical types wherever possible (by default). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2107) make package creates faulty tar.gz on macOS
[ https://issues.apache.org/jira/browse/MINIFICPP-2107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-2107: Resolution: Fixed Status: Resolved (was: Patch Available) > make package creates faulty tar.gz on macOS > --- > > Key: MINIFICPP-2107 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2107 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > INSTALL_RPATH is set to $ORIGIN, which doesnt seemed to work on macOS (we > should use @loader_path) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2101) Compilation failure in PutSFTPTests (libc++)
[ https://issues.apache.org/jira/browse/MINIFICPP-2101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-2101: Resolution: Fixed Status: Resolved (was: Patch Available) > Compilation failure in PutSFTPTests (libc++) > > > Key: MINIFICPP-2101 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2101 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Martin Zink >Assignee: Martin Zink >Priority: Minor > Time Spent: 4h 10m > Remaining Estimate: 0h > > file_clock::from_sys(modification_time) used in PutSFTPTests is not available > on every supported platform -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] szaszm closed pull request #1570: MINIFICPP-2107 fix macOS make package
szaszm closed pull request #1570: MINIFICPP-2107 fix macOS make package URL: https://github.com/apache/nifi-minifi-cpp/pull/1570 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm closed pull request #1560: MINIFICPP-2101 Compilation fix in PutSFTPTests (libc++)
szaszm closed pull request #1560: MINIFICPP-2101 Compilation fix in PutSFTPTests (libc++) URL: https://github.com/apache/nifi-minifi-cpp/pull/1560 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #4044: NIFI-4957: Add ability to get JOLT transform from file
exceptionfactory commented on code in PR #4044:
URL: https://github.com/apache/nifi/pull/4044#discussion_r1194307760
##
nifi-nar-bundles/nifi-jolt-record-bundle/nifi-jolt-record-processors/src/main/java/org/apache/nifi/processors/jolt/record/JoltTransformRecord.java:
##
@@ -141,12 +145,23 @@ public class JoltTransformRecord extends
AbstractProcessor {
static final PropertyDescriptor JOLT_SPEC = new
PropertyDescriptor.Builder()
.name("jolt-record-spec")
.displayName("Jolt Specification")
-.description("Jolt Specification for transform of record data.
This value is ignored if the Jolt Sort Transformation is selected.")
+.description("Jolt Specification for transform of record data.
Only one of 'Jolt Specification' or 'Path To Jolt Specification' may be used. "
++ "This value is ignored if the Jolt Sort Transformation
is selected.")
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
.required(false)
.build();
+static final PropertyDescriptor JOLT_SPEC_FILE = new
PropertyDescriptor.Builder()
+.name("jolt-spec-file")
+.displayName("Path To Jolt Specification")
+.description("Path to location of a JOLT specification file. Only
one of 'Jolt Specification' or 'Path To Jolt Specification' may be used. "
++ "This value is ignored if the Jolt Sort Transformation
is selected.")
+.required(false)
+.identifiesExternalResource(ResourceCardinality.SINGLE,
ResourceType.FILE)
Review Comment:
@mattyb149 Is it possible to enhance the existing `Jolt Specification`
property to support multiple resource types? That way the existing property
could be either a text string as it is now, or a file reference.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-11550) Refactor Groovy tests in nifi-commons/nifi-expression-language to Java (and JUnit 5)
[ https://issues.apache.org/jira/browse/NIFI-11550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17722920#comment-17722920 ] David Handermann commented on NIFI-11550: - Moving the test methods to TestQuery (without the logging, and with corrected code style), sounds like the best approach. > Refactor Groovy tests in nifi-commons/nifi-expression-language to Java (and > JUnit 5) > > > Key: NIFI-11550 > URL: https://issues.apache.org/jira/browse/NIFI-11550 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Minor > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (NIFI-11550) Refactor Groovy tests in nifi-commons/nifi-expression-language to Java (and JUnit 5)
[
https://issues.apache.org/jira/browse/NIFI-11550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17722919#comment-17722919
]
Daniel Stieglitz edited comment on NIFI-11550 at 5/15/23 7:52 PM:
--
[~exceptionfactory] There is already a Junit 5 test class for this
{code:java}
nifi-commons/nifi-expression-language/src/test/java/org/apache/nifi/attribute/expression/language/TestQuery.java
{code} should I just add the four tests from
{code:java}
nifi-commons/nifi-expression-language/src/test/groovy/org/apache/nifi/attribute/expression/language/QueryGroovyTest.groovy
{code}
to it or should I create new test class?
was (Author: JIRAUSER294662):
[~exceptionfactory] There is already a Junit 5 test class for this
{code:java}
nifi-commons/nifi-expression-language/src/test/java/org/apache/nifi/attribute/expression/language/TestQuery.java
{code} should I just add the four tests from
{code:java}
nifi-commons/nifi-expression-language/src/test/groovy/org/apache/nifi/attribute/expression/language/QueryGroovyTest.groovy
{code}
to it?
> Refactor Groovy tests in nifi-commons/nifi-expression-language to Java (and
> JUnit 5)
>
>
> Key: NIFI-11550
> URL: https://issues.apache.org/jira/browse/NIFI-11550
> Project: Apache NiFi
> Issue Type: Sub-task
>Reporter: Daniel Stieglitz
>Assignee: Daniel Stieglitz
>Priority: Minor
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #7222: NIFI-11513 Remove deprecated NarProvider and implementations
exceptionfactory commented on code in PR #7222:
URL: https://github.com/apache/nifi/pull/7222#discussion_r1194298330
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/conf/nifi.properties:
##
@@ -383,7 +383,3 @@
nifi.performance.tracking.percentage=${nifi.performance.tracking.percentage}
# nifi.nar.library.provider.hdfs.kerberos.principal=n...@nifi.com
# nifi.nar.library.provider.hdfs.kerberos.keytab=/path/to/nifi.keytab
# nifi.nar.library.provider.hdfs.kerberos.password=
-#
-# Example NiFi Registry NAR Provider:
-#
nifi.nar.library.provider.nifi-registry.implementation=org.apache.nifi.registry.extension.NiFiRegistryNarProvider
-# nifi.nar.library.provider.nifi-registry.url=http://localhost:18080
Review Comment:
@bbende Pushed an update restoring the Registry example using the preferred
Provider class name.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-11550) Refactor Groovy tests in nifi-commons/nifi-expression-language to Java (and JUnit 5)
[
https://issues.apache.org/jira/browse/NIFI-11550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17722919#comment-17722919
]
Daniel Stieglitz commented on NIFI-11550:
-
[~exceptionfactory] There is already a Junit 5 test class for this
{code:java}
nifi-commons/nifi-expression-language/src/test/java/org/apache/nifi/attribute/expression/language/TestQuery.java
{code} should I just add the four tests from
{code:java}
nifi-commons/nifi-expression-language/src/test/groovy/org/apache/nifi/attribute/expression/language/QueryGroovyTest.groovy
{code}
to it?
> Refactor Groovy tests in nifi-commons/nifi-expression-language to Java (and
> JUnit 5)
>
>
> Key: NIFI-11550
> URL: https://issues.apache.org/jira/browse/NIFI-11550
> Project: Apache NiFi
> Issue Type: Sub-task
>Reporter: Daniel Stieglitz
>Assignee: Daniel Stieglitz
>Priority: Minor
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-11550) Refactor Groovy tests in nifi-commons/nifi-expression-language to Java (and JUnit 5)
Daniel Stieglitz created NIFI-11550: --- Summary: Refactor Groovy tests in nifi-commons/nifi-expression-language to Java (and JUnit 5) Key: NIFI-11550 URL: https://issues.apache.org/jira/browse/NIFI-11550 Project: Apache NiFi Issue Type: Sub-task Reporter: Daniel Stieglitz Assignee: Daniel Stieglitz -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #7222: NIFI-11513 Remove deprecated NarProvider and implementations
exceptionfactory commented on code in PR #7222:
URL: https://github.com/apache/nifi/pull/7222#discussion_r1194270582
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/conf/nifi.properties:
##
@@ -383,7 +383,3 @@
nifi.performance.tracking.percentage=${nifi.performance.tracking.percentage}
# nifi.nar.library.provider.hdfs.kerberos.principal=n...@nifi.com
# nifi.nar.library.provider.hdfs.kerberos.keytab=/path/to/nifi.keytab
# nifi.nar.library.provider.hdfs.kerberos.password=
-#
-# Example NiFi Registry NAR Provider:
-#
nifi.nar.library.provider.nifi-registry.implementation=org.apache.nifi.registry.extension.NiFiRegistryNarProvider
-# nifi.nar.library.provider.nifi-registry.url=http://localhost:18080
Review Comment:
Good point, will make the adjustment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-11528) Refactor Groovy tests in nifi-commons/nifi-utils to Java (and JUnit 5)
[ https://issues.apache.org/jira/browse/NIFI-11528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Stieglitz updated NIFI-11528: Status: Patch Available (was: Open) > Refactor Groovy tests in nifi-commons/nifi-utils to Java (and JUnit 5) > -- > > Key: NIFI-11528 > URL: https://issues.apache.org/jira/browse/NIFI-11528 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Minor > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] dan-s1 opened a new pull request, #7248: [NIFI-11528] Refactored Groovy test in nifi-commons/nifi-utils to Jav…
dan-s1 opened a new pull request, #7248: URL: https://github.com/apache/nifi/pull/7248 …a (and JUnit 5) # Summary [NIFI-11528](https://issues.apache.org/jira/browse/NIFI-11528) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] bbende commented on a diff in pull request #7222: NIFI-11513 Remove deprecated NarProvider and implementations
bbende commented on code in PR #7222:
URL: https://github.com/apache/nifi/pull/7222#discussion_r1194258175
##
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/conf/nifi.properties:
##
@@ -383,7 +383,3 @@
nifi.performance.tracking.percentage=${nifi.performance.tracking.percentage}
# nifi.nar.library.provider.hdfs.kerberos.principal=n...@nifi.com
# nifi.nar.library.provider.hdfs.kerberos.keytab=/path/to/nifi.keytab
# nifi.nar.library.provider.hdfs.kerberos.password=
-#
-# Example NiFi Registry NAR Provider:
-#
nifi.nar.library.provider.nifi-registry.implementation=org.apache.nifi.registry.extension.NiFiRegistryNarProvider
-# nifi.nar.library.provider.nifi-registry.url=http://localhost:18080
Review Comment:
Could we leave this block in the properties file, but just fix the class
name from `NiFiRegistryNarProvider` to `NiFiRegistryExternalResourceProvider` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on pull request #7194: NIFI-11167 - Add Excel Record Reader
exceptionfactory commented on PR #7194: URL: https://github.com/apache/nifi/pull/7194#issuecomment-1548388099 Thanks @dan-s1, this is on my list to review, I plan to take a closer look soon. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-11539) Remove ZooKeeper Migrator Toolkit
[ https://issues.apache.org/jira/browse/NIFI-11539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Bende updated NIFI-11539: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Remove ZooKeeper Migrator Toolkit > - > > Key: NIFI-11539 > URL: https://issues.apache.org/jira/browse/NIFI-11539 > Project: Apache NiFi > Issue Type: Task > Components: Tools and Build >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Fix For: 2.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > The ZooKeeper Migrator Toolkit provided the ability to transfer ZooKeeper > state from one cluster to another using intermediate files for transferring > information. The current implementation does not support TLS, which limits > its usefulness for modern deployments. > With the introduction of generalized Cluster State migration in NIFI-10976, > the ZooKeeper Migrator Toolkit should be removed in the next major release > version. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11539) Remove ZooKeeper Migrator Toolkit
[ https://issues.apache.org/jira/browse/NIFI-11539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17722901#comment-17722901 ] ASF subversion and git services commented on NIFI-11539: Commit b9f4d02094e0e0d5f2bfb9fceecaccbb22988341 in nifi's branch refs/heads/main from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=b9f4d02094 ] NIFI-11539 Removed ZooKeeper Migrator Toolkit (#7237) > Remove ZooKeeper Migrator Toolkit > - > > Key: NIFI-11539 > URL: https://issues.apache.org/jira/browse/NIFI-11539 > Project: Apache NiFi > Issue Type: Task > Components: Tools and Build >Reporter: David Handermann >Assignee: David Handermann >Priority: Major > Fix For: 2.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > The ZooKeeper Migrator Toolkit provided the ability to transfer ZooKeeper > state from one cluster to another using intermediate files for transferring > information. The current implementation does not support TLS, which limits > its usefulness for modern deployments. > With the introduction of generalized Cluster State migration in NIFI-10976, > the ZooKeeper Migrator Toolkit should be removed in the next major release > version. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] bbende merged pull request #7237: NIFI-11539 Remove ZooKeeper Migrator Toolkit
bbende merged PR #7237: URL: https://github.com/apache/nifi/pull/7237 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-11548) Upgrade Apache Tika to 2.8.0
[ https://issues.apache.org/jira/browse/NIFI-11548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-11548: -- Fix Version/s: 2.0.0 (was: 2.latest) Resolution: Fixed Status: Resolved (was: Patch Available) > Upgrade Apache Tika to 2.8.0 > > > Key: NIFI-11548 > URL: https://issues.apache.org/jira/browse/NIFI-11548 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Fix For: 2.0.0 > > > [Apache Tika 2.8.0|https://tika.apache.org/2.8.0/index.html] includes a > number of bug fixes and type detection improvements. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11548) Upgrade Apache Tika to 2.8.0
[ https://issues.apache.org/jira/browse/NIFI-11548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17722899#comment-17722899 ] ASF subversion and git services commented on NIFI-11548: Commit bec6ceda1f549205f08cb66854cfc6c26f68bb2d in nifi's branch refs/heads/main from David Handermann [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=bec6ceda1f ] NIFI-11548 Upgraded Apache Tika from 2.7.0 to 2.8.0 Signed-off-by: Pierre Villard This closes #7247. > Upgrade Apache Tika to 2.8.0 > > > Key: NIFI-11548 > URL: https://issues.apache.org/jira/browse/NIFI-11548 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Fix For: 2.latest > > > [Apache Tika 2.8.0|https://tika.apache.org/2.8.0/index.html] includes a > number of bug fixes and type detection improvements. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11528) Refactor Groovy tests in nifi-commons/nifi-utils to Java (and JUnit 5)
[ https://issues.apache.org/jira/browse/NIFI-11528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17722897#comment-17722897 ] David Handermann commented on NIFI-11528: - Closed, thanks! > Refactor Groovy tests in nifi-commons/nifi-utils to Java (and JUnit 5) > -- > > Key: NIFI-11528 > URL: https://issues.apache.org/jira/browse/NIFI-11528 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Minor > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-11083) FormatUtils not converting milliseconds to nanoseconds correctly
[
https://issues.apache.org/jira/browse/NIFI-11083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann resolved NIFI-11083.
-
Resolution: Not A Problem
> FormatUtils not converting milliseconds to nanoseconds correctly
>
>
> Key: NIFI-11083
> URL: https://issues.apache.org/jira/browse/NIFI-11083
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Daniel Stieglitz
>Priority: Major
>
> While refactoring the unit tests in NIFI-11035 it was discovered that
> {code:java}
> nifi/nifi-commons/nifi-utils/src/main/java/org/apache/nifi/util/FormatUtils.java
> {code} method
> {code:java}
> makeWholeNumberTime
> {code}
> is not converting milliseconds to nanoseconds correctly. The given unit test
> was where milliseconds was 0.75, and in nanoseconds should be 75 but 750
> was returned.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-11528) Refactor Groovy tests in nifi-commons/nifi-utils to Java (and JUnit 5)
[ https://issues.apache.org/jira/browse/NIFI-11528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17722896#comment-17722896 ] Daniel Stieglitz commented on NIFI-11528: - [~exceptionfactory] So can NIFI-11803 be closed as this is not a bug? > Refactor Groovy tests in nifi-commons/nifi-utils to Java (and JUnit 5) > -- > > Key: NIFI-11528 > URL: https://issues.apache.org/jira/browse/NIFI-11528 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Minor > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (NIFI-11528) Refactor Groovy tests in nifi-commons/nifi-utils to Java (and JUnit 5)
[ https://issues.apache.org/jira/browse/NIFI-11528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17722896#comment-17722896 ] Daniel Stieglitz edited comment on NIFI-11528 at 5/15/23 6:27 PM: -- [~exceptionfactory] So can NIFI-11083 be closed as this is not a bug? was (Author: JIRAUSER294662): [~exceptionfactory] So can NIFI-11803 be closed as this is not a bug? > Refactor Groovy tests in nifi-commons/nifi-utils to Java (and JUnit 5) > -- > > Key: NIFI-11528 > URL: https://issues.apache.org/jira/browse/NIFI-11528 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Minor > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-11549) Add Azure Queue Storage Processors using Azure SDK 12
[
https://issues.apache.org/jira/browse/NIFI-11549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Emilio Setiadarma reassigned NIFI-11549:
Assignee: Emilio Setiadarma
> Add Azure Queue Storage Processors using Azure SDK 12
> -
>
> Key: NIFI-11549
> URL: https://issues.apache.org/jira/browse/NIFI-11549
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Reporter: David Handermann
>Assignee: Emilio Setiadarma
>Priority: Major
> Fix For: 2.latest
>
>
> The {{GetAzureQueueStorage}} and {{PutAzureQueueStorage}} Processors depend
> on the legacy Microsoft Azure SDK libraries. The current implementations also
> depend on the Storage Credentials Service based on the legacy SDK.
> Following the pattern of the Azure Blob Storage Processors, new
> implementations should be created that depend on the current Azure SDK
> version 12. Adding new Processors will enable subsequent deprecation of the
> current versions. The new Processors should leverage the Credentials Service
> with Azure SDK 12.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] mattyb149 commented on pull request #7236: NIFI-11537: Add support for Iceberg tables to UpdateHive3Table
mattyb149 commented on PR #7236: URL: https://github.com/apache/nifi/pull/7236#issuecomment-1548327414 Apache Hive 3.1.3 (the current version used in Apache NiFi) doesn't support `STORED BY` and `STORED AS` at the same time. Need to change the logic (for now) to use `STORED BY` if present and `STORED AS` otherwise, plus any other TBLPROPERTIES settings needed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-11528) Refactor Groovy tests in nifi-commons/nifi-utils to Java (and JUnit 5)
[ https://issues.apache.org/jira/browse/NIFI-11528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17722887#comment-17722887 ] David Handermann commented on NIFI-11528: - That sounds correct, thanks! > Refactor Groovy tests in nifi-commons/nifi-utils to Java (and JUnit 5) > -- > > Key: NIFI-11528 > URL: https://issues.apache.org/jira/browse/NIFI-11528 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Minor > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11528) Refactor Groovy tests in nifi-commons/nifi-utils to Java (and JUnit 5)
[
https://issues.apache.org/jira/browse/NIFI-11528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17722884#comment-17722884
]
Daniel Stieglitz commented on NIFI-11528:
-
[~exceptionfactory] Thanks for the reminder. Looking more closely at the code
makes me feel the author chose an incorrect test case and the code is working
as expected. It is true that .75 milliseconds equals 75 nanoseconds but
the point of method
{code:java}
makeWholeNumberTime
{code}
is not to find the smallest possible time unit but rather to find the next
smaller time unit with a whole number. The next possible smaller time unit
which has a whole number for .75 milliseconds is microseconds (not nanoseconds)
where the whole number is 750. I would think the test case should reflect the
conversion from milliseconds to microseconds. Please let me know whether you
concur. Thanks!
> Refactor Groovy tests in nifi-commons/nifi-utils to Java (and JUnit 5)
> --
>
> Key: NIFI-11528
> URL: https://issues.apache.org/jira/browse/NIFI-11528
> Project: Apache NiFi
> Issue Type: Sub-task
>Reporter: Daniel Stieglitz
>Assignee: Daniel Stieglitz
>Priority: Minor
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] exceptionfactory commented on pull request #7233: NIFI-11532 Remove JUnit 4 and Groovy Test from default dependencies
exceptionfactory commented on PR #7233: URL: https://github.com/apache/nifi/pull/7233#issuecomment-1548171034 Rebased to address conflicts with recent changes in main branch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] turcsanyip commented on a diff in pull request #7246: NIFI-11535: Transfer ConnectWebsocket HTTP header flowfile to relatio…
turcsanyip commented on code in PR #7246:
URL: https://github.com/apache/nifi/pull/7246#discussion_r1194007689
##
nifi-nar-bundles/nifi-websocket-bundle/nifi-websocket-processors/src/main/java/org/apache/nifi/processors/websocket/AbstractWebSocketGatewayProcessor.java:
##
@@ -77,6 +77,18 @@ public abstract class AbstractWebSocketGatewayProcessor
extends AbstractSessionF
.description("The WebSocket binary message output")
.build();
+public static final Relationship REL_SUCCESS = new Relationship.Builder()
+.name("success")
+.description("Connection HTTP header attributes in case of
successful connection")
+.autoTerminateDefault(true)
+.build();
+
+public static final Relationship REL_FAILURE = new Relationship.Builder()
+.name("failure")
+.description("Connection HTTP header attributes in case of
connection failure")
+.autoTerminateDefault(true)
+.build();
+
Review Comment:
"Connection HTTP header attributes" is a bit unclear in this context and
other config parameters can be passed also. I would suggest the following
descriptions:
```suggestion
public static final Relationship REL_SUCCESS = new Relationship.Builder()
.name("success")
.description("FlowFile holding connection configuration
attributes (like URL or HTTP headers) in case of successful connection")
.autoTerminateDefault(true)
.build();
public static final Relationship REL_FAILURE = new Relationship.Builder()
.name("failure")
.description("FlowFile holding connection configuration
attributes (like URL or HTTP headers) in case of connection failure")
.autoTerminateDefault(true)
.build();
```
##
nifi-nar-bundles/nifi-websocket-bundle/nifi-websocket-processors/src/test/java/org/apache/nifi/processors/websocket/TestConnectWebSocket.java:
##
@@ -162,10 +154,44 @@ void testDynamicUrlsParsedFromFlowFileAndAbleToConnect()
throws InitializationEx
final List flowFilesForRelationship =
runner.getFlowFilesForRelationship(ConnectWebSocket.REL_CONNECTED);
assertEquals(1, flowFilesForRelationship.size());
+final List flowFilesForSuccess =
runner.getFlowFilesForRelationship(ConnectWebSocket.REL_SUCCESS);
+assertEquals(1, flowFilesForSuccess.size());
+
runner.stop();
webSocketListener.stop();
}
+@Test
+void testDynamicUrlsTransferredToFailure() throws InitializationException {
Review Comment:
The following test name would describe the test scenario more
straightforward:
```suggestion
void testDynamicUrlsParsedFromFlowFileButNotAbleToConnect() throws
InitializationException {
```
##
nifi-nar-bundles/nifi-websocket-bundle/nifi-websocket-processors/src/test/java/org/apache/nifi/processors/websocket/TestConnectWebSocket.java:
##
@@ -162,10 +154,44 @@ void testDynamicUrlsParsedFromFlowFileAndAbleToConnect()
throws InitializationEx
final List flowFilesForRelationship =
runner.getFlowFilesForRelationship(ConnectWebSocket.REL_CONNECTED);
assertEquals(1, flowFilesForRelationship.size());
+final List flowFilesForSuccess =
runner.getFlowFilesForRelationship(ConnectWebSocket.REL_SUCCESS);
+assertEquals(1, flowFilesForSuccess.size());
+
runner.stop();
webSocketListener.stop();
}
+@Test
+void testDynamicUrlsTransferredToFailure() throws InitializationException {
+final TestRunner runner =
TestRunners.newTestRunner(ConnectWebSocket.class);
+
+final String serviceId = "ws-service";
+final String endpointId = "client-1";
+
+MockFlowFile flowFile = getFlowFile();
+runner.enqueue(flowFile);
+
+JettyWebSocketClient service = new JettyWebSocketClient();
+
+
+runner.addControllerService(serviceId, service);
+runner.setProperty(service, JettyWebSocketClient.WS_URI,
"ws://localhost/12345");
Review Comment:
No need to set a fake URL because the failure comes due to ListenWebSocket
not started.
```suggestion
runner.setProperty(service, JettyWebSocketClient.WS_URI,
"ws://localhost/${dynamicUrlPart}");
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-11540) Remove legacy FlowFile attributes from MergeContent and SegmentContent
[ https://issues.apache.org/jira/browse/NIFI-11540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17722844#comment-17722844 ] Michael W Moser commented on NIFI-11540: [~shreeju] we would welcome your help, thank you! > Remove legacy FlowFile attributes from MergeContent and SegmentContent > -- > > Key: NIFI-11540 > URL: https://issues.apache.org/jira/browse/NIFI-11540 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Michael W Moser >Priority: Trivial > > The following attributes were replaced and only maintained for backwards > compatibility. As part of removing technical debt in the 2.0 release, we > should no longer support them. > * segment.identifier > * segment.index > * segment.count > They were replaced by identical fragment.identifier, fragment.index and > fragment.count. I found them in MergeContent and SegmentContent, their > processor documentation and their unit tests. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2091) Update docker tests with ARM64 supported test images
[ https://issues.apache.org/jira/browse/MINIFICPP-2091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gábor Gyimesi updated MINIFICPP-2091: - Fix Version/s: 0.15.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Update docker tests with ARM64 supported test images > > > Key: MINIFICPP-2091 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2091 > Project: Apache NiFi MiNiFi C++ > Issue Type: Test >Reporter: Gábor Gyimesi >Assignee: Gábor Gyimesi >Priority: Major > Fix For: 0.15.0 > > Time Spent: 50m > Remaining Estimate: 0h > > We need to be able to run docker system tests on ARM64 to have the same test > coverage as on x86/x64 architectures. There are some docker images used for > testing that are not available with the currently used version on ARM64. > Images that only need a version upgrade to the latest version to support > ARM64: > mcr.microsoft.com/azure-storage/azurite:3.14.2 -> 3.23.0 > apache/nifi:1.16.0 -> 1.20.0 > adobe/s3mock:2.1.28 -> 2.11.0 > apache/nifi-minifi-c2:1.18.0 -> 1.20.0 > These images use the "latest" version should be pinned to a specific version: > fsouza/fake-gcs-server:latest -> 1.44.1 > opensearchproject/opensearch:latest -> 2.6.0 > The following images have no available ARM64 support: > > sameersbn/squid - Used for HTTP proxy, we may be able to find an alternative > proxy image with ARM64 > wurstmeister/zookeeper - Used for kafka tests, we need to find an alternative > zookeeper image > lordgamez/open62541 - Custom built image, as the official repository does not > regularily build their docker image, we need to build a new custom image on > ARM64 arch > splunk/splunk - No officially build image, but can be customly built using > [https://github.com/splunk/docker-splunk] repository according to > https://github.com/splunk/docker-splunk/issues/516 some custom images can > already be found at > [https://hub.docker.com/r/odidev/docker-splunk/tags|https://hub.docker.com/r/odidev/docker-splunk/tags,] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-11549) Add Azure Queue Storage Processors using Azure SDK 12
David Handermann created NIFI-11549:
---
Summary: Add Azure Queue Storage Processors using Azure SDK 12
Key: NIFI-11549
URL: https://issues.apache.org/jira/browse/NIFI-11549
Project: Apache NiFi
Issue Type: Improvement
Components: Extensions
Reporter: David Handermann
Fix For: 2.latest
The {{GetAzureQueueStorage}} and {{PutAzureQueueStorage}} Processors depend on
the legacy Microsoft Azure SDK libraries. The current implementations also
depend on the Storage Credentials Service based on the legacy SDK.
Following the pattern of the Azure Blob Storage Processors, new implementations
should be created that depend on the current Azure SDK version 12. Adding new
Processors will enable subsequent deprecation of the current versions. The new
Processors should leverage the Credentials Service with Azure SDK 12.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-11545) NiFi Registry integration test failures after changes of NIFI-11116
[ https://issues.apache.org/jira/browse/NIFI-11545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11545: Summary: NiFi Registry integration test failures after changes of NIFI-6 (was: NiFi registry integration test after changes of NIFI-6) > NiFi Registry integration test failures after changes of NIFI-6 > --- > > Key: NIFI-11545 > URL: https://issues.apache.org/jira/browse/NIFI-11545 > Project: Apache NiFi > Issue Type: Test > Components: NiFi Registry >Affects Versions: 1.21.0 >Reporter: Tan Luu >Assignee: Tan Luu >Priority: Trivial > Labels: pull-request-available > Fix For: 2.0.0, 1.22.0 > > Attachments: InvalidTestClass.log, > testConvertOIDCTokenToNiFiTokenShouldHandleBlankIdentityAndNoEmailClaim.log > > > After commit 889767683cb7ce3e3024ac41e11252ae93fb00b8 got merge, NiFi > registry IT test by command "mvn verify -Ptest-all-dbs" failed. Some test > classes does not have correct annotation for JUnit 5. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-11545) NiFi registry integration test after changes of NIFI-11116
[ https://issues.apache.org/jira/browse/NIFI-11545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann resolved NIFI-11545. - Fix Version/s: 2.0.0 1.22.0 Assignee: Tan Luu Resolution: Fixed > NiFi registry integration test after changes of NIFI-6 > -- > > Key: NIFI-11545 > URL: https://issues.apache.org/jira/browse/NIFI-11545 > Project: Apache NiFi > Issue Type: Test > Components: NiFi Registry >Affects Versions: 1.21.0 >Reporter: Tan Luu >Assignee: Tan Luu >Priority: Trivial > Labels: pull-request-available > Fix For: 2.0.0, 1.22.0 > > Attachments: InvalidTestClass.log, > testConvertOIDCTokenToNiFiTokenShouldHandleBlankIdentityAndNoEmailClaim.log > > > After commit 889767683cb7ce3e3024ac41e11252ae93fb00b8 got merge, NiFi > registry IT test by command "mvn verify -Ptest-all-dbs" failed. Some test > classes does not have correct annotation for JUnit 5. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11545) NiFi registry integration test after changes of NIFI-11116
[ https://issues.apache.org/jira/browse/NIFI-11545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17722829#comment-17722829 ] ASF subversion and git services commented on NIFI-11545: Commit 939c8c952458264d085f410a48fb6ab10c04f499 in nifi's branch refs/heads/support/nifi-1.x from nathluu [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=939c8c9524 ] NIFI-11545 Fixed test-all-dbs for Registry integration tests This closes #7244 Signed-off-by: David Handermann (cherry picked from commit 0308b5b404d99aa97166e11586ab348a8456b6d6) > NiFi registry integration test after changes of NIFI-6 > -- > > Key: NIFI-11545 > URL: https://issues.apache.org/jira/browse/NIFI-11545 > Project: Apache NiFi > Issue Type: Test > Components: NiFi Registry >Affects Versions: 1.21.0 >Reporter: Tan Luu >Priority: Trivial > Labels: pull-request-available > Attachments: InvalidTestClass.log, > testConvertOIDCTokenToNiFiTokenShouldHandleBlankIdentityAndNoEmailClaim.log > > > After commit 889767683cb7ce3e3024ac41e11252ae93fb00b8 got merge, NiFi > registry IT test by command "mvn verify -Ptest-all-dbs" failed. Some test > classes does not have correct annotation for JUnit 5. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11545) NiFi registry integration test after changes of NIFI-11116
[ https://issues.apache.org/jira/browse/NIFI-11545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17722828#comment-17722828 ] ASF subversion and git services commented on NIFI-11545: Commit 0308b5b404d99aa97166e11586ab348a8456b6d6 in nifi's branch refs/heads/main from nathluu [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=0308b5b404 ] NIFI-11545 Fixed test-all-dbs for Registry integration tests This closes #7244 Signed-off-by: David Handermann > NiFi registry integration test after changes of NIFI-6 > -- > > Key: NIFI-11545 > URL: https://issues.apache.org/jira/browse/NIFI-11545 > Project: Apache NiFi > Issue Type: Test > Components: NiFi Registry >Affects Versions: 1.21.0 >Reporter: Tan Luu >Priority: Trivial > Labels: pull-request-available > Attachments: InvalidTestClass.log, > testConvertOIDCTokenToNiFiTokenShouldHandleBlankIdentityAndNoEmailClaim.log > > > After commit 889767683cb7ce3e3024ac41e11252ae93fb00b8 got merge, NiFi > registry IT test by command "mvn verify -Ptest-all-dbs" failed. Some test > classes does not have correct annotation for JUnit 5. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory closed pull request #7244: NIFI-11545 Fix registry integration tests issued by 'mvn verify -Ptest-all-dbs'
exceptionfactory closed pull request #7244: NIFI-11545 Fix registry integration tests issued by 'mvn verify -Ptest-all-dbs' URL: https://github.com/apache/nifi/pull/7244 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] bbende commented on pull request #7244: NIFI-11545 Fix registry integration tests issued by 'mvn verify -Ptest-all-dbs'
bbende commented on PR #7244: URL: https://github.com/apache/nifi/pull/7244#issuecomment-1548081926 This looks good overall, there is one checkstyle issue shown in the Static Analysis run by CI ``` [INFO] --- maven-checkstyle-plugin:3.2.1:check (check-style) @ nifi-registry-web-api --- Warning: src/test/java/org/apache/nifi/registry/web/api/UnsecuredITBase.java:[24,8] (imports) UnusedImports: Unused import - org.springframework.test.context.junit4.SpringRunner. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink closed pull request #1551: MINIFICPP-2091 Add ARM64 support for docker system tests
martinzink closed pull request #1551: MINIFICPP-2091 Add ARM64 support for docker system tests URL: https://github.com/apache/nifi-minifi-cpp/pull/1551 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] exceptionfactory commented on a diff in pull request #7238: NIFI-11531: refactored tests in nifi-security-utils
exceptionfactory commented on code in PR #7238:
URL: https://github.com/apache/nifi/pull/7238#discussion_r1193937717
##
nifi-commons/nifi-security-utils/src/test/java/org/apache/nifi/security/util/crypto/AESKeyedCipherProviderTest.java:
##
@@ -14,311 +14,298 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.nifi.security.util.crypto
+package org.apache.nifi.security.util.crypto;
-import org.apache.commons.codec.binary.Hex
-import org.apache.nifi.security.util.EncryptionMethod
-import org.bouncycastle.jce.provider.BouncyCastleProvider
-import org.junit.jupiter.api.BeforeAll
-import org.junit.jupiter.api.Test
-import org.slf4j.Logger
-import org.slf4j.LoggerFactory
+import org.apache.commons.codec.DecoderException;
+import org.apache.commons.codec.binary.Hex;
+import org.apache.nifi.security.util.EncryptionMethod;
+import org.bouncycastle.jce.provider.BouncyCastleProvider;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.Test;
-import javax.crypto.Cipher
-import javax.crypto.SecretKey
-import javax.crypto.spec.SecretKeySpec
-import java.security.SecureRandom
-import java.security.Security
+import javax.crypto.Cipher;
+import javax.crypto.SecretKey;
+import javax.crypto.spec.SecretKeySpec;
+import java.security.NoSuchAlgorithmException;
+import java.security.SecureRandom;
+import java.security.Security;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+import java.util.stream.Collectors;
-import static org.junit.jupiter.api.Assertions.assertEquals
-import static org.junit.jupiter.api.Assertions.assertFalse
-import static org.junit.jupiter.api.Assertions.assertThrows
-import static org.junit.jupiter.api.Assertions.assertTrue
-import static org.junit.jupiter.api.Assumptions.assumeTrue
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+import static org.junit.jupiter.api.Assumptions.assumeTrue;
-class AESKeyedCipherProviderGroovyTest {
-private static final Logger logger =
LoggerFactory.getLogger(AESKeyedCipherProviderGroovyTest.class)
+public class AESKeyedCipherProviderTest {
+private static final String KEY_HEX = "0123456789ABCDEFFEDCBA9876543210";
-private static final String KEY_HEX = "0123456789ABCDEFFEDCBA9876543210"
+private static final String PLAINTEXT = "ExactBlockSizeRequiredForProcess";
-private static final String PLAINTEXT = "ExactBlockSizeRequiredForProcess"
+private static final List keyedEncryptionMethods =
Arrays.stream(EncryptionMethod.values())
+.filter(EncryptionMethod::isKeyedCipher)
+.collect(Collectors.toList());
-private static final List keyedEncryptionMethods =
EncryptionMethod.values().findAll { it.keyedCipher }
+private static SecretKey key;
-private static final SecretKey key = new
SecretKeySpec(Hex.decodeHex(KEY_HEX as char[]), "AES")
@BeforeAll
static void setUpOnce() throws Exception {
-Security.addProvider(new BouncyCastleProvider())
+Security.addProvider(new BouncyCastleProvider());
-logger.metaClass.methodMissing = { String name, args ->
-logger.info("[${name?.toUpperCase()}] ${(args as List).join(" ")}")
+try {
+key = new SecretKeySpec(Hex.decodeHex(KEY_HEX.toCharArray()),
"AES");
+} catch (final DecoderException e) {
+throw new RuntimeException(e);
}
}
private static boolean isUnlimitedStrengthCryptoAvailable() {
-Cipher.getMaxAllowedKeyLength("AES") > 128
+try {
+return Cipher.getMaxAllowedKeyLength("AES") > 128;
+} catch (final NoSuchAlgorithmException e) {
+throw new RuntimeException(e);
+}
}
Review Comment:
This check is no longer needed, so the method can be removed and anything
that depends on it can be adjusted to always test unlimited strength values.
##
nifi-commons/nifi-security-utils/src/test/java/org/apache/nifi/security/util/crypto/AESKeyedCipherProviderTest.java:
##
@@ -14,311 +14,298 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.nifi.security.util.crypto
+package org.apache.nifi.security.util.crypto;
-import org.apache.commons.codec.binary.Hex
-import org.apache.nifi.security.util.EncryptionMethod
-import org.bouncycastle.jce.provider.BouncyCastleProvider
-import org.junit.jupiter.api.BeforeAll
-import org.junit.jupiter.api.Test
-import org.slf4j.Logger
-import org.slf4j.LoggerFactory
+import org.apache.commons.codec.DecoderException;
+import org.apache.commons.codec.binary.Hex;
+import org.apache.nifi.security.util.EncryptionMethod;
+import
[GitHub] [nifi] bbende commented on a diff in pull request #7245: NIFI-8831 Support using MSSQL as nifi registry relation database engine
bbende commented on code in PR #7245:
URL: https://github.com/apache/nifi/pull/7245#discussion_r1193943130
##
nifi-registry/nifi-registry-core/nifi-registry-framework/src/main/java/org/apache/nifi/registry/security/authorization/database/DatabaseUserGroupProvider.java:
##
@@ -121,7 +129,9 @@ public void checkInheritability(final String
proposedFingerprint) throws Authori
@Override
public User addUser(final User user) throws AuthorizationAccessException {
Validate.notNull(user);
-final String sql = "INSERT INTO UGP_USER(IDENTIFIER, IDENTITY) VALUES
(?, ?)";
+final String sql = (databaseType instanceof SQLServerDatabaseType)
+? "INSERT INTO UGP_USER(IDENTIFIER, [IDENTITY]) VALUES (?, ?)"
+: "INSERT INTO UGP_USER(IDENTIFIER, IDENTITY) VALUES (?, ?)";
Review Comment:
In general I think the application shouldn't really be checking the type of
database, with the exception of the Flyway stuff that sets up the DB. It is not
clear to me what the different SQL here is doing, but can it be done in a
general way that conforms to standard SQL?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-11548) Upgrade Apache Tika to 2.8.0
[ https://issues.apache.org/jira/browse/NIFI-11548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-11548: Status: Patch Available (was: Open) > Upgrade Apache Tika to 2.8.0 > > > Key: NIFI-11548 > URL: https://issues.apache.org/jira/browse/NIFI-11548 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: David Handermann >Assignee: David Handermann >Priority: Minor > Labels: dependency-upgrade > Fix For: 2.latest > > > [Apache Tika 2.8.0|https://tika.apache.org/2.8.0/index.html] includes a > number of bug fixes and type detection improvements. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] exceptionfactory opened a new pull request, #7247: NIFI-11548 Upgrade Apache Tika from 2.7.0 to 2.8.0
exceptionfactory opened a new pull request, #7247: URL: https://github.com/apache/nifi/pull/7247 # Summary [NIFI-11548](https://issues.apache.org/jira/browse/NIFI-11548) Upgrades Apache Tika from 2.7.0 to [2.8.0](https://tika.apache.org/2.8.0/index.html) incorporating several bug fixes and type detection improvements. # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [X] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [X] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [X] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [X] Pull Request based on current revision of the `main` branch - [X] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [X] Build completed using `mvn clean install -P contrib-check` - [X] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-11548) Upgrade Apache Tika to 2.8.0
David Handermann created NIFI-11548: --- Summary: Upgrade Apache Tika to 2.8.0 Key: NIFI-11548 URL: https://issues.apache.org/jira/browse/NIFI-11548 Project: Apache NiFi Issue Type: Improvement Components: Core Framework, Extensions Reporter: David Handermann Assignee: David Handermann Fix For: 2.latest [Apache Tika 2.8.0|https://tika.apache.org/2.8.0/index.html] includes a number of bug fixes and type detection improvements. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-11547) DBCP Verification Error - Failes to configure data source
[
https://issues.apache.org/jira/browse/NIFI-11547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lehel Boér reassigned NIFI-11547:
-
Assignee: Lehel Boér
> DBCP Verification Error - Failes to configure data source
> --
>
> Key: NIFI-11547
> URL: https://issues.apache.org/jira/browse/NIFI-11547
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.21.0
>Reporter: Roman
>Assignee: Lehel Boér
>Priority: Critical
> Attachments: image.png
>
>
> After upgrading Nifi from 1.19.1 to 1.21.0 an Verification Error appears for
> DBCP Connection Pool Controller Service in Verify process.
> Nothing changed in configuration and happens for all DBCP Controller Services
> doesn't matter which database is configured.
> DB-Connection, Username, PW and Driver is everything fine and working.
> Also the enabled processor service is working in and configured flow
> You can see the error in screenshot and stacktrace as attachment.
>
> {code:java}
> 2023-05-15 14:53:12,346 ERROR [Verify Controller Service Config Thread-1]
> org.apache.nifi.dbcp.DBCPConnectionPool
> DBCPConnectionPool[id=1f437b6b-0188-1000--ff2d84f1] Failed to
> configure Data Source.
> java.lang.NullPointerException: null
> at
> org.apache.nifi.dbcp.AbstractDBCPConnectionPool.configureDataSource(AbstractDBCPConnectionPool.java:163)
> at
> org.apache.nifi.dbcp.AbstractDBCPConnectionPool.verify(AbstractDBCPConnectionPool.java:82)
> at
> org.apache.nifi.controller.service.StandardControllerServiceNode.verifyConfiguration(StandardControllerServiceNode.java:492)
> at
> org.apache.nifi.web.dao.impl.StandardControllerServiceDAO.verifyConfiguration(StandardControllerServiceDAO.java:424)
> at
> org.apache.nifi.web.dao.impl.StandardControllerServiceDAO$$FastClassBySpringCGLIB$$6b77954.invoke()
> at
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:708)
> at
> org.apache.nifi.web.dao.impl.StandardControllerServiceDAO$$EnhancerBySpringCGLIB$$2ebb6749.verifyConfiguration()
> at
> org.apache.nifi.web.StandardNiFiServiceFacade.performControllerServiceConfigVerification(StandardNiFiServiceFacade.java:2933)
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
> at
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:708)
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$$EnhancerBySpringCGLIB$$4af10f85.performControllerServiceConfigVerification()
> at
> org.apache.nifi.web.api.ControllerServiceResource.lambda$performAsyncConfigVerification$28(ControllerServiceResource.java:1110)
> at
> org.apache.nifi.web.api.concurrent.AsyncRequestManager$2.run(AsyncRequestManager.java:117)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown
[jira] [Updated] (NIFI-11535) Change WebSocket FlowFile removal when supplying HTTP headers
[
https://issues.apache.org/jira/browse/NIFI-11535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lehel Boér updated NIFI-11535:
--
Status: Patch Available (was: In Progress)
> Change WebSocket FlowFile removal when supplying HTTP headers
> -
>
> Key: NIFI-11535
> URL: https://issues.apache.org/jira/browse/NIFI-11535
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Lehel Boér
>Assignee: Lehel Boér
>Priority: Major
>
> final ProcessSession session =
> processSessionFactory.createSession();
> final FlowFile flowFile = session.get();
> try {
> webSocketClientService.connect(endpointId,
> flowFile.getAttributes());
> } finally {
> session.remove(flowFile);
> session.commitAsync();
> }
> In case of connection failure we lose data
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] Lehel44 opened a new pull request, #7246: NIFI-11535: Transfer ConnectWebsocket HTTP header flowfile to relatio…
Lehel44 opened a new pull request, #7246: URL: https://github.com/apache/nifi/pull/7246 …nships # Summary [NIFI-11535](https://issues.apache.org/jira/browse/NIFI-11535) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 11 - [ ] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-11547) DBCP Verification Error - Failes to configure data source
[
https://issues.apache.org/jira/browse/NIFI-11547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman updated NIFI-11547:
-
Description:
After upgrading Nifi from 1.19.1 to 1.21.0 an Verification Error appears for
DBCP Connection Pool Controller Service in Verify process.
Nothing changed in configuration and happens for all DBCP Controller Services
doesn't matter which database is configured.
DB-Connection, Username, PW and Driver is everything fine and working.
Also the enabled processor service is working in and configured flow
You can see the error in screenshot and stacktrace as attachment.
{code:java}
2023-05-15 14:53:12,346 ERROR [Verify Controller Service Config Thread-1]
org.apache.nifi.dbcp.DBCPConnectionPool
DBCPConnectionPool[id=1f437b6b-0188-1000--ff2d84f1] Failed to configure
Data Source.
java.lang.NullPointerException: null
at
org.apache.nifi.dbcp.AbstractDBCPConnectionPool.configureDataSource(AbstractDBCPConnectionPool.java:163)
at
org.apache.nifi.dbcp.AbstractDBCPConnectionPool.verify(AbstractDBCPConnectionPool.java:82)
at
org.apache.nifi.controller.service.StandardControllerServiceNode.verifyConfiguration(StandardControllerServiceNode.java:492)
at
org.apache.nifi.web.dao.impl.StandardControllerServiceDAO.verifyConfiguration(StandardControllerServiceDAO.java:424)
at
org.apache.nifi.web.dao.impl.StandardControllerServiceDAO$$FastClassBySpringCGLIB$$6b77954.invoke()
at
org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
at
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:708)
at
org.apache.nifi.web.dao.impl.StandardControllerServiceDAO$$EnhancerBySpringCGLIB$$2ebb6749.verifyConfiguration()
at
org.apache.nifi.web.StandardNiFiServiceFacade.performControllerServiceConfigVerification(StandardNiFiServiceFacade.java:2933)
at
org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
at
org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
at
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:708)
at
org.apache.nifi.web.StandardNiFiServiceFacade$$EnhancerBySpringCGLIB$$4af10f85.performControllerServiceConfigVerification()
at
org.apache.nifi.web.api.ControllerServiceResource.lambda$performAsyncConfigVerification$28(ControllerServiceResource.java:1110)
at
org.apache.nifi.web.api.concurrent.AsyncRequestManager$2.run(AsyncRequestManager.java:117)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown
Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
Source)
at java.base/java.lang.Thread.run(Unknown Source) {code}
was:
After upgrading Nifi from 1.19.1 to 1.21.0 an Verification Error appears for
DBCP Connection Pool Controller Service in Verify process.
Nothing changed in configuration and happens for all DBCP Controller Services
doesn't matter which database is configured.
You can see the error in screenshot and stacktrace as attachment.
{code:java}
2023-05-15 14:53:12,346 ERROR [Verify Controller Service Config Thread-1]
org.apache.nifi.dbcp.DBCPConnectionPool
[jira] [Created] (NIFI-11547) DBCP Verification Error - Failes to configure data source
Roman created NIFI-11547:
Summary: DBCP Verification Error - Failes to configure data source
Key: NIFI-11547
URL: https://issues.apache.org/jira/browse/NIFI-11547
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Affects Versions: 1.21.0
Reporter: Roman
Attachments: image.png
After upgrading Nifi from 1.19.1 to 1.21.0 an Verification Error appears for
DBCP Connection Pool Controller Service in Verify process.
Nothing changed in configuration and happens for all DBCP Controller Services
doesn't matter which database is configured.
You can see the error in screenshot and stacktrace as attachment.
{code:java}
2023-05-15 14:53:12,346 ERROR [Verify Controller Service Config Thread-1]
org.apache.nifi.dbcp.DBCPConnectionPool
DBCPConnectionPool[id=1f437b6b-0188-1000--ff2d84f1] Failed to configure
Data Source.
java.lang.NullPointerException: null
at
org.apache.nifi.dbcp.AbstractDBCPConnectionPool.configureDataSource(AbstractDBCPConnectionPool.java:163)
at
org.apache.nifi.dbcp.AbstractDBCPConnectionPool.verify(AbstractDBCPConnectionPool.java:82)
at
org.apache.nifi.controller.service.StandardControllerServiceNode.verifyConfiguration(StandardControllerServiceNode.java:492)
at
org.apache.nifi.web.dao.impl.StandardControllerServiceDAO.verifyConfiguration(StandardControllerServiceDAO.java:424)
at
org.apache.nifi.web.dao.impl.StandardControllerServiceDAO$$FastClassBySpringCGLIB$$6b77954.invoke()
at
org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
at
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:708)
at
org.apache.nifi.web.dao.impl.StandardControllerServiceDAO$$EnhancerBySpringCGLIB$$2ebb6749.verifyConfiguration()
at
org.apache.nifi.web.StandardNiFiServiceFacade.performControllerServiceConfigVerification(StandardNiFiServiceFacade.java:2933)
at
org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
at
org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
at
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:708)
at
org.apache.nifi.web.StandardNiFiServiceFacade$$EnhancerBySpringCGLIB$$4af10f85.performControllerServiceConfigVerification()
at
org.apache.nifi.web.api.ControllerServiceResource.lambda$performAsyncConfigVerification$28(ControllerServiceResource.java:1110)
at
org.apache.nifi.web.api.concurrent.AsyncRequestManager$2.run(AsyncRequestManager.java:117)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown
Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
Source)
at java.base/java.lang.Thread.run(Unknown Source) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-11528) Refactor Groovy tests in nifi-commons/nifi-utils to Java (and JUnit 5)
[ https://issues.apache.org/jira/browse/NIFI-11528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17722773#comment-17722773 ] David Handermann commented on NIFI-11528: - Thanks for working on this [~dstiegli1]! It looks like you noticed this problem some time ago and wrote up a separate Jira issue to correct the underlying problem: NIFI-11083 With that background, it sounds like the runtime fix should be handled in that Jira, and the test code should follow the current behavior until that issue is fixed. Alternatively, you could address both the runtime issue and the test refactor in a single pull request. > Refactor Groovy tests in nifi-commons/nifi-utils to Java (and JUnit 5) > -- > > Key: NIFI-11528 > URL: https://issues.apache.org/jira/browse/NIFI-11528 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Minor > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a diff in pull request #1567: MINIFICPP-2111 - Remove test-only method
adamdebreceni commented on code in PR #1567: URL: https://github.com/apache/nifi-minifi-cpp/pull/1567#discussion_r1193818426 ## extensions/standard-processors/tests/unit/YamlConfigurationTests.cpp: ## Review Comment: #1566 touches the same file, changed them here as well to see if everything else is ok, but will have to be careful on merge -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a diff in pull request #1567: MINIFICPP-2111 - Remove test-only method
adamdebreceni commented on code in PR #1567:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1567#discussion_r1193811911
##
extensions/standard-processors/tests/unit/YamlConfigurationTests.cpp:
##
@@ -456,7 +452,7 @@ Remote Process Groups:
NiFi Properties Overrides: {}
)";
std::istringstream configYamlStream(TEST_CONFIG_YAML);
Review Comment:
removed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-11453) Update ignite-core to 2.14.0
[ https://issues.apache.org/jira/browse/NIFI-11453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17722769#comment-17722769 ] David Handermann commented on NIFI-11453: - [~shreeju] Upgrading the Ignite libraries will require some code changes, together with an understanding of the details of the Ignite client library. If you are familiar with these details, this would be a helpful improvement. > Update ignite-core to 2.14.0 > - > > Key: NIFI-11453 > URL: https://issues.apache.org/jira/browse/NIFI-11453 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.21.0 >Reporter: Mike R >Priority: Major > > Update ignite-core to 2.14.0 from 1.6.0. The dependency has CVEs in 1.6.0 > resolved in 2.14.0: > [CVE-2020-1963|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-1963] > [CVE-2018-8018|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-8018] > [CVE-2018-1295|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-1295] > [CVE-2017-7686|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-7686] > [CVE-2016-6805|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-6805] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] szaszm commented on pull request #1566: MINIFICPP-2109 - Add extra logs to configuration parsing
szaszm commented on PR #1566:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1566#issuecomment-1547779890
> There are some clang-tidy warning you should check out, otherwise LGTM +1
```
/home/runner/work/nifi-minifi-cpp/nifi-minifi-cpp/extensions/standard-processors/tests/unit/YamlConfigurationTests.cpp:164:15:
error: loop variable is copied but only used as const reference; consider
making it a const reference [performance-for-range-copy,-warnings-as-errors]
for (auto it : connectionMap) {
^
const &
/home/runner/work/nifi-minifi-cpp/nifi-minifi-cpp/extensions/standard-processors/tests/unit/YamlConfigurationTests.cpp:479:13:
error: loop variable is copied but only used as const reference; consider
making it a const reference [performance-for-range-copy,-warnings-as-errors]
for (auto it : connectionMap) {
^
const &
/home/runner/work/nifi-minifi-cpp/nifi-minifi-cpp/extensions/standard-processors/tests/unit/YamlConfigurationTests.cpp:770:13:
error: loop variable is copied but only used as const reference; consider
making it a const reference [performance-for-range-copy,-warnings-as-errors]
for (auto it : connectionMap) {
^
const &
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1568: MINIFICPP-2112 Fix flow update and restart with minifi controller
szaszm commented on code in PR #1568:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1568#discussion_r1193765515
##
libminifi/include/c2/ControllerSocketProtocol.h:
##
@@ -61,6 +66,46 @@ class ControllerSocketProtocol {
std::weak_ptr controller_socket_reporter_;
std::shared_ptr configuration_;
std::shared_ptr logger_ =
core::logging::LoggerFactory::getLogger();
+ std::mutex initialization_mutex_;
+
+ // Some commands need to restart the controller socket to reinitialize the
socket with new data for example new SSL data in case of a flow update
+ // These commands are handled on a separate thread, and while these commands
are handled other incoming commands are dropped
+ class SocketRestartCommandProcessor {
+ public:
+explicit SocketRestartCommandProcessor(state::StateMonitor& update_sink_);
+~SocketRestartCommandProcessor();
+
+enum class Command {
+ FLOW_UPDATE,
+ START
+};
+
+struct CommandData {
+ Command command;
+ std::string data;
+};
+
+void enqueue(const CommandData& command_data) {
+ is_socket_restarting_ = true;
+ command_queue_.enqueue(command_data);
+ command_queue_condition_variable_.notify_all();
+}
+
+bool isSocketRestarting() const {
+ return is_socket_restarting_;
+}
+
+ private:
+mutable std::atomic_bool is_socket_restarting_ = false;
+state::StateMonitor& update_sink_;
+std::thread command_processor_thread_;
+std::mutex cv_mutex_;
+std::condition_variable command_queue_condition_variable_;
+std::atomic_bool running_ = true;
+moodycamel::ConcurrentQueue command_queue_;
Review Comment:
The minifi `utils::ConditionConcurrentQueue` could be a better alternative
here. It's not as performant in high load scenarios, but performance doesn't
really matter here, and it's less likely to change the order of commands, and
it has builtin waiting dequeue functions.
`moodycamel::ConcurrentQueue` works with per-thread arrays, and the consumer
always picks the first non-empty thread array. This means that it doesn't keep
any ordering between different sources, and it may return elements that were
inserted later if they happen to be in the first bucket/array. Strictly
speaking, this still gives happens-before guarantees within a producer thread,
but nothing between threads. If the second thread inserted something much
earlier, it will still return the item from the first thread if there are any.
We used to have strange priority and CPU starving issues when we used to use
this queue for scheduling processors. It's unlikely to cause any issues here,
other than occasional command reordering between multiple independent clients.
https://github.com/cameron314/concurrentqueue#high-level-design
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (NIFI-11546) MiNiFi IT Improvements
Ferenc Kis created NIFI-11546:
-
Summary: MiNiFi IT Improvements
Key: NIFI-11546
URL: https://issues.apache.org/jira/browse/NIFI-11546
Project: Apache NiFi
Issue Type: Improvement
Components: MiNiFi
Reporter: Ferenc Kis
1. We should add
{code:java}
mvn -pl minifi/minifi-c2/minifi-c2-assembly -am install -T1C -DskipTests -P
targz
mvn -pl minifi/minifi-c2/minifi-c2-docker -am install -T1C -DskipTests -P docker
{code}
to nifi/minifi/minifi-integration-tests/README.md to make it self contained, as
minifi-integration-tests require minifi-c2-docker image.
2. StandaloneYamlTest.verifyLogEntries test seems to be unstable as sometimes
the particular container is started too slowly, and in the verifyLogEntries
method the while loop exits before all the required log entries are found.
Possible solutions:
* quick and dirty: insert a Thread.sleep before verifyLogEntries, so the
docker container will have time to start
* proper: refactor the verifyLogEntries method, to not check on null, but poll
until the test criteria is met or a timeout is reached
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2116) Fix GCC 13 build
[
https://issues.apache.org/jira/browse/MINIFICPP-2116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Szasz updated MINIFICPP-2116:
Description:
1. update RocksDB to 8.0.0+ (latest: 8.1.1)
https://github.com/facebook/rocksdb/pull/8
2. Fix libminifi errors ( Annotation.h lacks )
3. update abseil:
https://github.com/abseil/abseil-cpp/commit/b957f0ccd00481cd4fd663d8320aa02ae0564f18
4. update opencv to 4.7.0:
https://github.com/opencv/opencv/commit/03130548eaa89e5491f669bf96af324890a26374
https://github.com/opencv/ade/commit/7cecc9138b89e1946e3e515727bb69b2ab119806
1.
{noformat}
In file included from
/home/szaszm/nifi-minifi-cpp/build/thirdparty/rocksdb-src/table/block_based/data_block_hash_index.cc:9:
/home/szaszm/nifi-minifi-cpp/build/thirdparty/rocksdb-src/table/block_based/data_block_hash_index.h:65:7:
error: ‘uint8_t’ does not name a type
65 | const uint8_t kNoEntry = 255;
| ^~~
/home/szaszm/nifi-minifi-cpp/build/thirdparty/rocksdb-src/table/block_based/data_block_hash_index.h:1:1:
note: ‘uint8_t’ is defined in header ‘’; did you forget to ‘#include
’?
+++ |+#include
1 | // Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
{noformat}
2.
{noformat}
In file included from
/home/szaszm/nifi-minifi-cpp/libminifi/include/agent/agent_docs.h:24,
from
/home/szaszm/nifi-minifi-cpp/libminifi/src/agent/agent_docs.cpp:18:
/home/szaszm/nifi-minifi-cpp/libminifi/include/core/Annotation.h:30:6: warning:
elaborated-type-specifier for a scoped enum must not use the ‘class’ keyword
30 | enum class Input : uint8_t {
| ^
| -
/home/szaszm/nifi-minifi-cpp/libminifi/include/core/Annotation.h:30:18: error:
found ‘:’ in nested-name-specifier, expected ‘::’
30 | enum class Input : uint8_t {
| ^
| ::
{noformat}
3.
{noformat}
In file included from
/home/szaszm/nifi-minifi-cpp/build/_deps/absl-src/absl/strings/internal/str_format/extension.cc:16:
/home/szaszm/nifi-minifi-cpp/build/_deps/absl-src/absl/strings/internal/str_format/extension.h:34:6:
warning: elaborated-type-specifier for a scoped enum must not use the ‘class’
keyword
34 | enum class FormatConversionChar : uint8_t;
| ^
| -
/home/szaszm/nifi-minifi-cpp/build/_deps/absl-src/absl/strings/internal/str_format/extension.h:34:33:
error: found ‘:’ in nested-name-specifier, expected ‘::’
34 | enum class FormatConversionChar : uint8_t;
| ^
| ::
/home/szaszm/nifi-minifi-cpp/build/_deps/absl-src/absl/strings/internal/str_format/extension.h:34:12:
error: ‘FormatConversionChar’ has not been declared
34 | enum class FormatConversionChar : uint8_t;
|^~~~
/home/szaszm/nifi-minifi-cpp/build/_deps/absl-src/absl/strings/internal/str_format/extension.h:35:6:
warning: elaborated-type-specifier for a scoped enum must not use the ‘class’
keyword
35 | enum class FormatConversionCharSet : uint64_t;
| ^
| -
{noformat}
4.
{noformat}
In file included from
/home/szaszm/nifi-minifi-cpp/build/extensions/opencv/opencv-external-prefix/src/opencv-external-build/3rdparty/ade/ade-0.1.1f/sources/ade/include/ade/passes/topological_sort.hpp:17,
from
/home/szaszm/nifi-minifi-cpp/build/extensions/opencv/opencv-external-prefix/src/opencv-external-build/3rdparty/ade/ade-0.1.1f/sources/ade/source/topological_sort.cpp:7:
/home/szaszm/nifi-minifi-cpp/build/extensions/opencv/opencv-external-prefix/src/opencv-external-build/3rdparty/ade/ade-0.1.1f/sources/ade/include/ade/typed_graph.hpp:101:10:
error: ‘uintptr_t’ in namespace ‘std’ does not name a type
101 | std::uintptr_t m_srcGraph;
| ^
/home/szaszm/nifi-minifi-cpp/build/extensions/opencv/opencv-external-prefix/src/opencv-external-build/3rdparty/ade/ade-0.1.1f/sources/ade/include/ade/typed_graph.hpp:22:1:
note: ‘std::uintptr_t’ is defined in header ‘’; did you forget to
‘#include ’?
21 | #include "typed_metadata.hpp"
+++ |+#include
22 |
/home/szaszm/nifi-minifi-cpp/build/extensions/opencv/opencv-external-prefix/src/opencv-external-build/3rdparty/ade/ade-0.1.1f/sources/ade/include/ade/typed_graph.hpp:
In member function ‘const ade::Graph& ade::ConstTypedGraph::getCGraph()
const’:
/home/szaszm/nifi-minifi-cpp/build/extensions/opencv/opencv-external-prefix/src/opencv-external-build/3rdparty/ade/ade-0.1.1f/sources/ade/include/ade/typed_graph.hpp:106:53:
error: ‘m_srcGraph’ was not declared in this scope
106 | return *reinterpret_cast(m_srcGraph);
| ^~
{noformat}
workaround:
{code:sh}
export CXXFLAGS="-include cstdint"
{code}
was:
1. update RocksDB to 8.0.0+ (latest: 8.1.1)
[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1532: MINIFICPP-2076 Implement logging metrics publisher
szaszm commented on code in PR #1532:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1532#discussion_r1193618193
##
libminifi/src/core/state/MetricsPublisherStore.cpp:
##
@@ -42,9 +42,9 @@ void
MetricsPublisherStore::initialize(core::controller::ControllerServiceProvid
addMetricsPublisher(c2::CONTROLLER_SOCKET_METRICS_PUBLISHER,
std::move(controller_socket_metrics_publisher));
}
- std::shared_ptr metrics_publisher =
minifi::state::createMetricsPublisher(configuration_, response_node_loader_);
- if (metrics_publisher) {
-addMetricsPublisher(minifi::Configure::nifi_metrics_publisher_class,
std::move(metrics_publisher));
+ for (auto&& publisher :
minifi::state::createMetricsPublishers(configuration_, response_node_loader_)) {
+auto name = publisher->getName();
+addMetricsPublisher(name, std::move(publisher));
Review Comment:
Consider pass-by-value and move for the name.
Can we change `addMetricsPublisher` to take a `unique_ptr` instead of a
`shared_ptr`? The store should probably not share ownership of the metrics
publishers.
##
libminifi/include/core/state/Value.h:
##
@@ -558,13 +558,14 @@ struct SerializedResponseNode {
return value.empty() && children.empty();
}
+ template>
Review Comment:
This won't work with the definition still in the .cpp file.
##
libminifi/test/unit/LogMetricsPublisherTests.cpp:
##
@@ -0,0 +1,209 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include
+#include
+
+#include "../TestBase.h"
+#include "../Catch.h"
+#include "core/state/LogMetricsPublisher.h"
+#include "core/state/nodes/ResponseNodeLoader.h"
+#include "core/RepositoryFactory.h"
+#include "utils/IntegrationTestUtils.h"
+
+using namespace std::literals::chrono_literals;
+
+namespace org::apache::nifi::minifi::test {
+
+class LogPublisherTestFixture {
+ public:
+ LogPublisherTestFixture()
+: configuration_(std::make_shared()),
+ provenance_repo_(core::createRepository("provenancerepository",
"provenancerepository")),
+ flow_file_repo_(core::createRepository("flowfilerepository",
"flowfilerepository")),
+
response_node_loader_(std::make_shared(configuration_,
provenance_repo_, flow_file_repo_, nullptr)),
+ publisher_("LogMetricsPublisher") {
+ }
+
+ protected:
+ std::shared_ptr configuration_;
+ std::shared_ptr provenance_repo_;
+ std::shared_ptr flow_file_repo_;
+ std::shared_ptr response_node_loader_;
+ minifi::state::LogMetricsPublisher publisher_;
+};
+
+TEST_CASE_METHOD(LogPublisherTestFixture, "Logging interval property is
mandatory", "[LogMetricsPublisher]") {
+
LogTestController::getInstance().setTrace();
+ SECTION("No logging interval is set") {
+REQUIRE_THROWS_WITH(publisher_.initialize(configuration_,
response_node_loader_), "General Operation: Metrics logging interval not
configured for log metrics publisher!");
+ }
+ SECTION("Logging interval is set to 2 seconds") {
+
configuration_->set(minifi::Configuration::nifi_metrics_publisher_log_metrics_logging_interval,
"2s");
+using org::apache::nifi::minifi::utils::verifyLogLinePresenceInPollTime;
+publisher_.initialize(configuration_, response_node_loader_);
+REQUIRE(verifyLogLinePresenceInPollTime(5s, "Metric logging interval is
set to 2000 milliseconds"));
+ }
+}
+
+TEST_CASE_METHOD(LogPublisherTestFixture, "Verify empty metrics if no valid
metrics are defined", "[LogMetricsPublisher]") {
+
LogTestController::getInstance().setTrace();
+
configuration_->set(minifi::Configuration::nifi_metrics_publisher_log_metrics_logging_interval,
"100ms");
+ SECTION("No metrics are defined") {}
+ SECTION("Only invalid metrics are defined") {
+configuration_->set(Configure::nifi_metrics_publisher_metrics,
"InvalidMetric,NotValidMetricNode");
+ }
+ publisher_.initialize(configuration_, response_node_loader_);
+ publisher_.loadMetricNodes();
+ using org::apache::nifi::minifi::utils::verifyLogLinePresenceInPollTime;
+ std::string expected_log = R"([info] {
+"LogMetrics": {}
+})";
+ REQUIRE(verifyLogLinePresenceInPollTime(5s, expected_log));
Review Comment:
If there is nothing useful to log, then we shouldn't
[jira] [Commented] (NIFI-10514) Add Instance ID to labels in PrometheusRecordSink
[ https://issues.apache.org/jira/browse/NIFI-10514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17722702#comment-17722702 ] Jeyassri Balachandran commented on NIFI-10514: -- Hi [~mattyb149] , I would like to work on it. May I take it up and raise PR? > Add Instance ID to labels in PrometheusRecordSink > - > > Key: NIFI-10514 > URL: https://issues.apache.org/jira/browse/NIFI-10514 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Burgess >Priority: Major > Labels: easyfix, newbie > > The PrometheusRecordSink offers an Instance ID property but does not make use > of it. PrometheusReportingTask uses the Instance ID as a label for data > points. PrometheusRecordSink should be updated to add Instance ID as a label -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] mark-bathori commented on pull request #7241: NIFI-11544 Fixed REL_FAILURE usage in AbstractIcebergProcessor
mark-bathori commented on PR #7241: URL: https://github.com/apache/nifi/pull/7241#issuecomment-1547393769 I'm reviewing this PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (MINIFICPP-2020) Fix the code which is supposed to stop two MiNiFi instances from interfering with each other
[ https://issues.apache.org/jira/browse/MINIFICPP-2020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Debreceni reassigned MINIFICPP-2020: - Assignee: Adam Debreceni > Fix the code which is supposed to stop two MiNiFi instances from interfering > with each other > > > Key: MINIFICPP-2020 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2020 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Ferenc Gerlits >Assignee: Adam Debreceni >Priority: Minor > Time Spent: 2h > Remaining Estimate: 0h > > On Windows, > * (x) if MiNiFi is already running as a service and you try to start another > instance from the console, then the second instance will exit with > %errorlevel% == -1, but without any message printed to either the console or > the log file, because GetTerminationEventHandle returns a null pointer; > * (/) if MiNiFi is not running as a service, you can start up two instances > with different MINIFI_HOMES from the console; > * (?) if you start up a MiNiFi instance from the console, then you start the > MiNiFi service, that works (assuming their MINIFI_HOMEs are different). > On Linux, > * (/) you can start up two instances as long as their MINIFI_HOMEs are > different. > On both, > * (?) if you try to start up two MiNiFi instances with the same MINIFI_HOME, > it will fail, as long as some repositories are RocksDB, because RocksDB can't > open the repo if another process has it open already. > > In theory, the process semaphore should stop the last case before trying to > open the DBs, but it doesn't, because we open the semaphore in non-exclusive > mode. Opening the process semaphore in exclusive mode (with no other > changes) would be a bad solution, because in the case of a crash or a kill > signal, the stale semaphore would stop the user from starting a new MiNiFi > instance and would have to be removed manually. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11453) Update ignite-core to 2.14.0
[ https://issues.apache.org/jira/browse/NIFI-11453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17722658#comment-17722658 ] Jeyassri Balachandran commented on NIFI-11453: -- Hi [~msr1716] , I would like to work on it. May I take it up and raise PR? > Update ignite-core to 2.14.0 > - > > Key: NIFI-11453 > URL: https://issues.apache.org/jira/browse/NIFI-11453 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.21.0 >Reporter: Mike R >Priority: Major > > Update ignite-core to 2.14.0 from 1.6.0. The dependency has CVEs in 1.6.0 > resolved in 2.14.0: > [CVE-2020-1963|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-1963] > [CVE-2018-8018|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-8018] > [CVE-2018-1295|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-1295] > [CVE-2017-7686|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-7686] > [CVE-2016-6805|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-6805] -- This message was sent by Atlassian Jira (v8.20.10#820010)
