[GitHub] [nifi] levilentz commented on pull request #7677: NIFI-7355: add bytecode submission to tinkerpop service
levilentz commented on PR #7677: URL: https://github.com/apache/nifi/pull/7677#issuecomment-1738364154 @joewitt @MikeThomsen @mattyb149 I have updated the code with your suggested changes, and they are working great! Let me know if you have any other changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] levilentz commented on a diff in pull request #7677: NIFI-7355: add bytecode submission to tinkerpop service
levilentz commented on code in PR #7677:
URL: https://github.com/apache/nifi/pull/7677#discussion_r1339455563
##
nifi-nar-bundles/nifi-graph-bundle/nifi-other-graph-services/src/main/java/org/apache/nifi/graph/TinkerpopClientService.java:
##
@@ -0,0 +1,525 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.graph;
+
+import groovy.lang.Binding;
+import groovy.lang.GroovyClassLoader;
+import groovy.lang.GroovyShell;
+import groovy.lang.Script;
+import io.netty.handler.ssl.ClientAuth;
+import io.netty.handler.ssl.JdkSslContext;
+import org.apache.commons.codec.digest.DigestUtils;
+import org.apache.nifi.annotation.behavior.RequiresInstanceClassLoading;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnDisabled;
+import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.components.resource.ResourceCardinality;
+import org.apache.nifi.components.resource.ResourceType;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.graph.gremlin.SimpleEntry;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.graph.ExecuteGraphQueryRecord;
+import org.apache.nifi.reporting.InitializationException;
+import org.apache.nifi.ssl.SSLContextService;
+import org.apache.nifi.util.StringUtils;
+import org.apache.nifi.util.file.classloader.ClassLoaderUtils;
+import org.apache.tinkerpop.gremlin.driver.Client;
+import org.apache.tinkerpop.gremlin.driver.Cluster;
+import org.apache.tinkerpop.gremlin.driver.Result;
+import org.apache.tinkerpop.gremlin.driver.remote.DriverRemoteConnection;
+import org.apache.tinkerpop.gremlin.process.traversal.AnonymousTraversalSource;
+import
org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+
+@Tags({"graph", "gremlin"})
+@CapabilityDescription("This service interacts with a tinkerpop-compliant
graph service, providing both script submission and bytecode submission
capabilities. " +
+"Script submission is the default, with the script command being sent
to the gremlin server as text. This should only be used for simple interactions
with a tinkerpop-compliant server " +
+"such as counts or other operations that do not require the injection
of custom classed. " +
+"Bytecode submission allows much more flexibility. When providing a
jar, custom serializers can be used and pre-compiled graph logic can be
utilized by groovy scripts" +
+"provided by processors such as the ExecuteGraphQueryRecord.")
+@SeeAlso({ExecuteGraphQueryRecord.class})
+@RequiresInstanceClassLoading
+public class TinkerpopClientService extends AbstractControllerService
implements GraphClientService {
+public static final String NOT_SUPPORTED = "NOT_SUPPORTED";
+private static final AllowableValue BYTECODE_SUBMISSION = new
AllowableValue("bytecode-submission", "ByteCode Submission",
+"Groovy scripts are compiled within NiFi, with the
GraphTraversalSource injected as a variable 'g'. Effectively allowing " +
+"your logic to directly manipulates the graph without
string serialization overheard."
+);
+
+private static final AllowableValue SCRIPT_SUBMISSION = new
AllowableValue("script-submission", "Script Submission",
+
[GitHub] [nifi] levilentz commented on a diff in pull request #7677: NIFI-7355: add bytecode submission to tinkerpop service
levilentz commented on code in PR #7677:
URL: https://github.com/apache/nifi/pull/7677#discussion_r1339455296
##
nifi-nar-bundles/nifi-graph-bundle/nifi-other-graph-services/src/main/java/org/apache/nifi/graph/TinkerpopClientService.java:
##
@@ -0,0 +1,525 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.graph;
+
+import groovy.lang.Binding;
+import groovy.lang.GroovyClassLoader;
+import groovy.lang.GroovyShell;
+import groovy.lang.Script;
+import io.netty.handler.ssl.ClientAuth;
+import io.netty.handler.ssl.JdkSslContext;
+import org.apache.commons.codec.digest.DigestUtils;
+import org.apache.nifi.annotation.behavior.RequiresInstanceClassLoading;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnDisabled;
+import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.components.resource.ResourceCardinality;
+import org.apache.nifi.components.resource.ResourceType;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.graph.gremlin.SimpleEntry;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.graph.ExecuteGraphQueryRecord;
+import org.apache.nifi.reporting.InitializationException;
+import org.apache.nifi.ssl.SSLContextService;
+import org.apache.nifi.util.StringUtils;
+import org.apache.nifi.util.file.classloader.ClassLoaderUtils;
+import org.apache.tinkerpop.gremlin.driver.Client;
+import org.apache.tinkerpop.gremlin.driver.Cluster;
+import org.apache.tinkerpop.gremlin.driver.Result;
+import org.apache.tinkerpop.gremlin.driver.remote.DriverRemoteConnection;
+import org.apache.tinkerpop.gremlin.process.traversal.AnonymousTraversalSource;
+import
org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+
+@Tags({"graph", "gremlin"})
+@CapabilityDescription("This service interacts with a tinkerpop-compliant
graph service, providing both script submission and bytecode submission
capabilities. " +
+"Script submission is the default, with the script command being sent
to the gremlin server as text. This should only be used for simple interactions
with a tinkerpop-compliant server " +
+"such as counts or other operations that do not require the injection
of custom classed. " +
+"Bytecode submission allows much more flexibility. When providing a
jar, custom serializers can be used and pre-compiled graph logic can be
utilized by groovy scripts" +
+"provided by processors such as the ExecuteGraphQueryRecord.")
+@SeeAlso({ExecuteGraphQueryRecord.class})
+@RequiresInstanceClassLoading
+public class TinkerpopClientService extends AbstractControllerService
implements GraphClientService {
+public static final String NOT_SUPPORTED = "NOT_SUPPORTED";
+private static final AllowableValue BYTECODE_SUBMISSION = new
AllowableValue("bytecode-submission", "ByteCode Submission",
+"Groovy scripts are compiled within NiFi, with the
GraphTraversalSource injected as a variable 'g'. Effectively allowing " +
+"your logic to directly manipulates the graph without
string serialization overheard."
+);
+
+private static final AllowableValue SCRIPT_SUBMISSION = new
AllowableValue("script-submission", "Script Submission",
+
[jira] [Comment Edited] (NIFI-12100) Remove ConvertExcelToCSVProcessor
[ https://issues.apache.org/jira/browse/NIFI-12100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769831#comment-17769831 ] Daniel Stieglitz edited comment on NIFI-12100 at 9/28/23 12:42 AM: --- [~exceptionfactory] I think I figured it out. I had to name the branch something other than NIFI-12100 as that was used for the branch I created to merge into main. Also, I could not build with Java 21 but I was able to build with Java 17. The PR is [7803|https://github.com/apache/nifi/pull/7803] For future knowledge what branch do I actually push to? I had done origin which I realize is not correct and then on github I had to switch to which branch I wanted to merge to. was (Author: JIRAUSER294662): [~exceptionfactory] I think I figured it out. I had to name the branch something other than NIFI-12100 as that was used for the branch I created to merge into main. Also, I could not build with Java 21 but I was able to build with Java 17. The PR is [7803|https://github.com/apache/nifi/pull/7803] > Remove ConvertExcelToCSVProcessor > --- > > Key: NIFI-12100 > URL: https://issues.apache.org/jira/browse/NIFI-12100 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > ConvertExcelToCSVProcessor is no longer needed since there is now the > ExcelReader which along with CSVRecordSetWriter can be used in ConvertRecord > to achieve the same thing. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-12135) Update 2 missing OIDC properties in update_oidc_properties.sh
[ https://issues.apache.org/jira/browse/NIFI-12135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt resolved NIFI-12135. - Fix Version/s: 2.0.0 Resolution: Fixed > Update 2 missing OIDC properties in update_oidc_properties.sh > - > > Key: NIFI-12135 > URL: https://issues.apache.org/jira/browse/NIFI-12135 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Docker >Affects Versions: 1.23.1, 1.23.2 >Reporter: Marcelo >Priority: Minor > Labels: pull-request-available > Fix For: 2.0.0 > > Time Spent: 10m > Remaining Estimate: 0h > > According this Nifi Documentation > ([https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#openid_connect)] > I've created a PR to update 2 missing OIDC properties in > *update_oidc_properties.sh* file: > - nifi.security.user.oidc.claim.groups > - nifi.security.user.oidc.token.refresh.window -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12135) Update 2 missing OIDC properties in update_oidc_properties.sh
[ https://issues.apache.org/jira/browse/NIFI-12135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769851#comment-17769851 ] ASF subversion and git services commented on NIFI-12135: Commit acd9b5b10b67096b8981e49299a165ff1afea0ff in nifi's branch refs/heads/main from Marcelo Vinícius de Sousa Campos [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=acd9b5b10b ] NIFI-12135 This closes #7799. Update update_oidc_properties.sh Update OIDC properties via update_oidc_properties.sh Signed-off-by: Joseph Witt > Update 2 missing OIDC properties in update_oidc_properties.sh > - > > Key: NIFI-12135 > URL: https://issues.apache.org/jira/browse/NIFI-12135 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Docker >Affects Versions: 1.23.1, 1.23.2 >Reporter: Marcelo >Priority: Minor > Labels: pull-request-available > > According this Nifi Documentation > ([https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#openid_connect)] > I've created a PR to update 2 missing OIDC properties in > *update_oidc_properties.sh* file: > - nifi.security.user.oidc.claim.groups > - nifi.security.user.oidc.token.refresh.window -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] asfgit closed pull request #7799: NIFI-12135 Update 2 missing OIDC properties in update_oidc_properties.sh
asfgit closed pull request #7799: NIFI-12135 Update 2 missing OIDC properties in update_oidc_properties.sh URL: https://github.com/apache/nifi/pull/7799 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] joewitt opened a new pull request, #7804: NIFI-12138 added volume for nar extensions
joewitt opened a new pull request, #7804: URL: https://github.com/apache/nifi/pull/7804 # Summary [NIFI-0](https://issues.apache.org/jira/browse/NIFI-0) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (NIFI-12100) Remove ConvertExcelToCSVProcessor
[ https://issues.apache.org/jira/browse/NIFI-12100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769831#comment-17769831 ] Daniel Stieglitz edited comment on NIFI-12100 at 9/27/23 10:20 PM: --- [~exceptionfactory] I think I figured it out. I had to name the branch something other than NIFI-12100 as that was used for the branch I created to merge into main. Also, I could not build with Java 21 but I was able to build with Java 17. The PR is [7803|https://github.com/apache/nifi/pull/7803] was (Author: JIRAUSER294662): [~exceptionfactory] I think I figured it out. I had to name the branch something other than NIFI-12100 as that was used for the branch I created to merge into main and I could not build with Java 21 but I was able to build with Java 17. The PR is [7803|https://github.com/apache/nifi/pull/7803] > Remove ConvertExcelToCSVProcessor > --- > > Key: NIFI-12100 > URL: https://issues.apache.org/jira/browse/NIFI-12100 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > ConvertExcelToCSVProcessor is no longer needed since there is now the > ExcelReader which along with CSVRecordSetWriter can be used in ConvertRecord > to achieve the same thing. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (NIFI-12100) Remove ConvertExcelToCSVProcessor
[ https://issues.apache.org/jira/browse/NIFI-12100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769831#comment-17769831 ] Daniel Stieglitz edited comment on NIFI-12100 at 9/27/23 10:10 PM: --- [~exceptionfactory] I think I figured it out. I had to name the branch something other than NIFI-12100 as that was used for the branch I created to merge into main and I could not build with Java 21 but I was able to build with Java 17. The PR is [7803|https://github.com/apache/nifi/pull/7803] was (Author: JIRAUSER294662): [~exceptionfactory] I think I figured it out. I had to name the branch something other than NIFI-12100 as that was used for the branch I created to merge into main and I could not build with Java 21 but I was able to build with Java 17. > Remove ConvertExcelToCSVProcessor > --- > > Key: NIFI-12100 > URL: https://issues.apache.org/jira/browse/NIFI-12100 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > ConvertExcelToCSVProcessor is no longer needed since there is now the > ExcelReader which along with CSVRecordSetWriter can be used in ConvertRecord > to achieve the same thing. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (NIFI-12100) Remove ConvertExcelToCSVProcessor
[ https://issues.apache.org/jira/browse/NIFI-12100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769831#comment-17769831 ] Daniel Stieglitz edited comment on NIFI-12100 at 9/27/23 10:08 PM: --- [~exceptionfactory] I think I figured it out. I had to name the branch something other than NIFI-12100 as that was used for the branch I created to merge into main and I could not build with Java 21 but I was able to build with Java 17. was (Author: JIRAUSER294662): [~exceptionfactory] I think I figured it out. I had to name the branch something other than NIFI-12100 as that was used for the branch I created to merge into main. > Remove ConvertExcelToCSVProcessor > --- > > Key: NIFI-12100 > URL: https://issues.apache.org/jira/browse/NIFI-12100 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > ConvertExcelToCSVProcessor is no longer needed since there is now the > ExcelReader which along with CSVRecordSetWriter can be used in ConvertRecord > to achieve the same thing. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12100) Remove ConvertExcelToCSVProcessor
[ https://issues.apache.org/jira/browse/NIFI-12100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769831#comment-17769831 ] Daniel Stieglitz commented on NIFI-12100: - [~exceptionfactory] I think I figured it out. I had to name the branch something other than NIFI-12100 as that was used for the branch I created to merge into main. > Remove ConvertExcelToCSVProcessor > --- > > Key: NIFI-12100 > URL: https://issues.apache.org/jira/browse/NIFI-12100 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > ConvertExcelToCSVProcessor is no longer needed since there is now the > ExcelReader which along with CSVRecordSetWriter can be used in ConvertRecord > to achieve the same thing. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] dan-s1 opened a new pull request, #7803: Nifi 12100 deprecate
dan-s1 opened a new pull request, #7803: URL: https://github.com/apache/nifi/pull/7803 # Summary [NIFI-12100](https://issues.apache.org/jira/browse/NIFI-12100) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] bbende commented on a diff in pull request #7796: NIFI-12104 Separate a non-atomic Redis DMC implementation from the ex…
bbende commented on code in PR #7796:
URL: https://github.com/apache/nifi/pull/7796#discussion_r1339239244
##
nifi-nar-bundles/nifi-redis-bundle/nifi-redis-extensions/src/main/java/org/apache/nifi/redis/service/RedisDistributedMapCacheClientService.java:
##
@@ -332,43 +127,4 @@ public boolean replace(final AtomicCacheEntry entry, final
});
}
-// - END Methods from AtomicDistributedMapCacheClient
-
-private Tuple serialize(final K key, final V value,
final Serializer keySerializer, final Serializer valueSerializer) throws
IOException {
-final ByteArrayOutputStream out = new ByteArrayOutputStream();
-
-keySerializer.serialize(key, out);
-final byte[] k = out.toByteArray();
-
-out.reset();
-
-valueSerializer.serialize(value, out);
-final byte[] v = out.toByteArray();
-
-return new Tuple<>(k, v);
-}
-
-private byte[] serialize(final K key, final Serializer
keySerializer) throws IOException {
-final ByteArrayOutputStream out = new ByteArrayOutputStream();
-
-keySerializer.serialize(key, out);
-return out.toByteArray();
-}
-
-private T withConnection(final RedisAction action) throws
IOException {
Review Comment:
The transactional code is in the methods from `AtomicDistributedMapCache`
that are implemented in `RedisDistributedMapCacheClientService`. Since we can't
rename the existing service (it would be better to name that one
`AtomicRedisDistributedMapCacheClientService`), we can leave that one alone and
move the methods that are from only `DistributedMapCacheClientService` down to
`SimpleRedisDistributedMapCacheClientService`. Then if a processor declares
that it only needs `DistributedMapCacheClient` (not atomic), then it can use
the Simple Redis one even when Redis is clustered.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] bbende commented on a diff in pull request #7796: NIFI-12104 Separate a non-atomic Redis DMC implementation from the ex…
bbende commented on code in PR #7796:
URL: https://github.com/apache/nifi/pull/7796#discussion_r1339239244
##
nifi-nar-bundles/nifi-redis-bundle/nifi-redis-extensions/src/main/java/org/apache/nifi/redis/service/RedisDistributedMapCacheClientService.java:
##
@@ -332,43 +127,4 @@ public boolean replace(final AtomicCacheEntry entry, final
});
}
-// - END Methods from AtomicDistributedMapCacheClient
-
-private Tuple serialize(final K key, final V value,
final Serializer keySerializer, final Serializer valueSerializer) throws
IOException {
-final ByteArrayOutputStream out = new ByteArrayOutputStream();
-
-keySerializer.serialize(key, out);
-final byte[] k = out.toByteArray();
-
-out.reset();
-
-valueSerializer.serialize(value, out);
-final byte[] v = out.toByteArray();
-
-return new Tuple<>(k, v);
-}
-
-private byte[] serialize(final K key, final Serializer
keySerializer) throws IOException {
-final ByteArrayOutputStream out = new ByteArrayOutputStream();
-
-keySerializer.serialize(key, out);
-return out.toByteArray();
-}
-
-private T withConnection(final RedisAction action) throws
IOException {
Review Comment:
The transactional code is in the methods from `AtomicDistributedMapCache`
that are implemented in `RedisDistributedMapCacheClientService`. Since we can't
rename the existing service (it would be better to name that one
`AtomicRedisDistributedMapCacheClientService`), we can leave that one alone and
move the methods that are from only `DistributedMapCacheClientService` down to
`SimpleRedisDistributedMapCacheClientService`. This if a processor declares
that it only needs `DistributedMapCacheClient` (not atomic), then it can use
the Simple Redis one even when Redis is clustered.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Created] (NIFI-12139) Allow for cleaner migration of extensions' configuration
Mark Payne created NIFI-12139:
-
Summary: Allow for cleaner migration of extensions' configuration
Key: NIFI-12139
URL: https://issues.apache.org/jira/browse/NIFI-12139
Project: Apache NiFi
Issue Type: Task
Components: Core Framework, Extensions
Reporter: Mark Payne
Assignee: Mark Payne
Over time, Processors, Controller Services, and Reporting Tasks need to evolve.
New capabilities become available in the library that it uses or the endpoint
it interacts with. The developer overlooked some configuration that is
important for some use cases, etc. Or we just have a typo or best practices for
naming conventions evolve.
Today, we have some ways to handle these scenarios:
* We can add a new property descriptor with a default value.
* We can add a displayName for Property Descriptors
But these mechanisms are lacking. Some of the problems that we have:
* If we add a new Property Descriptor, we generally have to set the default
value such that we don't change the behavior of existing components. This
sometimes means that we have to use a default value that's not really the best
default, just to maintain backward compatibility.
* We have to maintain both a 'name' and a 'displayName' for property
descriptors. This makes the UI and the API confusing. If the UI shows a
property is named 'My Property', setting the value of 'My Property' should
work. Instead, we have to set the value of 'my-property' because the that's the
property's 'name' - even though it's not made clear in the UI.
* If we add a new PropertyDescriptor that doesn't have a default value, all
existing instances are made invalid upon upgrade.
* If we want to add a new Relationship, unless it is auto-terminated, all
existing instances are made invalid upon upgrade - and making the new
relationship auto-terminated is rarely OK because it could result in data loss.
* There is no way to remove a Relationship.
* There is no way to remove a Property Descriptor. Once added, it must stay
there, even though it is ignored.
We need to introduce a new ability to migrate old configuration to a new
configuration. Something along the lines of:
{code:java}
public void migrateProperties(PropertyConfiguration existingConfig) {
}{code}
A default implementation would mean that nothing happens. But an implementation
might decide to implement this as such:
{code:java}
public void migrateProperties(PropertyConfiguration config) {
config.renameProperty("old-name", "New Name");
} {code}
Or, if a property is no longer necessary, instead of leaving it to be ignored,
we could simply use:
{code:java}
public void migrateProperties(PropertyConfiguration config) {
config.removeProperty("deprecated property name");
}{code}
This would mean we can actually eliminate the user of displayName and instead
just use clean, clear names for properties. This would lead to much less
confusion.
It gives us MUCH more freedom to evolve processors, as well. For example, let's
say that we have a Processor that processes a file and then deletes it from a
directory. We now decide that it should be configurable - and by default we
don't want to delete the file. But for existing processors, we don't want to
change their behavior. We can handle this by introducing the new Property
Descriptor with a default but having the migration ensure that we don't change
existing behavior:
{code:java}
static PropertyDescriptor DELETE_FILE_ON_COMPLETION = new
PropertyDescriptor.Builder()
.name("Delete File on Completion")
.description("Whether or not to delete the file after processing completes.")
.defaultValue("false")
.allowableValues("true", "false")
.build();
...
public void migrateProperties(PropertyConfiguration config) {
// Maintain existing behavior for processors that were created before
// the option was added to delete files or not.
if (!config.hasProperty(DELETE_FILE_ON_COMPLETION)) {
config.setProperty(DELETE_FILE_ON_COMPLETION, "true");
}
}{code}
Importantly, we also want the ability to handle evolution of Relationships:
{code:java}
public void migrateRelationships(RelationshipConfiguration config) {
} {code}
If we decide that we now want to rename the "comms.failure" relationship to
"Communications Failure" we can do so thusly:
{code:java}
public void migrateRelationships(RelationshipConfiguration config) {
config.renameRelationship("comms.failure", "Communications Failure");
} {code}
We also sometimes have a situation where we'd like to break a relationship into
two. For example, we have a "failure" relationship that we want to split into
"failure" and "comms failure". But introducing a new "comms failure"
relationship today would mean that existing processors become invalid. This
allows us to take care of this:
{code:java}
public void
[jira] [Created] (NIFI-12138) Make it easy to have a defined volume for nifi extensions in the nifi docker image
Joe Witt created NIFI-12138: --- Summary: Make it easy to have a defined volume for nifi extensions in the nifi docker image Key: NIFI-12138 URL: https://issues.apache.org/jira/browse/NIFI-12138 Project: Apache NiFi Issue Type: Task Reporter: Joe Witt Assignee: Joe Witt Allow users to define a volume so they can easily set nars in a specific point that will be picked up for their docker containers running NiFI -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12138) Make it easy to have a defined volume for nifi extensions in the nifi docker image
[ https://issues.apache.org/jira/browse/NIFI-12138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-12138: Fix Version/s: 2.latest > Make it easy to have a defined volume for nifi extensions in the nifi docker > image > -- > > Key: NIFI-12138 > URL: https://issues.apache.org/jira/browse/NIFI-12138 > Project: Apache NiFi > Issue Type: Task >Reporter: Joe Witt >Assignee: Joe Witt >Priority: Major > Fix For: 2.latest > > > Allow users to define a volume so they can easily set nars in a specific > point that will be picked up for their docker containers running NiFI -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] MikeThomsen commented on a diff in pull request #7796: NIFI-12104 Separate a non-atomic Redis DMC implementation from the ex…
MikeThomsen commented on code in PR #7796:
URL: https://github.com/apache/nifi/pull/7796#discussion_r1339203072
##
nifi-nar-bundles/nifi-redis-bundle/nifi-redis-extensions/src/main/java/org/apache/nifi/redis/service/RedisDistributedMapCacheClientService.java:
##
@@ -332,43 +127,4 @@ public boolean replace(final AtomicCacheEntry entry, final
});
}
-// - END Methods from AtomicDistributedMapCacheClient
-
-private Tuple serialize(final K key, final V value,
final Serializer keySerializer, final Serializer valueSerializer) throws
IOException {
-final ByteArrayOutputStream out = new ByteArrayOutputStream();
-
-keySerializer.serialize(key, out);
-final byte[] k = out.toByteArray();
-
-out.reset();
-
-valueSerializer.serialize(value, out);
-final byte[] v = out.toByteArray();
-
-return new Tuple<>(k, v);
-}
-
-private byte[] serialize(final K key, final Serializer
keySerializer) throws IOException {
-final ByteArrayOutputStream out = new ByteArrayOutputStream();
-
-keySerializer.serialize(key, out);
-return out.toByteArray();
-}
-
-private T withConnection(final RedisAction action) throws
IOException {
Review Comment:
The connection handling looks the same to me. Am I missing something? I am
not seeing where one use is transactional, but the other isn't.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Resolved] (NIFI-11917) Remove RPM build from NiFi POM/Assembly
[
https://issues.apache.org/jira/browse/NIFI-11917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt resolved NIFI-11917.
-
Fix Version/s: 2.0.0
(was: 2.latest)
Resolution: Fixed
> Remove RPM build from NiFi POM/Assembly
> ---
>
> Key: NIFI-11917
> URL: https://issues.apache.org/jira/browse/NIFI-11917
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.23.0
> Environment: Linux
>Reporter: Gregory M. Foreman
>Assignee: Joe Witt
>Priority: Major
> Fix For: 2.0.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Nifi 1.23.0 is unable to build from source when using the rpm profile setting.
> This command:
> {code:java}
> JAVA_OPTS="-Xms128m -Xmx4g"
> MAVEN_OPTS="-Dorg.slf4j.simpleLogger.defaultLogLevel=ERROR -Xms1024m
> -Xmx3076m -XX:MaxPermSize=256m"
> mvn -T C2.0 --batch-mode -Prpm -DskipTests clean install
> {code}
> produces the following error:
> {code:java}
> [ERROR] Failed to execute goal
> org.codehaus.mojo:rpm-maven-plugin:2.2.0:attached-rpm (build-bin-rpm) on
> project nifi-toolkit-assembly: Source location
> /root/test/nifi-1.23.0/nifi-toolkit/nifi-toolkit-assembly/target/nifi-toolkit-1.23.0-bin/nifi-toolkit-1.23.0/LICENSE
> does not exist -> [Help 1]{code}
> The path above ends at:
> /root/test2/nifi/nifi-toolkit/nifi-toolkit-assembly/target/
> There is a zip file within it:
> nifi-toolkit-1.23.0-bin.zip
> It seems like this unzip was missed.
> Environment info:
> {code:bash}
> mvn -version
> Apache Maven 3.8.4 (9b656c72d54e5bacbed989b64718c159fe39b537)
> Maven home: /opt/maven
> Java version: 1.8.0_372, vendor: Red Hat, Inc., runtime:
> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.372.b07-1.el7_9.x86_64/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "linux", version: "3.10.0-1160.95.1.el7.x86_64", arch: "amd64",
> family: "unix"{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (NIFI-12137) Improve Java Python processor behaviors in NiFi 2x
[ https://issues.apache.org/jira/browse/NIFI-12137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-12137: Fix Version/s: 2.latest > Improve Java Python processor behaviors in NiFi 2x > -- > > Key: NIFI-12137 > URL: https://issues.apache.org/jira/browse/NIFI-12137 > Project: Apache NiFi > Issue Type: Bug >Reporter: Joe Witt >Priority: Major > Fix For: 2.latest > > > In a flow using a FlowFileTransform I am seeing data when I route it to > faillure also going to original. If routed to failure it should not go to > original. Original should only be when success is the chosen route. > We should change ExpressionLanguageScope.LIMITED to ENVIRONMENT. > We should try to use python conventions on all method names instead of Java > conventions. > Can we define the 'logger' that gets injected to implementations so code > completion can work in IDEs in the meantime? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12137) Improve Java Python processor behaviors in NiFi 2x
[
https://issues.apache.org/jira/browse/NIFI-12137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt updated NIFI-12137:
Description:
In a flow using a FlowFileTransform I am seeing data when I route it to
faillure also going to original. If routed to failure it should not go to
original. Original should only be when success is the chosen route.
We should change ExpressionLanguageScope.LIMITED to ENVIRONMENT.
We should try to use python conventions on all method names instead of Java
conventions.
Can we define the 'logger' that gets injected to implementations so code
completion can work in IDEs in the meantime?
was:
Built a python processor. Deployed as version 0.0.1. Updated to 0.0.2
replacing original file. Updated to 0.0.3. Then started just keeping same
version and restarting processor. However, on restart I received this output
{noformat}
2023-09-26 13:00:10 2023-09-26 20:00:06,984 ERROR [NiFi Web Server-140]
o.a.n.n.StandardExtensionDiscoveringManager Could not instantiate class of type
python.LookupAddress using ClassLoader for bundle
org.apache.nifi:python-extensions:0.0.1-SNAPSHOT
2023-09-26 13:00:10 py4j.Py4JException: An exception was raised by the Python
Proxy. Return Message: Traceback (most recent call last):
2023-09-26 13:00:10 File
"/opt/nifi/nifi-current/python/framework/py4j/java_gateway.py", line 2466, in
_call_proxy
2023-09-26 13:00:10 return_value = getattr(self.pool[obj_id],
method)(*params)
2023-09-26 13:00:10 File
"/opt/nifi/nifi-current/./python/framework/Controller.py", line 52, in
createProcessor
2023-09-26 13:00:10 processorClass =
self.extensionManager.getProcessorClass(processorType, version, work_dir)
2023-09-26 13:00:10 File
"/opt/nifi/nifi-current/python/framework/ExtensionManager.py", line 138, in
getProcessorClass
2023-09-26 13:00:10 raise ValueError('Invalid Processor Type: No module is
known to contain Processor of type ' + type + ' version ' + version)
2023-09-26 13:00:10 ValueError: Invalid Processor Type: No module is known to
contain Processor of type LookupAddress version 0.0.1-SNAPSHOT
2023-09-26 13:00:10
2023-09-26 13:00:10 at py4j.Protocol.getReturnValue(Protocol.java:476)
2023-09-26 13:00:09 tail: '/opt/nifi/nifi-current/logs/nifi-app.log' has been
replaced; following new file
{noformat}
In a flow using a FlowFileTransform I am seeing data when I route it to
faillure also going to original. If routed to failure it should not go to
original. Original should only be when success is the chosen route. Will
replicate and add more info here.
We should change ExpressionLanguageScope.LIMITED to ENVIRONMENT.
We should try to use python conventions on all method names instead of Java
conventions.
Can we define the 'logger' that gets injected to implementations so code
completion can work in IDEs in the meantime?
> Improve Java Python processor behaviors in NiFi 2x
> --
>
> Key: NIFI-12137
> URL: https://issues.apache.org/jira/browse/NIFI-12137
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Joe Witt
>Priority: Major
>
> In a flow using a FlowFileTransform I am seeing data when I route it to
> faillure also going to original. If routed to failure it should not go to
> original. Original should only be when success is the chosen route.
> We should change ExpressionLanguageScope.LIMITED to ENVIRONMENT.
> We should try to use python conventions on all method names instead of Java
> conventions.
> Can we define the 'logger' that gets injected to implementations so code
> completion can work in IDEs in the meantime?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-12137) Improve Java Python processor behaviors in NiFi 2x
[ https://issues.apache.org/jira/browse/NIFI-12137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769771#comment-17769771 ] Joe Witt commented on NIFI-12137: - cc [~markap14] > Improve Java Python processor behaviors in NiFi 2x > -- > > Key: NIFI-12137 > URL: https://issues.apache.org/jira/browse/NIFI-12137 > Project: Apache NiFi > Issue Type: Bug >Reporter: Joe Witt >Priority: Major > Fix For: 2.latest > > > In a flow using a FlowFileTransform I am seeing data when I route it to > faillure also going to original. If routed to failure it should not go to > original. Original should only be when success is the chosen route. > We should change ExpressionLanguageScope.LIMITED to ENVIRONMENT. > We should try to use python conventions on all method names instead of Java > conventions. > Can we define the 'logger' that gets injected to implementations so code > completion can work in IDEs in the meantime? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11917) Remove RPM build from NiFi POM/Assembly
[
https://issues.apache.org/jira/browse/NIFI-11917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769762#comment-17769762
]
ASF subversion and git services commented on NIFI-11917:
Commit 249829af5f046790523753bced705ef29f0e8c83 in nifi's branch
refs/heads/main from Joe Witt
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=249829af5f ]
NIFI-11917 removing support for RPM building as we lack the
infrastructure/maintenance to keep it healthy and we dont publish the artifacts
(#7801)
Signed-off-by: Otto Fowler
> Remove RPM build from NiFi POM/Assembly
> ---
>
> Key: NIFI-11917
> URL: https://issues.apache.org/jira/browse/NIFI-11917
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.23.0
> Environment: Linux
>Reporter: Gregory M. Foreman
>Assignee: Joe Witt
>Priority: Major
> Fix For: 2.latest
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Nifi 1.23.0 is unable to build from source when using the rpm profile setting.
> This command:
> {code:java}
> JAVA_OPTS="-Xms128m -Xmx4g"
> MAVEN_OPTS="-Dorg.slf4j.simpleLogger.defaultLogLevel=ERROR -Xms1024m
> -Xmx3076m -XX:MaxPermSize=256m"
> mvn -T C2.0 --batch-mode -Prpm -DskipTests clean install
> {code}
> produces the following error:
> {code:java}
> [ERROR] Failed to execute goal
> org.codehaus.mojo:rpm-maven-plugin:2.2.0:attached-rpm (build-bin-rpm) on
> project nifi-toolkit-assembly: Source location
> /root/test/nifi-1.23.0/nifi-toolkit/nifi-toolkit-assembly/target/nifi-toolkit-1.23.0-bin/nifi-toolkit-1.23.0/LICENSE
> does not exist -> [Help 1]{code}
> The path above ends at:
> /root/test2/nifi/nifi-toolkit/nifi-toolkit-assembly/target/
> There is a zip file within it:
> nifi-toolkit-1.23.0-bin.zip
> It seems like this unzip was missed.
> Environment info:
> {code:bash}
> mvn -version
> Apache Maven 3.8.4 (9b656c72d54e5bacbed989b64718c159fe39b537)
> Maven home: /opt/maven
> Java version: 1.8.0_372, vendor: Red Hat, Inc., runtime:
> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.372.b07-1.el7_9.x86_64/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "linux", version: "3.10.0-1160.95.1.el7.x86_64", arch: "amd64",
> family: "unix"{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi] ottobackwards merged pull request #7801: NIFI-11917 removing support for RPM building as we lack the infrastru…
ottobackwards merged PR #7801: URL: https://github.com/apache/nifi/pull/7801 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-12137) Improve Java Python processor behaviors in NiFi 2x
[
https://issues.apache.org/jira/browse/NIFI-12137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt updated NIFI-12137:
Summary: Improve Java Python processor behaviors in NiFi 2x (was: Improve
Java 2x Python processor behaviors)
> Improve Java Python processor behaviors in NiFi 2x
> --
>
> Key: NIFI-12137
> URL: https://issues.apache.org/jira/browse/NIFI-12137
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Joe Witt
>Priority: Major
>
> Built a python processor. Deployed as version 0.0.1. Updated to 0.0.2
> replacing original file. Updated to 0.0.3. Then started just keeping same
> version and restarting processor. However, on restart I received this output
> {noformat}
> 2023-09-26 13:00:10 2023-09-26 20:00:06,984 ERROR [NiFi Web Server-140]
> o.a.n.n.StandardExtensionDiscoveringManager Could not instantiate class of
> type python.LookupAddress using ClassLoader for bundle
> org.apache.nifi:python-extensions:0.0.1-SNAPSHOT
> 2023-09-26 13:00:10 py4j.Py4JException: An exception was raised by the Python
> Proxy. Return Message: Traceback (most recent call last):
> 2023-09-26 13:00:10 File
> "/opt/nifi/nifi-current/python/framework/py4j/java_gateway.py", line 2466, in
> _call_proxy
> 2023-09-26 13:00:10 return_value = getattr(self.pool[obj_id],
> method)(*params)
> 2023-09-26 13:00:10 File
> "/opt/nifi/nifi-current/./python/framework/Controller.py", line 52, in
> createProcessor
> 2023-09-26 13:00:10 processorClass =
> self.extensionManager.getProcessorClass(processorType, version, work_dir)
> 2023-09-26 13:00:10 File
> "/opt/nifi/nifi-current/python/framework/ExtensionManager.py", line 138, in
> getProcessorClass
> 2023-09-26 13:00:10 raise ValueError('Invalid Processor Type: No module
> is known to contain Processor of type ' + type + ' version ' + version)
> 2023-09-26 13:00:10 ValueError: Invalid Processor Type: No module is known to
> contain Processor of type LookupAddress version 0.0.1-SNAPSHOT
> 2023-09-26 13:00:10
> 2023-09-26 13:00:10 at py4j.Protocol.getReturnValue(Protocol.java:476)
> 2023-09-26 13:00:09 tail: '/opt/nifi/nifi-current/logs/nifi-app.log' has been
> replaced; following new file
> {noformat}
> In a flow using a FlowFileTransform I am seeing data when I route it to
> faillure also going to original. If routed to failure it should not go to
> original. Original should only be when success is the chosen route. Will
> replicate and add more info here.
> We should change ExpressionLanguageScope.LIMITED to ENVIRONMENT.
> We should try to use python conventions on all method names instead of Java
> conventions.
> Can we define the 'logger' that gets injected to implementations so code
> completion can work in IDEs in the meantime?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (NIFI-12137) Improve Java 2x Python processor behaviors
Joe Witt created NIFI-12137:
---
Summary: Improve Java 2x Python processor behaviors
Key: NIFI-12137
URL: https://issues.apache.org/jira/browse/NIFI-12137
Project: Apache NiFi
Issue Type: Bug
Reporter: Joe Witt
Built a python processor. Deployed as version 0.0.1. Updated to 0.0.2
replacing original file. Updated to 0.0.3. Then started just keeping same
version and restarting processor. However, on restart I received this output
{noformat}
2023-09-26 13:00:10 2023-09-26 20:00:06,984 ERROR [NiFi Web Server-140]
o.a.n.n.StandardExtensionDiscoveringManager Could not instantiate class of type
python.LookupAddress using ClassLoader for bundle
org.apache.nifi:python-extensions:0.0.1-SNAPSHOT
2023-09-26 13:00:10 py4j.Py4JException: An exception was raised by the Python
Proxy. Return Message: Traceback (most recent call last):
2023-09-26 13:00:10 File
"/opt/nifi/nifi-current/python/framework/py4j/java_gateway.py", line 2466, in
_call_proxy
2023-09-26 13:00:10 return_value = getattr(self.pool[obj_id],
method)(*params)
2023-09-26 13:00:10 File
"/opt/nifi/nifi-current/./python/framework/Controller.py", line 52, in
createProcessor
2023-09-26 13:00:10 processorClass =
self.extensionManager.getProcessorClass(processorType, version, work_dir)
2023-09-26 13:00:10 File
"/opt/nifi/nifi-current/python/framework/ExtensionManager.py", line 138, in
getProcessorClass
2023-09-26 13:00:10 raise ValueError('Invalid Processor Type: No module is
known to contain Processor of type ' + type + ' version ' + version)
2023-09-26 13:00:10 ValueError: Invalid Processor Type: No module is known to
contain Processor of type LookupAddress version 0.0.1-SNAPSHOT
2023-09-26 13:00:10
2023-09-26 13:00:10 at py4j.Protocol.getReturnValue(Protocol.java:476)
2023-09-26 13:00:09 tail: '/opt/nifi/nifi-current/logs/nifi-app.log' has been
replaced; following new file
{noformat}
In a flow using a FlowFileTransform I am seeing data when I route it to
faillure also going to original. If routed to failure it should not go to
original. Original should only be when success is the chosen route. Will
replicate and add more info here.
We should change ExpressionLanguageScope.LIMITED to ENVIRONMENT.
We should try to use python conventions on all method names instead of Java
conventions.
Can we define the 'logger' that gets injected to implementations so code
completion can work in IDEs in the meantime?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (NIFI-12100) Remove ConvertExcelToCSVProcessor
[ https://issues.apache.org/jira/browse/NIFI-12100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769719#comment-17769719 ] Daniel Stieglitz edited comment on NIFI-12100 at 9/27/23 6:34 PM: -- [~exceptionfactory] I submittied a PR for this ticket. You had mentioned in NIFI-11172 you could help me with the deprecation PR which needs to be branched off of support/nifi-1.x. How do I branch off support/nifi-1.x and then submit a PR off there? I see on the [Contributor Guide|https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide#ContributorGuide-Keepingyourfeaturebranchcurrent] about checking out the main or '1.x.0' branch but not the 'support/nifi-1.x' branch. I assume before submitting the PR I still have to run the build with the contrib-check profile, right? Thank you for your help! was (Author: JIRAUSER294662): [~exceptionfactory] I will shortly be submitting the PR for this ticket. You had mentioned in NIFI-11172 you could help me with the deprecation PR which needs to be branched off of support/nifi-1.x. How do I branch off support/nifi-1.x and then submit a PR off there? I see on the [Contributor Guide|https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide#ContributorGuide-Keepingyourfeaturebranchcurrent] about checking out the main or '1.x.0' branch but not the 'support/nifi-1.x' branch. I assume before submitting the PR I still have to run the build with the contrib-check profile, right? Thank you for your help! > Remove ConvertExcelToCSVProcessor > --- > > Key: NIFI-12100 > URL: https://issues.apache.org/jira/browse/NIFI-12100 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > ConvertExcelToCSVProcessor is no longer needed since there is now the > ExcelReader which along with CSVRecordSetWriter can be used in ConvertRecord > to achieve the same thing. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12100) Remove ConvertExcelToCSVProcessor
[ https://issues.apache.org/jira/browse/NIFI-12100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Stieglitz updated NIFI-12100: Status: Patch Available (was: In Progress) > Remove ConvertExcelToCSVProcessor > --- > > Key: NIFI-12100 > URL: https://issues.apache.org/jira/browse/NIFI-12100 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > ConvertExcelToCSVProcessor is no longer needed since there is now the > ExcelReader which along with CSVRecordSetWriter can be used in ConvertRecord > to achieve the same thing. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] dan-s1 opened a new pull request, #7802: NIFI-12100 Removed the ConvertExcelToCSVProcessor
dan-s1 opened a new pull request, #7802: URL: https://github.com/apache/nifi/pull/7802 # Summary [NIFI-0](https://issues.apache.org/jira/browse/NIFI-0) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] joewitt opened a new pull request, #7801: NIFI-11917 removing support for RPM building as we lack the infrastru…
joewitt opened a new pull request, #7801: URL: https://github.com/apache/nifi/pull/7801 …cture/maintenance to keep it healthy and we dont publish the artifacts # Summary [NIFI-0](https://issues.apache.org/jira/browse/NIFI-0) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Comment Edited] (NIFI-12100) Remove ConvertExcelToCSVProcessor
[ https://issues.apache.org/jira/browse/NIFI-12100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769719#comment-17769719 ] Daniel Stieglitz edited comment on NIFI-12100 at 9/27/23 5:40 PM: -- [~exceptionfactory] I will shortly be submitting the PR for this ticket. You had mentioned in NIFI-11172 you could help me with the deprecation PR which needs to be branched off of support/nifi-1.x. How do I branch off support/nifi-1.x and then submit a PR off there? I see on the [Contributor Guide|https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide#ContributorGuide-Keepingyourfeaturebranchcurrent] about checking out the main or '1.x.0' branch but not the 'support/nifi-1.x' branch. I assume before submitting the PR I still have to run the build with the contrib-check profile, right? Thank you for your help! was (Author: JIRAUSER294662): [~exceptionfactory] I will shortly be submitting the PR for this ticket. You had mentioned in NIFI-11172 you could help me with the deprecation PR which needs to be branched off of support/nifi-1.x. How do I branch off support/nifi-1.x and then submit a PR off there? I assume before submitting the PR I still have to run the build with the contrib-check profile, right? Thank you for your help! > Remove ConvertExcelToCSVProcessor > --- > > Key: NIFI-12100 > URL: https://issues.apache.org/jira/browse/NIFI-12100 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Major > > ConvertExcelToCSVProcessor is no longer needed since there is now the > ExcelReader which along with CSVRecordSetWriter can be used in ConvertRecord > to achieve the same thing. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12100) Remove ConvertExcelToCSVProcessor
[ https://issues.apache.org/jira/browse/NIFI-12100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769719#comment-17769719 ] Daniel Stieglitz commented on NIFI-12100: - [~exceptionfactory] I will shortly be submitting the PR for this ticket. You had mentioned in NIFI-11172 you could help me with the deprecation PR which needs to be branched off of support/nifi-1.x. How do I branch off support/nifi-1.x and then submit a PR off there? I assume before submitting the PR I still have to run the build with the contrib-check profile, right? Thank you for your help! > Remove ConvertExcelToCSVProcessor > --- > > Key: NIFI-12100 > URL: https://issues.apache.org/jira/browse/NIFI-12100 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Daniel Stieglitz >Assignee: Daniel Stieglitz >Priority: Major > > ConvertExcelToCSVProcessor is no longer needed since there is now the > ExcelReader which along with CSVRecordSetWriter can be used in ConvertRecord > to achieve the same thing. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11917) Remove RPM build from NiFi POM/Assembly
[
https://issues.apache.org/jira/browse/NIFI-11917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt updated NIFI-11917:
Summary: Remove RPM build from NiFi POM/Assembly (was: RPM build not being
maintained - not working - not in testing chain - consider removal)
> Remove RPM build from NiFi POM/Assembly
> ---
>
> Key: NIFI-11917
> URL: https://issues.apache.org/jira/browse/NIFI-11917
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.23.0
> Environment: Linux
>Reporter: Gregory M. Foreman
>Assignee: Joe Witt
>Priority: Major
> Fix For: 2.latest
>
>
> Nifi 1.23.0 is unable to build from source when using the rpm profile setting.
> This command:
> {code:java}
> JAVA_OPTS="-Xms128m -Xmx4g"
> MAVEN_OPTS="-Dorg.slf4j.simpleLogger.defaultLogLevel=ERROR -Xms1024m
> -Xmx3076m -XX:MaxPermSize=256m"
> mvn -T C2.0 --batch-mode -Prpm -DskipTests clean install
> {code}
> produces the following error:
> {code:java}
> [ERROR] Failed to execute goal
> org.codehaus.mojo:rpm-maven-plugin:2.2.0:attached-rpm (build-bin-rpm) on
> project nifi-toolkit-assembly: Source location
> /root/test/nifi-1.23.0/nifi-toolkit/nifi-toolkit-assembly/target/nifi-toolkit-1.23.0-bin/nifi-toolkit-1.23.0/LICENSE
> does not exist -> [Help 1]{code}
> The path above ends at:
> /root/test2/nifi/nifi-toolkit/nifi-toolkit-assembly/target/
> There is a zip file within it:
> nifi-toolkit-1.23.0-bin.zip
> It seems like this unzip was missed.
> Environment info:
> {code:bash}
> mvn -version
> Apache Maven 3.8.4 (9b656c72d54e5bacbed989b64718c159fe39b537)
> Maven home: /opt/maven
> Java version: 1.8.0_372, vendor: Red Hat, Inc., runtime:
> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.372.b07-1.el7_9.x86_64/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "linux", version: "3.10.0-1160.95.1.el7.x86_64", arch: "amd64",
> family: "unix"{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (NIFI-12136) Add documentation about HTTPS and OpenID Authentication for Docker
[ https://issues.apache.org/jira/browse/NIFI-12136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt resolved NIFI-12136. - Fix Version/s: 2.0.0 Assignee: Joe Witt Resolution: Fixed > Add documentation about HTTPS and OpenID Authentication for Docker > -- > > Key: NIFI-12136 > URL: https://issues.apache.org/jira/browse/NIFI-12136 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Docker, Documentation Website, > Examples >Affects Versions: 1.23.2 >Reporter: Marcelo >Assignee: Joe Witt >Priority: Minor > Labels: documentation, pull-request-available > Fix For: 2.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > I've added documentation (Readme) on how to create a Nifi Docker Instance for > HTTPS and OpenID Authentication -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] joewitt commented on pull request #7800: NIFI-12136 Add documentation about HTTPS and OpenID Authentication for Docker
joewitt commented on PR #7800: URL: https://github.com/apache/nifi/pull/7800#issuecomment-1737795689 Cool - looks helpful. Merged. Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-12136) Add documentation about HTTPS and OpenID Authentication for Docker
[ https://issues.apache.org/jira/browse/NIFI-12136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769715#comment-17769715 ] ASF subversion and git services commented on NIFI-12136: Commit 01fb3e99dde95fc4e83c598536263eedf632ef65 in nifi's branch refs/heads/main from Marcelo Vinícius de Sousa Campos [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=01fb3e99dd ] NIFI-12136 This closes #7800. Update README.md to add an example how to using Nifi to connect to an OpenID server. Signed-off-by: Joseph Witt > Add documentation about HTTPS and OpenID Authentication for Docker > -- > > Key: NIFI-12136 > URL: https://issues.apache.org/jira/browse/NIFI-12136 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Docker, Documentation Website, > Examples >Affects Versions: 1.23.2 >Reporter: Marcelo >Priority: Minor > Labels: documentation, pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > I've added documentation (Readme) on how to create a Nifi Docker Instance for > HTTPS and OpenID Authentication -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] asfgit closed pull request #7800: NIFI-12136 Add documentation about HTTPS and OpenID Authentication for Docker
asfgit closed pull request #7800: NIFI-12136 Add documentation about HTTPS and OpenID Authentication for Docker URL: https://github.com/apache/nifi/pull/7800 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] joewitt commented on pull request #7800: NIFI-12136 Add documentation about HTTPS and OpenID Authentication for Docker
joewitt commented on PR #7800: URL: https://github.com/apache/nifi/pull/7800#issuecomment-1737789549 thanks @marcelo225 checking it out now -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-11917) RPM build not being maintained - not working - not in testing chain - consider removal
[
https://issues.apache.org/jira/browse/NIFI-11917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769711#comment-17769711
]
Joe Witt commented on NIFI-11917:
-
[~gforeman02] Ok thanks. I suspect we will have to drop the RPM from the build
logic since we're not maintaining it and the effort/energy seems to be on the
docker side increasingly.
If people step up to take on the work of maintaining and automating the testing
of the RPM build we could definitely bring it back in at some point.
Thanks
> RPM build not being maintained - not working - not in testing chain -
> consider removal
> --
>
> Key: NIFI-11917
> URL: https://issues.apache.org/jira/browse/NIFI-11917
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.23.0
> Environment: Linux
>Reporter: Gregory M. Foreman
>Assignee: Joe Witt
>Priority: Major
> Fix For: 2.latest
>
>
> Nifi 1.23.0 is unable to build from source when using the rpm profile setting.
> This command:
> {code:java}
> JAVA_OPTS="-Xms128m -Xmx4g"
> MAVEN_OPTS="-Dorg.slf4j.simpleLogger.defaultLogLevel=ERROR -Xms1024m
> -Xmx3076m -XX:MaxPermSize=256m"
> mvn -T C2.0 --batch-mode -Prpm -DskipTests clean install
> {code}
> produces the following error:
> {code:java}
> [ERROR] Failed to execute goal
> org.codehaus.mojo:rpm-maven-plugin:2.2.0:attached-rpm (build-bin-rpm) on
> project nifi-toolkit-assembly: Source location
> /root/test/nifi-1.23.0/nifi-toolkit/nifi-toolkit-assembly/target/nifi-toolkit-1.23.0-bin/nifi-toolkit-1.23.0/LICENSE
> does not exist -> [Help 1]{code}
> The path above ends at:
> /root/test2/nifi/nifi-toolkit/nifi-toolkit-assembly/target/
> There is a zip file within it:
> nifi-toolkit-1.23.0-bin.zip
> It seems like this unzip was missed.
> Environment info:
> {code:bash}
> mvn -version
> Apache Maven 3.8.4 (9b656c72d54e5bacbed989b64718c159fe39b537)
> Maven home: /opt/maven
> Java version: 1.8.0_372, vendor: Red Hat, Inc., runtime:
> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.372.b07-1.el7_9.x86_64/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "linux", version: "3.10.0-1160.95.1.el7.x86_64", arch: "amd64",
> family: "unix"{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1668: MINIFICPP-2226 Remove flakiness from LoggerTests
lordgamez commented on code in PR #1668:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1668#discussion_r1338932369
##
libminifi/test/unit/LoggerTests.cpp:
##
@@ -364,24 +363,14 @@ TEST_CASE("Test sending multiple segments at once",
"[ttl16]") {
log_config.initialize(properties);
auto logger = log_config.getLogger("CompressionTestMultiSegment");
- std::random_device rd;
- std::mt19937 eng(rd());
- constexpr const char * TEXT_CHARS =
"abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890";
- const int index_of_last_char = gsl::narrow(strlen(TEXT_CHARS)) - 1;
- std::uniform_int_distribution<> distr(0, index_of_last_char);
- std::vector data(100);
- std::string log_str;
- const size_t SEGMENT_COUNT = 5;
- for (size_t idx = 0; idx < SEGMENT_COUNT; ++idx) {
-std::generate_n(data.begin(), data.size(), [&] { return
TEXT_CHARS[static_cast(distr(eng))]; });
-log_str = std::string{data.begin(), data.end()} + "." +
std::to_string(idx);
+ std::string log_str =
"abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890";
+ for (size_t i = 0; i < 100; ++i) {
logger->log_error(log_str.c_str());
}
LoggerTestAccessor::runCompression(log_config);
-
auto compressed_logs = logging::LoggerConfiguration::getCompressedLogs();
- REQUIRE(compressed_logs.size() == SEGMENT_COUNT);
- auto logs = decompress(compressed_logs[SEGMENT_COUNT - 1]);
+ REQUIRE(compressed_logs.size() > 1);
Review Comment:
I think the inconsistency comes from the `LogCompressorSink` that creates a
separate thread for processing and compressing the incoming logs in parallel. I
will have to check it how the test can be changed to make the results more
consistent.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-11917) RPM build not being maintained - not working - not in testing chain - consider removal
[
https://issues.apache.org/jira/browse/NIFI-11917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769708#comment-17769708
]
Gregory M. Foreman commented on NIFI-11917:
---
[~joewitt] , having Nifi in the EPEL repo would remove the need to run the
maven build at all (for us anyway). I think the only thing needed by EPEL is
the binary version of Nifi, so testing should be easy and would be performed by
EPEL project resources. If none of the options is possible, then we will just
create the rpm on our own.
> RPM build not being maintained - not working - not in testing chain -
> consider removal
> --
>
> Key: NIFI-11917
> URL: https://issues.apache.org/jira/browse/NIFI-11917
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.23.0
> Environment: Linux
>Reporter: Gregory M. Foreman
>Assignee: Joe Witt
>Priority: Major
> Fix For: 2.latest
>
>
> Nifi 1.23.0 is unable to build from source when using the rpm profile setting.
> This command:
> {code:java}
> JAVA_OPTS="-Xms128m -Xmx4g"
> MAVEN_OPTS="-Dorg.slf4j.simpleLogger.defaultLogLevel=ERROR -Xms1024m
> -Xmx3076m -XX:MaxPermSize=256m"
> mvn -T C2.0 --batch-mode -Prpm -DskipTests clean install
> {code}
> produces the following error:
> {code:java}
> [ERROR] Failed to execute goal
> org.codehaus.mojo:rpm-maven-plugin:2.2.0:attached-rpm (build-bin-rpm) on
> project nifi-toolkit-assembly: Source location
> /root/test/nifi-1.23.0/nifi-toolkit/nifi-toolkit-assembly/target/nifi-toolkit-1.23.0-bin/nifi-toolkit-1.23.0/LICENSE
> does not exist -> [Help 1]{code}
> The path above ends at:
> /root/test2/nifi/nifi-toolkit/nifi-toolkit-assembly/target/
> There is a zip file within it:
> nifi-toolkit-1.23.0-bin.zip
> It seems like this unzip was missed.
> Environment info:
> {code:bash}
> mvn -version
> Apache Maven 3.8.4 (9b656c72d54e5bacbed989b64718c159fe39b537)
> Maven home: /opt/maven
> Java version: 1.8.0_372, vendor: Red Hat, Inc., runtime:
> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.372.b07-1.el7_9.x86_64/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "linux", version: "3.10.0-1160.95.1.el7.x86_64", arch: "amd64",
> family: "unix"{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1669: MINIFICPP-2210 Add C2 debug command to MiNiFi Controller
lordgamez commented on code in PR #1669:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1669#discussion_r1338924083
##
libminifi/include/utils/file/ArchiveUtils.h:
##
@@ -0,0 +1,31 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#pragma once
+
+#include
+#include
+#include
+
+#include "io/ArchiveStream.h"
+#include "utils/expected.h"
+#include "io/BufferStream.h"
+
+namespace org::apache::nifi::minifi::utils::archive {
+
+nonstd::expected, std::string>
createArchive(std::map>& files,
const std::shared_ptr& logger);
Review Comment:
I'm not sure we have to be this strict with the interface of a utility
function, it may be enough to be as generic as our use cases require it to be,
and shape the interface according to the new use cases of the function as
required. There were some good points, I removed the unneeded logger and
changed the `files` parameter to const ref, but as this can utility can be
restricted to out C2 use cases I moved it to C2Utils and renamed it to
`createDebugBundleArchive`. It can be modified and moved to a more generic
utility file later if a new use cases arises.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-12136) Add documentation about HTTPS and OpenID Authentication
[ https://issues.apache.org/jira/browse/NIFI-12136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo updated NIFI-12136: --- Labels: documentation pull-request-available (was: documentation) > Add documentation about HTTPS and OpenID Authentication > --- > > Key: NIFI-12136 > URL: https://issues.apache.org/jira/browse/NIFI-12136 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Docker, Documentation Website, > Examples >Affects Versions: 1.23.2 >Reporter: Marcelo >Priority: Minor > Labels: documentation, pull-request-available > > I've added documentation (Readme) on how to create a Nifi Docker Instance for > HTTPS and OpenID Authentication -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12136) Add documentation about HTTPS and OpenID Authentication for Docker
[ https://issues.apache.org/jira/browse/NIFI-12136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo updated NIFI-12136: --- Summary: Add documentation about HTTPS and OpenID Authentication for Docker (was: Add documentation about HTTPS and OpenID Authentication) > Add documentation about HTTPS and OpenID Authentication for Docker > -- > > Key: NIFI-12136 > URL: https://issues.apache.org/jira/browse/NIFI-12136 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Docker, Documentation Website, > Examples >Affects Versions: 1.23.2 >Reporter: Marcelo >Priority: Minor > Labels: documentation, pull-request-available > > I've added documentation (Readme) on how to create a Nifi Docker Instance for > HTTPS and OpenID Authentication -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12136) Add documentation about HTTPS and OpenID Authentication
Marcelo created NIFI-12136: -- Summary: Add documentation about HTTPS and OpenID Authentication Key: NIFI-12136 URL: https://issues.apache.org/jira/browse/NIFI-12136 Project: Apache NiFi Issue Type: Improvement Components: Configuration, Docker, Documentation Website, Examples Affects Versions: 1.23.2 Reporter: Marcelo I've added documentation (Readme) on how to create a Nifi Docker Instance for HTTPS and OpenID Authentication -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-12135) Update 2 missing OIDC properties in update_oidc_properties.sh
Marcelo created NIFI-12135: -- Summary: Update 2 missing OIDC properties in update_oidc_properties.sh Key: NIFI-12135 URL: https://issues.apache.org/jira/browse/NIFI-12135 Project: Apache NiFi Issue Type: Improvement Components: Configuration, Docker Affects Versions: 1.23.2, 1.23.1 Reporter: Marcelo According this Nifi Documentation ([https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#openid_connect)] I've created a PR to update 2 missing OIDC properties in *update_oidc_properties.sh* file: - nifi.security.user.oidc.claim.groups - nifi.security.user.oidc.token.refresh.window -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1669: MINIFICPP-2210 Add C2 debug command to MiNiFi Controller
lordgamez commented on code in PR #1669:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1669#discussion_r1338885235
##
docker/test/integration/cluster/DockerCommunicator.py:
##
@@ -66,3 +66,19 @@ def write_content_to_container(self, content,
container_name, dst_path):
tar.addfile(info, io.BytesIO(content.encode('utf-8')))
with open(os.path.join(td, 'content.tar'), 'rb') as data:
return self.__put_archive(container_name,
os.path.dirname(dst_path), data.read())
+
+def copy_file_from_container(self, container_name, src_path_in_container,
dest_dir_on_host) -> bool:
+try:
+container = self.client.containers.get(container_name)
+(bits, _) = container.get_archive(src_path_in_container)
+tmp_tar_path = os.path.join(dest_dir_on_host, "tmp_debug.tar")
Review Comment:
`tmp_debug.tar` is only temporary because the python docker API only offers
retrieving a file or a folder in a single tar file. This file is deleted after
extracted. I can rename it to something more generic, as it can be used for any
file. Updated in 257aff8003d0ca34e0537857637a6ba6be3ec8da
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1669: MINIFICPP-2210 Add C2 debug command to MiNiFi Controller
lordgamez commented on code in PR #1669:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1669#discussion_r1338885235
##
docker/test/integration/cluster/DockerCommunicator.py:
##
@@ -66,3 +66,19 @@ def write_content_to_container(self, content,
container_name, dst_path):
tar.addfile(info, io.BytesIO(content.encode('utf-8')))
with open(os.path.join(td, 'content.tar'), 'rb') as data:
return self.__put_archive(container_name,
os.path.dirname(dst_path), data.read())
+
+def copy_file_from_container(self, container_name, src_path_in_container,
dest_dir_on_host) -> bool:
+try:
+container = self.client.containers.get(container_name)
+(bits, _) = container.get_archive(src_path_in_container)
+tmp_tar_path = os.path.join(dest_dir_on_host, "tmp_debug.tar")
Review Comment:
`tmp_debug.tar` is only temporary because the python docker API only offers
retrieving a file or a folder in a single tar file. This file is deleted after
extracted. I can rename it to something more generic, as it can be used for any
file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1669: MINIFICPP-2210 Add C2 debug command to MiNiFi Controller
lordgamez commented on code in PR #1669:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1669#discussion_r1338875658
##
controller/Controller.cpp:
##
@@ -225,4 +227,43 @@ bool getJstacks(const utils::net::SocketData& socket_data,
std::ostream ) {
return true;
}
+nonstd::expected getDebugBundle(const
utils::net::SocketData& socket_data, const std::filesystem::path& target_dir) {
+ std::unique_ptr connection_stream =
std::make_unique(socket_data);
Review Comment:
Unfortunately that doesn't compile due to the virtual inheritance in
InputStream and OutputStream classes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Resolved] (NIFI-12134) Disable Directory Listing property is duplicated on PutSFTP processor
[ https://issues.apache.org/jira/browse/NIFI-12134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Csaba Bejan resolved NIFI-12134. Resolution: Fixed > Disable Directory Listing property is duplicated on PutSFTP processor > - > > Key: NIFI-12134 > URL: https://issues.apache.org/jira/browse/NIFI-12134 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.23.2 >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Minor > Fix For: 2.0.0, 1.24.0 > > Time Spent: 20m > Remaining Estimate: 0h > > The property is present both as static and dynamic property, this doesn't > make sense. The dynamic one should be removed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-12134) Disable Directory Listing property is duplicated on PutSFTP processor
[ https://issues.apache.org/jira/browse/NIFI-12134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Csaba Bejan updated NIFI-12134: --- Fix Version/s: 1.24.0 > Disable Directory Listing property is duplicated on PutSFTP processor > - > > Key: NIFI-12134 > URL: https://issues.apache.org/jira/browse/NIFI-12134 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.23.2 >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Minor > Fix For: 2.0.0, 1.24.0 > > Time Spent: 20m > Remaining Estimate: 0h > > The property is present both as static and dynamic property, this doesn't > make sense. The dynamic one should be removed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12134) Disable Directory Listing property is duplicated on PutSFTP processor
[ https://issues.apache.org/jira/browse/NIFI-12134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769679#comment-17769679 ] ASF subversion and git services commented on NIFI-12134: Commit cfb820be2dac54176857d7824ffa2b002279bed2 in nifi's branch refs/heads/support/nifi-1.x from Arpad Boda [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=cfb820be2d ] NIFI-12134 - Disable Directory Listing property is duplicated on PutSFTP processor Signed-off-by: Bejan Csaba This closes #7798. > Disable Directory Listing property is duplicated on PutSFTP processor > - > > Key: NIFI-12134 > URL: https://issues.apache.org/jira/browse/NIFI-12134 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.23.2 >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Minor > Fix For: 2.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > The property is present both as static and dynamic property, this doesn't > make sense. The dynamic one should be removed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12134) Disable Directory Listing property is duplicated on PutSFTP processor
[ https://issues.apache.org/jira/browse/NIFI-12134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769675#comment-17769675 ] ASF subversion and git services commented on NIFI-12134: Commit dbcc223b4057ede14038466bdcc38ec2e56298a5 in nifi's branch refs/heads/main from Arpad Boda [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=dbcc223b40 ] NIFI-12134 - Disable Directory Listing property is duplicated on PutSFTP processor Signed-off-by: Bejan Csaba This closes #7798. > Disable Directory Listing property is duplicated on PutSFTP processor > - > > Key: NIFI-12134 > URL: https://issues.apache.org/jira/browse/NIFI-12134 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.23.2 >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Minor > Fix For: 2.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > The property is present both as static and dynamic property, this doesn't > make sense. The dynamic one should be removed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] bejancsaba closed pull request #7798: NIFI-12134 - Disable Directory Listing property is duplicated on PutS…
bejancsaba closed pull request #7798: NIFI-12134 - Disable Directory Listing property is duplicated on PutS… URL: https://github.com/apache/nifi/pull/7798 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (MINIFICPP-2231) Avoid global CXX flags, especially -Werror
Marton Szasz created MINIFICPP-2231: --- Summary: Avoid global CXX flags, especially -Werror Key: MINIFICPP-2231 URL: https://issues.apache.org/jira/browse/MINIFICPP-2231 Project: Apache NiFi MiNiFi C++ Issue Type: Bug Reporter: Marton Szasz We should only set compiler flags on our own targets, not globally. All FetchContent dependencies inherit our global settings, but we don't want to compile our dependencies with Werror enabled. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1661: MINIFICPP-2196 Integrate cppcoreguideline clang-tidy checks in CI
lordgamez commented on code in PR #1661:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1661#discussion_r1338795626
##
libminifi/src/core/extension/ExtensionManager.cpp:
##
@@ -31,9 +31,13 @@ namespace org::apache::nifi::minifi::core::extension {
const std::shared_ptr ExtensionManager::logger_ =
logging::LoggerFactory::getLogger();
-ExtensionManager::ExtensionManager() {
- modules_.push_back(std::make_unique());
- active_module_ = modules_[0].get();
+ExtensionManager::ExtensionManager()
+: modules_([] {
+std::vector> modules;
+modules.push_back(std::make_unique());
+return modules;
+ }()),
Review Comment:
Unfortunately that does not compile:
```
In file included from /usr/lib/llvm-16/bin/../include/c++/v1/memory:884:
In file included from
/usr/lib/llvm-16/bin/../include/c++/v1/__memory/allocate_at_least.h:13:
/usr/lib/llvm-16/bin/../include/c++/v1/__memory/allocator_traits.h:304:9:
error: no matching function for call to 'construct_at'
_VSTD::construct_at(__p, _VSTD::forward<_Args>(__args)...);
^~~
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1661: MINIFICPP-2196 Integrate cppcoreguideline clang-tidy checks in CI
lordgamez commented on code in PR #1661:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1661#discussion_r1338792943
##
extensions/librdkafka/KafkaConnection.cpp:
##
@@ -69,8 +70,8 @@ void KafkaConnection::setConnection(gsl::owner
producer) {
startPoll();
}
-rd_kafka_t *KafkaConnection::getConnection() const {
- return kafka_connection_;
+rd_kafka_t* KafkaConnection::getConnection() const {
+ return static_cast(kafka_connection_);
Review Comment:
Clang tidy gives a warning as we return a gsl::owner<> type but the return
value does not reflect that. This way we make it explicit that we do not want
to move the ownership, only return the raw pointer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (MINIFICPP-2042) Address CWEL ambigous Output property
[
https://issues.apache.org/jira/browse/MINIFICPP-2042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769663#comment-17769663
]
Marton Szasz commented on MINIFICPP-2042:
-
[~aboda] idea: mark possible value as deprecated, remove from docs, but keep
behavior
> Address CWEL ambigous Output property
> --
>
> Key: MINIFICPP-2042
> URL: https://issues.apache.org/jira/browse/MINIFICPP-2042
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
>Reporter: Martin Zink
>Priority: Minor
>
> [~lordgamez] made a great observation
> [https://github.com/apache/nifi-minifi-cpp/pull/1482#issuecomment-1385016182]
> {noformat}
> Currently we have the options XML, Plaintext, Both, JSON for Output Format.
> The "Both" option is quite ambiguous now that we have 3 different output
> formats, should we revise this? Either have an "All" option for all 3 output
> formats, or rename it to "XML and Plaintext'.{noformat}
> I don't see an easy way to fix this without breaking compatibility.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-2040) Do not deserialize flow file just to be deleted
[ https://issues.apache.org/jira/browse/MINIFICPP-2040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz resolved MINIFICPP-2040. - Resolution: Fixed > Do not deserialize flow file just to be deleted > --- > > Key: MINIFICPP-2040 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2040 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Adam Debreceni >Assignee: Adam Debreceni >Priority: Major > Time Spent: 1h 20m > Remaining Estimate: 0h > > Currently FlowFileRepository deserializes each flow file just to be deleted, > it makes the deletion of the content possible. We should elide the > deserialization step. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-2038) Update SSL context of C2 agent after flow update
[ https://issues.apache.org/jira/browse/MINIFICPP-2038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz resolved MINIFICPP-2038. - Resolution: Won't Fix Reading C2 SSL properties from an SSLContextService should be dropped. > Update SSL context of C2 agent after flow update > > > Key: MINIFICPP-2038 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2038 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Gábor Gyimesi >Priority: Minor > > SSLContext for C2 communication is retrieved from the SSLContextService > defined in the flow configuration. In case of a flow configuration update we > should update the C2Agents's SSLContext if it is changed in the updated flow. > The same should be considered for ControllerSocketProtocol -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2038) Update SSL context of C2 agent after flow update
[ https://issues.apache.org/jira/browse/MINIFICPP-2038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-2038: Priority: Minor (was: Major) > Update SSL context of C2 agent after flow update > > > Key: MINIFICPP-2038 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2038 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Gábor Gyimesi >Priority: Minor > > SSLContext for C2 communication is retrieved from the SSLContextService > defined in the flow configuration. In case of a flow configuration update we > should update the C2Agents's SSLContext if it is changed in the updated flow. > The same should be considered for ControllerSocketProtocol -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2033) drop runDurationNanos
[ https://issues.apache.org/jira/browse/MINIFICPP-2033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-2033: Summary: drop runDurationNanos (was: Check if we need runDurationNanos) > drop runDurationNanos > - > > Key: MINIFICPP-2033 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2033 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Adam Debreceni >Priority: Major > > Currently we can set the runDurationNanos but it is unused, investigate if we > need it or it can be removed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (MINIFICPP-2002) Typecheck query arguments in SQL processors
[ https://issues.apache.org/jira/browse/MINIFICPP-2002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769654#comment-17769654 ] Gábor Gyimesi commented on MINIFICPP-2002: -- Check NiFi implementation > Typecheck query arguments in SQL processors > --- > > Key: MINIFICPP-2002 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2002 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Adam Debreceni >Priority: Trivial > > The used backend soci,odbc, do not seem to check the type of bound arguments > in a prepared statement. We should investigate if it is feasible to fail such > flowfiles or provide a "Check type" property to these sql processors. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2001) In SQL processors should fail flowfiles with too many arguments
[ https://issues.apache.org/jira/browse/MINIFICPP-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-2001: Labels: (was: beginner hacktoberfest starter) > In SQL processors should fail flowfiles with too many arguments > --- > > Key: MINIFICPP-2001 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2001 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Adam Debreceni >Priority: Major > > Currently the used libraries soci, odbc do not give feedback when more > arguments are provided than required by the query, e.g. query = `INSERT INTO > test VALUES (?);` args = ['1', '2'], we should give either a warning or fail > such flowfiles. > > Tests are present in > libminifi/test/sql-tests/ExecuteSQLTests.cpp > libminifi/test/sql-tests/PutSQLTests.cpp -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2002) Typecheck query arguments in SQL processors
[ https://issues.apache.org/jira/browse/MINIFICPP-2002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-2002: Priority: Trivial (was: Major) > Typecheck query arguments in SQL processors > --- > > Key: MINIFICPP-2002 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2002 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Adam Debreceni >Priority: Trivial > > The used backend soci,odbc, do not seem to check the type of bound arguments > in a prepared statement. We should investigate if it is feasible to fail such > flowfiles or provide a "Check type" property to these sql processors. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2001) In SQL processors should fail flowfiles with too many arguments
[ https://issues.apache.org/jira/browse/MINIFICPP-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gábor Gyimesi updated MINIFICPP-2001: - Description: Currently the used libraries soci, odbc do not give feedback when more arguments are provided than required by the query, e.g. query = `INSERT INTO test VALUES (?);` args = ['1', '2'], we should give either a warning or fail such flowfiles. Tests are present in libminifi/test/sql-tests/ExecuteSQLTests.cpp libminifi/test/sql-tests/PutSQLTests.cpp was:Currently the used libraries soci, odbc do not give feedback when more arguments are provided than required by the query, e.g. query = `INSERT INTO test VALUES (?);` args = ['1', '2'], we should give either a warning or fail such flowfiles. > In SQL processors should fail flowfiles with too many arguments > --- > > Key: MINIFICPP-2001 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2001 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Adam Debreceni >Priority: Major > Labels: beginner, hacktoberfest, starter > > Currently the used libraries soci, odbc do not give feedback when more > arguments are provided than required by the query, e.g. query = `INSERT INTO > test VALUES (?);` args = ['1', '2'], we should give either a warning or fail > such flowfiles. > > Tests are present in > libminifi/test/sql-tests/ExecuteSQLTests.cpp > libminifi/test/sql-tests/PutSQLTests.cpp -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2001) In SQL processors should fail flowfiles with too many arguments
[ https://issues.apache.org/jira/browse/MINIFICPP-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-2001: Labels: beginner hacktoberfest starter (was: ) > In SQL processors should fail flowfiles with too many arguments > --- > > Key: MINIFICPP-2001 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2001 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Adam Debreceni >Priority: Major > Labels: beginner, hacktoberfest, starter > > Currently the used libraries soci, odbc do not give feedback when more > arguments are provided than required by the query, e.g. query = `INSERT INTO > test VALUES (?);` args = ['1', '2'], we should give either a warning or fail > such flowfiles. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-2001) In SQL processors should fail flowfiles with too many arguments
[ https://issues.apache.org/jira/browse/MINIFICPP-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-2001: Summary: In SQL processors should fail flowfiles with too many arguments (was: In SQL processors fail flowfiles with more than required arguments) > In SQL processors should fail flowfiles with too many arguments > --- > > Key: MINIFICPP-2001 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2001 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Adam Debreceni >Priority: Major > > Currently the used libraries soci, odbc do not give feedback when more > arguments are provided than required by the query, e.g. query = `INSERT INTO > test VALUES (?);` args = ['1', '2'], we should give either a warning or fail > such flowfiles. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] MikeThomsen commented on a diff in pull request #7677: NIFI-7355: add bytecode submission to tinkerpop service
MikeThomsen commented on code in PR #7677:
URL: https://github.com/apache/nifi/pull/7677#discussion_r1338688499
##
nifi-nar-bundles/nifi-graph-bundle/nifi-other-graph-services/src/main/java/org/apache/nifi/graph/TinkerpopClientService.java:
##
@@ -0,0 +1,525 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.graph;
+
+import groovy.lang.Binding;
+import groovy.lang.GroovyClassLoader;
+import groovy.lang.GroovyShell;
+import groovy.lang.Script;
+import io.netty.handler.ssl.ClientAuth;
+import io.netty.handler.ssl.JdkSslContext;
+import org.apache.commons.codec.digest.DigestUtils;
+import org.apache.nifi.annotation.behavior.RequiresInstanceClassLoading;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnDisabled;
+import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.components.resource.ResourceCardinality;
+import org.apache.nifi.components.resource.ResourceType;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.graph.gremlin.SimpleEntry;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.graph.ExecuteGraphQueryRecord;
+import org.apache.nifi.reporting.InitializationException;
+import org.apache.nifi.ssl.SSLContextService;
+import org.apache.nifi.util.StringUtils;
+import org.apache.nifi.util.file.classloader.ClassLoaderUtils;
+import org.apache.tinkerpop.gremlin.driver.Client;
+import org.apache.tinkerpop.gremlin.driver.Cluster;
+import org.apache.tinkerpop.gremlin.driver.Result;
+import org.apache.tinkerpop.gremlin.driver.remote.DriverRemoteConnection;
+import org.apache.tinkerpop.gremlin.process.traversal.AnonymousTraversalSource;
+import
org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+
+@Tags({"graph", "gremlin"})
+@CapabilityDescription("This service interacts with a tinkerpop-compliant
graph service, providing both script submission and bytecode submission
capabilities. " +
+"Script submission is the default, with the script command being sent
to the gremlin server as text. This should only be used for simple interactions
with a tinkerpop-compliant server " +
+"such as counts or other operations that do not require the injection
of custom classed. " +
+"Bytecode submission allows much more flexibility. When providing a
jar, custom serializers can be used and pre-compiled graph logic can be
utilized by groovy scripts" +
+"provided by processors such as the ExecuteGraphQueryRecord.")
+@SeeAlso({ExecuteGraphQueryRecord.class})
+@RequiresInstanceClassLoading
+public class TinkerpopClientService extends AbstractControllerService
implements GraphClientService {
+public static final String NOT_SUPPORTED = "NOT_SUPPORTED";
+private static final AllowableValue BYTECODE_SUBMISSION = new
AllowableValue("bytecode-submission", "ByteCode Submission",
+"Groovy scripts are compiled within NiFi, with the
GraphTraversalSource injected as a variable 'g'. Effectively allowing " +
+"your logic to directly manipulates the graph without
string serialization overheard."
+);
+
+private static final AllowableValue SCRIPT_SUBMISSION = new
AllowableValue("script-submission", "Script Submission",
+
[GitHub] [nifi] marcelo225 commented on pull request #7800: Update README.md
marcelo225 commented on PR #7800: URL: https://github.com/apache/nifi/pull/7800#issuecomment-1737494434 @joewitt I created this PR to add this Nifi OIDC configuration I thought it was tricky to configure it, though. I believe it helps someone! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (MINIFICPP-1982) Fix documentation for BinFiles
[ https://issues.apache.org/jira/browse/MINIFICPP-1982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1982: Priority: Minor (was: Major) > Fix documentation for BinFiles > -- > > Key: MINIFICPP-1982 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1982 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Adam Debreceni >Priority: Minor > > BinFiles processor documentation is confusing, we should fix the meaning of > "bin", "group", "entry" and the general principle/purpose of operation. > MergeContent might also be affected, since it is a subclass of BinFiles. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1980) Enable multithreading in PutUDP
[ https://issues.apache.org/jira/browse/MINIFICPP-1980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1980: Labels: beginner hacktoberfest starter (was: ) > Enable multithreading in PutUDP > --- > > Key: MINIFICPP-1980 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1980 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Martin Zink >Priority: Trivial > Labels: beginner, hacktoberfest, starter > > There is nothing preventing PutUDP to be multi-threaded. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] marcelo225 opened a new pull request, #7800: Update README.md
marcelo225 opened a new pull request, #7800: URL: https://github.com/apache/nifi/pull/7800 Add an example how to using Nifi to connect to an OpenID server. # Summary [NIFI-0](https://issues.apache.org/jira/browse/NIFI-0) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [ ] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-0` - [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-0` ### Pull Request Formatting - [ ] Pull Request based on current revision of the `main` branch - [ ] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [ ] JDK 21 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (MINIFICPP-1958) CoreComponent should pass strings by value and move them in constructor
[ https://issues.apache.org/jira/browse/MINIFICPP-1958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz resolved MINIFICPP-1958. - Resolution: Done > CoreComponent should pass strings by value and move them in constructor > --- > > Key: MINIFICPP-1958 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1958 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Ádám Markovics >Assignee: Ádám Markovics >Priority: Minor > Time Spent: 50m > Remaining Estimate: 0h > > This way it is more efficient, preventing allocation and deallocation of > temporary strings. Also this affects subclasses. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1955) Drop Stream::seek
[ https://issues.apache.org/jira/browse/MINIFICPP-1955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1955: Summary: Drop Stream::seek (was: Revisit Stream::seek) > Drop Stream::seek > - > > Key: MINIFICPP-1955 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1955 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Adam Debreceni >Priority: Major > > Currently Stream::seek aims to set both the read and write offsets into the > stream, we should talk this through, possibly mimic seekg/seekp (also > BufferStream::seek is noop when writing) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1960) Add custom flow file allocator to track memory usage, and expose it as a metric
[ https://issues.apache.org/jira/browse/MINIFICPP-1960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1960: Priority: Minor (was: Major) > Add custom flow file allocator to track memory usage, and expose it as a > metric > --- > > Key: MINIFICPP-1960 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1960 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Marton Szasz >Priority: Minor > > Flow files are owned by shared_ptr, which erases allocator type, so it should > be possible to override the allocator with one that does some extra > accounting, without changing the whole world. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1954) Implement ConcatInputStream
[ https://issues.apache.org/jira/browse/MINIFICPP-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1954: Priority: Minor (was: Major) > Implement ConcatInputStream > --- > > Key: MINIFICPP-1954 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1954 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Adam Debreceni >Priority: Minor > > We currently have no means to read back from a stream we have appended to > when using BufferedContentSession, a ConcatInputStream could allow us to do > just that. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-1944) EncryptionTests sometimes fail
[ https://issues.apache.org/jira/browse/MINIFICPP-1944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz resolved MINIFICPP-1944. - Resolution: Cannot Reproduce > EncryptionTests sometimes fail > -- > > Key: MINIFICPP-1944 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1944 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Gábor Gyimesi >Priority: Minor > Attachments: EncryptionTests_failure_macos.log > > > EncryptionTests transiently fail in CI, see attached logs for more info -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-1937) High disk load when using MergeContent
[
https://issues.apache.org/jira/browse/MINIFICPP-1937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Szasz resolved MINIFICPP-1937.
-
Resolution: Fixed
> High disk load when using MergeContent
> --
>
> Key: MINIFICPP-1937
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1937
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
>Reporter: Adam Debreceni
>Assignee: Adam Debreceni
>Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> The rocksdb content repository backend could exhibit unacceptable disk usage
> on some flows, e.g. GenerateFlowFile\{File Size: 2kB, period: 1 s, Batch
> Size: 200} -> MergeContent\{Batch Size: 500, Bin Packing, Delimiter: ';'}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (MINIFICPP-1942) Implement MQTT 5 Request-Response pattern
[ https://issues.apache.org/jira/browse/MINIFICPP-1942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz reassigned MINIFICPP-1942: --- Assignee: (was: Ádám Markovics) > Implement MQTT 5 Request-Response pattern > - > > Key: MINIFICPP-1942 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1942 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Ádám Markovics >Priority: Minor > > Possibly as a new processor. > https://www.hivemq.com/blog/mqtt5-essentials-part9-request-response-pattern/ > onSchedule(): > - get response information from broker > onTrigger(): > - subscribe to response topic > - publish with correlation data and response topic set > - wait for response > Questions: > - what should happen to incoming flow files? replace content with response or > create new ones instead? > - how should response topic be set? concatenate to response information? > Further ideas: > - also implement for Last Will in CONNECT packet -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1928) Explore CWEL performance improvement options
[ https://issues.apache.org/jira/browse/MINIFICPP-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1928: Description: A few ideas: - Use a SAX-style parser, like expat, instead the DOM-style rapidxml, when converting to JSON - Use raw text operations instead of XML parsing when replacing double percent placeholders (e.g. "%%1842" to "Yes") - Decouple WinAPI event calls from later processing, and add multithread support for the processing steps. was: A few ideas: - Use a SAX-style parser, like expat, instead the DOM-style rapidxml, when converting to JSON - Use raw text operations instead of XML parsing when replacing double percent placeholders (e.g. "%%1842" to "Yes") - Decouple WinAPI event calls from later processing, and add multithread support for the processing steps. - Precompile the Identifier regex in onSchedule > Explore CWEL performance improvement options > > > Key: MINIFICPP-1928 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1928 > Project: Apache NiFi MiNiFi C++ > Issue Type: Epic >Reporter: Marton Szasz >Priority: Major > > A few ideas: > - Use a SAX-style parser, like expat, instead the DOM-style rapidxml, when > converting to JSON > - Use raw text operations instead of XML parsing when replacing double > percent placeholders (e.g. "%%1842" to "Yes") > - Decouple WinAPI event calls from later processing, and add multithread > support for the processing steps. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1928) Explore CWEL performance improvement options
[ https://issues.apache.org/jira/browse/MINIFICPP-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1928: Description: A few ideas: - Use raw text operations instead of XML parsing when replacing double percent placeholders (e.g. "%%1842" to "Yes") - Decouple WinAPI event calls from later processing, and add multithread support for the processing steps. was: A few ideas: - Use a SAX-style parser, like expat, instead the DOM-style rapidxml, when converting to JSON - Use raw text operations instead of XML parsing when replacing double percent placeholders (e.g. "%%1842" to "Yes") - Decouple WinAPI event calls from later processing, and add multithread support for the processing steps. > Explore CWEL performance improvement options > > > Key: MINIFICPP-1928 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1928 > Project: Apache NiFi MiNiFi C++ > Issue Type: Epic >Reporter: Marton Szasz >Priority: Major > > A few ideas: > - Use raw text operations instead of XML parsing when replacing double > percent placeholders (e.g. "%%1842" to "Yes") > - Decouple WinAPI event calls from later processing, and add multithread > support for the processing steps. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (NIFI-11172) Remove Duplicative Features for 2.0.0
[
https://issues.apache.org/jira/browse/NIFI-11172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769624#comment-17769624
]
Daniel Stieglitz edited comment on NIFI-11172 at 9/27/23 2:03 PM:
--
[~joewitt] Based on the work I am doing for NIFI-12116, I was wondering whether
NIFI wants to support both {{JoltTransformJSON}} and {{JoltTransformRecord?
}}{{JoltTransformJSON}} only works on a single document of JSON and does not
support conversion for JSON lines while {{JoltTransformRecord does.}}
was (Author: JIRAUSER294662):
[~joewitt] Based on the work I am doing for NIFI-12116, I was wondering whether
NIFI wants to support both {{JoltTransformJSON}} and {{JoltTransformRecord?
}}{{JoltTransformJSON}} only works on a single document of JSON and does not
support conversion for JSON lines while }}{{{}JoltTransformRecord
does.{}}}}}{}}}
> Remove Duplicative Features for 2.0.0
> -
>
> Key: NIFI-11172
> URL: https://issues.apache.org/jira/browse/NIFI-11172
> Project: Apache NiFi
> Issue Type: Epic
>Reporter: David Handermann
>Priority: Major
> Fix For: 2.0.0
>
>
> Duplicative features should be removed as highlighted in the [NiFi 2.0
> Release
> Goals|https://cwiki.apache.org/confluence/display/NIFI/NiFi+2.0+Release+Goals].
> Removal should include eliminating support for XML Templates and Variable
> Registry features.
> Separate tasks should be created for removing support for {{flow.xml.gz}} as
> the fallback flow configuration in absence of {{{}flow.json.gz{}}}.
> Individual feature removal efforts should be tracked separately in logical
> groups to encourage sufficient review.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (NIFI-11172) Remove Duplicative Features for 2.0.0
[
https://issues.apache.org/jira/browse/NIFI-11172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769624#comment-17769624
]
Daniel Stieglitz edited comment on NIFI-11172 at 9/27/23 2:03 PM:
--
[~joewitt] Based on the work I am doing for NIFI-12116, I was wondering whether
NIFI 2.x wants to support both {{JoltTransformJSON}} and {{JoltTransformRecord?
}}{{JoltTransformJSON}} only works on a single document of JSON and does not
support conversion for JSON lines while {{JoltTransformRecord does.}}
was (Author: JIRAUSER294662):
[~joewitt] Based on the work I am doing for NIFI-12116, I was wondering whether
NIFI wants to support both {{JoltTransformJSON}} and {{JoltTransformRecord?
}}{{JoltTransformJSON}} only works on a single document of JSON and does not
support conversion for JSON lines while {{JoltTransformRecord does.}}
> Remove Duplicative Features for 2.0.0
> -
>
> Key: NIFI-11172
> URL: https://issues.apache.org/jira/browse/NIFI-11172
> Project: Apache NiFi
> Issue Type: Epic
>Reporter: David Handermann
>Priority: Major
> Fix For: 2.0.0
>
>
> Duplicative features should be removed as highlighted in the [NiFi 2.0
> Release
> Goals|https://cwiki.apache.org/confluence/display/NIFI/NiFi+2.0+Release+Goals].
> Removal should include eliminating support for XML Templates and Variable
> Registry features.
> Separate tasks should be created for removing support for {{flow.xml.gz}} as
> the fallback flow configuration in absence of {{{}flow.json.gz{}}}.
> Individual feature removal efforts should be tracked separately in logical
> groups to encourage sufficient review.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (NIFI-11172) Remove Duplicative Features for 2.0.0
[
https://issues.apache.org/jira/browse/NIFI-11172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769624#comment-17769624
]
Daniel Stieglitz edited comment on NIFI-11172 at 9/27/23 2:02 PM:
--
[~joewitt] Based on the work I am doing for NIFI-12116, I was wondering whether
NIFI wants to support both {{JoltTransformJSON}} and {{JoltTransformRecord?
}}{{JoltTransformJSON}} only works on a single document of JSON and does not
support conversion for JSON lines while }}{{{}JoltTransformRecord
does.{}}}}}{}}}
was (Author: JIRAUSER294662):
[~joewitt] Based on the work I am doing for NIFI-12116, I was wondering whether
NIFI wants to support both {{JoltTransformJSON}} and {{JoltTransformRecord?
}}{{JoltTransformJSON}}{{ only works on a single document of JSON and does not
support conversion for JSON lines while }}{{{}JoltTransformRecord
does.{}}}{{{}{}}}
> Remove Duplicative Features for 2.0.0
> -
>
> Key: NIFI-11172
> URL: https://issues.apache.org/jira/browse/NIFI-11172
> Project: Apache NiFi
> Issue Type: Epic
>Reporter: David Handermann
>Priority: Major
> Fix For: 2.0.0
>
>
> Duplicative features should be removed as highlighted in the [NiFi 2.0
> Release
> Goals|https://cwiki.apache.org/confluence/display/NIFI/NiFi+2.0+Release+Goals].
> Removal should include eliminating support for XML Templates and Variable
> Registry features.
> Separate tasks should be created for removing support for {{flow.xml.gz}} as
> the fallback flow configuration in absence of {{{}flow.json.gz{}}}.
> Individual feature removal efforts should be tracked separately in logical
> groups to encourage sufficient review.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (NIFI-11172) Remove Duplicative Features for 2.0.0
[
https://issues.apache.org/jira/browse/NIFI-11172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17769624#comment-17769624
]
Daniel Stieglitz commented on NIFI-11172:
-
[~joewitt] Based on the work I am doing for NIFI-12116, I was wondering whether
NIFI wants to support both {{JoltTransformJSON}} and {{JoltTransformRecord?
}}{{JoltTransformJSON}}{{ only works on a single document of JSON and does not
support conversion for JSON lines while }}{{{}JoltTransformRecord
does.{}}}{{{}{}}}
> Remove Duplicative Features for 2.0.0
> -
>
> Key: NIFI-11172
> URL: https://issues.apache.org/jira/browse/NIFI-11172
> Project: Apache NiFi
> Issue Type: Epic
>Reporter: David Handermann
>Priority: Major
> Fix For: 2.0.0
>
>
> Duplicative features should be removed as highlighted in the [NiFi 2.0
> Release
> Goals|https://cwiki.apache.org/confluence/display/NIFI/NiFi+2.0+Release+Goals].
> Removal should include eliminating support for XML Templates and Variable
> Registry features.
> Separate tasks should be created for removing support for {{flow.xml.gz}} as
> the fallback flow configuration in absence of {{{}flow.json.gz{}}}.
> Individual feature removal efforts should be tracked separately in logical
> groups to encourage sufficient review.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-1892) Reduce automatic CI triggers, add manual trigger for GH actions CI runs
[ https://issues.apache.org/jira/browse/MINIFICPP-1892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz resolved MINIFICPP-1892. - Resolution: Won't Fix > Reduce automatic CI triggers, add manual trigger for GH actions CI runs > --- > > Key: MINIFICPP-1892 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1892 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Minor > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1876) Think about ways to exclude frequently changing properties, like flow url, from the C2 manifest/hash
[ https://issues.apache.org/jira/browse/MINIFICPP-1876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1876: Priority: Minor (was: Major) > Think about ways to exclude frequently changing properties, like flow url, > from the C2 manifest/hash > > > Key: MINIFICPP-1876 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1876 > Project: Apache NiFi MiNiFi C++ > Issue Type: Task >Reporter: Marton Szasz >Priority: Minor > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1847) Investigate high memory usage with high volume ListenSyslog
[ https://issues.apache.org/jira/browse/MINIFICPP-1847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1847: Description: Connections are limited to 10 MB by default IIRC, but we should make sure that this is enforced. was: ... except for those that occur with extreme configurations. Add Bug Jiras for any use case that can result in minifi using > 100MB of RAM in release builds. Connections are limited to 10 MB by default IIRC, but we should make sure that this is enforced. > Investigate high memory usage with high volume ListenSyslog > --- > > Key: MINIFICPP-1847 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1847 > Project: Apache NiFi MiNiFi C++ > Issue Type: Wish >Reporter: Marton Szasz >Priority: Major > Labels: MiNiFi-CPP-Hygiene > > > Connections are limited to 10 MB by default IIRC, but we should make sure > that this is enforced. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1847) Investigate high memory usage with high volume ListenSyslog
[ https://issues.apache.org/jira/browse/MINIFICPP-1847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1847: Summary: Investigate high memory usage with high volume ListenSyslog (was: Investigate high memory usage scenarios) > Investigate high memory usage with high volume ListenSyslog > --- > > Key: MINIFICPP-1847 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1847 > Project: Apache NiFi MiNiFi C++ > Issue Type: Wish >Reporter: Marton Szasz >Priority: Major > Labels: MiNiFi-CPP-Hygiene > > ... except for those that occur with extreme configurations. Add Bug Jiras > for any use case that can result in minifi using > 100MB of RAM in release > builds. > Connections are limited to 10 MB by default IIRC, but we should make sure > that this is enforced. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (MINIFICPP-1847) Investigate high memory usage scenarios
[ https://issues.apache.org/jira/browse/MINIFICPP-1847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17541484#comment-17541484 ] Marton Szasz edited comment on MINIFICPP-1847 at 9/27/23 1:51 PM: -- I could get GBs of memory usage with a high volume (5000 msg/sec) ListenSyslog. Ideally it should always stay below 100 MB, except if the messages are large. I was testing with short syslog messages, maybe 100 bytes each. Even if a lot of them a queued in memory, they shouldn't take up that much space. was (Author: szaszm): I could get GBs of memory usage with a high volume (5000 msg/sec) ListenSyslog. Ideally it should always stay below 100 MB, except if the messages are large. I was testing with short syslog messages, maybe 100 bytes each. Even if a lot of them a queued in memory, they should take up that much space. > Investigate high memory usage scenarios > --- > > Key: MINIFICPP-1847 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1847 > Project: Apache NiFi MiNiFi C++ > Issue Type: Wish >Reporter: Marton Szasz >Priority: Major > Labels: MiNiFi-CPP-Hygiene > > ... except for those that occur with extreme configurations. Add Bug Jiras > for any use case that can result in minifi using > 100MB of RAM in release > builds. > Connections are limited to 10 MB by default IIRC, but we should make sure > that this is enforced. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1838) Improve MQTT capabilities
[ https://issues.apache.org/jira/browse/MINIFICPP-1838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1838: Priority: Minor (was: Major) > Improve MQTT capabilities > - > > Key: MINIFICPP-1838 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1838 > Project: Apache NiFi MiNiFi C++ > Issue Type: Epic >Reporter: Ádám Markovics >Assignee: Ádám Markovics >Priority: Minor > > MiNiFi should have full MQTT client compatibility both as a publisher and a > subscriber. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (MINIFICPP-1838) Improve MQTT capabilities
[ https://issues.apache.org/jira/browse/MINIFICPP-1838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz reassigned MINIFICPP-1838: --- Assignee: (was: Ádám Markovics) > Improve MQTT capabilities > - > > Key: MINIFICPP-1838 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1838 > Project: Apache NiFi MiNiFi C++ > Issue Type: Epic >Reporter: Ádám Markovics >Priority: Minor > > MiNiFi should have full MQTT client compatibility both as a publisher and a > subscriber. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-1827) Fix secure InvokeHTTPTest
[ https://issues.apache.org/jira/browse/MINIFICPP-1827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz resolved MINIFICPP-1827. - Resolution: Done > Fix secure InvokeHTTPTest > - > > Key: MINIFICPP-1827 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1827 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Adam Debreceni >Assignee: Adam Debreceni >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Currently some *Secure tests don't actually take SSL into account and pass. > (e.g. TestInvokeHTTPPostSecure has two "Controller Services" nodes thus not > using the SSLContext) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-1822) Add alert capability
[ https://issues.apache.org/jira/browse/MINIFICPP-1822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz resolved MINIFICPP-1822. - Resolution: Done > Add alert capability > > > Key: MINIFICPP-1822 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1822 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Adam Debreceni >Assignee: Adam Debreceni >Priority: Major > Time Spent: 7h > Remaining Estimate: 0h > > An agent should be able to convey important events (full connection, expired > certificate, etc.) to the c2 agent separate from the heartbeat/response > system. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-1836) Verify server certificate in SSL test
[ https://issues.apache.org/jira/browse/MINIFICPP-1836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz resolved MINIFICPP-1836. - Resolution: Done > Verify server certificate in SSL test > - > > Key: MINIFICPP-1836 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1836 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Adam Debreceni >Assignee: Adam Debreceni >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Currently we try to test InvokeHTTP with secure connections, but we disable > peer verification making the addition of CA to the chain irrelevant and the > test always passes. We should remove this flag. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Reopened] (MINIFICPP-1827) Fix secure InvokeHTTPTest
[ https://issues.apache.org/jira/browse/MINIFICPP-1827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz reopened MINIFICPP-1827: - > Fix secure InvokeHTTPTest > - > > Key: MINIFICPP-1827 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1827 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Adam Debreceni >Assignee: Adam Debreceni >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Currently some *Secure tests don't actually take SSL into account and pass. > (e.g. TestInvokeHTTPPostSecure has two "Controller Services" nodes thus not > using the SSLContext) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-1827) Fix secure InvokeHTTPTest
[ https://issues.apache.org/jira/browse/MINIFICPP-1827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz resolved MINIFICPP-1827. - Resolution: Duplicate > Fix secure InvokeHTTPTest > - > > Key: MINIFICPP-1827 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1827 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Adam Debreceni >Assignee: Adam Debreceni >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Currently some *Secure tests don't actually take SSL into account and pass. > (e.g. TestInvokeHTTPPostSecure has two "Controller Services" nodes thus not > using the SSLContext) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1821) Improve the error message at http-curl/client/HTTPClient.cpp:387
[ https://issues.apache.org/jira/browse/MINIFICPP-1821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1821: Summary: Improve the error message at http-curl/client/HTTPClient.cpp:387 (was: Improve the error message at http-curl/client/HTTPClient.cpp:326) > Improve the error message at http-curl/client/HTTPClient.cpp:387 > > > Key: MINIFICPP-1821 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1821 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Affects Versions: 0.9.0, 0.7.0, 0.10.0, 0.11.0 >Reporter: Marton Szasz >Priority: Major > > [2022-05-04 16:58:28.615] > [org:!https://emoji.slack-edge.com/T024BEHTP/apache/82bb49e80093023f.gif!!https://emoji.slack-edge.com/T024BEHTP/nifi/dcde4542779c3a5a.png!!https://emoji.slack-edge.com/T024BEHTP/minifi/c985ef7d29a6b21e.png!:utils::HTTPClient] > [error] curl_easy_perform() failed SSL connect error on > [https://telemetrygatewaytokafka.inbound.dfx.pygwsmlz.xcu2-8y8x.dev.cldr.work:9898/data], > error code 35 > > > It's not clear that the error code and part of the error message refer to the > same thing. And we should retrieve the underlying OpenSSL error code in case > of an SSL error. -- This message was sent by Atlassian Jira (v8.20.10#820010)
