[jira] [Updated] (NIFI-10799) Enable OIDC support for NiFi and NiFi Registry Docker images
[ https://issues.apache.org/jira/browse/NIFI-10799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-10799: Issue Type: Improvement (was: New Feature) > Enable OIDC support for NiFi and NiFi Registry Docker images > > > Key: NIFI-10799 > URL: https://issues.apache.org/jira/browse/NIFI-10799 > Project: Apache NiFi > Issue Type: Improvement > Components: Docker, NiFi Registry >Reporter: Justin Rittenhouse >Assignee: Justin Rittenhouse >Priority: Major > Labels: Docker, OIDC > Fix For: 1.20.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > NiFi and the NiFi Registry both support OIDC authentication, but the Docker > images don't currently handle it. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] turcsanyip commented on a diff in pull request #6932: NIFI-11146 Proxy usage in BoxFile processors
turcsanyip commented on code in PR #6932: URL: https://github.com/apache/nifi/pull/6932#discussion_r1099405242 ## nifi-nar-bundles/nifi-box-bundle/nifi-box-services/src/main/java/org/apache/nifi/box/controllerservices/JsonConfigBasedBoxClientService.java: ## @@ -75,10 +78,13 @@ public class JsonConfigBasedBoxClientService extends AbstractControllerService i .expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY) .build(); +private static final ProxySpec[] PROXY_SPECS = {ProxySpec.HTTP_AUTH}; Review Comment: HTTP without authentication is also supported (handled in line 166). ```suggestion private static final ProxySpec[] PROXY_SPECS = {ProxySpec.HTTP, ProxySpec.HTTP_AUTH}; ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-11149) Add PutRedisRecord processor
Matt Burgess created NIFI-11149: --- Summary: Add PutRedisRecord processor Key: NIFI-11149 URL: https://issues.apache.org/jira/browse/NIFI-11149 Project: Apache NiFi Issue Type: New Feature Components: Extensions Reporter: Matt Burgess This case is to add a record enabled processor to send data into Redis. The hash should be chosen such that the field values can be retrieved later. It should use the existing RedisConnectionPool controller service -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11124) Add hadoop.file.url attribute to HDFS processors
[ https://issues.apache.org/jira/browse/NIFI-11124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-11124: --- Fix Version/s: 1.21.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Add hadoop.file.url attribute to HDFS processors > > > Key: NIFI-11124 > URL: https://issues.apache.org/jira/browse/NIFI-11124 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mark Bathori >Assignee: Mark Bathori >Priority: Major > Fix For: 1.21.0 > > Time Spent: 40m > Remaining Estimate: 0h > > HDFS processors are providing qualified path attribute to the Provenance > Reporter but it is not available on the FlowFile. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] asfgit closed pull request #6916: NIFI-11124: Add hadoop.file.url attribute to HDFS processors
asfgit closed pull request #6916: NIFI-11124: Add hadoop.file.url attribute to HDFS processors URL: https://github.com/apache/nifi/pull/6916 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-11124) Add hadoop.file.url attribute to HDFS processors
[ https://issues.apache.org/jira/browse/NIFI-11124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17685488#comment-17685488 ] ASF subversion and git services commented on NIFI-11124: Commit b375f4b1388dfb86712e7f197d88bd5580f5d978 in nifi's branch refs/heads/main from Mark Bathori [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=b375f4b138 ] NIFI-11124: Add hadoop.file.url attribute to HDFS processors This closes #6916. Signed-off-by: Peter Turcsanyi > Add hadoop.file.url attribute to HDFS processors > > > Key: NIFI-11124 > URL: https://issues.apache.org/jira/browse/NIFI-11124 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mark Bathori >Assignee: Mark Bathori >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > HDFS processors are providing qualified path attribute to the Provenance > Reporter but it is not available on the FlowFile. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-11124) Add hadoop.file.url attribute to HDFS processors
[ https://issues.apache.org/jira/browse/NIFI-11124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi updated NIFI-11124: --- Summary: Add hadoop.file.url attribute to HDFS processors (was: Add qualified path attribute to HDFS processors) > Add hadoop.file.url attribute to HDFS processors > > > Key: NIFI-11124 > URL: https://issues.apache.org/jira/browse/NIFI-11124 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mark Bathori >Assignee: Mark Bathori >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > HDFS processors are providing qualified path attribute to the Provenance > Reporter but it is not available on the FlowFile. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-11148) Race condition loading script engines in InvokeScriptedProcessor
Matt Burgess created NIFI-11148: --- Summary: Race condition loading script engines in InvokeScriptedProcessor Key: NIFI-11148 URL: https://issues.apache.org/jira/browse/NIFI-11148 Project: Apache NiFi Issue Type: Bug Reporter: Matt Burgess In the 1.20.0 release candidate I noticed this error when loading a flow with an InvokeScriptedProcessor: 2023-02-07 16:26:39,825 ERROR [NiFi Web Server-32] o.a.n.p.script.InvokeScriptedProcessor InvokeScriptedProcessor[id=2bbc053d-8b08-3206-7c47-cc00a08beb64] Error adding script engine Groovy 2023-02-07 16:26:39,826 ERROR [NiFi Web Server-32] o.a.n.p.script.InvokeScriptedProcessor InvokeScriptedProcessor[id=2bbc053d-8b08-3206-7c47-cc00a08beb64] Unable to load script: No script runner available org.apache.nifi.processor.exception.ProcessException: No script runner available at org.apache.nifi.processors.script.InvokeScriptedProcessor.reloadScript(InvokeScriptedProcessor.java:371) at org.apache.nifi.processors.script.InvokeScriptedProcessor.reloadScriptBody(InvokeScriptedProcessor.java:326) at org.apache.nifi.processors.script.InvokeScriptedProcessor.setup(InvokeScriptedProcessor.java:230) at org.apache.nifi.processors.script.InvokeScriptedProcessor.onConfigurationRestored(InvokeScriptedProcessor.java:222) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:145) at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:133) at org.apache.nifi.util.ReflectionUtils.quietlyInvokeMethodsWithAnnotations(ReflectionUtils.java:316) at org.apache.nifi.util.ReflectionUtils.quietlyInvokeMethodsWithAnnotation(ReflectionUtils.java:93) at org.apache.nifi.controller.StandardProcessorNode.onConfigurationRestored(StandardProcessorNode.java:2115) at org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.addProcessor(StandardVersionedComponentSynchronizer.java:2417) at org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronizeProcessors(StandardVersionedComponentSynchronizer.java:932) at org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:422) at org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.lambda$synchronize$0(StandardVersionedComponentSynchronizer.java:260) at org.apache.nifi.controller.flow.AbstractFlowManager.withParameterContextResolution(AbstractFlowManager.java:550) at org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:255) at org.apache.nifi.groups.StandardProcessGroup.synchronizeFlow(StandardProcessGroup.java:3972) at org.apache.nifi.groups.StandardProcessGroup.updateFlow(StandardProcessGroup.java:3952) at org.apache.nifi.web.dao.impl.StandardProcessGroupDAO.updateProcessGroupFlow(StandardProcessGroupDAO.java:435) at org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$FastClassBySpringCGLIB$$10a99b47.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89) at org.apache.nifi.audit.ProcessGroupAuditor.updateProcessGroupFlowAdvice(ProcessGroupAuditor.java:308) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624)
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1494: MINIFICPP-2035 - NiFi flow json configuration format support
lordgamez commented on code in PR #1494: URL: https://github.com/apache/nifi-minifi-cpp/pull/1494#discussion_r1099291974 ## libminifi/include/core/flow/FlowSchema.h: ## @@ -0,0 +1,88 @@ +/** + * + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#pragma once + +#include +#include + +namespace org::apache::nifi::minifi::core::flow { + +struct FlowSchema { + using Keys = std::vector; + + Keys FLOW_HEADER; Review Comment: I see, in that case I would not use full caps for the member names. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (NIFI-11147) Allow QuerySalesforceObject to query all existing fields
[ https://issues.apache.org/jira/browse/NIFI-11147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess reassigned NIFI-11147: --- Assignee: Matt Burgess > Allow QuerySalesforceObject to query all existing fields > > > Key: NIFI-11147 > URL: https://issues.apache.org/jira/browse/NIFI-11147 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > > Currently the Field Names property of QuerySalesforceObject is required and > must contain the names of the fields the user wants to return. However in a > schema drift use case, the user may want to add a field to a Salesforce > object and have the NiFi flow continue without needing alteration. > This Jira is to make it possible for QuerySalesforceObject to return all > fields from an object. A suggestion is to make Field Names optional and if it > is not set, all fields are queried. The documentation should be updated to > match the behavior. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-11147) Allow QuerySalesforceObject to query all existing fields
[ https://issues.apache.org/jira/browse/NIFI-11147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess reassigned NIFI-11147: --- Assignee: Lehel Boér (was: Matt Burgess) > Allow QuerySalesforceObject to query all existing fields > > > Key: NIFI-11147 > URL: https://issues.apache.org/jira/browse/NIFI-11147 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Matt Burgess >Assignee: Lehel Boér >Priority: Major > > Currently the Field Names property of QuerySalesforceObject is required and > must contain the names of the fields the user wants to return. However in a > schema drift use case, the user may want to add a field to a Salesforce > object and have the NiFi flow continue without needing alteration. > This Jira is to make it possible for QuerySalesforceObject to return all > fields from an object. A suggestion is to make Field Names optional and if it > is not set, all fields are queried. The documentation should be updated to > match the behavior. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10285) NIFI - Cassandra 4.0 connection
[ https://issues.apache.org/jira/browse/NIFI-10285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Handermann updated NIFI-10285: Priority: Major (was: Blocker) > NIFI - Cassandra 4.0 connection > --- > > Key: NIFI-10285 > URL: https://issues.apache.org/jira/browse/NIFI-10285 > Project: Apache NiFi > Issue Type: Wish > Components: Extensions >Affects Versions: 1.16.3 >Reporter: batbileg >Priority: Major > Labels: easyfix > > Cant connect cassandra 4.0. > Nifi components can connect to cassandra 3.0 which cassandra 3.0 only works > on python 2.7(outdated version of python). > is there any way that nifi's components support cassandra 4.0? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] mattyb149 commented on a diff in pull request #6903: NIFI-11111 add option to output Elasticsearch error responses as FlowFile to PutElasticsearchJson and PutElasticsearchRecord
mattyb149 commented on code in PR #6903: URL: https://github.com/apache/nifi/pull/6903#discussion_r1099188315 ## nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-restapi-processors/src/main/java/org/apache/nifi/processors/elasticsearch/AbstractPutElasticsearch.java: ## @@ -67,13 +69,28 @@ public abstract class AbstractPutElasticsearch extends AbstractProcessor impleme .required(true) .build(); +static final PropertyDescriptor OUTPUT_ERROR_RESPONSES = new PropertyDescriptor.Builder() +.name("put-es-output-error-responses") +.displayName("Output Error Responses") +.description("If this is enabled, errors will be output to the \"error_responses\" relationship.") Review Comment: Also might be worth mentioning that the relationship is auto-terminated by default so they also need to connect that relationship to a downstream processor. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-11146) Proxy usage in BoxFile processors
[ https://issues.apache.org/jira/browse/NIFI-11146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zsihovszki Krisztina updated NIFI-11146: Status: Patch Available (was: Open) > Proxy usage in BoxFile processors > - > > Key: NIFI-11146 > URL: https://issues.apache.org/jira/browse/NIFI-11146 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Zsihovszki Krisztina >Assignee: Zsihovszki Krisztina >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Proxy support was not implemented for List/FetchBoxFile and PutBoxFile since > the password authentication did not work due to a Box SDK issue. > See [https://github.com/apache/nifi/pull/6355#discussion_r972708814] > The [latest|https://github.com/box/box-java-sdk/releases/tag/v4.0.0] > box-java-sdk release contains support for OkHttp, proxy configuration would > work for Box processors using 4.0.0 version of the Box Java client. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-11147) Allow QuerySalesforceObject to query all existing fields
Matt Burgess created NIFI-11147: --- Summary: Allow QuerySalesforceObject to query all existing fields Key: NIFI-11147 URL: https://issues.apache.org/jira/browse/NIFI-11147 Project: Apache NiFi Issue Type: Improvement Reporter: Matt Burgess Currently the Field Names property of QuerySalesforceObject is required and must contain the names of the fields the user wants to return. However in a schema drift use case, the user may want to add a field to a Salesforce object and have the NiFi flow continue without needing alteration. This Jira is to make it possible for QuerySalesforceObject to return all fields from an object. A suggestion is to make Field Names optional and if it is not set, all fields are queried. The documentation should be updated to match the behavior. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] mattyb149 commented on a diff in pull request #6903: NIFI-11111 add option to output Elasticsearch error responses as FlowFile to PutElasticsearchJson and PutElasticsearchRecord
mattyb149 commented on code in PR #6903: URL: https://github.com/apache/nifi/pull/6903#discussion_r1099052170 ## nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-restapi-processors/src/main/java/org/apache/nifi/processors/elasticsearch/AbstractPutElasticsearch.java: ## @@ -67,13 +69,28 @@ public abstract class AbstractPutElasticsearch extends AbstractProcessor impleme .required(true) .build(); +static final PropertyDescriptor OUTPUT_ERROR_RESPONSES = new PropertyDescriptor.Builder() +.name("put-es-output-error-responses") +.displayName("Output Error Responses") +.description("If this is enabled, errors will be output to the \"error_responses\" relationship.") +.allowableValues("true", "false") +.defaultValue("false") +.addValidator(StandardValidators.BOOLEAN_VALIDATOR) +.build(); + static final Relationship REL_SUCCESS = new Relationship.Builder() .name("success") .description("All flowfiles that succeed in being transferred into Elasticsearch go here. " + "Documents received by the Elasticsearch _bulk API may still result in errors on the Elasticsearch side. " + "The Elasticsearch response will need to be examined to determine whether any Document(s)/Record(s) resulted in errors.") .build(); +static final Relationship REL_ERROR_RESPONSES = new Relationship.Builder() +.name("error_responses") +.description("Elasticsearch _bulk API responses marked as \"error\" (and optionally \"not_found\") go here.") Review Comment: How are the`not_found` responses optional? I didn't see a property or config option. ## nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-restapi-processors/src/main/java/org/apache/nifi/processors/elasticsearch/AbstractPutElasticsearch.java: ## @@ -67,13 +69,28 @@ public abstract class AbstractPutElasticsearch extends AbstractProcessor impleme .required(true) .build(); +static final PropertyDescriptor OUTPUT_ERROR_RESPONSES = new PropertyDescriptor.Builder() +.name("put-es-output-error-responses") +.displayName("Output Error Responses") +.description("If this is enabled, errors will be output to the \"error_responses\" relationship.") Review Comment: Are the other relationship(s) honored when this is enabled? I presume the original behavior is maintained and this is an extra feature that can be enabled for better error-handling. If so can you add documentation here that the other relationships' behaviors remain intact? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1494: MINIFICPP-2035 - NiFi flow json configuration format support
szaszm commented on code in PR #1494: URL: https://github.com/apache/nifi-minifi-cpp/pull/1494#discussion_r1098974788 ## libminifi/include/core/flow/FlowSchema.h: ## @@ -0,0 +1,88 @@ +/** + * + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#pragma once + +#include +#include + +namespace org::apache::nifi::minifi::core::flow { + +struct FlowSchema { + using Keys = std::vector; + + Keys FLOW_HEADER; Review Comment: Const data members disable move. I believe it's better to default to mutable non-static data members. You can always make the object const. :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1494: MINIFICPP-2035 - NiFi flow json configuration format support
szaszm commented on code in PR #1494: URL: https://github.com/apache/nifi-minifi-cpp/pull/1494#discussion_r1098974788 ## libminifi/include/core/flow/FlowSchema.h: ## @@ -0,0 +1,88 @@ +/** + * + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#pragma once + +#include +#include + +namespace org::apache::nifi::minifi::core::flow { + +struct FlowSchema { + using Keys = std::vector; + + Keys FLOW_HEADER; Review Comment: Const data members disable move. I believe it's better to default to mutable non-static data members. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a diff in pull request #1494: MINIFICPP-2035 - NiFi flow json configuration format support
szaszm commented on code in PR #1494: URL: https://github.com/apache/nifi-minifi-cpp/pull/1494#discussion_r1085471221 ## extensions/http-curl/protocols/RESTSender.cpp: ## @@ -162,6 +162,7 @@ C2Payload RESTSender::sendPayload(const std::string& url, const Direction direct if (payload.getOperation() == Operation::TRANSFER) { auto read = std::make_unique(std::numeric_limits::max()); client.setReadCallback(std::move(read)); +client.setRequestHeader("Accept", "application/vnd.minifi-c2+json;version=1, text/yml"); Review Comment: Is this compatible with the NiFi C2 service? (see the issue in the comment a few lines below, https://issues.apache.org/jira/browse/NIFI-10535) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1494: MINIFICPP-2035 - NiFi flow json configuration format support
lordgamez commented on code in PR #1494: URL: https://github.com/apache/nifi-minifi-cpp/pull/1494#discussion_r1098833813 ## libminifi/include/core/flow/FlowSchema.h: ## @@ -0,0 +1,88 @@ +/** + * + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#pragma once + +#include +#include + +namespace org::apache::nifi::minifi::core::flow { + +struct FlowSchema { + using Keys = std::vector; + + Keys FLOW_HEADER; Review Comment: I think these members could be made const. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] krisztina-zsihovszki opened a new pull request, #6932: NIFI-11146 Proxy usage in BoxFile processors
krisztina-zsihovszki opened a new pull request, #6932: URL: https://github.com/apache/nifi/pull/6932 # Summary [NIFI-11146](https://issues.apache.org/jira/browse/NIFI-11146) # Tracking Please complete the following tracking steps prior to pull request creation. ### Issue Tracking - [x] [Apache NiFi Jira](https://issues.apache.org/jira/browse/NIFI) issue created ### Pull Request Tracking - [x] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-11146` - [x] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-11146` ### Pull Request Formatting - [x] Pull Request based on current revision of the `main` branch - [x] Pull Request refers to a feature branch with one commit containing changes # Verification Please indicate the verification steps performed prior to pull request creation. ### Build - [ ] Build completed using `mvn clean install -P contrib-check` - [x] JDK 8 - [x] JDK 11 - [x] JDK 17 ### Licensing - [ ] New dependencies are compatible with the [Apache License 2.0](https://apache.org/licenses/LICENSE-2.0) according to the [License Policy](https://www.apache.org/legal/resolved.html) - [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files ### Documentation - [ ] Documentation formatting appears as expected in rendered files -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (MINIFICPP-1997) FileSystemRepositoryTests transiently fails
[ https://issues.apache.org/jira/browse/MINIFICPP-1997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gábor Gyimesi reassigned MINIFICPP-1997: Assignee: Gábor Gyimesi > FileSystemRepositoryTests transiently fails > --- > > Key: MINIFICPP-1997 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1997 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Gábor Gyimesi >Assignee: Gábor Gyimesi >Priority: Minor > Attachments: FileSystemRepositoryTests_failure.log > > > Test failed in run > [https://github.com/apache/nifi-minifi-cpp/actions/runs/3541637140/jobs/5946147018] > See attached log of failure. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi] mark-bathori commented on pull request #6916: NIFI-11124: Add qualified path attribute to HDFS processors
mark-bathori commented on PR #6916: URL: https://github.com/apache/nifi/pull/6916#issuecomment-1420863727 @turcsanyip Thanks for the review. I’ve renamed the qualified path attribute to `hadoop.file.url` and pushed a new commit with the changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink opened a new pull request, #1504: MINIFICPP-1961 Python scripting compatibility with multiple python minor versions
martinzink opened a new pull request, #1504: URL: https://github.com/apache/nifi-minifi-cpp/pull/1504 The bulk of this python rewrite was done by @dam4rus. I've took over because he got other responsibilities that made it unfeasable to finish the PR. --- This change replaces the previous Python extension implementation (based on PyBind11) with a pure Python C API implementation (with the limited stable API) https://docs.python.org/3/c-api/stable.html I've included a new ci workflow that uses the centos tar.gz and verifies this new extension works on a large variety of platforms with large variety of python versions, but I don't think we want to run these on every commit. (Maybe on every release?) I've also separated the current scripting extensions into lua/python extensions so it will be possible to build them together, and still use them without needing the other one's dependencies. --- Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically main)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the LICENSE file? - [x] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (MINIFICPP-1840) Add support for MQTT 5
[ https://issues.apache.org/jira/browse/MINIFICPP-1840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits resolved MINIFICPP-1840. --- Resolution: Fixed > Add support for MQTT 5 > -- > > Key: MINIFICPP-1840 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1840 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Ádám Markovics >Assignee: Ádám Markovics >Priority: Major > Fix For: 0.14.0 > > Time Spent: 17h 10m > Remaining Estimate: 0h > > Current supported version is the most widespread one, MQTT 3.1.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (MINIFICPP-1840) Add support for MQTT 5
[ https://issues.apache.org/jira/browse/MINIFICPP-1840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits updated MINIFICPP-1840: -- Fix Version/s: 0.14.0 > Add support for MQTT 5 > -- > > Key: MINIFICPP-1840 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1840 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Ádám Markovics >Assignee: Ádám Markovics >Priority: Major > Fix For: 0.14.0 > > Time Spent: 17h 10m > Remaining Estimate: 0h > > Current supported version is the most widespread one, MQTT 3.1.1 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (MINIFICPP-2029) Refactor the MQTT processors to better encapsulate dependence on the MQTT version
[ https://issues.apache.org/jira/browse/MINIFICPP-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ferenc Gerlits reassigned MINIFICPP-2029: - Assignee: (was: Ferenc Gerlits) > Refactor the MQTT processors to better encapsulate dependence on the MQTT > version > - > > Key: MINIFICPP-2029 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2029 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Ferenc Gerlits >Priority: Minor > > (from > [https://github.com/apache/nifi-minifi-cpp/pull/1432#discussion_r1031680118] > by [~lordgamez]) > [In the MQTT processors,] we rely too much on the mqtt_version_ and a lot of > the function implementations are if-else branches depending on this version. > A cleaner solution would be to collect these differing functionalities, > create a separate abstract class for these functions with two separate > inherited class implementations for the two versions. After we have the > version in {{onSchedule}} we can instantiate the proper one and call the > proper implementation in each case where it differs (strategy pattern). > If needed the common parts can also be moved to the base class and only the > differences implemented in the inherited classes (template method pattern). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (MINIFICPP-2016) Add session commit time metrics
[ https://issues.apache.org/jira/browse/MINIFICPP-2016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gábor Gyimesi resolved MINIFICPP-2016. -- Fix Version/s: 0.14.0 Resolution: Fixed > Add session commit time metrics > --- > > Key: MINIFICPP-2016 > URL: https://issues.apache.org/jira/browse/MINIFICPP-2016 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Gábor Gyimesi >Assignee: Gábor Gyimesi >Priority: Minor > Fix For: 0.14.0 > > Time Spent: 50m > Remaining Estimate: 0h > > Currently the last and the average time of the onTrigger calls is collected > which only consists of the runtime of the processor's onTrigger function > without the session.commit call. We should add the last and average runtime > of the session.commit call as well to the processor metrics. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1483: MINIFICPP-2019 Fix auto-generation of PROCESSORS.md
lordgamez commented on code in PR #1483: URL: https://github.com/apache/nifi-minifi-cpp/pull/1483#discussion_r1098616074 ## extensions/splunk/PutSplunkHTTP.h: ## @@ -40,7 +40,23 @@ class PutSplunkHTTP final : public SplunkHECProcessor { PutSplunkHTTP& operator=(PutSplunkHTTP&&) = delete; ~PutSplunkHTTP() override = default; - EXTENSIONAPI static constexpr const char* Description = "Sends the flow file contents to the specified Splunk HTTP Event Collector over HTTP or HTTPS. Supports HEC Index Acknowledgement."; + EXTENSIONAPI static constexpr const char* Description = + "Sends the flow file contents to the specified Splunk HTTP Event Collector (see https://docs.splunk.com/Documentation/SplunkCloud/latest/Data/UsetheHTTPEventCollector) over HTTP or HTTPS.\n" + "\n" + "The \"Source\", \"Source Type\", \"Host\" and \"Index\" properties are optional and will be set by Splunk if unspecified. If set,\n" Review Comment: Are the `\n` endline characters necessary to wrap the descriptions? Isn't the wrapping done automatically in the markdown files? ## extensions/aws/processors/PutS3Object.h: ## @@ -44,7 +44,10 @@ class PutS3Object : public S3Processor { static const std::set STORAGE_CLASSES; static const std::set SERVER_SIDE_ENCRYPTIONS; - EXTENSIONAPI static constexpr const char* Description = "This Processor puts FlowFiles to an Amazon S3 Bucket."; + EXTENSIONAPI static constexpr const char* Description = "Puts FlowFiles to an Amazon S3 Bucket. The upload uses either the PutS3Object method. " Review Comment: ```suggestion EXTENSIONAPI static constexpr const char* Description = "Puts FlowFiles to an Amazon S3 Bucket. The upload uses the PutS3Object method. " ``` ## minifi_main/TableFormatter.h: ## @@ -0,0 +1,47 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#pragma once + +#include +#include +#include + +#include "core/PropertyValue.h" + +namespace org::apache::nifi::minifi::docs { + +class Table { + public: + explicit Table(std::vector header) : header_{std::move(header)} {} + void addRow(std::vector row); + [[nodiscard]] std::string toString() const; + + private: + [[nodiscard]] std::vector find_widths() const; Review Comment: This should be camelCase ## minifi_main/TableFormatter.h: ## @@ -0,0 +1,47 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#pragma once + +#include +#include +#include + +#include "core/PropertyValue.h" + +namespace org::apache::nifi::minifi::docs { + +class Table { + public: + explicit Table(std::vector header) : header_{std::move(header)} {} + void addRow(std::vector row); + [[nodiscard]] std::string toString() const; + + private: + [[nodiscard]] std::vector find_widths() const; + + std::vector header_; + std::vector> rows_; +}; + +std::string formatName(const std::string& name, bool is_required); +std::string formatAllowableValues(const std::vector& values); +std::string formatDescription(std::string description, bool supports_expression_language = false); +std::string formatSeparator(const std::vector& widths); +std::string formatRow(const std::vector& items, const std::vector& widths); Review Comment: This might benefit from a few unit tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci
[GitHub] [nifi-minifi-cpp] martinzink closed pull request #1477: MINIFICPP-2016 Add session commit time metrics
martinzink closed pull request #1477: MINIFICPP-2016 Add session commit time metrics URL: https://github.com/apache/nifi-minifi-cpp/pull/1477 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink closed pull request #1432: MINIFICPP-1840 - Add support for MQTT 5
martinzink closed pull request #1432: MINIFICPP-1840 - Add support for MQTT 5 URL: https://github.com/apache/nifi-minifi-cpp/pull/1432 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink closed pull request #1498: MINIFICPP-2020 Improve error messages when MiNiFi is not able to start
martinzink closed pull request #1498: MINIFICPP-2020 Improve error messages when MiNiFi is not able to start URL: https://github.com/apache/nifi-minifi-cpp/pull/1498 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink closed pull request #1466: MINIFICPP-1994 - Route flowfiles with invalid SQL to Failure
martinzink closed pull request #1466: MINIFICPP-1994 - Route flowfiles with invalid SQL to Failure URL: https://github.com/apache/nifi-minifi-cpp/pull/1466 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] Lehel44 commented on a diff in pull request #6853: NIFI-11045: Sensitive dynamic property support for parameterized quer…
Lehel44 commented on code in PR #6853: URL: https://github.com/apache/nifi/pull/6853#discussion_r1098077362 ## nifi-nar-bundles/nifi-extension-utils/nifi-database-utils/src/main/java/org/apache/nifi/util/db/JdbcCommon.java: ## @@ -711,18 +711,59 @@ public static void setParameters(final PreparedStatement stmt, final Map attributes) throws SQLException { Review Comment: - According to @turcsanyip's remarks, since logging occurs only in the setParameter method if the flowfile attribute is "sql.arguments.*type," and - @exceptionfactory - filtering out the type is not needed we may revert to the old behavior. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez opened a new pull request, #1503: MINIFICPP-2039 Dust off minificontroller
lordgamez opened a new pull request, #1503: URL: https://github.com/apache/nifi-minifi-cpp/pull/1503 https://issues.apache.org/jira/browse/MINIFICPP-2039 -- Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically main)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1485: MINIFICPP-2024 Refactor repository hierarchy
lordgamez commented on code in PR #1485: URL: https://github.com/apache/nifi-minifi-cpp/pull/1485#discussion_r1098497944 ## libminifi/src/core/repository/VolatileRepository.cpp: ## @@ -23,16 +23,141 @@ #include #include "FlowFileRecord.h" -namespace org { -namespace apache { -namespace nifi { -namespace minifi { -namespace core { -namespace repository { - -} /* namespace repository */ -} /* namespace core */ -} /* namespace minifi */ -} /* namespace nifi */ -} /* namespace apache */ -} /* namespace org */ +namespace org::apache::nifi::minifi::core::repository { + +void VolatileRepository::loadComponent(const std::shared_ptr& /*content_repo*/) { +} + +VolatileRepository::~VolatileRepository() { +} Review Comment: Good point, removed in 688c4a816f3d4ecafa721771f8fa2452d9b4eb08 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (NIFI-11146) Proxy usage in BoxFile processors
[ https://issues.apache.org/jira/browse/NIFI-11146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zsihovszki Krisztina reassigned NIFI-11146: --- Assignee: Zsihovszki Krisztina > Proxy usage in BoxFile processors > - > > Key: NIFI-11146 > URL: https://issues.apache.org/jira/browse/NIFI-11146 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Zsihovszki Krisztina >Assignee: Zsihovszki Krisztina >Priority: Major > > Proxy support was not implemented for List/FetchBoxFile and PutBoxFile since > the password authentication did not work due to a Box SDK issue. > See [https://github.com/apache/nifi/pull/6355#discussion_r972708814] > The [latest|https://github.com/box/box-java-sdk/releases/tag/v4.0.0] > box-java-sdk release contains support for OkHttp, proxy configuration would > work for Box processors using 4.0.0 version of the Box Java client. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-11146) Proxy usage in BoxFile processors
Zsihovszki Krisztina created NIFI-11146: --- Summary: Proxy usage in BoxFile processors Key: NIFI-11146 URL: https://issues.apache.org/jira/browse/NIFI-11146 Project: Apache NiFi Issue Type: Improvement Reporter: Zsihovszki Krisztina Proxy support was not implemented for List/FetchBoxFile and PutBoxFile since the password authentication did not work due to a Box SDK issue. See [https://github.com/apache/nifi/pull/6355#discussion_r972708814] The [latest|https://github.com/box/box-java-sdk/releases/tag/v4.0.0] box-java-sdk release contains support for OkHttp, proxy configuration would work for Box processors using 4.0.0 version of the Box Java client. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] fgerlits commented on a diff in pull request #1485: MINIFICPP-2024 Refactor repository hierarchy
fgerlits commented on code in PR #1485: URL: https://github.com/apache/nifi-minifi-cpp/pull/1485#discussion_r1098475395 ## libminifi/src/core/repository/VolatileRepository.cpp: ## @@ -23,16 +23,141 @@ #include #include "FlowFileRecord.h" -namespace org { -namespace apache { -namespace nifi { -namespace minifi { -namespace core { -namespace repository { - -} /* namespace repository */ -} /* namespace core */ -} /* namespace minifi */ -} /* namespace nifi */ -} /* namespace apache */ -} /* namespace org */ +namespace org::apache::nifi::minifi::core::repository { + +void VolatileRepository::loadComponent(const std::shared_ptr& /*content_repo*/) { +} + +VolatileRepository::~VolatileRepository() { +} Review Comment: this could be removed now -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1485: MINIFICPP-2024 Refactor repository hierarchy
lordgamez commented on code in PR #1485: URL: https://github.com/apache/nifi-minifi-cpp/pull/1485#discussion_r1098471269 ## libminifi/src/core/repository/VolatileRepository.cpp: ## @@ -23,16 +23,144 @@ #include #include "FlowFileRecord.h" -namespace org { -namespace apache { -namespace nifi { -namespace minifi { -namespace core { -namespace repository { - -} /* namespace repository */ -} /* namespace core */ -} /* namespace minifi */ -} /* namespace nifi */ -} /* namespace apache */ -} /* namespace org */ +namespace org::apache::nifi::minifi::core::repository { + +void VolatileRepository::loadComponent(const std::shared_ptr& /*content_repo*/) { +} + +VolatileRepository::~VolatileRepository() { + for (auto ent : repo_data_.value_vector) { +delete ent; + } +} Review Comment: Updated in 79d5a6e5d2d061953ebc87b82d8cc10da6bf10d6 ## libminifi/src/core/repository/VolatileContentRepository.cpp: ## @@ -45,10 +48,10 @@ bool VolatileContentRepository::initialize(const std::shared_ptr &con } } if (!minimize_locking_) { -for (auto ent : value_vector_) { +for (auto ent : repo_data_.value_vector) { delete ent; } -value_vector_.clear(); +repo_data_.value_vector.clear(); Review Comment: Updated in 79d5a6e5d2d061953ebc87b82d8cc10da6bf10d6 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1485: MINIFICPP-2024 Refactor repository hierarchy
lordgamez commented on code in PR #1485: URL: https://github.com/apache/nifi-minifi-cpp/pull/1485#discussion_r1098470947 ## libminifi/include/core/repository/FileSystemRepository.h: ## @@ -22,39 +22,31 @@ #include #include -#include "core/Core.h" #include "../ContentRepository.h" #include "properties/Configure.h" #include "core/logging/LoggerFactory.h" namespace org::apache::nifi::minifi::core::repository { -/** - * FileSystemRepository is a content repository that stores data onto the local file system. - */ Review Comment: It's true that it is not obvious from the class name, but the definition of the class states that it is inherited from the `ContentRepository` which I think makes it clear that it is a content repository, so I don't think this comment is needed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1485: MINIFICPP-2024 Refactor repository hierarchy
lordgamez commented on code in PR #1485: URL: https://github.com/apache/nifi-minifi-cpp/pull/1485#discussion_r1098469358 ## libminifi/src/core/repository/VolatileContentRepository.cpp: ## @@ -86,7 +89,7 @@ std::shared_ptr VolatileContentRepository::write(const minifi::R if (claim_check != master_list_.end()) { return std::make_shared>(claim.getContentFullPath(), claim_check->second); } else { - auto *ent = new AtomicEntry(¤t_size_, &max_size_); + auto *ent = new AtomicEntry(&repo_data_.current_size, &repo_data_.max_size); Review Comment: I'm not sure if it's a good idea as it is stored in the `master_list_` as a raw pointer where the ownership is not trivial. We have the `value_vector` of the repo data and the `master_list_` and the ownership between those 2 needs to be clarified first to see how the ownership should be handled. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-11125) AbstracDBCPConnectionPool unset property handling
[ https://issues.apache.org/jira/browse/NIFI-11125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lehel Boér updated NIFI-11125: -- Status: Open (was: Patch Available) > AbstracDBCPConnectionPool unset property handling > - > > Key: NIFI-11125 > URL: https://issues.apache.org/jira/browse/NIFI-11125 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Lehel Boér >Assignee: Lehel Boér >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > AbstractDBCPConnectionPool currently does not accept null property values, > which can occur if a child class does not utilize an inherited one that is > subsequently processed. We should add additional logic to > "configureDatasource" checking unset properties before durations are > extracted. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-10836) Support Receiving RFC 3195 Syslog Messages
[ https://issues.apache.org/jira/browse/NIFI-10836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lehel Boér resolved NIFI-10836. --- Resolution: Won't Fix > Support Receiving RFC 3195 Syslog Messages > -- > > Key: NIFI-10836 > URL: https://issues.apache.org/jira/browse/NIFI-10836 > Project: Apache NiFi > Issue Type: Improvement > Components: MiNiFi >Reporter: CHANDAN KUMAR >Assignee: Lehel Boér >Priority: Major > > [RFC 3195|https://www.rfc-editor.org/rfc/rfc3195] defines a reliable delivery > format for syslog messages. The {{ListenTCP}} and {{ListenSyslog}} Processors > do not work with this format because messages span multiple lines and both > processors expect messages to be terminated by a single newline. A new > processor could be created to support handling RFC 3195 messages. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-9327) QueryNiFi reporting task does not keep track of state
[ https://issues.apache.org/jira/browse/NIFI-9327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lehel Boér updated NIFI-9327: - Resolution: Fixed Status: Resolved (was: Patch Available) > QueryNiFi reporting task does not keep track of state > - > > Key: NIFI-9327 > URL: https://issues.apache.org/jira/browse/NIFI-9327 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Lehel Boér >Assignee: Lehel Boér >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > > The QueryNiFi reporting task does not keep track of state which is > problematic. If I've configured my reporting task to run every minute with: > {code:java} > SELECT * FROM BULLETINS > {code} > Since the bulletins are available for 5 minutes in NiFi, I'll have up to 5 > occurrences of each bulletin sent to whatever sink is configured. > Some kind of state should be tracked for both provenance data and bulletins. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-9447) Fix SNMP test failures related to available UDP ports
[ https://issues.apache.org/jira/browse/NIFI-9447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lehel Boér resolved NIFI-9447. -- Resolution: Fixed > Fix SNMP test failures related to available UDP ports > - > > Key: NIFI-9447 > URL: https://issues.apache.org/jira/browse/NIFI-9447 > Project: Apache NiFi > Issue Type: Bug >Reporter: Lehel Boér >Assignee: Lehel Boér >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > > There are SNMP test failure on the pipeline due to unavailable ports. > NetworkUtils::availablePort is deprecated, the test should look for an > available UDP port. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1490: MINIFICPP-2022 Add valid repository size metrics for all repositories
lordgamez commented on code in PR #1490: URL: https://github.com/apache/nifi-minifi-cpp/pull/1490#discussion_r1098413827 ## extensions/rocksdb-repos/database/OpenRocksDb.cpp: ## @@ -118,8 +114,18 @@ rocksdb::DB* OpenRocksDb::get() { return impl_.get(); } -} // namespace internal -} // namespace minifi -} // namespace nifi -} // namespace apache -} // namespace org +std::optional OpenRocksDb::getApproximateSizes() const { + rocksdb::SizeApproximationOptions options; + options.include_memtabtles = true; + std::array ranges; + ranges[0].start = ""; + ranges[0].limit = "~"; + uint64_t value = 0; + auto status = impl_->GetApproximateSizes(options, column_->handle.get(), ranges.data(), 1, &value); Review Comment: Updated in 38e9c97ff6fbcfb6479eec3615c025d076d1274b. Yes `include_memtabtles` us a typo in the rocksdb API :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1465: MINIFICPP-1975 Volatile and persistent combination of repositories should be avoided
lordgamez commented on code in PR #1465: URL: https://github.com/apache/nifi-minifi-cpp/pull/1465#discussion_r1098388567 ## controller/Controller.h: ## @@ -259,13 +261,16 @@ std::shared_ptr if (!flow_repo) { throw org::apache::nifi::minifi::Exception(org::apache::nifi::minifi::REPOSITORY_EXCEPTION, "Could not create flowfile repository"); } - flow_repo->initialize(configuration); - configuration->get(org::apache::nifi::minifi::Configure::nifi_content_repository_class_name, content_repo_class); const std::shared_ptr content_repo = org::apache::nifi::minifi::core::createContentRepository(content_repo_class, true, "content"); + const bool is_flow_repo_non_persistent = flow_repo->isNoop() || std::dynamic_pointer_cast(flow_repo) != nullptr; + const bool is_content_repo_non_persistent = std::dynamic_pointer_cast(content_repo) != nullptr; Review Comment: Reverted to using dynamic_cast -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1482: MINIFICPP-2009 - CWEL should add resolved attributes with json output…
martinzink commented on code in PR #1482: URL: https://github.com/apache/nifi-minifi-cpp/pull/1482#discussion_r1098372406 ## extensions/windows-event-log/tests/ConsumeWindowsEventLogTests.cpp: ## @@ -238,6 +238,18 @@ TEST_CASE("ConsumeWindowsEventLog extracts some attributes by default", "[onTrig auto logger_processor = test_plan->addProcessor("LogAttribute", "logger", Success, true); test_plan->setProperty(logger_processor, LogAttribute::FlowFilesToLog.getName(), "0"); + SECTION("XML output") { +CHECK(test_plan->setProperty(cwel_processor, ConsumeWindowsEventLog ::OutputFormat.getName(), "XML")); Review Comment: You are right. I've switched to `REQUIRE` in https://github.com/apache/nifi-minifi-cpp/pull/1482/commits/efb169ad2b661a82f1608e80031dfcb7672eab06#diff-a2d991cd055149f79da7e62be4848113893ed6fc7104d9972c97cb671d4de7ceR242 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1482: MINIFICPP-2009 - CWEL should add resolved attributes with json output…
martinzink commented on code in PR #1482: URL: https://github.com/apache/nifi-minifi-cpp/pull/1482#discussion_r1098371820 ## extensions/windows-event-log/tests/ConsumeWindowsEventLogTests.cpp: ## @@ -238,6 +238,18 @@ TEST_CASE("ConsumeWindowsEventLog extracts some attributes by default", "[onTrig auto logger_processor = test_plan->addProcessor("LogAttribute", "logger", Success, true); test_plan->setProperty(logger_processor, LogAttribute::FlowFilesToLog.getName(), "0"); + SECTION("XML output") { +CHECK(test_plan->setProperty(cwel_processor, ConsumeWindowsEventLog ::OutputFormat.getName(), "XML")); Review Comment: agreed, don't why that was there, fixed it in https://github.com/apache/nifi-minifi-cpp/pull/1482/commits/efb169ad2b661a82f1608e80031dfcb7672eab06#diff-a2d991cd055149f79da7e62be4848113893ed6fc7104d9972c97cb671d4de7ceR242 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] martinzink commented on a diff in pull request #1482: MINIFICPP-2009 - CWEL should add resolved attributes with json output…
martinzink commented on code in PR #1482: URL: https://github.com/apache/nifi-minifi-cpp/pull/1482#discussion_r1098371050 ## extensions/windows-event-log/ConsumeWindowsEventLog.cpp: ## @@ -699,42 +699,38 @@ void ConsumeWindowsEventLog::refreshTimeZoneData() { } void ConsumeWindowsEventLog::putEventRenderFlowFileToSession(const EventRender& eventRender, core::ProcessSession& session) const { - auto commitFlowFile = [&] (const std::shared_ptr& flowFile, const std::string& content, const std::string& mimeType) { -session.writeBuffer(flowFile, content); -session.putAttribute(flowFile, core::SpecialFlowAttribute::MIME_TYPE, mimeType); -session.putAttribute(flowFile, "timezone.name", timezone_name_); -session.putAttribute(flowFile, "timezone.offset", timezone_offset_); -session.getProvenanceReporter()->receive(flowFile, provenanceUri_, getUUIDStr(), "Consume windows event logs", 0ms); -session.transfer(flowFile, Success); + auto commitFlowFile = [&] (const std::string& content, const std::string& mimeType) { +auto flow_file = session.create(); +addMatchedFieldsAsAttributes(eventRender, session, flow_file); +session.writeBuffer(flow_file, content); +session.putAttribute(flow_file, core::SpecialFlowAttribute::MIME_TYPE, mimeType); +session.putAttribute(flow_file, "timezone.name", timezone_name_); +session.putAttribute(flow_file, "timezone.offset", timezone_offset_); +session.getProvenanceReporter()->receive(flow_file, provenanceUri_, getUUIDStr(), "Consume windows event logs", 0ms); +session.transfer(flow_file, Success); }; if (output_.xml) { -auto flowFile = session.create(); logger_->log_trace("Writing rendered XML to a flow file"); - -for (const auto &fieldMapping : eventRender.matched_fields) { - if (!fieldMapping.second.empty()) { -session.putAttribute(flowFile, fieldMapping.first, fieldMapping.second); - } -} - -commitFlowFile(flowFile, eventRender.xml, "application/xml"); +commitFlowFile(eventRender.xml, "application/xml"); } if (output_.plaintext) { logger_->log_trace("Writing rendered plain text to a flow file"); -commitFlowFile(session.create(), eventRender.plaintext, "text/plain"); +commitFlowFile(eventRender.plaintext, "text/plain"); } - if (output_.json.type == JSONType::Raw) { -logger_->log_trace("Writing rendered raw JSON to a flow file"); -commitFlowFile(session.create(), eventRender.json, "application/json"); - } else if (output_.json.type == JSONType::Simple) { -logger_->log_trace("Writing rendered simple JSON to a flow file"); -commitFlowFile(session.create(), eventRender.json, "application/json"); - } else if (output_.json.type == JSONType::Flattened) { -logger_->log_trace("Writing rendered flattened JSON to a flow file"); -commitFlowFile(session.create(), eventRender.json, "application/json"); + if (output_.json.type != JSONType::None) { Review Comment: :+1: good idea, changed it in https://github.com/apache/nifi-minifi-cpp/pull/1482/commits/efb169ad2b661a82f1608e80031dfcb7672eab06#diff-0769252d38a0bd8b7b412439a306a061cd5da7b84419220a1aec6e100a853cecR723 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] lordgamez commented on a diff in pull request #1497: MINIFICPP-1869 Metrics-related refactoring
lordgamez commented on code in PR #1497: URL: https://github.com/apache/nifi-minifi-cpp/pull/1497#discussion_r1098364381 ## libminifi/src/FlowController.cpp: ## @@ -205,13 +177,12 @@ int16_t FlowController::stop() { * -Stopping the schedulers doesn't actually quit the onTrigger functions of processors * -They only guarantee that the processors are not scheduled anymore * -After the threadpool is stopped we can make sure that processors don't need repos and controllers anymore */ -if (this->root_) { - this->root_->drainConnections(); -} +root_wrapper_.drainConnections(); this->flow_file_repo_->stop(); this->provenance_repo_->stop(); // stop the ControllerServices disableAllControllerServices(); +initialized_ = false; Review Comment: I think the `unload` was just complicating things as even in tests either `unload` or `stop` (or both of them together) was used for the same functionality. `load` is a bit different as it also calls `stop` for the flow controller and reloads the flow controller if a new config is applied so I would keep that. I see what the original intent was, but it does not reflect the current functionality, so I would remove `unload` only. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits commented on a diff in pull request #1498: MINIFICPP-2020 Improve error messages when MiNiFi is not able to start
fgerlits commented on code in PR #1498: URL: https://github.com/apache/nifi-minifi-cpp/pull/1498#discussion_r1093407989 ## minifi_main/MiNiFiMain.cpp: ## @@ -258,7 +259,11 @@ int main(int argc, char **argv) { minifi::core::extension::ExtensionManager::get().initialize(configure); -if (argc >= 3 && std::string("docs") == argv[1]) { +if (argc >= 2 && std::string("docs") == argv[1]) { + if (argc == 2) { Review Comment: yes, good idea, done in 205ed38ec5e8677a58ba7900ff2a20409cb50ee3 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits commented on a diff in pull request #1498: MINIFICPP-2020 Improve error messages when MiNiFi is not able to start
fgerlits commented on code in PR #1498: URL: https://github.com/apache/nifi-minifi-cpp/pull/1498#discussion_r1093407232 ## minifi_main/MiNiFiWindowsService.h: ## @@ -20,10 +20,12 @@ #ifdef WIN32 #include +#include + #include "core/Core.h" void RunAsServiceIfNeeded(); -HANDLE GetTerminationEventHandle(bool* isStartedByService); +std::tuple GetTerminationEventHandle(); Review Comment: OK, done in 503afa2109daece2a98fe2450c2162ff9dd632c2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@nifi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org