[GitHub] [nifi] esecules commented on a change in pull request #4232: NIFI-7392: Initial commit for the ValidateJSON processor
esecules commented on a change in pull request #4232:
URL: https://github.com/apache/nifi/pull/4232#discussion_r456673497
##
File path:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
##
@@ -0,0 +1,206 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicReference;
+
+import com.fasterxml.jackson.core.JsonParseException;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.networknt.schema.JsonSchema;
+import com.networknt.schema.JsonSchemaFactory;
+import com.networknt.schema.ValidationMessage;
+import com.networknt.schema.SpecVersion.VersionFlag;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"JSON", "schema", "validation"})
+@WritesAttributes({
+@WritesAttribute(attribute = "validatejson.invalid.error", description =
"If the flow file is routed to the invalid relationship "
++ "the attribute will contain the error message resulting from the
validation failure.")
+})
+@CapabilityDescription("Validates the contents of FlowFiles against a
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final String ERROR_ATTRIBUTE_KEY =
"validatejson.invalid.error";
+
+public static final AllowableValue SCHEMA_VERSION_4 = new
AllowableValue("V4");
+public static final AllowableValue SCHEMA_VERSION_6 = new
AllowableValue("V6");
+public static final AllowableValue SCHEMA_VERSION_7 = new
AllowableValue("V7");
+public static final AllowableValue SCHEMA_VERSION_V201909 = new
AllowableValue("V201909");
+
+public static final PropertyDescriptor SCHEMA_VERSION = new
PropertyDescriptor
+.Builder().name("SCHEMA_VERSION")
+.displayName("Schema Version")
+.description("The JSON schema specification")
+.required(true)
+.allowableValues(SCHEMA_VERSION_4, SCHEMA_VERSION_6, SCHEMA_VERSION_7,
SCHEMA_VERSION_V201909)
+.defaultValue(SCHEMA_VERSION_V201909.getValue())
+.build();
+
+public static final PropertyDescriptor SCHEMA_TEXT = new PropertyDescriptor
+.Builder().name("SCHEMA_TEXT")
+.displayName("Schema Text")
+.description("The text of a JSON schema")
+.required(true)
+.expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final Relationship REL_VALID = new Relationship.Builder()
+.name("valid")
+
[jira] [Updated] (NIFI-7658) Lower log severity of expected exception for missing content length filter size value
[
https://issues.apache.org/jira/browse/NIFI-7658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy LoPresto updated NIFI-7658:
Description:
Every WAR loaded by NiFi now prints two INFO logs at startup due to the default
maximum web request size being unpopulated by default. This log severity should
be lowered and the message suppressed by default as it is an expected scenario
but causes concern for many users.
{code}
2020-07-17 16:15:46,429 INFO [main] org.apache.nifi.web.server.JettyServer
Can't parse valid max content length from
2020-07-17 16:15:46,429 INFO [main] org.apache.nifi.web.server.JettyServer Not
adding content-length filter because nifi.web.max.content.size is not set in
nifi.properties
{code}
A follow-on Jira should address removing the unnecessary request entirely.
was:
All unsecured instances report an INFO log at startup and on additional
requests due to the Kerberos authentication mechanism being unavailable. This
log severity should be lowered and the message suppressed by default as it is
an expected scenario but causes concern for many users.
{code}
2020-07-17 11:37:38,023 INFO [NiFi Web Server-18]
o.a.n.w.m.IllegalStateExceptionMapper java.lang.IllegalStateException: Access
tokens are only issued over HTTPS. Returning Conflict response.
{code}
A follow-on Jira should address removing the unnecessary request entirely.
> Lower log severity of expected exception for missing content length filter
> size value
> -
>
> Key: NIFI-7658
> URL: https://issues.apache.org/jira/browse/NIFI-7658
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Affects Versions: 1.11.4
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Major
> Labels: error, log, security
>
> Every WAR loaded by NiFi now prints two INFO logs at startup due to the
> default maximum web request size being unpopulated by default. This log
> severity should be lowered and the message suppressed by default as it is an
> expected scenario but causes concern for many users.
> {code}
> 2020-07-17 16:15:46,429 INFO [main] org.apache.nifi.web.server.JettyServer
> Can't parse valid max content length from
> 2020-07-17 16:15:46,429 INFO [main] org.apache.nifi.web.server.JettyServer
> Not adding content-length filter because nifi.web.max.content.size is not set
> in nifi.properties
> {code}
> A follow-on Jira should address removing the unnecessary request entirely.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (NIFI-7658) Lower log severity of expected exception for missing content length filter size value
Andy LoPresto created NIFI-7658:
---
Summary: Lower log severity of expected exception for missing
content length filter size value
Key: NIFI-7658
URL: https://issues.apache.org/jira/browse/NIFI-7658
Project: Apache NiFi
Issue Type: Improvement
Components: Core Framework
Affects Versions: 1.11.4
Reporter: Andy LoPresto
Assignee: Andy LoPresto
All unsecured instances report an INFO log at startup and on additional
requests due to the Kerberos authentication mechanism being unavailable. This
log severity should be lowered and the message suppressed by default as it is
an expected scenario but causes concern for many users.
{code}
2020-07-17 11:37:38,023 INFO [NiFi Web Server-18]
o.a.n.w.m.IllegalStateExceptionMapper java.lang.IllegalStateException: Access
tokens are only issued over HTTPS. Returning Conflict response.
{code}
A follow-on Jira should address removing the unnecessary request entirely.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] esecules commented on a change in pull request #4232: NIFI-7392: Initial commit for the ValidateJSON processor
esecules commented on a change in pull request #4232:
URL: https://github.com/apache/nifi/pull/4232#discussion_r456673497
##
File path:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateJson.java
##
@@ -0,0 +1,206 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.standard;
+
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicReference;
+
+import com.fasterxml.jackson.core.JsonParseException;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.networknt.schema.JsonSchema;
+import com.networknt.schema.JsonSchemaFactory;
+import com.networknt.schema.ValidationMessage;
+import com.networknt.schema.SpecVersion.VersionFlag;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"JSON", "schema", "validation"})
+@WritesAttributes({
+@WritesAttribute(attribute = "validatejson.invalid.error", description =
"If the flow file is routed to the invalid relationship "
++ "the attribute will contain the error message resulting from the
validation failure.")
+})
+@CapabilityDescription("Validates the contents of FlowFiles against a
user-specified JSON Schema file")
+public class ValidateJson extends AbstractProcessor {
+
+public static final String ERROR_ATTRIBUTE_KEY =
"validatejson.invalid.error";
+
+public static final AllowableValue SCHEMA_VERSION_4 = new
AllowableValue("V4");
+public static final AllowableValue SCHEMA_VERSION_6 = new
AllowableValue("V6");
+public static final AllowableValue SCHEMA_VERSION_7 = new
AllowableValue("V7");
+public static final AllowableValue SCHEMA_VERSION_V201909 = new
AllowableValue("V201909");
+
+public static final PropertyDescriptor SCHEMA_VERSION = new
PropertyDescriptor
+.Builder().name("SCHEMA_VERSION")
+.displayName("Schema Version")
+.description("The JSON schema specification")
+.required(true)
+.allowableValues(SCHEMA_VERSION_4, SCHEMA_VERSION_6, SCHEMA_VERSION_7,
SCHEMA_VERSION_V201909)
+.defaultValue(SCHEMA_VERSION_V201909.getValue())
+.build();
+
+public static final PropertyDescriptor SCHEMA_TEXT = new PropertyDescriptor
+.Builder().name("SCHEMA_TEXT")
+.displayName("Schema Text")
+.description("The text of a JSON schema")
+.required(true)
+.expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final Relationship REL_VALID = new Relationship.Builder()
+.name("valid")
+
[jira] [Resolved] (NIFI-7559) NiFi Cluster page shows multiple entries for same node identity
[ https://issues.apache.org/jira/browse/NIFI-7559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammed Nadeem resolved NIFI-7559. --- Resolution: Not A Problem This behavior is seen if Statefulset pods are not mounted to PVC (ephemeral). So, every time a pod is restarted without preserving its state. NiFi clustering would think its a new node though it same node that was part of it before. Mounting nodes state will solve this issue > NiFi Cluster page shows multiple entries for same node identity > --- > > Key: NIFI-7559 > URL: https://issues.apache.org/jira/browse/NIFI-7559 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework, Core UI >Affects Versions: 1.9.1, 1.9.2 > Environment: Linux >Reporter: Mohammed Nadeem >Priority: Major > Attachments: image-2020-06-19-02-52-35-420.png > > > In a containerized environment, When a NiFi node doesn't meet health probes > performed by Kubelet (Kubernetes), then the node is killed and restarted. > This may loop continuously causing *NiFi Cluster page* ( Hamburger Menu -> > Cluster ) to show multiple entries for the same node that gets restarted > again and again. > To replicate the issue: > 1. Setup single-node cluster ( statefulset application with replicaCount = 1) > 2. Change configured probes such as Startup or Liveness to duration less than > regular node start up time . > 3. Scale new node (replicaCount), for example from 1 to 2. Then new scaled > node *node-1* gets killed and restarted continuously by kubelet . > Please check the screenshot below. > *nifi-1:8080* is being added 7 times when it should be added just once, This > shows incorrect *1/8* nodes cluster summary when expected behavior is just > 1/2 nodes . > !image-2020-06-19-02-50-27-674.png! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-7656) TestZooKeeperStateServer fails on Win10 latest Java8
[ https://issues.apache.org/jira/browse/NIFI-7656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-7656: --- Description: [ERROR] testServerCreatePath(org.apache.nifi.controller.state.server.zookeeper.TestZooKeeperStateServer) Time elapsed: 0.163 s <<< ERROR! org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /test at org.apache.nifi.controller.state.server.zookeeper.TestZooKeeperStateServer.testServerCreatePath(TestZooKeeperStateServer.java:130) was: [ERROR] Tests run: 6, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 3.036 s <<< FAILURE! - in org.apache.nifi.processors.standard.TestListenTCP [ERROR] testTLSClientAuthRequiredAndClientCertProvided(org.apache.nifi.processors.standard.TestListenTCP) Time elapsed: 0.094 s <<< ERROR! org.apache.nifi.security.util.TlsException: The keystore properties are not valid at org.apache.nifi.processors.standard.TestListenTCP.testTLSClientAuthRequiredAndClientCertProvided(TestListenTCP.java:141) [ERROR] testTLSClientAuthRequiredAndClientCertProvided(org.apache.nifi.processors.standard.TestListenTCPRecord) Time elapsed: 0.047 s <<< ERROR! org.apache.nifi.security.util.TlsException: The keystore properties are not valid at org.apache.nifi.processors.standard.TestListenTCPRecord.testTLSClientAuthRequiredAndClientCertProvided(TestListenTCPRecord.java:178) > TestZooKeeperStateServer fails on Win10 latest Java8 > > > Key: NIFI-7656 > URL: https://issues.apache.org/jira/browse/NIFI-7656 > Project: Apache NiFi > Issue Type: Test > Components: Tools and Build > Environment: openjdk version "1.8.0_242" > OpenJDK Runtime Environment (build 1.8.0_242-b08) > OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode) > Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; > 2015-11-10T11:41:47-05:00) > Maven home: C:\development\tools\apache-maven-3.3.9 > Java version: 1.8.0_102, vendor: Oracle Corporation > Java home: C:\Program Files\Java\jdk1.8.0_102\jre > Default locale: en_US, platform encoding: Cp1252 > OS name: "windows 10", version: "10.0", arch: "amd64", family: "dos" >Reporter: Joe Witt >Priority: Major > Fix For: 1.12.0 > > > [ERROR] > testServerCreatePath(org.apache.nifi.controller.state.server.zookeeper.TestZooKeeperStateServer) > Time elapsed: 0.163 s <<< ERROR! > org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = > NodeExists for /test > at > org.apache.nifi.controller.state.server.zookeeper.TestZooKeeperStateServer.testServerCreatePath(TestZooKeeperStateServer.java:130) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-7656) TestZooKeeperStateServer fails on Win10 latest Java8
[ https://issues.apache.org/jira/browse/NIFI-7656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Witt updated NIFI-7656: --- Summary: TestZooKeeperStateServer fails on Win10 latest Java8 (was: TestListenTCP and TestListenTCPRecord fail in some windows environments) > TestZooKeeperStateServer fails on Win10 latest Java8 > > > Key: NIFI-7656 > URL: https://issues.apache.org/jira/browse/NIFI-7656 > Project: Apache NiFi > Issue Type: Test > Components: Tools and Build > Environment: openjdk version "1.8.0_242" > OpenJDK Runtime Environment (build 1.8.0_242-b08) > OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode) > Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; > 2015-11-10T11:41:47-05:00) > Maven home: C:\development\tools\apache-maven-3.3.9 > Java version: 1.8.0_102, vendor: Oracle Corporation > Java home: C:\Program Files\Java\jdk1.8.0_102\jre > Default locale: en_US, platform encoding: Cp1252 > OS name: "windows 10", version: "10.0", arch: "amd64", family: "dos" >Reporter: Joe Witt >Priority: Major > Fix For: 1.12.0 > > > [ERROR] Tests run: 6, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 3.036 > s <<< FAILURE! - in org.apache.nifi.processors.standard.TestListenTCP > [ERROR] > testTLSClientAuthRequiredAndClientCertProvided(org.apache.nifi.processors.standard.TestListenTCP) > Time elapsed: 0.094 s <<< ERROR! > org.apache.nifi.security.util.TlsException: The keystore properties are not > valid > at > org.apache.nifi.processors.standard.TestListenTCP.testTLSClientAuthRequiredAndClientCertProvided(TestListenTCP.java:141) > [ERROR] > testTLSClientAuthRequiredAndClientCertProvided(org.apache.nifi.processors.standard.TestListenTCPRecord) > Time elapsed: 0.047 s <<< ERROR! > org.apache.nifi.security.util.TlsException: The keystore properties are not > valid > at > org.apache.nifi.processors.standard.TestListenTCPRecord.testTLSClientAuthRequiredAndClientCertProvided(TestListenTCPRecord.java:178) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7656) TestListenTCP and TestListenTCPRecord fail in some windows environments
[ https://issues.apache.org/jira/browse/NIFI-7656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17160106#comment-17160106 ] Joe Witt commented on NIFI-7656: upgraded to latest available adopt open jdk 8 for windows 64 and the TLS tests are now fine. Updating to reflect new test issue > TestListenTCP and TestListenTCPRecord fail in some windows environments > --- > > Key: NIFI-7656 > URL: https://issues.apache.org/jira/browse/NIFI-7656 > Project: Apache NiFi > Issue Type: Test > Components: Tools and Build > Environment: openjdk version "1.8.0_242" > OpenJDK Runtime Environment (build 1.8.0_242-b08) > OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode) > Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; > 2015-11-10T11:41:47-05:00) > Maven home: C:\development\tools\apache-maven-3.3.9 > Java version: 1.8.0_102, vendor: Oracle Corporation > Java home: C:\Program Files\Java\jdk1.8.0_102\jre > Default locale: en_US, platform encoding: Cp1252 > OS name: "windows 10", version: "10.0", arch: "amd64", family: "dos" >Reporter: Joe Witt >Priority: Major > Fix For: 1.12.0 > > > [ERROR] Tests run: 6, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 3.036 > s <<< FAILURE! - in org.apache.nifi.processors.standard.TestListenTCP > [ERROR] > testTLSClientAuthRequiredAndClientCertProvided(org.apache.nifi.processors.standard.TestListenTCP) > Time elapsed: 0.094 s <<< ERROR! > org.apache.nifi.security.util.TlsException: The keystore properties are not > valid > at > org.apache.nifi.processors.standard.TestListenTCP.testTLSClientAuthRequiredAndClientCertProvided(TestListenTCP.java:141) > [ERROR] > testTLSClientAuthRequiredAndClientCertProvided(org.apache.nifi.processors.standard.TestListenTCPRecord) > Time elapsed: 0.047 s <<< ERROR! > org.apache.nifi.security.util.TlsException: The keystore properties are not > valid > at > org.apache.nifi.processors.standard.TestListenTCPRecord.testTLSClientAuthRequiredAndClientCertProvided(TestListenTCPRecord.java:178) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-7657) Lower log severity of expected exception for authentication on unsecured instance
[
https://issues.apache.org/jira/browse/NIFI-7657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy LoPresto updated NIFI-7657:
Description:
All unsecured instances report an INFO log at startup and on additional
requests due to the Kerberos authentication mechanism being unavailable. This
log severity should be lowered and the message suppressed by default as it is
an expected scenario but causes concern for many users.
{code}
2020-07-17 11:37:38,023 INFO [NiFi Web Server-18]
o.a.n.w.m.IllegalStateExceptionMapper java.lang.IllegalStateException: Access
tokens are only issued over HTTPS. Returning Conflict response.
{code}
A follow-on Jira should address removing the unnecessary request entirely.
was:
All unsecured instances report an INFO log at startup and on additional
requests due to the Kerberos authentication mechanism being unavailable. This
log severity should be lowered and the message suppressed by default as it is
an expected scenario but causes concern for many users.
{code}
2020-07-17 11:37:38,023 INFO [NiFi Web Server-18]
o.a.n.w.m.IllegalStateExceptionMapper java.lang.IllegalStateException: Access
tokens are only issued over HTTPS. Returning Conflict response.
{code}
> Lower log severity of expected exception for authentication on unsecured
> instance
> -
>
> Key: NIFI-7657
> URL: https://issues.apache.org/jira/browse/NIFI-7657
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Affects Versions: 1.11.4
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Major
> Labels: error, log, security
>
> All unsecured instances report an INFO log at startup and on additional
> requests due to the Kerberos authentication mechanism being unavailable. This
> log severity should be lowered and the message suppressed by default as it is
> an expected scenario but causes concern for many users.
> {code}
> 2020-07-17 11:37:38,023 INFO [NiFi Web Server-18]
> o.a.n.w.m.IllegalStateExceptionMapper java.lang.IllegalStateException: Access
> tokens are only issued over HTTPS. Returning Conflict response.
> {code}
> A follow-on Jira should address removing the unnecessary request entirely.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (NIFI-7657) Lower log severity of expected exception for authentication on unsecured instance
Andy LoPresto created NIFI-7657:
---
Summary: Lower log severity of expected exception for
authentication on unsecured instance
Key: NIFI-7657
URL: https://issues.apache.org/jira/browse/NIFI-7657
Project: Apache NiFi
Issue Type: Improvement
Components: Core Framework
Affects Versions: 1.11.4
Reporter: Andy LoPresto
Assignee: Andy LoPresto
All unsecured instances report an INFO log at startup and on additional
requests due to the Kerberos authentication mechanism being unavailable. This
log severity should be lowered and the message suppressed by default as it is
an expected scenario but causes concern for many users.
{code}
2020-07-17 11:37:38,023 INFO [NiFi Web Server-18]
o.a.n.w.m.IllegalStateExceptionMapper java.lang.IllegalStateException: Access
tokens are only issued over HTTPS. Returning Conflict response.
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (NIFI-7656) TestListenTCP and TestListenTCPRecord fail in some windows environments
Joe Witt created NIFI-7656: -- Summary: TestListenTCP and TestListenTCPRecord fail in some windows environments Key: NIFI-7656 URL: https://issues.apache.org/jira/browse/NIFI-7656 Project: Apache NiFi Issue Type: Test Components: Tools and Build Environment: openjdk version "1.8.0_242" OpenJDK Runtime Environment (build 1.8.0_242-b08) OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode) Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T11:41:47-05:00) Maven home: C:\development\tools\apache-maven-3.3.9 Java version: 1.8.0_102, vendor: Oracle Corporation Java home: C:\Program Files\Java\jdk1.8.0_102\jre Default locale: en_US, platform encoding: Cp1252 OS name: "windows 10", version: "10.0", arch: "amd64", family: "dos" Reporter: Joe Witt Fix For: 1.12.0 [ERROR] Tests run: 6, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 3.036 s <<< FAILURE! - in org.apache.nifi.processors.standard.TestListenTCP [ERROR] testTLSClientAuthRequiredAndClientCertProvided(org.apache.nifi.processors.standard.TestListenTCP) Time elapsed: 0.094 s <<< ERROR! org.apache.nifi.security.util.TlsException: The keystore properties are not valid at org.apache.nifi.processors.standard.TestListenTCP.testTLSClientAuthRequiredAndClientCertProvided(TestListenTCP.java:141) [ERROR] testTLSClientAuthRequiredAndClientCertProvided(org.apache.nifi.processors.standard.TestListenTCPRecord) Time elapsed: 0.047 s <<< ERROR! org.apache.nifi.security.util.TlsException: The keystore properties are not valid at org.apache.nifi.processors.standard.TestListenTCPRecord.testTLSClientAuthRequiredAndClientCertProvided(TestListenTCPRecord.java:178) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7552) Add "batch.output.XYZ" attribute when Process Group is using batch output mode
[ https://issues.apache.org/jira/browse/NIFI-7552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17159980#comment-17159980 ] ASF subversion and git services commented on NIFI-7552: --- Commit 2e803e0e8fb0ea2e58446cd2fabe64e96f7b5640 in nifi's branch refs/heads/main from Pierre Villard [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=2e803e0 ] NIFI-7552 - fix checkstyle > Add "batch.output.XYZ" attribute when Process Group is using batch output mode > -- > > Key: NIFI-7552 > URL: https://issues.apache.org/jira/browse/NIFI-7552 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.12.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > When a Process Group is configured to use a FlowFile Concurrency of Single > FlowFile per Node and an Outbound Policy of Batch Output, it would be helpful > to know how many FlowFiles are routed to each of the Output Ports. This is > often needed in order to detect whether or not anything in the input failed, > etc. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (NIFIREG-405) Update master to main in README
[ https://issues.apache.org/jira/browse/NIFIREG-405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Turcsanyi reassigned NIFIREG-405: --- Assignee: Peter Turcsanyi > Update master to main in README > --- > > Key: NIFIREG-405 > URL: https://issues.apache.org/jira/browse/NIFIREG-405 > Project: NiFi Registry > Issue Type: Improvement >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-registry] turcsanyip opened a new pull request #288: NIFIREG-405: Updated master to main in README
turcsanyip opened a new pull request #288: URL: https://github.com/apache/nifi-registry/pull/288 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFIREG-405) Update master to main in README
Peter Turcsanyi created NIFIREG-405: --- Summary: Update master to main in README Key: NIFIREG-405 URL: https://issues.apache.org/jira/browse/NIFIREG-405 Project: NiFi Registry Issue Type: Improvement Reporter: Peter Turcsanyi -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #837: MINIFICPP-1121 - Upgrade spdlog to version 1.7.0
szaszm commented on a change in pull request #837: URL: https://github.com/apache/nifi-minifi-cpp/pull/837#discussion_r456474581 ## File path: LICENSE ## @@ -258,23 +258,27 @@ This product bundles 'spdlog' which is available under an MIT license. Copyright (c) 2016 spdlog. - Permission is hereby granted, free of charge, to any person obtaining a copy - of this software and associated documentation files (the "Software"), to deal - in the Software without restriction, including without limitation the rights - to use, copy, modify, merge, publish, distribute, sublicense, and/or sell - copies of the Software, and to permit persons to whom the Software is - furnished to do so, subject to the following conditions: - - The above copyright notice and this permission notice shall be included in - all copies or substantial portions of the Software. - - THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR - IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, - FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE - AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER - LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, - OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN - THE SOFTWARE. + Permission is hereby granted, free of charge, to any person obtaining a copy + of this software and associated documentation files (the "Software"), to deal + in the Software without restriction, including without limitation the rights + to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + copies of the Software, and to permit persons to whom the Software is + furnished to do so, subject to the following conditions: + + The above copyright notice and this permission notice shall be included in + all copies or substantial portions of the Software. + + THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + THE SOFTWARE. + + -- NOTE: Third party dependency used by this software -- + This software depends on the fmt lib (MIT License), + and users must comply to its license: https://github.com/fmtlib/fmt/blob/master/LICENSE.rst Review comment: Good point from Aldrin, I take back that point. It's too bad that spdlog's LICENSE file is incomplete... :( This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #837: MINIFICPP-1121 - Upgrade spdlog to version 1.7.0
szaszm commented on a change in pull request #837: URL: https://github.com/apache/nifi-minifi-cpp/pull/837#discussion_r456473493 ## File path: NOTICE ## @@ -44,11 +44,13 @@ Amazon Technologies, Inc (http://www.amazon.com/). THIRD PARTY COMPONENTS ** This software includes third party software subject to the following copyrights: -- XML parsing and utility functions from TinyXml2 - Lee Thomason. -- JSON parsing and utility functions from JsonCpp - Copyright (c) 2007-2010 Baptiste Lepilleur. +- Very fast, header-only/compiled, C++ logging library from spdlog - Copyright (c) 2016 Gabi Melman +- An open-source formatting library for C++ from fmt - Copyright (c) 2012 - present, Victor Zverovich +- XML parsing and utility functions from TinyXml2 - Lee Thomason +- JSON parsing and utility functions from JsonCpp - Copyright (c) 2007-2010 Baptiste Lepilleur - OpenSSL build files for cmake used for Android Builds - Copyright (C) 2007-2012 LuaDist and Copyright (C) 2013 Brian Sidebotham - Android tool chain cmake build files - Copyright (c) 2010-2011, Ethan Rublee and Copyright (c) 2011-2014, Andrey Kamaev -- gsl-lite - Copyright (c) 2015 Martin Moene and Copyright (c) 2015 Microsoft Corporation. All rights reserved. +- gsl-lite - Copyright (c) 2015 Martin Moene and Copyright (c) 2015 Microsoft Corporation. All rights reserved Review comment: I don't know whether they need to be literal, but I tried to modify as little as possible while keeping then on a single line (for 1 3rd party) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on pull request #807: MINIFICPP-1228 - Resource ownership refactor + flowFile-owning processors.
szaszm commented on pull request #807: URL: https://github.com/apache/nifi-minifi-cpp/pull/807#issuecomment-660132934 Could you check the CI failures? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #837: MINIFICPP-1121 - Upgrade spdlog to version 1.7.0
hunyadi-dev commented on a change in pull request #837: URL: https://github.com/apache/nifi-minifi-cpp/pull/837#discussion_r456418735 ## File path: LICENSE ## @@ -258,23 +258,27 @@ This product bundles 'spdlog' which is available under an MIT license. Copyright (c) 2016 spdlog. - Permission is hereby granted, free of charge, to any person obtaining a copy - of this software and associated documentation files (the "Software"), to deal - in the Software without restriction, including without limitation the rights - to use, copy, modify, merge, publish, distribute, sublicense, and/or sell - copies of the Software, and to permit persons to whom the Software is - furnished to do so, subject to the following conditions: - - The above copyright notice and this permission notice shall be included in - all copies or substantial portions of the Software. - - THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR - IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, - FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE - AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER - LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, - OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN - THE SOFTWARE. + Permission is hereby granted, free of charge, to any person obtaining a copy + of this software and associated documentation files (the "Software"), to deal + in the Software without restriction, including without limitation the rights + to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + copies of the Software, and to permit persons to whom the Software is + furnished to do so, subject to the following conditions: + + The above copyright notice and this permission notice shall be included in + all copies or substantial portions of the Software. + + THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + THE SOFTWARE. + + -- NOTE: Third party dependency used by this software -- + This software depends on the fmt lib (MIT License), + and users must comply to its license: https://github.com/fmtlib/fmt/blob/master/LICENSE.rst Review comment: Ah, I have seen those lines, but I do not know why we have them. At no point did spdlog have them listed in its `LICENSE` ([[see here]](https://github.com/gabime/spdlog/commits/v1.x/LICENSE)), but their names were present in the codebase as copyright holders. For some reason [[Aldrin asked us]](https://github.com/apache/nifi-minifi-cpp/pull/55) to provide these in our notice as well. I updated the lines so that they are in correspondence with the current copyright notices in the spdlog files. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #837: MINIFICPP-1121 - Upgrade spdlog to version 1.7.0
hunyadi-dev commented on a change in pull request #837: URL: https://github.com/apache/nifi-minifi-cpp/pull/837#discussion_r456471246 ## File path: NOTICE ## @@ -44,11 +44,13 @@ Amazon Technologies, Inc (http://www.amazon.com/). THIRD PARTY COMPONENTS ** This software includes third party software subject to the following copyrights: -- XML parsing and utility functions from TinyXml2 - Lee Thomason. -- JSON parsing and utility functions from JsonCpp - Copyright (c) 2007-2010 Baptiste Lepilleur. +- Very fast, header-only/compiled, C++ logging library from spdlog - Copyright (c) 2016 Gabi Melman +- An open-source formatting library for C++ from fmt - Copyright (c) 2012 - present, Victor Zverovich +- XML parsing and utility functions from TinyXml2 - Lee Thomason +- JSON parsing and utility functions from JsonCpp - Copyright (c) 2007-2010 Baptiste Lepilleur - OpenSSL build files for cmake used for Android Builds - Copyright (C) 2007-2012 LuaDist and Copyright (C) 2013 Brian Sidebotham - Android tool chain cmake build files - Copyright (c) 2010-2011, Ethan Rublee and Copyright (c) 2011-2014, Andrey Kamaev -- gsl-lite - Copyright (c) 2015 Martin Moene and Copyright (c) 2015 Microsoft Corporation. All rights reserved. +- gsl-lite - Copyright (c) 2015 Martin Moene and Copyright (c) 2015 Microsoft Corporation. All rights reserved Review comment: Ah, I did not know that these notices are literal. Will add it back again :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #807: MINIFICPP-1228 - Resource ownership refactor + flowFile-owning processors.
szaszm commented on a change in pull request #807:
URL: https://github.com/apache/nifi-minifi-cpp/pull/807#discussion_r456470888
##
File path: libminifi/src/core/ProcessSession.cpp
##
@@ -540,111 +481,97 @@ void ProcessSession::import(const std::string& source,
std::vector buffer(getpagesize());
try {
-try {
- std::ifstream input{source, std::ios::in | std::ios::binary};
- logger_->log_debug("Opening %s", source);
- if (!input.is_open() || !input.good()) {
-throw Exception(FILE_OPERATION_EXCEPTION,
utils::StringUtils::join_pack("File Import Error: failed to open file \'",
source, "\'"));
+std::ifstream input{source, std::ios::in | std::ios::binary};
+logger_->log_debug("Opening %s", source);
+if (!input.is_open() || !input.good()) {
+ throw Exception(FILE_OPERATION_EXCEPTION,
utils::StringUtils::join_pack("File Import Error: failed to open file \'",
source, "\'"));
+}
+if (offset != 0U) {
+ input.seekg(offset, std::ifstream::beg);
+ if (!input.good()) {
+logger_->log_error("Seeking to %lu failed for file %s (does
file/filesystem support seeking?)", offset, source);
+throw Exception(FILE_OPERATION_EXCEPTION,
utils::StringUtils::join_pack("File Import Error: Couldn't seek to offset ",
std::to_string(offset)));
}
- if (offset != 0U) {
-input.seekg(offset, std::ifstream::beg);
-if (!input.good()) {
- logger_->log_error("Seeking to %lu failed for file %s (does
file/filesystem support seeking?)", offset, source);
- throw Exception(FILE_OPERATION_EXCEPTION,
utils::StringUtils::join_pack("File Import Error: Couldn't seek to offset ",
std::to_string(offset)));
-}
+}
+uint64_t startTime = 0U;
+while (input.good()) {
+ input.read(reinterpret_cast(buffer.data()), buffer.size());
+ std::streamsize read = input.gcount();
+ if (read < 0) {
+throw Exception(FILE_OPERATION_EXCEPTION, "std::ifstream::gcount
returned negative value");
}
- uint64_t startTime = 0U;
- while (input.good()) {
-input.read(reinterpret_cast(buffer.data()), buffer.size());
-std::streamsize read = input.gcount();
-if (read < 0) {
- throw Exception(FILE_OPERATION_EXCEPTION, "std::ifstream::gcount
returned negative value");
-}
-if (read == 0) {
- logger_->log_trace("Finished reading input %s", source);
+ if (read == 0) {
+logger_->log_trace("Finished reading input %s", source);
+break;
+ } else {
+logging::LOG_TRACE(logger_) << "Read input of " << read;
+ }
+ uint8_t* begin = buffer.data();
+ uint8_t* end = begin + read;
+ while (true) {
+startTime = getTimeMillis();
+uint8_t* delimiterPos = std::find(begin, end,
static_cast(inputDelimiter));
+const auto len = gsl::narrow(delimiterPos - begin);
+
+logging::LOG_TRACE(logger_) << "Read input of " << read << " length is
" << len << " is at end?" << (delimiterPos == end);
+/*
+ * We do not want to process the rest of the buffer after the last
delimiter if
+ * - we have reached EOF in the file (we would discard it anyway)
+ * - there is nothing to process (the last character in the buffer is
a delimiter)
+ */
+if (delimiterPos == end && (input.eof() || len == 0)) {
break;
-} else {
- logging::LOG_TRACE(logger_) << "Read input of " << read;
}
-uint8_t* begin = buffer.data();
-uint8_t* end = begin + read;
-while (true) {
- startTime = getTimeMillis();
- uint8_t* delimiterPos = std::find(begin, end,
static_cast(inputDelimiter));
- const auto len = gsl::narrow(delimiterPos - begin);
-
- logging::LOG_TRACE(logger_) << "Read input of " << read << " length
is " << len << " is at end?" << (delimiterPos == end);
- /*
- * We do not want to process the rest of the buffer after the last
delimiter if
- * - we have reached EOF in the file (we would discard it anyway)
- * - there is nothing to process (the last character in the buffer
is a delimiter)
- */
- if (delimiterPos == end && (input.eof() || len == 0)) {
-break;
- }
-
- /* Create claim and stream if needed and append data */
- if (claim == nullptr) {
-startTime = getTimeMillis();
-claim =
std::make_shared(process_context_->getContentRepository());
- }
- if (stream == nullptr) {
-stream = process_context_->getContentRepository()->write(claim);
- }
- if (stream == nullptr) {
-logger_->log_error("Stream is null");
-rollback();
-return;
- }
- if (stream->write(begin, len) != len) {
-logger_->log_error("Error while writing");
[GitHub] [nifi] bbende commented on pull request #4404: NIFI-7630 Upgraded H2 dependency.
bbende commented on pull request #4404: URL: https://github.com/apache/nifi/pull/4404#issuecomment-660120138 I would have expected 199 to work, but I also haven't tested any upgrade paths for NiFi. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #837: MINIFICPP-1121 - Upgrade spdlog to version 1.7.0
hunyadi-dev commented on a change in pull request #837: URL: https://github.com/apache/nifi-minifi-cpp/pull/837#discussion_r456418735 ## File path: LICENSE ## @@ -258,23 +258,27 @@ This product bundles 'spdlog' which is available under an MIT license. Copyright (c) 2016 spdlog. - Permission is hereby granted, free of charge, to any person obtaining a copy - of this software and associated documentation files (the "Software"), to deal - in the Software without restriction, including without limitation the rights - to use, copy, modify, merge, publish, distribute, sublicense, and/or sell - copies of the Software, and to permit persons to whom the Software is - furnished to do so, subject to the following conditions: - - The above copyright notice and this permission notice shall be included in - all copies or substantial portions of the Software. - - THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR - IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, - FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE - AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER - LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, - OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN - THE SOFTWARE. + Permission is hereby granted, free of charge, to any person obtaining a copy + of this software and associated documentation files (the "Software"), to deal + in the Software without restriction, including without limitation the rights + to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + copies of the Software, and to permit persons to whom the Software is + furnished to do so, subject to the following conditions: + + The above copyright notice and this permission notice shall be included in + all copies or substantial portions of the Software. + + THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + THE SOFTWARE. + + -- NOTE: Third party dependency used by this software -- + This software depends on the fmt lib (MIT License), + and users must comply to its license: https://github.com/fmtlib/fmt/blob/master/LICENSE.rst Review comment: Ah, I have seen those lines, but I do not know where they are coming from. At no point did spdlog have them listed in its `LICENSE` ([[see here]](https://github.com/gabime/spdlog/commits/v1.x/LICENSE)), but their names were present in the codebase as copyright holders. For some reason [[Aldrin asked us]](https://github.com/apache/nifi-minifi-cpp/pull/55) to provide these in our notice as well. I updated the lines so that they are in correspondence with the current copyright notices in the spdlog files. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-7410) Clob unreadable code when convertToAvroStream in JdbcCommon.java
[
https://issues.apache.org/jira/browse/NIFI-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-7410:
-
Fix Version/s: 1.12.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Clob unreadable code when convertToAvroStream in JdbcCommon.java
> -
>
> Key: NIFI-7410
> URL: https://issues.apache.org/jira/browse/NIFI-7410
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.11.4
>Reporter: ZhangCheng
>Assignee: ZhangCheng
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.12.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> when ExecuteSql or QueryDataBaseTable processor trigger below code in
> JdbcCommon.java ,clob would be unreadable code（with Chinese character）
> {code:java}
> if (javaSqlType == CLOB) {
> Clob clob = rs.getClob(i);
> if (clob != null) {
> long numChars = clob.length();
> char[] buffer = new char[(int) numChars];
> InputStream is = clob.getAsciiStream();
> int index = 0;
> int c = is.read();
> while (c >= 0) {
> buffer[index++] = (char) c;
> c = is.read();
> }
> rec.put(i - 1, new String(buffer));
> clob.free();
> } else {
> rec.put(i - 1, null);
> }
> continue;
> }
> {code}
> I konw this can be resoveld by using ExecuteSqlRecord and
> QueryDatabaseTableRecord. Then have new avroWriter(by using controller
> cervice), so I think ,can we change the DefaultAvroSqlWriter to the new
> avroWriter?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-7410) Clob unreadable code when convertToAvroStream in JdbcCommon.java
[
https://issues.apache.org/jira/browse/NIFI-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17159899#comment-17159899
]
ASF subversion and git services commented on NIFI-7410:
---
Commit e18b4f0c75849355d11519eab23815f894e5c365 in nifi's branch
refs/heads/main from zhangcheng
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=e18b4f0 ]
NIFI-7410 Update JdbcCommon.java when javaSqlType is CLOB or NCLOB in
convertToAvroStream method, use the DataTypeUtils.toString(clob/nClob,(String)
null, StandardCharsets.UTF_8)) method to get the String
NIFI-7410 Update JdbcCommon.java when javaSqlType is CLOB or NCLOB in
convertToAvroStream method, use the CharacterStream rto read the value of CLOB
NIFI-7410 Add a unit test. validate if it's unreadable when the clob value is
Chinese, Japanese, and Korean.
Signed-off-by: Pierre Villard
This closes #4243.
> Clob unreadable code when convertToAvroStream in JdbcCommon.java
> -
>
> Key: NIFI-7410
> URL: https://issues.apache.org/jira/browse/NIFI-7410
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.11.4
>Reporter: ZhangCheng
>Assignee: ZhangCheng
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> when ExecuteSql or QueryDataBaseTable processor trigger below code in
> JdbcCommon.java ,clob would be unreadable code（with Chinese character）
> {code:java}
> if (javaSqlType == CLOB) {
> Clob clob = rs.getClob(i);
> if (clob != null) {
> long numChars = clob.length();
> char[] buffer = new char[(int) numChars];
> InputStream is = clob.getAsciiStream();
> int index = 0;
> int c = is.read();
> while (c >= 0) {
> buffer[index++] = (char) c;
> c = is.read();
> }
> rec.put(i - 1, new String(buffer));
> clob.free();
> } else {
> rec.put(i - 1, null);
> }
> continue;
> }
> {code}
> I konw this can be resoveld by using ExecuteSqlRecord and
> QueryDatabaseTableRecord. Then have new avroWriter(by using controller
> cervice), so I think ,can we change the DefaultAvroSqlWriter to the new
> avroWriter?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-7410) Clob unreadable code when convertToAvroStream in JdbcCommon.java
[
https://issues.apache.org/jira/browse/NIFI-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17159901#comment-17159901
]
ASF subversion and git services commented on NIFI-7410:
---
Commit e18b4f0c75849355d11519eab23815f894e5c365 in nifi's branch
refs/heads/main from zhangcheng
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=e18b4f0 ]
NIFI-7410 Update JdbcCommon.java when javaSqlType is CLOB or NCLOB in
convertToAvroStream method, use the DataTypeUtils.toString(clob/nClob,(String)
null, StandardCharsets.UTF_8)) method to get the String
NIFI-7410 Update JdbcCommon.java when javaSqlType is CLOB or NCLOB in
convertToAvroStream method, use the CharacterStream rto read the value of CLOB
NIFI-7410 Add a unit test. validate if it's unreadable when the clob value is
Chinese, Japanese, and Korean.
Signed-off-by: Pierre Villard
This closes #4243.
> Clob unreadable code when convertToAvroStream in JdbcCommon.java
> -
>
> Key: NIFI-7410
> URL: https://issues.apache.org/jira/browse/NIFI-7410
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.11.4
>Reporter: ZhangCheng
>Assignee: ZhangCheng
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> when ExecuteSql or QueryDataBaseTable processor trigger below code in
> JdbcCommon.java ,clob would be unreadable code（with Chinese character）
> {code:java}
> if (javaSqlType == CLOB) {
> Clob clob = rs.getClob(i);
> if (clob != null) {
> long numChars = clob.length();
> char[] buffer = new char[(int) numChars];
> InputStream is = clob.getAsciiStream();
> int index = 0;
> int c = is.read();
> while (c >= 0) {
> buffer[index++] = (char) c;
> c = is.read();
> }
> rec.put(i - 1, new String(buffer));
> clob.free();
> } else {
> rec.put(i - 1, null);
> }
> continue;
> }
> {code}
> I konw this can be resoveld by using ExecuteSqlRecord and
> QueryDatabaseTableRecord. Then have new avroWriter(by using controller
> cervice), so I think ,can we change the DefaultAvroSqlWriter to the new
> avroWriter?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-7410) Clob unreadable code when convertToAvroStream in JdbcCommon.java
[
https://issues.apache.org/jira/browse/NIFI-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17159900#comment-17159900
]
ASF subversion and git services commented on NIFI-7410:
---
Commit e18b4f0c75849355d11519eab23815f894e5c365 in nifi's branch
refs/heads/main from zhangcheng
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=e18b4f0 ]
NIFI-7410 Update JdbcCommon.java when javaSqlType is CLOB or NCLOB in
convertToAvroStream method, use the DataTypeUtils.toString(clob/nClob,(String)
null, StandardCharsets.UTF_8)) method to get the String
NIFI-7410 Update JdbcCommon.java when javaSqlType is CLOB or NCLOB in
convertToAvroStream method, use the CharacterStream rto read the value of CLOB
NIFI-7410 Add a unit test. validate if it's unreadable when the clob value is
Chinese, Japanese, and Korean.
Signed-off-by: Pierre Villard
This closes #4243.
> Clob unreadable code when convertToAvroStream in JdbcCommon.java
> -
>
> Key: NIFI-7410
> URL: https://issues.apache.org/jira/browse/NIFI-7410
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.11.4
>Reporter: ZhangCheng
>Assignee: ZhangCheng
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> when ExecuteSql or QueryDataBaseTable processor trigger below code in
> JdbcCommon.java ,clob would be unreadable code（with Chinese character）
> {code:java}
> if (javaSqlType == CLOB) {
> Clob clob = rs.getClob(i);
> if (clob != null) {
> long numChars = clob.length();
> char[] buffer = new char[(int) numChars];
> InputStream is = clob.getAsciiStream();
> int index = 0;
> int c = is.read();
> while (c >= 0) {
> buffer[index++] = (char) c;
> c = is.read();
> }
> rec.put(i - 1, new String(buffer));
> clob.free();
> } else {
> rec.put(i - 1, null);
> }
> continue;
> }
> {code}
> I konw this can be resoveld by using ExecuteSqlRecord and
> QueryDatabaseTableRecord. Then have new avroWriter(by using controller
> cervice), so I think ,can we change the DefaultAvroSqlWriter to the new
> avroWriter?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #4243: NIFI-7410 Fix CLOB would be unreadable code in ExecuteSql or QueryDataBaseTable
asfgit closed pull request #4243: URL: https://github.com/apache/nifi/pull/4243 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] pvillard31 commented on pull request #4243: NIFI-7410 Fix CLOB would be unreadable code in ExecuteSql or QueryDataBaseTable
pvillard31 commented on pull request #4243: URL: https://github.com/apache/nifi/pull/4243#issuecomment-660085007 Manually fixed the checkstyle violations and added the license to make the RAT check happy. Merging to main. Thanks @KuKuDeCheng (next time, please make sure to add the header and check the build with ``-Pcontrib-check``) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-7372) Add test for ZooKeeperStateServer
[ https://issues.apache.org/jira/browse/NIFI-7372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-7372: - Component/s: Core Framework > Add test for ZooKeeperStateServer > - > > Key: NIFI-7372 > URL: https://issues.apache.org/jira/browse/NIFI-7372 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core Framework >Reporter: Joey Frazee >Assignee: Joey Frazee >Priority: Minor > Fix For: 1.12.0 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (NIFI-7372) Add test for ZooKeeperStateServer
[ https://issues.apache.org/jira/browse/NIFI-7372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-7372. -- Fix Version/s: 1.12.0 Resolution: Fixed > Add test for ZooKeeperStateServer > - > > Key: NIFI-7372 > URL: https://issues.apache.org/jira/browse/NIFI-7372 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Joey Frazee >Assignee: Joey Frazee >Priority: Minor > Fix For: 1.12.0 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #4241: NIFI-7372 Added test for ZooKeeperStateServer
asfgit closed pull request #4241: URL: https://github.com/apache/nifi/pull/4241 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-7372) Add test for ZooKeeperStateServer
[ https://issues.apache.org/jira/browse/NIFI-7372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17159894#comment-17159894 ] ASF subversion and git services commented on NIFI-7372: --- Commit a72c3d685cdd694249b71b6f579a2830b67bf9b1 in nifi's branch refs/heads/main from Joey Frazee [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=a72c3d6 ] NIFI-7372 Added test for ZooKeeperStateServer Signed-off-by: Pierre Villard This closes #4241. > Add test for ZooKeeperStateServer > - > > Key: NIFI-7372 > URL: https://issues.apache.org/jira/browse/NIFI-7372 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Joey Frazee >Assignee: Joey Frazee >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] arpadboda closed pull request #821: MINIFICPP-1251 - Implement and test RetryFlowFile processor
arpadboda closed pull request #821: URL: https://github.com/apache/nifi-minifi-cpp/pull/821 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #807: MINIFICPP-1228 - Resource ownership refactor + flowFile-owning processors.
adamdebreceni commented on a change in pull request #807:
URL: https://github.com/apache/nifi-minifi-cpp/pull/807#discussion_r456401011
##
File path: libminifi/src/core/ProcessSession.cpp
##
@@ -540,111 +481,97 @@ void ProcessSession::import(const std::string& source,
std::vector buffer(getpagesize());
try {
-try {
- std::ifstream input{source, std::ios::in | std::ios::binary};
- logger_->log_debug("Opening %s", source);
- if (!input.is_open() || !input.good()) {
-throw Exception(FILE_OPERATION_EXCEPTION,
utils::StringUtils::join_pack("File Import Error: failed to open file \'",
source, "\'"));
+std::ifstream input{source, std::ios::in | std::ios::binary};
+logger_->log_debug("Opening %s", source);
+if (!input.is_open() || !input.good()) {
+ throw Exception(FILE_OPERATION_EXCEPTION,
utils::StringUtils::join_pack("File Import Error: failed to open file \'",
source, "\'"));
+}
+if (offset != 0U) {
+ input.seekg(offset, std::ifstream::beg);
+ if (!input.good()) {
+logger_->log_error("Seeking to %lu failed for file %s (does
file/filesystem support seeking?)", offset, source);
+throw Exception(FILE_OPERATION_EXCEPTION,
utils::StringUtils::join_pack("File Import Error: Couldn't seek to offset ",
std::to_string(offset)));
}
- if (offset != 0U) {
-input.seekg(offset, std::ifstream::beg);
-if (!input.good()) {
- logger_->log_error("Seeking to %lu failed for file %s (does
file/filesystem support seeking?)", offset, source);
- throw Exception(FILE_OPERATION_EXCEPTION,
utils::StringUtils::join_pack("File Import Error: Couldn't seek to offset ",
std::to_string(offset)));
-}
+}
+uint64_t startTime = 0U;
+while (input.good()) {
+ input.read(reinterpret_cast(buffer.data()), buffer.size());
+ std::streamsize read = input.gcount();
+ if (read < 0) {
+throw Exception(FILE_OPERATION_EXCEPTION, "std::ifstream::gcount
returned negative value");
}
- uint64_t startTime = 0U;
- while (input.good()) {
-input.read(reinterpret_cast(buffer.data()), buffer.size());
-std::streamsize read = input.gcount();
-if (read < 0) {
- throw Exception(FILE_OPERATION_EXCEPTION, "std::ifstream::gcount
returned negative value");
-}
-if (read == 0) {
- logger_->log_trace("Finished reading input %s", source);
+ if (read == 0) {
+logger_->log_trace("Finished reading input %s", source);
+break;
+ } else {
+logging::LOG_TRACE(logger_) << "Read input of " << read;
+ }
+ uint8_t* begin = buffer.data();
+ uint8_t* end = begin + read;
+ while (true) {
+startTime = getTimeMillis();
+uint8_t* delimiterPos = std::find(begin, end,
static_cast(inputDelimiter));
+const auto len = gsl::narrow(delimiterPos - begin);
+
+logging::LOG_TRACE(logger_) << "Read input of " << read << " length is
" << len << " is at end?" << (delimiterPos == end);
+/*
+ * We do not want to process the rest of the buffer after the last
delimiter if
+ * - we have reached EOF in the file (we would discard it anyway)
+ * - there is nothing to process (the last character in the buffer is
a delimiter)
+ */
+if (delimiterPos == end && (input.eof() || len == 0)) {
break;
-} else {
- logging::LOG_TRACE(logger_) << "Read input of " << read;
}
-uint8_t* begin = buffer.data();
-uint8_t* end = begin + read;
-while (true) {
- startTime = getTimeMillis();
- uint8_t* delimiterPos = std::find(begin, end,
static_cast(inputDelimiter));
- const auto len = gsl::narrow(delimiterPos - begin);
-
- logging::LOG_TRACE(logger_) << "Read input of " << read << " length
is " << len << " is at end?" << (delimiterPos == end);
- /*
- * We do not want to process the rest of the buffer after the last
delimiter if
- * - we have reached EOF in the file (we would discard it anyway)
- * - there is nothing to process (the last character in the buffer
is a delimiter)
- */
- if (delimiterPos == end && (input.eof() || len == 0)) {
-break;
- }
-
- /* Create claim and stream if needed and append data */
- if (claim == nullptr) {
-startTime = getTimeMillis();
-claim =
std::make_shared(process_context_->getContentRepository());
- }
- if (stream == nullptr) {
-stream = process_context_->getContentRepository()->write(claim);
- }
- if (stream == nullptr) {
-logger_->log_error("Stream is null");
-rollback();
-return;
- }
- if (stream->write(begin, len) != len) {
-logger_->log_error("Error while writ
[jira] [Commented] (NIFI-7552) Add "batch.output.XYZ" attribute when Process Group is using batch output mode
[ https://issues.apache.org/jira/browse/NIFI-7552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17159881#comment-17159881 ] ASF subversion and git services commented on NIFI-7552: --- Commit 5da596ea8dfd70c181f518a8db1497bd8fe7b363 in nifi's branch refs/heads/main from Pierre Villard [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=5da596e ] NIFI-7552 - fixed checkstyle issue > Add "batch.output.XYZ" attribute when Process Group is using batch output mode > -- > > Key: NIFI-7552 > URL: https://issues.apache.org/jira/browse/NIFI-7552 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.12.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > When a Process Group is configured to use a FlowFile Concurrency of Single > FlowFile per Node and an Outbound Policy of Batch Output, it would be helpful > to know how many FlowFiles are routed to each of the Output Ports. This is > often needed in order to detect whether or not anything in the input failed, > etc. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #837: MINIFICPP-1121 - Upgrade spdlog to version 1.7.0
szaszm commented on a change in pull request #837: URL: https://github.com/apache/nifi-minifi-cpp/pull/837#discussion_r456392073 ## File path: NOTICE ## @@ -44,11 +44,13 @@ Amazon Technologies, Inc (http://www.amazon.com/). THIRD PARTY COMPONENTS ** This software includes third party software subject to the following copyrights: -- XML parsing and utility functions from TinyXml2 - Lee Thomason. -- JSON parsing and utility functions from JsonCpp - Copyright (c) 2007-2010 Baptiste Lepilleur. +- Very fast, header-only/compiled, C++ logging library from spdlog - Copyright (c) 2016 Gabi Melman +- An open-source formatting library for C++ from fmt - Copyright (c) 2012 - present, Victor Zverovich +- XML parsing and utility functions from TinyXml2 - Lee Thomason +- JSON parsing and utility functions from JsonCpp - Copyright (c) 2007-2010 Baptiste Lepilleur - OpenSSL build files for cmake used for Android Builds - Copyright (C) 2007-2012 LuaDist and Copyright (C) 2013 Brian Sidebotham - Android tool chain cmake build files - Copyright (c) 2010-2011, Ethan Rublee and Copyright (c) 2011-2014, Andrey Kamaev -- gsl-lite - Copyright (c) 2015 Martin Moene and Copyright (c) 2015 Microsoft Corporation. All rights reserved. +- gsl-lite - Copyright (c) 2015 Martin Moene and Copyright (c) 2015 Microsoft Corporation. All rights reserved Review comment: The dot was part of the original: https://github.com/gsl-lite/gsl-lite/blob/master/LICENSE#L4 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #807: MINIFICPP-1228 - Resource ownership refactor + flowFile-owning processors.
szaszm commented on a change in pull request #807:
URL: https://github.com/apache/nifi-minifi-cpp/pull/807#discussion_r456383996
##
File path: libminifi/src/core/ProcessSession.cpp
##
@@ -540,111 +481,97 @@ void ProcessSession::import(const std::string& source,
std::vector buffer(getpagesize());
try {
-try {
- std::ifstream input{source, std::ios::in | std::ios::binary};
- logger_->log_debug("Opening %s", source);
- if (!input.is_open() || !input.good()) {
-throw Exception(FILE_OPERATION_EXCEPTION,
utils::StringUtils::join_pack("File Import Error: failed to open file \'",
source, "\'"));
+std::ifstream input{source, std::ios::in | std::ios::binary};
+logger_->log_debug("Opening %s", source);
+if (!input.is_open() || !input.good()) {
+ throw Exception(FILE_OPERATION_EXCEPTION,
utils::StringUtils::join_pack("File Import Error: failed to open file \'",
source, "\'"));
+}
+if (offset != 0U) {
+ input.seekg(offset, std::ifstream::beg);
+ if (!input.good()) {
+logger_->log_error("Seeking to %lu failed for file %s (does
file/filesystem support seeking?)", offset, source);
+throw Exception(FILE_OPERATION_EXCEPTION,
utils::StringUtils::join_pack("File Import Error: Couldn't seek to offset ",
std::to_string(offset)));
}
- if (offset != 0U) {
-input.seekg(offset, std::ifstream::beg);
-if (!input.good()) {
- logger_->log_error("Seeking to %lu failed for file %s (does
file/filesystem support seeking?)", offset, source);
- throw Exception(FILE_OPERATION_EXCEPTION,
utils::StringUtils::join_pack("File Import Error: Couldn't seek to offset ",
std::to_string(offset)));
-}
+}
+uint64_t startTime = 0U;
+while (input.good()) {
+ input.read(reinterpret_cast(buffer.data()), buffer.size());
+ std::streamsize read = input.gcount();
+ if (read < 0) {
+throw Exception(FILE_OPERATION_EXCEPTION, "std::ifstream::gcount
returned negative value");
}
- uint64_t startTime = 0U;
- while (input.good()) {
-input.read(reinterpret_cast(buffer.data()), buffer.size());
-std::streamsize read = input.gcount();
-if (read < 0) {
- throw Exception(FILE_OPERATION_EXCEPTION, "std::ifstream::gcount
returned negative value");
-}
-if (read == 0) {
- logger_->log_trace("Finished reading input %s", source);
+ if (read == 0) {
+logger_->log_trace("Finished reading input %s", source);
+break;
+ } else {
+logging::LOG_TRACE(logger_) << "Read input of " << read;
+ }
+ uint8_t* begin = buffer.data();
+ uint8_t* end = begin + read;
+ while (true) {
+startTime = getTimeMillis();
+uint8_t* delimiterPos = std::find(begin, end,
static_cast(inputDelimiter));
+const auto len = gsl::narrow(delimiterPos - begin);
+
+logging::LOG_TRACE(logger_) << "Read input of " << read << " length is
" << len << " is at end?" << (delimiterPos == end);
+/*
+ * We do not want to process the rest of the buffer after the last
delimiter if
+ * - we have reached EOF in the file (we would discard it anyway)
+ * - there is nothing to process (the last character in the buffer is
a delimiter)
+ */
+if (delimiterPos == end && (input.eof() || len == 0)) {
break;
-} else {
- logging::LOG_TRACE(logger_) << "Read input of " << read;
}
-uint8_t* begin = buffer.data();
-uint8_t* end = begin + read;
-while (true) {
- startTime = getTimeMillis();
- uint8_t* delimiterPos = std::find(begin, end,
static_cast(inputDelimiter));
- const auto len = gsl::narrow(delimiterPos - begin);
-
- logging::LOG_TRACE(logger_) << "Read input of " << read << " length
is " << len << " is at end?" << (delimiterPos == end);
- /*
- * We do not want to process the rest of the buffer after the last
delimiter if
- * - we have reached EOF in the file (we would discard it anyway)
- * - there is nothing to process (the last character in the buffer
is a delimiter)
- */
- if (delimiterPos == end && (input.eof() || len == 0)) {
-break;
- }
-
- /* Create claim and stream if needed and append data */
- if (claim == nullptr) {
-startTime = getTimeMillis();
-claim =
std::make_shared(process_context_->getContentRepository());
- }
- if (stream == nullptr) {
-stream = process_context_->getContentRepository()->write(claim);
- }
- if (stream == nullptr) {
-logger_->log_error("Stream is null");
-rollback();
-return;
- }
- if (stream->write(begin, len) != len) {
-logger_->log_error("Error while writing");
[jira] [Commented] (NIFI-7493) XML Schema Inference can infer a type of String when it should be Record
[
https://issues.apache.org/jira/browse/NIFI-7493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17159852#comment-17159852
]
ASF subversion and git services commented on NIFI-7493:
---
Commit 958d9c4cb674a4723a6b8ac59f0a176447844f45 in nifi's branch
refs/heads/main from Pierre Villard
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=958d9c4 ]
NIFI-7493 - fix checkstyle issue
> XML Schema Inference can infer a type of String when it should be Record
>
>
> Key: NIFI-7493
> URL: https://issues.apache.org/jira/browse/NIFI-7493
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.12.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> From the mailing list:
> {quote}I have configured a XMLReader to use the Infer Schema. The other issue
> is that I have problems converting sub records. My records looks something
> like this: John Doe
> some there
> workingman
> New York
> A Company
>
>
>
>
> The issues are with the subrecords in part 3. I have configured the XMLReader
> property "Field Name for Content" = value
>
> When the data is being converted via a XMLWriter the output for the
> additionalInfo fields looks like this:
> MapRecord[\{name=Location,
> value=New York}]
> MapRecord[\{name=Company, value=A
> Company}]
>
>
>
> If I use a JSONWriter I gets this:
> "Part3": { "Details": {
> "additionalInfo": [ "MapRecord[\{name=Location, value=New York}]",
> "MapRecord[\{name=Company, value=A Company}]" ]
> }
> }{quote}
> The issue appears to be that "additionalInfo" is being inferred as a String,
> but the XML Reader is returning a Record.
>
> This is probably because the "additionalInfo" element contains String
> content and no child nodes. However, it does have attributes. As a result,
> the XML Reader will return a Record. I'm guessing that attributes are not
> taken into account in the schema inference, though, and since
> "additionalInfo" has no child nodes but has textual content, it must be a
> String.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #837: MINIFICPP-1121 - Upgrade spdlog to version 1.7.0
szaszm commented on a change in pull request #837: URL: https://github.com/apache/nifi-minifi-cpp/pull/837#discussion_r456351621 ## File path: LICENSE ## @@ -258,23 +258,27 @@ This product bundles 'spdlog' which is available under an MIT license. Copyright (c) 2016 spdlog. - Permission is hereby granted, free of charge, to any person obtaining a copy - of this software and associated documentation files (the "Software"), to deal - in the Software without restriction, including without limitation the rights - to use, copy, modify, merge, publish, distribute, sublicense, and/or sell - copies of the Software, and to permit persons to whom the Software is - furnished to do so, subject to the following conditions: - - The above copyright notice and this permission notice shall be included in - all copies or substantial portions of the Software. - - THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR - IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, - FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE - AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER - LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, - OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN - THE SOFTWARE. + Permission is hereby granted, free of charge, to any person obtaining a copy + of this software and associated documentation files (the "Software"), to deal + in the Software without restriction, including without limitation the rights + to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + copies of the Software, and to permit persons to whom the Software is + furnished to do so, subject to the following conditions: + + The above copyright notice and this permission notice shall be included in + all copies or substantial portions of the Software. + + THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + THE SOFTWARE. + + -- NOTE: Third party dependency used by this software -- + This software depends on the fmt lib (MIT License), + and users must comply to its license: https://github.com/fmtlib/fmt/blob/master/LICENSE.rst Review comment: first comment first bullet: In the `LICENSE` file lines 248-260, there is an outdated list of copyright notices for spdlog. Please update those to reflect the current version of spdlog's [`LICENSE`](https://github.com/gabime/spdlog/blob/v1.7.0/LICENSE#L3) file. Thanks for the updates. :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #837: MINIFICPP-1121 - Upgrade spdlog to version 1.7.0
szaszm commented on a change in pull request #837: URL: https://github.com/apache/nifi-minifi-cpp/pull/837#discussion_r456351621 ## File path: LICENSE ## @@ -258,23 +258,27 @@ This product bundles 'spdlog' which is available under an MIT license. Copyright (c) 2016 spdlog. - Permission is hereby granted, free of charge, to any person obtaining a copy - of this software and associated documentation files (the "Software"), to deal - in the Software without restriction, including without limitation the rights - to use, copy, modify, merge, publish, distribute, sublicense, and/or sell - copies of the Software, and to permit persons to whom the Software is - furnished to do so, subject to the following conditions: - - The above copyright notice and this permission notice shall be included in - all copies or substantial portions of the Software. - - THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR - IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, - FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE - AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER - LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, - OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN - THE SOFTWARE. + Permission is hereby granted, free of charge, to any person obtaining a copy + of this software and associated documentation files (the "Software"), to deal + in the Software without restriction, including without limitation the rights + to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + copies of the Software, and to permit persons to whom the Software is + furnished to do so, subject to the following conditions: + + The above copyright notice and this permission notice shall be included in + all copies or substantial portions of the Software. + + THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + THE SOFTWARE. + + -- NOTE: Third party dependency used by this software -- + This software depends on the fmt lib (MIT License), + and users must comply to its license: https://github.com/fmtlib/fmt/blob/master/LICENSE.rst Review comment: first comment first bullet: In the `LICENSE` file lines 248-260, there is an outdated list of copyright notices. Please update those to reflect the current version of spdlog's [`LICENSE`](https://github.com/gabime/spdlog/blob/v1.7.0/LICENSE#L3) file. Thanks for the updates. :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #837: MINIFICPP-1121 - Upgrade spdlog to version 1.7.0
szaszm commented on a change in pull request #837: URL: https://github.com/apache/nifi-minifi-cpp/pull/837#discussion_r456351621 ## File path: LICENSE ## @@ -258,23 +258,27 @@ This product bundles 'spdlog' which is available under an MIT license. Copyright (c) 2016 spdlog. - Permission is hereby granted, free of charge, to any person obtaining a copy - of this software and associated documentation files (the "Software"), to deal - in the Software without restriction, including without limitation the rights - to use, copy, modify, merge, publish, distribute, sublicense, and/or sell - copies of the Software, and to permit persons to whom the Software is - furnished to do so, subject to the following conditions: - - The above copyright notice and this permission notice shall be included in - all copies or substantial portions of the Software. - - THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR - IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, - FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE - AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER - LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, - OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN - THE SOFTWARE. + Permission is hereby granted, free of charge, to any person obtaining a copy + of this software and associated documentation files (the "Software"), to deal + in the Software without restriction, including without limitation the rights + to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + copies of the Software, and to permit persons to whom the Software is + furnished to do so, subject to the following conditions: + + The above copyright notice and this permission notice shall be included in + all copies or substantial portions of the Software. + + THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + THE SOFTWARE. + + -- NOTE: Third party dependency used by this software -- + This software depends on the fmt lib (MIT License), + and users must comply to its license: https://github.com/fmtlib/fmt/blob/master/LICENSE.rst Review comment: In the `LICENSE` file lines 248-260, there is an outdated list of copyright notices. Please update those to reflect the current version of spdlog's [`LICENSE`](https://github.com/gabime/spdlog/blob/v1.7.0/LICENSE#L3) file This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm edited a comment on pull request #821: MINIFICPP-1251 - Implement and test RetryFlowFile processor
szaszm edited a comment on pull request #821: URL: https://github.com/apache/nifi-minifi-cpp/pull/821#issuecomment-660013389 > Looks good to me, thanks! > > @adamdebreceni @szaszm could you get back to this and check if your comments are properly replied/handled? I only left a single reply comment, no reviews. I got a reply, so don't feel blocked by me. :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on pull request #821: MINIFICPP-1251 - Implement and test RetryFlowFile processor
szaszm commented on pull request #821: URL: https://github.com/apache/nifi-minifi-cpp/pull/821#issuecomment-660013389 > Looks good to me, thanks! > > @adamdebreceni @szaszm could you get back to this and check if your comments are properly replied/handled? I only left a single reply comment, no reviews. I got a reply, so don't feel blocked by me. :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] pvillard31 commented on a change in pull request #4321: NIFI-7506 - CompressContent Add Snappy-Hadoop
pvillard31 commented on a change in pull request #4321:
URL: https://github.com/apache/nifi/pull/4321#discussion_r456345824
##
File path:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/CompressContent.java
##
@@ -124,7 +126,7 @@
.build();
public static final PropertyDescriptor MODE = new
PropertyDescriptor.Builder()
.name("Mode")
-.description("Indicates whether the processor should compress content or
decompress content. Must be either 'compress' or 'decompress'")
+.description("Indicates whether the processor should compress content or
decompress content. Must be either 'compress' or 'decompress'. Data that is
compressed with Snappy Hadoop can not be decomressed using this processor.")
Review comment:
Instead of updating the description, could we add a customValidate
method to make the processor invalid in case both SNAPPY HADOOP is selected for
the compression format and DECOMPRESS is selected for the mode?
##
File path:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/CompressContent.java
##
@@ -124,7 +126,7 @@
.build();
public static final PropertyDescriptor MODE = new
PropertyDescriptor.Builder()
.name("Mode")
-.description("Indicates whether the processor should compress content or
decompress content. Must be either 'compress' or 'decompress'")
+.description("Indicates whether the processor should compress content or
decompress content. Must be either 'compress' or 'decompress'. Data that is
compressed with Snappy Hadoop can not be decomressed using this processor.")
Review comment:
```suggestion
.description("Indicates whether the processor should compress content or
decompress content. Must be either 'compress' or 'decompress'. Data that is
compressed with Snappy Hadoop can not be decompressed using this processor.")
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] asfgit closed pull request #4366: NIFI-6603 Add cancel button to the variables.
asfgit closed pull request #4366: URL: https://github.com/apache/nifi/pull/4366 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-6603) Variables: There is no way to cancel creating a new variable unless a variable name is provided.
[ https://issues.apache.org/jira/browse/NIFI-6603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17159834#comment-17159834 ] ASF subversion and git services commented on NIFI-6603: --- Commit 950437b2d77958d9a1e4e5191c79ed10cee08455 in nifi's branch refs/heads/main from Vasily Makarov [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=950437b ] NIFI-6603 Add cancel button to the variables. Signed-off-by: Pierre Villard This closes #4366. > Variables: There is no way to cancel creating a new variable unless a > variable name is provided. > > > Key: NIFI-6603 > URL: https://issues.apache.org/jira/browse/NIFI-6603 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Andrew M. Lim >Assignee: Makarov Vasiliy Nicolaevich >Priority: Minor > Time Spent: 1h 40m > Remaining Estimate: 0h > > Open the variables window and select "+" . In the New Variable window do not > enter a value for the Variable Name field and select "OK". A Configuration > Error dialog is shown, select "OK". There is no way for the user to stop > variable creation without entering a name and then selecting Cancel from the > Variables window. We should add a "Cancel" button to the "New Variable" > window. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-6603) Variables: There is no way to cancel creating a new variable unless a variable name is provided.
[ https://issues.apache.org/jira/browse/NIFI-6603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-6603: - Fix Version/s: 1.12.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Variables: There is no way to cancel creating a new variable unless a > variable name is provided. > > > Key: NIFI-6603 > URL: https://issues.apache.org/jira/browse/NIFI-6603 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Andrew M. Lim >Assignee: Makarov Vasiliy Nicolaevich >Priority: Minor > Fix For: 1.12.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Open the variables window and select "+" . In the New Variable window do not > enter a value for the Variable Name field and select "OK". A Configuration > Error dialog is shown, select "OK". There is no way for the user to stop > variable creation without entering a name and then selecting Cancel from the > Variables window. We should add a "Cancel" button to the "New Variable" > window. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] pvillard31 commented on pull request #4366: NIFI-6603 Add cancel button to the variables.
pvillard31 commented on pull request #4366: URL: https://github.com/apache/nifi/pull/4366#issuecomment-660007015 Confirmed the change, thanks for the improvement @feitgraph, merging to main. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] pvillard31 commented on pull request #4355: QueryElasticsearchHttp to support nested JSON objects
pvillard31 commented on pull request #4355: URL: https://github.com/apache/nifi/pull/4355#issuecomment-660003545 Is there a JIRA for this? If not, can you create one? https://issues.apache.org/jira/browse/NIFI This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-7582) Move Kafka 0.9 and 0.10 support to optional profiles to conserve space
[ https://issues.apache.org/jira/browse/NIFI-7582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17159815#comment-17159815 ] Pierre Villard commented on NIFI-7582: -- Updated the [migration guide|https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance] to reflect the change. > Move Kafka 0.9 and 0.10 support to optional profiles to conserve space > -- > > Key: NIFI-7582 > URL: https://issues.apache.org/jira/browse/NIFI-7582 > Project: Apache NiFi > Issue Type: Task > Components: Extensions, Tools and Build >Reporter: Mike Thomsen >Assignee: Mike Thomsen >Priority: Major > Fix For: 1.12.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (NIFI-7582) Move Kafka 0.9 and 0.10 support to optional profiles to conserve space
[ https://issues.apache.org/jira/browse/NIFI-7582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-7582. -- Fix Version/s: 1.12.0 Resolution: Fixed > Move Kafka 0.9 and 0.10 support to optional profiles to conserve space > -- > > Key: NIFI-7582 > URL: https://issues.apache.org/jira/browse/NIFI-7582 > Project: Apache NiFi > Issue Type: Task > Components: Extensions, Tools and Build >Reporter: Mike Thomsen >Assignee: Mike Thomsen >Priority: Major > Fix For: 1.12.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-7582) Move Kafka 0.9 and 0.10 support to optional profiles to conserve space
[ https://issues.apache.org/jira/browse/NIFI-7582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-7582: - Component/s: Tools and Build Extensions > Move Kafka 0.9 and 0.10 support to optional profiles to conserve space > -- > > Key: NIFI-7582 > URL: https://issues.apache.org/jira/browse/NIFI-7582 > Project: Apache NiFi > Issue Type: Task > Components: Extensions, Tools and Build >Reporter: Mike Thomsen >Assignee: Mike Thomsen >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7582) Move Kafka 0.9 and 0.10 support to optional profiles to conserve space
[ https://issues.apache.org/jira/browse/NIFI-7582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17159813#comment-17159813 ] ASF subversion and git services commented on NIFI-7582: --- Commit b8d51333d593fe2decade72079b805006dbdfb6a in nifi's branch refs/heads/main from Mike Thomsen [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=b8d5133 ] NIFI-7582 Added profile builds for kafka 0.9 and 0.10 so they can be removed from the convenience binary. Signed-off-by: Pierre Villard This closes #4364. > Move Kafka 0.9 and 0.10 support to optional profiles to conserve space > -- > > Key: NIFI-7582 > URL: https://issues.apache.org/jira/browse/NIFI-7582 > Project: Apache NiFi > Issue Type: Task >Reporter: Mike Thomsen >Assignee: Mike Thomsen >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #4364: NIFI-7582 Added profile builds for kafka 0.9 and 0.10 so they can be …
asfgit closed pull request #4364: URL: https://github.com/apache/nifi/pull/4364 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] pvillard31 commented on pull request #4364: NIFI-7582 Added profile builds for kafka 0.9 and 0.10 so they can be …
pvillard31 commented on pull request #4364: URL: https://github.com/apache/nifi/pull/4364#issuecomment-659998761 Merged to main, thanks @MikeThomsen This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-7493) XML Schema Inference can infer a type of String when it should be Record
[
https://issues.apache.org/jira/browse/NIFI-7493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-7493:
-
Resolution: Fixed
Status: Resolved (was: Patch Available)
> XML Schema Inference can infer a type of String when it should be Record
>
>
> Key: NIFI-7493
> URL: https://issues.apache.org/jira/browse/NIFI-7493
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.12.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> From the mailing list:
> {quote}I have configured a XMLReader to use the Infer Schema. The other issue
> is that I have problems converting sub records. My records looks something
> like this: John Doe
> some there
> workingman
> New York
> A Company
>
>
>
>
> The issues are with the subrecords in part 3. I have configured the XMLReader
> property "Field Name for Content" = value
>
> When the data is being converted via a XMLWriter the output for the
> additionalInfo fields looks like this:
> MapRecord[\{name=Location,
> value=New York}]
> MapRecord[\{name=Company, value=A
> Company}]
>
>
>
> If I use a JSONWriter I gets this:
> "Part3": { "Details": {
> "additionalInfo": [ "MapRecord[\{name=Location, value=New York}]",
> "MapRecord[\{name=Company, value=A Company}]" ]
> }
> }{quote}
> The issue appears to be that "additionalInfo" is being inferred as a String,
> but the XML Reader is returning a Record.
>
> This is probably because the "additionalInfo" element contains String
> content and no child nodes. However, it does have attributes. As a result,
> the XML Reader will return a Record. I'm guessing that attributes are not
> taken into account in the schema inference, though, and since
> "additionalInfo" has no child nodes but has textual content, it must be a
> String.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-7493) XML Schema Inference can infer a type of String when it should be Record
[
https://issues.apache.org/jira/browse/NIFI-7493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17159810#comment-17159810
]
ASF subversion and git services commented on NIFI-7493:
---
Commit 7e09e0db339a07670eba47c138959d79663400ee in nifi's branch
refs/heads/main from Mark Payne
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=7e09e0d ]
NIFI-7493: When inferring schema for XML data, if we find a text element that
also has attributes, infer it as a Record type, in order to match how the data
will be read when using the XML Reader
Signed-off-by: Pierre Villard
This closes #4375.
> XML Schema Inference can infer a type of String when it should be Record
>
>
> Key: NIFI-7493
> URL: https://issues.apache.org/jira/browse/NIFI-7493
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.12.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> From the mailing list:
> {quote}I have configured a XMLReader to use the Infer Schema. The other issue
> is that I have problems converting sub records. My records looks something
> like this: John Doe
> some there
> workingman
> New York
> A Company
>
>
>
>
> The issues are with the subrecords in part 3. I have configured the XMLReader
> property "Field Name for Content" = value
>
> When the data is being converted via a XMLWriter the output for the
> additionalInfo fields looks like this:
> MapRecord[\{name=Location,
> value=New York}]
> MapRecord[\{name=Company, value=A
> Company}]
>
>
>
> If I use a JSONWriter I gets this:
> "Part3": { "Details": {
> "additionalInfo": [ "MapRecord[\{name=Location, value=New York}]",
> "MapRecord[\{name=Company, value=A Company}]" ]
> }
> }{quote}
> The issue appears to be that "additionalInfo" is being inferred as a String,
> but the XML Reader is returning a Record.
>
> This is probably because the "additionalInfo" element contains String
> content and no child nodes. However, it does have attributes. As a result,
> the XML Reader will return a Record. I'm guessing that attributes are not
> taken into account in the schema inference, though, and since
> "additionalInfo" has no child nodes but has textual content, it must be a
> String.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] pvillard31 commented on pull request #4375: NIFI-7493: When inferring schema for XML data, if we find a text elem…
pvillard31 commented on pull request #4375: URL: https://github.com/apache/nifi/pull/4375#issuecomment-659996855 Merged to main, thanks @markap14 ! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] asfgit closed pull request #4375: NIFI-7493: When inferring schema for XML data, if we find a text elem…
asfgit closed pull request #4375: URL: https://github.com/apache/nifi/pull/4375 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-7654) Deprecate Client Auth property on AMQP processors
[ https://issues.apache.org/jira/browse/NIFI-7654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-7654: - Fix Version/s: 1.12.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Deprecate Client Auth property on AMQP processors > - > > Key: NIFI-7654 > URL: https://issues.apache.org/jira/browse/NIFI-7654 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Major > Fix For: 1.12.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > 'Client Auth' property has no effect and is obsolete. There is another > property - 'Use Certificate Authentication' - to turn on client certificate > authentication. > User Name/Password properties should not be mandatory when client certificate > authentication is turned on. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] pvillard31 commented on pull request #4412: NIFI-7654: Deprecated Client Auth property on AMQP processors
pvillard31 commented on pull request #4412: URL: https://github.com/apache/nifi/pull/4412#issuecomment-659993937 Went through the code, it LGTM and the changes make totale sense. Merging to main, thanks @turcsanyip This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-7654) Deprecate Client Auth property on AMQP processors
[ https://issues.apache.org/jira/browse/NIFI-7654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-7654: - Component/s: Extensions > Deprecate Client Auth property on AMQP processors > - > > Key: NIFI-7654 > URL: https://issues.apache.org/jira/browse/NIFI-7654 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Major > Fix For: 1.12.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > 'Client Auth' property has no effect and is obsolete. There is another > property - 'Use Certificate Authentication' - to turn on client certificate > authentication. > User Name/Password properties should not be mandatory when client certificate > authentication is turned on. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7654) Deprecate Client Auth property on AMQP processors
[ https://issues.apache.org/jira/browse/NIFI-7654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17159805#comment-17159805 ] ASF subversion and git services commented on NIFI-7654: --- Commit 21c3085d15a75ee2699e3cce06518125bf45202f in nifi's branch refs/heads/main from Peter Turcsanyi [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=21c3085 ] NIFI-7654: Deprecated Client Auth property on AMQP processors Signed-off-by: Pierre Villard This closes #4412. > Deprecate Client Auth property on AMQP processors > - > > Key: NIFI-7654 > URL: https://issues.apache.org/jira/browse/NIFI-7654 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Peter Turcsanyi >Assignee: Peter Turcsanyi >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > 'Client Auth' property has no effect and is obsolete. There is another > property - 'Use Certificate Authentication' - to turn on client certificate > authentication. > User Name/Password properties should not be mandatory when client certificate > authentication is turned on. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #4412: NIFI-7654: Deprecated Client Auth property on AMQP processors
asfgit closed pull request #4412: URL: https://github.com/apache/nifi/pull/4412 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] pvillard31 commented on pull request #4392: NIFI-7608 Add CLI command to replace contents of a process group
pvillard31 commented on pull request #4392: URL: https://github.com/apache/nifi/pull/4392#issuecomment-659992844 Merged to main, thanks @bbende This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-7608) Create CLI command for replacing process group content
[ https://issues.apache.org/jira/browse/NIFI-7608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-7608: - Fix Version/s: 1.12.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Create CLI command for replacing process group content > -- > > Key: NIFI-7608 > URL: https://issues.apache.org/jira/browse/NIFI-7608 > Project: Apache NiFi > Issue Type: Improvement > Components: Flow Versioning, SDLC, Tools and Build >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Minor > Fix For: 1.12.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Create a CLI command that uses the /replace-requests endpoint in the > ProcessGroupResource. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-7608) Create CLI command for replacing process group content
[ https://issues.apache.org/jira/browse/NIFI-7608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-7608: - Component/s: Tools and Build SDLC Flow Versioning > Create CLI command for replacing process group content > -- > > Key: NIFI-7608 > URL: https://issues.apache.org/jira/browse/NIFI-7608 > Project: Apache NiFi > Issue Type: Improvement > Components: Flow Versioning, SDLC, Tools and Build >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Minor > Time Spent: 20m > Remaining Estimate: 0h > > Create a CLI command that uses the /replace-requests endpoint in the > ProcessGroupResource. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7608) Create CLI command for replacing process group content
[ https://issues.apache.org/jira/browse/NIFI-7608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17159801#comment-17159801 ] ASF subversion and git services commented on NIFI-7608: --- Commit f6c2824f74990f22da4fc1f2e73607daa3fd00e6 in nifi's branch refs/heads/main from Bryan Bende [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=f6c2824 ] NIFI-7608 Add CLI command to replace contents of a process group Signed-off-by: Pierre Villard This closes #4392. > Create CLI command for replacing process group content > -- > > Key: NIFI-7608 > URL: https://issues.apache.org/jira/browse/NIFI-7608 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > Create a CLI command that uses the /replace-requests endpoint in the > ProcessGroupResource. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #4392: NIFI-7608 Add CLI command to replace contents of a process group
asfgit closed pull request #4392: URL: https://github.com/apache/nifi/pull/4392 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] pvillard31 commented on a change in pull request #4410: NIFI-7642: KafkaConsumers - improve batching into FlowFiles under high latency conditions
pvillard31 commented on a change in pull request #4410:
URL: https://github.com/apache/nifi/pull/4410#discussion_r456316349
##
File path:
nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-1-0-processors/src/main/java/org/apache/nifi/processors/kafka/pubsub/ConsumerLease.java
##
@@ -174,7 +177,7 @@ void poll() {
* This behavior has been fixed via Kafka KIP-62 and available from
Kafka client 0.10.1.0.
*/
try {
-final ConsumerRecords records =
kafkaConsumer.poll(10);
+final ConsumerRecords records =
kafkaConsumer.poll(maxWaitMillis / 10);
Review comment:
Why are we doing ``maxWaitMillis / 10`` instead of just
``maxWaitMillis``?
(sorry for the possibly dumb question here)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] pvillard31 commented on pull request #4405: NIFI-7633: Added FlowFileConcurrency of SINGLE_BATCH_PER_NODE to allo…
pvillard31 commented on pull request #4405: URL: https://github.com/apache/nifi/pull/4405#issuecomment-659962467 Hey @markap14 - now that I merged #4345, this PR has conflicts. Can you resolve the conflicts? Happy to review and merge this before 1.12.0 release. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-7552) Add "batch.output.XYZ" attribute when Process Group is using batch output mode
[ https://issues.apache.org/jira/browse/NIFI-7552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-7552: - Resolution: Fixed Status: Resolved (was: Patch Available) > Add "batch.output.XYZ" attribute when Process Group is using batch output mode > -- > > Key: NIFI-7552 > URL: https://issues.apache.org/jira/browse/NIFI-7552 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.12.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > When a Process Group is configured to use a FlowFile Concurrency of Single > FlowFile per Node and an Outbound Policy of Batch Output, it would be helpful > to know how many FlowFiles are routed to each of the Output Ports. This is > often needed in order to detect whether or not anything in the input failed, > etc. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7552) Add "batch.output.XYZ" attribute when Process Group is using batch output mode
[ https://issues.apache.org/jira/browse/NIFI-7552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17159771#comment-17159771 ] ASF subversion and git services commented on NIFI-7552: --- Commit 44fc4d9f27cad664186e1b7a397a785dd90644ec in nifi's branch refs/heads/main from Mark Payne [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=44fc4d9 ] NIFI-7552: When Process Group is configured to transfer data in batch, add an attribute to each outbound FlowFile that indicates how many FlowFiles went to each port. Updated user guide to explain the new attributes. Signed-off-by: Pierre Villard This closes #4345. > Add "batch.output.XYZ" attribute when Process Group is using batch output mode > -- > > Key: NIFI-7552 > URL: https://issues.apache.org/jira/browse/NIFI-7552 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.12.0 > > Time Spent: 20m > Remaining Estimate: 0h > > When a Process Group is configured to use a FlowFile Concurrency of Single > FlowFile per Node and an Outbound Policy of Batch Output, it would be helpful > to know how many FlowFiles are routed to each of the Output Ports. This is > often needed in order to detect whether or not anything in the input failed, > etc. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #4345: NIFI-7552: When Process Group is configured to transfer data in batch…
asfgit closed pull request #4345: URL: https://github.com/apache/nifi/pull/4345 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] pvillard31 commented on pull request #4345: NIFI-7552: When Process Group is configured to transfer data in batch…
pvillard31 commented on pull request #4345: URL: https://github.com/apache/nifi/pull/4345#issuecomment-659958025 I played with this for a bit and it behaved as expected. Merging to main, thanks @markap14 ! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #821: MINIFICPP-1251 - Implement and test RetryFlowFile processor
hunyadi-dev commented on a change in pull request #821:
URL: https://github.com/apache/nifi-minifi-cpp/pull/821#discussion_r456291644
##
File path: libminifi/include/core/PropertyValidation.h
##
@@ -259,6 +259,22 @@ class UnsignedLongValidator : public PropertyValidator {
}
};
+class NonBlankValidator : public PropertyValidator {
+ public:
+ explicit NonBlankValidator(const std::string& name)
+ : PropertyValidator(name) {
+ }
+ ~NonBlankValidator() override = default;
+
+ ValidationResult validate(const std::string& subject, const
std::shared_ptr& input) const final {
+return validate(subject, input->getStringValue());
+ }
+
+ ValidationResult validate(const std::string& subject, const std::string&
input) const final {
+return
ValidationResult::Builder::createBuilder().withSubject(subject).withInput(input).isValid(utils::StringUtils::trimLeft(input).size()).build();
Review comment:
Should not be a factor in readibility.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #821: MINIFICPP-1251 - Implement and test RetryFlowFile processor
adamdebreceni commented on a change in pull request #821:
URL: https://github.com/apache/nifi-minifi-cpp/pull/821#discussion_r456276773
##
File path: libminifi/include/core/PropertyValidation.h
##
@@ -259,6 +259,22 @@ class UnsignedLongValidator : public PropertyValidator {
}
};
+class NonBlankValidator : public PropertyValidator {
+ public:
+ explicit NonBlankValidator(const std::string& name)
+ : PropertyValidator(name) {
+ }
+ ~NonBlankValidator() override = default;
+
+ ValidationResult validate(const std::string& subject, const
std::shared_ptr& input) const final {
+return validate(subject, input->getStringValue());
+ }
+
+ ValidationResult validate(const std::string& subject, const std::string&
input) const final {
+return
ValidationResult::Builder::createBuilder().withSubject(subject).withInput(input).isValid(utils::StringUtils::trimLeft(input).size()).build();
Review comment:
I would prefer and explicit `.size() != 0`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #837: MINIFICPP-1121 - Upgrade spdlog to version 1.7.0
hunyadi-dev commented on a change in pull request #837: URL: https://github.com/apache/nifi-minifi-cpp/pull/837#discussion_r456271769 ## File path: LICENSE ## @@ -258,23 +258,27 @@ This product bundles 'spdlog' which is available under an MIT license. Copyright (c) 2016 spdlog. - Permission is hereby granted, free of charge, to any person obtaining a copy - of this software and associated documentation files (the "Software"), to deal - in the Software without restriction, including without limitation the rights - to use, copy, modify, merge, publish, distribute, sublicense, and/or sell - copies of the Software, and to permit persons to whom the Software is - furnished to do so, subject to the following conditions: - - The above copyright notice and this permission notice shall be included in - all copies or substantial portions of the Software. - - THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR - IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, - FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE - AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER - LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, - OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN - THE SOFTWARE. + Permission is hereby granted, free of charge, to any person obtaining a copy + of this software and associated documentation files (the "Software"), to deal + in the Software without restriction, including without limitation the rights + to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + copies of the Software, and to permit persons to whom the Software is + furnished to do so, subject to the following conditions: + + The above copyright notice and this permission notice shall be included in + all copies or substantial portions of the Software. + + THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + THE SOFTWARE. + + -- NOTE: Third party dependency used by this software -- + This software depends on the fmt lib (MIT License), + and users must comply to its license: https://github.com/fmtlib/fmt/blob/master/LICENSE.rst Review comment: I am not exactly sure how the first comments first bullet differs from the second comment. Updated the `NOTICE` and added a section for fmt in this `LICENSE` file. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (MINIFICPP-1293) PropertyTests fails on Windows in time zones east of Greenwich
[
https://issues.apache.org/jira/browse/MINIFICPP-1293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ferenc Gerlits resolved MINIFICPP-1293.
---
Resolution: Fixed
> PropertyTests fails on Windows in time zones east of Greenwich
> --
>
> Key: MINIFICPP-1293
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1293
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
>Reporter: Ferenc Gerlits
>Assignee: Ferenc Gerlits
>Priority: Minor
> Fix For: 0.8.0
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> In the "Test DateTime Conversion" in PropertyTests.cpp, the test
> {noformat}
> int64_t timestamp = 0LL;
> REQUIRE(true ==
> org::apache::nifi::minifi::core::Property::StringToDateTime("1970-01-01T00:00:00Z",
> timestamp));
> REQUIRE(0LL == timestamp);
> {noformat}
> fails, because {{StringToDateTime}} uses {{mktime}} internally to convert a
> UTC {{stuct tm}} to a localtime {{time_h}}, and then adjust it back to UTC by
> adding a time zone offset.
> "1970-01-01T00:00:00" in local time east of Greenwich results in a negative
> {{time_h}} value, which is not officially supported by {{mktime}}. On
> Windows, {{mktime}} returns -1 to signal an error, so the test fails. On
> Linux, if the time zone is UTC+1, {{mktime}} returns -3600, so the test
> passes; however, if we were to test "1970-01-01T00:59:59", then {{mktime}}
> would have to return -1 as the answer, which looks like an error, so this
> test would fail.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #807: MINIFICPP-1228 - Resource ownership refactor + flowFile-owning processors.
adamdebreceni commented on a change in pull request #807:
URL: https://github.com/apache/nifi-minifi-cpp/pull/807#discussion_r456267761
##
File path: libminifi/include/core/FlowFile.h
##
@@ -35,9 +35,56 @@ namespace minifi {
namespace core {
class FlowFile : public core::Connectable, public ReferenceContainer {
+ private:
+ class FlowFileOwnedResourceClaimPtr{
+ public:
+FlowFileOwnedResourceClaimPtr() = default;
+explicit FlowFileOwnedResourceClaimPtr(const
std::shared_ptr& claim) : claim_(claim) {
+ if (claim_) claim_->increaseFlowFileRecordOwnedCount();
+}
+explicit FlowFileOwnedResourceClaimPtr(std::shared_ptr&&
claim) : claim_(std::move(claim)) {
+ if (claim_) claim_->increaseFlowFileRecordOwnedCount();
+}
+FlowFileOwnedResourceClaimPtr(const FlowFileOwnedResourceClaimPtr& ref) :
claim_(ref.claim_) {
+ if (claim_) claim_->increaseFlowFileRecordOwnedCount();
+}
+FlowFileOwnedResourceClaimPtr(FlowFileOwnedResourceClaimPtr&& ref) :
claim_(std::move(ref.claim_)) {
+ // taking ownership of claim, no need to increment/decrement
+}
+FlowFileOwnedResourceClaimPtr& operator=(const
FlowFileOwnedResourceClaimPtr& ref) = delete;
+FlowFileOwnedResourceClaimPtr& operator=(FlowFileOwnedResourceClaimPtr&&
ref) = delete;
+
+FlowFileOwnedResourceClaimPtr& set(FlowFile& owner, const
FlowFileOwnedResourceClaimPtr& ref) {
+ return set(owner, ref.claim_);
+}
+FlowFileOwnedResourceClaimPtr& set(FlowFile& owner, const
std::shared_ptr& newClaim) {
+ auto oldClaim = claim_;
+ claim_ = newClaim;
+ // the order of increase/release is important
+ if (claim_) claim_->increaseFlowFileRecordOwnedCount();
+ if (oldClaim) owner.releaseClaim(oldClaim);
Review comment:
added comment
##
File path: libminifi/include/core/FlowFile.h
##
@@ -35,9 +36,58 @@ namespace minifi {
namespace core {
class FlowFile : public core::Connectable, public ReferenceContainer {
+ private:
+ class FlowFileOwnedResourceClaimPtr{
+ public:
+FlowFileOwnedResourceClaimPtr() = default;
+explicit FlowFileOwnedResourceClaimPtr(const
std::shared_ptr& claim) : claim_(claim) {
+ if (claim_) claim_->increaseFlowFileRecordOwnedCount();
+}
+explicit FlowFileOwnedResourceClaimPtr(std::shared_ptr&&
claim) : claim_(std::move(claim)) {
+ if (claim_) claim_->increaseFlowFileRecordOwnedCount();
+}
+FlowFileOwnedResourceClaimPtr(const FlowFileOwnedResourceClaimPtr& ref) :
claim_(ref.claim_) {
+ if (claim_) claim_->increaseFlowFileRecordOwnedCount();
+}
Review comment:
yep, added
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (MINIFICPP-1297) TestSuite called "Tests" should have a more talkative name
[ https://issues.apache.org/jira/browse/MINIFICPP-1297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpad Boda updated MINIFICPP-1297: -- Labels: MiNiFi-CPP-Hygiene (was: ) > TestSuite called "Tests" should have a more talkative name > -- > > Key: MINIFICPP-1297 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1297 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 0.7.0 >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Minor > Labels: MiNiFi-CPP-Hygiene > Fix For: 0.8.0 > > Time Spent: 10m > Remaining Estimate: 0h > > The current name is not talkative enough, should reflect what it was made > for. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1145) Create a common RocksdbRepository baseclass
[ https://issues.apache.org/jira/browse/MINIFICPP-1145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpad Boda updated MINIFICPP-1145: -- Labels: MiNiFi-CPP-Hygiene (was: ) > Create a common RocksdbRepository baseclass > --- > > Key: MINIFICPP-1145 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1145 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 0.7.0 >Reporter: Arpad Boda >Assignee: Arpad Boda >Priority: Major > Labels: MiNiFi-CPP-Hygiene > Fix For: 0.8.0 > > > As MINIFICPP-1126 and MINIFICPP-1127 highlighted there is a lot in common in > Provenance and FlowFile repositories. > Rocksdb-based content repo and the planned rocksdb-based processor state > storage would also result in some code duplication. > printStats should be moved to the baseclass and properly handle failed getter > calls. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1298) Improve test coverage of TimeUtil.h and move it to a proper namespace
[ https://issues.apache.org/jira/browse/MINIFICPP-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpad Boda updated MINIFICPP-1298: -- Labels: MiNiFi-CPP-Hygiene (was: ) > Improve test coverage of TimeUtil.h and move it to a proper namespace > - > > Key: MINIFICPP-1298 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1298 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 0.7.0 >Reporter: Arpad Boda >Priority: Minor > Labels: MiNiFi-CPP-Hygiene > > See summary -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (MINIFICPP-1298) Improve test coverage of TimeUtil.h and move it to a proper namespace
Arpad Boda created MINIFICPP-1298: - Summary: Improve test coverage of TimeUtil.h and move it to a proper namespace Key: MINIFICPP-1298 URL: https://issues.apache.org/jira/browse/MINIFICPP-1298 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Affects Versions: 0.7.0 Reporter: Arpad Boda See summary -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] arpadboda opened a new pull request #841: MINIFICPP-1297 - TestSuite called \
arpadboda opened a new pull request #841: URL: https://github.com/apache/nifi-minifi-cpp/pull/841 Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
