[jira] [Commented] (NIFI-1002) support for Listen WebSocket processor
[ https://issues.apache.org/jira/browse/NIFI-1002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15666346#comment-15666346 ] ASF GitHub Bot commented on NIFI-1002: -- Github user ijokarumawak commented on the issue: https://github.com/apache/nifi/pull/1184 @olegz Thanks again for the great review! I rebased the PR and done additional refactoring based on your feedback. Please let me know if you prefer it to be squashed for further review process. > support for Listen WebSocket processor > --- > > Key: NIFI-1002 > URL: https://issues.apache.org/jira/browse/NIFI-1002 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 0.4.0 >Reporter: sumanth chinthagunta >Priority: Minor > Labels: newbie > > A WebSocket listen processor will be helpful for IoT data ingestion. > I am playing with embedded Vert.X for WebSocket and also ability to put > FlowFiles back to WebSocket client via Vert.X EventBus. > https://github.com/xmlking/nifi-websocket > I am new to NiFi. any advise can be helpful. > PS: I feel forcing Interfaces for Controller Services is unnecessary as in > many cases Controller Services are only used by a set of Processors and > developers usually bundle them together. > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi issue #1184: NIFI-1002: Added WebSocket support.
Github user ijokarumawak commented on the issue: https://github.com/apache/nifi/pull/1184 @olegz Thanks again for the great review! I rebased the PR and done additional refactoring based on your feedback. Please let me know if you prefer it to be squashed for further review process. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-1002) support for Listen WebSocket processor
[ https://issues.apache.org/jira/browse/NIFI-1002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15666340#comment-15666340 ] ASF GitHub Bot commented on NIFI-1002: -- Github user ijokarumawak commented on a diff in the pull request: https://github.com/apache/nifi/pull/1184#discussion_r87958885 --- Diff: nifi-nar-bundles/nifi-websocket-bundle/nifi-websocket-processors/src/main/java/org/apache/nifi/processors/websocket/AbstractWebSocketGatewayProcessor.java --- @@ -0,0 +1,314 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.nifi.processors.websocket; + +import org.apache.commons.lang3.StringUtils; +import org.apache.nifi.annotation.behavior.TriggerSerially; +import org.apache.nifi.annotation.lifecycle.OnStopped; +import org.apache.nifi.annotation.lifecycle.OnUnscheduled; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.components.ValidationContext; +import org.apache.nifi.components.ValidationResult; +import org.apache.nifi.flowfile.FlowFile; +import org.apache.nifi.flowfile.attributes.CoreAttributes; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.ProcessSession; +import org.apache.nifi.processor.Relationship; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.util.StandardValidators; +import org.apache.nifi.websocket.BinaryMessageConsumer; +import org.apache.nifi.websocket.ConnectedListener; +import org.apache.nifi.websocket.TextMessageConsumer; +import org.apache.nifi.websocket.WebSocketClientService; +import org.apache.nifi.websocket.WebSocketConfigurationException; +import org.apache.nifi.websocket.WebSocketConnectedMessage; +import org.apache.nifi.websocket.WebSocketMessage; +import org.apache.nifi.websocket.WebSocketService; +import org.apache.nifi.websocket.WebSocketSessionInfo; + +import java.io.IOException; +import java.util.ArrayList; +import java.util.Collection; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.concurrent.LinkedBlockingQueue; + +@TriggerSerially +public abstract class AbstractWebSocketGatewayProcessor extends AbstractWebSocketProcessor implements ConnectedListener, TextMessageConsumer, BinaryMessageConsumer { + +public static final PropertyDescriptor PROP_MAX_QUEUE_SIZE = new PropertyDescriptor.Builder() +.name("max-queue-size") +.displayName("Max Queue Size") +.description("The WebSocket messages are kept in an on-memory queue," + +" then transferred to relationships when this processor is triggered." + +" If the 'Run Schedule' is significantly behind the rate" + +" at which the messages are arriving to this processor then a back up can occur." + +" This property specifies the maximum number of messages this processor will hold in memory at one time." + +" CAUTION: Any incoming WebSocket message arrived while the queue being full" + +" will be discarded and a warning message will be logged.") +.required(true) +.addValidator(StandardValidators.POSITIVE_INTEGER_VALIDATOR) +.defaultValue("1") +.build(); + +private volatile LinkedBlockingQueue incomingMessageQueue; --- End diff -- Thank you for bringing this up. I had been worrying about this queue, too. To make it simpler, I removed the intermediate queue, and let incoming messages to be transferred to NiFi relationships. > support for Listen WebSocket processor > --- > > Key: NIFI-1002 > URL: https://issues.apache.org/jira/browse/NIFI-1002 > Project
[GitHub] nifi pull request #1184: NIFI-1002: Added WebSocket support.
Github user ijokarumawak commented on a diff in the pull request: https://github.com/apache/nifi/pull/1184#discussion_r87958885 --- Diff: nifi-nar-bundles/nifi-websocket-bundle/nifi-websocket-processors/src/main/java/org/apache/nifi/processors/websocket/AbstractWebSocketGatewayProcessor.java --- @@ -0,0 +1,314 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.nifi.processors.websocket; + +import org.apache.commons.lang3.StringUtils; +import org.apache.nifi.annotation.behavior.TriggerSerially; +import org.apache.nifi.annotation.lifecycle.OnStopped; +import org.apache.nifi.annotation.lifecycle.OnUnscheduled; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.components.ValidationContext; +import org.apache.nifi.components.ValidationResult; +import org.apache.nifi.flowfile.FlowFile; +import org.apache.nifi.flowfile.attributes.CoreAttributes; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.ProcessSession; +import org.apache.nifi.processor.Relationship; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.util.StandardValidators; +import org.apache.nifi.websocket.BinaryMessageConsumer; +import org.apache.nifi.websocket.ConnectedListener; +import org.apache.nifi.websocket.TextMessageConsumer; +import org.apache.nifi.websocket.WebSocketClientService; +import org.apache.nifi.websocket.WebSocketConfigurationException; +import org.apache.nifi.websocket.WebSocketConnectedMessage; +import org.apache.nifi.websocket.WebSocketMessage; +import org.apache.nifi.websocket.WebSocketService; +import org.apache.nifi.websocket.WebSocketSessionInfo; + +import java.io.IOException; +import java.util.ArrayList; +import java.util.Collection; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.concurrent.LinkedBlockingQueue; + +@TriggerSerially +public abstract class AbstractWebSocketGatewayProcessor extends AbstractWebSocketProcessor implements ConnectedListener, TextMessageConsumer, BinaryMessageConsumer { + +public static final PropertyDescriptor PROP_MAX_QUEUE_SIZE = new PropertyDescriptor.Builder() +.name("max-queue-size") +.displayName("Max Queue Size") +.description("The WebSocket messages are kept in an on-memory queue," + +" then transferred to relationships when this processor is triggered." + +" If the 'Run Schedule' is significantly behind the rate" + +" at which the messages are arriving to this processor then a back up can occur." + +" This property specifies the maximum number of messages this processor will hold in memory at one time." + +" CAUTION: Any incoming WebSocket message arrived while the queue being full" + +" will be discarded and a warning message will be logged.") +.required(true) +.addValidator(StandardValidators.POSITIVE_INTEGER_VALIDATOR) +.defaultValue("1") +.build(); + +private volatile LinkedBlockingQueue incomingMessageQueue; --- End diff -- Thank you for bringing this up. I had been worrying about this queue, too. To make it simpler, I removed the intermediate queue, and let incoming messages to be transferred to NiFi relationships. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-1002) support for Listen WebSocket processor
[ https://issues.apache.org/jira/browse/NIFI-1002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15666337#comment-15666337 ] ASF GitHub Bot commented on NIFI-1002: -- Github user ijokarumawak commented on a diff in the pull request: https://github.com/apache/nifi/pull/1184#discussion_r87958669 --- Diff: nifi-nar-bundles/nifi-websocket-bundle/nifi-websocket-services-jetty/src/main/java/org/apache/nifi/websocket/jetty/JettyWebSocketServer.java --- @@ -0,0 +1,258 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.nifi.websocket.jetty; + +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnDisabled; +import org.apache.nifi.annotation.lifecycle.OnEnabled; +import org.apache.nifi.annotation.lifecycle.OnShutdown; +import org.apache.nifi.components.AllowableValue; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.controller.ConfigurationContext; +import org.apache.nifi.nar.NarCloseable; +import org.apache.nifi.processor.util.StandardValidators; +import org.apache.nifi.ssl.SSLContextService; +import org.apache.nifi.websocket.WebSocketConfigurationException; +import org.apache.nifi.websocket.WebSocketMessageRouter; +import org.apache.nifi.websocket.WebSocketServerService; +import org.eclipse.jetty.server.Connector; +import org.eclipse.jetty.server.Handler; +import org.eclipse.jetty.server.HttpConfiguration; +import org.eclipse.jetty.server.HttpConnectionFactory; +import org.eclipse.jetty.server.SecureRequestCustomizer; +import org.eclipse.jetty.server.Server; +import org.eclipse.jetty.server.ServerConnector; +import org.eclipse.jetty.server.SslConnectionFactory; +import org.eclipse.jetty.server.handler.ContextHandlerCollection; +import org.eclipse.jetty.servlet.ServletContextHandler; +import org.eclipse.jetty.servlet.ServletHandler; +import org.eclipse.jetty.util.ssl.SslContextFactory; +import org.eclipse.jetty.websocket.api.Session; +import org.eclipse.jetty.websocket.api.WebSocketPolicy; +import org.eclipse.jetty.websocket.server.WebSocketServerFactory; +import org.eclipse.jetty.websocket.servlet.ServletUpgradeRequest; +import org.eclipse.jetty.websocket.servlet.ServletUpgradeResponse; +import org.eclipse.jetty.websocket.servlet.WebSocketCreator; +import org.eclipse.jetty.websocket.servlet.WebSocketServlet; +import org.eclipse.jetty.websocket.servlet.WebSocketServletFactory; + +import java.net.URI; +import java.util.ArrayList; +import java.util.Collections; +import java.util.List; +import java.util.Map; +import java.util.concurrent.ConcurrentHashMap; + +@Tags({"WebSocket", "Jetty", "server"}) +@CapabilityDescription("Implementation of WebSocketServerService." + +" This service uses Jetty WebSocket server module to provide" + +" WebSocket session management throughout the application.") +public class JettyWebSocketServer extends AbstractJettyWebSocketService implements WebSocketServerService { + +/** + * A global map to refer a controller service instance by requested port number. + */ +private static final Map portToControllerService = new ConcurrentHashMap<>(); + +// Allowable values for client auth +public static final AllowableValue CLIENT_NONE = new AllowableValue("no", "No Authentication", +"Processor will not authenticate clients. Anyone can communicate with this Processor anonymously"); +public static final AllowableValue CLIENT_WANT = new AllowableValue("want", "Want Authentication", +"Processor will try to verify the client but if unable to verify will allow the client to communicate anonymously"); +public static final AllowableValue CLIENT_NEED = new AllowableValue("need", "Need Authentication",
[GitHub] nifi pull request #1184: NIFI-1002: Added WebSocket support.
Github user ijokarumawak commented on a diff in the pull request: https://github.com/apache/nifi/pull/1184#discussion_r87958669 --- Diff: nifi-nar-bundles/nifi-websocket-bundle/nifi-websocket-services-jetty/src/main/java/org/apache/nifi/websocket/jetty/JettyWebSocketServer.java --- @@ -0,0 +1,258 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.nifi.websocket.jetty; + +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnDisabled; +import org.apache.nifi.annotation.lifecycle.OnEnabled; +import org.apache.nifi.annotation.lifecycle.OnShutdown; +import org.apache.nifi.components.AllowableValue; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.controller.ConfigurationContext; +import org.apache.nifi.nar.NarCloseable; +import org.apache.nifi.processor.util.StandardValidators; +import org.apache.nifi.ssl.SSLContextService; +import org.apache.nifi.websocket.WebSocketConfigurationException; +import org.apache.nifi.websocket.WebSocketMessageRouter; +import org.apache.nifi.websocket.WebSocketServerService; +import org.eclipse.jetty.server.Connector; +import org.eclipse.jetty.server.Handler; +import org.eclipse.jetty.server.HttpConfiguration; +import org.eclipse.jetty.server.HttpConnectionFactory; +import org.eclipse.jetty.server.SecureRequestCustomizer; +import org.eclipse.jetty.server.Server; +import org.eclipse.jetty.server.ServerConnector; +import org.eclipse.jetty.server.SslConnectionFactory; +import org.eclipse.jetty.server.handler.ContextHandlerCollection; +import org.eclipse.jetty.servlet.ServletContextHandler; +import org.eclipse.jetty.servlet.ServletHandler; +import org.eclipse.jetty.util.ssl.SslContextFactory; +import org.eclipse.jetty.websocket.api.Session; +import org.eclipse.jetty.websocket.api.WebSocketPolicy; +import org.eclipse.jetty.websocket.server.WebSocketServerFactory; +import org.eclipse.jetty.websocket.servlet.ServletUpgradeRequest; +import org.eclipse.jetty.websocket.servlet.ServletUpgradeResponse; +import org.eclipse.jetty.websocket.servlet.WebSocketCreator; +import org.eclipse.jetty.websocket.servlet.WebSocketServlet; +import org.eclipse.jetty.websocket.servlet.WebSocketServletFactory; + +import java.net.URI; +import java.util.ArrayList; +import java.util.Collections; +import java.util.List; +import java.util.Map; +import java.util.concurrent.ConcurrentHashMap; + +@Tags({"WebSocket", "Jetty", "server"}) +@CapabilityDescription("Implementation of WebSocketServerService." + +" This service uses Jetty WebSocket server module to provide" + +" WebSocket session management throughout the application.") +public class JettyWebSocketServer extends AbstractJettyWebSocketService implements WebSocketServerService { + +/** + * A global map to refer a controller service instance by requested port number. + */ +private static final Map portToControllerService = new ConcurrentHashMap<>(); + +// Allowable values for client auth +public static final AllowableValue CLIENT_NONE = new AllowableValue("no", "No Authentication", +"Processor will not authenticate clients. Anyone can communicate with this Processor anonymously"); +public static final AllowableValue CLIENT_WANT = new AllowableValue("want", "Want Authentication", +"Processor will try to verify the client but if unable to verify will allow the client to communicate anonymously"); +public static final AllowableValue CLIENT_NEED = new AllowableValue("need", "Need Authentication", +"Processor will reject communications from any client unless the client provides a certificate that is trusted by the TrustStore" ++ "specified in the SSL Context Service"); + +public static final Proper
[jira] [Commented] (NIFI-1002) support for Listen WebSocket processor
[ https://issues.apache.org/jira/browse/NIFI-1002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15666200#comment-15666200 ] Koji Kawamura commented on NIFI-1002: - [~bbende] It used to be unable to load classes in nar properly without this, but after rebasing my branch with the latest master, it works without this. So I removed the use of NarCloseable.withComponentNarLoader from JettyWebSocketServer. I agree that we should avoid using NarCloseable from codes like custom processors. I'll update my PR shortly. Thanks! > support for Listen WebSocket processor > --- > > Key: NIFI-1002 > URL: https://issues.apache.org/jira/browse/NIFI-1002 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 0.4.0 >Reporter: sumanth chinthagunta >Priority: Minor > Labels: newbie > > A WebSocket listen processor will be helpful for IoT data ingestion. > I am playing with embedded Vert.X for WebSocket and also ability to put > FlowFiles back to WebSocket client via Vert.X EventBus. > https://github.com/xmlking/nifi-websocket > I am new to NiFi. any advise can be helpful. > PS: I feel forcing Interfaces for Controller Services is unnecessary as in > many cases Controller Services are only used by a set of Processors and > developers usually bundle them together. > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2943) tls-toolkit pkcs12 truststore 0 entries
[ https://issues.apache.org/jira/browse/NIFI-2943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15666110#comment-15666110 ] ASF GitHub Bot commented on NIFI-2943: -- Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/1165 I think the Admin Guide should also get an update indicating that the TLS Toolkit will ignore user requests to use PKCS12 format for truststores. > tls-toolkit pkcs12 truststore 0 entries > --- > > Key: NIFI-2943 > URL: https://issues.apache.org/jira/browse/NIFI-2943 > Project: Apache NiFi > Issue Type: Bug >Reporter: Bryan Rosander >Assignee: Bryan Rosander >Priority: Minor > > When pkcs12 is used by the tls-toolkit, the resulting truststore has no > entries when inspected by the keytool and the tls-toolkit certificate > authority certificate is not trusted by NiFi. > This seems to be due to the Java pkcs12 provider not supporting certificate > entries: > http://stackoverflow.com/questions/3614239/pkcs12-java-keystore-from-ca-and-user-certificate-in-java#answer-3614405 > The Bouncy Castle provider does seem to support certificates but we may not > want to explicitly use that provider from within NiFi. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi issue #1165: NIFI-2943 - pkcs12 keystore improvements
Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/1165 I think the Admin Guide should also get an update indicating that the TLS Toolkit will ignore user requests to use PKCS12 format for truststores. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (NIFI-3032) NiFiPropertiesLoader can cache stale key after migration
[ https://issues.apache.org/jira/browse/NIFI-3032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy LoPresto updated NIFI-3032: Resolution: Fixed Status: Resolved (was: Patch Available) > NiFiPropertiesLoader can cache stale key after migration > > > Key: NIFI-3032 > URL: https://issues.apache.org/jira/browse/NIFI-3032 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.0.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto >Priority: Blocker > Labels: encryption, security > Fix For: 1.1.0 > > > [~bryanrosan...@gmail.com] discovered an issue where the > {{NiFiPropertiesLoader}} can be initialized with a static > {{SensitivePropertyProviderFactory}} containing a specific {{key}}, and then > when {{NiFiPropertiesLoader.withKey()}} is called with a different key, the > factory is not refreshed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2943) tls-toolkit pkcs12 truststore 0 entries
[ https://issues.apache.org/jira/browse/NIFI-2943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15666050#comment-15666050 ] ASF GitHub Bot commented on NIFI-2943: -- Github user alopresto commented on a diff in the pull request: https://github.com/apache/nifi/pull/1165#discussion_r87947260 --- Diff: nifi-commons/nifi-security-utils/src/main/java/org/apache/nifi/security/util/KeyStoreUtils.java --- @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.nifi.security.util; + +import org.apache.commons.lang3.StringUtils; +import org.bouncycastle.jce.provider.BouncyCastleProvider; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.security.KeyStore; +import java.security.KeyStoreException; +import java.security.Security; + +public class KeyStoreUtils { +private static final Logger logger = LoggerFactory.getLogger(KeyStoreUtils.class); + +static { +Security.addProvider(new BouncyCastleProvider()); +} + +public static String getKeyStoreProvider(String keyStoreType) { +if (KeystoreType.PKCS12.toString().equalsIgnoreCase(keyStoreType)) { +return BouncyCastleProvider.PROVIDER_NAME; +} +return null; +} + +public static KeyStore getKeyStore(String keyStoreType) throws KeyStoreException { --- End diff -- We should add Javadoc comments on these methods to indicate that they do not return a specific instance, rather a new empty instance similar to `KeyStore.getInstance()`. > tls-toolkit pkcs12 truststore 0 entries > --- > > Key: NIFI-2943 > URL: https://issues.apache.org/jira/browse/NIFI-2943 > Project: Apache NiFi > Issue Type: Bug >Reporter: Bryan Rosander >Assignee: Bryan Rosander >Priority: Minor > > When pkcs12 is used by the tls-toolkit, the resulting truststore has no > entries when inspected by the keytool and the tls-toolkit certificate > authority certificate is not trusted by NiFi. > This seems to be due to the Java pkcs12 provider not supporting certificate > entries: > http://stackoverflow.com/questions/3614239/pkcs12-java-keystore-from-ca-and-user-certificate-in-java#answer-3614405 > The Bouncy Castle provider does seem to support certificates but we may not > want to explicitly use that provider from within NiFi. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi pull request #1165: NIFI-2943 - pkcs12 keystore improvements
Github user alopresto commented on a diff in the pull request: https://github.com/apache/nifi/pull/1165#discussion_r87947260 --- Diff: nifi-commons/nifi-security-utils/src/main/java/org/apache/nifi/security/util/KeyStoreUtils.java --- @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.nifi.security.util; + +import org.apache.commons.lang3.StringUtils; +import org.bouncycastle.jce.provider.BouncyCastleProvider; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.security.KeyStore; +import java.security.KeyStoreException; +import java.security.Security; + +public class KeyStoreUtils { +private static final Logger logger = LoggerFactory.getLogger(KeyStoreUtils.class); + +static { +Security.addProvider(new BouncyCastleProvider()); +} + +public static String getKeyStoreProvider(String keyStoreType) { +if (KeystoreType.PKCS12.toString().equalsIgnoreCase(keyStoreType)) { +return BouncyCastleProvider.PROVIDER_NAME; +} +return null; +} + +public static KeyStore getKeyStore(String keyStoreType) throws KeyStoreException { --- End diff -- We should add Javadoc comments on these methods to indicate that they do not return a specific instance, rather a new empty instance similar to `KeyStore.getInstance()`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2943) tls-toolkit pkcs12 truststore 0 entries
[ https://issues.apache.org/jira/browse/NIFI-2943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15666008#comment-15666008 ] ASF GitHub Bot commented on NIFI-2943: -- Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/1165 I think we should provide log output indicating that the user's choice of PKCS12 is not used for truststores. ``` hw12203:...assembly/target/nifi-toolkit-1.1.0-SNAPSHOT-bin/nifi-toolkit-1.1.0-SNAPSHOT (pr1165) alopresto 🔓 46s @ 19:28:29 $ ./bin/tls-toolkit.sh standalone -n 'localhost' -T PKCS12 -P password -S password 2016-11-14 19:52:11,629 INFO [main] o.a.n.t.t.s.TlsToolkitStandaloneCommandLine No nifiPropertiesFile specified, using embedded one. 2016-11-14 19:52:11,956 INFO [main] o.a.n.t.t.s.TlsToolkitStandalone Running standalone certificate generation with output directory ../nifi-toolkit-1.1.0-SNAPSHOT 2016-11-14 19:52:12,407 INFO [main] o.a.n.t.t.s.TlsToolkitStandalone Generated new CA certificate ../nifi-toolkit-1.1.0-SNAPSHOT/nifi-cert.pem and key ../nifi-toolkit-1.1.0-SNAPSHOT/nifi-key.key 2016-11-14 19:52:12,408 INFO [main] o.a.n.t.t.s.TlsToolkitStandalone Writing new ssl configuration to ../nifi-toolkit-1.1.0-SNAPSHOT/localhost 2016-11-14 19:52:13,382 INFO [main] o.a.n.t.t.s.TlsToolkitStandalone Successfully generated TLS configuration for localhost 1 in ../nifi-toolkit-1.1.0-SNAPSHOT/localhost 2016-11-14 19:52:13,382 INFO [main] o.a.n.t.t.s.TlsToolkitStandalone No clientCertDn specified, not generating any client certificates. 2016-11-14 19:52:13,382 INFO [main] o.a.n.t.t.s.TlsToolkitStandalone tls-toolkit standalone completed successfully hw12203:...assembly/target/nifi-toolkit-1.1.0-SNAPSHOT-bin/nifi-toolkit-1.1.0-SNAPSHOT (pr1165) alopresto 🔓 1424s @ 19:52:14 $ ll localhost/ total 40 drwx-- 5 alopresto staff 170B Nov 14 19:52 ./ drwxr-xr-x 11 alopresto staff 374B Nov 14 19:52 ../ -rw--- 1 alopresto staff 3.4K Nov 14 19:52 keystore.pkcs12 -rw--- 1 alopresto staff 8.5K Nov 14 19:52 nifi.properties -rw--- 1 alopresto staff 911B Nov 14 19:52 truststore.jks hw12203:...assembly/target/nifi-toolkit-1.1.0-SNAPSHOT-bin/nifi-toolkit-1.1.0-SNAPSHOT (pr1165) alopresto 🔓 196s @ 19:55:31 $ ``` > tls-toolkit pkcs12 truststore 0 entries > --- > > Key: NIFI-2943 > URL: https://issues.apache.org/jira/browse/NIFI-2943 > Project: Apache NiFi > Issue Type: Bug >Reporter: Bryan Rosander >Assignee: Bryan Rosander >Priority: Minor > > When pkcs12 is used by the tls-toolkit, the resulting truststore has no > entries when inspected by the keytool and the tls-toolkit certificate > authority certificate is not trusted by NiFi. > This seems to be due to the Java pkcs12 provider not supporting certificate > entries: > http://stackoverflow.com/questions/3614239/pkcs12-java-keystore-from-ca-and-user-certificate-in-java#answer-3614405 > The Bouncy Castle provider does seem to support certificates but we may not > want to explicitly use that provider from within NiFi. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi issue #1165: NIFI-2943 - pkcs12 keystore improvements
Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/1165 I think we should provide log output indicating that the user's choice of PKCS12 is not used for truststores. ``` hw12203:...assembly/target/nifi-toolkit-1.1.0-SNAPSHOT-bin/nifi-toolkit-1.1.0-SNAPSHOT (pr1165) alopresto 🔓 46s @ 19:28:29 $ ./bin/tls-toolkit.sh standalone -n 'localhost' -T PKCS12 -P password -S password 2016-11-14 19:52:11,629 INFO [main] o.a.n.t.t.s.TlsToolkitStandaloneCommandLine No nifiPropertiesFile specified, using embedded one. 2016-11-14 19:52:11,956 INFO [main] o.a.n.t.t.s.TlsToolkitStandalone Running standalone certificate generation with output directory ../nifi-toolkit-1.1.0-SNAPSHOT 2016-11-14 19:52:12,407 INFO [main] o.a.n.t.t.s.TlsToolkitStandalone Generated new CA certificate ../nifi-toolkit-1.1.0-SNAPSHOT/nifi-cert.pem and key ../nifi-toolkit-1.1.0-SNAPSHOT/nifi-key.key 2016-11-14 19:52:12,408 INFO [main] o.a.n.t.t.s.TlsToolkitStandalone Writing new ssl configuration to ../nifi-toolkit-1.1.0-SNAPSHOT/localhost 2016-11-14 19:52:13,382 INFO [main] o.a.n.t.t.s.TlsToolkitStandalone Successfully generated TLS configuration for localhost 1 in ../nifi-toolkit-1.1.0-SNAPSHOT/localhost 2016-11-14 19:52:13,382 INFO [main] o.a.n.t.t.s.TlsToolkitStandalone No clientCertDn specified, not generating any client certificates. 2016-11-14 19:52:13,382 INFO [main] o.a.n.t.t.s.TlsToolkitStandalone tls-toolkit standalone completed successfully hw12203:...assembly/target/nifi-toolkit-1.1.0-SNAPSHOT-bin/nifi-toolkit-1.1.0-SNAPSHOT (pr1165) alopresto 🔓 1424s @ 19:52:14 $ ll localhost/ total 40 drwx-- 5 alopresto staff 170B Nov 14 19:52 ./ drwxr-xr-x 11 alopresto staff 374B Nov 14 19:52 ../ -rw--- 1 alopresto staff 3.4K Nov 14 19:52 keystore.pkcs12 -rw--- 1 alopresto staff 8.5K Nov 14 19:52 nifi.properties -rw--- 1 alopresto staff 911B Nov 14 19:52 truststore.jks hw12203:...assembly/target/nifi-toolkit-1.1.0-SNAPSHOT-bin/nifi-toolkit-1.1.0-SNAPSHOT (pr1165) alopresto 🔓 196s @ 19:55:31 $ ``` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (NIFI-2654) Encrypted configs should handle login identity provider configs
[ https://issues.apache.org/jira/browse/NIFI-2654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated NIFI-2654: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Encrypted configs should handle login identity provider configs > --- > > Key: NIFI-2654 > URL: https://issues.apache.org/jira/browse/NIFI-2654 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Tools and Build >Affects Versions: 1.0.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto > Labels: config, encryption, ldap, security > Fix For: 1.1.0 > > > The encrypted configuration tool and internal logic to load unprotected > values should handle sensitive values contained in the login identity > providers (like LDAP Manager Password). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2654) Encrypted configs should handle login identity provider configs
[ https://issues.apache.org/jira/browse/NIFI-2654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665930#comment-15665930 ] ASF subversion and git services commented on NIFI-2654: --- Commit 59fea1cb4ed440ad0e68d96a412792c3e4551309 in nifi's branch refs/heads/master from [~alopresto] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=59fea1c ] NIFI-2654 Enabled encryption coverage for login-identity-providers.xml. Squashed commits: [5dd22a9] NIFI-2654 Updated administration guide with login-identity-providers.xml flags. Exposed master key retrieval code in NiFiPropertiesLoader. Added logic to decrypt login identity providers XML configuration. Updated login-identity-providers.xsd to include encryption scheme attribute. Added unit tests. (+18 squashed commits) Squashed commits: [57c815f] NIFI-2654 Resolved issue where empty LIP property elements could not be encrypted. Added unit test and resource. [27d7309] NIFI-2654 Wired in serialization logic to write logic for LIP. Added comprehensive unit test for LIP & NFP in same test. [b450eb2] NIFI-2654 Finalized logic for preserving comments in LIP parsing. [5aa6c9c] NIFI-2654 Added logic for maintaining XML formatting (comments and whitespace) for LIP. Added unit tests (w/o encryption works; w/ does not). [b53461f] NIFI-2654 Added unit test for full tool invocation migrating a login-identity-providers.xml file and updating file and bootstrap.conf with key. [2d9686c] NIFI-2654 Updated tool description and various logging statements. Added unit test for full tool invocation encrypting a login-identity-providers.xml file and updating file and bootstrap.conf with key. [8c67cb2] NIFI-2654 Added logic to encrypt LIP XML content. Added unit tests. [8682d19] NIFI-2654 Added logic to handle "empty" (commented) LIP files. Added unit tests. [077230e] NIFI-2654 Fixed logic to decrypt multiline and multiple-per-line XML elements. Added unit tests and resources. [d5bb8da] NIFI-2654 Ignored unit test for unreadable conf directory because directory was causing Maven build issues. Removed test resources. [7e50506] NIFI-2654 Fixed AESSensitivePropertyProvider bug handling cipher text with whitespace. Added unit test. [b69a661] NIFI-2654 Fixed AESSensitivePropertyProviderFactoryTest to reflect absence of key causes errors. [6f821b9] NIFI-2654 Added standard password to arbitrary encryption test for use in test resources. [d289ffa] NIFI-2654 Added LIP XML decryption. Added unit tests. [a482245] NIFI-2654 Added LIP test resources. [7204df4] NIFI-2654 Changed logic to only perform properties encryption when file path is provided. [729e1df] NIFI-2654 Removed population of default file locations for bootstrap.conf, nifi.properties, and login-identity-providers.xml as not all files may be desired. Added/updated unit tests. [7dba5ef] NIFI-2654 Started LIP work (arguments & parsing). Added unit tests. Signed-off-by: Yolanda M. Davis This closes #1216 > Encrypted configs should handle login identity provider configs > --- > > Key: NIFI-2654 > URL: https://issues.apache.org/jira/browse/NIFI-2654 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Tools and Build >Affects Versions: 1.0.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto > Labels: config, encryption, ldap, security > Fix For: 1.1.0 > > > The encrypted configuration tool and internal logic to load unprotected > values should handle sensitive values contained in the login identity > providers (like LDAP Manager Password). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2654) Encrypted configs should handle login identity provider configs
[ https://issues.apache.org/jira/browse/NIFI-2654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665928#comment-15665928 ] ASF subversion and git services commented on NIFI-2654: --- Commit 59fea1cb4ed440ad0e68d96a412792c3e4551309 in nifi's branch refs/heads/master from [~alopresto] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=59fea1c ] NIFI-2654 Enabled encryption coverage for login-identity-providers.xml. Squashed commits: [5dd22a9] NIFI-2654 Updated administration guide with login-identity-providers.xml flags. Exposed master key retrieval code in NiFiPropertiesLoader. Added logic to decrypt login identity providers XML configuration. Updated login-identity-providers.xsd to include encryption scheme attribute. Added unit tests. (+18 squashed commits) Squashed commits: [57c815f] NIFI-2654 Resolved issue where empty LIP property elements could not be encrypted. Added unit test and resource. [27d7309] NIFI-2654 Wired in serialization logic to write logic for LIP. Added comprehensive unit test for LIP & NFP in same test. [b450eb2] NIFI-2654 Finalized logic for preserving comments in LIP parsing. [5aa6c9c] NIFI-2654 Added logic for maintaining XML formatting (comments and whitespace) for LIP. Added unit tests (w/o encryption works; w/ does not). [b53461f] NIFI-2654 Added unit test for full tool invocation migrating a login-identity-providers.xml file and updating file and bootstrap.conf with key. [2d9686c] NIFI-2654 Updated tool description and various logging statements. Added unit test for full tool invocation encrypting a login-identity-providers.xml file and updating file and bootstrap.conf with key. [8c67cb2] NIFI-2654 Added logic to encrypt LIP XML content. Added unit tests. [8682d19] NIFI-2654 Added logic to handle "empty" (commented) LIP files. Added unit tests. [077230e] NIFI-2654 Fixed logic to decrypt multiline and multiple-per-line XML elements. Added unit tests and resources. [d5bb8da] NIFI-2654 Ignored unit test for unreadable conf directory because directory was causing Maven build issues. Removed test resources. [7e50506] NIFI-2654 Fixed AESSensitivePropertyProvider bug handling cipher text with whitespace. Added unit test. [b69a661] NIFI-2654 Fixed AESSensitivePropertyProviderFactoryTest to reflect absence of key causes errors. [6f821b9] NIFI-2654 Added standard password to arbitrary encryption test for use in test resources. [d289ffa] NIFI-2654 Added LIP XML decryption. Added unit tests. [a482245] NIFI-2654 Added LIP test resources. [7204df4] NIFI-2654 Changed logic to only perform properties encryption when file path is provided. [729e1df] NIFI-2654 Removed population of default file locations for bootstrap.conf, nifi.properties, and login-identity-providers.xml as not all files may be desired. Added/updated unit tests. [7dba5ef] NIFI-2654 Started LIP work (arguments & parsing). Added unit tests. Signed-off-by: Yolanda M. Davis This closes #1216 > Encrypted configs should handle login identity provider configs > --- > > Key: NIFI-2654 > URL: https://issues.apache.org/jira/browse/NIFI-2654 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Tools and Build >Affects Versions: 1.0.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto > Labels: config, encryption, ldap, security > Fix For: 1.1.0 > > > The encrypted configuration tool and internal logic to load unprotected > values should handle sensitive values contained in the login identity > providers (like LDAP Manager Password). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2654) Encrypted configs should handle login identity provider configs
[ https://issues.apache.org/jira/browse/NIFI-2654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665939#comment-15665939 ] ASF subversion and git services commented on NIFI-2654: --- Commit 59fea1cb4ed440ad0e68d96a412792c3e4551309 in nifi's branch refs/heads/master from [~alopresto] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=59fea1c ] NIFI-2654 Enabled encryption coverage for login-identity-providers.xml. Squashed commits: [5dd22a9] NIFI-2654 Updated administration guide with login-identity-providers.xml flags. Exposed master key retrieval code in NiFiPropertiesLoader. Added logic to decrypt login identity providers XML configuration. Updated login-identity-providers.xsd to include encryption scheme attribute. Added unit tests. (+18 squashed commits) Squashed commits: [57c815f] NIFI-2654 Resolved issue where empty LIP property elements could not be encrypted. Added unit test and resource. [27d7309] NIFI-2654 Wired in serialization logic to write logic for LIP. Added comprehensive unit test for LIP & NFP in same test. [b450eb2] NIFI-2654 Finalized logic for preserving comments in LIP parsing. [5aa6c9c] NIFI-2654 Added logic for maintaining XML formatting (comments and whitespace) for LIP. Added unit tests (w/o encryption works; w/ does not). [b53461f] NIFI-2654 Added unit test for full tool invocation migrating a login-identity-providers.xml file and updating file and bootstrap.conf with key. [2d9686c] NIFI-2654 Updated tool description and various logging statements. Added unit test for full tool invocation encrypting a login-identity-providers.xml file and updating file and bootstrap.conf with key. [8c67cb2] NIFI-2654 Added logic to encrypt LIP XML content. Added unit tests. [8682d19] NIFI-2654 Added logic to handle "empty" (commented) LIP files. Added unit tests. [077230e] NIFI-2654 Fixed logic to decrypt multiline and multiple-per-line XML elements. Added unit tests and resources. [d5bb8da] NIFI-2654 Ignored unit test for unreadable conf directory because directory was causing Maven build issues. Removed test resources. [7e50506] NIFI-2654 Fixed AESSensitivePropertyProvider bug handling cipher text with whitespace. Added unit test. [b69a661] NIFI-2654 Fixed AESSensitivePropertyProviderFactoryTest to reflect absence of key causes errors. [6f821b9] NIFI-2654 Added standard password to arbitrary encryption test for use in test resources. [d289ffa] NIFI-2654 Added LIP XML decryption. Added unit tests. [a482245] NIFI-2654 Added LIP test resources. [7204df4] NIFI-2654 Changed logic to only perform properties encryption when file path is provided. [729e1df] NIFI-2654 Removed population of default file locations for bootstrap.conf, nifi.properties, and login-identity-providers.xml as not all files may be desired. Added/updated unit tests. [7dba5ef] NIFI-2654 Started LIP work (arguments & parsing). Added unit tests. Signed-off-by: Yolanda M. Davis This closes #1216 > Encrypted configs should handle login identity provider configs > --- > > Key: NIFI-2654 > URL: https://issues.apache.org/jira/browse/NIFI-2654 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Tools and Build >Affects Versions: 1.0.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto > Labels: config, encryption, ldap, security > Fix For: 1.1.0 > > > The encrypted configuration tool and internal logic to load unprotected > values should handle sensitive values contained in the login identity > providers (like LDAP Manager Password). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2654) Encrypted configs should handle login identity provider configs
[ https://issues.apache.org/jira/browse/NIFI-2654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665936#comment-15665936 ] ASF subversion and git services commented on NIFI-2654: --- Commit 59fea1cb4ed440ad0e68d96a412792c3e4551309 in nifi's branch refs/heads/master from [~alopresto] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=59fea1c ] NIFI-2654 Enabled encryption coverage for login-identity-providers.xml. Squashed commits: [5dd22a9] NIFI-2654 Updated administration guide with login-identity-providers.xml flags. Exposed master key retrieval code in NiFiPropertiesLoader. Added logic to decrypt login identity providers XML configuration. Updated login-identity-providers.xsd to include encryption scheme attribute. Added unit tests. (+18 squashed commits) Squashed commits: [57c815f] NIFI-2654 Resolved issue where empty LIP property elements could not be encrypted. Added unit test and resource. [27d7309] NIFI-2654 Wired in serialization logic to write logic for LIP. Added comprehensive unit test for LIP & NFP in same test. [b450eb2] NIFI-2654 Finalized logic for preserving comments in LIP parsing. [5aa6c9c] NIFI-2654 Added logic for maintaining XML formatting (comments and whitespace) for LIP. Added unit tests (w/o encryption works; w/ does not). [b53461f] NIFI-2654 Added unit test for full tool invocation migrating a login-identity-providers.xml file and updating file and bootstrap.conf with key. [2d9686c] NIFI-2654 Updated tool description and various logging statements. Added unit test for full tool invocation encrypting a login-identity-providers.xml file and updating file and bootstrap.conf with key. [8c67cb2] NIFI-2654 Added logic to encrypt LIP XML content. Added unit tests. [8682d19] NIFI-2654 Added logic to handle "empty" (commented) LIP files. Added unit tests. [077230e] NIFI-2654 Fixed logic to decrypt multiline and multiple-per-line XML elements. Added unit tests and resources. [d5bb8da] NIFI-2654 Ignored unit test for unreadable conf directory because directory was causing Maven build issues. Removed test resources. [7e50506] NIFI-2654 Fixed AESSensitivePropertyProvider bug handling cipher text with whitespace. Added unit test. [b69a661] NIFI-2654 Fixed AESSensitivePropertyProviderFactoryTest to reflect absence of key causes errors. [6f821b9] NIFI-2654 Added standard password to arbitrary encryption test for use in test resources. [d289ffa] NIFI-2654 Added LIP XML decryption. Added unit tests. [a482245] NIFI-2654 Added LIP test resources. [7204df4] NIFI-2654 Changed logic to only perform properties encryption when file path is provided. [729e1df] NIFI-2654 Removed population of default file locations for bootstrap.conf, nifi.properties, and login-identity-providers.xml as not all files may be desired. Added/updated unit tests. [7dba5ef] NIFI-2654 Started LIP work (arguments & parsing). Added unit tests. Signed-off-by: Yolanda M. Davis This closes #1216 > Encrypted configs should handle login identity provider configs > --- > > Key: NIFI-2654 > URL: https://issues.apache.org/jira/browse/NIFI-2654 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Tools and Build >Affects Versions: 1.0.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto > Labels: config, encryption, ldap, security > Fix For: 1.1.0 > > > The encrypted configuration tool and internal logic to load unprotected > values should handle sensitive values contained in the login identity > providers (like LDAP Manager Password). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2654) Encrypted configs should handle login identity provider configs
[ https://issues.apache.org/jira/browse/NIFI-2654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665947#comment-15665947 ] ASF subversion and git services commented on NIFI-2654: --- Commit 59fea1cb4ed440ad0e68d96a412792c3e4551309 in nifi's branch refs/heads/master from [~alopresto] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=59fea1c ] NIFI-2654 Enabled encryption coverage for login-identity-providers.xml. Squashed commits: [5dd22a9] NIFI-2654 Updated administration guide with login-identity-providers.xml flags. Exposed master key retrieval code in NiFiPropertiesLoader. Added logic to decrypt login identity providers XML configuration. Updated login-identity-providers.xsd to include encryption scheme attribute. Added unit tests. (+18 squashed commits) Squashed commits: [57c815f] NIFI-2654 Resolved issue where empty LIP property elements could not be encrypted. Added unit test and resource. [27d7309] NIFI-2654 Wired in serialization logic to write logic for LIP. Added comprehensive unit test for LIP & NFP in same test. [b450eb2] NIFI-2654 Finalized logic for preserving comments in LIP parsing. [5aa6c9c] NIFI-2654 Added logic for maintaining XML formatting (comments and whitespace) for LIP. Added unit tests (w/o encryption works; w/ does not). [b53461f] NIFI-2654 Added unit test for full tool invocation migrating a login-identity-providers.xml file and updating file and bootstrap.conf with key. [2d9686c] NIFI-2654 Updated tool description and various logging statements. Added unit test for full tool invocation encrypting a login-identity-providers.xml file and updating file and bootstrap.conf with key. [8c67cb2] NIFI-2654 Added logic to encrypt LIP XML content. Added unit tests. [8682d19] NIFI-2654 Added logic to handle "empty" (commented) LIP files. Added unit tests. [077230e] NIFI-2654 Fixed logic to decrypt multiline and multiple-per-line XML elements. Added unit tests and resources. [d5bb8da] NIFI-2654 Ignored unit test for unreadable conf directory because directory was causing Maven build issues. Removed test resources. [7e50506] NIFI-2654 Fixed AESSensitivePropertyProvider bug handling cipher text with whitespace. Added unit test. [b69a661] NIFI-2654 Fixed AESSensitivePropertyProviderFactoryTest to reflect absence of key causes errors. [6f821b9] NIFI-2654 Added standard password to arbitrary encryption test for use in test resources. [d289ffa] NIFI-2654 Added LIP XML decryption. Added unit tests. [a482245] NIFI-2654 Added LIP test resources. [7204df4] NIFI-2654 Changed logic to only perform properties encryption when file path is provided. [729e1df] NIFI-2654 Removed population of default file locations for bootstrap.conf, nifi.properties, and login-identity-providers.xml as not all files may be desired. Added/updated unit tests. [7dba5ef] NIFI-2654 Started LIP work (arguments & parsing). Added unit tests. Signed-off-by: Yolanda M. Davis This closes #1216 > Encrypted configs should handle login identity provider configs > --- > > Key: NIFI-2654 > URL: https://issues.apache.org/jira/browse/NIFI-2654 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Tools and Build >Affects Versions: 1.0.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto > Labels: config, encryption, ldap, security > Fix For: 1.1.0 > > > The encrypted configuration tool and internal logic to load unprotected > values should handle sensitive values contained in the login identity > providers (like LDAP Manager Password). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2654) Encrypted configs should handle login identity provider configs
[ https://issues.apache.org/jira/browse/NIFI-2654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665938#comment-15665938 ] ASF subversion and git services commented on NIFI-2654: --- Commit 59fea1cb4ed440ad0e68d96a412792c3e4551309 in nifi's branch refs/heads/master from [~alopresto] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=59fea1c ] NIFI-2654 Enabled encryption coverage for login-identity-providers.xml. Squashed commits: [5dd22a9] NIFI-2654 Updated administration guide with login-identity-providers.xml flags. Exposed master key retrieval code in NiFiPropertiesLoader. Added logic to decrypt login identity providers XML configuration. Updated login-identity-providers.xsd to include encryption scheme attribute. Added unit tests. (+18 squashed commits) Squashed commits: [57c815f] NIFI-2654 Resolved issue where empty LIP property elements could not be encrypted. Added unit test and resource. [27d7309] NIFI-2654 Wired in serialization logic to write logic for LIP. Added comprehensive unit test for LIP & NFP in same test. [b450eb2] NIFI-2654 Finalized logic for preserving comments in LIP parsing. [5aa6c9c] NIFI-2654 Added logic for maintaining XML formatting (comments and whitespace) for LIP. Added unit tests (w/o encryption works; w/ does not). [b53461f] NIFI-2654 Added unit test for full tool invocation migrating a login-identity-providers.xml file and updating file and bootstrap.conf with key. [2d9686c] NIFI-2654 Updated tool description and various logging statements. Added unit test for full tool invocation encrypting a login-identity-providers.xml file and updating file and bootstrap.conf with key. [8c67cb2] NIFI-2654 Added logic to encrypt LIP XML content. Added unit tests. [8682d19] NIFI-2654 Added logic to handle "empty" (commented) LIP files. Added unit tests. [077230e] NIFI-2654 Fixed logic to decrypt multiline and multiple-per-line XML elements. Added unit tests and resources. [d5bb8da] NIFI-2654 Ignored unit test for unreadable conf directory because directory was causing Maven build issues. Removed test resources. [7e50506] NIFI-2654 Fixed AESSensitivePropertyProvider bug handling cipher text with whitespace. Added unit test. [b69a661] NIFI-2654 Fixed AESSensitivePropertyProviderFactoryTest to reflect absence of key causes errors. [6f821b9] NIFI-2654 Added standard password to arbitrary encryption test for use in test resources. [d289ffa] NIFI-2654 Added LIP XML decryption. Added unit tests. [a482245] NIFI-2654 Added LIP test resources. [7204df4] NIFI-2654 Changed logic to only perform properties encryption when file path is provided. [729e1df] NIFI-2654 Removed population of default file locations for bootstrap.conf, nifi.properties, and login-identity-providers.xml as not all files may be desired. Added/updated unit tests. [7dba5ef] NIFI-2654 Started LIP work (arguments & parsing). Added unit tests. Signed-off-by: Yolanda M. Davis This closes #1216 > Encrypted configs should handle login identity provider configs > --- > > Key: NIFI-2654 > URL: https://issues.apache.org/jira/browse/NIFI-2654 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Tools and Build >Affects Versions: 1.0.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto > Labels: config, encryption, ldap, security > Fix For: 1.1.0 > > > The encrypted configuration tool and internal logic to load unprotected > values should handle sensitive values contained in the login identity > providers (like LDAP Manager Password). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2654) Encrypted configs should handle login identity provider configs
[ https://issues.apache.org/jira/browse/NIFI-2654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665942#comment-15665942 ] ASF subversion and git services commented on NIFI-2654: --- Commit 59fea1cb4ed440ad0e68d96a412792c3e4551309 in nifi's branch refs/heads/master from [~alopresto] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=59fea1c ] NIFI-2654 Enabled encryption coverage for login-identity-providers.xml. Squashed commits: [5dd22a9] NIFI-2654 Updated administration guide with login-identity-providers.xml flags. Exposed master key retrieval code in NiFiPropertiesLoader. Added logic to decrypt login identity providers XML configuration. Updated login-identity-providers.xsd to include encryption scheme attribute. Added unit tests. (+18 squashed commits) Squashed commits: [57c815f] NIFI-2654 Resolved issue where empty LIP property elements could not be encrypted. Added unit test and resource. [27d7309] NIFI-2654 Wired in serialization logic to write logic for LIP. Added comprehensive unit test for LIP & NFP in same test. [b450eb2] NIFI-2654 Finalized logic for preserving comments in LIP parsing. [5aa6c9c] NIFI-2654 Added logic for maintaining XML formatting (comments and whitespace) for LIP. Added unit tests (w/o encryption works; w/ does not). [b53461f] NIFI-2654 Added unit test for full tool invocation migrating a login-identity-providers.xml file and updating file and bootstrap.conf with key. [2d9686c] NIFI-2654 Updated tool description and various logging statements. Added unit test for full tool invocation encrypting a login-identity-providers.xml file and updating file and bootstrap.conf with key. [8c67cb2] NIFI-2654 Added logic to encrypt LIP XML content. Added unit tests. [8682d19] NIFI-2654 Added logic to handle "empty" (commented) LIP files. Added unit tests. [077230e] NIFI-2654 Fixed logic to decrypt multiline and multiple-per-line XML elements. Added unit tests and resources. [d5bb8da] NIFI-2654 Ignored unit test for unreadable conf directory because directory was causing Maven build issues. Removed test resources. [7e50506] NIFI-2654 Fixed AESSensitivePropertyProvider bug handling cipher text with whitespace. Added unit test. [b69a661] NIFI-2654 Fixed AESSensitivePropertyProviderFactoryTest to reflect absence of key causes errors. [6f821b9] NIFI-2654 Added standard password to arbitrary encryption test for use in test resources. [d289ffa] NIFI-2654 Added LIP XML decryption. Added unit tests. [a482245] NIFI-2654 Added LIP test resources. [7204df4] NIFI-2654 Changed logic to only perform properties encryption when file path is provided. [729e1df] NIFI-2654 Removed population of default file locations for bootstrap.conf, nifi.properties, and login-identity-providers.xml as not all files may be desired. Added/updated unit tests. [7dba5ef] NIFI-2654 Started LIP work (arguments & parsing). Added unit tests. Signed-off-by: Yolanda M. Davis This closes #1216 > Encrypted configs should handle login identity provider configs > --- > > Key: NIFI-2654 > URL: https://issues.apache.org/jira/browse/NIFI-2654 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Tools and Build >Affects Versions: 1.0.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto > Labels: config, encryption, ldap, security > Fix For: 1.1.0 > > > The encrypted configuration tool and internal logic to load unprotected > values should handle sensitive values contained in the login identity > providers (like LDAP Manager Password). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2654) Encrypted configs should handle login identity provider configs
[ https://issues.apache.org/jira/browse/NIFI-2654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665945#comment-15665945 ] ASF subversion and git services commented on NIFI-2654: --- Commit 59fea1cb4ed440ad0e68d96a412792c3e4551309 in nifi's branch refs/heads/master from [~alopresto] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=59fea1c ] NIFI-2654 Enabled encryption coverage for login-identity-providers.xml. Squashed commits: [5dd22a9] NIFI-2654 Updated administration guide with login-identity-providers.xml flags. Exposed master key retrieval code in NiFiPropertiesLoader. Added logic to decrypt login identity providers XML configuration. Updated login-identity-providers.xsd to include encryption scheme attribute. Added unit tests. (+18 squashed commits) Squashed commits: [57c815f] NIFI-2654 Resolved issue where empty LIP property elements could not be encrypted. Added unit test and resource. [27d7309] NIFI-2654 Wired in serialization logic to write logic for LIP. Added comprehensive unit test for LIP & NFP in same test. [b450eb2] NIFI-2654 Finalized logic for preserving comments in LIP parsing. [5aa6c9c] NIFI-2654 Added logic for maintaining XML formatting (comments and whitespace) for LIP. Added unit tests (w/o encryption works; w/ does not). [b53461f] NIFI-2654 Added unit test for full tool invocation migrating a login-identity-providers.xml file and updating file and bootstrap.conf with key. [2d9686c] NIFI-2654 Updated tool description and various logging statements. Added unit test for full tool invocation encrypting a login-identity-providers.xml file and updating file and bootstrap.conf with key. [8c67cb2] NIFI-2654 Added logic to encrypt LIP XML content. Added unit tests. [8682d19] NIFI-2654 Added logic to handle "empty" (commented) LIP files. Added unit tests. [077230e] NIFI-2654 Fixed logic to decrypt multiline and multiple-per-line XML elements. Added unit tests and resources. [d5bb8da] NIFI-2654 Ignored unit test for unreadable conf directory because directory was causing Maven build issues. Removed test resources. [7e50506] NIFI-2654 Fixed AESSensitivePropertyProvider bug handling cipher text with whitespace. Added unit test. [b69a661] NIFI-2654 Fixed AESSensitivePropertyProviderFactoryTest to reflect absence of key causes errors. [6f821b9] NIFI-2654 Added standard password to arbitrary encryption test for use in test resources. [d289ffa] NIFI-2654 Added LIP XML decryption. Added unit tests. [a482245] NIFI-2654 Added LIP test resources. [7204df4] NIFI-2654 Changed logic to only perform properties encryption when file path is provided. [729e1df] NIFI-2654 Removed population of default file locations for bootstrap.conf, nifi.properties, and login-identity-providers.xml as not all files may be desired. Added/updated unit tests. [7dba5ef] NIFI-2654 Started LIP work (arguments & parsing). Added unit tests. Signed-off-by: Yolanda M. Davis This closes #1216 > Encrypted configs should handle login identity provider configs > --- > > Key: NIFI-2654 > URL: https://issues.apache.org/jira/browse/NIFI-2654 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Tools and Build >Affects Versions: 1.0.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto > Labels: config, encryption, ldap, security > Fix For: 1.1.0 > > > The encrypted configuration tool and internal logic to load unprotected > values should handle sensitive values contained in the login identity > providers (like LDAP Manager Password). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2654) Encrypted configs should handle login identity provider configs
[ https://issues.apache.org/jira/browse/NIFI-2654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665940#comment-15665940 ] ASF subversion and git services commented on NIFI-2654: --- Commit 59fea1cb4ed440ad0e68d96a412792c3e4551309 in nifi's branch refs/heads/master from [~alopresto] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=59fea1c ] NIFI-2654 Enabled encryption coverage for login-identity-providers.xml. Squashed commits: [5dd22a9] NIFI-2654 Updated administration guide with login-identity-providers.xml flags. Exposed master key retrieval code in NiFiPropertiesLoader. Added logic to decrypt login identity providers XML configuration. Updated login-identity-providers.xsd to include encryption scheme attribute. Added unit tests. (+18 squashed commits) Squashed commits: [57c815f] NIFI-2654 Resolved issue where empty LIP property elements could not be encrypted. Added unit test and resource. [27d7309] NIFI-2654 Wired in serialization logic to write logic for LIP. Added comprehensive unit test for LIP & NFP in same test. [b450eb2] NIFI-2654 Finalized logic for preserving comments in LIP parsing. [5aa6c9c] NIFI-2654 Added logic for maintaining XML formatting (comments and whitespace) for LIP. Added unit tests (w/o encryption works; w/ does not). [b53461f] NIFI-2654 Added unit test for full tool invocation migrating a login-identity-providers.xml file and updating file and bootstrap.conf with key. [2d9686c] NIFI-2654 Updated tool description and various logging statements. Added unit test for full tool invocation encrypting a login-identity-providers.xml file and updating file and bootstrap.conf with key. [8c67cb2] NIFI-2654 Added logic to encrypt LIP XML content. Added unit tests. [8682d19] NIFI-2654 Added logic to handle "empty" (commented) LIP files. Added unit tests. [077230e] NIFI-2654 Fixed logic to decrypt multiline and multiple-per-line XML elements. Added unit tests and resources. [d5bb8da] NIFI-2654 Ignored unit test for unreadable conf directory because directory was causing Maven build issues. Removed test resources. [7e50506] NIFI-2654 Fixed AESSensitivePropertyProvider bug handling cipher text with whitespace. Added unit test. [b69a661] NIFI-2654 Fixed AESSensitivePropertyProviderFactoryTest to reflect absence of key causes errors. [6f821b9] NIFI-2654 Added standard password to arbitrary encryption test for use in test resources. [d289ffa] NIFI-2654 Added LIP XML decryption. Added unit tests. [a482245] NIFI-2654 Added LIP test resources. [7204df4] NIFI-2654 Changed logic to only perform properties encryption when file path is provided. [729e1df] NIFI-2654 Removed population of default file locations for bootstrap.conf, nifi.properties, and login-identity-providers.xml as not all files may be desired. Added/updated unit tests. [7dba5ef] NIFI-2654 Started LIP work (arguments & parsing). Added unit tests. Signed-off-by: Yolanda M. Davis This closes #1216 > Encrypted configs should handle login identity provider configs > --- > > Key: NIFI-2654 > URL: https://issues.apache.org/jira/browse/NIFI-2654 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Tools and Build >Affects Versions: 1.0.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto > Labels: config, encryption, ldap, security > Fix For: 1.1.0 > > > The encrypted configuration tool and internal logic to load unprotected > values should handle sensitive values contained in the login identity > providers (like LDAP Manager Password). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2654) Encrypted configs should handle login identity provider configs
[ https://issues.apache.org/jira/browse/NIFI-2654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665931#comment-15665931 ] ASF subversion and git services commented on NIFI-2654: --- Commit 59fea1cb4ed440ad0e68d96a412792c3e4551309 in nifi's branch refs/heads/master from [~alopresto] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=59fea1c ] NIFI-2654 Enabled encryption coverage for login-identity-providers.xml. Squashed commits: [5dd22a9] NIFI-2654 Updated administration guide with login-identity-providers.xml flags. Exposed master key retrieval code in NiFiPropertiesLoader. Added logic to decrypt login identity providers XML configuration. Updated login-identity-providers.xsd to include encryption scheme attribute. Added unit tests. (+18 squashed commits) Squashed commits: [57c815f] NIFI-2654 Resolved issue where empty LIP property elements could not be encrypted. Added unit test and resource. [27d7309] NIFI-2654 Wired in serialization logic to write logic for LIP. Added comprehensive unit test for LIP & NFP in same test. [b450eb2] NIFI-2654 Finalized logic for preserving comments in LIP parsing. [5aa6c9c] NIFI-2654 Added logic for maintaining XML formatting (comments and whitespace) for LIP. Added unit tests (w/o encryption works; w/ does not). [b53461f] NIFI-2654 Added unit test for full tool invocation migrating a login-identity-providers.xml file and updating file and bootstrap.conf with key. [2d9686c] NIFI-2654 Updated tool description and various logging statements. Added unit test for full tool invocation encrypting a login-identity-providers.xml file and updating file and bootstrap.conf with key. [8c67cb2] NIFI-2654 Added logic to encrypt LIP XML content. Added unit tests. [8682d19] NIFI-2654 Added logic to handle "empty" (commented) LIP files. Added unit tests. [077230e] NIFI-2654 Fixed logic to decrypt multiline and multiple-per-line XML elements. Added unit tests and resources. [d5bb8da] NIFI-2654 Ignored unit test for unreadable conf directory because directory was causing Maven build issues. Removed test resources. [7e50506] NIFI-2654 Fixed AESSensitivePropertyProvider bug handling cipher text with whitespace. Added unit test. [b69a661] NIFI-2654 Fixed AESSensitivePropertyProviderFactoryTest to reflect absence of key causes errors. [6f821b9] NIFI-2654 Added standard password to arbitrary encryption test for use in test resources. [d289ffa] NIFI-2654 Added LIP XML decryption. Added unit tests. [a482245] NIFI-2654 Added LIP test resources. [7204df4] NIFI-2654 Changed logic to only perform properties encryption when file path is provided. [729e1df] NIFI-2654 Removed population of default file locations for bootstrap.conf, nifi.properties, and login-identity-providers.xml as not all files may be desired. Added/updated unit tests. [7dba5ef] NIFI-2654 Started LIP work (arguments & parsing). Added unit tests. Signed-off-by: Yolanda M. Davis This closes #1216 > Encrypted configs should handle login identity provider configs > --- > > Key: NIFI-2654 > URL: https://issues.apache.org/jira/browse/NIFI-2654 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Tools and Build >Affects Versions: 1.0.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto > Labels: config, encryption, ldap, security > Fix For: 1.1.0 > > > The encrypted configuration tool and internal logic to load unprotected > values should handle sensitive values contained in the login identity > providers (like LDAP Manager Password). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2654) Encrypted configs should handle login identity provider configs
[ https://issues.apache.org/jira/browse/NIFI-2654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665932#comment-15665932 ] ASF subversion and git services commented on NIFI-2654: --- Commit 59fea1cb4ed440ad0e68d96a412792c3e4551309 in nifi's branch refs/heads/master from [~alopresto] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=59fea1c ] NIFI-2654 Enabled encryption coverage for login-identity-providers.xml. Squashed commits: [5dd22a9] NIFI-2654 Updated administration guide with login-identity-providers.xml flags. Exposed master key retrieval code in NiFiPropertiesLoader. Added logic to decrypt login identity providers XML configuration. Updated login-identity-providers.xsd to include encryption scheme attribute. Added unit tests. (+18 squashed commits) Squashed commits: [57c815f] NIFI-2654 Resolved issue where empty LIP property elements could not be encrypted. Added unit test and resource. [27d7309] NIFI-2654 Wired in serialization logic to write logic for LIP. Added comprehensive unit test for LIP & NFP in same test. [b450eb2] NIFI-2654 Finalized logic for preserving comments in LIP parsing. [5aa6c9c] NIFI-2654 Added logic for maintaining XML formatting (comments and whitespace) for LIP. Added unit tests (w/o encryption works; w/ does not). [b53461f] NIFI-2654 Added unit test for full tool invocation migrating a login-identity-providers.xml file and updating file and bootstrap.conf with key. [2d9686c] NIFI-2654 Updated tool description and various logging statements. Added unit test for full tool invocation encrypting a login-identity-providers.xml file and updating file and bootstrap.conf with key. [8c67cb2] NIFI-2654 Added logic to encrypt LIP XML content. Added unit tests. [8682d19] NIFI-2654 Added logic to handle "empty" (commented) LIP files. Added unit tests. [077230e] NIFI-2654 Fixed logic to decrypt multiline and multiple-per-line XML elements. Added unit tests and resources. [d5bb8da] NIFI-2654 Ignored unit test for unreadable conf directory because directory was causing Maven build issues. Removed test resources. [7e50506] NIFI-2654 Fixed AESSensitivePropertyProvider bug handling cipher text with whitespace. Added unit test. [b69a661] NIFI-2654 Fixed AESSensitivePropertyProviderFactoryTest to reflect absence of key causes errors. [6f821b9] NIFI-2654 Added standard password to arbitrary encryption test for use in test resources. [d289ffa] NIFI-2654 Added LIP XML decryption. Added unit tests. [a482245] NIFI-2654 Added LIP test resources. [7204df4] NIFI-2654 Changed logic to only perform properties encryption when file path is provided. [729e1df] NIFI-2654 Removed population of default file locations for bootstrap.conf, nifi.properties, and login-identity-providers.xml as not all files may be desired. Added/updated unit tests. [7dba5ef] NIFI-2654 Started LIP work (arguments & parsing). Added unit tests. Signed-off-by: Yolanda M. Davis This closes #1216 > Encrypted configs should handle login identity provider configs > --- > > Key: NIFI-2654 > URL: https://issues.apache.org/jira/browse/NIFI-2654 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Tools and Build >Affects Versions: 1.0.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto > Labels: config, encryption, ldap, security > Fix For: 1.1.0 > > > The encrypted configuration tool and internal logic to load unprotected > values should handle sensitive values contained in the login identity > providers (like LDAP Manager Password). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2654) Encrypted configs should handle login identity provider configs
[ https://issues.apache.org/jira/browse/NIFI-2654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665944#comment-15665944 ] ASF subversion and git services commented on NIFI-2654: --- Commit 59fea1cb4ed440ad0e68d96a412792c3e4551309 in nifi's branch refs/heads/master from [~alopresto] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=59fea1c ] NIFI-2654 Enabled encryption coverage for login-identity-providers.xml. Squashed commits: [5dd22a9] NIFI-2654 Updated administration guide with login-identity-providers.xml flags. Exposed master key retrieval code in NiFiPropertiesLoader. Added logic to decrypt login identity providers XML configuration. Updated login-identity-providers.xsd to include encryption scheme attribute. Added unit tests. (+18 squashed commits) Squashed commits: [57c815f] NIFI-2654 Resolved issue where empty LIP property elements could not be encrypted. Added unit test and resource. [27d7309] NIFI-2654 Wired in serialization logic to write logic for LIP. Added comprehensive unit test for LIP & NFP in same test. [b450eb2] NIFI-2654 Finalized logic for preserving comments in LIP parsing. [5aa6c9c] NIFI-2654 Added logic for maintaining XML formatting (comments and whitespace) for LIP. Added unit tests (w/o encryption works; w/ does not). [b53461f] NIFI-2654 Added unit test for full tool invocation migrating a login-identity-providers.xml file and updating file and bootstrap.conf with key. [2d9686c] NIFI-2654 Updated tool description and various logging statements. Added unit test for full tool invocation encrypting a login-identity-providers.xml file and updating file and bootstrap.conf with key. [8c67cb2] NIFI-2654 Added logic to encrypt LIP XML content. Added unit tests. [8682d19] NIFI-2654 Added logic to handle "empty" (commented) LIP files. Added unit tests. [077230e] NIFI-2654 Fixed logic to decrypt multiline and multiple-per-line XML elements. Added unit tests and resources. [d5bb8da] NIFI-2654 Ignored unit test for unreadable conf directory because directory was causing Maven build issues. Removed test resources. [7e50506] NIFI-2654 Fixed AESSensitivePropertyProvider bug handling cipher text with whitespace. Added unit test. [b69a661] NIFI-2654 Fixed AESSensitivePropertyProviderFactoryTest to reflect absence of key causes errors. [6f821b9] NIFI-2654 Added standard password to arbitrary encryption test for use in test resources. [d289ffa] NIFI-2654 Added LIP XML decryption. Added unit tests. [a482245] NIFI-2654 Added LIP test resources. [7204df4] NIFI-2654 Changed logic to only perform properties encryption when file path is provided. [729e1df] NIFI-2654 Removed population of default file locations for bootstrap.conf, nifi.properties, and login-identity-providers.xml as not all files may be desired. Added/updated unit tests. [7dba5ef] NIFI-2654 Started LIP work (arguments & parsing). Added unit tests. Signed-off-by: Yolanda M. Davis This closes #1216 > Encrypted configs should handle login identity provider configs > --- > > Key: NIFI-2654 > URL: https://issues.apache.org/jira/browse/NIFI-2654 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Tools and Build >Affects Versions: 1.0.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto > Labels: config, encryption, ldap, security > Fix For: 1.1.0 > > > The encrypted configuration tool and internal logic to load unprotected > values should handle sensitive values contained in the login identity > providers (like LDAP Manager Password). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2654) Encrypted configs should handle login identity provider configs
[ https://issues.apache.org/jira/browse/NIFI-2654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665927#comment-15665927 ] ASF subversion and git services commented on NIFI-2654: --- Commit 59fea1cb4ed440ad0e68d96a412792c3e4551309 in nifi's branch refs/heads/master from [~alopresto] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=59fea1c ] NIFI-2654 Enabled encryption coverage for login-identity-providers.xml. Squashed commits: [5dd22a9] NIFI-2654 Updated administration guide with login-identity-providers.xml flags. Exposed master key retrieval code in NiFiPropertiesLoader. Added logic to decrypt login identity providers XML configuration. Updated login-identity-providers.xsd to include encryption scheme attribute. Added unit tests. (+18 squashed commits) Squashed commits: [57c815f] NIFI-2654 Resolved issue where empty LIP property elements could not be encrypted. Added unit test and resource. [27d7309] NIFI-2654 Wired in serialization logic to write logic for LIP. Added comprehensive unit test for LIP & NFP in same test. [b450eb2] NIFI-2654 Finalized logic for preserving comments in LIP parsing. [5aa6c9c] NIFI-2654 Added logic for maintaining XML formatting (comments and whitespace) for LIP. Added unit tests (w/o encryption works; w/ does not). [b53461f] NIFI-2654 Added unit test for full tool invocation migrating a login-identity-providers.xml file and updating file and bootstrap.conf with key. [2d9686c] NIFI-2654 Updated tool description and various logging statements. Added unit test for full tool invocation encrypting a login-identity-providers.xml file and updating file and bootstrap.conf with key. [8c67cb2] NIFI-2654 Added logic to encrypt LIP XML content. Added unit tests. [8682d19] NIFI-2654 Added logic to handle "empty" (commented) LIP files. Added unit tests. [077230e] NIFI-2654 Fixed logic to decrypt multiline and multiple-per-line XML elements. Added unit tests and resources. [d5bb8da] NIFI-2654 Ignored unit test for unreadable conf directory because directory was causing Maven build issues. Removed test resources. [7e50506] NIFI-2654 Fixed AESSensitivePropertyProvider bug handling cipher text with whitespace. Added unit test. [b69a661] NIFI-2654 Fixed AESSensitivePropertyProviderFactoryTest to reflect absence of key causes errors. [6f821b9] NIFI-2654 Added standard password to arbitrary encryption test for use in test resources. [d289ffa] NIFI-2654 Added LIP XML decryption. Added unit tests. [a482245] NIFI-2654 Added LIP test resources. [7204df4] NIFI-2654 Changed logic to only perform properties encryption when file path is provided. [729e1df] NIFI-2654 Removed population of default file locations for bootstrap.conf, nifi.properties, and login-identity-providers.xml as not all files may be desired. Added/updated unit tests. [7dba5ef] NIFI-2654 Started LIP work (arguments & parsing). Added unit tests. Signed-off-by: Yolanda M. Davis This closes #1216 > Encrypted configs should handle login identity provider configs > --- > > Key: NIFI-2654 > URL: https://issues.apache.org/jira/browse/NIFI-2654 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Tools and Build >Affects Versions: 1.0.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto > Labels: config, encryption, ldap, security > Fix For: 1.1.0 > > > The encrypted configuration tool and internal logic to load unprotected > values should handle sensitive values contained in the login identity > providers (like LDAP Manager Password). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2654) Encrypted configs should handle login identity provider configs
[ https://issues.apache.org/jira/browse/NIFI-2654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665943#comment-15665943 ] ASF subversion and git services commented on NIFI-2654: --- Commit 59fea1cb4ed440ad0e68d96a412792c3e4551309 in nifi's branch refs/heads/master from [~alopresto] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=59fea1c ] NIFI-2654 Enabled encryption coverage for login-identity-providers.xml. Squashed commits: [5dd22a9] NIFI-2654 Updated administration guide with login-identity-providers.xml flags. Exposed master key retrieval code in NiFiPropertiesLoader. Added logic to decrypt login identity providers XML configuration. Updated login-identity-providers.xsd to include encryption scheme attribute. Added unit tests. (+18 squashed commits) Squashed commits: [57c815f] NIFI-2654 Resolved issue where empty LIP property elements could not be encrypted. Added unit test and resource. [27d7309] NIFI-2654 Wired in serialization logic to write logic for LIP. Added comprehensive unit test for LIP & NFP in same test. [b450eb2] NIFI-2654 Finalized logic for preserving comments in LIP parsing. [5aa6c9c] NIFI-2654 Added logic for maintaining XML formatting (comments and whitespace) for LIP. Added unit tests (w/o encryption works; w/ does not). [b53461f] NIFI-2654 Added unit test for full tool invocation migrating a login-identity-providers.xml file and updating file and bootstrap.conf with key. [2d9686c] NIFI-2654 Updated tool description and various logging statements. Added unit test for full tool invocation encrypting a login-identity-providers.xml file and updating file and bootstrap.conf with key. [8c67cb2] NIFI-2654 Added logic to encrypt LIP XML content. Added unit tests. [8682d19] NIFI-2654 Added logic to handle "empty" (commented) LIP files. Added unit tests. [077230e] NIFI-2654 Fixed logic to decrypt multiline and multiple-per-line XML elements. Added unit tests and resources. [d5bb8da] NIFI-2654 Ignored unit test for unreadable conf directory because directory was causing Maven build issues. Removed test resources. [7e50506] NIFI-2654 Fixed AESSensitivePropertyProvider bug handling cipher text with whitespace. Added unit test. [b69a661] NIFI-2654 Fixed AESSensitivePropertyProviderFactoryTest to reflect absence of key causes errors. [6f821b9] NIFI-2654 Added standard password to arbitrary encryption test for use in test resources. [d289ffa] NIFI-2654 Added LIP XML decryption. Added unit tests. [a482245] NIFI-2654 Added LIP test resources. [7204df4] NIFI-2654 Changed logic to only perform properties encryption when file path is provided. [729e1df] NIFI-2654 Removed population of default file locations for bootstrap.conf, nifi.properties, and login-identity-providers.xml as not all files may be desired. Added/updated unit tests. [7dba5ef] NIFI-2654 Started LIP work (arguments & parsing). Added unit tests. Signed-off-by: Yolanda M. Davis This closes #1216 > Encrypted configs should handle login identity provider configs > --- > > Key: NIFI-2654 > URL: https://issues.apache.org/jira/browse/NIFI-2654 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Tools and Build >Affects Versions: 1.0.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto > Labels: config, encryption, ldap, security > Fix For: 1.1.0 > > > The encrypted configuration tool and internal logic to load unprotected > values should handle sensitive values contained in the login identity > providers (like LDAP Manager Password). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2654) Encrypted configs should handle login identity provider configs
[ https://issues.apache.org/jira/browse/NIFI-2654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665935#comment-15665935 ] ASF GitHub Bot commented on NIFI-2654: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1216 > Encrypted configs should handle login identity provider configs > --- > > Key: NIFI-2654 > URL: https://issues.apache.org/jira/browse/NIFI-2654 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Tools and Build >Affects Versions: 1.0.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto > Labels: config, encryption, ldap, security > Fix For: 1.1.0 > > > The encrypted configuration tool and internal logic to load unprotected > values should handle sensitive values contained in the login identity > providers (like LDAP Manager Password). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2654) Encrypted configs should handle login identity provider configs
[ https://issues.apache.org/jira/browse/NIFI-2654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665929#comment-15665929 ] ASF subversion and git services commented on NIFI-2654: --- Commit 59fea1cb4ed440ad0e68d96a412792c3e4551309 in nifi's branch refs/heads/master from [~alopresto] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=59fea1c ] NIFI-2654 Enabled encryption coverage for login-identity-providers.xml. Squashed commits: [5dd22a9] NIFI-2654 Updated administration guide with login-identity-providers.xml flags. Exposed master key retrieval code in NiFiPropertiesLoader. Added logic to decrypt login identity providers XML configuration. Updated login-identity-providers.xsd to include encryption scheme attribute. Added unit tests. (+18 squashed commits) Squashed commits: [57c815f] NIFI-2654 Resolved issue where empty LIP property elements could not be encrypted. Added unit test and resource. [27d7309] NIFI-2654 Wired in serialization logic to write logic for LIP. Added comprehensive unit test for LIP & NFP in same test. [b450eb2] NIFI-2654 Finalized logic for preserving comments in LIP parsing. [5aa6c9c] NIFI-2654 Added logic for maintaining XML formatting (comments and whitespace) for LIP. Added unit tests (w/o encryption works; w/ does not). [b53461f] NIFI-2654 Added unit test for full tool invocation migrating a login-identity-providers.xml file and updating file and bootstrap.conf with key. [2d9686c] NIFI-2654 Updated tool description and various logging statements. Added unit test for full tool invocation encrypting a login-identity-providers.xml file and updating file and bootstrap.conf with key. [8c67cb2] NIFI-2654 Added logic to encrypt LIP XML content. Added unit tests. [8682d19] NIFI-2654 Added logic to handle "empty" (commented) LIP files. Added unit tests. [077230e] NIFI-2654 Fixed logic to decrypt multiline and multiple-per-line XML elements. Added unit tests and resources. [d5bb8da] NIFI-2654 Ignored unit test for unreadable conf directory because directory was causing Maven build issues. Removed test resources. [7e50506] NIFI-2654 Fixed AESSensitivePropertyProvider bug handling cipher text with whitespace. Added unit test. [b69a661] NIFI-2654 Fixed AESSensitivePropertyProviderFactoryTest to reflect absence of key causes errors. [6f821b9] NIFI-2654 Added standard password to arbitrary encryption test for use in test resources. [d289ffa] NIFI-2654 Added LIP XML decryption. Added unit tests. [a482245] NIFI-2654 Added LIP test resources. [7204df4] NIFI-2654 Changed logic to only perform properties encryption when file path is provided. [729e1df] NIFI-2654 Removed population of default file locations for bootstrap.conf, nifi.properties, and login-identity-providers.xml as not all files may be desired. Added/updated unit tests. [7dba5ef] NIFI-2654 Started LIP work (arguments & parsing). Added unit tests. Signed-off-by: Yolanda M. Davis This closes #1216 > Encrypted configs should handle login identity provider configs > --- > > Key: NIFI-2654 > URL: https://issues.apache.org/jira/browse/NIFI-2654 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Tools and Build >Affects Versions: 1.0.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto > Labels: config, encryption, ldap, security > Fix For: 1.1.0 > > > The encrypted configuration tool and internal logic to load unprotected > values should handle sensitive values contained in the login identity > providers (like LDAP Manager Password). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2654) Encrypted configs should handle login identity provider configs
[ https://issues.apache.org/jira/browse/NIFI-2654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665946#comment-15665946 ] ASF subversion and git services commented on NIFI-2654: --- Commit 59fea1cb4ed440ad0e68d96a412792c3e4551309 in nifi's branch refs/heads/master from [~alopresto] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=59fea1c ] NIFI-2654 Enabled encryption coverage for login-identity-providers.xml. Squashed commits: [5dd22a9] NIFI-2654 Updated administration guide with login-identity-providers.xml flags. Exposed master key retrieval code in NiFiPropertiesLoader. Added logic to decrypt login identity providers XML configuration. Updated login-identity-providers.xsd to include encryption scheme attribute. Added unit tests. (+18 squashed commits) Squashed commits: [57c815f] NIFI-2654 Resolved issue where empty LIP property elements could not be encrypted. Added unit test and resource. [27d7309] NIFI-2654 Wired in serialization logic to write logic for LIP. Added comprehensive unit test for LIP & NFP in same test. [b450eb2] NIFI-2654 Finalized logic for preserving comments in LIP parsing. [5aa6c9c] NIFI-2654 Added logic for maintaining XML formatting (comments and whitespace) for LIP. Added unit tests (w/o encryption works; w/ does not). [b53461f] NIFI-2654 Added unit test for full tool invocation migrating a login-identity-providers.xml file and updating file and bootstrap.conf with key. [2d9686c] NIFI-2654 Updated tool description and various logging statements. Added unit test for full tool invocation encrypting a login-identity-providers.xml file and updating file and bootstrap.conf with key. [8c67cb2] NIFI-2654 Added logic to encrypt LIP XML content. Added unit tests. [8682d19] NIFI-2654 Added logic to handle "empty" (commented) LIP files. Added unit tests. [077230e] NIFI-2654 Fixed logic to decrypt multiline and multiple-per-line XML elements. Added unit tests and resources. [d5bb8da] NIFI-2654 Ignored unit test for unreadable conf directory because directory was causing Maven build issues. Removed test resources. [7e50506] NIFI-2654 Fixed AESSensitivePropertyProvider bug handling cipher text with whitespace. Added unit test. [b69a661] NIFI-2654 Fixed AESSensitivePropertyProviderFactoryTest to reflect absence of key causes errors. [6f821b9] NIFI-2654 Added standard password to arbitrary encryption test for use in test resources. [d289ffa] NIFI-2654 Added LIP XML decryption. Added unit tests. [a482245] NIFI-2654 Added LIP test resources. [7204df4] NIFI-2654 Changed logic to only perform properties encryption when file path is provided. [729e1df] NIFI-2654 Removed population of default file locations for bootstrap.conf, nifi.properties, and login-identity-providers.xml as not all files may be desired. Added/updated unit tests. [7dba5ef] NIFI-2654 Started LIP work (arguments & parsing). Added unit tests. Signed-off-by: Yolanda M. Davis This closes #1216 > Encrypted configs should handle login identity provider configs > --- > > Key: NIFI-2654 > URL: https://issues.apache.org/jira/browse/NIFI-2654 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Tools and Build >Affects Versions: 1.0.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto > Labels: config, encryption, ldap, security > Fix For: 1.1.0 > > > The encrypted configuration tool and internal logic to load unprotected > values should handle sensitive values contained in the login identity > providers (like LDAP Manager Password). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2654) Encrypted configs should handle login identity provider configs
[ https://issues.apache.org/jira/browse/NIFI-2654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665941#comment-15665941 ] ASF subversion and git services commented on NIFI-2654: --- Commit 59fea1cb4ed440ad0e68d96a412792c3e4551309 in nifi's branch refs/heads/master from [~alopresto] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=59fea1c ] NIFI-2654 Enabled encryption coverage for login-identity-providers.xml. Squashed commits: [5dd22a9] NIFI-2654 Updated administration guide with login-identity-providers.xml flags. Exposed master key retrieval code in NiFiPropertiesLoader. Added logic to decrypt login identity providers XML configuration. Updated login-identity-providers.xsd to include encryption scheme attribute. Added unit tests. (+18 squashed commits) Squashed commits: [57c815f] NIFI-2654 Resolved issue where empty LIP property elements could not be encrypted. Added unit test and resource. [27d7309] NIFI-2654 Wired in serialization logic to write logic for LIP. Added comprehensive unit test for LIP & NFP in same test. [b450eb2] NIFI-2654 Finalized logic for preserving comments in LIP parsing. [5aa6c9c] NIFI-2654 Added logic for maintaining XML formatting (comments and whitespace) for LIP. Added unit tests (w/o encryption works; w/ does not). [b53461f] NIFI-2654 Added unit test for full tool invocation migrating a login-identity-providers.xml file and updating file and bootstrap.conf with key. [2d9686c] NIFI-2654 Updated tool description and various logging statements. Added unit test for full tool invocation encrypting a login-identity-providers.xml file and updating file and bootstrap.conf with key. [8c67cb2] NIFI-2654 Added logic to encrypt LIP XML content. Added unit tests. [8682d19] NIFI-2654 Added logic to handle "empty" (commented) LIP files. Added unit tests. [077230e] NIFI-2654 Fixed logic to decrypt multiline and multiple-per-line XML elements. Added unit tests and resources. [d5bb8da] NIFI-2654 Ignored unit test for unreadable conf directory because directory was causing Maven build issues. Removed test resources. [7e50506] NIFI-2654 Fixed AESSensitivePropertyProvider bug handling cipher text with whitespace. Added unit test. [b69a661] NIFI-2654 Fixed AESSensitivePropertyProviderFactoryTest to reflect absence of key causes errors. [6f821b9] NIFI-2654 Added standard password to arbitrary encryption test for use in test resources. [d289ffa] NIFI-2654 Added LIP XML decryption. Added unit tests. [a482245] NIFI-2654 Added LIP test resources. [7204df4] NIFI-2654 Changed logic to only perform properties encryption when file path is provided. [729e1df] NIFI-2654 Removed population of default file locations for bootstrap.conf, nifi.properties, and login-identity-providers.xml as not all files may be desired. Added/updated unit tests. [7dba5ef] NIFI-2654 Started LIP work (arguments & parsing). Added unit tests. Signed-off-by: Yolanda M. Davis This closes #1216 > Encrypted configs should handle login identity provider configs > --- > > Key: NIFI-2654 > URL: https://issues.apache.org/jira/browse/NIFI-2654 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Tools and Build >Affects Versions: 1.0.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto > Labels: config, encryption, ldap, security > Fix For: 1.1.0 > > > The encrypted configuration tool and internal logic to load unprotected > values should handle sensitive values contained in the login identity > providers (like LDAP Manager Password). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2654) Encrypted configs should handle login identity provider configs
[ https://issues.apache.org/jira/browse/NIFI-2654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665933#comment-15665933 ] ASF subversion and git services commented on NIFI-2654: --- Commit 59fea1cb4ed440ad0e68d96a412792c3e4551309 in nifi's branch refs/heads/master from [~alopresto] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=59fea1c ] NIFI-2654 Enabled encryption coverage for login-identity-providers.xml. Squashed commits: [5dd22a9] NIFI-2654 Updated administration guide with login-identity-providers.xml flags. Exposed master key retrieval code in NiFiPropertiesLoader. Added logic to decrypt login identity providers XML configuration. Updated login-identity-providers.xsd to include encryption scheme attribute. Added unit tests. (+18 squashed commits) Squashed commits: [57c815f] NIFI-2654 Resolved issue where empty LIP property elements could not be encrypted. Added unit test and resource. [27d7309] NIFI-2654 Wired in serialization logic to write logic for LIP. Added comprehensive unit test for LIP & NFP in same test. [b450eb2] NIFI-2654 Finalized logic for preserving comments in LIP parsing. [5aa6c9c] NIFI-2654 Added logic for maintaining XML formatting (comments and whitespace) for LIP. Added unit tests (w/o encryption works; w/ does not). [b53461f] NIFI-2654 Added unit test for full tool invocation migrating a login-identity-providers.xml file and updating file and bootstrap.conf with key. [2d9686c] NIFI-2654 Updated tool description and various logging statements. Added unit test for full tool invocation encrypting a login-identity-providers.xml file and updating file and bootstrap.conf with key. [8c67cb2] NIFI-2654 Added logic to encrypt LIP XML content. Added unit tests. [8682d19] NIFI-2654 Added logic to handle "empty" (commented) LIP files. Added unit tests. [077230e] NIFI-2654 Fixed logic to decrypt multiline and multiple-per-line XML elements. Added unit tests and resources. [d5bb8da] NIFI-2654 Ignored unit test for unreadable conf directory because directory was causing Maven build issues. Removed test resources. [7e50506] NIFI-2654 Fixed AESSensitivePropertyProvider bug handling cipher text with whitespace. Added unit test. [b69a661] NIFI-2654 Fixed AESSensitivePropertyProviderFactoryTest to reflect absence of key causes errors. [6f821b9] NIFI-2654 Added standard password to arbitrary encryption test for use in test resources. [d289ffa] NIFI-2654 Added LIP XML decryption. Added unit tests. [a482245] NIFI-2654 Added LIP test resources. [7204df4] NIFI-2654 Changed logic to only perform properties encryption when file path is provided. [729e1df] NIFI-2654 Removed population of default file locations for bootstrap.conf, nifi.properties, and login-identity-providers.xml as not all files may be desired. Added/updated unit tests. [7dba5ef] NIFI-2654 Started LIP work (arguments & parsing). Added unit tests. Signed-off-by: Yolanda M. Davis This closes #1216 > Encrypted configs should handle login identity provider configs > --- > > Key: NIFI-2654 > URL: https://issues.apache.org/jira/browse/NIFI-2654 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Tools and Build >Affects Versions: 1.0.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto > Labels: config, encryption, ldap, security > Fix For: 1.1.0 > > > The encrypted configuration tool and internal logic to load unprotected > values should handle sensitive values contained in the login identity > providers (like LDAP Manager Password). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2654) Encrypted configs should handle login identity provider configs
[ https://issues.apache.org/jira/browse/NIFI-2654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665934#comment-15665934 ] ASF subversion and git services commented on NIFI-2654: --- Commit 59fea1cb4ed440ad0e68d96a412792c3e4551309 in nifi's branch refs/heads/master from [~alopresto] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=59fea1c ] NIFI-2654 Enabled encryption coverage for login-identity-providers.xml. Squashed commits: [5dd22a9] NIFI-2654 Updated administration guide with login-identity-providers.xml flags. Exposed master key retrieval code in NiFiPropertiesLoader. Added logic to decrypt login identity providers XML configuration. Updated login-identity-providers.xsd to include encryption scheme attribute. Added unit tests. (+18 squashed commits) Squashed commits: [57c815f] NIFI-2654 Resolved issue where empty LIP property elements could not be encrypted. Added unit test and resource. [27d7309] NIFI-2654 Wired in serialization logic to write logic for LIP. Added comprehensive unit test for LIP & NFP in same test. [b450eb2] NIFI-2654 Finalized logic for preserving comments in LIP parsing. [5aa6c9c] NIFI-2654 Added logic for maintaining XML formatting (comments and whitespace) for LIP. Added unit tests (w/o encryption works; w/ does not). [b53461f] NIFI-2654 Added unit test for full tool invocation migrating a login-identity-providers.xml file and updating file and bootstrap.conf with key. [2d9686c] NIFI-2654 Updated tool description and various logging statements. Added unit test for full tool invocation encrypting a login-identity-providers.xml file and updating file and bootstrap.conf with key. [8c67cb2] NIFI-2654 Added logic to encrypt LIP XML content. Added unit tests. [8682d19] NIFI-2654 Added logic to handle "empty" (commented) LIP files. Added unit tests. [077230e] NIFI-2654 Fixed logic to decrypt multiline and multiple-per-line XML elements. Added unit tests and resources. [d5bb8da] NIFI-2654 Ignored unit test for unreadable conf directory because directory was causing Maven build issues. Removed test resources. [7e50506] NIFI-2654 Fixed AESSensitivePropertyProvider bug handling cipher text with whitespace. Added unit test. [b69a661] NIFI-2654 Fixed AESSensitivePropertyProviderFactoryTest to reflect absence of key causes errors. [6f821b9] NIFI-2654 Added standard password to arbitrary encryption test for use in test resources. [d289ffa] NIFI-2654 Added LIP XML decryption. Added unit tests. [a482245] NIFI-2654 Added LIP test resources. [7204df4] NIFI-2654 Changed logic to only perform properties encryption when file path is provided. [729e1df] NIFI-2654 Removed population of default file locations for bootstrap.conf, nifi.properties, and login-identity-providers.xml as not all files may be desired. Added/updated unit tests. [7dba5ef] NIFI-2654 Started LIP work (arguments & parsing). Added unit tests. Signed-off-by: Yolanda M. Davis This closes #1216 > Encrypted configs should handle login identity provider configs > --- > > Key: NIFI-2654 > URL: https://issues.apache.org/jira/browse/NIFI-2654 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Tools and Build >Affects Versions: 1.0.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto > Labels: config, encryption, ldap, security > Fix For: 1.1.0 > > > The encrypted configuration tool and internal logic to load unprotected > values should handle sensitive values contained in the login identity > providers (like LDAP Manager Password). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2654) Encrypted configs should handle login identity provider configs
[ https://issues.apache.org/jira/browse/NIFI-2654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665937#comment-15665937 ] ASF subversion and git services commented on NIFI-2654: --- Commit 59fea1cb4ed440ad0e68d96a412792c3e4551309 in nifi's branch refs/heads/master from [~alopresto] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=59fea1c ] NIFI-2654 Enabled encryption coverage for login-identity-providers.xml. Squashed commits: [5dd22a9] NIFI-2654 Updated administration guide with login-identity-providers.xml flags. Exposed master key retrieval code in NiFiPropertiesLoader. Added logic to decrypt login identity providers XML configuration. Updated login-identity-providers.xsd to include encryption scheme attribute. Added unit tests. (+18 squashed commits) Squashed commits: [57c815f] NIFI-2654 Resolved issue where empty LIP property elements could not be encrypted. Added unit test and resource. [27d7309] NIFI-2654 Wired in serialization logic to write logic for LIP. Added comprehensive unit test for LIP & NFP in same test. [b450eb2] NIFI-2654 Finalized logic for preserving comments in LIP parsing. [5aa6c9c] NIFI-2654 Added logic for maintaining XML formatting (comments and whitespace) for LIP. Added unit tests (w/o encryption works; w/ does not). [b53461f] NIFI-2654 Added unit test for full tool invocation migrating a login-identity-providers.xml file and updating file and bootstrap.conf with key. [2d9686c] NIFI-2654 Updated tool description and various logging statements. Added unit test for full tool invocation encrypting a login-identity-providers.xml file and updating file and bootstrap.conf with key. [8c67cb2] NIFI-2654 Added logic to encrypt LIP XML content. Added unit tests. [8682d19] NIFI-2654 Added logic to handle "empty" (commented) LIP files. Added unit tests. [077230e] NIFI-2654 Fixed logic to decrypt multiline and multiple-per-line XML elements. Added unit tests and resources. [d5bb8da] NIFI-2654 Ignored unit test for unreadable conf directory because directory was causing Maven build issues. Removed test resources. [7e50506] NIFI-2654 Fixed AESSensitivePropertyProvider bug handling cipher text with whitespace. Added unit test. [b69a661] NIFI-2654 Fixed AESSensitivePropertyProviderFactoryTest to reflect absence of key causes errors. [6f821b9] NIFI-2654 Added standard password to arbitrary encryption test for use in test resources. [d289ffa] NIFI-2654 Added LIP XML decryption. Added unit tests. [a482245] NIFI-2654 Added LIP test resources. [7204df4] NIFI-2654 Changed logic to only perform properties encryption when file path is provided. [729e1df] NIFI-2654 Removed population of default file locations for bootstrap.conf, nifi.properties, and login-identity-providers.xml as not all files may be desired. Added/updated unit tests. [7dba5ef] NIFI-2654 Started LIP work (arguments & parsing). Added unit tests. Signed-off-by: Yolanda M. Davis This closes #1216 > Encrypted configs should handle login identity provider configs > --- > > Key: NIFI-2654 > URL: https://issues.apache.org/jira/browse/NIFI-2654 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Tools and Build >Affects Versions: 1.0.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto > Labels: config, encryption, ldap, security > Fix For: 1.1.0 > > > The encrypted configuration tool and internal logic to load unprotected > values should handle sensitive values contained in the login identity > providers (like LDAP Manager Password). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi pull request #1216: NIFI-2654 Enabled encryption coverage for login-ide...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1216 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1216: NIFI-2654 Enabled encryption coverage for login-identity-p...
Github user YolandaMDavis commented on the issue: https://github.com/apache/nifi/pull/1216 @alopresto I was able to test encrypt-config update with the following scenarios: tested encryption using new -l flag along with -n, - b, and -p flags (no migration). nifi.properties and login provider identity files encrypted properties as expected. Master key appears as expected in bootstrap.con tested encryption using -l, -n, and - b flag with migration of password using -m -w. nifi.properties and login provider identity files encrypted as expected. Master key appears as expected in bootstrap.conf Also tested encrypted configurations with secured NiFi against OpenLDAP (SIMPLE and TLS). In both scenarios NiFi was able to decrypt values and access LDAP for authentication as needed. +1 Thanks @alopresto! Will merge this in shortly --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2654) Encrypted configs should handle login identity provider configs
[ https://issues.apache.org/jira/browse/NIFI-2654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665802#comment-15665802 ] ASF GitHub Bot commented on NIFI-2654: -- Github user YolandaMDavis commented on the issue: https://github.com/apache/nifi/pull/1216 @alopresto I was able to test encrypt-config update with the following scenarios: tested encryption using new -l flag along with -n, - b, and -p flags (no migration). nifi.properties and login provider identity files encrypted properties as expected. Master key appears as expected in bootstrap.con tested encryption using -l, -n, and - b flag with migration of password using -m -w. nifi.properties and login provider identity files encrypted as expected. Master key appears as expected in bootstrap.conf Also tested encrypted configurations with secured NiFi against OpenLDAP (SIMPLE and TLS). In both scenarios NiFi was able to decrypt values and access LDAP for authentication as needed. +1 Thanks @alopresto! Will merge this in shortly > Encrypted configs should handle login identity provider configs > --- > > Key: NIFI-2654 > URL: https://issues.apache.org/jira/browse/NIFI-2654 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Tools and Build >Affects Versions: 1.0.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto > Labels: config, encryption, ldap, security > Fix For: 1.1.0 > > > The encrypted configuration tool and internal logic to load unprotected > values should handle sensitive values contained in the login identity > providers (like LDAP Manager Password). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2565) NiFi processor to parse logs using Grok patterns
[ https://issues.apache.org/jira/browse/NIFI-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665798#comment-15665798 ] Joseph Witt commented on NIFI-2565: --- removed the fix version for now. Once this is ready for merge the fix version can be assigned. > NiFi processor to parse logs using Grok patterns > > > Key: NIFI-2565 > URL: https://issues.apache.org/jira/browse/NIFI-2565 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Andre > > Following up on Ryan Ward to create a Grok capable parser > https://mail-archives.apache.org/mod_mbox/nifi-dev/201606.mbox/%3CCADD=rnPa8nHkJbeM280=PTQ=wurtwhstm5u+7btoo9pcym2...@mail.gmail.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2565) NiFi processor to parse logs using Grok patterns
[ https://issues.apache.org/jira/browse/NIFI-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-2565: -- Fix Version/s: (was: 1.1.0) > NiFi processor to parse logs using Grok patterns > > > Key: NIFI-2565 > URL: https://issues.apache.org/jira/browse/NIFI-2565 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Andre > > Following up on Ryan Ward to create a Grok capable parser > https://mail-archives.apache.org/mod_mbox/nifi-dev/201606.mbox/%3CCADD=rnPa8nHkJbeM280=PTQ=wurtwhstm5u+7btoo9pcym2...@mail.gmail.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2565) NiFi processor to parse logs using Grok patterns
[ https://issues.apache.org/jira/browse/NIFI-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665797#comment-15665797 ] ASF GitHub Bot commented on NIFI-2565: -- Github user joewitt commented on a diff in the pull request: https://github.com/apache/nifi/pull/1108#discussion_r87937612 --- Diff: nifi-assembly/LICENSE --- @@ -1729,4 +1729,20 @@ This product bundles 'jbzip2' which is available under an MIT license. AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN -THE SOFTWARE. \ No newline at end of file +THE SOFTWARE. --- End diff -- This whole license section can be removed. This is the assembly license which is to cover all binary artifacts and source in the build of nifi itself. The dependency of java-grok is binary only (not source) and is ASLv2 so nothing needs to be in this license for it. There should be an entry for this in the notice similar to the many ASLv2 examples in there. The only thing needing mentioned then is the copyright line from the project's license file https://github.com/thekrakken/java-grok/blob/master/LICENSE. Also, this nifi-asembly/NOTICE change needed will also need to be in the NOTICE of the nifi-standard-nar as well. Lots of words above but the short version is "No license change needed. Just add a small section to the nar NOTICE and assembly NOTICE to reflect this ASLv2 dependency specifically because it has a copyright reference in the license." > NiFi processor to parse logs using Grok patterns > > > Key: NIFI-2565 > URL: https://issues.apache.org/jira/browse/NIFI-2565 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Andre > > Following up on Ryan Ward to create a Grok capable parser > https://mail-archives.apache.org/mod_mbox/nifi-dev/201606.mbox/%3CCADD=rnPa8nHkJbeM280=PTQ=wurtwhstm5u+7btoo9pcym2...@mail.gmail.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi pull request #1108: NIFI-2565: add Grok parser
Github user joewitt commented on a diff in the pull request: https://github.com/apache/nifi/pull/1108#discussion_r87937612 --- Diff: nifi-assembly/LICENSE --- @@ -1729,4 +1729,20 @@ This product bundles 'jbzip2' which is available under an MIT license. AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN -THE SOFTWARE. \ No newline at end of file +THE SOFTWARE. --- End diff -- This whole license section can be removed. This is the assembly license which is to cover all binary artifacts and source in the build of nifi itself. The dependency of java-grok is binary only (not source) and is ASLv2 so nothing needs to be in this license for it. There should be an entry for this in the notice similar to the many ASLv2 examples in there. The only thing needing mentioned then is the copyright line from the project's license file https://github.com/thekrakken/java-grok/blob/master/LICENSE. Also, this nifi-asembly/NOTICE change needed will also need to be in the NOTICE of the nifi-standard-nar as well. Lots of words above but the short version is "No license change needed. Just add a small section to the nar NOTICE and assembly NOTICE to reflect this ASLv2 dependency specifically because it has a copyright reference in the license." --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-1002) support for Listen WebSocket processor
[ https://issues.apache.org/jira/browse/NIFI-1002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665780#comment-15665780 ] Bryan Bende commented on NIFI-1002: --- The change that is causing a compilation problem is to NarCloseable... We used to have: {code} public static NarCloseable withComponentNarLoader(final Class componentClass) { final ClassLoader current = Thread.currentThread().getContextClassLoader(); Thread.currentThread().setContextClassLoader(componentClass.getClassLoader()); return new NarCloseable(current); } {code} Which is replaced with: {code} public static NarCloseable withComponentNarLoader(final Class componentClass, final String componentIdentifier) { final ClassLoader current = Thread.currentThread().getContextClassLoader(); final ClassLoader instanceClassLoader = ExtensionManager.getClassLoader(componentClass.getName(), componentIdentifier); Thread.currentThread().setContextClassLoader(instanceClassLoader); return new NarCloseable(current); } {code} The WebSocket code was using NarCloseable.withComponentNarLoader(WebSocketServerFactory.class) to set the context ClassLoader to the class that loaded WebSocketServerFactory, but I feel like we should try to avoid using NarCloseable outside the framework bundle right? The withComponentNarLoader method was specifically created for wrapping processors, controller services, and reporting tasks. > support for Listen WebSocket processor > --- > > Key: NIFI-1002 > URL: https://issues.apache.org/jira/browse/NIFI-1002 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 0.4.0 >Reporter: sumanth chinthagunta >Priority: Minor > Labels: newbie > > A WebSocket listen processor will be helpful for IoT data ingestion. > I am playing with embedded Vert.X for WebSocket and also ability to put > FlowFiles back to WebSocket client via Vert.X EventBus. > https://github.com/xmlking/nifi-websocket > I am new to NiFi. any advise can be helpful. > PS: I feel forcing Interfaces for Controller Services is unnecessary as in > many cases Controller Services are only used by a set of Processors and > developers usually bundle them together. > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-3036) Resource Claim can violate its "isInUse" assumption causing Null Pointer Exceptions
[ https://issues.apache.org/jira/browse/NIFI-3036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-3036: -- Priority: Blocker (was: Critical) > Resource Claim can violate its "isInUse" assumption causing Null Pointer > Exceptions > --- > > Key: NIFI-3036 > URL: https://issues.apache.org/jira/browse/NIFI-3036 > Project: Apache NiFi > Issue Type: Bug >Reporter: Joseph Percivall >Priority: Blocker > Fix For: 1.1.0 > > > I hit a Null Pointer Exception when testing on the lastest master. After > digging I believe the root cause to be the assumption explained here[1] being > violated. > Specifically I was replaying events on a disk that was over the threshold for > not archiving and seeing this[2] stacktrace. So the claimnant count was > increased from 0 when "isWritable" was false causing the ResourceClaim to > still be a key in the "writeableClaimStreams" map but have a value of null. > I have tested on 1.0.0 and was not able to reproduce. > [1] > https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/repository/claim/StandardResourceClaim.java#L142-L142 > [2] 2016-11-14 16:00:19,783 INFO [Flow Service Tasks Thread-2] > o.a.nifi.controller.StandardFlowService Saved flow controller > org.apache.nifi.controller.FlowController@6191c12d // Another save pending = > false > 2016-11-14 16:00:21,016 INFO [StandardProcessScheduler Thread-4] > o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled > GenerateFlowFile[id=01581004-90ba-149c-7ccc-4fe3d2dcfe12] to run with 1 > threads > 2016-11-14 16:00:21,017 ERROR [Timer-Driven Process Thread-8] > o.a.n.p.standard.GenerateFlowFile > GenerateFlowFile[id=01581004-90ba-149c-7ccc-4fe3d2dcfe12] > GenerateFlowFile[id=01581004-90ba-149c-7ccc-4fe3d2dcfe12] failed to process > due to java.lang.NullPointerException; rolling back session: > java.lang.NullPointerException > 2016-11-14 16:00:21,018 ERROR [Timer-Driven Process Thread-8] > o.a.n.p.standard.GenerateFlowFile > java.lang.NullPointerException: null > at > org.apache.nifi.controller.repository.FileSystemRepository$2.write(FileSystemRepository.java:907) > ~[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] > at > org.apache.nifi.controller.repository.io.DisableOnCloseOutputStream.write(DisableOnCloseOutputStream.java:49) > ~[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] > at > org.apache.nifi.controller.repository.io.ByteCountingOutputStream.write(ByteCountingOutputStream.java:46) > ~[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] > at > org.apache.nifi.controller.repository.io.ByteCountingOutputStream.write(ByteCountingOutputStream.java:41) > ~[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] > at > org.apache.nifi.controller.repository.io.FlowFileAccessOutputStream.write(FlowFileAccessOutputStream.java:78) > ~[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] > at > org.apache.nifi.processors.standard.GenerateFlowFile$1.process(GenerateFlowFile.java:192) > ~[nifi-standard-processors-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] > at > org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2321) > ~[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] > at > org.apache.nifi.processors.standard.GenerateFlowFile.onTrigger(GenerateFlowFile.java:189) > ~[nifi-standard-processors-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > ~[nifi-api-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1089) > [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) > [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) > [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) > [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [na:1.8.0_74] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > [na:1.8.0_74] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > [na:1.8.0_74] > at > java.util.concur
[jira] [Commented] (NIFI-3036) Resource Claim can violate its "isInUse" assumption causing Null Pointer Exceptions
[ https://issues.apache.org/jira/browse/NIFI-3036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665758#comment-15665758 ] Joseph Witt commented on NIFI-3036: --- nice evaluation/analysis [~JPercivall]. I bumped the priority to blocker as this doesn't look like something we'd do the release without. > Resource Claim can violate its "isInUse" assumption causing Null Pointer > Exceptions > --- > > Key: NIFI-3036 > URL: https://issues.apache.org/jira/browse/NIFI-3036 > Project: Apache NiFi > Issue Type: Bug >Reporter: Joseph Percivall >Priority: Blocker > Fix For: 1.1.0 > > > I hit a Null Pointer Exception when testing on the lastest master. After > digging I believe the root cause to be the assumption explained here[1] being > violated. > Specifically I was replaying events on a disk that was over the threshold for > not archiving and seeing this[2] stacktrace. So the claimnant count was > increased from 0 when "isWritable" was false causing the ResourceClaim to > still be a key in the "writeableClaimStreams" map but have a value of null. > I have tested on 1.0.0 and was not able to reproduce. > [1] > https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/repository/claim/StandardResourceClaim.java#L142-L142 > [2] 2016-11-14 16:00:19,783 INFO [Flow Service Tasks Thread-2] > o.a.nifi.controller.StandardFlowService Saved flow controller > org.apache.nifi.controller.FlowController@6191c12d // Another save pending = > false > 2016-11-14 16:00:21,016 INFO [StandardProcessScheduler Thread-4] > o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled > GenerateFlowFile[id=01581004-90ba-149c-7ccc-4fe3d2dcfe12] to run with 1 > threads > 2016-11-14 16:00:21,017 ERROR [Timer-Driven Process Thread-8] > o.a.n.p.standard.GenerateFlowFile > GenerateFlowFile[id=01581004-90ba-149c-7ccc-4fe3d2dcfe12] > GenerateFlowFile[id=01581004-90ba-149c-7ccc-4fe3d2dcfe12] failed to process > due to java.lang.NullPointerException; rolling back session: > java.lang.NullPointerException > 2016-11-14 16:00:21,018 ERROR [Timer-Driven Process Thread-8] > o.a.n.p.standard.GenerateFlowFile > java.lang.NullPointerException: null > at > org.apache.nifi.controller.repository.FileSystemRepository$2.write(FileSystemRepository.java:907) > ~[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] > at > org.apache.nifi.controller.repository.io.DisableOnCloseOutputStream.write(DisableOnCloseOutputStream.java:49) > ~[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] > at > org.apache.nifi.controller.repository.io.ByteCountingOutputStream.write(ByteCountingOutputStream.java:46) > ~[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] > at > org.apache.nifi.controller.repository.io.ByteCountingOutputStream.write(ByteCountingOutputStream.java:41) > ~[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] > at > org.apache.nifi.controller.repository.io.FlowFileAccessOutputStream.write(FlowFileAccessOutputStream.java:78) > ~[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] > at > org.apache.nifi.processors.standard.GenerateFlowFile$1.process(GenerateFlowFile.java:192) > ~[nifi-standard-processors-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] > at > org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2321) > ~[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] > at > org.apache.nifi.processors.standard.GenerateFlowFile.onTrigger(GenerateFlowFile.java:189) > ~[nifi-standard-processors-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > ~[nifi-api-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1089) > [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) > [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) > [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) > [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > [na:1.8.0_74] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > [na:1.8.0_74] > at > java.uti
[jira] [Resolved] (NIFI-2380) ExtractEmailAttachments processor should support TNEF files (aka winmail.dat)
[ https://issues.apache.org/jira/browse/NIFI-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt resolved NIFI-2380. --- Resolution: Fixed > ExtractEmailAttachments processor should support TNEF files (aka winmail.dat) > - > > Key: NIFI-2380 > URL: https://issues.apache.org/jira/browse/NIFI-2380 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.0.0 >Reporter: Andre >Assignee: Joseph Witt > Fix For: 1.1.0 > > > during the review of NIFI-1899 Dan Marshall highlighted some use cases for > email processing that have not been addressed as part of the initial > development cycle. > One of these use cases was the decoding of Microsoft Transport Neutral > Encoding Files (TNEF). > This type of attachments is popularly know as winmail.dat and uses a non RFC > compliant structure to transfer attachments across different Microsoft > Outlook clients. > Given the prevalence of outlook and the issues with winmail.dat files, it > would be nice to be able to decode TNEF as we currently do with MIME > attachments. > Permalink to Dan's comments > http://mail-archives.apache.org/mod_mbox/nifi-dev/201607.mbox/%3C1468716836729-12827.post%40n7.nabble.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2380) ExtractEmailAttachments processor should support TNEF files (aka winmail.dat)
[ https://issues.apache.org/jira/browse/NIFI-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665752#comment-15665752 ] Joseph Witt commented on NIFI-2380: --- +1 merged to master. Needed to adjust some licensing/notice items and clear up an overlapping dependency issue with javax.mail having changed its group/artifact coordinates. Did some local testing and things still look good. Thanks [~trixpan] and [~ozhurakousky] > ExtractEmailAttachments processor should support TNEF files (aka winmail.dat) > - > > Key: NIFI-2380 > URL: https://issues.apache.org/jira/browse/NIFI-2380 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.0.0 >Reporter: Andre >Assignee: Joseph Witt > Fix For: 1.1.0 > > > during the review of NIFI-1899 Dan Marshall highlighted some use cases for > email processing that have not been addressed as part of the initial > development cycle. > One of these use cases was the decoding of Microsoft Transport Neutral > Encoding Files (TNEF). > This type of attachments is popularly know as winmail.dat and uses a non RFC > compliant structure to transfer attachments across different Microsoft > Outlook clients. > Given the prevalence of outlook and the issues with winmail.dat files, it > would be nice to be able to decode TNEF as we currently do with MIME > attachments. > Permalink to Dan's comments > http://mail-archives.apache.org/mod_mbox/nifi-dev/201607.mbox/%3C1468716836729-12827.post%40n7.nabble.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2380) ExtractEmailAttachments processor should support TNEF files (aka winmail.dat)
[ https://issues.apache.org/jira/browse/NIFI-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665745#comment-15665745 ] ASF subversion and git services commented on NIFI-2380: --- Commit a347ebf87b509d54b3fd2aac33f202920b59a4b6 in nifi's branch refs/heads/master from Andre F de Miranda [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=a347ebf ] NIFI-2380 - Introduce ExtractTNEFAttachment - A processor to extract attachments embedded within a Transport Neutral Encapsulation Format (TNEF) attachment (i.e. winmail.dat) > ExtractEmailAttachments processor should support TNEF files (aka winmail.dat) > - > > Key: NIFI-2380 > URL: https://issues.apache.org/jira/browse/NIFI-2380 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.0.0 >Reporter: Andre >Assignee: Joseph Witt > Fix For: 1.1.0 > > > during the review of NIFI-1899 Dan Marshall highlighted some use cases for > email processing that have not been addressed as part of the initial > development cycle. > One of these use cases was the decoding of Microsoft Transport Neutral > Encoding Files (TNEF). > This type of attachments is popularly know as winmail.dat and uses a non RFC > compliant structure to transfer attachments across different Microsoft > Outlook clients. > Given the prevalence of outlook and the issues with winmail.dat files, it > would be nice to be able to decode TNEF as we currently do with MIME > attachments. > Permalink to Dan's comments > http://mail-archives.apache.org/mod_mbox/nifi-dev/201607.mbox/%3C1468716836729-12827.post%40n7.nabble.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi pull request #817: NIFI-2380 - Introduce ExtractTNEFAttachments
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/817 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2380) ExtractEmailAttachments processor should support TNEF files (aka winmail.dat)
[ https://issues.apache.org/jira/browse/NIFI-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665747#comment-15665747 ] ASF GitHub Bot commented on NIFI-2380: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/817 > ExtractEmailAttachments processor should support TNEF files (aka winmail.dat) > - > > Key: NIFI-2380 > URL: https://issues.apache.org/jira/browse/NIFI-2380 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.0.0 >Reporter: Andre >Assignee: Joseph Witt > Fix For: 1.1.0 > > > during the review of NIFI-1899 Dan Marshall highlighted some use cases for > email processing that have not been addressed as part of the initial > development cycle. > One of these use cases was the decoding of Microsoft Transport Neutral > Encoding Files (TNEF). > This type of attachments is popularly know as winmail.dat and uses a non RFC > compliant structure to transfer attachments across different Microsoft > Outlook clients. > Given the prevalence of outlook and the issues with winmail.dat files, it > would be nice to be able to decode TNEF as we currently do with MIME > attachments. > Permalink to Dan's comments > http://mail-archives.apache.org/mod_mbox/nifi-dev/201607.mbox/%3C1468716836729-12827.post%40n7.nabble.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2380) ExtractEmailAttachments processor should support TNEF files (aka winmail.dat)
[ https://issues.apache.org/jira/browse/NIFI-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665746#comment-15665746 ] ASF subversion and git services commented on NIFI-2380: --- Commit 8ad31251687d492639bc6e1a8cf2dea85316b869 in nifi's branch refs/heads/master from [~joewitt] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=8ad3125 ] NIFI-2380 This closes #817. corrected licensing and notice and remove conflicting mail deps > ExtractEmailAttachments processor should support TNEF files (aka winmail.dat) > - > > Key: NIFI-2380 > URL: https://issues.apache.org/jira/browse/NIFI-2380 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.0.0 >Reporter: Andre >Assignee: Joseph Witt > Fix For: 1.1.0 > > > during the review of NIFI-1899 Dan Marshall highlighted some use cases for > email processing that have not been addressed as part of the initial > development cycle. > One of these use cases was the decoding of Microsoft Transport Neutral > Encoding Files (TNEF). > This type of attachments is popularly know as winmail.dat and uses a non RFC > compliant structure to transfer attachments across different Microsoft > Outlook clients. > Given the prevalence of outlook and the issues with winmail.dat files, it > would be nice to be able to decode TNEF as we currently do with MIME > attachments. > Permalink to Dan's comments > http://mail-archives.apache.org/mod_mbox/nifi-dev/201607.mbox/%3C1468716836729-12827.post%40n7.nabble.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1002) support for Listen WebSocket processor
[ https://issues.apache.org/jira/browse/NIFI-1002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665639#comment-15665639 ] Joseph Witt commented on NIFI-1002: --- [~bbende] [~ijokarumawak] [~ozhurakousky] i noticed olegs comment about compilation. Can you please confirm [~bbende] that no backward compatibility issues were introduced in your changes? > support for Listen WebSocket processor > --- > > Key: NIFI-1002 > URL: https://issues.apache.org/jira/browse/NIFI-1002 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 0.4.0 >Reporter: sumanth chinthagunta >Priority: Minor > Labels: newbie > > A WebSocket listen processor will be helpful for IoT data ingestion. > I am playing with embedded Vert.X for WebSocket and also ability to put > FlowFiles back to WebSocket client via Vert.X EventBus. > https://github.com/xmlking/nifi-websocket > I am new to NiFi. any advise can be helpful. > PS: I feel forcing Interfaces for Controller Services is unnecessary as in > many cases Controller Services are only used by a set of Processors and > developers usually bundle them together. > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-3032) NiFiPropertiesLoader can cache stale key after migration
[ https://issues.apache.org/jira/browse/NIFI-3032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665342#comment-15665342 ] ASF GitHub Bot commented on NIFI-3032: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1220 > NiFiPropertiesLoader can cache stale key after migration > > > Key: NIFI-3032 > URL: https://issues.apache.org/jira/browse/NIFI-3032 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.0.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto >Priority: Blocker > Labels: encryption, security > Fix For: 1.1.0 > > > [~bryanrosan...@gmail.com] discovered an issue where the > {{NiFiPropertiesLoader}} can be initialized with a static > {{SensitivePropertyProviderFactory}} containing a specific {{key}}, and then > when {{NiFiPropertiesLoader.withKey()}} is called with a different key, the > factory is not refreshed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-3032) NiFiPropertiesLoader can cache stale key after migration
[ https://issues.apache.org/jira/browse/NIFI-3032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665339#comment-15665339 ] ASF subversion and git services commented on NIFI-3032: --- Commit 4d1bcc808f5c75a7b7bb3a42d79b5a8f28c44d4f in nifi's branch refs/heads/master from [~alopresto] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=4d1bcc8 ] NIFI-3032 Resolved issue where multiple invocations of NiFiPropertiesLoader.withKey() used cached key. Added unit tests and resources. NIFI-3032 Fixed bug in AESSensitivePropertyProvider#getIdentifierKey where the result was always the max available key size, not the size of the current key. Added unit test. This closes #1220 Signed-off-by: Bryan Rosander > NiFiPropertiesLoader can cache stale key after migration > > > Key: NIFI-3032 > URL: https://issues.apache.org/jira/browse/NIFI-3032 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.0.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto >Priority: Blocker > Labels: encryption, security > Fix For: 1.1.0 > > > [~bryanrosan...@gmail.com] discovered an issue where the > {{NiFiPropertiesLoader}} can be initialized with a static > {{SensitivePropertyProviderFactory}} containing a specific {{key}}, and then > when {{NiFiPropertiesLoader.withKey()}} is called with a different key, the > factory is not refreshed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-3032) NiFiPropertiesLoader can cache stale key after migration
[ https://issues.apache.org/jira/browse/NIFI-3032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665340#comment-15665340 ] ASF subversion and git services commented on NIFI-3032: --- Commit 4d1bcc808f5c75a7b7bb3a42d79b5a8f28c44d4f in nifi's branch refs/heads/master from [~alopresto] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=4d1bcc8 ] NIFI-3032 Resolved issue where multiple invocations of NiFiPropertiesLoader.withKey() used cached key. Added unit tests and resources. NIFI-3032 Fixed bug in AESSensitivePropertyProvider#getIdentifierKey where the result was always the max available key size, not the size of the current key. Added unit test. This closes #1220 Signed-off-by: Bryan Rosander > NiFiPropertiesLoader can cache stale key after migration > > > Key: NIFI-3032 > URL: https://issues.apache.org/jira/browse/NIFI-3032 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.0.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto >Priority: Blocker > Labels: encryption, security > Fix For: 1.1.0 > > > [~bryanrosan...@gmail.com] discovered an issue where the > {{NiFiPropertiesLoader}} can be initialized with a static > {{SensitivePropertyProviderFactory}} containing a specific {{key}}, and then > when {{NiFiPropertiesLoader.withKey()}} is called with a different key, the > factory is not refreshed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi pull request #1220: NIFI-3032 Resolved issue where multiple invocations...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1220 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (NIFI-3036) Resource Claim can violate its "isInUse" assumption causing Null Pointer Exceptions
Joseph Percivall created NIFI-3036: -- Summary: Resource Claim can violate its "isInUse" assumption causing Null Pointer Exceptions Key: NIFI-3036 URL: https://issues.apache.org/jira/browse/NIFI-3036 Project: Apache NiFi Issue Type: Bug Reporter: Joseph Percivall Priority: Critical Fix For: 1.1.0 I hit a Null Pointer Exception when testing on the lastest master. After digging I believe the root cause to be the assumption explained here[1] being violated. Specifically I was replaying events on a disk that was over the threshold for not archiving and seeing this[2] stacktrace. So the claimnant count was increased from 0 when "isWritable" was false causing the ResourceClaim to still be a key in the "writeableClaimStreams" map but have a value of null. I have tested on 1.0.0 and was not able to reproduce. [1] https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/repository/claim/StandardResourceClaim.java#L142-L142 [2] 2016-11-14 16:00:19,783 INFO [Flow Service Tasks Thread-2] o.a.nifi.controller.StandardFlowService Saved flow controller org.apache.nifi.controller.FlowController@6191c12d // Another save pending = false 2016-11-14 16:00:21,016 INFO [StandardProcessScheduler Thread-4] o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled GenerateFlowFile[id=01581004-90ba-149c-7ccc-4fe3d2dcfe12] to run with 1 threads 2016-11-14 16:00:21,017 ERROR [Timer-Driven Process Thread-8] o.a.n.p.standard.GenerateFlowFile GenerateFlowFile[id=01581004-90ba-149c-7ccc-4fe3d2dcfe12] GenerateFlowFile[id=01581004-90ba-149c-7ccc-4fe3d2dcfe12] failed to process due to java.lang.NullPointerException; rolling back session: java.lang.NullPointerException 2016-11-14 16:00:21,018 ERROR [Timer-Driven Process Thread-8] o.a.n.p.standard.GenerateFlowFile java.lang.NullPointerException: null at org.apache.nifi.controller.repository.FileSystemRepository$2.write(FileSystemRepository.java:907) ~[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] at org.apache.nifi.controller.repository.io.DisableOnCloseOutputStream.write(DisableOnCloseOutputStream.java:49) ~[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] at org.apache.nifi.controller.repository.io.ByteCountingOutputStream.write(ByteCountingOutputStream.java:46) ~[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] at org.apache.nifi.controller.repository.io.ByteCountingOutputStream.write(ByteCountingOutputStream.java:41) ~[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] at org.apache.nifi.controller.repository.io.FlowFileAccessOutputStream.write(FlowFileAccessOutputStream.java:78) ~[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] at org.apache.nifi.processors.standard.GenerateFlowFile$1.process(GenerateFlowFile.java:192) ~[nifi-standard-processors-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] at org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2321) ~[nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] at org.apache.nifi.processors.standard.GenerateFlowFile.onTrigger(GenerateFlowFile.java:189) ~[nifi-standard-processors-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) ~[nifi-api-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1089) [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) [nifi-framework-core-1.1.0-SNAPSHOT.jar:1.1.0-SNAPSHOT] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_74] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_74] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_74] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_74] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_74] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_74] at java.lang.Thread.run(Thread.java:745) [na:1.8.
[jira] [Updated] (NIFI-3033) GenerateFlowFile Support Dynamic Properties for Generating Attributes
[ https://issues.apache.org/jira/browse/NIFI-3033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Wing updated NIFI-3033: - Status: Patch Available (was: Open) > GenerateFlowFile Support Dynamic Properties for Generating Attributes > - > > Key: NIFI-3033 > URL: https://issues.apache.org/jira/browse/NIFI-3033 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.1.0 >Reporter: James Wing >Priority: Trivial > > Following on the work in NIFI-2912 to allow custom text, GenerateFlowFile > should also accept dynamic properties to evaluate and apply as attributes in > generated flowfiles. This would remove the need for many GenerateFlowFile -> > UpdateAttribute sequences. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-3033) GenerateFlowFile Support Dynamic Properties for Generating Attributes
[ https://issues.apache.org/jira/browse/NIFI-3033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665320#comment-15665320 ] ASF GitHub Bot commented on NIFI-3033: -- GitHub user jvwing opened a pull request: https://github.com/apache/nifi/pull/1222 NIFI-3033 GenerateFlowFile Dynamic Properties Updated GenerateFlowFile for dynamic properties: * Enable dynamic properties with expression language * Apply dynamic property values as attributes to generated flowfiles * Unit test for dynamic properties Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [X] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? - [X] Is your initial contribution a single, squashed commit? ### For code changes: - [X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [X] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jvwing/nifi NIFI-3033-generateflowfile-attributes-1 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1222.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1222 commit c2978ff5cf9c1dac7f701bac2092f3c1b11a9863 Author: James Wing Date: 2016-11-14T21:32:31Z NIFI-3033 GenerateFlowFile Dynamic Properties > GenerateFlowFile Support Dynamic Properties for Generating Attributes > - > > Key: NIFI-3033 > URL: https://issues.apache.org/jira/browse/NIFI-3033 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.1.0 >Reporter: James Wing >Priority: Trivial > > Following on the work in NIFI-2912 to allow custom text, GenerateFlowFile > should also accept dynamic properties to evaluate and apply as attributes in > generated flowfiles. This would remove the need for many GenerateFlowFile -> > UpdateAttribute sequences. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi pull request #1222: NIFI-3033 GenerateFlowFile Dynamic Properties
GitHub user jvwing opened a pull request: https://github.com/apache/nifi/pull/1222 NIFI-3033 GenerateFlowFile Dynamic Properties Updated GenerateFlowFile for dynamic properties: * Enable dynamic properties with expression language * Apply dynamic property values as attributes to generated flowfiles * Unit test for dynamic properties Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [X] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? - [X] Is your initial contribution a single, squashed commit? ### For code changes: - [X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [X] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jvwing/nifi NIFI-3033-generateflowfile-attributes-1 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1222.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1222 commit c2978ff5cf9c1dac7f701bac2092f3c1b11a9863 Author: James Wing Date: 2016-11-14T21:32:31Z NIFI-3033 GenerateFlowFile Dynamic Properties --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2565) NiFi processor to parse logs using Grok patterns
[ https://issues.apache.org/jira/browse/NIFI-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665264#comment-15665264 ] ASF GitHub Bot commented on NIFI-2565: -- Github user trixpan commented on a diff in the pull request: https://github.com/apache/nifi/pull/1108#discussion_r87909770 --- Diff: nifi-commons/nifi-processor-utilities/src/main/java/org/apache/nifi/processor/util/StandardValidators.java --- @@ -26,6 +26,8 @@ import java.util.concurrent.TimeUnit; import java.util.regex.Pattern; +import oi.thekraken.grok.api.Grok; --- End diff -- This validation roytine should not be added to standard validators in order to avoud impirting grok into the standard validator > NiFi processor to parse logs using Grok patterns > > > Key: NIFI-2565 > URL: https://issues.apache.org/jira/browse/NIFI-2565 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Andre > Fix For: 1.1.0 > > > Following up on Ryan Ward to create a Grok capable parser > https://mail-archives.apache.org/mod_mbox/nifi-dev/201606.mbox/%3CCADD=rnPa8nHkJbeM280=PTQ=wurtwhstm5u+7btoo9pcym2...@mail.gmail.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi pull request #1108: NIFI-2565: add Grok parser
Github user trixpan commented on a diff in the pull request: https://github.com/apache/nifi/pull/1108#discussion_r87909770 --- Diff: nifi-commons/nifi-processor-utilities/src/main/java/org/apache/nifi/processor/util/StandardValidators.java --- @@ -26,6 +26,8 @@ import java.util.concurrent.TimeUnit; import java.util.regex.Pattern; +import oi.thekraken.grok.api.Grok; --- End diff -- This validation roytine should not be added to standard validators in order to avoud impirting grok into the standard validator --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (NIFI-2957) ZooKeeper migration toolkit
[ https://issues.apache.org/jira/browse/NIFI-2957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Storck updated NIFI-2957: -- Fix Version/s: (was: 1.2.0) 1.1.0 > ZooKeeper migration toolkit > --- > > Key: NIFI-2957 > URL: https://issues.apache.org/jira/browse/NIFI-2957 > Project: Apache NiFi > Issue Type: New Feature > Components: Tools and Build >Affects Versions: 1.0.0, 0.7.1 >Reporter: Jeff Storck >Assignee: Jeff Storck > Fix For: 1.1.0 > > > When upgrading from NiFi 0.x to 1.x, or when it is desired to move from the > embedded ZooKeeper to an external ZooKeeper, state from ZooKeeper needs to be > migrated. > Initial considerations: > * Username/password protection of nodes is not supported in NiFi 1.x.. Those > nodes that are configured that way in ZooKeeper need to be migrated to have > an open ACL. > * The toolkit will support a mode to read data from a configurable root node > in a source ZooKeeper, and the data will be written to a file designated via > CLI. > * The toolkit will support a mode to write data to a destination ZooKeeper > * The toolkit will not allow data to be written to the same ZooKeeper from > which the source data was obtained. > * The toolkit will not support reconnecting to ZooKeeper if it is > disconnected. The user can rerun the tool. > * The toolkit will support ZooKeepers configured with Kerberos. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi-minifi pull request #43: MINIFI-115 Upgrade to NiFi 1.0 API
Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi-minifi/pull/43#discussion_r87867569 --- Diff: minifi-nar-bundles/minifi-framework-bundle/minifi-framework-nar/pom.xml --- @@ -34,51 +34,70 @@ limitations under the License. true - + +org.apache.nifi.minifi +minifi-framework-core +provided + + +org.apache.nifi.minifi +minifi-api +provided + + + org.apache.nifi nifi-api -provided +compile org.apache.nifi nifi-runtime -provided +compile org.apache.nifi nifi-nar-utils -provided +compile org.apache.nifi nifi-properties -provided +compile -org.apache.nifi.minifi -minifi-framework-core -provided +org.apache.nifi +nifi-security +compile +1.0.0 -org.apache.nifi -nifi-administration -provided +org.eclipse.jetty +jetty-server -org.apache.nifi.minifi -minifi-api -provided +org.eclipse.jetty +jetty-servlet -org.apache.nifi -nifi-framework-core -provided +org.eclipse.jetty +jetty-webapp --- End diff -- This is what precipitated the changes to assembly. Ultimately the motivation is to treat these items as core framework dependencies given their necessity by the configuration change notifiers and should hopefully make items a bit more consistent in terms of where dependencies enter into the project. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2957) ZooKeeper migration toolkit
[ https://issues.apache.org/jira/browse/NIFI-2957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665181#comment-15665181 ] ASF subversion and git services commented on NIFI-2957: --- Commit 7e237341810763ed4631fec8f568c04d8f928788 in nifi's branch refs/heads/master from [~jtstorck] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=7e23734 ] NIFI-2957 Corrections of typos, documentation, and strings Updates made regarding Intellij code inspections This closes #1218 Signed-off-by: Bryan Rosander > ZooKeeper migration toolkit > --- > > Key: NIFI-2957 > URL: https://issues.apache.org/jira/browse/NIFI-2957 > Project: Apache NiFi > Issue Type: New Feature > Components: Tools and Build >Affects Versions: 1.0.0, 0.7.1 >Reporter: Jeff Storck >Assignee: Jeff Storck > Fix For: 1.2.0 > > > When upgrading from NiFi 0.x to 1.x, or when it is desired to move from the > embedded ZooKeeper to an external ZooKeeper, state from ZooKeeper needs to be > migrated. > Initial considerations: > * Username/password protection of nodes is not supported in NiFi 1.x.. Those > nodes that are configured that way in ZooKeeper need to be migrated to have > an open ACL. > * The toolkit will support a mode to read data from a configurable root node > in a source ZooKeeper, and the data will be written to a file designated via > CLI. > * The toolkit will support a mode to write data to a destination ZooKeeper > * The toolkit will not allow data to be written to the same ZooKeeper from > which the source data was obtained. > * The toolkit will not support reconnecting to ZooKeeper if it is > disconnected. The user can rerun the tool. > * The toolkit will support ZooKeepers configured with Kerberos. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2957) ZooKeeper migration toolkit
[ https://issues.apache.org/jira/browse/NIFI-2957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665183#comment-15665183 ] ASF GitHub Bot commented on NIFI-2957: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1218 > ZooKeeper migration toolkit > --- > > Key: NIFI-2957 > URL: https://issues.apache.org/jira/browse/NIFI-2957 > Project: Apache NiFi > Issue Type: New Feature > Components: Tools and Build >Affects Versions: 1.0.0, 0.7.1 >Reporter: Jeff Storck >Assignee: Jeff Storck > Fix For: 1.2.0 > > > When upgrading from NiFi 0.x to 1.x, or when it is desired to move from the > embedded ZooKeeper to an external ZooKeeper, state from ZooKeeper needs to be > migrated. > Initial considerations: > * Username/password protection of nodes is not supported in NiFi 1.x.. Those > nodes that are configured that way in ZooKeeper need to be migrated to have > an open ACL. > * The toolkit will support a mode to read data from a configurable root node > in a source ZooKeeper, and the data will be written to a file designated via > CLI. > * The toolkit will support a mode to write data to a destination ZooKeeper > * The toolkit will not allow data to be written to the same ZooKeeper from > which the source data was obtained. > * The toolkit will not support reconnecting to ZooKeeper if it is > disconnected. The user can rerun the tool. > * The toolkit will support ZooKeepers configured with Kerberos. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi pull request #1218: NIFI-2957 Corrections of typos, documentation, and ...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1218 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2957) ZooKeeper migration toolkit
[ https://issues.apache.org/jira/browse/NIFI-2957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665176#comment-15665176 ] ASF GitHub Bot commented on NIFI-2957: -- Github user brosander commented on the issue: https://github.com/apache/nifi/pull/1218 +1 will merge > ZooKeeper migration toolkit > --- > > Key: NIFI-2957 > URL: https://issues.apache.org/jira/browse/NIFI-2957 > Project: Apache NiFi > Issue Type: New Feature > Components: Tools and Build >Affects Versions: 1.0.0, 0.7.1 >Reporter: Jeff Storck >Assignee: Jeff Storck > Fix For: 1.2.0 > > > When upgrading from NiFi 0.x to 1.x, or when it is desired to move from the > embedded ZooKeeper to an external ZooKeeper, state from ZooKeeper needs to be > migrated. > Initial considerations: > * Username/password protection of nodes is not supported in NiFi 1.x.. Those > nodes that are configured that way in ZooKeeper need to be migrated to have > an open ACL. > * The toolkit will support a mode to read data from a configurable root node > in a source ZooKeeper, and the data will be written to a file designated via > CLI. > * The toolkit will support a mode to write data to a destination ZooKeeper > * The toolkit will not allow data to be written to the same ZooKeeper from > which the source data was obtained. > * The toolkit will not support reconnecting to ZooKeeper if it is > disconnected. The user can rerun the tool. > * The toolkit will support ZooKeepers configured with Kerberos. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi issue #1218: NIFI-2957 Corrections of typos, documentation, and strings
Github user brosander commented on the issue: https://github.com/apache/nifi/pull/1218 +1 will merge --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi pull request #43: MINIFI-115 Upgrade to NiFi 1.0 API
Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi-minifi/pull/43#discussion_r87867250 --- Diff: minifi-assembly/src/main/assembly/dependencies.xml --- @@ -64,15 +79,11 @@ true minifi-bootstrap -slf4j-api -logback-classic -nifi-api +minifi-api +minifi-commons-schema +minifi-utils +snakeyaml nifi-utils -jetty-server --- End diff -- These are still present, but transitively included in lib from the other module(s) that need them. Wanted to ensure we minimized noise in terms of items here to make things a bit more manageable. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi pull request #43: MINIFI-115 Upgrade to NiFi 1.0 API
Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi-minifi/pull/43#discussion_r87900582 --- Diff: minifi-assembly/pom.xml --- @@ -177,23 +182,57 @@ limitations under the License. +org.bouncycastle +bcprov-jdk15on +1.54 +compile + + org.eclipse.jetty -jetty-servlet +jetty-util +9.3.9.v20160517 compile +org.apache.commons +commons-lang3 + + +org.apache.httpcomponents +httpclient + + +com.google.guava +guava --- End diff -- This is brought in by the nifi-framework-nar, so believe it is additionally needed at a lower level to support those components that may use it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi pull request #43: MINIFI-115 Upgrade to NiFi 1.0 API
Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi-minifi/pull/43#discussion_r87866531 --- Diff: minifi-assembly/src/main/assembly/dependencies.xml --- @@ -36,23 +36,38 @@ minifi-bootstrap minifi-resources + org.apache.nifi:nifi-framework-core:1.0.0 +zookeeper spring-aop spring-context -spring-security-core spring-beans +spring-expression swagger-annotations slf4j-log4j12 aspectjweaver h2 netty jaxb-impl -httpclient mail log4j +lucene-analyzers-common lucene-queryparser commons-net +spring-context +spring-security-core + +runtime +false +lib --- End diff -- This is to provide explicit inclusion of the following artifacts. I found out the tough way that the include/exclude directives will do a simple pattern matching. So nifi-framework-core also excludes minifi-framework-core given the match. Will put a comment on this to clarify. Had one at some point, but got lost to the sands of time and many iterations. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi pull request #43: MINIFI-115 Upgrade to NiFi 1.0 API
Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi-minifi/pull/43#discussion_r87867638 --- Diff: minifi-nar-bundles/minifi-framework-bundle/minifi-framework/minifi-framework-core/pom.xml --- @@ -27,6 +27,48 @@ limitations under the License. +org.eclipse.jetty +jetty-server +9.3.9.v20160517 --- End diff -- yep, can migrate --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi pull request #43: MINIFI-115 Upgrade to NiFi 1.0 API
Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi-minifi/pull/43#discussion_r87867758 --- Diff: minifi-nar-bundles/minifi-framework-bundle/minifi-framework/minifi-framework-core/src/main/java/org/apache/nifi/minifi/status/StatusRequestParser.java --- @@ -122,15 +122,13 @@ static RemoteProcessGroupStatusBean parseRemoteProcessGroupStatusRequest(RemoteP new BulletinQuery.Builder() .sourceIdMatches(inputRemoteProcessGroupStatus.getId()) .build()); -List authorizationIssues = inputRemoteProcessGroupStatus.getAuthorizationIssues(); --- End diff -- Just a note to myself that I removed the corresponding doc item. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3032) NiFiPropertiesLoader can cache stale key after migration
[ https://issues.apache.org/jira/browse/NIFI-3032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665116#comment-15665116 ] ASF GitHub Bot commented on NIFI-3032: -- Github user brosander commented on a diff in the pull request: https://github.com/apache/nifi/pull/1220#discussion_r87898225 --- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-properties-loader/src/main/java/org/apache/nifi/properties/NiFiPropertiesLoader.java --- @@ -158,9 +158,7 @@ private static String getDefaultProviderKey() { } private void initializeSensitivePropertyProviderFactory() { -if (sensitivePropertyProviderFactory == null) { -sensitivePropertyProviderFactory = new AESSensitivePropertyProviderFactory(keyHex); -} +sensitivePropertyProviderFactory = new AESSensitivePropertyProviderFactory(keyHex); --- End diff -- Makes sense, probably not worth the effort since it won't be used that way. > NiFiPropertiesLoader can cache stale key after migration > > > Key: NIFI-3032 > URL: https://issues.apache.org/jira/browse/NIFI-3032 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.0.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto >Priority: Blocker > Labels: encryption, security > Fix For: 1.1.0 > > > [~bryanrosan...@gmail.com] discovered an issue where the > {{NiFiPropertiesLoader}} can be initialized with a static > {{SensitivePropertyProviderFactory}} containing a specific {{key}}, and then > when {{NiFiPropertiesLoader.withKey()}} is called with a different key, the > factory is not refreshed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi pull request #1220: NIFI-3032 Resolved issue where multiple invocations...
Github user brosander commented on a diff in the pull request: https://github.com/apache/nifi/pull/1220#discussion_r87898225 --- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-properties-loader/src/main/java/org/apache/nifi/properties/NiFiPropertiesLoader.java --- @@ -158,9 +158,7 @@ private static String getDefaultProviderKey() { } private void initializeSensitivePropertyProviderFactory() { -if (sensitivePropertyProviderFactory == null) { -sensitivePropertyProviderFactory = new AESSensitivePropertyProviderFactory(keyHex); -} +sensitivePropertyProviderFactory = new AESSensitivePropertyProviderFactory(keyHex); --- End diff -- Makes sense, probably not worth the effort since it won't be used that way. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3032) NiFiPropertiesLoader can cache stale key after migration
[ https://issues.apache.org/jira/browse/NIFI-3032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665107#comment-15665107 ] ASF GitHub Bot commented on NIFI-3032: -- Github user alopresto commented on a diff in the pull request: https://github.com/apache/nifi/pull/1220#discussion_r87897741 --- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-properties-loader/src/main/java/org/apache/nifi/properties/NiFiPropertiesLoader.java --- @@ -158,9 +158,7 @@ private static String getDefaultProviderKey() { } private void initializeSensitivePropertyProviderFactory() { -if (sensitivePropertyProviderFactory == null) { -sensitivePropertyProviderFactory = new AESSensitivePropertyProviderFactory(keyHex); -} +sensitivePropertyProviderFactory = new AESSensitivePropertyProviderFactory(keyHex); --- End diff -- It will if two competing resources are attempting to access it, but this scenario should not occur (there should not be two resources with different keys trying to load the data in the real application, as there is only one master key). The reason it was failing in your original test was due to sequential access not being cleared after migration. If we are concerned about race conditions, this should not be `static` at all and more of the organization needs to change. > NiFiPropertiesLoader can cache stale key after migration > > > Key: NIFI-3032 > URL: https://issues.apache.org/jira/browse/NIFI-3032 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.0.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto >Priority: Blocker > Labels: encryption, security > Fix For: 1.1.0 > > > [~bryanrosan...@gmail.com] discovered an issue where the > {{NiFiPropertiesLoader}} can be initialized with a static > {{SensitivePropertyProviderFactory}} containing a specific {{key}}, and then > when {{NiFiPropertiesLoader.withKey()}} is called with a different key, the > factory is not refreshed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi pull request #1220: NIFI-3032 Resolved issue where multiple invocations...
Github user alopresto commented on a diff in the pull request: https://github.com/apache/nifi/pull/1220#discussion_r87897741 --- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-properties-loader/src/main/java/org/apache/nifi/properties/NiFiPropertiesLoader.java --- @@ -158,9 +158,7 @@ private static String getDefaultProviderKey() { } private void initializeSensitivePropertyProviderFactory() { -if (sensitivePropertyProviderFactory == null) { -sensitivePropertyProviderFactory = new AESSensitivePropertyProviderFactory(keyHex); -} +sensitivePropertyProviderFactory = new AESSensitivePropertyProviderFactory(keyHex); --- End diff -- It will if two competing resources are attempting to access it, but this scenario should not occur (there should not be two resources with different keys trying to load the data in the real application, as there is only one master key). The reason it was failing in your original test was due to sequential access not being cleared after migration. If we are concerned about race conditions, this should not be `static` at all and more of the organization needs to change. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2957) ZooKeeper migration toolkit
[ https://issues.apache.org/jira/browse/NIFI-2957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665102#comment-15665102 ] ASF GitHub Bot commented on NIFI-2957: -- Github user brosander commented on the issue: https://github.com/apache/nifi/pull/1218 Reviewing > ZooKeeper migration toolkit > --- > > Key: NIFI-2957 > URL: https://issues.apache.org/jira/browse/NIFI-2957 > Project: Apache NiFi > Issue Type: New Feature > Components: Tools and Build >Affects Versions: 1.0.0, 0.7.1 >Reporter: Jeff Storck >Assignee: Jeff Storck > Fix For: 1.2.0 > > > When upgrading from NiFi 0.x to 1.x, or when it is desired to move from the > embedded ZooKeeper to an external ZooKeeper, state from ZooKeeper needs to be > migrated. > Initial considerations: > * Username/password protection of nodes is not supported in NiFi 1.x.. Those > nodes that are configured that way in ZooKeeper need to be migrated to have > an open ACL. > * The toolkit will support a mode to read data from a configurable root node > in a source ZooKeeper, and the data will be written to a file designated via > CLI. > * The toolkit will support a mode to write data to a destination ZooKeeper > * The toolkit will not allow data to be written to the same ZooKeeper from > which the source data was obtained. > * The toolkit will not support reconnecting to ZooKeeper if it is > disconnected. The user can rerun the tool. > * The toolkit will support ZooKeepers configured with Kerberos. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi issue #1218: NIFI-2957 Corrections of typos, documentation, and strings
Github user brosander commented on the issue: https://github.com/apache/nifi/pull/1218 Reviewing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3024) Encrypted configuration migrator should be able to update sensitive properties key and migrate flow.xml.gz
[ https://issues.apache.org/jira/browse/NIFI-3024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665097#comment-15665097 ] ASF GitHub Bot commented on NIFI-3024: -- GitHub user brosander opened a pull request: https://github.com/apache/nifi/pull/1221 NIFI-3024 - Encrypt-toolkit flow.xml.gz support Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [x] - N/A - If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] - N/A - If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [x] - N/A - If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [x] - N/A - If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/brosander/nifi NIFI-3024-rebase Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1221.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1221 commit f5e8db68f4aedeb9dc7a2a5b0ba12d115e80b931 Author: Bryan Rosander Date: 2016-11-11T17:24:35Z NIFI-3024 - Encrypt-toolkit flow.xml.gz support > Encrypted configuration migrator should be able to update sensitive > properties key and migrate flow.xml.gz > -- > > Key: NIFI-3024 > URL: https://issues.apache.org/jira/browse/NIFI-3024 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration, Tools and Build >Affects Versions: 1.0.0 >Reporter: Bryan Rosander >Assignee: Bryan Rosander > Labels: config, encryption, security, serialization > > In order to allow changing of nifi.sensitive.props.key and updating of the > flow.xml.gz, the ConfigEncryptionTool should be able to accept a new value > for that field and update encrypted values in the flow.xml.gz appropriately. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi pull request #1221: NIFI-3024 - Encrypt-toolkit flow.xml.gz support
GitHub user brosander opened a pull request: https://github.com/apache/nifi/pull/1221 NIFI-3024 - Encrypt-toolkit flow.xml.gz support Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [x] - N/A - If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] - N/A - If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [x] - N/A - If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [x] - N/A - If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/brosander/nifi NIFI-3024-rebase Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1221.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1221 commit f5e8db68f4aedeb9dc7a2a5b0ba12d115e80b931 Author: Bryan Rosander Date: 2016-11-11T17:24:35Z NIFI-3024 - Encrypt-toolkit flow.xml.gz support --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (NIFI-3023) UI - RPG transmission status
[ https://issues.apache.org/jira/browse/NIFI-3023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Aslan updated NIFI-3023: -- Resolution: Fixed Status: Resolved (was: Patch Available) > UI - RPG transmission status > > > Key: NIFI-3023 > URL: https://issues.apache.org/jira/browse/NIFI-3023 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman > Fix For: 1.1.0 > > > Remote Process Group transmission status is currently attempting to access > the component configuration. This is currently preventing the UI from loading > for a user without permissions. Need to ensure we are only assigning the > classes in question when the user has permissions to view the Remote Process > Group. > {code} > Uncaught TypeError: Cannot read property 'transmitting' of undefined > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi issue #1205: UI - Verifying permissions prior to checking Remote Proces...
Github user scottyaslan commented on the issue: https://github.com/apache/nifi/pull/1205 Thanks @mcgilman this has been merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3023) UI - RPG transmission status
[ https://issues.apache.org/jira/browse/NIFI-3023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665073#comment-15665073 ] ASF subversion and git services commented on NIFI-3023: --- Commit 2c91a1a33d2d085f50ac25addffd2981af347551 in nifi's branch refs/heads/master from [~mcgilman] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=2c91a1a ] NIFI-3023: - Verifying permissions prior to checking Remote Process Group transmission status. > UI - RPG transmission status > > > Key: NIFI-3023 > URL: https://issues.apache.org/jira/browse/NIFI-3023 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman > Fix For: 1.1.0 > > > Remote Process Group transmission status is currently attempting to access > the component configuration. This is currently preventing the UI from loading > for a user without permissions. Need to ensure we are only assigning the > classes in question when the user has permissions to view the Remote Process > Group. > {code} > Uncaught TypeError: Cannot read property 'transmitting' of undefined > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-3023) UI - RPG transmission status
[ https://issues.apache.org/jira/browse/NIFI-3023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665078#comment-15665078 ] ASF GitHub Bot commented on NIFI-3023: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1205 > UI - RPG transmission status > > > Key: NIFI-3023 > URL: https://issues.apache.org/jira/browse/NIFI-3023 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman > Fix For: 1.1.0 > > > Remote Process Group transmission status is currently attempting to access > the component configuration. This is currently preventing the UI from loading > for a user without permissions. Need to ensure we are only assigning the > classes in question when the user has permissions to view the Remote Process > Group. > {code} > Uncaught TypeError: Cannot read property 'transmitting' of undefined > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-3023) UI - RPG transmission status
[ https://issues.apache.org/jira/browse/NIFI-3023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665074#comment-15665074 ] ASF subversion and git services commented on NIFI-3023: --- Commit ae4f27a627a14960d764aa330424745263edde8b in nifi's branch refs/heads/master from [~mcgilman] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=ae4f27a ] NIFI-3023: - Ensuring there are no authorization issues prior to considering the remote process group's transmitting flag. This closes #1205 > UI - RPG transmission status > > > Key: NIFI-3023 > URL: https://issues.apache.org/jira/browse/NIFI-3023 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman > Fix For: 1.1.0 > > > Remote Process Group transmission status is currently attempting to access > the component configuration. This is currently preventing the UI from loading > for a user without permissions. Need to ensure we are only assigning the > classes in question when the user has permissions to view the Remote Process > Group. > {code} > Uncaught TypeError: Cannot read property 'transmitting' of undefined > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi pull request #1205: UI - Verifying permissions prior to checking Remote...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1205 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2926) Add a user-centric view for authorization policies
[ https://issues.apache.org/jira/browse/NIFI-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665019#comment-15665019 ] ASF GitHub Bot commented on NIFI-2926: -- Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/1188 @markap14 Sounds great. Thanks! > Add a user-centric view for authorization policies > -- > > Key: NIFI-2926 > URL: https://issues.apache.org/jira/browse/NIFI-2926 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Affects Versions: 1.0.0 >Reporter: Andrew Lim >Assignee: Scott Aslan > Labels: UI, authorization > > The UI for managing authorizations in 1.0.0 is policy-centric, meaning in > order to view which access privileges a specific user has, you need to > navigate to each individual policy and see if the user has been added to it. > We should add a view to the UI where you can select a user and then see all > the access policies that he/she has. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi issue #1188: [NIFI-2926] add policy action item to the user table and a...
Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/1188 @markap14 Sounds great. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1188: [NIFI-2926] add policy action item to the user table and a...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/1188 @mcgilman I'd be happy to review this if you need another reviewer given that you and @scottyaslan have both provided commits for this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2926) Add a user-centric view for authorization policies
[ https://issues.apache.org/jira/browse/NIFI-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665018#comment-15665018 ] ASF GitHub Bot commented on NIFI-2926: -- Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/1188 @mcgilman I'd be happy to review this if you need another reviewer given that you and @scottyaslan have both provided commits for this PR. > Add a user-centric view for authorization policies > -- > > Key: NIFI-2926 > URL: https://issues.apache.org/jira/browse/NIFI-2926 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Affects Versions: 1.0.0 >Reporter: Andrew Lim >Assignee: Scott Aslan > Labels: UI, authorization > > The UI for managing authorizations in 1.0.0 is policy-centric, meaning in > order to view which access privileges a specific user has, you need to > navigate to each individual policy and see if the user has been added to it. > We should add a view to the UI where you can select a user and then see all > the access policies that he/she has. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-3035) URL to display a particular process group in UI
Christine Draper created NIFI-3035: -- Summary: URL to display a particular process group in UI Key: NIFI-3035 URL: https://issues.apache.org/jira/browse/NIFI-3035 Project: Apache NiFi Issue Type: New Feature Components: Core UI Reporter: Christine Draper Our use case has multiple teams of users working on specific process groups. We would like to be able to give them a URL that will launch the UI on the specific group they are working on, rather than them having to navigate to it from the root group. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-3011) Support Elasticsearch 5.0 for Put/FetchElasticsearch processors
[ https://issues.apache.org/jira/browse/NIFI-3011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15664940#comment-15664940 ] Andy LoPresto commented on NIFI-3011: - It appears it does not mention it in the main X-Pack documentation, but in the [advanced settings|https://www.elastic.co/guide/en/x-pack/current/security-settings.html#_java_keystore_files], there does appear to still be an option for keystore and truststore files and passwords. In addition, it appears there is an option for {{xpack.ssl.key_passphrase}}, which would allow a password to be used with {{xpack.ssl.key}} as mentioned above. > Support Elasticsearch 5.0 for Put/FetchElasticsearch processors > --- > > Key: NIFI-3011 > URL: https://issues.apache.org/jira/browse/NIFI-3011 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess > > Now that Elastic has released a new major version (5.0) of Elasticsearch, the > Put/FetchElasticsearch processors would need to be upgraded (or duplicated) > as the major version of the transport client needs to match the major version > of the Elasticsearch cluster. > If upgrade is selected, then Put/FetchES will no longer work with > Elasticsearch 2.x clusters, so in that case users would want to switch to the > Http versions of those processors. However this might not be desirable (due > to performance concerns with the HTTP API vs the transport API), so care must > be taken when deciding whether to upgrade the existing processors or create > new ones. > Creating new versions of these processors (to use the 5.0 transport client) > will also take some consideration, as it is unlikely the different versions > can coexist in the same NAR due to classloading issues (multiple versions of > JARs containing the same class names, e.g.). It may be necessary to create an > "elasticsearch-5.0" version of the NAR, containing only the new versions of > these processors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-3011) Support Elasticsearch 5.0 for Put/FetchElasticsearch processors
[ https://issues.apache.org/jira/browse/NIFI-3011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15664929#comment-15664929 ] Andy LoPresto commented on NIFI-3011: - After discussing with [~mattyb149], it appears the way X-Pack (the new client) performs TLS configuration is less secure than it previously was. The old client lib accepted Java keystore and truststore paths with their respective passwords, providing a more secure approach compared to the new client lib which wants the raw private key in PEM encoded format, the server certificate in PEM format, and the CA certificate(s) in PEM format. This means that the server private key (as they refer to it, the *node private key* or {{xpack.ssl.key}} is no longer password-protected and is just stored on disk, accessible to anyone with OS-level access. In fact, the [documentation|https://www.elastic.co/guide/en/x-pack/current/ssl-tls.html] was clearly not updated completely, as it still refers to "keystore" and "password(s)" immediately above. {quote} To enable SSL, make the following changes in elasticsearch.yml: Specify the location of the node’s keystore and the password(s) needed to access the node’s certificate. For example: {code} xpack.ssl.key: /home/es/config/x-pack/node01.key xpack.ssl.certificate: /home/es/config/x-pack/node01.crt xpack.ssl.certificate_authorities: [ "/home/es/config/x-pack/ca.crt" ] {code} The full path to the node key file. This must be a location within the Elasticsearch configuration directory. The full path to the node certificate. This must be a location within the Elasticsearch configuration directory. An array of paths to the CA certificates that should be trusted. These paths must be a location within the Elasticsearch configuration directory. {quote} > Support Elasticsearch 5.0 for Put/FetchElasticsearch processors > --- > > Key: NIFI-3011 > URL: https://issues.apache.org/jira/browse/NIFI-3011 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess > > Now that Elastic has released a new major version (5.0) of Elasticsearch, the > Put/FetchElasticsearch processors would need to be upgraded (or duplicated) > as the major version of the transport client needs to match the major version > of the Elasticsearch cluster. > If upgrade is selected, then Put/FetchES will no longer work with > Elasticsearch 2.x clusters, so in that case users would want to switch to the > Http versions of those processors. However this might not be desirable (due > to performance concerns with the HTTP API vs the transport API), so care must > be taken when deciding whether to upgrade the existing processors or create > new ones. > Creating new versions of these processors (to use the 5.0 transport client) > will also take some consideration, as it is unlikely the different versions > can coexist in the same NAR due to classloading issues (multiple versions of > JARs containing the same class names, e.g.). It may be necessary to create an > "elasticsearch-5.0" version of the NAR, containing only the new versions of > these processors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-3032) NiFiPropertiesLoader can cache stale key after migration
[ https://issues.apache.org/jira/browse/NIFI-3032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15664916#comment-15664916 ] ASF GitHub Bot commented on NIFI-3032: -- Github user brosander commented on a diff in the pull request: https://github.com/apache/nifi/pull/1220#discussion_r87885510 --- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-properties-loader/src/main/java/org/apache/nifi/properties/AESSensitivePropertyProvider.java --- @@ -16,14 +16,15 @@ */ package org.apache.nifi.properties; +import static sun.security.util.KeyUtil.getKeySize; --- End diff -- @alopresto this may not work everywhere. > NiFiPropertiesLoader can cache stale key after migration > > > Key: NIFI-3032 > URL: https://issues.apache.org/jira/browse/NIFI-3032 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.0.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto >Priority: Blocker > Labels: encryption, security > Fix For: 1.1.0 > > > [~bryanrosan...@gmail.com] discovered an issue where the > {{NiFiPropertiesLoader}} can be initialized with a static > {{SensitivePropertyProviderFactory}} containing a specific {{key}}, and then > when {{NiFiPropertiesLoader.withKey()}} is called with a different key, the > factory is not refreshed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi pull request #1220: NIFI-3032 Resolved issue where multiple invocations...
Github user brosander commented on a diff in the pull request: https://github.com/apache/nifi/pull/1220#discussion_r87885510 --- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-properties-loader/src/main/java/org/apache/nifi/properties/AESSensitivePropertyProvider.java --- @@ -16,14 +16,15 @@ */ package org.apache.nifi.properties; +import static sun.security.util.KeyUtil.getKeySize; --- End diff -- @alopresto this may not work everywhere. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3032) NiFiPropertiesLoader can cache stale key after migration
[ https://issues.apache.org/jira/browse/NIFI-3032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15664903#comment-15664903 ] ASF GitHub Bot commented on NIFI-3032: -- Github user brosander commented on a diff in the pull request: https://github.com/apache/nifi/pull/1220#discussion_r87884181 --- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-properties-loader/src/main/java/org/apache/nifi/properties/NiFiPropertiesLoader.java --- @@ -158,9 +158,7 @@ private static String getDefaultProviderKey() { } private void initializeSensitivePropertyProviderFactory() { -if (sensitivePropertyProviderFactory == null) { -sensitivePropertyProviderFactory = new AESSensitivePropertyProviderFactory(keyHex); -} +sensitivePropertyProviderFactory = new AESSensitivePropertyProviderFactory(keyHex); --- End diff -- @alopresto won't this still have race conditions if more than one thread does NiFiPropertiesLoader.withKey() at the same time? > NiFiPropertiesLoader can cache stale key after migration > > > Key: NIFI-3032 > URL: https://issues.apache.org/jira/browse/NIFI-3032 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.0.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto >Priority: Blocker > Labels: encryption, security > Fix For: 1.1.0 > > > [~bryanrosan...@gmail.com] discovered an issue where the > {{NiFiPropertiesLoader}} can be initialized with a static > {{SensitivePropertyProviderFactory}} containing a specific {{key}}, and then > when {{NiFiPropertiesLoader.withKey()}} is called with a different key, the > factory is not refreshed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi pull request #1220: NIFI-3032 Resolved issue where multiple invocations...
Github user brosander commented on a diff in the pull request: https://github.com/apache/nifi/pull/1220#discussion_r87884181 --- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-properties-loader/src/main/java/org/apache/nifi/properties/NiFiPropertiesLoader.java --- @@ -158,9 +158,7 @@ private static String getDefaultProviderKey() { } private void initializeSensitivePropertyProviderFactory() { -if (sensitivePropertyProviderFactory == null) { -sensitivePropertyProviderFactory = new AESSensitivePropertyProviderFactory(keyHex); -} +sensitivePropertyProviderFactory = new AESSensitivePropertyProviderFactory(keyHex); --- End diff -- @alopresto won't this still have race conditions if more than one thread does NiFiPropertiesLoader.withKey() at the same time? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---