[jira] [Commented] (NIFI-1490) Add multipart request support to ListenHTTP Processor
[ https://issues.apache.org/jira/browse/NIFI-1490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022368#comment-16022368 ] David Patterson commented on NIFI-1490: --- Great to see this feature progressing. It would be helpful if you could name form fields to be used as flow file attributes in the processor config. > Add multipart request support to ListenHTTP Processor > - > > Key: NIFI-1490 > URL: https://issues.apache.org/jira/browse/NIFI-1490 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Andrew Serff > > The current ListenHTTP processor does not seem to support multipart requests > that are encoded with multipart/form-data. When a multipart request is > received, the ListenHTTPServlet just copies the Request InputStream to the > FlowFiles content which leaves the form encoding wrapper in the content and > in turn makes the file invalid. > Specifically, we want to be able to support file uploads in a multipart > request. > See this thread in the mailing list for more info: > http://mail-archives.apache.org/mod_mbox/nifi-users/201602.mbox/%3C6DE9CEEF-2A37-480F-8D3C-5028C590FD9E%40acesinc.net%3E -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-329) Provide processor(s) to interact with IRC
[
https://issues.apache.org/jira/browse/NIFI-329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022317#comment-16022317
]
ASF GitHub Bot commented on NIFI-329:
-
Github user joewitt commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1541#discussion_r118161898
--- Diff:
nifi-nar-bundles/nifi-irc-bundle/nifi-irc-client-service/src/main/java/org/apache/nifi/irc/StandardIRCClientService.java
---
@@ -0,0 +1,233 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.irc;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.security.KeyStore;
+import java.security.KeyStoreException;
+import java.security.NoSuchAlgorithmException;
+import java.security.Provider;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnDisabled;
+import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.irc.handlers.ServiceEventHandler;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.reporting.InitializationException;
+import org.apache.nifi.ssl.SSLContextService;
+import org.apache.nifi.security.util.KeyStoreUtils;
+import org.apache.nifi.util.StringUtils;
+import org.kitteh.irc.client.library.Client;
+import org.kitteh.irc.client.library.feature.sending.SingleDelaySender;
+
+
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.TrustManagerFactory;
+
+@Tags({ "irc"})
+@CapabilityDescription("IRC client controller.")
+public class StandardIRCClientService extends AbstractControllerService
implements IRCClientService {
+
+public static final PropertyDescriptor IRC_SERVER = new
PropertyDescriptor
+.Builder().name("IRC_SERVER")
+.displayName("IRC Server")
+.description("The IRC server you want to connect to")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+public static final PropertyDescriptor IRC_SERVER_PORT = new
PropertyDescriptor
+.Builder().name("IRC_SERVER_PORT")
+.displayName("IRC Server Port")
+.description("The IRC server port you want to connect to")
+.required(true)
+.defaultValue("6667")
+.addValidator(StandardValidators.PORT_VALIDATOR)
+.build();
+public static final PropertyDescriptor SSL_CONTEXT_SERVICE = new
PropertyDescriptor.Builder()
+.name("SSL_CONTEXT_SERVICE")
+.displayName("SSL Context Service")
+.description("The Controller Service to use in order to obtain
an SSL Context. If this property is set, "
++ "IRC connection will be established over a secure
connection.")
+.required(false)
+.identifiesControllerService(SSLContextService.class)
+.build();
+public static final PropertyDescriptor IRC_NICK = new
PropertyDescriptor
+.Builder().name("IRC_NICK")
+.displayName("Nickname")
+.description("The Nickname to use when conne
[GitHub] nifi pull request #1541: NIFI-329 - Introduce IRC Client Services and Consum...
Github user joewitt commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1541#discussion_r118161898
--- Diff:
nifi-nar-bundles/nifi-irc-bundle/nifi-irc-client-service/src/main/java/org/apache/nifi/irc/StandardIRCClientService.java
---
@@ -0,0 +1,233 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.irc;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.security.KeyStore;
+import java.security.KeyStoreException;
+import java.security.NoSuchAlgorithmException;
+import java.security.Provider;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnDisabled;
+import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.irc.handlers.ServiceEventHandler;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.reporting.InitializationException;
+import org.apache.nifi.ssl.SSLContextService;
+import org.apache.nifi.security.util.KeyStoreUtils;
+import org.apache.nifi.util.StringUtils;
+import org.kitteh.irc.client.library.Client;
+import org.kitteh.irc.client.library.feature.sending.SingleDelaySender;
+
+
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.TrustManagerFactory;
+
+@Tags({ "irc"})
+@CapabilityDescription("IRC client controller.")
+public class StandardIRCClientService extends AbstractControllerService
implements IRCClientService {
+
+public static final PropertyDescriptor IRC_SERVER = new
PropertyDescriptor
+.Builder().name("IRC_SERVER")
+.displayName("IRC Server")
+.description("The IRC server you want to connect to")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+public static final PropertyDescriptor IRC_SERVER_PORT = new
PropertyDescriptor
+.Builder().name("IRC_SERVER_PORT")
+.displayName("IRC Server Port")
+.description("The IRC server port you want to connect to")
+.required(true)
+.defaultValue("6667")
+.addValidator(StandardValidators.PORT_VALIDATOR)
+.build();
+public static final PropertyDescriptor SSL_CONTEXT_SERVICE = new
PropertyDescriptor.Builder()
+.name("SSL_CONTEXT_SERVICE")
+.displayName("SSL Context Service")
+.description("The Controller Service to use in order to obtain
an SSL Context. If this property is set, "
++ "IRC connection will be established over a secure
connection.")
+.required(false)
+.identifiesControllerService(SSLContextService.class)
+.build();
+public static final PropertyDescriptor IRC_NICK = new
PropertyDescriptor
+.Builder().name("IRC_NICK")
+.displayName("Nickname")
+.description("The Nickname to use when connecting to the IRC
server")
+.required(true)
+.defaultValue("NiFi")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+public static final PropertyDescriptor IRC_S
[jira] [Commented] (NIFI-329) Provide processor(s) to interact with IRC
[ https://issues.apache.org/jira/browse/NIFI-329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022315#comment-16022315 ] ASF GitHub Bot commented on NIFI-329: - Github user joewitt commented on a diff in the pull request: https://github.com/apache/nifi/pull/1541#discussion_r118161695 --- Diff: nifi-assembly/NOTICE --- @@ -1364,6 +1364,8 @@ The following binary components are provided under the MIT License. See project (MIT License) EWS Java API (com.microsoft.ews-java-api:ews-java-api:2.0 - https://github.com/OfficeDev/ews-java-api) (MIT License) libffi (libffi-3.2.1 - http://sourceware.org/libffi/) + (MIT License) Kitteh IRC Client Library (kitteh client-lib 3.0.3 - http://kicl.kitteh.org/) + (MIT License) 'mbassador' (mbassador 1.2.4 - https://github.com/bennidi/mbassador) --- End diff -- lines 1365-1368 should be deleted. MIT licensed dependencies get referenced in the LICENSE and do not end up in the NOTICE. > Provide processor(s) to interact with IRC > - > > Key: NIFI-329 > URL: https://issues.apache.org/jira/browse/NIFI-329 > Project: Apache NiFi > Issue Type: Sub-task > Components: Examples >Reporter: Joseph Witt >Assignee: Andre F de Miranda >Priority: Minor > Labels: beginner > > - Processor(s) to interact with IRC (sending/receiving) > One approach: A single processor which both sends and receives data from a > given IRC channel. The user can configure the IRC host, username, password, > channel, etc... The connection then is held open and the processor will > produce an output flow file for every message received in the IRC channel > which will have as attributes the message content, sender, time, etc.. That > same processor can also read flow files from its queue which contain message > text in an attribute. In this manner the processor can support bidirectional > interaction with IRC. > Would then also be interesting to make it really easy for a user to generate > a message via the UI as well as easily to consume a message via the UI. > These could be very generic processors/widgets for creation/consumption and > good for these sorts of cases. > There are active IRC channels which are great for demonstration of relatively > active datastreams. Weather updates, Wikipedia updates, etc... -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi pull request #1541: NIFI-329 - Introduce IRC Client Services and Consum...
Github user joewitt commented on a diff in the pull request: https://github.com/apache/nifi/pull/1541#discussion_r118161695 --- Diff: nifi-assembly/NOTICE --- @@ -1364,6 +1364,8 @@ The following binary components are provided under the MIT License. See project (MIT License) EWS Java API (com.microsoft.ews-java-api:ews-java-api:2.0 - https://github.com/OfficeDev/ews-java-api) (MIT License) libffi (libffi-3.2.1 - http://sourceware.org/libffi/) + (MIT License) Kitteh IRC Client Library (kitteh client-lib 3.0.3 - http://kicl.kitteh.org/) + (MIT License) 'mbassador' (mbassador 1.2.4 - https://github.com/bennidi/mbassador) --- End diff -- lines 1365-1368 should be deleted. MIT licensed dependencies get referenced in the LICENSE and do not end up in the NOTICE. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3497) ScanAttribute should support tagging a flowfile with metadata value from the supplied dictionary
[ https://issues.apache.org/jira/browse/NIFI-3497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022314#comment-16022314 ] ASF GitHub Bot commented on NIFI-3497: -- Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/1564 @joetrite my apologies for the big delay. Been hard to get back to this. I of course remember very little of this now which makes it harder to get back into it. have you been using this? > ScanAttribute should support tagging a flowfile with metadata value from the > supplied dictionary > > > Key: NIFI-3497 > URL: https://issues.apache.org/jira/browse/NIFI-3497 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joseph Witt >Assignee: Joseph Witt > > Today ScanAttribute just looks through the supplied dictionary and given > object for a string matching hit. If it hits then it is a match otherwise it > is a 'not found'. However, when a hit occurs it can often be quite useful to > gather additional metadata about that hit. This makes cases like > enrichment/tagging much easier. > So, plan is to have ScanAttribute support a dictionary value demarcator which > would separate the dictionary term from some string response that will be > added to the flowfile. For instance a dictionary might have > apples:These are red or green > bananas:These are yellow unless you should toss them or make bread > Then if a hit occurs on 'apples' the flowfile that contained such an > attribute would have a new attribute such as 'dictionary.hit.term' = 'apple' > and 'dictionary.hit.metadata' = 'These are red or green'. > This means downstream processors could extract that metadata and do > interesting things with it. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi issue #1564: NIFI-3497 ScanAttribute Tagging
Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/1564 @joetrite my apologies for the big delay. Been hard to get back to this. I of course remember very little of this now which makes it harder to get back into it. have you been using this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-385) Add Kerberos support in nifi-kite-nar
[
https://issues.apache.org/jira/browse/NIFI-385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022312#comment-16022312
]
ASF GitHub Bot commented on NIFI-385:
-
Github user joewitt commented on the issue:
https://github.com/apache/nifi/pull/1565
@williamnouet sorry for the delay on this. the kite stuff is tough as it
was contributed by someone who has not been active in a while and I dont think
there are many folks familiar with it. A lot of what it would be used to do we
have very strong alternatives for so that should be considered as well.
Anyone familiar with kite that could validate this work?
> Add Kerberos support in nifi-kite-nar
> -
>
> Key: NIFI-385
> URL: https://issues.apache.org/jira/browse/NIFI-385
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Reporter: Ryan Blue
>
> Kite should be able to connect to a Kerberized Hadoop cluster to store data.
> Kite's Flume connector has working code. The Kite dataset needs to be
> instantiated in a {{doPrivileged}} block and its internal {{FileSystem}}
> object will hold the credentials after that.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[GitHub] nifi issue #1565: NIFI-385 Add Kerberos Support to Kite
Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/1565 @williamnouet sorry for the delay on this. the kite stuff is tough as it was contributed by someone who has not been active in a while and I dont think there are many folks familiar with it. A lot of what it would be used to do we have very strong alternatives for so that should be considered as well. Anyone familiar with kite that could validate this work? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3572) Add a processor to support XProc XML Pipelines
[ https://issues.apache.org/jira/browse/NIFI-3572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022309#comment-16022309 ] ASF GitHub Bot commented on NIFI-3572: -- Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/1575 hello. Would definitely prefer/recommend we put this into its own nar bundle and not along side the standard processors bundle. 'nifi-xmlcalabash-bundle' for example. Are there any templates/examples of use to leverage for testing? > Add a processor to support XProc XML Pipelines > -- > > Key: NIFI-3572 > URL: https://issues.apache.org/jira/browse/NIFI-3572 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Steve Lawrence > > An XProc processor was developed and submitted to the NiFi dev list: > https://mail-archives.apache.org/mod_mbox/nifi-dev/201703.mbox/%3c5d4c2d48-79de-5543-c704-27059c264...@tresys.com%3E > It would be nice if this could be pulled into NiFi. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi issue #1575: NIFI-3572 Add PipelineXml processor to support XProc XML p...
Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/1575 hello. Would definitely prefer/recommend we put this into its own nar bundle and not along side the standard processors bundle. 'nifi-xmlcalabash-bundle' for example. Are there any templates/examples of use to leverage for testing? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3625) Add JSON support to PutHiveStreaming
[ https://issues.apache.org/jira/browse/NIFI-3625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022306#comment-16022306 ] ASF GitHub Bot commented on NIFI-3625: -- Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/1607 @ryanpersaud thanks! I think given the Record reader/writer work this processor is a very good candidate to refactor to leverage that or we should instead create a new one (so we dont break old flows). The new one would be PutHiveStreamingRecord and it would let users select a record reader and the processor then handles going from the record object into the structure Hive needs. This means the processor immediately becomes able to support any format and schema for which we have a reader. > Add JSON support to PutHiveStreaming > > > Key: NIFI-3625 > URL: https://issues.apache.org/jira/browse/NIFI-3625 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.2.0 >Reporter: Ryan Persaud > > As noted in a Hortonworks Community Connection post > (https://community.hortonworks.com/questions/88424/nifi-puthivestreaming-requires-avro.html), > PutHiveStreaming does not currently support JSON Flow File content. I've > completed the code to allow JSON flow files to be streamed into hive, and I'm > currently working on test cases and updated documentation. I should have a > PR to submit this week. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi issue #1607: NIFI-3625 Adding JSON support to PutHiveStreaming processo...
Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/1607 @ryanpersaud thanks! I think given the Record reader/writer work this processor is a very good candidate to refactor to leverage that or we should instead create a new one (so we dont break old flows). The new one would be PutHiveStreamingRecord and it would let users select a record reader and the processor then handles going from the record object into the structure Hive needs. This means the processor immediately becomes able to support any format and schema for which we have a reader. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3648) Address Excessive Garbage Collection
[ https://issues.apache.org/jira/browse/NIFI-3648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022305#comment-16022305 ] ASF GitHub Bot commented on NIFI-3648: -- Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/1637 @markap14 looks like the adjustment was made. Is this good to go? > Address Excessive Garbage Collection > > > Key: NIFI-3648 > URL: https://issues.apache.org/jira/browse/NIFI-3648 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Extensions >Reporter: Mark Payne >Assignee: Mark Payne > > We have a lot of places in the codebase where we generate lots of unnecessary > garbage - especially byte arrays. We need to clean this up to in order to > relieve stress on the garbage collector. > Specific points that I've found create unnecessary garbage: > Provenance CompressableRecordWriter creates a new BufferedOutputStream for > each 'compression block' that it creates. Each one has a 64 KB byte[]. This > is very wasteful. We should instead subclass BufferedOutputStream so that we > are able to provide a byte[] to use instead of an int that indicates the > size. This way, we can just keep re-using the same byte[] that we create for > each writer. This saves about 32,000 of these 64 KB byte[] for each writer > that we create. And we create more than 1 of these per minute. > EvaluateJsonPath uses a BufferedInputStream but it is not necessary, because > the underlying library will also buffer data. So we are unnecessarily > creating a lot of byte[]'s > CompressContent uses Buffered Input AND Output. And uses 64 KB byte[]. And > doesn't need them at all, because it reads and writes with its own byte[] > buffer via StreamUtils.copy > Site-to-site uses CompressionInputStream. This stream creates a new byte[] in > the readChunkHeader() method continually. We should instead only create a new > byte[] if we need a bigger buffer and otherwise just use an offset & length > variable. > Right now, SplitText uses TextLineDemarcator. The fill() method increases the > size of the internal byte[] by 8 KB each time. When dealing with a large > chunk of data, this is VERY expensive on GC because we continually create a > byte[] and then discard it to create a new one. Take for example an 800 KB > chunk. We would do this 100,000 times. If we instead double the size we would > only have to create 8 of these. > Other Processors that use Buffered streams unnecessarily: > ConvertJSONToSQL > ExecuteProcess > ExecuteStreamCommand > AttributesToJSON > EvaluateJsonPath (when writing to content) > ExtractGrok > JmsConsumer -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi issue #1637: NIFI-3648 removed cluster message copying when not in debu...
Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/1637 @markap14 looks like the adjustment was made. Is this good to go? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1644: NIFI-11 - Capture StringIndexOutOfBoundsException to preve...
Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/1644 @trixpan and @mcgilman is this ready/close or does it need another commit? Could not quite tell from the history if there was a new commit and already squashed or not. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-11) UpdateAttribute should penalize & roll back FlowFiles for which Expression Language fails to evaluate
[
https://issues.apache.org/jira/browse/NIFI-11?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022302#comment-16022302
]
ASF GitHub Bot commented on NIFI-11:
Github user joewitt commented on the issue:
https://github.com/apache/nifi/pull/1644
@trixpan and @mcgilman is this ready/close or does it need another commit?
Could not quite tell from the history if there was a new commit and already
squashed or not.
> UpdateAttribute should penalize & roll back FlowFiles for which Expression
> Language fails to evaluate
> -
>
> Key: NIFI-11
> URL: https://issues.apache.org/jira/browse/NIFI-11
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Matt Gilman
>Assignee: Andre F de Miranda
>Priority: Minor
>
> For example, if using ${filename:substring(1, 20)} and the filename is only
> 15 characters, this will throw an Exception, which causes UpdateAttribute to
> yield and not penalize the FlowFile. Instead, it should catch this Exception
> and rollback with penalization and NOT yield, as this is a problem with the
> data, not a problem with the processor's ability to function at this moment.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (NIFI-1763) Provide an integration with 'Schema registry for Kafka'
[ https://issues.apache.org/jira/browse/NIFI-1763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022301#comment-16022301 ] ASF GitHub Bot commented on NIFI-1763: -- Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/1661 @ssiahetiong thanks for contributing. The timing of this is excellent. That said the ConvertXtoY processors had a lot of challenges so with NiFi 1.2.0 we now have record reader and writers which various processors can leverage. This keeps all the format/schema specific logic where it belongs and where it can be readily reused. We already built in integration for the hwx schema registry service, a built-in simple avro registry, and we can certainly add the confluent schema registry as it appears to be the intent here. If interested in aligning this work to that model it would be a much better and integrated experience for the user. > Provide an integration with 'Schema registry for Kafka' > --- > > Key: NIFI-1763 > URL: https://issues.apache.org/jira/browse/NIFI-1763 > Project: Apache NiFi > Issue Type: Wish > Components: Extensions >Reporter: Joseph Witt >Priority: Minor > > Reported on a mailing list question on 13 April 2016 > https://github.com/confluentinc/schema-registry > The registry itself is an ASLv2 licensed codebase. It offers a REST-based > Web API for interaction. It would be good to support integration with it for > users of Kafka so it would register schemas if needed when writing to Kafka > and understand how to parse data based on the indicated schema when reading > from Kafka. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi issue #1661: NIFI-1763 [Provide an integration with 'Schema registry fo...
Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/1661 @ssiahetiong thanks for contributing. The timing of this is excellent. That said the ConvertXtoY processors had a lot of challenges so with NiFi 1.2.0 we now have record reader and writers which various processors can leverage. This keeps all the format/schema specific logic where it belongs and where it can be readily reused. We already built in integration for the hwx schema registry service, a built-in simple avro registry, and we can certainly add the confluent schema registry as it appears to be the intent here. If interested in aligning this work to that model it would be a much better and integrated experience for the user. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (NIFI-3878) Update AmbariReportingTask to set 1 minute as default run schedule
[ https://issues.apache.org/jira/browse/NIFI-3878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-3878: -- Resolution: Fixed Status: Resolved (was: Patch Available) +1 merged to master > Update AmbariReportingTask to set 1 minute as default run schedule > --- > > Key: NIFI-3878 > URL: https://issues.apache.org/jira/browse/NIFI-3878 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Trivial > Fix For: 1.3.0 > > > The AmbariReportingTask should be configured to run every one minute to match > the expectations of the Ambari Metrics System. Instead of the default 5 > minutes, change the reporting task to default to 1 minute. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3878) Update AmbariReportingTask to set 1 minute as default run schedule
[ https://issues.apache.org/jira/browse/NIFI-3878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022284#comment-16022284 ] ASF subversion and git services commented on NIFI-3878: --- Commit d63fa8a822a755e12a30c05a00c73786ade1c657 in nifi's branch refs/heads/master from [~pvillard] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=d63fa8a ] NIFI-3878 This closes #1790. AmbariReportingTask - Default run schedule set to 1 minute Signed-off-by: joewitt > Update AmbariReportingTask to set 1 minute as default run schedule > --- > > Key: NIFI-3878 > URL: https://issues.apache.org/jira/browse/NIFI-3878 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Trivial > Fix For: 1.3.0 > > > The AmbariReportingTask should be configured to run every one minute to match > the expectations of the Ambari Metrics System. Instead of the default 5 > minutes, change the reporting task to default to 1 minute. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3878) Update AmbariReportingTask to set 1 minute as default run schedule
[ https://issues.apache.org/jira/browse/NIFI-3878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-3878: -- Fix Version/s: 1.3.0 > Update AmbariReportingTask to set 1 minute as default run schedule > --- > > Key: NIFI-3878 > URL: https://issues.apache.org/jira/browse/NIFI-3878 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Trivial > Fix For: 1.3.0 > > > The AmbariReportingTask should be configured to run every one minute to match > the expectations of the Ambari Metrics System. Instead of the default 5 > minutes, change the reporting task to default to 1 minute. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3644) Add DetectDuplicateUsingHBase processor
[ https://issues.apache.org/jira/browse/NIFI-3644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022295#comment-16022295 ] ASF GitHub Bot commented on NIFI-3644: -- Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/1645 This looks like it could be pretty helpful! I wonder if in light of the recent LookupService work we should consider exposing/using this via that interface instead of or in addition to this distributed cache one. Thoughts? > Add DetectDuplicateUsingHBase processor > --- > > Key: NIFI-3644 > URL: https://issues.apache.org/jira/browse/NIFI-3644 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Bjorn Olsen >Priority: Minor > > The DetectDuplicate processor makes use of a distributed map cache for > maintaining a list of unique file identifiers (such as hashes). > The distributed map cache functionality could be provided by an HBase table, > which then allows for reliably storing a huge volume of file identifiers and > auditing information. The downside of this approach is of course that HBase > is required. > Storing the unique file identifiers in a reliable, query-able manner along > with some audit information is of benefit to several use cases. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi issue #1645: NIFI-3644 - Added HBase_1_1_2_ClientMapCacheService
Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/1645 This looks like it could be pretty helpful! I wonder if in light of the recent LookupService work we should consider exposing/using this via that interface instead of or in addition to this distributed cache one. Thoughts? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-725) Refactor when documentation is generated
[ https://issues.apache.org/jira/browse/NIFI-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022293#comment-16022293 ] ASF GitHub Bot commented on NIFI-725: - Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/1744 @kira1508 thanks for contributing. Can you please squash the contribs and rebase and update the commit message to say something like "NIFI-725 providing a maven plugin to alter when documentation is built". That said, can you describe a bit more about the approach and idea and how you feel it helps moves things forward. There was clearly a lot of work done here so just really want to make sure it can be well understood and discussed as needed. > Refactor when documentation is generated > > > Key: NIFI-725 > URL: https://issues.apache.org/jira/browse/NIFI-725 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Documentation & Website >Affects Versions: 0.1.0 >Reporter: Dan Bress >Assignee: Dan Bress >Priority: Minor > > Currently the DocGenerator, which generates all of our > Processor/ReportingTask/ControllerService documentation, is invoked directly > from the > [NiFi|https://github.com/apache/incubator-nifi/blob/develop/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-runtime/src/main/java/org/apache/nifi/NiFi.java#L123] > class in the runtime project. > Invoking the DocGenerator directly from the runtime hamstrings what libraries > the DocGenerator can use, because the runtime project should have the least > number of dependencies possible, and it currently depends on the > documentation library. > I'm proposing that we find a new way/place to invoke documentation generation > such that it may bring in whatever dependencies it needs. > The first dependency it needs is on nifi-framework-core > [ReflectionUtils|https://github.com/apache/incubator-nifi/blob/develop/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/util/ReflectionUtils.java], > which would let us [this > copy|https://github.com/apache/incubator-nifi/blob/NIFI-632/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-documentation/src/main/java/org/apache/nifi/documentation/util/ReflectionUtils.java] > of that class. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (NIFI-3958) Avro decimal logical type fail if database returns Zero precision
[
https://issues.apache.org/jira/browse/NIFI-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Koji Kawamura reassigned NIFI-3958:
---
Assignee: Koji Kawamura
> Avro decimal logical type fail if database returns Zero precision
> -
>
> Key: NIFI-3958
> URL: https://issues.apache.org/jira/browse/NIFI-3958
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.3.0
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
>
> NIFI-2624 introduced Avro logical type mapping capability. For
> NUMERIC/DECIMAL types, 'decimal' logical type is used, and positive precision
> is required to encode a numeric value as 'decimal logical type.
> However, database can return 0 (Zero) precision to represent unlimited
> precision, if a column is not configured with specific precision (some
> database engine has default precision, but some don't), or numeric value
> generated by functions or other means dynamically.
> We need to provide a way to define default precision in case Zero precision
> is returned, probably by adding another Processor property to specify default
> precision.
> {code}
> 2017-05-23 16:54:12,414 WARN [Timer-Driven Process Thread-5]
> o.a.n.c.t.ContinuallyRunProcessorTask
> java.lang.IllegalArgumentException: Invalid decimal precision: 0 (must be
> positive)
> at
> org.apache.avro.LogicalTypes$Decimal.validate(LogicalTypes.java:206)
> at org.apache.avro.LogicalType.addToSchema(LogicalType.java:70)
> at
> org.apache.avro.LogicalTypes$Decimal.addToSchema(LogicalTypes.java:182)
> at
> org.apache.nifi.processors.standard.util.JdbcCommon.lambda$createSchema$0(JdbcCommon.java:463)
> at
> org.apache.nifi.processors.standard.util.JdbcCommon.addNullableField(JdbcCommon.java:359)
> at
> org.apache.nifi.processors.standard.util.JdbcCommon.createSchema(JdbcCommon.java:461)
> at
> org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:192)
> at
> org.apache.nifi.processors.standard.ExecuteSQL$2.process(ExecuteSQL.java:204)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2529)
> at
> org.apache.nifi.processors.standard.ExecuteSQL.onTrigger(ExecuteSQL.java:195)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1120)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:144)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[GitHub] nifi issue #1744: Nifi 725 master
Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/1744 @kira1508 thanks for contributing. Can you please squash the contribs and rebase and update the commit message to say something like "NIFI-725 providing a maven plugin to alter when documentation is built". That said, can you describe a bit more about the approach and idea and how you feel it helps moves things forward. There was clearly a lot of work done here so just really want to make sure it can be well understood and discussed as needed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3878) Update AmbariReportingTask to set 1 minute as default run schedule
[ https://issues.apache.org/jira/browse/NIFI-3878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022286#comment-16022286 ] ASF GitHub Bot commented on NIFI-3878: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1790 > Update AmbariReportingTask to set 1 minute as default run schedule > --- > > Key: NIFI-3878 > URL: https://issues.apache.org/jira/browse/NIFI-3878 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Trivial > Fix For: 1.3.0 > > > The AmbariReportingTask should be configured to run every one minute to match > the expectations of the Ambari Metrics System. Instead of the default 5 > minutes, change the reporting task to default to 1 minute. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3954) javax.net.SSL.PeerUnverifiedException Hostname www.googleapis.com does the match the certificate subject provided by the peer
[
https://issues.apache.org/jira/browse/NIFI-3954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joseph Witt updated NIFI-3954:
--
Fix Version/s: 1.3.0
> javax.net.SSL.PeerUnverifiedException Hostname www.googleapis.com does the
> match the certificate subject provided by the peer
> -
>
> Key: NIFI-3954
> URL: https://issues.apache.org/jira/browse/NIFI-3954
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.2.0
>Reporter: Dmitry Lukyanov
>Assignee: Andy LoPresto
> Fix For: 1.3.0
>
>
> The description taken here:
> https://stackoverflow.com/questions/44112959/apache-nifi-with-youtube-data-api
> Use {{GetHTTP}} processor with url
> {{https://www.googleapis.com/youtube/v3/commentThreads?key=}}
> Expected response: should receive http error code 400 with json output, but
> got exception of host name verification
> {code}Hostname www.googleapis.com does the match the certificate subject
> provided by the peer *.googleapis.com{code}
> And reproduced with java 1.8 and nifi 1.2.0
> Also I found this issue in http client 4.4.x
> https://issues.apache.org/jira/browse/HTTPCLIENT-1613
> And nifi 1.2.0 uses httpclient-4.4.1.jar
> Seems just need to change version of httpclient to 4.5
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[GitHub] nifi pull request #1790: NIFI-3878 AmbariReportingTask - Default run schedul...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1790 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (NIFI-3954) javax.net.SSL.PeerUnverifiedException Hostname www.googleapis.com does the match the certificate subject provided by the peer
[
https://issues.apache.org/jira/browse/NIFI-3954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joseph Witt updated NIFI-3954:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
+1. full clean build and andy's test results look solid. Merged to master
> javax.net.SSL.PeerUnverifiedException Hostname www.googleapis.com does the
> match the certificate subject provided by the peer
> -
>
> Key: NIFI-3954
> URL: https://issues.apache.org/jira/browse/NIFI-3954
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.2.0
>Reporter: Dmitry Lukyanov
>Assignee: Andy LoPresto
>
> The description taken here:
> https://stackoverflow.com/questions/44112959/apache-nifi-with-youtube-data-api
> Use {{GetHTTP}} processor with url
> {{https://www.googleapis.com/youtube/v3/commentThreads?key=}}
> Expected response: should receive http error code 400 with json output, but
> got exception of host name verification
> {code}Hostname www.googleapis.com does the match the certificate subject
> provided by the peer *.googleapis.com{code}
> And reproduced with java 1.8 and nifi 1.2.0
> Also I found this issue in http client 4.4.x
> https://issues.apache.org/jira/browse/HTTPCLIENT-1613
> And nifi 1.2.0 uses httpclient-4.4.1.jar
> Seems just need to change version of httpclient to 4.5
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (NIFI-3954) javax.net.SSL.PeerUnverifiedException Hostname www.googleapis.com does the match the certificate subject provided by the peer
[
https://issues.apache.org/jira/browse/NIFI-3954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022282#comment-16022282
]
ASF GitHub Bot commented on NIFI-3954:
--
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/1841
> javax.net.SSL.PeerUnverifiedException Hostname www.googleapis.com does the
> match the certificate subject provided by the peer
> -
>
> Key: NIFI-3954
> URL: https://issues.apache.org/jira/browse/NIFI-3954
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.2.0
>Reporter: Dmitry Lukyanov
>Assignee: Andy LoPresto
>
> The description taken here:
> https://stackoverflow.com/questions/44112959/apache-nifi-with-youtube-data-api
> Use {{GetHTTP}} processor with url
> {{https://www.googleapis.com/youtube/v3/commentThreads?key=}}
> Expected response: should receive http error code 400 with json output, but
> got exception of host name verification
> {code}Hostname www.googleapis.com does the match the certificate subject
> provided by the peer *.googleapis.com{code}
> And reproduced with java 1.8 and nifi 1.2.0
> Also I found this issue in http client 4.4.x
> https://issues.apache.org/jira/browse/HTTPCLIENT-1613
> And nifi 1.2.0 uses httpclient-4.4.1.jar
> Seems just need to change version of httpclient to 4.5
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[GitHub] nifi pull request #1841: NIFI-3954 Upgraded version of httpclient library to...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1841 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-1490) Add multipart request support to ListenHTTP Processor
[ https://issues.apache.org/jira/browse/NIFI-1490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022280#comment-16022280 ] ASF GitHub Bot commented on NIFI-1490: -- Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/1795 hello @pipern. Thanks for contributing. If i'm reading this right it looks like what this will do is grab the first matching section of a multi-part request. The current behavior before the patch would be to, i believe, grab all of the request as the flow file content. Given that, it seems like rather than altering this processor to grab just the first match we'd either want to alter it to grab each section as new flow files OR perhaps better yet have a different processor which understands how to extract out the specific parts into individual flowfiles. ExtractMultiPartHttp for example could take as input the entire request body in a flowfile that came from ListenHTTP and this new processor could create child objects which are each of the parts. What do you think about that? > Add multipart request support to ListenHTTP Processor > - > > Key: NIFI-1490 > URL: https://issues.apache.org/jira/browse/NIFI-1490 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Andrew Serff > > The current ListenHTTP processor does not seem to support multipart requests > that are encoded with multipart/form-data. When a multipart request is > received, the ListenHTTPServlet just copies the Request InputStream to the > FlowFiles content which leaves the form encoding wrapper in the content and > in turn makes the file invalid. > Specifically, we want to be able to support file uploads in a multipart > request. > See this thread in the mailing list for more info: > http://mail-archives.apache.org/mod_mbox/nifi-users/201602.mbox/%3C6DE9CEEF-2A37-480F-8D3C-5028C590FD9E%40acesinc.net%3E -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3954) javax.net.SSL.PeerUnverifiedException Hostname www.googleapis.com does the match the certificate subject provided by the peer
[
https://issues.apache.org/jira/browse/NIFI-3954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022281#comment-16022281
]
ASF subversion and git services commented on NIFI-3954:
---
Commit 6d07ebf50c397458d639b3713525f6bbab4c7901 in nifi's branch
refs/heads/master from [~alopresto]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=6d07ebf ]
NIFI-3954 This closes #1841. Upgraded version of httpclient library to handle
TLS issues.
Added explicit dependency on httpcore for @ThreadSafe annotation.
Signed-off-by: joewitt
> javax.net.SSL.PeerUnverifiedException Hostname www.googleapis.com does the
> match the certificate subject provided by the peer
> -
>
> Key: NIFI-3954
> URL: https://issues.apache.org/jira/browse/NIFI-3954
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.2.0
>Reporter: Dmitry Lukyanov
>Assignee: Andy LoPresto
>
> The description taken here:
> https://stackoverflow.com/questions/44112959/apache-nifi-with-youtube-data-api
> Use {{GetHTTP}} processor with url
> {{https://www.googleapis.com/youtube/v3/commentThreads?key=}}
> Expected response: should receive http error code 400 with json output, but
> got exception of host name verification
> {code}Hostname www.googleapis.com does the match the certificate subject
> provided by the peer *.googleapis.com{code}
> And reproduced with java 1.8 and nifi 1.2.0
> Also I found this issue in http client 4.4.x
> https://issues.apache.org/jira/browse/HTTPCLIENT-1613
> And nifi 1.2.0 uses httpclient-4.4.1.jar
> Seems just need to change version of httpclient to 4.5
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[GitHub] nifi issue #1795: NIFI-1490 Add multipart request support to ListenHTTP Proc...
Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/1795 hello @pipern. Thanks for contributing. If i'm reading this right it looks like what this will do is grab the first matching section of a multi-part request. The current behavior before the patch would be to, i believe, grab all of the request as the flow file content. Given that, it seems like rather than altering this processor to grab just the first match we'd either want to alter it to grab each section as new flow files OR perhaps better yet have a different processor which understands how to extract out the specific parts into individual flowfiles. ExtractMultiPartHttp for example could take as input the entire request body in a flowfile that came from ListenHTTP and this new processor could create child objects which are each of the parts. What do you think about that? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3681) Controller Service for Processor Leader Elections
[ https://issues.apache.org/jira/browse/NIFI-3681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022277#comment-16022277 ] ASF GitHub Bot commented on NIFI-3681: -- Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/1800 couple of quick thoughts. - we'll need to get all the version numbers aligned with whatever nifi version this would be committed into. Currently that would be 1.3.0-SNAPSHOT. - It would probably be a good idea to have the notion of 'nifi-leaderelection-api' which is not about zookeeper but rather just generic election/tracking of a leader for a given thing (a partition?) Then there would be zookeeper based implementations of those. Processors then can leverage the api for their code but users can select whichever types of services exist (zookeeper being the obvious initial example). The structure appears already in place for this other than the current naming and perhaps the API referencing zookeeper. Thoughts? - It would be good to have a processor which leverages this or some docs that go along with it to show suggested usage. > Controller Service for Processor Leader Elections > -- > > Key: NIFI-3681 > URL: https://issues.apache.org/jira/browse/NIFI-3681 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joseph Niemiec >Assignee: Joseph Niemiec > > Today I find a need for NiFi Cluster to allow a specific set of processors to > perform a 'LeaderElection' of sorts to allow for a single processor to > update the process shared cluster state with assignments (both initial and > recovery.) The CS would be responsible for joining a Zookeeper group, the > logic itself, performing new elections should an leader die, etc... > At its core I envision a simple API provided by the CS > * String - whoIsLeader > * List[String] - aliveElectors > * Long- LastElectionEpoch ? - Not sure about this, but would it be good to > detect if an election occurred and the leader did not change but the election > occured. Maybe a UUID-4? > While thinking ZK is best as Clustered NiFi already requires it would there > be value in implementing a standalone RAFT or PAXOS that runs at the cluster > level? -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi issue #1800: NIFI-3681 - Controller Service for Processor Leader Electi...
Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/1800 couple of quick thoughts. - we'll need to get all the version numbers aligned with whatever nifi version this would be committed into. Currently that would be 1.3.0-SNAPSHOT. - It would probably be a good idea to have the notion of 'nifi-leaderelection-api' which is not about zookeeper but rather just generic election/tracking of a leader for a given thing (a partition?) Then there would be zookeeper based implementations of those. Processors then can leverage the api for their code but users can select whichever types of services exist (zookeeper being the obvious initial example). The structure appears already in place for this other than the current naming and perhaps the API referencing zookeeper. Thoughts? - It would be good to have a processor which leverages this or some docs that go along with it to show suggested usage. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1832: NIFI-1452 on timer-driven yield, use the greater of yield ...
Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/1832 the change looks solid to me and this is a really good catch for framework behavior. @markap14 would definitely appreciate you getting eyes on here given where it lives. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-1452) Yield Duration can short circuit long Timer Driven Run Schedule
[ https://issues.apache.org/jira/browse/NIFI-1452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022272#comment-16022272 ] ASF GitHub Bot commented on NIFI-1452: -- Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/1832 the change looks solid to me and this is a really good catch for framework behavior. @markap14 would definitely appreciate you getting eyes on here given where it lives. > Yield Duration can short circuit long Timer Driven Run Schedule > --- > > Key: NIFI-1452 > URL: https://issues.apache.org/jira/browse/NIFI-1452 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.4.1 >Reporter: Michael Moser >Assignee: Michael Moser >Priority: Trivial > > This may be a rare use case, but I configured a GetFile processor to be Timer > Driven with a Run Schedule of 30 secs. Its Yield Duration was default 1 sec. > I expected GetFile onTrigger() to be called every 30 secs, but it was being > called every 1 sec most of the time. > GetFile will call context.yield() when it polls a directory and gets an empty > list in return. It appears that a yield will ignore the Run Schedule. Many > standard processors call context.yield() when they have no work to do. > I changed my scheduling strategy to CRON Driven with its run schedule every > 30 seconds, and the onTrigger() was called every 30 seconds, even after a > yield. So CRON Driven scheduling is working as expected after a yield. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3962) Update ConsumeKafkaRecord_0_10 to better batch together messages into fewer flowfiles
[ https://issues.apache.org/jira/browse/NIFI-3962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-3962: -- Resolution: Fixed Status: Resolved (was: Patch Available) tested orig consumer and record consumer. Changes look good. All checks out. +1 merged to master > Update ConsumeKafkaRecord_0_10 to better batch together messages into fewer > flowfiles > - > > Key: NIFI-3962 > URL: https://issues.apache.org/jira/browse/NIFI-3962 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne > > Currently, ConsumeKafkaRecord will batch together only as many records in a > FlowFile as it is able to obtain for a single partition & topic. This should > be updated to allow for multiple iterations of KafkaConsumer.poll() in the > same way that ConsumeKafka does with a Demarcator configured. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3962) Update ConsumeKafkaRecord_0_10 to better batch together messages into fewer flowfiles
[ https://issues.apache.org/jira/browse/NIFI-3962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-3962: -- Fix Version/s: 1.3.0 > Update ConsumeKafkaRecord_0_10 to better batch together messages into fewer > flowfiles > - > > Key: NIFI-3962 > URL: https://issues.apache.org/jira/browse/NIFI-3962 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne > Fix For: 1.3.0 > > > Currently, ConsumeKafkaRecord will batch together only as many records in a > FlowFile as it is able to obtain for a single partition & topic. This should > be updated to allow for multiple iterations of KafkaConsumer.poll() in the > same way that ConsumeKafka does with a Demarcator configured. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3962) Update ConsumeKafkaRecord_0_10 to better batch together messages into fewer flowfiles
[ https://issues.apache.org/jira/browse/NIFI-3962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022258#comment-16022258 ] ASF subversion and git services commented on NIFI-3962: --- Commit 2c751a8e5bdf784513fabf1da029bb12cd076732 in nifi's branch refs/heads/master from [~markap14] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=2c751a8 ] NIFI-3962: This closes #1846. Updated ConsumerLease to better handle batching of messages into a single FlowFile in the same way that it is handled for demarcated data > Update ConsumeKafkaRecord_0_10 to better batch together messages into fewer > flowfiles > - > > Key: NIFI-3962 > URL: https://issues.apache.org/jira/browse/NIFI-3962 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne > > Currently, ConsumeKafkaRecord will batch together only as many records in a > FlowFile as it is able to obtain for a single partition & topic. This should > be updated to allow for multiple iterations of KafkaConsumer.poll() in the > same way that ConsumeKafka does with a Demarcator configured. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3962) Update ConsumeKafkaRecord_0_10 to better batch together messages into fewer flowfiles
[ https://issues.apache.org/jira/browse/NIFI-3962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022259#comment-16022259 ] ASF GitHub Bot commented on NIFI-3962: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1846 > Update ConsumeKafkaRecord_0_10 to better batch together messages into fewer > flowfiles > - > > Key: NIFI-3962 > URL: https://issues.apache.org/jira/browse/NIFI-3962 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne > > Currently, ConsumeKafkaRecord will batch together only as many records in a > FlowFile as it is able to obtain for a single partition & topic. This should > be updated to allow for multiple iterations of KafkaConsumer.poll() in the > same way that ConsumeKafka does with a Demarcator configured. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi pull request #1846: NIFI-3962: Updated ConsumerLease to better handle b...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1846 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3799) Document and / or visually show if a procesor accepts "input" connection
[ https://issues.apache.org/jira/browse/NIFI-3799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022184#comment-16022184 ] ASF GitHub Bot commented on NIFI-3799: -- Github user jvwing commented on the issue: https://github.com/apache/nifi/pull/1756 @pvillard31 - Everything works well in my testing. Why add the notation to the bottom of the Description section, rather than creating a new section with heading? For example, the Restricted section seems comparable, but it has a heading and then the text describing the restriction under the heading. > Document and / or visually show if a procesor accepts "input" connection > > > Key: NIFI-3799 > URL: https://issues.apache.org/jira/browse/NIFI-3799 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Juan C. Sequeiros >Assignee: Pierre Villard >Priority: Minor > > Today the only way of knowing if a processor accepts input connection is > either by trying it on the UI or looking at the processor code. > @InputRequirement(Requirement.INPUT_FORBIDDEN) > or > @InputRequirement(Requirement.INPUT_REQUIRED) > It would help a DFM if there is at least some documentation on this either > under usage link or a tag to have alternative ways of knowing > proactively. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi issue #1756: NIFI-3799 - added input requirement info for processor doc...
Github user jvwing commented on the issue: https://github.com/apache/nifi/pull/1756 @pvillard31 - Everything works well in my testing. Why add the notation to the bottom of the Description section, rather than creating a new section with heading? For example, the Restricted section seems comparable, but it has a heading and then the text describing the restriction under the heading. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3943) Help/hover text is misaligned in the Scope drop-down in the Enable Controller Service dialog
[ https://issues.apache.org/jira/browse/NIFI-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022028#comment-16022028 ] ASF GitHub Bot commented on NIFI-3943: -- GitHub user scottyaslan opened a pull request: https://github.com/apache/nifi/pull/1849 [NIFI-3943] align combo option item toolips to hover closer to the icon Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/scottyaslan/nifi NIFI-3943 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1849.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1849 commit b0c5e287d70aa4696f797724a0161f9af8b15c3c Author: Scott Aslan Date: 2017-05-23T22:54:42Z [NIFI-3943] align combo option item toolips to hover closer to the icon > Help/hover text is misaligned in the Scope drop-down in the Enable Controller > Service dialog > > > Key: NIFI-3943 > URL: https://issues.apache.org/jira/browse/NIFI-3943 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.2.0 > Environment: Chrome Version 58.0.3029.110 (64-bit) >Reporter: Andrew Lim >Assignee: Scott Aslan >Priority: Trivial > Attachments: NIFI-3943.png > > > When hovering over the "?" icons in the Scope drop-down, the help text > appears higher than it should. It is especially confusing for the help text > for "Service and referencing components", as the text appears closer to the > "Service Only" option. > Screenshot attached. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3943) Help/hover text is misaligned in the Scope drop-down in the Enable Controller Service dialog
[ https://issues.apache.org/jira/browse/NIFI-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Aslan updated NIFI-3943: -- Status: Patch Available (was: In Progress) > Help/hover text is misaligned in the Scope drop-down in the Enable Controller > Service dialog > > > Key: NIFI-3943 > URL: https://issues.apache.org/jira/browse/NIFI-3943 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.2.0 > Environment: Chrome Version 58.0.3029.110 (64-bit) >Reporter: Andrew Lim >Assignee: Scott Aslan >Priority: Trivial > Attachments: NIFI-3943.png > > > When hovering over the "?" icons in the Scope drop-down, the help text > appears higher than it should. It is especially confusing for the help text > for "Service and referencing components", as the text appears closer to the > "Service Only" option. > Screenshot attached. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi pull request #1849: [NIFI-3943] align combo option item toolips to hove...
GitHub user scottyaslan opened a pull request: https://github.com/apache/nifi/pull/1849 [NIFI-3943] align combo option item toolips to hover closer to the icon Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/scottyaslan/nifi NIFI-3943 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1849.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1849 commit b0c5e287d70aa4696f797724a0161f9af8b15c3c Author: Scott Aslan Date: 2017-05-23T22:54:42Z [NIFI-3943] align combo option item toolips to hover closer to the icon --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Assigned] (NIFI-3943) Help/hover text is misaligned in the Scope drop-down in the Enable Controller Service dialog
[ https://issues.apache.org/jira/browse/NIFI-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Aslan reassigned NIFI-3943: - Assignee: Scott Aslan > Help/hover text is misaligned in the Scope drop-down in the Enable Controller > Service dialog > > > Key: NIFI-3943 > URL: https://issues.apache.org/jira/browse/NIFI-3943 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.2.0 > Environment: Chrome Version 58.0.3029.110 (64-bit) >Reporter: Andrew Lim >Assignee: Scott Aslan >Priority: Trivial > Attachments: NIFI-3943.png > > > When hovering over the "?" icons in the Scope drop-down, the help text > appears higher than it should. It is especially confusing for the help text > for "Service and referencing components", as the text appears closer to the > "Service Only" option. > Screenshot attached. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3925) Add Template list with long template name
[ https://issues.apache.org/jira/browse/NIFI-3925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Aslan updated NIFI-3925: -- Status: Patch Available (was: In Progress) > Add Template list with long template name > - > > Key: NIFI-3925 > URL: https://issues.apache.org/jira/browse/NIFI-3925 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.2.0 >Reporter: Mark Bean >Assignee: Scott Aslan >Priority: Minor > > If a template has a very long name (roughly 40 characters or more), the > dropdown list of templates in "Add Template" does not display properly. The > "?" rollover which displays the template description is shown on the next > line. In effect, the long-named template line has no "?" rollover, and the > next template line has two adjacent "?" rollovers (one corresponding to the > previous, long-named template and one corresponding to itself.) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3925) Add Template list with long template name
[ https://issues.apache.org/jira/browse/NIFI-3925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021943#comment-16021943 ] ASF GitHub Bot commented on NIFI-3925: -- GitHub user scottyaslan opened a pull request: https://github.com/apache/nifi/pull/1848 [NIFI-3925] bound width of combo options drop down text Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/scottyaslan/nifi NIFI-3925 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1848.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1848 commit dd73254c0b8280670e70ff98d1646f56c47f739e Author: Scott Aslan Date: 2017-05-23T21:54:40Z [NIFI-3925] bound width of combo options drop down text > Add Template list with long template name > - > > Key: NIFI-3925 > URL: https://issues.apache.org/jira/browse/NIFI-3925 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.2.0 >Reporter: Mark Bean >Assignee: Scott Aslan >Priority: Minor > > If a template has a very long name (roughly 40 characters or more), the > dropdown list of templates in "Add Template" does not display properly. The > "?" rollover which displays the template description is shown on the next > line. In effect, the long-named template line has no "?" rollover, and the > next template line has two adjacent "?" rollovers (one corresponding to the > previous, long-named template and one corresponding to itself.) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi pull request #1848: [NIFI-3925] bound width of combo options drop down ...
GitHub user scottyaslan opened a pull request: https://github.com/apache/nifi/pull/1848 [NIFI-3925] bound width of combo options drop down text Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/scottyaslan/nifi NIFI-3925 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1848.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1848 commit dd73254c0b8280670e70ff98d1646f56c47f739e Author: Scott Aslan Date: 2017-05-23T21:54:40Z [NIFI-3925] bound width of combo options drop down text --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Assigned] (NIFI-3925) Add Template list with long template name
[ https://issues.apache.org/jira/browse/NIFI-3925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Aslan reassigned NIFI-3925: - Assignee: Scott Aslan > Add Template list with long template name > - > > Key: NIFI-3925 > URL: https://issues.apache.org/jira/browse/NIFI-3925 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.2.0 >Reporter: Mark Bean >Assignee: Scott Aslan >Priority: Minor > > If a template has a very long name (roughly 40 characters or more), the > dropdown list of templates in "Add Template" does not display properly. The > "?" rollover which displays the template description is shown on the next > line. In effect, the long-named template line has no "?" rollover, and the > next template line has two adjacent "?" rollovers (one corresponding to the > previous, long-named template and one corresponding to itself.) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3963) Remote Process Group does not honor yield duration if transmission is disabled then reenabled
[ https://issues.apache.org/jira/browse/NIFI-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021909#comment-16021909 ] Joseph Percivall commented on NIFI-3963: Also I verified in the flow.xml that the yield period is "10 sec" > Remote Process Group does not honor yield duration if transmission is > disabled then reenabled > - > > Key: NIFI-3963 > URL: https://issues.apache.org/jira/browse/NIFI-3963 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Joseph Percivall > Attachments: Screen Shot 2017-05-23 at 5.07.45 PM.png, > unable-to-reproduce-nifi-3963.png > > > To reproduce, create two NiFi instances properly set up for unsecure or > secure S2S. One on side add an input/output port and the other an RPG. > Enable transmission on the RPG. Stop the instance with the port. On the > running instance see NiFi honoring the yield duration. Then stop and start > transmission, see NiFi repeatedly attempt to connect to the instance (I saw > multiple log messages per millisecond). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3963) Remote Process Group does not honor yield duration if transmission is disabled then reenabled
[ https://issues.apache.org/jira/browse/NIFI-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Percivall updated NIFI-3963: --- Attachment: Screen Shot 2017-05-23 at 5.07.45 PM.png > Remote Process Group does not honor yield duration if transmission is > disabled then reenabled > - > > Key: NIFI-3963 > URL: https://issues.apache.org/jira/browse/NIFI-3963 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Joseph Percivall > Attachments: Screen Shot 2017-05-23 at 5.07.45 PM.png, > unable-to-reproduce-nifi-3963.png > > > To reproduce, create two NiFi instances properly set up for unsecure or > secure S2S. One on side add an input/output port and the other an RPG. > Enable transmission on the RPG. Stop the instance with the port. On the > running instance see NiFi honoring the yield duration. Then stop and start > transmission, see NiFi repeatedly attempt to connect to the instance (I saw > multiple log messages per millisecond). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3963) Remote Process Group does not honor yield duration if transmission is disabled then reenabled
[ https://issues.apache.org/jira/browse/NIFI-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021897#comment-16021897 ] Joseph Percivall commented on NIFI-3963: Hey [~mcgilman], I believe I've narrowed it down. I can reliably replicate when the RPG is pulling from an output port. So the steps are: Initial state: 2 NiFi instances. One instance with an output port (seen with a 1.1.1 or 1.2.0 instance here). The second being a 1.2.0 instance with an RPG set up to pull from that output port. 1: Enable transmission on the RPG to start pulling from the output port 2: Stop the output port instance (ie. "nifi.sh stop") 3: Disable transmission on the RPG 4: Enable transmission on the RPG 5: Wait a couple seconds 6: See a flood of failed requests in the logs. I will be attached an screenshot of the logs where I disable then reenable it. I have not been able to reproduce when it is an input port instead of an output port. > Remote Process Group does not honor yield duration if transmission is > disabled then reenabled > - > > Key: NIFI-3963 > URL: https://issues.apache.org/jira/browse/NIFI-3963 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Joseph Percivall > Attachments: unable-to-reproduce-nifi-3963.png > > > To reproduce, create two NiFi instances properly set up for unsecure or > secure S2S. One on side add an input/output port and the other an RPG. > Enable transmission on the RPG. Stop the instance with the port. On the > running instance see NiFi honoring the yield duration. Then stop and start > transmission, see NiFi repeatedly attempt to connect to the instance (I saw > multiple log messages per millisecond). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (NIFI-3966) Update URL depending on location in the documentation
Andrew Lim created NIFI-3966: Summary: Update URL depending on location in the documentation Key: NIFI-3966 URL: https://issues.apache.org/jira/browse/NIFI-3966 Project: Apache NiFi Issue Type: Improvement Components: Documentation & Website Reporter: Andrew Lim Priority: Trivial I've seen this in other websites and thought it might be useful for our documentation. When scrolling through a document, say the Admin Guide, the URL could change to include the anchor for that section in the doc. So if the user wanted to share a link to that specific place in the doc, the URL would be ready to copy and paste. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (NIFI-3965) Hexdump nifi processor
John Teabo created NIFI-3965: Summary: Hexdump nifi processor Key: NIFI-3965 URL: https://issues.apache.org/jira/browse/NIFI-3965 Project: Apache NiFi Issue Type: New Feature Components: Tools and Build Affects Versions: 1.2.0 Reporter: John Teabo It would be great to have a nifi processor that would do a hexdump of a file. An example of a groovy script already doing this in nifi can be seen in this hortonworks question: https://community.hortonworks.com/questions/60597/hexdump-nifi-processor-nifi-hexdump-processor.html -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3963) Remote Process Group does not honor yield duration if transmission is disabled then reenabled
[ https://issues.apache.org/jira/browse/NIFI-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021795#comment-16021795 ] Matt Gilman commented on NIFI-3963: --- I've also tried terminating the target instance but the yield duration seems to be honored. I do see some logging about trying to refresh the peer list in the background. However, the RemoteGroupPort does not appear to trigger again until the yield duration completes. > Remote Process Group does not honor yield duration if transmission is > disabled then reenabled > - > > Key: NIFI-3963 > URL: https://issues.apache.org/jira/browse/NIFI-3963 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Joseph Percivall > Attachments: unable-to-reproduce-nifi-3963.png > > > To reproduce, create two NiFi instances properly set up for unsecure or > secure S2S. One on side add an input/output port and the other an RPG. > Enable transmission on the RPG. Stop the instance with the port. On the > running instance see NiFi honoring the yield duration. Then stop and start > transmission, see NiFi repeatedly attempt to connect to the instance (I saw > multiple log messages per millisecond). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3962) Update ConsumeKafkaRecord_0_10 to better batch together messages into fewer flowfiles
[ https://issues.apache.org/jira/browse/NIFI-3962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021764#comment-16021764 ] Joseph Witt commented on NIFI-3962: --- reviewing > Update ConsumeKafkaRecord_0_10 to better batch together messages into fewer > flowfiles > - > > Key: NIFI-3962 > URL: https://issues.apache.org/jira/browse/NIFI-3962 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne > > Currently, ConsumeKafkaRecord will batch together only as many records in a > FlowFile as it is able to obtain for a single partition & topic. This should > be updated to allow for multiple iterations of KafkaConsumer.poll() in the > same way that ConsumeKafka does with a Demarcator configured. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (NIFI-3963) Remote Process Group does not honor yield duration if transmission is disabled then reenabled
[ https://issues.apache.org/jira/browse/NIFI-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021756#comment-16021756 ] Matt Gilman edited comment on NIFI-3963 at 5/23/17 8:16 PM: [~JPercivall] I have been unable to reproduce this issue with the supplied steps. I've attached a screenshot showing the yield duration being honored throughout multiple transmission cycles. Any other details that may help me reproduce? Can you check the flow.xml.gz to see the configuration value in there? Please let me know if I've misunderstood something. Thanks. was (Author: mcgilman): [~JPercivall] I have been unable to reproduce this issue with the supplied steps. I've attached a screenshot showing the yield duration being honored throughout multiple transmission cycles. Any other details that may help me reproduce? Can you check the flow.xml.gz to see the configuration value in there? > Remote Process Group does not honor yield duration if transmission is > disabled then reenabled > - > > Key: NIFI-3963 > URL: https://issues.apache.org/jira/browse/NIFI-3963 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Joseph Percivall > Attachments: unable-to-reproduce-nifi-3963.png > > > To reproduce, create two NiFi instances properly set up for unsecure or > secure S2S. One on side add an input/output port and the other an RPG. > Enable transmission on the RPG. Stop the instance with the port. On the > running instance see NiFi honoring the yield duration. Then stop and start > transmission, see NiFi repeatedly attempt to connect to the instance (I saw > multiple log messages per millisecond). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3963) Remote Process Group does not honor yield duration if transmission is disabled then reenabled
[ https://issues.apache.org/jira/browse/NIFI-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-3963: -- Attachment: unable-to-reproduce-nifi-3963.png > Remote Process Group does not honor yield duration if transmission is > disabled then reenabled > - > > Key: NIFI-3963 > URL: https://issues.apache.org/jira/browse/NIFI-3963 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Joseph Percivall > Attachments: unable-to-reproduce-nifi-3963.png > > > To reproduce, create two NiFi instances properly set up for unsecure or > secure S2S. One on side add an input/output port and the other an RPG. > Enable transmission on the RPG. Stop the instance with the port. On the > running instance see NiFi honoring the yield duration. Then stop and start > transmission, see NiFi repeatedly attempt to connect to the instance (I saw > multiple log messages per millisecond). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3963) Remote Process Group does not honor yield duration if transmission is disabled then reenabled
[ https://issues.apache.org/jira/browse/NIFI-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021756#comment-16021756 ] Matt Gilman commented on NIFI-3963: --- [~JPercivall] I have been unable to reproduce this issue with the supplied steps. I've attached a screenshot showing the yield duration being honored throughout multiple transmission cycles. Any other details that may help me reproduce? Can you check the flow.xml.gz to see the configuration value in there? > Remote Process Group does not honor yield duration if transmission is > disabled then reenabled > - > > Key: NIFI-3963 > URL: https://issues.apache.org/jira/browse/NIFI-3963 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Joseph Percivall > > To reproduce, create two NiFi instances properly set up for unsecure or > secure S2S. One on side add an input/output port and the other an RPG. > Enable transmission on the RPG. Stop the instance with the port. On the > running instance see NiFi honoring the yield duration. Then stop and start > transmission, see NiFi repeatedly attempt to connect to the instance (I saw > multiple log messages per millisecond). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3964) Grok processors - Allow multiple pattern configuration files
[ https://issues.apache.org/jira/browse/NIFI-3964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021727#comment-16021727 ] Joseph Witt commented on NIFI-3964: --- consider having them looked up via a simple lookup service. The current lookup api for simple strings could be a good fit. Lets really try to avoid file based approaches. > Grok processors - Allow multiple pattern configuration files > > > Key: NIFI-3964 > URL: https://issues.apache.org/jira/browse/NIFI-3964 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Trivial > > When using a lot of grok processors with different teams and use cases, it > would be useful to be able to split the pattern file in different files that > can be individually managed. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (NIFI-3964) Grok processors - Allow multiple pattern configuration files
Pierre Villard created NIFI-3964: Summary: Grok processors - Allow multiple pattern configuration files Key: NIFI-3964 URL: https://issues.apache.org/jira/browse/NIFI-3964 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: Pierre Villard Assignee: Pierre Villard Priority: Trivial When using a lot of grok processors with different teams and use cases, it would be useful to be able to split the pattern file in different files that can be individually managed. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (NIFI-3963) Remote Process Group does not honor yield duration if transmission is disabled then reenabled
Joseph Percivall created NIFI-3963: -- Summary: Remote Process Group does not honor yield duration if transmission is disabled then reenabled Key: NIFI-3963 URL: https://issues.apache.org/jira/browse/NIFI-3963 Project: Apache NiFi Issue Type: Bug Affects Versions: 1.2.0 Reporter: Joseph Percivall To reproduce, create two NiFi instances properly set up for unsecure or secure S2S. One on side add an input/output port and the other an RPG. Enable transmission on the RPG. Stop the instance with the port. On the running instance see NiFi honoring the yield duration. Then stop and start transmission, see NiFi repeatedly attempt to connect to the instance (I saw multiple log messages per millisecond). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Reopened] (NIFI-3961) HiveConnectionPool with yarn queue settings
[ https://issues.apache.org/jira/browse/NIFI-3961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard reopened NIFI-3961: -- > HiveConnectionPool with yarn queue settings > --- > > Key: NIFI-3961 > URL: https://issues.apache.org/jira/browse/NIFI-3961 > Project: Apache NiFi > Issue Type: Wish > Components: Extensions >Reporter: Hassan Faouaz >Priority: Minor > Fix For: 1.3.0 > > > Ability to specify yarn queue when getting a Hive connection. This is one > example. Possible use the same feature to pass in any parameter to a > connection string URL as parameter based -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (NIFI-3961) HiveConnectionPool with yarn queue settings
[ https://issues.apache.org/jira/browse/NIFI-3961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-3961. -- Resolution: Duplicate > HiveConnectionPool with yarn queue settings > --- > > Key: NIFI-3961 > URL: https://issues.apache.org/jira/browse/NIFI-3961 > Project: Apache NiFi > Issue Type: Wish > Components: Extensions >Reporter: Hassan Faouaz >Priority: Minor > Fix For: 1.3.0 > > > Ability to specify yarn queue when getting a Hive connection. This is one > example. Possible use the same feature to pass in any parameter to a > connection string URL as parameter based -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (NIFI-3867) Add Expression Language support to HiveConnectionPool
[ https://issues.apache.org/jira/browse/NIFI-3867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-3867. -- Resolution: Fixed > Add Expression Language support to HiveConnectionPool > - > > Key: NIFI-3867 > URL: https://issues.apache.org/jira/browse/NIFI-3867 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess > Fix For: 1.3.0 > > > The HiveConnectionPool controller service does not accept Expression Language > for fields such as the JDBC URL, Resource locations, etc. With the advent of > the Variable Registry, improving these fields to accept and evaluate NiFi > Expression Language (not against a flow file, but against the environment to > include a variable registry) would be helpful for the development lifecycle > (dev to test to production, e.g.). > This is an extension of NIFI-2407. NIFI-2923 will handle the Kerberos > properties (as they are shared amongst many processors) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3867) Add Expression Language support to HiveConnectionPool
[ https://issues.apache.org/jira/browse/NIFI-3867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021704#comment-16021704 ] ASF subversion and git services commented on NIFI-3867: --- Commit fb925fc182c78cdb31c699c203657ac55ec774a9 in nifi's branch refs/heads/master from [~mattyb149] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=fb925fc ] NIFI-3867: Fixed issue with getConnectionURL in HiveConnectionPool using Expression Language Signed-off-by: Pierre Villard This closes #1847. > Add Expression Language support to HiveConnectionPool > - > > Key: NIFI-3867 > URL: https://issues.apache.org/jira/browse/NIFI-3867 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess > Fix For: 1.3.0 > > > The HiveConnectionPool controller service does not accept Expression Language > for fields such as the JDBC URL, Resource locations, etc. With the advent of > the Variable Registry, improving these fields to accept and evaluate NiFi > Expression Language (not against a flow file, but against the environment to > include a variable registry) would be helpful for the development lifecycle > (dev to test to production, e.g.). > This is an extension of NIFI-2407. NIFI-2923 will handle the Kerberos > properties (as they are shared amongst many processors) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3867) Add Expression Language support to HiveConnectionPool
[ https://issues.apache.org/jira/browse/NIFI-3867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021705#comment-16021705 ] ASF GitHub Bot commented on NIFI-3867: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1847 > Add Expression Language support to HiveConnectionPool > - > > Key: NIFI-3867 > URL: https://issues.apache.org/jira/browse/NIFI-3867 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess > Fix For: 1.3.0 > > > The HiveConnectionPool controller service does not accept Expression Language > for fields such as the JDBC URL, Resource locations, etc. With the advent of > the Variable Registry, improving these fields to accept and evaluate NiFi > Expression Language (not against a flow file, but against the environment to > include a variable registry) would be helpful for the development lifecycle > (dev to test to production, e.g.). > This is an extension of NIFI-2407. NIFI-2923 will handle the Kerberos > properties (as they are shared amongst many processors) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3867) Add Expression Language support to HiveConnectionPool
[ https://issues.apache.org/jira/browse/NIFI-3867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021706#comment-16021706 ] ASF GitHub Bot commented on NIFI-3867: -- Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/1847 +1, contrib-check OK, merged to master, thanks @mattyb149! > Add Expression Language support to HiveConnectionPool > - > > Key: NIFI-3867 > URL: https://issues.apache.org/jira/browse/NIFI-3867 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess > Fix For: 1.3.0 > > > The HiveConnectionPool controller service does not accept Expression Language > for fields such as the JDBC URL, Resource locations, etc. With the advent of > the Variable Registry, improving these fields to accept and evaluate NiFi > Expression Language (not against a flow file, but against the environment to > include a variable registry) would be helpful for the development lifecycle > (dev to test to production, e.g.). > This is an extension of NIFI-2407. NIFI-2923 will handle the Kerberos > properties (as they are shared amongst many processors) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi pull request #1847: NIFI-3867: Fixed issue with getConnectionURL in Hiv...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1847 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1847: NIFI-3867: Fixed issue with getConnectionURL in HiveConnec...
Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/1847 +1, contrib-check OK, merged to master, thanks @mattyb149! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3867) Add Expression Language support to HiveConnectionPool
[ https://issues.apache.org/jira/browse/NIFI-3867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021702#comment-16021702 ] ASF GitHub Bot commented on NIFI-3867: -- GitHub user mattyb149 opened a pull request: https://github.com/apache/nifi/pull/1847 NIFI-3867: Fixed issue with getConnectionURL in HiveConnectionPool using EL Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mattyb149/nifi NIFI-3867_1 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1847.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1847 commit 86ba9f8ba495196fae7414ffe2f63da919214cbf Author: Matt Burgess Date: 2017-05-23T19:21:10Z NIFI-3867: Fixed issue with getConnectionURL in HiveConnectionPool using Expression Language > Add Expression Language support to HiveConnectionPool > - > > Key: NIFI-3867 > URL: https://issues.apache.org/jira/browse/NIFI-3867 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess > Fix For: 1.3.0 > > > The HiveConnectionPool controller service does not accept Expression Language > for fields such as the JDBC URL, Resource locations, etc. With the advent of > the Variable Registry, improving these fields to accept and evaluate NiFi > Expression Language (not against a flow file, but against the environment to > include a variable registry) would be helpful for the development lifecycle > (dev to test to production, e.g.). > This is an extension of NIFI-2407. NIFI-2923 will handle the Kerberos > properties (as they are shared amongst many processors) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi pull request #1847: NIFI-3867: Fixed issue with getConnectionURL in Hiv...
GitHub user mattyb149 opened a pull request: https://github.com/apache/nifi/pull/1847 NIFI-3867: Fixed issue with getConnectionURL in HiveConnectionPool using EL Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mattyb149/nifi NIFI-3867_1 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1847.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1847 commit 86ba9f8ba495196fae7414ffe2f63da919214cbf Author: Matt Burgess Date: 2017-05-23T19:21:10Z NIFI-3867: Fixed issue with getConnectionURL in HiveConnectionPool using Expression Language --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Reopened] (NIFI-3867) Add Expression Language support to HiveConnectionPool
[ https://issues.apache.org/jira/browse/NIFI-3867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess reopened NIFI-3867: The getConnectionURL() method returns an unevaluated value for the Hive Database URL, giving incorrect/incomplete information for provenance events. The method should use the same evaluated value used in the data source itself. > Add Expression Language support to HiveConnectionPool > - > > Key: NIFI-3867 > URL: https://issues.apache.org/jira/browse/NIFI-3867 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess > Fix For: 1.3.0 > > > The HiveConnectionPool controller service does not accept Expression Language > for fields such as the JDBC URL, Resource locations, etc. With the advent of > the Variable Registry, improving these fields to accept and evaluate NiFi > Expression Language (not against a flow file, but against the environment to > include a variable registry) would be helpful for the development lifecycle > (dev to test to production, e.g.). > This is an extension of NIFI-2407. NIFI-2923 will handle the Kerberos > properties (as they are shared amongst many processors) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (NIFI-3961) HiveConnectionPool with yarn queue settings
[ https://issues.apache.org/jira/browse/NIFI-3961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hassan Faouaz resolved NIFI-3961. - Resolution: Fixed Fix Version/s: 1.3.0 Thanks to [~pvillard] at pointing the Jira change. > HiveConnectionPool with yarn queue settings > --- > > Key: NIFI-3961 > URL: https://issues.apache.org/jira/browse/NIFI-3961 > Project: Apache NiFi > Issue Type: Wish > Components: Extensions >Reporter: Hassan Faouaz >Priority: Minor > Fix For: 1.3.0 > > > Ability to specify yarn queue when getting a Hive connection. This is one > example. Possible use the same feature to pass in any parameter to a > connection string URL as parameter based -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3961) HiveConnectionPool with yarn queue settings
[ https://issues.apache.org/jira/browse/NIFI-3961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021682#comment-16021682 ] Hassan Faouaz commented on NIFI-3961: - Yes, NIFI 3867 is taking care of it! On a side note, specifying queue, I meant specifying Yarn queue for capacity. We do have different flows of data and each has different priority to run on Hadoop. Anyhow, I think Nifi 1.3.0 will take care of the feature I was proposing and no need to do a pull-request. > HiveConnectionPool with yarn queue settings > --- > > Key: NIFI-3961 > URL: https://issues.apache.org/jira/browse/NIFI-3961 > Project: Apache NiFi > Issue Type: Wish > Components: Extensions >Reporter: Hassan Faouaz >Priority: Minor > > Ability to specify yarn queue when getting a Hive connection. This is one > example. Possible use the same feature to pass in any parameter to a > connection string URL as parameter based -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3962) Update ConsumeKafkaRecord_0_10 to better batch together messages into fewer flowfiles
[ https://issues.apache.org/jira/browse/NIFI-3962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-3962: - Assignee: Mark Payne Status: Patch Available (was: Open) > Update ConsumeKafkaRecord_0_10 to better batch together messages into fewer > flowfiles > - > > Key: NIFI-3962 > URL: https://issues.apache.org/jira/browse/NIFI-3962 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne > > Currently, ConsumeKafkaRecord will batch together only as many records in a > FlowFile as it is able to obtain for a single partition & topic. This should > be updated to allow for multiple iterations of KafkaConsumer.poll() in the > same way that ConsumeKafka does with a Demarcator configured. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3962) Update ConsumeKafkaRecord_0_10 to better batch together messages into fewer flowfiles
[ https://issues.apache.org/jira/browse/NIFI-3962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021678#comment-16021678 ] ASF GitHub Bot commented on NIFI-3962: -- GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/1846 NIFI-3962: Updated ConsumerLease to better handle batching of message… …s into a single FlowFile in the same way that it is handled for demarcated data Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-3962 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1846.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1846 commit 49888dfcf9cdd0a402fff3a75ace721da851f861 Author: Mark Payne Date: 2017-05-23T19:08:37Z NIFI-3962: Updated ConsumerLease to better handle batching of messages into a single FlowFile in the same way that it is handled for demarcated data > Update ConsumeKafkaRecord_0_10 to better batch together messages into fewer > flowfiles > - > > Key: NIFI-3962 > URL: https://issues.apache.org/jira/browse/NIFI-3962 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne > > Currently, ConsumeKafkaRecord will batch together only as many records in a > FlowFile as it is able to obtain for a single partition & topic. This should > be updated to allow for multiple iterations of KafkaConsumer.poll() in the > same way that ConsumeKafka does with a Demarcator configured. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi pull request #1846: NIFI-3962: Updated ConsumerLease to better handle b...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/1846 NIFI-3962: Updated ConsumerLease to better handle batching of message⦠â¦s into a single FlowFile in the same way that it is handled for demarcated data Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-3962 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1846.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1846 commit 49888dfcf9cdd0a402fff3a75ace721da851f861 Author: Mark Payne Date: 2017-05-23T19:08:37Z NIFI-3962: Updated ConsumerLease to better handle batching of messages into a single FlowFile in the same way that it is handled for demarcated data --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (NIFI-3962) Update ConsumeKafkaRecord_0_10 to better batch together messages into fewer flowfiles
Mark Payne created NIFI-3962: Summary: Update ConsumeKafkaRecord_0_10 to better batch together messages into fewer flowfiles Key: NIFI-3962 URL: https://issues.apache.org/jira/browse/NIFI-3962 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: Mark Payne Currently, ConsumeKafkaRecord will batch together only as many records in a FlowFile as it is able to obtain for a single partition & topic. This should be updated to allow for multiple iterations of KafkaConsumer.poll() in the same way that ConsumeKafka does with a Demarcator configured. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3961) HiveConnectionPool with yarn queue settings
[ https://issues.apache.org/jira/browse/NIFI-3961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021675#comment-16021675 ] Pierre Villard commented on NIFI-3961: -- The database URL will accept expression language so you could use variables (starting with 1.3.0 and NIFI-3867). Not sure to understand "we needed to pass queue name base on flows" though. Do you mean specify a queue based on an attribute of a flow file being processed, or by a different queue for different workflows? If the latter, I would create a controller service per workflow. If that should be per flow file, then that's a different story since the data source is created only when the controller service is configured. In any case, you can certainly fire a pull request with your code :) > HiveConnectionPool with yarn queue settings > --- > > Key: NIFI-3961 > URL: https://issues.apache.org/jira/browse/NIFI-3961 > Project: Apache NiFi > Issue Type: Wish > Components: Extensions >Reporter: Hassan Faouaz >Priority: Minor > > Ability to specify yarn queue when getting a Hive connection. This is one > example. Possible use the same feature to pass in any parameter to a > connection string URL as parameter based -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3961) HiveConnectionPool with yarn queue settings
[ https://issues.apache.org/jira/browse/NIFI-3961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hassan Faouaz updated NIFI-3961: Description: Ability to specify yarn queue when getting a Hive connection. This is one example. Possible use the same feature to pass in any parameter to a connection string URL as parameter based (was: Ability to specify yarn queue when getting a Hive connection) > HiveConnectionPool with yarn queue settings > --- > > Key: NIFI-3961 > URL: https://issues.apache.org/jira/browse/NIFI-3961 > Project: Apache NiFi > Issue Type: Wish > Components: Extensions >Reporter: Hassan Faouaz >Priority: Minor > > Ability to specify yarn queue when getting a Hive connection. This is one > example. Possible use the same feature to pass in any parameter to a > connection string URL as parameter based -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (NIFI-3961) HiveConnectionPool with yarn queue settings
[ https://issues.apache.org/jira/browse/NIFI-3961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021660#comment-16021660 ] Hassan Faouaz edited comment on NIFI-3961 at 5/23/17 6:55 PM: -- Bonjour [~pvillard] Yes we can do that, but we realized we needed to pass queue name base on flows and the URL connection string doesn't take a variable. or does it? Actually this feature can be leveraged to pass in any parameters to the Connection String. was (Author: hfaouaz): Bonjour [~pvillard] Yes we can do that, but we realized we needed to pass queue name base on flows and the URL connection string doesn't take a variable. or does it? > HiveConnectionPool with yarn queue settings > --- > > Key: NIFI-3961 > URL: https://issues.apache.org/jira/browse/NIFI-3961 > Project: Apache NiFi > Issue Type: Wish > Components: Extensions >Reporter: Hassan Faouaz >Priority: Minor > > Ability to specify yarn queue when getting a Hive connection -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3961) HiveConnectionPool with yarn queue settings
[ https://issues.apache.org/jira/browse/NIFI-3961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021660#comment-16021660 ] Hassan Faouaz commented on NIFI-3961: - Bonjour [~pvillard] Yes we can do that, but we realized we needed to pass queue name base on flows and the URL connection string doesn't take a variable. or does it? > HiveConnectionPool with yarn queue settings > --- > > Key: NIFI-3961 > URL: https://issues.apache.org/jira/browse/NIFI-3961 > Project: Apache NiFi > Issue Type: Wish > Components: Extensions >Reporter: Hassan Faouaz >Priority: Minor > > Ability to specify yarn queue when getting a Hive connection -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3961) HiveConnectionPool with yarn queue settings
[
https://issues.apache.org/jira/browse/NIFI-3961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021639#comment-16021639
]
Pierre Villard commented on NIFI-3961:
--
Hi [~hfaouaz], never tried but you should be able to set it in the connection
string, no? Something like:
{{jdbc:hive2://localhost:1?tez.queue.name=test}}
> HiveConnectionPool with yarn queue settings
> ---
>
> Key: NIFI-3961
> URL: https://issues.apache.org/jira/browse/NIFI-3961
> Project: Apache NiFi
> Issue Type: Wish
> Components: Extensions
>Reporter: Hassan Faouaz
>Priority: Minor
>
> Ability to specify yarn queue when getting a Hive connection
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[GitHub] nifi-minifi-cpp issue #103: MINIFI-177 Incorporating Apache RAT to build wit...
Github user apiri commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/103 @phrocker Good call. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3961) HiveConnectionPool with yarn queue settings
[ https://issues.apache.org/jira/browse/NIFI-3961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021630#comment-16021630 ] Hassan Faouaz commented on NIFI-3961: - We have extended the existing HiveConnectionPool with ability to pass a Tez Yarn queue to the connection settings. Perhaps we can roll this into the next version if not already in progress. > HiveConnectionPool with yarn queue settings > --- > > Key: NIFI-3961 > URL: https://issues.apache.org/jira/browse/NIFI-3961 > Project: Apache NiFi > Issue Type: Wish > Components: Extensions >Reporter: Hassan Faouaz >Priority: Minor > > Ability to specify yarn queue when getting a Hive connection -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (NIFI-3961) HiveConnectionPool with yarn queue settings
Hassan Faouaz created NIFI-3961: --- Summary: HiveConnectionPool with yarn queue settings Key: NIFI-3961 URL: https://issues.apache.org/jira/browse/NIFI-3961 Project: Apache NiFi Issue Type: Wish Components: Extensions Reporter: Hassan Faouaz Priority: Minor Ability to specify yarn queue when getting a Hive connection -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3960) ExtractGrok - Include exception message when compile fails
[ https://issues.apache.org/jira/browse/NIFI-3960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-3960: - Status: Patch Available (was: Open) > ExtractGrok - Include exception message when compile fails > -- > > Key: NIFI-3960 > URL: https://issues.apache.org/jira/browse/NIFI-3960 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Trivial > > When using ExtractGrok, I noticed that there is no way to have details when a > grok expression is not compiling. It's just saying "Not a valid Grok > Expression". Could be nice to include the message of the corresponding > exception in the validator. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3960) ExtractGrok - Include exception message when compile fails
[ https://issues.apache.org/jira/browse/NIFI-3960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021607#comment-16021607 ] ASF GitHub Bot commented on NIFI-3960: -- GitHub user pvillard31 opened a pull request: https://github.com/apache/nifi/pull/1845 NIFI-3960 - ExtractGrok - Include exception message when compile fails Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [X] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? - [X] Is your initial contribution a single, squashed commit? ### For code changes: - [X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/pvillard31/nifi grok-exception Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1845.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1845 commit 698f71ca43e21d024954e22c17e1a38203c47092 Author: Pierre Villard Date: 2017-05-23T18:27:22Z ExtractGrok - Include exception message when compile fails > ExtractGrok - Include exception message when compile fails > -- > > Key: NIFI-3960 > URL: https://issues.apache.org/jira/browse/NIFI-3960 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Trivial > > When using ExtractGrok, I noticed that there is no way to have details when a > grok expression is not compiling. It's just saying "Not a valid Grok > Expression". Could be nice to include the message of the corresponding > exception in the validator. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi pull request #1845: NIFI-3960 - ExtractGrok - Include exception message...
GitHub user pvillard31 opened a pull request: https://github.com/apache/nifi/pull/1845 NIFI-3960 - ExtractGrok - Include exception message when compile fails Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [X] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? - [X] Is your initial contribution a single, squashed commit? ### For code changes: - [X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/pvillard31/nifi grok-exception Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1845.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1845 commit 698f71ca43e21d024954e22c17e1a38203c47092 Author: Pierre Villard Date: 2017-05-23T18:27:22Z ExtractGrok - Include exception message when compile fails --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (NIFI-3960) ExtractGrok - Include exception message when compile fails
Pierre Villard created NIFI-3960: Summary: ExtractGrok - Include exception message when compile fails Key: NIFI-3960 URL: https://issues.apache.org/jira/browse/NIFI-3960 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: Pierre Villard Assignee: Pierre Villard Priority: Trivial When using ExtractGrok, I noticed that there is no way to have details when a grok expression is not compiling. It's just saying "Not a valid Grok Expression". Could be nice to include the message of the corresponding exception in the validator. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3903) Basic Unit Test Coverage for AWS Processors
[ https://issues.apache.org/jira/browse/NIFI-3903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021560#comment-16021560 ] ASF GitHub Bot commented on NIFI-3903: -- Github user jvwing commented on the issue: https://github.com/apache/nifi/pull/1814 Thanks again, @pvillard31 > Basic Unit Test Coverage for AWS Processors > --- > > Key: NIFI-3903 > URL: https://issues.apache.org/jira/browse/NIFI-3903 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: James Wing >Assignee: James Wing >Priority: Minor > Fix For: 1.3.0 > > > Many of the AWS processors rely heavily on integration tests for coverage of > major features. While there is no substitute for running the code against a > live AWS environment, the live environment requirement means that integration > tests are run infrequently. Regular unit tests should be included: > * Provide coverage of basic features without integration tests > * Demonstrate at least one mocking pattern for testing AWS request/response > * Remove excuses for not providing unit tests with new features > * Clarify integration tests vs. unit tests > Processors that need help include: > * InvokeLambda > * DeleteS3Object, FetchS3Object, PutS3Object, ListS3 > * PutSNS > * DeleteSQS, GetSQS, PutSQS -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi issue #1814: NIFI-3903 Unit Tests for AWS Processors
Github user jvwing commented on the issue: https://github.com/apache/nifi/pull/1814 Thanks again, @pvillard31 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (NIFI-3903) Basic Unit Test Coverage for AWS Processors
[ https://issues.apache.org/jira/browse/NIFI-3903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-3903: - Component/s: Extensions > Basic Unit Test Coverage for AWS Processors > --- > > Key: NIFI-3903 > URL: https://issues.apache.org/jira/browse/NIFI-3903 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: James Wing >Assignee: James Wing >Priority: Minor > Fix For: 1.3.0 > > > Many of the AWS processors rely heavily on integration tests for coverage of > major features. While there is no substitute for running the code against a > live AWS environment, the live environment requirement means that integration > tests are run infrequently. Regular unit tests should be included: > * Provide coverage of basic features without integration tests > * Demonstrate at least one mocking pattern for testing AWS request/response > * Remove excuses for not providing unit tests with new features > * Clarify integration tests vs. unit tests > Processors that need help include: > * InvokeLambda > * DeleteS3Object, FetchS3Object, PutS3Object, ListS3 > * PutSNS > * DeleteSQS, GetSQS, PutSQS -- This message was sent by Atlassian JIRA (v6.3.15#6346)
