[jira] [Commented] (NIFI-2835) GetAzureEventHub processor should leverage partition offset to better handle restarts
[ https://issues.apache.org/jira/browse/NIFI-2835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247046#comment-16247046 ] Koji Kawamura commented on NIFI-2835: - Hi [~Eulicny], To forgo or not, the decision is up to you. However, while I was implementing ConsumeAzureEventHub, I happened to look at Azure Event Hub client source code a lot. Then I felt if we had to do the similar thing ourselves in NiFi codebase, it'd be challenging.. So, I'd like you to try ConsumeAzureEventHub processor if possible, which is available as a PR for reviewing now. Any review comment would be appreciated. https://github.com/apache/nifi/pull/2264 If you find that is helpful and no need to do anything at GetAzureEventHub, then this JIRA can be closed by simply add 'SeeAlso' and probably 'DeprecationNotice' annotation to GetAzureEventHub so that users can notice the limitation and another solution. How do you think? > GetAzureEventHub processor should leverage partition offset to better handle > restarts > - > > Key: NIFI-2835 > URL: https://issues.apache.org/jira/browse/NIFI-2835 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joseph Percivall >Assignee: Eric Ulicny > > The GetAzureEventHub processor utilizes the Azure client that consists of > receivers for each partition. The processor stores them in a map[1] that gets > cleared every time the processor is stopped[2]. These receivers have > partition offsets which keep track of which message it's currently on and > which it should receive next. So currently, when the processor is > stopped/restarted, any tracking of which message is next to be received is > lost. > If instead of clearing the map each time, we hold onto the receivers, or kept > track of the partitionId/Offsets when stopping, (barring any relevant > configuration changes) the processor would restart exactly where it left off > with no loss of data. > This would work very well with NIFI-2826. > [1]https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/eventhub/GetAzureEventHub.java#L122 > [2] > https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/eventhub/GetAzureEventHub.java#L229 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #2264: NIFI-4595: Add ConsumeAzureEventHub.
GitHub user ijokarumawak opened a pull request: https://github.com/apache/nifi/pull/2264 NIFI-4595: Add ConsumeAzureEventHub. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [x] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijokarumawak/nifi nifi-4595 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2264.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2264 commit f45d305b2db38e2a2ca6ebd1398dc79c7aee4398 Author: Koji KawamuraDate: 2017-11-08T14:20:58Z NIFI-4595: Add ConsumeAzureEventHub. ---
[jira] [Commented] (NIFI-4595) Add ConsumeAzureEventHub processor
[ https://issues.apache.org/jira/browse/NIFI-4595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247038#comment-16247038 ] ASF GitHub Bot commented on NIFI-4595: -- GitHub user ijokarumawak opened a pull request: https://github.com/apache/nifi/pull/2264 NIFI-4595: Add ConsumeAzureEventHub. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [x] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijokarumawak/nifi nifi-4595 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2264.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2264 commit f45d305b2db38e2a2ca6ebd1398dc79c7aee4398 Author: Koji KawamuraDate: 2017-11-08T14:20:58Z NIFI-4595: Add ConsumeAzureEventHub. > Add ConsumeAzureEventHub processor > -- > > Key: NIFI-4595 > URL: https://issues.apache.org/jira/browse/NIFI-4595 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Koji Kawamura >Assignee: Koji Kawamura > > As reported by NIFI-2835, the existing GetAzureEventHub processor does not > handle partition distribution among nodes in a NiFi cluster, and handling > offset information across NiFi restart. > The processor uses PartitionReceiver directly, however, there's another > approach to receive events from Event Hub, that is using EventProcessorHost > class. > EventProcessorHost uses Azure Storage to store consumer group offset > information so that it can distribute consumer load among consumer instances > within the same group. Also this way can handle restart and NiFi cluster > scale scenario better. > In addition to above enhancements, we could also add Record data model > capability to new ConsumeAzureEventHub. Similar to ConsumeKafkaRecord > processor. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4595) Add ConsumeAzureEventHub processor
[ https://issues.apache.org/jira/browse/NIFI-4595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-4595: Status: Patch Available (was: In Progress) > Add ConsumeAzureEventHub processor > -- > > Key: NIFI-4595 > URL: https://issues.apache.org/jira/browse/NIFI-4595 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Koji Kawamura >Assignee: Koji Kawamura > > As reported by NIFI-2835, the existing GetAzureEventHub processor does not > handle partition distribution among nodes in a NiFi cluster, and handling > offset information across NiFi restart. > The processor uses PartitionReceiver directly, however, there's another > approach to receive events from Event Hub, that is using EventProcessorHost > class. > EventProcessorHost uses Azure Storage to store consumer group offset > information so that it can distribute consumer load among consumer instances > within the same group. Also this way can handle restart and NiFi cluster > scale scenario better. > In addition to above enhancements, we could also add Record data model > capability to new ConsumeAzureEventHub. Similar to ConsumeKafkaRecord > processor. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4595) Add ConsumeAzureEventHub processor
[ https://issues.apache.org/jira/browse/NIFI-4595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-4595: Issue Type: Improvement (was: Bug) > Add ConsumeAzureEventHub processor > -- > > Key: NIFI-4595 > URL: https://issues.apache.org/jira/browse/NIFI-4595 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Koji Kawamura >Assignee: Koji Kawamura > > As reported by NIFI-2835, the existing GetAzureEventHub processor does not > handle partition distribution among nodes in a NiFi cluster, and handling > offset information across NiFi restart. > The processor uses PartitionReceiver directly, however, there's another > approach to receive events from Event Hub, that is using EventProcessorHost > class. > EventProcessorHost uses Azure Storage to store consumer group offset > information so that it can distribute consumer load among consumer instances > within the same group. Also this way can handle restart and NiFi cluster > scale scenario better. > In addition to above enhancements, we could also add Record data model > capability to new ConsumeAzureEventHub. Similar to ConsumeKafkaRecord > processor. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (NIFI-4595) Add ConsumeAzureEventHub processor
Koji Kawamura created NIFI-4595: --- Summary: Add ConsumeAzureEventHub processor Key: NIFI-4595 URL: https://issues.apache.org/jira/browse/NIFI-4595 Project: Apache NiFi Issue Type: Bug Components: Extensions Reporter: Koji Kawamura Assignee: Koji Kawamura As reported by NIFI-2835, the existing GetAzureEventHub processor does not handle partition distribution among nodes in a NiFi cluster, and handling offset information across NiFi restart. The processor uses PartitionReceiver directly, however, there's another approach to receive events from Event Hub, that is using EventProcessorHost class. EventProcessorHost uses Azure Storage to store consumer group offset information so that it can distribute consumer load among consumer instances within the same group. Also this way can handle restart and NiFi cluster scale scenario better. In addition to above enhancements, we could also add Record data model capability to new ConsumeAzureEventHub. Similar to ConsumeKafkaRecord processor. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4594) Content Viewer not working when reverse proxy places nifi not at the root
[ https://issues.apache.org/jira/browse/NIFI-4594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Suckow updated NIFI-4594: Description: As a NiFi user I need to be able to use the configurable NiFi content viewer to view flow file content. Currently, it works perfectly if it is NOT behind a reverse proxy, like nginx, or AWS load balancer. It appears that the URL is not being properly formed. Below is an example of the issue with another example where a modified URL does work. NOTE: URLs will not be reachable and are presented in this ticket for illustrative purposes DOES NOT WORK: https://nifi-gray.unrulyfire.com/nifi-content-viewer/?ref=https%3A%2F%2Fnifi-gray.unrulyfire.com%3A443%2Fmynifi%2Fnifi-api%2Fflowfile-queues%2Fb4aa9ade-015e-1000--8856fa2d%2Fflowfiles%2Fb7c9b786-b85c-4feb-83a2-00bfcd0ffad5%2Fcontent%3FclusterNodeId%3D8b0cc92a-6266-4f3d-bcb0-95d994086c80 DOES WORK : https://nifi-gray.unrulyfire.com/mynifi/nifi-content-viewer/?ref=https%3A%2F%2Fnifi-gray.unrulyfire.com%3A443%2Fmynifi%2Fnifi-api%2Fflowfile-queues%2Fb4aa9ade-015e-1000--8856fa2d%2Fflowfiles%2Fb7c9b786-b85c-4feb-83a2-00bfcd0ffad5%2Fcontent%3FclusterNodeId%3D8b0cc92a-6266-4f3d-bcb0-95d994086c80 was: As a NiFi user I need to be able to use the configurable NiFi content viewer to view flow file content. Currently, it works perfectly if it is NOT behind a reverse proxy, like nginx, or AWS load balancer. It appears that the URL is not being properly formed. Below is an example of the issue with another example where a modified URL does work. NOTE: URLs will not be reachable and are presented in this ticket for illustrative purposes, but the attached screenshots show the effect of the URLs DOES NOT WORK: https://nifi-gray.unrulyfire.com/nifi-content-viewer/?ref=https%3A%2F%2Fnifi-gray.unrulyfire.com%3A443%2Fmynifi%2Fnifi-api%2Fflowfile-queues%2Fb4aa9ade-015e-1000--8856fa2d%2Fflowfiles%2Fb7c9b786-b85c-4feb-83a2-00bfcd0ffad5%2Fcontent%3FclusterNodeId%3D8b0cc92a-6266-4f3d-bcb0-95d994086c80 DOES WORK : https://nifi-gray.unrulyfire.com/mynifi/nifi-content-viewer/?ref=https%3A%2F%2Fnifi-gray.unrulyfire.com%3A443%2Fmynifi%2Fnifi-api%2Fflowfile-queues%2Fb4aa9ade-015e-1000--8856fa2d%2Fflowfiles%2Fb7c9b786-b85c-4feb-83a2-00bfcd0ffad5%2Fcontent%3FclusterNodeId%3D8b0cc92a-6266-4f3d-bcb0-95d994086c80 > Content Viewer not working when reverse proxy places nifi not at the root > - > > Key: NIFI-4594 > URL: https://issues.apache.org/jira/browse/NIFI-4594 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.4.0 >Reporter: Thomas Suckow > > As a NiFi user I need to be able to use the configurable NiFi content viewer > to view flow file content. Currently, it works perfectly if it is NOT behind > a reverse proxy, like nginx, or AWS load balancer. It appears that the URL is > not being properly formed. Below is an example of the issue with another > example where a modified URL does work. > NOTE: URLs will not be reachable and are presented in this ticket for > illustrative purposes > DOES NOT WORK: > https://nifi-gray.unrulyfire.com/nifi-content-viewer/?ref=https%3A%2F%2Fnifi-gray.unrulyfire.com%3A443%2Fmynifi%2Fnifi-api%2Fflowfile-queues%2Fb4aa9ade-015e-1000--8856fa2d%2Fflowfiles%2Fb7c9b786-b85c-4feb-83a2-00bfcd0ffad5%2Fcontent%3FclusterNodeId%3D8b0cc92a-6266-4f3d-bcb0-95d994086c80 > DOES WORK : > https://nifi-gray.unrulyfire.com/mynifi/nifi-content-viewer/?ref=https%3A%2F%2Fnifi-gray.unrulyfire.com%3A443%2Fmynifi%2Fnifi-api%2Fflowfile-queues%2Fb4aa9ade-015e-1000--8856fa2d%2Fflowfiles%2Fb7c9b786-b85c-4feb-83a2-00bfcd0ffad5%2Fcontent%3FclusterNodeId%3D8b0cc92a-6266-4f3d-bcb0-95d994086c80 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (NIFI-4594) Content Viewer not working when reverse proxy places nifi not at the root
Thomas Suckow created NIFI-4594: --- Summary: Content Viewer not working when reverse proxy places nifi not at the root Key: NIFI-4594 URL: https://issues.apache.org/jira/browse/NIFI-4594 Project: Apache NiFi Issue Type: Bug Components: Core UI Affects Versions: 1.4.0 Reporter: Thomas Suckow As a NiFi user I need to be able to use the configurable NiFi content viewer to view flow file content. Currently, it works perfectly if it is NOT behind a reverse proxy, like nginx, or AWS load balancer. It appears that the URL is not being properly formed. Below is an example of the issue with another example where a modified URL does work. NOTE: URLs will not be reachable and are presented in this ticket for illustrative purposes, but the attached screenshots show the effect of the URLs DOES NOT WORK: https://nifi-gray.unrulyfire.com/nifi-content-viewer/?ref=https%3A%2F%2Fnifi-gray.unrulyfire.com%3A443%2Fmynifi%2Fnifi-api%2Fflowfile-queues%2Fb4aa9ade-015e-1000--8856fa2d%2Fflowfiles%2Fb7c9b786-b85c-4feb-83a2-00bfcd0ffad5%2Fcontent%3FclusterNodeId%3D8b0cc92a-6266-4f3d-bcb0-95d994086c80 DOES WORK : https://nifi-gray.unrulyfire.com/mynifi/nifi-content-viewer/?ref=https%3A%2F%2Fnifi-gray.unrulyfire.com%3A443%2Fmynifi%2Fnifi-api%2Fflowfile-queues%2Fb4aa9ade-015e-1000--8856fa2d%2Fflowfiles%2Fb7c9b786-b85c-4feb-83a2-00bfcd0ffad5%2Fcontent%3FclusterNodeId%3D8b0cc92a-6266-4f3d-bcb0-95d994086c80 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #2262: NIFI-4591: Ensure View Details is button is visible when a...
Github user scottyaslan commented on the issue: https://github.com/apache/nifi/pull/2262 Reviewing... ---
[jira] [Commented] (NIFI-4591) Controller Service Details button missing
[ https://issues.apache.org/jira/browse/NIFI-4591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246600#comment-16246600 ] ASF GitHub Bot commented on NIFI-4591: -- Github user scottyaslan commented on the issue: https://github.com/apache/nifi/pull/2262 Reviewing... > Controller Service Details button missing > - > > Key: NIFI-4591 > URL: https://issues.apache.org/jira/browse/NIFI-4591 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.4.0 >Reporter: Matt Gilman >Assignee: Matt Gilman > > Following the modification in NIFI-3781 a regression was introduced that > prevents the rendering of the View Details button when the user has read > permissions but no write permissions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4325) Create a new ElasticSearch processor that supports the JSON DSL
[ https://issues.apache.org/jira/browse/NIFI-4325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246525#comment-16246525 ] ASF GitHub Bot commented on NIFI-4325: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/2113#discussion_r150086797 --- Diff: nifi-assembly/pom.xml --- @@ -338,6 +338,16 @@ org.apache.nifi +nifi-elasticsearch-client-service-nar +nar + + > Key: NIFI-4325 > URL: https://issues.apache.org/jira/browse/NIFI-4325 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mike Thomsen >Priority: Minor > > The existing ElasticSearch processors use the Lucene-style syntax for > querying, not the JSON DSL. A new processor is needed that can take a full > JSON query and execute it. It should also support aggregation queries in this > syntax. A user needs to be able to take a query as-is from Kibana and drop it > into NiFi and have it just run. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4325) Create a new ElasticSearch processor that supports the JSON DSL
[ https://issues.apache.org/jira/browse/NIFI-4325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246533#comment-16246533 ] ASF GitHub Bot commented on NIFI-4325: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/2113#discussion_r150087559 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-5-processors/src/main/java/org/apache/nifi/processors/elasticsearch/JsonQueryElasticsearch5.java --- @@ -0,0 +1,291 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.nifi.processors.elasticsearch; + +import com.fasterxml.jackson.databind.ObjectMapper; +import org.apache.nifi.annotation.behavior.EventDriven; +import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.annotation.lifecycle.OnUnscheduled; +import org.apache.nifi.components.AllowableValue; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.components.Validator; +import org.apache.nifi.elasticsearch.ElasticSearchClientService; +import org.apache.nifi.elasticsearch.ElasticSearchResponse; +import org.apache.nifi.flowfile.FlowFile; +import org.apache.nifi.processor.AbstractProcessor; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.ProcessSession; +import org.apache.nifi.processor.Relationship; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.io.OutputStreamCallback; +import org.apache.nifi.processor.util.StandardValidators; + +import java.io.IOException; +import java.io.OutputStream; +import java.util.ArrayList; +import java.util.Collections; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Optional; +import java.util.Set; + + +@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED) +@EventDriven +@Tags({"elasticsearch", "elasticsearch 5", "query", "read", "get", "json"}) +@CapabilityDescription("A processor that allows the user to run a query (with aggregations) written with the " + +"ElasticSearch JSON DSL. It currently does not support pagination.") +public class JsonQueryElasticsearch5 extends AbstractProcessor { +public static final Relationship REL_SUCCESS = new Relationship.Builder().name("success") +.description("All original flowfiles that don't cause an error to occur go to this relationship. " + +"This applies even if you select the \"break up hits\" option to send individual hits to the " + +"\"hits\" relationship.").build(); + +public static final Relationship REL_FAILURE = new Relationship.Builder().name("failure") +.description("All FlowFiles that cannot be read from Elasticsearch are routed to this relationship").build(); + +public static final Relationship REL_HITS = new Relationship.Builder().name("hits") +.description("Search hits are routed to this relationship.") +.build(); + +public static final Relationship REL_AGGREGATIONS = new Relationship.Builder().name("aggregations") +.description("Aggregations are routed to this relationship.") +.build(); + +public static final PropertyDescriptor INDEX = new PropertyDescriptor.Builder() +.name("el5-fetch-index") +.displayName("Index") +.description("The name of the index to read from") +.required(true) +.expressionLanguageSupported(true) +.addValidator(StandardValidators.NON_EMPTY_VALIDATOR) +.build(); + +public static final PropertyDescriptor
[GitHub] nifi pull request #2113: NIFI-4325 Added new processor that uses the JSON DS...
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/2113#discussion_r150087559 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-5-processors/src/main/java/org/apache/nifi/processors/elasticsearch/JsonQueryElasticsearch5.java --- @@ -0,0 +1,291 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.nifi.processors.elasticsearch; + +import com.fasterxml.jackson.databind.ObjectMapper; +import org.apache.nifi.annotation.behavior.EventDriven; +import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.annotation.lifecycle.OnUnscheduled; +import org.apache.nifi.components.AllowableValue; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.components.Validator; +import org.apache.nifi.elasticsearch.ElasticSearchClientService; +import org.apache.nifi.elasticsearch.ElasticSearchResponse; +import org.apache.nifi.flowfile.FlowFile; +import org.apache.nifi.processor.AbstractProcessor; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.ProcessSession; +import org.apache.nifi.processor.Relationship; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.io.OutputStreamCallback; +import org.apache.nifi.processor.util.StandardValidators; + +import java.io.IOException; +import java.io.OutputStream; +import java.util.ArrayList; +import java.util.Collections; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Optional; +import java.util.Set; + + +@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED) +@EventDriven +@Tags({"elasticsearch", "elasticsearch 5", "query", "read", "get", "json"}) +@CapabilityDescription("A processor that allows the user to run a query (with aggregations) written with the " + +"ElasticSearch JSON DSL. It currently does not support pagination.") +public class JsonQueryElasticsearch5 extends AbstractProcessor { +public static final Relationship REL_SUCCESS = new Relationship.Builder().name("success") +.description("All original flowfiles that don't cause an error to occur go to this relationship. " + +"This applies even if you select the \"break up hits\" option to send individual hits to the " + +"\"hits\" relationship.").build(); + +public static final Relationship REL_FAILURE = new Relationship.Builder().name("failure") +.description("All FlowFiles that cannot be read from Elasticsearch are routed to this relationship").build(); + +public static final Relationship REL_HITS = new Relationship.Builder().name("hits") +.description("Search hits are routed to this relationship.") +.build(); + +public static final Relationship REL_AGGREGATIONS = new Relationship.Builder().name("aggregations") +.description("Aggregations are routed to this relationship.") +.build(); + +public static final PropertyDescriptor INDEX = new PropertyDescriptor.Builder() +.name("el5-fetch-index") +.displayName("Index") +.description("The name of the index to read from") +.required(true) +.expressionLanguageSupported(true) +.addValidator(StandardValidators.NON_EMPTY_VALIDATOR) +.build(); + +public static final PropertyDescriptor TYPE = new PropertyDescriptor.Builder() +.name("el5-type") +.displayName("Type") +.description("The type of this document (used by Elasticsearch for indexing and searching)") +
[jira] [Commented] (NIFI-4325) Create a new ElasticSearch processor that supports the JSON DSL
[ https://issues.apache.org/jira/browse/NIFI-4325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246531#comment-16246531 ] ASF GitHub Bot commented on NIFI-4325: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/2113#discussion_r150087242 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-5-processors/src/main/java/org/apache/nifi/processors/elasticsearch/JsonQueryElasticsearch5.java --- @@ -0,0 +1,291 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.nifi.processors.elasticsearch; + +import com.fasterxml.jackson.databind.ObjectMapper; +import org.apache.nifi.annotation.behavior.EventDriven; +import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.annotation.lifecycle.OnUnscheduled; +import org.apache.nifi.components.AllowableValue; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.components.Validator; +import org.apache.nifi.elasticsearch.ElasticSearchClientService; +import org.apache.nifi.elasticsearch.ElasticSearchResponse; +import org.apache.nifi.flowfile.FlowFile; +import org.apache.nifi.processor.AbstractProcessor; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.ProcessSession; +import org.apache.nifi.processor.Relationship; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.io.OutputStreamCallback; +import org.apache.nifi.processor.util.StandardValidators; + +import java.io.IOException; +import java.io.OutputStream; +import java.util.ArrayList; +import java.util.Collections; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Optional; +import java.util.Set; + + +@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED) +@EventDriven +@Tags({"elasticsearch", "elasticsearch 5", "query", "read", "get", "json"}) +@CapabilityDescription("A processor that allows the user to run a query (with aggregations) written with the " + +"ElasticSearch JSON DSL. It currently does not support pagination.") +public class JsonQueryElasticsearch5 extends AbstractProcessor { +public static final Relationship REL_SUCCESS = new Relationship.Builder().name("success") +.description("All original flowfiles that don't cause an error to occur go to this relationship. " + +"This applies even if you select the \"break up hits\" option to send individual hits to the " + +"\"hits\" relationship.").build(); + +public static final Relationship REL_FAILURE = new Relationship.Builder().name("failure") +.description("All FlowFiles that cannot be read from Elasticsearch are routed to this relationship").build(); + +public static final Relationship REL_HITS = new Relationship.Builder().name("hits") +.description("Search hits are routed to this relationship.") +.build(); + +public static final Relationship REL_AGGREGATIONS = new Relationship.Builder().name("aggregations") +.description("Aggregations are routed to this relationship.") +.build(); + +public static final PropertyDescriptor INDEX = new PropertyDescriptor.Builder() +.name("el5-fetch-index") +.displayName("Index") +.description("The name of the index to read from") +.required(true) +.expressionLanguageSupported(true) +.addValidator(StandardValidators.NON_EMPTY_VALIDATOR) +.build(); + +public static final PropertyDescriptor
[jira] [Commented] (NIFI-4402) Add component location in Summary view
[ https://issues.apache.org/jira/browse/NIFI-4402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246520#comment-16246520 ] ASF GitHub Bot commented on NIFI-4402: -- Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/2195 @yuri1969 @pvillard31 Ugh sorry guys. I totally forgot about this. I've been swamped with some other things at the moment. I remember jumping on this PR because I wanted to verify the case when a user has doesn't have permissions to one of the groups in the chain of ancestors. If someone else wants to verify this case, please do. Otherwise, I can hopefully sometime soon. > Add component location in Summary view > -- > > Key: NIFI-4402 > URL: https://issues.apache.org/jira/browse/NIFI-4402 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Pierre Villard >Priority: Minor > > In the summary view, when there are hundreds of processors, it can be > difficult to know what exact processor we are looking at in the table. The > only options are: > - go to the processor location to know what processor it is (but then we are > not on the summary view anymore) > - have a naming convention to uniquely name every component > To improve the user experience, the following could be implemented: > - add a column "Process Group" displaying the name of the parent process > group containing the component > - when hovering the "Go to location" button change the tooltip to include the > path of the component. Something like: "Go to /NiFi Flow/My RPG/My Processor" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #2113: NIFI-4325 Added new processor that uses the JSON DS...
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/2113#discussion_r150087242 --- Diff: nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-5-processors/src/main/java/org/apache/nifi/processors/elasticsearch/JsonQueryElasticsearch5.java --- @@ -0,0 +1,291 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.nifi.processors.elasticsearch; + +import com.fasterxml.jackson.databind.ObjectMapper; +import org.apache.nifi.annotation.behavior.EventDriven; +import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.annotation.lifecycle.OnUnscheduled; +import org.apache.nifi.components.AllowableValue; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.components.Validator; +import org.apache.nifi.elasticsearch.ElasticSearchClientService; +import org.apache.nifi.elasticsearch.ElasticSearchResponse; +import org.apache.nifi.flowfile.FlowFile; +import org.apache.nifi.processor.AbstractProcessor; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.ProcessSession; +import org.apache.nifi.processor.Relationship; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.io.OutputStreamCallback; +import org.apache.nifi.processor.util.StandardValidators; + +import java.io.IOException; +import java.io.OutputStream; +import java.util.ArrayList; +import java.util.Collections; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Optional; +import java.util.Set; + + +@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED) +@EventDriven +@Tags({"elasticsearch", "elasticsearch 5", "query", "read", "get", "json"}) +@CapabilityDescription("A processor that allows the user to run a query (with aggregations) written with the " + +"ElasticSearch JSON DSL. It currently does not support pagination.") +public class JsonQueryElasticsearch5 extends AbstractProcessor { +public static final Relationship REL_SUCCESS = new Relationship.Builder().name("success") +.description("All original flowfiles that don't cause an error to occur go to this relationship. " + +"This applies even if you select the \"break up hits\" option to send individual hits to the " + +"\"hits\" relationship.").build(); + +public static final Relationship REL_FAILURE = new Relationship.Builder().name("failure") +.description("All FlowFiles that cannot be read from Elasticsearch are routed to this relationship").build(); + +public static final Relationship REL_HITS = new Relationship.Builder().name("hits") +.description("Search hits are routed to this relationship.") +.build(); + +public static final Relationship REL_AGGREGATIONS = new Relationship.Builder().name("aggregations") +.description("Aggregations are routed to this relationship.") +.build(); + +public static final PropertyDescriptor INDEX = new PropertyDescriptor.Builder() +.name("el5-fetch-index") +.displayName("Index") +.description("The name of the index to read from") +.required(true) +.expressionLanguageSupported(true) +.addValidator(StandardValidators.NON_EMPTY_VALIDATOR) +.build(); + +public static final PropertyDescriptor TYPE = new PropertyDescriptor.Builder() +.name("el5-type") +.displayName("Type") +.description("The type of this document (used by Elasticsearch for indexing and searching)") +
[GitHub] nifi pull request #2113: NIFI-4325 Added new processor that uses the JSON DS...
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/2113#discussion_r150086797 --- Diff: nifi-assembly/pom.xml --- @@ -338,6 +338,16 @@ org.apache.nifi +nifi-elasticsearch-client-service-nar +nar +
[jira] [Commented] (NIFI-4402) Add component location in Summary view
[ https://issues.apache.org/jira/browse/NIFI-4402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246496#comment-16246496 ] ASF GitHub Bot commented on NIFI-4402: -- Github user yuri1969 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2195#discussion_r150084382 --- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/summary/nf-summary-table.js --- @@ -417,7 +426,7 @@ var markup = ''; if (isInShell) { -markup += ''; +markup += ''; --- End diff -- I've pushed the change. Thanks @pvillard31. > Add component location in Summary view > -- > > Key: NIFI-4402 > URL: https://issues.apache.org/jira/browse/NIFI-4402 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Pierre Villard >Priority: Minor > > In the summary view, when there are hundreds of processors, it can be > difficult to know what exact processor we are looking at in the table. The > only options are: > - go to the processor location to know what processor it is (but then we are > not on the summary view anymore) > - have a naming convention to uniquely name every component > To improve the user experience, the following could be implemented: > - add a column "Process Group" displaying the name of the parent process > group containing the component > - when hovering the "Go to location" button change the tooltip to include the > path of the component. Something like: "Go to /NiFi Flow/My RPG/My Processor" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #2195: NIFI-4402 - Add component location in Summary view
Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/2195 @yuri1969 @pvillard31 Ugh sorry guys. I totally forgot about this. I've been swamped with some other things at the moment. I remember jumping on this PR because I wanted to verify the case when a user has doesn't have permissions to one of the groups in the chain of ancestors. If someone else wants to verify this case, please do. Otherwise, I can hopefully sometime soon. ---
[GitHub] nifi pull request #2195: NIFI-4402 - Add component location in Summary view
Github user yuri1969 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2195#discussion_r150084382 --- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/summary/nf-summary-table.js --- @@ -417,7 +426,7 @@ var markup = ''; if (isInShell) { -markup += ''; +markup += ''; --- End diff -- I've pushed the change. Thanks @pvillard31. ---
[jira] [Commented] (NIFI-4593) NoClassDefFoundError when using HortonworksSchemaRegistry
[ https://issues.apache.org/jira/browse/NIFI-4593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246451#comment-16246451 ] ASF GitHub Bot commented on NIFI-4593: -- GitHub user mcgilman opened a pull request: https://github.com/apache/nifi/pull/2263 NIFI-4593: Ensuring jackson dependencies are bundled NIFI-4593: - Ensuring the necessary jackson dependencies are bundled. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mcgilman/nifi NIFI-4593 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2263.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2263 commit 60fe4b0d91a252a38d40749f80dceef9ceed3918 Author: Matt GilmanDate: 2017-11-09T20:21:00Z NIFI-4593: - Ensuring the necessary jackson dependencies are bundled. > NoClassDefFoundError when using HortonworksSchemaRegistry > - > > Key: NIFI-4593 > URL: https://issues.apache.org/jira/browse/NIFI-4593 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.4.0 >Reporter: Matt Gilman >Assignee: Matt Gilman > > This appears to be a regression from the introduction of the newer Jersey > (NIFI-) or Jackson (NIFI-4297) dependency and their transitive > dependencies. > {code} > java.lang.NoClassDefFoundError: com/fasterxml/jackson/core/TreeNode > at > org.apache.nifi.schemaregistry.hortonworks.HortonworksSchemaRegistry.getClient(HortonworksSchemaRegistry.java:144) > at > org.apache.nifi.schemaregistry.hortonworks.HortonworksSchemaRegistry.retrieveSchema(HortonworksSchemaRegistry.java:228) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:497) > at > org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:89) > at com.sun.proxy.$Proxy113.retrieveSchema(Unknown Source) > at > org.apache.nifi.schema.access.SchemaNamePropertyStrategy.getSchema(SchemaNamePropertyStrategy.java:53) > at > org.apache.nifi.serialization.SchemaRegistryService.getSchema(SchemaRegistryService.java:117) > at > org.apache.nifi.csv.CSVReader.createRecordReader(CSVReader.java:108) > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4593) NoClassDefFoundError when using HortonworksSchemaRegistry
[ https://issues.apache.org/jira/browse/NIFI-4593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-4593: -- Affects Version/s: 1.4.0 Status: Patch Available (was: In Progress) > NoClassDefFoundError when using HortonworksSchemaRegistry > - > > Key: NIFI-4593 > URL: https://issues.apache.org/jira/browse/NIFI-4593 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.4.0 >Reporter: Matt Gilman >Assignee: Matt Gilman > > This appears to be a regression from the introduction of the newer Jersey > (NIFI-) or Jackson (NIFI-4297) dependency and their transitive > dependencies. > {code} > java.lang.NoClassDefFoundError: com/fasterxml/jackson/core/TreeNode > at > org.apache.nifi.schemaregistry.hortonworks.HortonworksSchemaRegistry.getClient(HortonworksSchemaRegistry.java:144) > at > org.apache.nifi.schemaregistry.hortonworks.HortonworksSchemaRegistry.retrieveSchema(HortonworksSchemaRegistry.java:228) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:497) > at > org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:89) > at com.sun.proxy.$Proxy113.retrieveSchema(Unknown Source) > at > org.apache.nifi.schema.access.SchemaNamePropertyStrategy.getSchema(SchemaNamePropertyStrategy.java:53) > at > org.apache.nifi.serialization.SchemaRegistryService.getSchema(SchemaRegistryService.java:117) > at > org.apache.nifi.csv.CSVReader.createRecordReader(CSVReader.java:108) > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #2263: NIFI-4593: Ensuring jackson dependencies are bundle...
GitHub user mcgilman opened a pull request: https://github.com/apache/nifi/pull/2263 NIFI-4593: Ensuring jackson dependencies are bundled NIFI-4593: - Ensuring the necessary jackson dependencies are bundled. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mcgilman/nifi NIFI-4593 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2263.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2263 commit 60fe4b0d91a252a38d40749f80dceef9ceed3918 Author: Matt GilmanDate: 2017-11-09T20:21:00Z NIFI-4593: - Ensuring the necessary jackson dependencies are bundled. ---
[jira] [Commented] (NIFI-4072) remove usage of RELEASE from dependency version in pom.xml
[ https://issues.apache.org/jira/browse/NIFI-4072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246414#comment-16246414 ] ASF GitHub Bot commented on NIFI-4072: -- Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/2258 Confirmed that's the only place to update and confirmed full build is OK. Merging to master, thanks @markobean > remove usage of RELEASE from dependency version in pom.xml > -- > > Key: NIFI-4072 > URL: https://issues.apache.org/jira/browse/NIFI-4072 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.3.0 >Reporter: Mark Bean >Assignee: Mark Bean > Fix For: 1.5.0 > > > There are several instances where a dependency version is identified as > "RELEASE" in the pom.xml. This should be replaced by a specific version for > reproducible build results. The following pom.xml have been identified as > containing RELEASE in the current master branch: > nifi-nar-bundles/nifi-cybersecurity-bundle/nifi-cybersecurity-processors/pom.xml > nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/pom.xml > nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/pom.xml > nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-security/pom.xml > > nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/pom.xml > nifi-nar-bundles/nifi-spring/bundle/nifi-spring-processors/pom.xml > nifi-toolkit/nifi-toolkit-admin/pom.xml > pom.xml -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4072) remove usage of RELEASE from dependency version in pom.xml
[ https://issues.apache.org/jira/browse/NIFI-4072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246417#comment-16246417 ] ASF GitHub Bot commented on NIFI-4072: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2258 > remove usage of RELEASE from dependency version in pom.xml > -- > > Key: NIFI-4072 > URL: https://issues.apache.org/jira/browse/NIFI-4072 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.3.0 >Reporter: Mark Bean >Assignee: Mark Bean > Fix For: 1.5.0 > > > There are several instances where a dependency version is identified as > "RELEASE" in the pom.xml. This should be replaced by a specific version for > reproducible build results. The following pom.xml have been identified as > containing RELEASE in the current master branch: > nifi-nar-bundles/nifi-cybersecurity-bundle/nifi-cybersecurity-processors/pom.xml > nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/pom.xml > nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/pom.xml > nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-security/pom.xml > > nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/pom.xml > nifi-nar-bundles/nifi-spring/bundle/nifi-spring-processors/pom.xml > nifi-toolkit/nifi-toolkit-admin/pom.xml > pom.xml -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4072) remove usage of RELEASE from dependency version in pom.xml
[ https://issues.apache.org/jira/browse/NIFI-4072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246415#comment-16246415 ] ASF subversion and git services commented on NIFI-4072: --- Commit f11e03886f51a570c05c7936e9472a7b9e643ba0 in nifi's branch refs/heads/master from [~markbean] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=f11e038 ] NIFI-4072 replace keyword RELEASE with actual version number Signed-off-by: Pierre VillardThis closes #2258. > remove usage of RELEASE from dependency version in pom.xml > -- > > Key: NIFI-4072 > URL: https://issues.apache.org/jira/browse/NIFI-4072 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.3.0 >Reporter: Mark Bean >Assignee: Mark Bean > Fix For: 1.5.0 > > > There are several instances where a dependency version is identified as > "RELEASE" in the pom.xml. This should be replaced by a specific version for > reproducible build results. The following pom.xml have been identified as > containing RELEASE in the current master branch: > nifi-nar-bundles/nifi-cybersecurity-bundle/nifi-cybersecurity-processors/pom.xml > nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/pom.xml > nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/pom.xml > nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-security/pom.xml > > nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/pom.xml > nifi-nar-bundles/nifi-spring/bundle/nifi-spring-processors/pom.xml > nifi-toolkit/nifi-toolkit-admin/pom.xml > pom.xml -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #2258: NIFI-4072 replace keyword RELEASE with actual versi...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2258 ---
[GitHub] nifi issue #2258: NIFI-4072 replace keyword RELEASE with actual version numb...
Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/2258 Confirmed that's the only place to update and confirmed full build is OK. Merging to master, thanks @markobean ---
[jira] [Commented] (NIFI-4402) Add component location in Summary view
[ https://issues.apache.org/jira/browse/NIFI-4402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246379#comment-16246379 ] ASF GitHub Bot commented on NIFI-4402: -- Github user pvillard31 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2195#discussion_r150066467 --- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/summary/nf-summary-table.js --- @@ -417,7 +426,7 @@ var markup = ''; if (isInShell) { -markup += ''; +markup += ''; --- End diff -- I'd suggest: Go To Processor in instead of: Go To Processor Or, otherwise, the path should contain the processor name itself but I feel like it would be redundant. > Add component location in Summary view > -- > > Key: NIFI-4402 > URL: https://issues.apache.org/jira/browse/NIFI-4402 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Pierre Villard >Priority: Minor > > In the summary view, when there are hundreds of processors, it can be > difficult to know what exact processor we are looking at in the table. The > only options are: > - go to the processor location to know what processor it is (but then we are > not on the summary view anymore) > - have a naming convention to uniquely name every component > To improve the user experience, the following could be implemented: > - add a column "Process Group" displaying the name of the parent process > group containing the component > - when hovering the "Go to location" button change the tooltip to include the > path of the component. Something like: "Go to /NiFi Flow/My RPG/My Processor" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #2195: NIFI-4402 - Add component location in Summary view
Github user pvillard31 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2195#discussion_r150066467 --- Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/summary/nf-summary-table.js --- @@ -417,7 +426,7 @@ var markup = ''; if (isInShell) { -markup += ''; +markup += ''; --- End diff -- I'd suggest: Go To Processor in instead of: Go To Processor Or, otherwise, the path should contain the processor name itself but I feel like it would be redundant. ---
[jira] [Updated] (NIFI-4593) NoClassDefFoundError when using HortonworksSchemaRegistry
[ https://issues.apache.org/jira/browse/NIFI-4593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-4593: -- Summary: NoClassDefFoundError when using HortonworksSchemaRegistry (was: ClassNotFound when using HortonworksSchemaRegistry) > NoClassDefFoundError when using HortonworksSchemaRegistry > - > > Key: NIFI-4593 > URL: https://issues.apache.org/jira/browse/NIFI-4593 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Gilman >Assignee: Matt Gilman > > This appears to be a regression from the introduction of the newer Jersey > (NIFI-) or Jackson (NIFI-4297) dependency and their transitive > dependencies. > {code} > java.lang.NoClassDefFoundError: com/fasterxml/jackson/core/TreeNode > at > org.apache.nifi.schemaregistry.hortonworks.HortonworksSchemaRegistry.getClient(HortonworksSchemaRegistry.java:144) > at > org.apache.nifi.schemaregistry.hortonworks.HortonworksSchemaRegistry.retrieveSchema(HortonworksSchemaRegistry.java:228) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:497) > at > org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:89) > at com.sun.proxy.$Proxy113.retrieveSchema(Unknown Source) > at > org.apache.nifi.schema.access.SchemaNamePropertyStrategy.getSchema(SchemaNamePropertyStrategy.java:53) > at > org.apache.nifi.serialization.SchemaRegistryService.getSchema(SchemaRegistryService.java:117) > at > org.apache.nifi.csv.CSVReader.createRecordReader(CSVReader.java:108) > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #2261: NIFI-4587: Fixing newlines in comment fields
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2261 ---
[jira] [Commented] (NIFI-4587) no newline in processor/PG comments tab when using Firefox browser,
[ https://issues.apache.org/jira/browse/NIFI-4587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246349#comment-16246349 ] ASF GitHub Bot commented on NIFI-4587: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2261 > no newline in processor/PG comments tab when using Firefox browser, > > > Key: NIFI-4587 > URL: https://issues.apache.org/jira/browse/NIFI-4587 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.3.0 >Reporter: Haimo Liu >Assignee: Matt Gilman > Fix For: 1.5.0 > > > (Firefox) when editing comments in processors or process groups, the > "comment" tab does not properly handle newlines (CR/LF). However we can see > correct newlines in Chrome. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4587) no newline in processor/PG comments tab when using Firefox browser,
[ https://issues.apache.org/jira/browse/NIFI-4587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246346#comment-16246346 ] ASF subversion and git services commented on NIFI-4587: --- Commit 9ecb26a52b3caa5ae1a3962ccd5582752a619324 in nifi's branch refs/heads/master from [~mcgilman] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=9ecb26a ] NIFI-4587 - Updating styles to allow for newlines in Process Group, Processor, Controller Service, and Reporting Task comments. Signed-off-by: Pierre VillardThis closes #2261. > no newline in processor/PG comments tab when using Firefox browser, > > > Key: NIFI-4587 > URL: https://issues.apache.org/jira/browse/NIFI-4587 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.3.0 >Reporter: Haimo Liu >Assignee: Matt Gilman > Fix For: 1.5.0 > > > (Firefox) when editing comments in processors or process groups, the > "comment" tab does not properly handle newlines (CR/LF). However we can see > correct newlines in Chrome. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4587) no newline in processor/PG comments tab when using Firefox browser,
[ https://issues.apache.org/jira/browse/NIFI-4587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-4587: - Resolution: Fixed Fix Version/s: 1.5.0 Status: Resolved (was: Patch Available) > no newline in processor/PG comments tab when using Firefox browser, > > > Key: NIFI-4587 > URL: https://issues.apache.org/jira/browse/NIFI-4587 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.3.0 >Reporter: Haimo Liu >Assignee: Matt Gilman > Fix For: 1.5.0 > > > (Firefox) when editing comments in processors or process groups, the > "comment" tab does not properly handle newlines (CR/LF). However we can see > correct newlines in Chrome. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4587) no newline in processor/PG comments tab when using Firefox browser,
[ https://issues.apache.org/jira/browse/NIFI-4587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246345#comment-16246345 ] ASF GitHub Bot commented on NIFI-4587: -- Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/2261 Reproduced the issue before the fix, and confirmed expected behaviour (Firefox, Chrome, Safari) once the fix is applied. LGTM, merging to master, thanks @mcgilman > no newline in processor/PG comments tab when using Firefox browser, > > > Key: NIFI-4587 > URL: https://issues.apache.org/jira/browse/NIFI-4587 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.3.0 >Reporter: Haimo Liu >Assignee: Matt Gilman > Fix For: 1.5.0 > > > (Firefox) when editing comments in processors or process groups, the > "comment" tab does not properly handle newlines (CR/LF). However we can see > correct newlines in Chrome. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #2261: NIFI-4587: Fixing newlines in comment fields
Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/2261 Reproduced the issue before the fix, and confirmed expected behaviour (Firefox, Chrome, Safari) once the fix is applied. LGTM, merging to master, thanks @mcgilman ---
[jira] [Created] (NIFI-4593) ClassNotFound when using HortonworksSchemaRegistry
Matt Gilman created NIFI-4593: - Summary: ClassNotFound when using HortonworksSchemaRegistry Key: NIFI-4593 URL: https://issues.apache.org/jira/browse/NIFI-4593 Project: Apache NiFi Issue Type: Bug Components: Extensions Reporter: Matt Gilman Assignee: Matt Gilman This appears to be a regression from the introduction of the newer Jersey (NIFI-) or Jackson (NIFI-4297) dependency and their transitive dependencies. {code} java.lang.NoClassDefFoundError: com/fasterxml/jackson/core/TreeNode at org.apache.nifi.schemaregistry.hortonworks.HortonworksSchemaRegistry.getClient(HortonworksSchemaRegistry.java:144) at org.apache.nifi.schemaregistry.hortonworks.HortonworksSchemaRegistry.retrieveSchema(HortonworksSchemaRegistry.java:228) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:89) at com.sun.proxy.$Proxy113.retrieveSchema(Unknown Source) at org.apache.nifi.schema.access.SchemaNamePropertyStrategy.getSchema(SchemaNamePropertyStrategy.java:53) at org.apache.nifi.serialization.SchemaRegistryService.getSchema(SchemaRegistryService.java:117) at org.apache.nifi.csv.CSVReader.createRecordReader(CSVReader.java:108) {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (NIFI-4586) Wrong URI can be created by ApplicationResource when X-Proxy and X-Forwarded headers exist in the request
[ https://issues.apache.org/jira/browse/NIFI-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Storck reassigned NIFI-4586: - Assignee: Jeff Storck > Wrong URI can be created by ApplicationResource when X-Proxy and X-Forwarded > headers exist in the request > - > > Key: NIFI-4586 > URL: https://issues.apache.org/jira/browse/NIFI-4586 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.4.0 >Reporter: Jeff Storck >Assignee: Jeff Storck >Priority: Critical > > While testing support for Knox proxying to a secure NiFi cluster, I was > unable to modify/move components after creating them. This is being caused > by behavior in ApplicationResource#generateResourceUri, which attempts to get > the first header value in the request between the X-Proxy* and X-Forwarded* > headers. > If the request has both sets of headers, the code will prioritize the > X-Proxy* headers, and will generate the URI with those values. Since Knox > sets the X-Forwarded* headers, if the X-Proxy* headers are on the request, > the headers supplied by Knox will be ignored. > The NiFi UI will then attempt to access the URI generated with the X-Proxied* > header values. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (NIFI-4592) Add support for running docker container as a different user than 1000
Thomas Suckow created NIFI-4592: --- Summary: Add support for running docker container as a different user than 1000 Key: NIFI-4592 URL: https://issues.apache.org/jira/browse/NIFI-4592 Project: Apache NiFi Issue Type: Improvement Components: Docker Affects Versions: 1.4.0 Reporter: Thomas Suckow If the docker container is run with the --user flag set to anything other than 1000, nifi will fail to run, even if --add-group 1000 is specified because some folders are only writeable by the nifi user, at the very least the nifi group should be allowed to write to necessary folders. The pid file appears to be the initial troublemaker. This solves issues with shared volumes being exposed to the container and user 1000 not being a valid user on them. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #2239: NIFI-4545: Improve Hive processors provenance transit URL
Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/2239 +1 LGTM, ran build with contrib-check and unit tests, also tried various HiveQL commands, verified the input and output tables attributes are being correctly populated. Thanks for this improvement! Merging to master ---
[jira] [Commented] (NIFI-4545) Improve Hive processors provenance transit URL
[ https://issues.apache.org/jira/browse/NIFI-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246162#comment-16246162 ] ASF GitHub Bot commented on NIFI-4545: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2239 > Improve Hive processors provenance transit URL > -- > > Key: NIFI-4545 > URL: https://issues.apache.org/jira/browse/NIFI-4545 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Koji Kawamura >Assignee: Koji Kawamura > Fix For: 1.5.0 > > > Hive-related processors report NiFi provenance events with transit URLs in a > format as 'jdbc:hive2://:,:/dbName'. The URL > format can identify a Hive environment, but not descriptive enough to derive > actual table names affecting or being affected by the query which generated > the provenance event. > Those table information can only be known by parsing query. This JIRA > improves following Hive related processors to write additional > 'query.input.tables' and 'query.output.tables' FlowFile attributes by parsing > Hive queries using Hive parser. > Target Processors: > * PutHiveQL > * SelectHiveQL > * PutHiveStreaming: This processor knows a table name without the need of > parsing queries. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4545) Improve Hive processors provenance transit URL
[ https://issues.apache.org/jira/browse/NIFI-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246149#comment-16246149 ] ASF subversion and git services commented on NIFI-4545: --- Commit 37ef2839e00a31c33b13da98dcb7aebe9ea77923 in nifi's branch refs/heads/master from [~ijokarumawak] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=37ef283 ] NIFI-4545: Improve Hive processors provenance transit URL Incorporated review comments: - Added 'input' to equals() method so that the same table name can appear as input and output tables. Signed-off-by: Matthew BurgessThis closes #2239 > Improve Hive processors provenance transit URL > -- > > Key: NIFI-4545 > URL: https://issues.apache.org/jira/browse/NIFI-4545 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Koji Kawamura >Assignee: Koji Kawamura > Fix For: 1.5.0 > > > Hive-related processors report NiFi provenance events with transit URLs in a > format as 'jdbc:hive2://:,:/dbName'. The URL > format can identify a Hive environment, but not descriptive enough to derive > actual table names affecting or being affected by the query which generated > the provenance event. > Those table information can only be known by parsing query. This JIRA > improves following Hive related processors to write additional > 'query.input.tables' and 'query.output.tables' FlowFile attributes by parsing > Hive queries using Hive parser. > Target Processors: > * PutHiveQL > * SelectHiveQL > * PutHiveStreaming: This processor knows a table name without the need of > parsing queries. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4545) Improve Hive processors provenance transit URL
[ https://issues.apache.org/jira/browse/NIFI-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-4545: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Improve Hive processors provenance transit URL > -- > > Key: NIFI-4545 > URL: https://issues.apache.org/jira/browse/NIFI-4545 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Koji Kawamura >Assignee: Koji Kawamura > Fix For: 1.5.0 > > > Hive-related processors report NiFi provenance events with transit URLs in a > format as 'jdbc:hive2://:,:/dbName'. The URL > format can identify a Hive environment, but not descriptive enough to derive > actual table names affecting or being affected by the query which generated > the provenance event. > Those table information can only be known by parsing query. This JIRA > improves following Hive related processors to write additional > 'query.input.tables' and 'query.output.tables' FlowFile attributes by parsing > Hive queries using Hive parser. > Target Processors: > * PutHiveQL > * SelectHiveQL > * PutHiveStreaming: This processor knows a table name without the need of > parsing queries. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #2239: NIFI-4545: Improve Hive processors provenance trans...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2239 ---
[jira] [Updated] (NIFI-4545) Improve Hive processors provenance transit URL
[ https://issues.apache.org/jira/browse/NIFI-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-4545: --- Fix Version/s: 1.5.0 > Improve Hive processors provenance transit URL > -- > > Key: NIFI-4545 > URL: https://issues.apache.org/jira/browse/NIFI-4545 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Koji Kawamura >Assignee: Koji Kawamura > Fix For: 1.5.0 > > > Hive-related processors report NiFi provenance events with transit URLs in a > format as 'jdbc:hive2://:,:/dbName'. The URL > format can identify a Hive environment, but not descriptive enough to derive > actual table names affecting or being affected by the query which generated > the provenance event. > Those table information can only be known by parsing query. This JIRA > improves following Hive related processors to write additional > 'query.input.tables' and 'query.output.tables' FlowFile attributes by parsing > Hive queries using Hive parser. > Target Processors: > * PutHiveQL > * SelectHiveQL > * PutHiveStreaming: This processor knows a table name without the need of > parsing queries. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4545) Improve Hive processors provenance transit URL
[ https://issues.apache.org/jira/browse/NIFI-4545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246146#comment-16246146 ] ASF GitHub Bot commented on NIFI-4545: -- Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/2239 +1 LGTM, ran build with contrib-check and unit tests, also tried various HiveQL commands, verified the input and output tables attributes are being correctly populated. Thanks for this improvement! Merging to master > Improve Hive processors provenance transit URL > -- > > Key: NIFI-4545 > URL: https://issues.apache.org/jira/browse/NIFI-4545 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Koji Kawamura >Assignee: Koji Kawamura > > Hive-related processors report NiFi provenance events with transit URLs in a > format as 'jdbc:hive2://:,:/dbName'. The URL > format can identify a Hive environment, but not descriptive enough to derive > actual table names affecting or being affected by the query which generated > the provenance event. > Those table information can only be known by parsing query. This JIRA > improves following Hive related processors to write additional > 'query.input.tables' and 'query.output.tables' FlowFile attributes by parsing > Hive queries using Hive parser. > Target Processors: > * PutHiveQL > * SelectHiveQL > * PutHiveStreaming: This processor knows a table name without the need of > parsing queries. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4591) Controller Service Details button missing
[ https://issues.apache.org/jira/browse/NIFI-4591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-4591: -- Affects Version/s: 1.4.0 Status: Patch Available (was: In Progress) > Controller Service Details button missing > - > > Key: NIFI-4591 > URL: https://issues.apache.org/jira/browse/NIFI-4591 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.4.0 >Reporter: Matt Gilman >Assignee: Matt Gilman > > Following the modification in NIFI-3781 a regression was introduced that > prevents the rendering of the View Details button when the user has read > permissions but no write permissions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4591) Controller Service Details button missing
[ https://issues.apache.org/jira/browse/NIFI-4591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246144#comment-16246144 ] ASF GitHub Bot commented on NIFI-4591: -- GitHub user mcgilman opened a pull request: https://github.com/apache/nifi/pull/2262 NIFI-4591: Ensure View Details is button is visible when appropriate NIFI-4591: - Ensuring View Details button is visible if the user has read access and the service is defined in the current Process Group. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mcgilman/nifi NIFI-4591 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2262.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2262 commit b5b6f4c8c14fc88508f5771cfded2637a9910d48 Author: Matt GilmanDate: 2017-11-09T17:48:42Z NIFI-4591: - Ensuring View Details button is visible if the user has read access and the service is defined in the current Process Group. > Controller Service Details button missing > - > > Key: NIFI-4591 > URL: https://issues.apache.org/jira/browse/NIFI-4591 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman > > Following the modification in NIFI-3781 a regression was introduced that > prevents the rendering of the View Details button when the user has read > permissions but no write permissions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #2262: NIFI-4591: Ensure View Details is button is visible...
GitHub user mcgilman opened a pull request: https://github.com/apache/nifi/pull/2262 NIFI-4591: Ensure View Details is button is visible when appropriate NIFI-4591: - Ensuring View Details button is visible if the user has read access and the service is defined in the current Process Group. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mcgilman/nifi NIFI-4591 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2262.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2262 commit b5b6f4c8c14fc88508f5771cfded2637a9910d48 Author: Matt GilmanDate: 2017-11-09T17:48:42Z NIFI-4591: - Ensuring View Details button is visible if the user has read access and the service is defined in the current Process Group. ---
[jira] [Commented] (MINIFICPP-33) Create GetGPS processor for acquiring GPS coordinates
[ https://issues.apache.org/jira/browse/MINIFICPP-33?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246120#comment-16246120 ] ASF GitHub Bot commented on MINIFICPP-33: - Github user minifirocks commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/172 @phrocker looks good, please merge the two commits into one. > Create GetGPS processor for acquiring GPS coordinates > - > > Key: MINIFICPP-33 > URL: https://issues.apache.org/jira/browse/MINIFICPP-33 > Project: NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Jeremy Dyer >Assignee: Jeremy Dyer > > GPSD is a popular framework for interacting with a multitude of GPS devices. > It drastically simplifies the interaction with vendor specific GPS devices by > providing a daemon service which communicates with the device, converts the > raw NMEA 0183 sentences into JSON objects, and then emits those JSON objects > over a socket for 0-N downstream devices to consume. > This feature would create a GetGPS processor that would listen to a running > instance of GPSD as one of those downstream consumers. The processor would > provide integration with the GPSD daemon to accept the JSON objects and > create new flowfiles for each of the JSON objects received. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi-minifi-cpp issue #172: MINIFI-218: GetGPS processor implementation
Github user minifirocks commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/172 @phrocker looks good, please merge the two commits into one. ---
[jira] [Commented] (NIFI-4587) no newline in processor/PG comments tab when using Firefox browser,
[ https://issues.apache.org/jira/browse/NIFI-4587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246095#comment-16246095 ] ASF GitHub Bot commented on NIFI-4587: -- GitHub user mcgilman opened a pull request: https://github.com/apache/nifi/pull/2261 NIFI-4587: Fixing newlines in comment fields NIFI-4587: - Updating styles to allow for newlines in Process Group, Processor, Controller Service, and Reporting Task comments. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mcgilman/nifi NIFI-4587 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2261.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2261 commit 8153aef234152f6845e71c60f5beab023a6b8426 Author: Matt GilmanDate: 2017-11-09T17:23:30Z NIFI-4587: - Updating styles to allow for newlines in Process Group, Processor, Controller Service, and Reporting Task comments. > no newline in processor/PG comments tab when using Firefox browser, > > > Key: NIFI-4587 > URL: https://issues.apache.org/jira/browse/NIFI-4587 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.3.0 >Reporter: Haimo Liu >Assignee: Matt Gilman > > (Firefox) when editing comments in processors or process groups, the > "comment" tab does not properly handle newlines (CR/LF). However we can see > correct newlines in Chrome. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4587) no newline in processor/PG comments tab when using Firefox browser,
[ https://issues.apache.org/jira/browse/NIFI-4587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-4587: -- Status: Patch Available (was: In Progress) > no newline in processor/PG comments tab when using Firefox browser, > > > Key: NIFI-4587 > URL: https://issues.apache.org/jira/browse/NIFI-4587 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.3.0 >Reporter: Haimo Liu >Assignee: Matt Gilman > > (Firefox) when editing comments in processors or process groups, the > "comment" tab does not properly handle newlines (CR/LF). However we can see > correct newlines in Chrome. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #2261: NIFI-4587: Fixing newlines in comment fields
GitHub user mcgilman opened a pull request: https://github.com/apache/nifi/pull/2261 NIFI-4587: Fixing newlines in comment fields NIFI-4587: - Updating styles to allow for newlines in Process Group, Processor, Controller Service, and Reporting Task comments. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mcgilman/nifi NIFI-4587 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2261.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2261 commit 8153aef234152f6845e71c60f5beab023a6b8426 Author: Matt GilmanDate: 2017-11-09T17:23:30Z NIFI-4587: - Updating styles to allow for newlines in Process Group, Processor, Controller Service, and Reporting Task comments. ---
[jira] [Created] (NIFI-4591) Controller Service Details button missing
Matt Gilman created NIFI-4591: - Summary: Controller Service Details button missing Key: NIFI-4591 URL: https://issues.apache.org/jira/browse/NIFI-4591 Project: Apache NiFi Issue Type: Bug Components: Core UI Reporter: Matt Gilman Assignee: Matt Gilman Following the modification in NIFI-3781 a regression was introduced that prevents the rendering of the View Details button when the user has read permissions but no write permissions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFICPP-250) Create a packet capture process
[ https://issues.apache.org/jira/browse/MINIFICPP-250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245991#comment-16245991 ] ASF GitHub Bot commented on MINIFICPP-250: -- Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/170#discussion_r150017984 --- Diff: .travis.yml --- @@ -84,4 +84,4 @@ matrix: - package='libpng'; [[ $(brew ls --versions ${package}) ]] && { brew outdated ${package} || brew upgrade ${package}; } || brew install ${package} script: - - mkdir ./build && cd ./build && cmake .. ${CMAKE_BUILD_OPTIONS} && make -j2 VERBOSE=1 && make test ARGS="-j2 --output-on-failure" && make linter && make apache-rat && make docs + - mkdir ./build && cd ./build && cmake -DENABLE_PCAP=TRUE .. ${CMAKE_BUILD_OPTIONS} && make -j2 VERBOSE=1 && make test ARGS="-j2 --output-on-failure" && make linter && make apache-rat && make docs --- End diff -- This is going to be removed after we show it working in travis. > Create a packet capture process > --- > > Key: MINIFICPP-250 > URL: https://issues.apache.org/jira/browse/MINIFICPP-250 > Project: NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Joseph Witt >Assignee: marco polo > > It would be really powerful to have minificpp agents be able to capture > packets and produce proper pcap bundles for them. This then can be sent via > s2s to a nifi which could use pcap readers and netflow/other writers to > create filtered results, do enrichment, send raw data to kafka/etc... -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi-minifi-cpp pull request #170: MINIFICPP-250: Initial implementation fo ...
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/170#discussion_r150017984 --- Diff: .travis.yml --- @@ -84,4 +84,4 @@ matrix: - package='libpng'; [[ $(brew ls --versions ${package}) ]] && { brew outdated ${package} || brew upgrade ${package}; } || brew install ${package} script: - - mkdir ./build && cd ./build && cmake .. ${CMAKE_BUILD_OPTIONS} && make -j2 VERBOSE=1 && make test ARGS="-j2 --output-on-failure" && make linter && make apache-rat && make docs + - mkdir ./build && cd ./build && cmake -DENABLE_PCAP=TRUE .. ${CMAKE_BUILD_OPTIONS} && make -j2 VERBOSE=1 && make test ARGS="-j2 --output-on-failure" && make linter && make apache-rat && make docs --- End diff -- This is going to be removed after we show it working in travis. ---
[jira] [Commented] (MINIFICPP-250) Create a packet capture process
[ https://issues.apache.org/jira/browse/MINIFICPP-250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245985#comment-16245985 ] ASF GitHub Bot commented on MINIFICPP-250: -- Github user calebj commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/170#discussion_r150017488 --- Diff: .travis.yml --- @@ -84,4 +84,4 @@ matrix: - package='libpng'; [[ $(brew ls --versions ${package}) ]] && { brew outdated ${package} || brew upgrade ${package}; } || brew install ${package} script: - - mkdir ./build && cd ./build && cmake .. ${CMAKE_BUILD_OPTIONS} && make -j2 VERBOSE=1 && make test ARGS="-j2 --output-on-failure" && make linter && make apache-rat && make docs + - mkdir ./build && cd ./build && cmake -DENABLE_PCAP=TRUE .. ${CMAKE_BUILD_OPTIONS} && make -j2 VERBOSE=1 && make test ARGS="-j2 --output-on-failure" && make linter && make apache-rat && make docs --- End diff -- I think this is better off in CMAKE_BUILD_OPTIONS above > Create a packet capture process > --- > > Key: MINIFICPP-250 > URL: https://issues.apache.org/jira/browse/MINIFICPP-250 > Project: NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Joseph Witt >Assignee: marco polo > > It would be really powerful to have minificpp agents be able to capture > packets and produce proper pcap bundles for them. This then can be sent via > s2s to a nifi which could use pcap readers and netflow/other writers to > create filtered results, do enrichment, send raw data to kafka/etc... -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi-minifi-cpp pull request #170: MINIFICPP-250: Initial implementation fo ...
Github user calebj commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/170#discussion_r150017488 --- Diff: .travis.yml --- @@ -84,4 +84,4 @@ matrix: - package='libpng'; [[ $(brew ls --versions ${package}) ]] && { brew outdated ${package} || brew upgrade ${package}; } || brew install ${package} script: - - mkdir ./build && cd ./build && cmake .. ${CMAKE_BUILD_OPTIONS} && make -j2 VERBOSE=1 && make test ARGS="-j2 --output-on-failure" && make linter && make apache-rat && make docs + - mkdir ./build && cd ./build && cmake -DENABLE_PCAP=TRUE .. ${CMAKE_BUILD_OPTIONS} && make -j2 VERBOSE=1 && make test ARGS="-j2 --output-on-failure" && make linter && make apache-rat && make docs --- End diff -- I think this is better off in CMAKE_BUILD_OPTIONS above ---
[jira] [Commented] (MINIFICPP-258) C2NullConfiguration fails due to a segfault on travis.
[ https://issues.apache.org/jira/browse/MINIFICPP-258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245912#comment-16245912 ] marco polo commented on MINIFICPP-258: -- [~calebj] The ticket was regarding a failure which wasn't proven to be addressed. I don't see a reason to close this quite yet since you've only re-enabled the test. I still suspect something may be amiss. Thanks for the contribution nonetheless. > C2NullConfiguration fails due to a segfault on travis. > --- > > Key: MINIFICPP-258 > URL: https://issues.apache.org/jira/browse/MINIFICPP-258 > Project: NiFi MiNiFi C++ > Issue Type: Bug >Reporter: marco polo >Assignee: marco polo > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFIREG-50) Add additional metadata to VersionedFlow and VersionedFlowSnapshotMetadata
[ https://issues.apache.org/jira/browse/NIFIREG-50?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245872#comment-16245872 ] ASF GitHub Bot commented on NIFIREG-50: --- Github user asfgit closed the pull request at: https://github.com/apache/nifi-registry/pull/35 > Add additional metadata to VersionedFlow and VersionedFlowSnapshotMetadata > -- > > Key: NIFIREG-50 > URL: https://issues.apache.org/jira/browse/NIFIREG-50 > Project: NiFi Registry > Issue Type: Improvement >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Minor > > It would be helpful for NiFi if VersionedFlow could include the bucket name, > and if VersionedFlowSnapshotMetadata could include the bucket name and flow > description. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (NIFIREG-50) Add additional metadata to VersionedFlow and VersionedFlowSnapshotMetadata
[ https://issues.apache.org/jira/browse/NIFIREG-50?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne resolved NIFIREG-50. --- Resolution: Fixed Fix Version/s: 0.0.1 > Add additional metadata to VersionedFlow and VersionedFlowSnapshotMetadata > -- > > Key: NIFIREG-50 > URL: https://issues.apache.org/jira/browse/NIFIREG-50 > Project: NiFi Registry > Issue Type: Improvement >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Minor > Fix For: 0.0.1 > > > It would be helpful for NiFi if VersionedFlow could include the bucket name, > and if VersionedFlowSnapshotMetadata could include the bucket name and flow > description. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFIREG-50) Add additional metadata to VersionedFlow and VersionedFlowSnapshotMetadata
[ https://issues.apache.org/jira/browse/NIFIREG-50?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245873#comment-16245873 ] ASF GitHub Bot commented on NIFIREG-50: --- Github user markap14 commented on the issue: https://github.com/apache/nifi-registry/pull/35 @bbende all looks good! +1 merged to master. > Add additional metadata to VersionedFlow and VersionedFlowSnapshotMetadata > -- > > Key: NIFIREG-50 > URL: https://issues.apache.org/jira/browse/NIFIREG-50 > Project: NiFi Registry > Issue Type: Improvement >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Minor > > It would be helpful for NiFi if VersionedFlow could include the bucket name, > and if VersionedFlowSnapshotMetadata could include the bucket name and flow > description. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi-registry issue #35: NIFIREG-50 Adding additional metadata to some objec...
Github user markap14 commented on the issue: https://github.com/apache/nifi-registry/pull/35 @bbende all looks good! +1 merged to master. ---
[GitHub] nifi-registry pull request #35: NIFIREG-50 Adding additional metadata to som...
Github user asfgit closed the pull request at: https://github.com/apache/nifi-registry/pull/35 ---
[jira] [Commented] (NIFI-4428) Implement PutDruid Processor and Controller
[ https://issues.apache.org/jira/browse/NIFI-4428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245854#comment-16245854 ] ASF GitHub Bot commented on NIFI-4428: -- Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2181 @joewitt @vakshorton If there is a newb-friendly way to set up Druid, let me know and I'll try to find some time to jump in. > Implement PutDruid Processor and Controller > --- > > Key: NIFI-4428 > URL: https://issues.apache.org/jira/browse/NIFI-4428 > Project: Apache NiFi > Issue Type: New Feature >Affects Versions: 1.3.0 >Reporter: Vadim Vaks > > Implement a PutDruid Processor and Controller using Tranquility API. This > will enable Nifi to index contents of flow files in Druid. The implementation > should also be able to handle late arriving data (event timestamp points to > Druid indexing task that has closed, segment granularity and grace window > period expired). Late arriving data is typically dropped. Nifi should allow > late arriving data to be diverted to FAILED or DROPPED relationship. That > would allow late arriving data to be stored on HDFS or S3 until a re-indexing > task can merge it into the correct segment in deep storage. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #2181: NIFI-4428: - Implement PutDruid Processor and Controller
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2181 @joewitt @vakshorton If there is a newb-friendly way to set up Druid, let me know and I'll try to find some time to jump in. ---
[jira] [Assigned] (NIFI-4589) Allow multiple keys in FetchDistributedMapCache
[ https://issues.apache.org/jira/browse/NIFI-4589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess reassigned NIFI-4589: -- Assignee: Matt Burgess > Allow multiple keys in FetchDistributedMapCache > --- > > Key: NIFI-4589 > URL: https://issues.apache.org/jira/browse/NIFI-4589 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess > > Currently (1.4.0) FetchDistributedMapCache will look up a single key and > place the value (if present) in either the flow file content or a user-named > attribute, depending on the value of the "Put Cache Value In Attribute" > property. If the user wishes to look up more than one value, they need > multiple FetchDistributedMapCache processors, and each would make one call to > the server to get a single key. > A useful improvement would be to allow multiple keys to be retrieved at once > by FetchDistributedMapCache. This would likely involve the following: > 1) Update documentation and code to accept a comma-separated list of Cache > Key Identifiers > 2) If a single Cache Key Identifier is specified, current behavior is > retained (i.e. output to flow file content or a specified attribute) > 3) If multiple Cache Key Identifiers are specified, then Put Cache Value In > Attribute must be set, and the attributes will be prefixed by the value of > said property, followed by a period, followed by the evaluated cache key. So > if Cache Key Identifier is set to "field1, field2" and Put Cache Value In > Attribute is set to "myattrs", then the value for field1 will be placed in > the "myattrs.field1" attribute, and field2's value in "myattrs.field2" > respectively. > 4) Due to the possible presence of Expression Language in the Cache Key > Identifier property, it may not be possible to determine whether multiple > cache keys are present (i.e. a single EL function that generates a > comma-separated list), so the requirement on Put Cache Value In Attribute > being set must be checked at validation time (if possible) and also run-time > 5) To make this fetch efficient, a method "subMap" can be created on the > DistributedMapCache API, so multiple keys can be passed and multiple > key/value pairs can be returned in a single call to the cache server. > 6) #5 implies a new protocol version (would be 3 at the time of this writing) > be added to the DistributedMapCache API > 7) If protocol negotiation results in a lower version being used, then the > client should gracefully degrade into using the "subMap" operation to make > multiple calls to the "get" operation, and fill in the result map manually. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4589) Allow multiple keys in FetchDistributedMapCache
[ https://issues.apache.org/jira/browse/NIFI-4589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-4589: --- Status: Patch Available (was: In Progress) > Allow multiple keys in FetchDistributedMapCache > --- > > Key: NIFI-4589 > URL: https://issues.apache.org/jira/browse/NIFI-4589 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess > > Currently (1.4.0) FetchDistributedMapCache will look up a single key and > place the value (if present) in either the flow file content or a user-named > attribute, depending on the value of the "Put Cache Value In Attribute" > property. If the user wishes to look up more than one value, they need > multiple FetchDistributedMapCache processors, and each would make one call to > the server to get a single key. > A useful improvement would be to allow multiple keys to be retrieved at once > by FetchDistributedMapCache. This would likely involve the following: > 1) Update documentation and code to accept a comma-separated list of Cache > Key Identifiers > 2) If a single Cache Key Identifier is specified, current behavior is > retained (i.e. output to flow file content or a specified attribute) > 3) If multiple Cache Key Identifiers are specified, then Put Cache Value In > Attribute must be set, and the attributes will be prefixed by the value of > said property, followed by a period, followed by the evaluated cache key. So > if Cache Key Identifier is set to "field1, field2" and Put Cache Value In > Attribute is set to "myattrs", then the value for field1 will be placed in > the "myattrs.field1" attribute, and field2's value in "myattrs.field2" > respectively. > 4) Due to the possible presence of Expression Language in the Cache Key > Identifier property, it may not be possible to determine whether multiple > cache keys are present (i.e. a single EL function that generates a > comma-separated list), so the requirement on Put Cache Value In Attribute > being set must be checked at validation time (if possible) and also run-time > 5) To make this fetch efficient, a method "subMap" can be created on the > DistributedMapCache API, so multiple keys can be passed and multiple > key/value pairs can be returned in a single call to the cache server. > 6) #5 implies a new protocol version (would be 3 at the time of this writing) > be added to the DistributedMapCache API > 7) If protocol negotiation results in a lower version being used, then the > client should gracefully degrade into using the "subMap" operation to make > multiple calls to the "get" operation, and fill in the result map manually. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4589) Allow multiple keys in FetchDistributedMapCache
[ https://issues.apache.org/jira/browse/NIFI-4589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245846#comment-16245846 ] ASF GitHub Bot commented on NIFI-4589: -- Github user mattyb149 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2260#discussion_r149995922 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/FetchDistributedMapCache.java --- @@ -151,6 +164,35 @@ public FetchDistributedMapCache() { } @Override +protected Collection customValidate(ValidationContext validationContext) { +List results = new ArrayList<>(super.customValidate(validationContext)); + +PropertyValue cacheEntryIdentifier = validationContext.getProperty(PROP_CACHE_ENTRY_IDENTIFIER); +boolean elPresent = false; +try { --- End diff -- Hoping to remove this try/catch if [NIFI-4590](https://issues.apache.org/jira/browse/NIFI-4590) is implemented first. > Allow multiple keys in FetchDistributedMapCache > --- > > Key: NIFI-4589 > URL: https://issues.apache.org/jira/browse/NIFI-4589 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Burgess > > Currently (1.4.0) FetchDistributedMapCache will look up a single key and > place the value (if present) in either the flow file content or a user-named > attribute, depending on the value of the "Put Cache Value In Attribute" > property. If the user wishes to look up more than one value, they need > multiple FetchDistributedMapCache processors, and each would make one call to > the server to get a single key. > A useful improvement would be to allow multiple keys to be retrieved at once > by FetchDistributedMapCache. This would likely involve the following: > 1) Update documentation and code to accept a comma-separated list of Cache > Key Identifiers > 2) If a single Cache Key Identifier is specified, current behavior is > retained (i.e. output to flow file content or a specified attribute) > 3) If multiple Cache Key Identifiers are specified, then Put Cache Value In > Attribute must be set, and the attributes will be prefixed by the value of > said property, followed by a period, followed by the evaluated cache key. So > if Cache Key Identifier is set to "field1, field2" and Put Cache Value In > Attribute is set to "myattrs", then the value for field1 will be placed in > the "myattrs.field1" attribute, and field2's value in "myattrs.field2" > respectively. > 4) Due to the possible presence of Expression Language in the Cache Key > Identifier property, it may not be possible to determine whether multiple > cache keys are present (i.e. a single EL function that generates a > comma-separated list), so the requirement on Put Cache Value In Attribute > being set must be checked at validation time (if possible) and also run-time > 5) To make this fetch efficient, a method "subMap" can be created on the > DistributedMapCache API, so multiple keys can be passed and multiple > key/value pairs can be returned in a single call to the cache server. > 6) #5 implies a new protocol version (would be 3 at the time of this writing) > be added to the DistributedMapCache API > 7) If protocol negotiation results in a lower version being used, then the > client should gracefully degrade into using the "subMap" operation to make > multiple calls to the "get" operation, and fill in the result map manually. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #2260: NIFI-4589: Allow multiple keys in FetchDistributedM...
Github user mattyb149 commented on a diff in the pull request: https://github.com/apache/nifi/pull/2260#discussion_r149995922 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/FetchDistributedMapCache.java --- @@ -151,6 +164,35 @@ public FetchDistributedMapCache() { } @Override +protected Collection customValidate(ValidationContext validationContext) { +List results = new ArrayList<>(super.customValidate(validationContext)); + +PropertyValue cacheEntryIdentifier = validationContext.getProperty(PROP_CACHE_ENTRY_IDENTIFIER); +boolean elPresent = false; +try { --- End diff -- Hoping to remove this try/catch if [NIFI-4590](https://issues.apache.org/jira/browse/NIFI-4590) is implemented first. ---
[jira] [Commented] (NIFI-4589) Allow multiple keys in FetchDistributedMapCache
[ https://issues.apache.org/jira/browse/NIFI-4589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245845#comment-16245845 ] ASF GitHub Bot commented on NIFI-4589: -- GitHub user mattyb149 opened a pull request: https://github.com/apache/nifi/pull/2260 NIFI-4589: Allow multiple keys in FetchDistributedMapCache, add subMap to API Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mattyb149/nifi NIFI-4589 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2260.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2260 commit 16b0fae5599f47857ae926b3c025d351a855987c Author: Matthew BurgessDate: 2017-11-09T15:31:16Z NIFI-4589: Allow multiple keys in FetchDistributedMapCache, add subMap operation to API > Allow multiple keys in FetchDistributedMapCache > --- > > Key: NIFI-4589 > URL: https://issues.apache.org/jira/browse/NIFI-4589 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Burgess > > Currently (1.4.0) FetchDistributedMapCache will look up a single key and > place the value (if present) in either the flow file content or a user-named > attribute, depending on the value of the "Put Cache Value In Attribute" > property. If the user wishes to look up more than one value, they need > multiple FetchDistributedMapCache processors, and each would make one call to > the server to get a single key. > A useful improvement would be to allow multiple keys to be retrieved at once > by FetchDistributedMapCache. This would likely involve the following: > 1) Update documentation and code to accept a comma-separated list of Cache > Key Identifiers > 2) If a single Cache Key Identifier is specified, current behavior is > retained (i.e. output to flow file content or a specified attribute) > 3) If multiple Cache Key Identifiers are specified, then Put Cache Value In > Attribute must be set, and the attributes will be prefixed by the value of > said property, followed by a period, followed by the evaluated cache key. So > if Cache Key Identifier is set to "field1, field2" and Put Cache Value In > Attribute is set to "myattrs", then the value for field1 will be placed in > the "myattrs.field1" attribute, and field2's value in "myattrs.field2" > respectively. > 4) Due to the possible presence of Expression Language in the Cache Key > Identifier property, it may not be possible to determine whether multiple > cache keys are present (i.e. a single EL function that generates a > comma-separated list), so the requirement on Put Cache Value In Attribute > being set must be checked at validation time (if possible) and also run-time > 5) To make this fetch efficient, a method "subMap" can be created on the > DistributedMapCache API, so multiple keys can be passed and
[GitHub] nifi pull request #2260: NIFI-4589: Allow multiple keys in FetchDistributedM...
GitHub user mattyb149 opened a pull request: https://github.com/apache/nifi/pull/2260 NIFI-4589: Allow multiple keys in FetchDistributedMapCache, add subMap to API Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mattyb149/nifi NIFI-4589 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2260.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2260 commit 16b0fae5599f47857ae926b3c025d351a855987c Author: Matthew BurgessDate: 2017-11-09T15:31:16Z NIFI-4589: Allow multiple keys in FetchDistributedMapCache, add subMap operation to API ---
[jira] [Commented] (MINIFICPP-297) Remove Boost as a dependency for archive extensions
[ https://issues.apache.org/jira/browse/MINIFICPP-297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245837#comment-16245837 ] ASF GitHub Bot commented on MINIFICPP-297: -- Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/181 Still doing some validation on older *nix releases to validate that the stat structure is the same everywhere. In some cases it will not be, so need to finish validation against all releases. > Remove Boost as a dependency for archive extensions > --- > > Key: MINIFICPP-297 > URL: https://issues.apache.org/jira/browse/MINIFICPP-297 > Project: NiFi MiNiFi C++ > Issue Type: Bug >Reporter: marco polo >Assignee: marco polo > Fix For: 0.3.0 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi-minifi-cpp issue #181: MINIFICPP-297: Remove Boost dependencies since w...
Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/181 Still doing some validation on older *nix releases to validate that the stat structure is the same everywhere. In some cases it will not be, so need to finish validation against all releases. ---
[jira] [Commented] (MINIFICPP-297) Remove Boost as a dependency for archive extensions
[ https://issues.apache.org/jira/browse/MINIFICPP-297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245832#comment-16245832 ] ASF GitHub Bot commented on MINIFICPP-297: -- GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/181 MINIFICPP-297: Remove Boost dependencies since we have very little re… …liance on it MINIFICPP-297: Ensure posix compliance Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFICPP-297 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/181.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #181 commit 1e4fb83511482e8048989adc9ab6e3d5e8d74d66 Author: Marc ParisiDate: 2017-11-08T19:03:22Z MINIFICPP-297: Remove Boost dependencies since we have very little reliance on it MINIFICPP-297: Ensure posix compliance > Remove Boost as a dependency for archive extensions > --- > > Key: MINIFICPP-297 > URL: https://issues.apache.org/jira/browse/MINIFICPP-297 > Project: NiFi MiNiFi C++ > Issue Type: Bug >Reporter: marco polo >Assignee: marco polo > Fix For: 0.3.0 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi-minifi-cpp pull request #181: MINIFICPP-297: Remove Boost dependencies ...
GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/181 MINIFICPP-297: Remove Boost dependencies since we have very little re⦠â¦liance on it MINIFICPP-297: Ensure posix compliance Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFICPP-297 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/181.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #181 commit 1e4fb83511482e8048989adc9ab6e3d5e8d74d66 Author: Marc ParisiDate: 2017-11-08T19:03:22Z MINIFICPP-297: Remove Boost dependencies since we have very little reliance on it MINIFICPP-297: Ensure posix compliance ---
[jira] [Created] (NIFI-4590) customValidate() fails during unit tests when isExpressionLanguagePresent() is called
Matt Burgess created NIFI-4590: -- Summary: customValidate() fails during unit tests when isExpressionLanguagePresent() is called Key: NIFI-4590 URL: https://issues.apache.org/jira/browse/NIFI-4590 Project: Apache NiFi Issue Type: Test Components: Core Framework Reporter: Matt Burgess When isExpressionLanguagePresent() is called from a customValidate() method in a processor during a unit test, a NullPointerException is thrown. Inside that method in MockPropertyValue, a Boolean (not boolean) reference "expectExpressions" appears that expectExpressions has not yet been populated by the time isExpressionLanguagePresent() is called, and the if(!expectExpressions) clause results in an NPE. Not sure if the fix is to change expectExpressions to be a boolean, or to allow for expectExpressions == null in isExpressionLanguagePresent(), or some third option to achieve the desired behavior while maintaining an accurate mock of true behavior. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFICPP-258) C2NullConfiguration fails due to a segfault on travis.
[ https://issues.apache.org/jira/browse/MINIFICPP-258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245770#comment-16245770 ] Caleb Johnson commented on MINIFICPP-258: - Fix merged, I think this can be closed. > C2NullConfiguration fails due to a segfault on travis. > --- > > Key: MINIFICPP-258 > URL: https://issues.apache.org/jira/browse/MINIFICPP-258 > Project: NiFi MiNiFi C++ > Issue Type: Bug >Reporter: marco polo >Assignee: marco polo > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (NIFI-4589) Allow multiple keys in FetchDistributedMapCache
Matt Burgess created NIFI-4589: -- Summary: Allow multiple keys in FetchDistributedMapCache Key: NIFI-4589 URL: https://issues.apache.org/jira/browse/NIFI-4589 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: Matt Burgess Currently (1.4.0) FetchDistributedMapCache will look up a single key and place the value (if present) in either the flow file content or a user-named attribute, depending on the value of the "Put Cache Value In Attribute" property. If the user wishes to look up more than one value, they need multiple FetchDistributedMapCache processors, and each would make one call to the server to get a single key. A useful improvement would be to allow multiple keys to be retrieved at once by FetchDistributedMapCache. This would likely involve the following: 1) Update documentation and code to accept a comma-separated list of Cache Key Identifiers 2) If a single Cache Key Identifier is specified, current behavior is retained (i.e. output to flow file content or a specified attribute) 3) If multiple Cache Key Identifiers are specified, then Put Cache Value In Attribute must be set, and the attributes will be prefixed by the value of said property, followed by a period, followed by the evaluated cache key. So if Cache Key Identifier is set to "field1, field2" and Put Cache Value In Attribute is set to "myattrs", then the value for field1 will be placed in the "myattrs.field1" attribute, and field2's value in "myattrs.field2" respectively. 4) Due to the possible presence of Expression Language in the Cache Key Identifier property, it may not be possible to determine whether multiple cache keys are present (i.e. a single EL function that generates a comma-separated list), so the requirement on Put Cache Value In Attribute being set must be checked at validation time (if possible) and also run-time 5) To make this fetch efficient, a method "subMap" can be created on the DistributedMapCache API, so multiple keys can be passed and multiple key/value pairs can be returned in a single call to the cache server. 6) #5 implies a new protocol version (would be 3 at the time of this writing) be added to the DistributedMapCache API 7) If protocol negotiation results in a lower version being used, then the client should gracefully degrade into using the "subMap" operation to make multiple calls to the "get" operation, and fill in the result map manually. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFICPP-258) C2NullConfiguration fails due to a segfault on travis.
[ https://issues.apache.org/jira/browse/MINIFICPP-258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245741#comment-16245741 ] ASF GitHub Bot commented on MINIFICPP-258: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/180 > C2NullConfiguration fails due to a segfault on travis. > --- > > Key: MINIFICPP-258 > URL: https://issues.apache.org/jira/browse/MINIFICPP-258 > Project: NiFi MiNiFi C++ > Issue Type: Bug >Reporter: marco polo >Assignee: marco polo > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi-minifi-cpp pull request #180: MINIFICPP-258 enable C2NullConfiguration ...
Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/180 ---
[jira] [Assigned] (NIFI-4587) no newline in processor/PG comments tab when using Firefox browser,
[ https://issues.apache.org/jira/browse/NIFI-4587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman reassigned NIFI-4587: - Assignee: Matt Gilman > no newline in processor/PG comments tab when using Firefox browser, > > > Key: NIFI-4587 > URL: https://issues.apache.org/jira/browse/NIFI-4587 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.3.0 >Reporter: Haimo Liu >Assignee: Matt Gilman > > (Firefox) when editing comments in processors or process groups, the > "comment" tab does not properly handle newlines (CR/LF). However we can see > correct newlines in Chrome. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (NIFI-4526) The REST API and NiFi UI should allow the target URL for a remote process group to be edited
[ https://issues.apache.org/jira/browse/NIFI-4526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman reassigned NIFI-4526: - Assignee: Matt Gilman > The REST API and NiFi UI should allow the target URL for a remote process > group to be edited > > > Key: NIFI-4526 > URL: https://issues.apache.org/jira/browse/NIFI-4526 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Core UI, Flow Versioning >Reporter: Joseph Witt >Assignee: Matt Gilman >Priority: Critical > > Today remote process groups (RPG) once established cannot have the target URL > edited. If a user wants to change the URL they have to add a new RPG for the > new target and change the relationships/connections to point to it. Then > they can remove the old one. > That process ensures that there is explicit (user provided mapping) to the > new ports available at the new target. However, this is very limiting in > templates today and would be very limiting to the effectiveness of the > versioned flow registry and porting between environments. > We could make the URL support expression language statements but at present > that would be the first non component property (like processor properties) > where EL is allowed and we need to have a more thoughtful and consistent > approach for that to include things like number of processor threads/etc.. A > cleaner and more consistent option is to do this like sensitive properties > are being handled in versioned flows which is they are not part of the > versioned flow definition but rather on import or when edited we allow the > user to set their environment/flow specific values all without making the > local flow version dirty in terms of versioned flow management. > So, we should allow the user via the UI and obviously through REST API calls > to change the URL. This would require stopping the RPG, changing the target > URL (resycing/establishing remote ports/auth/etc..), then starting it. We > need to make sure this change does not necessitate a new version of the flow. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4559) ExecuteStreamCommand should have a failure relationship
[ https://issues.apache.org/jira/browse/NIFI-4559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-4559: --- Affects Version/s: (was: 1.4.0) > ExecuteStreamCommand should have a failure relationship > --- > > Key: NIFI-4559 > URL: https://issues.apache.org/jira/browse/NIFI-4559 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Joseph Witt >Assignee: Matt Burgess > > ExecuteStreamCommand only has a success relationship. The gist is it > executes the command, captures the output stream, code, etc.. and stores the > necessary content and attributes. > However, for those wanting to see the status code as an indicator of failure > they have to create a flow which checks, loops, etc.. This is more work than > necessary since the processor can provide a better indicator. By adding a > failure relationship we can route results and status codes for non-zero to > this relationship. We'll have to consider backward compatibility friendly > ideas but the gist is to make this common failure/retry pattern easier to > configure and track. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4559) ExecuteStreamCommand should have a failure relationship
[ https://issues.apache.org/jira/browse/NIFI-4559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-4559: --- Status: Patch Available (was: In Progress) > ExecuteStreamCommand should have a failure relationship > --- > > Key: NIFI-4559 > URL: https://issues.apache.org/jira/browse/NIFI-4559 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.4.0 >Reporter: Joseph Witt >Assignee: Matt Burgess > > ExecuteStreamCommand only has a success relationship. The gist is it > executes the command, captures the output stream, code, etc.. and stores the > necessary content and attributes. > However, for those wanting to see the status code as an indicator of failure > they have to create a flow which checks, loops, etc.. This is more work than > necessary since the processor can provide a better indicator. By adding a > failure relationship we can route results and status codes for non-zero to > this relationship. We'll have to consider backward compatibility friendly > ideas but the gist is to make this common failure/retry pattern easier to > configure and track. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #2113: NIFI-4325 Added new processor that uses the JSON DSL.
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2113 Thanks @mattyb149 I'll try to find some time to address these today or tomorrow. I've been knee deep in some Mongo-related work for a client. I have two pull requests for Mongo and record api-related fixes/updates if you're up for reviewing two small patches (one is 3 lines of code :) ) ---
[jira] [Commented] (NIFI-4325) Create a new ElasticSearch processor that supports the JSON DSL
[ https://issues.apache.org/jira/browse/NIFI-4325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245682#comment-16245682 ] ASF GitHub Bot commented on NIFI-4325: -- Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2113 Thanks @mattyb149 I'll try to find some time to address these today or tomorrow. I've been knee deep in some Mongo-related work for a client. I have two pull requests for Mongo and record api-related fixes/updates if you're up for reviewing two small patches (one is 3 lines of code :) ) > Create a new ElasticSearch processor that supports the JSON DSL > --- > > Key: NIFI-4325 > URL: https://issues.apache.org/jira/browse/NIFI-4325 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mike Thomsen >Priority: Minor > > The existing ElasticSearch processors use the Lucene-style syntax for > querying, not the JSON DSL. A new processor is needed that can take a full > JSON query and execute it. It should also support aggregation queries in this > syntax. A user needs to be able to take a query as-is from Kibana and drop it > into NiFi and have it just run. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4588) Add ability to use update operators like $set and $push in PutMongo
[ https://issues.apache.org/jira/browse/NIFI-4588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16245678#comment-16245678 ] ASF GitHub Bot commented on NIFI-4588: -- GitHub user MikeThomsen opened a pull request: https://github.com/apache/nifi/pull/2259 NIFI-4588 Added the ability to use update operators like and to Put… …Mongo. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/MikeThomsen/nifi NIFI-4588 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2259.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2259 > Add ability to use update operators like $set and $push in PutMongo > --- > > Key: NIFI-4588 > URL: https://issues.apache.org/jira/browse/NIFI-4588 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mike Thomsen >Assignee: Mike Thomsen > > The current implementation of PutMongo just puts an entire document in with > an update. There are situations where this will definitely not work like when > a user needs to just update one field or push an element to array in an > existing document. > The update should provide a configurable ability to push either an entire > JSON document (using extended JSON, as that is required by MongoDB) or use an > update document like this: > {code:java} > { > "$set": { > "strField": "Message", > "dateField": { > "$date": 1510235376611 > } > }, > "$push": { > "arrayField": { > "field": "Something here" > } > } > } > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #2259: NIFI-4588 Added the ability to use update operators...
GitHub user MikeThomsen opened a pull request: https://github.com/apache/nifi/pull/2259 NIFI-4588 Added the ability to use update operators like and to Put⦠â¦Mongo. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/MikeThomsen/nifi NIFI-4588 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2259.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2259 ---
[jira] [Updated] (NIFI-4588) Add ability to use update operators like $set and $push in PutMongo
[ https://issues.apache.org/jira/browse/NIFI-4588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Thomsen updated NIFI-4588: --- Description: The current implementation of PutMongo just puts an entire document in with an update. There are situations where this will definitely not work like when a user needs to just update one field or push an element to array in an existing document. The update should provide a configurable ability to push either an entire JSON document (using extended JSON, as that is required by MongoDB) or use an update document like this: {code:java} { "$set": { "strField": "Message", "dateField": { "$date": 1510235376611 } }, "$push": { "arrayField": { "field": "Something here" } } } {code} was: The current implementation of PutMongo just puts an entire document in with an update. There are situations where this will definitely not work like when a user needs to just update one field or push an element to array in an existing document. The update should provide a configurable ability to push either an entire JSON document (using extended JSON, as that is required by MongoDB) or use an update document like this: { "$set": { "strField": "Message", "dateField": { "$date": 1510235376611 } }, "$push": { "arrayField": { "field": "Something here" } } } > Add ability to use update operators like $set and $push in PutMongo > --- > > Key: NIFI-4588 > URL: https://issues.apache.org/jira/browse/NIFI-4588 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mike Thomsen >Assignee: Mike Thomsen > > The current implementation of PutMongo just puts an entire document in with > an update. There are situations where this will definitely not work like when > a user needs to just update one field or push an element to array in an > existing document. > The update should provide a configurable ability to push either an entire > JSON document (using extended JSON, as that is required by MongoDB) or use an > update document like this: > {code:java} > { > "$set": { > "strField": "Message", > "dateField": { > "$date": 1510235376611 > } > }, > "$push": { > "arrayField": { > "field": "Something here" > } > } > } > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (NIFI-4588) Add ability to use update operators like $set and $push in PutMongo
Mike Thomsen created NIFI-4588: -- Summary: Add ability to use update operators like $set and $push in PutMongo Key: NIFI-4588 URL: https://issues.apache.org/jira/browse/NIFI-4588 Project: Apache NiFi Issue Type: Improvement Reporter: Mike Thomsen Assignee: Mike Thomsen The current implementation of PutMongo just puts an entire document in with an update. There are situations where this will definitely not work like when a user needs to just update one field or push an element to array in an existing document. The update should provide a configurable ability to push either an entire JSON document (using extended JSON, as that is required by MongoDB) or use an update document like this: { "$set": { "strField": "Message", "dateField": { "$date": 1510235376611 } }, "$push": { "arrayField": { "field": "Something here" } } } -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (NIFI-4587) no newline in processor/PG comments tab when using Firefox browser,
Haimo Liu created NIFI-4587: --- Summary: no newline in processor/PG comments tab when using Firefox browser, Key: NIFI-4587 URL: https://issues.apache.org/jira/browse/NIFI-4587 Project: Apache NiFi Issue Type: Bug Components: Core UI Affects Versions: 1.3.0 Reporter: Haimo Liu (Firefox) when editing comments in processors or process groups, the "comment" tab does not properly handle newlines (CR/LF). However we can see correct newlines in Chrome. -- This message was sent by Atlassian JIRA (v6.4.14#64029)