[jira] [Commented] (NIFI-4711) when i start consumerKafka processor more than eleven,all of the processors get into an indefinite stuck state
[
https://issues.apache.org/jira/browse/NIFI-4711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16296301#comment-16296301
]
clearlove commented on NIFI-4711:
-
When I connect kafka with javaAPI, when a thread is creating two consumers and
subscribing to two different topics, it looks like this
ConsumerRecords records = consumer.poll(10);
ConsumerRecords records1 = consumer1.poll(10);
for (ConsumerRecord consumerRecord : records){
System.out.println(consumerRecord.partition() + " Offset: "
+ consumerRecord.offset() + " Value: " + consumerRecord.value() + " ThreadID: "
+ Thread.currentThread().getId());
}
for (ConsumerRecord consumerRecord : records1){
System.out.println(consumerRecord.partition() + " Offset: "
+ consumerRecord.offset() + " Value: " + consumerRecord.value() + " ThreadID: "
+ Thread.currentThread().getId());
}
The two consumers will switch slowly.Maybe this season.
> when i start consumerKafka processor more than eleven,all of the processors
> get into an indefinite stuck state
> --
>
> Key: NIFI-4711
> URL: https://issues.apache.org/jira/browse/NIFI-4711
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.3.0
> Environment: Linux
>Reporter: clearlove
> Attachments: 2.png, KafkaTemple.xml, nifi-bootstrap.log
>
>
> hello,
> When i start one consumeKafka_0_10 processor ,all of the processors are all
> right.
> but when i start consumeKafka_0_10 processor more than eleven,all of the
> processors get into an indefinite stuck state.
> !1.png|thumbnail!
> like in the picture ,look like everything is all right, but all
> processor(even processor is not consumeKafka_0_10 ) are stop work.All of the
> flowfile don't move.
> And when i stop them , some of consumeKafka_0_10 processor are warning
> {{Was interrupted while trying to communicate with Kafka with lease {}. Will
> roll back session and discard any partially received data.}}
> in the nifi-app.log
> {noformat}
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
> 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-4]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ReplaceText[id=099a83f6-7ddc-3df8-fe07-5b901799ade3] to run
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.SplitJson
> 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-2]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> EvaluateJsonPath[id=fdd4850d-a619-3f17-6287-198cac3bb52a] to run
> 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-4]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> ConsumeKafka_0_10[id=bed469b9-b52e-3178-f629-07f98ee6c07e] to run
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.PutSQL
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
> 2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-2]
> o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
> SplitJson[id=f3f85270-6bb8-3f65-4347-6700426d6a2e] to run
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.ReplaceText
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.nifi.processors.standard.EvaluateJsonPath
> 2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
> o.a.n.controller.StandardProcessorNode Stopping processor: class
> org.apache.
[jira] [Created] (NIFI-4711) when i start consumerKafka processor more than eleven,all of the processors get into an indefinite stuck state
clearlove created NIFI-4711:
---
Summary: when i start consumerKafka processor more than eleven,all
of the processors get into an indefinite stuck state
Key: NIFI-4711
URL: https://issues.apache.org/jira/browse/NIFI-4711
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 1.3.0
Environment: Linux
Reporter: clearlove
Attachments: 2.png, KafkaTemple.xml, nifi-bootstrap.log
hello,
When i start one consumeKafka_0_10 processor ,all of the processors are all
right.
but when i start consumeKafka_0_10 processor more than eleven,all of the
processors get into an indefinite stuck state.
!1.png|thumbnail!
like in the picture ,look like everything is all right, but all processor(even
processor is not consumeKafka_0_10 ) are stop work.All of the flowfile don't
move.
And when i stop them , some of consumeKafka_0_10 processor are warning
{{Was interrupted while trying to communicate with Kafka with lease {}. Will
roll back session and discard any partially received data.}}
in the nifi-app.log
{noformat}
2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
o.a.n.controller.StandardProcessorNode Stopping processor: class
org.apache.nifi.processors.standard.EvaluateJsonPath
2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
o.a.n.controller.StandardProcessorNode Stopping processor: class
org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-4]
o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
ReplaceText[id=099a83f6-7ddc-3df8-fe07-5b901799ade3] to run
2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
o.a.n.controller.StandardProcessorNode Stopping processor: class
org.apache.nifi.processors.standard.SplitJson
2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-2]
o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
EvaluateJsonPath[id=fdd4850d-a619-3f17-6287-198cac3bb52a] to run
2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-4]
o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
ConsumeKafka_0_10[id=bed469b9-b52e-3178-f629-07f98ee6c07e] to run
2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
o.a.n.controller.StandardProcessorNode Stopping processor: class
org.apache.nifi.processors.standard.EvaluateJsonPath
2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
o.a.n.controller.StandardProcessorNode Stopping processor: class
org.apache.nifi.processors.standard.EvaluateJsonPath
2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
o.a.n.controller.StandardProcessorNode Stopping processor: class
org.apache.nifi.processors.standard.PutSQL
2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
o.a.n.controller.StandardProcessorNode Stopping processor: class
org.apache.nifi.processors.standard.EvaluateJsonPath
2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
o.a.n.controller.StandardProcessorNode Stopping processor: class
org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-2]
o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
SplitJson[id=f3f85270-6bb8-3f65-4347-6700426d6a2e] to run
2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
o.a.n.controller.StandardProcessorNode Stopping processor: class
org.apache.nifi.processors.standard.ReplaceText
2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
o.a.n.controller.StandardProcessorNode Stopping processor: class
org.apache.nifi.processors.standard.EvaluateJsonPath
2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
o.a.n.controller.StandardProcessorNode Stopping processor: class
org.apache.nifi.processors.standard.EvaluateJsonPath
2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
o.a.n.controller.StandardProcessorNode Stopping processor: class
org.apache.nifi.processors.standard.PutSQL
2017-12-05 10:12:02,128 INFO [NiFi Web Server-1179]
o.a.n.controller.StandardProcessorNode Stopping processor: class
org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
2017-12-05 10:12:02,128 INFO [StandardProcessScheduler Thread-2]
o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling
EvaluateJsonPath[id=6aacc9a9-c7e8-32b0-ed3c-3f89e72c30cf] to run
2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
o.a.n.controller.StandardProcessorNode Stopping processor: class
org.apache.nifi.processors.standard.ReplaceText
2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
o.a.n.controller.StandardProcessorNode Stopping processor: class
org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
o.a.n.controller.StandardProcessorNode Stopping processor: class
org.apache.nifi.processors.kafka.pubsub.ConsumeKafka_0_10
2017-12-05 10:12:02,129 INFO [NiFi Web Server-1179]
o.a.n.controller.StandardProcessorNode Stopping processor: class
org.apache.nifi.processors.standard.EvaluateJsonPath
2017-12
[GitHub] nifi pull request #2351: NIFI-4707: S2SProvenanceReportingTask not returning...
Github user ijokarumawak commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2351#discussion_r157667259
--- Diff:
nifi-nar-bundles/nifi-atlas-bundle/nifi-atlas-reporting-task/src/main/java/org/apache/nifi/atlas/reporting/ReportLineageToAtlas.java
---
@@ -640,7 +640,7 @@ private void
consumeNiFiProvenanceEvents(ReportingContext context, NiFiFlow nifi
final AnalysisContext analysisContext = new
StandardAnalysisContext(nifiFlow, clusterResolvers,
// FIXME: This class cast shouldn't be necessary to query
lineage. Possible refactor target in next major update.
(ProvenanceRepository)eventAccess.getProvenanceRepository());
-consumer.consumeEvents(eventAccess, context.getStateManager(),
events -> {
+consumer.consumeEvents(null, eventAccess,
context.getStateManager(), events -> {
--- End diff --
It would be more useful if we add another consumeEvents method signature
(or replace existing one) having the last argument as
`BiConsumer>`.
---
[jira] [Commented] (NIFI-4707) SiteToSiteProvenanceReportingTask not returning correct metadata
[
https://issues.apache.org/jira/browse/NIFI-4707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16296235#comment-16296235
]
ASF GitHub Bot commented on NIFI-4707:
--
Github user ijokarumawak commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2351#discussion_r157667259
--- Diff:
nifi-nar-bundles/nifi-atlas-bundle/nifi-atlas-reporting-task/src/main/java/org/apache/nifi/atlas/reporting/ReportLineageToAtlas.java
---
@@ -640,7 +640,7 @@ private void
consumeNiFiProvenanceEvents(ReportingContext context, NiFiFlow nifi
final AnalysisContext analysisContext = new
StandardAnalysisContext(nifiFlow, clusterResolvers,
// FIXME: This class cast shouldn't be necessary to query
lineage. Possible refactor target in next major update.
(ProvenanceRepository)eventAccess.getProvenanceRepository());
-consumer.consumeEvents(eventAccess, context.getStateManager(),
events -> {
+consumer.consumeEvents(null, eventAccess,
context.getStateManager(), events -> {
--- End diff --
It would be more useful if we add another consumeEvents method signature
(or replace existing one) having the last argument as
`BiConsumer>`.
> SiteToSiteProvenanceReportingTask not returning correct metadata
>
>
> Key: NIFI-4707
> URL: https://issues.apache.org/jira/browse/NIFI-4707
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Matt Burgess
>Assignee: Matt Burgess
>
> When the SiteToSiteProvenanceReportingTask emits flow files, some of them
> include a "componentName" field and some do not. Investigation shows that
> only the components (except connections) in the root process group have that
> field populated. Having this information can be very helpful to the user,
> even though the names might be duplicated, there would be a mapping between a
> component's ID and its name. At the very least the behavior (i.e. component
> name being available) should be consistent.
> Having a full map (by traversing the entire flow) also opens up the ability
> to include Process Group information for the various components. The
> reporting task could include the parent Process Group identifier and/or name,
> with perhaps a special ID for the root PG's "parent", such as "@ROOT@" or
> something unique.
> This could also allow for a PG ID in the list of filtered "component IDs",
> where any provenance event for a processor in a particular PG could be
> included in a filter when that PG's ID is in the filter list.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-4005) Add support for Azure Shared Access Signature (SAS) Tokens
[ https://issues.apache.org/jira/browse/NIFI-4005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16296231#comment-16296231 ] ASF GitHub Bot commented on NIFI-4005: -- Github user ijokarumawak commented on the issue: https://github.com/apache/nifi/pull/1886 @aperepel I took over updating the PR. The new one #2353 is submitted based on your commit. Would you close this PR? Thanks! > Add support for Azure Shared Access Signature (SAS) Tokens > -- > > Key: NIFI-4005 > URL: https://issues.apache.org/jira/browse/NIFI-4005 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.2.0 >Reporter: Andrew Grande >Assignee: Koji Kawamura >Priority: Minor > > Azure's account name and key are treated as admin, 'root' access credentials. > If one has those, every container under this account is fully accessible. An > MSFT-recommended approach is to use SAS policies, which provide for a fine > grained permission and object control, as well as defined expiration. > I already have working code, filing this ticket to formally track and submit > PR against next. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #1886: NIFI-4005 Add support for Azure Shared Access Signature (S...
Github user ijokarumawak commented on the issue: https://github.com/apache/nifi/pull/1886 @aperepel I took over updating the PR. The new one #2353 is submitted based on your commit. Would you close this PR? Thanks! ---
[jira] [Commented] (NIFI-4707) SiteToSiteProvenanceReportingTask not returning correct metadata
[
https://issues.apache.org/jira/browse/NIFI-4707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16296225#comment-16296225
]
ASF GitHub Bot commented on NIFI-4707:
--
Github user ijokarumawak commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2351#discussion_r157666543
--- Diff:
nifi-nar-bundles/nifi-site-to-site-reporting-bundle/nifi-site-to-site-reporting-task/src/main/java/org/apache/nifi/reporting/SiteToSiteProvenanceReportingTask.java
---
@@ -174,34 +177,47 @@ public void onUnscheduled() {
return properties;
}
-private Map createComponentMap(final ProcessGroupStatus
status) {
-final Map componentMap = new HashMap<>();
+private ComponentMapHolder createComponentMap(final ProcessGroupStatus
status) {
--- End diff --
This method probably should be in ProvenanceEventConsumer instead of each
ReportingTask implementation so that other ReportingTasks can get benefits from
it. How do you think?
> SiteToSiteProvenanceReportingTask not returning correct metadata
>
>
> Key: NIFI-4707
> URL: https://issues.apache.org/jira/browse/NIFI-4707
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Matt Burgess
>Assignee: Matt Burgess
>
> When the SiteToSiteProvenanceReportingTask emits flow files, some of them
> include a "componentName" field and some do not. Investigation shows that
> only the components (except connections) in the root process group have that
> field populated. Having this information can be very helpful to the user,
> even though the names might be duplicated, there would be a mapping between a
> component's ID and its name. At the very least the behavior (i.e. component
> name being available) should be consistent.
> Having a full map (by traversing the entire flow) also opens up the ability
> to include Process Group information for the various components. The
> reporting task could include the parent Process Group identifier and/or name,
> with perhaps a special ID for the root PG's "parent", such as "@ROOT@" or
> something unique.
> This could also allow for a PG ID in the list of filtered "component IDs",
> where any provenance event for a processor in a particular PG could be
> included in a filter when that PG's ID is in the filter list.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] nifi pull request #2351: NIFI-4707: S2SProvenanceReportingTask not returning...
Github user ijokarumawak commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2351#discussion_r157666543
--- Diff:
nifi-nar-bundles/nifi-site-to-site-reporting-bundle/nifi-site-to-site-reporting-task/src/main/java/org/apache/nifi/reporting/SiteToSiteProvenanceReportingTask.java
---
@@ -174,34 +177,47 @@ public void onUnscheduled() {
return properties;
}
-private Map createComponentMap(final ProcessGroupStatus
status) {
-final Map componentMap = new HashMap<>();
+private ComponentMapHolder createComponentMap(final ProcessGroupStatus
status) {
--- End diff --
This method probably should be in ProvenanceEventConsumer instead of each
ReportingTask implementation so that other ReportingTasks can get benefits from
it. How do you think?
---
[jira] [Commented] (NIFI-4005) Add support for Azure Shared Access Signature (SAS) Tokens
[ https://issues.apache.org/jira/browse/NIFI-4005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16296206#comment-16296206 ] ASF GitHub Bot commented on NIFI-4005: -- GitHub user ijokarumawak opened a pull request: https://github.com/apache/nifi/pull/2353 NIFI-4005: Add support for Azure Shared Access Signature (SAS) Tokens This PR incorporated review comments based on #1886. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijokarumawak/nifi nifi-4005 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2353.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2353 commit 87d09faac5a11e5a34edcd1bbf2be20ce9ffe880 Author: Andrew Grande Date: 2017-05-31T20:45:26Z NIFI-4005 Add support for Azure Shared Access Signature (SAS) Tokens. Upgraded the client library from 5.0.0 to 5.2.0 commit 4f69235019901dac4bd0e38a541498c877ab273d Author: Koji Kawamura Date: 2017-12-19T02:35:00Z NIFI-4005: Azure Blob Storage SAS support, incorporating review comments - Renamed Azure to AzureStorageUtils. - Fixed whitespacing in property description. - Renamed SAS String to SAS Token. > Add support for Azure Shared Access Signature (SAS) Tokens > -- > > Key: NIFI-4005 > URL: https://issues.apache.org/jira/browse/NIFI-4005 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.2.0 >Reporter: Andrew Grande >Assignee: Koji Kawamura >Priority: Minor > > Azure's account name and key are treated as admin, 'root' access credentials. > If one has those, every container under this account is fully accessible. An > MSFT-recommended approach is to use SAS policies, which provide for a fine > grained permission and object control, as well as defined expiration. > I already have working code, filing this ticket to formally track and submit > PR against next. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #2353: NIFI-4005: Add support for Azure Shared Access Sign...
GitHub user ijokarumawak opened a pull request: https://github.com/apache/nifi/pull/2353 NIFI-4005: Add support for Azure Shared Access Signature (SAS) Tokens This PR incorporated review comments based on #1886. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijokarumawak/nifi nifi-4005 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2353.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2353 commit 87d09faac5a11e5a34edcd1bbf2be20ce9ffe880 Author: Andrew Grande Date: 2017-05-31T20:45:26Z NIFI-4005 Add support for Azure Shared Access Signature (SAS) Tokens. Upgraded the client library from 5.0.0 to 5.2.0 commit 4f69235019901dac4bd0e38a541498c877ab273d Author: Koji Kawamura Date: 2017-12-19T02:35:00Z NIFI-4005: Azure Blob Storage SAS support, incorporating review comments - Renamed Azure to AzureStorageUtils. - Fixed whitespacing in property description. - Renamed SAS String to SAS Token. ---
[jira] [Created] (NIFI-4710) Kerberos
Aldrin Piri created NIFI-4710: - Summary: Kerberos Key: NIFI-4710 URL: https://issues.apache.org/jira/browse/NIFI-4710 Project: Apache NiFi Issue Type: Sub-task Components: Docker Reporter: Aldrin Piri -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4707) SiteToSiteProvenanceReportingTask not returning correct metadata
[ https://issues.apache.org/jira/browse/NIFI-4707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-4707: --- Description: When the SiteToSiteProvenanceReportingTask emits flow files, some of them include a "componentName" field and some do not. Investigation shows that only the components (except connections) in the root process group have that field populated. Having this information can be very helpful to the user, even though the names might be duplicated, there would be a mapping between a component's ID and its name. At the very least the behavior (i.e. component name being available) should be consistent. Having a full map (by traversing the entire flow) also opens up the ability to include Process Group information for the various components. The reporting task could include the parent Process Group identifier and/or name, with perhaps a special ID for the root PG's "parent", such as "@ROOT@" or something unique. This could also allow for a PG ID in the list of filtered "component IDs", where any provenance event for a processor in a particular PG could be included in a filter when that PG's ID is in the filter list. was: When the SiteToSiteProvenanceReportingTask emits flow files, some of them include a "componentName" field and some do not. Investigation shows that only the components (except connections) in the root process group have that field populated. Having this information can be very helpful to the user, even though the names might be duplicated, there would be a mapping between a component's ID and its name. At the very least the behavior (i.e. component name being available) should be consistent. Having a full map (by traversing the entire flow) also opens up the ability to include Process Group information for the various components. The reporting task could include the parent Process Group identifier and/or name, with perhaps a special ID for the root PG's "parent", such as "@ROOT@" or something unique. > SiteToSiteProvenanceReportingTask not returning correct metadata > > > Key: NIFI-4707 > URL: https://issues.apache.org/jira/browse/NIFI-4707 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess > > When the SiteToSiteProvenanceReportingTask emits flow files, some of them > include a "componentName" field and some do not. Investigation shows that > only the components (except connections) in the root process group have that > field populated. Having this information can be very helpful to the user, > even though the names might be duplicated, there would be a mapping between a > component's ID and its name. At the very least the behavior (i.e. component > name being available) should be consistent. > Having a full map (by traversing the entire flow) also opens up the ability > to include Process Group information for the various components. The > reporting task could include the parent Process Group identifier and/or name, > with perhaps a special ID for the root PG's "parent", such as "@ROOT@" or > something unique. > This could also allow for a PG ID in the list of filtered "component IDs", > where any provenance event for a processor in a particular PG could be > included in a filter when that PG's ID is in the filter list. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4707) SiteToSiteProvenanceReportingTask not returning correct metadata
[ https://issues.apache.org/jira/browse/NIFI-4707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-4707: --- Summary: SiteToSiteProvenanceReportingTask not returning correct metadata (was: SiteToSiteProvenanceReportingTask does not always return component name, nor ProcessGroup ID/Name) > SiteToSiteProvenanceReportingTask not returning correct metadata > > > Key: NIFI-4707 > URL: https://issues.apache.org/jira/browse/NIFI-4707 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess > > When the SiteToSiteProvenanceReportingTask emits flow files, some of them > include a "componentName" field and some do not. Investigation shows that > only the components (except connections) in the root process group have that > field populated. Having this information can be very helpful to the user, > even though the names might be duplicated, there would be a mapping between a > component's ID and its name. At the very least the behavior (i.e. component > name being available) should be consistent. > Having a full map (by traversing the entire flow) also opens up the ability > to include Process Group information for the various components. The > reporting task could include the parent Process Group identifier and/or name, > with perhaps a special ID for the root PG's "parent", such as "@ROOT@" or > something unique. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4709) ListAzureBlobStorage misunderstands target system timestamp precision as Minutes
[ https://issues.apache.org/jira/browse/NIFI-4709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-4709: Summary: ListAzureBlobStorage misunderstands target system timestamp precision as Minutes (was: ListAzureBlobStorage mistreat target system timestamp precision as Minutes) > ListAzureBlobStorage misunderstands target system timestamp precision as > Minutes > > > Key: NIFI-4709 > URL: https://issues.apache.org/jira/browse/NIFI-4709 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.4.0 >Reporter: Koji Kawamura >Assignee: Koji Kawamura > > NIFI-4069 added target system timestamp detection for List processors. > Defaults to auto detection. Most sub classes added 'Target System Timestamp > Precision' to their 'getSupportedPropertyDescriptors' method. But > ListAzureBlobStorage didn't. > Even though 'Target System Timestamp Precision' property has a default value, > if it's not included in getSupportedPropertyDescriptors method, the property > value becomes null, instead of the default value. This combination is not > handled well in AbstractListProcessor currently. That makes > ListAzureBlobStorage behaves as if Azure Blob Storage time precision is in > Minutes while it actually has Seconds precision. Incurs longer time for blob > files to be picked than required. > Not having 'Target System Timestamp Precision' at ListAzureBlobStorage seems > reasonable as the processor interact with only Azure Blob Storage, and its > timestamp precision should be fixed. AbstractListProcessor should provide an > extension point for sub-classes to define default precision. In case for > Azure Blob, it's SECONDS. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (NIFI-4709) ListAzureBlobStorage mistreat target system timestamp precision as Minutes
Koji Kawamura created NIFI-4709: --- Summary: ListAzureBlobStorage mistreat target system timestamp precision as Minutes Key: NIFI-4709 URL: https://issues.apache.org/jira/browse/NIFI-4709 Project: Apache NiFi Issue Type: Bug Components: Extensions Affects Versions: 1.4.0 Reporter: Koji Kawamura Assignee: Koji Kawamura NIFI-4069 added target system timestamp detection for List processors. Defaults to auto detection. Most sub classes added 'Target System Timestamp Precision' to their 'getSupportedPropertyDescriptors' method. But ListAzureBlobStorage didn't. Even though 'Target System Timestamp Precision' property has a default value, if it's not included in getSupportedPropertyDescriptors method, the property value becomes null, instead of the default value. This combination is not handled well in AbstractListProcessor currently. That makes ListAzureBlobStorage behaves as if Azure Blob Storage time precision is in Minutes while it actually has Seconds precision. Incurs longer time for blob files to be picked than required. Not having 'Target System Timestamp Precision' at ListAzureBlobStorage seems reasonable as the processor interact with only Azure Blob Storage, and its timestamp precision should be fixed. AbstractListProcessor should provide an extension point for sub-classes to define default precision. In case for Azure Blob, it's SECONDS. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFICPP-346) Add bustache to bootstrap
[ https://issues.apache.org/jira/browse/MINIFICPP-346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16296014#comment-16296014 ] ASF GitHub Bot commented on MINIFICPP-346: -- GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/222 MINIFICPP-346: Add raspi distro Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFICPP-346-RPI Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/222.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #222 commit 5c9e67fff2d9487ed676c22d87dfb98d9d1a624e Author: Marc Parisi Date: 2017-12-19T01:48:28Z MINIFICPP-346: Add raspi distro > Add bustache to bootstrap > - > > Key: MINIFICPP-346 > URL: https://issues.apache.org/jira/browse/MINIFICPP-346 > Project: NiFi MiNiFi C++ > Issue Type: Bug >Reporter: marco polo >Assignee: marco polo > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi-minifi-cpp pull request #222: MINIFICPP-346: Add raspi distro
GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/222 MINIFICPP-346: Add raspi distro Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFICPP-346-RPI Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/222.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #222 commit 5c9e67fff2d9487ed676c22d87dfb98d9d1a624e Author: Marc Parisi Date: 2017-12-19T01:48:28Z MINIFICPP-346: Add raspi distro ---
[jira] [Commented] (NIFI-4005) Add support for Azure Shared Access Signature (SAS) Tokens
[ https://issues.apache.org/jira/browse/NIFI-4005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295959#comment-16295959 ] Koji Kawamura commented on NIFI-4005: - [~aperepel] I got it, I will submit another PR based on your branch. Thanks! > Add support for Azure Shared Access Signature (SAS) Tokens > -- > > Key: NIFI-4005 > URL: https://issues.apache.org/jira/browse/NIFI-4005 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.2.0 >Reporter: Andrew Grande >Assignee: Koji Kawamura >Priority: Minor > > Azure's account name and key are treated as admin, 'root' access credentials. > If one has those, every container under this account is fully accessible. An > MSFT-recommended approach is to use SAS policies, which provide for a fine > grained permission and object control, as well as defined expiration. > I already have working code, filing this ticket to formally track and submit > PR against next. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (NIFI-4005) Add support for Azure Shared Access Signature (SAS) Tokens
[ https://issues.apache.org/jira/browse/NIFI-4005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura reassigned NIFI-4005: --- Assignee: Koji Kawamura > Add support for Azure Shared Access Signature (SAS) Tokens > -- > > Key: NIFI-4005 > URL: https://issues.apache.org/jira/browse/NIFI-4005 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.2.0 >Reporter: Andrew Grande >Assignee: Koji Kawamura >Priority: Minor > > Azure's account name and key are treated as admin, 'root' access credentials. > If one has those, every container under this account is fully accessible. An > MSFT-recommended approach is to use SAS policies, which provide for a fine > grained permission and object control, as well as defined expiration. > I already have working code, filing this ticket to formally track and submit > PR against next. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4707) SiteToSiteProvenanceReportingTask does not always return component name, nor ProcessGroup ID/Name
[ https://issues.apache.org/jira/browse/NIFI-4707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-4707: --- Description: When the SiteToSiteProvenanceReportingTask emits flow files, some of them include a "componentName" field and some do not. Investigation shows that only the components (except connections) in the root process group have that field populated. Having this information can be very helpful to the user, even though the names might be duplicated, there would be a mapping between a component's ID and its name. At the very least the behavior (i.e. component name being available) should be consistent. Having a full map (by traversing the entire flow) also opens up the ability to include Process Group information for the various components. The reporting task could include the parent Process Group identifier and/or name, with perhaps a special ID for the root PG's "parent", such as "@ROOT@" or something unique. was:When the SiteToSiteProvenanceReportingTask emits flow files, some of them include a "componentName" field and some do not. Investigation shows that only the components (except connections) in the root process group have that field populated. Having this information can be very helpful to the user, even though the names might be duplicated, there would be a mapping between a component's ID and its name. At the very least the behavior (i.e. component name being available) should be consistent. > SiteToSiteProvenanceReportingTask does not always return component name, nor > ProcessGroup ID/Name > - > > Key: NIFI-4707 > URL: https://issues.apache.org/jira/browse/NIFI-4707 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess > > When the SiteToSiteProvenanceReportingTask emits flow files, some of them > include a "componentName" field and some do not. Investigation shows that > only the components (except connections) in the root process group have that > field populated. Having this information can be very helpful to the user, > even though the names might be duplicated, there would be a mapping between a > component's ID and its name. At the very least the behavior (i.e. component > name being available) should be consistent. > Having a full map (by traversing the entire flow) also opens up the ability > to include Process Group information for the various components. The > reporting task could include the parent Process Group identifier and/or name, > with perhaps a special ID for the root PG's "parent", such as "@ROOT@" or > something unique. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4707) SiteToSiteProvenanceReportingTask does not always return component name, nor ProcessGroup ID/Name
[ https://issues.apache.org/jira/browse/NIFI-4707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-4707: --- Summary: SiteToSiteProvenanceReportingTask does not always return component name, nor ProcessGroup ID/Name (was: SiteToSiteProvenanceReportingTask does not always return component name) > SiteToSiteProvenanceReportingTask does not always return component name, nor > ProcessGroup ID/Name > - > > Key: NIFI-4707 > URL: https://issues.apache.org/jira/browse/NIFI-4707 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess > > When the SiteToSiteProvenanceReportingTask emits flow files, some of them > include a "componentName" field and some do not. Investigation shows that > only the components (except connections) in the root process group have that > field populated. Having this information can be very helpful to the user, > even though the names might be duplicated, there would be a mapping between a > component's ID and its name. At the very least the behavior (i.e. component > name being available) should be consistent. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4197) *GCP processors do not expose proxy settings
[ https://issues.apache.org/jira/browse/NIFI-4197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295741#comment-16295741 ] ASF GitHub Bot commented on NIFI-4197: -- Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/2017 @jvwing Apologies for the delay I believe the feedback has been addressed. > *GCP processors do not expose proxy settings > > > Key: NIFI-4197 > URL: https://issues.apache.org/jira/browse/NIFI-4197 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Andre F de Miranda >Assignee: Andre F de Miranda > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #2017: NIFI-4197 - Expose some proxy settings to GCS Processors
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/2017 @jvwing Apologies for the delay I believe the feedback has been addressed. ---
[jira] [Created] (MINIFICPP-352) verify and resolve issues with bootstrap on RPI
marco polo created MINIFICPP-352: Summary: verify and resolve issues with bootstrap on RPI Key: MINIFICPP-352 URL: https://issues.apache.org/jira/browse/MINIFICPP-352 Project: NiFi MiNiFi C++ Issue Type: Bug Reporter: marco polo Assignee: marco polo -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (MINIFICPP-351) Remove rocksdb tools that have been added back in
[ https://issues.apache.org/jira/browse/MINIFICPP-351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] marco polo updated MINIFICPP-351: - Fix Version/s: 0.4.0 > Remove rocksdb tools that have been added back in > - > > Key: MINIFICPP-351 > URL: https://issues.apache.org/jira/browse/MINIFICPP-351 > Project: NiFi MiNiFi C++ > Issue Type: Bug >Reporter: marco polo > Fix For: 0.4.0 > > > Remove tools that were added back into the build -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFIREG-66) Create LICENSE/NOTICE for assembly
[ https://issues.apache.org/jira/browse/NIFIREG-66?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295655#comment-16295655 ] ASF GitHub Bot commented on NIFIREG-66: --- GitHub user bbende opened a pull request: https://github.com/apache/nifi-registry/pull/62 NIFIREG-66 Initial LICENSE/NOTICE for assembly & WARs Also updated the README with some basic instructions and changed the default port to 18080 to avoid the default behavior conflicting with NiFi. You can merge this pull request into a Git repository by running: $ git pull https://github.com/bbende/nifi-registry NIFIREG-66 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-registry/pull/62.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #62 commit dd8c9ce84b56c34acd825e83c250bab08e58c222 Author: Bryan Bende Date: 2017-12-11T14:31:48Z NIFIREG-66 Initial LICENSE/NOTICE for assembly & WARs > Create LICENSE/NOTICE for assembly > -- > > Key: NIFIREG-66 > URL: https://issues.apache.org/jira/browse/NIFIREG-66 > Project: NiFi Registry > Issue Type: Improvement >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Blocker > Fix For: 0.0.1 > > > Need to correctly populate the LICENSE and NOTICE in the assembly before we > can make a release. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi-registry pull request #62: NIFIREG-66 Initial LICENSE/NOTICE for assemb...
GitHub user bbende opened a pull request: https://github.com/apache/nifi-registry/pull/62 NIFIREG-66 Initial LICENSE/NOTICE for assembly & WARs Also updated the README with some basic instructions and changed the default port to 18080 to avoid the default behavior conflicting with NiFi. You can merge this pull request into a Git repository by running: $ git pull https://github.com/bbende/nifi-registry NIFIREG-66 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-registry/pull/62.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #62 commit dd8c9ce84b56c34acd825e83c250bab08e58c222 Author: Bryan Bende Date: 2017-12-11T14:31:48Z NIFIREG-66 Initial LICENSE/NOTICE for assembly & WARs ---
[jira] [Commented] (NIFI-4704) get a view of the process as a list of directories
[ https://issues.apache.org/jira/browse/NIFI-4704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295648#comment-16295648 ] Yuri commented on NIFI-4704: There seems to be a need for a more hierarchical style of interaction with PGs and/or components within PGs (eg. NIFI-4402 or NIFI-4538). A treeview described here could help. Personally, I don't know whether other components, such as processors, should be included in the tree (acting as leaves). What comes to mind is a correlation to current breadcrumb "level" and maybe also to search results. I mean using different highlights for current "level", etc. > get a view of the process as a list of directories > -- > > Key: NIFI-4704 > URL: https://issues.apache.org/jira/browse/NIFI-4704 > Project: Apache NiFi > Issue Type: Wish > Components: Core UI >Reporter: Alexandre Côté >Priority: Minor > > Hello, > It's very long to move around all our componend of Nifi (nvagiation in the > group). > It could be very usefull to get something like a directory view as we can get > one in eclipse for exemple. > We can imagine in the next step to have some information about the process in > that view (state, error, number of flow, ...), and start and stop the process > directly in the view for exemple. > Thank's, > Alex -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4689) Write Ahead Provenance Repository properties missing from nifi.properties file
[ https://issues.apache.org/jira/browse/NIFI-4689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295641#comment-16295641 ] ASF GitHub Bot commented on NIFI-4689: -- GitHub user mcgilman opened a pull request: https://github.com/apache/nifi/pull/2352 NIFI-4689: WriteAheadProvenance Properties NIFI-4689: - Ensuring all provenance properties are represented in nifi.properties. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mcgilman/nifi NIFI-4689 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2352.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2352 commit 56f8d55812841b6b361b0a1b2c08ecbeaa37e87b Author: Matt Gilman Date: 2017-12-18T20:42:45Z NIFI-4689: - Ensuring all provenance properties are represented in nifi.properties. > Write Ahead Provenance Repository properties missing from nifi.properties file > -- > > Key: NIFI-4689 > URL: https://issues.apache.org/jira/browse/NIFI-4689 > Project: Apache NiFi > Issue Type: Bug > Components: Documentation & Website >Affects Versions: 1.4.0 > Environment: N/A >Reporter: Matthew Clarke >Assignee: Matt Gilman >Priority: Minor > Labels: newbie > > The following Write Ahead Provenance Repository properties are missing from > nifi.properties file: > 1. nifi.provenance.repository.concurrent.merge.threads > 2. nifi.provenance.repository.warm.cache.frequency > These properties are documented in the NiFi admin Guide. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4689) Write Ahead Provenance Repository properties missing from nifi.properties file
[ https://issues.apache.org/jira/browse/NIFI-4689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-4689: -- Status: Patch Available (was: In Progress) > Write Ahead Provenance Repository properties missing from nifi.properties file > -- > > Key: NIFI-4689 > URL: https://issues.apache.org/jira/browse/NIFI-4689 > Project: Apache NiFi > Issue Type: Bug > Components: Documentation & Website >Affects Versions: 1.4.0 > Environment: N/A >Reporter: Matthew Clarke >Assignee: Matt Gilman >Priority: Minor > Labels: newbie > > The following Write Ahead Provenance Repository properties are missing from > nifi.properties file: > 1. nifi.provenance.repository.concurrent.merge.threads > 2. nifi.provenance.repository.warm.cache.frequency > These properties are documented in the NiFi admin Guide. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #2352: NIFI-4689: WriteAheadProvenance Properties
GitHub user mcgilman opened a pull request: https://github.com/apache/nifi/pull/2352 NIFI-4689: WriteAheadProvenance Properties NIFI-4689: - Ensuring all provenance properties are represented in nifi.properties. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mcgilman/nifi NIFI-4689 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2352.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2352 commit 56f8d55812841b6b361b0a1b2c08ecbeaa37e87b Author: Matt Gilman Date: 2017-12-18T20:42:45Z NIFI-4689: - Ensuring all provenance properties are represented in nifi.properties. ---
[jira] [Assigned] (NIFI-4689) Write Ahead Provenance Repository properties missing from nifi.properties file
[ https://issues.apache.org/jira/browse/NIFI-4689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman reassigned NIFI-4689: - Assignee: Matt Gilman > Write Ahead Provenance Repository properties missing from nifi.properties file > -- > > Key: NIFI-4689 > URL: https://issues.apache.org/jira/browse/NIFI-4689 > Project: Apache NiFi > Issue Type: Bug > Components: Documentation & Website >Affects Versions: 1.4.0 > Environment: N/A >Reporter: Matthew Clarke >Assignee: Matt Gilman >Priority: Minor > Labels: newbie > > The following Write Ahead Provenance Repository properties are missing from > nifi.properties file: > 1. nifi.provenance.repository.concurrent.merge.threads > 2. nifi.provenance.repository.warm.cache.frequency > These properties are documented in the NiFi admin Guide. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi-minifi pull request #107: MINIFI-420 Migrate Windows service executable...
Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi/pull/107 ---
[jira] [Commented] (MINIFICPP-41) Create C++ library for core API
[ https://issues.apache.org/jira/browse/MINIFICPP-41?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295517#comment-16295517 ] ASF GitHub Bot commented on MINIFICPP-41: - Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/217 > Create C++ library for core API > --- > > Key: MINIFICPP-41 > URL: https://issues.apache.org/jira/browse/MINIFICPP-41 > Project: NiFi MiNiFi C++ > Issue Type: Task >Reporter: Aldrin Piri > > To make it possible for folks to create extensions and build on top of the > MiNiFi C++ framework, the establishment of a core API library extracted from > the existing codebase that can be linked should be provided. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi-minifi-cpp pull request #217: MINIFICPP-41: First iteration of C api
Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/217 ---
[jira] [Updated] (MINIFICPP-318) Python virtualenv fails when virtualenv defaults to python3
[ https://issues.apache.org/jira/browse/MINIFICPP-318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Doran updated MINIFICPP-318: -- Resolution: Fixed Fix Version/s: 0.4.0 Status: Resolved (was: Patch Available) > Python virtualenv fails when virtualenv defaults to python3 > --- > > Key: MINIFICPP-318 > URL: https://issues.apache.org/jira/browse/MINIFICPP-318 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Kevin Doran >Assignee: Kevin Doran >Priority: Minor > Fix For: 0.4.0 > > > In docker/DockerVerify.sh, the command that creates the Python virtualenv for > running the integration tests does not specify the python executable to use. > The tests require python2, so on a machine that virtualenv defaults to > python3, the virtualenv does not work with the integration tests resulting in > runtime errors. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (NIFI-4708) Add support for NiFi Registry to the encrypt-config tool in NiFi Toolkit
Kevin Doran created NIFI-4708: - Summary: Add support for NiFi Registry to the encrypt-config tool in NiFi Toolkit Key: NIFI-4708 URL: https://issues.apache.org/jira/browse/NIFI-4708 Project: Apache NiFi Issue Type: Improvement Reporter: Kevin Doran Assignee: Kevin Doran Fix For: 1.5.0 NiFi Registry now supports loading encrypted config files (e.g., nifi-registry.properties, authorizers.xml, login-identity-providers.xml). These files are very difficult to encrypt by hand, and is not recommended. Because NiFi Registry utilizes the same encryption algorithms supported by NiFi, the easiest way to build a tool for encrypting NiFi Registry config properties is to extend the the encrypt-config tool in NiFi Toolkit to support NiFi Registry as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4707) SiteToSiteProvenanceReportingTask does not always return component name
[ https://issues.apache.org/jira/browse/NIFI-4707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-4707: --- Status: Patch Available (was: In Progress) > SiteToSiteProvenanceReportingTask does not always return component name > --- > > Key: NIFI-4707 > URL: https://issues.apache.org/jira/browse/NIFI-4707 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess > > When the SiteToSiteProvenanceReportingTask emits flow files, some of them > include a "componentName" field and some do not. Investigation shows that > only the components (except connections) in the root process group have that > field populated. Having this information can be very helpful to the user, > even though the names might be duplicated, there would be a mapping between a > component's ID and its name. At the very least the behavior (i.e. component > name being available) should be consistent. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4701) Support encrypted properties in authorizers.xml
[ https://issues.apache.org/jira/browse/NIFI-4701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Doran updated NIFI-4701: -- Status: Patch Available (was: In Progress) > Support encrypted properties in authorizers.xml > --- > > Key: NIFI-4701 > URL: https://issues.apache.org/jira/browse/NIFI-4701 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration >Reporter: Kevin Doran >Assignee: Kevin Doran > Fix For: 1.5.0 > > > Since the addition of LdapUserGroupProvider (see NIFI-4059) in v1.4.0, > authorizers.xml can now contain properties for LDAP Server credentials. > This ticket is to enable properties in authorizers.xml to be encrypted, so > that the LDAP Server Manager credentials can be protected similar to > LdapProvider which is configured via login-identity-providers.xml. > The main changes are in nifi-authorizers are: > * authorizers.xsd to add an encryption attribute to Property > * to PropertyAuthorizerFactoryBean to check for that attribute and decrypt > the property value if necessary when creating the the configuration context > Additionally, support for creating an encrypted authorizers.xml, protected by > the NiFi master key, should be added to the Encrypt Tool in NiFi Toolkit. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4707) SiteToSiteProvenanceReportingTask does not always return component name
[ https://issues.apache.org/jira/browse/NIFI-4707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295431#comment-16295431 ] ASF GitHub Bot commented on NIFI-4707: -- GitHub user mattyb149 opened a pull request: https://github.com/apache/nifi/pull/2351 NIFI-4707: Build full component map for ID -> Name association in provenance reporting Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] ~~If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? ~~ - [ ] ~~If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?~~ - [ ] ~~If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?~~ - [ ] ~~If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?~~ ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mattyb149/nifi NIFI-4707 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2351.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2351 commit 749d5b6b0251392e54d48d8f7673d2a87f364543 Author: Matthew Burgess Date: 2017-12-18T18:44:21Z NIFI-4707: Build full component map for ID -> Name association in provenance reporting" > SiteToSiteProvenanceReportingTask does not always return component name > --- > > Key: NIFI-4707 > URL: https://issues.apache.org/jira/browse/NIFI-4707 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess > > When the SiteToSiteProvenanceReportingTask emits flow files, some of them > include a "componentName" field and some do not. Investigation shows that > only the components (except connections) in the root process group have that > field populated. Having this information can be very helpful to the user, > even though the names might be duplicated, there would be a mapping between a > component's ID and its name. At the very least the behavior (i.e. component > name being available) should be consistent. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #2351: NIFI-4707: Build full component map for ID -> Name ...
GitHub user mattyb149 opened a pull request: https://github.com/apache/nifi/pull/2351 NIFI-4707: Build full component map for ID -> Name association in provenance reporting Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] ~~If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? ~~ - [ ] ~~If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?~~ - [ ] ~~If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?~~ - [ ] ~~If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?~~ ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mattyb149/nifi NIFI-4707 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2351.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2351 commit 749d5b6b0251392e54d48d8f7673d2a87f364543 Author: Matthew Burgess Date: 2017-12-18T18:44:21Z NIFI-4707: Build full component map for ID -> Name association in provenance reporting" ---
[jira] [Commented] (MINIFICPP-341) Add CentOS6 build instructions
[ https://issues.apache.org/jira/browse/MINIFICPP-341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295418#comment-16295418 ] ASF GitHub Bot commented on MINIFICPP-341: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/216 > Add CentOS6 build instructions > -- > > Key: MINIFICPP-341 > URL: https://issues.apache.org/jira/browse/MINIFICPP-341 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Andrew Christianson >Assignee: Andrew Christianson > > Add docs for building on CentOS6 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi-minifi-cpp pull request #216: MINIFICPP-341 Add CentOS6 build instructi...
Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/216 ---
[jira] [Created] (NIFI-4707) SiteToSiteProvenanceReportingTask does not always return component name
Matt Burgess created NIFI-4707: -- Summary: SiteToSiteProvenanceReportingTask does not always return component name Key: NIFI-4707 URL: https://issues.apache.org/jira/browse/NIFI-4707 Project: Apache NiFi Issue Type: Bug Components: Extensions Reporter: Matt Burgess When the SiteToSiteProvenanceReportingTask emits flow files, some of them include a "componentName" field and some do not. Investigation shows that only the components (except connections) in the root process group have that field populated. Having this information can be very helpful to the user, even though the names might be duplicated, there would be a mapping between a component's ID and its name. At the very least the behavior (i.e. component name being available) should be consistent. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (NIFI-4707) SiteToSiteProvenanceReportingTask does not always return component name
[ https://issues.apache.org/jira/browse/NIFI-4707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess reassigned NIFI-4707: -- Assignee: Matt Burgess > SiteToSiteProvenanceReportingTask does not always return component name > --- > > Key: NIFI-4707 > URL: https://issues.apache.org/jira/browse/NIFI-4707 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess > > When the SiteToSiteProvenanceReportingTask emits flow files, some of them > include a "componentName" field and some do not. Investigation shows that > only the components (except connections) in the root process group have that > field populated. Having this information can be very helpful to the user, > even though the names might be duplicated, there would be a mapping between a > component's ID and its name. At the very least the behavior (i.e. component > name being available) should be consistent. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFICPP-41) Create C++ library for core API
[ https://issues.apache.org/jira/browse/MINIFICPP-41?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295408#comment-16295408 ] ASF GitHub Bot commented on MINIFICPP-41: - Github user minifirocks commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/217 yes, +1 > Create C++ library for core API > --- > > Key: MINIFICPP-41 > URL: https://issues.apache.org/jira/browse/MINIFICPP-41 > Project: NiFi MiNiFi C++ > Issue Type: Task >Reporter: Aldrin Piri > > To make it possible for folks to create extensions and build on top of the > MiNiFi C++ framework, the establishment of a core API library extracted from > the existing codebase that can be linked should be provided. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi-minifi-cpp issue #217: MINIFICPP-41: First iteration of C api
Github user minifirocks commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/217 yes, +1 ---
[jira] [Commented] (NIFIREG-61) Add support for encrypted properties in configuration files
[ https://issues.apache.org/jira/browse/NIFIREG-61?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295390#comment-16295390 ] ASF GitHub Bot commented on NIFIREG-61: --- Github user kevdoran commented on the issue: https://github.com/apache/nifi-registry/pull/51 Thanks @alopresto - updated that log level. Also rebased off the latest master. Let me know if you see anything else to be changed. > Add support for encrypted properties in configuration files > --- > > Key: NIFIREG-61 > URL: https://issues.apache.org/jira/browse/NIFIREG-61 > Project: NiFi Registry > Issue Type: New Feature >Reporter: Kevin Doran >Assignee: Kevin Doran > Fix For: 0.0.1 > > > The NiFi Registry server is configured by files on disk, e.g., > nifi-registry.properties, bootstrap.conf, and XML files for loading > extensions. Sometimes these files contain properties with sensitive values, > such as credentials. > We want to be able to support encrypting property values in NiFi Registry > configuration files on disk that get decrypted in memory at runtime. As an > initial step, the decryption key will be specified in the bootstrap.conf > file. In the future, it might be input to the bootstrap processes via other > means. > For the design of this feature, the NiFi implementation of this capability > should be used as a guide. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi-registry issue #51: NIFIREG-61 Add support for encrypted config files
Github user kevdoran commented on the issue: https://github.com/apache/nifi-registry/pull/51 Thanks @alopresto - updated that log level. Also rebased off the latest master. Let me know if you see anything else to be changed. ---
[GitHub] nifi issue #2219: NiFi-4436: Add UI controls for starting/stopping/reverting...
Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/2219 For those playing along at home the thing fails to build because it depends on the registry. Once we get a release of the registry this will sort itself out. But if you build locally the registry then this PR is all good ---
[GitHub] nifi-minifi pull request #107: MINIFI-420 Migrate Windows service executable...
GitHub user apiri opened a pull request: https://github.com/apache/nifi-minifi/pull/107 MINIFI-420 Migrate Windows service executable handling to minifi-assembly MINIFI-420 Migrate Windows service executable handling to minifi-assembly to avoid executables in source packages. Thank you for submitting a contribution to Apache NiFi - MiNiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [X] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? - [X] Is your initial contribution a single, squashed commit? ### For code changes: - [X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi-minifi folder? - [-] Have you written or updated unit tests to verify your changes? - [-] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [-] If applicable, have you updated the LICENSE file, including the main LICENSE file under minifi-assembly? - [-] If applicable, have you updated the NOTICE file, including the main NOTICE file found under minifi-assembly? ### For documentation related changes: - [-] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/apiri/nifi-minifi MINIFI-420 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi/pull/107.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #107 commit 0603a93b9142da75f930e517d41db0fad1b3a513 Author: Aldrin Piri Date: 2017-12-18T17:49:38Z MINIFI-420 Migrate Windows service executable handling to minifi-assembly to avoid executables in source packages. ---
[GitHub] nifi-minifi-cpp issue #217: MINIFICPP-41: First iteration of C api
Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/217 @minifirocks was that a +1 ? If so I can merge it. ---
[jira] [Commented] (MINIFICPP-41) Create C++ library for core API
[ https://issues.apache.org/jira/browse/MINIFICPP-41?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295337#comment-16295337 ] ASF GitHub Bot commented on MINIFICPP-41: - Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/217 @minifirocks was that a +1 ? If so I can merge it. > Create C++ library for core API > --- > > Key: MINIFICPP-41 > URL: https://issues.apache.org/jira/browse/MINIFICPP-41 > Project: NiFi MiNiFi C++ > Issue Type: Task >Reporter: Aldrin Piri > > To make it possible for folks to create extensions and build on top of the > MiNiFi C++ framework, the establishment of a core API library extracted from > the existing codebase that can be linked should be provided. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (NIFI-3709) Export NiFi flow dataset lineage to Apache Atlas
[ https://issues.apache.org/jira/browse/NIFI-3709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne resolved NIFI-3709. -- Resolution: Fixed Fix Version/s: 1.5.0 > Export NiFi flow dataset lineage to Apache Atlas > > > Key: NIFI-3709 > URL: https://issues.apache.org/jira/browse/NIFI-3709 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Koji Kawamura >Assignee: Koji Kawamura > Fix For: 1.5.0 > > > While Apache NiFi has provenance and event level lineage support within its > data flow, Apache Atlas also does manage lineage between dataset and process > those interacting with such data. > It would be beneficial for users who use both NiFi and Atlas and if they can > see end-to-end data lineage on Atlas lineage graph, as some type of dataset > are processed by both NiFi and technologies around Atlas such as Storm, > Falcon or Sqoop. For example, Kafka topics and Hive tables. > In order to make this integration happen, I propose a NiFi reporting task > that analyzes NiFi flow then creates DataSet and Process entities in Atlas. > The challenge is how to design NiFi flow dataset level lineage within Atlas > lineage graph. > If we just add a single NiFi process and connect every DataSet from/to it, it > would be too ambiguous since it won't be clear which part of a NiFi flow > actually interact with certain dataset. > But if we put every NiFi processor as independent process in Atlas, it would > be too granular, too. Also, we already have detailed event level lineage in > NiFi, we wouldn't need the same level in Atlas. > If we can group certain processors in a NiFI flow as a process in Atlas, it > would be a nice granularity. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-3709) Export NiFi flow dataset lineage to Apache Atlas
[ https://issues.apache.org/jira/browse/NIFI-3709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295292#comment-16295292 ] ASF GitHub Bot commented on NIFI-3709: -- Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/2335 @ijokarumawak all looks good after some local testing. Thanks for your diligence and the willingness to keep going on this until we got everything resolved! I have now merged this to master! > Export NiFi flow dataset lineage to Apache Atlas > > > Key: NIFI-3709 > URL: https://issues.apache.org/jira/browse/NIFI-3709 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Koji Kawamura >Assignee: Koji Kawamura > Fix For: 1.5.0 > > > While Apache NiFi has provenance and event level lineage support within its > data flow, Apache Atlas also does manage lineage between dataset and process > those interacting with such data. > It would be beneficial for users who use both NiFi and Atlas and if they can > see end-to-end data lineage on Atlas lineage graph, as some type of dataset > are processed by both NiFi and technologies around Atlas such as Storm, > Falcon or Sqoop. For example, Kafka topics and Hive tables. > In order to make this integration happen, I propose a NiFi reporting task > that analyzes NiFi flow then creates DataSet and Process entities in Atlas. > The challenge is how to design NiFi flow dataset level lineage within Atlas > lineage graph. > If we just add a single NiFi process and connect every DataSet from/to it, it > would be too ambiguous since it won't be clear which part of a NiFi flow > actually interact with certain dataset. > But if we put every NiFi processor as independent process in Atlas, it would > be too granular, too. Also, we already have detailed event level lineage in > NiFi, we wouldn't need the same level in Atlas. > If we can group certain processors in a NiFI flow as a process in Atlas, it > would be a nice granularity. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #2335: NIFI-3709: Export NiFi flow dataset lineage to Apac...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2335 ---
[GitHub] nifi issue #2335: NIFI-3709: Export NiFi flow dataset lineage to Apache Atla...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/2335 @ijokarumawak all looks good after some local testing. Thanks for your diligence and the willingness to keep going on this until we got everything resolved! I have now merged this to master! ---
[jira] [Commented] (NIFI-3709) Export NiFi flow dataset lineage to Apache Atlas
[ https://issues.apache.org/jira/browse/NIFI-3709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295289#comment-16295289 ] ASF GitHub Bot commented on NIFI-3709: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2335 > Export NiFi flow dataset lineage to Apache Atlas > > > Key: NIFI-3709 > URL: https://issues.apache.org/jira/browse/NIFI-3709 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Koji Kawamura >Assignee: Koji Kawamura > > While Apache NiFi has provenance and event level lineage support within its > data flow, Apache Atlas also does manage lineage between dataset and process > those interacting with such data. > It would be beneficial for users who use both NiFi and Atlas and if they can > see end-to-end data lineage on Atlas lineage graph, as some type of dataset > are processed by both NiFi and technologies around Atlas such as Storm, > Falcon or Sqoop. For example, Kafka topics and Hive tables. > In order to make this integration happen, I propose a NiFi reporting task > that analyzes NiFi flow then creates DataSet and Process entities in Atlas. > The challenge is how to design NiFi flow dataset level lineage within Atlas > lineage graph. > If we just add a single NiFi process and connect every DataSet from/to it, it > would be too ambiguous since it won't be clear which part of a NiFi flow > actually interact with certain dataset. > But if we put every NiFi processor as independent process in Atlas, it would > be too granular, too. Also, we already have detailed event level lineage in > NiFi, we wouldn't need the same level in Atlas. > If we can group certain processors in a NiFI flow as a process in Atlas, it > would be a nice granularity. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-3709) Export NiFi flow dataset lineage to Apache Atlas
[ https://issues.apache.org/jira/browse/NIFI-3709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295287#comment-16295287 ] ASF subversion and git services commented on NIFI-3709: --- Commit 9750cf2fcda0209e897e98e632f7e42f3f17e3d9 in nifi's branch refs/heads/master from [~markap14] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=9750cf2 ] NIFI-3709: Added XMLENC to L&N file. Moved inclusion of nifi-atlas-nar to a profile named include-atlas. This closes #2335. > Export NiFi flow dataset lineage to Apache Atlas > > > Key: NIFI-3709 > URL: https://issues.apache.org/jira/browse/NIFI-3709 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Koji Kawamura >Assignee: Koji Kawamura > > While Apache NiFi has provenance and event level lineage support within its > data flow, Apache Atlas also does manage lineage between dataset and process > those interacting with such data. > It would be beneficial for users who use both NiFi and Atlas and if they can > see end-to-end data lineage on Atlas lineage graph, as some type of dataset > are processed by both NiFi and technologies around Atlas such as Storm, > Falcon or Sqoop. For example, Kafka topics and Hive tables. > In order to make this integration happen, I propose a NiFi reporting task > that analyzes NiFi flow then creates DataSet and Process entities in Atlas. > The challenge is how to design NiFi flow dataset level lineage within Atlas > lineage graph. > If we just add a single NiFi process and connect every DataSet from/to it, it > would be too ambiguous since it won't be clear which part of a NiFi flow > actually interact with certain dataset. > But if we put every NiFi processor as independent process in Atlas, it would > be too granular, too. Also, we already have detailed event level lineage in > NiFi, we wouldn't need the same level in Atlas. > If we can group certain processors in a NiFI flow as a process in Atlas, it > would be a nice granularity. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-3709) Export NiFi flow dataset lineage to Apache Atlas
[ https://issues.apache.org/jira/browse/NIFI-3709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295286#comment-16295286 ] ASF subversion and git services commented on NIFI-3709: --- Commit fc73c609240de81d7379bda7c281f064ebe02714 in nifi's branch refs/heads/master from [~ijokarumawak] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=fc73c60 ] NIFI-3709: Export NiFi flow dataset lineage to Apache Atlas > Export NiFi flow dataset lineage to Apache Atlas > > > Key: NIFI-3709 > URL: https://issues.apache.org/jira/browse/NIFI-3709 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Koji Kawamura >Assignee: Koji Kawamura > > While Apache NiFi has provenance and event level lineage support within its > data flow, Apache Atlas also does manage lineage between dataset and process > those interacting with such data. > It would be beneficial for users who use both NiFi and Atlas and if they can > see end-to-end data lineage on Atlas lineage graph, as some type of dataset > are processed by both NiFi and technologies around Atlas such as Storm, > Falcon or Sqoop. For example, Kafka topics and Hive tables. > In order to make this integration happen, I propose a NiFi reporting task > that analyzes NiFi flow then creates DataSet and Process entities in Atlas. > The challenge is how to design NiFi flow dataset level lineage within Atlas > lineage graph. > If we just add a single NiFi process and connect every DataSet from/to it, it > would be too ambiguous since it won't be clear which part of a NiFi flow > actually interact with certain dataset. > But if we put every NiFi processor as independent process in Atlas, it would > be too granular, too. Also, we already have detailed event level lineage in > NiFi, we wouldn't need the same level in Atlas. > If we can group certain processors in a NiFI flow as a process in Atlas, it > would be a nice granularity. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi-minifi-cpp pull request #219: MINIFICPP-330 Implemented Expression Lang...
Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/219 ---
[jira] [Commented] (NIFI-3709) Export NiFi flow dataset lineage to Apache Atlas
[ https://issues.apache.org/jira/browse/NIFI-3709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295163#comment-16295163 ] ASF GitHub Bot commented on NIFI-3709: -- Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/2335 @ijokarumawak yes I believe your understanding to be correct. Thanks for the clarifications! In terms of code I am good with the PR. Just want to test this against a live instance and as soon as I can confirm that all works well, I can push to master. However, I do see that this nar alone is > 60 MB. For now, I think I it would be best to add a Maven profile for activating this so that those who don't need this don't have to pull in the extra weight. Is that OK with you? > Export NiFi flow dataset lineage to Apache Atlas > > > Key: NIFI-3709 > URL: https://issues.apache.org/jira/browse/NIFI-3709 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Koji Kawamura >Assignee: Koji Kawamura > > While Apache NiFi has provenance and event level lineage support within its > data flow, Apache Atlas also does manage lineage between dataset and process > those interacting with such data. > It would be beneficial for users who use both NiFi and Atlas and if they can > see end-to-end data lineage on Atlas lineage graph, as some type of dataset > are processed by both NiFi and technologies around Atlas such as Storm, > Falcon or Sqoop. For example, Kafka topics and Hive tables. > In order to make this integration happen, I propose a NiFi reporting task > that analyzes NiFi flow then creates DataSet and Process entities in Atlas. > The challenge is how to design NiFi flow dataset level lineage within Atlas > lineage graph. > If we just add a single NiFi process and connect every DataSet from/to it, it > would be too ambiguous since it won't be clear which part of a NiFi flow > actually interact with certain dataset. > But if we put every NiFi processor as independent process in Atlas, it would > be too granular, too. Also, we already have detailed event level lineage in > NiFi, we wouldn't need the same level in Atlas. > If we can group certain processors in a NiFI flow as a process in Atlas, it > would be a nice granularity. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFICPP-330) Implement substring operations in Expression Language
[ https://issues.apache.org/jira/browse/MINIFICPP-330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295165#comment-16295165 ] ASF GitHub Bot commented on MINIFICPP-330: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/219 > Implement substring operations in Expression Language > - > > Key: MINIFICPP-330 > URL: https://issues.apache.org/jira/browse/MINIFICPP-330 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Andrew Christianson >Assignee: Andrew Christianson > > Add support for these substring functions to EL: > substring > substringBefore > substringBeforeLast > substringAfter > substringAfterLast -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #2335: NIFI-3709: Export NiFi flow dataset lineage to Apache Atla...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/2335 @ijokarumawak yes I believe your understanding to be correct. Thanks for the clarifications! In terms of code I am good with the PR. Just want to test this against a live instance and as soon as I can confirm that all works well, I can push to master. However, I do see that this nar alone is > 60 MB. For now, I think I it would be best to add a Maven profile for activating this so that those who don't need this don't have to pull in the extra weight. Is that OK with you? ---
[jira] [Commented] (NIFI-4698) Nifi Open ID Connect with Azure Active Directory fails to extract email from UserInfoToken
[ https://issues.apache.org/jira/browse/NIFI-4698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295140#comment-16295140 ] ASF GitHub Bot commented on NIFI-4698: -- Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/2346 @rsomas Thanks for submitting a PR! Reading through the JIRA, I wanted to get your take on possibly taking a slightly different approach as it appears that the attribute that identifies the user may differ from provider to provider. Since there does not appear to be anything in the discovery document that specifies the attribute, what are your thoughts on adding a property to the nifi.properties that does? This value would default to 'email' but could be updated to suit any value that Identity Provider supports without requiring additional code changes. > Nifi Open ID Connect with Azure Active Directory fails to extract email from > UserInfoToken > -- > > Key: NIFI-4698 > URL: https://issues.apache.org/jira/browse/NIFI-4698 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.4.0 > Environment: Microsoft Azure >Reporter: Raghu Somasundaram > Fix For: 1.5.0 > > > While integrating with Microsoft Azure Active Directory through > OpenIDConnect, Azure AD sends back Json response without "email" attribute. > This causes Nifi to throw following exception: > 2017-12-12 16:34:20,442 ERROR [NiFi Web Server-67] > org.apache.nifi.web.api.AccessResource Unable to exchange authorization for > ID token: Unable to extract email from the UserInfo token. > java.lang.IllegalStateException: Unable to extract email from the UserInfo > token. > at > org.apache.nifi.web.security.oidc.StandardOidcIdentityProvider.lookupEmail(StandardOidcIdentityProvider.java:352) > at > org.apache.nifi.web.security.oidc.StandardOidcIdentityProvider.exchangeAuthorizationCode(StandardOidcIdentityProvider.java:306) > at > org.apache.nifi.web.security.oidc.OidcService.exchangeAuthorizationCode(OidcService.java:192) > at > org.apache.nifi.web.api.AccessResource.oidcCallback(AccessResource.java:256) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) > at > com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$VoidOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:167) > at > com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) > The workaround is to extract "upn" attribute for email. This code fix checks > the json response for "email" attribute. If email is empty, it returns "upn" > value as email. If both email and upn are empty, we thrown the same exception > as above. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #2346: NIFI-4698 Extract upn if email is not in OpenID response J...
Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/2346 @rsomas Thanks for submitting a PR! Reading through the JIRA, I wanted to get your take on possibly taking a slightly different approach as it appears that the attribute that identifies the user may differ from provider to provider. Since there does not appear to be anything in the discovery document that specifies the attribute, what are your thoughts on adding a property to the nifi.properties that does? This value would default to 'email' but could be updated to suit any value that Identity Provider supports without requiring additional code changes. ---
[GitHub] nifi pull request #2350: NIFI-4701 Support encrypted authorizers.xml
GitHub user kevdoran opened a pull request: https://github.com/apache/nifi/pull/2350 NIFI-4701 Support encrypted authorizers.xml Enable properties in authorizers.xml to be encrypted by the master key. Adds authorizers.xml to the files understood by the encrypt-config tool in the NiFi Toolkit. If passed to the tool using the CLI, then the sensitive properties for LdapUserGroupProvider will be encrypted. Also fixes a bug wherein encrypt-config replaces multiple XML nodes in login-indentity-providers.xml when LdapProvider is not the first provider listed in the file. --- Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/kevdoran/nifi NIFI-4701 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2350.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2350 commit 35d69e06cd878ae22e0de142803d9d0d9112c7a0 Author: Kevin Doran Date: 2017-12-17T16:40:06Z NIFI-4701 Support encrypted authorizers.xml Enable properties in authorizers.xml to be encrypted by the master key. commit a9c496e97a8e3f4fbcee3b7c0ab260e88ecc9b19 Author: Kevin Doran Date: 2017-12-18T04:46:39Z NIFI-4701 Add authorizers.xml support to toolkit Adds authorizers.xml to the files understood by the encrypt-config tool in the NiFi Toolkit. If passed to the tool using the CLI, then the sensitive properties for LdapUserGroupProvider will be encrypted. Also fixes a bug wherein encrypt-config replaces multiple XML nodes in login-indentity-providers.xml when LdapProvider is not the first provider listed in the file. ---
[jira] [Commented] (NIFI-4701) Support encrypted properties in authorizers.xml
[ https://issues.apache.org/jira/browse/NIFI-4701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295137#comment-16295137 ] ASF GitHub Bot commented on NIFI-4701: -- GitHub user kevdoran opened a pull request: https://github.com/apache/nifi/pull/2350 NIFI-4701 Support encrypted authorizers.xml Enable properties in authorizers.xml to be encrypted by the master key. Adds authorizers.xml to the files understood by the encrypt-config tool in the NiFi Toolkit. If passed to the tool using the CLI, then the sensitive properties for LdapUserGroupProvider will be encrypted. Also fixes a bug wherein encrypt-config replaces multiple XML nodes in login-indentity-providers.xml when LdapProvider is not the first provider listed in the file. --- Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/kevdoran/nifi NIFI-4701 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2350.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2350 commit 35d69e06cd878ae22e0de142803d9d0d9112c7a0 Author: Kevin Doran Date: 2017-12-17T16:40:06Z NIFI-4701 Support encrypted authorizers.xml Enable properties in authorizers.xml to be encrypted by the master key. commit a9c496e97a8e3f4fbcee3b7c0ab260e88ecc9b19 Author: Kevin Doran Date: 2017-12-18T04:46:39Z NIFI-4701 Add authorizers.xml support to toolkit Adds authorizers.xml to the files understood by the encrypt-config tool in the NiFi Toolkit. If passed to the tool using the CLI, then the sensitive properties for LdapUserGroupProvider will be encrypted. Also fixes a bug wherein encrypt-config replaces multiple XML nodes in login-indentity-providers.xml when LdapProvider is not the first provider listed in the file. > Support encrypted properties in authorizers.xml > --- > > Key: NIFI-4701 > URL: https://issues.apache.org/jira/browse/NIFI-4701 > Project: Apache NiFi > Issue Type: Improvement > Components: Configuration >Reporter: Kevin Doran >Assignee: Kevin Doran > Fix For: 1.5.0 > > > Since the addition of LdapUserGroupProvider (see NIFI-4059) in v1.4.0, > authorizers.xml can now contain properties for LDAP Server credentials. > This ticket is to enable properties in authorizers.xml to be encrypted, so > that the LDAP Server Manager credentials can be protected similar to > LdapProvider which is configured via login-identity-providers.xml. > The main changes are in nifi-authorizers are: > * authorizers.xsd to add an encryption attribute to Property > * to PropertyAuthorizerFactoryBean to check for that attribute and decrypt > the property value if necessary when creating the the configuration context > Additionally, support for creating an encrypted authorizers.xml, protected by > t
[jira] [Commented] (NIFIREG-73) Flow Name should be unique per bucket
[ https://issues.apache.org/jira/browse/NIFIREG-73?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295119#comment-16295119 ] ASF GitHub Bot commented on NIFIREG-73: --- Github user mcgilman commented on the issue: https://github.com/apache/nifi-registry/pull/59 Thanks @bbende! This has been merged to master. > Flow Name should be unique per bucket > - > > Key: NIFIREG-73 > URL: https://issues.apache.org/jira/browse/NIFIREG-73 > Project: NiFi Registry > Issue Type: Improvement >Reporter: Bryan Bende >Assignee: Bryan Bende > Fix For: 0.0.1 > > > Currently flow name is unique across all buckets which creates confusing > behavior when there is a flow with the same name in a bucket that the current > user doesn't have access to. > We should make flow name unique per bucket. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi-registry issue #59: NIFIREG-73 Making flow/item name be unique per buck...
Github user mcgilman commented on the issue: https://github.com/apache/nifi-registry/pull/59 Thanks @bbende! This has been merged to master. ---
[jira] [Resolved] (NIFIREG-73) Flow Name should be unique per bucket
[ https://issues.apache.org/jira/browse/NIFIREG-73?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman resolved NIFIREG-73. Resolution: Fixed > Flow Name should be unique per bucket > - > > Key: NIFIREG-73 > URL: https://issues.apache.org/jira/browse/NIFIREG-73 > Project: NiFi Registry > Issue Type: Improvement >Reporter: Bryan Bende >Assignee: Bryan Bende > Fix For: 0.0.1 > > > Currently flow name is unique across all buckets which creates confusing > behavior when there is a flow with the same name in a bucket that the current > user doesn't have access to. > We should make flow name unique per bucket. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFIREG-73) Flow Name should be unique per bucket
[ https://issues.apache.org/jira/browse/NIFIREG-73?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295117#comment-16295117 ] ASF GitHub Bot commented on NIFIREG-73: --- Github user asfgit closed the pull request at: https://github.com/apache/nifi-registry/pull/59 > Flow Name should be unique per bucket > - > > Key: NIFIREG-73 > URL: https://issues.apache.org/jira/browse/NIFIREG-73 > Project: NiFi Registry > Issue Type: Improvement >Reporter: Bryan Bende >Assignee: Bryan Bende > Fix For: 0.0.1 > > > Currently flow name is unique across all buckets which creates confusing > behavior when there is a flow with the same name in a bucket that the current > user doesn't have access to. > We should make flow name unique per bucket. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi-registry pull request #59: NIFIREG-73 Making flow/item name be unique p...
Github user asfgit closed the pull request at: https://github.com/apache/nifi-registry/pull/59 ---
[jira] [Created] (MINIFICPP-351) Remove rocksdb tools that have been added back in
marco polo created MINIFICPP-351: Summary: Remove rocksdb tools that have been added back in Key: MINIFICPP-351 URL: https://issues.apache.org/jira/browse/MINIFICPP-351 Project: NiFi MiNiFi C++ Issue Type: Bug Reporter: marco polo Remove tools that were added back into the build -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4706) Bump ParCEFone version on ParseCEF
[ https://issues.apache.org/jira/browse/NIFI-4706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-4706: -- Resolution: Fixed Fix Version/s: 1.5.0 Status: Resolved (was: Patch Available) +1 merged to master > Bump ParCEFone version on ParseCEF > -- > > Key: NIFI-4706 > URL: https://issues.apache.org/jira/browse/NIFI-4706 > Project: Apache NiFi > Issue Type: Bug >Reporter: Andre F de Miranda >Assignee: Andre F de Miranda > Fix For: 1.5.0 > > > ParCEFone, the underlying CEF parser used by ParseCEF has a new version that > improves its performance and corrects a bug where some fields could be not > parsed parsed -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4706) Bump ParCEFone version on ParseCEF
[ https://issues.apache.org/jira/browse/NIFI-4706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295105#comment-16295105 ] ASF GitHub Bot commented on NIFI-4706: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2348 > Bump ParCEFone version on ParseCEF > -- > > Key: NIFI-4706 > URL: https://issues.apache.org/jira/browse/NIFI-4706 > Project: Apache NiFi > Issue Type: Bug >Reporter: Andre F de Miranda >Assignee: Andre F de Miranda > Fix For: 1.5.0 > > > ParCEFone, the underlying CEF parser used by ParseCEF has a new version that > improves its performance and corrects a bug where some fields could be not > parsed parsed -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #2348: NIFI-4706 - Bump ParCEFone version
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2348 ---
[jira] [Commented] (NIFI-4706) Bump ParCEFone version on ParseCEF
[ https://issues.apache.org/jira/browse/NIFI-4706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295104#comment-16295104 ] ASF subversion and git services commented on NIFI-4706: --- Commit 44a3ac9eff6d0e4bda193d9169989915dd716d51 in nifi's branch refs/heads/master from trixpan [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=44a3ac9 ] NIFI-4706 - This closes #2348. Bump ParCEFone version Signed-off-by: joewitt > Bump ParCEFone version on ParseCEF > -- > > Key: NIFI-4706 > URL: https://issues.apache.org/jira/browse/NIFI-4706 > Project: Apache NiFi > Issue Type: Bug >Reporter: Andre F de Miranda >Assignee: Andre F de Miranda > > ParCEFone, the underlying CEF parser used by ParseCEF has a new version that > improves its performance and corrects a bug where some fields could be not > parsed parsed -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4706) Bump ParCEFone version on ParseCEF
[ https://issues.apache.org/jira/browse/NIFI-4706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295092#comment-16295092 ] ASF GitHub Bot commented on NIFI-4706: -- Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/2348 well if travis-ci passes then obviously the issue is with me :) > Bump ParCEFone version on ParseCEF > -- > > Key: NIFI-4706 > URL: https://issues.apache.org/jira/browse/NIFI-4706 > Project: Apache NiFi > Issue Type: Bug >Reporter: Andre F de Miranda >Assignee: Andre F de Miranda > > ParCEFone, the underlying CEF parser used by ParseCEF has a new version that > improves its performance and corrects a bug where some fields could be not > parsed parsed -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #2348: NIFI-4706 - Bump ParCEFone version
Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/2348 well if travis-ci passes then obviously the issue is with me :) ---
[jira] [Commented] (NIFIREG-73) Flow Name should be unique per bucket
[ https://issues.apache.org/jira/browse/NIFIREG-73?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295080#comment-16295080 ] ASF GitHub Bot commented on NIFIREG-73: --- Github user mcgilman commented on the issue: https://github.com/apache/nifi-registry/pull/59 Will review... > Flow Name should be unique per bucket > - > > Key: NIFIREG-73 > URL: https://issues.apache.org/jira/browse/NIFIREG-73 > Project: NiFi Registry > Issue Type: Improvement >Reporter: Bryan Bende >Assignee: Bryan Bende > Fix For: 0.0.1 > > > Currently flow name is unique across all buckets which creates confusing > behavior when there is a flow with the same name in a bucket that the current > user doesn't have access to. > We should make flow name unique per bucket. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi-registry issue #59: NIFIREG-73 Making flow/item name be unique per buck...
Github user mcgilman commented on the issue: https://github.com/apache/nifi-registry/pull/59 Will review... ---
[jira] [Updated] (NIFI-4702) Grok Record Reader applies regex to each line twice in some situations
[ https://issues.apache.org/jira/browse/NIFI-4702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-4702: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Grok Record Reader applies regex to each line twice in some situations > -- > > Key: NIFI-4702 > URL: https://issues.apache.org/jira/browse/NIFI-4702 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne > Fix For: 1.5.0 > > > When using the Grok Record Reader, it reads the next line from the stream to > check if it needs to be appended to the record, is the start of a stack > trace, etc. When this is done, if the next line matches the configured > pattern, it is stored as 'nextLine' but we don't store the Map that is > obtained from calling captures(). This means that the next iteration of > nextRecord() must re-evaluate the regexes against 'nextLine'. We should > instead just keep the Map as a member variable. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (MINIFICPP-350) Fix compilation warnings
[ https://issues.apache.org/jira/browse/MINIFICPP-350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295075#comment-16295075 ] ASF GitHub Bot commented on MINIFICPP-350: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/221 > Fix compilation warnings > > > Key: MINIFICPP-350 > URL: https://issues.apache.org/jira/browse/MINIFICPP-350 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Dustin Rodrigues >Priority: Minor > > Fix compilation warnings in PutFile and Expression -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4702) Grok Record Reader applies regex to each line twice in some situations
[ https://issues.apache.org/jira/browse/NIFI-4702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295074#comment-16295074 ] ASF GitHub Bot commented on NIFI-4702: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2349 > Grok Record Reader applies regex to each line twice in some situations > -- > > Key: NIFI-4702 > URL: https://issues.apache.org/jira/browse/NIFI-4702 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne > Fix For: 1.5.0 > > > When using the Grok Record Reader, it reads the next line from the stream to > check if it needs to be appended to the record, is the start of a stack > trace, etc. When this is done, if the next line matches the configured > pattern, it is stored as 'nextLine' but we don't store the Map that is > obtained from calling captures(). This means that the next iteration of > nextRecord() must re-evaluate the regexes against 'nextLine'. We should > instead just keep the Map as a member variable. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi-minifi-cpp pull request #221: MINIFICPP-350: fix PutFile and Expression...
Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/221 ---
[GitHub] nifi pull request #2349: NIFI-4702: When we check the next line for matches ...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2349 ---
[jira] [Commented] (NIFI-4702) Grok Record Reader applies regex to each line twice in some situations
[ https://issues.apache.org/jira/browse/NIFI-4702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295072#comment-16295072 ] ASF GitHub Bot commented on NIFI-4702: -- Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/2349 +1 LGTM, ran full build with unit tests and contrib-check, verified that the behavior is maintained, and code indicates the performance improvement. Thanks for the improvement! Merging to master > Grok Record Reader applies regex to each line twice in some situations > -- > > Key: NIFI-4702 > URL: https://issues.apache.org/jira/browse/NIFI-4702 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne > Fix For: 1.5.0 > > > When using the Grok Record Reader, it reads the next line from the stream to > check if it needs to be appended to the record, is the start of a stack > trace, etc. When this is done, if the next line matches the configured > pattern, it is stored as 'nextLine' but we don't store the Map that is > obtained from calling captures(). This means that the next iteration of > nextRecord() must re-evaluate the regexes against 'nextLine'. We should > instead just keep the Map as a member variable. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4702) Grok Record Reader applies regex to each line twice in some situations
[ https://issues.apache.org/jira/browse/NIFI-4702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295073#comment-16295073 ] ASF subversion and git services commented on NIFI-4702: --- Commit 406db4867a60d830218de70e00ad7b9f9686b54a in nifi's branch refs/heads/master from [~markap14] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=406db48 ] NIFI-4702: When we check the next line for matches in Grok Reader, store the Map that is returned so that we don't have to re-evaluate the regexes the next time that nextRecord() is called Signed-off-by: Matthew Burgess This closes #2349 > Grok Record Reader applies regex to each line twice in some situations > -- > > Key: NIFI-4702 > URL: https://issues.apache.org/jira/browse/NIFI-4702 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne > Fix For: 1.5.0 > > > When using the Grok Record Reader, it reads the next line from the stream to > check if it needs to be appended to the record, is the start of a stack > trace, etc. When this is done, if the next line matches the configured > pattern, it is stored as 'nextLine' but we don't store the Map that is > obtained from calling captures(). This means that the next iteration of > nextRecord() must re-evaluate the regexes against 'nextLine'. We should > instead just keep the Map as a member variable. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #2349: NIFI-4702: When we check the next line for matches in Grok...
Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/2349 +1 LGTM, ran full build with unit tests and contrib-check, verified that the behavior is maintained, and code indicates the performance improvement. Thanks for the improvement! Merging to master ---
[GitHub] nifi issue #2349: NIFI-4702: When we check the next line for matches in Grok...
Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/2349 Reviewing... ---
[jira] [Commented] (NIFI-4702) Grok Record Reader applies regex to each line twice in some situations
[ https://issues.apache.org/jira/browse/NIFI-4702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295068#comment-16295068 ] ASF GitHub Bot commented on NIFI-4702: -- Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/2349 Reviewing... > Grok Record Reader applies regex to each line twice in some situations > -- > > Key: NIFI-4702 > URL: https://issues.apache.org/jira/browse/NIFI-4702 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne > Fix For: 1.5.0 > > > When using the Grok Record Reader, it reads the next line from the stream to > check if it needs to be appended to the record, is the start of a stack > trace, etc. When this is done, if the next line matches the configured > pattern, it is stored as 'nextLine' but we don't store the Map that is > obtained from calling captures(). This means that the next iteration of > nextRecord() must re-evaluate the regexes against 'nextLine'. We should > instead just keep the Map as a member variable. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4702) Grok Record Reader applies regex to each line twice in some situations
[ https://issues.apache.org/jira/browse/NIFI-4702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-4702: - Fix Version/s: 1.5.0 Status: Patch Available (was: Open) > Grok Record Reader applies regex to each line twice in some situations > -- > > Key: NIFI-4702 > URL: https://issues.apache.org/jira/browse/NIFI-4702 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne > Fix For: 1.5.0 > > > When using the Grok Record Reader, it reads the next line from the stream to > check if it needs to be appended to the record, is the start of a stack > trace, etc. When this is done, if the next line matches the configured > pattern, it is stored as 'nextLine' but we don't store the Map that is > obtained from calling captures(). This means that the next iteration of > nextRecord() must re-evaluate the regexes against 'nextLine'. We should > instead just keep the Map as a member variable. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4702) Grok Record Reader applies regex to each line twice in some situations
[ https://issues.apache.org/jira/browse/NIFI-4702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295059#comment-16295059 ] ASF GitHub Bot commented on NIFI-4702: -- GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/2349 NIFI-4702: When we check the next line for matches in Grok Reader, st… …ore the Map that is returned so that we don't have to re-evaluate the regexes the next time that nextRecord() is called Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-4702 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2349.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2349 commit 914833e9afdfdbe09ebfe210604a661311fa0a54 Author: Mark Payne Date: 2017-12-18T14:29:52Z NIFI-4702: When we check the next line for matches in Grok Reader, store the Map that is returned so that we don't have to re-evaluate the regexes the next time that nextRecord() is called > Grok Record Reader applies regex to each line twice in some situations > -- > > Key: NIFI-4702 > URL: https://issues.apache.org/jira/browse/NIFI-4702 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne > Fix For: 1.5.0 > > > When using the Grok Record Reader, it reads the next line from the stream to > check if it needs to be appended to the record, is the start of a stack > trace, etc. When this is done, if the next line matches the configured > pattern, it is stored as 'nextLine' but we don't store the Map that is > obtained from calling captures(). This means that the next iteration of > nextRecord() must re-evaluate the regexes against 'nextLine'. We should > instead just keep the Map as a member variable. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi pull request #2349: NIFI-4702: When we check the next line for matches ...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/2349 NIFI-4702: When we check the next line for matches in Grok Reader, st⦠â¦ore the Map that is returned so that we don't have to re-evaluate the regexes the next time that nextRecord() is called Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-4702 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2349.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2349 commit 914833e9afdfdbe09ebfe210604a661311fa0a54 Author: Mark Payne Date: 2017-12-18T14:29:52Z NIFI-4702: When we check the next line for matches in Grok Reader, store the Map that is returned so that we don't have to re-evaluate the regexes the next time that nextRecord() is called ---
[jira] [Updated] (NIFI-4706) Bump ParCEFone version on ParseCEF
[ https://issues.apache.org/jira/browse/NIFI-4706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-4706: --- Status: Patch Available (was: Open) > Bump ParCEFone version on ParseCEF > -- > > Key: NIFI-4706 > URL: https://issues.apache.org/jira/browse/NIFI-4706 > Project: Apache NiFi > Issue Type: Bug >Reporter: Andre F de Miranda >Assignee: Andre F de Miranda > > ParCEFone, the underlying CEF parser used by ParseCEF has a new version that > improves its performance and corrects a bug where some fields could be not > parsed parsed -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFIREG-30) Integrate security/user/group endpoints
[ https://issues.apache.org/jira/browse/NIFIREG-30?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295046#comment-16295046 ] ASF GitHub Bot commented on NIFIREG-30: --- Github user bbende commented on the issue: https://github.com/apache/nifi-registry/pull/61 Reviewing... > Integrate security/user/group endpoints > --- > > Key: NIFIREG-30 > URL: https://issues.apache.org/jira/browse/NIFIREG-30 > Project: NiFi Registry > Issue Type: Improvement >Reporter: Scott Aslan >Assignee: Scott Aslan > Fix For: 0.0.1 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi-registry issue #61: [NIFIREG-30] add bucket side nav
Github user bbende commented on the issue: https://github.com/apache/nifi-registry/pull/61 Reviewing... ---
[jira] [Resolved] (NIFI-4655) NPE when starting InvokeHTTP in secure cluster.
[ https://issues.apache.org/jira/browse/NIFI-4655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt resolved NIFI-4655. --- Resolution: Fixed Fix Version/s: 1.5.0 +1 merged to master > NPE when starting InvokeHTTP in secure cluster. > --- > > Key: NIFI-4655 > URL: https://issues.apache.org/jira/browse/NIFI-4655 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.4.0 > Environment: Running NiFi 1.4.0 on a small 3 node cluster (Ubuntu > 14.04.5 LTS), configured to use the OIDC for Auth. I setup the ssl/tls using > the nifi-toolkit for the cluster config. >Reporter: dan young >Assignee: Josh Anderton > Fix For: 1.5.0 > > Attachments: invoke-http-1-config.png, invoke-http-2-config.png > > > We're trying to migrate from a unsecure NiFi 1.3.0 cluster (everything > working in the NiFi 1.3 cluster) to a secure 1.4.0 cluster. When we try to > start the configured InvokeHTTP processor, we're seeing the following in the > logs: > 2017-12-04 18:18:49,701 ERROR [StandardProcessScheduler Thread-7] > org.apache.nifi.engine.FlowEngine A flow controller task execution stopped > abnormally > java.util.concurrent.ExecutionException: > java.lang.reflect.InvocationTargetException > at java.util.concurrent.FutureTask.report(FutureTask.java:122) > at java.util.concurrent.FutureTask.get(FutureTask.java:192) > at org.apache.nifi.engine.FlowEngine.afterExecute(FlowEngine.java:100) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1157) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: java.lang.reflect.InvocationTargetException: null > at sun.reflect.GeneratedMethodAccessor1028.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:137) > at > org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:125) > at > org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:70) > at > org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:47) > at > org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1306) > at > org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1302) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > ... 2 common frames omitted > Caused by: java.lang.NullPointerException: null > 2017-12-04 18:18:49,702 ERROR [StandardProcessScheduler Thread-8] > o.a.nifi.processors.standard.InvokeHTTP > InvokeHTTP[id=ae055c76-88b8-3c86-bd1e-06ca4dcb43d5] > InvokeHTTP[id=ae055c76-88b8-3c86-bd1e-06ca4dcb43d5] failed to invoke > @OnScheduled method due to java.lang.RuntimeException: Failed while executing > one of processor's OnScheduled task.; processor will not be scheduled to run > for 30 seconds: java.lang.RuntimeException: Failed while executing one of > processor's OnScheduled task. > java.lang.RuntimeException: Failed while executing one of processor's > OnScheduled task. > at > org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1483) > at > org.apache.nifi.controller.StandardProcessorNode.access$000(StandardProcessorNode.java:103) > at > org.apache.nifi.controller.StandardProcessorNode$1.run(StandardProcessorNode.java:1302) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: java.util.concurrent.ExecutionException: > j
[jira] [Commented] (NIFI-4655) NPE when starting InvokeHTTP in secure cluster.
[ https://issues.apache.org/jira/browse/NIFI-4655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295038#comment-16295038 ] ASF GitHub Bot commented on NIFI-4655: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2347 > NPE when starting InvokeHTTP in secure cluster. > --- > > Key: NIFI-4655 > URL: https://issues.apache.org/jira/browse/NIFI-4655 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.4.0 > Environment: Running NiFi 1.4.0 on a small 3 node cluster (Ubuntu > 14.04.5 LTS), configured to use the OIDC for Auth. I setup the ssl/tls using > the nifi-toolkit for the cluster config. >Reporter: dan young >Assignee: Josh Anderton > Fix For: 1.5.0 > > Attachments: invoke-http-1-config.png, invoke-http-2-config.png > > > We're trying to migrate from a unsecure NiFi 1.3.0 cluster (everything > working in the NiFi 1.3 cluster) to a secure 1.4.0 cluster. When we try to > start the configured InvokeHTTP processor, we're seeing the following in the > logs: > 2017-12-04 18:18:49,701 ERROR [StandardProcessScheduler Thread-7] > org.apache.nifi.engine.FlowEngine A flow controller task execution stopped > abnormally > java.util.concurrent.ExecutionException: > java.lang.reflect.InvocationTargetException > at java.util.concurrent.FutureTask.report(FutureTask.java:122) > at java.util.concurrent.FutureTask.get(FutureTask.java:192) > at org.apache.nifi.engine.FlowEngine.afterExecute(FlowEngine.java:100) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1157) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: java.lang.reflect.InvocationTargetException: null > at sun.reflect.GeneratedMethodAccessor1028.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:137) > at > org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:125) > at > org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:70) > at > org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:47) > at > org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1306) > at > org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1302) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > ... 2 common frames omitted > Caused by: java.lang.NullPointerException: null > 2017-12-04 18:18:49,702 ERROR [StandardProcessScheduler Thread-8] > o.a.nifi.processors.standard.InvokeHTTP > InvokeHTTP[id=ae055c76-88b8-3c86-bd1e-06ca4dcb43d5] > InvokeHTTP[id=ae055c76-88b8-3c86-bd1e-06ca4dcb43d5] failed to invoke > @OnScheduled method due to java.lang.RuntimeException: Failed while executing > one of processor's OnScheduled task.; processor will not be scheduled to run > for 30 seconds: java.lang.RuntimeException: Failed while executing one of > processor's OnScheduled task. > java.lang.RuntimeException: Failed while executing one of processor's > OnScheduled task. > at > org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1483) > at > org.apache.nifi.controller.StandardProcessorNode.access$000(StandardProcessorNode.java:103) > at > org.apache.nifi.controller.StandardProcessorNode$1.run(StandardProcessorNode.java:1302) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.T
