[jira] [Commented] (NIFI-5542) Add support for node groups to FileAccessPolicyProvider
[
https://issues.apache.org/jira/browse/NIFI-5542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593244#comment-16593244
]
ASF GitHub Bot commented on NIFI-5542:
--
Github user pepov commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2965#discussion_r212879535
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-file-authorizer/src/main/java/org/apache/nifi/authorization/FileAccessPolicyProvider.java
---
@@ -604,12 +610,26 @@ private void populateInitialAdmin(final
Authorizations authorizations) {
* @param authorizations the overall authorizations
*/
private void populateNodes(Authorizations authorizations) {
+// authorize static nodes
+authorizeNodeIdentities(authorizations, nodeIdentities);
+
+// authorize dynamic nodes (node group)
+if (nodeGroupName != null) {
+Group nodeGroup = userGroupProvider.getGroup(nodeGroupName);
+if (nodeGroup == null) {
+throw new AuthorizerCreationException("Unable to locate
node group " + nodeGroupName + " to seed policies.");
+}
+Set nodeGroupUserIdentities = nodeGroup.getUsers();
+authorizeNodeIdentities(authorizations,
nodeGroupUserIdentities);
--- End diff --
I'm not intimate with how this works, but wouldn't this just authorize the
users in the group initially and not the group itself? I mean will this
authorization include nodes added later to the group? Is there a way to do the
same authorization on the group object directly?
> Add support for node groups to FileAccessPolicyProvider
> ---
>
> Key: NIFI-5542
> URL: https://issues.apache.org/jira/browse/NIFI-5542
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Andrew Christianson
>Priority: Major
>
> Currently in FileAccessPolicyProvider, it is possible to specify a set of
> node identities, which are given access to /proxy. This works well for static
> clusters, but does not work so well for dynamic clusters (scaling up/down #
> of nodes) because we don't know in advance what the node identities will be
> or how many there will be.
> In order to support dynamic sets of node identities, add support for
> specifying a "Node Group," for which all identities in the group will be
> granted access to /proxy. A UserGroupProvider can then be implemented to
> gather node identities dynamically from the cluster environment.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2965: NIFI-5542 Added support for node groups to FileAcce...
Github user pepov commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2965#discussion_r212879535
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-file-authorizer/src/main/java/org/apache/nifi/authorization/FileAccessPolicyProvider.java
---
@@ -604,12 +610,26 @@ private void populateInitialAdmin(final
Authorizations authorizations) {
* @param authorizations the overall authorizations
*/
private void populateNodes(Authorizations authorizations) {
+// authorize static nodes
+authorizeNodeIdentities(authorizations, nodeIdentities);
+
+// authorize dynamic nodes (node group)
+if (nodeGroupName != null) {
+Group nodeGroup = userGroupProvider.getGroup(nodeGroupName);
+if (nodeGroup == null) {
+throw new AuthorizerCreationException("Unable to locate
node group " + nodeGroupName + " to seed policies.");
+}
+Set nodeGroupUserIdentities = nodeGroup.getUsers();
+authorizeNodeIdentities(authorizations,
nodeGroupUserIdentities);
--- End diff --
I'm not intimate with how this works, but wouldn't this just authorize the
users in the group initially and not the group itself? I mean will this
authorization include nodes added later to the group? Is there a way to do the
same authorization on the group object directly?
---
[jira] [Commented] (NIFI-5537) Create Neo4J cypher execution processor
[ https://issues.apache.org/jira/browse/NIFI-5537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593057#comment-16593057 ] ASF GitHub Bot commented on NIFI-5537: -- Github user mans2singh commented on the issue: https://github.com/apache/nifi/pull/2956 Hi @ottobackwards @joewitt I've renamed the bundle based on your feedback. Please let me know if you have any more comments/feedback. Thanks for your advice. > Create Neo4J cypher execution processor > --- > > Key: NIFI-5537 > URL: https://issues.apache.org/jira/browse/NIFI-5537 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.7.1 > Environment: All >Reporter: Mans Singh >Assignee: Mans Singh >Priority: Minor > Labels: graph, neo4j, node, relationship > Fix For: 1.8.0 > > > Create Nifi Neo4J cypher queries processor -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2956: NIFI-5537 Create Neo4J cypher execution processor
Github user mans2singh commented on the issue: https://github.com/apache/nifi/pull/2956 Hi @ottobackwards @joewitt I've renamed the bundle based on your feedback. Please let me know if you have any more comments/feedback. Thanks for your advice. ---
[jira] [Commented] (NIFI-5543) Connection path is not visualizing as selected when hidden linepoint element is clicked
[ https://issues.apache.org/jira/browse/NIFI-5543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593056#comment-16593056 ] Mark Bean commented on NIFI-5543: - [~mcgilman] I removed the origin CSS "fix" to the problem, and created one based on your suggestion of moving the bendPointDrag below the mousedown. Seems to work just fine. > Connection path is not visualizing as selected when hidden linepoint element > is clicked > --- > > Key: NIFI-5543 > URL: https://issues.apache.org/jira/browse/NIFI-5543 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.7.1 >Reporter: Mark Bean >Assignee: Mark Bean >Priority: Major > Attachments: NIFI5543_template.xml > > > Select a connection. Then, select another connection on its bend point. The > UI still indicates the original connection is selected. However, it is > actually the second connection that is selected - or at least the second > connection can be modified even though it is not visually highlighted. This > can lead to misinformation on multiple, overlapping connections with the same > endpoint. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5456) PutKinesisStream - Fails to work with AWS Private Link endpoint
[ https://issues.apache.org/jira/browse/NIFI-5456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16592904#comment-16592904 ] ASF GitHub Bot commented on NIFI-5456: -- GitHub user zenfenan opened a pull request: https://github.com/apache/nifi/pull/2968 NIFI-5456: AWS clients now work with private link endpoints (VPC) Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/zenfenan/nifi NIFI-5456 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2968.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2968 commit 7b390a9d0b003301f22a89cecf07a622bbc78f5e Author: zenfenan Date: 2018-08-26T13:36:45Z NIFI-5456: AWS clients now work with private link endpoints (VPC) > PutKinesisStream - Fails to work with AWS Private Link endpoint > --- > > Key: NIFI-5456 > URL: https://issues.apache.org/jira/browse/NIFI-5456 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.6.0, 1.7.1 > Environment: RedHat 6 >Reporter: Ariel Godinez >Assignee: Sivaprasanna Sethuraman >Priority: Major > Labels: easyfix > Attachments: > 0001-NIFI-5456-AWS-clients-now-work-with-private-link-end.patch > > > NiFi version: 1.6.0 > PutKinesisStream fails to put due to invalid signing information when using > an AWS Private Link as the endpoint override URL. The endpoint override URL > pattern for private links is like below along with the error that NiFi > outputs when we attempt to use this type of URL as the 'Endpoint Override > URL' property value. > Endpoint Override URL: > [https://vpce-|https://vpce-/].kinesis.us-east-2.vpce.amazonaws.com > ERROR [Timer-Driven Process Thread-11] "o.a.n.p.a.k.stream.PutKinesisStream" > PutKinesisStream[id=4c314e25-0164-1000--9bd79c77] Failed to publish > due to exception com.amazonaws.services.kinesis.model.AmazonKinesisException: > Credential should be scoped to a valid region, not 'vpce'. (Service: > AmazonKinesis; Status Code: 400; Error Code: InvalidSignatureException; > Request ID: 6330b83c-a64e-4acf-b892-a505621cf78e) flowfiles > [StandardFlowFileRecord[uuid=ba299cec-7cbf-4750-a766-c348b5cd9c73,claim=StandardContentClaim > [resourceClaim=StandardResourceClaim[id=1532469012962-1, > container=content002, section=1], offset=2159750, > length=534625],offset=0,name=900966573101260,size=534625]] > > It looks like 'vpce' is being extracted from the url as the region name when > it should be getting 'us-east-2'. We were able to get this processor to work > correctly by explicitly passing in the region and service using > 'setEndpoint(String endpoint, String serviceName, String regionId)' instead > of 'setEndpoint(String endpoint)' in > 'nifi/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-abstract-processors/src/main/java/org/apa
[GitHub] nifi pull request #2968: NIFI-5456: AWS clients now work with private link e...
GitHub user zenfenan opened a pull request: https://github.com/apache/nifi/pull/2968 NIFI-5456: AWS clients now work with private link endpoints (VPC) Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/zenfenan/nifi NIFI-5456 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2968.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2968 commit 7b390a9d0b003301f22a89cecf07a622bbc78f5e Author: zenfenan Date: 2018-08-26T13:36:45Z NIFI-5456: AWS clients now work with private link endpoints (VPC) ---
[jira] [Commented] (NIFI-5456) PutKinesisStream - Fails to work with AWS Private Link endpoint
[ https://issues.apache.org/jira/browse/NIFI-5456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16592900#comment-16592900 ] Sivaprasanna Sethuraman commented on NIFI-5456: --- [~randy_b] Yep. Valid point. And sorry for the late response. Was held up with a couple of other nagging issues. I have attached a patch for this. Can you please test it and verify whether it is working correctly. I don't currently have a setup with VPC so can't validate it myself. Appreciate if you could verify and validate this patch. > PutKinesisStream - Fails to work with AWS Private Link endpoint > --- > > Key: NIFI-5456 > URL: https://issues.apache.org/jira/browse/NIFI-5456 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.6.0, 1.7.1 > Environment: RedHat 6 >Reporter: Ariel Godinez >Assignee: Sivaprasanna Sethuraman >Priority: Major > Labels: easyfix > Attachments: > 0001-NIFI-5456-AWS-clients-now-work-with-private-link-end.patch > > > NiFi version: 1.6.0 > PutKinesisStream fails to put due to invalid signing information when using > an AWS Private Link as the endpoint override URL. The endpoint override URL > pattern for private links is like below along with the error that NiFi > outputs when we attempt to use this type of URL as the 'Endpoint Override > URL' property value. > Endpoint Override URL: > [https://vpce-|https://vpce-/].kinesis.us-east-2.vpce.amazonaws.com > ERROR [Timer-Driven Process Thread-11] "o.a.n.p.a.k.stream.PutKinesisStream" > PutKinesisStream[id=4c314e25-0164-1000--9bd79c77] Failed to publish > due to exception com.amazonaws.services.kinesis.model.AmazonKinesisException: > Credential should be scoped to a valid region, not 'vpce'. (Service: > AmazonKinesis; Status Code: 400; Error Code: InvalidSignatureException; > Request ID: 6330b83c-a64e-4acf-b892-a505621cf78e) flowfiles > [StandardFlowFileRecord[uuid=ba299cec-7cbf-4750-a766-c348b5cd9c73,claim=StandardContentClaim > [resourceClaim=StandardResourceClaim[id=1532469012962-1, > container=content002, section=1], offset=2159750, > length=534625],offset=0,name=900966573101260,size=534625]] > > It looks like 'vpce' is being extracted from the url as the region name when > it should be getting 'us-east-2'. We were able to get this processor to work > correctly by explicitly passing in the region and service using > 'setEndpoint(String endpoint, String serviceName, String regionId)' instead > of 'setEndpoint(String endpoint)' in > 'nifi/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-abstract-processors/src/main/java/org/apache/nifi/processors/aws/AbstractAWSProcessor.java' > line 289 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-5456) PutKinesisStream - Fails to work with AWS Private Link endpoint
[ https://issues.apache.org/jira/browse/NIFI-5456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sivaprasanna Sethuraman updated NIFI-5456: -- Attachment: 0001-NIFI-5456-AWS-clients-now-work-with-private-link-end.patch Status: Patch Available (was: In Progress) > PutKinesisStream - Fails to work with AWS Private Link endpoint > --- > > Key: NIFI-5456 > URL: https://issues.apache.org/jira/browse/NIFI-5456 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.7.1, 1.6.0 > Environment: RedHat 6 >Reporter: Ariel Godinez >Assignee: Sivaprasanna Sethuraman >Priority: Major > Labels: easyfix > Attachments: > 0001-NIFI-5456-AWS-clients-now-work-with-private-link-end.patch > > > NiFi version: 1.6.0 > PutKinesisStream fails to put due to invalid signing information when using > an AWS Private Link as the endpoint override URL. The endpoint override URL > pattern for private links is like below along with the error that NiFi > outputs when we attempt to use this type of URL as the 'Endpoint Override > URL' property value. > Endpoint Override URL: > [https://vpce-|https://vpce-/].kinesis.us-east-2.vpce.amazonaws.com > ERROR [Timer-Driven Process Thread-11] "o.a.n.p.a.k.stream.PutKinesisStream" > PutKinesisStream[id=4c314e25-0164-1000--9bd79c77] Failed to publish > due to exception com.amazonaws.services.kinesis.model.AmazonKinesisException: > Credential should be scoped to a valid region, not 'vpce'. (Service: > AmazonKinesis; Status Code: 400; Error Code: InvalidSignatureException; > Request ID: 6330b83c-a64e-4acf-b892-a505621cf78e) flowfiles > [StandardFlowFileRecord[uuid=ba299cec-7cbf-4750-a766-c348b5cd9c73,claim=StandardContentClaim > [resourceClaim=StandardResourceClaim[id=1532469012962-1, > container=content002, section=1], offset=2159750, > length=534625],offset=0,name=900966573101260,size=534625]] > > It looks like 'vpce' is being extracted from the url as the region name when > it should be getting 'us-east-2'. We were able to get this processor to work > correctly by explicitly passing in the region and service using > 'setEndpoint(String endpoint, String serviceName, String regionId)' instead > of 'setEndpoint(String endpoint)' in > 'nifi/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-abstract-processors/src/main/java/org/apache/nifi/processors/aws/AbstractAWSProcessor.java' > line 289 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (NIFI-5546) PutAzureBlobStorage not able to configure
[
https://issues.apache.org/jira/browse/NIFI-5546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sivaprasanna Sethuraman closed NIFI-5546.
-
> PutAzureBlobStorage not able to configure
> -
>
> Key: NIFI-5546
> URL: https://issues.apache.org/jira/browse/NIFI-5546
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core UI
>Affects Versions: 1.6.0, 1.7.0
>Reporter: Rakesh Prasad
>Priority: Major
>
> to configure PutAzureBlobStorage processor we have 2 properties that are
> marked as not null.
> ACCOUNT_KEY = NON_EMPTY_VALIDATOR
> PROP_SAS_TOKEN = NON_EMPTY_VALIDATOR
> but inside "AzureStorageUtils" class we have additional code that is asking
> for either one on the value, but not both.
> String sasToken = validationContext.getProperty(PROP_SAS_TOKEN).getValue();
> String acctName = validationContext.getProperty(ACCOUNT_KEY).getValue();
> if ((StringUtils.isBlank(sasToken) && StringUtils.isBlank(acctName))
> || (StringUtils.isNotBlank(sasToken) && StringUtils.isNotBlank(acctName))) {
> results.add(new ValidationResult.Builder().subject("AzureStorageUtils
> Credentials")
> .valid(false)
> .explanation("either Azure Account Key or Shared Access Signature required,
> but not both")
> .build());
> }
>
> because of this, not able to use this processor.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5546) PutAzureBlobStorage not able to configure
[
https://issues.apache.org/jira/browse/NIFI-5546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16592892#comment-16592892
]
Sivaprasanna Sethuraman commented on NIFI-5546:
---
[~rakeshsilswal]
This is actually not a bug. Azure Storage account can be accessed by two ways:
# Storage account name & storage account key - This key is the master key. A
person who has this combination can do any operations on this storage account
# SAS Token - This provides more granular access control. You can create SAS
token to only read from a storage account.
When you are working SAS, you don't need a storage account key and vice versa,
but in both cases you would need the Storage account name and that is why the
properties are defined and validated like that.
> PutAzureBlobStorage not able to configure
> -
>
> Key: NIFI-5546
> URL: https://issues.apache.org/jira/browse/NIFI-5546
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core UI
>Affects Versions: 1.6.0, 1.7.0
>Reporter: Rakesh Prasad
>Priority: Major
>
> to configure PutAzureBlobStorage processor we have 2 properties that are
> marked as not null.
> ACCOUNT_KEY = NON_EMPTY_VALIDATOR
> PROP_SAS_TOKEN = NON_EMPTY_VALIDATOR
> but inside "AzureStorageUtils" class we have additional code that is asking
> for either one on the value, but not both.
> String sasToken = validationContext.getProperty(PROP_SAS_TOKEN).getValue();
> String acctName = validationContext.getProperty(ACCOUNT_KEY).getValue();
> if ((StringUtils.isBlank(sasToken) && StringUtils.isBlank(acctName))
> || (StringUtils.isNotBlank(sasToken) && StringUtils.isNotBlank(acctName))) {
> results.add(new ValidationResult.Builder().subject("AzureStorageUtils
> Credentials")
> .valid(false)
> .explanation("either Azure Account Key or Shared Access Signature required,
> but not both")
> .build());
> }
>
> because of this, not able to use this processor.
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-280) Move extension tests and test headers, lint extensions
[ https://issues.apache.org/jira/browse/MINIFICPP-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16592830#comment-16592830 ] ASF GitHub Bot commented on MINIFICPP-280: -- Github user calebj commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/168 @phrocker Sure thing. I'm not with the project that needed minifi anymore, but I'm still interested in seeing these changes implemented. > Move extension tests and test headers, lint extensions > -- > > Key: MINIFICPP-280 > URL: https://issues.apache.org/jira/browse/MINIFICPP-280 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Caleb Johnson >Assignee: Caleb Johnson >Priority: Minor > > * move extension tests into their respective folders > * separate source and header files > * remove unnecessary or nonexisting include directories > * run linter on extension source files as part of linter target > * clean up extensions according to linter > * add ability to specify more than one include and source folder for linter -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp issue #168: MINIFICPP-280 Move extension tests and test head...
Github user calebj commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/168 @phrocker Sure thing. I'm not with the project that needed minifi anymore, but I'm still interested in seeing these changes implemented. ---
