[jira] [Commented] (NIFI-786) Add other supporting options for configuring credentials for AWS processors
[
https://issues.apache.org/jira/browse/NIFI-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15297690#comment-15297690
]
ASF GitHub Bot commented on NIFI-786:
-
Github user jvwing commented on the pull request:
https://github.com/apache/nifi/pull/244#issuecomment-221167882
@jskora, thanks for your help reviewing and merging.
> Add other supporting options for configuring credentials for AWS processors
> ---
>
> Key: NIFI-786
> URL: https://issues.apache.org/jira/browse/NIFI-786
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 0.3.0
>Reporter: Michael Kobit
>Assignee: James Wing
>Priority: Minor
> Fix For: 0.7.0
>
>
> I was looking at https://issues.apache.org/jira/browse/NIFI-770 and looked at
> how the AWS processors credentials are currently configured. As a NFM you
> have a few options with the properties right now:
> 1) set basic, static credentials
> 2) set a credentials properties filepath
> 3) set neither, use anonymous credentials
> I think it would be better if each AWS could rely on a ControllerService that
> returns `AWSCredentialsProvider` (instead of `AWSCredentials`) that gives
> all of the possible implementations that could be used, rather than relying
> on a static credentials. *Provider implementations can be refreshed and can
> also other more complicated implementations, but already have built in
> support for the Static and Properties file that are provided by NiFi today.
> My thinking is that the controller service would be something like
> public interface AwsCredentialsProviderService extends ControllerService {
> AWSCredentialsProvider getCredentialsProvider();
> }
> and you could have `StaticAwsCredentialsProviderService`,
> `PropertiesFileAwsCredentialsProviderService`, and
> `AnonymousAwsCredentialsProviderService` to provide the functionality that is
> supported right now. Additional credential providers could be added later, as
> there a bunch more AWS provided versions that I think could fit in well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-786) Add other supporting options for configuring credentials for AWS processors
[
https://issues.apache.org/jira/browse/NIFI-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15297662#comment-15297662
]
ASF GitHub Bot commented on NIFI-786:
-
Github user jskora commented on the pull request:
https://github.com/apache/nifi/pull/244#issuecomment-221163835
This is really nice, it was great being able to just provide the AWS CLI
profile and have setting automatically picked up.
Great improvement, thanks James!
> Add other supporting options for configuring credentials for AWS processors
> ---
>
> Key: NIFI-786
> URL: https://issues.apache.org/jira/browse/NIFI-786
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 0.3.0
>Reporter: Michael Kobit
>Assignee: James Wing
>Priority: Minor
> Fix For: 0.7.0
>
>
> I was looking at https://issues.apache.org/jira/browse/NIFI-770 and looked at
> how the AWS processors credentials are currently configured. As a NFM you
> have a few options with the properties right now:
> 1) set basic, static credentials
> 2) set a credentials properties filepath
> 3) set neither, use anonymous credentials
> I think it would be better if each AWS could rely on a ControllerService that
> returns `AWSCredentialsProvider` (instead of `AWSCredentials`) that gives
> all of the possible implementations that could be used, rather than relying
> on a static credentials. *Provider implementations can be refreshed and can
> also other more complicated implementations, but already have built in
> support for the Static and Properties file that are provided by NiFi today.
> My thinking is that the controller service would be something like
> public interface AwsCredentialsProviderService extends ControllerService {
> AWSCredentialsProvider getCredentialsProvider();
> }
> and you could have `StaticAwsCredentialsProviderService`,
> `PropertiesFileAwsCredentialsProviderService`, and
> `AnonymousAwsCredentialsProviderService` to provide the functionality that is
> supported right now. Additional credential providers could be added later, as
> there a bunch more AWS provided versions that I think could fit in well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-786) Add other supporting options for configuring credentials for AWS processors
[
https://issues.apache.org/jira/browse/NIFI-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15297656#comment-15297656
]
ASF GitHub Bot commented on NIFI-786:
-
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/244
> Add other supporting options for configuring credentials for AWS processors
> ---
>
> Key: NIFI-786
> URL: https://issues.apache.org/jira/browse/NIFI-786
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 0.3.0
>Reporter: Michael Kobit
>Assignee: James Wing
>Priority: Minor
> Fix For: 0.7.0
>
>
> I was looking at https://issues.apache.org/jira/browse/NIFI-770 and looked at
> how the AWS processors credentials are currently configured. As a NFM you
> have a few options with the properties right now:
> 1) set basic, static credentials
> 2) set a credentials properties filepath
> 3) set neither, use anonymous credentials
> I think it would be better if each AWS could rely on a ControllerService that
> returns `AWSCredentialsProvider` (instead of `AWSCredentials`) that gives
> all of the possible implementations that could be used, rather than relying
> on a static credentials. *Provider implementations can be refreshed and can
> also other more complicated implementations, but already have built in
> support for the Static and Properties file that are provided by NiFi today.
> My thinking is that the controller service would be something like
> public interface AwsCredentialsProviderService extends ControllerService {
> AWSCredentialsProvider getCredentialsProvider();
> }
> and you could have `StaticAwsCredentialsProviderService`,
> `PropertiesFileAwsCredentialsProviderService`, and
> `AnonymousAwsCredentialsProviderService` to provide the functionality that is
> supported right now. Additional credential providers could be added later, as
> there a bunch more AWS provided versions that I think could fit in well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-786) Add other supporting options for configuring credentials for AWS processors
[
https://issues.apache.org/jira/browse/NIFI-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15297655#comment-15297655
]
ASF subversion and git services commented on NIFI-786:
--
Commit 9bb6dcb5e58e3557a8545b97fa9ceef26124903c in nifi's branch
refs/heads/master from [~jameswing]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=9bb6dcb ]
NIFI-786 AWS credential refactoring and enhancements
Signed-off-by: Joe Skora
This closes #244.
> Add other supporting options for configuring credentials for AWS processors
> ---
>
> Key: NIFI-786
> URL: https://issues.apache.org/jira/browse/NIFI-786
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 0.3.0
>Reporter: Michael Kobit
>Assignee: James Wing
>Priority: Minor
> Fix For: 0.7.0
>
>
> I was looking at https://issues.apache.org/jira/browse/NIFI-770 and looked at
> how the AWS processors credentials are currently configured. As a NFM you
> have a few options with the properties right now:
> 1) set basic, static credentials
> 2) set a credentials properties filepath
> 3) set neither, use anonymous credentials
> I think it would be better if each AWS could rely on a ControllerService that
> returns `AWSCredentialsProvider` (instead of `AWSCredentials`) that gives
> all of the possible implementations that could be used, rather than relying
> on a static credentials. *Provider implementations can be refreshed and can
> also other more complicated implementations, but already have built in
> support for the Static and Properties file that are provided by NiFi today.
> My thinking is that the controller service would be something like
> public interface AwsCredentialsProviderService extends ControllerService {
> AWSCredentialsProvider getCredentialsProvider();
> }
> and you could have `StaticAwsCredentialsProviderService`,
> `PropertiesFileAwsCredentialsProviderService`, and
> `AnonymousAwsCredentialsProviderService` to provide the functionality that is
> supported right now. Additional credential providers could be added later, as
> there a bunch more AWS provided versions that I think could fit in well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
nifi git commit: NIFI-786 AWS credential refactoring and enhancements
Repository: nifi
Updated Branches:
refs/heads/master 4dd50c80a -> 9bb6dcb5e
NIFI-786 AWS credential refactoring and enhancements
Signed-off-by: Joe Skora
This closes #244.
Project: http://git-wip-us.apache.org/repos/asf/nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/9bb6dcb5
Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/9bb6dcb5
Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/9bb6dcb5
Branch: refs/heads/master
Commit: 9bb6dcb5e58e3557a8545b97fa9ceef26124903c
Parents: 4dd50c8
Author: James Wing
Authored: Mon Feb 22 10:14:47 2016 -0800
Committer: Joe Skora
Committed: Tue May 24 00:27:13 2016 -0400
--
.../processors/aws/AbstractAWSProcessor.java| 26 +--
.../factory/CredentialPropertyDescriptors.java | 158 +++
.../factory/CredentialsProviderFactory.java | 125
.../provider/factory/CredentialsStrategy.java | 75 +++
.../AbstractBooleanCredentialsStrategy.java | 70 +++
.../strategies/AbstractCredentialsStrategy.java | 101 ++
.../AccessKeyPairCredentialsStrategy.java | 52 +
.../AnonymousCredentialsStrategy.java | 47 +
.../AssumeRoleCredentialsStrategy.java | 128
.../ExplicitDefaultCredentialsStrategy.java | 46 +
.../strategies/FileCredentialsStrategy.java | 53 +
.../ImplicitDefaultCredentialsStrategy.java | 43
.../NamedProfileCredentialsStrategy.java| 48 +
...AWSCredentialsProviderControllerService.java | 137 +++--
.../provider/factory/MockAWSProcessor.java | 104 ++
.../factory/TestCredentialsProviderFactory.java | 201 +++
16 files changed, 1283 insertions(+), 131 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/nifi/blob/9bb6dcb5/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/AbstractAWSProcessor.java
--
diff --git
a/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/AbstractAWSProcessor.java
b/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/AbstractAWSProcessor.java
index e2c2196..3d4e458 100644
---
a/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/AbstractAWSProcessor.java
+++
b/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/AbstractAWSProcessor.java
@@ -41,6 +41,7 @@ import org.apache.nifi.processor.ProcessContext;
import org.apache.nifi.processor.Relationship;
import org.apache.nifi.processor.exception.ProcessException;
import org.apache.nifi.processor.util.StandardValidators;
+import
org.apache.nifi.processors.aws.credentials.provider.factory.CredentialPropertyDescriptors;
import org.apache.nifi.ssl.SSLContextService;
import com.amazonaws.AmazonWebServiceClient;
@@ -72,28 +73,9 @@ public abstract class AbstractAWSProcessor relationships =
Collections.unmodifiableSet(
new HashSet<>(Arrays.asList(REL_SUCCESS, REL_FAILURE)));
-public static final PropertyDescriptor CREDENTIALS_FILE = new
PropertyDescriptor.Builder()
-.name("Credentials File")
-.expressionLanguageSupported(false)
-.required(false)
-.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
-.build();
-
-public static final PropertyDescriptor ACCESS_KEY = new
PropertyDescriptor.Builder()
-.name("Access Key")
-.expressionLanguageSupported(true)
-.required(false)
-.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
-.sensitive(true)
-.build();
-
-public static final PropertyDescriptor SECRET_KEY = new
PropertyDescriptor.Builder()
-.name("Secret Key")
-.expressionLanguageSupported(true)
-.required(false)
-.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
-.sensitive(true)
-.build();
+public static final PropertyDescriptor CREDENTIALS_FILE =
CredentialPropertyDescriptors.CREDENTIALS_FILE;
+public static final PropertyDescriptor ACCESS_KEY =
CredentialPropertyDescriptors.ACCESS_KEY;
+public static final PropertyDescriptor SECRET_KEY =
CredentialPropertyDescriptors.SECRET_KEY;
public static final PropertyDescriptor PROXY_HOST = new
PropertyDescriptor.Builder()
.name("Proxy Host")
http://git-wip-us.apache.org/repos/asf/nifi/blob/9bb6dcb5/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/credentials/provider/factory/CredentialPropertyDescriptors.java
--
diff -
[jira] [Commented] (NIFI-786) Add other supporting options for configuring credentials for AWS processors
[
https://issues.apache.org/jira/browse/NIFI-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15297653#comment-15297653
]
ASF subversion and git services commented on NIFI-786:
--
Commit de7ecd719a2e9907042628ea3a8283cfe2d4fbac in nifi's branch refs/heads/0.x
from [~jameswing]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=de7ecd7 ]
NIFI-786 AWS credential refactoring and enhancements
Signed-off-by: Joe Skora
This closes #244.
> Add other supporting options for configuring credentials for AWS processors
> ---
>
> Key: NIFI-786
> URL: https://issues.apache.org/jira/browse/NIFI-786
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 0.3.0
>Reporter: Michael Kobit
>Assignee: James Wing
>Priority: Minor
> Fix For: 0.7.0
>
>
> I was looking at https://issues.apache.org/jira/browse/NIFI-770 and looked at
> how the AWS processors credentials are currently configured. As a NFM you
> have a few options with the properties right now:
> 1) set basic, static credentials
> 2) set a credentials properties filepath
> 3) set neither, use anonymous credentials
> I think it would be better if each AWS could rely on a ControllerService that
> returns `AWSCredentialsProvider` (instead of `AWSCredentials`) that gives
> all of the possible implementations that could be used, rather than relying
> on a static credentials. *Provider implementations can be refreshed and can
> also other more complicated implementations, but already have built in
> support for the Static and Properties file that are provided by NiFi today.
> My thinking is that the controller service would be something like
> public interface AwsCredentialsProviderService extends ControllerService {
> AWSCredentialsProvider getCredentialsProvider();
> }
> and you could have `StaticAwsCredentialsProviderService`,
> `PropertiesFileAwsCredentialsProviderService`, and
> `AnonymousAwsCredentialsProviderService` to provide the functionality that is
> supported right now. Additional credential providers could be added later, as
> there a bunch more AWS provided versions that I think could fit in well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
nifi git commit: NIFI-786 AWS credential refactoring and enhancements
Repository: nifi
Updated Branches:
refs/heads/0.x 718212aac -> de7ecd719
NIFI-786 AWS credential refactoring and enhancements
Signed-off-by: Joe Skora
This closes #244.
Project: http://git-wip-us.apache.org/repos/asf/nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/de7ecd71
Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/de7ecd71
Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/de7ecd71
Branch: refs/heads/0.x
Commit: de7ecd719a2e9907042628ea3a8283cfe2d4fbac
Parents: 718212a
Author: James Wing
Authored: Mon Feb 22 10:14:47 2016 -0800
Committer: Joe Skora
Committed: Tue May 24 00:23:04 2016 -0400
--
.../processors/aws/AbstractAWSProcessor.java| 26 +--
.../factory/CredentialPropertyDescriptors.java | 158 +++
.../factory/CredentialsProviderFactory.java | 125
.../provider/factory/CredentialsStrategy.java | 75 +++
.../AbstractBooleanCredentialsStrategy.java | 70 +++
.../strategies/AbstractCredentialsStrategy.java | 101 ++
.../AccessKeyPairCredentialsStrategy.java | 52 +
.../AnonymousCredentialsStrategy.java | 47 +
.../AssumeRoleCredentialsStrategy.java | 128
.../ExplicitDefaultCredentialsStrategy.java | 46 +
.../strategies/FileCredentialsStrategy.java | 53 +
.../ImplicitDefaultCredentialsStrategy.java | 43
.../NamedProfileCredentialsStrategy.java| 48 +
...AWSCredentialsProviderControllerService.java | 137 +++--
.../provider/factory/MockAWSProcessor.java | 104 ++
.../factory/TestCredentialsProviderFactory.java | 201 +++
16 files changed, 1283 insertions(+), 131 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/nifi/blob/de7ecd71/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/AbstractAWSProcessor.java
--
diff --git
a/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/AbstractAWSProcessor.java
b/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/AbstractAWSProcessor.java
index e2c2196..3d4e458 100644
---
a/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/AbstractAWSProcessor.java
+++
b/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/AbstractAWSProcessor.java
@@ -41,6 +41,7 @@ import org.apache.nifi.processor.ProcessContext;
import org.apache.nifi.processor.Relationship;
import org.apache.nifi.processor.exception.ProcessException;
import org.apache.nifi.processor.util.StandardValidators;
+import
org.apache.nifi.processors.aws.credentials.provider.factory.CredentialPropertyDescriptors;
import org.apache.nifi.ssl.SSLContextService;
import com.amazonaws.AmazonWebServiceClient;
@@ -72,28 +73,9 @@ public abstract class AbstractAWSProcessor relationships =
Collections.unmodifiableSet(
new HashSet<>(Arrays.asList(REL_SUCCESS, REL_FAILURE)));
-public static final PropertyDescriptor CREDENTIALS_FILE = new
PropertyDescriptor.Builder()
-.name("Credentials File")
-.expressionLanguageSupported(false)
-.required(false)
-.addValidator(StandardValidators.FILE_EXISTS_VALIDATOR)
-.build();
-
-public static final PropertyDescriptor ACCESS_KEY = new
PropertyDescriptor.Builder()
-.name("Access Key")
-.expressionLanguageSupported(true)
-.required(false)
-.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
-.sensitive(true)
-.build();
-
-public static final PropertyDescriptor SECRET_KEY = new
PropertyDescriptor.Builder()
-.name("Secret Key")
-.expressionLanguageSupported(true)
-.required(false)
-.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
-.sensitive(true)
-.build();
+public static final PropertyDescriptor CREDENTIALS_FILE =
CredentialPropertyDescriptors.CREDENTIALS_FILE;
+public static final PropertyDescriptor ACCESS_KEY =
CredentialPropertyDescriptors.ACCESS_KEY;
+public static final PropertyDescriptor SECRET_KEY =
CredentialPropertyDescriptors.SECRET_KEY;
public static final PropertyDescriptor PROXY_HOST = new
PropertyDescriptor.Builder()
.name("Proxy Host")
http://git-wip-us.apache.org/repos/asf/nifi/blob/de7ecd71/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/credentials/provider/factory/CredentialPropertyDescriptors.java
--
diff --git
[jira] [Assigned] (NIFI-1915) ReplaceText infinite loops when attribute contains $ sign
[ https://issues.apache.org/jira/browse/NIFI-1915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Zhurakousky reassigned NIFI-1915: -- Assignee: Oleg Zhurakousky > ReplaceText infinite loops when attribute contains $ sign > - > > Key: NIFI-1915 > URL: https://issues.apache.org/jira/browse/NIFI-1915 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.6.1 >Reporter: Stephane Maarek >Assignee: Oleg Zhurakousky >Priority: Critical > Fix For: 1.0.0, 0.7.0 > > > I think the biggest issue is that the text inside of an attribute isn't > properly escaped when written to a String, which brings conflict when the > text contains dollar signs ($) > That's a big roadblock for me as I can't predict if and when some $ signs may > be present in the data > An easy way to reproduce is to take the csv to json template here: > https://cwiki.apache.org/confluence/download/attachments/57904847/CsvToJSON.xml?version=1&modificationDate=1442927496000&api=v2 > > In the first ReplaceText, replace a,b,c,d by a$a,b,c,d (it simulates data > that may or may not contain a $ sign) > Launch the flow, you'll see the errors / warning / infinite loop -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1915) ReplaceText infinite loops when attribute contains $ sign
[ https://issues.apache.org/jira/browse/NIFI-1915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Zhurakousky updated NIFI-1915: --- Fix Version/s: 0.7.0 1.0.0 > ReplaceText infinite loops when attribute contains $ sign > - > > Key: NIFI-1915 > URL: https://issues.apache.org/jira/browse/NIFI-1915 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.6.1 >Reporter: Stephane Maarek >Assignee: Oleg Zhurakousky >Priority: Critical > Fix For: 1.0.0, 0.7.0 > > > I think the biggest issue is that the text inside of an attribute isn't > properly escaped when written to a String, which brings conflict when the > text contains dollar signs ($) > That's a big roadblock for me as I can't predict if and when some $ signs may > be present in the data > An easy way to reproduce is to take the csv to json template here: > https://cwiki.apache.org/confluence/download/attachments/57904847/CsvToJSON.xml?version=1&modificationDate=1442927496000&api=v2 > > In the first ReplaceText, replace a,b,c,d by a$a,b,c,d (it simulates data > that may or may not contain a $ sign) > Launch the flow, you'll see the errors / warning / infinite loop -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-1915) ReplaceText infinite loops when attribute contains $ sign
Stephane Maarek created NIFI-1915: - Summary: ReplaceText infinite loops when attribute contains $ sign Key: NIFI-1915 URL: https://issues.apache.org/jira/browse/NIFI-1915 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 0.6.1 Reporter: Stephane Maarek Priority: Critical I think the biggest issue is that the text inside of an attribute isn't properly escaped when written to a String, which brings conflict when the text contains dollar signs ($) That's a big roadblock for me as I can't predict if and when some $ signs may be present in the data An easy way to reproduce is to take the csv to json template here: https://cwiki.apache.org/confluence/download/attachments/57904847/CsvToJSON.xml?version=1&modificationDate=1442927496000&api=v2 In the first ReplaceText, replace a,b,c,d by a$a,b,c,d (it simulates data that may or may not contain a $ sign) Launch the flow, you'll see the errors / warning / infinite loop -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1905) Stop action doesn't invoke ExecuteProcess.destroy()
[
https://issues.apache.org/jira/browse/NIFI-1905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15297155#comment-15297155
]
ASF subversion and git services commented on NIFI-1905:
---
Commit 718212aac430e47e60dd863a937f7380b34ebe3d in nifi's branch refs/heads/0.x
from [~ozhurakousky]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=718212a ]
NIFI-1905 fixed compilation error due to usage of non-existing method on
Process (part-two)
> Stop action doesn't invoke ExecuteProcess.destroy()
> ---
>
> Key: NIFI-1905
> URL: https://issues.apache.org/jira/browse/NIFI-1905
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 0.6.1
> Environment: Raspberry Pi 3
> Java(TM) SE Runtime Environment (build 1.8.0_65-b17)
> Java HotSpot(TM) Client VM (build 25.65-b01, mixed mode)
>Reporter: Andrew Grande
>Assignee: Oleg Zhurakousky
>Priority: Critical
> Fix For: 1.0.0, 0.7.0
>
> Attachments: NIFI-1905.xml
>
>
> Consider the attached template. There is an ExecuteProcess which wraps a
> python script (listed below).
> STR:
> * start ListenTcp
> * start ExecuteProcess
> * stop ExecuteProcess
> Expected: the ExecuteProcess component terminates successfully
> Actual: there's always a thread running (1 displayed in the top-right corner
> of the processor)
> The thread will only go away if the underlying command crashes. It's easy to
> replicate by stopping ListenTcp - the script will fail with a socket
> connection error and quit.
> I have been looking into the ExecuteProcess source, and it looks like
> OnUnscheduled must also invoke Process.destroy() in addition to terminating
> the stdio/stderr I/O threads, but there's nothing like that anywhere.
> {code}
> import serial
> import socket
> # set up the serial connection speed
> ser = serial.Serial('/dev/ttyUSB0', 115200)
> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
> s.connect(('localhost',3456))
> # main loop
> while 1:
> data = ser.readline()
> s.sendall(data)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
nifi git commit: NIFI-1905 fixed compilation error due to usage of non-existing method on Process (part-two)
Repository: nifi
Updated Branches:
refs/heads/0.x 06c5c695b -> 718212aac
NIFI-1905 fixed compilation error due to usage of non-existing method on
Process (part-two)
Project: http://git-wip-us.apache.org/repos/asf/nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/718212aa
Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/718212aa
Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/718212aa
Branch: refs/heads/0.x
Commit: 718212aac430e47e60dd863a937f7380b34ebe3d
Parents: 06c5c69
Author: Oleg Zhurakousky
Authored: Mon May 23 17:46:47 2016 -0400
Committer: Oleg Zhurakousky
Committed: Mon May 23 17:46:47 2016 -0400
--
.../nifi/processors/standard/ExecuteProcess.java | 17 ++---
.../processors/standard/TestExecuteProcess.java| 6 --
2 files changed, 10 insertions(+), 13 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/nifi/blob/718212aa/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteProcess.java
--
diff --git
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteProcess.java
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteProcess.java
index 302b5cf..bb4c53e 100644
---
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteProcess.java
+++
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteProcess.java
@@ -186,10 +186,7 @@ public class ExecuteProcess extends AbstractProcessor {
try {
executor.shutdown();
} finally {
-if (this.externalProcess.isAlive()) {
-this.getLogger().info("Process hasn't terminated, forcing the
interrupt");
-this.externalProcess.destroy();
-}
+this.externalProcess.destroy();
}
}
@@ -379,9 +376,15 @@ public class ExecuteProcess extends AbstractProcessor {
try {
// Since we are going to exit anyway, one sec gives it
an extra chance to exit gracefully.
// In the future consider exposing it via
configuration.
-boolean terminated = externalProcess.waitFor(1000,
TimeUnit.MILLISECONDS);
-int exitCode = terminated ?
externalProcess.exitValue() : -;
-getLogger().info("Process finished with exit code {}
", new Object[] { exitCode });
+externalProcess.waitFor();
+int exitCode = -;
+try {
+exitCode = externalProcess.exitValue();
+ExecuteProcess.this.getLogger().info("Process
finished with exit code {} ",
+new Object[] { exitCode });
+} catch (IllegalThreadStateException e) {
+ExecuteProcess.this.getLogger().warn("Failed to
terminate external process");
+}
} catch (InterruptedException e1) {
Thread.currentThread().interrupt();
}
http://git-wip-us.apache.org/repos/asf/nifi/blob/718212aa/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestExecuteProcess.java
--
diff --git
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestExecuteProcess.java
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestExecuteProcess.java
index 160bbdb..4ec199b 100644
---
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestExecuteProcess.java
+++
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestExecuteProcess.java
@@ -18,7 +18,6 @@ package org.apache.nifi.processors.standard;
import static org.junit.Assert.assertArrayEquals;
import static org.junit.Assert.assertEquals;
-import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertNotNull;
import static org.junit.Assert.assertTrue;
import static org.junit.Assert.fail;
@@ -117,11 +116,6 @@ public class TestExecuteProcess {
ExecutorService executor = (ExecutorService)
executorF.get(processor);
assertTrue(executor.isShutdown());
[jira] [Commented] (NIFI-1884) Add User & Group API
[ https://issues.apache.org/jira/browse/NIFI-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15297021#comment-15297021 ] ASF GitHub Bot commented on NIFI-1884: -- Github user mcgilman commented on the pull request: https://github.com/apache/nifi/pull/452#issuecomment-221086451 @bbende The updated PR looks good to me. A number of great iterations here. Definitely think this is a solid foundation to continue the policy based authorizers managed by NiFi. > Add User & Group API > > > Key: NIFI-1884 > URL: https://issues.apache.org/jira/browse/NIFI-1884 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core Framework >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Minor > Fix For: 1.0.0 > > > Define the API for managing users, groups, and policies. > This is to advance the work described in this feature proposal: > https://cwiki.apache.org/confluence/display/NIFI/Support+Authorizer+API > The parent JIRA for all authorization work is NIFI-1550. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1905) Stop action doesn't invoke ExecuteProcess.destroy()
[
https://issues.apache.org/jira/browse/NIFI-1905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296970#comment-15296970
]
ASF subversion and git services commented on NIFI-1905:
---
Commit 06c5c695be6c803ca0f046db5182c4d1ffcb29b3 in nifi's branch refs/heads/0.x
from [~ozhurakousky]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=06c5c69 ]
NIFI-1905 fixed compilation error due to usage of non-existing method on Process
> Stop action doesn't invoke ExecuteProcess.destroy()
> ---
>
> Key: NIFI-1905
> URL: https://issues.apache.org/jira/browse/NIFI-1905
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 0.6.1
> Environment: Raspberry Pi 3
> Java(TM) SE Runtime Environment (build 1.8.0_65-b17)
> Java HotSpot(TM) Client VM (build 25.65-b01, mixed mode)
>Reporter: Andrew Grande
>Assignee: Oleg Zhurakousky
>Priority: Critical
> Fix For: 1.0.0, 0.7.0
>
> Attachments: NIFI-1905.xml
>
>
> Consider the attached template. There is an ExecuteProcess which wraps a
> python script (listed below).
> STR:
> * start ListenTcp
> * start ExecuteProcess
> * stop ExecuteProcess
> Expected: the ExecuteProcess component terminates successfully
> Actual: there's always a thread running (1 displayed in the top-right corner
> of the processor)
> The thread will only go away if the underlying command crashes. It's easy to
> replicate by stopping ListenTcp - the script will fail with a socket
> connection error and quit.
> I have been looking into the ExecuteProcess source, and it looks like
> OnUnscheduled must also invoke Process.destroy() in addition to terminating
> the stdio/stderr I/O threads, but there's nothing like that anywhere.
> {code}
> import serial
> import socket
> # set up the serial connection speed
> ser = serial.Serial('/dev/ttyUSB0', 115200)
> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
> s.connect(('localhost',3456))
> # main loop
> while 1:
> data = ser.readline()
> s.sendall(data)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-1799) Auto adjust flow layout
[
https://issues.apache.org/jira/browse/NIFI-1799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296969#comment-15296969
]
ASF GitHub Bot commented on NIFI-1799:
--
Github user alopresto commented on a diff in the pull request:
https://github.com/apache/nifi/pull/442#discussion_r64277394
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/groups/StandardProcessGroup.java
---
@@ -2168,6 +2170,16 @@ private boolean isDisconnected(final Snippet
snippet) {
return true;
}
+@Override
+public Set findAllPositionables() {
+Set positionables = Sets.newHashSet();
+positionables.addAll(findAllConnectables(this, true));
+positionables.addAll(findAllProcessGroups());
--- End diff --
We should check to see if this recurses within nested process groups when
scaling top-level process group.
> Auto adjust flow layout
> ---
>
> Key: NIFI-1799
> URL: https://issues.apache.org/jira/browse/NIFI-1799
> Project: Apache NiFi
> Issue Type: Sub-task
> Components: Core Framework, Core UI
>Reporter: Matt Gilman
>Assignee: Jeff Storck
> Fix For: 1.0.0
>
>
> Updates to the canvas to reflect the modernized look and feel and component
> level authorization will increase the size of each component. We should auto
> adjust the layouts to the components will not overlap. Below are the current
> size of each component and the new size:
> {code}
> Processor 310x100 350x130
> Process Group 365x142 380x172
> Remote Process Group 365x140 380x176
> Connection (largest) 188x53 200x55
> Port (root) 160x56 240x75
> Port 160x40 240x50
> Funnel 61x61 48x48
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
nifi git commit: NIFI-1905 fixed compilation error due to usage of non-existing method on Process
Repository: nifi
Updated Branches:
refs/heads/0.x b81b05d13 -> 06c5c695b
NIFI-1905 fixed compilation error due to usage of non-existing method on Process
Project: http://git-wip-us.apache.org/repos/asf/nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/06c5c695
Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/06c5c695
Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/06c5c695
Branch: refs/heads/0.x
Commit: 06c5c695be6c803ca0f046db5182c4d1ffcb29b3
Parents: b81b05d
Author: Oleg Zhurakousky
Authored: Mon May 23 15:52:20 2016 -0400
Committer: Oleg Zhurakousky
Committed: Mon May 23 15:52:20 2016 -0400
--
.../java/org/apache/nifi/processors/standard/ExecuteProcess.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/nifi/blob/06c5c695/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteProcess.java
--
diff --git
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteProcess.java
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteProcess.java
index 74d05cd..302b5cf 100644
---
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteProcess.java
+++
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteProcess.java
@@ -188,7 +188,7 @@ public class ExecuteProcess extends AbstractProcessor {
} finally {
if (this.externalProcess.isAlive()) {
this.getLogger().info("Process hasn't terminated, forcing the
interrupt");
-this.externalProcess.destroyForcibly();
+this.externalProcess.destroy();
}
}
}
[jira] [Commented] (NIFI-1660) Enhance the expression language with jsonPath function
[
https://issues.apache.org/jira/browse/NIFI-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296948#comment-15296948
]
ASF GitHub Bot commented on NIFI-1660:
--
Github user ckmcd commented on the pull request:
https://github.com/apache/nifi/pull/303#issuecomment-221073806
I've rebase on master. Other suggested changes are in the works.
> Enhance the expression language with jsonPath function
> --
>
> Key: NIFI-1660
> URL: https://issues.apache.org/jira/browse/NIFI-1660
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Affects Versions: 0.5.1
>Reporter: Christopher McDermott
>Assignee: Joseph Witt
>Priority: Minor
> Fix For: 1.0.0, 0.7.0
>
>
> jsonPath would evaluate a JSON path provided, as an argument, against the
> subject.
> Example
> {quote}
> $\{kafka.key:jsonPath('$.foo.bar')\}
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-1907) SiteToSiteClient not properly using keystore and truststore properties
[ https://issues.apache.org/jira/browse/NIFI-1907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296911#comment-15296911 ] ASF GitHub Bot commented on NIFI-1907: -- Github user bbende commented on the pull request: https://github.com/apache/nifi/pull/457#issuecomment-221068355 @joewitt good idea regarding the error handling, i made the suggested changes and and rebased so the commit should now reflect those changes > SiteToSiteClient not properly using keystore and truststore properties > -- > > Key: NIFI-1907 > URL: https://issues.apache.org/jira/browse/NIFI-1907 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 0.6.1 >Reporter: Bryan Bende >Assignee: Bryan Bende > Fix For: 0.7.0 > > > SiteToSiteClient.Builder allows setting an SSLContext or setting all of the > individual SSL properties, it then has a method getSSLContext() which says > that if the sslContext is null return that, otherwise use the properties to > create one: > https://github.com/apache/nifi/blob/e4b7e47836edf47042973e604005058c28eed23b/nifi-commons/nifi-site-to-site-client/src/main/java/org/apache/nifi/remote/client/SiteToSiteClient.java#L575 > The problem is this getSSLContext() is never called. When the builder's > build() method is called, it passes the builder to > StandardSiteToSiteClientConfig and just assigns all the member variables with > direct access: > https://github.com/apache/nifi/blob/e4b7e47836edf47042973e604005058c28eed23b/nifi-commons/nifi-site-to-site-client/src/main/java/org/apache/nifi/remote/client/SiteToSiteClient.java#L722 > Later on in SocketClient it will call SiteToSiteClient.getSSLContext(): > https://github.com/apache/nifi/blob/e4b7e47836edf47042973e604005058c28eed23b/nifi-commons/nifi-site-to-site-client/src/main/java/org/apache/nifi/remote/client/socket/SocketClient.java#L52 > This will still be null here if only the SSL properties were initially > specified on the builder, and therefore won't end up creating an Https > connection and thus failing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1913) HandleHttpRequest doesn't correctly handle failures to register requests
[ https://issues.apache.org/jira/browse/NIFI-1913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296856#comment-15296856 ] ASF GitHub Bot commented on NIFI-1913: -- Github user pvillard31 commented on the pull request: https://github.com/apache/nifi/pull/462#issuecomment-221058590 There is a double space on line 160 in test class. Travis seems not to be run on 0.x branch so I don't know if contrib check would fail because of that (cannot personally check right now). Otherwise, +1. > HandleHttpRequest doesn't correctly handle failures to register requests > > > Key: NIFI-1913 > URL: https://issues.apache.org/jira/browse/NIFI-1913 > Project: Apache NiFi > Issue Type: Bug >Reporter: Joseph Percivall >Assignee: Joseph Percivall > > While looking at this[1] stack overflow question I came across the fact that > HandleHttpRequest doesn't properly handle the flowfiles when there is a > failure to register[2]. The created FlowFile should be removed just like it > is done here[3]. This lack of removal causes a FlowFileHandlingException to > occur when the context map fails to register the flowfile. > The fix is simply adding "session.remove(flowFile);" to the "if > (!registered)" block. > [1] > http://stackoverflow.com/questions/37394219/nifi-transfer-relationship-not-specified > > [2] > https://github.com/apache/nifi/blob/9064b976317e316f42ac279dd026105b54a17ddb/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/HandleHttpRequest.java#L601-L601 > [3] > https://github.com/apache/nifi/blob/9064b976317e316f42ac279dd026105b54a17ddb/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/HandleHttpRequest.java#L489-L489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1781) Update UI to reflect component level authorization
[ https://issues.apache.org/jira/browse/NIFI-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296804#comment-15296804 ] ASF GitHub Bot commented on NIFI-1781: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/461 > Update UI to reflect component level authorization > -- > > Key: NIFI-1781 > URL: https://issues.apache.org/jira/browse/NIFI-1781 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman > Fix For: 1.0.0 > > Attachments: 0001-NIFI-1781.patch, nifi-component-samples.png, > nifi-sample-flow.png > > > - Update to UI to visual access level per component -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1781) Update UI to reflect component level authorization
[ https://issues.apache.org/jira/browse/NIFI-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296801#comment-15296801 ] ASF subversion and git services commented on NIFI-1781: --- Commit 4dd50c80a4fe086d94998c8e7738dcf149734a84 in nifi's branch refs/heads/master from [~mcgilman] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=4dd50c8 ] NIFI-1781: - Including access policies in the breadcrumb's trail. - Updating toolbox according to group access policies. - Updating actions in palette based on selection access policies. NIFI-1554: - Introducing authorization during two phase commit. - Introducing snippet authorization according to the encapsulated components and the action performed. - This closes #461 > Update UI to reflect component level authorization > -- > > Key: NIFI-1781 > URL: https://issues.apache.org/jira/browse/NIFI-1781 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman > Fix For: 1.0.0 > > Attachments: 0001-NIFI-1781.patch, nifi-component-samples.png, > nifi-sample-flow.png > > > - Update to UI to visual access level per component -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1554) Incorporate new Authorizer API
[ https://issues.apache.org/jira/browse/NIFI-1554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296802#comment-15296802 ] ASF subversion and git services commented on NIFI-1554: --- Commit 4dd50c80a4fe086d94998c8e7738dcf149734a84 in nifi's branch refs/heads/master from [~mcgilman] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=4dd50c8 ] NIFI-1781: - Including access policies in the breadcrumb's trail. - Updating toolbox according to group access policies. - Updating actions in palette based on selection access policies. NIFI-1554: - Introducing authorization during two phase commit. - Introducing snippet authorization according to the encapsulated components and the action performed. - This closes #461 > Incorporate new Authorizer API > -- > > Key: NIFI-1554 > URL: https://issues.apache.org/jira/browse/NIFI-1554 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core Framework >Reporter: Matt Gilman >Assignee: Matt Gilman > Fix For: 1.0.0 > > > - Introduce checks on user actions using new Authorizer API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[3/6] nifi git commit: NIFI-1781: - Including access policies in the breadcrumb's trail. - Updating toolbox according to group access policies. - Updating actions in palette based on selection access
http://git-wip-us.apache.org/repos/asf/nifi/blob/4dd50c80/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java -- diff --git a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java index 097f214..65c8045 100644 --- a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java +++ b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java @@ -16,47 +16,28 @@ */ package org.apache.nifi.web.api; -import java.io.InputStream; -import java.net.URI; -import java.net.URISyntaxException; -import java.util.Date; -import java.util.HashMap; -import java.util.Map; -import java.util.Set; - -import javax.servlet.http.HttpServletRequest; -import javax.ws.rs.Consumes; -import javax.ws.rs.DELETE; -import javax.ws.rs.DefaultValue; -import javax.ws.rs.GET; -import javax.ws.rs.HttpMethod; -import javax.ws.rs.POST; -import javax.ws.rs.PUT; -import javax.ws.rs.Path; -import javax.ws.rs.PathParam; -import javax.ws.rs.Produces; -import javax.ws.rs.QueryParam; -import javax.ws.rs.WebApplicationException; -import javax.ws.rs.core.Context; -import javax.ws.rs.core.MediaType; -import javax.ws.rs.core.Response; -import javax.xml.bind.JAXBContext; -import javax.xml.bind.JAXBElement; -import javax.xml.bind.JAXBException; -import javax.xml.bind.Unmarshaller; -import javax.xml.transform.stream.StreamSource; - +import com.sun.jersey.api.core.ResourceContext; +import com.sun.jersey.multipart.FormDataParam; +import com.wordnik.swagger.annotations.Api; +import com.wordnik.swagger.annotations.ApiOperation; +import com.wordnik.swagger.annotations.ApiParam; +import com.wordnik.swagger.annotations.ApiResponse; +import com.wordnik.swagger.annotations.ApiResponses; +import com.wordnik.swagger.annotations.Authorization; import org.apache.commons.lang3.StringUtils; -import org.apache.nifi.cluster.coordination.http.replication.RequestReplicator; +import org.apache.nifi.authorization.Authorizer; +import org.apache.nifi.authorization.RequestAction; +import org.apache.nifi.authorization.resource.Authorizable; import org.apache.nifi.cluster.manager.impl.WebClusterManager; +import org.apache.nifi.controller.Snippet; import org.apache.nifi.util.NiFiProperties; +import org.apache.nifi.web.AuthorizableLookup; import org.apache.nifi.web.NiFiServiceFacade; import org.apache.nifi.web.Revision; import org.apache.nifi.web.UpdateResult; import org.apache.nifi.web.api.dto.ConnectionDTO; import org.apache.nifi.web.api.dto.ProcessGroupDTO; import org.apache.nifi.web.api.dto.RemoteProcessGroupDTO; -import org.apache.nifi.web.api.dto.SnippetDTO; import org.apache.nifi.web.api.dto.TemplateDTO; import org.apache.nifi.web.api.dto.flow.FlowDTO; import org.apache.nifi.web.api.entity.ConnectionEntity; @@ -80,7 +61,6 @@ import org.apache.nifi.web.api.entity.ProcessorEntity; import org.apache.nifi.web.api.entity.ProcessorsEntity; import org.apache.nifi.web.api.entity.RemoteProcessGroupEntity; import org.apache.nifi.web.api.entity.RemoteProcessGroupsEntity; -import org.apache.nifi.web.api.entity.SnippetEntity; import org.apache.nifi.web.api.entity.TemplateEntity; import org.apache.nifi.web.api.entity.TemplatesEntity; import org.apache.nifi.web.api.request.ClientIdParameter; @@ -88,14 +68,34 @@ import org.apache.nifi.web.api.request.LongParameter; import org.slf4j.Logger; import org.slf4j.LoggerFactory; -import com.sun.jersey.api.core.ResourceContext; -import com.sun.jersey.multipart.FormDataParam; -import com.wordnik.swagger.annotations.Api; -import com.wordnik.swagger.annotations.ApiOperation; -import com.wordnik.swagger.annotations.ApiParam; -import com.wordnik.swagger.annotations.ApiResponse; -import com.wordnik.swagger.annotations.ApiResponses; -import com.wordnik.swagger.annotations.Authorization; +import javax.servlet.http.HttpServletRequest; +import javax.ws.rs.Consumes; +import javax.ws.rs.DELETE; +import javax.ws.rs.DefaultValue; +import javax.ws.rs.GET; +import javax.ws.rs.HttpMethod; +import javax.ws.rs.POST; +import javax.ws.rs.PUT; +import javax.ws.rs.Path; +import javax.ws.rs.PathParam; +import javax.ws.rs.Produces; +import javax.ws.rs.QueryParam; +import javax.ws.rs.WebApplicationException; +import javax.ws.rs.core.Context; +import javax.ws.rs.core.MediaType; +import javax.ws.rs.core.Response; +import javax.xml.bind.JAXBContext; +import javax.xml.bind.JAXBElement; +import javax.xml.bind.JAXBException; +import javax.xml.bind.Unmarshaller; +import javax.xml.transform.stream.StreamSource; +import java
[2/6] nifi git commit: NIFI-1781: - Including access policies in the breadcrumb's trail. - Updating toolbox according to group access policies. - Updating actions in palette based on selection access
http://git-wip-us.apache.org/repos/asf/nifi/blob/4dd50c80/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/RemoteProcessGroupResource.java
--
diff --git
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/RemoteProcessGroupResource.java
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/RemoteProcessGroupResource.java
index 96183ea..cbd56b3 100644
---
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/RemoteProcessGroupResource.java
+++
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/RemoteProcessGroupResource.java
@@ -16,10 +16,28 @@
*/
package org.apache.nifi.web.api;
-import java.net.URI;
-import java.util.HashMap;
-import java.util.Map;
-import java.util.Set;
+import com.wordnik.swagger.annotations.Api;
+import com.wordnik.swagger.annotations.ApiOperation;
+import com.wordnik.swagger.annotations.ApiParam;
+import com.wordnik.swagger.annotations.ApiResponse;
+import com.wordnik.swagger.annotations.ApiResponses;
+import com.wordnik.swagger.annotations.Authorization;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.authorization.Authorizer;
+import org.apache.nifi.authorization.RequestAction;
+import org.apache.nifi.authorization.resource.Authorizable;
+import org.apache.nifi.cluster.manager.impl.WebClusterManager;
+import org.apache.nifi.util.NiFiProperties;
+import org.apache.nifi.web.NiFiServiceFacade;
+import org.apache.nifi.web.Revision;
+import org.apache.nifi.web.UpdateResult;
+import org.apache.nifi.web.api.dto.RemoteProcessGroupDTO;
+import org.apache.nifi.web.api.dto.RemoteProcessGroupPortDTO;
+import org.apache.nifi.web.api.dto.RevisionDTO;
+import org.apache.nifi.web.api.entity.RemoteProcessGroupEntity;
+import org.apache.nifi.web.api.entity.RemoteProcessGroupPortEntity;
+import org.apache.nifi.web.api.request.ClientIdParameter;
+import org.apache.nifi.web.api.request.LongParameter;
import javax.servlet.http.HttpServletRequest;
import javax.ws.rs.Consumes;
@@ -35,27 +53,8 @@ import javax.ws.rs.QueryParam;
import javax.ws.rs.core.Context;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
-
-import org.apache.commons.lang3.StringUtils;
-import org.apache.nifi.cluster.manager.impl.WebClusterManager;
-import org.apache.nifi.util.NiFiProperties;
-import org.apache.nifi.web.NiFiServiceFacade;
-import org.apache.nifi.web.Revision;
-import org.apache.nifi.web.UpdateResult;
-import org.apache.nifi.web.api.dto.RemoteProcessGroupDTO;
-import org.apache.nifi.web.api.dto.RemoteProcessGroupPortDTO;
-import org.apache.nifi.web.api.dto.RevisionDTO;
-import org.apache.nifi.web.api.entity.RemoteProcessGroupEntity;
-import org.apache.nifi.web.api.entity.RemoteProcessGroupPortEntity;
-import org.apache.nifi.web.api.request.ClientIdParameter;
-import org.apache.nifi.web.api.request.LongParameter;
-
-import com.wordnik.swagger.annotations.Api;
-import com.wordnik.swagger.annotations.ApiOperation;
-import com.wordnik.swagger.annotations.ApiParam;
-import com.wordnik.swagger.annotations.ApiResponse;
-import com.wordnik.swagger.annotations.ApiResponses;
-import com.wordnik.swagger.annotations.Authorization;
+import java.net.URI;
+import java.util.Set;
/**
* RESTful endpoint for managing a Remote group.
@@ -72,6 +71,7 @@ public class RemoteProcessGroupResource extends
ApplicationResource {
private NiFiServiceFacade serviceFacade;
private WebClusterManager clusterManager;
private NiFiProperties properties;
+private Authorizer authorizer;
/**
* Populates the remaining content for each remote process group. The uri
must be generated and the remote process groups name must be retrieved.
@@ -160,18 +160,24 @@ public class RemoteProcessGroupResource extends
ApplicationResource {
value = "Whether to include any encapulated ports or just
details about the remote process group.",
required = false
)
-@QueryParam("verbose") @DefaultValue(VERBOSE_DEFAULT_VALUE)
Boolean verbose,
+@QueryParam("verbose") @DefaultValue(VERBOSE_DEFAULT_VALUE) final
Boolean verbose,
@ApiParam(
value = "The remote process group id.",
required = true
)
-@PathParam("id") String id) {
+@PathParam("id") final String id) {
// replicate if cluster manager
if (properties.isClusterManager()) {
return clusterManager.applyRequest(HttpMethod.GET,
getAbsolutePath(), getRequestParameters(true), getHeaders()).getResponse();
}
+// authorize access
+service
[5/6] nifi git commit: NIFI-1781: - Including access policies in the breadcrumb's trail. - Updating toolbox according to group access policies. - Updating actions in palette based on selection access
http://git-wip-us.apache.org/repos/asf/nifi/blob/4dd50c80/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/StandardNiFiServiceFacade.java
--
diff --git
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/StandardNiFiServiceFacade.java
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/StandardNiFiServiceFacade.java
index 0600cb6..cb44834 100644
---
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/StandardNiFiServiceFacade.java
+++
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/StandardNiFiServiceFacade.java
@@ -16,30 +16,6 @@
*/
package org.apache.nifi.web;
-import java.io.IOException;
-import java.nio.charset.StandardCharsets;
-import java.util.ArrayList;
-import java.util.Arrays;
-import java.util.Collection;
-import java.util.Date;
-import java.util.HashMap;
-import java.util.HashSet;
-import java.util.LinkedHashMap;
-import java.util.LinkedHashSet;
-import java.util.List;
-import java.util.ListIterator;
-import java.util.Map;
-import java.util.Optional;
-import java.util.Set;
-import java.util.TimeZone;
-import java.util.UUID;
-import java.util.concurrent.TimeUnit;
-import java.util.function.Function;
-import java.util.function.Supplier;
-import java.util.stream.Collectors;
-
-import javax.ws.rs.WebApplicationException;
-
import org.apache.commons.lang3.StringUtils;
import org.apache.nifi.action.Action;
import org.apache.nifi.action.Component;
@@ -64,7 +40,6 @@ import org.apache.nifi.components.ValidationResult;
import org.apache.nifi.components.Validator;
import org.apache.nifi.components.state.Scope;
import org.apache.nifi.components.state.StateMap;
-import org.apache.nifi.connectable.Connectable;
import org.apache.nifi.connectable.Connection;
import org.apache.nifi.connectable.Funnel;
import org.apache.nifi.connectable.Port;
@@ -139,7 +114,6 @@ import org.apache.nifi.web.api.dto.action.ActionDTO;
import org.apache.nifi.web.api.dto.action.HistoryDTO;
import org.apache.nifi.web.api.dto.action.HistoryQueryDTO;
import org.apache.nifi.web.api.dto.flow.FlowDTO;
-import org.apache.nifi.web.api.dto.flow.ProcessGroupFlowDTO;
import org.apache.nifi.web.api.dto.provenance.ProvenanceDTO;
import org.apache.nifi.web.api.dto.provenance.ProvenanceEventDTO;
import org.apache.nifi.web.api.dto.provenance.ProvenanceOptionsDTO;
@@ -161,10 +135,12 @@ import org.apache.nifi.web.api.entity.FunnelEntity;
import org.apache.nifi.web.api.entity.LabelEntity;
import org.apache.nifi.web.api.entity.PortEntity;
import org.apache.nifi.web.api.entity.ProcessGroupEntity;
+import org.apache.nifi.web.api.entity.ProcessGroupFlowEntity;
import org.apache.nifi.web.api.entity.ProcessorEntity;
import org.apache.nifi.web.api.entity.RemoteProcessGroupEntity;
import org.apache.nifi.web.api.entity.RemoteProcessGroupPortEntity;
import org.apache.nifi.web.api.entity.ReportingTaskEntity;
+import org.apache.nifi.web.api.entity.ScheduleComponentsEntity;
import org.apache.nifi.web.api.entity.SnippetEntity;
import org.apache.nifi.web.controller.ControllerFacade;
import org.apache.nifi.web.dao.ConnectionDAO;
@@ -191,6 +167,29 @@ import org.apache.nifi.web.util.SnippetUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
+import javax.ws.rs.WebApplicationException;
+import java.io.IOException;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Date;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.LinkedHashMap;
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.ListIterator;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+import java.util.TimeZone;
+import java.util.UUID;
+import java.util.concurrent.TimeUnit;
+import java.util.function.Function;
+import java.util.function.Supplier;
+import java.util.stream.Collectors;
+
/**
* Implementation of NiFiServiceFacade that performs revision checking.
*/
@@ -232,12 +231,154 @@ public class StandardNiFiServiceFacade implements
NiFiServiceFacade {
private Authorizer authorizer;
+private final AuthorizableLookup authorizableLookup = new
AuthorizableLookup() {
+@Override
+public Authorizable getProcessor(final String id) {
+return processorDAO.getProcessor(id);
+}
+
+@Override
+public Authorizable getInputPort(final String id) {
+return inputPortDAO.getPort(id);
+}
+
+@Override
+public Authorizable getOutputPort(final String id) {
+return outputPortDAO.getPort(id);
+}
+
+@Override
+public
[6/6] nifi git commit: NIFI-1781: - Including access policies in the breadcrumb's trail. - Updating toolbox according to group access policies. - Updating actions in palette based on selection access
NIFI-1781: - Including access policies in the breadcrumb's trail. - Updating toolbox according to group access policies. - Updating actions in palette based on selection access policies. NIFI-1554: - Introducing authorization during two phase commit. - Introducing snippet authorization according to the encapsulated components and the action performed. - This closes #461 Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/4dd50c80 Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/4dd50c80 Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/4dd50c80 Branch: refs/heads/master Commit: 4dd50c80a4fe086d94998c8e7738dcf149734a84 Parents: d42ea46 Author: Matt Gilman Authored: Mon May 23 14:14:22 2016 -0400 Committer: Matt Gilman Committed: Mon May 23 14:15:32 2016 -0400 -- .../org/apache/nifi/controller/Snippet.java | 12 +- .../nifi/web/api/dto/ProcessGroupDTO.java | 15 - .../org/apache/nifi/web/api/dto/SnippetDTO.java | 17 - .../web/api/dto/flow/FlowBreadcrumbDTO.java | 17 - .../apache/nifi/web/api/dto/flow/FlowDTO.java | 15 - .../web/api/dto/flow/ProcessGroupFlowDTO.java | 7 +- .../web/api/entity/FlowBreadcrumbEntity.java| 99 +++ .../web/api/entity/ProcessGroupFlowEntity.java | 19 + .../api/entity/ScheduleComponentsEntity.java| 77 ++ .../nifi/web/api/entity/SnippetEntity.java | 7 +- .../nifi/controller/ConfiguredComponent.java| 3 +- .../nifi/controller/ReportingTaskNode.java | 9 +- .../service/ControllerServiceNode.java | 9 +- .../org/apache/nifi/groups/ProcessGroup.java| 34 +- .../apache/nifi/controller/StandardSnippet.java | 14 - .../nifi/groups/StandardProcessGroup.java | 92 +- .../service/mock/MockProcessGroup.java | 7 + .../apache/nifi/audit/ProcessGroupAuditor.java | 73 +- .../org/apache/nifi/audit/SnippetAuditor.java | 157 ++-- .../org/apache/nifi/web/AuthorizableLookup.java | 154 .../org/apache/nifi/web/AuthorizeAccess.java| 21 + .../org/apache/nifi/web/NiFiServiceFacade.java | 106 ++- .../nifi/web/StandardNiFiServiceFacade.java | 513 ++-- .../nifi/web/api/ApplicationResource.java | 187 - .../apache/nifi/web/api/ConnectionResource.java | 155 ++-- .../nifi/web/api/ControllerServiceResource.java | 195 +++-- .../nifi/web/api/FlowFileQueueResource.java | 159 ++-- .../org/apache/nifi/web/api/FlowResource.java | 209 - .../org/apache/nifi/web/api/FunnelResource.java | 153 ++-- .../apache/nifi/web/api/InputPortResource.java | 139 ++-- .../org/apache/nifi/web/api/LabelResource.java | 143 ++-- .../apache/nifi/web/api/OutputPortResource.java | 151 ++-- .../nifi/web/api/ProcessGroupResource.java | 833 --- .../apache/nifi/web/api/ProcessorResource.java | 154 ++-- .../apache/nifi/web/api/ProvenanceResource.java | 158 ++-- .../web/api/RemoteProcessGroupResource.java | 319 --- .../nifi/web/api/ReportingTaskResource.java | 137 +-- .../apache/nifi/web/api/ResourceResource.java | 35 + .../apache/nifi/web/api/SnippetResource.java| 327 .../nifi/web/api/SystemDiagnosticsResource.java | 33 +- .../apache/nifi/web/api/TemplateResource.java | 37 +- .../org/apache/nifi/web/api/dto/DtoFactory.java | 59 +- .../apache/nifi/web/api/dto/EntityFactory.java | 35 +- .../apache/nifi/web/dao/ProcessGroupDAO.java| 14 +- .../org/apache/nifi/web/dao/SnippetDAO.java | 23 +- .../web/dao/impl/StandardProcessGroupDAO.java | 66 +- .../nifi/web/dao/impl/StandardSnippetDAO.java | 98 +-- .../src/main/resources/nifi-web-api-context.xml | 19 + .../accesscontrol/DfmAccessControlTest.java | 1 - .../nifi/web/revision/NaiveRevisionManager.java | 8 +- .../WEB-INF/partials/canvas/canvas-header.jsp | 8 + .../WEB-INF/partials/canvas/navigation.jsp | 20 +- .../src/main/webapp/css/navigation.css | 1 + .../controllers/nf-ng-breadcrumbs-controller.js | 18 +- .../src/main/webapp/js/nf/canvas/nf-actions.js | 64 +- .../main/webapp/js/nf/canvas/nf-canvas-utils.js | 57 +- .../src/main/webapp/js/nf/canvas/nf-canvas.js | 17 + .../main/webapp/js/nf/canvas/nf-context-menu.js | 2 +- .../js/nf/canvas/nf-processor-configuration.js | 2 +- .../js/nf/canvas/nf-remote-process-group.js | 4 +- .../src/main/webapp/js/nf/canvas/nf-snippet.js | 73 +- .../main/webapp/js/nf/nf-connection-details.js | 2 +- .../main/webapp/js/nf/nf-processor-details.js | 2 +- .../views/nf-ng-breadcrumbs-directive-view.html | 2 +- 64 files changed, 3397 insertions(+), 2199 deletions(-) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/4dd50c80/nifi-api/src/main/java/org/apache/nifi/controller/Snippet.java
[4/6] nifi git commit: NIFI-1781: - Including access policies in the breadcrumb's trail. - Updating toolbox according to group access policies. - Updating actions in palette based on selection access
http://git-wip-us.apache.org/repos/asf/nifi/blob/4dd50c80/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/FlowFileQueueResource.java
--
diff --git
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/FlowFileQueueResource.java
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/FlowFileQueueResource.java
index f6dd081..0b8af7c 100644
---
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/FlowFileQueueResource.java
+++
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/FlowFileQueueResource.java
@@ -16,32 +16,16 @@
*/
package org.apache.nifi.web.api;
-import java.io.IOException;
-import java.io.InputStream;
-import java.io.OutputStream;
-import java.net.URI;
-import java.util.HashSet;
-import java.util.Set;
-
-import javax.servlet.http.HttpServletRequest;
-import javax.ws.rs.Consumes;
-import javax.ws.rs.DELETE;
-import javax.ws.rs.DefaultValue;
-import javax.ws.rs.GET;
-import javax.ws.rs.HttpMethod;
-import javax.ws.rs.POST;
-import javax.ws.rs.Path;
-import javax.ws.rs.PathParam;
-import javax.ws.rs.Produces;
-import javax.ws.rs.QueryParam;
-import javax.ws.rs.WebApplicationException;
-import javax.ws.rs.core.Context;
-import javax.ws.rs.core.MediaType;
-import javax.ws.rs.core.Response;
-import javax.ws.rs.core.Response.Status;
-import javax.ws.rs.core.StreamingOutput;
-
+import com.wordnik.swagger.annotations.Api;
+import com.wordnik.swagger.annotations.ApiOperation;
+import com.wordnik.swagger.annotations.ApiParam;
+import com.wordnik.swagger.annotations.ApiResponse;
+import com.wordnik.swagger.annotations.ApiResponses;
+import com.wordnik.swagger.annotations.Authorization;
import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.authorization.Authorizer;
+import org.apache.nifi.authorization.RequestAction;
+import org.apache.nifi.authorization.resource.Authorizable;
import org.apache.nifi.cluster.coordination.http.replication.RequestReplicator;
import org.apache.nifi.cluster.manager.exception.UnknownNodeException;
import org.apache.nifi.cluster.manager.impl.WebClusterManager;
@@ -60,12 +44,29 @@ import org.apache.nifi.web.api.entity.FlowFileEntity;
import org.apache.nifi.web.api.entity.ListingRequestEntity;
import org.apache.nifi.web.api.request.ClientIdParameter;
-import com.wordnik.swagger.annotations.Api;
-import com.wordnik.swagger.annotations.ApiOperation;
-import com.wordnik.swagger.annotations.ApiParam;
-import com.wordnik.swagger.annotations.ApiResponse;
-import com.wordnik.swagger.annotations.ApiResponses;
-import com.wordnik.swagger.annotations.Authorization;
+import javax.servlet.http.HttpServletRequest;
+import javax.ws.rs.Consumes;
+import javax.ws.rs.DELETE;
+import javax.ws.rs.DefaultValue;
+import javax.ws.rs.GET;
+import javax.ws.rs.HttpMethod;
+import javax.ws.rs.POST;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.QueryParam;
+import javax.ws.rs.WebApplicationException;
+import javax.ws.rs.core.Context;
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response;
+import javax.ws.rs.core.Response.Status;
+import javax.ws.rs.core.StreamingOutput;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.net.URI;
+import java.util.HashSet;
+import java.util.Set;
/**
* RESTful endpoint for managing a flowfile queue.
@@ -75,12 +76,12 @@ import com.wordnik.swagger.annotations.Authorization;
value = "/flowfile-queues",
description = "Endpoint for managing a FlowFile Queue."
)
-// TODO: Need revisions of the Connections for these endpoints!
public class FlowFileQueueResource extends ApplicationResource {
private NiFiServiceFacade serviceFacade;
private WebClusterManager clusterManager;
private NiFiProperties properties;
+private Authorizer authorizer;
/**
* Populate the URIs for the specified flowfile listing.
@@ -147,17 +148,17 @@ public class FlowFileQueueResource extends
ApplicationResource {
value = "The connection id.",
required = true
)
-@PathParam("connection-id") String connectionId,
+@PathParam("connection-id") final String connectionId,
@ApiParam(
value = "The flowfile uuid.",
required = true
)
-@PathParam("flowfile-uuid") String flowFileUuid,
+@PathParam("flowfile-uuid") final String flowFileUuid,
@ApiParam(
value = "The id of the node where the content exists if
clustered.",
required = false
[1/6] nifi git commit: NIFI-1781: - Including access policies in the breadcrumb's trail. - Updating toolbox according to group access policies. - Updating actions in palette based on selection access
Repository: nifi
Updated Branches:
refs/heads/master d42ea467c -> 4dd50c80a
http://git-wip-us.apache.org/repos/asf/nifi/blob/4dd50c80/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/dao/impl/StandardSnippetDAO.java
--
diff --git
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/dao/impl/StandardSnippetDAO.java
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/dao/impl/StandardSnippetDAO.java
index fa3da71..f41e7f9 100644
---
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/dao/impl/StandardSnippetDAO.java
+++
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/dao/impl/StandardSnippetDAO.java
@@ -16,11 +16,7 @@
*/
package org.apache.nifi.web.dao.impl;
-import java.util.HashMap;
-import java.util.Map;
-import java.util.Map.Entry;
-import java.util.Set;
-
+import org.apache.commons.lang3.StringUtils;
import org.apache.nifi.components.PropertyDescriptor;
import org.apache.nifi.controller.FlowController;
import org.apache.nifi.controller.ProcessorNode;
@@ -41,7 +37,11 @@ import org.apache.nifi.web.api.dto.RevisionDTO;
import org.apache.nifi.web.api.dto.SnippetDTO;
import org.apache.nifi.web.dao.SnippetDAO;
import org.apache.nifi.web.util.SnippetUtils;
-import org.apache.commons.lang3.StringUtils;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
public class StandardSnippetDAO implements SnippetDAO {
@@ -119,7 +119,6 @@ public class StandardSnippetDAO implements SnippetDAO {
final StandardSnippet snippet = new StandardSnippet();
snippet.setId(snippetDTO.getId());
snippet.setParentGroupId(snippetDTO.getParentGroupId());
-snippet.setLinked(snippetDTO.isLinked());
snippet.addProcessors(mapDtoToRevision(snippetDTO.getProcessors()));
snippet.addProcessGroups(mapDtoToRevision(snippetDTO.getProcessGroups()));
snippet.addRemoteProcessGroups(mapDtoToRevision(snippetDTO.getRemoteProcessGroups()));
@@ -146,39 +145,31 @@ public class StandardSnippetDAO implements SnippetDAO {
}
@Override
-public void verifyDelete(String snippetId) {
-final StandardSnippet snippet = locateSnippet(snippetId);
-
-// only need to check if the snippet is linked
-if (snippet.isLinked()) {
-// ensure the parent group exist
-final ProcessGroup processGroup =
flowController.getGroup(snippet.getParentGroupId());
-if (processGroup == null) {
-throw new IllegalArgumentException("The specified parent
process group could not be found.");
-}
+public void verifyDeleteSnippetComponents(String snippetId) {
+final Snippet snippet = locateSnippet(snippetId);
-// verify the processGroup can remove the snippet
-processGroup.verifyCanDelete(snippet);
+// ensure the parent group exist
+final ProcessGroup processGroup =
flowController.getGroup(snippet.getParentGroupId());
+if (processGroup == null) {
+throw new IllegalArgumentException("The specified parent process
group could not be found.");
}
+
+// verify the processGroup can remove the snippet
+processGroup.verifyCanDelete(snippet);
}
@Override
-public void deleteSnippet(String snippetId) {
-final StandardSnippet snippet = locateSnippet(snippetId);
-
-// if the snippet is linked, remove the contents
-if (snippet.isLinked()) {
-final ProcessGroup processGroup =
flowController.getGroup(snippet.getParentGroupId());
-if (processGroup == null) {
-throw new IllegalArgumentException("The specified parent
process group could not be found.");
-}
+public void deleteSnippetComponents(String snippetId) {
+final Snippet snippet = locateSnippet(snippetId);
-// remove the underlying components
-processGroup.remove(snippet);
+// remove the contents
+final ProcessGroup processGroup =
flowController.getGroup(snippet.getParentGroupId());
+if (processGroup == null) {
+throw new IllegalArgumentException("The specified parent process
group could not be found.");
}
-// delete the snippet itself
-flowController.getSnippetManager().removeSnippet(snippet);
+// remove the underlying components
+processGroup.remove(snippet);
}
@Override
@@ -192,11 +183,18 @@ public class StandardSnippetDAO implements SnippetDAO {
}
@Override
-public void verifyUpdate(SnippetDTO snippetDTO) {
-fin
[jira] [Commented] (NIFI-1781) Update UI to reflect component level authorization
[ https://issues.apache.org/jira/browse/NIFI-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296799#comment-15296799 ] ASF GitHub Bot commented on NIFI-1781: -- Github user mcgilman commented on the pull request: https://github.com/apache/nifi/pull/461#issuecomment-221051044 Thanks @markap14 All changes have been applied. > Update UI to reflect component level authorization > -- > > Key: NIFI-1781 > URL: https://issues.apache.org/jira/browse/NIFI-1781 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman > Fix For: 1.0.0 > > Attachments: 0001-NIFI-1781.patch, nifi-component-samples.png, > nifi-sample-flow.png > > > - Update to UI to visual access level per component -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1781) Update UI to reflect component level authorization
[
https://issues.apache.org/jira/browse/NIFI-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296789#comment-15296789
]
ASF GitHub Bot commented on NIFI-1781:
--
Github user mcgilman commented on a diff in the pull request:
https://github.com/apache/nifi/pull/461#discussion_r64262070
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
---
@@ -2024,6 +1843,24 @@ public Response instantiateTemplate(
// templates
// -
+private void authorizeSnippetUsage(final AuthorizableLookup lookup,
final String groupId, final String snippetId) {
+// ensure write access to the target process group
+lookup.getProcessGroup(groupId).authorize(authorizer,
RequestAction.WRITE);
+
+// ensure read permission to every component in the snippet
+final Snippet snippet = lookup.getSnippet(snippetId);
+final Set authorizables = new HashSet<>();
+
authorizables.addAll(snippet.getProcessGroups().keySet().stream().map(id ->
lookup.getProcessGroup(id)).collect(Collectors.toSet()));
--- End diff --
Awesome. Will do!
> Update UI to reflect component level authorization
> --
>
> Key: NIFI-1781
> URL: https://issues.apache.org/jira/browse/NIFI-1781
> Project: Apache NiFi
> Issue Type: Sub-task
> Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.0.0
>
> Attachments: 0001-NIFI-1781.patch, nifi-component-samples.png,
> nifi-sample-flow.png
>
>
> - Update to UI to visual access level per component
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MINIFI-6) Basic C++ native MiniFi implementation
[ https://issues.apache.org/jira/browse/MINIFI-6?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296773#comment-15296773 ] ASF GitHub Bot commented on MINIFI-6: - Github user simonellistonball commented on the pull request: https://github.com/apache/nifi-minifi-cpp/pull/4#issuecomment-221049133 Tried building this with the default settings (-O0), which yields 1.3MB binary. Is the intention to provide a different build script for more constrained environments, (-Os yields 459K binary). > Basic C++ native MiniFi implementation > -- > > Key: MINIFI-6 > URL: https://issues.apache.org/jira/browse/MINIFI-6 > Project: Apache NiFi MiNiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 0.1.0 > Environment: C++ > Linux >Reporter: bqiu > Labels: native > Fix For: 0.1.0 > > > A Basic C++ isolated native MiNiFi implementation (not communicated to the > master NiFI yet). > 1) Identify the all necessary software frameworks for C++ MiNiFi > implementation like database, xml parser, logging, etc. > 2) Flow configuration from flow.xml > 3) Processor init/enable/disable/running > 4) Processor Scheduling > 5) Processor Relationship/Connections > 6) Flow record creation/clone/transfer between Processor > 7) Flow record persistent -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1781) Update UI to reflect component level authorization
[
https://issues.apache.org/jira/browse/NIFI-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296739#comment-15296739
]
ASF GitHub Bot commented on NIFI-1781:
--
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/461#discussion_r64259092
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/SnippetResource.java
---
@@ -0,0 +1,397 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.web.api;
+
+import com.wordnik.swagger.annotations.Api;
+import com.wordnik.swagger.annotations.ApiOperation;
+import com.wordnik.swagger.annotations.ApiParam;
+import com.wordnik.swagger.annotations.ApiResponse;
+import com.wordnik.swagger.annotations.ApiResponses;
+import com.wordnik.swagger.annotations.Authorization;
+import org.apache.nifi.authorization.Authorizer;
+import org.apache.nifi.authorization.RequestAction;
+import org.apache.nifi.authorization.resource.Authorizable;
+import org.apache.nifi.authorization.user.NiFiUser;
+import org.apache.nifi.authorization.user.NiFiUserUtils;
+import org.apache.nifi.cluster.manager.impl.WebClusterManager;
+import org.apache.nifi.controller.Snippet;
+import org.apache.nifi.util.NiFiProperties;
+import org.apache.nifi.web.NiFiServiceFacade;
+import org.apache.nifi.web.Revision;
+import org.apache.nifi.web.api.dto.SnippetDTO;
+import org.apache.nifi.web.api.entity.SnippetEntity;
+
+import javax.servlet.http.HttpServletRequest;
+import javax.ws.rs.Consumes;
+import javax.ws.rs.DELETE;
+import javax.ws.rs.HttpMethod;
+import javax.ws.rs.POST;
+import javax.ws.rs.PUT;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.Context;
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response;
+import java.net.URI;
+import java.util.HashSet;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+/**
+ * RESTful endpoint for querying dataflow snippets.
+ */
+@Path("/snippets")
+@Api(
+value = "/snippets",
+description = "Endpoint for accessing dataflow snippets."
+)
+public class SnippetResource extends ApplicationResource {
+
+private NiFiServiceFacade serviceFacade;
+private WebClusterManager clusterManager;
+private NiFiProperties properties;
+private Authorizer authorizer;
+
+/**
+ * Populate the uri's for the specified snippet.
+ *
+ * @param entity processors
+ * @return dtos
+ */
+private SnippetEntity
populateRemainingSnippetEntityContent(SnippetEntity entity) {
+if (entity.getSnippet() != null) {
+populateRemainingSnippetContent(entity.getSnippet());
+}
+return entity;
+}
+
+/**
+ * Populates the uri for the specified snippet.
+ */
+private SnippetDTO populateRemainingSnippetContent(SnippetDTO snippet)
{
+String snippetGroupId = snippet.getParentGroupId();
+
+// populate the snippet href
+snippet.setUri(generateResourceUri("process-groups",
snippetGroupId, "snippets", snippet.getId()));
+
+return snippet;
+}
+
+//
+// snippets
+//
+
+/**
+ * Creates a snippet based off the specified configuration.
+ *
+ * @param httpServletRequest request
+ * @param snippetEntity A snippetEntity
+ * @return A snippetEntity
+ */
+@POST
+@Consumes(MediaType.APPLICATION_JSON)
+@Produces(MediaType.APPLICATION_JSON)
+// TODO - @PreAuthorize("hasRole('ROLE_DFM')")
+@ApiOperation(
+value = "Creates a snippet",
+response = SnippetEntity.class,
+au
[jira] [Commented] (NIFI-1781) Update UI to reflect component level authorization
[
https://issues.apache.org/jira/browse/NIFI-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296757#comment-15296757
]
ASF GitHub Bot commented on NIFI-1781:
--
Github user mcgilman commented on a diff in the pull request:
https://github.com/apache/nifi/pull/461#discussion_r64259949
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/FlowResource.java
---
@@ -321,6 +320,157 @@ public Response getControllerServices(
return clusterContext(generateOkResponse(entity)).build();
}
+/**
+ * Updates the specified process group.
+ *
+ * @param httpServletRequest request
+ * @param id The id of the process group.
+ * @param scheduleComponentsEntity A scheduleComponentsEntity.
+ * @return A processGroupEntity.
+ */
+@PUT
+@Consumes(MediaType.APPLICATION_JSON)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("process-groups/{id}")
+// TODO - @PreAuthorize("hasRole('ROLE_DFM')")
+@ApiOperation(
+value = "Updates a process group",
+response = ScheduleComponentsEntity.class,
+authorizations = {
+@Authorization(value = "Data Flow Manager", type = "ROLE_DFM")
+}
+)
+@ApiResponses(
+value = {
+@ApiResponse(code = 400, message = "NiFi was unable to
complete the request because it was invalid. The request should not be retried
without modification."),
+@ApiResponse(code = 401, message = "Client could not be
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized
to make this request."),
+@ApiResponse(code = 404, message = "The specified resource
could not be found."),
+@ApiResponse(code = 409, message = "The request was valid but
NiFi was not in the appropriate state to process it. Retrying the same request
later may be successful.")
+}
+)
+public Response scheduleComponents(
+@Context HttpServletRequest httpServletRequest,
+@ApiParam(
+value = "The process group id.",
+required = true
+)
+@PathParam("id") String id,
+ScheduleComponentsEntity scheduleComponentsEntity) {
+
+authorizeFlow();
+
+// ensure the same id is being used
+if (!id.equals(scheduleComponentsEntity.getId())) {
+throw new IllegalArgumentException(String.format("The process
group id (%s) in the request body does "
++ "not equal the process group id of the requested
resource (%s).", scheduleComponentsEntity.getId(), id));
+}
+
+final ScheduledState state;
+if (scheduleComponentsEntity.getState() == null) {
+throw new IllegalArgumentException("The scheduled state must
be specified.");
+} else {
+try {
+state =
ScheduledState.valueOf(scheduleComponentsEntity.getState());
+} catch (final IllegalArgumentException iae) {
+throw new IllegalArgumentException(String.format("The
scheduled must be one of [].",
StringUtils.join(EnumSet.of(ScheduledState.RUNNING, ScheduledState.STOPPED), ",
")));
+}
+}
+
+// ensure its a supported scheduled state
+if (ScheduledState.DISABLED.equals(state) ||
ScheduledState.STARTING.equals(state) || ScheduledState.STOPPING.equals(state))
{
+throw new IllegalArgumentException(String.format("The
scheduled must be one of [].",
StringUtils.join(EnumSet.of(ScheduledState.RUNNING, ScheduledState.STOPPED), ",
")));
+}
+
+// if the components are not specified, gather all components and
their current revision
+if (scheduleComponentsEntity.getComponents() == null) {
+// TODO - this will break while clustered until nodes are able
to process/replicate requests
+// get the current revisions for the components being updated
+final Set revisions =
serviceFacade.getRevisionsFromGroup(id, group -> {
+final Set componentIds = new HashSet<>();
+
+// ensure authorized for each processor we will attempt to
schedule
+
group.findAllProcessors().stream().filter(ScheduledState.RUNNING.equals(state)
? ProcessGroup.SCHEDULABLE_PROCESSORS :
ProcessGroup.UNSCHEDULABLE_PROCESSORS).forEach(processor -> {
+if (processor.isAuthorized(authorizer,
RequestAction.WRITE)) {
--- End diff --
Will do!
> Update UI to reflect component level authorization
>
[jira] [Commented] (NIFI-1884) Add User & Group API
[ https://issues.apache.org/jira/browse/NIFI-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296756#comment-15296756 ] ASF GitHub Bot commented on NIFI-1884: -- Github user bbende commented on the pull request: https://github.com/apache/nifi/pull/452#issuecomment-221046683 @mcgilman @jtstorck @alopresto pushed two new commits, the first contains minor changes to address some of Matt's comments, the second is an attempt at what Matt suggested about how the MutableAuthorizer could be an abstract class that provides an implementation of authorize(), let me know what you think > Add User & Group API > > > Key: NIFI-1884 > URL: https://issues.apache.org/jira/browse/NIFI-1884 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core Framework >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Minor > Fix For: 1.0.0 > > > Define the API for managing users, groups, and policies. > This is to advance the work described in this feature proposal: > https://cwiki.apache.org/confluence/display/NIFI/Support+Authorizer+API > The parent JIRA for all authorization work is NIFI-1550. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1781) Update UI to reflect component level authorization
[ https://issues.apache.org/jira/browse/NIFI-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296752#comment-15296752 ] ASF GitHub Bot commented on NIFI-1781: -- Github user markap14 commented on the pull request: https://github.com/apache/nifi/pull/461#issuecomment-221046258 @mcgilman looks good for the most part. I left some comments in-line (most of which you've already addressed before I finished the review even!) Excellent. Once the last few things are merged in I'm a +1. Thanks for knocking this out. > Update UI to reflect component level authorization > -- > > Key: NIFI-1781 > URL: https://issues.apache.org/jira/browse/NIFI-1781 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman > Fix For: 1.0.0 > > Attachments: 0001-NIFI-1781.patch, nifi-component-samples.png, > nifi-sample-flow.png > > > - Update to UI to visual access level per component -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1781) Update UI to reflect component level authorization
[
https://issues.apache.org/jira/browse/NIFI-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296741#comment-15296741
]
ASF GitHub Bot commented on NIFI-1781:
--
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/461#discussion_r64259242
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/SnippetResource.java
---
@@ -0,0 +1,397 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.web.api;
+
+import com.wordnik.swagger.annotations.Api;
+import com.wordnik.swagger.annotations.ApiOperation;
+import com.wordnik.swagger.annotations.ApiParam;
+import com.wordnik.swagger.annotations.ApiResponse;
+import com.wordnik.swagger.annotations.ApiResponses;
+import com.wordnik.swagger.annotations.Authorization;
+import org.apache.nifi.authorization.Authorizer;
+import org.apache.nifi.authorization.RequestAction;
+import org.apache.nifi.authorization.resource.Authorizable;
+import org.apache.nifi.authorization.user.NiFiUser;
+import org.apache.nifi.authorization.user.NiFiUserUtils;
+import org.apache.nifi.cluster.manager.impl.WebClusterManager;
+import org.apache.nifi.controller.Snippet;
+import org.apache.nifi.util.NiFiProperties;
+import org.apache.nifi.web.NiFiServiceFacade;
+import org.apache.nifi.web.Revision;
+import org.apache.nifi.web.api.dto.SnippetDTO;
+import org.apache.nifi.web.api.entity.SnippetEntity;
+
+import javax.servlet.http.HttpServletRequest;
+import javax.ws.rs.Consumes;
+import javax.ws.rs.DELETE;
+import javax.ws.rs.HttpMethod;
+import javax.ws.rs.POST;
+import javax.ws.rs.PUT;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.Context;
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response;
+import java.net.URI;
+import java.util.HashSet;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+/**
+ * RESTful endpoint for querying dataflow snippets.
+ */
+@Path("/snippets")
+@Api(
+value = "/snippets",
+description = "Endpoint for accessing dataflow snippets."
+)
+public class SnippetResource extends ApplicationResource {
+
+private NiFiServiceFacade serviceFacade;
+private WebClusterManager clusterManager;
+private NiFiProperties properties;
+private Authorizer authorizer;
+
+/**
+ * Populate the uri's for the specified snippet.
+ *
+ * @param entity processors
+ * @return dtos
+ */
+private SnippetEntity
populateRemainingSnippetEntityContent(SnippetEntity entity) {
+if (entity.getSnippet() != null) {
+populateRemainingSnippetContent(entity.getSnippet());
+}
+return entity;
+}
+
+/**
+ * Populates the uri for the specified snippet.
+ */
+private SnippetDTO populateRemainingSnippetContent(SnippetDTO snippet)
{
+String snippetGroupId = snippet.getParentGroupId();
+
+// populate the snippet href
+snippet.setUri(generateResourceUri("process-groups",
snippetGroupId, "snippets", snippet.getId()));
+
+return snippet;
+}
+
+//
+// snippets
+//
+
+/**
+ * Creates a snippet based off the specified configuration.
+ *
+ * @param httpServletRequest request
+ * @param snippetEntity A snippetEntity
+ * @return A snippetEntity
+ */
+@POST
+@Consumes(MediaType.APPLICATION_JSON)
+@Produces(MediaType.APPLICATION_JSON)
+// TODO - @PreAuthorize("hasRole('ROLE_DFM')")
+@ApiOperation(
+value = "Creates a snippet",
+response = SnippetEntity.class,
+au
[jira] [Commented] (NIFI-1781) Update UI to reflect component level authorization
[
https://issues.apache.org/jira/browse/NIFI-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296727#comment-15296727
]
ASF GitHub Bot commented on NIFI-1781:
--
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/461#discussion_r64258489
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
---
@@ -2024,6 +1843,24 @@ public Response instantiateTemplate(
// templates
// -
+private void authorizeSnippetUsage(final AuthorizableLookup lookup,
final String groupId, final String snippetId) {
+// ensure write access to the target process group
+lookup.getProcessGroup(groupId).authorize(authorizer,
RequestAction.WRITE);
+
+// ensure read permission to every component in the snippet
+final Snippet snippet = lookup.getSnippet(snippetId);
+final Set authorizables = new HashSet<>();
+
authorizables.addAll(snippet.getProcessGroups().keySet().stream().map(id ->
lookup.getProcessGroup(id)).collect(Collectors.toSet()));
--- End diff --
Or better yet, to get rid of the 'authorizables' Set all together and
define the final authorization lambda as an object and just call it on each of
these streams. E.g.,
```
final Consumer auth = authorizable ->
authorizable.authorize(authorizer, RequestAciton.READ);
snippet.getProcessGroups().keySet().stream().map(id ->
lookup.getProcessGroup(id)).forEach(auth);
snippet.getProcessors().keySet().stream().map(id ->
lookup.getProcessor(id)).forEach(auth);
```
> Update UI to reflect component level authorization
> --
>
> Key: NIFI-1781
> URL: https://issues.apache.org/jira/browse/NIFI-1781
> Project: Apache NiFi
> Issue Type: Sub-task
> Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.0.0
>
> Attachments: 0001-NIFI-1781.patch, nifi-component-samples.png,
> nifi-sample-flow.png
>
>
> - Update to UI to visual access level per component
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-1781) Update UI to reflect component level authorization
[
https://issues.apache.org/jira/browse/NIFI-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296713#comment-15296713
]
ASF GitHub Bot commented on NIFI-1781:
--
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/461#discussion_r64257999
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProcessGroupResource.java
---
@@ -2024,6 +1843,24 @@ public Response instantiateTemplate(
// templates
// -
+private void authorizeSnippetUsage(final AuthorizableLookup lookup,
final String groupId, final String snippetId) {
+// ensure write access to the target process group
+lookup.getProcessGroup(groupId).authorize(authorizer,
RequestAction.WRITE);
+
+// ensure read permission to every component in the snippet
+final Snippet snippet = lookup.getSnippet(snippetId);
+final Set authorizables = new HashSet<>();
+
authorizables.addAll(snippet.getProcessGroups().keySet().stream().map(id ->
lookup.getProcessGroup(id)).collect(Collectors.toSet()));
--- End diff --
It probably makes sense here to change this to:
`snippet.getProcessGroups().keySet().stream().map(id ->
lookup.getProcessGroup(id)).forEach(auth -> authorizables.add(auth));
`
This way we don't have to keep creating all of these intermediate Set
objects just so that we can add them to the 'authorizables' set.
> Update UI to reflect component level authorization
> --
>
> Key: NIFI-1781
> URL: https://issues.apache.org/jira/browse/NIFI-1781
> Project: Apache NiFi
> Issue Type: Sub-task
> Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.0.0
>
> Attachments: 0001-NIFI-1781.patch, nifi-component-samples.png,
> nifi-sample-flow.png
>
>
> - Update to UI to visual access level per component
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-1781) Update UI to reflect component level authorization
[
https://issues.apache.org/jira/browse/NIFI-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296694#comment-15296694
]
ASF GitHub Bot commented on NIFI-1781:
--
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/461#discussion_r64256472
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/FlowResource.java
---
@@ -321,6 +320,157 @@ public Response getControllerServices(
return clusterContext(generateOkResponse(entity)).build();
}
+/**
+ * Updates the specified process group.
+ *
+ * @param httpServletRequest request
+ * @param id The id of the process group.
+ * @param scheduleComponentsEntity A scheduleComponentsEntity.
+ * @return A processGroupEntity.
+ */
+@PUT
+@Consumes(MediaType.APPLICATION_JSON)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("process-groups/{id}")
+// TODO - @PreAuthorize("hasRole('ROLE_DFM')")
+@ApiOperation(
+value = "Updates a process group",
+response = ScheduleComponentsEntity.class,
+authorizations = {
+@Authorization(value = "Data Flow Manager", type = "ROLE_DFM")
+}
+)
+@ApiResponses(
+value = {
+@ApiResponse(code = 400, message = "NiFi was unable to
complete the request because it was invalid. The request should not be retried
without modification."),
+@ApiResponse(code = 401, message = "Client could not be
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized
to make this request."),
+@ApiResponse(code = 404, message = "The specified resource
could not be found."),
+@ApiResponse(code = 409, message = "The request was valid but
NiFi was not in the appropriate state to process it. Retrying the same request
later may be successful.")
+}
+)
+public Response scheduleComponents(
+@Context HttpServletRequest httpServletRequest,
+@ApiParam(
+value = "The process group id.",
+required = true
+)
+@PathParam("id") String id,
+ScheduleComponentsEntity scheduleComponentsEntity) {
+
+authorizeFlow();
+
+// ensure the same id is being used
+if (!id.equals(scheduleComponentsEntity.getId())) {
+throw new IllegalArgumentException(String.format("The process
group id (%s) in the request body does "
++ "not equal the process group id of the requested
resource (%s).", scheduleComponentsEntity.getId(), id));
+}
+
+final ScheduledState state;
+if (scheduleComponentsEntity.getState() == null) {
+throw new IllegalArgumentException("The scheduled state must
be specified.");
+} else {
+try {
+state =
ScheduledState.valueOf(scheduleComponentsEntity.getState());
+} catch (final IllegalArgumentException iae) {
+throw new IllegalArgumentException(String.format("The
scheduled must be one of [].",
StringUtils.join(EnumSet.of(ScheduledState.RUNNING, ScheduledState.STOPPED), ",
")));
+}
+}
+
+// ensure its a supported scheduled state
+if (ScheduledState.DISABLED.equals(state) ||
ScheduledState.STARTING.equals(state) || ScheduledState.STOPPING.equals(state))
{
+throw new IllegalArgumentException(String.format("The
scheduled must be one of [].",
StringUtils.join(EnumSet.of(ScheduledState.RUNNING, ScheduledState.STOPPED), ",
")));
+}
+
+// if the components are not specified, gather all components and
their current revision
+if (scheduleComponentsEntity.getComponents() == null) {
+// TODO - this will break while clustered until nodes are able
to process/replicate requests
+// get the current revisions for the components being updated
+final Set revisions =
serviceFacade.getRevisionsFromGroup(id, group -> {
+final Set componentIds = new HashSet<>();
+
+// ensure authorized for each processor we will attempt to
schedule
+
group.findAllProcessors().stream().filter(ScheduledState.RUNNING.equals(state)
? ProcessGroup.SCHEDULABLE_PROCESSORS :
ProcessGroup.UNSCHEDULABLE_PROCESSORS).forEach(processor -> {
+if (processor.isAuthorized(authorizer,
RequestAction.WRITE)) {
+componentIds.add(processor.getIdentifier());
+
[jira] [Commented] (NIFI-1781) Update UI to reflect component level authorization
[
https://issues.apache.org/jira/browse/NIFI-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296693#comment-15296693
]
ASF GitHub Bot commented on NIFI-1781:
--
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/461#discussion_r64256311
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/FlowResource.java
---
@@ -321,6 +320,157 @@ public Response getControllerServices(
return clusterContext(generateOkResponse(entity)).build();
}
+/**
+ * Updates the specified process group.
+ *
+ * @param httpServletRequest request
+ * @param id The id of the process group.
+ * @param scheduleComponentsEntity A scheduleComponentsEntity.
+ * @return A processGroupEntity.
+ */
+@PUT
+@Consumes(MediaType.APPLICATION_JSON)
+@Produces(MediaType.APPLICATION_JSON)
+@Path("process-groups/{id}")
+// TODO - @PreAuthorize("hasRole('ROLE_DFM')")
+@ApiOperation(
+value = "Updates a process group",
+response = ScheduleComponentsEntity.class,
+authorizations = {
+@Authorization(value = "Data Flow Manager", type = "ROLE_DFM")
+}
+)
+@ApiResponses(
+value = {
+@ApiResponse(code = 400, message = "NiFi was unable to
complete the request because it was invalid. The request should not be retried
without modification."),
+@ApiResponse(code = 401, message = "Client could not be
authenticated."),
+@ApiResponse(code = 403, message = "Client is not authorized
to make this request."),
+@ApiResponse(code = 404, message = "The specified resource
could not be found."),
+@ApiResponse(code = 409, message = "The request was valid but
NiFi was not in the appropriate state to process it. Retrying the same request
later may be successful.")
+}
+)
+public Response scheduleComponents(
+@Context HttpServletRequest httpServletRequest,
+@ApiParam(
+value = "The process group id.",
+required = true
+)
+@PathParam("id") String id,
+ScheduleComponentsEntity scheduleComponentsEntity) {
+
+authorizeFlow();
+
+// ensure the same id is being used
+if (!id.equals(scheduleComponentsEntity.getId())) {
+throw new IllegalArgumentException(String.format("The process
group id (%s) in the request body does "
++ "not equal the process group id of the requested
resource (%s).", scheduleComponentsEntity.getId(), id));
+}
+
+final ScheduledState state;
+if (scheduleComponentsEntity.getState() == null) {
+throw new IllegalArgumentException("The scheduled state must
be specified.");
+} else {
+try {
+state =
ScheduledState.valueOf(scheduleComponentsEntity.getState());
+} catch (final IllegalArgumentException iae) {
+throw new IllegalArgumentException(String.format("The
scheduled must be one of [].",
StringUtils.join(EnumSet.of(ScheduledState.RUNNING, ScheduledState.STOPPED), ",
")));
+}
+}
+
+// ensure its a supported scheduled state
+if (ScheduledState.DISABLED.equals(state) ||
ScheduledState.STARTING.equals(state) || ScheduledState.STOPPING.equals(state))
{
+throw new IllegalArgumentException(String.format("The
scheduled must be one of [].",
StringUtils.join(EnumSet.of(ScheduledState.RUNNING, ScheduledState.STOPPED), ",
")));
+}
+
+// if the components are not specified, gather all components and
their current revision
+if (scheduleComponentsEntity.getComponents() == null) {
+// TODO - this will break while clustered until nodes are able
to process/replicate requests
+// get the current revisions for the components being updated
+final Set revisions =
serviceFacade.getRevisionsFromGroup(id, group -> {
+final Set componentIds = new HashSet<>();
+
+// ensure authorized for each processor we will attempt to
schedule
+
group.findAllProcessors().stream().filter(ScheduledState.RUNNING.equals(state)
? ProcessGroup.SCHEDULABLE_PROCESSORS :
ProcessGroup.UNSCHEDULABLE_PROCESSORS).forEach(processor -> {
+if (processor.isAuthorized(authorizer,
RequestAction.WRITE)) {
--- End diff --
It seems cleaner to me to use a filter here to check if the proces
[jira] [Commented] (NIFI-1913) HandleHttpRequest doesn't correctly handle failures to register requests
[ https://issues.apache.org/jira/browse/NIFI-1913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296674#comment-15296674 ] ASF GitHub Bot commented on NIFI-1913: -- GitHub user JPercivall opened a pull request: https://github.com/apache/nifi/pull/462 NIFI-1913 Properly removing flowfile in HandleHttpRequest when the Co… …ntext Map fails to register the request You can merge this pull request into a Git repository by running: $ git pull https://github.com/JPercivall/nifi NIFI-1913 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/462.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #462 commit 44013976749d377706999fc49a1a4d3014909a69 Author: jpercivall Date: 2016-05-23T16:24:43Z NIFI-1913 Properly removing flowfile in HandleHttpRequest when the Context Map fails to register the request > HandleHttpRequest doesn't correctly handle failures to register requests > > > Key: NIFI-1913 > URL: https://issues.apache.org/jira/browse/NIFI-1913 > Project: Apache NiFi > Issue Type: Bug >Reporter: Joseph Percivall >Assignee: Joseph Percivall > > While looking at this[1] stack overflow question I came across the fact that > HandleHttpRequest doesn't properly handle the flowfiles when there is a > failure to register[2]. The created FlowFile should be removed just like it > is done here[3]. This lack of removal causes a FlowFileHandlingException to > occur when the context map fails to register the flowfile. > The fix is simply adding "session.remove(flowFile);" to the "if > (!registered)" block. > [1] > http://stackoverflow.com/questions/37394219/nifi-transfer-relationship-not-specified > > [2] > https://github.com/apache/nifi/blob/9064b976317e316f42ac279dd026105b54a17ddb/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/HandleHttpRequest.java#L601-L601 > [3] > https://github.com/apache/nifi/blob/9064b976317e316f42ac279dd026105b54a17ddb/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/HandleHttpRequest.java#L489-L489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1781) Update UI to reflect component level authorization
[
https://issues.apache.org/jira/browse/NIFI-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296644#comment-15296644
]
ASF GitHub Bot commented on NIFI-1781:
--
Github user mcgilman commented on a diff in the pull request:
https://github.com/apache/nifi/pull/461#discussion_r64250769
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core-api/src/main/java/org/apache/nifi/groups/ProcessGroup.java
---
@@ -46,6 +48,26 @@
public interface ProcessGroup extends Authorizable {
/**
+ * Predicate for filtering schedulable Processors.
+ */
+Predicate SCHEDULABLE_PROCESSORS = node ->
!node.isRunning() && node.getScheduledState() != ScheduledState.DISABLED;
+
+/**
+ * Predicate for filtering unschedulable Processors.
+ */
+Predicate UNSCHEDULABLE_PROCESSORS = node ->
node.isRunning();
+
+/**
+ * Predicate for filtering schedulable Ports
+ */
+Predicate SCHEDULABLE_PORTS = port -> port.getScheduledState()
!= ScheduledState.DISABLED;
--- End diff --
I did not modify any logic here. Simply relocated to not repeat. If that
logic is incorrect, I can certainly update it, but it hasn't changed here.
> Update UI to reflect component level authorization
> --
>
> Key: NIFI-1781
> URL: https://issues.apache.org/jira/browse/NIFI-1781
> Project: Apache NiFi
> Issue Type: Sub-task
> Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.0.0
>
> Attachments: 0001-NIFI-1781.patch, nifi-component-samples.png,
> nifi-sample-flow.png
>
>
> - Update to UI to visual access level per component
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-1781) Update UI to reflect component level authorization
[
https://issues.apache.org/jira/browse/NIFI-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296649#comment-15296649
]
ASF GitHub Bot commented on NIFI-1781:
--
Github user mcgilman commented on a diff in the pull request:
https://github.com/apache/nifi/pull/461#discussion_r64251051
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core-api/src/main/java/org/apache/nifi/groups/ProcessGroup.java
---
@@ -46,6 +48,26 @@
public interface ProcessGroup extends Authorizable {
/**
+ * Predicate for filtering schedulable Processors.
+ */
+Predicate SCHEDULABLE_PROCESSORS = node ->
!node.isRunning() && node.getScheduledState() != ScheduledState.DISABLED;
+
+/**
+ * Predicate for filtering unschedulable Processors.
+ */
+Predicate UNSCHEDULABLE_PROCESSORS = node ->
node.isRunning();
+
+/**
+ * Predicate for filtering schedulable Ports
+ */
+Predicate SCHEDULABLE_PORTS = port -> port.getScheduledState()
!= ScheduledState.DISABLED;
+
+/**
+ * Predicate for filtering schedulable Ports
+ */
+Predicate UNSCHEDULABLE_PORTS = port -> port.getScheduledState()
== ScheduledState.RUNNING;
--- End diff --
Happy to update the logic to whatever it needs to be. However, this is how
it was previously.
> Update UI to reflect component level authorization
> --
>
> Key: NIFI-1781
> URL: https://issues.apache.org/jira/browse/NIFI-1781
> Project: Apache NiFi
> Issue Type: Sub-task
> Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.0.0
>
> Attachments: 0001-NIFI-1781.patch, nifi-component-samples.png,
> nifi-sample-flow.png
>
>
> - Update to UI to visual access level per component
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-1781) Update UI to reflect component level authorization
[
https://issues.apache.org/jira/browse/NIFI-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296647#comment-15296647
]
ASF GitHub Bot commented on NIFI-1781:
--
Github user mcgilman commented on a diff in the pull request:
https://github.com/apache/nifi/pull/461#discussion_r64250889
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/StandardNiFiServiceFacade.java
---
@@ -568,164 +699,50 @@ public void verifyDeleteReportingTask(String
reportingTaskId) {
@Override
-public void verifyUpdateSnippet(SnippetDTO snippetDto) {
+public void verifyUpdateSnippet(SnippetDTO snippetDto, final
Set affectedComponentIds) {
try {
// if snippet does not exist, then the update request is
likely creating it
// so we don't verify since it will fail
if (snippetDAO.hasSnippet(snippetDto.getId())) {
snippetDAO.verifyUpdate(snippetDto);
}
} catch (final Exception e) {
-revisionManager.cancelClaim(snippetDto.getId());
+affectedComponentIds.forEach(id ->
revisionManager.cancelClaim(snippetDto.getId()));
throw e;
}
}
-private Set getRevisionsForGroup(final String groupId) {
-final Set revisions = new HashSet<>();
-
-revisions.add(revisionManager.getRevision(groupId));
-final ProcessGroup processGroup =
processGroupDAO.getProcessGroup(groupId);
-if (processGroup == null) {
-throw new IllegalArgumentException("Snippet contains a
reference to Process Group with ID " + groupId + " but no Process Group exists
with that ID");
-}
-
-processGroup.getConnections().stream().map(c ->
c.getIdentifier()).map(id -> revisionManager.getRevision(id)).forEach(rev ->
revisions.add(rev));
-processGroup.getFunnels().stream().map(c ->
c.getIdentifier()).map(id -> revisionManager.getRevision(id)).forEach(rev ->
revisions.add(rev));
-processGroup.getInputPorts().stream().map(c ->
c.getIdentifier()).map(id -> revisionManager.getRevision(id)).forEach(rev ->
revisions.add(rev));
-processGroup.getOutputPorts().stream().map(c ->
c.getIdentifier()).map(id -> revisionManager.getRevision(id)).forEach(rev ->
revisions.add(rev));
-processGroup.getLabels().stream().map(c ->
c.getIdentifier()).map(id -> revisionManager.getRevision(id)).forEach(rev ->
revisions.add(rev));
-processGroup.getProcessors().stream().map(c ->
c.getIdentifier()).map(id -> revisionManager.getRevision(id)).forEach(rev ->
revisions.add(rev));
-processGroup.getRemoteProcessGroups().stream().map(c ->
c.getIdentifier()).map(id -> revisionManager.getRevision(id)).forEach(rev ->
revisions.add(rev));
-processGroup.getProcessGroups().stream().map(c ->
c.getIdentifier()).forEach(id -> revisions.addAll(getRevisionsForGroup(id)));
-
-return revisions;
-}
-
-private Set getRevisionsForSnippet(final SnippetDTO
snippetDto) {
-final Set requiredRevisions = new HashSet<>();
-
requiredRevisions.add(revisionManager.getRevision(snippetDto.getId()));
-snippetDto.getConnections().entrySet().stream()
-.map(entry -> new Revision(entry.getValue().getVersion(),
entry.getValue().getClientId(), entry.getKey()))
-.forEach(rev -> requiredRevisions.add(rev));
-
-snippetDto.getFunnels().entrySet().stream()
-.map(entry -> new Revision(entry.getValue().getVersion(),
entry.getValue().getClientId(), entry.getKey()))
-.forEach(rev -> requiredRevisions.add(rev));
-
-snippetDto.getInputPorts().entrySet().stream()
-.map(entry -> new Revision(entry.getValue().getVersion(),
entry.getValue().getClientId(), entry.getKey()))
-.forEach(rev -> requiredRevisions.add(rev));
-
-snippetDto.getOutputPorts().entrySet().stream()
-.map(entry -> new Revision(entry.getValue().getVersion(),
entry.getValue().getClientId(), entry.getKey()))
-.forEach(rev -> requiredRevisions.add(rev));
-
-snippetDto.getLabels().entrySet().stream()
-.map(entry -> new Revision(entry.getValue().getVersion(),
entry.getValue().getClientId(), entry.getKey()))
-.forEach(rev -> requiredRevisions.add(rev));
-
-snippetDto.getProcessors().entrySet().stream()
-.map(entry -> new Revision(entry.getValue().getVersion(),
entry.getValue().getClientId(), entry.getKey()))
-.forEach(rev -> requiredRevisions.add(rev));
-
-sni
[jira] [Created] (NIFI-1914) Update framework method to use Java 8 where appropriate
Matt Gilman created NIFI-1914: - Summary: Update framework method to use Java 8 where appropriate Key: NIFI-1914 URL: https://issues.apache.org/jira/browse/NIFI-1914 Project: Apache NiFi Issue Type: Improvement Components: Core Framework Reporter: Matt Gilman Priority: Trivial Fix For: 1.0.0 Currently, there are a number methods for finding all of a given component. We should consider refactoring these to support Predicate's. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1781) Update UI to reflect component level authorization
[
https://issues.apache.org/jira/browse/NIFI-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296637#comment-15296637
]
ASF GitHub Bot commented on NIFI-1781:
--
Github user mcgilman commented on a diff in the pull request:
https://github.com/apache/nifi/pull/461#discussion_r64250267
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/groups/StandardProcessGroup.java
---
@@ -283,38 +283,21 @@ public boolean isRootGroup() {
public void startProcessing() {
readLock.lock();
try {
-for (final ProcessorNode node : processors.values()) {
+
findAllProcessors().stream().filter(SCHEDULABLE_PROCESSORS).forEach(node -> {
--- End diff --
Totally agree. There are a number of improvements that we can make with
Java 8 available. Let's create a separate JIRA to identify those.
> Update UI to reflect component level authorization
> --
>
> Key: NIFI-1781
> URL: https://issues.apache.org/jira/browse/NIFI-1781
> Project: Apache NiFi
> Issue Type: Sub-task
> Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.0.0
>
> Attachments: 0001-NIFI-1781.patch, nifi-component-samples.png,
> nifi-sample-flow.png
>
>
> - Update to UI to visual access level per component
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-1781) Update UI to reflect component level authorization
[
https://issues.apache.org/jira/browse/NIFI-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296627#comment-15296627
]
ASF GitHub Bot commented on NIFI-1781:
--
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/461#discussion_r64249465
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/StandardNiFiServiceFacade.java
---
@@ -568,164 +699,50 @@ public void verifyDeleteReportingTask(String
reportingTaskId) {
@Override
-public void verifyUpdateSnippet(SnippetDTO snippetDto) {
+public void verifyUpdateSnippet(SnippetDTO snippetDto, final
Set affectedComponentIds) {
try {
// if snippet does not exist, then the update request is
likely creating it
// so we don't verify since it will fail
if (snippetDAO.hasSnippet(snippetDto.getId())) {
snippetDAO.verifyUpdate(snippetDto);
}
} catch (final Exception e) {
-revisionManager.cancelClaim(snippetDto.getId());
+affectedComponentIds.forEach(id ->
revisionManager.cancelClaim(snippetDto.getId()));
throw e;
}
}
-private Set getRevisionsForGroup(final String groupId) {
-final Set revisions = new HashSet<>();
-
-revisions.add(revisionManager.getRevision(groupId));
-final ProcessGroup processGroup =
processGroupDAO.getProcessGroup(groupId);
-if (processGroup == null) {
-throw new IllegalArgumentException("Snippet contains a
reference to Process Group with ID " + groupId + " but no Process Group exists
with that ID");
-}
-
-processGroup.getConnections().stream().map(c ->
c.getIdentifier()).map(id -> revisionManager.getRevision(id)).forEach(rev ->
revisions.add(rev));
-processGroup.getFunnels().stream().map(c ->
c.getIdentifier()).map(id -> revisionManager.getRevision(id)).forEach(rev ->
revisions.add(rev));
-processGroup.getInputPorts().stream().map(c ->
c.getIdentifier()).map(id -> revisionManager.getRevision(id)).forEach(rev ->
revisions.add(rev));
-processGroup.getOutputPorts().stream().map(c ->
c.getIdentifier()).map(id -> revisionManager.getRevision(id)).forEach(rev ->
revisions.add(rev));
-processGroup.getLabels().stream().map(c ->
c.getIdentifier()).map(id -> revisionManager.getRevision(id)).forEach(rev ->
revisions.add(rev));
-processGroup.getProcessors().stream().map(c ->
c.getIdentifier()).map(id -> revisionManager.getRevision(id)).forEach(rev ->
revisions.add(rev));
-processGroup.getRemoteProcessGroups().stream().map(c ->
c.getIdentifier()).map(id -> revisionManager.getRevision(id)).forEach(rev ->
revisions.add(rev));
-processGroup.getProcessGroups().stream().map(c ->
c.getIdentifier()).forEach(id -> revisions.addAll(getRevisionsForGroup(id)));
-
-return revisions;
-}
-
-private Set getRevisionsForSnippet(final SnippetDTO
snippetDto) {
-final Set requiredRevisions = new HashSet<>();
-
requiredRevisions.add(revisionManager.getRevision(snippetDto.getId()));
-snippetDto.getConnections().entrySet().stream()
-.map(entry -> new Revision(entry.getValue().getVersion(),
entry.getValue().getClientId(), entry.getKey()))
-.forEach(rev -> requiredRevisions.add(rev));
-
-snippetDto.getFunnels().entrySet().stream()
-.map(entry -> new Revision(entry.getValue().getVersion(),
entry.getValue().getClientId(), entry.getKey()))
-.forEach(rev -> requiredRevisions.add(rev));
-
-snippetDto.getInputPorts().entrySet().stream()
-.map(entry -> new Revision(entry.getValue().getVersion(),
entry.getValue().getClientId(), entry.getKey()))
-.forEach(rev -> requiredRevisions.add(rev));
-
-snippetDto.getOutputPorts().entrySet().stream()
-.map(entry -> new Revision(entry.getValue().getVersion(),
entry.getValue().getClientId(), entry.getKey()))
-.forEach(rev -> requiredRevisions.add(rev));
-
-snippetDto.getLabels().entrySet().stream()
-.map(entry -> new Revision(entry.getValue().getVersion(),
entry.getValue().getClientId(), entry.getKey()))
-.forEach(rev -> requiredRevisions.add(rev));
-
-snippetDto.getProcessors().entrySet().stream()
-.map(entry -> new Revision(entry.getValue().getVersion(),
entry.getValue().getClientId(), entry.getKey()))
-.forEach(rev -> requiredRevisions.add(rev));
-
-sni
[jira] [Commented] (NIFI-1781) Update UI to reflect component level authorization
[
https://issues.apache.org/jira/browse/NIFI-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296608#comment-15296608
]
ASF GitHub Bot commented on NIFI-1781:
--
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/461#discussion_r64248276
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/groups/StandardProcessGroup.java
---
@@ -283,38 +283,21 @@ public boolean isRootGroup() {
public void startProcessing() {
readLock.lock();
try {
-for (final ProcessorNode node : processors.values()) {
+
findAllProcessors().stream().filter(SCHEDULABLE_PROCESSORS).forEach(node -> {
--- End diff --
It seems like here, rather than finding all processors and putting them
into a set, then converting to a stream and filtering, we should allow a
Predicate to be passed into the findAllProcessors() method? Same for
input/output ports
> Update UI to reflect component level authorization
> --
>
> Key: NIFI-1781
> URL: https://issues.apache.org/jira/browse/NIFI-1781
> Project: Apache NiFi
> Issue Type: Sub-task
> Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.0.0
>
> Attachments: 0001-NIFI-1781.patch, nifi-component-samples.png,
> nifi-sample-flow.png
>
>
> - Update to UI to visual access level per component
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-1781) Update UI to reflect component level authorization
[
https://issues.apache.org/jira/browse/NIFI-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296606#comment-15296606
]
ASF GitHub Bot commented on NIFI-1781:
--
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/461#discussion_r64248023
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core-api/src/main/java/org/apache/nifi/groups/ProcessGroup.java
---
@@ -46,6 +48,26 @@
public interface ProcessGroup extends Authorizable {
/**
+ * Predicate for filtering schedulable Processors.
+ */
+Predicate SCHEDULABLE_PROCESSORS = node ->
!node.isRunning() && node.getScheduledState() != ScheduledState.DISABLED;
+
+/**
+ * Predicate for filtering unschedulable Processors.
+ */
+Predicate UNSCHEDULABLE_PROCESSORS = node ->
node.isRunning();
+
+/**
+ * Predicate for filtering schedulable Ports
+ */
+Predicate SCHEDULABLE_PORTS = port -> port.getScheduledState()
!= ScheduledState.DISABLED;
+
+/**
+ * Predicate for filtering schedulable Ports
+ */
+Predicate UNSCHEDULABLE_PORTS = port -> port.getScheduledState()
== ScheduledState.RUNNING;
--- End diff --
Shouldn't this be "port.isRunning()" instead, like we do for Processor? It
seems like we could actually just have a SCHEDULABLE_TRIGGERABLE and an
UNSCHEDULABLE_TRIGGERABLE rather than separate ones for ports & processors?
> Update UI to reflect component level authorization
> --
>
> Key: NIFI-1781
> URL: https://issues.apache.org/jira/browse/NIFI-1781
> Project: Apache NiFi
> Issue Type: Sub-task
> Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.0.0
>
> Attachments: 0001-NIFI-1781.patch, nifi-component-samples.png,
> nifi-sample-flow.png
>
>
> - Update to UI to visual access level per component
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-1781) Update UI to reflect component level authorization
[
https://issues.apache.org/jira/browse/NIFI-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296605#comment-15296605
]
ASF GitHub Bot commented on NIFI-1781:
--
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/461#discussion_r64247888
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core-api/src/main/java/org/apache/nifi/groups/ProcessGroup.java
---
@@ -46,6 +48,26 @@
public interface ProcessGroup extends Authorizable {
/**
+ * Predicate for filtering schedulable Processors.
+ */
+Predicate SCHEDULABLE_PROCESSORS = node ->
!node.isRunning() && node.getScheduledState() != ScheduledState.DISABLED;
+
+/**
+ * Predicate for filtering unschedulable Processors.
+ */
+Predicate UNSCHEDULABLE_PROCESSORS = node ->
node.isRunning();
+
+/**
+ * Predicate for filtering schedulable Ports
+ */
+Predicate SCHEDULABLE_PORTS = port -> port.getScheduledState()
!= ScheduledState.DISABLED;
--- End diff --
It seems odd to me that we are checking "!node.isRunning()" for
ProcessorsNode but not checking "!port.isRunning()" for ports.
> Update UI to reflect component level authorization
> --
>
> Key: NIFI-1781
> URL: https://issues.apache.org/jira/browse/NIFI-1781
> Project: Apache NiFi
> Issue Type: Sub-task
> Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
> Fix For: 1.0.0
>
> Attachments: 0001-NIFI-1781.patch, nifi-component-samples.png,
> nifi-sample-flow.png
>
>
> - Update to UI to visual access level per component
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (NIFI-1913) HandleHttpRequest doesn't correctly handle failures to register requests
Joseph Percivall created NIFI-1913: -- Summary: HandleHttpRequest doesn't correctly handle failures to register requests Key: NIFI-1913 URL: https://issues.apache.org/jira/browse/NIFI-1913 Project: Apache NiFi Issue Type: Bug Reporter: Joseph Percivall Assignee: Joseph Percivall While looking at this[1] stack overflow question I came across the fact that HandleHttpRequest doesn't properly handle the flowfiles when there is a failure to register[2]. The created FlowFile should be removed just like it is done here[3]. This lack of removal causes a FlowFileHandlingException to occur when the context map fails to register the flowfile. The fix is simply adding "session.remove(flowFile);" to the "if (!registered)" block. [1] http://stackoverflow.com/questions/37394219/nifi-transfer-relationship-not-specified [2] https://github.com/apache/nifi/blob/9064b976317e316f42ac279dd026105b54a17ddb/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/HandleHttpRequest.java#L601-L601 [3] https://github.com/apache/nifi/blob/9064b976317e316f42ac279dd026105b54a17ddb/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/HandleHttpRequest.java#L489-L489 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1781) Update UI to reflect component level authorization
[ https://issues.apache.org/jira/browse/NIFI-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-1781: -- Attachment: 0001-NIFI-1781.patch GitHub PR is currently unavailable. Posting patch here as a workaround. > Update UI to reflect component level authorization > -- > > Key: NIFI-1781 > URL: https://issues.apache.org/jira/browse/NIFI-1781 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman > Fix For: 1.0.0 > > Attachments: 0001-NIFI-1781.patch, nifi-component-samples.png, > nifi-sample-flow.png > > > - Update to UI to visual access level per component -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1660) Enhance the expression language with jsonPath function
[
https://issues.apache.org/jira/browse/NIFI-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296572#comment-15296572
]
ASF GitHub Bot commented on NIFI-1660:
--
Github user ckmcd commented on the pull request:
https://github.com/apache/nifi/pull/303#issuecomment-221019823
@mattyb149, @markap14 I am working on your suggestions. Thanks.
> Enhance the expression language with jsonPath function
> --
>
> Key: NIFI-1660
> URL: https://issues.apache.org/jira/browse/NIFI-1660
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Affects Versions: 0.5.1
>Reporter: Christopher McDermott
>Assignee: Joseph Witt
>Priority: Minor
> Fix For: 1.0.0, 0.7.0
>
>
> jsonPath would evaluate a JSON path provided, as an argument, against the
> subject.
> Example
> {quote}
> $\{kafka.key:jsonPath('$.foo.bar')\}
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-1781) Update UI to reflect component level authorization
[ https://issues.apache.org/jira/browse/NIFI-1781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296543#comment-15296543 ] ASF GitHub Bot commented on NIFI-1781: -- GitHub user mcgilman opened a pull request: https://github.com/apache/nifi/pull/461 NIFI-1781: UI authorization updates NIFI-1781: - Including access policies in the breadcrumb's trail. - Updating toolbox according to group access policies. - Updating actions in palette based on selection access policies. NIFI-1554: - Introducing authorization during two phase commit. - Introducing snippet authorization according to the encapsulated components and the action performed. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mcgilman/nifi NIFI-1781 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/461.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #461 > Update UI to reflect component level authorization > -- > > Key: NIFI-1781 > URL: https://issues.apache.org/jira/browse/NIFI-1781 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman > Fix For: 1.0.0 > > Attachments: nifi-component-samples.png, nifi-sample-flow.png > > > - Update to UI to visual access level per component -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1660) Enhance the expression language with jsonPath function
[
https://issues.apache.org/jira/browse/NIFI-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296459#comment-15296459
]
ASF GitHub Bot commented on NIFI-1660:
--
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/303#discussion_r64235176
--- Diff:
nifi-commons/nifi-expression-language/src/test/java/org/apache/nifi/attribute/expression/language/TestQuery.java
---
@@ -233,6 +233,24 @@ public void testEmbeddedExpressionsAndQuotes() {
}
@Test
+public void testJsonPath() {
+final Map attributes = new HashMap<>();
+attributes.put("json",
--- End diff --
Would recommend we pull the JSON out into a file in src/test/resources.
Having this live in the codebase makes it very difficult to read & edit
> Enhance the expression language with jsonPath function
> --
>
> Key: NIFI-1660
> URL: https://issues.apache.org/jira/browse/NIFI-1660
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Affects Versions: 0.5.1
>Reporter: Christopher McDermott
>Assignee: Joseph Witt
>Priority: Minor
> Fix For: 1.0.0, 0.7.0
>
>
> jsonPath would evaluate a JSON path provided, as an argument, against the
> subject.
> Example
> {quote}
> $\{kafka.key:jsonPath('$.foo.bar')\}
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-1660) Enhance the expression language with jsonPath function
[
https://issues.apache.org/jira/browse/NIFI-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296455#comment-15296455
]
ASF GitHub Bot commented on NIFI-1660:
--
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/303#discussion_r64234770
--- Diff:
nifi-commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/JsonPathEvaluator.java
---
@@ -0,0 +1,98 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.attribute.expression.language.evaluation.functions;
+
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+
+import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
+import
org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
+import
org.apache.nifi.attribute.expression.language.evaluation.StringEvaluator;
+import
org.apache.nifi.attribute.expression.language.evaluation.StringQueryResult;
+
+import com.jayway.jsonpath.Configuration;
+import com.jayway.jsonpath.DocumentContext;
+import com.jayway.jsonpath.InvalidJsonException;
+import com.jayway.jsonpath.JsonPath;
+import com.jayway.jsonpath.spi.json.JacksonJsonProvider;
+import com.jayway.jsonpath.spi.json.JsonProvider;
+
+
+public class JsonPathEvaluator extends StringEvaluator {
+
+private static final StringQueryResult NULL_RESULT = new
StringQueryResult("");
+private static final Configuration STRICT_PROVIDER_CONFIGURATION =
Configuration.builder().jsonProvider(new JacksonJsonProvider()).build();
+private static final JsonProvider JSON_PROVIDER =
STRICT_PROVIDER_CONFIGURATION.jsonProvider();
+
+private final Evaluator subject;
+private final Evaluator jsonPathExp;
+
+public JsonPathEvaluator(final Evaluator subject, final
Evaluator jsonPathExp) {
+this.subject = subject;
+this.jsonPathExp = jsonPathExp;
+}
+
+@Override
+public QueryResult evaluate(final Map
attributes) {
+final String subjectValue =
subject.evaluate(attributes).getValue();
+if (subjectValue == null || subjectValue.length() == 0) {
+return NULL_RESULT;
+}
+DocumentContext documentContext = null;
+try {
+documentContext =
validateAndEstablishJsonContext(subjectValue);
+} catch (InvalidJsonException e) {
+return NULL_RESULT;
+}
+
+final JsonPath compiledJsonPath =
JsonPath.compile(jsonPathExp.evaluate(attributes).getValue());
--- End diff --
In the vast majority of use cases, I would expect that the JSON Path will
be a literal string that does not reference any attributes. In this case, I
would want to avoid compiling the JSON Path every time. Would recommend that if
the jsonPathExp is an instance of the StringLiteralEvaluator, we pre-compile
the JSON Path. We follow a similar pattern with the MatchesEvaluator for
pre-compiling regular expressions, and it can make a pretty huge difference on
performance.
> Enhance the expression language with jsonPath function
> --
>
> Key: NIFI-1660
> URL: https://issues.apache.org/jira/browse/NIFI-1660
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Affects Versions: 0.5.1
>Reporter: Christopher McDermott
>Assignee: Joseph Witt
>Priority: Minor
> Fix For: 1.0.0, 0.7.0
>
>
> jsonPath would evaluate a JSON path provided, as an argument, against the
> subject.
> Example
> {quote}
> $\{kafka.key:jsonPath('$.foo.bar')\}
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (NIFI-1323) UI visual design enhancement
[ https://issues.apache.org/jira/browse/NIFI-1323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Moran updated NIFI-1323: Attachment: nifi-global-menu-pending-u...@2x.png Attached *nifi-global-menu-pending-u...@2x.png* Demonstrates a generic notification with an aggregate count that appears on the global menu icon. When the menu is displayed, the same notification (and individual count) appears in line with the corresponding item. > UI visual design enhancement > > > Key: NIFI-1323 > URL: https://issues.apache.org/jira/browse/NIFI-1323 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Affects Versions: 1.0.0 >Reporter: Rob Moran >Priority: Trivial > Attachments: flowfont.zip, nifi-add-processor-dialog.png, > nifi-component-samples.png, > nifi-configure-processor-properties-set-value.png, > nifi-configure-processor-properties.png, > nifi-configure-processor-scheduling.png, > nifi-configure-processor-settings.png, nifi-dialog-samp...@800px.png, > nifi-drop.svg, nifi-global-menu-pending-u...@2x.png, nifi-global-menu.png, > nifi-interaction-and-menu-samples.png, nifi-lineage-gra...@800px.png, > nifi-logo.svg, nifi-sample-fl...@800px.png, nifi-sample-flow.png, > nifi-shell-samp...@800px.png > > > (I will attach mockups and supporting files as they become available) > I am starting to work on a design to modernize the look and feel of the NiFi > UI. The initial focus of the design is to freshen the UI (flat design, SVG > icons, etc.). Additionally, the new design will propose usability > improvements such as exposing more flow-related actions into collapsible > panes, improving hierarchy of information, etc. > Going forward, the design plan is to help lay the foundation for other UI/UX > related issues such as those documented in NIFI-951. > --- > *flowfont.zip* > Contains icon font and supporting files > *nifi-add-processor-dialog.png* > Dialog sample. This sample shows the 'Add Processor' dialog. > *nifi-component-samples.png* > To show styling for all components, as well as those components when a user > is unauthorized to access. > *nifi-configure-processor-properties* > *nifi-configure-processor-properties-set-value* > *nifi-configure-processor-scheduling* > *nifi-configure-processor-settings* > Configure Processor dialog. See related Comments below (in Activity section). > *nifi-dialog-sample-@800px* > Dialog sample in 800px wide viewport. This sample shows the the 'Details' tab > of a provenance event. > *nifi-drop.svg* > NiFi logo without 'nifi > *nifi-global-menu* > To show global menu > *nifi-interaction-and-menu-samples.png* > To demonstrate user interactions - hover states, tooltips, menus, etc. > *nifi-lineage-graph-@800px* > To show lineage graph with explicit action to get back to data provenance > event list. > *nifi-sample-flow-@800px* > Shows a very useable UI down to around 800px in width. The thinking here is > that at anything lower than this, the NiFi user experience will change to > more of a monitoring and/or administrative type workflow. Future mockups will > be created to illustrate this. > *nifi-logo.svg* > NiFi logo complete > *nifi-sample-flow.png* > Mockup of sample flow. Updated to show revised tool and status bars. > Management related actions will move to a menu via mouseover (see > _nifi-global-menu_). Added benefits here include reducing clutter and more > user-friendly menu with text labels to reduce time spent scanning only a > large set of icons. This also helps gain valuable viewport width in a browser > (see _nifi-sample-flow-@800px_) > *nifi-shell-sample-@800px* > Shell sample in 800px wide viewport. This sample shows the 'Data Provenance' > table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1907) SiteToSiteClient not properly using keystore and truststore properties
[ https://issues.apache.org/jira/browse/NIFI-1907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296387#comment-15296387 ] ASF GitHub Bot commented on NIFI-1907: -- Github user joewitt commented on the pull request: https://github.com/apache/nifi/pull/457#issuecomment-220988156 @bbende i think the move makes sense to improve the lifecycle of when such issues are detected. However, it does seem like throwing RuntimeExceptions for those could be improved. Could they instead be considered IllegalStateExceptions and be a documented part of that interface? Also for the last runtime exception should we include the underlying exception when we create a new exception or do we intentionally bury it here? That said these questions/concerns are not 'because of this change' and i do agree the context should be constructed during the building phase of the client so I'm a +1 anyway. > SiteToSiteClient not properly using keystore and truststore properties > -- > > Key: NIFI-1907 > URL: https://issues.apache.org/jira/browse/NIFI-1907 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 0.6.1 >Reporter: Bryan Bende >Assignee: Bryan Bende > Fix For: 0.7.0 > > > SiteToSiteClient.Builder allows setting an SSLContext or setting all of the > individual SSL properties, it then has a method getSSLContext() which says > that if the sslContext is null return that, otherwise use the properties to > create one: > https://github.com/apache/nifi/blob/e4b7e47836edf47042973e604005058c28eed23b/nifi-commons/nifi-site-to-site-client/src/main/java/org/apache/nifi/remote/client/SiteToSiteClient.java#L575 > The problem is this getSSLContext() is never called. When the builder's > build() method is called, it passes the builder to > StandardSiteToSiteClientConfig and just assigns all the member variables with > direct access: > https://github.com/apache/nifi/blob/e4b7e47836edf47042973e604005058c28eed23b/nifi-commons/nifi-site-to-site-client/src/main/java/org/apache/nifi/remote/client/SiteToSiteClient.java#L722 > Later on in SocketClient it will call SiteToSiteClient.getSSLContext(): > https://github.com/apache/nifi/blob/e4b7e47836edf47042973e604005058c28eed23b/nifi-commons/nifi-site-to-site-client/src/main/java/org/apache/nifi/remote/client/socket/SocketClient.java#L52 > This will still be null here if only the SSL properties were initially > specified on the builder, and therefore won't end up creating an Https > connection and thus failing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1323) UI visual design enhancement
[ https://issues.apache.org/jira/browse/NIFI-1323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Moran updated NIFI-1323: Attachment: flowfont.zip Attached updated *flowfont.zip* # Updated icon-template (0xe809) # New icon-template-import (0xe810) # New icon-template-create (0xe811) > UI visual design enhancement > > > Key: NIFI-1323 > URL: https://issues.apache.org/jira/browse/NIFI-1323 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Affects Versions: 1.0.0 >Reporter: Rob Moran >Priority: Trivial > Attachments: flowfont.zip, nifi-add-processor-dialog.png, > nifi-component-samples.png, > nifi-configure-processor-properties-set-value.png, > nifi-configure-processor-properties.png, > nifi-configure-processor-scheduling.png, > nifi-configure-processor-settings.png, nifi-dialog-samp...@800px.png, > nifi-drop.svg, nifi-global-menu.png, nifi-interaction-and-menu-samples.png, > nifi-lineage-gra...@800px.png, nifi-logo.svg, nifi-sample-fl...@800px.png, > nifi-sample-flow.png, nifi-shell-samp...@800px.png > > > (I will attach mockups and supporting files as they become available) > I am starting to work on a design to modernize the look and feel of the NiFi > UI. The initial focus of the design is to freshen the UI (flat design, SVG > icons, etc.). Additionally, the new design will propose usability > improvements such as exposing more flow-related actions into collapsible > panes, improving hierarchy of information, etc. > Going forward, the design plan is to help lay the foundation for other UI/UX > related issues such as those documented in NIFI-951. > --- > *flowfont.zip* > Contains icon font and supporting files > *nifi-add-processor-dialog.png* > Dialog sample. This sample shows the 'Add Processor' dialog. > *nifi-component-samples.png* > To show styling for all components, as well as those components when a user > is unauthorized to access. > *nifi-configure-processor-properties* > *nifi-configure-processor-properties-set-value* > *nifi-configure-processor-scheduling* > *nifi-configure-processor-settings* > Configure Processor dialog. See related Comments below (in Activity section). > *nifi-dialog-sample-@800px* > Dialog sample in 800px wide viewport. This sample shows the the 'Details' tab > of a provenance event. > *nifi-drop.svg* > NiFi logo without 'nifi > *nifi-global-menu* > To show global menu > *nifi-interaction-and-menu-samples.png* > To demonstrate user interactions - hover states, tooltips, menus, etc. > *nifi-lineage-graph-@800px* > To show lineage graph with explicit action to get back to data provenance > event list. > *nifi-sample-flow-@800px* > Shows a very useable UI down to around 800px in width. The thinking here is > that at anything lower than this, the NiFi user experience will change to > more of a monitoring and/or administrative type workflow. Future mockups will > be created to illustrate this. > *nifi-logo.svg* > NiFi logo complete > *nifi-sample-flow.png* > Mockup of sample flow. Updated to show revised tool and status bars. > Management related actions will move to a menu via mouseover (see > _nifi-global-menu_). Added benefits here include reducing clutter and more > user-friendly menu with text labels to reduce time spent scanning only a > large set of icons. This also helps gain valuable viewport width in a browser > (see _nifi-sample-flow-@800px_) > *nifi-shell-sample-@800px* > Shell sample in 800px wide viewport. This sample shows the 'Data Provenance' > table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1323) UI visual design enhancement
[ https://issues.apache.org/jira/browse/NIFI-1323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Moran updated NIFI-1323: Attachment: (was: flowfont.zip) > UI visual design enhancement > > > Key: NIFI-1323 > URL: https://issues.apache.org/jira/browse/NIFI-1323 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Affects Versions: 1.0.0 >Reporter: Rob Moran >Priority: Trivial > Attachments: nifi-add-processor-dialog.png, > nifi-component-samples.png, > nifi-configure-processor-properties-set-value.png, > nifi-configure-processor-properties.png, > nifi-configure-processor-scheduling.png, > nifi-configure-processor-settings.png, nifi-dialog-samp...@800px.png, > nifi-drop.svg, nifi-global-menu.png, nifi-interaction-and-menu-samples.png, > nifi-lineage-gra...@800px.png, nifi-logo.svg, nifi-sample-fl...@800px.png, > nifi-sample-flow.png, nifi-shell-samp...@800px.png > > > (I will attach mockups and supporting files as they become available) > I am starting to work on a design to modernize the look and feel of the NiFi > UI. The initial focus of the design is to freshen the UI (flat design, SVG > icons, etc.). Additionally, the new design will propose usability > improvements such as exposing more flow-related actions into collapsible > panes, improving hierarchy of information, etc. > Going forward, the design plan is to help lay the foundation for other UI/UX > related issues such as those documented in NIFI-951. > --- > *flowfont.zip* > Contains icon font and supporting files > *nifi-add-processor-dialog.png* > Dialog sample. This sample shows the 'Add Processor' dialog. > *nifi-component-samples.png* > To show styling for all components, as well as those components when a user > is unauthorized to access. > *nifi-configure-processor-properties* > *nifi-configure-processor-properties-set-value* > *nifi-configure-processor-scheduling* > *nifi-configure-processor-settings* > Configure Processor dialog. See related Comments below (in Activity section). > *nifi-dialog-sample-@800px* > Dialog sample in 800px wide viewport. This sample shows the the 'Details' tab > of a provenance event. > *nifi-drop.svg* > NiFi logo without 'nifi > *nifi-global-menu* > To show global menu > *nifi-interaction-and-menu-samples.png* > To demonstrate user interactions - hover states, tooltips, menus, etc. > *nifi-lineage-graph-@800px* > To show lineage graph with explicit action to get back to data provenance > event list. > *nifi-sample-flow-@800px* > Shows a very useable UI down to around 800px in width. The thinking here is > that at anything lower than this, the NiFi user experience will change to > more of a monitoring and/or administrative type workflow. Future mockups will > be created to illustrate this. > *nifi-logo.svg* > NiFi logo complete > *nifi-sample-flow.png* > Mockup of sample flow. Updated to show revised tool and status bars. > Management related actions will move to a menu via mouseover (see > _nifi-global-menu_). Added benefits here include reducing clutter and more > user-friendly menu with text labels to reduce time spent scanning only a > large set of icons. This also helps gain valuable viewport width in a browser > (see _nifi-sample-flow-@800px_) > *nifi-shell-sample-@800px* > Shell sample in 800px wide viewport. This sample shows the 'Data Provenance' > table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-786) Add other supporting options for configuring credentials for AWS processors
[
https://issues.apache.org/jira/browse/NIFI-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15296205#comment-15296205
]
Joe Skora commented on NIFI-786:
I've tested on 0.7.0 and it functions as intended, but I'm currently having
trouble getting a secure instance of 1.0.0 running to test against.
> Add other supporting options for configuring credentials for AWS processors
> ---
>
> Key: NIFI-786
> URL: https://issues.apache.org/jira/browse/NIFI-786
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 0.3.0
>Reporter: Michael Kobit
>Assignee: James Wing
>Priority: Minor
> Fix For: 0.7.0
>
>
> I was looking at https://issues.apache.org/jira/browse/NIFI-770 and looked at
> how the AWS processors credentials are currently configured. As a NFM you
> have a few options with the properties right now:
> 1) set basic, static credentials
> 2) set a credentials properties filepath
> 3) set neither, use anonymous credentials
> I think it would be better if each AWS could rely on a ControllerService that
> returns `AWSCredentialsProvider` (instead of `AWSCredentials`) that gives
> all of the possible implementations that could be used, rather than relying
> on a static credentials. *Provider implementations can be refreshed and can
> also other more complicated implementations, but already have built in
> support for the Static and Properties file that are provided by NiFi today.
> My thinking is that the controller service would be something like
> public interface AwsCredentialsProviderService extends ControllerService {
> AWSCredentialsProvider getCredentialsProvider();
> }
> and you could have `StaticAwsCredentialsProviderService`,
> `PropertiesFileAwsCredentialsProviderService`, and
> `AnonymousAwsCredentialsProviderService` to provide the functionality that is
> supported right now. Additional credential providers could be added later, as
> there a bunch more AWS provided versions that I think could fit in well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
