[jira] [Created] (NIFI-1305) Clarify Jacadoc on DistributedMapCacheClient.getAndPutIfAbsent
Michael Kobit created NIFI-1305: --- Summary: Clarify Jacadoc on DistributedMapCacheClient.getAndPutIfAbsent Key: NIFI-1305 URL: https://issues.apache.org/jira/browse/NIFI-1305 Project: Apache NiFi Issue Type: Improvement Components: Documentation & Website Affects Versions: 0.4.0 Reporter: Michael Kobit Priority: Trivial The Javadoc for this method on DistributedMapCacheClient (https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-services/nifi-distributed-cache-client-service-api/src/main/java/org/apache/nifi/distributed/cache/client/DistributedMapCacheClient.java#L53-L71) is unclear about what happens if the key does not exist in the cache. It looks like from the implementation of DistributedMapCacheClientService that it returns the value if it present, otherwise returns the computed value. The Javadoc doesn't state what is returned happens if an item does not exist. I spent some time trying to figure out what makes sense from a client's perspective, and I had some trouble trying to understand what should happen. It seems obvious to me now that the cache should return either the existing value or the newly computed value, but it took some time to get there. I think the documentation should be clearer about what is returned in the case where the item does not exist -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-786) Add other supporting options for configuring credentials for AWS processors
[
https://issues.apache.org/jira/browse/NIFI-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14951878#comment-14951878
]
Michael Kobit commented on NIFI-786:
[~markap14] That sounds good to me.
> Add other supporting options for configuring credentials for AWS processors
> ---
>
> Key: NIFI-786
> URL: https://issues.apache.org/jira/browse/NIFI-786
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 0.3.0
>Reporter: Michael Kobit
>Priority: Minor
>
> I was looking at https://issues.apache.org/jira/browse/NIFI-770 and looked at
> how the AWS processors credentials are currently configured. As a NFM you
> have a few options with the properties right now:
> 1) set basic, static credentials
> 2) set a credentials properties filepath
> 3) set neither, use anonymous credentials
> I think it would be better if each AWS could rely on a ControllerService that
> returns `AWSCredentialsProvider` (instead of `AWSCredentials`) that gives
> all of the possible implementations that could be used, rather than relying
> on a static credentials. *Provider implementations can be refreshed and can
> also other more complicated implementations, but already have built in
> support for the Static and Properties file that are provided by NiFi today.
> My thinking is that the controller service would be something like
> public interface AwsCredentialsProviderService extends ControllerService {
> AWSCredentialsProvider getCredentialsProvider();
> }
> and you could have `StaticAwsCredentialsProviderService`,
> `PropertiesFileAwsCredentialsProviderService`, and
> `AnonymousAwsCredentialsProviderService` to provide the functionality that is
> supported right now. Additional credential providers could be added later, as
> there a bunch more AWS provided versions that I think could fit in well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (NIFI-786) Add other supporting options for configuring credentials for AWS processors
[
https://issues.apache.org/jira/browse/NIFI-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14949828#comment-14949828
]
Michael Kobit edited comment on NIFI-786 at 10/9/15 3:20 AM:
-
[~markap14] Hey Mark, my initial idea was that the ControllerService}} would
return an
[{{AWSCredentialsProvider}}|http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/AWSCredentialsProvider.html]
not an
[|http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/AWSCredentials.html].
The {{*Provider}} can be used for supplying credentials that don't change, the
static access and secret key use case now, but also allows for complicated
implementations, like the ones that integrate with key management systems (like
via [STS|http://docs.aws.amazon.com/STS/latest/APIReference/Welcome.html]). I
am fairly certain it is supported by all of the SDK vended
{{Amazon*ServiceClient}}
For the use case of typing them in the UI, it would be supported by the, what I
called above, {{StaticAwsCredentialsProviderService}} {{ControllerService}}.
Users would type in the access/secret keys like they can do now, although they
would have to set it up at the controller service level rather than at a
processor level. This way it could also be reused across multiple AWS
processors.
For the properties file use case, it would be supported by the, what I called
above, {{PropertiesFileAwsCredentialsProviderService}} {{ControllerService}}.
This implementation I would see using the AWS SDK
[PropertiesFileCredentialsProvider|http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/PropertiesFileCredentialsProvider.html].
It covers the file based use case, but it also (currently) allows for
credential rotation in the file since it gets loaded every time.
One of the main use cases I was thinking of is when running on AWS and you are
["Using IAM Roles to Grant Access to AWS Resources on Amazon
EC2"|http://docs.aws.amazon.com/AWSSdkDocsJava/latest/DeveloperGuide/java-dg-roles.html].
Basically distributing the credentials to the EC2 instances themselves rather
than managing them in your application. The keys are temporary and AWS
automatically rotates them, so using the AWS Java SDK
{{AWSCredentialsProvider}} implementation is preferred.
A couple other possible use cases that could be supported fairly easily:
1. environment variables
2. Java system properties
And then there is the option of allowing consumers to define their own
{{ControllerService}} for retrieving credentials.
was (Author: mkobit):
[~markap14] Hey Mark, my initial idea was that the ControllerService}} would
return an
[AWSCredentialsProvider|http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/AWSCredentialsProvider.html]
not an
[|http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/AWSCredentials.html].
The {{*Provider}} can be used for supplying credentials that don't change, the
static access and secret key use case now, but also allows for complicated
implementations, like the ones that integrate with key management systems (like
via [STS|http://docs.aws.amazon.com/STS/latest/APIReference/Welcome.html]). I
am fairly certain it is supported by all of the SDK vended
{{Amazon*ServiceClient}}
For the use case of typing them in the UI, it would be supported by the, what I
called above, {{StaticAwsCredentialsProviderService}} {{ControllerService}}.
Users would type in the access/secret keys like they can do now, although they
would have to set it up at the controller service level rather than at a
processor level. This way it could also be reused across multiple AWS
processors.
For the properties file use case, it would be supported by the, what I called
above, {{PropertiesFileAwsCredentialsProviderService}} {{ControllerService}}.
This implementation I would see using the AWS SDK
[PropertiesFileCredentialsProvider|http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/PropertiesFileCredentialsProvider.html].
It covers the file based use case, but it also (currently) allows for
credential rotation in the file since it gets loaded every time.
One of the main use cases I was thinking of is when running on AWS and you are
["Using IAM Roles to Grant Access to AWS Resources on Amazon
EC2"|http://docs.aws.amazon.com/AWSSdkDocsJava/latest/DeveloperGuide/java-dg-roles.html].
Basically distributing the credentials to the EC2 instances themselves rather
than managing them in your application. The keys are temporary and AWS
automatically rotates them, so using the AWS Java SDK
{{AWSCredentialsProvider}} implementation is preferred.
A couple other possible use cases that could be supported fairly easily:
1. environment variables
2. Java system properties
And then there is the option of allowing consumers to define their own
{{Contro
[jira] [Comment Edited] (NIFI-786) Add other supporting options for configuring credentials for AWS processors
[
https://issues.apache.org/jira/browse/NIFI-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14949828#comment-14949828
]
Michael Kobit edited comment on NIFI-786 at 10/9/15 3:21 AM:
-
[~markap14] Hey Mark, my initial idea was that the {{ControllerService}} would
return an
[{{AWSCredentialsProvider}}|http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/AWSCredentialsProvider.html]
not an
[{{AWSCredentials}}|http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/AWSCredentials.html].
The {{*Provider}} can be used for supplying credentials that don't change, the
static access and secret key use case now, but also allows for complicated
implementations, like the ones that integrate with key management systems (like
via [STS|http://docs.aws.amazon.com/STS/latest/APIReference/Welcome.html]). I
am fairly certain it is supported by all of the SDK vended
{{Amazon*ServiceClient}}
For the use case of typing them in the UI, it would be supported by the, what I
called above, {{StaticAwsCredentialsProviderService}} {{ControllerService}}.
Users would type in the access/secret keys like they can do now, although they
would have to set it up at the controller service level rather than at a
processor level. This way it could also be reused across multiple AWS
processors.
For the properties file use case, it would be supported by the, what I called
above, {{PropertiesFileAwsCredentialsProviderService}} {{ControllerService}}.
This implementation I would see using the AWS SDK
[PropertiesFileCredentialsProvider|http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/PropertiesFileCredentialsProvider.html].
It covers the file based use case, but it also (currently) allows for
credential rotation in the file since it gets loaded every time.
One of the main use cases I was thinking of is when running on AWS and you are
["Using IAM Roles to Grant Access to AWS Resources on Amazon
EC2"|http://docs.aws.amazon.com/AWSSdkDocsJava/latest/DeveloperGuide/java-dg-roles.html].
Basically distributing the credentials to the EC2 instances themselves rather
than managing them in your application. The keys are temporary and AWS
automatically rotates them, so using the AWS Java SDK
{{AWSCredentialsProvider}} implementation is preferred.
A couple other possible use cases that could be supported fairly easily:
1. environment variables
2. Java system properties
And then there is the option of allowing consumers to define their own
{{ControllerService}} for retrieving credentials.
was (Author: mkobit):
[~markap14] Hey Mark, my initial idea was that the ControllerService}} would
return an
[{{AWSCredentialsProvider}}|http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/AWSCredentialsProvider.html]
not an
[|http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/AWSCredentials.html].
The {{*Provider}} can be used for supplying credentials that don't change, the
static access and secret key use case now, but also allows for complicated
implementations, like the ones that integrate with key management systems (like
via [STS|http://docs.aws.amazon.com/STS/latest/APIReference/Welcome.html]). I
am fairly certain it is supported by all of the SDK vended
{{Amazon*ServiceClient}}
For the use case of typing them in the UI, it would be supported by the, what I
called above, {{StaticAwsCredentialsProviderService}} {{ControllerService}}.
Users would type in the access/secret keys like they can do now, although they
would have to set it up at the controller service level rather than at a
processor level. This way it could also be reused across multiple AWS
processors.
For the properties file use case, it would be supported by the, what I called
above, {{PropertiesFileAwsCredentialsProviderService}} {{ControllerService}}.
This implementation I would see using the AWS SDK
[PropertiesFileCredentialsProvider|http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/PropertiesFileCredentialsProvider.html].
It covers the file based use case, but it also (currently) allows for
credential rotation in the file since it gets loaded every time.
One of the main use cases I was thinking of is when running on AWS and you are
["Using IAM Roles to Grant Access to AWS Resources on Amazon
EC2"|http://docs.aws.amazon.com/AWSSdkDocsJava/latest/DeveloperGuide/java-dg-roles.html].
Basically distributing the credentials to the EC2 instances themselves rather
than managing them in your application. The keys are temporary and AWS
automatically rotates them, so using the AWS Java SDK
{{AWSCredentialsProvider}} implementation is preferred.
A couple other possible use cases that could be supported fairly easily:
1. environment variables
2. Java system properties
And then there is the option of allowing consumers to de
[jira] [Commented] (NIFI-786) Add other supporting options for configuring credentials for AWS processors
[
https://issues.apache.org/jira/browse/NIFI-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14949828#comment-14949828
]
Michael Kobit commented on NIFI-786:
[~markap14] Hey Mark, my initial idea was that the ControllerService}} would
return an
[AWSCredentialsProvider|http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/AWSCredentialsProvider.html]
not an
[|http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/AWSCredentials.html].
The {{*Provider}} can be used for supplying credentials that don't change, the
static access and secret key use case now, but also allows for complicated
implementations, like the ones that integrate with key management systems (like
via [STS|http://docs.aws.amazon.com/STS/latest/APIReference/Welcome.html]). I
am fairly certain it is supported by all of the SDK vended
{{Amazon*ServiceClient}}
For the use case of typing them in the UI, it would be supported by the, what I
called above, {{StaticAwsCredentialsProviderService}} {{ControllerService}}.
Users would type in the access/secret keys like they can do now, although they
would have to set it up at the controller service level rather than at a
processor level. This way it could also be reused across multiple AWS
processors.
For the properties file use case, it would be supported by the, what I called
above, {{PropertiesFileAwsCredentialsProviderService}} {{ControllerService}}.
This implementation I would see using the AWS SDK
[PropertiesFileCredentialsProvider|http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/auth/PropertiesFileCredentialsProvider.html].
It covers the file based use case, but it also (currently) allows for
credential rotation in the file since it gets loaded every time.
One of the main use cases I was thinking of is when running on AWS and you are
["Using IAM Roles to Grant Access to AWS Resources on Amazon
EC2"|http://docs.aws.amazon.com/AWSSdkDocsJava/latest/DeveloperGuide/java-dg-roles.html].
Basically distributing the credentials to the EC2 instances themselves rather
than managing them in your application. The keys are temporary and AWS
automatically rotates them, so using the AWS Java SDK
{{AWSCredentialsProvider}} implementation is preferred.
A couple other possible use cases that could be supported fairly easily:
1. environment variables
2. Java system properties
And then there is the option of allowing consumers to define their own
{{ControllerService}} for retrieving credentials.
> Add other supporting options for configuring credentials for AWS processors
> ---
>
> Key: NIFI-786
> URL: https://issues.apache.org/jira/browse/NIFI-786
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 0.3.0
>Reporter: Michael Kobit
>Priority: Minor
>
> I was looking at https://issues.apache.org/jira/browse/NIFI-770 and looked at
> how the AWS processors credentials are currently configured. As a NFM you
> have a few options with the properties right now:
> 1) set basic, static credentials
> 2) set a credentials properties filepath
> 3) set neither, use anonymous credentials
> I think it would be better if each AWS could rely on a ControllerService that
> returns `AWSCredentialsProvider` (instead of `AWSCredentials`) that gives
> all of the possible implementations that could be used, rather than relying
> on a static credentials. *Provider implementations can be refreshed and can
> also other more complicated implementations, but already have built in
> support for the Static and Properties file that are provided by NiFi today.
> My thinking is that the controller service would be something like
> public interface AwsCredentialsProviderService extends ControllerService {
> AWSCredentialsProvider getCredentialsProvider();
> }
> and you could have `StaticAwsCredentialsProviderService`,
> `PropertiesFileAwsCredentialsProviderService`, and
> `AnonymousAwsCredentialsProviderService` to provide the functionality that is
> supported right now. Additional credential providers could be added later, as
> there a bunch more AWS provided versions that I think could fit in well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-786) Add other supporting options for configuring credentials for AWS processors
[
https://issues.apache.org/jira/browse/NIFI-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14945142#comment-14945142
]
Michael Kobit commented on NIFI-786:
Any thoughts on this for the 1.0.0 version?
It seems like there could be 2 ways to do this:
1. Maintain backwards compatibility by checking if the `Access Key` and `Secret
Key` are set or 'Credentials File', and if not use this new ControllerService
2. Get rid of those, and only use this ControllerService.
Since 1.0.0 (I believe) would be introducing some breaking changes, would this
be considered for it?
> Add other supporting options for configuring credentials for AWS processors
> ---
>
> Key: NIFI-786
> URL: https://issues.apache.org/jira/browse/NIFI-786
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 0.3.0
>Reporter: Michael Kobit
>Priority: Minor
>
> I was looking at https://issues.apache.org/jira/browse/NIFI-770 and looked at
> how the AWS processors credentials are currently configured. As a NFM you
> have a few options with the properties right now:
> 1) set basic, static credentials
> 2) set a credentials properties filepath
> 3) set neither, use anonymous credentials
> I think it would be better if each AWS could rely on a ControllerService that
> returns `AWSCredentialsProvider` (instead of `AWSCredentials`) that gives
> all of the possible implementations that could be used, rather than relying
> on a static credentials. *Provider implementations can be refreshed and can
> also other more complicated implementations, but already have built in
> support for the Static and Properties file that are provided by NiFi today.
> My thinking is that the controller service would be something like
> public interface AwsCredentialsProviderService extends ControllerService {
> AWSCredentialsProvider getCredentialsProvider();
> }
> and you could have `StaticAwsCredentialsProviderService`,
> `PropertiesFileAwsCredentialsProviderService`, and
> `AnonymousAwsCredentialsProviderService` to provide the functionality that is
> supported right now. Additional credential providers could be added later, as
> there a bunch more AWS provided versions that I think could fit in well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (NIFI-941) Same-page anchor links not present in URL after clicking
Michael Kobit created NIFI-941: -- Summary: Same-page anchor links not present in URL after clicking Key: NIFI-941 URL: https://issues.apache.org/jira/browse/NIFI-941 Project: Apache NiFi Issue Type: Bug Components: Documentation & Website Affects Versions: 0.2.1 Environment: Firefox and Chrome on Mac OSX Reporter: Michael Kobit Priority: Minor Clicking on some documentation inter-page links from a docs page, for example https://nifi.apache.org/docs.html or https://nifi.apache.org/developer-guide.html, causes you to move to that anchor in the page but the URL in the address bar remains the same. Back/forward seem to work fine. This is kind of annoying when I am trying to send somebody documentation and I usually copy the URL, but the URL hasn't changed to reflect where I currently am in the page -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-940) Expose 'filename' as a Property on PutFile, PutHDFS, PutFTP, PutSFTP, and other Put-esque processors
[
https://issues.apache.org/jira/browse/NIFI-940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Kobit updated NIFI-940:
---
Affects Version/s: 0.3.0
> Expose 'filename' as a Property on PutFile, PutHDFS, PutFTP, PutSFTP, and
> other Put-esque processors
>
>
> Key: NIFI-940
> URL: https://issues.apache.org/jira/browse/NIFI-940
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Affects Versions: 0.3.0
>Reporter: Michael Kobit
>
> Allow a NFM to set the filename to save files as. Should support Expression
> language, and default to '${filename}'
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (NIFI-940) Expose 'filename' as a Property on PutFile, PutHDFS, PutFTP, PutSFTP, and other Put-esque processors
Michael Kobit created NIFI-940:
--
Summary: Expose 'filename' as a Property on PutFile, PutHDFS,
PutFTP, PutSFTP, and other Put-esque processors
Key: NIFI-940
URL: https://issues.apache.org/jira/browse/NIFI-940
Project: Apache NiFi
Issue Type: Improvement
Components: Core Framework
Reporter: Michael Kobit
Allow a NFM to set the filename to save files as. Should support Expression
language, and default to '${filename}'
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (NIFI-786) Add other supporting options for configuring credentials for AWS processors
Michael Kobit created NIFI-786:
--
Summary: Add other supporting options for configuring credentials
for AWS processors
Key: NIFI-786
URL: https://issues.apache.org/jira/browse/NIFI-786
Project: Apache NiFi
Issue Type: Improvement
Affects Versions: 0.3.0
Reporter: Michael Kobit
Priority: Minor
I was looking at https://issues.apache.org/jira/browse/NIFI-770 and looked at
how the AWS processors credentials are currently configured. As a NFM you have
a few options with the properties right now:
1) set basic, static credentials
2) set a credentials properties filepath
3) set neither, use anonymous credentials
I think it would be better if each AWS could rely on a ControllerService that
returns `AWSCredentialsProvider` (instead of `AWSCredentials`) that gives all
of the possible implementations that could be used, rather than relying on a
static credentials. *Provider implementations can be refreshed and can also
other more complicated implementations, but already have built in support for
the Static and Properties file that are provided by NiFi today.
My thinking is that the controller service would be something like
public interface AwsCredentialsProviderService extends ControllerService {
AWSCredentialsProvider getCredentialsProvider();
}
and you could have `StaticAwsCredentialsProviderService`,
`PropertiesFileAwsCredentialsProviderService`, and
`AnonymousAwsCredentialsProviderService` to provide the functionality that is
supported right now. Additional credential providers could be added later, as
there a bunch more AWS provided versions that I think could fit in well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
