[jira] [Commented] (NIFI-974) Improve error messages included in Exceptions that are thrown by ProcessSession
[ https://issues.apache.org/jira/browse/NIFI-974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14946829#comment-14946829 ] Mark Payne commented on NIFI-974: - Though not actually in the ProcessSession, another error that I have seen recently that threw me for a loop when using Site-to-Site: "User DN is not known". This should have been much more informative, such as "Could not find any account for user with Distinguished Name . If you believe the user account does exist, please contact your administrator to ensure that the Authorization Providers are properly configured for your environment." > Improve error messages included in Exceptions that are thrown by > ProcessSession > --- > > Key: NIFI-974 > URL: https://issues.apache.org/jira/browse/NIFI-974 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mark Payne > Fix For: 0.4.0 > > > Exceptions that are thrown by StandardProcessSession tend to be geared toward > verbiage that makes sense within the context of the ProcessSession, but they > don't make much sense within the context of a Processor. Developers are often > confused by the meaning. > Examples include: > "... is not the most recent version of this FlowFIle." This should note that > the processor modified the FlowFIle via write, putAttribute, > putAllAttributes, etc. and then attempted to access an old version of the > FlowFile > "transfer relationship not specified". This should note that the Processor > has a bug that resulted in a FlowFile not being transferred or removed and > that all FlowFiles must be accounted for; this message should also indicate > whether the FlowFile was created in this session or if the FlowFile was > consumed from a queue. > There are likely other examples of poorly worded Exceptions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-516) MergeContent when merging 10,000s or more objects can cause too many open files
[
https://issues.apache.org/jira/browse/NIFI-516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14946840#comment-14946840
]
Mark Payne commented on NIFI-516:
-
Hey [~JPercivall] thanks for knocking this out! Looks good, for the most part,
but one section of the patch concerns me just a bit:
{code}
+
+// Allow processors to close the file after reading to avoid
too many files open or do smart session stream management.
+if(!allowSessionStreamManagement){
+currentReadClaimStream.close();
+}
{code}
If we close that read claim stream, we need to ensure that we also then set the
currentReadClaimStream to null and also null out the currentReadClaim.
Otherwise, some paths through the code could end up trying to read from a
stream that has already been closed.
> MergeContent when merging 10,000s or more objects can cause too many open
> files
> ---
>
> Key: NIFI-516
> URL: https://issues.apache.org/jira/browse/NIFI-516
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Joseph Witt
>Assignee: Joseph Percivall
> Fix For: 0.4.0
>
> Attachments: NIFI-516.patch, generateMerge.xml
>
>
> MergeContent really is just exposing an issue where the process session keeps
> file handles open longer than necessary.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-972) ExecuteSQL bug in createSchema() create Arvo Schema method
[ https://issues.apache.org/jira/browse/NIFI-972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14946843#comment-14946843 ] Mark Payne commented on NIFI-972: - Scott, I think that if there are no rows returned from the SQL query, it should not even create a FlowFile at all. Is there any value to creating an empty flowfile that indicates that the query was run and returned 0 results? It would also perhaps be beneficial to include an Attribute on the FlowFIle that indicates how many rows were selected. This would make it much easier to route FlowFiles. Thanks -Mark > ExecuteSQL bug in createSchema() create Arvo Schema method > -- > > Key: NIFI-972 > URL: https://issues.apache.org/jira/browse/NIFI-972 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Toivo Adams >Assignee: Toivo Adams > Attachments: nifi-972_01oct2015.patch > > > Jonathan Lyons reported: > Just getting started with NiFi here. I am attempting to run a static query > in MySQL using the ExecuteSQL processor. It is set to run on a 5 second > interval. Since ExecuteSQL appears to need an input flow file I'm using a > GenerateFlowFile processor to produce a random file every 5 seconds. > Unfortunately, I'm getting a very vague ArrayIndexOutOfBounds exception > when I hit play on the flow: > java.lang.ArrayIndexOutOfBoundsException: 8 > at org.apache.avro.generic.GenericData$Record.put(GenericData.java:129) > at org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream > at > org.apache.nifi.processors.standard.ExecuteSQL$1.process(ExecuteSQL.java:141) > It’s seem to be bug in createSchema() which creates Arvo schema. > When createSchema() will encounter unknown type, no Arvo column info will be > created. And Arvo schema has less columns than ResultSet row… -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-786) Add other supporting options for configuring credentials for AWS processors
[
https://issues.apache.org/jira/browse/NIFI-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14946860#comment-14946860
]
Mark Payne commented on NIFI-786:
-
This is certainly an interesting idea. What are the other types of credential
providers you're thinking of here? Browsing through the AWS SDK, it appears
that there are several implementations of AWSCredentials, but they really are
just different mechanisms for obtaining an Access Key and Secret Key and
perhaps a Session Token (for AWSSessionCredentials). In all cases, though, the
different implementations appear to be different mechanisms of determining
these 2-3 pieces of information (there is a BasicSessionCredentials for
instance that has the keys/token passed to it, and a PropertiesAWSCredentials
that reads them from a file. AnonymousAWSCredentials simply returns null).
If there are other types of credentials that I don't know about then a
Controller Service could be a great way to go about it, but if other
credentials are just more ways of determining what those access and secret keys
are, I would lean toward just having the user enter them into the UI.
So when you say "There are a bunch more AWS provided versions that I think
could fit in well," can you provide some details as to what those would be?
Thanks
-mark
> Add other supporting options for configuring credentials for AWS processors
> ---
>
> Key: NIFI-786
> URL: https://issues.apache.org/jira/browse/NIFI-786
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 0.3.0
>Reporter: Michael Kobit
>Priority: Minor
>
> I was looking at https://issues.apache.org/jira/browse/NIFI-770 and looked at
> how the AWS processors credentials are currently configured. As a NFM you
> have a few options with the properties right now:
> 1) set basic, static credentials
> 2) set a credentials properties filepath
> 3) set neither, use anonymous credentials
> I think it would be better if each AWS could rely on a ControllerService that
> returns `AWSCredentialsProvider` (instead of `AWSCredentials`) that gives
> all of the possible implementations that could be used, rather than relying
> on a static credentials. *Provider implementations can be refreshed and can
> also other more complicated implementations, but already have built in
> support for the Static and Properties file that are provided by NiFi today.
> My thinking is that the controller service would be something like
> public interface AwsCredentialsProviderService extends ControllerService {
> AWSCredentialsProvider getCredentialsProvider();
> }
> and you could have `StaticAwsCredentialsProviderService`,
> `PropertiesFileAwsCredentialsProviderService`, and
> `AnonymousAwsCredentialsProviderService` to provide the functionality that is
> supported right now. Additional credential providers could be added later, as
> there a bunch more AWS provided versions that I think could fit in well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-972) ExecuteSQL bug in createSchema() create Arvo Schema method
[ https://issues.apache.org/jira/browse/NIFI-972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14946870#comment-14946870 ] Toivo Adams commented on NIFI-972: -- Hi, Only value (empty flowfile) what I can see is that the query was run successfully and there were no rows which satisfied query criteria. How often such information is needed – don’t know. Thanks Toivo > ExecuteSQL bug in createSchema() create Arvo Schema method > -- > > Key: NIFI-972 > URL: https://issues.apache.org/jira/browse/NIFI-972 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Toivo Adams >Assignee: Toivo Adams > Attachments: nifi-972_01oct2015.patch > > > Jonathan Lyons reported: > Just getting started with NiFi here. I am attempting to run a static query > in MySQL using the ExecuteSQL processor. It is set to run on a 5 second > interval. Since ExecuteSQL appears to need an input flow file I'm using a > GenerateFlowFile processor to produce a random file every 5 seconds. > Unfortunately, I'm getting a very vague ArrayIndexOutOfBounds exception > when I hit play on the flow: > java.lang.ArrayIndexOutOfBoundsException: 8 > at org.apache.avro.generic.GenericData$Record.put(GenericData.java:129) > at org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream > at > org.apache.nifi.processors.standard.ExecuteSQL$1.process(ExecuteSQL.java:141) > It’s seem to be bug in createSchema() which creates Arvo schema. > When createSchema() will encounter unknown type, no Arvo column info will be > created. And Arvo schema has less columns than ResultSet row… -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-972) ExecuteSQL bug in createSchema() create Arvo Schema method
[ https://issues.apache.org/jira/browse/NIFI-972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14946896#comment-14946896 ] Toivo Adams commented on NIFI-972: -- Hi, Imagine following scenario: 1. We receive request. 2. Next we try to read some optional data from database. 3. Next we add optional data or skip it when it’s empty. 4. Next do something more… When we don’t create empty FlowFile how we can execute step 3? Thanks Toivo > ExecuteSQL bug in createSchema() create Arvo Schema method > -- > > Key: NIFI-972 > URL: https://issues.apache.org/jira/browse/NIFI-972 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Toivo Adams >Assignee: Toivo Adams > Attachments: nifi-972_01oct2015.patch > > > Jonathan Lyons reported: > Just getting started with NiFi here. I am attempting to run a static query > in MySQL using the ExecuteSQL processor. It is set to run on a 5 second > interval. Since ExecuteSQL appears to need an input flow file I'm using a > GenerateFlowFile processor to produce a random file every 5 seconds. > Unfortunately, I'm getting a very vague ArrayIndexOutOfBounds exception > when I hit play on the flow: > java.lang.ArrayIndexOutOfBoundsException: 8 > at org.apache.avro.generic.GenericData$Record.put(GenericData.java:129) > at org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream > at > org.apache.nifi.processors.standard.ExecuteSQL$1.process(ExecuteSQL.java:141) > It’s seem to be bug in createSchema() which creates Arvo schema. > When createSchema() will encounter unknown type, no Arvo column info will be > created. And Arvo schema has less columns than ResultSet row… -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-972) ExecuteSQL bug in createSchema() create Arvo Schema method
[
https://issues.apache.org/jira/browse/NIFI-972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14946967#comment-14946967
]
Mark Payne commented on NIFI-972:
-
Sorry, I was thinking that ExecuteSQL could be used only as a source processor,
but it is used as an in-line processor as well. We definitely will need to
output a FlowFIle. If we add an attribute to it though indicating how many rows
were selected, we could easily use a RouteOnAttribute with an expression like
${executesql.row.count:gt(0)} to route anything with at least 1 row in one
direction and anything with no rows to a different direction.
> ExecuteSQL bug in createSchema() create Arvo Schema method
> --
>
> Key: NIFI-972
> URL: https://issues.apache.org/jira/browse/NIFI-972
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Toivo Adams
>Assignee: Toivo Adams
> Attachments: nifi-972_01oct2015.patch
>
>
> Jonathan Lyons reported:
> Just getting started with NiFi here. I am attempting to run a static query
> in MySQL using the ExecuteSQL processor. It is set to run on a 5 second
> interval. Since ExecuteSQL appears to need an input flow file I'm using a
> GenerateFlowFile processor to produce a random file every 5 seconds.
> Unfortunately, I'm getting a very vague ArrayIndexOutOfBounds exception
> when I hit play on the flow:
> java.lang.ArrayIndexOutOfBoundsException: 8
> at org.apache.avro.generic.GenericData$Record.put(GenericData.java:129)
> at org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream
> at
> org.apache.nifi.processors.standard.ExecuteSQL$1.process(ExecuteSQL.java:141)
> It’s seem to be bug in createSchema() which creates Arvo schema.
> When createSchema() will encounter unknown type, no Arvo column info will be
> created. And Arvo schema has less columns than ResultSet row…
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (NIFI-516) MergeContent when merging 10,000s or more objects can cause too many open files
[ https://issues.apache.org/jira/browse/NIFI-516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Percivall updated NIFI-516: -- Attachment: NIFI-516.patch [~mpayne2] Good catch, I set it as null and rebased to master > MergeContent when merging 10,000s or more objects can cause too many open > files > --- > > Key: NIFI-516 > URL: https://issues.apache.org/jira/browse/NIFI-516 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Joseph Witt >Assignee: Joseph Percivall > Fix For: 0.4.0 > > Attachments: NIFI-516.patch, generateMerge.xml > > > MergeContent really is just exposing an issue where the process session keeps > file handles open longer than necessary. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-447) Add "Replacement Strategy" property to ReplaceText
[ https://issues.apache.org/jira/browse/NIFI-447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947038#comment-14947038 ] Mark Payne commented on NIFI-447: - Hey [~JPercivall] was just looking over this patch. I think the way that the Literal Replacement strategy was implemented is a little bit different than I was expecting. When the user chooses to do a Literal Replacement, there shouldn't be a regular expression entered at all. Instead, the value to search for is a literal string value. This means that the Regular Expression property should probably also be renamed to Search Value, because the search value is not always a regular expression anymore. > Add "Replacement Strategy" property to ReplaceText > -- > > Key: NIFI-447 > URL: https://issues.apache.org/jira/browse/NIFI-447 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne > Fix For: 0.4.0 > > Attachments: NIFI-447.patch > > > "Regular Expression" property should have displayName changed to "Search > Value" > ReplaceText should allow for a "Replacement Strategy" with values: > Prepend (Search Value is ignored; replacement value prepended to FlowFile) > Append (Search Value is ignored; replacement value appended to FlowFIle) > Regex Replace (current behavior, so this should be default) > Linewise Regex Replace (like current behavior if Evaluation Mode is set to > Line-by-Line) > Literal Replace (do not interpret the Replacement value > The Line-by-Line property should be removed; this is breaking change! -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-516) MergeContent when merging 10,000s or more objects can cause too many open files
[ https://issues.apache.org/jira/browse/NIFI-516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Percivall updated NIFI-516: -- Attachment: (was: NIFI-516.patch) > MergeContent when merging 10,000s or more objects can cause too many open > files > --- > > Key: NIFI-516 > URL: https://issues.apache.org/jira/browse/NIFI-516 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Joseph Witt >Assignee: Joseph Percivall > Fix For: 0.4.0 > > Attachments: generateMerge.xml > > > MergeContent really is just exposing an issue where the process session keeps > file handles open longer than necessary. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-447) Add "Replacement Strategy" property to ReplaceText
[ https://issues.apache.org/jira/browse/NIFI-447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Percivall updated NIFI-447: -- Attachment: (was: NIFI-447.patch) > Add "Replacement Strategy" property to ReplaceText > -- > > Key: NIFI-447 > URL: https://issues.apache.org/jira/browse/NIFI-447 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne > Fix For: 0.4.0 > > > "Regular Expression" property should have displayName changed to "Search > Value" > ReplaceText should allow for a "Replacement Strategy" with values: > Prepend (Search Value is ignored; replacement value prepended to FlowFile) > Append (Search Value is ignored; replacement value appended to FlowFIle) > Regex Replace (current behavior, so this should be default) > Linewise Regex Replace (like current behavior if Evaluation Mode is set to > Line-by-Line) > Literal Replace (do not interpret the Replacement value > The Line-by-Line property should be removed; this is breaking change! -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-447) Add "Replacement Strategy" property to ReplaceText
[ https://issues.apache.org/jira/browse/NIFI-447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Percivall updated NIFI-447: -- Attachment: NIFI-447.patch [~mpayne2] You're right. I changed the literal replace to use the search value as a character sequence which doesn't interpret it as a regular expression and added a proper unit tests for it. > Add "Replacement Strategy" property to ReplaceText > -- > > Key: NIFI-447 > URL: https://issues.apache.org/jira/browse/NIFI-447 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne > Fix For: 0.4.0 > > Attachments: NIFI-447.patch > > > "Regular Expression" property should have displayName changed to "Search > Value" > ReplaceText should allow for a "Replacement Strategy" with values: > Prepend (Search Value is ignored; replacement value prepended to FlowFile) > Append (Search Value is ignored; replacement value appended to FlowFIle) > Regex Replace (current behavior, so this should be default) > Linewise Regex Replace (like current behavior if Evaluation Mode is set to > Line-by-Line) > Literal Replace (do not interpret the Replacement value > The Line-by-Line property should be removed; this is breaking change! -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-828) Update default filter in new Processor Dialog
[ https://issues.apache.org/jira/browse/NIFI-828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-828: - Attachment: 0001-NIFI-828.patch > Update default filter in new Processor Dialog > - > > Key: NIFI-828 > URL: https://issues.apache.org/jira/browse/NIFI-828 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Trivial > Fix For: 0.4.0 > > Attachments: 0001-NIFI-828.patch > > > Update the default filter in the new processor dialog to use 'by tag' instead > of 'by name'. A user will likely want to search for items using terms tagged > on the processor rather than the formal processor name. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-828) Update default filter in new Processor Dialog
[ https://issues.apache.org/jira/browse/NIFI-828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-828: - Attachment: (was: 0001-NIFI-828.patch) > Update default filter in new Processor Dialog > - > > Key: NIFI-828 > URL: https://issues.apache.org/jira/browse/NIFI-828 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Trivial > Fix For: 0.4.0 > > Attachments: 0001-NIFI-828.patch > > > Update the default filter in the new processor dialog to use 'by tag' instead > of 'by name'. A user will likely want to search for items using terms tagged > on the processor rather than the formal processor name. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-828) Update default filter in new Processor Dialog
[ https://issues.apache.org/jira/browse/NIFI-828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947481#comment-14947481 ] Matt Gilman commented on NIFI-828: -- Removed previous patch that just swapped the order of the filter by and replaced with a patch that completely removed the filter by. > Update default filter in new Processor Dialog > - > > Key: NIFI-828 > URL: https://issues.apache.org/jira/browse/NIFI-828 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Trivial > Fix For: 0.4.0 > > Attachments: 0001-NIFI-828.patch > > > Update the default filter in the new processor dialog to use 'by tag' instead > of 'by name'. A user will likely want to search for items using terms tagged > on the processor rather than the formal processor name. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (NIFI-810) Create Annotation that indicates that a Processor cannot be scheduled to run without an incoming connection
[ https://issues.apache.org/jira/browse/NIFI-810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne reassigned NIFI-810: --- Assignee: Mark Payne > Create Annotation that indicates that a Processor cannot be scheduled to run > without an incoming connection > --- > > Key: NIFI-810 > URL: https://issues.apache.org/jira/browse/NIFI-810 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne > > Currently, if a Processor has no incoming connections but is started, it will > continually without ever accomplishing anything. We should have an > annotation, perhaps @RequiresInput, that indicates that the Processor should > not be scheduled to run unless it has an incoming connection. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-810) Create Annotation that indicates that a Processor cannot be scheduled to run without an incoming connection
[ https://issues.apache.org/jira/browse/NIFI-810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947667#comment-14947667 ] ASF subversion and git services commented on NIFI-810: -- Commit 5ecdb1858e94938cd426b12bc48d3725109c6e96 in nifi's branch refs/heads/NIFI-810-InputRequirement from [~markap14] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=5ecdb18 ] NIFI-810: Updated the wording on validation errors due to upstream connections > Create Annotation that indicates that a Processor cannot be scheduled to run > without an incoming connection > --- > > Key: NIFI-810 > URL: https://issues.apache.org/jira/browse/NIFI-810 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne > > Currently, if a Processor has no incoming connections but is started, it will > continually without ever accomplishing anything. We should have an > annotation, perhaps @RequiresInput, that indicates that the Processor should > not be scheduled to run unless it has an incoming connection. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[04/17] nifi git commit: Previous commits for NIFI-992 provide the functionality as specified in Github PR.
Previous commits for NIFI-992 provide the functionality as specified in Github PR. This closes #96. Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/88cb Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/88cb Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/88cb Branch: refs/heads/NIFI-810-InputRequirement Commit: 88cb00083fc5b0ee169122d1a4b41ee9d524 Parents: 033a155 Author: Bryan Bende Authored: Thu Oct 1 15:12:22 2015 -0400 Committer: Bryan Bende Committed: Thu Oct 1 15:12:22 2015 -0400 -- --
[03/17] nifi git commit: nifi-992 Improvements based on code review part II.
nifi-992 Improvements based on code review part II.
- Penalize or Yield based on ErrorHandlingStrategy.Penalty
- Add Retry relationship to PutCouchbaseKey
- Remove unnecessary try/catch and let the framework handle it
- Change CouchbaseException relation mapping for Fatal from Failure to Retry
Signed-off-by: Bryan Bende
Project: http://git-wip-us.apache.org/repos/asf/nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/033a1553
Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/033a1553
Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/033a1553
Branch: refs/heads/NIFI-810-InputRequirement
Commit: 033a1553abe7c83d4d9c49b753911355a881dcc5
Parents: 72eb64e
Author: ijokarumawak
Authored: Thu Oct 1 15:05:29 2015 +0900
Committer: Bryan Bende
Committed: Thu Oct 1 14:13:20 2015 -0400
--
.../couchbase/AbstractCouchbaseProcessor.java | 16 ++-
.../couchbase/ErrorHandlingStrategy.java| 28 +---
.../processors/couchbase/GetCouchbaseKey.java | 28 +---
.../processors/couchbase/PutCouchbaseKey.java | 14 --
.../couchbase/TestGetCouchbaseKey.java | 19 ++---
.../couchbase/TestPutCouchbaseKey.java | 13 ++---
6 files changed, 76 insertions(+), 42 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/nifi/blob/033a1553/nifi-nar-bundles/nifi-couchbase-bundle/nifi-couchbase-processors/src/main/java/org/apache/nifi/processors/couchbase/AbstractCouchbaseProcessor.java
--
diff --git
a/nifi-nar-bundles/nifi-couchbase-bundle/nifi-couchbase-processors/src/main/java/org/apache/nifi/processors/couchbase/AbstractCouchbaseProcessor.java
b/nifi-nar-bundles/nifi-couchbase-bundle/nifi-couchbase-processors/src/main/java/org/apache/nifi/processors/couchbase/AbstractCouchbaseProcessor.java
index 066b1ca..b879041 100644
---
a/nifi-nar-bundles/nifi-couchbase-bundle/nifi-couchbase-processors/src/main/java/org/apache/nifi/processors/couchbase/AbstractCouchbaseProcessor.java
+++
b/nifi-nar-bundles/nifi-couchbase-bundle/nifi-couchbase-processors/src/main/java/org/apache/nifi/processors/couchbase/AbstractCouchbaseProcessor.java
@@ -175,18 +175,32 @@ public abstract class AbstractCouchbaseProcessor extends
AbstractProcessor {
/**
* Handles the thrown CocuhbaseException accordingly.
+ * @param context a process context
* @param session a process session
* @param logger a logger
* @param inFile an input FlowFile
* @param e the thrown CouchbaseException
* @param errMsg a message to be logged
*/
-protected void handleCouchbaseException(final ProcessSession session,
+protected void handleCouchbaseException(final ProcessContext context,
final ProcessSession session,
final ProcessorLog logger, FlowFile inFile, CouchbaseException e,
String errMsg) {
logger.error(errMsg, e);
if(inFile != null){
ErrorHandlingStrategy strategy =
CouchbaseExceptionMappings.getStrategy(e);
+switch(strategy.penalty()) {
+case Penalize:
+if(logger.isDebugEnabled()) logger.debug("Penalized: {}", new
Object[]{inFile});
+inFile = session.penalize(inFile);
+break;
+case Yield:
+if(logger.isDebugEnabled()) logger.debug("Yielded context:
{}", new Object[]{inFile});
+context.yield();
+break;
+case None:
+break;
+}
+
switch(strategy.result()) {
case ProcessException:
throw new ProcessException(errMsg, e);
http://git-wip-us.apache.org/repos/asf/nifi/blob/033a1553/nifi-nar-bundles/nifi-couchbase-bundle/nifi-couchbase-processors/src/main/java/org/apache/nifi/processors/couchbase/ErrorHandlingStrategy.java
--
diff --git
a/nifi-nar-bundles/nifi-couchbase-bundle/nifi-couchbase-processors/src/main/java/org/apache/nifi/processors/couchbase/ErrorHandlingStrategy.java
b/nifi-nar-bundles/nifi-couchbase-bundle/nifi-couchbase-processors/src/main/java/org/apache/nifi/processors/couchbase/ErrorHandlingStrategy.java
index 75b8f46..bae35d5 100644
---
a/nifi-nar-bundles/nifi-couchbase-bundle/nifi-couchbase-processors/src/main/java/org/apache/nifi/processors/couchbase/ErrorHandlingStrategy.java
+++
b/nifi-nar-bundles/nifi-couchbase-bundle/nifi-couchbase-processors/src/main/java/org/apache/nifi/processors/couchbase/ErrorHandlingStrategy.java
@@ -16,6 +16,7 @@
*/
package org.apache.nifi.processors.couchbase;
+import static
org.apache.nifi.processors.couchbase.ErrorHandlingStrategy.Penalty.None;
import static
org.apache.nifi.processors.couchbase.ErrorHandlingStrate
[13/17] nifi git commit: NIFI-810: - Adding basic support for preventing connection when appropriate. - Updating validation when [dis]connecting processors.
NIFI-810:
- Adding basic support for preventing connection when appropriate.
- Updating validation when [dis]connecting processors.
Project: http://git-wip-us.apache.org/repos/asf/nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/13edcfda
Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/13edcfda
Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/13edcfda
Branch: refs/heads/NIFI-810-InputRequirement
Commit: 13edcfda2ef830e1b160b31f7cd3bea874ccd3f0
Parents: 4afd8f8
Author: Matt Gilman
Authored: Fri Sep 25 17:46:58 2015 -0400
Committer: Mark Payne
Committed: Wed Oct 7 17:26:14 2015 -0400
--
.../org/apache/nifi/web/api/dto/ProcessorDTO.java| 15 +++
.../org/apache/nifi/controller/TemplateManager.java | 1 +
.../java/org/apache/nifi/web/api/dto/DtoFactory.java | 1 +
.../src/main/webapp/js/nf/canvas/nf-actions.js | 4 +++-
.../src/main/webapp/js/nf/canvas/nf-canvas-utils.js | 13 ++---
.../js/nf/canvas/nf-connection-configuration.js | 8 ++--
6 files changed, 36 insertions(+), 6 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/nifi/blob/13edcfda/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-client-dto/src/main/java/org/apache/nifi/web/api/dto/ProcessorDTO.java
--
diff --git
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-client-dto/src/main/java/org/apache/nifi/web/api/dto/ProcessorDTO.java
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-client-dto/src/main/java/org/apache/nifi/web/api/dto/ProcessorDTO.java
index c65c46a..866d77c 100644
---
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-client-dto/src/main/java/org/apache/nifi/web/api/dto/ProcessorDTO.java
+++
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-client-dto/src/main/java/org/apache/nifi/web/api/dto/ProcessorDTO.java
@@ -36,6 +36,7 @@ public class ProcessorDTO extends NiFiComponentDTO {
private String description;
private Boolean supportsParallelProcessing;
private Boolean supportsEventDriven;
+private String inputRequirement;
private ProcessorConfigDTO config;
@@ -121,6 +122,20 @@ public class ProcessorDTO extends NiFiComponentDTO {
}
/**
+ * @return the input requirement of this processor
+ */
+@ApiModelProperty(
+value = "The input requirement for this processor."
+)
+public String getInputRequirement() {
+return inputRequirement;
+}
+
+public void setInputRequirement(String inputRequirement) {
+this.inputRequirement = inputRequirement;
+}
+
+/**
* @return whether this processor supports event driven scheduling
*/
@ApiModelProperty(
http://git-wip-us.apache.org/repos/asf/nifi/blob/13edcfda/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/TemplateManager.java
--
diff --git
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/TemplateManager.java
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/TemplateManager.java
index 7b8e173..a332e05 100644
---
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/TemplateManager.java
+++
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/TemplateManager.java
@@ -320,6 +320,7 @@ public class TemplateManager {
// remove validation errors
processorDTO.setValidationErrors(null);
+processorDTO.setInputRequirement(null);
}
}
http://git-wip-us.apache.org/repos/asf/nifi/blob/13edcfda/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/dto/DtoFactory.java
--
diff --git
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/dto/DtoFactory.java
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/dto/DtoFactory.java
index 76bce6f..16b114e 100644
---
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/dto/DtoFactory.java
+++
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/dto/DtoFactory.java
@@ -1402,6 +1402,7 @@ public final class DtoFactory {
dto.setPosition(createPositionDto(node.getPosition()));
[06/17] nifi git commit: Bumped nifi-kafka-processors Kafka version
Bumped nifi-kafka-processors Kafka version This closes #99. Signed-off-by: Aldrin Piri Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/31fba6b3 Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/31fba6b3 Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/31fba6b3 Branch: refs/heads/NIFI-810-InputRequirement Commit: 31fba6b3332978ca2f6a1d693f6053d719fb9daa Parents: 6b2f5ad Author: Randy Gelhausen Authored: Fri Oct 2 18:09:02 2015 -0400 Committer: Aldrin Piri Committed: Mon Oct 5 10:39:22 2015 -0400 -- nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-processors/pom.xml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/31fba6b3/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-processors/pom.xml -- diff --git a/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-processors/pom.xml b/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-processors/pom.xml index e4db495..ff10c09 100644 --- a/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-processors/pom.xml +++ b/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-processors/pom.xml @@ -36,8 +36,8 @@ org.apache.kafka -kafka_2.8.2 -0.8.1 +kafka_2.9.1 +0.8.2.2
[10/17] nifi git commit: NIFI-810: Created RequiresInput annotation and ensure that processors are invalid if connections do not agree
http://git-wip-us.apache.org/repos/asf/nifi/blob/4afd8f88/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/StandardProcessorNode.java
--
diff --git
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/StandardProcessorNode.java
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/StandardProcessorNode.java
index cbd0f88..0c39eda 100644
---
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/StandardProcessorNode.java
+++
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/StandardProcessorNode.java
@@ -16,14 +16,6 @@
*/
package org.apache.nifi.controller;
-import org.apache.nifi.annotation.behavior.EventDriven;
-import org.apache.nifi.annotation.behavior.SideEffectFree;
-import org.apache.nifi.annotation.behavior.SupportsBatching;
-import org.apache.nifi.annotation.behavior.TriggerSerially;
-import org.apache.nifi.annotation.behavior.TriggerWhenAnyDestinationAvailable;
-import org.apache.nifi.annotation.behavior.TriggerWhenEmpty;
-import org.apache.nifi.annotation.documentation.CapabilityDescription;
-
import static java.util.Objects.requireNonNull;
import java.util.ArrayList;
@@ -43,6 +35,17 @@ import java.util.concurrent.atomic.AtomicReference;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantReadWriteLock;
+import org.apache.commons.lang3.builder.EqualsBuilder;
+import org.apache.commons.lang3.builder.HashCodeBuilder;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.TriggerSerially;
+import org.apache.nifi.annotation.behavior.TriggerWhenAnyDestinationAvailable;
+import org.apache.nifi.annotation.behavior.TriggerWhenEmpty;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.components.ValidationContext;
import org.apache.nifi.components.ValidationResult;
import org.apache.nifi.connectable.Connectable;
@@ -61,8 +64,6 @@ import org.apache.nifi.processor.Processor;
import org.apache.nifi.processor.Relationship;
import org.apache.nifi.scheduling.SchedulingStrategy;
import org.apache.nifi.util.FormatUtils;
-import org.apache.commons.lang3.builder.EqualsBuilder;
-import org.apache.commons.lang3.builder.HashCodeBuilder;
import org.quartz.CronExpression;
import org.slf4j.LoggerFactory;
@@ -73,1185 +74,1242 @@ import org.slf4j.LoggerFactory;
*/
public class StandardProcessorNode extends ProcessorNode implements
Connectable {
-public static final String BULLETIN_OBSERVER_ID = "bulletin-observer";
-
-public static final TimeUnit DEFAULT_TIME_UNIT = TimeUnit.MILLISECONDS;
-public static final String DEFAULT_YIELD_PERIOD = "1 sec";
-public static final String DEFAULT_PENALIZATION_PERIOD = "30 sec";
-private final AtomicReference processGroup;
-private final Processor processor;
-private final AtomicReference identifier;
-private final Map destinations;
-private final Map> connections;
-private final AtomicReference>
undefinedRelationshipsToTerminate;
-private final AtomicReference> incomingConnectionsRef;
-private final ReentrantReadWriteLock rwLock;
-private final Lock readLock;
-private final Lock writeLock;
-private final AtomicBoolean isolated;
-private final AtomicBoolean lossTolerant;
-private final AtomicReference scheduledState;
-private final AtomicReference comments;
-private final AtomicReference name;
-private final AtomicReference position;
-private final AtomicReference annotationData;
-private final AtomicReference schedulingPeriod; // stored as
string so it's presented to user as they entered it
-private final AtomicReference yieldPeriod;
-private final AtomicReference penalizationPeriod;
-private final AtomicReference> style;
-private final AtomicInteger concurrentTaskCount;
-private final AtomicLong yieldExpiration;
-private final AtomicLong schedulingNanos;
-private final boolean triggerWhenEmpty;
-private final boolean sideEffectFree;
-private final boolean triggeredSerially;
-private final boolean triggerWhenAnyDestinationAvailable;
-private final boolean eventDrivenSupported;
-private final boolean batchSupported;
-private final ValidationContextFactory validationContextFactory;
-private final ProcessScheduler processScheduler;
-private long runNanos
[14/17] nifi git commit: NIFI-810: rebased from master
http://git-wip-us.apache.org/repos/asf/nifi/blob/b974445d/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/StandardProcessorNode.java
--
diff --cc
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/StandardProcessorNode.java
index 3c816d0,0c39eda..f69c510
---
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/StandardProcessorNode.java
+++
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/StandardProcessorNode.java
@@@ -74,1241 -74,1242 +74,1241 @@@ import org.slf4j.LoggerFactory
*/
public class StandardProcessorNode extends ProcessorNode implements
Connectable {
- public static final String BULLETIN_OBSERVER_ID = "bulletin-observer";
-
- public static final TimeUnit DEFAULT_TIME_UNIT = TimeUnit.MILLISECONDS;
- public static final String DEFAULT_YIELD_PERIOD = "1 sec";
- public static final String DEFAULT_PENALIZATION_PERIOD = "30 sec";
- private final AtomicReference processGroup;
- private final Processor processor;
- private final AtomicReference identifier;
- private final Map destinations;
- private final Map> connections;
- private final AtomicReference>
undefinedRelationshipsToTerminate;
- private final AtomicReference> incomingConnectionsRef;
- private final ReentrantReadWriteLock rwLock;
- private final Lock readLock;
- private final Lock writeLock;
- private final AtomicBoolean isolated;
- private final AtomicBoolean lossTolerant;
- private final AtomicReference scheduledState;
- private final AtomicReference comments;
- private final AtomicReference name;
- private final AtomicReference position;
- private final AtomicReference annotationData;
- private final AtomicReference schedulingPeriod; // stored as
string so it's presented to user as they entered it
- private final AtomicReference yieldPeriod;
- private final AtomicReference penalizationPeriod;
- private final AtomicReference> style;
- private final AtomicInteger concurrentTaskCount;
- private final AtomicLong yieldExpiration;
- private final AtomicLong schedulingNanos;
- private final boolean triggerWhenEmpty;
- private final boolean sideEffectFree;
- private final boolean triggeredSerially;
- private final boolean triggerWhenAnyDestinationAvailable;
- private final boolean eventDrivenSupported;
- private final boolean batchSupported;
- private final Requirement inputRequirement;
- private final ValidationContextFactory validationContextFactory;
- private final ProcessScheduler processScheduler;
- private long runNanos = 0L;
-
- private SchedulingStrategy schedulingStrategy; // guarded by
read/write lock
-
- @SuppressWarnings("deprecation")
- public StandardProcessorNode(final Processor processor, final String
uuid, final ValidationContextFactory validationContextFactory,
- final ProcessScheduler scheduler, final
ControllerServiceProvider controllerServiceProvider) {
- super(processor, uuid, validationContextFactory,
controllerServiceProvider);
-
- this.processor = processor;
- identifier = new AtomicReference<>(uuid);
- destinations = new HashMap<>();
- connections = new HashMap<>();
- incomingConnectionsRef = new
AtomicReference>(new ArrayList());
- scheduledState = new AtomicReference<>(ScheduledState.STOPPED);
- rwLock = new ReentrantReadWriteLock(false);
- readLock = rwLock.readLock();
- writeLock = rwLock.writeLock();
- lossTolerant = new AtomicBoolean(false);
- final Set emptySetOfRelationships = new
HashSet<>();
- undefinedRelationshipsToTerminate = new
AtomicReference<>(emptySetOfRelationships);
- comments = new AtomicReference<>("");
- name = new
AtomicReference<>(processor.getClass().getSimpleName());
- schedulingPeriod = new AtomicReference<>("0 sec");
- schedulingNanos = new AtomicLong(MINIMUM_SCHEDULING_NANOS);
- yieldPeriod = new AtomicReference<>(DEFAULT_YIELD_PERIOD);
- yieldExpiration = new AtomicLong(0L);
- concurrentTaskCount = new AtomicInteger(1);
- position = new AtomicReference<>(new Position(0D, 0D));
- style = new AtomicReference<>(Collections.unmodifiableMap(new
HashMap()));
- this.processGroup = new AtomicReference<>();
- processScheduler = scheduler;
- annota
[jira] [Commented] (NIFI-810) Create Annotation that indicates that a Processor cannot be scheduled to run without an incoming connection
[ https://issues.apache.org/jira/browse/NIFI-810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947669#comment-14947669 ] ASF subversion and git services commented on NIFI-810: -- Commit ccfb57fe9ff43f11319dcb1625bfc78b1d88f56a in nifi's branch refs/heads/NIFI-810-InputRequirement from [~markap14] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=ccfb57f ] NIFI-810: Addressed several checkstyle violations > Create Annotation that indicates that a Processor cannot be scheduled to run > without an incoming connection > --- > > Key: NIFI-810 > URL: https://issues.apache.org/jira/browse/NIFI-810 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne > > Currently, if a Processor has no incoming connections but is started, it will > continually without ever accomplishing anything. We should have an > annotation, perhaps @RequiresInput, that indicates that the Processor should > not be scheduled to run unless it has an incoming connection. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[09/17] nifi git commit: NIFI-810: Created RequiresInput annotation and ensure that processors are invalid if connections do not agree
http://git-wip-us.apache.org/repos/asf/nifi/blob/4afd8f88/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/CreateHadoopSequenceFile.java
--
diff --git
a/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/CreateHadoopSequenceFile.java
b/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/CreateHadoopSequenceFile.java
index cbcc54d..385ac73 100644
---
a/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/CreateHadoopSequenceFile.java
+++
b/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/CreateHadoopSequenceFile.java
@@ -23,7 +23,8 @@ import java.util.List;
import java.util.Set;
import org.apache.hadoop.io.SequenceFile;
-import org.apache.hadoop.io.SequenceFile.CompressionType;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
import org.apache.nifi.annotation.behavior.SideEffectFree;
import org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.annotation.documentation.SeeAlso;
@@ -57,6 +58,7 @@ import
org.apache.nifi.processors.hadoop.util.SequenceFileWriter;
*
*/
@SideEffectFree
+@InputRequirement(Requirement.INPUT_REQUIRED)
@Tags({"hadoop", "sequence file", "create", "sequencefile"})
@CapabilityDescription("Creates Hadoop Sequence Files from incoming flow
files")
@SeeAlso(PutHDFS.class)
http://git-wip-us.apache.org/repos/asf/nifi/blob/4afd8f88/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/FetchHDFS.java
--
diff --git
a/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/FetchHDFS.java
b/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/FetchHDFS.java
index 4a52fb7..aa03e73 100644
---
a/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/FetchHDFS.java
+++
b/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/FetchHDFS.java
@@ -29,6 +29,8 @@ import org.apache.hadoop.fs.FSDataInputStream;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.security.AccessControlException;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
import org.apache.nifi.annotation.behavior.SupportsBatching;
import org.apache.nifi.annotation.behavior.WritesAttribute;
import org.apache.nifi.annotation.documentation.CapabilityDescription;
@@ -44,6 +46,7 @@ import org.apache.nifi.processor.util.StandardValidators;
import org.apache.nifi.util.StopWatch;
@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
@Tags({"hadoop", "hdfs", "get", "ingest", "fetch", "source"})
@CapabilityDescription("Retrieves a file from HDFS. The content of the
incoming FlowFile is replaced by the content of the file in HDFS. "
+ "The file in HDFS is left intact without any changes being made to
it.")
http://git-wip-us.apache.org/repos/asf/nifi/blob/4afd8f88/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/GetHDFS.java
--
diff --git
a/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/GetHDFS.java
b/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/GetHDFS.java
index de776d4..4c9deea 100644
---
a/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/GetHDFS.java
+++
b/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/GetHDFS.java
@@ -41,6 +41,8 @@ import org.apache.hadoop.fs.Path;
import org.apache.hadoop.fs.PathFilter;
import org.apache.hadoop.io.compress.CompressionCodec;
import org.apache.hadoop.io.compress.CompressionCodecFactory;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
import org.apache.nifi.annotation.behavior.TriggerWhenEmpty;
import org.apache.nifi.annotation.behavior.WritesAttribute;
import org.apache.nifi.annotation.behavior.WritesAttributes;
@@ -62,6 +64,7 @@ import org.apache.nifi.processor.util.StandardValidators;
import org.apache.nifi.util.StopWatch;
@TriggerWhenEmpty
+@InputRequirement(Requirement.INPUT_ALLOWED)
@Tags({"hadoop",
[15/17] nifi git commit: NIFI-810: rebased from master
NIFI-810: rebased from master Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/b974445d Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/b974445d Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/b974445d Branch: refs/heads/NIFI-810-InputRequirement Commit: b974445ddd38ec7e84995225b86987e6af1af52c Parents: 5ecdb18 2215bc8 Author: Mark Payne Authored: Wed Oct 7 17:28:39 2015 -0400 Committer: Mark Payne Committed: Wed Oct 7 17:28:39 2015 -0400 -- .../nifi/controller/StandardProcessorNode.java| 18 +- 1 file changed, 9 insertions(+), 9 deletions(-) --
[11/17] nifi git commit: NIFI-810: Created RequiresInput annotation and ensure that processors are invalid if connections do not agree
NIFI-810: Created RequiresInput annotation and ensure that processors are invalid if connections do not agree Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/4afd8f88 Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/4afd8f88 Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/4afd8f88 Branch: refs/heads/NIFI-810-InputRequirement Commit: 4afd8f88f8a34cf87f2a06221667166a54c99a15 Parents: 31fba6b Author: Mark Payne Authored: Fri Sep 25 11:39:28 2015 -0400 Committer: Mark Payne Committed: Wed Oct 7 17:26:14 2015 -0400 -- .../annotation/behavior/InputRequirement.java | 51 + .../src/main/asciidoc/developer-guide.adoc | 11 + .../nifi/processors/avro/ConvertAvroToJSON.java |3 + .../processors/avro/ExtractAvroMetadata.java| 29 +- .../apache/nifi/processors/avro/SplitAvro.java | 27 +- .../nifi/processors/aws/s3/FetchS3Object.java |3 + .../nifi/processors/aws/s3/PutS3Object.java |6 +- .../apache/nifi/processors/aws/sns/PutSNS.java |3 + .../nifi/processors/aws/sqs/DeleteSQS.java |3 + .../apache/nifi/processors/aws/sqs/GetSQS.java |5 +- .../apache/nifi/processors/aws/sqs/PutSQS.java |5 +- .../nifi/processors/flume/ExecuteFlumeSink.java | 14 +- .../processors/flume/ExecuteFlumeSource.java| 14 +- .../apache/nifi/controller/ProcessorNode.java | 89 +- .../nifi/controller/StandardProcessorNode.java | 2440 +- .../org/apache/nifi/processors/GeoEnrichIP.java |3 + .../hadoop/CreateHadoopSequenceFile.java|4 +- .../nifi/processors/hadoop/FetchHDFS.java |3 + .../apache/nifi/processors/hadoop/GetHDFS.java |3 + .../apache/nifi/processors/hadoop/ListHDFS.java |3 + .../apache/nifi/processors/hadoop/PutHDFS.java |3 + .../processors/hl7/ExtractHL7Attributes.java|3 + .../apache/nifi/processors/hl7/RouteHL7.java|3 + .../processors/image/ExtractImageMetadata.java | 36 +- .../nifi/processors/image/ResizeImage.java | 38 +- .../apache/nifi/processors/kafka/GetKafka.java | 21 +- .../apache/nifi/processors/kafka/PutKafka.java | 10 +- .../nifi/processors/kite/ConvertCSVToAvro.java | 16 +- .../nifi/processors/kite/ConvertJSONToAvro.java | 14 +- .../processors/kite/StoreInKiteDataset.java |9 +- .../nifi/processors/yandex/YandexTranslate.java |3 + .../nifi-pcap-processors/.gitignore |1 + .../nifi/processors/twitter/GetTwitter.java |5 +- .../apache/nifi/processors/solr/GetSolr.java| 43 +- .../processors/solr/PutSolrContentStream.java | 33 +- .../standard/Base64EncodeContent.java | 171 +- .../processors/standard/CompressContent.java| 15 +- .../nifi/processors/standard/ControlRate.java | 683 ++--- .../standard/ConvertCharacterSet.java |3 + .../processors/standard/ConvertJSONToSQL.java |3 + .../processors/standard/DetectDuplicate.java|3 + .../processors/standard/DistributeLoad.java |3 + .../processors/standard/DuplicateFlowFile.java |3 + .../nifi/processors/standard/EncodeContent.java | 15 +- .../processors/standard/EncryptContent.java |3 + .../processors/standard/EvaluateJsonPath.java | 38 +- .../nifi/processors/standard/EvaluateXPath.java | 29 +- .../processors/standard/EvaluateXQuery.java | 25 +- .../processors/standard/ExecuteProcess.java |3 + .../nifi/processors/standard/ExecuteSQL.java|3 + .../standard/ExecuteStreamCommand.java |7 +- .../nifi/processors/standard/ExtractText.java |3 + .../processors/standard/GenerateFlowFile.java | 11 +- .../apache/nifi/processors/standard/GetFTP.java | 13 +- .../nifi/processors/standard/GetFile.java |7 +- .../nifi/processors/standard/GetHTTP.java |3 + .../nifi/processors/standard/GetJMSQueue.java |3 + .../nifi/processors/standard/GetJMSTopic.java |3 + .../nifi/processors/standard/GetSFTP.java |7 +- .../processors/standard/HandleHttpRequest.java |7 +- .../processors/standard/HandleHttpResponse.java |5 +- .../nifi/processors/standard/HashAttribute.java |5 +- .../nifi/processors/standard/HashContent.java |5 +- .../processors/standard/IdentifyMimeType.java |5 +- .../nifi/processors/standard/InvokeHTTP.java|3 + .../nifi/processors/standard/ListenHTTP.java| 16 +- .../nifi/processors/standard/ListenUDP.java | 18 +- .../nifi/processors/standard/LogAttribute.java | 16 +- .../nifi/processors/standard/MergeContent.java | 11 +- .../nifi/processors/standard/ModifyBytes.java | 14 +- .../processors/standard/MonitorActivity.java| 31 +- .../nifi/processors/standard/PostHTTP.java |3 + .../nifi/processors/standard/PutEmail.java |3 + .../apache/nifi/processors
[02/17] nifi git commit: nifi-992 Improvements based on code review.
nifi-992 Improvements based on code review. - Removed checkstyle and contrib-check profile since it's inherit from top-level pom. - Consolidate DOC_ID and DOC_ID_EXP into a single DOC_ID property. - Add capability description on GetCouchbaseKey. - Fixed documentation spell misses. - Handle Exceptions accordingly. - Add 'retry' relationship. Signed-off-by: Bryan Bende Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/72eb64e8 Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/72eb64e8 Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/72eb64e8 Branch: refs/heads/NIFI-810-InputRequirement Commit: 72eb64e8a43a08a5be988a3826b0a116c57915ea Parents: 2466a24 Author: ijokarumawak Authored: Wed Sep 30 00:58:39 2015 +0900 Committer: Bryan Bende Committed: Tue Sep 29 13:48:13 2015 -0400 -- .../nifi-couchbase-processors/pom.xml | 144 -- .../nifi/couchbase/CouchbaseAttributes.java | 4 + .../couchbase/AbstractCouchbaseProcessor.java | 94 --- .../couchbase/CouchbaseExceptionMappings.java | 128 + .../couchbase/ErrorHandlingStrategy.java| 59 .../processors/couchbase/GetCouchbaseKey.java | 45 ++- .../processors/couchbase/PutCouchbaseKey.java | 45 +-- .../couchbase/TestCouchbaseClusterService.java | 2 +- .../couchbase/TestGetCouchbaseKey.java | 282 +-- .../couchbase/TestPutCouchbaseKey.java | 95 +-- 10 files changed, 626 insertions(+), 272 deletions(-) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/72eb64e8/nifi-nar-bundles/nifi-couchbase-bundle/nifi-couchbase-processors/pom.xml -- diff --git a/nifi-nar-bundles/nifi-couchbase-bundle/nifi-couchbase-processors/pom.xml b/nifi-nar-bundles/nifi-couchbase-bundle/nifi-couchbase-processors/pom.xml index 33b0baa..257ef46 100644 --- a/nifi-nar-bundles/nifi-couchbase-bundle/nifi-couchbase-processors/pom.xml +++ b/nifi-nar-bundles/nifi-couchbase-bundle/nifi-couchbase-processors/pom.xml @@ -61,148 +61,4 @@ test - - - - - org.apache.maven.plugins - maven-checkstyle-plugin - 2.15 - - - com.puppycrawl.tools - checkstyle - 6.5 - - - - - - - -org.apache.maven.plugins -maven-checkstyle-plugin - - - - - - - - - - - - - - - - - - - - - - - -ftp://"; /> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
[16/17] nifi git commit: NIFI-810: Addressed several checkstyle violations
NIFI-810: Addressed several checkstyle violations
Project: http://git-wip-us.apache.org/repos/asf/nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/ccfb57fe
Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/ccfb57fe
Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/ccfb57fe
Branch: refs/heads/NIFI-810-InputRequirement
Commit: ccfb57fe9ff43f11319dcb1625bfc78b1d88f56a
Parents: b974445
Author: Mark Payne
Authored: Wed Oct 7 17:48:51 2015 -0400
Committer: Mark Payne
Committed: Wed Oct 7 17:48:51 2015 -0400
--
.../annotation/behavior/InputRequirement.java | 70 +-
.../nifi/processors/aws/s3/PutS3Object.java | 46 +-
.../apache/nifi/controller/ProcessorNode.java | 88 +--
.../nifi/controller/StandardProcessorNode.java | 10 +-
.../standard/Base64EncodeContent.java | 168 ++---
.../nifi/processors/standard/ControlRate.java | 672 +--
6 files changed, 534 insertions(+), 520 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/nifi/blob/ccfb57fe/nifi-api/src/main/java/org/apache/nifi/annotation/behavior/InputRequirement.java
--
diff --git
a/nifi-api/src/main/java/org/apache/nifi/annotation/behavior/InputRequirement.java
b/nifi-api/src/main/java/org/apache/nifi/annotation/behavior/InputRequirement.java
index 97e6b88..13f442c 100644
---
a/nifi-api/src/main/java/org/apache/nifi/annotation/behavior/InputRequirement.java
+++
b/nifi-api/src/main/java/org/apache/nifi/annotation/behavior/InputRequirement.java
@@ -1,3 +1,19 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
package org.apache.nifi.annotation.behavior;
import java.lang.annotation.Documented;
@@ -21,31 +37,31 @@ import java.lang.annotation.Target;
@Retention(RetentionPolicy.RUNTIME)
@Inherited
public @interface InputRequirement {
- Requirement value();
-
- public static enum Requirement {
- /**
-* This value is used to indicate that the Processor requires
input from other Processors
-* in order to run. As a result, the Processor will not be
valid if it does not have any
-* incoming connections.
-*/
- INPUT_REQUIRED,
-
- /**
-* This value is used to indicate that the Processor will
consume data from an incoming
-* connection but does not require an incoming connection in
order to perform its task.
-* If the {@link InputRequirement} annotation is not present,
this is the default value
-* that is used.
-*/
- INPUT_ALLOWED,
-
- /**
-* This value is used to indicate that the Processor is a
"Source Processor" and does
-* not accept incoming connections. Because the Processor does
not pull FlowFiles from
-* an incoming connection, it can be very confusing for users
who create incoming connections
-* to the Processor. As a result, this value can be used in
order to clarify that incoming
-* connections will not be used. This prevents the user from
even creating such a connection.
-*/
- INPUT_FORBIDDEN;
- }
+Requirement value();
+
+public static enum Requirement {
+/**
+ * This value is used to indicate that the Processor requires input
from other Processors
+ * in order to run. As a result, the Processor will not be valid if it
does not have any
+ * incoming connections.
+ */
+INPUT_REQUIRED,
+
+/**
+ * This value is used to indicate that the Processor will consume data
from an incoming
+ * connection but does not require an incoming connection in order to
perform its task.
+ * If the {@link InputRequirement} annotation is not present, this is
the default value
+ * that is used.
+ */
+INPUT_ALLOWED,
+
+/**
+ *
[jira] [Commented] (NIFI-810) Create Annotation that indicates that a Processor cannot be scheduled to run without an incoming connection
[ https://issues.apache.org/jira/browse/NIFI-810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947668#comment-14947668 ] ASF subversion and git services commented on NIFI-810: -- Commit b974445ddd38ec7e84995225b86987e6af1af52c in nifi's branch refs/heads/NIFI-810-InputRequirement from [~markap14] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=b974445 ] NIFI-810: rebased from master > Create Annotation that indicates that a Processor cannot be scheduled to run > without an incoming connection > --- > > Key: NIFI-810 > URL: https://issues.apache.org/jira/browse/NIFI-810 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne > > Currently, if a Processor has no incoming connections but is started, it will > continually without ever accomplishing anything. We should have an > annotation, perhaps @RequiresInput, that indicates that the Processor should > not be scheduled to run unless it has an incoming connection. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-810) Create Annotation that indicates that a Processor cannot be scheduled to run without an incoming connection
[ https://issues.apache.org/jira/browse/NIFI-810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947670#comment-14947670 ] ASF subversion and git services commented on NIFI-810: -- Commit 8e2308b78de480dd7848ffe8efb485a5ee61c42a in nifi's branch refs/heads/NIFI-810-InputRequirement from [~markap14] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=8e2308b ] NIFI-810: Reworded validation errors pertaining ot upstream connections > Create Annotation that indicates that a Processor cannot be scheduled to run > without an incoming connection > --- > > Key: NIFI-810 > URL: https://issues.apache.org/jira/browse/NIFI-810 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne > > Currently, if a Processor has no incoming connections but is started, it will > continually without ever accomplishing anything. We should have an > annotation, perhaps @RequiresInput, that indicates that the Processor should > not be scheduled to run unless it has an incoming connection. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[17/17] nifi git commit: NIFI-810: Reworded validation errors pertaining ot upstream connections
NIFI-810: Reworded validation errors pertaining ot upstream connections
Project: http://git-wip-us.apache.org/repos/asf/nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/8e2308b7
Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/8e2308b7
Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/8e2308b7
Branch: refs/heads/NIFI-810-InputRequirement
Commit: 8e2308b78de480dd7848ffe8efb485a5ee61c42a
Parents: ccfb57f
Author: Mark Payne
Authored: Wed Oct 7 17:53:04 2015 -0400
Committer: Mark Payne
Committed: Wed Oct 7 17:53:04 2015 -0400
--
.../org/apache/nifi/controller/StandardProcessorNode.java| 8
1 file changed, 4 insertions(+), 4 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/nifi/blob/8e2308b7/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/StandardProcessorNode.java
--
diff --git
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/StandardProcessorNode.java
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/StandardProcessorNode.java
index ad22c6d..2b0d413 100644
---
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/StandardProcessorNode.java
+++
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/StandardProcessorNode.java
@@ -1048,8 +1048,8 @@ public class StandardProcessorNode extends ProcessorNode
implements Connectable
final int incomingConnCount =
getIncomingConnections().size();
if (incomingConnCount != 0) {
results.add(new ValidationResult.Builder()
-.explanation("Processor does not accept Incoming
Connections but is currently configured with " + incomingConnCount + " Incoming
Connections")
-.subject("Incoming Connections")
+.explanation("Processor does not allow upstream
connections but currently has " + incomingConnCount)
+.subject("Upstream Connections")
.valid(false)
.build());
}
@@ -1058,8 +1058,8 @@ public class StandardProcessorNode extends ProcessorNode
implements Connectable
case INPUT_REQUIRED: {
if (getIncomingConnections().isEmpty()) {
results.add(new ValidationResult.Builder()
-.explanation("Processor required at least one
Incoming Connection in order to perform its function but currently has no
Incoming Connection")
-.subject("Incoming Connections")
+.explanation("Processor requires an upstream
connection but currently has none")
+.subject("Upstream Connections")
.valid(false)
.build());
}
[08/17] nifi git commit: NIFI-810: Created RequiresInput annotation and ensure that processors are invalid if connections do not agree
http://git-wip-us.apache.org/repos/asf/nifi/blob/4afd8f88/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/DetectDuplicate.java
--
diff --git
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/DetectDuplicate.java
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/DetectDuplicate.java
index b825972..39dc725 100644
---
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/DetectDuplicate.java
+++
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/DetectDuplicate.java
@@ -28,11 +28,13 @@ import java.util.concurrent.TimeUnit;
import org.apache.commons.lang3.StringUtils;
import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
import org.apache.nifi.annotation.behavior.SupportsBatching;
import org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.annotation.documentation.SeeAlso;
import org.apache.nifi.annotation.documentation.Tags;
import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
import org.apache.nifi.components.PropertyDescriptor;
import org.apache.nifi.distributed.cache.client.Deserializer;
import org.apache.nifi.distributed.cache.client.DistributedMapCacheClient;
@@ -52,6 +54,7 @@ import org.apache.nifi.processor.util.StandardValidators;
@EventDriven
@SupportsBatching
@Tags({"hash", "dupe", "duplicate", "dedupe"})
+@InputRequirement(Requirement.INPUT_REQUIRED)
@CapabilityDescription("Caches a value, computed from FlowFile attributes, for
each incoming FlowFile and determines if the cached value has already been
seen. "
+ "If so, routes the FlowFile to 'duplicate' with an attribute named
'original.identifier' that specifies the original FlowFile's"
+ "\"description\", which is specified in the
property. If the FlowFile is not determined to be a duplicate, the Processor "
http://git-wip-us.apache.org/repos/asf/nifi/blob/4afd8f88/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/DistributeLoad.java
--
diff --git
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/DistributeLoad.java
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/DistributeLoad.java
index afff3c4..73ada84 100644
---
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/DistributeLoad.java
+++
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/DistributeLoad.java
@@ -32,9 +32,11 @@ import java.util.concurrent.atomic.AtomicReference;
import org.apache.commons.lang3.StringUtils;
import org.apache.nifi.annotation.behavior.DynamicProperty;
import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
import org.apache.nifi.annotation.behavior.SideEffectFree;
import org.apache.nifi.annotation.behavior.SupportsBatching;
import org.apache.nifi.annotation.behavior.TriggerWhenAnyDestinationAvailable;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
import org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.annotation.behavior.DynamicRelationship;
import org.apache.nifi.annotation.documentation.Tags;
@@ -57,6 +59,7 @@ import org.apache.nifi.processor.util.StandardValidators;
@EventDriven
@SideEffectFree
@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
@TriggerWhenAnyDestinationAvailable
@Tags({"distribute", "load balance", "route", "round robin", "weighted"})
@CapabilityDescription("Distributes FlowFiles to downstream processors based
on a Distribution Strategy. If using the Round Robin "
http://git-wip-us.apache.org/repos/asf/nifi/blob/4afd8f88/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/DuplicateFlowFile.java
--
diff --git
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/DuplicateFlowFile.java
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/DuplicateFlowFile.java
index 7400821..021a94f 100644
---
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-proce
[12/17] nifi git commit: NIFI-810: Updated the wording on validation errors due to upstream connections
NIFI-810: Updated the wording on validation errors due to upstream connections
Project: http://git-wip-us.apache.org/repos/asf/nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/5ecdb185
Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/5ecdb185
Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/5ecdb185
Branch: refs/heads/NIFI-810-InputRequirement
Commit: 5ecdb1858e94938cd426b12bc48d3725109c6e96
Parents: 13edcfd
Author: Mark Payne
Authored: Wed Oct 7 17:26:04 2015 -0400
Committer: Mark Payne
Committed: Wed Oct 7 17:26:14 2015 -0400
--
.../nifi/controller/StandardProcessorNode.java | 2475 +-
1 file changed, 1237 insertions(+), 1238 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/nifi/blob/5ecdb185/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/StandardProcessorNode.java
--
diff --git
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/StandardProcessorNode.java
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/StandardProcessorNode.java
index 0c39eda..3c816d0 100644
---
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/StandardProcessorNode.java
+++
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/StandardProcessorNode.java
@@ -74,1242 +74,1241 @@ import org.slf4j.LoggerFactory;
*/
public class StandardProcessorNode extends ProcessorNode implements
Connectable {
- public static final String BULLETIN_OBSERVER_ID = "bulletin-observer";
-
- public static final TimeUnit DEFAULT_TIME_UNIT = TimeUnit.MILLISECONDS;
- public static final String DEFAULT_YIELD_PERIOD = "1 sec";
- public static final String DEFAULT_PENALIZATION_PERIOD = "30 sec";
- private final AtomicReference processGroup;
- private final Processor processor;
- private final AtomicReference identifier;
- private final Map destinations;
- private final Map> connections;
- private final AtomicReference>
undefinedRelationshipsToTerminate;
- private final AtomicReference> incomingConnectionsRef;
- private final ReentrantReadWriteLock rwLock;
- private final Lock readLock;
- private final Lock writeLock;
- private final AtomicBoolean isolated;
- private final AtomicBoolean lossTolerant;
- private final AtomicReference scheduledState;
- private final AtomicReference comments;
- private final AtomicReference name;
- private final AtomicReference position;
- private final AtomicReference annotationData;
- private final AtomicReference schedulingPeriod; // stored as
string so it's presented to user as they entered it
- private final AtomicReference yieldPeriod;
- private final AtomicReference penalizationPeriod;
- private final AtomicReference> style;
- private final AtomicInteger concurrentTaskCount;
- private final AtomicLong yieldExpiration;
- private final AtomicLong schedulingNanos;
- private final boolean triggerWhenEmpty;
- private final boolean sideEffectFree;
- private final boolean triggeredSerially;
- private final boolean triggerWhenAnyDestinationAvailable;
- private final boolean eventDrivenSupported;
- private final boolean batchSupported;
- private final Requirement inputRequirement;
- private final ValidationContextFactory validationContextFactory;
- private final ProcessScheduler processScheduler;
- private long runNanos = 0L;
-
- private SchedulingStrategy schedulingStrategy; // guarded by
read/write lock
-
- @SuppressWarnings("deprecation")
- public StandardProcessorNode(final Processor processor, final String
uuid, final ValidationContextFactory validationContextFactory,
- final ProcessScheduler scheduler, final
ControllerServiceProvider controllerServiceProvider) {
- super(processor, uuid, validationContextFactory,
controllerServiceProvider);
-
- this.processor = processor;
- identifier = new AtomicReference<>(uuid);
- destinations = new HashMap<>();
- connections = new HashMap<>();
- incomingConnectionsRef = new
AtomicReference>(new ArrayList());
- scheduledState = new AtomicReference<>(ScheduledState.STOPPED);
- rwLock = new ReentrantReadWriteLock(false);
- readLock = rwLock.readLock();
- writeLock = rwLock.writeLock();
-
[01/17] nifi git commit: nifi-992 Adding nifi-couchbase-bundle.
Repository: nifi Updated Branches: refs/heads/NIFI-810-InputRequirement 2215bc848 -> 8e2308b78 nifi-992 Adding nifi-couchbase-bundle. - new CouchbaseClusterControllerService - new Processors - GetCouchbaseKey - PutCouchbaseKey Signed-off-by: Bryan Bende Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/2466a245 Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/2466a245 Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/2466a245 Branch: refs/heads/NIFI-810-InputRequirement Commit: 2466a24530f493238024eb22dc041eebe96621f3 Parents: 96764ed Author: ijokarumawak Authored: Sat Sep 26 02:46:37 2015 +0900 Committer: Bryan Bende Committed: Mon Sep 28 11:21:42 2015 -0400 -- nifi-assembly/NOTICE| 10 + nifi-assembly/pom.xml | 5 + .../nifi-couchbase-nar/pom.xml | 37 +++ .../nifi-couchbase-processors/pom.xml | 208 +++ .../nifi/couchbase/CouchbaseAttributes.java | 59 + .../CouchbaseClusterControllerService.java | 38 +++ .../nifi/couchbase/CouchbaseClusterService.java | 130 ++ .../couchbase/AbstractCouchbaseProcessor.java | 174 + .../nifi/processors/couchbase/DocumentType.java | 36 +++ .../processors/couchbase/GetCouchbaseKey.java | 172 + .../processors/couchbase/PutCouchbaseKey.java | 164 ...org.apache.nifi.controller.ControllerService | 15 ++ .../org.apache.nifi.processor.Processor | 16 ++ .../couchbase/TestCouchbaseClusterService.java | 59 + .../couchbase/TestGetCouchbaseKey.java | 224 .../couchbase/TestPutCouchbaseKey.java | 254 +++ nifi-nar-bundles/nifi-couchbase-bundle/pom.xml | 35 +++ nifi-nar-bundles/pom.xml| 1 + pom.xml | 6 + 19 files changed, 1643 insertions(+) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/2466a245/nifi-assembly/NOTICE -- diff --git a/nifi-assembly/NOTICE b/nifi-assembly/NOTICE index 3362740..1f7e3f1 100644 --- a/nifi-assembly/NOTICE +++ b/nifi-assembly/NOTICE @@ -709,6 +709,16 @@ The following binary components are provided under the Apache Software License v Metadata-Extractor Copyright 2002-2015 Drew Noakes +(ASLv2) Couchbase Java SDK + The following NOTICE information applies: +Couchbase Java SDK +Copyright 2014 Couchbase, Inc. + +(ASLv2) RxJava + The following NOTICE information applies: +Couchbase Java SDK +Copyright 2012 Netflix, Inc. + Common Development and Distribution License 1.1 http://git-wip-us.apache.org/repos/asf/nifi/blob/2466a245/nifi-assembly/pom.xml -- diff --git a/nifi-assembly/pom.xml b/nifi-assembly/pom.xml index f162588..de4fdcb 100644 --- a/nifi-assembly/pom.xml +++ b/nifi-assembly/pom.xml @@ -227,6 +227,11 @@ language governing permissions and limitations under the License. --> nifi-image-nar nar + +org.apache.nifi +nifi-couchbase-nar +nar + http://git-wip-us.apache.org/repos/asf/nifi/blob/2466a245/nifi-nar-bundles/nifi-couchbase-bundle/nifi-couchbase-nar/pom.xml -- diff --git a/nifi-nar-bundles/nifi-couchbase-bundle/nifi-couchbase-nar/pom.xml b/nifi-nar-bundles/nifi-couchbase-bundle/nifi-couchbase-nar/pom.xml new file mode 100644 index 000..4f58d1f --- /dev/null +++ b/nifi-nar-bundles/nifi-couchbase-bundle/nifi-couchbase-nar/pom.xml @@ -0,0 +1,37 @@ + + +http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd";> +4.0.0 + + +org.apache.nifi +nifi-couchbase-bundle +0.3.1-SNAPSHOT + + +nifi-couchbase-nar +0.3.1-SNAPSHOT +nar + + + +org.apache.nifi +nifi-couchbase-processors +0.3.1-SNAPSHOT + + + + http://git-wip-us.apache.org/repos/asf/nifi/blob/2466a245/nifi-nar-bundles/nifi-couchbase-bundle/nifi-couchbase-processors/pom.xml -- diff --git a/nifi-nar-bundles/nifi-couchbase-bundle/nifi-couchbase-processors/pom.xml b/nifi-nar-bundles/nifi-couchbase-bundle/nifi-couchbase-processors/pom.xml new file mode 100644 index 000..33b0baa --- /dev/null +++ b/nifi-nar-bundles/nifi-couchbase-bundl
[05/17] nifi git commit: NIFI-992 Adding NOTICE to nifi-couchbase-nar
NIFI-992 Adding NOTICE to nifi-couchbase-nar Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/6b2f5ad1 Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/6b2f5ad1 Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/6b2f5ad1 Branch: refs/heads/NIFI-810-InputRequirement Commit: 6b2f5ad1f59c6cd2ced5e7225454260cefbbbcce Parents: 88c Author: Bryan Bende Authored: Thu Oct 1 16:56:49 2015 -0400 Committer: Bryan Bende Committed: Thu Oct 1 16:56:49 2015 -0400 -- .../src/main/resources/META-INF/NOTICE | 21 1 file changed, 21 insertions(+) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/6b2f5ad1/nifi-nar-bundles/nifi-couchbase-bundle/nifi-couchbase-nar/src/main/resources/META-INF/NOTICE -- diff --git a/nifi-nar-bundles/nifi-couchbase-bundle/nifi-couchbase-nar/src/main/resources/META-INF/NOTICE b/nifi-nar-bundles/nifi-couchbase-bundle/nifi-couchbase-nar/src/main/resources/META-INF/NOTICE new file mode 100644 index 000..ed69bda --- /dev/null +++ b/nifi-nar-bundles/nifi-couchbase-bundle/nifi-couchbase-nar/src/main/resources/META-INF/NOTICE @@ -0,0 +1,21 @@ +nifi-couchbase-nar +Copyright 2014-2015 The Apache Software Foundation + +This product includes software developed at +The Apache Software Foundation (http://www.apache.org/). + +** +Apache Software License v2 +** + +The following binary components are provided under the Apache Software License v2 + +(ASLv2) Couchbase Java SDK + The following NOTICE information applies: +Couchbase Java SDK +Copyright 2014 Couchbase, Inc. + +(ASLv2) RxJava + The following NOTICE information applies: +Couchbase Java SDK +Copyright 2012 Netflix, Inc. \ No newline at end of file
[jira] [Commented] (NIFI-810) Create Annotation that indicates that a Processor cannot be scheduled to run without an incoming connection
[ https://issues.apache.org/jira/browse/NIFI-810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947666#comment-14947666 ] ASF subversion and git services commented on NIFI-810: -- Commit 13edcfda2ef830e1b160b31f7cd3bea874ccd3f0 in nifi's branch refs/heads/NIFI-810-InputRequirement from [~mcgilman] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=13edcfd ] NIFI-810: - Adding basic support for preventing connection when appropriate. - Updating validation when [dis]connecting processors. > Create Annotation that indicates that a Processor cannot be scheduled to run > without an incoming connection > --- > > Key: NIFI-810 > URL: https://issues.apache.org/jira/browse/NIFI-810 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne > > Currently, if a Processor has no incoming connections but is started, it will > continually without ever accomplishing anything. We should have an > annotation, perhaps @RequiresInput, that indicates that the Processor should > not be scheduled to run unless it has an incoming connection. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[07/17] nifi git commit: NIFI-810: Created RequiresInput annotation and ensure that processors are invalid if connections do not agree
http://git-wip-us.apache.org/repos/asf/nifi/blob/4afd8f88/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ReplaceText.java
--
diff --git
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ReplaceText.java
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ReplaceText.java
index 68155d1..98a56bf 100644
---
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ReplaceText.java
+++
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ReplaceText.java
@@ -16,33 +16,7 @@
*/
package org.apache.nifi.processors.standard;
-import org.apache.nifi.processor.ProcessContext;
-import org.apache.nifi.processor.AbstractProcessor;
-import org.apache.nifi.processor.ProcessorInitializationContext;
-import org.apache.nifi.processor.DataUnit;
-import org.apache.nifi.processor.ProcessSession;
-import org.apache.nifi.processor.Relationship;
import java.io.BufferedWriter;
-
-import org.apache.nifi.components.PropertyDescriptor;
-import org.apache.nifi.components.Validator;
-import org.apache.nifi.expression.AttributeValueDecorator;
-import org.apache.nifi.flowfile.FlowFile;
-import org.apache.nifi.stream.io.StreamUtils;
-import org.apache.nifi.logging.ProcessorLog;
-import org.apache.nifi.annotation.documentation.CapabilityDescription;
-import org.apache.nifi.annotation.behavior.EventDriven;
-import org.apache.nifi.annotation.behavior.SideEffectFree;
-import org.apache.nifi.annotation.behavior.SupportsBatching;
-import org.apache.nifi.annotation.documentation.Tags;
-import org.apache.nifi.processor.exception.ProcessException;
-import org.apache.nifi.processor.io.OutputStreamCallback;
-import org.apache.nifi.processor.io.StreamCallback;
-import org.apache.nifi.processor.util.FlowFileFilters;
-import org.apache.nifi.processor.util.StandardValidators;
-import org.apache.nifi.processors.standard.util.NLKBufferedReader;
-import org.apache.nifi.util.StopWatch;
-
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
@@ -58,9 +32,37 @@ import java.util.concurrent.TimeUnit;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.expression.AttributeValueDecorator;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ProcessorLog;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.OutputStreamCallback;
+import org.apache.nifi.processor.io.StreamCallback;
+import org.apache.nifi.processor.util.FlowFileFilters;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.standard.util.NLKBufferedReader;
+import org.apache.nifi.stream.io.StreamUtils;
+import org.apache.nifi.util.StopWatch;
+
@EventDriven
@SideEffectFree
@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
@Tags({"Text", "Regular Expression", "Update", "Change", "Replace", "Modify",
"Regex"})
@CapabilityDescription("Updates the content of a FlowFile by evaluating a
Regular Expression against it and replacing the section of "
+ "the content that matches the Regular Expression with some alternate
value.")
http://git-wip-us.apache.org/repos/asf/nifi/blob/4afd8f88/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ReplaceTextWithMapping.java
--
diff --git
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ReplaceTextWithMapping.java
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ReplaceTextWithMapping.java
index 04a9c56..f68ac6c 100644
---
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standar
[jira] [Commented] (NIFI-810) Create Annotation that indicates that a Processor cannot be scheduled to run without an incoming connection
[ https://issues.apache.org/jira/browse/NIFI-810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947672#comment-14947672 ] Mark Payne commented on NIFI-810: - I addressed the wording in the validation errors, rebased from master, and pushed the changes. > Create Annotation that indicates that a Processor cannot be scheduled to run > without an incoming connection > --- > > Key: NIFI-810 > URL: https://issues.apache.org/jira/browse/NIFI-810 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne > > Currently, if a Processor has no incoming connections but is started, it will > continually without ever accomplishing anything. We should have an > annotation, perhaps @RequiresInput, that indicates that the Processor should > not be scheduled to run unless it has an incoming connection. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-828) Update default filter in new Processor Dialog
[ https://issues.apache.org/jira/browse/NIFI-828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947698#comment-14947698 ] Mark Payne commented on NIFI-828: - Looks good to me. Applied the patch and was able to verify functionality for new Processors, as well as Controller Services and Reporting Tasks. Looks this also addresses NIFI-442, right? +1 > Update default filter in new Processor Dialog > - > > Key: NIFI-828 > URL: https://issues.apache.org/jira/browse/NIFI-828 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Trivial > Fix For: 0.4.0 > > Attachments: 0001-NIFI-828.patch > > > Update the default filter in the new processor dialog to use 'by tag' instead > of 'by name'. A user will likely want to search for items using terms tagged > on the processor rather than the formal processor name. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
nifi git commit: NIFI-442: NIFI-828: - Always selecting the first item in the new component table. - Enabling adding the selected component by typing Enter. - Removing the 'filter by' in the new compo
Repository: nifi Updated Branches: refs/heads/master 31fba6b33 -> 2583d7869 NIFI-442: NIFI-828: - Always selecting the first item in the new component table. - Enabling adding the selected component by typing Enter. - Removing the 'filter by' in the new component dialogs and instead just searching every field. Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/2583d786 Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/2583d786 Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/2583d786 Branch: refs/heads/master Commit: 2583d7869acedcbe536d4dd5eaf2e7c8f9cf149c Parents: 31fba6b Author: Matt Gilman Authored: Wed Oct 7 19:11:55 2015 -0400 Committer: Matt Gilman Committed: Wed Oct 7 19:11:55 2015 -0400 -- .../canvas/new-controller-service-dialog.jsp| 1 - .../partials/canvas/new-processor-dialog.jsp| 1 - .../canvas/new-reporting-task-dialog.jsp| 1 - .../css/new-controller-service-dialog.css | 9 -- .../main/webapp/css/new-processor-dialog.css| 9 -- .../webapp/css/new-reporting-task-dialog.css| 9 -- .../webapp/js/nf/canvas/nf-canvas-toolbox.js| 60 .../src/main/webapp/js/nf/canvas/nf-settings.js | 140 +-- 8 files changed, 105 insertions(+), 125 deletions(-) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/2583d786/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/WEB-INF/partials/canvas/new-controller-service-dialog.jsp -- diff --git a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/WEB-INF/partials/canvas/new-controller-service-dialog.jsp b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/WEB-INF/partials/canvas/new-controller-service-dialog.jsp index eb54ace..9463cb9 100644 --- a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/WEB-INF/partials/canvas/new-controller-service-dialog.jsp +++ b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/WEB-INF/partials/canvas/new-controller-service-dialog.jsp @@ -20,7 +20,6 @@ - Displaying of http://git-wip-us.apache.org/repos/asf/nifi/blob/2583d786/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/WEB-INF/partials/canvas/new-processor-dialog.jsp -- diff --git a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/WEB-INF/partials/canvas/new-processor-dialog.jsp b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/WEB-INF/partials/canvas/new-processor-dialog.jsp index df7766c..ad33c1c 100644 --- a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/WEB-INF/partials/canvas/new-processor-dialog.jsp +++ b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/WEB-INF/partials/canvas/new-processor-dialog.jsp @@ -20,7 +20,6 @@ - Displaying of http://git-wip-us.apache.org/repos/asf/nifi/blob/2583d786/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/WEB-INF/partials/canvas/new-reporting-task-dialog.jsp -- diff --git a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/WEB-INF/partials/canvas/new-reporting-task-dialog.jsp b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/WEB-INF/partials/canvas/new-reporting-task-dialog.jsp index cfb3992..2025aeb 100644 --- a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/WEB-INF/partials/canvas/new-reporting-task-dialog.jsp +++ b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/WEB-INF/partials/canvas/new-reporting-task-dialog.jsp @@ -20,7 +20,6 @@ - Displaying of http://git-wip-us.apache.org/repos/asf/nifi/blob/2583d786/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/css/new-controller-service-dialog.css -- diff --git a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp
[jira] [Commented] (NIFI-828) Update default filter in new Processor Dialog
[ https://issues.apache.org/jira/browse/NIFI-828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947779#comment-14947779 ] ASF subversion and git services commented on NIFI-828: -- Commit 2583d7869acedcbe536d4dd5eaf2e7c8f9cf149c in nifi's branch refs/heads/master from [~mcgilman] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=2583d78 ] NIFI-442: NIFI-828: - Always selecting the first item in the new component table. - Enabling adding the selected component by typing Enter. - Removing the 'filter by' in the new component dialogs and instead just searching every field. > Update default filter in new Processor Dialog > - > > Key: NIFI-828 > URL: https://issues.apache.org/jira/browse/NIFI-828 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Trivial > Fix For: 0.4.0 > > Attachments: 0001-NIFI-828.patch > > > Update the default filter in the new processor dialog to use 'by tag' instead > of 'by name'. A user will likely want to search for items using terms tagged > on the processor rather than the formal processor name. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-442) New processor search box should auto-select first entry
[ https://issues.apache.org/jira/browse/NIFI-442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947778#comment-14947778 ] ASF subversion and git services commented on NIFI-442: -- Commit 2583d7869acedcbe536d4dd5eaf2e7c8f9cf149c in nifi's branch refs/heads/master from [~mcgilman] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=2583d78 ] NIFI-442: NIFI-828: - Always selecting the first item in the new component table. - Enabling adding the selected component by typing Enter. - Removing the 'filter by' in the new component dialogs and instead just searching every field. > New processor search box should auto-select first entry > --- > > Key: NIFI-442 > URL: https://issues.apache.org/jira/browse/NIFI-442 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Affects Versions: 0.0.2 >Reporter: Ryan Blue >Assignee: Matt Gilman >Priority: Minor > Labels: beginner > Fix For: 0.4.0 > > > When I go to add a processor to the graph, I can start typing the name of a > processor. (Great!) I often know what I'm looking for and type the name > directly, but when I hit enter, I get a message saying that I didn't select a > processor, even though there was only one result from my filter. It would be > really convenient if the first result were automatically selected so when I > hit enter, it's added instead of showing me an error. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (NIFI-442) New processor search box should auto-select first entry
[ https://issues.apache.org/jira/browse/NIFI-442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman resolved NIFI-442. -- Resolution: Fixed > New processor search box should auto-select first entry > --- > > Key: NIFI-442 > URL: https://issues.apache.org/jira/browse/NIFI-442 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Affects Versions: 0.0.2 >Reporter: Ryan Blue >Assignee: Matt Gilman >Priority: Minor > Labels: beginner > Fix For: 0.4.0 > > > When I go to add a processor to the graph, I can start typing the name of a > processor. (Great!) I often know what I'm looking for and type the name > directly, but when I hit enter, I get a message saying that I didn't select a > processor, even though there was only one result from my filter. It would be > really convenient if the first result were automatically selected so when I > hit enter, it's added instead of showing me an error. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-442) New processor search box should auto-select first entry
[ https://issues.apache.org/jira/browse/NIFI-442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947780#comment-14947780 ] Matt Gilman commented on NIFI-442: -- This ticket was address in the patch supplied on NIFI-828. > New processor search box should auto-select first entry > --- > > Key: NIFI-442 > URL: https://issues.apache.org/jira/browse/NIFI-442 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Affects Versions: 0.0.2 >Reporter: Ryan Blue >Assignee: Matt Gilman >Priority: Minor > Labels: beginner > Fix For: 0.4.0 > > > When I go to add a processor to the graph, I can start typing the name of a > processor. (Great!) I often know what I'm looking for and type the name > directly, but when I hit enter, I get a message saying that I didn't select a > processor, even though there was only one result from my filter. It would be > really convenient if the first result were automatically selected so when I > hit enter, it's added instead of showing me an error. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-828) Update default filter in new Processor Dialog
[ https://issues.apache.org/jira/browse/NIFI-828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947782#comment-14947782 ] Matt Gilman commented on NIFI-828: -- Yes it does. 442 is not closed as well. Thanks! > Update default filter in new Processor Dialog > - > > Key: NIFI-828 > URL: https://issues.apache.org/jira/browse/NIFI-828 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Trivial > Fix For: 0.4.0 > > Attachments: 0001-NIFI-828.patch > > > Update the default filter in the new processor dialog to use 'by tag' instead > of 'by name'. A user will likely want to search for items using terms tagged > on the processor rather than the formal processor name. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (NIFI-828) Update default filter in new Processor Dialog
[ https://issues.apache.org/jira/browse/NIFI-828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947782#comment-14947782 ] Matt Gilman edited comment on NIFI-828 at 10/7/15 11:45 PM: Yes it does. 442 is now closed as well. Thanks! was (Author: mcgilman): Yes it does. 442 is not closed as well. Thanks! > Update default filter in new Processor Dialog > - > > Key: NIFI-828 > URL: https://issues.apache.org/jira/browse/NIFI-828 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Trivial > Fix For: 0.4.0 > > Attachments: 0001-NIFI-828.patch > > > Update the default filter in the new processor dialog to use 'by tag' instead > of 'by name'. A user will likely want to search for items using terms tagged > on the processor rather than the formal processor name. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-817) Create Processors to interact with HBase
[ https://issues.apache.org/jira/browse/NIFI-817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947840#comment-14947840 ] Nick Dimiduk commented on NIFI-817: --- Replication endpoint might be tricky (I don't know NiFi architecture) as it's a "push" model rather than "pull". Meaning, you register the destination endpoint with the HBase cluster and it pushes edits that endpoint. Here "endpoint" means a service or cluster that can handle all the region servers in the source cluster sending data. > Create Processors to interact with HBase > > > Key: NIFI-817 > URL: https://issues.apache.org/jira/browse/NIFI-817 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Mark Payne >Assignee: Bryan Bende > Fix For: 0.4.0 > > Attachments: > 0001-NIFI-817-Initial-implementation-of-HBase-processors.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-447) Add "Replacement Strategy" property to ReplaceText
[ https://issues.apache.org/jira/browse/NIFI-447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-447: Attachment: 0001-NIFI-447-Fixed-some-typos-and-clarified-some-documen.patch > Add "Replacement Strategy" property to ReplaceText > -- > > Key: NIFI-447 > URL: https://issues.apache.org/jira/browse/NIFI-447 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne > Fix For: 0.4.0 > > Attachments: > 0001-NIFI-447-Fixed-some-typos-and-clarified-some-documen.patch, > NIFI-447.patch > > > "Regular Expression" property should have displayName changed to "Search > Value" > ReplaceText should allow for a "Replacement Strategy" with values: > Prepend (Search Value is ignored; replacement value prepended to FlowFile) > Append (Search Value is ignored; replacement value appended to FlowFIle) > Regex Replace (current behavior, so this should be default) > Linewise Regex Replace (like current behavior if Evaluation Mode is set to > Line-by-Line) > Literal Replace (do not interpret the Replacement value > The Line-by-Line property should be removed; this is breaking change! -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-447) Add "Replacement Strategy" property to ReplaceText
[ https://issues.apache.org/jira/browse/NIFI-447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947884#comment-14947884 ] Mark Payne commented on NIFI-447: - [~JPercivall] it looks like you mentioned Michael Payne in the last comment, not me :) Patch looks really good now. I did find one bug related to the character set. There were two places where it was calling getBytes() without passing in the appropriate character set. I went ahead and addressed those in a new patch. I also fixed a couple of typos in the documentation and (at least I think) clarified a bit of the documentation. There were also a couple of performance improvements that I noticed could be implemented. For example, since it's running over a few FlowFiles in each iteration of onTrigger, I created a single byte array and reuse it rather than creating a new byte array for each FlowFile. I also noticed that for the Literal Replace, it was using a regular expression in order to insert the Replacement Value before the line ending. This definitely appears to work okay but can be pretty expensive, especially since the regex starts with a .* which is a very expensive operation to perform in regex land. So I changed it a bit to just copy the data character-by-character from the (buffered) input stream to the (buffered) output stream until we find a \r or a \n and at that point write out the replacement value, and then write out the rest of the characters. I added unit tests for this to ensure that it's working right, and all of the existing tests that you created pass as well. I attached a new patch with these things merged in. Please review and make sure that you're okay with the changes that I made. Thanks -Mark > Add "Replacement Strategy" property to ReplaceText > -- > > Key: NIFI-447 > URL: https://issues.apache.org/jira/browse/NIFI-447 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne > Fix For: 0.4.0 > > Attachments: > 0001-NIFI-447-Fixed-some-typos-and-clarified-some-documen.patch, > NIFI-447.patch > > > "Regular Expression" property should have displayName changed to "Search > Value" > ReplaceText should allow for a "Replacement Strategy" with values: > Prepend (Search Value is ignored; replacement value prepended to FlowFile) > Append (Search Value is ignored; replacement value appended to FlowFIle) > Regex Replace (current behavior, so this should be default) > Linewise Regex Replace (like current behavior if Evaluation Mode is set to > Line-by-Line) > Literal Replace (do not interpret the Replacement value > The Line-by-Line property should be removed; this is breaking change! -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-447) Add "Replacement Strategy" property to ReplaceText
[ https://issues.apache.org/jira/browse/NIFI-447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947972#comment-14947972 ] Joseph Percivall commented on NIFI-447: --- Whops sorry Michael [~markap14] Ah that's a good catch with the getBytes. Why did you change the "name" of SEARCH_VALUE to "Regular Expression" and not keep it the same as the display name? That's a great work around to my regex hack for Append lol. I think I was stuck in the regex mind set when trying to do it. I ran a mvn install contrib check and it looks good. Joe > Add "Replacement Strategy" property to ReplaceText > -- > > Key: NIFI-447 > URL: https://issues.apache.org/jira/browse/NIFI-447 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne > Fix For: 0.4.0 > > Attachments: > 0001-NIFI-447-Fixed-some-typos-and-clarified-some-documen.patch, > NIFI-447.patch > > > "Regular Expression" property should have displayName changed to "Search > Value" > ReplaceText should allow for a "Replacement Strategy" with values: > Prepend (Search Value is ignored; replacement value prepended to FlowFile) > Append (Search Value is ignored; replacement value appended to FlowFIle) > Regex Replace (current behavior, so this should be default) > Linewise Regex Replace (like current behavior if Evaluation Mode is set to > Line-by-Line) > Literal Replace (do not interpret the Replacement value > The Line-by-Line property should be removed; this is breaking change! -- This message was sent by Atlassian JIRA (v6.3.4#6332)
