[jira] [Commented] (NIFI-1899) Create ListenSMTP & ExtractEmailAttachment processors
[ https://issues.apache.org/jira/browse/NIFI-1899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15311956#comment-15311956 ] ASF GitHub Bot commented on NIFI-1899: -- Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/483 @bbende Was aware of that possibility. What I couldn't find is a way of easily doing the same from Expression Language. It would be nice to be able to access the flowfile lineage from expression language. Anyhow, initial processor is ready and with test harness. Working on ExtractEmailAttributes. @joewitt: ExtractEmailAttributes will have the attributes: * email.headers.To * email.headers.From * email.headers.Message-ID * email.headers.Date * email.headers.Subject * email.attachment_count Plus the options of: A. Choosing what attributes get extracted and adding them to * email.headers.matched_name_here; B. OR extract all headers under * email.headers.* I fear the second option is a terrible security choice (import something you don't have much control over but I'll leave that to the user to judge) > Create ListenSMTP & ExtractEmailAttachment processors > - > > Key: NIFI-1899 > URL: https://issues.apache.org/jira/browse/NIFI-1899 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Andre > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1897) Allow Nodes to Federate Requests to other nodes
[ https://issues.apache.org/jira/browse/NIFI-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312241#comment-15312241 ] ASF GitHub Bot commented on NIFI-1897: -- GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/488 NIFI-1897: Refactoring to allow requests to be replicated from a node to other nodes Nodes are now capable of replicating requests across the cluster and a Cluster Coordinator is auto-elected to monitor heartbeats and provide the up-to-date flow to newly joining nodes. The WebClusterManager and associated components has been removed! You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-1897 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/488.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #488 commit 6633904b6db6b75880ad6b5e185081423b644d16 Author: Mark Payne Date: 2016-05-19T14:42:39Z NIFI-1897: Refactoring to allow requests to be replicated from a node to other nodes > Allow Nodes to Federate Requests to other nodes > --- > > Key: NIFI-1897 > URL: https://issues.apache.org/jira/browse/NIFI-1897 > Project: Apache NiFi > Issue Type: Task > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne > Fix For: 1.0.0 > > > When a user makes a web request to any node, it should be capable of > replicating the request to all nodes in the cluster, handling the logic that > the NCM currently handles for replicating requests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1951) Custom UIs
[ https://issues.apache.org/jira/browse/NIFI-1951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312259#comment-15312259 ] ASF GitHub Bot commented on NIFI-1951: -- GitHub user mcgilman opened a pull request: https://github.com/apache/nifi/pull/489 NIFI-1951: Custom UIs - Removing deprecated NiFiWebContext and related classes. - Adding authorization to Custom UIs. - Fixing issue when creating ControllerService inline. - Addressing contentType issue when attempting to clear component state. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mcgilman/nifi NIFI-1951 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/489.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #489 commit bfe0861b2a409aa84df450d712c2485121d0c6d3 Author: Matt Gilman Date: 2016-06-02T12:57:24Z NIFI-1951: - Removing deprecated NiFiWebContext and related classes. - Adding authorization to Custom UIs. - Fixing issue when creating ControllerService inline. - Addressing contentType issue when attempting to clear component state. > Custom UIs > -- > > Key: NIFI-1951 > URL: https://issues.apache.org/jira/browse/NIFI-1951 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Matt Gilman >Assignee: Matt Gilman > Fix For: 1.0.0 > > > - Remove deprecated NiFiWebContext and related classes > - Ensure appropriate access for calls into NiFiWebConfigurationContext -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1948) Templates created should include template encoding version
[ https://issues.apache.org/jira/browse/NIFI-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312277#comment-15312277 ] ASF GitHub Bot commented on NIFI-1948: -- Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/485 Reviewing... > Templates created should include template encoding version > -- > > Key: NIFI-1948 > URL: https://issues.apache.org/jira/browse/NIFI-1948 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core Framework >Affects Versions: 1.0.0 >Reporter: Jeff Storck >Assignee: Jeff Storck > Fix For: 1.0.0 > > > Templates created, starting with version Apache NiFi 1.0.0, should include > the template encoding version. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
nifi git commit: NIFI-1948 Fixed encoding-version attribute not getting created in exported templates. This closes #485
Repository: nifi
Updated Branches:
refs/heads/master 812731497 -> 3e064101e
NIFI-1948 Fixed encoding-version attribute not getting created in exported
templates. This closes #485
Project: http://git-wip-us.apache.org/repos/asf/nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/3e064101
Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/3e064101
Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/3e064101
Branch: refs/heads/master
Commit: 3e064101e9df8cfef037a20a69b7735771722370
Parents: 8127314
Author: Jeff Storck
Authored: Wed Jun 1 11:36:41 2016 -0400
Committer: Matt Gilman
Committed: Thu Jun 2 09:29:11 2016 -0400
--
.../src/main/java/org/apache/nifi/web/api/dto/DtoFactory.java | 2 ++
1 file changed, 2 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/nifi/blob/3e064101/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/dto/DtoFactory.java
--
diff --git
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/dto/DtoFactory.java
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/dto/DtoFactory.java
index 4e15e3b..d5c5f1d 100644
---
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/dto/DtoFactory.java
+++
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/dto/DtoFactory.java
@@ -746,6 +746,8 @@ public final class DtoFactory {
copy.setDescription(original.getDescription());
copy.setTimestamp(original.getTimestamp());
copy.setUri(original.getUri());
+copy.setEncodingVersion(original.getEncodingVersion());
+
return copy;
}
[jira] [Commented] (NIFI-1948) Templates created should include template encoding version
[ https://issues.apache.org/jira/browse/NIFI-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312303#comment-15312303 ] ASF subversion and git services commented on NIFI-1948: --- Commit 3e064101e9df8cfef037a20a69b7735771722370 in nifi's branch refs/heads/master from [~jtstorck] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=3e06410 ] NIFI-1948 Fixed encoding-version attribute not getting created in exported templates. This closes #485 > Templates created should include template encoding version > -- > > Key: NIFI-1948 > URL: https://issues.apache.org/jira/browse/NIFI-1948 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core Framework >Affects Versions: 1.0.0 >Reporter: Jeff Storck >Assignee: Jeff Storck > Fix For: 1.0.0 > > > Templates created, starting with version Apache NiFi 1.0.0, should include > the template encoding version. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (NIFI-1948) Templates created should include template encoding version
[ https://issues.apache.org/jira/browse/NIFI-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman resolved NIFI-1948. --- Resolution: Fixed > Templates created should include template encoding version > -- > > Key: NIFI-1948 > URL: https://issues.apache.org/jira/browse/NIFI-1948 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core Framework >Affects Versions: 1.0.0 >Reporter: Jeff Storck >Assignee: Jeff Storck > Fix For: 1.0.0 > > > Templates created, starting with version Apache NiFi 1.0.0, should include > the template encoding version. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1928) UI - Remove registration process
[ https://issues.apache.org/jira/browse/NIFI-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312307#comment-15312307 ] ASF GitHub Bot commented on NIFI-1928: -- Github user bbende commented on the issue: https://github.com/apache/nifi/pull/482 Reviewing... > UI - Remove registration process > > > Key: NIFI-1928 > URL: https://issues.apache.org/jira/browse/NIFI-1928 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman > Fix For: 1.0.0 > > > With the updated user authorization model, users will no longer be able to > request an account. Instead, users are authorized through the configured > Authorizer implementation (it's pluggable). Consequently, we can remove all > the account registration forms from the UI and OBE REST endpoints. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1943) PutFile should have ReadsAttribute annotation for "filename"
[ https://issues.apache.org/jira/browse/NIFI-1943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312314#comment-15312314 ] ASF GitHub Bot commented on NIFI-1943: -- Github user trkurc commented on the issue: https://github.com/apache/nifi/pull/480 I will start reviewing this. > PutFile should have ReadsAttribute annotation for "filename" > > > Key: NIFI-1943 > URL: https://issues.apache.org/jira/browse/NIFI-1943 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joseph Percivall >Assignee: Joseph Percivall >Priority: Minor > Fix For: 0.7.0 > > > The PutFile processor writes the flowfile to disk using the value contained > in the "filename" attribute. There should be documentation telling the user > this. > All that needs to be done is to add a "ReadsAttribute" annotation for this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1948) Templates created should include template encoding version
[ https://issues.apache.org/jira/browse/NIFI-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312306#comment-15312306 ] ASF GitHub Bot commented on NIFI-1948: -- Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/485 +1 Looks good. Verified functionality exporting/importing templates across 0.x and 1.x versions. Merged to master. > Templates created should include template encoding version > -- > > Key: NIFI-1948 > URL: https://issues.apache.org/jira/browse/NIFI-1948 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core Framework >Affects Versions: 1.0.0 >Reporter: Jeff Storck >Assignee: Jeff Storck > Fix For: 1.0.0 > > > Templates created, starting with version Apache NiFi 1.0.0, should include > the template encoding version. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1948) Templates created should include template encoding version
[ https://issues.apache.org/jira/browse/NIFI-1948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312304#comment-15312304 ] ASF GitHub Bot commented on NIFI-1948: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/485 > Templates created should include template encoding version > -- > > Key: NIFI-1948 > URL: https://issues.apache.org/jira/browse/NIFI-1948 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core Framework >Affects Versions: 1.0.0 >Reporter: Jeff Storck >Assignee: Jeff Storck > Fix For: 1.0.0 > > > Templates created, starting with version Apache NiFi 1.0.0, should include > the template encoding version. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1265) Precompiling message-page.jsp causes runtime error in CentOS 6.7 and Fedora 23
[
https://issues.apache.org/jira/browse/NIFI-1265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312323#comment-15312323
]
ASF subversion and git services commented on NIFI-1265:
---
Commit 4f101294ebb7fc3b10fd2ec53a7ce86487835005 in nifi's branch
refs/heads/master from [~JPercivall]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=4f10129 ]
NIFI-1265 Fixing InvokeHttp tests.
Signed-off-by: Matt Gilman
> Precompiling message-page.jsp causes runtime error in CentOS 6.7 and Fedora 23
> --
>
> Key: NIFI-1265
> URL: https://issues.apache.org/jira/browse/NIFI-1265
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Minor
> Fix For: 1.0.0
>
> Attachments: jetty.447816.patch
>
>
> During start up it appears that when message-page.jsp is precompiled and run
> in CentOS 6.7 or Fedora 23 the following method in ServletHolder (Jetty)
> fails with:
> {noformat}
> 2015-12-07 10:01:26,557 WARN [main] org.apache.nifi.web.server.JettyServer
> Failed to start web server... shutting down.
> java.lang.IllegalArgumentException: Comparison method violates its general
> contract!
> at java.util.ComparableTimSort.mergeHi(ComparableTimSort.java:866)
> ~[na:1.8.0_65]
> at java.util.ComparableTimSort.mergeAt(ComparableTimSort.java:483)
> ~[na:1.8.0_65]
> at
> java.util.ComparableTimSort.mergeForceCollapse(ComparableTimSort.java:422)
> ~[na:1.8.0_65]
> at java.util.ComparableTimSort.sort(ComparableTimSort.java:222)
> ~[na:1.8.0_65]
> at java.util.Arrays.sort(Arrays.java:1246) ~[na:1.8.0_65]
> at
> org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:865)
> ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
> at
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:298)
> ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
> at
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
> ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
> {noformat}
> {code}
> @Override
> public int compareTo(ServletHolder sh)
> {
> if (sh==this)
> return 0;
> if (sh._initOrder<_initOrder)
> return 1;
> if (sh._initOrder>_initOrder)
> return -1;
> int c=(_className!=null &&
> sh._className!=null)?_className.compareTo(sh._className):0;
> if (c==0)
> c=_name.compareTo(sh._name);
> return c;
> }
> {code}
> After a quick glance it may be that this method is not transitive when
> falling back to comparing by className and name. From the Java API
> {noformat}
> The implementor must also ensure that the relation is transitive:
> ((compare(x, y)>0) && (compare(y, z)>0)) implies compare(x, z)>0.
> {noformat}
> This could fail with the following scenario:
> {noformat}
> x._initOrder = -1
> x._className = null
> x._name = Login
> y._initOrder = -1
> y._className = org.apache.nifi.web.jsp.WEB_002dINF.pages.message_002dpage_jsp
> y._name = org.apache.nifi.web.jsp.WEB_002dINF.pages.message_002dpage_jsp
> z._initOrder = -1
> z._className = org.apache.nifi.web.servlet.DownloadSvg
> z._name = DownloadSvg
> {noformat}
> Unfortunately, I am unable to replicate the exception. However, my
> observation from debugging show that when JSPs are precompiled the _className
> and _name are the same (like y above). When a Servlet is referenced in the
> web.xml it's name is overridden (like z above). And when a JSP is referenced
> in the web.xml the _className is null and the _name is set to the name from
> the web.xml (like x above).
> In the scenario outlined: X < Y and Y < Z but Z < X. When running locally I
> did not see the exception mentioned above but the resulting sort order was
> not correct as it ultimately depended on which elements were compared to each
> other.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (NIFI-1265) Precompiling message-page.jsp causes runtime error in CentOS 6.7 and Fedora 23
[
https://issues.apache.org/jira/browse/NIFI-1265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne resolved NIFI-1265.
--
Resolution: Fixed
> Precompiling message-page.jsp causes runtime error in CentOS 6.7 and Fedora 23
> --
>
> Key: NIFI-1265
> URL: https://issues.apache.org/jira/browse/NIFI-1265
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Minor
> Fix For: 1.0.0
>
> Attachments: jetty.447816.patch
>
>
> During start up it appears that when message-page.jsp is precompiled and run
> in CentOS 6.7 or Fedora 23 the following method in ServletHolder (Jetty)
> fails with:
> {noformat}
> 2015-12-07 10:01:26,557 WARN [main] org.apache.nifi.web.server.JettyServer
> Failed to start web server... shutting down.
> java.lang.IllegalArgumentException: Comparison method violates its general
> contract!
> at java.util.ComparableTimSort.mergeHi(ComparableTimSort.java:866)
> ~[na:1.8.0_65]
> at java.util.ComparableTimSort.mergeAt(ComparableTimSort.java:483)
> ~[na:1.8.0_65]
> at
> java.util.ComparableTimSort.mergeForceCollapse(ComparableTimSort.java:422)
> ~[na:1.8.0_65]
> at java.util.ComparableTimSort.sort(ComparableTimSort.java:222)
> ~[na:1.8.0_65]
> at java.util.Arrays.sort(Arrays.java:1246) ~[na:1.8.0_65]
> at
> org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:865)
> ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
> at
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:298)
> ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
> at
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
> ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
> {noformat}
> {code}
> @Override
> public int compareTo(ServletHolder sh)
> {
> if (sh==this)
> return 0;
> if (sh._initOrder<_initOrder)
> return 1;
> if (sh._initOrder>_initOrder)
> return -1;
> int c=(_className!=null &&
> sh._className!=null)?_className.compareTo(sh._className):0;
> if (c==0)
> c=_name.compareTo(sh._name);
> return c;
> }
> {code}
> After a quick glance it may be that this method is not transitive when
> falling back to comparing by className and name. From the Java API
> {noformat}
> The implementor must also ensure that the relation is transitive:
> ((compare(x, y)>0) && (compare(y, z)>0)) implies compare(x, z)>0.
> {noformat}
> This could fail with the following scenario:
> {noformat}
> x._initOrder = -1
> x._className = null
> x._name = Login
> y._initOrder = -1
> y._className = org.apache.nifi.web.jsp.WEB_002dINF.pages.message_002dpage_jsp
> y._name = org.apache.nifi.web.jsp.WEB_002dINF.pages.message_002dpage_jsp
> z._initOrder = -1
> z._className = org.apache.nifi.web.servlet.DownloadSvg
> z._name = DownloadSvg
> {noformat}
> Unfortunately, I am unable to replicate the exception. However, my
> observation from debugging show that when JSPs are precompiled the _className
> and _name are the same (like y above). When a Servlet is referenced in the
> web.xml it's name is overridden (like z above). And when a JSP is referenced
> in the web.xml the _className is null and the _name is set to the name from
> the web.xml (like x above).
> In the scenario outlined: X < Y and Y < Z but Z < X. When running locally I
> did not see the exception mentioned above but the resulting sort order was
> not correct as it ultimately depended on which elements were compared to each
> other.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-1265) Precompiling message-page.jsp causes runtime error in CentOS 6.7 and Fedora 23
[
https://issues.apache.org/jira/browse/NIFI-1265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312329#comment-15312329
]
Mark Payne commented on NIFI-1265:
--
This all looks good. Was able to test that application starts up and runs
properly. Tested ListenHTTP, HandleHttpRequest, HandleHttpResponse processors
as well, including using a SSL and non-SSL configurations. All works as
expected. Merged to master.
> Precompiling message-page.jsp causes runtime error in CentOS 6.7 and Fedora 23
> --
>
> Key: NIFI-1265
> URL: https://issues.apache.org/jira/browse/NIFI-1265
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Minor
> Fix For: 1.0.0
>
> Attachments: jetty.447816.patch
>
>
> During start up it appears that when message-page.jsp is precompiled and run
> in CentOS 6.7 or Fedora 23 the following method in ServletHolder (Jetty)
> fails with:
> {noformat}
> 2015-12-07 10:01:26,557 WARN [main] org.apache.nifi.web.server.JettyServer
> Failed to start web server... shutting down.
> java.lang.IllegalArgumentException: Comparison method violates its general
> contract!
> at java.util.ComparableTimSort.mergeHi(ComparableTimSort.java:866)
> ~[na:1.8.0_65]
> at java.util.ComparableTimSort.mergeAt(ComparableTimSort.java:483)
> ~[na:1.8.0_65]
> at
> java.util.ComparableTimSort.mergeForceCollapse(ComparableTimSort.java:422)
> ~[na:1.8.0_65]
> at java.util.ComparableTimSort.sort(ComparableTimSort.java:222)
> ~[na:1.8.0_65]
> at java.util.Arrays.sort(Arrays.java:1246) ~[na:1.8.0_65]
> at
> org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:865)
> ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
> at
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:298)
> ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
> at
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
> ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
> {noformat}
> {code}
> @Override
> public int compareTo(ServletHolder sh)
> {
> if (sh==this)
> return 0;
> if (sh._initOrder<_initOrder)
> return 1;
> if (sh._initOrder>_initOrder)
> return -1;
> int c=(_className!=null &&
> sh._className!=null)?_className.compareTo(sh._className):0;
> if (c==0)
> c=_name.compareTo(sh._name);
> return c;
> }
> {code}
> After a quick glance it may be that this method is not transitive when
> falling back to comparing by className and name. From the Java API
> {noformat}
> The implementor must also ensure that the relation is transitive:
> ((compare(x, y)>0) && (compare(y, z)>0)) implies compare(x, z)>0.
> {noformat}
> This could fail with the following scenario:
> {noformat}
> x._initOrder = -1
> x._className = null
> x._name = Login
> y._initOrder = -1
> y._className = org.apache.nifi.web.jsp.WEB_002dINF.pages.message_002dpage_jsp
> y._name = org.apache.nifi.web.jsp.WEB_002dINF.pages.message_002dpage_jsp
> z._initOrder = -1
> z._className = org.apache.nifi.web.servlet.DownloadSvg
> z._name = DownloadSvg
> {noformat}
> Unfortunately, I am unable to replicate the exception. However, my
> observation from debugging show that when JSPs are precompiled the _className
> and _name are the same (like y above). When a Servlet is referenced in the
> web.xml it's name is overridden (like z above). And when a JSP is referenced
> in the web.xml the _className is null and the _name is set to the name from
> the web.xml (like x above).
> In the scenario outlined: X < Y and Y < Z but Z < X. When running locally I
> did not see the exception mentioned above but the resulting sort order was
> not correct as it ultimately depended on which elements were compared to each
> other.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (NIFI-1935) Added ConvertDynamicJsonToAvro processor
[ https://issues.apache.org/jira/browse/NIFI-1935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Kurc updated NIFI-1935: Assignee: Alex Halldin > Added ConvertDynamicJsonToAvro processor > > > Key: NIFI-1935 > URL: https://issues.apache.org/jira/browse/NIFI-1935 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Daniel Cave >Assignee: Alex Halldin >Priority: Minor > Fix For: 1.0.0, 0.7.0 > > > ConvertJsonToAvro required a predefined Avro schema to convert JSON and > required the presence of all field on the incoming JSON. > ConvertDynamicJsonToAvro functions similarly, however it now accepts the JSON > and schema as incoming flowfiles and creates the Avro dynamically. > This processor requires the InferAvroSchema processor in its upstream flow so > that it can use the original and schema flowfiles as input. These two > flowfiles will have the unique attribute inferredAvroId set on them by > InferAvroSchema so that they can be properly matched in > ConvertDynamicJsonToAvro. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1935) Added ConvertDynamicJsonToAvro processor
[ https://issues.apache.org/jira/browse/NIFI-1935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312331#comment-15312331 ] Tony Kurc commented on NIFI-1935: - done > Added ConvertDynamicJsonToAvro processor > > > Key: NIFI-1935 > URL: https://issues.apache.org/jira/browse/NIFI-1935 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Daniel Cave >Assignee: Alex Halldin >Priority: Minor > Fix For: 1.0.0, 0.7.0 > > > ConvertJsonToAvro required a predefined Avro schema to convert JSON and > required the presence of all field on the incoming JSON. > ConvertDynamicJsonToAvro functions similarly, however it now accepts the JSON > and schema as incoming flowfiles and creates the Avro dynamically. > This processor requires the InferAvroSchema processor in its upstream flow so > that it can use the original and schema flowfiles as input. These two > flowfiles will have the unique attribute inferredAvroId set on them by > InferAvroSchema so that they can be properly matched in > ConvertDynamicJsonToAvro. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[2/3] nifi git commit: NIFI-1265 Fixing InvokeHttp tests.
NIFI-1265 Fixing InvokeHttp tests.
Signed-off-by: Matt Gilman
Project: http://git-wip-us.apache.org/repos/asf/nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/4f101294
Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/4f101294
Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/4f101294
Branch: refs/heads/master
Commit: 4f101294ebb7fc3b10fd2ec53a7ce86487835005
Parents: e587c99
Author: jpercivall
Authored: Fri May 27 14:39:43 2016 -0400
Committer: Mark Payne
Committed: Thu Jun 2 09:49:57 2016 -0400
--
.../processors/standard/TestInvokeHTTP.java | 4 +-
.../standard/util/TestInvokeHttpCommon.java | 43 +++-
2 files changed, 26 insertions(+), 21 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/nifi/blob/4f101294/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestInvokeHTTP.java
--
diff --git
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestInvokeHTTP.java
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestInvokeHTTP.java
index e116f8c..b99f4b8 100644
---
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestInvokeHTTP.java
+++
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestInvokeHTTP.java
@@ -124,7 +124,7 @@ public class TestInvokeHTTP extends TestInvokeHttpCommon {
bundle1.assertAttributeEquals(InvokeHTTP.STATUS_CODE, "200");
bundle1.assertAttributeEquals(InvokeHTTP.STATUS_MESSAGE, "OK");
bundle1.assertAttributeEquals("Foo", "Bar");
-bundle1.assertAttributeEquals("Content-Type", "text/plain;
charset=ISO-8859-1");
+bundle1.assertAttributeEquals("Content-Type",
"text/plain;charset=iso-8859-1");
}
// Currently InvokeHttp does not support Proxy via Https
@@ -171,7 +171,7 @@ public class TestInvokeHTTP extends TestInvokeHttpCommon {
bundle1.assertAttributeEquals(InvokeHTTP.STATUS_CODE, "200");
bundle1.assertAttributeEquals(InvokeHTTP.STATUS_MESSAGE, "OK");
bundle1.assertAttributeEquals("Foo", "Bar");
-bundle1.assertAttributeEquals("Content-Type", "text/plain;
charset=ISO-8859-1");
+bundle1.assertAttributeEquals("Content-Type",
"text/plain;charset=iso-8859-1");
}
public static class MyProxyHandler extends AbstractHandler {
http://git-wip-us.apache.org/repos/asf/nifi/blob/4f101294/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/util/TestInvokeHttpCommon.java
--
diff --git
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/util/TestInvokeHttpCommon.java
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/util/TestInvokeHttpCommon.java
index d0f29b9..6e8468e 100644
---
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/util/TestInvokeHttpCommon.java
+++
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/util/TestInvokeHttpCommon.java
@@ -132,7 +132,7 @@ public abstract class TestInvokeHttpCommon {
bundle1.assertAttributeEquals(InvokeHTTP.STATUS_CODE, "200");
bundle1.assertAttributeEquals(InvokeHTTP.STATUS_MESSAGE, "OK");
bundle1.assertAttributeEquals("Foo", "Bar");
-bundle1.assertAttributeEquals("Content-Type", "text/plain;
charset=ISO-8859-1");
+bundle1.assertAttributeEquals("Content-Type",
"text/plain;charset=iso-8859-1");
final List provEvents =
runner.getProvenanceEvents();
assertEquals(2, provEvents.size());
@@ -185,7 +185,7 @@ public abstract class TestInvokeHttpCommon {
bundle1.assertAttributeEquals(InvokeHTTP.STATUS_CODE, "404");
bundle1.assertAttributeEquals(InvokeHTTP.STATUS_MESSAGE, "Not Found");
bundle1.assertAttributeEquals("Foo", "Bar");
-bundle1.assertAttributeEquals("Content-Type", "text/plain;
charset=ISO-8859-1");
+bundle1.assertAttributeEquals("Content-Type",
"text/plain;charset=iso-8859-1");
}
@Test
@@ -225,7 +225,7 @@ public abstract class TestInvokeHttpCommon {
bundle1.assertAttributeEquals(InvokeHTTP.STATUS_CODE, "404");
bundle1.assertAttributeEquals(InvokeHTTP.STATUS_MESSAGE, "Not Found");
bundle1.assertAttributeEquals("Foo", "Bar");
-
[jira] [Commented] (NIFI-1265) Precompiling message-page.jsp causes runtime error in CentOS 6.7 and Fedora 23
[
https://issues.apache.org/jira/browse/NIFI-1265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312322#comment-15312322
]
ASF subversion and git services commented on NIFI-1265:
---
Commit e587c998551d1652f480ab9f23d6a426b1068602 in nifi's branch
refs/heads/master from [~mcgilman]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=e587c99 ]
NIFI-1265:
- Upgrading to Jetty 9.3.
> Precompiling message-page.jsp causes runtime error in CentOS 6.7 and Fedora 23
> --
>
> Key: NIFI-1265
> URL: https://issues.apache.org/jira/browse/NIFI-1265
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Minor
> Fix For: 1.0.0
>
> Attachments: jetty.447816.patch
>
>
> During start up it appears that when message-page.jsp is precompiled and run
> in CentOS 6.7 or Fedora 23 the following method in ServletHolder (Jetty)
> fails with:
> {noformat}
> 2015-12-07 10:01:26,557 WARN [main] org.apache.nifi.web.server.JettyServer
> Failed to start web server... shutting down.
> java.lang.IllegalArgumentException: Comparison method violates its general
> contract!
> at java.util.ComparableTimSort.mergeHi(ComparableTimSort.java:866)
> ~[na:1.8.0_65]
> at java.util.ComparableTimSort.mergeAt(ComparableTimSort.java:483)
> ~[na:1.8.0_65]
> at
> java.util.ComparableTimSort.mergeForceCollapse(ComparableTimSort.java:422)
> ~[na:1.8.0_65]
> at java.util.ComparableTimSort.sort(ComparableTimSort.java:222)
> ~[na:1.8.0_65]
> at java.util.Arrays.sort(Arrays.java:1246) ~[na:1.8.0_65]
> at
> org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:865)
> ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
> at
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:298)
> ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
> at
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
> ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
> {noformat}
> {code}
> @Override
> public int compareTo(ServletHolder sh)
> {
> if (sh==this)
> return 0;
> if (sh._initOrder<_initOrder)
> return 1;
> if (sh._initOrder>_initOrder)
> return -1;
> int c=(_className!=null &&
> sh._className!=null)?_className.compareTo(sh._className):0;
> if (c==0)
> c=_name.compareTo(sh._name);
> return c;
> }
> {code}
> After a quick glance it may be that this method is not transitive when
> falling back to comparing by className and name. From the Java API
> {noformat}
> The implementor must also ensure that the relation is transitive:
> ((compare(x, y)>0) && (compare(y, z)>0)) implies compare(x, z)>0.
> {noformat}
> This could fail with the following scenario:
> {noformat}
> x._initOrder = -1
> x._className = null
> x._name = Login
> y._initOrder = -1
> y._className = org.apache.nifi.web.jsp.WEB_002dINF.pages.message_002dpage_jsp
> y._name = org.apache.nifi.web.jsp.WEB_002dINF.pages.message_002dpage_jsp
> z._initOrder = -1
> z._className = org.apache.nifi.web.servlet.DownloadSvg
> z._name = DownloadSvg
> {noformat}
> Unfortunately, I am unable to replicate the exception. However, my
> observation from debugging show that when JSPs are precompiled the _className
> and _name are the same (like y above). When a Servlet is referenced in the
> web.xml it's name is overridden (like z above). And when a JSP is referenced
> in the web.xml the _className is null and the _name is set to the name from
> the web.xml (like x above).
> In the scenario outlined: X < Y and Y < Z but Z < X. When running locally I
> did not see the exception mentioned above but the resulting sort order was
> not correct as it ultimately depended on which elements were compared to each
> other.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[3/3] nifi git commit: NIFI-1265: - Upgrading to Jetty 9.3.
NIFI-1265:
- Upgrading to Jetty 9.3.
Project: http://git-wip-us.apache.org/repos/asf/nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/e587c998
Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/e587c998
Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/e587c998
Branch: refs/heads/master
Commit: e587c998551d1652f480ab9f23d6a426b1068602
Parents: 3e06410
Author: Matt Gilman
Authored: Fri May 27 14:53:12 2016 -0400
Committer: Mark Payne
Committed: Thu Jun 2 09:49:57 2016 -0400
--
.../nifi-framework/nifi-web/nifi-jetty/pom.xml | 9 ++-
.../org/apache/nifi/web/server/JettyServer.java | 10 +++
.../nifi-web/nifi-web-content-viewer/pom.xml| 16 ---
.../nifi-web/nifi-web-docs/pom.xml | 16 ---
.../nifi/web/docs/DocumentationController.java | 17 +---
.../nifi-web/nifi-web-error/pom.xml | 16 ---
.../nifi-framework/nifi-web/nifi-web-ui/pom.xml | 16 ---
.../apache/nifi/web/servlet/DownloadSvg.java| 11
.../nifi-image-bundle/nifi-image-viewer/pom.xml | 12 -
.../apache/nifi/web/ImageViewerController.java | 7 ++---
nifi-nar-bundles/nifi-jetty-bundle/pom.xml | 20 --
.../nifi-standard-content-viewer/pom.xml| 16 ---
.../web/StandardContentViewerController.java| 10 +++
.../processors/standard/TestInvokeHTTP.java | 21 +++
.../nifi-update-attribute-ui/pom.xml| 16 ---
pom.xml | 28 ++--
16 files changed, 53 insertions(+), 188 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/nifi/blob/e587c998/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-jetty/pom.xml
--
diff --git
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-jetty/pom.xml
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-jetty/pom.xml
index 74322bb..ae5b94a 100644
---
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-jetty/pom.xml
+++
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-jetty/pom.xml
@@ -51,27 +51,22 @@
org.eclipse.jetty
jetty-server
-compile
org.eclipse.jetty
jetty-servlet
-compile
org.eclipse.jetty
jetty-webapp
-compile
org.eclipse.jetty
-jetty-servlets
-compile
+jetty-annotations
org.eclipse.jetty
-jetty-jsp
-compile
+jetty-servlets
com.sun.jersey
http://git-wip-us.apache.org/repos/asf/nifi/blob/e587c998/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-jetty/src/main/java/org/apache/nifi/web/server/JettyServer.java
--
diff --git
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-jetty/src/main/java/org/apache/nifi/web/server/JettyServer.java
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-jetty/src/main/java/org/apache/nifi/web/server/JettyServer.java
index a54f320..8be0a29 100644
---
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-jetty/src/main/java/org/apache/nifi/web/server/JettyServer.java
+++
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-jetty/src/main/java/org/apache/nifi/web/server/JettyServer.java
@@ -34,6 +34,7 @@ import org.apache.nifi.web.ContentAccess;
import org.apache.nifi.web.NiFiWebConfigurationContext;
import org.apache.nifi.web.NiFiWebContext;
import org.apache.nifi.web.UiExtensionType;
+import org.eclipse.jetty.annotations.AnnotationConfiguration;
import org.eclipse.jetty.server.Connector;
import org.eclipse.jetty.server.Handler;
import org.eclipse.jetty.server.HttpConfiguration;
@@ -51,6 +52,8 @@ import org.eclipse.jetty.util.resource.Resource;
import org.eclipse.jetty.util.resource.ResourceCollection;
import org.eclipse.jetty.util.ssl.SslContextFactory;
import org.eclipse.jetty.util.thread.QueuedThreadPool;
+import org.eclipse.jetty.webapp.Configuration;
+import org.eclipse.jetty.webapp.JettyWebXmlConfiguration;
import org.eclipse.jetty.webapp.WebAppClassLoader;
import org.eclipse.jetty.webapp.WebAppContext;
import org.slf4j.Logger;
@@ -133,6 +136,10 @@ public class JettyServer implements NiFiServer {
this.server = new Server(threadPool);
this.props = props;
+// enable the annotation based configuration to ensure the jsp
container is initialized properly
+final Configuration.ClassList classli
[jira] [Updated] (NIFI-1934) InferAvroSchema dynamically set record name
[ https://issues.apache.org/jira/browse/NIFI-1934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Kurc updated NIFI-1934: Assignee: Alex Halldin > InferAvroSchema dynamically set record name > --- > > Key: NIFI-1934 > URL: https://issues.apache.org/jira/browse/NIFI-1934 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Daniel Cave >Assignee: Alex Halldin >Priority: Minor > Fix For: 1.0.0, 0.7.0 > > > InferAvroSchema only allowed for hard set schema record names for the newly > generated schemas. Infer now allows these to be set dynamically based on the > record name from the incoming JSON. Infer now will also set a n > inferredAvroId onto the outgoing original JSON flowfile as well as the > outgoing schema flowfile. This is for matching in ConvertDynamicJsonToAvro -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[1/3] nifi git commit: NIFI-1265: - Upgrading the jspc maven plugin.
Repository: nifi
Updated Branches:
refs/heads/master 3e064101e -> 950e0cfa5
NIFI-1265:
- Upgrading the jspc maven plugin.
Project: http://git-wip-us.apache.org/repos/asf/nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/950e0cfa
Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/950e0cfa
Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/950e0cfa
Branch: refs/heads/master
Commit: 950e0cfa58269d9d64e799197a1833119bc71ddc
Parents: 4f10129
Author: Matt Gilman
Authored: Fri May 27 15:38:23 2016 -0400
Committer: Mark Payne
Committed: Thu Jun 2 09:49:57 2016 -0400
--
.../nifi-framework/nifi-web/nifi-web-ui/pom.xml| 6 ++
1 file changed, 2 insertions(+), 4 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/nifi/blob/950e0cfa/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/pom.xml
--
diff --git
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/pom.xml
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/pom.xml
index d8e970c..ffce69b 100644
---
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/pom.xml
+++
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/pom.xml
@@ -72,18 +72,16 @@
must exclude them here.
-->
-org.mortbay.jetty
+org.eclipse.jetty
jetty-jspc-maven-plugin
-8.1.10.v20130312
+${jetty.version}
jspc
-org.apache.nifi.web.jsp
true
-true
true
**/message-page.jsp,
[jira] [Commented] (NIFI-1934) InferAvroSchema dynamically set record name
[ https://issues.apache.org/jira/browse/NIFI-1934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312328#comment-15312328 ] Tony Kurc commented on NIFI-1934: - done > InferAvroSchema dynamically set record name > --- > > Key: NIFI-1934 > URL: https://issues.apache.org/jira/browse/NIFI-1934 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Daniel Cave >Assignee: Alex Halldin >Priority: Minor > Fix For: 1.0.0, 0.7.0 > > > InferAvroSchema only allowed for hard set schema record names for the newly > generated schemas. Infer now allows these to be set dynamically based on the > record name from the incoming JSON. Infer now will also set a n > inferredAvroId onto the outgoing original JSON flowfile as well as the > outgoing schema flowfile. This is for matching in ConvertDynamicJsonToAvro -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1265) Precompiling message-page.jsp causes runtime error in CentOS 6.7 and Fedora 23
[
https://issues.apache.org/jira/browse/NIFI-1265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312324#comment-15312324
]
ASF subversion and git services commented on NIFI-1265:
---
Commit 950e0cfa58269d9d64e799197a1833119bc71ddc in nifi's branch
refs/heads/master from [~mcgilman]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=950e0cf ]
NIFI-1265:
- Upgrading the jspc maven plugin.
> Precompiling message-page.jsp causes runtime error in CentOS 6.7 and Fedora 23
> --
>
> Key: NIFI-1265
> URL: https://issues.apache.org/jira/browse/NIFI-1265
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Minor
> Fix For: 1.0.0
>
> Attachments: jetty.447816.patch
>
>
> During start up it appears that when message-page.jsp is precompiled and run
> in CentOS 6.7 or Fedora 23 the following method in ServletHolder (Jetty)
> fails with:
> {noformat}
> 2015-12-07 10:01:26,557 WARN [main] org.apache.nifi.web.server.JettyServer
> Failed to start web server... shutting down.
> java.lang.IllegalArgumentException: Comparison method violates its general
> contract!
> at java.util.ComparableTimSort.mergeHi(ComparableTimSort.java:866)
> ~[na:1.8.0_65]
> at java.util.ComparableTimSort.mergeAt(ComparableTimSort.java:483)
> ~[na:1.8.0_65]
> at
> java.util.ComparableTimSort.mergeForceCollapse(ComparableTimSort.java:422)
> ~[na:1.8.0_65]
> at java.util.ComparableTimSort.sort(ComparableTimSort.java:222)
> ~[na:1.8.0_65]
> at java.util.Arrays.sort(Arrays.java:1246) ~[na:1.8.0_65]
> at
> org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:865)
> ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
> at
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:298)
> ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
> at
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
> ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
> {noformat}
> {code}
> @Override
> public int compareTo(ServletHolder sh)
> {
> if (sh==this)
> return 0;
> if (sh._initOrder<_initOrder)
> return 1;
> if (sh._initOrder>_initOrder)
> return -1;
> int c=(_className!=null &&
> sh._className!=null)?_className.compareTo(sh._className):0;
> if (c==0)
> c=_name.compareTo(sh._name);
> return c;
> }
> {code}
> After a quick glance it may be that this method is not transitive when
> falling back to comparing by className and name. From the Java API
> {noformat}
> The implementor must also ensure that the relation is transitive:
> ((compare(x, y)>0) && (compare(y, z)>0)) implies compare(x, z)>0.
> {noformat}
> This could fail with the following scenario:
> {noformat}
> x._initOrder = -1
> x._className = null
> x._name = Login
> y._initOrder = -1
> y._className = org.apache.nifi.web.jsp.WEB_002dINF.pages.message_002dpage_jsp
> y._name = org.apache.nifi.web.jsp.WEB_002dINF.pages.message_002dpage_jsp
> z._initOrder = -1
> z._className = org.apache.nifi.web.servlet.DownloadSvg
> z._name = DownloadSvg
> {noformat}
> Unfortunately, I am unable to replicate the exception. However, my
> observation from debugging show that when JSPs are precompiled the _className
> and _name are the same (like y above). When a Servlet is referenced in the
> web.xml it's name is overridden (like z above). And when a JSP is referenced
> in the web.xml the _className is null and the _name is set to the name from
> the web.xml (like x above).
> In the scenario outlined: X < Y and Y < Z but Z < X. When running locally I
> did not see the exception mentioned above but the resulting sort order was
> not correct as it ultimately depended on which elements were compared to each
> other.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-1937) GetHTTP should support configurable cookie policy
[ https://issues.apache.org/jira/browse/NIFI-1937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312341#comment-15312341 ] ASF GitHub Bot commented on NIFI-1937: -- Github user trkurc commented on the issue: https://github.com/apache/nifi/pull/479 I'll review this one. > GetHTTP should support configurable cookie policy > - > > Key: NIFI-1937 > URL: https://issues.apache.org/jira/browse/NIFI-1937 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 0.6.1 >Reporter: Michael Moser >Assignee: Michael Moser >Priority: Minor > Fix For: 0.7.0 > > > After changes to GetHTTP in NIFI-1714, I found a corporate web site where > GetHTTP fails to download content. GetHTTP could successfully download > content from this site before NIFI-1714 was implemented. So that change > effectively broke access to this site. > I propose we add a new property to GetHTTP that allows the NiFi user to > choose the HTTPClient (Apache HTTPComponents) cookie policy. The property > would be called Redirect Cookie Policy which would be "When a HTTP server > responds to a request with a redirect, this is the cookie specification used > to copy cookies to the following request" -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1943) PutFile should have ReadsAttribute annotation for "filename"
[ https://issues.apache.org/jira/browse/NIFI-1943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312351#comment-15312351 ] ASF GitHub Bot commented on NIFI-1943: -- Github user JPercivall commented on the issue: https://github.com/apache/nifi/pull/480 Thanks @trkurc, should be quite simple > PutFile should have ReadsAttribute annotation for "filename" > > > Key: NIFI-1943 > URL: https://issues.apache.org/jira/browse/NIFI-1943 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joseph Percivall >Assignee: Joseph Percivall >Priority: Minor > Fix For: 0.7.0 > > > The PutFile processor writes the flowfile to disk using the value contained > in the "filename" attribute. There should be documentation telling the user > this. > All that needs to be done is to add a "ReadsAttribute" annotation for this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1815) Tesseract OCR Processor
[
https://issues.apache.org/jira/browse/NIFI-1815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312382#comment-15312382
]
ASF GitHub Bot commented on NIFI-1815:
--
Github user olegz commented on a diff in the pull request:
https://github.com/apache/nifi/pull/397#discussion_r65548155
--- Diff:
nifi-nar-bundles/nifi-ocr-bundle/nifi-ocr-processors/src/main/java/org/apache/nifi/processors/ocr/TesseractOCRProcessor.java
---
@@ -0,0 +1,355 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.ocr;
+
+import java.awt.image.BufferedImage;
+import java.io.File;
+import java.io.FileFilter;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+
+import javax.imageio.ImageIO;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.StreamCallback;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import net.sourceforge.tess4j.ITesseract;
+import net.sourceforge.tess4j.Tesseract;
+import net.sourceforge.tess4j.TesseractException;
+
+@Tags({"ocr", "tesseract", "image", "text"})
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@CapabilityDescription("Extracts text from images using Optical Character
Recognition (OCR). The images are pulled from the incoming" +
+" Flowfile's content. Supported image types are TIFF, JPEG, GIF,
PNG, BMP, and PDF. Any Flowfile that doesn't contain" +
+" a supported image type in its content body will be routed to the
'unsupported image format' relationship and no OCR will be performed." +
+" This processor uses Tesseract to perform its duties and part of
that requires that a valid Tesseract data (Tessdata) directory" +
+" be specified in the 'Tessdata Directory' Property. This
processor considers a valid Tessdata directory to be an existing directory on
the" +
+" local NiFi instance that contains one or more files ending with
the '.traineddata' extension. The list of supported languages" +
+" is built from the Tessdata directory configured by listing all
files ending with '.traineddata' and considering those" +
+" Tesseract language models. You can create you own Tesseract
language models and place them in your Tessedata directory" +
+" and the processor will display it in the dropdown list of
languages available. All valid Tesseract configuration values" +
+" may be passed to this processor by use of the 'Tesseract
configuration values' which accepts a comma separated list" +
+" of key=value pairs representing Tesseract configurations.
'Tesseract configuration values' is where all of your tuning" +
+" values can be passed in to help increase the accuracy of your
OCR operations based on your expe
[jira] [Commented] (NIFI-1815) Tesseract OCR Processor
[ https://issues.apache.org/jira/browse/NIFI-1815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312389#comment-15312389 ] ASF GitHub Bot commented on NIFI-1815: -- Github user olegz commented on the issue: https://github.com/apache/nifi/pull/397 @jdye64 I left one last comment. Once addressed I'll begin the merge procedure. I will also take care of LICENSE and NOTICE > Tesseract OCR Processor > --- > > Key: NIFI-1815 > URL: https://issues.apache.org/jira/browse/NIFI-1815 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Jeremy Dyer >Assignee: Jeremy Dyer > Attachments: 0006-changes-to-the-OCR-processor.patch, > nifi_1815_1.x_patch.zip > > > This ticket is a follow-up to NIFI-1718 minus the use of the Tika library > Expose OCR capabilities through a new processor which uses the Tesseract > library. Use of this processor would require that Tesseract be installed on > the NiFi host. Since the processor will have a system dependency care must be > taken to ensure that the overall NiFi cluster continues to function properly > in the absence of the Tesseract system dependency even though the OCR > processor itself will be unable to perform its duties. In the event that the > system dependencies are not detected the processor should display a > validation warning rather than failing or preventing the NiFi instance from > booting properly. > Properties expose to configure Tesseract > tesseractPath - Path to tesseract installation folder, if not on system path. > language - Language ID (e.g. "eng"); language dictionary to be used. > pageSegMode - Tesseract page segmentation mode, defaults to 1. > minFileSizeToOcr - Minimum file size to submit file to OCR, defaults to 0. > maxFileSizeToOcr - Maximum file size to submit file to OCR, defaults to > Integer.MAX_VALUE. > timeout - Maximum time (in seconds) to wait for the OCR process termination; > defaults to 120. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1934) InferAvroSchema dynamically set record name
[ https://issues.apache.org/jira/browse/NIFI-1934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Halldin updated NIFI-1934: --- Attachment: 0001-NIFI-1934-Updates-to-InferAvroSchema-made-record-nam.patch > InferAvroSchema dynamically set record name > --- > > Key: NIFI-1934 > URL: https://issues.apache.org/jira/browse/NIFI-1934 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.0.0, 0.7.0 >Reporter: Daniel Cave >Assignee: Alex Halldin >Priority: Minor > Fix For: 1.0.0, 0.7.0 > > Attachments: > 0001-NIFI-1934-Updates-to-InferAvroSchema-made-record-nam.patch > > > InferAvroSchema only allowed for hard set schema record names for the newly > generated schemas. Infer now allows these to be set dynamically based on the > record name from the incoming JSON. Infer now will also set a n > inferredAvroId onto the outgoing original JSON flowfile as well as the > outgoing schema flowfile. This is for matching in ConvertDynamicJsonToAvro -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1933) SplitJson enhancements for flowfile names and additional options
[ https://issues.apache.org/jira/browse/NIFI-1933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Halldin updated NIFI-1933: --- Attachment: 0001-NIFI-1933-Updates-to-SplitJSON.java-now-can-split-on.patch > SplitJson enhancements for flowfile names and additional options > > > Key: NIFI-1933 > URL: https://issues.apache.org/jira/browse/NIFI-1933 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.0.0, 0.7.0 >Reporter: Daniel Cave >Priority: Minor > Fix For: 1.0.0, 0.7.0 > > Attachments: > 0001-NIFI-1933-Updates-to-SplitJSON.java-now-can-split-on.patch > > > 1) SplitJson output flowfiles are replicating the parent filename across all > children. This results in overwriting of failure if trying to write the > files without later renaming in another processor (that may or may not result > in meaningful names). SplitJson will now rename children with an appended > index number to the parent: -.json > 2) SplitJson now has the option to include the split upon element as the new > root element of each split. This configurable and disabled by default to > conform with backwards compatibility. > 3) SplitJson now will correctly handle single element splits and no longer > simply fail due to the element not being an array. Also configurable and > disabled by default. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1935) Added ConvertDynamicJsonToAvro processor
[ https://issues.apache.org/jira/browse/NIFI-1935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Halldin updated NIFI-1935: --- Attachment: 0001-NIFI-1935-Added-ConvertDynamicJSONToAvro.java.-Added.patch > Added ConvertDynamicJsonToAvro processor > > > Key: NIFI-1935 > URL: https://issues.apache.org/jira/browse/NIFI-1935 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Affects Versions: 1.0.0, 0.7.0 >Reporter: Daniel Cave >Assignee: Alex Halldin >Priority: Minor > Fix For: 1.0.0, 0.7.0 > > Attachments: > 0001-NIFI-1935-Added-ConvertDynamicJSONToAvro.java.-Added.patch > > > ConvertJsonToAvro required a predefined Avro schema to convert JSON and > required the presence of all field on the incoming JSON. > ConvertDynamicJsonToAvro functions similarly, however it now accepts the JSON > and schema as incoming flowfiles and creates the Avro dynamically. > This processor requires the InferAvroSchema processor in its upstream flow so > that it can use the original and schema flowfiles as input. These two > flowfiles will have the unique attribute inferredAvroId set on them by > InferAvroSchema so that they can be properly matched in > ConvertDynamicJsonToAvro. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1899) Create ListenSMTP & ExtractEmailAttachment processors
[ https://issues.apache.org/jira/browse/NIFI-1899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312432#comment-15312432 ] ASF GitHub Bot commented on NIFI-1899: -- Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/483 @joewitt Second processor should be ready, will work on harness later today. > Create ListenSMTP & ExtractEmailAttachment processors > - > > Key: NIFI-1899 > URL: https://issues.apache.org/jira/browse/NIFI-1899 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Andre > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1899) Create ListenSMTP & ExtractEmailAttachment processors
[ https://issues.apache.org/jira/browse/NIFI-1899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312439#comment-15312439 ] ASF GitHub Bot commented on NIFI-1899: -- Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/483 Sounds quite good to me @trixpan . I would just not worry about the additional attributes - yet. You identified good standard ones and if we find the need for more then we can iterate. You agree? > Create ListenSMTP & ExtractEmailAttachment processors > - > > Key: NIFI-1899 > URL: https://issues.apache.org/jira/browse/NIFI-1899 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Andre > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1928) UI - Remove registration process
[ https://issues.apache.org/jira/browse/NIFI-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312527#comment-15312527 ] ASF GitHub Bot commented on NIFI-1928: -- Github user bbende commented on the issue: https://github.com/apache/nifi/pull/482 +1 looks good, verified registration form no longer exists and access denied page is presented instead > UI - Remove registration process > > > Key: NIFI-1928 > URL: https://issues.apache.org/jira/browse/NIFI-1928 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman > Fix For: 1.0.0 > > > With the updated user authorization model, users will no longer be able to > request an account. Instead, users are authorized through the configured > Authorizer implementation (it's pluggable). Consequently, we can remove all > the account registration forms from the UI and OBE REST endpoints. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
nifi git commit: NIFI-1928: - Fixing UI action for bulk delete. - Removing registration form. - Fixing default visibility of the anonymous user warning.
Repository: nifi
Updated Branches:
refs/heads/master 950e0cfa5 -> f0f74fe40
NIFI-1928:
- Fixing UI action for bulk delete.
- Removing registration form.
- Fixing default visibility of the anonymous user warning.
Project: http://git-wip-us.apache.org/repos/asf/nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/f0f74fe4
Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/f0f74fe4
Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/f0f74fe4
Branch: refs/heads/master
Commit: f0f74fe4049f38a8c022c4bf47494b12f9ce7174
Parents: 950e0cf
Author: Matt Gilman
Authored: Wed Jun 1 08:09:04 2016 -0400
Committer: Matt Gilman
Committed: Thu Jun 2 11:56:10 2016 -0400
--
.../web/api/dto/AccessConfigurationDTO.java | 16 --
.../nifi/web/api/dto/AccessStatusDTO.java | 2 -
.../nifi/web/api/entity/IdentityEntity.java | 11 ++
.../org/apache/nifi/web/api/AccessResource.java | 170 +-
.../apache/nifi/web/api/ControllerResource.java | 50 --
.../org/apache/nifi/web/api/FlowResource.java | 171 ---
.../api/config/AccessDeniedExceptionMapper.java | 26 ++-
.../src/main/resources/nifi-web-api-context.xml | 7 +-
.../accesscontrol/AccessTokenEndpointTest.java | 29 +---
.../nifi/integration/util/NiFiTestUser.java | 32 +---
.../anonymous/NiFiAnonymousUserFilter.java | 8 -
.../src/main/webapp/WEB-INF/pages/login.jsp | 3 +-
.../WEB-INF/partials/canvas/canvas-header.jsp | 4 +-
.../partials/login/nifi-registration-form.jsp | 38 -
.../webapp/WEB-INF/partials/message-pane.jsp| 2 +-
.../nifi-web-ui/src/main/webapp/css/header.css | 1 +
.../nifi-web-ui/src/main/webapp/css/login.css | 16 --
.../src/main/webapp/js/nf/canvas/nf-canvas.js | 10 +-
.../src/main/webapp/js/nf/cluster/nf-cluster.js | 2 +-
.../main/webapp/js/nf/counters/nf-counters.js | 2 +-
.../src/main/webapp/js/nf/history/nf-history.js | 2 +-
.../src/main/webapp/js/nf/login/nf-login.js | 114 ++---
.../webapp/js/nf/provenance/nf-provenance.js| 2 +-
.../main/webapp/js/nf/templates/nf-templates.js | 2 +-
24 files changed, 269 insertions(+), 451 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/nifi/blob/f0f74fe4/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-client-dto/src/main/java/org/apache/nifi/web/api/dto/AccessConfigurationDTO.java
--
diff --git
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-client-dto/src/main/java/org/apache/nifi/web/api/dto/AccessConfigurationDTO.java
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-client-dto/src/main/java/org/apache/nifi/web/api/dto/AccessConfigurationDTO.java
index d9719b3..3c15912 100644
---
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-client-dto/src/main/java/org/apache/nifi/web/api/dto/AccessConfigurationDTO.java
+++
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-client-dto/src/main/java/org/apache/nifi/web/api/dto/AccessConfigurationDTO.java
@@ -26,7 +26,6 @@ import javax.xml.bind.annotation.XmlType;
public class AccessConfigurationDTO {
private Boolean supportsLogin;
-private Boolean supportsAnonymous;
/**
* @return Indicates whether or not this NiFi supports user login.
@@ -43,19 +42,4 @@ public class AccessConfigurationDTO {
this.supportsLogin = supportsLogin;
}
-/**
- * @return Indicates whether or not this NiFi supports anonymous access.
- */
-@ApiModelProperty(
-value = "Indicates whether or not this NiFi supports anonymous.",
-readOnly = true
-)
-public Boolean getSupportsAnonymous() {
-return supportsAnonymous;
-}
-
-public void setSupportsAnonymous(Boolean supportsAnonymous) {
-this.supportsAnonymous = supportsAnonymous;
-}
-
}
http://git-wip-us.apache.org/repos/asf/nifi/blob/f0f74fe4/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-client-dto/src/main/java/org/apache/nifi/web/api/dto/AccessStatusDTO.java
--
diff --git
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-client-dto/src/main/java/org/apache/nifi/web/api/dto/AccessStatusDTO.java
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-client-dto/src/main/java/org/apache/nifi/web/api/dto/AccessStatusDTO.java
index 712da0e..5962cc2 100644
---
a/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-client-dto/src/main/java/org/apache/nifi/web/api/dto/AccessStatusDTO.java
+++
b/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-client-dto/src/main/java/org/apache/nifi/web/api/dto/AccessStatusDTO.java
@@ -28,8 +28,6 @@ public class AccessStatusDTO {
public static enum Status {
[jira] [Commented] (NIFI-1928) UI - Remove registration process
[ https://issues.apache.org/jira/browse/NIFI-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312528#comment-15312528 ] ASF subversion and git services commented on NIFI-1928: --- Commit f0f74fe4049f38a8c022c4bf47494b12f9ce7174 in nifi's branch refs/heads/master from [~mcgilman] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=f0f74fe ] NIFI-1928: - Fixing UI action for bulk delete. - Removing registration form. - Fixing default visibility of the anonymous user warning. > UI - Remove registration process > > > Key: NIFI-1928 > URL: https://issues.apache.org/jira/browse/NIFI-1928 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman > Fix For: 1.0.0 > > > With the updated user authorization model, users will no longer be able to > request an account. Instead, users are authorized through the configured > Authorizer implementation (it's pluggable). Consequently, we can remove all > the account registration forms from the UI and OBE REST endpoints. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1265) Precompiling message-page.jsp causes runtime error in CentOS 6.7 and Fedora 23
[
https://issues.apache.org/jira/browse/NIFI-1265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312532#comment-15312532
]
ASF GitHub Bot commented on NIFI-1265:
--
Github user markap14 commented on the issue:
https://github.com/apache/nifi/pull/472
@mcgilman I merged this to master but forgot to update the commit message.
Can you close the PR?
> Precompiling message-page.jsp causes runtime error in CentOS 6.7 and Fedora 23
> --
>
> Key: NIFI-1265
> URL: https://issues.apache.org/jira/browse/NIFI-1265
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Minor
> Fix For: 1.0.0
>
> Attachments: jetty.447816.patch
>
>
> During start up it appears that when message-page.jsp is precompiled and run
> in CentOS 6.7 or Fedora 23 the following method in ServletHolder (Jetty)
> fails with:
> {noformat}
> 2015-12-07 10:01:26,557 WARN [main] org.apache.nifi.web.server.JettyServer
> Failed to start web server... shutting down.
> java.lang.IllegalArgumentException: Comparison method violates its general
> contract!
> at java.util.ComparableTimSort.mergeHi(ComparableTimSort.java:866)
> ~[na:1.8.0_65]
> at java.util.ComparableTimSort.mergeAt(ComparableTimSort.java:483)
> ~[na:1.8.0_65]
> at
> java.util.ComparableTimSort.mergeForceCollapse(ComparableTimSort.java:422)
> ~[na:1.8.0_65]
> at java.util.ComparableTimSort.sort(ComparableTimSort.java:222)
> ~[na:1.8.0_65]
> at java.util.Arrays.sort(Arrays.java:1246) ~[na:1.8.0_65]
> at
> org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:865)
> ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
> at
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:298)
> ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
> at
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
> ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
> {noformat}
> {code}
> @Override
> public int compareTo(ServletHolder sh)
> {
> if (sh==this)
> return 0;
> if (sh._initOrder<_initOrder)
> return 1;
> if (sh._initOrder>_initOrder)
> return -1;
> int c=(_className!=null &&
> sh._className!=null)?_className.compareTo(sh._className):0;
> if (c==0)
> c=_name.compareTo(sh._name);
> return c;
> }
> {code}
> After a quick glance it may be that this method is not transitive when
> falling back to comparing by className and name. From the Java API
> {noformat}
> The implementor must also ensure that the relation is transitive:
> ((compare(x, y)>0) && (compare(y, z)>0)) implies compare(x, z)>0.
> {noformat}
> This could fail with the following scenario:
> {noformat}
> x._initOrder = -1
> x._className = null
> x._name = Login
> y._initOrder = -1
> y._className = org.apache.nifi.web.jsp.WEB_002dINF.pages.message_002dpage_jsp
> y._name = org.apache.nifi.web.jsp.WEB_002dINF.pages.message_002dpage_jsp
> z._initOrder = -1
> z._className = org.apache.nifi.web.servlet.DownloadSvg
> z._name = DownloadSvg
> {noformat}
> Unfortunately, I am unable to replicate the exception. However, my
> observation from debugging show that when JSPs are precompiled the _className
> and _name are the same (like y above). When a Servlet is referenced in the
> web.xml it's name is overridden (like z above). And when a JSP is referenced
> in the web.xml the _className is null and the _name is set to the name from
> the web.xml (like x above).
> In the scenario outlined: X < Y and Y < Z but Z < X. When running locally I
> did not see the exception mentioned above but the resulting sort order was
> not correct as it ultimately depended on which elements were compared to each
> other.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-1265) Precompiling message-page.jsp causes runtime error in CentOS 6.7 and Fedora 23
[
https://issues.apache.org/jira/browse/NIFI-1265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312533#comment-15312533
]
ASF GitHub Bot commented on NIFI-1265:
--
Github user mcgilman commented on the issue:
https://github.com/apache/nifi/pull/472
Will do. Thanks!
> Precompiling message-page.jsp causes runtime error in CentOS 6.7 and Fedora 23
> --
>
> Key: NIFI-1265
> URL: https://issues.apache.org/jira/browse/NIFI-1265
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Minor
> Fix For: 1.0.0
>
> Attachments: jetty.447816.patch
>
>
> During start up it appears that when message-page.jsp is precompiled and run
> in CentOS 6.7 or Fedora 23 the following method in ServletHolder (Jetty)
> fails with:
> {noformat}
> 2015-12-07 10:01:26,557 WARN [main] org.apache.nifi.web.server.JettyServer
> Failed to start web server... shutting down.
> java.lang.IllegalArgumentException: Comparison method violates its general
> contract!
> at java.util.ComparableTimSort.mergeHi(ComparableTimSort.java:866)
> ~[na:1.8.0_65]
> at java.util.ComparableTimSort.mergeAt(ComparableTimSort.java:483)
> ~[na:1.8.0_65]
> at
> java.util.ComparableTimSort.mergeForceCollapse(ComparableTimSort.java:422)
> ~[na:1.8.0_65]
> at java.util.ComparableTimSort.sort(ComparableTimSort.java:222)
> ~[na:1.8.0_65]
> at java.util.Arrays.sort(Arrays.java:1246) ~[na:1.8.0_65]
> at
> org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:865)
> ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
> at
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:298)
> ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
> at
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
> ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
> {noformat}
> {code}
> @Override
> public int compareTo(ServletHolder sh)
> {
> if (sh==this)
> return 0;
> if (sh._initOrder<_initOrder)
> return 1;
> if (sh._initOrder>_initOrder)
> return -1;
> int c=(_className!=null &&
> sh._className!=null)?_className.compareTo(sh._className):0;
> if (c==0)
> c=_name.compareTo(sh._name);
> return c;
> }
> {code}
> After a quick glance it may be that this method is not transitive when
> falling back to comparing by className and name. From the Java API
> {noformat}
> The implementor must also ensure that the relation is transitive:
> ((compare(x, y)>0) && (compare(y, z)>0)) implies compare(x, z)>0.
> {noformat}
> This could fail with the following scenario:
> {noformat}
> x._initOrder = -1
> x._className = null
> x._name = Login
> y._initOrder = -1
> y._className = org.apache.nifi.web.jsp.WEB_002dINF.pages.message_002dpage_jsp
> y._name = org.apache.nifi.web.jsp.WEB_002dINF.pages.message_002dpage_jsp
> z._initOrder = -1
> z._className = org.apache.nifi.web.servlet.DownloadSvg
> z._name = DownloadSvg
> {noformat}
> Unfortunately, I am unable to replicate the exception. However, my
> observation from debugging show that when JSPs are precompiled the _className
> and _name are the same (like y above). When a Servlet is referenced in the
> web.xml it's name is overridden (like z above). And when a JSP is referenced
> in the web.xml the _className is null and the _name is set to the name from
> the web.xml (like x above).
> In the scenario outlined: X < Y and Y < Z but Z < X. When running locally I
> did not see the exception mentioned above but the resulting sort order was
> not correct as it ultimately depended on which elements were compared to each
> other.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-1265) Precompiling message-page.jsp causes runtime error in CentOS 6.7 and Fedora 23
[
https://issues.apache.org/jira/browse/NIFI-1265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312534#comment-15312534
]
ASF GitHub Bot commented on NIFI-1265:
--
Github user mcgilman closed the pull request at:
https://github.com/apache/nifi/pull/472
> Precompiling message-page.jsp causes runtime error in CentOS 6.7 and Fedora 23
> --
>
> Key: NIFI-1265
> URL: https://issues.apache.org/jira/browse/NIFI-1265
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core UI
>Reporter: Matt Gilman
>Assignee: Matt Gilman
>Priority: Minor
> Fix For: 1.0.0
>
> Attachments: jetty.447816.patch
>
>
> During start up it appears that when message-page.jsp is precompiled and run
> in CentOS 6.7 or Fedora 23 the following method in ServletHolder (Jetty)
> fails with:
> {noformat}
> 2015-12-07 10:01:26,557 WARN [main] org.apache.nifi.web.server.JettyServer
> Failed to start web server... shutting down.
> java.lang.IllegalArgumentException: Comparison method violates its general
> contract!
> at java.util.ComparableTimSort.mergeHi(ComparableTimSort.java:866)
> ~[na:1.8.0_65]
> at java.util.ComparableTimSort.mergeAt(ComparableTimSort.java:483)
> ~[na:1.8.0_65]
> at
> java.util.ComparableTimSort.mergeForceCollapse(ComparableTimSort.java:422)
> ~[na:1.8.0_65]
> at java.util.ComparableTimSort.sort(ComparableTimSort.java:222)
> ~[na:1.8.0_65]
> at java.util.Arrays.sort(Arrays.java:1246) ~[na:1.8.0_65]
> at
> org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:865)
> ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
> at
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:298)
> ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
> at
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
> ~[jetty-webapp-9.2.11.v20150529.jar:9.2.11.v20150529]
> {noformat}
> {code}
> @Override
> public int compareTo(ServletHolder sh)
> {
> if (sh==this)
> return 0;
> if (sh._initOrder<_initOrder)
> return 1;
> if (sh._initOrder>_initOrder)
> return -1;
> int c=(_className!=null &&
> sh._className!=null)?_className.compareTo(sh._className):0;
> if (c==0)
> c=_name.compareTo(sh._name);
> return c;
> }
> {code}
> After a quick glance it may be that this method is not transitive when
> falling back to comparing by className and name. From the Java API
> {noformat}
> The implementor must also ensure that the relation is transitive:
> ((compare(x, y)>0) && (compare(y, z)>0)) implies compare(x, z)>0.
> {noformat}
> This could fail with the following scenario:
> {noformat}
> x._initOrder = -1
> x._className = null
> x._name = Login
> y._initOrder = -1
> y._className = org.apache.nifi.web.jsp.WEB_002dINF.pages.message_002dpage_jsp
> y._name = org.apache.nifi.web.jsp.WEB_002dINF.pages.message_002dpage_jsp
> z._initOrder = -1
> z._className = org.apache.nifi.web.servlet.DownloadSvg
> z._name = DownloadSvg
> {noformat}
> Unfortunately, I am unable to replicate the exception. However, my
> observation from debugging show that when JSPs are precompiled the _className
> and _name are the same (like y above). When a Servlet is referenced in the
> web.xml it's name is overridden (like z above). And when a JSP is referenced
> in the web.xml the _className is null and the _name is set to the name from
> the web.xml (like x above).
> In the scenario outlined: X < Y and Y < Z but Z < X. When running locally I
> did not see the exception mentioned above but the resulting sort order was
> not correct as it ultimately depended on which elements were compared to each
> other.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (NIFI-1928) UI - Remove registration process
[ https://issues.apache.org/jira/browse/NIFI-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman resolved NIFI-1928. --- Resolution: Fixed > UI - Remove registration process > > > Key: NIFI-1928 > URL: https://issues.apache.org/jira/browse/NIFI-1928 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman > Fix For: 1.0.0 > > > With the updated user authorization model, users will no longer be able to > request an account. Instead, users are authorized through the configured > Authorizer implementation (it's pluggable). Consequently, we can remove all > the account registration forms from the UI and OBE REST endpoints. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1943) PutFile should have ReadsAttribute annotation for "filename"
[ https://issues.apache.org/jira/browse/NIFI-1943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312547#comment-15312547 ] ASF GitHub Bot commented on NIFI-1943: -- Github user trkurc commented on the issue: https://github.com/apache/nifi/pull/480 @JPercivall - I'm +1 on this. Would you like to merge it in, or prefer I do? > PutFile should have ReadsAttribute annotation for "filename" > > > Key: NIFI-1943 > URL: https://issues.apache.org/jira/browse/NIFI-1943 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joseph Percivall >Assignee: Joseph Percivall >Priority: Minor > Fix For: 0.7.0 > > > The PutFile processor writes the flowfile to disk using the value contained > in the "filename" attribute. There should be documentation telling the user > this. > All that needs to be done is to add a "ReadsAttribute" annotation for this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1943) PutFile should have ReadsAttribute annotation for "filename"
[ https://issues.apache.org/jira/browse/NIFI-1943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312549#comment-15312549 ] ASF GitHub Bot commented on NIFI-1943: -- Github user JPercivall commented on the issue: https://github.com/apache/nifi/pull/480 @trkurc I can take care of it, thanks! > PutFile should have ReadsAttribute annotation for "filename" > > > Key: NIFI-1943 > URL: https://issues.apache.org/jira/browse/NIFI-1943 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joseph Percivall >Assignee: Joseph Percivall >Priority: Minor > Fix For: 0.7.0 > > > The PutFile processor writes the flowfile to disk using the value contained > in the "filename" attribute. There should be documentation telling the user > this. > All that needs to be done is to add a "ReadsAttribute" annotation for this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1943) PutFile should have ReadsAttribute annotation for "filename"
[ https://issues.apache.org/jira/browse/NIFI-1943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312671#comment-15312671 ] ASF subversion and git services commented on NIFI-1943: --- Commit d01f2d910e0f1e457921c11f99ce5a91ada0d242 in nifi's branch refs/heads/0.x from [~JPercivall] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=d01f2d9 ] NIFI-1943 added proper ReadsAttribute to PutFile This closes #480 > PutFile should have ReadsAttribute annotation for "filename" > > > Key: NIFI-1943 > URL: https://issues.apache.org/jira/browse/NIFI-1943 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joseph Percivall >Assignee: Joseph Percivall >Priority: Minor > Fix For: 0.7.0 > > > The PutFile processor writes the flowfile to disk using the value contained > in the "filename" attribute. There should be documentation telling the user > this. > All that needs to be done is to add a "ReadsAttribute" annotation for this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
nifi git commit: NIFI-1943 added proper ReadsAttribute to PutFile
Repository: nifi
Updated Branches:
refs/heads/0.x 43d00b90f -> d01f2d910
NIFI-1943 added proper ReadsAttribute to PutFile
This closes #480
Project: http://git-wip-us.apache.org/repos/asf/nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/d01f2d91
Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/d01f2d91
Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/d01f2d91
Branch: refs/heads/0.x
Commit: d01f2d910e0f1e457921c11f99ce5a91ada0d242
Parents: 43d00b9
Author: jpercivall
Authored: Tue May 31 10:01:07 2016 -0400
Committer: jpercivall
Committed: Thu Jun 2 13:03:41 2016 -0400
--
.../src/main/java/org/apache/nifi/processors/standard/PutFile.java | 2 ++
1 file changed, 2 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/nifi/blob/d01f2d91/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/PutFile.java
--
diff --git
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/PutFile.java
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/PutFile.java
index 5b819b3..fcde107 100644
---
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/PutFile.java
+++
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/PutFile.java
@@ -36,6 +36,7 @@ import java.util.regex.Pattern;
import org.apache.nifi.annotation.behavior.InputRequirement;
import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.ReadsAttribute;
import org.apache.nifi.annotation.behavior.SupportsBatching;
import org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.annotation.documentation.SeeAlso;
@@ -58,6 +59,7 @@ import org.apache.nifi.util.StopWatch;
@Tags({"put", "local", "copy", "archive", "files", "filesystem"})
@CapabilityDescription("Writes the contents of a FlowFile to the local file
system")
@SeeAlso({FetchFile.class, GetFile.class})
+@ReadsAttribute(attribute = "filename", description = "The filename to use
when writing the FlowFile to disk.")
public class PutFile extends AbstractProcessor {
public static final String REPLACE_RESOLUTION = "replace";
[jira] [Commented] (NIFI-1928) UI - Remove registration process
[ https://issues.apache.org/jira/browse/NIFI-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312681#comment-15312681 ] ASF GitHub Bot commented on NIFI-1928: -- Github user mcgilman closed the pull request at: https://github.com/apache/nifi/pull/482 > UI - Remove registration process > > > Key: NIFI-1928 > URL: https://issues.apache.org/jira/browse/NIFI-1928 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman > Fix For: 1.0.0 > > > With the updated user authorization model, users will no longer be able to > request an account. Instead, users are authorized through the configured > Authorizer implementation (it's pluggable). Consequently, we can remove all > the account registration forms from the UI and OBE REST endpoints. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1928) UI - Remove registration process
[ https://issues.apache.org/jira/browse/NIFI-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312680#comment-15312680 ] ASF GitHub Bot commented on NIFI-1928: -- Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/482 This has been merged into master. Thanks! > UI - Remove registration process > > > Key: NIFI-1928 > URL: https://issues.apache.org/jira/browse/NIFI-1928 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman > Fix For: 1.0.0 > > > With the updated user authorization model, users will no longer be able to > request an account. Instead, users are authorized through the configured > Authorizer implementation (it's pluggable). Consequently, we can remove all > the account registration forms from the UI and OBE REST endpoints. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
nifi git commit: NIFI-1943 added proper ReadsAttribute to PutFile
Repository: nifi
Updated Branches:
refs/heads/master f0f74fe40 -> 54c5c28cb
NIFI-1943 added proper ReadsAttribute to PutFile
This closes #480
Project: http://git-wip-us.apache.org/repos/asf/nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/54c5c28c
Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/54c5c28c
Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/54c5c28c
Branch: refs/heads/master
Commit: 54c5c28cbb0f9f50d4aacf22cf6c819dcf0eb02d
Parents: f0f74fe
Author: jpercivall
Authored: Tue May 31 10:01:07 2016 -0400
Committer: jpercivall
Committed: Thu Jun 2 13:04:14 2016 -0400
--
.../src/main/java/org/apache/nifi/processors/standard/PutFile.java | 2 ++
1 file changed, 2 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/nifi/blob/54c5c28c/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/PutFile.java
--
diff --git
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/PutFile.java
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/PutFile.java
index 4fcd79a..8ba3288 100644
---
a/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/PutFile.java
+++
b/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/PutFile.java
@@ -36,6 +36,7 @@ import java.util.regex.Pattern;
import org.apache.nifi.annotation.behavior.InputRequirement;
import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.ReadsAttribute;
import org.apache.nifi.annotation.behavior.SupportsBatching;
import org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.annotation.documentation.SeeAlso;
@@ -58,6 +59,7 @@ import org.apache.nifi.util.StopWatch;
@Tags({"put", "local", "copy", "archive", "files", "filesystem"})
@CapabilityDescription("Writes the contents of a FlowFile to the local file
system")
@SeeAlso({FetchFile.class, GetFile.class})
+@ReadsAttribute(attribute = "filename", description = "The filename to use
when writing the FlowFile to disk.")
public class PutFile extends AbstractProcessor {
public static final String REPLACE_RESOLUTION = "replace";
[jira] [Commented] (NIFI-1943) PutFile should have ReadsAttribute annotation for "filename"
[ https://issues.apache.org/jira/browse/NIFI-1943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312708#comment-15312708 ] ASF subversion and git services commented on NIFI-1943: --- Commit 54c5c28cbb0f9f50d4aacf22cf6c819dcf0eb02d in nifi's branch refs/heads/master from [~JPercivall] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=54c5c28 ] NIFI-1943 added proper ReadsAttribute to PutFile This closes #480 > PutFile should have ReadsAttribute annotation for "filename" > > > Key: NIFI-1943 > URL: https://issues.apache.org/jira/browse/NIFI-1943 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joseph Percivall >Assignee: Joseph Percivall >Priority: Minor > Fix For: 0.7.0 > > > The PutFile processor writes the flowfile to disk using the value contained > in the "filename" attribute. There should be documentation telling the user > this. > All that needs to be done is to add a "ReadsAttribute" annotation for this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1943) PutFile should have ReadsAttribute annotation for "filename"
[ https://issues.apache.org/jira/browse/NIFI-1943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312712#comment-15312712 ] ASF GitHub Bot commented on NIFI-1943: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/480 > PutFile should have ReadsAttribute annotation for "filename" > > > Key: NIFI-1943 > URL: https://issues.apache.org/jira/browse/NIFI-1943 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joseph Percivall >Assignee: Joseph Percivall >Priority: Minor > Fix For: 0.7.0 > > > The PutFile processor writes the flowfile to disk using the value contained > in the "filename" attribute. There should be documentation telling the user > this. > All that needs to be done is to add a "ReadsAttribute" annotation for this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1323) UI visual design enhancement
[ https://issues.apache.org/jira/browse/NIFI-1323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Moran updated NIFI-1323: Attachment: (was: flowfont.zip) > UI visual design enhancement > > > Key: NIFI-1323 > URL: https://issues.apache.org/jira/browse/NIFI-1323 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Affects Versions: 1.0.0 >Reporter: Rob Moran >Priority: Trivial > Attachments: nifi-add-processor-dialog.png, > nifi-component-samples.png, > nifi-configure-processor-properties-set-value.png, > nifi-configure-processor-properties.png, > nifi-configure-processor-scheduling.png, > nifi-configure-processor-settings.png, nifi-dialog-samp...@800px.png, > nifi-drag-drop-compon...@2x.png, nifi-drop.svg, > nifi-global-menu-pending-u...@2x.png, nifi-global-menu.png, > nifi-interaction-and-menu-samples.png, nifi-lineage-gra...@800px.png, > nifi-logo.svg, nifi-sample-fl...@800px.png, nifi-sample-flow.png, > nifi-search-resu...@2x.png, nifi-shell-samp...@800px.png > > > (I will attach mockups and supporting files as they become available) > I am starting to work on a design to modernize the look and feel of the NiFi > UI. The initial focus of the design is to freshen the UI (flat design, SVG > icons, etc.). Additionally, the new design will propose usability > improvements such as exposing more flow-related actions into collapsible > panes, improving hierarchy of information, etc. > Going forward, the design plan is to help lay the foundation for other UI/UX > related issues such as those documented in NIFI-951. > --- > *flowfont.zip* > Contains icon font and supporting files > *nifi-add-processor-dialog.png* > Dialog sample. This sample shows the 'Add Processor' dialog. > *nifi-component-samples.png* > To show styling for all components, as well as those components when a user > is unauthorized to access. > *nifi-configure-processor-properties* > *nifi-configure-processor-properties-set-value* > *nifi-configure-processor-scheduling* > *nifi-configure-processor-settings* > Configure Processor dialog. See related Comments below (in Activity section). > *nifi-dialog-sample-@800px* > Dialog sample in 800px wide viewport. This sample shows the the 'Details' tab > of a provenance event. > *nifi-drag-drop-component@2x* > To show updated component toolbar button style and user interaction > improvements > *nifi-drop.svg* > NiFi logo without 'nifi > *nifi-global-menu* > To show global menu > *nifi-global-menu-pending-user@2x* > Demonstrates a generic notification with an aggregate count that appears on > the global menu icon. When the menu is displayed, the same notification (and > individual count) appears in line with the corresponding item. > *nifi-interaction-and-menu-samples.png* > To demonstrate user interactions - hover states, tooltips, menus, etc. > *nifi-lineage-graph-@800px* > To show lineage graph with explicit action to get back to data provenance > event list. > *nifi-sample-flow-@800px* > Shows a very useable UI down to around 800px in width. The thinking here is > that at anything lower than this, the NiFi user experience will change to > more of a monitoring and/or administrative type workflow. Future mockups will > be created to illustrate this. > *nifi-logo.svg* > NiFi logo complete > *nifi-sample-flow.png* > Mockup of sample flow. Updated to show revised tool and status bars. > Management related actions will move to a menu via mouseover (see > _nifi-global-menu_). Added benefits here include reducing clutter and more > user-friendly menu with text labels to reduce time spent scanning only a > large set of icons. This also helps gain valuable viewport width in a browser > (see _nifi-sample-flow-@800px_) > *nifi-search-resu...@2x.png* > Search results example > *nifi-shell-sample-@800px* > Shell sample in 800px wide viewport. This sample shows the 'Data Provenance' > table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1897) Allow Nodes to Federate Requests to other nodes
[ https://issues.apache.org/jira/browse/NIFI-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312715#comment-15312715 ] ASF GitHub Bot commented on NIFI-1897: -- Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/488 Reviewing... > Allow Nodes to Federate Requests to other nodes > --- > > Key: NIFI-1897 > URL: https://issues.apache.org/jira/browse/NIFI-1897 > Project: Apache NiFi > Issue Type: Task > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne > Fix For: 1.0.0 > > > When a user makes a web request to any node, it should be capable of > replicating the request to all nodes in the cluster, handling the logic that > the NCM currently handles for replicating requests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1323) UI visual design enhancement
[ https://issues.apache.org/jira/browse/NIFI-1323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rob Moran updated NIFI-1323: Attachment: flowfont.zip Attached updated *flowfont.zip* * Changed baseline to 14% > UI visual design enhancement > > > Key: NIFI-1323 > URL: https://issues.apache.org/jira/browse/NIFI-1323 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Affects Versions: 1.0.0 >Reporter: Rob Moran >Priority: Trivial > Attachments: flowfont.zip, nifi-add-processor-dialog.png, > nifi-component-samples.png, > nifi-configure-processor-properties-set-value.png, > nifi-configure-processor-properties.png, > nifi-configure-processor-scheduling.png, > nifi-configure-processor-settings.png, nifi-dialog-samp...@800px.png, > nifi-drag-drop-compon...@2x.png, nifi-drop.svg, > nifi-global-menu-pending-u...@2x.png, nifi-global-menu.png, > nifi-interaction-and-menu-samples.png, nifi-lineage-gra...@800px.png, > nifi-logo.svg, nifi-sample-fl...@800px.png, nifi-sample-flow.png, > nifi-search-resu...@2x.png, nifi-shell-samp...@800px.png > > > (I will attach mockups and supporting files as they become available) > I am starting to work on a design to modernize the look and feel of the NiFi > UI. The initial focus of the design is to freshen the UI (flat design, SVG > icons, etc.). Additionally, the new design will propose usability > improvements such as exposing more flow-related actions into collapsible > panes, improving hierarchy of information, etc. > Going forward, the design plan is to help lay the foundation for other UI/UX > related issues such as those documented in NIFI-951. > --- > *flowfont.zip* > Contains icon font and supporting files > *nifi-add-processor-dialog.png* > Dialog sample. This sample shows the 'Add Processor' dialog. > *nifi-component-samples.png* > To show styling for all components, as well as those components when a user > is unauthorized to access. > *nifi-configure-processor-properties* > *nifi-configure-processor-properties-set-value* > *nifi-configure-processor-scheduling* > *nifi-configure-processor-settings* > Configure Processor dialog. See related Comments below (in Activity section). > *nifi-dialog-sample-@800px* > Dialog sample in 800px wide viewport. This sample shows the the 'Details' tab > of a provenance event. > *nifi-drag-drop-component@2x* > To show updated component toolbar button style and user interaction > improvements > *nifi-drop.svg* > NiFi logo without 'nifi > *nifi-global-menu* > To show global menu > *nifi-global-menu-pending-user@2x* > Demonstrates a generic notification with an aggregate count that appears on > the global menu icon. When the menu is displayed, the same notification (and > individual count) appears in line with the corresponding item. > *nifi-interaction-and-menu-samples.png* > To demonstrate user interactions - hover states, tooltips, menus, etc. > *nifi-lineage-graph-@800px* > To show lineage graph with explicit action to get back to data provenance > event list. > *nifi-sample-flow-@800px* > Shows a very useable UI down to around 800px in width. The thinking here is > that at anything lower than this, the NiFi user experience will change to > more of a monitoring and/or administrative type workflow. Future mockups will > be created to illustrate this. > *nifi-logo.svg* > NiFi logo complete > *nifi-sample-flow.png* > Mockup of sample flow. Updated to show revised tool and status bars. > Management related actions will move to a menu via mouseover (see > _nifi-global-menu_). Added benefits here include reducing clutter and more > user-friendly menu with text labels to reduce time spent scanning only a > large set of icons. This also helps gain valuable viewport width in a browser > (see _nifi-sample-flow-@800px_) > *nifi-search-resu...@2x.png* > Search results example > *nifi-shell-sample-@800px* > Shell sample in 800px wide viewport. This sample shows the 'Data Provenance' > table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-1957) Modal dialogs do not handle browser window resize
Andy LoPresto created NIFI-1957: --- Summary: Modal dialogs do not handle browser window resize Key: NIFI-1957 URL: https://issues.apache.org/jira/browse/NIFI-1957 Project: Apache NiFi Issue Type: Bug Components: Core UI Affects Versions: 0.6.1 Reporter: Andy LoPresto Priority: Critical Fix For: 1.0.0, 0.7.0 When the processor configuration dialog is open and the browser window is resized, the dialog does not move or resize to fit in the new visible canvas. In master (1.0) branch, the processor component is also truncated after resizing the window, and refreshing the UI does not fix this issue. See screenshots attached. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1957) Modal dialogs do not handle browser window resize
[ https://issues.apache.org/jira/browse/NIFI-1957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy LoPresto updated NIFI-1957: Attachment: Screen Shot 2016-06-02 at 10.31.31 AM.png Screen Shot 2016-06-02 at 10.34.12 AM.png Screen Shot 2016-06-02 at 10.42.52 AM.png Screenshots of the dialog issue in 0.6.1 and 1.0. > Modal dialogs do not handle browser window resize > - > > Key: NIFI-1957 > URL: https://issues.apache.org/jira/browse/NIFI-1957 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 0.6.1 >Reporter: Andy LoPresto >Priority: Critical > Labels: dialog, ui > Fix For: 1.0.0, 0.7.0 > > Attachments: Screen Shot 2016-06-02 at 10.31.31 AM.png, Screen Shot > 2016-06-02 at 10.34.12 AM.png, Screen Shot 2016-06-02 at 10.42.52 AM.png > > > When the processor configuration dialog is open and the browser window is > resized, the dialog does not move or resize to fit in the new visible canvas. > In master (1.0) branch, the processor component is also truncated after > resizing the window, and refreshing the UI does not fix this issue. See > screenshots attached. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1957) Modal dialogs do not handle browser window resize
[ https://issues.apache.org/jira/browse/NIFI-1957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy LoPresto updated NIFI-1957: Description: When the processor configuration dialog is open and the browser window is resized, the dialog does not move or resize to fit in the new visible canvas. In master (1.0) branch, the processor component is also truncated after resizing the window, and refreshing the UI does not fix this issue. See screenshots attached. This is reported on Mac OS X 10.11.4 using Google Chrome Version 50.0.2661.102 (64-bit) and Safari Version 9.1 (11601.5.17.1). was:When the processor configuration dialog is open and the browser window is resized, the dialog does not move or resize to fit in the new visible canvas. In master (1.0) branch, the processor component is also truncated after resizing the window, and refreshing the UI does not fix this issue. See screenshots attached. > Modal dialogs do not handle browser window resize > - > > Key: NIFI-1957 > URL: https://issues.apache.org/jira/browse/NIFI-1957 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 0.6.1 >Reporter: Andy LoPresto >Priority: Critical > Labels: dialog, ui > Fix For: 1.0.0, 0.7.0 > > Attachments: Screen Shot 2016-06-02 at 10.31.31 AM.png, Screen Shot > 2016-06-02 at 10.34.12 AM.png, Screen Shot 2016-06-02 at 10.42.52 AM.png > > > When the processor configuration dialog is open and the browser window is > resized, the dialog does not move or resize to fit in the new visible canvas. > In master (1.0) branch, the processor component is also truncated after > resizing the window, and refreshing the UI does not fix this issue. See > screenshots attached. > This is reported on Mac OS X 10.11.4 using Google Chrome Version > 50.0.2661.102 (64-bit) and Safari Version 9.1 (11601.5.17.1). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1957) Modal dialogs do not handle browser window resize
[ https://issues.apache.org/jira/browse/NIFI-1957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy LoPresto updated NIFI-1957: Attachment: Screen Shot 2016-06-02 at 10.47.37 AM.png Screen Shot 2016-06-02 at 10.47.41 AM.png Screen Shot 2016-06-02 at 10.47.45 AM.png Screen Shot 2016-06-02 at 10.48.04 AM.png Screen Shot 2016-06-02 at 10.48.09 AM.png Screen Shot 2016-06-02 at 10.48.22 AM.png Screen Shot 2016-06-02 at 10.48.34 AM.png Screen Shot 2016-06-02 at 10.48.48 AM.png > Modal dialogs do not handle browser window resize > - > > Key: NIFI-1957 > URL: https://issues.apache.org/jira/browse/NIFI-1957 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 0.6.1 >Reporter: Andy LoPresto >Priority: Critical > Labels: dialog, ui > Fix For: 1.0.0, 0.7.0 > > Attachments: Screen Shot 2016-06-02 at 10.31.31 AM.png, Screen Shot > 2016-06-02 at 10.34.12 AM.png, Screen Shot 2016-06-02 at 10.42.52 AM.png, > Screen Shot 2016-06-02 at 10.47.37 AM.png, Screen Shot 2016-06-02 at 10.47.41 > AM.png, Screen Shot 2016-06-02 at 10.47.45 AM.png, Screen Shot 2016-06-02 at > 10.48.04 AM.png, Screen Shot 2016-06-02 at 10.48.09 AM.png, Screen Shot > 2016-06-02 at 10.48.22 AM.png, Screen Shot 2016-06-02 at 10.48.34 AM.png, > Screen Shot 2016-06-02 at 10.48.48 AM.png > > > When the processor configuration dialog is open and the browser window is > resized, the dialog does not move or resize to fit in the new visible canvas. > In master (1.0) branch, the processor component is also truncated after > resizing the window, and refreshing the UI does not fix this issue. See > screenshots attached. > This is reported on Mac OS X 10.11.4 using Google Chrome Version > 50.0.2661.102 (64-bit) and Safari Version 9.1 (11601.5.17.1). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-1958) FormatUtils does not handle time units larger than "day"
Andy LoPresto created NIFI-1958:
---
Summary: FormatUtils does not handle time units larger than "day"
Key: NIFI-1958
URL: https://issues.apache.org/jira/browse/NIFI-1958
Project: Apache NiFi
Issue Type: Improvement
Components: Core Framework
Affects Versions: 0.6.1
Reporter: Andy LoPresto
Fix For: 1.0.0, 0.7.0
{{FormatUtils}}, the class used to handle unit conversions from user-provided
configuration values, does not accept values using "week", "month", "year",
etc. These units should be added.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-1118) Enable SplitText processor to limit line length and filter header lines
[ https://issues.apache.org/jira/browse/NIFI-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312920#comment-15312920 ] ASF GitHub Bot commented on NIFI-1118: -- Github user mosermw commented on the issue: https://github.com/apache/nifi/pull/444 +1 LGTM. contrib-check on nifi-standard-bundle passes. Old unit tests from NIFI-1649 continue to work and new unit tests make sense. Templates from NIFI-1649 continue to work the same. Will squash, merge and push tomorrow if nobody else reviews. This is a long standing ticket on a heavily used processor, so I want to give others a chance to weigh in if they want to. > Enable SplitText processor to limit line length and filter header lines > --- > > Key: NIFI-1118 > URL: https://issues.apache.org/jira/browse/NIFI-1118 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Bean >Assignee: Mark Bean > Fix For: 0.7.0 > > > Include the following functionality to the SplitText processor: > 1) Maximum size limit of the split file(s) > A new split file will be created if the next line to be added to the current > split file exceeds a user-defined maximum file size > 2) Header line marker > User-defined character(s) can be used to identify the header line(s) of the > data file rather than a predetermined number of lines > These changes are additions, not a replacement of any property or behavior. > In the case of header line marker, the existing property "Header Line Count" > must be zero for the new property and behavior to be used. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1037) Hdfs Inotify processor
[ https://issues.apache.org/jira/browse/NIFI-1037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312934#comment-15312934 ] Josh Meyer commented on NIFI-1037: -- I believe I almost have this processor working. I'm currently testing it. Please assign to me and I'll create a PR as soon as my testing is wrapped up. > Hdfs Inotify processor > -- > > Key: NIFI-1037 > URL: https://issues.apache.org/jira/browse/NIFI-1037 > Project: Apache NiFi > Issue Type: New Feature >Reporter: nicolas maillard >Priority: Minor > > HDFS has an Inotify interface that enables to access the HDFS edit stream. > https://issues.apache.org/jira/browse/HDFS-6634 > Creating a processor to listen in and get notifications either for select > directories or select actions would have many applications. > - Stream to a search engine the activity on HDFS > - Wait for specific actions or files to trigger workflows, like duplication > to other clusters > - Validate ingestion processes > etc.. > probably more I don't think of. > I have a first working beta version that needs to evolve > it reuses the Hadoop-nar-bundle > Needs a HDFS 2.7 dependency currently done through editing the Hadop-lib > bundle > let me know if this idea makes sense and would be of interest to the community > would love to contribute the idea -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-856) Add Processor for Lumberjack protocol
[ https://issues.apache.org/jira/browse/NIFI-856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15312992#comment-15312992 ] ASF GitHub Bot commented on NIFI-856: - Github user apiri commented on the issue: https://github.com/apache/nifi/pull/290 Hey @trixpan, Looks good! Thanks for the changes. Will get this merged. > Add Processor for Lumberjack protocol > - > > Key: NIFI-856 > URL: https://issues.apache.org/jira/browse/NIFI-856 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Mike de Rhino > Labels: features > Attachments: NIFI-856.patch > > > It would be great if NIFI could support the [lumberjack > protocol|https://github.com/elastic/logstash-forwarder/blob/master/PROTOCOL.md] > so to enable the use of logstash forwarder as a source of data. > A lot of non Java shops tend to avoid installing Java at data producing nodes > and instead of Flume they end up using things like kafka, heka, fluentd or > logstash-forwarded as data shipping mechanisms. > Kafka is great but its architecture seem to be better focused on multi-DC > environments instead of multi-branch scenarios (imagine having to manager 80 > Zookeeper quorum, one for each country where you operate?) > [Heka|https://github.com/mozilla-services/heka] is fine, it has decent > backpressure buffering but no concept of acknowledgement on the receiving > side of a TCP stream. If the other end of a TCP stream is capable of > listening but gets stuck with its messages it will keep spitting data through > the pipe, oblivious to the woes at the other end. > Logstash forwarder in the other hand, is a quite simple tool, with a > reasonable implementation of acknowledgments on the receiving side but... it > depends on Logstash(and logstash has its own issues). > It would be great if NIFI could serve as a middle man, receiving lumberjack > messages and offloading some of the hard work Logstash seems to struggle with > (e.g. using NIFI to save to HDFS while a downstream Logstash writes into ES). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-1959) TailFile not ingesting data when tailed file is moved with no rolling pattern
Pierre Villard created NIFI-1959: Summary: TailFile not ingesting data when tailed file is moved with no rolling pattern Key: NIFI-1959 URL: https://issues.apache.org/jira/browse/NIFI-1959 Project: Apache NiFi Issue Type: Bug Components: Extensions Affects Versions: 0.6.1 Reporter: Pierre Villard Assignee: Pierre Villard Priority: Minor In case "Rolling Filename Pattern" is not set by user and if the tailed file is moved, then the new file will not be tailed. Besides, in such case, the processor will be endlessly triggered without ingesting data: it creates a lot of tasks and consumes CPU. The reason is it never goes in if statement L448. A solution is to look at size() and lastUpdated() of the tailed file to detect a "rollover". However it won't allow the processor to ingest the potential data added in the tailed file just before being moved. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1916) Make File Based Authorizer implement new User, Group, Policy API
[ https://issues.apache.org/jira/browse/NIFI-1916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313120#comment-15313120 ] ASF GitHub Bot commented on NIFI-1916: -- Github user bbende commented on the issue: https://github.com/apache/nifi/pull/473 @mcgilman updated the PR to address the thread-safety, a summary of the changes... - Created a single data structure to encapsulate all data structures used by the FileAuthorizer so there can be a single AtomicReference - Added synchronization to add, update, delete methods to ensure only one thread can modify the internal Authorizations reference - Included changes to expose the root group id to the authorizer which will be needed to auto-convert old users files - Brought back the old users.xsd and generation of the jaxb object to prep for auto-converting old users files > Make File Based Authorizer implement new User, Group, Policy API > > > Key: NIFI-1916 > URL: https://issues.apache.org/jira/browse/NIFI-1916 > Project: Apache NiFi > Issue Type: Sub-task > Components: Core Framework >Reporter: Bryan Bende >Assignee: Bryan Bende > Fix For: 1.0.0 > > > This ticket is a sub-task of NIFI-1550 which is the larger effort to refactor > NiFi's authorization API. > This ticket specifically is to update the FileAuthorizer introduced on > master, to now extend from AbstractPolicyBasedAuthorizer and implement all of > the CRUD operations for Users, Groups, and Policies. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1959) TailFile not ingesting data when tailed file is moved with no rolling pattern
[ https://issues.apache.org/jira/browse/NIFI-1959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313269#comment-15313269 ] ASF GitHub Bot commented on NIFI-1959: -- GitHub user pvillard31 opened a pull request: https://github.com/apache/nifi/pull/490 NIFI-1959 Added length and timestamp to detect rollover It improves robustness of the processor in case the property “Rolling filename pattern” is not set and the tailed file is moved for some reasons. You can merge this pull request into a Git repository by running: $ git pull https://github.com/pvillard31/nifi NIFI-1959 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/490.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #490 commit d3d59d58a51ded47ba1b7ebf87a79f678ab41cf2 Author: Pierre Villard Date: 2016-06-02T23:09:53Z NIFI-1959 Added length and timestamp to detect rollover It improves robustness of the processor in case the property “Rolling filename pattern” is not set and the tailed file is moved for some reasons. > TailFile not ingesting data when tailed file is moved with no rolling pattern > - > > Key: NIFI-1959 > URL: https://issues.apache.org/jira/browse/NIFI-1959 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 0.6.1 >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Minor > > In case "Rolling Filename Pattern" is not set by user and if the tailed file > is moved, then the new file will not be tailed. > Besides, in such case, the processor will be endlessly triggered without > ingesting data: it creates a lot of tasks and consumes CPU. The reason is it > never goes in if statement L448. > A solution is to look at size() and lastUpdated() of the tailed file to > detect a "rollover". However it won't allow the processor to ingest the > potential data added in the tailed file just before being moved. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1959) TailFile not ingesting data when tailed file is moved with no rolling pattern
[ https://issues.apache.org/jira/browse/NIFI-1959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313279#comment-15313279 ] ASF GitHub Bot commented on NIFI-1959: -- Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/490 It remains an issue I have not been able to solve at this point: the case where the new file contains more data than the tailed file before being moved. In this case we ends up in the same situation: the new data in not ingested and the processor is not yielded. This situation should not happen really often... but still... if someone has an idea. To reproduce: tail on /tmp/test.log echo "test" >> /tmp/test.log rm /tmp/test.log echo "testtest" >> /tmp/test.log In this case, both timestamp and size cannot be used to detect a pseudo-rollover and we are stuck with the previous state and reader. Not sure to see an easy solution. > TailFile not ingesting data when tailed file is moved with no rolling pattern > - > > Key: NIFI-1959 > URL: https://issues.apache.org/jira/browse/NIFI-1959 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 0.6.1 >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Minor > > In case "Rolling Filename Pattern" is not set by user and if the tailed file > is moved, then the new file will not be tailed. > Besides, in such case, the processor will be endlessly triggered without > ingesting data: it creates a lot of tasks and consumes CPU. The reason is it > never goes in if statement L448. > A solution is to look at size() and lastUpdated() of the tailed file to > detect a "rollover". However it won't allow the processor to ingest the > potential data added in the tailed file just before being moved. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1578) Create PutSlack processor
[ https://issues.apache.org/jira/browse/NIFI-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313364#comment-15313364 ] ASF GitHub Bot commented on NIFI-1578: -- Github user adamonduty commented on the issue: https://github.com/apache/nifi/pull/256 @rstjohn I fixed the channel validator issue, would you mind trying it again? I think the implementation of attachments with dynamic properties sounds reasonable, but I think it makes sense to try to get this module as-is into 0.7 and leave the attachments feature for a future implementation. @mattyb149 @alopresto Would either of you mind giving this another review? I copied the `TestServer` from the standard processors to the slack bundle and now all the tests interact with an embedded web server rather than touch the slack service. > Create PutSlack processor > - > > Key: NIFI-1578 > URL: https://issues.apache.org/jira/browse/NIFI-1578 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Adam Lamar >Assignee: Adam Lamar > Fix For: 0.7.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-1960) Documentation
Matt Gilman created NIFI-1960: - Summary: Documentation Key: NIFI-1960 URL: https://issues.apache.org/jira/browse/NIFI-1960 Project: Apache NiFi Issue Type: Task Components: Documentation & Website Reporter: Matt Gilman Priority: Blocker Fix For: 1.0.0 Several parts of the application are fundamentally different in 1.x. Specifically, documentation around clustering and authorization need to be completely revisited. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[2/2] nifi git commit: NIFI-856 - Implements ListenLumberjack Processor
NIFI-856 - Implements ListenLumberjack Processor This closes #290. Signed-off-by: Aldrin Piri Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/258fa660 Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/258fa660 Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/258fa660 Branch: refs/heads/0.x Commit: 258fa660d4a6c1c277b4d353498a7de6dbc90e1b Parents: d01f2d9 Author: Andre F de Miranda Authored: Sat Mar 19 01:41:35 2016 +1100 Committer: Aldrin Piri Committed: Thu Jun 2 20:55:20 2016 -0400 -- nifi-assembly/pom.xml | 5 + .../nifi-lumberjack-nar/pom.xml | 42 +++ .../src/main/resources/META-INF/LICENSE | 202 + .../src/main/resources/META-INF/NOTICE | 5 + .../nifi-lumberjack-processors/pom.xml | 77 + .../processors/lumberjack/ListenLumberjack.java | 232 ++ .../lumberjack/event/LumberjackEvent.java | 46 +++ .../event/LumberjackEventFactory.java | 38 +++ .../lumberjack/event/LumberjackMetadata.java| 26 ++ .../lumberjack/frame/LumberjackDecoder.java | 299 +++ .../lumberjack/frame/LumberjackEncoder.java | 47 +++ .../lumberjack/frame/LumberjackFrame.java | 115 +++ .../frame/LumberjackFrameException.java | 32 ++ .../lumberjack/frame/LumberjackState.java | 28 ++ .../handler/LumberjackFrameHandler.java | 113 +++ .../LumberjackSSLSocketChannelHandler.java | 94 ++ .../handler/LumberjackSocketChannelHandler.java | 103 +++ .../LumberjackSocketChannelHandlerFactory.java | 56 .../response/LumberjackChannelResponse.java | 42 +++ .../lumberjack/response/LumberjackResponse.java | 62 .../org.apache.nifi.processor.Processor | 15 + .../event/TestLumberjackEventFactory.java | 54 .../lumberjack/frame/TestLumberjackDecoder.java | 101 +++ .../lumberjack/frame/TestLumberjackEncoder.java | 49 +++ .../lumberjack/frame/TestLumberjackFrame.java | 38 +++ .../handler/TestLumberjackFrameHandler.java | 157 ++ .../TestLumberjackSocketChannelHandler.java | 206 + nifi-nar-bundles/nifi-lumberjack-bundle/pom.xml | 34 +++ nifi-nar-bundles/pom.xml| 1 + pom.xml | 6 + 30 files changed, 2325 insertions(+) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/258fa660/nifi-assembly/pom.xml -- diff --git a/nifi-assembly/pom.xml b/nifi-assembly/pom.xml index 485a2b7..17ec567 100644 --- a/nifi-assembly/pom.xml +++ b/nifi-assembly/pom.xml @@ -284,6 +284,11 @@ language governing permissions and limitations under the License. --> org.apache.nifi +nifi-lumberjack-nar +nar + + +org.apache.nifi nifi-amqp-nar nar http://git-wip-us.apache.org/repos/asf/nifi/blob/258fa660/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-nar/pom.xml -- diff --git a/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-nar/pom.xml b/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-nar/pom.xml new file mode 100644 index 000..1c926a3 --- /dev/null +++ b/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-nar/pom.xml @@ -0,0 +1,42 @@ + + +http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd";> +4.0.0 + + +org.apache.nifi +nifi-lumberjack-bundle +0.7.0-SNAPSHOT + + +nifi-lumberjack-nar +0.7.0-SNAPSHOT +nar + + + +org.apache.nifi +nifi-lumberjack-processors +0.7.0-SNAPSHOT + + +org.apache.nifi +nifi-standard-services-api-nar +nar + + + + http://git-wip-us.apache.org/repos/asf/nifi/blob/258fa660/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-nar/src/main/resources/META-INF/LICENSE -- diff --git a/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-nar/src/main/resources/META-INF/LICENSE b/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-nar/src/main/resources/META-INF/LICENSE new file mode 100644 index 000..489ee58 --- /dev/null +++ b/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-nar/src/main/resources/META-INF/LICENSE @@ -0,0 +1,202 @@ + + Apache License + Version
[1/2] nifi git commit: NIFI-856 - Implements ListenLumberjack Processor
Repository: nifi
Updated Branches:
refs/heads/0.x d01f2d910 -> 258fa660d
http://git-wip-us.apache.org/repos/asf/nifi/blob/258fa660/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-processors/src/main/java/org/apache/nifi/processors/lumberjack/response/LumberjackResponse.java
--
diff --git
a/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-processors/src/main/java/org/apache/nifi/processors/lumberjack/response/LumberjackResponse.java
b/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-processors/src/main/java/org/apache/nifi/processors/lumberjack/response/LumberjackResponse.java
new file mode 100644
index 000..2ace367
--- /dev/null
+++
b/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-processors/src/main/java/org/apache/nifi/processors/lumberjack/response/LumberjackResponse.java
@@ -0,0 +1,62 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.lumberjack.response;
+
+import org.apache.nifi.processors.lumberjack.frame.LumberjackFrame;
+
+import java.nio.ByteBuffer;
+
+/**
+ 'ack' frame type
+
+ SENT FROM READER ONLY
+ frame type value: ASCII 'A' aka byte value 0x41
+
+ Payload:
+ 32bit unsigned sequence number.
+
+ */
+public class LumberjackResponse {
+private final long seqNumber;
+final private byte version = 0x31, frameType = 0x41;
+
+
+
+public LumberjackResponse(final long seqNumber) {
+this.seqNumber = seqNumber;
+}
+
+/**
+ * Creates a LumberjackFrame where the data portion will contain this
response.
+ *
+ *
+ * @return a LumberjackFrame for for this response
+ */
+public LumberjackFrame toFrame() {
+
+return new LumberjackFrame.Builder()
+.version(version)
+.frameType(frameType)
+.payload(ByteBuffer.allocate(8).putLong(seqNumber).array())
+.build();
+}
+
+public static LumberjackResponse ok(final long seqNumber) {
+return new LumberjackResponse(seqNumber);
+}
+
+}
http://git-wip-us.apache.org/repos/asf/nifi/blob/258fa660/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-processors/src/main/resources/META-INF/services/org.apache.nifi.processor.Processor
--
diff --git
a/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-processors/src/main/resources/META-INF/services/org.apache.nifi.processor.Processor
b/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-processors/src/main/resources/META-INF/services/org.apache.nifi.processor.Processor
new file mode 100644
index 000..3c23391
--- /dev/null
+++
b/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-processors/src/main/resources/META-INF/services/org.apache.nifi.processor.Processor
@@ -0,0 +1,15 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+org.apache.nifi.processors.lumberjack.ListenLumberjack
http://git-wip-us.apache.org/repos/asf/nifi/blob/258fa660/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-processors/src/test/java/org/apache/nifi/processors/lumberjack/event/TestLumberjackEventFactory.java
--
diff --git
a/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-processors/src/test/java/org/apache/nifi/processors/lumberjack/event/TestLumberjackEventFactory.java
b/nifi-nar-bundles/nifi-lumberjack-bu
[jira] [Commented] (NIFI-856) Add Processor for Lumberjack protocol
[ https://issues.apache.org/jira/browse/NIFI-856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313433#comment-15313433 ] ASF subversion and git services commented on NIFI-856: -- Commit 258fa660d4a6c1c277b4d353498a7de6dbc90e1b in nifi's branch refs/heads/0.x from Andre F de Miranda [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=258fa66 ] NIFI-856 - Implements ListenLumberjack Processor This closes #290. Signed-off-by: Aldrin Piri > Add Processor for Lumberjack protocol > - > > Key: NIFI-856 > URL: https://issues.apache.org/jira/browse/NIFI-856 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Mike de Rhino > Labels: features > Attachments: NIFI-856.patch > > > It would be great if NIFI could support the [lumberjack > protocol|https://github.com/elastic/logstash-forwarder/blob/master/PROTOCOL.md] > so to enable the use of logstash forwarder as a source of data. > A lot of non Java shops tend to avoid installing Java at data producing nodes > and instead of Flume they end up using things like kafka, heka, fluentd or > logstash-forwarded as data shipping mechanisms. > Kafka is great but its architecture seem to be better focused on multi-DC > environments instead of multi-branch scenarios (imagine having to manager 80 > Zookeeper quorum, one for each country where you operate?) > [Heka|https://github.com/mozilla-services/heka] is fine, it has decent > backpressure buffering but no concept of acknowledgement on the receiving > side of a TCP stream. If the other end of a TCP stream is capable of > listening but gets stuck with its messages it will keep spitting data through > the pipe, oblivious to the woes at the other end. > Logstash forwarder in the other hand, is a quite simple tool, with a > reasonable implementation of acknowledgments on the receiving side but... it > depends on Logstash(and logstash has its own issues). > It would be great if NIFI could serve as a middle man, receiving lumberjack > messages and offloading some of the hard work Logstash seems to struggle with > (e.g. using NIFI to save to HDFS while a downstream Logstash writes into ES). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-1961) Test Resource - nifi.properties
Matt Gilman created NIFI-1961: - Summary: Test Resource - nifi.properties Key: NIFI-1961 URL: https://issues.apache.org/jira/browse/NIFI-1961 Project: Apache NiFi Issue Type: Task Components: Core Framework Reporter: Matt Gilman Priority: Trivial Fix For: 1.0.0 Many tests reference a copy of nifi.properties. These copies have become increasingly stale with as the application has evolved. With the changes in 1.x we should at the very least update the copies or better yet, update the tests to mock out the necessary configuration to avoid the copies all together. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-856) Add Processor for Lumberjack protocol
[ https://issues.apache.org/jira/browse/NIFI-856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aldrin Piri updated NIFI-856: - Fix Version/s: 0.7.0 1.0.0 > Add Processor for Lumberjack protocol > - > > Key: NIFI-856 > URL: https://issues.apache.org/jira/browse/NIFI-856 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Mike de Rhino > Labels: features > Fix For: 1.0.0, 0.7.0 > > Attachments: NIFI-856.patch > > > It would be great if NIFI could support the [lumberjack > protocol|https://github.com/elastic/logstash-forwarder/blob/master/PROTOCOL.md] > so to enable the use of logstash forwarder as a source of data. > A lot of non Java shops tend to avoid installing Java at data producing nodes > and instead of Flume they end up using things like kafka, heka, fluentd or > logstash-forwarded as data shipping mechanisms. > Kafka is great but its architecture seem to be better focused on multi-DC > environments instead of multi-branch scenarios (imagine having to manager 80 > Zookeeper quorum, one for each country where you operate?) > [Heka|https://github.com/mozilla-services/heka] is fine, it has decent > backpressure buffering but no concept of acknowledgement on the receiving > side of a TCP stream. If the other end of a TCP stream is capable of > listening but gets stuck with its messages it will keep spitting data through > the pipe, oblivious to the woes at the other end. > Logstash forwarder in the other hand, is a quite simple tool, with a > reasonable implementation of acknowledgments on the receiving side but... it > depends on Logstash(and logstash has its own issues). > It would be great if NIFI could serve as a middle man, receiving lumberjack > messages and offloading some of the hard work Logstash seems to struggle with > (e.g. using NIFI to save to HDFS while a downstream Logstash writes into ES). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[1/2] nifi git commit: NIFI-856 - Implements ListenLumberjack Processor
Repository: nifi
Updated Branches:
refs/heads/master 54c5c28cb -> 5a8979150
http://git-wip-us.apache.org/repos/asf/nifi/blob/5a897915/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-processors/src/main/java/org/apache/nifi/processors/lumberjack/response/LumberjackResponse.java
--
diff --git
a/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-processors/src/main/java/org/apache/nifi/processors/lumberjack/response/LumberjackResponse.java
b/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-processors/src/main/java/org/apache/nifi/processors/lumberjack/response/LumberjackResponse.java
new file mode 100644
index 000..2ace367
--- /dev/null
+++
b/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-processors/src/main/java/org/apache/nifi/processors/lumberjack/response/LumberjackResponse.java
@@ -0,0 +1,62 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.lumberjack.response;
+
+import org.apache.nifi.processors.lumberjack.frame.LumberjackFrame;
+
+import java.nio.ByteBuffer;
+
+/**
+ 'ack' frame type
+
+ SENT FROM READER ONLY
+ frame type value: ASCII 'A' aka byte value 0x41
+
+ Payload:
+ 32bit unsigned sequence number.
+
+ */
+public class LumberjackResponse {
+private final long seqNumber;
+final private byte version = 0x31, frameType = 0x41;
+
+
+
+public LumberjackResponse(final long seqNumber) {
+this.seqNumber = seqNumber;
+}
+
+/**
+ * Creates a LumberjackFrame where the data portion will contain this
response.
+ *
+ *
+ * @return a LumberjackFrame for for this response
+ */
+public LumberjackFrame toFrame() {
+
+return new LumberjackFrame.Builder()
+.version(version)
+.frameType(frameType)
+.payload(ByteBuffer.allocate(8).putLong(seqNumber).array())
+.build();
+}
+
+public static LumberjackResponse ok(final long seqNumber) {
+return new LumberjackResponse(seqNumber);
+}
+
+}
http://git-wip-us.apache.org/repos/asf/nifi/blob/5a897915/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-processors/src/main/resources/META-INF/services/org.apache.nifi.processor.Processor
--
diff --git
a/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-processors/src/main/resources/META-INF/services/org.apache.nifi.processor.Processor
b/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-processors/src/main/resources/META-INF/services/org.apache.nifi.processor.Processor
new file mode 100644
index 000..3c23391
--- /dev/null
+++
b/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-processors/src/main/resources/META-INF/services/org.apache.nifi.processor.Processor
@@ -0,0 +1,15 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+org.apache.nifi.processors.lumberjack.ListenLumberjack
http://git-wip-us.apache.org/repos/asf/nifi/blob/5a897915/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-processors/src/test/java/org/apache/nifi/processors/lumberjack/event/TestLumberjackEventFactory.java
--
diff --git
a/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-processors/src/test/java/org/apache/nifi/processors/lumberjack/event/TestLumberjackEventFactory.java
b/nifi-nar-bundles/nifi-lumberjack
[2/2] nifi git commit: NIFI-856 - Implements ListenLumberjack Processor
NIFI-856 - Implements ListenLumberjack Processor This closes #290. Signed-off-by: Aldrin Piri Project: http://git-wip-us.apache.org/repos/asf/nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/nifi/commit/5a897915 Tree: http://git-wip-us.apache.org/repos/asf/nifi/tree/5a897915 Diff: http://git-wip-us.apache.org/repos/asf/nifi/diff/5a897915 Branch: refs/heads/master Commit: 5a8979150ccef1bee9f90270dd1e37efa1d020ea Parents: 54c5c28 Author: Andre F de Miranda Authored: Sat Mar 19 01:41:35 2016 +1100 Committer: Aldrin Piri Committed: Thu Jun 2 21:58:41 2016 -0400 -- nifi-assembly/pom.xml | 5 + .../nifi-lumberjack-nar/pom.xml | 42 +++ .../src/main/resources/META-INF/LICENSE | 202 + .../src/main/resources/META-INF/NOTICE | 5 + .../nifi-lumberjack-processors/pom.xml | 76 + .../processors/lumberjack/ListenLumberjack.java | 232 ++ .../lumberjack/event/LumberjackEvent.java | 46 +++ .../event/LumberjackEventFactory.java | 38 +++ .../lumberjack/event/LumberjackMetadata.java| 26 ++ .../lumberjack/frame/LumberjackDecoder.java | 299 +++ .../lumberjack/frame/LumberjackEncoder.java | 47 +++ .../lumberjack/frame/LumberjackFrame.java | 115 +++ .../frame/LumberjackFrameException.java | 32 ++ .../lumberjack/frame/LumberjackState.java | 28 ++ .../handler/LumberjackFrameHandler.java | 114 +++ .../LumberjackSSLSocketChannelHandler.java | 94 ++ .../handler/LumberjackSocketChannelHandler.java | 103 +++ .../LumberjackSocketChannelHandlerFactory.java | 56 .../response/LumberjackChannelResponse.java | 42 +++ .../lumberjack/response/LumberjackResponse.java | 62 .../org.apache.nifi.processor.Processor | 15 + .../event/TestLumberjackEventFactory.java | 54 .../lumberjack/frame/TestLumberjackDecoder.java | 101 +++ .../lumberjack/frame/TestLumberjackEncoder.java | 49 +++ .../lumberjack/frame/TestLumberjackFrame.java | 38 +++ .../handler/TestLumberjackFrameHandler.java | 157 ++ .../TestLumberjackSocketChannelHandler.java | 207 + nifi-nar-bundles/nifi-lumberjack-bundle/pom.xml | 34 +++ nifi-nar-bundles/pom.xml| 1 + pom.xml | 12 +- 30 files changed, 2329 insertions(+), 3 deletions(-) -- http://git-wip-us.apache.org/repos/asf/nifi/blob/5a897915/nifi-assembly/pom.xml -- diff --git a/nifi-assembly/pom.xml b/nifi-assembly/pom.xml index aad2561..d226726 100644 --- a/nifi-assembly/pom.xml +++ b/nifi-assembly/pom.xml @@ -284,6 +284,11 @@ language governing permissions and limitations under the License. --> org.apache.nifi +nifi-lumberjack-nar +nar + + +org.apache.nifi nifi-amqp-nar nar http://git-wip-us.apache.org/repos/asf/nifi/blob/5a897915/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-nar/pom.xml -- diff --git a/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-nar/pom.xml b/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-nar/pom.xml new file mode 100644 index 000..747f4d9 --- /dev/null +++ b/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-nar/pom.xml @@ -0,0 +1,42 @@ + + +http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd";> +4.0.0 + + +org.apache.nifi +nifi-lumberjack-bundle +1.0.0-SNAPSHOT + + +nifi-lumberjack-nar +1.0.0-SNAPSHOT +nar + + + +org.apache.nifi +nifi-lumberjack-processors +1.0.0-SNAPSHOT + + +org.apache.nifi +nifi-standard-services-api-nar +nar + + + + http://git-wip-us.apache.org/repos/asf/nifi/blob/5a897915/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-nar/src/main/resources/META-INF/LICENSE -- diff --git a/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-nar/src/main/resources/META-INF/LICENSE b/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-nar/src/main/resources/META-INF/LICENSE new file mode 100644 index 000..489ee58 --- /dev/null +++ b/nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-nar/src/main/resources/META-INF/LICENSE @@ -0,0 +1,202 @@ + + Apache License +
[jira] [Commented] (NIFI-856) Add Processor for Lumberjack protocol
[ https://issues.apache.org/jira/browse/NIFI-856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313469#comment-15313469 ] ASF subversion and git services commented on NIFI-856: -- Commit 5a8979150ccef1bee9f90270dd1e37efa1d020ea in nifi's branch refs/heads/master from Andre F de Miranda [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=5a89791 ] NIFI-856 - Implements ListenLumberjack Processor This closes #290. Signed-off-by: Aldrin Piri > Add Processor for Lumberjack protocol > - > > Key: NIFI-856 > URL: https://issues.apache.org/jira/browse/NIFI-856 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Mike de Rhino > Labels: features > Fix For: 1.0.0, 0.7.0 > > Attachments: NIFI-856.patch > > > It would be great if NIFI could support the [lumberjack > protocol|https://github.com/elastic/logstash-forwarder/blob/master/PROTOCOL.md] > so to enable the use of logstash forwarder as a source of data. > A lot of non Java shops tend to avoid installing Java at data producing nodes > and instead of Flume they end up using things like kafka, heka, fluentd or > logstash-forwarded as data shipping mechanisms. > Kafka is great but its architecture seem to be better focused on multi-DC > environments instead of multi-branch scenarios (imagine having to manager 80 > Zookeeper quorum, one for each country where you operate?) > [Heka|https://github.com/mozilla-services/heka] is fine, it has decent > backpressure buffering but no concept of acknowledgement on the receiving > side of a TCP stream. If the other end of a TCP stream is capable of > listening but gets stuck with its messages it will keep spitting data through > the pipe, oblivious to the woes at the other end. > Logstash forwarder in the other hand, is a quite simple tool, with a > reasonable implementation of acknowledgments on the receiving side but... it > depends on Logstash(and logstash has its own issues). > It would be great if NIFI could serve as a middle man, receiving lumberjack > messages and offloading some of the hard work Logstash seems to struggle with > (e.g. using NIFI to save to HDFS while a downstream Logstash writes into ES). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-856) Add Processor for Lumberjack protocol
[ https://issues.apache.org/jira/browse/NIFI-856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313470#comment-15313470 ] ASF GitHub Bot commented on NIFI-856: - Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/290 > Add Processor for Lumberjack protocol > - > > Key: NIFI-856 > URL: https://issues.apache.org/jira/browse/NIFI-856 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Mike de Rhino > Labels: features > Fix For: 1.0.0, 0.7.0 > > Attachments: NIFI-856.patch > > > It would be great if NIFI could support the [lumberjack > protocol|https://github.com/elastic/logstash-forwarder/blob/master/PROTOCOL.md] > so to enable the use of logstash forwarder as a source of data. > A lot of non Java shops tend to avoid installing Java at data producing nodes > and instead of Flume they end up using things like kafka, heka, fluentd or > logstash-forwarded as data shipping mechanisms. > Kafka is great but its architecture seem to be better focused on multi-DC > environments instead of multi-branch scenarios (imagine having to manager 80 > Zookeeper quorum, one for each country where you operate?) > [Heka|https://github.com/mozilla-services/heka] is fine, it has decent > backpressure buffering but no concept of acknowledgement on the receiving > side of a TCP stream. If the other end of a TCP stream is capable of > listening but gets stuck with its messages it will keep spitting data through > the pipe, oblivious to the woes at the other end. > Logstash forwarder in the other hand, is a quite simple tool, with a > reasonable implementation of acknowledgments on the receiving side but... it > depends on Logstash(and logstash has its own issues). > It would be great if NIFI could serve as a middle man, receiving lumberjack > messages and offloading some of the hard work Logstash seems to struggle with > (e.g. using NIFI to save to HDFS while a downstream Logstash writes into ES). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1037) Hdfs Inotify processor
[ https://issues.apache.org/jira/browse/NIFI-1037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313480#comment-15313480 ] Aldrin Piri commented on NIFI-1037: --- Done. Thanks for your efforts! > Hdfs Inotify processor > -- > > Key: NIFI-1037 > URL: https://issues.apache.org/jira/browse/NIFI-1037 > Project: Apache NiFi > Issue Type: New Feature >Reporter: nicolas maillard >Assignee: Josh Meyer >Priority: Minor > > HDFS has an Inotify interface that enables to access the HDFS edit stream. > https://issues.apache.org/jira/browse/HDFS-6634 > Creating a processor to listen in and get notifications either for select > directories or select actions would have many applications. > - Stream to a search engine the activity on HDFS > - Wait for specific actions or files to trigger workflows, like duplication > to other clusters > - Validate ingestion processes > etc.. > probably more I don't think of. > I have a first working beta version that needs to evolve > it reuses the Hadoop-nar-bundle > Needs a HDFS 2.7 dependency currently done through editing the Hadop-lib > bundle > let me know if this idea makes sense and would be of interest to the community > would love to contribute the idea -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-1037) Hdfs Inotify processor
[ https://issues.apache.org/jira/browse/NIFI-1037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aldrin Piri updated NIFI-1037: -- Assignee: Josh Meyer > Hdfs Inotify processor > -- > > Key: NIFI-1037 > URL: https://issues.apache.org/jira/browse/NIFI-1037 > Project: Apache NiFi > Issue Type: New Feature >Reporter: nicolas maillard >Assignee: Josh Meyer >Priority: Minor > > HDFS has an Inotify interface that enables to access the HDFS edit stream. > https://issues.apache.org/jira/browse/HDFS-6634 > Creating a processor to listen in and get notifications either for select > directories or select actions would have many applications. > - Stream to a search engine the activity on HDFS > - Wait for specific actions or files to trigger workflows, like duplication > to other clusters > - Validate ingestion processes > etc.. > probably more I don't think of. > I have a first working beta version that needs to evolve > it reuses the Hadoop-nar-bundle > Needs a HDFS 2.7 dependency currently done through editing the Hadop-lib > bundle > let me know if this idea makes sense and would be of interest to the community > would love to contribute the idea -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1897) Allow Nodes to Federate Requests to other nodes
[ https://issues.apache.org/jira/browse/NIFI-1897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313533#comment-15313533 ] ASF GitHub Bot commented on NIFI-1897: -- Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/488 @markap14 I've been reviewing this PR and have some initial findings. As I'm reviewing I'm updating various comments and naming that still reflect the previous clustering model. I've gone ahead and updated those. Also, had some trouble starting the application in standalone mode but was able to resolve by updating the Spring Context where necessary. You can check out these modifications here [1]. Additionally, I noticed that none of the user documentation has been updated to reflect the new clustering model and a number of test resources have become stale. I've create separate JIRAs for addressing these issues [2] [3] at a later time. Will be doing some additional reviewing but so far so good. [1] https://github.com/mcgilman/nifi/commit/05cbb92ce35feb7aefbe607da22e1bbde8de5314 [2] https://issues.apache.org/jira/browse/NIFI-1960 [3] https://issues.apache.org/jira/browse/NIFI-1961 > Allow Nodes to Federate Requests to other nodes > --- > > Key: NIFI-1897 > URL: https://issues.apache.org/jira/browse/NIFI-1897 > Project: Apache NiFi > Issue Type: Task > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne > Fix For: 1.0.0 > > > When a user makes a web request to any node, it should be capable of > replicating the request to all nodes in the cluster, handling the logic that > the NCM currently handles for replicating requests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1922) Add support for WASB / Azure Blob Storage with HDFS processors
[ https://issues.apache.org/jira/browse/NIFI-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313534#comment-15313534 ] Alex Tilcock commented on NIFI-1922: I'm working on a project which needs to work around this problem so I've been doing a bit of debugging. The issue with using the existing HDFS processor is that it makes a call from AbstractHadoopProcessor (GetFileSystemAsUser) that calls down to the base java FileSystem class. An exception is raised in the FileSystem.getFileSystemClass method when it can't find a match for the wasb scheme. The good news is that the FileSystem class has a method (newFileSystem) that supports adding additional system schemas and so one approach might be to create an extension class that inherits from FileSystem (i.e AbstractHadoopFileSystem) that implements a method to add the wasb format from the azure hadoop/storage dependencies and modify the AbstractHadoopProcessor to call the new class. > Add support for WASB / Azure Blob Storage with HDFS processors > -- > > Key: NIFI-1922 > URL: https://issues.apache.org/jira/browse/NIFI-1922 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Joseph Witt > > https://hadoop.apache.org/docs/stable2/hadoop-azure/index.html > Received two mailing list questions on this in the past couple weeks. > We should look at what things we'd need to do in the processors to make this > possible. Adding the dependency is one, anything else? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
