[jira] [Updated] (NIFI-731) If content repo is unable to destroy content as fast as it is generated, nifi performance becomes very sporadic
[ https://issues.apache.org/jira/browse/NIFI-731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-731: Summary: If content repo is unable to destroy content as fast as it is generated, nifi performance becomes very sporadic (was: If content repo is unable to destroy content as fast as it is generated, nifi performance becomes very sporatic) If content repo is unable to destroy content as fast as it is generated, nifi performance becomes very sporadic --- Key: NIFI-731 URL: https://issues.apache.org/jira/browse/NIFI-731 Project: Apache NiFi Issue Type: Bug Components: Core Framework Reporter: Mark Payne Assignee: Mark Payne Fix For: 0.2.0 When the FlowFile Repository marks claims as destructable, it puts the notification on a queue that the content repo pulls from. If the content repo cannot keep up, the queue will fill, resulting in backpressure, that prevents the FlowFile repository from being updated. This, in turn, causes Processors to block, waiting on space to become available. This is by design. However, the capacity of this queue is quite large, and the content repo drains the entire queue, then destroys all content claims that are on it. As a result, this act of destroying claims can be quite long, and Processors can block for quite a period of time, leading to very sporadic performance. Instead, the content repo should pull from the queue and destroy the claims one at a time or in small batches, instead of draining the entire queue each time. This should result in much less sporadic behavior. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-736) Create a Maven Archetype for ControllerServices
Bryan Bende created NIFI-736: Summary: Create a Maven Archetype for ControllerServices Key: NIFI-736 URL: https://issues.apache.org/jira/browse/NIFI-736 Project: Apache NiFi Issue Type: Improvement Reporter: Bryan Bende Assignee: Bryan Bende Priority: Minor Fix For: 0.3.0 It would be helpful for people building extensions to have a good starting point for building controller services, and it would also serve as a good example for learning the dependency relationship between nars. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-736) Create a Maven Archetype for ControllerServices
[ https://issues.apache.org/jira/browse/NIFI-736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Bende updated NIFI-736: - Attachment: NIFI-736.patch Create a Maven Archetype for ControllerServices --- Key: NIFI-736 URL: https://issues.apache.org/jira/browse/NIFI-736 Project: Apache NiFi Issue Type: Improvement Reporter: Bryan Bende Assignee: Bryan Bende Priority: Minor Fix For: 0.3.0 Attachments: NIFI-736.patch It would be helpful for people building extensions to have a good starting point for building controller services, and it would also serve as a good example for learning the dependency relationship between nars. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-736) Create a Maven Archetype for ControllerServices
[ https://issues.apache.org/jira/browse/NIFI-736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604172#comment-14604172 ] Bryan Bende commented on NIFI-736: -- [~markap14] can you review this patch when you get a chance? This archetype creates a project structure similar to the controller service layout in the Developer Guide. It is geared towards someone developing a controller service that is separate from nifi-standard-services. Here is an example of using it: mvn archetype:generate -DarchetypeGroupId=org.apache.nifi -DarchetypeArtifactId=nifi-service-bundle-archetype -DarchetypeVersion=0.2.0-incubating-SNAPSHOT -DnifiVersion=0.2.0-incubating-SNAPSHOT Define value for property 'groupId': : org.apache.nifi Define value for property 'artifactId': : nifi-helloworld-services Define value for property 'version': 1.0-SNAPSHOT: : Define value for property 'artifactBaseName': : helloworld Define value for property 'package': org.apache.nifi.helloworld: : [INFO] Using property: nifiVersion = 0.2.0-incubating-SNAPSHOT Create a Maven Archetype for ControllerServices --- Key: NIFI-736 URL: https://issues.apache.org/jira/browse/NIFI-736 Project: Apache NiFi Issue Type: Improvement Reporter: Bryan Bende Assignee: Bryan Bende Priority: Minor Fix For: 0.3.0 Attachments: NIFI-736.patch It would be helpful for people building extensions to have a good starting point for building controller services, and it would also serve as a good example for learning the dependency relationship between nars. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (NIFI-632) Documentation Generation: Nar ClassLoader
[ https://issues.apache.org/jira/browse/NIFI-632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt reopened NIFI-632: -- need to resolve versioning issues present and use of version property rather than simply declaring in the parent with the explicit version. Will commit the resolution shortly. Documentation Generation: Nar ClassLoader - Key: NIFI-632 URL: https://issues.apache.org/jira/browse/NIFI-632 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 0.1.0 Reporter: Matt Gilman Assignee: Dan Bress Priority: Minor Fix For: 0.2.0 Attachments: 0001-Squashed-commit-of-the-following.patch When auto generating component documentation the NarThreadContextClassLoader needs to be set in case the component attempts to load anything off of its classpath. Without this, the documentation generation fails. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
incubator-nifi git commit: NIFI-632 fixed versioning issue
Repository: incubator-nifi Updated Branches: refs/heads/develop cc3be416b - 8f36ba1ab NIFI-632 fixed versioning issue Project: http://git-wip-us.apache.org/repos/asf/incubator-nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-nifi/commit/8f36ba1a Tree: http://git-wip-us.apache.org/repos/asf/incubator-nifi/tree/8f36ba1a Diff: http://git-wip-us.apache.org/repos/asf/incubator-nifi/diff/8f36ba1a Branch: refs/heads/develop Commit: 8f36ba1ab417bdd6f6d3c3884e4abf750c5ea226 Parents: cc3be41 Author: joewitt joew...@apache.org Authored: Sat Jun 27 13:45:00 2015 -0400 Committer: joewitt joew...@apache.org Committed: Sat Jun 27 13:45:00 2015 -0400 -- nifi/logs/nifi-app.log | 0 nifi/logs/nifi-bootstrap.log | 6 ++ nifi/logs/nifi-user.log | 0 .../nifi-example-bundle/nifi-nifi-example-nar/pom.xml| 5 ++--- .../nifi-nifi-example-processors/pom.xml | 2 +- nifi/nifi-external/nifi-example-bundle/pom.xml | 11 ++- 6 files changed, 19 insertions(+), 5 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/8f36ba1a/nifi/logs/nifi-app.log -- diff --git a/nifi/logs/nifi-app.log b/nifi/logs/nifi-app.log new file mode 100644 index 000..e69de29 http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/8f36ba1a/nifi/logs/nifi-bootstrap.log -- diff --git a/nifi/logs/nifi-bootstrap.log b/nifi/logs/nifi-bootstrap.log new file mode 100644 index 000..3d2aba2 --- /dev/null +++ b/nifi/logs/nifi-bootstrap.log @@ -0,0 +1,6 @@ +2015-06-27 13:43:15,941 INFO [main] org.apache.nifi.bootstrap.Command Starting Apache NiFi... +2015-06-27 13:43:15,944 INFO [main] org.apache.nifi.bootstrap.Command Working Directory: /development/git-repo/nifi/nifi-assembly/target/nifi-0.2.0-incubating-SNAPSHOT-bin/nifi-0.2.0-incubating-SNAPSHOT +2015-06-27 13:43:15,945 INFO [main] org.apache.nifi.bootstrap.Command Command: /usr/java/latest/bin/java -classpath /development/git-repo/nifi/nifi-assembly/target/nifi-0.2.0-incubating-SNAPSHOT-bin/nifi-0.2.0-incubating-SNAPSHOT/./conf:/development/git-repo/nifi/nifi-assembly/target/nifi-0.2.0-incubating-SNAPSHOT-bin/nifi-0.2.0-incubating-SNAPSHOT/./lib/nifi-runtime-0.2.0-incubating-SNAPSHOT.jar:/development/git-repo/nifi/nifi-assembly/target/nifi-0.2.0-incubating-SNAPSHOT-bin/nifi-0.2.0-incubating-SNAPSHOT/./lib/logback-core-1.1.3.jar:/development/git-repo/nifi/nifi-assembly/target/nifi-0.2.0-incubating-SNAPSHOT-bin/nifi-0.2.0-incubating-SNAPSHOT/./lib/slf4j-api-1.7.12.jar:/development/git-repo/nifi/nifi-assembly/target/nifi-0.2.0-incubating-SNAPSHOT-bin/nifi-0.2.0-incubating-SNAPSHOT/./lib/nifi-api-0.2.0-incubating-SNAPSHOT.jar:/development/git-repo/nifi/nifi-assembly/target/nifi-0.2.0-incubating-SNAPSHOT-bin/nifi-0.2.0-incubating-SNAPSHOT/./lib/nifi-properties-0.2.0-incubating-S NAPSHOT.jar:/development/git-repo/nifi/nifi-assembly/target/nifi-0.2.0-incubating-SNAPSHOT-bin/nifi-0.2.0-incubating-SNAPSHOT/./lib/log4j-over-slf4j-1.7.12.jar:/development/git-repo/nifi/nifi-assembly/target/nifi-0.2.0-incubating-SNAPSHOT-bin/nifi-0.2.0-incubating-SNAPSHOT/./lib/jul-to-slf4j-1.7.12.jar:/development/git-repo/nifi/nifi-assembly/target/nifi-0.2.0-incubating-SNAPSHOT-bin/nifi-0.2.0-incubating-SNAPSHOT/./lib/nifi-documentation-0.2.0-incubating-SNAPSHOT.jar:/development/git-repo/nifi/nifi-assembly/target/nifi-0.2.0-incubating-SNAPSHOT-bin/nifi-0.2.0-incubating-SNAPSHOT/./lib/jcl-over-slf4j-1.7.12.jar:/development/git-repo/nifi/nifi-assembly/target/nifi-0.2.0-incubating-SNAPSHOT-bin/nifi-0.2.0-incubating-SNAPSHOT/./lib/logback-classic-1.1.3.jar:/development/git-repo/nifi/nifi-assembly/target/nifi-0.2.0-incubating-SNAPSHOT-bin/nifi-0.2.0-incubating-SNAPSHOT/./lib/nifi-nar-utils-0.2.0-incubating-SNAPSHOT.jar -Djava.net.preferIPv4Stack=true -Dsun.net.http.allowRestrictedHeade rs=true -Djava.protocol.handler.pkgs=sun.net.www.protocol -Dorg.apache.jasper.compiler.disablejsr199=true -Xmx512m -Xms512m -Dnifi.properties.file.path=/development/git-repo/nifi/nifi-assembly/target/nifi-0.2.0-incubating-SNAPSHOT-bin/nifi-0.2.0-incubating-SNAPSHOT/./conf/nifi.properties -Dnifi.bootstrap.listen.port=59379 -Dapp=NiFi org.apache.nifi.NiFi +2015-06-27 13:43:16,332 INFO [NiFi Bootstrap Command Listener] org.apache.nifi.bootstrap.RunNiFi Apache NiFi now running and listening for Bootstrap requests on port 38812 +2015-06-27 13:44:24,728 INFO [main] org.apache.nifi.bootstrap.Command Apache NiFi has accepted the Shutdown Command and is shutting down now +2015-06-27 13:44:24,758 INFO [main] org.apache.nifi.bootstrap.Command NiFi has finished shutting down.
[jira] [Commented] (NIFI-632) Documentation Generation: Nar ClassLoader
[ https://issues.apache.org/jira/browse/NIFI-632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604277#comment-14604277 ] ASF subversion and git services commented on NIFI-632: -- Commit 8f36ba1ab417bdd6f6d3c3884e4abf750c5ea226 in incubator-nifi's branch refs/heads/develop from [~joewitt] [ https://git-wip-us.apache.org/repos/asf?p=incubator-nifi.git;h=8f36ba1 ] NIFI-632 fixed versioning issue Documentation Generation: Nar ClassLoader - Key: NIFI-632 URL: https://issues.apache.org/jira/browse/NIFI-632 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 0.1.0 Reporter: Matt Gilman Assignee: Dan Bress Priority: Minor Fix For: 0.2.0 Attachments: 0001-Squashed-commit-of-the-following.patch When auto generating component documentation the NarThreadContextClassLoader needs to be set in case the component attempts to load anything off of its classpath. Without this, the documentation generation fails. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (NIFI-632) Documentation Generation: Nar ClassLoader
[ https://issues.apache.org/jira/browse/NIFI-632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt resolved NIFI-632. -- Resolution: Fixed the merged set caused the build to be broken. You would probably not notice unless you had a clean maven repo or specifically one where 0.1.1 wasn't present. Documentation Generation: Nar ClassLoader - Key: NIFI-632 URL: https://issues.apache.org/jira/browse/NIFI-632 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 0.1.0 Reporter: Matt Gilman Assignee: Dan Bress Priority: Minor Fix For: 0.2.0 Attachments: 0001-Squashed-commit-of-the-following.patch When auto generating component documentation the NarThreadContextClassLoader needs to be set in case the component attempts to load anything off of its classpath. Without this, the documentation generation fails. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
incubator-nifi git commit: NIFI-632 removed logs dir that showed up - root issue still unknown
Repository: incubator-nifi Updated Branches: refs/heads/develop 8f36ba1ab - e35f348b8 NIFI-632 removed logs dir that showed up - root issue still unknown Project: http://git-wip-us.apache.org/repos/asf/incubator-nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-nifi/commit/e35f348b Tree: http://git-wip-us.apache.org/repos/asf/incubator-nifi/tree/e35f348b Diff: http://git-wip-us.apache.org/repos/asf/incubator-nifi/diff/e35f348b Branch: refs/heads/develop Commit: e35f348b831f5a27960d0d2dec4787860941fe29 Parents: 8f36ba1 Author: joewitt joew...@apache.org Authored: Sat Jun 27 13:55:33 2015 -0400 Committer: joewitt joew...@apache.org Committed: Sat Jun 27 13:55:33 2015 -0400 -- nifi/logs/nifi-app.log | 0 nifi/logs/nifi-bootstrap.log | 6 -- nifi/logs/nifi-user.log | 0 3 files changed, 6 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/e35f348b/nifi/logs/nifi-app.log -- diff --git a/nifi/logs/nifi-app.log b/nifi/logs/nifi-app.log deleted file mode 100644 index e69de29..000 http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/e35f348b/nifi/logs/nifi-bootstrap.log -- diff --git a/nifi/logs/nifi-bootstrap.log b/nifi/logs/nifi-bootstrap.log deleted file mode 100644 index 3d2aba2..000 --- a/nifi/logs/nifi-bootstrap.log +++ /dev/null @@ -1,6 +0,0 @@ -2015-06-27 13:43:15,941 INFO [main] org.apache.nifi.bootstrap.Command Starting Apache NiFi... -2015-06-27 13:43:15,944 INFO [main] org.apache.nifi.bootstrap.Command Working Directory: /development/git-repo/nifi/nifi-assembly/target/nifi-0.2.0-incubating-SNAPSHOT-bin/nifi-0.2.0-incubating-SNAPSHOT -2015-06-27 13:43:15,945 INFO [main] org.apache.nifi.bootstrap.Command Command: /usr/java/latest/bin/java -classpath /development/git-repo/nifi/nifi-assembly/target/nifi-0.2.0-incubating-SNAPSHOT-bin/nifi-0.2.0-incubating-SNAPSHOT/./conf:/development/git-repo/nifi/nifi-assembly/target/nifi-0.2.0-incubating-SNAPSHOT-bin/nifi-0.2.0-incubating-SNAPSHOT/./lib/nifi-runtime-0.2.0-incubating-SNAPSHOT.jar:/development/git-repo/nifi/nifi-assembly/target/nifi-0.2.0-incubating-SNAPSHOT-bin/nifi-0.2.0-incubating-SNAPSHOT/./lib/logback-core-1.1.3.jar:/development/git-repo/nifi/nifi-assembly/target/nifi-0.2.0-incubating-SNAPSHOT-bin/nifi-0.2.0-incubating-SNAPSHOT/./lib/slf4j-api-1.7.12.jar:/development/git-repo/nifi/nifi-assembly/target/nifi-0.2.0-incubating-SNAPSHOT-bin/nifi-0.2.0-incubating-SNAPSHOT/./lib/nifi-api-0.2.0-incubating-SNAPSHOT.jar:/development/git-repo/nifi/nifi-assembly/target/nifi-0.2.0-incubating-SNAPSHOT-bin/nifi-0.2.0-incubating-SNAPSHOT/./lib/nifi-properties-0.2.0-incubating-S NAPSHOT.jar:/development/git-repo/nifi/nifi-assembly/target/nifi-0.2.0-incubating-SNAPSHOT-bin/nifi-0.2.0-incubating-SNAPSHOT/./lib/log4j-over-slf4j-1.7.12.jar:/development/git-repo/nifi/nifi-assembly/target/nifi-0.2.0-incubating-SNAPSHOT-bin/nifi-0.2.0-incubating-SNAPSHOT/./lib/jul-to-slf4j-1.7.12.jar:/development/git-repo/nifi/nifi-assembly/target/nifi-0.2.0-incubating-SNAPSHOT-bin/nifi-0.2.0-incubating-SNAPSHOT/./lib/nifi-documentation-0.2.0-incubating-SNAPSHOT.jar:/development/git-repo/nifi/nifi-assembly/target/nifi-0.2.0-incubating-SNAPSHOT-bin/nifi-0.2.0-incubating-SNAPSHOT/./lib/jcl-over-slf4j-1.7.12.jar:/development/git-repo/nifi/nifi-assembly/target/nifi-0.2.0-incubating-SNAPSHOT-bin/nifi-0.2.0-incubating-SNAPSHOT/./lib/logback-classic-1.1.3.jar:/development/git-repo/nifi/nifi-assembly/target/nifi-0.2.0-incubating-SNAPSHOT-bin/nifi-0.2.0-incubating-SNAPSHOT/./lib/nifi-nar-utils-0.2.0-incubating-SNAPSHOT.jar -Djava.net.preferIPv4Stack=true -Dsun.net.http.allowRestrictedHeade rs=true -Djava.protocol.handler.pkgs=sun.net.www.protocol -Dorg.apache.jasper.compiler.disablejsr199=true -Xmx512m -Xms512m -Dnifi.properties.file.path=/development/git-repo/nifi/nifi-assembly/target/nifi-0.2.0-incubating-SNAPSHOT-bin/nifi-0.2.0-incubating-SNAPSHOT/./conf/nifi.properties -Dnifi.bootstrap.listen.port=59379 -Dapp=NiFi org.apache.nifi.NiFi -2015-06-27 13:43:16,332 INFO [NiFi Bootstrap Command Listener] org.apache.nifi.bootstrap.RunNiFi Apache NiFi now running and listening for Bootstrap requests on port 38812 -2015-06-27 13:44:24,728 INFO [main] org.apache.nifi.bootstrap.Command Apache NiFi has accepted the Shutdown Command and is shutting down now -2015-06-27 13:44:24,758 INFO [main] org.apache.nifi.bootstrap.Command NiFi has finished shutting down. http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/e35f348b/nifi/logs/nifi-user.log -- diff --git a/nifi/logs/nifi-user.log b/nifi/logs/nifi-user.log deleted file mode 100644 index e69de29..000
[jira] [Commented] (NIFI-632) Documentation Generation: Nar ClassLoader
[ https://issues.apache.org/jira/browse/NIFI-632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604284#comment-14604284 ] ASF subversion and git services commented on NIFI-632: -- Commit e35f348b831f5a27960d0d2dec4787860941fe29 in incubator-nifi's branch refs/heads/develop from [~joewitt] [ https://git-wip-us.apache.org/repos/asf?p=incubator-nifi.git;h=e35f348 ] NIFI-632 removed logs dir that showed up - root issue still unknown Documentation Generation: Nar ClassLoader - Key: NIFI-632 URL: https://issues.apache.org/jira/browse/NIFI-632 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 0.1.0 Reporter: Matt Gilman Assignee: Dan Bress Priority: Minor Fix For: 0.2.0 Attachments: 0001-Squashed-commit-of-the-following.patch When auto generating component documentation the NarThreadContextClassLoader needs to be set in case the component attempts to load anything off of its classpath. Without this, the documentation generation fails. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-717) nifi-bootstrap.log written to directory relative to current working directory
[ https://issues.apache.org/jira/browse/NIFI-717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604295#comment-14604295 ] Mark Payne commented on NIFI-717: - I think maybe the best solution is just to make sure that in the nifi.sh script we set the working directory before calling 'java' to launch the RunNiFi class nifi-bootstrap.log written to directory relative to current working directory - Key: NIFI-717 URL: https://issues.apache.org/jira/browse/NIFI-717 Project: Apache NiFi Issue Type: Bug Components: Core Framework Reporter: Matt Gilman Priority: Minor Fix For: 0.2.0 It appears that nifi-bootstrap.log is written to a directory that is relative to the current working directory. If NiFi is launched from outside $NIFI_HOME the logs end up outside of $NIFI_HOME. It is confusing since its configured to be written to logs/ just like nifi-app.log and nifi-user.log but it is written to logs/ in a different location. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-717) nifi-bootstrap.log written to directory relative to current working directory
[ https://issues.apache.org/jira/browse/NIFI-717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-717: Fix Version/s: (was: 0.3.0) 0.2.0 nifi-bootstrap.log written to directory relative to current working directory - Key: NIFI-717 URL: https://issues.apache.org/jira/browse/NIFI-717 Project: Apache NiFi Issue Type: Bug Components: Core Framework Reporter: Matt Gilman Priority: Minor Fix For: 0.2.0 It appears that nifi-bootstrap.log is written to a directory that is relative to the current working directory. If NiFi is launched from outside $NIFI_HOME the logs end up outside of $NIFI_HOME. It is confusing since its configured to be written to logs/ just like nifi-app.log and nifi-user.log but it is written to logs/ in a different location. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-737) Typo in developer guide 'lot message'
Dan Bress created NIFI-737: -- Summary: Typo in developer guide 'lot message' Key: NIFI-737 URL: https://issues.apache.org/jira/browse/NIFI-737 Project: Apache NiFi Issue Type: Improvement Components: Documentation Website Affects Versions: 0.1.0 Reporter: Dan Bress Priority: Trivial [The Developer Guide|https://nifi.incubator.apache.org/docs/nifi-docs/html/developer-guide.html#onshutdown] has a type: _If such a method throws an Exception, a *lot* message will be generated, and the Exception will otherwise be ignored; other methods with this annotation will still be invoked. Methods with this annotation must take zero arguments._ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-737) Typo in developer guide 'lot message'
[ https://issues.apache.org/jira/browse/NIFI-737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Bress updated NIFI-737: --- Description: [The Developer Guide|https://nifi.incubator.apache.org/docs/nifi-docs/html/developer-guide.html#onshutdown] has a typo: _If such a method throws an Exception, a *lot* message will be generated, and the Exception will otherwise be ignored; other methods with this annotation will still be invoked. Methods with this annotation must take zero arguments._ was: [The Developer Guide|https://nifi.incubator.apache.org/docs/nifi-docs/html/developer-guide.html#onshutdown] has a type: _If such a method throws an Exception, a *lot* message will be generated, and the Exception will otherwise be ignored; other methods with this annotation will still be invoked. Methods with this annotation must take zero arguments._ Typo in developer guide 'lot message' - Key: NIFI-737 URL: https://issues.apache.org/jira/browse/NIFI-737 Project: Apache NiFi Issue Type: Improvement Components: Documentation Website Affects Versions: 0.1.0 Reporter: Dan Bress Priority: Trivial [The Developer Guide|https://nifi.incubator.apache.org/docs/nifi-docs/html/developer-guide.html#onshutdown] has a typo: _If such a method throws an Exception, a *lot* message will be generated, and the Exception will otherwise be ignored; other methods with this annotation will still be invoked. Methods with this annotation must take zero arguments._ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
incubator-nifi git commit: NIFI-735: DocGenerator should on @OnUnscheduled and @OnStopped after its done generating the docs
Repository: incubator-nifi
Updated Branches:
refs/heads/NIFI-735 [created] bd9782cb4
NIFI-735: DocGenerator should on @OnUnscheduled and @OnStopped after its done
generating the docs
Project: http://git-wip-us.apache.org/repos/asf/incubator-nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-nifi/commit/bd9782cb
Tree: http://git-wip-us.apache.org/repos/asf/incubator-nifi/tree/bd9782cb
Diff: http://git-wip-us.apache.org/repos/asf/incubator-nifi/diff/bd9782cb
Branch: refs/heads/NIFI-735
Commit: bd9782cb43ece682c853d9a75279ebe76f968044
Parents: e35f348
Author: danbress dbr...@onyxconsults.com
Authored: Sat Jun 27 14:38:45 2015 -0400
Committer: danbress dbr...@onyxconsults.com
Committed: Sat Jun 27 14:38:45 2015 -0400
--
.../documentation/ConfigurableComponentInitializer.java | 9 -
.../java/org/apache/nifi/documentation/DocGenerator.java | 2 ++
.../documentation/init/ControllerServiceInitializer.java | 11 +--
.../nifi/documentation/init/ProcessorInitializer.java| 11 +--
.../documentation/init/ReportingTaskingInitializer.java | 11 +--
.../documentation/html/HtmlDocumentationWriterTest.java | 2 ++
.../html/ProcessorDocumentationWriterTest.java | 3 +++
7 files changed, 42 insertions(+), 7 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/bd9782cb/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-documentation/src/main/java/org/apache/nifi/documentation/ConfigurableComponentInitializer.java
--
diff --git
a/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-documentation/src/main/java/org/apache/nifi/documentation/ConfigurableComponentInitializer.java
b/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-documentation/src/main/java/org/apache/nifi/documentation/ConfigurableComponentInitializer.java
index bd07ab5..ad21f21 100644
---
a/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-documentation/src/main/java/org/apache/nifi/documentation/ConfigurableComponentInitializer.java
+++
b/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-documentation/src/main/java/org/apache/nifi/documentation/ConfigurableComponentInitializer.java
@@ -20,7 +20,7 @@ import org.apache.nifi.components.ConfigurableComponent;
import org.apache.nifi.reporting.InitializationException;
/**
- * An interface for initializing a ConfigurableComponent. It is up to the
+ * An interface for initializing and tearing down a ConfigurableComponent. It
is up to the
* implementer to call init so that you can call
* ConfigurableComponent.getPropertyDescriptors()
*
@@ -35,4 +35,11 @@ public interface ConfigurableComponentInitializer {
* @throws InitializationException if the component could not be
initialized
*/
void initialize(ConfigurableComponent component) throws
InitializationException;
+
+/**
+ * Calls the lifecycle methods that should be called when a flow is
shutdown.
+ *
+ * @param component the component to initialize
+ */
+void teardown(ConfigurableComponent component);
}
http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/bd9782cb/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-documentation/src/main/java/org/apache/nifi/documentation/DocGenerator.java
--
diff --git
a/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-documentation/src/main/java/org/apache/nifi/documentation/DocGenerator.java
b/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-documentation/src/main/java/org/apache/nifi/documentation/DocGenerator.java
index 7c32bf8..327da04 100644
---
a/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-documentation/src/main/java/org/apache/nifi/documentation/DocGenerator.java
+++
b/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-documentation/src/main/java/org/apache/nifi/documentation/DocGenerator.java
@@ -114,6 +114,8 @@ public class DocGenerator {
try (final OutputStream output = new BufferedOutputStream(new
FileOutputStream(baseDocumenationFile))) {
writer.write(component, output, hasAdditionalInfo(directory));
}
+
+initializer.teardown(component);
}
/**
http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/bd9782cb/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-documentation/src/main/java/org/apache/nifi/documentation/init/ControllerServiceInitializer.java
--
diff --git
[jira] [Commented] (NIFI-735) DocGenerator should on @OnUnscheduled and @OnStopped after its done generating the docs
[ https://issues.apache.org/jira/browse/NIFI-735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604311#comment-14604311 ] Dan Bress commented on NIFI-735: [~mcgilman], implementing this now. You sure you want to call @OnRemoved if I never call @OnAdded? DocGenerator should on @OnUnscheduled and @OnStopped after its done generating the docs --- Key: NIFI-735 URL: https://issues.apache.org/jira/browse/NIFI-735 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 0.1.0 Reporter: Dan Bress Assignee: Dan Bress Fix For: 0.2.0 Per [~mcgilman] +0 for triggering OnRemoved and OnShutdown after initialization and before being done with instance +0 for triggering OnRemoved and OnShutdown and not their deprecated counterparts Change code to call annotations *after* the documentation has been generated for that component. Also invoke both the new annotations and the deprecated annotations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-735) DocGenerator should on @OnUnscheduled and @OnStopped after its done generating the docs
[ https://issues.apache.org/jira/browse/NIFI-735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604312#comment-14604312 ] ASF subversion and git services commented on NIFI-735: -- Commit bd9782cb43ece682c853d9a75279ebe76f968044 in incubator-nifi's branch refs/heads/NIFI-735 from [~danbress] [ https://git-wip-us.apache.org/repos/asf?p=incubator-nifi.git;h=bd9782c ] NIFI-735: DocGenerator should on @OnUnscheduled and @OnStopped after its done generating the docs DocGenerator should on @OnUnscheduled and @OnStopped after its done generating the docs --- Key: NIFI-735 URL: https://issues.apache.org/jira/browse/NIFI-735 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 0.1.0 Reporter: Dan Bress Assignee: Dan Bress Fix For: 0.2.0 Per [~mcgilman] +0 for triggering OnRemoved and OnShutdown after initialization and before being done with instance +0 for triggering OnRemoved and OnShutdown and not their deprecated counterparts Change code to call annotations *after* the documentation has been generated for that component. Also invoke both the new annotations and the deprecated annotations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-735) DocGenerator should on @OnUnscheduled and @OnStopped after its done generating the docs
[ https://issues.apache.org/jira/browse/NIFI-735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Bress updated NIFI-735: --- Attachment: 0001-NIFI-735-DocGenerator-should-on-OnUnscheduled-and-On.patch DocGenerator should on @OnUnscheduled and @OnStopped after its done generating the docs --- Key: NIFI-735 URL: https://issues.apache.org/jira/browse/NIFI-735 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 0.1.0 Reporter: Dan Bress Assignee: Dan Bress Fix For: 0.2.0 Attachments: 0001-NIFI-735-DocGenerator-should-on-OnUnscheduled-and-On.patch Per [~mcgilman] +0 for triggering OnRemoved and OnShutdown after initialization and before being done with instance +0 for triggering OnRemoved and OnShutdown and not their deprecated counterparts Change code to call annotations *after* the documentation has been generated for that component. Also invoke both the new annotations and the deprecated annotations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-737) Typo in developer guide 'lot message'
[ https://issues.apache.org/jira/browse/NIFI-737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Bress updated NIFI-737: --- Attachment: 0001-NIFI-737-Typo-in-developer-guide-lot-message.patch changed 'lot' to 'log' Typo in developer guide 'lot message' - Key: NIFI-737 URL: https://issues.apache.org/jira/browse/NIFI-737 Project: Apache NiFi Issue Type: Improvement Components: Documentation Website Affects Versions: 0.1.0 Reporter: Dan Bress Assignee: Dan Bress Priority: Trivial Fix For: 0.2.0 Attachments: 0001-NIFI-737-Typo-in-developer-guide-lot-message.patch [The Developer Guide|https://nifi.incubator.apache.org/docs/nifi-docs/html/developer-guide.html#onshutdown] has a typo: _If such a method throws an Exception, a *lot* message will be generated, and the Exception will otherwise be ignored; other methods with this annotation will still be invoked. Methods with this annotation must take zero arguments._ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-632) Documentation Generation: Nar ClassLoader
[ https://issues.apache.org/jira/browse/NIFI-632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604316#comment-14604316 ] Dan Bress commented on NIFI-632: Joe, Ah. Sorry about that. Thanks for the quick fix. Dan Documentation Generation: Nar ClassLoader - Key: NIFI-632 URL: https://issues.apache.org/jira/browse/NIFI-632 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 0.1.0 Reporter: Matt Gilman Assignee: Dan Bress Priority: Minor Fix For: 0.2.0 Attachments: 0001-Squashed-commit-of-the-following.patch When auto generating component documentation the NarThreadContextClassLoader needs to be set in case the component attempts to load anything off of its classpath. Without this, the documentation generation fails. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (NIFI-712) Improve documentation of MergeContent
[ https://issues.apache.org/jira/browse/NIFI-712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Bress resolved NIFI-712. Resolution: Duplicate Looks like I created this one twice... Improve documentation of MergeContent - Key: NIFI-712 URL: https://issues.apache.org/jira/browse/NIFI-712 Project: Apache NiFi Issue Type: Improvement Components: Documentation Website Affects Versions: 0.1.0 Reporter: Dan Bress Priority: Minor MergeContent is an incredibly powerful processor, but its [documentation|https://nifi.incubator.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.MergeContent/index.html] is confusing, and makes it hard to understand how to configure it to do what you want. I'm proposing improving the documentation in the following ways: # Provide a graphical picture(a .png) to show how Bins, Bundles, Entries, and Groups relate to one another. Put this in additionalDetails.html. # Pick a term and stick with it. I think we use Bin, Bundle and Group in the documentation interchangeably when they all mean the same thing. Maybe I'm wrong about this, but this is part of my confusion. # I think [this line|https://github.com/apache/incubator-nifi/blob/develop/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/MergeContent.java#L80-L82] is a typo: _It is recommended that the Processor be configured with only a single incoming connection, *as Group of FlowFiles* will not be created from FlowFiles in different connections_ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-735) DocGenerator should on @OnUnscheduled and @OnStopped after its done generating the docs
[ https://issues.apache.org/jira/browse/NIFI-735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604340#comment-14604340 ] Matt Gilman commented on NIFI-735: -- That's a great point. Ultimately all I wanted to do was make sure that the processor's be given the opportunity to clean up any resources/state they may have initialized upon construction or during the init() call. If we think that OnShutdown is sufficient than that's fine by me. DocGenerator should on @OnUnscheduled and @OnStopped after its done generating the docs --- Key: NIFI-735 URL: https://issues.apache.org/jira/browse/NIFI-735 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 0.1.0 Reporter: Dan Bress Assignee: Dan Bress Fix For: 0.2.0 Attachments: 0001-NIFI-735-DocGenerator-should-on-OnUnscheduled-and-On.patch Per [~mcgilman] +0 for triggering OnRemoved and OnShutdown after initialization and before being done with instance +0 for triggering OnRemoved and OnShutdown and not their deprecated counterparts Change code to call annotations *after* the documentation has been generated for that component. Also invoke both the new annotations and the deprecated annotations. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-738) Do not write conversion error messages to flow file content
Ryan Blue created NIFI-738: -- Summary: Do not write conversion error messages to flow file content Key: NIFI-738 URL: https://issues.apache.org/jira/browse/NIFI-738 Project: Apache NiFi Issue Type: Bug Components: Extensions Affects Versions: 0.1.0 Reporter: Ryan Blue Assignee: Ryan Blue Fix For: 0.2.0 NIFI-551 extended the error handling provided by the ConvertJSONToAvro processor, but wrote error messages as the content of a file sent on the failure relationship. I think the right thing to do is to output the bad records as the file content and put the error messages in the outgoing attributes. NIFI-551 wasn't included in 0.1.0, so changing this behavior is safe. Consequently, I'd like to get this fix into 0.2.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-738) Do not write conversion error messages to flow file content
[ https://issues.apache.org/jira/browse/NIFI-738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604453#comment-14604453 ] ASF GitHub Bot commented on NIFI-738: - GitHub user rdblue opened a pull request: https://github.com/apache/incubator-nifi/pull/69 NIFI-738: Improve error handling for CSV and JSON conversion. Changes: * Send bad record information on the incompatible relationship * Use an attribute, errors for summarized error messages * Send no error content for now, original content can't be accessed * Summarize similar error messages with N similar failures * Add similar error handling to CSV conversion You can merge this pull request into a Git repository by running: $ git pull https://github.com/rdblue/incubator-nifi NIFI-738-fix-conversion-error-handling Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-nifi/pull/69.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #69 commit c380e68764aa318e740835011c2e8933f05feb19 Author: Ryan Blue b...@apache.org Date: 2015-06-28T00:36:42Z NIFI-738: Improve error handling for CSV and JSON conversion. Changes: * Send bad record information on the incompatible relationship * Use an attribute, errors for summarized error messages * Send no error content for now, original content can't be accessed * Summarize similar error messages with N similar failures * Add similar error handling to CSV conversion Do not write conversion error messages to flow file content --- Key: NIFI-738 URL: https://issues.apache.org/jira/browse/NIFI-738 Project: Apache NiFi Issue Type: Bug Components: Extensions Affects Versions: 0.1.0 Reporter: Ryan Blue Assignee: Ryan Blue Fix For: 0.2.0 NIFI-551 extended the error handling provided by the ConvertJSONToAvro processor, but wrote error messages as the content of a file sent on the failure relationship. I think the right thing to do is to output the bad records as the file content and put the error messages in the outgoing attributes. NIFI-551 wasn't included in 0.1.0, so changing this behavior is safe. Consequently, I'd like to get this fix into 0.2.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-589) Add processors that can run Apache Flume sources/sinks
[ https://issues.apache.org/jira/browse/NIFI-589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604529#comment-14604529 ] ASF GitHub Bot commented on NIFI-589: - Github user rdblue commented on the pull request: https://github.com/apache/incubator-nifi/pull/51#issuecomment-116191838 Is there any service provider construct or similar mechanism whereby we could provide a dropdown of identified sources and sinks I think this is primarily intended to allow people to use their current Flume config inside of NiFi and pave the way for more seamless integration as a next step. I would say we want to keep this as-is for compatibility (just paste in your current config) and create processors to wrap the widely used Flume components, like the Avro source. I think because channel.setSession(…) is being called as it is during the onTrigger method this means the processor needs to run serially Good point. Maybe we can keep a thread-local cache of the sessions and pass an accessor to the channel instead? Then we would be able to keep parallel execution. Add processors that can run Apache Flume sources/sinks -- Key: NIFI-589 URL: https://issues.apache.org/jira/browse/NIFI-589 Project: Apache NiFi Issue Type: New Feature Components: Extensions Affects Versions: 0.1.0 Reporter: Joey Echeverria Assignee: Joey Echeverria -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-738) Do not write conversion error messages to flow file content
[ https://issues.apache.org/jira/browse/NIFI-738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604480#comment-14604480 ] ASF GitHub Bot commented on NIFI-738: - Github user joewitt commented on the pull request: https://github.com/apache/incubator-nifi/pull/69#issuecomment-116186741 Builds cleanly including the contribution check for licenses and check style. ConvertCSVToAvro - Line 258 should use parameters passed in via object[] rather than string concatenation. - What happens if you have processed an entire input and produced no successful results due to a DatasetRecordExceptions? I think it would forward an empty flow file since nothing would have been written to it. Is that what you want? Such a case seems like a FAILURE to me. - Consider removing the incompatible relationship and simply adding an attribute to the success flow file which lists the reasons or add an attribute per reason for difficulty converting the original. If you will keep the incompatible relationship then I’d recommend sending it a clone of the original item and then adding the relationship. That would make troubleshooting or analysis of that object much easier. In any case be very careful about how large these attributes could become. If you’re storing a series of stack traces this could become unwieldy and memory inefficient really fast. ConvertJSONToAvro - Line 160 should use parameters passed in via object[] rather than string concatenation. - Same comments as Convert CSV to Avro. FailureTracker - This class is a cool idea. My concern with it is that it appears to store all the raw reasons which could be quite significant in large files with *many* thousands of lines. This could create issues with the heap, cause excessive GC, etc. Consider refactoring this class to both immediately store the summarized representation and to bound the total number of reasons it will store. It can just ignore additional errors as they arrive. This will allow it to behave well in terms of memory consumption even in the presence of massive failures as everything will be bounded. Thanks Joe Do not write conversion error messages to flow file content --- Key: NIFI-738 URL: https://issues.apache.org/jira/browse/NIFI-738 Project: Apache NiFi Issue Type: Bug Components: Extensions Affects Versions: 0.1.0 Reporter: Ryan Blue Assignee: Ryan Blue Fix For: 0.2.0 NIFI-551 extended the error handling provided by the ConvertJSONToAvro processor, but wrote error messages as the content of a file sent on the failure relationship. I think the right thing to do is to output the bad records as the file content and put the error messages in the outgoing attributes. NIFI-551 wasn't included in 0.1.0, so changing this behavior is safe. Consequently, I'd like to get this fix into 0.2.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
