[jira] [Updated] (NIFI-3863) Create a LookupRecord Processor
[ https://issues.apache.org/jira/browse/NIFI-3863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-3863: -- Resolution: Fixed Status: Resolved (was: Patch Available) +1 merged to master Also uploaded a good template used to test and validate this thing. Very very powerful! > Create a LookupRecord Processor > --- > > Key: NIFI-3863 > URL: https://issues.apache.org/jira/browse/NIFI-3863 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne > Fix For: 1.3.0 > > Attachments: template-for-lookup-parition-provenance-records.xml > > > We should provide a mechanism to fetch a value from a Record, and then lookup > that value in a "lookup table." If we get a match, we should add the result > back into a field in the Record. > The Processor would be configured with a Record Reader and a Record Writer. > It would also take a RecordPath as a 'Lookup Path' that indicates where in > the Record the desired value should be pulled from. We then lookup the value > from that field. There should also be a 'Destination Path' property that > indicates where the value should be added back to the record. The record > should then be routed to either a 'matched' relationship if there was a value > in the lookup table, or an 'unmatched' relationship if there was no value in > the lookup table. > The 'Destination Path' property should be optional. This way, if it is set, > this Processor would function as an 'enrichment processor.' If the property > is not set, this Processor would function as a 'routing processor', by > routing to either 'matched' or 'unmatched' depending on whether or not its > value is found in the Lookup Table. > This means that we also need to build a Lookup Table type of Controller > Service. It would be best to provide two implementations: a Properties > File-based lookup service and a Simple Key-Value Lookup Service. The Simple > Key-Value service would just allow the user to add properties where the > Property Name is the Key and the Property Value is the value. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3857) Create PartitionRecord processor
[ https://issues.apache.org/jira/browse/NIFI-3857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-3857: -- Resolution: Fixed Status: Resolved (was: Patch Available) +1 merged to master > Create PartitionRecord processor > > > Key: NIFI-3857 > URL: https://issues.apache.org/jira/browse/NIFI-3857 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne > Fix For: 1.3.0 > > > With the record readers and writers, it will be useful to be able to bring in > a stream of records as a FlowFile and output N different FlowFiles, where all > records in the output are "alike." Users should be able to define what it > means for two records to be "like records" by entering one or more > RecordPaths. > Since we know that all records in an outgoing FlowFile will have the same > value for a given field, we should also add an attribute to the FlowFile for > each of these, so that routing can easily be done and we can also track that > attribute via Provenance, etc. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3857) Create PartitionRecord processor
[ https://issues.apache.org/jira/browse/NIFI-3857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016941#comment-16016941 ] ASF subversion and git services commented on NIFI-3857: --- Commit ae9953db646f54f3731c56c8ff97503aacc041f3 in nifi's branch refs/heads/master from [~markap14] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=ae9953d ] NIFI-3857: This closes #1825. Added PartitionRecord processor Signed-off-by: joewitt > Create PartitionRecord processor > > > Key: NIFI-3857 > URL: https://issues.apache.org/jira/browse/NIFI-3857 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne > Fix For: 1.3.0 > > > With the record readers and writers, it will be useful to be able to bring in > a stream of records as a FlowFile and output N different FlowFiles, where all > records in the output are "alike." Users should be able to define what it > means for two records to be "like records" by entering one or more > RecordPaths. > Since we know that all records in an outgoing FlowFile will have the same > value for a given field, we should also add an attribute to the FlowFile for > each of these, so that routing can easily be done and we can also track that > attribute via Provenance, etc. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3863) Create a LookupRecord Processor
[ https://issues.apache.org/jira/browse/NIFI-3863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-3863: -- Fix Version/s: 1.3.0 > Create a LookupRecord Processor > --- > > Key: NIFI-3863 > URL: https://issues.apache.org/jira/browse/NIFI-3863 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne > Fix For: 1.3.0 > > Attachments: template-for-lookup-parition-provenance-records.xml > > > We should provide a mechanism to fetch a value from a Record, and then lookup > that value in a "lookup table." If we get a match, we should add the result > back into a field in the Record. > The Processor would be configured with a Record Reader and a Record Writer. > It would also take a RecordPath as a 'Lookup Path' that indicates where in > the Record the desired value should be pulled from. We then lookup the value > from that field. There should also be a 'Destination Path' property that > indicates where the value should be added back to the record. The record > should then be routed to either a 'matched' relationship if there was a value > in the lookup table, or an 'unmatched' relationship if there was no value in > the lookup table. > The 'Destination Path' property should be optional. This way, if it is set, > this Processor would function as an 'enrichment processor.' If the property > is not set, this Processor would function as a 'routing processor', by > routing to either 'matched' or 'unmatched' depending on whether or not its > value is found in the Lookup Table. > This means that we also need to build a Lookup Table type of Controller > Service. It would be best to provide two implementations: a Properties > File-based lookup service and a Simple Key-Value Lookup Service. The Simple > Key-Value service would just allow the user to add properties where the > Property Name is the Key and the Property Value is the value. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3863) Create a LookupRecord Processor
[ https://issues.apache.org/jira/browse/NIFI-3863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016940#comment-16016940 ] ASF subversion and git services commented on NIFI-3863: --- Commit 9bd0246a96cfcda01b972834823250a9cddf77a8 in nifi's branch refs/heads/master from [~markap14] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=9bd0246 ] NIFI-3863: Initial implementation of Lookup Services. Implemented LookupRecord processors. This required some refactoring of RecordSetWriter interface, so refactored that interface and all implementations and references of it > Create a LookupRecord Processor > --- > > Key: NIFI-3863 > URL: https://issues.apache.org/jira/browse/NIFI-3863 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne > Fix For: 1.3.0 > > Attachments: template-for-lookup-parition-provenance-records.xml > > > We should provide a mechanism to fetch a value from a Record, and then lookup > that value in a "lookup table." If we get a match, we should add the result > back into a field in the Record. > The Processor would be configured with a Record Reader and a Record Writer. > It would also take a RecordPath as a 'Lookup Path' that indicates where in > the Record the desired value should be pulled from. We then lookup the value > from that field. There should also be a 'Destination Path' property that > indicates where the value should be added back to the record. The record > should then be routed to either a 'matched' relationship if there was a value > in the lookup table, or an 'unmatched' relationship if there was no value in > the lookup table. > The 'Destination Path' property should be optional. This way, if it is set, > this Processor would function as an 'enrichment processor.' If the property > is not set, this Processor would function as a 'routing processor', by > routing to either 'matched' or 'unmatched' depending on whether or not its > value is found in the Lookup Table. > This means that we also need to build a Lookup Table type of Controller > Service. It would be best to provide two implementations: a Properties > File-based lookup service and a Simple Key-Value Lookup Service. The Simple > Key-Value service would just allow the user to add properties where the > Property Name is the Key and the Property Value is the value. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi pull request #1825: NIFI 3863 - Add LookupRecord, PartitionRecord proce...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1825 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (NIFI-3863) Create a LookupRecord Processor
[ https://issues.apache.org/jira/browse/NIFI-3863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-3863: -- Attachment: template-for-lookup-parition-provenance-records.xml this is a great template to test all this new goodness with > Create a LookupRecord Processor > --- > > Key: NIFI-3863 > URL: https://issues.apache.org/jira/browse/NIFI-3863 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne > Attachments: template-for-lookup-parition-provenance-records.xml > > > We should provide a mechanism to fetch a value from a Record, and then lookup > that value in a "lookup table." If we get a match, we should add the result > back into a field in the Record. > The Processor would be configured with a Record Reader and a Record Writer. > It would also take a RecordPath as a 'Lookup Path' that indicates where in > the Record the desired value should be pulled from. We then lookup the value > from that field. There should also be a 'Destination Path' property that > indicates where the value should be added back to the record. The record > should then be routed to either a 'matched' relationship if there was a value > in the lookup table, or an 'unmatched' relationship if there was no value in > the lookup table. > The 'Destination Path' property should be optional. This way, if it is set, > this Processor would function as an 'enrichment processor.' If the property > is not set, this Processor would function as a 'routing processor', by > routing to either 'matched' or 'unmatched' depending on whether or not its > value is found in the Lookup Table. > This means that we also need to build a Lookup Table type of Controller > Service. It would be best to provide two implementations: a Properties > File-based lookup service and a Simple Key-Value Lookup Service. The Simple > Key-Value service would just allow the user to add properties where the > Property Name is the Key and the Property Value is the value. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (NIFI-3942) IPLookupService should auto reload geo dictionary file
Joseph Witt created NIFI-3942: - Summary: IPLookupService should auto reload geo dictionary file Key: NIFI-3942 URL: https://issues.apache.org/jira/browse/NIFI-3942 Project: Apache NiFi Issue Type: Bug Affects Versions: 1.3.0 Reporter: Joseph Witt Right now the IPLookupService introduced in NIFI-3863 loads the database onEnabled for the controller service. It needs to periodically check for a new database file and reload it automatically during calls to lookup. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi pull request #1825: NIFI 3863 - Add LookupRecord, PartitionRecord proce...
Github user joewitt commented on a diff in the pull request: https://github.com/apache/nifi/pull/1825#discussion_r117406662 --- Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/record/script/ScriptedRecordSetWriter.java --- @@ -131,6 +129,7 @@ protected boolean reloadScript(final String scriptBody) { } } catch (final Exception ex) { +ex.printStackTrace(); --- End diff -- removing this and addressing a couple checkstyle issues --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Resolved] (NIFI-3940) older jruby-complete artifacts contained LGPL code
[ https://issues.apache.org/jira/browse/NIFI-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy LoPresto resolved NIFI-3940. - Resolution: Fixed > older jruby-complete artifacts contained LGPL code > -- > > Key: NIFI-3940 > URL: https://issues.apache.org/jira/browse/NIFI-3940 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.2.0 >Reporter: Joseph Witt >Assignee: Joseph Witt >Priority: Blocker > Fix For: 1.3.0 > > > From Mike Drob's report to legal > Legal, > I recently got done working with the JRuby dev team on an issue where they > were inadvertently including LGPL code in the jruby-complete jar for versions > 9.0.0.0 - 9.1.8.0. > I'm not sure if we have a good mechanism for tracking which projects are > using/bundling JRuby, but it would be good to let them know that they should > consider updating for license reasons. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3940) older jruby-complete artifacts contained LGPL code
[ https://issues.apache.org/jira/browse/NIFI-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016890#comment-16016890 ] ASF GitHub Bot commented on NIFI-3940: -- Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/1827 Ran a full build on OS X and everything worked. Verified `contrib-check` as well. Merged. > older jruby-complete artifacts contained LGPL code > -- > > Key: NIFI-3940 > URL: https://issues.apache.org/jira/browse/NIFI-3940 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.2.0 >Reporter: Joseph Witt >Assignee: Joseph Witt >Priority: Blocker > Fix For: 1.3.0 > > > From Mike Drob's report to legal > Legal, > I recently got done working with the JRuby dev team on an issue where they > were inadvertently including LGPL code in the jruby-complete jar for versions > 9.0.0.0 - 9.1.8.0. > I'm not sure if we have a good mechanism for tracking which projects are > using/bundling JRuby, but it would be good to let them know that they should > consider updating for license reasons. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi issue #1827: NIFI-3940 upgraded to latest jruby which resolves an appar...
Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/1827 Ran a full build on OS X and everything worked. Verified `contrib-check` as well. Merged. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3940) older jruby-complete artifacts contained LGPL code
[ https://issues.apache.org/jira/browse/NIFI-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016889#comment-16016889 ] ASF GitHub Bot commented on NIFI-3940: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1827 > older jruby-complete artifacts contained LGPL code > -- > > Key: NIFI-3940 > URL: https://issues.apache.org/jira/browse/NIFI-3940 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.2.0 >Reporter: Joseph Witt >Assignee: Joseph Witt >Priority: Blocker > Fix For: 1.3.0 > > > From Mike Drob's report to legal > Legal, > I recently got done working with the JRuby dev team on an issue where they > were inadvertently including LGPL code in the jruby-complete jar for versions > 9.0.0.0 - 9.1.8.0. > I'm not sure if we have a good mechanism for tracking which projects are > using/bundling JRuby, but it would be good to let them know that they should > consider updating for license reasons. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3940) older jruby-complete artifacts contained LGPL code
[ https://issues.apache.org/jira/browse/NIFI-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016888#comment-16016888 ] ASF subversion and git services commented on NIFI-3940: --- Commit 3b98abbf416974a7e94ef4e1ca3fc169bbf66393 in nifi's branch refs/heads/master from [~joewitt] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=3b98abb ] NIFI-3940 upgraded to latest jruby which resolves an apparent lgpl bunding issue This closes #1827. Signed-off-by: Andy LoPresto > older jruby-complete artifacts contained LGPL code > -- > > Key: NIFI-3940 > URL: https://issues.apache.org/jira/browse/NIFI-3940 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.2.0 >Reporter: Joseph Witt >Assignee: Joseph Witt >Priority: Blocker > Fix For: 1.3.0 > > > From Mike Drob's report to legal > Legal, > I recently got done working with the JRuby dev team on an issue where they > were inadvertently including LGPL code in the jruby-complete jar for versions > 9.0.0.0 - 9.1.8.0. > I'm not sure if we have a good mechanism for tracking which projects are > using/bundling JRuby, but it would be good to let them know that they should > consider updating for license reasons. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi pull request #1827: NIFI-3940 upgraded to latest jruby which resolves a...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1827 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3940) older jruby-complete artifacts contained LGPL code
[ https://issues.apache.org/jira/browse/NIFI-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016866#comment-16016866 ] ASF GitHub Bot commented on NIFI-3940: -- Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/1827 unreal. Great find. Pushing squashed in a sec. Thanks! > older jruby-complete artifacts contained LGPL code > -- > > Key: NIFI-3940 > URL: https://issues.apache.org/jira/browse/NIFI-3940 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.2.0 >Reporter: Joseph Witt >Assignee: Joseph Witt >Priority: Blocker > Fix For: 1.3.0 > > > From Mike Drob's report to legal > Legal, > I recently got done working with the JRuby dev team on an issue where they > were inadvertently including LGPL code in the jruby-complete jar for versions > 9.0.0.0 - 9.1.8.0. > I'm not sure if we have a good mechanism for tracking which projects are > using/bundling JRuby, but it would be good to let them know that they should > consider updating for license reasons. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi issue #1827: NIFI-3940 upgraded to latest jruby which resolves an appar...
Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/1827 unreal. Great find. Pushing squashed in a sec. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1827: NIFI-3940 upgraded to latest jruby which resolves an appar...
Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/1827 The issue is that `jython` uses a binary build instead of Maven so it is an "unshaded JAR" (described in far more detail [here](https://github.com/scijava/jython-shaded)). It includes libraries that conflict with `jruby` but Maven can't resolve them, so there are runtime errors. Rather than separate the dependencies into separate modules, we can use the `jython-shaded` dependency mentioned above to resolve this. I can't push the changes to this branch, but you can make the following changes in `nifi-scripting-processor/pom.xml`: ``` org.python jython-standalone 2.7.1b3 ``` with ``` org.scijava jython-shaded 2.7.0 ``` Tests all pass: ``` --- T E S T S --- Running org.apache.nifi.processors.script.ExecuteScriptGroovyTest Tests run: 4, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 2.533 sec - in org.apache.nifi.processors.script.ExecuteScriptGroovyTest Running org.apache.nifi.processors.script.TestExecuteClojure Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.029 sec - in org.apache.nifi.processors.script.TestExecuteClojure Running org.apache.nifi.processors.script.TestExecuteGroovy Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.648 sec - in org.apache.nifi.processors.script.TestExecuteGroovy Running org.apache.nifi.processors.script.TestExecuteJavascript Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.979 sec - in org.apache.nifi.processors.script.TestExecuteJavascript Running org.apache.nifi.processors.script.TestExecuteJRuby Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.77 sec - in org.apache.nifi.processors.script.TestExecuteJRuby Running org.apache.nifi.processors.script.TestExecuteJython Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.72 sec - in org.apache.nifi.processors.script.TestExecuteJython Running org.apache.nifi.processors.script.TestExecuteLua Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.068 sec - in org.apache.nifi.processors.script.TestExecuteLua Running org.apache.nifi.processors.script.TestInvokeGroovy Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.517 sec - in org.apache.nifi.processors.script.TestInvokeGroovy Running org.apache.nifi.processors.script.TestInvokeJavascript Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.396 sec - in org.apache.nifi.processors.script.TestInvokeJavascript Running org.apache.nifi.processors.script.TestInvokeJython Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.263 sec - in org.apache.nifi.processors.script.TestInvokeJython Running org.apache.nifi.record.script.ScriptedReaderTest Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.515 sec - in org.apache.nifi.record.script.ScriptedReaderTest Running org.apache.nifi.record.script.ScriptedRecordSetWriterTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.24 sec - in org.apache.nifi.record.script.ScriptedRecordSetWriterTest Running org.apache.nifi.reporting.script.ScriptedReportingTaskGroovyTest Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.269 sec - in org.apache.nifi.reporting.script.ScriptedReportingTaskGroovyTest Results : Tests run: 45, Failures: 0, Errors: 0, Skipped: 1 ``` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3940) older jruby-complete artifacts contained LGPL code
[ https://issues.apache.org/jira/browse/NIFI-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016865#comment-16016865 ] ASF GitHub Bot commented on NIFI-3940: -- Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/1827 The issue is that `jython` uses a binary build instead of Maven so it is an "unshaded JAR" (described in far more detail [here](https://github.com/scijava/jython-shaded)). It includes libraries that conflict with `jruby` but Maven can't resolve them, so there are runtime errors. Rather than separate the dependencies into separate modules, we can use the `jython-shaded` dependency mentioned above to resolve this. I can't push the changes to this branch, but you can make the following changes in `nifi-scripting-processor/pom.xml`: ``` org.python jython-standalone 2.7.1b3 ``` with ``` org.scijava jython-shaded 2.7.0 ``` Tests all pass: ``` --- T E S T S --- Running org.apache.nifi.processors.script.ExecuteScriptGroovyTest Tests run: 4, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 2.533 sec - in org.apache.nifi.processors.script.ExecuteScriptGroovyTest Running org.apache.nifi.processors.script.TestExecuteClojure Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.029 sec - in org.apache.nifi.processors.script.TestExecuteClojure Running org.apache.nifi.processors.script.TestExecuteGroovy Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.648 sec - in org.apache.nifi.processors.script.TestExecuteGroovy Running org.apache.nifi.processors.script.TestExecuteJavascript Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.979 sec - in org.apache.nifi.processors.script.TestExecuteJavascript Running org.apache.nifi.processors.script.TestExecuteJRuby Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.77 sec - in org.apache.nifi.processors.script.TestExecuteJRuby Running org.apache.nifi.processors.script.TestExecuteJython Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.72 sec - in org.apache.nifi.processors.script.TestExecuteJython Running org.apache.nifi.processors.script.TestExecuteLua Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.068 sec - in org.apache.nifi.processors.script.TestExecuteLua Running org.apache.nifi.processors.script.TestInvokeGroovy Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.517 sec - in org.apache.nifi.processors.script.TestInvokeGroovy Running org.apache.nifi.processors.script.TestInvokeJavascript Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.396 sec - in org.apache.nifi.processors.script.TestInvokeJavascript Running org.apache.nifi.processors.script.TestInvokeJython Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.263 sec - in org.apache.nifi.processors.script.TestInvokeJython Running org.apache.nifi.record.script.ScriptedReaderTest Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.515 sec - in org.apache.nifi.record.script.ScriptedReaderTest Running org.apache.nifi.record.script.ScriptedRecordSetWriterTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.24 sec - in org.apache.nifi.record.script.ScriptedRecordSetWriterTest Running org.apache.nifi.reporting.script.ScriptedReportingTaskGroovyTest Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.269 sec - in org.apache.nifi.reporting.script.ScriptedReportingTaskGroovyTest Results : Tests run: 45, Failures: 0, Errors: 0, Skipped: 1 ``` > older jruby-complete artifacts contained LGPL code > -- > > Key: NIFI-3940 > URL: https://issues.apache.org/jira/browse/NIFI-3940 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.2.0 >Reporter: Joseph Witt >Assignee: Joseph Witt >Priority: Blocker > Fix For: 1.3.0 > > > From Mike Drob's report to legal > Legal, > I recently got done working with the JRuby dev team on an issue where they > were inadvertently including LGPL code in the jruby-complete jar for versions > 9.0.0.0 - 9.1.8.0. > I'm not sure if we have a good mechanism for tracking which projects are > using/bundling JRuby, but it would be good to let them know that they should > consider updating for license reasons. -- This message was sent by Atlassian JIRA (v6.3.15#6346
[jira] [Commented] (NIFI-3911) Improve documentation for Controller Services scoping
[ https://issues.apache.org/jira/browse/NIFI-3911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016843#comment-16016843 ] Andrew Lim commented on NIFI-3911: -- Just wanted to note that [~mosermw] did add a sentence to the 0.7.0 to 1.0.0 section of the Migration Guide as he suggested in his comment above. Specifically, the sentence added was: Controller Services that will be used by Processors must be defined in the Operate Palette of the root process group or sub process group where they will be used. Controller Services defined in the Global - Controller Settings window can only be used by Reporting Tasks, not by any Processors. Thanks [~mosermw] for doing that! > Improve documentation for Controller Services scoping > - > > Key: NIFI-3911 > URL: https://issues.apache.org/jira/browse/NIFI-3911 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation & Website >Reporter: Andrew Lim >Assignee: Andrew Lim >Priority: Minor > > Filing to revisit and improve the documentation on Controller Services > scoping which has caused some confusion for users. There have been some > improvements made to improve the UX (NIFI-3128) but the doc can perhaps due a > better job highlighting this change in functionality from 0.x NiFi. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (NIFI-3941) Clarify tab name for controller level controller services dialog
Andy LoPresto created NIFI-3941: --- Summary: Clarify tab name for controller level controller services dialog Key: NIFI-3941 URL: https://issues.apache.org/jira/browse/NIFI-3941 Project: Apache NiFi Issue Type: Improvement Components: Core UI Affects Versions: 1.2.0 Reporter: Andy LoPresto Priority: Minor As a follow-on to [NIFI-3911|https://issues.apache.org/jira/browse/NIFI-3911], I believe the *Controller Services* tab in the "global"/"controller level" *Controller Settings* dialog should be renamed to *Reporting Task Controller Services* and a sentence added above the table explaining that any controller services added here are only accessible by reporting tasks and not accessible by any components on the flow itself. {quote} Would it make sense to change the title of the tab in the global Controller Settings dialog to "Reporting Task Controller Services" and add a sentence description above the table stating that these controller services are only accessible by the Reporting Tasks? {quote} This is helpful for users who do not read the extensive documentation in the User Guide. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3940) older jruby-complete artifacts contained LGPL code
[ https://issues.apache.org/jira/browse/NIFI-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016836#comment-16016836 ] ASF GitHub Bot commented on NIFI-3940: -- Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/1827 Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.231 sec <<< FAILURE! - in org.apache.nifi.processors.script.TestExecuteJRuby testReadFlowFileContentAndStoreInFlowFileAttribute(org.apache.nifi.processors.script.TestExecuteJRuby) Time elapsed: 1.205 sec <<< FAILURE! java.lang.AssertionError: java.lang.NoSuchFieldError: O_TMPFILE at org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:201) at org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:160) at org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:155) at org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:150) at org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:145) at org.apache.nifi.processors.script.TestExecuteJRuby.testReadFlowFileContentAndStoreInFlowFileAttribute(TestExecuteJRuby.java:52) Caused by: java.lang.NoSuchFieldError: O_TMPFILE at org.jruby.RubyFile.createFileClass(RubyFile.java:202) at org.jruby.Ruby.initCore(Ruby.java:1490) at org.jruby.Ruby.bootstrap(Ruby.java:1274) at org.jruby.Ruby.init(Ruby.java:1173) at org.jruby.Ruby.newInstance(Ruby.java:338) at org.jruby.embed.internal.AbstractLocalContextProvider.getGlobalRuntime(AbstractLocalContextProvider.java:82) at org.jruby.embed.internal.SingletonLocalContextProvider.getRuntime(SingletonLocalContextProvider.java:99) at org.jruby.embed.internal.BiVariableMap.getRuntime(BiVariableMap.java:109) at org.jruby.embed.internal.BiVariableMap.getTopSelf(BiVariableMap.java:241) at org.jruby.embed.internal.BiVariableMap.getReceiverObject(BiVariableMap.java:237) at org.jruby.embed.internal.BiVariableMap.put(BiVariableMap.java:332) at org.jruby.embed.ScriptingContainer.put(ScriptingContainer.java:1185) at org.jruby.embed.jsr223.Utils.put(Utils.java:234) at org.jruby.embed.jsr223.Utils.preEval(Utils.java:109) at org.jruby.embed.jsr223.JRubyEngine.eval(JRubyEngine.java:88) at org.jruby.embed.jsr223.JRubyEngine.eval(JRubyEngine.java:142) at org.apache.nifi.processors.script.ExecuteScript.onTrigger(ExecuteScript.java:220) at org.apache.nifi.util.StandardProcessorTestRunner$RunProcessor.call(StandardProcessorTestRunner.java:251) at org.apache.nifi.util.StandardProcessorTestRunner$RunProcessor.call(StandardProcessorTestRunner.java:245) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) > older jruby-complete artifacts contained LGPL code > -- > > Key: NIFI-3940 > URL: https://issues.apache.org/jira/browse/NIFI-3940 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.2.0 >Reporter: Joseph Witt >Assignee: Joseph Witt >Priority: Blocker > Fix For: 1.3.0 > > > From Mike Drob's report to legal > Legal, > I recently got done working with the JRuby dev team on an issue where they > were inadvertently including LGPL code in the jruby-complete jar for versions > 9.0.0.0 - 9.1.8.0. > I'm not sure if we have a good mechanism for tracking which projects are > using/bundling JRuby, but it would be good to let them know that they should > consider updating for license reasons. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi issue #1827: NIFI-3940 upgraded to latest jruby which resolves an appar...
Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/1827 Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.231 sec <<< FAILURE! - in org.apache.nifi.processors.script.TestExecuteJRuby testReadFlowFileContentAndStoreInFlowFileAttribute(org.apache.nifi.processors.script.TestExecuteJRuby) Time elapsed: 1.205 sec <<< FAILURE! java.lang.AssertionError: java.lang.NoSuchFieldError: O_TMPFILE at org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:201) at org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:160) at org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:155) at org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:150) at org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:145) at org.apache.nifi.processors.script.TestExecuteJRuby.testReadFlowFileContentAndStoreInFlowFileAttribute(TestExecuteJRuby.java:52) Caused by: java.lang.NoSuchFieldError: O_TMPFILE at org.jruby.RubyFile.createFileClass(RubyFile.java:202) at org.jruby.Ruby.initCore(Ruby.java:1490) at org.jruby.Ruby.bootstrap(Ruby.java:1274) at org.jruby.Ruby.init(Ruby.java:1173) at org.jruby.Ruby.newInstance(Ruby.java:338) at org.jruby.embed.internal.AbstractLocalContextProvider.getGlobalRuntime(AbstractLocalContextProvider.java:82) at org.jruby.embed.internal.SingletonLocalContextProvider.getRuntime(SingletonLocalContextProvider.java:99) at org.jruby.embed.internal.BiVariableMap.getRuntime(BiVariableMap.java:109) at org.jruby.embed.internal.BiVariableMap.getTopSelf(BiVariableMap.java:241) at org.jruby.embed.internal.BiVariableMap.getReceiverObject(BiVariableMap.java:237) at org.jruby.embed.internal.BiVariableMap.put(BiVariableMap.java:332) at org.jruby.embed.ScriptingContainer.put(ScriptingContainer.java:1185) at org.jruby.embed.jsr223.Utils.put(Utils.java:234) at org.jruby.embed.jsr223.Utils.preEval(Utils.java:109) at org.jruby.embed.jsr223.JRubyEngine.eval(JRubyEngine.java:88) at org.jruby.embed.jsr223.JRubyEngine.eval(JRubyEngine.java:142) at org.apache.nifi.processors.script.ExecuteScript.onTrigger(ExecuteScript.java:220) at org.apache.nifi.util.StandardProcessorTestRunner$RunProcessor.call(StandardProcessorTestRunner.java:251) at org.apache.nifi.util.StandardProcessorTestRunner$RunProcessor.call(StandardProcessorTestRunner.java:245) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Resolved] (NIFI-3911) Improve documentation for Controller Services scoping
[ https://issues.apache.org/jira/browse/NIFI-3911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy LoPresto resolved NIFI-3911. - Resolution: Fixed > Improve documentation for Controller Services scoping > - > > Key: NIFI-3911 > URL: https://issues.apache.org/jira/browse/NIFI-3911 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation & Website >Reporter: Andrew Lim >Assignee: Andrew Lim >Priority: Minor > > Filing to revisit and improve the documentation on Controller Services > scoping which has caused some confusion for users. There have been some > improvements made to improve the UX (NIFI-3128) but the doc can perhaps due a > better job highlighting this change in functionality from 0.x NiFi. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3940) older jruby-complete artifacts contained LGPL code
[ https://issues.apache.org/jira/browse/NIFI-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016833#comment-16016833 ] ASF GitHub Bot commented on NIFI-3940: -- Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/1827 isn't quite right yet. Breaks a test. I am not a ruby expert so phoning a friend. Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.446 sec - in org.apache.nifi.reporting.script.ScriptedReportingTaskGroovyTest Results : Failed tests: TestExecuteJRuby.testReadFlowFileContentAndStoreInFlowFileAttribute:52 java.lang.NoSuchFieldError: O_TMPFILE > older jruby-complete artifacts contained LGPL code > -- > > Key: NIFI-3940 > URL: https://issues.apache.org/jira/browse/NIFI-3940 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.2.0 >Reporter: Joseph Witt >Assignee: Joseph Witt >Priority: Blocker > Fix For: 1.3.0 > > > From Mike Drob's report to legal > Legal, > I recently got done working with the JRuby dev team on an issue where they > were inadvertently including LGPL code in the jruby-complete jar for versions > 9.0.0.0 - 9.1.8.0. > I'm not sure if we have a good mechanism for tracking which projects are > using/bundling JRuby, but it would be good to let them know that they should > consider updating for license reasons. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3940) older jruby-complete artifacts contained LGPL code
[ https://issues.apache.org/jira/browse/NIFI-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016834#comment-16016834 ] ASF GitHub Bot commented on NIFI-3940: -- Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/1827 Taking a look. > older jruby-complete artifacts contained LGPL code > -- > > Key: NIFI-3940 > URL: https://issues.apache.org/jira/browse/NIFI-3940 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.2.0 >Reporter: Joseph Witt >Assignee: Joseph Witt >Priority: Blocker > Fix For: 1.3.0 > > > From Mike Drob's report to legal > Legal, > I recently got done working with the JRuby dev team on an issue where they > were inadvertently including LGPL code in the jruby-complete jar for versions > 9.0.0.0 - 9.1.8.0. > I'm not sure if we have a good mechanism for tracking which projects are > using/bundling JRuby, but it would be good to let them know that they should > consider updating for license reasons. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi issue #1827: NIFI-3940 upgraded to latest jruby which resolves an appar...
Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/1827 Taking a look. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1827: NIFI-3940 upgraded to latest jruby which resolves an appar...
Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/1827 isn't quite right yet. Breaks a test. I am not a ruby expert so phoning a friend. Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.446 sec - in org.apache.nifi.reporting.script.ScriptedReportingTaskGroovyTest Results : Failed tests: TestExecuteJRuby.testReadFlowFileContentAndStoreInFlowFileAttribute:52 java.lang.NoSuchFieldError: O_TMPFILE --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3940) older jruby-complete artifacts contained LGPL code
[ https://issues.apache.org/jira/browse/NIFI-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016832#comment-16016832 ] ASF GitHub Bot commented on NIFI-3940: -- GitHub user joewitt opened a pull request: https://github.com/apache/nifi/pull/1827 NIFI-3940 upgraded to latest jruby which resolves an apparent lgpl bu… …nding issue Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/joewitt/incubator-nifi NIFI-3940 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1827.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1827 commit a7432f257ad09d73b724e69ed520ab8627935d9c Author: joewitt Date: 2017-05-19T02:57:49Z NIFI-3940 upgraded to latest jruby which resolves an apparent lgpl bunding issue > older jruby-complete artifacts contained LGPL code > -- > > Key: NIFI-3940 > URL: https://issues.apache.org/jira/browse/NIFI-3940 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.2.0 >Reporter: Joseph Witt >Assignee: Joseph Witt >Priority: Blocker > Fix For: 1.3.0 > > > From Mike Drob's report to legal > Legal, > I recently got done working with the JRuby dev team on an issue where they > were inadvertently including LGPL code in the jruby-complete jar for versions > 9.0.0.0 - 9.1.8.0. > I'm not sure if we have a good mechanism for tracking which projects are > using/bundling JRuby, but it would be good to let them know that they should > consider updating for license reasons. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi pull request #1827: NIFI-3940 upgraded to latest jruby which resolves a...
GitHub user joewitt opened a pull request: https://github.com/apache/nifi/pull/1827 NIFI-3940 upgraded to latest jruby which resolves an apparent lgpl bu⦠â¦nding issue Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/joewitt/incubator-nifi NIFI-3940 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1827.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1827 commit a7432f257ad09d73b724e69ed520ab8627935d9c Author: joewitt Date: 2017-05-19T02:57:49Z NIFI-3940 upgraded to latest jruby which resolves an apparent lgpl bunding issue --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (NIFI-3940) older jruby-complete artifacts contained LGPL code
Joseph Witt created NIFI-3940: - Summary: older jruby-complete artifacts contained LGPL code Key: NIFI-3940 URL: https://issues.apache.org/jira/browse/NIFI-3940 Project: Apache NiFi Issue Type: Bug Components: Extensions Affects Versions: 1.2.0 Reporter: Joseph Witt Assignee: Joseph Witt Priority: Blocker Fix For: 1.3.0 >From Mike Drob's report to legal Legal, I recently got done working with the JRuby dev team on an issue where they were inadvertently including LGPL code in the jruby-complete jar for versions 9.0.0.0 - 9.1.8.0. I'm not sure if we have a good mechanism for tracking which projects are using/bundling JRuby, but it would be good to let them know that they should consider updating for license reasons. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3911) Improve documentation for Controller Services scoping
[ https://issues.apache.org/jira/browse/NIFI-3911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016822#comment-16016822 ] Andrew Lim commented on NIFI-3911: -- Thanks [~alopresto] . I think we should create a separate Jira for any further UI/UX improvements for Controller Settings and close this one out since it was created with the intention of focusing on docs. I had some thoughts too on some possible usability improvements (not related to scoping). > Improve documentation for Controller Services scoping > - > > Key: NIFI-3911 > URL: https://issues.apache.org/jira/browse/NIFI-3911 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation & Website >Reporter: Andrew Lim >Assignee: Andrew Lim >Priority: Minor > > Filing to revisit and improve the documentation on Controller Services > scoping which has caused some confusion for users. There have been some > improvements made to improve the UX (NIFI-3128) but the doc can perhaps due a > better job highlighting this change in functionality from 0.x NiFi. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi issue #1826: NIFI-3911 Improve Controller Services and Reporting Task D...
Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/1826 I reviewed this and really like the improvements. I merged it, but I'm not closing the Jira yet because I still think changing the title of the tab in the Controller Settings dialog and adding a sentence there will help users who do not read the documentation (like that would ever happen). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3911) Improve documentation for Controller Services scoping
[ https://issues.apache.org/jira/browse/NIFI-3911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016805#comment-16016805 ] ASF GitHub Bot commented on NIFI-3911: -- Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/1826 I reviewed this and really like the improvements. I merged it, but I'm not closing the Jira yet because I still think changing the title of the tab in the Controller Settings dialog and adding a sentence there will help users who do not read the documentation (like that would ever happen). > Improve documentation for Controller Services scoping > - > > Key: NIFI-3911 > URL: https://issues.apache.org/jira/browse/NIFI-3911 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation & Website >Reporter: Andrew Lim >Assignee: Andrew Lim >Priority: Minor > > Filing to revisit and improve the documentation on Controller Services > scoping which has caused some confusion for users. There have been some > improvements made to improve the UX (NIFI-3128) but the doc can perhaps due a > better job highlighting this change in functionality from 0.x NiFi. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3911) Improve documentation for Controller Services scoping
[ https://issues.apache.org/jira/browse/NIFI-3911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016803#comment-16016803 ] ASF GitHub Bot commented on NIFI-3911: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1826 > Improve documentation for Controller Services scoping > - > > Key: NIFI-3911 > URL: https://issues.apache.org/jira/browse/NIFI-3911 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation & Website >Reporter: Andrew Lim >Assignee: Andrew Lim >Priority: Minor > > Filing to revisit and improve the documentation on Controller Services > scoping which has caused some confusion for users. There have been some > improvements made to improve the UX (NIFI-3128) but the doc can perhaps due a > better job highlighting this change in functionality from 0.x NiFi. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi pull request #1826: NIFI-3911 Improve Controller Services and Reporting...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1826 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3911) Improve documentation for Controller Services scoping
[ https://issues.apache.org/jira/browse/NIFI-3911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016802#comment-16016802 ] ASF subversion and git services commented on NIFI-3911: --- Commit 28eadaa683b41833987b6d20bd79d03f43d61259 in nifi's branch refs/heads/master from [~andrewmlim] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=28eadaa ] NIFI-3911 Improve Controller Services and Reporting Task Documentation This closes #1826. Signed-off-by: Andy LoPresto > Improve documentation for Controller Services scoping > - > > Key: NIFI-3911 > URL: https://issues.apache.org/jira/browse/NIFI-3911 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation & Website >Reporter: Andrew Lim >Assignee: Andrew Lim >Priority: Minor > > Filing to revisit and improve the documentation on Controller Services > scoping which has caused some confusion for users. There have been some > improvements made to improve the UX (NIFI-3128) but the doc can perhaps due a > better job highlighting this change in functionality from 0.x NiFi. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3922) Inconsistency in Resume, Stop, Start of Processors/Controller Services
[ https://issues.apache.org/jira/browse/NIFI-3922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-3922: -- Fix Version/s: (was: 1.3.0) > Inconsistency in Resume, Stop, Start of Processors/Controller Services > -- > > Key: NIFI-3922 > URL: https://issues.apache.org/jira/browse/NIFI-3922 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.2.0 >Reporter: n h > > There are some inconsistency in Resume, Stop, Start of Processors/Controller > Services. > For Example in rare situation when nifi starts, all processors are stopped > (not resumed), even if bdfs.flowcontroller.autoResumeState=true > Also, it is (rarely) seen that when a processor is stopped by rest api, all > processors are stopped. (the target processor and all others are in root > group) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3812) Add PutORC and FetchORC processors
[ https://issues.apache.org/jira/browse/NIFI-3812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016797#comment-16016797 ] Joseph Witt commented on NIFI-3812: --- bryan i removed the fix version for now until there is a pr and review traction. please advise if that is not appropriate. thanks > Add PutORC and FetchORC processors > -- > > Key: NIFI-3812 > URL: https://issues.apache.org/jira/browse/NIFI-3812 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Minor > > We should add processors for Apache ORC that can write records to HDFS as > ORC, and fetch ORC from HDFS into records, similar to the recently added > Parquet processors. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3812) Add PutORC and FetchORC processors
[ https://issues.apache.org/jira/browse/NIFI-3812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-3812: -- Fix Version/s: (was: 1.3.0) > Add PutORC and FetchORC processors > -- > > Key: NIFI-3812 > URL: https://issues.apache.org/jira/browse/NIFI-3812 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Bryan Bende >Assignee: Bryan Bende >Priority: Minor > > We should add processors for Apache ORC that can write records to HDFS as > ORC, and fetch ORC from HDFS into records, similar to the recently added > Parquet processors. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (NIFI-3939) Fix entity type mismatches in web API resource response types
Joe Skora created NIFI-3939: --- Summary: Fix entity type mismatches in web API resource response types Key: NIFI-3939 URL: https://issues.apache.org/jira/browse/NIFI-3939 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 1.2.0 Reporter: Joe Skora In {{ProcessGroupResource.java}} annotations on the {{/process-groups/\{id}/getProcessGroups}} endpoint incorrectly identifies the response as a {{ProcessorsEntity}} when it should be a {{ProcessGroupsEntity}}. Similar mismatches may exist in {{FlowFileQueueResponse}}, {{FlowResource}}, and possibly other classes. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3911) Improve documentation for Controller Services scoping
[ https://issues.apache.org/jira/browse/NIFI-3911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016702#comment-16016702 ] ASF GitHub Bot commented on NIFI-3911: -- GitHub user andrewmlim opened a pull request: https://github.com/apache/nifi/pull/1826 NIFI-3911 Improve Controller Services and Reporting Task Documentation Specifically added more information regarding Controller Scoping. Added more screenshots throughout and more content to Reporting Task Section. Moved Enabling/Disabling Controller Services into a new section. You can merge this pull request into a Git repository by running: $ git pull https://github.com/andrewmlim/nifi NIFI-3911 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1826.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1826 commit d49869872bfca1e95ebee33edd901cace29baf55 Author: Andrew Lim Date: 2017-05-18T23:58:29Z NIFI-3911 Improve Controller Services and Reporting Task Documentation > Improve documentation for Controller Services scoping > - > > Key: NIFI-3911 > URL: https://issues.apache.org/jira/browse/NIFI-3911 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation & Website >Reporter: Andrew Lim >Assignee: Andrew Lim >Priority: Minor > > Filing to revisit and improve the documentation on Controller Services > scoping which has caused some confusion for users. There have been some > improvements made to improve the UX (NIFI-3128) but the doc can perhaps due a > better job highlighting this change in functionality from 0.x NiFi. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi pull request #1826: NIFI-3911 Improve Controller Services and Reporting...
GitHub user andrewmlim opened a pull request: https://github.com/apache/nifi/pull/1826 NIFI-3911 Improve Controller Services and Reporting Task Documentation Specifically added more information regarding Controller Scoping. Added more screenshots throughout and more content to Reporting Task Section. Moved Enabling/Disabling Controller Services into a new section. You can merge this pull request into a Git repository by running: $ git pull https://github.com/andrewmlim/nifi NIFI-3911 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1826.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1826 commit d49869872bfca1e95ebee33edd901cace29baf55 Author: Andrew Lim Date: 2017-05-18T23:58:29Z NIFI-3911 Improve Controller Services and Reporting Task Documentation --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (NIFI-3857) Create PartitionRecord processor
[ https://issues.apache.org/jira/browse/NIFI-3857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-3857: - Fix Version/s: 1.3.0 Status: Patch Available (was: Open) > Create PartitionRecord processor > > > Key: NIFI-3857 > URL: https://issues.apache.org/jira/browse/NIFI-3857 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne > Fix For: 1.3.0 > > > With the record readers and writers, it will be useful to be able to bring in > a stream of records as a FlowFile and output N different FlowFiles, where all > records in the output are "alike." Users should be able to define what it > means for two records to be "like records" by entering one or more > RecordPaths. > Since we know that all records in an outgoing FlowFile will have the same > value for a given field, we should also add an attribute to the FlowFile for > each of these, so that routing can easily be done and we can also track that > attribute via Provenance, etc. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi pull request #1825: NIFI 3863 - Add LookupRecord, PartitionRecord proce...
Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1825#discussion_r117351574 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/PartitionRecord.java --- @@ -0,0 +1,412 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.nifi.processors.standard; + +import java.io.BufferedInputStream; +import java.io.IOException; +import java.io.InputStream; +import java.io.OutputStream; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.Collection; +import java.util.Collections; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.stream.Collectors; +import java.util.stream.Stream; + +import org.apache.nifi.annotation.behavior.DynamicProperty; +import org.apache.nifi.annotation.behavior.EventDriven; +import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.behavior.InputRequirement.Requirement; +import org.apache.nifi.annotation.behavior.SupportsBatching; +import org.apache.nifi.annotation.behavior.WritesAttribute; +import org.apache.nifi.annotation.behavior.WritesAttributes; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.SeeAlso; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.components.ValidationContext; +import org.apache.nifi.components.ValidationResult; +import org.apache.nifi.flowfile.FlowFile; +import org.apache.nifi.flowfile.attributes.CoreAttributes; +import org.apache.nifi.processor.AbstractProcessor; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.ProcessSession; +import org.apache.nifi.processor.Relationship; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.record.path.FieldValue; +import org.apache.nifi.record.path.RecordPath; +import org.apache.nifi.record.path.util.RecordPathCache; +import org.apache.nifi.record.path.validation.RecordPathValidator; +import org.apache.nifi.serialization.RecordReader; +import org.apache.nifi.serialization.RecordReaderFactory; +import org.apache.nifi.serialization.RecordSetWriter; +import org.apache.nifi.serialization.RecordSetWriterFactory; +import org.apache.nifi.serialization.WriteResult; +import org.apache.nifi.serialization.record.Record; +import org.apache.nifi.serialization.record.RecordSchema; +import org.apache.nifi.serialization.record.util.DataTypeUtils; + +@EventDriven +@SupportsBatching +@InputRequirement(Requirement.INPUT_REQUIRED) +@CapabilityDescription("Receives Record-oriented data (i.e., data that can be read by the configured Record Reader) and evaluates one or more RecordPaths against the " ++ "each record in the incoming FlowFile. Each record is then grouped with other \"like records\" and a FlowFile is created for each group of \"like records.\" What it means for " ++ "two records to be \"like records\" is determined by user-defined properties. The user is required to enter at least one user-defined property whose value is a RecordPath. Two " ++ "records are considered alike if they have the same value for all configured RecordPaths. Because we know that all records in a given output FlowFile have the same value for the " ++ "fields that are specified by the RecordPath, an attribute is added for each field. See Additional Details on the Usage page for more information and examples.") +@DynamicProperty(name="The name given to the dynamic property is the name of the attribute that will be used to denote the value of the associted RecordPath.", +value="A RecordPath that points to a field in the Record.", +description="Each dy
[GitHub] nifi-minifi-cpp issue #98: MINIFI-37: Create a volatile repository and confi...
Github user apiri commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/98 @phrocker Thanks for the heads up. Will look for that update. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #100: MINIFI-320: Move instantiation of OpenSSL...
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/100#discussion_r117346914 --- Diff: libminifi/test/unit/SocketTests.cpp --- @@ -183,3 +196,42 @@ TEST_CASE("TestSocketWriteTestAfterClose", "[TestSocket6]") { server.closeStream(); } + +std::atomic counter; +std::mt19937_64 seed { std::random_device { }() }; +bool createSocket() { + int mine = counter++; + std::shared_ptr configuration = std::make_shared< + minifi::Configure>(); + + std::uniform_int_distribution<> distribution { 10, 100 }; + std::this_thread::sleep_for(std::chrono::milliseconds { distribution(seed) }); + + for (int i = 0; i < 50; i++) { +std::shared_ptr socketA = std::make_shared( +configuration); +socketA->initialize(); + } + + return true; +} --- End diff -- add some comments to why this exists. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #100: MINIFI-320: Move instantiation of OpenSSL...
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/100#discussion_r117346816 --- Diff: libminifi/include/utils/ThreadPool.h --- @@ -189,7 +195,8 @@ class ThreadPool { // atomic running boolean std::atomic running_; // worker queue of worker objects - std::queue> worker_queue_; + //std::queue> worker_queue_; --- End diff -- remove comment --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #98: MINIFI-37: Create a volatile repository and confi...
Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/98 @apiri after addressing your comments, above, I ran tests which made me rethink part of how this was tested. To cover my bases I'm going to test a little more and comment when it's ready again. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Assigned] (NIFI-3938) Add a ScriptedLookupService
[ https://issues.apache.org/jira/browse/NIFI-3938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess reassigned NIFI-3938: -- Assignee: Matt Burgess > Add a ScriptedLookupService > --- > > Key: NIFI-3938 > URL: https://issues.apache.org/jira/browse/NIFI-3938 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess > > NIFI-3863 aims to implement a LookupRecord processor along with two > implementations of LookupServices, one for enriching records with Geo > information from IP addresses, and a static one from a list of provided > key/value pairs. > A very flexible and powerful improvement would be to offer a > ScriptedLookupService, where the same scripting capabilities that are > available to InvokeScriptedProcessor, ScriptedReader and > ScriptedRecordSetWriter are available to prototype/implement a LookupService > for use by the LookupRecord processor. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (NIFI-3938) Add a ScriptedLookupService
Matt Burgess created NIFI-3938: -- Summary: Add a ScriptedLookupService Key: NIFI-3938 URL: https://issues.apache.org/jira/browse/NIFI-3938 Project: Apache NiFi Issue Type: New Feature Components: Extensions Reporter: Matt Burgess NIFI-3863 aims to implement a LookupRecord processor along with two implementations of LookupServices, one for enriching records with Geo information from IP addresses, and a static one from a list of provided key/value pairs. A very flexible and powerful improvement would be to offer a ScriptedLookupService, where the same scripting capabilities that are available to InvokeScriptedProcessor, ScriptedReader and ScriptedRecordSetWriter are available to prototype/implement a LookupService for use by the LookupRecord processor. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (NIFI-3824) Perform Release Manager duties for 0.7.3 Release
[ https://issues.apache.org/jira/browse/NIFI-3824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Moser resolved NIFI-3824. - Resolution: Delivered Fix Version/s: 0.7.3 > Perform Release Manager duties for 0.7.3 Release > > > Key: NIFI-3824 > URL: https://issues.apache.org/jira/browse/NIFI-3824 > Project: Apache NiFi > Issue Type: Task >Reporter: Michael Moser >Assignee: Michael Moser >Priority: Minor > Fix For: 0.7.3 > > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3863) Create a LookupRecord Processor
[ https://issues.apache.org/jira/browse/NIFI-3863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-3863: --- Status: Patch Available (was: Open) > Create a LookupRecord Processor > --- > > Key: NIFI-3863 > URL: https://issues.apache.org/jira/browse/NIFI-3863 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne > > We should provide a mechanism to fetch a value from a Record, and then lookup > that value in a "lookup table." If we get a match, we should add the result > back into a field in the Record. > The Processor would be configured with a Record Reader and a Record Writer. > It would also take a RecordPath as a 'Lookup Path' that indicates where in > the Record the desired value should be pulled from. We then lookup the value > from that field. There should also be a 'Destination Path' property that > indicates where the value should be added back to the record. The record > should then be routed to either a 'matched' relationship if there was a value > in the lookup table, or an 'unmatched' relationship if there was no value in > the lookup table. > The 'Destination Path' property should be optional. This way, if it is set, > this Processor would function as an 'enrichment processor.' If the property > is not set, this Processor would function as a 'routing processor', by > routing to either 'matched' or 'unmatched' depending on whether or not its > value is found in the Lookup Table. > This means that we also need to build a Lookup Table type of Controller > Service. It would be best to provide two implementations: a Properties > File-based lookup service and a Simple Key-Value Lookup Service. The Simple > Key-Value service would just allow the user to add properties where the > Property Name is the Key and the Property Value is the value. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi pull request #1825: NIFI 3863 - Add LookupRecord, PartitionRecord proce...
Github user mattyb149 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1825#discussion_r117331376 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/PartitionRecord.java --- @@ -0,0 +1,412 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.nifi.processors.standard; + +import java.io.BufferedInputStream; +import java.io.IOException; +import java.io.InputStream; +import java.io.OutputStream; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.Collection; +import java.util.Collections; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.stream.Collectors; +import java.util.stream.Stream; + +import org.apache.nifi.annotation.behavior.DynamicProperty; +import org.apache.nifi.annotation.behavior.EventDriven; +import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.behavior.InputRequirement.Requirement; +import org.apache.nifi.annotation.behavior.SupportsBatching; +import org.apache.nifi.annotation.behavior.WritesAttribute; +import org.apache.nifi.annotation.behavior.WritesAttributes; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.SeeAlso; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.components.ValidationContext; +import org.apache.nifi.components.ValidationResult; +import org.apache.nifi.flowfile.FlowFile; +import org.apache.nifi.flowfile.attributes.CoreAttributes; +import org.apache.nifi.processor.AbstractProcessor; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.ProcessSession; +import org.apache.nifi.processor.Relationship; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.record.path.FieldValue; +import org.apache.nifi.record.path.RecordPath; +import org.apache.nifi.record.path.util.RecordPathCache; +import org.apache.nifi.record.path.validation.RecordPathValidator; +import org.apache.nifi.serialization.RecordReader; +import org.apache.nifi.serialization.RecordReaderFactory; +import org.apache.nifi.serialization.RecordSetWriter; +import org.apache.nifi.serialization.RecordSetWriterFactory; +import org.apache.nifi.serialization.WriteResult; +import org.apache.nifi.serialization.record.Record; +import org.apache.nifi.serialization.record.RecordSchema; +import org.apache.nifi.serialization.record.util.DataTypeUtils; + +@EventDriven +@SupportsBatching +@InputRequirement(Requirement.INPUT_REQUIRED) +@CapabilityDescription("Receives Record-oriented data (i.e., data that can be read by the configured Record Reader) and evaluates one or more RecordPaths against the " ++ "each record in the incoming FlowFile. Each record is then grouped with other \"like records\" and a FlowFile is created for each group of \"like records.\" What it means for " ++ "two records to be \"like records\" is determined by user-defined properties. The user is required to enter at least one user-defined property whose value is a RecordPath. Two " ++ "records are considered alike if they have the same value for all configured RecordPaths. Because we know that all records in a given output FlowFile have the same value for the " ++ "fields that are specified by the RecordPath, an attribute is added for each field. See Additional Details on the Usage page for more information and examples.") +@DynamicProperty(name="The name given to the dynamic property is the name of the attribute that will be used to denote the value of the associted RecordPath.", +value="A RecordPath that points to a field in the Record.", +description="Each d
[jira] [Updated] (NIFI-3897) Queue with flow files thinks it is empty
[ https://issues.apache.org/jira/browse/NIFI-3897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Gresock updated NIFI-3897: - Description: A particular queue is listed as having 80,000 flow files, but NiFi thinks there are none to list. I was originally inclined to believe that the queue is indeed empty, and that there is somehow a pointer to 80,000 flow files leftover on the queue. Unfortunately, the logs have rolled over so I can't see what error or warning messages may have happened at the time, and I can't reproduce it reliably. Update: I happened to reboot my cluster, and suddenly the 80,000 flow files went through the flow. This makes the problem considerably worse, since it appears the flow files were "invisible" until the reboot. was: A particular queue is listed as having 80,000 flow files, but NiFi thinks there are none to list. I am inclined to believe that the queue is indeed empty, and that there is somehow a pointer to 80,000 flow files leftover on the queue. Unfortunately, the logs have rolled over so I can't see what error or warning messages may have happened at the time, and I can't reproduce it reliably. Update: I happened to reboot my cluster, and suddenly the 80,000 flow files went through the flow. This makes the problem considerably worse, since it appears the flow files were "invisible" until the reboot. > Queue with flow files thinks it is empty > > > Key: NIFI-3897 > URL: https://issues.apache.org/jira/browse/NIFI-3897 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.2.0 >Reporter: Joseph Gresock > Attachments: queue-has-no-flowfiles.png > > > A particular queue is listed as having 80,000 flow files, but NiFi thinks > there are none to list. I was originally inclined to believe that the queue > is indeed empty, and that there is somehow a pointer to 80,000 flow files > leftover on the queue. > Unfortunately, the logs have rolled over so I can't see what error or warning > messages may have happened at the time, and I can't reproduce it reliably. > Update: I happened to reboot my cluster, and suddenly the 80,000 flow files > went through the flow. This makes the problem considerably worse, since it > appears the flow files were "invisible" until the reboot. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi-minifi-cpp pull request #98: MINIFI-37: Create a volatile repository an...
Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/98#discussion_r117326088 --- Diff: README.md --- @@ -299,7 +299,19 @@ Additionally, users can utilize the MiNiFi Toolkit Converter (version 0.0.1 - sc if you do not want to enable client certificate base authorization nifi.security.need.ClientAuth=false - + +### Configuring Volatile and NO-OP Repositories + + in minifi.properties + + # For Volatile Repositories: + nifi.flow.repository.class.name=VolatileRepository + nifi.provenance.repository.class.name=VolatileRepository + + # For NO-OP Repositories: +nifi.flow.repository.class.name=NoOpRepository --- End diff -- That sounds fair. Happy to leave it be for the time being and we can adjust as needed later. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #98: MINIFI-37: Create a volatile repository an...
Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/98#discussion_r117326106 --- Diff: libminifi/include/core/Repository.h --- @@ -47,14 +47,21 @@ namespace nifi { namespace minifi { namespace core { +#define REPOSITORY_DIRECTORY "./repo" --- End diff -- Created https://issues.apache.org/jira/browse/MINIFI-321 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1825: NIFI 3863 - Add LookupRecord, PartitionRecord processors
Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/1825 this thing is insanely useful. I will post my template when done reviewing. But this is supremely cool. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3935) TestPersistentProvenanceRepository frequently fails in travis / likely slow machine timing bug in test
[ https://issues.apache.org/jira/browse/NIFI-3935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016241#comment-16016241 ] ASF GitHub Bot commented on NIFI-3935: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1823 > TestPersistentProvenanceRepository frequently fails in travis / likely slow > machine timing bug in test > -- > > Key: NIFI-3935 > URL: https://issues.apache.org/jira/browse/NIFI-3935 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Joseph Witt >Assignee: Joseph Witt > Fix For: 1.3.0 > > > {quote} > Tests run: 35, Failures: 1, Errors: 0, Skipped: 2, Time elapsed: 99.013 sec > <<< FAILURE! - in > org.apache.nifi.provenance.TestPersistentProvenanceRepository > testAddAndRecover(org.apache.nifi.provenance.TestPersistentProvenanceRepository) > Time elapsed: 3.646 sec <<< FAILURE! > java.lang.AssertionError: expected:<10> but was:<9> > at org.junit.Assert.fail(Assert.java:88) > at org.junit.Assert.failNotEquals(Assert.java:834) > at org.junit.Assert.assertEquals(Assert.java:645) > at org.junit.Assert.assertEquals(Assert.java:631) > at > org.apache.nifi.provenance.TestPersistentProvenanceRepository.testAddAndRecover(TestPersistentProvenanceRepository.java:396) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3935) TestPersistentProvenanceRepository frequently fails in travis / likely slow machine timing bug in test
[ https://issues.apache.org/jira/browse/NIFI-3935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-3935: --- Fix Version/s: 1.3.0 > TestPersistentProvenanceRepository frequently fails in travis / likely slow > machine timing bug in test > -- > > Key: NIFI-3935 > URL: https://issues.apache.org/jira/browse/NIFI-3935 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Joseph Witt >Assignee: Joseph Witt > Fix For: 1.3.0 > > > {quote} > Tests run: 35, Failures: 1, Errors: 0, Skipped: 2, Time elapsed: 99.013 sec > <<< FAILURE! - in > org.apache.nifi.provenance.TestPersistentProvenanceRepository > testAddAndRecover(org.apache.nifi.provenance.TestPersistentProvenanceRepository) > Time elapsed: 3.646 sec <<< FAILURE! > java.lang.AssertionError: expected:<10> but was:<9> > at org.junit.Assert.fail(Assert.java:88) > at org.junit.Assert.failNotEquals(Assert.java:834) > at org.junit.Assert.assertEquals(Assert.java:645) > at org.junit.Assert.assertEquals(Assert.java:631) > at > org.apache.nifi.provenance.TestPersistentProvenanceRepository.testAddAndRecover(TestPersistentProvenanceRepository.java:396) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3935) TestPersistentProvenanceRepository frequently fails in travis / likely slow machine timing bug in test
[ https://issues.apache.org/jira/browse/NIFI-3935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-3935: --- Resolution: Fixed Status: Resolved (was: Patch Available) > TestPersistentProvenanceRepository frequently fails in travis / likely slow > machine timing bug in test > -- > > Key: NIFI-3935 > URL: https://issues.apache.org/jira/browse/NIFI-3935 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Joseph Witt >Assignee: Joseph Witt > Fix For: 1.3.0 > > > {quote} > Tests run: 35, Failures: 1, Errors: 0, Skipped: 2, Time elapsed: 99.013 sec > <<< FAILURE! - in > org.apache.nifi.provenance.TestPersistentProvenanceRepository > testAddAndRecover(org.apache.nifi.provenance.TestPersistentProvenanceRepository) > Time elapsed: 3.646 sec <<< FAILURE! > java.lang.AssertionError: expected:<10> but was:<9> > at org.junit.Assert.fail(Assert.java:88) > at org.junit.Assert.failNotEquals(Assert.java:834) > at org.junit.Assert.assertEquals(Assert.java:645) > at org.junit.Assert.assertEquals(Assert.java:631) > at > org.apache.nifi.provenance.TestPersistentProvenanceRepository.testAddAndRecover(TestPersistentProvenanceRepository.java:396) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi pull request #1823: NIFI-3935 made the old persistent prov test more fo...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1823 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3935) TestPersistentProvenanceRepository frequently fails in travis / likely slow machine timing bug in test
[ https://issues.apache.org/jira/browse/NIFI-3935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016237#comment-16016237 ] ASF GitHub Bot commented on NIFI-3935: -- Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/1823 +1 LGTM, tests pass locally and on Travis. Thanks much, merging to master > TestPersistentProvenanceRepository frequently fails in travis / likely slow > machine timing bug in test > -- > > Key: NIFI-3935 > URL: https://issues.apache.org/jira/browse/NIFI-3935 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Joseph Witt >Assignee: Joseph Witt > > {quote} > Tests run: 35, Failures: 1, Errors: 0, Skipped: 2, Time elapsed: 99.013 sec > <<< FAILURE! - in > org.apache.nifi.provenance.TestPersistentProvenanceRepository > testAddAndRecover(org.apache.nifi.provenance.TestPersistentProvenanceRepository) > Time elapsed: 3.646 sec <<< FAILURE! > java.lang.AssertionError: expected:<10> but was:<9> > at org.junit.Assert.fail(Assert.java:88) > at org.junit.Assert.failNotEquals(Assert.java:834) > at org.junit.Assert.assertEquals(Assert.java:645) > at org.junit.Assert.assertEquals(Assert.java:631) > at > org.apache.nifi.provenance.TestPersistentProvenanceRepository.testAddAndRecover(TestPersistentProvenanceRepository.java:396) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3935) TestPersistentProvenanceRepository frequently fails in travis / likely slow machine timing bug in test
[ https://issues.apache.org/jira/browse/NIFI-3935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-3935: --- Status: Patch Available (was: Open) > TestPersistentProvenanceRepository frequently fails in travis / likely slow > machine timing bug in test > -- > > Key: NIFI-3935 > URL: https://issues.apache.org/jira/browse/NIFI-3935 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Joseph Witt >Assignee: Joseph Witt > > {quote} > Tests run: 35, Failures: 1, Errors: 0, Skipped: 2, Time elapsed: 99.013 sec > <<< FAILURE! - in > org.apache.nifi.provenance.TestPersistentProvenanceRepository > testAddAndRecover(org.apache.nifi.provenance.TestPersistentProvenanceRepository) > Time elapsed: 3.646 sec <<< FAILURE! > java.lang.AssertionError: expected:<10> but was:<9> > at org.junit.Assert.fail(Assert.java:88) > at org.junit.Assert.failNotEquals(Assert.java:834) > at org.junit.Assert.assertEquals(Assert.java:645) > at org.junit.Assert.assertEquals(Assert.java:631) > at > org.apache.nifi.provenance.TestPersistentProvenanceRepository.testAddAndRecover(TestPersistentProvenanceRepository.java:396) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3937) Content viewer is always empty
[ https://issues.apache.org/jira/browse/NIFI-3937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016238#comment-16016238 ] Joseph Witt commented on NIFI-3937: --- [~jgresock] yeah there is plenty to do a 1.2.1 although frankly there is even plenty in my opinion to just do a 1.3.0 as well. We cherry-pick things into 1.2.1 (incrementals) rather than presuming we'll do one. However, given the crazy fast pace of progress even since the 1.2.0 release i'm frankly planning to advocate we just go directly after another minor. > Content viewer is always empty > -- > > Key: NIFI-3937 > URL: https://issues.apache.org/jira/browse/NIFI-3937 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Joseph Gresock > > After upgrading to 1.2.0, I started noticing that the content viewer is > always empty when I try to view a flow file. I've tried both JSON and text > files with the same result. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3935) TestPersistentProvenanceRepository frequently fails in travis / likely slow machine timing bug in test
[ https://issues.apache.org/jira/browse/NIFI-3935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016239#comment-16016239 ] ASF subversion and git services commented on NIFI-3935: --- Commit 762293c01cc9b843abd5378f5de51f626050f498 in nifi's branch refs/heads/master from [~joewitt] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=762293c ] NIFI-3935 made the old persistent prov test more forgiving of timing issues and extended lucene test timeouts for slower environments. These should be integration tests really. Signed-off-by: Matt Burgess This closes #1823 > TestPersistentProvenanceRepository frequently fails in travis / likely slow > machine timing bug in test > -- > > Key: NIFI-3935 > URL: https://issues.apache.org/jira/browse/NIFI-3935 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Joseph Witt >Assignee: Joseph Witt > > {quote} > Tests run: 35, Failures: 1, Errors: 0, Skipped: 2, Time elapsed: 99.013 sec > <<< FAILURE! - in > org.apache.nifi.provenance.TestPersistentProvenanceRepository > testAddAndRecover(org.apache.nifi.provenance.TestPersistentProvenanceRepository) > Time elapsed: 3.646 sec <<< FAILURE! > java.lang.AssertionError: expected:<10> but was:<9> > at org.junit.Assert.fail(Assert.java:88) > at org.junit.Assert.failNotEquals(Assert.java:834) > at org.junit.Assert.assertEquals(Assert.java:645) > at org.junit.Assert.assertEquals(Assert.java:631) > at > org.apache.nifi.provenance.TestPersistentProvenanceRepository.testAddAndRecover(TestPersistentProvenanceRepository.java:396) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi issue #1823: NIFI-3935 made the old persistent prov test more forgiving...
Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/1823 +1 LGTM, tests pass locally and on Travis. Thanks much, merging to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3937) Content viewer is always empty
[ https://issues.apache.org/jira/browse/NIFI-3937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016230#comment-16016230 ] Joseph Gresock commented on NIFI-3937: -- Thanks, I missed that one. Should NIFI-3865 be in 1.2.1 instead of 1.3.0? It seems important enough to put in a bug fix release. > Content viewer is always empty > -- > > Key: NIFI-3937 > URL: https://issues.apache.org/jira/browse/NIFI-3937 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Joseph Gresock > > After upgrading to 1.2.0, I started noticing that the content viewer is > always empty when I try to view a flow file. I've tried both JSON and text > files with the same result. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi issue #1825: NIFI 3863 - Add LookupRecord, PartitionRecord processors
Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/1825 I think for LookupAttribute it should either send all events to success (so it doesn't change/partition the stream other than to possibly enrich each record). I think if the lookup recordpath does not find anything the enrichment should just move on just like if the result record is empty. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Resolved] (NIFI-3937) Content viewer is always empty
[ https://issues.apache.org/jira/browse/NIFI-3937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt resolved NIFI-3937. --- Resolution: Duplicate https://issues.apache.org/jira/browse/NIFI-3865 > Content viewer is always empty > -- > > Key: NIFI-3937 > URL: https://issues.apache.org/jira/browse/NIFI-3937 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.2.0 >Reporter: Joseph Gresock > > After upgrading to 1.2.0, I started noticing that the content viewer is > always empty when I try to view a flow file. I've tried both JSON and text > files with the same result. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (NIFI-3937) Content viewer is always empty
Joseph Gresock created NIFI-3937: Summary: Content viewer is always empty Key: NIFI-3937 URL: https://issues.apache.org/jira/browse/NIFI-3937 Project: Apache NiFi Issue Type: Bug Affects Versions: 1.2.0 Reporter: Joseph Gresock After upgrading to 1.2.0, I started noticing that the content viewer is always empty when I try to view a flow file. I've tried both JSON and text files with the same result. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3897) Queue with flow files thinks it is empty
[ https://issues.apache.org/jira/browse/NIFI-3897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Gresock updated NIFI-3897: - Priority: Major (was: Minor) > Queue with flow files thinks it is empty > > > Key: NIFI-3897 > URL: https://issues.apache.org/jira/browse/NIFI-3897 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.2.0 >Reporter: Joseph Gresock > Attachments: queue-has-no-flowfiles.png > > > A particular queue is listed as having 80,000 flow files, but NiFi thinks > there are none to list. I am inclined to believe that the queue is indeed > empty, and that there is somehow a pointer to 80,000 flow files leftover on > the queue. > Unfortunately, the logs have rolled over so I can't see what error or warning > messages may have happened at the time, and I can't reproduce it reliably. > Update: I happened to reboot my cluster, and suddenly the 80,000 flow files > went through the flow. This makes the problem considerably worse, since it > appears the flow files were "invisible" until the reboot. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi-minifi-cpp pull request #100: MINIFI-320: Move instantion of OpenSSL to...
GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/100 MINIFI-320: Move instantion of OpenSSL to a singleton that's lazily created. Added test. In order to make the test work as expected I had to update thread pool to use a non blocking queue. Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFI-320 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/100.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #100 commit 68d7d6f14c96a73f89dead04e37895af6e1957fc Author: Marc Parisi Date: 2017-05-18T18:21:16Z MINIFI-320: Move instantion of OpenSSL to a singleton that's lazily created. Added test. In order to make the test work as expected I had to update thread pool to use a non blocking queue. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (NIFI-3897) Queue with flow files thinks it is empty
[ https://issues.apache.org/jira/browse/NIFI-3897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Gresock updated NIFI-3897: - Description: A particular queue is listed as having 80,000 flow files, but NiFi thinks there are none to list. I am inclined to believe that the queue is indeed empty, and that there is somehow a pointer to 80,000 flow files leftover on the queue. Unfortunately, the logs have rolled over so I can't see what error or warning messages may have happened at the time, and I can't reproduce it reliably. Update: I happened to reboot my cluster, and suddenly the 80,000 flow files went through the flow. This makes the problem considerably worse, since it appears the flow files were "invisible" until the reboot. was: A particular queue is listed as having 80,000 flow files, but NiFi thinks there are none to list. I am inclined to believe that the queue is indeed empty, and that there is somehow a pointer to 80,000 flow files leftover on the queue. Unfortunately, the logs have rolled over so I can't see what error or warning messages may have happened at the time, and I can't reproduce it reliably. > Queue with flow files thinks it is empty > > > Key: NIFI-3897 > URL: https://issues.apache.org/jira/browse/NIFI-3897 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.2.0 >Reporter: Joseph Gresock >Priority: Minor > Attachments: queue-has-no-flowfiles.png > > > A particular queue is listed as having 80,000 flow files, but NiFi thinks > there are none to list. I am inclined to believe that the queue is indeed > empty, and that there is somehow a pointer to 80,000 flow files leftover on > the queue. > Unfortunately, the logs have rolled over so I can't see what error or warning > messages may have happened at the time, and I can't reproduce it reliably. > Update: I happened to reboot my cluster, and suddenly the 80,000 flow files > went through the flow. This makes the problem considerably worse, since it > appears the flow files were "invisible" until the reboot. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3897) Queue with flow files thinks it is empty
[ https://issues.apache.org/jira/browse/NIFI-3897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Gresock updated NIFI-3897: - Summary: Queue with flow files thinks it is empty (was: Empty queue displays a non-zero flow file count) > Queue with flow files thinks it is empty > > > Key: NIFI-3897 > URL: https://issues.apache.org/jira/browse/NIFI-3897 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.2.0 >Reporter: Joseph Gresock >Priority: Minor > Attachments: queue-has-no-flowfiles.png > > > A particular queue is listed as having 80,000 flow files, but NiFi thinks > there are none to list. I am inclined to believe that the queue is indeed > empty, and that there is somehow a pointer to 80,000 flow files leftover on > the queue. > Unfortunately, the logs have rolled over so I can't see what error or warning > messages may have happened at the time, and I can't reproduce it reliably. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi issue #1825: NIFI 3863 - Add LookupRecord, PartitionRecord processors
Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/1825 @markap14 i think you dont want this in the PR create mode 100644 nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/hs_err_pid26624.log Can you please squash when it is time for a formal review. I am going through this already though. This is cool stuff! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3935) TestPersistentProvenanceRepository frequently fails in travis / likely slow machine timing bug in test
[ https://issues.apache.org/jira/browse/NIFI-3935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016145#comment-16016145 ] Joseph Witt commented on NIFI-3935: --- actually these look like the problems i just fixed on core framework. rebased to master now that this fix is in and will see what happens now > TestPersistentProvenanceRepository frequently fails in travis / likely slow > machine timing bug in test > -- > > Key: NIFI-3935 > URL: https://issues.apache.org/jira/browse/NIFI-3935 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Joseph Witt >Assignee: Joseph Witt > > {quote} > Tests run: 35, Failures: 1, Errors: 0, Skipped: 2, Time elapsed: 99.013 sec > <<< FAILURE! - in > org.apache.nifi.provenance.TestPersistentProvenanceRepository > testAddAndRecover(org.apache.nifi.provenance.TestPersistentProvenanceRepository) > Time elapsed: 3.646 sec <<< FAILURE! > java.lang.AssertionError: expected:<10> but was:<9> > at org.junit.Assert.fail(Assert.java:88) > at org.junit.Assert.failNotEquals(Assert.java:834) > at org.junit.Assert.assertEquals(Assert.java:645) > at org.junit.Assert.assertEquals(Assert.java:631) > at > org.apache.nifi.provenance.TestPersistentProvenanceRepository.testAddAndRecover(TestPersistentProvenanceRepository.java:396) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (NIFI-3853) Escape configurations prior to writing flow.xml
[ https://issues.apache.org/jira/browse/NIFI-3853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Bende resolved NIFI-3853. --- Resolution: Fixed > Escape configurations prior to writing flow.xml > --- > > Key: NIFI-3853 > URL: https://issues.apache.org/jira/browse/NIFI-3853 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Critical > Fix For: 1.3.0 > > > If invalid characters are entered through the UI, they can corrupt the > flow.xml.gz making it unparsable on future restarts. Once in this state, the > flow.xml.gz needs to be hand-edited to remove the invalid characters. > Characters outside the following ranges may cause issues: > {code} > #x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x1-#x10] > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3935) TestPersistentProvenanceRepository frequently fails in travis / likely slow machine timing bug in test
[ https://issues.apache.org/jira/browse/NIFI-3935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016138#comment-16016138 ] Joseph Witt commented on NIFI-3935: --- and now travis shows these Tests run: 30, Failures: 0, Errors: 28, Skipped: 2, Time elapsed: 0.16 sec <<< FAILURE! - in org.apache.nifi.controller.scheduling.TestProcessorLifecycle validateStopCallsAreMeaninglessIfProcessorNotStarted(org.apache.nifi.controller.scheduling.TestProcessorLifecycle) Time elapsed: 0.077 sec <<< ERROR! java.nio.channels.OverlappingFileLockException: null at sun.nio.ch.SharedFileLockTable.checkList(FileLockTable.java:255) at sun.nio.ch.SharedFileLockTable.add(FileLockTable.java:152) at sun.nio.ch.FileChannelImpl.lock(FileChannelImpl.java:1063) at java.nio.channels.FileChannel.lock(FileChannel.java:1053) at org.wali.MinimalLockingWriteAheadLog.(MinimalLockingWriteAheadLog.java:187) at org.wali.MinimalLockingWriteAheadLog.(MinimalLockingWriteAheadLog.java:108) at org.apache.nifi.controller.state.providers.local.WriteAheadLocalStateProvider.init(WriteAheadLocalStateProvider.java:131) at org.apache.nifi.controller.state.providers.AbstractStateProvider.initialize(AbstractStateProvider.java:34) at org.apache.nifi.controller.state.manager.StandardStateManagerProvider.createStateProvider(StandardStateManagerProvider.java:194) at org.apache.nifi.controller.state.manager.StandardStateManagerProvider.createLocalStateProvider(StandardStateManagerProvider.java:85) at org.apache.nifi.controller.state.manager.StandardStateManagerProvider.create(StandardStateManagerProvider.java:71) at org.apache.nifi.controller.FlowController.(FlowController.java:483) at org.apache.nifi.controller.FlowController.createStandaloneInstance(FlowController.java:389) at org.apache.nifi.controller.scheduling.TestProcessorLifecycle.buildFlowControllerForTest(TestProcessorLifecycle.java:738) at org.apache.nifi.controller.scheduling.TestProcessorLifecycle.buildFlowControllerForTest(TestProcessorLifecycle.java:746) at org.apache.nifi.controller.scheduling.TestProcessorLifecycle.validateStopCallsAreMeaninglessIfProcessorNotStarted(TestProcessorLifecycle.java:185) validateStopCallsAreMeaninglessIfProcessorNotStarted(org.apache.nifi.controller.scheduling.TestProcessorLifecycle) Time elapsed: 0.079 sec <<< ERROR! java.lang.NullPointerException: null at org.apache.nifi.controller.scheduling.TestProcessorLifecycle.after(TestProcessorLifecycle.java:99) validateProcessorUnscheduledAndStoppedWhenStopIsCalledBeforeProcessorFullyStarted(org.apache.nifi.controller.scheduling.TestProcessorLifecycle) Time elapsed: 0.006 sec <<< ERROR! java.lang.RuntimeException: java.nio.channels.OverlappingFileLockException at sun.nio.ch.SharedFileLockTable.checkList(FileLockTable.java:255) at sun.nio.ch.SharedFileLockTable.add(FileLockTable.java:152) at sun.nio.ch.FileChannelImpl.lock(FileChannelImpl.java:1063) at java.nio.channels.FileChannel.lock(FileChannel.java:1053) at org.wali.MinimalLockingWriteAheadLog.(MinimalLockingWriteAheadLog.java:187) at org.apache.nifi.controller.repository.WriteAheadFlowFileRepository.initialize(WriteAheadFlowFileRepository.java:161) at org.apache.nifi.controller.FlowController.createFlowFileRepository(FlowController.java:651) at org.apache.nifi.controller.FlowController.(FlowController.java:461) at org.apache.nifi.controller.FlowController.createStandaloneInstance(FlowController.java:389) at org.apache.nifi.controller.scheduling.TestProcessorLifecycle.buildFlowControllerForTest(TestProcessorLifecycle.java:738) at org.apache.nifi.controller.scheduling.TestProcessorLifecycle.buildFlowControllerForTest(TestProcessorLifecycle.java:746) at org.apache.nifi.controller.scheduling.TestProcessorLifecycle.validateProcessorUnscheduledAndStoppedWhenStopIsCalledBeforeProcessorFullyStarted(TestProcessorLifecycle.java:314) validateProcessorUnscheduledAndStoppedWhenStopIsCalledBeforeProcessorFullyStarted(org.apache.nifi.controller.scheduling.TestProcessorLifecycle) Time elapsed: 0.006 sec <<< ERROR! java.lang.NullPointerException: null at org.apache.nifi.controller.scheduling.TestProcessorLifecycle.after(TestProcessorLifecycle.java:99) validateProcessScheduledAfterAdministrativeDelayDueToTheOnScheduledException(org.apache.nifi.controller.scheduling.TestProcessorLifecycle) Time elapsed: 0.007 sec <<< ERROR! java.lang.RuntimeException: java.nio.channels.OverlappingFileLockException at sun.nio.ch.SharedFileLockTable.checkList(FileLockTable.java:255) at sun.nio.ch.SharedFileLockTable.add(FileLockTable.java:152) at sun.nio.ch.FileChannelImpl.lock(FileChannelImpl.java:1063) at java.nio.cha
[GitHub] nifi pull request #1825: NIFI 3863 - Add LookupRecord, PartitionRecord proce...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/1825 NIFI 3863 - Add LookupRecord, PartitionRecord processors I still need to add some more documentation and unit tests but wanted to go ahead and get this up so that if anyone has a chance to review or test they can do so. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-3863 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1825.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1825 commit f01aca3c9201fa687ab7139f70b10f22682e7319 Author: Mark Payne Date: 2017-05-10T17:09:45Z NIFI-3863: Initial implementation of Lookup Services commit 1255f6994de2380299c5b99eba714e3700c24bc0 Author: Mark Payne Date: 2017-05-15T14:32:36Z NIFI-3863: Implemented LookupRecord processors. This required some refactoring of RecordSetWriter interface, so refactored that interface and all implementations annd referenceerences of it commit fa0726347af4d0da23b8b203aa0e824c743cf03a Author: Mark Payne Date: 2017-05-18T17:36:31Z NIFI-3857: Added PartitionRecord processor --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3853) Escape configurations prior to writing flow.xml
[ https://issues.apache.org/jira/browse/NIFI-3853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016134#comment-16016134 ] ASF subversion and git services commented on NIFI-3853: --- Commit be67f3552ba9abbb98aa29ab48c06264b77ab661 in nifi's branch refs/heads/master from [~joewitt] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=be67f35 ] NIFI-3853 fixing tests to properly encapsulate their environments This closes #1822. Signed-off-by: Bryan Bende > Escape configurations prior to writing flow.xml > --- > > Key: NIFI-3853 > URL: https://issues.apache.org/jira/browse/NIFI-3853 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Critical > Fix For: 1.3.0 > > > If invalid characters are entered through the UI, they can corrupt the > flow.xml.gz making it unparsable on future restarts. Once in this state, the > flow.xml.gz needs to be hand-edited to remove the invalid characters. > Characters outside the following ranges may cause issues: > {code} > #x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x1-#x10] > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi pull request #1822: NIFI-3853 fixing tests to properly encapsulate thei...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1822 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3853) Escape configurations prior to writing flow.xml
[ https://issues.apache.org/jira/browse/NIFI-3853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016135#comment-16016135 ] ASF GitHub Bot commented on NIFI-3853: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1822 > Escape configurations prior to writing flow.xml > --- > > Key: NIFI-3853 > URL: https://issues.apache.org/jira/browse/NIFI-3853 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Critical > Fix For: 1.3.0 > > > If invalid characters are entered through the UI, they can corrupt the > flow.xml.gz making it unparsable on future restarts. Once in this state, the > flow.xml.gz needs to be hand-edited to remove the invalid characters. > Characters outside the following ranges may cause issues: > {code} > #x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x1-#x10] > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (NIFI-3936) Follow the instructions step, maven build fails
shuguo zheng created NIFI-3936: -- Summary: Follow the instructions step, maven build fails Key: NIFI-3936 URL: https://issues.apache.org/jira/browse/NIFI-3936 Project: Apache NiFi Issue Type: Bug Components: Tools and Build Affects Versions: 1.2.0 Environment: windows7,centos6 Reporter: shuguo zheng When i execute “mvn -T 2.0C clean install ” or "mvn clean install" to compile nifi source, The following error occurred. How can i fix it? [ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.0:ins tall-node-and-npm (install-node-and-npm) on project nifi-web-ui: Could not extra ct the npm archive: Could not extract archive: 'F:\maven\com\github\eirslett\npm \1.3.8\npm-1.3.8.tar.gz': EOFException -> [Help 1] The detail error: [ERROR] No goals have been specified for this build. You must specify a valid li fecycle phase or a goal in the format : or :[:]:. Available lifecycle phases are : validate, initialize, generate-sources, process-sources, generate-resources, p rocess-resources, compile, process-classes, generate-test-sources, process-test- sources, generate-test-resources, process-test-resources, test-compile, process- test-classes, test, prepare-package, package, pre-integration-test, integration- test, post-integration-test, verify, install, deploy, pre-clean, clean, post-cle an, pre-site, site, post-site, site-deploy. -> [Help 1] org.apache.maven.lifecycle.NoGoalSpecifiedException: No goals have been specifie d for this build. You must specify a valid lifecycle phase or a goal in the form at : or :[:]:. Available lifecycle phases are: validate, initialize, generate-so urces, process-sources, generate-resources, process-resources, compile, process- classes, generate-test-sources, process-test-sources, generate-test-resources, p rocess-test-resources, test-compile, process-test-classes, test, prepare-package , package, pre-integration-test, integration-test, post-integration-test, verify , install, deploy, pre-clean, clean, post-clean, pre-site, site, post-site, site -deploy. at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(Lifecycl eStarter.java:90) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:355) at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:155) at org.apache.maven.cli.MavenCli.execute(MavenCli.java:584) at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:216) at org.apache.maven.cli.MavenCli.main(MavenCli.java:160) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces sorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Laun cher.java:289) at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.jav a:229) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(La uncher.java:415) at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java: 356) [ERROR] [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please rea d the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/NoGoalSpecifie dException -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi-minifi-cpp pull request #99: MINIFI-270: Move Base RPG capabilities bac...
Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/99 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3933) Cluster - Cluster Coordinator removal due to lack of heartbeats
[ https://issues.apache.org/jira/browse/NIFI-3933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016042#comment-16016042 ] Neil Derraugh commented on NIFI-3933: - In 1.2.0 you can't delete a CONNECTED node with the REST API. If anybody has any suggestions for workarounds please comment. > Cluster - Cluster Coordinator removal due to lack of heartbeats > --- > > Key: NIFI-3933 > URL: https://issues.apache.org/jira/browse/NIFI-3933 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Matt Gilman > > If the process for the acting Cluster Coordinator is killed, the node will > remain in the cluster despite not receiving heartbeats past the configured > threshold. If this same action is taken against a node that is not the > Cluster Coordinator, it will be removed once the configured threshold has > elapsed. > From the mailing list [1] > [1] > https://mail-archives.apache.org/mod_mbox/nifi-users/201705.mbox/%3C1495107006491-1942.post%40n4.nabble.com%3E > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi-minifi-cpp issue #95: MINIFI-290 Refactor Tests
Github user kevdoran commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/95 Will do. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #95: MINIFI-290 Refactor Tests
Github user apiri commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/95 Hey @kevdoran, #96 is now merged. Would you be able to rebase and see if that solves the problem you were encountering before? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #96: MINIFI-304: Update testing strategy
Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/96 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #96: MINIFI-304: Update testing strategy
Github user apiri commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/96 Thanks for the rebase. Ran through the usual battery of stuff and looks good. Will get this merged. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3853) Escape configurations prior to writing flow.xml
[ https://issues.apache.org/jira/browse/NIFI-3853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016005#comment-16016005 ] ASF GitHub Bot commented on NIFI-3853: -- Github user bbende commented on the issue: https://github.com/apache/nifi/pull/1822 +1 looks good, full build passes with contrib-check, will merge > Escape configurations prior to writing flow.xml > --- > > Key: NIFI-3853 > URL: https://issues.apache.org/jira/browse/NIFI-3853 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Critical > Fix For: 1.3.0 > > > If invalid characters are entered through the UI, they can corrupt the > flow.xml.gz making it unparsable on future restarts. Once in this state, the > flow.xml.gz needs to be hand-edited to remove the invalid characters. > Characters outside the following ranges may cause issues: > {code} > #x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x1-#x10] > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi issue #1822: NIFI-3853 fixing tests to properly encapsulate their envir...
Github user bbende commented on the issue: https://github.com/apache/nifi/pull/1822 +1 looks good, full build passes with contrib-check, will merge --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3536) Add GetSOAP Processor
[ https://issues.apache.org/jira/browse/NIFI-3536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16015976#comment-16015976 ] ASF GitHub Bot commented on NIFI-3536: -- GitHub user apsaltis opened a pull request: https://github.com/apache/nifi/pull/1824 NIFI-3536 Adding GetSOAP processor which supports generically invoking SOAP endpoints This is a follow-on to P/R #1741 that I closed to both rebase off the 1.3.0-SNAPSHOT source and also update the NOTICE in nifi-assembly with the missing artifacts Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [X] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? - [X] Is your initial contribution a single, squashed commit? ### For code changes: - [X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [X] Have you written or updated unit tests to verify your changes? - [X] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [X] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [X] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/apsaltis/nifi NIFI-3536 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1824.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1824 commit 083951c70cbcabdf90ab333bafaa1837904f078f Author: Andrew Psaltis Date: 2017-05-03T15:08:06Z NIFI-3536 Adding GetSOAP processor which supports generically invoking SOAP endpoints > Add GetSOAP Processor > - > > Key: NIFI-3536 > URL: https://issues.apache.org/jira/browse/NIFI-3536 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Andrew Psaltis >Assignee: Andrew Psaltis > > Although it may seem like everyone is now using REST or at least RESTful > services there still remains a need for many enterprises to interact with > SOAP endpoints. This JIRA is for a new processor that supports interacting > with SOAP endpoints. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi pull request #1824: NIFI-3536 Adding GetSOAP processor which supports g...
GitHub user apsaltis opened a pull request: https://github.com/apache/nifi/pull/1824 NIFI-3536 Adding GetSOAP processor which supports generically invoking SOAP endpoints This is a follow-on to P/R #1741 that I closed to both rebase off the 1.3.0-SNAPSHOT source and also update the NOTICE in nifi-assembly with the missing artifacts Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [X] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? - [X] Is your initial contribution a single, squashed commit? ### For code changes: - [X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [X] Have you written or updated unit tests to verify your changes? - [X] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [X] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [X] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/apsaltis/nifi NIFI-3536 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1824.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1824 commit 083951c70cbcabdf90ab333bafaa1837904f078f Author: Andrew Psaltis Date: 2017-05-03T15:08:06Z NIFI-3536 Adding GetSOAP processor which supports generically invoking SOAP endpoints --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3935) TestPersistentProvenanceRepository frequently fails in travis / likely slow machine timing bug in test
[ https://issues.apache.org/jira/browse/NIFI-3935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16015967#comment-16015967 ] ASF GitHub Bot commented on NIFI-3935: -- Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/1823 Reviewing... > TestPersistentProvenanceRepository frequently fails in travis / likely slow > machine timing bug in test > -- > > Key: NIFI-3935 > URL: https://issues.apache.org/jira/browse/NIFI-3935 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Joseph Witt >Assignee: Joseph Witt > > {quote} > Tests run: 35, Failures: 1, Errors: 0, Skipped: 2, Time elapsed: 99.013 sec > <<< FAILURE! - in > org.apache.nifi.provenance.TestPersistentProvenanceRepository > testAddAndRecover(org.apache.nifi.provenance.TestPersistentProvenanceRepository) > Time elapsed: 3.646 sec <<< FAILURE! > java.lang.AssertionError: expected:<10> but was:<9> > at org.junit.Assert.fail(Assert.java:88) > at org.junit.Assert.failNotEquals(Assert.java:834) > at org.junit.Assert.assertEquals(Assert.java:645) > at org.junit.Assert.assertEquals(Assert.java:631) > at > org.apache.nifi.provenance.TestPersistentProvenanceRepository.testAddAndRecover(TestPersistentProvenanceRepository.java:396) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi issue #1823: NIFI-3935 made the old persistent prov test more forgiving...
Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/1823 Reviewing... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3935) TestPersistentProvenanceRepository frequently fails in travis / likely slow machine timing bug in test
[ https://issues.apache.org/jira/browse/NIFI-3935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16015962#comment-16015962 ] ASF GitHub Bot commented on NIFI-3935: -- GitHub user joewitt opened a pull request: https://github.com/apache/nifi/pull/1823 NIFI-3935 made the old persistent prov test more forgiving of timing … …issues and extended lucene test timeouts for slower environments. These should be integration tests really. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/joewitt/incubator-nifi NIFI-3935 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1823.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1823 commit 6220e8f1a79aab62bcc39990d771bd2880e8e184 Author: joewitt Date: 2017-05-18T15:43:46Z NIFI-3935 made the old persistent prov test more forgiving of timing issues and extended lucene test timeouts for slower environments. These should be integration tests really. > TestPersistentProvenanceRepository frequently fails in travis / likely slow > machine timing bug in test > -- > > Key: NIFI-3935 > URL: https://issues.apache.org/jira/browse/NIFI-3935 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Joseph Witt >Assignee: Joseph Witt > > {quote} > Tests run: 35, Failures: 1, Errors: 0, Skipped: 2, Time elapsed: 99.013 sec > <<< FAILURE! - in > org.apache.nifi.provenance.TestPersistentProvenanceRepository > testAddAndRecover(org.apache.nifi.provenance.TestPersistentProvenanceRepository) > Time elapsed: 3.646 sec <<< FAILURE! > java.lang.AssertionError: expected:<10> but was:<9> > at org.junit.Assert.fail(Assert.java:88) > at org.junit.Assert.failNotEquals(Assert.java:834) > at org.junit.Assert.assertEquals(Assert.java:645) > at org.junit.Assert.assertEquals(Assert.java:631) > at > org.apache.nifi.provenance.TestPersistentProvenanceRepository.testAddAndRecover(TestPersistentProvenanceRepository.java:396) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi pull request #1823: NIFI-3935 made the old persistent prov test more fo...
GitHub user joewitt opened a pull request: https://github.com/apache/nifi/pull/1823 NIFI-3935 made the old persistent prov test more forgiving of timing ⦠â¦issues and extended lucene test timeouts for slower environments. These should be integration tests really. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/joewitt/incubator-nifi NIFI-3935 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1823.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1823 commit 6220e8f1a79aab62bcc39990d771bd2880e8e184 Author: joewitt Date: 2017-05-18T15:43:46Z NIFI-3935 made the old persistent prov test more forgiving of timing issues and extended lucene test timeouts for slower environments. These should be integration tests really. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3853) Escape configurations prior to writing flow.xml
[ https://issues.apache.org/jira/browse/NIFI-3853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16015953#comment-16015953 ] ASF GitHub Bot commented on NIFI-3853: -- Github user bbende commented on the issue: https://github.com/apache/nifi/pull/1822 Reviewing... > Escape configurations prior to writing flow.xml > --- > > Key: NIFI-3853 > URL: https://issues.apache.org/jira/browse/NIFI-3853 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Critical > Fix For: 1.3.0 > > > If invalid characters are entered through the UI, they can corrupt the > flow.xml.gz making it unparsable on future restarts. Once in this state, the > flow.xml.gz needs to be hand-edited to remove the invalid characters. > Characters outside the following ranges may cause issues: > {code} > #x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x1-#x10] > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi issue #1822: NIFI-3853 fixing tests to properly encapsulate their envir...
Github user bbende commented on the issue: https://github.com/apache/nifi/pull/1822 Reviewing... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---