[jira] [Commented] (NIFI-5615) Can't format the Avro data for Converting processors
[ https://issues.apache.org/jira/browse/NIFI-5615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621411#comment-16621411 ] Kemix Koo commented on NIFI-5615: - Add the PR: [https://github.com/apache/nifi/pull/3016] > Can't format the Avro data for Converting processors > > > Key: NIFI-5615 > URL: https://issues.apache.org/jira/browse/NIFI-5615 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Kemix Koo >Priority: Minor > > When check the "List queue", and view the contents, it's Avro-binary > obviously, but can't view as "formatted". Missing to set MIME type for > FlowFile. > Processors: > * ConvertAvroSchema > * ConvertCSVToAvro > * ConvertJSONToAvro > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (NIFI-5615) Can't format the Avro data for Converting processors
[ https://issues.apache.org/jira/browse/NIFI-5615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kemix Koo updated NIFI-5615: Comment: was deleted (was: Add the PR: [https://github.com/apache/nifi/pull/3016] ) > Can't format the Avro data for Converting processors > > > Key: NIFI-5615 > URL: https://issues.apache.org/jira/browse/NIFI-5615 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Kemix Koo >Priority: Minor > > When check the "List queue", and view the contents, it's Avro-binary > obviously, but can't view as "formatted". Missing to set MIME type for > FlowFile. > Processors: > * ConvertAvroSchema > * ConvertCSVToAvro > * ConvertJSONToAvro > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5615) Can't format the Avro data for Converting processors
[ https://issues.apache.org/jira/browse/NIFI-5615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621409#comment-16621409 ] ASF GitHub Bot commented on NIFI-5615: -- GitHub user kemixkoo opened a pull request: https://github.com/apache/nifi/pull/3016 NIFI-5615: set the avro-binary MIME type for converting processors Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/kemixkoo/nifi master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3016.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3016 commit d6341bc2b86dd84b531ca2cb6b8dacf4b30e5270 Author: Kemix Koo Date: 2018-09-20T01:59:15Z NIFI-5615: set the avro-binary MIME type for converting processors > Can't format the Avro data for Converting processors > > > Key: NIFI-5615 > URL: https://issues.apache.org/jira/browse/NIFI-5615 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Kemix Koo >Priority: Minor > > When check the "List queue", and view the contents, it's Avro-binary > obviously, but can't view as "formatted". Missing to set MIME type for > FlowFile. > Processors: > * ConvertAvroSchema > * ConvertCSVToAvro > * ConvertJSONToAvro > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #3016: NIFI-5615: set the avro-binary MIME type for conver...
GitHub user kemixkoo opened a pull request: https://github.com/apache/nifi/pull/3016 NIFI-5615: set the avro-binary MIME type for converting processors Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/kemixkoo/nifi master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3016.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3016 commit d6341bc2b86dd84b531ca2cb6b8dacf4b30e5270 Author: Kemix Koo Date: 2018-09-20T01:59:15Z NIFI-5615: set the avro-binary MIME type for converting processors ---
[jira] [Created] (NIFI-5615) Can't format the Avro data for Converting processors
Kemix Koo created NIFI-5615: --- Summary: Can't format the Avro data for Converting processors Key: NIFI-5615 URL: https://issues.apache.org/jira/browse/NIFI-5615 Project: Apache NiFi Issue Type: Bug Components: Extensions Reporter: Kemix Koo When check the "List queue", and view the contents, it's Avro-binary obviously, but can't view as "formatted". Missing to set MIME type for FlowFile. Processors: * ConvertAvroSchema * ConvertCSVToAvro * ConvertJSONToAvro -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-614) Reduce overhead of AgentInformation::serialize
[ https://issues.apache.org/jira/browse/MINIFICPP-614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621390#comment-16621390 ] ASF GitHub Bot commented on MINIFICPP-614: -- GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/401 MINIFICPP-614: Don't rebuild agent information every call Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFICPP-614 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/401.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #401 commit 16caf42ca514d6896e384feaf51d65d44e48e1b7 Author: Marc Parisi Date: 2018-09-20T01:30:58Z MINIFICPP-614: Don't rebuild agent information every call > Reduce overhead of AgentInformation::serialize > -- > > Key: MINIFICPP-614 > URL: https://issues.apache.org/jira/browse/MINIFICPP-614 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > > Serialize is taking more cycles than necessary. we can improve this > dramatically. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #401: MINIFICPP-614: Don't rebuild agent inform...
GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/401 MINIFICPP-614: Don't rebuild agent information every call Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFICPP-614 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/401.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #401 commit 16caf42ca514d6896e384feaf51d65d44e48e1b7 Author: Marc Parisi Date: 2018-09-20T01:30:58Z MINIFICPP-614: Don't rebuild agent information every call ---
[jira] [Created] (MINIFICPP-614) Reduce overhead of AgentInformation::serialize
Mr TheSegfault created MINIFICPP-614: Summary: Reduce overhead of AgentInformation::serialize Key: MINIFICPP-614 URL: https://issues.apache.org/jira/browse/MINIFICPP-614 Project: NiFi MiNiFi C++ Issue Type: Improvement Reporter: Mr TheSegfault Assignee: Mr TheSegfault Serialize is taking more cycles than necessary. we can improve this dramatically. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5612) org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
[
https://issues.apache.org/jira/browse/NIFI-5612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621346#comment-16621346
]
Colin Dean commented on NIFI-5612:
--
My work thus far is on GitHub:
https://github.com/apache/nifi/compare/support/nifi-1.7.x...colindean:colindean/nifi-5612-unresolvedunion.
I'm not sure if I should be building off of the {{support/nifi-1.7.x}} branch
but it shouldn't be painful to go back off of master.
> org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
>
>
> Key: NIFI-5612
> URL: https://issues.apache.org/jira/browse/NIFI-5612
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.5.0, 1.6.0, 1.7.1
> Environment: Microsoft Windows, MySQL Enterprise 5.0.80
>Reporter: Colin Dean
>Priority: Major
> Labels: ExecuteSQL, avro, nifi
>
> I'm seeing this when I execute {{SELECT * FROM }} on a few tables
> but not on dozens of others in the same database.
> {code}
> 2018-09-13 01:11:31,434 WARN [Timer-Driven Process Thread-8]
> o.a.n.controller.tasks.ConnectableTask Administratively Yielding
> ExecuteSQL[id=cf5c0996-eddf-3e05-25a3-c407c5edf990] due to uncaught
> Exception: org.apache.avro.file.DataFileWriter$AppendWriteException:
> org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
> org.apache.avro.file.DataFileWriter$AppendWriteException:
> org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
> at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:308)
> at
> org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:462)
> at
> org.apache.nifi.processors.standard.ExecuteSQL.lambda$onTrigger$1(ExecuteSQL.java:252)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2625)
> at
> org.apache.nifi.processors.standard.ExecuteSQL.onTrigger(ExecuteSQL.java:242)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1165)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:203)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.avro.UnresolvedUnionException: Not in union
> ["null","int"]: 0
> at
> org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:709)
> at
> org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:192)
> at
> org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:110)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
> at
> org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:153)
> at
> org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:143)
> at
> org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:105)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:60)
> at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:302)
> ... 15 common frames omitted
> {code}
> I don't know if I can share the database schema – still working with my team
> on that – but looking at it, I think it has something to do with the
> signedness of int(1) or tinyint(1) because those two are the only numerical
> types common to all of the table.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5612) org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
[
https://issues.apache.org/jira/browse/NIFI-5612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621342#comment-16621342
]
Colin Dean commented on NIFI-5612:
--
I dug a little deeper into how the schemas are built. In my original error, the
union type is null or int. The big switch in
{{org.apache.nifi.processors.standard.util.JdbcCommon#createSchema(java.sql.ResultSet,
org.apache.nifi.processors.standard.util.JdbcCommon.AvroConversionOptions)}}
yields SQL column types that produce that union type in Avro:
1. INTEGER, but only when signed or precision is set and less than 9 (otherwise
null and long)
2. SMALLINT or TINYINT
That's it. So, I'll the error could only be happening on these columns (not all
in the same table):
{code:sql}
`FaxCoverPage` int(1) unsigned NOT NULL default '0',
`CopayChanged` int(6) unsigned NOT NULL default '0',
`ClaimRequired` int(6) unsigned NOT NULL default '0',
`ClaimReq` int(6) unsigned NOT NULL default '1',
`prNotesSendStatus` int(1) unsigned NOT NULL default '0',
`inactive` int(1) unsigned default '0',
`GrType` int(2) unsigned NOT NULL default '0',
`nostatements` int(6) unsigned NOT NULL default '0',
`interpretSvc` tinyint(1) unsigned default NULL,
`GlobalAlert` smallint(2) unsigned NOT NULL default '0',
`irisenroll` tinyint(1) unsigned NOT NULL default '0',
`GrRel` int(2) unsigned NOT NULL default '0',
`IsGrPt` int(2) unsigned NOT NULL default '1',
`OBFLabType` smallint(5) unsigned NOT NULL default '0',
`futureflag` int(2) unsigned NOT NULL default '0',
`DeleteFlag` tinyint(3) unsigned NOT NULL default '0',
`Type` tinyint(3) unsigned NOT NULL default '0',
`GivenToCollection` tinyint(3) unsigned NOT NULL default '0',
… a few more
{code}
{{int(1) unsigned}} exists in tables that are successfully retrieved, all the
way up to {{int(7)}} on a quick glance. No 8, no 9.
> org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
>
>
> Key: NIFI-5612
> URL: https://issues.apache.org/jira/browse/NIFI-5612
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.5.0, 1.6.0, 1.7.1
> Environment: Microsoft Windows, MySQL Enterprise 5.0.80
>Reporter: Colin Dean
>Priority: Major
> Labels: ExecuteSQL, avro, nifi
>
> I'm seeing this when I execute {{SELECT * FROM }} on a few tables
> but not on dozens of others in the same database.
> {code}
> 2018-09-13 01:11:31,434 WARN [Timer-Driven Process Thread-8]
> o.a.n.controller.tasks.ConnectableTask Administratively Yielding
> ExecuteSQL[id=cf5c0996-eddf-3e05-25a3-c407c5edf990] due to uncaught
> Exception: org.apache.avro.file.DataFileWriter$AppendWriteException:
> org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
> org.apache.avro.file.DataFileWriter$AppendWriteException:
> org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
> at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:308)
> at
> org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:462)
> at
> org.apache.nifi.processors.standard.ExecuteSQL.lambda$onTrigger$1(ExecuteSQL.java:252)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2625)
> at
> org.apache.nifi.processors.standard.ExecuteSQL.onTrigger(ExecuteSQL.java:242)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1165)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:203)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.avro.UnresolvedUnionException: Not in union
> ["null","int"]: 0
> at
> org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:709)
> at
> org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:192)
[jira] [Commented] (NIFI-5254) Upgrade to Groovy 2.5.0
[ https://issues.apache.org/jira/browse/NIFI-5254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621337#comment-16621337 ] ASF GitHub Bot commented on NIFI-5254: -- GitHub user MikeThomsen opened a pull request: https://github.com/apache/nifi/pull/3015 NIFI-5254 [WIP] Updated to Groovy 2.5.2 Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/MikeThomsen/nifi NIFI-5254 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3015.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3015 commit 248e2656f2216156fe59fcde0a93f6157140dab8 Author: Mike Thomsen Date: 2018-09-20T00:06:29Z NIFI-5254 [WIP] Updated to Groovy 2.5.2 > Upgrade to Groovy 2.5.0 > --- > > Key: NIFI-5254 > URL: https://issues.apache.org/jira/browse/NIFI-5254 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mike Thomsen >Assignee: Mike Thomsen >Priority: Major > > Groovy 2.5 has been released and support for it should be added. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #3015: NIFI-5254 [WIP] Updated to Groovy 2.5.2
GitHub user MikeThomsen opened a pull request: https://github.com/apache/nifi/pull/3015 NIFI-5254 [WIP] Updated to Groovy 2.5.2 Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/MikeThomsen/nifi NIFI-5254 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3015.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3015 commit 248e2656f2216156fe59fcde0a93f6157140dab8 Author: Mike Thomsen Date: 2018-09-20T00:06:29Z NIFI-5254 [WIP] Updated to Groovy 2.5.2 ---
[jira] [Commented] (NIFI-5612) org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
[
https://issues.apache.org/jira/browse/NIFI-5612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621329#comment-16621329
]
Colin Dean commented on NIFI-5612:
--
I set up a parameterized test that checks TINYINT, SMALLINT, INTEGER, and
BIGINT with precisions from 1 to 64, both signedness: 512 tests, 94 of which
fail. I'm not sure which of these that fail don't make sense.
* INTEGER fails at precision 9 through 64 only when unsigned.
* BIGINT fails at precision 1 through 19, regardless of signedness.
I've got tables with columns defined like so:
{code:sql}
`LoginID` int(11) unsigned NOT NULL default '0',
`DoctorID` int(11) unsigned NOT NULL default '0',
`PatientID` int(11) unsigned NOT NULL default '0',
{code}
This leads me to believe that these are the columns that may be triggering this
error in "production" ExecuteSQL. However, looking through tables that are
succeeding, I see {{int(11) unsigned NOT NULL default '0',}} in those columns.
> org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
>
>
> Key: NIFI-5612
> URL: https://issues.apache.org/jira/browse/NIFI-5612
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.5.0, 1.6.0, 1.7.1
> Environment: Microsoft Windows, MySQL Enterprise 5.0.80
>Reporter: Colin Dean
>Priority: Major
> Labels: ExecuteSQL, avro, nifi
>
> I'm seeing this when I execute {{SELECT * FROM }} on a few tables
> but not on dozens of others in the same database.
> {code}
> 2018-09-13 01:11:31,434 WARN [Timer-Driven Process Thread-8]
> o.a.n.controller.tasks.ConnectableTask Administratively Yielding
> ExecuteSQL[id=cf5c0996-eddf-3e05-25a3-c407c5edf990] due to uncaught
> Exception: org.apache.avro.file.DataFileWriter$AppendWriteException:
> org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
> org.apache.avro.file.DataFileWriter$AppendWriteException:
> org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
> at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:308)
> at
> org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:462)
> at
> org.apache.nifi.processors.standard.ExecuteSQL.lambda$onTrigger$1(ExecuteSQL.java:252)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2625)
> at
> org.apache.nifi.processors.standard.ExecuteSQL.onTrigger(ExecuteSQL.java:242)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1165)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:203)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.avro.UnresolvedUnionException: Not in union
> ["null","int"]: 0
> at
> org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:709)
> at
> org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:192)
> at
> org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:110)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
> at
> org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:153)
> at
> org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:143)
> at
> org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:105)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:60)
> at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:302)
> ... 15 common frames omitted
> {code}
> I don't know if I can share the database schema – still working with my team
> on that – but looking at it, I think
[jira] [Updated] (NIFI-5614) Upgrade commons-dbcp to latest commons-dbcp2
[ https://issues.apache.org/jira/browse/NIFI-5614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-5614: --- Status: Patch Available (was: In Progress) > Upgrade commons-dbcp to latest commons-dbcp2 > > > Key: NIFI-5614 > URL: https://issues.apache.org/jira/browse/NIFI-5614 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > > DBCPConnectionPool currently uses Commons DBCP 1.4. There have been [many > improvements and a major > release|https://commons.apache.org/proper/commons-dbcp/changes-report.html#a2.5.0] > since then, at the time of this writing the latest version is 2.5.0. > This Jira proposes to upgrade the version of commons-dbcp to commons-dbcp2 > version 2.5.0 in order to benefit from the improvements/additions. This will > involve some code changes in addition to the updated dependencies, as the API > has changed slightly to use different terminology e.g. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5614) Upgrade commons-dbcp to latest commons-dbcp2
[ https://issues.apache.org/jira/browse/NIFI-5614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621323#comment-16621323 ] ASF GitHub Bot commented on NIFI-5614: -- GitHub user mattyb149 opened a pull request: https://github.com/apache/nifi/pull/3014 NIFI-5614: Update commons-dbcp to commons-dbcp2 for DBCPConnectionPool Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [x] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mattyb149/nifi NIFI-5614 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3014.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3014 commit 752af4509dea48296ad5f561ee99eeb9c7674a9e Author: Matthew Burgess Date: 2018-09-19T23:34:17Z NIFI-5614: Update commons-dbcp to commons-dbcp2 for DBCPConnectionPool > Upgrade commons-dbcp to latest commons-dbcp2 > > > Key: NIFI-5614 > URL: https://issues.apache.org/jira/browse/NIFI-5614 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > > DBCPConnectionPool currently uses Commons DBCP 1.4. There have been [many > improvements and a major > release|https://commons.apache.org/proper/commons-dbcp/changes-report.html#a2.5.0] > since then, at the time of this writing the latest version is 2.5.0. > This Jira proposes to upgrade the version of commons-dbcp to commons-dbcp2 > version 2.5.0 in order to benefit from the improvements/additions. This will > involve some code changes in addition to the updated dependencies, as the API > has changed slightly to use different terminology e.g. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #3014: NIFI-5614: Update commons-dbcp to commons-dbcp2 for...
GitHub user mattyb149 opened a pull request: https://github.com/apache/nifi/pull/3014 NIFI-5614: Update commons-dbcp to commons-dbcp2 for DBCPConnectionPool Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [x] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mattyb149/nifi NIFI-5614 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3014.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3014 commit 752af4509dea48296ad5f561ee99eeb9c7674a9e Author: Matthew Burgess Date: 2018-09-19T23:34:17Z NIFI-5614: Update commons-dbcp to commons-dbcp2 for DBCPConnectionPool ---
[jira] [Assigned] (NIFI-5614) Upgrade commons-dbcp to latest commons-dbcp2
[ https://issues.apache.org/jira/browse/NIFI-5614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess reassigned NIFI-5614: -- Assignee: Matt Burgess > Upgrade commons-dbcp to latest commons-dbcp2 > > > Key: NIFI-5614 > URL: https://issues.apache.org/jira/browse/NIFI-5614 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > > DBCPConnectionPool currently uses Commons DBCP 1.4. There have been [many > improvements and a major > release|https://commons.apache.org/proper/commons-dbcp/changes-report.html#a2.5.0] > since then, at the time of this writing the latest version is 2.5.0. > This Jira proposes to upgrade the version of commons-dbcp to commons-dbcp2 > version 2.5.0 in order to benefit from the improvements/additions. This will > involve some code changes in addition to the updated dependencies, as the API > has changed slightly to use different terminology e.g. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFI-5614) Upgrade commons-dbcp to latest commons-dbcp2
Matt Burgess created NIFI-5614: -- Summary: Upgrade commons-dbcp to latest commons-dbcp2 Key: NIFI-5614 URL: https://issues.apache.org/jira/browse/NIFI-5614 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: Matt Burgess DBCPConnectionPool currently uses Commons DBCP 1.4. There have been [many improvements and a major release|https://commons.apache.org/proper/commons-dbcp/changes-report.html#a2.5.0] since then, at the time of this writing the latest version is 2.5.0. This Jira proposes to upgrade the version of commons-dbcp to commons-dbcp2 version 2.5.0 in order to benefit from the improvements/additions. This will involve some code changes in addition to the updated dependencies, as the API has changed slightly to use different terminology e.g. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5612) org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
[
https://issues.apache.org/jira/browse/NIFI-5612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621249#comment-16621249
]
Colin Dean commented on NIFI-5612:
--
I added a few tests and found one that triggers _an_ error: an unsigned integer
with a precision of 11.
{code}
[ERROR]
testConvertToAvroStreamForUnsignedIntegerWithPrecision11(org.apache.nifi.processors.standard.util.TestJdbcCommon)
Time elapsed: 0.012 s <<< ERROR!
java.lang.RuntimeException: Unable to resolve union for value 0 with type
java.lang.Integer
at
org.apache.nifi.processors.standard.util.TestJdbcCommon.convertResultSetToAvroInputStream(TestJdbcCommon.java:601)
at
org.apache.nifi.processors.standard.util.TestJdbcCommon.testConvertToAvroStreamForUnsignedIntegerWithPrecision11(TestJdbcCommon.java:650)
Caused by: org.apache.avro.file.DataFileWriter$AppendWriteException:
org.apache.avro.UnresolvedUnionException: Not in union ["null","long"]: 0
at
org.apache.nifi.processors.standard.util.TestJdbcCommon.convertResultSetToAvroInputStream(TestJdbcCommon.java:601)
at
org.apache.nifi.processors.standard.util.TestJdbcCommon.testConvertToAvroStreamForUnsignedIntegerWithPrecision11(TestJdbcCommon.java:650)
Caused by: org.apache.avro.UnresolvedUnionException: Not in union
["null","long"]: 0
at
org.apache.nifi.processors.standard.util.TestJdbcCommon.convertResultSetToAvroInputStream(TestJdbcCommon.java:601)
at
org.apache.nifi.processors.standard.util.TestJdbcCommon.testConvertToAvroStreamForUnsignedIntegerWithPrecision11(TestJdbcCommon.java:650)
{code}
However, this is not yet _my_ error so I have more tests to add.
> org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
>
>
> Key: NIFI-5612
> URL: https://issues.apache.org/jira/browse/NIFI-5612
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.5.0, 1.6.0, 1.7.1
> Environment: Microsoft Windows, MySQL Enterprise 5.0.80
>Reporter: Colin Dean
>Priority: Major
> Labels: ExecuteSQL, avro, nifi
>
> I'm seeing this when I execute {{SELECT * FROM }} on a few tables
> but not on dozens of others in the same database.
> {code}
> 2018-09-13 01:11:31,434 WARN [Timer-Driven Process Thread-8]
> o.a.n.controller.tasks.ConnectableTask Administratively Yielding
> ExecuteSQL[id=cf5c0996-eddf-3e05-25a3-c407c5edf990] due to uncaught
> Exception: org.apache.avro.file.DataFileWriter$AppendWriteException:
> org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
> org.apache.avro.file.DataFileWriter$AppendWriteException:
> org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
> at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:308)
> at
> org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:462)
> at
> org.apache.nifi.processors.standard.ExecuteSQL.lambda$onTrigger$1(ExecuteSQL.java:252)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2625)
> at
> org.apache.nifi.processors.standard.ExecuteSQL.onTrigger(ExecuteSQL.java:242)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1165)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:203)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.avro.UnresolvedUnionException: Not in union
> ["null","int"]: 0
> at
> org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:709)
> at
> org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:192)
> at
> org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:110)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
> at
>
[jira] [Commented] (NIFI-4806) Upgrade to Tika/Tika-Parsers 1.19 to resolve TIKA-2535 and CVEs reported against all releases prior to 1.19
[ https://issues.apache.org/jira/browse/NIFI-4806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621240#comment-16621240 ] Joseph Witt commented on NIFI-4806: --- updated all explicit refs to commons-text 1.4 commons-lang3 3.8 commons-compress 1.18 commons-io 2.6 commons-codec 1.11 commons-net 3.6 commons-jexl3 3.1 commons-email 1.5 commons-logging 1.2 commons-csv 1.5 commons-cli 1.4 commons-collections 4.2 commons-collections4 4.2 commons-beanutils 1.9.3 commons-pool 2.6.0 commons-dbcp 2.5.0 commons-configuration 2.3 > Upgrade to Tika/Tika-Parsers 1.19 to resolve TIKA-2535 and CVEs reported > against all releases prior to 1.19 > --- > > Key: NIFI-4806 > URL: https://issues.apache.org/jira/browse/NIFI-4806 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.5.0 >Reporter: Joseph Witt >Assignee: Joseph Witt >Priority: Blocker > Fix For: 1.8.0 > > > The nifi-media-processors depend on Tika-Parsers 1.16 as of now. They need > to be upgraded to at least Tika-Parsers 1.18 to resolve the licensing problem > identified to the Tika team and which they resolved in > https://issues.apache.org/jira/browse/TIKA-2535 > > We aren't susceptible to the licensing problem at this time because when we > did the last update for Tika-Parsers this was flagged and excluded though we > could be exposing bugs for certain datayptes we'd do mime detection on > (maybe). I have a comment about this in our pom. > > This Jira is to upgrade, ensure no invalid libs are used, and clean up the > comments and move on. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4685) Hardcoded path when creating Bootstrap ClassLoader in Nifi
[
https://issues.apache.org/jira/browse/NIFI-4685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620807#comment-16620807
]
Imre edited comment on NIFI-4685 at 9/19/18 9:44 PM:
-
So, looking at the source, in my case starting Java on the command-line from
the {{C:\Program Files\NiFi}} directory also does not work, because
{{/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/RunNiFi.java}} makes
the assumption that the parent dir of the bootstrap config file will be the
root (the "workingDir") of all things NiFi. Which in my case is also not true,
as I'm putting the configuration files in a different directory: {{C:\Program
Files\My Program\nifi-conf\}}...
{{public void start() throws IOException, InterruptedException {}}
{{...}}
{{final File bootstrapConfigAbsoluteFile =
bootstrapConfigFile.getAbsoluteFile();}}
{{final File binDir = bootstrapConfigAbsoluteFile.getParentFile();}}
{{final File workingDir = binDir.getParentFile();}}
{{...}}
{{}}}
was (Author: olajos):
So, looking at the source, in my case starting Java on the command-line from
the {{C:\Program Files\NiFi}} directory also does not work, because
{{/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/RunNiFi.java}} makes
the assumption that the second up parent dir of the bootstrap config file will
be the root (the "workingDir") of all things NiFi. Which in my case is also not
true, as I'm putting the configuration files in a different directory:
{{C:\Program Files\My Program\nifi-conf\}}...
{{public void start() throws IOException, InterruptedException {}}
{{...}}
{{final File bootstrapConfigAbsoluteFile =
bootstrapConfigFile.getAbsoluteFile();}}
{{final File binDir = bootstrapConfigAbsoluteFile.getParentFile();}}
{{final File workingDir = binDir.getParentFile();}}
{{...}}
{{}}}
> Hardcoded path when creating Bootstrap ClassLoader in Nifi
> --
>
> Key: NIFI-4685
> URL: https://issues.apache.org/jira/browse/NIFI-4685
> Project: Apache NiFi
> Issue Type: Bug
> Components: Configuration, Core Framework
>Affects Versions: 1.4.0
> Environment: Windows 7, JRE 1.8.0_144
>Reporter: Sorin Florea
>Priority: Minor
> Labels: easyfix, usability, windows
>
> Found a hardcoded path to {{lib/bootstrap}} in
> [org.apache.nifi.NiFi|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-runtime/src/main/java/org/apache/nifi/NiFi.java#L3]
> class when calling {{createBootstrapClassLoader()}} :
> {code:java}
> private static ClassLoader createBootstrapClassLoader() throws IOException {
> //Get list of files in bootstrap folder
> final List urls = new ArrayList<>();
> Files.list(Paths.get({color:red}_"lib/bootstrap"_{color})).forEach(p
> -> {
> try {
> urls.add(p.toUri().toURL());
> } catch (final MalformedURLException mef) {
> LOGGER.warn("Unable to load " + p.getFileName() + " due to "
> + mef, mef);
> }
> });
> //Create the bootstrap classloader
> return new URLClassLoader(urls.toArray(new URL[0]),
> Thread.currentThread().getContextClassLoader());
> }
> {code}
> There are multiple places where the {{.\lib}} directory is defined:
> * {{BOOTSTRAP_LIBS}} or {{%LIB_DIR%}} environment variables in {{nifi.sh}} or
> {{run-nifi.bat}}
> * {{lib.dir}} in {{bootstrap.conf}} is most likely it
> * {{nifi.nar.library.directory}} or {{nifi.web.war.directory}} identify the
> library folder but don't quite refer to the bootstrap
> Perhaps using {{lib.dir}} in {{bootstrap.conf}} or create a new
> {{lib.bootstrap.dir}} property.
> Using different library folders would allow configuring multiple instances of
> Nifi using the same resource. I managed changing the properties and
> environment variables above to point to a common {{lib}} directory but this
> harcoded prevented {{run-nifi.bat}} to run successfully. {{dump-nifi.bat}}
> and {{status-nifi.bat}} ran without issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4806) Upgrade to Tika/Tika-Parsers 1.19 to resolve TIKA-2535 and CVEs reported against all releases prior to 1.19
[ https://issues.apache.org/jira/browse/NIFI-4806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621228#comment-16621228 ] Joseph Witt commented on NIFI-4806: --- updating all commons-compress 1.16.1 to 1.18 > Upgrade to Tika/Tika-Parsers 1.19 to resolve TIKA-2535 and CVEs reported > against all releases prior to 1.19 > --- > > Key: NIFI-4806 > URL: https://issues.apache.org/jira/browse/NIFI-4806 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.5.0 >Reporter: Joseph Witt >Assignee: Joseph Witt >Priority: Blocker > Fix For: 1.8.0 > > > The nifi-media-processors depend on Tika-Parsers 1.16 as of now. They need > to be upgraded to at least Tika-Parsers 1.18 to resolve the licensing problem > identified to the Tika team and which they resolved in > https://issues.apache.org/jira/browse/TIKA-2535 > > We aren't susceptible to the licensing problem at this time because when we > did the last update for Tika-Parsers this was flagged and excluded though we > could be exposing bugs for certain datayptes we'd do mime detection on > (maybe). I have a comment about this in our pom. > > This Jira is to upgrade, ensure no invalid libs are used, and clean up the > comments and move on. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4806) Upgrade to Tika/Tika-Parsers 1.19 to resolve TIKA-2535 and CVEs reported against all releases prior to 1.19
[ https://issues.apache.org/jira/browse/NIFI-4806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621227#comment-16621227 ] Joseph Witt commented on NIFI-4806: --- also see a lot of deprecation warnings about StringEscapeUtils deprecated in commons-lang3 and now in commons-text. Fixing that. And updating all use of commons-lang3 > Upgrade to Tika/Tika-Parsers 1.19 to resolve TIKA-2535 and CVEs reported > against all releases prior to 1.19 > --- > > Key: NIFI-4806 > URL: https://issues.apache.org/jira/browse/NIFI-4806 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.5.0 >Reporter: Joseph Witt >Assignee: Joseph Witt >Priority: Blocker > Fix For: 1.8.0 > > > The nifi-media-processors depend on Tika-Parsers 1.16 as of now. They need > to be upgraded to at least Tika-Parsers 1.18 to resolve the licensing problem > identified to the Tika team and which they resolved in > https://issues.apache.org/jira/browse/TIKA-2535 > > We aren't susceptible to the licensing problem at this time because when we > did the last update for Tika-Parsers this was flagged and excluded though we > could be exposing bugs for certain datayptes we'd do mime detection on > (maybe). I have a comment about this in our pom. > > This Jira is to upgrade, ensure no invalid libs are used, and clean up the > comments and move on. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-5609) NPE thrown if user attempts to view Status History too early
[
https://issues.apache.org/jira/browse/NIFI-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess updated NIFI-5609:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
> NPE thrown if user attempts to view Status History too early
>
>
> Key: NIFI-5609
> URL: https://issues.apache.org/jira/browse/NIFI-5609
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.8.0
>
>
> I started NiFi and then quickly went to the UI and right-clicked on a
> processor and tried to view Status History. I got back an error. The
> user-logs show the following (partial) stack trace:
> {code:java}
> 2018-09-19 13:20:19,928 ERROR [NiFi Web Server-24]
> o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred:
> java.lang.NullPointerException. Returning Internal Server Error response.
> java.lang.NullPointerException: null
> at
> org.apache.nifi.controller.status.history.StatusHistoryUtil.createStatusHistoryDTO(StatusHistoryUtil.java:39)
> at
> org.apache.nifi.controller.FlowController.getProcessorStatusHistory(FlowController.java:5066)
> at
> org.apache.nifi.controller.FlowController.getProcessorStatusHistory(FlowController.java:5062)
> at
> org.apache.nifi.web.controller.ControllerFacade.getProcessorStatusHistory(ControllerFacade.java:299)
> at
> org.apache.nifi.web.controller.ControllerFacade$$FastClassBySpringCGLIB$$5a42ba54.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
> at
> org.apache.nifi.web.controller.ControllerFacade$$EnhancerBySpringCGLIB$$8aeed060.getProcessorStatusHistory()
> at
> org.apache.nifi.web.StandardNiFiServiceFacade.getProcessorStatusHistory(StandardNiFiServiceFacade.java:2950)
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5609) NPE thrown if user attempts to view Status History too early
[
https://issues.apache.org/jira/browse/NIFI-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621201#comment-16621201
]
ASF GitHub Bot commented on NIFI-5609:
--
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/3012
> NPE thrown if user attempts to view Status History too early
>
>
> Key: NIFI-5609
> URL: https://issues.apache.org/jira/browse/NIFI-5609
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.8.0
>
>
> I started NiFi and then quickly went to the UI and right-clicked on a
> processor and tried to view Status History. I got back an error. The
> user-logs show the following (partial) stack trace:
> {code:java}
> 2018-09-19 13:20:19,928 ERROR [NiFi Web Server-24]
> o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred:
> java.lang.NullPointerException. Returning Internal Server Error response.
> java.lang.NullPointerException: null
> at
> org.apache.nifi.controller.status.history.StatusHistoryUtil.createStatusHistoryDTO(StatusHistoryUtil.java:39)
> at
> org.apache.nifi.controller.FlowController.getProcessorStatusHistory(FlowController.java:5066)
> at
> org.apache.nifi.controller.FlowController.getProcessorStatusHistory(FlowController.java:5062)
> at
> org.apache.nifi.web.controller.ControllerFacade.getProcessorStatusHistory(ControllerFacade.java:299)
> at
> org.apache.nifi.web.controller.ControllerFacade$$FastClassBySpringCGLIB$$5a42ba54.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
> at
> org.apache.nifi.web.controller.ControllerFacade$$EnhancerBySpringCGLIB$$8aeed060.getProcessorStatusHistory()
> at
> org.apache.nifi.web.StandardNiFiServiceFacade.getProcessorStatusHistory(StandardNiFiServiceFacade.java:2950)
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5609) NPE thrown if user attempts to view Status History too early
[
https://issues.apache.org/jira/browse/NIFI-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621200#comment-16621200
]
ASF subversion and git services commented on NIFI-5609:
---
Commit 3dd548e807c80749a99f83e18857cab879cd9c21 in nifi's branch
refs/heads/master from [~markap14]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=3dd548e ]
NIFI-5609: Fixed NullPointer when attempting to view status history for a
component that has not yet run
Signed-off-by: Matthew Burgess
This closes #3012
> NPE thrown if user attempts to view Status History too early
>
>
> Key: NIFI-5609
> URL: https://issues.apache.org/jira/browse/NIFI-5609
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.8.0
>
>
> I started NiFi and then quickly went to the UI and right-clicked on a
> processor and tried to view Status History. I got back an error. The
> user-logs show the following (partial) stack trace:
> {code:java}
> 2018-09-19 13:20:19,928 ERROR [NiFi Web Server-24]
> o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred:
> java.lang.NullPointerException. Returning Internal Server Error response.
> java.lang.NullPointerException: null
> at
> org.apache.nifi.controller.status.history.StatusHistoryUtil.createStatusHistoryDTO(StatusHistoryUtil.java:39)
> at
> org.apache.nifi.controller.FlowController.getProcessorStatusHistory(FlowController.java:5066)
> at
> org.apache.nifi.controller.FlowController.getProcessorStatusHistory(FlowController.java:5062)
> at
> org.apache.nifi.web.controller.ControllerFacade.getProcessorStatusHistory(ControllerFacade.java:299)
> at
> org.apache.nifi.web.controller.ControllerFacade$$FastClassBySpringCGLIB$$5a42ba54.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
> at
> org.apache.nifi.web.controller.ControllerFacade$$EnhancerBySpringCGLIB$$8aeed060.getProcessorStatusHistory()
> at
> org.apache.nifi.web.StandardNiFiServiceFacade.getProcessorStatusHistory(StandardNiFiServiceFacade.java:2950)
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5609) NPE thrown if user attempts to view Status History too early
[
https://issues.apache.org/jira/browse/NIFI-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621199#comment-16621199
]
ASF GitHub Bot commented on NIFI-5609:
--
Github user mattyb149 commented on the issue:
https://github.com/apache/nifi/pull/3012
+1 LGTM, ran build with unit tests and contrib-check, tried on a live NiFi
instance, verified no NPE is thrown when PG status history is requested too
soon. Thanks for the fix! Merging to master
> NPE thrown if user attempts to view Status History too early
>
>
> Key: NIFI-5609
> URL: https://issues.apache.org/jira/browse/NIFI-5609
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.8.0
>
>
> I started NiFi and then quickly went to the UI and right-clicked on a
> processor and tried to view Status History. I got back an error. The
> user-logs show the following (partial) stack trace:
> {code:java}
> 2018-09-19 13:20:19,928 ERROR [NiFi Web Server-24]
> o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred:
> java.lang.NullPointerException. Returning Internal Server Error response.
> java.lang.NullPointerException: null
> at
> org.apache.nifi.controller.status.history.StatusHistoryUtil.createStatusHistoryDTO(StatusHistoryUtil.java:39)
> at
> org.apache.nifi.controller.FlowController.getProcessorStatusHistory(FlowController.java:5066)
> at
> org.apache.nifi.controller.FlowController.getProcessorStatusHistory(FlowController.java:5062)
> at
> org.apache.nifi.web.controller.ControllerFacade.getProcessorStatusHistory(ControllerFacade.java:299)
> at
> org.apache.nifi.web.controller.ControllerFacade$$FastClassBySpringCGLIB$$5a42ba54.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
> at
> org.apache.nifi.web.controller.ControllerFacade$$EnhancerBySpringCGLIB$$8aeed060.getProcessorStatusHistory()
> at
> org.apache.nifi.web.StandardNiFiServiceFacade.getProcessorStatusHistory(StandardNiFiServiceFacade.java:2950)
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #3012: NIFI-5609: Fixed NullPointer when attempting to vie...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/3012 ---
[GitHub] nifi issue #3012: NIFI-5609: Fixed NullPointer when attempting to view statu...
Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/3012 +1 LGTM, ran build with unit tests and contrib-check, tried on a live NiFi instance, verified no NPE is thrown when PG status history is requested too soon. Thanks for the fix! Merging to master ---
[jira] [Assigned] (NIFI-5609) NPE thrown if user attempts to view Status History too early
[
https://issues.apache.org/jira/browse/NIFI-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess reassigned NIFI-5609:
--
Assignee: Mark Payne (was: Matt Burgess)
> NPE thrown if user attempts to view Status History too early
>
>
> Key: NIFI-5609
> URL: https://issues.apache.org/jira/browse/NIFI-5609
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.8.0
>
>
> I started NiFi and then quickly went to the UI and right-clicked on a
> processor and tried to view Status History. I got back an error. The
> user-logs show the following (partial) stack trace:
> {code:java}
> 2018-09-19 13:20:19,928 ERROR [NiFi Web Server-24]
> o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred:
> java.lang.NullPointerException. Returning Internal Server Error response.
> java.lang.NullPointerException: null
> at
> org.apache.nifi.controller.status.history.StatusHistoryUtil.createStatusHistoryDTO(StatusHistoryUtil.java:39)
> at
> org.apache.nifi.controller.FlowController.getProcessorStatusHistory(FlowController.java:5066)
> at
> org.apache.nifi.controller.FlowController.getProcessorStatusHistory(FlowController.java:5062)
> at
> org.apache.nifi.web.controller.ControllerFacade.getProcessorStatusHistory(ControllerFacade.java:299)
> at
> org.apache.nifi.web.controller.ControllerFacade$$FastClassBySpringCGLIB$$5a42ba54.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
> at
> org.apache.nifi.web.controller.ControllerFacade$$EnhancerBySpringCGLIB$$8aeed060.getProcessorStatusHistory()
> at
> org.apache.nifi.web.StandardNiFiServiceFacade.getProcessorStatusHistory(StandardNiFiServiceFacade.java:2950)
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5609) NPE thrown if user attempts to view Status History too early
[
https://issues.apache.org/jira/browse/NIFI-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621195#comment-16621195
]
ASF GitHub Bot commented on NIFI-5609:
--
Github user mattyb149 commented on the issue:
https://github.com/apache/nifi/pull/3012
Reviewing...
> NPE thrown if user attempts to view Status History too early
>
>
> Key: NIFI-5609
> URL: https://issues.apache.org/jira/browse/NIFI-5609
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Priority: Major
> Fix For: 1.8.0
>
>
> I started NiFi and then quickly went to the UI and right-clicked on a
> processor and tried to view Status History. I got back an error. The
> user-logs show the following (partial) stack trace:
> {code:java}
> 2018-09-19 13:20:19,928 ERROR [NiFi Web Server-24]
> o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred:
> java.lang.NullPointerException. Returning Internal Server Error response.
> java.lang.NullPointerException: null
> at
> org.apache.nifi.controller.status.history.StatusHistoryUtil.createStatusHistoryDTO(StatusHistoryUtil.java:39)
> at
> org.apache.nifi.controller.FlowController.getProcessorStatusHistory(FlowController.java:5066)
> at
> org.apache.nifi.controller.FlowController.getProcessorStatusHistory(FlowController.java:5062)
> at
> org.apache.nifi.web.controller.ControllerFacade.getProcessorStatusHistory(ControllerFacade.java:299)
> at
> org.apache.nifi.web.controller.ControllerFacade$$FastClassBySpringCGLIB$$5a42ba54.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
> at
> org.apache.nifi.web.controller.ControllerFacade$$EnhancerBySpringCGLIB$$8aeed060.getProcessorStatusHistory()
> at
> org.apache.nifi.web.StandardNiFiServiceFacade.getProcessorStatusHistory(StandardNiFiServiceFacade.java:2950)
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (NIFI-5609) NPE thrown if user attempts to view Status History too early
[
https://issues.apache.org/jira/browse/NIFI-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess reassigned NIFI-5609:
--
Assignee: Matt Burgess
> NPE thrown if user attempts to view Status History too early
>
>
> Key: NIFI-5609
> URL: https://issues.apache.org/jira/browse/NIFI-5609
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Assignee: Matt Burgess
>Priority: Major
> Fix For: 1.8.0
>
>
> I started NiFi and then quickly went to the UI and right-clicked on a
> processor and tried to view Status History. I got back an error. The
> user-logs show the following (partial) stack trace:
> {code:java}
> 2018-09-19 13:20:19,928 ERROR [NiFi Web Server-24]
> o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred:
> java.lang.NullPointerException. Returning Internal Server Error response.
> java.lang.NullPointerException: null
> at
> org.apache.nifi.controller.status.history.StatusHistoryUtil.createStatusHistoryDTO(StatusHistoryUtil.java:39)
> at
> org.apache.nifi.controller.FlowController.getProcessorStatusHistory(FlowController.java:5066)
> at
> org.apache.nifi.controller.FlowController.getProcessorStatusHistory(FlowController.java:5062)
> at
> org.apache.nifi.web.controller.ControllerFacade.getProcessorStatusHistory(ControllerFacade.java:299)
> at
> org.apache.nifi.web.controller.ControllerFacade$$FastClassBySpringCGLIB$$5a42ba54.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
> at
> org.apache.nifi.web.controller.ControllerFacade$$EnhancerBySpringCGLIB$$8aeed060.getProcessorStatusHistory()
> at
> org.apache.nifi.web.StandardNiFiServiceFacade.getProcessorStatusHistory(StandardNiFiServiceFacade.java:2950)
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi issue #3012: NIFI-5609: Fixed NullPointer when attempting to view statu...
Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/3012 Reviewing... ---
[jira] [Comment Edited] (NIFI-5612) org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
[
https://issues.apache.org/jira/browse/NIFI-5612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621183#comment-16621183
]
Colin Dean edited comment on NIFI-5612 at 9/19/18 8:46 PM:
---
Looks like I have to go town the route of getting the
{{UnresolvedUnionException}} out of the {{DataFileWriter.AppendWriteException}}
because none of the other information is in scope.
I've got this:
{code:java}
try {
dataFileWriter.append(rec);
nrOfRows += 1;
} catch (DataFileWriter.AppendWriteException awe) {
Throwable rootCause = ExceptionUtils.getRootCause(awe);
if(rootCause instanceof UnresolvedUnionException) {
UnresolvedUnionException uue =
(UnresolvedUnionException) rootCause;
throw new RuntimeException(
"Unable to resolve union for value " +
uue.getUnresolvedDatum() +
" with type " +
uue.getUnresolvedDatum().getClass().getCanonicalName()
, awe);
} else {
throw awe;
}
}
{code}
but I've got some time before a maintenance window opens to try it out so I'm
going to look through the tests and see what I can add that's more specific
examples of what I'm seeing in the schemas of the affected tables.
was (Author: colindean):
Looks like I have to go town the route of getting the
{{UnresolvedUnionException}} out of the {{DataFileWriter.AppendWriteException}}
because none of the other information is in scope.
I've got this:
{code:java}
{code}
but I've got some time before a maintenance window opens to try it out so I'm
going to look through the tests and see what I can add that's more specific
examples of what I'm seeing in the schemas of the affected tables.
> org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
>
>
> Key: NIFI-5612
> URL: https://issues.apache.org/jira/browse/NIFI-5612
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.5.0, 1.6.0, 1.7.1
> Environment: Microsoft Windows, MySQL Enterprise 5.0.80
>Reporter: Colin Dean
>Priority: Major
> Labels: ExecuteSQL, avro, nifi
>
> I'm seeing this when I execute {{SELECT * FROM }} on a few tables
> but not on dozens of others in the same database.
> {code}
> 2018-09-13 01:11:31,434 WARN [Timer-Driven Process Thread-8]
> o.a.n.controller.tasks.ConnectableTask Administratively Yielding
> ExecuteSQL[id=cf5c0996-eddf-3e05-25a3-c407c5edf990] due to uncaught
> Exception: org.apache.avro.file.DataFileWriter$AppendWriteException:
> org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
> org.apache.avro.file.DataFileWriter$AppendWriteException:
> org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
> at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:308)
> at
> org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:462)
> at
> org.apache.nifi.processors.standard.ExecuteSQL.lambda$onTrigger$1(ExecuteSQL.java:252)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2625)
> at
> org.apache.nifi.processors.standard.ExecuteSQL.onTrigger(ExecuteSQL.java:242)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1165)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:203)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.avro.UnresolvedUnionException: Not in union
> ["null","int"]: 0
> at
> org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:709)
> at
>
[jira] [Commented] (NIFI-5612) org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
[
https://issues.apache.org/jira/browse/NIFI-5612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621183#comment-16621183
]
Colin Dean commented on NIFI-5612:
--
Looks like I have to go town the route of getting the
{{UnresolvedUnionException}} out of the {{DataFileWriter.AppendWriteException}}
because none of the other information is in scope.
I've got this:
{code:java}
{code}
but I've got some time before a maintenance window opens to try it out so I'm
going to look through the tests and see what I can add that's more specific
examples of what I'm seeing in the schemas of the affected tables.
> org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
>
>
> Key: NIFI-5612
> URL: https://issues.apache.org/jira/browse/NIFI-5612
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.5.0, 1.6.0, 1.7.1
> Environment: Microsoft Windows, MySQL Enterprise 5.0.80
>Reporter: Colin Dean
>Priority: Major
> Labels: ExecuteSQL, avro, nifi
>
> I'm seeing this when I execute {{SELECT * FROM }} on a few tables
> but not on dozens of others in the same database.
> {code}
> 2018-09-13 01:11:31,434 WARN [Timer-Driven Process Thread-8]
> o.a.n.controller.tasks.ConnectableTask Administratively Yielding
> ExecuteSQL[id=cf5c0996-eddf-3e05-25a3-c407c5edf990] due to uncaught
> Exception: org.apache.avro.file.DataFileWriter$AppendWriteException:
> org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
> org.apache.avro.file.DataFileWriter$AppendWriteException:
> org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
> at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:308)
> at
> org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:462)
> at
> org.apache.nifi.processors.standard.ExecuteSQL.lambda$onTrigger$1(ExecuteSQL.java:252)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2625)
> at
> org.apache.nifi.processors.standard.ExecuteSQL.onTrigger(ExecuteSQL.java:242)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1165)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:203)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.avro.UnresolvedUnionException: Not in union
> ["null","int"]: 0
> at
> org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:709)
> at
> org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:192)
> at
> org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:110)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
> at
> org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:153)
> at
> org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:143)
> at
> org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:105)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:60)
> at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:302)
> ... 15 common frames omitted
> {code}
> I don't know if I can share the database schema – still working with my team
> on that – but looking at it, I think it has something to do with the
> signedness of int(1) or tinyint(1) because those two are the only numerical
> types common to all of the table.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (NIFI-5612) org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
[
https://issues.apache.org/jira/browse/NIFI-5612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621183#comment-16621183
]
Colin Dean edited comment on NIFI-5612 at 9/19/18 8:46 PM:
---
Looks like I have to go town the route of getting the
{{UnresolvedUnionException}} out of the {{DataFileWriter.AppendWriteException}}
because none of the other information is in scope.
I've got this:
{code:java}
try {
dataFileWriter.append(rec);
nrOfRows += 1;
} catch (DataFileWriter.AppendWriteException awe) {
Throwable rootCause = ExceptionUtils.getRootCause(awe);
if(rootCause instanceof UnresolvedUnionException) {
UnresolvedUnionException uue = (UnresolvedUnionException) rootCause;
throw new RuntimeException(
"Unable to resolve union for value " + uue.getUnresolvedDatum()
+
" with type " +
uue.getUnresolvedDatum().getClass().getCanonicalName()
, awe);
} else {
throw awe;
}
}
{code}
but I've got some time before a maintenance window opens to try it out so I'm
going to look through the tests and see what I can add that's more specific
examples of what I'm seeing in the schemas of the affected tables.
was (Author: colindean):
Looks like I have to go town the route of getting the
{{UnresolvedUnionException}} out of the {{DataFileWriter.AppendWriteException}}
because none of the other information is in scope.
I've got this:
{code:java}
try {
dataFileWriter.append(rec);
nrOfRows += 1;
} catch (DataFileWriter.AppendWriteException awe) {
Throwable rootCause = ExceptionUtils.getRootCause(awe);
if(rootCause instanceof UnresolvedUnionException) {
UnresolvedUnionException uue =
(UnresolvedUnionException) rootCause;
throw new RuntimeException(
"Unable to resolve union for value " +
uue.getUnresolvedDatum() +
" with type " +
uue.getUnresolvedDatum().getClass().getCanonicalName()
, awe);
} else {
throw awe;
}
}
{code}
but I've got some time before a maintenance window opens to try it out so I'm
going to look through the tests and see what I can add that's more specific
examples of what I'm seeing in the schemas of the affected tables.
> org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
>
>
> Key: NIFI-5612
> URL: https://issues.apache.org/jira/browse/NIFI-5612
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.5.0, 1.6.0, 1.7.1
> Environment: Microsoft Windows, MySQL Enterprise 5.0.80
>Reporter: Colin Dean
>Priority: Major
> Labels: ExecuteSQL, avro, nifi
>
> I'm seeing this when I execute {{SELECT * FROM }} on a few tables
> but not on dozens of others in the same database.
> {code}
> 2018-09-13 01:11:31,434 WARN [Timer-Driven Process Thread-8]
> o.a.n.controller.tasks.ConnectableTask Administratively Yielding
> ExecuteSQL[id=cf5c0996-eddf-3e05-25a3-c407c5edf990] due to uncaught
> Exception: org.apache.avro.file.DataFileWriter$AppendWriteException:
> org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
> org.apache.avro.file.DataFileWriter$AppendWriteException:
> org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
> at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:308)
> at
> org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:462)
> at
> org.apache.nifi.processors.standard.ExecuteSQL.lambda$onTrigger$1(ExecuteSQL.java:252)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2625)
> at
> org.apache.nifi.processors.standard.ExecuteSQL.onTrigger(ExecuteSQL.java:242)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1165)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:203)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
>
[jira] [Commented] (NIFI-5613) Upgrade NiFi Enrichment bundle to geoip2 version 2.12.0
[ https://issues.apache.org/jira/browse/NIFI-5613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621169#comment-16621169 ] ASF GitHub Bot commented on NIFI-5613: -- GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/3013 NIFI-5613: Upgraded version of geoip2 library to version 2.12.0; fixe… …d unit tests Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-5613 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3013.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3013 commit 2435efce303766dd50c456e5da8465cabbde678e Author: Mark Payne Date: 2018-09-19T20:32:53Z NIFI-5613: Upgraded version of geoip2 library to version 2.12.0; fixed unit tests > Upgrade NiFi Enrichment bundle to geoip2 version 2.12.0 > --- > > Key: NIFI-5613 > URL: https://issues.apache.org/jira/browse/NIFI-5613 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Mark Payne >Priority: Major > Fix For: 1.8.0 > > > Currently, it has a dependency on version 2.1.0 of the library but the newest > version is 2.12.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #3013: NIFI-5613: Upgraded version of geoip2 library to ve...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/3013 NIFI-5613: Upgraded version of geoip2 library to version 2.12.0; fixe⦠â¦d unit tests Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-5613 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3013.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3013 commit 2435efce303766dd50c456e5da8465cabbde678e Author: Mark Payne Date: 2018-09-19T20:32:53Z NIFI-5613: Upgraded version of geoip2 library to version 2.12.0; fixed unit tests ---
[jira] [Updated] (NIFI-5613) Upgrade NiFi Enrichment bundle to geoip2 version 2.12.0
[ https://issues.apache.org/jira/browse/NIFI-5613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-5613: - Fix Version/s: 1.8.0 Status: Patch Available (was: Open) > Upgrade NiFi Enrichment bundle to geoip2 version 2.12.0 > --- > > Key: NIFI-5613 > URL: https://issues.apache.org/jira/browse/NIFI-5613 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Mark Payne >Priority: Major > Fix For: 1.8.0 > > > Currently, it has a dependency on version 2.1.0 of the library but the newest > version is 2.12.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFI-5613) Upgrade NiFi Enrichment bundle to geoip2 version 2.12.0
Mark Payne created NIFI-5613: Summary: Upgrade NiFi Enrichment bundle to geoip2 version 2.12.0 Key: NIFI-5613 URL: https://issues.apache.org/jira/browse/NIFI-5613 Project: Apache NiFi Issue Type: Bug Components: Extensions Reporter: Mark Payne Currently, it has a dependency on version 2.1.0 of the library but the newest version is 2.12.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5612) org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
[
https://issues.apache.org/jira/browse/NIFI-5612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621158#comment-16621158
]
Colin Dean commented on NIFI-5612:
--
Minimally, I'd like to see more information about the erroring field in the
exception thrown [by {{dataFileWriter.append(rec)}} in
{{JdbcCommon.convertToAvroStream}}|https://github.com/apache/nifi/blob/e959630c22c9a52ec717141f6cf9f018830a38bf/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/util/JdbcCommon.java#L462]
It'd be nice if
{{org.apache.avro.UnresolvedUnionException#UnresolvedUnionException}} emitted
type information in the exception message but having its datum and schema
accessible might be useful in implementing the above.
> org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
>
>
> Key: NIFI-5612
> URL: https://issues.apache.org/jira/browse/NIFI-5612
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.5.0, 1.6.0, 1.7.1
> Environment: Microsoft Windows, MySQL Enterprise 5.0.80
>Reporter: Colin Dean
>Priority: Major
> Labels: ExecuteSQL, avro, nifi
>
> I'm seeing this when I execute {{SELECT * FROM }} on a few tables
> but not on dozens of others in the same database.
> {code}
> 2018-09-13 01:11:31,434 WARN [Timer-Driven Process Thread-8]
> o.a.n.controller.tasks.ConnectableTask Administratively Yielding
> ExecuteSQL[id=cf5c0996-eddf-3e05-25a3-c407c5edf990] due to uncaught
> Exception: org.apache.avro.file.DataFileWriter$AppendWriteException:
> org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
> org.apache.avro.file.DataFileWriter$AppendWriteException:
> org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
> at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:308)
> at
> org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:462)
> at
> org.apache.nifi.processors.standard.ExecuteSQL.lambda$onTrigger$1(ExecuteSQL.java:252)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2625)
> at
> org.apache.nifi.processors.standard.ExecuteSQL.onTrigger(ExecuteSQL.java:242)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1165)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:203)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.avro.UnresolvedUnionException: Not in union
> ["null","int"]: 0
> at
> org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:709)
> at
> org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:192)
> at
> org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:110)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
> at
> org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:153)
> at
> org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:143)
> at
> org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:105)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:60)
> at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:302)
> ... 15 common frames omitted
> {code}
> I don't know if I can share the database schema – still working with my team
> on that – but looking at it, I think it has something to do with the
> signedness of int(1) or tinyint(1) because those two are the only numerical
> types common to all of the table.
--
This message was sent by Atlassian JIRA
[jira] [Updated] (NIFI-5609) NPE thrown if user attempts to view Status History too early
[
https://issues.apache.org/jira/browse/NIFI-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne updated NIFI-5609:
-
Fix Version/s: 1.8.0
Status: Patch Available (was: Open)
> NPE thrown if user attempts to view Status History too early
>
>
> Key: NIFI-5609
> URL: https://issues.apache.org/jira/browse/NIFI-5609
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Priority: Major
> Fix For: 1.8.0
>
>
> I started NiFi and then quickly went to the UI and right-clicked on a
> processor and tried to view Status History. I got back an error. The
> user-logs show the following (partial) stack trace:
> {code:java}
> 2018-09-19 13:20:19,928 ERROR [NiFi Web Server-24]
> o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred:
> java.lang.NullPointerException. Returning Internal Server Error response.
> java.lang.NullPointerException: null
> at
> org.apache.nifi.controller.status.history.StatusHistoryUtil.createStatusHistoryDTO(StatusHistoryUtil.java:39)
> at
> org.apache.nifi.controller.FlowController.getProcessorStatusHistory(FlowController.java:5066)
> at
> org.apache.nifi.controller.FlowController.getProcessorStatusHistory(FlowController.java:5062)
> at
> org.apache.nifi.web.controller.ControllerFacade.getProcessorStatusHistory(ControllerFacade.java:299)
> at
> org.apache.nifi.web.controller.ControllerFacade$$FastClassBySpringCGLIB$$5a42ba54.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
> at
> org.apache.nifi.web.controller.ControllerFacade$$EnhancerBySpringCGLIB$$8aeed060.getProcessorStatusHistory()
> at
> org.apache.nifi.web.StandardNiFiServiceFacade.getProcessorStatusHistory(StandardNiFiServiceFacade.java:2950)
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5609) NPE thrown if user attempts to view Status History too early
[
https://issues.apache.org/jira/browse/NIFI-5609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621153#comment-16621153
]
ASF GitHub Bot commented on NIFI-5609:
--
GitHub user markap14 opened a pull request:
https://github.com/apache/nifi/pull/3012
NIFI-5609: Fixed NullPointer when attempting to view status history f…
…or a component that has not yet run
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with NIFI- where is the JIRA number
you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically master)?
- [ ] Is your initial contribution a single, squashed commit?
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the LICENSE file, including the main
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in
which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/markap14/nifi NIFI-5609
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/3012.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3012
commit dc0cacf8c5476d53b531eb5323ff645fcd20d7c8
Author: Mark Payne
Date: 2018-09-19T20:17:29Z
NIFI-5609: Fixed NullPointer when attempting to view status history for a
component that has not yet run
> NPE thrown if user attempts to view Status History too early
>
>
> Key: NIFI-5609
> URL: https://issues.apache.org/jira/browse/NIFI-5609
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Mark Payne
>Priority: Major
>
> I started NiFi and then quickly went to the UI and right-clicked on a
> processor and tried to view Status History. I got back an error. The
> user-logs show the following (partial) stack trace:
> {code:java}
> 2018-09-19 13:20:19,928 ERROR [NiFi Web Server-24]
> o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred:
> java.lang.NullPointerException. Returning Internal Server Error response.
> java.lang.NullPointerException: null
> at
> org.apache.nifi.controller.status.history.StatusHistoryUtil.createStatusHistoryDTO(StatusHistoryUtil.java:39)
> at
> org.apache.nifi.controller.FlowController.getProcessorStatusHistory(FlowController.java:5066)
> at
> org.apache.nifi.controller.FlowController.getProcessorStatusHistory(FlowController.java:5062)
> at
> org.apache.nifi.web.controller.ControllerFacade.getProcessorStatusHistory(ControllerFacade.java:299)
> at
> org.apache.nifi.web.controller.ControllerFacade$$FastClassBySpringCGLIB$$5a42ba54.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
> at
>
[GitHub] nifi pull request #3012: NIFI-5609: Fixed NullPointer when attempting to vie...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/3012 NIFI-5609: Fixed NullPointer when attempting to view status history f⦠â¦or a component that has not yet run Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-5609 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3012.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3012 commit dc0cacf8c5476d53b531eb5323ff645fcd20d7c8 Author: Mark Payne Date: 2018-09-19T20:17:29Z NIFI-5609: Fixed NullPointer when attempting to view status history for a component that has not yet run ---
[jira] [Commented] (NIFI-5612) org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
[
https://issues.apache.org/jira/browse/NIFI-5612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621142#comment-16621142
]
Colin Dean commented on NIFI-5612:
--
Toggling Avro logical types had no effect on this.
> org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
>
>
> Key: NIFI-5612
> URL: https://issues.apache.org/jira/browse/NIFI-5612
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.5.0, 1.6.0, 1.7.1
> Environment: Microsoft Windows, MySQL Enterprise 5.0.80
>Reporter: Colin Dean
>Priority: Major
> Labels: ExecuteSQL, avro, nifi
>
> I'm seeing this when I execute {{SELECT * FROM }} on a few tables
> but not on dozens of others in the same database.
> {code}
> 2018-09-13 01:11:31,434 WARN [Timer-Driven Process Thread-8]
> o.a.n.controller.tasks.ConnectableTask Administratively Yielding
> ExecuteSQL[id=cf5c0996-eddf-3e05-25a3-c407c5edf990] due to uncaught
> Exception: org.apache.avro.file.DataFileWriter$AppendWriteException:
> org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
> org.apache.avro.file.DataFileWriter$AppendWriteException:
> org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
> at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:308)
> at
> org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:462)
> at
> org.apache.nifi.processors.standard.ExecuteSQL.lambda$onTrigger$1(ExecuteSQL.java:252)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2625)
> at
> org.apache.nifi.processors.standard.ExecuteSQL.onTrigger(ExecuteSQL.java:242)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1165)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:203)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.avro.UnresolvedUnionException: Not in union
> ["null","int"]: 0
> at
> org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:709)
> at
> org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:192)
> at
> org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:110)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
> at
> org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:153)
> at
> org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:143)
> at
> org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:105)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:60)
> at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:302)
> ... 15 common frames omitted
> {code}
> I don't know if I can share the database schema – still working with my team
> on that – but looking at it, I think it has something to do with the
> signedness of int(1) or tinyint(1) because those two are the only numerical
> types common to all of the table.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (NIFI-5612) org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
Colin Dean created NIFI-5612:
Summary: org.apache.avro.UnresolvedUnionException: Not in union
["null","int"]: 0
Key: NIFI-5612
URL: https://issues.apache.org/jira/browse/NIFI-5612
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Affects Versions: 1.7.1, 1.6.0, 1.5.0
Environment: Microsoft Windows, MySQL Enterprise 5.0.80
Reporter: Colin Dean
I'm seeing this when I execute {{SELECT * FROM }} on a few tables
but not on dozens of others in the same database.
{code}
2018-09-13 01:11:31,434 WARN [Timer-Driven Process Thread-8]
o.a.n.controller.tasks.ConnectableTask Administratively Yielding
ExecuteSQL[id=cf5c0996-eddf-3e05-25a3-c407c5edf990] due to uncaught Exception:
org.apache.avro.file.DataFileWriter$AppendWriteException:
org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
org.apache.avro.file.DataFileWriter$AppendWriteException:
org.apache.avro.UnresolvedUnionException: Not in union ["null","int"]: 0
at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:308)
at
org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:462)
at
org.apache.nifi.processors.standard.ExecuteSQL.lambda$onTrigger$1(ExecuteSQL.java:252)
at
org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2625)
at
org.apache.nifi.processors.standard.ExecuteSQL.onTrigger(ExecuteSQL.java:242)
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1165)
at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:203)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.avro.UnresolvedUnionException: Not in union
["null","int"]: 0
at
org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:709)
at
org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:192)
at
org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:110)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
at
org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:153)
at
org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:143)
at
org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:105)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:60)
at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:302)
... 15 common frames omitted
{code}
I don't know if I can share the database schema – still working with my team on
that – but looking at it, I think it has something to do with the signedness of
int(1) or tinyint(1) because those two are the only numerical types common to
all of the table.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2990: NIFI-375: Added operation policy
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2990 ---
[jira] [Commented] (NIFI-375) New user role: Operator who can start and stop components
[ https://issues.apache.org/jira/browse/NIFI-375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621095#comment-16621095 ] ASF GitHub Bot commented on NIFI-375: - Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2990 > New user role: Operator who can start and stop components > - > > Key: NIFI-375 > URL: https://issues.apache.org/jira/browse/NIFI-375 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Reporter: Daniel Ueberfluss >Assignee: Koji Kawamura >Priority: Major > Fix For: 1.8.0 > > > Would like to have a user role that allows a user to stop/start processors > but perform no other changes to the dataflow. > This would allow users to address simple problems without providing full > access to modifying a data flow. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-375) New user role: Operator who can start and stop components
[ https://issues.apache.org/jira/browse/NIFI-375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-375: - Resolution: Fixed Fix Version/s: 1.8.0 Status: Resolved (was: Patch Available) > New user role: Operator who can start and stop components > - > > Key: NIFI-375 > URL: https://issues.apache.org/jira/browse/NIFI-375 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Reporter: Daniel Ueberfluss >Assignee: Koji Kawamura >Priority: Major > Fix For: 1.8.0 > > > Would like to have a user role that allows a user to stop/start processors > but perform no other changes to the dataflow. > This would allow users to address simple problems without providing full > access to modifying a data flow. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-375) New user role: Operator who can start and stop components
[ https://issues.apache.org/jira/browse/NIFI-375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621094#comment-16621094 ] ASF subversion and git services commented on NIFI-375: -- Commit f570cb980ddbb8292044ba3ff30304ed157dc839 in nifi's branch refs/heads/master from [~ijokarumawak] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=f570cb9 ] NIFI-375: Added operation policy The operation policy allows that a user to operate components even if they does not have direct READ/WRITE permission of the component. Following operations are controlled by the new operate policy: - Start/stop/enable/disable Processors, ControllerServices, ReportingTasks, Input/OuputPorts - Enable/disable transmission of RemoteInput/OutputPorts and RemoteProcessGroups - Terminate Processor threads Refactored what API exposes The previous commit let API exposes few fields in DTO. But we should avoid returning partial DTO as it complicates authorization logic. Instead, this commit adds StatusDTO for ReportingTaskEntity and ControllerServiceEntity, so that it can be returned regardless of having READ permission. Component DTO can only be returned with a READ permission. Refactor RPG same as ControllerService. WIP incorporating review comments. Incorporated review comments - Cleaned up merger classes - Recreate DTO instance at each function during two phase commmit Restrict enabling ControllerService without read permission Revert the last commit. Fix review comments. - Renamed confusing static method names and its parameters - Removed unnecessary permission checks from UI condition Fixed delete action display condition. Fixed NPE at Summary. Apply operation policy to activateControllerServices. Removed OperationPermissible from ComponentEntity. This closes #2990 > New user role: Operator who can start and stop components > - > > Key: NIFI-375 > URL: https://issues.apache.org/jira/browse/NIFI-375 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Reporter: Daniel Ueberfluss >Assignee: Koji Kawamura >Priority: Major > Fix For: 1.8.0 > > > Would like to have a user role that allows a user to stop/start processors > but perform no other changes to the dataflow. > This would allow users to address simple problems without providing full > access to modifying a data flow. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-375) New user role: Operator who can start and stop components
[ https://issues.apache.org/jira/browse/NIFI-375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621091#comment-16621091 ] ASF GitHub Bot commented on NIFI-375: - Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/2990 @ijokarumawak Thanks this looks good. I'm going to squash and merged to master. I've created two new JIRAs that are related to this effort but separate from the initial concern. Thanks again for working through all the review comments! https://issues.apache.org/jira/browse/NIFI-5610 https://issues.apache.org/jira/browse/NIFI-5611 > New user role: Operator who can start and stop components > - > > Key: NIFI-375 > URL: https://issues.apache.org/jira/browse/NIFI-375 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Reporter: Daniel Ueberfluss >Assignee: Koji Kawamura >Priority: Major > > Would like to have a user role that allows a user to stop/start processors > but perform no other changes to the dataflow. > This would allow users to address simple problems without providing full > access to modifying a data flow. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2990: NIFI-375: Added operation policy
Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/2990 @ijokarumawak Thanks this looks good. I'm going to squash and merged to master. I've created two new JIRAs that are related to this effort but separate from the initial concern. Thanks again for working through all the review comments! https://issues.apache.org/jira/browse/NIFI-5610 https://issues.apache.org/jira/browse/NIFI-5611 ---
[jira] [Created] (NIFI-5611) UI - Controller Service References
Matt Gilman created NIFI-5611: - Summary: UI - Controller Service References Key: NIFI-5611 URL: https://issues.apache.org/jira/browse/NIFI-5611 Project: Apache NiFi Issue Type: Improvement Components: Core UI Reporter: Matt Gilman Currently, if the user does not have sufficient write permissions to a component referencing a controller service we do not allow them to enable the service and all referencing components. This behavior is ok, but there is no indication which referencing component has lacking permissions. We need this to be more clear to the user. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFI-5610) UI - Controller Service Enable/Disable Dialog
Matt Gilman created NIFI-5610: - Summary: UI - Controller Service Enable/Disable Dialog Key: NIFI-5610 URL: https://issues.apache.org/jira/browse/NIFI-5610 Project: Apache NiFi Issue Type: Improvement Components: Core UI Reporter: Matt Gilman Currently, the Controller Service enable/disable dialog allows the user to make a change that they may not have the permissions to undo. When enabling a Controller Service, we allow the user to specify if they want to enable just the service or if they also want to activate (enable or schedule) all referencing components. When disabling a Controller Service, we require the user to deactivate (disable or unschedule) all referencing components. This is because there is a requirement for referenced services to be available. However, this means that a user with permissions to the start only the service may not be able to stop it if they lack permissions to one of the referencing components. We should either not allow a user without permissions to all referencing components to even enable the service only. Or we should identify the case where no referencing components needs to be modified in order to disable the service in question. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (NIFI-5254) Upgrade to Groovy 2.5.0
[ https://issues.apache.org/jira/browse/NIFI-5254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Thomsen reassigned NIFI-5254: -- Assignee: Mike Thomsen > Upgrade to Groovy 2.5.0 > --- > > Key: NIFI-5254 > URL: https://issues.apache.org/jira/browse/NIFI-5254 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Mike Thomsen >Assignee: Mike Thomsen >Priority: Major > > Groovy 2.5 has been released and support for it should be added. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-613) Remove rooturi local option
[ https://issues.apache.org/jira/browse/MINIFICPP-613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621057#comment-16621057 ] ASF GitHub Bot commented on MINIFICPP-613: -- GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/400 MINIFICPP-613: Remove undocumented option that allows arbitrary paths… … that aren't needed. Defaulting to root Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFICPP-613 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/400.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #400 commit 322d9f05b8d8b4b7526fbe473fb008048c10a4fd Author: Marc Parisi Date: 2018-09-19T19:06:18Z MINIFICPP-613: Remove undocumented option that allows arbitrary paths that aren't needed. Defaulting to root > Remove rooturi local option > --- > > Key: MINIFICPP-613 > URL: https://issues.apache.org/jira/browse/MINIFICPP-613 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > > Remove an unused option for the listener root uri -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #400: MINIFICPP-613: Remove undocumented option...
GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/400 MINIFICPP-613: Remove undocumented option that allows arbitrary paths⦠⦠that aren't needed. Defaulting to root Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFICPP-613 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/400.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #400 commit 322d9f05b8d8b4b7526fbe473fb008048c10a4fd Author: Marc Parisi Date: 2018-09-19T19:06:18Z MINIFICPP-613: Remove undocumented option that allows arbitrary paths that aren't needed. Defaulting to root ---
[jira] [Created] (MINIFICPP-613) Remove rooturi local option
Mr TheSegfault created MINIFICPP-613: Summary: Remove rooturi local option Key: MINIFICPP-613 URL: https://issues.apache.org/jira/browse/MINIFICPP-613 Project: NiFi MiNiFi C++ Issue Type: Improvement Reporter: Mr TheSegfault Assignee: Mr TheSegfault Remove an unused option for the listener root uri -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5327) NetFlow Processors
[ https://issues.apache.org/jira/browse/NIFI-5327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620934#comment-16620934 ] ASF GitHub Bot commented on NIFI-5327: -- Github user PrashanthVenkatesan commented on the issue: https://github.com/apache/nifi/pull/2820 @ottobackwards @MikeThomsen Thanks for all your support and guidance in my first OS contribution. Soon, I will try to contribute for NetFlowv9 processor. > NetFlow Processors > -- > > Key: NIFI-5327 > URL: https://issues.apache.org/jira/browse/NIFI-5327 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 1.6.0 >Reporter: Prashanth Venkatesan >Assignee: Prashanth Venkatesan >Priority: Major > Fix For: 1.8.0 > > > As network traffic data scopes for the big data use case, would like NiFi to > have processors to support parsing of those protocols. > Netflow is a protocol introduced by Cisco that provides the ability to > collect IP network traffic as it enters or exits an interface and is > described in detail in here: > [https://www.cisco.com/c/en/us/td/docs/net_mgmt/netflow_collection_engine/3-6/user/guide/format.html] > > Currently, I have created the following processor: > *ParseNetflowv5*: Parses the ingress netflowv5 bytes and ingest as either > NiFi flowfile attributes or as a JSON content. This also sends > one-time-template. > > Further ahead, we can add many processor specific to network protocols in > this nar bundle. > I will create a pull request. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2820: NIFI-5327 Adding Netflowv5 protocol parser
Github user PrashanthVenkatesan commented on the issue: https://github.com/apache/nifi/pull/2820 @ottobackwards @MikeThomsen Thanks for all your support and guidance in my first OS contribution. Soon, I will try to contribute for NetFlowv9 processor. ---
[jira] [Commented] (NIFI-4806) Upgrade to Tika/Tika-Parsers 1.19 to resolve TIKA-2535 and CVEs reported against all releases prior to 1.19
[ https://issues.apache.org/jira/browse/NIFI-4806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620927#comment-16620927 ] Joseph Witt commented on NIFI-4806: --- ./jetty/nifi-web-content-viewer-1.8.0-SNAPSHOT.war/webapp/WEB-INF/lib/tika-core-1.17.jar ./nar/extensions/nifi-media-nar-1.8.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/tika-core-1.17.jar ./nar/extensions/nifi-media-nar-1.8.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/vorbis-java-tika-0.8.jar ./nar/extensions/nifi-media-nar-1.8.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/tika-parsers-1.17.jar ./nar/extensions/nifi-standard-nar-1.8.0-SNAPSHOT.nar-unpacked/NAR-INF/bundled-dependencies/tika-core-1.17.jar > Upgrade to Tika/Tika-Parsers 1.19 to resolve TIKA-2535 and CVEs reported > against all releases prior to 1.19 > --- > > Key: NIFI-4806 > URL: https://issues.apache.org/jira/browse/NIFI-4806 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.5.0 >Reporter: Joseph Witt >Assignee: Joseph Witt >Priority: Major > Fix For: 1.8.0 > > > The nifi-media-processors depend on Tika-Parsers 1.16 as of now. They need > to be upgraded to at least Tika-Parsers 1.18 to resolve the licensing problem > identified to the Tika team and which they resolved in > https://issues.apache.org/jira/browse/TIKA-2535 > > We aren't susceptible to the licensing problem at this time because when we > did the last update for Tika-Parsers this was flagged and excluded though we > could be exposing bugs for certain datayptes we'd do mime detection on > (maybe). I have a comment about this in our pom. > > This Jira is to upgrade, ensure no invalid libs are used, and clean up the > comments and move on. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-4806) Upgrade to Tika/Tika-Parsers 1.19 to resolve TIKA-2535 and CVEs reported against all releases prior to 1.19
[ https://issues.apache.org/jira/browse/NIFI-4806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-4806: -- Fix Version/s: 1.8.0 > Upgrade to Tika/Tika-Parsers 1.19 to resolve TIKA-2535 and CVEs reported > against all releases prior to 1.19 > --- > > Key: NIFI-4806 > URL: https://issues.apache.org/jira/browse/NIFI-4806 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.5.0 >Reporter: Joseph Witt >Assignee: Joseph Witt >Priority: Blocker > Fix For: 1.8.0 > > > The nifi-media-processors depend on Tika-Parsers 1.16 as of now. They need > to be upgraded to at least Tika-Parsers 1.18 to resolve the licensing problem > identified to the Tika team and which they resolved in > https://issues.apache.org/jira/browse/TIKA-2535 > > We aren't susceptible to the licensing problem at this time because when we > did the last update for Tika-Parsers this was flagged and excluded though we > could be exposing bugs for certain datayptes we'd do mime detection on > (maybe). I have a comment about this in our pom. > > This Jira is to upgrade, ensure no invalid libs are used, and clean up the > comments and move on. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-4806) Upgrade to Tika/Tika-Parsers 1.19 to resolve TIKA-2535 and CVEs reported against all releases prior to 1.19
[ https://issues.apache.org/jira/browse/NIFI-4806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-4806: -- Priority: Blocker (was: Major) > Upgrade to Tika/Tika-Parsers 1.19 to resolve TIKA-2535 and CVEs reported > against all releases prior to 1.19 > --- > > Key: NIFI-4806 > URL: https://issues.apache.org/jira/browse/NIFI-4806 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.5.0 >Reporter: Joseph Witt >Assignee: Joseph Witt >Priority: Blocker > Fix For: 1.8.0 > > > The nifi-media-processors depend on Tika-Parsers 1.16 as of now. They need > to be upgraded to at least Tika-Parsers 1.18 to resolve the licensing problem > identified to the Tika team and which they resolved in > https://issues.apache.org/jira/browse/TIKA-2535 > > We aren't susceptible to the licensing problem at this time because when we > did the last update for Tika-Parsers this was flagged and excluded though we > could be exposing bugs for certain datayptes we'd do mime detection on > (maybe). I have a comment about this in our pom. > > This Jira is to upgrade, ensure no invalid libs are used, and clean up the > comments and move on. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-4806) Upgrade to Tika/Tika-Parsers 1.19 to resolve TIKA-2535 and CVEs reported against all releases prior to 1.19
[ https://issues.apache.org/jira/browse/NIFI-4806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-4806: -- Summary: Upgrade to Tika/Tika-Parsers 1.19 to resolve TIKA-2535 and CVEs reported against all releases prior to 1.19 (was: Upgrade to Tika/Tika-Parsers 1.18 or newer to resolve TIKA-2535) > Upgrade to Tika/Tika-Parsers 1.19 to resolve TIKA-2535 and CVEs reported > against all releases prior to 1.19 > --- > > Key: NIFI-4806 > URL: https://issues.apache.org/jira/browse/NIFI-4806 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.5.0 >Reporter: Joseph Witt >Assignee: Joseph Witt >Priority: Major > Fix For: 1.8.0 > > > The nifi-media-processors depend on Tika-Parsers 1.16 as of now. They need > to be upgraded to at least Tika-Parsers 1.18 to resolve the licensing problem > identified to the Tika team and which they resolved in > https://issues.apache.org/jira/browse/TIKA-2535 > > We aren't susceptible to the licensing problem at this time because when we > did the last update for Tika-Parsers this was flagged and excluded though we > could be exposing bugs for certain datayptes we'd do mime detection on > (maybe). I have a comment about this in our pom. > > This Jira is to upgrade, ensure no invalid libs are used, and clean up the > comments and move on. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFI-5609) NPE thrown if user attempts to view Status History too early
Mark Payne created NIFI-5609:
Summary: NPE thrown if user attempts to view Status History too
early
Key: NIFI-5609
URL: https://issues.apache.org/jira/browse/NIFI-5609
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Reporter: Mark Payne
I started NiFi and then quickly went to the UI and right-clicked on a processor
and tried to view Status History. I got back an error. The user-logs show the
following (partial) stack trace:
{code:java}
2018-09-19 13:20:19,928 ERROR [NiFi Web Server-24]
o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred:
java.lang.NullPointerException. Returning Internal Server Error response.
java.lang.NullPointerException: null
at
org.apache.nifi.controller.status.history.StatusHistoryUtil.createStatusHistoryDTO(StatusHistoryUtil.java:39)
at
org.apache.nifi.controller.FlowController.getProcessorStatusHistory(FlowController.java:5066)
at
org.apache.nifi.controller.FlowController.getProcessorStatusHistory(FlowController.java:5062)
at
org.apache.nifi.web.controller.ControllerFacade.getProcessorStatusHistory(ControllerFacade.java:299)
at
org.apache.nifi.web.controller.ControllerFacade$$FastClassBySpringCGLIB$$5a42ba54.invoke()
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
at
org.apache.nifi.web.controller.ControllerFacade$$EnhancerBySpringCGLIB$$8aeed060.getProcessorStatusHistory()
at
org.apache.nifi.web.StandardNiFiServiceFacade.getProcessorStatusHistory(StandardNiFiServiceFacade.java:2950)
at
org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-5327) NetFlow Processors
[ https://issues.apache.org/jira/browse/NIFI-5327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Thomsen updated NIFI-5327: --- Resolution: Fixed Fix Version/s: 1.8.0 Status: Resolved (was: Patch Available) > NetFlow Processors > -- > > Key: NIFI-5327 > URL: https://issues.apache.org/jira/browse/NIFI-5327 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 1.6.0 >Reporter: Prashanth Venkatesan >Assignee: Prashanth Venkatesan >Priority: Major > Fix For: 1.8.0 > > > As network traffic data scopes for the big data use case, would like NiFi to > have processors to support parsing of those protocols. > Netflow is a protocol introduced by Cisco that provides the ability to > collect IP network traffic as it enters or exits an interface and is > described in detail in here: > [https://www.cisco.com/c/en/us/td/docs/net_mgmt/netflow_collection_engine/3-6/user/guide/format.html] > > Currently, I have created the following processor: > *ParseNetflowv5*: Parses the ingress netflowv5 bytes and ingest as either > NiFi flowfile attributes or as a JSON content. This also sends > one-time-template. > > Further ahead, we can add many processor specific to network protocols in > this nar bundle. > I will create a pull request. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5327) NetFlow Processors
[ https://issues.apache.org/jira/browse/NIFI-5327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620901#comment-16620901 ] ASF subversion and git services commented on NIFI-5327: --- Commit 9161b1787a57f158107653aa4cecc0f0b4f98499 in nifi's branch refs/heads/master from “PrashanthVenkatesan” [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=9161b17 ] NIFI-5327 Adding Netflowv5 protocol parser NIFI-5327 Formated Code to adhere NiFi checkstyle NIFI-5327 Incorporated code review comments NIFI-5327 Update additionalDetails docs NIFI-5327 Format IPV4 fields & rename nar NIFI-5327 Update dependency jar version NIFI-5327 Updated Code review comments NIFI-5327 Updated Code review comments NIFI-5327 Patch for code checkstyle violations NIFI-5327 commit for retriggering build This closes #2820 Signed-off-by: Mike Thomsen > NetFlow Processors > -- > > Key: NIFI-5327 > URL: https://issues.apache.org/jira/browse/NIFI-5327 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 1.6.0 >Reporter: Prashanth Venkatesan >Assignee: Prashanth Venkatesan >Priority: Major > > As network traffic data scopes for the big data use case, would like NiFi to > have processors to support parsing of those protocols. > Netflow is a protocol introduced by Cisco that provides the ability to > collect IP network traffic as it enters or exits an interface and is > described in detail in here: > [https://www.cisco.com/c/en/us/td/docs/net_mgmt/netflow_collection_engine/3-6/user/guide/format.html] > > Currently, I have created the following processor: > *ParseNetflowv5*: Parses the ingress netflowv5 bytes and ingest as either > NiFi flowfile attributes or as a JSON content. This also sends > one-time-template. > > Further ahead, we can add many processor specific to network protocols in > this nar bundle. > I will create a pull request. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5327) NetFlow Processors
[ https://issues.apache.org/jira/browse/NIFI-5327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620904#comment-16620904 ] ASF subversion and git services commented on NIFI-5327: --- Commit 9161b1787a57f158107653aa4cecc0f0b4f98499 in nifi's branch refs/heads/master from “PrashanthVenkatesan” [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=9161b17 ] NIFI-5327 Adding Netflowv5 protocol parser NIFI-5327 Formated Code to adhere NiFi checkstyle NIFI-5327 Incorporated code review comments NIFI-5327 Update additionalDetails docs NIFI-5327 Format IPV4 fields & rename nar NIFI-5327 Update dependency jar version NIFI-5327 Updated Code review comments NIFI-5327 Updated Code review comments NIFI-5327 Patch for code checkstyle violations NIFI-5327 commit for retriggering build This closes #2820 Signed-off-by: Mike Thomsen > NetFlow Processors > -- > > Key: NIFI-5327 > URL: https://issues.apache.org/jira/browse/NIFI-5327 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 1.6.0 >Reporter: Prashanth Venkatesan >Assignee: Prashanth Venkatesan >Priority: Major > > As network traffic data scopes for the big data use case, would like NiFi to > have processors to support parsing of those protocols. > Netflow is a protocol introduced by Cisco that provides the ability to > collect IP network traffic as it enters or exits an interface and is > described in detail in here: > [https://www.cisco.com/c/en/us/td/docs/net_mgmt/netflow_collection_engine/3-6/user/guide/format.html] > > Currently, I have created the following processor: > *ParseNetflowv5*: Parses the ingress netflowv5 bytes and ingest as either > NiFi flowfile attributes or as a JSON content. This also sends > one-time-template. > > Further ahead, we can add many processor specific to network protocols in > this nar bundle. > I will create a pull request. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5327) NetFlow Processors
[ https://issues.apache.org/jira/browse/NIFI-5327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620899#comment-16620899 ] ASF subversion and git services commented on NIFI-5327: --- Commit 9161b1787a57f158107653aa4cecc0f0b4f98499 in nifi's branch refs/heads/master from “PrashanthVenkatesan” [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=9161b17 ] NIFI-5327 Adding Netflowv5 protocol parser NIFI-5327 Formated Code to adhere NiFi checkstyle NIFI-5327 Incorporated code review comments NIFI-5327 Update additionalDetails docs NIFI-5327 Format IPV4 fields & rename nar NIFI-5327 Update dependency jar version NIFI-5327 Updated Code review comments NIFI-5327 Updated Code review comments NIFI-5327 Patch for code checkstyle violations NIFI-5327 commit for retriggering build This closes #2820 Signed-off-by: Mike Thomsen > NetFlow Processors > -- > > Key: NIFI-5327 > URL: https://issues.apache.org/jira/browse/NIFI-5327 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 1.6.0 >Reporter: Prashanth Venkatesan >Assignee: Prashanth Venkatesan >Priority: Major > > As network traffic data scopes for the big data use case, would like NiFi to > have processors to support parsing of those protocols. > Netflow is a protocol introduced by Cisco that provides the ability to > collect IP network traffic as it enters or exits an interface and is > described in detail in here: > [https://www.cisco.com/c/en/us/td/docs/net_mgmt/netflow_collection_engine/3-6/user/guide/format.html] > > Currently, I have created the following processor: > *ParseNetflowv5*: Parses the ingress netflowv5 bytes and ingest as either > NiFi flowfile attributes or as a JSON content. This also sends > one-time-template. > > Further ahead, we can add many processor specific to network protocols in > this nar bundle. > I will create a pull request. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5327) NetFlow Processors
[ https://issues.apache.org/jira/browse/NIFI-5327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620907#comment-16620907 ] ASF GitHub Bot commented on NIFI-5327: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2820 > NetFlow Processors > -- > > Key: NIFI-5327 > URL: https://issues.apache.org/jira/browse/NIFI-5327 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 1.6.0 >Reporter: Prashanth Venkatesan >Assignee: Prashanth Venkatesan >Priority: Major > > As network traffic data scopes for the big data use case, would like NiFi to > have processors to support parsing of those protocols. > Netflow is a protocol introduced by Cisco that provides the ability to > collect IP network traffic as it enters or exits an interface and is > described in detail in here: > [https://www.cisco.com/c/en/us/td/docs/net_mgmt/netflow_collection_engine/3-6/user/guide/format.html] > > Currently, I have created the following processor: > *ParseNetflowv5*: Parses the ingress netflowv5 bytes and ingest as either > NiFi flowfile attributes or as a JSON content. This also sends > one-time-template. > > Further ahead, we can add many processor specific to network protocols in > this nar bundle. > I will create a pull request. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5327) NetFlow Processors
[ https://issues.apache.org/jira/browse/NIFI-5327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620898#comment-16620898 ] ASF subversion and git services commented on NIFI-5327: --- Commit 9161b1787a57f158107653aa4cecc0f0b4f98499 in nifi's branch refs/heads/master from “PrashanthVenkatesan” [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=9161b17 ] NIFI-5327 Adding Netflowv5 protocol parser NIFI-5327 Formated Code to adhere NiFi checkstyle NIFI-5327 Incorporated code review comments NIFI-5327 Update additionalDetails docs NIFI-5327 Format IPV4 fields & rename nar NIFI-5327 Update dependency jar version NIFI-5327 Updated Code review comments NIFI-5327 Updated Code review comments NIFI-5327 Patch for code checkstyle violations NIFI-5327 commit for retriggering build This closes #2820 Signed-off-by: Mike Thomsen > NetFlow Processors > -- > > Key: NIFI-5327 > URL: https://issues.apache.org/jira/browse/NIFI-5327 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 1.6.0 >Reporter: Prashanth Venkatesan >Assignee: Prashanth Venkatesan >Priority: Major > > As network traffic data scopes for the big data use case, would like NiFi to > have processors to support parsing of those protocols. > Netflow is a protocol introduced by Cisco that provides the ability to > collect IP network traffic as it enters or exits an interface and is > described in detail in here: > [https://www.cisco.com/c/en/us/td/docs/net_mgmt/netflow_collection_engine/3-6/user/guide/format.html] > > Currently, I have created the following processor: > *ParseNetflowv5*: Parses the ingress netflowv5 bytes and ingest as either > NiFi flowfile attributes or as a JSON content. This also sends > one-time-template. > > Further ahead, we can add many processor specific to network protocols in > this nar bundle. > I will create a pull request. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5327) NetFlow Processors
[ https://issues.apache.org/jira/browse/NIFI-5327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620897#comment-16620897 ] ASF subversion and git services commented on NIFI-5327: --- Commit 9161b1787a57f158107653aa4cecc0f0b4f98499 in nifi's branch refs/heads/master from “PrashanthVenkatesan” [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=9161b17 ] NIFI-5327 Adding Netflowv5 protocol parser NIFI-5327 Formated Code to adhere NiFi checkstyle NIFI-5327 Incorporated code review comments NIFI-5327 Update additionalDetails docs NIFI-5327 Format IPV4 fields & rename nar NIFI-5327 Update dependency jar version NIFI-5327 Updated Code review comments NIFI-5327 Updated Code review comments NIFI-5327 Patch for code checkstyle violations NIFI-5327 commit for retriggering build This closes #2820 Signed-off-by: Mike Thomsen > NetFlow Processors > -- > > Key: NIFI-5327 > URL: https://issues.apache.org/jira/browse/NIFI-5327 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 1.6.0 >Reporter: Prashanth Venkatesan >Assignee: Prashanth Venkatesan >Priority: Major > > As network traffic data scopes for the big data use case, would like NiFi to > have processors to support parsing of those protocols. > Netflow is a protocol introduced by Cisco that provides the ability to > collect IP network traffic as it enters or exits an interface and is > described in detail in here: > [https://www.cisco.com/c/en/us/td/docs/net_mgmt/netflow_collection_engine/3-6/user/guide/format.html] > > Currently, I have created the following processor: > *ParseNetflowv5*: Parses the ingress netflowv5 bytes and ingest as either > NiFi flowfile attributes or as a JSON content. This also sends > one-time-template. > > Further ahead, we can add many processor specific to network protocols in > this nar bundle. > I will create a pull request. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5327) NetFlow Processors
[ https://issues.apache.org/jira/browse/NIFI-5327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620905#comment-16620905 ] ASF subversion and git services commented on NIFI-5327: --- Commit 9161b1787a57f158107653aa4cecc0f0b4f98499 in nifi's branch refs/heads/master from “PrashanthVenkatesan” [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=9161b17 ] NIFI-5327 Adding Netflowv5 protocol parser NIFI-5327 Formated Code to adhere NiFi checkstyle NIFI-5327 Incorporated code review comments NIFI-5327 Update additionalDetails docs NIFI-5327 Format IPV4 fields & rename nar NIFI-5327 Update dependency jar version NIFI-5327 Updated Code review comments NIFI-5327 Updated Code review comments NIFI-5327 Patch for code checkstyle violations NIFI-5327 commit for retriggering build This closes #2820 Signed-off-by: Mike Thomsen > NetFlow Processors > -- > > Key: NIFI-5327 > URL: https://issues.apache.org/jira/browse/NIFI-5327 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 1.6.0 >Reporter: Prashanth Venkatesan >Assignee: Prashanth Venkatesan >Priority: Major > > As network traffic data scopes for the big data use case, would like NiFi to > have processors to support parsing of those protocols. > Netflow is a protocol introduced by Cisco that provides the ability to > collect IP network traffic as it enters or exits an interface and is > described in detail in here: > [https://www.cisco.com/c/en/us/td/docs/net_mgmt/netflow_collection_engine/3-6/user/guide/format.html] > > Currently, I have created the following processor: > *ParseNetflowv5*: Parses the ingress netflowv5 bytes and ingest as either > NiFi flowfile attributes or as a JSON content. This also sends > one-time-template. > > Further ahead, we can add many processor specific to network protocols in > this nar bundle. > I will create a pull request. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5327) NetFlow Processors
[ https://issues.apache.org/jira/browse/NIFI-5327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620903#comment-16620903 ] ASF subversion and git services commented on NIFI-5327: --- Commit 9161b1787a57f158107653aa4cecc0f0b4f98499 in nifi's branch refs/heads/master from “PrashanthVenkatesan” [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=9161b17 ] NIFI-5327 Adding Netflowv5 protocol parser NIFI-5327 Formated Code to adhere NiFi checkstyle NIFI-5327 Incorporated code review comments NIFI-5327 Update additionalDetails docs NIFI-5327 Format IPV4 fields & rename nar NIFI-5327 Update dependency jar version NIFI-5327 Updated Code review comments NIFI-5327 Updated Code review comments NIFI-5327 Patch for code checkstyle violations NIFI-5327 commit for retriggering build This closes #2820 Signed-off-by: Mike Thomsen > NetFlow Processors > -- > > Key: NIFI-5327 > URL: https://issues.apache.org/jira/browse/NIFI-5327 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 1.6.0 >Reporter: Prashanth Venkatesan >Assignee: Prashanth Venkatesan >Priority: Major > > As network traffic data scopes for the big data use case, would like NiFi to > have processors to support parsing of those protocols. > Netflow is a protocol introduced by Cisco that provides the ability to > collect IP network traffic as it enters or exits an interface and is > described in detail in here: > [https://www.cisco.com/c/en/us/td/docs/net_mgmt/netflow_collection_engine/3-6/user/guide/format.html] > > Currently, I have created the following processor: > *ParseNetflowv5*: Parses the ingress netflowv5 bytes and ingest as either > NiFi flowfile attributes or as a JSON content. This also sends > one-time-template. > > Further ahead, we can add many processor specific to network protocols in > this nar bundle. > I will create a pull request. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5327) NetFlow Processors
[ https://issues.apache.org/jira/browse/NIFI-5327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620906#comment-16620906 ] ASF subversion and git services commented on NIFI-5327: --- Commit 9161b1787a57f158107653aa4cecc0f0b4f98499 in nifi's branch refs/heads/master from “PrashanthVenkatesan” [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=9161b17 ] NIFI-5327 Adding Netflowv5 protocol parser NIFI-5327 Formated Code to adhere NiFi checkstyle NIFI-5327 Incorporated code review comments NIFI-5327 Update additionalDetails docs NIFI-5327 Format IPV4 fields & rename nar NIFI-5327 Update dependency jar version NIFI-5327 Updated Code review comments NIFI-5327 Updated Code review comments NIFI-5327 Patch for code checkstyle violations NIFI-5327 commit for retriggering build This closes #2820 Signed-off-by: Mike Thomsen > NetFlow Processors > -- > > Key: NIFI-5327 > URL: https://issues.apache.org/jira/browse/NIFI-5327 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 1.6.0 >Reporter: Prashanth Venkatesan >Assignee: Prashanth Venkatesan >Priority: Major > > As network traffic data scopes for the big data use case, would like NiFi to > have processors to support parsing of those protocols. > Netflow is a protocol introduced by Cisco that provides the ability to > collect IP network traffic as it enters or exits an interface and is > described in detail in here: > [https://www.cisco.com/c/en/us/td/docs/net_mgmt/netflow_collection_engine/3-6/user/guide/format.html] > > Currently, I have created the following processor: > *ParseNetflowv5*: Parses the ingress netflowv5 bytes and ingest as either > NiFi flowfile attributes or as a JSON content. This also sends > one-time-template. > > Further ahead, we can add many processor specific to network protocols in > this nar bundle. > I will create a pull request. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5327) NetFlow Processors
[ https://issues.apache.org/jira/browse/NIFI-5327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620900#comment-16620900 ] ASF subversion and git services commented on NIFI-5327: --- Commit 9161b1787a57f158107653aa4cecc0f0b4f98499 in nifi's branch refs/heads/master from “PrashanthVenkatesan” [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=9161b17 ] NIFI-5327 Adding Netflowv5 protocol parser NIFI-5327 Formated Code to adhere NiFi checkstyle NIFI-5327 Incorporated code review comments NIFI-5327 Update additionalDetails docs NIFI-5327 Format IPV4 fields & rename nar NIFI-5327 Update dependency jar version NIFI-5327 Updated Code review comments NIFI-5327 Updated Code review comments NIFI-5327 Patch for code checkstyle violations NIFI-5327 commit for retriggering build This closes #2820 Signed-off-by: Mike Thomsen > NetFlow Processors > -- > > Key: NIFI-5327 > URL: https://issues.apache.org/jira/browse/NIFI-5327 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 1.6.0 >Reporter: Prashanth Venkatesan >Assignee: Prashanth Venkatesan >Priority: Major > > As network traffic data scopes for the big data use case, would like NiFi to > have processors to support parsing of those protocols. > Netflow is a protocol introduced by Cisco that provides the ability to > collect IP network traffic as it enters or exits an interface and is > described in detail in here: > [https://www.cisco.com/c/en/us/td/docs/net_mgmt/netflow_collection_engine/3-6/user/guide/format.html] > > Currently, I have created the following processor: > *ParseNetflowv5*: Parses the ingress netflowv5 bytes and ingest as either > NiFi flowfile attributes or as a JSON content. This also sends > one-time-template. > > Further ahead, we can add many processor specific to network protocols in > this nar bundle. > I will create a pull request. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5327) NetFlow Processors
[ https://issues.apache.org/jira/browse/NIFI-5327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620902#comment-16620902 ] ASF subversion and git services commented on NIFI-5327: --- Commit 9161b1787a57f158107653aa4cecc0f0b4f98499 in nifi's branch refs/heads/master from “PrashanthVenkatesan” [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=9161b17 ] NIFI-5327 Adding Netflowv5 protocol parser NIFI-5327 Formated Code to adhere NiFi checkstyle NIFI-5327 Incorporated code review comments NIFI-5327 Update additionalDetails docs NIFI-5327 Format IPV4 fields & rename nar NIFI-5327 Update dependency jar version NIFI-5327 Updated Code review comments NIFI-5327 Updated Code review comments NIFI-5327 Patch for code checkstyle violations NIFI-5327 commit for retriggering build This closes #2820 Signed-off-by: Mike Thomsen > NetFlow Processors > -- > > Key: NIFI-5327 > URL: https://issues.apache.org/jira/browse/NIFI-5327 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 1.6.0 >Reporter: Prashanth Venkatesan >Assignee: Prashanth Venkatesan >Priority: Major > > As network traffic data scopes for the big data use case, would like NiFi to > have processors to support parsing of those protocols. > Netflow is a protocol introduced by Cisco that provides the ability to > collect IP network traffic as it enters or exits an interface and is > described in detail in here: > [https://www.cisco.com/c/en/us/td/docs/net_mgmt/netflow_collection_engine/3-6/user/guide/format.html] > > Currently, I have created the following processor: > *ParseNetflowv5*: Parses the ingress netflowv5 bytes and ingest as either > NiFi flowfile attributes or as a JSON content. This also sends > one-time-template. > > Further ahead, we can add many processor specific to network protocols in > this nar bundle. > I will create a pull request. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2820: NIFI-5327 Adding Netflowv5 protocol parser
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2820 ---
[jira] [Updated] (NIFI-5598) Allow ConsumeJMS and PublishJMS processors to use JNDI to lookup Connection Factory
[ https://issues.apache.org/jira/browse/NIFI-5598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-5598: --- Status: Patch Available (was: Open) > Allow ConsumeJMS and PublishJMS processors to use JNDI to lookup Connection > Factory > --- > > Key: NIFI-5598 > URL: https://issues.apache.org/jira/browse/NIFI-5598 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-5598) Allow ConsumeJMS and PublishJMS processors to use JNDI to lookup Connection Factory
[ https://issues.apache.org/jira/browse/NIFI-5598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-5598: --- Fix Version/s: 1.8.0 > Allow ConsumeJMS and PublishJMS processors to use JNDI to lookup Connection > Factory > --- > > Key: NIFI-5598 > URL: https://issues.apache.org/jira/browse/NIFI-5598 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-5598) Allow ConsumeJMS and PublishJMS processors to use JNDI to lookup Connection Factory
[ https://issues.apache.org/jira/browse/NIFI-5598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-5598: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Allow ConsumeJMS and PublishJMS processors to use JNDI to lookup Connection > Factory > --- > > Key: NIFI-5598 > URL: https://issues.apache.org/jira/browse/NIFI-5598 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.8.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5598) Allow ConsumeJMS and PublishJMS processors to use JNDI to lookup Connection Factory
[ https://issues.apache.org/jira/browse/NIFI-5598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620852#comment-16620852 ] ASF GitHub Bot commented on NIFI-5598: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/3005 > Allow ConsumeJMS and PublishJMS processors to use JNDI to lookup Connection > Factory > --- > > Key: NIFI-5598 > URL: https://issues.apache.org/jira/browse/NIFI-5598 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5598) Allow ConsumeJMS and PublishJMS processors to use JNDI to lookup Connection Factory
[ https://issues.apache.org/jira/browse/NIFI-5598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620848#comment-16620848 ] ASF subversion and git services commented on NIFI-5598: --- Commit ad80f5f064a2895390bdff77278162a1fba68543 in nifi's branch refs/heads/master from [~markap14] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=ad80f5f ] NIFI-5598: Allow JMS Processors to lookup Connection Factory via JNDI NIFI-5598: Expose JNDI Principal & Credentails as explicit properties Signed-off-by: Matthew Burgess This closes #3005 > Allow ConsumeJMS and PublishJMS processors to use JNDI to lookup Connection > Factory > --- > > Key: NIFI-5598 > URL: https://issues.apache.org/jira/browse/NIFI-5598 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5598) Allow ConsumeJMS and PublishJMS processors to use JNDI to lookup Connection Factory
[ https://issues.apache.org/jira/browse/NIFI-5598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620846#comment-16620846 ] ASF GitHub Bot commented on NIFI-5598: -- Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/3005 +1 LGTM, reviewed code and ran unit tests. @bdesert tried with success on a running system. Thanks for the improvement! Merging to master > Allow ConsumeJMS and PublishJMS processors to use JNDI to lookup Connection > Factory > --- > > Key: NIFI-5598 > URL: https://issues.apache.org/jira/browse/NIFI-5598 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5598) Allow ConsumeJMS and PublishJMS processors to use JNDI to lookup Connection Factory
[ https://issues.apache.org/jira/browse/NIFI-5598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620849#comment-16620849 ] ASF subversion and git services commented on NIFI-5598: --- Commit ad80f5f064a2895390bdff77278162a1fba68543 in nifi's branch refs/heads/master from [~markap14] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=ad80f5f ] NIFI-5598: Allow JMS Processors to lookup Connection Factory via JNDI NIFI-5598: Expose JNDI Principal & Credentails as explicit properties Signed-off-by: Matthew Burgess This closes #3005 > Allow ConsumeJMS and PublishJMS processors to use JNDI to lookup Connection > Factory > --- > > Key: NIFI-5598 > URL: https://issues.apache.org/jira/browse/NIFI-5598 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #3005: NIFI-5598: Allow JMS Processors to lookup Connectio...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/3005 ---
[GitHub] nifi issue #3005: NIFI-5598: Allow JMS Processors to lookup Connection Facto...
Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/3005 +1 LGTM, reviewed code and ran unit tests. @bdesert tried with success on a running system. Thanks for the improvement! Merging to master ---
[jira] [Created] (NIFI-5608) PutDatabaseRecord will remove _s in update keys if translate columns is true
eric twilegar created NIFI-5608: --- Summary: PutDatabaseRecord will remove _s in update keys if translate columns is true Key: NIFI-5608 URL: https://issues.apache.org/jira/browse/NIFI-5608 Project: Apache NiFi Issue Type: Bug Reporter: eric twilegar I had a table where the columns names where all defined lower case. In the nifi records it was mixed case and sort of all over the place. translate was working well, but then i added a column with an "_" in it. So a column like my_id which was part of the primary key and so was used the as the WHERE clause in the update statement. So the where clause was "WHERE myid = 5" vs "WHERE my_id =5" -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4685) Hardcoded path when creating Bootstrap ClassLoader in Nifi
[
https://issues.apache.org/jira/browse/NIFI-4685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620807#comment-16620807
]
Imre edited comment on NIFI-4685 at 9/19/18 4:22 PM:
-
So, looking at the source, in my case starting Java on the command-line from
the {{C:\Program Files\NiFi}} directory also does not work, because
{{/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/RunNiFi.java}} makes
the assumption that the second up parent dir of the bootstrap config file will
be the root (the "workingDir") of all things NiFi. Which in my case is also not
true, as I'm putting the configuration files in a different directory:
{{C:\Program Files\My Program\nifi-conf\}}...
{{public void start() throws IOException, InterruptedException {}}
{{...}}
{{final File bootstrapConfigAbsoluteFile =
bootstrapConfigFile.getAbsoluteFile();}}
{{final File binDir = bootstrapConfigAbsoluteFile.getParentFile();}}
{{final File workingDir = binDir.getParentFile();}}
{{...}}
{{}}}
was (Author: olajos):
So, looking at the source, in my case starting Java on the command-line from
the {{C:\Program Files\NiFi}} directory also does not work, because
{{/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/RunNiFi.java}} makes
the assumption that the second up parent dir of the bootstrap config file will
be the root (the "workingDir") of all things NiFi. Which in my case is also not
true, as I'm putting the configuration files in a different directory:
\{{C:\Program Files\My Program\nifi-conf\}}
...
{{public void start() throws IOException, InterruptedException {}}
{{...}}
{{final File bootstrapConfigAbsoluteFile =
bootstrapConfigFile.getAbsoluteFile();}}
{{final File binDir = bootstrapConfigAbsoluteFile.getParentFile();}}
{{final File workingDir = binDir.getParentFile();}}
{{...}}
{{}}}
> Hardcoded path when creating Bootstrap ClassLoader in Nifi
> --
>
> Key: NIFI-4685
> URL: https://issues.apache.org/jira/browse/NIFI-4685
> Project: Apache NiFi
> Issue Type: Bug
> Components: Configuration, Core Framework
>Affects Versions: 1.4.0
> Environment: Windows 7, JRE 1.8.0_144
>Reporter: Sorin Florea
>Priority: Minor
> Labels: easyfix, usability, windows
>
> Found a hardcoded path to {{lib/bootstrap}} in
> [org.apache.nifi.NiFi|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-runtime/src/main/java/org/apache/nifi/NiFi.java#L3]
> class when calling {{createBootstrapClassLoader()}} :
> {code:java}
> private static ClassLoader createBootstrapClassLoader() throws IOException {
> //Get list of files in bootstrap folder
> final List urls = new ArrayList<>();
> Files.list(Paths.get({color:red}_"lib/bootstrap"_{color})).forEach(p
> -> {
> try {
> urls.add(p.toUri().toURL());
> } catch (final MalformedURLException mef) {
> LOGGER.warn("Unable to load " + p.getFileName() + " due to "
> + mef, mef);
> }
> });
> //Create the bootstrap classloader
> return new URLClassLoader(urls.toArray(new URL[0]),
> Thread.currentThread().getContextClassLoader());
> }
> {code}
> There are multiple places where the {{.\lib}} directory is defined:
> * {{BOOTSTRAP_LIBS}} or {{%LIB_DIR%}} environment variables in {{nifi.sh}} or
> {{run-nifi.bat}}
> * {{lib.dir}} in {{bootstrap.conf}} is most likely it
> * {{nifi.nar.library.directory}} or {{nifi.web.war.directory}} identify the
> library folder but don't quite refer to the bootstrap
> Perhaps using {{lib.dir}} in {{bootstrap.conf}} or create a new
> {{lib.bootstrap.dir}} property.
> Using different library folders would allow configuring multiple instances of
> Nifi using the same resource. I managed changing the properties and
> environment variables above to point to a common {{lib}} directory but this
> harcoded prevented {{run-nifi.bat}} to run successfully. {{dump-nifi.bat}}
> and {{status-nifi.bat}} ran without issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4685) Hardcoded path when creating Bootstrap ClassLoader in Nifi
[
https://issues.apache.org/jira/browse/NIFI-4685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620807#comment-16620807
]
Imre edited comment on NIFI-4685 at 9/19/18 4:22 PM:
-
So, looking at the source, in my case starting Java on the command-line from
the {{C:\Program Files\NiFi}} directory also does not work, because
{{/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/RunNiFi.java}} makes
the assumption that the second up parent dir of the bootstrap config file will
be the root (the "workingDir") of all things NiFi. Which in my case is also not
true, as I'm putting the configuration files in a different directory:
\{{C:\Program Files\My Program\nifi-conf\}}...
{{public void start() throws IOException, InterruptedException {}}
{{...}}
{{final File bootstrapConfigAbsoluteFile =
bootstrapConfigFile.getAbsoluteFile();}}
{{final File binDir = bootstrapConfigAbsoluteFile.getParentFile();}}
{{final File workingDir = binDir.getParentFile();}}
{{...}}
{{}}}
was (Author: olajos):
So, looking at the source, in my case starting Java on the command-line from
the {{C:\Program Files\NiFi}} directory also does not work, because
{{/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/RunNiFi.java}} makes
the assumption that the second up parent dir of the bootstrap config file will
be the root (the "workingDir") of all things NiFi. Which in my case is also not
true, as I'm putting the configuration files in a different directory...
{{public void start() throws IOException, InterruptedException {}}
{{...}}
{{final File bootstrapConfigAbsoluteFile =
bootstrapConfigFile.getAbsoluteFile();}}
{{final File binDir = bootstrapConfigAbsoluteFile.getParentFile();}}
{{final File workingDir = binDir.getParentFile();}}
{{...}}
{{}}}
> Hardcoded path when creating Bootstrap ClassLoader in Nifi
> --
>
> Key: NIFI-4685
> URL: https://issues.apache.org/jira/browse/NIFI-4685
> Project: Apache NiFi
> Issue Type: Bug
> Components: Configuration, Core Framework
>Affects Versions: 1.4.0
> Environment: Windows 7, JRE 1.8.0_144
>Reporter: Sorin Florea
>Priority: Minor
> Labels: easyfix, usability, windows
>
> Found a hardcoded path to {{lib/bootstrap}} in
> [org.apache.nifi.NiFi|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-runtime/src/main/java/org/apache/nifi/NiFi.java#L3]
> class when calling {{createBootstrapClassLoader()}} :
> {code:java}
> private static ClassLoader createBootstrapClassLoader() throws IOException {
> //Get list of files in bootstrap folder
> final List urls = new ArrayList<>();
> Files.list(Paths.get({color:red}_"lib/bootstrap"_{color})).forEach(p
> -> {
> try {
> urls.add(p.toUri().toURL());
> } catch (final MalformedURLException mef) {
> LOGGER.warn("Unable to load " + p.getFileName() + " due to "
> + mef, mef);
> }
> });
> //Create the bootstrap classloader
> return new URLClassLoader(urls.toArray(new URL[0]),
> Thread.currentThread().getContextClassLoader());
> }
> {code}
> There are multiple places where the {{.\lib}} directory is defined:
> * {{BOOTSTRAP_LIBS}} or {{%LIB_DIR%}} environment variables in {{nifi.sh}} or
> {{run-nifi.bat}}
> * {{lib.dir}} in {{bootstrap.conf}} is most likely it
> * {{nifi.nar.library.directory}} or {{nifi.web.war.directory}} identify the
> library folder but don't quite refer to the bootstrap
> Perhaps using {{lib.dir}} in {{bootstrap.conf}} or create a new
> {{lib.bootstrap.dir}} property.
> Using different library folders would allow configuring multiple instances of
> Nifi using the same resource. I managed changing the properties and
> environment variables above to point to a common {{lib}} directory but this
> harcoded prevented {{run-nifi.bat}} to run successfully. {{dump-nifi.bat}}
> and {{status-nifi.bat}} ran without issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (NIFI-4685) Hardcoded path when creating Bootstrap ClassLoader in Nifi
[
https://issues.apache.org/jira/browse/NIFI-4685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620807#comment-16620807
]
Imre edited comment on NIFI-4685 at 9/19/18 4:22 PM:
-
So, looking at the source, in my case starting Java on the command-line from
the {{C:\Program Files\NiFi}} directory also does not work, because
{{/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/RunNiFi.java}} makes
the assumption that the second up parent dir of the bootstrap config file will
be the root (the "workingDir") of all things NiFi. Which in my case is also not
true, as I'm putting the configuration files in a different directory:
\{{C:\Program Files\My Program\nifi-conf\}}
...
{{public void start() throws IOException, InterruptedException {}}
{{...}}
{{final File bootstrapConfigAbsoluteFile =
bootstrapConfigFile.getAbsoluteFile();}}
{{final File binDir = bootstrapConfigAbsoluteFile.getParentFile();}}
{{final File workingDir = binDir.getParentFile();}}
{{...}}
{{}}}
was (Author: olajos):
So, looking at the source, in my case starting Java on the command-line from
the {{C:\Program Files\NiFi}} directory also does not work, because
{{/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/RunNiFi.java}} makes
the assumption that the second up parent dir of the bootstrap config file will
be the root (the "workingDir") of all things NiFi. Which in my case is also not
true, as I'm putting the configuration files in a different directory:
\{{C:\Program Files\My Program\nifi-conf\}}...
{{public void start() throws IOException, InterruptedException {}}
{{...}}
{{final File bootstrapConfigAbsoluteFile =
bootstrapConfigFile.getAbsoluteFile();}}
{{final File binDir = bootstrapConfigAbsoluteFile.getParentFile();}}
{{final File workingDir = binDir.getParentFile();}}
{{...}}
{{}}}
> Hardcoded path when creating Bootstrap ClassLoader in Nifi
> --
>
> Key: NIFI-4685
> URL: https://issues.apache.org/jira/browse/NIFI-4685
> Project: Apache NiFi
> Issue Type: Bug
> Components: Configuration, Core Framework
>Affects Versions: 1.4.0
> Environment: Windows 7, JRE 1.8.0_144
>Reporter: Sorin Florea
>Priority: Minor
> Labels: easyfix, usability, windows
>
> Found a hardcoded path to {{lib/bootstrap}} in
> [org.apache.nifi.NiFi|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-runtime/src/main/java/org/apache/nifi/NiFi.java#L3]
> class when calling {{createBootstrapClassLoader()}} :
> {code:java}
> private static ClassLoader createBootstrapClassLoader() throws IOException {
> //Get list of files in bootstrap folder
> final List urls = new ArrayList<>();
> Files.list(Paths.get({color:red}_"lib/bootstrap"_{color})).forEach(p
> -> {
> try {
> urls.add(p.toUri().toURL());
> } catch (final MalformedURLException mef) {
> LOGGER.warn("Unable to load " + p.getFileName() + " due to "
> + mef, mef);
> }
> });
> //Create the bootstrap classloader
> return new URLClassLoader(urls.toArray(new URL[0]),
> Thread.currentThread().getContextClassLoader());
> }
> {code}
> There are multiple places where the {{.\lib}} directory is defined:
> * {{BOOTSTRAP_LIBS}} or {{%LIB_DIR%}} environment variables in {{nifi.sh}} or
> {{run-nifi.bat}}
> * {{lib.dir}} in {{bootstrap.conf}} is most likely it
> * {{nifi.nar.library.directory}} or {{nifi.web.war.directory}} identify the
> library folder but don't quite refer to the bootstrap
> Perhaps using {{lib.dir}} in {{bootstrap.conf}} or create a new
> {{lib.bootstrap.dir}} property.
> Using different library folders would allow configuring multiple instances of
> Nifi using the same resource. I managed changing the properties and
> environment variables above to point to a common {{lib}} directory but this
> harcoded prevented {{run-nifi.bat}} to run successfully. {{dump-nifi.bat}}
> and {{status-nifi.bat}} ran without issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4685) Hardcoded path when creating Bootstrap ClassLoader in Nifi
[
https://issues.apache.org/jira/browse/NIFI-4685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620807#comment-16620807
]
Imre commented on NIFI-4685:
So, looking at the source, in my case starting Java on the command-line from
the {{C:\Program Files\NiFi}} directory also does not work, because
{{/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/RunNiFi.java}} makes
the assumption that the second up parent dir of the bootstrap config file will
be the root (the "workingDir") of all things NiFi. Which in my case is also not
true, as I'm putting the configuration files in a different directory...
{{public void start() throws IOException, InterruptedException {}}
{{...}}
{{final File bootstrapConfigAbsoluteFile =
bootstrapConfigFile.getAbsoluteFile();}}
{{final File binDir = bootstrapConfigAbsoluteFile.getParentFile();}}
{{final File workingDir = binDir.getParentFile();}}
{{...}}
{{}}}
> Hardcoded path when creating Bootstrap ClassLoader in Nifi
> --
>
> Key: NIFI-4685
> URL: https://issues.apache.org/jira/browse/NIFI-4685
> Project: Apache NiFi
> Issue Type: Bug
> Components: Configuration, Core Framework
>Affects Versions: 1.4.0
> Environment: Windows 7, JRE 1.8.0_144
>Reporter: Sorin Florea
>Priority: Minor
> Labels: easyfix, usability, windows
>
> Found a hardcoded path to {{lib/bootstrap}} in
> [org.apache.nifi.NiFi|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-runtime/src/main/java/org/apache/nifi/NiFi.java#L3]
> class when calling {{createBootstrapClassLoader()}} :
> {code:java}
> private static ClassLoader createBootstrapClassLoader() throws IOException {
> //Get list of files in bootstrap folder
> final List urls = new ArrayList<>();
> Files.list(Paths.get({color:red}_"lib/bootstrap"_{color})).forEach(p
> -> {
> try {
> urls.add(p.toUri().toURL());
> } catch (final MalformedURLException mef) {
> LOGGER.warn("Unable to load " + p.getFileName() + " due to "
> + mef, mef);
> }
> });
> //Create the bootstrap classloader
> return new URLClassLoader(urls.toArray(new URL[0]),
> Thread.currentThread().getContextClassLoader());
> }
> {code}
> There are multiple places where the {{.\lib}} directory is defined:
> * {{BOOTSTRAP_LIBS}} or {{%LIB_DIR%}} environment variables in {{nifi.sh}} or
> {{run-nifi.bat}}
> * {{lib.dir}} in {{bootstrap.conf}} is most likely it
> * {{nifi.nar.library.directory}} or {{nifi.web.war.directory}} identify the
> library folder but don't quite refer to the bootstrap
> Perhaps using {{lib.dir}} in {{bootstrap.conf}} or create a new
> {{lib.bootstrap.dir}} property.
> Using different library folders would allow configuring multiple instances of
> Nifi using the same resource. I managed changing the properties and
> environment variables above to point to a common {{lib}} directory but this
> harcoded prevented {{run-nifi.bat}} to run successfully. {{dump-nifi.bat}}
> and {{status-nifi.bat}} ran without issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5327) NetFlow Processors
[ https://issues.apache.org/jira/browse/NIFI-5327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620803#comment-16620803 ] ASF GitHub Bot commented on NIFI-5327: -- Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2820 @PrashanthVenkatesan if any of the builds passed, don't worry about it. It's the same test, just localized to different locale settings. > NetFlow Processors > -- > > Key: NIFI-5327 > URL: https://issues.apache.org/jira/browse/NIFI-5327 > Project: Apache NiFi > Issue Type: New Feature > Components: Core Framework >Affects Versions: 1.6.0 >Reporter: Prashanth Venkatesan >Assignee: Prashanth Venkatesan >Priority: Major > > As network traffic data scopes for the big data use case, would like NiFi to > have processors to support parsing of those protocols. > Netflow is a protocol introduced by Cisco that provides the ability to > collect IP network traffic as it enters or exits an interface and is > described in detail in here: > [https://www.cisco.com/c/en/us/td/docs/net_mgmt/netflow_collection_engine/3-6/user/guide/format.html] > > Currently, I have created the following processor: > *ParseNetflowv5*: Parses the ingress netflowv5 bytes and ingest as either > NiFi flowfile attributes or as a JSON content. This also sends > one-time-template. > > Further ahead, we can add many processor specific to network protocols in > this nar bundle. > I will create a pull request. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2820: NIFI-5327 Adding Netflowv5 protocol parser
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2820 @PrashanthVenkatesan if any of the builds passed, don't worry about it. It's the same test, just localized to different locale settings. ---
***UNCHECKED*** [jira] [Commented] (NIFI-3469) Add multipart request support to HandleHttpRequest Processor
[
https://issues.apache.org/jira/browse/NIFI-3469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620797#comment-16620797
]
ASF GitHub Bot commented on NIFI-3469:
--
Github user alopresto commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2991#discussion_r218868215
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/HandleHttpRequest.java
---
@@ -521,161 +553,221 @@ public void onTrigger(final ProcessContext context,
final ProcessSession session
final long start = System.nanoTime();
final HttpServletRequest request = container.getRequest();
-FlowFile flowFile = session.create();
-try (OutputStream flowFileOut = session.write(flowFile)) {
-StreamUtils.copy(request.getInputStream(), flowFileOut);
-} catch (final IOException e) {
-// There may be many reasons which can produce an IOException
on the HTTP stream and in some of them, eg.
-// bad requests, the connection to the client is not closed.
In order to address also these cases, we try
-// and answer with a BAD_REQUEST, which lets the client know
that the request has not been correctly
-// processed and makes it aware that the connection can be
closed.
-getLogger().error("Failed to receive content from HTTP Request
from {} due to {}",
-new Object[]{request.getRemoteAddr(), e});
-session.remove(flowFile);
-try {
-HttpServletResponse response = container.getResponse();
-response.sendError(Status.BAD_REQUEST.getStatusCode());
-response.flushBuffer();
-container.getContext().complete();
-} catch (final IOException ioe) {
-getLogger().warn("Failed to send HTTP response to {} due
to {}",
-new Object[]{request.getRemoteAddr(), ioe});
+if (!Strings.isNullOrEmpty(request.getContentType()) &&
request.getContentType().contains(MIME_TYPE__MULTIPART_FORM_DATA)) {
+ final long maxRequestSize =
context.getProperty(MAX_REQUEST_SIZE).asLong();
+ request.setAttribute(Request.__MULTIPART_CONFIG_ELEMENT, new
MultipartConfigElement("/tmp", maxRequestSize, maxRequestSize, 0));
--- End diff --
This also opens up a lot of security concerns. We need to be very careful
about how we handle, sanitize, trust, store, and display this data.
Some good starting places for reading:
* https://www.owasp.org/index.php/Deserialization_of_untrusted_data
* https://www.owasp.org/index.php/Unrestricted_File_Upload
* https://www.owasp.org/index.php/Insecure_Temporary_File
* https://www.owasp.org/index.php/Protect_FileUpload_Against_Malicious_File
> Add multipart request support to HandleHttpRequest Processor
>
>
> Key: NIFI-3469
> URL: https://issues.apache.org/jira/browse/NIFI-3469
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Reporter: Koji Kawamura
>Assignee: Endre Kovacs
>Priority: Major
>
> Currently, HandleHttpRequest outputs a single FlowFile containing all
> multipart values as following:
> {code}
> --ef07e8bf36c274d3
> Content-Disposition: form-data; name="p1"
> v1
> --ef07e8bf36c274d3
> Content-Disposition: form-data; name="p2"
> v2
> --ef07e8bf36c274d3--
> {code}
> Many users requested adding upload files support to NiFi.
> In order for HandleHttpRequest to support multipart data we need to add
> followings (this is based on a brief researching and can be more complex or
> simple):
> We need to use HttpServletRequest#getParts() as written in this stackoverflow
> thread:
> http://stackoverflow.com/questions/3337056/convenient-way-to-parse-incoming-multipart-form-data-parameters-in-a-servlet
> Also, we probably need a custom MultiPartInputStreamParser implementation.
> Because Jetty's default implementation writes input data to temporary
> directory on file system, instead, we'd like NiFi to write those into output
> FlowFiles content in streaming fashion.
> And we need request size validation checks, threshold for those validation
> should be passed via javax.servlet.MultipartConfigElement.
> Finally, we have to do something with HandleHttpResponse processor.
> Once HandleHttpRequest processor start splitting incoming request into
> multiple output FlowFiles, we need to wait for every fragment to be
> processed, then execute HandleHttpRequest.
> I think Wait/Notify processors (available
[GitHub] nifi pull request #2991: NIFI-3469: multipart request support added to Handl...
Github user alopresto commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2991#discussion_r218868215
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/HandleHttpRequest.java
---
@@ -521,161 +553,221 @@ public void onTrigger(final ProcessContext context,
final ProcessSession session
final long start = System.nanoTime();
final HttpServletRequest request = container.getRequest();
-FlowFile flowFile = session.create();
-try (OutputStream flowFileOut = session.write(flowFile)) {
-StreamUtils.copy(request.getInputStream(), flowFileOut);
-} catch (final IOException e) {
-// There may be many reasons which can produce an IOException
on the HTTP stream and in some of them, eg.
-// bad requests, the connection to the client is not closed.
In order to address also these cases, we try
-// and answer with a BAD_REQUEST, which lets the client know
that the request has not been correctly
-// processed and makes it aware that the connection can be
closed.
-getLogger().error("Failed to receive content from HTTP Request
from {} due to {}",
-new Object[]{request.getRemoteAddr(), e});
-session.remove(flowFile);
-try {
-HttpServletResponse response = container.getResponse();
-response.sendError(Status.BAD_REQUEST.getStatusCode());
-response.flushBuffer();
-container.getContext().complete();
-} catch (final IOException ioe) {
-getLogger().warn("Failed to send HTTP response to {} due
to {}",
-new Object[]{request.getRemoteAddr(), ioe});
+if (!Strings.isNullOrEmpty(request.getContentType()) &&
request.getContentType().contains(MIME_TYPE__MULTIPART_FORM_DATA)) {
+ final long maxRequestSize =
context.getProperty(MAX_REQUEST_SIZE).asLong();
+ request.setAttribute(Request.__MULTIPART_CONFIG_ELEMENT, new
MultipartConfigElement("/tmp", maxRequestSize, maxRequestSize, 0));
--- End diff --
This also opens up a lot of security concerns. We need to be very careful
about how we handle, sanitize, trust, store, and display this data.
Some good starting places for reading:
* https://www.owasp.org/index.php/Deserialization_of_untrusted_data
* https://www.owasp.org/index.php/Unrestricted_File_Upload
* https://www.owasp.org/index.php/Insecure_Temporary_File
* https://www.owasp.org/index.php/Protect_FileUpload_Against_Malicious_File
---
***UNCHECKED*** [jira] [Commented] (NIFI-5333) Create GetMongoRecord processor
[ https://issues.apache.org/jira/browse/NIFI-5333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620789#comment-16620789 ] ASF GitHub Bot commented on NIFI-5333: -- GitHub user MikeThomsen opened a pull request: https://github.com/apache/nifi/pull/3011 NIFI-5333 Added GetMongoRecord. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/MikeThomsen/nifi NIFI-5333 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3011.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3011 commit d9fbe281b2a01c66cea9c4a6523c84c84c303be7 Author: Mike Thomsen Date: 2018-09-02T19:58:33Z NIFI-5333 Added GetMongoRecord. > Create GetMongoRecord processor > --- > > Key: NIFI-5333 > URL: https://issues.apache.org/jira/browse/NIFI-5333 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Mike Thomsen >Assignee: Mike Thomsen >Priority: Major > > A processor similar to GetMongo that uses the record API should be created. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #3011: NIFI-5333 Added GetMongoRecord.
GitHub user MikeThomsen opened a pull request: https://github.com/apache/nifi/pull/3011 NIFI-5333 Added GetMongoRecord. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/MikeThomsen/nifi NIFI-5333 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3011.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3011 commit d9fbe281b2a01c66cea9c4a6523c84c84c303be7 Author: Mike Thomsen Date: 2018-09-02T19:58:33Z NIFI-5333 Added GetMongoRecord. ---
***UNCHECKED*** [jira] [Comment Edited] (NIFI-4685) Hardcoded path when creating Bootstrap ClassLoader in Nifi
[
https://issues.apache.org/jira/browse/NIFI-4685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620727#comment-16620727
]
Imre edited comment on NIFI-4685 at 9/19/18 3:53 PM:
-
I just bumped into the same issue (NiFi v1.7.1). In my case, running on
Windows, I created my own batch file to start NiFi, with the lib directory of
NiFi being installed in {{c:\Program Files\NiFi}}, and all generated files
(logs, internal DB, etc.) pointed to {{C:\ProgramData\NiFi}}. (I modified
several NiFi config files to achieve this.) I happened to start the batch file
from somewhere other than the NiFi root dir in {{C:\Program Files\NiFi}}, and I
got the following exception (due to the above code):
{{2018-09-19 09:30:25,895 INFO [main] org.apache.nifi.NiFi Launching NiFi...}}
{{2018-09-19 09:30:25,906 ERROR [main] org.apache.nifi.NiFi Failure to launch
NiFi due to java.nio.file.NoSuchFileException: lib\bootstrap}}
{{java.nio.file.NoSuchFileException: lib\bootstrap}}
{{at
sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:79)}}
{{at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)}}
{{at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)}}
{{at sun.nio.fs.WindowsDirectoryStream.(WindowsDirectoryStream.java:86)}}
{{at
sun.nio.fs.WindowsFileSystemProvider.newDirectoryStream(WindowsFileSystemProvider.java:518)}}
{{at java.nio.file.Files.newDirectoryStream(Files.java:457)}}
{{at java.nio.file.Files.list(Files.java:3451)}}
{{at org.apache.nifi.NiFi.createBootstrapClassLoader(NiFi.java:197)}}
{{at
org.apache.nifi.NiFi.convertArgumentsToValidatedNiFiProperties(NiFi.java:299)}}
{{at org.apache.nifi.NiFi.main(NiFi.java:291)}}
Although, interestingly, starting Java on the command-line from the
{{C:\Program Files\NiFi}} directory also results in the same exception, even
though there's definitely a {{lib\bootstrap}} directory there relative to the
current dir...
was (Author: olajos):
I just bumped into the same issue (NiFi v1.7.1). In my case, running on
Windows, I created my own batch file to start NiFi, with the lib directory of
NiFi being installed in {{c:\Program Files\NiFi}}, and all generated files
(logs, internal DB, etc.) pointed to {{C:\ProgramData\NiFi}}. (I modified
several NiFi config files to achieve this.) I happened to start the batch file
from somewhere other than the NiFi root dir in {{C:\Program Files\NiFi}}, and I
got the following exception (due to the above code):
{{2018-09-19 09:30:25,895 INFO [main] org.apache.nifi.NiFi Launching NiFi...}}
{{2018-09-19 09:30:25,906 ERROR [main] org.apache.nifi.NiFi Failure to launch
NiFi due to java.nio.file.NoSuchFileException: lib\bootstrap}}
{{java.nio.file.NoSuchFileException: lib\bootstrap}}
{{at
sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:79)}}
{{at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)}}
{{at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)}}
{{at sun.nio.fs.WindowsDirectoryStream.(WindowsDirectoryStream.java:86)}}
{{at
sun.nio.fs.WindowsFileSystemProvider.newDirectoryStream(WindowsFileSystemProvider.java:518)}}
{{at java.nio.file.Files.newDirectoryStream(Files.java:457)}}
{{at java.nio.file.Files.list(Files.java:3451)}}
{{at org.apache.nifi.NiFi.createBootstrapClassLoader(NiFi.java:197)}}
{{at
org.apache.nifi.NiFi.convertArgumentsToValidatedNiFiProperties(NiFi.java:299)}}
{{at org.apache.nifi.NiFi.main(NiFi.java:291)}}
Although, interestingly, starting Java on the command-line from the
{{C:\Program Files\NiFi}} directory also results in the same exception, even
though there's definitely a {{lib\bootstrapper}} directory there relative to
the current dir...
> Hardcoded path when creating Bootstrap ClassLoader in Nifi
> --
>
> Key: NIFI-4685
> URL: https://issues.apache.org/jira/browse/NIFI-4685
> Project: Apache NiFi
> Issue Type: Bug
> Components: Configuration, Core Framework
>Affects Versions: 1.4.0
> Environment: Windows 7, JRE 1.8.0_144
>Reporter: Sorin Florea
>Priority: Minor
> Labels: easyfix, usability, windows
>
> Found a hardcoded path to {{lib/bootstrap}} in
> [org.apache.nifi.NiFi|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-runtime/src/main/java/org/apache/nifi/NiFi.java#L3]
> class when calling {{createBootstrapClassLoader()}} :
> {code:java}
> private static ClassLoader createBootstrapClassLoader() throws IOException {
> //Get list of files in bootstrap folder
> final List urls = new ArrayList<>();
> Files.list(Paths.get({color:red}_"lib/bootstrap"_{color})).forEach(p
> -> {
> try {
>
[jira] [Comment Edited] (NIFI-4685) Hardcoded path when creating Bootstrap ClassLoader in Nifi
[
https://issues.apache.org/jira/browse/NIFI-4685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620727#comment-16620727
]
Imre edited comment on NIFI-4685 at 9/19/18 3:53 PM:
-
I just bumped into the same issue (NiFi v1.7.1). In my case, running on
Windows, I created my own batch file to start NiFi, with the lib directory of
NiFi being installed in {{c:\Program Files\NiFi}}, and all generated files
(logs, internal DB, etc.) pointed to {{C:\ProgramData\NiFi}}. (I modified
several NiFi config files to achieve this.) I happened to start the batch file
from somewhere other than the NiFi root dir in {{C:\Program Files\NiFi}}, and I
got the following exception (due to the above code):
{{2018-09-19 09:30:25,895 INFO [main] org.apache.nifi.NiFi Launching NiFi...}}
{{2018-09-19 09:30:25,906 ERROR [main] org.apache.nifi.NiFi Failure to launch
NiFi due to java.nio.file.NoSuchFileException: lib\bootstrap}}
{{java.nio.file.NoSuchFileException: lib\bootstrap}}
{{at
sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:79)}}
{{at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)}}
{{at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)}}
{{at sun.nio.fs.WindowsDirectoryStream.(WindowsDirectoryStream.java:86)}}
{{at
sun.nio.fs.WindowsFileSystemProvider.newDirectoryStream(WindowsFileSystemProvider.java:518)}}
{{at java.nio.file.Files.newDirectoryStream(Files.java:457)}}
{{at java.nio.file.Files.list(Files.java:3451)}}
{{at org.apache.nifi.NiFi.createBootstrapClassLoader(NiFi.java:197)}}
{{at
org.apache.nifi.NiFi.convertArgumentsToValidatedNiFiProperties(NiFi.java:299)}}
{{at org.apache.nifi.NiFi.main(NiFi.java:291)}}
Although, interestingly, starting Java on the command-line from the
{{C:\Program Files\NiFi}} directory also results in the same exception, even
though there's definitely a {{lib\bootstrapper}} directory there relative to
the current dir...
was (Author: olajos):
I just bumped into the same issue. In my case, I'm running on Windows, I
created my own batch file to start NiFi, with the lib directory of NiFi being
installed in {{c:\Program Files\NiFi}}, and all generated files (logs, internal
DB, etc.) pointed to {{C:\ProgramData\NiFi}}. I happened to start the batch
file from somewhere other than the NiFi root dir in {{C:\Program Files\NiFi}},
and I got the following exception (due to the above code):
{{2018-09-19 09:30:25,895 INFO [main] org.apache.nifi.NiFi Launching NiFi...}}
{{2018-09-19 09:30:25,906 ERROR [main] org.apache.nifi.NiFi Failure to launch
NiFi due to java.nio.file.NoSuchFileException: lib\bootstrap}}
{{java.nio.file.NoSuchFileException: lib\bootstrap}}
{{at
sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:79)}}
{{at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)}}
{{at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)}}
{{at sun.nio.fs.WindowsDirectoryStream.(WindowsDirectoryStream.java:86)}}
{{at
sun.nio.fs.WindowsFileSystemProvider.newDirectoryStream(WindowsFileSystemProvider.java:518)}}
{{at java.nio.file.Files.newDirectoryStream(Files.java:457)}}
{{at java.nio.file.Files.list(Files.java:3451)}}
{{at org.apache.nifi.NiFi.createBootstrapClassLoader(NiFi.java:197)}}
{{at
org.apache.nifi.NiFi.convertArgumentsToValidatedNiFiProperties(NiFi.java:299)}}
{{at org.apache.nifi.NiFi.main(NiFi.java:291)}}
> Hardcoded path when creating Bootstrap ClassLoader in Nifi
> --
>
> Key: NIFI-4685
> URL: https://issues.apache.org/jira/browse/NIFI-4685
> Project: Apache NiFi
> Issue Type: Bug
> Components: Configuration, Core Framework
>Affects Versions: 1.4.0
> Environment: Windows 7, JRE 1.8.0_144
>Reporter: Sorin Florea
>Priority: Minor
> Labels: easyfix, usability, windows
>
> Found a hardcoded path to {{lib/bootstrap}} in
> [org.apache.nifi.NiFi|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-runtime/src/main/java/org/apache/nifi/NiFi.java#L3]
> class when calling {{createBootstrapClassLoader()}} :
> {code:java}
> private static ClassLoader createBootstrapClassLoader() throws IOException {
> //Get list of files in bootstrap folder
> final List urls = new ArrayList<>();
> Files.list(Paths.get({color:red}_"lib/bootstrap"_{color})).forEach(p
> -> {
> try {
> urls.add(p.toUri().toURL());
> } catch (final MalformedURLException mef) {
> LOGGER.warn("Unable to load " + p.getFileName() + " due to "
> + mef, mef);
> }
> });
> //Create the bootstrap classloader
> return new URLClassLoader(urls.toArray(new
