[jira] [Updated] (NIFI-4258) CSVUtils uses same AllowableValue for 'Informix Unload' and 'Informix Unload Escape Disabled'
[ https://issues.apache.org/jira/browse/NIFI-4258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wesley L Lawrence updated NIFI-4258: Attachment: NIFI-4258.patch > CSVUtils uses same AllowableValue for 'Informix Unload' and 'Informix Unload > Escape Disabled' > - > > Key: NIFI-4258 > URL: https://issues.apache.org/jira/browse/NIFI-4258 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.4.0 >Reporter: Wesley L Lawrence >Priority: Minor > Attachments: NIFI-4258.patch > > > Related to NIFI-4242, if you can't use 'Informix Unload Escape Disabled' as a > pre-defined CSV format, because 'Informix Unload' has the same allowable > value. > The WebUI for CSVRedaer/CSVRecordSetWriter seems to always display 'Informix > Unload', and when choosing from the drop down, says 'Informix Unload Escape > Delimited'. > Given that within CSVUtils, 'Informix Unload' is checked against first, I > suspect that's the one that gets chosen. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4258) CSVUtils uses same AllowableValue for 'Informix Unload' and 'Informix Unload Escape Disabled'
[ https://issues.apache.org/jira/browse/NIFI-4258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wesley L Lawrence updated NIFI-4258: Status: Patch Available (was: Open) > CSVUtils uses same AllowableValue for 'Informix Unload' and 'Informix Unload > Escape Disabled' > - > > Key: NIFI-4258 > URL: https://issues.apache.org/jira/browse/NIFI-4258 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.4.0 >Reporter: Wesley L Lawrence >Priority: Minor > Attachments: NIFI-4258.patch > > > Related to NIFI-4242, if you can't use 'Informix Unload Escape Disabled' as a > pre-defined CSV format, because 'Informix Unload' has the same allowable > value. > The WebUI for CSVRedaer/CSVRecordSetWriter seems to always display 'Informix > Unload', and when choosing from the drop down, says 'Informix Unload Escape > Delimited'. > Given that within CSVUtils, 'Informix Unload' is checked against first, I > suspect that's the one that gets chosen. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4242) CSVReader shouldn't require that an escape character be defined
[
https://issues.apache.org/jira/browse/NIFI-4242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wesley L Lawrence updated NIFI-4242:
Attachment: NIFI-4242.patch
> CSVReader shouldn't require that an escape character be defined
> ---
>
> Key: NIFI-4242
> URL: https://issues.apache.org/jira/browse/NIFI-4242
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Wesley L Lawrence
>Priority: Minor
> Attachments: NIFI-4242.patch, NIFI-4242.patch
>
>
> There are situations where, when parsing a CSV file, one doesn't want to
> define an escape character. For example, when using quote character ", the
> following is valid CSV;
> {code}
> a,"""b",c
> {code}
> The second column should be interpreted as "b. But when Apache Commons CSV is
> told that there's an escape character, the above row is invalid
> (interestingly, if it was """b""", it would be valid as "b").
> There are known formats that Apache Commons CSV provides, that doesn't define
> escape characters either.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (NIFI-4242) CSVReader shouldn't require that an escape character be defined
[
https://issues.apache.org/jira/browse/NIFI-4242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wesley L Lawrence updated NIFI-4242:
Status: Patch Available (was: Open)
> CSVReader shouldn't require that an escape character be defined
> ---
>
> Key: NIFI-4242
> URL: https://issues.apache.org/jira/browse/NIFI-4242
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Wesley L Lawrence
>Priority: Minor
> Attachments: NIFI-4242.patch
>
>
> There are situations where, when parsing a CSV file, one doesn't want to
> define an escape character. For example, when using quote character ", the
> following is valid CSV;
> {code}
> a,"""b",c
> {code}
> The second column should be interpreted as "b. But when Apache Commons CSV is
> told that there's an escape character, the above row is invalid
> (interestingly, if it was """b""", it would be valid as "b").
> There are known formats that Apache Commons CSV provides, that doesn't define
> escape characters either.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (NIFI-4242) CSVReader shouldn't require that an escape character be defined
[
https://issues.apache.org/jira/browse/NIFI-4242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wesley L Lawrence updated NIFI-4242:
Attachment: NIFI-4242.patch
> CSVReader shouldn't require that an escape character be defined
> ---
>
> Key: NIFI-4242
> URL: https://issues.apache.org/jira/browse/NIFI-4242
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.3.0
>Reporter: Wesley L Lawrence
>Priority: Minor
> Attachments: NIFI-4242.patch
>
>
> There are situations where, when parsing a CSV file, one doesn't want to
> define an escape character. For example, when using quote character ", the
> following is valid CSV;
> {code}
> a,"""b",c
> {code}
> The second column should be interpreted as "b. But when Apache Commons CSV is
> told that there's an escape character, the above row is invalid
> (interestingly, if it was """b""", it would be valid as "b").
> There are known formats that Apache Commons CSV provides, that doesn't define
> escape characters either.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-4215) Avro schemas with records that have a field of themselves fail to parse, causing stackoverflow exception
[
https://issues.apache.org/jira/browse/NIFI-4215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113158#comment-16113158
]
Wesley L Lawrence commented on NIFI-4215:
-
[~markap14] No worries, it's always better to be safe then sorry.
I'll re-submit the patch, with the schema identifier issues fixed. Thanks again
for taking a look at this!
> Avro schemas with records that have a field of themselves fail to parse,
> causing stackoverflow exception
>
>
> Key: NIFI-4215
> URL: https://issues.apache.org/jira/browse/NIFI-4215
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Wesley L Lawrence
>Priority: Minor
> Attachments: nifi-4215.patch
>
>
> Noticed this while attempting to use the AvroSchemaRegsitry with some complex
> schema. Boiled down, Avro lets you define a schema such as;
> {code}
> {
> "namespace": "org.apache.nifi.testing",
> "name": "CompositRecord",
> "type": "record",
> "fields": [
> {
> "name": "id",
> "type": "int"
> },
> {
> "name": "value",
> "type": "string"
> },
> {
> "name": "parent",
> "type": [
> "null",
> "CompositRecord"
> ]
> }
> ]
> }
> {code}
> The AvroSchemaRegistry (AvroTypeUtil specifically) will fail to parse, and
> generate a stackoverflow exception.
> I've whipped up a fix, tested it out in 1.4.0, and am just running through
> the contrib build before I submit a patch.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (NIFI-4258) CSVUtils uses same AllowableValue for 'Informix Unload' and 'Informix Unload Escape Disabled'
Wesley L Lawrence created NIFI-4258: --- Summary: CSVUtils uses same AllowableValue for 'Informix Unload' and 'Informix Unload Escape Disabled' Key: NIFI-4258 URL: https://issues.apache.org/jira/browse/NIFI-4258 Project: Apache NiFi Issue Type: Bug Affects Versions: 1.4.0 Reporter: Wesley L Lawrence Priority: Minor Related to NIFI-4242, if you can't use 'Informix Unload Escape Disabled' as a pre-defined CSV format, because 'Informix Unload' has the same allowable value. The WebUI for CSVRedaer/CSVRecordSetWriter seems to always display 'Informix Unload', and when choosing from the drop down, says 'Informix Unload Escape Delimited'. Given that within CSVUtils, 'Informix Unload' is checked against first, I suspect that's the one that gets chosen. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4215) Avro schemas with records that have a field of themselves fail to parse, causing stackoverflow exception
[
https://issues.apache.org/jira/browse/NIFI-4215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109209#comment-16109209
]
Wesley L Lawrence commented on NIFI-4215:
-
[~markap14] If the 'fields' and 'fieldIndices' are private (but not final) to
'SimpleRecordSchema', and 'fields' is still a 'Collections.unmodifiableList',
and 'fieldIndices' isn't available outside the class, what can be mutated from
outside 'SimpleRecordSchema'? While I agree that ideally they should also be
final, the fact that they aren't mutable outside the 'SimpleRecordSchema', and
that 'SimpleRecordSchema' currently doesn't mutate them (apart from a one-time
'setFields' call, which throws an exception if used a second time) should be
enough to maintain thread safety.
--Wes
> Avro schemas with records that have a field of themselves fail to parse,
> causing stackoverflow exception
>
>
> Key: NIFI-4215
> URL: https://issues.apache.org/jira/browse/NIFI-4215
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Wesley L Lawrence
>Priority: Blocker
> Fix For: 1.4.0
>
> Attachments: nifi-4215.patch
>
>
> Noticed this while attempting to use the AvroSchemaRegsitry with some complex
> schema. Boiled down, Avro lets you define a schema such as;
> {code}
> {
> "namespace": "org.apache.nifi.testing",
> "name": "CompositRecord",
> "type": "record",
> "fields": [
> {
> "name": "id",
> "type": "int"
> },
> {
> "name": "value",
> "type": "string"
> },
> {
> "name": "parent",
> "type": [
> "null",
> "CompositRecord"
> ]
> }
> ]
> }
> {code}
> The AvroSchemaRegistry (AvroTypeUtil specifically) will fail to parse, and
> generate a stackoverflow exception.
> I've whipped up a fix, tested it out in 1.4.0, and am just running through
> the contrib build before I submit a patch.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-4215) Avro schemas with records that have a field of themselves fail to parse, causing stackoverflow exception
[
https://issues.apache.org/jira/browse/NIFI-4215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109145#comment-16109145
]
Wesley L Lawrence commented on NIFI-4215:
-
Right. I didn't go with the Builder the first time, because the
SimpleRecordSchema fields needs a circular reference when a record has a field
of it's same type. They can't be final AFAIK...
I'll see what else I can think of.
--Wes
> Avro schemas with records that have a field of themselves fail to parse,
> causing stackoverflow exception
>
>
> Key: NIFI-4215
> URL: https://issues.apache.org/jira/browse/NIFI-4215
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Wesley L Lawrence
>Priority: Blocker
> Fix For: 1.4.0
>
> Attachments: nifi-4215.patch
>
>
> Noticed this while attempting to use the AvroSchemaRegsitry with some complex
> schema. Boiled down, Avro lets you define a schema such as;
> {code}
> {
> "namespace": "org.apache.nifi.testing",
> "name": "CompositRecord",
> "type": "record",
> "fields": [
> {
> "name": "id",
> "type": "int"
> },
> {
> "name": "value",
> "type": "string"
> },
> {
> "name": "parent",
> "type": [
> "null",
> "CompositRecord"
> ]
> }
> ]
> }
> {code}
> The AvroSchemaRegistry (AvroTypeUtil specifically) will fail to parse, and
> generate a stackoverflow exception.
> I've whipped up a fix, tested it out in 1.4.0, and am just running through
> the contrib build before I submit a patch.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (NIFI-4215) Avro schemas with records that have a field of themselves fail to parse, causing stackoverflow exception
[
https://issues.apache.org/jira/browse/NIFI-4215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109082#comment-16109082
]
Wesley L Lawrence edited comment on NIFI-4215 at 8/1/17 3:30 PM:
-
[~markap14] Thanks for adding a second eye to this code change.
I see there error that was added into AvroTypeUtil (on line 357,
createSchema(...) ). Sorry about that, I'll get that fixed back to 'schemaId'.
I was hesitant to change the immutability of SimleRecordSchema, but there needs
to be a way to create a place holder for the in-progress record parsing, so
that sub-records of the same type don't create a parsing loop.
Looking back with a fresh set of eyes; I could create a
'SimpleRecordSchemaBuilder' that's mutable, and once parsing is complete,
returns the immutable 'SimpleRecordSchema'.
Any other ideas or feedback are welcome, and I'll get around to addressing this
as soon as possible.
--Wes
was (Author: wesleylawrence):
[~markap14] Thanks for adding a second eye to this code change.
I see there error that was added into AvroTypeUtil (on line 357,
createSchema(...) ).
I was hesitant to change the immutability of SimleRecordSchema, but there needs
to be a way to create a place holder for the in-progress record parsing, so
that sub-records of the same type don't create a parsing loop.
Looking back with a fresh set of eyes; I could create a
'SimpleRecordSchemaBuilder' that's mutable, and once parsing is complete,
returns the immutable 'SimpleRecordSchema'.
Any other ideas or feedback are welcome, and I'll get around to addressing this
as soon as possible.
--Wes
> Avro schemas with records that have a field of themselves fail to parse,
> causing stackoverflow exception
>
>
> Key: NIFI-4215
> URL: https://issues.apache.org/jira/browse/NIFI-4215
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Wesley L Lawrence
>Priority: Blocker
> Fix For: 1.4.0
>
> Attachments: nifi-4215.patch
>
>
> Noticed this while attempting to use the AvroSchemaRegsitry with some complex
> schema. Boiled down, Avro lets you define a schema such as;
> {code}
> {
> "namespace": "org.apache.nifi.testing",
> "name": "CompositRecord",
> "type": "record",
> "fields": [
> {
> "name": "id",
> "type": "int"
> },
> {
> "name": "value",
> "type": "string"
> },
> {
> "name": "parent",
> "type": [
> "null",
> "CompositRecord"
> ]
> }
> ]
> }
> {code}
> The AvroSchemaRegistry (AvroTypeUtil specifically) will fail to parse, and
> generate a stackoverflow exception.
> I've whipped up a fix, tested it out in 1.4.0, and am just running through
> the contrib build before I submit a patch.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-4215) Avro schemas with records that have a field of themselves fail to parse, causing stackoverflow exception
[
https://issues.apache.org/jira/browse/NIFI-4215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16109082#comment-16109082
]
Wesley L Lawrence commented on NIFI-4215:
-
[~markap14] Thanks for adding a second eye to this code change.
I see there error that was added into AvroTypeUtil (on line 357,
createSchema(...) ).
I was hesitant to change the immutability of SimleRecordSchema, but there needs
to be a way to create a place holder for the in-progress record parsing, so
that sub-records of the same type don't create a parsing loop.
Looking back with a fresh set of eyes; I could create a
'SimpleRecordSchemaBuilder' that's mutable, and once parsing is complete,
returns the immutable 'SimpleRecordSchema'.
Any other ideas or feedback are welcome, and I'll get around to addressing this
as soon as possible.
--Wes
> Avro schemas with records that have a field of themselves fail to parse,
> causing stackoverflow exception
>
>
> Key: NIFI-4215
> URL: https://issues.apache.org/jira/browse/NIFI-4215
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Wesley L Lawrence
>Priority: Blocker
> Fix For: 1.4.0
>
> Attachments: nifi-4215.patch
>
>
> Noticed this while attempting to use the AvroSchemaRegsitry with some complex
> schema. Boiled down, Avro lets you define a schema such as;
> {code}
> {
> "namespace": "org.apache.nifi.testing",
> "name": "CompositRecord",
> "type": "record",
> "fields": [
> {
> "name": "id",
> "type": "int"
> },
> {
> "name": "value",
> "type": "string"
> },
> {
> "name": "parent",
> "type": [
> "null",
> "CompositRecord"
> ]
> }
> ]
> }
> {code}
> The AvroSchemaRegistry (AvroTypeUtil specifically) will fail to parse, and
> generate a stackoverflow exception.
> I've whipped up a fix, tested it out in 1.4.0, and am just running through
> the contrib build before I submit a patch.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (NIFI-4242) CSVReader shouldn't require that an escape character be defined
Wesley L Lawrence created NIFI-4242:
---
Summary: CSVReader shouldn't require that an escape character be
defined
Key: NIFI-4242
URL: https://issues.apache.org/jira/browse/NIFI-4242
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 1.3.0
Reporter: Wesley L Lawrence
Priority: Minor
There are situations where, when parsing a CSV file, one doesn't want to define
an escape character. For example, when using quote character ", the following
is valid CSV;
{code}
a,"""b",c
{code}
The second column should be interpreted as "b. But when Apache Commons CSV is
told that there's an escape character, the above row is invalid (interestingly,
if it was """b""", it would be valid as "b").
There are known formats that Apache Commons CSV provides, that doesn't define
escape characters either.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-3731) Excessive Curator messages
[ https://issues.apache.org/jira/browse/NIFI-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106632#comment-16106632 ] Wesley L Lawrence commented on NIFI-3731: - Might be something in Apache Curator, not NiFi itself. https://issues.apache.org/jira/browse/CURATOR-272 I've seen these messages spam as well when ZK goes down, or gets into a state where it can't be connected to. > Excessive Curator messages > -- > > Key: NIFI-3731 > URL: https://issues.apache.org/jira/browse/NIFI-3731 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.1.1 >Reporter: Mark Bean >Priority: Minor > > Occasionally, while testing scenarios on a 3-Node cluster a Node will be > removed from the cluster. Sometimes, the log files begin filling with an > inordinate amount of repeated messages from the Curator. I'm still trying to > track down what instigates this, but most likely it is when one of the 3 ZK > servers becomes unavailable. Is there a way to suppress or reduce the > frequency of the following messages? Currently, it is repeated more than > once/millisecond. > 2017-04-19 16:25:09,824 ERROR [Curator-Framework-0] > o.a.c.f.imps.CuratorFrameworkImpl Background retry gave up > org.apache.curator.CuratorConnectionLossException: KeeperErrorCode = > ConnectionLoss > at > org.apache.curator.framework.imps.CuratorFrameworkImpl.performBackgroundOperation(CuratorFramworkImpl.java:838) > [curator-framework-2.11.0.jar:na] > at > org.apache.curator.framework.imps.CuratorFrameworkImpl.backgroundOperationsLoop(CuratorFrameworkImpl.java:809) > [curator-framework-2.11.0.jar:na] > at > org.apache.curator.framework.imps.CuratorFrameworkImpl.access$300(CuratorFrameworkImpl.java:64) > [curator-framework-2.11.0.jar:na] > at > org.apache.curator.framework.imps.CuratorFrameworkImpl.$4.call(CuratorFrameworkImpl.java:267) > [curator-framework-2.11.0.jar:na] > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > [na:1.8.0_121] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) > [na:1.8.0_121] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > [na:1.8.0_121] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [na:1.8.0_121] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [na:1.8.0_121] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4215) Avro schemas with records that have a field of themselves fail to parse, causing stackoverflow exception
[
https://issues.apache.org/jira/browse/NIFI-4215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wesley L Lawrence updated NIFI-4215:
Attachment: nifi-4215.patch
> Avro schemas with records that have a field of themselves fail to parse,
> causing stackoverflow exception
>
>
> Key: NIFI-4215
> URL: https://issues.apache.org/jira/browse/NIFI-4215
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Wesley L Lawrence
>Priority: Minor
> Fix For: 1.4.0
>
> Attachments: nifi-4215.patch
>
>
> Noticed this while attempting to use the AvroSchemaRegsitry with some complex
> schema. Boiled down, Avro lets you define a schema such as;
> {code}
> {
> "namespace": "org.apache.nifi.testing",
> "name": "CompositRecord",
> "type": "record",
> "fields": [
> {
> "name": "id",
> "type": "int"
> },
> {
> "name": "value",
> "type": "string"
> },
> {
> "name": "parent",
> "type": [
> "null",
> "CompositRecord"
> ]
> }
> ]
> }
> {code}
> The AvroSchemaRegistry (AvroTypeUtil specifically) will fail to parse, and
> generate a stackoverflow exception.
> I've whipped up a fix, tested it out in 1.4.0, and am just running through
> the contrib build before I submit a patch.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (NIFI-4215) Avro schemas with records that have a field of themselves fail to parse, causing stackoverflow exception
[
https://issues.apache.org/jira/browse/NIFI-4215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wesley L Lawrence updated NIFI-4215:
Fix Version/s: 1.4.0
Status: Patch Available (was: Open)
> Avro schemas with records that have a field of themselves fail to parse,
> causing stackoverflow exception
>
>
> Key: NIFI-4215
> URL: https://issues.apache.org/jira/browse/NIFI-4215
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Wesley L Lawrence
>Priority: Minor
> Fix For: 1.4.0
>
> Attachments: nifi-4215.patch
>
>
> Noticed this while attempting to use the AvroSchemaRegsitry with some complex
> schema. Boiled down, Avro lets you define a schema such as;
> {code}
> {
> "namespace": "org.apache.nifi.testing",
> "name": "CompositRecord",
> "type": "record",
> "fields": [
> {
> "name": "id",
> "type": "int"
> },
> {
> "name": "value",
> "type": "string"
> },
> {
> "name": "parent",
> "type": [
> "null",
> "CompositRecord"
> ]
> }
> ]
> }
> {code}
> The AvroSchemaRegistry (AvroTypeUtil specifically) will fail to parse, and
> generate a stackoverflow exception.
> I've whipped up a fix, tested it out in 1.4.0, and am just running through
> the contrib build before I submit a patch.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (NIFI-4215) Avro schemas with records that have a field of themselves fail to parse, causing stackoverflow exception
Wesley L Lawrence created NIFI-4215:
---
Summary: Avro schemas with records that have a field of themselves
fail to parse, causing stackoverflow exception
Key: NIFI-4215
URL: https://issues.apache.org/jira/browse/NIFI-4215
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 1.4.0
Reporter: Wesley L Lawrence
Priority: Minor
Noticed this while attempting to use the AvroSchemaRegsitry with some complex
schema. Boiled down, Avro lets you define a schema such as;
{code}
{
"namespace": "org.apache.nifi.testing",
"name": "CompositRecord",
"type": "record",
"fields": [
{
"name": "id",
"type": "int"
},
{
"name": "value",
"type": "string"
},
{
"name": "parent",
"type": [
"null",
"CompositRecord"
]
}
]
}
{code}
The AvroSchemaRegistry (AvroTypeUtil specifically) will fail to parse, and
generate a stackoverflow exception.
I've whipped up a fix, tested it out in 1.4.0, and am just running through the
contrib build before I submit a patch.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (NIFI-4181) CSVReader and CSVRecordSetWriter services should be able to work given an explicit list of columns.
[ https://issues.apache.org/jira/browse/NIFI-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wesley L Lawrence updated NIFI-4181: Attachment: NIFI-4181.patch Some updates based on GitHub PR comments. > CSVReader and CSVRecordSetWriter services should be able to work given an > explicit list of columns. > --- > > Key: NIFI-4181 > URL: https://issues.apache.org/jira/browse/NIFI-4181 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Wesley L Lawrence >Priority: Minor > Attachments: NIFI-4181.patch > > > Currently, to read or write a CSV file with *Record processors, the CSVReader > and CSVRecordSetWriters need to be given an avro schema. For CSV, a simple > column definition can also work. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4181) CSVReader and CSVRecordSetWriter services should be able to work given an explicit list of columns.
[ https://issues.apache.org/jira/browse/NIFI-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wesley L Lawrence updated NIFI-4181: Attachment: (was: NIFI-4181.patch) > CSVReader and CSVRecordSetWriter services should be able to work given an > explicit list of columns. > --- > > Key: NIFI-4181 > URL: https://issues.apache.org/jira/browse/NIFI-4181 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Wesley L Lawrence >Priority: Minor > Attachments: NIFI-4181.patch > > > Currently, to read or write a CSV file with *Record processors, the CSVReader > and CSVRecordSetWriters need to be given an avro schema. For CSV, a simple > column definition can also work. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4181) CSVReader and CSVRecordSetWriter services should be able to work given an explicit list of columns.
[ https://issues.apache.org/jira/browse/NIFI-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wesley L Lawrence updated NIFI-4181: Attachment: NIFI-4181.patch Slight patch update based on running contrib checks. > CSVReader and CSVRecordSetWriter services should be able to work given an > explicit list of columns. > --- > > Key: NIFI-4181 > URL: https://issues.apache.org/jira/browse/NIFI-4181 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Wesley L Lawrence >Priority: Minor > Attachments: NIFI-4181.patch > > > Currently, to read or write a CSV file with *Record processors, the CSVReader > and CSVRecordSetWriters need to be given an avro schema. For CSV, a simple > column definition can also work. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4181) CSVReader and CSVRecordSetWriter services should be able to work given an explicit list of columns.
[ https://issues.apache.org/jira/browse/NIFI-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wesley L Lawrence updated NIFI-4181: Attachment: (was: NIFI-4181.patch) > CSVReader and CSVRecordSetWriter services should be able to work given an > explicit list of columns. > --- > > Key: NIFI-4181 > URL: https://issues.apache.org/jira/browse/NIFI-4181 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Wesley L Lawrence >Priority: Minor > Attachments: NIFI-4181.patch > > > Currently, to read or write a CSV file with *Record processors, the CSVReader > and CSVRecordSetWriters need to be given an avro schema. For CSV, a simple > column definition can also work. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (NIFI-4181) CSVReader and CSVRecordSetWriter services should be able to work given an explicit list of columns.
Wesley L Lawrence created NIFI-4181: --- Summary: CSVReader and CSVRecordSetWriter services should be able to work given an explicit list of columns. Key: NIFI-4181 URL: https://issues.apache.org/jira/browse/NIFI-4181 Project: Apache NiFi Issue Type: Improvement Reporter: Wesley L Lawrence Priority: Minor Currently, to read or write a CSV file with *Record processors, the CSVReader and CSVRecordSetWriters need to be given an avro schema. For CSV, a simple column definition can also work. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4181) CSVReader and CSVRecordSetWriter services should be able to work given an explicit list of columns.
[ https://issues.apache.org/jira/browse/NIFI-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wesley L Lawrence updated NIFI-4181: Status: Patch Available (was: Open) GH PR coming too, if that's easier to review. > CSVReader and CSVRecordSetWriter services should be able to work given an > explicit list of columns. > --- > > Key: NIFI-4181 > URL: https://issues.apache.org/jira/browse/NIFI-4181 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Wesley L Lawrence >Priority: Minor > Attachments: NIFI-4181.patch > > > Currently, to read or write a CSV file with *Record processors, the CSVReader > and CSVRecordSetWriters need to be given an avro schema. For CSV, a simple > column definition can also work. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-3503) Create a 'SplitCSV' processor
[ https://issues.apache.org/jira/browse/NIFI-3503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16068480#comment-16068480 ] Wesley L Lawrence commented on NIFI-3503: - I've been using the SplitRecord processor, and I also think it's sufficient. > Create a 'SplitCSV' processor > - > > Key: NIFI-3503 > URL: https://issues.apache.org/jira/browse/NIFI-3503 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Wesley L Lawrence >Priority: Minor > > While the 'SplitText' processor helps break up newline separated records into > individual files, it's not uncommon to have CSV files where records span > multiple lines, and 'SplitText' isn't able or meant to handle this. > Currently, one can replace, remove, or escape newline characters that exist > in a single CSV record by searching within quoted columns with 'ReplaceText', > before passing the data onto 'SplitText'. However, this may not work in all > cases, or could potentially remove the valid newline character at the end of > a CSV record, if all edge cases aren't properly covered with regex. > Having a dedicated 'SplitCSV' processor will solve this problem, and be a > simpler approach for users. > See the following [Apache NiFi user email > thread|https://mail-archives.apache.org/mod_mbox/nifi-users/201702.mbox/%3CCAFuL2BbgymFXwu5fRyd8pP-zu6WkToqPE2Ek7bkyBg0_-cknqQ%40mail.gmail.com%3E] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (NIFI-4064) Funnel with no outgoing connections and a queued Flow File uses high CPU
[ https://issues.apache.org/jira/browse/NIFI-4064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048137#comment-16048137 ] Wesley L Lawrence edited comment on NIFI-4064 at 6/13/17 5:25 PM: -- PR submitted through GitHub, but patch attached also. [^nifi-4064.patch] was (Author: wesleylawrence): PR submitted through GitHub, but patch attached also. > Funnel with no outgoing connections and a queued Flow File uses high CPU > > > Key: NIFI-4064 > URL: https://issues.apache.org/jira/browse/NIFI-4064 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.3.0 >Reporter: Wesley L Lawrence >Assignee: Wesley L Lawrence >Priority: Minor > Fix For: 1.4.0 > > Attachments: FlowWithOneFileInQueue.png, nifi-4064.patch, > NifiPostEmptyLowCpu.png, NifiPreEmptyHighCpu.png, WithFixPrePostCpu.png > > > I have a simple Flow where a file containing JSON data is read, and converted > to Avro. > !FlowWithOneFileInQueue.png! > However, I have the resulting Avro container file parked in a queue going to > a Funnel, and it causes my CPU to spike. > !NifiPreEmptyHighCpu.png! > Emptying the queue causes the CPU to return to normal. > !NifiPostEmptyLowCpu.png! > It seems that in the 'ContinuallyRunConnectableTask' class, it does correctly > determine that the Funnel shouldn't be run, but since there is both a > FlowFile in queue, and a relaiton, it also decides to not yield. > I've added both a fix for this (patch to follow), as well as a new unit test > to make sure Funnels without outgoing connections get yielded. With the fix, > the CPU spiking is gone. > !WithFixPrePostCpu.png! -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4064) Funnel with no outgoing connections and a queued Flow File uses high CPU
[ https://issues.apache.org/jira/browse/NIFI-4064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wesley L Lawrence updated NIFI-4064: Fix Version/s: 1.4.0 Status: Patch Available (was: In Progress) PR submitted through GitHub, but patch attached also. > Funnel with no outgoing connections and a queued Flow File uses high CPU > > > Key: NIFI-4064 > URL: https://issues.apache.org/jira/browse/NIFI-4064 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.3.0 >Reporter: Wesley L Lawrence >Assignee: Wesley L Lawrence >Priority: Minor > Fix For: 1.4.0 > > Attachments: FlowWithOneFileInQueue.png, nifi-4064.patch, > NifiPostEmptyLowCpu.png, NifiPreEmptyHighCpu.png, WithFixPrePostCpu.png > > > I have a simple Flow where a file containing JSON data is read, and converted > to Avro. > !FlowWithOneFileInQueue.png! > However, I have the resulting Avro container file parked in a queue going to > a Funnel, and it causes my CPU to spike. > !NifiPreEmptyHighCpu.png! > Emptying the queue causes the CPU to return to normal. > !NifiPostEmptyLowCpu.png! > It seems that in the 'ContinuallyRunConnectableTask' class, it does correctly > determine that the Funnel shouldn't be run, but since there is both a > FlowFile in queue, and a relaiton, it also decides to not yield. > I've added both a fix for this (patch to follow), as well as a new unit test > to make sure Funnels without outgoing connections get yielded. With the fix, > the CPU spiking is gone. > !WithFixPrePostCpu.png! -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (NIFI-4064) Funnel with no outgoing connections and a queued Flow File uses high CPU
[ https://issues.apache.org/jira/browse/NIFI-4064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wesley L Lawrence reassigned NIFI-4064: --- Assignee: Wesley L Lawrence > Funnel with no outgoing connections and a queued Flow File uses high CPU > > > Key: NIFI-4064 > URL: https://issues.apache.org/jira/browse/NIFI-4064 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.3.0 >Reporter: Wesley L Lawrence >Assignee: Wesley L Lawrence >Priority: Minor > Attachments: FlowWithOneFileInQueue.png, nifi-4064.patch, > NifiPostEmptyLowCpu.png, NifiPreEmptyHighCpu.png, WithFixPrePostCpu.png > > > I have a simple Flow where a file containing JSON data is read, and converted > to Avro. > !FlowWithOneFileInQueue.png! > However, I have the resulting Avro container file parked in a queue going to > a Funnel, and it causes my CPU to spike. > !NifiPreEmptyHighCpu.png! > Emptying the queue causes the CPU to return to normal. > !NifiPostEmptyLowCpu.png! > It seems that in the 'ContinuallyRunConnectableTask' class, it does correctly > determine that the Funnel shouldn't be run, but since there is both a > FlowFile in queue, and a relaiton, it also decides to not yield. > I've added both a fix for this (patch to follow), as well as a new unit test > to make sure Funnels without outgoing connections get yielded. With the fix, > the CPU spiking is gone. > !WithFixPrePostCpu.png! -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4064) Funnel with no outgoing connections and a queued Flow File uses high CPU
[ https://issues.apache.org/jira/browse/NIFI-4064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wesley L Lawrence updated NIFI-4064: Attachment: nifi-4064.patch > Funnel with no outgoing connections and a queued Flow File uses high CPU > > > Key: NIFI-4064 > URL: https://issues.apache.org/jira/browse/NIFI-4064 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.3.0 >Reporter: Wesley L Lawrence >Priority: Minor > Attachments: FlowWithOneFileInQueue.png, nifi-4064.patch, > NifiPostEmptyLowCpu.png, NifiPreEmptyHighCpu.png, WithFixPrePostCpu.png > > > I have a simple Flow where a file containing JSON data is read, and converted > to Avro. > !FlowWithOneFileInQueue.png! > However, I have the resulting Avro container file parked in a queue going to > a Funnel, and it causes my CPU to spike. > !NifiPreEmptyHighCpu.png! > Emptying the queue causes the CPU to return to normal. > !NifiPostEmptyLowCpu.png! > It seems that in the 'ContinuallyRunConnectableTask' class, it does correctly > determine that the Funnel shouldn't be run, but since there is both a > FlowFile in queue, and a relaiton, it also decides to not yield. > I've added both a fix for this (patch to follow), as well as a new unit test > to make sure Funnels without outgoing connections get yielded. With the fix, > the CPU spiking is gone. > !WithFixPrePostCpu.png! -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (NIFI-4064) Funnel with no outgoing connections and a queued Flow File uses high CPU
[ https://issues.apache.org/jira/browse/NIFI-4064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wesley L Lawrence updated NIFI-4064: Description: I have a simple Flow where a file containing JSON data is read, and converted to Avro. !FlowWithOneFileInQueue.png! However, I have the resulting Avro container file parked in a queue going to a Funnel, and it causes my CPU to spike. !NifiPreEmptyHighCpu.png! Emptying the queue causes the CPU to return to normal. !NifiPostEmptyLowCpu.png! It seems that in the 'ContinuallyRunConnectableTask' class, it does correctly determine that the Funnel shouldn't be run, but since there is both a FlowFile in queue, and a relaiton, it also decides to not yield. I've added both a fix for this (patch to follow), as well as a new unit test to make sure Funnels without outgoing connections get yielded. With the fix, the CPU spiking is gone. !WithFixPrePostCpu.png! was: I have a simple Flow where a file containing JSON data is read, and converted to Avro. !FlowWithOneFileInQueue.png|thumbnail! However, I have the resulting Avro container file parked in a queue going to a Funnel, and it causes my CPU to spike. !NifiPreEmptyHighCpu.png! Emptying the queue causes the CPU to return to normal. !NifiPostEmptyLowCpu.png! It seems that in the 'ContinuallyRunConnectableTask' class, it does correctly determine that the Funnel shouldn't be run, but since there is both a FlowFile in queue, and a relaiton, it also decides to not yield. I've added both a fix for this (patch to follow), as well as a new unit test to make sure Funnels without outgoing connections get yielded. With the fix, the CPU spiking is gone. !WithFixPrePostCpu.png! > Funnel with no outgoing connections and a queued Flow File uses high CPU > > > Key: NIFI-4064 > URL: https://issues.apache.org/jira/browse/NIFI-4064 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.3.0 >Reporter: Wesley L Lawrence >Priority: Minor > Attachments: FlowWithOneFileInQueue.png, NifiPostEmptyLowCpu.png, > NifiPreEmptyHighCpu.png, WithFixPrePostCpu.png > > > I have a simple Flow where a file containing JSON data is read, and converted > to Avro. > !FlowWithOneFileInQueue.png! > However, I have the resulting Avro container file parked in a queue going to > a Funnel, and it causes my CPU to spike. > !NifiPreEmptyHighCpu.png! > Emptying the queue causes the CPU to return to normal. > !NifiPostEmptyLowCpu.png! > It seems that in the 'ContinuallyRunConnectableTask' class, it does correctly > determine that the Funnel shouldn't be run, but since there is both a > FlowFile in queue, and a relaiton, it also decides to not yield. > I've added both a fix for this (patch to follow), as well as a new unit test > to make sure Funnels without outgoing connections get yielded. With the fix, > the CPU spiking is gone. > !WithFixPrePostCpu.png! -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (NIFI-4064) Funnel with no outgoing connections and a queued Flow File uses high CPU
Wesley L Lawrence created NIFI-4064: --- Summary: Funnel with no outgoing connections and a queued Flow File uses high CPU Key: NIFI-4064 URL: https://issues.apache.org/jira/browse/NIFI-4064 Project: Apache NiFi Issue Type: Bug Affects Versions: 1.3.0 Reporter: Wesley L Lawrence Priority: Minor Attachments: FlowWithOneFileInQueue.png, NifiPostEmptyLowCpu.png, NifiPreEmptyHighCpu.png, WithFixPrePostCpu.png I have a simple Flow where a file containing JSON data is read, and converted to Avro. !FlowWithOneFileInQueue.png|thumbnail! However, I have the resulting Avro container file parked in a queue going to a Funnel, and it causes my CPU to spike. !NifiPreEmptyHighCpu.png! Emptying the queue causes the CPU to return to normal. !NifiPostEmptyLowCpu.png! It seems that in the 'ContinuallyRunConnectableTask' class, it does correctly determine that the Funnel shouldn't be run, but since there is both a FlowFile in queue, and a relaiton, it also decides to not yield. I've added both a fix for this (patch to follow), as well as a new unit test to make sure Funnels without outgoing connections get yielded. With the fix, the CPU spiking is gone. !WithFixPrePostCpu.png! -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (NIFI-3503) Create a 'SplitCSV' processor
Wesley L Lawrence created NIFI-3503: --- Summary: Create a 'SplitCSV' processor Key: NIFI-3503 URL: https://issues.apache.org/jira/browse/NIFI-3503 Project: Apache NiFi Issue Type: New Feature Reporter: Wesley L Lawrence Priority: Minor While the 'SplitText' processor helps break up newline separated records into individual files, it's not uncommon to have CSV files where records span multiple lines, and 'SplitText' isn't able or meant to handle this. Currently, one can replace, remove, or escape newline characters that exist in a single CSV record by searching within quoted columns with 'ReplaceText', before passing the data onto 'SplitText'. However, this may not work in all cases, or could potentially remove the valid newline character at the end of a CSV record, if all edge cases aren't properly covered with regex. Having a dedicated 'SplitCSV' processor will solve this problem, and be a simpler approach for users. See the following [Apache NiFi user email thread|https://mail-archives.apache.org/mod_mbox/nifi-users/201702.mbox/%3CCAFuL2BbgymFXwu5fRyd8pP-zu6WkToqPE2Ek7bkyBg0_-cknqQ%40mail.gmail.com%3E] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (NIFI-3175) ValidateCSV processor throws NullPointerException when printing an empty CSV column
[
https://issues.apache.org/jira/browse/NIFI-3175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15733728#comment-15733728
]
Wesley L Lawrence edited comment on NIFI-3175 at 12/9/16 12:03 AM:
---
Fix available at; https://github.com/apache/nifi/pull/1311
{code}
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ Y ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ Y ] Does your PR title start with NIFI- where is the JIRA number
you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [ Y ] Has your PR been rebased against the latest commit within the target
branch (typically master)?
- [ Y ] Is your initial contribution a single, squashed commit?
### For code changes:
- [ Y ] Have you ensured that the full suite of tests is executed via mvn
-Pcontrib-check clean install at the root nifi folder?
- [ Y ] Have you written or updated unit tests to verify your changes?
- [N/A] If adding new dependencies to the code, are these dependencies licensed
in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [N/A] If applicable, have you updated the LICENSE file, including the main
LICENSE file under nifi-assembly?
- [N/A] If applicable, have you updated the NOTICE file, including the main
NOTICE file found under nifi-assembly?
- [N/A] If adding new Properties, have you added .displayName in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [N/A] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
{code}
was (Author: wesleylawrence):
Fix available at; https://github.com/Wesley-Lawrence/nifi/tree/NIFI-3175.
Reading through [Contributor
Guide|https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide] still.
> ValidateCSV processor throws NullPointerException when printing an empty CSV
> column
> ---
>
> Key: NIFI-3175
> URL: https://issues.apache.org/jira/browse/NIFI-3175
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Wesley L Lawrence
>Priority: Minor
>
> For example, given a CSV file with the line;
> {code}
> Hello,,World
> {code}
> And the ValidateCSV processor is given the Super-CSV schema of;
> {code}
> Null,Null,Null
> {code}
> And exception such as this will be thrown;
> {code}
> 2016-12-08 16:58:27,061 ERROR [Timer-Driven Process Thread-9]
> o.a.nifi.processors.standard.ValidateCsv
> ValidateCsv[id=e05389fa-0158-1000-75dc-d6a79554c96a]
> ValidateCsv[id=e05389fa-0158-1000-75dc-d6a7955
> 4c96a] failed to process due to java.lang.NullPointerException; rolling back
> session: java.lang.NullPointerException
> 2016-12-08 16:58:27,062 ERROR [Timer-Driven Process Thread-9]
> o.a.nifi.processors.standard.ValidateCsv
> java.lang.NullPointerException: null
> at
> org.apache.nifi.processors.standard.ValidateCsv.print(ValidateCsv.java:584)
> ~[nifi-standard-processors-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.processors.standard.ValidateCsv.access$000(ValidateCsv.java:89)
> ~[nifi-standard-processors-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.processors.standard.ValidateCsv$1$3.process(ValidateCsv.java:484)
> ~[nifi-standard-processors-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.append(StandardProcessSession.java:2412)
> ~[nifi-framework-core-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.processors.standard.ValidateCsv$1.process(ValidateCsv.java:481)
> ~[nifi-standard-processors-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2082)
> ~[nifi-framework-core-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2053)
> ~[nifi-framework-core-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.processors.standard.ValidateCsv.onTrigger(ValidateCsv.java:444)
> ~[nifi-standard-processors-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> ~[nifi-api-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1099)
> [nifi-framework-core-1.1.0.jar:1.1.0]
> at
>
[jira] [Comment Edited] (NIFI-3175) ValidateCSV processor throws NullPointerException when printing an empty CSV column
[
https://issues.apache.org/jira/browse/NIFI-3175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15733728#comment-15733728
]
Wesley L Lawrence edited comment on NIFI-3175 at 12/9/16 12:04 AM:
---
Fix available at; https://github.com/apache/nifi/pull/1311
was (Author: wesleylawrence):
Fix available at; https://github.com/apache/nifi/pull/1311
{code}
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ Y ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ Y ] Does your PR title start with NIFI- where is the JIRA number
you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [ Y ] Has your PR been rebased against the latest commit within the target
branch (typically master)?
- [ Y ] Is your initial contribution a single, squashed commit?
### For code changes:
- [ Y ] Have you ensured that the full suite of tests is executed via mvn
-Pcontrib-check clean install at the root nifi folder?
- [ Y ] Have you written or updated unit tests to verify your changes?
- [N/A] If adding new dependencies to the code, are these dependencies licensed
in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [N/A] If applicable, have you updated the LICENSE file, including the main
LICENSE file under nifi-assembly?
- [N/A] If applicable, have you updated the NOTICE file, including the main
NOTICE file found under nifi-assembly?
- [N/A] If adding new Properties, have you added .displayName in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [N/A] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
{code}
> ValidateCSV processor throws NullPointerException when printing an empty CSV
> column
> ---
>
> Key: NIFI-3175
> URL: https://issues.apache.org/jira/browse/NIFI-3175
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Wesley L Lawrence
>Priority: Minor
>
> For example, given a CSV file with the line;
> {code}
> Hello,,World
> {code}
> And the ValidateCSV processor is given the Super-CSV schema of;
> {code}
> Null,Null,Null
> {code}
> And exception such as this will be thrown;
> {code}
> 2016-12-08 16:58:27,061 ERROR [Timer-Driven Process Thread-9]
> o.a.nifi.processors.standard.ValidateCsv
> ValidateCsv[id=e05389fa-0158-1000-75dc-d6a79554c96a]
> ValidateCsv[id=e05389fa-0158-1000-75dc-d6a7955
> 4c96a] failed to process due to java.lang.NullPointerException; rolling back
> session: java.lang.NullPointerException
> 2016-12-08 16:58:27,062 ERROR [Timer-Driven Process Thread-9]
> o.a.nifi.processors.standard.ValidateCsv
> java.lang.NullPointerException: null
> at
> org.apache.nifi.processors.standard.ValidateCsv.print(ValidateCsv.java:584)
> ~[nifi-standard-processors-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.processors.standard.ValidateCsv.access$000(ValidateCsv.java:89)
> ~[nifi-standard-processors-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.processors.standard.ValidateCsv$1$3.process(ValidateCsv.java:484)
> ~[nifi-standard-processors-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.append(StandardProcessSession.java:2412)
> ~[nifi-framework-core-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.processors.standard.ValidateCsv$1.process(ValidateCsv.java:481)
> ~[nifi-standard-processors-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2082)
> ~[nifi-framework-core-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2053)
> ~[nifi-framework-core-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.processors.standard.ValidateCsv.onTrigger(ValidateCsv.java:444)
> ~[nifi-standard-processors-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> ~[nifi-api-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1099)
> [nifi-framework-core-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
> [nifi-framework-core-1.1.0.jar:1.1.0]
> at
>
[jira] [Commented] (NIFI-3175) ValidateCSV processor throws NullPointerException when printing an empty CSV column
[
https://issues.apache.org/jira/browse/NIFI-3175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15733728#comment-15733728
]
Wesley L Lawrence commented on NIFI-3175:
-
Fix available at; https://github.com/Wesley-Lawrence/nifi/tree/NIFI-3175.
Reading through [Contributor
Guide|https://cwiki.apache.org/confluence/display/NIFI/Contributor+Guide] still.
> ValidateCSV processor throws NullPointerException when printing an empty CSV
> column
> ---
>
> Key: NIFI-3175
> URL: https://issues.apache.org/jira/browse/NIFI-3175
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Wesley L Lawrence
>Priority: Minor
>
> For example, given a CSV file with the line;
> {code}
> Hello,,World
> {code}
> And the ValidateCSV processor is given the Super-CSV schema of;
> {code}
> Null,Null,Null
> {code}
> And exception such as this will be thrown;
> {code}
> 2016-12-08 16:58:27,061 ERROR [Timer-Driven Process Thread-9]
> o.a.nifi.processors.standard.ValidateCsv
> ValidateCsv[id=e05389fa-0158-1000-75dc-d6a79554c96a]
> ValidateCsv[id=e05389fa-0158-1000-75dc-d6a7955
> 4c96a] failed to process due to java.lang.NullPointerException; rolling back
> session: java.lang.NullPointerException
> 2016-12-08 16:58:27,062 ERROR [Timer-Driven Process Thread-9]
> o.a.nifi.processors.standard.ValidateCsv
> java.lang.NullPointerException: null
> at
> org.apache.nifi.processors.standard.ValidateCsv.print(ValidateCsv.java:584)
> ~[nifi-standard-processors-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.processors.standard.ValidateCsv.access$000(ValidateCsv.java:89)
> ~[nifi-standard-processors-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.processors.standard.ValidateCsv$1$3.process(ValidateCsv.java:484)
> ~[nifi-standard-processors-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.append(StandardProcessSession.java:2412)
> ~[nifi-framework-core-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.processors.standard.ValidateCsv$1.process(ValidateCsv.java:481)
> ~[nifi-standard-processors-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2082)
> ~[nifi-framework-core-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2053)
> ~[nifi-framework-core-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.processors.standard.ValidateCsv.onTrigger(ValidateCsv.java:444)
> ~[nifi-standard-processors-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> ~[nifi-api-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1099)
> [nifi-framework-core-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
> [nifi-framework-core-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
> [nifi-framework-core-1.1.0.jar:1.1.0]
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
> [nifi-framework-core-1.1.0.jar:1.1.0]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_60]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> [na:1.8.0_60]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> [na:1.8.0_60]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> [na:1.8.0_60]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_60]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_60]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (NIFI-3175) ValidateCSV processor throws NullPointerException when printing an empty CSV column
Wesley L Lawrence created NIFI-3175:
---
Summary: ValidateCSV processor throws NullPointerException when
printing an empty CSV column
Key: NIFI-3175
URL: https://issues.apache.org/jira/browse/NIFI-3175
Project: Apache NiFi
Issue Type: Bug
Reporter: Wesley L Lawrence
Priority: Minor
For example, given a CSV file with the line;
{code}
Hello,,World
{code}
And the ValidateCSV processor is given the Super-CSV schema of;
{code}
Null,Null,Null
{code}
And exception such as this will be thrown;
{code}
2016-12-08 16:58:27,061 ERROR [Timer-Driven Process Thread-9]
o.a.nifi.processors.standard.ValidateCsv
ValidateCsv[id=e05389fa-0158-1000-75dc-d6a79554c96a]
ValidateCsv[id=e05389fa-0158-1000-75dc-d6a7955
4c96a] failed to process due to java.lang.NullPointerException; rolling back
session: java.lang.NullPointerException
2016-12-08 16:58:27,062 ERROR [Timer-Driven Process Thread-9]
o.a.nifi.processors.standard.ValidateCsv
java.lang.NullPointerException: null
at
org.apache.nifi.processors.standard.ValidateCsv.print(ValidateCsv.java:584)
~[nifi-standard-processors-1.1.0.jar:1.1.0]
at
org.apache.nifi.processors.standard.ValidateCsv.access$000(ValidateCsv.java:89)
~[nifi-standard-processors-1.1.0.jar:1.1.0]
at
org.apache.nifi.processors.standard.ValidateCsv$1$3.process(ValidateCsv.java:484)
~[nifi-standard-processors-1.1.0.jar:1.1.0]
at
org.apache.nifi.controller.repository.StandardProcessSession.append(StandardProcessSession.java:2412)
~[nifi-framework-core-1.1.0.jar:1.1.0]
at
org.apache.nifi.processors.standard.ValidateCsv$1.process(ValidateCsv.java:481)
~[nifi-standard-processors-1.1.0.jar:1.1.0]
at
org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2082)
~[nifi-framework-core-1.1.0.jar:1.1.0]
at
org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2053)
~[nifi-framework-core-1.1.0.jar:1.1.0]
at
org.apache.nifi.processors.standard.ValidateCsv.onTrigger(ValidateCsv.java:444)
~[nifi-standard-processors-1.1.0.jar:1.1.0]
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
~[nifi-api-1.1.0.jar:1.1.0]
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1099)
[nifi-framework-core-1.1.0.jar:1.1.0]
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
[nifi-framework-core-1.1.0.jar:1.1.0]
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
[nifi-framework-core-1.1.0.jar:1.1.0]
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
[nifi-framework-core-1.1.0.jar:1.1.0]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_60]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
[na:1.8.0_60]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
[na:1.8.0_60]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
[na:1.8.0_60]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_60]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_60]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
