Re: [VOTE] Release Apache NiFi 1.4.0 (RC2)

2017-10-02 Thread Joe Gresock
I have to change my non-binding +1 vote to a 0, unless there is a
workaround for the bug [1] I just submitted, regarding the QueryRecord
processor being broken.


[1] https://issues.apache.org/jira/browse/NIFI-4452


On Mon, Oct 2, 2017 at 3:23 AM, Kevin Doran  wrote:

> +1, non-binding
>
> - Verified hashes
> - Full build with contrib-check passes
> - Was able to run nifi server and send flow files to remote input port
> over s2s protocol. Web UI and API working as expected.
>
> Platform:
> Apache Maven 3.5.0 (ff8f5e7444045639af65f6095c62210b5713f426;
> 2017-04-03T15:39:06-04:00)
> Java version: 1.8.0_131, vendor: Oracle Corporation
> OS name: "mac os x", version: "10.12.6", arch: "x86_64", family: "mac"
>
> Looks good! Nice work everyone!
>
> Kevin
>
> On 9/28/17, 15:54, "Jeff"  wrote:
>
> Hello,
>
> I am pleased to be calling this vote for the source release of Apache
> NiFi
> nifi-1.4.0.
>
> The source zip, including signatures, digests, etc. can be found at:
> https://repository.apache.org/content/repositories/orgapachenifi-
>
> The Git tag is nifi-1.4.0-RC2
> The Git commit ID is e6508ba7d3da5bba54abd6233a7a8f9dd4c32151
> https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=
> e6508ba7d3da5bba54abd6233a7a8f9dd4c32151
>
> Checksums of nifi-1.4.0-source-release.zip:
> MD5: 41e4083e602883a3e180032f32913414
> SHA1: 26770625138126f45bed4989adb0a6b65a767aa2
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/jstorck.asc
>
> KEYS file available here:
> https://dist.apache.org/repos/dist/release/nifi/KEYS
>
> 199 issues were closed/resolved for this release:
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12316020&version=12340589
>
> Release note highlights can be found here:
> https://cwiki.apache.org/confluence/display/NIFI/
> Release+Notes#ReleaseNotes-Version-1.4.0
>
> The vote will be open for 72 hours.
> Please download the release candidate and evaluate the necessary items
> including checking hashes, signatures, build
> from source, and test.  The please vote:
>
> [ ] +1 Release this package as nifi-1.4.0
> [ ] +0 no opinion
> [ ] -1 Do not release this package because...
>
>
>
>


-- 
I know what it is to be in need, and I know what it is to have plenty.  I
have learned the secret of being content in any and every situation,
whether well fed or hungry, whether living in plenty or in want.  I can do
all this through him who gives me strength.*-Philippians 4:12-13*


[RESULT][VOTE] Release Apache NiFi 1.4.0

2017-10-02 Thread Jeff
Apache NiFi Community,

I am pleased to announce that the 1.4.0 release of Apache NiFi passes with
  10 +1 (binding) votes
  9 +1 (non-binding) votes
  1 0 votes
  0 -1 votes

Thanks to all who helped make this release possible.

Here is the PMC vote thread:
http://mail-archives.apache.org/mod_mbox/nifi-dev/201709.mbox/%3ccaavbywuu7kzxnxo3in+mm5jua1xl+lb80dd6mcrorehufzq...@mail.gmail.com%3E


Re: [VOTE] Release Apache NiFi 1.4.0 (RC2)

2017-10-02 Thread Mark Payne
Hey Joe,

I've seen this error crop up a time or two before. The issue is a bug in how we 
are handling a failure
condition. Haven't yet figured out the best way to tackle it, but the thing to 
note is that this error occurs
when there is some other sort of failure -- typically, a failure to obtain the 
schema for the Record Reader
or Record Writer. Can you check your logs for any other errors that may be 
occurring?

Thanks
-Mark


> On Oct 2, 2017, at 11:32 AM, Joe Gresock  wrote:
> 
> I have to change my non-binding +1 vote to a 0, unless there is a
> workaround for the bug [1] I just submitted, regarding the QueryRecord
> processor being broken.
> 
> 
> [1] https://issues.apache.org/jira/browse/NIFI-4452
> 
> 
> On Mon, Oct 2, 2017 at 3:23 AM, Kevin Doran  wrote:
> 
>> +1, non-binding
>> 
>> - Verified hashes
>> - Full build with contrib-check passes
>> - Was able to run nifi server and send flow files to remote input port
>> over s2s protocol. Web UI and API working as expected.
>> 
>> Platform:
>> Apache Maven 3.5.0 (ff8f5e7444045639af65f6095c62210b5713f426;
>> 2017-04-03T15:39:06-04:00)
>> Java version: 1.8.0_131, vendor: Oracle Corporation
>> OS name: "mac os x", version: "10.12.6", arch: "x86_64", family: "mac"
>> 
>> Looks good! Nice work everyone!
>> 
>> Kevin
>> 
>> On 9/28/17, 15:54, "Jeff"  wrote:
>> 
>>Hello,
>> 
>>I am pleased to be calling this vote for the source release of Apache
>> NiFi
>>nifi-1.4.0.
>> 
>>The source zip, including signatures, digests, etc. can be found at:
>>https://repository.apache.org/content/repositories/orgapachenifi-
>> 
>>The Git tag is nifi-1.4.0-RC2
>>The Git commit ID is e6508ba7d3da5bba54abd6233a7a8f9dd4c32151
>>https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=
>> e6508ba7d3da5bba54abd6233a7a8f9dd4c32151
>> 
>>Checksums of nifi-1.4.0-source-release.zip:
>>MD5: 41e4083e602883a3e180032f32913414
>>SHA1: 26770625138126f45bed4989adb0a6b65a767aa2
>> 
>>Release artifacts are signed with the following key:
>>https://people.apache.org/keys/committer/jstorck.asc
>> 
>>KEYS file available here:
>>https://dist.apache.org/repos/dist/release/nifi/KEYS
>> 
>>199 issues were closed/resolved for this release:
>>https://issues.apache.org/jira/secure/ReleaseNote.jspa?
>> projectId=12316020&version=12340589
>> 
>>Release note highlights can be found here:
>>https://cwiki.apache.org/confluence/display/NIFI/
>> Release+Notes#ReleaseNotes-Version-1.4.0
>> 
>>The vote will be open for 72 hours.
>>Please download the release candidate and evaluate the necessary items
>>including checking hashes, signatures, build
>>from source, and test.  The please vote:
>> 
>>[ ] +1 Release this package as nifi-1.4.0
>>[ ] +0 no opinion
>>[ ] -1 Do not release this package because...
>> 
>> 
>> 
>> 
> 
> 
> -- 
> I know what it is to be in need, and I know what it is to have plenty.  I
> have learned the secret of being content in any and every situation,
> whether well fed or hungry, whether living in plenty or in want.  I can do
> all this through him who gives me strength.*-Philippians 4:12-13*



QueryRecord error

2017-10-02 Thread Joe Gresock
I'm changing the subject name so as not to muddy the [VOTE] thread.

I finally found the error in the logs.  This was also not happening prior
to the upgrade.  I thinkI'll debug the server to determine the root cause:

2017-10-02 16:09:47,004 ERROR [Timer-Driven Process Thread-81]
o.a.nifi.processors.standard.QueryRecord
QueryRecord[id=21570e18-015c-1000--58832883] Unable to query
StandardFlowFileRecord[uuid=2da6d0f9-3856-420e-99bd-9589f7f74911,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1506960462126-919,
container=default, section=919], offset=55116,
length=616757],offset=0,name=billing-detailed-line-items-with-resources-and-tags-2017-10.csv.avro,size=616757]
due to org.apache.nifi.processor.exception.ProcessException: Failed to read
next record in stream for
StandardFlowFileRecord[uuid=2da6d0f9-3856-420e-99bd-9589f7f74911,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1506960462126-919,
container=default, section=919], offset=55116,
length=616757],offset=0,name=billing-detailed-line-items-with-resources-and-tags-2017-10.csv.avro,size=616757]:
{}
org.apache.nifi.processor.exception.ProcessException: Failed to read next
record in stream for
StandardFlowFileRecord[uuid=2da6d0f9-3856-420e-99bd-9589f7f74911,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1506960462126-919,
container=default, section=919], offset=55116,
length=616757],offset=0,name=billing-detailed-line-items-with-resources-and-tags-2017-10.csv.avro,size=616757]
at
org.apache.nifi.queryrecord.FlowFileEnumerator.moveNext(FlowFileEnumerator.java:65)
at Baz$1$1.moveNext(Unknown Source)
at
org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.(Linq4j.java:664)
at
org.apache.calcite.linq4j.Linq4j.enumeratorIterator(Linq4j.java:98)
at
org.apache.calcite.linq4j.AbstractEnumerable.iterator(AbstractEnumerable.java:33)
at
org.apache.calcite.avatica.MetaImpl.createCursor(MetaImpl.java:89)
at
org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:196)
at
org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:67)
at
org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:44)
at
org.apache.calcite.avatica.AvaticaConnection.executeQueryInternal(AvaticaConnection.java:513)
at
org.apache.calcite.avatica.AvaticaPreparedStatement.executeQuery(AvaticaPreparedStatement.java:132)
at
org.apache.nifi.processors.standard.QueryRecord.queryWithCache(QueryRecord.java:445)
at
org.apache.nifi.processors.standard.QueryRecord.onTrigger(QueryRecord.java:276)
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1119)
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)


On Mon, Oct 2, 2017 at 4:03 PM, Mark Payne  wrote:

> Hey Joe,
>
> I've seen this error crop up a time or two before. The issue is a bug in
> how we are handling a failure
> condition. Haven't yet figured out the best way to tackle it, but the
> thing to note is that this error occurs
> when there is some other sort of failure -- typically, a failure to obtain
> the schema for the Record Reader
> or Record Writer. Can you check your logs for any other errors that may be
> occurring?
>
> Thanks
> -Mark
>
>
> > On Oct 2, 2017, at 11:32 AM, Joe Gresock  wrote:
> >
> > I have to change my non-binding +1 vote to a 0, unless there is a
> > workaround for the bug [1] I just submitted, regarding the QueryRecord
> > processor being broken.
> >
> >
> > [1] https://issues.apache.org/jira/browse/NIFI-4452
> >
> >
> > On Mon, Oct 2, 2017 at 3:23 AM, Kevin Doran 
> wrote:
> >
> >> +1, non-binding
> >>
> >> - Verified hashes
> >> - Full build with contrib-check passes
> >> - Was able to run nifi server and send flow files to remote input port
> >> over s2s protocol. Web UI and API working as expected.
> >>
> >> Platform:
> >> Apache Maven 3.5.0 (ff8f5e744

Re: QueryRecord error

2017-10-02 Thread Mark Payne
Joe,

Thanks. I was going to suggest we spawn off a new thread if this continued with 
more back-and-forth.

I did notice that in the JIRA you indicated that
you are using a CSVReader service to read the data. In the message, it 
indicates that the FlowFIle's
filename is 
"billing-detailed-line-items-with-resources-and-tags-2017-10.csv.avro" -- which 
would imply
to me that it is Avro (perhaps converted from CSV to avro upstream?) Is it 
possible that you just selected
the wrong reader?

Thanks
-Mark

> On Oct 2, 2017, at 12:24 PM, Joe Gresock  wrote:
> 
> I'm changing the subject name so as not to muddy the [VOTE] thread.
> 
> I finally found the error in the logs.  This was also not happening prior
> to the upgrade.  I thinkI'll debug the server to determine the root cause:
> 
> 2017-10-02 16:09:47,004 ERROR [Timer-Driven Process Thread-81]
> o.a.nifi.processors.standard.QueryRecord
> QueryRecord[id=21570e18-015c-1000--58832883] Unable to query
> StandardFlowFileRecord[uuid=2da6d0f9-3856-420e-99bd-9589f7f74911,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1506960462126-919,
> container=default, section=919], offset=55116,
> length=616757],offset=0,name=billing-detailed-line-items-with-resources-and-tags-2017-10.csv.avro,size=616757]
> due to org.apache.nifi.processor.exception.ProcessException: Failed to read
> next record in stream for
> StandardFlowFileRecord[uuid=2da6d0f9-3856-420e-99bd-9589f7f74911,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1506960462126-919,
> container=default, section=919], offset=55116,
> length=616757],offset=0,name=billing-detailed-line-items-with-resources-and-tags-2017-10.csv.avro,size=616757]:
> {}
> org.apache.nifi.processor.exception.ProcessException: Failed to read next
> record in stream for
> StandardFlowFileRecord[uuid=2da6d0f9-3856-420e-99bd-9589f7f74911,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1506960462126-919,
> container=default, section=919], offset=55116,
> length=616757],offset=0,name=billing-detailed-line-items-with-resources-and-tags-2017-10.csv.avro,size=616757]
>at
> org.apache.nifi.queryrecord.FlowFileEnumerator.moveNext(FlowFileEnumerator.java:65)
>at Baz$1$1.moveNext(Unknown Source)
>at
> org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.(Linq4j.java:664)
>at
> org.apache.calcite.linq4j.Linq4j.enumeratorIterator(Linq4j.java:98)
>at
> org.apache.calcite.linq4j.AbstractEnumerable.iterator(AbstractEnumerable.java:33)
>at
> org.apache.calcite.avatica.MetaImpl.createCursor(MetaImpl.java:89)
>at
> org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:196)
>at
> org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:67)
>at
> org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:44)
>at
> org.apache.calcite.avatica.AvaticaConnection.executeQueryInternal(AvaticaConnection.java:513)
>at
> org.apache.calcite.avatica.AvaticaPreparedStatement.executeQuery(AvaticaPreparedStatement.java:132)
>at
> org.apache.nifi.processors.standard.QueryRecord.queryWithCache(QueryRecord.java:445)
>at
> org.apache.nifi.processors.standard.QueryRecord.onTrigger(QueryRecord.java:276)
>at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
>at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1119)
>at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
>at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
>at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
>at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
>at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>at java.lang.Thread.run(Thread.java:748)
> 
> 
> On Mon, Oct 2, 2017 at 4:03 PM, Mark Payne  wrote:
> 
>> Hey Joe,
>> 
>> I've seen this error crop up a time or two before. The issue is a bug in
>> how we are handling a failure
>> condition. Haven't yet figured out the best way to tackle it, but the
>> thing to note is that this error occurs
>> when there is some other sort of failure -- typically, a failure to obtain
>> the schema for the Record Reader
>> or Record Writer. Can you check your logs for any other erro

Re: QueryRecord error

2017-10-02 Thread Joe Gresock
Good question -- no, this must be an artifact of running InferAvroSchema
before the QueryRecord processor.  InferAvroSchema infers the schema and
places it in a flow file attribute, which the CSVReader uses in its
configuration.

I just debugged nifi and found that it thinks one of my fields is null.
There is indeed a record with an empty string in the csv for this field,
but the same thing happens every day when this flow runs, so I suspect
something must have changed in the implementation of either CSVReader or
InferAvroSchema that made the parsing stricter.  I'm going to try to figure
out how to allow blank values in the CSV.

On Mon, Oct 2, 2017 at 4:29 PM, Mark Payne  wrote:

> Joe,
>
> Thanks. I was going to suggest we spawn off a new thread if this continued
> with more back-and-forth.
>
> I did notice that in the JIRA you indicated that
> you are using a CSVReader service to read the data. In the message, it
> indicates that the FlowFIle's
> filename is 
> "billing-detailed-line-items-with-resources-and-tags-2017-10.csv.avro"
> -- which would imply
> to me that it is Avro (perhaps converted from CSV to avro upstream?) Is it
> possible that you just selected
> the wrong reader?
>
> Thanks
> -Mark
>
> > On Oct 2, 2017, at 12:24 PM, Joe Gresock  wrote:
> >
> > I'm changing the subject name so as not to muddy the [VOTE] thread.
> >
> > I finally found the error in the logs.  This was also not happening prior
> > to the upgrade.  I thinkI'll debug the server to determine the root
> cause:
> >
> > 2017-10-02 16:09:47,004 ERROR [Timer-Driven Process Thread-81]
> > o.a.nifi.processors.standard.QueryRecord
> > QueryRecord[id=21570e18-015c-1000--58832883] Unable to query
> > StandardFlowFileRecord[uuid=2da6d0f9-3856-420e-99bd-9589f7f74911,claim=
> StandardContentClaim
> > [resourceClaim=StandardResourceClaim[id=1506960462126-919,
> > container=default, section=919], offset=55116,
> > length=616757],offset=0,name=billing-detailed-line-items-
> with-resources-and-tags-2017-10.csv.avro,size=616757]
> > due to org.apache.nifi.processor.exception.ProcessException: Failed to
> read
> > next record in stream for
> > StandardFlowFileRecord[uuid=2da6d0f9-3856-420e-99bd-9589f7f74911,claim=
> StandardContentClaim
> > [resourceClaim=StandardResourceClaim[id=1506960462126-919,
> > container=default, section=919], offset=55116,
> > length=616757],offset=0,name=billing-detailed-line-items-
> with-resources-and-tags-2017-10.csv.avro,size=616757]:
> > {}
> > org.apache.nifi.processor.exception.ProcessException: Failed to read
> next
> > record in stream for
> > StandardFlowFileRecord[uuid=2da6d0f9-3856-420e-99bd-9589f7f74911,claim=
> StandardContentClaim
> > [resourceClaim=StandardResourceClaim[id=1506960462126-919,
> > container=default, section=919], offset=55116,
> > length=616757],offset=0,name=billing-detailed-line-items-
> with-resources-and-tags-2017-10.csv.avro,size=616757]
> >at
> > org.apache.nifi.queryrecord.FlowFileEnumerator.moveNext(
> FlowFileEnumerator.java:65)
> >at Baz$1$1.moveNext(Unknown Source)
> >at
> > org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.<
> init>(Linq4j.java:664)
> >at
> > org.apache.calcite.linq4j.Linq4j.enumeratorIterator(Linq4j.java:98)
> >at
> > org.apache.calcite.linq4j.AbstractEnumerable.iterator(
> AbstractEnumerable.java:33)
> >at
> > org.apache.calcite.avatica.MetaImpl.createCursor(MetaImpl.java:89)
> >at
> > org.apache.calcite.avatica.AvaticaResultSet.execute(
> AvaticaResultSet.java:196)
> >at
> > org.apache.calcite.jdbc.CalciteResultSet.execute(
> CalciteResultSet.java:67)
> >at
> > org.apache.calcite.jdbc.CalciteResultSet.execute(
> CalciteResultSet.java:44)
> >at
> > org.apache.calcite.avatica.AvaticaConnection.executeQueryInternal(
> AvaticaConnection.java:513)
> >at
> > org.apache.calcite.avatica.AvaticaPreparedStatement.executeQuery(
> AvaticaPreparedStatement.java:132)
> >at
> > org.apache.nifi.processors.standard.QueryRecord.
> queryWithCache(QueryRecord.java:445)
> >at
> > org.apache.nifi.processors.standard.QueryRecord.
> onTrigger(QueryRecord.java:276)
> >at
> > org.apache.nifi.processor.AbstractProcessor.onTrigger(
> AbstractProcessor.java:27)
> >at
> > org.apache.nifi.controller.StandardProcessorNode.onTrigger(
> StandardProcessorNode.java:1119)
> >at
> > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(
> ContinuallyRunProcessorTask.java:147)
> >at
> > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(
> ContinuallyRunProcessorTask.java:47)
> >at
> > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(
> TimerDrivenSchedulingAgent.java:128)
> >at
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >at java.util.concurrent.FutureTask.runAndReset(
> FutureTask.java:308)
> >at
> > java.util.concurrent.ScheduledThre

Re: QueryRecord error

2017-10-02 Thread Joe Gresock
I was able to configure the reader to infer the schema from the first line
of the CSV, and the writer to use the same schema, and now the processor
works.  I think these are new properties in 1.4.0, and they certainly help
simplify the flow!  I was able to remove the InferAvroSchema processor as a
result.

Joe

On Mon, Oct 2, 2017 at 4:44 PM, Joe Gresock  wrote:

> Good question -- no, this must be an artifact of running InferAvroSchema
> before the QueryRecord processor.  InferAvroSchema infers the schema and
> places it in a flow file attribute, which the CSVReader uses in its
> configuration.
>
> I just debugged nifi and found that it thinks one of my fields is null.
> There is indeed a record with an empty string in the csv for this field,
> but the same thing happens every day when this flow runs, so I suspect
> something must have changed in the implementation of either CSVReader or
> InferAvroSchema that made the parsing stricter.  I'm going to try to figure
> out how to allow blank values in the CSV.
>
> On Mon, Oct 2, 2017 at 4:29 PM, Mark Payne  wrote:
>
>> Joe,
>>
>> Thanks. I was going to suggest we spawn off a new thread if this
>> continued with more back-and-forth.
>>
>> I did notice that in the JIRA you indicated that
>> you are using a CSVReader service to read the data. In the message, it
>> indicates that the FlowFIle's
>> filename is 
>> "billing-detailed-line-items-with-resources-and-tags-2017-10.csv.avro"
>> -- which would imply
>> to me that it is Avro (perhaps converted from CSV to avro upstream?) Is
>> it possible that you just selected
>> the wrong reader?
>>
>> Thanks
>> -Mark
>>
>> > On Oct 2, 2017, at 12:24 PM, Joe Gresock  wrote:
>> >
>> > I'm changing the subject name so as not to muddy the [VOTE] thread.
>> >
>> > I finally found the error in the logs.  This was also not happening
>> prior
>> > to the upgrade.  I thinkI'll debug the server to determine the root
>> cause:
>> >
>> > 2017-10-02 16:09:47,004 ERROR [Timer-Driven Process Thread-81]
>> > o.a.nifi.processors.standard.QueryRecord
>> > QueryRecord[id=21570e18-015c-1000--58832883] Unable to query
>> > StandardFlowFileRecord[uuid=2da6d0f9-3856-420e-99bd-9589f7f7
>> 4911,claim=StandardContentClaim
>> > [resourceClaim=StandardResourceClaim[id=1506960462126-919,
>> > container=default, section=919], offset=55116,
>> > length=616757],offset=0,name=billing-detailed-line-items-wit
>> h-resources-and-tags-2017-10.csv.avro,size=616757]
>> > due to org.apache.nifi.processor.exception.ProcessException: Failed to
>> read
>> > next record in stream for
>> > StandardFlowFileRecord[uuid=2da6d0f9-3856-420e-99bd-9589f7f7
>> 4911,claim=StandardContentClaim
>> > [resourceClaim=StandardResourceClaim[id=1506960462126-919,
>> > container=default, section=919], offset=55116,
>> > length=616757],offset=0,name=billing-detailed-line-items-wit
>> h-resources-and-tags-2017-10.csv.avro,size=616757]:
>> > {}
>> > org.apache.nifi.processor.exception.ProcessException: Failed to read
>> next
>> > record in stream for
>> > StandardFlowFileRecord[uuid=2da6d0f9-3856-420e-99bd-9589f7f7
>> 4911,claim=StandardContentClaim
>> > [resourceClaim=StandardResourceClaim[id=1506960462126-919,
>> > container=default, section=919], offset=55116,
>> > length=616757],offset=0,name=billing-detailed-line-items-wit
>> h-resources-and-tags-2017-10.csv.avro,size=616757]
>> >at
>> > org.apache.nifi.queryrecord.FlowFileEnumerator.moveNext(Flow
>> FileEnumerator.java:65)
>> >at Baz$1$1.moveNext(Unknown Source)
>> >at
>> > org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.(
>> Linq4j.java:664)
>> >at
>> > org.apache.calcite.linq4j.Linq4j.enumeratorIterator(Linq4j.java:98)
>> >at
>> > org.apache.calcite.linq4j.AbstractEnumerable.iterator(Abstra
>> ctEnumerable.java:33)
>> >at
>> > org.apache.calcite.avatica.MetaImpl.createCursor(MetaImpl.java:89)
>> >at
>> > org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaR
>> esultSet.java:196)
>> >at
>> > org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResu
>> ltSet.java:67)
>> >at
>> > org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResu
>> ltSet.java:44)
>> >at
>> > org.apache.calcite.avatica.AvaticaConnection.executeQueryInt
>> ernal(AvaticaConnection.java:513)
>> >at
>> > org.apache.calcite.avatica.AvaticaPreparedStatement.executeQ
>> uery(AvaticaPreparedStatement.java:132)
>> >at
>> > org.apache.nifi.processors.standard.QueryRecord.queryWithCac
>> he(QueryRecord.java:445)
>> >at
>> > org.apache.nifi.processors.standard.QueryRecord.onTrigger(
>> QueryRecord.java:276)
>> >at
>> > org.apache.nifi.processor.AbstractProcessor.onTrigger(Abstra
>> ctProcessor.java:27)
>> >at
>> > org.apache.nifi.controller.StandardProcessorNode.onTrigger(S
>> tandardProcessorNode.java:1119)
>> >at
>> > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask
>> .call(ContinuallyRunProcessorTask.j