[jira] [Created] (NIFI-9645) Re-enable "Idle Connection Expiration" property in PutSplunk
Anders Synstad created NIFI-9645:
Summary: Re-enable "Idle Connection Expiration" property in
PutSplunk
Key: NIFI-9645
URL: https://issues.apache.org/jira/browse/NIFI-9645
Project: Apache NiFi
Issue Type: Improvement
Affects Versions: 1.15.3, 1.15.2, 1.15.1, 1.15.0, 1.14.0
Reporter: Anders Synstad
Creating JIRA ticket as requested by [~exceptionfactory] in
https://apachenifi.slack.com/archives/C0L9VCD47/p1643633831171269
In NiFi version 1.14.0 and later
(https://issues.apache.org/jira/browse/NIFI-8616), the "Idle Connection
Expiration" property in PutSplunk is no longer used. This causes quite a bit of
warning spam with "Connection reset by peer" in certain configurations:
{code}
2022-01-31 16:18:17,268 WARN
[PutSplunk[19dc42e1-0b51-340d-a9a6-81ad014d79e1]-41-2]
o.a.nifi.processors.splunk.PutSplunk
PutSplunk[id=19dc42e1-0b51-340d-a9a6-81ad014d79e1] Communication Failed with
Remote Address [/1.2.3.4:12345]: java.io.IOException: Connection reset by peer
java.io.IOException: Connection reset by peer
at java.base/sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at java.base/sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at java.base/sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:276)
at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:233)
at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:223)
at
java.base/sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:356)
at io.netty.buffer.PooledByteBuf.setBytes(PooledByteBuf.java:253)
at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1132)
at
io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:350)
at
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:151)
at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
at
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
{code}
On example where this causes this problem is when using an LB between NiFi and
Splunk.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (NIFI-9613) QueryRecord fails with java.util.NoSuchElementException
[
https://issues.apache.org/jira/browse/NIFI-9613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479963#comment-17479963
]
Anders Synstad commented on NIFI-9613:
--
I believe you are correct :)
> QueryRecord fails with java.util.NoSuchElementException
> ---
>
> Key: NIFI-9613
> URL: https://issues.apache.org/jira/browse/NIFI-9613
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.15.2
> Environment: Ubuntu 18.04 with openjdk-11-jre-headless
> 11.0.13+8-0ubuntu1~18.04
>Reporter: Anders Synstad
>Assignee: Pierre Villard
>Priority: Minor
> Fix For: 1.16.0
>
> Attachments: JSON-test.xml
>
>
> I am getting java.util.NoSuchElementException in QueryRecord under certain
> conditions using JsonTreeReader + JsonRecordSetWriter + a simple json event.
> Generating a simple flowfile with the following json using GenerateFlowFile:
> {{
> {"epoch":1642669279,"thing":[1234],"foo":"bar"}
> }}
> Using QueryRecord with the following dynamic properties work:
> {{SELECT * FROM FLOWFILE WHERE foo = 'bar'}}
> {{SELECT * FROM FLOWFILE WHERE foo LIKE 'ba%'}}
> However, using the following generates an error:
> {{SELECT * FROM FLOWFILE WHERE foo LIKE 'fo%'}}
> It appears it will fail if the WHERE clause does not find any matches in the
> flowfile.Removing {{"thing":[12345]}} from the json in the flowfile causes
> all 3 dynamic
> properties in QueryRecord to work.The error generated is:
> {code}
> 2022-01-20 10:19:25,403 ERROR [Timer-Driven Process Thread-24]
> o.a.nifi.processors.standard.QueryRecord
> QueryRecord[id=9a2539a7-14df-159f-b3f4-83787721a05f] Unable to query
> StandardFlowFileRecord[uuid=db24c0b5-e874-4a55-82a7-cbdf69b0cf5a,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1642670361821-5989894,
> container=default, section=518], offset=517544,
> length=47],offset=0,name=db24c0b5-e874-4a55-82a7-cbdf69b0cf5a,size=47] due to
> org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException:
> java.util.NoSuchElementException: Expecting cursor position to be
> Position.OK, actual is Position.AFTER_END: java.util.NoSuchElementException:
> Expecting cursor position to be Position.OK, actual is Position.AFTER_END
> ↳ causes: java.sql.SQLException: java.util.NoSuchElementException: Expecting
> cursor position to be Position.OK, actual is Position.AFTER_END
> ↳ causes: org.apache.nifi.processor.exception.ProcessException:
> java.sql.SQLException: java.util.NoSuchElementException: Expecting cursor
> position to be Position.OK, actual is Position.AFTER_END
> org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException:
> java.util.NoSuchElementException: Expecting cursor position to be
> Position.OK, actual is Position.AFTER_END
> at
> org.apache.nifi.processors.standard.QueryRecord$1.process(QueryRecord.java:343)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2909)
> at
> org.apache.nifi.processors.standard.QueryRecord.onTrigger(QueryRecord.java:332)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1273)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)
> at
> org.apache.nifi.controller.scheduling.AbstractTimeBasedSchedulingAgent.lambda$doScheduleOnce$0(AbstractTimeBasedSchedulingAgent.java:63)
> at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:829)
> Caused by: java.sql.SQLException: java.util.NoSuchElementException: Expecting
> cursor position to be Position.OK, actual is Position.AFTER_END
> at
> org.apache.calcite.avatica.util.PositionedCursor$ArrayGetter.getObject(PositionedCursor.java:65)
> at
> org.apache.calcite.avatica.util.AbstractCursor$AccessorImpl.getObject(AbstractCursor.java:355)
> at
> org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.getObject(AbstractCursor.java:1286)
> at
> org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.getArray(AbstractCursor.java:1354)
> at
> org.apache.calcite.avatica.AvaticaResultSet.getArray(AvaticaResultSet.java:729)
>
[jira] [Updated] (NIFI-9613) QueryRecord fails with java.util.NoSuchElementException
[
https://issues.apache.org/jira/browse/NIFI-9613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anders Synstad updated NIFI-9613:
-
Description:
I am getting java.util.NoSuchElementException in QueryRecord under certain
conditions using JsonTreeReader + JsonRecordSetWriter + a simple json event.
Generating a simple flowfile with the following json using GenerateFlowFile:
{{
{"epoch":1642669279,"thing":[1234],"foo":"bar"}
}}
Using QueryRecord with the following dynamic properties work:
{{SELECT * FROM FLOWFILE WHERE foo = 'bar'}}
{{SELECT * FROM FLOWFILE WHERE foo LIKE 'ba%'}}
However, using the following generates an error:
{{SELECT * FROM FLOWFILE WHERE foo LIKE 'fo%'}}
It appears it will fail if the WHERE clause does not find any matches in the
flowfile.Removing {{"thing":[12345]}} from the json in the flowfile causes all
3 dynamic
properties in QueryRecord to work.The error generated is:
{code}
2022-01-20 10:19:25,403 ERROR [Timer-Driven Process Thread-24]
o.a.nifi.processors.standard.QueryRecord
QueryRecord[id=9a2539a7-14df-159f-b3f4-83787721a05f] Unable to query
StandardFlowFileRecord[uuid=db24c0b5-e874-4a55-82a7-cbdf69b0cf5a,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1642670361821-5989894,
container=default, section=518], offset=517544,
length=47],offset=0,name=db24c0b5-e874-4a55-82a7-cbdf69b0cf5a,size=47] due to
org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException:
java.util.NoSuchElementException: Expecting cursor position to be Position.OK,
actual is Position.AFTER_END: java.util.NoSuchElementException: Expecting
cursor position to be Position.OK, actual is Position.AFTER_END
↳ causes: java.sql.SQLException: java.util.NoSuchElementException: Expecting
cursor position to be Position.OK, actual is Position.AFTER_END
↳ causes: org.apache.nifi.processor.exception.ProcessException:
java.sql.SQLException: java.util.NoSuchElementException: Expecting cursor
position to be Position.OK, actual is Position.AFTER_END
org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException:
java.util.NoSuchElementException: Expecting cursor position to be Position.OK,
actual is Position.AFTER_END
at
org.apache.nifi.processors.standard.QueryRecord$1.process(QueryRecord.java:343)
at
org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2909)
at
org.apache.nifi.processors.standard.QueryRecord.onTrigger(QueryRecord.java:332)
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1273)
at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)
at
org.apache.nifi.controller.scheduling.AbstractTimeBasedSchedulingAgent.lambda$doScheduleOnce$0(AbstractTimeBasedSchedulingAgent.java:63)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.sql.SQLException: java.util.NoSuchElementException: Expecting
cursor position to be Position.OK, actual is Position.AFTER_END
at
org.apache.calcite.avatica.util.PositionedCursor$ArrayGetter.getObject(PositionedCursor.java:65)
at
org.apache.calcite.avatica.util.AbstractCursor$AccessorImpl.getObject(AbstractCursor.java:355)
at
org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.getObject(AbstractCursor.java:1286)
at
org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.getArray(AbstractCursor.java:1354)
at
org.apache.calcite.avatica.AvaticaResultSet.getArray(AvaticaResultSet.java:729)
at
org.apache.nifi.serialization.record.ResultSetRecordSet.getArrayDataType(ResultSetRecordSet.java:295)
at
org.apache.nifi.serialization.record.ResultSetRecordSet.getDataType(ResultSetRecordSet.java:207)
at
org.apache.nifi.serialization.record.ResultSetRecordSet.createSchema(ResultSetRecordSet.java:184)
at
org.apache.nifi.serialization.record.ResultSetRecordSet.(ResultSetRecordSet.java:86)
at
org.apache.nifi.serialization.record.ResultSetRecordSet.(ResultSetRecordSet.java:71)
at
org.apache.nifi.processors.standard.QueryRecord$1.process(QueryRecord.java:339)
... 13 common frames omitted
Caused by: java.util.NoSuchElementException: Expecting cursor position to be
Position.OK, actual is Position.AFTER_END
at
org.apache.calcite.avatica.util.IteratorCursor.current(IteratorCursor.java:71)
at
[jira] [Created] (NIFI-9613) QueryRecord fails with java.util.NoSuchElementException
Anders Synstad created NIFI-9613:
Summary: QueryRecord fails with java.util.NoSuchElementException
Key: NIFI-9613
URL: https://issues.apache.org/jira/browse/NIFI-9613
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Affects Versions: 1.15.2
Environment: Ubuntu 18.04 with openjdk-11-jre-headless
11.0.13+8-0ubuntu1~18.04
Reporter: Anders Synstad
Attachments: JSON-test.xml
I am getting java.util.NoSuchElementException in QueryRecord under certain
conditions using JsonTreeReader + JsonRecordSetWriter + a simple json event.
Generating a simple flowfile with the following json using GenerateFlowFile:
{{{"epoch":1642669279,"thing":[1234],"foo":"bar"}}}
Using QueryRecord with the following dynamic properties work:
{{SELECT * FROM FLOWFILE WHERE foo = 'bar'}}
{{SELECT * FROM FLOWFILE WHERE foo LIKE 'ba%'}}
However, using the following generates an error:
{{SELECT * FROM FLOWFILE WHERE foo LIKE 'fo%'}}
It appears it will fail if the WHERE clause does not find any matches in the
flowfile.Removing {{"thing":[12345]}} from the json in the flowfile causes all
3 dynamic
properties in QueryRecord to work.The error generated is:
2022-01-20 10:19:25,403 ERROR [Timer-Driven Process Thread-24]
o.a.nifi.processors.standard.QueryRecord
QueryRecord[id=9a2539a7-14df-159f-b3f4-83787721a05f] Unable to query
StandardFlowFileRecord[uuid=db24c0b5-e874-4a55-82a7-cbdf69b0cf5a,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1642670361821-5989894,
container=default, section=518], offset=517544,
length=47],offset=0,name=db24c0b5-e874-4a55-82a7-cbdf69b0cf5a,size=47] due to
org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException:
java.util.NoSuchElementException: Expecting cursor position to be Position.OK,
actual is Position.AFTER_END: java.util.NoSuchElementException: Expecting
cursor position to be Position.OK, actual is Position.AFTER_END
↳ causes: java.sql.SQLException: java.util.NoSuchElementException: Expecting
cursor position to be Position.OK, actual is Position.AFTER_END
↳ causes: org.apache.nifi.processor.exception.ProcessException:
java.sql.SQLException: java.util.NoSuchElementException: Expecting cursor
position to be Position.OK, actual is Position.AFTER_END
org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException:
java.util.NoSuchElementException: Expecting cursor position to be Position.OK,
actual is Position.AFTER_END
at
org.apache.nifi.processors.standard.QueryRecord$1.process(QueryRecord.java:343)
at
org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2909)
at
org.apache.nifi.processors.standard.QueryRecord.onTrigger(QueryRecord.java:332)
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1273)
at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)
at
org.apache.nifi.controller.scheduling.AbstractTimeBasedSchedulingAgent.lambda$doScheduleOnce$0(AbstractTimeBasedSchedulingAgent.java:63)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.sql.SQLException: java.util.NoSuchElementException: Expecting
cursor position to be Position.OK, actual is Position.AFTER_END
at
org.apache.calcite.avatica.util.PositionedCursor$ArrayGetter.getObject(PositionedCursor.java:65)
at
org.apache.calcite.avatica.util.AbstractCursor$AccessorImpl.getObject(AbstractCursor.java:355)
at
org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.getObject(AbstractCursor.java:1286)
at
org.apache.calcite.avatica.util.AbstractCursor$ArrayAccessor.getArray(AbstractCursor.java:1354)
at
org.apache.calcite.avatica.AvaticaResultSet.getArray(AvaticaResultSet.java:729)
at
org.apache.nifi.serialization.record.ResultSetRecordSet.getArrayDataType(ResultSetRecordSet.java:295)
at
org.apache.nifi.serialization.record.ResultSetRecordSet.getDataType(ResultSetRecordSet.java:207)
at
org.apache.nifi.serialization.record.ResultSetRecordSet.createSchema(ResultSetRecordSet.java:184)
at
