[
https://issues.apache.org/jira/browse/NIFI-1534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15154376#comment-15154376
]
Oleg Zhurakousky edited comment on NIFI-1534 at 2/19/16 3:48 PM:
-----------------------------------------------------------------
This is a real catch 22 situation. Basically PutKafka is annotated with
_@TriggerWhenEmpty_ and _@SupportsBatching_. On top of that the implementation
logic is structured that there may be a significant delay (even without
batching) on when the FlowFile is transferred to a Relationship and the actual
session for which messages have been sent is committed which in a way exposes
another bug. For example imagine your task runs once every hour. Even without
batching the FlowFile will be transferred and session will be committed an hour
after the actual message have been sent. That is because the logic in
onTrigger() processes internal queue first before building a new one and since
internal queue doesn't exist on the first invocation of _onTrigger()_, nothing
is being transferred and/or committed on the first invocation of _onTrigger()_.
And with each subsequent invocation the component is always one step behind.
The above is also at the core of immediate issue since on the first invocation
of the _onTrigger()_ nothing is transferred to any relationship, but given the
support for batching (via _@SupportsBatching_) and *Run-duration > 0* the
ContinuallyRunningProcessTask attempts to commit session after each invocation
of _onTrigger()_ resulting in "_transfer relationship not specified_"
So for the time being the workaround is to not set Run-duration to any value >
0.
was (Author: ozhurakousky):
This si a real catch 22 situation. Basically PutKafka is annotated with
_@TriggerWhenEmpty_ and _@SupportsBatching_. On top of that the implementation
logic is structured that there may be a significant delay (even without
batching) on when the FlowFile is transferred to a Relationship and the actual
session for which messages have been sent is committed which in a way exposes
another bug. For example imagine your task runs once every hour. Even without
batching the FlowFile will be transferred and session will be committed an hour
after the actual message have been sent. That is because the logic in
onTrigger() processes internal queue first before building a new one and since
internal queue doesn't exist on the first invocation of _onTrigger()_, nothing
is being transferred and/or committed on the first invocation of _onTrigger()_.
And with each subsequent invocation the component is always one step behind.
The above is also at the core of immediate issue since on the first invocation
of the _onTrigger()_ nothing is transferred to any relationship, but given the
support for batching (via _@SupportsBatching_) and *Run-duration > 0* the
ContinuallyRunningProcessTask attempts to commit session after each invocation
of _onTrigger()_ resulting in "_transfer relationship not specified_"
So for the time being the workaround is to not set Run-duration to any value >
0.
> PutKafka fails when Run Duration > 0
> ------------------------------------
>
> Key: NIFI-1534
> URL: https://issues.apache.org/jira/browse/NIFI-1534
> Project: Apache NiFi
> Issue Type: Bug
> Reporter: Oleg Zhurakousky
> Assignee: Oleg Zhurakousky
> Fix For: 0.5.1
>
>
> The following error shows up in the bulletin
> {code}
> PutKafka[id=80e8946f-349c-4716-9da9-5e766582f788] Failed to process session
> due to org.apache.nifi.processor.exception.FlowFileHandlingException:
> StandardFlowFileRecord[uuid=27030bf8-c217-4b59-a0ba-46f1b30ff093,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1455807429050-246,
> container=default, section=246], offset=563929,
> length=431],offset=0,name=hh77.33,size=431] is not known in this session
> {code}
> After some debugging I also see this
> {code}
> ERROR [Timer-Driven Process Thread-14] o.apache.nifi.processors.kafka.PutKafka
> org.apache.nifi.processor.exception.FlowFileHandlingException:
> StandardFlowFileRecord[uuid=dd373d71-ebd2-432e-bcf5-9980d7df4da4,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1454688235111-5, container=content2,
> section=5], offset=0, length=4778852],offset=0,name=boo,size=4778852]
> transfer relationship not specified
> at
> org.apache.nifi.controller.repository.StandardProcessSession.checkpoint(StandardProcessSession.java:210)
>
> at
> org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:280)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:179)
> . . . .
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
