[
https://issues.apache.org/jira/browse/NIFI-1619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15191023#comment-15191023
]
ASF GitHub Bot commented on NIFI-1619:
--------------------------------------
GitHub user mattyb149 opened a pull request:
https://github.com/apache/nifi/pull/269
NIFI-1619: Fix Elasticsearch processor bug when flow file missing ID
attribute
When the flow file was missing the attribute specified in the processor
config, there was still an attempt to insert it into Elasticsearch. This was a
simple missing "else" clause. However the flow files that had already been
transferred to failure due to this condition caused other problems of multiple
transfers. The fix is to keep track of the flow files that remain to be
transferred, starting with the original list and removing them as they are
transferred.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mattyb149/nifi NIFI-1619
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/269.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #269
----
commit bbf2c26fa38e1e3bd742a12ae34ad5bd1982889a
Author: Matt Burgess <mattyb...@gmail.com>
Date: 2016-03-11T14:44:15Z
NIFI-1619: Fix Elasticsearch processor bug when flow file missing ID
attribute
----
> PutElasticSearch throws FlowFileHandlingException when id attribute is missing
> ------------------------------------------------------------------------------
>
> Key: NIFI-1619
> URL: https://issues.apache.org/jira/browse/NIFI-1619
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Affects Versions: 0.5.1
> Reporter: Jim Wagoner
> Assignee: Matt Burgess
> Priority: Minor
> Labels: easyfix
> Fix For: 0.6.0
>
>
> When the processor can't find the id_attribute on the FlowFile while trying
> to be added to the Bulk Request it is transferred to failure relationship.
> The issue is after this happens the FlowFile is still added to the Bulk
> Request even though a invalid id will be used during the insert. This will
> cause the a failure to be returned to the BulkResponse and a attempt to
> transfer the already transferred FlowFile throws a FlowFileHandlingException .
> Looks like a "continue;" added to line 178 would solve this issue.
> STACK:
> 2016-03-11 07:49:39,053 ERROR [Timer-Driven Process Thread-12]
> o.a.n.p.elasticsearch.PutElasticsearch
> PutElasticsearch[id=9a22e93f-03d8-445f-95b8-7979d0016ee7] No value in
> identifier attribute ${metadataId} for
> StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1457700558252-17754,
> container=default, section=346], offset=648997,
> length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283]
> 2016-03-11 07:49:39,081 ERROR [Timer-Driven Process Thread-12]
> o.a.n.p.elasticsearch.PutElasticsearch
> PutElasticsearch[id=9a22e93f-03d8-445f-95b8-7979d0016ee7] Failed to insert
> into Elasticsearch due to
> StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1457700558252-17754,
> container=default, section=346], offset=648997,
> length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283]
> is already marked for transfer:
> org.apache.nifi.processor.exception.FlowFileHandlingException:
> StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1457700558252-17754,
> container=default, section=346], offset=648997,
> length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283]
> is already marked for transfer
> 2016-03-11 07:49:39,105 ERROR [Timer-Driven Process Thread-12]
> o.a.n.p.elasticsearch.PutElasticsearch
> org.apache.nifi.processor.exception.FlowFileHandlingException:
> StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1457700558252-17754,
> container=default, section=346], offset=648997,
> length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283]
> is already marked for transfer
> at
> org.apache.nifi.controller.repository.StandardProcessSession.validateRecordState(StandardProcessSession.java:2369)
> ~[na:na]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1782)
> ~[na:na]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1777)
> ~[na:na]
> at
> org.apache.nifi.processors.elasticsearch.PutElasticsearch.onTrigger(PutElasticsearch.java:171)
> ~[nifi-elasticsearch-processors-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> [nifi-api-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1139)
> [nifi-framework-core-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139)
> [nifi-framework-core-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49)
> [nifi-framework-core-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:124)
> [nifi-framework-core-0.5.1.jar:0.5.1]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> [na:1.7.0_95]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> [na:1.7.0_95]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> [na:1.7.0_95]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> [na:1.7.0_95]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_95]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_95]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_95]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
