[jira] [Commented] (NIFI-1619) PutElasticSearch throws FlowFileHandlingException when id attribute is missing
[
https://issues.apache.org/jira/browse/NIFI-1619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15191023#comment-15191023
]
ASF GitHub Bot commented on NIFI-1619:
--
GitHub user mattyb149 opened a pull request:
https://github.com/apache/nifi/pull/269
NIFI-1619: Fix Elasticsearch processor bug when flow file missing ID
attribute
When the flow file was missing the attribute specified in the processor
config, there was still an attempt to insert it into Elasticsearch. This was a
simple missing "else" clause. However the flow files that had already been
transferred to failure due to this condition caused other problems of multiple
transfers. The fix is to keep track of the flow files that remain to be
transferred, starting with the original list and removing them as they are
transferred.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mattyb149/nifi NIFI-1619
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/269.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #269
commit bbf2c26fa38e1e3bd742a12ae34ad5bd1982889a
Author: Matt Burgess
Date: 2016-03-11T14:44:15Z
NIFI-1619: Fix Elasticsearch processor bug when flow file missing ID
attribute
> PutElasticSearch throws FlowFileHandlingException when id attribute is missing
> --
>
> Key: NIFI-1619
> URL: https://issues.apache.org/jira/browse/NIFI-1619
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 0.5.1
>Reporter: Jim Wagoner
>Assignee: Matt Burgess
>Priority: Minor
> Labels: easyfix
> Fix For: 0.6.0
>
>
> When the processor can't find the id_attribute on the FlowFile while trying
> to be added to the Bulk Request it is transferred to failure relationship.
> The issue is after this happens the FlowFile is still added to the Bulk
> Request even though a invalid id will be used during the insert. This will
> cause the a failure to be returned to the BulkResponse and a attempt to
> transfer the already transferred FlowFile throws a FlowFileHandlingException .
> Looks like a "continue;" added to line 178 would solve this issue.
> STACK:
> 2016-03-11 07:49:39,053 ERROR [Timer-Driven Process Thread-12]
> o.a.n.p.elasticsearch.PutElasticsearch
> PutElasticsearch[id=9a22e93f-03d8-445f-95b8-7979d0016ee7] No value in
> identifier attribute ${metadataId} for
> StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1457700558252-17754,
> container=default, section=346], offset=648997,
> length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283]
> 2016-03-11 07:49:39,081 ERROR [Timer-Driven Process Thread-12]
> o.a.n.p.elasticsearch.PutElasticsearch
> PutElasticsearch[id=9a22e93f-03d8-445f-95b8-7979d0016ee7] Failed to insert
> into Elasticsearch due to
> StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1457700558252-17754,
> container=default, section=346], offset=648997,
> length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283]
> is already marked for transfer:
> org.apache.nifi.processor.exception.FlowFileHandlingException:
> StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1457700558252-17754,
> container=default, section=346], offset=648997,
> length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283]
> is already marked for transfer
> 2016-03-11 07:49:39,105 ERROR [Timer-Driven Process Thread-12]
> o.a.n.p.elasticsearch.PutElasticsearch
> org.apache.nifi.processor.exception.FlowFileHandlingException:
> StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1457700558252-17754,
> container=default, section=346], offset=648997,
> length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283]
> is already marked for transfer
> at
> org.apache.nifi.controller.repository.StandardProcessSession.validateRecordState(StandardProcessSession.java:2369)
> ~[na:na]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1782)
> ~[na:na]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1777)
> ~[na:na]
> at
> org.apache.nifi.processors.elasticsea
[jira] [Commented] (NIFI-1619) PutElasticSearch throws FlowFileHandlingException when id attribute is missing
[
https://issues.apache.org/jira/browse/NIFI-1619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15191071#comment-15191071
]
ASF GitHub Bot commented on NIFI-1619:
--
Github user apiri commented on the pull request:
https://github.com/apache/nifi/pull/269#issuecomment-195420771
reviewing
> PutElasticSearch throws FlowFileHandlingException when id attribute is missing
> --
>
> Key: NIFI-1619
> URL: https://issues.apache.org/jira/browse/NIFI-1619
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 0.5.1
>Reporter: Jim Wagoner
>Assignee: Matt Burgess
>Priority: Minor
> Labels: easyfix
> Fix For: 0.6.0
>
>
> When the processor can't find the id_attribute on the FlowFile while trying
> to be added to the Bulk Request it is transferred to failure relationship.
> The issue is after this happens the FlowFile is still added to the Bulk
> Request even though a invalid id will be used during the insert. This will
> cause the a failure to be returned to the BulkResponse and a attempt to
> transfer the already transferred FlowFile throws a FlowFileHandlingException .
> Looks like a "continue;" added to line 178 would solve this issue.
> STACK:
> 2016-03-11 07:49:39,053 ERROR [Timer-Driven Process Thread-12]
> o.a.n.p.elasticsearch.PutElasticsearch
> PutElasticsearch[id=9a22e93f-03d8-445f-95b8-7979d0016ee7] No value in
> identifier attribute ${metadataId} for
> StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1457700558252-17754,
> container=default, section=346], offset=648997,
> length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283]
> 2016-03-11 07:49:39,081 ERROR [Timer-Driven Process Thread-12]
> o.a.n.p.elasticsearch.PutElasticsearch
> PutElasticsearch[id=9a22e93f-03d8-445f-95b8-7979d0016ee7] Failed to insert
> into Elasticsearch due to
> StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1457700558252-17754,
> container=default, section=346], offset=648997,
> length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283]
> is already marked for transfer:
> org.apache.nifi.processor.exception.FlowFileHandlingException:
> StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1457700558252-17754,
> container=default, section=346], offset=648997,
> length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283]
> is already marked for transfer
> 2016-03-11 07:49:39,105 ERROR [Timer-Driven Process Thread-12]
> o.a.n.p.elasticsearch.PutElasticsearch
> org.apache.nifi.processor.exception.FlowFileHandlingException:
> StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1457700558252-17754,
> container=default, section=346], offset=648997,
> length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283]
> is already marked for transfer
> at
> org.apache.nifi.controller.repository.StandardProcessSession.validateRecordState(StandardProcessSession.java:2369)
> ~[na:na]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1782)
> ~[na:na]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1777)
> ~[na:na]
> at
> org.apache.nifi.processors.elasticsearch.PutElasticsearch.onTrigger(PutElasticsearch.java:171)
> ~[nifi-elasticsearch-processors-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> [nifi-api-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1139)
> [nifi-framework-core-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139)
> [nifi-framework-core-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49)
> [nifi-framework-core-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:124)
> [nifi-framework-core-0.5.1.jar:0.5.1]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> [na:1.7.0_95]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> [na:1.7.0_95]
> at
> java.util.concurrent.Sch
[jira] [Commented] (NIFI-1619) PutElasticSearch throws FlowFileHandlingException when id attribute is missing
[
https://issues.apache.org/jira/browse/NIFI-1619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15191115#comment-15191115
]
ASF GitHub Bot commented on NIFI-1619:
--
Github user apiri commented on the pull request:
https://github.com/apache/nifi/pull/269#issuecomment-195438823
Overall, looks good, was able to use it in conjunction with an ES docker
image and things seem to work as anticipated. Would be nice to see a unit test
capture the bug that crept up.
> PutElasticSearch throws FlowFileHandlingException when id attribute is missing
> --
>
> Key: NIFI-1619
> URL: https://issues.apache.org/jira/browse/NIFI-1619
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 0.5.1
>Reporter: Jim Wagoner
>Assignee: Matt Burgess
>Priority: Minor
> Labels: easyfix
> Fix For: 0.6.0
>
>
> When the processor can't find the id_attribute on the FlowFile while trying
> to be added to the Bulk Request it is transferred to failure relationship.
> The issue is after this happens the FlowFile is still added to the Bulk
> Request even though a invalid id will be used during the insert. This will
> cause the a failure to be returned to the BulkResponse and a attempt to
> transfer the already transferred FlowFile throws a FlowFileHandlingException .
> Looks like a "continue;" added to line 178 would solve this issue.
> STACK:
> 2016-03-11 07:49:39,053 ERROR [Timer-Driven Process Thread-12]
> o.a.n.p.elasticsearch.PutElasticsearch
> PutElasticsearch[id=9a22e93f-03d8-445f-95b8-7979d0016ee7] No value in
> identifier attribute ${metadataId} for
> StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1457700558252-17754,
> container=default, section=346], offset=648997,
> length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283]
> 2016-03-11 07:49:39,081 ERROR [Timer-Driven Process Thread-12]
> o.a.n.p.elasticsearch.PutElasticsearch
> PutElasticsearch[id=9a22e93f-03d8-445f-95b8-7979d0016ee7] Failed to insert
> into Elasticsearch due to
> StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1457700558252-17754,
> container=default, section=346], offset=648997,
> length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283]
> is already marked for transfer:
> org.apache.nifi.processor.exception.FlowFileHandlingException:
> StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1457700558252-17754,
> container=default, section=346], offset=648997,
> length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283]
> is already marked for transfer
> 2016-03-11 07:49:39,105 ERROR [Timer-Driven Process Thread-12]
> o.a.n.p.elasticsearch.PutElasticsearch
> org.apache.nifi.processor.exception.FlowFileHandlingException:
> StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1457700558252-17754,
> container=default, section=346], offset=648997,
> length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283]
> is already marked for transfer
> at
> org.apache.nifi.controller.repository.StandardProcessSession.validateRecordState(StandardProcessSession.java:2369)
> ~[na:na]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1782)
> ~[na:na]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1777)
> ~[na:na]
> at
> org.apache.nifi.processors.elasticsearch.PutElasticsearch.onTrigger(PutElasticsearch.java:171)
> ~[nifi-elasticsearch-processors-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> [nifi-api-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1139)
> [nifi-framework-core-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139)
> [nifi-framework-core-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49)
> [nifi-framework-core-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:124)
> [nifi-framework-core-0.5.1.jar:0.5.1]
> at
> java.util.concurrent.Executors$RunnableAdapter.c
[jira] [Commented] (NIFI-1619) PutElasticSearch throws FlowFileHandlingException when id attribute is missing
[
https://issues.apache.org/jira/browse/NIFI-1619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15191120#comment-15191120
]
ASF GitHub Bot commented on NIFI-1619:
--
Github user mattyb149 commented on the pull request:
https://github.com/apache/nifi/pull/269#issuecomment-195440926
good point, I just added one. Verified it failed on master, and passes with
the fix
> PutElasticSearch throws FlowFileHandlingException when id attribute is missing
> --
>
> Key: NIFI-1619
> URL: https://issues.apache.org/jira/browse/NIFI-1619
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 0.5.1
>Reporter: Jim Wagoner
>Assignee: Matt Burgess
>Priority: Minor
> Labels: easyfix
> Fix For: 0.6.0
>
>
> When the processor can't find the id_attribute on the FlowFile while trying
> to be added to the Bulk Request it is transferred to failure relationship.
> The issue is after this happens the FlowFile is still added to the Bulk
> Request even though a invalid id will be used during the insert. This will
> cause the a failure to be returned to the BulkResponse and a attempt to
> transfer the already transferred FlowFile throws a FlowFileHandlingException .
> Looks like a "continue;" added to line 178 would solve this issue.
> STACK:
> 2016-03-11 07:49:39,053 ERROR [Timer-Driven Process Thread-12]
> o.a.n.p.elasticsearch.PutElasticsearch
> PutElasticsearch[id=9a22e93f-03d8-445f-95b8-7979d0016ee7] No value in
> identifier attribute ${metadataId} for
> StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1457700558252-17754,
> container=default, section=346], offset=648997,
> length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283]
> 2016-03-11 07:49:39,081 ERROR [Timer-Driven Process Thread-12]
> o.a.n.p.elasticsearch.PutElasticsearch
> PutElasticsearch[id=9a22e93f-03d8-445f-95b8-7979d0016ee7] Failed to insert
> into Elasticsearch due to
> StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1457700558252-17754,
> container=default, section=346], offset=648997,
> length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283]
> is already marked for transfer:
> org.apache.nifi.processor.exception.FlowFileHandlingException:
> StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1457700558252-17754,
> container=default, section=346], offset=648997,
> length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283]
> is already marked for transfer
> 2016-03-11 07:49:39,105 ERROR [Timer-Driven Process Thread-12]
> o.a.n.p.elasticsearch.PutElasticsearch
> org.apache.nifi.processor.exception.FlowFileHandlingException:
> StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1457700558252-17754,
> container=default, section=346], offset=648997,
> length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283]
> is already marked for transfer
> at
> org.apache.nifi.controller.repository.StandardProcessSession.validateRecordState(StandardProcessSession.java:2369)
> ~[na:na]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1782)
> ~[na:na]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1777)
> ~[na:na]
> at
> org.apache.nifi.processors.elasticsearch.PutElasticsearch.onTrigger(PutElasticsearch.java:171)
> ~[nifi-elasticsearch-processors-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> [nifi-api-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1139)
> [nifi-framework-core-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139)
> [nifi-framework-core-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49)
> [nifi-framework-core-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:124)
> [nifi-framework-core-0.5.1.jar:0.5.1]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> [na:1.7.0_95]
> at java.util.concurrent.FutureTask.runAndRese
[jira] [Commented] (NIFI-1619) PutElasticSearch throws FlowFileHandlingException when id attribute is missing
[
https://issues.apache.org/jira/browse/NIFI-1619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15191181#comment-15191181
]
ASF GitHub Bot commented on NIFI-1619:
--
Github user apiri commented on the pull request:
https://github.com/apache/nifi/pull/269#issuecomment-195444057
Awesome, thanks. Will let Travis do its thing and get this merged in.
Thanks for patching this up.
> PutElasticSearch throws FlowFileHandlingException when id attribute is missing
> --
>
> Key: NIFI-1619
> URL: https://issues.apache.org/jira/browse/NIFI-1619
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 0.5.1
>Reporter: Jim Wagoner
>Assignee: Matt Burgess
>Priority: Minor
> Labels: easyfix
> Fix For: 0.6.0
>
>
> When the processor can't find the id_attribute on the FlowFile while trying
> to be added to the Bulk Request it is transferred to failure relationship.
> The issue is after this happens the FlowFile is still added to the Bulk
> Request even though a invalid id will be used during the insert. This will
> cause the a failure to be returned to the BulkResponse and a attempt to
> transfer the already transferred FlowFile throws a FlowFileHandlingException .
> Looks like a "continue;" added to line 178 would solve this issue.
> STACK:
> 2016-03-11 07:49:39,053 ERROR [Timer-Driven Process Thread-12]
> o.a.n.p.elasticsearch.PutElasticsearch
> PutElasticsearch[id=9a22e93f-03d8-445f-95b8-7979d0016ee7] No value in
> identifier attribute ${metadataId} for
> StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1457700558252-17754,
> container=default, section=346], offset=648997,
> length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283]
> 2016-03-11 07:49:39,081 ERROR [Timer-Driven Process Thread-12]
> o.a.n.p.elasticsearch.PutElasticsearch
> PutElasticsearch[id=9a22e93f-03d8-445f-95b8-7979d0016ee7] Failed to insert
> into Elasticsearch due to
> StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1457700558252-17754,
> container=default, section=346], offset=648997,
> length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283]
> is already marked for transfer:
> org.apache.nifi.processor.exception.FlowFileHandlingException:
> StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1457700558252-17754,
> container=default, section=346], offset=648997,
> length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283]
> is already marked for transfer
> 2016-03-11 07:49:39,105 ERROR [Timer-Driven Process Thread-12]
> o.a.n.p.elasticsearch.PutElasticsearch
> org.apache.nifi.processor.exception.FlowFileHandlingException:
> StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1457700558252-17754,
> container=default, section=346], offset=648997,
> length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283]
> is already marked for transfer
> at
> org.apache.nifi.controller.repository.StandardProcessSession.validateRecordState(StandardProcessSession.java:2369)
> ~[na:na]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1782)
> ~[na:na]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1777)
> ~[na:na]
> at
> org.apache.nifi.processors.elasticsearch.PutElasticsearch.onTrigger(PutElasticsearch.java:171)
> ~[nifi-elasticsearch-processors-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> [nifi-api-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1139)
> [nifi-framework-core-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139)
> [nifi-framework-core-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49)
> [nifi-framework-core-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:124)
> [nifi-framework-core-0.5.1.jar:0.5.1]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> [na:1.7.0_95]
> at java.util.concurrent.FutureTas
[jira] [Commented] (NIFI-1619) PutElasticSearch throws FlowFileHandlingException when id attribute is missing
[
https://issues.apache.org/jira/browse/NIFI-1619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15191220#comment-15191220
]
ASF GitHub Bot commented on NIFI-1619:
--
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/269
> PutElasticSearch throws FlowFileHandlingException when id attribute is missing
> --
>
> Key: NIFI-1619
> URL: https://issues.apache.org/jira/browse/NIFI-1619
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 0.5.1
>Reporter: Jim Wagoner
>Assignee: Matt Burgess
>Priority: Minor
> Labels: easyfix
> Fix For: 0.6.0
>
>
> When the processor can't find the id_attribute on the FlowFile while trying
> to be added to the Bulk Request it is transferred to failure relationship.
> The issue is after this happens the FlowFile is still added to the Bulk
> Request even though a invalid id will be used during the insert. This will
> cause the a failure to be returned to the BulkResponse and a attempt to
> transfer the already transferred FlowFile throws a FlowFileHandlingException .
> Looks like a "continue;" added to line 178 would solve this issue.
> STACK:
> 2016-03-11 07:49:39,053 ERROR [Timer-Driven Process Thread-12]
> o.a.n.p.elasticsearch.PutElasticsearch
> PutElasticsearch[id=9a22e93f-03d8-445f-95b8-7979d0016ee7] No value in
> identifier attribute ${metadataId} for
> StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1457700558252-17754,
> container=default, section=346], offset=648997,
> length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283]
> 2016-03-11 07:49:39,081 ERROR [Timer-Driven Process Thread-12]
> o.a.n.p.elasticsearch.PutElasticsearch
> PutElasticsearch[id=9a22e93f-03d8-445f-95b8-7979d0016ee7] Failed to insert
> into Elasticsearch due to
> StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1457700558252-17754,
> container=default, section=346], offset=648997,
> length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283]
> is already marked for transfer:
> org.apache.nifi.processor.exception.FlowFileHandlingException:
> StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1457700558252-17754,
> container=default, section=346], offset=648997,
> length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283]
> is already marked for transfer
> 2016-03-11 07:49:39,105 ERROR [Timer-Driven Process Thread-12]
> o.a.n.p.elasticsearch.PutElasticsearch
> org.apache.nifi.processor.exception.FlowFileHandlingException:
> StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1457700558252-17754,
> container=default, section=346], offset=648997,
> length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283]
> is already marked for transfer
> at
> org.apache.nifi.controller.repository.StandardProcessSession.validateRecordState(StandardProcessSession.java:2369)
> ~[na:na]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1782)
> ~[na:na]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1777)
> ~[na:na]
> at
> org.apache.nifi.processors.elasticsearch.PutElasticsearch.onTrigger(PutElasticsearch.java:171)
> ~[nifi-elasticsearch-processors-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> [nifi-api-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1139)
> [nifi-framework-core-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139)
> [nifi-framework-core-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49)
> [nifi-framework-core-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:124)
> [nifi-framework-core-0.5.1.jar:0.5.1]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> [na:1.7.0_95]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> [na:1.7.0_95]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTa
[jira] [Commented] (NIFI-1619) PutElasticSearch throws FlowFileHandlingException when id attribute is missing
[
https://issues.apache.org/jira/browse/NIFI-1619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15191219#comment-15191219
]
ASF subversion and git services commented on NIFI-1619:
---
Commit e1cf37fb8953cc04a8f92382b558cf89c1563a5d in nifi's branch
refs/heads/master from [~mattyb149]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=e1cf37f ]
NIFI-1619: Fix Elasticsearch processor bug when flow file missing ID attribute
This closes #269
Signed-off-by: Aldrin Piri
> PutElasticSearch throws FlowFileHandlingException when id attribute is missing
> --
>
> Key: NIFI-1619
> URL: https://issues.apache.org/jira/browse/NIFI-1619
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 0.5.1
>Reporter: Jim Wagoner
>Assignee: Matt Burgess
>Priority: Minor
> Labels: easyfix
> Fix For: 0.6.0
>
>
> When the processor can't find the id_attribute on the FlowFile while trying
> to be added to the Bulk Request it is transferred to failure relationship.
> The issue is after this happens the FlowFile is still added to the Bulk
> Request even though a invalid id will be used during the insert. This will
> cause the a failure to be returned to the BulkResponse and a attempt to
> transfer the already transferred FlowFile throws a FlowFileHandlingException .
> Looks like a "continue;" added to line 178 would solve this issue.
> STACK:
> 2016-03-11 07:49:39,053 ERROR [Timer-Driven Process Thread-12]
> o.a.n.p.elasticsearch.PutElasticsearch
> PutElasticsearch[id=9a22e93f-03d8-445f-95b8-7979d0016ee7] No value in
> identifier attribute ${metadataId} for
> StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1457700558252-17754,
> container=default, section=346], offset=648997,
> length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283]
> 2016-03-11 07:49:39,081 ERROR [Timer-Driven Process Thread-12]
> o.a.n.p.elasticsearch.PutElasticsearch
> PutElasticsearch[id=9a22e93f-03d8-445f-95b8-7979d0016ee7] Failed to insert
> into Elasticsearch due to
> StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1457700558252-17754,
> container=default, section=346], offset=648997,
> length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283]
> is already marked for transfer:
> org.apache.nifi.processor.exception.FlowFileHandlingException:
> StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1457700558252-17754,
> container=default, section=346], offset=648997,
> length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283]
> is already marked for transfer
> 2016-03-11 07:49:39,105 ERROR [Timer-Driven Process Thread-12]
> o.a.n.p.elasticsearch.PutElasticsearch
> org.apache.nifi.processor.exception.FlowFileHandlingException:
> StandardFlowFileRecord[uuid=2fad6fa6-f773-4ac1-b2c3-788e398122a3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1457700558252-17754,
> container=default, section=346], offset=648997,
> length=7283],offset=0,name=ce9f46e6-c89b-46d0-bede-bd111d05380e.xml,size=7283]
> is already marked for transfer
> at
> org.apache.nifi.controller.repository.StandardProcessSession.validateRecordState(StandardProcessSession.java:2369)
> ~[na:na]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1782)
> ~[na:na]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1777)
> ~[na:na]
> at
> org.apache.nifi.processors.elasticsearch.PutElasticsearch.onTrigger(PutElasticsearch.java:171)
> ~[nifi-elasticsearch-processors-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> [nifi-api-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1139)
> [nifi-framework-core-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139)
> [nifi-framework-core-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49)
> [nifi-framework-core-0.5.1.jar:0.5.1]
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:124)
> [nifi-framework-core-0.5.1.jar:0.5.1]
> at
> java.util.concurrent.Executors
