[ https://issues.apache.org/jira/browse/NIFI-11480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Sampson reassigned NIFI-11480: ------------------------------------ Assignee: Chris Sampson > PutElasticsearchRecord should have an option to output _bulk api response > errors as flowfile attributes > ------------------------------------------------------------------------------------------------------- > > Key: NIFI-11480 > URL: https://issues.apache.org/jira/browse/NIFI-11480 > Project: Apache NiFi > Issue Type: Improvement > Reporter: Chris Sampson > Assignee: Chris Sampson > Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > https://github.com/apache/nifi/pull/6903 for NIFI-11111 introduced a > [conversation|https://github.com/apache/nifi/pull/6903#issuecomment-1513872398] > about outputting the response error details for Records that are not > processed by Elasticsearch. > The same PR introduces a new {{elasticsearch.bulk.error}} attribute for the > {{PutElasticsearchJson}} processor, but explains why [it's not so simple for > PutElasticsearchRecord|https://github.com/apache/nifi/pull/6903#issuecomment-1514554132] > due to input FlowFiles potentially containing many Records and there being > no obvious way of expressing error details for all such Records in the single > output flowfile. > One [suggested > approach|https://github.com/apache/nifi/pull/6903#issuecomment-1517903668] > would be to "partition" the output {{errors}} Records into multiple > flowfiles, grouped by the error {{type}} provided by Elasticsearch. This > {{type}} could then be added to the flowfile(s) as the > {{elasticsearch.bulk.error}} attribute. Flows could then {{RouteOnAttribute}} > if they wanted to handle certain Elasticsearch errors in particular ways. > Leaving all errors flowfiles in the same output queue avoids the problem of > the [large (and changing) number of potential Elasticsearch error > types|https://github.com/apache/nifi/pull/6903#issuecomment-1517863606] > Such output partitioning (if implemented) should be optional, driven by a > processor property that maintains the current "all in one" flowfile output by > default. -- This message was sent by Atlassian Jira (v8.20.10#820010)