Hi Gaspar, I believe this is because 4294967295 is a too big value that cannot be contained in an integer. I am not a ES expert but you should look at your applicationId type if you can change it.
Hope this helps. 2016-11-07 19:09 GMT+01:00 Gaspar, Carson <[email protected]>: > It stays in the queue on a field type error, e.g.: > [2016-11-07 12:37:17,263][DEBUG][action.bulk ] [Cybele] > [estreamer_index_2016.11.07][0] failed to execute bulk item (index) index > {[estreamer_index_2016.11.07][estreamer_doc][AVg_3ZmnIxO_esWdRpuM], source > ...data... > MapperParsingException[failed to parse [applicationId]]; nested: > NumberFormatException[For input string: "4294967295"]; > at org.elasticsearch.index.mapper.FieldMapper.parse( > FieldMapper.java:329) > at org.elasticsearch.index.mapper.DocumentParser. > parseObjectOrField(DocumentParser.java:311) > at org.elasticsearch.index.mapper.DocumentParser. > parseValue(DocumentParser.java:438) > at org.elasticsearch.index.mapper.DocumentParser. > parseObject(DocumentParser.java:264) > at org.elasticsearch.index.mapper.DocumentParser. > parseDocument(DocumentParser.java:124) > at org.elasticsearch.index.mapper.DocumentMapper.parse( > DocumentMapper.java:309) > at org.elasticsearch.index.shard.IndexShard.prepareCreate( > IndexShard.java:529) > at org.elasticsearch.index.shard.IndexShard. > prepareCreateOnPrimary(IndexShard.java:506) > at org.elasticsearch.action.index.TransportIndexAction. > prepareIndexOperationOnPrimary(TransportIndexAction.java:214) > at org.elasticsearch.action.index.TransportIndexAction. > executeIndexRequestOnPrimary(TransportIndexAction.java:223) > at org.elasticsearch.action.bulk.TransportShardBulkAction. > shardIndexOperation(TransportShardBulkAction.java:327) > at org.elasticsearch.action.bulk.TransportShardBulkAction. > shardOperationOnPrimary(TransportShardBulkAction.java:120) > at org.elasticsearch.action.bulk.TransportShardBulkAction. > shardOperationOnPrimary(TransportShardBulkAction.java:68) > at org.elasticsearch.action.support.replication. > TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction. > java:648) > at org.elasticsearch.common.util.concurrent.AbstractRunnable. > run(AbstractRunnable.java:37) > at org.elasticsearch.action.support.replication. > TransportReplicationAction$PrimaryOperationTransportHandl > er.messageReceived(TransportReplicationAction.java:279) > at org.elasticsearch.action.support.replication. > TransportReplicationAction$PrimaryOperationTransportHandl > er.messageReceived(TransportReplicationAction.java:271) > at org.elasticsearch.transport.RequestHandlerRegistry. > processMessageReceived(RequestHandlerRegistry.java:77) > at org.elasticsearch.transport.TransportService$4.doRun( > TransportService.java:376) > at org.elasticsearch.common.util.concurrent.AbstractRunnable. > run(AbstractRunnable.java:37) > at java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.NumberFormatException: For input string: "4294967295" > at java.lang.NumberFormatException.forInputString( > NumberFormatException.java:65) > at java.lang.Integer.parseInt(Integer.java:583) > at java.lang.Integer.parseInt(Integer.java:615) > at org.elasticsearch.common.xcontent.support. > AbstractXContentParser.intValue(AbstractXContentParser.java:127) > at org.elasticsearch.index.mapper.core.IntegerFieldMapper. > innerParseCreateField(IntegerFieldMapper.java:293) > at org.elasticsearch.index.mapper.core.NumberFieldMapper. > parseCreateField(NumberFieldMapper.java:241) > at org.elasticsearch.index.mapper.FieldMapper.parse( > FieldMapper.java:321) > ... 22 more > > -----Original Message----- > From: Koji Kawamura [mailto:[email protected]] > Sent: Monday, November 07, 2016 12:10 AM > To: [email protected] > Subject: Re: PutElasticsearchHttp error behaviour > > Hello Gaspar, > > I looked at the PutElasticsearchHttp code, it distinguishes following two > types of error: > > 1. Error occurs when the processor tries to connect Elasticsearch, but it > couldn't. E.g. the specified Elasticsearch URL is not correct and > java.net.ConnectException is thrown. > When this happens, the processor throws ProcessException, and NiFi > framework keeps the FlowFile in the queue, so that it can be retried later. > This indicates that DataFlowManager(user) has to update the processor > configuration to make it work properly. > > 2. The processor successfully sends a request to Elasticsearch, but > Elasticsearch returns error response. This happens with operations such as > updating with a document Id that doesn't exist in the target index. > When this happens, the processor routes the incoming FlowFile which caused > the error, to failure relationship. > > I think the error you encountered the former type of error above (#1). > If the error was returned from Elasticsearch, it should be routed to > failure relationship as you suspected. Would you please let us know about > the specific error you got? A StackTrace would be helpful. > > BTW, PutElasticsearchHttp also has REL_RETRY relationship, but it seems > it's not used currently. > > Thanks! > Koji > > On Thu, Nov 3, 2016 at 5:47 AM, Gaspar, Carson <[email protected]> wrote: > > When PutElasticsearchHttp gets an error from Elasticsearch, it > > _appears_ to never dequeue the problem flow file and just try it over > > and over again. It certainly doesn’t route the problem flow file to > > the retry or failure queues. This seems wrong to me – am I missing > something? > > > > Nifi is from the Hortonworks > > nifi_2_0_0_0_579-1.0.0.2.0.0.0-579.el6.x86_64 > > RPM. > > > > >
