Hi All, I'm currently attempting to get a topology running for data into Elasticsearch. Tuples go through some minimal marshalling and preprocessing before being sent to partitionPersist, where they are transformed into JSON and indexed in Elasticsearch.
The cluster appears to work properly in local mode, but when deployed to my 4 node cluster, state updates do not seem to fire correctly (sometimes they don't fire at all). Tuple counter filters show data flowing through the topology at a healthy rate (approx 80,000 rec/second), however, the updateState function only rarely appears to be called. After a brief period of time, no further calls to updateState are seen. As a test, I wrote a filter that queues up tuples and batch sends them to Elasticsearch once a certain threshold is reached. This works perfectly fine and is capable of managing the processing load. I've seen discussion of this behavior before, but have not managed to find an explanation or solution. Has anybody else had similar issues or have a solution?
