() with a "counter" variable, that is held in a "ValueState", this counter will always be in sync with the number of elements in the kafka topic (because the counter
is reset on a failure).
On Tue, Feb 21, 2017 at 4:04 PM, Y. Sakamoto <phonypian...@gmail.com
<mailto:phon
k's state to Elastic. That way the writes to ElasticSearch
become "idempotent", which means duplicates simple make no additional changes.
Hope that helps!
Stephan
On Mon, Feb 20, 2017 at 5:53 PM, Y. Sakamoto <phonypian...@gmail.com
<mailto:phonypian...@gmail.com>> wro
Hi,
I'm using Flink 1.2.0 and try to do "exactly once" data transfer
from Kafka to Elasticsearch, but I cannot.
(Scala 2.11, Kafka 0.10, without YARN)
There are 2 Flink TaskManager nodes, and when processing
with 2 parallelism, shutdown one of them (simulating node failure).
Using
