Hi Is there a way to estimate read/write traffic between flink and zk?
I am looking for something like 1000 reads/sec or 1000 writes/sec. And the
size of the message.
Thanks
Hi,
Are you asking how to measure records/s or is it possible to achieve it? To
measure it you can check numRecordsInPerSecond metric.
As far if 1000 records/s is possible, it depends on many things like state
backend used, state size, complexity of your application, size of the records,
numbe
Thanks Piotr, does Flink read/write to zookeeper every time it process a
record?
I thought only JM uses ZK to keep some meta level data, not sure why `it
depends on many things like state backend used, state size, complexity of
your application, size of the records, number of machines, their hardwa
Hi,
I think Zookeeper is only used as a meta data store in HA mode. Interactions
with ZK are not part of the per-record stream processing code paths of Flink.
Things that are written to ZK can (also depending on your job) include e.g. the
job graph, Kafka offsets, or the meta data about availab
Great, thanks for the info, Stefan.
On Thu, Nov 16, 2017, 01:59 Stefan Richter
wrote:
> Hi,
>
> I think Zookeeper is only used as a meta data store in HA mode.
> Interactions with ZK are not part of the per-record stream processing code
> paths of Flink. Things that are written to ZK can (also d