Thanks Piotr, does Flink read/write to zookeeper every time it process a record? I thought only JM uses ZK to keep some meta level data, not sure why `it depends on many things like state backend used, state size, complexity of your application, size of the records, number of machines, their hardware and the network.`
On Thu, Oct 12, 2017 at 1:35 AM Piotr Nowojski <pi...@data-artisans.com> wrote: > Hi, > > Are you asking how to measure records/s or is it possible to achieve it? > To measure it you can check numRecordsInPerSecond metric. > > As far if 1000 records/s is possible, it depends on many things like state > backend used, state size, complexity of your application, size of the > records, number of machines, their hardware and the network. In the very > simplest cases it is possible to achieve millions of records per second per > machine. It would be best to try it out in your particular use case on some > small scale. > > Piotrek > > > On 11 Oct 2017, at 19:58, Hao Sun <ha...@zendesk.com> wrote: > > > > Hi Is there a way to estimate read/write traffic between flink and zk? > > I am looking for something like 1000 reads/sec or 1000 writes/sec. And > the size of the message. > > > > Thanks > >
