Austin -the flink benchmark is for testing Flink on single machines and not a cluster.
I did see this https://oceanrep.geomar.de/50729/1/bsc_nico_biernat_thesis.pdf but it is more for testing the Scaling of Flink instead of testing throughput and latency. On Mon, Nov 8, 2021 at 10:54 AM Vijay Balakrishnan <bvija...@gmail.com> wrote: > Thx, Austin. I was hoping there might be a newer benchmark run similar to > done by dataArtisans on Flink in 2016(old). > https://www.ververica.com/blog/extending-the-yahoo-streaming-benchmark > > Looks like Yahoo Streaming benchmark was an initial standard in 2016. > Hoping to see something updated for late 2021. > > TIA, > Vijay > > On Fri, Nov 5, 2021 at 3:16 PM Austin Cawley-Edwards < > austin.caw...@gmail.com> wrote: > >> Hi Vijay, >> >> I'm not too familiar with the subject, but maybe you could have a look at >> the flink-faker[1], which generates fake data. I would think you could use >> it to write to kafka in one Flink job, and then have another Flink job to >> ingest and run your benchmarks. >> >> There is also this microbenchmark repo[2], perhaps that could be useful >> to run on different CPUs. >> >> Hope those help, >> Austin >> >> [1]: https://github.com/knaufk/flink-faker >> [2]: https://github.com/apache/flink-benchmarks >> >> On Fri, Nov 5, 2021 at 5:14 PM Vijay Balakrishnan <bvija...@gmail.com> >> wrote: >> >>> Hi, >>> I am a newbie to running a performance benchmark load test of Flink on >>> new CPUs. >>> Is there an* existing workload generator* that I can use with Kafka and >>> then ingest it with Flink KafkaConnector & test the performance against >>> various new chips on servers ? >>> >>> Measuring CPU performance etc, vCPU usage, Latency, throughput etc. >>> Pls pardon my ignorance in a lot of these performance related topics. >>> >>> TIA, >>> Vijay >>> >>
