Hi Andrew and Pranav, Thanks for pointing out the current infrastructure we have in Beam. I have investigated the current Performance tests of Beam IOs in the Beam repo and summarized the current tools and infrastructure we have in this document: https://docs.google.com/document/d/11CgNVtyZSipoRiJ2O57hhqShUw_FQDTj6rEOzudwsK4/edit# . I also put some brief designs about how can we incorporate them into the test framework proposed by Pranav.
Best, Yi On Tue, Sep 27, 2022 at 12:06 PM Alexey Romanenko <[email protected]> wrote: > Thanks for raising this topic. > > > On 26 Sep 2022, at 23:32, Andrew Pilloud via dev <[email protected]> > wrote: > > > > I left some comments on your design. Your doc discusses a bunch of > > details about infrastructure such as testing frameworks, automation, > > and performance databases, but doesn't describe how it will fit in > > with our existing infrastructure (Load Tests, Nexmark, Jenkins, > > InfluxDB, Grafina). I would suspect we actually have most of the > > infrastructure already built? > > Right, I’m second on this question. We already have an infrastructure > ready to run a bunch of different benchmarks/load tests and > collect/present/analyse the results. Of course, there is a field for > improvements, but it would be great to take this into account and add the > details how this benchmark can be integrated into (to avoid a double job > for further support). > > > — > Alexey > > > On Mon, Sep 26, 2022 at 9:07 AM Pranav Bhandari > > <[email protected]> wrote: > >> > >> Hello, > >> > >> Hope this email finds you well. I have attached a link to a doc which > discusses the design for a performance and cost benchmarking framework to > be used by Beam IOs and Google-provided dataflow templates. > >> > >> Please feel free to comment on the doc with any questions, concerns or > ideas you might have. > >> > >> Thank you, > >> Pranav Bhandari > >> > >> > >> > https://docs.google.com/document/d/14GatBilwuR4jJGb-ZNpYeuB-KkVmDvEm/edit?usp=sharing&ouid=102139643796739130048&rtpof=true&sd=true > >
