Thanks Damon, I appreciate the data-driven effort you are making to test different approaches to rate limiting in Beam.
On Thu, Feb 9, 2023 at 12:54 AM Damon Douglas <douglas.da...@gmail.com> wrote: > Hello Everyone, > > The following exploratory study proposal aims to evaluate existing > features of Beam to prevent web service API overusage. API providers > typically design for application workloads smaller than parallelized data > processing. This presents a challenge when using these resources to read > from and write to in the context of Beam. > > The study defines primary and secondary measures, identifying experimental > groups based on a key Beam feature applied to its pipeline design, such as > windowing or the State API. The data will foster evidence based approaches > to designing a solution to this problem. > > > https://docs.google.com/document/d/1VZ9YphDO7kewBSz5oMXVPHWaib3S03Z6aZ66BhciB3E/edit?usp=sharing&resourcekey=0-ItxMSG72EzfSwVedSz-Zeg > > Best, > > Damon >
