Thanks for your suggestions. It's always good to reach the dev list! You're
right that we should focus more on what are we trying to test rather than
providing huge loads.
To stay transparent for everyone:
"what is it we're trying to test?"
I talked with some testing experts from the Dataflow te
Agreeing with Robert about "what is it we're trying to test?". Would a
smaller performance test find the same issues, faster and more reliably?
We have seen issues with the apache-beam-testing project exceeding quota
during dataflow jobs, resulting in spurious failures during precommits and
postco
I like the idea of creating separate project(s) for load tests so as
to not compete with other tests and the standard development cycle.
As for how many workers is too many, I would take the track "what is
it we're trying to test?" Unless your stress-testing the shuffle
itself, much of what Beam d
Hi,
pinging this thread (maybe some folks missed it). What do you think about
those concerns/ideas?
Łukasz
pon., 14 sty 2019 o 17:11 Łukasz Gajowy napisał(a):
> Hi all,
>
> one problem we need to solve while working with load tests we currently
> develop is that we don't really know how much G
Hi all,
one problem we need to solve while working with load tests we currently
develop is that we don't really know how much GCP/Jenkins resources can we
occupy. We did some initial testing with
beam_Java_LoadTests_GroupByKey_Dataflow_Small[1] and it seems that for:
- 1 000 000 000 (~ 23 GB) syn
