Thank you, Andrew! Exactly what I was looking for, that’s awesome! On 15.09.22, 06:37, "Alexey Romanenko" <[email protected]> wrote:
Ahh, great! I didn’t know that 'beam-perf’ label is used for that. Thanks! > On 14 Sep 2022, at 17:47, Andrew Pilloud <[email protected]> wrote: > > We do have a dedicated machine for benchmarks. This is a single > machine limited to running one test at a time. Set the > jenkinsExecutorLabel for the job to 'beam-perf' to use it. For > example: > https://urldefense.com/v3/__https://github.com/apache/beam/blob/66bbee84ed477d86008905646e68b100591b6f78/.test-infra/jenkins/job_PostCommit_Java_Nexmark_Direct.groovy*L36__;Iw!!CiXD_PY!Qat2J4NAyHVo4Cc32PKMn50yw8LgWHmEOm4Ltb7aRV-7KCfNamu3tGOiSYKDUZhLHKu3zlqbBXzJNiX_f_Qteg$<https://urldefense.com/v3/__https:/github.com/apache/beam/blob/66bbee84ed477d86008905646e68b100591b6f78/.test-infra/jenkins/job_PostCommit_Java_Nexmark_Direct.groovy*L36__;Iw!!CiXD_PY!Qat2J4NAyHVo4Cc32PKMn50yw8LgWHmEOm4Ltb7aRV-7KCfNamu3tGOiSYKDUZhLHKu3zlqbBXzJNiX_f_Qteg$> > > Andrew > > On Wed, Sep 14, 2022 at 8:28 AM Alexey Romanenko > <[email protected]> wrote: >> >> I think it depends on the goal why to run that benchmarks. In ideal case, we >> need to run them on the same dedicated machine(s) and with the same >> configuration all the time but I’m not sure that it can be achieved in >> current infrastructure reality. >> >> On the other hand, IIRC, the initial goal of benchmarks, like Nexmark, was >> to detect fast any major regressions, especially between releases, that are >> not so sensitive to ideal conditions. And here we a field for improvements. >> >> — >> Alexey >> >> On 13 Sep 2022, at 22:57, Kenneth Knowles <[email protected]> wrote: >> >> Good idea. I'm curious about our current benchmarks. Some of them run on >> clusters, but I think some of them are running locally and just being noisy. >> Perhaps this could improve that. (or if they are running on local >> Spark/Flink then maybe the results are not really meaningful anyhow) >> >> On Tue, Sep 13, 2022 at 2:54 AM Moritz Mack <[email protected]> wrote: >>> >>> Hi team, >>> >>> >>> >>> I’m looking for some help to setup infrastructure to periodically run Java >>> microbenchmarks (JMH). >>> >>> Results of these runs will be added to our community metrics (InfluxDB) to >>> help us track performance, see [1]. >>> >>> >>> >>> To prevent noisy runs this would require a dedicated Jenkins machine that >>> runs at most one job (benchmark) at a time. Benchmark runs take quite some >>> time, but on the other hand they don’t have to run very frequently (once a >>> week should be fine initially). >>> >>> >>> >>> Thanks so much, >>> >>> Moritz >>> >>> >>> >>> [1] >>> https://urldefense.com/v3/__https://github.com/apache/beam/pull/23041__;!!CiXD_PY!Qat2J4NAyHVo4Cc32PKMn50yw8LgWHmEOm4Ltb7aRV-7KCfNamu3tGOiSYKDUZhLHKu3zlqbBXzJNiUkaqlEKQ$<https://urldefense.com/v3/__https:/github.com/apache/beam/pull/23041__;!!CiXD_PY!Qat2J4NAyHVo4Cc32PKMn50yw8LgWHmEOm4Ltb7aRV-7KCfNamu3tGOiSYKDUZhLHKu3zlqbBXzJNiUkaqlEKQ$> >>> >>> As a recipient of an email from Talend, your contact personal data will be >>> on our systems. Please see our privacy notice. >>> >>> >>> >> As a recipient of an email from Talend, your contact personal data will be on our systems. Please see our privacy notice. <https://www.talend.com/privacy/>
