Is there somewhere we could document this? On Thu, Sep 15, 2022 at 6:45 AM Moritz Mack <mm...@talend.com> wrote:
> Thank you, Andrew! > > Exactly what I was looking for, that’s awesome! > > > > On 15.09.22, 06:37, "Alexey Romanenko" <aromanenko....@gmail.com> wrote: > > > > > > Ahh, great! I didn’t know that 'beam-perf’ label is used for that. > Thanks! > > > On 14 Sep 2022, at 17:47, Andrew Pilloud <apill...@apache.org> wrote: > > > > We do have a dedicated machine for benchmarks. This is a single > > machine limited to running one test at a time. Set the > > jenkinsExecutorLabel for the job to 'beam-perf' to use it. For > > example: > > > https://urldefense.com/v3/__https://github.com/apache/beam/blob/66bbee84ed477d86008905646e68b100591b6f78/.test-infra/jenkins/job_PostCommit_Java_Nexmark_Direct.groovy*L36__;Iw!!CiXD_PY!Qat2J4NAyHVo4Cc32PKMn50yw8LgWHmEOm4Ltb7aRV-7KCfNamu3tGOiSYKDUZhLHKu3zlqbBXzJNiX_f_Qteg$ > <https://urldefense.com/v3/__https:/github.com/apache/beam/blob/66bbee84ed477d86008905646e68b100591b6f78/.test-infra/jenkins/job_PostCommit_Java_Nexmark_Direct.groovy*L36__;Iw!!CiXD_PY!Qat2J4NAyHVo4Cc32PKMn50yw8LgWHmEOm4Ltb7aRV-7KCfNamu3tGOiSYKDUZhLHKu3zlqbBXzJNiX_f_Qteg$> > > > > > Andrew > > > > On Wed, Sep 14, 2022 at 8:28 AM Alexey Romanenko > > <aromanenko....@gmail.com> wrote: > >> > >> I think it depends on the goal why to run that benchmarks. In ideal > case, we need to run them on the same dedicated machine(s) and with the > same configuration all the time but I’m not sure that it can be achieved in > current infrastructure reality. > >> > >> On the other hand, IIRC, the initial goal of benchmarks, like Nexmark, > was to detect fast any major regressions, especially between releases, that > are not so sensitive to ideal conditions. And here we a field for > improvements. > >> > >> — > >> Alexey > >> > >> On 13 Sep 2022, at 22:57, Kenneth Knowles <k...@apache.org> wrote: > >> > >> Good idea. I'm curious about our current benchmarks. Some of them run > on clusters, but I think some of them are running locally and just being > noisy. Perhaps this could improve that. (or if they are running on local > Spark/Flink then maybe the results are not really meaningful anyhow) > >> > >> On Tue, Sep 13, 2022 at 2:54 AM Moritz Mack <mm...@talend.com> wrote: > >>> > >>> Hi team, > >>> > >>> > >>> > >>> I’m looking for some help to setup infrastructure to periodically run > Java microbenchmarks (JMH). > >>> > >>> Results of these runs will be added to our community metrics > (InfluxDB) to help us track performance, see [1]. > >>> > >>> > >>> > >>> To prevent noisy runs this would require a dedicated Jenkins machine > that runs at most one job (benchmark) at a time. Benchmark runs take quite > some time, but on the other hand they don’t have to run very frequently > (once a week should be fine initially). > >>> > >>> > >>> > >>> Thanks so much, > >>> > >>> Moritz > >>> > >>> > >>> > >>> [1] > https://urldefense.com/v3/__https://github.com/apache/beam/pull/23041__;!!CiXD_PY!Qat2J4NAyHVo4Cc32PKMn50yw8LgWHmEOm4Ltb7aRV-7KCfNamu3tGOiSYKDUZhLHKu3zlqbBXzJNiUkaqlEKQ$ > <https://urldefense.com/v3/__https:/github.com/apache/beam/pull/23041__;!!CiXD_PY!Qat2J4NAyHVo4Cc32PKMn50yw8LgWHmEOm4Ltb7aRV-7KCfNamu3tGOiSYKDUZhLHKu3zlqbBXzJNiUkaqlEKQ$> > > >>> > >>> As a recipient of an email from Talend, your contact personal data > will be on our systems. Please see our privacy notice. > >>> > >>> > >>> > >> > > *As a recipient of an email from Talend, your contact personal data will > be on our systems. Please see our privacy notice. > <https://www.talend.com/privacy/>* > > >
