+1 for the general idea of fixit day/week for Gradle. Agree with what Łukasz said. Some of these performance tests are new and are flaky due to other issues that were discovered during the process of adding the test.
I think the high level blocker is updating performance testing framework to use Gradle. This will involve adding Gradle-based logic for invoking PerfKitBenchmaker, for example [1], and updating PerfKitBenchmarker to issue a Gradle command for running the benchmark [2]. First task will be to find the work needed here and updating the relevant JIRA [3]. Thanks, Cham [1] https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/pom.xml#L79 [2] https://github.com/GoogleCloudPlatform/PerfKitBenchmarker/blob/master/perfkitbenchmarker/beam_benchmark_helper.py [3] https://issues.apache.org/jira/browse/BEAM-3251 On Thu, Mar 8, 2018 at 3:10 AM Łukasz Gajowy <lukasz.gaj...@gmail.com> wrote: > 2018-03-07 22:29 GMT+01:00 Kenneth Knowles <k...@google.com>: > >> >> Based on https://builds.apache.org/view/A-D/view/Beam/ and our failure >> spam level the performance tests are mostly not healthy anyhow. So is there >> any high level blocker to switching them or is it just someone sitting down >> with each one? >> >> > I thought I could share my point of view here as I am working on the > Performance Test part for a while now. I wouldn't say those are "mostly not > healthy". The situation is as follows: > > - totally not working: Python, Spark performance tests (don't know why, > I'm not maintaining those tests. Should we disable them?) > - flaky: the recently re-enabled JDBC Performance Test. It's seems to be > flaky mostly due to: https://issues.apache.org/jira/browse/BEAM-3798 > - working well, rarely failing: AvroIO, TextIO, Compressed Text, TFRecordIO > > Also, some test failures are caused due to pending PRs (we sometimes run > concrete tests to check if PRs won't break them). This also causes failures > sometimes. > > I can help with switching the Performance tests to Gradle as this part > seems to be free to take. > > > Łukasz > >
