[ https://issues.apache.org/jira/browse/BEAM-7772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kamil Wasilewski resolved BEAM-7772. ------------------------------------ Fix Version/s: Not applicable Resolution: Fixed > Stop using Perfkit Benchmarker tool in all tests > ------------------------------------------------ > > Key: BEAM-7772 > URL: https://issues.apache.org/jira/browse/BEAM-7772 > Project: Beam > Issue Type: Improvement > Components: testing > Reporter: Lukasz Gajowy > Priority: P2 > Labels: stale-P2 > Fix For: Not applicable > > Time Spent: 6h 50m > Remaining Estimate: 0h > > [Devlist thread > link|https://lists.apache.org/thread.html/dab1c093799248787e8b75e63b66d7389b594b649a4d9a4a5db1cfbb@%3Cdev.beam.apache.org%3E] > > Currently Python, IOIT and some Dataflow and Spark performance tests are > relying on Perfkit Benchmarker tool. Due to the reasons discussed on the > devlist it was decided to remove it from Beam's tests. > Problems that we face currently: > # Changes to Gradle tasks/build configuration in the Beam codebase have to > be reflected in Perfkit code. This required PRs to Perfkit which can last and > the tests break due to this sometimes (no change in Perfkit + change already > there in beam = incompatibility). This is what happened in PR 8919 (above), > # Can't run in Python3 (depends on python 2 only library like functools32), > # Black box testing which hard to collect pipeline related metrics, > # Measurement of run time is inaccurate, > # It offers relatively small elasticity in comparison with eg. Jenkins tasks > in terms of setting up the testing infrastructure (runners, databases). For > example, if we'd like to setup Flink runner, and reuse it in consequent tests > in one go, that would be impossible. We can easily do this in Jenkins. > Tests that use Perfkit: > # IO integration tests, > # Python performance tests, > # beam_PerformanceTests_Dataflow (disabled), > # beam_PerformanceTests_Spark (failing constantly - looks not maintained). > -- This message was sent by Atlassian Jira (v8.3.4#803005)