[ https://issues.apache.org/jira/browse/BEAM-3942?focusedWorklogId=89424&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89424 ]
ASF GitHub Bot logged work on BEAM-3942: ---------------------------------------- Author: ASF GitHub Bot Created on: 10/Apr/18 14:42 Start Date: 10/Apr/18 14:42 Worklog Time Spent: 10m Work Description: lgajowy commented on a change in pull request #5003: [BEAM-3942] Update performance testing framework to use Gradle URL: https://github.com/apache/beam/pull/5003#discussion_r180446870 ########## File path: build_rules.gradle ########## @@ -593,6 +593,118 @@ ext.applyJavaNature = { } } +// Reads and contains all necessary performance test parameters +class JavaPerformanceTestConfiguration { + String pkbLocation = System.getProperty('pkbLocation') + + String logLevel = System.getProperty('logLevel', 'INFO') + String gradleBinary = System.getProperty('gradleBinary', './gradlew') + String isOfficial = System.getProperty('official', 'true') + String benchmarks = System.getProperty('benchmarks', 'beam_integration_benchmark') + + String beamPrebuilt = System.getProperty('beamPrebuilt', 'true') + String beamSdk = System.getProperty('beamSdk', 'java') + + String timeout = System.getProperty('itTimeout', '1200') + + String kubeconfig = System.getProperty('kubeconfig', System.getProperty('user.home') + '/.kube/config') + String kubectl = System.getProperty('kubectl', 'kubectl') + + // PerfKit will have trouble reading 'null' path. It expects empty string if no scripts are expected. + String kubernetesScripts = System.getProperty('kubernetesScripts', '') + + String integrationTestPipelineOptions = System.getProperty('integrationTestPipelineOptions') + + // PerfKit will have trouble reading 'null' path. It expects empty string if no config file is expected. + String optionsConfigFile = System.getProperty('beamITOptions', '') + + String integrationTest = System.getProperty('integrationTest') + String itModule = System.getProperty('itModule') + + String runner = System.getProperty('integrationTestRunner', 'direct') + + String extraProperties = System.getProperty('beamExtraProperties', '') +} + +// Configures a project with a set of tasks needed for running performance tests +ext.enableJavaPerformanceTesting = { + println "enableJavaPerformanceTesting with ${it ? "$it" : "default configuration"} for project ${project.name}" + + // Use the implicit it parameter of the closure to handle zero argument or one argument map calls. + JavaPerformanceTestConfiguration configuration = it ? it as JavaPerformanceTestConfiguration : new JavaPerformanceTestConfiguration() + + // Add runners needed to run integration tests on + task packageIntegrationTests(type: Jar) { + if (gradle.startParameter.taskNames.contains('integrationTest')) { Review comment: This line effectively means that we should include the runner dependencies only when the `integrationTest` task is run. So when user runs `./gradlew integrationTest...` those dependencies are added while building the jar. Now I see that this is not enough due the following reason: there is no way to "prebuild" tests - we cannot build with a separate build command and then just run the tests. Such prebuilding is needed by perfkit so it's a necessity I forgot to take into account here. "Prebuilding" is invoked when we specify the "beam_prebuilt" flag to false, see more in [Perfkit's code](https://github.com/GoogleCloudPlatform/PerfKitBenchmarker/blob/a3e6c0382809cb68cd294f34cbd5b3b558e53f56/perfkitbenchmarker/beam_benchmark_helper.py#L147). I think we should remove this condition and include the runner dependency only if the command is run with appropriate "integrationTestRunner" property . I changed this in the next commit. So now we can prebuild in Perfkit using this command: `./gradlew clean build -xtest` -DintegrationTestRunner=dataflow and run the tests using `integrationTest` task, for example like this: and execute the tests using this one: `./gradlew clean integrationTest -p sdks/java/io/file-based-io-tests/ -DintegrationTestPipelineOptions='[]' --tests org.apache.beam.sdk.io.text.TextIOIT -DintegrationTestRunner=dataflow` WDYT? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking ------------------- Worklog Id: (was: 89424) Time Spent: 5h 20m (was: 5h 10m) > Update performance testing framework to use Gradle. > --------------------------------------------------- > > Key: BEAM-3942 > URL: https://issues.apache.org/jira/browse/BEAM-3942 > Project: Beam > Issue Type: Sub-task > Components: testing > Reporter: Chamikara Jayalath > Assignee: Ćukasz Gajowy > Priority: Major > Time Spent: 5h 20m > Remaining Estimate: 0h > > This requires performing updates to PerfKitBenchmarker and Beam so that we > can execute performance tests using Gradle. -- This message was sent by Atlassian JIRA (v7.6.3#76005)