Hi, Folks, I am working on fixes for BEAM-5931 <https://issues.apache.org/jira/projects/BEAM/issues/BEAM-5931>. Two Jenkins tests were affected. One is Nexmark, for which I’ve fixed in PR#6916 <https://github.com/apache/beam/pull/6916>. The other one is PerformanceTest_TextIO <https://github.com/apache/beam/blob/master/.test-infra/jenkins/job_PerformanceTests_FileBasedIO_IT.groovy>, where we’d like to apply similar changes, and this is the place I have trouble finding the best fix.
Would like to have suggestions before spending too much time trying things unnecessarily creative, while there are easier alternatives available. I’ve also created a draft PR#6921 <https://github.com/apache/beam/pull/6921/files> showing what I am about to do. The main issue is that in PerformanceTest_TextIO <https://github.com/apache/beam/blob/master/.test-infra/jenkins/job_PerformanceTests_FileBasedIO_IT.groovy> jenkins job uses shell to call python, to run another github project, based on which then run performance test related classes. Question#1: We will have to do a gradle shadowJar build, but how to pass the path string (i.e. project(“”).shadowJar.ArchivePath) between jenkins and gradle? Either way (jenkins -> gradle or gradle -> jenkins) in theory should work, but which one would be more easier? One idea is to use environment variables, but so far I have trouble making environment variable work across gradle task and shell task. Question#2: Is CommonJobProperties.buildPerformanceTest() the right place to update at all? It would be much much easier if we can just update other places (instead of jenkins job). Any suggestions appreciated and feel free to just comment on the draft PR#6921 <https://github.com/apache/beam/pull/6921/files>. Thanks! -- ================ Ruoyun Huang