[
https://issues.apache.org/jira/browse/BEAM-5931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16676936#comment-16676936
]
Lukasz Gajowy commented on BEAM-5931:
-------------------------------------
Hi! Thanks for taking care of this.
I debugged/tested Jenkins files in Pull requests - afaik this is the only
sensible way right now. Local Dockerized Jenkins does not have all the
necessary software installed inside (Perfkit, I think some of the plugins are
missing too) and you still won't be sure if everything in the docker container
is set up the same way as on Jenkins. I've seen some idea towards [migrating
our Jenkins
workers|https://docs.google.com/document/d/1y0YuQj_oZXC0uM5-gniG7r9-5gv2uiDhzbtgYYJW48c/edit#heading=h.30j0zll]
to Docker containers - maybe in the future, this can be used to clone beam's
Jenkins on local machines but afaik now it is not possible.
As for the second question: I added suggestions to your draft pr:
[https://github.com/apache/beam/pull/6921/files]. IMO "performanceTest" task is
not the one that you should modify. It simply wraps perfkit and is a high level
"interface" for users. What Perfkit invokes to run the tests is the
"integrationTest" task ([here's
proof|https://github.com/GoogleCloudPlatform/PerfKitBenchmarker/blob/7dd217450d2efd19213f5e9cd831bd4007d69cae/perfkitbenchmarker/beam_benchmark_helper.py#L197])
- so it's more "low level" and fits better such beam's implementation details
(IMO). See my suggestions in the PR and let me know if it's clear and feasible.
> Rollback PR/6899
> ----------------
>
> Key: BEAM-5931
> URL: https://issues.apache.org/jira/browse/BEAM-5931
> Project: Beam
> Issue Type: Task
> Components: beam-model, runner-dataflow
> Reporter: Luke Cwik
> Assignee: Ruoyun Huang
> Priority: Major
> Time Spent: 2h
> Remaining Estimate: 0h
>
> To rollback this change, one must either:
> 1) Update nexmark / perf test framework to use Dataflow worker jar.
> This requires adding the
> {code}
> "--dataflowWorkerJar=${dataflowWorkerJar}",
> "--workerHarnessContainerImage=",
> {code}
> when running the tests.
> OR
> 2) Update the dataflow worker image with code that contains the rollback of
> PR/6899 and then rollback PR/6899 in Github with the updated Dataflow worker
> image.
> #1 is preferable since we will no longer have tests running that don't use a
> Dataflow worker jar built from Github HEAD.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
