These jobs also require Dataflow which has various quotas on resource usage. I hit these while working on the Dataflow Nexmark tests for SQL. I'm not sure what the quota is on the account that Jenkins uses, but the default quota will max out at around 2 concurrent jobs.
Andrew On Wed, May 2, 2018 at 1:16 PM Scott Wegner <[email protected]> wrote: > While working on tuning our Gradle build for Jenkins, I noticed that our > Jenkins jobs often get queued up, even though we have low utilization of > our Jenkins executor pool [2]. For example, right now I see 4 instances of > beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle queued [1], while the > 'beam' label currently has 29 out of 32 executors available [2]. > > Is it possible to configure our Jenkins jobs to execute concurrently on > separate nodes? I'm not sure whether it's safe for the same job to execute > concurrently on the same node, for example if runner resources are shared. > But I believe executing on separate nodes should be safe. > > Who would know more about this or have access to the Jenkins config if we > did want to make changes? > > [1] > https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/ > [2] > https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/ > > -- > > > Got feedback? http://go/swegner-feedback >
