[
https://issues.apache.org/jira/browse/BEAM-6528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Oliveira resolved BEAM-6528.
-----------------------------------
Resolution: Fixed
Fix Version/s: Not applicable
> Dataflow jobs failing in multiple PostCommits, including Java PostCommit.
> -------------------------------------------------------------------------
>
> Key: BEAM-6528
> URL: https://issues.apache.org/jira/browse/BEAM-6528
> Project: Beam
> Issue Type: Bug
> Components: runner-dataflow, test-failures
> Reporter: Daniel Oliveira
> Assignee: Daniel Oliveira
> Priority: Critical
> Labels: currently-failing, flake
> Fix For: Not applicable
>
>
> * [Jenkins
> Job|https://builds.apache.org/job/beam_PostCommit_Python_Verify/7219/]
> * [Gradle Build Scan|https://scans.gradle.com/s/cdjxkpm7dgtxm]
> Initial investigation:
> It seems this test has failed four times just today with flakes in various
> integration tests. I've linked to the first instance of this failure above.
> In the Dataflow dashboard I see this error:
> {noformat}
> Startup of the worker pool in zone us-central1-f failed to bring up any of
> the desired 1 workers. ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS: The zone
> 'projects/apache-beam-testing/zones/us-central1-f' does not have enough
> resources available to fulfill the request. '(resource
> type:pd-standard)'.{noformat}
> Possibly some quota issue?
>
> Edit:
> I'm updating this bug because it seems like this isn't restricted to
> beam_PostCommit_Python_Verify after all. I've now seen this happening in
> various java postcommits:
> [beam_PostCommit_Java|https://builds.apache.org/job/beam_PostCommit_Java/2466/]
>
> [beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow|https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow/617/]
>
> [beam_PostCommit_Java_ValidatesRunner_Dataflow|https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/1959/]
> In each case the error looks something like this:
> {noformat}
> java.lang.RuntimeException: Startup of the worker pool in zone us-central1-f
> failed to bring up any of the desired 1 workers.
> ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS: The zone
> 'projects/apache-beam-testing/zones/us-central1-f' does not have enough
> resources available to fulfill the request. '(resource
> type:pd-standard)'.Workflow failed.{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
