The GitHub Actions job "Tests (AMD)" on airflow.git/fix/beam-deferrable-dataflow-missing-job-id-68279 has failed. Run started by GitHub user GayathriSrividya (triggered by GayathriSrividya).
Head commit for run: f8117237ac3ddc355953064c14cc6f65802cbb01 / Gayathri Srividya Rajavarapu <[email protected]> Fix deferrable Beam/Dataflow operator fails when job-id absent from launcher stdout When a DataflowRunner pipeline does not configure INFO-level logging the Beam SDK's 'Created job with id: [...]' line is suppressed, so JOB_ID_PATTERN never matches and self.dataflow_job_id stays None. The synchronous wait_for_done() path already handles this by resolving the job by name prefix; the deferrable path did not, so it deferred with job_id=None and the Dataflow API immediately rejected the trigger with '400 Request must contain a job and project id.' Fix: before constructing the trigger in execute_on_dataflow() for both BeamRunPythonPipelineOperator and BeamRunJavaPipelineOperator, if dataflow_job_id is still None call a new DataflowHook.get_job_id_by_name() helper that looks up the job by name prefix via the Dataflow REST API. This mirrors the existing synchronous fallback and ensures the trigger always receives a valid ID. Closes #68279 Report URL: https://github.com/apache/airflow/actions/runs/27870310598 With regards, GitHub Actions via GitBox --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
