GitHub user atulgandhi1137 closed a discussion: GKEStartJobOperator
(deferrable=True) does not show container logs on failure
Question:
I’m using GKEStartJobOperator to run jobs on GKE and noticed a difference in
logging behavior depending on whether deferrable is enabled.
What I’m seeing:
- With deferrable=True:
On success → container logs are visible in Airflow task logs
On failure (e.g., BackOffLimitExceeded) → only job-level error is
shown, container logs are missing
- With deferrable=False:
On failure → container logs are correctly shown in task logs
My questions:
- Is this expected behavior when using deferrable=True?
- Or should container logs also be available on failure in deferrable mode?
Why this matters:
When jobs fail, not having container logs makes debugging difficult.
If this is expected, is there a recommended workaround to fetch container logs
in deferrable mode?
GitHub link: https://github.com/apache/airflow/discussions/65647
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
