mrvarmazyar commented on issue #66158: URL: https://github.com/apache/airflow/issues/66158#issuecomment-4807416452
@tirkarthi We can reproduce this consistently on Airflow 3.2.1 with KubernetesExecutor. Here's how: Setup: - Executor: KubernetesExecutor - Triggerer: 1 replica - Remote logging: GCS enabled - Deferrable operator pattern: task defers to a custom trigger that polls an XCom key written by an upstream task (watcher pattern) Reproduction steps: 1. Create a deferrable task that calls `self.defer(trigger=MyTrigger(...))` mid-execution 2. Open the Airflow UI → task log while the task is in deferred state 3. Observe: logs stop exactly at the `self.defer()` call, nothing from the triggerer polling phase appears What you see: - Task log shows initial setup lines, then cuts off at deferral - No triggerer polling output visible in UI during deferred state - After task completes: logs remain cut off (triggerer phase still missing) - Task `state=success` in DB, normal execution duration, the task DID run correctly What makes it consistent for us: - Happens on every run, not intermittent - Triggerer pod is healthy, port 8794 is exposed and log server is running (verified via `curl http://<triggerer-pod-ip>:8794`) - GCS log files exist but the triggerer portion (`.log.trigger.*.log`) is absent or empty **My hypothesis:** the socket pipeline between triggerer subprocess and structlog is dropping messages before they reach the file writer, consistent with what this issue describes. The missing file on GCS (not just missing in UI) confirms it's not a rendering issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
