Robert Conroy created AIRFLOW-6522:
--------------------------------------
Summary: Sensors in reschedule mode with S3TaskHandler can cause
log duplication
Key: AIRFLOW-6522
URL: https://issues.apache.org/jira/browse/AIRFLOW-6522
Project: Apache Airflow
Issue Type: Bug
Components: logging
Affects Versions: 1.10.6
Reporter: Robert Conroy
Assignee: Robert Conroy
With sensors using {{reschedule}} mode and {{S3TaskHandler}} for logging, the
task instance log gets a bunch of duplicate messages. I believe this is
happening because contents of the local log file are appended to what's already
in S3. The local log file may contain log messages that have already been
uploaded to S3 if the task is sent back to a worker that had already processed
a poke for that task instance.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)