andytrigg opened a new issue #10026: URL: https://github.com/apache/airflow/issues/10026
**Apache Airflow version**: 1.10.11 **Kubernetes version (if you are using kubernetes)** (use `kubectl version`): NA **Environment**: - **Cloud provider or hardware configuration**: AWS (EC2 instances) - **OS** (e.g. from /etc/os-release): NAME="Amazon Linux" VERSION="2" ID="amzn" ID_LIKE="centos rhel fedora" VERSION_ID="2" PRETTY_NAME="Amazon Linux 2" ANSI_COLOR="0;33" CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2" HOME_URL="https://amazonlinux.com/"; - **Kernel** (e.g. `uname -a`): Linux airflow-scheduler-10-229-13-220 4.14.165-131.185.amzn2.x86_64 #1 SMP Wed Jan 15 14:19:56 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux - **Install tools**: - **Others**: **What happened**: When manually invoke a task from the task details dialog, we see the task running for approximately 22 seconds before we see the following appear in the log... `[2020-07-28 01:25:14,726] {local_task_job.py:150} WARNING - Recorded pid 26940 does not match the current pid 26751 [2020-07-28 01:25:14,728] {helpers.py:325} INFO - Sending Signals.SIGTERM to GPID 26757` The task then is killed. We notice this is accompanied with a second failure shortly afterwards that correlates to the new pid that has been written to the `task_instance` table. It is interesting to note that if the task is scheduled as part of a normal dag run, or by clearing state and allowing the schedular to schedule its execution then we do not experience any issue. We have attempted to specify `task_concurrency` on our operators with no effect. **What you expected to happen**: We expected a single process to be spawned for the manually executed task. **How to reproduce it**: Manually invoke a task via the task details dialog where that task execution is going to be longer than the heart rate interval that has been set. The heart rate checks the pid and sees a mismatch and so kills the task. **Anything else we need to know**: We can produce this reliably if the task execution time is > than the heart rate interval. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
