[ https://issues.apache.org/jira/browse/AIRFLOW-6843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kaxil Naik resolved AIRFLOW-6843. --------------------------------- Fix Version/s: 2.0.0 Resolution: Fixed > Add grace_period_seconds config option for delete_namespaced_pod kube client > request > ------------------------------------------------------------------------------------ > > Key: AIRFLOW-6843 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6843 > Project: Apache Airflow > Issue Type: Improvement > Components: executor-kubernetes > Affects Versions: 1.10.9 > Reporter: Pete DeJoy > Assignee: Pete DeJoy > Priority: Major > Fix For: 2.0.0 > > > If you're using the Kubernetes executor, Amazon EKS deletes pods very quickly > after the tasks complete. This can be an issue if you're scraping Airflow > logs from a service like FluentD, as it means that the task pod gets deleted > before FluentD can pick up logs for fast-running (or fast-failing) tasks. > The [`kube_client_request_args` |#kube-client-request-args]environment > variable is passed to the [`delete_namespaced_pod`|#L443] client request, but > passing the necessary [`grace_period_seconds`|#delete_namespaced_pod] as a > [`post_param`|#L108] on that object causes other client requests to fail, as > that `kube_client_request_args` is passed as a kwarg to all client requests, > even those that don't have a `post_params` option for `grace_period_seconds`. > In order to provide a config option that fixes this issue, there needs to be > a new configurable environment variable > `AIRFLOW__KUBERNETES____DELETE_POD_GRACE_PERIOD_SECONDS` option that we pass > to the [`delete_namespaced_pod`|#L443]] client request as its own argument. > This will be defaulted to 0. -- This message was sent by Atlassian Jira (v8.3.4#803005)