Re: [PR] fix: delete pod when base container completes [airflow]

2024-08-01 Thread via GitHub
github-actions[bot] commented on PR #39694: URL: https://github.com/apache/airflow/pull/39694#issuecomment-2264253160 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for

Re: [PR] fix: delete pod when base container completes [airflow]

2024-05-21 Thread via GitHub
aptenodytes-forsteri commented on code in PR #39694: URL: https://github.com/apache/airflow/pull/39694#discussion_r1608712949 ## airflow/providers/cncf/kubernetes/utils/pod_manager.py: ## @@ -618,7 +615,8 @@ def await_pod_completion( remote_pod = self.read_pod(pod)

Re: [PR] fix: delete pod when base container completes [airflow]

2024-05-21 Thread via GitHub
jonathan-ostrander commented on code in PR #39694: URL: https://github.com/apache/airflow/pull/39694#discussion_r1608702091 ## airflow/providers/cncf/kubernetes/utils/pod_manager.py: ## @@ -618,7 +615,8 @@ def await_pod_completion( remote_pod = self.read_pod(pod)

Re: [PR] fix: delete pod when base container completes [airflow]

2024-05-21 Thread via GitHub
jonathan-ostrander commented on code in PR #39694: URL: https://github.com/apache/airflow/pull/39694#discussion_r1608686830 ## airflow/providers/cncf/kubernetes/utils/pod_manager.py: ## @@ -618,7 +615,8 @@ def await_pod_completion( remote_pod = self.read_pod(pod)

Re: [PR] fix: delete pod when base container completes [airflow]

2024-05-21 Thread via GitHub
aptenodytes-forsteri commented on code in PR #39694: URL: https://github.com/apache/airflow/pull/39694#discussion_r1608675167 ## airflow/providers/cncf/kubernetes/utils/pod_manager.py: ## @@ -618,7 +615,8 @@ def await_pod_completion( remote_pod = self.read_pod(pod)

Re: [PR] fix: delete pod when base container completes [airflow]

2024-05-21 Thread via GitHub
jonathan-ostrander commented on code in PR #39694: URL: https://github.com/apache/airflow/pull/39694#discussion_r1608579117 ## airflow/providers/cncf/kubernetes/operators/pod.py: ## @@ -842,16 +838,14 @@ def cleanup(self, pod: k8s.V1Pod, remote_pod: k8s.V1Pod): if

Re: [PR] fix: delete pod when base container completes [airflow]

2024-05-20 Thread via GitHub
jonathan-ostrander commented on code in PR #39694: URL: https://github.com/apache/airflow/pull/39694#discussion_r1606765260 ## airflow/providers/cncf/kubernetes/operators/pod.py: ## @@ -842,16 +838,14 @@ def cleanup(self, pod: k8s.V1Pod, remote_pod: k8s.V1Pod): if

Re: [PR] fix: delete pod when base container completes [airflow]

2024-05-18 Thread via GitHub
dirrao commented on PR #39694: URL: https://github.com/apache/airflow/pull/39694#issuecomment-2119110636 closes: #39693 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] fix: delete pod when base container completes [airflow]

2024-05-18 Thread via GitHub
dirrao commented on code in PR #39694: URL: https://github.com/apache/airflow/pull/39694#discussion_r1605947085 ## airflow/providers/cncf/kubernetes/operators/pod.py: ## @@ -842,16 +838,14 @@ def cleanup(self, pod: k8s.V1Pod, remote_pod: k8s.V1Pod): if self._killed or

Re: [PR] fix: delete pod when base container completes [airflow]

2024-05-18 Thread via GitHub
dirrao commented on code in PR #39694: URL: https://github.com/apache/airflow/pull/39694#discussion_r1605947085 ## airflow/providers/cncf/kubernetes/operators/pod.py: ## @@ -842,16 +838,14 @@ def cleanup(self, pod: k8s.V1Pod, remote_pod: k8s.V1Pod): if self._killed or

Re: [PR] fix: delete pod when base container completes [airflow]

2024-05-18 Thread via GitHub
dirrao commented on code in PR #39694: URL: https://github.com/apache/airflow/pull/39694#discussion_r1605947085 ## airflow/providers/cncf/kubernetes/operators/pod.py: ## @@ -842,16 +838,14 @@ def cleanup(self, pod: k8s.V1Pod, remote_pod: k8s.V1Pod): if self._killed or

Re: [PR] fix: delete pod when base container completes [airflow]

2024-05-18 Thread via GitHub
dirrao commented on PR #39694: URL: https://github.com/apache/airflow/pull/39694#issuecomment-2119102465 > @ashb @dirrao That should happen when the pod is deleted. [According to the k8s docs, when a pod is marked for deletion the containers in the pod are issued a TERM with a 30 second

Re: [PR] fix: delete pod when base container completes [airflow]

2024-05-18 Thread via GitHub
jonathan-ostrander commented on PR #39694: URL: https://github.com/apache/airflow/pull/39694#issuecomment-2119039284 @ashb That should happen when the pod is deleted. [According to the k8s docs, when a pod is marked for deletion the containers in the pod are issued a TERM with a 30 second

Re: [PR] fix: delete pod when base container completes [airflow]

2024-05-18 Thread via GitHub
jonathan-ostrander commented on code in PR #39694: URL: https://github.com/apache/airflow/pull/39694#discussion_r1605915610 ## airflow/providers/cncf/kubernetes/operators/pod.py: ## @@ -842,16 +838,14 @@ def cleanup(self, pod: k8s.V1Pod, remote_pod: k8s.V1Pod): if

Re: [PR] fix: delete pod when base container completes [airflow]

2024-05-17 Thread via GitHub
dirrao commented on code in PR #39694: URL: https://github.com/apache/airflow/pull/39694#discussion_r1605659696 ## airflow/providers/cncf/kubernetes/operators/pod.py: ## @@ -842,16 +838,14 @@ def cleanup(self, pod: k8s.V1Pod, remote_pod: k8s.V1Pod): if self._killed or

Re: [PR] fix: delete pod when base container completes [airflow]

2024-05-17 Thread via GitHub
jonathan-ostrander commented on PR #39694: URL: https://github.com/apache/airflow/pull/39694#issuecomment-2118361921 @jedcunningham I'm not sure I understand your point about `OnFinishAction.KEEP_POD` or why the istio side car needs to be treated as a special case. Shouldn't that finish

Re: [PR] fix: delete pod when base container completes [airflow]

2024-05-17 Thread via GitHub
jedcunningham commented on PR #39694: URL: https://github.com/apache/airflow/pull/39694#issuecomment-2118342696 Yeah, it's tough to 1) do it generally 2) not delete pods and 3) not leave the pods running. That's why istio has (or tried to have it turns out) special handling. I'm

Re: [PR] fix: delete pod when base container completes [airflow]

2024-05-17 Thread via GitHub
jedcunningham commented on PR #39694: URL: https://github.com/apache/airflow/pull/39694#issuecomment-2118321909 I did a little digging, and that scenario was never covered when the feature was added in #33306 . I don't think we want to leave these pods still running. -- This is

Re: [PR] fix: delete pod when base container completes [airflow]

2024-05-17 Thread via GitHub
jedcunningham commented on PR #39694: URL: https://github.com/apache/airflow/pull/39694#issuecomment-2118312136 Did you test `on_finish_action="keep_pod"`? We don't always delete the pods. I doubt we can do away with our istio handling like this, and I suspect that got broken somehow

Re: [PR] fix: delete pod when base container completes [airflow]

2024-05-17 Thread via GitHub
boring-cyborg[bot] commented on PR #39694: URL: https://github.com/apache/airflow/pull/39694#issuecomment-2118226821 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our

[PR] fix: delete pod when base container completes [airflow]

2024-05-17 Thread via GitHub
jonathan-ostrander opened a new pull request, #39694: URL: https://github.com/apache/airflow/pull/39694 Closes #39693 Special logic was added to `KubernetesPodOperator`'s lifecycle to handle the case where an istio proxy sidecar is running and preventing the pod from completing, but