Re: [PR] Connecting airflow containers with shareProcessNamespace [airflow]

2023-11-22 Thread via GitHub
pankajkoti commented on PR #35748: URL: https://github.com/apache/airflow/pull/35748#issuecomment-1823904072 @amoghrajesh just if you wonder if Jed is jot able to respond anytime soon, Jed is on parental leave 👶 -- This is an automated message from the Apache Git Service. To respond to th

Re: [PR] Connecting airflow containers with shareProcessNamespace [airflow]

2023-11-22 Thread via GitHub
amoghrajesh commented on PR #35748: URL: https://github.com/apache/airflow/pull/35748#issuecomment-1823829483 > I think we have ways to generate different resources for 1.28+ . I believe helm takes K8s version into account when generating resources. Look in our chart I think we have a few c

Re: [PR] Connecting airflow containers with shareProcessNamespace [airflow]

2023-11-21 Thread via GitHub
potiuk commented on PR #35748: URL: https://github.com/apache/airflow/pull/35748#issuecomment-1820964266 > Thanks for letting me know about this. Was not very aware of this. If this is the case, we should consider this as "THE ONLY" solution along with lock file as you mentioned for earlier

Re: [PR] Connecting airflow containers with shareProcessNamespace [airflow]

2023-11-21 Thread via GitHub
amoghrajesh commented on PR #35748: URL: https://github.com/apache/airflow/pull/35748#issuecomment-1820451617 >So I'd say we should treat native side-containers as THE solution and the others are merely workarounds for versions of kubernwtes that do not have native side containers yet.

Re: [PR] Connecting airflow containers with shareProcessNamespace [airflow]

2023-11-21 Thread via GitHub
amoghrajesh commented on PR #35748: URL: https://github.com/apache/airflow/pull/35748#issuecomment-1820450496 >I also think native sidecar is the long term way to go. It's been long time in the making - KEP for it has been discussed for 3 years at least - and it handles many edge cases and

Re: [PR] Connecting airflow containers with shareProcessNamespace [airflow]

2023-11-20 Thread via GitHub
potiuk commented on PR #35748: URL: https://github.com/apache/airflow/pull/35748#issuecomment-1819032429 I think we have ways to generate different resources for 1.28+ . I believe helm takes K8s version into account when generating resources. Look in our chart I think we have a few cases o

Re: [PR] Connecting airflow containers with shareProcessNamespace [airflow]

2023-11-20 Thread via GitHub
amoghrajesh commented on PR #35748: URL: https://github.com/apache/airflow/pull/35748#issuecomment-1818986629 >we should figure out a different way how to communicate with the kerberos refreshing process and stop it - one of the options is to let the running process be stoppable by sending

Re: [PR] Connecting airflow containers with shareProcessNamespace [airflow]

2023-11-20 Thread via GitHub
amoghrajesh commented on PR #35748: URL: https://github.com/apache/airflow/pull/35748#issuecomment-1818979974 Thank you for the detailed review @potiuk. This is extremely helpful to me. > Another option (probably simpler to implement) is to keep some kind of "shutdown" lock file in t

Re: [PR] Connecting airflow containers with shareProcessNamespace [airflow]

2023-11-20 Thread via GitHub
potiuk commented on PR #35748: URL: https://github.com/apache/airflow/pull/35748#issuecomment-1818945067 Shared process namespace in this case kind of violates the whole assumption the separate kerberos container here is introduced for. The whole idea is that "airflow" containers do not see

Re: [PR] Connecting airflow containers with shareProcessNamespace [airflow]

2023-11-19 Thread via GitHub
amoghrajesh commented on code in PR #35748: URL: https://github.com/apache/airflow/pull/35748#discussion_r1398696229 ## chart/templates/workers/worker-deployment.yaml: ## @@ -98,6 +98,9 @@ spec: {{- toYaml $podAnnotations | nindent 8 }} {{- end }} spec:

[PR] Connecting airflow containers with shareProcessNamespace [airflow]

2023-11-19 Thread via GitHub
amoghrajesh opened a new pull request, #35748: URL: https://github.com/apache/airflow/pull/35748 As a part of the effort to enable kerberos to work e2e with airflow in a kubernetes deployment, some changes related to the pod template and worker deployment have gone in recent