potiuk commented on PR #33680: URL: https://github.com/apache/airflow/pull/33680#issuecomment-1695110419
> I am a strong advocate for allowing referencing Airflow conn_id. From functionality point of view it makes no sense that we have a connection details stored in Airflow but for K8s pod users will have to redefine it somewhere else. I am with Elad here. That's why https://github.com/apache/airflow/pull/33680#issuecomment-1692475222 comment. I think we can easily make such secrets 100% compatible with what Airflow's expectation is. > Yes but it wasn't that much of a pain. It's a simple script to run once a week and it takes less than 1 min to finish. See above: >> or the cleanup, there were different solutions, one of them is patching the ownerReferences for the secret after creating the pod, in this case the secret will be automatically killed when we kill the pod. > That one looks pretty cool actually Seems that `ownerReference` might be the way to go - no cleanup necessary and the secret will get automatically deleted when the owner (i.e. our k8s POD) is gone. Not sure if that is possible to set (I am afraid we might have a chicken-egg problem, but maybe: 1) creating secret 2) creating pod 3) patching secret with ovnerReference of the pod created Is the way to go. The only small race is if our code gets killed between 2) and 3) but this can be either neglected or cleanup script mentioned by Elad could help to cope with the edge case. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
