potiuk commented on PR #33680: URL: https://github.com/apache/airflow/pull/33680#issuecomment-1692475222
Looking at the discussion... just a thought (and I think it comes very closely to a thought that @bolkedebruin had about making standalone tasks without the task havint to reach out to the DB of airflow). Why don't we make those secrets to be COMPATIBLE with the way airflow tasks use it ? I believe K8S secrets can be mapped to environment variables.... Why don't we map the connection secrets to Kubernetes secrets that are in the format of environment variables that follow Airflow's connection defiinitions for env variables? If we can map such connections to AIRFLOW__CONNECTION_XXXX=<encoded url for connection> - then we can simply run any airflow-compatible Hook as K8S Pod and it will automatically retrieve connnection from the env variable. So basically we could run any Airflow Hook inside the K8S Pod run via K8SPodOperator and that Hook would **just work** (without accessing the DB of airflow - just reading the env variable coming from the mounted secret. Unless it was already going to work like that and I misunderstood it .. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
