GitHub user Throne3d edited a comment on the discussion: Worker logs not reachable from webserver while running
I had this problem for a while, and didn't want to switch to StatefulSet. My SRE team tries to avoid stateful applications in our main Kubernetes cluster, instead using external persistence stores, and I don't know enough to contradict it or what the other implications of this change would be :D Here's the best solution I could come up with! <details><summary>(Investigations I performed before reaching this solution)</summary> I dug into the DNSes being set up, and patched in a `subdomain: airflow-triggerer` field on the Deployment, but the hostnames never included the pod name, only the pod IP. After adding the subdomain, I was able to see the DNS entries via: ```bash $ kubectl run ubuntu --image=ubuntu:latest --it --rm -- /bin/bash root@ubuntu:/# apt update && apt install -y dnsutils ... root@ubuntu:/# nslookup -type=SRV airflow-triggerer.airflow-kube-namespace.svc.cluster.local ... airflow-triggerer.airflow-kube-namespace.svc.cluster.local service = 0 100 8794 10-2-3-4.airflow-triggerer.airflow-kube-namespace.svc.cluster.local. ``` (indicating a pod at IP `10.2.3.4`, but not including the pod's full name, and not matching the value of `hostname -A` in the triggerer pod itself) Maybe some of this information will be helpful if someone wants to set up the DNS entries without switching to IP. </details> **Ultimately**, I saw that we can avoid DNS lookup and just [access the worker/triggerer pods directly via IP](https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#hostname-callable) (following [the hint above about hostname_callable](https://github.com/apache/airflow/discussions/27558#discussioncomment-4087826)), which solved my issue: ```yml # in the helm values.yaml config: core: hostname_callable: "airflow.utils.net.get_host_ip_address" ``` Noting this down in case it's useful for anyone else in the future =) GitHub link: https://github.com/apache/airflow/discussions/27558#discussioncomment-11608491 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
