There isn't a great answer to this. It is something that has been on my mind we multi-tenant installations, but it is unsolved for now.
On Sep 25, 2017 7:30 AM, "Simone D'Andreta" <simone.dandr...@gmail.com> wrote: > I need to be able to overwrite the resolv.conf per pods. If I tweak the > kube-dns configmap, I will have additional nameservers per node, not per > pods. > The scenario is: I have a kubernetes cluster which is shared between > different teams, and each team has his own namespace to deploy pods. Anyway > these pods must be able to communicate with different consul clusters which > are in different environments. Since they are in different envs, we need to > point to different DNS. Since pods get dns resolutions from the nodes, but > the nodes are shared, we need a way to overwrite the dns settings per pods. > I know this is a bad practice - I get what Tim said above in this thread - > but this is just a proof of concept and we'd like to know whether there is > a way to do this (the cleaner the better :D) > Thanks > Simone > > Il giorno lunedì 25 settembre 2017 15:15:43 UTC+2, Rodrigo Campos ha > scritto: >> >> Can you explain what do you what to achieve? >> >> Maybe changing the configmap kube-dns uses it's enough (and is prepared >> to be changed easily, now). >> >> On Monday, September 25, 2017, Simone D'Andreta <simone....@gmail.com> >> wrote: >> >>> Hi Tim, >>> I know what you mean and that's definitely a big issue on our side. This >>> is for us more a proof of concept to understand if we can go this way. >>> Since I am not a big expert with Kubernetes, I wanted to know if there are >>> solutions I haven't considered that might solve my problem. >>> That said, I thank you a lot for your explanations here. I found >>> something that can help me, though it sets dns names at node level - not >>> per pod: >>> https://kubernetes.io/docs/tasks/administer-cluster/dns-cust >>> om-nameservers/ >>> If I include the different IPs in the stub domains I definitely should >>> get an answer for the service discovery part. Not ideal, but I think better >>> than mounting and overriding the resolv.conf. >>> Cheers >>> Simone >>> >>> Il giorno venerdì 22 settembre 2017 19:10:01 UTC+2, Tim Hockin ha >>> scritto: >>>> >>>> you're trying to mount a directory (emptyDir) onto a file >>>> (/etc/resolv.conf). Without seeing the error that is a wild guess. I >>>> can't stop you from doing this, but I strongly encourage you to >>>> re-read and internalize what I wrote about multiple nameserver >>>> records. >>>> >>>> On Fri, Sep 22, 2017 at 6:19 AM, Simone D'Andreta >>>> <simone....@gmail.com> wrote: >>>> > I am trying to mount that volume but my container won't start. I >>>> guess I am >>>> > doing something wrong. This is the yaml >>>> > apiVersion: extensions/v1beta1 >>>> > kind: Deployment >>>> > metadata: >>>> > name: {{ template "fullname" . }} >>>> > namespace: code >>>> > labels: >>>> > chart: "{{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}" >>>> > annotations: >>>> > commitSHA: {{ .Chart.AppVersion }} >>>> > isNotifiable : "true" >>>> > spec: >>>> > replicas: {{ .Values.replicaCount }} >>>> > template: >>>> > metadata: >>>> > labels: >>>> > app: {{ template "fullname" . }} >>>> > >>>> > spec: >>>> > >>>> > containers: >>>> > - name: {{ .Chart.Name }} >>>> > image: "{{ .Values.image.repository }}:{{ .Values.image.tag >>>> }}" >>>> > imagePullPolicy: {{ .Values.image.pullPolicy }} >>>> > volumeMounts: >>>> > - name: new-resolv >>>> > mountPath: /etc/resolv.conf >>>> > command: ["/bin/sh"] >>>> > args: ["-c", "echo nameserver 10.24.26.102 > >>>> /etc/resolv.conf"] >>>> > >>>> > >>>> > ports: >>>> > - name: frontend >>>> > containerPort: {{ .Values.port}} >>>> > readinessProbe: >>>> > httpGet: >>>> > path: {{ .Values.lifecheck }} >>>> > port: {{ .Values.port}} >>>> > >>>> > volumes: >>>> > - name: new-resolv >>>> > emptyDir: {} >>>> > >>>> > I am using helm to deploy so the variables get expanded via >>>> Values.yaml or >>>> > other template files. >>>> > I think I am just not able to mount that volume properly.. >>>> > Thanks >>>> > >>>> > >>>> > Il giorno giovedì 21 settembre 2017 19:21:39 UTC+2, Tim Hockin ha >>>> scritto: >>>> >> >>>> >> You'd have to craft a new file and mount it onto your resolv.conf, >>>> >> which makes it harder to "just add another line" because you don't >>>> >> have the base. >>>> >> >>>> >> But more than that, what you're asking for is really non-standard >>>> >> behavior. You can't safely add a nameserver record to resolv.conf >>>> >> that produces different results. The behavior of DNS resolvers >>>> varies >>>> >> widely, and this will cause you pain eventually (kubernetes used to >>>> do >>>> >> this, it was bad). >>>> >> >>>> >> Consider this - some resolvers ask all DNS servers in parallel, and >>>> >> take the first response. If one resolver can answer a query and >>>> >> another can't (NXDOMAIN), your app will sometimes get an address and >>>> >> will sometimes fail. This actually happens. >>>> >> >>>> >> >>>> >> On Thu, Sep 21, 2017 at 6:33 AM, Simone D'Andreta >>>> >> <simone....@gmail.com> wrote: >>>> >> > Bad news, my idea doesn't work. Could you explain me more about >>>> the >>>> >> > volumeMount? I know how to mount but I don't know how I can >>>> effectively >>>> >> > add >>>> >> > a nameserver on that file. >>>> >> > >>>> >> > Thanks >>>> >> > Simone >>>> >> > >>>> >> > >>>> >> > Il giorno giovedì 21 settembre 2017 10:26:48 UTC+2, Simone >>>> D'Andreta ha >>>> >> > scritto: >>>> >> >> >>>> >> >> Hi Tim, >>>> >> >> thanks for your answer. I don't actually need to override all the >>>> >> >> settings >>>> >> >> in the resolv.conf, I just need to add another nameserver at the >>>> top of >>>> >> >> the >>>> >> >> file. How about if I run a command in the pod such as: >>>> >> >> >>>> >> >> command: ['/bin/sh', '-c', 'echo 'nameserver x.y.z.w' | cat - >>>> >> >> /etc/resolv.conf > temp && mv temp /etc/resolv.conf'] >>>> >> >> would it work? >>>> >> >> Thanks >>>> >> >> >>>> >> >> Il giorno mercoledì 20 settembre 2017 17:29:16 UTC+2, Tim Hockin >>>> ha >>>> >> >> scritto: >>>> >> >>> >>>> >> >>> There's no supported way to do that, in part because it would >>>> give up >>>> >> >>> all of the Service names that kubernetes provides. I don't know >>>> what >>>> >> >>> would happen if you tried to volumeMount a file over >>>> /etc/resolv.conf >>>> >> >>> - might be worth a shot. >>>> >> >>> >>>> >> >>> On Wed, Sep 20, 2017 at 3:15 AM, Simone D'Andreta >>>> >> >>> <simone....@gmail.com> wrote: >>>> >> >>> > Hello, >>>> >> >>> > I wanted to know if there is a way to override DNS settings >>>> for pods >>>> >> >>> > (not >>>> >> >>> > per cluster). I tried to use HostAliases but it only creates >>>> an A >>>> >> >>> > record for >>>> >> >>> > that entry. I basically need a NS record cause I need to point >>>> to >>>> >> >>> > different >>>> >> >>> > Consul clusters and then Consul must be able to do service >>>> >> >>> > discovery. >>>> >> >>> > So I >>>> >> >>> > was thinking to change the resolv.conf for the pod to use >>>> Consul for >>>> >> >>> > specific requests. Any idea? >>>> >> >>> > Thank you, >>>> >> >>> > Simone >>>> >> >>> > >>>> >> >>> > -- >>>> >> >>> > You received this message because you are subscribed to the >>>> Google >>>> >> >>> > Groups >>>> >> >>> > "Kubernetes user discussion and Q&A" group. >>>> >> >>> > To unsubscribe from this group and stop receiving emails from >>>> it, >>>> >> >>> > send >>>> >> >>> > an >>>> >> >>> > email to kubernetes-use...@googlegroups.com. >>>> >> >>> > To post to this group, send email to >>>> kubernet...@googlegroups.com. >>>> >> >>> > Visit this group at >>>> >> >>> > https://groups.google.com/group/kubernetes-users. >>>> >> >>> > For more options, visit https://groups.google.com/d/optout. >>>> >> > >>>> >> > -- >>>> >> > You received this message because you are subscribed to the Google >>>> >> > Groups >>>> >> > "Kubernetes user discussion and Q&A" group. >>>> >> > To unsubscribe from this group and stop receiving emails from it, >>>> send >>>> >> > an >>>> >> > email to kubernetes-use...@googlegroups.com. >>>> >> > To post to this group, send email to kubernet...@googlegroups.com. >>>> >>>> >> > Visit this group at https://groups.google.com/grou >>>> p/kubernetes-users. >>>> >> > For more options, visit https://groups.google.com/d/optout. >>>> > >>>> > -- >>>> > You received this message because you are subscribed to the Google >>>> Groups >>>> > "Kubernetes user discussion and Q&A" group. >>>> > To unsubscribe from this group and stop receiving emails from it, >>>> send an >>>> > email to kubernetes-use...@googlegroups.com. >>>> > To post to this group, send email to kubernet...@googlegroups.com. >>>> > Visit this group at https://groups.google.com/group/kubernetes-users. >>>> >>>> > For more options, visit https://groups.google.com/d/optout. >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Kubernetes user discussion and Q&A" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to kubernetes-users+unsubscr...@googlegroups.com. >>> To post to this group, send email to kubernetes-users@googlegroups.com. >>> Visit this group at https://groups.google.com/group/kubernetes-users. >>> For more options, visit https://groups.google.com/d/optout. >>> >> -- > You received this message because you are subscribed to the Google Groups > "Kubernetes user discussion and Q&A" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to kubernetes-users+unsubscr...@googlegroups.com. > To post to this group, send email to kubernetes-users@googlegroups.com. > Visit this group at https://groups.google.com/group/kubernetes-users. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Kubernetes user discussion and Q&A" group. To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-users+unsubscr...@googlegroups.com. To post to this group, send email to kubernetes-users@googlegroups.com. Visit this group at https://groups.google.com/group/kubernetes-users. For more options, visit https://groups.google.com/d/optout.
