My script *is* always looking up the same domain, and I believe it is cached by dnsmasq. I *think* the limit is the kernel NAT connection tracking, because each DNS query comes from a new ephemeral port, so it ends up using up all NAT mappings on the node running kube-dns. This is why dnsPolicy: Default fixes the problem: It uses the host's DNS configuration which avoids the NAT connection limits.
Details including the Python code and configs to reproduce it on a brand new GKE cluster are at the bottom of https://github.com/ kubernetes/kubernetes/issues/45976 I did a separate test, using a Go DNS query generator, which was able to do 80000 DNS queries per second, so dnsmasq does not appear to be the limit. Thanks! Evan On Thu, Oct 5, 2017 at 5:26 PM, Rodrigo Campos <rodr...@sdfg.com.ar> wrote: > On Thu, Oct 05, 2017 at 04:29:21PM -0400, Evan Jones wrote: > > The sustained 1000 qps comes from an application making that many > outbound > > connections. I agree that the application is very inefficient and > shouldn't > > be doing a DNS lookup for every request it sends, but it's a python > program > > that uses urllib2.urlopen so it creates a new connection each time. I > > suspect this isn't that unusual? This could be a server that hits an > > external service for every user request, for example. Given the activity > on > > the GitHub issues I linked, it appears I'm not the only person to have > run > > into this. > > But is always on different domains? If not, it can probably be cached (as > long > as the TTL allows) by the DNS server and, even if your app makes so many > requests, it should be answered quite fast. > > -- > You received this message because you are subscribed to a topic in the > Google Groups "Kubernetes user discussion and Q&A" group. > To unsubscribe from this topic, visit https://groups.google.com/d/to > pic/kubernetes-users/7JBq6jhMZHc/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > kubernetes-users+unsubscr...@googlegroups.com. > To post to this group, send email to kubernetes-users@googlegroups.com. > Visit this group at https://groups.google.com/group/kubernetes-users. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Kubernetes user discussion and Q&A" group. To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-users+unsubscr...@googlegroups.com. To post to this group, send email to kubernetes-users@googlegroups.com. Visit this group at https://groups.google.com/group/kubernetes-users. For more options, visit https://groups.google.com/d/optout.
