Re: [kubernetes-users] Cluster DNS: bottleneck with ~1000 outbound connections per second

2018-03-20 Thread Evan Jones
instead of the confusingly named Default, but it seemed challenging enough that I never got around to it. Evan On Tue, Mar 20, 2018 at 1:55 AM, <m...@percy.io> wrote: > On Thursday, October 5, 2017 at 1:29:28 PM UTC-7, Evan Jones wrote: > > The sustained 1000 qps comes from an app

[kubernetes-users] Re: Connecting to Cloud SQL from container app on kubernetes built via google container builder.

2018-01-04 Thread Evan Jones
I recommend using the Google Cloud SQL proxy container so you don't need to mess with IP whitelists. I don't quite get what you mean about "manual work to edit my pod deployment file": You just need to copy and paste this "sidecar" definition into your .yaml and leave it there. We have been

Re: [kubernetes-users] Cluster DNS: bottleneck with ~1000 outbound connections per second

2017-10-05 Thread Evan Jones
a Go DNS query generator, which was able to do 8 DNS queries per second, so dnsmasq does not appear to be the limit. Thanks! Evan On Thu, Oct 5, 2017 at 5:26 PM, Rodrigo Campos <rodr...@sdfg.com.ar> wrote: > On Thu, Oct 05, 2017 at 04:29:21PM -0400, Evan Jones wrote: > > The

[kubernetes-users] Cluster DNS: bottleneck with ~1000 outbound connections per second

2017-10-05 Thread Evan Jones
*TL;DR*: Kubernetes dnsPolicy: ClusterFirst can become a bottleneck with a high rate of outbound connections. It seems like the problem is filling the nf_conntrack table, causing client applications to fail to do DNS lookups. I resolved this problem by switching my application to dnsPolicy:

Re: [kubernetes-users] Need some guidance/help: howto diagnose an oomkill

2017-09-27 Thread Evan Jones
Its been a while since I've dealt with this sort of issue, but there are various libraries that use "native" memory outside the Java heap. The -Xmx flag only limits the Java heap, so it isn't surprising that some processes may need a way higher container memory limit than the Java GC heap

Re: [kubernetes-users] Getting Google's Cloud SQL Proxy to work

2017-06-28 Thread Evan Jones
M, Traiano Welcome <trai...@gmail.com> wrote: > > > On Thursday, 22 June 2017 18:16:22 UTC+4, Evan Jones wrote: >> >> The Cloud SQL Proxy logs suggest to me that it may not be using the right >> credentials? It is possible that it is trying to use the cluster's &qu

Re: [kubernetes-users] Getting Google's Cloud SQL Proxy to work

2017-06-22 Thread Evan Jones
7 at 9:46:52 AM UTC-4, Traiano Welcome wrote: > > Hi Evan > > > On Thu, Jun 22, 2017 at 5:34 PM, Evan Jones <evan@triggermail.io > > wrote: > >> I know nothing about wordpress, but for what it is worth, we are using >> this Cloud SQL Proxy container with

Re: [kubernetes-users] Authentication on GKE

2017-06-09 Thread Evan Jones
On the cluster details page on https://console.cloud.google.com/kubernetes , if you have upgraded to 1.6 (I think?), you should see the following drop down to edit an existing cluster. I haven't yet attempted this personally:

[kubernetes-users] Re: Google Compute Engine is unusable, repeated "Failed: Create VM"

2017-06-01 Thread Evan Jones
A friend of mine ran into something that sounds suspiciously similar to this. I don't recall the details about it, but he did tweet about it: https://twitter.com/nicksantos/status/86997848164864 I seem to recall after the free trial expired, they literally had to delete everything and

[kubernetes-users] Re: Help me understand Kubernetes/Google LB options and architectures

2017-05-15 Thread Evan Jones
This won't directly help answer your questions, since I don't know the answers. However, I found this talk about Kubernetes networking to be extremely helpful to understand the basics. Whenever I'm running into weirdness I end up reviewing it: https://www.youtube.com/watch?v=y2bhV81MfKQ

Re: [kubernetes-users] Finding a way to get stable public IP for outbound connections

2017-05-03 Thread Evan Jones
this is also true on an AWS managed equivalent like CoreOS's CloudFormation > scripts. > > On Wed, May 3, 2017 at 8:52 AM, Evan Jones <evan.jo...@triggermail.io> > wrote: > >> As Rodrigo described, we are using Container Engine. I haven't fully >> tested this yet, but my pla

Re: [kubernetes-users] Finding a way to get stable public IP for outbound connections

2017-05-03 Thread Evan Jones
As Rodrigo described, we are using Container Engine. I haven't fully tested this yet, but my plan is to assign "dedicated IPs" to a set of nodes, probably in their own Node Pool as part of the cluster. Those are the IPs used by outbound connections from pods running those nodes, if I recalling

Re: [kubernetes-users] Finding a way to get stable public IP for outbound connections

2017-05-02 Thread Evan Jones
Thank you! I had forgotten about that feature, since we previously have not needed it. That will absolutely solve our problem, and be much better than needing an "exceptional" thing outside of Kubernetes. You are correct about what we need: We have a small number of services where their

Re: [kubernetes-users] Finding a way to get stable public IP for outbound connections

2017-05-01 Thread Evan Jones
It turns out I've just run into a requirement to have a stable outbound IP address as well. In looking into this: I think we will likely some kind of proxy server running outside of Kubernetes. This will allow services "opt in" to this special handling, rather than doing it for everything in