Re: [kubernetes-users] Need some guidance/help: howto diagnose an oomkill

2017-09-28 Thread Matthias Rampke
You can also run jstatd in a running pod and then attach JVisualVM. I haven't done it myself, but the general procedure is: - kubectl exec into the pod - Write the policy file to disk: echo 'grant codebase "file:${java.home}/../lib/tools.jar" { permission java.security.AllPermission; };' >

Re: [kubernetes-users] Need some guidance/help: howto diagnose an oomkill

2017-09-27 Thread Evan Jones
Its been a while since I've dealt with this sort of issue, but there are various libraries that use "native" memory outside the Java heap. The -Xmx flag only limits the Java heap, so it isn't surprising that some processes may need a way higher container memory limit than the Java GC heap

Re: [kubernetes-users] Need some guidance/help: howto diagnose an oomkill

2017-09-26 Thread John VanRyn
helps some... we made the kube pods have almost twice as much memory as we are allocating the jvm.. and it seems to get us out of the woods but it totally means we need to look into a jdk upgrade from 8. Thanks On Tue, Sep 26, 2017 at 7:50 AM, Davanum Srinivas wrote: >

Re: [kubernetes-users] Need some guidance/help: howto diagnose an oomkill

2017-09-26 Thread Davanum Srinivas
John, Does this help? https://developers.redhat.com/blog/2017/03/14/java-inside-docker/ There are some details here as well: https://github.com/moby/moby/issues/15020 Thanks, Dims On Tue, Sep 26, 2017 at 7:37 AM, John VanRyn wrote: > I have a kube cluster running on

[kubernetes-users] Need some guidance/help: howto diagnose an oomkill

2017-09-26 Thread John VanRyn
I have a kube cluster running on n1-highmem-16 (16 vCPUs, 104 GB memory), using the unmodified cos-stable-60-9592-84-0 image. I have a java app running under wildfly apiVersion: extensions/v1beta1 kind: Deployment metadata: name: cas-unicas-ws labels: name: cas-unicas-ws model: