I'm actually seeing now that the workers are being OOMkilled at some point. Which doesn't make any sense because all of my kubernetes workers have >50gb free and this pod only requires 256mb.
Name: jnlp-1sh50 Namespace: jenkins-test Node: kubeworker-rwva1-prod-10/10.0.0.217 Start Time: Thu, 22 Feb 2018 01:08:20 -0800 Labels: jenkins=slave jenkins/jenkins-test-jenkins-slave=true Annotations: <none> Status: Failed IP: 10.233.115.31 Containers: jnlp: Container ID: docker://4cf893a70ad3d074b9708f076baef9600b0abfa9d5726d41711ccb39adcc9814 Image: jenkins/jnlp-slave:3.10-1 Image ID: docker-pullable://jenkins/jnlp-slave@sha256:db1cb9e803fe2aeb440435cf0da4195b63685664c5976d2fefead839631d070e Port: <none> Args: 2200e9b3a7435e57a10c48570b0afe4210cbc87a910766503b5eace2d4f32a86 jnlp-1sh50 State: Terminated Reason: OOMKilled Exit Code: 137 Started: Thu, 22 Feb 2018 01:08:21 -0800 Finished: Thu, 22 Feb 2018 01:09:10 -0800 Ready: False Restart Count: 0 Limits: cpu: 200m memory: 256Mi Requests: cpu: 200m memory: 256Mi Environment: JENKINS_SECRET: 2200e9b3a7435e57a10c48570b0afe4210cbc87a910766503b5eace2d4f32a86 JENKINS_TUNNEL: jenkins-test-jenkins-agent:50000 JENKINS_NAME: jnlp-1sh50 JENKINS_URL: http://jenkins-test-jenkins:8080 HOME: /home/jenkins Mounts: /home/jenkins from workspace-volume (rw) /var/run/secrets/kubernetes.io/serviceaccount from default-token-dtwdj (ro) Conditions: Type Status Initialized True Ready False PodScheduled True Volumes: workspace-volume: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: default-token-dtwdj: Type: Secret (a volume populated by a Secret) SecretName: default-token-dtwdj Optional: false QoS Class: Guaranteed Node-Selectors: <none> Tolerations: <none> Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 2m default-scheduler Successfully assigned jnlp-1sh50 to kubeworker-rwva1-prod-10 Normal SuccessfulMountVolume 2m kubelet, kubeworker-rwva1-prod-10 MountVolume.SetUp succeeded for volume "workspace-volume" Normal SuccessfulMountVolume 2m kubelet, kubeworker-rwva1-prod-10 MountVolume.SetUp succeeded for volume "default-token-dtwdj" Normal Pulled 2m kubelet, kubeworker-rwva1-prod-10 Container image "jenkins/jnlp-slave:3.10-1" already present on machine Normal Created 2m kubelet, kubeworker-rwva1-prod-10 Created container Normal Started 2m kubelet, kubeworker-rwva1-prod-10 Started container On Thursday, February 22, 2018 at 12:04:31 AM UTC-8, Carlos Sanchez wrote: > > That log suggests that the slave is not "online" for jenkins. If you go to > the jenkins nodes page under /computer/ what do you see? maybe the slave is > offline for some reason, lack of space or something > > -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/5507122d-833a-422f-9461-7290ec92fc9d%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.