Hello Guys, 1. I'm getting for the first time a Readiness Probe issue with ly Prom running on AWS EKS, the pod was running nice since weeks ant it suddenly turns into failing. *$ kdpon monitoring prometheus *(Kube pod describe command ) Unhealthy 4m (x182535 over 6d) kubelet, ip-100-xxxxxxxx.ec2.internal Readiness probe failed: HTTP probe failed with statuscode: 503 2. However, seems my logs are doing well, and restarting pods doesn't fix the issue but this is a normal. behaviour for readiness Prob *$ klon monitoring prometheus *(Kube log command ) level=info ts=2020-05-25T09:38:56.653582783Z caller=main.go:243 msg="Starting Prometheus" version="(version=2.6.1, branch=HEAD, revision=b639fe140c1f71b2cbad3fc322b17efe60839e7e)" level=info ts=2020-05-25T09:38:56.653650678Z caller=main.go:244 build_context="(go=go1.11.4, user=root@4c0e286fe2b3, date=20190115-19:12:04)" level=info ts=2020-05-25T09:38:56.653691321Z caller=main.go:245 host_details="(Linux 4.14.154-128.181.amzn2.x86_64 #1 SMP Sat Nov 16 21:49:00 UTC 2019 x86_64 prometheus-0 (none))" level=info ts=2020-05-25T09:38:56.653727427Z caller=main.go:246 fd_limits="(soft=65536, hard=65536)" level=info ts=2020-05-25T09:38:56.653762149Z caller=main.go:247 vm_limits="(soft=unlimited, hard=unlimited)" level=info ts=2020-05-25T09:38:56.65457275Z caller=main.go:561 msg="Starting TSDB ..." level=info ts=2020-05-25T09:38:56.655755388Z caller=web.go:429 component=web msg="Start listening for connections" address=0.0.0.0:9090 level=info ts=2020-05-25T09:38:56.817276722Z caller=repair.go:48 component=tsdb msg="found healthy block" mint=1583940235972 maxt=1583971200000 ulid=01E36HT83DK3TG22QWE02AP50X level=info ts=2020-05-25T09:38:56.822556254Z caller=repair.go:48 component=tsdb msg="found healthy block" mint=1583971200000 maxt=1584165600000 ulid=01E3C4BCPDEWX998E1EMR8RWTB level=info ts=2020-05-25T09:38:56.826661565Z caller=repair.go:48 component=tsdb msg="found healthy block" mint=1584165600000 maxt=1584360000000 ulid=01E3HXRADQ09H3PD3KHPS5GAXK level=info ts=2020-05-25T09:38:56.831874475Z caller=repair.go:48 component=tsdb msg="found healthy block" mint=1584360000000 maxt=1584554400000 ulid=01E3QQ4PWYR2D0TF0ERWR7KTZA level=info ts=2020-05-25T09:38:56.837772421Z caller=repair.go:48 component=tsdb msg="found healthy block" mint=1584554400000 maxt=1584748800000 ulid=01E3XGHHPWR4QHYZBZGGSR41FS level=info ts=2020-05-25T09:38:56.842330027Z caller=repair.go:48 component=tsdb msg="found healthy block" mint=1584748800000 maxt=1584943200000 ulid=01E439YBW2HYS58CK67540JRDP level=info ts=2020-05-25T09:38:56.847023184Z caller=repair.go:48 component=tsdb msg="found healthy block" mint=1584943200000 maxt=1585137600000 ulid=01E493B0FCHAT74EZFVX63AA8J level=info ts=2020-05-25T09:38:56.852126117Z caller=repair.go:48 component=tsdb msg="found healthy block" mint=1585137600000 maxt=1585332000000 ulid=01E4EWV0G5ZV7G0M5TH3XEV3X5 level=info ts=2020-05-25T09:38:56.856215058Z caller=repair.go:48 component=tsdb msg="found healthy block" mint=1585332000000 maxt=1585526400000 ulid=01E4MP49E017G3SF46C4MXND0A level=info ts=2020-05-25T09:38:56.860292298Z caller=repair.go:48 component=tsdb msg="found healthy block" mint=1585526400000 maxt=1585720800000 ulid=01E4TFMEZ1YMEM8PJAQJA9WZA7 level=info ts=2020-05-25T09:38:56.865162002Z caller=repair.go:48 component=tsdb msg="found healthy block" mint=1585720800000 maxt=1585915200000 ulid=01E508Y5F1V3RBPRMPVYMQCFQM level=info ts=2020-05-25T09:38:56.869511088Z caller=repair.go:48 component=tsdb msg="found healthy block" mint=1585915200000 maxt=1586109600000 ulid=01E562AXM9EHH3BV11MQYHGX80 I'm trying to figure if the issue is rrmated to Code or infrastructure layer Thanks you in advance for you Help Best, ABH -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/8ae22e72-c753-4af0-b5b1-d28a60a1654e%40googlegroups.com.