On 07/11/2015 8:01 AM, "Sébastien ROHAUT" <sebastien.rohaut....@gmail.com> wrote: > > Hi, > > We encountered a big problem this afternoon, which crashed for a while one of our websites, a java (tomcat+lift) application. We are using Haproxy 1.5. > > For our backend, we're doing something like this, using tcp-check because we need to check status AND a string, which is not possible with http-check : > > backend backend-mywebsite > balance roundrobin > option redispatch > option tcp-check > tcp-check send GET\ /check \ HTTP/1.1\r\nHost:\ > www.mywebsite.fr\r\nConnection:\ close\r\n > tcp-check send \r\n > tcp-check expect string HTTP/1.1\ 200\ OK > tcp-check expect rstring "healthStatus":"(Healthy|DegradedMode)" > cookie JSESSIONID prefix nocache > > > server s1 s1:11503 weight 1 check inter 10s fall 3 rise 2 ssl cookie s1 > server s2 s2:11503 weight 1 check inter 10s fall 3 rise 2 ssl cookie s2 > server s3 s3:11503 weight 1 check inter 10s fall 3 rise 2 ssl cookie s3 > server s4 s4:11503 weight 1 check inter 10s fall 3 rise 2 ssl cookie s4 > > For some reasons, the /check page didn't returned the correct application status and our / returned a 500 even if /check was OK, so we decided to check /. > > After 20 minutes, our application crashed. In fact, our 4 fronts crashed at the same time, and if we restarted them, 20 minutes after, they crashed again. We lost some time because we were really thinking on a software bug, before we realize the root cause. > > * Each tcp-check send opens a session on the application > * Each session, on the / page, consumes 500 KB > * session duration : 30 minutes > * We have 4 Haproxy, doing 2 checks (the app provides 2 websites, so one check for each Host: ), 6 times per minute = 48 checks, each minute. On each front. > * After 20 minutes : more than 450 MB used in the app for sessions > * Full GC, crash > > So, my question is : > > Is it possible to get and store the JSESSIONID cookie returned by the tcp-check expect (or something like this), and send it with the tcp-check send, to reuse the same session ? > > Is there a way for a health check to use persistent cookie session (always the same, one per server), returned by the check ? > > Thank you very much, > > Sebastien Rohaut
What we did in our case is simply not produce a session in the app for the health check path.
