Re: Possible causes for termination state "LR"
On 10/21/2014 03:59 PM, Cyril Bonté wrote:
Hi Colin,
Le 21/10/2014 22:05, Colin Ingarfield a écrit :
Hello,
Please help me understand this output in the haproxy log file:
Oct 21 14:53:17 host haproxy[4526]: 206.x.x.x:56480
[21/Oct/2014:14:53:16.560] http-in~ servers/server1 60/0/0/527/587 200
243 - - LRVN 12/12/1/2/0 0/0 "POST /ctx/cometd/connect HTTP/1.1"
The first two letters of the termination state are confusing me.
"L" indicates the session was processed by haproxy and not sent to a
server
"R" proxy was waiting for a complete request from the client
It also shows a status code of 200.
What circumstances could cause this? I'm testing haproxy now and it
(and the 2 servers behind it) are under no real load. So far I have
only observed this with Chrome Version 38.0.2125.104 (64-bit) linux. I
do not see any errors in the Chrome Network panel and the application
appears to work correctly.
Any insight greatly appreciated.
From a quick test I made, I think you have some redirect rules in your
haroxy configuration.
And it appears that flags are not reset on a keep-alived connection.
Please check in your logs if a previous request already used port
56480 with a 302 status.
This is maybe a bug, but I won't have time tonight to investigate more.
To reproduce it, I used this minimal configuration :
listen testflag :9000
mode http
option httplog
log /dev/log local7 debug err
http-request redirect location /redirected if { path / }
server s localhost:80
$ curl -L http://localhost:9000/
Oct 21 22:53:40 asus haproxy[11282]: 127.0.0.1:33985
[21/Oct/2014:22:53:40.094] testflag testflag/ 0/-1/-1/-1/0 302
89 - - LR-- 1/1/0/0/3 0/0 "GET / HTTP/1.1"
Oct 21 22:53:40 asus haproxy[11282]: 127.0.0.1:33985
[21/Oct/2014:22:53:40.094] testflag testflag/s 0/0/1/2/3 404 550 - -
LR-- 1/1/0/1/0 0/0 "GET /redirected HTTP/1.1"
Here, the second request inherits LR from the first one.
The same test with curl using HTTP/1.0 :
$ curl -0 -L http://localhost:9000/
Oct 21 22:55:26 asus haproxy[11282]: 127.0.0.1:33997
[21/Oct/2014:22:55:26.272] testflag testflag/ 0/-1/-1/-1/0 302
108 - - LR-- 0/0/0/0/3 0/0 "GET / HTTP/1.0"
Oct 21 22:55:26 asus haproxy[11282]: 127.0.0.1:33999
[21/Oct/2014:22:55:26.272] testflag testflag/s 0/0/0/2/2 404 529 - -
0/0/0/0/0 0/0 "GET /redirected HTTP/1.0"
Here, it's OK.
Thank you for the quick reply.
I searched the log and did not find any request from port 56480 with a
302 status. I do not have any explicit redirect rules in my config, but
perhaps some rules can cause a redirect. I'm quite new to haproxy.
I've attached my cleaned config file. If you could take a look I'd
really appreciate it. And the haproxy version:
$ /usr/local/sbin/haproxy -vv
HA-Proxy version 1.5.4 2014/09/02
Copyright 2000-2014 Willy Tarreau
Build options :
TARGET = linux2628
CPU = generic
CC = gcc
CFLAGS = -O2 -g -fno-strict-aliasing
OPTIONS = USE_LINUX_SPLICE=1 USE_GETADDRINFO=1 USE_ZLIB=1
USE_OPENSSL=1 USE_PCRE=1
Default settings :
maxconn = 2000, bufsize = 16384, maxrewrite = 8192, maxpollevents = 200
Encrypted password support via crypt(3): yes
Built with zlib version : 1.2.8
Compression algorithms supported : identity, deflate, gzip
Built with OpenSSL version : OpenSSL 1.0.1f 6 Jan 2014
Running on OpenSSL version : OpenSSL 1.0.1f 6 Jan 2014
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports prefer-server-ciphers : yes
Built with PCRE version : 8.31 2012-07-06
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Built with transparent proxy support using: IP_TRANSPARENT
IPV6_TRANSPARENT IP_FREEBIND
Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.
$ uname -a
Linux ip-x-x-x-x 3.13.0-36-generic #63-Ubuntu SMP Wed Sep 3 21:30:07 UTC
2014 x86_64 x86_64 x86_64 GNU/Linux
Thank you again for looking into this.
-- Colin
global
log /dev/loglocal0
#log /dev/log local1 notice
chroot /var/lib/haproxy
user haproxy
group haproxy
daemon
# Tuning
# must set to 1024 to suppress a warning. docs say > 1024 not
# supported by many clients (eg java 7)
tune.ssl.default-dh-param 1024
# users (controls access to the haproxy admin page)
userlist users
group adminusers user1
group admin_ro users user2
user user1 password XX
user user2 password XX
defaults
log global
mode http
option httplog
option splice-auto
timeout connect 5000
timeout client 9
timeout server 9
errorfile 400 /etc/haproxy/errors/400.http
errorfile 403 /etc/haproxy/errors/403.http
errorfile 408 /etc/haproxy/errors/408.http
Possible causes for termination state "LR"
Hello, Please help me understand this output in the haproxy log file: Oct 21 14:53:17 host haproxy[4526]: 206.x.x.x:56480 [21/Oct/2014:14:53:16.560] http-in~ servers/server1 60/0/0/527/587 200 243 - - LRVN 12/12/1/2/0 0/0 "POST /ctx/cometd/connect HTTP/1.1" The first two letters of the termination state are confusing me. "L" indicates the session was processed by haproxy and not sent to a server "R" proxy was waiting for a complete request from the client It also shows a status code of 200. What circumstances could cause this? I'm testing haproxy now and it (and the 2 servers behind it) are under no real load. So far I have only observed this with Chrome Version 38.0.2125.104 (64-bit) linux. I do not see any errors in the Chrome Network panel and the application appears to work correctly. Any insight greatly appreciated. Thanks, Colin
cookie persistence when a server is down
Hello, I'm testing HAProxy's cookie based persistence feature(s) and I have a question. Currently I have 2 test servers set up behind HAProxy. They use a JSESSIONID cookie like many java application servers. In haproxy.cfg I have these persistence settings: server server1 127.0.0.1:9443ssl verify none check cookie server1 server server2 172.28.128.3:9443 ssl verify none check cookie server2 cookie JSESSIONID prefix This works as expected. HAProxy adds the prefix to the cookie and this enables "sticky" sessions. When I put, for example, server1 into maintenance, HAProxy routes server1 clients to server2. I can see this in the HAProxy logs with termination flags "--DN". When I put server1 back in service, it routes server1 clients back to server1, because the cookie has not changed (flag "--VN"). But what if I had a server3? When I put server1 in maint, will server1 clients be randomly routed to server2 & 3 on each request? Or are they somehow temporarily persisted to server 2 or 3 until server1 becomes available again? Thank you, Colin Ingarfield
Re: prefer epoll or sepoll on linux?
On 09/09/2014 10:12 AM, Manfred Hollstein wrote: Hi Colin, On Tue, 09 Sep 2014, 17:00:37 +0200, Colin Ingarfield wrote: Hello, I'm running linux Mint 17 qiana. $ uname -a Linux cingarfield-dt 3.13.0-24-generic #47-Ubuntu SMP Fri May 2 23:30:00 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux The packaged haproxy for this distro is version 1.4.24: $ /usr/sbin/haproxy -vv HA-Proxy version 1.4.24 2013/06/17 [...] My question is why does the packaged 1.4.24 use sepoll but from source 1.5.3 use epoll? Is there a meaningful difference, eg, in terms of performance? from <http://www.haproxy.org/news.html>: Nov 22th, 2012 : Development 1.5-dev13 with Compression! 4. Polling o speculative polling was generalized to all pollers, and sepoll disappeared as it was superseded by epoll. The main reason for this important change is the way OpenSSL works and the fact that it can easily get stuck with some data in buffers with no I/O event to unblock them. So we needed to engage into this difficult change. I'd have preferred to delay it to 1.6 if I was offered the choice ! But in the end this is good because it's done and it improves both performance and reliability. Even select() and poll() are now fast. Thank you, Colin Ingarfield HTH, cheers. l8er manfred This page did not turn up in my brief googling -- I should have tried harder. But it's exactly what I was looking for. Thanks!
prefer epoll or sepoll on linux?
Hello, I'm running linux Mint 17 qiana. $ uname -a Linux cingarfield-dt 3.13.0-24-generic #47-Ubuntu SMP Fri May 2 23:30:00 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux The packaged haproxy for this distro is version 1.4.24: $ /usr/sbin/haproxy -vv HA-Proxy version 1.4.24 2013/06/17 Copyright 2000-2013 Willy Tarreau Build options : TARGET = linux2628 CPU = generic CC = gcc CFLAGS = -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 OPTIONS = USE_PCRE=1 Default settings : maxconn = 2000, bufsize = 16384, maxrewrite = 8192, maxpollevents = 200 Encrypted password support via crypt(3): yes Available polling systems : sepoll : pref=400, test result OK epoll : pref=300, test result OK poll : pref=200, test result OK select : pref=150, test result OK Total: 4 (4 usable), will use sepoll. However I would like to use SSL so I built 1.5.3 from source: ~/dev/haproxy/haproxy-1.5.3$ ./haproxy -vv HA-Proxy version 1.5.3 2014/07/25 Copyright 2000-2014 Willy Tarreau Build options : TARGET = linux2628 CPU = generic CC = gcc CFLAGS = -O2 -g -fno-strict-aliasing OPTIONS = USE_GETADDRINFO=1 USE_ZLIB=1 USE_OPENSSL=1 USE_PCRE=1 Default settings : maxconn = 2000, bufsize = 16384, maxrewrite = 8192, maxpollevents = 200 Encrypted password support via crypt(3): yes Built with zlib version : 1.2.8 Compression algorithms supported : identity, deflate, gzip Built with OpenSSL version : OpenSSL 1.0.1f 6 Jan 2014 Running on OpenSSL version : OpenSSL 1.0.1f 6 Jan 2014 OpenSSL library supports TLS extensions : yes OpenSSL library supports SNI : yes OpenSSL library supports prefer-server-ciphers : yes Built with PCRE version : 8.31 2012-07-06 PCRE library supports JIT : no (USE_PCRE_JIT not set) Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND Available polling systems : epoll : pref=300, test result OK poll : pref=200, test result OK select : pref=150, test result OK Total: 3 (3 usable), will use epoll. My question is why does the packaged 1.4.24 use sepoll but from source 1.5.3 use epoll? Is there a meaningful difference, eg, in terms of performance? Thank you, Colin Ingarfield
Re: Spam to this list?
On 09/05/2014 08:07 AM, Steven Haigh wrote: Sorry, this is a stupid suggestion. 1) The spam still makes it onto the list archives - See: http://marc.info/?l=haproxy&r=1&b=201409&w=2 2) It dilutes the mailing list content by creating a massive drop in signal to noise (24+ spam messages in the past day?!?) 3) It causes the reputation as a mail sender of the haproxy mailing lists to be greatly reduced - meaning eventually hosts will reject mail from the list - legit or not. 4) This is a problem that has been 'solved' on other mailing lists for at least a decade. 5) It gives the impression to people wanting to use haproxy that the admin team don't know what they're doing. I've gotten more spam from this single list in the last 24 hours than I have from the total of ALL my other mailing lists in the past few months. If you aren't going to do proper spam filtering, AT LEAST do moderation of non-member posts. This single action will just about cure the spam problem. Its starting to become a joke. I agree w/ every point here. I do run my own spam filter (spamassassin) and it works very well. But it mostly trusts email that has a valid DKIM signature. Since the spam from list does have valid DKIM sigs it makes it through my filter easily. To fix this I'd have to make special spam rules just for this list to ignore DKIM signatures. That of course defeats the purpose of them in the first place. Please either filter the spam or require some kind of registration before posting to the list. Thank you, Colin Ingarfield
