Hello
This is my first post to this mailing list.
I am not good at using English, so feel free to ask me
if my text is hard to understand.
I noticed haproxy-1.8 sometimes sent incomplete htmls to
clients when running haproxy with the attached haproxy.cfg
that uses multithread function.
# I also attached content01.html and check.html that
# are deployed on backend servers.
# content01.html is used in the confirmnation test
# described below, check.html is for health check
# purpose.
In this case, the client receives a shorter html.
I confirmed haoproxy's log message corresponded to the
shorter html, the following line is one of them.
2021-03-23T15:54:46.869626+09:00 lb01 [err] haproxy[703957]:
192.168.1.73:60572 [23/Mar/2021:15:54:46.868] vs_http01 rs_http01web/web01
0/0/0/0/0 200 2896 - - SD-- 12/12/7/2/0 0/0 "GET
/content01.html?x=170026897 HTTP/1.1"
I ran the curl commnad like the foolowing line at the client
to get this log message.
$ curl -v "http://192.168.1.21/content01.html?x=[17000-17049]";
1>17000.sto 2>17000.ste
# The x parameter is for identification.
The following line is the log message when a HTTP access is
normally done.
2021-03-23T15:54:05.918522+09:00 lb01 [info] haproxy[703957]:
192.168.1.73:44490 [23/Mar/2021:15:54:05.917] vs_http01 rs_http01web/web02
0/0/0/0/0 200 4864 - - 9/9/5/4/0 0/0 "GET /content01.html?x=17000
HTTP/1.1"
The following list is the frequency of abnormal access
when a total of 10 million accesses are made in 20 parallel
curl processes.
status_code bytes_read occurence
200 4344 1
200 2896 9
200 1448 6
408 2162
The following line is the log messages
in the case of 408.
2021-03-23T16:02:42.444084+09:00 lb01 [err] haproxy[703957]:
192.168.1.73:37052 [23/Mar/2021:16:02:32.472] vs_http01 vs_http01/
-1/-1/-1/-1/1 408 212 - - cR-- 14/14/0/0/0 0/0 ""
When I first met this behavior, I used haproxy-1.8.25 shipped
with RHEL8.3. So I obtained haproxy-1.8.30 from http://git.haproxy.org/
and built it, ran the test and got the result of the same sort.
This behavior didn't happen without using multithread function.
Next, I tried on haproxy-2.0.0 and confirmed it ran normally
without this behavior.
Then I picked up several versions of haproxy
between 1.8.0 and 2.0.0 and built them, test them
and found the commit below resolved this behavior.
===
commit 524344b4e0434b86d83869ef051f98d24505c08f
Author: Olivier Houchard
Date: Wed Sep 12 17:12:53 2018 +0200
MEDIUM: connections: Don't reset the polling flags in conn_fd_handler().
Resetting the polling flags at the end of conn_fd_handler() shouldn't be
needed anymore, and it will create problem when we won't handle
send/recv
from conn_fd_handler() anymore.
diff --git a/src/connection.c b/src/connection.c
index ab32567b..e303f2c3 100644
--- a/src/connection.c
+++ b/src/connection.c
@@ -203,9 +203,6 @@ void conn_fd_handler(int fd)
conn->mux->wake(conn) < 0)
return;
- /* remove the events before leaving */
- fdtab[fd].ev &= FD_POLL_STICKY;
-
/* commit polling changes */
conn->flags &= ~CO_FL_WILL_UPDATE;
conn_cond_update_polling(conn);
===
So I commented out "fdtab[fd].ev &= FD_POLL_STICKY;"
from both of haproxy-1.8.25 and haproxy-1.8.30,
then the behavior is resolved.
I don't know why this commit resolves the behavior,
I just tried and ran the test.
My question is:
Does this commit make side effects such like
data structure corruption or memory leak on haprox-1.8 ?
And if someone tell me how the behavior happens and
how the commmit resolves it, I will be appreciated it.
Best regards.
P.S.
I also tried haproxy-2.0.22 and I didn't find
messages about shorter htmls but the number of
log messages is smaller than the number of HTTP
accesses.
I am considering this would be another issue.
Both of haproxy-2.3.0 and haproxy-2.3.10 don't make
shorter html issue and insufficient log messages issue.
--
Takenaka Kazuhiro
email: or
Title: content01
0001 123456789 123456789 123456789 123456789 123456789X
0002 123456789 123456789 123456789 123456789 123456789X
0003 123456789 123456789 123456789 123456789 123456789X
0004 123456789 123456789 123456789 123456789 123456789X
0005 123456789 123456789 123456789 123456789 123456789X
0006 123456789 123456789 123456789 123456789 123456789X
0007 123456789 123456789 123456789 123456789 123456789X
0008 123456789 123456789 123456789 123456789 123456789X
0009 123456789 123456789 123456789 123456789 123456789X
0010 123456789 123456789 123456789 123456789 123456789X
0011 123456789 123456789 123456789 123456789 123456789X
0012 123456789 123456789 123456789 123456789 123456789X
0013 123456789 123456789 123456789 123456789 123456789X
0014 123456789 123456789 123456789 123456789 123456789X
0015 123456789 123456789 123456
