On Mon, Apr 24, 2017 at 05:29:14PM +0800, jaseywang wrote: > Hi, > > Here is the 1.7.5 output with CDN, before 05:22:00 PM with timestamps in > the file, there is no request, since 05:22:00 PM, I began the benchmark, so > you can check from 05:22:00 PM. > 61.155.222.157 is cdn itself > > > The file is large here is the download link: > https://www.dropbox.com/s/yrv7l3m8hw32rr9/1.7.5-sess?dl=0 > https://www.dropbox.com/s/pb7zglhnyovo79f/1.7.5-tcp?dl=0
OK these ones are interesting. First, none of the CLOSE_WAIT connections in the netstat file is present in the "show sess" output. Might it be possible that these ones are left-overs from a previous run ? Could you please double-check that you don't have an older process still running ? Second, for an unknown reason, the source address reported in "show sess" backends shows the CDN instead of haproxy. I double-checked the code to be sure and it's the correct address that is reported. Are you using some form of transparent proxying that didn't appear in your first config, like "usesrc clientip" ? Third, you have countless SYN_RECV on port 443. There can be two explanations : - too small a socket backlog - high loss rate between the CDN and your haproxy. The previous dumps used to report a lot of FIN_WAIT, indicating a difficulty to get ACKs back from the CDN which could thus also fuel the packet loss hypothesis, but on this test there are much less, about 0.2% which thus seems completely normal. However its still important to keep this hypothesis in mind as it could also explain your huge number of concurrent connections. Regardless, I'm seeing one strange thing. There are a lot of "state=DIS" in the output. This is a transient state which should not remain for a long time, it's used to report an imminent close. It indicates that the client-facing connection was just terminated, but not executed. Strangely, these tasks are present in the run queue but were delayed. Would you happen to build with Clang ? We've faced an integer overflow bug on very recent versions. Willy
