Hi Willy, > OK so instead of sending you a boring series, I can propose you to run > a test on 2.0-dev, which contains all the fixes I had to go through > because of tiny issues everywhere related to this. If you're using git, > just clone the master and checkout commit f7a259d46f8. > you can simply wait for the next nightly snapshot.
Sounds good. My compilation playbook uses tarballs, so I'll just use the last nightly. I assume I should wait for these fixes to be backported (1.9.3?) before trying anything in production? > But now you have a new server parameter called > "max-reuse". This allows to limit the number of times a server connection > is reused. For example you can set it to 990 when you know that the > server limits to 1000. That's great! I didn't expect to get a new configuration option. I'll definitely make sure these are in sync across our infrastructure. > Regarding the fact that in your case the client's close seems to cause > the server-side issue, I couldn't yet reproduce it though I have a few > theories about it. One of them would be an unexpected response from > the server causing the connection to turn to an error state. The other > one would be that we'd incorrectly abort our stream and/or session and > bring the connection down with us. I'll submit these theories to Olivier > once he's back so that he can tell me I'm saying crap regarding some of > them and we can focus on what remains :-) Sounds good. I'll report back my results from the latest snapshot and we can go from there. Perhaps the client causing the issues was a red herring for the server-side bugs. Thanks again for deep-diving and resolving this! I won't ask how many hours it took to find all these small edge cases. . . Best, Luke — Luke Seelenbinder Stadia Maps | Founder stadiamaps.com ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Thursday, January 24, 2019 7:55 PM, Willy Tarreau <w...@1wt.eu> wrote: > Hi Luke, > > On Wed, Jan 23, 2019 at 05:16:04PM +0000, Luke Seelenbinder wrote: > > > Hi Willy, > > This is all very good to hear. I'm glad you were able to get to the bottom > > of > > it all! > > Feel free to send along patches if you want me to test before the 1.9.3 > > release. I'm more than happy to do so. > > OK so instead of sending you a boring series, I can propose you to run > a test on 2.0-dev, which contains all the fixes I had to go through > because of tiny issues everywhere related to this. If you're using git, > just clone the master and checkout commit f7a259d46f8. > you can simply wait for the next nightly snapshot. > > Just let me know if that's OK for you. > > I found a number of issues that were causing server aborts, mainly > due to the late GOAWAY frame. Once we hit this one, the connection > is quickly closed by the server, causing our output packets to be > rejected and the connection to be in error. I have not yet investigated > in details to see if the close happens after we got the last data or in > the middle though. But now you have a new server parameter called > "max-reuse". This allows to limit the number of times a server connection > is reused. For example you can set it to 990 when you know that the > server limits to 1000. > > On the tests I've run here, I managed to address all the problems > related to excessive use of idle connections resulting in too many > streams being sent. In addition most of the rare cases that still > happen when you don't have max-reuse are properly handled as a retry. > > Regarding the fact that in your case the client's close seems to cause > the server-side issue, I couldn't yet reproduce it though I have a few > theories about it. One of them would be an unexpected response from > the server causing the connection to turn to an error state. The other > one would be that we'd incorrectly abort our stream and/or session and > bring the connection down with us. I'll submit these theories to Olivier > once he's back so that he can tell me I'm saying crap regarding some of > them and we can focus on what remains :-) > > Regards, > Willy
publickey - luke.seelenbinder@stadiamaps.com - 0xB23C1E8A.asc
Description: application/pgp-keys
signature.asc
Description: OpenPGP digital signature
