Re: Disable client keep-alive using ACL
Christopher, Am 19.11.20 um 11:47 schrieb Christopher Faulet: > Le 19/11/2020 à 10:49, Tim Düsterhus, WoltLab GmbH a écrit : >> John, >> >> Am 19.11.20 um 06:57 schrieb John Lauro: >>> A couple of possible options... >>> You could use tcp-request inspect-delay to delay the response a >>> number of >>> seconds (and accept it quick if legitimate traffic). >> >> I believe the tcp-(request|response) rules only apply to the very first >> buffer of a single TCP connection and thus do not apply here. Let me >> give an example of what I want to do. >> > > Tim, > > tcp-(request|response) content rules are evaluated on all HTTP > transactions. tcp-request connection/session rules are only evaluated on > the client connection establishment. Oh, indeed and it's even documented. I just applied my knowledge about `mode tcp` without checking. I never had a use for tcp-request content rules for HTTP traffic, the http rules are simply much better. > However, this does not help you, because in H2, adding a delay on a > stream will not stop the traffic for other streams on the same connection. And even for H1 it would not help much for my use case, because having a connection sit idle for a few seconds still allows a rogue client to pay the TLS cost only once per connection, just increasing the number of connections required by a constant factor instead of linearly increasing the number of connections. > Thanks for the feature request. > Do you have a first estimation whether this would require intimate knowledge of the architecture and larger changes to implement? If it's a matter of adding a flag somewhere and a few lines checking that flag after handling the response then I might attempt it myself. Best regards Tim Düsterhus Developer WoltLab GmbH -- WoltLab GmbH Nedlitzer Str. 27B 14469 Potsdam Tel.: +49 331 96784338 duester...@woltlab.com www.woltlab.com Managing director: Marcel Werk AG Potsdam HRB 26795 P
Re: Disable client keep-alive using ACL
Le 19/11/2020 à 10:49, Tim Düsterhus, WoltLab GmbH a écrit : John, Am 19.11.20 um 06:57 schrieb John Lauro: A couple of possible options... You could use tcp-request inspect-delay to delay the response a number of seconds (and accept it quick if legitimate traffic). I believe the tcp-(request|response) rules only apply to the very first buffer of a single TCP connection and thus do not apply here. Let me give an example of what I want to do. Tim, tcp-(request|response) content rules are evaluated on all HTTP transactions. tcp-request connection/session rules are only evaluated on the client connection establishment. However, this does not help you, because in H2, adding a delay on a stream will not stop the traffic for other streams on the same connection. Thanks for the feature request. -- Christopher Faulet
Re: Disable client keep-alive using ACL
John, Am 19.11.20 um 06:57 schrieb John Lauro: > A couple of possible options... > You could use tcp-request inspect-delay to delay the response a number of > seconds (and accept it quick if legitimate traffic). I believe the tcp-(request|response) rules only apply to the very first buffer of a single TCP connection and thus do not apply here. Let me give an example of what I want to do. 1) Client connects 2) Client performs TLS handshake 3) Client sends HTTP request 4) Server sends response 5) Client sends HTTP request over the same connection (H1 keep-alive / H2). tcp-request / tcp-response does not apply here. 6) Server sends response 7) Repeat steps 4 and 5. If I start detecting unusual request rates for the client I would like to piggy back a clean (!) close onto step 6, forcing the client to perform steps 1 and 2 again. > You could use redirects which will have the clients do more requests > (Possibly with the inspect delays). Redirects do not play well with POST requests and they are very cheap on keep-alive connections. The costly part is the TLS handshake. > That said, it would be useful to force a client connection closed at times, > but there are ways to protect the backends and slow some clients without > completely blocking them. I went ahead and filed a feature request in the tracker now: https://github.com/haproxy/haproxy/issues/969 Thanks for the suggestions to both of you :-) Best regards Tim Düsterhus Developer WoltLab GmbH -- WoltLab GmbH Nedlitzer Str. 27B 14469 Potsdam Tel.: +49 331 96784338 duester...@woltlab.com www.woltlab.com Managing director: Marcel Werk AG Potsdam HRB 26795 P
Re: Disable client keep-alive using ACL
A couple of possible options... You could use tcp-request inspect-delay to delay the response a number of seconds (and accept it quick if legitimate traffic). You could use redirects which will have the clients do more requests (Possibly with the inspect delays). That said, it would be useful to force a client connection closed at times, but there are ways to protect the backends and slow some clients without completely blocking them. On Wed, Nov 18, 2020 at 3:14 AM Tim Düsterhus, WoltLab GmbH < duester...@woltlab.com> wrote: > Lukas, > > > The reason is that we want to avoid outright blocking with e.g. a 429 > Too Many Requests, because that could affect legitimate traffic. Forcing > the client to re-establish the connection should not be noticeable for a > properly implemented client, other than an increased latency. > > I'm aware that this will be more costly for us as well, but we have > plenty of spare capacity at the load balancers. > > >
Re: Disable client keep-alive using ACL
Lukas, Am 17.11.20 um 17:37 schrieb Lukas Tribus: >>> is it possible to reliably disable client keep-alive on demand based on >>> the result of an ACL? >>> >>> I was successful for HTTP/1 requests by using: >>> >>> http-after-response set-header connection close if foo >>> >>> But apparently that has no effect for HTTP/2 requests. I was unable to >>> find anything within the documentation with regard to this either. > > I don't think there is a way. In HTTP/2 you'd need to send a GOAWAY > message to close the connection. There are no instructions in the HTTP > headers regarding the connection. I would be fine with some other method of communicating the Close / GOAWAY, e.g. using http-response stop-serving-this-client. > I *think/hope* we are actually sending GOAWAY messages when: > > - some timeouts are reached > - hard-stop-after triggers > - a "shutdown session ..." is triggered > > > You could check if sending a "421 Misdirected Request" error to the > client could achieve your goal, but it certainly behaves differently > than a close in H1 (you can't get a successful answer to the client). > It's also a workaround. It will not. What I was attempting to do was forcing clients to re-establish the TCP connection after every request when rate limits are exceeded to slow them down and / or spend more resources. The reason is that we want to avoid outright blocking with e.g. a 429 Too Many Requests, because that could affect legitimate traffic. Forcing the client to re-establish the connection should not be noticeable for a properly implemented client, other than an increased latency. I'm aware that this will be more costly for us as well, but we have plenty of spare capacity at the load balancers. > Triggering GOAWAY/full H2 connection teardown dynamically would need > to be implemented. I think in HTX all connection headers are > immediately dropped (they are not "translated" and applied to the > connection). > Can you (or anyone else) comment on whether that would actually be feasible? I would create a feature request in the tracker then. If not I'll save the effort. If I can get some pointers I might even have a stab at implementing this myself. Best regards Tim Düsterhus Developer WoltLab GmbH -- WoltLab GmbH Nedlitzer Str. 27B 14469 Potsdam Tel.: +49 331 96784338 duester...@woltlab.com www.woltlab.com Managing director: Marcel Werk AG Potsdam HRB 26795 P
Re: Disable client keep-alive using ACL
Hi Tim, On Tue, 17 Nov 2020 at 13:35, Tim Düsterhus, WoltLab GmbH wrote: > > Hi > > Am 09.11.20 um 12:36 schrieb Tim Düsterhus, WoltLab GmbH: > > is it possible to reliably disable client keep-alive on demand based on > > the result of an ACL? > > > > I was successful for HTTP/1 requests by using: > > > > http-after-response set-header connection close if foo > > > > But apparently that has no effect for HTTP/2 requests. I was unable to > > find anything within the documentation with regard to this either. I don't think there is a way. In HTTP/2 you'd need to send a GOAWAY message to close the connection. There are no instructions in the HTTP headers regarding the connection. I *think/hope* we are actually sending GOAWAY messages when: - some timeouts are reached - hard-stop-after triggers - a "shutdown session ..." is triggered You could check if sending a "421 Misdirected Request" error to the client could achieve your goal, but it certainly behaves differently than a close in H1 (you can't get a successful answer to the client). It's also a workaround. Triggering GOAWAY/full H2 connection teardown dynamically would need to be implemented. I think in HTX all connection headers are immediately dropped (they are not "translated" and applied to the connection). cheers, lukas [1] https://tools.ietf.org/html/rfc7540#section-9.1.2
Re: Disable client keep-alive using ACL
Hi Am 09.11.20 um 12:36 schrieb Tim Düsterhus, WoltLab GmbH: > is it possible to reliably disable client keep-alive on demand based on > the result of an ACL? > > I was successful for HTTP/1 requests by using: > > http-after-response set-header connection close if foo > > But apparently that has no effect for HTTP/2 requests. I was unable to > find anything within the documentation with regard to this either. > I'd like to bump this. Do you need any additional information to be able to answer the question? Best regards Tim Düsterhus Developer WoltLab GmbH -- WoltLab GmbH Nedlitzer Str. 27B 14469 Potsdam Tel.: +49 331 96784338 duester...@woltlab.com www.woltlab.com Managing director: Marcel Werk AG Potsdam HRB 26795 P
Disable client keep-alive using ACL
Hi List is it possible to reliably disable client keep-alive on demand based on the result of an ACL? I was successful for HTTP/1 requests by using: http-after-response set-header connection close if foo But apparently that has no effect for HTTP/2 requests. I was unable to find anything within the documentation with regard to this either. Best regards Tim Düsterhus Developer WoltLab GmbH -- WoltLab GmbH Nedlitzer Str. 27B 14469 Potsdam Tel.: +49 331 96784338 duester...@woltlab.com www.woltlab.com Managing director: Marcel Werk AG Potsdam HRB 26795 P
