On Wed, 2020-05-27 at 14:23 +0200, Michael Osipov wrote: > Am 2020-05-27 um 12:14 schrieb Oleg Kalnichevski: > > On Wed, 2020-05-27 at 00:01 +0200, Michael Osipov wrote: > > > Am 2020-05-26 um 23:09 schrieb Oleg Kalnichevski: > > > > On Tue, 2020-05-26 at 22:28 +0200, Michael Osipov wrote: > > > > > Am 2020-05-26 um 20:20 schrieb Oleg Kalnichevski: > > > > > > On Tue, 2020-05-26 at 17:58 +0000, Bernd Eckenfels wrote: > > > > > > > Michael, this looks a bit like the packets in between > > > > > > > have > > > > > > > been > > > > > > > TLS > > > > > > > handshakes which have not been carried out because the > > > > > > > engine > > > > > > > was > > > > > > > not > > > > > > > kicked off. Maybe a starHandshake() would help? Or can > > > > > > > you > > > > > > > share > > > > > > > the > > > > > > > full traces? Did you try http as well? > > > > > > > > > > > > > > > > > > > Michael, > > > > > > > > > > > > I can only second what Bernd has said. Try to simplify the > > > > > > setup > > > > > > and > > > > > > see if you can get things to work with plain HTTP first. > > > > > > > > > > > > Also feel free to tweak my code as you see fit. > > > > > > > > > > I followed Bernd's advise and disabled TLS. Although I > > > > > already > > > > > had a > > > > > clue what is going wrong. Attached is a patch on top of > > > > > Oleg's > > > > > branch > > > > > which makes it work unencrypted and encrypted. I will explain > > > > > in > > > > > detail > > > > > why the changes are necessary. > > > > > > > > > > > - public static final Timeout > > > > > > DEFAULT_WAIT_FOR_EARLY_RESPONSE = > > > > > > Timeout.ofMilliseconds(5); > > > > > > + public static final Timeout > > > > > > DEFAULT_WAIT_FOR_EARLY_RESPONSE = > > > > > > Timeout.ofMilliseconds(50); > > > > > > > > > > unless client and server are physically next to each other 5 > > > > > ms > > > > > are > > > > > virtually impossible. I have tried with our server at work. I > > > > > am > > > > > connected with my laptop via VPN to the corporate network, I > > > > > don't > > > > > know > > > > > where the peering point of our VPN provider is, but the > > > > > server is > > > > > physically 10 km away from me and with TLS 1.3 it takes 28 ms > > > > > to > > > > > produce > > > > > the 401. > > > > > > > > > > Maybe both timeouts (expect and early) should be configurable > > > > > via > > > > > RequestConfig? > > > > > > > > > > > + conn.flush(); > > > > > > > > Michael > > > > > > > > You are absolutely right. Not flushing the request head was the > > > > cause > > > > of the problem. > > > > > > > > Could you please try out a slightly different take on your > > > > original > > > > patch, though? > > > > > > > > > > > > https://github.com/ok2c/httpcomponents-core/commit/39f6d69a87586c147dc080431d45d6d48acb2c80 > > > > > > > > HttpRequestExecutor ought not be flushing all request message > > > > heads > > > > indiscriminately and should still try to pack small payload > > > > messages > > > > into a single IP frame for performance reasons. > > > > > > Makes sense! > > > > > > > I also would not overload RequestConfig with too many options > > > > but > > > > we > > > > can discuss that later. > > > > > > Agreed. > > > > > > > > This is why it did not work before is that headers were stuck > > > > > in > > > > > the > > > > > local buffer and were flushed only when the body was sent. So > > > > > the > > > > > server > > > > > never saw the headers before the body. The wait was > > > > > pointless. > > > > > > > > > > Please also note that even the buffer for headers might be > > > > > large > > > > > due > > > > > to > > > > > some authnz tokens. E.g., JWT or SPNEGO, cookies, etc. > > > > > > > > > > > + response = > > > > > > conn.receiveResponseHeader(); > > > > > > > > > > One needs to read the headers because the client shall be > > > > > able to > > > > > see > > > > > the status code and if fast enough the response body. In my > > > > > case > > > > > the > > > > > error page from Tomcat. > > > > > > > > > > RFC 7230, section 6.5 says: > > > > > > A client sending a message body SHOULD monitor the > > > > > > network > > > > > > connection > > > > > > for an error response while it is transmitting the > > > > > > request. If > > > > > > the > > > > > > client sees a response that indicates the server does > > > > > > not > > > > > > wish > > > > > > to > > > > > > receive the message body and is closing the > > > > > > connection, the > > > > > > client > > > > > > SHOULD immediately cease transmitting the body and > > > > > > close > > > > > > its > > > > > > side of > > > > > > the connection. > > > > > > > > > > I have issued another request and HttpClient reuses the > > > > > connection > > > > > because Tomcat forgot to send "Connection: close". > > > > > > > > Hmm. That sounds weird. HttpClient must terminate the > > > > connection if > > > > it > > > > failed to submit the entire message body declared in `Content- > > > > Length` > > > > header. The #terminateRequest should have marked the connection > > > > as > > > > `inconsistent`. > > > > > > > > Could you please send me a wire log of the session? > > > > > > So far, the patch works and I can access the first 401 response. > > > > > > > This should fix the problem with incorrect connection re-use. > > Please > > review: > > > > https://github.com/apache/httpcomponents-client/commit/a554aadabafe26ae5412b26a311ffa105e623cc2 > > Tried with various timeouts, 5 ms, 15 ms, 25 ms, 50 ms, 500 ms. > Connections are correctly closed when: > * early response is received > * early response did not arrive in time and the server reached its > swallow limit (worked already) > > I ran multiple requests in a loop as well as a subsequent one with > authentication header. > > I am happily accepting this patch! >
Great, but I still do not quite like the patch, though 1. There can be extra latency for normal request / response exchanges due to the early response check. 2. The early response check is performed once and never retried. An out of sequence response can still be sent as a later point. I working on a better fix. Please bear with me. Oleg --------------------------------------------------------------------- To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org For additional commands, e-mail: httpclient-users-h...@hc.apache.org
