Thanks Greg. Just so I’m clear, what does Jetty key on to know whether to close the connection? Just the 4xx/5xx response code? I’m trying to understand the difference between this case and the “normal unconsumed input” case you describe. Also, I did notice that Jetty does not set the Connection: close header when it does this, is that intentional?
> On Sep 25, 2018, at 6:37 PM, Greg Wilkins <gr...@webtide.com> wrote: > > > Thomas, > > There is no configuration to avoid this behaviour. If jetty sees and > exception in the application it will send the 400 and close the connection. > > However, as Simone says, your application can be setup to avoid this > situation by catching the exception and consuming any input. You can do this > in a filter that catches Throwable, it can then check the request input > stream (and/or reader) for unconsumed input and read & discard to end of > file. If the response is not committed, it can then send a 400 or any other > response that you like. > > Just remember that this may make your application somewhat vulnerable to DOS > attacks as it will be easy to hold a thread in that filter slowly consuming > data. I would suggest imposing a total time and total data limit on the > input consumption. > > Note that for normal unconsumed input, jetty 9.4 does make some attempt to > consume it... but if the reading of that data would block, it gives up and > closes the connection, as there is no point blocking for data that will be > discarded. > > regards > > > > > > > > > > > On Wed, 26 Sep 2018 at 07:35, Thomas Becker <twbec...@gmail.com > <mailto:twbec...@gmail.com>> wrote: > Thanks so much again for your response, this is great information. What you > say makes sense, but I now see I failed to mention the most critical part of > this problem. Which is that the client never actually sees the 400 response > we are sending from Jetty. When varnish sees the RST, it considers the > backend request failed and returns 503 Service Unavailable to the client, > effectively swallowing our application’s response. We can pursue a solution > to this on the Varnish side, but in the interim I’m guessing there is no way > to configure this behavior in Jetty? > > > >> On Sep 25, 2018, at 4:28 PM, Simone Bordet <sbor...@webtide.com >> <mailto:sbor...@webtide.com>> wrote: >> >> Hi, >> >> On Tue, Sep 25, 2018 at 8:34 PM Tommy Becker <twbec...@gmail.com >> <mailto:twbec...@gmail.com>> wrote: >>> >>> Update: we setup an environment with the old Jetty 9.2 code and this does >>> not occur. 9.2 does not send the FIN in #5 above, and seems happy to >>> receive the rest of the content, despite having sent a response already. >>> >>> On Tue, Sep 25, 2018 at 10:01 AM Tommy Becker <twbec...@gmail.com >>> <mailto:twbec...@gmail.com>> wrote: >>>> >>>> Thanks for your response. I managed to snag a tcp dump of what's going on >>>> in this scenario. From what I can see the sequence of events is the >>>> following. Recall that our Jetty server is fronted by a Varnish cache. >>>> >>>> 1) Varnish sends the headers and initial part of the content for a large >>>> POST. >>>> 2) On the Jetty server, we use a streaming parser and begin validating the >>>> content. >>>> 3) We detect a problem with the content and throw an exception that >>>> results in a 400 Bad Request to the client (via JAX-RS exception mapper) >>>> 4) An ACK is sent for the segment containing the 400 error. >>>> 5) The Jetty server sends a FIN. >>>> 6) An ACK is sent for the FIN >>>> 7) Varnish sends another segment that continues the content from #1. >>>> 8) The Jetty server sends a RST. >>>> >>>> In the server logs, we see an Early EOF from our JAX-RS resource that is >>>> parsing the content. This all seems pretty ok from the Jetty side, and it >>>> certainly seems like Varnish is misbehaving here (I'm thinking it may be >>>> this bug https://github.com/varnishcache/varnish-cache/issues/2332 >>>> <https://github.com/varnishcache/varnish-cache/issues/2332>). But I'm >>>> still unclear as to why this started after our upgrade from Jetty 9.2 -> >>>> 9.4. Any thoughts? >>>> >> >> This is normal. >> In Jetty 9.4 we are more aggressive in closing the connection because >> we don't want to be at the mercy of a possible nasty client sending us >> GiB of data when we know the application does not want to handle them. >> Varnish behavior is correct too: it sees the FIN from Jetty but does >> not know that Jetty does not want to read until it tries to send more >> content and gets a RST. >> At that point, it should relay the RST (or FIN) back to the client. >> >> So you have 2 choices: you catch the exception during your validation, >> and finish to read (and discard) the content in the application; or >> you ignore the early EOFs in the logs. >> I don't think that those early EOFs are logged above DEBUG level, is >> that correct? >> >> -- >> Simone Bordet >> ---- >> http://cometd.org <http://cometd.org/> >> http://webtide.com <http://webtide.com/> >> Developer advice, training, services and support >> from the Jetty & CometD experts. >> _______________________________________________ >> jetty-users mailing list >> jetty-users@eclipse.org <mailto:jetty-users@eclipse.org> >> To change your delivery options, retrieve your password, or unsubscribe from >> this list, visit >> https://dev.eclipse.org/mailman/listinfo/jetty-users >> <https://dev.eclipse.org/mailman/listinfo/jetty-users> > _______________________________________________ > jetty-users mailing list > jetty-users@eclipse.org <mailto:jetty-users@eclipse.org> > To change your delivery options, retrieve your password, or unsubscribe from > this list, visit > https://dev.eclipse.org/mailman/listinfo/jetty-users > <https://dev.eclipse.org/mailman/listinfo/jetty-users> > > -- > Greg Wilkins <gr...@webtide.com <mailto:gr...@webtide.com>> CTO > http://webtide.com <http://webtide.com/> > _______________________________________________ > jetty-users mailing list > jetty-users@eclipse.org > To change your delivery options, retrieve your password, or unsubscribe from > this list, visit > https://dev.eclipse.org/mailman/listinfo/jetty-users
_______________________________________________ jetty-users mailing list jetty-users@eclipse.org To change your delivery options, retrieve your password, or unsubscribe from this list, visit https://dev.eclipse.org/mailman/listinfo/jetty-users
