Update: we setup an environment with the old Jetty 9.2 code and this does not occur. 9.2 does not send the FIN in #5 above, and seems happy to receive the rest of the content, despite having sent a response already.
On Tue, Sep 25, 2018 at 10:01 AM Tommy Becker <twbec...@gmail.com> wrote: > Thanks for your response. I managed to snag a tcp dump of what's going on > in this scenario. From what I can see the sequence of events is the > following. Recall that our Jetty server is fronted by a Varnish cache. > > 1) Varnish sends the headers and initial part of the content for a large > POST. > 2) On the Jetty server, we use a streaming parser and begin validating the > content. > 3) We detect a problem with the content and throw an exception that > results in a 400 Bad Request to the client (via JAX-RS exception mapper) > 4) An ACK is sent for the segment containing the 400 error. > 5) The Jetty server sends a FIN. > 6) An ACK is sent for the FIN > 7) Varnish sends another segment that continues the content from #1. > 8) The Jetty server sends a RST. > > In the server logs, we see an Early EOF from our JAX-RS resource that is > parsing the content. This all seems pretty ok from the Jetty side, and it > certainly seems like Varnish is misbehaving here (I'm thinking it may be > this bug https://github.com/varnishcache/varnish-cache/issues/2332). But > I'm still unclear as to why this started after our upgrade from Jetty 9.2 > -> 9.4. Any thoughts? > > > > > > > On Thu, Sep 20, 2018 at 5:14 AM Simone Bordet <sbor...@webtide.com> wrote: > >> Hi, >> >> On Wed, Sep 19, 2018 at 4:06 PM Tommy Becker <twbec...@gmail.com> wrote: >> > >> > Hi folks, >> > We recently upgraded our application from Jetty 9.2 to 9.4. After doing >> so, we've noticed a lot of Early EOF stacktraces happening on POST >> requests. Of note, our application is fronted by a Varnish cache, which >> holds connections to the Jetty backend open indefinitely, even though we >> have the Jetty idle timeout configured to 50s. We're aware this is sub >> optimal, but it has been this way for some time, and worked fine under >> Jetty 9.2. To be clear, we do not have any evidence, or even believe that >> Jetty is doing anything wrong here per-se. But I understand that a fair bit >> of work happened in Jetty in this area (timeout/socket close) between 9.2 >> and 9.4 and I'm trying to get a high level description of what changed in >> an attempt to explain the behavior we're seeing now. Any info you could >> provide would be appreciated! >> > >> >> There have been many changes between Jetty 9.2 and 9.4 in many places. >> As an example, we have improved and made more compliant/strict HTTP >> parsing, and that may be the cause of your POSTs not going through. >> >> The best thing you can do to figure out this issue is to grab a >> network trace via Wireshark on the Jetty host, along with possibly a >> stack trace (that you did not report and you don't even say if it's on >> client or on server), and possibly server DEBUG logs. >> >> The network trace will be important to know who's closing the >> connection. Stack traces and logs to know why it was closed. >> >> -- >> Simone Bordet >> ---- >> http://cometd.org >> http://webtide.com >> Developer advice, training, services and support >> from the Jetty & CometD experts. >> _______________________________________________ >> jetty-users mailing list >> jetty-users@eclipse.org >> To change your delivery options, retrieve your password, or unsubscribe >> from this list, visit >> https://dev.eclipse.org/mailman/listinfo/jetty-users >> >
_______________________________________________ jetty-users mailing list jetty-users@eclipse.org To change your delivery options, retrieve your password, or unsubscribe from this list, visit https://dev.eclipse.org/mailman/listinfo/jetty-users
