I'm looking into this... I think the connection thing is a bit of red herring. If you allow HTTP/1.1 then your test uses 6 connections total until the crash, the first 5 to just load the static content. However, if you remove it, then every repl test is its own connection. This has the effect of each repl test getting its own thread and custodian. So, it's not that connections are leaking, but that by re-using the same connection, some other thing that is leaking lasts a long time. [BTW, you don't need to hack the code to do this, just pass #:connection-close? #t to serve/servlet.]
I gathered per-connection memory stats and found that it takes about 160 reqs to GC. It dies after 24 such cycles. Most allocate 30mbs. Most leak 3 mbs. I cut out the entire continuation apparatus of the server and had two interesting results. First, the memory still grows unboundedly and second the sandbox doesn't catch it. This suggests to me that the leaked memory is not accounted to the sandbox custodian. The part of the server that is left uses no global data structures, is only using six threads at most, and does not use continuations or non-tail recursive loops. It should run in constant memory. So, now it's my turn to cry wolf and say that something strange is happening in the VM. My results, including logs and modification to the test server are here: https://github.com/jeapostrophe/web-server-under-sandbox to get the graphs and analysis, just link one of the logs to "log" and run "log.rkt" Jay On Sat, Apr 20, 2013 at 5:57 PM, Danny Yoo <d...@hashcollision.org> wrote: >> Let me post code and see if anyone agrees: >> >> https://github.com/dyoo/web-server-under-sandbox >> >> I create a trivial web server with a servlet that immediately errors >> out. I have it also serve static JavaScript that's essentially a >> torture test: it keeps sending requests to the web server, over and >> over. (Ripped out of whalesong's REPL testing code). > > > Followup: I've gotten independent confirmation from Eric Hanchrow that > he can replicate my results, so at least I'm not hallucinating. > > I've been able to trace down the source of the garbage somewhere due > to the reuse of the connection to support HTTP/1.1. If I hack this > line in the codebase: > > > https://github.com/plt/racket/blob/master/collects/web-server/private/dispatch-server-unit.rkt#L106 > > > so that it _always_ closes the connection, then the resulting web > server survives my torture test. > > > So there's some kind of state that's accumulating due to the > connection reuse to support HTTP/1.1. Still trying to trace the exact > nature of the problem down. -- Jay McCarthy <j...@cs.byu.edu> Assistant Professor / Brigham Young University http://faculty.cs.byu.edu/~jay "The glory of God is Intelligence" - D&C 93 ____________________ Racket Users list: http://lists.racket-lang.org/users
