Hi, I'm a new subscriber to this list, but not a complete stranger either. My name is Ben Collins-Sussman, and I'm one of the original Subversion developers. I'm also an APR committer, and well... I know a whole lot of people on this list, and met many more of you at the last Apachecon.
I'm writing because I plan to do some major overhauling of mod_dav. No need to fear, because I'm essentially a disciple of gstein. :-) I'm reasonably familiar with mod_dav code already, because I'm intimate with the mod_dav_svn provider (Subversion's main server.) The problem here is that mod_dav is very inefficient about its pool usage. Back in the subversion project, we've developed a set of "best practices" regarding pool usage; it took us a couple years to discover and refine these rules. But mod_dav isn't following these practices, as it was written long ago. As a result, mod_dav is having serious problems scaling to large Subversion repositories. For example: if a repository directory contains thousands (or even hundreds) of entries, a PROPFIND of depth 1 will cause the httpd process to grow absolutely enormous, sometimes using up all available memory. Greg Stein has proposed a 4-part revamp plan for mod_dav in httpd-2.1, which I'd like to run by folks: 1. revamp mod_dav's use of pools internally. This means adding pool arguments to the various "layers" of function calls within mod_dav. Each caller controls subpool creation, and possibly passes a subpool into a subordinate function call. Whereever there are loops that allocate temporary memory, we use (and *clear*) a subpool within the loop. These changes are made up to the point of the provider backend; but we halt there, and do not pass pools into provider callbacks. 2. make all mod_dav_svn callbacks take pools. But then we wrap these callbacks with wrappers that *don't* take pools, so mod_dav_svn can still plug-in to mod_dav. 3. use the API versioning which is present in httpd-2.0 (and later) to define two back-end interfaces for mod_dav. The current interface is v1, and the new "pool accepting" interface becomes v2. mod_dav is updated so that it can use either interface. mod_dav_svn is updated to supply both: the old wrapper-based form, and the new form. This will allow a mix/match between httpd-2.0.X (and 2.X.Y) and mod_dav_svn. 4. we can eventually backport these changes from httpd-2.1 to the httpd-2.0 branch; it still maintains compatibility with v1 providers, such as mod_dav_fs. Comments?
