I looked at the code a bit closer. The ns_returnfile and ns_respond
commands both call Ns_ConnReturnFile, the public API to the underlying
FastPath. It does more than just blast the content -- it handles:
-- respond "not found" if file doesn't exist
-- sets "last modified header" based on mtime
-- returns "not modified" if client sent "if-modified-since"
-- returns headers only if a HEAD request
-- sets mimetype and contentlength
-- caches, mmap's, or simply opens the fd and sends, chunk by chunk
The code is a bit nested, with a few layers of code which handle
similar cases (send file, FILE *, fd, etc.). It's not 100% perfect
because the file could change between the stat to check existence and
get size/mtime for header construction and the actual send. There's
even a note that say something like "silently ignore truncated
file..." deeper down so that if fastpath stats a file of size x but
opens and starts sending a smaller file, it'll just silently quit
after sending what it found. By comparison, the ADP template read/
parse/cache code attempts to catch that case -- it tries up to 10
times in ParseFile to get a consistent fstat/read on the open file
(you can see that in adpeval.c).
Overall, this means ns_returnfile is designed to mimic the core file-
send/fastpath stuff as if there was no Tcl code involved at all --
performance, pitfalls and all. Whether fastpath is significantly
smarter or dumber sending static content than, say, Apache, I can't
say. This also means that a new "-cache 1/0" option to
ns_returnfile (default 0 or 1, subject to debate) could be useful but
not quite the same as "just send this file..." One would still get
the result AOLserver would have generated (not found, not modified,
etc.) unless all that was disabled as well.
By comparison, ns_returnfp is more basic -- it assumes it's sending
stuff from an open channel, just sets basic headers, and has none of
the "not found", "not modified", etc. type logic. Because the docs
don't mention this, I think some folks expect ns_returnfile to be just
this:
set fp [open file]
ns_returnfp $fp
close $fp
BTW: I mentioned that I thought it worked that way in the past but I
think I was wrong. Checking the CVS logs and diff'ing the files with
rev 1.1 way back in 2000 ns_returnfile was calling the fastpath and
fastpath was using the dev/inode pair for the cache key on Unix.
Overall, it seems one thing to do would be to switch to filename-based
cache keys by default, leaving the dev/inode pair as an option for
folks who run sites with large symlinks and want to benefit from
caching objects just once. I think that should avoid the data
corruption cases John pointed out with minimal downside. We could
also modify ns_returnfile to behave as above, calling ns_returnfp, but
I'm thinking that may break more than it fixes if folks are assuming
the behavior it has now (e.g, an ns_returnfile of an image where they
expect the not-modified responses).
-Jim
On Aug 20, 2008, at 9:23 AM, Titi Alailima wrote:
-----Original Message-----
From: AOLserver Discussion [mailto:[EMAIL PROTECTED] On
Behalf Of Jim Davidson
Sent: Tuesday, August 19, 2008 8:39 PM
To: AOLSERVER@LISTSERV.AOL.COM
Subject: Re: [AOLSERVER] Data "corruption" with fastpath caching
Your right, the code snippet below could trip over a race condition
as
you've described. But, that's not reason enough to change the
fastpath, it's reason to better document the behavior so folks don't
write code that uses ns_returnfile for temporary, dynamic content.
Although fastpath takes care to be correct in most cases (e.g.,
stat'ing the file on each request and serializing read on cache
miss),
the "fast" in fastpath is because it's primarily designed to return
simple static content with minimal overhead.
BTW: I believe the ns_returnfile command didn't use the fastpath
originally -- I think it just opened and sent the content. It was
changed because folks asked for it to go faster I think -- can't
recall.
This sounds like the problem. Not a bug with fastpath, but a
questionable decision to take an apparently generic command and
apply a situation-specific optimizer to it that returns bad results
when applied outside of its design parameters. It seemed like a
valid assumption at the time, just as John's code seemed to be
making valid assumptions, but it just seems to have led to asses
being made left and right.
Is there any way to make ns_returnfile's use of fastpath
configurable, and turn _that_ off by default? Or would that be
essentially the same as just turning fastpath off entirely? I must
confess my ignorance of where fastpath gets used.
Titi Ala'ilima
Lead Architect
MedTouch LLC
1100 Massachusetts Avenue
Cambridge, MA 02138
617.621.8670 x309
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]
> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the
Subject: field of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]>
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject:
field of your email blank.
