Re: sendfile in darwin

2008-05-16 Thread Jim Jagielski
On May 16, 2008, at 10:43 AM, William A. Rowe, Jr. wrote: Jim Jagielski wrote: I am assuming that this is NOT with the latest apr-1.3 HEAD code. Can you confirm that with the latest, all is well? Right - these issues were pre-patch. Once you applied r656722, all is well now. woot! Base

Re: sendfile in darwin

2008-05-16 Thread William A. Rowe, Jr.
Jim Jagielski wrote: I am assuming that this is NOT with the latest apr-1.3 HEAD code. Can you confirm that with the latest, all is well? Right - these issues were pre-patch. Once you applied r656722, all is well now. Based on Roy's legitimately rabid reaction to the cruft that blows into his

Re: sendfile in darwin

2008-05-16 Thread Dirk-Willem van Gulik
On May 14, 2008, at 7:33 PM, Jim Jagielski wrote: I'll have to try that... in the meantime, does re-adding the 'rv=0' line fix it? For me - yes ! Dw.

Re: sendfile in darwin

2008-05-16 Thread Jim Jagielski
I am assuming that this is NOT with the latest apr-1.3 HEAD code. Can you confirm that with the latest, all is well? I believe this is the only outstanding issue with apr?-1.3, meaning that we can hopefully release APR and then httpd (I don't want to delay httpd, but it is waiting for either 1.3

Re: sendfile in darwin

2008-05-15 Thread William A. Rowe, Jr.
FYI - I'm not testing darwin <> darwin, it's just as easy to test darwin to another OS such as linux or solaris :) I actually tested both directions, but really we are interested in OS X as the ./sendfile client. The ./sendfile server could be anything. Results (on a loopback test) attached. B

Re: sendfile in darwin

2008-05-15 Thread Jim Jagielski
After pouring over the Darwin sendfile() code and setting up 2 Leopards to do the APR sendfile tests (which *did* result in the errors popping up), I've committed r656659. The main issue is that we should *always* return if sendfile returns -1, with EAGAIN and some data sent; previously that would

Re: sendfile in darwin

2008-05-15 Thread Jim Jagielski
On May 14, 2008, at 7:56 PM, William A. Rowe, Jr. wrote: Jim Jagielski wrote: Trying to recreate so far, no luck. This was at today's trunk. Again I'd hint, try across the wire with ./sendfile server sitting on yet another box. It's really effective at making these errors pop out at

Re: sendfile in darwin

2008-05-14 Thread William A. Rowe, Jr.
Jim Jagielski wrote: Trying to recreate so far, no luck. This was at today's trunk. Again I'd hint, try across the wire with ./sendfile server sitting on yet another box. It's really effective at making these errors pop out at us.

Re: sendfile in darwin

2008-05-14 Thread Jim Jagielski
Trying to recreate so far, no luck. On Wed, May 14, 2008 at 04:06:34PM -0500, William A. Rowe, Jr. wrote: > sendfile client timeout now works locally. > > sendfile client nonblocking test now fails. > > server dies at apr_socket_recv()->70014/End of file found > > client dies at apr_socket_

Re: sendfile in darwin

2008-05-14 Thread William A. Rowe, Jr.
sendfile client timeout now works locally. sendfile client nonblocking test now fails. server dies at apr_socket_recv()->70014/End of file found client dies at apr_socket_sendfile()->70014, sent 0 bytes client problem: sent 227008 of 370049 bytes Jim Jagielski wrote: GOT IT! That was it. Pat

Re: sendfile in darwin

2008-05-14 Thread Jim Jagielski
GOT IT! That was it. Patching apr-trunk and apr-1.3 now. On May 14, 2008, at 2:35 PM, Jim Jagielski wrote: Hmmm I'm thinking it has to do when the value passed to sendfile for *len is 0... doing some checking now.

Re: sendfile in darwin

2008-05-14 Thread Jim Jagielski
Hmmm I'm thinking it has to do when the value passed to sendfile for *len is 0... doing some checking now.

Re: sendfile in darwin

2008-05-14 Thread William A. Rowe, Jr.
Corrupt patch, would you mind resending at a text/plain attach? Jim Jagielski wrote: On May 14, 2008, at 1:52 PM, William A. Rowe, Jr. wrote: Jim Jagielski wrote: I'll have to try that... in the meantime, does re-adding the 'rv=0' line fix it? With flying colors. Yes. But I'm guessing

Re: sendfile in darwin

2008-05-14 Thread Jim Jagielski
On May 14, 2008, at 1:52 PM, William A. Rowe, Jr. wrote: Jim Jagielski wrote: I'll have to try that... in the meantime, does re-adding the 'rv=0' line fix it? With flying colors. Yes. But I'm guessing it dies with the httpd test framework. Can you try with the below patch: Index: netwo

Re: sendfile in darwin

2008-05-14 Thread William A. Rowe, Jr.
Jim Jagielski wrote: I'll have to try that... in the meantime, does re-adding the 'rv=0' line fix it? With flying colors. Yes. Bill

Re: sendfile in darwin

2008-05-14 Thread Jim Jagielski
On May 14, 2008, at 1:12 PM, William A. Rowe, Jr. wrote: Jim Jagielski wrote: I wonder if you are seeing timing issues rather than "real" errors... Confirmed this is not so, and that you can reproduce it. Although your machine is too fast to observe the error (parallel CPU's), ./ sendfil

Re: sendfile in darwin

2008-05-14 Thread William A. Rowe, Jr.
Jim Jagielski wrote: I wonder if you are seeing timing issues rather than "real" errors... Confirmed this is not so, and that you can reproduce it. Although your machine is too fast to observe the error (parallel CPU's), ./sendfile's authors already anticipated that. From another box; ./sen

Re: sendfile in darwin

2008-05-13 Thread William A. Rowe, Jr.
Jim Jagielski wrote: On May 13, 2008, at 4:28 PM, William A. Rowe, Jr. wrote: Jim Jagielski wrote: I wonder if you are seeing timing issues rather than "real" errors... are client messages. However, this could be either 1. timing errors, or 2. unusual EINTR or other signal noise. I also

Re: sendfile in darwin

2008-05-13 Thread Jim Jagielski
On May 13, 2008, at 4:28 PM, William A. Rowe, Jr. wrote: Jim Jagielski wrote: I wonder if you are seeing timing issues rather than "real" errors... ./sendfile server & ./sendfile client timeout I get > Waiting for a client to connect... > Processing a client... and then failure. >

Re: sendfile in darwin

2008-05-13 Thread William A. Rowe, Jr.
Jim Jagielski wrote: I wonder if you are seeing timing issues rather than "real" errors... ./sendfile server & ./sendfile client timeout I get > Waiting for a client to connect... > Processing a client... and then failure. > Creating a test file... > Sending the file... are cl

Re: sendfile in darwin

2008-05-13 Thread Jim Jagielski
On May 13, 2008, at 2:59 PM, William A. Rowe, Jr. wrote: Jim Jagielski wrote: On May 7, 2008, at 5:58 PM, Geoff Greer wrote: OK, so there are two problems here: 1. Timeout sockets are broken in r654185. To test this cd to apr/ test, make, then run ./sendfile server & ./sendfile client tim

Re: sendfile in darwin

2008-05-13 Thread William A. Rowe, Jr.
Jim Jagielski wrote: On May 7, 2008, at 5:58 PM, Geoff Greer wrote: OK, so there are two problems here: 1. Timeout sockets are broken in r654185. To test this cd to apr/test, make, then run ./sendfile server & ./sendfile client timeout I cannot confirm that. Works fine here. Still broken

Re: sendfile in darwin

2008-05-12 Thread William A. Rowe, Jr.
Jim Jagielski wrote: On May 11, 2008, at 10:36 PM, William A. Rowe, Jr. wrote: A 10 minute old report on [EMAIL PROTECTED] implies this could very well be an issue on FreeBSD. Can you point out that post on [EMAIL PROTECTED] Not sure if it implies that the rv reset is "required" on FreeBSD

Re: sendfile in darwin

2008-05-12 Thread Jim Jagielski
On May 11, 2008, at 10:36 PM, William A. Rowe, Jr. wrote: Jim Jagielski wrote: On May 7, 2008, at 5:58 PM, Geoff Greer wrote: Of course reverting the change (adding the rv=0 line again) causes the httpd tests to fail. The Darwin impl assumes that it is very much like FreeBSDs except for

Re: sendfile in darwin

2008-05-11 Thread William A. Rowe, Jr.
Jim Jagielski wrote: On May 7, 2008, at 5:58 PM, Geoff Greer wrote: Of course reverting the change (adding the rv=0 line again) causes the httpd tests to fail. The Darwin impl assumes that it is very much like FreeBSDs except for the variation in arguments and the fact that we "aren't sure"

Re: sendfile in darwin

2008-05-09 Thread Jim Jagielski
FWIW, the current impl of sendfile() in Darwin (as coded in xnu-1228.3.13 at least) internally uses writev() (but with real error checking :) ) to send headers and footers, so breaking them out in APR really isn't that bad :) However: if (uap->nbytes != USER_A

Re: sendfile in darwin

2008-05-09 Thread Jim Jagielski
On May 8, 2008, at 10:48 PM, William A. Rowe, Jr. wrote: Jim Jagielski wrote: Can anyone else confirm that r654186 (for apr-trunk) and r654186 (for apr-1.3) fixes sendfile to work under Darwin? In particular, both the perl-test framework for httpd as well as APR's test suite (and test/sendfile

Re: sendfile in darwin

2008-05-09 Thread Jim Jagielski
Yeah... both trunk and 1.3 do not handle the case where the writing of the headers/footers are quite correct with handling that case (and partial writes). My other version (which uses apr_socket_sendv is much better. I think I'll go ahead and commit. On May 8, 2008, at 7:49 PM, Roy T. Fielding w

Re: sendfile in darwin

2008-05-09 Thread Dirk-Willem van Gulik
On May 8, 2008, at 10:07 PM, Jim Jagielski wrote: Can anyone else confirm that r654186 (for apr-trunk) and r654186 (for apr-1.3) fixes sendfile to work under Darwin? In particular, both the perl-test framework for httpd as well as APR's test suite (and test/sendfile ) pass for me now. I c

Re: sendfile in darwin

2008-05-08 Thread William A. Rowe, Jr.
Jim Jagielski wrote: Can anyone else confirm that r654186 (for apr-trunk) and r654186 (for apr-1.3) fixes sendfile to work under Darwin? In particular, both the perl-test framework for httpd as well as APR's test suite (and test/sendfile ) pass for me now. ggreer reports 9.2.2 Darwin Kerne

Re: sendfile in darwin

2008-05-08 Thread Roy T. Fielding
On May 8, 2008, at 1:07 PM, Jim Jagielski wrote: Can anyone else confirm that r654186 (for apr-trunk) and r654186 (for apr-1.3) fixes sendfile to work under Darwin? In particular, both the perl-test framework for httpd as well as APR's test suite (and test/sendfile ) pass for me now. I lo

Re: sendfile in darwin

2008-05-08 Thread Jim Jagielski
Can anyone else confirm that r654186 (for apr-trunk) and r654186 (for apr-1.3) fixes sendfile to work under Darwin? In particular, both the perl-test framework for httpd as well as APR's test suite (and test/sendfile ) pass for me now.

Re: sendfile in darwin

2008-05-08 Thread Jim Jagielski
On May 7, 2008, at 5:58 PM, Geoff Greer wrote: Of course reverting the change (adding the rv=0 line again) causes the httpd tests to fail. The Darwin impl assumes that it is very much like FreeBSDs except for the variation in arguments and the fact that we "aren't sure" about the data countin

Re: sendfile in darwin

2008-05-08 Thread Jim Jagielski
On May 7, 2008, at 5:58 PM, Geoff Greer wrote: OK, so there are two problems here: 1. Timeout sockets are broken in r654185. To test this cd to apr/test, make, then run ./sendfile server & ./sendfile client timeout I cannot confirm that. Works fine here. % ./sendfile server & ./sendfile c

Re: sendfile in darwin

2008-05-08 Thread Dirk-Willem van Gulik
On May 7, 2008, at 11:58 PM, Geoff Greer wrote: 2. httpd on OS X sometimes hangs for a couple seconds. This could have many causes, including a kernel bug. I don't know what version of apr and httpd Dirk was using, but anything outside the 1.3 branch or trunk on 10.5 has sendfile disabled

Re: sendfile in darwin

2008-05-07 Thread Geoff Greer
OK, so there are two problems here: 1. Timeout sockets are broken in r654185. To test this cd to apr/test, make, then run ./sendfile server & ./sendfile client timeout Of course reverting the change (adding the rv=0 line again) causes the httpd tests to fail. I don't have the httpd tests set up

Re: sendfile in darwin

2008-05-07 Thread Dirk-Willem van Gulik
On May 7, 2008, at 8:58 PM, Mads Toftum wrote: On Wed, May 07, 2008 at 02:50:46PM -0400, Jim Jagielski wrote: To clarify: I have no idea what you are referring to as far as what issue you were discussing. This is because it was on IRC. For myself and everyone else who did not happen to be on I

Re: sendfile in darwin

2008-05-07 Thread Dirk-Willem van Gulik
On May 7, 2008, at 8:42 PM, Jim Jagielski wrote: On May 7, 2008, at 2:31 PM, Dirk-Willem van Gulik wrote: Hmm - Jim - that does not quite solve the issue I was discussing on IRC; On IRC? What happened to onlist discussions?? :) Sorry - I was debugging something totally different (I thoug

Re: sendfile in darwin

2008-05-07 Thread Mads Toftum
On Wed, May 07, 2008 at 02:50:46PM -0400, Jim Jagielski wrote: > To clarify: I have no idea what you are referring to > as far as what issue you were discussing. This is > because it was on IRC. For myself and everyone else who > did not happen to be on IRC at that moment of time, we > have no idea

Re: sendfile in darwin

2008-05-07 Thread Jim Jagielski
On May 7, 2008, at 2:42 PM, Jim Jagielski wrote: On May 7, 2008, at 2:31 PM, Dirk-Willem van Gulik wrote: Hmm - Jim - that does not quite solve the issue I was discussing on IRC; On IRC? What happened to onlist discussions?? :) To clarify: I have no idea what you are referring to as far

Re: sendfile in darwin

2008-05-07 Thread Jim Jagielski
On May 7, 2008, at 2:31 PM, Dirk-Willem van Gulik wrote: Hmm - Jim - that does not quite solve the issue I was discussing on IRC; On IRC? What happened to onlist discussions?? :) I think below is needed (which does solve the TimeOut issue). --> diff with your version -- anticipate nbytes

Re: sendfile in darwin

2008-05-07 Thread Dirk-Willem van Gulik
Hmm - Jim - that does not quite solve the issue I was discussing on IRC; I think below is needed (which does solve the TimeOut issue). --> diff with your version -- anticipate nbytes set to 0 (which has whole file semantics on BSD and Darwin). Does that make sense ? Dw Index: network_io/u

Re: sendfile in darwin

2008-05-07 Thread Jim Jagielski
On May 7, 2008, at 1:29 PM, Jim Jagielski wrote: Squashed... Spent a LOT of time on this (even ported the Linux impl to Darwin, which actually is nicer since it uses apr_socket_sendv() and does things like checking partial header/footer stuff, which I may eventually commit to trunk) and the fix

Re: sendfile in darwin

2008-05-07 Thread Jim Jagielski
On May 7, 2008, at 9:31 AM, Jim Jagielski wrote: On May 7, 2008, at 9:25 AM, Jim Jagielski wrote: On May 7, 2008, at 9:14 AM, Jim Jagielski wrote: Just a quick FYI: Simply using the FreeBSD logic flow for sendfile (ie: adding || defined(DARWIN) and pulling out the existing DARWIN-specific

Re: sendfile in darwin

2008-05-07 Thread Jim Jagielski
On May 7, 2008, at 9:25 AM, Jim Jagielski wrote: On May 7, 2008, at 9:14 AM, Jim Jagielski wrote: Just a quick FYI: Simply using the FreeBSD logic flow for sendfile (ie: adding || defined(DARWIN) and pulling out the existing DARWIN-specific section) clears up the problems I saw with the getf

Re: sendfile in darwin

2008-05-07 Thread Jim Jagielski
On May 7, 2008, at 9:14 AM, Jim Jagielski wrote: Just a quick FYI: Simply using the FreeBSD logic flow for sendfile (ie: adding || defined(DARWIN) and pulling out the existing DARWIN-specific section) clears up the problems I saw with the getfile tests... So I guess I need to dig deeper on thi

Re: sendfile in darwin

2008-05-07 Thread Jim Jagielski
Just a quick FYI: Simply using the FreeBSD logic flow for sendfile (ie: adding || defined(DARWIN) and pulling out the existing DARWIN-specific section) clears up the problems I saw with the getfile tests... So I guess I need to dig deeper on this... BTW: This is on OS X 10.5.2 (Darwin 9.2.2) On

Re: sendfile in darwin

2008-05-06 Thread Jim Jagielski
I should have some cycles to dig deeper on this tomorrow...

Re: sendfile in darwin

2008-05-06 Thread Jim Jagielski
On May 6, 2008, at 1:53 PM, William A. Rowe, Jr. wrote: Jim Jagielski wrote: Will see how the latest 1.2.x does... Uhm - 1.2.x deliberately disables sendfile detection on Darwin. Yep, and so the test runs cleanly (confirmed)...

Re: sendfile in darwin

2008-05-06 Thread William A. Rowe, Jr.
Jim Jagielski wrote: Will see how the latest 1.2.x does... Uhm - 1.2.x deliberately disables sendfile detection on Darwin.

sendfile in darwin

2008-05-06 Thread Jim Jagielski
Hmmm at least in apr-1.3.0 (HEAD), sendfile in Darwin still seems broken... getting: [Tue May 06 13:35:26 2008] [crit] [Tue May 06 13:35:26 2008] file core_filters.c, line 391, assertion "total_bytes_left > 0 && tmplen > 0" failed from the test framewor