Re: FieldStorage and multiline headers in multipart/form.
Short answer: Yes, multiline headers are allowed there, and yes, mod_python will fail to read them properly. Longer answer: I checked throught the util.py code, and nope, the fix is not in there. I browsed through the appropriate RFC documents, in particular RFC1521 http://www.faqs.org/rfcs/rfc1521.html This clearly states that RFC822 headers are to be used, which means that those multiline headers are indeed allowed. The 1521 document even contains samples with multiline headers. Just a thought: Can we (re)use the rfc822 mime parser that's already built-in in Python to do the work for us? -- Mike Looijmans Philips Natlab / Topic Automation Graham Dumpleton wrote: With FieldStorage being discussed on main user mailing list, came across this old post of the mailing list: http://www.modpython.org/pipermail/mod_python/2001-November/012256.html What it is saying is that some HTTP clients use multi line headers in sections of multipart/form postings and that mod_python doesn't handle this. Looking at FieldStorage code, which I don't grok right at this minute because of an intensive coding frenzy today on the back of not enough sleep last night, I can't see that it has ever been modified to accomodate such multiline headers if indeed it needs to. Anyone who is more intimate with FieldStorage code want to have a better look at validity of original mailing list post and whether multiline headers are legitimate and whether there indeed could be a problem in mod_python. Graham
Re: Progressing 3.2.9.
Is there any of these you would like me to do, perhaps 77, 94 or 137 since I am familiar with the code? Not knowing what you are up to, wary of just going ahead and doing any in case you are already doing it. Any hints on how to use subversion to merge diff automatically from the other branch? Graham On 11/04/2006, at 12:47 PM, Jim Gallacher wrote: Here is the list of things I still need to backport. Fixes have already been committed to trunk. MODPYTHON-77 The Simplified GIL Aquisition patches. MODPYTHON-94 Support for optional mod_ssl functions on request object. MODPYTHON-131 Make mutex directory configurable. MODPYTHON-145 Make number of mutex locks configurable at apache startup MODPYTHON-137 Add req.server.get_options() for obtain PythonOption values set at global level. MODPYTHON-93 Improved FieldStorage The backports shouldn't take more than an hour and I can likely get to them tommorow. I'd do it now but my head is not entirely clean (too sleepy) so I'd better wait until morning. That just leaves the review and backporting of the logging stuff, and deciding if we need to worry about the auth test failure issue for apache 2.2. Once we sort those out I'll be able to roll a tarball that we can offer as a release candidate. I'm heading out of town on Thursday so if I don't get to it by then it'll have to wait until next Monday. Jim
Re: svn commit: r393037 - in /httpd/httpd/trunk: CHANGES server/protocol.c
Davi Arnaut wrote: It is legal. And Nick is right, that should be fixed in apr_uri. I will send a patch tomorrow. Once again, you describe the http: scheme. Let's be careful about our assumptions, and ensure we don't further constrain apr_uri to the http behavior :) I'm all for apr_uri 'doing the right thing'. We may *also* want to fix (on 2.2) the ap_fn - because we do *not* insist on any particular apr (other than apr 1.2.0 and later). But fixing in apr would ensure we need not leave such a patch on trunk/, because we can then insist on apr 1.3 and later for httpd 2.3/2.4. Bill
Re: svn commit: r392948 - /httpd/httpd/branches/2.2.x/STATUS
Jorge Schrauwen wrote: I was wondering this too while parsing htpasswd files in php. all others have a identified in front to see what encrypt it used... Why doesn't crypt have one? Because why would anyone have a passwd file that doesn't use crypt()? Have to remember htpasswd followed the passwd shadow file, and that was always using crypt. Nobody at the time foresaw a reason to do anything else. The identifier tags came later. Why plain didn't gain an identifier tag is beyond me. Bill
Re: svn commit: r393037 - in /httpd/httpd/trunk: CHANGES server/protocol.c
Nick Kew wrote: Yes, it's legal. The leading slash is implied. Note: that's leading slash, which is firmly different to a trailing slash in a path. for abspath you are correct. but in general; URI = scheme : hier-part [ ? query ] [ # fragment ] hier-part = // authority path-abempty / path-absolute / path-rootless / path-empty injecting a leading slash where an empty (rootless) path is implied will change the meaning on the fly :( Bill
Re: svn commit: r393037 - in /httpd/httpd/trunk: CHANGES server/protocol.c
Roy T. Fielding wrote: On Apr 10, 2006, at 2:50 PM, Ruediger Pluem wrote: I also thought initially to fix this in apr-util, but right know I am not sure about it, because IMHO apr_uri_parse should do generic uri parsing. Setting an empty uri to / seems to be HTTP specific, so I am not sure if we should do this in apr_uri_parse. At least we would need to check whether the scheme is http or https. It probably needs to be updated for RFC 3986 anyway. The path should be set to , not NULL. The HTTP server should take care of the redirect from to /, which in this case means the http-proxy needs to check for when it sends a request and respond with a redirect that adds the /. +1
Re: Intend to tag 2.2.1 on Friday
You mean this Friday or next? Alpha -- View this message in context: http://www.nabble.com/Intend-to-tag-2.2.1-on-Friday-t1359442.html#a3866575 Sent from the Apache HTTP Server - Dev forum at Nabble.com.
Re: svn commit: r393032 - /httpd/httpd/branches/2.0.x/STATUS
On 4/11/06, William A. Rowe, Jr. [EMAIL PROTECTED] wrote: Jeff Trawick wrote: On 4/10/06, William A. Rowe, Jr. [EMAIL PROTECTED] wrote: This pool is or is not created in the parent? If its the parent it's gotta be proc mutexed even if using prefork. It is created in the parent and used only during request processing. Why does it need to be proc mutexed? Each process has its own copy. There is no shared memory involved. Ah... of course. The pages are copy-on-write. So there is no proc mutexing issue :) If it's process-local, then please don't depend on APR_THREADED, please please please poll the MPM to determine if the mpm has_threads? It's a huge waste for prefork users with a shared (threaded) apr :( That minor performance improvement for some (lots of?) prefork users should first be addressed in trunk, and implementation/backport of that should not hold up this critical bug fix. But the code is there, and what we are using in mod_ssl and others. Please don't inject a regression where we already cleaned these up once before :( Nobody ever cleaned up ldap. Now there are perhaps 9 occurrences of a pattern to replace instead of 8. (didn't count exactly) It is a HUGE stretch to call that a regression, since anybody that wants to fix it has to search for the pattern anyway. People have had several years to get around to that performance improvement and nobody cared so far.
Re: perl glue docs?
Jonathan Vanasco wrote: they've disappeared from the website :( were they forked off already ? maybe then add a link to http://search.cpan.org/~joesuf/libapreq2-2.07/glue/perl/lib/Apache2 No, the version of doxygen on minotaur where they were built blew them... I forgot to fix that. Pester me again (directly) in about 2 hours if I haven't. -- Philip M. Gollucci ([EMAIL PROTECTED]) 323.219.4708 Consultant / http://p6m7g8.net/Resume/resume.shtml Senior Software Engineer - TicketMaster - http://ticketmaster.com 1024D/A79997FA F357 0FDD 2301 6296 690F 6A47 D55A 7172 A799 97F It takes a minute to have a crush on someone, an hour to like someone, and a day to love someone, but it takes a lifetime to forget someone...
Re: Intend to tag 2.2.1 on Friday
Alpha Huang wrote: You mean this Friday or next? It's come and gone, I didn't see the RM actually release 2.2.1... folks are rapidly fixing various additional bugs in 2.2 to make it stable for yet another 2.2.2 tag and roll.
Re: Intend to tag 2.2.1 on Friday
I've got a lot of poeple (aka 4 sofar) mailing me (no idea why really) where the tarball's are.Maybe the list avaible in [EMAIL PROTECTED] should be posted on the website aswel?I'm talking about the list that sais which releases where tagged, rolled, GA,... and when. JorgeOn 4/11/06, William A. Rowe, Jr. [EMAIL PROTECTED] wrote: Alpha Huang wrote: You mean this Friday or next?It's come and gone, I didn't see the RM actually release 2.2.1...folksare rapidly fixing various additional bugs in 2.2 to make it stable for yetanother 2.2.2 tag and roll.-- ~Jorge
Re: svn commit: r392948 - /httpd/httpd/branches/2.2.x/STATUS
Thats the problem i think plain and crypt both don't have a Identifier tag,The since the script will be used on win only i went for always plain text.Can emulaion crypt on platforms that don't support it? or will this be to slow? On 4/11/06, William A. Rowe, Jr. [EMAIL PROTECTED] wrote: Jorge Schrauwen wrote: I was wondering this too while parsing htpasswd files in php. all others have a identified in front to see what encrypt it used... Why doesn't crypt have one?Because why would anyone have a passwd file that doesn't use crypt()? Have to remember htpasswd followed the passwd shadow file, and that was alwaysusing crypt.Nobody at the time foresaw a reason to do anything else.Theidentifier tags came later.Why plain didn't gain an identifier tag is beyond me. Bill-- ~Jorge
[PATCH] #39275 MaxClients on startup [Was: Bug in 2.0.56-dev]
Hi --
Alexander Lazic wrote:
After 'make install' i started apache, then some seconds later i got the
message '...MaxClients reached...' but there was no entry in the access
log, and nobody have make a request to this server.
Jeff Trawick wrote:
There are problems accounting for child processes which are trying to
initialize that result in the parent thinking it needs to create more
children. The less harmful flavor is when it thinks (incorrectly) it
is already at MaxClients and issues the reached MaxClients message.
More disturbing is when MaxClients is very high and the parent keeps
creating new children using exponential ramp-up. That can be very
painful.
I have been seeing something similar with 2.2.0 using the worker
MPM, where with the following settings, I get over 10 child processes
initializing immediately (e.g., up to 15), and then they drop back to
10. I see the server reached MaxClients message as well right
after httpd startup, although nothing is connecting yet.
IfModule mpm_worker_module
StartServers 10
MaxClients 150
MinSpareThreads 25
MaxSpareThreads 100
ThreadsPerChild 10
/IfModule
In my case, the problem relates to how long the child_init phase
takes to execute. I can tune this by raising DBDMin (and DBDKeep)
so that mod_dbd attempts to open increasingly large numbers of
DB connections during child_init. With DBDMin set to 0 or 1,
all is well; no funny behaviour. Up at DBDMin and DBDKeep at 3,
that's when (for me) things go pear-shaped.
In server/mpm/worker/worker.c, after make_child() creates a
child process it immediately sets the scoreboard parent slot's pid
value. The main process goes into server_main_loop() and begins
executing perform_idle_server_maintenance() every second; this
looks at any process with a non-zero pid in the scoreboard and
assumes that any of its worker threads marked SERVER_DEAD are,
in fact, dead.
However, if the child processes are starting slowly because
ap_run_child_init() in child_main() is taking its time, then
start_threads() hasn't even been run yet, so the threads aren't
marked SERVER_STARTING -- they're just set to 0 as the default
value. But 0 == SERVER_DEAD, so the main process sees a lot
of dead worker threads and begins spawning new child processes,
up to MaxClients/ThreadsPerChild in the worst case. In this case,
when no worker threads have started yet, but all possible child
processes have been spawned (and are working through their
child_init phases), then the following is true and the
server reached MaxClients message is printed, even though
the server hasn't started accepting connections yet:
else if (idle_thread_count min_spare_threads) {
/* terminate the free list */
if (free_length == 0) {
I considered wedging another thread status into the
scoreboard, between SERVER_DEAD (the initial value) and
SERVER_STARTING. The make_child() would set all the thread
slots to this value and start_threads() would later flip them
to SERVER_STARTING after actually creating the worker threads.
That would have various ripple effects on other bits of
httpd, though, like mod_status and other MPMs, etc. So instead
I tried adding a status field to the process_score scoreboard
structure, and making the following changes to worker.c such that
this field is set by make_child to SERVER_STARTING and then
changed to SERVER_READY once the start thread that runs
start_threads() has done its initial work.
During this period, while the new child process is running
ap_run_child_init() and friends, perform_idle_server_maintenance()
just counts that child process's worker threads as all being
effectively in SERVER_STARTING mode. Once the process_score.status
field changes to SERVER_READY, perform_idle_server_maintenance()
begins to look at the individual thread status values.
Any thoughts? The patch in Bugzilla doesn't address other
MPMs that might see the same behaviour (event, and maybe prefork?)
http://issues.apache.org/bugzilla/show_bug.cgi?id=39275
It also doesn't necessarily play ideally well with the fact that
new child processes can gradually take over thread slots in
the scoreboard from a gracefully exiting old process -- the
count of idle threads for that process will be pegged (only
by perform_idle_server_maintenance()) at ap_threads_per_child
until the new process creates its first new worker thread.
But, that may be just fine I'll keep poking around and
testing and maybe a better idea will present itself.
Chris.
Re: svn commit: r393037 - in /httpd/httpd/trunk: CHANGES server/protocol.c
On 04/11/2006 04:00 AM, Roy T. Fielding wrote: It probably needs to be updated for RFC 3986 anyway. The path should be set to , not NULL. The HTTP server should take care of the redirect from to /, which in this case means the http-proxy needs to check for when it sends a request and respond with a redirect that adds the /. In general I agree, but I think the transformation from to / must happen very early to avoid disturbing the cache code in the quick handler. So I guess this could be only done in the post_read_request hook of the proxy, but I don't think that we can trigger a redirect from this hook. Provided my thoughts are not wrong, anybody an idea how to solve this? Regards Rüdiger
Re: svn commit: r393037 - in /httpd/httpd/trunk: CHANGES server/protocol.c
Ruediger Pluem wrote: On 04/11/2006 04:00 AM, Roy T. Fielding wrote: It probably needs to be updated for RFC 3986 anyway. The path should be set to , not NULL. The HTTP server should take care of the redirect from to /, which in this case means the http-proxy needs to check for when it sends a request and respond with a redirect that adds the /. In general I agree, but I think the transformation from to / must happen very early to avoid disturbing the cache code in the quick handler. So I guess this could be only done in the post_read_request hook of the proxy, but I don't think that we can trigger a redirect from this hook. Provided my thoughts are not wrong, anybody an idea how to solve this? If I read this correctly, http://foo.example.com is wrong, whereas http://foo.example.com/ is correct. Does it merit an external redirect, much as http://foo.example.com/dirname would externally redirect the user to http://foo.example.com/dirname/ Do I understand this correctly? If so, isn't this entire conversation focusing on the wrong section of code?
Re: svn commit: r393037 - in /httpd/httpd/trunk: CHANGES server/protocol.c
On Tuesday 11 April 2006 22:10, William A. Rowe, Jr. wrote: Ruediger Pluem wrote: On 04/11/2006 04:00 AM, Roy T. Fielding wrote: It probably needs to be updated for RFC 3986 anyway. The path should be set to , not NULL. The HTTP server should take care of the redirect from to /, which in this case means the http-proxy needs to check for when it sends a request and respond with a redirect that adds the /. Um, it's not really a redirect; it's just a normalisation. Shouldn't really invoke any redirect logic, whether internal or external. In general I agree, but I think the transformation from to / must happen very early to avoid disturbing the cache code in the quick handler. So I guess this could be only done in the post_read_request hook of the proxy, but I don't think that we can trigger a redirect from this hook. Provided my thoughts are not wrong, anybody an idea how to solve this? Just adjust the URL itself in that hook? If I read this correctly, http://foo.example.com is wrong, whereas http://foo.example.com/ is correct. No. Both forms are correct, and equivalent (by definition). Does it merit an external redirect, much as http://foo.example.com/dirname would externally redirect the user to http://foo.example.com/dirname/ That's different, because the two are of course distinct in URL-space. -- Nick Kew
Re: svn commit: r393037 - in /httpd/httpd/trunk: CHANGES server/protocol.c
On Apr 11, 2006, at 2:55 PM, Nick Kew wrote: On Tuesday 11 April 2006 22:10, William A. Rowe, Jr. wrote: Ruediger Pluem wrote: On 04/11/2006 04:00 AM, Roy T. Fielding wrote: It probably needs to be updated for RFC 3986 anyway. The path should be set to , not NULL. The HTTP server should take care of the redirect from to /, which in this case means the http-proxy needs to check for when it sends a request and respond with a redirect that adds the /. Um, it's not really a redirect; it's just a normalisation. Shouldn't really invoke any redirect logic, whether internal or external. The server should redirect any time the characters in the request URI are changed, since that impacts the digests used in various access control mechanisms. Roy
