Re: mod_fcgid Release

[Jan Ehrhardt]

Mario Brandt in gmane.comp.apache.devel (Tue, 15 Dec 2020 14:14:43
+0100):
>Ping
>
>On Thu, 24 Jan 2019 at 11:23, Mario Brandt  wrote:
>>
>> Hi,
>> it has beens a while since there was the last release of mod_fcgid.
>> There are some important fixes in trunk, but never made it to a
>> release since 2013.
>> Is anyone willing to file a release?

+1
-- 
Jan

Re: mod_fcgid Release

[Jeff McKenna]

Seconded.

-jeff



On 2020-12-15 9:14 a.m., Mario Brandt wrote:

Ping

On Thu, 24 Jan 2019 at 11:23, Mario Brandt  wrote:


Hi,
it has beens a while since there was the last release of mod_fcgid.
There are some important fixes in trunk, but never made it to a
release since 2013.
Is anyone willing to file a release?

Cheers
Mario

Re: mod_fcgid Release

[Mario Brandt]

Ping

On Thu, 24 Jan 2019 at 11:23, Mario Brandt  wrote:
>
> Hi,
> it has beens a while since there was the last release of mod_fcgid.
> There are some important fixes in trunk, but never made it to a
> release since 2013.
> Is anyone willing to file a release?
>
> Cheers
> Mario

mod_fcgid Release

[Mario Brandt]

Hi,
it has beens a while since there was the last release of mod_fcgid.
There are some important fixes in trunk, but never made it to a
release since 2013.
Is anyone willing to file a release?

Cheers
Mario

Re: mod_fcgid: Immediate HTTP error 503 if the max total process count is reached

[Ivan Zahariev]

Hi,

Thanks for the review. You can find my comments below in the reply.


On 20.9.2016 г. 16:33 ч., Eric Covener wrote:

Unfortunately there are not many reviewers for mod_fcgid.
I tried to take as a relative laymen and had a few comments/questions:

* a few C99 // comments were added and should be zapped

Fixed.


* the assert looks funny -- no assert.   maybe ap_debug_assert() so it
blows up in maintainer builds only?
It's a paranoid check. If it ever blows the whistle, this will be in 
some very hard to reproduce case, and I don't expect that we catch this 
on a maintainer build. I made this an APLOG_CRIT message because a 
mismatch in the counts between proctable_count_all_nonfree_nodes() and 
"g_total_process" could make your FastCGI requests fail erroneously with 
an immediate 503 HTTP error. A System Administrator who could encounter 
this bug will appreciate the immediate APLOG_CRIT message in the 
"error_log".


I don't insist this to remain like this and I will change it as you say, 
in order to comply with the usual development practices. Or we can give 
this function check a better name.



* The new flag is merged, but cannot be unset?
** Probably need to make it look like an AP_INIT_FLAG

Right. I've converted this to an AP_INIT_FLAG which makes more sense.

I don't understand what you mean by "it cannot be unset". I'm merging it 
in the same way as other mod_fcgid config options. Please read below for 
more information regarding this matter.



* I couldn't follow the concept in the comments around
max_process_count moving or being per-request. Is it maybe out of
date? I think the current impl takes an existing global only and
merges it with vhost server confs but it will always be a copy.


The original implementation allows you to set "max_process_count" in the 
GLOBAL_ONLY context. This is enforced by ap_check_cmd_context() in the 
AP_INIT_TAKE1() handler, because RSRC_CONF would otherwise allow us to 
define this config option also "per vhost".


I've added the new variable "max_process_used_no_wait_enable" in the 
same way.


The original implementation works well without further magic, because it 
really uses "max_process_count" in code which works with the 
"main_server" (global) context. Now I need to access the value of 
"max_process_count" inside a "request_rec" (per VirtualHost) variable. 
If one or more mod_fcgid directives are defined not only within the main 
server config, but also as additional settings in a VirtualHost context, 
then we need to merge "max_process_count", too. If we don't merge, the 
value of "max_process_count" returned by the "request_rec" variable will 
be the default init value (zero, in our case).


Long story short -- you are right, this is a global only variable and it 
merges with vhost confs always as a copy of the global value. This is 
just a behind-the-scenes action and doesn't affect Apache 
administrators, since they can't define this variable in a vhost context 
anyway. I tried to summarize this as the comment ["global only" but we 
use its main_server value in a "request_rec"] but maybe someone can 
propose a better text.


I've committed the source code changes and you can review them again. 
When we're satisfied with the changes, I'll test the final version on 
our server fleet.


Best regards.
--Ivan

Re: mod_fcgid: Immediate HTTP error 503 if the max total process count is reached

[Eric Covener]

On Mon, Sep 19, 2016 at 4:30 AM, Ivan Zahariev <fam...@famzah.net> wrote:
> Hello devs,
>
> It's been four months since I originally proposed this new feature in
> "mod_fcgid". During this time I've tested and deployed it on hundreds of
> busy production servers. It works as expected. If enabled, web visitors get
> an immediate response when FastCGI is overloaded, and no RAM is
> over-utilized.
>
> You can find all the information and a patch in the enhancement request at
> Bugzilla: https://bz.apache.org/bugzilla/show_bug.cgi?id=59656
>
> Can we get this merged into "mod_fcgid"?
>
> Best regards.
> --Ivan

Unfortunately there are not many reviewers for mod_fcgid.

I tried to take as a relative laymen and had a few comments/questions:

* a few C99 // comments were added and should be zapped
* the assert looks funny -- no assert.   maybe ap_debug_assert() so it
blows up in maintainer builds only?
* The new flag is merged, but cannot be unset?
** Probably need to make it look like an AP_INIT_FLAG
* I couldn't follow the concept in the comments around
max_process_count moving or being per-request. Is it maybe out of
date? I think the current impl takes an existing global only and
merges it with vhost server confs but it will always be a copy.

Re: mod_fcgid: Immediate HTTP error 503 if the max total process count is reached

[Ivan Zahariev]

Hello devs,

It's been four months since I originally proposed this new feature in 
"mod_fcgid". During this time I've tested and deployed it on hundreds of 
busy production servers. It works as expected. If enabled, web visitors 
get an immediate response when FastCGI is overloaded, and no RAM is 
over-utilized.


You can find all the information and a patch in the enhancement request 
at Bugzilla: https://bz.apache.org/bugzilla/show_bug.cgi?id=59656


Can we get this merged into "mod_fcgid"?

Best regards.
--Ivan


On 2.6.2016 г. 11:57 ч., Ivan Zahariev wrote:

Hi Nick,

Thanks for the info.

I've followed your instructions and submitted an enhancement request: 
https://bz.apache.org/bugzilla/show_bug.cgi?id=59656


Cheers.
--Ivan


On 31.5.2016 г. 13:45 ч., Nick Kew wrote:

On Tue, 2016-05-31 at 11:15 +0300, Ivan Zahariev wrote:

Hello,

I got no feedback. Am I posting this suggestion at the right mailing
list?

Sorry, I see your original post marked for attention in my mail
folder, but languishing hitherto unattended.  Just now opened your
link in a browser to take a look.  There could be others who
have done something similar.

As a general reply to this question, yes, this is the best
available mailinglist.  The other place to post it would be
as an enhancement request in bugzilla (issues.apache.org).
The keyword "PatchAvailable" there may help by marking it as
low-hanging fruit.

Re: mod_fcgid: Immediate HTTP error 503 if the max total process count is reached

[Ivan Zahariev]

Hi Nick,

Thanks for the info.

I've followed your instructions and submitted an enhancement request: 
https://bz.apache.org/bugzilla/show_bug.cgi?id=59656


Cheers.
--Ivan


On 31.5.2016 г. 13:45 ч., Nick Kew wrote:

On Tue, 2016-05-31 at 11:15 +0300, Ivan Zahariev wrote:

Hello,

I got no feedback. Am I posting this suggestion at the right mailing
list?

Sorry, I see your original post marked for attention in my mail
folder, but languishing hitherto unattended.  Just now opened your
link in a browser to take a look.  There could be others who
have done something similar.

As a general reply to this question, yes, this is the best
available mailinglist.  The other place to post it would be
as an enhancement request in bugzilla (issues.apache.org).
The keyword "PatchAvailable" there may help by marking it as
low-hanging fruit.

Re: mod_fcgid: Immediate HTTP error 503 if the max total process count is reached

[Nick Kew]

On Tue, 2016-05-31 at 11:15 +0300, Ivan Zahariev wrote:
> Hello,
> 
> I got no feedback. Am I posting this suggestion at the right mailing
> list?

Sorry, I see your original post marked for attention in my mail
folder, but languishing hitherto unattended.  Just now opened your
link in a browser to take a look.  There could be others who
have done something similar.

As a general reply to this question, yes, this is the best
available mailinglist.  The other place to post it would be
as an enhancement request in bugzilla (issues.apache.org).
The keyword "PatchAvailable" there may help by marking it as
low-hanging fruit.

-- 
Nick Kew

Re: mod_fcgid: Immediate HTTP error 503 if the max total process count is reached

[Ivan Zahariev]

Hello,

I got no feedback. Am I posting this suggestion at the right mailing list?

Best regards.
--Ivan

On 19.5.2016 г. 10:40 ч., Ivan Zahariev wrote:

Hi all,

I'd like to propose a new configuration setting for "mod_fcgid". The 
source code changes to review follow:


  * The whole patch compared to version 2.3.9:

https://github.com/famzah/mod_fcgid/compare/2.3.9...maxnowait?diff=split=maxnowait
  * The whole patch as a single file:

https://github.com/famzah/mod_fcgid/compare/2.3.9...maxnowait.patch?diff=unified=maxnowait
  * Every single commit compared to version 2.3.9:
https://github.com/famzah/mod_fcgid/commits/maxnowait
  * There should be no merge conflicts with the current "trunk"
version 2.3.10.


The motivation is that the current behavior to queue up new pending 
requests differs from the RLimitNPROC directive behavior. When there 
is a spike in the web hits, lots of Apache children get busy just 
waiting for up to 30+ seconds to get an idle FastCGI process. Thus we 
"waste" Apache children doing nothing while they could serve static 
content. This also puts unneeded memory pressure. Additionally, users 
today won't wait for a page to load, if it takes more than a few 
seconds, and we'd rather notify them that we are currently overloaded 
by sending them a 503 HTTP response immediately.


Here is the documentation for the new directive: 
http://www.famzah.net/temp/FcgidMaxProcessesUsedNoWait.docs.txt


Let me know what you think.

Best regards.
--Ivan

mod_fcgid: Immediate HTTP error 503 if the max total process count is reached

[Ivan Zahariev]

  
  
Hi all,

I'd like to propose a new configuration setting for "mod_fcgid". The
source code changes to review follow:

  The whole patch compared to version 2.3.9:
https://github.com/famzah/mod_fcgid/compare/2.3.9...maxnowait?diff=split=maxnowait
  The whole patch as a single file:
https://github.com/famzah/mod_fcgid/compare/2.3.9...maxnowait.patch?diff=unified=maxnowait
  Every single commit compared to version 2.3.9:
https://github.com/famzah/mod_fcgid/commits/maxnowait
  There should be no merge conflicts with the current "trunk"
version 2.3.10.


The motivation is that the current behavior to queue up new pending
requests differs from the RLimitNPROC directive behavior. When there
is a spike in the web hits, lots of Apache children get busy just
waiting for up to 30+ seconds to get an idle FastCGI process. Thus
we "waste" Apache children doing nothing while they could serve
static content. This also puts unneeded memory pressure.
Additionally, users today won't wait for a page to load, if it takes
more than a few seconds, and we'd rather notify them that we are
currently overloaded by sending them a 503 HTTP response
immediately.

Here is the documentation for the new directive:
http://www.famzah.net/temp/FcgidMaxProcessesUsedNoWait.docs.txt

Let me know what you think.

Best regards.
--Ivan
  

Re: mod_fcgid and broken doc links

[Jim Jagielski]

There is a need and an audience for both. We provide
both. I see no reason for us to stop doing that unless
we feel that we can no longer support both in the
manner in which our community expects. FWIW, I don't
see that as the current situation.

> On Jan 18, 2016, at 9:29 PM, William A Rowe Jr  wrote:
> 
> On Mon, Jan 18, 2016 at 3:29 PM, Jim Jagielski  wrote:
> 
> > On Jan 18, 2016, at 3:28 PM, William A Rowe Jr  wrote:
> >
> > On Fri, Jan 15, 2016 at 7:44 AM, Jim Jagielski  wrote:
> >
> > > On Jan 14, 2016, at 5:19 PM, William A Rowe Jr  
> > > wrote:
> > >
> > > Good point with your example, this is something that should
> > > be benchmarked and the winner-take-all, loser bumped from the
> > > trunk/ copy of httpd.
> >
> > -1
> >
> > You are implying that one would be a winner in all cases. The
> > whole idea is that there are cases where one is better than
> > the other. We provide both.
> >
> > You might have made that inference, but I'm going to assert that
> > for this one module, for s/{literal}/{repl}, mod_sed is going to
> > outperform mod_substitute /if/ we wrote the code correctly.
> 
> I disagree... it's kind of obvious by simple inspection that
> mod_substitute has fast paths that mod_sed lacks and, as thus,
> can be quite performant and the "better" choice in numerous cases
> where that fast path is used.
> 
> Well, only benchmarking is going to prove that one way or the other,
> something I don't have free cycles for right now (committed to way too
> many other project priorities.)  And it perhaps opens up opportunities
> to optimize mod_sed in ways that might have been initially overlooked
> when the code was thrown into trunk ;-)
>  
> Me, I don't tend to think of myself as "smarter" than all of
> our users, nor do I try to act as Big Brother and remove choices
> from people in cases and situations where they are using them.
> The ASF itself doesn't do that, nor do projects... so it seems
> kinda weird that you would want the httpd project to all of
> a sudden start removing choice and options for end users instead
> of helping them out and trusting them to know which impl is
> best for them.
> 
> Thankfully, we aren't, but our users do look at us as experts in the
> code they consume, considering that we collectively have authored
> and maintain the stuff.  So they do look to us to make the best
> choices they don't have the individual time to compare and elect.
> Why would I want us to collectively (and for me specifically) to
> propose the best solutions to the common use cases, and to
> depreciate less maintained code in favor of maintained code?  
> 
> Can't imagine why I or other project members would be possessed 
> to do that.  If you figure it out, please share :)

Re: mod_fcgid and broken doc links

[William A Rowe Jr]

On Mon, Jan 18, 2016 at 3:29 PM, Jim Jagielski  wrote:

>
> > On Jan 18, 2016, at 3:28 PM, William A Rowe Jr 
> wrote:
> >
> > On Fri, Jan 15, 2016 at 7:44 AM, Jim Jagielski  wrote:
> >
> > > On Jan 14, 2016, at 5:19 PM, William A Rowe Jr 
> wrote:
> > >
> > > Good point with your example, this is something that should
> > > be benchmarked and the winner-take-all, loser bumped from the
> > > trunk/ copy of httpd.
> >
> > -1
> >
> > You are implying that one would be a winner in all cases. The
> > whole idea is that there are cases where one is better than
> > the other. We provide both.
> >
> > You might have made that inference, but I'm going to assert that
> > for this one module, for s/{literal}/{repl}, mod_sed is going to
> > outperform mod_substitute /if/ we wrote the code correctly.
>
> I disagree... it's kind of obvious by simple inspection that
> mod_substitute has fast paths that mod_sed lacks and, as thus,
> can be quite performant and the "better" choice in numerous cases
> where that fast path is used.
>

Well, only benchmarking is going to prove that one way or the other,
something I don't have free cycles for right now (committed to way too
many other project priorities.)  And it perhaps opens up opportunities
to optimize mod_sed in ways that might have been initially overlooked
when the code was thrown into trunk ;-)


> Me, I don't tend to think of myself as "smarter" than all of
> our users, nor do I try to act as Big Brother and remove choices
> from people in cases and situations where they are using them.
> The ASF itself doesn't do that, nor do projects... so it seems
> kinda weird that you would want the httpd project to all of
> a sudden start removing choice and options for end users instead
> of helping them out and trusting them to know which impl is
> best for them.
>

Thankfully, we aren't, but our users do look at us as experts in the
code they consume, considering that we collectively have authored
and maintain the stuff.  So they do look to us to make the best
choices they don't have the individual time to compare and elect.
Why would I want us to collectively (and for me specifically) to
propose the best solutions to the common use cases, and to
depreciate less maintained code in favor of maintained code?

Can't imagine why I or other project members would be possessed
to do that.  If you figure it out, please share :)

Re: mod_fcgid and broken doc links

[William A Rowe Jr]

On Fri, Jan 15, 2016 at 7:44 AM, Jim Jagielski <j...@jagunet.com> wrote:

>
> > On Jan 14, 2016, at 5:19 PM, William A Rowe Jr <wr...@rowe-clan.net>
> wrote:
> >
> > Good point with your example, this is something that should
> > be benchmarked and the winner-take-all, loser bumped from the
> > trunk/ copy of httpd.
>
> -1
>
> You are implying that one would be a winner in all cases. The
> whole idea is that there are cases where one is better than
> the other. We provide both.
>

You might have made that inference, but I'm going to assert that
for this one module, for s/{literal}/{repl}, mod_sed is going to
outperform mod_substitute /if/ we wrote the code correctly.  Whas
is missing is a mod_sed directive for adding literal s/{orig}/{repl}/
patterns that follows the substitute syntax and that admins don't
have to wrap their heads around the special character escaping.
At that point, there is no reason for the redundant module, but
we aren't at that point.

Would you also recommend such a scenario with MPMs, and
> having the "winner" take all? I certainly hope not!!!
>

No, I generally agree with TMTOWTDI.  Glad you raised the MPMs
issue, because my exact complaint about folding in mod_fcgid
"as is" and overlapping with mod_proxy_fcgi was addressed by the
MPM community in pulling out and sharing the mpm_common.c
logic. Great example, thanks Jim!

Re: mod_fcgid and broken doc links

[Jim Jagielski]

> On Jan 18, 2016, at 3:28 PM, William A Rowe Jr  wrote:
> 
> On Fri, Jan 15, 2016 at 7:44 AM, Jim Jagielski  wrote:
> 
> > On Jan 14, 2016, at 5:19 PM, William A Rowe Jr  wrote:
> >
> > Good point with your example, this is something that should
> > be benchmarked and the winner-take-all, loser bumped from the
> > trunk/ copy of httpd.
> 
> -1
> 
> You are implying that one would be a winner in all cases. The
> whole idea is that there are cases where one is better than
> the other. We provide both.
> 
> You might have made that inference, but I'm going to assert that
> for this one module, for s/{literal}/{repl}, mod_sed is going to 
> outperform mod_substitute /if/ we wrote the code correctly.

I disagree... it's kind of obvious by simple inspection that
mod_substitute has fast paths that mod_sed lacks and, as thus,
can be quite performant and the "better" choice in numerous cases
where that fast path is used.

Me, I don't tend to think of myself as "smarter" than all of
our users, nor do I try to act as Big Brother and remove choices
from people in cases and situations where they are using them.
The ASF itself doesn't do that, nor do projects... so it seems
kinda weird that you would want the httpd project to all of
a sudden start removing choice and options for end users instead
of helping them out and trusting them to know which impl is
best for them.

Re: mod_fcgid and broken doc links

[Jim Jagielski]

> On Jan 14, 2016, at 5:19 PM, William A Rowe Jr  wrote:
> 
> 
> Good point with your example, this is something that should 
> be benchmarked and the winner-take-all, loser bumped from the
> trunk/ copy of httpd.

-1

You are implying that one would be a winner in all cases. The
whole idea is that there are cases where one is better than
the other. We provide both.

Would you also recommend such a scenario with MPMs, and
having the "winner" take all? I certainly hope not!!!

Re: mod_fcgid and broken doc links

[Jim Jagielski]

> On Jan 13, 2016, at 12:28 PM, William A Rowe Jr <wr...@rowe-clan.net> wrote:
> 
> 
> I can see us moving those modules into trunk (not 2.4), retaining the
> mmn tests for 2.2 and 2.4 compat, and then deriving an fcgid release
> out of trunk/modules/fcgid/.  But I'm not clear why we would want to
> maintain the duplication between mod_proxy_fcgi and mod_fcgid?
> Individually they get little enough attention as it is.
> 

Because they are separate solutions to a similar problem,
ala mod_sed and mod_substitute for example. I can site several
more if that is worthwhile (but the point is made) mod_fcgid does
not require the whole mod_proxy overhead, for example, but
lacks some features that mod_proxy_fcgi does have, since the
latter can leverage some of that "overhead".

> Just my 2c USD, I'm open to ideas.

That is good to know.

Re: mod_fcgid and broken doc links

[William A Rowe Jr]

On Thu, Jan 14, 2016 at 6:19 AM, Jim Jagielski <j...@jagunet.com> wrote:

>
> > On Jan 13, 2016, at 12:28 PM, William A Rowe Jr <wr...@rowe-clan.net>
> wrote:
> >
> > I can see us moving those modules into trunk (not 2.4), retaining the
> > mmn tests for 2.2 and 2.4 compat, and then deriving an fcgid release
> > out of trunk/modules/fcgid/.  But I'm not clear why we would want to
> > maintain the duplication between mod_proxy_fcgi and mod_fcgid?
> > Individually they get little enough attention as it is.
>
> Because they are separate solutions to a similar problem,
> ala mod_sed and mod_substitute for example. I can site several
> more if that is worthwhile (but the point is made) mod_fcgid does
> not require the whole mod_proxy overhead, for example, but
> lacks some features that mod_proxy_fcgi does have, since the
> latter can leverage some of that "overhead".


Good point with your example, this is something that should
be benchmarked and the winner-take-all, loser bumped from the
trunk/ copy of httpd.  They are slightly different in that sed is one
specific grammar, while substitute uses literal replacements, but
that could be a syntax tweak to mod_sed to allow some literal
replacements.  In any case, it relieves the dozen or so regular
httpd contributors from maintaining two modules which do, for
the most part, the very same thing expressed with redundant
yet different code.

Likewise with fcgid/proxy_fcgi - /if/ we bring mod_fcgid sources
into the trunk, I'd add the caveat that we do so if there is an offer
to eliminate all of the code duplication in the process, or reject
this proposition.  They do perform two different purposes and
overlap in interesting ways, so there has to be something gained
by pushing fcgid upon the entire httpd committer community
to maintain.  It was one thing for the handful of us who commit
to mod_fcgid to deal with that legacy, but another to push it
mainstream.


> > Just my 2c USD, I'm open to ideas.
>
> That is good to know.


Was simply offering backstory for why things ended up as they
have, sans snark, and no snark needed in response :)  Would
love to learn who is interested in helping merge mod_fcgid and
mod_proxy_ftp functionality into a util_fcgi, or otherwise condense
this functionality into something manageable, long term.

Re: mod_fcgid and broken doc links

[Jim Jagielski]

Does it make sense to "officially" bundle mod_fcgid w/ httpd?

> On Jan 12, 2016, at 1:13 PM, Rich Bowen <rbo...@rcbowen.com> wrote:
> 
> mod_fcgid is in a separate repo from the main httpd tree, due to
> historical reasons. I presume there are good reasons for this. JimJag
> suggested on IRC it's due to its independent release cycle.
> 
> Be that as it may, because it uses the standard documentation tools for
> the module docs, https://httpd.apache.org/mod_fcgid/mod/mod_fcgid.html
> is full of broken links. In particular, try any of the directive and
> links to other modules - try the mod_cgi or AddHandler links in the
> intro paragraph, and you'll see immediately what the problem is.
> 
> Now, we could of course have a separate version of the docs building
> tools just for this module, or we could patch the doc manually, but I
> was wondering, if there's no strong current reason for the module to be
> kept separate, can we please move it into the main httpd tree?
> 
> (Note that exactly the same situation applies to mod_ftp, but there's
> just fewer links from that doc so we don't hear it as often.)
> 
> --Rich
> 
> -- 
> Rich Bowen - rbo...@rcbowen.com - @rbowen
> http://apachecon.com/ - @apachecon

Re: mod_fcgid and broken doc links

[Jan Ehrhardt]

Jim Jagielski in gmane.comp.apache.devel (Wed, 13 Jan 2016 07:33:43
-0500):
>Does it make sense to "officially" bundle mod_fcgid w/ httpd?

FWIW: I always compile mod_fcgid.so together with Apache httpd. I have
made it part of my VC 9/11/14 solution files. I guess that many Windows
users of httpd install mod_fcgid, so for them it makes sense.

Jan

Re: mod_fcgid and broken doc links

[Rich Bowen]

On 01/13/2016 12:28 PM, William A Rowe Jr wrote:

> The reason for mod_ftp and mod_fcgid separate builds was historically
> that the same module, releasing on a different calendar than httpd, have
> been build-able independently against 2.0, 2.2 or 2.4.  Maintaining the 
> sources across the different branches was also something of a PITA.
> 
> Maybe more significantly, mod_proxy_fcgi was our 'adopted' solution 
> to the fcgi case.  mod_fcgid is a different beast with process pool 
> management. I was always under the impression that for 2.4 and later, 
> we collectively wanted to express process pools independently of the
> mod_proxy_fcgi structure, much like we and tomcat would love to see
> folks use mod_proxy_http or _ajp rather than mod_jk.
> 
> mod_ftp clearly isn't http:// so it never quite felt appropriate in that
> tree, but then again neither is mod_proxy_ajp :)  Which goes to the
> gist of it all, code bloat.  We've successfully only killed a tiny handful
> of modules in our entire history (imagemap, mem_cache etc). 

mod_imagemap still lives on in trunk. As does mod_cern_meta.

Point taken.

> So once merge to trunk, we own that code bloat for a very long time,
> but if it exists separate these can be enhanced or retired based on
> our desires.  E.g. if mod_aspdotnet had lived in modules/os/win32/
> we would still probably be shipping it, irrespective of how out of date
> that module becomes.
> 
> I can see us moving those modules into trunk (not 2.4), retaining the
> mmn tests for 2.2 and 2.4 compat, and then deriving an fcgid release
> out of trunk/modules/fcgid/.  But I'm not clear why we would want to
> maintain the duplication between mod_proxy_fcgi and mod_fcgid?
> Individually they get little enough attention as it is.

Yes, it would be nice to merge them, from the perspective of explaining
things to users.

-- 
Rich Bowen - rbo...@rcbowen.com - @rbowen
http://apachecon.com/ - @apachecon

Re: mod_fcgid and broken doc links

[William A Rowe Jr]

On Jan 13, 2016 15:50, "Rich Bowen"  wrote:
>
> Yes, it would be nice to merge them, from the perspective of explaining
> things to users.

Guess I am still confused what you suggest to merge... Docs or both docs
and code...

Also curious about released vs unreleased with respect to docs.  Our
pointers and resources for developers mostly live under
http://httpd.a.o/dev/ per ASF policy, but docs is an exception.  We are
supposed to be pointing users at the released artifacts, and not at
unreleased drafts.

Even http://httpd.apache.org/docs/dev/  might be clearer than trunk/ for
those who stumble upon them.

Thoughts?

Re: mod_fcgid and broken doc links

[Ruediger Pluem]

On 01/13/2016 01:33 PM, Jim Jagielski wrote:
> Does it make sense to "officially" bundle mod_fcgid w/ httpd?

Just for fixing the documention?
In this case I would prefer to investigate other solutions for the 
documentation and keep it separate.

Regards

Rüdiger

Re: mod_fcgid and broken doc links

[Mike Rumph]

A background for this request can be seen in bug report 56121.
- https://bz.apache.org/bugzilla/show_bug.cgi?id=56121#c4
This bug also describes a manual method for working around this problem.

On 1/12/2016 10:13 AM, Rich Bowen wrote:

mod_fcgid is in a separate repo from the main httpd tree, due to
historical reasons. I presume there are good reasons for this. JimJag
suggested on IRC it's due to its independent release cycle.

Be that as it may, because it uses the standard documentation tools for
the module docs, https://httpd.apache.org/mod_fcgid/mod/mod_fcgid.html
is full of broken links. In particular, try any of the directive and
links to other modules - try the mod_cgi or AddHandler links in the
intro paragraph, and you'll see immediately what the problem is.

Now, we could of course have a separate version of the docs building
tools just for this module, or we could patch the doc manually, but I
was wondering, if there's no strong current reason for the module to be
kept separate, can we please move it into the main httpd tree?

(Note that exactly the same situation applies to mod_ftp, but there's
just fewer links from that doc so we don't hear it as often.)

--Rich

Re: mod_fcgid and broken doc links

[Jim Jagielski]

Not just for that, but to make it easier for people
to use it... Seems stable enough that making it a
bundled module makes sense.

> On Jan 13, 2016, at 9:43 AM, Ruediger Pluem <rpl...@apache.org> wrote:
> 
> 
> 
> On 01/13/2016 01:33 PM, Jim Jagielski wrote:
>> Does it make sense to "officially" bundle mod_fcgid w/ httpd?
> 
> Just for fixing the documention?
> In this case I would prefer to investigate other solutions for the 
> documentation and keep it separate.
> 
> Regards
> 
> Rüdiger
> 

Re: mod_fcgid and broken doc links

[William A Rowe Jr]

On Tue, Jan 12, 2016 at 12:13 PM, Rich Bowen <rbo...@rcbowen.com> wrote:

> mod_fcgid is in a separate repo from the main httpd tree, due to
> historical reasons. I presume there are good reasons for this. JimJag
> suggested on IRC it's due to its independent release cycle.
>
> Be that as it may, because it uses the standard documentation tools for
> the module docs, https://httpd.apache.org/mod_fcgid/mod/mod_fcgid.html
> is full of broken links. In particular, try any of the directive and
> links to other modules - try the mod_cgi or AddHandler links in the
> intro paragraph, and you'll see immediately what the problem is.
>
> Now, we could of course have a separate version of the docs building
> tools just for this module, or we could patch the doc manually, but I
> was wondering, if there's no strong current reason for the module to be
> kept separate, can we please move it into the main httpd tree?
>

The reason for mod_ftp and mod_fcgid separate builds was historically
that the same module, releasing on a different calendar than httpd, have
been build-able independently against 2.0, 2.2 or 2.4.  Maintaining the
sources across the different branches was also something of a PITA.

Maybe more significantly, mod_proxy_fcgi was our 'adopted' solution
to the fcgi case.  mod_fcgid is a different beast with process pool
management. I was always under the impression that for 2.4 and later,
we collectively wanted to express process pools independently of the
mod_proxy_fcgi structure, much like we and tomcat would love to see
folks use mod_proxy_http or _ajp rather than mod_jk.

mod_ftp clearly isn't http:// so it never quite felt appropriate in that
tree, but then again neither is mod_proxy_ajp :)  Which goes to the
gist of it all, code bloat.  We've successfully only killed a tiny handful
of modules in our entire history (imagemap, mem_cache etc).
So once merge to trunk, we own that code bloat for a very long time,
but if it exists separate these can be enhanced or retired based on
our desires.  E.g. if mod_aspdotnet had lived in modules/os/win32/
we would still probably be shipping it, irrespective of how out of date
that module becomes.

I can see us moving those modules into trunk (not 2.4), retaining the
mmn tests for 2.2 and 2.4 compat, and then deriving an fcgid release
out of trunk/modules/fcgid/.  But I'm not clear why we would want to
maintain the duplication between mod_proxy_fcgi and mod_fcgid?
Individually they get little enough attention as it is.

Just my 2c USD, I'm open to ideas.

mod_fcgid and broken doc links

[Rich Bowen]

mod_fcgid is in a separate repo from the main httpd tree, due to
historical reasons. I presume there are good reasons for this. JimJag
suggested on IRC it's due to its independent release cycle.

Be that as it may, because it uses the standard documentation tools for
the module docs, https://httpd.apache.org/mod_fcgid/mod/mod_fcgid.html
is full of broken links. In particular, try any of the directive and
links to other modules - try the mod_cgi or AddHandler links in the
intro paragraph, and you'll see immediately what the problem is.

Now, we could of course have a separate version of the docs building
tools just for this module, or we could patch the doc manually, but I
was wondering, if there's no strong current reason for the module to be
kept separate, can we please move it into the main httpd tree?

(Note that exactly the same situation applies to mod_ftp, but there's
just fewer links from that doc so we don't hear it as often.)

--Rich

-- 
Rich Bowen - rbo...@rcbowen.com - @rbowen
http://apachecon.com/ - @apachecon

Re: Next release of mod_fcgid

[Jeff Trawick]

On Mon, Jun 22, 2015 at 11:42 AM, Mario Brandt jbl...@gmail.com wrote:

 r1604123: Lower log graceful kill fail, sending SIGKILL level to
 DEBUG on Windows. PR 54597.
 and
 r16041237: Follow up to r1604123: make it compile.

 I've now listed the fix in CHANGES-FCGID.  IMO it isn't worth rolling a
release for this.  I'm happy to TR and help test a mod_fcgid release that
has more fixes.  IIRC, there's some low-hanging fruit in Bugzilla that
interested developers can help test and promote.

(Not to say that this fix isn't important to some people, but they can
change WARNING to DEBUG  in the right place with zero risk.)



 Cheers
 Mario

 On 22 June 2015 at 16:59, Jeff Trawick traw...@gmail.com wrote:
  On Mon, Jun 22, 2015 at 10:14 AM, Mario Brandt jbl...@gmail.com wrote:
 
  Hi,
 
  the last release of mod_fcgid was been a long while.
 
  There was an important bugfix on windows side in June last year.
 
  Is there anyone willing to tag and release a new version?
 
 
  There's nothing in CHANGES.  What was it?
 
  --
  Born in Roswell... married an alien...
  http://emptyhammock.com/
 




-- 
Born in Roswell... married an alien...
http://emptyhammock.com/

Re: Next release of mod_fcgid

[Mario Brandt]

r1604123: Lower log graceful kill fail, sending SIGKILL level to
DEBUG on Windows. PR 54597.
and
r16041237: Follow up to r1604123: make it compile.

Cheers
Mario

On 22 June 2015 at 16:59, Jeff Trawick traw...@gmail.com wrote:
 On Mon, Jun 22, 2015 at 10:14 AM, Mario Brandt jbl...@gmail.com wrote:

 Hi,

 the last release of mod_fcgid was been a long while.

 There was an important bugfix on windows side in June last year.

 Is there anyone willing to tag and release a new version?


 There's nothing in CHANGES.  What was it?

 --
 Born in Roswell... married an alien...
 http://emptyhammock.com/

Re: Next release of mod_fcgid

[Jeff Trawick]

On Mon, Jun 22, 2015 at 10:14 AM, Mario Brandt jbl...@gmail.com wrote:

 Hi,

 the last release of mod_fcgid was been a long while.

 There was an important bugfix on windows side in June last year.

 Is there anyone willing to tag and release a new version?


There's nothing in CHANGES.  What was it?

-- 
Born in Roswell... married an alien...
http://emptyhammock.com/

Next release of mod_fcgid

[Mario Brandt]

Hi,

the last release of mod_fcgid was been a long while.

There was an important bugfix on windows side in June last year.

Is there anyone willing to tag and release a new version?


Cheers
Mario

Re: mod_fcgid: WIN32 compile-error and problem with slepp for 1 second

[Jeff Trawick]

Hello Michael,

This is a mailing list for developers of plug-in modules for httpd.  
us...@httpd.apache.org and d...@httpd.apache.org are better choices.  (I 
guess these two topics are code-related so dev@ is the best choice.)  
But see below.  (If further discussion is needed outside of the bug db, 
please post to d...@httpd.apache.org.)



On 04/22/2015 02:26 AM, Koperek, Michael wrote:

Hi,
using the current mod_fcgid 2.3.9 sources, im not able to compile on windows 
using msdev 6.0:

Compiling...
fcgid_conf.c
…\fcgid_conf.c(815) : error C2065: 'JOBOBJECT_EXTENDED_LIMIT_INFORMATION' : 
undeclared identifier
…\fcgid_conf.c(815) : error C2146: syntax error : missing ';' before identifier 
'job_info'
…\fcgid_conf.c(815) : error C2065: 'job_info' : undeclared identifier
…\fcgid_conf.c(815) : error C2059: syntax error : '{'
…\fcgid_conf.c(827) : error C2224: left of '.BasicLimitInformation' must have 
struct/union type
…\fcgid_conf.c(828) : error C2065: 'JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE' : 
undeclared identifier
…\fcgid_conf.c(830) : error C2065: 'JobObjectExtendedLimitInformation' : 
undeclared identifier

Seems to be not compilable code. Any ideas what to do?


You need a newer SDK from Microsoft that defines that API.  (Maybe a web 
search will help?)  It would be fair to open a bug against mod_fcgid 
asking for some compatibility with ancient SDK bundled with your 
compiler, but I'm not sure anyone would be able to work on that.





There is a sleep for 1 second in fcgid_bridge.c, function handle_request.
One second is for my application to high (user has to wait for each image to 
load on a web side with multiple images 1 second).
I changed the code from
apr_sleep(apr_time_from_sec(1));
to
   apr_sleep(APR_USEC_PER_SEC / 50); // 20 msecs

That works fine.
Is it possible to configure the sleep time?


Please comment on this bug to raise awareness that it affects more people:

https://bz.apache.org/bugzilla/show_bug.cgi?id=56308



Kind regards

Michael Koperek
e.bootis ag




e.bootis ag
Aktiengesellschaft mit Sitz in Essen
Amtsgericht Essen HRB 21782
Vorstand: Dr. Karl Langenstein, Ludger Langenstein, Tim Langenstein
Aufsichtsratsvorsitzender: Prof. Dr. Martin Užik

Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. 
Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten 
haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. 
Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail ist nicht 
gestattet.
This e-mail may contain confidential and/or privileged information. If you are 
not the intended recipient (or have received this e-mail in error) please 
notify the sender immediately and destroy this e-mail. Any unauthorized 
copying, disclosure or distribution of the material in this e-mail is strictly 
forbidden.

mod_fcgid: WIN32 compile-error and problem with slepp for 1 second

[Koperek, Michael]

Hi,
using the current mod_fcgid 2.3.9 sources, im not able to compile on windows 
using msdev 6.0:

Compiling...
fcgid_conf.c
…\fcgid_conf.c(815) : error C2065: 'JOBOBJECT_EXTENDED_LIMIT_INFORMATION' : 
undeclared identifier
…\fcgid_conf.c(815) : error C2146: syntax error : missing ';' before identifier 
'job_info'
…\fcgid_conf.c(815) : error C2065: 'job_info' : undeclared identifier
…\fcgid_conf.c(815) : error C2059: syntax error : '{'
…\fcgid_conf.c(827) : error C2224: left of '.BasicLimitInformation' must have 
struct/union type
…\fcgid_conf.c(828) : error C2065: 'JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE' : 
undeclared identifier
…\fcgid_conf.c(830) : error C2065: 'JobObjectExtendedLimitInformation' : 
undeclared identifier

Seems to be not compilable code. Any ideas what to do?


There is a sleep for 1 second in fcgid_bridge.c, function handle_request.
One second is for my application to high (user has to wait for each image to 
load on a web side with multiple images 1 second).
I changed the code from
   apr_sleep(apr_time_from_sec(1));
to
  apr_sleep(APR_USEC_PER_SEC / 50); // 20 msecs

That works fine.
Is it possible to configure the sleep time?

Kind regards

Michael Koperek
e.bootis ag




e.bootis ag
Aktiengesellschaft mit Sitz in Essen
Amtsgericht Essen HRB 21782
Vorstand: Dr. Karl Langenstein, Ludger Langenstein, Tim Langenstein
Aufsichtsratsvorsitzender: Prof. Dr. Martin Užik

Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. 
Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten 
haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. 
Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail ist nicht 
gestattet.
This e-mail may contain confidential and/or privileged information. If you are 
not the intended recipient (or have received this e-mail in error) please 
notify the sender immediately and destroy this e-mail. Any unauthorized 
copying, disclosure or distribution of the material in this e-mail is strictly 
forbidden.

Re: mod_fcgid kill all subprocesses in reload

[Stefan Priebe]

Am 26.12.2014 um 17:49 schrieb Mario Brandt:

Hi,
there is a patch for busy processes
https://issues.apache.org/bugzilla/show_bug.cgi?id=48769 Maybe that is
a starting point.


Thanks, this helps a lot. Still wondering why it isn't upstream.

Stefan

Re: mod_fcgid kill all subprocesses in reload

[Mario Brandt]

Hi,
there is a patch for busy processes
https://issues.apache.org/bugzilla/show_bug.cgi?id=48769 Maybe that is
a starting point.

Cheers
Mario

On 26 December 2014 at 07:05, Graham Dumpleton grah...@apache.org wrote:
 Sounds like it would perhaps be for the same reason as mod_wsgi has issues
 with that sort of thing.

 Only Apache child worker processes get special dispensation as far as
 graceful shutdowns or reloads are concerned. If instead a module creates
 additional processes using the other child API calls in APR:

 http://apr.apache.org/docs/apr/1.4/group__apr__thread__proc.html#gaf8d2be452a819161aa4cd6205a17761e

 then when a shutdown or restart occurs they get a hard 3 seconds to shutdown
 or they will be killed with SIGKILL. Up to that 3 seconds they will be sent
 a normal SIGTERM each second to try and get them to exit.

 Thus there is no facility to allow them to linger longer that I know of
 unless something has been added in more recent times.

 Graham


 On 26 December 2014 at 16:57, Stefan Priebe - Profihost AG
 s.pri...@profihost.ag wrote:

 Hi list,

 i like mod fcgid a lot but there's one bug which makes me crazy.

 On DSO unload (Apache reload ) all child's get killed no matter if they
 process requests or not. This makes no sense to me httpd processes itself
 are also kept until all requests are served.

 Stefan

 Excuse my typo sent from my mobile phone.

mod_fcgid kill all subprocesses in reload

[Stefan Priebe - Profihost AG]

Hi list,

i like mod fcgid a lot but there's one bug which makes me crazy.

On DSO unload (Apache reload ) all child's get killed no matter if they process 
requests or not. This makes no sense to me httpd processes itself are also kept 
until all requests are served.

Stefan

Excuse my typo sent from my mobile phone.

Re: mod_fcgid kill all subprocesses in reload

[Graham Dumpleton]

Sounds like it would perhaps be for the same reason as mod_wsgi has issues
with that sort of thing.

Only Apache child worker processes get special dispensation as far as
graceful shutdowns or reloads are concerned. If instead a module creates
additional processes using the other child API calls in APR:

http://apr.apache.org/docs/apr/1.4/group__apr__thread__proc.html#gaf8d2be452a819161aa4cd6205a17761e

then when a shutdown or restart occurs they get a hard 3 seconds to
shutdown or they will be killed with SIGKILL. Up to that 3 seconds they
will be sent a normal SIGTERM each second to try and get them to exit.

Thus there is no facility to allow them to linger longer that I know of
unless something has been added in more recent times.

Graham


On 26 December 2014 at 16:57, Stefan Priebe - Profihost AG 
s.pri...@profihost.ag wrote:

 Hi list,

 i like mod fcgid a lot but there's one bug which makes me crazy.

 On DSO unload (Apache reload ) all child's get killed no matter if they
 process requests or not. This makes no sense to me httpd processes itself
 are also kept until all requests are served.

 Stefan

 Excuse my typo sent from my mobile phone.

mod_fcgid and streaming large request bodies

[Justin Erenkrantz]

Hi all!

I'm currently using FastCGI for an application - in particular, Ceph's
radosgw (S3 endpoint).  I was told to use mod_fastcgi as mod_fcgid doesn't
handle large request bodies appropriately.  Yet, when I looked at the
mod_fastcgi code, I shrieked in horror.  =)

In looking at add_request_body in fcgid_bridge.c, it looks like we read the
request body into memory and if it crosses a threshold, we spool it to
disk.  After everything is read from the client, then mod_fcgid begins to
write to the FastCGI socket.  In my use case, we would be PUT'ing large
request bodies that are many GB in size - so, I'd rather just stream it to
the FastCGI socket and take the hit on potentially blocking other incoming
requests.  (In our case, it makes no sense to spool the request body to
local disk.)

Anyway, I'm diving into the code a bit - but, I figured it might be useful
to see if anyone else has any thoughts about how to handle large request
bodies with mod_fcgid.

Happy holidays!  -- justin

Re: mod_fcgid and streaming large request bodies

[Justin Erenkrantz]

On Fri, Dec 27, 2013 at 8:49 AM, Justin Erenkrantz
jus...@erenkrantz.com wrote:
 Anyway, I'm diving into the code a bit - but, I figured it might be useful
 to see if anyone else has any thoughts about how to handle large request
 bodies with mod_fcgid.

Here's a first-cut patch that compiles at least.  Cheers.  -- justin

Add FcgidStreamRequestBody directive to send request body as it arrives without
storing it in memory or on disk.

* modules/fcgid/mod_fcgid.c
  (fcgid_cmds): Add FcgidStreamRequestBody directive.
* modules/fcgid/fcgid_conf.h
  (fcgid_server_conf): Add stream_request_body/stream_request_body_set flags.
* modules/fcgid/fcgid_conf.c
  (DEFAULT_STREAM_REQUEST_BODY): Set to 0.
  (create_fcgid_server_config): Init stream_request_body flag.
  (merge_fcgid_server_config): Merge new stream_request_body flag.
  (set_stream_request_body): New config helper.
* modules/fcgid/fcgid_bridge.c
  (add_request_body): Add a forward declaration to reduce diff (for now);
  take fcgid_bucket_ctx param; if stream_request_body is set, don't copy
  bucket and then call proc_write_ipc to send out that brigade and clear it
  out before next loop iteration.
  (bridge_request): Delay reading the request body until later.
  (handle_request_ipc): Move add_request_body call to here and append the EOS
  bucket before we do the final write of the request body.

Index: modules/fcgid/fcgid_bridge.c
===
--- modules/fcgid/fcgid_bridge.c (revision 1553671)
+++ modules/fcgid/fcgid_bridge.c (working copy)
@@ -287,6 +287,10 @@ static int getsfunc_fcgid_BRIGADE(char *buf, int l
 return done;
 }

+static int add_request_body(request_rec *r, apr_pool_t *request_pool,
+apr_bucket_brigade *output_brigade,
+fcgid_bucket_ctx *bucket_ctx);
+
 static int
 handle_request_ipc(request_rec *r, int role,
apr_bucket_brigade *output_brigade,
@@ -295,9 +299,21 @@ handle_request_ipc(request_rec *r, int role,
 int cond_status;
 apr_status_t rv;
 apr_bucket_brigade *brigade_stdout;
+apr_bucket *bucket_eos;
 char sbuf[MAX_STRING_LEN];
 const char *location;

+if (role == FCGI_RESPONDER) {
+rv = add_request_body(r, r-pool, output_brigade, bucket_ctx);
+if (rv) {
+return rv;
+}
+}
+
+/* The eos bucket now */
+bucket_eos = apr_bucket_eos_create(r-connection-bucket_alloc);
+APR_BRIGADE_INSERT_TAIL(output_brigade, bucket_eos);
+
 /* Write output_brigade to fastcgi server */
 if ((rv = proc_write_ipc(bucket_ctx-ipc,
  output_brigade)) != APR_SUCCESS) {
@@ -304,6 +320,7 @@ handle_request_ipc(request_rec *r, int role,
 bucket_ctx-has_error = 1;
 return HTTP_INTERNAL_SERVER_ERROR;
 }
+apr_brigade_cleanup(output_brigade);

 /* Create brigade */
 brigade_stdout =
@@ -522,7 +539,8 @@ handle_request(request_rec * r, int role, fcgid_cm
 }

 static int add_request_body(request_rec *r, apr_pool_t *request_pool,
-apr_bucket_brigade *output_brigade)
+apr_bucket_brigade *output_brigade,
+fcgid_bucket_ctx *bucket_ctx)
 {
 apr_bucket *bucket_input, *bucket_header;
 apr_file_t *fd = NULL;
@@ -563,8 +581,6 @@ static int add_request_body(request_rec *r, apr_po
 return HTTP_INTERNAL_SERVER_ERROR;
 }

-
-
 while ((bucket_input = APR_BRIGADE_FIRST(input_brigade)) !=
APR_BRIGADE_SENTINEL(input_brigade)) {
 const char *data;
 apr_size_t len;
@@ -615,7 +631,9 @@ static int add_request_body(request_rec *r, apr_po
 return HTTP_INTERNAL_SERVER_ERROR;
 }

-if (request_size  sconf-max_mem_request_len) {
+if (sconf-stream_request_body) {
+bucket_stdin = bucket_input;
+} else if (request_size  sconf-max_mem_request_len) {
 apr_size_t wrote_len;
 static const char *fd_key = fcgid_fd;

@@ -701,6 +719,16 @@ static int add_request_body(request_rec *r, apr_po
 }
 APR_BRIGADE_INSERT_TAIL(output_brigade, bucket_header);
 APR_BRIGADE_INSERT_TAIL(output_brigade, bucket_stdin);
+
+if (sconf-stream_request_body) {
+/* Write output_brigade to fastcgi server */
+if ((rv = proc_write_ipc(bucket_ctx-ipc,
+ output_brigade)) != APR_SUCCESS) {
+bucket_ctx-has_error = 1;
+return HTTP_INTERNAL_SERVER_ERROR;
+}
+apr_brigade_cleanup(output_brigade);
+}
 }

 apr_brigade_cleanup(input_brigade);
@@ -731,7 +759,6 @@ static int add_request_body(request_rec *r, apr_po
 int bridge_request(request_rec * r, int role, fcgid_cmd_conf *cmd_conf

Re: mod_fcgid and streaming large request bodies

[Jeff Trawick]

On Fri, Dec 27, 2013 at 6:14 PM, Justin Erenkrantz jus...@erenkrantz.comwrote:

 On Fri, Dec 27, 2013 at 8:49 AM, Justin Erenkrantz
 jus...@erenkrantz.com wrote:
  Anyway, I'm diving into the code a bit - but, I figured it might be
 useful
  to see if anyone else has any thoughts about how to handle large request
  bodies with mod_fcgid.

 Here's a first-cut patch that compiles at least.  Cheers.  -- justin

 Add FcgidStreamRequestBody directive to send request body as it arrives
 without
 storing it in memory or on disk.


It would be quite valuable if there is a limit on how much can be pre-read
(0 for pure streaming).  Pre-reading the request body reduces the number of
application processes or threads required, and they are usually fatter
(sometimes dramatically so) than httpd workers.  While the current
implementation is truly ridiculous, there's some goodness that could be
kept.  I'll try to look through this in the next few days and see if it is
practical to keep the best of both versions.  If no round tuits are
available we should at least proceed with this, perhaps changing the
directive syntax to allow a hybrid implementation in the future with no
legacy cruft.  (Wild suggestion: FcgiPreReadRequestBody
Off|On|Unlimited|K, where On is defaultK)


 * modules/fcgid/mod_fcgid.c
   (fcgid_cmds): Add FcgidStreamRequestBody directive.
 * modules/fcgid/fcgid_conf.h
   (fcgid_server_conf): Add stream_request_body/stream_request_body_set
 flags.
 * modules/fcgid/fcgid_conf.c
   (DEFAULT_STREAM_REQUEST_BODY): Set to 0.
   (create_fcgid_server_config): Init stream_request_body flag.
   (merge_fcgid_server_config): Merge new stream_request_body flag.
   (set_stream_request_body): New config helper.
 * modules/fcgid/fcgid_bridge.c
   (add_request_body): Add a forward declaration to reduce diff (for now);
   take fcgid_bucket_ctx param; if stream_request_body is set, don't copy
   bucket and then call proc_write_ipc to send out that brigade and clear it
   out before next loop iteration.
   (bridge_request): Delay reading the request body until later.
   (handle_request_ipc): Move add_request_body call to here and append the
 EOS
   bucket before we do the final write of the request body.

 Index: modules/fcgid/fcgid_bridge.c
 ===
 --- modules/fcgid/fcgid_bridge.c (revision 1553671)
 +++ modules/fcgid/fcgid_bridge.c (working copy)
 @@ -287,6 +287,10 @@ static int getsfunc_fcgid_BRIGADE(char *buf, int l
  return done;
  }

 +static int add_request_body(request_rec *r, apr_pool_t *request_pool,
 +apr_bucket_brigade *output_brigade,
 +fcgid_bucket_ctx *bucket_ctx);
 +
  static int
  handle_request_ipc(request_rec *r, int role,
 apr_bucket_brigade *output_brigade,
 @@ -295,9 +299,21 @@ handle_request_ipc(request_rec *r, int role,
  int cond_status;
  apr_status_t rv;
  apr_bucket_brigade *brigade_stdout;
 +apr_bucket *bucket_eos;
  char sbuf[MAX_STRING_LEN];
  const char *location;

 +if (role == FCGI_RESPONDER) {
 +rv = add_request_body(r, r-pool, output_brigade, bucket_ctx);
 +if (rv) {
 +return rv;
 +}
 +}
 +
 +/* The eos bucket now */
 +bucket_eos = apr_bucket_eos_create(r-connection-bucket_alloc);
 +APR_BRIGADE_INSERT_TAIL(output_brigade, bucket_eos);
 +
  /* Write output_brigade to fastcgi server */
  if ((rv = proc_write_ipc(bucket_ctx-ipc,
   output_brigade)) != APR_SUCCESS) {
 @@ -304,6 +320,7 @@ handle_request_ipc(request_rec *r, int role,
  bucket_ctx-has_error = 1;
  return HTTP_INTERNAL_SERVER_ERROR;
  }
 +apr_brigade_cleanup(output_brigade);

  /* Create brigade */
  brigade_stdout =
 @@ -522,7 +539,8 @@ handle_request(request_rec * r, int role, fcgid_cm
  }

  static int add_request_body(request_rec *r, apr_pool_t *request_pool,
 -apr_bucket_brigade *output_brigade)
 +apr_bucket_brigade *output_brigade,
 +fcgid_bucket_ctx *bucket_ctx)
  {
  apr_bucket *bucket_input, *bucket_header;
  apr_file_t *fd = NULL;
 @@ -563,8 +581,6 @@ static int add_request_body(request_rec *r, apr_po
  return HTTP_INTERNAL_SERVER_ERROR;
  }

 -
 -
  while ((bucket_input = APR_BRIGADE_FIRST(input_brigade)) !=
 APR_BRIGADE_SENTINEL(input_brigade)) {
  const char *data;
  apr_size_t len;
 @@ -615,7 +631,9 @@ static int add_request_body(request_rec *r, apr_po
  return HTTP_INTERNAL_SERVER_ERROR;
  }

 -if (request_size  sconf-max_mem_request_len) {
 +if (sconf-stream_request_body) {
 +bucket_stdin = bucket_input;
 +} else if (request_size  sconf-max_mem_request_len) {
  apr_size_t wrote_len

Re: mod_fcgid and streaming large request bodies

[Justin Erenkrantz]

On Fri, Dec 27, 2013 at 6:42 PM, Jeff Trawick traw...@gmail.com wrote:
 It would be quite valuable if there is a limit on how much can be pre-read
 (0 for pure streaming).  Pre-reading the request body reduces the number of
 application processes or threads required, and they are usually fatter
 (sometimes dramatically so) than httpd workers.  While the current
 implementation is truly ridiculous, there's some goodness that could be
 kept.  I'll try to look through this in the next few days and see if it is
 practical to keep the best of both versions.  If no round tuits are
 available we should at least proceed with this, perhaps changing the
 directive syntax to allow a hybrid implementation in the future with no
 legacy cruft.  (Wild suggestion: FcgiPreReadRequestBody
 Off|On|Unlimited|K, where On is defaultK)

I definitely agree that doing some type of pre-read may be often
useful before we start sending stuff via FastCGI.  Yet, the current
code structure doesn't lend itself to that very well - I guess what we
could do is convert the IPC to be opened lazily rather than always
open it.  If we do that, then I think the code changes would be
manageable.  (In our use case, reading the first say 64KB into RAM is
fine, but we definitely want to avoid spooling to disk.)

I'll see if I can find some round tuits...

Cheers.  -- justin

[ANNOUNCE] mod_fcgid 2.3.9 released

[Jeff Trawick]

 The Apache Software Foundation and the Apache HTTP Server Project are
  pleased to announce the release of version 2.3.9 of mod_fcgid, a
  FastCGI implementation for Apache HTTP Server versions 2.0, 2.2, and
  2.4.  This version of mod_fcgid is a security release, resolving a
  defect that could result in a denial of service with some applications.
  Other fixes and improvements are also included in this release.

  mod_fcgid is available for download from:

http://httpd.apache.org/download.cgi#mod_fcgid

  A full list of changes in this release follows:

  *) SECURITY: CVE-2013-4365 (cve.mitre.org)
 Fix possible heap buffer overwrite.  Reported and solved by:
 [Robert Matthews rob tigertech.com]

  *) Add experimental cmake-based build system for Windows.  [Jeff Trawick]

  *) Correctly parse quotation and escaped spaces in FcgidWrapper and the
 AAA Authenticator/Authorizor/Access directives' command line argument,
 as currently documented.  PR 51194  [William Rowe]

  *) Honor quoted FcgidCmdOptions arguments (notably for InitialEnv
 assignments).  PR 51657  [William Rowe]

  *) Conform script response parsing with mod_cgid and ensure no response
 body is sent when ap_meets_conditions() determines that request
 conditions are met.  [Chris Darroch]

  *) Improve logging in access control hook functions.  [Chris Darroch]

  *) Avoid making internal sub-requests and processing Location headers
 when in FCGI_AUTHORIZER mode, as the auth hook functions already
 treat Location headers returned by scripts as an error since
 redirections are not meaningful in this mode.  [Chris Darroch]

Re: [VOTE] Release mod_fcgid 2.3.9

[Jeff Trawick]

On Fri, Oct 4, 2013 at 5:16 PM, Jeff Trawick traw...@gmail.com wrote:

 Tarballs/zips are at http://httpd.apache.org/dev/dist/mod_fcgid/

 Shortcut to changes:
 http://httpd.apache.org/dev/dist/mod_fcgid/CHANGES-FCGID

 The difference with 2.3.8 is that one of the regressions found in 2.3.8
 has been reverted and and the other has been fixed.

 +/-1
 [  ] Release mod_fcgid 2.3.9 as GA

 I'll hold the vote open for 72 hours unless something out of the ordinary
 occurs.


We're getting fairly close.  If someone still plans to test and might need
a little extra time, please let me know on this thread.


 Thanks in advance for testing!

 --
 Born in Roswell... married an alien...
 http://emptyhammock.com/




-- 
Born in Roswell... married an alien...
http://emptyhammock.com/

Re: [VOTE] Release mod_fcgid 2.3.9

[Piotr Kloc]

W dniu 2013-10-07 18:13, Jeff Trawick pisze:
On Fri, Oct 4, 2013 at 5:16 PM, Jeff Trawick traw...@gmail.com 
mailto:traw...@gmail.com wrote:


Tarballs/zips are at http://httpd.apache.org/dev/dist/mod_fcgid/

Shortcut to changes:
http://httpd.apache.org/dev/dist/mod_fcgid/CHANGES-FCGID

The difference with 2.3.8 is that one of the regressions found in
2.3.8 has been reverted and and the other has been fixed.

+/-1
[  ] Release mod_fcgid 2.3.9 as GA

I'll hold the vote open for 72 hours unless something out of the
ordinary occurs.


We're getting fairly close.  If someone still plans to test and might 
need a little extra time, please let me know on this thread.



I have tested this on few servers with about 2000 vhosts and all is ok

Piotr

Re: [VOTE] Release mod_fcgid 2.3.9

[Jeff Trawick]

On Fri, Oct 4, 2013 at 5:16 PM, Jeff Trawick traw...@gmail.com wrote:

 Tarballs/zips are at http://httpd.apache.org/dev/dist/mod_fcgid/

 Shortcut to changes:
 http://httpd.apache.org/dev/dist/mod_fcgid/CHANGES-FCGID

 The difference with 2.3.8 is that one of the regressions found in 2.3.8
 has been reverted and and the other has been fixed.

 +/-1
 [  ] Release mod_fcgid 2.3.9 as GA

 I'll hold the vote open for 72 hours unless something out of the ordinary
 occurs.

 Thanks in advance for testing!


The vote is successful, and mod_fcgid 2.3.9 is on its way to the mirrors.

Thanks to members of the community for your invaluable assistance with
getting this new release ready!

Re: [VOTE] Release mod_fcgid 2.3.9

[William A. Rowe Jr.]

Excellent, although I was not able to test, I was more than happy to laud
one extra 'officious' +1 based on non-member, community review.  This has
been subject to almost more scrutiny than several core httpd releases :)

Re: [VOTE] Release mod_fcgid 2.3.9

[Steffen]

Maybe other testers can have a look:

Tested with 2.2.25 Win32.

Stopping Apache does not exit/kill the running processes

Processes with FcgidIdleTimeout more then 300 (default) is still 
there, a process has 2016 seconds and stays with 20 accesses and it 
does not exit/kill:


PidActiveIdleAccessesState15682357201620Ready596023594036Ready

Ok exit, when processes reach:
FcgidInitialEnv PHP_FCGI_MAX_REQUESTS 1000

FcgidMaxRequestsPerProcess 1000





On Friday 04/10/2013 at 23:16, Jeff Trawick  wrote:



Tarballs/zips are at http://httpd.apache.org/dev/dist/mod_fcgid/

Shortcut to changes: 
http://httpd.apache.org/dev/dist/mod_fcgid/CHANGES-FCGID


The difference with 2.3.8 is that one of the regressions found in 
2.3.8 has been reverted and and the other has been fixed.


+/-1
[  ] Release mod_fcgid 2.3.9 as GA

I'll hold the vote open for 72 hours unless something out of the 
ordinary occurs.


Thanks in advance for testing!
--
Born in Roswell... married an alien...
http://emptyhammock.com/

Re: [VOTE] Release mod_fcgid 2.3.9

[Steffen]

From the docu: FcgidIdleTimeout , Application processes which have not 
handled a request for this period of time will be terminated, if the 
number of processses for the class exceeds FcgidMinProcessesPerClass.


The  FcgidMinProcessesPerClass is default (3), so not exit with two 
processes should be ok. had read the docu before.



Not tested for more processes (yet)


Left to have attention: Stopping Apache does not exit/kill the running 
processes, not sure it is regression



On Sunday 06/10/2013 at 12:13, Steffen  wrote:



Maybe other testers can have a look:

Tested with 2.2.25 Win32.

Stopping Apache does not exit/kill the running processes

Processes with FcgidIdleTimeout more then 300 (default) is still 
there, a process has 2016 seconds and stays with 20 accesses and it 
does not exit/kill:


PidActiveIdleAccessesState15682357201620Ready596023594036Ready

Ok exit, when processes reach:
FcgidInitialEnv PHP_FCGI_MAX_REQUESTS 1000

FcgidMaxRequestsPerProcess 1000





On Friday 04/10/2013 at 23:16, Jeff Trawick  wrote:



Tarballs/zips are at http://httpd.apache.org/dev/dist/mod_fcgid/

Shortcut to changes: 
http://httpd.apache.org/dev/dist/mod_fcgid/CHANGES-FCGID


The difference with 2.3.8 is that one of the regressions found in 
2.3.8 has been reverted and and the other has been fixed.


+/-1
[  ] Release mod_fcgid 2.3.9 as GA

I'll hold the vote open for 72 hours unless something out of the 
ordinary occurs.


Thanks in advance for testing!
--
Born in Roswell... married an alien...
http://emptyhammock.com/

Re: [VOTE] Release mod_fcgid 2.3.9

[Jeff Trawick]

On Sun, Oct 6, 2013 at 6:44 AM, Steffen i...@apachelounge.com wrote:

  From the docu: FcgidIdleTimeout , Application processes which have not
 handled a request for this period of time will be terminated, if the number
 of processses for the class exceeds FcgidMinProcessesPerClass.

 The  FcgidMinProcessesPerClass is default (3), so not exit with two
 processes should be ok. had read the docu before.

 Not tested for more processes (yet)


I could see processes exiting not long after FcgidIdleTimeout with this
setting:

FcgidIdleTimeout 30
FcgidMinProcessesPerClass 1
FcgidIdleScanInterval 3

Without that last directive, it will only look for idle processes every two
minutes.  FcgidZombieScanInterval might reduce the window where a process
is thought to be exiting, but I haven't tried that.

With both 2.3.7 and 2.3.9, the message mod_fcgid: process  graceful
kill fail, sending SIGKILL seems to be prevalent.  I wonder how many times
the graceful kill works.



 Left to have attention: Stopping Apache does not exit/kill the running
 processes, not sure it is regression


I only tested with php-cgi.exe processes, and they seem to die a few
seconds after I initiated shutdown, with both 2.3.7 and 2.3.9.

2.3.7 added this directive, which I haven't tried:

http://httpd.apache.org/mod_fcgid/mod/mod_fcgid.html#fcgidwin32preventorphans

 But I haven't seen an issue with exiting processes anyway.


 On Sunday 06/10/2013 at 12:13, Steffen wrote:

 Maybe other testers can have a look:

 Tested with 2.2.25 Win32.

 Stopping Apache does not exit/kill the running processes

 Processes with FcgidIdleTimeout more then 300 (default) is still there, a
 process has 2016 seconds and stays with 20 accesses and it does not
 exit/kill:

 PidActiveIdleAccessesState15682357201620Ready596023594036Ready

 Ok exit, when processes reach:
 FcgidInitialEnv PHP_FCGI_MAX_REQUESTS 1000
 FcgidMaxRequestsPerProcess 1000




 On Friday 04/10/2013 at 23:16, Jeff Trawick wrote:

 Tarballs/zips are at http://httpd.apache.org/dev/dist/mod_fcgid/

 Shortcut to changes:
 http://httpd.apache.org/dev/dist/mod_fcgid/CHANGES-FCGID

 The difference with 2.3.8 is that one of the regressions found in 2.3.8
 has been reverted and and the other has been fixed.

 +/-1
 [  ] Release mod_fcgid 2.3.9 as GA

 I'll hold the vote open for 72 hours unless something out of the ordinary
 occurs.

 Thanks in advance for testing!

 --
 Born in Roswell... married an alien...
 http://emptyhamm**ock.com/ http://emptyhammock.com/






-- 
Born in Roswell... married an alien...
http://emptyhammock.com/

Re: [VOTE] Release mod_fcgid 2.3.9

[Chris Darroch]

Jeff Trawick wrote:

[ ] Release mod_fcgid 2.3.9 as GA

+1

Chris.

--
GPG Key ID: 088335A9
GPG Key Fingerprint: 86CD 3297 7493 75BC F820  6715 F54F E648 0883 35A9

Re: [VOTE] Release mod_fcgid 2.3.9

[Jeff Trawick]

On Fri, Oct 4, 2013 at 5:16 PM, Jeff Trawick traw...@gmail.com wrote:

 Tarballs/zips are at http://httpd.apache.org/dev/dist/mod_fcgid/

 Shortcut to changes:
 http://httpd.apache.org/dev/dist/mod_fcgid/CHANGES-FCGID

 The difference with 2.3.8 is that one of the regressions found in 2.3.8
 has been reverted and and the other has been fixed.

 +/-1



 [X] Release mod_fcgid 2.3.9 as GA

Ubuntu 12.10 64-bit: built with configure.apxs, tested with 64-bit builds
of httpd 2.2.x HEAD and httpd 2.4.x HEAD
Windows 7: built with VS2012 and cmaket, tested with 64-bit build of httpd
2.4.x HEAD

-- 
Born in Roswell... married an alien...
http://emptyhammock.com/

Re: [VOTE] Release mod_fcgid 2.3.8

[Steffen]

Re: [VOTE] Release mod_fcgid 2.3.8

[Jeff Trawick]

On Fri, Oct 4, 2013 at 6:23 AM, Steffen i...@apachelounge.com wrote:

  That looks better and so far I can see it is the behavior as with 2.3.7.

 Keep it running at AL. When I see some strange, I shall report.


That is good news.  I hope to tag and roll mod_fcgid 2.3.9 later in the day
unless I hear of problems.



 For your info, the build warnings:


Yeah :(  The whole stack from APR on up needs attention.  For now I
verified that they don't get worse between 2.3.7 and current svn sources.


 warning C4267: 'function' : conversion from 'size_t' to 'DWORD', possible
 loss of data E:\VC11\Win64\2.4.6\modules\mod_fcgid-2.3.8\fcgid_proc_win.c
 339
 warning C4267: 'function' : conversion from 'size_t' to 'DWORD', possible
 loss of data E:\VC11\Win64\2.4.6\modules\mod_fcgid-2.3.8\fcgid_proc_win.c
 411
 warning C4267: 'function' : conversion from 'size_t' to 'int', possible
 loss of data E:\VC11\Win64\2.4.6\include\http_protocol.h 414

 warning C4244: '=' : conversion from '__int64' to 'int', possible loss of
 data E:\VC11\Win64\2.4.6\modules\mod_fcgid-2.3.8\fcgid_bridge.c 69
 warning C4244: '=' : conversion from '__int64' to 'int', possible loss of
 data E:\VC11\Win64\2.4.6\modules\mod_fcgid-2.3.8\fcgid_bridge.c 116
 warning C4244: '=' : conversion from '__int64' to 'int', possible loss of
 data E:\VC11\Win64\2.4.6\modules\mod_fcgid-2.3.8\fcgid_bridge.c 121
 warning C4244: '=' : conversion from '__int64' to 'int', possible loss of
 data E:\VC11\Win64\2.4.6\modules\mod_fcgid-2.3.8\fcgid_pm_main.c 40
 warning C4244: '=' : conversion from '__int64' to 'int', possible loss of
 data E:\VC11\Win64\2.4.6\modules\mod_fcgid-2.3.8\fcgid_pm_main.c 104
 warning C4244: '=' : conversion from '__int64' to 'int', possible loss of
 data E:\VC11\Win64\2.4.6\modules\mod_fcgid-2.3.8\fcgid_pm_main.c 209
 warning C4244: '=' : conversion from '__int64' to 'int', possible loss of
 data E:\VC11\Win64\2.4.6\modules\mod_fcgid-2.3.8\fcgid_pm_main.c 312
 warning C4244: '=' : conversion from '__int64' to 'int', possible loss of
 data E:\VC11\Win64\2.4.6\modules\mod_fcgid-2.3.8\fcgid_proctbl_win.c 71


 On Friday 04/10/2013 at 03:59, Jeff Trawick wrote:

 On Thu, Oct 3, 2013 at 5:45 AM, Steffen i...@apachelounge.com wrote:

 Running in real at AL for an hour with patch revert-r1377398.txt

 Results, see www.apachelounge.com/status-revert-r1377398.html

 Observation:

 No hanging with working and no accesses anymore

 Still quite some more processes: with 2.3.7 1-3 and now 8

 Processes with idle time more then 300 (default) is still there, a
 process has 1818 seconds and stays with 11 accesses and it does not
 stop/kill.

 The diff with 2.3.7 that there are more entries with Process:

 In 2.3.7 I saw only one entry, never more:

 *Process: php-cgi.exe*  (d:/servers/apache/php/php-cgi.exe)
 ..
 ..
 ..

 As you can see in 2.3.8 more:

  *Process: php-cgi.exe*  (d:/servers/apache/php/php-cgi.exe)
 ..
 ..
 *Process: php-cgi.exe*  (d:/servers/apache/php/php-cgi.exe)
 ...
 ..
 etc.



 Back to 2.3.7


 Steffen, thanks again for sticking with this.  I've had trouble finding
 the time myself :(

 I've committed the last patch you tried, as well as one additional fix
 which could explain your latest results.  (r1529062)

 Please try the latest from svn, or add this very minor change on top of
 the revert-r1377398.txt patch:


 http://svn.apache.org/viewvc/httpd/mod_fcgid/trunk/modules/fcgid/fcgid_pm_win.c?r1=1529062r2=1529061pathrev=1529062

 These two uninitialized fields on Windows could prevent finding a
 suitable, existing FastCGI process when one is needed.  (This was a
 regression in 2.3.8.)

 Thanks!!




 *From:* Jeff Trawick traw...@gmail.com
 *Sent:* Wednesday, October 2, 2013 2:10 AM
 *To:* Apache HTTP Server Development List dev@httpd.apache.org
 *Subject:* Re: [VOTE] Release mod_fcgid 2.3.8

 On Tue, Oct 1, 2013 at 6:57 PM, Jeff Trawick traw...@gmail.comwrote:

  On Tue, Oct 1, 2013 at 8:35 AM, Jeff Trawick traw...@gmail.com wrote:

  On Sun, Sep 29, 2013 at 2:00 PM, Jeff Trawick traw...@gmail.comwrote:

 Tarballs/zips are at http://httpd.apache.org/dev/dist/mod_fcgid/

 Shortcut to changes:
 http://httpd.apache.org/dev/dist/mod_fcgid/CHANGES-FCGID

  +/-1
 [  ] Release mod_fcgid 2.3.8 as GA

 I'll hold the vote open for 72 hours unless something out of the
 ordinary occurs.


 This release is scrapped based on Steffen's test results.

 I know of one 2.3.8 change to back out that restores better behavior in
 my testing, but we don't have a proposed fix for everything that Steffen
 saw:

 - just one process is serving.
 - rest is just “hanging”  as working with no accesses and high idle
 time.

 I suspect that all I can manage today is to move my test to Windows and
 see if that behavior shows up.



 The previous patch I suggested was just a small bit of 1377398.  I am
 testing a complete revert of that, as well as the two follow-up changes
 1397778 and 1527358.  With luck I'll even see Steffen's problem on Windows
 without

Re: [VOTE] Release mod_fcgid 2.3.8

[Jeff Trawick]

On Fri, Oct 4, 2013 at 8:19 AM, Jeff Trawick traw...@gmail.com wrote:

 On Fri, Oct 4, 2013 at 6:23 AM, Steffen i...@apachelounge.com wrote:

  That looks better and so far I can see it is the behavior as with 2.3.7.

 Keep it running at AL. When I see some strange, I shall report.


 That is good news.  I hope to tag and roll mod_fcgid 2.3.9 later in the
 day unless I hear of problems.


Starting that now...





 For your info, the build warnings:


 Yeah :(  The whole stack from APR on up needs attention.  For now I
 verified that they don't get worse between 2.3.7 and current svn sources.


 warning C4267: 'function' : conversion from 'size_t' to 'DWORD', possible
 loss of data E:\VC11\Win64\2.4.6\modules\mod_fcgid-2.3.8\fcgid_proc_win.c
 339
 warning C4267: 'function' : conversion from 'size_t' to 'DWORD', possible
 loss of data E:\VC11\Win64\2.4.6\modules\mod_fcgid-2.3.8\fcgid_proc_win.c
 411
 warning C4267: 'function' : conversion from 'size_t' to 'int', possible
 loss of data E:\VC11\Win64\2.4.6\include\http_protocol.h 414

 warning C4244: '=' : conversion from '__int64' to 'int', possible loss of
 data E:\VC11\Win64\2.4.6\modules\mod_fcgid-2.3.8\fcgid_bridge.c 69
 warning C4244: '=' : conversion from '__int64' to 'int', possible loss of
 data E:\VC11\Win64\2.4.6\modules\mod_fcgid-2.3.8\fcgid_bridge.c 116
 warning C4244: '=' : conversion from '__int64' to 'int', possible loss of
 data E:\VC11\Win64\2.4.6\modules\mod_fcgid-2.3.8\fcgid_bridge.c 121
 warning C4244: '=' : conversion from '__int64' to 'int', possible loss of
 data E:\VC11\Win64\2.4.6\modules\mod_fcgid-2.3.8\fcgid_pm_main.c 40
 warning C4244: '=' : conversion from '__int64' to 'int', possible loss of
 data E:\VC11\Win64\2.4.6\modules\mod_fcgid-2.3.8\fcgid_pm_main.c 104
 warning C4244: '=' : conversion from '__int64' to 'int', possible loss of
 data E:\VC11\Win64\2.4.6\modules\mod_fcgid-2.3.8\fcgid_pm_main.c 209
 warning C4244: '=' : conversion from '__int64' to 'int', possible loss of
 data E:\VC11\Win64\2.4.6\modules\mod_fcgid-2.3.8\fcgid_pm_main.c 312
 warning C4244: '=' : conversion from '__int64' to 'int', possible loss of
 data E:\VC11\Win64\2.4.6\modules\mod_fcgid-2.3.8\fcgid_proctbl_win.c 71


 On Friday 04/10/2013 at 03:59, Jeff Trawick wrote:

 On Thu, Oct 3, 2013 at 5:45 AM, Steffen i...@apachelounge.com wrote:

 Running in real at AL for an hour with patch revert-r1377398.txt

 Results, see www.apachelounge.com/status-revert-r1377398.html

 Observation:

 No hanging with working and no accesses anymore

 Still quite some more processes: with 2.3.7 1-3 and now 8

 Processes with idle time more then 300 (default) is still there, a
 process has 1818 seconds and stays with 11 accesses and it does not
 stop/kill.

 The diff with 2.3.7 that there are more entries with Process:

 In 2.3.7 I saw only one entry, never more:

 *Process: php-cgi.exe*  (d:/servers/apache/php/php-cgi.exe)
 ..
 ..
 ..

 As you can see in 2.3.8 more:

  *Process: php-cgi.exe*  (d:/servers/apache/php/php-cgi.exe)
 ..
 ..
 *Process: php-cgi.exe*  (d:/servers/apache/php/php-cgi.exe)
 ...
 ..
 etc.



 Back to 2.3.7


 Steffen, thanks again for sticking with this.  I've had trouble finding
 the time myself :(

 I've committed the last patch you tried, as well as one additional fix
 which could explain your latest results.  (r1529062)

 Please try the latest from svn, or add this very minor change on top of
 the revert-r1377398.txt patch:


 http://svn.apache.org/viewvc/httpd/mod_fcgid/trunk/modules/fcgid/fcgid_pm_win.c?r1=1529062r2=1529061pathrev=1529062

 These two uninitialized fields on Windows could prevent finding a
 suitable, existing FastCGI process when one is needed.  (This was a
 regression in 2.3.8.)

 Thanks!!




 *From:* Jeff Trawick traw...@gmail.com
 *Sent:* Wednesday, October 2, 2013 2:10 AM
 *To:* Apache HTTP Server Development List dev@httpd.apache.org
 *Subject:* Re: [VOTE] Release mod_fcgid 2.3.8

 On Tue, Oct 1, 2013 at 6:57 PM, Jeff Trawick traw...@gmail.comwrote:

  On Tue, Oct 1, 2013 at 8:35 AM, Jeff Trawick traw...@gmail.comwrote:

  On Sun, Sep 29, 2013 at 2:00 PM, Jeff Trawick traw...@gmail.comwrote:

 Tarballs/zips are at http://httpd.apache.org/dev/dist/mod_fcgid/

 Shortcut to changes:
 http://httpd.apache.org/dev/dist/mod_fcgid/CHANGES-FCGID

  +/-1
 [  ] Release mod_fcgid 2.3.8 as GA

 I'll hold the vote open for 72 hours unless something out of the
 ordinary occurs.


 This release is scrapped based on Steffen's test results.

 I know of one 2.3.8 change to back out that restores better behavior
 in my testing, but we don't have a proposed fix for everything that 
 Steffen
 saw:

 - just one process is serving.
 - rest is just “hanging”  as working with no accesses and high idle
 time.

 I suspect that all I can manage today is to move my test to Windows
 and see if that behavior shows up.



 The previous patch I suggested was just a small bit of 1377398.  I am
 testing a complete revert of that, as well as the two follow

Re: svn commit: r1357986 - /httpd/mod_fcgid/trunk/modules/fcgid/fcgid_bridge.c

[Jeff Trawick]

On Thu, Oct 3, 2013 at 1:06 AM, Chris Darroch chr...@pearsoncmg.com wrote:

 Chris Darroch wrote:

 The intent of r1357986 was to deal with a particular, wonky
 sub-case, when the Authorizer returns 200 (so the spec paragraph
 doesn't apply in this case, as it's a 200 OK response), but adds
 a Location header with a relative (not absolute) path.  In this case,
 2.3.7 and previous will run the sub-request and return whatever comes
 out of that -- sometimes munging the end result as a consequence.
 (Note that a 200 with an absolute URL in a Location header just produces
 a 401 response.)


   Actually, I have to take back that last parenthetical note --
 with 2.3.7, a 200 + absolute Location URL produces a 302 + Location header,
 and with trunk, it produces a 401 like other 200 + Location header cases.

   That might be, I suppose, considered a regression ... let me
 think on it a bit.  It's not covered by the spec case you mention, since
 the script is returning 200.  The 2.3.7 behaviour is inconsistent depending
 on the nature of the URL in the Location header, given a 200.  Trunk
 makes that behaviour consistent, but perhaps that's a regression?  Hmm.


 The app is out of spec either way.  I think the trunk behavior is better.




 Chris.

 --
 GPG Key ID: 088335A9
 GPG Key Fingerprint: 86CD 3297 7493 75BC F820  6715 F54F E648 0883 35A9




-- 
Born in Roswell... married an alien...
http://emptyhammock.com/

Re: [VOTE] Release mod_fcgid 2.3.8

[Steffen]

K

 On 4 okt. 2013, at 22:15, Jeff Trawick traw...@gmail.com wrote:
 
 On Fri, Oct 4, 2013 at 8:19 AM, Jeff Trawick traw...@gmail.com wrote:
 On Fri, Oct 4, 2013 at 6:23 AM, Steffen i...@apachelounge.com wrote:
 That looks better and so far I can see it is the behavior as with 2.3.7.
 
 Keep it running at AL. When I see some strange, I shall report.
 
 That is good news.  I hope to tag and roll mod_fcgid 2.3.9 later in the day 
 unless I hear of problems.
 
 Starting that now...
  
  
 
 For your info, the build warnings:
 
 Yeah :(  The whole stack from APR on up needs attention.  For now I verified 
 that they don't get worse between 2.3.7 and current svn sources.
  
 warning C4267: 'function' : conversion from 'size_t' to 'DWORD', possible 
 loss of data  
 E:\VC11\Win64\2.4.6\modules\mod_fcgid-2.3.8\fcgid_proc_win.c339
 warning C4267: 'function' : conversion from 'size_t' to 'DWORD', possible 
 loss of data  
 E:\VC11\Win64\2.4.6\modules\mod_fcgid-2.3.8\fcgid_proc_win.c411
 warning C4267: 'function' : conversion from 'size_t' to 'int', possible 
 loss of dataE:\VC11\Win64\2.4.6\include\http_protocol.h 414
 
 warning C4244: '=' : conversion from '__int64' to 'int', possible loss of 
 data  E:\VC11\Win64\2.4.6\modules\mod_fcgid-2.3.8\fcgid_bridge.c  69
 warning C4244: '=' : conversion from '__int64' to 'int', possible loss of 
 data  E:\VC11\Win64\2.4.6\modules\mod_fcgid-2.3.8\fcgid_bridge.c  
 116
 warning C4244: '=' : conversion from '__int64' to 'int', possible loss of 
 data  E:\VC11\Win64\2.4.6\modules\mod_fcgid-2.3.8\fcgid_bridge.c  
 121
 warning C4244: '=' : conversion from '__int64' to 'int', possible loss of 
 data  E:\VC11\Win64\2.4.6\modules\mod_fcgid-2.3.8\fcgid_pm_main.c 40
 warning C4244: '=' : conversion from '__int64' to 'int', possible loss of 
 data  E:\VC11\Win64\2.4.6\modules\mod_fcgid-2.3.8\fcgid_pm_main.c 
 104
 warning C4244: '=' : conversion from '__int64' to 'int', possible loss of 
 data  E:\VC11\Win64\2.4.6\modules\mod_fcgid-2.3.8\fcgid_pm_main.c 
 209
 warning C4244: '=' : conversion from '__int64' to 'int', possible loss of 
 data  E:\VC11\Win64\2.4.6\modules\mod_fcgid-2.3.8\fcgid_pm_main.c 
 312
 warning C4244: '=' : conversion from '__int64' to 'int', possible loss of 
 data  E:\VC11\Win64\2.4.6\modules\mod_fcgid-2.3.8\fcgid_proctbl_win.c 71
 
  
 On Friday 04/10/2013 at 03:59, Jeff Trawick wrote:
 On Thu, Oct 3, 2013 at 5:45 AM, Steffen i...@apachelounge.com wrote:
 Running in real at AL for an hour with patch revert-r1377398.txt
  
 Results, see www.apachelounge.com/status-revert-r1377398.html
  
 Observation:
  
 No hanging with working and no accesses anymore
  
 Still quite some more processes: with 2.3.7 1-3 and now 8
  
 Processes with idle time more then 300 (default) is still there, a 
 process has 1818 seconds and stays with 11 accesses and it does not 
 stop/kill.
  
 The diff with 2.3.7 that there are more entries with Process:
  
 In 2.3.7 I saw only one entry, never more:
  
 Process: php-cgi.exe  (d:/servers/apache/php/php-cgi.exe)
 ..
 ..
 ..
  
 As you can see in 2.3.8 more:
  
  Process: php-cgi.exe  (d:/servers/apache/php/php-cgi.exe)
 ..
 ..
 Process: php-cgi.exe  (d:/servers/apache/php/php-cgi.exe)
 ...
 ..
 etc.
  
  
  
 Back to 2.3.7
 
 Steffen, thanks again for sticking with this.  I've had trouble finding 
 the time myself :(
 
 I've committed the last patch you tried, as well as one additional fix 
 which could explain your latest results.  (r1529062)
 
 Please try the latest from svn, or add this very minor change on top of 
 the revert-r1377398.txt patch:
 
 http://svn.apache.org/viewvc/httpd/mod_fcgid/trunk/modules/fcgid/fcgid_pm_win.c?r1=1529062r2=1529061pathrev=1529062
 
 These two uninitialized fields on Windows could prevent finding a 
 suitable, existing FastCGI process when one is needed.  (This was a 
 regression in 2.3.8.)
 
 Thanks!!
 
 
 
  
 From: Jeff Trawick
 Sent: Wednesday, October 2, 2013 2:10 AM
 To: Apache HTTP Server Development List
 Subject: Re: [VOTE] Release mod_fcgid 2.3.8
  
 On Tue, Oct 1, 2013 at 6:57 PM, Jeff Trawick traw...@gmail.comwrote:
 On Tue, Oct 1, 2013 at 8:35 AM, Jeff Trawick traw...@gmail.com wrote:
 On Sun, Sep 29, 2013 at 2:00 PM, Jeff Trawick traw...@gmail.com wrote:
 Tarballs/zips are at http://httpd.apache.org/dev/dist/mod_fcgid/
  
 Shortcut to changes: 
 http://httpd.apache.org/dev/dist/mod_fcgid/CHANGES-FCGID
  
 +/-1
 [  ] Release mod_fcgid 2.3.8 as GA
  
 I'll hold the vote open for 72 hours unless something out of the 
 ordinary occurs.
  
 This release is scrapped based on Steffen's test results.
  
 I know of one 2.3.8 change to back out that restores better behavior in 
 my testing, but we don't have a proposed fix for everything that 
 Steffen saw:
  
 - just one process is serving. 
 - rest is just “hanging”  as working with no accesses and high idle 
 time.
  
 I suspect that all I can manage

[VOTE] Release mod_fcgid 2.3.9

[Jeff Trawick]

Tarballs/zips are at http://httpd.apache.org/dev/dist/mod_fcgid/

Shortcut to changes:
http://httpd.apache.org/dev/dist/mod_fcgid/CHANGES-FCGID

The difference with 2.3.8 is that one of the regressions found in 2.3.8 has
been reverted and and the other has been fixed.

+/-1
[  ] Release mod_fcgid 2.3.9 as GA

I'll hold the vote open for 72 hours unless something out of the ordinary
occurs.

Thanks in advance for testing!

-- 
Born in Roswell... married an alien...
http://emptyhammock.com/

Re: [VOTE] Release mod_fcgid 2.3.9

[Gregg Smith]

Works for me
VC9 x86  x64
VC12 x86  x64
XP, Server 2003, Vista  Server 2012

[+1] Release mod_fcgid 2.3.9 as GA

Re: svn commit: r1357986 - /httpd/mod_fcgid/trunk/modules/fcgid/fcgid_bridge.c

[Chris Darroch]

Jeff Trawick wrote:


 The app is out of spec either way.  I think the trunk behavior is better.


  I'd agree on both counts (the latter IMHO, of course).  For reference,
here's a breakdown of 2.3.7 vs. trunk behaviour for Authorizers:

Authorizer response2.3.7   trunk
====   =
200proceed proceed
200 + rel Location 200 + munged out + err  401
200 + bad rel Location 404 + munged err + err  401
200 + abs Location 302 + Location  401
all other  401 401

  I'll try to run some quick smoke tests on your 2.3.9 over the weekend;
thanks again for pushing this along.

Chris.

--
GPG Key ID: 088335A9
GPG Key Fingerprint: 86CD 3297 7493 75BC F820  6715 F54F E648 0883 35A9

Re: [VOTE] Release mod_fcgid 2.3.9

[Mario Brandt]

 [+1 ] Release mod_fcgid 2.3.9 as GA
works for me on Debian 7 (x64)

Re: [VOTE] Release mod_fcgid 2.3.8

[Steffen]

Running in real at AL for an hour with patch revert-r1377398.txt 

Results, see www.apachelounge.com/status-revert-r1377398.html

Observation:

No hanging with working and no accesses anymore 

Still quite some more processes: with 2.3.7 1-3 and now 8 

Processes with idle time more then 300 (default) is still there, a process has 
1818 seconds and stays with 11 accesses and it does not stop/kill.

The diff with 2.3.7 that there are more entries with Process:

In 2.3.7 I saw only one entry, never more:

Process: php-cgi.exe  (d:/servers/apache/php/php-cgi.exe)
..
..
..

As you can see in 2.3.8 more:

 Process: php-cgi.exe  (d:/servers/apache/php/php-cgi.exe)
..
..
Process: php-cgi.exe  (d:/servers/apache/php/php-cgi.exe)
...
..
etc.



Back to 2.3.7





From: Jeff Trawick 
Sent: Wednesday, October 2, 2013 2:10 AM
To: Apache HTTP Server Development List 
Subject: Re: [VOTE] Release mod_fcgid 2.3.8

On Tue, Oct 1, 2013 at 6:57 PM, Jeff Trawick traw...@gmail.com wrote:

  On Tue, Oct 1, 2013 at 8:35 AM, Jeff Trawick traw...@gmail.com wrote:

On Sun, Sep 29, 2013 at 2:00 PM, Jeff Trawick traw...@gmail.com wrote:

  Tarballs/zips are at http://httpd.apache.org/dev/dist/mod_fcgid/ 

  Shortcut to changes: 
http://httpd.apache.org/dev/dist/mod_fcgid/CHANGES-FCGID

  +/-1
  [  ] Release mod_fcgid 2.3.8 as GA

  I'll hold the vote open for 72 hours unless something out of the ordinary 
occurs.

This release is scrapped based on Steffen's test results.

I know of one 2.3.8 change to back out that restores better behavior in my 
testing, but we don't have a proposed fix for everything that Steffen saw:

- just one process is serving.  
- rest is just “hanging”  as working with no accesses and high idle time.

I suspect that all I can manage today is to move my test to Windows and see 
if that behavior shows up.


  The previous patch I suggested was just a small bit of 1377398.  I am testing 
a complete revert of that, as well as the two follow-up changes 1397778 and 
1527358.  With luck I'll even see Steffen's problem on Windows without it, but 
even if not I'll share the new one.





  Thanks in advance for testing!




Steffen, by chance can you test the full revert of r1377398 (and follow-on 
fixes), at

http://people.apache.org/~trawick/revert-r1377398.txt


(CR-LF to make your GNU patch happy ;) )

I tested before and after on Windows using a trivial PHP script and 
php-cgi.exe.  I didn't see the issue Steffen saw, but it definitely used a lot 
fewer processes after reverting.

-- 
Born in Roswell... married an alien...
http://emptyhammock.com/

Re: [VOTE] Release mod_fcgid 2.3.8

[Jeff Trawick]

On Thu, Oct 3, 2013 at 5:45 AM, Steffen i...@apachelounge.com wrote:

   Running in real at AL for an hour with patch revert-r1377398.txt

 Results, see www.apachelounge.com/status-revert-r1377398.html

 Observation:

 No hanging with working and no accesses anymore

 Still quite some more processes: with 2.3.7 1-3 and now 8

 Processes with idle time more then 300 (default) is still there, a process
 has 1818 seconds and stays with 11 accesses and it does not stop/kill.

 The diff with 2.3.7 that there are more entries with Process:

 In 2.3.7 I saw only one entry, never more:

 *Process: php-cgi.exe*  (d:/servers/apache/php/php-cgi.exe)
 ..
 ..
 ..

 As you can see in 2.3.8 more:

  *Process: php-cgi.exe*  (d:/servers/apache/php/php-cgi.exe)
 ..
 ..
 *Process: php-cgi.exe*  (d:/servers/apache/php/php-cgi.exe)
 ...
 ..
 etc.



 Back to 2.3.7


Steffen, thanks again for sticking with this.  I've had trouble finding the
time myself :(

I've committed the last patch you tried, as well as one additional fix
which could explain your latest results.  (r1529062)

Please try the latest from svn, or add this very minor change on top of the
revert-r1377398.txt patch:

http://svn.apache.org/viewvc/httpd/mod_fcgid/trunk/modules/fcgid/fcgid_pm_win.c?r1=1529062r2=1529061pathrev=1529062

These two uninitialized fields on Windows could prevent finding a suitable,
existing FastCGI process when one is needed.  (This was a regression in
2.3.8.)

Thanks!!




  *From:* Jeff Trawick traw...@gmail.com
 *Sent:* Wednesday, October 2, 2013 2:10 AM
 *To:* Apache HTTP Server Development List dev@httpd.apache.org
 *Subject:* Re: [VOTE] Release mod_fcgid 2.3.8

  On Tue, Oct 1, 2013 at 6:57 PM, Jeff Trawick traw...@gmail.com wrote:

  On Tue, Oct 1, 2013 at 8:35 AM, Jeff Trawick traw...@gmail.com wrote:

  On Sun, Sep 29, 2013 at 2:00 PM, Jeff Trawick traw...@gmail.comwrote:

 Tarballs/zips are at http://httpd.apache.org/dev/dist/mod_fcgid/

 Shortcut to changes:
 http://httpd.apache.org/dev/dist/mod_fcgid/CHANGES-FCGID

  +/-1
 [  ] Release mod_fcgid 2.3.8 as GA

 I'll hold the vote open for 72 hours unless something out of the
 ordinary occurs.


 This release is scrapped based on Steffen's test results.

 I know of one 2.3.8 change to back out that restores better behavior in
 my testing, but we don't have a proposed fix for everything that Steffen
 saw:

 - just one process is serving.
 - rest is just “hanging”  as working with no accesses and high idle time.

 I suspect that all I can manage today is to move my test to Windows and
 see if that behavior shows up.



 The previous patch I suggested was just a small bit of 1377398.  I am
 testing a complete revert of that, as well as the two follow-up changes
 1397778 and 1527358.  With luck I'll even see Steffen's problem on Windows
 without it, but even if not I'll share the new one.








 Thanks in advance for testing!




 Steffen, by chance can you test the full revert of r1377398 (and follow-on
 fixes), at

 http://people.apache.org/~trawick/revert-r1377398.txt

 (CR-LF to make your GNU patch happy ;) )

 I tested before and after on Windows using a trivial PHP script and
 php-cgi.exe.  I didn't see the issue Steffen saw, but it definitely used a
 lot fewer processes after reverting.

 --
 Born in Roswell... married an alien...
 http://emptyhammock.com/




-- 
Born in Roswell... married an alien...
http://emptyhammock.com/

Re: svn commit: r1357986 - /httpd/mod_fcgid/trunk/modules/fcgid/fcgid_bridge.c

[Chris Darroch]

Jeff Trawick wrote:


URL: http://svn.apache.org/viewvc?rev=1357986view=rev
http://svn.apache.org/viewvc?rev=1357986view=rev
Log:
Avoid internal sub-requests and processing of Location headers when
in FCGI_AUTHORIZER mode, as the mod_fcgid_authenticator(), etc. hook
functions report an error if the script returned a Location header and
redirections are nonsensical in this mode.

Previously, the handle_request_ipc() and handle_request() functions
would
examine this header when in FCGI_AUTHORIZER mode and then possibly
execute
an internal sub-request, which has no particular use, as its return
value
is ignored and its output may conflict with that of the actual content
generation phase.

 From the FastCGI spec (6.3):

For Authorizer response status values other than 200 (OK), the Web 
server denies access and sends the response status, headers, and content 
back to the HTTP client.


I was initially confused when looking at this commit (nothing like 
reviewing one year later) wondering if it broke this requirement, but 
AFAICT 2.3.7 didn't support the feature anyway, so no regression.  (Some 
iff statements in this code are what control it.)


  Yes, I believe that's correct (it's been a while).  It's a really
good question and worth having a second pair of eyes.

  I did a smidge of testing today, comparing 2.3.7 and trunk,
with a test Authorizer that returns things like 403, 500, etc.
I believe there's no difference for most of these cases; the 2.3.7
behaviour is to return a 401, which is still the case.  (This may
not follow the spec, as you point out, but is the existing behaviour;
I'll double-check on that tomorrow.)

  The intent of r1357986 was to deal with a particular, wonky
sub-case, when the Authorizer returns 200 (so the spec paragraph
doesn't apply in this case, as it's a 200 OK response), but adds
a Location header with a relative (not absolute) path.  In this case,
2.3.7 and previous will run the sub-request and return whatever comes
out of that -- sometimes munging the end result as a consequence.
(Note that a 200 with an absolute URL in a Location header just produces
a 401 response.)

  For example, if the Authorizer returns Location: /cgi-bin/printenv
and a 200, you get something strange like:


HTTP/1.1 200 OK
...

CONTEXT_PREFIX=/cgi-bin/
...
SERVER_SOFTWARE=Apache/2.5.0-dev (Unix) mod_fcgid/2.3.7
!DOCTYPE HTML PUBLIC -//IETF//DTD HTML 2.0//EN
htmlhead
title200 Unknown Reason/title
/headbody
h1Unknown Reason/h1
...


  Or if the Location is bad (say, Location: /cgi-bin/printenv.FOO)
then you get a 404 with a 200 Unknown Reason glued on:


HTTP/1.1 404 Not Found
...

!DOCTYPE HTML PUBLIC -//IETF//DTD HTML 2.0//EN
htmlhead
title404 Not Found/title
/headbody
h1Not Found/h1
pThe requested URL /cgi-bin/printenv.FOO was not found on this server./p
/body/html
!DOCTYPE HTML PUBLIC -//IETF//DTD HTML 2.0//EN
htmlhead
title200 Unknown Reason/title
/headbody


  So r1357986 is just trying to handle those 200 OK + Location: /...
header cases.  If it does more than that, we should review further,
but that was the only intent ... IIRC after about a year or so.

Chris.

--
GPG Key ID: 088335A9
GPG Key Fingerprint: 86CD 3297 7493 75BC F820  6715 F54F E648 0883 35A9

Re: svn commit: r1357986 - /httpd/mod_fcgid/trunk/modules/fcgid/fcgid_bridge.c

[Chris Darroch]

Chris Darroch wrote:


   The intent of r1357986 was to deal with a particular, wonky
sub-case, when the Authorizer returns 200 (so the spec paragraph
doesn't apply in this case, as it's a 200 OK response), but adds
a Location header with a relative (not absolute) path.  In this case,
2.3.7 and previous will run the sub-request and return whatever comes
out of that -- sometimes munging the end result as a consequence.
(Note that a 200 with an absolute URL in a Location header just produces
a 401 response.)


  Actually, I have to take back that last parenthetical note --
with 2.3.7, a 200 + absolute Location URL produces a 302 + Location header,
and with trunk, it produces a 401 like other 200 + Location header cases.

  That might be, I suppose, considered a regression ... let me
think on it a bit.  It's not covered by the spec case you mention, since
the script is returning 200.  The 2.3.7 behaviour is inconsistent depending
on the nature of the URL in the Location header, given a 200.  Trunk
makes that behaviour consistent, but perhaps that's a regression?  Hmm.

Chris.

--
GPG Key ID: 088335A9
GPG Key Fingerprint: 86CD 3297 7493 75BC F820  6715 F54F E648 0883 35A9

Re: [VOTE] Release mod_fcgid 2.3.8

[Jeff Trawick]

On Sun, Sep 29, 2013 at 2:00 PM, Jeff Trawick traw...@gmail.com wrote:

 Tarballs/zips are at http://httpd.apache.org/dev/dist/mod_fcgid/

 Shortcut to changes:
 http://httpd.apache.org/dev/dist/mod_fcgid/CHANGES-FCGID

 +/-1
 [  ] Release mod_fcgid 2.3.8 as GA

 I'll hold the vote open for 72 hours unless something out of the ordinary
 occurs.


This release is scrapped based on Steffen's test results.

I know of one 2.3.8 change to back out that restores better behavior in my
testing, but we don't have a proposed fix for everything that Steffen saw:

- just one process is serving.
- rest is just “hanging”  as working with no accesses and high idle time.

I suspect that all I can manage today is to move my test to Windows and see
if that behavior shows up.


 Thanks in advance for testing!

 --
 Born in Roswell... married an alien...
 http://emptyhammock.com/




-- 
Born in Roswell... married an alien...
http://emptyhammock.com/

Re: [VOTE] Release mod_fcgid 2.3.8

[Jim Jagielski]

I can confirm that this makes 2.3.8 act more like 2.3.7...

On Sep 30, 2013, at 8:12 AM, Jeff Trawick traw...@gmail.com wrote:

 On Mon, Sep 30, 2013 at 5:23 AM, Steffen i...@apachelounge.com wrote:
 
 Running at AL now 2.3.7 again, all fine again with 1-3 processes instead of 
 the 30+ with 2.3.8.
 
 The config live running here and your config for synthetic test  is 
 different. And not using fat php processes and max processes is in place.
 
 Not any mod_fcgid directives here in a vhost, only in server the commonly 
 used config for php. All vhosts here serving .php, including the default 
 vhost. Running server 2012.
 
 
 IfModule fcgid_module
 FcgidInitialEnv PHPRC d:/servers/apache/conf/
 FcgidInitialEnv PATH 
 d:/servers/apache/php;C:/WINDOWS/system32;C:/WINDOWS;C:/WINDOWS/System32/Wbem;
 FcgidInitialEnv SystemRoot C:/Windows
 FcgidInitialEnv SystemDrive C:
 FcgidInitialEnv TEMP C:/WINDOWS/Temp
 FcgidInitialEnv TMP C:/WINDOWS/Temp
 FcgidInitialEnv windir C:/WINDOWS
 
 FcgidInitialEnv PHP_FCGI_MAX_REQUESTS 1000
 FcgidMaxRequestsPerProcess 1000
 FcgidMaxProcesses 50
 FcgidMaxRequestLen 8131072
 
 # added for z-push iPhone
 FcgidIOTimeout  7200
 FcgidConnectTimeout 180
 FcgidBusyTimeout 7200
 /IfModule
 
 
 
 Files ~ \.php$
  AddHandler fcgid-script .php
  FcgidWrapper d:/servers/apache/php/php-cgi.exe .php
 /Files
 
 Hi Steffen,
 
 Are you able to try this patch?
 
 http://people.apache.org/~trawick/restore-fcgid-2.3.7-process-count-r1.txt
 
 This reverts one of the changes in 2.3.8 that affects how soon mod_fcgid will 
 create a new process when one isn't immediately available.  With this backed 
 out, my simple testcase uses 11-12 processes like 2.3.7 instead of 20 
 processes like 2.3.8.
 
 TIA!
  
 
 
 On Monday 30/09/2013 at 01:36, Jeff Trawick  wrote:
 
 On Sun, Sep 29, 2013 at 5:14 PM, Jeff Trawick traw...@gmail.com wrote:
 
 
 
 
 
 
 On Sun, Sep 29, 2013 at 4:04 PM, Steffen i...@apachelounge.com wrote:
 
 
 
 
 
 
 Becoming dramatic here, already running over 30 processes. Running out of 
 memory this way.
 
 
 All time high here is 5 processes, and while writing this mail it is already 
 34 and all 34 have an entry in the mod_status page.
 
 Also looks like it is not stopping/killing processes any more, have entries 
 with 1784 seconds idle (FcgidIdleTimeout is default, 300)
 
 Going back to 2.3.7 at AL.
 
 
 On Sunday 29/09/2013 at 21:15, Steffen  wrote:
 
 
 
 Observe a different behavior compared to 2.3.7
 
 
 - It spawns a lot more mod_fcgid processes, looks like vhost is in charge 
 (mod_fcgid only global defined here)
 
 - I see in Windows taskmanager and  in mod_status 5 processes and the error 
 log says that the are 3 started, a mismatch.
 
 - Also different in mod_status page, see more then one entry for Process: 
 php-cgi.exe
 
 With 2.3.8 splitted now :
 
 Total FastCGI processes: 5
 Process: php-cgi.exe  (d:/servers/apache/php/php-cgi.exe)
 PidActiveIdleAccessesState
 46204874317Ready
 Process: php-cgi.exe  (d:/servers/apache/php/php-cgi.exe)
 PidActiveIdleAccessesState
 38405151095Ready
 Process: php-cgi.exe  (d:/servers/apache/php/php-cgi.exe)
 PidActiveIdleAccessesState
 65525574566Ready
 Process: php-cgi.exe  (d:/servers/apache/php/php-cgi.exe)
 PidActiveIdleAccessesState
 320419627187Ready
 5036 2143 17192Ready
 
 
 With 2.3.7 was only one entry, like:
 
 Process: php-cgi.exe  (d:/servers/apache/php/php-cgi.exe)
 PidActiveIdleAccessesState
 320419627187Ready
 5036 2143 17  �� 192Ready
 ...
 
 Not really trust 2.3.8 (yet), give me a few days to observe more.
 
 
 
 Thanks, Steffen.  I'll try to reproduce soon and see which commit changed 
 that.  (Maybe 1377398?)
 
 
 
 So far I've only set up a simple test...  Simple perl FastCGI script, ab -n 
 20 -c 20, default main vhost configuration, this for the vhost:
 
 
 VirtualHost *:8082
 FCGIDCmdOptions /home/trawick/myhg/apache/fcgid/apps/altinfo.pl \
InitialEnv VHOST=any \
InitialEnv PERL5LIB=/home/trawick/perl5/lib/perl5
 /VirtualHost
 
 2.3.7 grows up to about 12 (vs. max 20 concurrent clients).
 2.3.8 grows up to about 20.
 
 I got both the fastest and slowest times for 200,000 requests using 2.3.8.  
 Generally I suspect 2.3.7 is slightly faster, but I don't have a good overall 
 summary.
 
 If you're using FcgidCmdOptions, I'd recommend using the MaxProcesses 
 parameter to something that your system can handle.  Otherwise, see 
 FcgidMaxProcesses and FcgidMaxProcessesPerClass.  Regardless of 2.3.7 or 
 2.3.8.
 
 Still, for this simple scenario + configuration, 2.3.7 would have been better 
 (generally not worse performance, uses 40% fewer processes).  Different 
 scenarios would have different results, but I think that the common, fat PHP 
 processes would have bigger problems with 2.3.8 if there is no reasonable 
 configured limit

Re: svn commit: r1357986 - /httpd/mod_fcgid/trunk/modules/fcgid/fcgid_bridge.c

[Jeff Trawick]

On Thu, Jul 5, 2012 at 7:01 PM, chr...@apache.org wrote:

 Author: chrisd
 Date: Thu Jul  5 23:01:09 2012
 New Revision: 1357986

 URL: http://svn.apache.org/viewvc?rev=1357986view=rev
 Log:
 Avoid internal sub-requests and processing of Location headers when
 in FCGI_AUTHORIZER mode, as the mod_fcgid_authenticator(), etc. hook
 functions report an error if the script returned a Location header and
 redirections are nonsensical in this mode.

 Previously, the handle_request_ipc() and handle_request() functions would
 examine this header when in FCGI_AUTHORIZER mode and then possibly execute
 an internal sub-request, which has no particular use, as its return value
 is ignored and its output may conflict with that of the actual content
 generation phase.


From the FastCGI spec (6.3):

For Authorizer response status values other than 200 (OK), the Web
server denies access and sends the response status, headers, and content
back to the HTTP client.

I was initially confused when looking at this commit (nothing like
reviewing one year later) wondering if it broke this requirement, but
AFAICT 2.3.7 didn't support the feature anyway, so no regression.  (Some
iff statements in this code are what control it.)



 Modified:
 httpd/mod_fcgid/trunk/modules/fcgid/fcgid_bridge.c

 Modified: httpd/mod_fcgid/trunk/modules/fcgid/fcgid_bridge.c
 URL:
 http://svn.apache.org/viewvc/httpd/mod_fcgid/trunk/modules/fcgid/fcgid_bridge.c?rev=1357986r1=1357985r2=1357986view=diff

 ==
 --- httpd/mod_fcgid/trunk/modules/fcgid/fcgid_bridge.c (original)
 +++ httpd/mod_fcgid/trunk/modules/fcgid/fcgid_bridge.c Thu Jul  5 23:01:09
 2012
 @@ -320,6 +320,10 @@ handle_request_ipc(request_rec *r, int r
  return cond_status;
  }

 +if (role == FCGI_AUTHORIZER) {
 +return cond_status;
 +}
 +
  /* Check redirect */
  location = apr_table_get(r-headers_out, Location);

 @@ -347,9 +351,8 @@ handle_request_ipc(request_rec *r, int r
  }

  /* Now pass to output filter */
 -if (role == FCGI_RESPONDER
 - (rv = ap_pass_brigade(r-output_filters,
 - brigade_stdout)) != APR_SUCCESS) {
 +if ((rv = ap_pass_brigade(r-output_filters,
 +  brigade_stdout)) != APR_SUCCESS) {
  if (!APR_STATUS_IS_ECONNABORTED(rv)) {
  ap_log_rerror(APLOG_MARK, APLOG_WARNING, rv, r,
mod_fcgid: ap_pass_brigade failed in 





-- 
Born in Roswell... married an alien...
http://emptyhammock.com/

Re: [VOTE] Release mod_fcgid 2.3.8

[Jeff Trawick]

On Tue, Oct 1, 2013 at 8:35 AM, Jeff Trawick traw...@gmail.com wrote:

 On Sun, Sep 29, 2013 at 2:00 PM, Jeff Trawick traw...@gmail.com wrote:

 Tarballs/zips are at http://httpd.apache.org/dev/dist/mod_fcgid/

 Shortcut to changes:
 http://httpd.apache.org/dev/dist/mod_fcgid/CHANGES-FCGID

 +/-1
 [  ] Release mod_fcgid 2.3.8 as GA

 I'll hold the vote open for 72 hours unless something out of the ordinary
 occurs.


 This release is scrapped based on Steffen's test results.

 I know of one 2.3.8 change to back out that restores better behavior in my
 testing, but we don't have a proposed fix for everything that Steffen saw:

 - just one process is serving.
 - rest is just “hanging”  as working with no accesses and high idle time.

 I suspect that all I can manage today is to move my test to Windows and
 see if that behavior shows up.


The previous patch I suggested was just a small bit of 1377398.  I am
testing a complete revert of that, as well as the two follow-up changes
1397778 and 1527358.  With luck I'll even see Steffen's problem on Windows
without it, but even if not I'll share the new one.




 Thanks in advance for testing!

 --
 Born in Roswell... married an alien...
 http://emptyhammock.com/




 --
 Born in Roswell... married an alien...
 http://emptyhammock.com/




-- 
Born in Roswell... married an alien...
http://emptyhammock.com/

Re: [VOTE] Release mod_fcgid 2.3.8

[Jeff Trawick]

On Tue, Oct 1, 2013 at 6:57 PM, Jeff Trawick traw...@gmail.com wrote:

 On Tue, Oct 1, 2013 at 8:35 AM, Jeff Trawick traw...@gmail.com wrote:

 On Sun, Sep 29, 2013 at 2:00 PM, Jeff Trawick traw...@gmail.com wrote:

 Tarballs/zips are at http://httpd.apache.org/dev/dist/mod_fcgid/

 Shortcut to changes:
 http://httpd.apache.org/dev/dist/mod_fcgid/CHANGES-FCGID

 +/-1
 [  ] Release mod_fcgid 2.3.8 as GA

 I'll hold the vote open for 72 hours unless something out of the
 ordinary occurs.


 This release is scrapped based on Steffen's test results.

 I know of one 2.3.8 change to back out that restores better behavior in
 my testing, but we don't have a proposed fix for everything that Steffen
 saw:

 - just one process is serving.
 - rest is just “hanging”  as working with no accesses and high idle time.

 I suspect that all I can manage today is to move my test to Windows and
 see if that behavior shows up.


 The previous patch I suggested was just a small bit of 1377398.  I am
 testing a complete revert of that, as well as the two follow-up changes
 1397778 and 1527358.  With luck I'll even see Steffen's problem on Windows
 without it, but even if not I'll share the new one.







 Thanks in advance for testing!

 --
 Born in Roswell... married an alien...
 http://emptyhammock.com/




 --
 Born in Roswell... married an alien...
 http://emptyhammock.com/




 --
 Born in Roswell... married an alien...
 http://emptyhammock.com/



Steffen, by chance can you test the full revert of r1377398 (and follow-on
fixes), at

http://people.apache.org/~trawick/revert-r1377398.txt

(CR-LF to make your GNU patch happy ;) )

I tested before and after on Windows using a trivial PHP script and
php-cgi.exe.  I didn't see the issue Steffen saw, but it definitely used a
lot fewer processes after reverting.

-- 
Born in Roswell... married an alien...
http://emptyhammock.com/

Re: [VOTE] Release mod_fcgid 2.3.8

[Steffen]

Running at AL now 2.3.7 again, all fine again with 1-3 processes 
instead of the 30+ with 2.3.8.


The config live running here and your config for synthetic test  is 
different. And not using fat php processes and max processes is in 
place.


Not any mod_fcgid directives here in a vhost, only in server the 
commonly used config for php. All vhosts here serving .php, including 
the default vhost. Running server 2012.



IfModule fcgid_module
FcgidInitialEnv PHPRC d:/servers/apache/conf/
FcgidInitialEnv PATH 
d:/servers/apache/php;C:/WINDOWS/system32;C:/WINDOWS;C:/WINDOWS/System32/Wbem;

FcgidInitialEnv SystemRoot C:/Windows
FcgidInitialEnv SystemDrive C:
FcgidInitialEnv TEMP C:/WINDOWS/Temp
FcgidInitialEnv TMP C:/WINDOWS/Temp
FcgidInitialEnv windir C:/WINDOWS

FcgidInitialEnv PHP_FCGI_MAX_REQUESTS 1000
FcgidMaxRequestsPerProcess 1000
FcgidMaxProcesses 50
FcgidMaxRequestLen 8131072

# added for z-push iPhone
FcgidIOTimeout  7200
FcgidConnectTimeout 180
FcgidBusyTimeout 7200
/IfModule



Files ~ \.php$
 AddHandler fcgid-script .php
 FcgidWrapper d:/servers/apache/php/php-cgi.exe .php
/Files


On Monday 30/09/2013 at 01:36, Jeff Trawick  wrote:


On Sun, Sep 29, 2013 at 5:14 PM, Jeff Trawick traw...@gmail.com 
wrote:








On Sun, Sep 29, 2013 at 4:04 PM, Steffen i...@apachelounge.com 
wrote:








Becoming dramatic here, already running over 30 processes. Running out 
of memory this way.



All time high here is 5 processes, and while writing this mail it is 
already 34 and all 34 have an entry in the mod_status page.


Also looks like it is not stopping/killing processes any more, have 
entries with 1784 seconds idle (FcgidIdleTimeout is default, 300)


Going back to 2.3.7 at AL.


On Sunday 29/09/2013 at 21:15, Steffen  wrote:




Observe a different behavior compared to 2.3.7


- It spawns a lot more mod_fcgid processes, looks like vhost is in 
charge (mod_fcgid only global defined here)


- I see in Windows taskmanager and  in mod_status 5 processes and the 
error log says that the are 3 started, a mismatch.


- Also different in mod_status page, see more then one entry for 
Process: php-cgi.exe


With 2.3.8 splitted now :

Total FastCGI processes: 5
Process: php-cgi.exe  (d:/servers/apache/php/php-cgi.exe)
PidActiveIdleAccessesState
46204874317Ready
Process: php-cgi.exe  (d:/servers/apache/php/php-cgi.exe)
PidActiveIdleAccessesState
38405151095Ready
Process: php-cgi.exe  (d:/servers/apache/php/php-cgi.exe)
PidActiveIdleAccessesState
65525574566Ready
Process: php-cgi.exe  (d:/servers/apache/php/php-cgi.exe)
PidActiveIdleAccessesState
320419627187Ready
5036214317192Ready


With 2.3.7 was only one entry, like:

Process: php-cgi.exe  (d:/servers/apache/php/php-cgi.exe)
PidActiveIdleAccessesState
320419627187Ready
5036214317  �� 192Ready
...

Not really trust 2.3.8 (yet), give me a few days to observe more.




Thanks, Steffen.  I'll try to reproduce soon and see which commit 
changed that.  (Maybe 1377398?)





So far I've only set up a simple test...  Simple perl FastCGI script, 
ab -n 20 -c 20, default main vhost configuration, this for the 
vhost:



VirtualHost *:8082
FCGIDCmdOptions /home/trawick/myhg/apache/fcgid/apps/altinfo.pl \
   InitialEnv VHOST=any \
   InitialEnv PERL5LIB=/home/trawick/perl5/lib/perl5
/VirtualHost

2.3.7 grows up to about 12 (vs. max 20 concurrent clients).
2.3.8 grows up to about 20.

I got both the fastest and slowest times for 200,000 requests using 
2.3.8.  Generally I suspect 2.3.7 is slightly faster, but I don't have 
a good overall summary.


If you're using FcgidCmdOptions, I'd recommend using the MaxProcesses 
parameter to something that your system can handle.  Otherwise, see 
FcgidMaxProcesses and FcgidMaxProcessesPerClass.  Regardless of 2.3.7 
or 2.3.8.


Still, for this simple scenario + configuration, 2.3.7 would have been 
better (generally not worse performance, uses 40% fewer processes).  
Different scenarios would have different results, but I think that the 
common, fat PHP processes would have bigger problems with 2.3.8 if 
there is no reasonable configured limit on the max to spawn.


Does anyone else have time to play?


















On Sunday 29/09/2013 at 20:01, Jeff Trawick  wrote:




Tarballs/zips are at http://httpd.apache.org/dev/dist/mod_fcgid/

Shortcut to changes: 
http://httpd.apache.org/dev/dist/mod_fcgid/CHANGES-FCGID



+/-1
[  ] Release mod_fcgid 2.3.8 as GA

I'll hold the vote open for 72 hours unless something out of the  
ordinary occurs.


Thanks in advance for testing!
--
Born in Roswell... married an alien...
http://emptyhammock.com/












--
Born in Roswell... married an alien...
http://emptyhammock.com/




--
Born in Roswell... married an alien...
http://emptyhammock.com/

Re: [VOTE] Release mod_fcgid 2.3.8

[Jeff Trawick]

On Mon, Sep 30, 2013 at 5:23 AM, Steffen i...@apachelounge.com wrote:


 Running at AL now 2.3.7 again, all fine again with 1-3 processes instead
 of the 30+ with 2.3.8.

 The config live running here and your config for synthetic test  is
 different. And not using fat php processes and max processes is in place.

 Not any mod_fcgid directives here in a vhost, only in server the commonly
 used config for php. All vhosts here serving .php, including the default
 vhost. Running server 2012.


 IfModule fcgid_module
 FcgidInitialEnv PHPRC d:/servers/apache/conf/
 FcgidInitialEnv PATH d:/servers/apache/php;C:/**
 WINDOWS/system32;C:/WINDOWS;C:**/WINDOWS/System32/Wbem;
 FcgidInitialEnv SystemRoot C:/Windows
 FcgidInitialEnv SystemDrive C:
 FcgidInitialEnv TEMP C:/WINDOWS/Temp
 FcgidInitialEnv TMP C:/WINDOWS/Temp
 FcgidInitialEnv windir C:/WINDOWS

 FcgidInitialEnv PHP_FCGI_MAX_REQUESTS 1000
 FcgidMaxRequestsPerProcess 1000
 FcgidMaxProcesses 50
 FcgidMaxRequestLen 8131072

 # added for z-push iPhone
 FcgidIOTimeout  7200
 FcgidConnectTimeout 180
 FcgidBusyTimeout 7200
 /IfModule



 Files ~ \.php$
  AddHandler fcgid-script .php
  FcgidWrapper d:/servers/apache/php/php-**cgi.exe .php
 /Files


Hi Steffen,

Are you able to try this patch?

http://people.apache.org/~trawick/restore-fcgid-2.3.7-process-count-r1.txt

This reverts one of the changes in 2.3.8 that affects how soon mod_fcgid
will create a new process when one isn't immediately available.  With this
backed out, my simple testcase uses 11-12 processes like 2.3.7 instead of
20 processes like 2.3.8.

TIA!




 On Monday 30/09/2013 at 01:36, Jeff Trawick  wrote:


 On Sun, Sep 29, 2013 at 5:14 PM, Jeff Trawick traw...@gmail.com wrote:






 On Sun, Sep 29, 2013 at 4:04 PM, Steffen i...@apachelounge.com wrote:






 Becoming dramatic here, already running over 30 processes. Running out
 of memory this way.


 All time high here is 5 processes, and while writing this mail it is
 already 34 and all 34 have an entry in the mod_status page.

 Also looks like it is not stopping/killing processes any more, have
 entries with 1784 seconds idle (FcgidIdleTimeout is default, 300)

 Going back to 2.3.7 at AL.


 On Sunday 29/09/2013 at 21:15, Steffen  wrote:



 Observe a different behavior compared to 2.3.7


 - It spawns a lot more mod_fcgid processes, looks like vhost is in
 charge (mod_fcgid only global defined here)

 - I see in Windows taskmanager and  in mod_status 5 processes and the
 error log says that the are 3 started, a mismatch.

 - Also different in mod_status page, see more then one entry for
 Process: php-cgi.exe

 With 2.3.8 splitted now :

 Total FastCGI processes: 5
 Process: php-cgi.exe  (d:/servers/apache/php/php-**cgi.exe)
 PidActiveIdleAccessesState
 46204874317Ready
 Process: php-cgi.exe  (d:/servers/apache/php/php-**cgi.exe)
 PidActiveIdleAccessesState
 38405151095Ready
 Process: php-cgi.exe  (d:/servers/apache/php/php-**cgi.exe)
 PidActiveIdleAccessesState
 65525574566Ready
 Process: php-cgi.exe  (d:/servers/apache/php/php-**cgi.exe)
 PidActiveIdleAccessesState
 320419627187Ready
 5036 2143 17192Ready


 With 2.3.7 was only one entry, like:

 Process: php-cgi.exe  (d:/servers/apache/php/php-**cgi.exe)
 PidActiveIdleAccessesState
 320419627187Ready
 5036 2143 17  �� 192Ready
 ...

 Not really trust 2.3.8 (yet), give me a few days to observe more.



 Thanks, Steffen.  I'll try to reproduce soon and see which commit
 changed that.  (Maybe 1377398?)



 So far I've only set up a simple test...  Simple perl FastCGI script, ab
 -n 20 -c 20, default main vhost configuration, this for the vhost:


 VirtualHost *:8082
 FCGIDCmdOptions /home/trawick/myhg/apache/**fcgid/apps/altinfo.pl \
InitialEnv VHOST=any \
InitialEnv PERL5LIB=/home/trawick/perl5/**lib/perl5
 /VirtualHost

 2.3.7 grows up to about 12 (vs. max 20 concurrent clients).
 2.3.8 grows up to about 20.

 I got both the fastest and slowest times for 200,000 requests using
 2.3.8.  Generally I suspect 2.3.7 is slightly faster, but I don't have a
 good overall summary.

 If you're using FcgidCmdOptions, I'd recommend using the MaxProcesses
 parameter to something that your system can handle.  Otherwise, see
 FcgidMaxProcesses and FcgidMaxProcessesPerClass.  Regardless of 2.3.7 or
 2.3.8.

 Still, for this simple scenario + configuration, 2.3.7 would have been
 better (generally not worse performance, uses 40% fewer processes).
  Different scenarios would have different results, but I think that the
 common, fat PHP processes would have bigger problems with 2.3.8 if there is
 no reasonable configured limit on the max to spawn.

 Does anyone else have time to play?















 On Sunday 29/09/2013 at 20:01, Jeff Trawick  wrote:




 Tarballs/zips are at 
 http://httpd.apache.org/dev/**dist/mod_fcgid/http

Re: [VOTE] Release mod_fcgid 2.3.8

[Steffen]

Still not going well with patch restore-fcgid-2.3.7-process-count-r1.txt

See for details http://www.apachelounge.com/status.html

I notice:

- just one process is serving.
- rest is just “hanging”  as working with no accesses and high idle time.


Back to 2.3.7


From: Jeff Trawick
Sent: Monday, September 30, 2013 2:12 PM
To: Apache HTTP Server Development List
Subject: Re: [VOTE] Release mod_fcgid 2.3.8

On Mon, Sep 30, 2013 at 5:23 AM, Steffen i...@apachelounge.com wrote:


Running at AL now 2.3.7 again, all fine again with 1-3 processes instead of 
the 30+ with 2.3.8.


The config live running here and your config for synthetic test  is 
different. And not using fat php processes and max processes is in place.


Not any mod_fcgid directives here in a vhost, only in server the commonly 
used config for php. All vhosts here serving .php, including the default 
vhost. Running server 2012.



IfModule fcgid_module
FcgidInitialEnv PHPRC d:/servers/apache/conf/
FcgidInitialEnv PATH 
d:/servers/apache/php;C:/WINDOWS/system32;C:/WINDOWS;C:/WINDOWS/System32/Wbem;

FcgidInitialEnv SystemRoot C:/Windows
FcgidInitialEnv SystemDrive C:
FcgidInitialEnv TEMP C:/WINDOWS/Temp
FcgidInitialEnv TMP C:/WINDOWS/Temp
FcgidInitialEnv windir C:/WINDOWS

FcgidInitialEnv PHP_FCGI_MAX_REQUESTS 1000
FcgidMaxRequestsPerProcess 1000
FcgidMaxProcesses 50
FcgidMaxRequestLen 8131072

# added for z-push iPhone
FcgidIOTimeout  7200
FcgidConnectTimeout 180
FcgidBusyTimeout 7200
/IfModule



Files ~ \.php$
AddHandler fcgid-script .php
FcgidWrapper d:/servers/apache/php/php-cgi.exe .php
/Files


Hi Steffen,

Are you able to try this patch?

http://people.apache.org/~trawick/restore-fcgid-2.3.7-process-count-r1.txt

This reverts one of the changes in 2.3.8 that affects how soon mod_fcgid 
will create a new process when one isn't immediately available.  With this 
backed out, my simple testcase uses 11-12 processes like 2.3.7 instead of 20 
processes like 2.3.8.


TIA!

Re: [VOTE] Release mod_fcgid 2.3.8

[Jeff Trawick]

On Mon, Sep 30, 2013 at 12:22 PM, Steffen i...@apachelounge.com wrote:

 Still not going well with patch restore-fcgid-2.3.7-process-**count-r1.txt

 See for details 
 http://www.apachelounge.com/**status.htmlhttp://www.apachelounge.com/status.html

 I notice:

 - just one process is serving.
 - rest is just “hanging”  as working with no accesses and high idle time.


 Back to 2.3.7


Thanks...  I'll see if I can encounter this issue and look for another
change to revert.



 From: Jeff Trawick
 Sent: Monday, September 30, 2013 2:12 PM
 To: Apache HTTP Server Development List
 Subject: Re: [VOTE] Release mod_fcgid 2.3.8


 On Mon, Sep 30, 2013 at 5:23 AM, Steffen i...@apachelounge.com wrote:


 Running at AL now 2.3.7 again, all fine again with 1-3 processes instead
 of the 30+ with 2.3.8.

 The config live running here and your config for synthetic test  is
 different. And not using fat php processes and max processes is in place.

 Not any mod_fcgid directives here in a vhost, only in server the commonly
 used config for php. All vhosts here serving .php, including the default
 vhost. Running server 2012.


 IfModule fcgid_module
 FcgidInitialEnv PHPRC d:/servers/apache/conf/
 FcgidInitialEnv PATH d:/servers/apache/php;C:/**
 WINDOWS/system32;C:/WINDOWS;C:**/WINDOWS/System32/Wbem;
 FcgidInitialEnv SystemRoot C:/Windows
 FcgidInitialEnv SystemDrive C:
 FcgidInitialEnv TEMP C:/WINDOWS/Temp
 FcgidInitialEnv TMP C:/WINDOWS/Temp
 FcgidInitialEnv windir C:/WINDOWS

 FcgidInitialEnv PHP_FCGI_MAX_REQUESTS 1000
 FcgidMaxRequestsPerProcess 1000
 FcgidMaxProcesses 50
 FcgidMaxRequestLen 8131072

 # added for z-push iPhone
 FcgidIOTimeout  7200
 FcgidConnectTimeout 180
 FcgidBusyTimeout 7200
 /IfModule



 Files ~ \.php$
 AddHandler fcgid-script .php
 FcgidWrapper d:/servers/apache/php/php-**cgi.exe .php
 /Files


 Hi Steffen,

 Are you able to try this patch?

 http://people.apache.org/~**trawick/restore-fcgid-2.3.7-**
 process-count-r1.txthttp://people.apache.org/~trawick/restore-fcgid-2.3.7-process-count-r1.txt

 This reverts one of the changes in 2.3.8 that affects how soon mod_fcgid
 will create a new process when one isn't immediately available.  With this
 backed out, my simple testcase uses 11-12 processes like 2.3.7 instead of
 20 processes like 2.3.8.

 TIA!




-- 
Born in Roswell... married an alien...
http://emptyhammock.com/

Re: [PATCH 49220] mod_fcgid - restrict arbitrary command execution from .htaccess files

[Jeff Trawick]

On Fri, Sep 27, 2013 at 1:56 PM, Benjamin Coddington bcodd...@uvm.eduwrote:

 On Sep 27, 2013, at 1:50 PM, Benjamin Coddington bcodd...@uvm.edu wrote:

  since I'll now need to generate a large number of
  AllowOverrideList configurations in order to implement this across our
  hosting - which requires I walk our modules to find all the directives in
  FileInfo and explicitly allow them to disable these mod_fcgid directives.

 Or... just use this:

 httpd -L | grep -B3 FileInfo | grep -B2 .htaccess | egrep '^\w'

 Ben


As you have a solution with httpd 2.4.x, I won't think any more about this
for fcgid 2.3.next.  (And as time goes on it is less useful to others
anyway.)

[VOTE] Release mod_fcgid 2.3.8

[Jeff Trawick]

Tarballs/zips are at http://httpd.apache.org/dev/dist/mod_fcgid/

Shortcut to changes:
http://httpd.apache.org/dev/dist/mod_fcgid/CHANGES-FCGID

+/-1
[  ] Release mod_fcgid 2.3.8 as GA

I'll hold the vote open for 72 hours unless something out of the ordinary
occurs.

Thanks in advance for testing!

-- 
Born in Roswell... married an alien...
http://emptyhammock.com/

Re: [VOTE] Release mod_fcgid 2.3.8

[Mario Brandt]

On 29 September 2013 20:00, Jeff Trawick traw...@gmail.com wrote:
 Tarballs/zips are at http://httpd.apache.org/dev/dist/mod_fcgid/

 Shortcut to changes:
 http://httpd.apache.org/dev/dist/mod_fcgid/CHANGES-FCGID

 +/-1
[ +1 ] Release mod_fcgid 2.3.8 as GA

tested on Debian 7 with PHP 5.4 (distro) and 5.5 (dotdeb)


 I'll hold the vote open for 72 hours unless something out of the ordinary
 occurs.

 Thanks in advance for testing!

 --
 Born in Roswell... married an alien...
 http://emptyhammock.com/

Re: [VOTE] Release mod_fcgid 2.3.8

[Steffen]

Observe a different behavior compared to 2.3.7


- It spawns a lot more mod_fcgid processes, looks like vhost is in 
charge (mod_fcgid only global defined here)


- I see in Windows taskmanager and  in mod_status 5 processes and the 
error log says that the are 3 started, a mismatch.


- Also different in mod_status page, see more then one entry for 
Process: php-cgi.exe


With 2.3.8 splitted now :

Total FastCGI processes: 5
Process: php-cgi.exe  (d:/servers/apache/php/php-cgi.exe)
PidActiveIdleAccessesState
46204874317Ready
Process: php-cgi.exe  (d:/servers/apache/php/php-cgi.exe)
PidActiveIdleAccessesState
38405151095Ready
Process: php-cgi.exe  (d:/servers/apache/php/php-cgi.exe)
PidActiveIdleAccessesState
65525574566Ready
Process: php-cgi.exe  (d:/servers/apache/php/php-cgi.exe)
PidActiveIdleAccessesState
320419627187Ready
5036214317192Ready


With 2.3.7 was only one entry, like:

Process: php-cgi.exe  (d:/servers/apache/php/php-cgi.exe)
PidActiveIdleAccessesState
320419627187Ready
5036214317192Ready
...

Not really trust 2.3.8 (yet), give me a few days to observe more.




On Sunday 29/09/2013 at 20:01, Jeff Trawick  wrote:


Tarballs/zips are at http://httpd.apache.org/dev/dist/mod_fcgid/

Shortcut to changes: 
http://httpd.apache.org/dev/dist/mod_fcgid/CHANGES-FCGID



+/-1
[  ] Release mod_fcgid 2.3.8 as GA

I'll hold the vote open for 72 hours unless something out of the 
ordinary occurs.


Thanks in advance for testing!
--
Born in Roswell... married an alien...
http://emptyhammock.com/

Re: [VOTE] Release mod_fcgid 2.3.8

[Steffen]

Becoming dramatic here, already running over 30 processes. Running out 
of memory this way.



All time high here is 5 processes, and while writing this mail it is 
already 34 and all 34 have an entry in the mod_status page.


Also looks like it is not stopping/killing processes any more, have 
entries with 1784 seconds idle (FcgidIdleTimeout is default, 300)


Going back to 2.3.7 at AL.


On Sunday 29/09/2013 at 21:15, Steffen  wrote:


Observe a different behavior compared to 2.3.7


- It spawns a lot more mod_fcgid processes, looks like vhost is in 
charge (mod_fcgid only global defined here)


- I see in Windows taskmanager and  in mod_status 5 processes and the 
error log says that the are 3 started, a mismatch.


- Also different in mod_status page, see more then one entry for 
Process: php-cgi.exe


With 2.3.8 splitted now :

Total FastCGI processes: 5
Process: php-cgi.exe  (d:/servers/apache/php/php-cgi.exe)
PidActiveIdleAccessesState
46204874317Ready
Process: php-cgi.exe  (d:/servers/apache/php/php-cgi.exe)
PidActiveIdleAccessesState
38405151095Ready
Process: php-cgi.exe  (d:/servers/apache/php/php-cgi.exe)
PidActiveIdleAccessesState
65525574566Ready
Process: php-cgi.exe  (d:/servers/apache/php/php-cgi.exe)
PidActiveIdleAccessesState
320419627187Ready
5036214317192Ready


With 2.3.7 was only one entry, like:

Process: php-cgi.exe  (d:/servers/apache/php/php-cgi.exe)
PidActiveIdleAccessesState
320419627187Ready
5036214317192Ready
...

Not really trust 2.3.8 (yet), give me a few days to observe more.




On Sunday 29/09/2013 at 20:01, Jeff Trawick  wrote:



Tarballs/zips are at http://httpd.apache.org/dev/dist/mod_fcgid/

Shortcut to changes: 
http://httpd.apache.org/dev/dist/mod_fcgid/CHANGES-FCGID



+/-1
[  ] Release mod_fcgid 2.3.8 as GA

I'll hold the vote open for 72 hours unless something out of the  
ordinary occurs.


Thanks in advance for testing!
--
Born in Roswell... married an alien...
http://emptyhammock.com/

Re: [VOTE] Release mod_fcgid 2.3.8

[Jeff Trawick]

On Sun, Sep 29, 2013 at 4:04 PM, Steffen i...@apachelounge.com wrote:


 Becoming dramatic here, already running over 30 processes. Running out of
 memory this way.


 All time high here is 5 processes, and while writing this mail it is
 already 34 and all 34 have an entry in the mod_status page.

 Also looks like it is not stopping/killing processes any more, have
 entries with 1784 seconds idle (FcgidIdleTimeout is default, 300)

 Going back to 2.3.7 at AL.


 On Sunday 29/09/2013 at 21:15, Steffen  wrote:


 Observe a different behavior compared to 2.3.7


 - It spawns a lot more mod_fcgid processes, looks like vhost is in charge
 (mod_fcgid only global defined here)

 - I see in Windows taskmanager and  in mod_status 5 processes and the
 error log says that the are 3 started, a mismatch.

 - Also different in mod_status page, see more then one entry for Process:
 php-cgi.exe

 With 2.3.8 splitted now :

 Total FastCGI processes: 5
 Process: php-cgi.exe  (d:/servers/apache/php/php-**cgi.exe)
 PidActiveIdleAccessesState
 46204874317Ready
 Process: php-cgi.exe  (d:/servers/apache/php/php-**cgi.exe)
 PidActiveIdleAccessesState
 38405151095Ready
 Process: php-cgi.exe  (d:/servers/apache/php/php-**cgi.exe)
 PidActiveIdleAccessesState
 65525574566Ready
 Process: php-cgi.exe  (d:/servers/apache/php/php-**cgi.exe)
 PidActiveIdleAccessesState
 320419627187Ready
 5036 2143 17192Ready


 With 2.3.7 was only one entry, like:

 Process: php-cgi.exe  (d:/servers/apache/php/php-**cgi.exe)
 PidActiveIdleAccessesState
 320419627187Ready
 5036 2143 17192Ready
 ...

 Not really trust 2.3.8 (yet), give me a few days to observe more.


Thanks, Steffen.  I'll try to reproduce soon and see which commit changed
that.  (Maybe 1377398?)






 On Sunday 29/09/2013 at 20:01, Jeff Trawick  wrote:



 Tarballs/zips are at 
 http://httpd.apache.org/dev/**dist/mod_fcgid/http://httpd.apache.org/dev/dist/mod_fcgid/

 Shortcut to changes: http://httpd.apache.org/dev/**
 dist/mod_fcgid/CHANGES-FCGIDhttp://httpd.apache.org/dev/dist/mod_fcgid/CHANGES-FCGID


 +/-1
 [  ] Release mod_fcgid 2.3.8 as GA

 I'll hold the vote open for 72 hours unless something out of the
  ordinary occurs.

 Thanks in advance for testing!
 --
 Born in Roswell... married an alien...
 http://emptyhammock.com/










-- 
Born in Roswell... married an alien...
http://emptyhammock.com/

Re: [VOTE] Release mod_fcgid 2.3.8

[Jeff Trawick]

On Sun, Sep 29, 2013 at 5:14 PM, Jeff Trawick traw...@gmail.com wrote:

 On Sun, Sep 29, 2013 at 4:04 PM, Steffen i...@apachelounge.com wrote:


 Becoming dramatic here, already running over 30 processes. Running out of
 memory this way.


 All time high here is 5 processes, and while writing this mail it is
 already 34 and all 34 have an entry in the mod_status page.

 Also looks like it is not stopping/killing processes any more, have
 entries with 1784 seconds idle (FcgidIdleTimeout is default, 300)

 Going back to 2.3.7 at AL.


 On Sunday 29/09/2013 at 21:15, Steffen  wrote:


 Observe a different behavior compared to 2.3.7


 - It spawns a lot more mod_fcgid processes, looks like vhost is in
 charge (mod_fcgid only global defined here)

 - I see in Windows taskmanager and  in mod_status 5 processes and the
 error log says that the are 3 started, a mismatch.

 - Also different in mod_status page, see more then one entry for
 Process: php-cgi.exe

 With 2.3.8 splitted now :

 Total FastCGI processes: 5
 Process: php-cgi.exe  (d:/servers/apache/php/php-**cgi.exe)
 PidActiveIdleAccessesState
 46204874317Ready
 Process: php-cgi.exe  (d:/servers/apache/php/php-**cgi.exe)
 PidActiveIdleAccessesState
 38405151095Ready
 Process: php-cgi.exe  (d:/servers/apache/php/php-**cgi.exe)
 PidActiveIdleAccessesState
 65525574566Ready
 Process: php-cgi.exe  (d:/servers/apache/php/php-**cgi.exe)
 PidActiveIdleAccessesState
 320419627187Ready
 5036 2143 17192Ready


 With 2.3.7 was only one entry, like:

 Process: php-cgi.exe  (d:/servers/apache/php/php-**cgi.exe)
 PidActiveIdleAccessesState
 320419627187Ready
 5036 2143 17192Ready
 ...

 Not really trust 2.3.8 (yet), give me a few days to observe more.


 Thanks, Steffen.  I'll try to reproduce soon and see which commit changed
 that.  (Maybe 1377398?)


So far I've only set up a simple test...  Simple perl FastCGI script, ab -n
20 -c 20, default main vhost configuration, this for the vhost:

VirtualHost *:8082
FCGIDCmdOptions /home/trawick/myhg/apache/fcgid/apps/altinfo.pl \
InitialEnv VHOST=any \
InitialEnv PERL5LIB=/home/trawick/perl5/lib/perl5
/VirtualHost

2.3.7 grows up to about 12 (vs. max 20 concurrent clients).
2.3.8 grows up to about 20.

I got both the fastest and slowest times for 200,000 requests using 2.3.8.
 Generally I suspect 2.3.7 is slightly faster, but I don't have a good
overall summary.

If you're using FcgidCmdOptions, I'd recommend using the MaxProcesses
parameter to something that your system can handle.  Otherwise, see
FcgidMaxProcesses and FcgidMaxProcessesPerClass.  Regardless of 2.3.7 or
2.3.8.

Still, for this simple scenario + configuration, 2.3.7 would have been
better (generally not worse performance, uses 40% fewer processes).
 Different scenarios would have different results, but I think that the
common, fat PHP processes would have bigger problems with 2.3.8 if there is
no reasonable configured limit on the max to spawn.

Does anyone else have time to play?







 On Sunday 29/09/2013 at 20:01, Jeff Trawick  wrote:



 Tarballs/zips are at 
 http://httpd.apache.org/dev/**dist/mod_fcgid/http://httpd.apache.org/dev/dist/mod_fcgid/

 Shortcut to changes: http://httpd.apache.org/dev/**
 dist/mod_fcgid/CHANGES-FCGIDhttp://httpd.apache.org/dev/dist/mod_fcgid/CHANGES-FCGID


 +/-1
 [  ] Release mod_fcgid 2.3.8 as GA

 I'll hold the vote open for 72 hours unless something out of the
  ordinary occurs.

 Thanks in advance for testing!
 --
 Born in Roswell... married an alien...
 http://emptyhammock.com/










 --
 Born in Roswell... married an alien...
 http://emptyhammock.com/




-- 
Born in Roswell... married an alien...
http://emptyhammock.com/

Re: [VOTE] Release mod_fcgid 2.3.8

[Mario Brandt]

Hi Jeff,
at least I can say from another build that I have made that r1500307
does not have that issue on my box Debian Box. There are the number of
processes that I see in htop and on the server status page the same.
So the issue might happen in a later revision.

Hope that helps. (May some other can try r1500307 on Windows, too).

On 29 September 2013 23:14, Jeff Trawick traw...@gmail.com wrote:
 On Sun, Sep 29, 2013 at 4:04 PM, Steffen i...@apachelounge.com wrote:


 Becoming dramatic here, already running over 30 processes. Running out of
 memory this way.


 All time high here is 5 processes, and while writing this mail it is
 already 34 and all 34 have an entry in the mod_status page.

 Also looks like it is not stopping/killing processes any more, have
 entries with 1784 seconds idle (FcgidIdleTimeout is default, 300)

 Going back to 2.3.7 at AL.


 On Sunday 29/09/2013 at 21:15, Steffen  wrote:


 Observe a different behavior compared to 2.3.7


 - It spawns a lot more mod_fcgid processes, looks like vhost is in charge
 (mod_fcgid only global defined here)

 - I see in Windows taskmanager and  in mod_status 5 processes and the
 error log says that the are 3 started, a mismatch.

 - Also different in mod_status page, see more then one entry for Process:
 php-cgi.exe

 With 2.3.8 splitted now :

 Total FastCGI processes: 5
 Process: php-cgi.exe  (d:/servers/apache/php/php-cgi.exe)
 PidActiveIdleAccessesState
 46204874317Ready
 Process: php-cgi.exe  (d:/servers/apache/php/php-cgi.exe)
 PidActiveIdleAccessesState
 38405151095Ready
 Process: php-cgi.exe  (d:/servers/apache/php/php-cgi.exe)
 PidActiveIdleAccessesState
 65525574566Ready
 Process: php-cgi.exe  (d:/servers/apache/php/php-cgi.exe)
 PidActiveIdleAccessesState
 320419627187Ready
 5036 2143 17192Ready


 With 2.3.7 was only one entry, like:

 Process: php-cgi.exe  (d:/servers/apache/php/php-cgi.exe)
 PidActiveIdleAccessesState
 320419627187Ready
 5036 2143 17192Ready
 ...

 Not really trust 2.3.8 (yet), give me a few days to observe more.


 Thanks, Steffen.  I'll try to reproduce soon and see which commit changed
 that.  (Maybe 1377398?)






 On Sunday 29/09/2013 at 20:01, Jeff Trawick  wrote:



 Tarballs/zips are at http://httpd.apache.org/dev/dist/mod_fcgid/

 Shortcut to changes:
 http://httpd.apache.org/dev/dist/mod_fcgid/CHANGES-FCGID


 +/-1
 [  ] Release mod_fcgid 2.3.8 as GA

 I'll hold the vote open for 72 hours unless something out of the
 ordinary occurs.

 Thanks in advance for testing!
 --
 Born in Roswell... married an alien...
 http://emptyhammock.com/










 --
 Born in Roswell... married an alien...
 http://emptyhammock.com/

Re: [PATCH 49220] mod_fcgid - restrict arbitrary command execution from .htaccess files

[Jeff Trawick]

On Fri, Sep 20, 2013 at 4:31 PM, Benjamin Coddington bcodd...@uvm.eduwrote:

 Hello everyone,

 We're looking at moving our shared hosting execution behind mod_fcgid and
 suexec, but we need to continue to allow our users .htaccess 'Files'
 overrides.  The current mod_fcgid allows users to execute arbitrary
 commands by configuring the FcgidAccessChecker, FcgidAuthenticator,
 FcgidAuthorizer, and FcgidWrapper directives within .htaccess files.

  - https://issues.apache.org/bugzilla/show_bug.cgi?id=49220

 I've approached a fix by creating a directive that would disable the
 application of those directives within .htaccess files if set; that patch
 has been submitted to the httpd bug 49220.

 You might shrewdly wonder how can this matter - this is cgi after all,
 we're just going to try to exec the resulting file!, but we're able to get
 away from that by disabling ExecCGI globally and setting it per-request in
 separate module which also ensures the request is mapped to our specific
 FcgidWrapper.

 I see mod_fcgid 2.3.8 is closing in a few days; any chance to sneak this
 in?  Thanks for your time and consideration.

 Ben


I'd like to see this aligned with 2.4's AllowOverrideList as much as
practical, but AllowOverrideList is more flexible and I haven't yet looked
at what changes to the patch would be necessary.  The feature should be
disabled when building for 2.4/trunk since those server versions already
have an appropriate feature.  It would be nice if the only change when
moving between server versions is
FcgidAllowOverrideList-AllowOverrideList.

I'll look more in the next day.

-- 
Born in Roswell... married an alien...
http://emptyhammock.com/

Re: [PATCH 49220] mod_fcgid - restrict arbitrary command execution from .htaccess files

[Benjamin Coddington]

On Sep 27, 2013, at 8:41 AM, Jeff Trawick traw...@gmail.com wrote:

 On Fri, Sep 20, 2013 at 4:31 PM, Benjamin Coddington bcodd...@uvm.eduwrote:
 
 Hello everyone,
 
 We're looking at moving our shared hosting execution behind mod_fcgid and
 suexec, but we need to continue to allow our users .htaccess 'Files'
 overrides.  The current mod_fcgid allows users to execute arbitrary
 commands by configuring the FcgidAccessChecker, FcgidAuthenticator,
 FcgidAuthorizer, and FcgidWrapper directives within .htaccess files.
 
 - https://issues.apache.org/bugzilla/show_bug.cgi?id=49220
 
 I've approached a fix by creating a directive that would disable the
 application of those directives within .htaccess files if set; that patch
 has been submitted to the httpd bug 49220.
 
 You might shrewdly wonder how can this matter - this is cgi after all,
 we're just going to try to exec the resulting file!, but we're able to get
 away from that by disabling ExecCGI globally and setting it per-request in
 separate module which also ensures the request is mapped to our specific
 FcgidWrapper.
 
 I see mod_fcgid 2.3.8 is closing in a few days; any chance to sneak this
 in?  Thanks for your time and consideration.
 
 Ben
 
 
 I'd like to see this aligned with 2.4's AllowOverrideList as much as
 practical, but AllowOverrideList is more flexible and I haven't yet looked
 at what changes to the patch would be necessary.  The feature should be
 disabled when building for 2.4/trunk since those server versions already
 have an appropriate feature.  It would be nice if the only change when
 moving between server versions is
 FcgidAllowOverrideList-AllowOverrideList.

After your comments, I've looked closer at using AllowOverrideList to
accomplish the same thing in configuration alone.  I realized that when I
initially tested this I had overlooked an 'AllowOverride FileInfo' in my
config.

So, while this approach may help the listed issue, it's not something we
require on 2.4.  It would be nice to have the inverse of AllowOverrideList
(RestrictOverrideList?) that would explicitly deny directives allowed by the
AllowOverride groups, since I'll now need to generate a large number of
AllowOverrideList configurations in order to implement this across our
hosting - which requires I walk our modules to find all the directives in
FileInfo and explicitly allow them to disable these mod_fcgid directives.

Ben

Re: [PATCH 49220] mod_fcgid - restrict arbitrary command execution from .htaccess files

[Benjamin Coddington]

On Sep 27, 2013, at 1:50 PM, Benjamin Coddington bcodd...@uvm.edu wrote:

 since I'll now need to generate a large number of
 AllowOverrideList configurations in order to implement this across our
 hosting - which requires I walk our modules to find all the directives in
 FileInfo and explicitly allow them to disable these mod_fcgid directives.

Or... just use this:

httpd -L | grep -B3 FileInfo | grep -B2 .htaccess | egrep '^\w'

Ben

mod_fcgid: consolidate command and vhost comparison

[Andriy Gapon]

We would like to propose the attached patch.
It is inspired by the several iterations of changing the respective checks.

use macros for the code that checks for command and virtual host sameness

That code consists of several lines which were duplicated in several
places.  Now the code is slightly more compact and all the logic is
in a single place.

We use macros as opposed to inline functions because we use them with
different argument types.  The only requirement on the types is that
the field names must follow a common convention.

Possibly the next logical step could be to introduce a new struct that would
hold all the ID fields and then use that struct as a member in all other structs
that currently keep the ID information as an assortment of the fields.
In other words, to make the following public (and with a better name):
+struct last_id {
+apr_ino_t inode;
+apr_dev_t deviceid;
+const char *cmdline;
+gid_t gid;
+uid_t uid;
+int vhost_id;
+} last_id;
-- 
Andriy Gapon
commit 60b1c2d2662da6ecc04c03b99c485292f4c3df50
Author: Andriy Gapon andriy.ga...@hybridcluster.com
Date:   Mon Sep 2 14:54:11 2013 +0300

use macros for the code that checks for command and virtual host sameness

That code consists of several lines which were duplicated in several
places.  Now the code is slightly more compact and all the logic is
in a single place.

We use macros as opposed to inline functions because we use them with
different argument types.  The only requirement on the types is that
the field names must follow a common convention.

diff --git a/modules/fcgid/fcgid_bridge.c b/modules/fcgid/fcgid_bridge.c
index f863008..adf355b 100644
--- a/modules/fcgid/fcgid_bridge.c
+++ b/modules/fcgid/fcgid_bridge.c
@@ -56,11 +56,8 @@ static fcgid_procnode *apply_free_procnode(request_rec *r,
 while (current_node != proc_table) {
 next_node = proc_table[current_node-next_index];
 
-if (current_node-inode == inode
- current_node-deviceid == deviceid
- !strcmp(current_node-cmdline, cmdline)
- current_node-vhost_id == command-vhost_id
- current_node-uid == uid  current_node-gid == gid) {
+if (IS_SAME_COMMAND(current_node, command)
+ IS_SAME_VHOST(current_node, command)) {
 /* Unlink from idle list */
 previous_node-next_index = current_node-next_index;
 
@@ -136,12 +133,8 @@ static int count_busy_processes(request_rec *r, 
fcgid_command *command)
 previous_node = busy_list_header;
 current_node = proc_table[previous_node-next_index];
 while (current_node != proc_table) {
-if (current_node-inode == command-inode
- current_node-deviceid == command-deviceid
- !strcmp(current_node-cmdline, command-cmdline)
- current_node-vhost_id == command-vhost_id
- current_node-uid == command-uid
- current_node-gid == command-gid) {
+if (IS_SAME_COMMAND(current_node, command)
+ IS_SAME_VHOST(current_node, command)) {
 result++;
 }
 next_node = proc_table[current_node-next_index];
diff --git a/modules/fcgid/fcgid_global.h b/modules/fcgid/fcgid_global.h
index d52a2fb..1ae0745 100644
--- a/modules/fcgid/fcgid_global.h
+++ b/modules/fcgid/fcgid_global.h
@@ -57,4 +57,14 @@ APLOG_USE_MODULE(fcgid);
 
 #define fcgid_min(a,b)(((a)  (b)) ? (a) : (b))
 
+#define IS_SAME_COMMAND(x, y)  \
+((x)-inode == (y)-inode  \
+  (x)-deviceid == (y)-deviceid \
+  !strcmp((x)-cmdline, (y)-cmdline)\
+  (x)-uid == (y)-uid   \
+  (x)-gid == (y)-gid)
+
+#define IS_SAME_VHOST(x, y)\
+((x)-vhost_id == (y)-vhost_id)
+
 #endif
diff --git a/modules/fcgid/fcgid_spawn_ctl.c b/modules/fcgid/fcgid_spawn_ctl.c
index 2d0b39c..1b561dc 100644
--- a/modules/fcgid/fcgid_spawn_ctl.c
+++ b/modules/fcgid/fcgid_spawn_ctl.c
@@ -58,12 +58,8 @@ register_life_death(server_rec * main_server,
 previous_node = g_stat_list_header;
 for (current_node = previous_node;
  current_node != NULL; current_node = current_node-next) {
-if (current_node-inode == procnode-inode
- current_node-deviceid == procnode-deviceid
- !strcmp(current_node-cmdline, procnode-cmdline)
- current_node-vhost_id == procnode-vhost_id
- current_node-uid == procnode-uid
- current_node-gid == procnode-gid)
+if (IS_SAME_COMMAND(current_node, procnode)
+ IS_SAME_VHOST(current_node, procnode))
 break;
 previous_node = current_node;
 }
@@ -175,12 +171,8 @@ int is_spawn_allowed(server_rec * main_server, 
fcgid_command * command)
 /* Can I find the node base on inode, device id and cmdline? */
 for (current_node = g_stat_list_header;
  

Re: FYI... Planning to TR mod_fcgid 2.3.8 in 7-10 days

[Chris Darroch]

+1 with many thanks,

Chris.

--
GPG Key ID: 088335A9
GPG Key Fingerprint: 86CD 3297 7493 75BC F820  6715 F54F E648 0883 35A9

Re: unreaped zombie children of mod_fcgid

[Andriy Gapon]

on 11/09/2013 09:11 Andriy Gapon said the following:
 I've been thinking about this problem and the only theory that I have got so 
 far
 is that perhaps an owner httpd process could terminate ungracefully (e.g.
 crash).  In that case the pool cleanup would never be run.  That's OK for
 process local resources like memory or file descriptors, which would be freed 
 by
 OS because the process dies anyway.  But that's not OK for external resources
 like other processes.
 In other words, if an httpd process marks an fcgid process as busy and then
 suddenly dies, then there is nobody to move the fcgid process back to the 
 idle list.

Just an idea: perhaps scan_busylist should move a process to a different list
(the error list?) after treating it with proc_kill_force?
Currently the processes on the busy list are never checked for being a zombie.
With the proposed change they should be correctly reaped after all the waiting
and killing.

-- 
Andriy Gapon

Re: mod_fcgid and mpm-itk

[William A. Rowe Jr.]

On Sun, 28 Jul 2013 15:17:41 +0200
AZ 9901 az9...@gmail.com wrote:

 Hello all,
 
 I run into troubles using mod_fcgid (for PHP) and mpm-itk together.
 
 As soon as I add the ITK directive AssignUserID to my vhost,
 specifying a different user than the global Apache one, mod_fcgid
 fails returning the following error : [Sun Jul 28 15:09:37 2013]
 [emerg] [client x.x.x.x] (13)Permission denied: mod_fcgid: can't lock
 process table in pid 1973
 
 I noted that the socket file is created with permissions 700.
 So, as a test, I tried the following, in fcgid_proc_unix.c, I
 replaced : rv = apr_file_perms_set(unix_addr.sun_path,
 
 APR_FPROT_UREAD|APR_FPROT_UWRITE|APR_FPROT_UEXECUTE);
 by :
 rv = apr_file_perms_set(unix_addr.sun_path,
 
 APR_FPROT_UREAD|APR_FPROT_UWRITE|APR_FPROT_UEXECUTE|APR_FPROT_GREAD|APR_FPROT_GWRITE|APR_FPROT_GEXECUTE|APR_FPROT_WREAD|APR_FPROT_WWRITE|APR_FPROT_WEXECUTE);
 
 The socket is then created with 777 permissions, but I does not solve
 the issue.
 
 Certainly that the issue is somewhere else, perhaps Apache tries to
 instance mod_fcgid with global Apache user instead of ITK defined
 user.
 
 Could you help me with that ?

You might also try any mpm-itk user list since the behavior is going
to be mpm-specific.  And please, fill us in if you learn more!

mod_fcgid and mpm-itk

[AZ 9901]

Hello all,

I run into troubles using mod_fcgid (for PHP) and mpm-itk together.

As soon as I add the ITK directive AssignUserID to my vhost, specifying a 
different user than the global Apache one, mod_fcgid fails returning the 
following error :
[Sun Jul 28 15:09:37 2013] [emerg] [client x.x.x.x] (13)Permission denied: 
mod_fcgid: can't lock process table in pid 1973

I noted that the socket file is created with permissions 700.
So, as a test, I tried the following, in fcgid_proc_unix.c, I replaced :
rv = apr_file_perms_set(unix_addr.sun_path,

APR_FPROT_UREAD|APR_FPROT_UWRITE|APR_FPROT_UEXECUTE);
by :
rv = apr_file_perms_set(unix_addr.sun_path,

APR_FPROT_UREAD|APR_FPROT_UWRITE|APR_FPROT_UEXECUTE|APR_FPROT_GREAD|APR_FPROT_GWRITE|APR_FPROT_GEXECUTE|APR_FPROT_WREAD|APR_FPROT_WWRITE|APR_FPROT_WEXECUTE);

The socket is then created with 777 permissions, but I does not solve the issue.

Certainly that the issue is somewhere else, perhaps Apache tries to instance 
mod_fcgid with global Apache user instead of ITK defined user.

Could you help me with that ?
Perhaps a patch already exists ?

Thank you very much !

Best regards,

Ben

Re: Anyone interested in a patch to mod_fcgid(with pay)

[William A. Rowe Jr.]

On Sat, 20 Jul 2013 10:39:20 +0800 (CST)
Pqf 潘庆峰 p...@mailtech.cn wrote:

 Hi, guys
A company need a TCP/IP patch of mod_fcgid or alternative, and
 will pay for it, anyone interested? I really like to take it but I
 don't have too much time... Anyone interested please reply to me and
 I will forward the email address of them.
 
 ...
 Neti only listens on TCP/IP socket, it assumes both an authorizer
 role and a responder role in the Fast CGI request. Here's the 3
 candidate Apache modules to interface Neti:
 ...

 2. Mod_proxy_fcgi: this module supports TCP socket, it can connect to
 Neti, but it doesn't support authorizer role. So in the first FCGI
 request, it forwards the request to Neti as a responder instead of an
 authorizer, Neti cannot simply let it through without properly
 authorizing it first, thus the request fails;
 
 3. Mod_fcgid: this module supports authorizer role while doesn't
 support TCP connection. We cannot confirm its authorizer role since
 it doesn't even connect to Neti due to lack of TCP;
 
 So our choice is between either adding authorizer role to
 mod_proxy_fcgi or adding TCP/IP to mod_fcgid. 
 
 We’re really willing to pay to have this project done, I mean either
 adding proxy to mod_fcgid or adding authorizer to mod_proxy_fcgi. Are
 you willing to work on this with reward or do you know anyone else
 who’s interested in doing so with pay, for example, author of
 mod_proxy_fcgi? (I cannot find his name)

No cycles myself at this instant, but it seems overtime to break apart
the mod_fcgid process-control so that it can then stack on top of a
single mod_proxy_fcgi communications pipe, dispatched through different
fcgi-stream methods (including child process stdio), including the
various authn/authz roles.  It would take more time to refactor it in
this way, but would get rid of any discrepancies between proxy_fcgi
and fcgid and serve as a good excuse to draw the remaining mod_fcgid
development back into trunk/, now that fcgid is generally sufficient
for 2.2 users.

Re: Re: Anyone interested in a patch to mod_fcgid(with pay)

[Pqf 潘庆峰]

Yes, split process control from mod_fcgid, merge proxy_fcgi(with 
load balance) and mod_fcgid(with authXX support) is a good idea,
admins can use httpd as process manager, or 3rd party process managers as they 
like.
But don't just make a patch to make mod_fcgid support TCP/IP, it's ugly...



 
  Hi, guys
 A company need a TCP/IP patch of mod_fcgid or alternative, and
  will pay for it, anyone interested? I really like to take it but I
  don't have too much time... Anyone interested please reply to me and
  I will forward the email address of them.
  
  ...
  Neti only listens on TCP/IP socket, it assumes both an authorizer
  role and a responder role in the Fast CGI request. Here's the 3
  candidate Apache modules to interface Neti:
  ...
 
  2. Mod_proxy_fcgi: this module supports TCP socket, it can connect to
  Neti, but it doesn't support authorizer role. So in the first FCGI
  request, it forwards the request to Neti as a responder instead of an
  authorizer, Neti cannot simply let it through without properly
  authorizing it first, thus the request fails;
  
  3. Mod_fcgid: this module supports authorizer role while doesn't
  support TCP connection. We cannot confirm its authorizer role since
  it doesn't even connect to Neti due to lack of TCP;
  
  So our choice is between either adding authorizer role to
  mod_proxy_fcgi or adding TCP/IP to mod_fcgid. 
  
  We’re really willing to pay to have this project done, I mean either
  adding proxy to mod_fcgid or adding authorizer to mod_proxy_fcgi. Are
  you willing to work on this with reward or do you know anyone else
  who’s interested in doing so with pay, for example, author of
  mod_proxy_fcgi? (I cannot find his name)
 
 No cycles myself at this instant, but it seems overtime to break apart
 the mod_fcgid process-control so that it can then stack on top of a
 single mod_proxy_fcgi communications pipe, dispatched through different
 fcgi-stream methods (including child process stdio), including the
 various authn/authz roles.  It would take more time to refactor it in
 this way, but would get rid of any discrepancies between proxy_fcgi
 and fcgid and serve as a good excuse to draw the remaining mod_fcgid
 development back into trunk/, now that fcgid is generally sufficient
 for 2.2 users.
 
 
 

Re: Anyone interested in a patch to mod_fcgid(with pay)

[William A. Rowe Jr.]

On Tue, 23 Jul 2013 09:46:58 +0800 (CST)
Pqf 潘庆峰 p...@mailtech.cn wrote:

 Yes, split process control from mod_fcgid, merge proxy_fcgi(with 
 load balance) and mod_fcgid(with authXX support) is a good idea,
 admins can use httpd as process manager, or 3rd party process
 managers as they like. But don't just make a patch to make mod_fcgid
 support TCP/IP, it's ugly...

Nice synopsis :)

Anyone interested in a patch to mod_fcgid(with pay)

[Pqf 潘庆峰]

Hi, guys
   A company need a TCP/IP patch of mod_fcgid or alternative, and will pay 
for it, anyone interested? I really like to take it but I don't have too much 
time...
   Anyone interested please reply to me and I will forward the email address of 
them.


Our company is hosting a website with a lot of users. Each user needs login to 
access our server. We have an in-house FCGI program responsible for 
authenticate user's credentials. The FCGI program is called “Neti”;. Right now 
Neti works fine with Zeus server, however we want to switch to Apache server.

Neti only listens on TCP/IP socket, it assumes both an authorizer role and a 
responder role in the Fast CGI request. Here's the 3 candidate Apache modules 
to interface Neti:

1. Mod_fastcgi: this module has NOT been updated since 2007. It will break 
randomly, (not only with Neti, but with some other FCGIs), so we're not 
planning to pursue it;

2. Mod_proxy_fcgi: this module supports TCP socket, it can connect to Neti, but 
it doesn't support authorizer role. So in the first FCGI request, it forwards 
the request to Neti as a responder instead of an authorizer, Neti cannot simply 
let it through without properly authorizing it first, thus the request fails;

3. Mod_fcgid: this module supports authorizer role while doesn't support TCP 
connection. We cannot confirm its authorizer role since it doesn't even connect 
to Neti due to lack of TCP;

So our choice is between either adding authorizer role to mod_proxy_fcgi or 
adding TCP/IP to mod_fcgid. 

We’re really willing to pay to have this project done, I mean either adding 
proxy to mod_fcgid or adding authorizer to mod_proxy_fcgi. Are you willing to 
work on this with reward or do you know anyone else who’s interested in doing 
so with pay, for example, author of mod_proxy_fcgi? (I cannot find his name)


Ryan

apache2-mpm-itk different php versions per user php.ini / mod_fcgid

[m...@unimx.de]

Hello Group,

the problem using ITK+mod_fcgid is the fcgid socket. the socket is 
owned by wwwuser (wwwrun / www-data) of apache.

ITK set vhost to user1, so fcgid says:

Permission denied: mod_fcgid: can't lock process table in pid

socket dir:

srwx-- 1 wwwrun wwwrun 0 21. Feb 14:11 11585.0
srwx-- 1 wwwrun wwwrun 0 21. Feb 13:12 2666.0
srwx-- 1 wwwrun wwwrun 0 21. Feb 13:12 2666.1
srwx-- 1 wwwrun wwwrun 0 21. Feb 13:17 3083.0
srwx-- 1 wwwrun wwwrun 0 21. Feb 13:25 3938.0

The way using fcgi is, we have different php-versions and per user 
php.ini
The way we use ITK is, we want to safe homedirs of all user not have 
to be readable by wwwrun (f.e. remember symlink problem)


So we think, we must patch mod_fcgid (source 
http://httpd.apache.org/mod_fcgid/)
May like modwsgi: 
http://code.google.com/p/modwsgi/issues/detail?id=187


if (!geteuid()) {
+#if defined(MPM_ITK)
+if (chown(process-socket, process-uid, -1)  0) {
+#else
   if (chown(process-socket, ap_unixd_config.user_id, -1)  0) {
+#endif
   ap_log_error(APLOG_MARK, WSGI_LOG_ALERT(errno), 
wsgi_server,
mod_wsgi (pid=%d): Couldn't change owner of 
unix 

domain socket '%s'., getpid(),

Thus, set UNIX listener socket to be owned by the same user as daemon 
process runs as.



Any ideas how to PATCH mod_fcgid ?
mpm-itk with mod_fcgid would be more than great :)


OR: Any way to user mpm-itk and different php-versions / per user 
php.ini ?


Thanks
Martin

Re: Re: Re: mod_fcgid concurrency bottleneck, issue#53693

[pqf]

Hi, all
I took some times to setup a stress test for mod_fcgid, to see is there any 
other concurrency bottleneck left, it seems fine to me.

I did a stress test with callgrind on mod_fcgid, this is the big picture: 
http://people.apache.org/~pqf/mod_fcgid_performance/performance.png and this is 
the output of callgrind: 
http://people.apache.org/~pqf/mod_fcgid_performance/callgrind.out
As we can see, most cpu time are used in ap_create_environment() + 
init_environment(). ap_create_environment() copy environment vars to a char[][] 
buffer, and init_environment() copy char[][] to a char[] buffer for FCGI 
protocol. I have to keep using ap_create_environment() to be compatible with 
other cgi modules(cgid, cgi...), I will not shortcut these two calls for less 
cpu utils.

And I make a strace trace to the system calls, to make sure there is no obvious 
IO throughput bottleneck, and no luck either

4622  13:17:16.737888 ... futex resumed ) = 0
4622  13:17:16.737920 futex(0x95ce65c, FUTEX_WAKE, 1 unfinished ...
4622  13:17:16.738038 ... futex resumed ) = 0
4622  13:17:16.740735 gettimeofday({1346217436, 740787}, NULL) = 0
4622  13:17:16.740835 getsockname(12, {sa_family=AF_INET6, 
sin6_port=htons(8080), inet_pton(AF_INET6, :::127.0.0.1, si
n6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [28]) = 0
4622  13:17:16.741008 fcntl64(12, F_GETFL) = 0x2 (flags O_RDWR)
4622  13:17:16.741056 fcntl64(12, F_SETFL, O_RDWR|O_NONBLOCK) = 0
4622  13:17:16.741824 gettimeofday({1346217436, 741849}, NULL) = 0
4622  13:17:16.741890 gettimeofday({1346217436, 741911}, NULL) = 0
4622  13:17:16.741969 read(12, GET /fcgi-bin/foo.pl HTTP/1.0\r\nH..., 8000) = 
97
4622  13:17:16.742049 gettimeofday({1346217436, 742070}, NULL) = 0
4622  13:17:16.742108 gettimeofday({1346217436, 742129}, NULL) = 0
4622  13:17:16.742165 gettimeofday({1346217436, 742185}, NULL) = 0
4622  13:17:16.74 gettimeofday({1346217436, 742242}, NULL) = 0
4622  13:17:16.742278 gettimeofday({1346217436, 742299}, NULL) = 0
4622  13:17:16.742345 gettimeofday({1346217436, 742367}, NULL) = 0
4622  13:17:16.742404 gettimeofday({1346217436, 742424}, NULL) = 0
4622  13:17:16.742467 stat64(/usr/local/apache2/htdocs/fcgi-bin/foo.pl, 
{st_mode=S_IFREG|0755, st_size=229, ...}) = 0
4622  13:17:16.742697 semop(4292662, 0x17234c, 1) = 0
4622  13:17:16.742750 semop(4292662, 0x172358, 1) = 0
4622  13:17:16.742804 socket(PF_FILE, SOCK_STREAM, 0) = 13
4622  13:17:16.742864 connect(13, {sa_family=AF_FILE, 
path=/usr/local/apache2/logs/fcgidsock/4617.11}, 110) = 0
4622  13:17:16.743028 fcntl64(13, F_GETFL) = 0x2 (flags O_RDWR)
4622  13:17:16.743074 fcntl64(13, F_SETFL, O_RDWR|O_NONBLOCK) = 0
4622  13:17:16.743118 gettimeofday({1346217436, 743138}, NULL) = 0
4622  13:17:16.743174 writev(13, [{\1\1\0\1\0\10\0\0, 8}, 
{\0\1\0\0\0\0\0\0, 8}, {\1\4\0\1\3U\0\0, 8}, {\t\30UNIQUE_ID
UD2l3MCoqlsAABILpXoAA..., 853}, {\1\4\0\1\0\0\0\0, 8}, {\1\5\0\1\0\0\0\0, 
8}], 6) = 893
4622  13:17:16.743337 read(13, 0xa6f4c5b8, 8192) = -1 EAGAIN (Resource 
temporarily unavailable)
4622  13:17:16.743386 poll([{fd=13, events=POLLIN, revents=POLLIN}], 1, 4) 
= 1
4622  13:17:16.743789 read(13, \1\6\0\1\3\266\2\0Content-Type: text/plain..., 
8192) = 984
4622  13:17:16.743879 gettimeofday( unfinished ...
4622  13:17:16.743968 ... gettimeofday resumed {1346217436, 743943}, NULL) = 0
4622  13:17:16.744037 close(13) = 0
4622  13:17:16.744101 semop(4292662, 0x17234c, 1) = 0
4622  13:17:16.744149 semop(4292662, 0x172358, 1) = 0
4622  13:17:16.744209 gettimeofday({1346217436, 744231}, NULL) = 0
4622  13:17:16.744266 gettimeofday({1346217436, 744287}, NULL) = 0
4622  13:17:16.744343 writev(12, [{HTTP/1.1 200 OK\r\nDate: Wed, 29 A..., 
154}, {CONTEXT_DOCUMENT_ROOT=\/usr/loca..., 92
4}], 2 unfinished ...
4622  13:17:16.745033 ... writev resumed ) = 1078
4622  13:17:16.745076 gettimeofday({1346217436, 745098}, NULL) = 0
4622  13:17:16.745148 times({tms_utime=1416, tms_stime=1464, tms_cutime=0, 
tms_cstime=0}) = 499844951
4622  13:17:16.745255 gettimeofday( unfinished ...
4622  13:17:16.745800 ... gettimeofday resumed {1346217436, 745791}, NULL) = 0
4622  13:17:16.746013 shutdown(12, 1 /* send */ unfinished ...
4622  13:17:16.746554 ... shutdown resumed ) = 0
4622  13:17:16.746625 gettimeofday( unfinished ...
4622  13:17:16.746684 ... gettimeofday resumed {1346217436, 746658}, NULL) = 0
4622  13:17:16.746765 epoll_ctl(11, EPOLL_CTL_ADD, 12, {EPOLLIN, 
{u32=158210764, u64=158210764}} unfinished ...
4622  13:17:16.746833 ... epoll_ctl resumed ) = 0
4622  13:17:16.747587 gettimeofday( unfinished ...
4622  13:17:16.747904 ... gettimeofday resumed {1346217436, 747649}, NULL) = 0
4622  13:17:16.748672 futex(0x95ce690, FUTEX_WAIT, 176390, NULL unfinished ...



2012-09-05



pqf



发件人：pqf
发送时间：2012-08-28 09:34
主题：回复: Re: Re: Re: mod_fcgid concurrency bottleneck, issue#53693
收件人：devdev@httpd.apache.org
抄送：

So what can mod_fcgid do in this overloaded?
1. mod_fcgid get a request
2. mod_fcgid can't apply a free slot

Re: Re: Re: Re: mod_fcgid concurrency bottleneck, issue#53693

[Lazy]

2012/8/28 pqf p...@mailtech.cn:
 So what can mod_fcgid do in this overloaded?
 1. mod_fcgid get a request
 2. mod_fcgid can't apply a free slot of FCGI handler
 3. mod_fcgid send a spawn request to PM
 4. PM deny the request(for too much process already)
 5. Now
for( i=1; i64; i++)
   {
  a) mod_fcgid delay a while, and then send another spawn request to PM
 and try apply free slot again.
  b) mod_fcgid send another spawn request at once, even the last request
 is denied.
  c) ??
 (now is a, b maybe not a good idea, any new idea?)
   }

 I think the bottleneck is too much request, too less FCGI handler. httpd(or
 mod_fcgid) either drop client connections or delay a while, there is no
 other way out?

My idea is to add a availability number to each class. If a wait fails
it will be decreased or increased if wait is successful. Let's say we
want to wait max 100 times 250ms


int connected=0;

   for(i=0; !connected  i = (get_clas_avail()/MAX_AVAIL)*100; i++) {
/* Apply a process slot */
bucket_ctx-procnode = apply_free_procnode(r, fcgi_request);

/* Send a spawn request if I can't get a process slot */
/* procmgr_send_spawn_cmd() return APR_SUCCESS if a
process is created */
if( !bucket_ctx-procnode 
(procmgr_send_spawn_cmd(fcgi_request, r) != APR_SUCCESS) )
apr_sleep(apr_time_from_msec(250));
else
/* Apply a process slot */
bucket_ctx-procnode = apply_free_procnode(r, fcgi_request);

if (bucket_ctx-procnode) {
if (proc_connect_ipc(bucket_ctx-procnode,
 bucket_ctx-ipc) != APR_SUCCESS) {
proc_close_ipc(bucket_ctx-ipc);
bucket_ctx-procnode-diewhy = FCGID_DIE_CONNECT_ERROR;
return_procnode(r, bucket_ctx-procnode, 1 /* has
error */ );
bucket_ctx-procnode = NULL;
decrease_avail();
}else {
increase_avail();
connected=1;
}
}
}

if (!connected) {
decrease_avail();
ap_log_rerror(APLOG_MARK, APLOG_WARNING, 0, r,
  mod_fcgid: can't apply process slot for %s,
  cmd_conf-cmdline);
return HTTP_SERVICE_UNAVAILABLE;
}

decrease_avail() might halve availability each time called.


Availability should be dynamic maybe controlled by processmanager and
returned to the threads handling connections by
procmgr_send_spawn_cmd(), it can depend on total number of denied
spawn requests for specific class in a similar way as score, without
this connections will be 503 not sooner then 25 seconds, which is
still IMHO to long. Another improvement would bo to make wait time
shorter for not overloaded classes to keep the penalty of denied spawn
as low as possible.


I plan to work on that later.



 Another question. Is it necessary to call procmgr_init_spawn_cmd() from
 inside the for loop ?
 I took a brief look, it seems not necessary. I will move it out of loop and
 test.

 2012-08-28
 
 pqf
 
 发件人：Lazy
 发送时间：2012-08-27 21:47
 主题：Re: Re: Re: mod_fcgid concurrency bottleneck, issue#53693
 收件人：devdev@httpd.apache.org
 抄送：

 2012/8/16 pqf p...@mailtech.cn:
 How about this:
 1. procmgr_post_spawn_cmd() now return a status code from PM, so process
 handler now know the spawn request is denyed or not.
 2. if a new process is created, no sleep is needed.
 3. if no process is created, sleep a while

 sorry for the late reply,

 in the old code there ware no sleep() between procmgr_post_spawn_cmd()
 and apply_free_procnode()

 sleep() was invoked only if there ware no free procnode.

 This happened only if we ware denied spawning new process or in some
 cases if some other thread managed to use that procnode before us.

 Your change adresses cases if some other thread stole our newly
 spawned fcgi process, old code was waiting 1s before trying to spawn
 another/recheck, new code doesn't, I guess this is the orginal issue
 in stress tests when total number of simultaneous connections doesn't
 exceed max fcgi processes. But when spawning is denied recovery time
 is still long 1s.


 I was refering to cases when spawn is denied.

 If a vhost is overloaded or someone added sleep(60) in the code,
 mod_fcgid blocks on all request to that vhost
 for over a minute and it is possible to occupy 1000 threads using
 under 20 new connections to slow vhost
 per second. This can be mitingated by adding avaiability which will
 impact time spend on waiting for free process. Overloaded vhost will
 start to drop connections faster preventing the web-server reaching
 MaxClients
 limit.

 Another question. Is it necessary to call procmgr_init_spawn_cmd()
 from inside the for loop ?



 2012-08-16
 
 pqf

Re: Re: Re: mod_fcgid concurrency bottleneck, issue#53693

[Lazy]

2012/8/16 pqf p...@mailtech.cn:
 How about this:
 1. procmgr_post_spawn_cmd() now return a status code from PM, so process
 handler now know the spawn request is denyed or not.
 2. if a new process is created, no sleep is needed.
 3. if no process is created, sleep a while

sorry for the late reply,

in the old code there ware no sleep() between procmgr_post_spawn_cmd()
and apply_free_procnode()

sleep() was invoked only if there ware no free procnode.

This happened only if we ware denied spawning new process or in some
cases if some other thread managed to use that procnode before us.

Your change adresses cases if some other thread stole our newly
spawned fcgi process, old code was waiting 1s before trying to spawn
another/recheck, new code doesn't, I guess this is the orginal issue
in stress tests when total number of simultaneous connections doesn't
exceed max fcgi processes. But when spawning is denied recovery time
is still long 1s.


I was refering to cases when spawn is denied.

If a vhost is overloaded or someone added sleep(60) in the code,
mod_fcgid blocks on all request to that vhost
for over a minute and it is possible to occupy 1000 threads using
under 20 new connections to slow vhost
per second. This can be mitingated by adding avaiability which will
impact time spend on waiting for free process. Overloaded vhost will
start to drop connections faster preventing the web-server reaching
MaxClients
limit.

Another question. Is it necessary to call procmgr_init_spawn_cmd()
from inside the for loop ?



 2012-08-16
 
 pqf
 
 发件人：Lazy
 发送时间：2012-08-16 16:47
 主题：Re: Re: mod_fcgid concurrency bottleneck, issue#53693
 收件人：devdev@httpd.apache.org
 抄送：

 2012/8/16 pqf p...@mailtech.cn:
 Hi, Michal
 My solution do add availability to each class, which is the
 procmgr_post_spawn_cmd() call in each loop do.
 The sleep() call is intrudused for a stress test without warm up time, in
 this case, mod_fcgid will create more processes than a slow start one(each
 process handler can't apply a free slot on the very begining, so send a
 request to process manager to create one, it's easy to reach the max # of
 process limit while httpd startup, but the idle process will be killed
 later), the sleep() call is a little like a server side warm up delay.
 But since someone said remove this sleep(), the server work fine without
 bottleneck(Maybe he didn't notise the warm up issue?), so I thought remove
 the sleep() is a good idea. But reduce the time of sleep() is fine to me
 too.

 I was referring to the case where all processes are busy, without
 sleep(), handle_request() wil quickly send spawn requsts, whith will
 be denyed by process menager, with sleep() handle_request() will
 always wait quite a long time,
 occupying slots

 --
 Michal Grzedzicki

回复: Re: Re: Re: mod_fcgid concurrency bottleneck, issue#53693

[pqf]

So what can mod_fcgid do in this overloaded?
1. mod_fcgid get a request
2. mod_fcgid can't apply a free slot of FCGI handler
3. mod_fcgid send a spawn request to PM
4. PM deny the request(for too much process already)
5. Now
   for( i=1; i64; i++)
  {
 a) mod_fcgid delay a while, and then send another spawn request to PM and 
try apply free slot again.
 b) mod_fcgid send another spawn request at once, even the last request is 
denied.
 c) ??
(now is a, b maybe not a good idea, any new idea?)
  }

I think the bottleneck is too much request, too less FCGI handler. httpd(or 
mod_fcgid) either drop client connections or delay a while, there is no other 
way out?


Another question. Is it necessary to call procmgr_init_spawn_cmd() from inside 
the for loop ? 
I took a brief look, it seems not necessary. I will move it out of loop and 
test.

2012-08-28



pqf



发件人：Lazy
发送时间：2012-08-27 21:47
主题：Re: Re: Re: mod_fcgid concurrency bottleneck, issue#53693
收件人：devdev@httpd.apache.org
抄送：

2012/8/16 pqf p...@mailtech.cn: 
 How about this: 
 1. procmgr_post_spawn_cmd() now return a status code from PM, so process 
 handler now know the spawn request is denyed or not. 
 2. if a new process is created, no sleep is needed. 
 3. if no process is created, sleep a while 

sorry for the late reply, 

in the old code there ware no sleep() between procmgr_post_spawn_cmd() 
and apply_free_procnode() 

sleep() was invoked only if there ware no free procnode. 

This happened only if we ware denied spawning new process or in some 
cases if some other thread managed to use that procnode before us. 

Your change adresses cases if some other thread stole our newly 
spawned fcgi process, old code was waiting 1s before trying to spawn 
another/recheck, new code doesn't, I guess this is the orginal issue 
in stress tests when total number of simultaneous connections doesn't 
exceed max fcgi processes. But when spawning is denied recovery time 
is still long 1s. 


I was refering to cases when spawn is denied. 

If a vhost is overloaded or someone added sleep(60) in the code, 
mod_fcgid blocks on all request to that vhost 
for over a minute and it is possible to occupy 1000 threads using 
under 20 new connections to slow vhost 
per second. This can be mitingated by adding avaiability which will 
impact time spend on waiting for free process. Overloaded vhost will 
start to drop connections faster preventing the web-server reaching 
MaxClients 
limit. 

Another question. Is it necessary to call procmgr_init_spawn_cmd() 
from inside the for loop ? 


 
 2012-08-16 
  
 pqf 
  
 发件人：Lazy 
 发送时间：2012-08-16 16:47 
 主题：Re: Re: mod_fcgid concurrency bottleneck, issue#53693 
 收件人：devdev@httpd.apache.org 
 抄送： 
 
 2012/8/16 pqf p...@mailtech.cn: 
 Hi, Michal 
 My solution do add availability to each class, which is the 
 procmgr_post_spawn_cmd() call in each loop do. 
 The sleep() call is intrudused for a stress test without warm up time, in 
 this case, mod_fcgid will create more processes than a slow start one(each 
 process handler can't apply a free slot on the very begining, so send a 
 request to process manager to create one, it's easy to reach the max # of 
 process limit while httpd startup, but the idle process will be killed 
 later), the sleep() call is a little like a server side warm up delay. 
 But since someone said remove this sleep(), the server work fine without 
 bottleneck(Maybe he didn't notise the warm up issue?), so I thought remove 
 the sleep() is a good idea. But reduce the time of sleep() is fine to me 
 too. 
 
 I was referring to the case where all processes are busy, without 
 sleep(), handle_request() wil quickly send spawn requsts, whith will 
 be denyed by process menager, with sleep() handle_request() will 
 always wait quite a long time, 
 occupying slots 
 
 -- 
 Michal Grzedzicki 
 
 

Re: mod_fcgid concurrency bottleneck, issue#53693

[William A. Rowe Jr.]

On 8/15/2012 8:23 PM, pqf wrote:
 But since someone said remove this sleep(), the server work fine without 
 bottleneck(Maybe
 he didn't notise the warm up issue?), so I thought remove the sleep() is a 
 good idea. But
 reduce the time of sleep() is fine to me too.
 Any other idea?

Can we introduce a blocking call on 'pipe ready' based on the child spawn
being ready to accept data?  The equivalent of a ping-test?

Re: Re: mod_fcgid concurrency bottleneck, issue#53693

[pqf]

procmgr_post_spawn_cmd(maybe it should be renamed to procmgr_send_spawn_cmd) 
will keep blocking while PM is creating a new process, after 
procmgr_post_spawn_cmd() return, there should be a process ready there waiting 
for accept.

2012-08-16



pqf



发件人：William A. Rowe Jr.
发送时间：2012-08-16 15:23
主题：Re: mod_fcgid concurrency bottleneck, issue#53693
收件人：devdev@httpd.apache.org
抄送：pqfp...@mailtech.cn

On 8/15/2012 8:23 PM, pqf wrote: 
 But since someone said remove this sleep(), the server work fine without 
 bottleneck(Maybe 
 he didn't notise the warm up issue?), so I thought remove the sleep() is a 
 good idea. But 
 reduce the time of sleep() is fine to me too. 
 Any other idea? 

Can we introduce a blocking call on 'pipe ready' based on the child spawn 
being ready to accept data?  The equivalent of a ping-test?