Re: mod_fcgid Release

[Jan Ehrhardt]

Mario Brandt in gmane.comp.apache.devel (Tue, 15 Dec 2020 14:14:43
+0100):
>Ping
>
>On Thu, 24 Jan 2019 at 11:23, Mario Brandt  wrote:
>>
>> Hi,
>> it has beens a while since there was the last release of mod_fcgid.
>> There are some important fixes in trunk, but never made it to a
>> release since 2013.
>> Is anyone willing to file a release?

+1
-- 
Jan

Re: mod_fcgid Release

[Jeff McKenna]

Seconded.

-jeff



On 2020-12-15 9:14 a.m., Mario Brandt wrote:

Ping

On Thu, 24 Jan 2019 at 11:23, Mario Brandt  wrote:


Hi,
it has beens a while since there was the last release of mod_fcgid.
There are some important fixes in trunk, but never made it to a
release since 2013.
Is anyone willing to file a release?

Cheers
Mario

Re: mod_fcgid Release

[Mario Brandt]

Ping

On Thu, 24 Jan 2019 at 11:23, Mario Brandt  wrote:
>
> Hi,
> it has beens a while since there was the last release of mod_fcgid.
> There are some important fixes in trunk, but never made it to a
> release since 2013.
> Is anyone willing to file a release?
>
> Cheers
> Mario

Re: mod_fcgid: Immediate HTTP error 503 if the max total process count is reached

[Ivan Zahariev]

Hi,

Thanks for the review. You can find my comments below in the reply.


On 20.9.2016 г. 16:33 ч., Eric Covener wrote:

Unfortunately there are not many reviewers for mod_fcgid.
I tried to take as a relative laymen and had a few comments/questions:

* a few C99 // comments were added and should be zapped

Fixed.


* the assert looks funny -- no assert.   maybe ap_debug_assert() so it
blows up in maintainer builds only?
It's a paranoid check. If it ever blows the whistle, this will be in 
some very hard to reproduce case, and I don't expect that we catch this 
on a maintainer build. I made this an APLOG_CRIT message because a 
mismatch in the counts between proctable_count_all_nonfree_nodes() and 
"g_total_process" could make your FastCGI requests fail erroneously with 
an immediate 503 HTTP error. A System Administrator who could encounter 
this bug will appreciate the immediate APLOG_CRIT message in the 
"error_log".


I don't insist this to remain like this and I will change it as you say, 
in order to comply with the usual development practices. Or we can give 
this function check a better name.



* The new flag is merged, but cannot be unset?
** Probably need to make it look like an AP_INIT_FLAG

Right. I've converted this to an AP_INIT_FLAG which makes more sense.

I don't understand what you mean by "it cannot be unset". I'm merging it 
in the same way as other mod_fcgid config options. Please read below for 
more information regarding this matter.



* I couldn't follow the concept in the comments around
max_process_count moving or being per-request. Is it maybe out of
date? I think the current impl takes an existing global only and
merges it with vhost server confs but it will always be a copy.


The original implementation allows you to set "max_process_count" in the 
GLOBAL_ONLY context. This is enforced by ap_check_cmd_context() in the 
AP_INIT_TAKE1() handler, because RSRC_CONF would otherwise allow us to 
define this config option also "per vhost".


I've added the new variable "max_process_used_no_wait_enable" in the 
same way.


The original implementation works well without further magic, because it 
really uses "max_process_count" in code which works with the 
"main_server" (global) context. Now I need to access the value of 
"max_process_count" inside a "request_rec" (per VirtualHost) variable. 
If one or more mod_fcgid directives are defined not only within the main 
server config, but also as additional settings in a VirtualHost context, 
then we need to merge "max_process_count", too. If we don't merge, the 
value of "max_process_count" returned by the "request_rec" variable will 
be the default init value (zero, in our case).


Long story short -- you are right, this is a global only variable and it 
merges with vhost confs always as a copy of the global value. This is 
just a behind-the-scenes action and doesn't affect Apache 
administrators, since they can't define this variable in a vhost context 
anyway. I tried to summarize this as the comment ["global only" but we 
use its main_server value in a "request_rec"] but maybe someone can 
propose a better text.


I've committed the source code changes and you can review them again. 
When we're satisfied with the changes, I'll test the final version on 
our server fleet.


Best regards.
--Ivan

Re: mod_fcgid: Immediate HTTP error 503 if the max total process count is reached

[Eric Covener]

On Mon, Sep 19, 2016 at 4:30 AM, Ivan Zahariev  wrote:
> Hello devs,
>
> It's been four months since I originally proposed this new feature in
> "mod_fcgid". During this time I've tested and deployed it on hundreds of
> busy production servers. It works as expected. If enabled, web visitors get
> an immediate response when FastCGI is overloaded, and no RAM is
> over-utilized.
>
> You can find all the information and a patch in the enhancement request at
> Bugzilla: https://bz.apache.org/bugzilla/show_bug.cgi?id=59656
>
> Can we get this merged into "mod_fcgid"?
>
> Best regards.
> --Ivan

Unfortunately there are not many reviewers for mod_fcgid.

I tried to take as a relative laymen and had a few comments/questions:

* a few C99 // comments were added and should be zapped
* the assert looks funny -- no assert.   maybe ap_debug_assert() so it
blows up in maintainer builds only?
* The new flag is merged, but cannot be unset?
** Probably need to make it look like an AP_INIT_FLAG
* I couldn't follow the concept in the comments around
max_process_count moving or being per-request. Is it maybe out of
date? I think the current impl takes an existing global only and
merges it with vhost server confs but it will always be a copy.

Re: mod_fcgid: Immediate HTTP error 503 if the max total process count is reached

[Ivan Zahariev]

Hello devs,

It's been four months since I originally proposed this new feature in 
"mod_fcgid". During this time I've tested and deployed it on hundreds of 
busy production servers. It works as expected. If enabled, web visitors 
get an immediate response when FastCGI is overloaded, and no RAM is 
over-utilized.


You can find all the information and a patch in the enhancement request 
at Bugzilla: https://bz.apache.org/bugzilla/show_bug.cgi?id=59656


Can we get this merged into "mod_fcgid"?

Best regards.
--Ivan


On 2.6.2016 г. 11:57 ч., Ivan Zahariev wrote:

Hi Nick,

Thanks for the info.

I've followed your instructions and submitted an enhancement request: 
https://bz.apache.org/bugzilla/show_bug.cgi?id=59656


Cheers.
--Ivan


On 31.5.2016 г. 13:45 ч., Nick Kew wrote:

On Tue, 2016-05-31 at 11:15 +0300, Ivan Zahariev wrote:

Hello,

I got no feedback. Am I posting this suggestion at the right mailing
list?

Sorry, I see your original post marked for attention in my mail
folder, but languishing hitherto unattended.  Just now opened your
link in a browser to take a look.  There could be others who
have done something similar.

As a general reply to this question, yes, this is the best
available mailinglist.  The other place to post it would be
as an enhancement request in bugzilla (issues.apache.org).
The keyword "PatchAvailable" there may help by marking it as
low-hanging fruit.

Re: mod_fcgid: Immediate HTTP error 503 if the max total process count is reached

[Ivan Zahariev]

Hi Nick,

Thanks for the info.

I've followed your instructions and submitted an enhancement request: 
https://bz.apache.org/bugzilla/show_bug.cgi?id=59656


Cheers.
--Ivan


On 31.5.2016 г. 13:45 ч., Nick Kew wrote:

On Tue, 2016-05-31 at 11:15 +0300, Ivan Zahariev wrote:

Hello,

I got no feedback. Am I posting this suggestion at the right mailing
list?

Sorry, I see your original post marked for attention in my mail
folder, but languishing hitherto unattended.  Just now opened your
link in a browser to take a look.  There could be others who
have done something similar.

As a general reply to this question, yes, this is the best
available mailinglist.  The other place to post it would be
as an enhancement request in bugzilla (issues.apache.org).
The keyword "PatchAvailable" there may help by marking it as
low-hanging fruit.

Re: mod_fcgid: Immediate HTTP error 503 if the max total process count is reached

[Nick Kew]

On Tue, 2016-05-31 at 11:15 +0300, Ivan Zahariev wrote:
> Hello,
> 
> I got no feedback. Am I posting this suggestion at the right mailing
> list?

Sorry, I see your original post marked for attention in my mail
folder, but languishing hitherto unattended.  Just now opened your
link in a browser to take a look.  There could be others who
have done something similar.

As a general reply to this question, yes, this is the best
available mailinglist.  The other place to post it would be
as an enhancement request in bugzilla (issues.apache.org).
The keyword "PatchAvailable" there may help by marking it as
low-hanging fruit.

-- 
Nick Kew

Re: mod_fcgid: Immediate HTTP error 503 if the max total process count is reached

[Ivan Zahariev]

Hello,

I got no feedback. Am I posting this suggestion at the right mailing list?

Best regards.
--Ivan

On 19.5.2016 г. 10:40 ч., Ivan Zahariev wrote:

Hi all,

I'd like to propose a new configuration setting for "mod_fcgid". The 
source code changes to review follow:


  * The whole patch compared to version 2.3.9:

https://github.com/famzah/mod_fcgid/compare/2.3.9...maxnowait?diff=split=maxnowait
  * The whole patch as a single file:

https://github.com/famzah/mod_fcgid/compare/2.3.9...maxnowait.patch?diff=unified=maxnowait
  * Every single commit compared to version 2.3.9:
https://github.com/famzah/mod_fcgid/commits/maxnowait
  * There should be no merge conflicts with the current "trunk"
version 2.3.10.


The motivation is that the current behavior to queue up new pending 
requests differs from the RLimitNPROC directive behavior. When there 
is a spike in the web hits, lots of Apache children get busy just 
waiting for up to 30+ seconds to get an idle FastCGI process. Thus we 
"waste" Apache children doing nothing while they could serve static 
content. This also puts unneeded memory pressure. Additionally, users 
today won't wait for a page to load, if it takes more than a few 
seconds, and we'd rather notify them that we are currently overloaded 
by sending them a 503 HTTP response immediately.


Here is the documentation for the new directive: 
http://www.famzah.net/temp/FcgidMaxProcessesUsedNoWait.docs.txt


Let me know what you think.

Best regards.
--Ivan

Re: mod_fcgid and broken doc links

[Jim Jagielski]

There is a need and an audience for both. We provide
both. I see no reason for us to stop doing that unless
we feel that we can no longer support both in the
manner in which our community expects. FWIW, I don't
see that as the current situation.

> On Jan 18, 2016, at 9:29 PM, William A Rowe Jr  wrote:
> 
> On Mon, Jan 18, 2016 at 3:29 PM, Jim Jagielski  wrote:
> 
> > On Jan 18, 2016, at 3:28 PM, William A Rowe Jr  wrote:
> >
> > On Fri, Jan 15, 2016 at 7:44 AM, Jim Jagielski  wrote:
> >
> > > On Jan 14, 2016, at 5:19 PM, William A Rowe Jr  
> > > wrote:
> > >
> > > Good point with your example, this is something that should
> > > be benchmarked and the winner-take-all, loser bumped from the
> > > trunk/ copy of httpd.
> >
> > -1
> >
> > You are implying that one would be a winner in all cases. The
> > whole idea is that there are cases where one is better than
> > the other. We provide both.
> >
> > You might have made that inference, but I'm going to assert that
> > for this one module, for s/{literal}/{repl}, mod_sed is going to
> > outperform mod_substitute /if/ we wrote the code correctly.
> 
> I disagree... it's kind of obvious by simple inspection that
> mod_substitute has fast paths that mod_sed lacks and, as thus,
> can be quite performant and the "better" choice in numerous cases
> where that fast path is used.
> 
> Well, only benchmarking is going to prove that one way or the other,
> something I don't have free cycles for right now (committed to way too
> many other project priorities.)  And it perhaps opens up opportunities
> to optimize mod_sed in ways that might have been initially overlooked
> when the code was thrown into trunk ;-)
>  
> Me, I don't tend to think of myself as "smarter" than all of
> our users, nor do I try to act as Big Brother and remove choices
> from people in cases and situations where they are using them.
> The ASF itself doesn't do that, nor do projects... so it seems
> kinda weird that you would want the httpd project to all of
> a sudden start removing choice and options for end users instead
> of helping them out and trusting them to know which impl is
> best for them.
> 
> Thankfully, we aren't, but our users do look at us as experts in the
> code they consume, considering that we collectively have authored
> and maintain the stuff.  So they do look to us to make the best
> choices they don't have the individual time to compare and elect.
> Why would I want us to collectively (and for me specifically) to
> propose the best solutions to the common use cases, and to
> depreciate less maintained code in favor of maintained code?  
> 
> Can't imagine why I or other project members would be possessed 
> to do that.  If you figure it out, please share :)

Re: mod_fcgid and broken doc links

[William A Rowe Jr]

On Mon, Jan 18, 2016 at 3:29 PM, Jim Jagielski  wrote:

>
> > On Jan 18, 2016, at 3:28 PM, William A Rowe Jr 
> wrote:
> >
> > On Fri, Jan 15, 2016 at 7:44 AM, Jim Jagielski  wrote:
> >
> > > On Jan 14, 2016, at 5:19 PM, William A Rowe Jr 
> wrote:
> > >
> > > Good point with your example, this is something that should
> > > be benchmarked and the winner-take-all, loser bumped from the
> > > trunk/ copy of httpd.
> >
> > -1
> >
> > You are implying that one would be a winner in all cases. The
> > whole idea is that there are cases where one is better than
> > the other. We provide both.
> >
> > You might have made that inference, but I'm going to assert that
> > for this one module, for s/{literal}/{repl}, mod_sed is going to
> > outperform mod_substitute /if/ we wrote the code correctly.
>
> I disagree... it's kind of obvious by simple inspection that
> mod_substitute has fast paths that mod_sed lacks and, as thus,
> can be quite performant and the "better" choice in numerous cases
> where that fast path is used.
>

Well, only benchmarking is going to prove that one way or the other,
something I don't have free cycles for right now (committed to way too
many other project priorities.)  And it perhaps opens up opportunities
to optimize mod_sed in ways that might have been initially overlooked
when the code was thrown into trunk ;-)


> Me, I don't tend to think of myself as "smarter" than all of
> our users, nor do I try to act as Big Brother and remove choices
> from people in cases and situations where they are using them.
> The ASF itself doesn't do that, nor do projects... so it seems
> kinda weird that you would want the httpd project to all of
> a sudden start removing choice and options for end users instead
> of helping them out and trusting them to know which impl is
> best for them.
>

Thankfully, we aren't, but our users do look at us as experts in the
code they consume, considering that we collectively have authored
and maintain the stuff.  So they do look to us to make the best
choices they don't have the individual time to compare and elect.
Why would I want us to collectively (and for me specifically) to
propose the best solutions to the common use cases, and to
depreciate less maintained code in favor of maintained code?

Can't imagine why I or other project members would be possessed
to do that.  If you figure it out, please share :)

Re: mod_fcgid and broken doc links

[William A Rowe Jr]

On Fri, Jan 15, 2016 at 7:44 AM, Jim Jagielski  wrote:

>
> > On Jan 14, 2016, at 5:19 PM, William A Rowe Jr 
> wrote:
> >
> > Good point with your example, this is something that should
> > be benchmarked and the winner-take-all, loser bumped from the
> > trunk/ copy of httpd.
>
> -1
>
> You are implying that one would be a winner in all cases. The
> whole idea is that there are cases where one is better than
> the other. We provide both.
>

You might have made that inference, but I'm going to assert that
for this one module, for s/{literal}/{repl}, mod_sed is going to
outperform mod_substitute /if/ we wrote the code correctly.  Whas
is missing is a mod_sed directive for adding literal s/{orig}/{repl}/
patterns that follows the substitute syntax and that admins don't
have to wrap their heads around the special character escaping.
At that point, there is no reason for the redundant module, but
we aren't at that point.

Would you also recommend such a scenario with MPMs, and
> having the "winner" take all? I certainly hope not!!!
>

No, I generally agree with TMTOWTDI.  Glad you raised the MPMs
issue, because my exact complaint about folding in mod_fcgid
"as is" and overlapping with mod_proxy_fcgi was addressed by the
MPM community in pulling out and sharing the mpm_common.c
logic. Great example, thanks Jim!

Re: mod_fcgid and broken doc links

[Jim Jagielski]

> On Jan 18, 2016, at 3:28 PM, William A Rowe Jr  wrote:
> 
> On Fri, Jan 15, 2016 at 7:44 AM, Jim Jagielski  wrote:
> 
> > On Jan 14, 2016, at 5:19 PM, William A Rowe Jr  wrote:
> >
> > Good point with your example, this is something that should
> > be benchmarked and the winner-take-all, loser bumped from the
> > trunk/ copy of httpd.
> 
> -1
> 
> You are implying that one would be a winner in all cases. The
> whole idea is that there are cases where one is better than
> the other. We provide both.
> 
> You might have made that inference, but I'm going to assert that
> for this one module, for s/{literal}/{repl}, mod_sed is going to 
> outperform mod_substitute /if/ we wrote the code correctly.

I disagree... it's kind of obvious by simple inspection that
mod_substitute has fast paths that mod_sed lacks and, as thus,
can be quite performant and the "better" choice in numerous cases
where that fast path is used.

Me, I don't tend to think of myself as "smarter" than all of
our users, nor do I try to act as Big Brother and remove choices
from people in cases and situations where they are using them.
The ASF itself doesn't do that, nor do projects... so it seems
kinda weird that you would want the httpd project to all of
a sudden start removing choice and options for end users instead
of helping them out and trusting them to know which impl is
best for them.

Re: mod_fcgid and broken doc links

[Jim Jagielski]

> On Jan 14, 2016, at 5:19 PM, William A Rowe Jr  wrote:
> 
> 
> Good point with your example, this is something that should 
> be benchmarked and the winner-take-all, loser bumped from the
> trunk/ copy of httpd.

-1

You are implying that one would be a winner in all cases. The
whole idea is that there are cases where one is better than
the other. We provide both.

Would you also recommend such a scenario with MPMs, and
having the "winner" take all? I certainly hope not!!!

Re: mod_fcgid and broken doc links

[Jim Jagielski]

> On Jan 13, 2016, at 12:28 PM, William A Rowe Jr  wrote:
> 
> 
> I can see us moving those modules into trunk (not 2.4), retaining the
> mmn tests for 2.2 and 2.4 compat, and then deriving an fcgid release
> out of trunk/modules/fcgid/.  But I'm not clear why we would want to
> maintain the duplication between mod_proxy_fcgi and mod_fcgid?
> Individually they get little enough attention as it is.
> 

Because they are separate solutions to a similar problem,
ala mod_sed and mod_substitute for example. I can site several
more if that is worthwhile (but the point is made) mod_fcgid does
not require the whole mod_proxy overhead, for example, but
lacks some features that mod_proxy_fcgi does have, since the
latter can leverage some of that "overhead".

> Just my 2c USD, I'm open to ideas.

That is good to know.

Re: mod_fcgid and broken doc links

[William A Rowe Jr]

On Thu, Jan 14, 2016 at 6:19 AM, Jim Jagielski  wrote:

>
> > On Jan 13, 2016, at 12:28 PM, William A Rowe Jr 
> wrote:
> >
> > I can see us moving those modules into trunk (not 2.4), retaining the
> > mmn tests for 2.2 and 2.4 compat, and then deriving an fcgid release
> > out of trunk/modules/fcgid/.  But I'm not clear why we would want to
> > maintain the duplication between mod_proxy_fcgi and mod_fcgid?
> > Individually they get little enough attention as it is.
>
> Because they are separate solutions to a similar problem,
> ala mod_sed and mod_substitute for example. I can site several
> more if that is worthwhile (but the point is made) mod_fcgid does
> not require the whole mod_proxy overhead, for example, but
> lacks some features that mod_proxy_fcgi does have, since the
> latter can leverage some of that "overhead".


Good point with your example, this is something that should
be benchmarked and the winner-take-all, loser bumped from the
trunk/ copy of httpd.  They are slightly different in that sed is one
specific grammar, while substitute uses literal replacements, but
that could be a syntax tweak to mod_sed to allow some literal
replacements.  In any case, it relieves the dozen or so regular
httpd contributors from maintaining two modules which do, for
the most part, the very same thing expressed with redundant
yet different code.

Likewise with fcgid/proxy_fcgi - /if/ we bring mod_fcgid sources
into the trunk, I'd add the caveat that we do so if there is an offer
to eliminate all of the code duplication in the process, or reject
this proposition.  They do perform two different purposes and
overlap in interesting ways, so there has to be something gained
by pushing fcgid upon the entire httpd committer community
to maintain.  It was one thing for the handful of us who commit
to mod_fcgid to deal with that legacy, but another to push it
mainstream.


> > Just my 2c USD, I'm open to ideas.
>
> That is good to know.


Was simply offering backstory for why things ended up as they
have, sans snark, and no snark needed in response :)  Would
love to learn who is interested in helping merge mod_fcgid and
mod_proxy_ftp functionality into a util_fcgi, or otherwise condense
this functionality into something manageable, long term.

Re: mod_fcgid and broken doc links

[Jim Jagielski]

Does it make sense to "officially" bundle mod_fcgid w/ httpd?

> On Jan 12, 2016, at 1:13 PM, Rich Bowen  wrote:
> 
> mod_fcgid is in a separate repo from the main httpd tree, due to
> historical reasons. I presume there are good reasons for this. JimJag
> suggested on IRC it's due to its independent release cycle.
> 
> Be that as it may, because it uses the standard documentation tools for
> the module docs, https://httpd.apache.org/mod_fcgid/mod/mod_fcgid.html
> is full of broken links. In particular, try any of the directive and
> links to other modules - try the mod_cgi or AddHandler links in the
> intro paragraph, and you'll see immediately what the problem is.
> 
> Now, we could of course have a separate version of the docs building
> tools just for this module, or we could patch the doc manually, but I
> was wondering, if there's no strong current reason for the module to be
> kept separate, can we please move it into the main httpd tree?
> 
> (Note that exactly the same situation applies to mod_ftp, but there's
> just fewer links from that doc so we don't hear it as often.)
> 
> --Rich
> 
> -- 
> Rich Bowen - rbo...@rcbowen.com - @rbowen
> http://apachecon.com/ - @apachecon

Re: mod_fcgid and broken doc links

[Jan Ehrhardt]

Jim Jagielski in gmane.comp.apache.devel (Wed, 13 Jan 2016 07:33:43
-0500):
>Does it make sense to "officially" bundle mod_fcgid w/ httpd?

FWIW: I always compile mod_fcgid.so together with Apache httpd. I have
made it part of my VC 9/11/14 solution files. I guess that many Windows
users of httpd install mod_fcgid, so for them it makes sense.

Jan

Re: mod_fcgid and broken doc links

[Rich Bowen]

On 01/13/2016 12:28 PM, William A Rowe Jr wrote:

> The reason for mod_ftp and mod_fcgid separate builds was historically
> that the same module, releasing on a different calendar than httpd, have
> been build-able independently against 2.0, 2.2 or 2.4.  Maintaining the 
> sources across the different branches was also something of a PITA.
> 
> Maybe more significantly, mod_proxy_fcgi was our 'adopted' solution 
> to the fcgi case.  mod_fcgid is a different beast with process pool 
> management. I was always under the impression that for 2.4 and later, 
> we collectively wanted to express process pools independently of the
> mod_proxy_fcgi structure, much like we and tomcat would love to see
> folks use mod_proxy_http or _ajp rather than mod_jk.
> 
> mod_ftp clearly isn't http:// so it never quite felt appropriate in that
> tree, but then again neither is mod_proxy_ajp :)  Which goes to the
> gist of it all, code bloat.  We've successfully only killed a tiny handful
> of modules in our entire history (imagemap, mem_cache etc). 

mod_imagemap still lives on in trunk. As does mod_cern_meta.

Point taken.

> So once merge to trunk, we own that code bloat for a very long time,
> but if it exists separate these can be enhanced or retired based on
> our desires.  E.g. if mod_aspdotnet had lived in modules/os/win32/
> we would still probably be shipping it, irrespective of how out of date
> that module becomes.
> 
> I can see us moving those modules into trunk (not 2.4), retaining the
> mmn tests for 2.2 and 2.4 compat, and then deriving an fcgid release
> out of trunk/modules/fcgid/.  But I'm not clear why we would want to
> maintain the duplication between mod_proxy_fcgi and mod_fcgid?
> Individually they get little enough attention as it is.

Yes, it would be nice to merge them, from the perspective of explaining
things to users.

-- 
Rich Bowen - rbo...@rcbowen.com - @rbowen
http://apachecon.com/ - @apachecon

Re: mod_fcgid and broken doc links

[William A Rowe Jr]

On Jan 13, 2016 15:50, "Rich Bowen"  wrote:
>
> Yes, it would be nice to merge them, from the perspective of explaining
> things to users.

Guess I am still confused what you suggest to merge... Docs or both docs
and code...

Also curious about released vs unreleased with respect to docs.  Our
pointers and resources for developers mostly live under
http://httpd.a.o/dev/ per ASF policy, but docs is an exception.  We are
supposed to be pointing users at the released artifacts, and not at
unreleased drafts.

Even http://httpd.apache.org/docs/dev/  might be clearer than trunk/ for
those who stumble upon them.

Thoughts?

Re: mod_fcgid and broken doc links

[Ruediger Pluem]

On 01/13/2016 01:33 PM, Jim Jagielski wrote:
> Does it make sense to "officially" bundle mod_fcgid w/ httpd?

Just for fixing the documention?
In this case I would prefer to investigate other solutions for the 
documentation and keep it separate.

Regards

Rüdiger

Re: mod_fcgid and broken doc links

[Mike Rumph]

A background for this request can be seen in bug report 56121.
- https://bz.apache.org/bugzilla/show_bug.cgi?id=56121#c4
This bug also describes a manual method for working around this problem.

On 1/12/2016 10:13 AM, Rich Bowen wrote:

mod_fcgid is in a separate repo from the main httpd tree, due to
historical reasons. I presume there are good reasons for this. JimJag
suggested on IRC it's due to its independent release cycle.

Be that as it may, because it uses the standard documentation tools for
the module docs, https://httpd.apache.org/mod_fcgid/mod/mod_fcgid.html
is full of broken links. In particular, try any of the directive and
links to other modules - try the mod_cgi or AddHandler links in the
intro paragraph, and you'll see immediately what the problem is.

Now, we could of course have a separate version of the docs building
tools just for this module, or we could patch the doc manually, but I
was wondering, if there's no strong current reason for the module to be
kept separate, can we please move it into the main httpd tree?

(Note that exactly the same situation applies to mod_ftp, but there's
just fewer links from that doc so we don't hear it as often.)

--Rich

Re: mod_fcgid and broken doc links

[Jim Jagielski]

Not just for that, but to make it easier for people
to use it... Seems stable enough that making it a
bundled module makes sense.

> On Jan 13, 2016, at 9:43 AM, Ruediger Pluem  wrote:
> 
> 
> 
> On 01/13/2016 01:33 PM, Jim Jagielski wrote:
>> Does it make sense to "officially" bundle mod_fcgid w/ httpd?
> 
> Just for fixing the documention?
> In this case I would prefer to investigate other solutions for the 
> documentation and keep it separate.
> 
> Regards
> 
> Rüdiger
> 

Re: mod_fcgid and broken doc links

[William A Rowe Jr]

On Tue, Jan 12, 2016 at 12:13 PM, Rich Bowen  wrote:

> mod_fcgid is in a separate repo from the main httpd tree, due to
> historical reasons. I presume there are good reasons for this. JimJag
> suggested on IRC it's due to its independent release cycle.
>
> Be that as it may, because it uses the standard documentation tools for
> the module docs, https://httpd.apache.org/mod_fcgid/mod/mod_fcgid.html
> is full of broken links. In particular, try any of the directive and
> links to other modules - try the mod_cgi or AddHandler links in the
> intro paragraph, and you'll see immediately what the problem is.
>
> Now, we could of course have a separate version of the docs building
> tools just for this module, or we could patch the doc manually, but I
> was wondering, if there's no strong current reason for the module to be
> kept separate, can we please move it into the main httpd tree?
>

The reason for mod_ftp and mod_fcgid separate builds was historically
that the same module, releasing on a different calendar than httpd, have
been build-able independently against 2.0, 2.2 or 2.4.  Maintaining the
sources across the different branches was also something of a PITA.

Maybe more significantly, mod_proxy_fcgi was our 'adopted' solution
to the fcgi case.  mod_fcgid is a different beast with process pool
management. I was always under the impression that for 2.4 and later,
we collectively wanted to express process pools independently of the
mod_proxy_fcgi structure, much like we and tomcat would love to see
folks use mod_proxy_http or _ajp rather than mod_jk.

mod_ftp clearly isn't http:// so it never quite felt appropriate in that
tree, but then again neither is mod_proxy_ajp :)  Which goes to the
gist of it all, code bloat.  We've successfully only killed a tiny handful
of modules in our entire history (imagemap, mem_cache etc).
So once merge to trunk, we own that code bloat for a very long time,
but if it exists separate these can be enhanced or retired based on
our desires.  E.g. if mod_aspdotnet had lived in modules/os/win32/
we would still probably be shipping it, irrespective of how out of date
that module becomes.

I can see us moving those modules into trunk (not 2.4), retaining the
mmn tests for 2.2 and 2.4 compat, and then deriving an fcgid release
out of trunk/modules/fcgid/.  But I'm not clear why we would want to
maintain the duplication between mod_proxy_fcgi and mod_fcgid?
Individually they get little enough attention as it is.

Just my 2c USD, I'm open to ideas.

Re: mod_fcgid: WIN32 compile-error and problem with slepp for 1 second

[Jeff Trawick]

Hello Michael,

This is a mailing list for developers of plug-in modules for httpd.  
us...@httpd.apache.org and d...@httpd.apache.org are better choices.  (I 
guess these two topics are code-related so dev@ is the best choice.)  
But see below.  (If further discussion is needed outside of the bug db, 
please post to d...@httpd.apache.org.)



On 04/22/2015 02:26 AM, Koperek, Michael wrote:

Hi,
using the current mod_fcgid 2.3.9 sources, im not able to compile on windows 
using msdev 6.0:

Compiling...
fcgid_conf.c
…\fcgid_conf.c(815) : error C2065: 'JOBOBJECT_EXTENDED_LIMIT_INFORMATION' : 
undeclared identifier
…\fcgid_conf.c(815) : error C2146: syntax error : missing ';' before identifier 
'job_info'
…\fcgid_conf.c(815) : error C2065: 'job_info' : undeclared identifier
…\fcgid_conf.c(815) : error C2059: syntax error : '{'
…\fcgid_conf.c(827) : error C2224: left of '.BasicLimitInformation' must have 
struct/union type
…\fcgid_conf.c(828) : error C2065: 'JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE' : 
undeclared identifier
…\fcgid_conf.c(830) : error C2065: 'JobObjectExtendedLimitInformation' : 
undeclared identifier

Seems to be not compilable code. Any ideas what to do?


You need a newer SDK from Microsoft that defines that API.  (Maybe a web 
search will help?)  It would be fair to open a bug against mod_fcgid 
asking for some compatibility with ancient SDK bundled with your 
compiler, but I'm not sure anyone would be able to work on that.





There is a sleep for 1 second in fcgid_bridge.c, function handle_request.
One second is for my application to high (user has to wait for each image to 
load on a web side with multiple images 1 second).
I changed the code from
apr_sleep(apr_time_from_sec(1));
to
   apr_sleep(APR_USEC_PER_SEC / 50); // 20 msecs

That works fine.
Is it possible to configure the sleep time?


Please comment on this bug to raise awareness that it affects more people:

https://bz.apache.org/bugzilla/show_bug.cgi?id=56308



Kind regards

Michael Koperek
e.bootis ag




e.bootis ag
Aktiengesellschaft mit Sitz in Essen
Amtsgericht Essen HRB 21782
Vorstand: Dr. Karl Langenstein, Ludger Langenstein, Tim Langenstein
Aufsichtsratsvorsitzender: Prof. Dr. Martin Užik

Diese E-Mail enthält vertrauliche und/oder rechtlich geschützte Informationen. 
Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten 
haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. 
Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail ist nicht 
gestattet.
This e-mail may contain confidential and/or privileged information. If you are 
not the intended recipient (or have received this e-mail in error) please 
notify the sender immediately and destroy this e-mail. Any unauthorized 
copying, disclosure or distribution of the material in this e-mail is strictly 
forbidden.

Re: mod_fcgid kill all subprocesses in reload

[Stefan Priebe]

Am 26.12.2014 um 17:49 schrieb Mario Brandt:

Hi,
there is a patch for busy processes
https://issues.apache.org/bugzilla/show_bug.cgi?id=48769 Maybe that is
a starting point.


Thanks, this helps a lot. Still wondering why it isn't upstream.

Stefan

Re: mod_fcgid kill all subprocesses in reload

[Mario Brandt]

Hi,
there is a patch for busy processes
https://issues.apache.org/bugzilla/show_bug.cgi?id=48769 Maybe that is
a starting point.

Cheers
Mario

On 26 December 2014 at 07:05, Graham Dumpleton grah...@apache.org wrote:
 Sounds like it would perhaps be for the same reason as mod_wsgi has issues
 with that sort of thing.

 Only Apache child worker processes get special dispensation as far as
 graceful shutdowns or reloads are concerned. If instead a module creates
 additional processes using the other child API calls in APR:

 http://apr.apache.org/docs/apr/1.4/group__apr__thread__proc.html#gaf8d2be452a819161aa4cd6205a17761e

 then when a shutdown or restart occurs they get a hard 3 seconds to shutdown
 or they will be killed with SIGKILL. Up to that 3 seconds they will be sent
 a normal SIGTERM each second to try and get them to exit.

 Thus there is no facility to allow them to linger longer that I know of
 unless something has been added in more recent times.

 Graham


 On 26 December 2014 at 16:57, Stefan Priebe - Profihost AG
 s.pri...@profihost.ag wrote:

 Hi list,

 i like mod fcgid a lot but there's one bug which makes me crazy.

 On DSO unload (Apache reload ) all child's get killed no matter if they
 process requests or not. This makes no sense to me httpd processes itself
 are also kept until all requests are served.

 Stefan

 Excuse my typo sent from my mobile phone.

Re: mod_fcgid kill all subprocesses in reload

[Graham Dumpleton]

Sounds like it would perhaps be for the same reason as mod_wsgi has issues
with that sort of thing.

Only Apache child worker processes get special dispensation as far as
graceful shutdowns or reloads are concerned. If instead a module creates
additional processes using the other child API calls in APR:

http://apr.apache.org/docs/apr/1.4/group__apr__thread__proc.html#gaf8d2be452a819161aa4cd6205a17761e

then when a shutdown or restart occurs they get a hard 3 seconds to
shutdown or they will be killed with SIGKILL. Up to that 3 seconds they
will be sent a normal SIGTERM each second to try and get them to exit.

Thus there is no facility to allow them to linger longer that I know of
unless something has been added in more recent times.

Graham


On 26 December 2014 at 16:57, Stefan Priebe - Profihost AG 
s.pri...@profihost.ag wrote:

 Hi list,

 i like mod fcgid a lot but there's one bug which makes me crazy.

 On DSO unload (Apache reload ) all child's get killed no matter if they
 process requests or not. This makes no sense to me httpd processes itself
 are also kept until all requests are served.

 Stefan

 Excuse my typo sent from my mobile phone.

Re: mod_fcgid and streaming large request bodies

[Justin Erenkrantz]

On Fri, Dec 27, 2013 at 8:49 AM, Justin Erenkrantz
jus...@erenkrantz.com wrote:
 Anyway, I'm diving into the code a bit - but, I figured it might be useful
 to see if anyone else has any thoughts about how to handle large request
 bodies with mod_fcgid.

Here's a first-cut patch that compiles at least.  Cheers.  -- justin

Add FcgidStreamRequestBody directive to send request body as it arrives without
storing it in memory or on disk.

* modules/fcgid/mod_fcgid.c
  (fcgid_cmds): Add FcgidStreamRequestBody directive.
* modules/fcgid/fcgid_conf.h
  (fcgid_server_conf): Add stream_request_body/stream_request_body_set flags.
* modules/fcgid/fcgid_conf.c
  (DEFAULT_STREAM_REQUEST_BODY): Set to 0.
  (create_fcgid_server_config): Init stream_request_body flag.
  (merge_fcgid_server_config): Merge new stream_request_body flag.
  (set_stream_request_body): New config helper.
* modules/fcgid/fcgid_bridge.c
  (add_request_body): Add a forward declaration to reduce diff (for now);
  take fcgid_bucket_ctx param; if stream_request_body is set, don't copy
  bucket and then call proc_write_ipc to send out that brigade and clear it
  out before next loop iteration.
  (bridge_request): Delay reading the request body until later.
  (handle_request_ipc): Move add_request_body call to here and append the EOS
  bucket before we do the final write of the request body.

Index: modules/fcgid/fcgid_bridge.c
===
--- modules/fcgid/fcgid_bridge.c (revision 1553671)
+++ modules/fcgid/fcgid_bridge.c (working copy)
@@ -287,6 +287,10 @@ static int getsfunc_fcgid_BRIGADE(char *buf, int l
 return done;
 }

+static int add_request_body(request_rec *r, apr_pool_t *request_pool,
+apr_bucket_brigade *output_brigade,
+fcgid_bucket_ctx *bucket_ctx);
+
 static int
 handle_request_ipc(request_rec *r, int role,
apr_bucket_brigade *output_brigade,
@@ -295,9 +299,21 @@ handle_request_ipc(request_rec *r, int role,
 int cond_status;
 apr_status_t rv;
 apr_bucket_brigade *brigade_stdout;
+apr_bucket *bucket_eos;
 char sbuf[MAX_STRING_LEN];
 const char *location;

+if (role == FCGI_RESPONDER) {
+rv = add_request_body(r, r-pool, output_brigade, bucket_ctx);
+if (rv) {
+return rv;
+}
+}
+
+/* The eos bucket now */
+bucket_eos = apr_bucket_eos_create(r-connection-bucket_alloc);
+APR_BRIGADE_INSERT_TAIL(output_brigade, bucket_eos);
+
 /* Write output_brigade to fastcgi server */
 if ((rv = proc_write_ipc(bucket_ctx-ipc,
  output_brigade)) != APR_SUCCESS) {
@@ -304,6 +320,7 @@ handle_request_ipc(request_rec *r, int role,
 bucket_ctx-has_error = 1;
 return HTTP_INTERNAL_SERVER_ERROR;
 }
+apr_brigade_cleanup(output_brigade);

 /* Create brigade */
 brigade_stdout =
@@ -522,7 +539,8 @@ handle_request(request_rec * r, int role, fcgid_cm
 }

 static int add_request_body(request_rec *r, apr_pool_t *request_pool,
-apr_bucket_brigade *output_brigade)
+apr_bucket_brigade *output_brigade,
+fcgid_bucket_ctx *bucket_ctx)
 {
 apr_bucket *bucket_input, *bucket_header;
 apr_file_t *fd = NULL;
@@ -563,8 +581,6 @@ static int add_request_body(request_rec *r, apr_po
 return HTTP_INTERNAL_SERVER_ERROR;
 }

-
-
 while ((bucket_input = APR_BRIGADE_FIRST(input_brigade)) !=
APR_BRIGADE_SENTINEL(input_brigade)) {
 const char *data;
 apr_size_t len;
@@ -615,7 +631,9 @@ static int add_request_body(request_rec *r, apr_po
 return HTTP_INTERNAL_SERVER_ERROR;
 }

-if (request_size  sconf-max_mem_request_len) {
+if (sconf-stream_request_body) {
+bucket_stdin = bucket_input;
+} else if (request_size  sconf-max_mem_request_len) {
 apr_size_t wrote_len;
 static const char *fd_key = fcgid_fd;

@@ -701,6 +719,16 @@ static int add_request_body(request_rec *r, apr_po
 }
 APR_BRIGADE_INSERT_TAIL(output_brigade, bucket_header);
 APR_BRIGADE_INSERT_TAIL(output_brigade, bucket_stdin);
+
+if (sconf-stream_request_body) {
+/* Write output_brigade to fastcgi server */
+if ((rv = proc_write_ipc(bucket_ctx-ipc,
+ output_brigade)) != APR_SUCCESS) {
+bucket_ctx-has_error = 1;
+return HTTP_INTERNAL_SERVER_ERROR;
+}
+apr_brigade_cleanup(output_brigade);
+}
 }

 apr_brigade_cleanup(input_brigade);
@@ -731,7 +759,6 @@ static int add_request_body(request_rec *r, apr_po
 int bridge_request(request_rec * r, int role, fcgid_cmd_conf *cmd_conf)
 {
 

Re: mod_fcgid and streaming large request bodies

[Jeff Trawick]

On Fri, Dec 27, 2013 at 6:14 PM, Justin Erenkrantz jus...@erenkrantz.comwrote:

 On Fri, Dec 27, 2013 at 8:49 AM, Justin Erenkrantz
 jus...@erenkrantz.com wrote:
  Anyway, I'm diving into the code a bit - but, I figured it might be
 useful
  to see if anyone else has any thoughts about how to handle large request
  bodies with mod_fcgid.

 Here's a first-cut patch that compiles at least.  Cheers.  -- justin

 Add FcgidStreamRequestBody directive to send request body as it arrives
 without
 storing it in memory or on disk.


It would be quite valuable if there is a limit on how much can be pre-read
(0 for pure streaming).  Pre-reading the request body reduces the number of
application processes or threads required, and they are usually fatter
(sometimes dramatically so) than httpd workers.  While the current
implementation is truly ridiculous, there's some goodness that could be
kept.  I'll try to look through this in the next few days and see if it is
practical to keep the best of both versions.  If no round tuits are
available we should at least proceed with this, perhaps changing the
directive syntax to allow a hybrid implementation in the future with no
legacy cruft.  (Wild suggestion: FcgiPreReadRequestBody
Off|On|Unlimited|K, where On is defaultK)


 * modules/fcgid/mod_fcgid.c
   (fcgid_cmds): Add FcgidStreamRequestBody directive.
 * modules/fcgid/fcgid_conf.h
   (fcgid_server_conf): Add stream_request_body/stream_request_body_set
 flags.
 * modules/fcgid/fcgid_conf.c
   (DEFAULT_STREAM_REQUEST_BODY): Set to 0.
   (create_fcgid_server_config): Init stream_request_body flag.
   (merge_fcgid_server_config): Merge new stream_request_body flag.
   (set_stream_request_body): New config helper.
 * modules/fcgid/fcgid_bridge.c
   (add_request_body): Add a forward declaration to reduce diff (for now);
   take fcgid_bucket_ctx param; if stream_request_body is set, don't copy
   bucket and then call proc_write_ipc to send out that brigade and clear it
   out before next loop iteration.
   (bridge_request): Delay reading the request body until later.
   (handle_request_ipc): Move add_request_body call to here and append the
 EOS
   bucket before we do the final write of the request body.

 Index: modules/fcgid/fcgid_bridge.c
 ===
 --- modules/fcgid/fcgid_bridge.c (revision 1553671)
 +++ modules/fcgid/fcgid_bridge.c (working copy)
 @@ -287,6 +287,10 @@ static int getsfunc_fcgid_BRIGADE(char *buf, int l
  return done;
  }

 +static int add_request_body(request_rec *r, apr_pool_t *request_pool,
 +apr_bucket_brigade *output_brigade,
 +fcgid_bucket_ctx *bucket_ctx);
 +
  static int
  handle_request_ipc(request_rec *r, int role,
 apr_bucket_brigade *output_brigade,
 @@ -295,9 +299,21 @@ handle_request_ipc(request_rec *r, int role,
  int cond_status;
  apr_status_t rv;
  apr_bucket_brigade *brigade_stdout;
 +apr_bucket *bucket_eos;
  char sbuf[MAX_STRING_LEN];
  const char *location;

 +if (role == FCGI_RESPONDER) {
 +rv = add_request_body(r, r-pool, output_brigade, bucket_ctx);
 +if (rv) {
 +return rv;
 +}
 +}
 +
 +/* The eos bucket now */
 +bucket_eos = apr_bucket_eos_create(r-connection-bucket_alloc);
 +APR_BRIGADE_INSERT_TAIL(output_brigade, bucket_eos);
 +
  /* Write output_brigade to fastcgi server */
  if ((rv = proc_write_ipc(bucket_ctx-ipc,
   output_brigade)) != APR_SUCCESS) {
 @@ -304,6 +320,7 @@ handle_request_ipc(request_rec *r, int role,
  bucket_ctx-has_error = 1;
  return HTTP_INTERNAL_SERVER_ERROR;
  }
 +apr_brigade_cleanup(output_brigade);

  /* Create brigade */
  brigade_stdout =
 @@ -522,7 +539,8 @@ handle_request(request_rec * r, int role, fcgid_cm
  }

  static int add_request_body(request_rec *r, apr_pool_t *request_pool,
 -apr_bucket_brigade *output_brigade)
 +apr_bucket_brigade *output_brigade,
 +fcgid_bucket_ctx *bucket_ctx)
  {
  apr_bucket *bucket_input, *bucket_header;
  apr_file_t *fd = NULL;
 @@ -563,8 +581,6 @@ static int add_request_body(request_rec *r, apr_po
  return HTTP_INTERNAL_SERVER_ERROR;
  }

 -
 -
  while ((bucket_input = APR_BRIGADE_FIRST(input_brigade)) !=
 APR_BRIGADE_SENTINEL(input_brigade)) {
  const char *data;
  apr_size_t len;
 @@ -615,7 +631,9 @@ static int add_request_body(request_rec *r, apr_po
  return HTTP_INTERNAL_SERVER_ERROR;
  }

 -if (request_size  sconf-max_mem_request_len) {
 +if (sconf-stream_request_body) {
 +bucket_stdin = bucket_input;
 +} else if (request_size  sconf-max_mem_request_len) {
  apr_size_t wrote_len;
   

Re: mod_fcgid and streaming large request bodies

[Justin Erenkrantz]

On Fri, Dec 27, 2013 at 6:42 PM, Jeff Trawick traw...@gmail.com wrote:
 It would be quite valuable if there is a limit on how much can be pre-read
 (0 for pure streaming).  Pre-reading the request body reduces the number of
 application processes or threads required, and they are usually fatter
 (sometimes dramatically so) than httpd workers.  While the current
 implementation is truly ridiculous, there's some goodness that could be
 kept.  I'll try to look through this in the next few days and see if it is
 practical to keep the best of both versions.  If no round tuits are
 available we should at least proceed with this, perhaps changing the
 directive syntax to allow a hybrid implementation in the future with no
 legacy cruft.  (Wild suggestion: FcgiPreReadRequestBody
 Off|On|Unlimited|K, where On is defaultK)

I definitely agree that doing some type of pre-read may be often
useful before we start sending stuff via FastCGI.  Yet, the current
code structure doesn't lend itself to that very well - I guess what we
could do is convert the IPC to be opened lazily rather than always
open it.  If we do that, then I think the code changes would be
manageable.  (In our use case, reading the first say 64KB into RAM is
fine, but we definitely want to avoid spooling to disk.)

I'll see if I can find some round tuits...

Cheers.  -- justin

Re: mod_fcgid and mpm-itk

[William A. Rowe Jr.]

On Sun, 28 Jul 2013 15:17:41 +0200
AZ 9901 az9...@gmail.com wrote:

 Hello all,
 
 I run into troubles using mod_fcgid (for PHP) and mpm-itk together.
 
 As soon as I add the ITK directive AssignUserID to my vhost,
 specifying a different user than the global Apache one, mod_fcgid
 fails returning the following error : [Sun Jul 28 15:09:37 2013]
 [emerg] [client x.x.x.x] (13)Permission denied: mod_fcgid: can't lock
 process table in pid 1973
 
 I noted that the socket file is created with permissions 700.
 So, as a test, I tried the following, in fcgid_proc_unix.c, I
 replaced : rv = apr_file_perms_set(unix_addr.sun_path,
 
 APR_FPROT_UREAD|APR_FPROT_UWRITE|APR_FPROT_UEXECUTE);
 by :
 rv = apr_file_perms_set(unix_addr.sun_path,
 
 APR_FPROT_UREAD|APR_FPROT_UWRITE|APR_FPROT_UEXECUTE|APR_FPROT_GREAD|APR_FPROT_GWRITE|APR_FPROT_GEXECUTE|APR_FPROT_WREAD|APR_FPROT_WWRITE|APR_FPROT_WEXECUTE);
 
 The socket is then created with 777 permissions, but I does not solve
 the issue.
 
 Certainly that the issue is somewhere else, perhaps Apache tries to
 instance mod_fcgid with global Apache user instead of ITK defined
 user.
 
 Could you help me with that ?

You might also try any mpm-itk user list since the behavior is going
to be mpm-specific.  And please, fill us in if you learn more!

Re: Re: Re: mod_fcgid concurrency bottleneck, issue#53693

[pqf]

Hi, all
I took some times to setup a stress test for mod_fcgid, to see is there any 
other concurrency bottleneck left, it seems fine to me.

I did a stress test with callgrind on mod_fcgid, this is the big picture: 
http://people.apache.org/~pqf/mod_fcgid_performance/performance.png and this is 
the output of callgrind: 
http://people.apache.org/~pqf/mod_fcgid_performance/callgrind.out
As we can see, most cpu time are used in ap_create_environment() + 
init_environment(). ap_create_environment() copy environment vars to a char[][] 
buffer, and init_environment() copy char[][] to a char[] buffer for FCGI 
protocol. I have to keep using ap_create_environment() to be compatible with 
other cgi modules(cgid, cgi...), I will not shortcut these two calls for less 
cpu utils.

And I make a strace trace to the system calls, to make sure there is no obvious 
IO throughput bottleneck, and no luck either

4622  13:17:16.737888 ... futex resumed ) = 0
4622  13:17:16.737920 futex(0x95ce65c, FUTEX_WAKE, 1 unfinished ...
4622  13:17:16.738038 ... futex resumed ) = 0
4622  13:17:16.740735 gettimeofday({1346217436, 740787}, NULL) = 0
4622  13:17:16.740835 getsockname(12, {sa_family=AF_INET6, 
sin6_port=htons(8080), inet_pton(AF_INET6, :::127.0.0.1, si
n6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [28]) = 0
4622  13:17:16.741008 fcntl64(12, F_GETFL) = 0x2 (flags O_RDWR)
4622  13:17:16.741056 fcntl64(12, F_SETFL, O_RDWR|O_NONBLOCK) = 0
4622  13:17:16.741824 gettimeofday({1346217436, 741849}, NULL) = 0
4622  13:17:16.741890 gettimeofday({1346217436, 741911}, NULL) = 0
4622  13:17:16.741969 read(12, GET /fcgi-bin/foo.pl HTTP/1.0\r\nH..., 8000) = 
97
4622  13:17:16.742049 gettimeofday({1346217436, 742070}, NULL) = 0
4622  13:17:16.742108 gettimeofday({1346217436, 742129}, NULL) = 0
4622  13:17:16.742165 gettimeofday({1346217436, 742185}, NULL) = 0
4622  13:17:16.74 gettimeofday({1346217436, 742242}, NULL) = 0
4622  13:17:16.742278 gettimeofday({1346217436, 742299}, NULL) = 0
4622  13:17:16.742345 gettimeofday({1346217436, 742367}, NULL) = 0
4622  13:17:16.742404 gettimeofday({1346217436, 742424}, NULL) = 0
4622  13:17:16.742467 stat64(/usr/local/apache2/htdocs/fcgi-bin/foo.pl, 
{st_mode=S_IFREG|0755, st_size=229, ...}) = 0
4622  13:17:16.742697 semop(4292662, 0x17234c, 1) = 0
4622  13:17:16.742750 semop(4292662, 0x172358, 1) = 0
4622  13:17:16.742804 socket(PF_FILE, SOCK_STREAM, 0) = 13
4622  13:17:16.742864 connect(13, {sa_family=AF_FILE, 
path=/usr/local/apache2/logs/fcgidsock/4617.11}, 110) = 0
4622  13:17:16.743028 fcntl64(13, F_GETFL) = 0x2 (flags O_RDWR)
4622  13:17:16.743074 fcntl64(13, F_SETFL, O_RDWR|O_NONBLOCK) = 0
4622  13:17:16.743118 gettimeofday({1346217436, 743138}, NULL) = 0
4622  13:17:16.743174 writev(13, [{\1\1\0\1\0\10\0\0, 8}, 
{\0\1\0\0\0\0\0\0, 8}, {\1\4\0\1\3U\0\0, 8}, {\t\30UNIQUE_ID
UD2l3MCoqlsAABILpXoAA..., 853}, {\1\4\0\1\0\0\0\0, 8}, {\1\5\0\1\0\0\0\0, 
8}], 6) = 893
4622  13:17:16.743337 read(13, 0xa6f4c5b8, 8192) = -1 EAGAIN (Resource 
temporarily unavailable)
4622  13:17:16.743386 poll([{fd=13, events=POLLIN, revents=POLLIN}], 1, 4) 
= 1
4622  13:17:16.743789 read(13, \1\6\0\1\3\266\2\0Content-Type: text/plain..., 
8192) = 984
4622  13:17:16.743879 gettimeofday( unfinished ...
4622  13:17:16.743968 ... gettimeofday resumed {1346217436, 743943}, NULL) = 0
4622  13:17:16.744037 close(13) = 0
4622  13:17:16.744101 semop(4292662, 0x17234c, 1) = 0
4622  13:17:16.744149 semop(4292662, 0x172358, 1) = 0
4622  13:17:16.744209 gettimeofday({1346217436, 744231}, NULL) = 0
4622  13:17:16.744266 gettimeofday({1346217436, 744287}, NULL) = 0
4622  13:17:16.744343 writev(12, [{HTTP/1.1 200 OK\r\nDate: Wed, 29 A..., 
154}, {CONTEXT_DOCUMENT_ROOT=\/usr/loca..., 92
4}], 2 unfinished ...
4622  13:17:16.745033 ... writev resumed ) = 1078
4622  13:17:16.745076 gettimeofday({1346217436, 745098}, NULL) = 0
4622  13:17:16.745148 times({tms_utime=1416, tms_stime=1464, tms_cutime=0, 
tms_cstime=0}) = 499844951
4622  13:17:16.745255 gettimeofday( unfinished ...
4622  13:17:16.745800 ... gettimeofday resumed {1346217436, 745791}, NULL) = 0
4622  13:17:16.746013 shutdown(12, 1 /* send */ unfinished ...
4622  13:17:16.746554 ... shutdown resumed ) = 0
4622  13:17:16.746625 gettimeofday( unfinished ...
4622  13:17:16.746684 ... gettimeofday resumed {1346217436, 746658}, NULL) = 0
4622  13:17:16.746765 epoll_ctl(11, EPOLL_CTL_ADD, 12, {EPOLLIN, 
{u32=158210764, u64=158210764}} unfinished ...
4622  13:17:16.746833 ... epoll_ctl resumed ) = 0
4622  13:17:16.747587 gettimeofday( unfinished ...
4622  13:17:16.747904 ... gettimeofday resumed {1346217436, 747649}, NULL) = 0
4622  13:17:16.748672 futex(0x95ce690, FUTEX_WAIT, 176390, NULL unfinished ...



2012-09-05



pqf



发件人：pqf
发送时间：2012-08-28 09:34
主题：回复: Re: Re: Re: mod_fcgid concurrency bottleneck, issue#53693
收件人：devdev@httpd.apache.org
抄送：

So what can mod_fcgid do in this overloaded?
1. mod_fcgid get a request
2. mod_fcgid can't apply a free slot

Re: Re: Re: Re: mod_fcgid concurrency bottleneck, issue#53693

[Lazy]

2012/8/28 pqf p...@mailtech.cn:
 So what can mod_fcgid do in this overloaded?
 1. mod_fcgid get a request
 2. mod_fcgid can't apply a free slot of FCGI handler
 3. mod_fcgid send a spawn request to PM
 4. PM deny the request(for too much process already)
 5. Now
for( i=1; i64; i++)
   {
  a) mod_fcgid delay a while, and then send another spawn request to PM
 and try apply free slot again.
  b) mod_fcgid send another spawn request at once, even the last request
 is denied.
  c) ??
 (now is a, b maybe not a good idea, any new idea?)
   }

 I think the bottleneck is too much request, too less FCGI handler. httpd(or
 mod_fcgid) either drop client connections or delay a while, there is no
 other way out?

My idea is to add a availability number to each class. If a wait fails
it will be decreased or increased if wait is successful. Let's say we
want to wait max 100 times 250ms


int connected=0;

   for(i=0; !connected  i = (get_clas_avail()/MAX_AVAIL)*100; i++) {
/* Apply a process slot */
bucket_ctx-procnode = apply_free_procnode(r, fcgi_request);

/* Send a spawn request if I can't get a process slot */
/* procmgr_send_spawn_cmd() return APR_SUCCESS if a
process is created */
if( !bucket_ctx-procnode 
(procmgr_send_spawn_cmd(fcgi_request, r) != APR_SUCCESS) )
apr_sleep(apr_time_from_msec(250));
else
/* Apply a process slot */
bucket_ctx-procnode = apply_free_procnode(r, fcgi_request);

if (bucket_ctx-procnode) {
if (proc_connect_ipc(bucket_ctx-procnode,
 bucket_ctx-ipc) != APR_SUCCESS) {
proc_close_ipc(bucket_ctx-ipc);
bucket_ctx-procnode-diewhy = FCGID_DIE_CONNECT_ERROR;
return_procnode(r, bucket_ctx-procnode, 1 /* has
error */ );
bucket_ctx-procnode = NULL;
decrease_avail();
}else {
increase_avail();
connected=1;
}
}
}

if (!connected) {
decrease_avail();
ap_log_rerror(APLOG_MARK, APLOG_WARNING, 0, r,
  mod_fcgid: can't apply process slot for %s,
  cmd_conf-cmdline);
return HTTP_SERVICE_UNAVAILABLE;
}

decrease_avail() might halve availability each time called.


Availability should be dynamic maybe controlled by processmanager and
returned to the threads handling connections by
procmgr_send_spawn_cmd(), it can depend on total number of denied
spawn requests for specific class in a similar way as score, without
this connections will be 503 not sooner then 25 seconds, which is
still IMHO to long. Another improvement would bo to make wait time
shorter for not overloaded classes to keep the penalty of denied spawn
as low as possible.


I plan to work on that later.



 Another question. Is it necessary to call procmgr_init_spawn_cmd() from
 inside the for loop ?
 I took a brief look, it seems not necessary. I will move it out of loop and
 test.

 2012-08-28
 
 pqf
 
 发件人：Lazy
 发送时间：2012-08-27 21:47
 主题：Re: Re: Re: mod_fcgid concurrency bottleneck, issue#53693
 收件人：devdev@httpd.apache.org
 抄送：

 2012/8/16 pqf p...@mailtech.cn:
 How about this:
 1. procmgr_post_spawn_cmd() now return a status code from PM, so process
 handler now know the spawn request is denyed or not.
 2. if a new process is created, no sleep is needed.
 3. if no process is created, sleep a while

 sorry for the late reply,

 in the old code there ware no sleep() between procmgr_post_spawn_cmd()
 and apply_free_procnode()

 sleep() was invoked only if there ware no free procnode.

 This happened only if we ware denied spawning new process or in some
 cases if some other thread managed to use that procnode before us.

 Your change adresses cases if some other thread stole our newly
 spawned fcgi process, old code was waiting 1s before trying to spawn
 another/recheck, new code doesn't, I guess this is the orginal issue
 in stress tests when total number of simultaneous connections doesn't
 exceed max fcgi processes. But when spawning is denied recovery time
 is still long 1s.


 I was refering to cases when spawn is denied.

 If a vhost is overloaded or someone added sleep(60) in the code,
 mod_fcgid blocks on all request to that vhost
 for over a minute and it is possible to occupy 1000 threads using
 under 20 new connections to slow vhost
 per second. This can be mitingated by adding avaiability which will
 impact time spend on waiting for free process. Overloaded vhost will
 start to drop connections faster preventing the web-server reaching
 MaxClients
 limit.

 Another question. Is it necessary to call procmgr_init_spawn_cmd()
 from inside the for loop ?



 2012-08-16
 
 pqf

Re: Re: Re: mod_fcgid concurrency bottleneck, issue#53693

[Lazy]

2012/8/16 pqf p...@mailtech.cn:
 How about this:
 1. procmgr_post_spawn_cmd() now return a status code from PM, so process
 handler now know the spawn request is denyed or not.
 2. if a new process is created, no sleep is needed.
 3. if no process is created, sleep a while

sorry for the late reply,

in the old code there ware no sleep() between procmgr_post_spawn_cmd()
and apply_free_procnode()

sleep() was invoked only if there ware no free procnode.

This happened only if we ware denied spawning new process or in some
cases if some other thread managed to use that procnode before us.

Your change adresses cases if some other thread stole our newly
spawned fcgi process, old code was waiting 1s before trying to spawn
another/recheck, new code doesn't, I guess this is the orginal issue
in stress tests when total number of simultaneous connections doesn't
exceed max fcgi processes. But when spawning is denied recovery time
is still long 1s.


I was refering to cases when spawn is denied.

If a vhost is overloaded or someone added sleep(60) in the code,
mod_fcgid blocks on all request to that vhost
for over a minute and it is possible to occupy 1000 threads using
under 20 new connections to slow vhost
per second. This can be mitingated by adding avaiability which will
impact time spend on waiting for free process. Overloaded vhost will
start to drop connections faster preventing the web-server reaching
MaxClients
limit.

Another question. Is it necessary to call procmgr_init_spawn_cmd()
from inside the for loop ?



 2012-08-16
 
 pqf
 
 发件人：Lazy
 发送时间：2012-08-16 16:47
 主题：Re: Re: mod_fcgid concurrency bottleneck, issue#53693
 收件人：devdev@httpd.apache.org
 抄送：

 2012/8/16 pqf p...@mailtech.cn:
 Hi, Michal
 My solution do add availability to each class, which is the
 procmgr_post_spawn_cmd() call in each loop do.
 The sleep() call is intrudused for a stress test without warm up time, in
 this case, mod_fcgid will create more processes than a slow start one(each
 process handler can't apply a free slot on the very begining, so send a
 request to process manager to create one, it's easy to reach the max # of
 process limit while httpd startup, but the idle process will be killed
 later), the sleep() call is a little like a server side warm up delay.
 But since someone said remove this sleep(), the server work fine without
 bottleneck(Maybe he didn't notise the warm up issue?), so I thought remove
 the sleep() is a good idea. But reduce the time of sleep() is fine to me
 too.

 I was referring to the case where all processes are busy, without
 sleep(), handle_request() wil quickly send spawn requsts, whith will
 be denyed by process menager, with sleep() handle_request() will
 always wait quite a long time,
 occupying slots

 --
 Michal Grzedzicki

回复: Re: Re: Re: mod_fcgid concurrency bottleneck, issue#53693

[pqf]

So what can mod_fcgid do in this overloaded?
1. mod_fcgid get a request
2. mod_fcgid can't apply a free slot of FCGI handler
3. mod_fcgid send a spawn request to PM
4. PM deny the request(for too much process already)
5. Now
   for( i=1; i64; i++)
  {
 a) mod_fcgid delay a while, and then send another spawn request to PM and 
try apply free slot again.
 b) mod_fcgid send another spawn request at once, even the last request is 
denied.
 c) ??
(now is a, b maybe not a good idea, any new idea?)
  }

I think the bottleneck is too much request, too less FCGI handler. httpd(or 
mod_fcgid) either drop client connections or delay a while, there is no other 
way out?


Another question. Is it necessary to call procmgr_init_spawn_cmd() from inside 
the for loop ? 
I took a brief look, it seems not necessary. I will move it out of loop and 
test.

2012-08-28



pqf



发件人：Lazy
发送时间：2012-08-27 21:47
主题：Re: Re: Re: mod_fcgid concurrency bottleneck, issue#53693
收件人：devdev@httpd.apache.org
抄送：

2012/8/16 pqf p...@mailtech.cn: 
 How about this: 
 1. procmgr_post_spawn_cmd() now return a status code from PM, so process 
 handler now know the spawn request is denyed or not. 
 2. if a new process is created, no sleep is needed. 
 3. if no process is created, sleep a while 

sorry for the late reply, 

in the old code there ware no sleep() between procmgr_post_spawn_cmd() 
and apply_free_procnode() 

sleep() was invoked only if there ware no free procnode. 

This happened only if we ware denied spawning new process or in some 
cases if some other thread managed to use that procnode before us. 

Your change adresses cases if some other thread stole our newly 
spawned fcgi process, old code was waiting 1s before trying to spawn 
another/recheck, new code doesn't, I guess this is the orginal issue 
in stress tests when total number of simultaneous connections doesn't 
exceed max fcgi processes. But when spawning is denied recovery time 
is still long 1s. 


I was refering to cases when spawn is denied. 

If a vhost is overloaded or someone added sleep(60) in the code, 
mod_fcgid blocks on all request to that vhost 
for over a minute and it is possible to occupy 1000 threads using 
under 20 new connections to slow vhost 
per second. This can be mitingated by adding avaiability which will 
impact time spend on waiting for free process. Overloaded vhost will 
start to drop connections faster preventing the web-server reaching 
MaxClients 
limit. 

Another question. Is it necessary to call procmgr_init_spawn_cmd() 
from inside the for loop ? 


 
 2012-08-16 
  
 pqf 
  
 发件人：Lazy 
 发送时间：2012-08-16 16:47 
 主题：Re: Re: mod_fcgid concurrency bottleneck, issue#53693 
 收件人：devdev@httpd.apache.org 
 抄送： 
 
 2012/8/16 pqf p...@mailtech.cn: 
 Hi, Michal 
 My solution do add availability to each class, which is the 
 procmgr_post_spawn_cmd() call in each loop do. 
 The sleep() call is intrudused for a stress test without warm up time, in 
 this case, mod_fcgid will create more processes than a slow start one(each 
 process handler can't apply a free slot on the very begining, so send a 
 request to process manager to create one, it's easy to reach the max # of 
 process limit while httpd startup, but the idle process will be killed 
 later), the sleep() call is a little like a server side warm up delay. 
 But since someone said remove this sleep(), the server work fine without 
 bottleneck(Maybe he didn't notise the warm up issue?), so I thought remove 
 the sleep() is a good idea. But reduce the time of sleep() is fine to me 
 too. 
 
 I was referring to the case where all processes are busy, without 
 sleep(), handle_request() wil quickly send spawn requsts, whith will 
 be denyed by process menager, with sleep() handle_request() will 
 always wait quite a long time, 
 occupying slots 
 
 -- 
 Michal Grzedzicki 
 
 

Re: mod_fcgid concurrency bottleneck, issue#53693

[William A. Rowe Jr.]

On 8/15/2012 8:23 PM, pqf wrote:
 But since someone said remove this sleep(), the server work fine without 
 bottleneck(Maybe
 he didn't notise the warm up issue?), so I thought remove the sleep() is a 
 good idea. But
 reduce the time of sleep() is fine to me too.
 Any other idea?

Can we introduce a blocking call on 'pipe ready' based on the child spawn
being ready to accept data?  The equivalent of a ping-test?

Re: Re: mod_fcgid concurrency bottleneck, issue#53693

[pqf]

procmgr_post_spawn_cmd(maybe it should be renamed to procmgr_send_spawn_cmd) 
will keep blocking while PM is creating a new process, after 
procmgr_post_spawn_cmd() return, there should be a process ready there waiting 
for accept.

2012-08-16



pqf



发件人：William A. Rowe Jr.
发送时间：2012-08-16 15:23
主题：Re: mod_fcgid concurrency bottleneck, issue#53693
收件人：devdev@httpd.apache.org
抄送：pqfp...@mailtech.cn

On 8/15/2012 8:23 PM, pqf wrote: 
 But since someone said remove this sleep(), the server work fine without 
 bottleneck(Maybe 
 he didn't notise the warm up issue?), so I thought remove the sleep() is a 
 good idea. But 
 reduce the time of sleep() is fine to me too. 
 Any other idea? 

Can we introduce a blocking call on 'pipe ready' based on the child spawn 
being ready to accept data?  The equivalent of a ping-test? 

Re: Re: mod_fcgid concurrency bottleneck, issue#53693

[Lazy]

2012/8/16 pqf p...@mailtech.cn:
 Hi, Michal
 My solution do add availability to each class, which is the
 procmgr_post_spawn_cmd() call in each loop do.
 The sleep() call is intrudused for a stress test without warm up time, in
 this case, mod_fcgid will create more processes than a slow start one(each
 process handler can't apply a free slot on the very begining, so send a
 request to process manager to create one, it's easy to reach the max # of
 process limit while httpd startup, but the idle process will be killed
 later), the sleep() call is a little like a server side warm up delay.
 But since someone said remove this sleep(), the server work fine without
 bottleneck(Maybe he didn't notise the warm up issue?), so I thought remove
 the sleep() is a good idea. But reduce the time of sleep() is fine to me
 too.

I was referring to the case where all processes are busy, without
sleep(), handle_request() wil quickly send spawn requsts, whith will
be denyed by process menager, with sleep() handle_request() will
always wait quite a long time,
occupying slots

-- 
Michal Grzedzicki

回复: Re: Re: mod_fcgid concurrency bottleneck, issue#53693

[pqf]

How about this:
1. procmgr_post_spawn_cmd() now return a status code from PM, so process 
handler now know the spawn request is denyed or not.
2. if a new process is created, no sleep is needed.
3. if no process is created, sleep a while

2012-08-16



pqf



发件人：Lazy
发送时间：2012-08-16 16:47
主题：Re: Re: mod_fcgid concurrency bottleneck, issue#53693
收件人：devdev@httpd.apache.org
抄送：

2012/8/16 pqf p...@mailtech.cn: 
 Hi, Michal 
 My solution do add availability to each class, which is the 
 procmgr_post_spawn_cmd() call in each loop do. 
 The sleep() call is intrudused for a stress test without warm up time, in 
 this case, mod_fcgid will create more processes than a slow start one(each 
 process handler can't apply a free slot on the very begining, so send a 
 request to process manager to create one, it's easy to reach the max # of 
 process limit while httpd startup, but the idle process will be killed 
 later), the sleep() call is a little like a server side warm up delay. 
 But since someone said remove this sleep(), the server work fine without 
 bottleneck(Maybe he didn't notise the warm up issue?), so I thought remove 
 the sleep() is a good idea. But reduce the time of sleep() is fine to me 
 too. 

I was referring to the case where all processes are busy, without 
sleep(), handle_request() wil quickly send spawn requsts, whith will 
be denyed by process menager, with sleep() handle_request() will 
always wait quite a long time, 
occupying slots 

--  
Michal Grzedzicki 

Re: mod_fcgid concurrency bottleneck, issue#53693

[pqf]

Hi, all
I prefer the following solution, can anyone review it? 
procmgr_post_spawn_cmd() will be blocked until process manager create a new 
fcgid process, the worst case is someone else take the new created process 
before I do, and I have to post another spawn command to PM again. The extreme 
case is loop FCGID_APPLY_TRY_COUNT but get no process slot.


Index: fcgid_bridge.c
===
--- fcgid_bridge.c  (revision 1373226)
+++ fcgid_bridge.c  (working copy)
@@ -30,7 +30,7 @@
 #include fcgid_spawn_ctl.h
 #include fcgid_protocol.h
 #include fcgid_bucket.h
-#define FCGID_APPLY_TRY_COUNT 2
+#define FCGID_APPLY_TRY_COUNT 4
 #define FCGID_REQUEST_COUNT 32
 #define FCGID_BRIGADE_CLEAN_STEP 32
 
@@ -447,19 +447,13 @@
 if (bucket_ctx-procnode)
 break;
 
-/* Avoid sleeping the very first time through if there are no
-   busy processes; the problem is just that we haven't spawned
-   anything yet, so waiting is pointless */
-if (i  0 || j  0 || count_busy_processes(r, fcgi_request)) {
-apr_sleep(apr_time_from_sec(1));
-
-bucket_ctx-procnode = apply_free_procnode(r, fcgi_request);
-if (bucket_ctx-procnode)
-break;
-}
-
 /* Send a spawn request if I can't get a process slot */
 procmgr_post_spawn_cmd(fcgi_request, r);
+
+/* Try again */
+bucket_ctx-procnode = apply_free_procnode(r, fcgi_request);
+if (bucket_ctx-procnode)
+break;
 }
 
 /* Connect to the fastcgi server */
2012-08-15



pqf



发件人：Mike M
发送时间：2012-08-15 04:29
主题：mod_fcgid concurrency bottleneck, issue#53693
收件人：devdev@httpd.apache.org
抄送：

Ahoy! 

With mod_fcgid I've observed a performance bottleneck in which concurrent 
requests to a virtualhost where the number of concurrent requests exceed 
the number of available fcgid-spawned processes result in a significant 
delay in responding to requests. 

Details of this situation, how to reproduce, various scenarios tried, etc 
are detailed in the issue submission [1]. 

In file  modules/fcgid/fcgid_bridge.c, there is this section of code: 

if (i  0 || j  0 || count_busy_processes(r, fcgi_request)) { 
apr_sleep(apr_time_from_sec(1)); 

If I change the sleep time from 1 second to 0 seconds (or, comment out 
this section of code entirely), the bottleneck goes away.  It seems like 
the more appropriate action here would be to turn the sleep time into a 
configurable value, with the current 1s value as the default. 

I presume this code is meant to be a way to help defer spawn requests so 
that a server is not over-whelmed with spin-up related I/O. 

My questions are: 
- Is this intended to be a throttling mechanism and if so, is there a more 
efficient way to handle this throttling? 
- If I outright reduce that sleep delay to 0 (or comment out this code), 
what potential or expected problems am I introducing to mod_fcgid? 


[1] https://issues.apache.org/bugzilla/show_bug.cgi?id=53693 


--- 
Mike M 

You can't learn in school what the world is going to do next year. 

Re: mod_fcgid concurrency bottleneck, issue#53693

[Lazy]

2012/8/15 pqf p...@mailtech.cn:
 Hi, all
 I prefer the following solution, can anyone review it?
 procmgr_post_spawn_cmd() will be blocked until process manager create a new
 fcgid process, the worst case is someone else take the new created process
 before I do, and I have to post another spawn command to PM again. The
 extreme case is loop FCGID_APPLY_TRY_COUNT but get no process slot.


 Index: fcgid_bridge.c
 ===
 --- fcgid_bridge.c  (revision 1373226)
 +++ fcgid_bridge.c  (working copy)
 @@ -30,7 +30,7 @@
  #include fcgid_spawn_ctl.h
  #include fcgid_protocol.h
  #include fcgid_bucket.h
 -#define FCGID_APPLY_TRY_COUNT 2
 +#define FCGID_APPLY_TRY_COUNT 4
  #define FCGID_REQUEST_COUNT 32
  #define FCGID_BRIGADE_CLEAN_STEP 32

 @@ -447,19 +447,13 @@
  if (bucket_ctx-procnode)
  break;

 -/* Avoid sleeping the very first time through if there are no
 -   busy processes; the problem is just that we haven't spawned
 -   anything yet, so waiting is pointless */
 -if (i  0 || j  0 || count_busy_processes(r, fcgi_request)) {
 -apr_sleep(apr_time_from_sec(1));
 -
 -bucket_ctx-procnode = apply_free_procnode(r,
 fcgi_request);
 -if (bucket_ctx-procnode)
 -break;
 -}
 -
  /* Send a spawn request if I can't get a process slot */
  procmgr_post_spawn_cmd(fcgi_request, r);
 +
 +/* Try again */
 +bucket_ctx-procnode = apply_free_procnode(r, fcgi_request);
 +if (bucket_ctx-procnode)
 +break;
  }

  /* Connect to the fastcgi server */


if You get rid of sleep apache will not wait for free process if all
of them are busy, this will lead to 503 errors

currently mod_fcgid waits FCGID_APPLY_TRY_COUNT * FCGID_REQUEST_COUNT
* 1 second and this is usually 64 seconds, this means if You have an
overloaded vhost with low FcgidMaxProcessesPerClass it can bring whole
server down, each thread waits 64 seconds so it doesn't take long
before all threads are occupied.

In my setup we lowerd the wait time and FCGID_REQUEST_COUNT to lower
the impact of overloaded class but I think
the best solution will be to add availability to each class. Total
wait time will be related to it (by changing sleep time, and
FCGID_APPLY_TRY_COUNT). If request is unsuccessful availability will
be halved so next time the wait time will be shorter. This way
congested class will get 0% availability, and new connections will
instantly get 503 it there are no free slots. A successful wait will
increase availability.


Regards,

Michal Grzedzicki

Re: Re: mod_fcgid concurrency bottleneck, issue#53693

[pqf]

Hi, Michal
My solution do add availability to each class, which is the 
procmgr_post_spawn_cmd() call in each loop do.
The sleep() call is intrudused for a stress test without warm up time, in this 
case, mod_fcgid will create more processes than a slow start one(each process 
handler can't apply a free slot on the very begining, so send a request to 
process manager to create one, it's easy to reach the max # of process limit 
while httpd startup, but the idle process will be killed later), the sleep() 
call is a little like a server side warm up delay.
But since someone said remove this sleep(), the server work fine without 
bottleneck(Maybe he didn't notise the warm up issue?), so I thought remove the 
sleep() is a good idea. But reduce the time of sleep() is fine to me too.
Any other idea?

Regards,

Ryan Pan

2012-08-16



pqf



发件人：Lazy
发送时间：2012-08-15 21:48
主题：Re: mod_fcgid concurrency bottleneck, issue#53693
收件人：devdev@httpd.apache.org
抄送：

2012/8/15 pqf p...@mailtech.cn: 
 Hi, all 
 I prefer the following solution, can anyone review it? 
 procmgr_post_spawn_cmd() will be blocked until process manager create a new 
 fcgid process, the worst case is someone else take the new created process 
 before I do, and I have to post another spawn command to PM again. The 
 extreme case is loop FCGID_APPLY_TRY_COUNT but get no process slot. 
 
 
 Index: fcgid_bridge.c 
 === 
 --- fcgid_bridge.c  (revision 1373226) 
 +++ fcgid_bridge.c  (working copy) 
 @@ -30,7 +30,7 @@ 
  #include fcgid_spawn_ctl.h 
  #include fcgid_protocol.h 
  #include fcgid_bucket.h 
 -#define FCGID_APPLY_TRY_COUNT 2 
 +#define FCGID_APPLY_TRY_COUNT 4 
  #define FCGID_REQUEST_COUNT 32 
  #define FCGID_BRIGADE_CLEAN_STEP 32 
 
 @@ -447,19 +447,13 @@ 
  if (bucket_ctx-procnode) 
  break; 
 
 -/* Avoid sleeping the very first time through if there are no 
 -   busy processes; the problem is just that we haven't spawned 
 -   anything yet, so waiting is pointless */ 
 -if (i  0 || j  0 || count_busy_processes(r, fcgi_request)) { 
 -apr_sleep(apr_time_from_sec(1)); 
 - 
 -bucket_ctx-procnode = apply_free_procnode(r, 
 fcgi_request); 
 -if (bucket_ctx-procnode) 
 -break; 
 -} 
 - 
  /* Send a spawn request if I can't get a process slot */ 
  procmgr_post_spawn_cmd(fcgi_request, r); 
 + 
 +/* Try again */ 
 +bucket_ctx-procnode = apply_free_procnode(r, fcgi_request); 
 +if (bucket_ctx-procnode) 
 +break; 
  } 
 
  /* Connect to the fastcgi server */ 


if You get rid of sleep apache will not wait for free process if all 
of them are busy, this will lead to 503 errors 

currently mod_fcgid waits FCGID_APPLY_TRY_COUNT * FCGID_REQUEST_COUNT 
* 1 second and this is usually 64 seconds, this means if You have an 
overloaded vhost with low FcgidMaxProcessesPerClass it can bring whole 
server down, each thread waits 64 seconds so it doesn't take long 
before all threads are occupied. 

In my setup we lowerd the wait time and FCGID_REQUEST_COUNT to lower 
the impact of overloaded class but I think 
the best solution will be to add availability to each class. Total 
wait time will be related to it (by changing sleep time, and 
FCGID_APPLY_TRY_COUNT). If request is unsuccessful availability will 
be halved so next time the wait time will be shorter. This way 
congested class will get 0% availability, and new connections will 
instantly get 503 it there are no free slots. A successful wait will 
increase availability. 


Regards, 

Michal Grzedzicki 

Re: mod_fcgid support for ap_meets_conditions()

[Chris Darroch]

William A. Rowe Jr. wrote:


On 6/29/2012 6:25 PM, Chris Darroch wrote:


  If this looks OK, I'd love to commit and move on to the next
patch ... assuming I can keep paddling near the shore and avoid those
dangerous undertows!  Thanks and cheers,


Looks sensible to me.


  Thanks, Bill -- committed with minor edits in r1362533.

Chris.

--
GPG Key ID: 088335A9
GPG Key Fingerprint: 86CD 3297 7493 75BC F820  6715 F54F E648 0883 35A9

Re: mod_fcgid support for ap_meets_conditions()

[William A. Rowe Jr.]

On 6/29/2012 6:25 PM, Chris Darroch wrote:
 
   If this looks OK, I'd love to commit and move on to the next
 patch ... assuming I can keep paddling near the shore and avoid those
 dangerous undertows!  Thanks and cheers,

Looks sensible to me.

Re: mod_fcgid can kill all the services on the server via kill -15 -1

[William A. Rowe Jr.]

On 5/3/2011 11:51 PM, pqf wrote:
 Here is the new patch, anyone review it? I will commmit it if no one respond 
 :)

+1, commit away

Re: Re: mod_fcgid can kill all the services on the server via kill -15 -1

[pqf]

Hi, guys
Here is the new patch, anyone review it? I will commmit it if no one respond :)

Index: fcgid_proc_unix.c
===
--- fcgid_proc_unix.c   (revision 1099314)
+++ fcgid_proc_unix.c   (working copy)
@@ -402,6 +402,7 @@
 procnode-proc_id = tmpproc;
 
 if (rv != APR_SUCCESS) {
+memset(procnode-proc_id, 0, sizeof(procnode-proc_id));
 ap_log_error(APLOG_MARK, APLOG_ERR, rv, procinfo-main_server,
  mod_fcgid: can't run %s, wargv[0]);
 }
@@ -414,6 +415,11 @@
 /* su as root before sending signal, for suEXEC */
 apr_status_t rv;
 
+if (procnode-proc_id.pid == 0) {
+/* procnode-proc_id.pid be 0 while fcgid_create_privileged_process() 
fail */
+return APR_SUCCESS; 
+}
+
 if (ap_unixd_config.suexec_enabled  seteuid(0) != 0) {
 
 /* can't gain privileges to send signal (should not occur); do NOT
@@ -461,6 +467,7 @@
 /* Destroy pool */
 apr_pool_destroy(procnode-proc_pool);
 procnode-proc_pool = NULL;
+memset(procnode-proc_id, 0, sizeof(procnode-proc_id));
 
 return APR_CHILD_DONE;
 }



2011-05-04



Ryan 


发件人: pqfp...@mailtech.cn
发送时间: 2011-04-18 09:58
主 题: Re: mod_fcgid can kill all the services on the server via kill -15 -1
收件人: devdev@httpd.apache.org



Hi, all
Another question, does proc_wait_process() should update procnode-proc_id to 0 
too? or else mod_fcgid may send a signal to another irrelevant process while 
apache is shutting down? I don't follow up mod_fcgid for a while, I just took a 
glance, maybe it's updated somewhere else?
By the way, procnode-proc_id is set to 0 while apache startup, so why not 
update procnode-proc_id to 0 while fcgid_create_privileged_process() is fail? 
And then check magic number 0 rather than both -1 and 0,  in both 
proc_kill_gracefully() and proc_kill_force().

Cheers.
Ryan


 
Hello,


There is a very interesting, and quite a rare bug in mod_fcgid. It is easy to 
reproduce if you can cause fork to fail (which can be done with CloudLinux -- 
if anyone wants to replicate it).


Here is how it works: 
mod_fcgid tries to spawn a new process (proc_spawn_process in 
fcgid_proc_unix.c), but fork returns -1. 
More exactly fcgid_create_privileged_process function call returns error, and 
fills in tmpproc.pid with -1  tmpproc is assiged to procnode-proc_id).


Now, if at the same time service httpd restart is executed, function 
kill_all_subprocess in fcgid_pm_main.c will execute, and it will try to go 
through all procnodes, sending SIGTERM via proc_kill_gracefully, (and then 
SIGKILL via proc_kill_force) to procnode-proc_id.pid
Yet, one procnode will be pointing to procnode-proc_id.pid, causing kill -15 
-1 (kill all).
The end results all services on the server failing, including SSH, apache, 
syslogd, etc..


I guess the problem is really rare for most people. Also it is quite hard to 
diagnose, as it is completely not clear where the signal came from, and it took 
us some time to correlate them with apache restarts.. Yet due to our OS being 
used by shared hosts (where httpd restart is common thing), and our ability to 
limit memory available to processes on per virtual host bases (which causes 
fork to fail once that virtual host reaches memory limit), we see the issue 
quite often.


The solution is quite simple (not sure if it is the best / right solution), in 
file: fcgid_proc_unix.c, in methods proc_kill_gracefully, line:


rv = apr_proc_kill((procnode-proc_id), SIGTERM);


should be changed to:
   if (procnode-proc_id.pid != -1) {
  rv = apr_proc_kill((procnode-proc_id), SIGTERM);
   } else {
  rv = APR_SUCCESS;
   }


Similarly in proc_kill_force
rv = apr_proc_kill((procnode-proc_id), SIGKILL);
should be changed to:
   if (procnode-proc_id.pid != -1) {
  rv = apr_proc_kill((procnode-proc_id), SIGKILL);
   } else {
  rv = APR_SUCCESS;
   }


Regards,
Igor Seletskiy
CEO @ Cloud Linux Inc

Re: mod_fcgid can kill all the services on the server via kill -15 -1

[Igor Seletskiy]

Ryan,

I like this approach. Do you want us to prepare a patch against latest
version?
Also, we never committed to apache project. Would it be enough if we post a
patch here. Or what would be the process?


On Sun, Apr 17, 2011 at 9:58 PM, pqf p...@mailtech.cn wrote:

  Hi, all
 Another question, does proc_wait_process() should update procnode-proc_id
 to 0 too? or else mod_fcgid may send a signal to another irrelevant process
 while apache is shutting down? I don't follow up mod_fcgid for a while, I
 just took a glance, maybe it's updated somewhere else?
 By the way, procnode-proc_id is set to 0 while apache startup, so why not
 update procnode-proc_id to 0 while fcgid_create_privileged_process() is
 fail? And then check magic number 0 rather than both -1 and 0,  in both
 proc_kill_gracefully() and proc_kill_force().

 Cheers.
 Ryan



  Hello,

 There is a very interesting, and quite a rare bug in mod_fcgid. It is easy
 to reproduce if you can cause fork to fail (which can be done with
 CloudLinux -- if anyone wants to replicate it).

 *Here is how it works: *
 mod_fcgid tries to spawn a new process (proc_spawn_process in
 fcgid_proc_unix.c), but fork returns -1.
 More exactly fcgid_create_privileged_process function call returns error,
 and fills in tmpproc.pid with -1  tmpproc is assiged to procnode-proc_id).

 Now, if at the same time service httpd restart is executed, function
 kill_all_subprocess in fcgid_pm_main.c will execute, and it will try to go
 through all procnodes, sending SIGTERM via proc_kill_gracefully, (and then
 SIGKILL via proc_kill_force) to procnode-proc_id.pid
 Yet, one procnode will be pointing to procnode-proc_id.pid, causing kill
 -15 -1 (kill all).
 The end results all services on the server failing, including SSH, apache,
 syslogd, etc..

 I guess the problem is really rare for most people. Also it is quite hard
 to diagnose, as it is completely not clear where the signal came from, and
 it took us some time to correlate them with apache restarts.. Yet due to our
 OS being used by shared hosts (where httpd restart is common thing), and our
 ability to limit memory available to processes on per virtual host bases
 (which causes fork to fail once that virtual host reaches memory limit), we
 see the issue quite often.

 The solution is quite simple (not sure if it is the best / right solution),
 in file: fcgid_proc_unix.c, in methods proc_kill_gracefully, line:

 rv = apr_proc_kill((procnode-proc_id), SIGTERM);

 should be changed to:
if (procnode-proc_id.pid != -1) {
   rv = apr_proc_kill((procnode-proc_id), SIGTERM);
} else {
   rv = APR_SUCCESS;
}

 Similarly in proc_kill_force
 rv = apr_proc_kill((procnode-proc_id), SIGKILL);
 should be changed to:
if (procnode-proc_id.pid != -1) {
   rv = apr_proc_kill((procnode-proc_id), SIGKILL);
} else {
   rv = APR_SUCCESS;
}

 Regards,
 Igor Seletskiy
 CEO @ Cloud Linux Inc

Re: mod_fcgid can kill all the services on the server via kill -15 -1

[Igor Seletskiy]

I like this idea better then just checking for pid == -1, though sending
TERM signal to 0 shouldn't be that damaging (if damaging at all).
Mostly because apachectl will run in different process group, so it will not
be killed, and will finish restarting apache.
And unless apache itself is embeded or started by some other software --
signal shouldn't kill anything but apache.


On Sun, Apr 17, 2011 at 9:58 PM, pqf p...@mailtech.cn wrote:

  Hi, all
 Another question, does proc_wait_process() should update procnode-proc_id
 to 0 too? or else mod_fcgid may send a signal to another irrelevant process
 while apache is shutting down? I don't follow up mod_fcgid for a while, I
 just took a glance, maybe it's updated somewhere else?
 By the way, procnode-proc_id is set to 0 while apache startup, so why not
 update procnode-proc_id to 0 while fcgid_create_privileged_process() is
 fail? And then check magic number 0 rather than both -1 and 0,  in both
 proc_kill_gracefully() and proc_kill_force().

 Cheers.
 Ryan



  Hello,

 There is a very interesting, and quite a rare bug in mod_fcgid. It is easy
 to reproduce if you can cause fork to fail (which can be done with
 CloudLinux -- if anyone wants to replicate it).

 *Here is how it works: *
 mod_fcgid tries to spawn a new process (proc_spawn_process in
 fcgid_proc_unix.c), but fork returns -1.
 More exactly fcgid_create_privileged_process function call returns error,
 and fills in tmpproc.pid with -1  tmpproc is assiged to procnode-proc_id).

 Now, if at the same time service httpd restart is executed, function
 kill_all_subprocess in fcgid_pm_main.c will execute, and it will try to go
 through all procnodes, sending SIGTERM via proc_kill_gracefully, (and then
 SIGKILL via proc_kill_force) to procnode-proc_id.pid
 Yet, one procnode will be pointing to procnode-proc_id.pid, causing kill
 -15 -1 (kill all).
 The end results all services on the server failing, including SSH, apache,
 syslogd, etc..

 I guess the problem is really rare for most people. Also it is quite hard
 to diagnose, as it is completely not clear where the signal came from, and
 it took us some time to correlate them with apache restarts.. Yet due to our
 OS being used by shared hosts (where httpd restart is common thing), and our
 ability to limit memory available to processes on per virtual host bases
 (which causes fork to fail once that virtual host reaches memory limit), we
 see the issue quite often.

 The solution is quite simple (not sure if it is the best / right solution),
 in file: fcgid_proc_unix.c, in methods proc_kill_gracefully, line:

 rv = apr_proc_kill((procnode-proc_id), SIGTERM);

 should be changed to:
if (procnode-proc_id.pid != -1) {
   rv = apr_proc_kill((procnode-proc_id), SIGTERM);
} else {
   rv = APR_SUCCESS;
}

 Similarly in proc_kill_force
 rv = apr_proc_kill((procnode-proc_id), SIGKILL);
 should be changed to:
if (procnode-proc_id.pid != -1) {
   rv = apr_proc_kill((procnode-proc_id), SIGKILL);
} else {
   rv = APR_SUCCESS;
}

 Regards,
 Igor Seletskiy
 CEO @ Cloud Linux Inc

Re: mod_fcgid can kill all the services on the server via kill -15 -1

[pqf]

Hi, all
Another question, does proc_wait_process() should update procnode-proc_id to 0 
too? or else mod_fcgid may send a signal to another irrelevant process while 
apache is shutting down? I don't follow up mod_fcgid for a while, I just took a 
glance, maybe it's updated somewhere else?
By the way, procnode-proc_id is set to 0 while apache startup, so why not 
update procnode-proc_id to 0 while fcgid_create_privileged_process() is fail? 
And then check magic number 0 rather than both -1 and 0,  in both 
proc_kill_gracefully() and proc_kill_force().

Cheers.
Ryan


 
Hello,


There is a very interesting, and quite a rare bug in mod_fcgid. It is easy to 
reproduce if you can cause fork to fail (which can be done with CloudLinux -- 
if anyone wants to replicate it).


Here is how it works: 
mod_fcgid tries to spawn a new process (proc_spawn_process in 
fcgid_proc_unix.c), but fork returns -1. 
More exactly fcgid_create_privileged_process function call returns error, and 
fills in tmpproc.pid with -1  tmpproc is assiged to procnode-proc_id).


Now, if at the same time service httpd restart is executed, function 
kill_all_subprocess in fcgid_pm_main.c will execute, and it will try to go 
through all procnodes, sending SIGTERM via proc_kill_gracefully, (and then 
SIGKILL via proc_kill_force) to procnode-proc_id.pid
Yet, one procnode will be pointing to procnode-proc_id.pid, causing kill -15 
-1 (kill all).
The end results all services on the server failing, including SSH, apache, 
syslogd, etc..


I guess the problem is really rare for most people. Also it is quite hard to 
diagnose, as it is completely not clear where the signal came from, and it took 
us some time to correlate them with apache restarts.. Yet due to our OS being 
used by shared hosts (where httpd restart is common thing), and our ability to 
limit memory available to processes on per virtual host bases (which causes 
fork to fail once that virtual host reaches memory limit), we see the issue 
quite often.


The solution is quite simple (not sure if it is the best / right solution), in 
file: fcgid_proc_unix.c, in methods proc_kill_gracefully, line:


rv = apr_proc_kill((procnode-proc_id), SIGTERM);


should be changed to:
   if (procnode-proc_id.pid != -1) {
  rv = apr_proc_kill((procnode-proc_id), SIGTERM);
   } else {
  rv = APR_SUCCESS;
   }


Similarly in proc_kill_force
rv = apr_proc_kill((procnode-proc_id), SIGKILL);
should be changed to:
   if (procnode-proc_id.pid != -1) {
  rv = apr_proc_kill((procnode-proc_id), SIGKILL);
   } else {
  rv = APR_SUCCESS;
   }


Regards,
Igor Seletskiy
CEO @ Cloud Linux Inc

Re: mod_fcgid in httpd tarball?

[Jim Jagielski]

On Mar 23, 2011, at 7:49 AM, Jeff Trawick wrote:

 On Wed, Mar 23, 2011 at 7:43 AM, Mark Montague m...@catseye.org wrote:
  On March 23, 2011 7:37 , Graham Leggett minf...@sharp.fm  wrote:
 
 Do we want to introduce mod_fcgid now into httpd 2.3.x for the next beta?
 
 How do we reconcile mod_fcgid with mod_proxy_fcgid?
 
 Do they need to be reconciled?  Each currently has strengths the other
 lacks.
 
 That's how they are reconciled ;)
 
 * mod_proxy_fcgi ONLY routes to separately managed application processes.
 * mod_fcgid ONLY routes to application processes it manages.
 

Yeppers. mod_proxy_fcgi simply adds the FCGI protocol to mod_proxy
(and mod_proxy_balancer). mod_fcgid is an FastCGI application
manager.

Re: mod_fcgid in httpd tarball?

[Graham Leggett]

On 19 Mar 2011, at 12:07 AM, William A. Rowe Jr. wrote:


It seems like mod_fcgid has made huge progress and is now in a much
more stable bugfix epoch of it's life, similar to how mod_proxy had
progressed when development was kicked out of core for major http/1.1
rework, and brought back in when a vast percentage of it's bugs had
been addressed.

Do we want to introduce mod_fcgid now into httpd 2.3.x for the next  
beta?


How do we reconcile mod_fcgid with mod_proxy_fcgid?

Regards,
Graham
--

Re: mod_fcgid in httpd tarball?

[Mark Montague]

 On March 23, 2011 7:37 , Graham Leggett minf...@sharp.fm  wrote:
Do we want to introduce mod_fcgid now into httpd 2.3.x for the next 
beta?


How do we reconcile mod_fcgid with mod_proxy_fcgid?


Do they need to be reconciled?  Each currently has strengths the other 
lacks.  I'd be fine with having both in future httpd 2.3.x betas and 
2.4, at least until one clearly becomes redundant compared to the other.


--
  Mark Montague
  m...@catseye.org

Re: mod_fcgid in httpd tarball?

[William A. Rowe Jr.]

On 3/23/2011 6:37 AM, Graham Leggett wrote:
 On 19 Mar 2011, at 12:07 AM, William A. Rowe Jr. wrote:
 
 It seems like mod_fcgid has made huge progress and is now in a much
 more stable bugfix epoch of it's life, similar to how mod_proxy had
 progressed when development was kicked out of core for major http/1.1
 rework, and brought back in when a vast percentage of it's bugs had
 been addressed.

 Do we want to introduce mod_fcgid now into httpd 2.3.x for the next beta?
 
 How do we reconcile mod_fcgid with mod_proxy_fcgid?

We clarify the docs if they are misaligned.

AIUI, mod_proxy_fcgid does not perform process control, while mod_fcgid
follows the processing spawning example of mod_cgi, mod_cgid etc.

Re: mod_fcgid in httpd tarball?

[Jeff Trawick]

On Wed, Mar 23, 2011 at 7:43 AM, Mark Montague m...@catseye.org wrote:
  On March 23, 2011 7:37 , Graham Leggett minf...@sharp.fm  wrote:

 Do we want to introduce mod_fcgid now into httpd 2.3.x for the next beta?

 How do we reconcile mod_fcgid with mod_proxy_fcgid?

 Do they need to be reconciled?  Each currently has strengths the other
 lacks.

That's how they are reconciled ;)

* mod_proxy_fcgi ONLY routes to separately managed application processes.
* mod_fcgid ONLY routes to application processes it manages.

Re: mod_fcgid in httpd tarball?

[Jim Jagielski]

On Mar 18, 2011, at 6:07 PM, William A. Rowe Jr. wrote:

 It seems like mod_fcgid has made huge progress and is now in a much
 more stable bugfix epoch of it's life, similar to how mod_proxy had
 progressed when development was kicked out of core for major http/1.1
 rework, and brought back in when a vast percentage of it's bugs had
 been addressed.
 
 Do we want to introduce mod_fcgid now into httpd 2.3.x for the next beta?
 

I'd support that, yeah.

Re: mod_fcgid in httpd tarball?

[Jeff Trawick]

On Fri, Mar 18, 2011 at 6:07 PM, William A. Rowe Jr.
wr...@rowe-clan.net wrote:
 It seems like mod_fcgid has made huge progress and is now in a much
 more stable bugfix epoch of it's life, similar to how mod_proxy had
 progressed when development was kicked out of core for major http/1.1
 rework, and brought back in when a vast percentage of it's bugs had
 been addressed.

 Do we want to introduce mod_fcgid now into httpd 2.3.x for the next beta?

I'm thinking about it :)

(I think some process management changes (including directives) should
be changed before the next fcgid bump (e.g., fcgid 2.4.x or httpd
2.4.x if bundled).  Are there cycles to do that in the short term?
Are the changes really important?)

Re: mod_fcgid in httpd tarball?

[Mario Brandt]

I really like to see this included in 2.3.x For me it runs great since
ages on Windows[1] and Linux.

Mario


[1] expect of Bug 50309 (patch included)

Re: mod_fcgid in httpd tarball?

[Mark Montague]

 On March 18, 2011 18:07 , William A. Rowe Jr. wr...@rowe-clan.net 
wrote:

It seems like mod_fcgid has made huge progress and is now in a much
more stable bugfix epoch of it's life, similar to how mod_proxy had
progressed when development was kicked out of core for major http/1.1
rework, and brought back in when a vast percentage of it's bugs had
been addressed.

Do we want to introduce mod_fcgid now into httpd 2.3.x for the next beta?


For what it's worth, on the systems I'm deploying, I'm using 
mod_proxy_fcgi and putting in as much effort as necessary to fix any 
bugs, add features I need to it, etc., simply because mod_proxy_fcgi is 
a core module, while mod_fcgid is not.  If mod_fcgid were in core, I may 
have wound up putting the effort there instead.  (I say may have 
because I've come to think that mod_proxy_fcgi is actually a better 
choice for my particular needs, anyway).


--
  Mark Montague
  m...@catseye.org

Re: mod_fcgid does not appear to bound max processes

[Jeff Trawick]

On Fri, Nov 12, 2010 at 6:00 PM, Edward Z. Yang ezy...@mit.edu wrote:
 Hello Jeff,

 We're running the latest released version of mod_fcgid and everything seems
 to be well. Thanks for getting this fixed.

great news!

Re: mod_fcgid does not appear to bound max processes

[Edward Z. Yang]

Hello Jeff,

We're running the latest released version of mod_fcgid and everything seems
to be well. Thanks for getting this fixed.

Cheers,
Edward

Re: mod_fcgid does not appear to bound max processes

[Jeff Trawick]

On Thu, Oct 28, 2010 at 9:07 PM, Edward Z. Yang ezy...@mit.edu wrote:
 Hello,

 Latest trunk segfaults for us.

 Program received signal SIGSEGV, Segmentation fault.
 0x7f37938786f8 in pm_main () from /etc/httpd/modules/mod_fcgid.so
 (gdb) gcore /root/core.fcgid
 Saved corefile /root/core.fcgid
 (gdb) bt
 #0  0x7f37938786f8 in pm_main () from /etc/httpd/modules/mod_fcgid.so
 #1  0x7f379387a072 in procmgr_post_config () from 
 /etc/httpd/modules/mod_fcgid.so
 #2  0x7f3793873f7a in ?? () from /etc/httpd/modules/mod_fcgid.so
 #3  0x7f379c8e7f09 in ap_run_post_config ()
 #4  0x7f379c8d4267 in main ()

There's not much in pm_main(); perhaps some inlining is occurring?  I
see some code there that is dependent on addressability of the
server_rec in the fcgid daemon, which of course blows when server_rec
is created on the fly.  (This is unrelated to picking the right
process.)  We're getting further.

I have a patch to handle this additional case.  It is running fine
with my usual tests, but I'll try to create a simple module that makes
a copy of the server_rec for every request and run tests with that.

Re: mod_fcgid does not appear to bound max processes

[Jeff Trawick]

On Fri, Oct 29, 2010 at 6:25 AM, Jeff Trawick traw...@gmail.com wrote:
 On Thu, Oct 28, 2010 at 9:07 PM, Edward Z. Yang ezy...@mit.edu wrote:
 Hello,

 Latest trunk segfaults for us.

 Program received signal SIGSEGV, Segmentation fault.
 0x7f37938786f8 in pm_main () from /etc/httpd/modules/mod_fcgid.so
 (gdb) gcore /root/core.fcgid
 Saved corefile /root/core.fcgid
 (gdb) bt
 #0  0x7f37938786f8 in pm_main () from /etc/httpd/modules/mod_fcgid.so
 #1  0x7f379387a072 in procmgr_post_config () from 
 /etc/httpd/modules/mod_fcgid.so
 #2  0x7f3793873f7a in ?? () from /etc/httpd/modules/mod_fcgid.so
 #3  0x7f379c8e7f09 in ap_run_post_config ()
 #4  0x7f379c8d4267 in main ()

 There's not much in pm_main(); perhaps some inlining is occurring?  I
 see some code there that is dependent on addressability of the
 server_rec in the fcgid daemon, which of course blows when server_rec
 is created on the fly.  (This is unrelated to picking the right
 process.)  We're getting further.

 I have a patch to handle this additional case.  It is running fine
 with my usual tests, but I'll try to create a simple module that makes
 a copy of the server_rec for every request and run tests with that.

I now have a test module that can duplicate server_rec; I duplicated
your crash with trunk (well, as far as I can tell from your truncated
backtrace) and confirmed that it works with the latest updates, which
I've committed in r1028683.

Hopefully this will do it!

Re: mod_fcgid does not appear to bound max processes

[Anders Kaseorg]

On Fri, 29 Oct 2010, Jeff Trawick wrote:
 I now have a test module that can duplicate server_rec; I duplicated 
 your crash with trunk (well, as far as I can tell from your truncated 
 backtrace) and confirmed that it works with the latest updates, which 
 I've committed in r1028683.

I’m not sure it’s sufficient just to store server-module_config instead 
of server in fcgid_command.  mod_vhost_ldap also duplicates the 
module_config vector out of the request pool, in order to be able to 
update core_module’s module_config without mutating it in place.

Storing vhost_id directly in fcgid_command would probably work.  Though I 
suspect it would be more robust to directly compare the fcgid_cmd_options 
structures, thereby avoiding the need to track VirtualHost identities 
altogether.

Anders

Re: mod_fcgid does not appear to bound max processes

[Jeff Trawick]

On Fri, Oct 29, 2010 at 3:29 PM, Anders Kaseorg ande...@mit.edu wrote:
 On Fri, 29 Oct 2010, Jeff Trawick wrote:
 I now have a test module that can duplicate server_rec; I duplicated
 your crash with trunk (well, as far as I can tell from your truncated
 backtrace) and confirmed that it works with the latest updates, which
 I've committed in r1028683.

 I’m not sure it’s sufficient just to store server-module_config instead
 of server in fcgid_command.  mod_vhost_ldap also duplicates the
 module_config vector out of the request pool, in order to be able to
 update core_module’s module_config without mutating it in place.

 Storing vhost_id directly in fcgid_command would probably work.

yes, I believe so; I'm not sure why I didn't do that already :)

(and the module_config duplication is another tweak to make to my fake
mass vhost code)

   
 Though I
 suspect it would be more robust to directly compare the fcgid_cmd_options
 structures, thereby avoiding the need to track VirtualHost identities
 altogether.

while I'm concerned about the amount of storage to compare (possibly
mitigated with a hash if it is really that bad), this is probably the
right long-term solution after some restructures of the various
settings; then the people who want to share across vhosts can do so
provided that they don't have any settings which differ

thanks!

Re: mod_fcgid does not appear to bound max processes

[Jeff Trawick]

On Fri, Oct 29, 2010 at 3:53 PM, Jeff Trawick traw...@gmail.com wrote:
 On Fri, Oct 29, 2010 at 3:29 PM, Anders Kaseorg ande...@mit.edu wrote:
 On Fri, 29 Oct 2010, Jeff Trawick wrote:
 I now have a test module that can duplicate server_rec; I duplicated
 your crash with trunk (well, as far as I can tell from your truncated
 backtrace) and confirmed that it works with the latest updates, which
 I've committed in r1028683.

 I’m not sure it’s sufficient just to store server-module_config instead
 of server in fcgid_command.  mod_vhost_ldap also duplicates the
 module_config vector out of the request pool, in order to be able to
 update core_module’s module_config without mutating it in place.

 Storing vhost_id directly in fcgid_command would probably work.

 yes, I believe so; I'm not sure why I didn't do that already :)

 (and the module_config duplication is another tweak to make to my fake
 mass vhost code)

ugly as heck

yeah, it crashed at that point, and AFAICT copying out vhost_id for
fcgid_command works fine

committed yet again (r1028901)

I had a look at mod_vhost_ldap and can't see any other such data
structure tricks.


                                                                              
          Though I
 suspect it would be more robust to directly compare the fcgid_cmd_options
 structures, thereby avoiding the need to track VirtualHost identities
 altogether.

 while I'm concerned about the amount of storage to compare (possibly
 mitigated with a hash if it is really that bad), this is probably the
 right long-term solution after some restructures of the various
 settings; then the people who want to share across vhosts can do so
 provided that they don't have any settings which differ

another day

Re: mod_fcgid does not appear to bound max processes

[Jeff Trawick]

On Wed, Oct 27, 2010 at 6:30 PM, Edward Z. Yang ezy...@mit.edu wrote:
 Hello Jeff,

 We tested the latest version of mod_fcgid in trunk (r1028094) and
 found that, while the total number of fcgis did not explode and overwhelm
 the server, that after a ramp up time new FCGIs were not being spawned
 and we were getting responses like:

    $ curl http://andersk.scripts.mit.edu/hello/hello.c.fcgi
    curl: (52) Empty reply from server

 Our hypothesis is that the new code interacts poorly with mod_vhost_ldap
 because mod_vhost_ldap copies the server_rec to a dynamically allocated buffer
 (see 
 http://mail-archives.apache.org/mod_mbox/httpd-dev/201007.mbox/%3c4c3465d9.1090...@rowe-clan.net%3e
 so the current_node-server == procnode-server check never succeeds.  mod_ftp
 does something similar.

Interesting...  I guess mod_fcgid has to leave some droplet attached
to a server_rec that can used to identify the different VirtualHost 
containers in lieu of comparing the ServerName setting (original
implementation) or the server_rec address (my subsequent fix).

(I don't know precisely what is failing in your case.)

I guess you could rip out the server_rec comparison, as long as you
have no VirtualHost-specific settings related to the FastCGI
applications (i.e., if you can share your apps across vhosts).

When somebody gets time/gumption I'm sure there will be a way to share
apps across vhosts but that seems ugly to sanity check the feasibility
(i.e., refuse to allow vhost-specific configuration of anything that
would be impacted by reusing another vhost's process when there's some
share-across-vhost flag at global scope.  Hopefully your case can be
made to work without that other work.

Re: mod_fcgid does not appear to bound max processes

[Jeff Trawick]

On Thu, Oct 28, 2010 at 8:57 AM, Jeff Trawick traw...@gmail.com wrote:
 On Wed, Oct 27, 2010 at 6:30 PM, Edward Z. Yang ezy...@mit.edu wrote:
 Hello Jeff,

 We tested the latest version of mod_fcgid in trunk (r1028094) and
 found that, while the total number of fcgis did not explode and overwhelm
 the server, that after a ramp up time new FCGIs were not being spawned
 and we were getting responses like:

    $ curl http://andersk.scripts.mit.edu/hello/hello.c.fcgi
    curl: (52) Empty reply from server

 Our hypothesis is that the new code interacts poorly with mod_vhost_ldap
 because mod_vhost_ldap copies the server_rec to a dynamically allocated 
 buffer
 (see 
 http://mail-archives.apache.org/mod_mbox/httpd-dev/201007.mbox/%3c4c3465d9.1090...@rowe-clan.net%3e
 so the current_node-server == procnode-server check never succeeds.  
 mod_ftp
 does something similar.

 Interesting...  I guess mod_fcgid has to leave some droplet attached
 to a server_rec that can used to identify the different VirtualHost 
 containers in lieu of comparing the ServerName setting (original
 implementation) or the server_rec address (my subsequent fix).

 (I don't know precisely what is failing in your case.)

 I guess you could rip out the server_rec comparison, as long as you
 have no VirtualHost-specific settings related to the FastCGI
 applications (i.e., if you can share your apps across vhosts).

 When somebody gets time/gumption I'm sure there will be a way to share
 apps across vhosts but that seems ugly to sanity check the feasibility
 (i.e., refuse to allow vhost-specific configuration of anything that
 would be impacted by reusing another vhost's process when there's some
 share-across-vhost flag at global scope.  Hopefully your case can be
 made to work without that other work.

please try the attached patch if at all possible

when comparing vhosts it requires only that the per-server module
config be left alone in the server_rec copy
Index: modules/fcgid/fcgid_conf.c
===
--- modules/fcgid/fcgid_conf.c  (revision 1027924)
+++ modules/fcgid/fcgid_conf.c  (working copy)
@@ -57,10 +57,32 @@
 #define DEFAULT_WRAPPER_KEY ALL
 #define WRAPPER_FLAG_VIRTUAL virtual
 
+int fcgid_same_vhost(const server_rec *s1, const server_rec *s2)
+{
+fcgid_server_conf *s1cfg, *s2cfg;
+
+if (s1 == s2) {
+return 1;
+}
+
+s1cfg = ap_get_module_config(s1-module_config, fcgid_module);
+s2cfg = ap_get_module_config(s2-module_config, fcgid_module);
+
+return s1cfg-server_id == s2cfg-server_id;
+}
+
 void *create_fcgid_server_config(apr_pool_t * p, server_rec * s)
 {
 fcgid_server_conf *config = apr_pcalloc(p, sizeof(*config));
+static int server_id = 0;
 
+/* allow vhost comparison even when some mass-vhost module
+ * makes a copy of the server_rec to override docroot or
+ * other such settings
+ */
+++server_id;
+config-server_id = server_id;
+
 if (!s-is_virtual) {
 config-busy_scan_interval = DEFAULT_BUSY_SCAN_INTERVAL;
 config-error_scan_interval = DEFAULT_ERROR_SCAN_INTERVAL;
Index: modules/fcgid/fcgid_spawn_ctl.c
===
--- modules/fcgid/fcgid_spawn_ctl.c (revision 1027924)
+++ modules/fcgid/fcgid_spawn_ctl.c (working copy)
@@ -60,9 +60,9 @@
 if (current_node-inode == procnode-inode
  current_node-deviceid == procnode-deviceid
  !strcmp(current_node-cmdline, procnode-cmdline)
- current_node-server == procnode-server
  current_node-uid == procnode-uid
- current_node-gid == procnode-gid)
+ current_node-gid == procnode-gid
+ fcgid_same_vhost(current_node-server, procnode-server))
 break;
 previous_node = current_node;
 }
@@ -173,9 +173,9 @@
 if (current_node-inode == command-inode
  current_node-deviceid == command-deviceid
  !strcmp(current_node-cmdline, command-cmdline)
- current_node-server == command-server
  current_node-uid == command-uid
- current_node-gid == command-gid)
+ current_node-gid == command-gid
+ fcgid_same_vhost(current_node-server, command-server))
 break;
 }
 
@@ -233,9 +233,9 @@
 if (current_node-inode == procnode-inode
  current_node-deviceid == procnode-deviceid
  !strcmp(current_node-cmdline, procnode-cmdline)
- current_node-server == procnode-server
  current_node-uid == procnode-uid
- current_node-gid == procnode-gid)
+ current_node-gid == procnode-gid
+ fcgid_same_vhost(current_node-server, procnode-server))
 break;
 }
 
Index: modules/fcgid/fcgid_conf.h
===
--- modules/fcgid/fcgid_conf.h  (revision 1027924)
+++ 

Re: mod_fcgid does not appear to bound max processes

[William A. Rowe Jr.]

On 10/28/2010 2:04 PM, Jeff Trawick wrote:
 On Thu, Oct 28, 2010 at 8:57 AM, Jeff Trawick traw...@gmail.com wrote:

 When somebody gets time/gumption I'm sure there will be a way to share
 apps across vhosts but that seems ugly to sanity check the feasibility
 (i.e., refuse to allow vhost-specific configuration of anything that
 would be impacted by reusing another vhost's process when there's some
 share-across-vhost flag at global scope.  Hopefully your case can be
 made to work without that other work.
 
 please try the attached patch if at all possible
 
 when comparing vhosts it requires only that the per-server module
 config be left alone in the server_rec copy

Good solution.  Multiple requests could be all over the place in terms of
which Host: they are directed to, so the mod_ftp solution to this wouldn't
be appropriate.  That was simply to compare r-connection-base_server,
but since ftp toggles are effectively per-connection for determining the
ftp active port bindings, your more complex solution doesn't seem to be
needed in mod_ftp, either.

Re: mod_fcgid does not appear to bound max processes

[Edward Z. Yang]

Hello,

Latest trunk segfaults for us.

Program received signal SIGSEGV, Segmentation fault.
0x7f37938786f8 in pm_main () from /etc/httpd/modules/mod_fcgid.so
(gdb) gcore /root/core.fcgid
Saved corefile /root/core.fcgid
(gdb) bt
#0  0x7f37938786f8 in pm_main () from /etc/httpd/modules/mod_fcgid.so
#1  0x7f379387a072 in procmgr_post_config () from 
/etc/httpd/modules/mod_fcgid.so
#2  0x7f3793873f7a in ?? () from /etc/httpd/modules/mod_fcgid.so
#3  0x7f379c8e7f09 in ap_run_post_config ()
#4  0x7f379c8d4267 in main ()

Cheers,
Edward

Re: mod_fcgid does not appear to bound max processes

[Edward Z. Yang]

Hello Jeff,

We tested the latest version of mod_fcgid in trunk (r1028094) and
found that, while the total number of fcgis did not explode and overwhelm
the server, that after a ramp up time new FCGIs were not being spawned
and we were getting responses like:

$ curl http://andersk.scripts.mit.edu/hello/hello.c.fcgi
curl: (52) Empty reply from server

Our hypothesis is that the new code interacts poorly with mod_vhost_ldap
because mod_vhost_ldap copies the server_rec to a dynamically allocated buffer
(see 
http://mail-archives.apache.org/mod_mbox/httpd-dev/201007.mbox/%3c4c3465d9.1090...@rowe-clan.net%3e
so the current_node-server == procnode-server check never succeeds.  mod_ftp
does something similar.

Cheers,
Edward

Re: mod_fcgid does not appear to bound max processes

[Jeff Trawick]

On Sat, Oct 23, 2010 at 7:14 AM, Edward Z. Yang ezy...@mit.edu wrote:
 Hello all,

 We recently switched from Fedora 11 to Fedora 13, and were quite
 puzzled by the fact that our servers, despite running on all the
 same hardware, were being extremely hosed.  We tracked down the
 problem to continually spawning FastCGI processes: one virtualhost had
 200 of dispatch.fcgi running, by our count!  No unusual messages
 were found in the error log.

Do you have some sort of mass vhost configuration, or is ServerName not set?

Have a look at http://svn.apache.org/viewvc?view=revisionrevision=939472

I suggest trying mod_fcgid trunk anyway, as that commit changes the
implementation of the one you identified.


 We did a little snooping and found that downgrading from 2.3.5 to 2.2
 fixed the problem.  We think that the bug might have been caused
 by this commit:

    http://svn.apache.org/viewvc?view=revisionrevision=753578

 A similar bug was reported here, but it's been marked FIXED
 and we don't think it addresses our issue.

    https://issues.apache.org/bugzilla/show_bug.cgi?id=48981

 Please let us know if we should submit another bug report,
 or provide more debugging info.

 Name        : mod_fcgid
 Arch        : x86_64
 Version     : 2.2
 Release     : 13.fc12
 Size        : 130 k
 Repo        : installed
 Summary     : Apache2 module for high-performance server-side
            : scripting
 URL         : http://fastcgi.coremail.cn/
 License     : GPL+
 Description : mod_fcgid is a binary-compatible alternative to
            : the Apache module mod_fastcgi. mod_fcgid has a
            : new process management strategy, which
            : concentrates on reducing the number of fastcgi
            : servers, and kicking out corrupt fastcgi servers
            : as soon as possible.

 Available Packages
 Name        : mod_fcgid
 Arch        : x86_64
 Version     : 2.3.5
 Release     : 1.fc13
 Size        : 68 k
 Repo        : fedora
 Summary     : FastCGI interface module for Apache 2
 URL         : http://httpd.apache.org/mod_fcgid/
 License     : ASL 2.0
 Description : mod_fcgid is a binary-compatible alternative to
            : the Apache module mod_fastcgi. mod_fcgid has a
            : new process management strategy, which
            : concentrates on reducing the number of fastcgi
            : servers, and kicking out corrupt fastcgi servers
            : as soon as possible.

 Cheers,
 Edward




-- 
Born in Roswell... married an alien...

Re: mod_fcgid does not appear to bound max processes

[Edward Z. Yang]

Excerpts from Jeff Trawick's message of Sat Oct 23 08:29:50 -0400 2010:
 Do you have some sort of mass vhost configuration, or is ServerName not set?

We use mod_vhost_ldap.

 Have a look at http://svn.apache.org/viewvc?view=revisionrevision=939472
 
 I suggest trying mod_fcgid trunk anyway, as that commit changes the
 implementation of the one you identified.

Will do, thanks.

Cheers,
Edward

Re: mod_fcgid Was Re: mod_status.h - include directory

[Rainer Jung]

On 01.07.2010 04:23, Gregg L. Smith wrote:

Speaking of 2.3.6, it would be nice to see it soonish too? Looking at
changes there are not a lot of them but the segfault is a bear, and it
brings downs Apache. It's been six months as well, seems a perfect
reason for a release.


Not sure what you mean here. Version 2.3.6 alpha has been released on 
June 21st.


Regards,

Rainer

Re: mod_fcgid Was Re: mod_status.h - include directory

[Gregg L. Smith]

Hello Rainer,

Sorry for my not being clear enough, these numbers need to change. It sure 
would make it easier.

mod_fcgid/2.3.6 is what I was talking about.

However, something was pointed out to me by Mario Brandt regarding mod_fcgid 
and graceful restarts on Windows. We've been here before back with 
mod_fcgid/2.2 and it seems the proposed changes never made it in before 
mod_fcgid was donated to the ASF, I can see the old code in fcgid_pm_win.c and 
fcgid_pm_main.c. I need to do some digging to find the old bug reports of Tom 
Donovan's, who supplied the patch to fix them that I am looking at. The fix has 
been well tested by people at the Apache Lounge and in some cases I'd imagine 
still in use today.

Regards,

Gregg



Original Message ---
On 01.07.2010 04:23, Gregg L. Smith wrote:
 Speaking of 2.3.6, it would be nice to see it soonish too? Looking at
 changes there are not a lot of them but the segfault is a bear, and it
 brings downs Apache. It's been six months as well, seems a perfect
 reason for a release.

Not sure what you mean here. Version 2.3.6 alpha has been released on
June 21st.

Regards,

Rainer

Re: mod_fcgid: process PID graceful kill fail, sending SIGKILL

[Weedy]

Can the SIGTERM be logged? Just to see how long it takes?

On Fri, Apr 23, 2010 at 3:39 PM, Jeff Trawick traw...@gmail.com wrote:
 On Fri, Apr 23, 2010 at 3:33 PM, Jeff Trawick traw...@gmail.com wrote:

 Keeping zombies cleaned up requires a small scan interval, but that
 can keep some scripts from having adequate time to terminate before
 getting KILL-ed.

 whoops, ignore that; I'm getting scans confused :)
 (FcgidZombieScanInterval controls zombie cleanup)

Re: mod_fcgid: process PID graceful kill fail, sending SIGKILL

[Jeff Trawick]

On Mon, Apr 26, 2010 at 11:02 AM, Weedy weedy2...@gmail.com wrote:
 Can the SIGTERM be logged? Just to see how long it takes?

The patches attached add extra logging to the error scan.

1. report_graceful_terminations.txt

It logs the number of processes receiving SIGTERM every time the error
scan runs, as long as there is at least one process.  This could
probably be committed permanently.

It requires LogLevel debug.

2. noisily_report_graceful_terminations.txt

This logs the pid of every process that is sent SIGTERM.  It is too
noisy to be committed permanently.

It requires LogLevel debug.
Index: modules/fcgid/fcgid_pm_main.c
===
--- modules/fcgid/fcgid_pm_main.c   (revision 938085)
+++ modules/fcgid/fcgid_pm_main.c   (working copy)
@@ -282,6 +282,7 @@
 fcgid_server_conf *sconf =
 ap_get_module_config(main_server-module_config,
  fcgid_module);
+int graceful_terminations = 0;
 
 /* Should I check the busy list? */
 if (procmgr_must_exit()
@@ -326,6 +327,7 @@
   current_node-proc_pool);
 if (!dummy) {
 proc_kill_gracefully(current_node, main_server);
+++graceful_terminations;
 apr_pool_userdata_set(set, HAS_GRACEFUL_KILL,
   apr_pool_cleanup_null,
   current_node-proc_pool);
@@ -350,6 +352,12 @@
 }
 previous_node-next_index = temp_error_header.next_index;
 proctable_pm_unlock(main_server);
+
+if (graceful_terminations) {
+ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, main_server,
+ mod_fcgid: gracefully terminated %d processes,
+ graceful_terminations);
+}
 }
 
 static void kill_all_subprocess(server_rec * main_server)
Index: modules/fcgid/fcgid_pm_main.c
===
--- modules/fcgid/fcgid_pm_main.c   (revision 938091)
+++ modules/fcgid/fcgid_pm_main.c   (working copy)
@@ -326,6 +326,9 @@
   current_node-proc_pool);
 if (!dummy) {
 proc_kill_gracefully(current_node, main_server);
+ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, main_server,
+ mod_fcgid: gracefully killing process % 
APR_PID_T_FMT,
+ current_node-proc_id.pid);
 apr_pool_userdata_set(set, HAS_GRACEFUL_KILL,
   apr_pool_cleanup_null,
   current_node-proc_pool);

Re: mod_fcgid: process PID graceful kill fail, sending SIGKILL

[Jeff Trawick]

On Tue, Apr 20, 2010 at 1:36 AM, Weedy weedy2...@gmail.com wrote:
 For the life of me I can't seem to find a way to make mod_fcgid wait
 longer. I don't mean FcgidMaxRequestsPerProcess or
 FcgidProcessLifeTime. I mean what controls the time between the signal
 -2/-11/-15/whatever and the -9.
 Any help would be appreciated.

During steady state:

FcgidErrorScanInterval controls normal termination of idle/extra
processes.  The directive controls the time between scans of the error
list, and during one scan it can send SIGTERM and during the next
scan, if it hasn't exited yet, it can send SIGKILL.

Keeping zombies cleaned up requires a small scan interval, but that
can keep some scripts from having adequate time to terminate before
getting KILL-ed.

At shutdown:

1 second hard-coded sleep between graceful kill and SIGKILL

--/--

Does this match what you are seeing?

Re: mod_fcgid: process PID graceful kill fail, sending SIGKILL

[Jeff Trawick]

On Fri, Apr 23, 2010 at 3:33 PM, Jeff Trawick traw...@gmail.com wrote:

 Keeping zombies cleaned up requires a small scan interval, but that
 can keep some scripts from having adequate time to terminate before
 getting KILL-ed.

whoops, ignore that; I'm getting scans confused :)
(FcgidZombieScanInterval controls zombie cleanup)

Re: [mod_fcgid] Windows and TCP/IP

[Jeff Trawick]

On Fri, Apr 9, 2010 at 9:49 AM, Darren Garvey darren.gar...@gmail.com wrote:
 On 9 April 2010 13:39, Jeff Trawick traw...@gmail.com wrote:

 On Wed, Mar 31, 2010 at 10:56 AM, Darren Garvey darren.gar...@gmail.com
 wrote:
  On 31 March 2010 15:49, Jeff Trawick traw...@gmail.com wrote:
 
  On Wed, Mar 31, 2010 at 7:05 AM, William A. Rowe Jr.
  wr...@rowe-clan.net wrote:
   On 3/31/2010 5:41 AM, Jeff Trawick wrote:
   On Wed, Mar 31, 2010 at 6:26 AM, Darren Garvey
   darren.gar...@gmail.com wrote:

   Does anyone know if there is any other way to communicate with
   mod_fcgid on Windows besides anonymous pipes?

   Isn't httpd 2.3 alpha mod_proxy_scgi ment to do this?

  mod_proxy_fcgi at least ;)  (I meant to add that earlier)
 
  mod_proxy_* is httpd's way to route/loadbalance/manage connections to
  externally managed servers of various types

  Ahh, this might be what I'm looking for. Is
  http://mproxyfcgi.sourceforge.net/ the same thing? It looks like
  mod_proxy_fcgi is in trunk now but this link has a user guide. :)

 hmmm, looks like the same idea (hopefully that's a good sign)...
 mod_proxy_fcgid in httpd trunk was started from scratch some 4 yrs
 ago...

 Ok so, does mod_proxy_fcgid in trunk work? It would be nice to test my
 FastCGI code with an external configuration (with async I/O) on Windows. For
 now I'm just using pipes which - apart from blocking on read/write - seem to
 work fine.

Does it work?  Well, sort of...

Here's my first attempt:

$ cat ~/inst/23/conf/conf.d/proxy-fcgi.conf
ProxyPass /info fcgi://127.0.0.1:/

$ support/fcgistarter -c ~/myhg/apache/fcgid/apps/info.pl -p 

(uh, fcgidstarter needed a fix to work on my system)

$ cat ~/myhg/apache/fcgid/apps/info.pl
#!/usr/bin/perl

use strict;
use CGI::Fast;

my $count = 0;
while (my $q = CGI::Fast-new) {
print(Content-Type: text/plain\n\n);
print(Process ID: $$; Count is:  . ++$count . \n);

print VHOST: $ENV{'VHOST'}\n;
print WHO: $ENV{'WHO'}\n;
print User-AgenT: $ENV{'User_AgenT'}\n;
}

It worked for a few requests, after which the Perl FastCGI engine got
out of sync and blocked in read().

Perhaps others have had better results?  (Yeah, I should shut up and
find the problem.)


  Does mod_proxy_fcgi set FCGI_WEB_SERVER_ADDRS as per the FastCGI spec?

 That should be set when the TCP-based FastCGI application is spawned
 and checked when a new connection from the web server/gateway is
 accepted?

 I see PHP logic to check it, but spawn-fcgi (Lighttpd) and fcgistarter
 (httpd) don't handle that automatically.  (I think you'd need to
 create a wrapper for your app that sets it, then tell
 spawn-fcgi/fcgistarter/whatever to run the wrapper.)

 Yes, of course. Silly question now I think about it. This is an
 installation-specific configuration variable, I guess PassEnv should do too.

You'd want to set FCGI_WEB_SERVER_ADDRS as the FastCGI app process is
created, not during request context (PassEnv), since you're trying to
validate what happens in the request context.

Re: [mod_fcgid] Windows and TCP/IP

[Jeff Trawick]

On Wed, Apr 14, 2010 at 1:52 PM, Jeff Trawick traw...@gmail.com wrote:
 On Fri, Apr 9, 2010 at 9:49 AM, Darren Garvey darren.gar...@gmail.com wrote:
 On 9 April 2010 13:39, Jeff Trawick traw...@gmail.com wrote:

 On Wed, Mar 31, 2010 at 10:56 AM, Darren Garvey darren.gar...@gmail.com
 wrote:
  On 31 March 2010 15:49, Jeff Trawick traw...@gmail.com wrote:
 
  On Wed, Mar 31, 2010 at 7:05 AM, William A. Rowe Jr.
  wr...@rowe-clan.net wrote:
   On 3/31/2010 5:41 AM, Jeff Trawick wrote:
   On Wed, Mar 31, 2010 at 6:26 AM, Darren Garvey
   darren.gar...@gmail.com wrote:

   Does anyone know if there is any other way to communicate with
   mod_fcgid on Windows besides anonymous pipes?

   Isn't httpd 2.3 alpha mod_proxy_scgi ment to do this?

  mod_proxy_fcgi at least ;)  (I meant to add that earlier)
 
  mod_proxy_* is httpd's way to route/loadbalance/manage connections to
  externally managed servers of various types

  Ahh, this might be what I'm looking for. Is
  http://mproxyfcgi.sourceforge.net/ the same thing? It looks like
  mod_proxy_fcgi is in trunk now but this link has a user guide. :)

 hmmm, looks like the same idea (hopefully that's a good sign)...
 mod_proxy_fcgid in httpd trunk was started from scratch some 4 yrs
 ago...

 Ok so, does mod_proxy_fcgid in trunk work? It would be nice to test my
 FastCGI code with an external configuration (with async I/O) on Windows. For
 now I'm just using pipes which - apart from blocking on read/write - seem to
 work fine.

 Does it work?  Well, sort of...

 Here's my first attempt:

 $ cat ~/inst/23/conf/conf.d/proxy-fcgi.conf
 ProxyPass /info fcgi://127.0.0.1:/

 $ support/fcgistarter -c ~/myhg/apache/fcgid/apps/info.pl -p 

 (uh, fcgidstarter needed a fix to work on my system)

 $ cat ~/myhg/apache/fcgid/apps/info.pl
 #!/usr/bin/perl

 use strict;
 use CGI::Fast;

 my $count = 0;
 while (my $q = CGI::Fast-new) {
    print(Content-Type: text/plain\n\n);
    print(Process ID: $$; Count is:  . ++$count . \n);

    print VHOST: $ENV{'VHOST'}\n;
    print WHO: $ENV{'WHO'}\n;
    print User-AgenT: $ENV{'User_AgenT'}\n;
 }

 It worked for a few requests, after which the Perl FastCGI engine got
 out of sync and blocked in read().

 Perhaps others have had better results?  (Yeah, I should shut up and
 find the problem.)

Proxy holds open the back-end connection by default, so that it
doesn't have to reconnect when the same httpd child process needs to
send the app a request.  I have a single instance, single threaded
FastCGI app, effectively waiting for a subsequent request on that
held-open back-end connection, which won't happen until the same httpd
child process handles another request.

Disabling reuse resolves that problem:

ProxyPass /info fcgi://127.0.0.1:/ disablereuse=on

Re: [mod_fcgid] Windows and TCP/IP

[Jeff Trawick]

On Wed, Mar 31, 2010 at 10:56 AM, Darren Garvey darren.gar...@gmail.com wrote:
 On 31 March 2010 15:49, Jeff Trawick traw...@gmail.com wrote:

 On Wed, Mar 31, 2010 at 7:05 AM, William A. Rowe Jr.
 wr...@rowe-clan.net wrote:
  On 3/31/2010 5:41 AM, Jeff Trawick wrote:
  On Wed, Mar 31, 2010 at 6:26 AM, Darren Garvey
  darren.gar...@gmail.com wrote:

  I can't remember now if mod_fcgid used to support external FastCGI
  daemons
  or not, but mod_fastcgi did (via. FastCgiExternalServer IIRC).
 
  No; mod_fcgid only supports FastCGI app processes which it has spawned.
 
 
  Does anyone know if there is any other way to communicate with
  mod_fcgid on
  Windows besides anonymous pipes?
 
  Isn't httpd 2.3 alpha mod_proxy_scgi ment to do this?

 mod_proxy_fcgi at least ;)  (I meant to add that earlier)

 mod_proxy_* is httpd's way to route/loadbalance/manage connections to
 externally managed servers of various types

 Ahh, this might be what I'm looking for. Is
 http://mproxyfcgi.sourceforge.net/ the same thing? It looks like
 mod_proxy_fcgi is in trunk now but this link has a user guide. :)

hmmm, looks like the same idea (hopefully that's a good sign)...
mod_proxy_fcgid in httpd trunk was started from scratch some 4 yrs
ago...

 Does mod_proxy_fcgi set FCGI_WEB_SERVER_ADDRS as per the FastCGI spec?

That should be set when the TCP-based FastCGI application is spawned
and checked when a new connection from the web server/gateway is
accepted?

I see PHP logic to check it, but spawn-fcgi (Lighttpd) and fcgistarter
(httpd) don't handle that automatically.  (I think you'd need to
create a wrapper for your app that sets it, then tell
spawn-fcgi/fcgistarter/whatever to run the wrapper.)

Re: [mod_fcgid] Windows and TCP/IP

[Darren Garvey]

On 9 April 2010 13:39, Jeff Trawick traw...@gmail.com wrote:

 On Wed, Mar 31, 2010 at 10:56 AM, Darren Garvey darren.gar...@gmail.com
 wrote:
  On 31 March 2010 15:49, Jeff Trawick traw...@gmail.com wrote:
 
  On Wed, Mar 31, 2010 at 7:05 AM, William A. Rowe Jr.
  wr...@rowe-clan.net wrote:
   On 3/31/2010 5:41 AM, Jeff Trawick wrote:
   On Wed, Mar 31, 2010 at 6:26 AM, Darren Garvey
   darren.gar...@gmail.com wrote:

   Does anyone know if there is any other way to communicate with
   mod_fcgid on Windows besides anonymous pipes?

   Isn't httpd 2.3 alpha mod_proxy_scgi ment to do this?

  mod_proxy_fcgi at least ;)  (I meant to add that earlier)
 
  mod_proxy_* is httpd's way to route/loadbalance/manage connections to
  externally managed servers of various types

  Ahh, this might be what I'm looking for. Is
  http://mproxyfcgi.sourceforge.net/ the same thing? It looks like
  mod_proxy_fcgi is in trunk now but this link has a user guide. :)

 hmmm, looks like the same idea (hopefully that's a good sign)...
 mod_proxy_fcgid in httpd trunk was started from scratch some 4 yrs
 ago...


Ok so, does mod_proxy_fcgid in trunk work? It would be nice to test my
FastCGI code with an external configuration (with async I/O) on Windows. For
now I'm just using pipes which - apart from blocking on read/write - seem to
work fine.


   Does mod_proxy_fcgi set FCGI_WEB_SERVER_ADDRS as per the FastCGI spec?

 That should be set when the TCP-based FastCGI application is spawned
 and checked when a new connection from the web server/gateway is
 accepted?

 I see PHP logic to check it, but spawn-fcgi (Lighttpd) and fcgistarter
 (httpd) don't handle that automatically.  (I think you'd need to
 create a wrapper for your app that sets it, then tell
 spawn-fcgi/fcgistarter/whatever to run the wrapper.)


Yes, of course. Silly question now I think about it. This is an
installation-specific configuration variable, I guess PassEnv should do too.

Thanks for your help.

Cheers,
Darren

Re: [mod_fcgid] Windows and TCP/IP

[Jeff Trawick]

On Wed, Mar 31, 2010 at 6:26 AM, Darren Garvey darren.gar...@gmail.com wrote:
 Hi all,

 I've been trying out the latest mod_fcgid and having some good results; good
 to see it back in apache-proper.

FWLIW, it never was in apache-proper until now.  (ISTR seeing some
discussion of mod_fastcgi in the project, but that was before my time
here.)

 I can't remember now if mod_fcgid used to support external FastCGI daemons
 or not, but mod_fastcgi did (via. FastCgiExternalServer IIRC).

No; mod_fcgid only supports FastCGI app processes which it has spawned.


 Does anyone know if there is any other way to communicate with mod_fcgid on
 Windows besides anonymous pipes?

There is not.

 I'm trying to communicate using IO Completion Ports and async I/O (in C++
 code) and getting stuck trying to figure out if it's possible. Reading
 through the source for libfcgi isn't giving me a huge amount of hope...

 Unfortunately I'm a bit stuck even knowing what questions I need to ask
 Google on this.

No help here on that :(

Re: [mod_fcgid] Windows and TCP/IP

[Darren Garvey]

Hi Jeff,

On 31 March 2010 11:41, Jeff Trawick traw...@gmail.com wrote:

 On Wed, Mar 31, 2010 at 6:26 AM, Darren Garvey darren.gar...@gmail.com
 wrote:
  I've been trying out the latest mod_fcgid and having some good results;
 good
  to see it back in apache-proper.

 FWLIW, it never was in apache-proper until now.  (ISTR seeing some
 discussion of mod_fastcgi in the project, but that was before my time
 here.)


You're right, it was only mod_fastcgi which used to be distributed with
httpd. I probably meant to say good to see support back in apache.  ;)
Thanks for the effort Jeff.


   I can't remember now if mod_fcgid used to support external FastCGI
 daemons
  or not, but mod_fastcgi did (via. FastCgiExternalServer IIRC).

 No; mod_fcgid only supports FastCGI app processes which it has spawned.


Ok. FWIW, I find external servers very useful for debugging.


   Does anyone know if there is any other way to communicate with mod_fcgid
 on
  Windows besides anonymous pipes?

 There is not.


Is there any motivation to add to this? Does this mean that FcgidIPCDir has
no effect on Windows? This isn't mentioned in the docs.

I guess this means it's not possible to write a server-side library on
Windows that uses async I/O then.


   I'm trying to communicate using IO Completion Ports and async I/O (in
 C++
  code) and getting stuck trying to figure out if it's possible. Reading
  through the source for libfcgi isn't giving me a huge amount of hope...
 
  Unfortunately I'm a bit stuck even knowing what questions I need to ask
  Google on this.

 No help here on that :(


Ok, thanks.

Cheers,
Darren

Re: [mod_fcgid] Windows and TCP/IP

[William A. Rowe Jr.]

On 3/31/2010 5:41 AM, Jeff Trawick wrote:
 On Wed, Mar 31, 2010 at 6:26 AM, Darren Garvey darren.gar...@gmail.com 
 wrote:
 Hi all,

 I've been trying out the latest mod_fcgid and having some good results; good
 to see it back in apache-proper.
 
 FWLIW, it never was in apache-proper until now.  (ISTR seeing some
 discussion of mod_fastcgi in the project, but that was before my time
 here.)
 
 I can't remember now if mod_fcgid used to support external FastCGI daemons
 or not, but mod_fastcgi did (via. FastCgiExternalServer IIRC).
 
 No; mod_fcgid only supports FastCGI app processes which it has spawned.
 

 Does anyone know if there is any other way to communicate with mod_fcgid on
 Windows besides anonymous pipes?
 
 There is not.
 
 I'm trying to communicate using IO Completion Ports and async I/O (in C++
 code) and getting stuck trying to figure out if it's possible. Reading
 through the source for libfcgi isn't giving me a huge amount of hope...

 Unfortunately I'm a bit stuck even knowing what questions I need to ask
 Google on this.
 
 No help here on that :(

Isn't httpd 2.3 alpha mod_proxy_scgi ment to do this?

Re: [mod_fcgid] Windows and TCP/IP

[Jeff Trawick]

On Wed, Mar 31, 2010 at 7:05 AM, William A. Rowe Jr.
wr...@rowe-clan.net wrote:
 On 3/31/2010 5:41 AM, Jeff Trawick wrote:
 On Wed, Mar 31, 2010 at 6:26 AM, Darren Garvey darren.gar...@gmail.com 
 wrote:
 Hi all,

 I've been trying out the latest mod_fcgid and having some good results; good
 to see it back in apache-proper.

 FWLIW, it never was in apache-proper until now.  (ISTR seeing some
 discussion of mod_fastcgi in the project, but that was before my time
 here.)

 I can't remember now if mod_fcgid used to support external FastCGI daemons
 or not, but mod_fastcgi did (via. FastCgiExternalServer IIRC).

 No; mod_fcgid only supports FastCGI app processes which it has spawned.


 Does anyone know if there is any other way to communicate with mod_fcgid on
 Windows besides anonymous pipes?

 There is not.

 I'm trying to communicate using IO Completion Ports and async I/O (in C++
 code) and getting stuck trying to figure out if it's possible. Reading
 through the source for libfcgi isn't giving me a huge amount of hope...

 Unfortunately I'm a bit stuck even knowing what questions I need to ask
 Google on this.

 No help here on that :(

 Isn't httpd 2.3 alpha mod_proxy_scgi ment to do this?

mod_proxy_fcgi at least ;)  (I meant to add that earlier)

mod_proxy_* is httpd's way to route/loadbalance/manage connections to
externally managed servers of various types

Re: [mod_fcgid] Windows and TCP/IP

[Darren Garvey]

On 31 March 2010 15:49, Jeff Trawick traw...@gmail.com wrote:

 On Wed, Mar 31, 2010 at 7:05 AM, William A. Rowe Jr.
 wr...@rowe-clan.net wrote:
  On 3/31/2010 5:41 AM, Jeff Trawick wrote:
  On Wed, Mar 31, 2010 at 6:26 AM, Darren Garvey darren.gar...@gmail.com
 wrote:

  I can't remember now if mod_fcgid used to support external FastCGI
 daemons
  or not, but mod_fastcgi did (via. FastCgiExternalServer IIRC).
 
  No; mod_fcgid only supports FastCGI app processes which it has spawned.
 
 
  Does anyone know if there is any other way to communicate with
 mod_fcgid on
  Windows besides anonymous pipes?
 
  Isn't httpd 2.3 alpha mod_proxy_scgi ment to do this?

 mod_proxy_fcgi at least ;)  (I meant to add that earlier)

 mod_proxy_* is httpd's way to route/loadbalance/manage connections to
 externally managed servers of various types


Ahh, this might be what I'm looking for. Is
http://mproxyfcgi.sourceforge.net/ the same thing? It looks like
mod_proxy_fcgi is in trunk now but this link has a user guide. :)

Does mod_proxy_fcgi set FCGI_WEB_SERVER_ADDRS as per the FastCGI spec?

Cheers,
Darren

Re: mod_fcgid: Problem serving binary content with Apache 2.2.13 - PHP 5.2.9

[Marcus Merz]

Jeff Trawick traw...@gmail.com schrieb im Newsbeitrag 
news:cc67648e0910121343j7e922d94q1f96ee89cd897...@mail.gmail.com...
 On Mon, Oct 12, 2009 at 4:14 PM, Jeff Trawick traw...@gmail.com wrote:
 On Sun, Oct 11, 2009 at 6:16 AM, Marcus Merz mm...@gmx.de wrote:
 Is this issue solved in 2.3.4?

 no

 BTW, I'm sorry I haven't followed up to your last couple of posts.  I
 should be honest with you:  All that Plesk-generated config and/or
 attempts to interject tweaks to that config makes my head explode.
 Maybe somebody else here has the time and will power to dig through
 it.

...

 Good luck!


Well, it's been a while but as i want other people to find a solution should 
they dig in old archives via Google, i wanted to share wiht you how i got 
around 'the wonderful world of Plesk' (no, i did not deinstall it...). 
Please acknowledge that i am neither a guru of Apache, mod_fcgid nor PHP. I 
simply want to help people who might face the same problem than i did about 
6 months ago. If you read this whole thread, then you know about the 
problems arising in certain -special- environments when you want to use 
mod_fcgid (i.e. active watermarking images like i did).

My current environment as of writing this is:
- suse-10.3-x86_64-plesk9
- Plesk 9.2.3
- Apache 2.2.4 with MPM-worker enabled
- PHP 5.2.9
- mod_fcgid 2.3.5 (with 'FcgidFixPathinfo 1' in 
/etc/apache2/conf.d/mod_fcgid.conf)

The 'FcgidFixPathinfo 1' in /etc/apache2/conf.d/mod_fcgid.conf corresponds 
to
---
; cgi.fix_pathinfo provides *real* PATH_INFO/PATH_TRANSLATED support for 
CGI.  PHP's
; previous behaviour was to set PATH_TRANSLATED to SCRIPT_FILENAME, and to 
not grok
; what PATH_INFO is.  For more information on PATH_INFO, see the cgi specs. 
Setting
; this to 1 will cause PHP CGI to fix its paths to conform to the spec.  A 
setting
; of zero causes PHP to behave as before.  Default is 1.  You should fix 
your scripts
; to use SCRIPT_FILENAME rather than PATH_TRANSLATED.
; http://php.net/cgi.fix-pathinfo
;cgi.fix_pathinfo=1
---

in /etc/php5/fastcgi/php.ini as cgi.fix_pathinfo=1 is the default. I mention 
it here because i refer to it later.


To avoid problems in an active watermark php script (caused by SetHandler 
and FCGIWrapper directives - please read the entire thread for details), 
this is what i did:

In Plesk, choose PHP Support via CGI from the dropdown menue in your 
domain Web Hosting Setup.

This will add
---
Files ~ (\.php)
AddHandler php-script .php
Options +ExecCGI
allow from all
/Files
---

to your domain/conf/httpd.include file. Now, how do we get to use 
mod_fcgid then?

In /etc/sysconfig/apache2 you (hopefully) have:
---
APACHE_MODULES=modules cgi moremodules fcgid evenmoremodules suexec
---

so mod_fcgid.so will be included in 
/etc/apache2/sysconfig.d/loadmodule.conf.

In the order you included your modules above, mod_cgi.so will be implemented 
before mod_fcgid.so and therefore /etc/apache2/conf.d/php_cgi.conf will be 
parsed before /etc/apache2/conf.d/mod_fcgid.conf. Do i make sense?

What does this mean?

In /etc/apache2/conf.d/php_cgi.conf we have
---
scriptAlias /phppath/ /usr/bin/
Action php-script /phppath/php-cgi5
---

(this was mentioned in here before). Actually this causes all PHP scripts to 
be parsed by mod_cgi. Hmm Fine. Let's add a vhost.conf in 
/srv/www/vhosts/domain/conf/

like this: /srv/www/vhosts/domain/conf/vhost.conf
---
OtherImportantStuff
Include /srv/www/vhosts/global.vhost.conf
---

What? This is because i want ALL domains to use PHP via mod_fcgid yet i do 
not want to change every vhost.conf manually which is why i use this 
'Include'.

/srv/www/vhosts/global.vhost.conf:
---
IfModule mod_fcgid.c
Alias /phpfcgidpath/ /usr/bin/
Action php-script /phpfcgidpath/php-cgi5
Location /phpfcgidpath/
SetHandler fcgid-script
Options +ExecCGI
/Location
/IfModule
---

Now, what do we do here? Well, we define an alias similiar like in 
/etc/apache2/conf.d/php_cgi.conf but via the Action statement and the 
following Location container, all Files ending (\.php) will now get parsed 
using mod_fcgid because of the
---
SetHandler fcgid-script
---

If you have set 'FcgidFixPathinfo 1' in 
/etc/apache2/conf.d/mod_fcgid.conf -as explained above- calling phpinfo() 
will tell you this
---
_SERVER[ORIG_SCRIPT_NAME]   /phpfcgidpath/php-cgi5
---

which means, mod_fcgid has been used to parse the php file. This is what we 
wanted to achieve.

Instead of using a (global) vhost.conf file, you could probably put 
everything into /etc/apache2/conf.d/mod_fcgid.conf but this is like with 
Rome: There are more than one ways to achieve your goal. I just wrote about 
how I succeeded. Feel free to find your own way.

HTH.
Marcus

Re: mod_fcgid: Problem serving binary content with Apache 2.2.13 - PHP 5.2.9

[Jeff Trawick]

On Tue, Feb 23, 2010 at 4:47 PM, Marcus Merz mm...@gmx.de wrote:
 Well, it's been a while but as i want other people to find a solution should
 they dig in old archives via Google,

Thanks for the good summary.

Re: mod_fcgid versioning question

[Jeff Trawick]

On Fri, Jan 22, 2010 at 2:14 PM, Chris Darroch chr...@pearsoncmg.com wrote:
 Chris Darroch wrote:

 Jeff Trawick wrote:

 Chris Darroch has a patch for that, which is a small part of


 http://people.apache.org/~chrisd/patches/mod_fcgid_auth/mod_fcgid-1auth-trunk.patch

 [snip]

   Yes -- and I'm really, really going to prioritize breaking this
 patchset up (I know I'm a moron for having run everything together)
 next week and trying to commit portions one at a time to mod_fcgid trunk.


  It's a new year and I've got a window of time I'm aiming to devote
 to breaking up this patchset.  (Lord, what a fool this mortal is.)
 I've begun doing that work, and I've got some initial bits I'd like to
 commit soon.

Cool!


  However, I want to check on our versioning rules for a module like
 mod_fcgid.  Are we following APR's guidelines, or something else?
 Specifically, what counts as a public function or structure, if anything?
 Things defined in mod_fcgid.h?  Things starting with fcgid_?  Nothing?

mod_fcgid doesn't currently provide a callable API.  There are no
public functions or structures.

  Two concrete examples.  First, just for cleanliness, I'd like to
 rename auth_conf to fcgid_auth_conf in order to match fcgid_wrapper_conf
 and other things in fcgid_conf.h.  Second, I need to add an argument
 to handle_request()'s signature.  May I do both of these in 2.3.6?

Yes

  If I read the STATUS-FCGID file correctly, all 2.3.x releases are
 alpha/beta ... does that mean I can change anything at all up until we
 cut a 2.4.x?

2.3.4 is GA already.  2.3.5 is being tested now and hopefully will be a GA too.

Plz hold off on any serious changes until we get a new GA release, in
case any regressions in the as-yet-untested/unreleased 2.3.5 need to
be corrected.

Re: mod_fcgid versioning question

[Chris Darroch]

Jeff Trawick wrote:


mod_fcgid doesn't currently provide a callable API.  There are no
public functions or structures.


  That's what I thought, but it's great to have it confirmed.


2.3.4 is GA already.  2.3.5 is being tested now and hopefully will be a GA too.

Plz hold off on any serious changes until we get a new GA release, in
case any regressions in the as-yet-untested/unreleased 2.3.5 need to
be corrected.


  Will do.  I've tested 2.3.5 lightly; will vote shortly.

Chris.

--
GPG Key ID: 366A375B
GPG Key Fingerprint: 485E 5041 17E1 E2BB C263  E4DE C8E3 FA36 366A 375B

Re: [mod_fcgid PATCH] catch errors from setuid()/seteuid()

[Jeff Trawick]

On Wed, Jan 20, 2010 at 8:19 PM, pqf p...@mailtech.cn wrote:
 I man seteuid in my Linux box, there are two types of errors:
 ERRORS
       The seteuid() function shall fail if:

       EINVAL The value of the uid argument is invalid and is not supported by 
 the implementation.

       EPERM  The  process  does not have appropriate privileges and uid does 
 not match the real group ID or the saved set-group-
              ID.

 If directly pass 0 in setuid(), EINVAL may not happend
 If this process is seteuid from root, EPERM may not happend

 so, I think the check is just a textbook logic check?

yes, until somebody changes code or some other bug results in this
being called in a different environment

 just call _exit(1) if it fail?

two concerns with that minimal change:

1. seteuid() works once then fails n times now (at least on Solaris),
so some extra logic is needed
2. even if these calls never fail, the presence of the exit() without
a log message may cause somebody to lose a lot of time investigating a
mysterious disappearance of the new process

--/--

I'll punt on this until after 2.3.5 since I'd like to spend the time
to watch it work on another platform or two.  (suexec is not something
I use more than once every ~3 years, so it is worth setting up in
multiple environments.)

Re: [mod_fcgid PATCH] catch errors from setuid()/seteuid()

[pqf]

Hi, Jeff
Your concerns are right, +1 for your patch :)

Thanks

--
From: Jeff Trawick traw...@gmail.com
Sent: Thursday, January 21, 2010 9:23 PM
To: dev@httpd.apache.org
Subject: Re: [mod_fcgid PATCH] catch errors from setuid()/seteuid()

 On Wed, Jan 20, 2010 at 8:19 PM, pqf p...@mailtech.cn wrote:
 I man seteuid in my Linux box, there are two types of errors:
 ERRORS
   The seteuid() function shall fail if:

   EINVAL The value of the uid argument is invalid and is not supported 
 by the implementation.

   EPERM  The  process  does not have appropriate privileges and uid does 
 not match the real group ID or the saved set-group-
  ID.

 If directly pass 0 in setuid(), EINVAL may not happend
 If this process is seteuid from root, EPERM may not happend

 so, I think the check is just a textbook logic check?
 
 yes, until somebody changes code or some other bug results in this
 being called in a different environment
 
 just call _exit(1) if it fail?
 
 two concerns with that minimal change:
 
 1. seteuid() works once then fails n times now (at least on Solaris),
 so some extra logic is needed
 2. even if these calls never fail, the presence of the exit() without
 a log message may cause somebody to lose a lot of time investigating a
 mysterious disappearance of the new process
 
 --/--
 
 I'll punt on this until after 2.3.5 since I'd like to spend the time
 to watch it work on another platform or two.  (suexec is not something
 I use more than once every ~3 years, so it is worth setting up in
 multiple environments.)
 

Re: [mod_fcgid PATCH] catch errors from setuid()/seteuid()

[pqf]

I man seteuid in my Linux box, there are two types of errors:
ERRORS
   The seteuid() function shall fail if:

   EINVAL The value of the uid argument is invalid and is not supported by 
the implementation.

   EPERM  The  process  does not have appropriate privileges and uid does 
not match the real group ID or the saved set-group-
  ID.

If directly pass 0 in setuid(), EINVAL may not happend
If this process is seteuid from root, EPERM may not happend

so, I think the check is just a textbook logic check? just call _exit(1) if it 
fail? 


--
From: Jeff Trawick traw...@gmail.com
Sent: Thursday, January 21, 2010 5:38 AM
To: dev@httpd.apache.org
Subject: [mod_fcgid PATCH] catch errors from setuid()/seteuid()

 During the last hackathon, Paul was kind enough to run the clang/llvm
 static analysis on mod_fcgid
 (http://zeus.kimaker.com/~chip/fcgid-scan/).  That pointed out these
 setuid()/seteuid() calls that aren't checked prior to running a child.
 
 The error checking itself is simple enough, but there's an ugly aspect
 of the implementation that results in trying to switch effective/real
 uids multiple times that I worked around.  (See the FIXME text in the
 patch.  I'm not aware of a simple solution, especially one simple
 enough to get into 2.3.5)  The seteuid() call would otherwise fail on
 subsequent invocations for the same child.
 
 IIRC Joe thought that the seteuid() wasn't needed at all, but the
 setuid() fails without it on Solaris.
 
 Concerns?
 
 Is there some reason that testing on Linux and Solaris wouldn't be sufficient?
 

Re: mod_fcgid, graceful restart

[Brad Plant]

On a slightly related note - I have seen that the cgi-handler continues to 
function for a GET request after a graceful restart, but not for a POST request.

Cheers,

Brad


On Wed, 13 Jan 2010 12:57:10 +0300
Alexey Vlasov ren...@renton.name wrote:

 Hi.
 
 It seems to me that graceful restart doesn't work in mod_fcgid at all.
 
 My configuration:
 IfModule mod_fcgid.c
 FcgidProcessTableFile /tmp/fcgidshm_aux7
 FcgidIPCDir /tmp/.fcgidsock_aux7
 FcgidMinProcessesPerClass 0
 FcgidIdleTimeout 300
 FcgidIdleScanInterval 1
 FcgidProcessLifeTime 150
 FcgidErrorScanInterval 150
 FcgidFixPathinfo 1
 /IfModule
 ...
 VirtualHost *
 IfModule fcgid_module
 FcgidWrapper /full/path/to/wrapper .php
 FcgidIOTimeout 150
 /IfModule
 /VirtualHost
 
 I made a simple script to check my guess:
 $ cat sleep.php
 ?php sleep (60); echo OK; ?
 
 Checking without Apache restart:
 $ time wget http://test-aux7/sleep.php
 --2010-01-12 16:53:08--  http://test-l12-aux7/sleep.php
 Resolving test-aux7... 111.222.11.22
 Connecting to test-aux7|111.222.11.22|:80... connected.
 HTTP request sent, awaiting response... 200 OK
 Length: unspecified [text/html]
 Saving to: `sleep.php'
 
 [ = 
] 2   --.-K/s   in 0s
 
 2010-01-12 16:54:08 (258 KB/s) - `sleep.php' saved
 
 real1m0.466s
 user0m0.000s
 sys 0m0.004s
 
 $ cat sleep.php
 OK
 Here is everything ok.
 
 Now with graceful restart:
 time wget http://test-aux7:80/sleep.php
 --2010-01-12 16:57:15--  http://test-aux7:80/sleep.php
 Resolving test-aux7... 111.222.11.22
 Connecting to test-aux7|111.222.11.22|:80... connected.
 HTTP request sent, awaiting response... No data received.
 Retrying.
 
 --2010-01-12 16:57:22--  (try: 2)  http://test-aux7:80/sleep.php
 Connecting to test-aux7|111.222.11.22|:80... connected.
 HTTP request sent, awaiting response...
 
 Here what goes in error_log:
 [Tue Jan 12 16:55:27 2010] [emerg] [client 10.0.2.11] (22)Invalid argument: 
 mod_fcgid: can't lock process table in pid 4172
 

Re: mod_fcgid 2.3.5 release ?

[Jeff Trawick]

On Mon, Dec 21, 2009 at 11:11 AM, Hanno Böck ha...@hboeck.de wrote:
 Hi,

 Today I stepped over the upload corruption bug affecting the current 2.3.4
 version of mod_fcgid:

 http://drupal.org/node/635270
 http://www.mail-archive.com/dev@httpd.apache.org/msg46230.html

 As this is a rather serious issue, I wanted to ask if you can push a new
 release of mod_fcgid. I think this really deserves a fast new release.

This was discussed briefly last week, and some initial testing was
done in preparation.

Re: mod_fcgid, mod_ftp both ready to tag?

[Jeff Trawick]

On Mon, Dec 14, 2009 at 6:17 PM, Jeff Trawick traw...@gmail.com wrote:
 On Mon, Dec 14, 2009 at 5:36 PM, William A. Rowe Jr.
 wr...@rowe-clan.net wrote:
 I was going to make a couple fixes in the morning to abstract out the
 QoS related calls in mod_ftp, and then tag both of these if folks think
 they are ready for a tag.  Any commits still incoming?

 Tomorrow after noon EST would work great for me.  I don't have any
 commits close to ready, but I'd like to look around first.

fcgid looks good here...

Re: mod_fcgid, mod_ftp both ready to tag?

[Jeff Trawick]

On Mon, Dec 14, 2009 at 5:36 PM, William A. Rowe Jr.
wr...@rowe-clan.net wrote:
 I was going to make a couple fixes in the morning to abstract out the
 QoS related calls in mod_ftp, and then tag both of these if folks think
 they are ready for a tag.  Any commits still incoming?

Tomorrow after noon EST would work great for me.  I don't have any
commits close to ready, but I'd like to look around first.

Re: [mod_fcgid] can't find apr headers

[Carsten Wiedmann]

Jeff Trawick schrieb:
 /usr/local/build/config_vars.mk includes /usr/local/include/apr-1 in
 EXTRA_INCLUDES
   (if not, what is EXTRA_INCLUDES set to?)
 No, the apr include dir is not part of EXTRA_INCLUDES. The apr inlude dir is
 defined in APR_INCLUDEDIR.
 
 The APR include dir should appear within both APR_INCLUDEDIR and
 EXTRA_INCLUDES.  Report that to your httpd provider, or open a bug at
 https://issues.apache.org/bugzilla/ if you built it yourself.

This only happens, if you build httpd with a preinstalled APR, but not with
the bundled APR (With a preinstalled APR httpd is also using apr-1-config to
get the values, like apxs -c). With the bundled APR the APR_INCLUDEDIR dir
is not added to EXTRA_INCLUDES.

With the bundled APR, that's only working for (most) people, because
apr/apr-util doesn't (normally) respect it's own config.layout (if
--enable-layout is used), and so the httpd and APR headers are going to the
same directory, which is the directory defined in includedir in
condig_vars.mk.


 And and least I have to set e.g.:
 | FcgidIPCDir /var/run/
 | FcgidProcessTableFile /var/run/fcgid_shm
 Or Apache can't start: No access/exists to the default values of these two
 directives.
 
 the enclosing directories exist but you can't write to it?

Correct, and my example above is also wrong (sorry, a problem with my copy
'n paste). It must be e.g.:
| FcgidIPCDir /var/run/fcgidsock

It's just that people must know, that
a) FcgidProcessTableFile is created by the startup process (uid: root)
b) the files in FcgidIPCDir are created by a worker process (e.g. uid: www)

and for b) and if the configured FcgidIPCDir already exists, this user must
have privs to create something in this directory.
If only the last part of FcgidIPCDir does not exists, the startup process is
creating this directory with the correct owner.


 Maybe e.g. TMPDIR should be used as default?
 
 The defaults are logs/fcgidsock and logs/fcgid_shm under
 ServerRoot.  I think DEFAULT_REL_RUNTIMEDIR is a better choice than
 hard-coded logs, but that wouldn't help you since your ServerRoot
 has no logs directory or you have no write access anyway.  I don't
 think a system-wide directory is an appropriate default.  (mod_fcgid
 currently creates lock files there :( ).

TMPDIR is just an example, because:
- there is always a way to get a valid TMPDIR path
- on my system it has the sticky bit, so others can't e.g. delete my
FcgidProcessTableFile
- if we assume there is no fcgidsock subdir at this time, the startup
process is creating this dir with exclusive privs for e.g. the www user.

(- well, and a log dir is for logfiles and not for e.g. sockets ;-) )


 So for me there is only one problem left. php_apc does not work together
 with mod_fcgid. And the question, who the guilty party is (mod_fcgid or
 apc...). Only the first request for a file is working, a core dump at the
 second, and so on.
 
 As for the core dump, open a bug with the component that is
 segfaulting. (I'm guessing PHP or APC, which are developed at
 php.net.)
 
 Somewhat separately, can APC work as desired with mod_fcgid barring
 any implementation bugs?  I don't know.  See the discussion in
 https://issues.apache.org/bugzilla/show_bug.cgi?id=48110

Hm, I guess it's a problem with APC. There are also bug reports in the APC
bug tracker regarding fast-cgi. Of course, eAccelerator is working with
fast-cgi. Well, I don't need this progressbar feature from APC, so
eAccelerator is also ok.


Regards,
Carsten