Re: project website update

[Dave Fisher]

> On Sep 8, 2021, at 5:05 AM, ste...@eissing.org wrote:
> 
> 
> 
>> Am 08.09.2021 um 12:53 schrieb Christophe JAILLET 
>> :
>> 
>> Le 08/09/2021 à 12:13, ste...@eissing.org a écrit :
>>> Need help to figure out how out website updates now after the "move" 
>>> (whatever that means).
>>> Formerly:
>>> - changes committed into 
>>> https://svn.apache.org/repos/asf/httpd/site/trunk/content
>> Changes committed in https://github.com/apache/httpd-site
>> (so needs to be listed as a contributor, I guess)
> 
> Thanks, Christophe. So that needs some git handling to update the site. Hmm, 
> either
> being submitter or make a fork and create a PR. The former sounds simpler.

As the one who moved the site for Infra I suggest that you clone from either 
Gitbox or Github. Make your changes, add the files, commit, and push.

If you use Github then you may need to associate your GitHub with your Apache 
ID. Go to id.apache.org <http://id.apache.org/> or check your Apache details at 
whimsy.apache.org <http://whimsy.apache.org/>

Your Github account must have 2FA setup. Changes to Github access rights are 
pushed hourly from Gitbox to Github.

Regards,
Dave

> 
>>> - TMP_KEY for from https://cms.apache.org/httpd/publish
>>> - POST with same form fields done to trigger the update (I assume)
>> Nothing to do, it is now done automatically by some magic put in place by 
>> infra.
>> When pushed on github, it lands on h.a.o.
> 
> Ack.
> 
>> CJ
>> 
>>> So, what is now the way?
>>> - Stefan
>> 
> 

Re: release?

[Dave Fisher]

> On Aug 31, 2021, at 4:12 AM, Daniel Ruggeri  wrote:
> 
> 
> On 8/30/2021 3:53 PM, Christophe JAILLET wrote:
>> 
>> Le 30/08/2021 à 13:53, Eric Covener a écrit : 
>>> On Mon, Aug 30, 2021 at 7:36 AM ste...@eissing.org 
>>>   
>>>  wrote: 
 In what state is our release handling? Given someone holding my hand, 
 could I do it? Or is it better to look someone over the shoulder while he 
 does it? 
>>> If there is an over-the-shoulder session I would like to tag along.  I 
>>> am flexible on time of day but I am GMT+4 (EDT).  I can host on webex. 
>>> Otherwise if you just want to struggle through it I can tag along but 
>>> I have no experience. 
>> 
>> I can give another try with my limited experience. 
>> 
>> I definitively would like to add a --dry-run option for all scripts so that 
>> they can be run for learning purpose without the fear of un-expected impact 
>> on svn. 
> FWIW, the announce.sh script which collates all the security "stuff" and 
> sends the announce emails drops the user to a shell to inspect/examine what 
> WILL happen before actually doing anything. Any non-zero return code of that 
> shell will abort the process. I used the heck out of that several times :-)
> 
> 
> 
>> 
>> Existing scripts are not that easy to read at first, but are understanbdable 
>> and followinghttp://httpd.apache.org/dev/release.html#how-to-do-a-release 
>>  helps a lot. 
>> The scripts should also be tweaked because of the latest changes in several 
>> places (at least web site update (now on github) and CVE announcement (if 
>> any) now that part of the process is handled elsewhere). 
>> 
> 
> +1
> 
> To my knowledge, the publishing of the site and overhaul of the new CVE 
> process are the things requiring updates.
> 

The JSON files for the release’s CVEs should be committed here: 
https://github.com/apache/httpd-site/tree/main/content/security/json 
 : 
https://gitbox.apache.org/repos/asf?p=httpd-site.git;a=tree;f=content/security/json;hb=HEAD
 



> -- 
> Daniel Ruggeri
>> The CVE announcement should be much easier, now that we have a "Send these 
>> Emails" on cveprocess.a.o. This should simplify part of the process where we 
>> were preparing some scripts to send the announcement emails. 
>> 
>> I've been lacking time for httpd since many weeks, but I should be able to 
>> RM next week if needed. 
>> 
>> CJ 
>> 
>>> Also: Anyone who has a showstopper to delay a release (even if not yet 
>>> proposed) please add it to 2.4.x STATUS so we can get things in order. 
>>> 

Re: svn commit: r1891407 - /httpd/site/trunk/content/docs-project/contribute.mdtext

[Dave Fisher]

I went ahead and made this change to GitHub.com/apache/httpd-site

I also made a change to the newest pelican build method in 
GitHub.com/apache/infrastructure-pelican

I’ll be monitoring dev@httpd.a.o <mailto:dev@httpd.a.o> for a few more days 
before I unsubscribe.

All The Best,
Dave

> On Jul 9, 2021, at 3:41 PM, Dave Fisher  wrote:
> 
> Yes.
> 
> Sent from my iPhone
> 
>> On Jul 9, 2021, at 1:17 PM, Ruediger Pluem  wrote:
>> 
>> 
>> 
>>> On 7/9/21 2:39 PM, rbo...@apache.org wrote:
>>> Author: rbowen
>>> Date: Fri Jul  9 12:39:29 2021
>>> New Revision: 1891407
>>> 
>>> URL: http://svn.apache.org/viewvc?rev=1891407=rev
>>> Log:
>>> s/freenode/libera/
>>> 
>>> Modified:
>>>   httpd/site/trunk/content/docs-project/contribute.mdtext
>>> 
>>> Modified: httpd/site/trunk/content/docs-project/contribute.mdtext
>>> URL: 
>>> http://svn.apache.org/viewvc/httpd/site/trunk/content/docs-project/contribute.mdtext?rev=1891407=1891406=1891407=diff
>>> ==
>>> --- httpd/site/trunk/content/docs-project/contribute.mdtext (original)
>>> +++ httpd/site/trunk/content/docs-project/contribute.mdtext Fri Jul  9 
>>> 12:39:29 2021
>>> @@ -40,7 +40,7 @@ You can:
>>> list](mailto:docs-subscr...@httpd.apache.org) and tell us something that
>>> could be improved.
>>> 
>>> -- Join #httpd on irc.freenode.net and tell us there.
>>> +- Join #httpd on irc.libera.chat and tell us there.
>>> 
>>> - Submit a ticket to the [bug database](../bug_report.html).
>> 
>> I guess with the site now moved over to Pelican / github this would need to 
>> fixed here:
>> https://github.com/apache/httpd-site/blob/main/content/docs-project/contribute.md
>> 
>> See also: https://infra.apache.org/asf-pelican.html
>> 
>> Regards
>> 
>> Rüdiger
>> 
> 

Re: svn commit: r1891407 - /httpd/site/trunk/content/docs-project/contribute.mdtext

[Dave Fisher]

Yes.

Sent from my iPhone

> On Jul 9, 2021, at 1:17 PM, Ruediger Pluem  wrote:
> 
> 
> 
>> On 7/9/21 2:39 PM, rbo...@apache.org wrote:
>> Author: rbowen
>> Date: Fri Jul  9 12:39:29 2021
>> New Revision: 1891407
>> 
>> URL: http://svn.apache.org/viewvc?rev=1891407=rev
>> Log:
>> s/freenode/libera/
>> 
>> Modified:
>>httpd/site/trunk/content/docs-project/contribute.mdtext
>> 
>> Modified: httpd/site/trunk/content/docs-project/contribute.mdtext
>> URL: 
>> http://svn.apache.org/viewvc/httpd/site/trunk/content/docs-project/contribute.mdtext?rev=1891407=1891406=1891407=diff
>> ==
>> --- httpd/site/trunk/content/docs-project/contribute.mdtext (original)
>> +++ httpd/site/trunk/content/docs-project/contribute.mdtext Fri Jul  9 
>> 12:39:29 2021
>> @@ -40,7 +40,7 @@ You can:
>> list](mailto:docs-subscr...@httpd.apache.org) and tell us something that
>> could be improved.
>> 
>> -- Join #httpd on irc.freenode.net and tell us there.
>> +- Join #httpd on irc.libera.chat and tell us there.
>> 
>> - Submit a ticket to the [bug database](../bug_report.html).
> 
> I guess with the site now moved over to Pelican / github this would need to 
> fixed here:
> https://github.com/apache/httpd-site/blob/main/content/docs-project/contribute.md
> 
> See also: https://infra.apache.org/asf-pelican.html
> 
> Regards
> 
> Rüdiger
> 

Re: migration of the HTTPD project website

[Dave Fisher]

> On Jul 1, 2021, at 11:40 PM, Ruediger Pluem  wrote:
> 
> 
> 
> On 7/1/21 10:49 PM, Christophe JAILLET wrote:
>> Le 01/07/2021 à 18:37, Dave Fisher a écrit :
>>> I see that there is already a PR to fix the modules hyperlink. Should this 
>>> be applied 
>> 
>> From my point of view, no.
>> But /modules/ needs some clean-up.
>> 
>>> and the ASF Pelican version of httpd.apache.org be put into production in 
>>> 24 hours?
>> 
>> Still, from my point of view, +1.
> 
> +1

The migrated httpd-site is now in production. Enjoy!

Regards,
Dave

> 
> Regards
> 
> Rüdiger
> 
> 

Re: migration of the HTTPD project website

[Dave Fisher]

I see that there is already a PR to fix the modules hyperlink.

Should this be applied and the ASF Pelican version of httpd.apache.org be put 
into production in 24 hours?

HTH,
Dave

> On Jun 29, 2021, at 2:23 PM, Dave Fisher  wrote:
> 
> Hi -
> 
>> On Jun 29, 2021, at 1:17 PM, Marion & Christophe JAILLET 
>>  wrote:
>> 
>> Hi Dave,
>> 
>> Thanks for having done it.
>> 
>> You did faster and better than what I had started.
>> 
>> 
>> Here are a few details spotted here and there:
>>   - The download page looks broken
>> (https://httpd.staged.apache.org/download.cgi)
> 
> This is due to the staging environment not picking up the production pickup 
> of cgi to mirror mapping into the download.html template.
> 
>> 
>>   - some '&' have been turned into 
>> (https://httpd.staged.apache.org/dev/debugging.htmll#gcore)
> 
> Fixed by switching from ``` fenced code to 
> 
>> 
>>   - some extra spaces in a command (around the use of awk)
>> (https://httpd.staged.apache.org/dev/release.html#how-to-do-a-release)
> 
> Removed. BTW - this page and associated scripts need to be edited to reflect 
> the website being git based.
> 
>> 
>>   - a bold string left with **
>> (https://httpd.staged.apache.org/dev/styleguide.html)
> 
> Good catch Github Flavored Markdown (GFM) does not like spaces for emphasis. 
> If you want to do bold italic: it’s ‘**_word_**'
> 
>> 
>>   - a link that is left as [foo](bar) instead of an hypertext link
>> (at the very bottom of https://httpd.staged.apache.org/contributors/)
> 
> Also GFM does not do any markdown within HTML blocks. I switched from  to 
> ### heading.
> 
>> 
>> Only the first point is an issue. It is maybe linked to the fact that it is 
>> in staging?
> 
> Yes.
> 
>> 
>> I guess that all the other tiny things could be fixed easily but don't have 
>> time to look at it by myself in the coming days/weeks.
>> 
>> 
>> I don't know if you hand modified a few things, but several places looks 
>> better now (some spacing between paragraphs which are smaller now, some 
>> alignment, some missing spaces between words that have been fixed, some 
>> numbering that were broken and fixed now, some links that have been added 
>> for URL or mails). So, great work!
> 
> There were a few tweaks to obviously broken content. URLs and emails are 
> automatically turned to hyperlinks by GFM.
> 
>> Thanks a lot.
> 
> You’re welcome.
> 
> Dave
> 
>> 
>> CJ
>> 
>> 
>> 
>> Le 25/06/2021 à 19:12, Dave Fisher a écrit :
>>> The Migration from CMS to ASF-Pelican is staged!
>>> 
>>> https://httpd.staged.apache.org/ is ready.
>>> 
>>> https://github.com/apache/httpd-site/
>>> 
>>> See the README on GitHub for details.
>>> 
>>> All The Best,
>>> Dave
>>> 
>>>> On Jun 22, 2021, at 11:00 AM, Dave Fisher  wrote:
>>>> 
>>>> 
>>>> 
>>>>> On Jun 21, 2021, at 6:26 AM, Eric Covener  wrote:
>>>>> 
>>>>> On Sun, Jun 20, 2021 at 3:25 PM Andrew Wetmore  wrote:
>>>>>> Hi, Eric:
>>>>>> 
>>>>>> Yes, committers can use either GitHub or Gitbox.
>>>>>> 
>>>>>> I am not sure what the "can you give a hint about the migration aspect" 
>>>>>> means. Maybe I was not clear. We have to move all projects off the 
>>>>>> Apache CMS and to some other technology, such as Pelican. Infra can help 
>>>>>> with that move to a Git repository. Sometime this summer the Apache CMS 
>>>>>> will stop functioning, which would mean you would have a hard time 
>>>>>> updating your website.
>>>>> I was referring to  "...migration process available that could speed
>>>>> things along." not the overall migration off the CMS.
>>>>> Daniel referred to
>>>>> https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features
>>>>> but I didn't find any reference to "migration".
>>>>> 
>>>>> https://github.com/apache/httpd-site was created but it's empty.  I
>>>>> was assuming we'd have some starting point based on the CMS-based site
>>>>> content and whatever additional template/scaffolding is needed that
>>>>> we'd be able to see w/o replacing our currently published site.
>>>>> I don't want anyone to start from scratch if there's a better way to
>>>>> get started.
>>>> I have the go ahead to do the migration for you. The goal is to create a 
>>>> staged site that will be nearly identical to your current site.
>>>> 
>>>> Expect more information this week.
>>>> 
>>>> All The Best,
>>>> Dave
>>>> 
>>>> 
> 

Re: migration of the HTTPD project website

[Dave Fisher]

Sent from my iPhone

> On Jun 29, 2021, at 11:50 PM, Ruediger Pluem  wrote:
> 
> 
> 
>> On 6/29/21 11:23 PM, Dave Fisher wrote:
>> Hi -
>> 
>>>> On Jun 29, 2021, at 1:17 PM, Marion & Christophe JAILLET 
>>>>  wrote:
>>> 
> 
>>> 
>>>   - some extra spaces in a command (around the use of awk)
>>> (https://httpd.staged.apache.org/dev/release.html#how-to-do-a-release)
>> 
>> Removed. BTW - this page and associated scripts need to be edited to reflect 
>> the website being git based.
>> 
> 
> I guess he talks about
> 
> # Get the tooling
> svn co https://svn.apache.org/repos/asf/httpd/site/trunk/tools tools
> 
> I think we should find a new location for this as this is not really site 
> related.
> How about
> 
> https://svn.apache.org/repos/asf/httpd/dev-tools/trunk
> 
> (svn mv https://svn.apache.org/repos/asf/httpd/site/trunk/tools 
> https://svn.apache.org/repos/asf/httpd/dev-tools/trunk)
> 

That’s the first part.

Then scripts like announce.sh need to be modified.

Regards,
Dave

> Regards
> 
> Rüdiger

Re: migration of the HTTPD project website

[Dave Fisher]

Hi -

> On Jun 29, 2021, at 1:17 PM, Marion & Christophe JAILLET 
>  wrote:
> 
> Hi Dave,
> 
> Thanks for having done it.
> 
> You did faster and better than what I had started.
> 
> 
> Here are a few details spotted here and there:
>- The download page looks broken
>  (https://httpd.staged.apache.org/download.cgi)

This is due to the staging environment not picking up the production pickup of 
cgi to mirror mapping into the download.html template.

> 
>- some '&' have been turned into 
>  (https://httpd.staged.apache.org/dev/debugging.htmll#gcore)

Fixed by switching from ``` fenced code to 

> 
>- some extra spaces in a command (around the use of awk)
> (https://httpd.staged.apache.org/dev/release.html#how-to-do-a-release)

Removed. BTW - this page and associated scripts need to be edited to reflect 
the website being git based.

> 
>- a bold string left with **
>  (https://httpd.staged.apache.org/dev/styleguide.html)

Good catch Github Flavored Markdown (GFM) does not like spaces for emphasis. If 
you want to do bold italic: it’s ‘**_word_**'

> 
>- a link that is left as [foo](bar) instead of an hypertext link
>  (at the very bottom of https://httpd.staged.apache.org/contributors/)

Also GFM does not do any markdown within HTML blocks. I switched from  to 
### heading.

> 
> Only the first point is an issue. It is maybe linked to the fact that it is 
> in staging?

Yes.

> 
> I guess that all the other tiny things could be fixed easily but don't have 
> time to look at it by myself in the coming days/weeks.
> 
> 
> I don't know if you hand modified a few things, but several places looks 
> better now (some spacing between paragraphs which are smaller now, some 
> alignment, some missing spaces between words that have been fixed, some 
> numbering that were broken and fixed now, some links that have been added for 
> URL or mails). So, great work!

There were a few tweaks to obviously broken content. URLs and emails are 
automatically turned to hyperlinks by GFM.

> Thanks a lot.

You’re welcome.

Dave

> 
> CJ
> 
> 
> 
> Le 25/06/2021 à 19:12, Dave Fisher a écrit :
>> The Migration from CMS to ASF-Pelican is staged!
>> 
>> https://httpd.staged.apache.org/ is ready.
>> 
>> https://github.com/apache/httpd-site/
>> 
>> See the README on GitHub for details.
>> 
>> All The Best,
>> Dave
>> 
>>> On Jun 22, 2021, at 11:00 AM, Dave Fisher  wrote:
>>> 
>>> 
>>> 
>>>> On Jun 21, 2021, at 6:26 AM, Eric Covener  wrote:
>>>> 
>>>> On Sun, Jun 20, 2021 at 3:25 PM Andrew Wetmore  wrote:
>>>>> Hi, Eric:
>>>>> 
>>>>> Yes, committers can use either GitHub or Gitbox.
>>>>> 
>>>>> I am not sure what the "can you give a hint about the migration aspect" 
>>>>> means. Maybe I was not clear. We have to move all projects off the Apache 
>>>>> CMS and to some other technology, such as Pelican. Infra can help with 
>>>>> that move to a Git repository. Sometime this summer the Apache CMS will 
>>>>> stop functioning, which would mean you would have a hard time updating 
>>>>> your website.
>>>> I was referring to  "...migration process available that could speed
>>>> things along." not the overall migration off the CMS.
>>>> Daniel referred to
>>>> https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features
>>>> but I didn't find any reference to "migration".
>>>> 
>>>> https://github.com/apache/httpd-site was created but it's empty.  I
>>>> was assuming we'd have some starting point based on the CMS-based site
>>>> content and whatever additional template/scaffolding is needed that
>>>> we'd be able to see w/o replacing our currently published site.
>>>> I don't want anyone to start from scratch if there's a better way to
>>>> get started.
>>> I have the go ahead to do the migration for you. The goal is to create a 
>>> staged site that will be nearly identical to your current site.
>>> 
>>> Expect more information this week.
>>> 
>>> All The Best,
>>> Dave
>>> 
>>> 

Re: migration of the HTTPD project website

[Dave Fisher]

> On Jun 28, 2021, at 10:40 AM, Eric Covener  wrote:
> 
> I spot checked and it looked good to me.  Thanks Dave!

You’re welcome! My pleasure!

> 
> Can anyone else review please so we can cut over and get off the naughty list?

Let me know and I’ll make the changes it will take a few minutes.

All the best,
Dave

> 
> On Fri, Jun 25, 2021 at 1:13 PM Dave Fisher  wrote:
>> 
>> The Migration from CMS to ASF-Pelican is staged!
>> 
>> https://httpd.staged.apache.org/ is ready.
>> 
>> https://github.com/apache/httpd-site/
>> 
>> See the README on GitHub for details.
>> 
>> All The Best,
>> Dave
>> 
>>> On Jun 22, 2021, at 11:00 AM, Dave Fisher  wrote:
>>> 
>>> 
>>> 
>>>> On Jun 21, 2021, at 6:26 AM, Eric Covener  wrote:
>>>> 
>>>> On Sun, Jun 20, 2021 at 3:25 PM Andrew Wetmore  wrote:
>>>>> 
>>>>> Hi, Eric:
>>>>> 
>>>>> Yes, committers can use either GitHub or Gitbox.
>>>>> 
>>>>> I am not sure what the "can you give a hint about the migration aspect" 
>>>>> means. Maybe I was not clear. We have to move all projects off the Apache 
>>>>> CMS and to some other technology, such as Pelican. Infra can help with 
>>>>> that move to a Git repository. Sometime this summer the Apache CMS will 
>>>>> stop functioning, which would mean you would have a hard time updating 
>>>>> your website.
>>>> 
>>>> I was referring to  "...migration process available that could speed
>>>> things along." not the overall migration off the CMS.
>>>> Daniel referred to
>>>> https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features
>>>> but I didn't find any reference to "migration".
>>>> 
>>>> https://github.com/apache/httpd-site was created but it's empty.  I
>>>> was assuming we'd have some starting point based on the CMS-based site
>>>> content and whatever additional template/scaffolding is needed that
>>>> we'd be able to see w/o replacing our currently published site.
>>>> I don't want anyone to start from scratch if there's a better way to
>>>> get started.
>>> 
>>> I have the go ahead to do the migration for you. The goal is to create a 
>>> staged site that will be nearly identical to your current site.
>>> 
>>> Expect more information this week.
>>> 
>>> All The Best,
>>> Dave
>>> 
>>> 
>> 
> 
> 
> -- 
> Eric Covener
> cove...@gmail.com

Re: where do we want to send website bot notices?

[Dave Fisher]

FYI - for the website the locations are controlled from

https://github.com/apache/httpd-site/blob/main/.asf.yaml

github:
  features:
# Enable issue management
issues: true

notifications:
  commits: c...@httpd.apache.org
  issues: dev@httpd.apache.org
  pullrequests: dev@httpd.apache.org

pelican:
  notify: dev@httpd.apache.org
  autobuild: preview/*
  target: asf-site
  theme: theme/apache
  whoami: main

Github issues is optional and can be turned off.

notifications.commits are the GitHub commits.
notifications.pullrequests happen if anyone suggestions an update to the site.
pelican.notify are buildbot runs of the website.

HTH,
Dave

> On Jun 28, 2021, at 1:35 AM, Ruediger Pluem  wrote:
> 
> 
> 
> On 6/28/21 10:12 AM, Joe Orton wrote:
>> On Fri, Jun 25, 2021 at 03:50:33PM -0700, Roy T. Fielding wrote:
>>> I was about to update the site config so that it wouldn't send notices
>>> to dev, but I don't know whether they should instead go to cvs, docs, or
>>> a new list (like notices at httpd).  Any opinions?
>> 
>> We already have notifications@ which is used for GitHub PR 
>> notifications, that makes sense to me.
>> 
> 
> I don't care if it is notifications@ or cvs@ / commits@. Both locations make 
> some sense.
> 
> Regards
> 
> Rüdiger

Re: migration of the HTTPD project website

[Dave Fisher]

Hi -

Greg and discussed this situation further and I misinterpreted a comment.

I reverted this change.

Regards,
Dave

> On Jun 26, 2021, at 9:17 AM, Dave Fisher  wrote:
> 
> FYI -
> 
> The regeneration of vulnerability reports is now a manual exercise. On review 
> with Infra it was decided that running arbitrary shell scripts during a 
> pelican build is a security risk.
> 
> All the best,
> Dave
> 
>> On Jun 25, 2021, at 10:12 AM, Dave Fisher  wrote:
>> 
>> The Migration from CMS to ASF-Pelican is staged!
>> 
>> https://httpd.staged.apache.org/ is ready.
>> 
>> https://github.com/apache/httpd-site/
>> 
>> See the README on GitHub for details.
>> 
>> All The Best,
>> Dave
>> 
>>> On Jun 22, 2021, at 11:00 AM, Dave Fisher  wrote:
>>> 
>>> 
>>> 
>>>> On Jun 21, 2021, at 6:26 AM, Eric Covener  wrote:
>>>> 
>>>> On Sun, Jun 20, 2021 at 3:25 PM Andrew Wetmore  wrote:
>>>>> 
>>>>> Hi, Eric:
>>>>> 
>>>>> Yes, committers can use either GitHub or Gitbox.
>>>>> 
>>>>> I am not sure what the "can you give a hint about the migration aspect" 
>>>>> means. Maybe I was not clear. We have to move all projects off the Apache 
>>>>> CMS and to some other technology, such as Pelican. Infra can help with 
>>>>> that move to a Git repository. Sometime this summer the Apache CMS will 
>>>>> stop functioning, which would mean you would have a hard time updating 
>>>>> your website.
>>>> 
>>>> I was referring to  "...migration process available that could speed
>>>> things along." not the overall migration off the CMS.
>>>> Daniel referred to
>>>> https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features
>>>> but I didn't find any reference to "migration".
>>>> 
>>>> https://github.com/apache/httpd-site was created but it's empty.  I
>>>> was assuming we'd have some starting point based on the CMS-based site
>>>> content and whatever additional template/scaffolding is needed that
>>>> we'd be able to see w/o replacing our currently published site.
>>>> I don't want anyone to start from scratch if there's a better way to
>>>> get started.
>>> 
>>> I have the go ahead to do the migration for you. The goal is to create a 
>>> staged site that will be nearly identical to your current site.
>>> 
>>> Expect more information this week.
>>> 
>>> All The Best,
>>> Dave
>>> 
>>> 
>> 
> 

Re: migration of the HTTPD project website

[Dave Fisher]

The Migration from CMS to ASF-Pelican is staged!

https://httpd.staged.apache.org/ is ready.

https://github.com/apache/httpd-site/

See the README on GitHub for details.

All The Best,
Dave

> On Jun 22, 2021, at 11:00 AM, Dave Fisher  wrote:
> 
> 
> 
>> On Jun 21, 2021, at 6:26 AM, Eric Covener  wrote:
>> 
>> On Sun, Jun 20, 2021 at 3:25 PM Andrew Wetmore  wrote:
>>> 
>>> Hi, Eric:
>>> 
>>> Yes, committers can use either GitHub or Gitbox.
>>> 
>>> I am not sure what the "can you give a hint about the migration aspect" 
>>> means. Maybe I was not clear. We have to move all projects off the Apache 
>>> CMS and to some other technology, such as Pelican. Infra can help with that 
>>> move to a Git repository. Sometime this summer the Apache CMS will stop 
>>> functioning, which would mean you would have a hard time updating your 
>>> website.
>> 
>> I was referring to  "...migration process available that could speed
>> things along." not the overall migration off the CMS.
>> Daniel referred to
>> https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features
>> but I didn't find any reference to "migration".
>> 
>> https://github.com/apache/httpd-site was created but it's empty.  I
>> was assuming we'd have some starting point based on the CMS-based site
>> content and whatever additional template/scaffolding is needed that
>> we'd be able to see w/o replacing our currently published site.
>> I don't want anyone to start from scratch if there's a better way to
>> get started.
> 
> I have the go ahead to do the migration for you. The goal is to create a 
> staged site that will be nearly identical to your current site.
> 
> Expect more information this week.
> 
> All The Best,
> Dave
> 
> 

Re: migration of the HTTPD project website

[Dave Fisher]

> On Jun 21, 2021, at 6:26 AM, Eric Covener  wrote:
> 
> On Sun, Jun 20, 2021 at 3:25 PM Andrew Wetmore  wrote:
>> 
>> Hi, Eric:
>> 
>> Yes, committers can use either GitHub or Gitbox.
>> 
>> I am not sure what the "can you give a hint about the migration aspect" 
>> means. Maybe I was not clear. We have to move all projects off the Apache 
>> CMS and to some other technology, such as Pelican. Infra can help with that 
>> move to a Git repository. Sometime this summer the Apache CMS will stop 
>> functioning, which would mean you would have a hard time updating your 
>> website.
> 
> I was referring to  "...migration process available that could speed
> things along." not the overall migration off the CMS.
> Daniel referred to
> https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features
> but I didn't find any reference to "migration".
> 
> https://github.com/apache/httpd-site was created but it's empty.  I
> was assuming we'd have some starting point based on the CMS-based site
> content and whatever additional template/scaffolding is needed that
> we'd be able to see w/o replacing our currently published site.
> I don't want anyone to start from scratch if there's a better way to
> get started.

I have the go ahead to do the migration for you. The goal is to create a staged 
site that will be nearly identical to your current site.

Expect more information this week.

All The Best,
Dave

Re: migration of the HTTPD project website

[Dave Fisher]

Sent from my iPhone

> On Jun 18, 2021, at 5:35 AM, Daniel Gruno  wrote:
> 
> On 18/06/2021 14.33, Eric Covener wrote:
>>> On Fri, Jun 18, 2021 at 8:20 AM Daniel Gruno  wrote:
>>> 
>>> On 18/06/2021 14.16, Eric Covener wrote:
>>>> I reviewed Christophes old post, I think we should proceed with the
>>>> ASF template and migration process. Worst case Christophe can still
>>>> apply what he has learned to further refine the output of the
>>>> migration.
>>>> 
>>>> I'll create an infra ticket to ask for the repo and migration on
>>>> Monday unless someone objects.
>>> 
>>> repo request is done via selfserve.apache.org
>> Since we do not use gitbox yet: So this would result in e.g.
>> github.com/apache/httpd-site and users can then choose to interact
>> with github or gitbox.a.o, right?
> 
> Yep.

The repository is already created.

> 
>>> migrating the site (and using pelican/jbake/hugo??) is done also
>>> self-serve-y via .asf.yaml - s.apache.org/asfyaml
>> A lot of info there, I see the pelican config stuff but can you give a
>> hint about the migration aspect there?
> 
> The migration would largely just happen automatically once .asf.yaml is in 
> place and working. There is some httpd docs aliasing that infra needs to 
> figure out, but that's tangential to this.

Your whole approach to the security pages will need to change. You will need to 
produce the html as a separate step.

Also the mdtext files need to be converted to md with a manual review in order 
to catch the discrepancies between the CMS’s flavor of markdown and the GFM.

Regards,
Dave

Re: rational behind not checking the return value of apr_palloc and apr_pcalloc

[dave b]

 first the attacker has to find  a way to reduce system memory to an
 almost oom condition
 Say, by attacking several httpd threads and/or unrelated processes to
 get them to eat up memory.

 --
 Sent from my toaster.


If you know something why not share it ;) ?
imho Apache is pretty good - so perhaps you could find a commonly used
module that leaks memory?

Also, I hope your toaster is running netbsd with apache ^^

--
As flies to wanton boys are we to the gods; they kill us for their
sport.  -- Shakespeare, King Lear

Re: rational behind not checking the return value of apr_palloc and apr_pcalloc

[dave b]

On 2 September 2010 13:29, William A. Rowe Jr. wr...@rowe-clan.net wrote:
 On 9/1/2010 10:17 PM, dave b wrote:

 Why not just fix it now and not worry? ...

 It will help if you can provide a specific use case for graceful failure.

 A segfault/dereference of NULL pointer provides a very specific exception
 for this general case, and 'recovers' neatly from a process which has simply
 consumed far too much memory.  There are very few alloc exceptions which are
 recoverable a multi-client application.

 But if you can illustrate a few, the community is happy to evaluate your
 examples, which is what Jeff has politely suggested to you.



http://blog.ksplice.com/2010/03/null-pointers-part-i/
So what if NULL doesn't crash the app ^^ ?

https://www.securecoding.cert.org/confluence/display/seccode/EXP34-C.+Do+not+dereference+null+pointers

I can't see MAP_FIXED anywhere in APR or apache code though.
I am just suggesting it would be wise to prevent this from occurring.
Not every platform / linux has  /proc/sys/vm/mmap_min_addr .

Sure it is not likely to be null, but it could be :)
--
It is a wise father that knows his own child.   -- William Shakespeare,
The Merchant of Venice

Re: rational behind not checking the return value of apr_palloc and apr_pcalloc

[dave b]

 And if you can't illustrate a few explicit cases, further abstract arguments
 are likely to be politely, but firmly, ignored.  There are good C language
 forums for folks to carry on such religious arguments.

 Or to put it another way, the dev@ group here is most certainly not worried
 about the general case, as the current design is effective at terminating
 httpd when faced with runaway allocations.

Sure ok :)
You have no complains from me really here. Just this could be an issue
on some platform with some mods potentially :)

I only asked this list because the mod_wsgi guy wasn't checking the
result of memory allocation. The rational as I see it is: there is
only a few cases where this can happen 1: and 2: first the attacker
has to find  a way to reduce system memory to an almost oom condition
by the looks of it.


--
I dote on his very absence. -- William Shakespeare, The Merchant 
of Venice

Re: rational behind not checking the return value of apr_palloc and apr_pcalloc

[dave b]

On 1 September 2010 22:08, Jeff Trawick traw...@gmail.com wrote:
 On Wed, Sep 1, 2010 at 6:37 AM, Graham Dumpleton
 graham.dumple...@gmail.com wrote:

 On 1 September 2010 20:15, Graham Leggett minf...@sharp.fm wrote:
  On 01 Sep 2010, at 6:07 AM, dave b wrote:
 
  What is the rational behind not checking the return value of
  apr_palloc and apr_pcalloc?
 
  The rationale is to not be forced to check for and handle hundreds of
  potential failure cases when you're probably doomed anyway.

Yes I agree :)
Hence, why the patch I put in the bug report used assert :)


  The APR pools API gives you the apr_pool_abort_set() function, which
  specifies a function to call if the memory allocation fails. In the case
  of
  httpd, a function is registered which gracefully shuts down that
  particular
  server process if the allocation fails, and apr_palloc() is in the
  process
  guaranteed to never return NULL.

 Noting that apr_pool_abort_set() is only setup in Apache 2.3 and not
 in Apache 2.2.16 or earlier. Not being in 2.X explains why I couldn't
 find it before.

 Any reason why setting up apr_pool_abort_set() wasn't back ported to
 Apache 2.2?

 dunno, but here's the commit and original discussion in case anyone wants to
 read
 http://svn.apache.org/viewvc?view=revisionrevision=406953 (dunno if it was
 subsequently fixed)
 http://www.mail-archive.com/dev@httpd.apache.org/msg32257.html


Interesting patch, um but... I think there maybe a problem here:

from apr :)
include/apr_pools.h


#if defined(DOXYGEN)
APR_DECLARE(apr_status_t) apr_pool_create(apr_pool_t **newpool,
  apr_pool_t *parent);
#else
#if APR_POOL_DEBUG
#define apr_pool_create(newpool, parent) \
apr_pool_create_ex_debug(newpool, parent, NULL, NULL, \
 APR_POOL__FILE_LINE__)
#else
#define apr_pool_create(newpool, parent) \
apr_pool_create_ex(newpool, parent, NULL, NULL)
#endif
#endif

And apr_pool_create_ex is:


APR_DECLARE(apr_status_t) apr_pool_create_ex(apr_pool_t **newpool,
 apr_pool_t *parent,
 apr_abortfunc_t abort_fn,
 apr_allocator_t *allocator);



First thing to note is that in main apr_app_initialize[1] in
server/main.c via init_process. When this happens the global_pool will
not have a abort_fn. *However*, cntx will (it is set :) ),
process-pconf will.

Now we look in the actual code :)
See [0]
Right so if apr_pool_create gets called with null as the parent from
now on - there will be no default abort_fn.
Lets go hunting for this happens ;) see [2]

Ok so lets pull out the scoreboard now ?

/* ToDo: This function should be made to handle setting up
 * a scoreboard shared between processes using any IPC technique,
 * not just a shared memory segment
 */
static apr_status_t open_scoreboard(apr_pool_t *pconf)
{
#if APR_HAS_SHARED_MEMORY
apr_status_t rv;
char *fname = NULL;
apr_pool_t *global_pool;

/* We don't want to have to recreate the scoreboard after
 * restarts, so we'll create a global pool and never clean it.
 */
rv = apr_pool_create(global_pool, NULL);


MMM I wonder where else httpd does these kind of things ;)
Remember my simple grep can *easily* miss some :)

Ok great just get the rest of world to use apache 2.3 which has had
this patch for over 4 years now. Just one problem: I don't see anyone
running apache 2.3 alpha 8 on their production servers.

So perhaps apache 2.2 should get a backport of this 'fix' :-)
Apparently my fix that I suggested on the bug referenced in my
original email is also not going to work :) (it will not be enough :-)
)

Also, note I hit that code path via apr testdir (in one of the tests
;) ) - and I was not out of memory - I haven't checked which test it
is yet (it would be nice if the apr tests told me this...).
If the caller has hit that code path and they haven't defined an
abort_fn, I think it best to exit / abort for them - If a user wants
not to have that happen then they can just simply ensure they are
providing their own abort_fn :)
A little less conversation, a little more action please.



[0]

APR_DECLARE(apr_status_t) apr_pool_create_ex(apr_pool_t **newpool,
 apr_pool_t *parent,
 apr_abortfunc_t abort_fn,
 apr_allocator_t *allocator)
{
apr_pool_t *pool;
apr_memnode_t *node;

*newpool = NULL;

if (!parent)
parent = global_pool;

/* parent will always be non-NULL here except the first time a
 * pool is created, in which case allocator is guaranteed to be
 * non-NULL. */

if (!abort_fn  parent)
abort_fn = parent-abort_fn;

if (allocator == NULL)
allocator = parent-allocator;

if ((node = allocator_alloc(allocator,
MIN_ALLOC

Re: rational behind not checking the return value of apr_palloc and apr_pcalloc

[dave b]

 My 2 cents:

 I doubt that any of the core devs are going to match you for devotion to
 this topic, but I'm sure we will review patches to trunk to fix somewhat
 practical scenarios, such as ensuring that memory allocation failures during
 request processing go through the common abort function, if the existing
 code doesn't already handle that.  (OTOH, I wouldn't think many would find
 addressing un-aborted alloc failures during initialization so interesting.)

Sorry I have a habit of measuring things twice and cutting once ;)


 That requires some investigation/understanding of the context of the code
 that passes NULL for the parent pool, which hopefully you can make an
 attempt at without dumping the grep output and selected source code to the
 mailing list ;)
 (I know that sounds rude, but its my best advice to pursuing that thread of
 investigation.)

It doesn't sound rude. It sounds like you want me to do the work for you.
The code dump and grep was to give some examples.
I think without the code dump some would have problems following  that
the global_pool abort_fn is not to abort when it would otherwise
return NULL ( I believe this to be the case atm at least :) ).


 What's the big picture here from your standpoint?  Why is 2.2 noticeably
 impaired without such changes?  Are there really many users who need that
 abort function to tell them that httpd can't alloc more heap?  Anyway, if
 2.3 doesn't really have this suitably addressed it would be better to get
 that finished first.

I would need to do a lot more testing and I don't have much time for
that presently. I will have a poke around later in a vm after making
some modifications to the code base to try and trigger somethings :)
IMHO the apr init function, which inits the global_pool, could take in
an argument for the default abort_fn.



 Also, note I hit that code path via apr testdir (in one of the tests
 ;) ) - and I was not out of memory - I haven't checked which test it
 is yet (it would be nice if the apr tests told me this...).
 If the caller has hit that code path and they haven't defined an
 abort_fn, I think it best to exit / abort for them - If a user wants
 not to have that happen then they can just simply ensure they are
 providing their own abort_fn :)

 d...@apr.apache.org, but first try gdb or testall -v or anything else that
 you need to find exactly what/where the failure is

ah there is a -v !

--
The ripest fruit falls first.   -- William Shakespeare, Richard II

Re: rational behind not checking the return value of apr_palloc and apr_pcalloc

[dave b]

 no, I don't want you to do anything for me; I'm just sharing my educated
 guess at what it takes to make progress on this topic you're apparently very
 interested in

 with a little luck you'll be able to find somebody here to analyze the code
 you pointed out to see which cases actually matter, and to what extent


Why not just fix it now and not worry? ...

--
For a light heart lives long.   -- Shakespeare, Love's Labour's Lost

rational behind not checking the return value of apr_palloc and apr_pcalloc

[dave b]

What is the rational behind not checking the return value of
apr_palloc and apr_pcalloc?


code memory/unix/apr_pools.c from apr-1.4.2
APR_DECLARE(void *) apr_pcalloc(apr_pool_t *pool, apr_size_t size);
APR_DECLARE(void *) apr_pcalloc(apr_pool_t *pool, apr_size_t size)
{
void *mem;

if ((mem = apr_palloc(pool, size)) != NULL) {
memset(mem, 0, size);
}

return mem;
}

and
apr_palloc can return NULL.
So I modified the code and the testdir test failed in one place -

node = active-next;
if (size = node_free_space(node)) {
list_remove(node);
}
else {
if ((node = allocator_alloc(pool-allocator, size)) == NULL) {
if (pool-abort_fn)
pool-abort_fn(APR_ENOMEM); /* HERE */

return NULL;
}
}

When you run the testdir (test). If you change the above to be:


.
if ((node = allocator_alloc(pool-allocator, size)) == NULL) {
if (!   pool-abort_fn) /* note the ! added */
pool-abort_fn(APR_ENOMEM);

return NULL; /* you end up here */
}
}
and you will fail one of the tests. This to me suggests that this scenario is
possible if the pool is like that one failed test *but* pool-abort_fn is not
true :)


So what is the rational behind most users of these method *not*
checking the return code - because from what I have seen / know it is
possible return NULL.

Also see:  https://issues.apache.org/bugzilla/show_bug.cgi?id=49847

Recommended way to use autotools

[Dave Ingram]

Hi all,

I've been wondering if there is a recommended way of using autotools 
when writing an Apache module? I've seen a tutorial 
http://threebit.net/tutorials/apache2_modules/tut1/tutorial1.html on 
this, but it was last updated in 2005, so I'm not sure if there are any 
better ways or things to do. I've also looked at a couple 
http://code.google.com/p/mod-authn-otp/ of projects 
http://code.google.com/p/modskeleton/, and they seem to do things 
differently. I also found a couple of previous 
http://marc.info/?l=apache-modulesm=109754513810791w=2 questions 
http://marc.info/?l=apache-modulesm=97517123707363w=2 on this 
subject, but they're very old and don't answer my question.


I essentially want to allow the module to be compiled against one or two 
different libraries, depending on availability and user choice.


Non-essential background details:
8-
I'm building a module that accesses a database at configuration parse 
time, but I can't use APR DBD for a few reasons. Firstly, I need 
features from apr-util 1.3 which is not in many major distributions. 
Secondly, apr-util and mod_php can conflict by linking against 
libmysqlclient_r and libmysqlclient, respectively. Both libraries export 
the same symbols, which then causes all sorts of hell (generally 
manifesting as bizarre segfaults). I've also had issues with the 
distro-packaged versions of apr-util being compiled without database 
support (or DSO support, making it impossible to load the DB drivers). 
Then again, this could be due to me missing something. My DB 
driver-loading code was inspired from mod_dbd, so I'm fairly sure it's 
making the right calls.


Basically, I want the user/package maintainer to be able to force 
selection of libmysqlclient/libmysqlclient_r as appropriate (as some 
distributions have fixed this issue), although it would be even better 
to autodetect this via configure rules. I then also plan to add 
support for other databases, such as Postgres.


Of course, if anyone has any better suggestions on how to do this that 
don't involve me writing my own reduced-functionality DB abstraction 
layer, I would love to hear them.

8-


Thanks,


Dave

Re: MySQL Virtual Host and Traffic Module

[Dave Ingram]

Hi Eldho,

I'm the author mod_sqltemplate, which sounds like it does what you're 
after (as Jorge kindly pointed out). It's currently under mostly-active 
development, and I'm definitely open to bug reports and suggestions.



Dave


Eldho wrote:

Hi Vaughan,

What is the status of this module. Actually I am searching for a module 
that write all the vhost  configuration to a database and read it from db

also.

thanks.

Eldho



Dave Ingram wrote:
  

Hi Vaughan,


Thanks for the response. I haven't thought of doing the SQL query the way
you suggested, however I agree that it will cause unnecessary load on
busy
servers and I would like to keep this as efficient as possible.

The second option sounds more reasonable. I have already used threading
to
make a function which ticks on a configurable interval so I suppose each
child process would dump data for each of its vhosts at this interval,
using
a query similar to what you have suggested.
  
  
I think that's probably the most sensible approach. It does mean that 
you won't have up-to-the-moment statistics, and I would guess that you'd 
have to play about with different intervals as the number of hosts grows 
in order for it to scale. You may also want to consider somehow 
staggering the updates, so they don't all happen at once. It may also be 
advisable to perform an UPDATE rather than an INSERT... ON DUPLICATE 
UPDATE once your module knows that there is a value that can be updated 
(i.e. after the query has run once in the simple case, or once this 
day/hour/etc in the complex case).




I think I might go with the second option for the time being and see how
it
goes but I am still interested to know if there is a way to store per
vhost
data across children?
  
  
I would be interested to know how things turn out, and I'd be interested 
to see the final module. I've been thinking about writing a custom 
bandwidth monitoring/limiting module myself, but if I don't need to 
reinvent the wheel...


I'm afraid I can't answer this question in a definite way, though. One 
module that should store per-vhost data like this is mod_cband 
http://sourceforge.net/projects/cband/, so that might be worth looking 
into.


As a side note, I'd be interested to know how you create/template the 
virtual hosts. I myself have written a database-backed templating module 
that could be used for virtual hosting 
(http://www.dmi.me.uk/code/apache/mod_sqltemplate/) and I'm curious to 
see other approaches.


Thanks,


Dave



Thanks,
Vaughan

-Original Message-
From: Dave Ingram [mailto:d...@dmi.me.uk] 
Sent: Thursday, 19 March 2009 12:28 AM

To: modules-dev@httpd.apache.org
Subject: Re: MySQL Virtual Host and Traffic Module

Vaughan,

  
  

What I have so far are 2 filters which gather the inbound traffic and
outbound traffic for each transaction. These work ok and when logging
transactions to file all of the in/out byte amounts appear to be
correct.
The first problem however, is that each child has its own set of memory



and
  
  

therefore keeps its own totals per virtual host. This also means that
multiple logging events occur for each transaction. I could just log
this
all to database but it would 1) be inefficient and 2) cause the size of



the
  
  

database to grow quite quickly.
  



It sounds to me like you could go two ways with this. I don't know the
format of your database table, but it should be possible to update it
atomically using something like:

INSERT INTO bandwidth (vhost_id, bw_in, bw_out) VALUES (42, 1124,
5023409) ON DUPLICATE KEY UPDATE bw_in = bw_in + 1124, bw_out = bw_out +
5023409

but that could lead to a lot of load. Another way might be for each
child to collect statistics and only flush to the database periodically,
say every 30 seconds (perhaps configurable on a per-vhost basis, so that
load-heavy sites could have larger update intervals). It would still be
possible to use the query above though.

This query could probably even be updated to split statistics on a
date/time basis, if you require more granular reporting.

Or have I missed/misunderstood something?


Dave

  
  





  

Re: APR feature detection

[Dave Ingram]

Sorin Manolache wrote:
 On Thu, Apr 2, 2009 at 19:45, Dave Ingram d...@dmi.me.uk wrote:
   
 Hi guys,

 Is there any way that my module can detect which APR features are
 enabled in Apache? It relies on DBD, and if that's not available then my
 module just segfaults, which isn't very friendly. I'd prefer to give an
 error message to the user, telling them that DBD is required.

 Does anyone have any tips, or should I ask the APR list instead?
 

 dbd does this:
 APR_REGISTER_OPTIONAL_FN(ap_dbd_prepare);
 APR_REGISTER_OPTIONAL_FN(ap_dbd_open);
 APR_REGISTER_OPTIONAL_FN(ap_dbd_close);
 APR_REGISTER_OPTIONAL_FN(ap_dbd_acquire);
 APR_REGISTER_OPTIONAL_FN(ap_dbd_cacquire);

 APR_RETRIEVE_OPTIONAL_FN(one of the functions above) should give you
 NULL if DBD is absent.

 The two macros are declared in apr_optional.h.
   

Actually, that's not quite what I was after -- those are defined by
mod_dbd, which I don't rely on (as it's initialised too late for my
module's purposes). I do all of the DBD interaction myself. Is there any
other way to be sure it's enabled? Would:

if (apr_dbd_open != NULL) {
  // error...
}

work?


Dave

Re: custom background thread and module sharing a data structure

[Dave Ingram]

Hi Sorin,

Sorin Manolache wrote:
 On Wed, Mar 11, 2009 at 02:08, Andrej van der Zee
 andrejvander...@gmail.com wrote:
   
 Hi,

 I need to modify Apache and run one custom background thread. In addition,
 my custom modules have to be able to share a data structure with this
 background thread. Did anybody do this before? Is there an example I can use
 as a starting point?
 

 Yes, I did this. I will send you a commented source file sometimes
 today or early tomorrow (Central European Time), I don't really have
 time now.
   
I'm interested in something similar -- would you please send me some
example sources as well?

Thanks,


Dave

APR feature detection

[Dave Ingram]

Hi guys,

Is there any way that my module can detect which APR features are
enabled in Apache? It relies on DBD, and if that's not available then my
module just segfaults, which isn't very friendly. I'd prefer to give an
error message to the user, telling them that DBD is required.

Does anyone have any tips, or should I ask the APR list instead?

Thanks,


Dave

Re: MySQL Virtual Host and Traffic Module

[Dave Ingram]

Hi Vaughan,

Thanks for the response. I haven't thought of doing the SQL query the way
you suggested, however I agree that it will cause unnecessary load on busy
servers and I would like to keep this as efficient as possible.

The second option sounds more reasonable. I have already used threading to
make a function which ticks on a configurable interval so I suppose each
child process would dump data for each of its vhosts at this interval, using
a query similar to what you have suggested.
  
I think that's probably the most sensible approach. It does mean that 
you won't have up-to-the-moment statistics, and I would guess that you'd 
have to play about with different intervals as the number of hosts grows 
in order for it to scale. You may also want to consider somehow 
staggering the updates, so they don't all happen at once. It may also be 
advisable to perform an UPDATE rather than an INSERT... ON DUPLICATE 
UPDATE once your module knows that there is a value that can be updated 
(i.e. after the query has run once in the simple case, or once this 
day/hour/etc in the complex case).



I think I might go with the second option for the time being and see how it
goes but I am still interested to know if there is a way to store per vhost
data across children?
  
I would be interested to know how things turn out, and I'd be interested 
to see the final module. I've been thinking about writing a custom 
bandwidth monitoring/limiting module myself, but if I don't need to 
reinvent the wheel...


I'm afraid I can't answer this question in a definite way, though. One 
module that should store per-vhost data like this is mod_cband 
http://sourceforge.net/projects/cband/, so that might be worth looking 
into.


As a side note, I'd be interested to know how you create/template the 
virtual hosts. I myself have written a database-backed templating module 
that could be used for virtual hosting 
(http://www.dmi.me.uk/code/apache/mod_sqltemplate/) and I'm curious to 
see other approaches.


Thanks,


Dave



Thanks,
Vaughan

-Original Message-
From: Dave Ingram [mailto:d...@dmi.me.uk] 
Sent: Thursday, 19 March 2009 12:28 AM

To: modules-dev@httpd.apache.org
Subject: Re: MySQL Virtual Host and Traffic Module

Vaughan,

  

What I have so far are 2 filters which gather the inbound traffic and
outbound traffic for each transaction. These work ok and when logging
transactions to file all of the in/out byte amounts appear to be correct.
The first problem however, is that each child has its own set of memory


and
  

therefore keeps its own totals per virtual host. This also means that
multiple logging events occur for each transaction. I could just log this
all to database but it would 1) be inefficient and 2) cause the size of


the
  

database to grow quite quickly.
  



It sounds to me like you could go two ways with this. I don't know the
format of your database table, but it should be possible to update it
atomically using something like:

INSERT INTO bandwidth (vhost_id, bw_in, bw_out) VALUES (42, 1124,
5023409) ON DUPLICATE KEY UPDATE bw_in = bw_in + 1124, bw_out = bw_out +
5023409

but that could lead to a lot of load. Another way might be for each
child to collect statistics and only flush to the database periodically,
say every 30 seconds (perhaps configurable on a per-vhost basis, so that
load-heavy sites could have larger update intervals). It would still be
possible to use the query above though.

This query could probably even be updated to split statistics on a
date/time basis, if you require more granular reporting.

Or have I missed/misunderstood something?


Dave

  

Re: MySQL Virtual Host and Traffic Module

[Dave Ingram]

Vaughan,

 What I have so far are 2 filters which gather the inbound traffic and
 outbound traffic for each transaction. These work ok and when logging
 transactions to file all of the in/out byte amounts appear to be correct.
 The first problem however, is that each child has its own set of memory and
 therefore keeps its own totals per virtual host. This also means that
 multiple logging events occur for each transaction. I could just log this
 all to database but it would 1) be inefficient and 2) cause the size of the
 database to grow quite quickly.
   

It sounds to me like you could go two ways with this. I don't know the
format of your database table, but it should be possible to update it
atomically using something like:

INSERT INTO bandwidth (vhost_id, bw_in, bw_out) VALUES (42, 1124,
5023409) ON DUPLICATE KEY UPDATE bw_in = bw_in + 1124, bw_out = bw_out +
5023409

but that could lead to a lot of load. Another way might be for each
child to collect statistics and only flush to the database periodically,
say every 30 seconds (perhaps configurable on a per-vhost basis, so that
load-heavy sites could have larger update intervals). It would still be
possible to use the query above though.

This query could probably even be updated to split statistics on a
date/time basis, if you require more granular reporting.

Or have I missed/misunderstood something?


Dave

Re: mod_vhost_dbd

[Dave Ingram]

Hi Jorge,

Something with the same name:

http://code.google.com/p/dbd-modules/wiki/mod_vhost_dbd

self-promotion
I don't know if that's what you're after, but I've written something
vaguely similar but more flexible:

http://www.dmi.me.uk/code/apache/mod_sqltemplate/
/self-promotion

Hope this helps!


Dave


Jorge Bastos wrote:
 Hi people,

 I just signed up, to report a typo.

  

 http://modules.apache.org/search.php?id=1753

  

 the link is broken.

 I'd love to test this, I'm looking for it for some time.

 Is the maintainer out there? J

  

 Jorge


   

Re: mod_vhost_dbd

[Dave Ingram]

Jorge Bastos wrote:
 This is possible using mod_sqltemplate. Suppose in your database, you
 have a php_register_globals column, which is either On or Off.
 Then this will work:

 [snip]

 So I think my problem is resolved!
 Resuming, I can have as many columns I want in the hosts table, correct? If
 so, I can have the config I want!!! That's fantastic, I've been looking for
 something like this for some years!
   
Yup. I place no restrictions at all. It doesn't even have to be a single
table -- it can be the result of a multi-table join, as long as all of
the columns have different names. They can be renamed using AS, like
SELECT a.id, a.name, h.name AS hname FROM table1 a, table2 h for
example, which would then be accessed as ${id}, ${name}, ${hname}. Take
a quick glance at the sample configuration
http://www.dmi.me.uk/code/apache/mod_sqltemplate/99_mod_sqltemplate.conf
from the mod_sqltemplate site.

 You can tell Apache to perform a config reload (e.g. via apache2ctl
 graceful), which will force the configuration to be re-read from the
 database.
 

 Hum but this only via commandline correct?
   
It can be from anything that can send Apache a signal (USR1 causes a
graceful restart) as well. Running apache2ctl can be done from anything
that has the right permissions to signal Apache.

As an incidental note, I'm planning on writing a PHP patch that allows
you to force Apache to do a graceful restart, but I haven't had the time
to think about the best way to do this yet.

 Another thing, apache read's this, when it loads and stays with this info on
 memory correct?
   
That's correct.

 The existing virtualhosts that I have, I can leave that intact, and start
 using the new virtualhosts with this configuration, correct?
   
You can leave your existing configuration files and create a completely
new one that handles all of your new virtualhosts.


Dave

Re: mod_vhost_dbd

[Dave Ingram]

Jorge Bastos wrote:
 Dave,
 You could have a Install.txt file to explain how to compile.
 I'm not very familiar with apxs2 :)

 Can you post here?
   

Erm. I think all you need to use is:

apxs2 -i -c mod_sqltemplate.c

as root, to compile and install in one step. Don't forget to add the
LoadModule line!


Dave

Re: mod_vhost_dbd

[Dave Ingram]

Jorge Bastos wrote:
 Sorry about apxs.
   
That's fine -- everyone has to learn somewhere.

 I'm just curious, libmysql shouldn't be linked agains the .so module
Nope - it uses the built-in APR DBD in Apache, which already links
against MySQL.


Dave

Re: WELCOME to modules-...@httpd.apache.org

[dave]

On Wed, Feb 11, 2009 at 02:16, Joe Lewis j...@joe-lewis.com wrote:



 I understand - you are using both directory AND server side configs.
  directory configs do not get created until a request.  And a merge should
 never return the same structure, meaning you should get new configs.  Even
 the directory configs don't get created until the request comes in.  In the
 handler, try setting two distinct variables, one to the server side and one
 to the directory config, and you should definitely see a difference.

 Well done!
 Joe



Well, sort of. I think I can hack it to fix the problem now, but I still
don't understand why the problem is occurring. I don't actually have any
directory configs, they're all virtuat-host-level configs (unless those
qualify as directory configs?) When I output the results of my merge, I see
the server-merge occur, followed by the child_init, but the child_init does
not receive the results of server-merge. This is simplest to show with the
end of the output:


acfg and bcfg are the arguments to the merge calback. cfg is the result that
the merge callback returns
...
merge_server{acfg: 0x2b45d35e79d0bcfg:0x2b45dc2385f0cfg:
0x2b45dc26fff0}
merge_dir{acfg: 0x2b45d35e79e8bcfg:0x2b45dc2385d8cfg:
0x2b45dc270458}
child_init{pid: 2800cfg:0x2b45d35e79d0server:
0x2b45d355f968} -- recieving the acfg from the merge_server, not the return
value of the merge_server
create new
handler{pid: 2800cfg:0x2b45dc26fff0server:
0x2b45dc2361b8}  -- recieving the return value of the merge_server
create new


Is this the way it is supposed to work?


-dave
man...@gmail.com

Re: WELCOME to modules-...@httpd.apache.org

[dave]

On Wed, Feb 11, 2009 at 13:21, Eric Covener cove...@gmail.com wrote:

 On Wed, Feb 11, 2009 at 1:18 PM, dave man...@gmail.com wrote:
  acfg and bcfg are the arguments to the merge calback. cfg is the result
 that
  the merge callback returns
  ...
  merge_server{acfg: 0x2b45d35e79d0bcfg:0x2b45dc2385f0cfg:
  0x2b45dc26fff0}
  merge_dir{acfg: 0x2b45d35e79e8bcfg:0x2b45dc2385d8cfg:
  0x2b45dc270458}
  child_init{pid: 2800cfg:0x2b45d35e79d0server:
  0x2b45d355f968} -- recieving the acfg from the merge_server, not the
 return
  value of the merge_server
  create new
  handler{pid: 2800cfg:0x2b45dc26fff0server:
  0x2b45dc2361b8}  -- recieving the return value of the merge_server
  create new

 is this from passing r-server? Are you sure your request was mapped
 to a virtualhost?

 --
 Eric Covener
 cove...@gmail.com


It's definitely setup as a virtual host.


I have this in my child_init:

static void my_child_init(apr_pool_t *p, server_rec *s) {
printf(child_init\t{pid: %d\tcfg:%p\t\tserver: %p}\n, getpid(),
ap_get_module_config(s-module_config, my_module), s);

//...
}

My handler:

static int my_handler(request_rec *r) {
//...
printf(handler\t\t{pid: %d\tcfg:%p\t\tserver: %p}\n, getpid(),
ap_get_module_config(r-server-module_config, my_module), r-server);
//...
}

And my merge_server:

static void *my_merge_server_config(apr_pool_t *p, void *base, void *add) {
my_config *cfg = (my_config *)apr_pcalloc(p, sizeof(my_config));
//...

printf(merge_server\t{acfg: %p\tbcfg:%p\tcfg: %p}\n, base, add, cfg);

return (void *)cfg;
}


Here was the full output of  my debug code if its of any use:

server_config{pid: 2800cfg:0x2b45d35e0ac0server: 0x2b45d3562568}
dir_config{pid: 2800cfg:0x2b45d35e0ad8path: (null)}
dir_config{pid: 2800cfg:0x2b45dc2365c8path: (null)}
server_config{pid: 2800cfg:0x2b45dc2365e0server: 0x2b45dc2341a8}
mapper{pid: 2800cfg:0x2b45dc2365c8cmd_parms: 0x7fffd77e4cd0}
merge_server{acfg: 0x2b45d35e0ac0bcfg:0x2b45dc2365e0cfg:
0x2b45dc237e30}
merge_dir{acfg: 0x2b45d35e0ad8bcfg:0x2b45dc2365c8cfg:
0x2b45dc238298}
server_config{pid: 2800cfg:0x2b45d35e79d0server: 0x2b45d355f968}
dir_config{pid: 2800cfg:0x2b45d35e79e8path: (null)}
dir_config{pid: 2800cfg:0x2b45dc2385d8path: (null)}
server_config{pid: 2800cfg:0x2b45dc2385f0server: 0x2b45dc2361b8}
mapper{pid: 2800cfg:0x2b45dc2385d8cmd_parms: 0x7fffd77e4cd0}
merge_server{acfg: 0x2b45d35e79d0bcfg:0x2b45dc2385f0cfg:
0x2b45dc26fff0}
merge_dir{acfg: 0x2b45d35e79e8bcfg:0x2b45dc2385d8cfg:
0x2b45dc270458}
child_init{pid: 2800cfg:0x2b45d35e79d0server:
0x2b45d355f968}
create new
handler{pid: 2800cfg:0x2b45dc26fff0server:
0x2b45dc2361b8}
create new


-dave
man...@gmail.com

Re: WELCOME to modules-...@httpd.apache.org

[dave]

On Wed, Feb 11, 2009 at 13:21, Eric Covener cove...@gmail.com wrote:

 On Wed, Feb 11, 2009 at 1:18 PM, dave man...@gmail.com wrote:
  acfg and bcfg are the arguments to the merge calback. cfg is the result
 that
  the merge callback returns
  ...
  merge_server{acfg: 0x2b45d35e79d0bcfg:0x2b45dc2385f0cfg:
  0x2b45dc26fff0}
  merge_dir{acfg: 0x2b45d35e79e8bcfg:0x2b45dc2385d8cfg:
  0x2b45dc270458}
  child_init{pid: 2800cfg:0x2b45d35e79d0server:
  0x2b45d355f968} -- recieving the acfg from the merge_server, not the
 return
  value of the merge_server
  create new
  handler{pid: 2800cfg:0x2b45dc26fff0server:
  0x2b45dc2361b8}  -- recieving the return value of the merge_server
  create new

 is this from passing r-server? Are you sure your request was mapped
 to a virtualhost?

 --
 Eric Covener
 cove...@gmail.com



Eric, I think your question sent me down the correct line of reasoning...

Even though my request is on a virtual host, apache does not fork individual
processes for each host. Thus the child_init gets called with the main
server config, but the virtual host's server config is set before child_init
is called. Knowing this, that would explain the problem. Frustrating, but
explainable. I can hack around that assuming my explanation is correct.


-dave
man...@gmail.com

Re: WELCOME to modules-dev@httpd.apache.org

[dave]

Hi All,

I'm having trouble with the server_rec-module_config variable and perhaps I
am misunderstanding something.

Using the apache2 C api, I have created a child_init callback (registered
through ap_hook_child_init) that is supposed to set up a per-process
resource when the server is first started (based off the pid.) Then, as each
request comes in through the handler (registered through ap_hook_handler) I
want the handler to look up that resource for its own use.

Currently I am attempting to store/retrieve this data using the
server_rec-module_config pointer. What I am having trouble with is that the
module_config pointed to during the child_init process seems to be different
than the module_config pointed to by the handler, so the resource
initialization isn't working.

Here's some a snippet code that illustrates my problem. Note that my_module
is a global variable. Am I doing something wrong?


static void get_resource(void *module_config) {
my_config *cfg = (my_config *)ap_get_module_config(module_config,
my_module);
printf(module_config: %d\n, cfg);

//...
}


static int my_child_init(apr_pool_t *p, server_rec *s) {
printf(init server_config: %d\n, s-module_config);
get_resource(s-module_config);
//...
}

static int my_handler(request_rec *r){
//...
printf(request server_config: %d\n, r-server-module_config);
get_resource(r-server-module_config);
//...
}


This code prints out four different address locations where I would
expect/hope for two - one for the server_rec-module_config and one for the
returned value of ap_get_module_config().





-dave
man...@gmail.com

Re: WELCOME to modules-dev@httpd.apache.org

[dave]

 So, how is the s-module_config being set?  I hope you are creating it in
 the per-server config create function.  Do you have that code for us to look
 at?

 Joe


I am instantiating it in the per-server callback, ala:

static void *my_create_server_config(apr_pool_t *p, server_rec *s) {
   my_config *cfg = (my_config *)apr_pcalloc(p, sizeof(my_config));
   cfg-prop1 = NULL;
   cfg-prop2 = NULL;
   //...
   return (void *)cfg;
}

For reference, one of those properties becomes the start of a linked list
that the get_resource function uses to do it's lookups.

Re: WELCOME to modules-dev@httpd.apache.org

[dave]

On Tue, Feb 10, 2009 at 19:11, Joe Lewis j...@joe-lewis.com wrote:

 With your last post (the creation), it looks like you understand it well.
  Are you only creating one process?  For example, run it with a -X parameter
 (to prevent fork()ing) so you can ensure that you aren't working across
 processes?  Printing the PID along with the %d?  Is the %d in the handler
 always the same?

 Joe


So I was in the process of creating a simple test module when I discovered
the problem had gone away (both in the test module and in the module I am
developing.) I did some more digging and discovered that it was a
configuration that is actually causing the problem. In my virtual host
definition, I have this:

#   SetHandler test
SetHandler mymod
MyModConfig /var/www/file

If I comment out that last line, I get a relatively simple configuration
sequence when Apache first boots (yes, this is all run with -X). cfg is the
address of the module_config variable. server/path is the address of either
the server_rec or the path variable pass into the callback.

sudo APACHE_RUN_USER=www-data APACHE_RUN_GROUP=www-data /usr/sbin/apache2 -X
server_config{pid: 13878cfg:0x2b7c26e32ac0server:
0x2b7c26db4568}
dir_config{pid: 13878cfg:0x2b7c26e32ad8path: (null)}
server_config{pid: 13878cfg:0x2b7c26e399d0server:
0x2b7c26db1968}
dir_config{pid: 13878cfg:0x2b7c26e399e8path: (null)}
child_init{pid: 13878cfg:0x2b7c26e399d0server:
0x2b7c26db1968}
create new -- this is my code creating the resources that it needs for the
process
handler{pid: 13878cfg:0x2b7c26e399d0server:
0x2b7c2fa881b8} -- notice the cfg/server addresses are the same as the line
above


When I run with the line uncommented, I get the following:

server_config{pid: 13979cfg:0x2b2cf82ebac0server:
0x2b2cf826d568}
dir_config{pid: 13979cfg:0x2b2cf82ebad8path: (null)}
dir_config{pid: 13979cfg:0x2b2d00f415c8path: (null)}
server_config{pid: 13979cfg:0x2b2d00f415e0server:
0x2b2d00f3f1a8}
mymodconfig{pid: 13979cfg:0x2b2d00f415c8cmd_parms:
0x7fffb2ad90c0} -- call to process the config
server_config{pid: 13979cfg:0x2b2cf82f29d0server:
0x2b2cf826a968}
dir_config{pid: 13979cfg:0x2b2cf82f29e8path: (null)}
dir_config{pid: 13979cfg:0x2b2d00f435d8path: (null)}
server_config{pid: 13979cfg:0x2b2d00f435f0server:
0x2b2d00f411b8}
mymodconnfig{pid: 13979cfg:0x2b2d00f435d8cmd_parms:
0x7fffb2ad90c0} -- call to process the config again
child_init{pid: 13979cfg:0x2b2cf82f29d0server:
0x2b2cf826a968}
create new -- creating the resources where it should
handler{pid: 13979cfg:0x2b2d00f7aff0server:
0x2b2d00f411b8} -- notice the cfg/server addresses are different than the
line above
create new -- thinks the resources have not been created, so doing it again

I tried defining a merge callback for both the server level and the
directory level, but that doesn't seem to change anything.

-dave
man...@gmail.com

Re: Making mod_auth_digest mysql

[Dave Ingram]

The -f and -d flags for RewriteCond are for checking the file system,
not environment variables, although they can use environment variables
if necessary. For example:

RewriteCond %{DOCUMENT_ROOT}/%{ENV:foo} -d

would check that the folder named by the environment variable foo
exists in the document root.


Dave


Michele Waldman wrote:
 RewriteCond has flags -f -d ...
 But not -e for exists.
 It looks like:
 RewriteCond ${REMOTE_USER} != always evaluates to true if REMOTE_USER does
 not exist.  Am I wrong?
 I'm thinking about adding a -e flag for environment variable does not exist
 to httpd on my server.  It would return true if the variable exists or
 false, otherwise.
 Is there a way to already do this?
 Thoughts?

 Michele


   

Announcement: mod_sqltemplate

[Dave Ingram]

Hi all,

A while ago, I discussed[1] the idea of a flexible SQL-based
configuration templating module for Apache 2.2. I've since written the
code, and (after letting it fall by the wayside for a couple of months),
I'm ready for people to test it, poke it and break it (and then send me
a bug report).

If you would like to help, then please visit my site[2] for information
on the module, its capabilities, and usage information. The source is
currently held in Subversion[3], so if there are any volunteers to just
check the code over and offer some advice/pointers/criticisms then that
would be very welcome. This is my first foray into APR, DBD, and Apache
module coding -- I want to make it as robust and well-coded as possible.

Thanks,


Dave

[1]
http://mail-archives.apache.org/mod_mbox/httpd-modules-dev/200808.mbox/%3c48a99724.50...@dmi.me.uk%3e
[2] http://www.dmi.me.uk/code/apache/mod_sqltemplate/
[3] http://svn.dmi.me.uk/mod_sqltemplate/trunk

Re: How can i decrypt a cookie in a module

[Dave Ingram]

Christian Klinger wrote:
 thanks for the answers, but i don´t get it.

 I try to add the Icrypto in my Makefile i get the error again:

 [snip]
That's a font issue - it's actually a lowercase L (for library) rather
than an uppercase i (for include directory).

 What file should i include to the LoadFile? Blowfish.h mod_auth_tkt.so?
I believe the LoadFile should be the path to the OpenSSL crypto library,
which would be something like /usr/lib/libcrypto.so (but you may need to
select a specific version). See the docs at
http://httpd.apache.org/docs/2.2/mod/mod_so.html

 Sorry i´m a newbie in this stuff.
Everyone starts somewhere :-)


Dave

Re: Logging authentication requests

[Dave Ingram]

To clarify:
 Is it currently possible to log authentication requests (ideally both
 success and failure, individually)? If not, is it possible?

Meant: is it currently possible to do this with Apache 2.2 (perhaps
using an existing module)? If not, is it even theoretically possible?


Dave

Re: number of bytes/packets sent/received

[Dave Ingram]

 I forgot to mention, but I need the number of packets / bytes per HTTP
 request and log it to a database with other information. So I guess
 that would take case of all the hairy stuff. I think I will give it a
 go as you recommend with an input and output filter that only counts
 the bytes and just passes on the data, unless somebody comes with a
 better idea (what about the content-length for example, can I trust
 this?). I understood from your email that the headers are counted with
 it (I never wrote a filter before) and that's exactly what I need.
   
Would mod_logio be at all helpful?

http://httpd.apache.org/docs/2.2/mod/mod_logio.html

I haven't looked into it, but the source might give you some ideas. You
may even have a direct solution if you use a module that logs directly
to a database. I must admit that I don't know offhand if such a module
exists.


Dave

Re: number of bytes/packets sent/received

[Dave Ingram]

 Would mod_logio be at all helpful?
 
 You really made it easy for me huh? Thanks alot.
   
No use in doing more work than you need to ;-)

To be honest, I only found out about it today while researching some
obscure logging options.

A quick search reveals that there are some modules out there that do log
to databases, but I don't know how well-maintained they are (mod_log_sql
only mentions Apache 2.0, for example) or how much work they would be to
use (an O'Reilly page
http://www.onlamp.com/pub/a/apache/2005/02/10/database_logs.html?page=2
suggests that Apache's default mod_log_config needs to be rebuilt).


Dave

Re: number of bytes/packets sent/received

[Dave Ingram]

 A quick search reveals that there are some modules out there that do log
 to databases, but I don't know how well-maintained they are (mod_log_sql
 only mentions Apache 2.0, for example) or how much work they would be to
 use (an O'Reilly page
 http://www.onlamp.com/pub/a/apache/2005/02/10/database_logs.html?page=2
 suggests that Apache's default mod_log_config needs to be rebuilt).

 

 Since Apache 2.1 there is mod_dbd that makes it pretty easy to log to
 a MySQL, sqlite2/3 and Postgres database. It also takes care of
 connection pooling.
   
Ah. I'm planning to use mod_dbd myself for a project, but I wasn't aware
it provided any logging ability.


Dave

Logging authentication requests

[Dave Ingram]

Hi all,

Just thinking about logging...

Is it currently possible to log authentication requests (ideally both
success and failure, individually)? If not, is it possible? And where
should the module fit into the authentication chain? Is this related to
Rick Houser's earlier posts about wrapping an existing hook?


Dave

Re: APR DBD: Column names from query

[Dave Ingram]

Nick Kew wrote:
 Dave Ingram wrote:

 First off, if this isn't the correct place to ask this then I apologise,
 but it seemed the most appropriate list. If there's somewhere more
 appropriate, please let me know.

 I've had a quick look for some information on accessing a database via
 APR, and after glancing through the header files, I still have a
 question: is it possible to get the column names from the query results?

 Yes, with sufficiently up-to-date versions of apr_dbd and mod_dbd.
 Hints about how it works can be found in the relevant CHANGES files.

 Sorry to be terse, but it would take too long to dig up URLs
 from here.
That's fine - all I really wanted was to know if it's possible, and a
nudge in the right direction -- thanks!

 [snip - moved lower and improved]

 Interesting idea.  Are you familiar with mod_vhost_dbd, and even
 mod_macro, which do somewhat-related things?
Yes indeed, although I only used the VirtualHost example as it's the
most often-used. Unless I'm much mistaken, mod_vhost_dbd only allows you
to set the document root, and all other mass virtualhosting solutions
that I've seen are quite limited -- nowhere near this flexible.

Say you wanted to add some redirects to your virtual hosts too, after
the directories have been rearranged, for example... as far as I know,
no module allows you to do anything like this:

SQLRepeat SELECT baz, bar, qux FROM sometable WHERE baz='foo'
VirtualHost *:80
  DocumentRoot /www/root/path/$baz
  ServerName $bar
  ServerAdmin $qux
  SQLRepeat SELECT was, now FROM redirects WHERE type='permanent' AND
servername = ? $bar
RedirectPermanent $was $now
  /SQLRepeat
/VirtualHost
/SQLRepeat

If you're interested, I've posted a bit more about it on my blog while
I'm still fleshing out the idea:
http://www.dmi.me.uk/blog/2008/08/13/modules-for-apache-and-php/

I've been looking at mod_macro for inspiration and to see how it does
the substitution/re-parsing, and I think I may end up borrowing a few
chunks of its code to avoid re-inventing the wheel...


Dave

Re: Configuration file templating

[Dave Ingram]

Hi Albert,

 Its not a module, but I've been working on a generic configuration
 file templating system to be powered by a variety of data sources,
 including SQL. You might enjoy checking it out:

 http://www.hypermagnet.com/
   
Thanks for the information and link - I'll definitely have a look and
see if it serves my needs. At the moment, I use a set of scripts to
auto-generate the Apache configuration files (and for other things, like
BIND) and then to poke the daemons with SIGHUP (or equivalent) to
reload. I know BIND can be patched to read directly from a database --
I'd just prefer to skip the configuration file generation step for
Apache too if I can, while still maintaining flexibility :-)

If anyone's interested, I've put some preliminary information in a blog
post:
http://www.daveingram.me.uk/blog/2008/08/13/modules-for-apache-and-php/

Comments and criticisms welcome!

Thanks,


Dave

Re: Should we release 2.10?

[Dave Viner]

what is required to run the perl test?

thanks
dave viner

On Jul 11, 2008, at 12:54 AM, Bojan Smojver wrote:


On Thu, 2008-07-10 at 18:09 -0700, Joe Schaefer wrote:

 Are you willing to give it a shot?


I can give it a try, but I have to warn that my Perl skills are
non-existent. How much of that is involved in the release?

--
Bojan

Re: APR::Request::Apache2 param() method

[Dave Viner]

removing the enctype makes no difference.  still encounter the same  
error.


i also added discard_request_body...  the code now reads:

my $req = APR::Request::Apache2-handle($r);
$r-discard_request_body;
my($arg_status,$body_status) = $req-param_status();
$r-log-debug(Parsing returned $arg_status, $body_status);  
#this is line 43


if($r-method_number() == Apache2::Const::M_POST)
{
my $ps = $req-param();
foreach my $k (keys(%$ps))
{
$r-log-warn(Got $k =  . $ps-{$k});
}
## verify that we got an  alert id
my $alert_id = $req-param('alerts_id');
my $alert_email = $req-param('alerts_email');
$r-log-debug( Checking params);
if(!$alert_id)
{
$r-log-warn(no alert id given);



here's the output in the error log:

[Wed Jan 31 13:17:32 2007] [debug] /Users/dviner/Documents/amb/dev/ 
aboutmybaby/AMB/Handlers/MailingList/Remove.pm(17): [client  
127.0.0.1] using mod_perl handler for removing mailing list, referer:  
http://lalala.aboutmybaby.localhost/mailing_list/remove? 
[EMAIL PROTECTED]
Use of uninitialized value in concatenation (.) or string at /Users/ 
dviner/Documents/amb/dev/aboutmybaby/AMB/Handlers/MailingList/ 
Remove.pm line 43.
[Wed Jan 31 13:17:32 2007] [debug] /Users/dviner/Documents/amb/dev/ 
aboutmybaby/AMB/Handlers/MailingList/Remove.pm(43): [client  
127.0.0.1] Parsing returned Missing input data, , referer: http:// 
lalala.aboutmybaby.localhost/mailing_list/remove? 
[EMAIL PROTECTED]
[Wed Jan 31 13:17:32 2007] [debug] /Users/dviner/Documents/amb/dev/ 
aboutmybaby/AMB/Handlers/MailingList/Remove.pm(55): [client  
127.0.0.1]  Checking params, referer: http:// 
lalala.aboutmybaby.localhost/mailing_list/remove? 
[EMAIL PROTECTED]
[Wed Jan 31 13:17:32 2007] [warn] [client 127.0.0.1] no alert id  
given, referer: http://lalala.aboutmybaby.localhost/mailing_list/ 
[EMAIL PROTECTED]



I also tried to verify my sanity by using CGI.pm to print out what  
the parameters are that are received...

here's that output and code:

[Wed Jan 31 13:23:30 2007] [debug] /Users/dviner/Documents/amb/dev/ 
aboutmybaby/AMB/Handlers/MailingList/Remove.pm(18): [client  
127.0.0.1] using mod_perl handler for removing mailing list, referer:  
http://lalala.aboutmybaby.localhost/mailing_list/remove? 
[EMAIL PROTECTED]
Use of uninitialized value in concatenation (.) or string at /Users/ 
dviner/Documents/amb/dev/aboutmybaby/AMB/Handlers/MailingList/ 
Remove.pm line 44.
[Wed Jan 31 13:23:30 2007] [debug] /Users/dviner/Documents/amb/dev/ 
aboutmybaby/AMB/Handlers/MailingList/Remove.pm(44): [client  
127.0.0.1] Parsing returned Missing input data, , referer: http:// 
lalala.aboutmybaby.localhost/mailing_list/remove? 
[EMAIL PROTECTED]
[Wed Jan 31 13:23:30 2007] [warn] [client 127.0.0.1] Got [CGI]  
alerts_id = 44524, referer: http://lalala.aboutmybaby.localhost/ 
mailing_list/[EMAIL PROTECTED]
[Wed Jan 31 13:23:30 2007] [warn] [client 127.0.0.1] Got [CGI]  
alerts_email = [EMAIL PROTECTED], referer: http:// 
lalala.aboutmybaby.localhost/mailing_list/remove? 
[EMAIL PROTECTED]
[Wed Jan 31 13:23:30 2007] [warn] [client 127.0.0.1] Got [CGI] submit  
= Remove, referer: http://lalala.aboutmybaby.localhost/mailing_list/ 
[EMAIL PROTECTED]
[Wed Jan 31 13:23:30 2007] [debug] /Users/dviner/Documents/amb/dev/ 
aboutmybaby/AMB/Handlers/MailingList/Remove.pm(62): [client  
127.0.0.1]  Checking params, referer: http:// 
lalala.aboutmybaby.localhost/mailing_list/remove? 
[EMAIL PROTECTED]
[Wed Jan 31 13:23:30 2007] [warn] [client 127.0.0.1] no alert id  
given, referer: http://lalala.aboutmybaby.localhost/mailing_list/ 
[EMAIL PROTECTED]


CODE:

my $req = APR::Request::Apache2-handle($r);
$r-discard_request_body;
my($arg_status,$body_status) = $req-param_status();
$r-log-debug(Parsing returned $arg_status, $body_status);

if($r-method_number() == Apache2::Const::M_POST)
{
my $ps = $req-param();
foreach my $k (keys(%$ps))
{
$r-log-warn(Got [APR::Request::Apache2] $k =  . $ps- 
{$k});

}
my $cgi = CGI-new($r);
my $pscgi = $cgi-Vars();
foreach my $k (keys(%$pscgi))
{
$r-log-warn(Got [CGI] $k =  . $pscgi-{$k});
}
## verify that we got an  alert id
my $alert_id = $req-param('alerts_id');
my $alert_email = $req-param('alerts_email');
$r-log-debug( Checking params);
if(!$alert_id)
{
$r-log-warn(no alert id given);


How is it that CGI.pm finds the parameter and APR::Request::Apache2  
does not??


thanks
dave




On Jan 30, 2007, at 12:49 PM, Joe Schaefer wrote:


Dave Viner [EMAIL PROTECTED] writes:


Hi,

I am having a strange problem with APR::Request::Apache2 and the  
param
() method.  When I send a POST from Firefox, my code works  
perfectly.  When I

send a POST from Safari or Windows/IE

APR::Request::Apache2 param() method

[Dave Viner]

 mod_apreq2-20051231/2.6.0 mod_perl/2.0.2 Perl/v5.8.8 configured  
-- resuming normal operations


thanks
dave

Re: Problems with apreq2 on OS X

[Dave Viner]

this might be a dumb question, but have you checked that the apreq  
module is loaded?


LoadModule apreq_modulemodules/mod_apreq2.so

?

dave

On Oct 29, 2006, at 12:23 PM, Patrick Galbraith wrote:


Fred Moyer wrote:


Patrick Galbraith wrote:


Fred,

Ok: I have this failure on
1. OS X
2. Suse 10.0 amd 64
3. Suse 9.3   intel 32

Has anyone addressed this? This is what I would call severely  
broke. I would prefer not to use CGI. After this week, I think  
maybe the universe is telling me to learn PHP after all these  
years of being a perl developer.



The short answer would be to use an earlier release of libapreq -  
this only happened to me with the latest release.  Or use CGI for  
now until it's fixed.  There's nothing wrong with using CGI while  
this bug gets worked out, the api is the same for the most part.   
libapreq is still in a development version, and chances are that  
your application will not bottleneck on CGI.


If you go to PHP, you should not expect a trouble free life :) I  
don't have anything against PHP, but it has it's own set of  
problems.  With development in any language, you need to make sure  
that you keep a tight hold on your versions.  Using the latest  
version of something isn't always the best move, as experience has  
taught me.  It's nice to try it out and report bugs back, and  
helpful to the development of the project, but use the version  
that works for you.


I've dug around a bit in the code trying to resolve this issue,  
but it's a bit above my head right now.


Fred,

I know, I'm just frustrated and venting after losing many hours of  
dev time. The reason CGI won't work in my case is I've written all  
this code as a handler and putting into CGI seems like it'd be a  
lot of work.  Maybe not, I'm not sure what I would have to change  
since I rely so much on the request object.


When you say to use an earlier version of libapreq, do you mean  
version 1.0? That won't work because all the linux dists I deal  
with are ones with pre-packaged mod_perl2 and apache2 (but haven't  
been able to get apreq to compile correctly against those pre- 
package versions, trying everything from source).


Thanks for your replies!

Patrick






Patrick

Fred Moyer wrote:


Patrick Galbraith wrote:

[Sun Oct 29 12:38:27 2006] [notice] Apache/2.2.3 (Unix) mod_ssl/ 
2.2.3 OpenSSL/0.9.8d DAV/2 mod_perl/2.0.2 Perl/v5.8.8  
configured -- resuming normal operations
dyld: lazy symbol binding failed: Symbol not found:  
_apreq_handle_apache2
 Referenced from: /opt/local/lib/perl5/vendor_perl/5.8.8/ 
darwin-2level/auto/APR/Request/Apache2/Apache2.bundle

 Expected in: dynamic lookup

dyld: Symbol not found: _apreq_handle_apache2
 Referenced from: /opt/local/lib/perl5/vendor_perl/5.8.8/ 
darwin-2level/auto/APR/Request/Apache2/Apache2.bundle

 Expected in: dynamic lookup

[Sun Oct 29 12:38:38 2006] [notice] child pid 11206 exit signal  
Trace/BPT trap (5)


OS X version: Darwin radha.local 8.8.1 Darwin Kernel Version  
8.8.1: Mon Sep 25 19:42:00 PDT 2006; root:xnu-792.13.8.obj~1/ 
RELEASE_I386 i386 i386


Not sure what this is. Anyone encountered this before?




I ran into this also, same platform.  I have been digging around  
a bit to see if I can resolve it but no luck so far - my foo in  
this area isn't quite where it needs to be.  This works fine for  
me on Linux though.


Also, is there a way to have access to things like $rec-param  
without having to use Apache2::Request/libapreq2? I ask this in  
case there is no solution for getting this to work, as well as  
on linux distributions I cannot get libapreq2 working.




You can use CGI.  Are you hitting this same issue on Linux?

use libapreq2 to read POST params

[Dave Viner]

Hi,
I'm trying to use libapreq2 to read some parameters submitted via  
POST, but I'm having trouble.  Should I use $req-body($myparam) ? or  
$req-param($myparam)?



My code looks like this:

$r-log-debug(Testing param);
my ($md5, $entered);
eval { $md5 = $req-param('cv'); } ;
if($@)
{
use Data::Dumper;
$r-log-error(ACK: no cv param:  . [EMAIL PROTECTED]strerror() .   
--  . Dumper($@));

return $class-handle_get($r, $Site, $req,
'The code entered was not  
valid.brPlease try again.');

}
$r-log-debug(For cv got... $md5);


The error log shows the following:

[Fri Sep 22 10:59:56 2006] [debug] /Users/dviner/Documents/amb/dev/ 
aboutmybaby/AMB/Handlers/GuestbookSign.pm(57): [client 127.0.0.1]  
Testing param
[Fri Sep 22 10:59:56 2006] [debug] filter.c(269): [client 127.0.0.1]  
prefetching 65536 bytes
Use of uninitialized value in concatenation (.) or string at /Users/ 
dviner/Documents/amb/dev/aboutmybaby/AMB/Handlers/GuestbookSign.pm  
line 67.
[Fri Sep 22 10:59:56 2006] [debug] /Users/dviner/Documents/amb/dev/ 
aboutmybaby/AMB/Handlers/GuestbookSign.pm(67): [client 127.0.0.1] For  
cv got...



The request I sent is as follows:

$ cat sign.params | lwp-request -m POST -SUsue http:// 
lalala.aboutmybaby.localhost/guestbook/sign

$ cat sign.params
gbentry_from=hitheregbentry_body=foobarcv=111cg=abcdef

$

Here are the details of my installation:

[Fri Sep 22 10:59:55 2006] [notice] Apache/2.0.58 (Unix) PHP/5.1.4  
DAV/2 mod_apreq2-20051231/2.6.0 mod_perl/2.0.2 Perl/v5.8.8 configured  
-- resuming normal operations



Thanks for your help.
Dave Viner

possible bug in Apache 2.0 (sending chunked body with HTTP/1.0)

[Dave O'Hair]

This is low priority.  The only reason I'm hitting it is because I'm
working with a custom Apache module that does URL-based redirection to
an Application Server, and I've managed to work around it pretty easily.
 I doubt anyone using Apache out of the box would ever see it.

When SSL is being used the default ssl.conf file includes these
settings:

SetEnvIf User-Agent .*MSIE.* \
 nokeepalive ssl-unclean-shutdown \
 downgrade-1.0 force-response-1.0

The force-response-1.0 environment variable is handled in
basic_http_header_check(), in http_protocol.c:

if (r-proto_num == HTTP_VERSION(1,0)
 apr_table_get(r-subprocess_env, force-response-1.0)) {
*protocol = HTTP/1.0;
r-connection-keepalive = AP_CONN_CLOSE;
r-chunked = 0; /* NEW CODE */
}

So in addition to changing the protocol header it disables the HTTP
keepalive which is not part of the HTTP 1.0 specification.  At this
point I think it should also disable chunking of the request back to the
client (as in the above example), since chunking was also not part of
the spec, and in fact if you send a chunked response back with a HTTP
1.0 protocol header it causes some browsers (IE) to display the response
body incorrectly.

Re: stable 2.0 trees

[Dave]

Just an opinion..
Lots of people trust you lot. Next time there is a security issue and
you do release 2.0.x, if there is a change/new functionality that is
beta, alpha or worse then that is extremely bad for a GA product. Start
the 2.1.x branch! Agree rough timescales bearing in mind you may need to
do a 2.0.x release at any time. Say 2.x every 9-12 months. 2.0.x as
needed but within a week or two. If something is going to take longer
than a couple of weeks to get GA, then put it in 2.1.x. If that feature
is needed in 2.0.x, then back port it once it is GA. Look at how they do
the Linux kernel.

Sorry if this pisses people off, but it's just common sense.
Dave.

Re: mod_proxy and Content-Length

[Dave Seidel]

lurk state=off/

Pardon me for butting in here, but as someone who is building a product
based in part upon Apache/mod_proxy, I *strongly* agree with Graham.  I've
had to hack the mod_proxy code more than once to deal with this issue, and
I'd rather not have to.  I agree that it should be entirely up to any
modifying filter to be responsible for Content-Length changes.  The proxy
itself should, IMHO, be always be considered a non-modifying passthough
(with the obvious exception of proxy-specific HTTP headers, which
Content-Length is not).

- Dave

lurk state==on/

- Original Message -
From: Graham Leggett [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED]
Sent: Friday, October 11, 2002 5:26 AM
Subject: Re: mod_proxy and Content-Length


 [EMAIL PROTECTED] wrote:

  Problems with 2.0.42 and mod_proxy with Content-Length:
 
- 2.0.39 stripped C-L from all HTTP/1.0 responses.
- 2.0.40 retained C-L on HTTP/1.0 responses for GETs, but stripped it
  for HEAD.
- 2.0.42 strips C-L from all HTTP/1.0 responses.
 
  Do people think that the 2.0.42 behaviour (stripping C-L from all
  HTTP/1.0 responses) is correct?  The messages referenced below would
  suggest not, but .42 has reverted to .39's behaviour.

 I don't think proxy should touch content-length at all.

 If a filter fiddles with content length, then it should be responsible
 for removing the content-length header as needed so that it can be
 readded later.

 Proxy doesn't change content-length in itself - so it really has no
 business touching it.

 Regards,
 Graham
 --
 -
 [EMAIL PROTECTED] There's a moon
 over Bourbon Street
 tonight...

alloca() issue on tru64

[Dave Hill]

Hi all,
While digging into that mysterious hang of Apache 2 on Tru64,
I found that it seemed to be hanging under apr_poll in alloca() 
While I found that the problem seems to go away when not using a
prerelease version of the OS, it did bring something to my attention.

According to Those Who Know, in a threaded environment on Tru64, we
should be including alloca.h which causes the alloca() function to
become a builtin (which They say is the one true way of avoiding
threaded issues). This affects one (yes 1) file in the pool,
srclib/apr/poll/unix/poll.c. Attached is a suggested change to include
alloca.h if HAVE_ALLOCA is set. If my assumption is wrong that alloca
is a universal include file (for machines that have alloca), then this
could be restricted to Tru64, or maybe another conditional in 
configure :-p

regards,
Dave Hill

--- srclib/apr/poll/unix/poll.c.orig2002-09-09 11:01:27.0 -0400
+++ srclib/apr/poll/unix/poll.c 2002-09-09 11:02:25.0 -0400
 -64,6 +64,10 
 #if HAVE_SYS_POLL_H
 #include sys/poll.h
 #endif
+#if HAVE_ALLOCA
+#include alloca.h
+#endif
+
 
 #ifdef NETWARE
 #define HAS_SOCKETS(dt) (dt == APR_POLL_SOCKET) ? 1 : 0

apachectl and options take 2

[Dave Hill]

Hi again,
I am back again with a proposed change to apachectl
With the input from Aaron, I reworked my proposed fix/enhancement.
In a nutshell, for a wide variety of reasons I want apachectl to
pass -DWARP and other define toggles through to the server invocation.
Customers like apachectl, and I like giving them example code wrapped in
Defined blocks.

The proposed change does 2 things which follows the all things to 
all people approach that we seem to like.

First, ./apachectl start -DWARP -DSSL would now work with this change.
Currently adding the defines to the command line results in a error
message.

Second, despite Aarons reservations. (no one else responded), this
change also supports the setting of HTTPD_OPTIONS in the environment,
which would then be added to the httpd invocation. This feature is 
important to me because I have this bad case of CRS (can't remember stuff)
an occasionally forget what toggles I have enabled. Adding this to the
script means that I can add the environment var into bin/envars or somewhere
else (like my init.d script) and have things work.

So. any reactions to this proposal out there ?
Anyone want to sponsor this into the tree for me ?

thanks,
Dave Hill

*** apachectl.in.orig   Thu Aug 29 11:20:36 2002
--- apachectl.inThu Aug 29 14:19:29 2002
***
*** 22,28 
  # When multiple arguments are given, only the error from the _last_
  # one is reported.  Run apachectl help for usage info
  #
- ARGV=$
  #
  #  START CONFIGURATION SECTION  
  #   
--- 22,27 
***
*** 58,78 
  fi
  
  ERROR=0
! if [ x$ARGV = x ] ; then 
! ARGV=-h
  fi
  
! case $ARGV in
  start|stop|restart|graceful)
! $HTTPD -k $ARGV
  ERROR=$?
  ;;
  startssl|sslstart|start-SSL)
! $HTTPD -k start -DSSL
  ERROR=$?
  ;;
  configtest)
! $HTTPD -t
  ERROR=$?
  ;;
  status)
--- 57,82 
  fi
  
  ERROR=0
! if [ x$1 = x ] ; then 
! FIRST_ARG=-h
! ARGV=
! else
! FIRST_ARG=$1
! shift
! ARGV=$
  fi
  
! case $FIRST_ARG in
  start|stop|restart|graceful)
! $HTTPD -k $FIRST_ARG $ARGV $HTTPD_OPTIONS
  ERROR=$?
  ;;
  startssl|sslstart|start-SSL)
! $HTTPD -k start -DSSL $ARGV $HTTPD_OPTIONS
  ERROR=$?
  ;;
  configtest)
! $HTTPD -t $ARGV $HTTPD_OPTIONS
  ERROR=$?
  ;;
  status)
***
*** 82,88 
  $LYNX $STATUSURL
  ;;
  *)
! $HTTPD $ARGV
  ERROR=$?
  esac
  
--- 86,92 
  $LYNX $STATUSURL
  ;;
  *)
! $HTTPD $FIRST_ARG $ARGV
  ERROR=$?
  esac
  

Re: apachectl and options take 2

[Dave Hill]

Hi again,

Attached is the diff -u version of the proposes apachectl changes.
If I am outvoted on the env veriable (and so far it is 2 against me :-)
just drop the $HTTPD_OPTIONS from the end of the line in the few places 
it occurs. 

thanks,
Dave Hill

--- apachectl.in.orig   2002-08-29 11:20:36.0 -0400
+++ apachectl.in2002-08-29 14:19:29.0 -0400
 -22,7 +22,6 
 # When multiple arguments are given, only the error from the _last_
 # one is reported.  Run apachectl help for usage info
 #
-ARGV=$
 #
 #  START CONFIGURATION SECTION  
 #   
 -58,21 +57,26 
 fi
 
 ERROR=0
-if [ x$ARGV = x ] ; then 
-ARGV=-h
+if [ x$1 = x ] ; then 
+FIRST_ARG=-h
+ARGV=
+else
+FIRST_ARG=$1
+shift
+ARGV=$
 fi
 
-case $ARGV in
+case $FIRST_ARG in
 start|stop|restart|graceful)
-$HTTPD -k $ARGV
+$HTTPD -k $FIRST_ARG $ARGV $HTTPD_OPTIONS
 ERROR=$?
 ;;
 startssl|sslstart|start-SSL)
-$HTTPD -k start -DSSL
+$HTTPD -k start -DSSL $ARGV $HTTPD_OPTIONS
 ERROR=$?
 ;;
 configtest)
-$HTTPD -t
+$HTTPD -t $ARGV $HTTPD_OPTIONS
 ERROR=$?
 ;;
 status)
 -82,7 +86,7 
 $LYNX $STATUSURL
 ;;
 *)
-$HTTPD $ARGV
+$HTTPD $FIRST_ARG $ARGV
 ERROR=$?
 esac
 

adding options to apachectl

[Dave Hill]

Hi all,
I would like to propose a change to apachectl (for 2.0 although
work with 1.3 as well)

I like using IfDefined toggles in the default httpd.conf files
I provide to customers (for things like Tomcat connectors, PHP...),
The problem with the toggles is getting them on the command line.
With a very minor generic change to apachectl the options can be
put into bin/envvars (which is already read by apachectl) and
added to the invocation of httpd. The nice thing about this approach
is that you don't have to hack apachectl with specific options, and 
envvars is much much easier to parse  modify.
If HTTPD_OPTIONS is not set, it defaults to blank which is just fine.

Proposed patch follows:

regards,
Dave Hill

*** apachectl.orig  Tue Aug 27 13:58:52 2002
--- apachectl   Tue Aug 27 14:02:30 2002
***
*** 54,68 
  
  case $ARGV in
  start|stop|restart|graceful)
! $HTTPD -k $ARGV
  ERROR=$?
  ;;
  startssl|sslstart|start-SSL)
! $HTTPD -k start -DSSL
  ERROR=$?
  ;;
  configtest)
! $HTTPD -t
  ERROR=$?
  ;;
  status)
--- 54,68 
  
  case $ARGV in
  start|stop|restart|graceful)
! $HTTPD -k $ARGV $HTTPD_OPTIONS
  ERROR=$?
  ;;
  startssl|sslstart|start-SSL)
! $HTTPD -k start -DSSL $HTTPD_OPTIONS
  ERROR=$?
  ;;
  configtest)
! $HTTPD -t $HTTPD_OPTIONS
  ERROR=$?
  ;;
  status)
***
*** 72,78 
  $LYNX $STATUSURL
  ;;
  *)
! $HTTPD $ARGV
  ERROR=$?
  esac
  
--- 72,78 
  $LYNX $STATUSURL
  ;;
  *)
! $HTTPD $ARGV $HTTPD_OPTIONS
  ERROR=$?
  esac
  

Re: New full log format definition

[Dave Jones]

In message [EMAIL PROTECTED],
  William A. Rowe, Jr. [EMAIL PROTECTED] writes:
When it comes to a Complete reference, stop and look at the number
of HTTP headers that might be interesting.  Look at the breadth of SSL
tokens available from mod_ssl.  It is reasonably incomprehensible to try
and inventory EVERY relevant field that might spew forth from servers
today.  Not to mention the incidental log bloat.

Sounds like the log file should be XML :-)

--
David L. Jones   |  Phone:(614) 292-6929
Ohio State University|  Internet:
140 W. 19th St. Rm. 231a |   [EMAIL PROTECTED]
Columbus, OH 43210   |   [EMAIL PROTECTED]

Disclaimer: I'm looking for marbles all day long.

Re: Patch for PR# 4634

[Dave Dribin]

On Sun, Feb 10, 2002 at 11:57:34PM -0600, Dave Dribin wrote:
 Hi,
 
 I've attached a patch against 1.3.23 that modifies ./configure to take
 a --force-suffix option as a solution to PR# 4634.
 
   http://bugs.apache.org/index.cgi/full/4634

It's been over a month, and I never got any real response to this
patch.  I figured a patch against an open bug would have gotten *some*
response, even if it was that patch sucks, we don't want it. :) Did
I not follow some protocol for submitting a patch or something?

Thanks,

-Dave

Re: Copyright year bumping

[Dave Jones]

On Sat, Mar 09, 2002 at 12:20:23PM +0800, Stas Bekman wrote:
 Sander Striker wrote:
 Hi,
 
 Should we bump the copyright year on all the files?
 Anyone have a script handy?
 
 find . -type f -exec perl -pi -e 's|2000-2001|2000-2002|' {} \;

It always seems to me that if you are going to put a copyright date on each
individual source file, you shouldn't change it unless you actually make a 
change to that file.


David L. Jones   |  Phone:(614) 292-6929
Ohio State University|  Internet:
140 W. 19th St. Rm. 231a |   [EMAIL PROTECTED]
Columbus, OH 43210   |   [EMAIL PROTECTED]

Disclaimer: I'm looking for marbles all day long.

Patch for PR# 4634

[Dave Dribin]

Hi,

I've attached a patch against 1.3.23 that modifies ./configure to take
a --force-suffix option as a solution to PR# 4634.

  http://bugs.apache.org/index.cgi/full/4634

This keeps the default behavior as is, but allows people who want to
have the additional httpd tacked on the directories to do so, even
if it is already in an apache subdirectory.

BTW, for those wondering why this would be useful, I use epkg to
install Apache as an encap package.

  http://www.encap.org/epkg/

The extra httpd subdirectories become useful when epkg makes
symlinks to the encap directory, such as:

  /usr/local/etc/httpd/httpd.conf - ../../encap/apache-1.3.23/etc/httpd/httpd.conf

I'd rather keep the files in /usr/local/etc/httpd rather than
/usr/local/etc.

-Dave


diff -uNr apache_1.3.23/configure apache_1.3.23.1/configure
--- apache_1.3.23/configure Thu Sep 27 13:12:03 2001
+++ apache_1.3.23.1/configure   Tue Feb  5 09:38:37 2002
@@ -237,6 +237,7 @@
 #   layout configuration
 with_layout=0
 show_layout=0
+force_suffix=0
 
 #   suexec defaults
 suexec=0
@@ -434,6 +435,7 @@
 echo  --runtimedir=DIR   install runtime data in DIR
 echo  --logfiledir=DIR   install logfile data in DIR
 echo  --proxycachedir=DIRinstall proxy cache data in DIR
+echo  --force-suffix always use TARGET as a suffix
 echo 
 echo Configuration options:
 echo  --enable-rule=NAME enable  a particular Rule named 'NAME'
@@ -703,6 +705,9 @@
 proxycachedir=$apc_optarg 
 autosuffix_proxycachedir=no
 ;;
+--force-suffix) 
+force_suffix=1
+;;
 --add-module=*) 
 file=$apc_optarg
 if [ x`echo $file | egrep '/?mod_[a-zA-Z0-9][a-zA-Z0-9_]*\.c$'` = x 
]; then
@@ -1124,10 +1129,14 @@
 #   still not part of path)
 eval autosuffix=\$autosuffix_$var
 if [ x$autosuffix = xyes ]; then
-addtarget=no
-if [ x`echo $val | grep apache` = x ]; then
-if [ x`echo $val | grep $thetarget` = x ]; then
-addtarget=yes
+if [ x$force_suffix = x1 ]; then
+addtarget=yes
+else
+addtarget=no
+if [ x`echo $val | grep apache` = x ]; then
+if [ x`echo $val | grep $thetarget` = x ]; then
+addtarget=yes
+fi
 fi
 fi
 if [ x$addtarget = xyes ]; then

RE: [STATUS] (httpd-2.0) Wed Jan 2 23:45:06 EST 2002

[Dave Seidel]

Just a reminder, that I and at least one other person (Dwayne Miller
[[EMAIL PROTECTED]]) have reported that we have been unable to get Apache
to run as an NT service when we do our own builds.  Should I add this to a
bug database somewhere?

- Dave

-Original Message-
From: Rodent of Unusual Size [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, January 02, 2002 11:45 PM
To: Apache HTTP server developers
Subject: [STATUS] (httpd-2.0) Wed Jan 2 23:45:06 EST 2002


APACHE 2.0 STATUS:  -*-text-*-
Last modified at [$Date: 2002/01/02 19:34:47 $]

[BIG SNIP]

RE: problem running 2.0.28 as Win2k service (was 2.0.18)

[Dave Seidel]

Any other thoughts on this one?  It's going to become important very soon
that I be able to run my own build as a service.

Thanks.

- Dave

-Original Message-
From: Dave Seidel [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, December 12, 2001 3:29 PM
To: William A. Rowe, Jr.; [EMAIL PROTECTED]
Subject: RE: problem running 2.0.28 as Win2k service (was 2.0.18)


OK, I installed the August 2001 Platform SDK from the November MSDN issue
(if there's a later one, they haven't sent it, and the website is down).
Made sure to install the Visual Studio integration, etc.  Wiped out
httpd-2_0_28, unzipped a fresh copy, rebuilt.  Same problem.

Then I reinstalled VS6SP5, rebuilt from scratch again.  Same problem.

-Original Message-
From: William A. Rowe, Jr. [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, December 12, 2001 12:38 PM
To: [EMAIL PROTECTED]; [EMAIL PROTECTED]
Subject: Re: problem running 2.0.28 as Win2k service (was 2.0.18)


From: Dave Seidel [EMAIL PROTECTED]
Sent: Wednesday, December 12, 2001 9:10 AM


 FYI, I did my local build using VC 6.0, using the InstallBin target in the
 IDE, using all the project files as they came from the zip archive.

Dave, if you grab a recent PlatformSDK from Microsoft's site, and install it
into your DevStudio, does it work?  That will help us break apart those two
issues, since distributions are always built with a more-recent PlatformSDK,
and VC6.0 SP5.

RE: problem running 2.0.28 as Win2k service (was 2.0.18)

[Dave Seidel]

More info:

I just installed the official 2.0.28 msi installer, and (of course) put it
into a different directory from the one into which I installed the 2.0.28
binary I built myself.  The httpd.conf configuration is identical except for
Listen (7171 vs. 7070), the server name, and the directory locations.  The
service is configured (by default) to logon as the LocalSystem account, and
the install directory has the usual full control for everyone permissions
as inherited from the root directory, all of which is exactly the way I had
the locally-built version configured, at least initially.

Although the installer did *not* install the service, I did so using -k
install, then I fired up ApacheMonitor and started up the service from
there.

IT WORKS.

So now my question is: what is the difference between the binaries I built
and the binaries ASF built, and why?  What exactly is causing the
service-based configuration to fail in my build, but not in the ASF build?
Since at least one other person has the same problem, there must be
something there.  I can't reply on the ASF-supplied binary, because I'm
developing something that will require a custom-built binary.

FYI, I did my local build using VC 6.0, using the InstallBin target in the
IDE, using all the project files as they came from the zip archive.

-Original Message-
From: Dave Seidel [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, December 11, 2001 3:53 PM
To: [EMAIL PROTECTED]
Subject: RE: problem running 2.0.28 as Win2k service (was 2.0.18)


This is precisely the same as what I'm seeing, the only difference being
that I built from the 2.0.28 archive.

-Original Message-
From: Dwayne Miller [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, December 11, 2001 3:46 PM
To: [EMAIL PROTECTED]
Subject: Re: problem running 2.0.28 as Win2k service (was 2.0.18)


I just compiled the latest CVS tree under Win32.  Runs fine when started
as a normal user from command line.  However, I see the following error
when trying to start the service as an administrator. I have installed
the service prior to this with no apparent errors (it appears in the
services manager).

D:\Apache2bin\apache -k start -n Apache2
Starting the Apache2 service
[Tue Dec 11 15:32:33 2001] [crit] (23497)Overlapped I/O operation is in
progress
  : Apache2: Failed to start the service process.

There are no errors in the log files
Using net start Apache2 simply reports that service could not start

RE: problem running 2.0.28 as Win2k service (was 2.0.18)

[Dave Seidel]

No SSL in either.

-Original Message-
From: Dwayne Miller [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, December 12, 2001 10:45 AM
To: [EMAIL PROTECTED]
Subject: Re: problem running 2.0.28 as Win2k service (was 2.0.18)


Did you build with SSL support?  Does the MSI file contain SSL support? 
 I've also experienced your problem and would like to track it down.

RE: problem running 2.0.28 as Win2k service (was 2.0.18)

[Dave Seidel]

OK, I installed the August 2001 Platform SDK from the November MSDN issue
(if there's a later one, they haven't sent it, and the website is down).
Made sure to install the Visual Studio integration, etc.  Wiped out
httpd-2_0_28, unzipped a fresh copy, rebuilt.  Same problem.

Then I reinstalled VS6SP5, rebuilt from scratch again.  Same problem.

-Original Message-
From: William A. Rowe, Jr. [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, December 12, 2001 12:38 PM
To: [EMAIL PROTECTED]; [EMAIL PROTECTED]
Subject: Re: problem running 2.0.28 as Win2k service (was 2.0.18)


From: Dave Seidel [EMAIL PROTECTED]
Sent: Wednesday, December 12, 2001 9:10 AM


 FYI, I did my local build using VC 6.0, using the InstallBin target in the
 IDE, using all the project files as they came from the zip archive.

Dave, if you grab a recent PlatformSDK from Microsoft's site, and install it
into your DevStudio, does it work?  That will help us break apart those two
issues, since distributions are always built with a more-recent PlatformSDK,
and VC6.0 SP5.

problem running 2.0.18 as Win2k service

[Dave Seidel]

I just built a 2.0.18 binary from scratch from the zip archive, using VC6 on
Win2K.  Works fine from the console, but it does not work as a service.
Here's a summary:

Command   Result
Apache -k install   Installing the Apache2 service
The Apache2 service is successfully installed.

net start Apache2   The Apache2 service is starting.
The Apache2 service could not be started.
The service did not report an error.
More help is available by typing NET HELPMSG 3534.

Apache -k start Starting the Apache2 service
[Tue Dec 11 11:03:57 2001] [crit] (23497)Overlapped 
I/O operation is in
progress.  :
Apache2: Failed to start the service process.


Any ideas?  Is this a tested configuration in this beta (I don't see much
about it in the status reports)?  What else can I try to provide more data?

- Dave

--
Dave Seidel, Founder
Mindreef, LLC

RE: problem running 2.0.28 as Win2k service (was 2.0.18)

[Dave Seidel]

Whoops, that was a typo.  I am indeed using the 2.0.28 beta release.  Sorry
for the confusion.

-Original Message-
From: Cliff Woolley [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, December 11, 2001 11:40 AM
To: Apache-Dev; [EMAIL PROTECTED]
Subject: Re: problem running 2.0.18 as Win2k service


On Tue, 11 Dec 2001, Dave Seidel wrote:

 I just built a 2.0.18 binary from scratch from the zip archive, using VC6
on
 Win2K.  Works fine from the console, but it does not work as a service.
 Any ideas?  Is this a tested configuration in this beta (I don't see much
 about it in the status reports)?  What else can I try to provide more
data?

Please try again with 2.0.28, the most recent beta.  2.0.18, besides being
very old at this point, was released as an alpha release, not beta.

Thanks!
--Cliff

--
   Cliff Woolley
   [EMAIL PROTECTED]
   Charlottesville, VA

RE: problem running 2.0.28 as Win2k service (was 2.0.18)

[Dave Seidel]

Nope, 0 bytes in the error log.

-Original Message-
From: Bill Stoddard [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, December 11, 2001 12:01 PM
To: [EMAIL PROTECTED]; [EMAIL PROTECTED]
Subject: Re: problem running 2.0.28 as Win2k service (was 2.0.18)


Any messages in the error log?

Bill

 Whoops, that was a typo.  I am indeed using the 2.0.28 beta release.
Sorry
 for the confusion.

 -Original Message-
 From: Cliff Woolley [mailto:[EMAIL PROTECTED]]
 Sent: Tuesday, December 11, 2001 11:40 AM
 To: Apache-Dev; [EMAIL PROTECTED]
 Subject: Re: problem running 2.0.18 as Win2k service


 On Tue, 11 Dec 2001, Dave Seidel wrote:

  I just built a 2.0.18 binary from scratch from the zip archive, using
VC6
 on
  Win2K.  Works fine from the console, but it does not work as a service.
  Any ideas?  Is this a tested configuration in this beta (I don't see
much
  about it in the status reports)?  What else can I try to provide more
 data?

 Please try again with 2.0.28, the most recent beta.  2.0.18, besides being
 very old at this point, was released as an alpha release, not beta.

 Thanks!
 --Cliff

 --
Cliff Woolley
[EMAIL PROTECTED]
Charlottesville, VA

RE: problem running 2.0.28 as Win2k service (was 2.0.18)

[Dave Seidel]

Good suggestion, since I hadn't been looking at permissions at all, but
there seems to be some other problem.  I just created a new user account in
the Users group, gave it full control permissions in the Apache2
directory, and the problem is still there.  Even if I make the user a member
of the Administrators group, it still doesn't work.

-Original Message-
From: Bill Stoddard [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, December 11, 2001 12:56 PM
To: [EMAIL PROTECTED]; [EMAIL PROTECTED]
Subject: Re: problem running 2.0.28 as Win2k service (was 2.0.18)


This sounds like a permissions problem.  Make sure that the user id you are
running Apache
under has permission to read/write to all the appropriate directories.

Bill

 Nope, 0 bytes in the error log.

 -Original Message-
 From: Bill Stoddard [mailto:[EMAIL PROTECTED]]
 Sent: Tuesday, December 11, 2001 12:01 PM
 To: [EMAIL PROTECTED]; [EMAIL PROTECTED]
 Subject: Re: problem running 2.0.28 as Win2k service (was 2.0.18)


 Any messages in the error log?

 Bill

  Whoops, that was a typo.  I am indeed using the 2.0.28 beta release.
 Sorry
  for the confusion.
 
  -Original Message-
  From: Cliff Woolley [mailto:[EMAIL PROTECTED]]
  Sent: Tuesday, December 11, 2001 11:40 AM
  To: Apache-Dev; [EMAIL PROTECTED]
  Subject: Re: problem running 2.0.18 as Win2k service
 
 
  On Tue, 11 Dec 2001, Dave Seidel wrote:
 
   I just built a 2.0.18 binary from scratch from the zip archive, using
 VC6
  on
   Win2K.  Works fine from the console, but it does not work as a
service.
   Any ideas?  Is this a tested configuration in this beta (I don't see
 much
   about it in the status reports)?  What else can I try to provide more
  data?
 
  Please try again with 2.0.28, the most recent beta.  2.0.18, besides
being
  very old at this point, was released as an alpha release, not beta.
 
  Thanks!
  --Cliff
 
  --
 Cliff Woolley
 [EMAIL PROTECTED]
 Charlottesville, VA
 
 
 
 
 
 

RE: problem running 2.0.28 as Win2k service (was 2.0.18)

[Dave Seidel]

Sorry, makes no difference.  Although I hadn't previously used the -n arg,
using it didn't change the behavior.

-Original Message-
From: William A. Rowe, Jr. [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, December 11, 2001 2:01 PM
To: [EMAIL PROTECTED]; [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Subject: Re: problem running 2.0.28 as Win2k service (was 2.0.18)


bin\apache -k install -n Apache2

 THEN;

apache -k start -n apache2

 -OR-

net start Apache2

The Apache2 (default) may have tripped you up.  Keeps it distinct from
1.3 generation apache services, but you can use the -n Apache arg instead,
if you like.

Bill

- Original Message -
From: Dave Seidel [EMAIL PROTECTED]
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Sent: Tuesday, December 11, 2001 10:58 AM
Subject: RE: problem running 2.0.28 as Win2k service (was 2.0.18)


 Whoops, that was a typo.  I am indeed using the 2.0.28 beta release.
Sorry
 for the confusion.

 -Original Message-
 From: Cliff Woolley [mailto:[EMAIL PROTECTED]]
 Sent: Tuesday, December 11, 2001 11:40 AM
 To: Apache-Dev; [EMAIL PROTECTED]
 Subject: Re: problem running 2.0.18 as Win2k service


 On Tue, 11 Dec 2001, Dave Seidel wrote:

  I just built a 2.0.18 binary from scratch from the zip archive, using
VC6
 on
  Win2K.  Works fine from the console, but it does not work as a service.
  Any ideas?  Is this a tested configuration in this beta (I don't see
much
  about it in the status reports)?  What else can I try to provide more
 data?

 Please try again with 2.0.28, the most recent beta.  2.0.18, besides being
 very old at this point, was released as an alpha release, not beta.

 Thanks!
 --Cliff

 --
Cliff Woolley
[EMAIL PROTECTED]
Charlottesville, VA

RE: problem running 2.0.28 as Win2k service (was 2.0.18)

[Dave Seidel]

Well, since this is a developer machine, everyone (actually, Everyone
group) has full permissions from the root of this drive down.  And I would
have thought that adding my user to the Admin group would also give it
privileges.

In any case, I went full out and added the user to both the Admin *and*
Backup Operator groups, gave it full, inheritable permissions from the root
on down, triple-checked that the Apache2 service is set to log on as this
specific user...and the problem still occurs.

-Original Message-
From: William A. Rowe, Jr. [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, December 11, 2001 2:27 PM
To: [EMAIL PROTECTED]; [EMAIL PROTECTED]
Subject: Re: problem running 2.0.28 as Win2k service (was 2.0.18)


Dave, win32 canonicalization requires list/read access to each directory
above Apache directories, to the root.  It doesn't have to have access to
their contents, but it needs to be able to list files, so it can determine
that c:\rootdir is spelled rootdir and not RootDir, so we don't
escape from filename normalization, and bypass Directory  restrictions.

To get file attributes of directories, the 'Backup Operator' flag may have
to be toggled, or they may not have permission to open a filehandle to
the directory.

Bill

RE: problem running 2.0.28 as Win2k service (was 2.0.18)

[Dave Seidel]

This is precisely the same as what I'm seeing, the only difference being
that I built from the 2.0.28 archive.

-Original Message-
From: Dwayne Miller [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, December 11, 2001 3:46 PM
To: [EMAIL PROTECTED]
Subject: Re: problem running 2.0.28 as Win2k service (was 2.0.18)


I just compiled the latest CVS tree under Win32.  Runs fine when started
as a normal user from command line.  However, I see the following error
when trying to start the service as an administrator. I have installed
the service prior to this with no apparent errors (it appears in the
services manager).

D:\Apache2bin\apache -k start -n Apache2
Starting the Apache2 service
[Tue Dec 11 15:32:33 2001] [crit] (23497)Overlapped I/O operation is in
progress
  : Apache2: Failed to start the service process.

There are no errors in the log files
Using net start Apache2 simply reports that service could not start