status report for ASF board

[Roy T. Fielding]

I need to prepare a status report for the HTTP server project
for the ASF board meeting on Wednesday (IIRC).  Given that
I'll be on vacation for two weeks starting Monday, it would
be nice to have some text ready to send in by Sunday night.

Our last report on 2005_05_18 consisted of

===
   Attachment D: Status report for the Apache HTTP Server Project

   Activity is picking up again.  We are happy to say we have
   invited a number of new committers into apreq, docs, httpd
   and mod_python.

   There were a couple of releases: libapreq2-2.05-dev and
   httpd-2.0.54.

   There is increased interest in branching httpd 2.1 and working
   towards an initial 2.2 release.
===

Since that time, we have made no official changes to the PMC
(that will change very soon) and have added a number of
committers for the Google SoC projects.  Do we have a summary
of the SoC projects and committers somewhere?

We have also cut one alpha release called httpd 2.1.6. Other
releases are pending some fixes in APR.

Sander Striker has decided to step down as chair and VP of the
HTTP server project after having been volunteered into being
President of ASF.  I have been asked to come out of retirement
and be the new conveyer of choice words between this project
and the board.

Should I add anything else?  Could the people who accepted PMC
or committer status in the past three months please remind me
to include your name in the report?  Thanks.

Cheers,

Roy T. Fielding
Chief Scientist, Day Software  

New mod_smtpd release

[Rian Hunter]

Hi,

I've checked in mod_smtpd 0.9 and its API should be completely frozen  
by now. This version of mod_smtpd is heavily based on Qpsmtpd, so the  
same extensibility you expect from Qpsmtpd can be achieved with this  
version of mod_smtpd. I haven't written any documentation yet but  
here is a quick run-down of how to use it:


In your httpd.conf, make sure you have SmtpProtocol On, if you are  
setting up a virtualhost make sure the virtualHost container has the  
ServerName directive (duh).


This version of mod_smtpd is callback based, very similar to Qpsmtpd.  
Here is a list of all the hooks you can register:


smtpd_run_unrecognized_command
smtpd_run_connect
smtpd_run_reset_transaction
smtpd_run_helo
smtpd_run_ehlo
smtpd_run_mail
smtpd_run_rcpt
smtpd_run_vrfy
smtpd_run_quit
smtpd_run_data
smtpd_run_data_post
smtpd_run_data_queue

You can register a hook to one of these by calling:

APR_OPTIONAL_HOOK(smtpd, /* hook name */ vrfy, /* function address */  
default_vrfy, NULL, NULL, APR_HOOK_FIRST);


In your register hooks function. Each hook you register should return  
smtpd_retcode, to see what retcodes make sense for each callback you  
should look at smtp_protocol.c until I write better documentation. To  
see what argument each different type of hook takes look at smtp_core.c.


The code is very small and simple and shouldn't be too hard to figure  
out if you're familiar with apache modules. Currently it works with  
httpd 2.0 and up.


You can checkout this code out from:
http://svn.apache.org/repos/asf/httpd/mod_smtpd/trunk/

Have Fun!
-rian

Re: svn commit: r232406 - in /httpd/mod_smtpd/trunk: configure configure.ac

[Garrett Rooney]

[EMAIL PROTECTED] wrote:

Author: soc-rian
Date: Fri Aug 12 16:56:59 2005
New Revision: 232406

URL: http://svn.apache.org/viewcvs?rev=232406&view=rev
Log:
More robuse apache version checking in configure

Modified:
httpd/mod_smtpd/trunk/configure
httpd/mod_smtpd/trunk/configure.ac


Would it be possible to not have configure (and any other generated 
files) checked in to svn?  It makes the diffs rather hard to read.


-garrett

Re: Questions about SSL 0.9.8 and Apache 2.0.54 compiling errors on released versions

[William A. Rowe, Jr.]

At 04:28 AM 8/12/2005, Fields Marshall wrote:

>I have installed and compiled OpenSSL 0.9.8 and then started to compile apache 
>now I am getting the `PEM_F_DEF_CALLBACK' undeclared error.

Fixed in the next release.  Sorry, our psychic skills have
been falling down lately, and the steam rising out of my
coffee cup didn't foreshadow their API change early enough
to have this pre-fixed in the now-current release 2.0.54 :)

Bill

Re: Apache2 FIPS Certified?

[William A. Rowe, Jr.]

At 08:12 AM 8/12/2005, Jess Holle wrote:
>Thanks for the information, Bill.
>
>As best I could tell it looks like the OpenSSL folk have not gotten around to 
>bringing the fips mode forward into 0.9.8 yet either...

That's not as likely to happen on any particular schedule, and
would be a pointless exercise until the implementation under
test passes muster.  Who knows, certain parts may be sent back
to the OpenSSL project for complete rework.  Why port what may
be a moving target?

You have to understand that FIPS testing is an expensive, time
consuming, cyclic process.  The crypto code was *FROZEN* at a
specific point in time.  There is a certain threshold for 
allowable fixes before the module must be re-certified, but
you won't be seeing many rapid releases of crypto code changes,
as is the general course for OpenSSL project development.


http://oss-institute.org/index.php?option=content&task=view&id=109

is the current news, such as it is.

Bill 

Re: [PATCH] typo in manual

[Wilfredo Sánchez Vega]

  On HEAD and 2.2.

Thanks,
-wsv

On Aug 9, 2005, at 7:36 AM, Ben Collins-Sussman wrote:


[[[
Fix typo in manual.

* docs/manual/logs.xml:  typo.  "flexibly"-->"flexible".
]]]


Index: docs/manual/logs.xml
===
--- docs/manual/logs.xml(revision 231041)
+++ docs/manual/logs.xml(working copy)
@@ -425,7 +425,7 @@
   
   Although we have just shown that conditional logging is very
-  powerful and flexibly, it is not the only way to control the
+  powerful and flexible, it is not the only way to control the
   contents of the logs. Log files are more useful when they
   contain a complete record of server activity. It is often
   easier to simply post-process the log files to remove requests





smime.p7s
Description: S/MIME cryptographic signature

Re: [PATCH] Make caching hash more deterministic

[Jim Jagielski]

On Aug 12, 2005, at 1:10 PM, Colm MacCarthaigh wrote:


On Fri, Aug 12, 2005 at 04:59:20PM +0100, Colm MacCarthaigh wrote:


On Fri, Aug 12, 2005 at 11:54:44AM -0400, Brian Akins wrote:


Should this honor usecanonicalname?  If so, could just use
ap_get_servername(r) in stead of r->hostname.  This may further  
compact

the number of entries.



Yes, but I think there'd have to be additional code to detect the  
proxy

cases. And you pointing that out has just reminded me of a bug in my
patch - it doesn't work for;

GET ftp://ftp.heanet.ie/pub/heanet/100.txt HTTP/1.0

I'll go make that work too.



Here's a more involved patch that gets the logic right. It's 6pm on a
Friday for me, so I have only tested it a little, but thought I'd  
share

for comment before the weekend.



+1 on inspection... testing to be done over
the weekend :)

mod_headers: changes to support direct HTTP header mapping (+Vary)

[Paul Rhodes]

Hi,

I've been using mod_headers extensively for a number of years to modify 
headers from as they enter and exit the network. We use apache as a 
front end to our appservers and we get requests from multiple sources so 
we often have issues with header translation. This means we have to do 
something like this:


 RewriteCond %{HTTP:INPUTHEADER1}  ([0-9]+)
 RewriteRule (.*) $1 [E=HEADER:%1]

 RewriteCond %{HTTP:INPUTHEADER2}  ([0-9]+)
 RewriteCond %{ENV:HEADER} (.+)
 RewriteRule (.*) $1 [E=HEADER:%1]

 RewriteCond %{ENV:HEADER} (.+)
 RewriteRule (.*) $1 [E=HEADER:"default"]


 RequestHeader set OUTPUTHEADER %{HEADER}e env=HEADER

This has worked kind of ok until now, but we are now looking at caching 
issues and wish to ensure that the corresponding Vary is mapped. This 
makes the situation somewhat more complicated and I don't think it's 
really possible to continue doing this via rewrite rules.


I've looked at mod_headers and have started implementing a patch. The 
patch provides the following functionality:


 Header|RequestHeader default  

 The header  is only set if  does not already exist.


 Header|RequestHeader copy  

 This will copy the value of  to  if and 
only if  is not already present.



 Header|RequestHeader rename  

 This will copy the value of  to  if and 
only if  is not already present. In addition to this it will 
remove .



The combination of the above will allow me to put something like the 
following..


 RequestHeader rename OUTPUTHEADER INPUTHEADER1
 RequestHeader rename OUTPUTHEADER INPUTHEADER2
 RequestHeader default "default"

This is obviously neater, but the main reason to do this is to have an 
explicit link between the headers so that when a response comes back, 
the Vary can be catered for. So if 'Vary: OUTPUTHEADER' is sent back 
from the appserver, apache can convert this to 'Vary: INPUTHEADER1' or 
'Vary: INPUTHEADER2' as appropriate.


Ok. That is a lot of background, but my question is what is the best way 
to track which of the translations took place? I was trying to avoid 
bodging it with setting an environment variable or using notes. Is there 
somewhere within the datastructures where I can keep track of which 
rules fired?


Also, I have made an effort to genericise this implementation so it fits 
logically on top of mod_headers. Is there any possibility that such a 
patch might be commited for mod_headers?


thanks in advance,

Paul

Re: [PATCH] Make caching hash more deterministic

[Colm MacCarthaigh]

On Fri, Aug 12, 2005 at 04:59:20PM +0100, Colm MacCarthaigh wrote:
> On Fri, Aug 12, 2005 at 11:54:44AM -0400, Brian Akins wrote:
> > Should this honor usecanonicalname?  If so, could just use 
> > ap_get_servername(r) in stead of r->hostname.  This may further compact 
> > the number of entries.
> 
> Yes, but I think there'd have to be additional code to detect the proxy
> cases. And you pointing that out has just reminded me of a bug in my
> patch - it doesn't work for;
> 
>   GET ftp://ftp.heanet.ie/pub/heanet/100.txt HTTP/1.0
> 
> I'll go make that work too.

Here's a more involved patch that gets the logic right. It's 6pm on a
Friday for me, so I have only tested it a little, but thought I'd share
for comment before the weekend.

-- 
Colm MacCárthaighPublic Key: [EMAIL PROTECTED]
Index: modules/cache/cache_storage.c
===
--- modules/cache/cache_storage.c   (revision 232304)
+++ modules/cache/cache_storage.c   (working copy)
@@ -318,12 +318,90 @@
 apr_status_t cache_generate_key_default(request_rec *r, apr_pool_t* p,
 char**key)
 {
-if (r->hostname) {
-*key = apr_pstrcat(p, r->hostname, r->uri, "?", r->args, NULL);
+const char *hostname;
+char *port_str, *scheme, *hn;
+int i;
+
+/* Use the canonical name to improve cache hit rate, but only if this is
+ * not a proxy request. 
+ */ 
+if (!r->proxyreq) {
+/* Use _default_ as the hostname if none present, as in mod_vhost
+ */
+hostname = ap_get_server_name(r);
+if (!hostname) {
+hostname = "_default_";
+}
 }
+else if(r->parsed_uri.hostname) {
+/* Copy the parsed uri hostname */
+hn = apr_pcalloc(p, strlen(r->parsed_uri.hostname) + 1);
+for (i = 0; r->parsed_uri.hostname[i]; i++) {
+hn[i] = apr_tolower(r->parsed_uri.hostname[i]);
+}
+
+/* const work-around */
+hostname = hn;
+}
 else {
-*key = apr_pstrcat(p, r->uri, "?", r->args, NULL);
+/* We are a proxied request, with no hostname. Unlikely
+ * to get very far - but just in case */
+hostname = "_default_";
 }
+  
+/* Copy the scheme, ensuring that it is lower case. If the parsed uri
+ * contains no string or if this is not a proxy request, we use "local" as
+ * the default. 
+ *
+ * Why "local"? Ans: to indicate that the content is locally generated, and
+ * because  Apache can serve multiple protocols, lets not get tied to a
+ * single one. This way a mod_[ftp|bittorrent|foobar] front-end can share
+ * our content cache.
+ */
+if (r->proxyreq && r->parsed_uri.scheme) {
+/* Copy the scheme */
+scheme = apr_pcalloc(p, strlen(r->parsed_uri.scheme) + 1);
+for (i = 0; r->parsed_uri.scheme[i]; i++) {
+scheme[i] = apr_tolower(r->parsed_uri.scheme[i]);
+}
+}
+else {
+scheme = "local";
+}
+
+/* If the content is locally generated, use the port-number of the
+ * current server. Otherwise. copy the URI's port-string (which may be a
+ * service name). If the URI contains no port-string, use apr-util's
+ * notion of the default port for that scheme - if available.
+ */
+if(r->proxyreq) {
+if (r->parsed_uri.port_str) {
+port_str = apr_pcalloc(p, strlen(r->parsed_uri.port_str) + 2);
+port_str[0] = ':';
+for (i = 0; r->parsed_uri.port_str[i]; i++) {
+port_str[i + 1] = apr_tolower(r->parsed_uri.port_str[i]);
+}
+}
+else if (apr_uri_port_of_scheme(scheme)) {
+port_str = apr_psprintf(p, ":%u", apr_uri_port_of_scheme(scheme));
+}
+else {
+/* No port string given in the AbsoluteUri, and we have no
+ * idea what the default port for the scheme is. Leave it
+ * blank and live with the inefficiency of some extra cached
+ * entities.
+ */
+port_str = "";
+}   
+}   
+else {
+/* Use the server port */
+port_str = apr_psprintf(p, ":%u", ap_get_server_port(r));
+}
+
+/* Key format is a URI */
+*key = apr_pstrcat(p, scheme, "://", hostname, port_str,
+   r->parsed_uri.path, "?", r->args, NULL);
+
 return APR_SUCCESS;
 }
-

Re: [PATCH] fix incorrect 304's responses when cache is unwritable

[Justin Erenkrantz]

--On August 8, 2005 9:46:52 PM +0200 [EMAIL PROTECTED] wrote:


log_correction.diff:

...

dir_removal_patch.diff:


Committed in r232335 and r232334, respectively.

Thanks!  -- justin

Re: [PATCH] Make caching hash more deterministic

[Colm MacCarthaigh]

On Fri, Aug 12, 2005 at 11:54:44AM -0400, Brian Akins wrote:
> Should this honor usecanonicalname?  If so, could just use 
> ap_get_servername(r) in stead of r->hostname.  This may further compact 
> the number of entries.

Yes, but I think there'd have to be additional code to detect the proxy
cases. And you pointing that out has just reminded me of a bug in my
patch - it doesn't work for;

GET ftp://ftp.heanet.ie/pub/heanet/100.txt HTTP/1.0

I'll go make that work too.

-- 
Colm MacCárthaighPublic Key: [EMAIL PROTECTED]

Re: [PATCH] Make caching hash more deterministic

[Brian Akins]

Should this honor usecanonicalname?  If so, could just use 
ap_get_servername(r) in stead of r->hostname.  This may further compact 
the number of entries.




--
Brian Akins
Lead Systems Engineer
CNN Internet Technologies

[PATCH] Make caching hash more deterministic

[Colm MacCarthaigh]

Currently;

GET / HTTP/1.1
Host: ftp.heanet.ie

GET http://ftp.heanet.ie/ HTTP/1.0

GET HTTP://Ftp.Heanet.Ie/ HTTP/1.0

are all mapped to different hashes by mod_cache; despite being the same
content, this is an inefficient waste of disk space and really awkward
for me trying to write a debug/admin tool.

The attached patch makes it deterministic, by mapping them all to;

"http://ftp.heanet.ie:80/?"; 

Instead of "ftp.heanet.ie/?". For for a cached webserver, this really
won't make much of a difference since the Host-header is forcably
lower-cased anyway, but for a proxy it definitely helps.  Looking
through my logs I'm seeing lots of simple domain case variations - no
point storing them twice and handling all of the expires multiple times.

It also solves the colision that happens if aan administrator wants to
run Apache listening on multiple ports, but has mod_cache enabled. 

The only awkwardness I can see with this approach, is that;

GET / HTTP/1.0

would look like this;

"http://:80/?";

So, I've re-used the _default_ "convention" (underscores are not
permitted in DNS anyway) for such keys;

"http://_default_:80/?";

Which should at least make a familiar sort of sence to an administrator.

-- 
Colm MacCárthaighPublic Key: [EMAIL PROTECTED]
Index: modules/cache/cache_storage.c
===
--- modules/cache/cache_storage.c   (revision 232304)
+++ modules/cache/cache_storage.c   (working copy)
@@ -318,12 +318,46 @@
 apr_status_t cache_generate_key_default(request_rec *r, apr_pool_t* p,
 char**key)
 {
-if (r->hostname) {
-*key = apr_pstrcat(p, r->hostname, r->uri, "?", r->args, NULL);
+const char *hostname;
+char *port_str, *scheme;
+int i;
+
+/* Use _default_ as the hostname if none present, as in mod_vhost
+ * Note: r->hostname is always lowercase
+ */
+hostname = r->hostname ? r->hostname : "_default_";
+  
+/* Copy the scheme, ensuring that it is lower case. If the parsed uri
+ * contains no string, we use "http" as the default. This is a fair
+ * assumption, as request_rec is HTTP-specific.
+ */
+if (r->parsed_uri.scheme) {
+scheme = apr_pcalloc(p, strlen(r->parsed_uri.scheme) + 1);
+for (i = 0; r->parsed_uri.scheme[i]; i++) {
+scheme[i] = apr_tolower(r->parsed_uri.scheme[i]);
+}
 }
 else {
-*key = apr_pstrcat(p, r->uri, "?", r->args, NULL);
+scheme = "http";
 }
+
+/* Copy the port string, ensuring that it is lower case (it may be a
+ * service name. If not present, use the connection to determine port
+ * number
+ */
+if (r->parsed_uri.port_str) {
+port_str = apr_pcalloc(p, strlen(r->parsed_uri.port_str) + 1);
+for (i = 0; r->parsed_uri.port_str[i]; i++) {
+port_str[i] = apr_tolower(r->parsed_uri.port_str[i]);
+}
+}
+else {
+port_str = apr_psprintf(p, "%u", ap_get_server_port(r));
+}
+
+/* Key format is a URI */
+*key = apr_pstrcat(p, scheme, "://", hostname, ":", port_str,
+   r->parsed_uri.path, "?", r->args, NULL);
+
 return APR_SUCCESS;
 }
-

RE: Apache2 FIPS Certified?

[Fenlason, Josh]

Thanks for the info.
,
Josh.

> -Original Message-
> From: William A. Rowe, Jr. [mailto:[EMAIL PROTECTED] 
> Sent: Thursday, August 11, 2005 6:44 PM
> To: dev@httpd.apache.org
> Cc: dev@httpd.apache.org
> Subject: Re: Apache2 FIPS Certified?
> 
> 
> Plenty.  First, OpenSSL is -not- FIPS certified.  It's in
> the certification under test (CUT) phase, and no word of 
> exactly what will come of that phase.  Second, you would have 
> to enable OpenSSL's fips-only mode, and stop using all 
> prohibited entropy, hashing and crypto.
> 
> The http project has a little side-repository Ben and I have 
> been working on which will throw these flags appropriately, 
> and replace some components of httpd and apr.  I'd point you 
> at it, but the caveat remains that you still won't have any 
> fips web server after all your effort.  Not until OpenSSL has 
> completed the process.
> 
> FWIW, any designation of "FIPS certification pending" happens 
> to be expressly prohibited by the FIPS requirements 
> themselves, so it's not possible to proactively provide a 
> solution with any claims whatsoever.
> 
> Ben and I started this sandbox as a proof of concept to 
> determine what needed to change in apr, httpd, etc, and it's 
> very likely that those features will become part of httpd 
> after the certification process is complete.  If you want to 
> take a look at our unreleased efforts, that repository is in
> 
>   http://svn.apache.org/repos/asf/httpd/httpd/branches/fips-dev/
> 
> Bill
> 
> At 03:59 PM 8/11/2005, Fenlason, Josh wrote:
> >Would anyone be able to tell me if Apache2 is FIPS certified?  If I 
> >build OpenSSL with the FIPS flag, is there anything else I 
> have to do 
> >when building Apache with OpenSSL?  Thanks. , Josh.
> 
> 

Re: Apache2 FIPS Certified?

[Jess Holle]

Thanks for the information, Bill.

As best I could tell it looks like the OpenSSL folk have not gotten
around to bringing the fips mode forward into 0.9.8 yet either...

--
Jess Holle

William A. Rowe, Jr. wrote:

  Plenty.  First, OpenSSL is -not- FIPS certified.  It's in
the certification under test (CUT) phase, and no word of
exactly what will come of that phase.  Second, you would
have to enable OpenSSL's fips-only mode, and stop using
all prohibited entropy, hashing and crypto.

The http project has a little side-repository Ben and I have
been working on which will throw these flags appropriately,
and replace some components of httpd and apr.  I'd point you
at it, but the caveat remains that you still won't have any
fips web server after all your effort.  Not until OpenSSL
has completed the process.

FWIW, any designation of "FIPS certification pending" happens
to be expressly prohibited by the FIPS requirements themselves,
so it's not possible to proactively provide a solution with
any claims whatsoever.

Ben and I started this sandbox as a proof of concept to 
determine what needed to change in apr, httpd, etc, and it's
very likely that those features will become part of httpd after
the certification process is complete.  If you want to take a
look at our unreleased efforts, that repository is in

  http://svn.apache.org/repos/asf/httpd/httpd/branches/fips-dev/

Bill

At 03:59 PM 8/11/2005, Fenlason, Josh wrote:
  
  
Would anyone be able to tell me if Apache2 is FIPS certified?  If I build OpenSSL with the FIPS flag, is there anything else I have to do when building Apache with OpenSSL?  Thanks.
,
Josh.

  
  

  

Re: svn commit: r231355 - in /httpd/httpd/trunk/modules/proxy: mod_proxy.c mod_proxy.h mod_proxy_http.c

[Nick Kew]

Joe Orton wrote:


plus this looks equally broken:


OK, I've bitten the bullet and updated my build environment here to
to build the whole proxy in context.  Apologies for assuming this
fix was too simple to need it.

Note that the 2.0 fix is a straight diff between 2.0.54 and a fully
working fixed version.  That's basically what I was copying from, but
I hadn't appreciated how widespread *relevant* changes had become.

--
Nick Kew

Questions about SSL 0.9.8 and Apache 2.0.54 compiling errors on released versions

[Fields Marshall]

Title: Message




Hi 
this is my first post to the mailing 
list asking perhaps a rather simple question to you apache developers. 

I am writing this email because I see 
that this will be a common problem with people trying to compile the latest SSL 
and Apache.
I think I fit a common 
scenario -- average webmaster/programmer trying to install latest released 
versions of apache with ssl.  Familiar 
with webprogramming but not really in C. However still able 
to Configure, Make and Make install. 
I have searched the net and read the 
faq but no comprehensive information exists perhaps I don’t know how to use the 
patch command well maybe this is causing problems.
I have installed and compiled OpenSSL 
0.9.8 and then started to compile apache now I am getting the 
`PEM_F_DEF_CALLBACK' undeclared error.
I have researched this and have found 
reference to patch file in email archive but I have been unable to apply this 
patch to get something compiled.
When I run the patch command in 
httpd/modules/ssl directory
patch -p0 ssl_toolkit_compat.h 
patchfile

patch:  Only garbage was found in 
the patch input.
Could someone please point me in the 
right direction or a faq on how to properly patch this.  
 I tried the latest version from CVS but 
wasn’t able to configure it in the normal manner.
I can see this being a common 
problem/scenario that is one reason why I am posting this.
Thanks in advance for help
Fields

Memory leak not fixed from 2003

[sternmarc]

Bug #25659 is about a memory leak.
 
The (quite trivial) patch has been provided in 2003, and the 
bug is still not corrected !!!
Could somebody include this is next version ?
 
Thanks,
 
Marc

Re: [PATCH] fix incorrect 304's responses when cache is unwritable

[r . pluem]

Justin Erenkrantz wrote:
> On Fri, Aug 12, 2005 at 05:38:40AM +0200, Plm, Rdiger, VIS wrote:
> 
>>In the case that you are caching a response from a backend app server or
>>a cgi script I can imagine situations where one variant is 404 and another
>>one is not. Dw also pointed that out.
>>From my personal point of view we should keep them and let the next 
>>revalidation
>>on them caused by a client decide whether they should be removed or not.
> 
> 
> I guess I just don't buy that as a legitimate (and compliant) use case; but if
> that's how some servers work, I guess.

Agreed. I confess that I sometimes misuse mod_cache to get the performance of 
odd
commercial web applications fixed. And it is easier to misuse mod_cache and 
sometimes
to patch it for this misuse then to get commercial vendors fix their sloppy 
software :-(.

> 
> So, that should mean that the code is fine as-is.
> 
> 

Basicly yes, from my personal point of view. BTW: Can you have a look at the 
patch I
proposed at 
http://mail-archives.apache.org/mod_mbox/httpd-dev/200508.mbox/[EMAIL PROTECTED]
to delete the empty directories (dir_removal_patch.diff) for the cache entries 
that
get removed? That would be very nice.

[..cut..]

Thanks

Rüdiger

htcacheadmin was: Re: [PATCH] fix incorrect 304's responses when cache is unwritable

[Colm MacCarthaigh]

On Thu, Aug 11, 2005 at 11:48:21PM -0700, Justin Erenkrantz wrote:
> Right.  I think Paul mentioned that we also need to fix up htcacheclean to
> remove the .vary subdirectories as well.  -- justin

Next time someone is commiting to htcacheclean; it's define's for
VARY_FORMAT_VERSION and DISK_FORMAT_VERSION are wrong. They should be 3,
and 4 respectively - as per mod_disk_cache.c. Right now they are 1 and
2.

I'm writing something else now, to help me debug cache edge cases, I'm
still seeing some misbehaviour that I need to track down, but I'm having
trouble re-creating vary caching. 

If the vary content is locally generated, it is saved as per content
location. And right now, I can't get it cache proxied vary content at
all. Though it doesn't help that I've hamfisting the mod_proxy code in
trunk to even compile.

Does anyone have a remote URI on some webserver somewhere that reliably
returns a Vary response that is cacheable?

That something else;  htcacheadmin 

Now that I'm running with an expanded cache, and trying to debug things
while I'm at it, I keep coming accross needing to do the same tasks, but
"find ./ -type f | xargs grep" just isn't a reliable way of tracking
down cache entities. So I've written htcacheadmin.

It's *extremely* useful for debugging, which is why I'm posting it now -
it's helped me a lot. It's useful for administrators too, but I'm not
sure if it's useful enough Vs confusing enough for support/

Anyway, right now it's got most of its functionality working, but it
only works for locally generated non-vary content. I'm going to add a
loop to allow it to work through the various key permutations for proxy
support, eg.

ftp.heanet.ie/pub/heanet/100.txt?
ftp.heanet.iehttp://ftp.heanet.ie/pub/heanet/100.txt?
ftp.heanet.iehttp://ftp.heanet.ie:80/pub/heanet/100.txt?

to track down likely cache matches. Though I wonder is there a hope of
convinving anyone to change the cache key to something utterly
determinstic like;

http://ftp.heanet.ie:80/pub/heanet/100.txt?

including the scheme will help me when I go to make caching work for
ftp, and the rest just helps make it reliably deterministic.

Anyway, other features I want to add including allowing the
administrator to extend the expiry of particular entities, and I think I
might split it's functionality so that it has one mode which locates
(and only locates) cache entities, and then another mode which does the
information retrieval taking a .header file as an argument (more useful
to admins since they can guarantee the output is for one instance only).

See attachment, feedback appreciated.

-- 
Colm MacCárthaighPublic Key: [EMAIL PROTECTED]
/* Copyright 2001-2005 The Apache Software Foundation or its licensors, as
 * applicable.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 * http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

/*
 * htcacheadmin.c: a utility to allow administrators to track down urls
 * in their caches, and perform actions on that basis.
 *
 * Contributed by Colm MacCarthaigh  
 * 11 Aug 2005
 */

#include "apr.h"
#include "apr_lib.h"
#include "apr_strings.h"
#include "apr_file_io.h"
#include "apr_file_info.h"
#include "apr_pools.h"
#include "apr_md5.h"
#include "apr_getopt.h"
#include "apr_date.h"
#include "apr_uri.h"

#if APR_HAVE_UNISTD_H
#include 
#endif
#if APR_HAVE_STDLIB_H
#include 
#endif

/* mod_disk_cache.c extract start */

#define VARY_FORMAT_VERSION 3
#define DISK_FORMAT_VERSION 4

typedef struct
{
/* Indicates the format of the header struct stored on-disk. */
apr_uint32_t format;
/* The HTTP status code returned for this response.  */
int status;
/* The size of the entity name that follows. */
apr_size_t name_len;
/* The number of times we've cached this entity. */
apr_size_t entity_version;
/* Miscellaneous time values. */
apr_time_t date;
apr_time_t expire;
apr_time_t request_time;
apr_time_t response_time;
} disk_cache_info_t;

/* mod_disk_cache.c extract end */

/* cache_util.c extract started */

static void cache_hash(const char *it, char *val, int ndepth, int nlength)
{
apr_md5_ctx_t context;
unsigned char digest[16];
char tmp[22];
int i, k, d;
unsigned int x;
static const char enc_table[64] =
"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789_@";

apr_md5_init(&context);
apr_md5_update(&context, (const unsigned char *) it, strlen(it));
apr_md5_final(digest, &context);

/* en

Re: svn commit: r231355 - in /httpd/httpd/trunk/modules/proxy: mod_proxy.c mod_proxy.h mod_proxy_http.c

[Joe Orton]

On Fri, Aug 12, 2005 at 07:59:49AM +0100, Joe Orton wrote:
> On Thu, Aug 11, 2005 at 07:56:38PM +0100, Nick Kew wrote:
> > Justin Erenkrantz wrote:
> > 
> > >>Fix ProxyPassReverse & family to work correctly in 
> > >
> > >
> > >This commit broke the build.
> > 
> > Aaargh!  Careless cut&paste.  Sorry.  Fixed - thanks.
> 
> proxy_util.c: In function `ap_proxy_location_reverse_map':
> proxy_util.c:1015: error: structure has no member named `raliases'
> proxy_util.c:1016: error: structure has no member named `raliases'

plus this looks equally broken:

mod_proxy_http.c: In function 'process_proxy_header':
mod_proxy_http.c:998: warning: passing argument 2 of 'transform_hdrs[i].func' 
from incompatible pointer type

joe

Re: svn commit: r231355 - in /httpd/httpd/trunk/modules/proxy: mod_proxy.c mod_proxy.h mod_proxy_http.c

[Joe Orton]

On Thu, Aug 11, 2005 at 07:56:38PM +0100, Nick Kew wrote:
> Justin Erenkrantz wrote:
> 
> >>Fix ProxyPassReverse & family to work correctly in 
> >
> >
> >This commit broke the build.
> 
> Aaargh!  Careless cut&paste.  Sorry.  Fixed - thanks.

proxy_util.c: In function `ap_proxy_location_reverse_map':
proxy_util.c:1015: error: structure has no member named `raliases'
proxy_util.c:1016: error: structure has no member named `raliases'
proxy_util.c: In function `ap_proxy_cookie_reverse_map':
proxy_util.c:1053: error: structure has no member named `cookie_path_str'
proxy_util.c:1053: error: structure has no member named `cookie_path_str'
proxy_util.c:1053: warning: left-hand operand of comma expression has no effect
proxy_util.c:1059: error: structure has no member named `cookie_paths'
proxy_util.c:1060: error: structure has no member named `cookie_paths'
proxy_util.c:1070: error: structure has no member named `cookie_domain_str'
proxy_util.c:1070: error: structure has no member named `cookie_domain_str'
proxy_util.c:1070: warning: left-hand operand of comma expression has no effect
proxy_util.c:1076: error: structure has no member named `cookie_domains'
proxy_util.c:1077: error: structure has no member named `cookie_domains'