Re: [VOTE] move all mod_*.h with public APIs to ./include folder
Graham Leggett wrote: Guenter Knauf wrote: in order to simplify future configuration, and most important to have same include path structure with both in-tree and installed compilations I think it makes sense to move all mod_*.h headers with public APIs to the common ./include folder. Current votes: +1 wrowe, fuankg +1. Regards, Graham -- -1 Silver W.Zuse
Re: [VOTE] move all mod_*.h with public APIs to ./include folder
Guenter Knauf wrote: in order to simplify future configuration, and most important to have same include path structure with both in-tree and installed compilations I think it makes sense to move all mod_*.h headers with public APIs to the common ./include folder. +1 since it simplifies my life. Chris. -- GPG Key ID: 366A375B GPG Key Fingerprint: 485E 5041 17E1 E2BB C263 E4DE C8E3 FA36 366A 375B
Re: [VOTE] move all mod_*.h with public APIs to ./include folder
Guenter Knauf wrote: in order to simplify future configuration, and most important to have same include path structure with both in-tree and installed compilations I think it makes sense to move all mod_*.h headers with public APIs to the common ./include folder. Current votes: +1 wrowe, fuankg +1. Regards, Graham -- smime.p7s Description: S/MIME Cryptographic Signature
[VOTE] move all mod_*.h with public APIs to ./include folder
Hi, in order to simplify future configuration, and most important to have same include path structure with both in-tree and installed compilations I think it makes sense to move all mod_*.h headers with public APIs to the common ./include folder. Current votes: +1 wrowe, fuankg please vote, and also explain a reason if you vote against. Guenter.
Re: svn commit: r646445 - in /httpd/httpd/trunk/modules/aaa:mod_auth_basic.c mod_auth_digest.c
Hi, >Yes, and in fact, there's no need for a function call at all, you > can just look up the provider list directly in that case. Much cleaner, > except for all the config magic changes (as discussed by others); > I can more or less guarantee that I'll break the Windows and/or the > Netware builds trying to get mod_auth.h to be included. :-) But > a +1 from me on the idea, if others are willing to do the repair > work afterwards for those platforms. no prob for NetWare (I'll change that if we dont move the mod_*.h), so my +1 for this change. Guenter.
Re: Solaris sed based apache filtering module (mod_sed)
> Furthermore from a first glance of mod_sed I would guess that it runs into
> trouble with large static files. So could you test mod_sed with one of
> the simple cases above and a 3 GB file and monitor httpd's memory consumption
> in comparison to mod_substitute?
I did testing with large files. Memory consumption was not a problem with large
files but response time was a big problem.
The performance of mod_sed was terrible with large files. I was not buffering
the output data in sed code which was causing too many buckets creation. This
didn't cause much performance drop for small files but for big files, it made
a large difference. I introduced the 4K buffer and that's solved the problem.
For a 3M file :
download time : 255 seconds
After my changes :
download time : 0.5 seconds
I also tried with 300M file and response time/memory consumption was not bad.
I have revised the mod_sed code and new code can be retrieve from mercurial :
$ hg clone ssh://[EMAIL PROTECTED]/hg/webstack/mod_sed
It should soon be visible on http url too.
http://src.opensolaris.org/source/xref/webstack/mod_sed/
Diffs are attached at the bottom. With these new changes here are the
performance comparison results. The changes improved performance in general.
Performance result with changes in mod_sed and trunk version of mod_substitute.
Apache : 2.2.6 64 bit worker mpm.
OS : Solaris snv_79
-
test2.html : 500 substitution s/hello/hi/g.
Requests Per Second (RPS) as measured by ab for s/hello/hi/ substitution.
Enabled CPUs : 4
testname Concur MaxReq mod_sedmod_line_edit mod_substitute
mod_substitute
RPS RPS RPS (with
n flag) RPS
test2.html 400 40 2788 1363 2557
2811
Summary :
On simple string substitution (s/hello/hi/) mod_sed perform better than
mod_substitue without [n] parameter and it is close to mod_substitute with [n]
parameter.
-
Test case summary : test for simple regular expression.
Regular expression used : OutputSed 's/.*one.*two/1 2/'
: Substitute "s/.*one.*two/1 2/"
String : hello one 1 two 2 three 3 four 4 five 5.
Enabled CPUs : 4
test22.html : 500 lines containing the above string (Result in 500
substitution).
testname Concur MaxReq mod_sedmod_substitute
RPS RPS
test22.html 40040 21681688
Summary :
With ordinary regular expression, mod_sed performed better than
mod_substitute.
-
Test case summary : pattern capture
Regular expression used : OutputSed
's/.*\(one\).*\(two\).*\(three\).*\(four\).*\(five\).*/\5 \4 \3 \2 \1/'
: Substitute
"s/.*(one).*(two).*(three).*(four).*(five).*/$5 $4 $3 $2 $1/"
String : hello one 1 two 2 three 3 four 4 five 5.
Enabled CPUs : 4
test12.html : 500 lines containing the above string (Result in 500
substitution).
testname Concur MaxReq mod_sedmod_substitute
RPS RPS
test12.html 40040 15233135
Summary :
With pattern capture, mod_substitute performed much better than mod_sed.
-
Regards,
Basant.
diff -r b16bfdb8759b libsed.h
--- a/libsed.h Fri Apr 11 15:00:36 2008 -0700
+++ b/libsed.h Fri Apr 11 15:04:44 2008 -0700
@@ -133,6 +133,10 @@ struct sed_eval_s {
char *gbend;
char *lcomend;
+char *outbuf;
+char *curoutbuf;
+char *outbend;
+
apr_file_t*fcode[NWFILES];
sed_reptr_t*abuf[SED_ABUFSIZE];
sed_reptr_t**aptr;
diff -r b16bfdb8759b sed1.c
--- a/sed1.cFri Apr 11 15:00:36 2008 -0700
+++ b/sed1.cFri Apr 11 15:04:37 2008 -0700
@@ -73,6 +73,8 @@ static apr_status_t command(sed_eval_t *
step_vars_storage *step_vars);
static void wline(sed_eval_t *eval, char *buf, int sz);
static void arout(sed_eval_t *eval);
+static void sed_write(sed_eval_t *eval, char *buf, int sz);
+static void sed_flush_output_buffer(sed_eval_t *eval, char* buf, int sz);
static void eval_errf(sed_eval_t *eval, const char *fmt, ...)
{
@@ -124,6 +126,13 @@ apr_status_t sed_reset_eval(sed_eval_t *
eval->genbuf = apr_palloc(eval->pool, LBSIZE + 1);
eval->gbend = eval->genbuf + LBSIZE;
}
+if (eval->outbuf == NULL) {
+int bufsize = 4 * LBSIZE;
+eval->outbuf = apr_palloc(eval->pool, bufsize + 1);
+eval->outbend = eval->outbuf + bufsize;
+}
+eval->curoutbuf = eval->outbuf;
+
eval->lspend = eval->linebuf;
eval->hspend = eval->holdbuf;
eval->lcomend = &eval->genbuf[
Re: svn commit: r646445 - in /httpd/httpd/trunk/modules/aaa: mod_auth_basic.c mod_auth_digest.c
Ruediger Pluem wrote: So we are using the optional function only because we do not have the defines for AUTHN_PROVIDER_GROUP and the version number around in request.c. If they would be around it would be quite fine to call ap_list_provider_names directly from there, correct? Yes, and in fact, there's no need for a function call at all, you can just look up the provider list directly in that case. Much cleaner, except for all the config magic changes (as discussed by others); I can more or less guarantee that I'll break the Windows and/or the Netware builds trying to get mod_auth.h to be included. :-) But a +1 from me on the idea, if others are willing to do the repair work afterwards for those platforms. Chris. -- GPG Key ID: 366A375B GPG Key Fingerprint: 485E 5041 17E1 E2BB C263 E4DE C8E3 FA36 366A 375B
Re: svn commit: r646281 - in /httpd/httpd/trunk: CHANGES include/http_protocol.h modules/http/http_filters.c
On Apr 11, 2008, at 8:46 PM, Graham Leggett wrote: Roy T. Fielding wrote: New features belong in modules so that they don't get compiled into the server by those of us who don't want that feature. I don't recall why or how KeptBodySize made it into http_filter (where it does not belong), but I am quite certain that the core will not be parsing HTTP body content any time soon. So, either find another way or remove the feature as well. Done (r647263). Thanks Graham, Roy
Re: svn commit: r646281 - in /httpd/httpd/trunk: CHANGES include/http_protocol.h modules/http/http_filters.c
Roy T. Fielding wrote: New features belong in modules so that they don't get compiled into the server by those of us who don't want that feature. I don't recall why or how KeptBodySize made it into http_filter (where it does not belong), but I am quite certain that the core will not be parsing HTTP body content any time soon. So, either find another way or remove the feature as well. Done (r647263). Regards, Graham -- smime.p7s Description: S/MIME Cryptographic Signature
Re: Ship 1.3.0 apr in httpd 2.2.9
> So if APR ships 1.3.0 - would we be prepared to accept 1.3.0 as the new > minimum version, modulo the minor MMN bump? +1 BTW. wouldnt it make sense to introduce a similar versioning as we now use with httpd? I mean that current trunk 1.3.0-unstable becomes 1.4.0-stable, and we start on 1.5.0-unstable? Guenter.
Re: Ship 1.3.0 apr in httpd 2.2.9
On Fri, Apr 11, 2008 at 4:39 PM, William A. Rowe, Jr. <[EMAIL PROTECTED]> wrote: > So if APR ships 1.3.0 - would we be prepared to accept 1.3.0 as the new > minimum version, modulo the minor MMN bump? +1. -- justin
Re: svn commit: r646445 - in /httpd/httpd/trunk/modules/aaa:mod_auth_basic.c mod_auth_digest.c
Guenter Knauf wrote: We recently found that at least for Win32 install we missed to pack a bunch of mod_*.h includes with the distros (Bill fixed this then, and this reminds me that I should do same for NetWare...); however I'm asking me why we dont just put those mod_*.h files which are known to be non-private to one module only into the common ./include dir rather than keeping them in the module's folders? Wouldnt that simplify all installs + no need for config.m4 tweaks since the common ./include dir is anyway already with the include path? unpublished internal API --- modules/foo/foo.h external API -- include/mod_foo.h +1
Re: Ship 1.3.0 apr in httpd 2.2.9
On 11 Apr 2008, at 15:39, William A. Rowe, Jr. wrote: So if APR ships 1.3.0 - would we be prepared to accept 1.3.0 as the new minimum version, modulo the minor MMN bump? -0.9 People who want 1.3 can get it for themselves. People who want packages want stable. And there's stuff in 1.3 that falls way short of what wants to go into a packaged products. Example: apr_dbd_freetds is wide open to SQL injection if used as you would use AN Other driver, because of the prepared statements implementation. -- Nick Kew
Re: Ship 1.3.0 apr in httpd 2.2.9
On 4/11/08 10:39 AM, "William A. Rowe, Jr." <[EMAIL PROTECTED]> wrote: > So if APR ships 1.3.0 - would we be prepared to accept 1.3.0 as the new > minimum version, modulo the minor MMN bump? +1 -- Brian Akins Chief Operations Engineer Turner Digital Media Technologies
Ship 1.3.0 apr in httpd 2.2.9
So here's a thought from Davi's^WColm's APR talk in speaking with
chipig, jorton and others...
We have several inheritance quirks addressed by 1.3.0 apr, and the
new memcache, ssl and other features are present in 1.3. The features
in APR[-util] 1.3 would be a definite benefit, today.
When we add a few new functions to HTTPD - we bump the MMN and then
guarantee that these functions are binary compatible from that given
MMN bump ... all older builds against that HTTPD are binary forward
compatible until we move along to 2.4.
In APR - we guarantee backwards binary compat... apr 1.{any} will
continue to behave once we we bump the version minor.
Combine these, and it seems that with a Minor MMN bump, we can throw
the switch to require APR 1.3.x (or later) for the remaining life
of httpd 2.2.
The other side effects of requiring apr 1.3 over 1.2.x all seem to be
build related and really have no binary compatibility issues whatsoever.
So if APR ships 1.3.0 - would we be prepared to accept 1.3.0 as the new
minimum version, modulo the minor MMN bump?
Bill
Re: svn commit: r646445 - in /httpd/httpd/trunk/modules/aaa:mod_auth_basic.c mod_auth_digest.c
Hi Chris, >True -- an alternative might be to do the following: > - #define AUTHN/Z_PROVIDER_VERSION "0" in mod_auth.h > - change all the authn/z modules to use #defines in place of hard-coded > "0" provider versions (maybe not a bad thing to do anyway) > - #include mod_auth.h in request.c (mostly involves lots of fiddling > with config.m4 files and the like, for each platform) > - eliminate the optional function call entirely >I worked on this approach but concluded people might not appreciate > all the changes to the config files and so forth, although the actual > code per se is a little simpler and cleaner. If this is preferred by > a majority, though, I'm happy to implement it this way instead. Thoughts? We recently found that at least for Win32 install we missed to pack a bunch of mod_*.h includes with the distros (Bill fixed this then, and this reminds me that I should do same for NetWare...); however I'm asking me why we dont just put those mod_*.h files which are known to be non-private to one module only into the common ./include dir rather than keeping them in the module's folders? Wouldnt that simplify all installs + no need for config.m4 tweaks since the common ./include dir is anyway already with the include path? Any thoughts about such a change? is there any reason why they should stay in the module folders? Guenter.
Re: svn commit: r645978 - in /httpd/httpd/trunk/modules: cache/ap_socache.h cache/mod_socache_dbm.c cache/mod_socache_dc.c cache/mod_socache_memcache.c cache/mod_socache_shmcb.c ssl/ssl_scache.c (fwd)
Author: jorton
Date: Tue Apr 8 08:47:48 2008
New Revision: 645978
URL: http://svn.apache.org/viewvc?rev=645978&view=rev
Log:
Adjust socache init interface to take sizing hints, and namespace tag
for memcache:
* modules/cache/ap_socache.h (struct ap_socache_hints): New structure.
Change init callback to take namespace string and hints structure pointer.
* modules/cache/mod_socache_dc.c (socache_dc_init): Adjust accordingly.
* modules/cache/mod_socache_dbm.c (struct ap_socache_instance_t): Rename
timeout field to expiry_interval.
(socache_dbm_init, socache_dbm_create): Take expiry interval from
hints rather than hard-code to 30.
(socache_dbm_expire): Update for timeout field rename.
* modules/cache/mod_socache_shmcb.c (socache_shmcb_init): Adjust for
hints and namespace; adjust subcache index sizing heuristics to use
passed-in hints.
* modules/cache/mod_socache_memcache.c (struct ap_socache_instance_t):
Add tag, taglen fields.
(socache_mc_init): Store the passed-in namespace in instance
structure.
(mc_session_id2sz): Adjust to not take context, use configured
tag as string prefix, and not use a return value.
(socache_mc_store, socache_mc_retrieve, socache_mc_remove):
Adjust for mc_session_id2sz interface changes.
* modules/ssl/ssl_scache.c (ssl_scache_init): Pass namespace and hints
to socache provider init function.
Modified:
httpd/httpd/trunk/modules/cache/ap_socache.h
httpd/httpd/trunk/modules/cache/mod_socache_dbm.c
httpd/httpd/trunk/modules/cache/mod_socache_dc.c
httpd/httpd/trunk/modules/cache/mod_socache_memcache.c
httpd/httpd/trunk/modules/cache/mod_socache_shmcb.c
httpd/httpd/trunk/modules/ssl/ssl_scache.c
Modified: httpd/httpd/trunk/modules/cache/mod_socache_memcache.c
URL:
http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/cache/mod_socache_memcache.c?rev=645978&r1=645977&r2=645978&view=diff
==
--- httpd/httpd/trunk/modules/cache/mod_socache_memcache.c (original)
+++ httpd/httpd/trunk/modules/cache/mod_socache_memcache.c Tue Apr 8 08:47:48
2008
@@ -164,22 +167,21 @@
/* noop. */
}
-static char *mc_session_id2sz(const unsigned char *id, unsigned int idlen,
- char *str, int strsize)
+static void mc_session_id2sz(ap_socache_instance_t *ctx,
+ const unsigned char *id, unsigned int idlen,
+ char *buf, apr_size_t buflen)
{
+apr_size_t maxlen = (buflen - ctx->taglen) / 2;
What happens if ctx-taglen > buflen?
char *cp;
int n;
-int maxlen = (strsize - MC_TAG_LEN)/2;
-cp = apr_cpystrn(str, MC_TAG, MC_TAG_LEN);
+cp = apr_cpystrn(buf, ctx->tag, ctx->taglen);
Se above. IMHO we should be defensive here and ensure that the last
parameter to apr_cpystrn is not larger than the buffer size.
for (n = 0; n < idlen && n < maxlen; n++) {
apr_snprintf(cp, 3, "%02X", (unsigned) id[n]);
cp += 2;
}
*cp = '\0';
-
-return str;
}
static apr_status_t socache_mc_store(ap_socache_instance_t *ctx, server_rec *s,
@@ -188,21 +190,16 @@
unsigned char *ucaData, unsigned int
nData)
{
char buf[MC_KEY_LEN];
-char *strkey = NULL;
apr_status_t rv;
-strkey = mc_session_id2sz(id, idlen, buf, sizeof(buf));
-if(!strkey) {
-ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, s, "scache_mc: Key generation
borked.");
-return APR_EGENERAL;
-}
+mc_session_id2sz(ctx, id, idlen, buf, sizeof(buf));
-rv = apr_memcache_set(ctx->mc, strkey, (char*)ucaData, nData, timeout, 0);
+rv = apr_memcache_set(ctx->mc, buf, (char*)ucaData, nData, timeout, 0);
if (rv != APR_SUCCESS) {
ap_log_error(APLOG_MARK, APLOG_CRIT, rv, s,
"scache_mc: error setting key '%s' "
- "with %d bytes of data", strkey, nData);
+ "with %d bytes of data", buf, nData);
return rv;
}
@@ -217,21 +214,14 @@
{
apr_size_t der_len;
char buf[MC_KEY_LEN], *der;
-char *strkey = NULL;
apr_status_t rv;
-strkey = mc_session_id2sz(id, idlen, buf, sizeof(buf));
-
-if (!strkey) {
-ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, s,
- "scache_mc: Key generation borked.");
-return APR_EGENERAL;
-}
+mc_session_id2sz(ctx, id, idlen, buf, sizeof(buf));
/* ### this could do with a subpool, but _getp looks like it will
* eat memory like it's going out of fashion anyway. */
-rv = apr_memcache_getp(ctx->mc, p, strkey,
+rv = apr_memcache_getp(ctx->mc, p, buf,
&der, &der_len, NULL);
if (rv) {
if (rv != APR_NOTFOUND) {
Modified: httpd/httpd/trunk/modules/cache/mod_socache_shmcb.c
URL:
http://svn.apache.org/viewvc/httpd/httpd/trunk/modules/cache/mod_socache_shmcb.c?rev=645978&r
Re: svn commit: r646445 - in /httpd/httpd/trunk/modules/aaa: mod_auth_basic.c mod_auth_digest.c
On 11.04.2008 04:00, Chris Darroch wrote: Ruediger Pluem wrote: The implementation of the function. IMHO it must be the same in all modules. Otherwise it depends on the module load order what gets called and done. True -- an alternative might be to do the following: - #define AUTHN/Z_PROVIDER_VERSION "0" in mod_auth.h - change all the authn/z modules to use #defines in place of hard-coded "0" provider versions (maybe not a bad thing to do anyway) - #include mod_auth.h in request.c (mostly involves lots of fiddling with config.m4 files and the like, for each platform) - eliminate the optional function call entirely I worked on this approach but concluded people might not appreciate all the changes to the config files and so forth, although the actual code per se is a little simpler and cleaner. If this is preferred by a majority, though, I'm happy to implement it this way instead. Thoughts? So we are using the optional function only because we do not have the defines for AUTHN_PROVIDER_GROUP and the version number around in request.c. If they would be around it would be quite fine to call ap_list_provider_names directly from there, correct? Regards Rüdiger
Re: svn commit: r646582 - /httpd/httpd/trunk/modules/ldap/util_ldap.c
On 11.04.2008 01:34, Brad Nicholes wrote: It is protected by the code at the beginning of util_ldap_post_config() that calls apr_pool_userdata_get() and checks a user data tag. If the tag is empty then this is the first time that the function was called. It then puts something in the tag and returns. The next time it reads the user data tag, it finds what it put there the first time. Now it knows that this is the second time and goes ahead with initializations. This is actually the preferred way to do it rather than using a static variable. Static global variables only work on platforms that have process separation for data areas. NetWare *isn't* one of those platforms. A global variable is global to everything running on the box. That was why I had to make the other NetWare specific changes in ldap_rebind.c. Static globals are bad. :( As always thanks for the detailed explanation and the hints for solving this kind of problem in a platform independent manner. Now I am all fine with this. Regards Rüdiger
