rse 98/02/23 00:27:41
Modified: src CHANGES
htdocs/manual new_features_1_3.html
htdocs/manual/mod mod_rewrite.html
src/modules/standard mod_rewrite.c mod_rewrite.h
. STATUS
Log:
Add the new RewriteMap types `rnd' and `int' to mod_rewrite to allow Apache to
be used as a Reverse Proxy (where the backend servers are choosen via a `rnd'
map) and to allow mass virtual hosting without <VirtualHost> sections (where
you have to fix the case of server names when translating the Host-Header to a
directory structure).
Together with the comitted ProxyPassReverse directive we now have solved
two things the users have asked in the past:
1. The ability to use Apache as a full-featured Reverse Proxy
2. The ability to do mass virtual hosting without <VirtualHost> sections.
For both topics we should write stand-alone documents (perhaps inside
htdocs/manual/misc/) because they are not trivial to do, even when we now have
the functionality ;-)
Submitted by: Ralf S. Engelschall
Reviewed by: Dean Gaudet, Ralf S. Engelschall
Revision Changes Path
1.659 +12 -0 apache-1.3/src/CHANGES
Index: CHANGES
===================================================================
RCS file: /export/home/cvs/apache-1.3/src/CHANGES,v
retrieving revision 1.658
retrieving revision 1.659
diff -u -r1.658 -r1.659
--- CHANGES 1998/02/23 08:04:05 1.658
+++ CHANGES 1998/02/23 08:27:32 1.659
@@ -1,5 +1,17 @@
Changes with Apache 1.3b6
+ *) Add new RewriteMap types: First, `rnd' which is equivalent to the `txt'
+ type but with a special post-processing for the looked-up value: It
+ parses it into alternatives according to `|' chars and then only one
+ particular alternative is choosen randomly (this is an essential
+ functionality needed for balancing between backend-servers when using
+ Apache as a Reverse Proxy. The looked up value here is a list of
+ servers). Second, `int' with the built-in maps named `tolower' and
+ `toupper' which can be used to map URL parts to a fixed case (this is an
+ essential feature to fix the case of server names when doing mass
+ virtual-hosting with the help of mod_rewrite instead of using
+ <VirtualHost> sections).
+
*) Add a new directive to mod_proxy similar to ProxyPass:
`ProxyPassReverse'.
This directive lets Apache adjust the URL in Location-headers on HTTP
redirect responses sent by the remote server. This way the virtually
1.46 +11 -0 apache-1.3/htdocs/manual/new_features_1_3.html
Index: new_features_1_3.html
===================================================================
RCS file: /export/home/cvs/apache-1.3/htdocs/manual/new_features_1_3.html,v
retrieving revision 1.45
retrieving revision 1.46
diff -u -r1.45 -r1.46
--- new_features_1_3.html 1998/02/23 08:04:08 1.45
+++ new_features_1_3.html 1998/02/23 08:27:35 1.46
@@ -562,6 +562,17 @@
as a reverse proxy to avoid by-passing the reverse proxy because of HTTP
redirects on the backend servers which stay behind the reverse proxy.
</LI>
+
+ <LI><STRONG>New map types for <SAMP><A
HREF="mod/mod_rewrite.html#RewriteMap">RewriteMap</A></SAMP> directive</STRONG>
+ <BR>
+ The new map types `Randomized Plain Text' and `Internal Function' were
added
+ to the <SAMP>RewriteMap</SAMP> directive of mod_rewrite. They provide two
+ new features: First, you now can randomly choose a sub-value from a value
+ which was looked-up in a rewriting map (which is useful when choosing
+ between backend servers in a Reverse Proxy situation). Second, you now can
+ translate URL parts to fixed (upper or lower) case (which is useful when
+ doing mass virtual hosting by the help of mod_rewrite).
+ </LI>
</UL>
<!--#include virtual="footer.html" -->
1.23 +103 -42 apache-1.3/htdocs/manual/mod/mod_rewrite.html
Index: mod_rewrite.html
===================================================================
RCS file: /export/home/cvs/apache-1.3/htdocs/manual/mod/mod_rewrite.html,v
retrieving revision 1.22
retrieving revision 1.23
diff -u -r1.22 -r1.23
--- mod_rewrite.html 1998/02/05 21:20:01 1.22
+++ mod_rewrite.html 1998/02/23 08:27:36 1.23
@@ -256,7 +256,7 @@
<A
HREF="directive-dict.html#Syntax"
REL="Help"
-><STRONG>Syntax:</STRONG></A> <CODE>RewriteMap</CODE> <EM>Mapname</EM>
<CODE>{txt,dbm,prg}:</CODE><EM>Filename</EM><BR>
+><STRONG>Syntax:</STRONG></A> <CODE>RewriteMap</CODE> <EM>MapName</EM>
<EM>MapType</EM><CODE>:</CODE><EM>MapSource</EM><BR>
<A
HREF="directive-dict.html#Default"
REL="Help"
@@ -267,82 +267,146 @@
><STRONG>Context:</STRONG></A> server config, virtual host<BR>
<P>
-The <TT>RewriteMap</TT> directive defines an external <EM>Rewriting Map</EM>
+The <TT>RewriteMap</TT> directive defines a <EM>Rewriting Map</EM>
which can be used inside rule substitution strings by the mapping-functions
-to insert/substitute fields through a key lookup.
+to insert/substitute fields through a key lookup. The source of this
+lookup can be of various types.
<P>
-The <A NAME="mapfunc"><EM>Mapname</EM></A> is the name of the map and will
+The <A NAME="mapfunc"><EM>MapName</EM></A> is the name of the map and will
be used to specify a mapping-function for the substitution strings of a
-rewriting rule via
+rewriting rule via one of the following constructs:
<BLOCKQUOTE><STRONG>
-<CODE>${</CODE> <EM>Mapname</EM> <CODE>:</CODE> <EM>LookupKey</EM>
+<CODE>${</CODE> <EM>MapName</EM> <CODE>:</CODE> <EM>LookupKey</EM>
+<CODE>}</CODE><BR>
+<CODE>${</CODE> <EM>MapName</EM> <CODE>:</CODE> <EM>LookupKey</EM>
<CODE>|</CODE> <EM>DefaultValue</EM> <CODE>}</CODE>
</STRONG></BLOCKQUOTE>
-When such a directive occurs the map <EM>Mapname</EM>
+When such a construct occurs the map <EM>MapName</EM>
is consulted and the key <EM>LookupKey</EM> is looked-up. If the key is
-found, the map-function directive is substituted by <EM>SubstValue</EM>. If
-the key is not found then it is substituted by <EM>DefaultValue</EM>.
+found, the map-function construct is substituted by <EM>SubstValue</EM>. If
+the key is not found then it is substituted by <EM>DefaultValue</EM> or
+the empty string if no <EM>DefaultValue</EM> was specified.
<P>
-The <EM>Filename</EM> must be a valid Unix filepath, containing one
-of the following formats:
+The following combinations for <EM>MapType</EM> and <EM>MapSource</EM>
+can be used:
-<OL>
-<LI><STRONG>Plain Text Format</STRONG>
+<UL>
+<LI><STRONG>Standard Plain Text</STRONG><BR>
+ MapType: <CODE>txt</CODE>, MapSource: Unix filesystem path to valid
regular file
<P>
- This is a ASCII file which contains either blank lines, comment lines
- (starting with a '#' character) or
+ This is the standard rewriting map feature where the <EM>MapSource</EM>
is
+ a plain ASCII file containing either blank lines, comment lines
(starting
+ with a '#' character) or pairs like the following - one per line.
<BLOCKQUOTE><STRONG>
<EM>MatchingKey</EM> <EM>SubstValue</EM>
</STRONG></BLOCKQUOTE>
- pairs - one per line. You can create such files either manually,
- using your favorite editor, or by using the programs
- <TT>mapcollect</TT> and <TT>mapmerge</TT> from the <TT>support</TT>
- directory of the <STRONG>mod_rewrite</STRONG> distribution.
<P>
- To declare such a map prefix, <EM>Filename</EM> with a <CODE>txt:</CODE>
- string as in the following example:
-
+ Example:
<P>
<table border=0 cellspacing=1 cellpadding=5 bgcolor="#f0f0f0">
<TR><TD><PRE>
-#
-# map.real-to-user -- maps realnames to usernames
-#
+##
+## map.txt -- rewriting map
+##
Ralf.S.Engelschall rse # Bastard Operator From Hell
-Dr.Fred.Klabuster fred # Mr. DAU
+Mr.Joe.Average joe # Mr. Average
</PRE></TD></TR>
</TABLE>
<P>
<table border=0 cellspacing=1 cellpadding=5 bgcolor="#f0f0f0">
<TR><TD><PRE>
-RewriteMap real-to-host txt:/path/to/file/map.real-to-user
+RewriteMap real-to-host txt:/path/to/file/map.txt
</PRE></TD></TR>
</TABLE>
<P>
-<LI><STRONG>DBM Hashfile Format</STRONG>
+<LI><STRONG>Randomized Plain Text</STRONG><BR>
+ MapType: <CODE>rnd</CODE>, MapSource: Unix filesystem path to valid
regular file
<P>
- This is a binary NDBM format file containing the
- same contents as the <EM>Plain Text Format</EM> files. You can create
- such a file with any NDBM tool or with the <TT>dbmmanage</TT> program
- from the <TT>support</TT> directory of the Apache distribution.
+ This is identical to the Standard Plain Text variant above but with a
special
+ post-processing feature: After looking up a value it is parsed according
+ to contained ``<TT>|</TT>'' characters which have the meaning of ``or''.
Or
+ in other words: they indicate a set of alternatives from which the actual
+ returned value is choosen randomly. Although this sounds crazy and
useless, it
+ was actually designed for load balancing in a reverse proxy situation
where
+ the looked up values are server names.
+ Example:
+<P>
+<table border=0 cellspacing=1 cellpadding=5 bgcolor="#f0f0f0">
+<TR><TD><PRE>
+##
+## map.txt -- rewriting map
+##
+
+static www1|www2|www3|www4
+dynamic www5|www6
+</PRE></TD></TR>
+</TABLE>
+
+<P>
+<table border=0 cellspacing=1 cellpadding=5 bgcolor="#f0f0f0">
+<TR><TD><PRE>
+RewriteMap servers rnd:/path/to/file/map.txt
+</PRE></TD></TR>
+</TABLE>
+
+<P>
+<LI><STRONG>Hash File</STRONG><BR>
+ MapType: <CODE>dbm</CODE>, MapSource: Unix filesystem path to valid
regular file
<P>
- To declare such a map prefix <EM>Filename</EM> with a <CODE>dbm:</CODE>
- string.
+ Here the source is a binary NDBM format file containing the same
contents
+ as a <EM>Plain Text</EM> format file, but in a special representation
+ which is optimized for really fast lookups. You can create such a file
with
+ any NDBM tool or with the following Perl script:
+ <P>
+ <table border=0 cellspacing=1 cellpadding=5 bgcolor="#f0f0f0">
+ <TR><TD><PRE>
+#!/path/to/bin/perl
+##
+## txt2dbm -- convert txt map to dbm format
+##
+
+($txtmap, $dbmmap) = @ARGV;
+open(TXT, "<$txtmap");
+dbmopen(%DB, $dbmmap, 0644);
+while (<TXT>) {
+ next if (m|^s*#.*| or m|^s*$|);
+ $DB{$1} = $2 if (m|^\s*(\S+)\s+(\S+)$|);
+}
+dbmclose(%DB);
+close(TXT)</PRE></TD></TR>
+ </TABLE>
+ <P>
+ <table border=0 cellspacing=1 cellpadding=5 bgcolor="#f0f0f0">
+ <TR><TD><PRE>$ txt2dbm map.txt map.db </PRE></TD></TR>
+ </TABLE>
+<P>
+<LI><STRONG>Internal Function</STRONG><BR>
+ MapType: <CODE>int</CODE>, MapSource: Internal Apache function
+ <P>
+ Here the source is an internal Apache function. Currently you cannot
+ create your own, but the following functions already exists:
+ <UL>
+ <LI><STRONG>toupper</STRONG>:<BR>
+ Converts the looked up key to all upper case.
+ <LI><STRONG>tolower</STRONG>:<BR>
+ Converts the looked up key to all lower case.
+ </UL>
<P>
-<LI><STRONG>Program Format</STRONG>
+<LI><STRONG>External Rewriting Program</STRONG><BR>
+ MapType: <CODE>prg</CODE>, MapSource: Unix filesystem path to valid
regular file
<P>
- This is a Unix executable, not a lookup file. To create it you can use
+ Here the source is a Unix program, not a map file. To create it you can
use
the language of your choice, but the result has to be a run-able Unix
- binary (i.e. either object-code or a script with the
+ executable (i.e. either object-code or a script with the
magic cookie trick '<TT>#!/path/to/interpreter</TT>' as the first line).
<P>
This program gets started once at startup of the Apache servers and then
@@ -367,7 +431,7 @@
</PRE></TD></TR>
</TABLE>
<P>
- <STRONG>But be very careful:</STRONG><BR>
+ But be very careful:<BR>
<OL>
<LI>``<EM>Keep the program simple, stupid</EM>'' (KISS), because
if this program hangs it will lead to a hang of the Apache server
@@ -376,10 +440,7 @@
This will cause a deadloop! Hence the ``<TT>$|=1</TT>'' in the above
example...
</OL>
- <P>
- To declare such a map prefix <EM>Filename</EM> with a <CODE>prg:</CODE>
- string.
-</OL>
+</UL>
The <TT>RewriteMap</TT> directive can occur more than once. For each
mapping-function use one <TT>RewriteMap</TT> directive to declare its
1.67 +147 -2 apache-1.3/src/modules/standard/mod_rewrite.c
Index: mod_rewrite.c
===================================================================
RCS file: /export/home/cvs/apache-1.3/src/modules/standard/mod_rewrite.c,v
retrieving revision 1.66
retrieving revision 1.67
diff -u -r1.66 -r1.67
--- mod_rewrite.c 1998/02/02 22:33:40 1.66
+++ mod_rewrite.c 1998/02/23 08:27:38 1.67
@@ -93,6 +93,7 @@
#include <time.h>
#include <signal.h>
#include <errno.h>
+#include <ctype.h>
#include <sys/types.h>
#include <sys/stat.h>
#ifdef WIN32
@@ -435,11 +436,17 @@
new = push_array(sconf->rewritemaps);
new->name = a1;
+ new->func = NULL;
if (strncmp(a2, "txt:", 4) == 0) {
new->type = MAPTYPE_TXT;
new->datafile = a2+4;
new->checkfile = a2+4;
}
+ else if (strncmp(a2, "rnd:", 4) == 0) {
+ new->type = MAPTYPE_RND;
+ new->datafile = a2+4;
+ new->checkfile = a2+4;
+ }
else if (strncmp(a2, "dbm:", 4) == 0) {
#ifdef HAS_NDBM_LIB
new->type = MAPTYPE_DBM;
@@ -455,6 +462,18 @@
new->datafile = a2+4;
new->checkfile = a2+4;
}
+ else if (strncmp(a2, "int:", 4) == 0) {
+ new->type = MAPTYPE_INT;
+ new->datafile = NULL;
+ new->checkfile = NULL;
+ if (strcmp(a2+4, "tolower") == 0)
+ new->func = rewrite_mapfunc_tolower;
+ else if (strcmp(a2+4, "toupper") == 0)
+ new->func = rewrite_mapfunc_toupper;
+ else if (sconf->state == ENGINE_ENABLED)
+ return pstrcat(cmd->pool, "RewriteMap: internal map not found:",
+ a2+4, NULL);
+ }
else {
new->type = MAPTYPE_TXT;
new->datafile = a2;
@@ -2499,6 +2518,52 @@
s->name, key);
}
}
+ else if (s->type == MAPTYPE_INT) {
+ if ((value = lookup_map_internal(r, s->func, key)) != NULL) {
+ rewritelog(r, 5, "map lookup OK: map=%s key=%s ->
val=%s",
+ s->name, key, value);
+ return value;
+ }
+ else {
+ rewritelog(r, 5, "map lookup FAILED: map=%s key=%s",
+ s->name, key);
+ }
+ }
+ else if (s->type == MAPTYPE_RND) {
+ if (stat(s->checkfile, &st) == -1) {
+ aplog_error(APLOG_MARK, APLOG_ERR, r->server,
+ "mod_rewrite: can't access text RewriteMap "
+ "file %s", s->checkfile);
+ rewritelog(r, 1,
+ "can't open RewriteMap file, see error log");
+ return NULL;
+ }
+ value = get_cache_string(cachep, s->name, CACHEMODE_TS,
+ st.st_mtime, key);
+ if (value == NULL) {
+ rewritelog(r, 6, "cache lookup FAILED, forcing new "
+ "map lookup");
+ if ((value =
+ lookup_map_txtfile(r, s->datafile, key)) != NULL) {
+ rewritelog(r, 5, "map lookup OK: map=%s key=%s[txt] "
+ "-> val=%s", s->name, key, value);
+ set_cache_string(cachep, s->name, CACHEMODE_TS,
+ st.st_mtime, key, value);
+ }
+ else {
+ rewritelog(r, 5, "map lookup FAILED: map=%s[txt] "
+ "key=%s", s->name, key);
+ return NULL;
+ }
+ }
+ else {
+ rewritelog(r, 5, "cache lookup OK: map=%s[txt] key=%s "
+ "-> val=%s", s->name, key, value);
+ }
+ value = select_random_value_part(r, value);
+ rewritelog(r, 5, "randomly choosen the subvalue `%s'",
value);
+ return value;
+ }
}
}
return NULL;
@@ -2605,7 +2670,87 @@
return pstrdup(r->pool, buf);
}
+static char *lookup_map_internal(request_rec *r,
+ char *(*func)(request_rec *, char *),
+ char *key)
+{
+ /* currently we just let the function convert
+ the key to a corresponding value */
+ return func(r, key);
+}
+
+static char *rewrite_mapfunc_toupper(request_rec *r, char *key)
+{
+ char *value, *cp;
+
+ for (cp = value = pstrdup(r->pool, key); cp != NULL && *cp != '\0'; cp++)
+ *cp = toupper(*cp);
+ return value;
+}
+
+static char *rewrite_mapfunc_tolower(request_rec *r, char *key)
+{
+ char *value, *cp;
+
+ for (cp = value = pstrdup(r->pool, key); cp != NULL && *cp != '\0'; cp++)
+ *cp = tolower(*cp);
+ return value;
+}
+
+static int rewrite_rand_init_done = 0;
+void rewrite_rand_init(void)
+{
+ if (!rewrite_rand_init_done) {
+ srand((unsigned)(getpid()));
+ rewrite_rand_init_done = 1;
+ }
+ return;
+}
+
+int rewrite_rand(int l, int h)
+{
+ int i;
+ char buf[50];
+
+ rewrite_rand_init();
+ sprintf(buf, "%.0f", (((double)rand()/RAND_MAX)*(h-l)));
+ i = atoi(buf)+1;
+ if (i < l) i = l;
+ if (i > h) i = h;
+ return i;
+}
+
+static char *select_random_value_part(request_rec *r, char *value)
+{
+ char *buf;
+ int n, i, k;
+
+ /* count number of distinct values */
+ for (n = 1, i = 0; value[i] != '\0'; i++)
+ if (value[i] == '|')
+ n++;
+
+ /* when only one value we have no option to choose */
+ if (n == 1)
+ return value;
+
+ /* else randomly select one */
+ k = rewrite_rand(1, n);
+
+ /* and grep it out */
+ for (n = 1, i = 0; value[i] != '\0'; i++) {
+ if (n == k)
+ break;
+ if (value[i] == '|')
+ n++;
+ }
+ buf = pstrdup(r->pool, &value[i]);
+ for (i = 0; buf[i] != '\0' && buf[i] != '|'; i++)
+ ;
+ buf[i] = '\0';
+ return buf;
+}
/*
@@ -2912,7 +3057,7 @@
continue;
}
}
- outp = ap_cpystrn(outp, cp, endp - outp);
+ outp = ap_cpystrn(outp, cp, endp - outp);
break;
}
return expanded ? pstrdup(r->pool, output) : str;
@@ -3010,7 +3155,7 @@
result = r->server->server_admin;
}
else if (strcasecmp(var, "SERVER_NAME") == 0) {
- result = get_server_name(r);
+ result = get_server_name(r);
}
else if (strcasecmp(var, "SERVER_PORT") == 0) {
ap_snprintf(resultbuf, sizeof(resultbuf), "%u", get_server_port(r));
1.37 +13 -3 apache-1.3/src/modules/standard/mod_rewrite.h
Index: mod_rewrite.h
===================================================================
RCS file: /export/home/cvs/apache-1.3/src/modules/standard/mod_rewrite.h,v
retrieving revision 1.36
retrieving revision 1.37
diff -u -r1.36 -r1.37
--- mod_rewrite.h 1998/01/07 16:46:56 1.36
+++ mod_rewrite.h 1998/02/23 08:27:39 1.37
@@ -184,6 +184,8 @@
#define MAPTYPE_TXT 1<<0
#define MAPTYPE_DBM 1<<1
#define MAPTYPE_PRG 1<<2
+#define MAPTYPE_INT 1<<3
+#define MAPTYPE_RND 1<<4
#define ENGINE_DISABLED 1<<0
#define ENGINE_ENABLED 1<<1
@@ -228,8 +230,10 @@
char *datafile; /* filename for map data files */
char *checkfile; /* filename to check for map existence */
int type; /* the type of the map */
- int fpin; /* in filepointer for program maps */
- int fpout; /* out filepointer for program maps */
+ int fpin; /* in file pointer for program maps */
+ int fpout; /* out file pointer for program maps */
+ char *(*func)(request_rec *, /* function pointer for internal maps */
+ char *);
} rewritemap_entry;
typedef struct {
@@ -385,13 +389,19 @@
static char *expand_tildepaths(request_rec *r, char *uri);
static void expand_map_lookups(request_rec *r, char *uri, int uri_len);
- /* DBM hashfile support functions */
+ /* rewrite map support functions */
static char *lookup_map(request_rec *r, char *name, char *key);
static char *lookup_map_txtfile(request_rec *r, char *file, char *key);
#if HAS_NDBM_LIB
static char *lookup_map_dbmfile(request_rec *r, char *file, char *key);
#endif
static char *lookup_map_program(request_rec *r, int fpin, int fpout, char
*key);
+static char *lookup_map_internal(request_rec *r, char *(*func)(request_rec
*r, char *key), char *key);
+static char *rewrite_mapfunc_toupper(request_rec *r, char *key);
+static char *rewrite_mapfunc_tolower(request_rec *r, char *key);
+static char *select_random_value_part(request_rec *r, char *value);
+static void rewrite_rand_init(void);
+static int rewrite_rand(int l, int h);
/* rewriting logfile support */
static void open_rewritelog(server_rec *s, pool *p);
1.162 +0 -4 apache-1.3/STATUS
Index: STATUS
===================================================================
RCS file: /export/home/cvs/apache-1.3/STATUS,v
retrieving revision 1.161
retrieving revision 1.162
diff -u -r1.161 -r1.162
--- STATUS 1998/02/23 08:04:13 1.161
+++ STATUS 1998/02/23 08:27:40 1.162
@@ -30,10 +30,6 @@
Status: Dean +1, Martin +0 (duplicates /server-info?server),
Alexei -1 (shared lib concerns)
- * Ralf's [PATCH] New RewriteMap types for mod_rewrite
- <[EMAIL PROTECTED]>
- Status: Ralf +1
-
Concepts:
* Dean's [PRE-PATCH] expanding ap_snprintf()
