Re: Report on mod_accel and mod_deflate
On Thu, 20 Dec 2001, Jeremy Howard wrote: Note that mod_accel can also be called by utilising the mod_rewrite [P] directive, just like with mod_proxy. If I put [P] in a RewriteRule, how does Apache know whether I want it to use mod_proxy or mod_accel? AccelSet* adds X-* headers to the request to the backend. This is useful to know what the original request details were. In ftp://ftp.lexa.ru/pub/apache-rus/contrib/ (where I have been told to download mod_accel/mod_deflate from before), I see another file called mod_realip-1.0.tar.gz just released one week ago. From looking at the keywords in the documentation, it looks like a module to be installed on the backend httpd that will parse these X-* headers to retrieve the original IP address. By default only text/html is compressed. I think it's safe to compress text/plain by default, too; I've never seen any browser problems with compressed text/plain (only text/js and text/css).
Re: Report on mod_accel and mod_deflate
Philip Mak wrote: In ftp://ftp.lexa.ru/pub/apache-rus/contrib/ (where I have been told to download mod_accel/mod_deflate from before), I see another file called mod_realip-1.0.tar.gz just released one week ago. From looking at the keywords in the documentation, it looks like a module to be installed on the backend httpd that will parse these X-* headers to retrieve the original IP address. Correct. This is a simple module that fills out $R-connection-ip with the X- header added by mod_deflate.
Re: Report on mod_accel and mod_deflate
On Thu, 20 Dec 2001, Philip Mak wrote: On Thu, 20 Dec 2001, Jeremy Howard wrote: Note that mod_accel can also be called by utilising the mod_rewrite [P] directive, just like with mod_proxy. Even more. You can use mod_accel/mod_rewrite/mod_include: RewriteRule ^/one.html$ http://backend/$1 [P] !--#include virtual=/one.html?arg=some -- If I put [P] in a RewriteRule, how does Apache know whether I want it to use mod_proxy or mod_accel? mod_proxy and mod_accel can work together expect one case - mod_rewrite [P]. You can disable mod_accel's [P] with --without-mod_rewrite mod_accel configure parameter. AccelSet* adds X-* headers to the request to the backend. This is useful to know what the original request details were. In ftp://ftp.lexa.ru/pub/apache-rus/contrib/ (where I have been told to download mod_accel/mod_deflate from before), I see another file called mod_realip-1.0.tar.gz just released one week ago. From looking at the keywords in the documentation, it looks like a module to be installed on the backend httpd that will parse these X-* headers to retrieve the original IP address. Yes, it can set IP using X-Real-IP header (default) or X-Forwarde-For header. Also mod_realip set IP in one of three phases - postread, header and fixups. By default only text/html is compressed. I think it's safe to compress text/plain by default, too; I've never seen any browser problems with compressed text/plain (only text/js and text/css). There is no text/js type - application/x-javascript instead. Macromedia FlashPlayer 4.x-5.x doesn't understand compressed text files received via loadVariables() function. These files can have any type but usually they have text/plain. If you site doesn't use such flash movies - you can safely compress text/plain: DeflateTypes text/plain Igor Sysoev
Re: Report on mod_accel and mod_deflate
On Thu, 20 Dec 2001, Jeremy Howard wrote: Thank you, Jeremy. Here is some comments. OK, enough evangelism. On to practical matters. Many of the following snippets and comments were provided to me by Igor in private email, so thank him, not me. Firstly, here's how to compile everything (including mod_ssl on the front-end) and install it into /usr/local/apache_accel: tar -zxvf mod_accel-1.0.7.tar.gz tar -zxvf mod_deflate-1.0.9.tar.gz tar -zxvf mm-1.1.3.tar.gz cd mm-1.1.3 ./configure --disable-shared make cd ../mod_ssl-2.8.5-1.3.22 ./configure --with-apache=../apache_1.3.22_accel/ \ --with-crt=/INSERT_APPROPRIATE_PATH.crt \ --with-key=/INSERT_APPROPRIATE_PATH.key cd ../mod_accel-1.0.7 ./configure --with-apache=../apache_1.3.22_accel/ \ --with-eapi=../mod_ssl-2.8.5-1.3.22/pkg.eapi/ make You don't need to use mod_ssl but you need EAPI distributed with mod_ssl. So if you don't use mod_ssl you need untar it only and set path to EAPI: cd ../mod_accel-1.0.7 ./configure --with-apache=../apache_1.3.22_accel/ \ --with-eapi=../mod_ssl-2.8.5-1.3.22/pkg.eapi/ make But if use mod_ssl then you can not to specify EAPI path - mod_ssl's configure set it on Apache source itself: cd ../mod_accel-1.0.7 ./configure --with-apache=../apache_1.3.22_accel/ make cd ../mod_deflate-1.0.9 ./configure --with-apache=../apache_1.3.22_accel/ make cd ../apache_1.3.22_accel/ EAPI_MM=../mm-1.1.3 SSLBASE=SYSTEM ./configure --enable-rule=EAPI \ --prefix=/usr/local/apache_accel --disable-rule=EXPAT \ --enable-module=most \ --enable-module=rewrite \ --activate-module=src/modules/accel/libaccel.a \ --activate-module=src/modules/extra/mod_deflate.o make make install mkdir /var/accelcache chown nobody:nobody /var/accelcache/ /usr/local/apache_accel/bin/apachectl startssl Now, in /usr/local/apache_accel/conf/httpd.conf, append the following: AccelCacheRoot /var/accelcache 1 AccelNoCacheon AccelPass/mail/http://127.0.0.1:8080/mail/ AccelWaitBeforeBodyRead 100 AccelUnlinkNoCached off AccelSetXHost on AccelSetXRealIP on AccelSetXURL on This creates an HTTP accelerator without any caching. I don't know much about the caching directives and haven't tried them out, so I can't provide any guidance there. The AccelCacheRoot directive sets the number of levels of subdirectories in the cache. 16 first level subdirectories named 0 .. f and 256 second level subdirectories named 00 .. ff are used by default (i.e. AccelCacheRoot path 1 2). Since I'm not caching, my config above does not specify any directory hashing (i.e. AccelCacheRoot path 1). AccelCacheRoot allow to set up to 3 levels of two type subdirectories - named 0 .. f and 00 .. ff. Also AccelCacheRoot has parameter 'noauto' that specify do not create automaticaly all nested subdirectories. They shoud be created before with create_cache script. Here's the config for mod_deflate: DeflateEnable On DeflateMinLength 3000 DeflateCompLevel 1 DeflateProxied Off This is default settings of DeflateCompLevel and DeflateProxied. DeflateHTTP 1.0 DeflateDisableRange MSIE 4. DeflateProxied Off means that anything with a 'Via' header is not proxied--Igor says that some proxies are broken so this is best to be safe. is not compressed And MSIE 4 apparently is a bit broken too. By default only text/html is compressed. DeflateCompLevel 1 provides adequate compression for text. Igor Sysoev
Re: Report on mod_accel and mod_deflate
On Thu, 20 Dec 2001, Thomas Eibner wrote: On Thu, Dec 20, 2001 at 07:04:50AM -0500, Philip Mak wrote: AccelSet* adds X-* headers to the request to the backend. This is useful to know what the original request details were. In ftp://ftp.lexa.ru/pub/apache-rus/contrib/ (where I have been told to download mod_accel/mod_deflate from before), I see another file called mod_realip-1.0.tar.gz just released one week ago. From looking at the keywords in the documentation, it looks like a module to be installed on the backend httpd that will parse these X-* headers to retrieve the original IP address. I basically wrote something similar for the users of mod_proxy_add_forward 2-3 months ago, it's called mod_rpaf and sets r-connection-remote_ip and moved the X-Host: headers into the incoming Host: header and then updates the vhost structure so virtualhosting via mod_proxy_add_forward works. The module can be found at: http://stderr.net/apache/rpaf/ There is one drawback in setting r-connection-remote_addr. If you want to disable direct access to backend then you can't do it with mod_access - you need to configure firewall. Igor Sysoev
Re: Report on mod_accel and mod_deflate
On Thu, 20 Dec 2001, Igor Sysoev wrote: On Thu, 20 Dec 2001, Thomas Eibner wrote: On Thu, Dec 20, 2001 at 07:04:50AM -0500, Philip Mak wrote: AccelSet* adds X-* headers to the request to the backend. This is useful to know what the original request details were. In ftp://ftp.lexa.ru/pub/apache-rus/contrib/ (where I have been told to download mod_accel/mod_deflate from before), I see another file called mod_realip-1.0.tar.gz just released one week ago. From looking at the keywords in the documentation, it looks like a module to be installed on the backend httpd that will parse these X-* headers to retrieve the original IP address. I basically wrote something similar for the users of mod_proxy_add_forward 2-3 months ago, it's called mod_rpaf and sets r-connection-remote_ip and moved the X-Host: headers into the incoming Host: header and then updates the vhost structure so virtualhosting via mod_proxy_add_forward works. The module can be found at: http://stderr.net/apache/rpaf/ There is one drawback in setting r-connection-remote_addr. If you want to disable direct access to backend then you can't do it with mod_access - you need to configure firewall. Sorry, I did not notice RPAFproxy_ips. Igor Sysoev
Re: Report on mod_accel and mod_deflate
Jeremy Howard wrote: Igor Sysoev mentioned recently on this list that he has written a module called 'mod_accel' that provides a caching HTTP accelerator, as well as a mod_gzip replacement called 'mod_deflate'. These modules are both used on Kaspersky labs' busy sites, as well as at the popular portal http://www.rambler.ru. mod_deflate is used at around 700 sites. Igor, sorry to tell you but httpd-2.0 has introduced mod_deflate as a replacement for mod_gzip, so you are going to have a lot of confused users when httpd 2.0 hits the streets. I suggest that you raise this issue on the httpd dev list before it's too late. May be it's too late already, since there was already a long discussion about why it should be called deflate and not gzip (check the archives). The modules is located here: httpd-2.0/modules/experimental/mod_deflate.c _ Stas Bekman JAm_pH -- Just Another mod_perl Hacker http://stason.org/ mod_perl Guide http://perl.apache.org/guide mailto:[EMAIL PROTECTED] http://ticketmaster.com http://apacheweek.com http://singlesheaven.com http://perl.apache.org http://perlmonth.com/
Re: Report on mod_accel and mod_deflate
IS == Igor Sysoev [EMAIL PROTECTED] writes: IS There is one drawback in setting r-connection-remote_addr. IS If you want to disable direct access to backend then you can't do it IS with mod_access - you need to configure firewall. I always bind my back-end to a private IP space (either localhost or a 192.168.* type address). -- =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Vivek Khera, Ph.D.Khera Communications, Inc. Internet: [EMAIL PROTECTED] Rockville, MD +1-240-453-8497 AIM: vivekkhera Y!: vivek_khera http://www.khera.org/~vivek/
Re: Report on mod_accel and mod_deflate
On Thu, 20 Dec 2001, Stas Bekman wrote: Jeremy Howard wrote: Igor Sysoev mentioned recently on this list that he has written a module called 'mod_accel' that provides a caching HTTP accelerator, as well as a mod_gzip replacement called 'mod_deflate'. These modules are both used on Kaspersky labs' busy sites, as well as at the popular portal http://www.rambler.ru. mod_deflate is used at around 700 sites. Igor, sorry to tell you but httpd-2.0 has introduced mod_deflate as a replacement for mod_gzip, so you are going to have a lot of confused users when httpd 2.0 hits the streets. I suggest that you raise this issue on the httpd dev list before it's too late. May be it's too late already, since there was already a long discussion about why it should be called deflate and not gzip (check the archives). Thank you. I'd overlooked it. Last time I saw discussion about mod_gzip and mod_gz in new-httpd list was September. By the way do you speak Russian ? Igor Sysoev
Re: Report on mod_accel and mod_deflate
Igor Sysoev wrote: On Thu, 20 Dec 2001, Stas Bekman wrote: Jeremy Howard wrote: Igor Sysoev mentioned recently on this list that he has written a module called 'mod_accel' that provides a caching HTTP accelerator, as well as a mod_gzip replacement called 'mod_deflate'. These modules are both used on Kaspersky labs' busy sites, as well as at the popular portal http://www.rambler.ru. mod_deflate is used at around 700 sites. Igor, sorry to tell you but httpd-2.0 has introduced mod_deflate as a replacement for mod_gzip, so you are going to have a lot of confused users when httpd 2.0 hits the streets. I suggest that you raise this issue on the httpd dev list before it's too late. May be it's too late already, since there was already a long discussion about why it should be called deflate and not gzip (check the archives). Thank you. I'd overlooked it. Last time I saw discussion about mod_gzip and mod_gz in new-httpd list was September. I think they have decided on this name somewhere in October, after a very long dispute of whether it should be a part of the core or a 3rd party module. By the way do you speak Russian ? I do :) _ Stas Bekman JAm_pH -- Just Another mod_perl Hacker http://stason.org/ mod_perl Guide http://perl.apache.org/guide mailto:[EMAIL PROTECTED] http://ticketmaster.com http://apacheweek.com http://singlesheaven.com http://perl.apache.org http://perlmonth.com/
Report on mod_accel and mod_deflate
Igor Sysoev mentioned recently on this list that he has written a module
called 'mod_accel' that provides a caching HTTP accelerator, as well as a
mod_gzip replacement called 'mod_deflate'. These modules are both used on
Kaspersky labs' busy sites, as well as at the popular portal
http://www.rambler.ru. mod_deflate is used at around 700 sites.
So why haven't we heard more of them? Because the docs are only in Russian,
and almost all implementations have occured on Russian sites. Well, the good
news is that I've now implemented a mod_accel/mod_deflate front-end on
http://www.fastmail.fm, allowing me to announce a substantial speedup to our
users:
http://www.emailaddresses.com/forum/showthread.php?threadid=1603
You may notice in the linked announcement that I mention I've been able to
remove the restriction that attachments have to be downloaded within an
hour. We previously had this restriction because even with mod_proxy as a
front-end, uploads larger than the TCP buffer get fed through bit by bit,
keeping the mod_perl handler open the whole time... So we had code like:
|| my $oldalarm=alarm(0);
|| alarm 3600;
|| my $Status = $R-parse;
|| alarm $oldalarm;
Without that, the backend app server can hang around for ever serving one
client. We had similar code for downloads, as well as uploads. BTW, Joe
Schaefer has posted a mod_proxy patch to this list that fixes this for POST
uploads.
Well, mod_accel gets around this problem by automatically buffering large
uploads and downloads onto the filesystem! :-) In fact, mod_accel reads the
backend response and sends it to client asynchronously, so the client sees
the first part of the response even before the backend has completed the
request. But the backend is never blocked by the client, since it is
buffered to the file-system as required.
mod_deflate works really well with mod_accel so that both dynamic forwarded
requests and static requests are compressed. mod_deflate also has lots of
intelligence to avoid compressing content to buggy browsers that can't
handle it.
OK, enough evangelism. On to practical matters. Many of the following
snippets and comments were provided to me by Igor in private email, so thank
him, not me. Firstly, here's how to compile everything (including mod_ssl on
the front-end) and install it into /usr/local/apache_accel:
tar -zxvf mod_accel-1.0.7.tar.gz
tar -zxvf mod_deflate-1.0.9.tar.gz
tar -zxvf mm-1.1.3.tar.gz
cd mm-1.1.3
./configure --disable-shared
make
cd ../mod_ssl-2.8.5-1.3.22
./configure --with-apache=../apache_1.3.22_accel/ \
--with-crt=/INSERT_APPROPRIATE_PATH.crt \
--with-key=/INSERT_APPROPRIATE_PATH.key
cd ../mod_accel-1.0.7
./configure --with-apache=../apache_1.3.22_accel/ \
--with-eapi=../mod_ssl-2.8.5-1.3.22/pkg.eapi/
make
cd ../mod_deflate-1.0.9
./configure --with-apache=../apache_1.3.22_accel/
make
cd ../apache_1.3.22_accel/
EAPI_MM=../mm-1.1.3 SSLBASE=SYSTEM ./configure --enable-rule=EAPI \
--prefix=/usr/local/apache_accel --disable-rule=EXPAT \
--enable-module=most \
--enable-module=rewrite \
--activate-module=src/modules/accel/libaccel.a \
--activate-module=src/modules/extra/mod_deflate.o
make
make install
mkdir /var/accelcache
chown nobody:nobody /var/accelcache/
/usr/local/apache_accel/bin/apachectl startssl
Now, in /usr/local/apache_accel/conf/httpd.conf, append the following:
AccelCacheRoot /var/accelcache 1
AccelNoCacheon
AccelPass/mail/http://127.0.0.1:8080/mail/
AccelWaitBeforeBodyRead 100
AccelUnlinkNoCached off
AccelSetXHost on
AccelSetXRealIP on
AccelSetXURL on
This creates an HTTP accelerator without any caching. I don't know much
about the caching directives and haven't tried them out, so I can't provide
any guidance there.
The AccelCacheRoot directive sets the number of levels of subdirectories in
the cache. 16 first level subdirectories named 0 .. f and 256 second level
subdirectories named 00 .. ff are used by default (i.e. AccelCacheRoot
path 1 2). Since I'm not caching, my config above does not specify any
directory hashing (i.e. AccelCacheRoot path 1).
AccelNoCache simply turns off caching.
AccelPass specifies that I want everything under /mail/ forwarded to the
backend. You can also specify '/' for AccelPass, and then use something like
AccelNoPass /download /images ~*\.jpg$ to create exceptions (uses
standard regex syntax).
Note that mod_accel can also be called by utilising the mod_rewrite [P]
directive, just like with mod_proxy.
AccelWaitBeforeBodyRead sets the time in milliseconds to wait between
receiving the response header and the first part of the body. It is useful
over Ethernet where packets are small. If mod_accel waits a little it can
read most of the body or even the whole body with one just one
select()/read() pair of syscalls. You can see the first read size and the
number of reads in the log with %{accel}x in a CustomLog directive. Igor
usually uses 100-200
