Re: [AOLSERVER] gzip and content acceleration
On Tue, 2005-09-27 at 11:24 -0400, Mark Mcgaha wrote: > I do not check time stamps in the tcl, but this is what I use: Good alternative solution. Thank you. Daniel -- | --- | Daniel P. Stasinski | http://www.disabilities-r-us.com/ | [EMAIL PROTECTED]| http://www.scriptkitties.com/ | --- | - | Jabber: [EMAIL PROTECTED] | Google Talk: mooo -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] gzip and content acceleration
I do not check time stamps in the tcl, but this is what I use:
ns_register_filter postauth GET /*.html filter_get_pregz
ns_register_filter postauth GET /*.css filter_get_pregz
ns_register_filter postauth GET /*.js filter_get_pregz
proc filter_get_pregz {why} {
if {[string first {gzip} [ns_set get [ns_conn headers]
{Accept-Encoding}]] != -1} {
set file [ns_url2file [ns_conn url]]
set gz_file "${file}.gz"
if {[file exists $gz_file]} {
ns_set put [ns_conn outputheaders] Content-Encoding gzip
ns_returnfile 200 [ns_guesstype $file] $gz_file
return "filter_return"
}
}
return "filter_ok"
}
I let the shell script that makes the gz files worry about when the
files were updated.
#!/bin/sh
umask 002
#
# delete old gz files
#
find . -type f -print |grep -E '\.css\.gz$|\.html\.gz$|\.js\.gz$'
|(while read gzfile; do
source=`expr $gzfile : '\(.*\)\.gz$'`
if ! [ -f $source ] || [ $gzfile -ot $source ]; then
rm $gzfile
fi
done)
#
# generate missing gz files
#
find . -type f -print |grep -E '\.css$|\.html$|\.js$' |(while read file; do
if ! [ -f $file.gz ]; then
gzip -9 -c <$file >$file.gz
fi
done)
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]>
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject:
field of your email blank.
Re: [AOLSERVER] gzip and content acceleration
Yes, the new driver model in the head version is much better suited for serving static content, although we have yet to do any performance testing to actually back this up. ;-) That said, the motivation wasn't necessarily to replace Tux, and as I mentioned earlier, the server is primarily optimized for dynamic content serving. One feature that was discussed, however, is providing a way to register async callbacks that would fire off before before the request was dispatched to the connection threads, which we've asserted are relatively "expensive" resources. One possible use case was something like the following: 1. Request received by Web server 2. Request consumed (HTTP headers parsed) 3. Callback, registration based on request path, triggered 4. Callback does some interesting work such as a server-side ad call, or other Web-Service type request 5. Once callback completes, the original request information and callback results get passed to connection thread for processing The idea is that for I/O related async activities at least, having the events get processed in the single-threaded driver thread should be faster. Think of it as a mix of the best aspects of single- threaded event programming (driver thread), and multi-threaded programming (connection threads). - n On Sep 26, 2005, at 3:45 PM, Daniel P. Stasinski wrote: On Mon, 2005-09-26 at 13:44 -0400, Nathan Folkman wrote: To be honest, the Tux approach mentioned below might be your best solution. AOLserver is primarily optimized for dynamic serving, with static assets (flat HTML, images, JS, CSS, etc.) generally served by servers optimized for static serving. I was happy with that whole concept too until I got the 4.5cvs and noticed that aolserver was now serving a lot of the static content that Tux was serving. Though I haven't gone far through the code yet, I am assuming that pipelining support was added to aolserver and that would supersede Tux's ability to serve the static content. Daniel -- | --- | Daniel P. Stasinski | http://www.disabilities-r-us.com/ | [EMAIL PROTECTED]| http://www.scriptkitties.com/ | --- | - | Jabber: [EMAIL PROTECTED] | Google Talk: mooo -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] gzip and content acceleration
On Mon, 2005-09-26 at 13:44 -0400, Nathan Folkman wrote: > To be honest, the Tux approach mentioned below might be your best > solution. AOLserver is primarily optimized for dynamic serving, with > static assets (flat HTML, images, JS, CSS, etc.) generally served by > servers optimized for static serving. I was happy with that whole concept too until I got the 4.5cvs and noticed that aolserver was now serving a lot of the static content that Tux was serving. Though I haven't gone far through the code yet, I am assuming that pipelining support was added to aolserver and that would supersede Tux's ability to serve the static content. Daniel -- | --- | Daniel P. Stasinski | http://www.disabilities-r-us.com/ | [EMAIL PROTECTED]| http://www.scriptkitties.com/ | --- | - | Jabber: [EMAIL PROTECTED] | Google Talk: mooo -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] gzip and content acceleration
To be honest, the Tux approach mentioned below might be your best solution. AOLserver is primarily optimized for dynamic serving, with static assets (flat HTML, images, JS, CSS, etc.) generally served by servers optimized for static serving. - n On Sep 26, 2005, at 1:24 PM, Daniel P. Stasinski wrote: On Mon, 2005-09-26 at 12:56 -0400, Nathan Folkman wrote: Makes sense... Is there anything preventing you from adding some code to implement this logic yourself, on top of a "vanilla" AOLserver with something like the ns_gzip module? There is nothing preventing me from doing it, but it seems like one hell of a great feature that is better done as a feature rather than a hack. I don't see a need for the static file compression to be part of the server core itself. There's no sense in repeatedly compressing the same content over and over, even if aolserver caches it afterward. I just occasionally run console script that recursively goes through all the html, js and css files under the pageroot and creates an alternate .gz file in the same location. At the moment I am using Tux to do it. It listens for http requests at the kernel level and responds accordingly. If it is unable to serve the requested content, it passes the request to aolserver which is listening on localhost. Daniel -- | --- | Daniel P. Stasinski | http://www.disabilities-r-us.com/ | [EMAIL PROTECTED]| http://www.scriptkitties.com/ | --- | - | Jabber: [EMAIL PROTECTED] | Google Talk: mooo -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] gzip and content acceleration
On 2005.09.26, Nathan Folkman <[EMAIL PROTECTED]> wrote: > Take a look at the head version, specifically: > [... ChangeLog excerpt snipped ...] Ah, except Ns_Compress() isn't available through a Tcl command, yet. A lot of times, the expression "XYZ isn't available in AOLserver" is roughly translatable to "XYZ isn't available as a Tcl command in AOLserver." -- Dossy -- Dossy Shiobara [EMAIL PROTECTED] | http://dossy.org/ Panoptic Computer Network http://panoptic.com/ "He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on." (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] gzip and content acceleration
On Mon, 2005-09-26 at 12:56 -0400, Nathan Folkman wrote: > Makes sense... Is there anything preventing you from adding some code > to implement this logic yourself, on top of a "vanilla" AOLserver > with something like the ns_gzip module? There is nothing preventing me from doing it, but it seems like one hell of a great feature that is better done as a feature rather than a hack. I don't see a need for the static file compression to be part of the server core itself. There's no sense in repeatedly compressing the same content over and over, even if aolserver caches it afterward. I just occasionally run console script that recursively goes through all the html, js and css files under the pageroot and creates an alternate .gz file in the same location. At the moment I am using Tux to do it. It listens for http requests at the kernel level and responds accordingly. If it is unable to serve the requested content, it passes the request to aolserver which is listening on localhost. Daniel -- | --- | Daniel P. Stasinski | http://www.disabilities-r-us.com/ | [EMAIL PROTECTED]| http://www.scriptkitties.com/ | --- | - | Jabber: [EMAIL PROTECTED] | Google Talk: mooo -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] gzip and content acceleration
Makes sense... Is there anything preventing you from adding some code to implement this logic yourself, on top of a "vanilla" AOLserver with something like the ns_gzip module? - n On Sep 26, 2005, at 12:41 PM, Daniel P. Stasinski wrote: On Mon, 2005-09-26 at 11:30 -0400, Dossy Shiobara wrote: On 2005.09.26, Daniel P. Stasinski <[EMAIL PROTECTED]> wrote: Is the question "will ns_gzip be included as part of the core" or "is there a better way of integrating gzip into the core than just using ns_gzip"? The question is neither. Let's say a user requests index.html. Instead of just serving up index.html, I want AOLserver to see if there is a file called index.html.gz already existing in the same directory. If it is there, serve index.html.gz instead of index.html. If there is no .gz version there, serve the plain old index.html. This is how Redhat Content Accelerator does it. http://www.redhat.ru/docs/manuals/tux/TUX-2.2-Manual/gzip.html The web server does not provide compression services, it only provides compressed content if someone else has already compressed it beforehand. Daniel -- | --- | Daniel P. Stasinski | http://www.disabilities-r-us.com/ | [EMAIL PROTECTED]| http://www.scriptkitties.com/ | --- | - | Jabber: [EMAIL PROTECTED] | Google Talk: mooo -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] gzip and content acceleration
On Mon, 2005-09-26 at 11:30 -0400, Dossy Shiobara wrote: > On 2005.09.26, Daniel P. Stasinski <[EMAIL PROTECTED]> wrote: > Is the question "will ns_gzip be included as part of the core" or "is > there a better way of integrating gzip into the core than just using > ns_gzip"? The question is neither. Let's say a user requests index.html. Instead of just serving up index.html, I want AOLserver to see if there is a file called index.html.gz already existing in the same directory. If it is there, serve index.html.gz instead of index.html. If there is no .gz version there, serve the plain old index.html. This is how Redhat Content Accelerator does it. http://www.redhat.ru/docs/manuals/tux/TUX-2.2-Manual/gzip.html The web server does not provide compression services, it only provides compressed content if someone else has already compressed it beforehand. Daniel -- | --- | Daniel P. Stasinski | http://www.disabilities-r-us.com/ | [EMAIL PROTECTED]| http://www.scriptkitties.com/ | --- | - | Jabber: [EMAIL PROTECTED] | Google Talk: mooo -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] gzip and content acceleration
On 2005.09.26, Daniel P. Stasinski <[EMAIL PROTECTED]> wrote: > > Well of course, I was wondering if this would ever be part of the core? > ns_gzip works well but it is better suited for on-the-fly compression. Is the question "will ns_gzip be included as part of the core" or "is there a better way of integrating gzip into the core than just using ns_gzip"? Former, at some point but low priority now. Latter, I'm not sure; the performance difference is probably immeasurably small, if there is a real benefit at all. -- Dossy -- Dossy Shiobara [EMAIL PROTECTED] | http://dossy.org/ Panoptic Computer Network http://panoptic.com/ "He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on." (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] gzip and content acceleration
Take a look at the head version, specifically: 2004-11-19 Dossy Shiobara <[EMAIL PROTECTED]> * configure (1.23), configure.in (1.18), include/Makefile.global.in (1.17), nsd/compress.c (1.2): Add --with-zlib configure option (on by default) and add ifdef's to nsd/compress.c. 2004-11-19 Dossy Shiobara <[EMAIL PROTECTED]> * nsd/Makefile (1.42), nsd/compress.c (1.1), include/ns.h (1.71): Add new Ns_Compress() function to gzip data using Zlib.- nOn Sep 26, 2005, at 9:51 AM, Daniel P. Stasinski wrote:Well of course, I was wondering if this would ever be part of the core? ns_gzip works well but it is better suited for on-the-fly compression. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] gzip and content acceleration
On Mon, 2005-09-26 at 07:59 -0400, Dossy Shiobara wrote: > (Nitpick: I think you mean "same or newer.") Ok, nitpick proof text to follow. The client has explicitly stated to support gzip encoding. The original file exists, is a regular file, and has the proper permissions. The .gz file exists, is a regular file, and has the proper permissions. The .gz file is newer than or has the same-date as the original file. The size of the .gz file is smaller than original file. > Sure -- I think with a registered filter and Vlad's ns_gzip, you should > be able to implement this with a few lines of Tcl. Well of course, I was wondering if this would ever be part of the core? ns_gzip works well but it is better suited for on-the-fly compression. Daniel -- | --- | Daniel P. Stasinski | http://www.disabilities-r-us.com/ | [EMAIL PROTECTED]| http://www.scriptkitties.com/ | --- | - | Jabber: [EMAIL PROTECTED] | Google Talk: mooo -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] gzip and content acceleration
On 2005.09.25, Daniel P. Stasinski <[EMAIL PROTECTED]> wrote: > A few years ago at one of the online meetings we had discussed content > acceleration with gzip. I know that 4.0.10 and above support on-the-fly > ADP compression but are there plans to extend it to static files? > > With Tux, when a URL is requested, say index.html, the system will look > for an existing index.html.gz also. If the .gz file is smaller than the > regular one -and- the timestamp is the same or older, it will serve > the .gz version instead. (Nitpick: I think you mean "same or newer.") > Possible? Sure -- I think with a registered filter and Vlad's ns_gzip, you should be able to implement this with a few lines of Tcl. -- Dossy -- Dossy Shiobara [EMAIL PROTECTED] | http://dossy.org/ Panoptic Computer Network http://panoptic.com/ "He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on." (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
[AOLSERVER] gzip and content acceleration
A few years ago at one of the online meetings we had discussed content acceleration with gzip. I know that 4.0.10 and above support on-the-fly ADP compression but are there plans to extend it to static files? With Tux, when a URL is requested, say index.html, the system will look for an existing index.html.gz also. If the .gz file is smaller than the regular one -and- the timestamp is the same or older, it will serve the .gz version instead. Possible? Daniel -- | --- | Daniel P. Stasinski | http://www.disabilities-r-us.com/ | [EMAIL PROTECTED]| http://www.scriptkitties.com/ | --- | - | Jabber: [EMAIL PROTECTED] | Google Talk: mooo -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
