Re: [AOLSERVER] gzip compression
Don't think so as I believe it relies on some newly added C APIs (Ns_SetGzipProc, Ns_Gzip, etc.), which are only available in AOLserver 4.5. Have you looked at the "nszlib" AOLserver module? It's available from the AOLserver CVS tree: aolserver/nszlib Hope that helps! - n [EMAIL PROTECTED] wrote: On Monday 26 June 2006 21:47, Nathan Folkman wrote: Yes. In fact, there's a new nszlib AOLserver module as part of the AOLserver 4.5 release. Thanks !!! Can it also be used on/in Aolserver 4.0.10 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] gzip compression
On Monday 26 June 2006 21:47, Nathan Folkman wrote: > Yes. In fact, there's a new nszlib AOLserver module as part of the > AOLserver 4.5 release. Thanks !!! Can it also be used on/in Aolserver 4.0.10 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] gzip compression
Yes. In fact, there's a new nszlib AOLserver module as part of the AOLserver 4.5 release. Here's the relevant release notes: nszlib: The popular nszlib module written by Vlad Seryakov has been integrated into the core release with modifications to enable the new Ns_Gzip routine. The module also provides the "ns_zlib" command for compressing and uncompressing strings and files. C APIs: Ns_Gzip: Ns_SetGzipProc: Routine to gzip-compress a buffer if a compression function has been registered with a call to Ns_SetGzipProc. The new "nszlib" module registers such a callback. Tcl APIs: ns_zlib ns_zlib option ?arg? ns_adp_ctl gzip ?bool?; # Gzip output if enabled in server? Also be sure to check out the README file (aolserver/nszlib/README) - n [EMAIL PROTECTED] wrote: Hello, I would like to ask if AOLServer supports gzip compression, similar to Apache's mod_gzip. Thanks, Hamilton -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] gzip compression
Having said that, the preferred future direction seems to be
ns_adp_compress:
http://sourceforge.net/tracker/index.php?
func=detail&group_id=3152&atid=353152&aid=1099613
Does this gzip option work with the AOLserver source in the current
CVS tree?
I tried it, and added the recommended config section:
> ns_section "ns/server/${servername}/adp/gzip"
> ns_param enabled on
> ns_param level 4
> ns_param minsize 0
and couldn't get a compressed result back with:
> curl -H "Accept-Encoding: gzip" http://londonmini
Which I think should work.
Digging in the source, I see references to config options "gzipmin"
and "gziplevel"
if (!Ns_ConfigGetBool(path, "gzip", &i) || i) {
servPtr->opts.flags |= SERV_GZIP;
}
if (!Ns_ConfigGetInt(path, "gzipmin", &i) || i <= 0) {
i = 4 * 1024;
}
servPtr->opts.gzipmin = i;
if (!Ns_ConfigGetInt(path, "gziplevel", &i) || i < 0 || i > 9) {
i = 4;
}
servPtr->opts.gziplevel = i;
-john
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]>
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject:
field of your email blank.
Re: [AOLSERVER] gzip compression
Sure does. Against aolserver 3 I very successfully use the rlreturnz module - see: http://openacs.org/forums/message-view?message_id=32589 I believe that this was the basis for nsreturnz which is in the aolserver cvs: http://aolserver.cvs.sourceforge.net/aolserver/nsreturnz/ Having said that, the preferred future direction seems to be ns_adp_compress: http://sourceforge.net/tracker/index.php?func=detail&group_id=3152&atid=353152&aid=1099613 On 6/26/06, Hamilton Chua <[EMAIL PROTECTED]> wrote: Hello, I would like to ask if AOLServer supports gzip compression, similar to Apache's mod_gzip. Thanks, Hamilton -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank. -- Mark Aufflick e: [EMAIL PROTECTED] w: mark.aufflick.com p: +61 438 700 647 f: +61 2 9436 4737 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
[AOLSERVER] gzip compression
Hello, I would like to ask if AOLServer supports gzip compression, similar to Apache's mod_gzip. Thanks, Hamilton -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Gzip compression
On 2005.10.24, Daniel P. Stasinski <[EMAIL PROTECTED]> wrote: > On Mon, 2005-10-24 at 10:19 -0400, Dossy Shiobara wrote: > > On 2005.10.24, Daniel P. Stasinski <[EMAIL PROTECTED]> wrote: > > > I don't have commit access to anything other than nsecrypt() so I'll > > > just send the file to you directly. > > You've said this before but I just don't understand why you say this. > > My apologies. I have always been under the impression that there was a > module level access list so, not wanting to poke the bunny, I never > tried. No worries. I just didn't know if you were getting some kind of SourceForge CVS error and interpreting that as some restricted access, but from what I can tell, all signs point to the fact that you have just as much CVS commit access as I do right now. Update: I just saw you commit the gzip fixes to either HEAD or the 4.0 branch, so it looks like everything "works" -- awesome! -- Dossy -- Dossy Shiobara [EMAIL PROTECTED] | http://dossy.org/ Panoptic Computer Network http://panoptic.com/ "He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on." (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Gzip compression
On Mon, 2005-10-24 at 10:19 -0400, Dossy Shiobara wrote: > On 2005.10.24, Daniel P. Stasinski <[EMAIL PROTECTED]> wrote: > > I don't have commit access to anything other than nsecrypt() so I'll > > just send the file to you directly. > You've said this before but I just don't understand why you say this. My apologies. I have always been under the impression that there was a module level access list so, not wanting to poke the bunny, I never tried. Daniel -- | --- | Daniel P. Stasinski | http://www.disabilities-r-us.com/ | [EMAIL PROTECTED]| http://www.scriptkitties.com/ | --- | - | Jabber: [EMAIL PROTECTED] | Google Talk: mooo -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Gzip compression
On 2005.10.24, Daniel P. Stasinski <[EMAIL PROTECTED]> wrote: > I don't have commit access to anything other than nsecrypt() so I'll > just send the file to you directly. You've said this before but I just don't understand why you say this. cvs_acls isn't set up for the AOLserver project at SourceForge: anyone with CVS commit access to the project has commit access to the /entire/ project. So, you can't be referring to a technologically-implemented restriction. The "social policy" around CVS committing is described by the CVS commit guidelines. As it states, if you have CVS commit access, you are welcome to commit changes to any file in the entire CVS repository. Go ahead. Merge in your changes. If it really does refuse to allow you to commit them, then there's a problem at SourceForge and I'll work with you to get the necessary information to open a support ticket and have them resolve it for us. -- Dossy -- Dossy Shiobara [EMAIL PROTECTED] | http://dossy.org/ Panoptic Computer Network http://panoptic.com/ "He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on." (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Gzip compression
On Mon, 2005-10-24 at 07:50 -0400, Dossy Shiobara wrote: > Actually, Daniel, is there a reason you couldn't commit your fixes to > the aolserver_v40_bp branch? Is gzip encoding still broke in HEAD, too? Yep, it's broken in the head too. I posted a diff to the list several weeks ago and never heard mention of it again. I don't have commit access to anything other than nsecrypt() so I'll just send the file to you directly. Daniel -- | --- | Daniel P. Stasinski | http://www.disabilities-r-us.com/ | [EMAIL PROTECTED]| http://www.scriptkitties.com/ | --- | - | Jabber: [EMAIL PROTECTED] | Google Talk: mooo -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Gzip compression
On Mon, 2005-10-24 at 09:59 +0400, Nader Henein wrote:
> Can I get a copy of that, tried to get this working a few months back
> and even after diving into the original C code figured it wasn't working.
Sure, see attached.
Daniel
--
| ---
| Daniel P. Stasinski | http://www.disabilities-r-us.com/
| [EMAIL PROTECTED]| http://www.scriptkitties.com/
| --- | -
| Jabber: [EMAIL PROTECTED] | Google Talk: mooo
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]>
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject:
field of your email blank.
/*
* The contents of this file are subject to the AOLserver Public License
* Version 1.1 (the "License"); you may not use this file except in
* compliance with the License. You may obtain a copy of the License at
* http://aolserver.com/.
*
* Software distributed under the License is distributed on an "AS IS"
* basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See
* the License for the specific language governing rights and limitations
* under the License.
*
* The Original Code is AOLserver Code and related documentation
* distributed by AOL.
*
* The Initial Developer of the Original Code is America Online,
* Inc. Portions created by AOL are Copyright (C) 1999 America Online,
* Inc. All Rights Reserved.
*
* Alternatively, the contents of this file may be used under the terms
* of the GNU General Public License (the "GPL"), in which case the
* provisions of GPL are applicable instead of those above. If you wish
* to allow use of your version of this file only under the terms of the
* GPL and not to allow others to use your version of this file under the
* License, indicate your decision by deleting the provisions above and
* replace them with the notice and other provisions required by the GPL.
* If you do not delete the provisions above, a recipient may use your
* version of this file under either the License or the GPL.
*/
/*
* compress.c --
*
* Support for simple gzip compression using Zlib.
*/
static const char *RCSID = "@(#) $Header: /cvsroot/aolserver/aolserver/nsd/compress.c,v 1.1.2.3 2005/01/12 19:11:46 dossy Exp $, compiled: " __DATE__ " " __TIME__;
#include "ns.h"
#ifdef HAVE_ZLIB_H
#include
static char header[] = {
037, 0213, /* GZIP magic number. */
010,/* Z_DEFLATED */
0, /* flags */
0,0,0,0,/* timestamp */
0, /* xflags */
03};/* Unix OS_CODE */
/*
*--
*
* Ns_CompressGzip --
*
* Compress a string using gzip with RFC 1952 header/footer.
*
* Results:
* NS_OK if compression worked, NS_ERROR otherwise.
*
* Side effects:
* Will write compressed content to given Tcl_DString.
*
*--
*/
int
Ns_CompressGzip(char *buf, int len, Tcl_DString *outPtr, int level)
{
uLongf glen;
char *gbuf;
uLong crc;
int skip;
uint32_t footer[2];
/*
* Grow buffer for header, footer, and maximum compressed size.
*/
glen = compressBound(len) + sizeof(header) + sizeof(footer);
Tcl_DStringSetLength(outPtr, glen);
/*
* Compress output starting 2-bytes from the end of the header.
*/
gbuf = outPtr->string;
skip = sizeof(header) - 2;
glen -= skip;
if (compress2(gbuf + skip, &glen, buf, (uLong) len, level) != Z_OK) {
return NS_ERROR;
}
glen -= 4;
memcpy(gbuf, header, sizeof(header));
Tcl_DStringSetLength(outPtr, glen + skip);
/*
* Append footer of CRC and uncompressed length.
*/
crc = crc32(0, Z_NULL, 0);
crc = crc32(crc, buf, len);
footer[0] = crc;
footer[1] = len;
Tcl_DStringAppend(outPtr, (char *) footer, sizeof(footer));
return NS_OK;
}
#else
int
Ns_CompressGzip(char *buf, int len, Tcl_DString *outPtr, int level)
{
return NS_ERROR;
}
#endif
/*
*--
*
* Ns_Compress --
*
* Deprecated: Used by nsjk2 module. Remove this once nsjk2 has
* been updated to use Ns_CompressGzip directly.
*
* Results:
* NS_OK if compression worked, NS_ERROR otherwise.
*
* Side effects:
* Will write compressed content to given Tcl_DString.
*
*--
*/
int
Ns_Compress(char *buf, int len, Tcl_DString *outPtr, int level)
{
return Ns_CompressGzip(buf, len, outPtr, level);
}
Re: [AOLSERVER] Gzip compression
On 2005.10.24, Nader Henein <[EMAIL PROTECTED]> wrote: > Can I get a copy of that, tried to get this working a few months back > and even after diving into the original C code figured it wasn't working. > > Daniel P. Stasinski wrote: > >On Fri, 2005-10-21 at 09:48 -0700, C. R. Oldham wrote: > >>I've added this to my servers but no OpenACS pages seem to be gzipped. > >>Am I missing something? These servers are on OpenACS 5.0. > > > >Add this to all adp's before sending any data: > > > > ns_adp_compress 1 > > > >Also, the gzip code there is very broken. I can send you my working > >compress.c file if you need it. Actually, Daniel, is there a reason you couldn't commit your fixes to the aolserver_v40_bp branch? Is gzip encoding still broke in HEAD, too? -- Dossy -- Dossy Shiobara [EMAIL PROTECTED] | http://dossy.org/ Panoptic Computer Network http://panoptic.com/ "He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on." (p. 70) -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Gzip compression
Can I get a copy of that, tried to get this working a few months back and even after diving into the original C code figured it wasn't working. Nader Henein Daniel P. Stasinski wrote: On Fri, 2005-10-21 at 09:48 -0700, C. R. Oldham wrote: I've added this to my servers but no OpenACS pages seem to be gzipped. Am I missing something? These servers are on OpenACS 5.0. Add this to all adp's before sending any data: ns_adp_compress 1 Also, the gzip code there is very broken. I can send you my working compress.c file if you need it. Daniel -- Nader S. Henein Senior Applications Architect Bayt Technologies * Bayt.com Dubai Internet City Building 2, Office 314 P.O. Box 500245 Tel. +971 4 3911900 Dir. +971 4 3613675 Cell. +971 50 5659557 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Gzip compression
On Fri, 2005-10-21 at 09:48 -0700, C. R. Oldham wrote: > I've added this to my servers but no OpenACS pages seem to be gzipped. > Am I missing something? These servers are on OpenACS 5.0. Add this to all adp's before sending any data: ns_adp_compress 1 Also, the gzip code there is very broken. I can send you my working compress.c file if you need it. Daniel -- | --- | Daniel P. Stasinski | http://www.disabilities-r-us.com/ | [EMAIL PROTECTED]| http://www.scriptkitties.com/ | --- | - | Jabber: [EMAIL PROTECTED] | Google Talk: mooo -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
[AOLSERVER] Gzip compression
Greetings,
With 4.0.10, the release notes say that gzip compression is now
available in the core server instead of using the ns_returnz or
rl_returnz modules. In reading the source it looks like in the
ns/${servername}/adp
section you have to say
compress true
I've added this to my servers but no OpenACS pages seem to be gzipped.
Am I missing something? These servers are on OpenACS 5.0.
--cro
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]>
with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject:
field of your email blank.
Re: [AOLSERVER] gzip compression
On Thu, Jan 09, 2003 at 11:16:34AM -0500, Gabriel Ricard wrote: > Has anyone investigated gzip transport encoding at all? > > I played around with the rlreturnz stuff and it's nice and it works, > but my main problem with gzip encoded content is that the browser waits > until it's downloaded the entire file before it decompresses the data > and begins rendering the page. A side-effect is that it makes pages > appear to take longer to load, even though they take less time to > download. Of course it depends on the content that was compressed in My experience is quite the opposite. With gzip encoding I get broadband-like speeds from dial-up connections. -Roberto -- +|Roberto Mello -http://www.brasileiro.net/ |--+ + Computer Science Graduate Student, Utah State University + + USU Free Software & GNU/Linux Club - http://fslc.usu.edu/ + Insert New Disk for Drive C: Press ENTER when ready.
Re: [AOLSERVER] gzip compression
I think it is browser-dependant whether gzip decompression is performed after all data is received. Here is a quote from the Mozilla site, at http://www.mozilla.org/projects/apache/gzip/ "The current Mozilla source already sends Accept-encoding: gzip and can do a streaming decompression of HTML data received with Content-encoding: gzip." gzip compression is inherently stream-oriented, so the decompression can/should be stream-oriented as well. One effect you may be seeing is that if you have a web page with unconstrained tables, the browser will have to download the entire page, decompress it, and layout the entire table before anything can be displayed. This happens with or without gzip. But if you are on a very slow client omputer and have a fast connection, the gzip time could outweigh the uncompressed data transfer time and make the page display slower. I don't know whether MSIE decompresses on-the-fly or waits until all data is downloaded. Jim > > Has anyone investigated gzip transport encoding at all? > > I played around with the rlreturnz stuff and it's nice and it works, > but my main problem with gzip encoded content is that the browser waits > until it's downloaded the entire file before it decompresses the data > and begins rendering the page. A side-effect is that it makes pages > appear to take longer to load, even though they take less time to > download. Of course it depends on the content that was compressed in > the first place, but for HTML pages it's a noticeable difference, which > is why I never adopted the use of the gzip content encoding. > > The other possibility I read about was using gzip for HTTP transfer > encoding. So at the protocol level it would compress and stream the > data (I believe...). I assume this would be pretty much transparent to > the generation of HTML (or any other content) and stuffing it down the > pipe and then the browser decompressing and rendering the page as it > did before (without the noticeable delay). > > I have no idea what, if any, web browsers support this yet. > > Anyone got a clue or two they can share about this? > > - Gabriel >
Re: [AOLSERVER] gzip compression
Hi Gabriel, > I played around with the rlreturnz stuff and it's nice and it works, > but my main problem with gzip encoded content is that the browser waits > until it's downloaded the entire file before it decompresses the data > and begins rendering the page. A side-effect is that it makes pages > appear to take longer to load, even though they take less time to > download. Of course it depends on the content that was compressed in > the first place, but for HTML pages it's a noticeable difference, which > is why I never adopted the use of the gzip content encoding. I think you win more than you loose, I have some statistics from a gzip-module currently in development, with some good numbers: [08/Jan/2003:20:12:00 +0100]20598 Bytes 3819 Bytes 0.01sec [08/Jan/2003:20:12:38 +0100]20598 Bytes 3819 Bytes 0.01sec [08/Jan/2003:20:13:13 +0100]11336 Bytes 2479 Bytes 0.00sec [08/Jan/2003:20:13:53 +0100]18025 Bytes 3131 Bytes 0.00sec [08/Jan/2003:20:13:54 +0100]18017 Bytes 3133 Bytes 0.00sec [08/Jan/2003:20:13:57 +0100]18642 Bytes 3503 Bytes 0.00sec [08/Jan/2003:20:14:04 +0100]13407 Bytes 2748 Bytes 0.01sec The difference between compression level 5 and 9, e.g, results only in about 150 bytes; rlreturnz.c uses 3, if I remember right. Using the highest level and then caching it similar like the fastpath stuff would eliminate thinking about processor load, too. Compression IMHO is most of the time a big win for the vast majority of web sites. I really don't "feel" that it takes longer to wait for the page. I have some pages with very long result sets, 300k and more, that can be compressed very good and take a fraction of the time loaded and rendered than uncompressed. Regards, Bernd.
Re: [AOLSERVER] gzip compression
Has anyone investigated gzip transport encoding at all? I played around with the rlreturnz stuff and it's nice and it works, but my main problem with gzip encoded content is that the browser waits until it's downloaded the entire file before it decompresses the data and begins rendering the page. A side-effect is that it makes pages appear to take longer to load, even though they take less time to download. Of course it depends on the content that was compressed in the first place, but for HTML pages it's a noticeable difference, which is why I never adopted the use of the gzip content encoding. The other possibility I read about was using gzip for HTTP transfer encoding. So at the protocol level it would compress and stream the data (I believe...). I assume this would be pretty much transparent to the generation of HTML (or any other content) and stuffing it down the pipe and then the browser decompressing and rendering the page as it did before (without the noticeable delay). I have no idea what, if any, web browsers support this yet. Anyone got a clue or two they can share about this? - Gabriel
Re: [AOLSERVER] gzip compression
On Monday, January 6, 2003, at 11:10 PM, Jeffrey Hobbs wrote: That's a good question ... I've never actually done performance analysis on the stacked channel stuff, but it is fairly efficient. It just passes buffers from one stack to the next as they are consumed. Special purpose filters may do a better job as they can suck in large amounts of data as they operate, but I'm not sure that you'd win if you have to have extra exec overhead. What would be a good test? I suppose timing SSL connection throughput using Tcl stacked channels via the Tcl C API and a straight C implementation. If the channels have configurable buffer sizes and such they may work well, i.e. all channels in a stack should have the same buffer sizes, the exception being for a channel whose output is larger than its input (a decompression channel, for example) -- the channel that gets that output should also reasonably be expected to have a larger buffer size. Jim D., Nathan and I were just discussing the comm model in AOLserver. For 4.x, Jim has created a very aggressive read-ahead comm model that isn't suitable for all types of network usage but is very suitable for dealing with HTTP content. I'd be surprised if Tcl stacked channels could match the speed and efficiency of this model, and in AOLserver's case, specialized code to do this is worth the fact that it's not generalized to handle any type of case. My suggestion was that we allow multiple comm models in AOLserver (and Tcl?) such that the one best suited for a particular task could be selected by the module or code that needs it. An HTTP connection might work great with the aggressive read-ahead model, for example, but SSL connections might not. Having a means in AOLserver's C API (and/or in Tcl's API) to choose which comm model, or transformation model (i.e. what channel type) to put into a stack would allow more flexibility. /s. In any case, I did want to mention that since 8.2 Tcl has had stacked channels in the core. Extensions like Trf, memchan and TLS use this to do compression and/or encryption on channels transparent to the user. I create a new channel type in tclcmds.c in the nsopenssl module to allow Tcl's gets, puts and friends to work with an SSL "socket", though I didn't use channels for the C API portion of the process. Are there any performance issues with stacking channels? Jeff Hobbs The Tcl Guy Senior Developer http://www.ActiveState.com/
Re: [AOLSERVER] gzip compression
> > In any case, I did want to mention that since 8.2 Tcl has had stacked > > channels in the core. Extensions like Trf, memchan and TLS use this > > to do compression and/or encryption on channels transparent to the > > user. > I create a new channel type in tclcmds.c in the nsopenssl module to > allow Tcl's gets, puts and friends to work with an SSL "socket", though > I didn't use channels for the C API portion of the process. > > Are there any performance issues with stacking channels? That's a good question ... I've never actually done performance analysis on the stacked channel stuff, but it is fairly efficient. It just passes buffers from one stack to the next as they are consumed. Special purpose filters may do a better job as they can suck in large amounts of data as they operate, but I'm not sure that you'd win if you have to have extra exec overhead. What would be a good test? Jeff Hobbs The Tcl Guy Senior Developer http://www.ActiveState.com/
Re: [AOLSERVER] gzip compression
On Monday, January 6, 2003, at 04:30 PM, Jeff Hobbs wrote: In any case, I did want to mention that since 8.2 Tcl has had stacked channels in the core. Extensions like Trf, memchan and TLS use this to do compression and/or encryption on channels transparent to the user. gzip gets a bit more complicated because you have to stack multiple things to get it right (compression and checksums), but it can still be wrapped in a transparent manner. The same could go for ssl encryption, which is in fact what tclhttpd uses, simply like so: I create a new channel type in tclcmds.c in the nsopenssl module to allow Tcl's gets, puts and friends to work with an SSL "socket", though I didn't use channels for the C API portion of the process. Are there any performance issues with stacking channels? /s.
Re: [AOLSERVER] gzip compression
> I wouldn't mind seeing hooks, especially if they can be made more generic; > gzip is today's problem, but I'm sure there will be another down the road. > It would be nice to be able to have a stack of stream postprocessors, of > which gzip compression could be one. Another module to get pushed onto > the stack might be rate-limiting (so that a greedy client gets no more > than x bytes per second; different from connection throttling, which needs > to be done at the front of the request). I have not had the opportunity to delve too far into how the latest AOLServer handles its I/O, although I know that it is somewhat (mostly?) independent of the core Tcl_Channel's based on an earlier I18N discussion (and I understand this is in large part historical). In any case, I did want to mention that since 8.2 Tcl has had stacked channels in the core. Extensions like Trf, memchan and TLS use this to do compression and/or encryption on channels transparent to the user. gzip gets a bit more complicated because you have to stack multiple things to get it right (compression and checksums), but it can still be wrapped in a transparent manner. The same could go for ssl encryption, which is in fact what tclhttpd uses, simply like so: package require http package require tls http::register https 443 \ [list ::tls::socket -require 1 -cafile ./server.pem] set tok [http::geturl https://developer.netscape.com/] That builds on hooks in the http library, but basically says that if we are accessing something on port 443 (default https port), apply the tls::socket transform (stack) onto the channel (with a few options that pertain to SSL stuff). It would be nice to move towards this model in the AOLServer internals. BTW, with the attached gzip.tcl file and Trf (part of ActiveTcl 8.4.1), the following code would create a valid gzip file: package require gzip; # in turn requires Trf set filename example.txt set ifid [open $filename] set ofid [open $filename.gz w] gzip::open $ofid; # actually attaches transforms, writes header fcopy $ifid $ofid gzip::close $ofid; # detaches transforms, writes crc trailer close $ifid close $ofid Consider the gzip open/close could also be named attach/detach. Jeff Hobbs The Tcl Guy Senior Developer http://www.ActiveState.com/ Tcl Support and Productivity Solutions gzip.tcl Description: Binary data
Re: [AOLSERVER] gzip compression
On Sun, 5 Jan 2003, Jim Davidson wrote: > I'm not sure this could be done well in a single, global manner. Instead, > hooks for the three most common request paths directly in the aolserver core > may be needed: [snip] > perhaps this stuff should be a compile time option triggered on with a > --with-gzip= style autoconf thing (ugly as that may be). I wouldn't mind seeing hooks, especially if they can be made more generic; gzip is today's problem, but I'm sure there will be another down the road. It would be nice to be able to have a stack of stream postprocessors, of which gzip compression could be one. Another module to get pushed onto the stack might be rate-limiting (so that a greedy client gets no more than x bytes per second; different from connection throttling, which needs to be done at the front of the request).
Re: [AOLSERVER] gzip compression
In a message dated 1/5/2003 1:45:26 PM Eastern Standard Time, [EMAIL PROTECTED] writes: Before investing a lot of time/energy into storing the gzip results, someone may want to do some benchmarks. For a low-volume site, nothing matters. On a high-volume site, storing a large cache of gzip results on disk will put more pressure on the memory subsystem. If someone decides that storing the gzip results on disk is a good thing, the cache size and all that should probably be a config parameter, and an option for disabling it altogether would be good (cache size of zero maybe). Static file handling would also need to check the mime type and not compress images and other already-compressed data. Ah - good point. Another option is to use some other server for static content which does the right thing, leaving AOLserver for the dynamic parts. That's what we do although I know most folks mix the static and dynamic on the same server. AOLserver's threaded architecture isn't great for static content - I wouldn't expect Apache's multiprocess or newer multithreaded architectures are any better. Single threaded or kernel-based approaches work better. -Jim
Re: [AOLSERVER] gzip compression
Before investing a lot of time/energy into storing the gzip results, someone may want to do some benchmarks. For a low-volume site, nothing matters. On a high-volume site, storing a large cache of gzip results on disk will put more pressure on the memory subsystem. If someone decides that storing the gzip results on disk is a good thing, the cache size and all that should probably be a config parameter, and an option for disabling it altogether would be good (cache size of zero maybe). Static file handling would also need to check the mime type and not compress images and other already-compressed data. Jim ... > > - In Fastpath (i.e., the static file code) do an internal-redirect to a > compressed file on disk, creating it when it doesn't exist or is stale (i.e., > stat both original and compressed each request which should be fine).
Re: [AOLSERVER] gzip compression
In a message dated 1/5/2003 12:58:33 PM Eastern Standard Time, [EMAIL PROTECTED] writes: rename ns_return ns_return_safe rename ns_returnz ns_return Thanks to everyone who has responded thus far - great info! What about having a server config boolean to enable or disable gzip encoding globally for all content? Anyone familiar with how Apache, or other web servers, handle this? I'm not sure this could be done well in a single, global manner. Instead, hooks for the three most common request paths directly in the aolserver core may be needed: - In ADP, set headers and gzip output right after text encoding translation in AdpFlush. In streaming mode (rarely used or useful), ignore. - In Fastpath (i.e., the static file code) do an internal-redirect to a compressed file on disk, creating it when it doesn't exist or is stale (i.e., stat both original and compressed each request which should be fine). - For Tcl procs, modify the various ns_return functions to compress text results as with ADP, ignoring cases where Tcl procs are used to send files and such (better aligning those commands with the Fastpath is general is another low-priority todo). I'm not familiar with how portable/clean/thread-safe the gzip library stuff is - perhaps this stuff should be a compile time option triggered on with a --with-gzip= style autoconf thing (ugly as that may be). -Jim
Re: [AOLSERVER] gzip compression
In a message dated 1/3/03 3:12:12 PM, [EMAIL PROTECTED] writes: rename ns_return ns_return_safe rename ns_returnz ns_return Thanks to everyone who has responded thus far - great info! What about having a server config boolean to enable or disable gzip encoding globally for all content? Anyone familiar with how Apache, or other web servers, handle this? - Nathan
Re: [AOLSERVER] gzip compression
The version from rubylane works flawlessly for us at www.bets4all.com. The only thing we had to patch (as we provide our site in different languages) was a call to NsConnEncodeForOutput for german and spanish special characters. It's in production now for several weeks and we are compressing our main page from up to 200K to less then 20K without hitting the server. It would be nice if someone could check in a version at sourceforge. Wolfgang
Re: [AOLSERVER] gzip compression
I've taken Jim's module and changed it to create the ns_returnz command and a module name of nsreturnz. I also added the original makefile with modifications. It is at http://zmbh.com/rlreturnz/ It is running in AOLserver 3.5.1 at http://zmbh.com:/returnz.tcl $ telnet zmbh.com Host: zmbh.com: Accept-Encoding: gzip, compress should return messed up data. I'm not sure what happens if the ns_returnz command isn't called. Jeff Davis suggested renaming the commands like this: rename ns_return ns_return_safe rename ns_returnz ns_return Jim Wilcoxson wrote: http://www.rubylane.com/public/rlreturnz/rlreturnz.c
Re: [AOLSERVER] gzip compression
Nathan, Jim's module has the changes I was hoping would work, plus removal of unnecessary header settings and additions of ones I guess he found useful from lots of experience. Jim -- Where is the makefile? I'll try it with my version. --Tom Jackson Jim Wilcoxson wrote: Happy New Year Nathan! Check here: http://www.rubylane.com/public/rlreturnz/rlreturnz.c
Re: [AOLSERVER] gzip compression
> John Caruso wrote > >A client of mine is using rlreturnz with AOLserver 3.4.2. We picked > >up the source in May of 2002 from www.theashergroup.com, and it has the > >following difference from yours: > > > That is interesting, because the version I am looking at (at > http://zmbh.com/rlreturnz/ ) uses > Ns_ConnReturnRawData which isn't in the current AOLserver C API. > However, it looks like > Ns_ConnReturnData should do the same thing now. The original code was > written for AOLserver 3.3+ad13 which has the old API. We're using a (locally-generated) port of the 3.3+ad13 patches which applies cleanly against 3.4.2, so that explains why it's working for us as well. BTW, our 3.4.2 encoding patch is available on SourceForge: http://sourceforge.net/tracker/index.php?func=detail&aid=570072&group_id=3152&atid=303152 One of the developers (a CVS wizard) generated it via incremental diffs off of the various Ars Digita source trees. It's been field tested for many months now and seems to work without a hitch. - John
Re: [AOLSERVER] gzip compression
John Caruso wrote A client of mine is using rlreturnz with AOLserver 3.4.2. We picked up the source in May of 2002 from www.theashergroup.com, and it has the following difference from yours: That is interesting, because the version I am looking at (at http://zmbh.com/rlreturnz/ ) uses Ns_ConnReturnRawData which isn't in the current AOLserver C API. However, it looks like Ns_ConnReturnData should do the same thing now. The original code was written for AOLserver 3.3+ad13 which has the old API. Would someone post a link to a source that works with the new API. --Tom Jackson
Re: [AOLSERVER] gzip compression
> I couldn't find it on Google, both sites that had it are gone.
> I moved a copy of it to
> http://zmbh.com/rlreturnz/
>
> Maybe this should get cleaned up and moved into SourceForge?
A client of mine is using rlreturnz with AOLserver 3.4.2. We picked
up the source in May of 2002 from www.theashergroup.com, and it has the
following difference from yours:
- 8< ---
--- rlreturnz-alt.c Fri Jan 3 06:53:51 2003
+++ rlreturnz.c Mon Oct 8 11:02:02 2001
@@ -55,6 +55,9 @@
if (headers != NULL) {
unsigned char *accept = Ns_SetIGet(headers, "Accept-Encoding");
unsigned char *agent = Ns_SetIGet(headers, "User-Agent");
+ if (accept != NULL && strstr(accept, "gzip") != NULL) {
+ do_gzip = 1;
+ }
/* IE 5 has an inexcusably buggy implementation
on a refresh, it will (sometimes) discard about 2K of text! */
if (do_gzip && agent != NULL &&
- 8< ---
Otherwise it's identical.
By the way just echoing what others have said: it works great, and it
hasn't had any apparent negative impact on the web servers. It also
compiles without a hitch (though we're still using relatively old Linux
servers and egcs 1.1.2, so there might be problems with new releases
of gcc).
- John
Re: [AOLSERVER] gzip compression
- Original Message - From: "Dossy" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Friday, January 03, 2003 10:55 AM Subject: Re: [AOLSERVER] gzip compression > On 2003.01.03, Daniel P. Stasinski <[EMAIL PROTECTED]> wrote: > > > > What I would really like to see, at least for static content, is to be > > able to have a gz cache. On first access of any static page, the > > gz'ed content would be cached for subsequent accesses. > > This could be implemented using registered procs or trace filters. > > Would people really use this, though? I think they would.. why recompress static content over and over? if you are going to compress, why not free up CPU cycles when you can as well?
Re: [AOLSERVER] gzip compression
> I moved a copy of it to http://zmbh.com/rlreturnz/ Has anyone used this with OpenACS? --cro
Re: [AOLSERVER] gzip compression
Nathan, I couldn't find it on Google, both sites that had it are gone. I moved a copy of it to http://zmbh.com/rlreturnz/ Maybe this should get cleaned up and moved into SourceForge? --Tom Jackson
Re: [AOLSERVER] gzip compression
On 2003.01.03, Daniel P. Stasinski <[EMAIL PROTECTED]> wrote: > > What I would really like to see, at least for static content, is to be > able to have a gz cache. On first access of any static page, the > gz'ed content would be cached for subsequent accesses. This could be implemented using registered procs or trace filters. Would people really use this, though? -- Dossy -- Dossy Shiobara mail: [EMAIL PROTECTED] Panoptic Computer Network web: http://www.panoptic.com/ "He realized the fastest way to change is to laugh at your own folly -- then you can let go and quickly move on." (p. 70)
Re: [AOLSERVER] gzip compression
We've used a slightly modified version of Jim Wilcoxson's rl_returnz for the past 18 months or so. It's been very stable. gzipping at the level of -3 the additional CPU cycles haven't even been noticable, though the drop in our bandwidth usage has been. On Fri, 3 Jan 2003, Nathan Folkman wrote: > Happy New Year! > > I was wondering if anyone is currently running an AOLserver which delivers > gzipped content to the browser? If so, what module are you using, what kind > of performance impact does this have on the server side, and have you > encountered any browser support issues? Thanks! > > - Nathan >
Re: [AOLSERVER] gzip compression
Happy New Year Nathan! Check here: http://www.rubylane.com/public/rlreturnz/rlreturnz.c If it has a performance impact, we have not noticed it. There are some browser issues, for example, with Netscape 3 on a framed site. My theory is that Netscape's gzip decompression routine has issues when multiple frames are being decompressed and rendered simultaneously. Most of the time a frameset page will display correctly, but occassionally it will goof and crash the browser or render a blank page. There may also be issues with certain version of MSIE for the Mac. Once in a blue moon a Mac user will tell us they cannot use our site, and they all report the same details. But we have staff that uses our site all the time with a Mac and doesn't have problems, so I believe it is only specific versions that have trouble. Disabling gzip compression for MSIE Mac users seemed to help some users, but did not make things work 100%, so it very well could be frame related. Jim > Happy New Year! > > I was wondering if anyone is currently running an AOLserver which delivers > gzipped content to the browser? If so, what module are you using, what kind > of performance impact does this have on the server side, and have you > encountered any browser support issues? Thanks! > > - Nathan > > --part1_da.219d8f87.2b471f53_boundary > Content-Type: text/html; charset="US-ASCII" > Content-Transfer-Encoding: 7bit > > FAMILY="SANSSERIF" SIZE="2">Happy New Year! > > I was wondering if anyone is currently running an AOLserver which delivers gzipped >content to the browser? If so, what module are you using, what kind of performance >impact does this have on the server side, and have you encountered any browser >support issues? Thanks! > > - NathanSIZE="2"> > > --part1_da.219d8f87.2b471f53_boundary-- >
Re: [AOLSERVER] gzip compression
On Fri, Jan 03, 2003 at 12:16:03PM -0500, Nathan Folkman wrote: > Happy New Year! > > I was wondering if anyone is currently running an AOLserver which delivers > gzipped content to the browser? If so, what module are you using, what kind > of performance impact does this have on the server side, and have you > encountered any browser support issues? Thanks! There's rl_returnz (which I really whish had implemented the command as ns_returnz for consistency sake). Jeff Davis modified rl_returnz so it treats encoding correctly and has been using it on his site. It's available at http://xarg.net/code You can test it in the following way: telnet xarg.net 80 GET / HTTP/1.1 Host: xarg.net Accept-Encoding: gzip, compress If you get a bunch of binary output, that's gzipped content. I installed his modified version, but haven't gotten gzipped output, not sure why. -Roberto -- +|Roberto Mello -http://www.brasileiro.net/ |--+ + Computer Science Graduate Student, Utah State University + + USU Free Software & GNU/Linux Club - http://fslc.usu.edu/ + Microsoft has been doing a really bad job on their OS - Linus Torvalds
Re: [AOLSERVER] gzip compression
Nathan, Happy New Year! Mydomain.com used to run a module called rlreturnz written by Petru Paler. When we upgraded AOLserver to 3.4.2, I couldn't compile it, but it ran for several years with no problems reported. I think the compile problem was a slight change in the C api, so it should be easy to fix, and maybe someone has. theashergroup had a copy and Petru asked me to look there for an update. --Tom Jackson Nathan Folkman wrote: Happy New Year! I was wondering if anyone is currently running an AOLserver which delivers gzipped content to the browser? If so, what module are you using, what kind of performance impact does this have on the server side, and have you encountered any browser support issues? Thanks! - Nathan
Re: [AOLSERVER] gzip compression
I only saw one module available and from what I understand it is broke with current versions. What I would really like to see, at least for static content, is to be able to have a gz cache. On first access of any static page, the gz'ed content would be cached for subsequent accesses. Daniel P. StasinskiSoftware EngineerMayor Pharmaceutical Laboratories[EMAIL PROTECTED] I was wondering if anyone is currently running an AOLserver which delivers gzipped content to the browser? If so, what module are you using, what kind of performance impact does this have on the server side, and have you encountered any browser support issues? Thanks!
[AOLSERVER] gzip compression
Happy New Year! I was wondering if anyone is currently running an AOLserver which delivers gzipped content to the browser? If so, what module are you using, what kind of performance impact does this have on the server side, and have you encountered any browser support issues? Thanks! - Nathan
