Andrew M. Bishop wrote (subject was: Re: Bug#140617: wwwoffle: zlib problem with
groups.yahoo.com):
> We have discussed before on this list servers that cannot send
> correctly compressed data using the deflate method. It does seem
> problematic for some people.
I've had a problem with the website abcnews.go.com that also turned to be
deflate-compression related.
When I use Mozilla the site displays OK, but the Internet Explorer users on my
LAN got a blank page.
It turns out the server recognizes Mozilla by the User-Agent string and avoids
sending compressed data in that case. It does "deflate" the content for
other browsers, but WWWOFFLE can't "inflate" it.
Putting "<http://*abcnews*.com> request-compressed-data = no" in the
configuration file solves the problem, but I do not find it totally
satisfactory, because it turns out that gzip compression does work correctly in
this case.
What I needed was being able to specify something like
"<http://*abcnews*.com> request-compressed-data = gzip" instead, so that I can
still enjoy the benefits of compression.
It turned out not to be too difficult to implement. I've attached a patch to
this message for people who want to try it out for themselves.
After patching and recompiling the WWWOFFLE source, you should be able to
specify "[<URL-SPEC>] request-compressed-data = yes|no|gzip|deflate" in your
config file. "yes" and "no" give the standard WWWOFFLE behaviour, "gzip" makes
WWWOFFLE only request gzip type compression, etc.
You will need to make the changes to the config file using a text editor. Some
further changes need to be made to the files README.CONF.txt and
ConfigurationItem-Head.html in the directory /var/spool/wwwoffle/html/ before
the interactive configuration editing webpages work correctly for this modified
option, but I'm still working on that.
Can someone give me an example of a site that deflates its content correctly, so
that WWWOFFLE can inflate it?
Maybe it sounds a bit cynical, but up to now I have only seen examples to the
contrary.
--
Paul A. Rombouts <[EMAIL PROTECTED]>
--- wwwoffle.orig/wwwoffle-2.7b/src/configdata.c Fri May 3 23:43:25 2002
+++ wwwoffle.test/wwwoffle-2.7b/src/configdata.c Fri Jun 14 16:16:56 2002
@@ -197,7 +197,7 @@
{"intr-download-size" ,&IntrDownloadSize ,1,0,Fixed,FileSize ,"1" },
{"intr-download-percent" ,&IntrDownloadPercent ,1,0,Fixed,Percentage,"80" },
{"timeout-download-keep" ,&TimeoutDownloadKeep ,1,0,Fixed,Boolean ,"no" },
- {"request-compressed-data",&RequestCompressedData,1,0,Fixed,Boolean ,"yes"}
+ {"request-compressed-data",&RequestCompressedData,1,0,Fixed,CompressSpec,"yes"}
};
/*+ OnlineOptions section. +*/
--- wwwoffle.orig/wwwoffle-2.7b/src/configmisc.c Fri May 3 23:43:25 2002
+++ wwwoffle.test/wwwoffle-2.7b/src/configmisc.c Fri Jun 14 16:14:33 2002
@@ -292,6 +292,7 @@
case UserId:
case GroupId:
case FileMode:
+ case CompressSpec:
break;
/* String */
@@ -482,6 +483,8 @@
return "UserId"; /* val */
case GroupId:
return "GroupId"; /* val */
+ case CompressSpec:
+ return "CompressionSpecification"; /* var */
case String:
return "String"; /* key */ /* val */
case PathName:
@@ -790,6 +793,12 @@
}
break;
+ case CompressSpec:
+ return strdup((key_or_val.integer==0 ) ? "no":
+ (key_or_val.integer==1 ) ? "deflate":
+ (key_or_val.integer==2 ) ? "gzip":
+ "yes");
+
/* String */
case String:
--- wwwoffle.orig/wwwoffle-2.7b/src/configpriv.h Fri May 3 23:43:25 2002
+++ wwwoffle.test/wwwoffle-2.7b/src/configpriv.h Fri Jun 14 16:08:48 2002
@@ -104,6 +104,8 @@
UserId, /*+ For user IDs, (numeric or string). +*/
GroupId, /*+ For group IDs, (numeric or string). +*/
+ CompressSpec, /*+ For compression specifiers (0=identity,
+1=deflate, 2=gzip, 3= wwwoffle default) +*/
+
String, /*+ For an arbitrary string. +*/
PathName, /*+ For pathname values (string starting with '/').
+*/
--- wwwoffle.orig/wwwoffle-2.7b/src/configrdwr.c Fri May 3 23:43:25 2002
+++ wwwoffle.test/wwwoffle-2.7b/src/configrdwr.c Fri Jun 14 16:11:01 2002
@@ -1157,6 +1157,21 @@
}
break;
+ case CompressSpec:
+ if(!*text)
+ {errmsg=(char*)malloc(48);strcpy(errmsg,"Expecting a CompressionSpecifier, got
+nothing.");}
+ else if(!strcasecmp(text,"no") || !strcasecmp(text,"false") ||
+!strcasecmp(text,"none"))
+ pointer->integer=0;
+ else if(!strcasecmp(text,"deflate"))
+ pointer->integer=1;
+ else if(!strcasecmp(text,"gzip"))
+ pointer->integer=2;
+ else if(!strcasecmp(text,"yes") || !strcasecmp(text,"true"))
+ pointer->integer=3;
+ else
+ {errmsg=(char*)malloc(48+strlen(text));sprintf(errmsg,"Expecting a
+CompressionSpecifier, got '%s'.",text);}
+ break;
+
case String:
if(!*text || !strcasecmp(text,"none"))
pointer->string=NULL;
--- wwwoffle.orig/wwwoffle-2.7b/src/wwwoffles.c Fri May 3 23:43:25 2002
+++ wwwoffle.test/wwwoffle-2.7b/src/wwwoffles.c Fri Jun 14 16:06:58 2002
@@ -1486,11 +1486,17 @@
/* Add the compression header */
#if USE_ZLIB
- if(ConfigBooleanURL(RequestCompressedData,Url) && !NotCompressed(NULL,Url->path))
- {
- request_compression=1;
- AddToHeader(request_head,"Accept-Encoding","deflate; q=1.0, x-gzip; q=0.9,
gzip; q=0.9, identity; q=0.1");
- }
+ {
+ int compress_spec=ConfigIntegerURL(RequestCompressedData,Url);
+ if(compress_spec && !NotCompressed(NULL,Url->path))
+ {
+ request_compression=1;
+ AddToHeader(request_head,"Accept-Encoding",
+ (compress_spec==1) ? "deflate; q=1.0, identity; q=0.1":
+ (compress_spec==2) ? "x-gzip; q=1.0, gzip; q=1.0, identity;
+q=0.1":
+ "deflate; q=1.0, x-gzip; q=0.9, gzip;
+q=0.9, identity; q=0.1");
+ }
+ }
#endif
}
