After a long time without any new versions of WWWOFFLE I have finally
reached the point where I think that the code is good enough for the
brave few to try.
I don't know of any specific problems with this version, but I can't
test all of the options in the configuration file and all the
different ways of using it. The best I can say is that "it works for
me" and I hope that it works for you as well.
The particular new features in this version that you might want to
test out are:
Chunked encoding to/from clients/servers
Better handling of various compressed formats
These two together make the largest change in the WWWOFFLE
source code with a complete re-write of the low level file and
network read and write functions. The changes to the low
level functions have caused changes throughout the program
code. As far as I can tell the chunked encoding transfer from
servers and to clients is robust and the handling of
compressed data is much better with 4 formats now recognised
(gzip and deflate as defined in the HTTP specification,
deflate as everybody seems to use it and a weird hybrid sent
by some buggy servers).
Better HTML output
The HTML code that WWWOFFLE produces from its internal web
pages is now all HTML 4.01 compliant and should validate in
any HTML validator. Also included in this version is a
stylesheet that can be edited to change the appearance of the
web pages to some extent.
Removal of temporary files
No web pages from the server to the client through WWWOFFLE
are now stored in temporary files. This means that page
downloading will be faster if you use WWWOFFLE when online.
You can enable the HTML modifications and not have to wait for
the server to send the page to WWWOFFLE before WWWOFFLE sends
it to the browser.
The side effect of this is that there are no Content-Length
headers from WWWOFFLE to the clients any more. This means
that web browsers will not be able to tell you what fraction
of the page has been downloaded.
If you try this version then you will need to edit your configuration
file. The default location for the configuration file is now
/etc/wwwoffle/wwwoffle.conf instead of /etc/wwwoffle.conf. The other
thing that needs changing is that the enable-modify-online option is
no longer valid due to the removal of temporary files. You can run
the upgrade-config.pl script that is in the conf directory to update
your configuration file contents to the latest version.
The source code can be downloaded from the web:
http://www.gedanken.freeserve.co.uk/download-wwwoffle/wwwoffle-2.8-beta.tgz
The WWWOFFLE version 2.8 web-pages are also in place (although there
is not much to see yet):
http://www.gedanken.demon.co.uk/wwwoffle/version-2.8/
The beta testing period will last about a month, depending on the
amount of bugs that are found and the amount of time that I have to
fix them.
I am interested to have your feedback during the beta testing period,
good or bad, bug reports and unexpected behaviour in particular. Send
bug reports direct to me and use this mailing list only for discussion
of the new features.
The changes in this version compared to version 2.7h are listed below.
-------------------- NEWS --------------------
Bug Fixes:
Fix some small memory leaks. Purge unmatched O* and U* files from outgoing.
Improve spool error messages. Validate the -d option in wwwoffled. Keep the
config file permissions when writing new one. Don't call freeaddrinfo(NULL).
Running 'wwwoffle URL' when online now actually fetches the URL. Improve
lexical analyser code for EOF condition, speed and new version of flex. Remove
some lockfile race conditions. Better handling of non-ASCII URLs when parsing.
The info page for a URL now shows all links. Index sorting by file type is
case insensitive. Handle & in HTML tags like '&'. Better memory freeing
in certain cases. Make the wwwoffle -[drR] options handle spaces before
number. Allow wwwoffle program to request recursive fetching of depth 0. FTP
requests with passwords work like HTTP. Running 'wwwoffle http://www/bar#foo'
now does the right thing. Correctly handle recursive fetch options. Running
'wwwoffle http://aaa:[EMAIL PROTECTED]/bar' now does the right thing. Allow files on
the local web server with spaces in them. Fix overwriting of old error message
with page. Remove title attributes from DontGet images when modifying. Check
form entries for unwanted whitespace.
New Features:
Chunked encoding from servers and to clients is now possible.
Changed all HTML output to HTML 4.01 DTD and validated most output pages.
Added in a WWWOFFLE stylesheet and some styles to the interal web pages.
Removed the use of temporary files between server/cache and client.
Added a parser for CSS (Stylesheets) to detect included files and images.
The default directory for configuration files is /etc/wwwoffle.
Remove the enable-modify-online option (no penalty for modifying online).
Handle deflate compression (the commonly implemented but wrong version).
Guess the compression type coming from servers (don't believe the headers).
Stop infinite recursion when following Location headers or Meta Refresh tags.
Stop infinite recursion if images are actually HTML (e.g. non-404 error pages).
Fetch images etc in pages with error status codes.
New Options
Added options to enable chunked encoding from servers and to clients.
Added an option to disable the use of Etag as a cache validator.
Added an option to force the insertion of a User-Agent header.
Added an option to not make conditional requests to specified hosts.
Added an option to fetch the favourite/shortcut icons automatically.
The *-no-cache options now appear in OnlineOptions and OfflineOptions.
Add an option to disable cookies from being set by HTML meta tags.
Programs:
Added '-g' option to wwwoffle to fetch no images, stylesheets etc.
*NOTE* The enable-modify-online option has been removed (there is now no extra
penalty to pay to perform modifications online compared to offline)
*NOTE* The default location for the configuration file(s) is now /etc/wwwoffle/.
-------------------- NEWS --------------------
--
Andrew.
----------------------------------------------------------------------
Andrew M. Bishop [EMAIL PROTECTED]
http://www.gedanken.demon.co.uk/
WWWOFFLE users page:
http://www.gedanken.demon.co.uk/wwwoffle/version-2.8/user.html
