Andrew,
Thanks for wwwoffle, and for taking the time to address our issues.
There are a number of things we are trying to fix, with different
priorities.
On Sun, 03 Mar 2002, Andrew M. Bishop wrote:
> Andy Rabagliati <[EMAIL PROTECTED]> writes:
>
> > Is there a better way to do it ?
>
> I think that serializing the requests is the best way of doing it
> I don't agree with your objection to it and it solves another problem.
> I don't see how it can be any slower to serialize.
If two schools are 'surfing' one of the sites may be slow, thus not
using the full bandwidth available to the fetching machine ? We are not
in the USA. The 'max-servers' parameter could be upped, though.
> If you have more than one fetch occuring in parallel another problem
> that you get is that you end up with an inconsistent state. A second
> fetch may be updating one of the pages while you are selecting the
> modified pages from the cache. This could see you sending an
> incomplete page back to the first requestor.
This could be a problem.
> For example if the first requestor asks for a particular page it will
> be fetched as will the images. The second requestor then asks for the
> same page. The WWWOFFLE fetch process doesn't fetch it again because
> it is already up to date (due to the first request), but the page that
> is in the cache is newer than the one that the second requestor has.
All client requested pages be fed back to the client. When processing
the 'wwwoffle -fetch' output, I also note when I see log lines like :-
($_ =~ m%^Not fetching http://([^/]+)/(.*) .Page Unchanged.$%)
and pack these up as well.
It would be nice not pass the client wwwoffle copies of pages it already
has. This requires keeping a great deal of state on the fetching machine -
your suggestion being a complete duplicate cache. The bandwidth
overhead of sending duplicate copies is less onerous, as it is our own
802.11b transport.
I am prepared to forgo this at this moment. Serialising the requests
would break this requirement.
> I think that the best way of doing this (if you can afford the disk
> space) is to run multiple WWWOFFLE servers on the well connected
> machine.
>
> To reduce the total network bandwidth on the well connected site you
> can make all of the WWWOFFLE proxies go through another proxy so that
> you don't need to fetch common pages more than once.
So it seems that to add multiple outgoing queues might improve speed,
at the expense of internal state breakage.
Serialising solves all problems except some extra traffic to the client.
I will do some performance measurements on serialisation.
Thanks for your suggestions.
> --
> Andrew.
Cheers, Andy!
