Andrew M. Bishop wrote:
Ian Stirling <[EMAIL PROTECTED]> writes:
So, naturally, when I realised a need for a possibly related bit of software,
and lacking results from google. I started wondering if anyone had thought of,
or knows of an implementation or proof of concept with wwwoffled.
Basically, it's a two part web proxy to drastically reduce web usage bandwidth.
One part resides on a mobile device with a (usually) poor bandwidth link, but
relatively large amount of storage, that may occasionally be plugged into a high
speed network.
The other part is on server, connected via a fast connection to the internet.
Have you seen the (no longer maintained) rsync-over-HTTP protocol that
is at http://rproxy.samba.org/. This combines rsync and HTTP to be
able to only transfer the updates just like you want.
They describe just what you are looking for, but since nobody was
interested about using it they seem to have given up.
Interesting, thanks!
Googling had found me little.
I remembered having seen the combination of rsync and HTTP before so
it was easy for me to find.
Before you ask, I don't plan to add it into WWWOFFLE.
Initial design was planned to be a little server on host and client, that
implemented this by changing the files in the wwwoffle cache, rather than
properly implementing stuff.
If eventually I generated a patch for WWWOFFLE, that mostly 'just worked',
without greatly complicating the core logic, are there any circumstances in
which it might be included?
The reason that I say I don't plan to add it in to WWWOFFLE is that
this is my standard reply when people ask me to extend WWWOFFLE way
beyond its primary task.
Fair enough.
What you are suggesting is a pair of proxies, one each end of the low
bandwidth link, that act together to minimise the data flowing over
that link. The one on the internet end of the link (not the client
end) could get new pages from the internet, or from WWWOFFLE or use
the WWWOFFLE cache directly.
<snip>
There seem to be very few cases where a user would have access to an
rsync supporting proxy at the the internet end to make this useful.
This is basically the sole thing I'm targetting.
You have a mobile phone (or dialup), with a slow or expensive connection
to the internet.
You choose to install software on your phone and home computer (or a
server farm, or in principle a paid service providing this cache) to
reduce your bills, or speed up your internet.
Webservers supporting rsync extensions is irrelevant, as you get better
performance if it all goes through one proxy cache - as it can know
exactly what files you have.
As a goal, it's nice, but expecting ebay/microsoft/google/... to deploy
it in the near term is unlikely.
The one user proxy really is quite simple.
Where it gets rather more complex is a public proxy with a few dozen users.
Then rather more with a few thousand.
Initial proof of concept would be basically using two stock WWWOFFLEs,
and a couple of tiny servers that poke about in the cache.
Then it would be nice to teach wwwoffle how to store, list and manage
versioned pages (by date and possibly hash).
The next steps would be of limited general use - adding the protocol to
transfer compressed differentials, searching for similar pages with
different URLs already in cache to diff from, understanding
compressed/shortened requests, and responding in similar vein.
Thanks for the comments!
