On Mon, 04 Mar 2002, Andy Rabagliati wrote: > > For example if the first requestor asks for a particular page it will > > be fetched as will the images. The second requestor then asks for the > > same page. The WWWOFFLE fetch process doesn't fetch it again because > > it is already up to date (due to the first request), but the page that > > is in the cache is newer than the one that the second requestor has. > > All client requested pages be fed back to the client. When processing > the 'wwwoffle -fetch' output, I also note when I see log lines like :- > > ($_ =~ m%^Not fetching http://([^/]+)/(.*) .Page Unchanged.$%) > > and pack these up as well.
I have found a workable solution for this. As remote requestor batches arrive, I wait on a lockfile (to serialise), unpack the requests, fetch and parse the fetch output, pack up the files, and pass them back (via UUCP). Now my problem is as follows :- The remote (un-connected) wwwoffled server does not recognise the unpacked pages. If I re-request the same pages, I expect it to find them in the cache, where I unpacked them. [root@esangweni wwwoffle]# ls -la http/news.bbc.co.uk/ total 68 drwxrwxrwx 2 uucp uucp 4096 Mar 20 12:02 . drwxrwxr-x 11 wwwoffle uucp 4096 Mar 20 12:02 .. -rw-rw-r-- 1 uucp uucp 11516 Mar 19 17:31 DajFGKPX5ggcKWJ8lfdYxQA -rw-rw-r-- 1 uucp uucp 3213 Mar 20 11:13 Dh6OtFmVQUW15Tr1X6nHNkQ -rw-rw-r-- 1 uucp uucp 3284 Mar 20 11:13 Dkr1U8LMnEvgZ+wuUkF3OxQ -rw-rw-r-- 1 uucp uucp 14434 Mar 20 12:05 DkuAAiJm7MiuHw9aecXwXyw -rw-rw-r-- 1 uucp uucp 3262 Mar 20 12:05 Do5beYFdjhsKAgrv+AW5+Jw -rw-rw-r-- 1 uucp uucp 17494 Mar 19 17:31 DsSiiyoVCf9IdProuiDqXsA [root@esangweni wwwoffle]# ls -la outgoing/ total 16 drwxr-x--- 2 wwwoffle uucp 4096 Mar 20 12:13 . drwxr-xr-x 15 wwwoffle uucp 4096 Mar 19 17:05 .. -rw-rw-r-- 1 wwwoffle uucp 695 Mar 20 12:13 OkuAAiJm7MiuHw9aecXwXyw -rw-rw-r-- 1 wwwoffle uucp 58 Mar 20 12:13 UkuAAiJm7MiuHw9aecXwXyw The server runs as user wwwoffle / group uucp. As can be seen, there are now outgoing requests for the same URLs that are in the cache. The wwwoffled server debug trace on the fetch above looks like this :- [root@esangweni wwwoffle]# /usr/sbin/wwwoffled -d 5 -c /etc/wwwoffle.conf wwwoffled[13908] Important: WWWOFFLE Demon Version 2.6d (with zlib,without ipv6) started. wwwoffled[13908] Information: WWWOFFLE Read Configuration File '/etc/wwwoffle.conf'. wwwoffled[13908] Information: Running with uid=100, gid=14. wwwoffled[13908] Information: HTTP Proxy connection from host dhcp68 (172.16.1.68). wwwoffles[13909] Information: URL='http://news.bbc.co.uk/low/english/world/africa/default.stm'. wwwoffles[13909] Debug: proto='http'; host='news.bbc.co.uk'; path='/low/english/world/africa/default.stm'; args='(null)'; user:pass='(null):(null)'. wwwoffles[13909] Information: Cache Access Status='New Page'. wwwoffled[13908] Information: Forked wwwoffles -spool (pid=13909). wwwoffled[13908] Information: Child wwwoffles exited with status 0 (pid=13909). wwwoffled[13908] Debug: Currently running: 0 servers total, 0 fetchers. Why doesn't it see the pages in the cache ? It does have read access. My config file on this server has the following in the Offline options (it is always offline) OfflineOptions { pragma-no-cache = yes confirm-requests = no } Thanks for your help, Cheers, Andy!
