Hi, we are trying to upgrade a newer version of wget (1.5.1 -> 1.9.1), but for some reason updating/mirroring of a ftp site is failing with the new version.
We tried to tweak alomst any wget option to get it running again. What what are we doing, what happens: We are trying to do a daily (incremental mirror) of a ftp site using a chain of proxies. Whatever we do, the new wget is just downloading the "index.html" for the directory and then is stopping. Wget does not try to check, if there are changes in a subtree (or in the current directory). This worked for version 1.5.1 fine, but 1.5.1 has (IMO) a timestamp bug: Downloading a file, which was changed within a short timeperiod seems to fail, because wget seems to ignore timezone differences. Maybe this happens, if wget doesn't get timestamps for a file. So I would like to propose to options: - --resume-on-errors=count (count = to prevent endless loops) - some more incremental mirror handling options Please see the wget output for the problem. Is there any way to solve this problem using wget? Tnx Rainer Example: ------------------------------------- snip ------------------------ (antivirus-test)# /var/ftpd/sbin/wget-1.9.1 -v -S -c -m --cache=off -np -P /var/ftpd/pub/antivirus-test/ --follow-ftp -d ftp://ftp.nai.com/pub/antivirus/datfiles/4.x/ DEBUG output created by Wget 1.9.1 on solaris2.8. --13:48:03-- ftp://ftp.nai.com/pub/antivirus/datfiles/4.x/ => `/var/ftpd/pub/antivirus-test/ftp.nai.com/pub/antivirus/datfiles/4.x/index.h tml' Resolving proxy1.abcdef.com... 10.105.193.55 Caching proxy1.abcdef.com => 10.105.193.55 Connecting to proxy1.abcdef.com[10.105.193.55]:8080... connected. Created socket 4. Releasing 56ef0 (new refcount 1). ---request begin--- HEAD ftp://ftp.nai.com/pub/antivirus/datfiles/4.x/ HTTP/1.0 User-Agent: Wget/1.9.1 Host: ftp.nai.com Accept: */* Range: bytes=11373- Pragma: no-cache ---request end--- Proxy request sent, awaiting response... HTTP/1.0 200 OK Server: Squid/2.4.STABLE2 2 Server: Squid/2.4.STABLE2Mime-Version: 1.0 3 Mime-Version: 1.0Date: Tue, 13 Jan 2004 12:34:43 GMT 4 Date: Tue, 13 Jan 2004 12:34:43 GMTContent-Type: text/html 5 Content-Type: text/htmlX-Cache: MISS from XXXXXXXXXXXXXXXX 6 X-Cache: MISS from XXXXXXXXXXXXXXXXX X-Cache: MISS from XXXXXXXXXXXXXXXXXX 7 X-Cache: MISS from XXXXXXXXXXXXXXXXXXXXXXXProxy-Connection: close 8 Proxy-Connection: close Continued download failed on this file, which conflicts with `-c'. Refusing to truncate existing file `/var/ftpd/pub/antivirus-test/ftp.nai.com/pub/antivirus/datfiles/4.x/index.h tml'. Closing fd 4 FINISHED --13:48:05-- Downloaded: 0 bytes in 0 files ------------------------------------- snap ------------------------- --- Rainer Scherg BRI/TDV6 - Internet,IntraNet,eMail Industrial Hydraulics Bosch Rexroth AG "The Drive & Control Company" 97813, Lohr am Main Phone: +49-(0)9352-18-1510 Fax: +49-(0)9352-18-1500 www.boschrexroth.de
