Rajesh wrote: > Thanks for your reply. I have tried using the command wget > --user-agent="Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)", but it didn't > work.
Adding the user agent helps some people -- I think most often with web servers from the evil empire. > I have one more question. In each directory I have a welcome.cfm file on the > main server (DirectoryIndex order is welcome.cfm welcome.htm welcome.html > index.html). But, when I run wget on the mirror server, wget renames welcome.cfm > to index.html and downloads to mirror server. > > Why does it change the file name from welcome.cfm to index.html. It appears to me that wget assumes that the result of getting a directory (such as http://www.sl.nsw.gov.au/collections/) is index.html. (See the debug output below.) > > How can I mirror a web site using scp?? I can only copy one file at a time using > scp. The following works for me: scp [EMAIL PROTECTED]:path/to/directory/* -r ********** The promised debug output: wget http://www.sl.nsw.gov.au/collections --debug DEBUG output created by Wget 1.8.1 on linux-gnu. --20:16:36-- http://www.sl.nsw.gov.au/collections => `collections' Resolving www.sl.nsw.gov.au... done. Caching www.sl.nsw.gov.au => 192.231.59.40 Connecting to www.sl.nsw.gov.au[192.231.59.40]:80... connected. Created socket 3. Releasing 0x810dc38 (new refcount 1). ---request begin--- GET /collections HTTP/1.0 User-Agent: Wget/1.8.1 Host: www.sl.nsw.gov.au Accept: */* Connection: Keep-Alive ---request end--- HTTP request sent, awaiting response... HTTP/1.1 301 Moved Permanently Date: Fri, 04 Jul 2003 03:16:36 GMT Server: Apache/1.3.19 (Unix) Location: http://www.sl.nsw.gov.au/collections/ Connection: close Content-Type: text/html; charset=iso-8859-1 Location: http://www.sl.nsw.gov.au/collections/ [following] Closing fd 3 --20:16:37-- http://www.sl.nsw.gov.au/collections/ => `index.html' Found www.sl.nsw.gov.au in host_name_addresses_map (0x810dc38) Connecting to www.sl.nsw.gov.au[192.231.59.40]:80... connected. Created socket 3. Releasing 0x810dc38 (new refcount 1). ---request begin--- GET /collections/ HTTP/1.0 User-Agent: Wget/1.8.1 Host: www.sl.nsw.gov.au Accept: */* Connection: Keep-Alive ---request end--- HTTP request sent, awaiting response... HTTP/1.1 200 OK Date: Fri, 04 Jul 2003 03:16:37 GMT Server: Apache/1.3.19 (Unix) Connection: close Content-Type: text/html; charset=iso-8859-1 Length: unspecified [text/html] [ ] 21,284 20.83K/s Closing fd 3 20:16:38 (20.83 KB/s) - `index.html' saved [21284]
