Re: [Bug-wget] -e robots=off problem

2011-06-20 Thread Micah Cowan
On 06/12/2011 06:34 AM, david chou wrote: > If there is a robots file , > how could I retrieve file from > that site? I 've used the -e robots=off instruction , > but it still can not mirror a web site. Use the --debug flag to diagnose, and compare the output with http://wget.addictivecode.org/Fr

[Bug-wget] -e robots=off problem

2011-06-12 Thread david chou
If there is a robots file , how could I retrieve file from that site? I 've used the -e robots=off instruction , but it still can not mirror a web site.