Re: [us...@httpd] disable wget-like user-agents

2011-01-05 Thread Igor Galić
- Mark Montague m...@catseye.org wrote: On January 4, 2011 22:32 , Carlos S neu...@gmail.com wrote: Recently I was trying to download a package using wget, but the website prevented access to it. I tried --user-agent option but it didn't work either. So I was curious to know what

[us...@httpd] disable wget-like user-agents

2011-01-04 Thread Carlos S
Is there any way to disable download/traffic from wget-like user agents? Can this be done using user-agent string? Any documentation link or example will be really helpful. -- cs. - The official User-To-User support forum of the

Re: [us...@httpd] disable wget-like user-agents

2011-01-04 Thread Mark Montague
On January 4, 2011 18:03 , Carlos S neu...@gmail.com wrote: Is there any way to disable download/traffic from wget-like user agents? Can this be done using user-agent string? Any documentation link or example will be really helpful. Follow the example below, but use only the user agent

Re: [us...@httpd] disable wget-like user-agents

2011-01-04 Thread Doug McNutt
At 18:19 -0500 1/4/11, Mark Montague wrote: Follow the example below, but use only the user agent condition, omit the IP condition, and suitably adjust the RewriteRule regular expression to match the URL(s) you wish to block:

Re: [us...@httpd] disable wget-like user-agents

2011-01-04 Thread Carlos S
Thanks for the links Mark and Doug. The webscrapers thing looks interesting.. I had looked at mod_rewrite and User-Agent header solution. Recently I was trying to download a package using wget, but the website prevented access to it. I tried --user-agent option but it didn't work either. So I

Re: [us...@httpd] disable wget-like user-agents

2011-01-04 Thread Mark Montague
On January 4, 2011 22:32 , Carlos S neu...@gmail.com wrote: Recently I was trying to download a package using wget, but the website prevented access to it. I tried --user-agent option but it didn't work either. So I was curious to know what strategy this web admin must have implemented.