On Mon, 20 May 2002 22:47:21 -0700 (PDT) Andrew Daviel <[EMAIL PROTECTED]> wrote:
> > (I haven't really been following so this may not be news. > Couldn't see"google" in a recent subject line though.) > > FYI, Google is blocking LWP. Part of an effort to discourage > parasite search engines I think, i.e. claiming google's results > as your own, or trying to figure out the "secret sauce". > > Google gives a status 403 forbidden error if User-Agent is set > to lwp-request. Using the /usr/bin/GET from lwp-request.PL > v1.35(OK, probably a bit old...) the body is stripped off and > lost, although RFC 2068 states that the reason may be given in > the body and in fact it is, as may be seen using a different > client but using the same user-agent header. Basicially it says > to read the terms of service at > http://www.google.com/terms_of_service.html and implies that if > you are writing a proxy or something they'll be glad to help. Use the api. It works great. http://www.google.com/apis/ -Tim
