The problem is I am getting tens of thousands of these requests per day and they ARE impacting my quota and they come from different IPs tho most seem to be in Europe. They are all curl requests so they are not genuine.
I'll have to move to a new ISP who can help me in blocking them as I can't afford this. My "real" requests are about .01% of these curl requests! On Thursday, August 2, 2012 6:45:04 PM UTC-4, Jay wrote: > > HEAD requests like this often come from social sites or search engines to > test a url. This may or may not be the case for you, but that is my > experience. > > Depending on the framework you are using , you should be able to handle > HEAD requests if you want to. > > Handling the request, even if it errors does consume some resources, so > there is likely some small impact to quota. If you don't want the systems > hitting your site, probably the best option is robots.txt > > On Wednesday, August 1, 2012 3:02:02 PM UTC-5, Kate wrote: >> >> Thanks for that, but these requests are coming from many different ip >> addresses and I can see no pattern to them. >> >> I am just wondering why there are so many and if it is affecting my quota. >> >> Also I do not have a main.py. I have an index.py. I'm wondering now how >> it works at all! >> >> >> On Wednesday, August 1, 2012 3:36:55 PM UTC-4, Joshua Smith wrote: >>> >>> Looks to me like somebody is doing head requests to your app using curl. >>> curl is a command line program that downloads web pages, similar to wget. >>> This particular user is coming from linux, possibly from here: Portugal, >>> Braga, Universidade do Minho (although reverse IP search results aren't >>> always accurate). >>> >>> If I were you, I'd just ignore the errors. You aren't in business to >>> make some college kid's half-baked web scraper work correctly. >>> >>> However, if you do want to handle the HEAD requests, you just add a >>> method analogous to the post or get handlers you have implemented already >>> in your main.py. >>> >>> -Joshua >>> >>> On Aug 1, 2012, at 3:26 PM, Kate <mss.ka...@gmail.com> wrote: >>> >>> > >>> > I can't work out why I am getting these errors. In 13 hours I have >>> 26,205 of them. I have no forms in my code. >>> > >>> > I do have urlfetch >>> > >>> > If as I have been told the error means that the head method is noot >>> implemented, how do I set up my app to support the Head methiod? I cannot >>> find anywhere I am using it, >>> > >>> > Here is an example of the error message. >>> > >>> > 2012-08-01 14:25:52.668 / 405 51ms 0kb curl/7.18.2 >>> (i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3 >>> libidn/0.6.14 libssh2/0.18 >>> > >>> > 193.136.19.13 - - [01/Aug/2012:12:25:52 -0700] "HEAD / HTTP/1.1" 405 >>> 124 - "curl/7.18.2 (i386-redhat-linux-gnu) libcurl/7.18.2 >>> NSS/3.12.2.0zlib/1.2.3 libidn/0.6.14 libssh2/0.18" " >>> aussieclouds.appspot.com" ms=51 cpu_ms=0 api_cpu_ms=0 cpm_usd=0.000045 >>> instance=00c61b117c01c1a2dec223c75dcfeaab60fbd7 >>> > >>> > Kate >>> > >>> >>> >>> -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/9NUDM-mRDTcJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
