Hi, On Thu, Apr 20, 2017 at 12:10:42PM -0700, Dima Pasechnik wrote: > > > > Strange, well, I created a firewall rule blocking that IP for now and > > everything seems to be fine. > > > > Does this mean that we need some robots.txt somewhere, perhaps after some > restructuring, > which would protect expensive resources from this sort of overload?
If the "culprit" host does not announce itself as a friendly bot (look at the webserver access logs), i doubt it will kindly follow what is suggested in robots.txt. Firewalling is the way to go, it is at the lowest level, hence much lighter than application-level solutions (.htaccess or whatever). Ciao, Thierry -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at https://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
