> Le 7 juin 2017 à 13:44, Gautier de Montmollin <gdem...@hotmail.com> a écrit : > > Thanks a lot! > You probably guessed well, these kind of robots are not so polite and will > ignore robots.txt. > The good ones will visit once a day and respect robots.txt... > > So I'll have a look at Gnoga.Gui.Navigator. > Perhaps I can figure out where the request come from and at least limit the > number of requests from the same IP address during a certain amount of time > T2 (say 1 day) if there were too many in a row within a time frame of length > T1 (say 1 minute). It is what search engines do.
Hello Gautier, I've added some enhanced logs with user agent, platform and IP of client browser: https://sourceforge.net/p/gnoga/code/ci/b8900663d4f1e5b13e60f8bd29540155018f79e2/ HTH, Pascal. http://blady.pagesperso-orange.fr _______________________________________________ Gnoga-list mailing list Gnoga-list@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/gnoga-list
