On Thu, 27 Apr 2017 08:07:02 +0200 Jonathan Aquilina <[email protected]> said:
> Hi Carsten, > > What advantage does http for git have? Isn't git better to use to pull > and push stuff to the repositories or is cgit more a web based ui to > view the repository history etc? it's an advantage for people behind firewalls that only allow outside access via http proxies for example. > --- > Regards, > > Jonathan Aquilina > > On 2017-04-26 04:18, Carsten Haitzler wrote: > > > I've had to disable the whole http support for now for git.enlightenment.org > > because several bots are crawling it causing our VM to basically be loaded > > with 10-20 cgit cgi's running git queries for history etc. continually. I/O > > and system load is going through the roof as a result and causing other > > stuff like phab to crawl and begin timing out. > > > > So anyone using HTTP for doing cmdline git stuff is, at this moment, going > > to find things not working. SSH and GIT protocol should still work. I'll > > keep this shut down for a few hours hoping the bots give up. > > > > I added a robots.txt and edited the cigtrc to deny all bots from indexing > > git.enlightenment.org - but the bots seem to be ignoring that now that they > > have decided to start indexing. > > > > I am wondering if this has been the cause of our issues - being overloaded > > by indexer bots. FYI I counted 3 different bots indexing cgit at the same > > time: Baiduspider, Semrushbot, Ahrefsbot. > > > > I hope later they will start listening to robots.txt, but for now I need to > > keep things off until the bots give up. -- ------------- Codito, ergo sum - "I code, therefore I am" -------------- The Rasterman (Carsten Haitzler) [email protected] ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ enlightenment-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
