Hi Carsten, What advantage does http for git have? Isn't git better to use to pull and push stuff to the repositories or is cgit more a web based ui to view the repository history etc?
--- Regards, Jonathan Aquilina On 2017-04-26 04:18, Carsten Haitzler wrote: > I've had to disable the whole http support for now for git.enlightenment.org > because several bots are crawling it causing our VM to basically be loaded > with > 10-20 cgit cgi's running git queries for history etc. continually. I/O and > system load is going through the roof as a result and causing other stuff like > phab to crawl and begin timing out. > > So anyone using HTTP for doing cmdline git stuff is, at this moment, going to > find things not working. SSH and GIT protocol should still work. I'll keep > this > shut down for a few hours hoping the bots give up. > > I added a robots.txt and edited the cigtrc to deny all bots from indexing > git.enlightenment.org - but the bots seem to be ignoring that now that they > have decided to start indexing. > > I am wondering if this has been the cause of our issues - being overloaded by > indexer bots. FYI I counted 3 different bots indexing cgit at the same time: > Baiduspider, Semrushbot, Ahrefsbot. > > I hope later they will start listening to robots.txt, but for now I need to > keep things off until the bots give up. ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ enlightenment-devel mailing list enlightenment-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
