Re: [Computer-go] Scraping lower-ranked games from kgs
Thanks for the tip. I agree and I have no intention to misbehave. Arthur > On Nov 13, 2015, at 8:38 AM, Petri Pitkanen > wrote: > > Yes scraping for large amounts of data from a smallish server is not really > polite. May overload the server. Besides quite inefficient. You could make a > request to owner of site instead. Assuming you can present good enough reason > you might get lucky ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Scraping lower-ranked games from kgs
Thank you Josef for that script. > > If you want to download more players, you need to search through opponents of > players you know. But do not spam the kgs server too much please :-) > Point taken, I would not want to spoil the kgs experience for users. I think a few hundred a day would be easily handled, after all I’ve heard it said that approx 25k games a day are played on it. Clearly if I tried to download all games from say 2010 onward in one go that would be horrendous and most uncivilised of me. Arthur ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Scraping lower-ranked games from kgs
Yes scraping for large amounts of data from a smallish server is not really polite. May overload the server. Besides quite inefficient. You could make a request to owner of site instead. Assuming you can present good enough reason you might get lucky 2015-11-13 8:39 GMT+02:00 Josef Moudrik : > Hello, > > I have a simple script for downloading info (gamelist, list of oponents, > games) one by one here: > > http://repo.or.cz/gostyle.git/blob/HEAD:/kgs/kgs.py > > If you want to download more players, you need to search through opponents > of players you know. But do not spam the kgs server too much please :-) > > Regards, > Josef > > > On Thu, Nov 12, 2015 at 2:35 PM Arthur Cater wrote: > >> Hello, >> I would be interested for research purposes in getting lots of game sgfs >> of >> lower-ranked players, to contrast with the high dan and pro games that >> get put into collections. Even kyu games. I was thinking of getting them >> from kgs archives. >> >> I wonder if anyone already has a script that could (or could be easily >> adapted to) do this? And would they share? >> >> Thanks, >> >> Arthur >> >> ___ >> Computer-go mailing list >> Computer-go@computer-go.org >> http://computer-go.org/mailman/listinfo/computer-go > > > ___ > Computer-go mailing list > Computer-go@computer-go.org > http://computer-go.org/mailman/listinfo/computer-go > ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go
Re: [Computer-go] Scraping lower-ranked games from kgs
Hello, I have a simple script for downloading info (gamelist, list of oponents, games) one by one here: http://repo.or.cz/gostyle.git/blob/HEAD:/kgs/kgs.py If you want to download more players, you need to search through opponents of players you know. But do not spam the kgs server too much please :-) Regards, Josef On Thu, Nov 12, 2015 at 2:35 PM Arthur Cater wrote: > Hello, > I would be interested for research purposes in getting lots of game sgfs of > lower-ranked players, to contrast with the high dan and pro games that > get put into collections. Even kyu games. I was thinking of getting them > from kgs archives. > > I wonder if anyone already has a script that could (or could be easily > adapted to) do this? And would they share? > > Thanks, > > Arthur > > ___ > Computer-go mailing list > Computer-go@computer-go.org > http://computer-go.org/mailman/listinfo/computer-go ___ Computer-go mailing list Computer-go@computer-go.org http://computer-go.org/mailman/listinfo/computer-go