Re: [Computer-go] Scraping lower-ranked games from kgs

[Arthur Cater]

Thanks for the tip. I agree and I have no intention to misbehave.

Arthur

> On Nov 13, 2015, at 8:38 AM, Petri Pitkanen  
> wrote:
> 
> Yes scraping for large amounts of data from  a smallish server is not really 
> polite. May overload the server. Besides quite inefficient. You could make a 
> request to owner of site instead. Assuming you can present good enough reason 
>  you might get lucky

___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Scraping lower-ranked games from kgs

[Petri Pitkanen]

Yes scraping for large amounts of data from  a smallish server is not
really polite. May overload the server. Besides quite inefficient. You
could make a request to owner of site instead. Assuming you can present
good enough reason  you might get lucky

2015-11-13 8:39 GMT+02:00 Josef Moudrik :

> Hello,
>
> I have a simple script for downloading info (gamelist, list of oponents,
> games) one by one here:
>
> http://repo.or.cz/gostyle.git/blob/HEAD:/kgs/kgs.py
>
> If you want to download more players, you need to search through opponents
> of players you know. But do not spam the kgs server too much please :-)
>
> Regards,
> Josef
>
>
> On Thu, Nov 12, 2015 at 2:35 PM Arthur Cater  wrote:
>
>> Hello,
>> I would be interested for research purposes in getting lots of game sgfs
>> of
>> lower-ranked players, to contrast with the high dan and pro games that
>> get put into collections. Even kyu games. I was thinking of getting them
>> from kgs archives.
>>
>> I wonder if anyone already has a script that could (or could be easily
>> adapted to) do this? And would they share?
>>
>> Thanks,
>>
>> Arthur
>>
>> ___
>> Computer-go mailing list
>> Computer-go@computer-go.org
>> http://computer-go.org/mailman/listinfo/computer-go
>
>
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go
>
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Scraping lower-ranked games from kgs

[Arthur Cater]

Thank you Josef for that script.

> 
> If you want to download more players, you need to search through opponents of 
> players you know. But do not spam the kgs server too much please :-)
> 

Point taken, I would not want to spoil the kgs experience for users.
I think a few hundred a day would be easily handled, after all I’ve heard it 
said
that approx 25k games a day are played on it. Clearly if I tried to download
all games from say 2010 onward in one go that would be horrendous and
most uncivilised of me.

Arthur


___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

Re: [Computer-go] Scraping lower-ranked games from kgs

[Josef Moudrik]

Hello,

I have a simple script for downloading info (gamelist, list of oponents,
games) one by one here:

http://repo.or.cz/gostyle.git/blob/HEAD:/kgs/kgs.py

If you want to download more players, you need to search through opponents
of players you know. But do not spam the kgs server too much please :-)

Regards,
Josef


On Thu, Nov 12, 2015 at 2:35 PM Arthur Cater  wrote:

> Hello,
> I would be interested for research purposes in getting lots of game sgfs of
> lower-ranked players, to contrast with the high dan and pro games that
> get put into collections. Even kyu games. I was thinking of getting them
> from kgs archives.
>
> I wonder if anyone already has a script that could (or could be easily
> adapted to) do this? And would they share?
>
> Thanks,
>
> Arthur
>
> ___
> Computer-go mailing list
> Computer-go@computer-go.org
> http://computer-go.org/mailman/listinfo/computer-go
___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go

[Computer-go] Scraping lower-ranked games from kgs

[Arthur Cater]

Hello,
I would be interested for research purposes in getting lots of game sgfs of
lower-ranked players, to contrast with the high dan and pro games that 
get put into collections. Even kyu games. I was thinking of getting them
from kgs archives.

I wonder if anyone already has a script that could (or could be easily adapted 
to) do this? And would they share?

Thanks,

Arthur

___
Computer-go mailing list
Computer-go@computer-go.org
http://computer-go.org/mailman/listinfo/computer-go