I've been wanting to know where there is a list of crawlers.
Here is this link to look at: http://www.robotstxt.org/wc/active.html Regards, Gethsemani Palacios Computer Specialist Austin Automation Center 512-326-6708 512-762-7295 -----Original Message----- From: Jeremy C. Reed [mailto:[EMAIL PROTECTED]] Sent: Thursday, July 25, 2002 6:00 PM To: [EMAIL PROTECTED] Subject: [Robots] Who is "Rumours-Agent"? I've been crawled a lot by Rumours-Agent of 202.214.69.189 the past few days. Does anyone know what they crawl for? Thanks, Jeremy C. Reed echo '9,J8HD,fDGG8B@?:536FC5=8@I;C5?@H5B0D@5GBIELD54DL>@8L?:5GDEJ8LDG1' |\ sed ss,s50EBsg | tr 0-M 'p.wBt SgiIlxmLhan:o,erDsduv/cyP'
