Yup. This has been occuring for years using the Archie protocol:
http://www.whatis.com/archie.htm
Mark Symons
Augusta, Ga
> -----Original Message-----
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED]]On Behalf Of Steve Meyers
> Sent: Friday, 03 September, 1999 9:57 AM
> To: [EMAIL PROTECTED]
> Subject: Re: [IMail Forum] Can you identify this?
>
>
> Do FTP sites get spidered as well as web sites ?
>
> Thanks in advance
> Steve
>
> ---------- Original Message ----------------------------------
> From: hostmaster <[EMAIL PROTECTED]>
> Date: Mon, 30 Aug 1999 02:54:19 -0500
>
> >Aside from a few bytes of bandwidth, no, none at all. In fact,
> most of the
> >sites on your web server are probably being spidered. You can do some
> >reading on using the robots.txt file to prevent it, in the event
> you have
> >content that you would prefer didn't show up in altavista.
> >Here's a doc you may find interesting:
> >http://www.altavista.com/av/content/addurl_exclude.htm
> >
> >Best Regards,
> >
> >Jonathan
> >
> >At 10:09 AM 8/30/99 +0300, you wrote:
> >>Aha!
> >>sorry for my naive question but "being spidered" does it cause any
> >>problems to the functionality of Server?
> >>>The Scooter agent is Altavista's spider. Your web interface
> (or at least
> >>>the front page, as it doesn't have a logon) is being spidered. You'll
> >>>also notice that it's coming from DEC.com, which is where
> Scooter runs from.
> >>>
> >>>Jonathan
> >>>
> >>>At 09:17 AM 8/30/99 +0300, you wrote:
> >>>>Hi guys!
> >>>>
> >>>>I was looking the Web Access logs (W2XXXX.log files) and then
> I noticed
> >>>>that the following record appeared sometimes in regural basis....
> >>>>
> >>>>.......
> >>>>19990828 014703 204.123.9.102, Scooter/2.0 G.R.A.B. V1.1.0, ,
> GET /HTTP/1.0
> >>>>19990828 015621 204.123.9.102, Scooter/2.0 G.R.A.B. V1.1.0, ,
> GET /HTTP/1.0
> >>>>19990828 015649 204.123.9.102, Scooter/2.0 G.R.A.B. V1.1.0, ,
> GET /HTTP/1.0
> >>>>19990828 022550 204.123.9.102, Scooter/2.0 G.R.A.B. V1.1.0, ,
> GET /HTTP/1.0
> >>>>............
> >>>>
> >>>>Dou you know what is this???
> >>>>
> >>>>
> >>>>thanks in advance
> >>>>Periklis Drakos
> >>>>Sys Admin
> >>>>School of Social Sciences
> >>>>Univ. of Crete, Greece
> >
> >
> >
> Please visit http://www.ipswitch.com/support/mailing-lists.html
> to be removed from this list.
>
> Please visit http://www.ipswitch.com/support/mailing-lists.html
> to be removed from this list.
>
Please visit http://www.ipswitch.com/support/mailing-lists.html
to be removed from this list.
