Michael Roberts wrote:
> And what I found was that someone had spidered my customer's entire (27-page)
> site, and each line had Referer of www.persiankitty.com and a Browser of
> "Mozilla/5.0".
>
> The host IP was 63.73.211.5 and that resolves to saqnad.neurotic.org -- oddly,
> the timing of the hits seemed nearly human, with many seconds between hits,
> but each and every link was followed, sometimes quickly, so it may have been
> automated. No hit to robots.txt.
>
> My question: has anyone else seen this, and does everybody think this would be a
> reasonable sort of thing to add to a spider exclusion list? Is it my
> paranoia that makes this seem like porn spam? Wouldn't access log spamming be a
> really pathetic way to advertise? But can anybody think of some other
> motivation to have done that?
Some companies will got to pethetic lengths to advertise. Once in a while I get
'spam' emails in Chinese. My email software doesn't parse that, which is fine,
because I don't either.
But my guess is that this is a 'robot' (like wget or LWP) that someone used to
grab the entire content of the site so that it can be reposted (or just read
locally). I might recommend to the client that they follow-up on the IP and other
leads for possible copyright violations. On the otherhand, why you would
explicitly specify a referrer header when you were copying I site, I can't explain
(but I would assume that persiankitty.com had nothing to do with the requests -
for that matter, the neurotic.org IP may be faked as well).
HTH,
Jeremy Wadsack
Wadsack-Allen Digital Group
------------------------------------------------------------------------
This is the analog-help mailing list. To unsubscribe from this
mailing list, send mail to [EMAIL PROTECTED]
with "unsubscribe" in the main BODY OF THE MESSAGE.
List archived at http://www.mail-archive.com/analog-help@lists.isite.net/
------------------------------------------------------------------------
