In the past week or so, I've been seeing many many portions of my site
sucked down in rapid fire. These sucks followed patterns of being a
spider -- some autogenerated URLs from pod2html are definitely bad,
and I saw these bad hits, indicating that someone is following every
link in every one of my WT columns. robots.txt is *occasionally*
being fetched, but not always.
The only thing in common with these rude intrusions is a windows-IE
user agent, along with a new string "DigExt".
Has anyone else seen this? Can we trace this back to some lousy user
interface somewhere? I tried blocking them, and I got back some users
that wondered why I was blocking them.
Is it a new version of IE that permits heavy rapid download?
--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<[EMAIL PROTECTED]> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!
