I have this same robot on my site. Can i Block this
robot using .htaccess files..???
Chris
http://www.truefootball.com
http://www.worldofjerseys.com
I have this same robot on my site. Can i Block this
robot using .htaccess files..???
Chris
http://www.truefootball.com
http://www.worldofjerseys.com
On 8 Jan 2002, at 9:56, Jesse Goerz wrote:
> On Tuesday 08 January 2002 01:38, Russell Coker wrote:
> > On Mon, 7 Jan 2002 23:31, Nathan Strom wrote:
> > > > I have a nasty web spider with an agent name of
> > > > "LinkWalker" downloading everythi
On Tuesday 08 January 2002 01:38, Russell Coker wrote:
> On Mon, 7 Jan 2002 23:31, Nathan Strom wrote:
> > > I have a nasty web spider with an agent name of
> > > "LinkWalker" downloading everything on my site (including
> > > .tgz files). Does any
On 8 Jan 2002, at 9:56, Jesse Goerz wrote:
> On Tuesday 08 January 2002 01:38, Russell Coker wrote:
> > On Mon, 7 Jan 2002 23:31, Nathan Strom wrote:
> > > > I have a nasty web spider with an agent name of
> > > > "LinkWalker" downloading everythi
On Tuesday 08 January 2002 01:38, Russell Coker wrote:
> On Mon, 7 Jan 2002 23:31, Nathan Strom wrote:
> > > I have a nasty web spider with an agent name of
> > > "LinkWalker" downloading everything on my site (including
> > > .tgz files). Does any
On Mon, 7 Jan 2002 23:31, Nathan Strom wrote:
> > I have a nasty web spider with an agent name of "LinkWalker" downloading
> > everything on my site (including .tgz files). Does anyone know anything
> > about it?
>
> It's apparantly a link-valida
On Mon, 7 Jan 2002 23:31, Nathan Strom wrote:
> > I have a nasty web spider with an agent name of "LinkWalker" downloading
> > everything on my site (including .tgz files). Does anyone know anything
> > about it?
>
> It's apparantly a link-valida
Bwahahaha!! Man, that is low. Advertising to sysadmins through the access
logs Sheesh. But now that you mention 7-24, I think I recognize that.
I think they are a spam marketing outfit.
At 02:31 PM 1/7/02 -0800, Nathan Strom wrote:
>Personally, I think this is a rogue organization -- there
Bwahahaha!! Man, that is low. Advertising to sysadmins through the access
logs Sheesh. But now that you mention 7-24, I think I recognize that.
I think they are a spam marketing outfit.
At 02:31 PM 1/7/02 -0800, Nathan Strom wrote:
>Personally, I think this is a rogue organization -- ther
> site does NOT have a link to us. Likely Seven24 is trying to clutter
> people's logs with references as a form of advertising.
... a practise we see more and more often here as well! Even
'respectable' major isp's are starting to do it!
It's a strange world ...
Frank Louwers
Openminds b.v.b.a.
[EMAIL PROTECTED] (Russell Coker) wrote in message news:<[EMAIL PROTECTED]>...
> I wasn't aware that there was any format to robots.txt, I thought that the
> mere presense of such a file would prevent robots from visiting.
Nope; see:
http://www.robotstxt.org/wc/robots.html
[EMAIL PROTECTED] (Russell Coker) wrote in message news:<[EMAIL PROTECTED]>...
> I have a nasty web spider with an agent name of "LinkWalker" downloading
> everything on my site (including .tgz files). Does anyone know anything
> about it?
It's apparantly a link-
> site does NOT have a link to us. Likely Seven24 is trying to clutter
> people's logs with references as a form of advertising.
... a practise we see more and more often here as well! Even
'respectable' major isp's are starting to do it!
It's a strange world ...
Frank Louwers
Openminds b.v.b.a
[EMAIL PROTECTED] (Russell Coker) wrote in message
news:<[EMAIL PROTECTED]>...
> I wasn't aware that there was any format to robots.txt, I thought that the
> mere presense of such a file would prevent robots from visiting.
Nope; see:
http://www.robotstxt.org/wc/robots.html
--
To UNSUBSCRIBE,
[EMAIL PROTECTED] (Russell Coker) wrote in message
news:<[EMAIL PROTECTED]>...
> I have a nasty web spider with an agent name of "LinkWalker" downloading
> everything on my site (including .tgz files). Does anyone know anything
> about it?
It's apparantly a lin
> > Why don't you just update your robots.txt to explicitly specify which
> > files you don't or do, allow spiders access to. If it's a rule-obiding
> > spider, that will be the end of it.
>
> I wasn't aware that there was any format to robots.txt, I thought that the
> mere presense of such a f
> > Why don't you just update your robots.txt to explicitly specify which
> > files you don't or do, allow spiders access to. If it's a rule-obiding
> > spider, that will be the end of it.
>
> I wasn't aware that there was any format to robots.txt, I thought that the
> mere presense of such a
On Mon, Dec 24, 2001 at 11:43:09AM +0100, Russell Coker wrote:
> > > I have a nasty web spider with an agent name of "LinkWalker" downloading
> > > everything on my site (including .tgz files). Does anyone know anything
> > > about it?
> >
> > S
On Mon, 24 Dec 2001 06:42, Jeremy Lunn wrote:
> On Sun, Dec 23, 2001 at 05:41:47PM +0100, Russell Coker wrote:
> > I have a nasty web spider with an agent name of "LinkWalker" downloading
> > everything on my site (including .tgz files). Does anyone know anything
> >
On Mon, Dec 24, 2001 at 11:43:09AM +0100, Russell Coker wrote:
> > > I have a nasty web spider with an agent name of "LinkWalker" downloading
> > > everything on my site (including .tgz files). Does anyone know anything
> > > about it?
> >
> > S
On Mon, 24 Dec 2001 06:42, Jeremy Lunn wrote:
> On Sun, Dec 23, 2001 at 05:41:47PM +0100, Russell Coker wrote:
> > I have a nasty web spider with an agent name of "LinkWalker" downloading
> > everything on my site (including .tgz files). Does anyone know anything
>
On Sun, Dec 23, 2001 at 05:41:47PM +0100, Russell Coker wrote:
> I have a nasty web spider with an agent name of "LinkWalker" downloading
> everything on my site (including .tgz files). Does anyone know anything
> about it?
Surely you'd be able to disallow access to it
On Sun, Dec 23, 2001 at 05:41:47PM +0100, Russell Coker wrote:
> I have a nasty web spider with an agent name of "LinkWalker" downloading
> everything on my site (including .tgz files). Does anyone know anything
> about it?
Surely you'd be able to disallow
You should be able to tell if it cares about robots.txt by looking in the
logs to see if it's downloading /robots.txt. If it is then something like:
User-agent: LinkWalker
Disallow: /
will keep it off your site. If it doesn't, then iptables will keep it away.
Robots info:
http://
You should be able to tell if it cares about robots.txt by looking in the
logs to see if it's downloading /robots.txt. If it is then something like:
User-agent: LinkWalker
Disallow: /
will keep it off your site. If it doesn't, then iptables will keep it away.
Robots info:
http://
On Sun, Dec 23, 2001 at 09:17:54PM +0100, Russell Coker wrote:
>
> I wasn't aware that there was any format to robots.txt, I thought that the
> mere presense of such a file would prevent robots from visiting.
Here is an example of my robots.txt
User-agent: *
Disallow: /webalizer/
Disallow: cont
responsible web spiders such as google
when browsing my web logs!
> On Sun, Dec 23, 2001 at 05:41:47PM +0100, Russell Coker wrote:
> > I have a nasty web spider with an agent name of "LinkWalker" downloading
> > everything on my site (including .tgz files). Does anyone know a
h an agent name of "LinkWalker" downloading
> everything on my site (including .tgz files). Does anyone know anything
> about it?
>
> I've added the following to my firewall setup to stop further attacks...
>
> # crappy LinkWalker - evil spider that downloads every file
On Sun, Dec 23, 2001 at 09:17:54PM +0100, Russell Coker wrote:
>
> I wasn't aware that there was any format to robots.txt, I thought that the
> mere presense of such a file would prevent robots from visiting.
Here is an example of my robots.txt
User-agent: *
Disallow: /webalizer/
Disallow: con
responsible web spiders such as google
when browsing my web logs!
> On Sun, Dec 23, 2001 at 05:41:47PM +0100, Russell Coker wrote:
> > I have a nasty web spider with an agent name of "LinkWalker" downloading
> > everything on my site (including .tgz files). Does anyone know a
h an agent name of "LinkWalker" downloading
> everything on my site (including .tgz files). Does anyone know anything
> about it?
>
> I've added the following to my firewall setup to stop further attacks...
>
> # crappy LinkWalker - evil spider that downloads ever
I have a nasty web spider with an agent name of "LinkWalker" downloading
everything on my site (including .tgz files). Does anyone know anything
about it?
I've added the following to my firewall setup to stop further attacks...
# crappy LinkWalker - evil spider that downl
I have a nasty web spider with an agent name of "LinkWalker" downloading
everything on my site (including .tgz files). Does anyone know anything
about it?
I've added the following to my firewall setup to stop further attacks...
# crappy LinkWalker - evil spider that downl
34 matches
Mail list logo
