A client of ours recently had a security audit on their web site. The audit
recommended that we remove all 'disallow: /xyz/' entries since a potential
hacker could read the robots.txt file and surmise which folders may be
sensitive.
Here's my question, if I remove all of the [disallow: /xyz/]
I think the distinction needs to be make between what's sensitive and what you
don't want to be indexed by a spider.
If the information is sensitive, it shouldn't be where a spider can get to it
at all - and robots.txt is no protection.
If the information just shouldn't be indexed, then
Can you, or are you able to, password protect the directories? IIS Auth?
htaccess? IP restrictions?
-Original Message-
From: Che Vilnonis [mailto:[EMAIL PROTECTED]
Sent: Friday, May 27, 2005 7:30 AM
To: CF-Talk
Subject: Robot.txt question... security issue?
A client of ours recently
audited.
~che
-Original Message-
From: Jacob [mailto:[EMAIL PROTECTED]
Sent: Friday, May 27, 2005 11:29 AM
To: CF-Talk
Subject: RE: Robot.txt question... security issue?
Can you, or are you able to, password protect the directories? IIS Auth?
htaccess? IP restrictions?
-Original
-Original Message-
From: Che Vilnonis [mailto:[EMAIL PROTECTED]
Sent: Friday, May 27, 2005 7:30 AM
To: CF-Talk
Subject: Robot.txt question... security issue?
A client of ours recently had a security audit on their web site. The
audit
recommended that we remove all 'disallow: /xyz
-Original Message-
From: Jim Davis [mailto:[EMAIL PROTECTED]
Sent: Friday, May 27, 2005 11:39 AM
To: CF-Talk
Subject: RE: Robot.txt question... security issue?
-Original Message-
From: Che Vilnonis [mailto:[EMAIL PROTECTED]
Sent: Friday, May 27, 2005 7:30 AM
To: CF-Talk
-Original Message-
From: Che Vilnonis [mailto:[EMAIL PROTECTED]
Sent: Friday, May 27, 2005 11:49 AM
To: CF-Talk
Subject: RE: Robot.txt question... security issue?
Jim, that is the recommendation I told my boss. :)
The client in question is being audited by Visa/Mastercard
A client of ours recently had a security audit on their web
site. The audit recommended that we remove all 'disallow: /xyz/'
entries since a potential hacker could read the robots.txt file
and surmise which folders may be sensitive.
Here's my question, if I remove all of the [disallow:
question... security issue?
Jim, that is the recommendation I told my boss. :)
The client in question is being audited by Visa/Mastercard and they are
using a 'canned' software package/service to perform the audit. In the
future, that recommendation may become a requirement in order to help
9 matches
Mail list logo
