Denis Jedig wrote:
> n3td3v wrote:
>
>> Today's disclosure involves Google and Yahoo search engines:
>>
>> All you need to do is put in the code to a web page, when Google and
>> Yahoo visit it, then the code exploits the software they use and
>> makes them start caching 'other' pages. Including 'no index' pages,
>> where sites have setup a robot text file on their server to protect
>> corporate and consumer interests.
>
> I think you missed the concept here. Whatever is on the webservers and
> is available to the public is... well... available to the public.
>
> It does not help security matters to introduce a robots.txt - the
> purpose of this directives file is not to secure something but to
> reduce traffic and keep irrelevant content out of search engines.
>
> If you need security, you introduce some kind of authentication
> *before* access is allowed to sensitive data. You will find that a
> sign reading "Do not enter and do not steal any gold" will not help
> much at the Fort Knox entrance if it is the only security measure.
Also, Google and Yahoo *do* respect the robots.txt file and do check it
for every server they fetch files from, and the whole thing is garbage. His
so-called 'example' is a fraud because it shows yahoo caching a page from
the site mtf.news.yahoo.com, which DOES NOT HAVE A ROBOTS.TXT FILE.
cheers,
DaveK
--
Can't think of a witty .sigline today....
_______________________________________________
Full-Disclosure - We believe in it.
Charter: http://lists.grok.org.uk/full-disclosure-charter.html
Hosted and sponsored by Secunia - http://secunia.com/
