- - - - - - - - - - - - - - - - - - - - - - - - - - - -
Name: Maxime
Subject: Re: robots.txt:
Allow/Disallow commands are looking in order of appearance, and only the first
found applies. So Disallow *.cgi will still exclude *.cgi in this case.
- - - - - - - - - - - - - - - - - - - - - - - - -
- - - - - - - - - - - - - - - - - - - - - - - - - - - -
Name: Maxime
Subject: Re: robots.txt:
Yes, absolutely right.
- - - - - - - - - - - - - - - - - - - - - - - - - - - -
Read the full topic here:
http://www.dataparksearch.org/cgi-bin/simpleforum.cgi?fid=02;topic_id=1185818418;page=2