Hi!
Saturday, November 11, 2000, 12:30:22 AM, you wrote:
AS> The Dict table has information in it, just not enough information!
AS> Here is my allow/disallow statement:
AS> # Exclude Apache and Squid directory lists in different sort order
AS> Disallow \?D=A$ \?D=A$ \?D=D$ \?M=A$ \?M=D$ \?N=A$ \?N=D$ \?S=A$ \?S=D$
AS> # Exclude ./. and ./.. from directory list
AS> Disallow /[.]{1,2} /\%2e /\%2f # Retrieve only
AS> directory list, check other files.
AS> CheckOnly [^/]$
so, you are CheckOnly ALL urls which are not ended with '/'.
So if file.html contains links to other pages than they will not be
added into database because of this checkonly statement.
--
Regards, Sergey aka gluke.
______________
If you want to unsubscribe send "unsubscribe udmsearch"
to [EMAIL PROTECTED]
