--- Alexander Barkov <[EMAIL PROTECTED]> wrote:
> Caffeinate The World wrote:
> >
> > if i insert a bunch of URLs into the db via a text file.. and those
> > urls have no corresponding server command, by default, it should
> have a
> > Follow of path and NOT site.
> >
> > here is one example. i pull URLs from a sub section of dmoz using
> the
> > Href directive. Some of those are site URLs and some are a path of
> a
> > site. now, when i pull all the urls from that sub section of dmoz,
> the
> > current setting will index thru all the "whole" sites/servers
> instead
> > of just the paths.
> >
> > if default is to follow path, then the whole site would still get
> > indexed. i hope that made sense. i brought this topic up about a
> month
> > ago and i still see this behavior in -current.
>
>
> When "DeleteNoServer no" is specified, indexer follows whole site
> if there is no "Server" command for some URL.
i understand this, but is this the correct behavior? if it is, i can't
think of any situation where it's appropriate.
__________________________________________________
Do You Yahoo!?
Yahoo! Auctions - Buy the things you want at great prices.
http://auctions.yahoo.com/
___________________________________________
If you want to unsubscribe send "unsubscribe general"
to [EMAIL PROTECTED]
