The thing is, your URL doesn't tecnically come under the disallowed set of
things.
For that particular case, try adding /cgi/ to the Disallow path. [I'm
assuming www.pg.com/cgi is a place that you keep CGI script on your web
server]
Hope this helps,
Gary (-;
> -----Original Message-----
> From: The Hermit Hacker [SMTP:[EMAIL PROTECTED]]
> Sent: Tuesday, March 06, 2001 4:58 PM
> To: [EMAIL PROTECTED]
> Subject: [3.1.11] indexing .cgi's?
>
>
> %sbin/indexer -h
>
> indexer from mnogosearch-3.1.11/PgSQL
> http://search.mnogo.ru (C) 1998-2000, LavTech Corp.
>
> I have it disallow'd in my config file:
>
> # Exclude cgi-bin and non-parsed-headers
> Disallow /cgi-bin/ \.cgi /nph \?
>
> yet, if I run the indexer, its indexing a *load* of cgi URLs, of the
> format:
>
> Indexer[83032]: [1]
> http://www.postgresql.org/cgi/cvsweb.cgi/pgsql/configure.in.diff?r1=1.59&r
> 2=1.45&sortby=rev&only_with_tag=MAIN
>
> why? and how can I get it to stop?
>
> thanks ...
>
>
> Marc G. Fournier ICQ#7615664 IRC Nick:
> Scrappy
> Systems Administrator @ hub.org
> primary: [EMAIL PROTECTED] secondary:
> scrappy@{freebsd|postgresql}.org
>
>
> ___________________________________________
> If you want to unsubscribe send "unsubscribe general"
> to [EMAIL PROTECTED]
>
___________________________________________
If you want to unsubscribe send "unsubscribe general"
to [EMAIL PROTECTED]
