Why not just use the "htdig-noindex" meta tag on the top
page of any collections you don't want to index? Or am I
missing something. I haven't tried the noindex tag, but
the e-mail notification ones work so well I just imagined the
others would, too.
Jim Augeri
Honeywell International Inc.
Defense Avionics Systems
9201 San Mateo Blvd NE
Albuquerque, NM 87113
Off: 505.828.5808
Fax: 505.828.5500
E-mail: [EMAIL PROTECTED]
-----Original Message-----
From: Gilles Detillieux [mailto:[EMAIL PROTECTED]]
Sent: Thursday, June 14, 2001 3:12 PM
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Subject: Re: [htdig] Excluding protected dirs and local_urls_only
According to Alex Rousskov:
> I need to index a site that has a few .htaccess protected
> directories. I do not want protected directories to be indexed. I want
> to index that site via local file system only. I do not want to have a
> static list of directories that htdig must avoid. Instead, I simply
> want htdig to exclude any directory that has .htaccess file in it.
>
> I tried to find a solution in the archives, but it seems like
> my problem is unique. Would I have to code the solution?
Yes, this does seem to be a rather unique problem, as far as I recall,
but not that different from other problems. I'd recommend using a
find command, similar to the one in http://www.htdig.org/FAQ.html#q5.25,
to find all directories that have a .htaccess file and make URLs out of
the directory names. Put these in a file, and use that file for your
exclude_urls attribute (rather than start_url as in the example). The
whole process can be automated with a script which runs this before
running htdig and htmerge.
--
Gilles R. Detillieux E-mail: <[EMAIL PROTECTED]>
Spinal Cord Research Centre WWW:
http://www.scrc.umanitoba.ca/~grdetil
Dept. Physiology, U. of Manitoba Phone: (204)789-3766
Winnipeg, MB R3E 3J7 (Canada) Fax: (204)789-3930
_______________________________________________
htdig-general mailing list <[EMAIL PROTECTED]>
To unsubscribe, send a message to
<[EMAIL PROTECTED]> with a subject of unsubscribe
FAQ: http://htdig.sourceforge.net/FAQ.html
_______________________________________________
htdig-general mailing list <[EMAIL PROTECTED]>
To unsubscribe, send a message to <[EMAIL PROTECTED]> with a
subject of unsubscribe
FAQ: http://htdig.sourceforge.net/FAQ.html
