I understand that
when I see the "url rejected" in the output, that means it is in the excluded
list. I have excluded /calendar/ from htdig, however, it seems to find it
necessarry to repeatedly hit
/calendar/index.php?page=XX&day=XX&month=XX&year=XX (where XX are
repeadetly replaced with increasing values, so far from 2001 to 2007). Is this
because it is finding those links somewhere or is it just doing a zombie-like,
brain-dead, recursive search on the url which is then
rejected?
Thanks,
Sjan
Evardsson
Webmaster
Alaska Pacific
University
