In local.h3dig-dev you write:
>>
>> That was my thinking too. I also felt that if duplicate detection was a
>> configuration attribute (potentially on a pre-URL or per-server basis),
>> then it wouldn't be as big of an issue.
>Oh, right. It didn't occur to me when reading Toivo's patches that this
>was not an optional feature. Making this feature selectable by a config
>attribute is a must, IMHO!
Yes, it think it should be optional, the duplicate detection and also
the extracting links from duplicate pages.
I've been running my htdig version against the web here for about
a day (it takes two days to complete) and looking at the output
the relative link problem is not a great one but I have found
one case of it happening so there will have to be a way of handling
it if it causes a problem, it won't be in the first version of
my code though.
--
Toivo Pedaste Email: [EMAIL PROTECTED]
University Communications Services, Phone: +61 8 9 380 2605
University of Western Australia Fax: +61 8 9 380 1109
"The time has come", the Walrus said, "to talk of many things"...
------------------------------------
To unsubscribe from the htdig3-dev mailing list, send a message to
[EMAIL PROTECTED]
You will receive a message to confirm this.
