[EMAIL PROTECTED] wrote:
On Thu, 15 Dec 2005, Paul A. Rubin wrote:
I've had good luck with Xenu Link Sleuth
(http://home.snafu.de/tilman/xenulink.html), but it's a Windows-only
program (although the web site says it appears to run under Wine).
Tried it on the Wiki, using 30 threads (could do more) and checking
internal links only.
That sounds good. Could you let it run to completion and let me know the
result? (Send the log privately if it's too big)
<sigh>Ran it from my office. After a few hours, running with the
maximum 100 threads, it had identified about 30,000+ links and checked
about 16,000 of them (excluding links that required authentication,
since I didn't have a password). Watching progress, I know that it
found at least a half-dozen or so broken links in that time. So I left
it running overnight, and when I came in today it had finished and
wanted to know if I wanted a report. I said yes and it got stuck in a
massive CPU-cycle-sucking exercise that it did not survive.</sigh>
I read elsewhere in the thread that you had validator running. Do you
still need me to run a link check here? If so, I think the job needs to
be partitioned (scan the Wiki to a modest depth on the first pass, then
scan deeper parts in separate runs).
Paul
