On 8/18/07, Micah Cowan <[EMAIL PROTECTED]> wrote: > I'm not convinced. To me, the name "spider" implies recursion, and it's > counter-intuitive for it not to. > > As to wasted functionality, what's wrong with "-O /dev/null" (or NUL or > whatever) for simply checking existence?
I see his point. The difference is that the --spider option will only look for broken links on a given page, such as the bookmarks.html example. If we were to force recursion, it would finger out across the different pages. Perhaps we only want to check the links on _that_one_ page. Recursion wouldn't be helpful in that instance. But it could be argued that you could just set the recursion level to zero (or is it one?) to prevent that behavior.
