On Tue, 05 Nov 2002 at 01:07:29 -0600, Searcher wrote: > Am I reading this correctly? 8 Million docs or URL's in the queue? I changed the > default hop from 256 to 4 and now the queue is even higher than before?
Did you clear the database and start from scratch? Changing MaxHops count without clearing the database first will have little effect on the queue. URLs that exceed MaxHops will be deleted but they have to be queued first. Bear in mind also that if you have MinDelay set it will be honored even if all URLs for a given site are to be deleted due to exceeding MaxHops. Actually, I've made a number of changes which I'm testing at present that fix most of this behavior but they're not ready to be committed to CVS yet. Matt.
