Author: Alex Barkov
Email: [EMAIL PROTECTED]
Message:
Check your tables structure. It seems that something is wrong.
Probably your database "lost" and unique index on url.url field.
> I ran like this:
>
> indexer -k -v 6 -c 1200 -n 1000
>
> And then it stops after 1200 seconds. Good. But when I search for something, I get
>duplicate results (2 search answers looking the same, pointing to the exact same
>doc/url). If I give it a 5 minute timeout, I do not get duplicates.
>
> This sort of confirms my theory that something is very wrong and it is not
>detecting/tagging pages it "sees", making it visit/download/index the same
>page over and over again, looping forever in indexer.
>
> Does it not use a hashtable to keep track of urls visited ? Does it not store them
>in canonical form to detect variations shuch as ../.. , etc ?
>
> Ideas ? I posted my conf file already (I changed the server url because that's
>private info)
>
> marcio
Reply: <http://search.mnogo.ru/board/message.php?id=2319>
___________________________________________
If you want to unsubscribe send "unsubscribe general"
to [EMAIL PROTECTED]
