I recently upgraded to 0.9, and I've started encountering a problem. I began
with a single url and crawled with a depth of 10, assuming I would get every
page on my site. This same configuration worked for me in 0.8. However, I
noticed a particular url that I was especially interested in was not in the
index. So I added the url explicitly and crawled again. And it still was not in
the index. So I checked the logs, and it is being fetched. So I tried a lower
depth, and it worked. With a depth of 6, the url does appear in the index. Any
ideas on what would be causing this? I'm very confused.
Thanks,
Ann
____________________________________________________________________________________Pinpoint
customers who are looking for what you sell.
http://searchmarketing.yahoo.com/-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Nutch-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-general
