current leaseholder is trying to recreate file.

2010-03-31 Thread hareesh
Anyone have insight on the following message attempt_201003301923_0007_m_00_0: -activeThreads=100, spinWaiting=0, fetchQueues.totalSize=4998 attempt_201003301923_0007_m_00_0: -activeThreads=100, spinWaiting=0, fetchQueues.totalSize=4998 attempt_201003301923_0007_m_00_0: Aborting

Re: Crawl yahoo search result page

2010-03-31 Thread reinhard schwab
it is not allowed for robots. http://search.yahoo.com/robots.txt User-agent: * Disallow: /search Disallow: /bin Disallow: /myweb Disallow: /myresults Disallow: /language Kim Theng Chong schrieb: Hi all, Can Nutch crawl Yahoo search result page? eg :

Problem at the end of fetching

2010-03-31 Thread hareesh
Iam trying to crawl a seed list of 5000. it was working fine, but at the end of fetching at depth 1 the process failed showing message like this. can any one suggest what may be problem.. attempt_201003311259_0003_m_03_2: fetching http://www.law.louisville.edu/news-events/admissions/feed