Increasing the depth and topN parameters of the crawl command should do the trick. Although, it will increase the time of each crawl cycle and the overall crawl time
Le 2011-05-15 à 11:03, jeffersonzhou <[email protected]> a écrit : > Hi, > > I notice that some urls can be unfetched after a crawl job. How can I get > all the unfetched urls, evaluate them and if possible put them into the > fetch list again? > > Thanks >
