Dear all,
I am trying to crawl a large index (maybe more than 4Gb) for web search benchmark. However, after 11 hours crawling, I only found 1.2GB in the crawl directory with the size of the index 50MB. Could someone tell me how to configure the crawling so that I can retrieve a large enough index. And could anyone share the experience that how long it takes to crawl a 4Gb index? Thank you! Best Hailong
