Thanks in advance for any help you can provide. Not sure exactly what's relevant here, but I have not been able to complete a full bin/crawl since I had a "No space left on device" error.
Using nutch-1.6. - bin/crawl had been running as expected for 20+ iterations - On one run, the disk ran out of space and threw the "No space left on device" error. - The db.fetch.interval.default is set at 80,000 (less than 24 hours) - To clear disk space I removed all segments - Ever since, re-running bin/crawl fails at the fetch point with multiple "Read time out" errors bin/crawl exits when it attempts 'crawl parse' because the crawl_parse, crawl_data, etc. directories do not exist. (The only directory in segments/xxxxxx is crawl_generate) What might be the solution?
