I am just staring to learn nutch. One question I wanted to know is that can nutch pause, stop and start indexing a site on a incremental daily basis? My concern with nutch is that nutch behaving like a hog and crawling everything with huge bandwidth consumption and pissing off the many site owners.
Can some experts shed some light in this?
