The estimated size of Google's index is 15 billion. So even for 1KB per page, it will be 15 TBs. But I think the average page size is way more than 1k.
On Sun, 2009-10-04 at 17:28 -0700, Gaurang Patel wrote: > All- > > I am novice to using Nutch. Can anyone tell me the estimated size in > (I suppose, in TBs) that will be required to store the crawled results > for whole web? I want to get estimate of the memory requirements for > my project, that uses Nutch web crawler. > > > > Regards, > Gaurang Patel
