Thanks for sharing. > > For comparison, on a 1400 node cluster, I can checksum 100 TB in > around 10 minutes, which means I'm seeing read averages of roughly 166 > GB/sec. For writes with replication of 3, I see roughly 40-50 minutes > to write 100TB, so roughly 33 GB/sec average. Of course the peaks are > much higher. Each node has 4 SATA disks, dual quad core, and 8 GB of > ram. >
>From your experience, how RAM "hungry" HDFS is? Meaning, additional 4GB or ram (to make it 8GB aas in your case), really change anything? Regards.
