On Feb 10, 2009, at 4:10 PM, Wasim Bari wrote:
Hi,
Could someone help me to find some real Figures (transfer rate)
about Hadoop File transfer from local filesystem to HDFS, S3 etc
and among Storage Systems (HDFS to S3 etc)
Thanks,
Wasim
What are you looking for? Maximum possible transfer rate? Maximum
possible transfer rate per client? Generally, if you're using the
Java client, transfer rate to/from HDFS is limited by the hardware you
have and the network connection (if you have 1Gbps per client).
I could give you a graph showing a peak of 9Gbps from our Hadoop
instance to the WAN, but that's not very interesting if you don't have
a 10Gbps pipe...
Brian