On Jan 28, 2011, at 9:53 PM, Harsh J wrote:
> Moving discussion to hdfs-user mailing list: hdfs-u...@hadoop.apache.com
>
> On Sat, Jan 29, 2011 at 5:06 AM, Pei HE <pei...@gmail.com> wrote:
>> Hi all,
>> I want to know the detailed performance of Hadoop.
>>
>> I am writing a client code to test the latency, transfer speed of
>> HDFS, and the time of initiating a JVM for a task.
>>
>> Does anyone know that which classes can be called without running a
>> whole map/reduce job to find out the performance of Hadoop?
You can get the performance of HDFS without Hadoop by using it directly:
######## Creating 1GB file ###########
#ftg-007:/usr/local/hadoop # dd if=/dev/zero of=/hdfs/demofile bs=1M count=1000
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB) copied, 8.78527 s, 119 MB/s
######### Copy from local to HDFS ########
#ftg-007:/usr/local/hadoop # time ./bin/hadoop dfs -copyFromLocal
/hdfs/demofile /
real 0m51.911s
user 0m9.945s
sys 0m2.128s
######### Copy to Local from HDFS ########
#ftg-007:/usr/local/hadoop # time ./bin/hadoop dfs -copyToLocal /demofile
/hdfs/test/
real 0m19.068s
user 0m8.205s
sys 0m4.948s
>>
>> I am trying to create a DFSClient in the following way.
>> Configuration conf = new Configuration();
>> DFSClient client = new DFSClient(conf);
>> client.exists("hdfs://localhost:9900/home/");
>> But, a RemoteException was thrown.
>
> Would help to know what the cause for the throw of a RemoteException was.
>
> --
> Harsh J
> www.harshj.com
