-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Frank Singleton wrote: > Hi, > > Did a test on recursive chown on a fedora 9 box here (2xquad core,16Gram) > Took about 12.5 minutes to complete for 45000 files. (hmm approx 60 files/sec) > > This was the namenode that I executed the command on > > Q1. Is this rate (60 files/sec) typical of what other folks are seeing ? > Q2. Are there any dfs/jvm parameters I should look at to see if I can improve > this > > time /home/hadoop/hadoop-0.18.1/bin/hadoop dfs -chown -R frank:frank > /home/frank/proj100 > > real 12m38.631s > user 1m54.662s > sys 0m33.124s > > time /home/hadoop/hadoop-0.18.1/bin/hadoop dfs -count /home/frank/proj100 > 220 45891 3965996260 > hdfs://namenode:9000/home/frank/proj100 > > real 0m1.579s > user 0m0.686s > sys 0m0.129s > > > cheers / frank
Just to clarify, this is for when the chown will modify all files owner attributes eg: toggle all from frank:frank to hadoop:hadoop (see below) for chown -R from frank:frank to frank:frank , the results is only 5 or 6 seconds. at this point , all files under /home/frank/proj100 are frank:frank, and the command executes in 6 seconds or so. [EMAIL PROTECTED] ~]$ time /home/hadoop/hadoop-0.18.1/bin/hadoop dfs -chown -R frank:frank /home/frank/proj100 real 0m5.624s user 0m6.744s sys 0m0.402s #now lets change all to hadoop:hadoop [EMAIL PROTECTED] ~]$ time /home/hadoop/hadoop-0.18.1/bin/hadoop dfs -chown -R hadoop:hadoop /home/frank/proj100 real 12m43.732s user 0m53.781s sys 0m10.655s # now toggle back to frank:frank [EMAIL PROTECTED] ~]$ time /home/hadoop/hadoop-0.18.1/bin/hadoop dfs -chown -R frank:frank /home/frank/proj100 real 12m40.700s user 0m45.757s sys 0m8.173s # now frank:frank to frank:frank [EMAIL PROTECTED] ~]$ time /home/hadoop/hadoop-0.18.1/bin/hadoop dfs -chown -R frank:frank /home/frank/proj100 real 0m5.648s user 0m6.734s sys 0m0.593s [EMAIL PROTECTED] ~]$ cheers / frank -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iEYEARECAAYFAkjlvKwACgkQpZzN+MMic6eO4ACfVYEJ3DqWXo1Mg/4StUhG2Vii r2AAn2YpDmDi2l2a4Bn/1CHAHQtLDgrg =Dq3d -----END PGP SIGNATURE-----
