[jira] [Updated] (HDFS-3295) Mismatch in number of bytes already moved and number of bytes being moved in balancer report

2014-02-24 Thread Junping Du (JIRA)

 [ 
https://issues.apache.org/jira/browse/HDFS-3295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Junping Du updated HDFS-3295:
-

Resolution: Cannot Reproduce
Status: Resolved  (was: Patch Available)

This issue has been fixed in HDFS-4261, let's resolve it as Cannot Reproduce.

 Mismatch in number of bytes already moved and number of bytes being moved in 
 balancer report
 

 Key: HDFS-3295
 URL: https://issues.apache.org/jira/browse/HDFS-3295
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: balancer
Affects Versions: 0.23.0
Reporter: J.Andreina
Assignee: Ashish Singhi
 Attachments: HDFS-3295.patch, HDFS-3295.patch, HDFS-3295.patch, 
 HDFS-3295.patch, HDFS-3295.patch


 Scenario:
 Replication factor = 
 1,fs.defaultFS=hdfs://namenodeip:port,dfs.namenode.rpc-address=namenodeip:port.
 step 1: started DN1.Pumped 4.67GB of data
 step 2: started DN2.
 step 3: issued the balancer cmd(./start-balancer.sh -threshold 1)
 Totally  848.41 MB  has been moved to 2DN and took 4 iterations to move the 
 blocks to 2DN.
 But in balancer output in all the iterations the number of bytes already 
 moved is alway 0KB  and there is mismatch in the Bytes being moved with the 
 bytes left to move
 Balancer output for 2nd Iteration
 =
 {noformat}
 Time Stamp   Iteration#  Bytes Already Moved  Bytes Left To Move  
 Bytes Being Moved
 Apr 18, 2012 12:41:20 PM  0 0 KB   848.41 MB  
 151.18 MB
 Apr 18, 2012 1:06:46 PM   1 0 KB   646.02 MB  
 151.18 MB
 Apr 18, 2012 1:10:28 PM   2 0 KB   417.43 MB  
 151.18 MB
 Apr 18, 2012 1:14:04 PM   3 0 KB   223.84 MB  
 151.18 MB
 Apr 18, 2012 1:18:10 PM   4 0 KB30.34 MB  
 151.18 MB{noformat}
 In the above balancer output
 •Bytes already moved is 0KB
 • In 3 rd iterarion Bytes left to move is 223.84 MB but Bytes being moved is 
 151.18 MB . So in next iteration the bytes left to move should be 223.84 MB - 
 151.18 MB 



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)


[jira] [Updated] (HDFS-3295) Mismatch in number of bytes already moved and number of bytes being moved in balancer report

2013-04-10 Thread Harsh J (JIRA)

 [ 
https://issues.apache.org/jira/browse/HDFS-3295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Harsh J updated HDFS-3295:
--

Fix Version/s: (was: 0.24.0)

 Mismatch in number of bytes already moved and number of bytes being moved in 
 balancer report
 

 Key: HDFS-3295
 URL: https://issues.apache.org/jira/browse/HDFS-3295
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: balancer
Affects Versions: 0.23.0
Reporter: J.Andreina
Assignee: Ashish Singhi
 Attachments: HDFS-3295.patch, HDFS-3295.patch, HDFS-3295.patch, 
 HDFS-3295.patch, HDFS-3295.patch


 Scenario:
 Replication factor = 
 1,fs.defaultFS=hdfs://namenodeip:port,dfs.namenode.rpc-address=namenodeip:port.
 step 1: started DN1.Pumped 4.67GB of data
 step 2: started DN2.
 step 3: issued the balancer cmd(./start-balancer.sh -threshold 1)
 Totally  848.41 MB  has been moved to 2DN and took 4 iterations to move the 
 blocks to 2DN.
 But in balancer output in all the iterations the number of bytes already 
 moved is alway 0KB  and there is mismatch in the Bytes being moved with the 
 bytes left to move
 Balancer output for 2nd Iteration
 =
 {noformat}
 Time Stamp   Iteration#  Bytes Already Moved  Bytes Left To Move  
 Bytes Being Moved
 Apr 18, 2012 12:41:20 PM  0 0 KB   848.41 MB  
 151.18 MB
 Apr 18, 2012 1:06:46 PM   1 0 KB   646.02 MB  
 151.18 MB
 Apr 18, 2012 1:10:28 PM   2 0 KB   417.43 MB  
 151.18 MB
 Apr 18, 2012 1:14:04 PM   3 0 KB   223.84 MB  
 151.18 MB
 Apr 18, 2012 1:18:10 PM   4 0 KB30.34 MB  
 151.18 MB{noformat}
 In the above balancer output
 •Bytes already moved is 0KB
 • In 3 rd iterarion Bytes left to move is 223.84 MB but Bytes being moved is 
 151.18 MB . So in next iteration the bytes left to move should be 223.84 MB - 
 151.18 MB 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[jira] [Updated] (HDFS-3295) Mismatch in number of bytes already moved and number of bytes being moved in balancer report

2012-05-26 Thread Ashish Singhi (JIRA)

 [ 
https://issues.apache.org/jira/browse/HDFS-3295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi updated HDFS-3295:


Status: Open  (was: Patch Available)

 Mismatch in number of bytes already moved and number of bytes being moved in 
 balancer report
 

 Key: HDFS-3295
 URL: https://issues.apache.org/jira/browse/HDFS-3295
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: balancer
Affects Versions: 0.23.0
Reporter: J.Andreina
Assignee: Ashish Singhi
 Fix For: 0.24.0

 Attachments: HDFS-3295.patch, HDFS-3295.patch, HDFS-3295.patch


 Scenario:
 Replication factor = 
 1,fs.defaultFS=hdfs://namenodeip:port,dfs.namenode.rpc-address=namenodeip:port.
 step 1: started DN1.Pumped 4.67GB of data
 step 2: started DN2.
 step 3: issued the balancer cmd(./start-balancer.sh -threshold 1)
 Totally  848.41 MB  has been moved to 2DN and took 4 iterations to move the 
 blocks to 2DN.
 But in balancer output in all the iterations the number of bytes already 
 moved is alway 0KB  and there is mismatch in the Bytes being moved with the 
 bytes left to move
 Balancer output for 2nd Iteration
 =
 {noformat}
 Time Stamp   Iteration#  Bytes Already Moved  Bytes Left To Move  
 Bytes Being Moved
 Apr 18, 2012 12:41:20 PM  0 0 KB   848.41 MB  
 151.18 MB
 Apr 18, 2012 1:06:46 PM   1 0 KB   646.02 MB  
 151.18 MB
 Apr 18, 2012 1:10:28 PM   2 0 KB   417.43 MB  
 151.18 MB
 Apr 18, 2012 1:14:04 PM   3 0 KB   223.84 MB  
 151.18 MB
 Apr 18, 2012 1:18:10 PM   4 0 KB30.34 MB  
 151.18 MB{noformat}
 In the above balancer output
 •Bytes already moved is 0KB
 • In 3 rd iterarion Bytes left to move is 223.84 MB but Bytes being moved is 
 151.18 MB . So in next iteration the bytes left to move should be 223.84 MB - 
 151.18 MB 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HDFS-3295) Mismatch in number of bytes already moved and number of bytes being moved in balancer report

2012-05-26 Thread Ashish Singhi (JIRA)

 [ 
https://issues.apache.org/jira/browse/HDFS-3295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi updated HDFS-3295:


Attachment: HDFS-3295.patch

 Mismatch in number of bytes already moved and number of bytes being moved in 
 balancer report
 

 Key: HDFS-3295
 URL: https://issues.apache.org/jira/browse/HDFS-3295
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: balancer
Affects Versions: 0.23.0
Reporter: J.Andreina
Assignee: Ashish Singhi
 Fix For: 0.24.0

 Attachments: HDFS-3295.patch, HDFS-3295.patch, HDFS-3295.patch, 
 HDFS-3295.patch


 Scenario:
 Replication factor = 
 1,fs.defaultFS=hdfs://namenodeip:port,dfs.namenode.rpc-address=namenodeip:port.
 step 1: started DN1.Pumped 4.67GB of data
 step 2: started DN2.
 step 3: issued the balancer cmd(./start-balancer.sh -threshold 1)
 Totally  848.41 MB  has been moved to 2DN and took 4 iterations to move the 
 blocks to 2DN.
 But in balancer output in all the iterations the number of bytes already 
 moved is alway 0KB  and there is mismatch in the Bytes being moved with the 
 bytes left to move
 Balancer output for 2nd Iteration
 =
 {noformat}
 Time Stamp   Iteration#  Bytes Already Moved  Bytes Left To Move  
 Bytes Being Moved
 Apr 18, 2012 12:41:20 PM  0 0 KB   848.41 MB  
 151.18 MB
 Apr 18, 2012 1:06:46 PM   1 0 KB   646.02 MB  
 151.18 MB
 Apr 18, 2012 1:10:28 PM   2 0 KB   417.43 MB  
 151.18 MB
 Apr 18, 2012 1:14:04 PM   3 0 KB   223.84 MB  
 151.18 MB
 Apr 18, 2012 1:18:10 PM   4 0 KB30.34 MB  
 151.18 MB{noformat}
 In the above balancer output
 •Bytes already moved is 0KB
 • In 3 rd iterarion Bytes left to move is 223.84 MB but Bytes being moved is 
 151.18 MB . So in next iteration the bytes left to move should be 223.84 MB - 
 151.18 MB 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HDFS-3295) Mismatch in number of bytes already moved and number of bytes being moved in balancer report

2012-05-26 Thread Ashish Singhi (JIRA)

 [ 
https://issues.apache.org/jira/browse/HDFS-3295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi updated HDFS-3295:


Status: Patch Available  (was: Open)

 Mismatch in number of bytes already moved and number of bytes being moved in 
 balancer report
 

 Key: HDFS-3295
 URL: https://issues.apache.org/jira/browse/HDFS-3295
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: balancer
Affects Versions: 0.23.0
Reporter: J.Andreina
Assignee: Ashish Singhi
 Fix For: 0.24.0

 Attachments: HDFS-3295.patch, HDFS-3295.patch, HDFS-3295.patch, 
 HDFS-3295.patch


 Scenario:
 Replication factor = 
 1,fs.defaultFS=hdfs://namenodeip:port,dfs.namenode.rpc-address=namenodeip:port.
 step 1: started DN1.Pumped 4.67GB of data
 step 2: started DN2.
 step 3: issued the balancer cmd(./start-balancer.sh -threshold 1)
 Totally  848.41 MB  has been moved to 2DN and took 4 iterations to move the 
 blocks to 2DN.
 But in balancer output in all the iterations the number of bytes already 
 moved is alway 0KB  and there is mismatch in the Bytes being moved with the 
 bytes left to move
 Balancer output for 2nd Iteration
 =
 {noformat}
 Time Stamp   Iteration#  Bytes Already Moved  Bytes Left To Move  
 Bytes Being Moved
 Apr 18, 2012 12:41:20 PM  0 0 KB   848.41 MB  
 151.18 MB
 Apr 18, 2012 1:06:46 PM   1 0 KB   646.02 MB  
 151.18 MB
 Apr 18, 2012 1:10:28 PM   2 0 KB   417.43 MB  
 151.18 MB
 Apr 18, 2012 1:14:04 PM   3 0 KB   223.84 MB  
 151.18 MB
 Apr 18, 2012 1:18:10 PM   4 0 KB30.34 MB  
 151.18 MB{noformat}
 In the above balancer output
 •Bytes already moved is 0KB
 • In 3 rd iterarion Bytes left to move is 223.84 MB but Bytes being moved is 
 151.18 MB . So in next iteration the bytes left to move should be 223.84 MB - 
 151.18 MB 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HDFS-3295) Mismatch in number of bytes already moved and number of bytes being moved in balancer report

2012-05-26 Thread Ashish Singhi (JIRA)

 [ 
https://issues.apache.org/jira/browse/HDFS-3295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi updated HDFS-3295:


Status: Open  (was: Patch Available)

 Mismatch in number of bytes already moved and number of bytes being moved in 
 balancer report
 

 Key: HDFS-3295
 URL: https://issues.apache.org/jira/browse/HDFS-3295
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: balancer
Affects Versions: 0.23.0
Reporter: J.Andreina
Assignee: Ashish Singhi
 Fix For: 0.24.0

 Attachments: HDFS-3295.patch, HDFS-3295.patch, HDFS-3295.patch, 
 HDFS-3295.patch


 Scenario:
 Replication factor = 
 1,fs.defaultFS=hdfs://namenodeip:port,dfs.namenode.rpc-address=namenodeip:port.
 step 1: started DN1.Pumped 4.67GB of data
 step 2: started DN2.
 step 3: issued the balancer cmd(./start-balancer.sh -threshold 1)
 Totally  848.41 MB  has been moved to 2DN and took 4 iterations to move the 
 blocks to 2DN.
 But in balancer output in all the iterations the number of bytes already 
 moved is alway 0KB  and there is mismatch in the Bytes being moved with the 
 bytes left to move
 Balancer output for 2nd Iteration
 =
 {noformat}
 Time Stamp   Iteration#  Bytes Already Moved  Bytes Left To Move  
 Bytes Being Moved
 Apr 18, 2012 12:41:20 PM  0 0 KB   848.41 MB  
 151.18 MB
 Apr 18, 2012 1:06:46 PM   1 0 KB   646.02 MB  
 151.18 MB
 Apr 18, 2012 1:10:28 PM   2 0 KB   417.43 MB  
 151.18 MB
 Apr 18, 2012 1:14:04 PM   3 0 KB   223.84 MB  
 151.18 MB
 Apr 18, 2012 1:18:10 PM   4 0 KB30.34 MB  
 151.18 MB{noformat}
 In the above balancer output
 •Bytes already moved is 0KB
 • In 3 rd iterarion Bytes left to move is 223.84 MB but Bytes being moved is 
 151.18 MB . So in next iteration the bytes left to move should be 223.84 MB - 
 151.18 MB 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HDFS-3295) Mismatch in number of bytes already moved and number of bytes being moved in balancer report

2012-05-26 Thread Ashish Singhi (JIRA)

 [ 
https://issues.apache.org/jira/browse/HDFS-3295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi updated HDFS-3295:


Attachment: HDFS-3295.patch

 Mismatch in number of bytes already moved and number of bytes being moved in 
 balancer report
 

 Key: HDFS-3295
 URL: https://issues.apache.org/jira/browse/HDFS-3295
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: balancer
Affects Versions: 0.23.0
Reporter: J.Andreina
Assignee: Ashish Singhi
 Fix For: 0.24.0

 Attachments: HDFS-3295.patch, HDFS-3295.patch, HDFS-3295.patch, 
 HDFS-3295.patch, HDFS-3295.patch


 Scenario:
 Replication factor = 
 1,fs.defaultFS=hdfs://namenodeip:port,dfs.namenode.rpc-address=namenodeip:port.
 step 1: started DN1.Pumped 4.67GB of data
 step 2: started DN2.
 step 3: issued the balancer cmd(./start-balancer.sh -threshold 1)
 Totally  848.41 MB  has been moved to 2DN and took 4 iterations to move the 
 blocks to 2DN.
 But in balancer output in all the iterations the number of bytes already 
 moved is alway 0KB  and there is mismatch in the Bytes being moved with the 
 bytes left to move
 Balancer output for 2nd Iteration
 =
 {noformat}
 Time Stamp   Iteration#  Bytes Already Moved  Bytes Left To Move  
 Bytes Being Moved
 Apr 18, 2012 12:41:20 PM  0 0 KB   848.41 MB  
 151.18 MB
 Apr 18, 2012 1:06:46 PM   1 0 KB   646.02 MB  
 151.18 MB
 Apr 18, 2012 1:10:28 PM   2 0 KB   417.43 MB  
 151.18 MB
 Apr 18, 2012 1:14:04 PM   3 0 KB   223.84 MB  
 151.18 MB
 Apr 18, 2012 1:18:10 PM   4 0 KB30.34 MB  
 151.18 MB{noformat}
 In the above balancer output
 •Bytes already moved is 0KB
 • In 3 rd iterarion Bytes left to move is 223.84 MB but Bytes being moved is 
 151.18 MB . So in next iteration the bytes left to move should be 223.84 MB - 
 151.18 MB 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HDFS-3295) Mismatch in number of bytes already moved and number of bytes being moved in balancer report

2012-05-26 Thread Ashish Singhi (JIRA)

 [ 
https://issues.apache.org/jira/browse/HDFS-3295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi updated HDFS-3295:


Status: Patch Available  (was: Open)

In test case the shutdownDatanode thread before shutting down the datanode 
balancer completed all the iterations. Thus using CountDownLatch to make sure 
that first datanode gets shutdown in the second iteration and then balancer 
resumes with its iterations.

 Mismatch in number of bytes already moved and number of bytes being moved in 
 balancer report
 

 Key: HDFS-3295
 URL: https://issues.apache.org/jira/browse/HDFS-3295
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: balancer
Affects Versions: 0.23.0
Reporter: J.Andreina
Assignee: Ashish Singhi
 Fix For: 0.24.0

 Attachments: HDFS-3295.patch, HDFS-3295.patch, HDFS-3295.patch, 
 HDFS-3295.patch, HDFS-3295.patch


 Scenario:
 Replication factor = 
 1,fs.defaultFS=hdfs://namenodeip:port,dfs.namenode.rpc-address=namenodeip:port.
 step 1: started DN1.Pumped 4.67GB of data
 step 2: started DN2.
 step 3: issued the balancer cmd(./start-balancer.sh -threshold 1)
 Totally  848.41 MB  has been moved to 2DN and took 4 iterations to move the 
 blocks to 2DN.
 But in balancer output in all the iterations the number of bytes already 
 moved is alway 0KB  and there is mismatch in the Bytes being moved with the 
 bytes left to move
 Balancer output for 2nd Iteration
 =
 {noformat}
 Time Stamp   Iteration#  Bytes Already Moved  Bytes Left To Move  
 Bytes Being Moved
 Apr 18, 2012 12:41:20 PM  0 0 KB   848.41 MB  
 151.18 MB
 Apr 18, 2012 1:06:46 PM   1 0 KB   646.02 MB  
 151.18 MB
 Apr 18, 2012 1:10:28 PM   2 0 KB   417.43 MB  
 151.18 MB
 Apr 18, 2012 1:14:04 PM   3 0 KB   223.84 MB  
 151.18 MB
 Apr 18, 2012 1:18:10 PM   4 0 KB30.34 MB  
 151.18 MB{noformat}
 In the above balancer output
 •Bytes already moved is 0KB
 • In 3 rd iterarion Bytes left to move is 223.84 MB but Bytes being moved is 
 151.18 MB . So in next iteration the bytes left to move should be 223.84 MB - 
 151.18 MB 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HDFS-3295) Mismatch in number of bytes already moved and number of bytes being moved in balancer report

2012-05-21 Thread Ashish Singhi (JIRA)

 [ 
https://issues.apache.org/jira/browse/HDFS-3295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi updated HDFS-3295:


Status: Open  (was: Patch Available)

 Mismatch in number of bytes already moved and number of bytes being moved in 
 balancer report
 

 Key: HDFS-3295
 URL: https://issues.apache.org/jira/browse/HDFS-3295
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: balancer
Affects Versions: 0.23.0
Reporter: J.Andreina
Assignee: Ashish Singhi
 Fix For: 0.24.0

 Attachments: HDFS-3295.patch, HDFS-3295.patch


 Scenario:
 Replication factor = 
 1,fs.defaultFS=hdfs://namenodeip:port,dfs.namenode.rpc-address=namenodeip:port.
 step 1: started DN1.Pumped 4.67GB of data
 step 2: started DN2.
 step 3: issued the balancer cmd(./start-balancer.sh -threshold 1)
 Totally  848.41 MB  has been moved to 2DN and took 4 iterations to move the 
 blocks to 2DN.
 But in balancer output in all the iterations the number of bytes already 
 moved is alway 0KB  and there is mismatch in the Bytes being moved with the 
 bytes left to move
 Balancer output for 2nd Iteration
 =
 {noformat}
 Time Stamp   Iteration#  Bytes Already Moved  Bytes Left To Move  
 Bytes Being Moved
 Apr 18, 2012 12:41:20 PM  0 0 KB   848.41 MB  
 151.18 MB
 Apr 18, 2012 1:06:46 PM   1 0 KB   646.02 MB  
 151.18 MB
 Apr 18, 2012 1:10:28 PM   2 0 KB   417.43 MB  
 151.18 MB
 Apr 18, 2012 1:14:04 PM   3 0 KB   223.84 MB  
 151.18 MB
 Apr 18, 2012 1:18:10 PM   4 0 KB30.34 MB  
 151.18 MB{noformat}
 In the above balancer output
 •Bytes already moved is 0KB
 • In 3 rd iterarion Bytes left to move is 223.84 MB but Bytes being moved is 
 151.18 MB . So in next iteration the bytes left to move should be 223.84 MB - 
 151.18 MB 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HDFS-3295) Mismatch in number of bytes already moved and number of bytes being moved in balancer report

2012-05-21 Thread Ashish Singhi (JIRA)

 [ 
https://issues.apache.org/jira/browse/HDFS-3295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi updated HDFS-3295:


Attachment: HDFS-3295.patch

 Mismatch in number of bytes already moved and number of bytes being moved in 
 balancer report
 

 Key: HDFS-3295
 URL: https://issues.apache.org/jira/browse/HDFS-3295
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: balancer
Affects Versions: 0.23.0
Reporter: J.Andreina
Assignee: Ashish Singhi
 Fix For: 0.24.0

 Attachments: HDFS-3295.patch, HDFS-3295.patch, HDFS-3295.patch


 Scenario:
 Replication factor = 
 1,fs.defaultFS=hdfs://namenodeip:port,dfs.namenode.rpc-address=namenodeip:port.
 step 1: started DN1.Pumped 4.67GB of data
 step 2: started DN2.
 step 3: issued the balancer cmd(./start-balancer.sh -threshold 1)
 Totally  848.41 MB  has been moved to 2DN and took 4 iterations to move the 
 blocks to 2DN.
 But in balancer output in all the iterations the number of bytes already 
 moved is alway 0KB  and there is mismatch in the Bytes being moved with the 
 bytes left to move
 Balancer output for 2nd Iteration
 =
 {noformat}
 Time Stamp   Iteration#  Bytes Already Moved  Bytes Left To Move  
 Bytes Being Moved
 Apr 18, 2012 12:41:20 PM  0 0 KB   848.41 MB  
 151.18 MB
 Apr 18, 2012 1:06:46 PM   1 0 KB   646.02 MB  
 151.18 MB
 Apr 18, 2012 1:10:28 PM   2 0 KB   417.43 MB  
 151.18 MB
 Apr 18, 2012 1:14:04 PM   3 0 KB   223.84 MB  
 151.18 MB
 Apr 18, 2012 1:18:10 PM   4 0 KB30.34 MB  
 151.18 MB{noformat}
 In the above balancer output
 •Bytes already moved is 0KB
 • In 3 rd iterarion Bytes left to move is 223.84 MB but Bytes being moved is 
 151.18 MB . So in next iteration the bytes left to move should be 223.84 MB - 
 151.18 MB 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HDFS-3295) Mismatch in number of bytes already moved and number of bytes being moved in balancer report

2012-05-21 Thread Ashish Singhi (JIRA)

 [ 
https://issues.apache.org/jira/browse/HDFS-3295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi updated HDFS-3295:


Status: Patch Available  (was: Open)

 Mismatch in number of bytes already moved and number of bytes being moved in 
 balancer report
 

 Key: HDFS-3295
 URL: https://issues.apache.org/jira/browse/HDFS-3295
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: balancer
Affects Versions: 0.23.0
Reporter: J.Andreina
Assignee: Ashish Singhi
 Fix For: 0.24.0

 Attachments: HDFS-3295.patch, HDFS-3295.patch, HDFS-3295.patch


 Scenario:
 Replication factor = 
 1,fs.defaultFS=hdfs://namenodeip:port,dfs.namenode.rpc-address=namenodeip:port.
 step 1: started DN1.Pumped 4.67GB of data
 step 2: started DN2.
 step 3: issued the balancer cmd(./start-balancer.sh -threshold 1)
 Totally  848.41 MB  has been moved to 2DN and took 4 iterations to move the 
 blocks to 2DN.
 But in balancer output in all the iterations the number of bytes already 
 moved is alway 0KB  and there is mismatch in the Bytes being moved with the 
 bytes left to move
 Balancer output for 2nd Iteration
 =
 {noformat}
 Time Stamp   Iteration#  Bytes Already Moved  Bytes Left To Move  
 Bytes Being Moved
 Apr 18, 2012 12:41:20 PM  0 0 KB   848.41 MB  
 151.18 MB
 Apr 18, 2012 1:06:46 PM   1 0 KB   646.02 MB  
 151.18 MB
 Apr 18, 2012 1:10:28 PM   2 0 KB   417.43 MB  
 151.18 MB
 Apr 18, 2012 1:14:04 PM   3 0 KB   223.84 MB  
 151.18 MB
 Apr 18, 2012 1:18:10 PM   4 0 KB30.34 MB  
 151.18 MB{noformat}
 In the above balancer output
 •Bytes already moved is 0KB
 • In 3 rd iterarion Bytes left to move is 223.84 MB but Bytes being moved is 
 151.18 MB . So in next iteration the bytes left to move should be 223.84 MB - 
 151.18 MB 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HDFS-3295) Mismatch in number of bytes already moved and number of bytes being moved in balancer report

2012-05-20 Thread Ashish Singhi (JIRA)

 [ 
https://issues.apache.org/jira/browse/HDFS-3295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi updated HDFS-3295:


Status: Open  (was: Patch Available)

 Mismatch in number of bytes already moved and number of bytes being moved in 
 balancer report
 

 Key: HDFS-3295
 URL: https://issues.apache.org/jira/browse/HDFS-3295
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: balancer
Affects Versions: 0.23.0
Reporter: J.Andreina
Assignee: Ashish Singhi
 Fix For: 0.24.0

 Attachments: HDFS-3295.patch


 Scenario:
 Replication factor = 
 1,fs.defaultFS=hdfs://namenodeip:port,dfs.namenode.rpc-address=namenodeip:port.
 step 1: started DN1.Pumped 4.67GB of data
 step 2: started DN2.
 step 3: issued the balancer cmd(./start-balancer.sh -threshold 1)
 Totally  848.41 MB  has been moved to 2DN and took 4 iterations to move the 
 blocks to 2DN.
 But in balancer output in all the iterations the number of bytes already 
 moved is alway 0KB  and there is mismatch in the Bytes being moved with the 
 bytes left to move
 Balancer output for 2nd Iteration
 =
 {noformat}
 Time Stamp   Iteration#  Bytes Already Moved  Bytes Left To Move  
 Bytes Being Moved
 Apr 18, 2012 12:41:20 PM  0 0 KB   848.41 MB  
 151.18 MB
 Apr 18, 2012 1:06:46 PM   1 0 KB   646.02 MB  
 151.18 MB
 Apr 18, 2012 1:10:28 PM   2 0 KB   417.43 MB  
 151.18 MB
 Apr 18, 2012 1:14:04 PM   3 0 KB   223.84 MB  
 151.18 MB
 Apr 18, 2012 1:18:10 PM   4 0 KB30.34 MB  
 151.18 MB{noformat}
 In the above balancer output
 •Bytes already moved is 0KB
 • In 3 rd iterarion Bytes left to move is 223.84 MB but Bytes being moved is 
 151.18 MB . So in next iteration the bytes left to move should be 223.84 MB - 
 151.18 MB 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HDFS-3295) Mismatch in number of bytes already moved and number of bytes being moved in balancer report

2012-05-20 Thread Ashish Singhi (JIRA)

 [ 
https://issues.apache.org/jira/browse/HDFS-3295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi updated HDFS-3295:


Attachment: HDFS-3295.patch

 Mismatch in number of bytes already moved and number of bytes being moved in 
 balancer report
 

 Key: HDFS-3295
 URL: https://issues.apache.org/jira/browse/HDFS-3295
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: balancer
Affects Versions: 0.23.0
Reporter: J.Andreina
Assignee: Ashish Singhi
 Fix For: 0.24.0

 Attachments: HDFS-3295.patch, HDFS-3295.patch


 Scenario:
 Replication factor = 
 1,fs.defaultFS=hdfs://namenodeip:port,dfs.namenode.rpc-address=namenodeip:port.
 step 1: started DN1.Pumped 4.67GB of data
 step 2: started DN2.
 step 3: issued the balancer cmd(./start-balancer.sh -threshold 1)
 Totally  848.41 MB  has been moved to 2DN and took 4 iterations to move the 
 blocks to 2DN.
 But in balancer output in all the iterations the number of bytes already 
 moved is alway 0KB  and there is mismatch in the Bytes being moved with the 
 bytes left to move
 Balancer output for 2nd Iteration
 =
 {noformat}
 Time Stamp   Iteration#  Bytes Already Moved  Bytes Left To Move  
 Bytes Being Moved
 Apr 18, 2012 12:41:20 PM  0 0 KB   848.41 MB  
 151.18 MB
 Apr 18, 2012 1:06:46 PM   1 0 KB   646.02 MB  
 151.18 MB
 Apr 18, 2012 1:10:28 PM   2 0 KB   417.43 MB  
 151.18 MB
 Apr 18, 2012 1:14:04 PM   3 0 KB   223.84 MB  
 151.18 MB
 Apr 18, 2012 1:18:10 PM   4 0 KB30.34 MB  
 151.18 MB{noformat}
 In the above balancer output
 •Bytes already moved is 0KB
 • In 3 rd iterarion Bytes left to move is 223.84 MB but Bytes being moved is 
 151.18 MB . So in next iteration the bytes left to move should be 223.84 MB - 
 151.18 MB 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HDFS-3295) Mismatch in number of bytes already moved and number of bytes being moved in balancer report

2012-05-20 Thread Ashish Singhi (JIRA)

 [ 
https://issues.apache.org/jira/browse/HDFS-3295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi updated HDFS-3295:


Status: Patch Available  (was: Open)

Updated the test case in the patch. This time before starting the balancer 
waiting for the replication set for the file to complete.

 Mismatch in number of bytes already moved and number of bytes being moved in 
 balancer report
 

 Key: HDFS-3295
 URL: https://issues.apache.org/jira/browse/HDFS-3295
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: balancer
Affects Versions: 0.23.0
Reporter: J.Andreina
Assignee: Ashish Singhi
 Fix For: 0.24.0

 Attachments: HDFS-3295.patch, HDFS-3295.patch


 Scenario:
 Replication factor = 
 1,fs.defaultFS=hdfs://namenodeip:port,dfs.namenode.rpc-address=namenodeip:port.
 step 1: started DN1.Pumped 4.67GB of data
 step 2: started DN2.
 step 3: issued the balancer cmd(./start-balancer.sh -threshold 1)
 Totally  848.41 MB  has been moved to 2DN and took 4 iterations to move the 
 blocks to 2DN.
 But in balancer output in all the iterations the number of bytes already 
 moved is alway 0KB  and there is mismatch in the Bytes being moved with the 
 bytes left to move
 Balancer output for 2nd Iteration
 =
 {noformat}
 Time Stamp   Iteration#  Bytes Already Moved  Bytes Left To Move  
 Bytes Being Moved
 Apr 18, 2012 12:41:20 PM  0 0 KB   848.41 MB  
 151.18 MB
 Apr 18, 2012 1:06:46 PM   1 0 KB   646.02 MB  
 151.18 MB
 Apr 18, 2012 1:10:28 PM   2 0 KB   417.43 MB  
 151.18 MB
 Apr 18, 2012 1:14:04 PM   3 0 KB   223.84 MB  
 151.18 MB
 Apr 18, 2012 1:18:10 PM   4 0 KB30.34 MB  
 151.18 MB{noformat}
 In the above balancer output
 •Bytes already moved is 0KB
 • In 3 rd iterarion Bytes left to move is 223.84 MB but Bytes being moved is 
 151.18 MB . So in next iteration the bytes left to move should be 223.84 MB - 
 151.18 MB 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HDFS-3295) Mismatch in number of bytes already moved and number of bytes being moved in balancer report

2012-05-18 Thread Ashish Singhi (JIRA)

 [ 
https://issues.apache.org/jira/browse/HDFS-3295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi updated HDFS-3295:


Attachment: HDFS-3295.patch

 Mismatch in number of bytes already moved and number of bytes being moved in 
 balancer report
 

 Key: HDFS-3295
 URL: https://issues.apache.org/jira/browse/HDFS-3295
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: balancer
Affects Versions: 0.23.0
Reporter: J.Andreina
Assignee: Ashish Singhi
 Fix For: 0.24.0

 Attachments: HDFS-3295.patch


 Scenario:
 Replication factor = 
 1,fs.defaultFS=hdfs://namenodeip:port,dfs.namenode.rpc-address=namenodeip:port.
 step 1: started DN1.Pumped 4.67GB of data
 step 2: started DN2.
 step 3: issued the balancer cmd(./start-balancer.sh -threshold 1)
 Totally  848.41 MB  has been moved to 2DN and took 4 iterations to move the 
 blocks to 2DN.
 But in balancer output in all the iterations the number of bytes already 
 moved is alway 0KB  and there is mismatch in the Bytes being moved with the 
 bytes left to move
 Balancer output for 2nd Iteration
 =
 {noformat}
 Time Stamp   Iteration#  Bytes Already Moved  Bytes Left To Move  
 Bytes Being Moved
 Apr 18, 2012 12:41:20 PM  0 0 KB   848.41 MB  
 151.18 MB
 Apr 18, 2012 1:06:46 PM   1 0 KB   646.02 MB  
 151.18 MB
 Apr 18, 2012 1:10:28 PM   2 0 KB   417.43 MB  
 151.18 MB
 Apr 18, 2012 1:14:04 PM   3 0 KB   223.84 MB  
 151.18 MB
 Apr 18, 2012 1:18:10 PM   4 0 KB30.34 MB  
 151.18 MB{noformat}
 In the above balancer output
 •Bytes already moved is 0KB
 • In 3 rd iterarion Bytes left to move is 223.84 MB but Bytes being moved is 
 151.18 MB . So in next iteration the bytes left to move should be 223.84 MB - 
 151.18 MB 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HDFS-3295) Mismatch in number of bytes already moved and number of bytes being moved in balancer report

2012-05-18 Thread Ashish Singhi (JIRA)

 [ 
https://issues.apache.org/jira/browse/HDFS-3295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi updated HDFS-3295:


Status: Patch Available  (was: Open)

 Mismatch in number of bytes already moved and number of bytes being moved in 
 balancer report
 

 Key: HDFS-3295
 URL: https://issues.apache.org/jira/browse/HDFS-3295
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: balancer
Affects Versions: 0.23.0
Reporter: J.Andreina
Assignee: Ashish Singhi
 Fix For: 0.24.0

 Attachments: HDFS-3295.patch


 Scenario:
 Replication factor = 
 1,fs.defaultFS=hdfs://namenodeip:port,dfs.namenode.rpc-address=namenodeip:port.
 step 1: started DN1.Pumped 4.67GB of data
 step 2: started DN2.
 step 3: issued the balancer cmd(./start-balancer.sh -threshold 1)
 Totally  848.41 MB  has been moved to 2DN and took 4 iterations to move the 
 blocks to 2DN.
 But in balancer output in all the iterations the number of bytes already 
 moved is alway 0KB  and there is mismatch in the Bytes being moved with the 
 bytes left to move
 Balancer output for 2nd Iteration
 =
 {noformat}
 Time Stamp   Iteration#  Bytes Already Moved  Bytes Left To Move  
 Bytes Being Moved
 Apr 18, 2012 12:41:20 PM  0 0 KB   848.41 MB  
 151.18 MB
 Apr 18, 2012 1:06:46 PM   1 0 KB   646.02 MB  
 151.18 MB
 Apr 18, 2012 1:10:28 PM   2 0 KB   417.43 MB  
 151.18 MB
 Apr 18, 2012 1:14:04 PM   3 0 KB   223.84 MB  
 151.18 MB
 Apr 18, 2012 1:18:10 PM   4 0 KB30.34 MB  
 151.18 MB{noformat}
 In the above balancer output
 •Bytes already moved is 0KB
 • In 3 rd iterarion Bytes left to move is 223.84 MB but Bytes being moved is 
 151.18 MB . So in next iteration the bytes left to move should be 223.84 MB - 
 151.18 MB 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Updated] (HDFS-3295) Mismatch in number of bytes already moved and number of bytes being moved in balancer report

2012-04-24 Thread Ashish Singhi (JIRA)

 [ 
https://issues.apache.org/jira/browse/HDFS-3295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi updated HDFS-3295:


Assignee: (was: Ashish Singhi)

 Mismatch in number of bytes already moved and number of bytes being moved in 
 balancer report
 

 Key: HDFS-3295
 URL: https://issues.apache.org/jira/browse/HDFS-3295
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: balancer
Affects Versions: 0.23.0
Reporter: J.Andreina
 Fix For: 0.24.0


 Scenario:
 Replication factor = 
 1,fs.defaultFS=hdfs://namenodeip:port,dfs.namenode.rpc-address=namenodeip:port.
 step 1: started DN1.Pumped 4.67GB of data
 step 2: started DN2.
 step 3: issued the balancer cmd(./start-balancer.sh -threshold 1)
 Totally  848.41 MB  has been moved to 2DN and took 4 iterations to move the 
 blocks to 2DN.
 But in balancer output in all the iterations the number of bytes already 
 moved is alway 0KB  and there is mismatch in the Bytes being moved with the 
 bytes left to move
 Balancer output for 2nd Iteration
 =
 {noformat}
 Time Stamp   Iteration#  Bytes Already Moved  Bytes Left To Move  
 Bytes Being Moved
 Apr 18, 2012 12:41:20 PM  0 0 KB   848.41 MB  
 151.18 MB
 Apr 18, 2012 1:06:46 PM   1 0 KB   646.02 MB  
 151.18 MB
 Apr 18, 2012 1:10:28 PM   2 0 KB   417.43 MB  
 151.18 MB
 Apr 18, 2012 1:14:04 PM   3 0 KB   223.84 MB  
 151.18 MB
 Apr 18, 2012 1:18:10 PM   4 0 KB30.34 MB  
 151.18 MB{noformat}
 In the above balancer output
 •Bytes already moved is 0KB
 • In 3 rd iterarion Bytes left to move is 223.84 MB but Bytes being moved is 
 151.18 MB . So in next iteration the bytes left to move should be 223.84 MB - 
 151.18 MB 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira