[jira] [Updated] (HDFS-3286) When the threshold value for balancer is 0(zero) ,unexpected output is displayed
[
https://issues.apache.org/jira/browse/HDFS-3286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashish Singhi updated HDFS-3286:
Target Version/s: 2.0.0, 3.0.0 (was: 3.0.0, 2.0.0)
Status: Open (was: Patch Available)
When the threshold value for balancer is 0(zero) ,unexpected output is
displayed
Key: HDFS-3286
URL: https://issues.apache.org/jira/browse/HDFS-3286
Project: Hadoop HDFS
Issue Type: Bug
Components: balancer
Affects Versions: 0.23.0
Reporter: J.Andreina
Assignee: Ashish Singhi
Fix For: 0.24.0
Attachments: HDFS-3286.patch, HDFS-3286.patch
Replication factor =1
Step 1: Start NN,DN1.write 4 GB of data
Step 2: Start DN2
Step 3: issue the balancer command(./hdfs balancer -threshold 0)
The threshold parameter is a fraction in the range of (0%, 100%) with a
default value of 10%
When the above scenario is executed the Source DN and Target DN is choosen
and the number of bytes to be moved from source to target DN is also
calculated .
Then the balancer is exiting with the following message No block can be
moved. Exiting... which is not expected.
{noformat}
HOST-xx-xx-xx-xx:/home/Andreina/APril10/install/hadoop/namenode/bin # ./hdfs
balancer -threshold 0
12/04/16 16:22:07 INFO balancer.Balancer: Using a threshold of 0.0
12/04/16 16:22:07 INFO balancer.Balancer: namenodes =
[hdfs://HOST-xx-xx-xx-xx:9000]
12/04/16 16:22:07 INFO balancer.Balancer: p =
Balancer.Parameters[BalancingPolicy.Node, threshold=0.0]
Time Stamp Iteration# Bytes Already Moved Bytes Left To Move
Bytes Being Moved
12/04/16 16:22:10 INFO net.NetworkTopology: Adding a new node:
/default-rack/yy.yy.yy.yy:50176
12/04/16 16:22:10 INFO net.NetworkTopology: Adding a new node:
/default-rack/xx.xx.xx.xx:50010
12/04/16 16:22:10 INFO balancer.Balancer: 1 over-utilized:
[Source[xx.xx.xx.xx:50010, utilization=7.212458091389678]]
12/04/16 16:22:10 INFO balancer.Balancer: 1 underutilized:
[BalancerDatanode[yy.yy.yy.yy:50176, utilization=4.650670324367203E-5]]
12/04/16 16:22:10 INFO balancer.Balancer: Need to move 1.77 GB to make the
cluster balanced.
No block can be moved. Exiting...
Balancing took 5.142 seconds
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3286) When the threshold value for balancer is 0(zero) ,unexpected output is displayed
[
https://issues.apache.org/jira/browse/HDFS-3286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashish Singhi updated HDFS-3286:
Attachment: HDFS-3286.patch
Thanks Uma and Aaron for taking a look at the patch.
Uploaded a patch addressing all your comments.
When the threshold value for balancer is 0(zero) ,unexpected output is
displayed
Key: HDFS-3286
URL: https://issues.apache.org/jira/browse/HDFS-3286
Project: Hadoop HDFS
Issue Type: Bug
Components: balancer
Affects Versions: 0.23.0
Reporter: J.Andreina
Assignee: Ashish Singhi
Fix For: 0.24.0
Attachments: HDFS-3286.patch, HDFS-3286.patch, HDFS-3286.patch
Replication factor =1
Step 1: Start NN,DN1.write 4 GB of data
Step 2: Start DN2
Step 3: issue the balancer command(./hdfs balancer -threshold 0)
The threshold parameter is a fraction in the range of (0%, 100%) with a
default value of 10%
When the above scenario is executed the Source DN and Target DN is choosen
and the number of bytes to be moved from source to target DN is also
calculated .
Then the balancer is exiting with the following message No block can be
moved. Exiting... which is not expected.
{noformat}
HOST-xx-xx-xx-xx:/home/Andreina/APril10/install/hadoop/namenode/bin # ./hdfs
balancer -threshold 0
12/04/16 16:22:07 INFO balancer.Balancer: Using a threshold of 0.0
12/04/16 16:22:07 INFO balancer.Balancer: namenodes =
[hdfs://HOST-xx-xx-xx-xx:9000]
12/04/16 16:22:07 INFO balancer.Balancer: p =
Balancer.Parameters[BalancingPolicy.Node, threshold=0.0]
Time Stamp Iteration# Bytes Already Moved Bytes Left To Move
Bytes Being Moved
12/04/16 16:22:10 INFO net.NetworkTopology: Adding a new node:
/default-rack/yy.yy.yy.yy:50176
12/04/16 16:22:10 INFO net.NetworkTopology: Adding a new node:
/default-rack/xx.xx.xx.xx:50010
12/04/16 16:22:10 INFO balancer.Balancer: 1 over-utilized:
[Source[xx.xx.xx.xx:50010, utilization=7.212458091389678]]
12/04/16 16:22:10 INFO balancer.Balancer: 1 underutilized:
[BalancerDatanode[yy.yy.yy.yy:50176, utilization=4.650670324367203E-5]]
12/04/16 16:22:10 INFO balancer.Balancer: Need to move 1.77 GB to make the
cluster balanced.
No block can be moved. Exiting...
Balancing took 5.142 seconds
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3286) When the threshold value for balancer is 0(zero) ,unexpected output is displayed
[
https://issues.apache.org/jira/browse/HDFS-3286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashish Singhi updated HDFS-3286:
Attachment: HDFS-3286-1.patch
Correct patch HDFS-3286-1
When the threshold value for balancer is 0(zero) ,unexpected output is
displayed
Key: HDFS-3286
URL: https://issues.apache.org/jira/browse/HDFS-3286
Project: Hadoop HDFS
Issue Type: Bug
Components: balancer
Affects Versions: 0.23.0
Reporter: J.Andreina
Assignee: Ashish Singhi
Fix For: 0.24.0
Attachments: HDFS-3286-1.patch, HDFS-3286.patch, HDFS-3286.patch,
HDFS-3286.patch
Replication factor =1
Step 1: Start NN,DN1.write 4 GB of data
Step 2: Start DN2
Step 3: issue the balancer command(./hdfs balancer -threshold 0)
The threshold parameter is a fraction in the range of (0%, 100%) with a
default value of 10%
When the above scenario is executed the Source DN and Target DN is choosen
and the number of bytes to be moved from source to target DN is also
calculated .
Then the balancer is exiting with the following message No block can be
moved. Exiting... which is not expected.
{noformat}
HOST-xx-xx-xx-xx:/home/Andreina/APril10/install/hadoop/namenode/bin # ./hdfs
balancer -threshold 0
12/04/16 16:22:07 INFO balancer.Balancer: Using a threshold of 0.0
12/04/16 16:22:07 INFO balancer.Balancer: namenodes =
[hdfs://HOST-xx-xx-xx-xx:9000]
12/04/16 16:22:07 INFO balancer.Balancer: p =
Balancer.Parameters[BalancingPolicy.Node, threshold=0.0]
Time Stamp Iteration# Bytes Already Moved Bytes Left To Move
Bytes Being Moved
12/04/16 16:22:10 INFO net.NetworkTopology: Adding a new node:
/default-rack/yy.yy.yy.yy:50176
12/04/16 16:22:10 INFO net.NetworkTopology: Adding a new node:
/default-rack/xx.xx.xx.xx:50010
12/04/16 16:22:10 INFO balancer.Balancer: 1 over-utilized:
[Source[xx.xx.xx.xx:50010, utilization=7.212458091389678]]
12/04/16 16:22:10 INFO balancer.Balancer: 1 underutilized:
[BalancerDatanode[yy.yy.yy.yy:50176, utilization=4.650670324367203E-5]]
12/04/16 16:22:10 INFO balancer.Balancer: Need to move 1.77 GB to make the
cluster balanced.
No block can be moved. Exiting...
Balancing took 5.142 seconds
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3286) When the threshold value for balancer is 0(zero) ,unexpected output is displayed
[
https://issues.apache.org/jira/browse/HDFS-3286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashish Singhi updated HDFS-3286:
Target Version/s: 2.0.0, 3.0.0 (was: 3.0.0, 2.0.0)
Status: Patch Available (was: Open)
When the threshold value for balancer is 0(zero) ,unexpected output is
displayed
Key: HDFS-3286
URL: https://issues.apache.org/jira/browse/HDFS-3286
Project: Hadoop HDFS
Issue Type: Bug
Components: balancer
Affects Versions: 0.23.0
Reporter: J.Andreina
Assignee: Ashish Singhi
Fix For: 0.24.0
Attachments: HDFS-3286-1.patch, HDFS-3286.patch, HDFS-3286.patch,
HDFS-3286.patch
Replication factor =1
Step 1: Start NN,DN1.write 4 GB of data
Step 2: Start DN2
Step 3: issue the balancer command(./hdfs balancer -threshold 0)
The threshold parameter is a fraction in the range of (0%, 100%) with a
default value of 10%
When the above scenario is executed the Source DN and Target DN is choosen
and the number of bytes to be moved from source to target DN is also
calculated .
Then the balancer is exiting with the following message No block can be
moved. Exiting... which is not expected.
{noformat}
HOST-xx-xx-xx-xx:/home/Andreina/APril10/install/hadoop/namenode/bin # ./hdfs
balancer -threshold 0
12/04/16 16:22:07 INFO balancer.Balancer: Using a threshold of 0.0
12/04/16 16:22:07 INFO balancer.Balancer: namenodes =
[hdfs://HOST-xx-xx-xx-xx:9000]
12/04/16 16:22:07 INFO balancer.Balancer: p =
Balancer.Parameters[BalancingPolicy.Node, threshold=0.0]
Time Stamp Iteration# Bytes Already Moved Bytes Left To Move
Bytes Being Moved
12/04/16 16:22:10 INFO net.NetworkTopology: Adding a new node:
/default-rack/yy.yy.yy.yy:50176
12/04/16 16:22:10 INFO net.NetworkTopology: Adding a new node:
/default-rack/xx.xx.xx.xx:50010
12/04/16 16:22:10 INFO balancer.Balancer: 1 over-utilized:
[Source[xx.xx.xx.xx:50010, utilization=7.212458091389678]]
12/04/16 16:22:10 INFO balancer.Balancer: 1 underutilized:
[BalancerDatanode[yy.yy.yy.yy:50176, utilization=4.650670324367203E-5]]
12/04/16 16:22:10 INFO balancer.Balancer: Need to move 1.77 GB to make the
cluster balanced.
No block can be moved. Exiting...
Balancing took 5.142 seconds
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2743) Streamline usage of bookkeeper journal manager
[ https://issues.apache.org/jira/browse/HDFS-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264235#comment-13264235 ] Uma Maheswara Rao G commented on HDFS-2743: --- +1 Streamline usage of bookkeeper journal manager -- Key: HDFS-2743 URL: https://issues.apache.org/jira/browse/HDFS-2743 Project: Hadoop HDFS Issue Type: Improvement Reporter: Ivan Kelly Assignee: Ivan Kelly Fix For: 0.24.0 Attachments: HDFS-2743.diff, HDFS-2743.diff The current method of installing bkjournal manager involves generating a tarball, and extracting it with special flags over the hdfs distribution. This is cumbersome and prone to being broken by other changes (see https://svn.apache.org/repos/asf/hadoop/common/trunk@1220940). I think a cleaner way to doing this is to generate a single jar that can be placed in the lib dir of hdfs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3286) When the threshold value for balancer is 0(zero) ,unexpected output is displayed
[
https://issues.apache.org/jira/browse/HDFS-3286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264251#comment-13264251
]
Hadoop QA commented on HDFS-3286:
-
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12524968/HDFS-3286-1.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 1 new or modified test
files.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9)
warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/2349//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2349//console
This message is automatically generated.
When the threshold value for balancer is 0(zero) ,unexpected output is
displayed
Key: HDFS-3286
URL: https://issues.apache.org/jira/browse/HDFS-3286
Project: Hadoop HDFS
Issue Type: Bug
Components: balancer
Affects Versions: 0.23.0
Reporter: J.Andreina
Assignee: Ashish Singhi
Fix For: 0.24.0
Attachments: HDFS-3286-1.patch, HDFS-3286.patch, HDFS-3286.patch,
HDFS-3286.patch
Replication factor =1
Step 1: Start NN,DN1.write 4 GB of data
Step 2: Start DN2
Step 3: issue the balancer command(./hdfs balancer -threshold 0)
The threshold parameter is a fraction in the range of (0%, 100%) with a
default value of 10%
When the above scenario is executed the Source DN and Target DN is choosen
and the number of bytes to be moved from source to target DN is also
calculated .
Then the balancer is exiting with the following message No block can be
moved. Exiting... which is not expected.
{noformat}
HOST-xx-xx-xx-xx:/home/Andreina/APril10/install/hadoop/namenode/bin # ./hdfs
balancer -threshold 0
12/04/16 16:22:07 INFO balancer.Balancer: Using a threshold of 0.0
12/04/16 16:22:07 INFO balancer.Balancer: namenodes =
[hdfs://HOST-xx-xx-xx-xx:9000]
12/04/16 16:22:07 INFO balancer.Balancer: p =
Balancer.Parameters[BalancingPolicy.Node, threshold=0.0]
Time Stamp Iteration# Bytes Already Moved Bytes Left To Move
Bytes Being Moved
12/04/16 16:22:10 INFO net.NetworkTopology: Adding a new node:
/default-rack/yy.yy.yy.yy:50176
12/04/16 16:22:10 INFO net.NetworkTopology: Adding a new node:
/default-rack/xx.xx.xx.xx:50010
12/04/16 16:22:10 INFO balancer.Balancer: 1 over-utilized:
[Source[xx.xx.xx.xx:50010, utilization=7.212458091389678]]
12/04/16 16:22:10 INFO balancer.Balancer: 1 underutilized:
[BalancerDatanode[yy.yy.yy.yy:50176, utilization=4.650670324367203E-5]]
12/04/16 16:22:10 INFO balancer.Balancer: Need to move 1.77 GB to make the
cluster balanced.
No block can be moved. Exiting...
Balancing took 5.142 seconds
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3334) ByteRangeInputStream leaks streams
[
https://issues.apache.org/jira/browse/HDFS-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264308#comment-13264308
]
Hudson commented on HDFS-3334:
--
Integrated in Hadoop-Hdfs-0.23-Build #241 (See
[https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/241/])
svn merge -c 1331570 from trunk for HDFS-3334. Fix ByteRangeInputStream
stream leakage. (Revision 1331573)
Result = SUCCESS
szetszwo :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1331573
Files :
* /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs
*
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java
*
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/ByteRangeInputStream.java
*
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestByteRangeInputStream.java
*
/hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestHftpFileSystem.java
ByteRangeInputStream leaks streams
--
Key: HDFS-3334
URL: https://issues.apache.org/jira/browse/HDFS-3334
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs client
Affects Versions: 0.24.0, 0.23.3, 2.0.0
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Fix For: 0.23.3
Attachments: HDFS-3334-1.patch, HDFS-3334.patch
{{HftpFileSystem.ByteRangeInputStream}} does not implement {{close}} so it
leaks the underlying stream(s).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3331) setBalancerBandwidth do not checkSuperuserPrivilege
[ https://issues.apache.org/jira/browse/HDFS-3331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264309#comment-13264309 ] Hudson commented on HDFS-3331: -- Integrated in Hadoop-Hdfs-0.23-Build #241 (See [https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/241/]) HDFS-3331. In namenode, check superuser privilege for setBalancerBandwidth and acquire the write lock for finalizeUpgrade. (Revision 1331600) Result = SUCCESS szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1331600 Files : * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java setBalancerBandwidth do not checkSuperuserPrivilege --- Key: HDFS-3331 URL: https://issues.apache.org/jira/browse/HDFS-3331 Project: Hadoop HDFS Issue Type: Bug Components: name-node Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.23.3 Attachments: h3331_20120426.patch, h3331_20120427.patch, h3331_20120427_0.23.patch - setBalancerBandwidth should checkSuperuserPrivilege - finalizeUpgrade should acquire the write lock. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3334) ByteRangeInputStream leaks streams
[
https://issues.apache.org/jira/browse/HDFS-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264313#comment-13264313
]
Hudson commented on HDFS-3334:
--
Integrated in Hadoop-Hdfs-trunk #1028 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1028/])
HDFS-3334. Fix ByteRangeInputStream stream leakage. Contributed by Daryn
Sharp (Revision 1331570)
Result = FAILURE
szetszwo :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1331570
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/ByteRangeInputStream.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestByteRangeInputStream.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestHftpFileSystem.java
ByteRangeInputStream leaks streams
--
Key: HDFS-3334
URL: https://issues.apache.org/jira/browse/HDFS-3334
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs client
Affects Versions: 0.24.0, 0.23.3, 2.0.0
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Fix For: 0.23.3
Attachments: HDFS-3334-1.patch, HDFS-3334.patch
{{HftpFileSystem.ByteRangeInputStream}} does not implement {{close}} so it
leaks the underlying stream(s).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3309) HttpFS (Hoop) chmod not supporting octal and sticky bit permissions
[
https://issues.apache.org/jira/browse/HDFS-3309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264312#comment-13264312
]
Hudson commented on HDFS-3309:
--
Integrated in Hadoop-Hdfs-trunk #1028 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1028/])
HDFS-3309. HttpFS (Hoop) chmod not supporting octal and sticky bit
permissions. (tucu) (Revision 1331493)
Result = FAILURE
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1331493
Files :
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/HttpFSParams.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/client/TestHttpFSFileSystem.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
HttpFS (Hoop) chmod not supporting octal and sticky bit permissions
---
Key: HDFS-3309
URL: https://issues.apache.org/jira/browse/HDFS-3309
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Romain Rigaux
Assignee: Alejandro Abdelnur
Fix For: 2.0.0
Attachments: HDFS-3309.patch, HDFS-3309.patch
HttpFs supports only the permissions: [0-7][0-7][0-7]
In order to be compatible with webhdfs in needs to understand octal and
sticky bit permissions (e.g. 0777, 01777...)
Example of error:
curl -L -X PUT
http://localhost:14000/webhdfs/v1/user/romain/test?permission=01777op=SETPERMISSIONuser.name=romain;
{RemoteException:{message:java.lang.IllegalArgumentException: Parameter
[permission], invalid value [01777], value must be
[default|(-[-r][-w][-x][-r][-w][-x][-r][-w][-x])|[0-7][0-7][0-7]],exception:QueryParamException,javaClassName:com.sun.jersey.api.ParamException$QueryParamException}}
Works with WebHdfs:
curl -L -X PUT
http://localhost:50070/webhdfs/v1/user/romain/test?permission=01777op=SETPERMISSIONuser.name=romain;
echo $?
0
curl -L -X PUT
http://localhost:14000/webhdfs/v1/user/romain/test?permission=99op=SETPERMISSIONuser.name=romain;
{RemoteException:{message:java.lang.IllegalArgumentException: Parameter
[permission], invalid value [99], value must be
[default|(-[-r][-w][-x][-r][-w][-x][-r][-w][-x])|[0-7][0-7][0-7]],exception:QueryParamException,javaClassName:com.sun.jersey.api.ParamException$QueryParamException}}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3326) Append enabled log message uses the wrong variable
[
https://issues.apache.org/jira/browse/HDFS-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264316#comment-13264316
]
Hudson commented on HDFS-3326:
--
Integrated in Hadoop-Hdfs-trunk #1028 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1028/])
HDFS-3326. Append enabled log message uses the wrong variable. Contributed
by Matthew Jacobs (Revision 1331626)
Result = FAILURE
eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1331626
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
Append enabled log message uses the wrong variable
--
Key: HDFS-3326
URL: https://issues.apache.org/jira/browse/HDFS-3326
Project: Hadoop HDFS
Issue Type: Bug
Components: name-node
Affects Versions: 2.0.0
Reporter: J.Andreina
Assignee: Matthew Jacobs
Priority: Trivial
Labels: newbie
Fix For: 2.0.0
Attachments: hdfs-3326.txt
dfs.support.append is set to true
started NN in non-HA mode
At the NN side log the append enable is set to false.
This is because in code append enabled is set to HA enabled value.Since
Started NN in non-HA mode the value for append is false
Code:
=
{noformat}
this.supportAppends = conf.getBoolean(DFS_SUPPORT_APPEND_KEY,
DFS_SUPPORT_APPEND_DEFAULT);
LOG.info(Append Enabled: + haEnabled);{noformat}
NN logs
{noformat}
2012-04-25 21:11:09,693 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: HA Enabled: false
2012-04-25 21:11:09,702 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Append Enabled:
false{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3331) setBalancerBandwidth do not checkSuperuserPrivilege
[ https://issues.apache.org/jira/browse/HDFS-3331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264314#comment-13264314 ] Hudson commented on HDFS-3331: -- Integrated in Hadoop-Hdfs-trunk #1028 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1028/]) HDFS-3331. In namenode, check superuser privilege for setBalancerBandwidth and acquire the write lock for finalizeUpgrade. (Revision 1331598) Result = FAILURE szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1331598 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java setBalancerBandwidth do not checkSuperuserPrivilege --- Key: HDFS-3331 URL: https://issues.apache.org/jira/browse/HDFS-3331 Project: Hadoop HDFS Issue Type: Bug Components: name-node Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.23.3 Attachments: h3331_20120426.patch, h3331_20120427.patch, h3331_20120427_0.23.patch - setBalancerBandwidth should checkSuperuserPrivilege - finalizeUpgrade should acquire the write lock. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3334) ByteRangeInputStream leaks streams
[
https://issues.apache.org/jira/browse/HDFS-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264331#comment-13264331
]
Hudson commented on HDFS-3334:
--
Integrated in Hadoop-Mapreduce-trunk #1063 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1063/])
HDFS-3334. Fix ByteRangeInputStream stream leakage. Contributed by Daryn
Sharp (Revision 1331570)
Result = FAILURE
szetszwo :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1331570
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/ByteRangeInputStream.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestByteRangeInputStream.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestHftpFileSystem.java
ByteRangeInputStream leaks streams
--
Key: HDFS-3334
URL: https://issues.apache.org/jira/browse/HDFS-3334
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs client
Affects Versions: 0.24.0, 0.23.3, 2.0.0
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Fix For: 0.23.3
Attachments: HDFS-3334-1.patch, HDFS-3334.patch
{{HftpFileSystem.ByteRangeInputStream}} does not implement {{close}} so it
leaks the underlying stream(s).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3309) HttpFS (Hoop) chmod not supporting octal and sticky bit permissions
[
https://issues.apache.org/jira/browse/HDFS-3309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264330#comment-13264330
]
Hudson commented on HDFS-3309:
--
Integrated in Hadoop-Mapreduce-trunk #1063 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1063/])
HDFS-3309. HttpFS (Hoop) chmod not supporting octal and sticky bit
permissions. (tucu) (Revision 1331493)
Result = FAILURE
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1331493
Files :
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/HttpFSParams.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/client/TestHttpFSFileSystem.java
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
HttpFS (Hoop) chmod not supporting octal and sticky bit permissions
---
Key: HDFS-3309
URL: https://issues.apache.org/jira/browse/HDFS-3309
Project: Hadoop HDFS
Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Romain Rigaux
Assignee: Alejandro Abdelnur
Fix For: 2.0.0
Attachments: HDFS-3309.patch, HDFS-3309.patch
HttpFs supports only the permissions: [0-7][0-7][0-7]
In order to be compatible with webhdfs in needs to understand octal and
sticky bit permissions (e.g. 0777, 01777...)
Example of error:
curl -L -X PUT
http://localhost:14000/webhdfs/v1/user/romain/test?permission=01777op=SETPERMISSIONuser.name=romain;
{RemoteException:{message:java.lang.IllegalArgumentException: Parameter
[permission], invalid value [01777], value must be
[default|(-[-r][-w][-x][-r][-w][-x][-r][-w][-x])|[0-7][0-7][0-7]],exception:QueryParamException,javaClassName:com.sun.jersey.api.ParamException$QueryParamException}}
Works with WebHdfs:
curl -L -X PUT
http://localhost:50070/webhdfs/v1/user/romain/test?permission=01777op=SETPERMISSIONuser.name=romain;
echo $?
0
curl -L -X PUT
http://localhost:14000/webhdfs/v1/user/romain/test?permission=99op=SETPERMISSIONuser.name=romain;
{RemoteException:{message:java.lang.IllegalArgumentException: Parameter
[permission], invalid value [99], value must be
[default|(-[-r][-w][-x][-r][-w][-x][-r][-w][-x])|[0-7][0-7][0-7]],exception:QueryParamException,javaClassName:com.sun.jersey.api.ParamException$QueryParamException}}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3331) setBalancerBandwidth do not checkSuperuserPrivilege
[ https://issues.apache.org/jira/browse/HDFS-3331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264332#comment-13264332 ] Hudson commented on HDFS-3331: -- Integrated in Hadoop-Mapreduce-trunk #1063 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1063/]) HDFS-3331. In namenode, check superuser privilege for setBalancerBandwidth and acquire the write lock for finalizeUpgrade. (Revision 1331598) Result = FAILURE szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1331598 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java setBalancerBandwidth do not checkSuperuserPrivilege --- Key: HDFS-3331 URL: https://issues.apache.org/jira/browse/HDFS-3331 Project: Hadoop HDFS Issue Type: Bug Components: name-node Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Fix For: 0.23.3 Attachments: h3331_20120426.patch, h3331_20120427.patch, h3331_20120427_0.23.patch - setBalancerBandwidth should checkSuperuserPrivilege - finalizeUpgrade should acquire the write lock. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3326) Append enabled log message uses the wrong variable
[
https://issues.apache.org/jira/browse/HDFS-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264334#comment-13264334
]
Hudson commented on HDFS-3326:
--
Integrated in Hadoop-Mapreduce-trunk #1063 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1063/])
HDFS-3326. Append enabled log message uses the wrong variable. Contributed
by Matthew Jacobs (Revision 1331626)
Result = FAILURE
eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1331626
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
Append enabled log message uses the wrong variable
--
Key: HDFS-3326
URL: https://issues.apache.org/jira/browse/HDFS-3326
Project: Hadoop HDFS
Issue Type: Bug
Components: name-node
Affects Versions: 2.0.0
Reporter: J.Andreina
Assignee: Matthew Jacobs
Priority: Trivial
Labels: newbie
Fix For: 2.0.0
Attachments: hdfs-3326.txt
dfs.support.append is set to true
started NN in non-HA mode
At the NN side log the append enable is set to false.
This is because in code append enabled is set to HA enabled value.Since
Started NN in non-HA mode the value for append is false
Code:
=
{noformat}
this.supportAppends = conf.getBoolean(DFS_SUPPORT_APPEND_KEY,
DFS_SUPPORT_APPEND_DEFAULT);
LOG.info(Append Enabled: + haEnabled);{noformat}
NN logs
{noformat}
2012-04-25 21:11:09,693 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: HA Enabled: false
2012-04-25 21:11:09,702 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Append Enabled:
false{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2743) Streamline usage of bookkeeper journal manager
[ https://issues.apache.org/jira/browse/HDFS-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264352#comment-13264352 ] Hudson commented on HDFS-2743: -- Integrated in Hadoop-Hdfs-trunk-Commit #2223 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2223/]) HDFS-2743. Streamline usage of bookkeeper journal manager. Contributed by Ivan Kelly. (Revision 1331790) Result = SUCCESS umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1331790 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/README.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml Streamline usage of bookkeeper journal manager -- Key: HDFS-2743 URL: https://issues.apache.org/jira/browse/HDFS-2743 Project: Hadoop HDFS Issue Type: Improvement Reporter: Ivan Kelly Assignee: Ivan Kelly Fix For: 0.24.0 Attachments: HDFS-2743.diff, HDFS-2743.diff The current method of installing bkjournal manager involves generating a tarball, and extracting it with special flags over the hdfs distribution. This is cumbersome and prone to being broken by other changes (see https://svn.apache.org/repos/asf/hadoop/common/trunk@1220940). I think a cleaner way to doing this is to generate a single jar that can be placed in the lib dir of hdfs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2743) Streamline usage of bookkeeper journal manager
[ https://issues.apache.org/jira/browse/HDFS-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264353#comment-13264353 ] Hudson commented on HDFS-2743: -- Integrated in Hadoop-Common-trunk-Commit #2149 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2149/]) HDFS-2743. Streamline usage of bookkeeper journal manager. Contributed by Ivan Kelly. (Revision 1331790) Result = SUCCESS umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1331790 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/README.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml Streamline usage of bookkeeper journal manager -- Key: HDFS-2743 URL: https://issues.apache.org/jira/browse/HDFS-2743 Project: Hadoop HDFS Issue Type: Improvement Reporter: Ivan Kelly Assignee: Ivan Kelly Fix For: 0.24.0 Attachments: HDFS-2743.diff, HDFS-2743.diff The current method of installing bkjournal manager involves generating a tarball, and extracting it with special flags over the hdfs distribution. This is cumbersome and prone to being broken by other changes (see https://svn.apache.org/repos/asf/hadoop/common/trunk@1220940). I think a cleaner way to doing this is to generate a single jar that can be placed in the lib dir of hdfs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2743) Streamline usage of bookkeeper journal manager
[ https://issues.apache.org/jira/browse/HDFS-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264360#comment-13264360 ] Hudson commented on HDFS-2743: -- Integrated in Hadoop-Mapreduce-trunk-Commit #2166 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2166/]) HDFS-2743. Streamline usage of bookkeeper journal manager. Contributed by Ivan Kelly. (Revision 1331790) Result = ABORTED umamahesh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1331790 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/README.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml Streamline usage of bookkeeper journal manager -- Key: HDFS-2743 URL: https://issues.apache.org/jira/browse/HDFS-2743 Project: Hadoop HDFS Issue Type: Improvement Reporter: Ivan Kelly Assignee: Ivan Kelly Fix For: 0.24.0 Attachments: HDFS-2743.diff, HDFS-2743.diff The current method of installing bkjournal manager involves generating a tarball, and extracting it with special flags over the hdfs distribution. This is cumbersome and prone to being broken by other changes (see https://svn.apache.org/repos/asf/hadoop/common/trunk@1220940). I think a cleaner way to doing this is to generate a single jar that can be placed in the lib dir of hdfs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2743) Streamline usage of bookkeeper journal manager
[ https://issues.apache.org/jira/browse/HDFS-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-2743: -- Resolution: Fixed Target Version/s: 3.0.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I have just committed to trunk! Thanks a lot, Ivan. Streamline usage of bookkeeper journal manager -- Key: HDFS-2743 URL: https://issues.apache.org/jira/browse/HDFS-2743 Project: Hadoop HDFS Issue Type: Improvement Reporter: Ivan Kelly Assignee: Ivan Kelly Fix For: 0.24.0 Attachments: HDFS-2743.diff, HDFS-2743.diff The current method of installing bkjournal manager involves generating a tarball, and extracting it with special flags over the hdfs distribution. This is cumbersome and prone to being broken by other changes (see https://svn.apache.org/repos/asf/hadoop/common/trunk@1220940). I think a cleaner way to doing this is to generate a single jar that can be placed in the lib dir of hdfs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3275) Format command overwrites contents of non-empty shared edits dir if name dirs are empty without any prompting
[ https://issues.apache.org/jira/browse/HDFS-3275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-3275: -- Attachment: HDFS-3275-4.patch Attached the patch same as Amith and corrected the failures. Let's run the Jenkins. Format command overwrites contents of non-empty shared edits dir if name dirs are empty without any prompting - Key: HDFS-3275 URL: https://issues.apache.org/jira/browse/HDFS-3275 Project: Hadoop HDFS Issue Type: Bug Components: ha, name-node Affects Versions: 2.0.0 Reporter: Vinithra Varadharajan Assignee: amith Fix For: 3.0.0 Attachments: HDFS-3275-4.patch, HDFS-3275.patch, HDFS-3275_1.patch, HDFS-3275_1.patch, HDFS-3275_2.patch, HDFS-3275_2.patch, HDFS-3275_3.patch To reproduce: # Configure a NameNode with namedirs and a shared edits dir, all of which are empty. # Run hdfs namenode -format. Namedirs and shared edits dir gets populated. # Delete the contents of the namedirs. Leave the shared edits dir as is. Check the timestamps of the shared edits dir contents. # Run format again. The namedirs as well as the shared edits dir get formatted. The shared edits dir's contents have been replaced without any prompting. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3275) Format command overwrites contents of non-empty shared edits dir if name dirs are empty without any prompting
[ https://issues.apache.org/jira/browse/HDFS-3275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-3275: -- Hadoop Flags: Reviewed Status: Patch Available (was: Open) Format command overwrites contents of non-empty shared edits dir if name dirs are empty without any prompting - Key: HDFS-3275 URL: https://issues.apache.org/jira/browse/HDFS-3275 Project: Hadoop HDFS Issue Type: Bug Components: ha, name-node Affects Versions: 2.0.0 Reporter: Vinithra Varadharajan Assignee: amith Fix For: 3.0.0 Attachments: HDFS-3275-4.patch, HDFS-3275.patch, HDFS-3275_1.patch, HDFS-3275_1.patch, HDFS-3275_2.patch, HDFS-3275_2.patch, HDFS-3275_3.patch To reproduce: # Configure a NameNode with namedirs and a shared edits dir, all of which are empty. # Run hdfs namenode -format. Namedirs and shared edits dir gets populated. # Delete the contents of the namedirs. Leave the shared edits dir as is. Check the timestamps of the shared edits dir contents. # Run format again. The namedirs as well as the shared edits dir get formatted. The shared edits dir's contents have been replaced without any prompting. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3286) When the threshold value for balancer is 0(zero) ,unexpected output is displayed
[
https://issues.apache.org/jira/browse/HDFS-3286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264381#comment-13264381
]
Uma Maheswara Rao G commented on HDFS-3286:
---
+1, looks good.
When the threshold value for balancer is 0(zero) ,unexpected output is
displayed
Key: HDFS-3286
URL: https://issues.apache.org/jira/browse/HDFS-3286
Project: Hadoop HDFS
Issue Type: Bug
Components: balancer
Affects Versions: 0.23.0
Reporter: J.Andreina
Assignee: Ashish Singhi
Fix For: 0.24.0
Attachments: HDFS-3286-1.patch, HDFS-3286.patch, HDFS-3286.patch,
HDFS-3286.patch
Replication factor =1
Step 1: Start NN,DN1.write 4 GB of data
Step 2: Start DN2
Step 3: issue the balancer command(./hdfs balancer -threshold 0)
The threshold parameter is a fraction in the range of (0%, 100%) with a
default value of 10%
When the above scenario is executed the Source DN and Target DN is choosen
and the number of bytes to be moved from source to target DN is also
calculated .
Then the balancer is exiting with the following message No block can be
moved. Exiting... which is not expected.
{noformat}
HOST-xx-xx-xx-xx:/home/Andreina/APril10/install/hadoop/namenode/bin # ./hdfs
balancer -threshold 0
12/04/16 16:22:07 INFO balancer.Balancer: Using a threshold of 0.0
12/04/16 16:22:07 INFO balancer.Balancer: namenodes =
[hdfs://HOST-xx-xx-xx-xx:9000]
12/04/16 16:22:07 INFO balancer.Balancer: p =
Balancer.Parameters[BalancingPolicy.Node, threshold=0.0]
Time Stamp Iteration# Bytes Already Moved Bytes Left To Move
Bytes Being Moved
12/04/16 16:22:10 INFO net.NetworkTopology: Adding a new node:
/default-rack/yy.yy.yy.yy:50176
12/04/16 16:22:10 INFO net.NetworkTopology: Adding a new node:
/default-rack/xx.xx.xx.xx:50010
12/04/16 16:22:10 INFO balancer.Balancer: 1 over-utilized:
[Source[xx.xx.xx.xx:50010, utilization=7.212458091389678]]
12/04/16 16:22:10 INFO balancer.Balancer: 1 underutilized:
[BalancerDatanode[yy.yy.yy.yy:50176, utilization=4.650670324367203E-5]]
12/04/16 16:22:10 INFO balancer.Balancer: Need to move 1.77 GB to make the
cluster balanced.
No block can be moved. Exiting...
Balancing took 5.142 seconds
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3275) Format command overwrites contents of non-empty shared edits dir if name dirs are empty without any prompting
[ https://issues.apache.org/jira/browse/HDFS-3275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264384#comment-13264384 ] Hadoop QA commented on HDFS-3275: - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12524985/HDFS-3275-4.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified test files. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2350//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2350//console This message is automatically generated. Format command overwrites contents of non-empty shared edits dir if name dirs are empty without any prompting - Key: HDFS-3275 URL: https://issues.apache.org/jira/browse/HDFS-3275 Project: Hadoop HDFS Issue Type: Bug Components: ha, name-node Affects Versions: 2.0.0 Reporter: Vinithra Varadharajan Assignee: amith Fix For: 3.0.0 Attachments: HDFS-3275-4.patch, HDFS-3275.patch, HDFS-3275_1.patch, HDFS-3275_1.patch, HDFS-3275_2.patch, HDFS-3275_2.patch, HDFS-3275_3.patch To reproduce: # Configure a NameNode with namedirs and a shared edits dir, all of which are empty. # Run hdfs namenode -format. Namedirs and shared edits dir gets populated. # Delete the contents of the namedirs. Leave the shared edits dir as is. Check the timestamps of the shared edits dir contents. # Run format again. The namedirs as well as the shared edits dir get formatted. The shared edits dir's contents have been replaced without any prompting. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3275) Format command overwrites contents of non-empty shared edits dir if name dirs are empty without any prompting
[ https://issues.apache.org/jira/browse/HDFS-3275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13264454#comment-13264454 ] Uma Maheswara Rao G commented on HDFS-3275: --- +1, Looks good, I will commit this patch. Format command overwrites contents of non-empty shared edits dir if name dirs are empty without any prompting - Key: HDFS-3275 URL: https://issues.apache.org/jira/browse/HDFS-3275 Project: Hadoop HDFS Issue Type: Bug Components: ha, name-node Affects Versions: 2.0.0 Reporter: Vinithra Varadharajan Assignee: amith Fix For: 3.0.0 Attachments: HDFS-3275-4.patch, HDFS-3275.patch, HDFS-3275_1.patch, HDFS-3275_1.patch, HDFS-3275_2.patch, HDFS-3275_2.patch, HDFS-3275_3.patch To reproduce: # Configure a NameNode with namedirs and a shared edits dir, all of which are empty. # Run hdfs namenode -format. Namedirs and shared edits dir gets populated. # Delete the contents of the namedirs. Leave the shared edits dir as is. Check the timestamps of the shared edits dir contents. # Run format again. The namedirs as well as the shared edits dir get formatted. The shared edits dir's contents have been replaced without any prompting. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
