[jira] [Commented] (HDFS-4443) Remove trailing '`' character from HDFS nodelist jsp
[ https://issues.apache.org/jira/browse/HDFS-4443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13563736#comment-13563736 ] Christian Rohling commented on HDFS-4443: - Thank you guys for moving this patch along. > Remove trailing '`' character from HDFS nodelist jsp > > > Key: HDFS-4443 > URL: https://issues.apache.org/jira/browse/HDFS-4443 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Reporter: Christian Rohling >Assignee: Christian Rohling >Priority: Trivial > Fix For: 2.0.3-alpha > > Attachments: HDFS-4443.txt > > > Removing trailing '`' character from HDFS nodelist jsp page. The character is > attached to the "pcremaining" class attribute in the td element. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4442) Initialization failed for block (...) Invalid volume failure config value: 1
[
https://issues.apache.org/jira/browse/HDFS-4442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13563637#comment-13563637
]
Steve Loughran commented on HDFS-4442:
--
see also http://wiki.apache.org/hadoop/InvalidJiraIssues
> Initialization failed for block (...) Invalid volume failure config value: 1
> -
>
> Key: HDFS-4442
> URL: https://issues.apache.org/jira/browse/HDFS-4442
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
> Environment: Amazon Linux (Centos 6), Cloudera nightly RPMs
>Reporter: Alexandre Fouché
> Labels: datanode, hdfs
>
> (Note: Some of the message are similar to HDFS-4201)
> Just after i created a new HDFS cluster, and this time using Cloudera nightly
> RPM hadoop-hdfs-datanode-2.0.0+898-1.cdh4.2.0.p0.939.el6.x86_64, HDFS
> datanodes were unable to initialize or store anything. It stays alive, but
> keeps logging exceptions every few seconds.
> It was "Initialization failed for block pool Block pool (...)"
> "org.apache.hadoop.util.DiskChecker$DiskErrorException: Invalid volume
> failure config value: 1" and then repeatedly "Exception in BPOfferService
> for Block pool (...)"
> My config was :
>
> dfs.datanode.data.dir
> file:///opt/hadoop/dn1/data
>
> After a bit of tweaking, it worked once i added a second EBS volume to the
> node. Yet it does not explain the initial error. A bug ?
>
> dfs.datanode.data.dir
> file:///opt/hadoop/dn1/data,file:///opt/hadoop/dn2/data
>
> Original exceptions:
> {code}
> (...)
> 2013-01-25 15:04:28,573 INFO org.apache.hadoop.hdfs.server.common.Storage:
> Formatting block pool BP-1342054845-10.118.50.25-1359125000145 directory
> /opt/hadoop/dn1/data/current/BP-1342054845-10.118.50.25-1359125000145/current
> 2013-01-25 15:04:28,581 INFO org.apache.hadoop.hdfs.server.datanode.DataNode:
> Setting up storage:
> nsid=1786416716;bpid=BP-1342054845-10.118.50.25-1359125000145;lv=-40;nsInfo=lv=-40;cid=CID-3c2cfe5f-da56-4115-90db-81e06c14bc50;nsid=1786416716;c=0;bpid=BP-1342054845-10.118.50.25-1359125000145
> 2013-01-25 15:04:28,601 INFO org.apache.hadoop.hdfs.server.datanode.DataNode:
> Block pool BP-1342054845-10.118.50.25-1359125000145 (storage id
> DS-404982471-10.194.189.193-50010-1359126268221) service to
> namenode2.somedomain.com/10.2.118.169:8020 beginning handshake with NN
> 2013-01-25 15:04:28,605 FATAL
> org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for
> block pool Block pool BP-1342054845-10.118.50.25-1359125000145 (storage id
> DS-404982471-10.194.189.193-50010-1359126268221) service to
> namenode1.somedomain.com/10.118.50.25:8020
> org.apache.hadoop.util.DiskChecker$DiskErrorException: Invalid volume failure
> config value: 1
> at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.(FsDatasetImpl.java:182)
> at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetFactory.newInstance(FsDatasetFactory.java:34)
> at
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetFactory.newInstance(FsDatasetFactory.java:30)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:910)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:872)
> at
> org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:308)
> at
> org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:218)
> at
> org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:660)
> at java.lang.Thread.run(Unknown Source)
> 2013-01-25 15:04:28,702 INFO org.apache.hadoop.hdfs.server.datanode.DataNode:
> Block pool Block pool BP-1342054845-10.118.50.25-1359125000145 (storage id
> DS-404982471-10.194.189.193-50010-1359126268221) service to
> namenode2.somedomain.com/10.2.118.169:8020 successfully registered with NN
> 2013-01-25 15:04:28,863 INFO org.apache.hadoop.hdfs.server.datanode.DataNode:
> For namenode namenode2.somedomain.com/10.2.118.169:8020 using
> DELETEREPORT_INTERVAL of 30 msec BLOCKREPORT_INTERVAL of 2160msec
> Initial delay: 0msec; heartBeatInterval=3000
> 2013-01-25 15:04:28,864 ERROR
> org.apache.hadoop.hdfs.server.datanode.DataNode: Exception in BPOfferService
> for Block pool BP-1342054845-10.118.50.25-1359125000145 (storage id
> DS-404982471-10.194.189.193-50010-1359126268221) service to
> namenode2.somedomain.com/10.2.118.169:8020
> java.lang.NullPointerException
> at
> org.apache.hadoop.hdfs.server.datanode.BPServiceActor.sendHeartBeat(BPServiceActor.java:435)
> at
> org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:521)
> at
> org.a
[jira] [Commented] (HDFS-4261) TestBalancerWithNodeGroup times out
[ https://issues.apache.org/jira/browse/HDFS-4261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13563569#comment-13563569 ] Junping Du commented on HDFS-4261: -- Suresh, I am good to get this in branch-1 and we can figure out very occasional failure later. > TestBalancerWithNodeGroup times out > --- > > Key: HDFS-4261 > URL: https://issues.apache.org/jira/browse/HDFS-4261 > Project: Hadoop HDFS > Issue Type: Bug > Components: balancer >Affects Versions: 1.0.4, 1.1.1, 2.0.2-alpha >Reporter: Tsz Wo (Nicholas), SZE >Assignee: Junping Du > Fix For: 3.0.0 > > Attachments: HDFS-4261-branch-1.patch, HDFS-4261-branch-1-v2.patch, > HDFS-4261.patch, HDFS-4261-v2.patch, HDFS-4261-v3.patch, HDFS-4261-v4.patch, > HDFS-4261-v5.patch, HDFS-4261-v6.patch, HDFS-4261-v7.patch, > HDFS-4261-v8.patch, jstack-mac-18567, jstack-win-5488, > org.apache.hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup-output.txt.mac, > > org.apache.hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup-output.txt.win, > test-balancer-with-node-group-timeout.txt > > > When I manually ran TestBalancerWithNodeGroup, it always timed out in my > machine. Looking at the Jerkins report [build > #3573|https://builds.apache.org/job/PreCommit-HDFS-Build/3573//testReport/org.apache.hadoop.hdfs.server.balancer/], > TestBalancerWithNodeGroup somehow was skipped so that the problem was not > detected. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4445) All BKJM ledgers are not checked while tailing, So failover will fail.
[
https://issues.apache.org/jira/browse/HDFS-4445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Uma Maheswara Rao G updated HDFS-4445:
--
Issue Type: Sub-task (was: Bug)
Parent: HDFS-3399
> All BKJM ledgers are not checked while tailing, So failover will fail.
> --
>
> Key: HDFS-4445
> URL: https://issues.apache.org/jira/browse/HDFS-4445
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Affects Versions: 2.0.3-alpha
>Reporter: Vinay
>Assignee: Vinay
>Priority: Blocker
>
> After the fix of HDFS-4130, all editlog nodes are not iterated if first edit
> are below fromTxId
> Problem part is below code inside
> BookKeeperJournalManager#selectInputStreams(..)
> {code}if (fromTxId >= l.getFirstTxId() && fromTxId <= lastTxId) {
> LedgerHandle h;
> if (l.isInProgress()) { // we don't want to fence the current
> journal
> h = bkc.openLedgerNoRecovery(l.getLedgerId(), digestType,
> digestpw.getBytes());
> } else {
> h = bkc
> .openLedger(l.getLedgerId(), digestType, digestpw.getBytes());
> }
> elis = new BookKeeperEditLogInputStream(h, l);
> elis.skipTo(fromTxId);
> } else {
> return;
> }{code}
> The else block should have continue statement instead of return.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4441) Refactor INodeDirectoryWithSnapshot.Diff as a top level generic class
[ https://issues.apache.org/jira/browse/HDFS-4441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13563489#comment-13563489 ] Hudson commented on HDFS-4441: -- Integrated in Hadoop-Hdfs-Snapshots-Branch-build #82 (See [https://builds.apache.org/job/Hadoop-Hdfs-Snapshots-Branch-build/82/]) HDFS-4441. Move INodeDirectoryWithSnapshot.Diff and the related classes to a package. (Revision 1438782) Result = FAILURE szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1438782 Files : * /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/CHANGES.HDFS-2802.txt * /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java * /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java * /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INode.java * /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeDirectory.java * /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java * /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeSymlink.java * /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeDirectoryWithSnapshot.java * /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileUnderConstructionWithSnapshot.java * /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/SnapshotFSImageFormat.java * /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/diff * /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/diff/Diff.java * /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestINodeDirectoryWithSnapshot.java * /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestINodeFileUnderConstructionWithSnapshot.java * /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestNestedSnapshots.java * /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/diff * /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/diff/TestDiff.java > Refactor INodeDirectoryWithSnapshot.Diff as a top level generic class > - > > Key: HDFS-4441 > URL: https://issues.apache.org/jira/browse/HDFS-4441 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: namenode >Reporter: Tsz Wo (Nicholas), SZE >Assignee: Tsz Wo (Nicholas), SZE > Fix For: Snapshot (HDFS-2802) > > Attachments: h4441_20130124.patch, h4441_20130125.patch > > > INodeDirectoryWithSnapshot.Diff is for recording the difference between the > current state and a previous state of the children list of an INodeDirectory. > It is easy to refactor the code to support generic lists. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4443) Remove trailing '`' character from HDFS nodelist jsp
[ https://issues.apache.org/jira/browse/HDFS-4443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13563487#comment-13563487 ] Hudson commented on HDFS-4443: -- Integrated in Hadoop-Mapreduce-trunk #1325 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1325/]) HDFS-4443. Remove a trailing '`' character from the HTML code generated by NamenodeJspHelper.generateNodeData(..). Contributed by Christian Rohling (Revision 1438808) Result = FAILURE szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1438808 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeJspHelper.java > Remove trailing '`' character from HDFS nodelist jsp > > > Key: HDFS-4443 > URL: https://issues.apache.org/jira/browse/HDFS-4443 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Reporter: Christian Rohling >Assignee: Christian Rohling >Priority: Trivial > Fix For: 2.0.3-alpha > > Attachments: HDFS-4443.txt > > > Removing trailing '`' character from HDFS nodelist jsp page. The character is > attached to the "pcremaining" class attribute in the td element. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4443) Remove trailing '`' character from HDFS nodelist jsp
[ https://issues.apache.org/jira/browse/HDFS-4443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13563481#comment-13563481 ] Hudson commented on HDFS-4443: -- Integrated in Hadoop-Hdfs-trunk #1297 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1297/]) HDFS-4443. Remove a trailing '`' character from the HTML code generated by NamenodeJspHelper.generateNodeData(..). Contributed by Christian Rohling (Revision 1438808) Result = FAILURE szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1438808 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeJspHelper.java > Remove trailing '`' character from HDFS nodelist jsp > > > Key: HDFS-4443 > URL: https://issues.apache.org/jira/browse/HDFS-4443 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Reporter: Christian Rohling >Assignee: Christian Rohling >Priority: Trivial > Fix For: 2.0.3-alpha > > Attachments: HDFS-4443.txt > > > Removing trailing '`' character from HDFS nodelist jsp page. The character is > attached to the "pcremaining" class attribute in the td element. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-4442) Initialization failed for block (...) Invalid volume failure config value: 1
[
https://issues.apache.org/jira/browse/HDFS-4442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Harsh J updated HDFS-4442:
--
Description:
(Note: Some of the message are similar to HDFS-4201)
Just after i created a new HDFS cluster, and this time using Cloudera nightly
RPM hadoop-hdfs-datanode-2.0.0+898-1.cdh4.2.0.p0.939.el6.x86_64, HDFS datanodes
were unable to initialize or store anything. It stays alive, but keeps logging
exceptions every few seconds.
It was "Initialization failed for block pool Block pool (...)"
"org.apache.hadoop.util.DiskChecker$DiskErrorException: Invalid volume failure
config value: 1" and then repeatedly "Exception in BPOfferService for Block
pool (...)"
My config was :
dfs.datanode.data.dir
file:///opt/hadoop/dn1/data
After a bit of tweaking, it worked once i added a second EBS volume to the
node. Yet it does not explain the initial error. A bug ?
dfs.datanode.data.dir
file:///opt/hadoop/dn1/data,file:///opt/hadoop/dn2/data
Original exceptions:
{code}
(...)
2013-01-25 15:04:28,573 INFO org.apache.hadoop.hdfs.server.common.Storage:
Formatting block pool BP-1342054845-10.118.50.25-1359125000145 directory
/opt/hadoop/dn1/data/current/BP-1342054845-10.118.50.25-1359125000145/current
2013-01-25 15:04:28,581 INFO org.apache.hadoop.hdfs.server.datanode.DataNode:
Setting up storage:
nsid=1786416716;bpid=BP-1342054845-10.118.50.25-1359125000145;lv=-40;nsInfo=lv=-40;cid=CID-3c2cfe5f-da56-4115-90db-81e06c14bc50;nsid=1786416716;c=0;bpid=BP-1342054845-10.118.50.25-1359125000145
2013-01-25 15:04:28,601 INFO org.apache.hadoop.hdfs.server.datanode.DataNode:
Block pool BP-1342054845-10.118.50.25-1359125000145 (storage id
DS-404982471-10.194.189.193-50010-1359126268221) service to
namenode2.somedomain.com/10.2.118.169:8020 beginning handshake with NN
2013-01-25 15:04:28,605 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode:
Initialization failed for block pool Block pool
BP-1342054845-10.118.50.25-1359125000145 (storage id
DS-404982471-10.194.189.193-50010-1359126268221) service to
namenode1.somedomain.com/10.118.50.25:8020
org.apache.hadoop.util.DiskChecker$DiskErrorException: Invalid volume failure
config value: 1
at
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.(FsDatasetImpl.java:182)
at
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetFactory.newInstance(FsDatasetFactory.java:34)
at
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetFactory.newInstance(FsDatasetFactory.java:30)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:910)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:872)
at
org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:308)
at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:218)
at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:660)
at java.lang.Thread.run(Unknown Source)
2013-01-25 15:04:28,702 INFO org.apache.hadoop.hdfs.server.datanode.DataNode:
Block pool Block pool BP-1342054845-10.118.50.25-1359125000145 (storage id
DS-404982471-10.194.189.193-50010-1359126268221) service to
namenode2.somedomain.com/10.2.118.169:8020 successfully registered with NN
2013-01-25 15:04:28,863 INFO org.apache.hadoop.hdfs.server.datanode.DataNode:
For namenode namenode2.somedomain.com/10.2.118.169:8020 using
DELETEREPORT_INTERVAL of 30 msec BLOCKREPORT_INTERVAL of 2160msec
Initial delay: 0msec; heartBeatInterval=3000
2013-01-25 15:04:28,864 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode:
Exception in BPOfferService for Block pool
BP-1342054845-10.118.50.25-1359125000145 (storage id
DS-404982471-10.194.189.193-50010-1359126268221) service to
namenode2.somedomain.com/10.2.118.169:8020
java.lang.NullPointerException
at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.sendHeartBeat(BPServiceActor.java:435)
at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:521)
at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:673)
at java.lang.Thread.run(Unknown Source)
2013-01-25 15:04:28,864 WARN org.apache.hadoop.hdfs.server.datanode.DataNode:
Ending block pool service for: Block pool
BP-1342054845-10.118.50.25-1359125000145 (storage id
DS-404982471-10.194.189.193-50010-1359126268221) service to
namenode1.somedomain.com/10.118.50.25:8020
2013-01-25 15:04:33,864 INFO org.apache.hadoop.hdfs.server.datanode.DataNode:
For namenode namenode2.somedomain.com/10.2.118.169:8020 using
DELETEREPORT_INTERVAL of 30 msec BLOCKREPORT_INTERVAL of 2160msec
Initial delay: 0msec; heartBeatInterval=3000
2013-01-25 15:04:33,864 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode:
Exception in BPOfferService for Block
[jira] [Commented] (HDFS-4443) Remove trailing '`' character from HDFS nodelist jsp
[ https://issues.apache.org/jira/browse/HDFS-4443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13563437#comment-13563437 ] Hudson commented on HDFS-4443: -- Integrated in Hadoop-Yarn-trunk #108 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/108/]) HDFS-4443. Remove a trailing '`' character from the HTML code generated by NamenodeJspHelper.generateNodeData(..). Contributed by Christian Rohling (Revision 1438808) Result = FAILURE szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1438808 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeJspHelper.java > Remove trailing '`' character from HDFS nodelist jsp > > > Key: HDFS-4443 > URL: https://issues.apache.org/jira/browse/HDFS-4443 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Reporter: Christian Rohling >Assignee: Christian Rohling >Priority: Trivial > Fix For: 2.0.3-alpha > > Attachments: HDFS-4443.txt > > > Removing trailing '`' character from HDFS nodelist jsp page. The character is > attached to the "pcremaining" class attribute in the td element. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4444) Add space between total transaction time and number of transactions in FSEditLog#printStatistics
[
https://issues.apache.org/jira/browse/HDFS-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13563423#comment-13563423
]
Stephen Chu commented on HDFS-:
---
No new unit tests are necessary to cover this case (just an extra whitespace
character in the log).
I verified this manually.
> Add space between total transaction time and number of transactions in
> FSEditLog#printStatistics
>
>
> Key: HDFS-
> URL: https://issues.apache.org/jira/browse/HDFS-
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha
>Reporter: Stephen Chu
>Assignee: Stephen Chu
>Priority: Trivial
> Attachments: HDFS-.patch.001
>
>
> Currently, when we log statistics, we see something like
> {code}
> 13/01/25 23:16:59 INFO namenode.FSNamesystem: Number of transactions: 0 Total
> time for transactions(ms): 0Number of transactions batched in Syncs: 0 Number
> of syncs: 0 SyncTimes(ms): 0
> {code}
> Notice how the value for total transactions time and "Number of transactions
> batched in Syncs" needs a space to separate them.
> FSEditLog#printStatistics:
> {code}
> private void printStatistics(boolean force) {
> long now = now();
> if (lastPrintTime + 6 > now && !force) {
> return;
> }
> lastPrintTime = now;
> StringBuilder buf = new StringBuilder();
> buf.append("Number of transactions: ");
> buf.append(numTransactions);
> buf.append(" Total time for transactions(ms): ");
> buf.append(totalTimeTransactions);
> buf.append("Number of transactions batched in Syncs: ");
> buf.append(numTransactionsBatchedInSync);
> buf.append(" Number of syncs: ");
> buf.append(editLogStream.getNumSync());
> buf.append(" SyncTimes(ms): ");
> buf.append(journalSet.getSyncTimes());
> LOG.info(buf);
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4444) Add space between total transaction time and number of transactions in FSEditLog#printStatistics
[
https://issues.apache.org/jira/browse/HDFS-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13563409#comment-13563409
]
Hadoop QA commented on HDFS-:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12566620/HDFS-.patch.001
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/3886//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3886//console
This message is automatically generated.
> Add space between total transaction time and number of transactions in
> FSEditLog#printStatistics
>
>
> Key: HDFS-
> URL: https://issues.apache.org/jira/browse/HDFS-
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha
>Reporter: Stephen Chu
>Assignee: Stephen Chu
>Priority: Trivial
> Attachments: HDFS-.patch.001
>
>
> Currently, when we log statistics, we see something like
> {code}
> 13/01/25 23:16:59 INFO namenode.FSNamesystem: Number of transactions: 0 Total
> time for transactions(ms): 0Number of transactions batched in Syncs: 0 Number
> of syncs: 0 SyncTimes(ms): 0
> {code}
> Notice how the value for total transactions time and "Number of transactions
> batched in Syncs" needs a space to separate them.
> FSEditLog#printStatistics:
> {code}
> private void printStatistics(boolean force) {
> long now = now();
> if (lastPrintTime + 6 > now && !force) {
> return;
> }
> lastPrintTime = now;
> StringBuilder buf = new StringBuilder();
> buf.append("Number of transactions: ");
> buf.append(numTransactions);
> buf.append(" Total time for transactions(ms): ");
> buf.append(totalTimeTransactions);
> buf.append("Number of transactions batched in Syncs: ");
> buf.append(numTransactionsBatchedInSync);
> buf.append(" Number of syncs: ");
> buf.append(editLogStream.getNumSync());
> buf.append(" SyncTimes(ms): ");
> buf.append(journalSet.getSyncTimes());
> LOG.info(buf);
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-4428) FsDatasetImpl should disclose what the error is when a rename fails
[
https://issues.apache.org/jira/browse/HDFS-4428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13563401#comment-13563401
]
Chris Nauroth commented on HDFS-4428:
-
Hi Colin,
This looks like it will be very helpful. I applied the patch locally on an
Ubuntu VM and ran {{TestNativeIO}} successfully. Here are a few questions:
1. It appears that the rename function can also fail with ENOTEMPTY. Can we
add that to the errno mappings?
2. In {{NativeIO.c}}:
{code}
if (rename(src, dst) < 0) {
throw_ioe(env, errno);
}
{code}
I've seen references for rename that say "On failure, a nonzero value is
returned" rather than specifically guaranteeing a negative number gets
returned. Maybe it's safest to change this to check for != 0?
3. In {{TestNativeIO}}:
{code}
// Test attempting to rename a nonexistent file.
try {
NativeIO.renameTo(nonExistentFile, targetFile);
} catch (NativeIOException e) {
Assume.assumeThat(e.getErrno(), equalTo(Errno.ENOENT));
}
{code}
I think all of the assume calls were supposed to be assert calls. With assume,
if the condition is not true, then JUnit marks the test skipped instead of
failed, and it does not cause a build failure. For example, in the above test,
I hacked my local copy to look for the wrong errno: EIO instead of ENOENT.
JUnit reported a skipped test, but not a failure.
Also, the test would pass even if the {{NativeIOException}} was not thrown.
We'll need to add fail calls in the try blocks after the rename calls.
4. Also in {{TestNativeIO}}:
{code}
File sourceFile = new File(TEST_DIR, "source");
FileOutputStream fos = new FileOutputStream(sourceFile);
fos.write(101);
fos.close();
{code}
It appears that the contents of the files aren't important for the test, so
this could simplify to {{File#createNewFile}} instead of a {{FileOutputStream}}.
5. In {{FsDatasetImpl}}:
{code}
try {
NativeIO.renameTo(blkfile, newBlkFile);
} catch (IOException e) {
if (!newmeta.renameTo(oldmeta)) { // restore the meta file
LOG.warn("Cannot move meta file " + newmeta +
"back to the finalized directory " + oldmeta);
}
throw new IOException("Block " + replicaInfo + " reopen failed. " +
" Unable to move block file " + blkfile +
" to rbw dir " + newBlkFile);
" to rbw dir " + newBlkFile, e);
}
{code}
Do you want to use the native call for restoring the meta file too? Then, we
could log the exception and see the errno from that call.
> FsDatasetImpl should disclose what the error is when a rename fails
> ---
>
> Key: HDFS-4428
> URL: https://issues.apache.org/jira/browse/HDFS-4428
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.0.3-alpha
>Reporter: Colin Patrick McCabe
>Assignee: Colin Patrick McCabe
>Priority: Minor
> Attachments: HDFS-4428.001.patch
>
>
> It would be nice if {{FsDatasetImpl}} would print out an error message when a
> rename fails, describing what went wrong. This would make it a lot easier to
> investigate and resolve test failures like HDFS-4051.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
