[jira] [Commented] (HDFS-6658) Namenode memory optimization - Block replicas list
[ https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075574#comment-14075574 ] Amir Langer commented on HDFS-6658: --- Re [~drankye]: Using on or off heap memory is an orthogonal issue to actually reducing it. Its probably better to keep looking at it in [HDFS-6709|https://issues.apache.org/jira/browse/HDFS-6709] but stick here to discussion around reduction of memory. Namenode memory optimization - Block replicas list --- Key: HDFS-6658 URL: https://issues.apache.org/jira/browse/HDFS-6658 Project: Hadoop HDFS Issue Type: Improvement Components: namenode Affects Versions: 2.4.1 Reporter: Amir Langer Assignee: Amir Langer Attachments: Namenode Memory Optimizations - Block replicas list.docx Part of the memory consumed by every BlockInfo object in the Namenode is a linked list of block references for every DatanodeStorageInfo (called triplets). We propose to change the way we store the list in memory. Using primitive integer indexes instead of object references will reduce the memory needed for every block replica (when compressed oops is disabled) and in our new design the list overhead will be per DatanodeStorageInfo and not per block replica. see attached design doc. for details and evaluation results. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6742) Support sorting datanode list on the new NN webUI
[ https://issues.apache.org/jira/browse/HDFS-6742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075779#comment-14075779 ] Ravi Prakash commented on HDFS-6742: IMHO, we should enable the filters on Datatables rather than just sorting capability. Support sorting datanode list on the new NN webUI - Key: HDFS-6742 URL: https://issues.apache.org/jira/browse/HDFS-6742 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Ming Ma Assignee: Chen He The legacy webUI allows sorting datanode list based on specific column such as hostname. It is handy for admins can find pattern more quickly, especially for big clusters. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HDFS-6509) create a /.reserved/raw filesystem namespace
[ https://issues.apache.org/jira/browse/HDFS-6509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charles Lamb updated HDFS-6509: --- Attachment: HDFS-6509.002.patch [~andrew.wang], Thanks for your review. I've attached a .002 version which I believe addresses your comments. FSN and FSDir: bq. You can just do isReservedRawName(src) in place rather than declaring a new variable if it's not being reused. Very cheap function too. But since resolvePathname changes src to remove /.reserved/raw, I needed to call iiRRN prior to the resolvePathname call. It's a moot point now that I've fixed the logAudit calls to log the /.reserved/raw pathnames. bq. Lines longer than 80 chars I fixed the one in constructRemainingPath (I left that in the first diffs since the diffs didn't show that particular line as a change). Were there others besides that one? The comments for resolvePath ran right up to 80, but I don't think they were over 80. bq. Need p/ to line break in a javadoc comment. Fixed. bq. I shy away from the term namespace for raw, can we just say directory? It is just a directory after all. Fixed. bq. FSDirectory, not a fan of even mildly complicated ternary statements. Can we turn these into ifs, or at least not inline them into method calls? Fixed. There was only one inside a method call. I moved it outside. The other ternary statements didn't look too complicated so I didn't change them, but if there still some that you don't like, let me know. bq. Calling up to FSNamesystem from FSDirectory is a no-no, since we want to separate FSD and FSN locks and promote a lock hierarchy. Can we do the superuser check in FSNamesystem? I've turned all of the FSD.resolvePath() calls in FSN into a new FSN#resolvePath. This looked better than adding a call to to checkAccessForReservedRaw(src) before every call to FSD#resolvePath. bq. Stripping the prefix off is scary because we lose context. At the very least, we should log the original path in the audit log. Various log messages will be more confusing. I think using the stripped path is okay in the edit log. Agree. I've made all of the relevant logAuditEvent calls use the pre-resolvePathname path rather than the non-/.reserved/raw pathname. bq. Some javadocs needs updating for the new isRawPath parameter. I added @param isRawPath to FSD#getFileInfo. I think that was the only one. Test: bq. Helper methods look duplicated from TestEncryptionZones. Let's extract the common code somewhere like DFSTestUtil or GenericTestUtils. Done. bq. Why is the special casing necessary in TestDFSUtil for mkdirs? If mkdirs behaves incorrectly on a raw path, then we should fix mkdirs. A normal user could call mkdirs on a raw path after all. Artifact. Removed. bq. Can we name the test TestReservedRawPaths? Done. bq. The .. tests, let's do some tests where the .. is in between .reserved and raw. Done. bq. There's a dfsadmin defined for the class, why create a new one in each test? The dfsAdmin in testRelativePathnames also isn't used. Fixed. bq. Why can't we list /.reserved? Seems like it should work. That feels a bit ugly. While I agree that it would be nice to be able to list /.reserved and have it show raw and .inodes, we'd have to create and return a DirectoryListing with a pair of HdfsFileStatus instances, one for raw, one for .inodes. There would be a bunch of special case code. If you feel strongly about this, then I'd rather do this work in a followon Jira. bq. Should also assert the entry that we get back is the entry we expect, not just the length. Fixed. create a /.reserved/raw filesystem namespace Key: HDFS-6509 URL: https://issues.apache.org/jira/browse/HDFS-6509 Project: Hadoop HDFS Issue Type: Sub-task Components: security Affects Versions: fs-encryption (HADOOP-10150 and HDFS-6134) Reporter: Charles Lamb Assignee: Charles Lamb Attachments: HDFS-6509.001.patch, HDFS-6509.002.patch, HDFS-6509distcpandDataatRestEncryption-2.pdf, HDFS-6509distcpandDataatRestEncryption-3.pdf, HDFS-6509distcpandDataatRestEncryption.pdf This is part of the work for making distcp work with Data at Rest Encryption. Per the attached document, create a /.reserved/raw HDFS filesystem namespace that allows access to the encrypted bytes of a file. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HDFS-6441) Add ability to exclude/include few datanodes while balancing
[ https://issues.apache.org/jira/browse/HDFS-6441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benoy Antony updated HDFS-6441: --- Attachment: HDFS-6441.patch Add ability to exclude/include few datanodes while balancing Key: HDFS-6441 URL: https://issues.apache.org/jira/browse/HDFS-6441 Project: Hadoop HDFS Issue Type: Improvement Components: balancer Affects Versions: 2.4.0 Reporter: Benoy Antony Assignee: Benoy Antony Attachments: HDFS-6441.patch, HDFS-6441.patch, HDFS-6441.patch, HDFS-6441.patch, HDFS-6441.patch, HDFS-6441.patch, HDFS-6441.patch, HDFS-6441.patch, HDFS-6441.patch, HDFS-6441.patch, HDFS-6441.patch In some use cases, it is desirable to ignore a few data nodes while balancing. The administrator should be able to specify a list of data nodes in a file similar to the hosts file and the balancer should ignore these data nodes while balancing so that no blocks are added/removed on these nodes. Similarly it will be beneficial to specify that only a particular list of datanodes should be considered for balancing. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6441) Add ability to exclude/include few datanodes while balancing
[
https://issues.apache.org/jira/browse/HDFS-6441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075821#comment-14075821
]
Benoy Antony commented on HDFS-6441:
Updating the patch including changes from HDFS-6010.
The main change is ability to specify the data nodes to include or exclude via
command line or file.
The second change is the ability to specify the data nodes as a host name or ip
address, with or without port address.
Data nodes can be excluded while balancing as follows :
{panel}
run-balancer.sh -exclude -f hosts-file
{panel}
OR
{panel}
run-balancer.sh -exclude comma-separated-list-of-hosts
{panel}
The specified datanodes will not be used while balancing.
Specific Data nodes can included while balancing as follows:
{panel}
run-balancer.sh -include -f hosts-file
{panel}
OR
{panel}
run-balancer.sh -include comma-separated-list-of-hosts
{panel}
With this, only the specified datanodes will be used for balancing.
Add ability to exclude/include few datanodes while balancing
Key: HDFS-6441
URL: https://issues.apache.org/jira/browse/HDFS-6441
Project: Hadoop HDFS
Issue Type: Improvement
Components: balancer
Affects Versions: 2.4.0
Reporter: Benoy Antony
Assignee: Benoy Antony
Attachments: HDFS-6441.patch, HDFS-6441.patch, HDFS-6441.patch,
HDFS-6441.patch, HDFS-6441.patch, HDFS-6441.patch, HDFS-6441.patch,
HDFS-6441.patch, HDFS-6441.patch, HDFS-6441.patch, HDFS-6441.patch
In some use cases, it is desirable to ignore a few data nodes while
balancing. The administrator should be able to specify a list of data nodes
in a file similar to the hosts file and the balancer should ignore these data
nodes while balancing so that no blocks are added/removed on these nodes.
Similarly it will be beneficial to specify that only a particular list of
datanodes should be considered for balancing.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6441) Add ability to exclude/include few datanodes while balancing
[
https://issues.apache.org/jira/browse/HDFS-6441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075881#comment-14075881
]
Hadoop QA commented on HDFS-6441:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12658052/HDFS-6441.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.ha.TestZKFailoverControllerStress
org.apache.hadoop.hdfs.server.balancer.TestBalancer
org.apache.hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
org.apache.hadoop.TestGenericRefresh
org.apache.hadoop.TestRefreshCallQueue
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/7471//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7471//console
This message is automatically generated.
Add ability to exclude/include few datanodes while balancing
Key: HDFS-6441
URL: https://issues.apache.org/jira/browse/HDFS-6441
Project: Hadoop HDFS
Issue Type: Improvement
Components: balancer
Affects Versions: 2.4.0
Reporter: Benoy Antony
Assignee: Benoy Antony
Attachments: HDFS-6441.patch, HDFS-6441.patch, HDFS-6441.patch,
HDFS-6441.patch, HDFS-6441.patch, HDFS-6441.patch, HDFS-6441.patch,
HDFS-6441.patch, HDFS-6441.patch, HDFS-6441.patch, HDFS-6441.patch
In some use cases, it is desirable to ignore a few data nodes while
balancing. The administrator should be able to specify a list of data nodes
in a file similar to the hosts file and the balancer should ignore these data
nodes while balancing so that no blocks are added/removed on these nodes.
Similarly it will be beneficial to specify that only a particular list of
datanodes should be considered for balancing.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HDFS-6755) There is an unnecessary sleep in the code path where DFSOutputStream#close gives up its attempt to contact the namenode
[
https://issues.apache.org/jira/browse/HDFS-6755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephen Chu updated HDFS-6755:
--
Resolution: Fixed
Fix Version/s: 2.6.0
3.0.0
Status: Resolved (was: Patch Available)
There is an unnecessary sleep in the code path where DFSOutputStream#close
gives up its attempt to contact the namenode
---
Key: HDFS-6755
URL: https://issues.apache.org/jira/browse/HDFS-6755
Project: Hadoop HDFS
Issue Type: Improvement
Affects Versions: 2.6.0
Reporter: Mit Desai
Assignee: Mit Desai
Fix For: 3.0.0, 2.6.0
Attachments: HDFS-6755.patch
DFSOutputStream#close has a loop where it tries to contact the NameNode, to
call {{complete}} on the file which is open-for-write. This loop includes a
sleep which increases exponentially (exponential backoff). It makes sense to
sleep before re-contacting the NameNode, but the code also sleeps even in the
case where it has already decided to give up and throw an exception back to
the user. It should not sleep after it has already decided to give up, since
there's no point.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6709) Implement off-heap data structures for NameNode and other HDFS memory optimization
[
https://issues.apache.org/jira/browse/HDFS-6709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075894#comment-14075894
]
Colin Patrick McCabe commented on HDFS-6709:
bq. I'm just asking leading questions to make sure this approach is sound. Y!
stands to lose a lot if this doesn't actually scale
The questions are good... hopefully the answers are too! I'm just trying to
make my answers as complete as I can.
bq. To clarify the RTTI, I thought you meant more than just a per-instance
reference to the class would be saved - although saving a reference is indeed
great
Yeah. It will shrink objects by 4 or 8 bytes each. It's not immaterial!
Savings like these are why I think it will shrink memory consumption
bq. Regarding atomicity/CAS, it's relevant because using misalignment
(over-optimization?) prevents adding concurrency to data structures that aren't
but should allow concurrency. I digress
Isn't this a minor implementation detail, though? We don't currently use
atomic ops on these data structures. If we go ahead with a layout that uses
unaligned access, and someone later decides to make things atomic, we can
always switch to an aligned layout.
bq. I know about generational collection but I'm admittedly not an expert.
Which young gen GC method does not pause? ParNew+CMS definitively pauses...
Here are some quickly gathered 12-day observations from a moderately loaded,
multi-thousand node, non-production cluster:
I'm not a GC expert either. But from what I've read, does not pause is a
pretty high bar to clear. I think even Azul's GC pauses on occasion for
sub-millisecond intervals. For CMS and G1, everything I've read talks about
tuning the young-gen collection in terms of target pause times.
bq. We have production clusters over 2.5X larger that sustained over 3X
ops/sec. This non-prod cluster is generating ~625MB of garbage/sec. How do you
predict dynamic instantiation of INode and BlockInfo objects will help? They
generally won't be promoted to old gen which should reduce the infrequent CMS
collection times. BUT, will it dramatically increase the frequency of young
collection and/or lead to premature tenuring?
If you look at the code, we create temporary objects all over the place.
For example, look at setTimes:
{code}
private void setTimesInt(String src, long mtime, long atime)
throws IOException, UnresolvedLinkException {
HdfsFileStatus resultingStat = null;
FSPermissionChecker pc = getPermissionChecker();
checkOperation(OperationCategory.WRITE);
byte[][] pathComponents = FSDirectory.getPathComponentsForReservedPath(src);
writeLock();
try {
checkOperation(OperationCategory.WRITE);
checkNameNodeSafeMode(Cannot set times + src);
src = FSDirectory.resolvePath(src, pathComponents, dir);
// Write access is required to set access and modification times
if (isPermissionEnabled) {
checkPathAccess(pc, src, FsAction.WRITE);
}
final INodesInPath iip = dir.getINodesInPath4Write(src);
final INode inode = iip.getLastINode();
{code}
You can see we create:
HdfsFileStatus (with at least 5 sub-objects. one of those, FsPermission, has 3
sub-objects of its own)
FSPermissionChecker (which has at least 5 sub-objects inside it)
pathComponents
new src string
INodesInPath (at least 2 sub-objects of its own)
That's at least 21 temporary objects just in this code snippet, and I'm sure I
missed a lot of things. I'm not including any of the functions that called or
were called by this function, or any of the RPC or protobuf machinations. The
average path depth is maybe between 5 and 8... would having 5 to 8 extra
temporary objects to represent INodes we traversed substantially increase the
GC load? I would say no.
Maybe you think I've chosen an easy example. Hmm... the operation that I can
think of that touches the most inodes is recursive delete. But we've known
about the problems with this for a while... that's why JIRAs like HDFS-2938
addressed the problem. Arguably, an off-heap implementation is actually better
here since we avoid creating a lot of trash in the tenured generation. And
trash in the tenured generation leads to heap fragmentations (at least in CMS),
and the dreaded full GC.
Implement off-heap data structures for NameNode and other HDFS memory
optimization
--
Key: HDFS-6709
URL: https://issues.apache.org/jira/browse/HDFS-6709
Project: Hadoop HDFS
Issue Type: Improvement
Components: namenode
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
Attachments: HDFS-6709.001.patch
We should investigate implementing off-heap data structures for NameNode and
other HDFS memory
[jira] [Created] (HDFS-6759) Namenode Web UI only display 1 datanode when multiple datanodes exist on same hostname under different ports
Konnjuta created HDFS-6759: -- Summary: Namenode Web UI only display 1 datanode when multiple datanodes exist on same hostname under different ports Key: HDFS-6759 URL: https://issues.apache.org/jira/browse/HDFS-6759 Project: Hadoop HDFS Issue Type: Bug Components: datanode, namenode Affects Versions: 2.4.1, 2.4.0 Environment: Centos 6.5 Reporter: Konnjuta Priority: Trivial NameNode Web UI only display 1 datanode when multiple datanodes exist on same hostname under different ports. This seems to affect only version 2.x Version 1.x working as expected. This is a development environment so multiple datanodes on a single hostname is common. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HDFS-6247) Avoid timeouts for replaceBlock() call by sending intermediate responses to Balancer
[ https://issues.apache.org/jira/browse/HDFS-6247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-6247: Attachment: HDFS-6247.patch Attaching the patch to shorten lines exceeding 80 chars Avoid timeouts for replaceBlock() call by sending intermediate responses to Balancer Key: HDFS-6247 URL: https://issues.apache.org/jira/browse/HDFS-6247 Project: Hadoop HDFS Issue Type: Bug Components: balancer, datanode Affects Versions: 2.4.0 Reporter: Vinayakumar B Assignee: Vinayakumar B Attachments: HDFS-6247.patch, HDFS-6247.patch, HDFS-6247.patch, HDFS-6247.patch, HDFS-6247.patch, HDFS-6247.patch Currently there is no response sent from target Datanode to Balancer for the replaceBlock() calls. Since the Block movement for balancing is throttled, complete block movement will take time and this could result in timeout at Balancer, which will be trying to read the status message. To Avoid this during replaceBlock() call in in progress Datanode can send IN_PROGRESS status messages to Balancer to avoid timeouts and treat BlockMovement as failed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HDFS-6247) Avoid timeouts for replaceBlock() call by sending intermediate responses to Balancer
[ https://issues.apache.org/jira/browse/HDFS-6247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-6247: Attachment: HDFS-6247.patch Excluding indentation changes. Avoid timeouts for replaceBlock() call by sending intermediate responses to Balancer Key: HDFS-6247 URL: https://issues.apache.org/jira/browse/HDFS-6247 Project: Hadoop HDFS Issue Type: Bug Components: balancer, datanode Affects Versions: 2.4.0 Reporter: Vinayakumar B Assignee: Vinayakumar B Attachments: HDFS-6247.patch, HDFS-6247.patch, HDFS-6247.patch, HDFS-6247.patch, HDFS-6247.patch, HDFS-6247.patch Currently there is no response sent from target Datanode to Balancer for the replaceBlock() calls. Since the Block movement for balancing is throttled, complete block movement will take time and this could result in timeout at Balancer, which will be trying to read the status message. To Avoid this during replaceBlock() call in in progress Datanode can send IN_PROGRESS status messages to Balancer to avoid timeouts and treat BlockMovement as failed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HDFS-6247) Avoid timeouts for replaceBlock() call by sending intermediate responses to Balancer
[ https://issues.apache.org/jira/browse/HDFS-6247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinayakumar B updated HDFS-6247: Attachment: (was: HDFS-6247.patch) Avoid timeouts for replaceBlock() call by sending intermediate responses to Balancer Key: HDFS-6247 URL: https://issues.apache.org/jira/browse/HDFS-6247 Project: Hadoop HDFS Issue Type: Bug Components: balancer, datanode Affects Versions: 2.4.0 Reporter: Vinayakumar B Assignee: Vinayakumar B Attachments: HDFS-6247.patch, HDFS-6247.patch, HDFS-6247.patch, HDFS-6247.patch, HDFS-6247.patch, HDFS-6247.patch Currently there is no response sent from target Datanode to Balancer for the replaceBlock() calls. Since the Block movement for balancing is throttled, complete block movement will take time and this could result in timeout at Balancer, which will be trying to read the status message. To Avoid this during replaceBlock() call in in progress Datanode can send IN_PROGRESS status messages to Balancer to avoid timeouts and treat BlockMovement as failed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6247) Avoid timeouts for replaceBlock() call by sending intermediate responses to Balancer
[
https://issues.apache.org/jira/browse/HDFS-6247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14075919#comment-14075919
]
Vinayakumar B commented on HDFS-6247:
-
bq. At least one line exceeds the 80 char width.
Fixed
bq. In receiveBlock, you changed the indentation which causes unnecessary code
churn.
excluded in latest patch
{quote}
It's my understanding that general patching protocol is that you should have
each new patch file be named with a sequential number. Hence,
HDFS-6247.001.patch
HDFS-6247.002.patch
...{quote}
Thanks for the suggestion. I will follow this from now onwards for new patch
submissions.
Avoid timeouts for replaceBlock() call by sending intermediate responses to
Balancer
Key: HDFS-6247
URL: https://issues.apache.org/jira/browse/HDFS-6247
Project: Hadoop HDFS
Issue Type: Bug
Components: balancer, datanode
Affects Versions: 2.4.0
Reporter: Vinayakumar B
Assignee: Vinayakumar B
Attachments: HDFS-6247.patch, HDFS-6247.patch, HDFS-6247.patch,
HDFS-6247.patch, HDFS-6247.patch, HDFS-6247.patch
Currently there is no response sent from target Datanode to Balancer for the
replaceBlock() calls.
Since the Block movement for balancing is throttled, complete block movement
will take time and this could result in timeout at Balancer, which will be
trying to read the status message.
To Avoid this during replaceBlock() call in in progress Datanode can send
IN_PROGRESS status messages to Balancer to avoid timeouts and treat
BlockMovement as failed.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HDFS-6760) Deletion of directories with snapshots will not output reason for trash move failure
Stephen Chu created HDFS-6760:
-
Summary: Deletion of directories with snapshots will not output
reason for trash move failure
Key: HDFS-6760
URL: https://issues.apache.org/jira/browse/HDFS-6760
Project: Hadoop HDFS
Issue Type: Improvement
Affects Versions: 2.4.0
Reporter: Stephen Chu
Assignee: Stephen Chu
Priority: Minor
When using trash-enabled FsShell to delete a directory that has snapshots, we
se an error message saying Failed to move to trash but no explanation.
{code}
[hdfs@schu-enc2 ~]$ hdfs dfs -rm -r snap
2014-07-28 05:45:29,527 INFO [main] fs.TrashPolicyDefault
(TrashPolicyDefault.java:initialize(92)) - Namenode trash configuration:
Deletion interval = 1440 minutes, Emptier interval = 0 minutes.
rm: Failed to move to trash: hdfs://schu-enc2.vpc.com:8020/user/hdfs/snap.
Consider using -skipTrash option
{code}
If we use -skipTrash, then we'll get the explanation: rm: The directory
/user/hdfs/snap cannot be deleted since /user/hdfs/snap is snapshottable and
already has snapshots
It'd be an improvement to make it clear that dirs with snapshots cannot be
deleted when we're using the trash.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
