Manoj Govindassamy created HDFS-11058:
-----------------------------------------
Summary: Implement 'hadoop fs -df' command for ViewFileSystem
Key: HDFS-11058
URL: https://issues.apache.org/jira/browse/HDFS-11058
Project: Hadoop HDFS
Issue Type: Task
Affects Versions: 3.0.0-alpha1
Reporter: Manoj Govindassamy
Assignee: Manoj Govindassamy
Df command doesn't seem to work well with ViewFileSystem. It always reports
used data as 0. Here is the client mount table configuration I am using against
a federated clusters of 2 NameNodes and 2 DataNoes.
{code}
1 <?xml version="1.0" ?>
2 <configuration>
3 <property>
4 <name>fs.defaultFS</name>
5 <value>viewfs://ClusterX/</value>
6 </property>
..
11 <property>
12 <name>fs.default.name</name>
13 <value>viewfs://ClusterX/</value>
14 </property>
..
23 <property>
24 <name>fs.viewfs.mounttable.ClusterX.link./nn0</name>
25 <value>hdfs://127.0.0.1:50001/</value>
26 </property>
27 <property>
28 <name>fs.viewfs.mounttable.ClusterX.link./nn1</name>
29 <value>hdfs://127.0.0.1:51001/</value>
30 </property>
31 <property>
32 <name>fs.viewfs.mounttable.ClusterX.link./nn2</name>
33 <value>hdfs://127.0.0.1:52001/nn2</value>
34 </property>
35 <property>
36 <name>fs.viewfs.mounttable.ClusterX.link./nn3</name>
37 <value>hdfs://127.0.0.1:52001/nn3</value>
38 </property>
39 <property>
40 <name>fs.viewfs.mounttable.ClusterY.linkMergeSlash</name>
41 <value>hdfs://127.0.0.1:50001/</value>
42 </property>
43 </configuration>
{code}
{{Df}} command always reports Size/Available as 8.0E and the usage as 0 for any
federated cluster.
{noformat}
# hadoop fs -fs viewfs://ClusterX/ -df /
Filesystem Size Used Available Use%
viewfs://ClusterX/ 9223372036854775807 0 9223372036854775807 0%
# hadoop fs -fs viewfs://ClusterX/ -df -h /
Filesystem Size Used Available Use%
viewfs://ClusterX/ 8.0 E 0 8.0 E 0%
# hadoop fs -fs viewfs://ClusterY/ -df -h /
Filesystem Size Used Available Use%
viewfs://ClusterY/ 8.0 E 0 8.0 E 0%
{noformat}
Whereas {{Du}} command seems to work as expected even with ViewFileSystem.
{noformat}
# hadoop fs -fs viewfs://ClusterY/ -du -h /
10.6 K 31.8 K /build.log.16y
0 0 /user
# hadoop fs -fs viewfs://ClusterX/ -du -h /
10.6 K 31.8 K /nn0
0 0 /nn1
20.2 K 35.8 K /nn3
40.6 K 34.3 K /nn4
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
