HDFS-11407. Document the missing usages of OfflineImageViewer processors. 
Contributed by Yiqun Lin.


Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/719df99c
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/719df99c
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/719df99c

Branch: refs/heads/YARN-2915
Commit: 719df99c05a4e0bc62a7cad6d22c8dbe5d92bde0
Parents: 71c23c9
Author: Yiqun Lin <yq...@apache.org>
Authored: Tue Feb 14 18:48:08 2017 +0800
Committer: Yiqun Lin <yq...@apache.org>
Committed: Tue Feb 14 18:48:08 2017 +0800

----------------------------------------------------------------------
 .../src/site/markdown/HdfsImageViewer.md        | 63 ++++++++++++++++++++
 1 file changed, 63 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/hadoop/blob/719df99c/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsImageViewer.md
----------------------------------------------------------------------
diff --git 
a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsImageViewer.md 
b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsImageViewer.md
index b677f6a..10d98b2 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsImageViewer.md
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsImageViewer.md
@@ -133,6 +133,69 @@ Applying the Offline Image Viewer with XML processor would 
result in the followi
          </inode>
        ...remaining output omitted...
 
+### ReverseXML Processor
+
+ReverseXML processor is the opposite of the XML processor. Users can specify 
input XML file and output fsimage file via -i and -o command-line.
+
+       bash$ bin/hdfs oiv -p ReverseXML -i fsimage.xml -o fsimage
+
+This will reconstruct an fsimage from an XML file.
+
+### FileDistribution Processor
+
+FileDistribution processor can analyze file sizes in the namespace image. 
Users can specify maxSize (128GB by default) and step (2MB by default) in bytes 
via -maxSize and -step command-line.
+
+       bash$ bin/hdfs oiv -p FileDistribution -maxSize maxSize -step size -i 
fsimage -o output
+
+The processor will calculate how many files in the system fall into each 
segment. The output file is formatted as a tab separated two column table 
showed as the following output:
+
+       Size    NumFiles
+       4       1
+       12      1
+       16      1
+       20      1
+       totalFiles = 4
+       totalDirectories = 2
+       totalBlocks = 4
+       totalSpace = 48
+       maxFileSize = 21
+
+To make the output result look more readable, users can specify -format option 
in addition.
+
+       bash$ bin/hdfs oiv -p FileDistribution -maxSize maxSize -step size 
-format -i fsimage -o output
+
+This would result in the following output:
+
+       Size Range      NumFiles
+       (0 B, 4 B]      1
+       (8 B, 12 B]     1
+       (12 B, 16 B]    1
+       (16 B, 21 B]    1
+       totalFiles = 4
+       totalDirectories = 2
+       totalBlocks = 4
+       totalSpace = 48
+       maxFileSize = 21
+
+### Delimited Processor
+
+Delimited processor generates a text representation of the fsimage, with each 
element separated by a delimiter string (\t by default). Users can specify a 
new delimiter string by -delimiter option.
+
+       bash$ bin/hdfs oiv -p Delimited -delimiter delimiterString -i fsimage 
-o output
+
+In addition, users can specify a temporary dir to cache intermediate result by 
the following command:
+
+       bash$ bin/hdfs oiv -p Delimited -delimiter delimiterString -t 
temporaryDir -i fsimage -o output
+
+If not set, Delimited processor will construct the namespace in memory before 
outputting text. The output result of this processor should be like the 
following output:
+
+       Path    Replication     ModificationTime        AccessTime      
PreferredBlockSize      BlocksCount     FileSize        NSQUOTA DSQUOTA 
Permission      UserName        GroupName
+       /       0       2017-02-13 10:39        1970-01-01 08:00        0       
0       0       9223372036854775807     -1      drwxr-xr-x      root    
supergroup
+       /dir0   0       2017-02-13 10:39        1970-01-01 08:00        0       
0       0       -1      -1      drwxr-xr-x      root    supergroup
+       /dir0/file0     1       2017-02-13 10:39        2017-02-13 10:39        
134217728       1       1       0       0       -rw-r--r--      root    
supergroup
+       /dir0/file1     1       2017-02-13 10:39        2017-02-13 10:39        
134217728       1       1       0       0       -rw-r--r--      root    
supergroup
+       /dir0/file2     1       2017-02-13 10:39        2017-02-13 10:39        
134217728       1       1       0       0       -rw-r--r--      root    
supergroup
+
 Options
 -------
 


---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-commits-h...@hadoop.apache.org

Reply via email to