[
https://issues.apache.org/jira/browse/HADOOP-5752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jakob Homan updated HADOOP-5752:
--------------------------------
Attachment: HADOOP-5752.patch
The OIV's output data are ripe for analysis. The attached patch:
* Creates a new image processor, Delimited, that creates a (by default)
tab-delimited file of the namespace that is suitable for analysis by other
tools.
* Updates the the oiv documentation to provide examples of how to analyze these
files using Pig to find probable duplicate files, files that have never been
accessed and the total number of files of each user in the namespace. These are
meant as examples to help ops and such build other useful scripts.
* Provides unit test for new DelimitedImageVisitor
Right now the script files themselves are not included in the patch because I
couldn't figure out a good place to stash them in the file structure.
Konstantin suggested adding them to the wiki, which would be nice as other
users could add other scripts as they are created, but I don't see where the
wiki hosts files like these. If it can, can someone please point me to them?
Santhosh from the Pig team kindly reviewed and blessed the pig scripts.
> Provide examples of using offline image viewer (oiv) to analyze hadoop file
> systems
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-5752
> URL: https://issues.apache.org/jira/browse/HADOOP-5752
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Reporter: Jakob Homan
> Assignee: Jakob Homan
> Attachments: HADOOP-5752.patch
>
>
> The offline image viewer provides the ability to generate large amounts of
> data about an hdfs namespace. It would be good to provide tools, examples,
> etc. on how to analyze this data to find useful information.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
