[
https://issues.apache.org/jira/browse/HDFS-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14290278#comment-14290278
]
Haohui Mai commented on HDFS-6673:
----------------------------------
-0
bq. ... a fairly large fsimage here. ... I think the current level of
performance is sufficient for the vast majority of our customers.
The test is on a 3G fsimage which can easily fit in the working set of your
laptop.
Multiple users are running much larger clusters, where their fsimage can be as
big as 40G (see HDFS-5698). I see the value of using LevelDB as a swap space to
handle fsimages that are bigger than the working set, but what are the net
benefits that the tool bring in if it can only handle fsimages that are 10x
smaller than the ones in some of the production clusters?
bq. Furthermore, this is a boolean improvement over the previous state of
affairs; currently, we have no delimited OIV tool, and with this patch, we do.
This is not true. Delimited OIV was such a headache thus we had to revived the
legacy fsimage saver / loader / oiv in HDFS-6293.
bq. This is the result of a few rounds of performance tuning.
You guys deserve all the credits of getting this tool working, but given we
have HDFS-6293 as a solid solution today, I would much rather to see this tool
to be capable of handling fsimages from the real, large-scale production runs
(at least from the design point of view), instead of putting in a half-baked
solution as-is. I'm also happy to provide help if necessary.
> Add Delimited format supports for PB OIV tool
> ---------------------------------------------
>
> Key: HDFS-6673
> URL: https://issues.apache.org/jira/browse/HDFS-6673
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: 2.4.0
> Reporter: Lei (Eddy) Xu
> Assignee: Lei (Eddy) Xu
> Priority: Minor
> Attachments: HDFS-6673.000.patch, HDFS-6673.001.patch,
> HDFS-6673.002.patch, HDFS-6673.003.patch, HDFS-6673.004.patch,
> HDFS-6673.005.patch
>
>
> The new oiv tool, which is designed for Protobuf fsimage, lacks a few
> features supported in the old {{oiv}} tool.
> This task adds supports of _Delimited_ processor to the oiv tool.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
