[jira] Commented: (HDFS-1435) Provide an option to store fsimage compressed

2010-10-23 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12924286#action_12924286 ] Hudson commented on HDFS-1435: -- Integrated in Hadoop-Hdfs-trunk-Commit #415 (See [https://huds

[jira] Commented: (HDFS-1435) Provide an option to store fsimage compressed

2010-10-20 Thread dhruba borthakur (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923213#action_12923213 ] dhruba borthakur commented on HDFS-1435: +1, code looks good to me. > Provide an o

[jira] Commented: (HDFS-1435) Provide an option to store fsimage compressed

2010-10-20 Thread Aaron T. Myers (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12922987#action_12922987 ] Aaron T. Myers commented on HDFS-1435: -- Thanks for the clarification, Hairong. Konstan

[jira] Commented: (HDFS-1435) Provide an option to store fsimage compressed

2010-10-19 Thread Hairong Kuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12922702#action_12922702 ] Hairong Kuang commented on HDFS-1435: - Sorry for the confusion. The bug is introduced by

[jira] Commented: (HDFS-1435) Provide an option to store fsimage compressed

2010-10-19 Thread Aaron T. Myers (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12922695#action_12922695 ] Aaron T. Myers commented on HDFS-1435: -- Hi Hairong, what bug in the OIV did this fix? a

[jira] Commented: (HDFS-1435) Provide an option to store fsimage compressed

2010-10-15 Thread Hairong Kuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12921494#action_12921494 ] Hairong Kuang commented on HDFS-1435: - [exec] +1 overall. [exec] [exec

[jira] Commented: (HDFS-1435) Provide an option to store fsimage compressed

2010-10-14 Thread dhruba borthakur (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12921193#action_12921193 ] dhruba borthakur commented on HDFS-1435: I looked at the code, looks good. Have two

[jira] Commented: (HDFS-1435) Provide an option to store fsimage compressed

2010-10-14 Thread Hairong Kuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12921178#action_12921178 ] Hairong Kuang commented on HDFS-1435: - Todd, this is a wonderful idea! When I discussed

[jira] Commented: (HDFS-1435) Provide an option to store fsimage compressed

2010-10-14 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12921170#action_12921170 ] Todd Lipcon commented on HDFS-1435: --- Hey Hairong. Another idea which you may want to exper

[jira] Commented: (HDFS-1435) Provide an option to store fsimage compressed

2010-10-14 Thread Hairong Kuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12921068#action_12921068 ] Hairong Kuang commented on HDFS-1435: - Oops! The header of the table above should shift

[jira] Commented: (HDFS-1435) Provide an option to store fsimage compressed

2010-10-14 Thread Hairong Kuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12921060#action_12921060 ] Hairong Kuang commented on HDFS-1435: - I did experiments with a secondary namenode using

[jira] Commented: (HDFS-1435) Provide an option to store fsimage compressed

2010-10-14 Thread Lu Yilei (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12920929#action_12920929 ] Lu Yilei commented on HDFS-1435: to Kuang, If the fsimage is very big. The network is full

[jira] Commented: (HDFS-1435) Provide an option to store fsimage compressed

2010-10-13 Thread Hairong Kuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12920770#action_12920770 ] Hairong Kuang commented on HDFS-1435: - @Lu, compressing the fsimage has additional advan

[jira] Commented: (HDFS-1435) Provide an option to store fsimage compressed

2010-10-13 Thread Doug Cutting (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12920513#action_12920513 ] Doug Cutting commented on HDFS-1435: Hairong, I don' think that using Avro is critical h

[jira] Commented: (HDFS-1435) Provide an option to store fsimage compressed

2010-10-12 Thread Lu Yilei (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12920237#action_12920237 ] Lu Yilei commented on HDFS-1435: We also encountered the same problem, our fsimage has 12G.

[jira] Commented: (HDFS-1435) Provide an option to store fsimage compressed

2010-10-11 Thread Hairong Kuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12920008#action_12920008 ] Hairong Kuang commented on HDFS-1435: - Sorry that I did not get to work on Avro's file f

[jira] Commented: (HDFS-1435) Provide an option to store fsimage compressed

2010-10-07 Thread Doug Cutting (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12919079#action_12919079 ] Doug Cutting commented on HDFS-1435: Hairong, Avro's file format has little overhead. I

[jira] Commented: (HDFS-1435) Provide an option to store fsimage compressed

2010-10-07 Thread Jeff Hammerbacher (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12918800#action_12918800 ] Jeff Hammerbacher commented on HDFS-1435: - bq. Jeff, I'd like to take a look at the

[jira] Commented: (HDFS-1435) Provide an option to store fsimage compressed

2010-10-05 Thread Hairong Kuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12918222#action_12918222 ] Hairong Kuang commented on HDFS-1435: - LZO compression codec is not supported in Hadoop

[jira] Commented: (HDFS-1435) Provide an option to store fsimage compressed

2010-10-04 Thread dhruba borthakur (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12917876#action_12917876 ] dhruba borthakur commented on HDFS-1435: +1 on compressing the entire file. The VER

[jira] Commented: (HDFS-1435) Provide an option to store fsimage compressed

2010-10-04 Thread Hairong Kuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12917768#action_12917768 ] Hairong Kuang commented on HDFS-1435: - I thought more about Philip's suggestion. So inst

[jira] Commented: (HDFS-1435) Provide an option to store fsimage compressed

2010-10-01 Thread dhruba borthakur (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12917164#action_12917164 ] dhruba borthakur commented on HDFS-1435: > time to load the image and the time to d

[jira] Commented: (HDFS-1435) Provide an option to store fsimage compressed

2010-10-01 Thread Hairong Kuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916938#action_12916938 ] Hairong Kuang commented on HDFS-1435: - Phillip, your suggestion definitely has value tha

[jira] Commented: (HDFS-1435) Provide an option to store fsimage compressed

2010-09-30 Thread Jeff Hammerbacher (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916795#action_12916795 ] Jeff Hammerbacher commented on HDFS-1435: - Could we use the Avro file format to stor

[jira] Commented: (HDFS-1435) Provide an option to store fsimage compressed

2010-09-30 Thread Philip Zeyliger (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916793#action_12916793 ] Philip Zeyliger commented on HDFS-1435: --- Instead of changing the binary format, could

[jira] Commented: (HDFS-1435) Provide an option to store fsimage compressed

2010-09-30 Thread Hairong Kuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916747#action_12916747 ] Hairong Kuang commented on HDFS-1435: - This depends on the compression algorithm to be u

[jira] Commented: (HDFS-1435) Provide an option to store fsimage compressed

2010-09-30 Thread Dmytro Molkov (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916737#action_12916737 ] Dmytro Molkov commented on HDFS-1435: - Do you expect the time to load the image and the