On Sun, Feb 17, 2013 at 5:08 PM, Harsh J wrote:
> Hi Robert,
>
> Are you by any chance adding files carrying unusual encoding?
I don't believe so. The only files I push to HDFS are SequenceFiles (with
protobuf objects in them) and HBase's regions, which again is just protobuf
objects. I don't
Hi Robert,
Are you by any chance adding files carrying unusual encoding? If its
possible, can we be sent a bundle of the corrupted log set (all of the
dfs.name.dir contents) to inspect what seems to be causing the
corruption?
The only identified (but rarely occurring) bug around this part in
1.0.
On Sun, Feb 17, 2013 at 4:41 PM, Mohammad Tariq wrote:
> You can make use of offine image viewer to diagnose
> the fsimage file.
>
Is this not included in the 1.0.x branch? All of the documentation I find
for it says to run 'bin/hdfs oev' but I do not have a 'bin/hdfs'.
> Warm Regards,
> Tari
On Sun, Feb 17, 2013 at 4:41 PM, Mohammad Tariq wrote:
> Hello Robert,
>
> It seems that your edit logs and fsimage have got
> corrupted somehow. It looks somewhat similar to this one
> https://issues.apache.org/jira/browse/HDFS-686
>
Similar, but the trace is different.
> Have you ma
Hello Robert,
It seems that your edit logs and fsimage have got
corrupted somehow. It looks somewhat similar to this one
https://issues.apache.org/jira/browse/HDFS-686
Have you made any changes to the 'dfs.name.dir' directory
lately?Do you have enough space where metadata is getting
stor
It just happened again. This was after a fresh format of HDFS/HBase and I
am attempting to re-import the (backed up) data.
http://pastebin.com/3fsWCNQY
So now if I restart the namenode, I will lose data from the past 3 hours.
What is causing this? How can I avoid it in the future? Is there
Forgot to mention: Hadoop 1.0.4
On Sat, Feb 16, 2013 at 2:38 PM, Robert Dyer wrote:
> I am at a bit of wits end here. Every single time I restart the namenode,
> I get this crash:
>
> 2013-02-16 14:32:42,616 INFO org.apache.hadoop.hdfs.server.common.Storage:
> Image file of size 168058 loaded
