[
https://issues.apache.org/jira/browse/HDFS-200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12730666#action_12730666
]
stack commented on HDFS-200:
----------------------------
(Thanks for review Konstantin)
In my last few test runs, NameNode has shut itself down with the below:
{code}
...
009-07-14 00:17:46,586 DEBUG org.apache.hadoop.hdfs.StateChange: BLOCK*
NameSystem.removeStoredBlock: blk_-9156287469566772234_2527 from
XX.XX.XX.142:51010
2009-07-14 00:17:46,586 DEBUG org.apache.hadoop.hdfs.StateChange: BLOCK*
NameSystem.removeStoredBlock: blk_-9181830129071396520_2355 from
XX.XX.XX.142:51010
2009-07-14 00:17:46,586 DEBUG org.apache.hadoop.hdfs.StateChange: BLOCK*
NameSystem.removeStoredBlock: blk_-9205119721509648294_2410 from
XX.XX.XX.142:51010
2009-07-14 00:17:46,586 INFO org.apache.hadoop.hdfs.StateChange: BLOCK*
NameSystem.processReport: block blk_-7011715647341740217_1 on
XX.XX.XX.142:51010 size 47027149 does not belong to any file.
2009-07-14 00:17:46,586 INFO org.apache.hadoop.hdfs.StateChange: BLOCK*
NameSystem.addToInvalidates: blk_-7011715647341740217 is added to invalidSet of
XX.XX.XX.142:51010
2009-07-14 00:17:46,586 INFO org.apache.hadoop.hdfs.StateChange: BLOCK*
NameSystem.processReport: block blk_-280166356715716926_1 on XX.XX.XX.142:51010
size 6487 does not belong to any file.
2009-07-14 00:17:46,586 INFO org.apache.hadoop.hdfs.StateChange: BLOCK*
NameSystem.addToInvalidates: blk_-280166356715716926 is added to invalidSet of
XX.XX.XX.142:51010
2009-07-14 00:17:46,586 INFO org.apache.hadoop.hdfs.StateChange: BLOCK*
NameSystem.processReport: block blk_1532053033915429278_1 on XX.XX.XX.142:51010
size 3869 does not belong to any file.
2009-07-14 00:17:46,586 INFO org.apache.hadoop.hdfs.StateChange: BLOCK*
NameSystem.addToInvalidates: blk_1532053033915429278 is added to invalidSet of
XX.XX.XX.142:51010
2009-07-14 00:17:47,303 WARN
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: ReplicationMonitor thread
received Runtime exception. java.lang.IllegalStateException: generationStamp
(=1) == GenerationStamp.WILDCARD_STAMP
2009-07-14 00:17:47,304 INFO org.apache.hadoop.hdfs.server.namenode.NameNode:
SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at aa0-000-12.u.powerset.com/XX.XX.XX.139
************************************************************/
{code}
My guess this is a bug only fellas with dfs.support.append=true set run in to?
Here is code from ReplicationMonitor:
{code}
} catch (Throwable t) {
LOG.warn("ReplicationMonitor thread received Runtime exception. " +
t);
Runtime.getRuntime().exit(-1);
}
{code}
Thats a rough call I'd say?
There are no more detailed exceptions in NN log.
Dig in more and stick what I find in another issue?
> In HDFS, sync() not yet guarantees data available to the new readers
> --------------------------------------------------------------------
>
> Key: HDFS-200
> URL: https://issues.apache.org/jira/browse/HDFS-200
> Project: Hadoop HDFS
> Issue Type: New Feature
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: dhruba borthakur
> Priority: Blocker
> Attachments: 4379_20081010TC3.java, fsyncConcurrentReaders.txt,
> fsyncConcurrentReaders11_20.txt, fsyncConcurrentReaders3.patch,
> fsyncConcurrentReaders4.patch, fsyncConcurrentReaders5.txt,
> fsyncConcurrentReaders6.patch, fsyncConcurrentReaders9.patch,
> hadoop-stack-namenode-aa0-000-12.u.powerset.com.log.gz,
> hypertable-namenode.log.gz, namenode.log, namenode.log, Reader.java,
> Reader.java, reopen_test.sh, ReopenProblem.java, Writer.java, Writer.java
>
>
> In the append design doc
> (https://issues.apache.org/jira/secure/attachment/12370562/Appends.doc), it
> says
> * A reader is guaranteed to be able to read data that was 'flushed' before
> the reader opened the file
> However, this feature is not yet implemented. Note that the operation
> 'flushed' is now called "sync".
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
