[
https://issues.apache.org/jira/browse/HADOOP-5744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12710547#action_12710547
]
Hairong Kuang commented on HADOOP-5744:
---------------------------------------
Jim/Stack, thanks for your comments. This makes us to understand our users'
requirements better.
> It doesn't have to be closed as long the reader is able to go all the ways up
> to the last sync made by the writer before crash.
Yes, all flushed data are guaranteed to be read by a new reader. I also want
you to be aware that the reader may read beyond the flushed data.
> The file may have more bytes than when it was previously read. This is a norm
> case. Will this be an issue to hbase? How would this circumstance arise?
Our idea is that not all the data received by a datanode are visible to a user.
For example. if a dfs application calls flush, the data left the client machine
and are being pushed to all datanodes, but the application dies before it
receives the flush ack. This flush is considered as a failure. The flushed data
may have reached none, one, or all datanodes. So the data is not guaranteed to
be visible to a reader. But the lease recovery MAY still keep the data in the
file if the data have reached one or all datanodes.
> Revisit append
> --------------
>
> Key: HADOOP-5744
> URL: https://issues.apache.org/jira/browse/HADOOP-5744
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Affects Versions: 0.20.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.21.0
>
> Attachments: AppendSpec.pdf
>
>
> HADOOP-1700 and related issues have put a lot of efforts to provide the first
> implementation of append. However, append is such a complex feature. It turns
> out that there are issues that were initially seemed trivial but needs a
> careful design. This jira revisits append, aiming for a design and
> implementation supporting a semantics that are acceptable to its users.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
