[
https://issues.apache.org/jira/browse/HDFS-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13740166#comment-13740166
]
Andrew Wang commented on HDFS-5051:
-----------------------------------
I included the gen stamp and length in the {{cacheReport}} to handle caching
newly appended data. I guess the gen stamp is unnecessary, but the DN isn't
going to automatically mlock newly appended data, so the NN needs to somehow
realize that the cached length is shorter than the new length and ask the DN to
recache at the new length. Alternatively, I guess the DN could automatically
mlock appended data, but there are quota implications there.
On startup, I agree that we can skip cache reports until the cache is
populated. I also agree that jittering doesn't matter as much if it's ticking
on such a short time scale. I guess I could have cleaned this up rather than
just changing the default cache report period like Colin asked.
However, since we want to eventually have both incremental and full reports,
let's just ape how block reports work; don't jitter the incremental reports,
but do jitter the start time for the full reports and afterwards tick at a
regular interval. Let's clean up all these issues in the incremental cache
report JIRA (HDFS-5092); if this sounds good, I'll edit the JIRA description
with these todo items.
> Propagate cache status information from the DataNode to the NameNode
> --------------------------------------------------------------------
>
> Key: HDFS-5051
> URL: https://issues.apache.org/jira/browse/HDFS-5051
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode, namenode
> Reporter: Colin Patrick McCabe
> Assignee: Andrew Wang
> Attachments: hdfs-5051-1.patch, hdfs-5051-2.patch
>
>
> The DataNode needs to inform the NameNode of its current cache state. Let's
> wire up the RPCs and stub out the relevant methods on the DN and NN side.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
