[jira] [Commented] (HDFS-5051) Propagate cache status information from the DataNode to the NameNode
[ https://issues.apache.org/jira/browse/HDFS-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13740866#comment-13740866 ] Hudson commented on HDFS-5051: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #302 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/302/]) HDFS-5051. nn fails to download checkpointed image from snn in some setups. Contributed by Vinay and Suresh Srinivas. (suresh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1514110) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/GetImageServlet.java Propagate cache status information from the DataNode to the NameNode Key: HDFS-5051 URL: https://issues.apache.org/jira/browse/HDFS-5051 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Colin Patrick McCabe Assignee: Andrew Wang Attachments: hdfs-5051-1.patch, hdfs-5051-2.patch The DataNode needs to inform the NameNode of its current cache state. Let's wire up the RPCs and stub out the relevant methods on the DN and NN side. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-5051) Propagate cache status information from the DataNode to the NameNode
[ https://issues.apache.org/jira/browse/HDFS-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13740938#comment-13740938 ] Hudson commented on HDFS-5051: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1492 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1492/]) HDFS-5051. nn fails to download checkpointed image from snn in some setups. Contributed by Vinay and Suresh Srinivas. (suresh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1514110) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/GetImageServlet.java Propagate cache status information from the DataNode to the NameNode Key: HDFS-5051 URL: https://issues.apache.org/jira/browse/HDFS-5051 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Colin Patrick McCabe Assignee: Andrew Wang Attachments: hdfs-5051-1.patch, hdfs-5051-2.patch The DataNode needs to inform the NameNode of its current cache state. Let's wire up the RPCs and stub out the relevant methods on the DN and NN side. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-5051) Propagate cache status information from the DataNode to the NameNode
[ https://issues.apache.org/jira/browse/HDFS-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741009#comment-13741009 ] Hudson commented on HDFS-5051: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1519 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1519/]) HDFS-5051. nn fails to download checkpointed image from snn in some setups. Contributed by Vinay and Suresh Srinivas. (suresh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1514110) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/GetImageServlet.java Propagate cache status information from the DataNode to the NameNode Key: HDFS-5051 URL: https://issues.apache.org/jira/browse/HDFS-5051 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Colin Patrick McCabe Assignee: Andrew Wang Attachments: hdfs-5051-1.patch, hdfs-5051-2.patch The DataNode needs to inform the NameNode of its current cache state. Let's wire up the RPCs and stub out the relevant methods on the DN and NN side. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-5051) Propagate cache status information from the DataNode to the NameNode
[ https://issues.apache.org/jira/browse/HDFS-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13739343#comment-13739343 ] Colin Patrick McCabe commented on HDFS-5051: The random jitter code was taken from the block report code. The goal is the same-- to avoid overloading the NameNode with too many reports at the same time. I don't see any reason to take out the jitter code here, although it will not be as important as it was in the block report case. As far as I can tell, genstamp and block length should not be included in the cache report. They aren't included in the regular block report in StorageBlockReportProto. When asking a DataNode to lock a block, the NameNode can specify the genstamp and minimum length it wants at that time, and the DataNode can fail the request if it doesn't have that genstamp / length. This issue starts getting into the NN to DN communicaion (HDFS-5053). That's why I suggested discussing it there-- although I'm happy to discuss it here as well. Propagate cache status information from the DataNode to the NameNode Key: HDFS-5051 URL: https://issues.apache.org/jira/browse/HDFS-5051 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Colin Patrick McCabe Assignee: Andrew Wang Attachments: hdfs-5051-1.patch, hdfs-5051-2.patch The DataNode needs to inform the NameNode of its current cache state. Let's wire up the RPCs and stub out the relevant methods on the DN and NN side. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-5051) Propagate cache status information from the DataNode to the NameNode
[ https://issues.apache.org/jira/browse/HDFS-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13739707#comment-13739707 ] Suresh Srinivas commented on HDFS-5051: --- bq. The random jitter code was taken from the block report code. The goal is the same-- to avoid overloading the NameNode with too many reports at the same time. I don't see any reason to take out the jitter code here, although it will not be as important as it was in the block report case. Quoting my own question: bq. When a datanode starts, do we expect any thing to be in the cache at all? Hence the question why is jitter code is important? bq. They aren't included in the regular block report in StorageBlockReportProto. That is not correct. Please see the code in BlockListAsLongs. We need to decide the following (and I do not think with the current summary in HDFS-5053, that is the right place): # Do we need to include generation stamp and length? My early thought is, it may not be necessary. Current code includes both generation stamp and length. # When there are no cache entries in the datanode, my preference is not to send a cache report at all, including the first time datanode starts up. I agree that we could have incremental cache report. Propagate cache status information from the DataNode to the NameNode Key: HDFS-5051 URL: https://issues.apache.org/jira/browse/HDFS-5051 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Colin Patrick McCabe Assignee: Andrew Wang Attachments: hdfs-5051-1.patch, hdfs-5051-2.patch The DataNode needs to inform the NameNode of its current cache state. Let's wire up the RPCs and stub out the relevant methods on the DN and NN side. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-5051) Propagate cache status information from the DataNode to the NameNode
[
https://issues.apache.org/jira/browse/HDFS-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13740166#comment-13740166
]
Andrew Wang commented on HDFS-5051:
---
I included the gen stamp and length in the {{cacheReport}} to handle caching
newly appended data. I guess the gen stamp is unnecessary, but the DN isn't
going to automatically mlock newly appended data, so the NN needs to somehow
realize that the cached length is shorter than the new length and ask the DN to
recache at the new length. Alternatively, I guess the DN could automatically
mlock appended data, but there are quota implications there.
On startup, I agree that we can skip cache reports until the cache is
populated. I also agree that jittering doesn't matter as much if it's ticking
on such a short time scale. I guess I could have cleaned this up rather than
just changing the default cache report period like Colin asked.
However, since we want to eventually have both incremental and full reports,
let's just ape how block reports work; don't jitter the incremental reports,
but do jitter the start time for the full reports and afterwards tick at a
regular interval. Let's clean up all these issues in the incremental cache
report JIRA (HDFS-5092); if this sounds good, I'll edit the JIRA description
with these todo items.
Propagate cache status information from the DataNode to the NameNode
Key: HDFS-5051
URL: https://issues.apache.org/jira/browse/HDFS-5051
Project: Hadoop HDFS
Issue Type: Sub-task
Components: datanode, namenode
Reporter: Colin Patrick McCabe
Assignee: Andrew Wang
Attachments: hdfs-5051-1.patch, hdfs-5051-2.patch
The DataNode needs to inform the NameNode of its current cache state. Let's
wire up the RPCs and stub out the relevant methods on the DN and NN side.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-5051) Propagate cache status information from the DataNode to the NameNode
[ https://issues.apache.org/jira/browse/HDFS-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13740174#comment-13740174 ] Suresh Srinivas commented on HDFS-5051: --- bq. I included the gen stamp and length in the cacheReport to handle caching newly appended data. We need to specify what the cache behavior in this case is. My understanding was that for the first phase new data written will not be cached automatically. In fact any file that is being written to will not be cached until it is closed. Lets clearly define the behavior in these cases. Rest sounds good. Thank you [~andrew.wang] for comprehensive look at the comments. Propagate cache status information from the DataNode to the NameNode Key: HDFS-5051 URL: https://issues.apache.org/jira/browse/HDFS-5051 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Colin Patrick McCabe Assignee: Andrew Wang Attachments: hdfs-5051-1.patch, hdfs-5051-2.patch The DataNode needs to inform the NameNode of its current cache state. Let's wire up the RPCs and stub out the relevant methods on the DN and NN side. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-5051) Propagate cache status information from the DataNode to the NameNode
[ https://issues.apache.org/jira/browse/HDFS-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13740200#comment-13740200 ] Andrew Wang commented on HDFS-5051: --- Gotcha, makes sense. I definitely only wanted to address caching finalized blocks at first, but I was thinking about the case where an append+write+close would lead to a finalized block with a new longer length. Let's punt that out to an auto-caching subtask (will file). So, I'll remove the gen stamp and length in HDFS-5092; will edit it with this and the other todo items. Propagate cache status information from the DataNode to the NameNode Key: HDFS-5051 URL: https://issues.apache.org/jira/browse/HDFS-5051 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Colin Patrick McCabe Assignee: Andrew Wang Attachments: hdfs-5051-1.patch, hdfs-5051-2.patch The DataNode needs to inform the NameNode of its current cache state. Let's wire up the RPCs and stub out the relevant methods on the DN and NN side. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-5051) Propagate cache status information from the DataNode to the NameNode
[ https://issues.apache.org/jira/browse/HDFS-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13740555#comment-13740555 ] Hudson commented on HDFS-5051: -- SUCCESS: Integrated in Hadoop-trunk-Commit #4262 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/4262/]) HDFS-5051. nn fails to download checkpointed image from snn in some setups. Contributed by Vinay and Suresh Srinivas. (suresh: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1514110) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/GetImageServlet.java Propagate cache status information from the DataNode to the NameNode Key: HDFS-5051 URL: https://issues.apache.org/jira/browse/HDFS-5051 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Colin Patrick McCabe Assignee: Andrew Wang Attachments: hdfs-5051-1.patch, hdfs-5051-2.patch The DataNode needs to inform the NameNode of its current cache state. Let's wire up the RPCs and stub out the relevant methods on the DN and NN side. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-5051) Propagate cache status information from the DataNode to the NameNode
[ https://issues.apache.org/jira/browse/HDFS-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13738808#comment-13738808 ] Colin Patrick McCabe commented on HDFS-5051: +1 Propagate cache status information from the DataNode to the NameNode Key: HDFS-5051 URL: https://issues.apache.org/jira/browse/HDFS-5051 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Colin Patrick McCabe Assignee: Andrew Wang Attachments: hdfs-5051-1.patch, hdfs-5051-2.patch The DataNode needs to inform the NameNode of its current cache state. Let's wire up the RPCs and stub out the relevant methods on the DN and NN side. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-5051) Propagate cache status information from the DataNode to the NameNode
[ https://issues.apache.org/jira/browse/HDFS-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13738821#comment-13738821 ] Suresh Srinivas commented on HDFS-5051: --- Andrew, quick comment - do you expect datanode to send cache report every 10 seconds or only when datanode has cache enabled? Propagate cache status information from the DataNode to the NameNode Key: HDFS-5051 URL: https://issues.apache.org/jira/browse/HDFS-5051 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Colin Patrick McCabe Assignee: Andrew Wang Attachments: hdfs-5051-1.patch, hdfs-5051-2.patch The DataNode needs to inform the NameNode of its current cache state. Let's wire up the RPCs and stub out the relevant methods on the DN and NN side. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-5051) Propagate cache status information from the DataNode to the NameNode
[ https://issues.apache.org/jira/browse/HDFS-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13738835#comment-13738835 ] Colin Patrick McCabe commented on HDFS-5051: bq. Andrew, quick comment - do you expect datanode to send cache report every 10 seconds or only when datanode has cache enabled? Once we piggyback cache additions and removals on the existing DN heartbeats (see HDFS-5092), we only need to send full cache reports rarely-- maybe every few minutes. At that point, it should not be a huge issue to send an empty full cache report every 10 minutes if caching is disabled. If we want to optimize it further for the no-cache case, we can follow up with that later. Propagate cache status information from the DataNode to the NameNode Key: HDFS-5051 URL: https://issues.apache.org/jira/browse/HDFS-5051 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Colin Patrick McCabe Assignee: Andrew Wang Attachments: hdfs-5051-1.patch, hdfs-5051-2.patch The DataNode needs to inform the NameNode of its current cache state. Let's wire up the RPCs and stub out the relevant methods on the DN and NN side. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-5051) Propagate cache status information from the DataNode to the NameNode
[ https://issues.apache.org/jira/browse/HDFS-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13738841#comment-13738841 ] Suresh Srinivas commented on HDFS-5051: --- Some comments: # Nit: Document the @return DatanodeProtocol#cacheReport # Nit: used to communicated blocks - used to communicate blocks # In cache report, do we need to have block length and generation stamp or just the block ID suffices? # I see the code related to initial cache report after random jitter. When a datanode starts, do we expect any thing to be in the cache at all? Also 10 seconds cache report means, quite a lot of messages from datanode to namenode. Should we send cache report only if some thing in the cache changes, instead of blindly every 10 seconds? This could happen in a later patch/jira. Propagate cache status information from the DataNode to the NameNode Key: HDFS-5051 URL: https://issues.apache.org/jira/browse/HDFS-5051 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Colin Patrick McCabe Assignee: Andrew Wang Attachments: hdfs-5051-1.patch, hdfs-5051-2.patch The DataNode needs to inform the NameNode of its current cache state. Let's wire up the RPCs and stub out the relevant methods on the DN and NN side. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-5051) Propagate cache status information from the DataNode to the NameNode
[ https://issues.apache.org/jira/browse/HDFS-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13738949#comment-13738949 ] Colin Patrick McCabe commented on HDFS-5051: bq. Nit: Document the \@return DatanodeProtocol#cacheReport. Nit: used to communicated blocks - used to communicate blocks This JIRA is already committed (it's marked as fixed). Let's do this in HDFS-5053. I have pasted the comments there so we won't forget. bq. In cache report, do we need to have block length and generation stamp or just the block ID suffices? Let's address this later in HDFS-5053. bq. Should we send cache report only if some thing in the cache changes, instead of blindly every 10 seconds? This could happen in a later patch/jira. We filed HDFS-5092 for this. Propagate cache status information from the DataNode to the NameNode Key: HDFS-5051 URL: https://issues.apache.org/jira/browse/HDFS-5051 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Colin Patrick McCabe Assignee: Andrew Wang Attachments: hdfs-5051-1.patch, hdfs-5051-2.patch The DataNode needs to inform the NameNode of its current cache state. Let's wire up the RPCs and stub out the relevant methods on the DN and NN side. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-5051) Propagate cache status information from the DataNode to the NameNode
[ https://issues.apache.org/jira/browse/HDFS-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13739042#comment-13739042 ] Suresh Srinivas commented on HDFS-5051: --- Can you comment on this: bq. I see the code related to initial cache report after random jitter. When a datanode starts, do we expect any thing to be in the cache at all? Propagate cache status information from the DataNode to the NameNode Key: HDFS-5051 URL: https://issues.apache.org/jira/browse/HDFS-5051 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Colin Patrick McCabe Assignee: Andrew Wang Attachments: hdfs-5051-1.patch, hdfs-5051-2.patch The DataNode needs to inform the NameNode of its current cache state. Let's wire up the RPCs and stub out the relevant methods on the DN and NN side. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-5051) Propagate cache status information from the DataNode to the NameNode
[ https://issues.apache.org/jira/browse/HDFS-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13739045#comment-13739045 ] Suresh Srinivas commented on HDFS-5051: --- bq. In cache report, do we need to have block length and generation stamp or just the block ID suffices? I do not think this is captured in HDFS-5053. I also think that is not the right place to do this change. Lets discuss if generation stamp and block length is required in cache report in this jira. Based on the outcome we can create another jira to address it. Propagate cache status information from the DataNode to the NameNode Key: HDFS-5051 URL: https://issues.apache.org/jira/browse/HDFS-5051 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Colin Patrick McCabe Assignee: Andrew Wang Attachments: hdfs-5051-1.patch, hdfs-5051-2.patch The DataNode needs to inform the NameNode of its current cache state. Let's wire up the RPCs and stub out the relevant methods on the DN and NN side. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-5051) Propagate cache status information from the DataNode to the NameNode
[
https://issues.apache.org/jira/browse/HDFS-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13735587#comment-13735587
]
Colin Patrick McCabe commented on HDFS-5051:
{code}
+ public static final String DFS_CACHEREPORT_INTERVAL_MSEC_KEY =
dfs.cachereport.intervalMsec;
+ public static final longDFS_CACHEREPORT_INTERVAL_MSEC_DEFAULT = 60 * 60
* 1000;
{code}
I'm not sure a cache report every hour is going to be ideal.
{code}
+ArrayListLong blocksAsList = new ArrayListLong(blocks.length);
+for (int i=0; iblocks.length; i++) {
+ blocksAsList.add(blocks[i]);
+}
+builder.addAllBlocks(blocksAsList);
{code}
Let's avoid creating this ArrayList. We could potentially have a thousands of
blocks here, and duplicating the whole array is not necessary. You can just
use a for loop and {{builder.setBlock(index, blocks[i])}} or something like
that.
{code}
+ boolean resetCacheReportTime = true;
{code}
In block reports, this mechanism is used to inject some randomness into the
timing. Here, you never read from (as opposed to write to) this boolean, so
it's effectively unused. Since cache reports are going to be pretty frequent,
I think it would be easier to put the delay in nanoseconds and inject a random
amount of jitter in each rescheduling, than to use this boolean.
I realize the block report code makes this mistake, but you should not use
wall-clock time to calculate a duration. {{System#nanoTime}} is probably what
you want.
{code}
+ repeated int64 blocks = 3;
{code}
I think you need to add {{[packed=1]}} after this to get the more efficient
protobuf format.
Propagate cache status information from the DataNode to the NameNode
Key: HDFS-5051
URL: https://issues.apache.org/jira/browse/HDFS-5051
Project: Hadoop HDFS
Issue Type: Sub-task
Components: datanode, namenode
Reporter: Colin Patrick McCabe
Assignee: Andrew Wang
Attachments: hdfs-5051-1.patch
The DataNode needs to inform the NameNode of its current cache state. Let's
wire up the RPCs and stub out the relevant methods on the DN and NN side.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-5051) Propagate cache status information from the DataNode to the NameNode
[ https://issues.apache.org/jira/browse/HDFS-5051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13732737#comment-13732737 ] Andrew Wang commented on HDFS-5051: --- Attached patch does the protobuf mangling for the RPCs and sets up some basic stubs in the right places in the NN and DN. Propagate cache status information from the DataNode to the NameNode Key: HDFS-5051 URL: https://issues.apache.org/jira/browse/HDFS-5051 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Colin Patrick McCabe Assignee: Andrew Wang Attachments: hdfs-5051-1.patch The DataNode needs to inform the NameNode of its current cache state. Let's wire up the RPCs and stub out the relevant methods on the DN and NN side. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
