[
https://issues.apache.org/jira/browse/HADOOP-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12528580
]
Raghu Angadi commented on HADOOP-1908:
--------------------------------------
The overall structure looks good. A few comments and a few corner cases :
# Optional startOffset for OP_READ_BLOCK. Is there any use of making it
optional? (See the first comment).
# Extra closeStream() calls are not necessary (set the above comment).
# BlockSender needs a close() method that closes any files that were open. If
there is an exception around DataNode.java:1339, this could leak file
descriptors. Also this close() method can used inside BlockSender's 'finally'
blocks as well.
# Same for BlockReceiver
# in writeBlock(), failure to read status from mirror results in this datanode
reporting error to source. This is not expected. It should return success as
long as local writeBlock() is successful.
# In BlockSender constructor, the initiall 'XXX' comment and the following line
should be removed since they are not relevant.
# The following code prints a success log which could be followed by a failure
log : {code}
blockSender.sendBlock(out, null);
//REVIEW we are printing this even though it could be an error.
LOG.info("Transmitted block " + b + " to " + curTarget);
DataInputStream in = new DataInputStream(new BufferedInputStream(
sock.getInputStream(), BUFFER_SIZE));
short opStatus = in.readShort();
if(opStatus != FSConstants.OP_STATUS_SUCCESS ) {
LOG.info("Block " +b + " transfer failed.");
}
{code} You could just use the warning in following catch {}.
# Another log related : Inside writeBlock() we try to write status back to
remote side even if there were read errors from the remote side, which will
again fail resulting in a spurious error message.
> Restructure data node code so that block sending/receiving is seperated from
> data transfer header handling
> ----------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-1908
> URL: https://issues.apache.org/jira/browse/HADOOP-1908
> Project: Hadoop
> Issue Type: Improvement
> Components: dfs
> Affects Versions: 0.14.1
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.15.0
>
> Attachments: datanode.patch
>
>
> This jira is intended for code sharing. I'd like to have a BlockSender which
> is resposible for reading a block from a disk and writing it to an output
> stream, and a BlockReceiver which receives a block from an input stream,
> writes it to a disk, and possibly writes it a mirror ouput stream to support
> pipeline writes. Block sender and receiver code is independent of any block
> IO protocols so it could be shared by block transfer, block write, block
> read, and future block replacement introduced by data node rebalancing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
