[jira] [Commented] (HDFS-11997) ChunkManager functions do not use the argument keyName
[ https://issues.apache.org/jira/browse/HDFS-11997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16054822#comment-16054822 ] Anu Engineer commented on HDFS-11997: - [~vagarychen] Thanks for your comments. You are absolutely correct. Chunk Manager never really needs to know which key it is part of. However, some time ago we did a prototype for Disaster Recovery (DR) using the containers. In that prototype we explored writing the chunks directly from the datanode to S3 buckets. If my memory serves me correct, that is when we added this. Since having the context made it easier to write the chunk to the remote cluster. If we decide to do that in future this might be useful. You are absolutely right from a layering perspective, this does not make sense -- A chunk never needs to know the parent context. This was only for propagating that information over to another cluster. Right now, we have no strong use case for this and DR has to be re-developed anyway (probably will not need this info for that). So this is prototyping residue that can indeed be removed. Please feel free to remove it. Completely your call. Thanks > ChunkManager functions do not use the argument keyName > -- > > Key: HDFS-11997 > URL: https://issues.apache.org/jira/browse/HDFS-11997 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Chen Liang >Assignee: Chen Liang > > {{ChunkManagerImpl}}'s functions i.e. {{writeChunk}} {{readChunk}} > {{deleteChunk}} all take a {{keyName}} argument, which is not being used by > any of them. > I think this makes sense because conceptually {{ChunkManager}} should not > have to know keyName to do anything, probably except for some sort of sanity > check or logging, which is not there either. We should revisit whether we > need it here. I think we should remove it to make the Chunk syntax, and the > function signatures more cleanly abstracted. > Any comments? [~anu] -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11997) ChunkManager functions do not use the argument keyName
[ https://issues.apache.org/jira/browse/HDFS-11997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16054779#comment-16054779 ] Chen Liang commented on HDFS-11997: --- I think from the perspective of abstraction, {{ChunkManager}} should work (read/write/delete a chunk) given just the metadata of the chunk. This is not causing any issue for now and will more likely never, but I felt having this field but not being used causes confusions. I simply didn't see any cases where this field should be used by ChunkManager as part of any of the operations. In fact, an implementation of chunk manager that relies on key name seems breaking the abstraction in some way to me... > ChunkManager functions do not use the argument keyName > -- > > Key: HDFS-11997 > URL: https://issues.apache.org/jira/browse/HDFS-11997 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Chen Liang >Assignee: Chen Liang > > {{ChunkManagerImpl}}'s functions i.e. {{writeChunk}} {{readChunk}} > {{deleteChunk}} all take a {{keyName}} argument, which is not being used by > any of them. > I think this makes sense because conceptually {{ChunkManager}} should not > have to know keyName to do anything, probably except for some sort of sanity > check or logging, which is not there either. We should revisit whether we > need it here. I think we should remove it to make the Chunk syntax, and the > function signatures more cleanly abstracted. > Any comments? [~anu] -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11997) ChunkManager functions do not use the argument keyName
[ https://issues.apache.org/jira/browse/HDFS-11997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16054754#comment-16054754 ] Anu Engineer commented on HDFS-11997: - As an API for chunk calls, isn't it better to have these in the API ?. This particular implementation of containers is not using it. But many others might find it useful. If it is just part of signature, I don't know if it is causing any issue. > ChunkManager functions do not use the argument keyName > -- > > Key: HDFS-11997 > URL: https://issues.apache.org/jira/browse/HDFS-11997 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Chen Liang >Assignee: Chen Liang > > {{ChunkManagerImpl}}'s functions i.e. {{writeChunk}} {{readChunk}} > {{deleteChunk}} all take a {{keyName}} argument, which is not being used by > any of them. > I think this makes sense because conceptually {{ChunkManager}} should not > have to know keyName to do anything, probably except for some sort of sanity > check or logging, which is not there either. We should revisit whether we > need it here. I think we should remove it to make the Chunk syntax, and the > function signatures more cleanly abstracted. > Any comments? [~anu] -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org