[
https://issues.apache.org/jira/browse/HDFS-11780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16025228#comment-16025228
]
Chen Liang commented on HDFS-11780:
-----------------------------------
Thanks [~cheersyang] for the comments!
bq. why KsmKeyBlock has only a containerName but not a PipeLine
The reason is that, by design, maintaining the mapping from container name to
actual container instance should be the responsibility of SCM, while KSM should
not be supposed to understand the internal of container implementation. Also,
Another important thing is that, {{KsmKeyBlock}} (now KsmKeyInfo) is written to
ksm.db for getKey(), keeping one {{Pipeline}} instance for every single key can
be too expensive, when storing just the container name is sufficient already.
Nonetheless, a downside of storing just the name rather than Pipeline is that,
client side needs to make another call to SCM to look up Pipeline given
container name. I think this can be optimized by changing XceiverClientManager
to cache container connections by containerName instead of Pipeline. Will
probably follow up in another JIRA.
bq. And I don't understand the comment...
Sorry for the confusion, will elaborate in next path... What I was trying to
say is that this call to KSM is only making sure a block is allocated, but
putKey is not complete yet. After this allocated block gets returned to client
side, client side {{DistributedStorageHandler}} will make another call to
datanode to actually writes the data. So KSM is in some sense making the
assumption that {{DistributedStorageHandler}} will complete the writes
successfully. In the case that the block is allocated on KSM and returned to
{{DistributedStorageHandler}}, but {{DistributedStorageHandler}} failed the
following writes for any reason, we may end up having an entry in ksm.db, but
no such key in that container.
(Also I thought allocateKeyBlock has been changed to allocateKey, but seems
it's not... will update also)
> Ozone: KSM : Add putKey
> -----------------------
>
> Key: HDFS-11780
> URL: https://issues.apache.org/jira/browse/HDFS-11780
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
> Reporter: Anu Engineer
> Assignee: Chen Liang
> Attachments: HDFS-11780-HDFS-7240.001.patch,
> HDFS-11780-HDFS-7240.002.patch, HDFS-11780-HDFS-7240.003.patch,
> HDFS-11780-HDFS-7240.004.patch
>
>
> Support putting a key into an Ozone bucket.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
