[
https://issues.apache.org/jira/browse/HDFS-11886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16026564#comment-16026564
]
Anu Engineer commented on HDFS-11886:
-------------------------------------
[~vagarychen] Very important questions. Let me try to share the view from top
down of ozone.
bq. In short, this probably comes down to do we favor less RPC calls? or do we
favor reliable getKey at any time?
Another way to ask this same question would be, when does a *key appear in the
namespace* when a putKey call is made ?
In the current approach what we are suggesting is that a key under construction
is visible to other users. This is true in HDFS world but not true in S3.
Since Ozone is an object store, I am rethinking my position and I am thinking
that a key is visible to other users *if and only if* it is committed
successfully.
This means that we would need *two rpc calls* for a putKey -- The first one
that writes to a *ksm-keys-under-progress.db*, and then a commit call, where we
move the key to *ksm.db*. This way a key is visible for an external user if it
is completely written out.
This architecture makes it easy to implement multi-part file puts later.
[~xyao], [~szetszwo], [~cheersyang] Please share your thoughts on this. Do you
see any issues with -- visible after commit semantics for putKey., IMHO from a
coding and usage perspective this makes it easy for us at various levels.
> Ozone : improve error handling for putkey operation
> ---------------------------------------------------
>
> Key: HDFS-11886
> URL: https://issues.apache.org/jira/browse/HDFS-11886
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
> Reporter: Chen Liang
>
> Ozone's putKey operations involve a couple steps:
> 1. KSM calls allocateBlock to SCM, writes this info to KSM's local metastore
> 2. allocatedBlock gets returned to client, client checks to see if container
> needs to be created on datanode, if yes, create the container
> 3. writes the data to container.
> it is possible that 1 succeeded, but 2 or 3 failed, in this case there will
> be an entry in KSM's local metastore, but the key is actually nowhere to be
> found. We need to revert 1 is 2 or 3 failed in this case.
> To resolve this, we need at least two things to be implemented first.
> 1. We need deleteKey() to be added KSM first.
> 2. We also need container reports to be implemented first such that SCM can
> track whether the container is actually added.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
