[
https://issues.apache.org/jira/browse/HDFS-11886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16030685#comment-16030685
]
Anu Engineer commented on HDFS-11886:
-------------------------------------
bq. do we really need to persist this? Can we store the state in memory only?
Only if all succeed, commit this to ksm.db, otherwise dispose it.
I am fine with doing that and It does make for a simpler system. Thanks for
pointing it out. There are 2 things that we must keep in mind.
# In the long run, we will have KSM HA, that is KSM state will be replicated
via RAFT(Ratis). This means that you have to write this data to a log so if the
active KSM dies, a follower is able to pick up the state.
# Again in future, we will have to support an API which allows end users to put
single files that is larger than 5 GB. This will be achieved via putting
multiple files into the system and using an API to connect these independent
uploads into a single file. While these independent uploads are going on,
saving that state might be useful.
But as I said, in the short term it is ok to do what you are suggesting, but in
the long run we will have to persist this data. Please note that we have 1 I/O
for a full block write, assuming a block a size of 256 MB, KSM is doing one
local I/O for each 256 MB * 3 I/O on datanodes. Yes, it can be expensive, but
nothing more than what HDFS does today.
bq. Another question: why we need to return a flag to OzoneHandler to determine
if a container needs to be created? I
We have debated this a lot and gone both ways, originally we had SCM talk to
datanodes to create the containers. Then we realized that reason about the
state of SCM <->Datanode is more complex if SCM is actively connecting to
datanodes. Hence we borrowed the model that is used in HDFS.
That is datanodes talk to SCM via heartbeats and all commands that SCM wants to
send is done via HB response. The client who wants to write to a block ends up
creating a block in HDFS world, so in ozone world, the ozone handler acts as
the DFSclient of HDFS world.
bq. ksm-keys-under-progress.db to ksm.db. This introduces more times of writes
to disk
Agree, in fact, [~xyao] suggested that we use the same database with a flag.
This has the advantage the we use a LSM tree (levelDB) where these writes are
optimized.
> Ozone : improve error handling for putkey operation
> ---------------------------------------------------
>
> Key: HDFS-11886
> URL: https://issues.apache.org/jira/browse/HDFS-11886
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
> Reporter: Chen Liang
> Attachments: design-notes-putkey.pdf
>
>
> Ozone's putKey operations involve a couple steps:
> 1. KSM calls allocateBlock to SCM, writes this info to KSM's local metastore
> 2. allocatedBlock gets returned to client, client checks to see if container
> needs to be created on datanode, if yes, create the container
> 3. writes the data to container.
> it is possible that 1 succeeded, but 2 or 3 failed, in this case there will
> be an entry in KSM's local metastore, but the key is actually nowhere to be
> found. We need to revert 1 is 2 or 3 failed in this case.
> To resolve this, we need at least two things to be implemented first.
> 1. We need deleteKey() to be added KSM first.
> 2. We also need container reports to be implemented first such that SCM can
> track whether the container is actually added.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
