[
https://issues.apache.org/jira/browse/HDFS-12283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136363#comment-16136363
]
Yuanbo Liu commented on HDFS-12283:
-----------------------------------
[~cheersyang] Thanks for your comments.
{quote}
ozone-default.xml
{quote}
addressed
{quote}
DeletedBlockLog.java
{quote}
addressed
{quote}
can we move line 67 - 71
{quote}
Those fields are designed to be kept in memory as global variables.
{quote}
line 64: can we use AtomicLong
{quote}
Since we change the value in a locked field, so I guess we don't have to use
AtomicLong here.
{quote}
exception handling of addTransaction, what if deletedStore.writeBatch(batch)
{quote}
The situation is much worse if we write batch before updating latestTxid, it
would lead to repeated txid, so we update latestTxid before writing batch, in
this case, txid may be discontinuous, but it's acceptable.
Other comments will be addressed in the new patch, thanks again for your review.
> Ozone: DeleteKey-5: Implement SCM DeletedBlockLog
> -------------------------------------------------
>
> Key: HDFS-12283
> URL: https://issues.apache.org/jira/browse/HDFS-12283
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone, scm
> Reporter: Weiwei Yang
> Assignee: Yuanbo Liu
> Attachments: HDFS-12283.001.patch, HDFS-12283-HDFS-7240.001.patch,
> HDFS-12283-HDFS-7240.002.patch, HDFS-12283-HDFS-7240.003.patch,
> HDFS-12283-HDFS-7240.004.patch, HDFS-12283-HDFS-7240.005.patch,
> HDFS-12283-HDFS-7240.006.patch
>
>
> The DeletedBlockLog is a persisted log in SCM to keep tracking container
> blocks which are under deletion. It maintains info about under-deletion
> container blocks that notified by KSM, and the state how it is processed. We
> can use RocksDB to implement the 1st version of the log, the schema looks like
> ||TxID||ContainerName||Block List||ProcessedCount||
> |0|c1|b1,b2,b3|0|
> |1|c2|b1|3|
> |2|c2|b2, b3|-1|
> Some explanations
> # TxID is an incremental long value transaction ID for ONE container and
> multiple blocks
> # Container name is the name of the container
> # Block list is a list of block IDs
> # ProcessedCount is the number of times SCM has sent this record to datanode,
> it represents the "state" of the transaction, it is in range of \[-1, 5\], -1
> means the transaction eventually failed after some retries, 5 is the max
> number times of retries.
> We need to define {{DeletedBlockLog}} as an interface and implement this with
> RocksDB {{MetadataStore}} as the first version.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
