[
https://issues.apache.org/jira/browse/HDFS-12543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chen Liang updated HDFS-12543:
------------------------------
Attachment: HDFS-12543-HDFS-7240.010.patch
Thanks [~nandakumar131] for the review and the comments! Post v010 patch to
rebase. Also addressed [~nandakumar131]'s comments, all the other comments are
addressed.
bq. I understand the optimization done with key size, but if we are going to
remove it later why depend on it now?
In general case, client opens a key, then start writing to block. So in
original design, when a key is opened, a single "pre-allocated block" is also
allocated to it, such that client does not need to issue another allocate block
call after open key.
But turns out, the tricky part is that, it is not that clear how many of such
"pre-allocated" blocks should actually be allocated. It could be 0, when client
tries to write an empty data array (as in some test cases). It could be some X
> 1 blocks, if client already knows more than one will be written. So this is
size is used as a "hint" from client to tell how many such "pre-allocated"
blocks should be allocated. If client is about to write 0 length data, or
client does not know how much will be written, it sets the hint as 0 and no
"pre-allocated" happens at open key. This makes "size" only as an optimization
that is optional.
I would consider "pre-allocated" blocks as potentially very helpful
optimization. So I'm not entirely sure whether key size here is completely
redundant and should be removed. That's why I wanted to resolve this in another
JIRA, and follow up when we have a better idea of how useful this turns out to
be, or when we have a better approach to do this.
> Ozone : allow create key without specifying size
> ------------------------------------------------
>
> Key: HDFS-12543
> URL: https://issues.apache.org/jira/browse/HDFS-12543
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Chen Liang
> Assignee: Chen Liang
> Labels: ozoneMerge
> Attachments: HDFS-12543-HDFS-7240.001.patch,
> HDFS-12543-HDFS-7240.002.patch, HDFS-12543-HDFS-7240.003.patch,
> HDFS-12543-HDFS-7240.004.patch, HDFS-12543-HDFS-7240.005.patch,
> HDFS-12543-HDFS-7240.006.patch, HDFS-12543-HDFS-7240.007.patch,
> HDFS-12543-HDFS-7240.008.patch, HDFS-12543-HDFS-7240.009.patch,
> HDFS-12543-HDFS-7240.010.patch
>
>
> Currently when creating a key, it is required to specify the total size of
> the key. This makes it inconvenient for the case where a key is created and
> data keeps coming and being appended. This JIRA is remove the requirement of
> specifying the size on key creation, and allows appending to the key
> indefinitely.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
