[
https://issues.apache.org/jira/browse/HDFS-12000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chen Liang updated HDFS-12000:
------------------------------
Attachment: HDFS-12000-HDFS-7240.009.patch
Thanks [~xyao] for the review and the comments! Update v009 patch.
bq. I think it will work when we only have one version, i.e., latest version.
Correct me if I'm wrong, say we have K1 (B1V1, B2V2), with
getBlocksLatestVersionOnly, are we going to get B2V2 instead of (B1V1, B2V2)?
Exactly right, for now, getBlocksLatestVersionOnly() will only return the
blocks from the most recent version, B2V2 only in your case. In the next few
steps, my plan for multiple versions is to augment APIs that specifies which
historical version to read. For example, an API that specifies read(K1,
version=1), then it will ignore B2V2 but only look at B1V1.
bq. Line 173: NIT: more comment does not valid any more
Fixed
Additionally, I have one major right now, that I'm looking for advices, is that
I believe the dominating majority of read will be reading only the most recent
version, in this case always iterating all the blocks including old version can
become inefficient. Any comments on this are appreciated.
bq. Line 268: should this openVersion as part of the request so that the client
can request open certain version? It is ok to assume open the latest version
for now. Maybe add a TODO for next JIRA on this feature.
Yes, like mentioned above, in follow JIRA, there will be API that allows
request specific version, old or recent. It will likely be using this field.
Added a TODO note to the comments and follow up in next JIRAs.
bq. Line 111: Is there a reason why the KsyKeyLocationInfo#Builder does not
support setCreateVersion? Do we expect it to be set directly on the
KsyKeyLocationInfo afterwards?
I found that when a block is allocated, it gets allocated first then version is
set based on whether it's appending to current version, or added as a new
version. I think conceptually, a block itself does not have the notion of
version. So, yes, I leave the block builder not setting version but the caller
should set it after creating the block.
bq. Line 22: NIT: unused imports
Fixed
bq. Line 58: if the version starts from 0, the special handling for
currentVersion==-1 is not needed. Can you confirm?
You are right. Thanks for the catch. Fixed
bq. Line 30: can the open version be committed without close, something like
hsync to populate the write without closing file.
We don't need the open version, as it is only used when opening a key to
disambiguate preallocated blocks.
More specifically, when a key is opened, depending on whether size is
specified, KeyManager *may or may not* have pre-allocated some blocks and
returned in open session. If pre-allocation does happen, the returned latest
version is the version to write. But if pre-allocation did not happen, the
returned latest version is actually an old, already committed version that
should not be written. The only purpose of this open version field is to
distinguish these two cases. This value gets checked one time on client when
loading pre-allocated blocks, then never used. So I think we don't need to
commit it.
> Ozone: Container : Add key versioning support-1
> -----------------------------------------------
>
> Key: HDFS-12000
> URL: https://issues.apache.org/jira/browse/HDFS-12000
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
> Affects Versions: HDFS-7240
> Reporter: Anu Engineer
> Assignee: Chen Liang
> Labels: OzonePostMerge
> Attachments: HDFS-12000-HDFS-7240.001.patch,
> HDFS-12000-HDFS-7240.002.patch, HDFS-12000-HDFS-7240.003.patch,
> HDFS-12000-HDFS-7240.004.patch, HDFS-12000-HDFS-7240.005.patch,
> HDFS-12000-HDFS-7240.007.patch, HDFS-12000-HDFS-7240.008.patch,
> HDFS-12000-HDFS-7240.009.patch, OzoneVersion.001.pdf
>
>
> The rest interface of ozone supports versioning of keys. This support comes
> from the containers and how chunks are managed to support this feature. This
> JIRA tracks that feature. Will post a detailed design doc so that we can talk
> about this feature.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
