[
https://issues.apache.org/jira/browse/HDDS-284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692490#comment-16692490
]
Hanisha Koneru commented on HDDS-284:
-------------------------------------
Thank you for the review [~shashikant].
{quote} 2. With the patch it aways seems to be computing the checksum in
writeChunkToContainerCall. With HTTP headers, if the checksum is already
available in a Rest call, we might not require to recompute again. Are we going
to address such cases later?{quote}
We can add support for this later. If the checksum is already provided in the
HTTP header, we can use that and skip the computation. Let's open a new Jira to
track this?
{quote} 3. ChunkManagerImpl#writeChunk:-> while handling the overWrites of a
chunkFile we can just verify the checksum if its already present and return
accordingly without actually doing I/O ( addressed as TODO in the code). We can
also add the checksum verification here, though these can be addressed in a
separate patch as well.{quote}
Yes, let's address this also in a separate patch.
{quote} 4. ChunkInputStream.java : L213-215 : why is this change specifically
required? Is it just for making the tests added to work?{quote}
Added this to propagate the actual exception to the client. Otherwise, we just
get "Unexpected OzoneException" without specifying the actual reason for the
failure.
> CRC for ChunksData
> ------------------
>
> Key: HDDS-284
> URL: https://issues.apache.org/jira/browse/HDDS-284
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Reporter: Bharat Viswanadham
> Assignee: Hanisha Koneru
> Priority: Major
> Attachments: CRC and Error Detection for Containers.pdf,
> HDDS-284.00.patch, HDDS-284.005.patch, HDDS-284.01.patch, HDDS-284.02.patch,
> HDDS-284.03.patch, HDDS-284.04.patch, Interleaving CRC and Error Detection
> for Containers.pdf
>
>
> This Jira is to add CRC for chunks data.
> Right now a Chunk Info structure looks like this:
> _message ChunkInfo {_
> _required string chunkName =_ _1__;_
> _required uint64 offset =_ _2__;_
> _required uint64 len =_ _3__;_
> _optional string checksum =_ _4__;_
> _repeated KeyValue metadata =_ _5__;_
> _}_
> _Proposal is to change ChunkInfo structure as below:_
> _message ChunkInfo {_
> _required string chunkName =_ _1__;_
> _required uint64 offset =_ _2__;_
> _required uint64 len =_ _3__;_
> _optional bytes checksum =_ _4__;_
> _optional CRCType checksumType =_ _5__;_
> _optional string legacyMetadata =_ _6__;_
> _optional string legacyData =_ _7__;_
> _repeated KeyValue metadata =_ _8__;_
> _}_
> _Instead of changing disk format, we put the checksum, checksumtype and
> legacy data fields in to chunkInfo._
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
