[
https://issues.apache.org/jira/browse/HDDS-14185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Swaminathan Balachandran updated HDDS-14185:
--------------------------------------------
Description:
HDDS-14027 introduced usage of CodecBuffers in creation of RDBSstFileWriter.
However when adding a tombstone entry in the SST file a ManagedDirectSlice is
used because of unavailability of ByteBuffer direct API. The usage of
DirectSlice differs where the rocksdb doesn't read the position and limit of
the passed ByteBuffer instance and always performs on the entire ByteBuffer by
default. This can have wrong results in flushing the sst file since the key
written would be wrong.
Consider this case:
CodecBuffer initialized with Key_Renamed entry in the byte Array [pos: 0, limit
: 12]
CodecBuffer is cleared and entry Key entry in the byte Array [pos: 0, limit: 3]
Write this value as delete entry in sst file. Here with the existing
implementation Key_Renamed would be written into sst file instead of Key which
is wrong because of the ManagedDirectSlice misuse.
CodecBuffer is cleared and entry Key_Renamed entry is added.[pos: 0, limit : 12]
Adding this entry to sst file will fail since it has the value needs to
increase monotonically since Key_Renamed was already added erroneously thus
fetching wrong results.
The patch focuses on making ManagedDirectSlice a ManagedRocksObject so that it
becomes easier to use it as mere wrapper for ByteBuffer making it easy to use
with CodecBuffers.
Rocksdb Source Code for reference:
[https://github.com/facebook/rocksdb/blob/06c8afeff5b9fd38a79bdd4ba1bbb9df572c8096/java/rocksjni/slice.cc#L292-L308]
[https://github.com/facebook/rocksdb/blob/1001bc01c9ea06785c2d5261ca9fe7ac0eac0625/java/src/main/java/org/rocksdb/DirectSlice.java#L70-L80]
was:
HDDS-14027 introduced usage of CodecBuffers in creation of RDBSstFileWriter.
However when adding a tombstone entry in the SST file a ManagedDirectSlice is
used because of unavailability of ByteBuffer direct API. The usage of
DirectSlice differs where the rocksdb doesn't read the position and limit of
the passed ByteBuffer instance and always performs on the entire ByteBuffer by
default. This can have wrong results in flushing the sst file since the key
written would be wrong.
Consider this case:
CodecBuffer initialized with Key_Renamed entry in the byte Array [pos: 0, limit
: 12]
CodecBuffer is cleared and entry Key entry in the byte Array [pos: 0, limit: 3]
Write this value as delete entry in sst file. Here with the existing
implementation Key_Renamed would be written into sst file instead of Key which
is wrong because of the ManagedDirectSlice misuse.
CodecBuffer is cleared and entry Key_Renamed entry is added.[pos: 0, limit : 12]
Adding this entry to sst file will fail since it has the value needs to
increase monotonically since Key_Renamed was already added erroneously thus
fetching wrong results.
The patch focuses on making ManagedDirectSlice a ManagedRocksObject so that it
becomes easier to use it as mere wrapper for ByteBuffer making it easy to use
with CodecBuffers.
> Fix ManagedDirectSlice usage as wrapper for ByteBuffer class
> ------------------------------------------------------------
>
> Key: HDDS-14185
> URL: https://issues.apache.org/jira/browse/HDDS-14185
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Swaminathan Balachandran
> Assignee: Swaminathan Balachandran
> Priority: Major
>
> HDDS-14027 introduced usage of CodecBuffers in creation of RDBSstFileWriter.
> However when adding a tombstone entry in the SST file a ManagedDirectSlice is
> used because of unavailability of ByteBuffer direct API. The usage of
> DirectSlice differs where the rocksdb doesn't read the position and limit of
> the passed ByteBuffer instance and always performs on the entire ByteBuffer
> by default. This can have wrong results in flushing the sst file since the
> key written would be wrong.
> Consider this case:
> CodecBuffer initialized with Key_Renamed entry in the byte Array [pos: 0,
> limit : 12]
> CodecBuffer is cleared and entry Key entry in the byte Array [pos: 0, limit:
> 3]
> Write this value as delete entry in sst file. Here with the existing
> implementation Key_Renamed would be written into sst file instead of Key
> which is wrong because of the ManagedDirectSlice misuse.
> CodecBuffer is cleared and entry Key_Renamed entry is added.[pos: 0, limit :
> 12]
> Adding this entry to sst file will fail since it has the value needs to
> increase monotonically since Key_Renamed was already added erroneously thus
> fetching wrong results.
> The patch focuses on making ManagedDirectSlice a ManagedRocksObject so that
> it becomes easier to use it as mere wrapper for ByteBuffer making it easy to
> use with CodecBuffers.
> Rocksdb Source Code for reference:
> [https://github.com/facebook/rocksdb/blob/06c8afeff5b9fd38a79bdd4ba1bbb9df572c8096/java/rocksjni/slice.cc#L292-L308]
> [https://github.com/facebook/rocksdb/blob/1001bc01c9ea06785c2d5261ca9fe7ac0eac0625/java/src/main/java/org/rocksdb/DirectSlice.java#L70-L80]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
