[
https://issues.apache.org/jira/browse/IGNITE-26270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksandr Polovtsev reassigned IGNITE-26270:
--------------------------------------------
Assignee: Aleksandr Polovtsev
> Implement append and segment switching
> --------------------------------------
>
> Key: IGNITE-26270
> URL: https://issues.apache.org/jira/browse/IGNITE-26270
> Project: Ignite
> Issue Type: Improvement
> Reporter: Ivan Bessonov
> Assignee: Aleksandr Polovtsev
> Priority: Major
> Labels: ignite-3
>
> In this ticket we need to implement the basic log storage class with
> {{append}} function. The log will initially consist of a number of files with
> the following naming convention:
> {code:java}
> segment-N-G.bin
> {code}
> * N is a sequence number of the segment. Zero-based, fixed-width, encoded as
> decimal.
> * G is a generation number assigned to particular segment number. Also
> zero-based, fixed-width, encoded as decimal.
> The first ever file when we start fresh log storage will be called
> {{{}segment-00000000-00000000.bin{}}}. This way the lexicographic order will
> match the natural {{(N,G)}} ascending order.
> The file structure is the following:
> {code:java}
> +--------+---------+-----+---------+
> | Header | Payload | ... | Payload |
> +--------+---------+-----+---------+
> Header (8 bytes):
> +--------------+---------+
> | Magic number | Version |
> +--------------+---------+
> Payload (16+ bytes):
> +---------------+--------+-------+------+
> | Raft Group ID | Length | Bytes | Hash |
> +---------------+--------+-------+------+{code}
> * {{Magic number}} is for checking that this is the right file.
> * {{Version}} is for format extensibility.
> * {{Raft Group ID}} is an 8-bytes identifier, unique for each raft group.
> * {{Length}} is 4 bytes.
> * 32-bit hash covers everything starting with {{Raft Group ID}} and ending
> with the last byte of {{Bytes}} section.
> These files will have a fixed size in order to use {{{}mmap{}}}. Default size
> should be 1Gb. Max size is 4 Gb. We will write to the file using
> {{SegmentRingByteBuffer}} as {{IO}} (maybe with necessary refactorings).
> Hash algorithm could be anything, but it should be fast.
> Switch segment record will either be an 8-bytes payload filled with zeroes
> (raft group ID can't be 0), or any tail of the file if there's less than 8
> bytes left (no need for padding with zeroes then).
> While reaching the end of file, {{append}} operation should optionally write
> a switch segment record and atomically switch to next file. Thread-safe
> implementation is presumed.
> {{Append}} should only return execution to the caller when its bytes and all
> previous bytes have been written into segments, like in AI2.
> {{V1Encoder}} can already be utilized in API, but we may want to extend it or
> even duplicate.
> Unit tests (including concurrency tests) must be included in the
> implementation.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
