[
https://issues.apache.org/jira/browse/IGNITE-17232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Bessonov updated IGNITE-17232:
-----------------------------------
Description:
When creating *DeleteFilePageStore* at checkpoint, we sort the list of all
dirty pages of the partition by {*}pageIdx{*}, write to disk the sorted list of
*pageIdx* (for *pageId -> pageIdx* binary lookup), the contents of the dirty
pages, and the current *pageIdx* of the page allocations.
I propose to optimize this a bit.
In {*}DeleteFilePageStore{*}, store only changes in existing pages, and write
all new pages immediately to {*}FilePageStore{*}, so we will reduce the work
for the compacter (it will need to write less to the main partition file) and
the sorted list of *pageIdx* will be smaller.
Since the allocation index becomes logical (which is stored in the
{*}FilePageStore{*}) and depends on the first (newest)
{*}DeleteFilePageStore{*}, then if the checkpoint is not completed, we will not
lose or break anything in the *FilePageStore* and on the new checkpoint we will
write new pages on over of those that we write on the unfinished previous
checkpoint.
was:
When creating *DelateFilePageStore* at checkpoint, we sort the list of all
dirty pages of the partition by *pageIdx*, write to disk the sorted list of
*pageIdx* (for *pageId -> pageIdx* binary lookup), the contents of the dirty
pages, and the current *pageIdx* of the page allocations.
I propose to optimize this a bit.
In *DelateFilePageStore*, store only changes in existing pages, and write all
new pages immediately to *FilePageStore*, so we will reduce the work for the
compacter (it will need to write less to the main partition file) and the
sorted list of *pageIdx* will be smaller.
Since the allocation index becomes logical (which is stored in the
*FilePageStore*) and depends on the first (newest) *DelateFilePageStore*, then
if the checkpoint is not completed, we will not lose or break anything in the
*FilePageStore* and on the new checkpoint we will write new pages on over of
those that we write on the unfinished previous checkpoint.
> Optimization of DeltaFilePageStore: write new pages directly to FilePageStore
> -----------------------------------------------------------------------------
>
> Key: IGNITE-17232
> URL: https://issues.apache.org/jira/browse/IGNITE-17232
> Project: Ignite
> Issue Type: Improvement
> Reporter: Kirill Tkalenko
> Assignee: Philipp Shergalis
> Priority: Major
> Labels: ignite-3
> Fix For: 3.1
>
>
> When creating *DeleteFilePageStore* at checkpoint, we sort the list of all
> dirty pages of the partition by {*}pageIdx{*}, write to disk the sorted list
> of *pageIdx* (for *pageId -> pageIdx* binary lookup), the contents of the
> dirty pages, and the current *pageIdx* of the page allocations.
> I propose to optimize this a bit.
> In {*}DeleteFilePageStore{*}, store only changes in existing pages, and write
> all new pages immediately to {*}FilePageStore{*}, so we will reduce the work
> for the compacter (it will need to write less to the main partition file) and
> the sorted list of *pageIdx* will be smaller.
> Since the allocation index becomes logical (which is stored in the
> {*}FilePageStore{*}) and depends on the first (newest)
> {*}DeleteFilePageStore{*}, then if the checkpoint is not completed, we will
> not lose or break anything in the *FilePageStore* and on the new checkpoint
> we will write new pages on over of those that we write on the unfinished
> previous checkpoint.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
