[
https://issues.apache.org/jira/browse/IGNITE-15818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Bessonov updated IGNITE-15818:
-----------------------------------
Description:
h2. Goal
Port and refactor core classes implementing page-based persistent store in
Ignite 2.x: GridCacheOffheapManager, GridCacheDatabaseSharedManager,
PageMemoryImpl, Checkpointer, FileWriteAheadLogManager.
New checkpoint implementation to avoid excessive logging.
Store lifecycle clarification to avoid complicated and invasive code of custom
lifecycle managed mostly by DatabaseSharedManager.
h2. Items to pay attention to
New checkpoint implementation based on split-file storage, new page index
structure to maintain disk-memory page mapping.
File page store implementation should be extracted from GridCacheOffheapManager
to a separate entity, target implementation should support new version of
checkpoint (split-file store to enable always-consistent store and to eliminate
binary recovery phase).
Support of big pages (256+ kB).
Support of throttling algorithms.
h2. References
New checkpoint design overview is available
[here|https://github.com/apache/ignite-3/blob/ignite-14647/modules/vault/README.md]
h2. Thoughts
On some occasions, physical WAL records are the only way to debug data
corruption. Having new persistence without binary log is unacceptable. There
are at least two choices:
* use WAL for binary recovery, as before. This eliminates
* don't use WAL for recovery and enable it by premise.
Or, log binary data in different format. But that's too radical, there are
already tools for reading WAL that can be adopted as well.
was:
h2. Goal
Port and refactor core classes implementing page-based persistent store in
Ignite 2.x: GridCacheOffheapManager, GridCacheDatabaseSharedManager,
PageMemoryImpl, Checkpointer, FileWriteAheadLogManager.
New checkpoint implementation to avoid excessive logging.
Store lifecycle clarification to avoid complicated and invasive code of custom
lifecycle managed mostly by DatabaseSharedManager.
h2. Items to pay attention to
New checkpoint implementation based on split-file storage, new page index
structure to maintain disk-memory page mapping.
File page store implementation should be extracted from GridCacheOffheapManager
to a separate entity, target implementation should support new version of
checkpoint (split-file store to enable always-consistent store and to eliminate
binary recovery phase).
Support of big pages (256+ kB).
Support of throttling algorithms.
h2. References
New checkpoint design overview is available
[here|https://github.com/apache/ignite-3/blob/ignite-14647/modules/vault/README.md]
> [Native Persistence 3.0] Checkpoint, lifecycle and file store refactoring and
> re-implementation
> -----------------------------------------------------------------------------------------------
>
> Key: IGNITE-15818
> URL: https://issues.apache.org/jira/browse/IGNITE-15818
> Project: Ignite
> Issue Type: Task
> Reporter: Sergey Chugunov
> Priority: Major
> Labels: ignite-3
>
> h2. Goal
> Port and refactor core classes implementing page-based persistent store in
> Ignite 2.x: GridCacheOffheapManager, GridCacheDatabaseSharedManager,
> PageMemoryImpl, Checkpointer, FileWriteAheadLogManager.
> New checkpoint implementation to avoid excessive logging.
> Store lifecycle clarification to avoid complicated and invasive code of
> custom lifecycle managed mostly by DatabaseSharedManager.
> h2. Items to pay attention to
> New checkpoint implementation based on split-file storage, new page index
> structure to maintain disk-memory page mapping.
> File page store implementation should be extracted from
> GridCacheOffheapManager to a separate entity, target implementation should
> support new version of checkpoint (split-file store to enable
> always-consistent store and to eliminate binary recovery phase).
> Support of big pages (256+ kB).
> Support of throttling algorithms.
> h2. References
> New checkpoint design overview is available
> [here|https://github.com/apache/ignite-3/blob/ignite-14647/modules/vault/README.md]
> h2. Thoughts
> On some occasions, physical WAL records are the only way to debug data
> corruption. Having new persistence without binary log is unacceptable. There
> are at least two choices:
> * use WAL for binary recovery, as before. This eliminates
> * don't use WAL for recovery and enable it by premise.
> Or, log binary data in different format. But that's too radical, there are
> already tools for reading WAL that can be adopted as well.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
