Igniters,
I've added some info about Pages Write Throttling and WAL compaction to
the "Ignite Persistent Store - under the hood" document.
Please let me know if anything is not clear enough.
Best Regards,
Ivan Rakov
On 26.03.2018 0:18, Dmitry Pavlov wrote:
Hi Igniters,
I've done several updates in section
https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Durable+Memory+-+under+the+hood#IgniteDurableMemory-underthehood-Regionandsegmentstructure
These updates releated to optimizations of LoadedPageTable in Ignite 2.5
(Replacement of storage algorithm corresponding to the ID in the page, and
the location of a physical address within the segment algorithm is used
Robin Hood with backward shift. Previous implementation was not resistant
to the ‘page rotation’, a lot of the cells were flooded with Removed state,
and new algorithm does not have ‘removed’ state of cell.)
New section has now picture of segments structure, and describes locking
optimisations in 2.5.
Sincerely,
Dmitriy Pavlov
чт, 19 окт. 2017 г. в 22:33, Dmitry Pavlov <[email protected]>:
Hi Denis,
I've updated the section in relation to your questions, please see
https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Durable+Memory+-+under+the+hood#IgniteDurableMemory-underthehood-IgniteDurableMemory
I’ve also moved paragraph related to page offset calculation and usage in
files into second article (PDS)
Sincerely,
Dmitriy Pavlov
пт, 1 сент. 2017 г. в 19:50, Denis Magda <[email protected]>:
Dmitriy,
First of all, thanks for putting this low level details on paper! Current
and future community members will be grateful to you.
Second, let me be a reviewer of the page and suggest some improvements.
Until the review is over I would encourage us to hold on with any public
announcements such as blog post.
I’ve reviewed and edited Ignite Durable Memory section (
https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Durable+Memory+-+under+the+hood#IgniteDurableMemory-underthehood-IgniteDurableMemory
<
https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Durable+Memory+-+under+the+hood#IgniteDurableMemory-underthehood-IgniteDurableMemory>).
Please address the questions below and I’ll move on with the new sections
afterwards.
1) "Different caches and its partitions bring more complex addressing
scheme but it is still possible to map from page ID to position in
particular file: pageId => file + offset in this file”.
Probably this appears too early in the doc because above we just
introduced page ID notion with durable memory and now we’re talking about
some files? Are these files of ignite persistence? What if the persistence
is disabled? Honestly, the sentence gets out of the story context.
2) "Also, the tree pages can be merged into one if < 50% of space is
used.”
50% of the whole memory available? Please clarify.
3) "There is one page ID table per memory segment.”
It’s the same segment as it’s described in the main doc, right?
https://apacheignite.readme.io/docs/memory-architecture#section-memory-segment
<
https://apacheignite.readme.io/docs/memory-architecture#section-memory-segment
4) "within unsafe segment”
If we want to mention “usafe” here we need to explain what that is.
5) "If memory amount less than whole index size, Ignite still can
operate.”
How do we define “index size”? Is this the page size?
—
Denis
On Aug 30, 2017, at 8:24 AM, Dmitry Pavlov <[email protected]>
wrote:
Hi Igniters,
I am glad to introduce two new wiki articles. These articles contain
high-level description of
- Ignite Durable Memory (
https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Durable+Memory+-+under+the+hood
)
- and Ignite Persistent Store (
https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood
)
These articles cover basic principles and internal design. Pages may be
useful for new contributors during start of working with persistent
store
code base.
I would like to thank all contributors who helped me with my questions.
Their answers and support made it possible to start this description.
I will appreciate any feedback and questions. Suggestions and edits are
welcome, of course.
Sincerely,
Dmitriy Pavlov
