Hi Ivan, Should we update the write-throttling section on readme.io? It makes sense to do if any recommendations have changed: https://apacheignite.readme.io/docs/durable-memory-tuning#section-pages-writes-throttling
The compaction technique will be introduced and described on readme: https://issues.apache.org/jira/browse/IGNITE-8090 -- Denis On Fri, Mar 30, 2018 at 1:38 AM, Ivan Rakov <[email protected]> wrote: > Igniters, > > I've added some info about Pages Write Throttling and WAL compaction to > the "Ignite Persistent Store - under the hood" document. > Please let me know if anything is not clear enough. > > Best Regards, > Ivan Rakov > > > On 26.03.2018 0:18, Dmitry Pavlov wrote: > >> Hi Igniters, >> >> I've done several updates in section >> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+ >> Durable+Memory+-+under+the+hood#IgniteDurableMemory-under >> thehood-Regionandsegmentstructure >> >> These updates releated to optimizations of LoadedPageTable in Ignite 2.5 >> (Replacement of storage algorithm corresponding to the ID in the page, and >> the location of a physical address within the segment algorithm is used >> Robin Hood with backward shift. Previous implementation was not resistant >> to the ‘page rotation’, a lot of the cells were flooded with Removed >> state, >> and new algorithm does not have ‘removed’ state of cell.) >> >> New section has now picture of segments structure, and describes locking >> optimisations in 2.5. >> >> Sincerely, >> Dmitriy Pavlov >> >> чт, 19 окт. 2017 г. в 22:33, Dmitry Pavlov <[email protected]>: >> >> Hi Denis, >>> >>> I've updated the section in relation to your questions, please see >>> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+ >>> Durable+Memory+-+under+the+hood#IgniteDurableMemory-under >>> thehood-IgniteDurableMemory >>> >>> I’ve also moved paragraph related to page offset calculation and usage in >>> files into second article (PDS) >>> >>> Sincerely, >>> Dmitriy Pavlov >>> >>> пт, 1 сент. 2017 г. в 19:50, Denis Magda <[email protected]>: >>> >>> Dmitriy, >>>> >>> >>> First of all, thanks for putting this low level details on paper! Current >>>> and future community members will be grateful to you. >>>> >>>> Second, let me be a reviewer of the page and suggest some improvements. >>>> Until the review is over I would encourage us to hold on with any public >>>> announcements such as blog post. >>>> >>>> I’ve reviewed and edited Ignite Durable Memory section ( >>>> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+ >>>> Durable+Memory+-+under+the+hood#IgniteDurableMemory-under >>>> thehood-IgniteDurableMemory >>>> < >>>> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+ >>>> Durable+Memory+-+under+the+hood#IgniteDurableMemory-under >>>> thehood-IgniteDurableMemory>). >>>> Please address the questions below and I’ll move on with the new >>>> sections >>>> afterwards. >>>> >>>> 1) "Different caches and its partitions bring more complex addressing >>>> scheme but it is still possible to map from page ID to position in >>>> particular file: pageId => file + offset in this file”. >>>> >>>> Probably this appears too early in the doc because above we just >>>> introduced page ID notion with durable memory and now we’re talking >>>> about >>>> some files? Are these files of ignite persistence? What if the >>>> persistence >>>> is disabled? Honestly, the sentence gets out of the story context. >>>> >>>> >>>> 2) "Also, the tree pages can be merged into one if < 50% of space is >>>> used.” >>>> >>>> 50% of the whole memory available? Please clarify. >>>> >>>> >>>> 3) "There is one page ID table per memory segment.” >>>> >>>> It’s the same segment as it’s described in the main doc, right? >>>> https://apacheignite.readme.io/docs/memory-architecture#sect >>>> ion-memory-segment >>>> < >>>> https://apacheignite.readme.io/docs/memory-architecture#sect >>>> ion-memory-segment >>>> >>>> 4) "within unsafe segment” >>>> >>>> If we want to mention “usafe” here we need to explain what that is. >>>> >>>> 5) "If memory amount less than whole index size, Ignite still can >>>> operate.” >>>> >>>> How do we define “index size”? Is this the page size? >>>> >>>> >>>> — >>>> Denis >>>> >>>> >>>> On Aug 30, 2017, at 8:24 AM, Dmitry Pavlov <[email protected]> >>>>> >>>> wrote: >>>> >>>>> Hi Igniters, >>>>> >>>>> >>>>> I am glad to introduce two new wiki articles. These articles contain >>>>> high-level description of >>>>> >>>>> - Ignite Durable Memory ( >>>>> >>>>> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+ >>>> Durable+Memory+-+under+the+hood >>>> >>>>> ) >>>>> >>>>> - and Ignite Persistent Store ( >>>>> >>>>> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+ >>>> Persistent+Store+-+under+the+hood >>>> >>>>> ) >>>>> >>>>> These articles cover basic principles and internal design. Pages may be >>>>> useful for new contributors during start of working with persistent >>>>> >>>> store >>>> >>>>> code base. >>>>> >>>>> >>>>> I would like to thank all contributors who helped me with my questions. >>>>> Their answers and support made it possible to start this description. >>>>> >>>>> >>>>> I will appreciate any feedback and questions. Suggestions and edits are >>>>> welcome, of course. >>>>> >>>>> >>>>> Sincerely, >>>>> >>>>> Dmitriy Pavlov >>>>> >>>> >>>> >
