Ivan, Sure, not a problem. Did more, created a hard-not-to-notice block titled Pages Write Throttling in a Nutshell.
Thanks! Denis On Fri, Mar 30, 2018 at 10:00 AM, Ivan Rakov <[email protected]> wrote: > Denis, thanks for pointing this out. > > I think that section on readme.io is good enough. > I'd only recommend to highlight "write throttling algorithm" words with > link to "Pages Write Throttling" section of "Persistence under the hood". > What's already described on readme.io is enough for "Durable Memory > Tuning" page, but if advanced user wants to know details about throttling, > he'll be navigated to page with internals. > > Best Regards, > Ivan Rakov > > > On 30.03.2018 19:48, Denis Magda wrote: > >> Hi Ivan, >> >> Should we update the write-throttling section on readme.io? It makes >> sense >> to do if any recommendations have changed: >> https://apacheignite.readme.io/docs/durable-memory-tuning#se >> ction-pages-writes-throttling >> >> The compaction technique will be introduced and described on readme: >> https://issues.apache.org/jira/browse/IGNITE-8090 >> >> -- >> Denis >> >> On Fri, Mar 30, 2018 at 1:38 AM, Ivan Rakov <[email protected]> >> wrote: >> >> Igniters, >>> >>> I've added some info about Pages Write Throttling and WAL compaction to >>> the "Ignite Persistent Store - under the hood" document. >>> Please let me know if anything is not clear enough. >>> >>> Best Regards, >>> Ivan Rakov >>> >>> >>> On 26.03.2018 0:18, Dmitry Pavlov wrote: >>> >>> Hi Igniters, >>>> >>>> I've done several updates in section >>>> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+ >>>> Durable+Memory+-+under+the+hood#IgniteDurableMemory-under >>>> thehood-Regionandsegmentstructure >>>> >>>> These updates releated to optimizations of LoadedPageTable in Ignite 2.5 >>>> (Replacement of storage algorithm corresponding to the ID in the page, >>>> and >>>> the location of a physical address within the segment algorithm is used >>>> Robin Hood with backward shift. Previous implementation was not >>>> resistant >>>> to the ‘page rotation’, a lot of the cells were flooded with Removed >>>> state, >>>> and new algorithm does not have ‘removed’ state of cell.) >>>> >>>> New section has now picture of segments structure, and describes locking >>>> optimisations in 2.5. >>>> >>>> Sincerely, >>>> Dmitriy Pavlov >>>> >>>> чт, 19 окт. 2017 г. в 22:33, Dmitry Pavlov <[email protected]>: >>>> >>>> Hi Denis, >>>> >>>>> I've updated the section in relation to your questions, please see >>>>> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+ >>>>> Durable+Memory+-+under+the+hood#IgniteDurableMemory-under >>>>> thehood-IgniteDurableMemory >>>>> >>>>> I’ve also moved paragraph related to page offset calculation and usage >>>>> in >>>>> files into second article (PDS) >>>>> >>>>> Sincerely, >>>>> Dmitriy Pavlov >>>>> >>>>> пт, 1 сент. 2017 г. в 19:50, Denis Magda <[email protected]>: >>>>> >>>>> Dmitriy, >>>>> First of all, thanks for putting this low level details on paper! >>>>> Current >>>>> >>>>>> and future community members will be grateful to you. >>>>>> >>>>>> Second, let me be a reviewer of the page and suggest some >>>>>> improvements. >>>>>> Until the review is over I would encourage us to hold on with any >>>>>> public >>>>>> announcements such as blog post. >>>>>> >>>>>> I’ve reviewed and edited Ignite Durable Memory section ( >>>>>> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+ >>>>>> Durable+Memory+-+under+the+hood#IgniteDurableMemory-under >>>>>> thehood-IgniteDurableMemory >>>>>> < >>>>>> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+ >>>>>> Durable+Memory+-+under+the+hood#IgniteDurableMemory-under >>>>>> thehood-IgniteDurableMemory>). >>>>>> Please address the questions below and I’ll move on with the new >>>>>> sections >>>>>> afterwards. >>>>>> >>>>>> 1) "Different caches and its partitions bring more complex addressing >>>>>> scheme but it is still possible to map from page ID to position in >>>>>> particular file: pageId => file + offset in this file”. >>>>>> >>>>>> Probably this appears too early in the doc because above we just >>>>>> introduced page ID notion with durable memory and now we’re talking >>>>>> about >>>>>> some files? Are these files of ignite persistence? What if the >>>>>> persistence >>>>>> is disabled? Honestly, the sentence gets out of the story context. >>>>>> >>>>>> >>>>>> 2) "Also, the tree pages can be merged into one if < 50% of space is >>>>>> used.” >>>>>> >>>>>> 50% of the whole memory available? Please clarify. >>>>>> >>>>>> >>>>>> 3) "There is one page ID table per memory segment.” >>>>>> >>>>>> It’s the same segment as it’s described in the main doc, right? >>>>>> https://apacheignite.readme.io/docs/memory-architecture#sect >>>>>> ion-memory-segment >>>>>> < >>>>>> https://apacheignite.readme.io/docs/memory-architecture#sect >>>>>> ion-memory-segment >>>>>> >>>>>> 4) "within unsafe segment” >>>>>> >>>>>> If we want to mention “usafe” here we need to explain what that is. >>>>>> >>>>>> 5) "If memory amount less than whole index size, Ignite still can >>>>>> operate.” >>>>>> >>>>>> How do we define “index size”? Is this the page size? >>>>>> >>>>>> >>>>>> — >>>>>> Denis >>>>>> >>>>>> >>>>>> On Aug 30, 2017, at 8:24 AM, Dmitry Pavlov <[email protected]> >>>>>> wrote: >>>>>> >>>>>> Hi Igniters, >>>>>>> >>>>>>> >>>>>>> I am glad to introduce two new wiki articles. These articles contain >>>>>>> high-level description of >>>>>>> >>>>>>> - Ignite Durable Memory ( >>>>>>> >>>>>>> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+ >>>>>>> >>>>>> Durable+Memory+-+under+the+hood >>>>>> >>>>>> ) >>>>>>> >>>>>>> - and Ignite Persistent Store ( >>>>>>> >>>>>>> https://cwiki.apache.org/confluence/display/IGNITE/Ignite+ >>>>>>> >>>>>> Persistent+Store+-+under+the+hood >>>>>> >>>>>> ) >>>>>>> >>>>>>> These articles cover basic principles and internal design. Pages may >>>>>>> be >>>>>>> useful for new contributors during start of working with persistent >>>>>>> >>>>>>> store >>>>>> >>>>>> code base. >>>>>>> >>>>>>> >>>>>>> I would like to thank all contributors who helped me with my >>>>>>> questions. >>>>>>> Their answers and support made it possible to start this description. >>>>>>> >>>>>>> >>>>>>> I will appreciate any feedback and questions. Suggestions and edits >>>>>>> are >>>>>>> welcome, of course. >>>>>>> >>>>>>> >>>>>>> Sincerely, >>>>>>> >>>>>>> Dmitriy Pavlov >>>>>>> >>>>>>> >>>>>> >
