Just FYI, we have held off any memory pressure changes in the meantime while we continue to investigate the memory issues we have.
On Tue, 18 Jul 2023 at 9:07 AM, Raymond Wilson <raymond_wil...@trimble.com> wrote: > Hi Pavel, > > This area is confusing. There is no indication that the memory pressure > applies to any individual object or allocation, so there is clearly no > association between memory pressure and any particular resource. > > I get your argument that .Net can 'see' allocated memory. What is unclear > is whether it cares about actually allocated and used pages, or committed > pages. > > I see there is a LazyMemoryAllocation (default: true) for data regions. > Some data regions set this to false, eg: > > ^-- sysMemPlc region [type=internal, persistence=true, > lazyAlloc=false, > ^-- metastoreMemPlc region [type=internal, persistence=true, > lazyAlloc=false, > ^-- TxLog region [type=internal, persistence=true, lazyAlloc=false, > > The documentation is not clear on the effect of this flag other than to > say it is for 'Lazy memory allocation'. If this flag is true will Ignite > proactively allocate and use all pages in a data region, rather than > incrementally? > > Thanks, > Raymond. > > > On Tue, Jul 11, 2023 at 10:55 PM Pavel Tupitsyn <ptupit...@apache.org> > wrote: > >> > I can’t see another way of letting . Net know that it can’t have access >> to all the ‘free’ memory in the process >> >> You don't need to tell .NET how much memory is currently available. It is >> the job of the OS. .NET can "see" the size of the unmanaged heap. >> >> To quote another explanation [1]: >> >> > The point of AddMemoryPressure is to tell the garbage collector that >> there's a large amount of memory allocated with that object. >> > If it's unmanaged, the garbage collector doesn't know about it; only >> the managed portion. >> > Since the managed portion is relatively small, the GC may let it pass >> for garbage collection several times, essentially wasting memory that might >> need to be freed. >> >> I really don't think AddMemoryPressure is the right thing to do in your >> case. >> If you run into OOM issues, then look into Ignite memory region settings >> [2] and/or adjust application memory usage on the .NET side, so that the >> sum of those is not bigger than available RAM. >> >> [1] >> https://stackoverflow.com/questions/1149181/what-is-the-point-of-using-gc-addmemorypressure-with-an-unmanaged-resource >> [2] >> https://ignite.apache.org/docs/latest/memory-configuration/data-regions#configuring-default-data-region >> >> On Tue, Jul 11, 2023 at 11:48 AM Raymond Wilson < >> raymond_wil...@trimble.com> wrote: >> >>> How do Ignite .Net server nodes manage this memory issue in other >>> projects? >>> >>> On Tue, Jul 11, 2023 at 5:32 PM Raymond Wilson < >>> raymond_wil...@trimble.com> wrote: >>> >>>> Oops, commutes => committed >>>> >>>> On Tue, 11 Jul 2023 at 4:34 PM, Raymond Wilson < >>>> raymond_wil...@trimble.com> wrote: >>>> >>>>> I can’t see another way of letting . Net know that it can’t have >>>>> access to all the ‘free’ memory in the process when a large slab of that >>>>> is >>>>> spoken for in terms of memory commutes to Ignite data regions. >>>>> >>>>> In the current setup, as time goes on and Ignite progressively fills >>>>> the allocated cache ram then system behaviour changes and can result in >>>>> out >>>>> of memory issues. I think I would prefer consistent system behaviour wrt >>>>> to >>>>> allocated resources from the start. >>>>> >>>>> Raymond. >>>>> >>>>> On Tue, 11 Jul 2023 at 3:57 PM, Pavel Tupitsyn <ptupit...@apache.org> >>>>> wrote: >>>>> >>>>>> Are you sure this is necessary? >>>>>> >>>>>> GC.AddMemoryPressure documentation [1] states that this will "improve >>>>>> performance only for types that exclusively depend on finalizers". >>>>>> >>>>>> [1] >>>>>> https://learn.microsoft.com/en-us/dotnet/api/system.gc.addmemorypressure?view=net-7.0 >>>>>> >>>>>> On Tue, Jul 11, 2023 at 1:02 AM Raymond Wilson < >>>>>> raymond_wil...@trimble.com> wrote: >>>>>> >>>>>>> I'm making changes to add memory pressure to the GC to take into >>>>>>> account memory committed to the Ignite data regions as this will be >>>>>>> unmanaged memory allocations from the perspective of the GC. >>>>>>> >>>>>>> I don't call seeing anything related to this for .Net clients in the >>>>>>> documentation. Are you aware of any? >>>>>>> >>>>>>> Raymond. >>>>>>> >>>>>>> On Mon, Jul 10, 2023 at 9:41 PM Raymond Wilson < >>>>>>> raymond_wil...@trimble.com> wrote: >>>>>>> >>>>>>>> Thanks Pavel, this makes sense. >>>>>>>> >>>>>>>> Querying the .Net Process instance shows this as the difference >>>>>>>> between PagesMemorySize (includes committed) versus WorkingSet >>>>>>>> (includes >>>>>>>> uses/written to) size. >>>>>>>> Raymond. >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> <http://www.trimble.com/> >>>>>>> Raymond Wilson >>>>>>> Trimble Distinguished Engineer, Civil Construction Software (CCS) >>>>>>> 11 Birmingham Drive | >>>>>>> <https://www.google.com/maps/search/11+Birmingham+Drive%C2%A0%7C%C2%A0+Christchurch,+New+Zealand?entry=gmail&source=g>Christchurch, >>>>>>> New Zealand >>>>>>> <https://www.google.com/maps/search/11+Birmingham+Drive%C2%A0%7C%C2%A0+Christchurch,+New+Zealand?entry=gmail&source=g> >>>>>>> raymond_wil...@trimble.com >>>>>>> >>>>>>> >>>>>>> <https://worksos.trimble.com/?utm_source=Trimble&utm_medium=emailsign&utm_campaign=Launch> >>>>>>> >>>>>> -- >>>>> <http://www.trimble.com/> >>>>> Raymond Wilson >>>>> Trimble Distinguished Engineer, Civil Construction Software (CCS) >>>>> 11 Birmingham Drive | Christchurch, New Zealand >>>>> raymond_wil...@trimble.com >>>>> >>>>> >>>>> <https://worksos.trimble.com/?utm_source=Trimble&utm_medium=emailsign&utm_campaign=Launch> >>>>> >>>> -- >>>> <http://www.trimble.com/> >>>> Raymond Wilson >>>> Trimble Distinguished Engineer, Civil Construction Software (CCS) >>>> 11 Birmingham Drive | Christchurch, New Zealand >>>> raymond_wil...@trimble.com >>>> >>>> >>>> <https://worksos.trimble.com/?utm_source=Trimble&utm_medium=emailsign&utm_campaign=Launch> >>>> >>> >>> >>> -- >>> <http://www.trimble.com/> >>> Raymond Wilson >>> Trimble Distinguished Engineer, Civil Construction Software (CCS) >>> 11 Birmingham Drive | Christchurch, New Zealand >>> raymond_wil...@trimble.com >>> >>> >>> <https://worksos.trimble.com/?utm_source=Trimble&utm_medium=emailsign&utm_campaign=Launch> >>> >> > > -- > <http://www.trimble.com/> > Raymond Wilson > Trimble Distinguished Engineer, Civil Construction Software (CCS) > 11 Birmingham Drive | Christchurch, New Zealand > raymond_wil...@trimble.com > > > <https://worksos.trimble.com/?utm_source=Trimble&utm_medium=emailsign&utm_campaign=Launch> > -- <http://www.trimble.com/> Raymond Wilson Trimble Distinguished Engineer, Civil Construction Software (CCS) 11 Birmingham Drive | Christchurch, New Zealand raymond_wil...@trimble.com <https://worksos.trimble.com/?utm_source=Trimble&utm_medium=emailsign&utm_campaign=Launch>
