Hi Pavel,
This area is confusing. There is no indication that the memory pressure
applies to any individual object or allocation, so there is clearly no
association between memory pressure and any particular resource.
I get your argument that .Net can 'see' allocated memory. What is unclear
is whether it cares about actually allocated and used pages, or committed
pages.
I see there is a LazyMemoryAllocation (default: true) for data regions.
Some data regions set this to false, eg:
^-- sysMemPlc region [type=internal, persistence=true,
lazyAlloc=false,
^-- metastoreMemPlc region [type=internal, persistence=true,
lazyAlloc=false,
^-- TxLog region [type=internal, persistence=true, lazyAlloc=false,
The documentation is not clear on the effect of this flag other than to say
it is for 'Lazy memory allocation'. If this flag is true will Ignite
proactively allocate and use all pages in a data region, rather than
incrementally?
Thanks,
Raymond.
On Tue, Jul 11, 2023 at 10:55 PM Pavel Tupitsyn <ptupit...@apache.org>
wrote:
> > I can’t see another way of letting . Net know that it can’t have access
> to all the ‘free’ memory in the process
>
> You don't need to tell .NET how much memory is currently available. It is
> the job of the OS. .NET can "see" the size of the unmanaged heap.
>
> To quote another explanation [1]:
>
> > The point of AddMemoryPressure is to tell the garbage collector that
> there's a large amount of memory allocated with that object.
> > If it's unmanaged, the garbage collector doesn't know about it; only the
> managed portion.
> > Since the managed portion is relatively small, the GC may let it pass
> for garbage collection several times, essentially wasting memory that might
> need to be freed.
>
> I really don't think AddMemoryPressure is the right thing to do in your
> case.
> If you run into OOM issues, then look into Ignite memory region settings
> [2] and/or adjust application memory usage on the .NET side, so that the
> sum of those is not bigger than available RAM.
>
> [1]
> https://stackoverflow.com/questions/1149181/what-is-the-point-of-using-gc-addmemorypressure-with-an-unmanaged-resource
> [2]
> https://ignite.apache.org/docs/latest/memory-configuration/data-regions#configuring-default-data-region
>
> On Tue, Jul 11, 2023 at 11:48 AM Raymond Wilson <
> raymond_wil...@trimble.com> wrote:
>
>> How do Ignite .Net server nodes manage this memory issue in other
>> projects?
>>
>> On Tue, Jul 11, 2023 at 5:32 PM Raymond Wilson <
>> raymond_wil...@trimble.com> wrote:
>>
>>> Oops, commutes => committed
>>>
>>> On Tue, 11 Jul 2023 at 4:34 PM, Raymond Wilson <
>>> raymond_wil...@trimble.com> wrote:
>>>
>>>> I can’t see another way of letting . Net know that it can’t have access
>>>> to all the ‘free’ memory in the process when a large slab of that is spoken
>>>> for in terms of memory commutes to Ignite data regions.
>>>>
>>>> In the current setup, as time goes on and Ignite progressively fills
>>>> the allocated cache ram then system behaviour changes and can result in out
>>>> of memory issues. I think I would prefer consistent system behaviour wrt to
>>>> allocated resources from the start.
>>>>
>>>> Raymond.
>>>>
>>>> On Tue, 11 Jul 2023 at 3:57 PM, Pavel Tupitsyn <ptupit...@apache.org>
>>>> wrote:
>>>>
>>>>> Are you sure this is necessary?
>>>>>
>>>>> GC.AddMemoryPressure documentation [1] states that this will "improve
>>>>> performance only for types that exclusively depend on finalizers".
>>>>>
>>>>> [1]
>>>>> https://learn.microsoft.com/en-us/dotnet/api/system.gc.addmemorypressure?view=net-7.0
>>>>>
>>>>> On Tue, Jul 11, 2023 at 1:02 AM Raymond Wilson <
>>>>> raymond_wil...@trimble.com> wrote:
>>>>>
>>>>>> I'm making changes to add memory pressure to the GC to take into
>>>>>> account memory committed to the Ignite data regions as this will be
>>>>>> unmanaged memory allocations from the perspective of the GC.
>>>>>>
>>>>>> I don't call seeing anything related to this for .Net clients in the
>>>>>> documentation. Are you aware of any?
>>>>>>
>>>>>> Raymond.
>>>>>>
>>>>>> On Mon, Jul 10, 2023 at 9:41 PM Raymond Wilson <
>>>>>> raymond_wil...@trimble.com> wrote:
>>>>>>
>>>>>>> Thanks Pavel, this makes sense.
>>>>>>>
>>>>>>> Querying the .Net Process instance shows this as the difference
>>>>>>> between PagesMemorySize (includes committed) versus WorkingSet (includes
>>>>>>> uses/written to) size.
>>>>>>> Raymond.
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> <http://www.trimble.com/>
>>>>>> Raymond Wilson
>>>>>> Trimble Distinguished Engineer, Civil Construction Software (CCS)
>>>>>> 11 Birmingham Drive |
>>>>>> <https://www.google.com/maps/search/11+Birmingham+Drive%C2%A0%7C%C2%A0+Christchurch,+New+Zealand?entry=gmail&source=g>Christchurch,
>>>>>> New Zealand
>>>>>> <https://www.google.com/maps/search/11+Birmingham+Drive%C2%A0%7C%C2%A0+Christchurch,+New+Zealand?entry=gmail&source=g>
>>>>>> raymond_wil...@trimble.com
>>>>>>
>>>>>>
>>>>>> <https://worksos.trimble.com/?utm_source=Trimble&utm_medium=emailsign&utm_campaign=Launch>
>>>>>>
>>>>> --
>>>> <http://www.trimble.com/>
>>>> Raymond Wilson
>>>> Trimble Distinguished Engineer, Civil Construction Software (CCS)
>>>> 11 Birmingham Drive | Christchurch, New Zealand
>>>> raymond_wil...@trimble.com
>>>>
>>>>
>>>> <https://worksos.trimble.com/?utm_source=Trimble&utm_medium=emailsign&utm_campaign=Launch>
>>>>
>>> --
>>> <http://www.trimble.com/>
>>> Raymond Wilson
>>> Trimble Distinguished Engineer, Civil Construction Software (CCS)
>>> 11 Birmingham Drive | Christchurch, New Zealand
>>> raymond_wil...@trimble.com
>>>
>>>
>>> <https://worksos.trimble.com/?utm_source=Trimble&utm_medium=emailsign&utm_campaign=Launch>
>>>
>>
>>
>> --
>> <http://www.trimble.com/>
>> Raymond Wilson
>> Trimble Distinguished Engineer, Civil Construction Software (CCS)
>> 11 Birmingham Drive | Christchurch, New Zealand
>> raymond_wil...@trimble.com
>>
>>
>> <https://worksos.trimble.com/?utm_source=Trimble&utm_medium=emailsign&utm_campaign=Launch>
>>
>
--
<http://www.trimble.com/>
Raymond Wilson
Trimble Distinguished Engineer, Civil Construction Software (CCS)
11 Birmingham Drive | Christchurch, New Zealand
raymond_wil...@trimble.com
<https://worksos.trimble.com/?utm_source=Trimble&utm_medium=emailsign&utm_campaign=Launch>
