Re: Global off-heap memory limit

[Denis Magda]

Sergi,

What is a JIRA ticket number? I’m struggling to find it using variety of search 
filters.

—
Denis

> On Sep 16, 2016, at 9:54 AM, Sergi Vladykin <sergi.vlady...@gmail.com> wrote:
> 
> That is true, in Ignite 2.0 you will be able to setup one or more
> differently configured memory pools per node. When you start a cache you
> have to decide which one to use for this cache. The caches attached to the
> same pool together will not be able to grow beyond the maximum size of that
> pool.
> 
> Sergi
> 
> 2016-09-16 4:20 GMT+03:00 Andrey Kornev <andrewkor...@hotmail.com>:
> 
>> +1
>> I'm particularly interested in the use case where I have caches that are
>> created dynamically and in general I don't know in advance how many of them
>> will exist at any point in time. But I do know the total amount of memory
>> I'm willing to make available to the caches.
>> So it would be nice if the newly introduced Memory Policies would allow
>> this type of configuration  possible.
>> Regards
>> Andrey
>> _
>> From: Dmitriy Setrakyan <dsetrak...@apache.org> dsetrak...@apache.org>>
>> Sent: Thursday, September 15, 2016 12:44 PM
>> Subject: Re: Global off-heap memory limit
>> To: <dev@ignite.apache.org<mailto:dev@ignite.apache.org>>
>> 
>> 
>> To my knowledge, in Ignite 2.0 we will have Memory Policies, and multiple
>> caches will be able to reuse the same policy. Since the memory size will be
>> defined at the policy level, it will essentially solve this problem.
>> 
>> Am I wrong?
>> 
>> D.
>> 
>> On Thu, Sep 15, 2016 at 12:15 PM, Igor Rudyak <irud...@gmail.com> irud...@gmail.com>> wrote:
>> 
>>> +1
>>> 
>>> On Thu, Sep 15, 2016 at 12:11 PM, Denis Magda <dma...@gridgain.com
>> <mailto:dma...@gridgain.com>> wrote:
>>> 
>>>> +1
>>>> 
>>>> I've faced with such injuries as well and this feature sounds
>> reasonable
>>>> for me.
>>>> 
>>>> However, is this going to be supported in the new memory architecture
>> the
>>>> community has been working on? Alex G., can you comment?
>>>> 
>>>> -
>>>> Denis
>>>> 
>>>>> On Sep 15, 2016, at 12:07 PM, Valentin Kulichenko <
>>>> valentin.kuliche...@gmail.com<mailto:valentin.kuliche...@gmail.com>>
>> wrote:
>>>>> 
>>>>> Folks,
>>>>> 
>>>>> I'm seeing more and more requests from users to add an option to
>>> specify
>>>>> the off-heap memory limit globally, for all the caches in total.
>>>> Currently
>>>>> we allow this only on per-cache level, which can be not very usable
>>>>> sometimes.
>>>>> 
>>>>> E.g., imagine that you have 100GB of RAM and two caches with 50GB
>> limit
>>>> for
>>>>> each of them. At some point one of the caches can consume all 50GB,
>>> while
>>>>> the second is almost empty. You can't add anything to the first cache
>>>>> regardless of the fact that you still have lots of memory available.
>>>>> 
>>>>> Can we add such parameter?
>>>>> 
>>>>> -Val
>>>> 
>>>> 
>>> 
>> 
>> 
>> 

Re: Global off-heap memory limit

[Sergi Vladykin]

That is true, in Ignite 2.0 you will be able to setup one or more
differently configured memory pools per node. When you start a cache you
have to decide which one to use for this cache. The caches attached to the
same pool together will not be able to grow beyond the maximum size of that
pool.

Sergi

2016-09-16 4:20 GMT+03:00 Andrey Kornev <andrewkor...@hotmail.com>:

> +1
> I'm particularly interested in the use case where I have caches that are
> created dynamically and in general I don't know in advance how many of them
> will exist at any point in time. But I do know the total amount of memory
> I'm willing to make available to the caches.
> So it would be nice if the newly introduced Memory Policies would allow
> this type of configuration  possible.
> Regards
> Andrey
> _
> From: Dmitriy Setrakyan <dsetrak...@apache.org dsetrak...@apache.org>>
> Sent: Thursday, September 15, 2016 12:44 PM
> Subject: Re: Global off-heap memory limit
> To: <dev@ignite.apache.org<mailto:dev@ignite.apache.org>>
>
>
> To my knowledge, in Ignite 2.0 we will have Memory Policies, and multiple
> caches will be able to reuse the same policy. Since the memory size will be
> defined at the policy level, it will essentially solve this problem.
>
> Am I wrong?
>
> D.
>
> On Thu, Sep 15, 2016 at 12:15 PM, Igor Rudyak <irud...@gmail.com irud...@gmail.com>> wrote:
>
> > +1
> >
> > On Thu, Sep 15, 2016 at 12:11 PM, Denis Magda <dma...@gridgain.com
> <mailto:dma...@gridgain.com>> wrote:
> >
> > > +1
> > >
> > > I've faced with such injuries as well and this feature sounds
> reasonable
> > > for me.
> > >
> > > However, is this going to be supported in the new memory architecture
> the
> > > community has been working on? Alex G., can you comment?
> > >
> > > -
> > > Denis
> > >
> > > > On Sep 15, 2016, at 12:07 PM, Valentin Kulichenko <
> > > valentin.kuliche...@gmail.com<mailto:valentin.kuliche...@gmail.com>>
> wrote:
> > > >
> > > > Folks,
> > > >
> > > > I'm seeing more and more requests from users to add an option to
> > specify
> > > > the off-heap memory limit globally, for all the caches in total.
> > > Currently
> > > > we allow this only on per-cache level, which can be not very usable
> > > > sometimes.
> > > >
> > > > E.g., imagine that you have 100GB of RAM and two caches with 50GB
> limit
> > > for
> > > > each of them. At some point one of the caches can consume all 50GB,
> > while
> > > > the second is almost empty. You can't add anything to the first cache
> > > > regardless of the fact that you still have lots of memory available.
> > > >
> > > > Can we add such parameter?
> > > >
> > > > -Val
> > >
> > >
> >
>
>
>

Re: Global off-heap memory limit

[Andrey Kornev]

+1
I'm particularly interested in the use case where I have caches that are 
created dynamically and in general I don't know in advance how many of them 
will exist at any point in time. But I do know the total amount of memory I'm 
willing to make available to the caches.
So it would be nice if the newly introduced Memory Policies would allow this 
type of configuration  possible.
Regards
Andrey
_
From: Dmitriy Setrakyan <dsetrak...@apache.org<mailto:dsetrak...@apache.org>>
Sent: Thursday, September 15, 2016 12:44 PM
Subject: Re: Global off-heap memory limit
To: <dev@ignite.apache.org<mailto:dev@ignite.apache.org>>


To my knowledge, in Ignite 2.0 we will have Memory Policies, and multiple
caches will be able to reuse the same policy. Since the memory size will be
defined at the policy level, it will essentially solve this problem.

Am I wrong?

D.

On Thu, Sep 15, 2016 at 12:15 PM, Igor Rudyak 
<irud...@gmail.com<mailto:irud...@gmail.com>> wrote:

> +1
>
> On Thu, Sep 15, 2016 at 12:11 PM, Denis Magda 
> <dma...@gridgain.com<mailto:dma...@gridgain.com>> wrote:
>
> > +1
> >
> > I've faced with such injuries as well and this feature sounds reasonable
> > for me.
> >
> > However, is this going to be supported in the new memory architecture the
> > community has been working on? Alex G., can you comment?
> >
> > -
> > Denis
> >
> > > On Sep 15, 2016, at 12:07 PM, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com<mailto:valentin.kuliche...@gmail.com>> wrote:
> > >
> > > Folks,
> > >
> > > I'm seeing more and more requests from users to add an option to
> specify
> > > the off-heap memory limit globally, for all the caches in total.
> > Currently
> > > we allow this only on per-cache level, which can be not very usable
> > > sometimes.
> > >
> > > E.g., imagine that you have 100GB of RAM and two caches with 50GB limit
> > for
> > > each of them. At some point one of the caches can consume all 50GB,
> while
> > > the second is almost empty. You can't add anything to the first cache
> > > regardless of the fact that you still have lots of memory available.
> > >
> > > Can we add such parameter?
> > >
> > > -Val
> >
> >
>

Re: Global off-heap memory limit

[Dmitriy Setrakyan]

To my knowledge, in Ignite 2.0 we will have Memory Policies, and multiple
caches will be able to reuse the same policy. Since the memory size will be
defined at the policy level, it will essentially solve this problem.

Am I wrong?

D.

On Thu, Sep 15, 2016 at 12:15 PM, Igor Rudyak  wrote:

> +1
>
> On Thu, Sep 15, 2016 at 12:11 PM, Denis Magda  wrote:
>
> > +1
> >
> > I’ve faced with such injuries as well and this feature sounds reasonable
> > for me.
> >
> > However, is this going to be supported in the new memory architecture the
> > community has been working on? Alex G., can you comment?
> >
> > —
> > Denis
> >
> > > On Sep 15, 2016, at 12:07 PM, Valentin Kulichenko <
> > valentin.kuliche...@gmail.com> wrote:
> > >
> > > Folks,
> > >
> > > I'm seeing more and more requests from users to add an option to
> specify
> > > the off-heap memory limit globally, for all the caches in total.
> > Currently
> > > we allow this only on per-cache level, which can be not very usable
> > > sometimes.
> > >
> > > E.g., imagine that you have 100GB of RAM and two caches with 50GB limit
> > for
> > > each of them. At some point one of the caches can consume all 50GB,
> while
> > > the second is almost empty. You can't add anything to the first cache
> > > regardless of the fact that you still have lots of memory available.
> > >
> > > Can we add such parameter?
> > >
> > > -Val
> >
> >
>

Re: Global off-heap memory limit

[Igor Rudyak]

+1

On Thu, Sep 15, 2016 at 12:11 PM, Denis Magda  wrote:

> +1
>
> I’ve faced with such injuries as well and this feature sounds reasonable
> for me.
>
> However, is this going to be supported in the new memory architecture the
> community has been working on? Alex G., can you comment?
>
> —
> Denis
>
> > On Sep 15, 2016, at 12:07 PM, Valentin Kulichenko <
> valentin.kuliche...@gmail.com> wrote:
> >
> > Folks,
> >
> > I'm seeing more and more requests from users to add an option to specify
> > the off-heap memory limit globally, for all the caches in total.
> Currently
> > we allow this only on per-cache level, which can be not very usable
> > sometimes.
> >
> > E.g., imagine that you have 100GB of RAM and two caches with 50GB limit
> for
> > each of them. At some point one of the caches can consume all 50GB, while
> > the second is almost empty. You can't add anything to the first cache
> > regardless of the fact that you still have lots of memory available.
> >
> > Can we add such parameter?
> >
> > -Val
>
>

Re: Global off-heap memory limit

[Denis Magda]

+1

I’ve faced with such injuries as well and this feature sounds reasonable for me.

However, is this going to be supported in the new memory architecture the 
community has been working on? Alex G., can you comment?

—
Denis

> On Sep 15, 2016, at 12:07 PM, Valentin Kulichenko 
>  wrote:
> 
> Folks,
> 
> I'm seeing more and more requests from users to add an option to specify
> the off-heap memory limit globally, for all the caches in total. Currently
> we allow this only on per-cache level, which can be not very usable
> sometimes.
> 
> E.g., imagine that you have 100GB of RAM and two caches with 50GB limit for
> each of them. At some point one of the caches can consume all 50GB, while
> the second is almost empty. You can't add anything to the first cache
> regardless of the fact that you still have lots of memory available.
> 
> Can we add such parameter?
> 
> -Val