Re: How much data is actually in RAM?

2019-06-14 Thread Michael Cherkasov
Hi John,

if you have native persistence enabled that ignite will evict data to a
disk if there's no space in RAM.
BUT you have to specify maxSize for RAM! so Ignite really doesn't have a
limit for space, it's just limited by your disk and you will get IgniteOOM
at some point.

So your current configuration should fail because Ignite will try to
allocate 72 GB in memory on start.

>1- If I get this right... Each node is using about 1GB on-heap JVM memory
each for the ignite process to run. Correct? Should this be set a bit
higher or that's ok?
Ignite store data in off heap, but heap will be used for everything else,
like message processing, marshaling and unmarshaling data and etc.
If you use SQL, ignite will load data from off heap to on heap to send
record as a response to client. Usually SQL the main consumer of heap, so
you need to tune you heap size based on your needs.

>2- There is total 216GB off-heap so about 72GB off-heap per node. So how
much of that is stored in actual RAM in the remainder
>of what is left from 8GB of the physical host? Is that the 20% value
indicated in the docs? Or should maxSize be set to 6L and not 72L giving
2GB free to the OS and ignite process?
As I said, this option defines not max size for storage, but the max size
for storage in RAM! Ignite always will use RAM memory that you allocate to
it for data storage, only if it's not enough memory for data storage it
will evict data to a disk.

Thanks,
Mike.


пт, 14 июн. 2019 г. в 14:02, John Smith :

> Hi, so I have 3 machines with 8GB RAM and 96GB disk each.
>
> I have configured the persistence as
>
>  class="org.apache.ignite.configuration.DataRegionConfiguration">
>   
>
>   
>   
> 
>   
>
> Looking at the logs:
> Topology snapshot [ver=3, locNode=xx, servers=3, clients=0,
> state=INACTIVE, CPUs=12, offheap=216.0GB, heap=3.0GB]
>
> 1- If I get this right... Each node is using about 1GB on-heap JVM memory
> each for the ignite process to run. Correct? Should this be set a bit
> higher or that's ok?
> 2- There is total 216GB off-heap so about 72GB off-heap per node. So how
> much of that is stored in actual RAM in the remainder of what is left from
> 8GB of the physical host? Is that the 20% value indicated in the docs? Or
> should maxSize be set to 6L and not 72L giving 2GB free to the OS and
> ignite process?
>
>
>
>
>


Re: How much data is actually in RAM?

2019-06-14 Thread Shane Duan
It guess really depends how you use the cluster. For my use case, I have to
bump up heap to 8G to support concurrent queries (JDBC); otherwise, Ignite
could crash since Ignite need process results in heap.

On Fri, Jun 14, 2019 at 2:02 PM John Smith  wrote:

> Hi, so I have 3 machines with 8GB RAM and 96GB disk each.
>
> I have configured the persistence as
>
>  class="org.apache.ignite.configuration.DataRegionConfiguration">
>   
>
>   
>   
> 
>   
>
> Looking at the logs:
> Topology snapshot [ver=3, locNode=xx, servers=3, clients=0,
> state=INACTIVE, CPUs=12, offheap=216.0GB, heap=3.0GB]
>
> 1- If I get this right... Each node is using about 1GB on-heap JVM memory
> each for the ignite process to run. Correct? Should this be set a bit
> higher or that's ok?
> 2- There is total 216GB off-heap so about 72GB off-heap per node. So how
> much of that is stored in actual RAM in the remainder of what is left from
> 8GB of the physical host? Is that the 20% value indicated in the docs? Or
> should maxSize be set to 6L and not 72L giving 2GB free to the OS and
> ignite process?
>
>
>
>
>


How much data is actually in RAM?

2019-06-14 Thread John Smith
Hi, so I have 3 machines with 8GB RAM and 96GB disk each.

I have configured the persistence as


  

  
  

  

Looking at the logs:
Topology snapshot [ver=3, locNode=xx, servers=3, clients=0,
state=INACTIVE, CPUs=12, offheap=216.0GB, heap=3.0GB]

1- If I get this right... Each node is using about 1GB on-heap JVM memory
each for the ignite process to run. Correct? Should this be set a bit
higher or that's ok?
2- There is total 216GB off-heap so about 72GB off-heap per node. So how
much of that is stored in actual RAM in the remainder of what is left from
8GB of the physical host? Is that the 20% value indicated in the docs? Or
should maxSize be set to 6L and not 72L giving 2GB free to the OS and
ignite process?