Re: cache OS memory and spark usage of it

2018-04-10 Thread yncxcw
hi, Raúl First, the most of the OS memory cache is used by Page Cache which OS use for caching the recent read/write I/O. I think the understanding of OS memory cache should be discussed in two different perspectives. From a perspective of user-spac

Re: cache OS memory and spark usage of it

2018-04-10 Thread Jose Raul Perez Rodriguez
it was helpful, Then, the OS needs to fill some pressure from the applications requesting memory to free some memory cache? Exactly under which circumstances the OS free that memory to give it to applications requesting it? I mean if the total memory is 16GB and 10GB are used for OS cache,

Re: cache OS memory and spark usage of it

2018-04-11 Thread yncxcw
hi, Raúl (1)&(2) yes, the OS needs some pressure to release it. For example, if you have a total 16GB ram in your machine, then you read a file of 8GB and immediately close it. Noe the page cache would cache 8GB the file data. Then you start a program requesting memory from OS, the OS will releas