On 12/04/2014 06:03 PM, Dominig ar Foll wrote:
> Le 04/12/2014 00:49, Carsten Haitzler (The Rasterman) a écrit :
>>> All the Telco product that I worked on before joining Intel, use that
>>> model. Checking has a cost.
>>> The trick is to ask for enough in one go to be able to take a decision
>>> on Kernel refusal without forcing the kernel to take action by itself.
>>> Not that nice, but works pretty well.
>>> Dominig
>> problem is that this won't work well on linux...
> All those products that I have in mind were running Linux.
>> 1. swap - if you have it the above method won't be that great
> In my experience swap is never activated on embedded critical system.
then i think this code was getting something seriously wrong.
>> 2. until you touch the pages (read or write) they don't exist so it will have
>> no impact
> I have not checked that detail with latest kernel. but is use to be that
> when the kernel was giving you the allocation, the RAM was there.
> Remember tha tI assume that there is no swap.
so last i checked, linux overcommit was enabled by default. so unless
you disable it:
cat /proc/sys/vm/overcommit_memory
to see if it is on or not (1 or 0).
linux will allow you to allocate as much as you like:
6:14PM ~ > free -m
total used free shared buff/cache
available
Mem: 3867 1684 158 247
2024 1653
Swap: 0 0 0
my sample code:
#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <stdlib.h>
int main(int argc, char **argv)
{
size_t size = atoi(argv[1]);
void *ptr = malloc(size * 10124 * 1024);
printf("malloc result = %p\n", ptr);
return 0;
}
and me running it:
6:14PM ~ > ./tt 10000
malloc result = 0x7fc5b0a37010
i just allocated 10gb... on a machine with only 4gb of physical ram
(only 3.8 total usable) and only 1.7g or so actually available given
existing memory usage... and zero swap to spill over into. how is it
possible to allocate 10g on this machine? overcommit. if i were to now
walk through this memory and read/write to each and every page.. the
kernel will be forced to ACTUALLY realize the alocation. once i end up
touching about 1.6gb of memory... it'll actually crash as that memory
doesn't actually exist at all even though allocation succeeded.
there is a reason overcimmit exists.
1. lots of apps allocate memory and then only use some of it. via malloc
or mmap or brk or sbrk (heap/stack). so forcing the real memory to exist
for all of it is a waste
2. biggest reason - fork().
since fork makes a COPY of the current process in a new one. let us
asume we have a machine with 4gb ram.. and i have a process using 2.5g
of ram (no swap) that simply wishes to fork+exec some binary. the first
thing it does after a fork is a simple exec. you cannot do this. the
fork will fail unless you overcommit. why? by definition the child could
dirty (write to) any pages it has and cause a copy-on-write and thus the
kernel MUSt ensure the memory exists for theat copy-on-write to happen.
even if all that happens now is exec() .. it coudl be a ssimpel as
execcing "ls"... just from a large binary. this is also commonly the
case with apache where it forks off a lot of slaves.
>> 3. a malloc won't give you linear/continuous memory. either will a mmap etc.
>> unless it's a specialized device guaranteeing that (i know of no general
>> device
>> providing that)
>> 4. due to linux doing overcommit to ensure we don't waste lots of memory any
>> allocation is never guaranteed. you can turn this off, but then you basically
>> need a chunk more real memory to make up for it. i might guess 50% to 100%
>> more
>>
>> you really need the kernel to give userspace more information.
> Kernel certainly has a better and more detailed view but you still need
> to extract that information to take decision in time.
>
>
--
The above message is intended solely for the named addressee and may
contain trade secret, industrial technology or privileged and
confidential information otherwise protected under applicable law
including the Unfair Competition Prevention and Trade Secret Protection
Act. Any unauthorized dissemination, distribution, copying or use of the
information contained in this communication is strictly prohibited. If
you have received this communication in error, please notify the sender
by email and delete this communication immediately.
_______________________________________________
Dev mailing list
[email protected]
https://lists.tizen.org/listinfo/dev
