Re: Kernel memory management problem

2009-03-03 Thread Alexander Indenbaum
Indenbaum alexander.indenb...@gmail.com Subject: Re: Kernel memory management problem To: guy keren c...@actcom.co.il Cc: linux-il. linux-il@cs.huji.ac.il Date: Monday, March 2, 2009, 4:24 PM On Mon, Mar 2, 2009 at 1:16 PM, guy keren c...@actcom.co.il wrote: if you don't care about the speed

Kernel memory management problem

2009-03-02 Thread Alexander Indenbaum
Guys/Girls/Dudes, I'm working with embedded Linux box powered by 2.4 kernel ( I know :) we are in process of switching to 2.6 ) with 1G physical memory, no swap defined and some slow flash storage mounted. We noticed that if we copy large media file ( 1.4G ) to flash storage filesystem, using

Re: Kernel memory management problem

2009-03-02 Thread guy keren
if you don't care about the speed of copying the data and of slowing down the disk media consderably during this copying, try to change the copy program, so it will use the O_DIRECT flag when opening the file (and then you'll need to make sure the buffers you pass to write() are aligned to

Re: Kernel memory management problem

2009-03-02 Thread Valery Reznic
--- On Mon, 3/2/09, Alexander Indenbaum alexander.indenb...@gmail.com wrote: We noticed that if we copy large media file ( 1.4G ) to flash storage filesystem, using wget/sftp/whatever kernel grabs almost all the available physical memory and buffers are not released even after

Re: Kernel memory management problem

2009-03-02 Thread Alexander Indenbaum
Valery, See answer inline. On Mon, Mar 2, 2009 at 1:11 PM, Valery Reznic valery_rez...@yahoo.com wrote: --- On Mon, 3/2/09, Alexander Indenbaum alexander.indenb...@gmail.com wrote: We noticed that if we copy large media file ( 1.4G ) to flash storage filesystem, using wget/sftp/whatever

Re: Kernel memory management problem

2009-03-02 Thread Alexander Indenbaum
On Mon, Mar 2, 2009 at 1:16 PM, guy keren c...@actcom.co.il wrote: if you don't care about the speed of copying the data and of slowing down the disk media consderably during this copying, try to change the copy program, so it will use the O_DIRECT flag when opening the file (and then you'll

Re: Kernel memory management problem

2009-03-02 Thread Valery Reznic
--- On Mon, 3/2/09, Alexander Indenbaum alexander.indenb...@gmail.com wrote: From: Alexander Indenbaum alexander.indenb...@gmail.com Subject: Re: Kernel memory management problem To: guy keren c...@actcom.co.il Cc: linux-il. linux-il@cs.huji.ac.il Date: Monday, March 2, 2009, 4:24 PM

Re: Kernel memory management problem

2009-03-02 Thread Alexander Indenbaum
On Mon, Mar 2, 2009 at 4:35 PM, Valery Reznic valery_rez...@yahoo.com wrote: Unfortunately this is real problem: if you do something like while(true) { transfer } then application crashes and burns in hell :) Are you sure that culprit is kernel ? May be transfer part has memory leak ? Did

Re: Kernel memory management problem

2009-03-02 Thread Alexander Indenbaum
On Mon, Mar 2, 2009 at 4:44 PM, Valery Reznic valery_rez...@yahoo.com wrote: --- On Mon, 3/2/09, Alexander Indenbaum alexander.indenb...@gmail.com wrote: From: Alexander Indenbaum alexander.indenb...@gmail.com Subject: Re: Kernel memory management problem To: guy keren c...@actcom.co.il Cc

Re: Kernel memory management problem

2009-03-02 Thread Valery Reznic
Unfortunately this is real problem: if you do something like while(true) { transfer } then application crashes and burns in hell :) Are you sure that culprit is kernel ? May be transfer part has memory leak ? Did you try to run it under valgrind ? or run top ? Nondeterministically -

Re: Kernel memory management problem

2009-03-02 Thread Alexander Indenbaum
On Mon, Mar 2, 2009 at 5:00 PM, Valery Reznic valery_rez...@yahoo.com wrote: --- On Mon, 3/2/09, Alexander Indenbaum alexander.indenb...@gmail.com wrote: From: Alexander Indenbaum alexander.indenb...@gmail.com Subject: Re: Kernel memory management problem To: valery_rez...@yahoo.com Cc

Re: Kernel memory management problem

2009-03-02 Thread Valery Reznic
--- On Mon, 3/2/09, Alexander Indenbaum alexander.indenb...@gmail.com wrote: From: Alexander Indenbaum alexander.indenb...@gmail.com Subject: Re: Kernel memory management problem To: valery_rez...@yahoo.com Cc: linux-il. linux-il@cs.huji.ac.il Date: Monday, March 2, 2009, 4:45 PM On Mon

Re: Kernel memory management problem

2009-03-02 Thread guy keren
Alexander Indenbaum wrote: On Mon, Mar 2, 2009 at 4:44 PM, Valery Reznic valery_rez...@yahoo.com wrote: --- On Mon, 3/2/09, Alexander Indenbaum alexander.indenb...@gmail.com wrote: From: Alexander Indenbaum alexander.indenb...@gmail.com Subject: Re: Kernel memory management problem To: guy