Thanks for pointing me towards the cache direction, Stephan. I read more on Linux memory usage, and apparently, the amount of memory my application uses should be (memory used - memory buffers - memory cached). When taking memory buffers & memory cached into account, the memory usage for my application remains constant.
So, there's on leak. I just didn't account for all the players. Khoa. From: Stephan Wiesand <[email protected]> To: Khoa To <[email protected]> Cc: [email protected] Date: 06/21/2010 01:08 PM Subject: Re: Mem leak with fopen/fseek/fclose ? Sent by: [email protected] On Jun 21, 2010, at 18:47 , Khoa To wrote: > > I don't think we use afs file system. slabtop does not have afs_inode_cache as one of its entries. Sorry, I confused your mail with one on another list. Checking the slabs may still be a good idea, though. Some of the caches may only be shrunk (garbage collected) periodically, or even only under memory pressure. Imagine the kernel really needs just a few bytes bytes of memory for assigning a new filehandle (though more likely 16 or 32 than 20): On a system with a GB of free RAM, it would seem strange to actually traverse the cache for an entry that can be reused (and keep your app waiting), instead of allocating a new one right away and garbage collecting free entries later occasionally. Is the behavior you observe an actual problem? > Regarding things that don't really make sense in sample code: > - In the real application, we're getting the file size by doing > fseek(ptr, 0, SEEK_END); > fileSize = ftell(ptr); // This line was omitted in the sample code > fseek(ptr, 0, SEEK_SET); > - The "&&" was something that my internet browser did, it's supposed to be just "&&" Ah. Ok. Regards, Stephan > - print("Success\n") instead of printf("Success") was a cut-n-paste error. > > Khoa. > > > > From: Stephan Wiesand <[email protected]> > To: [email protected] > Cc: Khoa To <[email protected]> > Date: 06/21/2010 10:56 AM > Subject: Re: Mem leak with fopen/fseek/fclose ? > > > > > > On Jun 21, 2010, at 3:12, Khoa To wrote: > > > I observed a memory leak (about 20 bytes per iteration) when I ran this > > > > simple program on my Scientific Linux 5.4 box (kernel version 2.6.18- > > 128.1.1.e15). > > Pretty old kernel (the one from the 5.3 initial release). What's the AFS client version? (output of "rxdebug localhost 7001 -v") > > > What I did was noting the output of "free -m" before and after I ran the > > > > program (which loops for 300000). I observed a 6MB increase in memory use > > d. > > No error was printed during the run. > > > > If I comment out the 2 fseek statements, then I don't see any memory > > increase. > > Check the amount of memory used for afs_inode_cache before and after the run (using slabtop). Does the difference account for the "leak"? > > > > The file size is about 700KB. > > > > I was wondering if someone can tell me if this is a known issue or if I'm > > > > doing something wrong. > > Well, the code below is not what you're actually running since amp is undefined. And the whole test for the while() loop doesn't make too much sense to me (although the first statement could probably simply be omitted). > > Regards, > Stephan > > > > > Thanks, > > Khoa. > > > > ======Source file, compiled with gcc========= > > > > #include <stdio.h> > > #include <errno.h> > > > > int main() > > { > > FILE * ptr = NULL; > > int count = 0; > > > > ptr = fopen("myBinaryFile","r"); > > while (ptr != NULL && count++ < 300000) > > { > > usleep(1000); > > > > if (fseek(ptr, 0, SEEK_END)) > > printf("Error 1: %d\n", errno); > > if (fseek(ptr, 0, SEEK_SET)) > > printf("Error 2: %d\n", errno); > > > > if (fclose(ptr)) > > break; > > > > usleep(10000); > > > > ptr = fopen("myBinaryFile","r"); > > } > > > > fclose(ptr); > > > > if (count >= 300000) > > print("Success\n"); > > > > return 0; > > } > > -- > Stephan Wiesand > DESY -DV- > Platanenallee 6 > 15738 Zeuthen, Germany > > > > > > -- Stephan Wiesand DESY -DV- Platanenenallee 6 15738 Zeuthen, Germany
