On Sat, Nov 23, 2013 at 2:31 AM, Steve Freegard <st...@stevefreegard.com> wrote: > On 23/11/13 01:02, Ben Noordhuis wrote: >> Right. I see that your test case creates 1M file watchers in a loop. >> Those won't be released right away and calling gc() won't change that. >> Disposing a file watcher takes two event loop ticks or more but in >> your test case, everything is done in a single tick. If you smear out >> the loop iterations with setImmediate(), you should see more >> consistent memory usage. > > Ok - I tried that. I also added a gc() call after the loop, followed by > a setTimeout() of 60 seconds with a further call to gc() and then set an > process.on('exit') handler to output process.memoryUsage() so I would > only get the output once all of the watchers were closed. > > I'm still seeing memory leaking: > > using fs.watch: > > [root@mta41 ~]# node --expose_gc --max_old_space_size=8 watchfile_leak.js > { rss: 8491008, heapTotal: 6163968, heapUsed: 1919544 } > { rss: 13017088, heapTotal: 13487360, heapUsed: 1966256 } > > using fs.watchFile: > > [root@mta41 ~]# node --expose_gc --max_old_space_size=8 watchfile_leak.js > { rss: 8495104, heapTotal: 6163968, heapUsed: 1919632 } > { rss: 13275136, heapTotal: 21875968, heapUsed: 1974984 } > > This is with 10,000 watchers being added and immediately removed, so the > process would not exit until the watchers have been terminated. > > Based on what we saw in Haraka - there is absolutely no doubt in my mind > that there is something amiss here. We were incorrectly calling > fs.unwatchFile() followed by fs.watchFile() thousands of times per > connection and the memory usage grew massively over a short space of > time. The moment I changed the code to avoid the calls to > fs.watchFile/unwatchFile we went from having an RSS of > 1Gb in about an > hour to around 100Mb. > > Cheers, > Steve.
I can't really reproduce that. Below is what IMO the test case reasonably should look like and that has RSS hovering around 60 MB with --max_old_space_size=64. Tested with HEAD of v0.10 and master on a x86_64 FC20 system. var fs = require('fs'); var n = 0; f(); function f() { if (++n % 1e4 === 0) stats(); fs.unwatchFile('test'); fs.watchFile('test', function() {}); fs.unwatchFile('test'); if (n < 1e6) setImmediate(f); } function stats() { var m = process.memoryUsage(); m.rss >>>= 10, m.heapUsed >>>= 10, m.heapTotal >>>= 10; console.log(m); } -- -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to nodejs@googlegroups.com To unsubscribe from this group, send email to nodejs+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en --- You received this message because you are subscribed to the Google Groups "nodejs" group. To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.