Ok, i waited till ps showed around 50MB (still out-of-the box compiled node 0.8.1 without arguments)
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND nodejs 12655 0.0 2.4 731204 50528 pts/0 Sl+ Jul05 0:13 node /opt/turntable/bin/hook_luncher.js --hook stats at the same time i see debug output from nodes gc like this: 46965999 ms: Mark-sweep 8.7 (95.0) -> 8.3 (95.0) MB, 0 ms (+ 7 ms in 1 steps since start of marking, biggest step 7.866211 ms) [idle notification: finalize incremental] [GC in old space requested]. Memory allocator, used: 99598336, available: 1435516928 New space, used: 0, available: 1048576 Old pointers, used: 50052944, available: 4272, waste: 0 Old data space, used: 6388224, available: 0, waste: 0 Code space, used: 2606944, available: 1472672, waste: 0 Map space, used: 252056, available: 910952, waste: 0 Cell space, used: 98240, available: 1032000, waste: 0 Large object space, used: 0, available: 1434451712 46966009 ms: Mark-sweep 8.7 (95.0) -> 8.3 (95.0) MB, 0 ms (+ 7 ms in 1 steps since start of marking, biggest step 7.776855 ms) [idle notification: finalize incremental] [GC in old space requested]. Memory allocator, used: 99598336, available: 1435516928 New space, used: 0, available: 1048576 Old pointers, used: 50052944, available: 4272, waste: 0 Old data space, used: 6388224, available: 0, waste: 0 Code space, used: 2606944, available: 1472672, waste: 0 Map space, used: 252056, available: 910952, waste: 0 Cell space, used: 98240, available: 1032000, waste: 0 Large object space, used: 0, available: 1434451712 how does this output reflect the 50MB RSS? Can you see something that is wrong? i also posted the whole trace (from the beginning (-13h) till now) at http://pastebin.com/K1RmPKUC On Thursday, July 5, 2012 12:37:09 PM UTC+2, Vyacheslav Egorov wrote: > > --trace-gc-verbose should be used together with --trace-gc. > > When enabled should produce something like: > > 1087 ms: Scavenge 2.6 (20.4) -> 1.6 (20.4) MB, 0 ms [allocation > failure]. > Memory allocator, used: 21409792, available: 746147840 > New space, used: 0, available: 1048576 > Old pointers, used: 1277824, available: 0, waste: 0 > Old data space, used: 1173980, available: 38308, waste: 0 > Code space, used: 1396000, available: 736, waste: 0 > Map space, used: 131072, available: 0, waste: 0 > Cell space, used: 98304, available: 0, waste: 0 > Large object space, used: 0, available: 745066368 > > I need to see this kind of fine grained statistics to make a guess. > > You can also try to update V8 inside node from 3.11.10.12 to 3.11.10.13 so > that > CL improving memory usage > (https://chromiumcodereview.appspot.com/10699051) is included. > > -- > Vyacheslav Egorov > > > On Thu, Jul 5, 2012 at 9:40 AM, chusi wrote: > > > > the --trace-gc-verbose parameter didn't print anything (at least not in > the > > minute i waited) and until i reach the 50 MB (monitoring using > --trace-gc) > > at least some hours have to pass by. I will try it later. > > > > here's a pic of running the same code once with node v0.6.17 (chris > lea's > > packet) and node v.0.8.1 (my own compiled version). > > > > http://troll.ws/i/nJQJFr.png > > > > before I updated to the new node version i was fighting against some > memory > > leaks in my code, but was able to fix them. when i switched to v.0.8.1 I > was > > worried that they seemed to be back (when i saw the graph). besides the > "red > > line" that goes crazy, i guess the rest can be explained by the new more > > lazy garbage collection approach. > > > > btw, the app is almost idle. the different modules shown in the graph > just > > communicate from time to time with each other and only exchange tiny > bits of > > data. > > > > > > > > On Wednesday, July 4, 2012 5:33:34 PM UTC+2, Vyacheslav Egorov wrote: > >> > >> If you run you app with --trace-gc what does it print (I am interested > >> in GC statistics for the time when app reaches 50MB)? > >> > >> I am also curious in --trace-gc-verbose output to see how memory is > >> distributed between different spaces in V8? > >> > >> It might be that the fragmentation in the heap became higher due to > >> lazyness of the new collector (it is tunes for interactive performance > >> so it tries to do things in small increments). > >> > >> Also I am not sure what the current situation with idle notifications > >> in node is but they are very important for the new collector to > >> perform bigger steps when app is idle. > >> > >> Additionally there were some changes recently that increased > >> aggressiveness with which new collector sweeps and compacts things out > >> from almost empty pages. You might be being bitten by this issues. > >> > >> -- > >> Vyacheslav Egorov > >> > >> > >> On Wed, Jul 4, 2012 at 12:42 PM, Ben Noordhuis <i...@bnoordhuis.nl> > wrote: > >> > On Wed, Jul 4, 2012 at 11:39 AM, chusi wrote: > >> >> On Tuesday, July 3, 2012 7:53:26 PM UTC+2, Ben Noordhuis wrote: > >> >>> Sanity check - what do `file /path/to/old/node` and `file > >> >>> /path/to/new/node` print? > >> >> > >> >> old 0.6.17: > >> >> ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically > linked > >> >> (uses shared libs), for GNU/Linux 2.6.15, stripped > >> >> > >> >> new 0.8.1 > >> >> ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically > linked > >> >> (uses shared libs), for GNU/Linux 2.6.15, stripped > >> >> > >> >> node is installed using chis-lea's package as described here: > >> >> > >> >> > https://github.com/joyent/node/wiki/Installing-Node.js-via-package-manager > >> > > >> > Output from `file` looks good. If one binary was built for ia32 and > >> > the other for x64, you'd expect to see different memory profiles, but > >> > that's not the case here. > >> > > >> > I don't know if Chris lands patches on top in a way that changes > >> > memory consumption. Do you see the same behavior with a (tarball, > git) > >> > source build? > >> > > >> > One thing to note is that V8 has a new and improved (but also more > >> > lazy) garbage collector. Run your app with --expose-gc and call gc() > >> > from time to time if memory pressure is a real issue - but I would > >> > generally advise against that. > >> > > >> > -- > >> > Job Board: http://jobs.nodejs.org/ > >> > Posting guidelines: > >> > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > >> > You received this message because you are subscribed to the Google > >> > Groups "nodejs" group. > >> > To post to this group, send email to nodejs@googlegroups.com > >> > To unsubscribe from this group, send email to > >> > nodejs+unsubscr...@googlegroups.com > >> > For more options, visit this group at > >> > http://groups.google.com/group/nodejs?hl=en?hl=en > > > > -- > > Job Board: http://jobs.nodejs.org/ > > Posting guidelines: > > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > > You received this message because you are subscribed to the Google > > Groups "nodejs" group. > > To post to this group, send email to nodejs@googlegroups.com > > To unsubscribe from this group, send email to > > nodejs+unsubscr...@googlegroups.com > > For more options, visit this group at > > http://groups.google.com/group/nodejs?hl=en?hl=en > -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to nodejs@googlegroups.com To unsubscribe from this group, send email to nodejs+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en
