Hi Philip, Thanks for the report. Would you mind coping it to https://bugs.chromium.org/p/v8/issues/entry so that is tracked properly?
V8 is trying to grow the heap as expected, but one of the fs stream operations is allocating memory outside the heap and reporting it to V8 using AdjustAmountOfExternalAllocatedMemory. GCs are triggered to free the external memory by collecting dead JS objects that reference it. I think we can improve the heuristics for triggering GCs on external memory pressure. Cheers, Ulan. On Mon, Sep 9, 2019 at 7:21 AM Jakob Gruber <jgru...@chromium.org> wrote: > +Ulan Degenbaev <u...@google.com> > > On Thu, Sep 5, 2019 at 9:06 PM Philip Zeyliger < > philip.zeyli...@airtable.com> wrote: > >> Hi, >> >> I'm running into what seems to be a lot of v8 GC activity when piping a >> 1GB file to /dev/null. The performance varies considerably depending on >> whether or not I have an empty heap or one that I've filled to the tune of >> 500MB. This is happening within node, but everything I can find is pointing >> to some interaction between Node's IO and v8 garbage collection. >> >> My reproduction is at >> https://gist.githubusercontent.com/philz/3e55a1a3377797d1fbc47b10a219ec6f/raw/1a6faadd60a0b920db08b39b759a2065bcd25831/slow.js; >> running this with node will see the first 5 iterations happen more quickly >> than the second 5 iterations. >> >> Looking at --trace_gc, both the "slow" and "fast" paths do ~80 mark-sweep >> collections, but, when the heap is small, these take ~1-2ms whereas in the >> slow case (bigger heap) they take ~200ms. >> >> Example slow: [2417:0x55f66b6913b0] 6858 ms: Mark-sweep 5.5 (11.0) -> >> 4.3 (11.0) MB, 1.0 / 0.0 ms (+ 2.4 ms in 4 steps since start of marking, >> biggest step 1.0 ms, walltime since start of marking 3 ms) finalize >> incremental marking via task GC in old space requested >> Example fast: [2417:0x55f66b6913b0] 35062 ms: Mark-sweep 770.9 (888.5) >> -> 769.6 (888.5) MB, 2.8 / 0.0 ms (+ 224.0 ms in 247 steps since start of >> marking, biggest step 3.3 ms, walltime since start of marking 227 ms) >> finalize incremental marking via task GC in old space requested >> >> It's not surprising that mark-sweep takes longer on a bigger heap, but >> it's reclaiming very little space, and I'd have expected it to either grow >> the heap or use an incremental collector. >> >> I'd appreciate any suggestions you may have! (The real use case here >> involves JSON.stringify() and uploading to S3 using the AWS SDK, but this >> is the whittled down mystery.) >> >> Thanks, >> >> -- Philip >> >> -- >> -- >> v8-users mailing list >> v8-users@googlegroups.com >> http://groups.google.com/group/v8-users >> --- >> You received this message because you are subscribed to the Google Groups >> "v8-users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to v8-users+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/v8-users/c25cc22b-9c76-4657-bb45-927238bff613%40googlegroups.com >> <https://groups.google.com/d/msgid/v8-users/c25cc22b-9c76-4657-bb45-927238bff613%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- -- v8-users mailing list v8-users@googlegroups.com http://groups.google.com/group/v8-users --- You received this message because you are subscribed to the Google Groups "v8-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to v8-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/v8-users/CABNJt2LZ__qqkme0n7o0PcCZhFnky7OYkSWru6XcBG07Mb4EfQ%40mail.gmail.com.
