Re: [v8-users] ArrayBuffer BackingStore mutex causing high system load

['Ulan Degenbaev' via v8-users]

Hi Andrew!

Could you please file a bug at crbug.com/v8 with steps to reproduce? We
will take a look and see if we can optimize this case.

Until it is fixed, I think caching the backing store pointer in the
internal field would work. Then you'd need to ensure that you keep a handle
to ArrayBuffer while using the pointer (to prevent ArrayBuffer from being
garbage collected). Additionally, the pointer may become invalid if
ArrayBuffer is detached. You can detect that case by checking ByteLength()
== 0.

Cheers,
Ulan.

On Sun, Oct 18, 2020 at 12:45 AM billywhizz  wrote:

> Hi,
>
> I have put some details here on an issue i have come across while doing
> some benchmarking:
> https://gist.github.com/billywhizz/ff4c83c37142198d2e70992e438bf045
>
> I have a few questions around this:
>
> 1) Do we need a pthread_mutex_lock and pthread_mutex_unlock in every
> access to ArrayBuffer->GetBackingStore? In the absence of a deep dive into
> the code (which I will do soon hopefully) I am guessing this is in case the
> GC, which is running on another thread, has moved the backing data
> elsewhere.
>
> 2) If i want to access backing data at high frequency from JS (e.g. doing
> lots of writing or reading of an arraybuffer on a socket) is it possible
> (or advisable) to do something like this in order to avoid the lock
> contention in GetBackingStore?
> https://gist.github.com/billywhizz/ff4c83c37142198d2e70992e438bf045#file-test-cc.
> i.e. cache the pointer to the backing data in an internal field, assuming
> it will not be moved.
>
> 3) If the approach in 2) is not possible, is there a known approach to
> this problem which can avoid all the syscall overhead of locking the mutex?
>
> If you need more info or something easy to reproduce please let me know.
>
> Many Thanks,
>
> Andrew
>
> --
> --
> v8-users mailing list
> v8-users@googlegroups.com
> http://groups.google.com/group/v8-users
> ---
> You received this message because you are subscribed to the Google Groups
> "v8-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to v8-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/v8-users/a4ea069b-4271-4590-af99-4446a6a041e4n%40googlegroups.com
> 
> .
>

-- 
-- 
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
--- 
You received this message because you are subscribed to the Google Groups 
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to v8-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/v8-users/CABNJt2LxOnoPKOjKss-xMo0Dcxcg6HqsQfmMRGwMjF2CfSB1Gw%40mail.gmail.com.

Re: [v8-users] Is it safe to take heap snapshots in the NearHeapLimitCallback?

['Ulan Degenbaev' via v8-users]

> Which one would be more preferable?
Either option would work. Please note that there is also a new-large-object
space that is also a part of the young generation. Perhaps, a simpler
overapproximation would be to double the current heap limit? That is
definitely larger than the young generation size.

> Also, is it  safe to assume that the GC triggered by snapshot generation
will not invoke the NearHeapLimitCallback more than once?
I think so. If the limit is increased sufficiently, then the
NearHeapLimitCallback more than once by snapshot GCs.

Cheers,
Ulan.


On Thu, Apr 23, 2020 at 2:56 PM Joyee Cheung  wrote:

> Hi Ulan,
>
> Thanks for the explanation!
>
> > If the second callback does not increase the heap limit sufficiently
> (e.g. max young generation size ~ 32MB by default) then the GC may crash
> with OOM because it may need to promote young objects into the old
> generation, which checks the limit.
>
> So IIUC, the safe thing to do should be to return a new limit that's
> initial_heap_limit + max_young_generation_size in the second callback. From
> what I can tell, there should two ways to get to the young generation size:
>
> - Save the max_young_generation_size_in_bytes() from the resource
> constraints used to create the isolate.
> - Do a GetHeapSpaceStatistics() in the callback, and filter out the one
> with the name "new_space", then use the space_size() of that particular
> item.
>
> Which one would be more preferable? My guess is the first one might be
> larger than the second one, but probably not by lot, and it saves the
> filtering code and  thus is more reliable.
>
> Also, is it  safe to assume that the GC triggered by snapshot generation
> will not invoke the NearHeapLimitCallback more than once? Or should I guard
> against more potential invocations?
>
> Joyee
>
> On Thursday, April 23, 2020 at 3:01:42 PM UTC+8, Ulan Degenbaev wrote:
>>
>> Hi Joyee,
>>
>> The heap snapshot indeed uses native memory. The only caveat with taking
>> heap snapshot inside NearHeapLimitCallback is that the callback may be
>> invoked by one of the GCs that are triggered by the heap snapshot
>> generator. So the callback has to be re-entrant (it seems that your PR is
>> already handling this case). If the second callback does not increase the
>> heap limit sufficiently (e.g. max young generation size ~ 32MB by default)
>> then the GC may crash with OOM because it may need to promote young objects
>> into the old generation, which checks the limit.
>>
>> Other than that it should be safe.
>>
>> Cheers,
>> Ulan.
>>
>> On Thu, Apr 23, 2020 at 1:50 AM Joyee Cheung  wrote:
>>
>>> I had been thinking that taking heap snapshots should not be viable
>>> since this also consumes memory, but it seems to work just fine when I
>>> tried to implement it in Node.js
>>> https://github.com/nodejs/node/pull/33010 - probably because taking the
>>> heap snapshot doesn't really consume that much memory from the V8 heap, it
>>> just mostly uses usual native memory ?
>>>
>>> And the fact that taking heap snapshots triggers GC seem to help
>>> generating multiple snapshots for users to compare, compared to writing
>>> just one when V8 is about to crash which is what
>>> https://github.com/blueconic/node-oom-heapdump does with the OOM
>>> callback. Though thinking about it a bit more, if node-oom-heapdump has
>>> been working, then at least taking heap snapshots is already safe even when
>>> the heap limit is already reached.
>>>
>>> My question is, is this safe? Or to what extent is this safe?
>>>
>>> --
>>> --
>>> v8-users mailing list
>>> v8-u...@googlegroups.com
>>> http://groups.google.com/group/v8-users
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "v8-users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to v8-u...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/v8-users/3bdb87b9-4833-4629-bf98-2ce5a66746cc%40googlegroups.com
>>> 
>>> .
>>>
>> --
> --
> v8-users mailing list
> v8-users@googlegroups.com
> http://groups.google.com/group/v8-users
> ---
> You received this message because you are subscribed to the Google Groups
> "v8-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to v8-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/v8-users/1b3c41d8-8c40-48b2-9086-ce96953c%40googlegroups.com
> 
> .
>

-- 
-- 
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
--- 
You received this message because you are subscribed to the Google Groups 
"v8-users" group.
To unsubscribe 

Re: [v8-users] Is it safe to take heap snapshots in the NearHeapLimitCallback?

['Ulan Degenbaev' via v8-users]

Hi Joyee,

The heap snapshot indeed uses native memory. The only caveat with taking
heap snapshot inside NearHeapLimitCallback is that the callback may be
invoked by one of the GCs that are triggered by the heap snapshot
generator. So the callback has to be re-entrant (it seems that your PR is
already handling this case). If the second callback does not increase the
heap limit sufficiently (e.g. max young generation size ~ 32MB by default)
then the GC may crash with OOM because it may need to promote young objects
into the old generation, which checks the limit.

Other than that it should be safe.

Cheers,
Ulan.

On Thu, Apr 23, 2020 at 1:50 AM Joyee Cheung  wrote:

> I had been thinking that taking heap snapshots should not be viable since
> this also consumes memory, but it seems to work just fine when I tried to
> implement it in Node.js https://github.com/nodejs/node/pull/33010 -
> probably because taking the heap snapshot doesn't really consume that much
> memory from the V8 heap, it just mostly uses usual native memory ?
>
> And the fact that taking heap snapshots triggers GC seem to help
> generating multiple snapshots for users to compare, compared to writing
> just one when V8 is about to crash which is what
> https://github.com/blueconic/node-oom-heapdump does with the OOM
> callback. Though thinking about it a bit more, if node-oom-heapdump has
> been working, then at least taking heap snapshots is already safe even when
> the heap limit is already reached.
>
> My question is, is this safe? Or to what extent is this safe?
>
> --
> --
> v8-users mailing list
> v8-users@googlegroups.com
> http://groups.google.com/group/v8-users
> ---
> You received this message because you are subscribed to the Google Groups
> "v8-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to v8-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/v8-users/3bdb87b9-4833-4629-bf98-2ce5a66746cc%40googlegroups.com
> 
> .
>

-- 
-- 
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
--- 
You received this message because you are subscribed to the Google Groups 
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to v8-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/v8-users/CABNJt2KgQwDV9TXnZSsFS%2BTk0Xzits41DHxHrNOT2bXEC9fSoQ%40mail.gmail.com.

Re: [v8-users] Performance mystery with largish heaps

['Ulan Degenbaev' via v8-users]

Hi Philip,

Thanks for the report. Would you mind coping it to
https://bugs.chromium.org/p/v8/issues/entry so that is tracked properly?

V8 is trying to grow the heap as expected, but one of the fs stream
operations is allocating memory outside the heap and reporting it to V8
using AdjustAmountOfExternalAllocatedMemory.
GCs are triggered to free the external memory by collecting dead JS objects
that reference it. I think we can improve the heuristics for triggering GCs
on external memory pressure.

Cheers,
Ulan.


On Mon, Sep 9, 2019 at 7:21 AM Jakob Gruber  wrote:

> +Ulan Degenbaev 
>
> On Thu, Sep 5, 2019 at 9:06 PM Philip Zeyliger <
> philip.zeyli...@airtable.com> wrote:
>
>> Hi,
>>
>> I'm running into what seems to be a lot of v8 GC activity when piping a
>> 1GB file to /dev/null. The performance varies considerably depending on
>> whether or not I have an empty heap or one that I've filled to the tune of
>> 500MB. This is happening within node, but everything I can find is pointing
>> to some interaction between Node's IO and v8 garbage collection.
>>
>> My reproduction is at
>> https://gist.githubusercontent.com/philz/3e55a1a3377797d1fbc47b10a219ec6f/raw/1a6faadd60a0b920db08b39b759a2065bcd25831/slow.js;
>> running this with node will see the first 5 iterations happen more quickly
>> than the second 5 iterations.
>>
>> Looking at --trace_gc, both the "slow" and "fast" paths do ~80 mark-sweep
>> collections, but, when the heap is small, these take ~1-2ms whereas in the
>> slow case (bigger heap) they take ~200ms.
>>
>> Example slow: [2417:0x55f66b6913b0] 6858 ms: Mark-sweep 5.5 (11.0) ->
>> 4.3 (11.0) MB, 1.0 / 0.0 ms  (+ 2.4 ms in 4 steps since start of marking,
>> biggest step 1.0 ms, walltime since start of marking 3 ms) finalize
>> incremental marking via task GC in old space requested
>> Example fast: [2417:0x55f66b6913b0]35062 ms: Mark-sweep 770.9 (888.5)
>> -> 769.6 (888.5) MB, 2.8 / 0.0 ms  (+ 224.0 ms in 247 steps since start of
>> marking, biggest step 3.3 ms, walltime since start of marking 227 ms)
>> finalize incremental marking via task GC in old space requested
>>
>> It's not surprising that mark-sweep takes longer on a bigger heap, but
>> it's reclaiming very little space, and I'd have expected it to either grow
>> the heap or use an incremental collector.
>>
>> I'd appreciate any suggestions you may have! (The real use case here
>> involves JSON.stringify() and uploading to S3 using the AWS SDK, but this
>> is the whittled down mystery.)
>>
>> Thanks,
>>
>> -- Philip
>>
>> --
>> --
>> v8-users mailing list
>> v8-users@googlegroups.com
>> http://groups.google.com/group/v8-users
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "v8-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to v8-users+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/v8-users/c25cc22b-9c76-4657-bb45-927238bff613%40googlegroups.com
>> 
>> .
>>
>

-- 
-- 
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
--- 
You received this message because you are subscribed to the Google Groups 
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to v8-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/v8-users/CABNJt2LZ__qqkme0n7o0PcCZhFnky7OYkSWru6XcBG07Mb4EfQ%40mail.gmail.com.

Re: [v8-users] V8 garbage collector doesnt collect deleted Mat instances using OpenCV.js

['Ulan Degenbaev' via v8-users]

Hi Zigmas,

Does the leak also reproduce in other browsers? If so, it might be a leak
in the application.

If the leak happens only in Chrome, would you mind filing a bug
https://bugs.chromium.org/p/chromium/issues/entry and providing steps to
reproduce the leak (e.g. all necessary js files that use Vue and include
opencv.js).

Thanks!
Ulan.

On Tue, Jul 9, 2019 at 7:35 PM Zigmas Slušnys 
wrote:

> the new link is https://jsfiddle.net/slushnys/vesx2yr0
>
> On Tuesday, July 9, 2019 at 7:04:23 PM UTC+3, Jakob Kummerow wrote:
>>
>> The fiddle is broken ("createFrames" is not defined, "frames.push" is not
>> a function, "deleteFrames" is never called; that's when I gave up); please
>> fix it such that it actually reproduces the issue.
>>
>> On Tue, Jul 9, 2019 at 5:52 PM Zigmas Slušnys 
>> wrote:
>>
>>> I am reading a video and storing the Mat instance clones of each frame
>>> to an array so I could play it back on demand and also analyze the frames.
>>> Once i process the video and store the frames in the array - the memory
>>> snapshot increases from 150Mb to 1000Mb within the JSArrayBufferData.
>>>
>>> I have to mention that i'm using Vue.js frontend framework to store the
>>> array within vue's data observable that's an array.
>>>
>>> Same thing happens also if I'm just creating a regular type of class
>>> object having an array in.
>>>
>>> I've tried deleting and setting each frame to zero just to see if that
>>> would release the memory, but nothing seems to work.
>>>
>>> I've got a simple example of this here: https://jsfiddle.net/9s51poyd/
>>>
>>>
>>> I would like to release the memory from chrome, because every time I
>>> analize video now or just copy these Mat instances into an array, memory
>>> footprint could go up to 9GB. As you can imagine that's not the desired
>>> outcome.
>>>
>>>
>>> Is there a way to release the memory, because garbage collection
>>> currently does not work in this case example.
>>>
>>> --
>>> --
>>> v8-users mailing list
>>> v8-u...@googlegroups.com
>>> http://groups.google.com/group/v8-users
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "v8-users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to v8-u...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/v8-users/e8ee6213-7ba5-48de-ad27-c80752b4a368%40googlegroups.com
>>> 
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> --
> --
> v8-users mailing list
> v8-users@googlegroups.com
> http://groups.google.com/group/v8-users
> ---
> You received this message because you are subscribed to the Google Groups
> "v8-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to v8-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/v8-users/21b4e5d0-b150-4c18-9018-a5c783c92b17%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
-- 
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
--- 
You received this message because you are subscribed to the Google Groups 
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to v8-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/v8-users/CABNJt2K2UjpybDkzCeQORc-pCm9LpaoEiMAZzQJnku7jqC4u-A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [v8-users] Behaviour of isolate->IdleNotificationDeadline()

['Ulan Degenbaev' via v8-users]

Here is the patch:
https://chromium-review.googlesource.com/c/v8/v8/+/1478695

On Wed, Feb 20, 2019 at 11:21 AM Ulan Degenbaev  wrote:

> Hi Wilfried,
>
> Thanks for the report. The "do-nothing" case should occur limited number
> of times and then switch to "done" [1].
> There seems to be a bug that can lead to unbounded "do-nothing".
>
> I think the "do-nothing" case is no longer useful, so we can simply merge
> it with the "done" case. I'll upload a patch for that.
>
> Cheers,
> Ulan.
>
> [1]:
> https://cs.chromium.org/chromium/src/v8/src/heap/gc-idle-time-handler.h?rcl=fee9be7abb565fc2f2ae7c20e7597bece4fc7144=123
>
> On Tue, Feb 19, 2019 at 6:02 PM Wilfried Gösgens 
> wrote:
>
>> Hi,
>> I saw that *IdleNotificationDeadline()* in term invokes
>> *Heap::PerformIdleTimeAction() (heap.cc), *
>> And in case of `action.type == DO_NOTHING` it will return false.
>> So ArangoDB kept on invoking IdleNotificationDeadline() without any
>> effect.
>>
>> Did we set up the preconditions wrong (so type wouldn't be DO_NOTHING
>> here?) or should there be in fact a
>>
>>* result = true;*
>>
>> Cheers,
>> Wilfried Goesgens
>>
>> --
>> --
>> v8-users mailing list
>> v8-users@googlegroups.com
>> http://groups.google.com/group/v8-users
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "v8-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to v8-users+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
-- 
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
--- 
You received this message because you are subscribed to the Google Groups 
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [v8-users] Behaviour of isolate->IdleNotificationDeadline()

['Ulan Degenbaev' via v8-users]

Hi Wilfried,

Thanks for the report. The "do-nothing" case should occur limited number of
times and then switch to "done" [1].
There seems to be a bug that can lead to unbounded "do-nothing".

I think the "do-nothing" case is no longer useful, so we can simply merge
it with the "done" case. I'll upload a patch for that.

Cheers,
Ulan.

[1]:
https://cs.chromium.org/chromium/src/v8/src/heap/gc-idle-time-handler.h?rcl=fee9be7abb565fc2f2ae7c20e7597bece4fc7144=123

On Tue, Feb 19, 2019 at 6:02 PM Wilfried Gösgens 
wrote:

> Hi,
> I saw that *IdleNotificationDeadline()* in term invokes
> *Heap::PerformIdleTimeAction() (heap.cc), *
> And in case of `action.type == DO_NOTHING` it will return false.
> So ArangoDB kept on invoking IdleNotificationDeadline() without any
> effect.
>
> Did we set up the preconditions wrong (so type wouldn't be DO_NOTHING
> here?) or should there be in fact a
>
>* result = true;*
>
> Cheers,
> Wilfried Goesgens
>
> --
> --
> v8-users mailing list
> v8-users@googlegroups.com
> http://groups.google.com/group/v8-users
> ---
> You received this message because you are subscribed to the Google Groups
> "v8-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to v8-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
-- 
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
--- 
You received this message because you are subscribed to the Google Groups 
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [v8-users] Re: OOM and AddNearHeapLimitCallback

['Ulan Degenbaev' via v8-users]

Hi Prakash,

I added the AddNearHeapLimitCallback API. The provided callback is invoked
when V8's heap is close to the heap limit specified by the
ResourceConstraints::max_old_space_size during V8 setup.

If the callback extends the heap limit by returning a number greater then
the provided current_heap_limit, then the application will continue running
until the new heap limit is reached. At that point the callback is invoked
again.
If the callback returns the current_heap_limit (or smaller number) then the
application will crash with OOM if it continues allocating.

The initial_heap_limit is the original limit that V8 was set up with. It is
provided for convenience, so that the callback can figure out if it has
already extended the limit without maintaining state.

The callback is called on the same thread as the running JavaScript. So
during callback invocation JavaScript is paused.

Overall, I think the API should work for your case.

Caveats:
- the callback may be invoked multiple times.
- if the callback does not extend the heap limit, then the application is
not guaranteed to OOM. For example, if the application stops allocating and
never reaches the heap limit.
- if the heap limit is extended by small number, then the application may
still crash with OOM. For example, if the application wants to allocate
10MB, but the limit is extended by 5MB.
- if V8 runs out of system memory without reaching the heap limit, then it
will crash with OOM without invoking the callback. For example, if
malloc/mmap fails there is no way to safely continue execution.

I hope that answers your questions. Please let me know if I missed anything.

Cheers,
Ulan.

On Tue, Jun 5, 2018 at 11:28 AM, Prakash Bailkeri <
prakash.bailk...@gmail.com> wrote:

> Thanks for the response.
>
> Any advice on usage of "AddNearHeapLimitCallback"? [Please see my sample
> program]
>
> Thanks, Prakash
>
>
> --
> --
> v8-users mailing list
> v8-users@googlegroups.com
> http://groups.google.com/group/v8-users
> ---
> You received this message because you are subscribed to the Google Groups
> "v8-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to v8-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
-- 
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
--- 
You received this message because you are subscribed to the Google Groups 
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [v8-users] Re: How incremental can incremental marking be?

['Ulan Degenbaev' via v8-users]

I filed https://github.com/nodejs/node/issues/19937

On Wed, Apr 11, 2018 at 12:03 PM, Ulan Degenbaev  wrote:

> Thank you, John.
>
> This looks like an unfortunate interaction of V8 incremental marker with
> NodeJS's implementation of v8::Platform.
>
> The incremental marker posts a 1ms task that does a marking step and
> reposts itself (using  v8::Platform::CallOnForegroundThread).
>
> This works well in Chrome, but in NodeJS this results in effectively
> non-incremental marking because v8::Platform implementation in NodeJS
> drains
> 
> the foreground queue all at once until the queue becomes empty:
> while (std::unique_ptr task = foreground_tasks_.Pop()) {
>did_work = true;
>RunForegroundTask(std::move(task));
> }
>
> Running the incremental marking tasks results in adding a new task to
> foreground_tasks_:
> void PerIsolatePlatformData::PostTask(std::unique_ptr task) {
>   foreground_tasks_.Push(std::move(task));
>   uv_async_send(flush_tasks_);
> }
>
> I see two possible fixes:
> 1) Limit the task draining loop to only run tasks that were in the queue
> at the beginning of the loop.
> 2) Add time based heuristics to incremental marker to stop rescheduling
> tasks or to alternate delayed tasks with normal tasks.
>
> I'll check with NodeJS team if option 1 is possible because option 2 seems
> a bit hacky.
>
> Cheers,
> Ulan.
>
>
>
>
>
>
>
> On Wed, Apr 11, 2018 at 11:00 AM, Hannes Payer 
> wrote:
>
>> +Ulan Degenbaev 
>>
>> On Wed, Apr 11, 2018 at 10:57 AM  wrote:
>>
>>> Thanks John for your detailed report.
>>>
>>> You are right, incremental marking steps should ideally be spread out
>>> over the whole program execution. It seems like the incremental marking
>>> task is the only task running in the observed trace. Do you have any
>>> JavaScript work pending? JavaScript should take turns with incremental
>>> marking work. Is your workload somehow reproducible for us?
>>>
>>> We have some good news for you: V8 6.4 introduces concurrent marking
>>> which moves most of the marking work on a concurrent thread. I would be
>>> interested in seeing how your workload performance changes with a more
>>> recent V8 version.
>>>
>>>
>>> On Wednesday, April 11, 2018 at 10:17:01 AM UTC+2, John Edwards wrote:

 Hi all! Just want to lead with a quick thanks to everyone who makes v8
 happen! It's been a super useful piece of technology for me for many years.
 That said, it can still be quite mysterious, and, most recently, I've been
 trying to understand a bit more about the incremental marking part of the
 garbage collector.

 I'm working on the server for a competitive online game using node.js
 (primarily version 8.11.1 with v8 version 6.2.414.50). The server updates
 the game state around 100 times a second, and normally takes less than 1ms
 per update. Occasionally, though, we were seeing 16-50ms pauses between
 frames, which temporarily degraded the player experience. The first suspect
 was the garbage collector, but on my initial (incorrect) read, I didn't see
 anything taking over a few milliseconds. This log of a mark-sweep pass
 appeared to me to only take 2.1ms, for instance:

 [2944:0x356a660] 76591 ms: Mark-sweep 20.2 (40.9) -> 12.5 (40.9) MB,
 2.1 / 0.0 ms (+ 26.1 ms in 18 steps since start of marking, biggest
 step 2.3 ms, walltime since start of marking 29 ms) finalize
 incremental marking via task GC in old space requested

 After much confusion, I eventually stumbled on the
 --trace_incremental_marking flag, which gave the following output:

 [2944:0x356a660] 76562 ms: [IncrementalMarking] Start (idle task): old
 generation 19MB, limit 27MB, slack 8MB
 [2944:0x356a660] 76562 ms: [IncrementalMarking] Start marking
 [2944:0x356a660] 76562 ms: [IncrementalMarking] Running
 [2944:0x356a660] 76564 ms: [IncrementalMarking] Step in task 827176
 bytes (827100) in 1.9
 [2944:0x356a660] 76566 ms: [IncrementalMarking] Step in task 827136
 bytes (827100) in 1.9
 [2944:0x356a660] 76567 ms: [IncrementalMarking] Step in task 827216
 bytes (827100) in 1.1
 [2944:0x356a660] 76569 ms: [IncrementalMarking] Step in task 878480
 bytes (827100) in 1.5
 [2944:0x356a660] 76570 ms: [IncrementalMarking] Step in task 827168
 bytes (827100) in 1.1
 [2944:0x356a660] 76571 ms: [IncrementalMarking] Step in task 827184
 bytes (827100) in 1.0
 [2944:0x356a660] 76573 ms: [IncrementalMarking] Step in task 849536
 bytes (827100) in 1.7
 [2944:0x356a660] 76575 ms: [IncrementalMarking] Step in task 827104
 bytes (827100) in 2.2
 [2944:0x356a660] 76576 ms: [IncrementalMarking] Step in task 857840
 bytes (827100) in 1.1
 [2944:0x356a660] 76577 ms: [IncrementalMarking] 

Re: [v8-users] Re: How incremental can incremental marking be?

['Ulan Degenbaev' via v8-users]

Thank you, John.

This looks like an unfortunate interaction of V8 incremental marker with
NodeJS's implementation of v8::Platform.

The incremental marker posts a 1ms task that does a marking step and
reposts itself (using  v8::Platform::CallOnForegroundThread).

This works well in Chrome, but in NodeJS this results in effectively
non-incremental marking because v8::Platform implementation in NodeJS drains

the foreground queue all at once until the queue becomes empty:
while (std::unique_ptr task = foreground_tasks_.Pop()) {
   did_work = true;
   RunForegroundTask(std::move(task));
}

Running the incremental marking tasks results in adding a new task to
foreground_tasks_:
void PerIsolatePlatformData::PostTask(std::unique_ptr task) {
  foreground_tasks_.Push(std::move(task));
  uv_async_send(flush_tasks_);
}

I see two possible fixes:
1) Limit the task draining loop to only run tasks that were in the queue at
the beginning of the loop.
2) Add time based heuristics to incremental marker to stop rescheduling
tasks or to alternate delayed tasks with normal tasks.

I'll check with NodeJS team if option 1 is possible because option 2 seems
a bit hacky.

Cheers,
Ulan.







On Wed, Apr 11, 2018 at 11:00 AM, Hannes Payer  wrote:

> +Ulan Degenbaev 
>
> On Wed, Apr 11, 2018 at 10:57 AM  wrote:
>
>> Thanks John for your detailed report.
>>
>> You are right, incremental marking steps should ideally be spread out
>> over the whole program execution. It seems like the incremental marking
>> task is the only task running in the observed trace. Do you have any
>> JavaScript work pending? JavaScript should take turns with incremental
>> marking work. Is your workload somehow reproducible for us?
>>
>> We have some good news for you: V8 6.4 introduces concurrent marking
>> which moves most of the marking work on a concurrent thread. I would be
>> interested in seeing how your workload performance changes with a more
>> recent V8 version.
>>
>>
>> On Wednesday, April 11, 2018 at 10:17:01 AM UTC+2, John Edwards wrote:
>>>
>>> Hi all! Just want to lead with a quick thanks to everyone who makes v8
>>> happen! It's been a super useful piece of technology for me for many years.
>>> That said, it can still be quite mysterious, and, most recently, I've been
>>> trying to understand a bit more about the incremental marking part of the
>>> garbage collector.
>>>
>>> I'm working on the server for a competitive online game using node.js
>>> (primarily version 8.11.1 with v8 version 6.2.414.50). The server updates
>>> the game state around 100 times a second, and normally takes less than 1ms
>>> per update. Occasionally, though, we were seeing 16-50ms pauses between
>>> frames, which temporarily degraded the player experience. The first suspect
>>> was the garbage collector, but on my initial (incorrect) read, I didn't see
>>> anything taking over a few milliseconds. This log of a mark-sweep pass
>>> appeared to me to only take 2.1ms, for instance:
>>>
>>> [2944:0x356a660] 76591 ms: Mark-sweep 20.2 (40.9) -> 12.5 (40.9) MB, 2.1
>>> / 0.0 ms (+ 26.1 ms in 18 steps since start of marking, biggest step 2.3
>>> ms, walltime since start of marking 29 ms) finalize incremental marking
>>> via task GC in old space requested
>>>
>>> After much confusion, I eventually stumbled on the
>>> --trace_incremental_marking flag, which gave the following output:
>>>
>>> [2944:0x356a660] 76562 ms: [IncrementalMarking] Start (idle task): old
>>> generation 19MB, limit 27MB, slack 8MB
>>> [2944:0x356a660] 76562 ms: [IncrementalMarking] Start marking
>>> [2944:0x356a660] 76562 ms: [IncrementalMarking] Running
>>> [2944:0x356a660] 76564 ms: [IncrementalMarking] Step in task 827176
>>> bytes (827100) in 1.9
>>> [2944:0x356a660] 76566 ms: [IncrementalMarking] Step in task 827136
>>> bytes (827100) in 1.9
>>> [2944:0x356a660] 76567 ms: [IncrementalMarking] Step in task 827216
>>> bytes (827100) in 1.1
>>> [2944:0x356a660] 76569 ms: [IncrementalMarking] Step in task 878480
>>> bytes (827100) in 1.5
>>> [2944:0x356a660] 76570 ms: [IncrementalMarking] Step in task 827168
>>> bytes (827100) in 1.1
>>> [2944:0x356a660] 76571 ms: [IncrementalMarking] Step in task 827184
>>> bytes (827100) in 1.0
>>> [2944:0x356a660] 76573 ms: [IncrementalMarking] Step in task 849536
>>> bytes (827100) in 1.7
>>> [2944:0x356a660] 76575 ms: [IncrementalMarking] Step in task 827104
>>> bytes (827100) in 2.2
>>> [2944:0x356a660] 76576 ms: [IncrementalMarking] Step in task 857840
>>> bytes (827100) in 1.1
>>> [2944:0x356a660] 76577 ms: [IncrementalMarking] Step in task 1038576
>>> bytes (827100) in 0.8
>>> [2944:0x356a660] 76578 ms: [IncrementalMarking] Step in task 827160
>>> bytes (827100) in 0.7
>>> [2944:0x356a660] 76580 ms: [IncrementalMarking] Step in task 827128
>>> bytes (827100) in 2.0
>>> [2944:0x356a660] 76582