Re: [v8-users] JavaScript heap out of memory issue

[Abhishek Singh]

Hi Ben,

It turned out to be memory leak in my C++ wrapper on top of V8 - fixed it. 
Thanks for the suggestion around using v8::HeapProfiler::TakeHeapSnapshot().

> On 24-Sep-2016, at 00:08, Ben Noordhuis  wrote:
> 
> On Fri, Sep 23, 2016 at 12:54 PM, Abhishek Singh
>  wrote:
>> Hi,
>> 
>> I noticed a V8 crash with following message:
>> 
>> 
>> <--- Last few GCs --->
>> 
>>  251206 ms: Mark-sweep 1400.2 (1456.1) -> 1400.2 (1456.1) MB, 3688.5 / 0.0 
>> ms [allocation failure] [GC in old space requested].
>>  254854 ms: Mark-sweep 1400.2 (1456.1) -> 1400.2 (1456.1) MB, 3648.2 / 0.0 
>> ms [allocation failure] [GC in old space requested].
>>  258562 ms: Mark-sweep 1400.2 (1456.1) -> 1400.2 (1426.1) MB, 3707.6 / 0.0 
>> ms [last resort gc].
>>  262315 ms: Mark-sweep 1400.2 (1426.1) -> 1400.2 (1426.1) MB, 3752.6 / 0.0 
>> ms [last resort gc].
>> 
>> 
>> <--- JS stacktrace --->
>> 
>>  JS stack trace =
>> 
>> Security context: 0x3c20e87ca141 
>>1: OnHTTPGet [0x3c20e8704311 :~64] 
>> [pc=0x328074967a8e](this=0x94116b170d1 ,req=0x6ba99d91ed1 
>> ,res=0x6ba99d91f29 )
>> 
>>  Details 
>> 
>> [1]: OnHTTPGet [0x3c20e8704311 :~64] 
>> [pc=0x328074967a8e](this=0x94116b170d1 ,req=0x6ba99d91ed1 
>> > 
>> 
>> What the application does is - it executes JS function call when an end user 
>> fires a HTTP request and response object is written back to the end user. 
>> Call volume was around 10K HTTP requests/sec(JS code being executed is 
>> trivial) - snippet of it:
>> 
>> function OnHTTPGet(req, res) {
>>  res.body.msg = “Hello world”; // res is nested map exposed from C++ to JS 
>> world
>> }
>> 
>> ===
>> 
>> I tried simulating behaviour on a standalone environment and could replicate 
>> the problem, here is code to do that: 
>> https://gist.github.com/abhi-bit/505538ac45d4b11014c3681e3bb74a2b
>> 
>> What I noticed that my machine has probably 4GB of memory left but V8’s 
>> memory footprint was around 1.6GB before the crash. To summarise my 
>> questions:
>> 
>> Is 10K request/sec(JS function calls/sec) too much for GC to handle in this 
>> scenario? Any tunable to control this behaviour or am I doing wrong in 
>> standalone code?
> 
> Modulo bugs in the garbage collector the request rate shouldn't
> matter.  Your application can not allocate faster than the collector
> can reclaim because it will simply block your application until it
> catches up.  Not necessarily true for a truly concurrent GC but V8's
> is still stop-the-world when it has to be.
> 
> A more likely cause is that you have a memory leak somewhere.  Try
> making snapshots of the heap that you can inspect in Chrome DevTools;
> see HeapProfiler::TakeHeapSnapshot() in include/v8-profiler.h or
> https://github.com/bnoordhuis/node-heapdump for a more full-fledged
> example.
> 
>> Why V8 mem_size could only grow upto 1.6GB, even when system has 4GB of 
>> memory? I’ve seen similar problem in couchdb(map-reduce index based on top 
>> of V8) as well, where app V8 binding crashed when footprint when close to 
>> 1.6GB(but that with V8 3.23) - error dump looked like this:
> 
> The default limit is ~1.6 GB.  You can turn it up with
> --max_old_space_size or through
> Isolate::CreateParams::constraints::set_max_old_space_size().
> 
> -- 
> -- 
> v8-users mailing list
> v8-users@googlegroups.com
> http://groups.google.com/group/v8-users
> --- 
> You received this message because you are subscribed to the Google Groups 
> "v8-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to v8-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
-- 
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
--- 
You received this message because you are subscribed to the Google Groups 
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [v8-users] JavaScript heap out of memory issue

[Ben Noordhuis]

On Fri, Sep 23, 2016 at 12:54 PM, Abhishek Singh
 wrote:
> Hi,
>
> I noticed a V8 crash with following message:
>
>
> <--- Last few GCs --->
>
>   251206 ms: Mark-sweep 1400.2 (1456.1) -> 1400.2 (1456.1) MB, 3688.5 / 0.0 
> ms [allocation failure] [GC in old space requested].
>   254854 ms: Mark-sweep 1400.2 (1456.1) -> 1400.2 (1456.1) MB, 3648.2 / 0.0 
> ms [allocation failure] [GC in old space requested].
>   258562 ms: Mark-sweep 1400.2 (1456.1) -> 1400.2 (1426.1) MB, 3707.6 / 0.0 
> ms [last resort gc].
>   262315 ms: Mark-sweep 1400.2 (1426.1) -> 1400.2 (1426.1) MB, 3752.6 / 0.0 
> ms [last resort gc].
>
>
> <--- JS stacktrace --->
>
>  JS stack trace =
>
> Security context: 0x3c20e87ca141 
> 1: OnHTTPGet [0x3c20e8704311 :~64] 
> [pc=0x328074967a8e](this=0x94116b170d1 ,req=0x6ba99d91ed1 
> ,res=0x6ba99d91f29 )
>
>  Details 
>
> [1]: OnHTTPGet [0x3c20e8704311 :~64] 
> [pc=0x328074967a8e](this=0x94116b170d1 ,req=0x6ba99d91ed1 
> 
>
> What the application does is - it executes JS function call when an end user 
> fires a HTTP request and response object is written back to the end user. 
> Call volume was around 10K HTTP requests/sec(JS code being executed is 
> trivial) - snippet of it:
>
> function OnHTTPGet(req, res) {
>   res.body.msg = “Hello world”; // res is nested map exposed from C++ to JS 
> world
> }
>
> ===
>
> I tried simulating behaviour on a standalone environment and could replicate 
> the problem, here is code to do that: 
> https://gist.github.com/abhi-bit/505538ac45d4b11014c3681e3bb74a2b
>
> What I noticed that my machine has probably 4GB of memory left but V8’s 
> memory footprint was around 1.6GB before the crash. To summarise my questions:
>
> Is 10K request/sec(JS function calls/sec) too much for GC to handle in this 
> scenario? Any tunable to control this behaviour or am I doing wrong in 
> standalone code?

Modulo bugs in the garbage collector the request rate shouldn't
matter.  Your application can not allocate faster than the collector
can reclaim because it will simply block your application until it
catches up.  Not necessarily true for a truly concurrent GC but V8's
is still stop-the-world when it has to be.

A more likely cause is that you have a memory leak somewhere.  Try
making snapshots of the heap that you can inspect in Chrome DevTools;
see HeapProfiler::TakeHeapSnapshot() in include/v8-profiler.h or
https://github.com/bnoordhuis/node-heapdump for a more full-fledged
example.

> Why V8 mem_size could only grow upto 1.6GB, even when system has 4GB of 
> memory? I’ve seen similar problem in couchdb(map-reduce index based on top of 
> V8) as well, where app V8 binding crashed when footprint when close to 
> 1.6GB(but that with V8 3.23) - error dump looked like this:

The default limit is ~1.6 GB.  You can turn it up with
--max_old_space_size or through
Isolate::CreateParams::constraints::set_max_old_space_size().

-- 
-- 
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
--- 
You received this message because you are subscribed to the Google Groups 
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[v8-users] JavaScript heap out of memory issue

[Abhishek Singh]

Hi,

I noticed a V8 crash with following message:


<--- Last few GCs --->

  251206 ms: Mark-sweep 1400.2 (1456.1) -> 1400.2 (1456.1) MB, 3688.5 / 0.0 ms 
[allocation failure] [GC in old space requested].
  254854 ms: Mark-sweep 1400.2 (1456.1) -> 1400.2 (1456.1) MB, 3648.2 / 0.0 ms 
[allocation failure] [GC in old space requested].
  258562 ms: Mark-sweep 1400.2 (1456.1) -> 1400.2 (1426.1) MB, 3707.6 / 0.0 ms 
[last resort gc].
  262315 ms: Mark-sweep 1400.2 (1426.1) -> 1400.2 (1426.1) MB, 3752.6 / 0.0 ms 
[last resort gc].


<--- JS stacktrace --->

 JS stack trace =

Security context: 0x3c20e87ca141 
1: OnHTTPGet [0x3c20e8704311 :~64] 
[pc=0x328074967a8e](this=0x94116b170d1 ,req=0x6ba99d91ed1 ,res=0x6ba99d91f29 )

 Details 

[1]: OnHTTPGet [0x3c20e8704311 :~64] 
[pc=0x328074967a8e](this=0x94116b170d1 ,req=0x6ba99d91ed1 https://gist.github.com/abhi-bit/505538ac45d4b11014c3681e3bb74a2b 


What I noticed that my machine has probably 4GB of memory left but V8’s memory 
footprint was around 1.6GB before the crash. To summarise my questions:

Is 10K request/sec(JS function calls/sec) too much for GC to handle in this 
scenario? Any tunable to control this behaviour or am I doing wrong in 
standalone code?
Why V8 mem_size could only grow upto 1.6GB, even when system has 4GB of memory? 
I’ve seen similar problem in couchdb(map-reduce index based on top of V8) as 
well, where app V8 binding crashed when footprint when close to 1.6GB(but that 
with V8 3.23) - error dump looked like this:

[ns_server:info,2016-09-03T2:28:27.006,babysitter_of_ns_1@127.0.0.1:<0.68.0>:ns_port_server:log:169]ns_server<0.68.0>:
ns_server<0.68.0>: #
ns_server<0.68.0>: # Fatal error in CodeRange::GetNextAllocationBlock
ns_server<0.68.0>: # Allocation failed - process out of memory
ns_server<0.68.0>: #
ns_server<0.68.0>:

Thanks,
Abhishek


-- 
-- 
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
--- 
You received this message because you are subscribed to the Google Groups 
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.