Re: [v8-users] V8 uses huge amounts of memory for "dictionary" objects with ~50 properties
Thanks for the fast reply. I did notice a big jump from 40 to 50 entries,
but I just figured some internal representation had changed due to the size
increasing. Your explanation makes a lot more sense. I agree that a 4x
growth rate is odd, I've usually seen 2x in C/C++ libs.
Normally I would use the constructor to initialize everything exactly as
you say, but I can't do it in this case -- there are about 300 or so
possible property names, but usually around 10 - 70 of them actually get
filled in per object. It's a somewhat odd use case, but it was still
surprising to see such high memory overhead compared to other browsers.
I'm glad to hear you're working on improving the dictionary-mode object,
hopefully that 4x growth bug can be fixed at some point too!
Russ
On Thursday, April 13, 2017 at 5:15:01 AM UTC-7, Jakob Kummerow wrote:
>
> We are aware that dictionary-mode objects are not particularly memory
> efficient right now, and are working on improving that.
>
> With the current implementation, every used entry in a property dictionary
> requires 3 words (=8 bytes each on 64-bit), one word each for the
> property's name, value, and details (writable/enumerable/configurable).
> Additionally, dictionaries grow when usage exceeds 2/3 of capacity; this
> is in order to keep the probability of collisions low.
> When they grow, they grow by a factor of 4x. (That smells like a bug; I
> would guess that the intention was to let them grow by 2x, but the
> multiplication is done twice.)
>
> What ends up happening is that for objects with 43 properties, the
> dictionary has capacity 64, consuming 64 * 3 (words per entry) * 8 (bytes
> per word) = 1536 bytes (+ 7 words metadata). When you add the 44th
> property, it is grown to capacity 256, now consuming 256 * 3 * 8 = 6144
> bytes (+7 words metadata). On the bright side, you can then grow your
> objects all the way up to 170 properties without increasing memory usage.
> So the overhead you're measuring will depend a lot on just how many
> properties you're adding.
>
>
> Side note: if you know the set of properties that will be added in advance
> (just not their order), then it might be a good idea to initialize all of
> them in the object's constructor, and fill in their actual values later, so
> that you'll benefit from faster and slimmer objects. Roughly:
> function MyObject() {
> this.prop1 = undefined;
> this.prop2 = undefined;
> // etc...
> }
> function OnNetworkReceived(object, prop, value) {
> object[prop] = value; // prop is "prop1" or "prop2" or ...
> }
>
>
> On Thu, Apr 13, 2017 at 1:20 PM, Russ P >
> wrote:
>
>> Hello,
>>
>> I've been working on a project that adds properties to objects in a
>> variable order as they come in over the network (hidden classes and similar
>> optimizations can't be applied). I've noticed that V8, on both Chrome and
>> Node, uses huge amounts of memory to represent them compared to other JS
>> engines. I'm testing on Windows.
>>
>> In 64-bit V8, an object with 50 simple properties with integer values
>> uses ~6000 bytes of memory, which works out to >100 bytes overhead per
>> entry. 32-bit is somewhat better at ~3000 bytes per object and >45 bytes
>> overhead per entry, but it's still really bad.
>>
>> In comparison, the same 50-property objects were only ~900 bytes each in
>> Firefox and Edge. That's 3-6X less!
>>
>> I've included a test that shows the issue if you'd like to try it out:
>> https://repl.it/HIMc/7. Be sure to turn off infinite loop protection in
>> settings. After running it, you can force a gc and check the browser's
>> memory usage (divide by 20, that's the test # of objects).
>>
>> Anyone else seen similar problems? That seems like a crazy amount of
>> overhead per property.
>>
>> -Russ
>>
>> --
>> --
>> v8-users mailing list
>> v8-u...@googlegroups.com
>> http://groups.google.com/group/v8-users
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "v8-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to v8-users+u...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
--
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
---
You received this message because you are subscribed to the Google Groups
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [v8-users] V8 uses huge amounts of memory for "dictionary" objects with ~50 properties
We are aware that dictionary-mode objects are not particularly memory
efficient right now, and are working on improving that.
With the current implementation, every used entry in a property dictionary
requires 3 words (=8 bytes each on 64-bit), one word each for the
property's name, value, and details (writable/enumerable/configurable).
Additionally, dictionaries grow when usage exceeds 2/3 of capacity; this is
in order to keep the probability of collisions low.
When they grow, they grow by a factor of 4x. (That smells like a bug; I
would guess that the intention was to let them grow by 2x, but the
multiplication is done twice.)
What ends up happening is that for objects with 43 properties, the
dictionary has capacity 64, consuming 64 * 3 (words per entry) * 8 (bytes
per word) = 1536 bytes (+ 7 words metadata). When you add the 44th
property, it is grown to capacity 256, now consuming 256 * 3 * 8 = 6144
bytes (+7 words metadata). On the bright side, you can then grow your
objects all the way up to 170 properties without increasing memory usage.
So the overhead you're measuring will depend a lot on just how many
properties you're adding.
Side note: if you know the set of properties that will be added in advance
(just not their order), then it might be a good idea to initialize all of
them in the object's constructor, and fill in their actual values later, so
that you'll benefit from faster and slimmer objects. Roughly:
function MyObject() {
this.prop1 = undefined;
this.prop2 = undefined;
// etc...
}
function OnNetworkReceived(object, prop, value) {
object[prop] = value; // prop is "prop1" or "prop2" or ...
}
On Thu, Apr 13, 2017 at 1:20 PM, Russ P wrote:
> Hello,
>
> I've been working on a project that adds properties to objects in a
> variable order as they come in over the network (hidden classes and similar
> optimizations can't be applied). I've noticed that V8, on both Chrome and
> Node, uses huge amounts of memory to represent them compared to other JS
> engines. I'm testing on Windows.
>
> In 64-bit V8, an object with 50 simple properties with integer values uses
> ~6000 bytes of memory, which works out to >100 bytes overhead per entry.
> 32-bit is somewhat better at ~3000 bytes per object and >45 bytes overhead
> per entry, but it's still really bad.
>
> In comparison, the same 50-property objects were only ~900 bytes each in
> Firefox and Edge. That's 3-6X less!
>
> I've included a test that shows the issue if you'd like to try it out:
> https://repl.it/HIMc/7. Be sure to turn off infinite loop protection in
> settings. After running it, you can force a gc and check the browser's
> memory usage (divide by 20, that's the test # of objects).
>
> Anyone else seen similar problems? That seems like a crazy amount of
> overhead per property.
>
> -Russ
>
> --
> --
> v8-users mailing list
> v8-users@googlegroups.com
> http://groups.google.com/group/v8-users
> ---
> You received this message because you are subscribed to the Google Groups
> "v8-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to v8-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
--
--
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
---
You received this message because you are subscribed to the Google Groups
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[v8-users] V8 uses huge amounts of memory for "dictionary" objects with ~50 properties
Hello, I've been working on a project that adds properties to objects in a variable order as they come in over the network (hidden classes and similar optimizations can't be applied). I've noticed that V8, on both Chrome and Node, uses huge amounts of memory to represent them compared to other JS engines. I'm testing on Windows. In 64-bit V8, an object with 50 simple properties with integer values uses ~6000 bytes of memory, which works out to >100 bytes overhead per entry. 32-bit is somewhat better at ~3000 bytes per object and >45 bytes overhead per entry, but it's still really bad. In comparison, the same 50-property objects were only ~900 bytes each in Firefox and Edge. That's 3-6X less! I've included a test that shows the issue if you'd like to try it out: https://repl.it/HIMc/7. Be sure to turn off infinite loop protection in settings. After running it, you can force a gc and check the browser's memory usage (divide by 20, that's the test # of objects). Anyone else seen similar problems? That seems like a crazy amount of overhead per property. -Russ -- -- v8-users mailing list v8-users@googlegroups.com http://groups.google.com/group/v8-users --- You received this message because you are subscribed to the Google Groups "v8-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to v8-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
