Re: [v8-users] BSON to v8::Object performance issue
Thank you!
Great help here. Amazing folks!
On Nov 17, 2017 6:21 PM, "J Decker" <d3c...@gmail.com> wrote:
>
>
> On Fri, Nov 17, 2017 at 7:25 AM, Jean-Marc Le Roux <
> jeanmarc.ler...@aerys.in> wrote:
>
>> My assumption is that my test app terminates before the GC has a chance
>> to run.
>> To test this:
>>
>>- I run node with the --expose-gc option
>>- I call global.gc() at the end of my test app
>>
>> In this case, the destructor of my ObjectWrap is indeed called.
>>
>> *So it seams the GC doesn't simply "collect everything" when the app
>> terminates.*
>> *Why is that?*
>>
>> It sounds like a pretty bad things to do: some behaviors might expect the
>> destructor to be called in order to close a socket, etc...
>>
>
> because all memory is released. when the app exits. Why do extra work of
> minor side effects?
> but ya, if tf you were returning like a file to a handle and expected the
> close in order to flush all data out to a device it's bad...
>
> You can attach to Node::AtExit( ... )
> https://nodejs.org/api/addons.html#addons_atexit_hooks
>
> which should be called so you an finalize such things.
>
> but sockets get closed by the system when a application exits also. and
> files are closed
>
>
>
>>
>> On Friday, November 17, 2017 at 10:27:22 AM UTC+1, Jean-Marc Le Roux
>> wrote:
>>>
>>> Thanks for the great help!
>>> I think it makes a lot of sense now.
>>>
>>> So I've extended ObjectWrap like so:
>>>
>>> class BSONObject : public node::ObjectWrap
>>>> {
>>>> private:
>>>> bson_t* _bson;
>>>> public:
>>>> BSONObject(bson_t* bson, Local tpl) :
>>>> _bson(bson)
>>>> {
>>>> Local obj = tpl->NewInstance();
>>>> initialize(v8::Isolate::GetCurrent(), obj, bson);
>>>> Wrap(obj);
>>>> }
>>>
>>>
>>>
>>> ~BSONObject()
>>>> {
>>>> std::cout << "~BSONObject()" << std::endl;
>>>> bson_destroy(_bson);
>>>> }
>>>
>>>
>>>
>>> static
>>>> Local
>>>> create_template(Isolate* isolate, const bson_t* document)
>>>> {
>>>> Local tpl = ObjectTemplate::New(isolate);
>>>> tpl->SetInternalFieldCount(1);
>>>
>>>
>>> And the "insert" function that I expose in JS and that instanciates the
>>> BSONObject is written like this:
>>>
>>> void
>>>> insert(const FunctionCallbackInfo& args)
>>>> {
>>>> Isolate* isolate = args.GetIsolate();
>>>>
>>>> db->insert(
>>>> json_stringify(isolate, args[0]),
>>>> [, isolate](const bson_t* document)
>>>> {
>>>> if (!args[1]->IsUndefined())
>>>> {
>>>> Local callback =
>>>> Local::Cast(args[1]);
>>>> Local obj = Object::New(isolate);
>>>> *Local argv[] = { (new
>>>> BSONObject(bson_copy(document)))->handle(isolate) };*
>>>> callback->Call(isolate->GetCurrentContext()->Global(),
>>>> 1, argv);
>>>> }
>>>> }
>>>> );
>>>> }
>>>
>>>
>>> It runs fine.
>>> But the destructor is never called (I never see "~BSONObject()" on
>>> stdout).
>>>
>>> Did I miss something?
>>>
>>> Thanks !
>>>
>>> 2017-11-16 22:01 GMT+01:00 J Decker <d3c...@gmail.com>:
>>>
>>>> You don't manage the instance, the v8 engine does. When it's no longer
>>>> referenced in the engine, it's garbage collected which triggers the
>>>> callback set when the perisstent object is made weak.
>>>>
>>>> You don't need to keep it anywhere, the object returned into the engine
>>>> has Internal Fields ( SetInternalFieldCount(1) ) which is used to point to
>>>> your native class instance. So when the callback is called, it knows what
>>>> to call to destruct it.
>>>>
>>>> You have to 'keep it' in the javascript scrpit if you want to keep it,
>>>> it's not 'kept' in the native code
Re: [v8-users] BSON to v8::Object performance issue
My assumption is that my test app terminates before the GC has a chance to
run.
To test this:
- I run node with the --expose-gc option
- I call global.gc() at the end of my test app
In this case, the destructor of my ObjectWrap is indeed called.
*So it seams the GC doesn't simply "collect everything" when the app
terminates.*
*Why is that?*
It sounds like a pretty bad things to do: some behaviors might expect the
destructor to be called in order to close a socket, etc...
On Friday, November 17, 2017 at 10:27:22 AM UTC+1, Jean-Marc Le Roux wrote:
>
> Thanks for the great help!
> I think it makes a lot of sense now.
>
> So I've extended ObjectWrap like so:
>
> class BSONObject : public node::ObjectWrap
>> {
>> private:
>> bson_t* _bson;
>> public:
>> BSONObject(bson_t* bson, Local tpl) :
>> _bson(bson)
>> {
>> Local obj = tpl->NewInstance();
>> initialize(v8::Isolate::GetCurrent(), obj, bson);
>> Wrap(obj);
>> }
>
>
>
> ~BSONObject()
>> {
>> std::cout << "~BSONObject()" << std::endl;
>> bson_destroy(_bson);
>> }
>
>
>
> static
>> Local
>> create_template(Isolate* isolate, const bson_t* document)
>> {
>> Local tpl = ObjectTemplate::New(isolate);
>> tpl->SetInternalFieldCount(1);
>
>
> And the "insert" function that I expose in JS and that instanciates the
> BSONObject is written like this:
>
> void
>> insert(const FunctionCallbackInfo& args)
>> {
>> Isolate* isolate = args.GetIsolate();
>>
>> db->insert(
>> json_stringify(isolate, args[0]),
>> [, isolate](const bson_t* document)
>> {
>> if (!args[1]->IsUndefined())
>> {
>> Local callback = Local::Cast(args[1]);
>> Local obj = Object::New(isolate);
>> *Local argv[] = { (new
>> BSONObject(bson_copy(document)))->handle(isolate) };*
>> callback->Call(isolate->GetCurrentContext()->Global(), 1,
>> argv);
>> }
>> }
>> );
>> }
>
>
> It runs fine.
> But the destructor is never called (I never see "~BSONObject()" on stdout).
>
> Did I miss something?
>
> Thanks !
>
> 2017-11-16 22:01 GMT+01:00 J Decker <d3c...@gmail.com>:
>
>> You don't manage the instance, the v8 engine does. When it's no longer
>> referenced in the engine, it's garbage collected which triggers the
>> callback set when the perisstent object is made weak.
>>
>> You don't need to keep it anywhere, the object returned into the engine
>> has Internal Fields ( SetInternalFieldCount(1) ) which is used to point to
>> your native class instance. So when the callback is called, it knows what
>> to call to destruct it.
>>
>> You have to 'keep it' in the javascript scrpit if you want to keep it,
>> it's not 'kept' in the native code...
>>
>>
>> --- This is some code I've done that uses setweak directly; but it's
>> fairly scattered
>>
>> https://github.com/d3x0r/sack.vfs/blob/master/src/jsonParse.cc#L18
>>
>> You can create an instance of your wrapped object with
>>
>> Local cons = Local::New( isolate, constructor );
>> cons->NewInstance( isolate->GetCurrentContext(), argc, argv
>> ).ToLocalChecked() ;
>>
>> Hmm that's not the best example source...
>>
>> https://github.com/d3x0r/sack.vfs/blob/master/src/vfs_module.cc#L407
>>
>> This creates a buffer, and makes it weak...
>> https://github.com/d3x0r/sack.vfs/blob/master/src/vfs_module.cc#L438
>>
>> Local arrayBuffer = ArrayBuffer::New( isolate, buf, len );
>> PARRAY_BUFFER_HOLDER holder = GetHolder();
>> holder->o.Reset( isolate, arrayBuffer );
>> holder->o.SetWeak( holder, releaseBuffer,
>> WeakCallbackType::kParameter );
>> holder->buffer = buf;
>>
>> PARRAY_BUFFER_HOLDER is just a type that holds the thing to be free (or
>> object containing things to be released), and the persistent reference that
>> has been made weak...
>>
>> struct arrayBufferHolder {
>> void *buffer;
>> Persistent o;
>> };
>>
>>
>>
>> On Thu, Nov 16, 2017 at 9:56 AM, Jean-Marc Le Roux <
>> jeanmarc.ler...@aerys.in> wrote:
>>
>>> Node offers a C++ Clss extention ObjectWrap
>>>
>>>
>>> Thanks !
>>>
Re: [v8-users] BSON to v8::Object performance issue
Thanks for the great help!
I think it makes a lot of sense now.
So I've extended ObjectWrap like so:
class BSONObject : public node::ObjectWrap
> {
> private:
> bson_t* _bson;
> public:
> BSONObject(bson_t* bson, Local tpl) :
> _bson(bson)
> {
> Local obj = tpl->NewInstance();
> initialize(v8::Isolate::GetCurrent(), obj, bson);
> Wrap(obj);
> }
~BSONObject()
> {
> std::cout << "~BSONObject()" << std::endl;
> bson_destroy(_bson);
> }
static
> Local
> create_template(Isolate* isolate, const bson_t* document)
> {
> Local tpl = ObjectTemplate::New(isolate);
> tpl->SetInternalFieldCount(1);
And the "insert" function that I expose in JS and that instanciates the
BSONObject is written like this:
void
> insert(const FunctionCallbackInfo& args)
> {
> Isolate* isolate = args.GetIsolate();
>
> db->insert(
> json_stringify(isolate, args[0]),
> [, isolate](const bson_t* document)
> {
> if (!args[1]->IsUndefined())
> {
> Local callback = Local::Cast(args[1]);
> Local obj = Object::New(isolate);
> *Local argv[] = { (new
> BSONObject(bson_copy(document)))->handle(isolate) };*
> callback->Call(isolate->GetCurrentContext()->Global(), 1,
> argv);
> }
> }
> );
> }
It runs fine.
But the destructor is never called (I never see "~BSONObject()" on stdout).
Did I miss something?
Thanks !
2017-11-16 22:01 GMT+01:00 J Decker <d3c...@gmail.com>:
> You don't manage the instance, the v8 engine does. When it's no longer
> referenced in the engine, it's garbage collected which triggers the
> callback set when the perisstent object is made weak.
>
> You don't need to keep it anywhere, the object returned into the engine
> has Internal Fields ( SetInternalFieldCount(1) ) which is used to point to
> your native class instance. So when the callback is called, it knows what
> to call to destruct it.
>
> You have to 'keep it' in the javascript scrpit if you want to keep it,
> it's not 'kept' in the native code...
>
>
> --- This is some code I've done that uses setweak directly; but it's
> fairly scattered
>
> https://github.com/d3x0r/sack.vfs/blob/master/src/jsonParse.cc#L18
>
> You can create an instance of your wrapped object with
>
> Local cons = Local::New( isolate, constructor );
> cons->NewInstance( isolate->GetCurrentContext(), argc, argv
> ).ToLocalChecked() ;
>
> Hmm that's not the best example source...
>
> https://github.com/d3x0r/sack.vfs/blob/master/src/vfs_module.cc#L407
>
> This creates a buffer, and makes it weak...
> https://github.com/d3x0r/sack.vfs/blob/master/src/vfs_module.cc#L438
>
> Local arrayBuffer = ArrayBuffer::New( isolate, buf, len );
> PARRAY_BUFFER_HOLDER holder = GetHolder();
> holder->o.Reset( isolate, arrayBuffer );
> holder->o.SetWeak( holder, releaseBuffer,
> WeakCallbackType::kParameter );
> holder->buffer = buf;
>
> PARRAY_BUFFER_HOLDER is just a type that holds the thing to be free (or
> object containing things to be released), and the persistent reference that
> has been made weak...
>
> struct arrayBufferHolder {
> void *buffer;
> Persistent o;
> };
>
>
>
> On Thu, Nov 16, 2017 at 9:56 AM, Jean-Marc Le Roux <
> jeanmarc.ler...@aerys.in> wrote:
>
>> Node offers a C++ Clss extention ObjectWrap
>>
>>
>> Thanks !
>> It is related to node. Yet I'm not sure how I'm supposed to use
>> ObjectWrap.
>>
>> I understand I have to create my own class that extends ObjectWrap and
>> implement a proper destructor.
>> Then I'll instanciate that class to wrap my Local. But how do I
>> manage such instance?
>> If I don't keep it somewhere, it will be destructed.
>> If I keep it around, then I'm keeping references to potentially
>> deallocated memory.
>>
>> It feels like I end up with the same problem...
>>
>> 2017-11-16 18:36 GMT+01:00 J Decker <d3c...@gmail.com>:
>>
>>>
>>>
>>> On Thu, Nov 16, 2017 at 1:24 AM, Jean-Marc Le Roux <
>>> jeanmarc.ler...@aerys.in> wrote:
>>>
>>>> I assume you're storing references to the objects as Persistent?
>>>>
>>>>
>>>> Nope. You've got the whole code up there.
>>>> So if I understand correctly:
>>>>
>>>>- the objects I need to keep track of the lifecycle should be made
>>>
Re: [v8-users] BSON to v8::Object performance issue
>
> Node offers a C++ Clss extention ObjectWrap
Thanks !
It is related to node. Yet I'm not sure how I'm supposed to use ObjectWrap.
I understand I have to create my own class that extends ObjectWrap and
implement a proper destructor.
Then I'll instanciate that class to wrap my Local. But how do I
manage such instance?
If I don't keep it somewhere, it will be destructed.
If I keep it around, then I'm keeping references to potentially deallocated
memory.
It feels like I end up with the same problem...
2017-11-16 18:36 GMT+01:00 J Decker <d3c...@gmail.com>:
>
>
> On Thu, Nov 16, 2017 at 1:24 AM, Jean-Marc Le Roux <
> jeanmarc.ler...@aerys.in> wrote:
>
>> I assume you're storing references to the objects as Persistent?
>>
>>
>> Nope. You've got the whole code up there.
>> So if I understand correctly:
>>
>>- the objects I need to keep track of the lifecycle should be made
>>Persistent ;
>>- all the other values can be set as Local ;
>>- I should set the weak callback using MakeWeak ;
>>- callback will be called when such object is GCed and I can free my
>>C/C++ memory then.
>>
>> Is that correct ?
>>
>
> Yes. I sthis perhaps related to Node?
> Node offers a C++ Clss extention ObjectWrap
>
> https://nodejs.org/api/addons.html#addons_wrapping_c_objects
> https://github.com/nodejs/node/blob/master/src/node_object_wrap.h
>
> which simplifies that whole process... when the object is GC'd the class
> destructor will get called so you can release values there.
>
>
>
>>
>> 2017-11-15 19:50 GMT+01:00 J Decker <d3c...@gmail.com>:
>>
>>> I assume you're storing references to the objects as Persistent? All
>>> you need to do then is call SetWeak() https://v8docs.nodes
>>> ource.com/node-0.10/d2/d78/classv8_1_1_persistent.html
>>> void MakeWeak (void *parameters, WeakReferenceCallback callback)
>>>
>>> the callback is called when the object is GC'd.
>>>
>>>
>>> On Wed, Nov 15, 2017 at 8:37 AM, Jean-Marc Le Roux <
>>> jeanmarc.ler...@aerys.in> wrote:
>>>
>>>> So I've found a solution to make things lazy: I set an accessor instead
>>>> of decoding/setting the actual value.
>>>>
>>>> void
>>>> getter(Local property, const PropertyCallbackInfo& info)
>>>> {
>>>> Isolate* isolate = info.GetIsolate();
>>>>
>>>> const bson_t* document = reinterpret_cast<bson_t*>(
>>>> info.This()
>>>> ->Get(String::NewFromUtf8(isolate, "__bson_document_ptr"))
>>>> ->ToInt32()->Value()
>>>> );
>>>>
>>>> char key[255];
>>>> property->ToString(isolate)->WriteUtf8(key, 255);
>>>>
>>>> bson_iter_t iter;
>>>> bson_iter_init(, document);
>>>> // FIXME: index the property so we don't have to find it
>>>> bson_iter_find(, key);
>>>>
>>>> // FIXME: replace the accessor with the deserialized value
>>>>
>>>> info.GetReturnValue().Set(iterator_to_value(isolate, ));
>>>> }
>>>>
>>>> Local
>>>> fill_object(Isolate* isolate, Local& obj, const bson_t*
>>>> document)
>>>> {
>>>> obj->Set(
>>>> String::NewFromUtf8(isolate, "__bson_document_ptr"),
>>>> Int32::New(isolate, reinterpret_cast(document))
>>>> );
>>>>
>>>> bson_iter_t iter;
>>>> if (bson_iter_init(, document))
>>>> {
>>>> while (bson_iter_next())
>>>> {
>>>> const char* key = bson_iter_key();
>>>>
>>>> if (!obj->Has(String::NewFromUtf8(isolate,
>>>> key)))
>>>> {
>>>> obj->SetAccessor(
>>>> isolate->GetCurrentContext(),
>>>> String::NewFromUtf8(isolate, key),
>>>>
>>>> );
>>>> }
>>>> }
>>>> }
>>>> }
>>>>
>>>> The secret sauce is to :
>>>>
>>>>- keep the original data (the bson_t) allocated
>>>>- store the corresponding pointer in a v8::Object
>>>>
>>>> But now I have a memory leak
Re: [v8-users] BSON to v8::Object performance issue
>
> I assume you're storing references to the objects as Persistent?
Nope. You've got the whole code up there.
So if I understand correctly:
- the objects I need to keep track of the lifecycle should be made
Persistent ;
- all the other values can be set as Local ;
- I should set the weak callback using MakeWeak ;
- callback will be called when such object is GCed and I can free my
C/C++ memory then.
Is that correct ?
2017-11-15 19:50 GMT+01:00 J Decker <d3c...@gmail.com>:
> I assume you're storing references to the objects as Persistent? All you
> need to do then is call SetWeak() https://v8docs.
> nodesource.com/node-0.10/d2/d78/classv8_1_1_persistent.html
> void MakeWeak (void *parameters, WeakReferenceCallback callback)
>
> the callback is called when the object is GC'd.
>
>
> On Wed, Nov 15, 2017 at 8:37 AM, Jean-Marc Le Roux <
> jeanmarc.ler...@aerys.in> wrote:
>
>> So I've found a solution to make things lazy: I set an accessor instead
>> of decoding/setting the actual value.
>>
>> void
>> getter(Local property, const PropertyCallbackInfo& info)
>> {
>> Isolate* isolate = info.GetIsolate();
>>
>> const bson_t* document = reinterpret_cast<bson_t*>(
>> info.This()
>> ->Get(String::NewFromUtf8(isolate, "__bson_document_ptr"))
>> ->ToInt32()->Value()
>> );
>>
>> char key[255];
>> property->ToString(isolate)->WriteUtf8(key, 255);
>>
>> bson_iter_t iter;
>> bson_iter_init(, document);
>> // FIXME: index the property so we don't have to find it
>> bson_iter_find(, key);
>>
>> // FIXME: replace the accessor with the deserialized value
>>
>> info.GetReturnValue().Set(iterator_to_value(isolate, ));
>> }
>>
>> Local
>> fill_object(Isolate* isolate, Local& obj, const bson_t* document)
>> {
>> obj->Set(
>> String::NewFromUtf8(isolate, "__bson_document_ptr"),
>> Int32::New(isolate, reinterpret_cast(document))
>> );
>>
>> bson_iter_t iter;
>> if (bson_iter_init(, document))
>> {
>> while (bson_iter_next())
>> {
>> const char* key = bson_iter_key();
>>
>> if (!obj->Has(String::NewFromUtf8(isolate,
>> key)))
>> {
>> obj->SetAccessor(
>> isolate->GetCurrentContext(),
>> String::NewFromUtf8(isolate, key),
>>
>> );
>> }
>> }
>> }
>> }
>>
>> The secret sauce is to :
>>
>>- keep the original data (the bson_t) allocated
>>- store the corresponding pointer in a v8::Object
>>
>> But now I have a memory leak because those pointers will never be freed.
>> How can I know when the Object will be disposed by the GC?
>>
>> Thanks,
>>
>> --
>> --
>> v8-users mailing list
>> v8-users@googlegroups.com
>> http://groups.google.com/group/v8-users
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "v8-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to v8-users+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> --
> v8-users mailing list
> v8-users@googlegroups.com
> http://groups.google.com/group/v8-users
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "v8-users" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/v8-users/aVeevQcHJ2c/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> v8-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
--
*Jean-Marc Le Roux*
Founder and CEO of Aerys (http://aerys.in)
Blog: http://blogs.aerys.in/jeanmarc-leroux
Cell: (+33)6 20 56 45 78
Phone: (+33)9 72 40 17 58
--
--
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
---
You received this message because you are subscribed to the Google Groups
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [v8-users] BSON to v8::Object performance issue
So I've found a solution to make things lazy: I set an accessor instead of
decoding/setting the actual value.
void
getter(Local property, const PropertyCallbackInfo& info)
{
Isolate* isolate = info.GetIsolate();
const bson_t* document = reinterpret_cast(
info.This()
->Get(String::NewFromUtf8(isolate, "__bson_document_ptr"))
->ToInt32()->Value()
);
char key[255];
property->ToString(isolate)->WriteUtf8(key, 255);
bson_iter_t iter;
bson_iter_init(, document);
// FIXME: index the property so we don't have to find it
bson_iter_find(, key);
// FIXME: replace the accessor with the deserialized value
info.GetReturnValue().Set(iterator_to_value(isolate, ));
}
Local
fill_object(Isolate* isolate, Local& obj, const bson_t* document)
{
obj->Set(
String::NewFromUtf8(isolate, "__bson_document_ptr"),
Int32::New(isolate, reinterpret_cast(document))
);
bson_iter_t iter;
if (bson_iter_init(, document))
{
while (bson_iter_next())
{
const char* key = bson_iter_key();
if (!obj->Has(String::NewFromUtf8(isolate, key)))
{
obj->SetAccessor(
isolate->GetCurrentContext(),
String::NewFromUtf8(isolate, key),
);
}
}
}
}
The secret sauce is to :
- keep the original data (the bson_t) allocated
- store the corresponding pointer in a v8::Object
But now I have a memory leak because those pointers will never be freed.
How can I know when the Object will be disposed by the GC?
Thanks,
--
--
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
---
You received this message because you are subscribed to the Google Groups
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [v8-users] BSON to v8::Object performance issue
>
> Now it takes *30ms. That's still 5x faster* than the native code.
300ms. (not 30)
On Wed, Nov 15, 2017 at 10:27 AM, Jean-Marc Le Roux <
jeanmarc.ler...@aerys.in> wrote:
> Thanks for the quick feedback!
>
> I rewrote the JS version to be closer to the native one:
>
> var array = [];
> var input = {
> "_id": {
> "$oid": "5a00bad8f759511811e030ba"
> },
> "attributes": [
> {
> "key": "smartshape.scene.node.path|default",
> "value": "/scene/Lot444.scene",
> "type": "imported"
> },
> {
> "key": "max x",
> "value": 196.5773162841797,
> "type": "computed"
> },
> {
> "key": "max y",
> "value": 18.55002021789551,
> "type": "computed"
> },
> {
> "key": "max z",
> "value": 22.87815856933594,
> "type": "computed"
> },
> {
> "key": "min x",
> "value": 149.9346771240234,
> "type": "computed"
> },
> {
> "key": "min y",
> "value": 18.54999732971191,
> "type": "computed"
> },
> {
> "key": "min z",
> "value": -23.35353088378906,
> "type": "computed"
> },
> {
> "key": "box radius",
> "value": 23.32131958007814,
> "type": "computed"
> },
> {
> "key": "center x",
> "value": 173.25599670410156,
> "type": "computed"
> },
> {
> "key": "center y",
> "value": 18.55000877380371,
> "type": "computed"
> },
> {
> "key": "center z",
> "value": -0.23768615722655895,
> "type": "computed"
> },
> {
> "key": "width",
> "value": 46.64263916015628,
> "type": "computed"
> },
> {
> "key": "height",
> "value": 2.2888183600855427e-05,
> "type": "computed"
> },
> {
> "key": "depth",
> "value": 46.231689453125,
> "type": "computed"
> },
> {
> "key": "box volume",
> "value": 0.04935534689932106,
> "type": "computed"
> },
> {
> "key": "box surface",
> "value": 4312.740269302394,
> "type": "computed"
> }
> ],
> "name": "default1161",
> "uuid": "70bf7d72-1fa9-5c8f-21ff-03ef209d4404",
> "surfaces": [
> "6f5201a2-31a7-14d0-6b2a-5130007d2a51"
> ],
> "aabb": [
> 196.5773162841797,
> 18.55002021789551,
> 22.87815856933594,
> 149.9346771240234,
> 18.54999732971191,
> -23.35353088378906
> ],
> "position": null,
> "scale": null,
> "rotation": null,
> "parents": [
> "18497b66-3f32-6e98-1899-2998203e6397",
> null
> ],
> "file": "5a00b9a4aa5d2517d32d03da",
> "numChildren": 0,
> "sceneTreeIndices": [
> 0
> ]
> };
>
> function process(value) {
> if (typeof value === "number") {
> return value;
> } else if (typeof value === "string") {
> return value;
> } else if (Array.isArray(value)) {
> var array = [];
>
> for (var item of value) {
> array.push(process(item));
> }
>
> return array;
> } else if (typeof value === "object") {
>
> if (!value) {
> return value;
> }
>
> var obj = {};
> for (var key of Object.keys(value)) {
> obj[key] = process(value[key]);
> }
>
> return obj;
> } else {
> return value;
> }
>
> return value;
> }
>
> for (var i = 0; i < 26000; ++i) {
> var obj = process(input);
>
> array.push(obj);
> }
>
>
>
> Now it takes *30ms. That's still 5x faster* than the native code.
>
> I'm suspecting my Local are copied/GCed multiple time to build the
> final object.
> Isn't that the case?
> Is there a better way to do it?
>
> using a constructor function and just filling in the values should be
>> faster than assembling them one property at a time.
>
>
> The root object is expected to be the same 99% of the time.
> Each of his field too.
> So I could definitely have a map<key, function>.
> How can I implement such a constructor function in v8 ?
>
> Thanks !
>
> --
> --
> v8-users mailing list
> v8-users@googlegroups.com
> http://groups.google.com/group/v8-users
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "v8-users" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/v8-users/aVeevQcHJ2c/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> v8-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
--
*Jean-Marc Le Roux*
Founder and CEO of Aerys (http://aerys.in)
Blog: http://blogs.aerys.in/jeanmarc-leroux
Phone: (+33)6 20 56 45 78
--
--
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
---
You received this message because you are subscribed to the Google Groups
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [v8-users] BSON to v8::Object performance issue
Thanks for the quick feedback!
I rewrote the JS version to be closer to the native one:
var array = [];
var input = {
"_id": {
"$oid": "5a00bad8f759511811e030ba"
},
"attributes": [
{
"key": "smartshape.scene.node.path|default",
"value": "/scene/Lot444.scene",
"type": "imported"
},
{
"key": "max x",
"value": 196.5773162841797,
"type": "computed"
},
{
"key": "max y",
"value": 18.55002021789551,
"type": "computed"
},
{
"key": "max z",
"value": 22.87815856933594,
"type": "computed"
},
{
"key": "min x",
"value": 149.9346771240234,
"type": "computed"
},
{
"key": "min y",
"value": 18.54999732971191,
"type": "computed"
},
{
"key": "min z",
"value": -23.35353088378906,
"type": "computed"
},
{
"key": "box radius",
"value": 23.32131958007814,
"type": "computed"
},
{
"key": "center x",
"value": 173.25599670410156,
"type": "computed"
},
{
"key": "center y",
"value": 18.55000877380371,
"type": "computed"
},
{
"key": "center z",
"value": -0.23768615722655895,
"type": "computed"
},
{
"key": "width",
"value": 46.64263916015628,
"type": "computed"
},
{
"key": "height",
"value": 2.2888183600855427e-05,
"type": "computed"
},
{
"key": "depth",
"value": 46.231689453125,
"type": "computed"
},
{
"key": "box volume",
"value": 0.04935534689932106,
"type": "computed"
},
{
"key": "box surface",
"value": 4312.740269302394,
"type": "computed"
}
],
"name": "default1161",
"uuid": "70bf7d72-1fa9-5c8f-21ff-03ef209d4404",
"surfaces": [
"6f5201a2-31a7-14d0-6b2a-5130007d2a51"
],
"aabb": [
196.5773162841797,
18.55002021789551,
22.87815856933594,
149.9346771240234,
18.54999732971191,
-23.35353088378906
],
"position": null,
"scale": null,
"rotation": null,
"parents": [
"18497b66-3f32-6e98-1899-2998203e6397",
null
],
"file": "5a00b9a4aa5d2517d32d03da",
"numChildren": 0,
"sceneTreeIndices": [
0
]
};
function process(value) {
if (typeof value === "number") {
return value;
} else if (typeof value === "string") {
return value;
} else if (Array.isArray(value)) {
var array = [];
for (var item of value) {
array.push(process(item));
}
return array;
} else if (typeof value === "object") {
if (!value) {
return value;
}
var obj = {};
for (var key of Object.keys(value)) {
obj[key] = process(value[key]);
}
return obj;
} else {
return value;
}
return value;
}
for (var i = 0; i < 26000; ++i) {
var obj = process(input);
array.push(obj);
}
Now it takes *30ms. That's still 5x faster* than the native code.
I'm suspecting my Local are copied/GCed multiple time to build the
final object.
Isn't that the case?
Is there a better way to do it?
using a constructor function and just filling in the values should be
> faster than assembling them one property at a time.
The root object is expected to be the same 99% of the time.
Each of his field too.
So I could definitely have a map.
How can I implement such a constructor function in v8 ?
Thanks !
--
--
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
---
You received this message because you are subscribed to the Google Groups
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[v8-users] BSON to v8::Object performance issue
Hello,
I'm trying to create NodeJS bindings for libbson.
One of the things I need is to be able to transform a bson_t document into
a v8::Object.
Loading, iterating on bson_t documents and matching them with MongoDB-style
queries it very fast: 50ms to load from my HDD, iterate and filter 26000
bson_t documents.
On the same machine, converting the same BSON documents into v8::Object is
painfully slow.
So I suspect I missed something.
Here is my code:
Local
iterator_to_value(Isolate* isolate, const bson_iter_t* iter)
{
switch (bson_iter_type(iter))
{
case BSON_TYPE_INT32:
return Number::New(isolate, bson_iter_int32(iter));
break;
case BSON_TYPE_INT64:
return Number::New(isolate, bson_iter_int64(iter));
break;
case BSON_TYPE_DOUBLE:
return Number::New(isolate, bson_iter_double(iter));
break;
case BSON_TYPE_DOCUMENT:
{
bson_iter_t sub_iter;
bson_iter_recurse(iter, _iter);
Local obj = Object::New(isolate);
while (bson_iter_next(_iter))
{
const char* key = bson_iter_key(_iter);
obj->Set(
String::NewFromUtf8(isolate, key),
iterator_to_value(isolate, _iter)
);
}
return obj;
}
break;
case BSON_TYPE_ARRAY:
{
bson_iter_t sub_iter;
uint32_t length;
const uint8_t* array_data;
bson_iter_array(iter, , _data);
bson_iter_recurse(iter, _iter);
Local array = Array::New(isolate);
int i = 0;
while (bson_iter_next(_iter))
{
array->Set(i++, iterator_to_value(isolate, _iter));
}
return array;
}
break;
case BSON_TYPE_OID:
{
const bson_oid_t* oid = bson_iter_oid(iter);
char oid_buffer[25];
bson_oid_to_string(oid, oid_buffer);
return String::NewFromOneByte(isolate, (uint8_t*)oid_buffer);
}
break;
case BSON_TYPE_UTF8:
{
uint32_t length;
return String::NewFromUtf8(isolate, bson_iter_utf8(iter,
));
}
break;
}
return Null(isolate);
}
Local
fill_object(Isolate* isolate, Local& obj, const bson_t* document)
{
bson_iter_t iter;
if (bson_iter_init(, document))
{
while (bson_iter_next())
{
const char* key = bson_iter_key();
obj->Set(
String::NewFromUtf8(isolate, key),
iterator_to_value(isolate, )
);
}
}
return obj;
}
As you can see, it's very straight forward.
*Transforming 26400 bson_t into v_::Object takes 1.23s.*
I tried doing what I believe to be the same code in pure JS with a
representative sample Object :
var array = [];
for (var i = 0; i < 25000; ++i)
{
array.push({
"_id": {
"$oid": "5a00bad8f759511811e030ba"
},
"attributes": [
{
"key": "smartshape.scene.node.path|default",
"value": "/scene/Lot444.scene",
"type": "imported"
},
{
"key": "max x",
"value": 196.5773162841797,
"type": "computed"
},
{
"key": "max y",
"value": 18.55002021789551,
"type": "computed"
},
{
"key": "max z",
"value": 22.87815856933594,
"type": "computed"
},
{
"key": "min x",
"value": 149.9346771240234,
"type": "computed"
},
{
"key": "min y",
"value": 18.54999732971191,
"type": "computed"
},
{
"key": "min z",
"value": -23.35353088378906,
"type": "computed"
},
{
"key": "box radius",
"value": 23.32131958007814,
"type": "computed"
},
{
"key": "center x",
"value": 173.25599670410156,
"type": "computed"
},
{
"key": "center y",
"value": 18.55000877380371,
"type": "computed"
},
{
"key": "center z",
"value": -0.23768615722655895,
"type": "computed"
},
{
"key": "width",
"value": 46.64263916015628,
"type": "computed"
},
{
"key": "height",
"value": 2.2888183600855427e-05,
"type": "computed"
},
{
"key": "depth",
"value": 46.231689453125,
"type": "computed"
},
{
"key": "box volume",
"value": 0.04935534689932106,
"type":
