Re: [v8-users] BSON to v8::Object performance issue

[Jean-Marc Le Roux]

Thank you!

Great help here. Amazing folks!

On Nov 17, 2017 6:21 PM, "J Decker"  wrote:

>
>
> On Fri, Nov 17, 2017 at 7:25 AM, Jean-Marc Le Roux <
> jeanmarc.ler...@aerys.in> wrote:
>
>> My assumption is that my test app terminates before the GC has a chance
>> to run.
>> To test this:
>>
>>- I run node with the --expose-gc option
>>- I call global.gc() at the end of my test app
>>
>> In this case, the destructor of my ObjectWrap is indeed called.
>>
>> *So it seams the GC doesn't simply "collect everything" when the app
>> terminates.*
>> *Why is that?*
>>
>> It sounds like a pretty bad things to do: some behaviors might expect the
>> destructor to be called in order to close a socket, etc...
>>
>
> because all memory is released. when the app exits.  Why do extra work of
> minor side effects?
> but ya, if tf you were returning like a file to a handle and expected the
> close in order to flush all data out to a device it's bad...
>
> You can attach to Node::AtExit( ... )
> https://nodejs.org/api/addons.html#addons_atexit_hooks
>
> which should be called so you an finalize such things.
>
> but sockets get closed by the system when a application exits also.  and
> files are closed
>
>
>
>>
>> On Friday, November 17, 2017 at 10:27:22 AM UTC+1, Jean-Marc Le Roux
>> wrote:
>>>
>>> Thanks for the great help!
>>> I think it makes a lot of sense now.
>>>
>>> So I've extended ObjectWrap like so:
>>>
>>> class BSONObject : public node::ObjectWrap
 {
 private:
 bson_t* _bson;
 public:
 BSONObject(bson_t* bson, Local tpl) :
 _bson(bson)
 {
 Local obj = tpl->NewInstance();
 initialize(v8::Isolate::GetCurrent(), obj, bson);
 Wrap(obj);
 }
>>>
>>>
>>>
>>>  ~BSONObject()
 {
 std::cout << "~BSONObject()" << std::endl;
 bson_destroy(_bson);
 }
>>>
>>>
>>>
>>> static
 Local
 create_template(Isolate* isolate, const bson_t* document)
 {
 Local tpl = ObjectTemplate::New(isolate);
 tpl->SetInternalFieldCount(1);
>>>
>>>
>>> And the "insert" function that I expose in JS and that instanciates the
>>> BSONObject is written like this:
>>>
>>> void
 insert(const FunctionCallbackInfo& args)
 {
 Isolate* isolate = args.GetIsolate();

 db->insert(
 json_stringify(isolate, args[0]),
 [, isolate](const bson_t* document)
 {
 if (!args[1]->IsUndefined())
 {
 Local callback =
 Local::Cast(args[1]);
 Local obj = Object::New(isolate);
 *Local argv[] = { (new
 BSONObject(bson_copy(document)))->handle(isolate) };*
 callback->Call(isolate->GetCurrentContext()->Global(),
 1, argv);
 }
 }
 );
 }
>>>
>>>
>>> It runs fine.
>>> But the destructor is never called (I never see "~BSONObject()" on
>>> stdout).
>>>
>>> Did I miss something?
>>>
>>> Thanks !
>>>
>>> 2017-11-16 22:01 GMT+01:00 J Decker :
>>>
 You don't manage the instance, the v8 engine does.  When it's no longer
 referenced in the engine, it's garbage collected which triggers the
 callback set when the perisstent object is made weak.

 You don't need to keep it anywhere, the object returned into the engine
 has Internal Fields ( SetInternalFieldCount(1) ) which is used to point to
 your native class instance.   So when the callback is called, it knows what
 to call to destruct it.

 You have to 'keep it' in the javascript scrpit if you want to keep it,
 it's not 'kept' in the native code...


 --- This is some code I've done that uses setweak directly; but it's
 fairly scattered

 https://github.com/d3x0r/sack.vfs/blob/master/src/jsonParse.cc#L18

 You can create an instance of your wrapped object with

 Local cons = Local::New( isolate, constructor );
 cons->NewInstance( isolate->GetCurrentContext(), argc, argv
 ).ToLocalChecked() ;

 Hmm that's not the best example source...

 https://github.com/d3x0r/sack.vfs/blob/master/src/vfs_module.cc#L407

 This creates a buffer, and makes it weak...
 https://github.com/d3x0r/sack.vfs/blob/master/src/vfs_module.cc#L438

 Local arrayBuffer = ArrayBuffer::New( isolate, buf, len );
 PARRAY_BUFFER_HOLDER holder = GetHolder();
 holder->o.Reset( isolate, arrayBuffer );
 holder->o.SetWeak( holder, releaseBuffer,
 WeakCallbackType::kParameter );
 holder->buffer = buf;

 PARRAY_BUFFER_HOLDER is just a type that holds the thing to be free (or
 object containing things to be released), and the persistent reference that
 has been made weak...

 struct arrayBufferHolder {
 void *buffer;
 Persistent o;
 };

Re: [v8-users] BSON to v8::Object performance issue

[J Decker]

On Fri, Nov 17, 2017 at 7:25 AM, Jean-Marc Le Roux  wrote:

> My assumption is that my test app terminates before the GC has a chance to
> run.
> To test this:
>
>- I run node with the --expose-gc option
>- I call global.gc() at the end of my test app
>
> In this case, the destructor of my ObjectWrap is indeed called.
>
> *So it seams the GC doesn't simply "collect everything" when the app
> terminates.*
> *Why is that?*
>
> It sounds like a pretty bad things to do: some behaviors might expect the
> destructor to be called in order to close a socket, etc...
>

because all memory is released. when the app exits.  Why do extra work of
minor side effects?
but ya, if tf you were returning like a file to a handle and expected the
close in order to flush all data out to a device it's bad...

You can attach to Node::AtExit( ... )
https://nodejs.org/api/addons.html#addons_atexit_hooks

which should be called so you an finalize such things.

but sockets get closed by the system when a application exits also.  and
files are closed



>
> On Friday, November 17, 2017 at 10:27:22 AM UTC+1, Jean-Marc Le Roux wrote:
>>
>> Thanks for the great help!
>> I think it makes a lot of sense now.
>>
>> So I've extended ObjectWrap like so:
>>
>> class BSONObject : public node::ObjectWrap
>>> {
>>> private:
>>> bson_t* _bson;
>>> public:
>>> BSONObject(bson_t* bson, Local tpl) :
>>> _bson(bson)
>>> {
>>> Local obj = tpl->NewInstance();
>>> initialize(v8::Isolate::GetCurrent(), obj, bson);
>>> Wrap(obj);
>>> }
>>
>>
>>
>>  ~BSONObject()
>>> {
>>> std::cout << "~BSONObject()" << std::endl;
>>> bson_destroy(_bson);
>>> }
>>
>>
>>
>> static
>>> Local
>>> create_template(Isolate* isolate, const bson_t* document)
>>> {
>>> Local tpl = ObjectTemplate::New(isolate);
>>> tpl->SetInternalFieldCount(1);
>>
>>
>> And the "insert" function that I expose in JS and that instanciates the
>> BSONObject is written like this:
>>
>> void
>>> insert(const FunctionCallbackInfo& args)
>>> {
>>> Isolate* isolate = args.GetIsolate();
>>>
>>> db->insert(
>>> json_stringify(isolate, args[0]),
>>> [, isolate](const bson_t* document)
>>> {
>>> if (!args[1]->IsUndefined())
>>> {
>>> Local callback = Local::Cast(args[1])
>>> ;
>>> Local obj = Object::New(isolate);
>>> *Local argv[] = { (new
>>> BSONObject(bson_copy(document)))->handle(isolate) };*
>>> callback->Call(isolate->GetCurrentContext()->Global(),
>>> 1, argv);
>>> }
>>> }
>>> );
>>> }
>>
>>
>> It runs fine.
>> But the destructor is never called (I never see "~BSONObject()" on
>> stdout).
>>
>> Did I miss something?
>>
>> Thanks !
>>
>> 2017-11-16 22:01 GMT+01:00 J Decker :
>>
>>> You don't manage the instance, the v8 engine does.  When it's no longer
>>> referenced in the engine, it's garbage collected which triggers the
>>> callback set when the perisstent object is made weak.
>>>
>>> You don't need to keep it anywhere, the object returned into the engine
>>> has Internal Fields ( SetInternalFieldCount(1) ) which is used to point to
>>> your native class instance.   So when the callback is called, it knows what
>>> to call to destruct it.
>>>
>>> You have to 'keep it' in the javascript scrpit if you want to keep it,
>>> it's not 'kept' in the native code...
>>>
>>>
>>> --- This is some code I've done that uses setweak directly; but it's
>>> fairly scattered
>>>
>>> https://github.com/d3x0r/sack.vfs/blob/master/src/jsonParse.cc#L18
>>>
>>> You can create an instance of your wrapped object with
>>>
>>> Local cons = Local::New( isolate, constructor );
>>> cons->NewInstance( isolate->GetCurrentContext(), argc, argv
>>> ).ToLocalChecked() ;
>>>
>>> Hmm that's not the best example source...
>>>
>>> https://github.com/d3x0r/sack.vfs/blob/master/src/vfs_module.cc#L407
>>>
>>> This creates a buffer, and makes it weak...
>>> https://github.com/d3x0r/sack.vfs/blob/master/src/vfs_module.cc#L438
>>>
>>> Local arrayBuffer = ArrayBuffer::New( isolate, buf, len );
>>> PARRAY_BUFFER_HOLDER holder = GetHolder();
>>> holder->o.Reset( isolate, arrayBuffer );
>>> holder->o.SetWeak( holder, releaseBuffer,
>>> WeakCallbackType::kParameter );
>>> holder->buffer = buf;
>>>
>>> PARRAY_BUFFER_HOLDER is just a type that holds the thing to be free (or
>>> object containing things to be released), and the persistent reference that
>>> has been made weak...
>>>
>>> struct arrayBufferHolder {
>>> void *buffer;
>>> Persistent o;
>>> };
>>>
>>>
>>>
>>> On Thu, Nov 16, 2017 at 9:56 AM, Jean-Marc Le Roux <
>>> jeanmarc.ler...@aerys.in> wrote:
>>>
 Node offers a C++ Clss extention ObjectWrap


 Thanks !
 It is related to node. Yet I'm not sure how I'm supposed to use
 ObjectWrap.

 I understand I 

Re: [v8-users] BSON to v8::Object performance issue

[Jean-Marc Le Roux]

My assumption is that my test app terminates before the GC has a chance to 
run.
To test this:

   - I run node with the --expose-gc option
   - I call global.gc() at the end of my test app

In this case, the destructor of my ObjectWrap is indeed called.

*So it seams the GC doesn't simply "collect everything" when the app 
terminates.*
*Why is that?*

It sounds like a pretty bad things to do: some behaviors might expect the 
destructor to be called in order to close a socket, etc...

On Friday, November 17, 2017 at 10:27:22 AM UTC+1, Jean-Marc Le Roux wrote:
>
> Thanks for the great help!
> I think it makes a lot of sense now.
>
> So I've extended ObjectWrap like so:
>
> class BSONObject : public node::ObjectWrap
>> {
>> private:
>> bson_t* _bson;
>> public:
>> BSONObject(bson_t* bson, Local tpl) :
>> _bson(bson)
>> {
>> Local obj = tpl->NewInstance();
>> initialize(v8::Isolate::GetCurrent(), obj, bson);
>> Wrap(obj);
>> }
>
>  
>
>  ~BSONObject()
>> {
>> std::cout << "~BSONObject()" << std::endl;
>> bson_destroy(_bson);
>> }
>
>  
>
> static
>> Local
>> create_template(Isolate* isolate, const bson_t* document)
>> {
>> Local tpl = ObjectTemplate::New(isolate);
>> tpl->SetInternalFieldCount(1); 
>
>
> And the "insert" function that I expose in JS and that instanciates the 
> BSONObject is written like this:
>
> void
>> insert(const FunctionCallbackInfo& args)
>> {
>> Isolate* isolate = args.GetIsolate();
>> 
>> db->insert(
>> json_stringify(isolate, args[0]),
>> [, isolate](const bson_t* document)
>> {
>> if (!args[1]->IsUndefined())
>> {
>> Local callback = Local::Cast(args[1]);
>> Local obj = Object::New(isolate);
>> *Local argv[] = { (new 
>> BSONObject(bson_copy(document)))->handle(isolate) };*
>> callback->Call(isolate->GetCurrentContext()->Global(), 1, 
>> argv);
>> }
>> }
>> );
>> }
>
>
> It runs fine.
> But the destructor is never called (I never see "~BSONObject()" on stdout).
>
> Did I miss something?
>
> Thanks !
>
> 2017-11-16 22:01 GMT+01:00 J Decker :
>
>> You don't manage the instance, the v8 engine does.  When it's no longer 
>> referenced in the engine, it's garbage collected which triggers the 
>> callback set when the perisstent object is made weak.
>>
>> You don't need to keep it anywhere, the object returned into the engine 
>> has Internal Fields ( SetInternalFieldCount(1) ) which is used to point to 
>> your native class instance.   So when the callback is called, it knows what 
>> to call to destruct it.
>>
>> You have to 'keep it' in the javascript scrpit if you want to keep it, 
>> it's not 'kept' in the native code...
>>
>>
>> --- This is some code I've done that uses setweak directly; but it's 
>> fairly scattered
>>
>> https://github.com/d3x0r/sack.vfs/blob/master/src/jsonParse.cc#L18
>>
>> You can create an instance of your wrapped object with
>>
>> Local cons = Local::New( isolate, constructor );
>> cons->NewInstance( isolate->GetCurrentContext(), argc, argv  
>> ).ToLocalChecked() ;
>>
>> Hmm that's not the best example source... 
>>
>> https://github.com/d3x0r/sack.vfs/blob/master/src/vfs_module.cc#L407
>>
>> This creates a buffer, and makes it weak... 
>> https://github.com/d3x0r/sack.vfs/blob/master/src/vfs_module.cc#L438
>>
>> Local arrayBuffer = ArrayBuffer::New( isolate, buf, len );
>> PARRAY_BUFFER_HOLDER holder = GetHolder();
>> holder->o.Reset( isolate, arrayBuffer );
>> holder->o.SetWeak( holder, releaseBuffer, 
>> WeakCallbackType::kParameter );
>> holder->buffer = buf;
>>
>> PARRAY_BUFFER_HOLDER is just a type that holds the thing to be free (or 
>> object containing things to be released), and the persistent reference that 
>> has been made weak...
>>
>> struct arrayBufferHolder {
>> void *buffer;
>> Persistent o;
>> };
>>
>>
>>
>> On Thu, Nov 16, 2017 at 9:56 AM, Jean-Marc Le Roux <
>> jeanmarc.ler...@aerys.in> wrote:
>>
>>> Node offers a C++ Clss extention ObjectWrap 
>>>
>>>
>>> Thanks !
>>> It is related to node. Yet I'm not sure how I'm supposed to use 
>>> ObjectWrap.
>>>
>>> I understand I have to create my own class that extends ObjectWrap and 
>>> implement a proper destructor.
>>> Then I'll instanciate that class to wrap my Local. But how do I 
>>> manage such instance?
>>> If I don't keep it somewhere, it will be destructed.
>>> If I keep it around, then I'm keeping references to potentially 
>>> deallocated memory.
>>>
>>> It feels like I end up with the same problem...
>>>
>>> 2017-11-16 18:36 GMT+01:00 J Decker :
>>>


 On Thu, Nov 16, 2017 at 1:24 AM, Jean-Marc Le Roux <
 jeanmarc.ler...@aerys.in> wrote:

> I assume you're storing references to the objects as Persistent?
>
>
> Nope. You've got the whole code up 

Re: [v8-users] BSON to v8::Object performance issue

[Jean-Marc Le Roux]

Thanks for the great help!
I think it makes a lot of sense now.

So I've extended ObjectWrap like so:

class BSONObject : public node::ObjectWrap
> {
> private:
> bson_t* _bson;
> public:
> BSONObject(bson_t* bson, Local tpl) :
> _bson(bson)
> {
> Local obj = tpl->NewInstance();
> initialize(v8::Isolate::GetCurrent(), obj, bson);
> Wrap(obj);
> }



 ~BSONObject()
> {
> std::cout << "~BSONObject()" << std::endl;
> bson_destroy(_bson);
> }



static
> Local
> create_template(Isolate* isolate, const bson_t* document)
> {
> Local tpl = ObjectTemplate::New(isolate);
> tpl->SetInternalFieldCount(1);


And the "insert" function that I expose in JS and that instanciates the
BSONObject is written like this:

void
> insert(const FunctionCallbackInfo& args)
> {
> Isolate* isolate = args.GetIsolate();
>
> db->insert(
> json_stringify(isolate, args[0]),
> [, isolate](const bson_t* document)
> {
> if (!args[1]->IsUndefined())
> {
> Local callback = Local::Cast(args[1]);
> Local obj = Object::New(isolate);
> *Local argv[] = { (new
> BSONObject(bson_copy(document)))->handle(isolate) };*
> callback->Call(isolate->GetCurrentContext()->Global(), 1,
> argv);
> }
> }
> );
> }


It runs fine.
But the destructor is never called (I never see "~BSONObject()" on stdout).

Did I miss something?

Thanks !

2017-11-16 22:01 GMT+01:00 J Decker :

> You don't manage the instance, the v8 engine does.  When it's no longer
> referenced in the engine, it's garbage collected which triggers the
> callback set when the perisstent object is made weak.
>
> You don't need to keep it anywhere, the object returned into the engine
> has Internal Fields ( SetInternalFieldCount(1) ) which is used to point to
> your native class instance.   So when the callback is called, it knows what
> to call to destruct it.
>
> You have to 'keep it' in the javascript scrpit if you want to keep it,
> it's not 'kept' in the native code...
>
>
> --- This is some code I've done that uses setweak directly; but it's
> fairly scattered
>
> https://github.com/d3x0r/sack.vfs/blob/master/src/jsonParse.cc#L18
>
> You can create an instance of your wrapped object with
>
> Local cons = Local::New( isolate, constructor );
> cons->NewInstance( isolate->GetCurrentContext(), argc, argv
> ).ToLocalChecked() ;
>
> Hmm that's not the best example source...
>
> https://github.com/d3x0r/sack.vfs/blob/master/src/vfs_module.cc#L407
>
> This creates a buffer, and makes it weak...
> https://github.com/d3x0r/sack.vfs/blob/master/src/vfs_module.cc#L438
>
> Local arrayBuffer = ArrayBuffer::New( isolate, buf, len );
> PARRAY_BUFFER_HOLDER holder = GetHolder();
> holder->o.Reset( isolate, arrayBuffer );
> holder->o.SetWeak( holder, releaseBuffer,
> WeakCallbackType::kParameter );
> holder->buffer = buf;
>
> PARRAY_BUFFER_HOLDER is just a type that holds the thing to be free (or
> object containing things to be released), and the persistent reference that
> has been made weak...
>
> struct arrayBufferHolder {
> void *buffer;
> Persistent o;
> };
>
>
>
> On Thu, Nov 16, 2017 at 9:56 AM, Jean-Marc Le Roux <
> jeanmarc.ler...@aerys.in> wrote:
>
>> Node offers a C++ Clss extention ObjectWrap
>>
>>
>> Thanks !
>> It is related to node. Yet I'm not sure how I'm supposed to use
>> ObjectWrap.
>>
>> I understand I have to create my own class that extends ObjectWrap and
>> implement a proper destructor.
>> Then I'll instanciate that class to wrap my Local. But how do I
>> manage such instance?
>> If I don't keep it somewhere, it will be destructed.
>> If I keep it around, then I'm keeping references to potentially
>> deallocated memory.
>>
>> It feels like I end up with the same problem...
>>
>> 2017-11-16 18:36 GMT+01:00 J Decker :
>>
>>>
>>>
>>> On Thu, Nov 16, 2017 at 1:24 AM, Jean-Marc Le Roux <
>>> jeanmarc.ler...@aerys.in> wrote:
>>>
 I assume you're storing references to the objects as Persistent?


 Nope. You've got the whole code up there.
 So if I understand correctly:

- the objects I need to keep track of the lifecycle should be made
Persistent ;
- all the other values can be set as Local ;
- I should set the weak callback using MakeWeak ;
- callback will be called when such object is GCed and I can free
my C/C++ memory then.

 Is that correct ?

>>>
>>> Yes.  I sthis perhaps related to Node?
>>> Node offers a C++ Clss extention ObjectWrap
>>>
>>> https://nodejs.org/api/addons.html#addons_wrapping_c_objects
>>> https://github.com/nodejs/node/blob/master/src/node_object_wrap.h
>>>
>>> which simplifies that whole process... when the object is GC'd the class
>>> destructor will get called so you can release values there.
>>>