Re: [v8-users] v8 3.28, getting data from typed arrays, I'm unsure if this approach is correct (and doesn't leak)
*Correction, free(dataPtr) should be replaced with a call to your
ArrayBufferAllocator's .Free(ptr, length) call
On Friday, February 26, 2016 at 1:22:26 PM UTC, George Corney wrote:
>
> *For the next person trying to do this:*
>
> It turns the approach in the post above doesn't completely work because
> GetIndexedPropertiesExternalArrayData is only supported on
> ArrayBufferViews and not ArrayBuffers. My solution now is to grab the
> Contents::Data() pointer provided when ArrayBuffer::Externalize() is called
> and store it on the ArrayBuffer in a hidden field (as well as creating a
> weak persistent handle to free it later). My implementation is provided:
>
> struct WeakCallbackData{
> v8::Persistent* persistent;
> uintptr_t dataPtrValue;
> };
>
> //when we get typed array data we take responsibility for freeing the data
> void ArrayBufferWeakCallback(const v8::WeakCallbackData WeakCallbackData>& info){
> WeakCallbackData* callbackData = info.GetParameter();
> v8::Persistent* persistent = callbackData->persistent;
> void* dataPtr = (void*) info.GetParameter()->dataPtrValue;
>
> Print::Debug("ArrayBufferWeakCallback fired (byte length: %d, dataPtr*:
> %p)", info.GetValue()->ByteLength(), dataPtr);
>
> //dealloc
> info.GetValue().Clear();
> persistent->Reset();
> delete callbackData;
> free(dataPtr);
> }
>
> void AttachPointerObject(v8::Isolate* isolate,
> v8::Handle& buffer, void* dataPtr){
> v8::Local ptrObjTmpl =
> v8::ObjectTemplate::New(isolate);
> ptrObjTmpl->SetInternalFieldCount(1);
> v8::Local ptrObj = ptrObjTmpl->NewInstance();
> ptrObj->SetAlignedPointerInInternalField(0, dataPtr);
> buffer->SetHiddenValue(v8::String::NewFromUtf8(isolate, "ptrObj"), ptrObj);
> }
>
> //Public API
>
> const void SafelyExternalizeArrayBuffer(v8::Isolate* isolate,
> v8::Handle& buffer){
> if(!buffer->IsExternal()){
> v8::ArrayBuffer::Contents contents = buffer->Externalize();
> RegisterExternalData(isolate, buffer, contents.Data());
> }
> }
>
> void RegisterExternalData(v8::Isolate* isolate,
> v8::Handle& buffer, void* dataPtr){
> Print::Debug("RegisterExternalData (byte length: %d, dataPtr*: %p)",
> buffer->ByteLength(), dataPtr);
>
> AttachPointerObject(isolate, buffer, dataPtr);
>
> v8::Persistent* persistent = new
> v8::Persistent();
> persistent->Reset(isolate, buffer);
>
> WeakCallbackData* callbackData = new WeakCallbackData();
> callbackData->persistent = persistent;
> callbackData->dataPtrValue = (uintptr_t)(void *)dataPtr;
>
> persistent->SetWeak(callbackData,
> ArrayBufferWeakCallback);
> }
>
> const unsigned char* GetArrayBufferBytes(v8::Isolate* isolate,
> v8::Handle& buffer){
> SafelyExternalizeArrayBuffer(isolate, buffer);
>
> v8::Local v =
> buffer->GetHiddenValue(v8::String::NewFromUtf8(isolate, "ptrObj"));
>
> if(!v->IsObject()){
> Print::Error("%s: Internal data pointer not found on ArrayBuffer, ensure
> RegisterExternalData was at initialization", __FUNCTION__);
> return NULL;
> }
>
> v8::Local ptrObj = v8::Local::Cast(v);
> const void* dataPtr = ptrObj->GetAlignedPointerFromInternalField(0);
> const unsigned char* bytePtr = static_cast(dataPtr);
> return bytePtr;
> }
>
>
> const unsigned char* GetArrayBufferViewBytes(v8::Isolate* isolate,
> v8::Handle& bufferView){
> v8::Local buffer = bufferView->Buffer();
> SafelyExternalizeArrayBuffer(isolate, buffer);
> const void* dataPtr = bufferView->GetIndexedPropertiesExternalArrayData();
> const unsigned char* bytePtr = static_cast(dataPtr)
> + bufferView->ByteOffset();
> return bytePtr;
> }
>
>
> (You'll need to replace the Print:: functions with whatever system you're
> using for logging)
>
> - If the TypedArrays are coming from JS land, you'll only need
> *GetArrayBufferBytes* and *GetArrayBufferViewBytes* (depending on if
> you're using an ArrayBufferView or ArrayBuffer) to access the data safely
> - If the TypedArray is created in C++ land and the data buffer is provided
> to it so it's already external, then you need to use
> *RegisterExternalData* for to make the ArrayBuffer safe to access with
> these functions (the buffer will be freed with the WeakCallback)
> - If you want to externalize without accessing the data, you need to use
> *SafelyExternalizeArrayBuffer* *and not* *ArrayBuffer::Externalize()*
>
>
> All the best,
> George Corney
>
> On Monday, February 22, 2016 at 3:38:08 PM UTC, George Corney wrote:
>>
>> Thanks for the tips
>>
>> I agree regarding upgrading from 3.28, unfortunately for the time being
>> I'm stuck with it as a consequence of various trade offs (I'm using JXcore
>> which is built on 3.28 and I can't migrate away because of time constraints)
>>
>> I'm trying the externalize + weak persistent handle method, here's my
>> code now
>>
>> //weak callback, frees the raw data and handle
>> void ArrayBufferWeakCallback(const v8::WeakCallbackData> v8::Persistent>& info){
>> Print::Debug("ArrayBufferWeakCallback fired");
>> void* rawData = info.GetValue()->GetIndexedPropertiesExternalArrayData
>> ()
Re: [v8-users] v8 3.28, getting data from typed arrays, I'm unsure if this approach is correct (and doesn't leak)
*For the next person trying to do this:*
It turns the approach in the post above doesn't completely work because
GetIndexedPropertiesExternalArrayData is only supported on ArrayBufferViews
and not ArrayBuffers. My solution now is to grab the Contents::Data()
pointer provided when ArrayBuffer::Externalize() is called and store it on
the ArrayBuffer in a hidden field (as well as creating a weak persistent
handle to free it later). My implementation is provided:
struct WeakCallbackData{
v8::Persistent* persistent;
uintptr_t dataPtrValue;
};
//when we get typed array data we take responsibility for freeing the data
void ArrayBufferWeakCallback(const v8::WeakCallbackData& info){
WeakCallbackData* callbackData = info.GetParameter();
v8::Persistent* persistent = callbackData->persistent;
void* dataPtr = (void*) info.GetParameter()->dataPtrValue;
Print::Debug("ArrayBufferWeakCallback fired (byte length: %d, dataPtr*:
%p)", info.GetValue()->ByteLength(), dataPtr);
//dealloc
info.GetValue().Clear();
persistent->Reset();
delete callbackData;
free(dataPtr);
}
void AttachPointerObject(v8::Isolate* isolate, v8::Handle&
buffer, void* dataPtr){
v8::Local ptrObjTmpl = v8::ObjectTemplate::New(isolate);
ptrObjTmpl->SetInternalFieldCount(1);
v8::Local ptrObj = ptrObjTmpl->NewInstance();
ptrObj->SetAlignedPointerInInternalField(0, dataPtr);
buffer->SetHiddenValue(v8::String::NewFromUtf8(isolate, "ptrObj"), ptrObj);
}
//Public API
const void SafelyExternalizeArrayBuffer(v8::Isolate* isolate,
v8::Handle& buffer){
if(!buffer->IsExternal()){
v8::ArrayBuffer::Contents contents = buffer->Externalize();
RegisterExternalData(isolate, buffer, contents.Data());
}
}
void RegisterExternalData(v8::Isolate* isolate,
v8::Handle& buffer, void* dataPtr){
Print::Debug("RegisterExternalData (byte length: %d, dataPtr*: %p)",
buffer->ByteLength(), dataPtr);
AttachPointerObject(isolate, buffer, dataPtr);
v8::Persistent* persistent = new
v8::Persistent();
persistent->Reset(isolate, buffer);
WeakCallbackData* callbackData = new WeakCallbackData();
callbackData->persistent = persistent;
callbackData->dataPtrValue = (uintptr_t)(void *)dataPtr;
persistent->SetWeak(callbackData,
ArrayBufferWeakCallback);
}
const unsigned char* GetArrayBufferBytes(v8::Isolate* isolate,
v8::Handle& buffer){
SafelyExternalizeArrayBuffer(isolate, buffer);
v8::Local v =
buffer->GetHiddenValue(v8::String::NewFromUtf8(isolate, "ptrObj"));
if(!v->IsObject()){
Print::Error("%s: Internal data pointer not found on ArrayBuffer, ensure
RegisterExternalData was at initialization", __FUNCTION__);
return NULL;
}
v8::Local ptrObj = v8::Local::Cast(v);
const void* dataPtr = ptrObj->GetAlignedPointerFromInternalField(0);
const unsigned char* bytePtr = static_cast(dataPtr);
return bytePtr;
}
const unsigned char* GetArrayBufferViewBytes(v8::Isolate* isolate,
v8::Handle& bufferView){
v8::Local buffer = bufferView->Buffer();
SafelyExternalizeArrayBuffer(isolate, buffer);
const void* dataPtr = bufferView->GetIndexedPropertiesExternalArrayData();
const unsigned char* bytePtr = static_cast(dataPtr) +
bufferView->ByteOffset();
return bytePtr;
}
(You'll need to replace the Print:: functions with whatever system you're
using for logging)
- If the TypedArrays are coming from JS land, you'll only need
*GetArrayBufferBytes* and *GetArrayBufferViewBytes* (depending on if you're
using an ArrayBufferView or ArrayBuffer) to access the data safely
- If the TypedArray is created in C++ land and the data buffer is provided
to it so it's already external, then you need to use *RegisterExternalData*
for to make the ArrayBuffer safe to access with these functions (the buffer
will be freed with the WeakCallback)
- If you want to externalize without accessing the data, you need to use
*SafelyExternalizeArrayBuffer* *and not* *ArrayBuffer::Externalize()*
All the best,
George Corney
On Monday, February 22, 2016 at 3:38:08 PM UTC, George Corney wrote:
>
> Thanks for the tips
>
> I agree regarding upgrading from 3.28, unfortunately for the time being
> I'm stuck with it as a consequence of various trade offs (I'm using JXcore
> which is built on 3.28 and I can't migrate away because of time constraints)
>
> I'm trying the externalize + weak persistent handle method, here's my code
> now
>
> //weak callback, frees the raw data and handle
> void ArrayBufferWeakCallback(const v8::WeakCallbackData v8::Persistent>& info){
> Print::Debug("ArrayBufferWeakCallback fired");
> void* rawData = info.GetValue()->GetIndexedPropertiesExternalArrayData();
> info.GetValue().Clear();
> info.GetParameter()->Reset();
> free(rawData);
> free(info.GetParameter());
> }
>
>
> //...
>
>
> v8::Local bufferView = dataArg.As >();
> size_t byteLength = bufferView->ByteLength();
> size_t byteOffset = bufferView->ByteOffset();
>
>
> bufferView->Buffer()->Externalize();
>
>
> //create weak persistent handle with callback to above code
> v8::Persistent* persistent = new v8::Persistent ArrayBuf
Re: [v8-users] v8 3.28, getting data from typed arrays, I'm unsure if this approach is correct (and doesn't leak)
Thanks for the tips
I agree regarding upgrading from 3.28, unfortunately for the time being I'm
stuck with it as a consequence of various trade offs (I'm using JXcore
which is built on 3.28 and I can't migrate away because of time constraints)
I'm trying the externalize + weak persistent handle method, here's my code
now
//weak callback, frees the raw data and handle
void ArrayBufferWeakCallback(const v8::WeakCallbackData>& info){
Print::Debug("ArrayBufferWeakCallback fired");
void* rawData = info.GetValue()->GetIndexedPropertiesExternalArrayData();
info.GetValue().Clear();
info.GetParameter()->Reset();
free(rawData);
free(info.GetParameter());
}
//...
v8::Local bufferView = dataArg.As();
size_t byteLength = bufferView->ByteLength();
size_t byteOffset = bufferView->ByteOffset();
bufferView->Buffer()->Externalize();
//create weak persistent handle with callback to above code
v8::Persistent* persistent = new v8::Persistent();
persistent->Reset(isolate, buffer);
persistent->SetWeak>(persistent,
ArrayBufferWeakCallback);
const void* rawData = bufferView->GetIndexedPropertiesExternalArrayData();
const unsigned char* bytePtr = static_cast(rawData);
glBufferData(target, byteLength, bytePtr + byteOffset, usage);
REPORT_GL_ERRORS();
The callback seems to fire as expected and there's no segfaults so I think
this might be the solution! Could you have a quick read to see I'm not
setting myself up for failure by rolling this approach out across my code?
Many thanks,
George
On Monday, February 22, 2016 at 7:59:15 AM UTC, Jochen Eisinger wrote:
>
> Please note that 3.28 is a really old version of V8 which is full of known
> issues and no longer maintained.
>
> In general, you'll wait to externalize the buffer and keep a weak
> persistent handle to the buffer - once the weak callback was triggered, you
> know that you can free the backing store (if you no longer need it
> otherwise).
>
> More current versions of V8 also have a method
> ArrayBufferView::CopyContents that allows you to access the backing store
> without externalizing the underlying buffer.
>
> On Sun, Feb 21, 2016 at 11:27 PM George Corney > wrote:
>
>> Hello,
>>
>> I'm working with typed arrays in v8 3.28, I've got a native function that
>> takes a Float32Array from javascript and passes the float* data off to a gl
>> call (glBufferData).
>>
>> I discovered that using just
>> const void* rawData = bufferView->GetIndexedPropertiesExternalArrayData
>> ();
>> works, but not reliably, the data gets passes to glBufferData correctly
>> most of the time, but not always!
>>
>> I've found creating a persistent handle before getting the data pointer,
>> and then reseting handle after the glBufferData solves the problem, but is
>> it the right approach? Am I leaking memory / risking sending incorrect data
>> to glBufferData? Here's the snippet
>>
>> v8::Local bufferView =
>> dataArg.As();
>> size_t byteLength = bufferView->ByteLength();
>> size_t byteOffset = bufferView->ByteOffset();
>>
>> v8::Persistent persistent;
>> persistent.Reset(__contextORisolate, bufferView->Buffer());
>>
>> const void* rawData = bufferView->GetIndexedPropertiesExternalArrayData();
>> const unsigned char* bytePtr = static_cast(rawData);
>>
>> glBufferData(target, byteLength, bytePtr + byteOffset, usage);
>> REPORT_GL_ERRORS();
>>
>> persistent.Reset();
>>
>>
>> ---
>>
>> If this the wrong approach, I think the next thing is to use
>> bufferView->Buffer()->Externalize();
>> before getting the data pointer. In this case I'm then responsible for
>> freeing the data - If this is necessary, could you explain how to do this?
>> Can it be done without altering the ArrayBufferAllocator?
>>
>>
>> Many thanks!
>> George Corney
>>
>> --
>> --
>> v8-users mailing list
>> v8-u...@googlegroups.com
>> http://groups.google.com/group/v8-users
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "v8-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to v8-users+u...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
--
--
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
---
You received this message because you are subscribed to the Google Groups
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [v8-users] v8 3.28, getting data from typed arrays, I'm unsure if this approach is correct (and doesn't leak)
On Sun, Feb 21, 2016 at 11:27 PM, George Corney wrote: > Hello, > > I'm working with typed arrays in v8 3.28, I've got a native function that > takes a Float32Array from javascript and passes the float* data off to a gl > call (glBufferData). > > I discovered that using just > const void* rawData = bufferView->GetIndexedPropertiesExternalArrayData(); > works, but not reliably, the data gets passes to glBufferData correctly most > of the time, but not always! > > I've found creating a persistent handle before getting the data pointer, and > then reseting handle after the glBufferData solves the problem, but is it > the right approach? Am I leaking memory / risking sending incorrect data to > glBufferData? Here's the snippet > > v8::Local bufferView = > dataArg.As(); > size_t byteLength = bufferView->ByteLength(); > size_t byteOffset = bufferView->ByteOffset(); > > v8::Persistent persistent; > persistent.Reset(__contextORisolate, bufferView->Buffer()); > > const void* rawData = bufferView->GetIndexedPropertiesExternalArrayData(); > const unsigned char* bytePtr = static_cast(rawData); > > glBufferData(target, byteLength, bytePtr + byteOffset, usage); > REPORT_GL_ERRORS(); > > persistent.Reset(); > > > --- > > If this the wrong approach, I think the next thing is to use > bufferView->Buffer()->Externalize(); > before getting the data pointer. In this case I'm then responsible for > freeing the data - If this is necessary, could you explain how to do this? > Can it be done without altering the ArrayBufferAllocator? > > > Many thanks! > George Corney GetIndexedPropertiesExternalArrayData() is the wrong approach if you plan on upgrading someday - it was removed in V8 3.29. In your case, I'd use GetContents() coupled with a Persistent for keeping the arraybuffer object alive until you're done with it. If GetContents() doesn't exist in the version of V8 you're using, upgrade. That's a good idea anyway because the 3.28 branch is retired, the last update was in 2014. -- -- v8-users mailing list v8-users@googlegroups.com http://groups.google.com/group/v8-users --- You received this message because you are subscribed to the Google Groups "v8-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to v8-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [v8-users] v8 3.28, getting data from typed arrays, I'm unsure if this approach is correct (and doesn't leak)
Please note that 3.28 is a really old version of V8 which is full of known issues and no longer maintained. In general, you'll wait to externalize the buffer and keep a weak persistent handle to the buffer - once the weak callback was triggered, you know that you can free the backing store (if you no longer need it otherwise). More current versions of V8 also have a method ArrayBufferView::CopyContents that allows you to access the backing store without externalizing the underlying buffer. On Sun, Feb 21, 2016 at 11:27 PM George Corney wrote: > Hello, > > I'm working with typed arrays in v8 3.28, I've got a native function that > takes a Float32Array from javascript and passes the float* data off to a gl > call (glBufferData). > > I discovered that using just > const void* rawData = bufferView->GetIndexedPropertiesExternalArrayData(); > works, but not reliably, the data gets passes to glBufferData correctly > most of the time, but not always! > > I've found creating a persistent handle before getting the data pointer, > and then reseting handle after the glBufferData solves the problem, but is > it the right approach? Am I leaking memory / risking sending incorrect data > to glBufferData? Here's the snippet > > v8::Local bufferView = > dataArg.As(); > size_t byteLength = bufferView->ByteLength(); > size_t byteOffset = bufferView->ByteOffset(); > > v8::Persistent persistent; > persistent.Reset(__contextORisolate, bufferView->Buffer()); > > const void* rawData = bufferView->GetIndexedPropertiesExternalArrayData(); > const unsigned char* bytePtr = static_cast(rawData); > > glBufferData(target, byteLength, bytePtr + byteOffset, usage); > REPORT_GL_ERRORS(); > > persistent.Reset(); > > > --- > > If this the wrong approach, I think the next thing is to use > bufferView->Buffer()->Externalize(); > before getting the data pointer. In this case I'm then responsible for > freeing the data - If this is necessary, could you explain how to do this? > Can it be done without altering the ArrayBufferAllocator? > > > Many thanks! > George Corney > > -- > -- > v8-users mailing list > v8-users@googlegroups.com > http://groups.google.com/group/v8-users > --- > You received this message because you are subscribed to the Google Groups > "v8-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to v8-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- -- v8-users mailing list v8-users@googlegroups.com http://groups.google.com/group/v8-users --- You received this message because you are subscribed to the Google Groups "v8-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to v8-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[v8-users] v8 3.28, getting data from typed arrays, I'm unsure if this approach is correct (and doesn't leak)
Hello, I'm working with typed arrays in v8 3.28, I've got a native function that takes a Float32Array from javascript and passes the float* data off to a gl call (glBufferData). I discovered that using just const void* rawData = bufferView->GetIndexedPropertiesExternalArrayData(); works, but not reliably, the data gets passes to glBufferData correctly most of the time, but not always! I've found creating a persistent handle before getting the data pointer, and then reseting handle after the glBufferData solves the problem, but is it the right approach? Am I leaking memory / risking sending incorrect data to glBufferData? Here's the snippet v8::Local bufferView = dataArg.As(); size_t byteLength = bufferView->ByteLength(); size_t byteOffset = bufferView->ByteOffset(); v8::Persistent persistent; persistent.Reset(__contextORisolate, bufferView->Buffer()); const void* rawData = bufferView->GetIndexedPropertiesExternalArrayData(); const unsigned char* bytePtr = static_cast(rawData); glBufferData(target, byteLength, bytePtr + byteOffset, usage); REPORT_GL_ERRORS(); persistent.Reset(); --- If this the wrong approach, I think the next thing is to use bufferView->Buffer()->Externalize(); before getting the data pointer. In this case I'm then responsible for freeing the data - If this is necessary, could you explain how to do this? Can it be done without altering the ArrayBufferAllocator? Many thanks! George Corney -- -- v8-users mailing list v8-users@googlegroups.com http://groups.google.com/group/v8-users --- You received this message because you are subscribed to the Google Groups "v8-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to v8-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
