Re: [v8-dev] Guidance on internals of `v8::ArrayBuffer::Data()` method
It seem I got a clean bill of health locally after fixing the obvious bug I had: >>> Running tests for x64.debug >>> Running with test processors [70:10|% 96|+ 18516|- 0]: Done >>> 19139 base tests produced 18516 (96%) non-filtered tests >>> 18516 tests ran On Wednesday, 27 September 2023 at 15:25:18 UTC+3 Aapo Alasuutari wrote: > Oops, yeah. > > Locally I already have related failed DCHECKs so this ain't gonna be > pretty. > > # Fatal error in ../../src/sandbox/sandboxed-pointer-inl.h, line 35 > # Check failed: GetProcessWideSandbox()->Contains(pointer). > # > # > # > #FailureMessage Object: 0x7fad9b7e54f8 > C stack trace === > > > /sources/aapoalas/v8/v8/out/x64.debug/libv8_libbase.so(v8::base::debug::StackTrace::StackTrace()+0x1e) > > [0x7fae87a1ef0e] > /sources/aapoalas/v8/v8/out/x64.debug/libv8_libplatform.so(+0x5219d) > [0x7fae8797419d] > /sources/aapoalas/v8/v8/out/x64.debug/libv8_libbase.so(V8_Fatal(char > const*, int, char const*, ...)+0x1ac) [0x7fae879c] > > /sources/aapoalas/v8/v8/out/x64.debug/cctest(v8::internal::WriteSandboxedPointerField(unsigned > > long, v8::internal::PtrComprCageBase, unsigned long)+0x59) [0x5561832b6a49] > > /sources/aapoalas/v8/v8/out/x64.debug/libv8_for_testing.so(v8::internal::HeapObject::WriteSandboxedPointerField(unsigned > > long, v8::internal::Isolate*, unsigned long)+0x47) [0x7fae8ce2eee7] > > /sources/aapoalas/v8/v8/out/x64.debug/libv8_for_testing.so(v8::internal::JSArrayBuffer::set_backing_store(v8::internal::Isolate*, > > void*)+0x40) [0x7fae8da717d0] > > /sources/aapoalas/v8/v8/out/x64.debug/libv8_for_testing.so(v8::internal::JSArrayBuffer::Attach(std::__Cr::shared_ptr)+0x45c) > > [0x7fae8da6e81c] > > On Wednesday, 27 September 2023 at 15:18:43 UTC+3 Clemens Backes wrote: > >> I guess you meant to link to https://crrev.com/c/4896678. I triggered >> dry-runs, let's see what happens. >> >> On Wed, Sep 27, 2023 at 2:12 PM Aapo Alasuutari >> wrote: >> >>> Thank you for the response! >>> >>> Hopefully this was then much easier than I even expected. I opened this >>> CL: https://groups.google.com/g/v8-dev/c/wuncGizO1EU >>> Unfortunately I'm not a dry-runner so I cannot start try-bots on this >>> myself. I'll try running some tests locally at least. >>> >>> -Aapo >>> >>> On Wednesday, 27 September 2023 at 14:15:23 UTC+3 Clemens Backes wrote: >>> >>>> This is the place where we store the special "empty backing store >>>> buffer" in the ArrayBuffer if the passed BackingStore is empty: >>>> >>>> https://source.chromium.org/chromium/chromium/src/+/main:v8/src/objects/js-array-buffer.cc;l=82;drc=57bf7660f3e50a0f68f329059f0dff8f641effc4 >>>> >>>> In a non-sandbox build, this will just store nullptr. >>>> >>>> That said, I can't tell you why we have this optimization and which >>>> part(s) of the system depend on that. As the backing store is kept alive >>>> by >>>> the ArrayBuffer anyway, I guess we could also just store the actual >>>> buffer's start in the ArrayBuffer::backing_store field. >>>> Waiting to be corrected :) >>>> >>>> On Wed, Sep 27, 2023 at 11:22 AM Aapo Alasuutari >>>> wrote: >>>> >>>>> Hello >>>>> >>>>> I'm trying to take a look at the `v8::ArrayBuffer::Data()` method with >>>>> the intention of fixing this bug: >>>>> https://bugs.chromium.org/p/v8/issues/detail?id=13488 (and by >>>>> extension possibly unblock >>>>> https://bugs.chromium.org/p/v8/issues/detail?id=13489) >>>>> >>>>> Put it short: The method returns a null pointer for all zero-length >>>>> buffers even when the ArrayBuffer is internally backed by an external >>>>> pointer. These sorts of externally backed zero-length buffers are >>>>> sometimes >>>>> used in eg. Node API to pass opaque pointers to and from JavaScript. >>>>> Getting the proper pointer requires using the >>>>> `v8::ArrayBuffer::GetBackingStore()` API, after which its `Data()` API >>>>> returns the real external pointer. >>>>> >>>>> I've been trying to track where this difference springs from but >>>>> haven't had much success. The `Data()` method calls the `backing_store()` >>&g
Re: [v8-dev] Guidance on internals of `v8::ArrayBuffer::Data()` method
Oops, yeah.
Locally I already have related failed DCHECKs so this ain't gonna be pretty.
# Fatal error in ../../src/sandbox/sandboxed-pointer-inl.h, line 35
# Check failed: GetProcessWideSandbox()->Contains(pointer).
#
#
#
#FailureMessage Object: 0x7fad9b7e54f8
C stack trace ===
/sources/aapoalas/v8/v8/out/x64.debug/libv8_libbase.so(v8::base::debug::StackTrace::StackTrace()+0x1e)
[0x7fae87a1ef0e]
/sources/aapoalas/v8/v8/out/x64.debug/libv8_libplatform.so(+0x5219d)
[0x7fae8797419d]
/sources/aapoalas/v8/v8/out/x64.debug/libv8_libbase.so(V8_Fatal(char
const*, int, char const*, ...)+0x1ac) [0x7fae879c]
/sources/aapoalas/v8/v8/out/x64.debug/cctest(v8::internal::WriteSandboxedPointerField(unsigned
long, v8::internal::PtrComprCageBase, unsigned long)+0x59) [0x5561832b6a49]
/sources/aapoalas/v8/v8/out/x64.debug/libv8_for_testing.so(v8::internal::HeapObject::WriteSandboxedPointerField(unsigned
long, v8::internal::Isolate*, unsigned long)+0x47) [0x7fae8ce2eee7]
/sources/aapoalas/v8/v8/out/x64.debug/libv8_for_testing.so(v8::internal::JSArrayBuffer::set_backing_store(v8::internal::Isolate*,
void*)+0x40) [0x7fae8da717d0]
/sources/aapoalas/v8/v8/out/x64.debug/libv8_for_testing.so(v8::internal::JSArrayBuffer::Attach(std::__Cr::shared_ptr)+0x45c)
[0x7fae8da6e81c]
On Wednesday, 27 September 2023 at 15:18:43 UTC+3 Clemens Backes wrote:
> I guess you meant to link to https://crrev.com/c/4896678. I triggered
> dry-runs, let's see what happens.
>
> On Wed, Sep 27, 2023 at 2:12 PM Aapo Alasuutari
> wrote:
>
>> Thank you for the response!
>>
>> Hopefully this was then much easier than I even expected. I opened this
>> CL: https://groups.google.com/g/v8-dev/c/wuncGizO1EU
>> Unfortunately I'm not a dry-runner so I cannot start try-bots on this
>> myself. I'll try running some tests locally at least.
>>
>> -Aapo
>>
>> On Wednesday, 27 September 2023 at 14:15:23 UTC+3 Clemens Backes wrote:
>>
>>> This is the place where we store the special "empty backing store
>>> buffer" in the ArrayBuffer if the passed BackingStore is empty:
>>>
>>> https://source.chromium.org/chromium/chromium/src/+/main:v8/src/objects/js-array-buffer.cc;l=82;drc=57bf7660f3e50a0f68f329059f0dff8f641effc4
>>>
>>> In a non-sandbox build, this will just store nullptr.
>>>
>>> That said, I can't tell you why we have this optimization and which
>>> part(s) of the system depend on that. As the backing store is kept alive by
>>> the ArrayBuffer anyway, I guess we could also just store the actual
>>> buffer's start in the ArrayBuffer::backing_store field.
>>> Waiting to be corrected :)
>>>
>>> On Wed, Sep 27, 2023 at 11:22 AM Aapo Alasuutari
>>> wrote:
>>>
>>>> Hello
>>>>
>>>> I'm trying to take a look at the `v8::ArrayBuffer::Data()` method with
>>>> the intention of fixing this bug:
>>>> https://bugs.chromium.org/p/v8/issues/detail?id=13488 (and by
>>>> extension possibly unblock
>>>> https://bugs.chromium.org/p/v8/issues/detail?id=13489)
>>>>
>>>> Put it short: The method returns a null pointer for all zero-length
>>>> buffers even when the ArrayBuffer is internally backed by an external
>>>> pointer. These sorts of externally backed zero-length buffers are
>>>> sometimes
>>>> used in eg. Node API to pass opaque pointers to and from JavaScript.
>>>> Getting the proper pointer requires using the
>>>> `v8::ArrayBuffer::GetBackingStore()` API, after which its `Data()` API
>>>> returns the real external pointer.
>>>>
>>>> I've been trying to track where this difference springs from but
>>>> haven't had much success. The `Data()` method calls the `backing_store()`
>>>> method of a `i::Handle` which I _think_ is defined with
>>>> the `DEF_GETTER` macro in `js-array-buffer-inl.h` line 48:
>>>>
>>>> DEF_GETTER(JSArrayBuffer, backing_store, void*) {
>>>> Address value = ReadSandboxedPointerField(kBackingStoreOffset,
>>>> cage_base);
>>>> return reinterpret_cast(value);
>>>> }
>>>>
>>>> But here I get confused: `ReadSandboxedPointerField` (in
>>>> `sandboxed-pointer-inl.h`) seems simple enough that there should be
>>>> absolutely no checks against the length of the buffer, nor does it seem
>>>> particularly likely for the backing store offset parameter to be somehow
>>>> wrong.
>>>>
>>>> If anyone has an i
Re: [v8-dev] Guidance on internals of `v8::ArrayBuffer::Data()` method
Thank you for the response!
Hopefully this was then much easier than I even expected. I opened this
CL: https://groups.google.com/g/v8-dev/c/wuncGizO1EU
Unfortunately I'm not a dry-runner so I cannot start try-bots on this
myself. I'll try running some tests locally at least.
-Aapo
On Wednesday, 27 September 2023 at 14:15:23 UTC+3 Clemens Backes wrote:
> This is the place where we store the special "empty backing store buffer"
> in the ArrayBuffer if the passed BackingStore is empty:
>
> https://source.chromium.org/chromium/chromium/src/+/main:v8/src/objects/js-array-buffer.cc;l=82;drc=57bf7660f3e50a0f68f329059f0dff8f641effc4
>
> In a non-sandbox build, this will just store nullptr.
>
> That said, I can't tell you why we have this optimization and which
> part(s) of the system depend on that. As the backing store is kept alive by
> the ArrayBuffer anyway, I guess we could also just store the actual
> buffer's start in the ArrayBuffer::backing_store field.
> Waiting to be corrected :)
>
> On Wed, Sep 27, 2023 at 11:22 AM Aapo Alasuutari
> wrote:
>
>> Hello
>>
>> I'm trying to take a look at the `v8::ArrayBuffer::Data()` method with
>> the intention of fixing this bug:
>> https://bugs.chromium.org/p/v8/issues/detail?id=13488 (and by extension
>> possibly unblock https://bugs.chromium.org/p/v8/issues/detail?id=13489)
>>
>> Put it short: The method returns a null pointer for all zero-length
>> buffers even when the ArrayBuffer is internally backed by an external
>> pointer. These sorts of externally backed zero-length buffers are sometimes
>> used in eg. Node API to pass opaque pointers to and from JavaScript.
>> Getting the proper pointer requires using the
>> `v8::ArrayBuffer::GetBackingStore()` API, after which its `Data()` API
>> returns the real external pointer.
>>
>> I've been trying to track where this difference springs from but haven't
>> had much success. The `Data()` method calls the `backing_store()` method of
>> a `i::Handle` which I _think_ is defined with the
>> `DEF_GETTER` macro in `js-array-buffer-inl.h` line 48:
>>
>> DEF_GETTER(JSArrayBuffer, backing_store, void*) {
>> Address value = ReadSandboxedPointerField(kBackingStoreOffset,
>> cage_base);
>> return reinterpret_cast(value);
>> }
>>
>> But here I get confused: `ReadSandboxedPointerField` (in
>> `sandboxed-pointer-inl.h`) seems simple enough that there should be
>> absolutely no checks against the length of the buffer, nor does it seem
>> particularly likely for the backing store offset parameter to be somehow
>> wrong.
>>
>> If anyone has an idea of where I should look into, that'd be much
>> appreciated
>> -Aapo Alasuutari
>>
>> --
>> --
>> v8-dev mailing list
>> v8-...@googlegroups.com
>> http://groups.google.com/group/v8-dev
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "v8-dev" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to v8-dev+un...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/v8-dev/350ede0e-0c6d-433e-b9b3-a85525c7049fn%40googlegroups.com
>>
>> <https://groups.google.com/d/msgid/v8-dev/350ede0e-0c6d-433e-b9b3-a85525c7049fn%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>
>
> --
>
> Clemens Backes
>
> Software Engineer
>
> clem...@google.com
>
> Google Germany GmbH
>
> Erika-Mann-Straße 33
>
> 80636 München
>
> Geschäftsführer: Paul Manicle, Liana Sebastian
>
> Registergericht und -nummer: Hamburg, HRB 86891
>
> Sitz der Gesellschaft: Hamburg
>
> Diese E-Mail ist vertraulich. Falls sie diese fälschlicherweise erhalten
> haben sollten, leiten Sie diese bitte nicht an jemand anderes weiter,
> löschen Sie alle Kopien und Anhänge davon und lassen Sie mich bitte wissen,
> dass die E-Mail an die falsche Person gesendet wurde.
>
>
> This e-mail is confidential. If you received this communication by
> mistake, please don't forward it to anyone else, please erase all copies
> and attachments, and please let me know that it has gone to the wrong
> person.
>
>
>
--
--
v8-dev mailing list
v8-dev@googlegroups.com
http://groups.google.com/group/v8-dev
---
You received this message because you are subscribed to the Google Groups
"v8-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to v8-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/v8-dev/25a7a109-5b44-4250-b5f2-bc060ef307b4n%40googlegroups.com.
[v8-dev] Guidance on internals of `v8::ArrayBuffer::Data()` method
Hello
I'm trying to take a look at the `v8::ArrayBuffer::Data()` method with the
intention of fixing this
bug: https://bugs.chromium.org/p/v8/issues/detail?id=13488 (and by
extension possibly
unblock https://bugs.chromium.org/p/v8/issues/detail?id=13489)
Put it short: The method returns a null pointer for all zero-length buffers
even when the ArrayBuffer is internally backed by an external pointer.
These sorts of externally backed zero-length buffers are sometimes used in
eg. Node API to pass opaque pointers to and from JavaScript. Getting the
proper pointer requires using the `v8::ArrayBuffer::GetBackingStore()` API,
after which its `Data()` API returns the real external pointer.
I've been trying to track where this difference springs from but haven't
had much success. The `Data()` method calls the `backing_store()` method of
a `i::Handle` which I _think_ is defined with the
`DEF_GETTER` macro in `js-array-buffer-inl.h` line 48:
DEF_GETTER(JSArrayBuffer, backing_store, void*) {
Address value = ReadSandboxedPointerField(kBackingStoreOffset, cage_base);
return reinterpret_cast(value);
}
But here I get confused: `ReadSandboxedPointerField` (in
`sandboxed-pointer-inl.h`) seems simple enough that there should be
absolutely no checks against the length of the buffer, nor does it seem
particularly likely for the backing store offset parameter to be somehow
wrong.
If anyone has an idea of where I should look into, that'd be much
appreciated
-Aapo Alasuutari
--
--
v8-dev mailing list
v8-dev@googlegroups.com
http://groups.google.com/group/v8-dev
---
You received this message because you are subscribed to the Google Groups
"v8-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to v8-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/v8-dev/350ede0e-0c6d-433e-b9b3-a85525c7049fn%40googlegroups.com.
Re: [v8-dev] `void*` pointer support for fast calls
this out pointer system and its (slight) performance overhead could be >>>> removed. (And most importantly, numbers-as-pointers insecurity could be >>>> eliminated.) >>>> Returning of C strings would allow Deno FFI to have "native" support of >>>> those (currently C string extraction is done via a separate method). >>>> >>>> 2. Deno's ops layer has recently moved to using Fast API by default >>>> where possible. Deno's binding functions are written as normal Rust >>>> functions and an ops macro takes care of writing the binding logic to V8's >>>> FunctionTemplate. >>>> Due to the near-universality of the ops macro, any Fast API binding >>>> logic needs to only be written once and the macro will take care of taking >>>> setting up the bindings for all ops that are bindable. Thus, here even >>>> more >>>> than with FFI, having more supported types leads near-automatically to >>>> faster binding layer in Deno, which is very much of interest to the Deno >>>> team. >>>> Some examples: >>>> * FFI might not benefit from Strings as parameters that much, since >>>> foreign APIs would only expect C strings. Deno ops however very much would >>>> like to get arbitrary (UTF8) strings in fast calls. They would also love >>>> to >>>> return arbitrary UTF8 strings. >>>> * FFI only cares about returning pointers in some form, External being >>>> the most logical. Deno ops would very much want to return TypedArrays of >>>> varying sizes, and they would not mind being explicit about memory >>>> management either. >>>> * ops have cases where eg. a String or TypedArray parameter might be >>>> optional. Overloads are already supported to a degree, but eg. null >>>> parameters in the middle currently are not supported directly (except as >>>> v8::Values which I'm not sure if it would ruin the "better typed" overload >>>> matching) >>>> * (Completely impossible stretch goal): Some ops take objects of some >>>> given shape. If V8 were to match its JS object shapes to a declared >>>> parameter struct shape, now that would be impossibly cool. Also, probably >>>> too hard to feasibly do but a man can dream. >>>> >>>> >>>> This has become a massive, meandering writeup. Sorry about that. >>>> >>>> Back onto topic: If you can give me some pointers on where I should >>>> look to add the External stuff for, I would much >>>> appreciate it. I would personally also prefer to write the code such that >>>> the C callback receives not the v8::External object but is directly called >>>> with the pointer that the External represents. This I expect to require >>>> some changes in the lowering code. >>>> >>>> Thank you for your time >>>> -Aapo Alasuutari >>>> >>>> On Friday, 30 September 2022 at 11:23:56 UTC+3 msle...@chromium.org >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> First of all I'm really sorry for the late reply, I didn't see >>>>> Leszek's ping in time. >>>>> >>>>> External sounds like the right type to represent embedder pointers, >>>>> though the poor performance you report sounds unfortunate. Tbh I'm not >>>>> aware of particular efforts to optimize it, but it might be indeed due to >>>>> the ExternalMap. I'll check with colleagues if it's possible to do >>>>> something about the performance there. >>>>> >>>>> On the main topic, adding C callbacks that accept an argument of type >>>>> External>>>> <https://source.chromium.org/chromium/chromium/src/+/main:v8/src/objects/js-objects.h;drc=ca79bd5301566d1a3fc573c6e6858b5880c00fbd;bpv=1;bpt=1;l=911?gsn=JSExternalObject=kythe%3A%2F%2Fchromium.googlesource.com%2Fchromium%2Fsrc%3Flang%3Dc%252B%252B%3Fpath%3Dv8%2Fsrc%2Fobjects%2Fjs-objects.h%23JSExternalObject%253Ainternal%253Av8%2523c%2523iz6AV1GPx3E=kythe%3A%2F%2Fchromium.googlesource.com%2Fchromium%2Fsrc%3Flang%3Dc%252B%252B%3Fpath%3Dv8%2Fsrc%2Fobjects%2Fjs-objects.h%23JSExternalObject%253Ainternal%253Av8%2523c%2523bNyn58S6iE1=kythe%3A%2F%2Fchromium.googlesource.com%2Fchromium%2Fsrc%3Flang%3Dc%252B%252B%3Fpath%3Dv8%2Fsrc%2Fobjects%2Fjs-objects.h%23E1nu-FvBjuQ-EDx8Ny1DO3ZL7UJtt6bOOeiU34UFYGw=kythe%3A%2F%2Fchromium.googlesource.com%2Fchrom
Re: [v8-dev] `void*` pointer support for fast calls
what the team >> considers important and am just speaking for myself. That being said: >> 1. Deno's FFI API relies heavily on Fast API. Every foreign library's >> symbol (C function) that a user wants to use will by default use the Fast >> API. Only symbols that call back into V8 need to / should opt out of this >> using a "callback" boolean option. >> Adding more supported types to Fast API directly adds to wider and better >> support for Deno FFI. As an example, currently returning of 64 bit integers >> (eg. pointers) is done via a TypedArray out pointer, where the pointer is >> written into. If returning of External objects was possible, this out >> pointer system and its (slight) performance overhead could be removed. (And >> most importantly, numbers-as-pointers insecurity could be eliminated.) >> Returning of C strings would allow Deno FFI to have "native" support of >> those (currently C string extraction is done via a separate method). >> >> 2. Deno's ops layer has recently moved to using Fast API by default where >> possible. Deno's binding functions are written as normal Rust functions and >> an ops macro takes care of writing the binding logic to V8's >> FunctionTemplate. >> Due to the near-universality of the ops macro, any Fast API binding logic >> needs to only be written once and the macro will take care of taking >> setting up the bindings for all ops that are bindable. Thus, here even more >> than with FFI, having more supported types leads near-automatically to >> faster binding layer in Deno, which is very much of interest to the Deno >> team. >> Some examples: >> * FFI might not benefit from Strings as parameters that much, since >> foreign APIs would only expect C strings. Deno ops however very much would >> like to get arbitrary (UTF8) strings in fast calls. They would also love to >> return arbitrary UTF8 strings. >> * FFI only cares about returning pointers in some form, External being >> the most logical. Deno ops would very much want to return TypedArrays of >> varying sizes, and they would not mind being explicit about memory >> management either. >> * ops have cases where eg. a String or TypedArray parameter might be >> optional. Overloads are already supported to a degree, but eg. null >> parameters in the middle currently are not supported directly (except as >> v8::Values which I'm not sure if it would ruin the "better typed" overload >> matching) >> * (Completely impossible stretch goal): Some ops take objects of some >> given shape. If V8 were to match its JS object shapes to a declared >> parameter struct shape, now that would be impossibly cool. Also, probably >> too hard to feasibly do but a man can dream. >> >> >> This has become a massive, meandering writeup. Sorry about that. >> >> Back onto topic: If you can give me some pointers on where I should look >> to add the External stuff for, I would much appreciate >> it. I would personally also prefer to write the code such that the C >> callback receives not the v8::External object but is directly called with >> the pointer that the External represents. This I expect to require some >> changes in the lowering code. >> >> Thank you for your time >> -Aapo Alasuutari >> >> On Friday, 30 September 2022 at 11:23:56 UTC+3 msle...@chromium.org >> wrote: >> >>> Hi, >>> >>> First of all I'm really sorry for the late reply, I didn't see Leszek's >>> ping in time. >>> >>> External sounds like the right type to represent embedder pointers, >>> though the poor performance you report sounds unfortunate. Tbh I'm not >>> aware of particular efforts to optimize it, but it might be indeed due to >>> the ExternalMap. I'll check with colleagues if it's possible to do >>> something about the performance there. >>> >>> On the main topic, adding C callbacks that accept an argument of type >>> External>> <https://source.chromium.org/chromium/chromium/src/+/main:v8/src/objects/js-objects.h;drc=ca79bd5301566d1a3fc573c6e6858b5880c00fbd;bpv=1;bpt=1;l=911?gsn=JSExternalObject=kythe%3A%2F%2Fchromium.googlesource.com%2Fchromium%2Fsrc%3Flang%3Dc%252B%252B%3Fpath%3Dv8%2Fsrc%2Fobjects%2Fjs-objects.h%23JSExternalObject%253Ainternal%253Av8%2523c%2523iz6AV1GPx3E=kythe%3A%2F%2Fchromium.googlesource.com%2Fchromium%2Fsrc%3Flang%3Dc%252B%252B%3Fpath%3Dv8%2Fsrc%2Fobjects%2Fjs-objects.h%23JSExternalObject%253Ainternal%253Av8%2523c%2523bNyn58S6iE1=kythe%3A%2F%2Fchromium.googlesource.com%2Fchromium%2Fsrc%3Flang%3Dc%252B%
Re: [v8-dev] `void*` pointer support for fast calls
be optional. Overloads are already supported to a degree, but eg. null parameters in the middle currently are not supported directly (except as v8::Values which I'm not sure if it would ruin the "better typed" overload matching) * (Completely impossible stretch goal): Some ops take objects of some given shape. If V8 were to match its JS object shapes to a declared parameter struct shape, now that would be impossibly cool. Also, probably too hard to feasibly do but a man can dream. This has become a massive, meandering writeup. Sorry about that. Back onto topic: If you can give me some pointers on where I should look to add the External stuff for, I would much appreciate it. I would personally also prefer to write the code such that the C callback receives not the v8::External object but is directly called with the pointer that the External represents. This I expect to require some changes in the lowering code. Thank you for your time -Aapo Alasuutari On Friday, 30 September 2022 at 11:23:56 UTC+3 msle...@chromium.org wrote: > Hi, > > First of all I'm really sorry for the late reply, I didn't see Leszek's > ping in time. > > External sounds like the right type to represent embedder pointers, though > the poor performance you report sounds unfortunate. Tbh I'm not aware of > particular efforts to optimize it, but it might be indeed due to the > ExternalMap. I'll check with colleagues if it's possible to do something > about the performance there. > > On the main topic, adding C callbacks that accept an argument of type > External <https://source.chromium.org/chromium/chromium/src/+/main:v8/src/objects/js-objects.h;drc=ca79bd5301566d1a3fc573c6e6858b5880c00fbd;bpv=1;bpt=1;l=911?gsn=JSExternalObject=kythe%3A%2F%2Fchromium.googlesource.com%2Fchromium%2Fsrc%3Flang%3Dc%252B%252B%3Fpath%3Dv8%2Fsrc%2Fobjects%2Fjs-objects.h%23JSExternalObject%253Ainternal%253Av8%2523c%2523iz6AV1GPx3E=kythe%3A%2F%2Fchromium.googlesource.com%2Fchromium%2Fsrc%3Flang%3Dc%252B%252B%3Fpath%3Dv8%2Fsrc%2Fobjects%2Fjs-objects.h%23JSExternalObject%253Ainternal%253Av8%2523c%2523bNyn58S6iE1=kythe%3A%2F%2Fchromium.googlesource.com%2Fchromium%2Fsrc%3Flang%3Dc%252B%252B%3Fpath%3Dv8%2Fsrc%2Fobjects%2Fjs-objects.h%23E1nu-FvBjuQ-EDx8Ny1DO3ZL7UJtt6bOOeiU34UFYGw=kythe%3A%2F%2Fchromium.googlesource.com%2Fchromium%2Fsrc%3Flang%3Dc%252B%252B%3Fpath%3Dout%2Fandroid-Debug%2Fgen%2Fv8%2Ftorque-generated%2Fclass-forward-declarations.h%23oOxlJQnIiQ9TdjrwzIe-NzNBbjuKOvOptxzBUUoilOc=kythe%3A%2F%2Fchromium.googlesource.com%2Fchromium%2Fsrc%3Flang%3Dc%252B%252B%3Fpath%3Dout%2Fandroid-Debug%2Fgen%2Fv8%2Ftorque-generated%2Fsrc%2Fobjects%2Fjs-objects-tq.inc%23L9uwfd2l6uOWvbRRILcxKp1VYllIsCCFPIecleuaEFI=kythe%3A%2F%2Fchromium.googlesource.com%2Fchromium%2Fsrc%3Flang%3Dc%252B%252B%3Fpath%3Dv8%2Fsrc%2Fheap%2Fobjects-visiting.h%23TyseKlOYyb_hrIxmiwvcWeGQvP1INehKCer2kV7xG6o=kythe%3A%2F%2Fchromium.googlesource.com%2Fchromium%2Fsrc%3Flang%3Dc%252B%252B%3Fpath%3Dout%2Fchromeos-Debug%2Fgen%2Fv8%2Ftorque-generated%2Fclass-forward-declarations.h%23-xwKU2vBSmUrnUm0jR5GI1mRjxJU6CM4EWCILgXHArg=kythe%3A%2F%2Fchromium.googlesource.com%2Fchromium%2Fsrc%3Flang%3Dc%252B%252B%3Fpath%3Dout%2Fchromeos-Debug%2Fgen%2Fv8%2Ftorque-generated%2Fsrc%2Fobjects%2Fjs-objects-tq.inc%233_drfRrSdKh0O1Osknb0zTSBaGNV_S6BOBULYV3JPWM=kythe%3A%2F%2Fchromium.googlesource.com%2Fchromium%2Fsrc%3Flang%3Dc%252B%252B%3Fpath%3Dout%2Ffuchsia-Debug%2Fgen%2Fv8%2Ftorque-generated%2Fclass-forward-declarations.h%234v0WVozoJqd2EihNGkWvPiA8BoPsJcdwXuoyVp6QHOQ=kythe%3A%2F%2Fchromium.googlesource.com%2Fchromium%2Fsrc%3Flang%3Dc%252B%252B%3Fpath%3Dout%2Ffuchsia-Debug%2Fgen%2Fv8%2Ftorque-generated%2Fsrc%2Fobjects%2Fjs-objects-tq.inc%23LSy3_Kz2Yjawt37W-b83WmwOsNDUY4ajMD0DAb7V3Mc=kythe%3A%2F%2Fchromium.googlesource.com%2Fchromium%2Fsrc%3Flang%3Dc%252B%252B%3Fpath%3Dout%2FDebug%2Fgen%2Fv8%2Ftorque-generated%2Fclass-forward-declarations.h%23mTH4BKG-IJiVtR9UIw5dWrwbrfkz9qT70rqpwU4XjrU=kythe%3A%2F%2Fchromium.googlesource.com%2Fchromium%2Fsrc%3Flang%3Dc%252B%252B%3Fpath%3Dout%2FDebug%2Fgen%2Fv8%2Ftorque-generated%2Fsrc%2Fobjects%2Fjs-objects-tq.inc%23whuyQbV2vlE-tTFI1uVnQfjP17lKMmgcfoUlNVo-Klw=kythe%3A%2F%2Fchromium.googlesource.com%2Fchromium%2Fsrc%3Flang%3Dc%252B%252B%3Fpath%3Dout%2Fmac-Debug%2Fgen%2Fv8%2Ftorque-generated%2Fclass-forward-declarations.h%23GzTsTV0SjMocKlp1gFc9rdY5cM8CLb1snxpk-K_Yl6w=kythe%3A%2F%2Fchromium.googlesource.com%2Fchromium%2Fsrc%3Flang%3Dc%252B%252B%3Fpath%3Dout%2Fmac-Debug%2Fgen%2Fv8%2Ftorque-generated%2Fsrc%2Fobjects%2Fjs-objects-tq.inc%23e8ogLQjp7_4HdRE3K0x5oqhCsc1cPbcQrRxJ93YsGgY=kythe%3A%2F%2Fchromium.googlesource.com%2Fchromium%2Fsrc%3Flang%3Dc%252B%252B%3Fpath%3Dout%2Fwebview-Debug%2Fclang_x86_v8_arm%2Fgen%2Fv8%2Ftorque-generated%2Fclass-forward-declarations.h%23eCSrpBoyJxJqKLLRwerXtYRgNRCFPzhqMlFvXB1GgXk=kythe%3A%2F%2Fchromium.googlesource.com%2Fchromium%2Fsrc%3Flang%3Dc%252B%252B%3Fpath%3Dout%2F
Re: [v8-dev] `void*` pointer support for fast calls
Still hoping to get some guidance with this. I'm also interested in support, even if limited, for string value parameters (or even return values) and returning of TypedArray buffers. Though, I expect those to be much harder to implement than returning External objects for void pointers. I guess a somewhat related option is to return external pointers as zero-sized TypedArrays / ArrayBuffers, but that sounds quite wrong compared to External objects. On Friday, 23 September 2022 at 15:15:10 UTC+3 Aapo Alasuutari wrote: > I presume Maya might now be back be at the office? > > Would it be possible to get some guidance regarding implementing void > pointer support, either here on Groups or possibly by organizing an online > meeting of some sort? > > -Aapo > > On Tuesday, 23 August 2022 at 11:32:30 UTC+3 cbr...@chromium.org wrote: > >> Hi, yes, Maya is out until mid-september. >> Cheers, Camillo >> >> On Tue, 23 Aug 2022 at 07:07, Aapo Alasuutari >> wrote: >> >>> Has Maya possibly returned from vacation? Or is their leave still >>> continuing? >>> >>> On Friday, 29 July 2022 at 12:08:53 UTC+3 ah...@google.com wrote: >>> >>>> Maya is on leave over the summer, unfortunately. >>>> >>>> On Fri, Jul 29, 2022 at 11:02 AM Leszek Swirski >>>> wrote: >>>> >>>>> +Maya, you're probably the best person to answer this. >>>>> >>>>> On Tue, Jul 26, 2022 at 9:05 PM Aapo Alasuutari >>>>> wrote: >>>>> >>>>>> Hello, >>>>>> >>>>>> I'm interested in implementing `void*` pointer support for Fast API >>>>>> calls. My thinking was that V8's `External` objects are appropriate to >>>>>> stand in for external `void*` pointers coming in from external code and >>>>>> going back out, since that's what they're (presumably) meant for. >>>>>> >>>>>> Unfortunately this seems to be a complex endeavour, a bit more than I >>>>>> can start hacking together directly. I'm also not sure if the >>>>>> `Sandboxify >>>>>> JSExternalObject external pointer` PR will complicate this plan of mine. >>>>>> >>>>>> The origin of my interest is Deno FFI support, that is calling native >>>>>> libraries from Deno JS runtime that uses the V8 engine. Recent changes >>>>>> to >>>>>> the FFI have added V8 Fast API support and made the FFI a lot faster, >>>>>> but >>>>>> unfortunately we're bound to using plain numbers as pointers, meaning >>>>>> both >>>>>> that creating pointers is as easy as just writing a number and that >>>>>> (Fast >>>>>> API compatible) pointers are limited to 53 bit numbers which will not be >>>>>> enough for eg. pointer cryptography on ARM v8.3. >>>>>> >>>>>> It believe it would be preferable if Deno could use `External` >>>>>> objects to stand for pointers but this would negate the current Fast API >>>>>> performance benefits. Thus, `void*` pointer support for fast calls. >>>>>> >>>>>> >>>>>> Any comments? Suggestions on how I might best proceed with this to >>>>>> implement it? Or is this perhaps not a reasonable idea? >>>>>> >>>>>> Side note: I was sad to find that getting the pointer value out of an >>>>>> `Local` is measurably slower than getting the pointer number >>>>>> value out of a `Local`. This is presumably due to the `External` >>>>>> internally saving the pointer in the `ExternalMap`. The slower >>>>>> performance >>>>>> is still a bit sad, from having expected `External` to be the main >>>>>> public >>>>>> API meant to handle external pointers. >>>>>> >>>>>> -- >>>>>> -- >>>>>> v8-dev mailing list >>>>>> v8-...@googlegroups.com >>>>>> http://groups.google.com/group/v8-dev >>>>>> --- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "v8-dev" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to v8-dev+un...@googlegroups.com. >>>&
Re: [v8-dev] `void*` pointer support for fast calls
I presume Maya might now be back be at the office? Would it be possible to get some guidance regarding implementing void pointer support, either here on Groups or possibly by organizing an online meeting of some sort? -Aapo On Tuesday, 23 August 2022 at 11:32:30 UTC+3 cbr...@chromium.org wrote: > Hi, yes, Maya is out until mid-september. > Cheers, Camillo > > On Tue, 23 Aug 2022 at 07:07, Aapo Alasuutari > wrote: > >> Has Maya possibly returned from vacation? Or is their leave still >> continuing? >> >> On Friday, 29 July 2022 at 12:08:53 UTC+3 ah...@google.com wrote: >> >>> Maya is on leave over the summer, unfortunately. >>> >>> On Fri, Jul 29, 2022 at 11:02 AM Leszek Swirski >>> wrote: >>> >>>> +Maya, you're probably the best person to answer this. >>>> >>>> On Tue, Jul 26, 2022 at 9:05 PM Aapo Alasuutari >>>> wrote: >>>> >>>>> Hello, >>>>> >>>>> I'm interested in implementing `void*` pointer support for Fast API >>>>> calls. My thinking was that V8's `External` objects are appropriate to >>>>> stand in for external `void*` pointers coming in from external code and >>>>> going back out, since that's what they're (presumably) meant for. >>>>> >>>>> Unfortunately this seems to be a complex endeavour, a bit more than I >>>>> can start hacking together directly. I'm also not sure if the `Sandboxify >>>>> JSExternalObject external pointer` PR will complicate this plan of mine. >>>>> >>>>> The origin of my interest is Deno FFI support, that is calling native >>>>> libraries from Deno JS runtime that uses the V8 engine. Recent changes to >>>>> the FFI have added V8 Fast API support and made the FFI a lot faster, but >>>>> unfortunately we're bound to using plain numbers as pointers, meaning >>>>> both >>>>> that creating pointers is as easy as just writing a number and that (Fast >>>>> API compatible) pointers are limited to 53 bit numbers which will not be >>>>> enough for eg. pointer cryptography on ARM v8.3. >>>>> >>>>> It believe it would be preferable if Deno could use `External` objects >>>>> to stand for pointers but this would negate the current Fast API >>>>> performance benefits. Thus, `void*` pointer support for fast calls. >>>>> >>>>> >>>>> Any comments? Suggestions on how I might best proceed with this to >>>>> implement it? Or is this perhaps not a reasonable idea? >>>>> >>>>> Side note: I was sad to find that getting the pointer value out of an >>>>> `Local` is measurably slower than getting the pointer number >>>>> value out of a `Local`. This is presumably due to the `External` >>>>> internally saving the pointer in the `ExternalMap`. The slower >>>>> performance >>>>> is still a bit sad, from having expected `External` to be the main public >>>>> API meant to handle external pointers. >>>>> >>>>> -- >>>>> -- >>>>> v8-dev mailing list >>>>> v8-...@googlegroups.com >>>>> http://groups.google.com/group/v8-dev >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "v8-dev" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to v8-dev+un...@googlegroups.com. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/v8-dev/a491-88bf-4238-828c-9ec3f2e09878n%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/v8-dev/a491-88bf-4238-828c-9ec3f2e09878n%40googlegroups.com?utm_medium=email_source=footer> >>>>> . >>>>> >>>> -- >>>> -- >>>> v8-dev mailing list >>>> v8-...@googlegroups.com >>>> http://groups.google.com/group/v8-dev >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "v8-dev" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to v8-dev+un...@googlegroups.com. >>>> >>> To view this discussion on the web visit >>>> https://groups.g
Re: [v8-dev] `void*` pointer support for fast calls
Has Maya possibly returned from vacation? Or is their leave still continuing? On Friday, 29 July 2022 at 12:08:53 UTC+3 ah...@google.com wrote: > Maya is on leave over the summer, unfortunately. > > On Fri, Jul 29, 2022 at 11:02 AM Leszek Swirski > wrote: > >> +Maya, you're probably the best person to answer this. >> >> On Tue, Jul 26, 2022 at 9:05 PM Aapo Alasuutari >> wrote: >> >>> Hello, >>> >>> I'm interested in implementing `void*` pointer support for Fast API >>> calls. My thinking was that V8's `External` objects are appropriate to >>> stand in for external `void*` pointers coming in from external code and >>> going back out, since that's what they're (presumably) meant for. >>> >>> Unfortunately this seems to be a complex endeavour, a bit more than I >>> can start hacking together directly. I'm also not sure if the `Sandboxify >>> JSExternalObject external pointer` PR will complicate this plan of mine. >>> >>> The origin of my interest is Deno FFI support, that is calling native >>> libraries from Deno JS runtime that uses the V8 engine. Recent changes to >>> the FFI have added V8 Fast API support and made the FFI a lot faster, but >>> unfortunately we're bound to using plain numbers as pointers, meaning both >>> that creating pointers is as easy as just writing a number and that (Fast >>> API compatible) pointers are limited to 53 bit numbers which will not be >>> enough for eg. pointer cryptography on ARM v8.3. >>> >>> It believe it would be preferable if Deno could use `External` objects >>> to stand for pointers but this would negate the current Fast API >>> performance benefits. Thus, `void*` pointer support for fast calls. >>> >>> >>> Any comments? Suggestions on how I might best proceed with this to >>> implement it? Or is this perhaps not a reasonable idea? >>> >>> Side note: I was sad to find that getting the pointer value out of an >>> `Local` is measurably slower than getting the pointer number >>> value out of a `Local`. This is presumably due to the `External` >>> internally saving the pointer in the `ExternalMap`. The slower performance >>> is still a bit sad, from having expected `External` to be the main public >>> API meant to handle external pointers. >>> >>> -- >>> -- >>> v8-dev mailing list >>> v8-...@googlegroups.com >>> http://groups.google.com/group/v8-dev >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "v8-dev" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to v8-dev+un...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/v8-dev/a491-88bf-4238-828c-9ec3f2e09878n%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/v8-dev/a491-88bf-4238-828c-9ec3f2e09878n%40googlegroups.com?utm_medium=email_source=footer> >>> . >>> >> -- >> -- >> v8-dev mailing list >> v8-...@googlegroups.com >> http://groups.google.com/group/v8-dev >> --- >> You received this message because you are subscribed to the Google Groups >> "v8-dev" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to v8-dev+un...@googlegroups.com. >> > To view this discussion on the web visit >> https://groups.google.com/d/msgid/v8-dev/CAGRskv_o%3DdZTXdYAceSM%2BdaabpJKFYZwEFMjvzS3_8jy3e0TuQ%40mail.gmail.com >> >> <https://groups.google.com/d/msgid/v8-dev/CAGRskv_o%3DdZTXdYAceSM%2BdaabpJKFYZwEFMjvzS3_8jy3e0TuQ%40mail.gmail.com?utm_medium=email_source=footer> >> . >> > -- -- v8-dev mailing list v8-dev@googlegroups.com http://groups.google.com/group/v8-dev --- You received this message because you are subscribed to the Google Groups "v8-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to v8-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/v8-dev/17c3b560-e88d-41a7-b64d-d792b4021613n%40googlegroups.com.
[v8-dev] `void*` pointer support for fast calls
Hello, I'm interested in implementing `void*` pointer support for Fast API calls. My thinking was that V8's `External` objects are appropriate to stand in for external `void*` pointers coming in from external code and going back out, since that's what they're (presumably) meant for. Unfortunately this seems to be a complex endeavour, a bit more than I can start hacking together directly. I'm also not sure if the `Sandboxify JSExternalObject external pointer` PR will complicate this plan of mine. The origin of my interest is Deno FFI support, that is calling native libraries from Deno JS runtime that uses the V8 engine. Recent changes to the FFI have added V8 Fast API support and made the FFI a lot faster, but unfortunately we're bound to using plain numbers as pointers, meaning both that creating pointers is as easy as just writing a number and that (Fast API compatible) pointers are limited to 53 bit numbers which will not be enough for eg. pointer cryptography on ARM v8.3. It believe it would be preferable if Deno could use `External` objects to stand for pointers but this would negate the current Fast API performance benefits. Thus, `void*` pointer support for fast calls. Any comments? Suggestions on how I might best proceed with this to implement it? Or is this perhaps not a reasonable idea? Side note: I was sad to find that getting the pointer value out of an `Local` is measurably slower than getting the pointer number value out of a `Local`. This is presumably due to the `External` internally saving the pointer in the `ExternalMap`. The slower performance is still a bit sad, from having expected `External` to be the main public API meant to handle external pointers. -- -- v8-dev mailing list v8-dev@googlegroups.com http://groups.google.com/group/v8-dev --- You received this message because you are subscribed to the Google Groups "v8-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to v8-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/v8-dev/a491-88bf-4238-828c-9ec3f2e09878n%40googlegroups.com.
