Re: [v8-users] Is running a v8 instance built as a static library supported in the same process that loaded a different v8 instance in a DLL?

[A.M.]

A quick update for those who are interested in running a v8-based 
multi-threaded application in a Node process, but completely isolated from 
the Node. 

I built static v8 libraries using a GN build. This is release arguments 
file I used (`args.gn`):

is_debug = false
target_cpu = "x64"
v8_static_library = true
is_component_build = false


The build failed close to the end with this error message:

> ninja -C out.gn/x64.release
ninja: Entering directory `out.gn/x64.release'
[1/1] Regenerating ninja files
[1389/1543] LINK cctest.exe cctest.exe.pdb
FAILED: cctest.exe cctest.exe.pdb
...\depot_tools/win_tools-2_7_6_bin/python/bin/python.exe 
../../build/toolchain/win/tool_wrapper.py link-wrapper environment.x64 
False link.exe /nologo /OUT:./cctest.exe /PDB:./cctest.exe.pdb 
@./cctest.exe.rsp
LINK : warning LNK4001: no object files specified; libraries used
LINK : error LNK1218: warning treated as error; no output file generated
[1394/1543] CXX 
obj/test/unittests/unittests_sources/unoptimized-compile-job-unittest.obj
ninja: build stopped: subcommand failed.

, but it did manage to produce all static v8 libraries to link against. 
This is the resulting list of libraries in `out.gn/x64.release/obj` and 
some subdirectories:


icui18n.lib
icuuc.lib
v8_base_0.lib
v8_base_1.lib
v8_external_snapshot.lib
v8_init.lib
v8_initializers.lib
v8_libbase.lib
v8_libplatform.lib
v8_libsampler.lib
v8_nosnapshot.lib
v8_snapshot.lib


As with a non-static v8 build, you will need `snapshot_blob.bin`, 
`natives_blob.bin` and `icudtl.dat` in the directory where the 
initialization can find them. 

My test setup was to run Node v6.9.1, which called a C++ modules every 50ms 
and created a couple of objects (a string and a function) and a couple of 
threads making a ton of v8 calls against multiple isolates from a static v8 
library v6.6, which used most commonly-used v8 functionality, including 
first and second level weak pointer callbacks.









-- 
-- 
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
--- 
You received this message because you are subscribed to the Google Groups 
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [v8-users] Is running a v8 instance built as a static library supported in the same process that loaded a different v8 instance in a DLL?

[A.M.]

 > For example, on 64-bit platforms, V8 requires all executable memory to 
be within a 2GB section of address space so that calls can use 32-bit 
offsets, so it reserves that amount of address space on initialization 
(which is a very cheap operation), whereas actual memory is allocated later 
as needed.

I think we are talking about the same thing - reserved vs. committed memory 
and I was suggesting that if v8 knows its memory limits upfront, I thought 
it could reserve memory up to the upper limit it may need in the future. As 
I'm writing this, however, I realized that memory limits are set 
per-isolate and any number of isolates can be allocated at run time, which 
would explain a large chunk of memory v8 reserves upfront. Makes sense. 
Thanks for clarifying. 

Ben, Jakob, thanks a lot for the feedback. It is much appreciated. I will 
post an update here after I work with this configuration a bit. 

-- 
-- 
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
--- 
You received this message because you are subscribed to the Google Groups 
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [v8-users] Is running a v8 instance built as a static library supported in the same process that loaded a different v8 instance in a DLL?

[A.M.]

> For example, on 64-bit platforms, V8 requires all executable memory to be 
within a 2GB section of address space so that calls can use 32-bit offsets, 
so it reserves that amount of address space on initialization (which is a 
very cheap operation), whereas actual memory is allocated later as needed.

I think we are talking about the same thing - reserved vs. committed memory 
and I was suggesting that if v8 knows its memory limits upfront, I thought 
it could reserve memory up to the upper limit it may need in the future. As 
I'm writing this, however, I realized that memory limits are set 
per-isolate and any number of isolates can be allocated at run time, which 
would explain a large chunk of memory v8 reserves upfront. Makes sense. 
Thanks for clarifying. 

Ben, Jakon, thanks a lot for the feedback. It is much appreciated. I will 
post an update here after I work with this configuration a bit. 

-- 
-- 
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
--- 
You received this message because you are subscribed to the Google Groups 
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [v8-users] Is running a v8 instance built as a static library supported in the same process that loaded a different v8 instance in a DLL?

[Jakob Kummerow]

>
> > What Ben pointed out was virtual memory reservations, which is not the
> same as restricted space sizes.
>
> Would you guys mind expanding on this? My thinking is that the v8 commits
> as much memory as it is allowed for semi/old/code spaces (considering that
> the app doesn't allocate more than fits into these spaces), plus for the
> C++ machinery to maintain these objects and if those sizes are limited to
> some reasonable size, why would v8 reserve more memory that it cannot
> commit?
>

Memory address space is separate from actual memory. There are cases where
it makes sense to reserve a large chunk of address space without requesting
it to be backed by actual memory, usually when a contiguous range may be
required later.
For example, on 64-bit platforms, V8 requires all executable memory to be
within a 2GB section of address space so that calls can use 32-bit offsets,
so it reserves that amount of address space on initialization (which is a
very cheap operation), whereas actual memory is allocated later as needed.
WebAssembly has a similar use case where a large section of address space
is reserved so that *if* it needs to use that memory later, the addresses
will be contiguous.

-- 
-- 
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
--- 
You received this message because you are subscribed to the Google Groups 
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [v8-users] Is running a v8 instance built as a static library supported in the same process that loaded a different v8 instance in a DLL?

[A.M.]

 Great. Thank you for jumping in, Jakob. Considering that both, you and Ben 
didn't reject this approach with some obvious reason, like some resources 
created with a process ID in the name, I can go ahead and try things out 
with a static v8 build. Without the input from the both of you, a simple 
working test wouldn't be worth much.

> What Ben pointed out was virtual memory reservations, which is not the 
same as restricted space sizes.

Would you guys mind expanding on this? My thinking is that the v8 commits 
as much memory as it is allowed for semi/old/code spaces (considering that 
the app doesn't allocate more than fits into these spaces), plus for the 
C++ machinery to maintain these objects and if those sizes are limited to 
some reasonable size, why would v8 reserve more memory that it cannot 
commit?

-- 
-- 
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
--- 
You received this message because you are subscribed to the Google Groups 
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [v8-users] Is running a v8 instance built as a static library supported in the same process that loaded a different v8 instance in a DLL?

[A.M.]

Great. Thank you for jumping in, Jakob. Considering that both, you and Ben 
didn't reject this approach with some obvious reason, like some resources 
created with a process ID in the name, I can go ahead and try things out 
with a static v8 build. Without the input from the both of you, a simple 
working test wouldn't worth much.

> What Ben pointed out was virtual memory reservations, which is not the 
same as restricted space sizes.

Would you guys mind expanding on this? My thinking is that the v8 commits 
as much memory as it is allowed for semi/old/code spaces (considering that 
the app doesn't allocate more than fits into these spaces), plus for the 
C++ machinery to maintain these objects and if those sizes are limited to 
some reasonable size, why would v8 reserve more memory that it cannot 
commit?

-- 
-- 
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
--- 
You received this message because you are subscribed to the Google Groups 
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [v8-users] Is running a v8 instance built as a static library supported in the same process that loaded a different v8 instance in a DLL?

[Jakob Kummerow]

I'm on the V8 team, and I defer to Ben on this question :-)

I've never tried embedding two different versions of V8 into the same
process, and I don't think we'd consider that scenario supported, but it
may well work. You can probably find out with a fairly small example (i.e.
without sinking lots of time into it) whether it works.

With regards to memory, I implemented first/second chance weak handle
> callbacks and tested it with low memory limits for semi/old space sizes and
> it appears to deal with memory restrictions well.


What Ben pointed out was virtual memory reservations, which is not the same
as restricted space sizes. As he said, on 64-bit platforms it shouldn't be
a problem.

-- 
-- 
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
--- 
You received this message because you are subscribed to the Google Groups 
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [v8-users] Is running a v8 instance built as a static library supported in the same process that loaded a different v8 instance in a DLL?

[A.M.]

Thanks Ben. Your feedback is much appreciated. I realize you are not on the 
v8 team. Just thought may be I'm missing some MSDN-like way to contact 
Google.

-- 
-- 
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
--- 
You received this message because you are subscribed to the Google Groups 
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [v8-users] Is running a v8 instance built as a static library supported in the same process that loaded a different v8 instance in a DLL?

[Ben Noordhuis]

On Tue, Apr 17, 2018 at 4:31 PM, A.M.  wrote:
> Thank you, Ben. That is reassuring. With regards to memory, I implemented
> first/second chance weak handle callbacks and tested it with low memory
> limits for semi/old space sizes and it appears to deal with memory
> restrictions well.
>
> The project I'm working on is rather large and it would be really helpful if
> there was a way to get some feedback from the v8 team. Do you know if there
> is a way to get in touch with somebody from the v8 team?

I'm not on the V8 team so I can't really say but I expect
"self-support" sums it up.  If you have questions, this mailing list
is probably the best place to ask them.

-- 
-- 
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
--- 
You received this message because you are subscribed to the Google Groups 
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [v8-users] Is running a v8 instance built as a static library supported in the same process that loaded a different v8 instance in a DLL?

[A.M.]

Thank you, Ben. That is reassuring. With regards to memory, I implemented 
first/second chance weak handle callbacks and tested it with low memory 
limits for semi/old space sizes and it appears to deal with memory 
restrictions well. 

The project I'm working on is rather large and it would be really helpful 
if there was a way to get some feedback from the v8 team. Do you know if 
there is a way to get in touch with somebody from the v8 team? 

-- 
-- 
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
--- 
You received this message because you are subscribed to the Google Groups 
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [v8-users] Is running a v8 instance built as a static library supported in the same process that loaded a different v8 instance in a DLL?

[Ben Noordhuis]

On Tue, Apr 17, 2018 at 12:27 AM, A.M.  wrote:
> I have an application that uses the v8 to run concurrent script jobs using
> multiple v8 isolates. There is a desire to move this application to Node.js,
> which puts the v8 used by the application in conflict with how Node.js uses
> its instance of the v8 (e.g. the v8 platform isn't exposed outside Node.js,
> etc).
>
> So, the question is, if I build the v8 as a static library and link it into
> the application, so it has its own globals, etc, will it still conflict with
> the instance of the v8 loaded by Node.js? I am thinking of any Windows
> global atom names, any names used within the v8 derived from the process ID
> (e.g. named mutexes, shared memory, etc) and so on.
>
> Application threads making calls into the static instance of the v8 will
> never cross into the Node.js v8 instance and vice versa, so if there is any
> thread local storage used allocated in any of the v8 calls, it should not
> cause any mix-up with TLS values. Likewise, any handles allocated in one v8
> instance will never make it into calls against another v8 instance. From the
> programmatic point of view both v8 instances are completely separated. I'm
> only worried about is any system resources that can be accessed via some OS
> mechanisms, like those mentioned above.
>
> Any thoughts on this would be greatly appreciated. Thanks.

Interesting question!

V8 doesn't use extra-process resources like named shm segments so I
don't see why it wouldn't work.  Not quite the same thing as
"officially supported" but there are no technical blockers that I can
think of.

Perhaps one caveat to keep in mind: V8 is pretty greedy when it comes
to reserving virtual memory.  It won't be a problem on 64 bits
platforms but on 32 bits you might run out of address space with two
V8 instances.

-- 
-- 
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
--- 
You received this message because you are subscribed to the Google Groups 
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[v8-users] Is running a v8 instance built as a static library supported in the same process that loaded a different v8 instance in a DLL?

[A.M.]

I have an application that uses the v8 to run concurrent script jobs using 
multiple v8 isolates. There is a desire to move this application to 
Node.js, which puts the v8 used by the application in conflict with how 
Node.js uses its instance of the v8 (e.g. the v8 platform isn't exposed 
outside Node.js, etc).

So, the question is, if I build the v8 as a static library and link it into 
the application, so it has its own globals, etc, will it still conflict 
with the instance of the v8 loaded by Node.js? I am thinking of any Windows 
global atom names, any names used within the v8 derived from the process ID 
(e.g. named mutexes, shared memory, etc) and so on. 

Application threads making calls into the static instance of the v8 will 
never cross into the Node.js v8 instance and vice versa, so if there is any 
thread local storage used allocated in any of the v8 calls, it should not 
cause any mix-up with TLS values. Likewise, any handles allocated in one v8 
instance will never make it into calls against another v8 instance. From 
the programmatic point of view both v8 instances are completely separated. 
I'm only worried about is any system resources that can be accessed via 
some OS mechanisms, like those mentioned above.

Any thoughts on this would be greatly appreciated. Thanks.


-- 
-- 
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
--- 
You received this message because you are subscribed to the Google Groups 
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.