Re: [v8-users] Can "threading" be implemented by mapping v8 heap to shared memory in linux?

[J Decker]

On Tue, Jan 9, 2018 at 8:30 AM,  wrote:

> There is active work on parallel JS with a range of approaches to shared
> memory parallelism.  I maintain a list of parallel JS projects
>  and
> am interested in finding applications that use them.  Are you working with
> a particular application that lead to a requirement for shared-memory
> parallel JS?
>
>
A voxel based 3d engine.  Offloading rebuilding meshes to other threads is
of great benefit; allowing much more complex (smoothed) meshing.  The
meshes, once available, should be immediately available to use in the
render thread.  A single sector 32x32x32 can be a meg as a mesh; the
visible area is 125 sectors.
Also physics for things moving across the voxels; which updates back for
the few hundred entities' positions and orientations could be message
based... not requiring shared memory.
https://github.com/d3x0r/voxelarium.js  (still very much a work in progress
and non-stable; and is only a hobby-time project)


>   -J
>
>
> On Monday, January 8, 2018 at 7:30:33 AM UTC-8, Bogdan wrote:
>>
>> I understand that updating object in shared memory need synchronization
>> and even with in-memory database task in general there will be a global
>> lock (not per-object because transaction will spread by different tables
>> and object) and only one process will write to memory so I think just
>> global lock will be enough for most tasks. But reading from shared memory
>> is safe and there is no reason to not use all cpu cores to achieve more
>> performance and that's why go and other languages will overcome js in this
>> tasks. But I hope js will be able to compete and new features like
>> SharedArrayBuffer and atomics seems like moving forward javascript to the
>> right direction and I don't see why "JavaScript is not designed for such
>> purposes.". Does SharedArrayBuffer supposed to do zero copy reading in
>> different workers? How it implemented - by multiple threads or maybe it
>> already using shared memory and different process?
>>
>> On Mon, Jan 8, 2018 at 6:04 PM, Jakob Kummerow 
>> wrote:
>>
>>> No, you cannot simply share all memory to get multi-threading. For
>>> safely/correctly working with shared memory, you need
>>> locking/synchronization primitives as well as certain guarantees in the
>>> language's memory model. JavaScript is not designed for such purposes.
>>>
>>>
>>> On Mon, Jan 8, 2018 at 3:52 PM Bogdan  wrote:
>>>
 I don't know all details, I just a regular node developer who came up
 with idea of "threading" in js and hope anybody with deep knowledge will
 clarify some things. Javascript have old problem of lacking threading and
 all advices of using multiple processes doesn't help because there are
 important tasks  (for example in-memory database or managing application
 cache) which needs shared memory to utilize all cpu cores and having full
 copy of that memory and synchronizing it by coping data over channels
 doesn't make sense. Recently I've found out an interesting linux feature
 like shader memory with shm_open and mmap which can allow zero-copy and
 zero-overhead access to shared memory from different processes. There are
 also some node modules which exposes this feature to javascript but seems
 like all they can do is exposing  shared memory to javascript as an array
 which means all object managing, garbage collector, and other useful
 features we need to do all by ourselves. So I am wondering can v8 (if
 cannot now, maybe in it will with little hacking) allow to just map all
 heap into shared buffer so we can use the same objects from another process
 without copying?

 --
 --
 v8-users mailing list
 v8-u...@googlegroups.com
 http://groups.google.com/group/v8-users
 ---
 You received this message because you are subscribed to the Google
 Groups "v8-users" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to v8-users+u...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

>>> --
>>> --
>>> v8-users mailing list
>>> v8-u...@googlegroups.com
>>> http://groups.google.com/group/v8-users
>>> ---
>>> You received this message because you are subscribed to a topic in the
>>> Google Groups "v8-users" group.
>>> To unsubscribe from this topic, visit https://groups.google.com/d/to
>>> pic/v8-users/2Z2UFFQrziE/unsubscribe.
>>> To unsubscribe from this group and all its topics, send an email to
>>> v8-users+u...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> --
> --
> v8-users mailing list
> v8-users@googlegroups.com
> http://groups.google.com/group/v8-users
> ---
> You received this message because you are subscribed to the Google Groups
> "v8-users" group.
> To unsubscribe from this group and stop receiving emails from it, sen

Re: [v8-users] Can "threading" be implemented by mapping v8 heap to shared memory in linux?

[mogill]

There is active work on parallel JS with a range of approaches to shared 
memory parallelism.  I maintain a list of parallel JS projects 
 and 
am interested in finding applications that use them.  Are you working with 
a particular application that lead to a requirement for shared-memory 
parallel JS?

  -J


On Monday, January 8, 2018 at 7:30:33 AM UTC-8, Bogdan wrote:
>
> I understand that updating object in shared memory need synchronization 
> and even with in-memory database task in general there will be a global 
> lock (not per-object because transaction will spread by different tables 
> and object) and only one process will write to memory so I think just 
> global lock will be enough for most tasks. But reading from shared memory 
> is safe and there is no reason to not use all cpu cores to achieve more 
> performance and that's why go and other languages will overcome js in this 
> tasks. But I hope js will be able to compete and new features like 
> SharedArrayBuffer and atomics seems like moving forward javascript to the 
> right direction and I don't see why "JavaScript is not designed for such 
> purposes.". Does SharedArrayBuffer supposed to do zero copy reading in 
> different workers? How it implemented - by multiple threads or maybe it 
> already using shared memory and different process?
>
> On Mon, Jan 8, 2018 at 6:04 PM, Jakob Kummerow  > wrote:
>
>> No, you cannot simply share all memory to get multi-threading. For 
>> safely/correctly working with shared memory, you need 
>> locking/synchronization primitives as well as certain guarantees in the 
>> language's memory model. JavaScript is not designed for such purposes.
>>
>>
>> On Mon, Jan 8, 2018 at 3:52 PM Bogdan > 
>> wrote:
>>
>>> I don't know all details, I just a regular node developer who came up 
>>> with idea of "threading" in js and hope anybody with deep knowledge will 
>>> clarify some things. Javascript have old problem of lacking threading and 
>>> all advices of using multiple processes doesn't help because there are 
>>> important tasks  (for example in-memory database or managing application 
>>> cache) which needs shared memory to utilize all cpu cores and having full 
>>> copy of that memory and synchronizing it by coping data over channels 
>>> doesn't make sense. Recently I've found out an interesting linux feature 
>>> like shader memory with shm_open and mmap which can allow zero-copy and 
>>> zero-overhead access to shared memory from different processes. There are 
>>> also some node modules which exposes this feature to javascript but seems 
>>> like all they can do is exposing  shared memory to javascript as an array 
>>> which means all object managing, garbage collector, and other useful 
>>> features we need to do all by ourselves. So I am wondering can v8 (if 
>>> cannot now, maybe in it will with little hacking) allow to just map all 
>>> heap into shared buffer so we can use the same objects from another process 
>>> without copying?
>>>
>>> -- 
>>> -- 
>>> v8-users mailing list
>>> v8-u...@googlegroups.com 
>>> http://groups.google.com/group/v8-users
>>> --- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "v8-users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to v8-users+u...@googlegroups.com .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> -- 
>> -- 
>> v8-users mailing list
>> v8-u...@googlegroups.com 
>> http://groups.google.com/group/v8-users
>> --- 
>> You received this message because you are subscribed to a topic in the 
>> Google Groups "v8-users" group.
>> To unsubscribe from this topic, visit 
>> https://groups.google.com/d/topic/v8-users/2Z2UFFQrziE/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to 
>> v8-users+u...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
-- 
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
--- 
You received this message because you are subscribed to the Google Groups 
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [v8-users] Can "threading" be implemented by mapping v8 heap to shared memory in linux?

[J Decker]

nwjs claims to support web worker with node, allowing true threading.  Each
thread gets its own javascript context and they don't share anything.  They
only communicate with messages.  The workers are entirely separate from
each other.

https://github.com/nwjs/nw.js/issues/494
https://groups.google.com/forum/#!msg/nwjs-general/jljGxjIyxK4/pVqyidTrBAAJ

pretty sure you'll need an addon if you want to share large chunks of data
using typed arrays... and/or provide other synchronization around the
buffers.


On Mon, Jan 8, 2018 at 7:30 AM, Bogdan Orlov  wrote:

> I understand that updating object in shared memory need synchronization
> and even with in-memory database task in general there will be a global
> lock (not per-object because transaction will spread by different tables
> and object) and only one process will write to memory so I think just
> global lock will be enough for most tasks. But reading from shared memory
> is safe and there is no reason to not use all cpu cores to achieve more
> performance and that's why go and other languages will overcome js in this
> tasks. But I hope js will be able to compete and new features like
> SharedArrayBuffer and atomics seems like moving forward javascript to the
> right direction and I don't see why "JavaScript is not designed for such
> purposes.". Does SharedArrayBuffer supposed to do zero copy reading in
> different workers? How it implemented - by multiple threads or maybe it
> already using shared memory and different process?
>
> On Mon, Jan 8, 2018 at 6:04 PM, Jakob Kummerow 
> wrote:
>
>> No, you cannot simply share all memory to get multi-threading. For
>> safely/correctly working with shared memory, you need
>> locking/synchronization primitives as well as certain guarantees in the
>> language's memory model. JavaScript is not designed for such purposes.
>>
>
>>
>> On Mon, Jan 8, 2018 at 3:52 PM Bogdan  wrote:
>>
>>> I don't know all details, I just a regular node developer who came up
>>> with idea of "threading" in js and hope anybody with deep knowledge will
>>> clarify some things. Javascript have old problem of lacking threading and
>>> all advices of using multiple processes doesn't help because there are
>>> important tasks  (for example in-memory database or managing application
>>> cache) which needs shared memory to utilize all cpu cores and having full
>>> copy of that memory and synchronizing it by coping data over channels
>>> doesn't make sense. Recently I've found out an interesting linux feature
>>> like shader memory with shm_open and mmap which can allow zero-copy and
>>> zero-overhead access to shared memory from different processes. There are
>>> also some node modules which exposes this feature to javascript but seems
>>> like all they can do is exposing  shared memory to javascript as an array
>>> which means all object managing, garbage collector, and other useful
>>> features we need to do all by ourselves. So I am wondering can v8 (if
>>> cannot now, maybe in it will with little hacking) allow to just map all
>>> heap into shared buffer so we can use the same objects from another process
>>> without copying?
>>>
>>> --
>>> --
>>> v8-users mailing list
>>> v8-users@googlegroups.com
>>> http://groups.google.com/group/v8-users
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "v8-users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to v8-users+unsubscr...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> --
>> --
>> v8-users mailing list
>> v8-users@googlegroups.com
>> http://groups.google.com/group/v8-users
>> ---
>> You received this message because you are subscribed to a topic in the
>> Google Groups "v8-users" group.
>> To unsubscribe from this topic, visit https://groups.google.com/d/to
>> pic/v8-users/2Z2UFFQrziE/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> v8-users+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> --
> v8-users mailing list
> v8-users@googlegroups.com
> http://groups.google.com/group/v8-users
> ---
> You received this message because you are subscribed to the Google Groups
> "v8-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to v8-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
-- 
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
--- 
You received this message because you are subscribed to the Google Groups 
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [v8-users] Can "threading" be implemented by mapping v8 heap to shared memory in linux?

[Bogdan Orlov]

I understand that updating object in shared memory need synchronization and
even with in-memory database task in general there will be a global lock
(not per-object because transaction will spread by different tables and
object) and only one process will write to memory so I think just global
lock will be enough for most tasks. But reading from shared memory is safe
and there is no reason to not use all cpu cores to achieve more performance
and that's why go and other languages will overcome js in this tasks. But I
hope js will be able to compete and new features like SharedArrayBuffer and
atomics seems like moving forward javascript to the right direction and I
don't see why "JavaScript is not designed for such purposes.". Does
SharedArrayBuffer supposed to do zero copy reading in different workers?
How it implemented - by multiple threads or maybe it already using shared
memory and different process?

On Mon, Jan 8, 2018 at 6:04 PM, Jakob Kummerow 
wrote:

> No, you cannot simply share all memory to get multi-threading. For
> safely/correctly working with shared memory, you need
> locking/synchronization primitives as well as certain guarantees in the
> language's memory model. JavaScript is not designed for such purposes.
>
>
> On Mon, Jan 8, 2018 at 3:52 PM Bogdan  wrote:
>
>> I don't know all details, I just a regular node developer who came up
>> with idea of "threading" in js and hope anybody with deep knowledge will
>> clarify some things. Javascript have old problem of lacking threading and
>> all advices of using multiple processes doesn't help because there are
>> important tasks  (for example in-memory database or managing application
>> cache) which needs shared memory to utilize all cpu cores and having full
>> copy of that memory and synchronizing it by coping data over channels
>> doesn't make sense. Recently I've found out an interesting linux feature
>> like shader memory with shm_open and mmap which can allow zero-copy and
>> zero-overhead access to shared memory from different processes. There are
>> also some node modules which exposes this feature to javascript but seems
>> like all they can do is exposing  shared memory to javascript as an array
>> which means all object managing, garbage collector, and other useful
>> features we need to do all by ourselves. So I am wondering can v8 (if
>> cannot now, maybe in it will with little hacking) allow to just map all
>> heap into shared buffer so we can use the same objects from another process
>> without copying?
>>
>> --
>> --
>> v8-users mailing list
>> v8-users@googlegroups.com
>> http://groups.google.com/group/v8-users
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "v8-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to v8-users+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
> --
> v8-users mailing list
> v8-users@googlegroups.com
> http://groups.google.com/group/v8-users
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "v8-users" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/v8-users/2Z2UFFQrziE/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> v8-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
-- 
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
--- 
You received this message because you are subscribed to the Google Groups 
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [v8-users] Can "threading" be implemented by mapping v8 heap to shared memory in linux?

[Jakob Kummerow]

No, you cannot simply share all memory to get multi-threading. For
safely/correctly working with shared memory, you need
locking/synchronization primitives as well as certain guarantees in the
language's memory model. JavaScript is not designed for such purposes.


On Mon, Jan 8, 2018 at 3:52 PM Bogdan  wrote:

> I don't know all details, I just a regular node developer who came up with
> idea of "threading" in js and hope anybody with deep knowledge will clarify
> some things. Javascript have old problem of lacking threading and all
> advices of using multiple processes doesn't help because there are
> important tasks  (for example in-memory database or managing application
> cache) which needs shared memory to utilize all cpu cores and having full
> copy of that memory and synchronizing it by coping data over channels
> doesn't make sense. Recently I've found out an interesting linux feature
> like shader memory with shm_open and mmap which can allow zero-copy and
> zero-overhead access to shared memory from different processes. There are
> also some node modules which exposes this feature to javascript but seems
> like all they can do is exposing  shared memory to javascript as an array
> which means all object managing, garbage collector, and other useful
> features we need to do all by ourselves. So I am wondering can v8 (if
> cannot now, maybe in it will with little hacking) allow to just map all
> heap into shared buffer so we can use the same objects from another process
> without copying?
>
> --
> --
> v8-users mailing list
> v8-users@googlegroups.com
> http://groups.google.com/group/v8-users
> ---
> You received this message because you are subscribed to the Google Groups
> "v8-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to v8-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
-- 
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
--- 
You received this message because you are subscribed to the Google Groups 
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[v8-users] Can "threading" be implemented by mapping v8 heap to shared memory in linux?

[Bogdan]

I don't know all details, I just a regular node developer who came up with 
idea of "threading" in js and hope anybody with deep knowledge will clarify 
some things. Javascript have old problem of lacking threading and all 
advices of using multiple processes doesn't help because there are 
important tasks  (for example in-memory database or managing application 
cache) which needs shared memory to utilize all cpu cores and having full 
copy of that memory and synchronizing it by coping data over channels 
doesn't make sense. Recently I've found out an interesting linux feature 
like shader memory with shm_open and mmap which can allow zero-copy and 
zero-overhead access to shared memory from different processes. There are 
also some node modules which exposes this feature to javascript but seems 
like all they can do is exposing  shared memory to javascript as an array 
which means all object managing, garbage collector, and other useful 
features we need to do all by ourselves. So I am wondering can v8 (if 
cannot now, maybe in it will with little hacking) allow to just map all 
heap into shared buffer so we can use the same objects from another process 
without copying?

-- 
-- 
v8-users mailing list
v8-users@googlegroups.com
http://groups.google.com/group/v8-users
--- 
You received this message because you are subscribed to the Google Groups 
"v8-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to v8-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.