Thanks! The SharedArrayBuffer and pthreads work is a carried out separately from the webgl in worker effort, by mostly different people, but they are both part of a well coordinated simultaneous strategy, where the goal is to empower the workers so that they are useful for a wide range of serious tasks. In Emscripten we are looking to eventually improve the proxy to worker feature by needing fewer items proxied, and researching the feasibility of synchronous blocking loops. That would get much closer to a programming model parity with native, and therefore would enable easier porting of native code. However there are still a lot of unknowns, but we will keep you posted how this progresses. On Jun 7, 2015 8:24 AM, "chronotext" <[email protected]> wrote:
> Hey there, > > Congrats for the hard-work. > > I'm trying to assemble the parts of the puzzle > <https://github.com/kripken/worker-ui/issues/5>, and I was wondering: > > Is the Emscripten-pthread + mozilla-SharedArrayBuffer effort correlated to > the ongoing mozilla-WorkerCanvas effort > <https://bugzilla.mozilla.org/show_bug.cgi?id=709490>? > > Thanks, > Ariel > > On Thursday, June 4, 2015 at 7:21:31 PM UTC+3, jj wrote: >> >> Hello all, >> >> we've got some really cool news to share! Earlier this week, the pthreads >> pull request was merged in to Emscripten upstream, which adds support for >> the POSIX threads API to the Emscripten incoming branch. >> >> The main reason why Emscripten has not had threading support before, is >> that the current JavaScript Web Worker specification only allows the >> workers (== pthreads to Emscripten) to communicate by passing messages to >> each other. In native world, this corresponds to a multiprocess-like >> parallel environment, where processes do interprocess pipes to push data to >> each other, or like the MPI API that is used in big computing clusters that >> consist of multiple separate computers that send messages through the >> network. Web Workers have followed this approach, and they limit workers >> from accessing each other's memory directly. This simplifies parallel >> programming greatly, but it is a different paradigm that also limits the >> kind of parallel algorithms that one could implement. >> >> Native threading builds strictly on the assumption that threads can share >> memory with each other. This has been the hard and painful part for JS Web >> Workers from Emscripten native threading perspective, since shared memory >> has not been available. However for more than a year now, there have been >> experiments going on to imagine what such a direct memory sharing approach >> for JavaScript Web Workers would look like. This research process is a >> collaboration between multiple browser vendors, and you can follow that >> discussion here: >> https://docs.google.com/document/d/1NDGA_gZJ7M7w1Bh8S0AoDyEqwDdRh4uSoTPSNn77PFk/edit#heading=h.a6o4dubw5qla >> , where the SharedArrayBuffer specification is being drafted. Having direct >> shared memory enables more flexible options for parallelism compared to the >> more limited message passing approaches, and this flexibility benefits both >> native JS developers and compilers such as Emscripten. Alongside the >> SharedArrayBuffer research, we have been working to add support for >> pthreads to Emscripten that backs on to this research, and today we feel >> that we are at the point where it makes sense to push the feature to the >> incoming branch for everyone to test. >> >> It should be stressed that this feature is very experimental at the >> moment, and the only browser to currently implement support for it is >> Firefox Nightly. Being experimental means that there can be any number of >> changes made to the specification, which can mean that pages that you have >> compiled against it can stop working in Nightly in the future, if the spec >> and the implementation changes. We don't want to wait however until the >> standard is fixed, before bringing it to Emscripten, because your feedback >> will be extremely valuable in shaping the draft further, and that we can >> ensure that we get it right, and that the draft that will be proposed for >> adoption will be a good one that covers the important Emscripten needs. So, >> if you have some cycles to spare on experimenting with this feature to give >> us feedback, please do, so that we can make sure we don't miss anything! >> >> What the pthreads support for Emscripten means for developers in practice: >> >> - Default build mode is still fully singlethreaded. You should not see >> any pthreads-related code "leak in" to your non-pthreads builds ("you don't >> pay for what you don't use"). If you do see this, please submit a bug >> report! >> - Pass the -s USE_PTHREADS=1 compiler and linker flag to enable >> targeting pthreads. This enables the code to call the pthreads API, as well >> as use the GCC built-in lockfree atomics operations and the futex wait&wake >> API. See >> https://github.com/kripken/emscripten/blob/incoming/system/include/emscripten/threading.h >> for a list of function calls enabled when targeting pthreads. >> - When building with pthreads enabled, the asm.js HEAP is fully shared >> between all pthreads, but all objects in handwritten JS code you write are >> all thread local (JS objects are not shared between threads) >> - Run the compiled output in Firefox Nightly. Remember to deploy the >> new output file pthread-main.js alongside the rest of the build outputs, >> that contains the "launcher" for pthreads code. >> - The implementation should be fairly mature at this point, so don't be >> shy to stress it. It has been tested on three large fronts so far: >> - The Emscripten unit test suite itself has a section for pthreads >> tests: browser.test_pthread_*, see >> https://github.com/kripken/emscripten/blob/incoming/tests/test_browser.py#L2517 >> - The Open POSIX Test suite has been ported over to build and run >> with Emscripten: https://github.com/juj/posixtestsuite/commits/master . >> The interface tests should all pass, and hence our implementation should be >> POSIX conformant, except for features that Emscripten can't support >> (inter-thread signals is one feature that is not available) >> - We have been working with Unity to develop multithreading support >> to Unity3D WebGL export code, and it is running quite nicely. Currently we >> are using this development as a source for scalability benchmarking like >> this: http://clb.demon.fi/emcc/pthreads/webgl_benchmark_pthreads.html >> >> You can find the documentation for pthreads here: >> https://github.com/kripken/emscripten/blob/incoming/site/source/docs/porting/pthreads.rst >> . Emscripten bug tracker also has a new label 'multithreading', and you can >> use that as a filter to follow the multithreading development. See >> https://github.com/kripken/emscripten/issues?q=is%3Aopen+is%3Aissue+label%3Amultithreading >> . >> >> Please do give a go to test how well your code empthreadizes, and let us >> know about the issues you are running into. Thanks! >> >> Jukka >> >> -- > You received this message because you are subscribed to the Google Groups > "emscripten-discuss" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
