Hey there, Congrats for the hard-work.
I'm trying to assemble the parts of the puzzle <https://github.com/kripken/worker-ui/issues/5>, and I was wondering: Is the Emscripten-pthread + mozilla-SharedArrayBuffer effort correlated to the ongoing mozilla-WorkerCanvas effort <https://bugzilla.mozilla.org/show_bug.cgi?id=709490>? Thanks, Ariel On Thursday, June 4, 2015 at 7:21:31 PM UTC+3, jj wrote: > > Hello all, > > we've got some really cool news to share! Earlier this week, the pthreads > pull request was merged in to Emscripten upstream, which adds support for > the POSIX threads API to the Emscripten incoming branch. > > The main reason why Emscripten has not had threading support before, is > that the current JavaScript Web Worker specification only allows the > workers (== pthreads to Emscripten) to communicate by passing messages to > each other. In native world, this corresponds to a multiprocess-like > parallel environment, where processes do interprocess pipes to push data to > each other, or like the MPI API that is used in big computing clusters that > consist of multiple separate computers that send messages through the > network. Web Workers have followed this approach, and they limit workers > from accessing each other's memory directly. This simplifies parallel > programming greatly, but it is a different paradigm that also limits the > kind of parallel algorithms that one could implement. > > Native threading builds strictly on the assumption that threads can share > memory with each other. This has been the hard and painful part for JS Web > Workers from Emscripten native threading perspective, since shared memory > has not been available. However for more than a year now, there have been > experiments going on to imagine what such a direct memory sharing approach > for JavaScript Web Workers would look like. This research process is a > collaboration between multiple browser vendors, and you can follow that > discussion here: > https://docs.google.com/document/d/1NDGA_gZJ7M7w1Bh8S0AoDyEqwDdRh4uSoTPSNn77PFk/edit#heading=h.a6o4dubw5qla > > , where the SharedArrayBuffer specification is being drafted. Having direct > shared memory enables more flexible options for parallelism compared to the > more limited message passing approaches, and this flexibility benefits both > native JS developers and compilers such as Emscripten. Alongside the > SharedArrayBuffer research, we have been working to add support for > pthreads to Emscripten that backs on to this research, and today we feel > that we are at the point where it makes sense to push the feature to the > incoming branch for everyone to test. > > It should be stressed that this feature is very experimental at the > moment, and the only browser to currently implement support for it is > Firefox Nightly. Being experimental means that there can be any number of > changes made to the specification, which can mean that pages that you have > compiled against it can stop working in Nightly in the future, if the spec > and the implementation changes. We don't want to wait however until the > standard is fixed, before bringing it to Emscripten, because your feedback > will be extremely valuable in shaping the draft further, and that we can > ensure that we get it right, and that the draft that will be proposed for > adoption will be a good one that covers the important Emscripten needs. So, > if you have some cycles to spare on experimenting with this feature to give > us feedback, please do, so that we can make sure we don't miss anything! > > What the pthreads support for Emscripten means for developers in practice: > > - Default build mode is still fully singlethreaded. You should not see > any pthreads-related code "leak in" to your non-pthreads builds ("you don't > pay for what you don't use"). If you do see this, please submit a bug > report! > - Pass the -s USE_PTHREADS=1 compiler and linker flag to enable > targeting pthreads. This enables the code to call the pthreads API, as well > as use the GCC built-in lockfree atomics operations and the futex wait&wake > API. See > https://github.com/kripken/emscripten/blob/incoming/system/include/emscripten/threading.h > > for a list of function calls enabled when targeting pthreads. > - When building with pthreads enabled, the asm.js HEAP is fully shared > between all pthreads, but all objects in handwritten JS code you write are > all thread local (JS objects are not shared between threads) > - Run the compiled output in Firefox Nightly. Remember to deploy the new > output file pthread-main.js alongside the rest of the build outputs, that > contains the "launcher" for pthreads code. > - The implementation should be fairly mature at this point, so don't be > shy to stress it. It has been tested on three large fronts so far: > - The Emscripten unit test suite itself has a section for pthreads > tests: browser.test_pthread_*, see > https://github.com/kripken/emscripten/blob/incoming/tests/test_browser.py#L2517 > - The Open POSIX Test suite has been ported over to build and run > with Emscripten: https://github.com/juj/posixtestsuite/commits/master . > The interface tests should all pass, and hence our implementation should be > POSIX conformant, except for features that Emscripten can't support > (inter-thread signals is one feature that is not available) > - We have been working with Unity to develop multithreading support > to Unity3D WebGL export code, and it is running quite nicely. Currently we > are using this development as a source for scalability benchmarking like > this: http://clb.demon.fi/emcc/pthreads/webgl_benchmark_pthreads.html > > You can find the documentation for pthreads here: > https://github.com/kripken/emscripten/blob/incoming/site/source/docs/porting/pthreads.rst > > . Emscripten bug tracker also has a new label 'multithreading', and you can > use that as a filter to follow the multithreading development. See > https://github.com/kripken/emscripten/issues?q=is%3Aopen+is%3Aissue+label%3Amultithreading > > . > > Please do give a go to test how well your code empthreadizes, and let us > know about the issues you are running into. Thanks! > > Jukka > > -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
