Hello all, we've got some really cool news to share! Earlier this week, the pthreads pull request was merged in to Emscripten upstream, which adds support for the POSIX threads API to the Emscripten incoming branch.
The main reason why Emscripten has not had threading support before, is that the current JavaScript Web Worker specification only allows the workers (== pthreads to Emscripten) to communicate by passing messages to each other. In native world, this corresponds to a multiprocess-like parallel environment, where processes do interprocess pipes to push data to each other, or like the MPI API that is used in big computing clusters that consist of multiple separate computers that send messages through the network. Web Workers have followed this approach, and they limit workers from accessing each other's memory directly. This simplifies parallel programming greatly, but it is a different paradigm that also limits the kind of parallel algorithms that one could implement. Native threading builds strictly on the assumption that threads can share memory with each other. This has been the hard and painful part for JS Web Workers from Emscripten native threading perspective, since shared memory has not been available. However for more than a year now, there have been experiments going on to imagine what such a direct memory sharing approach for JavaScript Web Workers would look like. This research process is a collaboration between multiple browser vendors, and you can follow that discussion here: https://docs.google.com/document/d/1NDGA_gZJ7M7w1Bh8S0AoDyEqwDdRh4uSoTPSNn77PFk/edit#heading=h.a6o4dubw5qla , where the SharedArrayBuffer specification is being drafted. Having direct shared memory enables more flexible options for parallelism compared to the more limited message passing approaches, and this flexibility benefits both native JS developers and compilers such as Emscripten. Alongside the SharedArrayBuffer research, we have been working to add support for pthreads to Emscripten that backs on to this research, and today we feel that we are at the point where it makes sense to push the feature to the incoming branch for everyone to test. It should be stressed that this feature is very experimental at the moment, and the only browser to currently implement support for it is Firefox Nightly. Being experimental means that there can be any number of changes made to the specification, which can mean that pages that you have compiled against it can stop working in Nightly in the future, if the spec and the implementation changes. We don't want to wait however until the standard is fixed, before bringing it to Emscripten, because your feedback will be extremely valuable in shaping the draft further, and that we can ensure that we get it right, and that the draft that will be proposed for adoption will be a good one that covers the important Emscripten needs. So, if you have some cycles to spare on experimenting with this feature to give us feedback, please do, so that we can make sure we don't miss anything! What the pthreads support for Emscripten means for developers in practice: - Default build mode is still fully singlethreaded. You should not see any pthreads-related code "leak in" to your non-pthreads builds ("you don't pay for what you don't use"). If you do see this, please submit a bug report! - Pass the -s USE_PTHREADS=1 compiler and linker flag to enable targeting pthreads. This enables the code to call the pthreads API, as well as use the GCC built-in lockfree atomics operations and the futex wait&wake API. See https://github.com/kripken/emscripten/blob/incoming/system/include/emscripten/threading.h for a list of function calls enabled when targeting pthreads. - When building with pthreads enabled, the asm.js HEAP is fully shared between all pthreads, but all objects in handwritten JS code you write are all thread local (JS objects are not shared between threads) - Run the compiled output in Firefox Nightly. Remember to deploy the new output file pthread-main.js alongside the rest of the build outputs, that contains the "launcher" for pthreads code. - The implementation should be fairly mature at this point, so don't be shy to stress it. It has been tested on three large fronts so far: - The Emscripten unit test suite itself has a section for pthreads tests: browser.test_pthread_*, see https://github.com/kripken/emscripten/blob/incoming/tests/test_browser.py#L2517 - The Open POSIX Test suite has been ported over to build and run with Emscripten: https://github.com/juj/posixtestsuite/commits/master . The interface tests should all pass, and hence our implementation should be POSIX conformant, except for features that Emscripten can't support (inter-thread signals is one feature that is not available) - We have been working with Unity to develop multithreading support to Unity3D WebGL export code, and it is running quite nicely. Currently we are using this development as a source for scalability benchmarking like this: http://clb.demon.fi/emcc/pthreads/webgl_benchmark_pthreads.html You can find the documentation for pthreads here: https://github.com/kripken/emscripten/blob/incoming/site/source/docs/porting/pthreads.rst . Emscripten bug tracker also has a new label 'multithreading', and you can use that as a filter to follow the multithreading development. See https://github.com/kripken/emscripten/issues?q=is%3Aopen+is%3Aissue+label%3Amultithreading . Please do give a go to test how well your code empthreadizes, and let us know about the issues you are running into. Thanks! Jukka -- You received this message because you are subscribed to the Google Groups "emscripten-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
