Hello Here is the latest OCaml Weekly News, for the week of April 06 to 13, 2021.
Table of Contents ───────────────── ocaml-wayland (pure OCaml wayland protocol library) Reparse v2.1.0 release Open source editor for iOS, iPadOS and macOS OCaml Café - Tue, Apr 13 @ 7pm Central Directories, an ocaml library Multicore OCaml: March 2021 ML Family Workshop 2021: first call for short abstracts and presentations ppx_let_locs - improve stack traces Converting typing information from one OCaml version to a later one? Set up OCaml 1.1.10 Old CWN ocaml-wayland (pure OCaml wayland protocol library) ═══════════════════════════════════════════════════ Archive: <https://discuss.ocaml.org/t/ann-ocaml-wayland-pure-ocaml-wayland-protocol-library/7616/1> Thomas Leonard announced ──────────────────────── I've just released [ocaml-wayland], a pure OCaml Wayland implementation. You can get it with: ┌──── │ opam install wayland └──── Wayland is a communications system intended for use between processes on a single computer. It is mainly used by graphical applications (clients) to talk to display servers, but nothing about it is specific to graphics and it could be used for other things. A Wayland protocol is defined by writing a schema file. Typed bindings can then be created automatically from that for various languages. Wayland's main interesting feature is that you can pass file descriptors as message arguments. For example, to send a frame of video you can get a file descriptor to some shared memory, draw the image to it, and then pass the FD to the server for rendering, with no need to copy the data. The ocaml-wayland library can be used to write applications, display servers, or proxies. The [example] in the repository opens a window showing some scrolling squares. Clicking reverses the direction and pressing keys changes the colours. I used the library to write [wayland-virtwl-proxy], which allows applications running in VMs to open windows directly on the host desktop, as described in my recent blog post "[Qubes-lite with KVM and Wayland]". Note that, unlike the older X11 protocol, Wayland doesn't provide any graphics primitives (such as drawing rectangles or text). Instead, applications render the window contents however they please and then simply share the buffer with the compositor. So if you want to write a normal application using it, you'll need to find or write a separate GUI toolkit too. [ocaml-wayland] <https://github.com/talex5/ocaml-wayland> [example] <https://github.com/talex5/ocaml-wayland/blob/master/example/test.ml> [wayland-virtwl-proxy] <https://github.com/talex5/wayland-virtwl-proxy> [Qubes-lite with KVM and Wayland] <https://roscidus.com/blog/blog/2021/03/07/qubes-lite-with-kvm-and-wayland/> Reparse v2.1.0 release ══════════════════════ Archive: <https://discuss.ocaml.org/t/ann-reparse-v2-1-0-release/7619/1> Bikal Lem announced ─────────────────── I have just released v2.1.0 of reparse to opam. Below are the CHANGES for this release. CHANGES: This release has backwards incompatible changes. • Infix functions are now available (opened by default) in `Parser' module itself. • Add support for let operators - `let+' , `let*' , `and+' and `and*' . • `bind' and `map' function are now labelled following `base' library dependency convention. • Items in `all_unit' are now `unit t' rather than `_ t' following monad combinator convention in `base' library dependency. • `pure' is now deprecated. Use `return' instead. This is to stay consistent with monad conventions in `base' library dependency. • `>|=' is deprecated. Use `>>|' instead. This is to stay consistent with monad conventions in `base' library dependency. • Removed `map4' function. • Add support for `ppx_let' . • Deprecate `Parser' module. Use `Reparse' instead. Open source editor for iOS, iPadOS and macOS ════════════════════════════════════════════ Archive: <https://discuss.ocaml.org/t/open-source-editor-for-ios-ipados-and-macos/7624/1> Nathan Fallet announced ─────────────────────── I'm proud to share with you the editor I've been working on theses last weeks, which is also open source! The editor is available here: [App Store link] And the source code here: [GitHub repository] A Pull Request just got merged, it is now part of the "Mobile" section on the Install page on the OCaml website. Feel free to give your feedback, and/or contribute to the project on GitHub to help us to make a better tool! [App Store link] <https://apps.apple.com/app/ocaml-learn-code/id1547506826> [GitHub repository] <https://github.com/GroupeMINASTE/OCaml-iOS> EduardoRFS then said and borisd replied ─────────────────────────────────────── I have dream, a day where this kind of app will be written in OCaml I've done some experimental Cocoa bindings, and the basics are working, but the scope of the project is too large for a single person effort. Here's an example basic app: <https://github.com/dboris/ocaml-cocoa/blob/master/examples/count_clicks.ml> Would be interested to contribute to a project with a similar goal, if someone has a better idea how to approach this. Yawar Amin added ──────────────── Jeffrey Scofield has built some iOS apps with OCaml, so it's certainly possible: <https://psellos.com/ocaml/compile-to-iphone.html> Returning to the original post, Nathan Fallet answered a question ───────────────────────────────────────────────────────────────── Yes, this app can be used offline! It works with a toplevel built with js_of_ocaml, stored in the app, and everything runs on the device (You can browse on GitHub the `OCaml/JavaScript/' folder which contains the toplevel itself). So as everything is already built-in, no internet connection is required. I will clarify it on the App Store page, because even if it was evident for me, I notice it is not for everyone. Last thing, I'm myself in CPGE (MPSI to be exact), and that's the reason why I started working on this app on my free time. Lilian Besson asked and Nathan Fallet replied ───────────────────────────────────────────── • how is automated the build and the upload of the app on the App Store? If a new OCaml version comes every couple of months, how much time would it require to rebuild and upgrade the app and publish/update it on the App Store? • I build the app manually, and upload the file to the App Store. So if a new version of OCaml is released, I need to rebuild the top level with Js_of_ocaml, and build a new version of the app that I upload. • Can users save (export) and import `.ml' files? If so, advertise this feature too! Can users write in more than one file at a time (like it's possible for BetterOCaml.ml in a single browser tab!) • Users can open and save their .ml files, and even open them from the file explorer (double click on a .ml file will open it in the app), it's like a real code editor. For now only one file can be opened at time, but I plan to make a way to open multiple files (still thinking about it…) • I guess I know the answer but… the technology you chose to build the app is limited to Apple-products only, and it's not a cross-platform technology like Flutter.dev or others recent technologies based on HTML/CSS/Javascript? If it is, how hard do you think it would be for you to also publish your app on Google Play store, and F-Droid.org ? The only app on Google Play is 10 year-old, for OCaml 3.12! <https://play.google.com/store/apps/details?id=fr.vernoux.ocaml>, it would be so great to publish a more recent and modern alternative! • Of course I wrote the app with pure Swift (except the top level which is built with JS), but an Android version of the app can be written without problem (it's just about the user interface), and then published to the Google Play Store and whatever other store for Android phones. As holidays are starting now in France, I will consider improving the app with feedbacks I got those last days, publish an update to the App Store, and start working on an Android version. OCaml Café - Tue, Apr 13 @ 7pm Central ══════════════════════════════════════ Archive: <https://discuss.ocaml.org/t/ocaml-cafe-tue-apr-13-7pm-central/7627/1> Claude Jager-Rubinson announced ─────────────────────────────── Please join us this coming Tuesday, April 13 at 7pm Central for OCaml Café. Zoom info is at [https://hfpug.org/event/ocaml-cafe-3/] OCaml Café offers a friendly, low stakes opportunity to ask questions about the OCaml language and ecosystem, work through programming problems that you’re stuck on, and get feedback on your code. Especially geared toward new and intermediate users, experienced OCaml developers will be available to answer your questions. Whether you’re still trying to make sense of currying or can spot non-tail-recursive code from across the room, we hope that you’ll join us with your questions about OCaml, or just to hang out with the OCaml community. [https://hfpug.org/event/ocaml-cafe-3/] <https://hfpug.org/event/ocaml-cafe-3/> Directories, an ocaml library ═════════════════════════════ Archive: <https://discuss.ocaml.org/t/directories-an-ocaml-library/6753/2> zapashcanon announced ───────────────────── We released directories 0.3 recently. In this release, we dropped the Windows dependency on ctypes-foreign and use ctypes.stubs instead. See [#10] This allowed us to clean the code and to remove [some dirty hacks]. [#10] <https://github.com/OCamlPro/directories/pull/10> [some dirty hacks] <https://github.com/OCamlPro/directories/pull/11> Multicore OCaml: March 2021 ═══════════════════════════ Archive: <https://discuss.ocaml.org/t/multicore-ocaml-march-2021/7639/1> Anil Madhavapeddy announced ─────────────────────────── Welcome to the March 2021 [Multicore OCaml] monthly report! The following update and the [previous ones] have been compiled by me, @kayceesrk and @shakthimaan. We remain broadly on track to integrate the last of the multicore prerequisites into the next (4.13) release, and to propose domains-only parallelism for OCaml 5.0. [Multicore OCaml] <https://github.com/ocaml-multicore/ocaml-multicore> [previous ones] <https://discuss.ocaml.org/tag/multicore-monthly> Upstream OCaml 4.13 development ╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌ The complex safe points PR ([#10039]) is continuing to make progress, with more refinement towards reducing the binary size increase that results from the introduction of more polling points. Special thanks to @damiendoligez for leaping in with a [PR-to-the-PR] to home in on a workable algorithm! [#10039] <https://github.com/ocaml/ocaml/pull/10039> [PR-to-the-PR] <https://github.com/sadiqj/ocaml/pull/3> Multicore OCaml trees ╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌ If there's one thing we're not going to miss, it's git rebasing. The multicore journey began many moons ago with OCaml [4.02], and then [4.04], [4.06], and the current [4.10]. We're pleased to announce the hopefully-last rebase of the multicore OCaml trees to OCaml 4.12.0 are now available. There is now a simpler naming scheme as well to reflect our upstreaming strategy more closely: • OCaml 4.12.0+domains is the domains-only parallelism that will be submitted for OCaml 5.0 • OCaml 4.12.0+domains+effects is the version with domains parallelism and effects-based concurrency. You can find opam installation instructions for these over at [the multicore-opam] repository. There is even an ocaml-lsp-server available, so that your favourite IDE should just work! [4.02] <https://github.com/ocaml-multicore/ocaml-multicore/commits/master-4.02.2> [4.04] <https://github.com/ocaml-multicore/ocaml-multicore/tree/4.04.2+multicore> [4.06] <https://github.com/ocaml-multicore/ocaml-multicore/tree/4.06.1+multicore> [4.10] <https://github.com/ocaml-multicore/ocaml-multicore/commits/parallel_minor_gc> [the multicore-opam] <https://github.com/ocaml-multicore/multicore-opam> Domains-only parallelism trees ┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄ The bulk of effort this month has been around the integration and debugging of Domain Local Allocation Buffers (DLABs), and also chasing down corner-case failures from stress testing and opam bulk builds. For details, see the long list of PRs in the next section. We're also cleaning up historical vestiges in order to reduce the diff to OCaml trunk, in order to clear the path to a clean diff for generating OCaml 5.0 PRs for upstream integration. Concurrency and Effects trees ┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄ *The camera-ready paper for PLDI 2021 on [Retrofitting Effect handlers onto OCaml] is now available on arXiv.* The code described in the paper can be used via the `4.12.0+domains+effects' opam switches. Please feel free to keep any comments coming to @kayceesrk and myself. We've also been hacking on the multicore IO stack and just beginning to combine concurrency (via effects) and parallelism (via domains) into Linux io_uring, macOS' Grand Central Dispatch and Windows iocp. We'll have more to report on this over the next few months, but early benchmarking numbers on Linux are promising. [Retrofitting Effect handlers onto OCaml] <https://arxiv.org/abs/2104.00250> CI and Benchmarking ╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌ We are continuing to expand the testing for different CI configurations for the project. With respect to Sandmark benchmarking, we are in the process of adding the Irmin layers.ml benchmark. There is also an end-to-end pipeline of using the OCurrent [current-bench] framework to give us benchmarking results from PRs that can be compared to previous runs. As always, we begin with the Multicore OCaml updates, which are then followed by the ongoing and completed tasks for the Sandmark benchmarking project. Finally, the upstream OCaml work is listed for your reference. [current-bench] <https://github.com/ocurrent/current-bench> Detailed Updates ╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌ Multicore OCaml ╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌ Ongoing ┄┄┄┄┄┄┄ DLAB ┈┈┈┈ • [ocaml-multicore/ocaml-multicore#484] Thread allocation buffers The PR provides an implementation for thread local allocation buffers or `Domain Local Allocation Buffers'. Code review and testing of the changes is in progress. • [ocaml-multicore/ocaml-multicore#508] Domain Local Allocation Buffers This is an extension to the `Thread allocation buffers' PR with initialization, lazy resizing of the global minor heap size, and rebase to 4.12 branch. [ocaml-multicore/ocaml-multicore#484] <https://github.com/ocaml-multicore/ocaml-multicore/pull/484> [ocaml-multicore/ocaml-multicore#508] <https://github.com/ocaml-multicore/ocaml-multicore/pull/508> Testing ┈┈┈┈┈┈┈ • [ocaml-multicore/ocaml-multicore#522] Building the runtime with -O0 rather than -O2 causes testsuite to fail The runtime tests fail when using `-O0' instead of `-O2' and this needs to be investigated further. • [ocaml-multicore/ocaml-multicore#526] weak-ephe-final issue468 can fail with really small minor heaps The `weak-ephe-final' tests with a small minor heap (4096 words) cause the issue468 test to fail. • [ocaml-multicore/ocaml-multicore#528] Expand CI runs A list of requirements to expand the scope and execution of our existing CI runs for comprehensive testing. [ocaml-multicore/ocaml-multicore#522] <https://github.com/ocaml-multicore/ocaml-multicore/issues/522> [ocaml-multicore/ocaml-multicore#526] <https://github.com/ocaml-multicore/ocaml-multicore/issues/526> [ocaml-multicore/ocaml-multicore#528] <https://github.com/ocaml-multicore/ocaml-multicore/pull/528> Sundries ┈┈┈┈┈┈┈┈ • [ocaml-multicore/ocaml-multicore#514] Update instructions in ocaml-variants.opam The `ocaml-variants.opam' and `configure.ac' files have been updated to use the Multicore OCaml repository, and to use a local switch instead of a global one. The current Multicore OCaml is at the 4.12 branch. • [ocaml-multicore/ocaml-multicore#523] Systhreads Mutex raises Sys_error The error checking for Systhreads Mutex should be inline with trunk, instead of the fatal errors reported by Multicore OCaml. • [ocaml-multicore/ocaml-multicore#527] Port eventlog to CTF The `eventlog' implementation has to be ported to the Common Trace Format. The log output should be consistent with the parallel_minor_gc output, and stress testing need to be performed. [ocaml-multicore/ocaml-multicore#514] <https://github.com/ocaml-multicore/ocaml-multicore/pull/514> [ocaml-multicore/ocaml-multicore#523] <https://github.com/ocaml-multicore/ocaml-multicore/pull/523> [ocaml-multicore/ocaml-multicore#527] <https://github.com/ocaml-multicore/ocaml-multicore/pull/527> Completed ┄┄┄┄┄┄┄┄┄ Upstream ┈┈┈┈┈┈┈┈ • [ocaml-multicore/ocaml-multicore#490] Remove getmutablefield from bytecode The bytecode compiler and interpreter have been updated by removing the `getmutablefield' opcodes. • [ocaml-multicore/ocaml-multicore#496] Replace caml_initialize_field with caml_initialize A patch to replace `caml_initialize_field', which was earlier used with the concurrent minor collector, is now replaced with `caml_initialize'. • [ocaml-multicore/ocaml-multicore#503] Re-enable lib-obj and asmcomp/is_static tests The `lib-obj' and `asmcomp/is_static' tests have been re-enabled and the configure settings have been updated for Multicore NO_NAKED_POINTERS. • [ocaml-multicore/ocaml-multicore#506] Replace `Op_val' with `Field' The use of `Op_val (x)[i]' has been replaced with `Field (x, i)' to be consistent with trunk implementation. • [ocaml-multicore/ocaml-multicore#507] Change interpreter to use naked code pointers The changes have been made to identify naked pointers in the interpreter stack to be compatible with trunk. • [ocaml-multicore/ocaml-multicore#516] Remove caml_root API The `caml_root' variables have been changed to `value' type and are managed as generational global roots. Hence, the `caml_root' API is now removed. [ocaml-multicore/ocaml-multicore#490] <https://github.com/ocaml-multicore/ocaml-multicore/pull/490> [ocaml-multicore/ocaml-multicore#496] <https://github.com/ocaml-multicore/ocaml-multicore/pull/496> [ocaml-multicore/ocaml-multicore#503] <https://github.com/ocaml-multicore/ocaml-multicore/pull/503> [ocaml-multicore/ocaml-multicore#506] <https://github.com/ocaml-multicore/ocaml-multicore/pull/506> [ocaml-multicore/ocaml-multicore#507] <https://github.com/ocaml-multicore/ocaml-multicore/pull/507> [ocaml-multicore/ocaml-multicore#516] <https://github.com/ocaml-multicore/ocaml-multicore/pull/516> DLAB ┈┈┈┈ • [ocaml-multicore/ocaml-multicore#511] Allocate unique root token on the major heap instead of the minor The unique root token allocation is now done on the major heap allocation that does not raise any exception, and exits cleanly when a domain creation fails. • [ocaml-multicore/ocaml-multicore#513] Clear the minor heap at the end of a collection in debug runtime A debug value is written to every element of the minor heap for debugging failures. We now clear the minor heap at the end of a minor collection. • [ocaml-multicore/ocaml-multicore#519] Make timing test more robust The `timing.ml' test has been updated to be more resilient for testing with DLABs. [ocaml-multicore/ocaml-multicore#511] <https://github.com/ocaml-multicore/ocaml-multicore/pull/511> [ocaml-multicore/ocaml-multicore#513] <https://github.com/ocaml-multicore/ocaml-multicore/pull/513> [ocaml-multicore/ocaml-multicore#519] <https://github.com/ocaml-multicore/ocaml-multicore/pull/519> Enhancements ┈┈┈┈┈┈┈┈┈┈┈┈ • [ocaml-multicore/ocaml-multicore#477] Move TLS areas to a dedicated memory space In order to support Domain Local Allocation Buffer, we now move the TLS areas to its own memory alloted space thereby changing the way we allocate an individual domain's TLS. • [ocaml-multicore/ocaml-multicore#480] Remove leave_when_done and friends from STW API The barriers from `caml_try_run_on_all_domains*' and `stw_request' are removed by cleaning up the `stw_request.leave_when_done' implementation. • [ocaml-multicore/ocaml-multicore#481] Don't share array amongst domains in gc-roots tests Every domain should have its own array, and the parallel global roots tests have been updated with this change. • [ocaml-multicore/ocaml-multicore#494] Stronger invariants on unix_fork We now enforce stronger invariants such that no other domain can run alongside domain 0 (`caml_domain_alone') for `unix_fork'. • [ocaml-multicore/ocaml-multicore#515] Add memprof stubs to build and stdlib The required `memprof' functions have been added to build `stdlib', and also to build memprof for the runtime. [ocaml-multicore/ocaml-multicore#477] <https://github.com/ocaml-multicore/ocaml-multicore/pull/477> [ocaml-multicore/ocaml-multicore#480] <https://github.com/ocaml-multicore/ocaml-multicore/pull/480> [ocaml-multicore/ocaml-multicore#481] <https://github.com/ocaml-multicore/ocaml-multicore/pull/481> [ocaml-multicore/ocaml-multicore#494] <https://github.com/ocaml-multicore/ocaml-multicore/pull/494> [ocaml-multicore/ocaml-multicore#515] <https://github.com/ocaml-multicore/ocaml-multicore/pull/515> Lazy Updates ┈┈┈┈┈┈┈┈┈┈┈┈ • [ocaml-multicore/ocaml-multicore#501] Safepoints lazy fix The lazy implementation need to be aware of safe points, and we need to differentiate between recursive forcing of lazy values from parallel forcing. These fixes are from [ocaml-multicore#492] and [ocaml-multicore#493]. • [ocaml-multicore/ocaml-multicore#505] Add a unique domain token to distinguish lazy forcing failure A `caml_ml_domain_unique_token' has been added to handle racy access by multiple mutators. This fixes the [using domain id (int)] to identify forcing domain of lazy block issue. [ocaml-multicore/ocaml-multicore#501] <https://github.com/ocaml-multicore/ocaml-multicore/pull/501> [ocaml-multicore#492] <https://github.com/ocaml-multicore/ocaml-multicore/pull/492> [ocaml-multicore#493] <https://github.com/ocaml-multicore/ocaml-multicore/pull/493> [ocaml-multicore/ocaml-multicore#505] <https://github.com/ocaml-multicore/ocaml-multicore/pull/505> [using domain id (int)] <https://github.com/ocaml-multicore/ocaml-multicore/issues/504> Fixes ┈┈┈┈┈ • [ocaml-multicore/ocaml-multicore#487] systhreads: set gc_regs_buckets and friends to NULL at thread startup Pointers have been initialized to NULL in `systhreads/st_stubs.c' which solves the [segmentation fault] observed when running the Layers benchmark. • [ocaml-multicore/ocaml-multicore#491] Reinitialize child locks after fork The runtime needs to operate correctly after a `fork', and this patch fixes it with proper resetting of domain lock. • [ocaml-multicore/ocaml-multicore#495] Fix problems with finaliser orphaning A fix for how we merge finalization tables for orphaned finaliser work. A test case has also been added to the PR. • [ocaml-multicore/ocaml-multicore#499] Fix backtrace unwind The unwinding of stacks over callbacks was not happening correctly and the discrepancy in `caml_next_frame_descriptior' is now resolved. • [ocaml-multicore/ocaml-multicore#509] Fix for bad setup of Continuation_already_taken exception in bytecode A patch to fix the `Continuation_already_taken' exception which was not set up as needed in the bytecode execution. • [ocaml-multicore/ocaml-multicore#510] Update a testcase in principality-and-gadts.ml A change in `principality-and-gadts.ml' to log the correct output as compared to 4.12 branch in ocaml/ocaml. [ocaml-multicore/ocaml-multicore#487] <https://github.com/ocaml-multicore/ocaml-multicore/pull/487> [segmentation fault] <https://github.com/ocaml-multicore/ocaml-multicore/issues/485> [ocaml-multicore/ocaml-multicore#491] <https://github.com/ocaml-multicore/ocaml-multicore/pull/491> [ocaml-multicore/ocaml-multicore#495] <https://github.com/ocaml-multicore/ocaml-multicore/pull/495> [ocaml-multicore/ocaml-multicore#499] <https://github.com/ocaml-multicore/ocaml-multicore/pull/499> [ocaml-multicore/ocaml-multicore#509] <https://github.com/ocaml-multicore/ocaml-multicore/pull/509> [ocaml-multicore/ocaml-multicore#510] <https://github.com/ocaml-multicore/ocaml-multicore/pull/510> Ecosystem ┈┈┈┈┈┈┈┈┈ • [ocaml-multicore/multicore-opam#46] Multicore compatible ocaml-migrate-parsetree.2.1.0 The `ocaml-migrate-parsetree' package uses the effect syntax and now builds with Multicore OCaml `parallel_minor_gc' branch. • [ocaml-multicore/multicore-opam#47] Multicore compatible ppxlib The effect syntax has been added to `ppxlib' and is also now compatible with Multicore OCaml. • [ocaml-multicore/multicore-opam#49] 4.12 Multicore configs Added configurations to install `4.12.0+domains+effects' and `4.12.0+domains' OCaml variants. • [ocaml-multicore/ocaml-multicore#473] Building on musl requires dynamically linked execinfo The opam files to allow installation on musl-based environments for Multicore OCaml have been added to the repository. • [ocaml-multicore/ocaml-multicore#482] Check for -lexecinfo in order to build on musl/alpine A `configure' script has been added which checks for `-lexecinfo' in order to support building Multicore OCaml on musl/alpine. [ocaml-multicore/multicore-opam#46] <https://github.com/ocaml-multicore/multicore-opam/pull/46> [ocaml-multicore/multicore-opam#47] <https://github.com/ocaml-multicore/multicore-opam/pull/47> [ocaml-multicore/multicore-opam#49] <https://github.com/ocaml-multicore/multicore-opam/pull/49> [ocaml-multicore/ocaml-multicore#473] <https://github.com/ocaml-multicore/ocaml-multicore/issues/473> [ocaml-multicore/ocaml-multicore#482] <https://github.com/ocaml-multicore/ocaml-multicore/pull/482> Documentation ┈┈┈┈┈┈┈┈┈┈┈┈┈ • [ocaml-multicore/ocaml-multicore#502] Update README to introduce 4.12+domains+effects and 4.12+domains We have updated the README file with the current list of active branches, and the names of the historic variants. • [ocaml-multicore/ocaml-multicore#520] Clarify comment on RacyLazy A documentation update in `stdlib/lazy.mli' that clarifies when `RacyLazy' and `Undefined' are raised. [ocaml-multicore/ocaml-multicore#502] <https://github.com/ocaml-multicore/ocaml-multicore/pull/502> [ocaml-multicore/ocaml-multicore#520] <https://github.com/ocaml-multicore/ocaml-multicore/pull/520> Sundries ┈┈┈┈┈┈┈┈ • [ocaml-multicore/ocaml-multicore#486] Sync no-effects-syntax to parallel_minor_gc branch The `ocaml-multicore:no-effects-syntax' branch is now up to date with the `parallel_minor_gc' branch changes. • [ocaml-multicore/ocaml-multicore#489] Remove promote_to The `promote_to' function was used in the concurrent minor GC. It is not required any more and hence has been removed. • [ocaml-multicore/ocaml-multicore#500] Replace caml_modify_field with caml_modify The `caml_modify_field' is no longer necessary and has been replaced with `caml_modify'. [ocaml-multicore/ocaml-multicore#486] <https://github.com/ocaml-multicore/ocaml-multicore/pull/486> [ocaml-multicore/ocaml-multicore#489] <https://github.com/ocaml-multicore/ocaml-multicore/pull/489> [ocaml-multicore/ocaml-multicore#500] <https://github.com/ocaml-multicore/ocaml-multicore/pull/500> Benchmarking ╌╌╌╌╌╌╌╌╌╌╌╌ Ongoing ┄┄┄┄┄┄┄ • [ocaml-bench/sandmark#204] Adding layers.ml as a benchmark to Sandmark The inclusion of Irmin layers.ml benchmark with updates to all its dependency requirements. • [ocaml-bench/sandmark#209] Use rule target kronecker.txt and remove from macro_bench A review of the graph500seq `kernel1.ml' implementation has been done, and code changes have been proposed. The `macro_bench' tag will be retained for the `graph500' benchmarks. • [ocaml-bench/sandmark#212] Increasing the major heap allocation on some benchmarks A work in progress to add more longer running benchmarks that involve major heap allocation. Some of the parameters have been updated with higher values, and more loops have been added as well. • We now have integrated the build of Sandmark 2.0 with [current-bench] for CI. The results of the benchmark runs are now pushed to a PostgreSQL database as shown below: ┌──── │ docker=# select * from benchmarks; │ -[ RECORD 1 ]--+------------------------------------------------------- │ run_at | 2021-03-26 11:21:20.64 │ repo_id | local/local │ commit | 55c6fb6416548737b715d6d8fde6c0f690526e42 │ branch | 2.0.0-alpha+001 │ pull_number | │ benchmark_name | │ test_name | coq.BasicSyntax.v │ metrics | {"maxrss_kB": 678096, "time_secs": 101.99969387054443} │ duration | 00:37:52.776357 │ -[ RECORD 2 ]--+------------------------------------------------------- │ run_at | 2021-03-26 11:21:20.64 │ repo_id | local/local │ commit | 55c6fb6416548737b715d6d8fde6c0f690526e42 │ branch | 2.0.0-alpha+001 │ pull_number | │ benchmark_name | │ test_name | thread_ring_lwt_mvar.20_000 │ metrics | {"maxrss_kB": 8096, "time_secs": 2.6146790981292725} │ duration | 00:37:52.776357 │ ... └──── We will continue to work on adding more workflows and features to `current-bench' to support Sandmark builds. [ocaml-bench/sandmark#204] <https://github.com/ocaml-bench/sandmark/pull/204> [ocaml-bench/sandmark#209] <https://github.com/ocaml-bench/sandmark/pull/209> [ocaml-bench/sandmark#212] <https://github.com/ocaml-bench/sandmark/pull/212> [current-bench] <https://github.com/ocurrent/current-bench> Completed ┄┄┄┄┄┄┄┄┄ • [ocaml-bench/sandmark#202] Added bench clean target in the Makefile A `benchclean' target to remove the generated benchmarks and its results while still retaining the `_opam' folder has been added to the Makefile. • [ocaml-bench/sandmark#203] Implement ITER support The use of ITER variable is now supported in Sandmark, and you can run multiple iterations of the benchmarks. For example, with `ITER=2', a couple of summary .bench files are created with the benchmark results as shown below: ┌──── │ $ TAG='"run_in_ci"' make run_config_filtered.json │ $ ITER=2 RUN_CONFIG_JSON=run_config_filtered.json make ocaml-versions/4.10.0+multicore.bench │ │ $ ls _results/ │ 4.10.0+multicore_1.orun.summary.bench 4.10.0+multicore_2.orun.summary.bench └──── • [ocaml-bench/sandmark#208] Fix params for simple-tests/capi A minor fix in `run_config.json' to correctly pass the arguments to the `simple-tests/capi' benchmark execution. You can verify the same using the following commands: ┌──── │ $ TAG='"lt_1s"' make run_config_filtered.json │ $ RUN_CONFIG_JSON=run_config_filtered.json make ocaml-versions/4.10.0+multicore.bench └──── • [ocaml-bench/sandmark#210] Don't share array in global roots parallel benchmarks A patch to not share array in global roots implementation for parallel benchmarks. • [ocaml-bench/sandmark#213] Resolve dependencies for 4.12.1+trunk, 4.12.0+domains and 4.12.0+domains+effects The `dependencies/packages' have now been updated to be able to build `4.12.1+trunk', `4.12.0+domains' and `4.12.0+domains+effects' branches with Sandmark. [ocaml-bench/sandmark#202] <https://github.com/ocaml-bench/sandmark/pull/202> [ocaml-bench/sandmark#203] <https://github.com/ocaml-bench/sandmark/pull/203> [ocaml-bench/sandmark#208] <https://github.com/ocaml-bench/sandmark/pull/208> [ocaml-bench/sandmark#210] <https://github.com/ocaml-bench/sandmark/pull/210> [ocaml-bench/sandmark#213] <https://github.com/ocaml-bench/sandmark/pull/213> OCaml ╌╌╌╌╌ Ongoing ┄┄┄┄┄┄┄ • [ocaml/ocaml#10039] Safepoints The review of the Safepoints PR is in progress. Special thanks to Damien Doligez for his [code suggestions] on safepoints and inserting polls. There is still work to be done on optimizations. Many thanks to all the OCaml users, developers and contributors in the community for their support to the project. Stay safe! [ocaml/ocaml#10039] <https://github.com/ocaml/ocaml/pull/10039> [code suggestions] <https://github.com/sadiqj/ocaml/pull/3> Acronyms ╌╌╌╌╌╌╌╌ • API: Application Programming Interface • CI: Continuous Integration • CTF: Common Trace Format • DLAB: Domain Local Allocation Buffer • GC: Garbage Collector • OPAM: OCaml Package Manager • PR: Pull Request • STW: Stop The World • TLS: Thread Local Storage Nate asked and Anil Madhavapeddy replied ──────────────────────────────────────── Sorry if this has been answered elsewhere – am I reading this correctly that effects aren’t intended for OCaml 5? If not, are there plans to merge effect handlers into mainline OCaml at some later stage, or is it going to be kept separate? Just to be clear, effects are not slated for any official release of OCaml yet. The focus is on making sure that OCaml 5.0 is a really solid release with domains-only parallelism, and then to look into subsequent changes required for concurrency. There are some considerations about how precisely effects should work when they materialise in a mainline OCaml release: • *semantics:* how will the introduction of effect handlers complicate (or improve!) reasoning about OCaml programs? @fpottier and Paulo Emílio de Vilhena have a paper on ["separation logic for effect handlers"] at POPL 2021 that is a good starting point to read from. • *performance:* on the other side, we want to make sure that effect handlers do not introduce a runtime overhead to sequential OCaml code. Our paper on "[retrofitting effect handlers onto OCaml]" addresses this, with a particular restriction on making sure handlers are one-shot only (a position conveniently reinforced by @fpottier's paper above, as it also makes program reasoning much easier). • *type safety:* our current implementation of effects is untyped, which doesn't fit that well into the spirit of ML. There is active research ongoing into developing a theory of typed effects lead by @lpw25, and you can see a [tech talk about this]. • *portability:* OCaml is an extremely portable language, we want to make sure we can maintain clean compilation to existing targets (such as js_of_ocaml) and new ones (such as webassembly). @kayceesrk is active in the [wasm working groups] on this topic, and we're using MirageOS (which has compiler forks for esp32 embedded systems) to experiment with overheads using unikernels. • *ergonomics:* and last but not least, the hardest one to answer: once we develop a workable model for typed effects in OCaml, does it work at scale on large codebases and also for writing incremental code that can be locally reasoned about? The only answer to this is to build prototypes of a significant size, which is something we're working on at OCaml Labs as part of efforts like our multicore IO stack. I don't talk about this much at the moment as we regularly thrown prototypes away, but we're converging on some stability in recent months. To risk an analogy, imagine that we're flying from Cambridge to Fiji for a nice post-pandemic vacation. To get to our end goal, we have to fly via a few intermediate cities (say, London and Sydney). We need to be very careful not to screw up our landing into Sydney because we're so excited about eventually getting to Fiji – if we do that, there is no glorious holiday for us! And you know what? Sydney's not so bad a place to hang out and adjust to jetlag while we carefully plan the flight path to Fiji. We could really use the community's help with OCaml 5.0, as it's going to be a major release. Rest assured that we're thinking (and working hard) on how concurrency should also show up in OCaml, but it won't happen until we get OCaml 5.0 out of the door. Core OCaml development is all about careful, iterative progress to avoid the risk of losing the properties that make OCaml so reliable for all the downstream projects that depend on us. Having said all that, I can't wait to get to get to Fiji either. It's going to be an awesome vacation when we finally arrive. ["separation logic for effect handlers"] <http://cambium.inria.fr/~fpottier/publis/de-vilhena-pottier-sleh.pdf> [retrofitting effect handlers onto OCaml] <https://arxiv.org/abs/2104.00250> [tech talk about this] <https://www.janestreet.com/tech-talks/effective-programming/> [wasm working groups] <https://kcsrk.info/slides/WASM_CG_4Aug20.pdf> ML Family Workshop 2021: first call for short abstracts and presentations ═════════════════════════════════════════════════════════════════════════ Archive: <https://sympa.inria.fr/sympa/arc/caml-list/2021-04/msg00010.html> Jonathan Protzenko announced ──────────────────────────── (tl;dr) The ML family workshop is back, and will be held virtually along with ICFP 2021. The workshop does not have proceedings, making it the perfect venue to run some ideas with the community or present some work in progress within a friendly environment. The PC has a broad expertise and submissions are 3 pages long: when in doubt, just submit! (long version) We are happy to announce that the ML Family Workshop is back for its 2021 edition, which we will be held online on Thursday August 26th, in conjunction with ICFP 2021. The ML family workshop warmly welcomes submission touching on the programming languages traditionally seen as part of the "ML family" (Standard ML, OCaml, F#, CakeML, SML#, Manticore, MetaOCaml, etc.). The scope of the workshop includes all aspects of the design, semantics, theory, application, implementation, and teaching of the members of the ML family. We also encourage presentations from related languages (such as Haskell, Scala, Rust, Nemerle, Links, Koka, F*, Eff, ATS, etc), to exchange experience of further developing ML ideas. Submission details ╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌ Submissions must be at most three pages long; see the full call for papers <https://icfp21.sigplan.org/home/mlfamilyworkshop-2021#Call-for-Presentations> for details. Submission site: <https://ml21.hotcrp.com/> Important dates ╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌ • Thu, May 27th 2021 (AoE): submission deadline • Thu, Jun 17th 2021 (AoE): author notification • Thu, Aug 26th: workshop (time slots TBD) Program committee ╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌ • Danel Ahman (University of Ljubljana) • Robert Atkey (University of Strathclyde) • Frédéric Bour (Tarides) • Ezgi Çiçek (Facebook London) • Youyou Cong (Tokyo Institute of Technology) • Richard A. Eisenberg (Tweag I/O) • Martin Elsman (University of Copenhagen, Denmark) • Ohad Kammar (University of Edinburgh) • Naoki Kobayashi (University of Tokyo, Japan) • Benoît Montagu (Inria) • Jonathan Protzenko (Microsoft Research) (Chair) • Kristina Sojakova (INRIA Paris) • Don Syme (Microsoft) • Matías Toro (University of Chile) • Katsuhiro Ueno (Tohoku University) ppx_let_locs - improve stack traces ═══════════════════════════════════ Archive: <https://discuss.ocaml.org/t/ann-ppx-let-locs-improve-stack-traces/7646/1> EduardoRFS announced ──────────────────── A common problem that I personally had when developing async code in OCaml is that both `Lwt' and `Async' leads to useless stack traces. When using `lwt_ppx' that is handled by the ppx, but the same is not possible with `let*', `let.await' or `>>='. If you want to solve that, you can use [ppx_let_locs], it will detect your calls and when there is a replacement function available that would allow stack traces to be enhanced it will enhance that by default for you. [ppx_let_locs] <https://github.com/EduardoRFS/ppx_let_locs> How it works ╌╌╌╌╌╌╌╌╌╌╌╌ Everytime an ident is applied it tries to find a `backtrace_ident' or `ident_backtrace' version of it, if that is present and it has the right signature, the PPX will then apply the backtraced one with the additional argument. This means it also works with operators, like `>>=_backtrace', as they're just normal function application. For letop there is a special case, the signature of the letop must be slightly different to ensure that the typing order was not changed, essentially the `let*_backtrace' must have the following signature `(exn -> exn * 'a) -> 'b -> 'c'. To achieve that this is actually a typed PPX, which runs a slightly patched version of the OCaml typechecker, then untype it and pass to the OCaml compiler. This implies a couple of things: • it needs to be executed as `(staged_pps ppx_let_locs)' on dune • versions must be explicitly supported, for now only 4.10 and 4.12 are supported, if you need any other version feel free to request. • it can be quite slow on some codebases To solve the above problems, an important property is that, assuming the backtraced version behaves identically to the non backtraced one, it is in theory a noop, as removing or adding the PPX should not change the behavior of the code. So it can be used only for development or for production, as adding or removing the PPX will not break your code. Async ╌╌╌╌╌ Currently it seems like Async doesn't provide anything similar to `Lwt.backtrace_bind' and `Async.try_with' also looses the stack trace, so I would love help in this area. Examples ╌╌╌╌╌╌╌╌ *from:* <https://aws1.discourse-cdn.com/standard11/uploads/ocaml/original/2X/1/14332ae62e5262a9d4e960e6f7324145b56713f7.png> **to:** <https://aws1.discourse-cdn.com/standard11/uploads/ocaml/original/2X/d/d67eaf5bb0fbb94a84f602de9bd02580f350ff68.png> As you can see, on the second stack trace you can find out that it was actually being called on `test_interpretation.ml'. Converting typing information from one OCaml version to a later one? ════════════════════════════════════════════════════════════════════ Archive: <https://discuss.ocaml.org/t/converting-typing-information-from-one-ocaml-version-to-a-later-one/7608/6> Deep in this thread, EduardoRFS announced ───────────────────────────────────────── For me reading the `cmi' is enough, so I implemented it on `ocaml-migrate-types', it can do 411 -> 412 types, will implement 410 -> 411, which is probably painful because of the Uid <https://github.com/EduardoRFS/ocaml-migrate-types/blob/main/src/migrate_411_412.ml> Set up OCaml 1.1.10 ═══════════════════ Archive: <https://discuss.ocaml.org/t/ann-set-up-ocaml-1-1-10/7667/1> Sora Morimoto announced ─────────────────────── Changed ╌╌╌╌╌╌╌ • Run `brew update' before set up to avoid an issue with Bintray transition period. <https://github.com/avsm/setup-ocaml/releases/tag/v1.1.10> Old CWN ═══════ If you happen to miss a CWN, you can [send me a message] and I'll mail it to you, or go take a look at [the archive] or the [RSS feed of the archives]. If you also wish to receive it every week by mail, you may subscribe [online]. [Alan Schmitt] [send me a message] <mailto:alan.schm...@polytechnique.org> [the archive] <https://alan.petitepomme.net/cwn/> [RSS feed of the archives] <https://alan.petitepomme.net/cwn/cwn.rss> [online] <http://lists.idyll.org/listinfo/caml-news-weekly/> [Alan Schmitt] <https://alan.petitepomme.net/>
_______________________________________________ caml-news-weekly mailing list caml-news-weekly@lists.idyll.org http://lists.idyll.org/listinfo/caml-news-weekly