I'm quite new to the world of Racket, and this made for some nice reading! If I understand it correctly, this will be quite a massive change. I'm sure a lot of folks, including myself will be happy to pitch in whichever way we can (documentation, testing, coding, etc.) in this process!
On Wednesday, February 15, 2017 at 8:37:19 PM UTC+5:30, mflatt wrote: > > TL;DR: I expect the main Racket distribution to run on Chez Scheme > instead of the current Racket VM sometime in the next couple of years. > > > Background > ---------- > > The core Racket implementation relies on a lot of C code, and that's > a problem for maintenance, for porting it to new platforms (say, > JavaScript), and for improving performance. > > My long-term plan has been to rewrite it all in Racket (except for > things like GC). Last summer, I started by rewriting Racket's macro > and module system from scratch. Although I was able to get a fresh > implementation working, the new implementation's performance was not > good enough to drop in as a replacement for our current > implementation. > > After making sure that the new expander is really working with data > structures and algorithms that are at least as good as the old ones, > the new implementation is still off by up to a factor of two in time > and 25% or so in space. I'm convinced that Racket's compiler and > runtime system will have to be better to support a reimplementation of > the core in Racket. > > Although I have specific ideas on how the current Racket VM could be > improved, there are already other compilers and runtime systems that > perform better. Notably, Chez Scheme has recently become available > under an open-source license (Apache). > > As a first experiment, I have been able to use the new Racket expander > to compile a new Racket regexp implementation to run on Chez. As > expected, Chez runs the matcher about 2 times as fast as Racket, > putting it generally on par with the C implementation that's currently > built into Racket. > > So far, so good. After more experiments along these lines, I see no > scenario where it's easier or more effective to improve Racket than to > build on Chez. Although there are some mismatches between the primitive > features that Chez and Racket provide, it will mostly work; in rare > cases where there's no other solution, we can contribute pull requests > to Chez. > > A Racket on Chez won't be compatible with the current Racket VM at the > level of the C API. For that reason and others, I expect the current > Racket VM to stick around for projects that use the C API, that need > to embed Racket, or so on. The goal of rewriting Racket in Racket has > always been to support multiple VMs, so there's no conflict with > keeping the current Racket VM, too. Other VM efforts, like Pycket and > RacketScript, should also benefit. Still, I expect mainline Racket > development to move to Chez as the primary backend. > > This shift in direction will affect many projects planned to use or > improve Racket. Although I expect to implement most of the conversion > myself, I've talked with others who are invested in the current Racket > direction. So far, everyone seems to be on board. > > Rewriting the Racket core has always seemed prohibitively difficult > and time-consuming, but I think now's the time, especially with the > expander and module system out of the way. The other subsystems look > easy compared to the part that is done, but there's plenty of room for > surprises. > > > Status > ------ > > I've reimplemented the Racket expander, reader, and regexp matcher in > Racket, and I've started on the port and filesystem path subsystems. > The new subsystem implementations have been developed in the "linklet" > branch here: > > https://github.com/mflatt/racket/tree/linklet > > See "expander" and "regexp" in the "pkgs" directory, for example. I > expect this branch to eventually move to a branch or repo in the > "racket" account on GitHub. > > Here's my work-in-progress toward making Racket reimplementations run > on Chez: > > https://github.com/mflatt/not-a-box/ > > For example, you can run the regexp implementation in Chez and compare > to Racket's using its built-in regexp implemenation in C or to Racket > running the new regexp implementation. > > > Plan > ---- > > My immediate plan is to make Racket's reader + expander + module > system run on Chez. It could be two weeks or two months. From there, > I'll continue rewriting all of the Racket subsystems in Racket, > compiling them down to layers of Chez. See "README.txt" in the > "not-a-box" repo for a status report and thoughts on various Racket > subsystems. > > When the current Racket VM becomes the legacy Racket VM, I expect that > it will retain most of its C implementation, which means that it will > keep its performance for backward compatibility. Even this legacy VM, > however, will use the new reader, expander, and module system --- > essentially as the linklet branch is now. The expander performs well > enough to be practical without affecting the runtime performance of > the programs that it expands. > > Currently, I see no obstacle to getting everything running on Chez by > the end of the year, but we'll see how it goes. > > -- You received this message because you are subscribed to the Google Groups "Racket Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/racket-dev/1dbd8dce-f198-4b47-bc07-887dcb1ab3e7%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
