On Thu, Sep 26, 2024 at 03:23:11PM +0100, Alex Bennée wrote: > During the various conversations I didn't hear anyone speak against the > proposed migration although some concerns where raised about review and > knowledge gaps.
Yep, this apparent broad acceptance (or at least tolerance) for use of Rust is a clear difference from any previous discussion about introducing new languages in QEMU n the past. > > One output from this discussion should be a clear statement that we are > going forward with this work and the road map. A rough roadmap might > look like: > > - 9.2 --enable-rust is available and developers can build with it. > rust devices have -x-device or -rust-device CLI flags for > runtime selection. > > - 10.x rust devices feature complete and migration compatible, enabled > by default when rust compiler detected. No CLI selection > required as legacy portions won't be built. Any partial > conversions should be behind --enable-prototype-rust configure > flag. > > - 11.x distros have enough infrastructure to build on supported > platforms. Rust becomes a mandatory dependency, old C versions > of converted code removed from build. > > - xx.y QEMU becomes a pure native rust program and all C is expunged. > We may never get to this point. Yeah, I think this last step is soo unlikely (or far away) that we could reasonably just not include it at all. Perhaps any future grandchildren will do this part for us ;-P > We should publish the intention and the road map prominently although it > was unclear if a blog post would be the best place vs expanding a > section in the developers manual. Perhaps both make sense with a blog > post for the statement of intent and rough timeline and the developer > manual being expanded with any new rules and standards to follow? We should include plans about Rust at the top of the release notes too for all forthcoming versions until we have it turned on permanently. > There was some concern about the missing gaps in the support matrix > especially as we support a number of "legacy" TCG backends. While *-user > support is more insulated from the effects of rust conversions due to > its relatively low set of dependencies it will still be a problem if we > convert the core CPU QOM classes to rust. > > Some of this is made simpler if we deprecate 32 bit host support? I feel like we're pretty close to wanting todo this regardless of Rust support. > What are candidates for conversion? > ----------------------------------- > > One area of discussion was what areas of the code base are likely to be > converted. This will give maintainers some idea of what to expect over > the next few cycles as we move ahead. There's a topical blog from Google about their experiance with Android that just hit LWN: https://security.googleblog.com/2024/09/eliminating-memory-safety-vulnerabilities-Android.html "vulnerabilities decay exponentially. They have a half-life. ... the density of Android’s memory safety bugs decreased with the age of the code, primarily residing in recent changes. This leads to two important takeaways: * The problem is overwhelmingly with new code, necessitating a fundamental change in how we develop code. * Code matures and gets safer with time, exponentially, making the returns on investments like rewrites diminish over time as code gets older." IOW, if we rush into conversion of existing code, we're likely to harm QEMU's quality in the short-medium term. We should thus encourage/prioritize use of Rust for *new* code, and be more cautious / targetted about converting existing code, to mitigate the risks inherant in any rewrites. > Manos' initial patch series [1] adds a pl011 serial device and there is > an intention to add a pflash device model to exercise the block layer. > It was suggested other device models such as a network card and watchdog > device would also be worth doing a conversion for to exercise other > common backends. If we want arbitrary new devices to be able to be written in Rust, we want to make sure we have the backends accessible to the Rust frontend device, for developers to consume. To be able to design & validate any Rust abstractions for backends, we need to have at least 1 Rust device consuming each different backend type eg, chardev, netdev, blockdev, tpm, rng, watchdog. Similarly for other helper code like IO channels, crypto. > Markus asked when will QAPI need rust bindings? As it is the route for > the public API into QEMU it is spread widely across the code base. While > the hand written serialisation code can likely be replaced with Rust's > serde crate it will need to interface to the internal APIs of both rust > and C modules. Probably need some kind of QAPI support sooner than we think, given how widely it spreads. > One issue that came up is how we handle adequately reviewing code when > most of the maintainers are experienced C coders but might not know much > about Rust. While we want to avoid the situation of developers vetoing > conversion there should be communication ahead of any serious work to > avoid rust contributions coming out of the blue. If a maintainer feels > they cannot maintain a bunch of unfamiliar rust code the submitter > should be prepared to find people willing to become a maintainers as > unmaintained drive-by submissions are not useful for the long term > health of the project. Yep, communication is critical, if proposing to rewrite existing functionality. Drowning maintainers in conversion patches without warning is a guaranteed way to create friction between people. > With relative inexperience there was a concern we could inadvertently > introduce technical debt in the code base (C-like Rust vs Rusty rust). > What can we do to mitigate that issue? On a long enough time frame, all exiting code can be considered technical debt. Given the relatively sparse Rust experiance across our community, we're guaranteed to make more design mistakes in the first few years. Mitigating this is important, but at the same time, we should also accept we're not going to get everything perfect. One thing our community is very good at is obsessing about perfection of patch series. I can forsee us doing that to an even greater extent with any Rust conversions of code. If we are not careful we could really harm our overall productivity by spending too much time striving for a perfect Rust abstraction first time out, even though many of us would still be relative newcomers to Rust, such that we don't know what we don't know. IOW, we need to be pragmatic about accepting some level of technical debt at times. Especially if there are cases where our Rust design is held back by existing C code, we might be better off temporarily accepting sub-optimal Rust, to avoid immediately refactoring piles of C code, and tackle the problems later. > Dependencies and Packaging concerns > ----------------------------------- > > Finally there is the topic of dependencies and how that affects > packaging. One of the benefits of Rust is a large library of crates for > common dependencies. Typically the management of those crates is handled > by the cargo build tool and projects tend to regularly update their > dependencies to stay current with the state of the art. However this > cargo driven approach is at odds with a lot of distros desire to package > a single version of a library and manage its updates during the stable > lifetime of a distro. Some distros do allow exceptions for "vendoring" > dependencies as part of the build but it is generally discouraged. > Another challenge is updating versions of crates can often lead to > additional transitive (indirect) dependencies which then need to be > checked if they are supported by our distro matrix. In terms of our distro support matrix, I think we should follow the same practice as Python. ie we only check that the distro provides the required toolchain / base language, and never check the crates. IMHO vendoring is really the only viable option for distros with non-trivial applications, as IME the de-vendoring crates waaaaay too much busy-work in the distro. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
