On Mon, 10 Jun 2024 at 16:27, Manos Pitsidianakis <manos.pitsidiana...@linaro.org> wrote: > > On Mon, 10 Jun 2024 22:59, Stefan Hajnoczi <stefa...@gmail.com> wrote: > >> What are the issues with not using the compiler, rustc, directly? > >> ----------------------------------------------------------------- > >> [whataretheissueswith] Back to [TOC] > >> > >> 1. Tooling > >> Mostly writing up the build-sys tooling to do so. Ideally we'd > >> compile everything without cargo but rustc directly. > > > >Why would that be ideal? > > It remove the indirection level of meson<->cargo<->rustc. I don't have a > concrete idea on how to tackle this, but if cargo ends up not strictly > necessary, I don't see why we cannot use one build system.
The convenience of being able to use cargo dependencies without special QEMU meson build system effort seems worth the overhead of meson<->cargo<->rustc to me. There is a blog post that explores using cargo crates using meson's wrap dependencies here, and it seems like extra work: https://coaxion.net/blog/2023/04/building-a-gstreamer-plugin-in-rust-with-meson-instead-of-cargo/ It's possible to use just meson today, but I don't think it's practical when using cargo dependencies. > > > > >> > >> If we decide we need Rust's `std` library support, we could > >> investigate whether building it from scratch is a good solution. This > >> will only build the bits we need in our devices. > > > >Whether or not to use std is a fundamental decision. It might be > >difficult to back from std later on. This is something that should be > >discussed in more detail. > > > >Do you want to avoid std for maximum flexibility in the future, or are > >there QEMU use cases today where std is unavailable? > > For flexibility, and for being compatible with more versions. > > But I do not want to avoid it, what I am saying is we can do a custom > build of it instead of linking to the rust toolchain's prebuilt version. What advantages does a custom build of std bring? > > > > >> > >> 2. Rust dependencies > >> We could go without them completely. I chose deliberately to include > >> one dependency in my UART implementation, `bilge`[0], because it has > >> an elegant way of representing typed bitfields for the UART's > >> registers. > >> > >> [0]: Article: https://hecatia-elegua.github.io/blog/no-more-bit-fiddling/ > >> Crates.io page: https://crates.io/crates/bilge > >> Repository: https://github.com/hecatia-elegua/bilge > > > >I guess there will be interest in using rust-vmm crates in some way. > > > >Bindings to platform features that are not available in core or std > >will also be desirable. We probably don't want to reinvent them. > > > Agreed. > > > > >> > >> Should QEMU use third-party dependencies? > >> ----------------------------------------- > >> [shouldqemuusethirdparty] Back to [TOC] > >> > >> In my personal opinion, if we need a dependency we need a strong > >> argument for it. A dependency needs a trusted upstream source, a QEMU > >> maintainer to make sure it us up-to-date in QEMU etc. > >> > >> We already fetch some projects with meson subprojects, so this is not a > >> new reality. Cargo allows you to define "locked" dependencies which is > >> the same as only fetching specific commits by SHA. No suspicious > >> tarballs, and no disappearing dependencies a la left-pad in npm. > >> > >> However, I believe it's worth considering vendoring every dependency by > >> default, if they prove to be few, for the sake of having a local QEMU > >> git clone buildable without network access. > > > >Do you mean vendoring by committing them to qemu.git or just the > >practice of running `cargo vendor` locally for users who decide they > >want to keep a copy of the dependencies? > > > Committing, with an option to opt-out. They are generally not big in > size. I am not of strong opinion on this one, I'm very open to > alternatives. Fedora and Debian want Rust applications to use distro-packaged crates. No vendoring and no crates.io online access. It's a bit of a pain because Rust developers need to make sure their code works with whatever version of crates Fedora and Debian provide. The `cargo vendor` command makes it easy for anyone wishing to collect the required dependencies for offline builds (something I've used for CentOS builds where vendoring is allowed). I suggest not vendoring packages in qemu.git. Users can still run `cargo vendor` for easy offline builds. > > > >> > >> Should QEMU provide wrapping Rust APIs over QEMU internals? > >> ----------------------------------------------------------- > >> [qemuprovidewrappingrustapis] Back to [TOC] > >> > >> My personal opinion is no, with the reasoning being that QEMU internals > >> are not documented or stable. However I do not see why creating stable > >> opt-in interfaces is bad. It just needs someone to volunteer to maintain > >> it and ensure there are no breakages through versions. > > > >Rust code will need to interface with QEMU's C APIs, so Rust wrappers > >seem unavoidable. Using a protocol like vhost-user might be possible > >in some cases. It separates the two codebases so they can both be > >native and without bindings, but that won't work for all parts of the > >QEMU source tree. > > > >Stable APIs aren't necessary if most developers in the QEMU community > >are willing to work in both languages. They can adjust both C and Rust > >code when making changes to APIs. I find this preferable to having > >Rust maintainers whose job is to keep wrappers up-to-date. Those Rust > >maintainers would probably burn out. This seems like a question of > >which approach the developer community is comfortable with. > > > Me too. > > > > >> > >> Will QEMU now depend on Rust and thus not build on my XYZ platform? > >> ------------------------------------------------------------------- > >> [qemudependonrustnotbuildonxyz] Back to [TOC] > >> > >> No, worry about this in some years if this experiment takes off. Rust > >> has broad platform support and is present in most distro package > >> managers. In the future we might have gcc support for it as well. > >> > >> For now, Rust will have an experimental status, and will be aimed to > >> those who wish to try it. I leave it to the project leaders to make > >> proper decisions and statements on this if necessary. > > > >This can be discussed in a separate email thread if you prefer, but I > >do think it needs agreement soon so that people have the confidence to > >invest their time in writing Rust. They need to know that the code > >they develop will be available on most platforms where QEMU is > >available and that others in the community won't object or insist on a > >C implementation for platform support reasons. > > Definitely, also it's out of scope for this RFC since we're not writing > and rules/guidelines yet. > > > > >> > >> > >> How is the compilation structured? > >> ================================== > >> [howisthecompilationstructured] Back to [TOC] > >> > >> First, a meson target that runs `bindgen` on a bunch of header files > >> (defined in `rust/wrapper.h`) is created as a target and as a dependency > >> for any rust hardware device that needs it. You can see the generated > >> bindings by running > >> > >> ninja generated.rs > >> > >> inside your build directory. > >> > >> The devices are defined as dictionaries in rust/meson.build because they > >> depend on the bindgen dependency, which is available much later in the > >> meson process (when the static qemu lib and target emulator executables > >> are defined). > >> > >> A cargo wrapper python script under scripts/ exists to build the crate > >> library, by providing the path to the generated.rs bindings via the > >> environment. Then, the qemu-system-aarch64 binary links against the > >> staticlib archive (i.e. libpl011.a) > >> > >> The generated.rs rust file includes a bunch of junk definitions? > >> ================================================================ > >> [generatedrsincludesjunk] Back to [TOC] > >> > >> Yes, bindgen allows you to block certain types and identifiers from > >> being generated but they are simply too many. I have trimmed some of the > >> fat but vast improvements can be made. > >> > >> The staticlib artifact contains a bunch of mangled .o objects? > >> ============================================================== > >> [staticlibmangledobjects] Back to [TOC] > >> > >> Yes, until we compile without the `std` module library or we compile it > >> manually instead of linking it, we will have some junk in it. > > > >What is the consequence of this? As long as the linker is bringing in > >.o files from the .a only through symbol dependencies, then unused .o > >files in the .a won't be linked into the final QEMU binary. > > No consequence, I just want to warn anyone peeking into the rust output > (not the final qemu binary) to expect junk. Okay, cool!
