Hi, This is a design RFC, not a patch series. It follows review of
[PATCH 00/12] single-binary: link ARM and RISC-V into qemu-system https://patchew.org/QEMU/[email protected]/ [PATCH v2 00/13] single-binary: link ARM and RISC-V into qemu-system https://patchew.org/QEMU/[email protected]/ and Daniel P. Berrange's measurements of that v1. What v2 already did =================== v2 of that series already takes the v1 review that got maintainer support: - Keep real qemu-system-$TARGET (and $TARGETw on Windows). v1 installed per-arch names as symlinks to qemu-system; v2 does not. qemu-system is an extra binary from the ARM+RISC-V union, behind a meson option. - argv[0] still selects the target for qemu-system-aarch64. - Drop v1 little- and big-endian TargetInfo variants (target-info-<name>-le/be, -target aarch64-le). One TargetInfo per target_name. Endian variants are later work. - -target ? lists target_name only. Unknown names point at -target ? rather than printing the list and exit(1). - Do not build qemu-systemw for the combined binary, so Windows data exports (qdev_prop_array / qdev_prop_*) do not block enabling modules later. - TargetCpuOps (b444b4fb) for QMP CPU queries, dump, and ARM/RISC-V semihosting, still as one process-wide table filtered at register time. This RFC revises that into a map. v2 still required -target on bare qemu-system, still used TYPE_TARGET_SPECIFIC on the combined binary, and did not mark qemu-system experimental. Those are what this RFC asks to freeze before posting the next series: extra binary stays, filtering is not a gate, proposal A vs B. Heterogeneous (no target: on -M) stays an error until later. Why this RFC now ================ We are very close to a linked qemu-system that contains more than one target (ARM + RISC-V; see v2 above). v2 already dropped the symlink. The remaining delay is not "can it link", it is unresolved product questions: -target vs -M only, complete filtering before the first binary exists, experimental union vs looking like $TARGET. This RFC is to freeze those questions so the next series can land an experimental qemu-system quickly, instead of another patch bomb that still debates the end result. Filtering, heterogeneous machines, modules for RSS, and CLI polish can follow against a real binary. Please comment on the split and on proposal A vs B; that is what unblocks the merge. Thesis ====== Land an extra, experimental qemu-system first, without completing target/device filtering. Use that binary as the test vehicle for filtering. Keep today's qemu-system-$TARGET as real binaries, not symlinks. qemu.git has no combined qemu-system today. Filtering, uniquify, and CPU-op dispatch are therefore untested assumptions. Completing TYPE_TARGET_SPECIFIC against qemu-system-riscv64 (a single-target link) does not prove the union case. We need a real qemu-system in the tree before filtering can be designed against measured behavior. Berrange compared a v1 qemu-system-riscv64 *symlink* to today's standalone binary: - qom-list-types went from about 770 classes to about 1530 (ARM types leak into RISC-V) - -device help grew - RSS for -nodefaults -display none -M none went from 17.3 MB to 20.7 MB - A full-union guess was much worse for RISC-V That is a packaging blocker if $TARGET becomes a symlink. Extra RAM on a *new* experimental qemu-system is acceptable if we document that this binary exposes the union. It is also not the last word on RSS. Once modules are always enabled for qemu-system, unused machines and devices can live in .so / .dll / .dylib and stay out of the startup working set. One simple shape is one module per target, so qemu-system -M arm:virt (or -target aarch64) loads the ARM module and does not map RISC-V boards. That is later work, not a blocker for the first extra binary (Mark Cave-Ayland already suggested mandatory modules for qemu-system; Berrange did not want to block on that). The first merge still links the union into one executable so we have something to test. This split is meant to take both review positions at the layer they apply to: - Berrange / Paolo: do not replace qemu-system-$TARGET with a symlink. A new qemu-system can be its own thing. - Pierrick: select TargetInfo when the user wants a single-target view; TYPE_TARGET_SPECIFIC as the filtering mechanism. Matching $TARGET introspection is the *goal* of a later filtering series, not a gate on the first link. Two CLI proposals for how to select one TargetInfo are below. They share argv[0] and -M target:machine. They differ on whether -target exists. A run with no selected TargetInfo (heterogeneous / SYS_EMU_TARGET_HETEROGENEOUS) is deferred: the first series only prints an error. experimental-qemu-system ======================== Keep building real qemu-system-$TARGET as today (and $TARGETw on Windows). Add qemu-system alongside them, from the union of ARM and RISC-V for the first pair, behind a meson option, marked experimental. Do not install per-arch names as symlinks to qemu-system. Distro packaging of qemu-system-riscv64 stays on the old RSS and QOM list. The combined binary is documented as exposing everything that was linked. Incomplete device filtering is an accepted experimental defect, not a release blocker for this first binary. -M help / -device help with no selected TargetInfo may still list the union. Starting a machine that way is not supported yet (error; see below). Binding TargetInfo ================== A combined process has more than one TargetInfo (target_name, page bits, long width, CPU_RESOLVING_TYPE, endianness). In this mail, single-target means the process has one selected TargetInfo / SysEmuTarget, like qemu-system-$TARGET today. Heterogeneous would mean it does not (SYS_EMU_TARGET_HETEROGENEOUS). That mode is not implemented yet. Single-target runs still pick one TargetInfo before the chosen machine and accelerators initialize. Shared in both proposals: - argv[0]: qemu-system-aarch64 infers aarch64. Existing $TARGET command lines stay valid. Single-target. -M virt works as today. - -M arm:virt on bare qemu-system: single-target. The target: prefix selects TargetInfo arm and machine virt together. The token is target_name (arm, aarch64, riscv32, riscv64). Properties stay after the machine name: -M aarch64:virt,gic-version=3. - Bare qemu-system and -M has no target: prefix: not supported yet. Fail with an error that this would be a heterogeneous machine (SYS_EMU_TARGET_HETEROGENEOUS), which is deferred, and point at -M arm:virt and (proposal A) -target. Do not start the VM. -M help / -device help with no selected TargetInfo may still list the union so users can pick a qualified name. If argv[0] and a target: prefix are both given, they must match. Unknown prefix names fail. -M help may also print qualified names (arm:virt, riscv64:virt) so users can opt into a single-target run. Proposal A: -target plus -M target:machine ------------------------------------------ Pierrick suggested -target. Keep it as an optional way to select TargetInfo on the combined binary. It is not mandatory on qemu-system. - -target aarch64: single-target. Later filtering can make this look like qemu-system-aarch64. -M virt works as today. - If both -target and a target: prefix are given, they must match. Unknown -target names fail and point at -target ?. qemu-system -M help qemu-system -M virt # error: heterogeneous not supported qemu-system -M arm:virt ... qemu-system -target aarch64 -M virt ... qemu-system-riscv64 -M virt ... Proposal B: target only in -M or argv[0] (alternative) ----------------------------------------------------- Do not add -target. The target is always embedded in the machine argument, or inferred from argv[0]. - qemu-system-aarch64 -M virt: single-target from argv[0]. - qemu-system -M arm:virt: single-target from -M. - qemu-system -M virt: error (heterogeneous not supported yet). There is no global option that selects TargetInfo before -M. Early option parsing still has to read -M (and argv[0]) before machines initialize, the same as proposal A reads -target. qemu-system -M help qemu-system -M virt # error: heterogeneous not supported qemu-system -M arm:virt ... qemu-system -M aarch64:virt,gic-version=3 ... qemu-system-riscv64 -M virt ... Compared with A: one place to write the target on the combined binary (-M), no new flag, no -target vs -M mismatch cases. The cost is that qemu-system -M virt is never a hidden alias for aarch64; colliding names must be qualified, and a single-target view of the union binary is qemu-system -M aarch64:virt, not qemu-system -target aarch64 -M virt. Please pick A or B; the rest of this RFC is the same either way. SYS_EMU_TARGET_HETEROGENEOUS (deferred) -------------------------------------- QAPI SysEmuTarget today is the configure list (aarch64, arm, riscv64, ...). A later process with no selected TargetInfo and no target: on -M would need a value that is not a single linked target: add heterogeneous (C name SYS_EMU_TARGET_HETEROGENEOUS). It would be a process mode, not a ./configure target and not qemu-system-heterogeneous. The first series does not add that enum and does not run that mode. qemu-system -M virt with no argv[0] target and no -target fails with an error: heterogeneous machines are not supported yet; use -M arm:virt (and -target in proposal A). How such a machine would name per-CPU / per-device targets is later work. target-info-def.c ================= The minimum compile unit for several targets in one process is target-info-def.c built once per target (COMPILING_PER_TARGET). Each translation unit registers one TargetInfo and a TYPE_TARGET_INFO QOM class. meson already does that for qemu-system-$TARGET. Combined qemu-system extracts those objects from every participating target, plus arch objects from a primary per base architecture (aarch64 and riscv64 for the first pair). That is enough to have several TargetInfo values. Selecting one TargetInfo (-M arm:virt, argv[0], and -target in proposal A) picks one value for the run. With none of those, the first series errors out; it does not invent SYS_EMU_TARGET_HETEROGENEOUS. That is not enough to look like a single-target binary. That is the point of the first merge. TYPE_TARGET_SPECIFIC ==================== Pierrick's series "single-binary: implement dynamic filtering for machine types" is a prerequisite and is already upstream. For a usable combined binary we need it at least for machines: without is_available(), -M virt in a union link is ambiguous or can start the wrong board against the selected TargetInfo. Help and introspection depend on whether TargetInfo is already selected: - No selected TargetInfo and no target: on -M: -M help / -device help may list the union (qualified names). Starting a machine is an error (heterogeneous not supported yet). - Selected TargetInfo (argv[0], -M arm:virt, or -target in proposal A): filter with is_available() as today. -M virt is unambiguous once TargetInfo is selected. We do not need to finish filtering every QOM class before the first qemu-system exists. Machine-level TYPE_TARGET_SPECIFIC (already wired for ARM and RISC-V boards) is enough to select a board after TargetInfo is selected. Device and class leakage with -target is what Berrange measured; on experimental qemu-system that is accepted until filtering-against-qemu-system. Without a selected TargetInfo and without a target: prefix on -M, do not run; error. New TYPE_TARGET_SPECIFIC annotations in the first series should be limited to what the combined link needs. Not a campaign to tag every class. TargetCpuOps must be a map ========================== A combined link cannot keep one global C symbol per architecture for query-cpu-definitions, dump notes, or Angel semihosting. Commit b444b4fb6ecb ("target-info: dispatch CPU QMP, dump, and arm/riscv semihosting") moved those handlers onto TargetCpuOps and TARGET_INFO_CPU_OP (one member at a time, type_init). That direction is required. The implementation is not complete. It uses a single process-wide static TargetCpuOps target_cpu_ops; and target_info_register_cpu_op() stores a member only when cpu_type matches the already selected TargetInfo. Other architectures are dropped. That depends on MODULE_INIT_QOM running after target_info_qom_set_target(). It is the same class of assumption we cannot test until a combined binary exists, and it is the wrong shape even for qemu-system-*: one global filled or ignored by init order, instead of a table keyed by the selected target. Proposal: keep TARGET_INFO_CPU_OP. Replace the single global with a map from the selected target to its TargetCpuOps. Lookup uses target_arch() (SysEmuTarget): target_info_cpu_ops() -> &map[target_arch()] Registration always fills that architecture's slot. It does not compare against the currently selected TargetInfo and drop the rest. qemu-system-aarch64 has one occupied slot. qemu-system has one slot per linked target. Same API, same callers in hw/core and semihosting. Help paths with no selected TargetInfo (qemu-system -M help) list the union and must not dispatch CPU QMP / dump / semihosting through a single-arch ops table. A VM start still needs argv[0], -M arm:virt, or -target (proposal A). There is no map[SYS_EMU_TARGET_HETEROGENEOUS] until that mode exists. CPU_RESOLVING_TYPE is not a unique map key. TYPE_ARM_CPU is shared by arm and aarch64, TYPE_RISCV_CPU by riscv32 and riscv64, TYPE_X86_CPU by i386 and x86_64. Those siblings share one object file in the combined link (the primary softmmu target). Registration fills the compiling TARGET_ARCH, then aliases the same TargetCpuOps to other SysEmuTarget values whose TargetInfo.cpu_type matches. Lookup is still map[target_arch()]. What the first series is not ============================ - Completing Kconfig / target / device filtering - Implementing a heterogeneous machine. First series: error only. SYS_EMU_TARGET_HETEROGENEOUS and how it names per-CPU / per-device targets come later. - Mandatory modules / splitting machines and devices (or a whole target) into .so / .dll / .dylib. That is the RSS follow-up once qemu-system exists, not a gate on linking it. - A new experimental CLI (-nodefaults default, no default machine, dropping -hda / -drive, ordered args) - Replacing qemu-system-$TARGET with a symlink Symbol uniquify (QOM names, helpers, TYPE_ACCEL_CPU) remains necessary to *link*. That is engineering, not a change in user semantics. filtering-against-qemu-system ============================= Once qemu-system exists in the tree, filtering can be implemented and tested for real: qemu-system -M riscv64:virt ... qemu-system -target riscv64 ... # proposal A only qemu-system-riscv64 ... That is the series that should drive qom-list-types, -device help, and RSS toward the single-target numbers Berrange measured, for the single-target path. Modules (always on for qemu-system, unused targets not mapped) are the other RSS tool, after the binary exists. Distros keep shipping qemu-system-$TARGET until they opt into the combined binary. The no-selected-TargetInfo path is an error until heterogeneous support exists. Filtering must not make qemu-system -M help hide other arches. Please comment on this split, and on proposal A vs B. We are close enough that a decision here is what lets the next series land experimental qemu-system instead of waiting on filtering. -- 此致 礼 罗勇刚 Yours sincerely, Yonggang Luo
