On 7/10/2026 2:26 PM, Ziyang Zhang wrote: > Document the dlcall plugin under Example Plugins: what it does, the trusted- > guests and guest_base == 0 constraints, how to load it, and a pointer to > Lorelei, one end-to-end userspace implementation, for the toolchain and a > runnable example. > > Co-authored-by: Kailiang Xu <[email protected]> > Co-authored-by: Mingyuan Xia <[email protected]> > Signed-off-by: Ziyang Zhang <[email protected]> > --- > docs/about/emulation.rst | 156 +++++++++++++++++++++++++++++++++++++++ > 1 file changed, 156 insertions(+) > > diff --git a/docs/about/emulation.rst b/docs/about/emulation.rst > index 3b4c365933..4f5ad1b3c6 100644 > --- a/docs/about/emulation.rst > +++ b/docs/about/emulation.rst > @@ -1046,6 +1046,162 @@ Count traps > This plugin counts the number of interrupts (asynchronous events), exceptions > (synchronous events) and host calls (e.g. semihosting) per cpu. > > +Dynamic Linking Call > +.................... > + > +``contrib/plugins/dlcall.c`` > + > +This plugin provides a dynamic linking function call interception mechanism > +for linux-user guests: the guest hands a call off to the host, where the > plugin > +runs native code in its place instead of the guest emulating it. Interception > +alone enables several uses, for instance tracing or auditing guest calls. > +One use is acceleration by leveraging the host's native shared libraries. For > +example, a thunk layer can run the stock zlib ``minizip`` utility under > +emulation while forwarding its ``deflate`` calls to the host's native zlib > +library (libz). This avoids emulating those selected library calls > instruction > +by instruction. > + > +The guest issues a reserved "magic" system call (4096 by default, > configurable > +with ``syscall_num=N``) whose first argument selects a pass-through > operation: > +dlopen/dlclose a host library, dlsym a symbol, and invoke a resolved host > +function. The plugin performs the operation on the host and consumes the > +syscall, so the real kernel never sees it. > + > +.. warning:: > + > + Trusted guests only. The guest can load arbitrary host libraries and run > + arbitrary code in the QEMU host process. The plugin is not a sandbox and > + provides no isolation. It also requires ``guest_base == 0`` (qemu-user's > + default), as guest pointers are dereferenced as host addresses with no > + translation. > + > +The plugin intentionally keeps the QEMU side lightweight and knows nothing > +about any particular library or its calling convention. Turning a real > library > +into working thunks, including argument marshalling, callbacks and variadic > +functions, is done entirely in userspace, and any toolchain can implement the > +interface. > + > +Loading the plugin is all that is required from QEMU's side: > + > +.. code-block:: shell > + > + qemu-x86_64 -plugin contrib/plugins/libdlcall.so <guest-program> ... > + > +`Lorelei <https://github.com/rover2024/lorelei>`_ is one end-to-end userspace > +implementation of this: it provides the guest and host runtimes and an > +automated toolchain that generates the thunks from a library's headers, so > guest > +library calls run on the host's native libraries. It supports an x86_64 guest > +running on an x86_64, aarch64 or riscv64 host. > + > +A minimal end-to-end example uses a one-function library, ``libhello.so``, > built > +two ways: the guest build tags its output ``(from the guest)`` and the host > +build ``(from the host)``. An unmodified guest program ``main`` calls > +``hello("World", 7)``, and the thunk makes that same binary reach the host > build > +in place of its own. The sources live under ``src/``: > + > +.. code-block:: c > + > + /* src/hello.h */ > + void hello(const char *name, int lucky); > + > +.. code-block:: c > + > + /* src/hello_guest.c */ > + #include "hello.h" > + #include <stdio.h> > + > + void hello(const char *name, int lucky) > + { > + printf("Hello, %s! Your lucky number is %d. (from the guest)\n", > name, lucky); > + } > + > +.. code-block:: c > + > + /* src/hello_host.c */ > + #include "hello.h" > + #include <stdio.h> > + > + void hello(const char *name, int lucky) > + { > + printf("Hello, %s! Your lucky number is %d. (from the host)\n", name, > lucky); > + } > + > +.. code-block:: c > + > + /* src/main.c */ > + #include "hello.h" > + > + int main(void) > + { > + hello("World", 7); > + return 0; > + } > + > +Lorelei ships a prebuilt toolchain (a "devkit") in its releases. Download the > +one for your host and unpack it: > + > +.. code-block:: shell > + > + # for an aarch64 host; see https://github.com/rover2024/lorelei/releases > + ARCH=aarch64 > + TAG=$(curl -fsSL -o /dev/null -w '%{url_effective}' \ > + https://github.com/rover2024/lorelei/releases/latest | sed > 's|.*/tag/||') > + wget > "https://github.com/rover2024/lorelei/releases/download/$TAG/lorelei-devkit-$ARCH-${TAG#v}.tar.xz"; > + tar -xf lorelei-devkit-$ARCH-*.tar.xz > + DEVKIT=lorelei-devkit-$ARCH > + > +Build the guest ``libhello.so`` (x86_64) and the host ``libhello.so`` (this > +host's own architecture), then the guest program: > + > +.. code-block:: shell > + > + mkdir -p build/guest build/host > + $DEVKIT/bin/x86_64-linux-gnu-clang -shared -fPIC src/hello_guest.c -o > build/guest/libhello.so > + cc -shared -fPIC src/hello_host.c -o build/host/libhello.so > + $DEVKIT/bin/x86_64-linux-gnu-clang src/main.c -Isrc -Lbuild/guest -lhello > -o build/guest/main > + > +Run it under qemu: > + > +.. code-block:: shell > + > + qemu-x86_64 -E LD_LIBRARY_PATH=build/guest build/guest/main > + > +which prints:: > + > + Hello, World! Your lucky number is 7. (from the guest) > + > +Now generate the thunk from the host ``libhello.so``. This produces a > guest-side > +``libhello.so`` that stands in for the guest build, and a host-side thunk > library > +that dispatches to the host build: > + > +.. code-block:: shell > + > + $DEVKIT/bin/LoreMakeThunk.py --name hello --lib build/host/libhello.so \ > + --header hello.h -o thunks -- -Isrc > + > +Run the same ``main`` under the plugin. The call reaches the host build now: > + > +.. code-block:: shell > + > + LD_LIBRARY_PATH=$DEVKIT/lib:build/host \ > + qemu-x86_64 -plugin contrib/plugins/libdlcall.so \ > + -E LD_LIBRARY_PATH=$DEVKIT/x86_64/lib:thunks/x86_64 \ > + build/guest/main > + > +which prints:: > + > + Hello, World! Your lucky number is 7. (from the host) > + > +.. list-table:: Dynamic Linking Call arguments > + :widths: 20 80 > + :header-rows: 1 > + > + * - Option > + - Description > + * - syscall_num=N > + - The magic syscall number the guest issues (default 4096). Must be high > + enough not to clash with a real syscall. > + > Other emulation features > ------------------------ >
Great work, the workflow is now ready! A few small issues are left with lib dependencies. libstdc++ --------- LD_LIBRARY_PATH=$DEVKIT/lib:build/host \ ./build/qemu-x86_64 -plugin ./build/contrib/plugins/libdlcall.so \ -E LD_LIBRARY_PATH=$DEVKIT/x86_64/lib:thunks/x86_64 \ -L /usr/x86_64-linux-gnu/ \ build/guest/main build/guest/main: error while loading shared libraries: libstdc++.so.6: cannot open shared object file: No such file or directory Indeed, libstc++ is not available in machine's x86_64 sysroot by default. It works with -L $DEVKIT/x86_64/sysroot but as mentioned previously, we don't want to use another sysroot than the one that user has, since it can contain extra libraries we are not aware about (or Lorelei one can bring extra dependencies that we don't want). So the last thing needed is to link statically libstdc++ for guest thunk and host thunk. In addition, you can do it for lorelei .so (I mean .so included in devkit), so thunking is fully self contained and does not require extra dependencies from the system where it runs. Same for libgcc. Link parameters you need are -static-libstdc++ -static-libgcc. Once added, all thunks and lorelei devkit should be portable between different machines 👍. The only base dependency we can assume will be present is libc (which also includes libm.so). We could argue if c++ should be considered as a base dependency also. For debian (+ubuntu), fedora and archlinux, it's not. See what is included in debian libc package. For instance, in our case: https://packages.debian.org/trixie/all/libc6-amd64-cross/filelist doc --- While at it, please add -L /usr/x86_64-linux-gnu/ to command line so it matches the default people will use. rpath ----- It seems that .so in thunks have rpath set to $ORIGIN:$ORIGIN/../../lib, which should not be needed. LD_LIBRARY_PATH will specify where to find the correct libraries, please remove it. You can use this command to double check this: readelf -d thunks/libhello_HTL.so thunks/x86_64/libhello.so No RUNPATH should be present. thunks/libhello_HTL.so has an explicit dependency to build/host/libhello.so. It should not include build/host and just link to libhello.so. Remove rpath also. thunks/x86_64/libhello.so, remove rpath. To clarify, lorelei-devkit-*/*/lib/* can use rpath since devkit itself is self contained. Thunks can not. ----- Once solved, we'll have a clean and minimal workflow, which ensure thunks do not require any extra dependency. With this, any user will be able to grab a lorelei devkit + a thunk tarball and use it as a drag and drop replacement for their system libraries, as long as it matches their version of course. Keep up the good work! Pierrick
