Hi Paolo,
That is correct.
On RETF(with 32bit operand) when we are in long mode(LMA=1), the operand
size(32) should be popped from the RSP even if it points at a 64bit
address. Furthermore, `get_sp_mask()` tries to calculate the mask of
RSP, based on attributes of SS segment, which is incorrect when we are
in long mode(AMD manual 4.5.3). Maybe the fix should be the correct
calculation of sp_mask in `get_sp_mask()` while we are in long mode.
I've seen the use of setting sp_mask equal to -1 in the same function,
when it does `if (env->hflags & HF_CS64_MASK) sa.sp_mask = -1;` before
calling SET_ESP(sa.sp, sa.sp_mask);. And I thought that I should just
use that.
The other two bugs have the same root cause.
Best,
Nick
On 11/9/2026 8:56 PM, Paolo Bonzini wrote:
On 10/26/26 00:18, NickSampanis wrote:
Found the bug while debugging a kernel using qemu-tcg where a retf with
a 32-bit operand size executed at an RSP above 4 GiB in long mode,
leading
to a triple fault. This led me to investigate the retf handler of
qemu-tcg, and subsequently find two additional bugs in lcall.
This series fixes three distinct cases where 64-bit stack pointers (RSP)
pointing above 4 GiB were incorrectly truncated to 16-bit or 32-bit
values
during far control-flow instructions in 64-bit/long mode:
target/i386/tcg: Fix 64-bit RSP truncation on retf in 64-bit mode
target/i386/tcg: Fix 64-bit RSP truncation on far call in 64-bit
mode
target/i386/tcg: Fix 64-bit RSP truncation on far call through a
call gate
For the first two patches, I used the HF_CS64_MASK flag of env->hflags
based on the similar check in the retf handler that sets the sp_mask
to -1.
https://github.com/qemu/qemu/blob/d73149283205/target/i386/tcg/seg_helper.c#L2207
For the third patch (far call through a call gate), patch 3 checks
shift == 2 instead, since the call-gate path increments shift under
EFER.LMA, so shift == 2 for any valid 64-bit gate regardless of the
caller's mode.
https://github.com/qemu/qemu/blob/d73149283205/target/i386/tcg/seg_helper.c#L1834
Proof-of-Concept & Testing:
I wrote a proof of concept for each bug. Each PoC is a 512-byte boot
sector that sets up a basic 4-level long mode page table mapping the
first 2 MiB of memory 1:1, maps a 2 MiB page at the 4 GiB virtual
address,
points RSP above 4 GiB, and executes the instruction. The PoC source
files are available on GitHub Gist:
1) 64-bit RSP truncation on retf in 64-bit mode PoC
https://gist.github.com/NickSampanis/3d67093b5b99a5e9627912e2f8f4b297
2) 64-bit RSP truncation on far call in 64-bit mode PoC
https://gist.github.com/NickSampanis/d4dea9948a34fbdedeed91d77586ff05
3) 64-bit RSP truncation on far call through a call gate PoC
https://gist.github.com/NickSampanis/7e9c6a9c994ef67a1aea2b13814be441
Interesting. And really complicated... The gist is that in 64-bit
mode, the operation size only governs how many bytes are popped, not
the stack size. And for far CALL, in addition, your patch only
touches the same-stack case.
Is that correct?
Paolo