Hi Samuel,
On 03/04/2026 12:54, Samuel Thibault wrote:
Michael Kelly, le ven. 03 avril 2026 09:32:48 +0100, a ecrit:
It seems to be related to the physical address map strategy of Qemu (see
attached note from Qemu's pc_piix.c). This layout starts to alter once the
physical memory total exceeds 3.5G. Any system with less memory than this
has all of it in the physical address range 0..0xe0000000. Once 3.5G is
exceeded, the default layout is then 0..0xc0000000 and the remaining memory
starts at 0x100000000. The q35 layout is from 0..0x7fffffff then
0x100000000.
So possibly somehow the physical address gets truncated at some
point inside rumpdisk. Probably inside rump some drivers may not
support addresses beyond 32b. In principle, the vm_allocate_contiguous
inside rumpdisk allocates memory only inside 32b, but possibly that
fails. Possibly we are passing to the driver the physical address of the
RPC data buffer, which can be beyond 32b, and then we need a bounce
buffer. All these should be checked to see where we miss 64b support.
Your suggestions were very relevant. I had expected that the rumpkernel
would have allocated DMA space as a bounce buffer that was later
transferred to the user buffer after the DMA operation. That is not the
case because the physical address that ends up being referenced in the
DMA request is that of the page that was allocated by
rumpdisk_device_read (vm_address_t buf) using vm_allocate(). Once memory
exceeds 3.5G we know that the HIGHMEM segment is created and is used for
all normal user page allocations. The physical address of buf now
exceeds the 32 bit range and, like you said, the address will be
truncated in the DMA request.
I did modify gnumach to not use the HIGHMEM segment for any allocations
and unsurprisingly Hurd then boots using i440fx with both 4G and 8G RAM.
I'm confident that a solution to the rumdisk_device_read/write could be
implemented using bounce buffers allocated using vm_allocate_contiguous.
There is however another problem which might be more tricky. When
rumpdisk first opens the device it calls rump_sys_open(). Deep within
that a call is made to readdisklabel() which allocates some memory and
makes several DMA requests. That memory is also allocated from the
HIGHMEM segment and so the DMA request fails in the same way. It's not
clear to me how the allocation of that memory could be altered. I don't
really know how the uvm_km_alloc() is translated for rumpkernel but I
think it is something based on libc.
Regarding, q35, perhaps the SATA drivers do use a bounce buffer and that
is why there are no problem for Hurd on q35. I've not researched all the
details. There is only so much time...
A very simple (theoretically anyway) work around that would solve these
difficulties is to prevent rumpdisk from allocating pages outside the 32
bit space. There'd need to be some kind of privileged system call to
specify acceptable physical memory ranges for vm_page_alloc_pa() on a
per task basis. You could limit rumpdisk to 32 bit ranges whilst devices
were being opened but then at other times permit normal page allocation
and use bounce buffers. All a bit of a hack though I think.
Any other ideas?
Cheers,
Mike.
