On 6/10/20 3:50 PM, Julien Grall wrote: > On 10/06/2020 09:13, Oleksandr Andrushchenko wrote: >> Hello, > > Hi, > >> While adding support for para-virtualized block device in u-boot >> >> I faced an issue with HYPERVISOR_console_io + CONSOLEIO_write. >> >> I tried to use the hypercall during MMU setup stage while enabling data cache >> >> on arm64 platform (e.g. data cache is not yet enabled) and saw in guest's >> console >> >> either old data or some sort of garbage. Printing constant strings, just >> like mini-os >> >> does on boot, works as expected. > > Per the hypercall ABI (see include/public/arch-arm.h) all the buffers must > reside in memory which is mapped as Normal Inner Write-Back Inner-Shareable. > > You are passing non-cacheable memory, so the behavior you see is expected. This is what we come up with as well > >> So, looking at the Xen code [1] it seems >> >> that we should copy guest's buffer with COPY_flush_dcache set in this case > > COPY_flush_dcache is only meant to be used when copy to guest memory to make > sure the data reached the point of coherency in your system. It is not meant > to be used when copying from guest. Understood and agree > >> as (usually?) this hypercall is used in guest's code which doesn't have any >> other means to print yet, e.g. early printk case. > > I have been using it after the MMU is setup The thing is that in u-boot a lot of things happen before the MMU setup... > but before the console is properly setup by the guest (there is a lot that > can happen in Linux between the two). In my case, the flush is not necessary > and would be wrong. ... and without the flush you lose the ability to debug that code. > > In general, the cache is never off on Arm64 because you may have system cache > or, in the cache of virtualization, cacheable mapping in the hypervisor (all > the memory is mapped). > > When updating data with MMU/D-cache off the normal approach is: > 1) Remove any dirty lines from the cache, otherwise they may get evicted > while updating the memory and override any value written with MMU off. > 2) Update the memory > 3) Remove any lines that may have been speculated so when you turn onthe > MMU and D-cache, U-boot can obverse the correct data. > > Step 1 is only necessary if you are going to write outside of the loaded > Image (BSS is not part of it). > > You most likely already have similar steps in U-boot for other part of the > memory access with MMU/D-Cache off. So I think U-boot is the best place to > invalidate the cache before issuing the hypercall.
Yes, I have invalidated the buffers and everything works like a charm now, thanks Probably my use-case should be somewhere documented, so another time someone steps into similar issues it is explained. > > Cheers, > Thank you for helping, Oleksandr
