"Aneesh Kumar K.V" <aneesh.ku...@linux.ibm.com> writes: > David Gibson <da...@gibson.dropbear.id.au> writes: > >> On Mon, Mar 14, 2022 at 07:10:10PM -0300, Fabiano Rosas wrote: >>> David Gibson <da...@gibson.dropbear.id.au> writes: >>> >>> > On Tue, Mar 08, 2022 at 10:23:59PM -0300, Fabiano Rosas wrote: >>> > > ... > >>> To satisfy TCG we could keep a spapr capability as ON and usually the >>> guest would pass cap-gtse=off when running with KVM. However this >>> doesn't work because this crash happens precisely because the nested >>> guest doesn't know that it needs to use cap-rpt-invalidate=on. Another >>> cap wouldn't help. >>> >>> So I think the only way to have a spapr capability for this is if TCG >>> always defaults to ON and KVM always defaults to OFF. But then we would >>> be changing guest visible behaviour depending on host properties. >> >> Ok, I'd forgotten we already have cap-rpt-invalidate. It still >> defaults to OFF for now, which might help us. >> >> What's clear is that we should never disable GTSE if >> cap-rpt-invalidate is off - qemu should enforce that before even >> starting the guest if at all possible. >> >> What's less clear to me is if we want to enable GTSE by default or >> not, in the cases where we're able to choose. Would always disabling >> GTSE when cap-rpt-invalidate=on be ok? Or do we want to be able to >> control GTSE separately. In that case we might need a second cap, but >> it would need inverted sense, so e.g. cap-disable-gtse. > > > GTSE and cap-rpt-invalidate can be looked at as independent such that we > can do GTSE=1 or GTSE=0 with cap-rpt-invalidate=on. But GTSE=0 with > cap-rpt-invalidate=off is not allowed/possible. GTSE value is what is > negotiated via CAS so we should let the hypervisor inform the guest whether it > can do GTSE 0 or 1. The challenge IIUC is Qemu always assumed GTSE=1 > which is not true in the case of nested virt where L1 guest that is booted > with GTSE=0. > > with cap-disable-gtse how would one interpret that? Whether hypervisor > have the capability to disable gtse?
The spapr capability would mean "disable GTSE if KVM allows it". Although I'd prefer using cap-gtse=<on/off> because it gives us more flexibility if we ever want to change the default value. On the KVM side I am testing a KVM_CAP_PPC_GTSE_DISABLE with the semantics of "whether QEMU is allowed to disable GTSE". It reports the inverse of MMU_FTR_GTSE. So if L1 runs with GTSE=0, then the capability returns 1 and therefore QEMU can disable GTSE. If the capability is not present, then QEMU is not allowed to disable GTSE. With David's idea of disallowing cap-rpt-invalidate=off,cap-gtse=off we can simply deny the nested guest command line if it doesn't include cap-rpt-invalidate=on when KVM L1 reports KVM_CAP_PPC_GTSE_DISABLE. That way cap-gtse can default to ON to keep TCG working. On a first look, I think the above works. I'm still running some tests with different QEMU/kernel versions.