On Wed, 30 Aug 2023, Anthony Chan wrote: > On Tue, 29 Aug 2023, Stefano Stabellini wrote: > > On Tue, 29 Aug 2023, Anthony Chan wrote: > > > Hi all, > > > > > > My name is Tony and I've been researching/developing using Xen for > > potential upcoming uses in our embedded systems. I started with Xen > > using Xilinx tools about a year ago and still have lots to learn about what > > it > > can to do in the embedded space. So far, I've managed to integrate Xen > > and Linux into an existing product that exclusively runs bare-metal code on > > a ZynqMP SoC and migrate some of the functionality into custom Linux > > driver/userspace. > > > > > > I'm now looking at low power support, for now at least between Xen > > (4.16) and Linux (5.15) dom0. I've tried a few different Linux kernel > > configs around power management and each time I try to suspend from > > linux dom0 (via sysfs or systemctl), Xen will watchdog on dom0 guest. > > AFAIK, Xen should trap on a 'WFI' from guests, but from what I can tell > > debugging through the linux suspend process is it's spinning in a 'suspend- > > to-idle' loop before it can get to issuing a 'WFI' or using PSCI interface > > to > > notify Xen. I'm beginning to suspect that 'low power' support for > > embedded arm64 just isn't quite there yet, or am I missing something in > > the configs? > > > > > > I realize this could very well be a Linux 'issue' but checking here > > > first. I > > know Xen presents a flattened device tree to Linux without CPU idle-state > > nodes and maybe this is causing the linux guest to only do the suspend- > > to-idle mode? I should mention that I'm booting up using dom0less > > feature if that matters. > > > > > > Hi Anthony, > > > > Assuming you are using the default Xen command line parameters for > > Xilinx boards: sched=null vwfi=native, then if the guest uses WFI, the CPU > > will execute WFI directly and go into low power mode. > Yes, using these command line params. > > > Given the issue you are describing, I am suspecting the guest is not issuing > > WFI: that is simple and known to work. Instead, I suspect that Linux might > > be trying to use PSCI_suspend in a way that is not supported or well- > > implemented by Xen. > > > > Can you check? You can add a printk in Linux > > drivers/firmware/psci/psci.c:__psci_cpu_suspend or in Xen > > xen/arch/arm/vpsci.c:do_psci_0_2_cpu_suspend > Instrumented both places it doesn't appear to reach there. In > kernel/power/suspend.c, there's a call to s2idle_loop that it's currently > 'stuck' in and I think it doesn't get to the psci suspend your referring till > afterwards, when suspend_ops->enter is called. Unfortunately, without any > idle-states nodes in the FDT, the only suspend state Linux is defaults to is > 'suspend to idle'. The fact that Linux uses "suspend to idle" is not a problem because as I mentioned WFI or PSCI_suspent are not different on Xen. That part is OK.
However, if the issue is not PSCI_suspend then I don't have another easy guess. Please post a full stack trace or more information about the error in Linux and I might be able to see where it is coming from.