On Feb 03 10:01, Peter Maydell wrote: > On Wed, 3 Feb 2021 at 04:01, Aaron Lindsay <aa...@os.amperecomputing.com> > wrote: > > Note that I'm using qemu-system-aarch64 with -loadvm. > > You don't say what machine type and command line you're using. Strictly, > Strictly speaking, the intended guarantee only covers versioned > machines, eg "virt-4.1" on QEMU 4.1 to "virt-4.1" on QEMU 5.2.
Sorry for the omission, `-M virt` was used to capture the snapshots on 4.1.0 and I'm using `-M virt-4.1` to restore it on 5.2.0. You get a fairly helpful error message about that one if you screw it up: "qemu-system-aarch64: Machine type received is 'virt-4.1' and local is 'virt-5.2'" I'm restoring using the following command-line on 5.2.0. Snapshots were taken with the same, except `-M virt` and without `-loadvm` on 4.1.0: $ qemu-system-aarch64 -smp 1 -M virt-4.1 -cpu max -m 8G -icount 0 -nographic -semihosting -kernel ./Image -append 'console=ttyAMA0 root=/dev/vda3 rw earlycon=smh' -drive if=none,id=image,file=rootfs.qcow2,format=qcow2 -device virtio-blk-pci,drive=image -loadvm "snapshot-name" I suspect that some of the differences I raised in this section: > > These hit errors detected in `write_list_to_cpustate` in > > target/arm/helper.c. may be from the fact that I'm running on -max, which understandably may cause changes in the ID registers between QEMU versions. I'm less concerned about issues related to my usage of -max than I am about the others. > > The third is that meanings of the bits in env->features (as defined by > > `enum arm_features` in target/arm/cpu.h) has shifted. For example, > > ARM_FEATURE_PXN, ARM_FEATURE_CRC, ARM_FEATURE_VFP, ARM_FEATURE_VFP3, > > ARM_FEATURE_VFP4 have all been removed and ARM_FEATURE_V8_1M has been > > added since 4.1.0. Heck, even I have added a field there in the past. > > Unfortunately, these additions/removals mean that when env->features is > > saved on one version and restored on another the bits can mean different > > things. Notably, the removal of the *VFP features means that a snapshot > > of a CPU reporting it supports ARM_FEATURE_VFP3 on 4.1.0 thinks it's now > > ARM_FEATURE_M on 5.2.0! > > Ow. I didn't realize the env->features was in the migration state :-( > There is no reason for it to be, because it's a constant property > of the CPU. The easy fix is to replace > VMSTATE_UINT64(env.features, ARMCPU), > in target/arm/machine.c with whatever the syntax is for "ignore > 64 bits of data here". Then we'll ignore whatever is coming in > from the source, which we don't need, and we'll stop sending it > out if we're the destination. I'll look into this. -Aaron