On Wed, Nov 09, 2016 at 12:48:09PM +0100, Andrew Jones wrote: > On Wed, Nov 09, 2016 at 11:37:35AM +0000, Daniel P. Berrange wrote: > > On Wed, Nov 09, 2016 at 12:26:17PM +0100, Laszlo Ersek wrote: > > > On 11/09/16 11:40, Andrew Jones wrote: > > > > On Wed, Nov 09, 2016 at 11:01:46AM +0800, Dave Young wrote: > > > >> Hi, > > > >> > > > >> Latest linux kernel enabled kaslr to randomiz phys/virt memory > > > >> addresses, we had some effort to support kexec/kdump so that crash > > > >> utility can still works in case crashed kernel has kaslr enabled. > > > >> > > > >> But according to Dave Anderson virsh dump does not work, quoted > > > >> messages > > > >> from Dave below: > > > >> > > > >> """ > > > >> with virsh dump, there's no way of even knowing that KASLR > > > >> has randomized the kernel __START_KERNEL_map region, because there is > > > >> no > > > >> virtual address information -- e.g., like "SYMBOL(_stext)" in the kdump > > > >> vmcoreinfo data to compare against the vmlinux file symbol value. > > > >> Unless virsh dump can export some basic virtual memory data, which > > > >> they say it can't, I don't see how KASLR can ever be supported. > > > >> """ > > > >> > > > >> I assume virsh dump is using qemu guest memory dump facility so it > > > >> should be first addressed in qemu. Thus post this query to qemu devel > > > >> list. If this is not correct please let me know. > > > >> > > > >> Could you qemu dump people make it work? Or we can not support virt > > > >> dump > > > >> as long as KASLR being enabled. Latest Fedora kernel has enabled it in > > > >> x86_64. > > > >> > > > > > > > > When the -kernel command line option is used, then it may be possible > > > > to extract some information that could be used to supplement the memory > > > > dump that dump-guest-memory provides. However, that would be a specific > > > > use. In general, QEMU knows nothing about the guest kernel. It doesn't > > > > know where it is in the disk image, and it doesn't even know if it's > > > > Linux. > > > > > > > > Is there anything a guest userspace application could probe from e.g. > > > > /proc that would work? If so, then the guest agent could gain a new > > > > feature providing that. > > > > > > I fully agree. This is exactly what I suggested too, independently, in > > > the downstream thread, before arriving at this upstream thread. Let me > > > quote that email: > > > > > > On 11/09/16 12:09, Laszlo Ersek wrote: > > > > [...] the dump-guest-memory QEMU command supports an option called > > > > "paging". Here's its documentation, from the "qapi-schema.json" source > > > > file: > > > > > > > >> # @paging: if true, do paging to get guest's memory mapping. This > > > >> allows > > > >> # using gdb to process the core file. > > > >> # > > > >> # IMPORTANT: this option can make QEMU allocate several > > > >> gigabytes > > > >> # of RAM. This can happen for a large guest, or a > > > >> # malicious guest pretending to be large. > > > >> # > > > >> # Also, paging=true has the following limitations: > > > >> # > > > >> # 1. The guest may be in a catastrophic state or can have > > > >> corrupted > > > >> # memory, which cannot be trusted > > > >> # 2. The guest can be in real-mode even if paging is > > > >> enabled. For > > > >> # example, the guest uses ACPI to sleep, and ACPI sleep > > > >> state > > > >> # goes in real-mode > > > >> # 3. Currently only supported on i386 and x86_64. > > > >> # > > > > > > > > "virsh dump --memory-only" sets paging=false, for obvious reasons. > > > > > > > > [...] the dump-guest-memory command provides a raw snapshot of the > > > > virtual machine's memory (and of the registers of the VCPUs); it is > > > > not enlightened about the guest. > > > > > > > > If the additional information you are looking for can be retrieved > > > > within the running Linux guest, using an appropriately privieleged > > > > userspace process, then I would recommend considering an extension to > > > > the qemu guest agent. The management layer (libvirt, [...]) could > > > > first invoke the guest agent (a process with root privileges running > > > > in the guest) from the host side, through virtio-serial. The new guest > > > > agent command would return the information necessary to deal with > > > > KASLR. Then the management layer would initiate the dump like always. > > > > Finally, the extra information would be combined with (or placed > > > > beside) the dump file in some way. > > > > > > > > So, this proposal would affect the guest agent and the management > > > > layer (= libvirt). > > > > > > Given that we already dislike "paging=true", enlightening > > > dump-guest-memory with even more guest-specific insight is the wrong > > > approach, IMO. That kind of knowledge belongs to the guest agent. > > > > If you're trying to debug a hung/panicked guest, then using a guest > > agent to fetch info is a complete non-starter as it'll be dead. > > So don't wait. Management software can make this query immediately > after the guest agent goes live. The information needed won't change.
That doesn't help with trying to diagnose a crash during boot up, since the guest agent isn't running till fairly late. I'm also concerned that the QEMU guest agent is likely to be far from widely deployed in guests, so reliance on the guest agent will mean the dump facility is no longer reliably available. Regards, Daniel -- |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://entangle-photo.org -o- http://search.cpan.org/~danberr/ :|
