Re: [Qemu-devel] [RFC][PATCH 09/15] introduce a new monitor command 'dump' to dump guest's memory

2012-01-30 Thread Wen Congyang 
At 01/31/2012 01:19 AM, Eric Blake Wrote:
> On 01/29/2012 10:36 PM, Wen Congyang wrote:
 +++ b/hmp-commands.hx
 @@ -828,6 +828,22 @@ new parameters (if specified) once the vm migration 
 finished successfully.
  ETEXI
  
  {
 +.name   = "dump",
 +.args_type  = "file:s",
 +.params = "file",
 +.help   = "dump to file",
 +.user_print = monitor_user_noop,
 +.mhandler.cmd = hmp_dump,
 +},
>>>
>>> What if I want to dump only a fraction of the memory?  I think you need
>>> optional start and length parameters, to limit how much memory to be
>>> dumped, rather than forcing me to dump all memory at once.
>>>
>>
>> It is OK to support it, but I do not know why do you want it?
>>
>> The purpose of this command is dumping the memory when the guest is paniced.
>> And then we can use crash/gdb(or other application) to investigate why the 
>> guest
>> is paniced. So we should dump the whole memory.
> 
> That's one purpose, but not the only purpose.  We shouldn't be
> artificially constraining things into requiring the entire memory region
> in order to use this command.
> 
> Libvirt provides virDomainMemoryPeek which currently wraps the 'memsave'
> and 'pmemsave' monitor commands, but these commands output raw memory.
> Your command is introducing a new memory format into ELF images, and if
> 'memsave' can already do a subset of memory, it also makes sense for
> 'dump' to do a subset when creating the ELF image.  That is, if a
> management app every has a reason to access a subset of memory, then
> this reason exists whether the subset is raw or ELF formatted when
> presented to the management app.

OK. I know why you want it, and will support it. Please wait for some
days.

Thanks
Wen Congyang

> 
> Meanwhile, on the libvirt side, the virDomainMemoryPeek API to
> management apps is constrained - it sends the data inline with the
> command, rather than on a side channel.  Someday, I'd like to enhance
> libvirt to have a dump-to-stream command, and reuse the existing libvirt
> ability to stream large amounts of data on side channels, in order to
> let management apps directly and atomically query a subset of memory
> into a file with the desired formatting, rather than the current
> approach of constraining the management app to only query 64k at a time
> and to have to manually pause the guest if they need to atomically
> inspect more memory.
>

Re: [Qemu-devel] [RFC][PATCH 09/15] introduce a new monitor command 'dump' to dump guest's memory

2012-01-30 Thread Eric Blake 
On 01/29/2012 10:36 PM, Wen Congyang wrote:
>>> +++ b/hmp-commands.hx
>>> @@ -828,6 +828,22 @@ new parameters (if specified) once the vm migration 
>>> finished successfully.
>>>  ETEXI
>>>  
>>>  {
>>> +.name   = "dump",
>>> +.args_type  = "file:s",
>>> +.params = "file",
>>> +.help   = "dump to file",
>>> +.user_print = monitor_user_noop,
>>> +.mhandler.cmd = hmp_dump,
>>> +},
>>
>> What if I want to dump only a fraction of the memory?  I think you need
>> optional start and length parameters, to limit how much memory to be
>> dumped, rather than forcing me to dump all memory at once.
>>
> 
> It is OK to support it, but I do not know why do you want it?
> 
> The purpose of this command is dumping the memory when the guest is paniced.
> And then we can use crash/gdb(or other application) to investigate why the 
> guest
> is paniced. So we should dump the whole memory.

That's one purpose, but not the only purpose.  We shouldn't be
artificially constraining things into requiring the entire memory region
in order to use this command.

Libvirt provides virDomainMemoryPeek which currently wraps the 'memsave'
and 'pmemsave' monitor commands, but these commands output raw memory.
Your command is introducing a new memory format into ELF images, and if
'memsave' can already do a subset of memory, it also makes sense for
'dump' to do a subset when creating the ELF image.  That is, if a
management app every has a reason to access a subset of memory, then
this reason exists whether the subset is raw or ELF formatted when
presented to the management app.

Meanwhile, on the libvirt side, the virDomainMemoryPeek API to
management apps is constrained - it sends the data inline with the
command, rather than on a side channel.  Someday, I'd like to enhance
libvirt to have a dump-to-stream command, and reuse the existing libvirt
ability to stream large amounts of data on side channels, in order to
let management apps directly and atomically query a subset of memory
into a file with the desired formatting, rather than the current
approach of constraining the management app to only query 64k at a time
and to have to manually pause the guest if they need to atomically
inspect more memory.

-- 
Eric Blake   ebl...@redhat.com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [RFC][PATCH 09/15] introduce a new monitor command 'dump' to dump guest's memory

2012-01-29 Thread Wen Congyang 
At 01/20/2012 12:32 AM, Eric Blake Wrote:
> On 01/18/2012 08:07 PM, Wen Congyang wrote:
>> Signed-off-by: Wen Congyang 
>> ---
>>  Makefile.target  |8 +-
>>  dump.c   |  590 
>> ++
>>  dump.h   |3 +
>>  hmp-commands.hx  |   16 ++
>>  hmp.c|9 +
>>  hmp.h|1 +
>>  monitor.c|3 +
>>  qapi-schema.json |   13 ++
>>  qmp-commands.hx  |   26 +++
>>  9 files changed, 665 insertions(+), 4 deletions(-)
>>  create mode 100644 dump.c
>>
> 
>> +void qmp_dump(const char *file, Error **errp)
>> +{
>> +const char *p;
>> +int fd = -1;
>> +DumpState *s;
>> +
>> +#if !defined(WIN32)
>> +if (strstart(file, "fd:", &p)) {
>> +fd = qemu_get_fd(p);
>> +if (fd == -1) {
>> +error_set(errp, QERR_FD_NOT_FOUND, p);
>> +return;
>> +}
>> +}
>> +#endif
> 
> Thanks for implementing fd support off the bat.
> 
>> +
>> +if  (strstart(file, "file:", &p)) {
>> +fd = open(p, O_WRONLY | O_CREAT | O_TRUNC | O_BINARY);
> 
> Use of O_CREAT requires that you pass a third argument to open()
> specifying the mode_t to use.

Yes, I forgot it, and will fix it.

> 
>> +++ b/hmp-commands.hx
>> @@ -828,6 +828,22 @@ new parameters (if specified) once the vm migration 
>> finished successfully.
>>  ETEXI
>>  
>>  {
>> +.name   = "dump",
>> +.args_type  = "file:s",
>> +.params = "file",
>> +.help   = "dump to file",
>> +.user_print = monitor_user_noop,
>> +.mhandler.cmd = hmp_dump,
>> +},
> 
> What if I want to dump only a fraction of the memory?  I think you need
> optional start and length parameters, to limit how much memory to be
> dumped, rather than forcing me to dump all memory at once.
> 

It is OK to support it, but I do not know why do you want it?

The purpose of this command is dumping the memory when the guest is paniced.
And then we can use crash/gdb(or other application) to investigate why the guest
is paniced. So we should dump the whole memory.

Thanks
Wen Congyang

Re: [Qemu-devel] [RFC][PATCH 09/15] introduce a new monitor command 'dump' to dump guest's memory

2012-01-19 Thread Eric Blake 
On 01/18/2012 08:07 PM, Wen Congyang wrote:
> Signed-off-by: Wen Congyang 
> ---
>  Makefile.target  |8 +-
>  dump.c   |  590 
> ++
>  dump.h   |3 +
>  hmp-commands.hx  |   16 ++
>  hmp.c|9 +
>  hmp.h|1 +
>  monitor.c|3 +
>  qapi-schema.json |   13 ++
>  qmp-commands.hx  |   26 +++
>  9 files changed, 665 insertions(+), 4 deletions(-)
>  create mode 100644 dump.c
> 

> +void qmp_dump(const char *file, Error **errp)
> +{
> +const char *p;
> +int fd = -1;
> +DumpState *s;
> +
> +#if !defined(WIN32)
> +if (strstart(file, "fd:", &p)) {
> +fd = qemu_get_fd(p);
> +if (fd == -1) {
> +error_set(errp, QERR_FD_NOT_FOUND, p);
> +return;
> +}
> +}
> +#endif

Thanks for implementing fd support off the bat.

> +
> +if  (strstart(file, "file:", &p)) {
> +fd = open(p, O_WRONLY | O_CREAT | O_TRUNC | O_BINARY);

Use of O_CREAT requires that you pass a third argument to open()
specifying the mode_t to use.

> +++ b/hmp-commands.hx
> @@ -828,6 +828,22 @@ new parameters (if specified) once the vm migration 
> finished successfully.
>  ETEXI
>  
>  {
> +.name   = "dump",
> +.args_type  = "file:s",
> +.params = "file",
> +.help   = "dump to file",
> +.user_print = monitor_user_noop,
> +.mhandler.cmd = hmp_dump,
> +},

What if I want to dump only a fraction of the memory?  I think you need
optional start and length parameters, to limit how much memory to be
dumped, rather than forcing me to dump all memory at once.

-- 
Eric Blake   ebl...@redhat.com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

[Qemu-devel] [RFC][PATCH 09/15] introduce a new monitor command 'dump' to dump guest's memory

2012-01-18 Thread Wen Congyang 
Signed-off-by: Wen Congyang 
---
 Makefile.target  |8 +-
 dump.c   |  590 ++
 dump.h   |3 +
 hmp-commands.hx  |   16 ++
 hmp.c|9 +
 hmp.h|1 +
 monitor.c|3 +
 qapi-schema.json |   13 ++
 qmp-commands.hx  |   26 +++
 9 files changed, 665 insertions(+), 4 deletions(-)
 create mode 100644 dump.c

diff --git a/Makefile.target b/Makefile.target
index d869550..abfb057 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -110,7 +110,7 @@ $(call set-vpath, 
$(SRC_PATH)/linux-user:$(SRC_PATH)/linux-user/$(TARGET_ABI_DIR
 QEMU_CFLAGS+=-I$(SRC_PATH)/linux-user/$(TARGET_ABI_DIR) 
-I$(SRC_PATH)/linux-user
 obj-y = main.o syscall.o strace.o mmap.o signal.o thunk.o \
   elfload.o linuxload.o uaccess.o gdbstub.o cpu-uname.o \
-  user-exec.o $(oslib-obj-y)
+  user-exec.o $(oslib-obj-y) dump.o
 
 obj-$(TARGET_HAS_BFLT) += flatload.o
 
@@ -148,7 +148,7 @@ LDFLAGS+=-Wl,-segaddr,__STD_PROG_ZONE,0x1000 -image_base 
0x0e00
 LIBS+=-lmx
 
 obj-y = main.o commpage.o machload.o mmap.o signal.o syscall.o thunk.o \
-gdbstub.o user-exec.o
+gdbstub.o user-exec.o dump.o
 
 obj-i386-y += ioport-user.o
 
@@ -170,7 +170,7 @@ $(call set-vpath, $(SRC_PATH)/bsd-user)
 QEMU_CFLAGS+=-I$(SRC_PATH)/bsd-user -I$(SRC_PATH)/bsd-user/$(TARGET_ARCH)
 
 obj-y = main.o bsdload.o elfload.o mmap.o signal.o strace.o syscall.o \
-gdbstub.o uaccess.o user-exec.o
+gdbstub.o uaccess.o user-exec.o dump.o
 
 obj-i386-y += ioport-user.o
 
@@ -186,7 +186,7 @@ endif #CONFIG_BSD_USER
 # System emulator target
 ifdef CONFIG_SOFTMMU
 
-obj-y = arch_init.o cpus.o monitor.o machine.o gdbstub.o balloon.o ioport.o
+obj-y = arch_init.o cpus.o monitor.o machine.o gdbstub.o balloon.o ioport.o 
dump.o
 # virtio has to be here due to weird dependency between PCI and virtio-net.
 # need to fix this properly
 obj-$(CONFIG_NO_PCI) += pci-stub.o
diff --git a/dump.c b/dump.c
new file mode 100644
index 000..2951b8b
--- /dev/null
+++ b/dump.c
@@ -0,0 +1,590 @@
+/*
+ * QEMU dump
+ *
+ * Copyright Fujitsu, Corp. 2011
+ *
+ * Authors:
+ * Wen Congyang 
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2.  See
+ * the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu-common.h"
+#include 
+#include 
+#include 
+#include 
+#include "cpu.h"
+#include "cpu-all.h"
+#include "targphys.h"
+#include "monitor.h"
+#include "kvm.h"
+#include "dump.h"
+#include "sysemu.h"
+#include "bswap.h"
+#include "memory_mapping.h"
+#include "error.h"
+#include "qmp-commands.h"
+
+#define CPU_CONVERT_TO_TARGET16(val) \
+({ \
+uint16_t _val = (val); \
+if (endian == ELFDATA2LSB) { \
+_val = cpu_to_le16(_val); \
+} else {\
+_val = cpu_to_be16(_val); \
+} \
+_val; \
+})
+
+#define CPU_CONVERT_TO_TARGET32(val) \
+({ \
+uint32_t _val = (val); \
+if (endian == ELFDATA2LSB) { \
+_val = cpu_to_le32(_val); \
+} else {\
+_val = cpu_to_be32(_val); \
+} \
+_val; \
+})
+
+#define CPU_CONVERT_TO_TARGET64(val) \
+({ \
+uint64_t _val = (val); \
+if (endian == ELFDATA2LSB) { \
+_val = cpu_to_le64(_val); \
+} else {\
+_val = cpu_to_be64(_val); \
+} \
+_val; \
+})
+
+enum {
+DUMP_STATE_ERROR,
+DUMP_STATE_SETUP,
+DUMP_STATE_CANCELLED,
+DUMP_STATE_ACTIVE,
+DUMP_STATE_COMPLETED,
+};
+
+typedef struct DumpState {
+ArchDumpInfo dump_info;
+MemoryMappingList list;
+int phdr_num;
+int state;
+char *error;
+int fd;
+target_phys_addr_t memory_offset;
+} DumpState;
+
+static DumpState *dump_get_current(void)
+{
+static DumpState current_dump = {
+.state = DUMP_STATE_SETUP,
+};
+
+return ¤t_dump;
+}
+
+static int dump_cleanup(DumpState *s)
+{
+int ret = 0;
+
+free_memory_mapping_list(&s->list);
+if (s->fd != -1) {
+close(s->fd);
+s->fd = -1;
+}
+
+return ret;
+}
+
+static void dump_error(DumpState *s, const char *reason)
+{
+s->state = DUMP_STATE_ERROR;
+s->error = g_strdup(reason);
+dump_cleanup(s);
+}
+
+static inline int cpuid(CPUState *env)
+{
+#if defined(CONFIG_USER_ONLY) && defined(CONFIG_USE_NPTL)
+return env->host_tid;
+#else
+return env->cpu_index + 1;
+#endif
+}
+
+static int write_elf64_header(DumpState *s)
+{
+Elf64_Ehdr elf_header;
+int ret;
+int endian = s->dump_info.d_endian;
+
+memset(&elf_header, 0, sizeof(Elf64_Ehdr));
+memcpy(&elf_header, ELFMAG, 4);
+elf_header.e_ident[EI_CLASS] = ELFCLASS64;
+elf_header.e_ident[EI_DATA] = s->dump_info.d_endian;
+elf_header.e_ident[EI_VERSION] = EV_CURRENT;
+elf_header.e_type = CPU_CONVERT_TO_TARGET16(ET_CORE);
+elf_header.e_machine = CPU_CONVERT_TO_TARGET16(s->dump_info.d_machine);
+elf_header.e_version = CPU_CONVERT_TO_TARGET32(EV_CURRENT);
+elf_header.e_ehsize = CPU_CONVERT_TO_TARGET16(sizeof(elf_hea