[PATCH 2/2] tcg: Allocate sufficient storage in temp_allocate_frame
This function should have been updated for vector types
when they were introduced.
Cc: qemu-sta...@nongnu.org
Fixes: d2fd745fe8b
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/367
Signed-off-by: Richard Henderson
---
tcg/tcg.c | 32 +++-
1 file changed, 27 insertions(+), 5 deletions(-)
diff --git a/tcg/tcg.c b/tcg/tcg.c
index 52e858523c..1bad423bde 100644
--- a/tcg/tcg.c
+++ b/tcg/tcg.c
@@ -3015,17 +3015,39 @@ static void check_regs(TCGContext *s)
static void temp_allocate_frame(TCGContext *s, TCGTemp *ts)
{
-if (s->current_frame_offset + (tcg_target_long)sizeof(tcg_target_long) >
-s->frame_end) {
-tcg_abort();
+size_t size, align;
+intptr_t off;
+
+switch (ts->type) {
+case TCG_TYPE_I32:
+size = align = 4;
+break;
+case TCG_TYPE_I64:
+case TCG_TYPE_V64:
+size = align = 8;
+break;
+case TCG_TYPE_V128:
+size = align = 16;
+break;
+case TCG_TYPE_V256:
+/* Note that we do not require aligned storage for V256. */
+size = 32, align = TCG_TARGET_STACK_ALIGN;
+break;
+default:
+g_assert_not_reached();
}
-ts->mem_offset = s->current_frame_offset;
+
+assert(align <= TCG_TARGET_STACK_ALIGN);
+off = ROUND_UP(s->current_frame_offset, align);
+assert(off + size <= s->frame_end);
+s->current_frame_offset = off + size;
+
+ts->mem_offset = off;
#if defined(__sparc__)
ts->mem_offset += TCG_TARGET_STACK_BIAS;
#endif
ts->mem_base = s->frame_temp;
ts->mem_allocated = 1;
-s->current_frame_offset += sizeof(tcg_target_long);
}
static void temp_load(TCGContext *, TCGTemp *, TCGRegSet, TCGRegSet,
TCGRegSet);
--
2.25.1
[PATCH 1/2] tcg/sparc: Fix temp_allocate_frame vs sparc stack bias
We should not be aligning the offset in temp_allocate_frame,
because the odd offset produces an aligned address in the end.
Instead, pass the logical offset into tcg_set_frame and add
the stack bias last.
Cc: qemu-sta...@nongnu.org
Cc: mark.cave-ayl...@ilande.co.uk
Signed-off-by: Richard Henderson
---
tcg/tcg.c | 9 +++--
tcg/sparc/tcg-target.c.inc | 16 ++--
2 files changed, 13 insertions(+), 12 deletions(-)
diff --git a/tcg/tcg.c b/tcg/tcg.c
index dd584f3bba..52e858523c 100644
--- a/tcg/tcg.c
+++ b/tcg/tcg.c
@@ -3015,17 +3015,14 @@ static void check_regs(TCGContext *s)
static void temp_allocate_frame(TCGContext *s, TCGTemp *ts)
{
-#if !(defined(__sparc__) && TCG_TARGET_REG_BITS == 64)
-/* Sparc64 stack is accessed with offset of 2047 */
-s->current_frame_offset = (s->current_frame_offset +
- (tcg_target_long)sizeof(tcg_target_long) - 1) &
-~(sizeof(tcg_target_long) - 1);
-#endif
if (s->current_frame_offset + (tcg_target_long)sizeof(tcg_target_long) >
s->frame_end) {
tcg_abort();
}
ts->mem_offset = s->current_frame_offset;
+#if defined(__sparc__)
+ts->mem_offset += TCG_TARGET_STACK_BIAS;
+#endif
ts->mem_base = s->frame_temp;
ts->mem_allocated = 1;
s->current_frame_offset += sizeof(tcg_target_long);
diff --git a/tcg/sparc/tcg-target.c.inc b/tcg/sparc/tcg-target.c.inc
index ce39ac2d86..a6ec94a094 100644
--- a/tcg/sparc/tcg-target.c.inc
+++ b/tcg/sparc/tcg-target.c.inc
@@ -984,14 +984,18 @@ static void tcg_target_qemu_prologue(TCGContext *s)
{
int tmp_buf_size, frame_size;
-/* The TCG temp buffer is at the top of the frame, immediately
- below the frame pointer. */
+/*
+ * The TCG temp buffer is at the top of the frame, immediately
+ * below the frame pointer. Use the logical (aligned) offset here;
+ * the stack bias is applied in temp_allocate_frame().
+ */
tmp_buf_size = CPU_TEMP_BUF_NLONGS * (int)sizeof(long);
-tcg_set_frame(s, TCG_REG_I6, TCG_TARGET_STACK_BIAS - tmp_buf_size,
- tmp_buf_size);
+tcg_set_frame(s, TCG_REG_I6, -tmp_buf_size, tmp_buf_size);
-/* TCG_TARGET_CALL_STACK_OFFSET includes the stack bias, but is
- otherwise the minimal frame usable by callees. */
+/*
+ * TCG_TARGET_CALL_STACK_OFFSET includes the stack bias, but is
+ * otherwise the minimal frame usable by callees.
+ */
frame_size = TCG_TARGET_CALL_STACK_OFFSET - TCG_TARGET_STACK_BIAS;
frame_size += TCG_STATIC_CALL_ARGS_SIZE + tmp_buf_size;
frame_size += TCG_TARGET_STACK_ALIGN - 1;
--
2.25.1
[PATCH 0/2] tcg: Fixes for temp_allocate_frame
Stefan, I think this will resolve #367 for you -- please test. r~ Richard Henderson (2): tcg/sparc: Fix temp_allocate_frame vs sparc stack bias tcg: Allocate sufficient storage in temp_allocate_frame tcg/tcg.c | 41 -- tcg/sparc/tcg-target.c.inc | 16 +-- 2 files changed, 40 insertions(+), 17 deletions(-) -- 2.25.1
[Bug 1825452] Re: Pulse audio backend doesn't work in v4.0.0-rc4 release
[Expired for QEMU because there has been no activity for 60 days.]
** Changed in: qemu
Status: Incomplete => Expired
--
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1825452
Title:
Pulse audio backend doesn't work in v4.0.0-rc4 release
Status in QEMU:
Expired
Bug description:
Using Gentoo linux, build from source: qemu v4.0.0-rc4 release
(eeba63fc7fface36f438bcbc0d3b02e7dcb59983)
Pulse audio backend doesn't initialize because of the:
audio/paaudio.c:
-if (!popts->has_server) {
-char pidfile[64];
-char *runtime;
-struct stat st;
-
-runtime = getenv("XDG_RUNTIME_DIR");
-if (!runtime) {
-return NULL;
-}
-snprintf(pidfile, sizeof(pidfile), "%s/pulse/pid", runtime);
-if (stat(pidfile, ) != 0) {
-return NULL;
-}
-}
XDG_RUNTIME_DIR is not set for me. There is no /run/user directory exist in
my system.
Also:
$ less ~/.pulse/client.conf
default-server = unix:/home/ivan/.pulse_server
To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1825452/+subscriptions
Re: [PATCH 00/12] linux-user: Load a vdso for x86_64 and hppa
Patchew URL:
https://patchew.org/QEMU/20210619034329.532318-1-richard.hender...@linaro.org/
Hi,
This series seems to have some coding style problems. See output below for
more information:
Type: series
Message-id: 20210619034329.532318-1-richard.hender...@linaro.org
Subject: [PATCH 00/12] linux-user: Load a vdso for x86_64 and hppa
=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===
Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
* [new tag]
patchew/20210619034329.532318-1-richard.hender...@linaro.org ->
patchew/20210619034329.532318-1-richard.hender...@linaro.org
Switched to a new branch 'test'
eb8d6c9 linux-user/hppa: Add vdso and use it for rt_sigreturn
52cca21 linux-user/x86_64: Add vdso
fd7f59e linux-user: Add gen-vdso tool
5228b92 linux-user: Load vdso image if available
0fc0b1b linux-user: Introduce imgsrc_mmap
e397043 linux-user: Replace bprm->fd with bprm->src.fd
832f702 linux-user: Use ImageSource in load_symbols
9d65bf5 linux-user: Use ImageSource in load_elf_image
b04a74d linux-user: Do not clobber bprm_buf swapping ehdr
acdaddc linux-user: Tidy loader_exec
63442e1 linux-user: Introduce imgsrc_read, imgsrc_read_alloc
a4d2a45 linux-user: Fix style problems in linuxload.c
=== OUTPUT BEGIN ===
1/12 Checking commit a4d2a4511b66 (linux-user: Fix style problems in
linuxload.c)
2/12 Checking commit 63442e15ed05 (linux-user: Introduce imgsrc_read,
imgsrc_read_alloc)
3/12 Checking commit acdaddc5dc89 (linux-user: Tidy loader_exec)
4/12 Checking commit b04a74dee4d8 (linux-user: Do not clobber bprm_buf swapping
ehdr)
5/12 Checking commit 9d65bf56e0c7 (linux-user: Use ImageSource in
load_elf_image)
6/12 Checking commit 832f702ccea5 (linux-user: Use ImageSource in load_symbols)
7/12 Checking commit e3970430bd3b (linux-user: Replace bprm->fd with
bprm->src.fd)
8/12 Checking commit 0fc0b1bd7d79 (linux-user: Introduce imgsrc_mmap)
9/12 Checking commit 5228b920709c (linux-user: Load vdso image if available)
10/12 Checking commit fd7f59e94124 (linux-user: Add gen-vdso tool)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#21:
new file mode 100644
ERROR: trailing whitespace
#31: FILE: linux-user/gen-vdso-elfn.c.inc:6:
+ * $
ERROR: spaces required around that '*' (ctx:WxV)
#92: FILE: linux-user/gen-vdso-elfn.c.inc:67:
+static void elfN(process)(FILE *outf, void *buf, long total_len,
^
ERROR: trailing whitespace
#184: FILE: linux-user/gen-vdso-elfn.c.inc:159:
+$
ERROR: trailing whitespace
#335: FILE: linux-user/gen-vdso.c:5:
+ * $
ERROR: spaces required around that ':' (ctx:VxW)
#350: FILE: linux-user/gen-vdso.c:20:
+uint16_t: __builtin_bswap16, \
^
ERROR: spaces required around that ':' (ctx:VxW)
#351: FILE: linux-user/gen-vdso.c:21:
+uint32_t: __builtin_bswap32, \
^
ERROR: spaces required around that ':' (ctx:VxW)
#352: FILE: linux-user/gen-vdso.c:22:
+uint64_t: __builtin_bswap64, \
^
ERROR: spaces required around that ':' (ctx:VxW)
#353: FILE: linux-user/gen-vdso.c:23:
+int16_t: __builtin_bswap16,\
^
ERROR: spaces required around that ':' (ctx:VxW)
#354: FILE: linux-user/gen-vdso.c:24:
+int32_t: __builtin_bswap32,\
^
ERROR: spaces required around that ':' (ctx:VxW)
#355: FILE: linux-user/gen-vdso.c:25:
+int64_t: __builtin_bswap64)
^
WARNING: Block comments use a leading /* on a separate line
#436: FILE: linux-user/gen-vdso.c:106:
+fputs("/* Automatically generated from linux-user/gen-vdso.c. */\n"
total: 10 errors, 2 warnings, 481 lines checked
Patch 10/12 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
11/12 Checking commit 52cca2161834 (linux-user/x86_64: Add vdso)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#38:
new file mode 100644
ERROR: trailing whitespace
#70: FILE: linux-user/x86_64/vdso.S:5:
+ * $
ERROR: trailing whitespace
#198: FILE: linux-user/x86_64/vdso.ld:5:
+ * $
total: 2 errors, 1 warnings, 217 lines checked
Patch 11/12 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
12/12 Checking commit eb8d6c9db1a6 (linux-user/hppa: Add vdso and use it for
rt_sigreturn)
WARNING: added, moved or deleted file(s),
[PATCH 12/12] linux-user/hppa: Add vdso and use it for rt_sigreturn
Building the vdso itself is not actually wired up to anything, since
we require a cross-compiler. Just check in that file for now.
Drop the now-unused 9 trampoline words, and describe the frame
without the trampoline in __kernel_rt_sigreturn.
Signed-off-by: Richard Henderson
---
linux-user/elfload.c | 3 +
linux-user/hppa/signal.c | 8 +-
linux-user/hppa/Makefile.vdso | 4 +
linux-user/hppa/meson.build | 6 ++
linux-user/hppa/vdso.S| 149 ++
linux-user/hppa/vdso.ld | 75 +
linux-user/hppa/vdso.so | Bin 0 -> 5192 bytes
7 files changed, 238 insertions(+), 7 deletions(-)
create mode 100644 linux-user/hppa/Makefile.vdso
create mode 100644 linux-user/hppa/vdso.S
create mode 100644 linux-user/hppa/vdso.ld
create mode 100755 linux-user/hppa/vdso.so
diff --git a/linux-user/elfload.c b/linux-user/elfload.c
index b70a5c48a2..6cea556b70 100644
--- a/linux-user/elfload.c
+++ b/linux-user/elfload.c
@@ -1457,6 +1457,9 @@ static inline void init_thread(struct target_pt_regs
*regs,
#define STACK_GROWS_DOWN 0
#define STACK_ALIGNMENT 64
+#define HAVE_VDSO 1
+#include "vdso.c.inc"
+
static inline void init_thread(struct target_pt_regs *regs,
struct image_info *infop)
{
diff --git a/linux-user/hppa/signal.c b/linux-user/hppa/signal.c
index 0e266f472d..44e2db6d3e 100644
--- a/linux-user/hppa/signal.c
+++ b/linux-user/hppa/signal.c
@@ -40,7 +40,6 @@ struct target_ucontext {
};
struct target_rt_sigframe {
-abi_uint tramp[9];
target_siginfo_t info;
struct target_ucontext uc;
/* hidden location of upper halves of pa2.0 64-bit gregs */
@@ -138,14 +137,9 @@ void setup_rt_frame(int sig, struct target_sigaction *ka,
setup_sigcontext(>uc.tuc_mcontext, env);
-__put_user(0x3419, frame->tramp + 0); /* ldi 0,%r25 */
-__put_user(0x3414015a, frame->tramp + 1); /* ldi __NR_rt_sigreturn,%r20 */
-__put_user(0xe4008200, frame->tramp + 2); /* be,l 0x100(%sr2,%r0) */
-__put_user(0x08000240, frame->tramp + 3); /* nop */
-
unlock_user_struct(frame, frame_addr, 1);
-env->gr[2] = h2g(frame->tramp);
+env->gr[2] = default_rt_sigreturn;
env->gr[30] = sp;
env->gr[26] = sig;
env->gr[25] = h2g(>info);
diff --git a/linux-user/hppa/Makefile.vdso b/linux-user/hppa/Makefile.vdso
new file mode 100644
index 00..ce92d51a37
--- /dev/null
+++ b/linux-user/hppa/Makefile.vdso
@@ -0,0 +1,4 @@
+vdso.so: vdso.S vdso.ld Makefile.vdso
+ hppa-linux-gnu-gcc -nostdlib -shared -Wl,-T,vdso.ld \
+ -Wl,-h,linux-vdso.so.1 -Wl,--build-id=none \
+ -Wl,--hash-style=both vdso.S -o $@
diff --git a/linux-user/hppa/meson.build b/linux-user/hppa/meson.build
index 4709508a09..3febe8523a 100644
--- a/linux-user/hppa/meson.build
+++ b/linux-user/hppa/meson.build
@@ -3,3 +3,9 @@ syscall_nr_generators += {
arguments: [ meson.current_source_dir() / 'syscallhdr.sh',
'@INPUT@', '@OUTPUT@', '@EXTRA_ARGS@' ],
output: '@BASENAME@_nr.h')
}
+
+gen = [
+ gen_vdso.process('vdso.so')
+]
+
+linux_user_ss.add(when: 'TARGET_HPPA', if_true: gen)
diff --git a/linux-user/hppa/vdso.S b/linux-user/hppa/vdso.S
new file mode 100644
index 00..eeae2c999a
--- /dev/null
+++ b/linux-user/hppa/vdso.S
@@ -0,0 +1,149 @@
+/*
+ * hppa linux kernel vdso replacement.
+ *
+ * Copyright 2021 Linaro, Ltd.
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include
+
+ .text
+
+#define sizeof_rt_sigframe 696
+#define offsetof_sigcontext152
+#define offsetof_sigcontext_gr offsetof_sigcontext + 4
+#define offsetof_sigcontext_fr offsetof_sigcontext_gr + 32 * 4
+#define offsetof_sigcontext_iasq offsetof_sigcontext_fr + 32 * 8
+#define offsetof_sigcontext_iaoq offsetof_sigcontext_iasq + 8
+#define offsetof_sigcontext_saroffsetof_sigcontext_iaoq + 8
+
+ /*
+* While this frame is marked as a signal frame, that only applies
+* to how this return address is handled for the outer frame.
+* The return address that arrived here, from the inner frame, is
+* not marked as a signal frame and so the unwinder still tries to
+* subtract 1 to examine the presumed call insn. Thus we must
+* extend the unwind info to a nop before the start.
+*/
+
+ .cfi_startproc simple
+ .cfi_signal_frame
+
+ /* Compare pa32_fallback_frame_state from libgcc. */
+
+ /* Record the size of the stack frame. */
+ .cfi_def_cfa30, -sizeof_rt_sigframe
+
+ /* Record save offset of general registers. */
+ .cfi_offset 1, offsetof_sigcontext_gr + 1 * 4
+ .cfi_offset 2, offsetof_sigcontext_gr + 2 * 4
+ .cfi_offset 3, offsetof_sigcontext_gr + 3 * 4
+ .cfi_offset 4, offsetof_sigcontext_gr + 4 * 4
+ .cfi_offset 5, offsetof_sigcontext_gr + 5 * 4
+
[PATCH 05/12] linux-user: Use ImageSource in load_elf_image
Change parse_elf_properties as well, as the bprm_buf argument
ties the two functions closely.
Signed-off-by: Richard Henderson
---
linux-user/elfload.c | 124 ---
1 file changed, 47 insertions(+), 77 deletions(-)
diff --git a/linux-user/elfload.c b/linux-user/elfload.c
index 1f9a69703a..425420f0cc 100644
--- a/linux-user/elfload.c
+++ b/linux-user/elfload.c
@@ -2525,10 +2525,9 @@ static bool parse_elf_property(const uint32_t *data, int
*off, int datasz,
}
/* Process NT_GNU_PROPERTY_TYPE_0. */
-static bool parse_elf_properties(int image_fd,
+static bool parse_elf_properties(const ImageSource *src,
struct image_info *info,
const struct elf_phdr *phdr,
- char bprm_buf[BPRM_BUF_SIZE],
Error **errp)
{
union {
@@ -2556,14 +2555,8 @@ static bool parse_elf_properties(int image_fd,
return false;
}
-if (phdr->p_offset + n <= BPRM_BUF_SIZE) {
-memcpy(, bprm_buf + phdr->p_offset, n);
-} else {
-ssize_t len = pread(image_fd, , n, phdr->p_offset);
-if (len != n) {
-error_setg_errno(errp, errno, "Error reading file header");
-return false;
-}
+if (!imgsrc_read(, phdr->p_offset, n, src, errp)) {
+return false;
}
/*
@@ -2609,30 +2602,34 @@ static bool parse_elf_properties(int image_fd,
}
}
-/* Load an ELF image into the address space.
+/**
+ * load_elf_image: Load an ELF image into the address space.
+ * @image_name: the filename of the image, to use in error messages.
+ * @src: the ImageSource from which to read.
+ * @info: info collected from the loaded image.
+ * @ehdr: the ELF header, not yet bswapped.
+ * @pinterp_name: record any PT_INTERP string found.
+ *
+ * On return: @info values will be filled in, as necessary or available.
+ */
- IMAGE_NAME is the filename of the image, to use in error messages.
- IMAGE_FD is the open file descriptor for the image.
-
- BPRM_BUF is a copy of the beginning of the file; this of course
- contains the elf file header at offset 0. It is assumed that this
- buffer is sufficiently aligned to present no problems to the host
- in accessing data at aligned offsets within the buffer.
-
- On return: INFO values will be filled in, as necessary or available. */
-
-static void load_elf_image(const char *image_name, int image_fd,
+static void load_elf_image(const char *image_name, const ImageSource *src,
struct image_info *info, struct elfhdr *ehdr,
- char **pinterp_name,
- char bprm_buf[BPRM_BUF_SIZE])
+ char **pinterp_name)
{
-struct elf_phdr *phdr;
+g_autofree struct elf_phdr *phdr = NULL;
abi_ulong load_addr, load_bias, loaddr, hiaddr, error;
-int i, retval, prot_exec;
+int i, prot_exec;
Error *err = NULL;
-/* First of all, some simple consistency checks */
-memcpy(ehdr, bprm_buf, sizeof(*ehdr));
+/*
+ * First of all, some simple consistency checks.
+ * Note that we rely on the bswapped ehdr staying in bprm_buf,
+ * for later use by load_elf_binary and create_elf_tables.
+ */
+if (!imgsrc_read(ehdr, 0, sizeof(*ehdr), src, )) {
+goto exit_errmsg;
+}
if (!elf_check_ident(ehdr)) {
error_setg(, "Invalid ELF image for this architecture");
goto exit_errmsg;
@@ -2643,15 +2640,11 @@ static void load_elf_image(const char *image_name, int
image_fd,
goto exit_errmsg;
}
-i = ehdr->e_phnum * sizeof(struct elf_phdr);
-if (ehdr->e_phoff + i <= BPRM_BUF_SIZE) {
-phdr = (struct elf_phdr *)(bprm_buf + ehdr->e_phoff);
-} else {
-phdr = (struct elf_phdr *) alloca(i);
-retval = pread(image_fd, phdr, i, ehdr->e_phoff);
-if (retval != i) {
-goto exit_read;
-}
+phdr = imgsrc_read_alloc(ehdr->e_phoff,
+ ehdr->e_phnum * sizeof(struct elf_phdr),
+ src, );
+if (phdr == NULL) {
+goto exit_errmsg;
}
bswap_phdr(phdr, ehdr->e_phnum);
@@ -2687,17 +2680,10 @@ static void load_elf_image(const char *image_name, int
image_fd,
goto exit_errmsg;
}
-interp_name = g_malloc(eppnt->p_filesz);
-
-if (eppnt->p_offset + eppnt->p_filesz <= BPRM_BUF_SIZE) {
-memcpy(interp_name, bprm_buf + eppnt->p_offset,
- eppnt->p_filesz);
-} else {
-retval = pread(image_fd, interp_name, eppnt->p_filesz,
- eppnt->p_offset);
-if (retval != eppnt->p_filesz) {
-goto exit_read;
-}
+interp_name = imgsrc_read_alloc(eppnt->p_offset, eppnt->p_filesz,
+
[PATCH 11/12] linux-user/x86_64: Add vdso
Building the vdso itself is not actually wired up to anything, since
we require a cross-compiler. Just check in that file for now.
Signed-off-by: Richard Henderson
---
linux-user/elfload.c| 3 +
linux-user/x86_64/Makefile.vdso | 3 +
linux-user/x86_64/meson.build | 6 ++
linux-user/x86_64/vdso.S| 122
linux-user/x86_64/vdso.ld | 74 +++
linux-user/x86_64/vdso.so | Bin 0 -> 5912 bytes
6 files changed, 208 insertions(+)
create mode 100644 linux-user/x86_64/Makefile.vdso
create mode 100644 linux-user/x86_64/vdso.S
create mode 100644 linux-user/x86_64/vdso.ld
create mode 100755 linux-user/x86_64/vdso.so
diff --git a/linux-user/elfload.c b/linux-user/elfload.c
index a6ad454617..b70a5c48a2 100644
--- a/linux-user/elfload.c
+++ b/linux-user/elfload.c
@@ -154,6 +154,9 @@ static uint32_t get_elf_hwcap(void)
#define ELF_CLASS ELFCLASS64
#define ELF_ARCH EM_X86_64
+#define HAVE_VDSO 1
+#include "vdso.c.inc"
+
static inline void init_thread(struct target_pt_regs *regs, struct image_info
*infop)
{
regs->rax = 0;
diff --git a/linux-user/x86_64/Makefile.vdso b/linux-user/x86_64/Makefile.vdso
new file mode 100644
index 00..6111d6f21c
--- /dev/null
+++ b/linux-user/x86_64/Makefile.vdso
@@ -0,0 +1,3 @@
+vdso.so: vdso.S vdso.ld Makefile.vdso
+ $(CC) -nostdlib -shared -Wl,-T,vdso.ld -Wl,--build-id=none \
+ -Wl,-h,linux-vdso.so.1 -Wl,--hash-style=both vdso.S -o $@
diff --git a/linux-user/x86_64/meson.build b/linux-user/x86_64/meson.build
index 203af9a60c..f6a0015953 100644
--- a/linux-user/x86_64/meson.build
+++ b/linux-user/x86_64/meson.build
@@ -3,3 +3,9 @@ syscall_nr_generators += {
arguments: [ meson.current_source_dir() /
'syscallhdr.sh', '@INPUT@', '@OUTPUT@', '@EXTRA_ARGS@' ],
output: '@BASENAME@_nr.h')
}
+
+gen = [
+ gen_vdso.process('vdso.so')
+]
+
+linux_user_ss.add(when: 'TARGET_X86_64', if_true: gen)
diff --git a/linux-user/x86_64/vdso.S b/linux-user/x86_64/vdso.S
new file mode 100644
index 00..7d0d653526
--- /dev/null
+++ b/linux-user/x86_64/vdso.S
@@ -0,0 +1,122 @@
+/*
+ * x86-64 linux replacement vdso.
+ *
+ * Copyright 2021 Linaro, Ltd.
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include
+
+ .globl __vdso_clock_gettime
+ .type __vdso_clock_gettime, @function
+ .balign 16
+ .cfi_startproc
+__vdso_clock_gettime:
+ mov $__NR_clock_gettime, %eax
+ syscall
+ ret
+ .cfi_endproc
+ .size __vdso_clock_gettime, . - __vdso_clock_gettime
+
+clock_gettime = __vdso_clock_gettime
+ .weak clock_gettime
+
+ .globl __vdso_clock_getres
+ .type __vdso_clock_getres, @function
+ .balign 16
+ .cfi_startproc
+__vdso_clock_getres:
+ mov $__NR_clock_getres, %eax
+ syscall
+ ret
+ .cfi_endproc
+ .size __vdso_clock_getres, . - __vdso_clock_getres
+
+clock_getres = __vdso_clock_getres
+ .weak clock_getres
+
+ .globl __vdso_gettimeofday
+ .type __vdso_gettimeofday, @function
+ .balign 16
+ .cfi_startproc
+__vdso_gettimeofday:
+ mov $__NR_gettimeofday, %eax
+ syscall
+ ret
+ .cfi_endproc
+ .size __vdso_gettimeofday, . - __vdso_gettimeofday
+
+gettimeofday = __vdso_gettimeofday
+ .weak gettimeofday
+
+
+ .globl __vdso_time
+ .type __vdso_time, @function
+ .balign 16
+ .cfi_startproc
+__vdso_time:
+ mov $__NR_time, %eax
+ syscall
+ ret
+ .cfi_endproc
+ .size __vdso_time, . - __vdso_time
+
+time = __vdso_time
+ .weak time
+
+
+ .globl __vdso_getcpu
+ .type __vdso_getcpu, @function
+ .balign 16
+ .cfi_startproc
+__vdso_getcpu:
+ /*
+ * ??? There is no syscall number for this allocated on x64.
+* We can handle this several ways:
+ *
+* (1) Invent a syscall number for use within qemu.
+ * It should be easy enough to pick a number that
+ * is well out of the way of the kernel numbers.
+ *
+ * (2) Force the emulated cpu to support the rdtscp insn,
+* and initialize the TSC_AUX value the appropriate value.
+ *
+* (3) Pretend that we're always running on cpu 0.
+ *
+* This last is the one that's implemented here, with the
+* tiny bit of extra code to support rdtscp in place.
+ */
+ xor %ecx, %ecx /* rdtscp w/ tsc_aux = 0 */
+
+ /* if (cpu != NULL) *cpu = (ecx & 0xfff); */
+ test%rdi, %rdi
+ jz 1f
+ mov %ecx, %eax
+ and $0xfff, %eax
+ mov %eax, (%rdi)
+
+ /* if (node != NULL) *node = (ecx >> 12); */
+1: test%rsi, %rsi
+ jz 2f
+ shr $12, %ecx
+ mov %ecx, (%rsi)
+
+2: xor
[PATCH 06/12] linux-user: Use ImageSource in load_symbols
Aside from the section headers, we're unlikely to hit the
ImageSource cache on guest executables. But the interface
for imgsrc_read_* is better.
Signed-off-by: Richard Henderson
---
linux-user/elfload.c | 87
1 file changed, 48 insertions(+), 39 deletions(-)
diff --git a/linux-user/elfload.c b/linux-user/elfload.c
index 425420f0cc..3c31a5e3b0 100644
--- a/linux-user/elfload.c
+++ b/linux-user/elfload.c
@@ -1729,7 +1729,8 @@ static inline void
bswap_mips_abiflags(Mips_elf_abiflags_v0 *abiflags) { }
#ifdef USE_ELF_CORE_DUMP
static int elf_core_dump(int, const CPUArchState *);
#endif /* USE_ELF_CORE_DUMP */
-static void load_symbols(struct elfhdr *hdr, int fd, abi_ulong load_bias);
+static void load_symbols(struct elfhdr *hdr, const ImageSource *src,
+ abi_ulong load_bias);
/* Verify the portions of EHDR within E_IDENT for the target.
This can be performed before bswapping the entire header. */
@@ -2896,7 +2897,7 @@ static void load_elf_image(const char *image_name, const
ImageSource *src,
}
if (qemu_log_enabled()) {
-load_symbols(ehdr, src->fd, load_bias);
+load_symbols(ehdr, src, load_bias);
}
mmap_unlock();
@@ -2984,19 +2985,20 @@ static int symcmp(const void *s0, const void *s1)
}
/* Best attempt to load symbols from this ELF object. */
-static void load_symbols(struct elfhdr *hdr, int fd, abi_ulong load_bias)
+static void load_symbols(struct elfhdr *hdr, const ImageSource *src,
+ abi_ulong load_bias)
{
int i, shnum, nsyms, sym_idx = 0, str_idx = 0;
-uint64_t segsz;
-struct elf_shdr *shdr;
+g_autofree struct elf_shdr *shdr = NULL;
char *strings = NULL;
-struct syminfo *s = NULL;
-struct elf_sym *new_syms, *syms = NULL;
+struct elf_sym *syms = NULL;
+struct elf_sym *new_syms;
+uint64_t segsz;
shnum = hdr->e_shnum;
-i = shnum * sizeof(struct elf_shdr);
-shdr = (struct elf_shdr *)alloca(i);
-if (pread(fd, shdr, i, hdr->e_shoff) != i) {
+shdr = imgsrc_read_alloc(hdr->e_shoff, shnum * sizeof(struct elf_shdr),
+ src, NULL);
+if (shdr == NULL) {
return;
}
@@ -3014,31 +3016,33 @@ static void load_symbols(struct elfhdr *hdr, int fd,
abi_ulong load_bias)
found:
/* Now know where the strtab and symtab are. Snarf them. */
-s = g_try_new(struct syminfo, 1);
-if (!s) {
-goto give_up;
-}
segsz = shdr[str_idx].sh_size;
-s->disas_strtab = strings = g_try_malloc(segsz);
-if (!strings ||
-pread(fd, strings, segsz, shdr[str_idx].sh_offset) != segsz) {
+strings = g_try_malloc(segsz);
+if (!strings) {
+goto give_up;
+}
+if (!imgsrc_read(strings, shdr[str_idx].sh_offset, segsz, src, NULL)) {
goto give_up;
}
segsz = shdr[sym_idx].sh_size;
-syms = g_try_malloc(segsz);
-if (!syms || pread(fd, syms, segsz, shdr[sym_idx].sh_offset) != segsz) {
-goto give_up;
-}
-
if (segsz / sizeof(struct elf_sym) > INT_MAX) {
-/* Implausibly large symbol table: give up rather than ploughing
- * on with the number of symbols calculation overflowing
+/*
+ * Implausibly large symbol table: give up rather than ploughing
+ * on with the number of symbols calculation overflowing.
*/
goto give_up;
}
nsyms = segsz / sizeof(struct elf_sym);
+syms = g_try_malloc(segsz);
+if (!syms) {
+goto give_up;
+}
+if (!imgsrc_read(syms, shdr[sym_idx].sh_offset, segsz, src, NULL)) {
+goto give_up;
+}
+
for (i = 0; i < nsyms; ) {
bswap_sym(syms + i);
/* Throw away entries which we do not need. */
@@ -3063,10 +3067,12 @@ static void load_symbols(struct elfhdr *hdr, int fd,
abi_ulong load_bias)
goto give_up;
}
-/* Attempt to free the storage associated with the local symbols
- that we threw away. Whether or not this has any effect on the
- memory allocation depends on the malloc implementation and how
- many symbols we managed to discard. */
+/*
+ * Attempt to free the storage associated with the local symbols
+ * that we threw away. Whether or not this has any effect on the
+ * memory allocation depends on the malloc implementation and how
+ * many symbols we managed to discard.
+ */
new_syms = g_try_renew(struct elf_sym, syms, nsyms);
if (new_syms == NULL) {
goto give_up;
@@ -3075,20 +3081,23 @@ static void load_symbols(struct elfhdr *hdr, int fd,
abi_ulong load_bias)
qsort(syms, nsyms, sizeof(*syms), symcmp);
-s->disas_num_syms = nsyms;
-#if ELF_CLASS == ELFCLASS32
-s->disas_symtab.elf32 = syms;
-#else
-s->disas_symtab.elf64 = syms;
-#endif
-s->lookup_symbol = lookup_symbolxx;
-s->next = syminfos;
-syminfos = s;
+{
+struct
[PATCH 10/12] linux-user: Add gen-vdso tool
This tool will be used for post-processing the linked vdso image,
turning it into something that is easy to include into elfload.c.
Signed-off-by: Richard Henderson
---
linux-user/gen-vdso.c | 168 ++
linux-user/gen-vdso-elfn.c.inc | 299 +
linux-user/meson.build | 6 +-
3 files changed, 472 insertions(+), 1 deletion(-)
create mode 100644 linux-user/gen-vdso.c
create mode 100644 linux-user/gen-vdso-elfn.c.inc
diff --git a/linux-user/gen-vdso.c b/linux-user/gen-vdso.c
new file mode 100644
index 00..ccbb6c4725
--- /dev/null
+++ b/linux-user/gen-vdso.c
@@ -0,0 +1,168 @@
+/*
+ * Post-process a vdso elf image for inclusion into qemu.
+ *
+ * Copyright 2021 Linaro, Ltd.
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include "elf.h"
+
+
+#define bswap_(p) _Generic(*(p), \
+uint16_t: __builtin_bswap16, \
+uint32_t: __builtin_bswap32, \
+uint64_t: __builtin_bswap64, \
+int16_t: __builtin_bswap16,\
+int32_t: __builtin_bswap32,\
+int64_t: __builtin_bswap64)
+#define bswaps(p) (*(p) = bswap_(p)(*(p)))
+
+static void output_reloc(FILE *outf, void *buf, void *loc)
+{
+fprintf(outf, "0x%08lx,\n", loc - buf);
+}
+
+#define N 32
+#define elfN(x) elf32_##x
+#define ElfN(x) Elf32_##x
+#include "gen-vdso-elfn.c.inc"
+#undef N
+#undef elfN
+#undef ElfN
+
+#define N 64
+#define elfN(x) elf64_##x
+#define ElfN(x) Elf64_##x
+#include "gen-vdso-elfn.c.inc"
+#undef N
+#undef elfN
+#undef ElfN
+
+
+int main(int ac, char **av)
+{
+FILE *inf, *outf;
+long total_len;
+const char *inf_name;
+const char *outf_name;
+unsigned char *buf;
+bool need_bswap;
+
+if (ac != 3) {
+fprintf(stderr, "usage: input-file output-file\n");
+return EXIT_FAILURE;
+}
+inf_name = av[1];
+outf_name = av[2];
+
+/*
+ * Open the input and output files.
+ */
+inf = fopen(inf_name, "rb");
+if (inf == NULL) {
+goto perror_inf;
+}
+outf = fopen(outf_name, "w");
+if (outf == NULL) {
+goto perror_outf;
+}
+
+/*
+ * Read the input file into a buffer.
+ * We expect the vdso to be small, on the order of one page,
+ * therefore we do not expect a partial read.
+ */
+fseek(inf, 0, SEEK_END);
+total_len = ftell(inf);
+fseek(inf, 0, SEEK_SET);
+
+buf = malloc(total_len);
+if (buf == NULL) {
+goto perror_inf;
+}
+
+errno = 0;
+if (fread(buf, 1, total_len, inf) != total_len) {
+if (errno) {
+goto perror_inf;
+}
+fprintf(stderr, "%s: incomplete read\n", inf_name);
+return EXIT_FAILURE;
+}
+fclose(inf);
+
+/*
+ * Write out the vdso image now, before we make local changes.
+ */
+
+fputs("/* Automatically generated from linux-user/gen-vdso.c. */\n"
+ "\n"
+ "static const uint8_t vdso_image[] = {",
+ outf);
+for (long i = 0; i < total_len; ++i) {
+if (i % 12 == 0) {
+fputs("\n ", outf);
+}
+fprintf(outf, " 0x%02x,", buf[i]);
+}
+fputs("\n};\n\n", outf);
+
+/*
+ * Identify which elf flavor we're processing.
+ * The first 16 bytes of the file are e_ident.
+ */
+
+if (buf[EI_MAG0] != ELFMAG0 || buf[EI_MAG1] != ELFMAG1 ||
+buf[EI_MAG2] != ELFMAG2 || buf[EI_MAG3] != ELFMAG3) {
+fprintf(stderr, "%s: not an elf file\n", inf_name);
+return EXIT_FAILURE;
+}
+switch (buf[EI_DATA]) {
+case ELFDATA2LSB:
+need_bswap = BYTE_ORDER != LITTLE_ENDIAN;
+break;
+case ELFDATA2MSB:
+need_bswap = BYTE_ORDER != BIG_ENDIAN;
+break;
+default:
+fprintf(stderr, "%s: invalid elf EI_DATA (%u)\n",
+inf_name, buf[EI_DATA]);
+return EXIT_FAILURE;
+}
+switch (buf[EI_CLASS]) {
+case ELFCLASS32:
+elf32_process(outf, buf, total_len, need_bswap);
+break;
+case ELFCLASS64:
+elf64_process(outf, buf, total_len, need_bswap);
+break;
+default:
+fprintf(stderr, "%s: invalid elf EI_CLASS (%u)\n",
+inf_name, buf[EI_CLASS]);
+return EXIT_FAILURE;
+}
+
+/*
+ * Everything should have gone well.
+ */
+if (fclose(outf)) {
+goto perror_outf;
+}
+return EXIT_SUCCESS;
+
+ perror_inf:
+perror(inf_name);
+return EXIT_FAILURE;
+
+ perror_outf:
+perror(outf_name);
+return EXIT_FAILURE;
+}
diff --git a/linux-user/gen-vdso-elfn.c.inc b/linux-user/gen-vdso-elfn.c.inc
new file mode 100644
index 00..4e9277aeff
--- /dev/null
+++ b/linux-user/gen-vdso-elfn.c.inc
@@ -0,0 +1,299 @@
+/*
+ *
[PATCH 02/12] linux-user: Introduce imgsrc_read, imgsrc_read_alloc
Introduced and initialized, but not yet really used.
These will tidy the current tests vs BPRM_BUF_SIZE.
Signed-off-by: Richard Henderson
---
linux-user/qemu.h | 50 ++
linux-user/linuxload.c | 46 ++
2 files changed, 87 insertions(+), 9 deletions(-)
diff --git a/linux-user/qemu.h b/linux-user/qemu.h
index 9e5e2aa499..f4cdfb16b3 100644
--- a/linux-user/qemu.h
+++ b/linux-user/qemu.h
@@ -168,6 +168,37 @@ extern unsigned long mmap_min_addr;
/* ??? See if we can avoid exposing so much of the loader internals. */
+typedef struct {
+const void *cache;
+unsigned int cache_size;
+int fd;
+} ImageSource;
+
+/**
+ * imgsrc_read: Read from ImageSource
+ * @dst: destination for read
+ * @offset: offset within file for read
+ * @len: size of the read
+ * @img: ImageSource to read from
+ * @errp: Error details.
+ *
+ * Read into @dst, using the cache when possible.
+ */
+bool imgsrc_read(void *dst, off_t offset, size_t len,
+ const ImageSource *img, Error **errp);
+
+/**
+ * imgsrc_read_alloc: Read from ImageSource
+ * @offset: offset within file for read
+ * @size: size of the read
+ * @img: ImageSource to read from
+ * @errp: Error details.
+ *
+ * Read into newly allocated memory, using the cache when possible.
+ */
+void *imgsrc_read_alloc(off_t offset, size_t len,
+const ImageSource *img, Error **errp);
+
/* Read a good amount of data initially, to hopefully get all the
program headers loaded. */
#define BPRM_BUF_SIZE 1024
@@ -177,15 +208,16 @@ extern unsigned long mmap_min_addr;
* used when loading binaries.
*/
struct linux_binprm {
-char buf[BPRM_BUF_SIZE] __attribute__((aligned));
-abi_ulong p;
-int fd;
-int e_uid, e_gid;
-int argc, envc;
-char **argv;
-char **envp;
-char * filename;/* Name of binary */
-int (*core_dump)(int, const CPUArchState *); /* coredump routine */
+char buf[BPRM_BUF_SIZE] __attribute__((aligned));
+ImageSource src;
+abi_ulong p;
+int fd;
+int e_uid, e_gid;
+int argc, envc;
+char **argv;
+char **envp;
+char *filename; /* Name of binary */
+int (*core_dump)(int, const CPUArchState *); /* coredump routine */
};
typedef struct IOCTLEntry IOCTLEntry;
diff --git a/linux-user/linuxload.c b/linux-user/linuxload.c
index 9d4eb5e94b..3b0bafc490 100644
--- a/linux-user/linuxload.c
+++ b/linux-user/linuxload.c
@@ -2,6 +2,7 @@
#include "qemu/osdep.h"
#include "qemu.h"
+#include "qapi/error.h"
#define NGROUPS 32
@@ -74,6 +75,10 @@ static int prepare_binprm(struct linux_binprm *bprm)
/* Make sure the rest of the loader won't read garbage. */
memset(bprm->buf + retval, 0, BPRM_BUF_SIZE - retval);
}
+
+bprm->src.cache = bprm->buf;
+bprm->src.cache_size = retval;
+
return retval;
}
@@ -129,6 +134,7 @@ int loader_exec(int fdexec, const char *filename, char
**argv, char **envp,
int retval;
bprm->fd = fdexec;
+bprm->src.fd = fdexec;
bprm->filename = (char *)filename;
bprm->argc = count(argv);
bprm->argv = argv;
@@ -163,3 +169,43 @@ int loader_exec(int fdexec, const char *filename, char
**argv, char **envp,
return retval;
}
+
+bool imgsrc_read(void *dst, off_t offset, size_t len,
+ const ImageSource *img, Error **errp)
+{
+ssize_t ret;
+
+if (offset + len <= img->cache_size) {
+memcpy(dst, img->cache + offset, len);
+return true;
+}
+
+if (img->fd < 0) {
+error_setg(errp, "read past end of buffer");
+return false;
+}
+
+ret = pread(img->fd, dst, len, offset);
+if (ret == len) {
+return true;
+}
+if (ret < 0) {
+error_setg_errno(errp, errno, "Error reading file header");
+} else {
+error_setg(errp, "Incomplete read of file header");
+}
+return false;
+}
+
+void *imgsrc_read_alloc(off_t offset, size_t len,
+const ImageSource *img, Error **errp)
+{
+void *alloc = g_malloc(len);
+bool ok = imgsrc_read(alloc, offset, len, img, errp);
+
+if (!ok) {
+g_free(alloc);
+alloc = NULL;
+}
+return alloc;
+}
--
2.25.1
Re: [RFC PATCH] tests/tcg: skip the signals test for hppa for now
On 6/18/21 2:32 AM, Alex Bennée wrote: While Richard has some patches that fix the instability on other architectures the hppa signal support still needs vdso support before we can make this reliable. So for now skip the test. Patches for a vdso posted. r~ Signed-off-by: Alex Bennée Cc: Richard Henderson --- tests/tcg/hppa/Makefile.target | 4 1 file changed, 4 insertions(+) diff --git a/tests/tcg/hppa/Makefile.target b/tests/tcg/hppa/Makefile.target index 8bf01966bd..71791235f6 100644 --- a/tests/tcg/hppa/Makefile.target +++ b/tests/tcg/hppa/Makefile.target @@ -4,3 +4,7 @@ # On parisc Linux supports 4K/16K/64K (but currently only 4k works) EXTRA_RUNS+=run-test-mmap-4096 # run-test-mmap-16384 run-test-mmap-65536 + +# There is a race that causes this to fail about 1% of the time +run-signals: signals + $(call skip-test, $<, "BROKEN awaiting vdso support")
[PATCH 09/12] linux-user: Load vdso image if available
The vdso image will be pre-processed into a C data array, with
a simple list of relocations to perform, and identifying the
location of signal trampolines.
Signed-off-by: Richard Henderson
---
linux-user/elfload.c | 75 +++-
1 file changed, 67 insertions(+), 8 deletions(-)
diff --git a/linux-user/elfload.c b/linux-user/elfload.c
index 8a3a7ae3ac..a6ad454617 100644
--- a/linux-user/elfload.c
+++ b/linux-user/elfload.c
@@ -1966,7 +1966,8 @@ static abi_ulong loader_build_fdpic_loadmap(struct
image_info *info, abi_ulong s
static abi_ulong create_elf_tables(abi_ulong p, int argc, int envc,
struct elfhdr *exec,
struct image_info *info,
- struct image_info *interp_info)
+ struct image_info *interp_info,
+ struct image_info *vdso_info)
{
abi_ulong sp;
abi_ulong u_argc, u_argv, u_envp, u_auxv;
@@ -2038,8 +2039,12 @@ static abi_ulong create_elf_tables(abi_ulong p, int
argc, int envc,
}
size = (DLINFO_ITEMS + 1) * 2;
-if (k_platform)
+if (k_platform) {
size += 2;
+}
+if (vdso_info) {
+size += 4;
+}
#ifdef DLINFO_ARCH_ITEMS
size += DLINFO_ARCH_ITEMS * 2;
#endif
@@ -2116,6 +2121,10 @@ static abi_ulong create_elf_tables(abi_ulong p, int
argc, int envc,
if (u_platform) {
NEW_AUX_ENT(AT_PLATFORM, u_platform);
}
+if (vdso_info) {
+NEW_AUX_ENT(AT_SYSINFO, vdso_info->entry);
+NEW_AUX_ENT(AT_SYSINFO_EHDR, vdso_info->load_addr);
+}
NEW_AUX_ENT (AT_NULL, 0);
#undef NEW_AUX_ENT
@@ -2942,6 +2951,53 @@ static void load_elf_interp(const char *filename, struct
image_info *info,
load_elf_image(filename, , info, , NULL);
}
+#ifndef HAVE_VDSO
+#define HAVE_VDSO 0
+static uint8_t vdso_image[] = { };
+static uint32_t vdso_relocs[] = { };
+#define vdso_sigreturn 0
+#define vdso_rt_sigreturn 0
+#endif
+
+static void load_elf_vdso(struct image_info *info)
+{
+ImageSource src;
+struct elfhdr ehdr;
+abi_ulong load_bias, load_addr;
+
+src.fd = -1;
+src.cache = vdso_image;
+src.cache_size = sizeof(vdso_image);
+
+load_elf_image("", , info, , NULL);
+load_addr = info->load_addr;
+load_bias = info->load_bias;
+
+/*
+ * We need to relocate the VDSO image. The one built into the kernel
+ * is built for a fixed address. The one built for QEMU is not, since
+ * that requires close control of the guest address space.
+ * We pre-processed the image to locate all of the addresses that need
+ * to be updated.
+ */
+for (size_t i = 0, n = ARRAY_SIZE(vdso_relocs); i < n; i++) {
+abi_ulong *addr = g2h_untagged(load_addr + vdso_relocs[i]);
+*addr = tswapal(tswapal(*addr) + load_bias);
+}
+
+/* Install signal trampolines, if present. */
+if (vdso_sigreturn) {
+default_sigreturn = load_addr + vdso_sigreturn;
+}
+if (vdso_rt_sigreturn) {
+default_rt_sigreturn = load_addr + vdso_rt_sigreturn;
+}
+
+/* Mark the VDSO writable segment read-only. */
+target_mprotect(info->start_data, info->end_data - info->start_data,
+PROT_READ);
+}
+
static int symfind(const void *s0, const void *s1)
{
target_ulong addr = *(target_ulong *)s0;
@@ -3146,7 +3202,7 @@ int load_elf_binary(struct linux_binprm *bprm, struct
image_info *info)
* and let elf_load_image do any swapping that may be required.
*/
struct elfhdr ehdr;
-struct image_info interp_info;
+struct image_info interp_info, vdso_info;
char *elf_interpreter = NULL;
char *scratch;
@@ -3216,10 +3272,12 @@ int load_elf_binary(struct linux_binprm *bprm, struct
image_info *info)
}
/*
- * TODO: load a vdso, which would also contain the signal trampolines.
- * Otherwise, allocate a private page to hold them.
+ * Load a vdso if available, which will amongst other things contain the
+ * signal trampolines. Otherwise, allocate a separate page for them.
*/
-if (TARGET_ARCH_HAS_SIGTRAMP_PAGE) {
+if (HAVE_VDSO) {
+load_elf_vdso(_info);
+} else if (TARGET_ARCH_HAS_SIGTRAMP_PAGE) {
abi_ulong tramp_page = target_mmap(0, TARGET_PAGE_SIZE,
PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANON, -1, 0);
@@ -3227,8 +3285,9 @@ int load_elf_binary(struct linux_binprm *bprm, struct
image_info *info)
target_mprotect(tramp_page, TARGET_PAGE_SIZE, PROT_READ | PROT_EXEC);
}
-bprm->p = create_elf_tables(bprm->p, bprm->argc, bprm->envc, ,
-info, (elf_interpreter ? _info : NULL));
+bprm->p = create_elf_tables(bprm->p, bprm->argc, bprm->envc, , info,
+elf_interpreter ? _info : NULL,
+
[PATCH 08/12] linux-user: Introduce imgsrc_mmap
Signed-off-by: Richard Henderson
---
linux-user/qemu.h | 11 +++
linux-user/elfload.c | 4 ++--
linux-user/linuxload.c | 44 ++
3 files changed, 57 insertions(+), 2 deletions(-)
diff --git a/linux-user/qemu.h b/linux-user/qemu.h
index dafaae6293..255182e133 100644
--- a/linux-user/qemu.h
+++ b/linux-user/qemu.h
@@ -199,6 +199,17 @@ bool imgsrc_read(void *dst, off_t offset, size_t len,
void *imgsrc_read_alloc(off_t offset, size_t len,
const ImageSource *img, Error **errp);
+/**
+ * imgsrc_mmap: Map from ImageSource
+ *
+ * If @src has a file descriptor, pass on to target_mmap. Otherwise,
+ * this is "mapping" from a host buffer, which resolves to memcpy.
+ * Therefore, flags must be MAP_PRIVATE | MAP_FIXED; the argument is
+ * retained for clarity.
+ */
+abi_long imgsrc_mmap(abi_ulong start, abi_ulong len, int prot,
+ int flags, const ImageSource *src, abi_ulong offset);
+
/* Read a good amount of data initially, to hopefully get all the
program headers loaded. */
#define BPRM_BUF_SIZE 1024
diff --git a/linux-user/elfload.c b/linux-user/elfload.c
index 3c31a5e3b0..8a3a7ae3ac 100644
--- a/linux-user/elfload.c
+++ b/linux-user/elfload.c
@@ -2832,9 +2832,9 @@ static void load_elf_image(const char *image_name, const
ImageSource *src,
*/
if (eppnt->p_filesz != 0) {
vaddr_len = TARGET_ELF_PAGELENGTH(eppnt->p_filesz + vaddr_po);
-error = target_mmap(vaddr_ps, vaddr_len, elf_prot,
+error = imgsrc_mmap(vaddr_ps, vaddr_len, elf_prot,
MAP_PRIVATE | MAP_FIXED,
-src->fd, eppnt->p_offset - vaddr_po);
+src, eppnt->p_offset - vaddr_po);
if (error == -1) {
goto exit_mmap;
diff --git a/linux-user/linuxload.c b/linux-user/linuxload.c
index d0d3f2ed0e..a437a22b49 100644
--- a/linux-user/linuxload.c
+++ b/linux-user/linuxload.c
@@ -208,3 +208,47 @@ void *imgsrc_read_alloc(off_t offset, size_t len,
}
return alloc;
}
+
+abi_long imgsrc_mmap(abi_ulong start, abi_ulong len, int prot,
+ int flags, const ImageSource *src, abi_ulong offset)
+{
+abi_long ret;
+int prot_write;
+void *haddr;
+
+assert(flags == (MAP_PRIVATE | MAP_FIXED));
+
+if (src->fd >= 0) {
+return target_mmap(start, len, prot, flags, src->fd, offset);
+}
+
+/*
+ * This case is for the vdso; we don't expect bad images.
+ * The mmap may extend beyond the end of the image, especially
+ * to the end of the page. Zero fill.
+ */
+assert(offset < src->cache_size);
+
+prot_write = prot | PROT_WRITE;
+ret = target_mmap(start, len, prot_write, flags | MAP_ANON, -1, 0);
+if (ret == -1) {
+return ret;
+}
+
+haddr = lock_user(VERIFY_WRITE, start, len, 0);
+assert(haddr != NULL);
+if (offset + len < src->cache_size) {
+memcpy(haddr, src->cache + offset, len);
+} else {
+size_t rest = src->cache_size - offset;
+memcpy(haddr, src->cache + offset, rest);
+memset(haddr + rest, 0, len - rest);
+}
+unlock_user(haddr, start, len);
+
+if (prot != prot_write) {
+target_mprotect(start, len, prot);
+}
+
+return ret;
+}
--
2.25.1
[PATCH 03/12] linux-user: Tidy loader_exec
Reorg the if cases to reduce indentation.
Test for 4 bytes in the file before checking the signatures.
Signed-off-by: Richard Henderson
---
linux-user/linuxload.c | 42 +-
1 file changed, 21 insertions(+), 21 deletions(-)
diff --git a/linux-user/linuxload.c b/linux-user/linuxload.c
index 3b0bafc490..8b93b9704c 100644
--- a/linux-user/linuxload.c
+++ b/linux-user/linuxload.c
@@ -143,31 +143,31 @@ int loader_exec(int fdexec, const char *filename, char
**argv, char **envp,
retval = prepare_binprm(bprm);
-if (retval >= 0) {
-if (bprm->buf[0] == 0x7f
-&& bprm->buf[1] == 'E'
-&& bprm->buf[2] == 'L'
-&& bprm->buf[3] == 'F') {
-retval = load_elf_binary(bprm, infop);
-#if defined(TARGET_HAS_BFLT)
-} else if (bprm->buf[0] == 'b'
-&& bprm->buf[1] == 'F'
-&& bprm->buf[2] == 'L'
-&& bprm->buf[3] == 'T') {
-retval = load_flt_binary(bprm, infop);
-#endif
-} else {
-return -ENOEXEC;
-}
+if (retval < 4) {
+return -ENOEXEC;
}
-
-if (retval >= 0) {
-/* success. Initialize important registers */
-do_init_thread(regs, infop);
+if (bprm->buf[0] == 0x7f
+&& bprm->buf[1] == 'E'
+&& bprm->buf[2] == 'L'
+&& bprm->buf[3] == 'F') {
+retval = load_elf_binary(bprm, infop);
+#if defined(TARGET_HAS_BFLT)
+} else if (bprm->buf[0] == 'b'
+ && bprm->buf[1] == 'F'
+ && bprm->buf[2] == 'L'
+ && bprm->buf[3] == 'T') {
+retval = load_flt_binary(bprm, infop);
+#endif
+} else {
+return -ENOEXEC;
+}
+if (retval < 0) {
return retval;
}
-return retval;
+/* Success. Initialize important registers. */
+do_init_thread(regs, infop);
+return 0;
}
bool imgsrc_read(void *dst, off_t offset, size_t len,
--
2.25.1
[PATCH 07/12] linux-user: Replace bprm->fd with bprm->src.fd
There are only a couple of uses of bprm->fd remaining.
Migrate to the other field.
Signed-off-by: Richard Henderson
---
linux-user/qemu.h | 1 -
linux-user/flatload.c | 8
linux-user/linuxload.c | 5 ++---
3 files changed, 6 insertions(+), 8 deletions(-)
diff --git a/linux-user/qemu.h b/linux-user/qemu.h
index f4cdfb16b3..dafaae6293 100644
--- a/linux-user/qemu.h
+++ b/linux-user/qemu.h
@@ -211,7 +211,6 @@ struct linux_binprm {
char buf[BPRM_BUF_SIZE] __attribute__((aligned));
ImageSource src;
abi_ulong p;
-int fd;
int e_uid, e_gid;
int argc, envc;
char **argv;
diff --git a/linux-user/flatload.c b/linux-user/flatload.c
index 3e5594cf89..58d0d9352c 100644
--- a/linux-user/flatload.c
+++ b/linux-user/flatload.c
@@ -460,7 +460,7 @@ static int load_flat_file(struct linux_binprm * bprm,
DBG_FLT("BINFMT_FLAT: ROM mapping of file (we hope)\n");
textpos = target_mmap(0, text_len, PROT_READ|PROT_EXEC,
- MAP_PRIVATE, bprm->fd, 0);
+ MAP_PRIVATE, bprm->src.fd, 0);
if (textpos == -1) {
fprintf(stderr, "Unable to mmap process text\n");
return -1;
@@ -487,7 +487,7 @@ static int load_flat_file(struct linux_binprm * bprm,
} else
#endif
{
-result = target_pread(bprm->fd, datapos,
+result = target_pread(bprm->src.fd, datapos,
data_len + (relocs * sizeof(abi_ulong)),
fpos);
}
@@ -537,10 +537,10 @@ static int load_flat_file(struct linux_binprm * bprm,
else
#endif
{
-result = target_pread(bprm->fd, textpos,
+result = target_pread(bprm->src.fd, textpos,
text_len, 0);
if (result >= 0) {
-result = target_pread(bprm->fd, datapos,
+result = target_pread(bprm->src.fd, datapos,
data_len + (relocs * sizeof(abi_ulong)),
ntohl(hdr->data_start));
}
diff --git a/linux-user/linuxload.c b/linux-user/linuxload.c
index 8b93b9704c..d0d3f2ed0e 100644
--- a/linux-user/linuxload.c
+++ b/linux-user/linuxload.c
@@ -36,7 +36,7 @@ static int prepare_binprm(struct linux_binprm *bprm)
int mode;
int retval;
-if (fstat(bprm->fd, ) < 0) {
+if (fstat(bprm->src.fd, ) < 0) {
return -errno;
}
@@ -66,7 +66,7 @@ static int prepare_binprm(struct linux_binprm *bprm)
bprm->e_gid = st.st_gid;
}
-retval = read(bprm->fd, bprm->buf, BPRM_BUF_SIZE);
+retval = read(bprm->src.fd, bprm->buf, BPRM_BUF_SIZE);
if (retval < 0) {
perror("prepare_binprm");
exit(-1);
@@ -133,7 +133,6 @@ int loader_exec(int fdexec, const char *filename, char
**argv, char **envp,
{
int retval;
-bprm->fd = fdexec;
bprm->src.fd = fdexec;
bprm->filename = (char *)filename;
bprm->argc = count(argv);
--
2.25.1
[PATCH 00/12] linux-user: Load a vdso for x86_64 and hppa
Supersedes: 20200519194452.9009-1-richard.hender...@linaro.org
Based-on: 20210618192951.125651-1-richard.hender...@linaro.org
("[PATCH v2 00/23] linux-user: Move signal trampolines to new page")
Previous cover letter:
> The subject of AT_SYSINFO came up on launchpad recently.
>
> There is definite room for improvement in all of this:
>
> (1) We could build the vdso binary into qemu instead of really
> loading it from the file system. This would obviate the
> several problems of locating the .so file. It would also
> mean that --static builds continue to create a standalone
> qemu binary.
>
> (2) We could use our cross-build system to build the vdso.
> Though we'd still likely want to keep the image in git
> along side the other rom images for when cross-build is
> not available.
>
> (3) There are some ??? comments where some decisions could be made,
> and other ??? that are merely commenting on weirdness.
>
> (4) It shouldn't take too much effort to create vdsos for the
> other architectures. But we should get this one as clean
> as we can first.
Amusingly, this patch set has turned 11 years old.
First posted April 4, 2010.
Change since previous:
Point (1) has been addressed: the vdso binary pre-processed,
validating the VDSO image, emitting a C array for the image,
collecting the set of relocations required, and identifying
any signal return trampolines.
Point (2) is still an issue in that the cross-build system is
tied up in tests/ makefiles. It might be time to bite the bullet
and convert it all to meson.
I think there are fewer ??? than before.
r~
Richard Henderson (12):
linux-user: Fix style problems in linuxload.c
linux-user: Introduce imgsrc_read, imgsrc_read_alloc
linux-user: Tidy loader_exec
linux-user: Do not clobber bprm_buf swapping ehdr
linux-user: Use ImageSource in load_elf_image
linux-user: Use ImageSource in load_symbols
linux-user: Replace bprm->fd with bprm->src.fd
linux-user: Introduce imgsrc_mmap
linux-user: Load vdso image if available
linux-user: Add gen-vdso tool
linux-user/x86_64: Add vdso
linux-user/hppa: Add vdso and use it for rt_sigreturn
linux-user/qemu.h | 60 ++-
linux-user/elfload.c| 305 ++--
linux-user/flatload.c | 8 +-
linux-user/gen-vdso.c | 168 ++
linux-user/hppa/signal.c| 8 +-
linux-user/linuxload.c | 171 +-
linux-user/gen-vdso-elfn.c.inc | 299 +++
linux-user/hppa/Makefile.vdso | 4 +
linux-user/hppa/meson.build | 6 +
linux-user/hppa/vdso.S | 149
linux-user/hppa/vdso.ld | 75
linux-user/hppa/vdso.so | Bin 0 -> 5192 bytes
linux-user/meson.build | 6 +-
linux-user/x86_64/Makefile.vdso | 3 +
linux-user/x86_64/meson.build | 6 +
linux-user/x86_64/vdso.S| 122 +
linux-user/x86_64/vdso.ld | 74
linux-user/x86_64/vdso.so | Bin 0 -> 5912 bytes
18 files changed, 1272 insertions(+), 192 deletions(-)
create mode 100644 linux-user/gen-vdso.c
create mode 100644 linux-user/gen-vdso-elfn.c.inc
create mode 100644 linux-user/hppa/Makefile.vdso
create mode 100644 linux-user/hppa/vdso.S
create mode 100644 linux-user/hppa/vdso.ld
create mode 100755 linux-user/hppa/vdso.so
create mode 100644 linux-user/x86_64/Makefile.vdso
create mode 100644 linux-user/x86_64/vdso.S
create mode 100644 linux-user/x86_64/vdso.ld
create mode 100755 linux-user/x86_64/vdso.so
--
2.25.1
[PATCH 01/12] linux-user: Fix style problems in linuxload.c
Signed-off-by: Richard Henderson
---
linux-user/linuxload.c | 42 --
1 file changed, 20 insertions(+), 22 deletions(-)
diff --git a/linux-user/linuxload.c b/linux-user/linuxload.c
index a27e1d0d8b..9d4eb5e94b 100644
--- a/linux-user/linuxload.c
+++ b/linux-user/linuxload.c
@@ -1,59 +1,57 @@
/* Code for loading Linux executables. Mostly linux kernel code. */
#include "qemu/osdep.h"
-
#include "qemu.h"
#define NGROUPS 32
/* ??? This should really be somewhere else. */
-abi_long memcpy_to_target(abi_ulong dest, const void *src,
- unsigned long len)
+abi_long memcpy_to_target(abi_ulong dest, const void *src, unsigned long len)
{
void *host_ptr;
host_ptr = lock_user(VERIFY_WRITE, dest, len, 0);
-if (!host_ptr)
+if (!host_ptr) {
return -TARGET_EFAULT;
+}
memcpy(host_ptr, src, len);
unlock_user(host_ptr, dest, 1);
return 0;
}
-static int count(char ** vec)
+static int count(char **vec)
{
-inti;
+int i;
-for(i = 0; *vec; i++) {
+for (i = 0; *vec; i++) {
vec++;
}
-
-return(i);
+return i;
}
static int prepare_binprm(struct linux_binprm *bprm)
{
-struct statst;
+struct stat st;
int mode;
int retval;
-if(fstat(bprm->fd, ) < 0) {
-return(-errno);
+if (fstat(bprm->fd, ) < 0) {
+return -errno;
}
mode = st.st_mode;
-if(!S_ISREG(mode)) { /* Must be regular file */
-return(-EACCES);
+if (!S_ISREG(mode)) { /* Must be regular file */
+return -EACCES;
}
-if(!(mode & 0111)) { /* Must have at least one execute bit set */
-return(-EACCES);
+if (!(mode & 0111)) { /* Must have at least one execute bit set */
+return -EACCES;
}
bprm->e_uid = geteuid();
bprm->e_gid = getegid();
/* Set-uid? */
-if(mode & S_ISUID) {
+if (mode & S_ISUID) {
bprm->e_uid = st.st_uid;
}
@@ -125,8 +123,8 @@ abi_ulong loader_build_argptr(int envc, int argc, abi_ulong
sp,
}
int loader_exec(int fdexec, const char *filename, char **argv, char **envp,
- struct target_pt_regs * regs, struct image_info *infop,
- struct linux_binprm *bprm)
+struct target_pt_regs *regs, struct image_info *infop,
+struct linux_binprm *bprm)
{
int retval;
@@ -139,7 +137,7 @@ int loader_exec(int fdexec, const char *filename, char
**argv, char **envp,
retval = prepare_binprm(bprm);
-if(retval>=0) {
+if (retval >= 0) {
if (bprm->buf[0] == 0x7f
&& bprm->buf[1] == 'E'
&& bprm->buf[2] == 'L'
@@ -157,11 +155,11 @@ int loader_exec(int fdexec, const char *filename, char
**argv, char **envp,
}
}
-if(retval>=0) {
+if (retval >= 0) {
/* success. Initialize important registers */
do_init_thread(regs, infop);
return retval;
}
-return(retval);
+return retval;
}
--
2.25.1
[PATCH 04/12] linux-user: Do not clobber bprm_buf swapping ehdr
Rearrange the allocation of storage for ehdr between load_elf_image
and load_elf_binary. The same set of copies are done, but we don't
modify bprm_buf, which will be important later.
Signed-off-by: Richard Henderson
---
linux-user/elfload.c | 25 ++---
1 file changed, 14 insertions(+), 11 deletions(-)
diff --git a/linux-user/elfload.c b/linux-user/elfload.c
index c0236a0b09..1f9a69703a 100644
--- a/linux-user/elfload.c
+++ b/linux-user/elfload.c
@@ -2622,16 +2622,17 @@ static bool parse_elf_properties(int image_fd,
On return: INFO values will be filled in, as necessary or available. */
static void load_elf_image(const char *image_name, int image_fd,
- struct image_info *info, char **pinterp_name,
+ struct image_info *info, struct elfhdr *ehdr,
+ char **pinterp_name,
char bprm_buf[BPRM_BUF_SIZE])
{
-struct elfhdr *ehdr = (struct elfhdr *)bprm_buf;
struct elf_phdr *phdr;
abi_ulong load_addr, load_bias, loaddr, hiaddr, error;
int i, retval, prot_exec;
Error *err = NULL;
/* First of all, some simple consistency checks */
+memcpy(ehdr, bprm_buf, sizeof(*ehdr));
if (!elf_check_ident(ehdr)) {
error_setg(, "Invalid ELF image for this architecture");
goto exit_errmsg;
@@ -2944,6 +2945,7 @@ static void load_elf_image(const char *image_name, int
image_fd,
static void load_elf_interp(const char *filename, struct image_info *info,
char bprm_buf[BPRM_BUF_SIZE])
{
+struct elfhdr ehdr;
int fd, retval;
Error *err = NULL;
@@ -2965,7 +2967,7 @@ static void load_elf_interp(const char *filename, struct
image_info *info,
memset(bprm_buf + retval, 0, BPRM_BUF_SIZE - retval);
}
-load_elf_image(filename, fd, info, NULL, bprm_buf);
+load_elf_image(filename, fd, info, , NULL, bprm_buf);
}
static int symfind(const void *s0, const void *s1)
@@ -3157,8 +3159,14 @@ uint32_t get_elf_eflags(int fd)
int load_elf_binary(struct linux_binprm *bprm, struct image_info *info)
{
+/*
+ * We need a copy of the elf header for passing to create_elf_tables.
+ * We will have overwritten the original when we re-use bprm->buf
+ * while loading the interpreter. Allocate the storage for this now
+ * and let elf_load_image do any swapping that may be required.
+ */
+struct elfhdr ehdr;
struct image_info interp_info;
-struct elfhdr elf_ex;
char *elf_interpreter = NULL;
char *scratch;
@@ -3170,12 +3178,7 @@ int load_elf_binary(struct linux_binprm *bprm, struct
image_info *info)
info->start_mmap = (abi_ulong)ELF_START_MMAP;
load_elf_image(bprm->filename, bprm->fd, info,
- _interpreter, bprm->buf);
-
-/* ??? We need a copy of the elf header for passing to create_elf_tables.
- If we do nothing, we'll have overwritten this when we re-use bprm->buf
- when we load the interpreter. */
-elf_ex = *(struct elfhdr *)bprm->buf;
+ , _interpreter, bprm->buf);
/* Do this so that we can load the interpreter, if need be. We will
change some of these later */
@@ -3245,7 +3248,7 @@ int load_elf_binary(struct linux_binprm *bprm, struct
image_info *info)
target_mprotect(tramp_page, TARGET_PAGE_SIZE, PROT_READ | PROT_EXEC);
}
-bprm->p = create_elf_tables(bprm->p, bprm->argc, bprm->envc, _ex,
+bprm->p = create_elf_tables(bprm->p, bprm->argc, bprm->envc, ,
info, (elf_interpreter ? _info : NULL));
info->start_stack = bprm->p;
--
2.25.1
Shortcoming (oversight) in Windows installer
Hello, I'm new and can't use GitHub/GitLab, but I noticed this about the Windows installer: Does not add (nor offer to add) "C:\Program Files\qemu" to path. It should maybe be an optional choice in the installer. Have a wonderful day!
[qemu-web PATCH] (typo)
Hello, Sorry I cannot use github/gitlab. I'm new to QEMU but I found this typo (mismatched braces) in the *installed* HTML manual: In: "file:///C:/Program%20Files/qemu/share/doc/tools/qemu-img.html" This should *not* have ending ']': measure [--output=OFMT] [-O OUTPUT_FMT] [-o OPTIONS] [--size N | [--object OBJECTDEF] [--image-opts] [-f FMT] [-l SNAPSHOT_PARAM] FILENAME]
[PULL 71/72] python: add qmp-shell entry point
now 'qmp-shell' should be available from the command line when installing the python package. Signed-off-by: John Snow Message-id: 20210607200649.1840382-42-js...@redhat.com Signed-off-by: John Snow --- python/setup.cfg | 1 + 1 file changed, 1 insertion(+) diff --git a/python/setup.cfg b/python/setup.cfg index 7f3c59d74e..85cecbb41b 100644 --- a/python/setup.cfg +++ b/python/setup.cfg @@ -51,6 +51,7 @@ console_scripts = qom-tree = qemu.qmp.qom:QOMTree.entry_point qom-fuse = qemu.qmp.qom_fuse:QOMFuse.entry_point [fuse] qemu-ga-client = qemu.qmp.qemu_ga_client:main +qmp-shell = qemu.qmp.qmp_shell:main [flake8] extend-ignore = E722 # Prefer pylint's bare-except checks to flake8's -- 2.31.1
[PULL 69/72] scripts/qmp-shell: add docstrings
Signed-off-by: John Snow
Message-id: 20210607200649.1840382-40-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qmp-shell | 39 ++-
1 file changed, 38 insertions(+), 1 deletion(-)
diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell
index 1a8a4ba18a..15aedb80c2 100755
--- a/scripts/qmp/qmp-shell
+++ b/scripts/qmp/qmp-shell
@@ -106,15 +106,20 @@ LOG = logging.getLogger(__name__)
class QMPCompleter:
+"""
+QMPCompleter provides a readline library tab-complete behavior.
+"""
# NB: Python 3.9+ will probably allow us to subclass list[str] directly,
# but pylint as of today does not know that List[str] is simply 'list'.
def __init__(self) -> None:
self._matches: List[str] = []
def append(self, value: str) -> None:
+"""Append a new valid completion to the list of possibilities."""
return self._matches.append(value)
def complete(self, text: str, state: int) -> Optional[str]:
+"""readline.set_completer() callback implementation."""
for cmd in self._matches:
if cmd.startswith(text):
if state == 0:
@@ -124,7 +129,9 @@ class QMPCompleter:
class QMPShellError(qmp.QMPError):
-pass
+"""
+QMP Shell Base error class.
+"""
class FuzzyJSON(ast.NodeTransformer):
@@ -137,6 +144,9 @@ class FuzzyJSON(ast.NodeTransformer):
@classmethod
def visit_Name(cls, # pylint: disable=invalid-name
node: ast.Name) -> ast.AST:
+"""
+Transform Name nodes with certain values into Constant (keyword) nodes.
+"""
if node.id == 'true':
return ast.Constant(value=True)
if node.id == 'false':
@@ -147,6 +157,13 @@ class FuzzyJSON(ast.NodeTransformer):
class QMPShell(qmp.QEMUMonitorProtocol):
+"""
+QMPShell provides a basic readline-based QMP shell.
+
+:param address: Address of the QMP server.
+:param pretty: Pretty-print QMP messages.
+:param verbose: Echo outgoing QMP messages to console.
+"""
def __init__(self, address: qmp.SocketAddrT,
pretty: bool = False, verbose: bool = False):
super().__init__(address)
@@ -333,6 +350,9 @@ class QMPShell(qmp.QEMUMonitorProtocol):
def show_banner(self,
msg: str = 'Welcome to the QMP low-level shell!') -> None:
+"""
+Print to stdio a greeting, and the QEMU version if available.
+"""
print(msg)
if not self._greeting:
print('Connected')
@@ -342,6 +362,9 @@ class QMPShell(qmp.QEMUMonitorProtocol):
@property
def prompt(self) -> str:
+"""
+Return the current shell prompt, including a trailing space.
+"""
if self._transmode:
return 'TRANS> '
return '(QEMU) '
@@ -367,6 +390,9 @@ class QMPShell(qmp.QEMUMonitorProtocol):
return self._execute_cmd(cmdline)
def repl(self) -> Iterator[None]:
+"""
+Return an iterator that implements the REPL.
+"""
self.show_banner()
while self.read_exec_command():
yield
@@ -374,6 +400,13 @@ class QMPShell(qmp.QEMUMonitorProtocol):
class HMPShell(QMPShell):
+"""
+HMPShell provides a basic readline-based HMP shell, tunnelled via QMP.
+
+:param address: Address of the QMP server.
+:param pretty: Pretty-print QMP messages.
+:param verbose: Echo outgoing QMP messages to console.
+"""
def __init__(self, address: qmp.SocketAddrT,
pretty: bool = False, verbose: bool = False):
super().__init__(address, pretty, verbose)
@@ -451,11 +484,15 @@ class HMPShell(QMPShell):
def die(msg: str) -> NoReturn:
+"""Write an error to stderr, then exit with a return code of 1."""
sys.stderr.write('ERROR: %s\n' % msg)
sys.exit(1)
def main() -> None:
+"""
+qmp-shell entry point: parse command line arguments and start the REPL.
+"""
parser = argparse.ArgumentParser()
parser.add_argument('-H', '--hmp', action='store_true',
help='Use HMP interface')
--
2.31.1
[PULL 65/72] scripts/qmp-shell: Remove too-broad-exception
We are only anticipating QMPShellErrors here, for syntax we weren't able
to understand. Other errors, if any, should be allowed to percolate
upwards.
Signed-off-by: John Snow
Message-id: 20210607200649.1840382-36-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qmp-shell | 11 +++
1 file changed, 7 insertions(+), 4 deletions(-)
diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell
index 78e4eae007..8d5845ab48 100755
--- a/scripts/qmp/qmp-shell
+++ b/scripts/qmp/qmp-shell
@@ -291,10 +291,13 @@ class QMPShell(qmp.QEMUMonitorProtocol):
def _execute_cmd(self, cmdline: str) -> bool:
try:
qmpcmd = self.__build_cmd(cmdline)
-except Exception as err:
-print('Error while parsing command line: %s' % err)
-print('command format: ', end=' ')
-print('[arg-name1=arg1] ... [arg-nameN=argN]')
+except QMPShellError as err:
+print(
+f"Error while parsing command line: {err!s}\n"
+"command format: "
+"[arg-name1=arg1] ... [arg-nameN=argN",
+file=sys.stderr
+)
return True
# For transaction mode, we may have just cached the action:
if qmpcmd is None:
--
2.31.1
[PULL 62/72] scripts/qmp-shell: use logging to show warnings
A perfect candidate is non-fatal shell history messages.
Signed-off-by: John Snow
Message-id: 20210607200649.1840382-33-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qmp-shell | 10 --
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell
index ec028d662e..0199a13a34 100755
--- a/scripts/qmp/qmp-shell
+++ b/scripts/qmp/qmp-shell
@@ -67,6 +67,7 @@
import argparse
import ast
import json
+import logging
import os
import re
import readline
@@ -85,6 +86,9 @@ from qemu import qmp
from qemu.qmp import QMPMessage
+LOG = logging.getLogger(__name__)
+
+
class QMPCompleter:
# NB: Python 3.9+ will probably allow us to subclass list[str] directly,
# but pylint as of today does not know that List[str] is simply 'list'.
@@ -167,13 +171,15 @@ class QMPShell(qmp.QEMUMonitorProtocol):
except FileNotFoundError:
pass
except IOError as err:
-print(f"Failed to read history '{self._histfile}': {err!s}")
+msg = f"Failed to read history '{self._histfile}': {err!s}"
+LOG.warning(msg)
def _save_history(self) -> None:
try:
readline.write_history_file(self._histfile)
except IOError as err:
-print(f"Failed to save history file '{self._histfile}': {err!s}")
+msg = f"Failed to save history file '{self._histfile}': {err!s}"
+LOG.warning(msg)
@classmethod
def __parse_value(cls, val: str) -> object:
--
2.31.1
[PULL 72/72] scripts/qmp-shell: add redirection shim
qmp-shell has a new home, add a redirect for a little while as the dust settles. Signed-off-by: John Snow Message-id: 20210607200649.1840382-43-js...@redhat.com Signed-off-by: John Snow --- scripts/qmp/qmp-shell | 11 +++ 1 file changed, 11 insertions(+) create mode 100755 scripts/qmp/qmp-shell diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell new file mode 100755 index 00..4a20f97db7 --- /dev/null +++ b/scripts/qmp/qmp-shell @@ -0,0 +1,11 @@ +#!/usr/bin/env python3 + +import os +import sys + +sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python')) +from qemu.qmp import qmp_shell + + +if __name__ == '__main__': +qmp_shell.main() -- 2.31.1
[PULL 61/72] scripts/qmp-shell: Use context manager instead of atexit
We can invoke the shell history writing when we leave the QMPShell scope
instead of relying on atexit. Doing so may be preferable to avoid global
state being registered from within a class instead of from the
application logic directly.
Use QMP's context manager to hook this history saving at close time,
which gets invoked when we leave the context block.
Signed-off-by: John Snow
Message-id: 20210607200649.1840382-32-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qmp-shell | 33 ++---
1 file changed, 18 insertions(+), 15 deletions(-)
diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell
index f14fe211cc..ec028d662e 100755
--- a/scripts/qmp/qmp-shell
+++ b/scripts/qmp/qmp-shell
@@ -66,7 +66,6 @@
# sent to QEMU, which is useful for debugging and documentation generation.
import argparse
import ast
-import atexit
import json
import os
import re
@@ -142,6 +141,11 @@ class QMPShell(qmp.QEMUMonitorProtocol):
self.pretty = pretty
self.verbose = verbose
+def close(self) -> None:
+# Hook into context manager of parent to save shell history.
+self._save_history()
+super().close()
+
def _fill_completion(self) -> None:
cmds = self.cmd('query-commands')
if 'error' in cmds:
@@ -164,9 +168,8 @@ class QMPShell(qmp.QEMUMonitorProtocol):
pass
except IOError as err:
print(f"Failed to read history '{self._histfile}': {err!s}")
-atexit.register(self.__save_history)
-def __save_history(self) -> None:
+def _save_history(self) -> None:
try:
readline.write_history_file(self._histfile)
except IOError as err:
@@ -448,25 +451,25 @@ def main() -> None:
parser.error("QMP socket or TCP address must be specified")
shell_class = HMPShell if args.hmp else QMPShell
+
try:
address = shell_class.parse_address(args.qmp_server)
except qmp.QMPBadPortError:
parser.error(f"Bad port number: {args.qmp_server}")
return # pycharm doesn't know error() is noreturn
-qemu = shell_class(address, args.pretty, args.verbose)
+with shell_class(address, args.pretty, args.verbose) as qemu:
+try:
+qemu.connect(negotiate=not args.skip_negotiation)
+except qmp.QMPConnectError:
+die("Didn't get QMP greeting message")
+except qmp.QMPCapabilitiesError:
+die("Couldn't negotiate capabilities")
+except OSError as err:
+die(f"Couldn't connect to {args.qmp_server}: {err!s}")
-try:
-qemu.connect(negotiate=not args.skip_negotiation)
-except qmp.QMPConnectError:
-die("Didn't get QMP greeting message")
-except qmp.QMPCapabilitiesError:
-die("Couldn't negotiate capabilities")
-except OSError as err:
-die(f"Couldn't connect to {args.qmp_server}: {err!s}")
-
-for _ in qemu.repl():
-pass
+for _ in qemu.repl():
+pass
if __name__ == '__main__':
--
2.31.1
[PULL 67/72] scripts/qmp-shell: remove double-underscores
They're not needed; single underscore is enough to express intent that
these methods are "internal". double underscore is used as a weak name
mangling, but that isn't beneficial for us here.
Signed-off-by: John Snow
Message-id: 20210607200649.1840382-38-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qmp-shell | 52 +--
1 file changed, 26 insertions(+), 26 deletions(-)
diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell
index 82fe16cff8..40ff9e0a82 100755
--- a/scripts/qmp/qmp-shell
+++ b/scripts/qmp/qmp-shell
@@ -171,7 +171,7 @@ class QMPShell(qmp.QEMUMonitorProtocol):
for cmd in cmds['return']:
self._completer.append(cmd['name'])
-def __completer_setup(self) -> None:
+def _completer_setup(self) -> None:
self._completer = QMPCompleter()
self._fill_completion()
readline.set_history_length(1024)
@@ -196,7 +196,7 @@ class QMPShell(qmp.QEMUMonitorProtocol):
LOG.warning(msg)
@classmethod
-def __parse_value(cls, val: str) -> object:
+def _parse_value(cls, val: str) -> object:
try:
return int(val)
except ValueError:
@@ -221,9 +221,9 @@ class QMPShell(qmp.QEMUMonitorProtocol):
pass
return val
-def __cli_expr(self,
- tokens: Sequence[str],
- parent: qmp.QMPObject) -> None:
+def _cli_expr(self,
+ tokens: Sequence[str],
+ parent: qmp.QMPObject) -> None:
for arg in tokens:
(key, sep, val) = arg.partition('=')
if sep != '=':
@@ -231,7 +231,7 @@ class QMPShell(qmp.QEMUMonitorProtocol):
f"Expected a key=value pair, got '{arg!s}'"
)
-value = self.__parse_value(val)
+value = self._parse_value(val)
optpath = key.split('.')
curpath = []
for path in optpath[:-1]:
@@ -249,7 +249,7 @@ class QMPShell(qmp.QEMUMonitorProtocol):
raise QMPShellError(f'Cannot set "{key}" multiple times')
parent[optpath[-1]] = value
-def __build_cmd(self, cmdline: str) -> Optional[QMPMessage]:
+def _build_cmd(self, cmdline: str) -> Optional[QMPMessage]:
"""
Build a QMP input object from a user provided command-line in the
following format:
@@ -289,13 +289,13 @@ class QMPShell(qmp.QEMUMonitorProtocol):
if cmdargs[-1] == ')':
cmdargs.pop(-1)
finalize = True
-self.__cli_expr(cmdargs[1:], action['data'])
+self._cli_expr(cmdargs[1:], action['data'])
self._actions.append(action)
-return self.__build_cmd(')') if finalize else None
+return self._build_cmd(')') if finalize else None
# Standard command: parse and return it to be executed.
qmpcmd = {'execute': cmdargs[0], 'arguments': {}}
-self.__cli_expr(cmdargs[1:], qmpcmd['arguments'])
+self._cli_expr(cmdargs[1:], qmpcmd['arguments'])
return qmpcmd
def _print(self, qmp_message: object) -> None:
@@ -306,7 +306,7 @@ class QMPShell(qmp.QEMUMonitorProtocol):
def _execute_cmd(self, cmdline: str) -> bool:
try:
-qmpcmd = self.__build_cmd(cmdline)
+qmpcmd = self._build_cmd(cmdline)
except QMPShellError as err:
print(
f"Error while parsing command line: {err!s}\n"
@@ -329,7 +329,7 @@ class QMPShell(qmp.QEMUMonitorProtocol):
def connect(self, negotiate: bool = True) -> None:
self._greeting = super().connect(negotiate)
-self.__completer_setup()
+self._completer_setup()
def show_banner(self,
msg: str = 'Welcome to the QMP low-level shell!') -> None:
@@ -377,10 +377,10 @@ class HMPShell(QMPShell):
def __init__(self, address: qmp.SocketAddrT,
pretty: bool = False, verbose: bool = False):
super().__init__(address, pretty, verbose)
-self.__cpu_index = 0
+self._cpu_index = 0
-def __cmd_completion(self) -> None:
-for cmd in self.__cmd_passthrough('help')['return'].split('\r\n'):
+def _cmd_completion(self) -> None:
+for cmd in self._cmd_passthrough('help')['return'].split('\r\n'):
if cmd and cmd[0] != '[' and cmd[0] != '\t':
name = cmd.split()[0] # drop help text
if name == 'info':
@@ -396,22 +396,22 @@ class HMPShell(QMPShell):
self._completer.append(name)
self._completer.append('help ' + name) # help completion
-def __info_completion(self) -> None:
-for cmd in self.__cmd_passthrough('info')['return'].split('\r\n'):
+def _info_completion(self) -> None:
+for cmd in self._cmd_passthrough('info')['return'].split('\r\n'):
if cmd:
[PULL 64/72] scripts/qmp-shell: Fix empty-transaction invocation
calling "transaction( )" is pointless, but valid. Rework the parser to
allow this kind of invocation. This helps clean up exception handling
later by removing accidental breakages of the parser that aren't
explicitly forbidden.
Signed-off-by: John Snow
Message-id: 20210607200649.1840382-35-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qmp-shell | 14 --
1 file changed, 8 insertions(+), 6 deletions(-)
diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell
index 3c32b576a3..78e4eae007 100755
--- a/scripts/qmp/qmp-shell
+++ b/scripts/qmp/qmp-shell
@@ -244,11 +244,14 @@ class QMPShell(qmp.QEMUMonitorProtocol):
cmdargs = re.findall(argument_regex, cmdline)
qmpcmd: QMPMessage
-# Transactional CLI entry/exit:
-if cmdargs[0] == 'transaction(':
+# Transactional CLI entry:
+if cmdargs and cmdargs[0] == 'transaction(':
self._transmode = True
+self._actions = []
cmdargs.pop(0)
-elif cmdargs[0] == ')' and self._transmode:
+
+# Transactional CLI exit:
+if cmdargs and cmdargs[0] == ')' and self._transmode:
self._transmode = False
if len(cmdargs) > 1:
msg = 'Unexpected input after close of Transaction sub-shell'
@@ -257,15 +260,14 @@ class QMPShell(qmp.QEMUMonitorProtocol):
'execute': 'transaction',
'arguments': {'actions': self._actions}
}
-self._actions = list()
return qmpcmd
-# Nothing to process?
+# No args, or no args remaining
if not cmdargs:
return None
-# Parse and then cache this Transactional Action
if self._transmode:
+# Parse and cache this Transactional Action
finalize = False
action = {'type': cmdargs[0], 'data': {}}
if cmdargs[-1] == ')':
--
2.31.1
[PULL 58/72] scripts/qmp-shell: Accept SocketAddrT instead of string
Don't "extend" QEMUMonitorProtocol by changing the argument types. Move
the string parsing just outside of the class instead.
Signed-off-by: John Snow
Message-id: 20210607200649.1840382-29-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qmp-shell | 12 +++-
1 file changed, 7 insertions(+), 5 deletions(-)
diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell
index 2d0e85b5f7..b465c7f9e2 100755
--- a/scripts/qmp/qmp-shell
+++ b/scripts/qmp/qmp-shell
@@ -130,9 +130,9 @@ class FuzzyJSON(ast.NodeTransformer):
# TODO: QMPShell's interface is a bit ugly (eg. _fill_completion() and
# _execute_cmd()). Let's design a better one.
class QMPShell(qmp.QEMUMonitorProtocol):
-def __init__(self, address: str, pretty: bool = False,
- verbose: bool = False):
-super().__init__(self.parse_address(address))
+def __init__(self, address: qmp.SocketAddrT,
+ pretty: bool = False, verbose: bool = False):
+super().__init__(address)
self._greeting: Optional[QMPMessage] = None
self._completer = QMPCompleter()
self._pretty = pretty
@@ -347,7 +347,7 @@ class QMPShell(qmp.QEMUMonitorProtocol):
class HMPShell(QMPShell):
-def __init__(self, address: str,
+def __init__(self, address: qmp.SocketAddrT,
pretty: bool = False, verbose: bool = False):
super().__init__(address, pretty, verbose)
self.__cpu_index = 0
@@ -450,11 +450,13 @@ def main() -> None:
shell_class = HMPShell if args.hmp else QMPShell
try:
-qemu = shell_class(args.qmp_server, args.pretty, args.verbose)
+address = shell_class.parse_address(args.qmp_server)
except qmp.QMPBadPortError:
parser.error(f"Bad port number: {args.qmp_server}")
return # pycharm doesn't know error() is noreturn
+qemu = shell_class(address, args.pretty, args.verbose)
+
try:
qemu.connect(negotiate=not args.skip_negotiation)
except qmp.QMPConnectError:
--
2.31.1
[PULL 66/72] scripts/qmp-shell: convert usage comment to docstring
The nice usage comment should be a docstring instead of a comment, so
that it's visible from other python tooling.
Signed-off-by: John Snow
Message-id: 20210607200649.1840382-37-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qmp-shell | 128 --
1 file changed, 72 insertions(+), 56 deletions(-)
diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell
index 8d5845ab48..82fe16cff8 100755
--- a/scripts/qmp/qmp-shell
+++ b/scripts/qmp/qmp-shell
@@ -1,7 +1,5 @@
#!/usr/bin/env python3
#
-# Low-level QEMU shell on top of QMP.
-#
# Copyright (C) 2009, 2010 Red Hat Inc.
#
# Authors:
@@ -10,60 +8,78 @@
# This work is licensed under the terms of the GNU GPL, version 2. See
# the COPYING file in the top-level directory.
#
-# Usage:
-#
-# Start QEMU with:
-#
-# # qemu [...] -qmp unix:./qmp-sock,server
-#
-# Run the shell:
-#
-# $ qmp-shell ./qmp-sock
-#
-# Commands have the following format:
-#
-#< command-name > [ arg-name1=arg1 ] ... [ arg-nameN=argN ]
-#
-# For example:
-#
-# (QEMU) device_add driver=e1000 id=net1
-# {u'return': {}}
-# (QEMU)
-#
-# key=value pairs also support Python or JSON object literal subset notations,
-# without spaces. Dictionaries/objects {} are supported as are arrays [].
-#
-#example-command arg-name1={'key':'value','obj'={'prop':"value"}}
-#
-# Both JSON and Python formatting should work, including both styles of
-# string literal quotes. Both paradigms of literal values should work,
-# including null/true/false for JSON and None/True/False for Python.
-#
-#
-# Transactions have the following multi-line format:
-#
-#transaction(
-#action-name1 [ arg-name1=arg1 ] ... [arg-nameN=argN ]
-#...
-#action-nameN [ arg-name1=arg1 ] ... [arg-nameN=argN ]
-#)
-#
-# One line transactions are also supported:
-#
-#transaction( action-name1 ... )
-#
-# For example:
-#
-# (QEMU) transaction(
-# TRANS> block-dirty-bitmap-add node=drive0 name=bitmap1
-# TRANS> block-dirty-bitmap-clear node=drive0 name=bitmap0
-# TRANS> )
-# {"return": {}}
-# (QEMU)
-#
-# Use the -v and -p options to activate the verbose and pretty-print options,
-# which will echo back the properly formatted JSON-compliant QMP that is being
-# sent to QEMU, which is useful for debugging and documentation generation.
+
+"""
+Low-level QEMU shell on top of QMP.
+
+usage: qmp-shell [-h] [-H] [-N] [-v] [-p] qmp_server
+
+positional arguments:
+ qmp_server< UNIX socket path | TCP address:port >
+
+optional arguments:
+ -h, --helpshow this help message and exit
+ -H, --hmp Use HMP interface
+ -N, --skip-negotiation
+Skip negotiate (for qemu-ga)
+ -v, --verbose Verbose (echo commands sent and received)
+ -p, --pretty Pretty-print JSON
+
+
+Start QEMU with:
+
+# qemu [...] -qmp unix:./qmp-sock,server
+
+Run the shell:
+
+$ qmp-shell ./qmp-sock
+
+Commands have the following format:
+
+ < command-name > [ arg-name1=arg1 ] ... [ arg-nameN=argN ]
+
+For example:
+
+(QEMU) device_add driver=e1000 id=net1
+{'return': {}}
+(QEMU)
+
+key=value pairs also support Python or JSON object literal subset notations,
+without spaces. Dictionaries/objects {} are supported as are arrays [].
+
+ example-command arg-name1={'key':'value','obj'={'prop':"value"}}
+
+Both JSON and Python formatting should work, including both styles of
+string literal quotes. Both paradigms of literal values should work,
+including null/true/false for JSON and None/True/False for Python.
+
+
+Transactions have the following multi-line format:
+
+ transaction(
+ action-name1 [ arg-name1=arg1 ] ... [arg-nameN=argN ]
+ ...
+ action-nameN [ arg-name1=arg1 ] ... [arg-nameN=argN ]
+ )
+
+One line transactions are also supported:
+
+ transaction( action-name1 ... )
+
+For example:
+
+(QEMU) transaction(
+TRANS> block-dirty-bitmap-add node=drive0 name=bitmap1
+TRANS> block-dirty-bitmap-clear node=drive0 name=bitmap0
+TRANS> )
+{"return": {}}
+(QEMU)
+
+Use the -v and -p options to activate the verbose and pretty-print options,
+which will echo back the properly formatted JSON-compliant QMP that is being
+sent to QEMU, which is useful for debugging and documentation generation.
+"""
+
import argparse
import ast
import json
--
2.31.1
[PULL 59/72] scripts/qmp-shell: unprivatize 'pretty' property
Similar to verbose, there's no reason this needs to be hidden.
Signed-off-by: John Snow
Message-id: 20210607200649.1840382-30-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qmp-shell | 9 -
1 file changed, 4 insertions(+), 5 deletions(-)
diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell
index b465c7f9e2..f14fe211cc 100755
--- a/scripts/qmp/qmp-shell
+++ b/scripts/qmp/qmp-shell
@@ -135,11 +135,11 @@ class QMPShell(qmp.QEMUMonitorProtocol):
super().__init__(address)
self._greeting: Optional[QMPMessage] = None
self._completer = QMPCompleter()
-self._pretty = pretty
self._transmode = False
self._actions: List[QMPMessage] = []
self._histfile = os.path.join(os.path.expanduser('~'),
'.qmp-shell_history')
+self.pretty = pretty
self.verbose = verbose
def _fill_completion(self) -> None:
@@ -274,10 +274,9 @@ class QMPShell(qmp.QEMUMonitorProtocol):
return qmpcmd
def _print(self, qmp_message: object) -> None:
-indent = None
-if self._pretty:
-indent = 4
-jsobj = json.dumps(qmp_message, indent=indent, sort_keys=self._pretty)
+jsobj = json.dumps(qmp_message,
+ indent=4 if self.pretty else None,
+ sort_keys=self.pretty)
print(str(jsobj))
def _execute_cmd(self, cmdline: str) -> bool:
--
2.31.1
[PULL 68/72] scripts/qmp-shell: make QMPShellError inherit QMPError
In preparation for moving qmp-shell into the qemu.qmp package, make QMPShellError inherit from QMPError so that all custom errors in this package all derive from QMPError. Signed-off-by: John Snow Message-id: 20210607200649.1840382-39-js...@redhat.com Signed-off-by: John Snow --- scripts/qmp/qmp-shell | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell index 40ff9e0a82..1a8a4ba18a 100755 --- a/scripts/qmp/qmp-shell +++ b/scripts/qmp/qmp-shell @@ -123,7 +123,7 @@ class QMPCompleter: return None -class QMPShellError(Exception): +class QMPShellError(qmp.QMPError): pass -- 2.31.1
[PULL 70/72] scripts/qmp-shell: move to python/qemu/qmp/qmp_shell.py
The script will be unavailable for a commit or two, which will help preserve development history attached to the new file. A forwarder will be added shortly afterwards. With qmp_shell in the python qemu.qmp package, now it is fully type checked, linted, etc. via the Python CI. It will be quite a bit harder to accidentally break it again in the future. Signed-off-by: John Snow Message-id: 20210607200649.1840382-41-js...@redhat.com Signed-off-by: John Snow --- scripts/qmp/qmp-shell => python/qemu/qmp/qmp_shell.py | 3 --- 1 file changed, 3 deletions(-) rename scripts/qmp/qmp-shell => python/qemu/qmp/qmp_shell.py (99%) mode change 100755 => 100644 diff --git a/scripts/qmp/qmp-shell b/python/qemu/qmp/qmp_shell.py old mode 100755 new mode 100644 similarity index 99% rename from scripts/qmp/qmp-shell rename to python/qemu/qmp/qmp_shell.py index 15aedb80c2..337acfce2d --- a/scripts/qmp/qmp-shell +++ b/python/qemu/qmp/qmp_shell.py @@ -1,4 +1,3 @@ -#!/usr/bin/env python3 # # Copyright (C) 2009, 2010 Red Hat Inc. # @@ -96,8 +95,6 @@ Sequence, ) - -sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python')) from qemu import qmp from qemu.qmp import QMPMessage -- 2.31.1
[PULL 63/72] scripts/qmp-shell: remove TODO
We still want to revamp qmp-shell again, but there's much more to the idea than the comment now intuits. Remove it. Signed-off-by: John Snow Message-id: 20210607200649.1840382-34-js...@redhat.com Signed-off-by: John Snow --- scripts/qmp/qmp-shell | 2 -- 1 file changed, 2 deletions(-) diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell index 0199a13a34..3c32b576a3 100755 --- a/scripts/qmp/qmp-shell +++ b/scripts/qmp/qmp-shell @@ -130,8 +130,6 @@ class FuzzyJSON(ast.NodeTransformer): return node -# TODO: QMPShell's interface is a bit ugly (eg. _fill_completion() and -# _execute_cmd()). Let's design a better one. class QMPShell(qmp.QEMUMonitorProtocol): def __init__(self, address: qmp.SocketAddrT, pretty: bool = False, verbose: bool = False): -- 2.31.1
[PULL 52/72] scripts/qmp-shell: move the REPL functionality into QMPShell
Instead of doing this in main, move it into the class itself. (This
makes it easier to put into the qemu.qmp package later by removing as
much as we can from the main() function.)
Signed-off-by: John Snow
Message-id: 20210607200649.1840382-23-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qmp-shell | 10 +++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell
index 31269859c4..aa148517a8 100755
--- a/scripts/qmp/qmp-shell
+++ b/scripts/qmp/qmp-shell
@@ -318,6 +318,12 @@ class QMPShell(qmp.QEMUMonitorProtocol):
return self._execute_cmd(cmdline)
+def repl(self):
+self.show_banner()
+while self.read_exec_command():
+yield
+self.close()
+
class HMPShell(QMPShell):
def __init__(self, address, pretty=False, verbose=False):
@@ -435,10 +441,8 @@ def main():
except OSError as err:
die(f"Couldn't connect to {args.qmp_server}: {err!s}")
-qemu.show_banner()
-while qemu.read_exec_command():
+for _ in qemu.repl():
pass
-qemu.close()
if __name__ == '__main__':
--
2.31.1
[PULL 60/72] python/qmp: return generic type from context manager
__enter__ can be invoked from a subclass, so it needs a more flexible
type.
Signed-off-by: John Snow
Message-id: 20210607200649.1840382-31-js...@redhat.com
Signed-off-by: John Snow
---
python/qemu/qmp/__init__.py | 5 -
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/python/qemu/qmp/__init__.py b/python/qemu/qmp/__init__.py
index ba0d2281d6..376954cb6d 100644
--- a/python/qemu/qmp/__init__.py
+++ b/python/qemu/qmp/__init__.py
@@ -30,6 +30,7 @@
TextIO,
Tuple,
Type,
+TypeVar,
Union,
cast,
)
@@ -220,7 +221,9 @@ def __get_events(self, wait: Union[bool, float] = False) ->
None:
if ret is None:
raise QMPConnectError("Error while reading from socket")
-def __enter__(self) -> 'QEMUMonitorProtocol':
+T = TypeVar('T')
+
+def __enter__(self: T) -> T:
# Implement context manager enter function.
return self
--
2.31.1
[PULL 56/72] python/qmp: add QMPObject type alias
This is meant to represent any generic object seen in a QMPMessage, not just the root object itself. Signed-off-by: John Snow Message-id: 20210607200649.1840382-27-js...@redhat.com Signed-off-by: John Snow --- python/qemu/qmp/__init__.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/python/qemu/qmp/__init__.py b/python/qemu/qmp/__init__.py index a6e1a7b857..ba0d2281d6 100644 --- a/python/qemu/qmp/__init__.py +++ b/python/qemu/qmp/__init__.py @@ -41,6 +41,9 @@ #: QMPReturnValue is the 'return' value of a command. QMPReturnValue = object +#: QMPObject is any object in a QMP message. +QMPObject = Dict[str, object] + # QMPMessage can be outgoing commands or incoming events/returns. # QMPReturnValue is usually a dict/json object, but due to QAPI's # 'returns-whitelist', it can actually be anything. -- 2.31.1
[PULL 49/72] scripts/qmp-shell: Make verbose a public attribute
No real reason to hide this behind an underscore; make it part of the
initializer and make it a regular RW attribute.
Signed-off-by: John Snow
Message-id: 20210607200649.1840382-20-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qmp-shell | 16 ++--
1 file changed, 6 insertions(+), 10 deletions(-)
diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell
index de5fa189f0..cfcefb95f9 100755
--- a/scripts/qmp/qmp-shell
+++ b/scripts/qmp/qmp-shell
@@ -113,7 +113,7 @@ class FuzzyJSON(ast.NodeTransformer):
# TODO: QMPShell's interface is a bit ugly (eg. _fill_completion() and
# _execute_cmd()). Let's design a better one.
class QMPShell(qmp.QEMUMonitorProtocol):
-def __init__(self, address, pretty=False):
+def __init__(self, address, pretty=False, verbose=False):
super().__init__(self.parse_address(address))
self._greeting = None
self._completer = None
@@ -122,7 +122,7 @@ class QMPShell(qmp.QEMUMonitorProtocol):
self._actions = list()
self._histfile = os.path.join(os.path.expanduser('~'),
'.qmp-shell_history')
-self._verbose = False
+self.verbose = verbose
def _fill_completion(self):
cmds = self.cmd('query-commands')
@@ -271,7 +271,7 @@ class QMPShell(qmp.QEMUMonitorProtocol):
# For transaction mode, we may have just cached the action:
if qmpcmd is None:
return True
-if self._verbose:
+if self.verbose:
self._print(qmpcmd)
resp = self.cmd_obj(qmpcmd)
if resp is None:
@@ -317,13 +317,10 @@ class QMPShell(qmp.QEMUMonitorProtocol):
return self._execute_cmd(cmdline)
-def set_verbosity(self, verbose):
-self._verbose = verbose
-
class HMPShell(QMPShell):
-def __init__(self, address, pretty=False):
-super().__init__(address, pretty)
+def __init__(self, address, pretty=False, verbose=False):
+super().__init__(address, pretty, verbose)
self.__cpu_index = 0
def __cmd_completion(self):
@@ -423,7 +420,7 @@ def main():
shell_class = HMPShell if args.hmp else QMPShell
try:
-qemu = shell_class(args.qmp_server, args.pretty)
+qemu = shell_class(args.qmp_server, args.pretty, args.verbose)
except qmp.QMPBadPortError:
parser.error(f"Bad port number: {args.qmp_server}")
return # pycharm doesn't know error() is noreturn
@@ -438,7 +435,6 @@ def main():
die(f"Couldn't connect to {args.qmp_server}: {err!s}")
qemu.show_banner()
-qemu.set_verbosity(args.verbose)
while qemu.read_exec_command(qemu.get_prompt()):
pass
qemu.close()
--
2.31.1
[PULL 54/72] scripts/qmp-shell: refactor QMPCompleter
list is a generic type, but we expect to use strings directly. We could subclass list[str], but pylint does not presently understand that invocation. Change this class to envelop a list instead of *being* a list, for simpler mypy typing. Signed-off-by: John Snow Message-id: 20210607200649.1840382-25-js...@redhat.com Signed-off-by: John Snow --- scripts/qmp/qmp-shell | 14 +++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell index 847d34890f..73694035b2 100755 --- a/scripts/qmp/qmp-shell +++ b/scripts/qmp/qmp-shell @@ -78,9 +78,17 @@ sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python')) from qemu import qmp -class QMPCompleter(list): -def complete(self, text, state): -for cmd in self: +class QMPCompleter: +# NB: Python 3.9+ will probably allow us to subclass list[str] directly, +# but pylint as of today does not know that List[str] is simply 'list'. +def __init__(self) -> None: +self._matches: List[str] = [] + +def append(self, value: str) -> None: +return self._matches.append(value) + +def complete(self, text: str, state: int) -> Optional[str]: +for cmd in self._matches: if cmd.startswith(text): if state == 0: return cmd -- 2.31.1
[PULL 53/72] scripts/qmp-shell: Fix "FuzzyJSON" parser
I'm not sure when this regressed (Or maybe if it was ever working right to begin with?), but the Python AST requires you to change "Names" to "Constants" in order to truly convert `false` to `False`. Signed-off-by: John Snow Message-id: 20210607200649.1840382-24-js...@redhat.com Signed-off-by: John Snow --- scripts/qmp/qmp-shell | 20 ++-- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell index aa148517a8..847d34890f 100755 --- a/scripts/qmp/qmp-shell +++ b/scripts/qmp/qmp-shell @@ -95,18 +95,19 @@ class QMPShellError(Exception): class FuzzyJSON(ast.NodeTransformer): """ This extension of ast.NodeTransformer filters literal "true/false/null" -values in an AST and replaces them by proper "True/False/None" values that -Python can properly evaluate. +values in a Python AST and replaces them by proper "True/False/None" values +that Python can properly evaluate. """ @classmethod -def visit_Name(cls, node): # pylint: disable=invalid-name +def visit_Name(cls, # pylint: disable=invalid-name + node: ast.Name) -> ast.AST: if node.id == 'true': -node.id = 'True' +return ast.Constant(value=True) if node.id == 'false': -node.id = 'False' +return ast.Constant(value=False) if node.id == 'null': -node.id = 'None' +return ast.Constant(value=None) return node @@ -174,10 +175,9 @@ class QMPShell(qmp.QEMUMonitorProtocol): # Try once again as FuzzyJSON: try: tree = ast.parse(val, mode='eval') -return ast.literal_eval(FuzzyJSON().visit(tree)) -except SyntaxError: -pass -except ValueError: +transformed = FuzzyJSON().visit(tree) +return ast.literal_eval(transformed) +except (SyntaxError, ValueError): pass return val -- 2.31.1
[PULL 57/72] scripts/qmp-shell: add mypy types
As per my usual, this patch is annotations only. Any changes with side
effects are done elsewhere.
Note: pylint does not understand the subscripts for Collection in Python 3.6,
so use the stronger Sequence type as a workaround.
Signed-off-by: John Snow
Message-id: 20210607200649.1840382-28-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qmp-shell | 67 ++-
1 file changed, 41 insertions(+), 26 deletions(-)
diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell
index 670361322c..2d0e85b5f7 100755
--- a/scripts/qmp/qmp-shell
+++ b/scripts/qmp/qmp-shell
@@ -72,10 +72,18 @@ import os
import re
import readline
import sys
+from typing import (
+Iterator,
+List,
+NoReturn,
+Optional,
+Sequence,
+)
sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
from qemu import qmp
+from qemu.qmp import QMPMessage
class QMPCompleter:
@@ -122,25 +130,26 @@ class FuzzyJSON(ast.NodeTransformer):
# TODO: QMPShell's interface is a bit ugly (eg. _fill_completion() and
# _execute_cmd()). Let's design a better one.
class QMPShell(qmp.QEMUMonitorProtocol):
-def __init__(self, address, pretty=False, verbose=False):
+def __init__(self, address: str, pretty: bool = False,
+ verbose: bool = False):
super().__init__(self.parse_address(address))
-self._greeting = None
+self._greeting: Optional[QMPMessage] = None
self._completer = QMPCompleter()
self._pretty = pretty
self._transmode = False
-self._actions = list()
+self._actions: List[QMPMessage] = []
self._histfile = os.path.join(os.path.expanduser('~'),
'.qmp-shell_history')
self.verbose = verbose
-def _fill_completion(self):
+def _fill_completion(self) -> None:
cmds = self.cmd('query-commands')
if 'error' in cmds:
return
for cmd in cmds['return']:
self._completer.append(cmd['name'])
-def __completer_setup(self):
+def __completer_setup(self) -> None:
self._completer = QMPCompleter()
self._fill_completion()
readline.set_history_length(1024)
@@ -157,14 +166,14 @@ class QMPShell(qmp.QEMUMonitorProtocol):
print(f"Failed to read history '{self._histfile}': {err!s}")
atexit.register(self.__save_history)
-def __save_history(self):
+def __save_history(self) -> None:
try:
readline.write_history_file(self._histfile)
except IOError as err:
print(f"Failed to save history file '{self._histfile}': {err!s}")
@classmethod
-def __parse_value(cls, val):
+def __parse_value(cls, val: str) -> object:
try:
return int(val)
except ValueError:
@@ -189,7 +198,9 @@ class QMPShell(qmp.QEMUMonitorProtocol):
pass
return val
-def __cli_expr(self, tokens, parent):
+def __cli_expr(self,
+ tokens: Sequence[str],
+ parent: qmp.QMPObject) -> None:
for arg in tokens:
(key, sep, val) = arg.partition('=')
if sep != '=':
@@ -215,7 +226,7 @@ class QMPShell(qmp.QEMUMonitorProtocol):
raise QMPShellError(f'Cannot set "{key}" multiple times')
parent[optpath[-1]] = value
-def __build_cmd(self, cmdline):
+def __build_cmd(self, cmdline: str) -> Optional[QMPMessage]:
"""
Build a QMP input object from a user provided command-line in the
following format:
@@ -224,6 +235,7 @@ class QMPShell(qmp.QEMUMonitorProtocol):
"""
argument_regex = r'''(?:[^\s"']|"(?:\\.|[^"])*"|'(?:\\.|[^'])*')+'''
cmdargs = re.findall(argument_regex, cmdline)
+qmpcmd: QMPMessage
# Transactional CLI entry/exit:
if cmdargs[0] == 'transaction(':
@@ -261,14 +273,14 @@ class QMPShell(qmp.QEMUMonitorProtocol):
self.__cli_expr(cmdargs[1:], qmpcmd['arguments'])
return qmpcmd
-def _print(self, qmp_message):
+def _print(self, qmp_message: object) -> None:
indent = None
if self._pretty:
indent = 4
jsobj = json.dumps(qmp_message, indent=indent, sort_keys=self._pretty)
print(str(jsobj))
-def _execute_cmd(self, cmdline):
+def _execute_cmd(self, cmdline: str) -> bool:
try:
qmpcmd = self.__build_cmd(cmdline)
except Exception as err:
@@ -288,11 +300,12 @@ class QMPShell(qmp.QEMUMonitorProtocol):
self._print(resp)
return True
-def connect(self, negotiate: bool = True):
+def connect(self, negotiate: bool = True) -> None:
self._greeting = super().connect(negotiate)
self.__completer_setup()
-def show_banner(self, msg='Welcome to the QMP low-level shell!'):
+def show_banner(self,
+
[PULL 50/72] scripts/qmp-shell: move get_prompt() to prompt property
Small tidying; treat "prompt" like an immutable property instead of
function/method/routine.
Signed-off-by: John Snow
Message-id: 20210607200649.1840382-21-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qmp-shell | 9 +
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell
index cfcefb95f9..3b86ef7d88 100755
--- a/scripts/qmp/qmp-shell
+++ b/scripts/qmp/qmp-shell
@@ -292,10 +292,11 @@ class QMPShell(qmp.QEMUMonitorProtocol):
version = self._greeting['QMP']['version']['qemu']
print("Connected to QEMU {major}.{minor}.{micro}\n".format(**version))
-def get_prompt(self):
+@property
+def prompt(self):
if self._transmode:
-return "TRANS> "
-return "(QEMU) "
+return 'TRANS> '
+return '(QEMU) '
def read_exec_command(self, prompt):
"""
@@ -435,7 +436,7 @@ def main():
die(f"Couldn't connect to {args.qmp_server}: {err!s}")
qemu.show_banner()
-while qemu.read_exec_command(qemu.get_prompt()):
+while qemu.read_exec_command(qemu.prompt):
pass
qemu.close()
--
2.31.1
[PULL 44/72] scripts/qmp-shell: fix shell history exception handling
We want to remove exceptions that are too broad here; we only want to
catch IOErrors that get raised as a direct result of the open call.
Signed-off-by: John Snow
Message-id: 20210607200649.1840382-15-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qmp-shell | 15 ++-
1 file changed, 6 insertions(+), 9 deletions(-)
diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell
index afb4b0c544..80cd432607 100755
--- a/scripts/qmp/qmp-shell
+++ b/scripts/qmp/qmp-shell
@@ -67,7 +67,6 @@
import ast
import atexit
-import errno
import json
import os
import re
@@ -143,19 +142,17 @@ class QMPShell(qmp.QEMUMonitorProtocol):
readline.set_completer_delims('')
try:
readline.read_history_file(self._histfile)
-except Exception as e:
-if isinstance(e, IOError) and e.errno == errno.ENOENT:
-# File not found. No problem.
-pass
-else:
-print("Failed to read history '%s'; %s" % (self._histfile, e))
+except FileNotFoundError:
+pass
+except IOError as err:
+print(f"Failed to read history '{self._histfile}': {err!s}")
atexit.register(self.__save_history)
def __save_history(self):
try:
readline.write_history_file(self._histfile)
-except Exception as e:
-print("Failed to save history file '%s'; %s" % (self._histfile, e))
+except IOError as err:
+print(f"Failed to save history file '{self._histfile}': {err!s}")
@classmethod
def __parse_value(cls, val):
--
2.31.1
[PULL 51/72] scripts/qmp-shell: remove prompt argument from read_exec_command
It's only ever used by one caller, we can just absorb that logic.
Signed-off-by: John Snow
Message-id: 20210607200649.1840382-22-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qmp-shell | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell
index 3b86ef7d88..31269859c4 100755
--- a/scripts/qmp/qmp-shell
+++ b/scripts/qmp/qmp-shell
@@ -298,14 +298,14 @@ class QMPShell(qmp.QEMUMonitorProtocol):
return 'TRANS> '
return '(QEMU) '
-def read_exec_command(self, prompt):
+def read_exec_command(self):
"""
Read and execute a command.
@return True if execution was ok, return False if disconnected.
"""
try:
-cmdline = input(prompt)
+cmdline = input(self.prompt)
except EOFError:
print()
return False
@@ -436,7 +436,7 @@ def main():
die(f"Couldn't connect to {args.qmp_server}: {err!s}")
qemu.show_banner()
-while qemu.read_exec_command(qemu.prompt):
+while qemu.read_exec_command():
pass
qemu.close()
--
2.31.1
[PULL 55/72] scripts/qmp-shell: initialize completer early
Add an empty completer as a more type-safe placeholder instead of 'None'. Signed-off-by: John Snow Message-id: 20210607200649.1840382-26-js...@redhat.com Signed-off-by: John Snow --- scripts/qmp/qmp-shell | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell index 73694035b2..670361322c 100755 --- a/scripts/qmp/qmp-shell +++ b/scripts/qmp/qmp-shell @@ -125,7 +125,7 @@ class QMPShell(qmp.QEMUMonitorProtocol): def __init__(self, address, pretty=False, verbose=False): super().__init__(self.parse_address(address)) self._greeting = None -self._completer = None +self._completer = QMPCompleter() self._pretty = pretty self._transmode = False self._actions = list() -- 2.31.1
[PULL 48/72] scripts/qmp-shell: Add pretty attribute to HMP shell
It's less useful, but it makes the initialization methods LSP
consistent, which quiets a mypy complaint.
Signed-off-by: John Snow
Message-id: 20210607200649.1840382-19-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qmp-shell | 11 ---
1 file changed, 4 insertions(+), 7 deletions(-)
diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell
index 5317dcd516..de5fa189f0 100755
--- a/scripts/qmp/qmp-shell
+++ b/scripts/qmp/qmp-shell
@@ -322,8 +322,8 @@ class QMPShell(qmp.QEMUMonitorProtocol):
class HMPShell(QMPShell):
-def __init__(self, address):
-super().__init__(address)
+def __init__(self, address, pretty=False):
+super().__init__(address, pretty)
self.__cpu_index = 0
def __cmd_completion(self):
@@ -421,12 +421,9 @@ def main():
if args.qmp_server is None:
parser.error("QMP socket or TCP address must be specified")
-qemu: QMPShell
+shell_class = HMPShell if args.hmp else QMPShell
try:
-if args.hmp:
-qemu = HMPShell(args.qmp_server)
-else:
-qemu = QMPShell(args.qmp_server, args.pretty)
+qemu = shell_class(args.qmp_server, args.pretty)
except qmp.QMPBadPortError:
parser.error(f"Bad port number: {args.qmp_server}")
return # pycharm doesn't know error() is noreturn
--
2.31.1
[PULL 41/72] scripts/qmp-shell: ignore visit_Name name
Not something I control, sorry, pylint. Signed-off-by: John Snow Message-id: 20210607200649.1840382-12-js...@redhat.com Signed-off-by: John Snow --- scripts/qmp/qmp-shell | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell index c46f4f516b..ea6a87e0b3 100755 --- a/scripts/qmp/qmp-shell +++ b/scripts/qmp/qmp-shell @@ -101,7 +101,7 @@ class FuzzyJSON(ast.NodeTransformer): """ @classmethod -def visit_Name(cls, node): +def visit_Name(cls, node): # pylint: disable=invalid-name if node.id == 'true': node.id = 'True' if node.id == 'false': -- 2.31.1
[PULL 40/72] scripts/qmp-shell: use triple-double-quote docstring style
(2014 me had never written python before.) Signed-off-by: John Snow Message-id: 20210607200649.1840382-11-js...@redhat.com Signed-off-by: John Snow --- scripts/qmp/qmp-shell | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell index 4027454324..c46f4f516b 100755 --- a/scripts/qmp/qmp-shell +++ b/scripts/qmp/qmp-shell @@ -94,9 +94,12 @@ class QMPShellError(Exception): class FuzzyJSON(ast.NodeTransformer): -'''This extension of ast.NodeTransformer filters literal "true/false/null" +""" +This extension of ast.NodeTransformer filters literal "true/false/null" values in an AST and replaces them by proper "True/False/None" values that -Python can properly evaluate.''' +Python can properly evaluate. +""" + @classmethod def visit_Name(cls, node): if node.id == 'true': -- 2.31.1
[PULL 45/72] scripts/qmp-shell: remove if-raise-else patterns
Shushes pylint. I don't always mind these patterns personally, but I'm
not as sure that I want to remove the warning from pylint's repertoire
entirely. Oh well.
Signed-off-by: John Snow
Message-id: 20210607200649.1840382-16-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qmp-shell | 8
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell
index 80cd432607..bf7a49dfc1 100755
--- a/scripts/qmp/qmp-shell
+++ b/scripts/qmp/qmp-shell
@@ -204,8 +204,7 @@ class QMPShell(qmp.QEMUMonitorProtocol):
if type(parent[optpath[-1]]) is dict:
msg = 'Cannot use "{:s}" as both leaf and non-leaf key'
raise QMPShellError(msg.format('.'.join(curpath)))
-else:
-raise QMPShellError(f'Cannot set "{key}" multiple times')
+raise QMPShellError(f'Cannot set "{key}" multiple times')
parent[optpath[-1]] = value
def __build_cmd(self, cmdline):
@@ -309,13 +308,14 @@ class QMPShell(qmp.QEMUMonitorProtocol):
except EOFError:
print()
return False
+
if cmdline == '':
for event in self.get_events():
print(event)
self.clear_events()
return True
-else:
-return self._execute_cmd(cmdline)
+
+return self._execute_cmd(cmdline)
def set_verbosity(self, verbose):
self._verbose = verbose
--
2.31.1
[PULL 37/72] scripts/qmp-shell: use @classmethod where appropriate
Methods with no self-use should belong to the class.
Signed-off-by: John Snow
Message-id: 20210607200649.1840382-8-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qmp-shell | 6 --
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell
index ae3f04534a..f354549bf2 100755
--- a/scripts/qmp/qmp-shell
+++ b/scripts/qmp/qmp-shell
@@ -97,7 +97,8 @@ class FuzzyJSON(ast.NodeTransformer):
'''This extension of ast.NodeTransformer filters literal "true/false/null"
values in an AST and replaces them by proper "True/False/None" values that
Python can properly evaluate.'''
-def visit_Name(self, node):
+@classmethod
+def visit_Name(cls, node):
if node.id == 'true':
node.id = 'True'
if node.id == 'false':
@@ -152,7 +153,8 @@ class QMPShell(qmp.QEMUMonitorProtocol):
except Exception as e:
print("Failed to save history file '%s'; %s" % (self._histfile, e))
-def __parse_value(self, val):
+@classmethod
+def __parse_value(cls, val):
try:
return int(val)
except ValueError:
--
2.31.1
[PULL 30/72] scripts/qemu-ga-client: Add forwarder shim
Add a little forwarder shim until we are sure that everyone is comfortable with how to use the tools in their new packaged location. Signed-off-by: John Snow Message-id: 20210604155532.1499282-12-js...@redhat.com Signed-off-by: John Snow --- scripts/qmp/qemu-ga-client | 11 +++ 1 file changed, 11 insertions(+) create mode 100755 scripts/qmp/qemu-ga-client diff --git a/scripts/qmp/qemu-ga-client b/scripts/qmp/qemu-ga-client new file mode 100755 index 00..102fd2cad9 --- /dev/null +++ b/scripts/qmp/qemu-ga-client @@ -0,0 +1,11 @@ +#!/usr/bin/env python3 + +import os +import sys + +sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python')) +from qemu.qmp import qemu_ga_client + + +if __name__ == '__main__': +sys.exit(qemu_ga_client.main()) -- 2.31.1
[PULL 43/72] scripts/qmp-shell: rename one and two-letter variables
A bit of churn and housekeeping for pylint, flake8 et al.
Signed-off-by: John Snow
Message-id: 20210607200649.1840382-14-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qmp-shell | 24
1 file changed, 12 insertions(+), 12 deletions(-)
diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell
index 8d84467b53..afb4b0c544 100755
--- a/scripts/qmp/qmp-shell
+++ b/scripts/qmp/qmp-shell
@@ -176,8 +176,8 @@ class QMPShell(qmp.QEMUMonitorProtocol):
pass
# Try once again as FuzzyJSON:
try:
-st = ast.parse(val, mode='eval')
-return ast.literal_eval(FuzzyJSON().visit(st))
+tree = ast.parse(val, mode='eval')
+return ast.literal_eval(FuzzyJSON().visit(tree))
except SyntaxError:
pass
except ValueError:
@@ -195,14 +195,14 @@ class QMPShell(qmp.QEMUMonitorProtocol):
value = self.__parse_value(val)
optpath = key.split('.')
curpath = []
-for p in optpath[:-1]:
-curpath.append(p)
-d = parent.get(p, {})
-if type(d) is not dict:
+for path in optpath[:-1]:
+curpath.append(path)
+obj = parent.get(path, {})
+if type(obj) is not dict:
msg = 'Cannot use "{:s}" as both leaf and non-leaf key'
raise QMPShellError(msg.format('.'.join(curpath)))
-parent[p] = d
-parent = d
+parent[path] = obj
+parent = obj
if optpath[-1] in parent:
if type(parent[optpath[-1]]) is dict:
msg = 'Cannot use "{:s}" as both leaf and non-leaf key'
@@ -267,8 +267,8 @@ class QMPShell(qmp.QEMUMonitorProtocol):
def _execute_cmd(self, cmdline):
try:
qmpcmd = self.__build_cmd(cmdline)
-except Exception as e:
-print('Error while parsing command line: %s' % e)
+except Exception as err:
+print('Error while parsing command line: %s' % err)
print('command format: ', end=' ')
print('[arg-name1=arg1] ... [arg-nameN=argN]')
return True
@@ -313,8 +313,8 @@ class QMPShell(qmp.QEMUMonitorProtocol):
print()
return False
if cmdline == '':
-for ev in self.get_events():
-print(ev)
+for event in self.get_events():
+print(event)
self.clear_events()
return True
else:
--
2.31.1
[PULL 47/72] scripts/qmp-shell: use argparse
Use argparse instead of an open-coded CLI parser, for consistency with
everything else.
Signed-off-by: John Snow
Message-id: 20210607200649.1840382-18-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qmp-shell | 82 +--
1 file changed, 32 insertions(+), 50 deletions(-)
diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell
index 970f43dd00..5317dcd516 100755
--- a/scripts/qmp/qmp-shell
+++ b/scripts/qmp/qmp-shell
@@ -64,7 +64,7 @@
# Use the -v and -p options to activate the verbose and pretty-print options,
# which will echo back the properly formatted JSON-compliant QMP that is being
# sent to QEMU, which is useful for debugging and documentation generation.
-
+import argparse
import ast
import atexit
import json
@@ -401,65 +401,47 @@ def die(msg):
sys.exit(1)
-def fail_cmdline(option=None):
-if option:
-sys.stderr.write('ERROR: bad command-line option \'%s\'\n' % option)
-sys.stderr.write(
-'qmp-shell [ -v ] [ -p ] [ -H ] [ -N ] '
-'< UNIX socket path> | < TCP address:port >\n'
-)
-sys.stderr.write('-v Verbose (echo command sent and received)\n')
-sys.stderr.write('-p Pretty-print JSON\n')
-sys.stderr.write('-H Use HMP interface\n')
-sys.stderr.write('-N Skip negotiate (for qemu-ga)\n')
-sys.exit(1)
-
-
def main():
-addr = ''
-qemu = None
-hmp = False
-pretty = False
-verbose = False
-negotiate = True
+parser = argparse.ArgumentParser()
+parser.add_argument('-H', '--hmp', action='store_true',
+help='Use HMP interface')
+parser.add_argument('-N', '--skip-negotiation', action='store_true',
+help='Skip negotiate (for qemu-ga)')
+parser.add_argument('-v', '--verbose', action='store_true',
+help='Verbose (echo commands sent and received)')
+parser.add_argument('-p', '--pretty', action='store_true',
+help='Pretty-print JSON')
+default_server = os.environ.get('QMP_SOCKET')
+parser.add_argument('qmp_server', action='store',
+default=default_server,
+help='< UNIX socket path | TCP address:port >')
+
+args = parser.parse_args()
+if args.qmp_server is None:
+parser.error("QMP socket or TCP address must be specified")
+
+qemu: QMPShell
try:
-for arg in sys.argv[1:]:
-if arg == "-H":
-if qemu is not None:
-fail_cmdline(arg)
-hmp = True
-elif arg == "-p":
-pretty = True
-elif arg == "-N":
-negotiate = False
-elif arg == "-v":
-verbose = True
-else:
-if qemu is not None:
-fail_cmdline(arg)
-if hmp:
-qemu = HMPShell(arg)
-else:
-qemu = QMPShell(arg, pretty)
-addr = arg
-
-if qemu is None:
-fail_cmdline()
+if args.hmp:
+qemu = HMPShell(args.qmp_server)
+else:
+qemu = QMPShell(args.qmp_server, args.pretty)
except qmp.QMPBadPortError:
-die('bad port number in command-line')
+parser.error(f"Bad port number: {args.qmp_server}")
+return # pycharm doesn't know error() is noreturn
try:
-qemu.connect(negotiate)
+qemu.connect(negotiate=not args.skip_negotiation)
except qmp.QMPConnectError:
-die('Didn\'t get QMP greeting message')
+die("Didn't get QMP greeting message")
except qmp.QMPCapabilitiesError:
-die('Could not negotiate capabilities')
-except OSError:
-die('Could not connect to %s' % addr)
+die("Couldn't negotiate capabilities")
+except OSError as err:
+die(f"Couldn't connect to {args.qmp_server}: {err!s}")
qemu.show_banner()
-qemu.set_verbosity(verbose)
+qemu.set_verbosity(args.verbose)
while qemu.read_exec_command(qemu.get_prompt()):
pass
qemu.close()
--
2.31.1
[PULL 29/72] python/qemu-ga-client: add entry point
Remove the shebang, and add a package-defined entry point instead. Now, it can be accessed using 'qemu-ga-client' from the command line after installing the package. The next commit adds a forwarder shim that allows the running of this script without needing to install the package again. Signed-off-by: John Snow Message-id: 20210604155532.1499282-11-js...@redhat.com Signed-off-by: John Snow --- python/qemu/qmp/qemu_ga_client.py | 2 -- python/setup.cfg | 1 + 2 files changed, 1 insertion(+), 2 deletions(-) mode change 100755 => 100644 python/qemu/qmp/qemu_ga_client.py diff --git a/python/qemu/qmp/qemu_ga_client.py b/python/qemu/qmp/qemu_ga_client.py old mode 100755 new mode 100644 index d2938ad47c..67ac0b4211 --- a/python/qemu/qmp/qemu_ga_client.py +++ b/python/qemu/qmp/qemu_ga_client.py @@ -1,5 +1,3 @@ -#!/usr/bin/env python3 - """ QEMU Guest Agent Client diff --git a/python/setup.cfg b/python/setup.cfg index 6b6be8b03c..7f3c59d74e 100644 --- a/python/setup.cfg +++ b/python/setup.cfg @@ -50,6 +50,7 @@ console_scripts = qom-list = qemu.qmp.qom:QOMList.entry_point qom-tree = qemu.qmp.qom:QOMTree.entry_point qom-fuse = qemu.qmp.qom_fuse:QOMFuse.entry_point [fuse] +qemu-ga-client = qemu.qmp.qemu_ga_client:main [flake8] extend-ignore = E722 # Prefer pylint's bare-except checks to flake8's -- 2.31.1
[PULL 42/72] scripts/qmp-shell: make QMPCompleter returns explicit
This function returns None when it doesn't find a match; do that explicitly. Signed-off-by: John Snow Message-id: 20210607200649.1840382-13-js...@redhat.com Signed-off-by: John Snow --- scripts/qmp/qmp-shell | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell index ea6a87e0b3..8d84467b53 100755 --- a/scripts/qmp/qmp-shell +++ b/scripts/qmp/qmp-shell @@ -83,10 +83,10 @@ class QMPCompleter(list): def complete(self, text, state): for cmd in self: if cmd.startswith(text): -if not state: +if state == 0: return cmd -else: -state -= 1 +state -= 1 +return None class QMPShellError(Exception): -- 2.31.1
[PULL 34/72] scripts/qmp-shell: fix exception handling
Fixes: 50d189c
Signed-off-by: John Snow
Message-id: 20210607200649.1840382-5-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qmp-shell | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell
index 18bf49bb26..413dd4d2de 100755
--- a/scripts/qmp/qmp-shell
+++ b/scripts/qmp/qmp-shell
@@ -452,7 +452,7 @@ def main():
die('Didn\'t get QMP greeting message')
except qmp.QMPCapabilitiesError:
die('Could not negotiate capabilities')
-except qemu.error:
+except OSError:
die('Could not connect to %s' % addr)
qemu.show_banner()
--
2.31.1
[PULL 46/72] scripts/qmp-shell: use isinstance() instead of type()
A bit more idiomatic, and quiets some linter warnings.
Signed-off-by: John Snow
Message-id: 20210607200649.1840382-17-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qmp-shell | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell
index bf7a49dfc1..970f43dd00 100755
--- a/scripts/qmp/qmp-shell
+++ b/scripts/qmp/qmp-shell
@@ -195,13 +195,13 @@ class QMPShell(qmp.QEMUMonitorProtocol):
for path in optpath[:-1]:
curpath.append(path)
obj = parent.get(path, {})
-if type(obj) is not dict:
+if not isinstance(obj, dict):
msg = 'Cannot use "{:s}" as both leaf and non-leaf key'
raise QMPShellError(msg.format('.'.join(curpath)))
parent[path] = obj
parent = obj
if optpath[-1] in parent:
-if type(parent[optpath[-1]]) is dict:
+if isinstance(parent[optpath[-1]], dict):
msg = 'Cannot use "{:s}" as both leaf and non-leaf key'
raise QMPShellError(msg.format('.'.join(curpath)))
raise QMPShellError(f'Cannot set "{key}" multiple times')
--
2.31.1
[PULL 36/72] scripts/qmp-shell: remove shadowed variable from _print()
Don't use 'qmp' here, which shadows the qmp module. Signed-off-by: John Snow Message-id: 20210607200649.1840382-7-js...@redhat.com Signed-off-by: John Snow --- scripts/qmp/qmp-shell | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell index 04ca6a25ae..ae3f04534a 100755 --- a/scripts/qmp/qmp-shell +++ b/scripts/qmp/qmp-shell @@ -251,11 +251,11 @@ class QMPShell(qmp.QEMUMonitorProtocol): self.__cli_expr(cmdargs[1:], qmpcmd['arguments']) return qmpcmd -def _print(self, qmp): +def _print(self, qmp_message): indent = None if self._pretty: indent = 4 -jsobj = json.dumps(qmp, indent=indent, sort_keys=self._pretty) +jsobj = json.dumps(qmp_message, indent=indent, sort_keys=self._pretty) print(str(jsobj)) def _execute_cmd(self, cmdline): -- 2.31.1
[PULL 28/72] scripts/qemu-ga-client: move to python/qemu/qmp/qemu_ga_client.py
The script itself will be unavailable for a few commits before being restored, with no way to run it right after this commit. This helps move git history into the new file. To prevent linter regressions, though, we do need to immediately touch up the filename to remove dashes (to make the module importable), and remove the executable bit. Signed-off-by: John Snow Message-id: 20210604155532.1499282-10-js...@redhat.com Signed-off-by: John Snow --- scripts/qmp/qemu-ga-client => python/qemu/qmp/qemu_ga_client.py | 2 -- 1 file changed, 2 deletions(-) rename scripts/qmp/qemu-ga-client => python/qemu/qmp/qemu_ga_client.py (99%) diff --git a/scripts/qmp/qemu-ga-client b/python/qemu/qmp/qemu_ga_client.py similarity index 99% rename from scripts/qmp/qemu-ga-client rename to python/qemu/qmp/qemu_ga_client.py index a7d0ef8347..d2938ad47c 100755 --- a/scripts/qmp/qemu-ga-client +++ b/python/qemu/qmp/qemu_ga_client.py @@ -52,8 +52,6 @@ Sequence, ) - -sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python')) from qemu import qmp from qemu.qmp import SocketAddrT -- 2.31.1
[PULL 38/72] scripts/qmp-shell: Use python3-style super()
Signed-off-by: John Snow Message-id: 20210607200649.1840382-9-js...@redhat.com Signed-off-by: John Snow --- scripts/qmp/qmp-shell | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell index f354549bf2..3066e37ae5 100755 --- a/scripts/qmp/qmp-shell +++ b/scripts/qmp/qmp-shell @@ -112,7 +112,7 @@ class FuzzyJSON(ast.NodeTransformer): # _execute_cmd()). Let's design a better one. class QMPShell(qmp.QEMUMonitorProtocol): def __init__(self, address, pretty=False): -super(QMPShell, self).__init__(self.parse_address(address)) +super().__init__(self.parse_address(address)) self._greeting = None self._completer = None self._pretty = pretty @@ -281,7 +281,7 @@ class QMPShell(qmp.QEMUMonitorProtocol): return True def connect(self, negotiate: bool = True): -self._greeting = super(QMPShell, self).connect(negotiate) +self._greeting = super().connect(negotiate) self.__completer_setup() def show_banner(self, msg='Welcome to the QMP low-level shell!'): @@ -322,7 +322,7 @@ class QMPShell(qmp.QEMUMonitorProtocol): class HMPShell(QMPShell): def __init__(self, address): -QMPShell.__init__(self, address) +super().__init__(address) self.__cpu_index = 0 def __cmd_completion(self): -- 2.31.1
[PULL 39/72] scripts/qmp-shell: declare verbose in __init__
Linters get angry when we don't define state at init time.
Signed-off-by: John Snow
Message-id: 20210607200649.1840382-10-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qmp-shell | 1 +
1 file changed, 1 insertion(+)
diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell
index 3066e37ae5..4027454324 100755
--- a/scripts/qmp/qmp-shell
+++ b/scripts/qmp/qmp-shell
@@ -120,6 +120,7 @@ class QMPShell(qmp.QEMUMonitorProtocol):
self._actions = list()
self._histfile = os.path.join(os.path.expanduser('~'),
'.qmp-shell_history')
+self._verbose = False
def _fill_completion(self):
cmds = self.cmd('query-commands')
--
2.31.1
[PULL 33/72] scripts/qmp-shell: fix show_banner signature
The signatures need to match.
Signed-off-by: John Snow
Message-id: 20210607200649.1840382-4-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qmp-shell | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell
index 62a6377e06..18bf49bb26 100755
--- a/scripts/qmp/qmp-shell
+++ b/scripts/qmp/qmp-shell
@@ -389,8 +389,8 @@ class HMPShell(QMPShell):
print('%s: %s' % (resp['error']['class'], resp['error']['desc']))
return True
-def show_banner(self):
-QMPShell.show_banner(self, msg='Welcome to the HMP shell!')
+def show_banner(self, msg='Welcome to the HMP shell!'):
+QMPShell.show_banner(self, msg)
def die(msg):
--
2.31.1
[PULL 32/72] scripts/qmp-shell: Apply flake8 rules
A lot of fiddling around to get us below 80 columns.
Signed-off-by: John Snow
Message-id: 20210607200649.1840382-3-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qmp-shell | 64 +--
1 file changed, 43 insertions(+), 21 deletions(-)
diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell
index a00efe6fea..62a6377e06 100755
--- a/scripts/qmp/qmp-shell
+++ b/scripts/qmp/qmp-shell
@@ -88,6 +88,7 @@ class QMPCompleter(list):
else:
state -= 1
+
class QMPShellError(Exception):
pass
@@ -105,6 +106,7 @@ class FuzzyJSON(ast.NodeTransformer):
node.id = 'None'
return node
+
# TODO: QMPShell's interface is a bit ugly (eg. _fill_completion() and
# _execute_cmd()). Let's design a better one.
class QMPShell(qmp.QEMUMonitorProtocol):
@@ -131,8 +133,8 @@ class QMPShell(qmp.QEMUMonitorProtocol):
readline.set_history_length(1024)
readline.set_completer(self._completer.complete)
readline.parse_and_bind("tab: complete")
-# XXX: default delimiters conflict with some command names (eg.
query-),
-# clearing everything as it doesn't seem to matter
+# NB: default delimiters conflict with some command names
+# (eg. query-), clearing everything as it doesn't seem to matter
readline.set_completer_delims('')
try:
readline.read_history_file(self._histfile)
@@ -180,7 +182,9 @@ class QMPShell(qmp.QEMUMonitorProtocol):
for arg in tokens:
(key, sep, val) = arg.partition('=')
if sep != '=':
-raise QMPShellError("Expected a key=value pair, got '%s'" %
arg)
+raise QMPShellError(
+f"Expected a key=value pair, got '{arg!s}'"
+)
value = self.__parse_value(val)
optpath = key.split('.')
@@ -189,14 +193,16 @@ class QMPShell(qmp.QEMUMonitorProtocol):
curpath.append(p)
d = parent.get(p, {})
if type(d) is not dict:
-raise QMPShellError('Cannot use "%s" as both leaf and
non-leaf key' % '.'.join(curpath))
+msg = 'Cannot use "{:s}" as both leaf and non-leaf key'
+raise QMPShellError(msg.format('.'.join(curpath)))
parent[p] = d
parent = d
if optpath[-1] in parent:
if type(parent[optpath[-1]]) is dict:
-raise QMPShellError('Cannot use "%s" as both leaf and
non-leaf key' % '.'.join(curpath))
+msg = 'Cannot use "{:s}" as both leaf and non-leaf key'
+raise QMPShellError(msg.format('.'.join(curpath)))
else:
-raise QMPShellError('Cannot set "%s" multiple times' % key)
+raise QMPShellError(f'Cannot set "{key}" multiple times')
parent[optpath[-1]] = value
def __build_cmd(self, cmdline):
@@ -206,7 +212,8 @@ class QMPShell(qmp.QEMUMonitorProtocol):
< command-name > [ arg-name1=arg1 ] ... [ arg-nameN=argN ]
"""
-cmdargs =
re.findall(r'''(?:[^\s"']|"(?:\\.|[^"])*"|'(?:\\.|[^'])*')+''', cmdline)
+argument_regex = r'''(?:[^\s"']|"(?:\\.|[^"])*"|'(?:\\.|[^'])*')+'''
+cmdargs = re.findall(argument_regex, cmdline)
# Transactional CLI entry/exit:
if cmdargs[0] == 'transaction(':
@@ -215,9 +222,12 @@ class QMPShell(qmp.QEMUMonitorProtocol):
elif cmdargs[0] == ')' and self._transmode:
self._transmode = False
if len(cmdargs) > 1:
-raise QMPShellError("Unexpected input after close of
Transaction sub-shell")
-qmpcmd = { 'execute': 'transaction',
- 'arguments': { 'actions': self._actions } }
+msg = 'Unexpected input after close of Transaction sub-shell'
+raise QMPShellError(msg)
+qmpcmd = {
+'execute': 'transaction',
+'arguments': {'actions': self._actions}
+}
self._actions = list()
return qmpcmd
@@ -228,7 +238,7 @@ class QMPShell(qmp.QEMUMonitorProtocol):
# Parse and then cache this Transactional Action
if self._transmode:
finalize = False
-action = { 'type': cmdargs[0], 'data': {} }
+action = {'type': cmdargs[0], 'data': {}}
if cmdargs[-1] == ')':
cmdargs.pop(-1)
finalize = True
@@ -237,7 +247,7 @@ class QMPShell(qmp.QEMUMonitorProtocol):
return self.__build_cmd(')') if finalize else None
# Standard command: parse and return it to be executed.
-qmpcmd = { 'execute': cmdargs[0], 'arguments': {} }
+qmpcmd = {'execute': cmdargs[0], 'arguments': {}}
self.__cli_expr(cmdargs[1:], qmpcmd['arguments'])
[PULL 16/72] python: add optional FUSE dependencies
In preparation for moving qom-fuse over to the python package, we need
some new dependencies to support it.
Add an optional 'fusepy' dependency that users of the package can opt
into with e.g. "pip install qemu[fuse]" which installs the requirements
necessary to obtain the additional functionality.
Add the same fusepy dependency to the 'devel' extras group --
unfortunately I do not see a way for optional groups to imply other
optional groups at present, so the dependency is repeated. The
development group needs to include the full set of dependencies for the
purpose of static analysis of all features offered by this library.
Lastly, add the [fuse] extras group to tox's configuration as a
workaround so that if a stale tox environment is found when running
`make check-tox`, tox will know to rebuild its environments.
Signed-off-by: John Snow
Message-id: 20210603003719.1321369-17-js...@redhat.com
Signed-off-by: John Snow
---
python/Pipfile.lock | 6 ++
python/setup.cfg| 9 -
2 files changed, 14 insertions(+), 1 deletion(-)
diff --git a/python/Pipfile.lock b/python/Pipfile.lock
index f2a3f91d0f..5bb3f1b635 100644
--- a/python/Pipfile.lock
+++ b/python/Pipfile.lock
@@ -67,6 +67,12 @@
"markers": "python_version >= '2.7' and python_version not in
'3.0, 3.1, 3.2, 3.3, 3.4'",
"version": "==3.9.2"
},
+"fusepy": {
+"hashes": [
+
"sha256:72ff783ec2f43de3ab394e3f7457605bf04c8cf288a2f4068b4cde141d4ee6bd"
+],
+"version": "==3.0.1"
+},
"importlib-metadata": {
"hashes": [
"sha256:8c501196e49fb9df5df43833bdb1e4328f64847763ec8a50703148b73784d581",
diff --git a/python/setup.cfg b/python/setup.cfg
index ba8d29fd62..aca6f31185 100644
--- a/python/setup.cfg
+++ b/python/setup.cfg
@@ -32,11 +32,16 @@ packages =
devel =
avocado-framework >= 87.0
flake8 >= 3.6.0
+fusepy >= 2.0.4
isort >= 5.1.2
mypy >= 0.770
pylint >= 2.8.0
tox >= 3.18.0
+# Provides qom-fuse functionality
+fuse =
+fusepy >= 2.0.4
+
[options.entry_points]
console_scripts =
qom = qemu.qmp.qom:main
@@ -114,6 +119,8 @@ envlist = py36, py37, py38, py39, py310
[testenv]
allowlist_externals = make
-deps = .[devel]
+deps =
+.[devel]
+.[fuse] # Workaround to trigger tox venv rebuild
commands =
make check
--
2.31.1
[PULL 26/72] python/qmp: Correct type of QMPReturnValue
It's only a Dict[str, Any] most of the time. It's not actually
guaranteed to be anything in particular. Fix this type to be
more accurate to the reality we live in.
Signed-off-by: John Snow
Message-id: 20210604155532.1499282-8-js...@redhat.com
Signed-off-by: John Snow
---
python/qemu/qmp/__init__.py | 25 +++--
1 file changed, 15 insertions(+), 10 deletions(-)
diff --git a/python/qemu/qmp/__init__.py b/python/qemu/qmp/__init__.py
index 822c793c32..a6e1a7b857 100644
--- a/python/qemu/qmp/__init__.py
+++ b/python/qemu/qmp/__init__.py
@@ -35,14 +35,19 @@
)
-# QMPMessage is a QMP Message of any kind.
-# e.g. {'yee': 'haw'}
-#
-# QMPReturnValue is the inner value of return values only.
-# {'return': {}} is the QMPMessage,
-# {} is the QMPReturnValue.
+#: QMPMessage is an entire QMP message of any kind.
QMPMessage = Dict[str, Any]
-QMPReturnValue = Dict[str, Any]
+
+#: QMPReturnValue is the 'return' value of a command.
+QMPReturnValue = object
+
+# QMPMessage can be outgoing commands or incoming events/returns.
+# QMPReturnValue is usually a dict/json object, but due to QAPI's
+# 'returns-whitelist', it can actually be anything.
+#
+# {'return': {}} is a QMPMessage,
+# {} is the QMPReturnValue.
+
InternetAddrT = Tuple[str, int]
UnixAddrT = str
@@ -297,8 +302,8 @@ def cmd_obj(self, qmp_cmd: QMPMessage) -> QMPMessage:
return resp
def cmd(self, name: str,
-args: Optional[Dict[str, Any]] = None,
-cmd_id: Optional[Any] = None) -> QMPMessage:
+args: Optional[Dict[str, object]] = None,
+cmd_id: Optional[object] = None) -> QMPMessage:
"""
Build a QMP command and send it to the QMP Monitor.
@@ -313,7 +318,7 @@ def cmd(self, name: str,
qmp_cmd['id'] = cmd_id
return self.cmd_obj(qmp_cmd)
-def command(self, cmd: str, **kwds: Any) -> QMPReturnValue:
+def command(self, cmd: str, **kwds: object) -> QMPReturnValue:
"""
Build and send a QMP command to the monitor, report errors if any
"""
--
2.31.1
[PULL 22/72] scripts/qemu-ga-client: Fix exception handling
Fixes: 50d189c
These error classes aren't available anymore. Fix the bitrot.
Signed-off-by: John Snow
Message-id: 20210604155532.1499282-4-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qemu-ga-client | 10 --
1 file changed, 4 insertions(+), 6 deletions(-)
diff --git a/scripts/qmp/qemu-ga-client b/scripts/qmp/qemu-ga-client
index 566bddc89d..7aba09f0fe 100755
--- a/scripts/qmp/qemu-ga-client
+++ b/scripts/qmp/qemu-ga-client
@@ -56,8 +56,6 @@ class QemuGuestAgent(qmp.QEMUMonitorProtocol):
class QemuGuestAgentClient:
-error = QemuGuestAgent.error
-
def __init__(self, address):
self.qga = QemuGuestAgent(address)
self.qga.connect(negotiate=False)
@@ -137,7 +135,7 @@ class QemuGuestAgentClient:
self.qga.settimeout(timeout)
try:
self.qga.ping()
-except self.qga.timeout:
+except TimeoutError:
return False
return True
@@ -269,11 +267,11 @@ def main(address, cmd, args):
try:
client = QemuGuestAgentClient(address)
-except QemuGuestAgent.error as e:
+except OSError as err:
import errno
-print(e)
-if e.errno == errno.ECONNREFUSED:
+print(err)
+if err.errno == errno.ECONNREFUSED:
print('Hint: qemu is not running?')
sys.exit(1)
--
2.31.1
[PULL 21/72] scripts/qemu-ga-client: apply (most) flake8 rules
- Line length should be < 80
- You shouldn't perform unscoped imports except at the top of the module
Notably, the sys.path hack creates problems with the import rule. This
will be fixed later.
Signed-off-by: John Snow
Message-id: 20210604155532.1499282-3-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qemu-ga-client | 28
1 file changed, 16 insertions(+), 12 deletions(-)
diff --git a/scripts/qmp/qemu-ga-client b/scripts/qmp/qemu-ga-client
index 97f4047a62..566bddc89d 100755
--- a/scripts/qmp/qemu-ga-client
+++ b/scripts/qmp/qemu-ga-client
@@ -12,7 +12,8 @@
# Start QEMU with:
#
# # qemu [...] -chardev socket,path=/tmp/qga.sock,server=on,wait=off,id=qga0 \
-# -device virtio-serial -device
virtserialport,chardev=qga0,name=org.qemu.guest_agent.0
+# -device virtio-serial \
+# -device virtserialport,chardev=qga0,name=org.qemu.guest_agent.0
#
# Run the script:
#
@@ -37,6 +38,7 @@
#
import base64
+import optparse
import os
import random
import sys
@@ -94,9 +96,11 @@ class QemuGuestAgentClient:
msgs = []
msgs.append('version: ' + info['version'])
msgs.append('supported_commands:')
-enabled = [c['name'] for c in info['supported_commands'] if
c['enabled']]
+enabled = [c['name'] for c in info['supported_commands']
+ if c['enabled']]
msgs.append('\tenabled: ' + ', '.join(enabled))
-disabled = [c['name'] for c in info['supported_commands'] if not
c['enabled']]
+disabled = [c['name'] for c in info['supported_commands']
+if not c['enabled']]
msgs.append('\tdisabled: ' + ', '.join(disabled))
return '\n'.join(msgs)
@@ -119,11 +123,11 @@ class QemuGuestAgentClient:
if ipaddr['ip-address-type'] == 'ipv4':
addr = ipaddr['ip-address']
mask = self.__gen_ipv4_netmask(int(ipaddr['prefix']))
-msgs.append("\tinet %s netmask %s" % (addr, mask))
+msgs.append(f"\tinet {addr} netmask {mask}")
elif ipaddr['ip-address-type'] == 'ipv6':
addr = ipaddr['ip-address']
prefix = ipaddr['prefix']
-msgs.append("\tinet6 %s prefixlen %s" % (addr,
prefix))
+msgs.append(f"\tinet6 {addr} prefixlen {prefix}")
if nif['hardware-address'] != '00:00:00:00:00:00':
msgs.append("\tether " + nif['hardware-address'])
@@ -237,6 +241,8 @@ def _cmd_suspend(client, args):
def _cmd_shutdown(client, args):
client.shutdown()
+
+
_cmd_powerdown = _cmd_shutdown
@@ -280,17 +286,15 @@ def main(address, cmd, args):
if __name__ == '__main__':
-import optparse
-import os
-import sys
+address = os.environ.get('QGA_CLIENT_ADDRESS')
-address = os.environ['QGA_CLIENT_ADDRESS'] if 'QGA_CLIENT_ADDRESS' in
os.environ else None
-
-usage = "%prog [--address=|]
[args...]\n"
+usage = ("%prog [--address=|]"
+ " [args...]\n")
usage += ': ' + ', '.join(commands)
parser = optparse.OptionParser(usage=usage)
parser.add_option('--address', action='store', type='string',
- default=address, help='Specify a ip:port pair or a unix
socket path')
+ default=address,
+ help='Specify a ip:port pair or a unix socket path')
options, args = parser.parse_args()
address = options.address
--
2.31.1
[PULL 35/72] scripts/qmp-shell: fix connect method signature
It needs to match the parent's signature -- the negotiate parameter must be optional. Signed-off-by: John Snow Message-id: 20210607200649.1840382-6-js...@redhat.com Signed-off-by: John Snow --- scripts/qmp/qmp-shell | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell index 413dd4d2de..04ca6a25ae 100755 --- a/scripts/qmp/qmp-shell +++ b/scripts/qmp/qmp-shell @@ -278,7 +278,7 @@ class QMPShell(qmp.QEMUMonitorProtocol): self._print(resp) return True -def connect(self, negotiate): +def connect(self, negotiate: bool = True): self._greeting = super(QMPShell, self).connect(negotiate) self.__completer_setup() -- 2.31.1
[PULL 24/72] scripts/qemu-ga-client: add module docstring
Turn that nice usage comment into a docstring. Signed-off-by: John Snow Message-id: 20210604155532.1499282-6-js...@redhat.com Signed-off-by: John Snow --- scripts/qmp/qemu-ga-client | 65 +++--- 1 file changed, 33 insertions(+), 32 deletions(-) diff --git a/scripts/qmp/qemu-ga-client b/scripts/qmp/qemu-ga-client index 8eb4015e61..e81937e0ea 100755 --- a/scripts/qmp/qemu-ga-client +++ b/scripts/qmp/qemu-ga-client @@ -1,41 +1,42 @@ #!/usr/bin/env python3 -# QEMU Guest Agent Client -# +""" +QEMU Guest Agent Client + +Usage: + +Start QEMU with: + +# qemu [...] -chardev socket,path=/tmp/qga.sock,server,wait=off,id=qga0 \ + -device virtio-serial \ + -device virtserialport,chardev=qga0,name=org.qemu.guest_agent.0 + +Run the script: + +$ qemu-ga-client --address=/tmp/qga.sock [args...] + +or + +$ export QGA_CLIENT_ADDRESS=/tmp/qga.sock +$ qemu-ga-client [args...] + +For example: + +$ qemu-ga-client cat /etc/resolv.conf +# Generated by NetworkManager +nameserver 10.0.2.3 +$ qemu-ga-client fsfreeze status +thawed +$ qemu-ga-client fsfreeze freeze +2 filesystems frozen + +See also: https://wiki.qemu.org/Features/QAPI/GuestAgent +""" + # Copyright (C) 2012 Ryota Ozaki # # This work is licensed under the terms of the GNU GPL, version 2. See # the COPYING file in the top-level directory. -# -# Usage: -# -# Start QEMU with: -# -# # qemu [...] -chardev socket,path=/tmp/qga.sock,server=on,wait=off,id=qga0 \ -# -device virtio-serial \ -# -device virtserialport,chardev=qga0,name=org.qemu.guest_agent.0 -# -# Run the script: -# -# $ qemu-ga-client --address=/tmp/qga.sock [args...] -# -# or -# -# $ export QGA_CLIENT_ADDRESS=/tmp/qga.sock -# $ qemu-ga-client [args...] -# -# For example: -# -# $ qemu-ga-client cat /etc/resolv.conf -# # Generated by NetworkManager -# nameserver 10.0.2.3 -# $ qemu-ga-client fsfreeze status -# thawed -# $ qemu-ga-client fsfreeze freeze -# 2 filesystems frozen -# -# See also: https://wiki.qemu.org/Features/QAPI/GuestAgent -# import argparse import base64 -- 2.31.1
[PULL 15/72] scripts/qom-fuse: add static type hints
Because fusepy does not have type hints, add some targeted warning
suppressions.
Namely, we need to allow subclassing something of an unknown type (in
qom_fuse.py), and we need to allow missing imports (recorded against
fuse itself) because mypy will be unable to import fusepy (even when
installed) as it has no types nor type stubs available.
Note: Until now, it was possible to run invocations like 'mypy qemu/'
from ./python and have that work. However, these targeted suppressions
require that you run 'mypy -p qemu/' instead. The correct, canonical
invocation is recorded in ./python/tests/mypy.sh and all of the various
CI invocations always use this correct form.
Signed-off-by: John Snow
Message-id: 20210603003719.1321369-16-js...@redhat.com
Signed-off-by: John Snow
---
python/setup.cfg | 8
scripts/qmp/qom-fuse | 26 +-
2 files changed, 25 insertions(+), 9 deletions(-)
diff --git a/python/setup.cfg b/python/setup.cfg
index c9b9445af9..ba8d29fd62 100644
--- a/python/setup.cfg
+++ b/python/setup.cfg
@@ -57,6 +57,14 @@ python_version = 3.6
warn_unused_configs = True
namespace_packages = True
+[mypy-qemu.qmp.qom_fuse]
+# fusepy has no type stubs:
+allow_subclassing_any = True
+
+[mypy-fuse]
+# fusepy has no type stubs:
+ignore_missing_imports = True
+
[pylint.messages control]
# Disable the message, report, category or checker with the given id(s). You
# can either give multiple identifiers separated by comma (,) or put this
diff --git a/scripts/qmp/qom-fuse b/scripts/qmp/qom-fuse
index 0d11f73152..a5a7a304a3 100755
--- a/scripts/qmp/qom-fuse
+++ b/scripts/qmp/qom-fuse
@@ -38,7 +38,14 @@ from errno import ENOENT, EPERM
import os
import stat
import sys
-from typing import Dict
+from typing import (
+IO,
+Dict,
+Iterator,
+Mapping,
+Optional,
+Union,
+)
import fuse
from fuse import FUSE, FuseOSError, Operations
@@ -83,7 +90,7 @@ class QOMFuse(QOMCommand, Operations):
self.fuse = FUSE(self, self.mount, foreground=True)
return 0
-def get_ino(self, path):
+def get_ino(self, path: str) -> int:
"""Get an inode number for a given QOM path."""
if path in self.ino_map:
return self.ino_map[path]
@@ -91,7 +98,7 @@ class QOMFuse(QOMCommand, Operations):
self.ino_count += 1
return self.ino_map[path]
-def is_object(self, path):
+def is_object(self, path: str) -> bool:
"""Is the given QOM path an object?"""
try:
self.qom_list(path)
@@ -99,7 +106,7 @@ class QOMFuse(QOMCommand, Operations):
except QMPResponseError:
return False
-def is_property(self, path):
+def is_property(self, path: str) -> bool:
"""Is the given QOM path a property?"""
path, prop = path.rsplit('/', 1)
if path == '':
@@ -112,7 +119,7 @@ class QOMFuse(QOMCommand, Operations):
except QMPResponseError:
return False
-def is_link(self, path):
+def is_link(self, path: str) -> bool:
"""Is the given QOM path a link?"""
path, prop = path.rsplit('/', 1)
if path == '':
@@ -125,7 +132,7 @@ class QOMFuse(QOMCommand, Operations):
except QMPResponseError:
return False
-def read(self, path, size, offset, fh):
+def read(self, path: str, size: int, offset: int, fh: IO[bytes]) -> bytes:
if not self.is_property(path):
raise FuseOSError(ENOENT)
@@ -143,7 +150,7 @@ class QOMFuse(QOMCommand, Operations):
return bytes(data[offset:][:size], encoding='utf-8')
-def readlink(self, path):
+def readlink(self, path: str) -> Union[bool, str]:
if not self.is_link(path):
return False
path, prop = path.rsplit('/', 1)
@@ -151,7 +158,8 @@ class QOMFuse(QOMCommand, Operations):
return prefix + str(self.qmp.command('qom-get', path=path,
property=prop))
-def getattr(self, path, fh=None):
+def getattr(self, path: str,
+fh: Optional[IO[bytes]] = None) -> Mapping[str, object]:
if self.is_link(path):
value = {
'st_mode': 0o755 | stat.S_IFLNK,
@@ -195,7 +203,7 @@ class QOMFuse(QOMCommand, Operations):
raise FuseOSError(ENOENT)
return value
-def readdir(self, path, fh):
+def readdir(self, path: str, fh: IO[bytes]) -> Iterator[str]:
yield '.'
yield '..'
for item in self.qom_list(path):
--
2.31.1
[PULL 31/72] scripts/qmp-shell: apply isort rules
Signed-off-by: John Snow Message-id: 20210607200649.1840382-2-js...@redhat.com Signed-off-by: John Snow --- scripts/qmp/qmp-shell | 12 +++- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell index d5ae8a9b21..a00efe6fea 100755 --- a/scripts/qmp/qmp-shell +++ b/scripts/qmp/qmp-shell @@ -65,18 +65,20 @@ # which will echo back the properly formatted JSON-compliant QMP that is being # sent to QEMU, which is useful for debugging and documentation generation. -import json import ast -import readline -import sys -import os -import errno import atexit +import errno +import json +import os import re +import readline +import sys + sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python')) from qemu import qmp + class QMPCompleter(list): def complete(self, text, state): for cmd in self: -- 2.31.1
[PULL 20/72] scripts/qemu-ga-client: apply isort rules
Hint: > ln -s scripts/qmp/qemu-ga-client python/qemu/qmp/qemu_ga_client.py > cd python > isort qemu Signed-off-by: John Snow Message-id: 20210604155532.1499282-2-js...@redhat.com Signed-off-by: John Snow --- scripts/qmp/qemu-ga-client | 9 + 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/scripts/qmp/qemu-ga-client b/scripts/qmp/qemu-ga-client index 348d85864c..97f4047a62 100755 --- a/scripts/qmp/qemu-ga-client +++ b/scripts/qmp/qemu-ga-client @@ -36,10 +36,11 @@ # See also: https://wiki.qemu.org/Features/QAPI/GuestAgent # -import os -import sys import base64 +import os import random +import sys + sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python')) from qemu import qmp @@ -279,9 +280,9 @@ def main(address, cmd, args): if __name__ == '__main__': -import sys -import os import optparse +import os +import sys address = os.environ['QGA_CLIENT_ADDRESS'] if 'QGA_CLIENT_ADDRESS' in os.environ else None -- 2.31.1
[PULL 13/72] scripts/qom-fuse: use QOMCommand.qom_list()
the qom_list method provides a type-safe object that's easier to type
check, so switch to using it.
Signed-off-by: John Snow
Reviewed-by: Philippe Mathieu-Daudé
Message-id: 20210603003719.1321369-14-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qom-fuse | 18 --
1 file changed, 8 insertions(+), 10 deletions(-)
diff --git a/scripts/qmp/qom-fuse b/scripts/qmp/qom-fuse
index 1676fb78d9..703a97e75f 100755
--- a/scripts/qmp/qom-fuse
+++ b/scripts/qmp/qom-fuse
@@ -94,7 +94,7 @@ class QOMFuse(QOMCommand, Operations):
def is_object(self, path):
"""Is the given QOM path an object?"""
try:
-self.qmp.command('qom-list', path=path)
+self.qom_list(path)
return True
except QMPResponseError:
return False
@@ -105,8 +105,8 @@ class QOMFuse(QOMCommand, Operations):
if path == '':
path = '/'
try:
-for item in self.qmp.command('qom-list', path=path):
-if item['name'] == prop:
+for item in self.qom_list(path):
+if item.name == prop:
return True
return False
except QMPResponseError:
@@ -118,11 +118,9 @@ class QOMFuse(QOMCommand, Operations):
if path == '':
path = '/'
try:
-for item in self.qmp.command('qom-list', path=path):
-if item['name'] == prop:
-if item['type'].startswith('link<'):
-return True
-return False
+for item in self.qom_list(path):
+if item.name == prop and item.link:
+return True
return False
except QMPResponseError:
return False
@@ -200,8 +198,8 @@ class QOMFuse(QOMCommand, Operations):
def readdir(self, path, fh):
yield '.'
yield '..'
-for item in self.qmp.command('qom-list', path=path):
-yield str(item['name'])
+for item in self.qom_list(path):
+yield item.name
if __name__ == '__main__':
--
2.31.1
[PULL 27/72] scripts/qemu-ga-client: add mypy type hints
This script is in slightly rough shape, but it still works. A lot of
care went into its initial development. In good faith, I'm updating it
to the latest Python coding standards. If there is in interest in this
script, though, I'll be asking for a contributor to take care of it
further.
Signed-off-by: John Snow
Message-id: 20210604155532.1499282-9-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qemu-ga-client | 89 +-
1 file changed, 49 insertions(+), 40 deletions(-)
diff --git a/scripts/qmp/qemu-ga-client b/scripts/qmp/qemu-ga-client
index ece9f74fa8..a7d0ef8347 100755
--- a/scripts/qmp/qemu-ga-client
+++ b/scripts/qmp/qemu-ga-client
@@ -44,10 +44,18 @@ import errno
import os
import random
import sys
+from typing import (
+Any,
+Callable,
+Dict,
+Optional,
+Sequence,
+)
sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
from qemu import qmp
+from qemu.qmp import SocketAddrT
# This script has not seen many patches or careful attention in quite
@@ -58,18 +66,18 @@ from qemu import qmp
class QemuGuestAgent(qmp.QEMUMonitorProtocol):
-def __getattr__(self, name):
-def wrapper(**kwds):
+def __getattr__(self, name: str) -> Callable[..., Any]:
+def wrapper(**kwds: object) -> object:
return self.command('guest-' + name.replace('_', '-'), **kwds)
return wrapper
class QemuGuestAgentClient:
-def __init__(self, address):
+def __init__(self, address: SocketAddrT):
self.qga = QemuGuestAgent(address)
self.qga.connect(negotiate=False)
-def sync(self, timeout=3):
+def sync(self, timeout: Optional[float] = 3) -> None:
# Avoid being blocked forever
if not self.ping(timeout):
raise EnvironmentError('Agent seems not alive')
@@ -79,9 +87,9 @@ class QemuGuestAgentClient:
if isinstance(ret, int) and int(ret) == uid:
break
-def __file_read_all(self, handle):
+def __file_read_all(self, handle: int) -> bytes:
eof = False
-data = ''
+data = b''
while not eof:
ret = self.qga.file_read(handle=handle, count=1024)
_data = base64.b64decode(ret['buf-b64'])
@@ -89,7 +97,7 @@ class QemuGuestAgentClient:
eof = ret['eof']
return data
-def read(self, path):
+def read(self, path: str) -> bytes:
handle = self.qga.file_open(path=path)
try:
data = self.__file_read_all(handle)
@@ -97,7 +105,7 @@ class QemuGuestAgentClient:
self.qga.file_close(handle=handle)
return data
-def info(self):
+def info(self) -> str:
info = self.qga.info()
msgs = []
@@ -113,14 +121,14 @@ class QemuGuestAgentClient:
return '\n'.join(msgs)
@classmethod
-def __gen_ipv4_netmask(cls, prefixlen):
+def __gen_ipv4_netmask(cls, prefixlen: int) -> str:
mask = int('1' * prefixlen + '0' * (32 - prefixlen), 2)
return '.'.join([str(mask >> 24),
str((mask >> 16) & 0xff),
str((mask >> 8) & 0xff),
str(mask & 0xff)])
-def ifconfig(self):
+def ifconfig(self) -> str:
nifs = self.qga.network_get_interfaces()
msgs = []
@@ -141,7 +149,7 @@ class QemuGuestAgentClient:
return '\n'.join(msgs)
-def ping(self, timeout):
+def ping(self, timeout: Optional[float]) -> bool:
self.qga.settimeout(timeout)
try:
self.qga.ping()
@@ -149,37 +157,40 @@ class QemuGuestAgentClient:
return False
return True
-def fsfreeze(self, cmd):
+def fsfreeze(self, cmd: str) -> object:
if cmd not in ['status', 'freeze', 'thaw']:
raise Exception('Invalid command: ' + cmd)
-
+# Can be int (freeze, thaw) or GuestFsfreezeStatus (status)
return getattr(self.qga, 'fsfreeze' + '_' + cmd)()
-def fstrim(self, minimum=0):
-return getattr(self.qga, 'fstrim')(minimum=minimum)
+def fstrim(self, minimum: int) -> Dict[str, object]:
+# returns GuestFilesystemTrimResponse
+ret = getattr(self.qga, 'fstrim')(minimum=minimum)
+assert isinstance(ret, dict)
+return ret
-def suspend(self, mode):
+def suspend(self, mode: str) -> None:
if mode not in ['disk', 'ram', 'hybrid']:
raise Exception('Invalid mode: ' + mode)
try:
getattr(self.qga, 'suspend' + '_' + mode)()
# On error exception will raise
-except self.qga.timeout:
+except TimeoutError:
# On success command will timed out
return
-def shutdown(self, mode='powerdown'):
+def shutdown(self, mode: str = 'powerdown') -> None:
if mode not in ['powerdown', 'halt', 'reboot']:
raise Exception('Invalid
[PULL 10/72] scripts/qom-fuse: Apply pylint rules
- Catch specific exceptions from QMP
- Reraise errors with explicit context
- method parameters should match parent's names
Signed-off-by: John Snow
Reviewed-by: Philippe Mathieu-Daudé
Message-id: 20210603003719.1321369-11-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qom-fuse | 16
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/scripts/qmp/qom-fuse b/scripts/qmp/qom-fuse
index ca30e92867..805e99c8ec 100755
--- a/scripts/qmp/qom-fuse
+++ b/scripts/qmp/qom-fuse
@@ -23,7 +23,7 @@ from fuse import FUSE, FuseOSError, Operations
sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
-from qemu.qmp import QEMUMonitorProtocol
+from qemu.qmp import QEMUMonitorProtocol, QMPResponseError
fuse.fuse_python_api = (0, 2)
@@ -47,7 +47,7 @@ class QOMFS(Operations):
try:
self.qmp.command('qom-list', path=path)
return True
-except:
+except QMPResponseError:
return False
def is_property(self, path):
@@ -59,7 +59,7 @@ class QOMFS(Operations):
if item['name'] == prop:
return True
return False
-except:
+except QMPResponseError:
return False
def is_link(self, path):
@@ -73,10 +73,10 @@ class QOMFS(Operations):
return True
return False
return False
-except:
+except QMPResponseError:
return False
-def read(self, path, length, offset, fh):
+def read(self, path, size, offset, fh):
if not self.is_property(path):
return -ENOENT
@@ -86,13 +86,13 @@ class QOMFS(Operations):
try:
data = self.qmp.command('qom-get', path=path, property=prop)
data += '\n' # make values shell friendly
-except:
-raise FuseOSError(EPERM)
+except QMPResponseError as err:
+raise FuseOSError(EPERM) from err
if offset > len(data):
return ''
-return bytes(data[offset:][:length], encoding='utf-8')
+return bytes(data[offset:][:size], encoding='utf-8')
def readlink(self, path):
if not self.is_link(path):
--
2.31.1
[PULL 08/72] scripts/qom-fuse: apply flake8 rules
flake8 still has one warning because of the sys.path hack, but that will
be going away by the end of this patch series.
Signed-off-by: John Snow
Reviewed-by: Philippe Mathieu-Daudé
Message-id: 20210603003719.1321369-9-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qom-fuse | 81 +++-
1 file changed, 43 insertions(+), 38 deletions(-)
diff --git a/scripts/qmp/qom-fuse b/scripts/qmp/qom-fuse
index 62deb9adb1..ca30e92867 100755
--- a/scripts/qmp/qom-fuse
+++ b/scripts/qmp/qom-fuse
@@ -9,13 +9,12 @@
# Anthony Liguori
# Markus Armbruster
#
-# This work is licensed under the terms of the GNU GPL, version 2 or later.
See
-# the COPYING file in the top-level directory.
+# This work is licensed under the terms of the GNU GPL, version 2 or later.
+# See the COPYING file in the top-level directory.
##
-from errno import *
+from errno import ENOENT, EPERM
import os
-import posix
import stat
import sys
@@ -29,6 +28,7 @@ from qemu.qmp import QEMUMonitorProtocol
fuse.fuse_python_api = (0, 2)
+
class QOMFS(Operations):
def __init__(self, qmp):
self.qmp = qmp
@@ -45,7 +45,7 @@ class QOMFS(Operations):
def is_object(self, path):
try:
-items = self.qmp.command('qom-list', path=path)
+self.qmp.command('qom-list', path=path)
return True
except:
return False
@@ -85,7 +85,7 @@ class QOMFS(Operations):
path = '/'
try:
data = self.qmp.command('qom-get', path=path, property=prop)
-data += '\n' # make values shell friendly
+data += '\n' # make values shell friendly
except:
raise FuseOSError(EPERM)
@@ -104,38 +104,44 @@ class QOMFS(Operations):
def getattr(self, path, fh=None):
if self.is_link(path):
-value = { 'st_mode': 0o755 | stat.S_IFLNK,
- 'st_ino': self.get_ino(path),
- 'st_dev': 0,
- 'st_nlink': 2,
- 'st_uid': 1000,
- 'st_gid': 1000,
- 'st_size': 4096,
- 'st_atime': 0,
- 'st_mtime': 0,
- 'st_ctime': 0 }
+value = {
+'st_mode': 0o755 | stat.S_IFLNK,
+'st_ino': self.get_ino(path),
+'st_dev': 0,
+'st_nlink': 2,
+'st_uid': 1000,
+'st_gid': 1000,
+'st_size': 4096,
+'st_atime': 0,
+'st_mtime': 0,
+'st_ctime': 0
+}
elif self.is_object(path):
-value = { 'st_mode': 0o755 | stat.S_IFDIR,
- 'st_ino': self.get_ino(path),
- 'st_dev': 0,
- 'st_nlink': 2,
- 'st_uid': 1000,
- 'st_gid': 1000,
- 'st_size': 4096,
- 'st_atime': 0,
- 'st_mtime': 0,
- 'st_ctime': 0 }
+value = {
+'st_mode': 0o755 | stat.S_IFDIR,
+'st_ino': self.get_ino(path),
+'st_dev': 0,
+'st_nlink': 2,
+'st_uid': 1000,
+'st_gid': 1000,
+'st_size': 4096,
+'st_atime': 0,
+'st_mtime': 0,
+'st_ctime': 0
+}
elif self.is_property(path):
-value = { 'st_mode': 0o644 | stat.S_IFREG,
- 'st_ino': self.get_ino(path),
- 'st_dev': 0,
- 'st_nlink': 1,
- 'st_uid': 1000,
- 'st_gid': 1000,
- 'st_size': 4096,
- 'st_atime': 0,
- 'st_mtime': 0,
- 'st_ctime': 0 }
+value = {
+'st_mode': 0o644 | stat.S_IFREG,
+'st_ino': self.get_ino(path),
+'st_dev': 0,
+'st_nlink': 1,
+'st_uid': 1000,
+'st_gid': 1000,
+'st_size': 4096,
+'st_atime': 0,
+'st_mtime': 0,
+'st_ctime': 0
+}
else:
raise FuseOSError(ENOENT)
return value
@@ -146,8 +152,7 @@ class QOMFS(Operations):
for item in self.qmp.command('qom-list', path=path):
yield str(item['name'])
+
if __name__ == '__main__':
-import os
-
fuse = FUSE(QOMFS(QEMUMonitorProtocol(os.environ['QMP_SOCKET'])),
sys.argv[1], foreground=True)
--
2.31.1
[PULL 18/72] scripts/qom-fuse: add redirection shim to python/qemu/qmp/qom-fuse.py
By leaving the script absent for a commit, git-blame travels to the new file instead of staying on the shim. Signed-off-by: John Snow Reviewed-by: Philippe Mathieu-Daudé Message-id: 20210603003719.1321369-19-js...@redhat.com Signed-off-by: John Snow --- scripts/qmp/qom-fuse | 11 +++ 1 file changed, 11 insertions(+) create mode 100755 scripts/qmp/qom-fuse diff --git a/scripts/qmp/qom-fuse b/scripts/qmp/qom-fuse new file mode 100755 index 00..a58c8ef979 --- /dev/null +++ b/scripts/qmp/qom-fuse @@ -0,0 +1,11 @@ +#!/usr/bin/env python3 + +import os +import sys + +sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python')) +from qemu.qmp.qom_fuse import QOMFuse + + +if __name__ == '__main__': +sys.exit(QOMFuse.entry_point()) -- 2.31.1
[PULL 06/72] scripts/qmp: redirect qom-xxx scripts to python/qemu/qmp/
Redirect to the new qom scripts. These forwarders can be deleted
eventually when there has been more time for the dust on the Python
packaging to settle and people understand how to find these commands.
Note: You can run these by setting $PYTHONPATH in your shell and then
running "python3 -m qemu.qmp.qom", or you can install the qemu namespace
package and use the "qom" or "qom-set" scripts.
I've written how to install the package elsewhere, but for the sake of
git-blame, cd to ./python, and then do:
- pip3 install [--user] [-e] .
--user will install to your local user install (will not work inside of
a venv), omitting this flag installs to your system-wide packages
(outside of a venv) or to your current virtual environment (inside the
venv).
When installing to a venv or to your system-wide packages, "qom"
should be in your $PATH already. If you do a user install, you may
need to add ~/.local/bin to your $PATH if you haven't already.
-e installs in editable mode: the installed package is effectively just
a symlink to this folder; so changes to your git working tree are
reflected in the installed package.
Note: installing these packages to an environment outside a venv can be
dangerous: Many QEMU scripts will begin to prefer the installed version
instead of the version directly in the tree. Use with caution. editable
mode is recommended when working outside of a venv.
Signed-off-by: John Snow
Reviewed-by: Philippe Mathieu-Daudé
Message-id: 20210603003719.1321369-7-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qom-get | 66 +++
scripts/qmp/qom-list | 63 +++--
scripts/qmp/qom-set | 63 +++--
scripts/qmp/qom-tree | 74 +++-
4 files changed, 16 insertions(+), 250 deletions(-)
diff --git a/scripts/qmp/qom-get b/scripts/qmp/qom-get
index 666df71832..e4f3e0c013 100755
--- a/scripts/qmp/qom-get
+++ b/scripts/qmp/qom-get
@@ -1,69 +1,11 @@
#!/usr/bin/env python3
-##
-# QEMU Object Model test tools
-#
-# Copyright IBM, Corp. 2011
-#
-# Authors:
-# Anthony Liguori
-#
-# This work is licensed under the terms of the GNU GPL, version 2 or later.
See
-# the COPYING file in the top-level directory.
-##
-import sys
import os
+import sys
sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
-from qemu.qmp import QEMUMonitorProtocol
+from qemu.qmp.qom import QOMGet
-cmd, args = sys.argv[0], sys.argv[1:]
-socket_path = None
-path = None
-prop = None
-def usage():
-return '''environment variables:
-QMP_SOCKET=
-usage:
-%s [-h] [-s ] .
-''' % cmd
-
-def usage_error(error_msg = "unspecified error"):
-sys.stderr.write('%s\nERROR: %s\n' % (usage(), error_msg))
-exit(1)
-
-if len(args) > 0:
-if args[0] == "-h":
-print(usage())
-exit(0);
-elif args[0] == "-s":
-try:
-socket_path = args[1]
-except:
-usage_error("missing argument: QMP socket path or address");
-args = args[2:]
-
-if not socket_path:
-if 'QMP_SOCKET' in os.environ:
-socket_path = os.environ['QMP_SOCKET']
-else:
-usage_error("no QMP socket path or address given");
-
-if len(args) > 0:
-try:
-path, prop = args[0].rsplit('.', 1)
-except:
-usage_error("invalid format for path/property/value")
-else:
-usage_error("not enough arguments")
-
-srv = QEMUMonitorProtocol(socket_path)
-srv.connect()
-
-rsp = srv.command('qom-get', path=path, property=prop)
-if type(rsp) == dict:
-for i in rsp.keys():
-print('%s: %s' % (i, rsp[i]))
-else:
-print(rsp)
+if __name__ == '__main__':
+sys.exit(QOMGet.entry_point())
diff --git a/scripts/qmp/qom-list b/scripts/qmp/qom-list
index 5074fd939f..7a071a54e1 100755
--- a/scripts/qmp/qom-list
+++ b/scripts/qmp/qom-list
@@ -1,66 +1,11 @@
#!/usr/bin/env python3
-##
-# QEMU Object Model test tools
-#
-# Copyright IBM, Corp. 2011
-#
-# Authors:
-# Anthony Liguori
-#
-# This work is licensed under the terms of the GNU GPL, version 2 or later.
See
-# the COPYING file in the top-level directory.
-##
-import sys
import os
+import sys
sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
-from qemu.qmp import QEMUMonitorProtocol
+from qemu.qmp.qom import QOMList
-cmd, args = sys.argv[0], sys.argv[1:]
-socket_path = None
-path = None
-prop = None
-def usage():
-return '''environment variables:
-QMP_SOCKET=
-usage:
-%s [-h] [-s ] []
-''' % cmd
-
-def usage_error(error_msg = "unspecified error"):
-sys.stderr.write('%s\nERROR: %s\n' % (usage(), error_msg))
-exit(1)
-
-if len(args) > 0:
-if args[0] == "-h":
-print(usage())
-exit(0);
-elif args[0] == "-s":
-try:
-socket_path = args[1]
-except:
-usage_error("missing argument: QMP socket path or address");
-
[PULL 09/72] python: Add 'fh' to known-good variable names
fd and fh are fine: we often use these for "file descriptor" or "file handle" accordingly. It is rarely the case that you need to enforce a more semantically meaningful name beyond "This is the file we are using right now." While we're here: add comments for all of the non-standard pylint names. (And the underscore.) Signed-off-by: John Snow Message-id: 20210603003719.1321369-10-js...@redhat.com Signed-off-by: John Snow --- python/setup.cfg | 7 --- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/python/setup.cfg b/python/setup.cfg index a19029d538..c9b9445af9 100644 --- a/python/setup.cfg +++ b/python/setup.cfg @@ -78,9 +78,10 @@ good-names=i, k, ex, Run, - _, - fd, - c, + _, # By convention: Unused variable + fh, # fh = open(...) + fd, # fd = os.open(...) + c, # for c in string: ... [pylint.similarities] # Ignore imports when computing similarities. -- 2.31.1
[PULL 19/72] python/qmp: add fuse command to 'qom' tools
The 'fuse' command will be unavailable if 'fusepy' is not installed. It
will simply not load and subsequently be unavailable as a subcommand.
Signed-off-by: John Snow
Reviewed-by: Philippe Mathieu-Daudé
Message-id: 20210603003719.1321369-20-js...@redhat.com
Signed-off-by: John Snow
---
python/qemu/qmp/qom.py | 14 --
python/setup.cfg | 1 +
2 files changed, 13 insertions(+), 2 deletions(-)
diff --git a/python/qemu/qmp/qom.py b/python/qemu/qmp/qom.py
index 7fe1448b5d..7ec7843d57 100644
--- a/python/qemu/qmp/qom.py
+++ b/python/qemu/qmp/qom.py
@@ -1,7 +1,7 @@
"""
QEMU Object Model testing tools.
-usage: qom [-h] {set,get,list,tree} ...
+usage: qom [-h] {set,get,list,tree,fuse} ...
Query and manipulate QOM data
@@ -9,11 +9,12 @@
-h, --help show this help message and exit
QOM commands:
- {set,get,list,tree}
+ {set,get,list,tree,fuse}
setSet a QOM property value
getGet a QOM property value
list List QOM properties at a given path
tree Show QOM tree from a given path
+fuse Mount a QOM tree as a FUSE filesystem
"""
##
# Copyright John Snow 2020, for Red Hat, Inc.
@@ -35,6 +36,15 @@
from .qom_common import QOMCommand
+try:
+from .qom_fuse import QOMFuse
+except ModuleNotFoundError as err:
+if err.name != 'fuse':
+raise
+else:
+assert issubclass(QOMFuse, QOMCommand)
+
+
class QOMSet(QOMCommand):
"""
QOM Command - Set a property to a given value.
diff --git a/python/setup.cfg b/python/setup.cfg
index aca6f31185..6b6be8b03c 100644
--- a/python/setup.cfg
+++ b/python/setup.cfg
@@ -49,6 +49,7 @@ console_scripts =
qom-get = qemu.qmp.qom:QOMGet.entry_point
qom-list = qemu.qmp.qom:QOMList.entry_point
qom-tree = qemu.qmp.qom:QOMTree.entry_point
+qom-fuse = qemu.qmp.qom_fuse:QOMFuse.entry_point [fuse]
[flake8]
extend-ignore = E722 # Prefer pylint's bare-except checks to flake8's
--
2.31.1
[PULL 17/72] scripts/qom-fuse: move to python/qemu/qmp/qom_fuse.py
Move qom-fuse over to the python package now that it passes the linter. Update the import paradigms so that it continues to pass in the context of the Python package. Signed-off-by: John Snow Message-id: 20210603003719.1321369-18-js...@redhat.com Signed-off-by: John Snow --- scripts/qmp/qom-fuse => python/qemu/qmp/qom_fuse.py | 12 ++-- 1 file changed, 2 insertions(+), 10 deletions(-) rename scripts/qmp/qom-fuse => python/qemu/qmp/qom_fuse.py (95%) mode change 100755 => 100644 diff --git a/scripts/qmp/qom-fuse b/python/qemu/qmp/qom_fuse.py old mode 100755 new mode 100644 similarity index 95% rename from scripts/qmp/qom-fuse rename to python/qemu/qmp/qom_fuse.py index a5a7a304a3..43f4671fdb --- a/scripts/qmp/qom-fuse +++ b/python/qemu/qmp/qom_fuse.py @@ -1,4 +1,3 @@ -#!/usr/bin/env python3 """ QEMU Object Model FUSE filesystem tool @@ -35,7 +34,6 @@ import argparse from errno import ENOENT, EPERM -import os import stat import sys from typing import ( @@ -50,10 +48,8 @@ import fuse from fuse import FUSE, FuseOSError, Operations - -sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python')) -from qemu.qmp import QMPResponseError -from qemu.qmp.qom_common import QOMCommand +from . import QMPResponseError +from .qom_common import QOMCommand fuse.fuse_python_api = (0, 2) @@ -208,7 +204,3 @@ def readdir(self, path: str, fh: IO[bytes]) -> Iterator[str]: yield '..' for item in self.qom_list(path): yield item.name - - -if __name__ == '__main__': -sys.exit(QOMFuse.entry_point()) -- 2.31.1
[PULL 11/72] scripts/qom-fuse: Add docstrings
The methods inherited from fuse don't need docstrings; that's up to
fusepy to handle.
Signed-off-by: John Snow
Message-id: 20210603003719.1321369-12-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qom-fuse | 21 +++--
1 file changed, 19 insertions(+), 2 deletions(-)
diff --git a/scripts/qmp/qom-fuse b/scripts/qmp/qom-fuse
index 805e99c8ec..1fb3008a16 100755
--- a/scripts/qmp/qom-fuse
+++ b/scripts/qmp/qom-fuse
@@ -1,7 +1,19 @@
#!/usr/bin/env python3
+"""
+QEMU Object Model FUSE filesystem tool
+
+This script offers a simple FUSE filesystem within which the QOM tree
+may be browsed, queried and edited using traditional shell tooling.
+
+This script requires the 'fusepy' python package.
+
+ENV:
+QMP_SOCKET: Path to the QMP server socket
+
+Usage:
+qom-fuse /mount/to/here
+"""
##
-# QEMU Object Model test tools
-#
# Copyright IBM, Corp. 2012
# Copyright (C) 2020 Red Hat, Inc.
#
@@ -30,6 +42,7 @@ fuse.fuse_python_api = (0, 2)
class QOMFS(Operations):
+"""QOMFS implements fuse.Operations to provide a QOM filesystem."""
def __init__(self, qmp):
self.qmp = qmp
self.qmp.connect()
@@ -37,6 +50,7 @@ class QOMFS(Operations):
self.ino_count = 1
def get_ino(self, path):
+"""Get an inode number for a given QOM path."""
if path in self.ino_map:
return self.ino_map[path]
self.ino_map[path] = self.ino_count
@@ -44,6 +58,7 @@ class QOMFS(Operations):
return self.ino_map[path]
def is_object(self, path):
+"""Is the given QOM path an object?"""
try:
self.qmp.command('qom-list', path=path)
return True
@@ -51,6 +66,7 @@ class QOMFS(Operations):
return False
def is_property(self, path):
+"""Is the given QOM path a property?"""
path, prop = path.rsplit('/', 1)
if path == '':
path = '/'
@@ -63,6 +79,7 @@ class QOMFS(Operations):
return False
def is_link(self, path):
+"""Is the given QOM path a link?"""
path, prop = path.rsplit('/', 1)
if path == '':
path = '/'
--
2.31.1
[PULL 12/72] scripts/qom-fuse: Convert to QOMCommand
Move qom-fuse onto the QOMCommand base established in
python/qemu/qmp/qom_common.py. The interface doesn't change
incompatibly, "qom-fuse mountpoint" still works as an invocation, and
QMP_SOCKET is still used as the environment variable.
Signed-off-by: John Snow
Message-id: 20210603003719.1321369-13-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qom-fuse | 59 ++--
1 file changed, 46 insertions(+), 13 deletions(-)
diff --git a/scripts/qmp/qom-fuse b/scripts/qmp/qom-fuse
index 1fb3008a16..1676fb78d9 100755
--- a/scripts/qmp/qom-fuse
+++ b/scripts/qmp/qom-fuse
@@ -7,11 +7,19 @@ may be browsed, queried and edited using traditional shell
tooling.
This script requires the 'fusepy' python package.
-ENV:
-QMP_SOCKET: Path to the QMP server socket
-Usage:
-qom-fuse /mount/to/here
+usage: qom-fuse [-h] [--socket SOCKET]
+
+Mount a QOM tree as a FUSE filesystem
+
+positional arguments:
+ Mount point
+
+optional arguments:
+ -h, --helpshow this help message and exit
+ --socket SOCKET, -s SOCKET
+QMP socket path or address (addr:port). May also be
+set via QMP_SOCKET environment variable.
"""
##
# Copyright IBM, Corp. 2012
@@ -25,30 +33,56 @@ Usage:
# See the COPYING file in the top-level directory.
##
+import argparse
from errno import ENOENT, EPERM
import os
import stat
import sys
+from typing import Dict
import fuse
from fuse import FUSE, FuseOSError, Operations
sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
-from qemu.qmp import QEMUMonitorProtocol, QMPResponseError
+from qemu.qmp import QMPResponseError
+from qemu.qmp.qom_common import QOMCommand
fuse.fuse_python_api = (0, 2)
-class QOMFS(Operations):
-"""QOMFS implements fuse.Operations to provide a QOM filesystem."""
-def __init__(self, qmp):
-self.qmp = qmp
-self.qmp.connect()
-self.ino_map = {}
+class QOMFuse(QOMCommand, Operations):
+"""
+QOMFuse implements both fuse.Operations and QOMCommand.
+
+Operations implements the FS, and QOMCommand implements the CLI command.
+"""
+name = 'fuse'
+help = 'Mount a QOM tree as a FUSE filesystem'
+fuse: FUSE
+
+@classmethod
+def configure_parser(cls, parser: argparse.ArgumentParser) -> None:
+super().configure_parser(parser)
+parser.add_argument(
+'mount',
+metavar='',
+action='store',
+help="Mount point",
+)
+
+def __init__(self, args: argparse.Namespace):
+super().__init__(args)
+self.mount = args.mount
+self.ino_map: Dict[str, int] = {}
self.ino_count = 1
+def run(self) -> int:
+print(f"Mounting QOMFS to '{self.mount}'", file=sys.stderr)
+self.fuse = FUSE(self, self.mount, foreground=True)
+return 0
+
def get_ino(self, path):
"""Get an inode number for a given QOM path."""
if path in self.ino_map:
@@ -171,5 +205,4 @@ class QOMFS(Operations):
if __name__ == '__main__':
-fuse = FUSE(QOMFS(QEMUMonitorProtocol(os.environ['QMP_SOCKET'])),
-sys.argv[1], foreground=True)
+sys.exit(QOMFuse.entry_point())
--
2.31.1
[PULL 25/72] scripts/qemu-ga-client: apply (most) pylint rules
I'm only doing a very quick best-effort to preserve this script, to help
keep it from breaking further. I think there are pending ideas swirling
on the right way to implement better SDKs and better clients, and this
script might be a handy reference for those discussions. It presents
some interesting design problems, like static type safety when using a
dynamic RPC mechanism.
I believe it's worth preserving the effort and care that went into
making this script by updating it to work with our current
infrastructure. However, I am disabling the requirement for docstrings
in this file.
If you would like to help improve this script, please add docstrings
alongside any refactors or rejuvenations you might apply at that time.
Signed-off-by: John Snow
Message-id: 20210604155532.1499282-7-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qemu-ga-client | 18 +++---
1 file changed, 15 insertions(+), 3 deletions(-)
diff --git a/scripts/qmp/qemu-ga-client b/scripts/qmp/qemu-ga-client
index e81937e0ea..ece9f74fa8 100755
--- a/scripts/qmp/qemu-ga-client
+++ b/scripts/qmp/qemu-ga-client
@@ -40,6 +40,7 @@ See also: https://wiki.qemu.org/Features/QAPI/GuestAgent
import argparse
import base64
+import errno
import os
import random
import sys
@@ -49,6 +50,13 @@ sys.path.append(os.path.join(os.path.dirname(__file__),
'..', '..', 'python'))
from qemu import qmp
+# This script has not seen many patches or careful attention in quite
+# some time. If you would like to improve it, please review the design
+# carefully and add docstrings at that point in time. Until then:
+
+# pylint: disable=missing-docstring
+
+
class QemuGuestAgent(qmp.QEMUMonitorProtocol):
def __getattr__(self, name):
def wrapper(**kwds):
@@ -104,7 +112,8 @@ class QemuGuestAgentClient:
return '\n'.join(msgs)
-def __gen_ipv4_netmask(self, prefixlen):
+@classmethod
+def __gen_ipv4_netmask(cls, prefixlen):
mask = int('1' * prefixlen + '0' * (32 - prefixlen), 2)
return '.'.join([str(mask >> 24),
str((mask >> 16) & 0xff),
@@ -207,10 +216,12 @@ def _cmd_fstrim(client, args):
def _cmd_ifconfig(client, args):
+assert not args
print(client.ifconfig())
def _cmd_info(client, args):
+assert not args
print(client.info())
@@ -239,6 +250,7 @@ def _cmd_suspend(client, args):
def _cmd_shutdown(client, args):
+assert not args
client.shutdown()
@@ -246,10 +258,12 @@ _cmd_powerdown = _cmd_shutdown
def _cmd_halt(client, args):
+assert not args
client.shutdown('halt')
def _cmd_reboot(client, args):
+assert not args
client.shutdown('reboot')
@@ -269,8 +283,6 @@ def send_command(address, cmd, args):
try:
client = QemuGuestAgentClient(address)
except OSError as err:
-import errno
-
print(err)
if err.errno == errno.ECONNREFUSED:
print('Hint: qemu is not running?')
--
2.31.1
[PULL 01/72] python/pipenv: Update Pipfile.lock
In a previous commit, I added tox to the development requirements of the
Python library. I never bothered to add them to the Pipfile, because
they aren't needed there. Here, I sync it anyway in its own commit so
that when we add new packages later that the diffstats will not
confusingly appear to pull in lots of extra packages.
Ideally I could tell Pipenv simply not to install these, but it doesn't
seem to support that, exactly. The alternative is removing Tox from the
development requires, which I'd rather not do.
The other alternative is re-specifying all of the dependencies of
setup.cfg in the Pipfile, which I'd also rather not do.
Picking what feels least-worst here.
Signed-off-by: John Snow
Message-id: 20210603003719.1321369-2-js...@redhat.com
Signed-off-by: John Snow
---
python/Pipfile.lock | 91 +++--
1 file changed, 88 insertions(+), 3 deletions(-)
diff --git a/python/Pipfile.lock b/python/Pipfile.lock
index 6e344f5fad..f2a3f91d0f 100644
--- a/python/Pipfile.lock
+++ b/python/Pipfile.lock
@@ -22,6 +22,13 @@
}
},
"develop": {
+"appdirs": {
+"hashes": [
+
"sha256:7d5d0167b2b1ba821647616af46a749d1c653740dd0d2415100fe26e27afdf41",
+
"sha256:a841dacd6b99318a741b166adb07e19ee71a274450e68237b4650ca1055ab128"
+],
+"version": "==1.4.4"
+},
"astroid": {
"hashes": [
"sha256:4db03ab5fc3340cf619dbc25e42c2cc3755154ce6009469766d7143d1fc2ee4e",
@@ -38,6 +45,20 @@
"markers": "python_version >= '3.6'",
"version": "==88.1"
},
+"distlib": {
+"hashes": [
+
"sha256:106fef6dc37dd8c0e2c0a60d3fca3e77460a48907f335fa28420463a6f799736",
+
"sha256:23e223426b28491b1ced97dc3bbe183027419dfc7982b4fa2f05d5f3ff10711c"
+],
+"version": "==0.3.2"
+},
+"filelock": {
+"hashes": [
+
"sha256:18d82244ee114f543149c66a6e0c14e9c4f8a1044b5cdaadd0f82159d6a6ff59",
+
"sha256:929b7d63ec5b7d6b71b0fa5ac14e030b3f70b75747cef1b10da9b879fef15836"
+],
+"version": "==3.0.12"
+},
"flake8": {
"hashes": [
"sha256:07528381786f2a6237b061f6e96610a4167b226cb926e2aa2b6b1d78057c576b",
@@ -54,6 +75,14 @@
"markers": "python_version < '3.8'",
"version": "==4.0.1"
},
+"importlib-resources": {
+"hashes": [
+
"sha256:54161657e8ffc76596c4ede7080ca68cb02962a2e074a2586b695a93a925d36e",
+
"sha256:e962bff7440364183203d179d7ae9ad90cb1f2b74dcb84300e88ecc42dca3351"
+],
+"markers": "python_version < '3.7'",
+"version": "==5.1.4"
+},
"isort": {
"hashes": [
"sha256:0a943902919f65c5684ac4e0154b1ad4fac6dcaa5d9f3426b732f1c8b5419be6",
@@ -132,6 +161,30 @@
],
"version": "==0.4.3"
},
+"packaging": {
+"hashes": [
+
"sha256:5b327ac1320dc863dca72f4514ecc086f31186744b84a230374cc1fd776feae5",
+
"sha256:67714da7f7bc052e064859c05c595155bd1ee9f69f76557e21f051443c20947a"
+],
+"markers": "python_version >= '2.7' and python_version not in
'3.0, 3.1, 3.2, 3.3'",
+"version": "==20.9"
+},
+"pluggy": {
+"hashes": [
+
"sha256:15b2acde666561e1298d71b523007ed7364de07029219b604cf808bfa1c765b0",
+
"sha256:966c145cd83c96502c3c3868f50408687b38434af77734af1e9ca461a4081d2d"
+],
+"markers": "python_version >= '2.7' and python_version not in
'3.0, 3.1, 3.2, 3.3'",
+"version": "==0.13.1"
+},
+"py": {
+"hashes": [
+
"sha256:21b81bda15b66ef5e1a777a21c4dcd9c20ad3efd0b3f817e7a809035269e1bd3",
+
"sha256:3b80836aa6d1feeaa108e046da6423ab8f6ceda6468545ae8d02d9d58d18818a"
+],
+"markers": "python_version >= '2.7' and python_version not in
'3.0, 3.1, 3.2, 3.3'",
+"version": "==1.10.0"
+},
"pycodestyle": {
"hashes": [
"sha256:514f76d918fcc0b55c6680472f0a37970994e07bbb80725808c17089be302068",
@@ -156,18 +209,42 @@
"markers": "python_version ~= '3.6'",
"version": "==2.8.2"
},
+"pyparsing": {
+"hashes": [
+
"sha256:c203ec8783bf771a155b207279b9bccb8dea02d8f0c9e5f8ead507bc3246ecc1",
+
"sha256:ef9d7589ef3c200abe66653d3f1ab1033c3c419ae9b9bdb1240a85b024efc88b"
+],
+"markers": "python_version >= '2.6' and python_version not in
'3.0, 3.1, 3.2, 3.3'",
+"version": "==2.4.7"
+},
"qemu": {
"editable": true,
"path": "."
[PULL 23/72] scripts/qemu-ga-client: replace deprecated optparse with argparse
optparse isn't supported anymore, it's from the python2 days. Replace it
with the mostly similar argparse.
Signed-off-by: John Snow
Message-id: 20210604155532.1499282-5-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qemu-ga-client | 32 +++-
1 file changed, 15 insertions(+), 17 deletions(-)
diff --git a/scripts/qmp/qemu-ga-client b/scripts/qmp/qemu-ga-client
index 7aba09f0fe..8eb4015e61 100755
--- a/scripts/qmp/qemu-ga-client
+++ b/scripts/qmp/qemu-ga-client
@@ -37,8 +37,8 @@
# See also: https://wiki.qemu.org/Features/QAPI/GuestAgent
#
+import argparse
import base64
-import optparse
import os
import random
import sys
@@ -255,7 +255,7 @@ def _cmd_reboot(client, args):
commands = [m.replace('_cmd_', '') for m in dir() if '_cmd_' in m]
-def main(address, cmd, args):
+def send_command(address, cmd, args):
if not os.path.exists(address):
print('%s not found' % address)
sys.exit(1)
@@ -283,25 +283,23 @@ def main(address, cmd, args):
globals()['_cmd_' + cmd](client, args)
-if __name__ == '__main__':
+def main():
address = os.environ.get('QGA_CLIENT_ADDRESS')
-usage = ("%prog [--address=|]"
- " [args...]\n")
-usage += ': ' + ', '.join(commands)
-parser = optparse.OptionParser(usage=usage)
-parser.add_option('--address', action='store', type='string',
- default=address,
- help='Specify a ip:port pair or a unix socket path')
-options, args = parser.parse_args()
+parser = argparse.ArgumentParser()
+parser.add_argument('--address', action='store',
+default=address,
+help='Specify a ip:port pair or a unix socket path')
+parser.add_argument('command', choices=commands)
+parser.add_argument('args', nargs='*')
-address = options.address
-if address is None:
+args = parser.parse_args()
+if args.address is None:
parser.error('address is not specified')
sys.exit(1)
-if len(args) == 0:
-parser.error('Less argument')
-sys.exit(1)
+send_command(args.address, args.command, args.args)
-main(address, args[0], args[1:])
+
+if __name__ == '__main__':
+main()
--
2.31.1
[PULL 04/72] python/qmp: Add qom script rewrites
Inspired by qom-set, qom-get, qom-tree and qom-list; combine all four of
those scripts into a single script.
A later addition of qom-fuse as an 'extension' necessitates that some
common features are split out and shared between them.
Signed-off-by: John Snow
Message-id: 20210603003719.1321369-5-js...@redhat.com
Signed-off-by: John Snow
---
python/qemu/qmp/qom.py| 262 ++
python/qemu/qmp/qom_common.py | 178 +++
2 files changed, 440 insertions(+)
create mode 100644 python/qemu/qmp/qom.py
create mode 100644 python/qemu/qmp/qom_common.py
diff --git a/python/qemu/qmp/qom.py b/python/qemu/qmp/qom.py
new file mode 100644
index 00..7fe1448b5d
--- /dev/null
+++ b/python/qemu/qmp/qom.py
@@ -0,0 +1,262 @@
+"""
+QEMU Object Model testing tools.
+
+usage: qom [-h] {set,get,list,tree} ...
+
+Query and manipulate QOM data
+
+optional arguments:
+ -h, --help show this help message and exit
+
+QOM commands:
+ {set,get,list,tree}
+setSet a QOM property value
+getGet a QOM property value
+list List QOM properties at a given path
+tree Show QOM tree from a given path
+"""
+##
+# Copyright John Snow 2020, for Red Hat, Inc.
+# Copyright IBM, Corp. 2011
+#
+# Authors:
+# John Snow
+# Anthony Liguori
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or later.
+# See the COPYING file in the top-level directory.
+#
+# Based on ./scripts/qmp/qom-[set|get|tree|list]
+##
+
+import argparse
+
+from . import QMPResponseError
+from .qom_common import QOMCommand
+
+
+class QOMSet(QOMCommand):
+"""
+QOM Command - Set a property to a given value.
+
+usage: qom-set [-h] [--socket SOCKET] .
+
+Set a QOM property value
+
+positional arguments:
+ . QOM path and property, separated by a period '.'
+ new QOM property value
+
+optional arguments:
+ -h, --helpshow this help message and exit
+ --socket SOCKET, -s SOCKET
+QMP socket path or address (addr:port). May also be
+set via QMP_SOCKET environment variable.
+"""
+name = 'set'
+help = 'Set a QOM property value'
+
+@classmethod
+def configure_parser(cls, parser: argparse.ArgumentParser) -> None:
+super().configure_parser(parser)
+cls.add_path_prop_arg(parser)
+parser.add_argument(
+'value',
+metavar='',
+action='store',
+help='new QOM property value'
+)
+
+def __init__(self, args: argparse.Namespace):
+super().__init__(args)
+self.path, self.prop = args.path_prop.rsplit('.', 1)
+self.value = args.value
+
+def run(self) -> int:
+rsp = self.qmp.command(
+'qom-set',
+path=self.path,
+property=self.prop,
+value=self.value
+)
+print(rsp)
+return 0
+
+
+class QOMGet(QOMCommand):
+"""
+QOM Command - Get a property's current value.
+
+usage: qom-get [-h] [--socket SOCKET] .
+
+Get a QOM property value
+
+positional arguments:
+ . QOM path and property, separated by a period '.'
+
+optional arguments:
+ -h, --helpshow this help message and exit
+ --socket SOCKET, -s SOCKET
+QMP socket path or address (addr:port). May also be
+set via QMP_SOCKET environment variable.
+"""
+name = 'get'
+help = 'Get a QOM property value'
+
+@classmethod
+def configure_parser(cls, parser: argparse.ArgumentParser) -> None:
+super().configure_parser(parser)
+cls.add_path_prop_arg(parser)
+
+def __init__(self, args: argparse.Namespace):
+super().__init__(args)
+try:
+tmp = args.path_prop.rsplit('.', 1)
+except ValueError as err:
+raise ValueError('Invalid format for .') from err
+self.path = tmp[0]
+self.prop = tmp[1]
+
+def run(self) -> int:
+rsp = self.qmp.command(
+'qom-get',
+path=self.path,
+property=self.prop
+)
+if isinstance(rsp, dict):
+for key, value in rsp.items():
+print(f"{key}: {value}")
+else:
+print(rsp)
+return 0
+
+
+class QOMList(QOMCommand):
+"""
+QOM Command - List the properties at a given path.
+
+usage: qom-list [-h] [--socket SOCKET]
+
+List QOM properties at a given path
+
+positional arguments:
+ QOM path
+
+optional arguments:
+ -h, --helpshow this help message and exit
+ --socket SOCKET, -s SOCKET
+QMP socket path or address (addr:port). May also be
+set via QMP_SOCKET environment variable.
+"""
+name = 'list'
+
[PULL 03/72] python/qmp: add parse_address classmethod
This takes the place of qmp-shell's __get_address function. It also
allows other utilities to share the same parser and syntax for
specifying QMP locations.
Signed-off-by: John Snow
Message-id: 20210603003719.1321369-4-js...@redhat.com
Signed-off-by: John Snow
---
python/qemu/qmp/__init__.py | 26 ++
scripts/qmp/qmp-shell | 21 ++---
2 files changed, 28 insertions(+), 19 deletions(-)
diff --git a/python/qemu/qmp/__init__.py b/python/qemu/qmp/__init__.py
index 5fb970f8a8..822c793c32 100644
--- a/python/qemu/qmp/__init__.py
+++ b/python/qemu/qmp/__init__.py
@@ -92,6 +92,12 @@ def __init__(self, reply: QMPMessage):
self.reply = reply
+class QMPBadPortError(QMPError):
+"""
+Unable to parse socket address: Port was non-numerical.
+"""
+
+
class QEMUMonitorProtocol:
"""
Provide an API to connect to QEMU via QEMU Monitor Protocol (QMP) and then
@@ -219,6 +225,26 @@ def __exit__(self,
# Implement context manager exit function.
self.close()
+@classmethod
+def parse_address(cls, address: str) -> SocketAddrT:
+"""
+Parse a string into a QMP address.
+
+Figure out if the argument is in the port:host form.
+If it's not, it's probably a file path.
+"""
+components = address.split(':')
+if len(components) == 2:
+try:
+port = int(components[1])
+except ValueError:
+msg = f"Bad port: '{components[1]}' in '{address}'."
+raise QMPBadPortError(msg) from None
+return (components[0], port)
+
+# Treat as filepath.
+return address
+
def connect(self, negotiate: bool = True) -> Optional[QMPMessage]:
"""
Connect to the QMP Monitor and perform capabilities negotiation.
diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell
index b4d06096ab..d5ae8a9b21 100755
--- a/scripts/qmp/qmp-shell
+++ b/scripts/qmp/qmp-shell
@@ -89,8 +89,6 @@ class QMPCompleter(list):
class QMPShellError(Exception):
pass
-class QMPShellBadPort(QMPShellError):
-pass
class FuzzyJSON(ast.NodeTransformer):
'''This extension of ast.NodeTransformer filters literal "true/false/null"
@@ -109,7 +107,7 @@ class FuzzyJSON(ast.NodeTransformer):
# _execute_cmd()). Let's design a better one.
class QMPShell(qmp.QEMUMonitorProtocol):
def __init__(self, address, pretty=False):
-super(QMPShell, self).__init__(self.__get_address(address))
+super(QMPShell, self).__init__(self.parse_address(address))
self._greeting = None
self._completer = None
self._pretty = pretty
@@ -118,21 +116,6 @@ class QMPShell(qmp.QEMUMonitorProtocol):
self._histfile = os.path.join(os.path.expanduser('~'),
'.qmp-shell_history')
-def __get_address(self, arg):
-"""
-Figure out if the argument is in the port:host form, if it's not it's
-probably a file path.
-"""
-addr = arg.split(':')
-if len(addr) == 2:
-try:
-port = int(addr[1])
-except ValueError:
-raise QMPShellBadPort
-return ( addr[0], port )
-# socket path
-return arg
-
def _fill_completion(self):
cmds = self.cmd('query-commands')
if 'error' in cmds:
@@ -437,7 +420,7 @@ def main():
if qemu is None:
fail_cmdline()
-except QMPShellBadPort:
+except qmp.QMPBadPortError:
die('bad port number in command-line')
try:
--
2.31.1
[PULL 07/72] scripts/qom-fuse: apply isort rules
Hint: you can use symlinks to create qom_fuse.py in python/qemu/qmp/ and point to scripts/qom-fuse to apply the standard linting rules to this script. Signed-off-by: John Snow Reviewed-by: Philippe Mathieu-Daudé Message-id: 20210603003719.1321369-8-js...@redhat.com Signed-off-by: John Snow --- scripts/qmp/qom-fuse | 12 +--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/scripts/qmp/qom-fuse b/scripts/qmp/qom-fuse index 7c7cff8edf..62deb9adb1 100755 --- a/scripts/qmp/qom-fuse +++ b/scripts/qmp/qom-fuse @@ -13,14 +13,20 @@ # the COPYING file in the top-level directory. ## -import fuse, stat -from fuse import FUSE, FuseOSError, Operations -import os, posix, sys from errno import * +import os +import posix +import stat +import sys + +import fuse +from fuse import FUSE, FuseOSError, Operations + sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python')) from qemu.qmp import QEMUMonitorProtocol + fuse.fuse_python_api = (0, 2) class QOMFS(Operations): -- 2.31.1
[PULL 05/72] python/qmp: add qom script entry points
Add the 'qom', 'qom-set', 'qom-get', 'qom-list', and 'qom-tree' scripts to the qemu.qmp package. When you install this package, these scripts will become available on your command line. (e.g. when inside of a venv, `cd python && pip install .` will add 'qom', 'qom-set', etc to your $PATH.) Signed-off-by: John Snow Reviewed-by: Philippe Mathieu-Daudé Message-id: 20210603003719.1321369-6-js...@redhat.com Signed-off-by: John Snow --- python/setup.cfg | 8 1 file changed, 8 insertions(+) diff --git a/python/setup.cfg b/python/setup.cfg index 0fcdec6f32..a19029d538 100644 --- a/python/setup.cfg +++ b/python/setup.cfg @@ -37,6 +37,14 @@ devel = pylint >= 2.8.0 tox >= 3.18.0 +[options.entry_points] +console_scripts = +qom = qemu.qmp.qom:main +qom-set = qemu.qmp.qom:QOMSet.entry_point +qom-get = qemu.qmp.qom:QOMGet.entry_point +qom-list = qemu.qmp.qom:QOMList.entry_point +qom-tree = qemu.qmp.qom:QOMTree.entry_point + [flake8] extend-ignore = E722 # Prefer pylint's bare-except checks to flake8's exclude = __pycache__, -- 2.31.1
[PULL 14/72] scripts/qom-fuse: ensure QOMFuse.read always returns bytes
- Use FuseOSError to signal ENOENT instead of returning it
- Wrap qom-get in str(), as we don't always know its type
- The empty return should be b'', not ''.
Signed-off-by: John Snow
Reviewed-by: Philippe Mathieu-Daudé
Message-id: 20210603003719.1321369-15-js...@redhat.com
Signed-off-by: John Snow
---
scripts/qmp/qom-fuse | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/scripts/qmp/qom-fuse b/scripts/qmp/qom-fuse
index 703a97e75f..0d11f73152 100755
--- a/scripts/qmp/qom-fuse
+++ b/scripts/qmp/qom-fuse
@@ -127,19 +127,19 @@ class QOMFuse(QOMCommand, Operations):
def read(self, path, size, offset, fh):
if not self.is_property(path):
-return -ENOENT
+raise FuseOSError(ENOENT)
path, prop = path.rsplit('/', 1)
if path == '':
path = '/'
try:
-data = self.qmp.command('qom-get', path=path, property=prop)
+data = str(self.qmp.command('qom-get', path=path, property=prop))
data += '\n' # make values shell friendly
except QMPResponseError as err:
raise FuseOSError(EPERM) from err
if offset > len(data):
-return ''
+return b''
return bytes(data[offset:][:size], encoding='utf-8')
--
2.31.1
[PULL 02/72] python/qmp: Fix type of SocketAddrT
In porting the qom tools, qmp-shell, etc; it becomes evident that this type is wrong. This is an integer, not a string. We didn't catch this before because none of QEMUMonitorProtocol's *users* happen to be checked, and the internal logic of this class is otherwise self-consistent. Additionally, mypy was not introspecting into the socket() interface to realize we were passing a bad type for AF_INET. Fixed now. Signed-off-by: John Snow Reviewed-by: Philippe Mathieu-Daudé Message-id: 20210603003719.1321369-3-js...@redhat.com Signed-off-by: John Snow --- python/qemu/qmp/__init__.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/python/qemu/qmp/__init__.py b/python/qemu/qmp/__init__.py index 9606248a3d..5fb970f8a8 100644 --- a/python/qemu/qmp/__init__.py +++ b/python/qemu/qmp/__init__.py @@ -44,7 +44,7 @@ QMPMessage = Dict[str, Any] QMPReturnValue = Dict[str, Any] -InternetAddrT = Tuple[str, str] +InternetAddrT = Tuple[str, int] UnixAddrT = str SocketAddrT = Union[InternetAddrT, UnixAddrT] -- 2.31.1
[PULL 00/72] Python patches
The following changes since commit 3ccf6cd0e3e1dfd663814640b3b18b55715d7a75: Merge remote-tracking branch 'remotes/kraxel/tags/audio-20210617-pull-request' into staging (2021-06-18 09:54:42 +0100) are available in the Git repository at: https://gitlab.com/jsnow/qemu.git tags/python-pull-request for you to fetch changes up to d08caefe6648fc0713af5361e2b88bee53b67ebb: scripts/qmp-shell: add redirection shim (2021-06-18 16:10:07 -0400) Python Pull request Moves QMP-related tools not used for build or automatic testing from scripts/ to python/qemu/qmp/ where they will be protected from bitrot by the check-python-* CI jobs. stub forwarders are left in the old locations for now. John Snow (72): python/pipenv: Update Pipfile.lock python/qmp: Fix type of SocketAddrT python/qmp: add parse_address classmethod python/qmp: Add qom script rewrites python/qmp: add qom script entry points scripts/qmp: redirect qom-xxx scripts to python/qemu/qmp/ scripts/qom-fuse: apply isort rules scripts/qom-fuse: apply flake8 rules python: Add 'fh' to known-good variable names scripts/qom-fuse: Apply pylint rules scripts/qom-fuse: Add docstrings scripts/qom-fuse: Convert to QOMCommand scripts/qom-fuse: use QOMCommand.qom_list() scripts/qom-fuse: ensure QOMFuse.read always returns bytes scripts/qom-fuse: add static type hints python: add optional FUSE dependencies scripts/qom-fuse: move to python/qemu/qmp/qom_fuse.py scripts/qom-fuse: add redirection shim to python/qemu/qmp/qom-fuse.py python/qmp: add fuse command to 'qom' tools scripts/qemu-ga-client: apply isort rules scripts/qemu-ga-client: apply (most) flake8 rules scripts/qemu-ga-client: Fix exception handling scripts/qemu-ga-client: replace deprecated optparse with argparse scripts/qemu-ga-client: add module docstring scripts/qemu-ga-client: apply (most) pylint rules python/qmp: Correct type of QMPReturnValue scripts/qemu-ga-client: add mypy type hints scripts/qemu-ga-client: move to python/qemu/qmp/qemu_ga_client.py python/qemu-ga-client: add entry point scripts/qemu-ga-client: Add forwarder shim scripts/qmp-shell: apply isort rules scripts/qmp-shell: Apply flake8 rules scripts/qmp-shell: fix show_banner signature scripts/qmp-shell: fix exception handling scripts/qmp-shell: fix connect method signature scripts/qmp-shell: remove shadowed variable from _print() scripts/qmp-shell: use @classmethod where appropriate scripts/qmp-shell: Use python3-style super() scripts/qmp-shell: declare verbose in __init__ scripts/qmp-shell: use triple-double-quote docstring style scripts/qmp-shell: ignore visit_Name name scripts/qmp-shell: make QMPCompleter returns explicit scripts/qmp-shell: rename one and two-letter variables scripts/qmp-shell: fix shell history exception handling scripts/qmp-shell: remove if-raise-else patterns scripts/qmp-shell: use isinstance() instead of type() scripts/qmp-shell: use argparse scripts/qmp-shell: Add pretty attribute to HMP shell scripts/qmp-shell: Make verbose a public attribute scripts/qmp-shell: move get_prompt() to prompt property scripts/qmp-shell: remove prompt argument from read_exec_command scripts/qmp-shell: move the REPL functionality into QMPShell scripts/qmp-shell: Fix "FuzzyJSON" parser scripts/qmp-shell: refactor QMPCompleter scripts/qmp-shell: initialize completer early python/qmp: add QMPObject type alias scripts/qmp-shell: add mypy types scripts/qmp-shell: Accept SocketAddrT instead of string scripts/qmp-shell: unprivatize 'pretty' property python/qmp: return generic type from context manager scripts/qmp-shell: Use context manager instead of atexit scripts/qmp-shell: use logging to show warnings scripts/qmp-shell: remove TODO scripts/qmp-shell: Fix empty-transaction invocation scripts/qmp-shell: Remove too-broad-exception scripts/qmp-shell: convert usage comment to docstring scripts/qmp-shell: remove double-underscores scripts/qmp-shell: make QMPShellError inherit QMPError scripts/qmp-shell: add docstrings scripts/qmp-shell: move to python/qemu/qmp/qmp_shell.py python: add qmp-shell entry point scripts/qmp-shell: add redirection shim python/Pipfile.lock | 97 +- python/qemu/qmp/__init__.py | 61 +++- python/qemu/qmp/qemu_ga_client.py | 323 ++ python/qemu/qmp/qmp_shell.py | 535 ++ python/qemu/qmp/qom.py| 272 +++ python/qemu/qmp/qom_common.py | 178 ++ python/qemu/qmp/qom_fuse.py | 206 python/setup.cfg | 35 +- scripts/qmp/qemu-ga-client| 297 + scripts/qmp/qmp-shell | 454 + scripts/qmp/qom-fuse | 144 +--- scripts/qmp/qom-get
Re: [PATCH 9/9] target/mips: Optimize regnames[] arrays
On 6/17/21 10:43 AM, Philippe Mathieu-Daudé wrote: Since all entries are no more than 3/4/6 bytes (including nul terminator), can save space and pie runtime relocations by declaring regnames[] as array of 3/4/6 const char. Inspired-by: Richard Henderson Signed-off-by: Philippe Mathieu-Daudé --- target/mips/internal.h | 2 +- target/mips/cpu.c | 2 +- target/mips/tcg/msa_translate.c | 2 +- target/mips/tcg/mxu_translate.c | 4 ++-- target/mips/tcg/translate.c | 4 ++-- 5 files changed, 7 insertions(+), 7 deletions(-) Reviewed-by: Richard Henderson r~
Re: [PATCH 8/9] target/mips: Constify host_to_mips_errno[]
On 6/17/21 10:43 AM, Philippe Mathieu-Daudé wrote: Keep host_to_mips_errno[] in .rodata by marking the array const. Signed-off-by: Philippe Mathieu-Daudé --- target/mips/tcg/sysemu/mips-semi.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) Reviewed-by: Richard Henderson r~
Re: [PATCH 7/9] target/mips: Remove microMIPS BPOSGE32 / BPOSGE64 unuseful cases
On 6/17/21 10:43 AM, Philippe Mathieu-Daudé wrote:
These switch cases for the microMIPS BPOSGE32 / BPOSGE64 opcodes have
been added commit 3c824109da0 ("target-mips: microMIPS ASE support").
More than 11 years later it is safe to assume there won't be added
soon. The cases fall back to the default which generates a RESERVED
INSTRUCTION, so it is safe to remove them.
Signed-off-by: Philippe Mathieu-Daudé
---
target/mips/tcg/translate.c | 6 --
1 file changed, 6 deletions(-)
Reviewed-by: Richard Henderson
r~
Re: [PATCH 6/9] target/mips: Remove SmartMIPS / MDMX unuseful comments
On 6/17/21 10:43 AM, Philippe Mathieu-Daudé wrote:
These placeholder comments for SmartMIPS and MDMX extensions have
been added commit 3c824109da0 ("target-mips: microMIPS ASE support").
More than 11 years later it is safe to assume there won't be added
soon, so remove these unuseful comments.
Signed-off-by: Philippe Mathieu-Daudé
---
target/mips/tcg/translate.c | 8
1 file changed, 8 deletions(-)
Reviewed-by: Richard Henderson
r~
Re: [PATCH 5/9] target/mips: Restrict some system specific declarations to sysemu
On 6/17/21 10:43 AM, Philippe Mathieu-Daudé wrote:
Commit 043715d1e0f ("target/mips: Update ITU to utilize SAARI
and SAAR CP0 registers") declared itc_reconfigure() in public
namespace, while it is restricted to system emulation.
Similarly commit 5679479b9a1 ("target/mips: Move CP0 helpers
to sysemu/cp0.c") restricted cpu_mips_soft_irq() definition to
system emulation, but forgot to restrict its declaration.
Commit a2b0a27d33e ("target/mips: Move TCG source files under
tcg/ sub directory") restricted mips_cpu_do_unaligned_access()
to system emulation but forgot to guard the declaration.
FWIW, I think the unaligned_access tcg_ops hook should be available for user-only as well,
like for tlb_fill. That it is sysemu only at the moment is a bug that affects user-only
atomic access.
Anyway,
Reviewed-by: Richard Henderson
r~
To avoid polluting user-mode emulation with these declarations,
restrict them to sysemu. Also restrict the sysemu ITU/ITC/IRQ
fields from CPUMIPSState.
Signed-off-by: Philippe Mathieu-Daudé
---
target/mips/cpu.h | 10 +++---
target/mips/tcg/tcg-internal.h | 6 +++---
2 files changed, 10 insertions(+), 6 deletions(-)
Re: [PATCH 4/9] target/mips: Move translate.h to tcg/ sub directory
On 6/17/21 10:43 AM, Philippe Mathieu-Daudé wrote: diff --git a/target/mips/tcg/msa_translate.c b/target/mips/tcg/msa_translate.c index ae6587edf69..b3b06352bf2 100644 --- a/target/mips/tcg/msa_translate.c +++ b/target/mips/tcg/msa_translate.c @@ -13,7 +13,7 @@ #include "qemu/osdep.h" #include "tcg/tcg-op.h" #include "exec/helper-gen.h" -#include "translate.h" +#include "tcg/translate.h" #include "fpu_helper.h" #include "internal.h" You shouldn't need to make include changes, because the search path for "" begins in the $cwd. But I guess it works. Reviewed-by: Richard Henderson r~
