[PATCH v3 0/5] plugins/cache: multicore cache modelling and minor tweaks

[Mahmoud Mandour]

Hello,

This series introduce some minor improvements/bug fixes in the cache
plugins and multicore cache modelling.

Multi-core cache modelling is handled such that for full-system
emulation, a private L1 cache is maintained to each core available to
the system. For multi-threaded userspace emulation, a static number of
cores is maintained for the overall system, and every memory access go
through one of these, even if the number of fired threads is more than
that number.

Also, raising the levels of warnings induced some warnings, this is
fixed in it's own patch.

Patches that still need review (other patches are already queued):
1. plugins/cache: Supported multicore cache modelling
2. docs/devel/tcg-plugins: added cores arg to cache plugin

v2 -> v3:
1. Included the link of the patch solving the race.
2. Fixed the title so that the series appears in Patchew.

v1 -> v2:
1. Dropped the patch with mitigating the use-after-free bug since
it's not needed anymore (fixed by Alex Bennée here
<20210720232703.10650-1-alex.ben...@linaro.org> with patch name:
tcg/plugins: implement a qemu_plugin_user_exit helper)
2. Summed cache performance data as a post-processing step.
3. Refactored appending core data to it's own function

Mahmoud Mandour (5):
  plugins/cache: Fixed a bug with destroying FIFO metadata
  plugins/cache: limited the scope of a mutex lock
  plugins/cache: Supported multicore cache modelling
  docs/devel/tcg-plugins: added cores arg to cache plugin
  plugins/cache: Fixed "function decl. is not a prototype" warnings

 contrib/plugins/cache.c| 165 -
 docs/devel/tcg-plugins.rst |  13 +--
 2 files changed, 132 insertions(+), 46 deletions(-)

-- 
2.25.1

[PATCH v3 5/5] plugins/cache: Fixed "function decl. is not a prototype" warnings

[Mahmoud Mandour]

Signed-off-by: Mahmoud Mandour 
---
 contrib/plugins/cache.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/contrib/plugins/cache.c b/contrib/plugins/cache.c
index 496d6e7d49..cc9020b2d5 100644
--- a/contrib/plugins/cache.c
+++ b/contrib/plugins/cache.c
@@ -535,7 +535,7 @@ static void sum_stats(void)
 }
 }
 
-static void log_stats()
+static void log_stats(void)
 {
 int i, iters;
 
@@ -559,7 +559,7 @@ static void log_stats()
 qemu_plugin_outs(rep->str);
 }
 
-static void log_top_insns()
+static void log_top_insns(void)
 {
 int i;
 GList *curr, *miss_insns;
@@ -610,7 +610,7 @@ static void plugin_exit(qemu_plugin_id_t id, void *p)
 g_free(stats);
 }
 
-static void policy_init()
+static void policy_init(void)
 {
 switch (policy) {
 case LRU:
-- 
2.25.1

[PATCH v3 4/5] docs/devel/tcg-plugins: added cores arg to cache plugin

[Mahmoud Mandour]

Signed-off-by: Mahmoud Mandour 
---
 docs/devel/tcg-plugins.rst | 13 -
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/docs/devel/tcg-plugins.rst b/docs/devel/tcg-plugins.rst
index 595b8e0ea4..370c11373f 100644
--- a/docs/devel/tcg-plugins.rst
+++ b/docs/devel/tcg-plugins.rst
@@ -330,11 +330,8 @@ configuration when a given working set is run::
 
 will report the following::
 
-Data accesses: 996479, Misses: 507
-Miss rate: 0.050879%
-
-Instruction accesses: 2641737, Misses: 18617
-Miss rate: 0.704726%
+core #, data accesses, data misses, dmiss rate, insn accesses, insn 
misses, imiss rate
+0   996695 508 0.0510%  264279918617   
0.7044%
 
 address, data misses, instruction
 0x424f1e (_int_malloc), 109, movq %rax, 8(%rcx)
@@ -378,3 +375,9 @@ The plugin has a number of arguments, all of them are 
optional:
   Sets the eviction policy to POLICY. Available policies are: :code:`lru`,
   :code:`fifo`, and :code:`rand`. The plugin will use the specified policy for
   both instruction and data caches. (default: POLICY = :code:`lru`)
+
+  * arg="cores=N"
+
+  Sets the number of cores for which we maintain separate icache and dcache.
+  (default: for linux-user, N = 1, for full system emulation: N = cores
+  available to guest)
-- 
2.25.1

[PATCH v3 1/5] plugins/cache: Fixed a bug with destroying FIFO metadata

[Mahmoud Mandour]

This manifests itself when associativity degree is greater than the
number of sets and FIFO is used, otherwise it's also a memory leak
whenever FIFO was used.

Signed-off-by: Mahmoud Mandour 
---
 contrib/plugins/cache.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/contrib/plugins/cache.c b/contrib/plugins/cache.c
index bf0d2f6097..4a71602639 100644
--- a/contrib/plugins/cache.c
+++ b/contrib/plugins/cache.c
@@ -200,7 +200,7 @@ static void fifo_destroy(Cache *cache)
 {
 int i;
 
-for (i = 0; i < cache->assoc; i++) {
+for (i = 0; i < cache->num_sets; i++) {
 g_queue_free(cache->sets[i].fifo_queue);
 }
 }
-- 
2.25.1

Re: Installation challenges

[Stefan Weil]

Am 22.07.21 um 05:02 schrieb Nicholas Kyanda:


Hello,

Thank you for your software. I just installed the 64 bit windows exe 
unfortunately, I cannot find it on my desktop. How do I start using it?

Sincerely,
Nicholas Kyanda



QEMU is a console application, therefore there is neither an icon on 
your desktop nor a menu entry to run it.


You have to run cmd.exe or the Windows power shell and start the desired 
QEMU executable from that command line.


Regards

Stefan Weil

Re: [PATCH for-6.2 v2 10/11] machine: Split out the smp parsing code

[wangyanan (Y)]

On 2021/7/20 1:20, Andrew Jones wrote:

On Mon, Jul 19, 2021 at 11:20:42AM +0800, Yanan Wang wrote:

We are going to introduce an unit test for the parser smp_parse()
in hw/core/machine.c, but now machine.c is only built in softmmu.

In order to solve the build dependency on the smp parsing code and
avoid building unrelated stuff for the unit tests, move the related
code from machine.c into a new common file, i.e., machine-smp.c.

Signed-off-by: Yanan Wang 
---
  MAINTAINERS   |   1 +
  hw/core/machine-smp.c | 124 ++
  hw/core/machine.c | 109 -
  hw/core/meson.build   |   1 +
  include/hw/boards.h   |   1 +
  5 files changed, 127 insertions(+), 109 deletions(-)
  create mode 100644 hw/core/machine-smp.c

diff --git a/MAINTAINERS b/MAINTAINERS
index 9100f9a043..70633e3bf4 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1626,6 +1626,7 @@ F: cpu.c
  F: hw/core/cpu.c
  F: hw/core/machine-qmp-cmds.c
  F: hw/core/machine.c
+F: hw/core/machine-smp.c
  F: hw/core/null-machine.c
  F: hw/core/numa.c
  F: hw/cpu/cluster.c
diff --git a/hw/core/machine-smp.c b/hw/core/machine-smp.c
new file mode 100644
index 00..6a00cfe44a
--- /dev/null
+++ b/hw/core/machine-smp.c
@@ -0,0 +1,124 @@
+/*
+ * QEMU Machine (related to SMP configuration)
+ *
+ * Copyright (C) 2014 Red Hat Inc
+ *
+ * Authors:
+ *   Marcel Apfelbaum 

This header was obviously copy+pasted without being updated.

Yes, the header was kept unchanged.

But actually I'm not completely sure which field should be updated. :)
Should I add "Copyright (C) 2021, Huawei, Inc." and also the authorship
"Yanan Wang " behind the existing ones
or just replace them?

+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "hw/boards.h"
+#include "qapi/error.h"
+
+/*
+ * smp_parse - Generic function used to parse the given SMP configuration
+ *
+ * The topology parameters must be specified equal to or great than one
+ * or just omitted, explicit configuration like "cpus=0" is not allowed.
+ * The omitted parameters will be calculated based on the provided ones.
+ *
+ * maxcpus will default to the value of cpus if omitted and will be used
+ * to compute the missing sockets/cores/threads. cpus will be calculated
+ * from the computed parametrs if omitted.
+ *
+ * In calculation of omitted arch-netural sockets/cores/threads, we prefer
+ * sockets over cores over threads before 6.2, while prefer cores over
+ * sockets over threads since 6.2 on. The arch-specific dies will directly
+ * default to 1 if omitted.
+ */
+void smp_parse(MachineState *ms, SMPConfiguration *config, Error **errp)
+{
+MachineClass *mc = MACHINE_GET_CLASS(ms);
+unsigned cpus= config->has_cpus ? config->cpus : 0;
+unsigned sockets = config->has_sockets ? config->sockets : 0;
+unsigned dies= config->has_dies ? config->dies : 1;
+unsigned cores   = config->has_cores ? config->cores : 0;
+unsigned threads = config->has_threads ? config->threads : 0;
+unsigned maxcpus = config->has_maxcpus ? config->maxcpus : 0;
+
+if ((config->has_cpus && config->cpus == 0) ||
+(config->has_sockets && config->sockets == 0) ||
+(config->has_dies && config->dies == 0) ||
+(config->has_cores && config->cores == 0) ||
+(config->has_threads && config->threads == 0) ||
+(config->has_maxcpus && config->maxcpus == 0)) {
+error_setg(errp, "parameters must be equal to or greater than one"
+   "if provided");
+return;
+}
+
+if (!mc->smp_dies_supported && dies > 1) {
+error_setg(errp, "dies not supported by this machine's CPU topology");
+return;
+}
+
+maxcpus = maxcpus > 0 ? maxcpus : cpus;
+
+/* prefer sockets over cores over threads before 6.2 */
+if (mc->smp_prefer_sockets) {
+if (sockets == 0) {
+cores = cores > 0 ? cores : 1;
+threads = threads > 0 ? threads : 1;
+sockets = maxcpus / (dies * cores * threads);
+sockets = sockets > 0 ? sockets : 1;
+} else if (cores == 0) {
+threads = threads > 0 ? threads : 1;
+cores = maxcpus / (sockets * dies * threads);
+cores = cores > 0 ? cores : 1;
+} else if (threads == 0) {
+threads = maxcpus / (sockets * dies * cores);
+threads = threads > 0 ? threads : 1;
+}
+/* prefer cores over sockets over threads since 6.2 */
+} else {
+if (cores == 0) {
+sockets = sockets > 0 ? sockets : 1;
+threads = threads > 0 ? threads : 1;
+cores = maxcpus / (sockets * dies * threads);
+cores = cores > 0 ? cores : 1;
+} else if (sockets == 0) {
+threads = threads > 0 ? threads : 1;
+sockets = maxcpus / (dies * cores * threa

Installation challenges

[Nicholas Kyanda]

Hello,

Thank you for your software. I just installed the 64 bit windows exe
unfortunately, I cannot find it on my desktop. How do I start using it?
Sincerely,
Nicholas Kyanda

[PATCH] Makefile: ignore long options

[Alexey Neyman]

When searching for options like -n in MAKEFLAGS, current code may result
in a false positive match when make is invoked with long options like
--no-print-directory. This has been observed with certain versions of
host make (e.g. 3.82) while building the Qemu package in buildroot.

Filter out such long options before searching for one-character options.

Signed-off-by: Alexey Neyman 
---
 Makefile | 8 +---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/Makefile b/Makefile
index 6c36330eef..401c623a65 100644
--- a/Makefile
+++ b/Makefile
@@ -129,9 +129,11 @@ endif
 # 4. Rules to bridge to other makefiles
 
 ifneq ($(NINJA),)
-MAKE.n = $(findstring n,$(firstword $(MAKEFLAGS)))
-MAKE.k = $(findstring k,$(firstword $(MAKEFLAGS)))
-MAKE.q = $(findstring q,$(firstword $(MAKEFLAGS)))
+# Filter out long options to avoid flags like --no-print-directory which
+# may result in false positive match for MAKE.n
+MAKE.n = $(findstring n,$(firstword $(filter-out --%,$(MAKEFLAGS
+MAKE.k = $(findstring k,$(firstword $(filter-out --%,$(MAKEFLAGS
+MAKE.q = $(findstring q,$(firstword $(filter-out --%,$(MAKEFLAGS
 MAKE.nq = $(if $(word 2, $(MAKE.n) $(MAKE.q)),nq)
 NINJAFLAGS = $(if $V,-v) $(if $(MAKE.n), -n) $(if $(MAKE.k), -k0) \
 $(filter-out -j, $(lastword -j1 $(filter -l% -j%, $(MAKEFLAGS \
-- 
2.27.0

Re: [PATCH for-6.2 v2 11/11] tests/unit: Add a unit test for smp parsing

[wangyanan (Y)]

On 2021/7/20 2:57, Andrew Jones wrote:

On Mon, Jul 19, 2021 at 11:20:43AM +0800, Yanan Wang wrote:

Add a QEMU unit test for the parsing of given SMP configuration.
Since all the parsing logic is in generic function smp_parse(),
this test passes diffenent SMP configurations to the function
and compare the parsing result with what is expected.

In the test, all possible collections of the topology parameters
and the corressponding expected results are listed, including the
valid and invalid ones.

The preference of sockets over cores and the preference of cores
over sockets, and the support of multi-dies are also considered.

Signed-off-by: Yanan Wang 
---
  MAINTAINERS |1 +
  tests/unit/meson.build  |1 +
  tests/unit/test-smp-parse.c | 1117 +++
  3 files changed, 1119 insertions(+)
  create mode 100644 tests/unit/test-smp-parse.c

diff --git a/MAINTAINERS b/MAINTAINERS
index 70633e3bf4..160dba2e57 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1636,6 +1636,7 @@ F: include/hw/boards.h
  F: include/hw/core/cpu.h
  F: include/hw/cpu/cluster.h
  F: include/sysemu/numa.h
+F: tests/unit/test-smp-parse.c
  T: git https://gitlab.com/ehabkost/qemu.git machine-next
  
  Xtensa Machines

diff --git a/tests/unit/meson.build b/tests/unit/meson.build
index 3e0504dd21..694a924627 100644
--- a/tests/unit/meson.build
+++ b/tests/unit/meson.build
@@ -44,6 +44,7 @@ tests = {
'test-uuid': [],
'ptimer-test': ['ptimer-test-stubs.c', meson.source_root() / 
'hw/core/ptimer.c'],
'test-qapi-util': [],
+  'test-smp-parse': [qom, meson.source_root() / 'hw/core/machine-smp.c'],
  }
  
  if have_system or have_tools

diff --git a/tests/unit/test-smp-parse.c b/tests/unit/test-smp-parse.c
new file mode 100644
index 00..bc1d324c3d
--- /dev/null
+++ b/tests/unit/test-smp-parse.c
@@ -0,0 +1,1117 @@
+/*
+ * SMP parsing unit-tests
+ *
+ * Copyright (C) 2021, Huawei, Inc.
+ *
+ * Authors:
+ *  Yanan Wang 
+ *
+ * This work is licensed under the terms of the GNU LGPL, version 2.1 or later.
+ * See the COPYING.LIB file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qom/object.h"
+#include "qemu/module.h"
+#include "qapi/error.h"
+
+#include "hw/boards.h"
+
+#define T true
+#define F false
+
+/**
+ * SMPTestData:
+ * @config - the given SMP configuration for parsing
+ * @should_be_valid - whether the given configuration is supposed to be valid
+ * @expect - the CPU topology info expected to be parsed out
+ */
+typedef struct SMPTestData {
+SMPConfiguration config;
+bool should_be_valid;

Long way to say 'valid'.

Indeed..., "valid" should be enough.

+CpuTopology expect;
+} SMPTestData;
+
+/* the specific machine type info for this test */
+static const TypeInfo smp_machine_info = {
+.name = TYPE_MACHINE,
+.parent = TYPE_OBJECT,
+.class_size = sizeof(MachineClass),
+.instance_size = sizeof(MachineState),
+};
+
+/*
+ * prefer sockets over cores over threads before 6.2.
+ * all possible SMP configurations and the corressponding expected outputs

corresponding (please run spell check on your commit messages)


Ok, I missed the check.

+ * are listed for testing, including the valid and invalid ones.
+ */
+static struct SMPTestData prefer_sockets[] = {
+{
+/* config: no smp configuration provided
+ * expect: cpus=1,sockets=1,dies=1,cores=1,threads=1,maxcpus=1 */
+.config = (SMPConfiguration) { F, 0, F, 0, F, 0, F, 0, F, 0, F, 0 },

SMPConfiguration and CpuTopology have named fields so we could drop the
'expect: ...' comment line and instead do

  {
   /* no configuration provided */
   .config = { .has_cpus = F, .has_maxcpus = F, ... },
   .valid = T,
   .expect = { .sockets = 1, .cores = 1, ... },
  }, {
   ...
  }

which may be easier to maintain. OTOH, the concise form this approach has
is also nice.
I tried the structure initialization with explicit name fields in it 
like above,

actually we are supposed to do in this way so that we don't have to worry
about the order change of the structure members.

But this would break the 80-char line limit or introduce more lines for
for each SMP configuration. If this is not a real problem, I also prefer
above format.

I don't think you should need the casts in the assignments
though.

Yes, the casts may be unnecessary, will remove them.



+.should_be_valid = true,
+.expect = (CpuTopology) { 1, 1, 1, 1, 1, 1 },
+}, {
+/* config: -smp 8
+ * expect: cpus=8,sockets=8,dies=1,cores=1,threads=1,maxcpus=8 */
+.config = (SMPConfiguration) { T, 8, F, 0, F, 0, F, 0, F, 0, F, 0 },
+.should_be_valid = true,
+.expect = (CpuTopology) { 8, 8, 1, 1, 1, 8 },
+}, {
+/* config: -smp sockets=2
+ * expect: cpus=2,sockets=2,dies=1,cores=1,threads=1,maxcpus=2 */
+.config = (SMPConfiguration) { F, 0, T, 2, F, 0, F, 0, F, 0, F, 0 },
+.should_be_valid = true,
+.expect =

Re: [PATCH for-6.1 0/1] machine: Disallow specifying topology parameters as zero

[Cornelia Huck]

On Thu, Jul 22 2021, Yanan Wang  wrote:

> In the SMP configuration, we should either specify a topology
> parameter with a reasonable value (equal to or greater than 1)
> or just leave it omitted and QEMU will calculate its value.
> Configurations which explicitly specify the topology parameters
> as zero like "sockets=0" are meaningless, so disallow them.
>
> However; the commit 1e63fe685804d
> (machine: pass QAPI struct to mc->smp_parse) has documented that
> '0' has the same semantics as omitting a parameter in the qapi
> comment for SMPConfiguration. So this patch fixes the doc and
> also adds the corresponding sanity check in the smp parsers.

Are we expecting any real users to have used that 'parameter=0'
behaviour? I expect that libvirt and other management software already
did the right thing; unfortunately, sometimes weird configuration lines
tend to persist in search results.

>
> This patch originly comes form [1], and it was suggested that
> this patch fixing the doc should be sent for 6.1 to avoid a
> deprecation process in the future.
>
> [1] https://lore.kernel.org/qemu-devel/ypwsthpiza3mf...@redhat.com/
>
> Yanan Wang (1):
>   machine: Disallow specifying topology parameters as zero
>
>  hw/core/machine.c | 30 ++
>  hw/i386/pc.c  | 33 -
>  qapi/machine.json |  6 +++---
>  3 files changed; 49 insertions(+); 20 deletions(-)

Re: [PATCH for-6.2 v2 07/11] machine: Prefer cores over sockets in smp parsing since 6.2

[wangyanan (Y)]

On 2021/7/20 1:13, Andrew Jones wrote:

On Mon, Jul 19, 2021 at 11:20:39AM +0800, Yanan Wang wrote:

In the real SMP hardware topology world, it's much more likely that
we have high cores-per-socket counts and few sockets totally. While
the current preference of sockets over cores in smp parsing results
in a virtual cpu topology with low cores-per-sockets counts and a
large number of sockets, which is just contrary to the real world.

Given that it is better to make the virtual cpu topology be more
reflective of the real world and also for the sake of compatibility,
we start to prefer cores over sockets over threads in smp parsing
since machine type 6.2 for different arches.

In this patch, a boolean "smp_prefer_sockets" is added, and we only
enable the old preference on older machines and enable the new one
since type 6.2 for all arches by using the machine compat mechanism.

Suggested-by: Daniel P. Berrange 
Signed-off-by: Yanan Wang 
---
  hw/arm/virt.c  |  1 +
  hw/core/machine.c  | 59 +-
  hw/i386/pc_piix.c  |  1 +
  hw/i386/pc_q35.c   |  1 +
  hw/ppc/spapr.c |  1 +
  hw/s390x/s390-virtio-ccw.c |  1 +
  include/hw/boards.h|  1 +
  qemu-options.hx|  4 ++-
  8 files changed, 55 insertions(+), 14 deletions(-)

diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 01165f7f53..7babea40dc 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -2797,6 +2797,7 @@ static void virt_machine_6_1_options(MachineClass *mc)
  {
  virt_machine_6_2_options(mc);
  compat_props_add(mc->compat_props, hw_compat_6_1, hw_compat_6_1_len);
+mc->smp_prefer_sockets = true;
  }
  DEFINE_VIRT_MACHINE(6, 1)
  
diff --git a/hw/core/machine.c b/hw/core/machine.c

index 63439c4a6d..c074425015 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -744,6 +744,22 @@ void machine_set_cpu_numa_node(MachineState *machine,
  }
  }
  
+/*

+ * smp_parse - Generic function used to parse the given SMP configuration
+ *
+ * The topology parameters must be specified equal to or great than one
+ * or just omitted, explicit configuration like "cpus=0" is not allowed.
+ * The omitted parameters will be calculated based on the provided ones.
+ *
+ * maxcpus will default to the value of cpus if omitted and will be used
+ * to compute the missing sockets/cores/threads. cpus will be calculated
+ * from the computed parametrs if omitted.

parameters

Or how about something like this:

When both maxcpus and cpus are omitted maxcpus will be calculated from the
given parameters and cpus will be set equal to maxcpus. When only one of
maxcpus and cpus is given then the omitted one will be set to its given
counterpart's value. Both maxcpus and cpus may be specified, but cpus must
be less than or equal to maxcpus.

Yes, this is much clearer.
Now I understand why you suggested in patch #4 to change the format from
  cpus = cpus > 0 ? cpus : sockets * dies * cores * threads;
  maxcpus = maxcpus > 0 ? maxcpus : cpus;
to
  maxcpus = maxcpus > 0 ? maxcpus : sockets * dies * cores * threads;
  cpus = cpus > 0 ? cpus : maxcpus;

What you suggest seems more reasonable according to above doc,
I will adjust the related part as you suggested.

+ *
+ * In calculation of omitted arch-netural sockets/cores/threads, we prefer
+ * sockets over cores over threads before 6.2, while prefer cores over

while preferring


+ * sockets over threads since 6.2 on. The arch-specific dies will directly

s/on//

Right.

+ * default to 1 if omitted.
+ */
  static void smp_parse(MachineState *ms, SMPConfiguration *config, Error 
**errp)
  {
  MachineClass *mc = MACHINE_GET_CLASS(ms);
@@ -772,19 +788,36 @@ static void smp_parse(MachineState *ms, SMPConfiguration 
*config, Error **errp)
  
  maxcpus = maxcpus > 0 ? maxcpus : cpus;
  
-/* compute missing values, prefer sockets over cores over threads */

-if (sockets == 0) {
-cores = cores > 0 ? cores : 1;
-threads = threads > 0 ? threads : 1;
-sockets = maxcpus / (dies * cores * threads);
-sockets = sockets > 0 ? sockets : 1;
-} else if (cores == 0) {
-threads = threads > 0 ? threads : 1;
-cores = maxcpus / (sockets * dies * threads);
-cores = cores > 0 ? cores : 1;
-} else if (threads == 0) {
-threads = maxcpus / (sockets * dies * cores);
-threads = threads > 0 ? threads : 1;
+/* prefer sockets over cores over threads before 6.2 */
+if (mc->smp_prefer_sockets) {

please move the comment into the if, so...


+if (sockets == 0) {
+cores = cores > 0 ? cores : 1;
+threads = threads > 0 ? threads : 1;
+sockets = maxcpus / (dies * cores * threads);
+sockets = sockets > 0 ? sockets : 1;
+} else if (cores == 0) {
+threads = threads > 0 ? threads : 1;
+cores = maxcpus / (sockets * dies * threads);
+cores = cores > 0 ? cores : 1;
+  

Re: [PATCH] acpi/gpex: Inform os to keep firmware resource map

[Guenter Roeck]

Hi,

On Thu, Dec 17, 2020 at 09:29:26PM +0800, Jiahui Cen wrote:
> There may be some differences in pci resource assignment between guest os
> and firmware.
> 
> Eg. A Bridge with Bus [d2]
> -+-[:d2]---01.0-[d3]01.0
> 
> where [d2:01.00] is a pcie-pci-bridge with BAR0 (mem, 64-bit, non-pref) 
> [size=256]
>   [d3:01.00] is a PCI Device with BAR0 (mem, 64-bit, pref) [size=128K]
>   BAR4 (mem, 64-bit, pref) [size=64M]
> 
> In EDK2, the Resource Map would be:
> PciBus: Resource Map for Bridge [D2|01|00]
> Type = PMem64; Base = 0x800400; Length = 0x410; 
> Alignment = 0x3FF
>Base = 0x800400; Length = 0x400; Alignment = 
> 0x3FF;  Owner = PCI [D3|01|00:20]
>Base = 0x800800; Length = 0x2;   Alignment = 0x1;  
>   Owner = PCI [D3|01|00:10]
> Type =  Mem64; Base = 0x800810; Length = 0x100; Alignment = 
> 0xFFF
> 
> While in Linux, kernel will use 0x2FF as the alignment to calculate
> the PMem64 size, which would be 0x600.
> 
> The diffences could result in resource assignment failure.
> 
> Using _DSM #5 method to inform guest os not to ignore the PCI configuration
> that firmware has done at boot time could handle the differences.
> 
> Signed-off-by: Jiahui Cen 

Since this patch is in qemu (ie starting with qemu v6.0), some of my qemu
tests booting aarch64 systems with efi bios no longer work. For example,
the following command fails to instantiate the Ethernet interface.

CMDLINE="root=/dev/vda console=ttyAMA0"
ROOTFS="rootfs.ext2"

qemu-system-aarch64 -M virt -kernel arch/arm64/boot/Image -no-reboot \
-m 512 -cpu cortex-a57 -no-reboot \
-device tulip,netdev=net0 -netdev user,id=net0 \
-bios QEMU_EFI-aarch64.fd \
-snapshot \
-device virtio-blk-device,drive=d0 \
-drive file=${ROOTFS},if=none,id=d0,format=raw \
-nographic -serial stdio -monitor none \
--append "${CMDLINE}"

QEMU_EFI-aarch64.fd is from https://retrage.github.io/edk2-nightly/.

Key difference is PCI BAR assignment.

good (qemu v5.2):

[3.921801] pci :00:01.0: [1011:0019] type 00 class 0x02
[3.922207] pci :00:01.0: reg 0x10: [io  0x-0x007f]
[3.922505] pci :00:01.0: reg 0x14: [mem 0x1000-0x107f]
[3.927111] pci :00:01.0: BAR 0: assigned [io  0x1000-0x107f]
[3.927455] pci :00:01.0: BAR 1: assigned [mem 0x1000-0x107f]

bad (qemu v6.0 and later):

[3.922887] pci :00:01.0: [1011:0019] type 00 class 0x02
[3.923278] pci :00:01.0: reg 0x10: [io  0x-0x007f]
[3.923451] pci :00:01.0: reg 0x14: [mem 0x1000-0x107f]

Reverting this patch fixes the problem.

Does this mean that I can no longer run aarch64 efi boot tests
with qemu v6.0 and later if the test involves PCI devices ?
Or is there some alternative command line which would still work ?

Thanks,
Guenter

Re: [PATCH for-6.2 v2 07/11] machine: Prefer cores over sockets in smp parsing since 6.2

[wangyanan (Y)]

On 2021/7/19 11:40, David Gibson wrote:

On Mon, Jul 19, 2021 at 11:20:39AM +0800, Yanan Wang wrote:

In the real SMP hardware topology world, it's much more likely that
we have high cores-per-socket counts and few sockets totally. While
the current preference of sockets over cores in smp parsing results
in a virtual cpu topology with low cores-per-sockets counts and a
large number of sockets, which is just contrary to the real world.

Given that it is better to make the virtual cpu topology be more
reflective of the real world and also for the sake of compatibility,
we start to prefer cores over sockets over threads in smp parsing
since machine type 6.2 for different arches.

In this patch, a boolean "smp_prefer_sockets" is added, and we only
enable the old preference on older machines and enable the new one
since type 6.2 for all arches by using the machine compat mechanism.

Suggested-by: Daniel P. Berrange 
Signed-off-by: Yanan Wang 

ppc parts

Acked-by: David Gibson 

Note that for the pseries machine types, being paravirtual, there is
essentially no guest visible difference between "cores" and "sockets.

I see. When the difference start to make some sense for the pseries guest,
then I think high-count cores may also be preferred and we will already have
the preference of cores at that time because of today's work.

Thanks,
Yanan
.

---
  hw/arm/virt.c  |  1 +
  hw/core/machine.c  | 59 +-
  hw/i386/pc_piix.c  |  1 +
  hw/i386/pc_q35.c   |  1 +
  hw/ppc/spapr.c |  1 +
  hw/s390x/s390-virtio-ccw.c |  1 +
  include/hw/boards.h|  1 +
  qemu-options.hx|  4 ++-
  8 files changed, 55 insertions(+), 14 deletions(-)

diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 01165f7f53..7babea40dc 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -2797,6 +2797,7 @@ static void virt_machine_6_1_options(MachineClass *mc)
  {
  virt_machine_6_2_options(mc);
  compat_props_add(mc->compat_props, hw_compat_6_1, hw_compat_6_1_len);
+mc->smp_prefer_sockets = true;
  }
  DEFINE_VIRT_MACHINE(6, 1)
  
diff --git a/hw/core/machine.c b/hw/core/machine.c

index 63439c4a6d..c074425015 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -744,6 +744,22 @@ void machine_set_cpu_numa_node(MachineState *machine,
  }
  }
  
+/*

+ * smp_parse - Generic function used to parse the given SMP configuration
+ *
+ * The topology parameters must be specified equal to or great than one
+ * or just omitted, explicit configuration like "cpus=0" is not allowed.
+ * The omitted parameters will be calculated based on the provided ones.
+ *
+ * maxcpus will default to the value of cpus if omitted and will be used
+ * to compute the missing sockets/cores/threads. cpus will be calculated
+ * from the computed parametrs if omitted.
+ *
+ * In calculation of omitted arch-netural sockets/cores/threads, we prefer
+ * sockets over cores over threads before 6.2, while prefer cores over
+ * sockets over threads since 6.2 on. The arch-specific dies will directly
+ * default to 1 if omitted.
+ */
  static void smp_parse(MachineState *ms, SMPConfiguration *config, Error 
**errp)
  {
  MachineClass *mc = MACHINE_GET_CLASS(ms);
@@ -772,19 +788,36 @@ static void smp_parse(MachineState *ms, SMPConfiguration 
*config, Error **errp)
  
  maxcpus = maxcpus > 0 ? maxcpus : cpus;
  
-/* compute missing values, prefer sockets over cores over threads */

-if (sockets == 0) {
-cores = cores > 0 ? cores : 1;
-threads = threads > 0 ? threads : 1;
-sockets = maxcpus / (dies * cores * threads);
-sockets = sockets > 0 ? sockets : 1;
-} else if (cores == 0) {
-threads = threads > 0 ? threads : 1;
-cores = maxcpus / (sockets * dies * threads);
-cores = cores > 0 ? cores : 1;
-} else if (threads == 0) {
-threads = maxcpus / (sockets * dies * cores);
-threads = threads > 0 ? threads : 1;
+/* prefer sockets over cores over threads before 6.2 */
+if (mc->smp_prefer_sockets) {
+if (sockets == 0) {
+cores = cores > 0 ? cores : 1;
+threads = threads > 0 ? threads : 1;
+sockets = maxcpus / (dies * cores * threads);
+sockets = sockets > 0 ? sockets : 1;
+} else if (cores == 0) {
+threads = threads > 0 ? threads : 1;
+cores = maxcpus / (sockets * dies * threads);
+cores = cores > 0 ? cores : 1;
+} else if (threads == 0) {
+threads = maxcpus / (sockets * dies * cores);
+threads = threads > 0 ? threads : 1;
+}
+/* prefer cores over sockets over threads since 6.2 */
+} else {
+if (cores == 0) {
+sockets = sockets > 0 ? sockets : 1;
+threads = threads > 0 ? threads : 1;
+cores = maxcpus / (sockets * dies * threads);
+cores = cores > 0 ? cores : 1;
+} else if (sock

Re: [RFC PATCH 10/27] virtio-snd: Add code for the realize function

[Shreyansh Chouhan]

Hi,

On Thu, 22 Jul 2021 at 07:48, Deepa gowda  wrote:

> Hi, Shreyansh,
>
> When is virtio-snd expected to be completed and available in Qemu GitHub?
>
>
Sorry for the recent absence of activity on this patch series. I have the
sound card
working with alsa. The output works just fine. The input needs a little bit
of polishing to do.
To answer your question, it is still going to take some time because I
recently got selected
for an internship/mentorship program and I cannot give as much time to the
patch as I
would like to. It could still take me over a month to complete this.

Hope you understand.

(I've cc'd the mailing list and Gerd so that they too can know about this.)

Thanks,
Shreyansh Chouhan

> Regards Deepa
>
> On Thu, 29 Apr 2021, 17:58 Shreyansh Chouhan, <
> chouhan.shreyansh2...@gmail.com> wrote:
>
>> Signed-off-by: Shreyansh Chouhan 
>> ---
>>  hw/audio/virtio-snd.c | 35 +++
>>  1 file changed, 35 insertions(+)
>>
>> diff --git a/hw/audio/virtio-snd.c b/hw/audio/virtio-snd.c
>> index edaeffd6b7..caad157705 100644
>> --- a/hw/audio/virtio-snd.c
>> +++ b/hw/audio/virtio-snd.c
>> @@ -97,8 +97,43 @@ static uint64_t virtio_snd_get_features(VirtIODevice
>> *vdev, uint64_t features,
>>  return vdev->host_features;
>>  }
>>
>> +/*
>> + * Initializes the VirtIOSound card device. Validates the configuration
>> + * passed by the command line. Initializes the virtqueues. Allocates
>> resources
>> + * for and initializes streams, jacks and chmaps.
>> + *
>> + * @dev: VirtIOSound card device
>> + * @errp: Set if there is an error
>> + */
>>  static void virtio_snd_device_realize(DeviceState *dev, Error **errp)
>>  {
>> +VirtIODevice *vdev = VIRTIO_DEVICE(dev);
>> +VirtIOSound *s = VIRTIO_SOUND(dev);
>> +
>> +virtio_init(vdev, "virtio-snd", VIRTIO_ID_SOUND,
>> sizeof(virtio_snd_config));
>> +
>> +/* set number of jacks and streams */
>> +if (s->snd_conf.jacks > 8) {
>> +error_setg(errp, "Invalid number of jacks: %d",
>> s->snd_conf.jacks);
>> +return;
>> +}
>> +if (s->snd_conf.streams < 1 || s->snd_conf.streams > 10) {
>> +error_setg(errp, "Invalid number of streams: %d",
>> s->snd_conf.streams);
>> +return;
>> +}
>> +
>> +if (s->snd_conf.chmaps > VIRTIO_SND_CHMAP_MAX_SIZE) {
>> +error_setg(errp, "Invalid number of channel maps: %d",
>> +   s->snd_conf.chmaps);
>> +return;
>> +}
>> +
>> +/* set up QEMUSoundCard and audiodev */
>> +AUD_register_card ("virtio_snd_card", &s->card);
>> +
>> +s->streams = g_new0(virtio_snd_pcm_stream *, s->snd_conf.streams);
>> +s->pcm_params = g_new0(virtio_snd_pcm_params *, s->snd_conf.streams);
>> +s->jacks = g_new0(virtio_snd_jack *, s->snd_conf.jacks);
>>  }
>>
>>  static void virtio_snd_device_unrealize(DeviceState *dev)
>> --
>> 2.25.1
>>
>>
>>

Re: [PATCH for-6.2 v2 04/11] machine: Use the computed parameters to calculate omitted cpus

[wangyanan (Y)]

On 2021/7/20 0:42, Andrew Jones wrote:

On Mon, Jul 19, 2021 at 11:20:36AM +0800, Yanan Wang wrote:

Currently we directly calculate the omitted cpus based on the already
provided parameters. This makes some cmdlines like:
   -smp maxcpus=16
   -smp sockets=2,maxcpus=16
   -smp sockets=2,dies=2,maxcpus=16
   -smp sockets=2,cores=4,maxcpus=16
not work. We should probably use the computed paramters to calculate
cpus when maxcpus is provided while cpus is omitted, which will make
above configs start to work.

Note: change in this patch won't affect any existing working cmdlines
but allows more incomplete configs to be valid.

Signed-off-by: Yanan Wang 
---
  hw/core/machine.c | 17 +
  1 file changed, 9 insertions(+), 8 deletions(-)

diff --git a/hw/core/machine.c b/hw/core/machine.c
index c9f15b15a5..668f0a1553 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -767,26 +767,27 @@ static void smp_parse(MachineState *ms, SMPConfiguration 
*config, Error **errp)
  return;
  }
  
-/* compute missing values, prefer sockets over cores over threads */

  maxcpus = maxcpus > 0 ? maxcpus : cpus;
  
-if (cpus == 0) {

-sockets = sockets > 0 ? sockets : 1;
-cores = cores > 0 ? cores : 1;
-threads = threads > 0 ? threads : 1;
-cpus = sockets * dies * cores * threads;
-maxcpus = maxcpus > 0 ? maxcpus : cpus;
-} else if (sockets == 0) {
+/* compute missing values, prefer sockets over cores over threads */
+if (sockets == 0) {
  cores = cores > 0 ? cores : 1;
  threads = threads > 0 ? threads : 1;
  sockets = maxcpus / (dies * cores * threads);
+sockets = sockets > 0 ? sockets : 1;
  } else if (cores == 0) {
  threads = threads > 0 ? threads : 1;
  cores = maxcpus / (sockets * dies * threads);
+cores = cores > 0 ? cores : 1;
  } else if (threads == 0) {
  threads = maxcpus / (sockets * dies * cores);
+threads = threads > 0 ? threads : 1;
  }

I didn't think we wanted this rounding which this patch adds back into
cores and threads and now also sockets.

Firstly, I think we can agree that with or without the rounding, the invalid
configs will still be reported as invalid. So the only difference is in 
the err

message (either report 0 or 1 of a fractional parameter). :)

I added back the rounding because this patch indeed need it, we start
to use the computed parameters to calculate cpus, so we have to ensure
that the computed parameters are at least 1. If both cpus and maxcpus
are omitted (e.g. -smp sockets=16), without the rounding we will get
zeroed cpus and maxcpus, and with the rounding we will get valid result
like "cpus=16,sockets=16,cores=1,threads=1,maxcpus=16".

If a "0" or "1" in the error message doesn't make so much difference as
assumed for the error reporting, I suggest that we probably can keep the
rounding which makes the parser code concise.
  
+/* use the computed parameters to calculate the omitted cpus */

+cpus = cpus > 0 ? cpus : sockets * dies * cores * threads;
+maxcpus = maxcpus > 0 ? maxcpus : cpus;

It doesn't really matter, but I think I'd rather write this like

  maxcpus = maxcpus > 0 ? maxcpus : sockets * dies * cores * threads;
  cpus = cpus > 0 ? cpus : maxcpus;

Yes, there is no functional difference. But I think maybe we'd better keep
some consistence with the "maxcpus = maxcpus > 0 ? maxcpus : cpus"
at top this function and also with the smp doc in qemu-option.hx i.e.
"If omitted the maximum number of CPUs will be set to match the initial
CPU count" ?

Thanks,
Yanan
.

+
  if (sockets * dies * cores * threads < cpus) {
  g_autofree char *dies_msg = g_strdup_printf(
  mc->smp_dies_supported ? " * dies (%u)" : "", dies);
--
2.19.1


Thanks,
drew

.

[PATCH] Makefile: ignore long options

[Alexey Neyman]

When searching for options like -n in MAKEFLAGS, current code may result
in a false positive match when make is invoked with long options like
--no-print-directory. This has been observed with certain versions of
host make (e.g. 3.82) while building the Qemu package in buildroot.

Filter out such long options before searching for one-character options.

Signed-off-by: Alexey Neyman 
---
 Makefile | 8 +---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/Makefile b/Makefile
index 6c36330eef..401c623a65 100644
--- a/Makefile
+++ b/Makefile
@@ -129,9 +129,11 @@ endif
 # 4. Rules to bridge to other makefiles
 
 ifneq ($(NINJA),)
-MAKE.n = $(findstring n,$(firstword $(MAKEFLAGS)))
-MAKE.k = $(findstring k,$(firstword $(MAKEFLAGS)))
-MAKE.q = $(findstring q,$(firstword $(MAKEFLAGS)))
+# Filter out long options to avoid flags like --no-print-directory which
+# may result in false positive match for MAKE.n
+MAKE.n = $(findstring n,$(firstword $(filter-out --%,$(MAKEFLAGS
+MAKE.k = $(findstring k,$(firstword $(filter-out --%,$(MAKEFLAGS
+MAKE.q = $(findstring q,$(firstword $(filter-out --%,$(MAKEFLAGS
 MAKE.nq = $(if $(word 2, $(MAKE.n) $(MAKE.q)),nq)
 NINJAFLAGS = $(if $V,-v) $(if $(MAKE.n), -n) $(if $(MAKE.k), -k0) \
 $(filter-out -j, $(lastword -j1 $(filter -l% -j%, $(MAKEFLAGS \
-- 
2.27.0

Re: [PATCH for-6.2 v2 03/11] machine: Uniformly use maxcpus to calculate the omitted parameters

[wangyanan (Y)]

On 2021/7/20 0:36, Andrew Jones wrote:

On Mon, Jul 19, 2021 at 11:20:35AM +0800, Yanan Wang wrote:

We are currently using maxcpus to calculate the omitted sockets
but using cpus to calculate the omitted cores/threads. This makes
cmdlines like:
   -smp cpus=8,maxcpus=16
   -smp cpus=8,cores=4,maxcpus=16
   -smp cpus=8,threads=2,maxcpus=16
work fine but the ones like:
   -smp cpus=8,sockets=2,maxcpus=16
   -smp cpus=8,sockets=2,cores=4,maxcpus=16
   -smp cpus=8,sockets=2,threads=2,maxcpus=16
break the invalid cpu topology check.

Since we require for the valid config that the sum of "sockets * cores
* dies * threads" should equal to the maxcpus, we should uniformly use
maxcpus to calculate their omitted values.

Also the if-branch of "cpus == 0 || sockets == 0" was splited into two
branches of "cpus == 0" and "sockets == 0" so that we can clearly read
that we are parsing -smp cmdlines with a preference of cpus over sockets
over cores over threads.

Note: change in this patch won't affect any existing working cmdlines
but improves consistency and allow more incomplete configs to be valid.

We also remove rounding of cores and threads when the math doesn't come
out right, which could possible start reporting a bad config as invalid
which worked before. Or were you able to prove that that can't happen with
your testing?

I also had this concern, but I think that can't happen for sure.

Take the if-branch of "cores == 0" as an example,
Before this patch:
We use cpus to calculate the missing cores and round it up to 1 if the
result is zero, and at last set maxcpus to match cpus if it's omitted.
So the parsing result must have met the two conditions:
1) sockets * cores * threads == maxcpus
2) sockets * cores * threads >= cpus

After this patch:
We start to use maxcpus to calculate the missing cores and also remove
the rounding. For the same config mentioned above, it still works and the
parsing result will also not change, because we will never get a fractional
value of cores (maxcpus is multiple of (sockets * threads) ).

Like the valid config "-smp 8,sockets=16,maxcpus=16", we will get
"cpus=8,sockets=16,cores=1,threads=1,maxcpus=16" before, and
still get the same result after.

Please correct me if I missed something, thanks.

Signed-off-by: Yanan Wang 
---
  hw/core/machine.c | 30 +++---
  1 file changed, 15 insertions(+), 15 deletions(-)

diff --git a/hw/core/machine.c b/hw/core/machine.c
index ed6712e964..c9f15b15a5 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -768,24 +768,26 @@ static void smp_parse(MachineState *ms, SMPConfiguration 
*config, Error **errp)
  }
  
  /* compute missing values, prefer sockets over cores over threads */

-if (cpus == 0 || sockets == 0) {
+maxcpus = maxcpus > 0 ? maxcpus : cpus;
+
+if (cpus == 0) {
+sockets = sockets > 0 ? sockets : 1;
  cores = cores > 0 ? cores : 1;
  threads = threads > 0 ? threads : 1;
-if (cpus == 0) {
-sockets = sockets > 0 ? sockets : 1;
-cpus = sockets * dies * cores * threads;
-} else {
-maxcpus = maxcpus > 0 ? maxcpus : cpus;
-sockets = maxcpus / (dies * cores * threads);
-}
+cpus = sockets * dies * cores * threads;
+maxcpus = maxcpus > 0 ? maxcpus : cpus;
+} else if (sockets == 0) {
+cores = cores > 0 ? cores : 1;
+threads = threads > 0 ? threads : 1;
+sockets = maxcpus / (dies * cores * threads);
  } else if (cores == 0) {
  threads = threads > 0 ? threads : 1;
-cores = cpus / (sockets * dies * threads);
-cores = cores > 0 ? cores : 1;
+cores = maxcpus / (sockets * dies * threads);
  } else if (threads == 0) {
-threads = cpus / (sockets * dies * cores);
-threads = threads > 0 ? threads : 1;
-} else if (sockets * dies * cores * threads < cpus) {
+threads = maxcpus / (sockets * dies * cores);
+}
+
+if (sockets * dies * cores * threads < cpus) {
  g_autofree char *dies_msg = g_strdup_printf(
  mc->smp_dies_supported ? " * dies (%u)" : "", dies);
  error_setg(errp, "cpu topology: "
@@ -795,8 +797,6 @@ static void smp_parse(MachineState *ms, SMPConfiguration 
*config, Error **errp)
  return;
  }
  
-maxcpus = maxcpus > 0 ? maxcpus : cpus;

-
  if (maxcpus < cpus) {
  error_setg(errp, "maxcpus must be equal to or greater than smp");
  return;
--
2.19.1


Reviewed-by: Andrew Jones 

Thanks,
Yanan
.

Re: [PATCH v4 3/3] target/ppc: moved store_40x_sler to helper_regs.c

[David Gibson]

On Wed, Jul 21, 2021 at 10:21:44AM -0300, Lucas Mateus Castro (alqotel) wrote:
> moved store_40x_sler from mmu_common.c to helper_regs.c as it was
> more appropriate

This message needs a rationale to explain why this is more appropriate.

> 
> Signed-off-by: Lucas Mateus Castro (alqotel) 
> ---
>  target/ppc/helper_regs.c | 12 
>  target/ppc/mmu_common.c  | 10 --
>  2 files changed, 12 insertions(+), 10 deletions(-)
> 
> diff --git a/target/ppc/helper_regs.c b/target/ppc/helper_regs.c
> index 3723872aa6..405450d863 100644
> --- a/target/ppc/helper_regs.c
> +++ b/target/ppc/helper_regs.c
> @@ -258,6 +258,18 @@ int hreg_store_msr(CPUPPCState *env, target_ulong value, 
> int alter_hv)
>  return excp;
>  }
>  
> +#ifdef CONFIG_SOFTMMU
> +void store_40x_sler(CPUPPCState *env, uint32_t val)
> +{
> +/* XXX: TO BE FIXED */
> +if (val != 0x) {
> +cpu_abort(env_cpu(env),
> +  "Little-endian regions are not supported by now\n");
> +}
> +env->spr[SPR_405_SLER] = val;
> +}
> +#endif /* CONFIG_SOFTMMU */
> +
>  #ifndef CONFIG_USER_ONLY
>  void check_tlb_flush(CPUPPCState *env, bool global)
>  {
> diff --git a/target/ppc/mmu_common.c b/target/ppc/mmu_common.c
> index a8bd418f18..e206be9d05 100644
> --- a/target/ppc/mmu_common.c
> +++ b/target/ppc/mmu_common.c
> @@ -622,16 +622,6 @@ static int mmu40x_get_physical_address(CPUPPCState *env, 
> mmu_ctx_t *ctx,
>  return ret;
>  }
>  
> -void store_40x_sler(CPUPPCState *env, uint32_t val)
> -{
> -/* XXX: TO BE FIXED */
> -if (val != 0x) {
> -cpu_abort(env_cpu(env),
> -  "Little-endian regions are not supported by now\n");
> -}
> -env->spr[SPR_405_SLER] = val;
> -}
> -
>  static int mmubooke_check_tlb(CPUPPCState *env, ppcemb_tlb_t *tlb,
>hwaddr *raddr, int *prot, target_ulong address,
>MMUAccessType access_type, int i)

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [PATCH v4 1/3] target/ppc: divided mmu_helper.c in 2 files

[David Gibson]

On Wed, Jul 21, 2021 at 10:21:42AM -0300, Lucas Mateus Castro (alqotel) wrote:
> Divided mmu_helper.c in 2 files, functions inside #ifdef CONFIG_SOFTMMU
> stayed in mmu_helper.c, other functions moved to mmu_common.c. Updated
> meson.build to compile mmu_common.c and only compile mmu_helper.c when
> CONFIG_TCG is set.
> Moved function declarations, #define and structs used by both files to
> internal.h except for functions that use structures defined in cpu.h,
> those were moved to cpu.h.
> 
> Signed-off-by: Lucas Mateus Castro (alqotel)
> 

LGTM, with the exception of one detail:

[snip]

> diff --git a/target/ppc/mmu_common.c b/target/ppc/mmu_common.c
> new file mode 100644
> index 00..776aedce29
> --- /dev/null
> +++ b/target/ppc/mmu_common.c
> @@ -0,0 +1,1604 @@
> +/*
> + *  PowerPC CPU initialization for qemu.
> + *
> + *  Copyright (C) 2021 Instituto de Pesquisas Eldorado (eldorado.org.br)

I know the copyright headers are pretty inaccurate anyway, but putting
an entirely new copyright on a file created more or less completely
from code motion seems a bit much.  I think you need to include the
old copyright banner from mmu_helper.c, though you can add your own as
well if you like.

> + * This library is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * This library is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with this library; if not, see 
> .
> + */
> +
> +#include "qemu/osdep.h"
> +#include "qemu/units.h"
> +#include "cpu.h"
> +#include "sysemu/kvm.h"
> +#include "kvm_ppc.h"
> +#include "mmu-hash64.h"
> +#include "mmu-hash32.h"
> +#include "exec/exec-all.h"
> +#include "exec/log.h"
> +#include "helper_regs.h"
> +#include "qemu/error-report.h"
> +#include "qemu/main-loop.h"
> +#include "qemu/qemu-print.h"
> +#include "internal.h"
> +#include "mmu-book3s-v3.h"
> +#include "mmu-radix64.h"
> +
> +/* #define DEBUG_MMU */
> +/* #define DEBUG_BATS */
> +/* #define DEBUG_SOFTWARE_TLB */
> +/* #define DUMP_PAGE_TABLES */
> +/* #define FLUSH_ALL_TLBS */
> +
> +#ifdef DEBUG_MMU
> +#  define LOG_MMU_STATE(cpu) log_cpu_state_mask(CPU_LOG_MMU, (cpu), 0)
> +#else
> +#  define LOG_MMU_STATE(cpu) do { } while (0)
> +#endif
> +
> +#ifdef DEBUG_SOFTWARE_TLB
> +#  define LOG_SWTLB(...) qemu_log_mask(CPU_LOG_MMU, __VA_ARGS__)
> +#else
> +#  define LOG_SWTLB(...) do { } while (0)
> +#endif
> +
> +#ifdef DEBUG_BATS
> +#  define LOG_BATS(...) qemu_log_mask(CPU_LOG_MMU, __VA_ARGS__)
> +#else
> +#  define LOG_BATS(...) do { } while (0)
> +#endif
> +
> +/*/
> +/* PowerPC MMU emulation */
> +
> +static int pp_check(int key, int pp, int nx)
> +{
> +int access;
> +
> +/* Compute access rights */
> +access = 0;
> +if (key == 0) {
> +switch (pp) {
> +case 0x0:
> +case 0x1:
> +case 0x2:
> +access |= PAGE_WRITE;
> +/* fall through */
> +case 0x3:
> +access |= PAGE_READ;
> +break;
> +}
> +} else {
> +switch (pp) {
> +case 0x0:
> +access = 0;
> +break;
> +case 0x1:
> +case 0x3:
> +access = PAGE_READ;
> +break;
> +case 0x2:
> +access = PAGE_READ | PAGE_WRITE;
> +break;
> +}
> +}
> +if (nx == 0) {
> +access |= PAGE_EXEC;
> +}
> +
> +return access;
> +}
> +
> +static int check_prot(int prot, MMUAccessType access_type)
> +{
> +return prot & prot_for_access_type(access_type) ? 0 : -2;
> +}
> +
> +int ppc6xx_tlb_getnum(CPUPPCState *env, target_ulong eaddr,
> +int way, int is_code)
> +{
> +int nr;
> +
> +/* Select TLB num in a way from address */
> +nr = (eaddr >> TARGET_PAGE_BITS) & (env->tlb_per_way - 1);
> +/* Select TLB way */
> +nr += env->tlb_per_way * way;
> +/* 6xx have separate TLBs for instructions and data */
> +if (is_code && env->id_tlbs == 1) {
> +nr += env->nb_tlb;
> +}
> +
> +return nr;
> +}
> +
> +static int ppc6xx_tlb_pte_check(mmu_ctx_t *ctx, target_ulong pte0,
> +target_ulong pte1, int h,
> +MMUAccessType access_type)
> +{
> +target_ulong ptem, mmask;
> +int access, ret, pteh, ptev, pp;
> +
> +ret = -1;
> +/* Check validity and table match */
> +ptev = pte_is_valid(pte0);
> +pteh = (

Re: [PATCH v4 2/3] target/ppc: moved ppc_store_sdr1 to mmu_common.c

[David Gibson]

On Wed, Jul 21, 2021 at 10:21:43AM -0300, Lucas Mateus Castro (alqotel) wrote:
> ppc_store_sdr1 was at first in mmu_helper.c and was moved as part
> the patches to enable the disable-tcg option, now it's being moved
> back to a file that will be compiled with that option
> 
> Signed-off-by: Lucas Mateus Castro (alqotel)
> 

Reviewed-by: David Gibson 

> ---
>  target/ppc/cpu.c| 28 
>  target/ppc/mmu_common.c | 26 ++
>  2 files changed, 26 insertions(+), 28 deletions(-)
> 
> diff --git a/target/ppc/cpu.c b/target/ppc/cpu.c
> index a29299882a..7ad9bd6044 100644
> --- a/target/ppc/cpu.c
> +++ b/target/ppc/cpu.c
> @@ -67,34 +67,6 @@ uint32_t ppc_get_vscr(CPUPPCState *env)
>  return env->vscr | (sat << VSCR_SAT);
>  }
>  
> -#ifdef CONFIG_SOFTMMU
> -void ppc_store_sdr1(CPUPPCState *env, target_ulong value)
> -{
> -PowerPCCPU *cpu = env_archcpu(env);
> -qemu_log_mask(CPU_LOG_MMU, "%s: " TARGET_FMT_lx "\n", __func__, value);
> -assert(!cpu->env.has_hv_mode || !cpu->vhyp);
> -#if defined(TARGET_PPC64)
> -if (mmu_is_64bit(env->mmu_model)) {
> -target_ulong sdr_mask = SDR_64_HTABORG | SDR_64_HTABSIZE;
> -target_ulong htabsize = value & SDR_64_HTABSIZE;
> -
> -if (value & ~sdr_mask) {
> -qemu_log_mask(LOG_GUEST_ERROR, "Invalid bits 0x"TARGET_FMT_lx
> - " set in SDR1", value & ~sdr_mask);
> -value &= sdr_mask;
> -}
> -if (htabsize > 28) {
> -qemu_log_mask(LOG_GUEST_ERROR, "Invalid HTABSIZE 0x" 
> TARGET_FMT_lx
> - " stored in SDR1", htabsize);
> -return;
> -}
> -}
> -#endif /* defined(TARGET_PPC64) */
> -/* FIXME: Should check for valid HTABMASK values in 32-bit case */
> -env->spr[SPR_SDR1] = value;
> -}
> -#endif /* CONFIG_SOFTMMU */
> -
>  /* GDBstub can read and write MSR... */
>  void ppc_store_msr(CPUPPCState *env, target_ulong value)
>  {
> diff --git a/target/ppc/mmu_common.c b/target/ppc/mmu_common.c
> index 776aedce29..a8bd418f18 100644
> --- a/target/ppc/mmu_common.c
> +++ b/target/ppc/mmu_common.c
> @@ -58,6 +58,32 @@
>  #  define LOG_BATS(...) do { } while (0)
>  #endif
>  
> +void ppc_store_sdr1(CPUPPCState *env, target_ulong value)
> +{
> +PowerPCCPU *cpu = env_archcpu(env);
> +qemu_log_mask(CPU_LOG_MMU, "%s: " TARGET_FMT_lx "\n", __func__, value);
> +assert(!cpu->env.has_hv_mode || !cpu->vhyp);
> +#if defined(TARGET_PPC64)
> +if (mmu_is_64bit(env->mmu_model)) {
> +target_ulong sdr_mask = SDR_64_HTABORG | SDR_64_HTABSIZE;
> +target_ulong htabsize = value & SDR_64_HTABSIZE;
> +
> +if (value & ~sdr_mask) {
> +qemu_log_mask(LOG_GUEST_ERROR, "Invalid bits 0x"TARGET_FMT_lx
> + " set in SDR1", value & ~sdr_mask);
> +value &= sdr_mask;
> +}
> +if (htabsize > 28) {
> +qemu_log_mask(LOG_GUEST_ERROR, "Invalid HTABSIZE 0x" 
> TARGET_FMT_lx
> + " stored in SDR1", htabsize);
> +return;
> +}
> +}
> +#endif /* defined(TARGET_PPC64) */
> +/* FIXME: Should check for valid HTABMASK values in 32-bit case */
> +env->spr[SPR_SDR1] = value;
> +}
> +
>  
> /*/
>  /* PowerPC MMU emulation */
>  

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [RFC PATCH v3 0/5] pSeries FORM2 affinity support

[David Gibson]

On Tue, Jul 13, 2021 at 04:40:40PM -0300, Daniel Henrique Barboza wrote:
> Hi,
> 
> This new version drops all the NVDIMM related changes from the
> previous iteraction after the reviews done in the kernel mailing
> list [1]. FORM2 will not use ibm,associativity-reference-points to
> determine a second NUMA mode for different operation modes of
> PAPR-SCM.
> 
> [1]
> https://lore.kernel.org/linuxppc-dev/20210628151117.545935-1-aneesh.ku...@linux.ibm.com/

Sorry, I'm going to punt on reviewing this again, until the kernel
side patches are sorted out.

> 
> 
> changes from v2:
> - patch 3:
>   * reworded commit msg and comments to reflect the current state of
> the specification
> - patches 5 and 6: removed
> - v2 link: 
> https://lists.gnu.org/archive/html/qemu-devel/2021-06/msg04056.html 
> 
> 
> changes from v1:
> - patches 1 and 2: switched places
> - patch 3: folded into patch 2
> - patch 2:
> * only make CAS related changes when using the newest
> machine version
> - patch 3 (former 4):
> * only advertise FORM2 support for the newest machine version
> - patches 5 and 6 (former 6 and 7):
> * detect if 'device-node' was set in the command line, and if not,
> use the 'node' value when writing in the device tree
> 
> v1 link: https://lists.gnu.org/archive/html/qemu-devel/2021-06/msg03617.html
> 
> 
> Daniel Henrique Barboza (5):
>   spapr_numa.c: split FORM1 code into helpers
>   spapr: move NUMA data init to post-CAS
>   spapr_numa.c: base FORM2 NUMA affinity support
>   spapr: simplify spapr_numa_associativity_init params
>   spapr: move memory/cpu less check to spapr_numa_FORM1_affinity_init()
> 
>  hw/ppc/spapr.c  |  60 --
>  hw/ppc/spapr_hcall.c|   4 +
>  hw/ppc/spapr_numa.c | 224 +---
>  include/hw/ppc/spapr.h  |   1 +
>  include/hw/ppc/spapr_numa.h |   3 +-
>  include/hw/ppc/spapr_ovec.h |   1 +
>  6 files changed, 236 insertions(+), 57 deletions(-)
> 

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [PATCH for-6.2 v2 00/11] machine: smp parsing fixes and improvement

[wangyanan (Y)]

On 2021/7/21 21:52, Pankaj Gupta wrote:

On Mon, Jul 19 2021, Yanan Wang  wrote:


Hi,

This is v2 of the series [1] that I have posted to introduce some smp parsing
fixes and improvement, much more work has been processed compared to RFC v1.

[1] https://lists.gnu.org/archive/html/qemu-devel/2021-07/msg00259.html

The purpose of this series is to improve/fix the parsing logic. Explicitly
specifying a CPU topology parameter as zero is not allowed any more, and
maxcpus is now uniformly used to calculate the omitted parameters. It's also
suggested that we should start to prefer cores over sockets over threads on
the newer machine types, which will make the computed virtual topology more
reflective of the real hardware.

In order to reduce code duplication and ease the code maintenance, smp_parse
in now converted into a parser generic enough for all arches, so that the PC
specific one can be removed. It's also convenient to introduce more topology
members (e.g. cluster) to the generic parser in the future.

Cc:ing Pierre, as he also had been looking at the smp parsing code (for
s390x) recently.

Also, please keep me on cc: for patches that touch s390x.

Sure, I will. Sorry about the missing. :)

Thanks,
Yanan
.

Finally, a QEMU unit test for the parsing of given SMP configuration is added.
Since all the parsing logic is in generic function smp_parse(), this test
passes diffenent SMP configurations to the function and compare the parsing
result with what is expected. In the test, all possible collections of the
topology parameters and the corressponding expected results are listed,
including the valid and invalid ones. The preference of sockets over cores
and the preference of cores over sockets, and the support of multi-dies are
also taken into consideration.

---

Changelogs:

v1->v2:
- disallow "anything=0" in the smp configuration (Andrew)
- make function smp_parse() a generic helper for all arches
- improve the error reporting in the parser
- start to prefer cores over sockets since 6.2 (Daniel)
- add a unit test for the smp parsing (Daniel)

---

Yanan Wang (11):
machine: Disallow specifying topology parameters as zero
machine: Make smp_parse generic enough for all arches
machine: Uniformly use maxcpus to calculate the omitted parameters
machine: Use the computed parameters to calculate omitted cpus
machine: Improve the error reporting of smp parsing
hw: Add compat machines for 6.2
machine: Prefer cores over sockets in smp parsing since 6.2
machine: Use ms instead of global current_machine in sanity-check
machine: Tweak the order of topology members in struct CpuTopology
machine: Split out the smp parsing code
tests/unit: Add a unit test for smp parsing

   MAINTAINERS |2 +
   hw/arm/virt.c   |   10 +-
   hw/core/machine-smp.c   |  124 
   hw/core/machine.c   |   68 +--
   hw/core/meson.build |1 +
   hw/i386/pc.c|   66 +--
   hw/i386/pc_piix.c   |   15 +-
   hw/i386/pc_q35.c|   14 +-
   hw/ppc/spapr.c  |   16 +-
   hw/s390x/s390-virtio-ccw.c  |   15 +-
   include/hw/boards.h |   13 +-
   include/hw/i386/pc.h|3 +
   qapi/machine.json   |6 +-
   qemu-options.hx |4 +-
   tests/unit/meson.build  |1 +
   tests/unit/test-smp-parse.c | 1117 +++
   16 files changed, 1338 insertions(+), 137 deletions(-)
   create mode 100644 hw/core/machine-smp.c
   create mode 100644 tests/unit/test-smp-parse.c


Please add me in CC as I reviewed one of the patch and was in middle
of reviewing few other patches
but missed the latest version.

Ok, I will, and thanks for the reviewing. v2 i.e. this version is now 
the latest. :)


Thanks,
Yanan

[PATCH for-6.1 0/1] machine: Disallow specifying topology parameters as zero

[Yanan Wang]

In the SMP configuration, we should either specify a topology
parameter with a reasonable value (equal to or greater than 1)
or just leave it omitted and QEMU will calculate its value.
Configurations which explicitly specify the topology parameters
as zero like "sockets=0" are meaningless, so disallow them.

However; the commit 1e63fe685804d
(machine: pass QAPI struct to mc->smp_parse) has documented that
'0' has the same semantics as omitting a parameter in the qapi
comment for SMPConfiguration. So this patch fixes the doc and
also adds the corresponding sanity check in the smp parsers.

This patch originly comes form [1], and it was suggested that
this patch fixing the doc should be sent for 6.1 to avoid a
deprecation process in the future.

[1] https://lore.kernel.org/qemu-devel/ypwsthpiza3mf...@redhat.com/

Yanan Wang (1):
  machine: Disallow specifying topology parameters as zero

 hw/core/machine.c | 30 ++
 hw/i386/pc.c  | 33 -
 qapi/machine.json |  6 +++---
 3 files changed; 49 insertions(+); 20 deletions(-)

--
2.19.1

[PATCH for-6.1 1/1] machine: Disallow specifying topology parameters as zero

[Yanan Wang]

In the SMP configuration, we should either specify a topology
parameter with a reasonable value (equal to or greater than 1)
or just leave it omitted and QEMU will calculate its value.
Configurations which explicitly specify the topology parameters
as zero like "sockets=0" are meaningless, so disallow them.

However, the commit 1e63fe685804d
(machine: pass QAPI struct to mc->smp_parse) has documented that
'0' has the same semantics as omitting a parameter in the qapi
comment for SMPConfiguration. So this patch fixes the doc and
also adds the corresponding sanity check in the smp parsers.

Reviewed-by: Andrew Jones 
Suggested-by: Andrew Jones 
Signed-off-by: Yanan Wang 
---
 hw/core/machine.c | 30 ++
 hw/i386/pc.c  | 33 -
 qapi/machine.json |  6 +++---
 3 files changed, 49 insertions(+), 20 deletions(-)

diff --git a/hw/core/machine.c b/hw/core/machine.c
index 775add0795..ada300542b 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -745,11 +745,24 @@ static void smp_parse(MachineState *ms, SMPConfiguration 
*config, Error **errp)
 {
 unsigned cpus= config->has_cpus ? config->cpus : 0;
 unsigned sockets = config->has_sockets ? config->sockets : 0;
+unsigned dies= config->has_dies ? config->dies : 0;
 unsigned cores   = config->has_cores ? config->cores : 0;
 unsigned threads = config->has_threads ? config->threads : 0;
+unsigned maxcpus = config->has_maxcpus ? config->maxcpus : 0;
+
+if ((config->has_cpus && config->cpus == 0) ||
+(config->has_sockets && config->sockets == 0) ||
+(config->has_dies && config->dies == 0) ||
+(config->has_cores && config->cores == 0) ||
+(config->has_threads && config->threads == 0) ||
+(config->has_maxcpus && config->maxcpus == 0)) {
+error_setg(errp, "parameters must be equal to or greater than one if 
provided");
+return;
+}
 
-if (config->has_dies && config->dies != 0 && config->dies != 1) {
+if (dies > 1) {
 error_setg(errp, "dies not supported by this machine's CPU topology");
+return;
 }
 
 /* compute missing values, prefer sockets over cores over threads */
@@ -760,8 +773,8 @@ static void smp_parse(MachineState *ms, SMPConfiguration 
*config, Error **errp)
 sockets = sockets > 0 ? sockets : 1;
 cpus = cores * threads * sockets;
 } else {
-ms->smp.max_cpus = config->has_maxcpus ? config->maxcpus : cpus;
-sockets = ms->smp.max_cpus / (cores * threads);
+maxcpus = maxcpus > 0 ? maxcpus : cpus;
+sockets = maxcpus / (cores * threads);
 }
 } else if (cores == 0) {
 threads = threads > 0 ? threads : 1;
@@ -778,26 +791,27 @@ static void smp_parse(MachineState *ms, SMPConfiguration 
*config, Error **errp)
 return;
 }
 
-ms->smp.max_cpus = config->has_maxcpus ? config->maxcpus : cpus;
+maxcpus = maxcpus > 0 ? maxcpus : cpus;
 
-if (ms->smp.max_cpus < cpus) {
+if (maxcpus < cpus) {
 error_setg(errp, "maxcpus must be equal to or greater than smp");
 return;
 }
 
-if (sockets * cores * threads != ms->smp.max_cpus) {
+if (sockets * cores * threads != maxcpus) {
 error_setg(errp, "Invalid CPU topology: "
"sockets (%u) * cores (%u) * threads (%u) "
"!= maxcpus (%u)",
sockets, cores, threads,
-   ms->smp.max_cpus);
+   maxcpus);
 return;
 }
 
 ms->smp.cpus = cpus;
+ms->smp.sockets = sockets;
 ms->smp.cores = cores;
 ms->smp.threads = threads;
-ms->smp.sockets = sockets;
+ms->smp.max_cpus = maxcpus;
 }
 
 static void machine_get_smp(Object *obj, Visitor *v, const char *name,
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index c2b9d62a35..db93841097 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -716,9 +716,23 @@ static void pc_smp_parse(MachineState *ms, 
SMPConfiguration *config, Error **err
 {
 unsigned cpus= config->has_cpus ? config->cpus : 0;
 unsigned sockets = config->has_sockets ? config->sockets : 0;
-unsigned dies= config->has_dies ? config->dies : 1;
+unsigned dies= config->has_dies ? config->dies : 0;
 unsigned cores   = config->has_cores ? config->cores : 0;
 unsigned threads = config->has_threads ? config->threads : 0;
+unsigned maxcpus = config->has_maxcpus ? config->maxcpus : 0;
+
+if ((config->has_cpus && config->cpus == 0) ||
+(config->has_sockets && config->sockets == 0) ||
+(config->has_dies && config->dies == 0) ||
+(config->has_cores && config->cores == 0) ||
+(config->has_threads && config->threads == 0) ||
+(config->has_maxcpus && config->maxcpus == 0)) {
+error_setg(errp, "parameters must be equal to or greater than one if 
provided");
+return;
+}
+
+/* directly default dies to 

Re: [PATCH for-6.2 33/34] target/arm: Implement MVE scatter-gather immediate forms

[Richard Henderson]

On 7/13/21 3:37 AM, Peter Maydell wrote:

Implement the MVE VLDR/VSTR insns which do scatter-gather using base
addresses from Qm plus or minus an immediate offset (possibly with
writeback). Note that writeback is not predicated but it does have
to honour ECI state, so we have to add an eci_mask check to the
VSTR_SG macros (the VLDR_SG macros already needed this to be able
to distinguish "skip beat" from "set predicated element to 0").

Signed-off-by: Peter Maydell
---
  target/arm/helper-mve.h|  5 +++
  target/arm/mve.decode  | 10 +
  target/arm/mve_helper.c| 91 --
  target/arm/translate-mve.c | 66 +++
  4 files changed, 140 insertions(+), 32 deletions(-)


Reviewed-by: Richard Henderson 

r~

Re: [PATCH v1 1/1] vfio: Make migration support non experimental by default.

[Liang Yan]

On 7/14/21 6:19 AM, Kirti Wankhede wrote:
>
>
> On 7/10/2021 1:14 PM, Claudio Fontana wrote:
>> On 3/8/21 5:09 PM, Tarun Gupta wrote:
>>> VFIO migration support in QEMU is experimental as of now, which was
>>> done to
>>> provide soak time and resolve concerns regarding bit-stream.
>>> But, with the patches discussed in
>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mail-archive.com%2Fqemu-devel%40nongnu.org%2Fmsg784931.html&data=04%7C01%7Ckwankhede%40nvidia.com%7C98194e8a856f4e6b611c08d943769ab5%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637614998961553398%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=A2EY9LEqGE0BSrT25h2WtWonb5oi0O%2B6%2BQmvhVf8Wd4%3D&reserved=0
>>> , we have
>>> corrected ordering of saving PCI config space and bit-stream.
>>>
>>> So, this patch proposes to make vfio migration support in QEMU to be
>>> enabled
>>> by default. Tested by successfully migrating mdev device.
>>>
>>> Signed-off-by: Tarun Gupta 
>>> Signed-off-by: Kirti Wankhede 
>>> ---
>>>   hw/vfio/pci.c | 2 +-
>>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
>>> index f74be78209..15e26f460b 100644
>>> --- a/hw/vfio/pci.c
>>> +++ b/hw/vfio/pci.c
>>> @@ -3199,7 +3199,7 @@ static Property vfio_pci_dev_properties[] = {
>>>   DEFINE_PROP_BIT("x-igd-opregion", VFIOPCIDevice, features,
>>>   VFIO_FEATURE_ENABLE_IGD_OPREGION_BIT, false),
>>>   DEFINE_PROP_BOOL("", VFIOPCIDevice,
>>> - vbasedev.enable_migration, false),
>>> + vbasedev.enable_migration, true),
>>>   DEFINE_PROP_BOOL("x-no-mmap", VFIOPCIDevice, vbasedev.no_mmap,
>>> false),
>>>   DEFINE_PROP_BOOL("x-balloon-allowed", VFIOPCIDevice,
>>>    vbasedev.ram_block_discard_allowed, false),
>>>
>>
>> Hello,
>>
>> has plain snapshot been tested?
>
> Yes.
>
>> If I issue the HMP command "savevm", and then "loadvm", will things
>> work fine?
>
> Yes
>

Hello Kirti,

I enabled x-enable-migration and did some hack on failover_pair_id,
finally made  "virsh save/restore" and "savevm/loadvm"work through.
However, it seems vGPU did not get involved in the real migration
process, the qemu trace file confirmed it, there is no vfio section for
savevm_section_start at all.

I am using kernel 5.8 and latest qemu, vGPU 12.2 with one V100. I am
wondering if there is a version compatible requirement or need extra
setup. Could you share your test setup here? Thanks in advance.

Regards,

Liang



> Thanks,
> Kirti
>

Re: [PATCH for-6.2 32/34] target/arm: Implement MVE scatter-gather insns

[Richard Henderson]

On 7/13/21 3:37 AM, Peter Maydell wrote:

+static bool do_ldst_sg(DisasContext *s, arg_vldst_sg *a, MVEGenLdStSGFn fn)
+{
+TCGv_i32 addr;
+TCGv_ptr qd, qm;
+
+if (!dc_isar_feature(aa32_mve, s) ||
+!mve_check_qreg_bank(s, a->qd | a->qm) ||
+!fn || a->rn == 15) {
+/* Rn case is UNPREDICTABLE */
+return false;
+}


No Qd != Qm check for loads?  Given that we know in advance that it simply won't work for 
VLDRD, it would be nice to diagnose the error.



+static bool trans_VLDR_S_sg(DisasContext *s, arg_vldst_sg *a)
+{
+static MVEGenLdStSGFn * const fns[2][4][4] = { {
+{ NULL, F(vldrb_sg_sh), F(vldrb_sg_sw), NULL },
+{ NULL, NULL,   F(vldrh_sg_sw), NULL },
+{ NULL, NULL,   NULL,   NULL },
+{ NULL, NULL,   NULL,   NULL }
+}, {
+{ NULL, NULL,  NULL,  NULL },
+{ NULL, NULL,  F(vldrh_sg_os_sw), NULL },
+{ NULL, NULL,  NULL,  NULL },
+{ NULL, NULL,  NULL,  NULL }
+}
+};


A little bit unfortunate with table density here, but whatever.

Reviewed-by: Richard Henderson 


r~

Re: [RFC PATCH 2/6] i386/sev: extend sev-guest property to include SEV-SNP

[Michael Roth via]

On Wed, Jul 21, 2021 at 03:08:37PM +0200, Markus Armbruster wrote:
> Daniel P. Berrangé  writes:
> 
> > On Tue, Jul 20, 2021 at 02:42:12PM -0500, Michael Roth wrote:
> >> On Tue, Jul 13, 2021 at 03:46:19PM +0200, Markus Armbruster wrote:
> 
> [...]
> 
> >> > I recommend to do exactly what we've done before for complex
> >> > configuration: define it in the QAPI schema, so we can use both dotted
> >> > keys and JSON on the command line, and can have QMP, too.  Examples:
> >> > -blockdev, -display, -compat.
> >> > 
> >> > Questions?
> >> 
> >> Hi Markus, Daniel,
> >> 
> >> I'm dealing with similar considerations with some SNP config options
> >> relating to CPUID enforcement, so I've started looking into this as
> >> well, but am still a little confused on the best way to proceed.
> >> 
> >> I see that -blockdev supports both JSON command-line arguments (via
> >> qobject_input_visitor_new) and dotted keys
> >> (via qobject_input_vistior_new_keyval).
> 
> Yes.  Convenience function qobject_input_visitor_new_str() provides
> this.
> 
> >> We could introduce a new config group do the same, maybe something specific
> >> to ConfidentialGuestSupport objects, e.g.:
> >> 
> >>   -confidential-guest-support sev-guest,id=sev0,key_a.subkey_b=...
> >
> > We don't wnt to be adding new CLI options anymore. -object with json
> > syntx should ultimately be able to cover everything we'll ever need
> > to do.
> 
> Depends.  When you want a CLI option to create a single QOM object, then
> -object is commonly the way to go.

So if I've read things correctly the fact that this is a question of how to
define properties of a single QOM object lends itself to using -object rather
than attempting a new -blockdev-like option group, and as such if we
want to allow structured parameters we should use pure JSON instead of
attempting to layer anything on top of the current keyval parser.

> 
> >> and use the same mechanisms to parse the options, but this seems to
> >> either involve adding a layer of option translations between command-line
> >> and the underlying object properties, or, if we keep the 1:1 mapping
> >> between QAPI-defined keys and object properties, it basically becomes a
> >> way to pass a different Visitor implementation into object_property_set(),
> >> in this case one created by object_input_visitor_new_keyval() instead of
> >> opts_visitor_new().
> 
> qobject_input_visitor_new_str() provides 1:1, i.e. common abstract
> syntax, and concrete syntax either JSON or dotted keys.  Note that the
> latter is slightly less expressive (can't do empty arrays and objects,
> may fall apart for type 'any').  If you run into these limitations, use
> JSON.  Machines should always use JSON.
> 
> qobject_input_visitor_new_str() works by wrapping the "right" visitor
> around the option argument.  Another way to provide a human-friendly
> interface in addition to a machine-friendly one is to translate from
> human to the machine interface.  HMP works that way: HMP commands wrap
> around QMP commands.  The QMP commands are generated from the QAPI
> schema.  The HMP commands are written by hand, which is maximally
> flexible, but also laborious.
> 
> >> In either case, genericizing it beyond CGS/SEV would basically be
> >> introducing:
> >> 
> >>   -object2 sev-guest,id=sev0,key_a.subkey_b=...
> 
> That's because existing -object doesn't use keyval_parse() + the keyval
> QObjct input visitor, it uses QemuOpts and the options visitor, for
> backward compatibility with all their (barely understood) features and
> warts.
> 
> Unfortunate, because even new user-creatable objects can't escape
> QemuOpts.
> 
> QemuOpts needs to go.  But replacing it is difficult and scary in
> places.  -object is such a place.
> 
> >> Which one seems preferable? Or is the answer neither?
> >
> > Yep, neither is the answer.
> 
> Welcome to the QemuOpts swamp, here's your waders and a leaky bucket.

*backs slowly away from swamp* :)

So back to the question of whether we need structured parameters. The
main driver for this seems to be that the options are currently defined
via a config file, which was originally introduced to cope with:

a) lots of parameters (8)

   - not really that significant compared to some other objects/options.

b) large page-size parameters

   - mainly applies to 'id_auth', which could be broken out as individual
 files/blobs and passed in via normal file path string arguments
   - already how we handle dh-cert-file and session-file

c) separating SNP-specific parameters from the base sev-guest object
   properties

   - could possibly be done with a new 'sev-snp-guest' object, which
 would also help disambiguate stuff like the 32-bit sev/sev-es
 'policy' arguments from the 64-bit version in SNP, and can still
 re-use common properties like 'cbitpos' via some base object
   - maybe some other benefits, need to look into it more.

If they aren't any objections I'll take a stab at this early next week.
Will be on PTO unti

Re: [PATCH] docs: convert writing-qmp-commands.txt to writing-qmp-commands.rst

[Connor Kuehl]

On 7/21/21 11:50 AM, John Snow wrote:

This does about the bare minimum, converting section headers to ReST
ones and adding an indent for code blocks.

Signed-off-by: John Snow 
---


Looks like ReST! The generated document also looks good to me.

Reviewed-by: Connor Kuehl 

Re: [PATCH for-6.2 31/34] target/arm: Implement MVE VCTP

[Richard Henderson]

On 7/13/21 3:37 AM, Peter Maydell wrote:

Implement the MVE VCTP insn, which sets the VPR.P0 predicate bits so
as to predicate any element at index Rn or greater is predicated.  As
with VPNOT, this insn itself is predicable and subject to beatwise
execution.

The calculation of the mask is the same as is used to determine
ltpmask in mve_element_mask(), but we precalculate masklen in
generated code to avoid having to have 4 helpers specialized by size.

We put the decode line in with the low-overhead-loop insns in
t32.decode because it's logically part of that collection of insn
patterns, even though it is an MVE only insn.

Signed-off-by: Peter Maydell
---
  target/arm/helper-mve.h|  2 ++
  target/arm/translate-a32.h |  1 +
  target/arm/t32.decode  |  1 +
  target/arm/mve_helper.c| 20 
  target/arm/translate-mve.c |  2 +-
  target/arm/translate.c | 33 +
  6 files changed, 58 insertions(+), 1 deletion(-)


Reviewed-by: Richard Henderson 

r~

Re: [PATCH for-6.2 30/34] target/arm: Implement MVE VPNOT

[Richard Henderson]

On 7/13/21 3:37 AM, Peter Maydell wrote:

Implement the MVE VPNOT insn, which inverts the bits in VPR.P0
(subject to both predication and to beatwise execution).

Signed-off-by: Peter Maydell
---
  target/arm/helper-mve.h|  1 +
  target/arm/mve.decode  |  1 +
  target/arm/mve_helper.c| 17 +
  target/arm/translate-mve.c | 19 +++
  4 files changed, 38 insertions(+)


Reviewed-by: Richard Henderson 

r~

Re: [PATCH for-6.2 29/34] target/arm: Implement MVE VMOV to/from 2 general-purpose registers

[Richard Henderson]

On 7/13/21 3:37 AM, Peter Maydell wrote:

Implement the MVE VMOV forms that move data between 2 general-purpose
registers and 2 32-bit lanes in a vector register.

Signed-off-by: Peter Maydell
---
  target/arm/translate-a32.h |  1 +
  target/arm/mve.decode  |  4 ++
  target/arm/translate-mve.c | 85 ++
  target/arm/translate-vfp.c |  2 +-
  4 files changed, 91 insertions(+), 1 deletion(-)


Reviewed-by: Richard Henderson 

r~

Re: [PATCH for-6.2 28/34] target/arm: Implement MVE VMAXA, VMINA

[Richard Henderson]

On 7/13/21 3:37 AM, Peter Maydell wrote:

Implement the MVE VMAXA and VMINA insns, which take the absolute
value of the signed elements in the input vector and then accumulate
the unsigned max or min into the destination vector.

Signed-off-by: Peter Maydell
---
  target/arm/helper-mve.h|  8 
  target/arm/mve.decode  |  4 
  target/arm/mve_helper.c| 26 ++
  target/arm/translate-mve.c |  2 ++
  4 files changed, 40 insertions(+)


Reviewed-by: Richard Henderson 

r~

Re: [PATCH for-6.2 27/34] target/arm: Implement MVE VQABS, VQNEG

[Richard Henderson]

On 7/13/21 3:37 AM, Peter Maydell wrote:

Implement the MVE 1-operand saturating operations VQABS and VQNEG.

Signed-off-by: Peter Maydell
---
  target/arm/helper-mve.h|  8 
  target/arm/mve.decode  |  3 +++
  target/arm/mve_helper.c| 37 +
  target/arm/translate-mve.c |  2 ++
  4 files changed, 50 insertions(+)


Reviewed-by: Richard Henderson 

r~

Re: [PATCH for-6.2 26/34] target/arm: Implement MVE saturating doubling multiply accumulates

[Richard Henderson]

On 7/13/21 3:37 AM, Peter Maydell wrote:

Implement the MVE saturating doubling multiply accumulate insns
VQDMLAH, VQRDMLAH, VQDMLASH and VQRDMLASH.  These perform a multiply,
double, add the accumulator shifted by the element size, possibly
round, saturate to twice the element size, then take the high half of
the result.  The *MLAH insns do vector * scalar + vector, and the
*MLASH insns do vector * vector + scalar.

Signed-off-by: Peter Maydell
---
  target/arm/helper-mve.h| 16 +++
  target/arm/mve.decode  |  5 ++
  target/arm/mve_helper.c| 95 ++
  target/arm/translate-mve.c |  4 ++
  4 files changed, 120 insertions(+)


Reviewed-by: Richard Henderson 

r~

Re: [PATCH for-6.2 25/34] target/arm: Implement MVE VMLA

[Richard Henderson]

On 7/13/21 3:37 AM, Peter Maydell wrote:

Implement the MVE VMLA insn, which multiplies a vector by a scalar
and accumulates into another vector.

Signed-off-by: Peter Maydell
---
  target/arm/helper-mve.h| 8 
  target/arm/mve.decode  | 3 +++
  target/arm/mve_helper.c| 6 ++
  target/arm/translate-mve.c | 2 ++
  4 files changed, 19 insertions(+)


Reviewed-by: Richard Henderson 

r~

Re: [PATCH for-6.2 24/34] target/arm: Implement MVE VMLADAV and VMLSLDAV

[Richard Henderson]

On 7/13/21 3:37 AM, Peter Maydell wrote:

Implement the MVE VMLADAV and VMLSLDAV insns.  Like the VMLALDAV and
VMLSLDAV insns already implemented, these accumulate multiplied
vector elements; but they accumulate a 32-bit result rather than a
64-bit one.

Note that these encodings overlap with what would be RdaHi=0b111 for
VMLALDAV, VMLSLDAV, VRMLALDAVH and VRMLSLDAVH.

Signed-off-by: Peter Maydell
---
  target/arm/helper-mve.h| 17 ++
  target/arm/mve.decode  | 33 +---
  target/arm/mve_helper.c| 41 
  target/arm/translate-mve.c | 64 ++
  4 files changed, 150 insertions(+), 5 deletions(-)


Reviewed-by: Richard Henderson 

r~

Re: [PATCH for-6.1?] iotest: Further enhance qemu-img-bitmaps

[Nir Soffer]

On 7/21/21 11:46 PM, Eric Blake wrote:

Add a regression test to make sure we detect attempts to use 'qemu-img
bitmap' to modify an in-use local file.

Suggested-by: Nir Soffer 
Signed-off-by: Eric Blake 
---

Sadly, this missed my bitmaps pull request today.  If there's any
reason to respin that pull request, I'm inclined to add this in, as it
just touches the iotests; otherwise, if it slips to 6.2 it's not too
bad.

  tests/qemu-iotests/tests/qemu-img-bitmaps | 6 ++
  tests/qemu-iotests/tests/qemu-img-bitmaps.out | 5 +
  2 files changed, 11 insertions(+)

diff --git a/tests/qemu-iotests/tests/qemu-img-bitmaps 
b/tests/qemu-iotests/tests/qemu-img-bitmaps
index 7a3fe8c3d37a..3b6fade11735 100755
--- a/tests/qemu-iotests/tests/qemu-img-bitmaps
+++ b/tests/qemu-iotests/tests/qemu-img-bitmaps
@@ -129,6 +129,12 @@ $QEMU_IMG map --output=json --image-opts \
  $QEMU_IMG map --output=json --image-opts \
  "$IMG,x-dirty-bitmap=qemu:dirty-bitmap:b3" | _filter_qemu_img_map

+echo
+echo "=== bitmap command fails to modify image already in use ==="
+echo
+
+$QEMU_IMG bitmap --add "$TEST_IMG" b4 2>&1 | _filter_testdir
+
  nbd_server_stop

  echo
diff --git a/tests/qemu-iotests/tests/qemu-img-bitmaps.out 
b/tests/qemu-iotests/tests/qemu-img-bitmaps.out
index e851f0320ecb..c6e12dd700aa 100644
--- a/tests/qemu-iotests/tests/qemu-img-bitmaps.out
+++ b/tests/qemu-iotests/tests/qemu-img-bitmaps.out
@@ -116,6 +116,11 @@ Format specific information:
  { "start": 2097152, "length": 1048576, "depth": 0, "present": false, "zero": false, 
"data": false},
  { "start": 3145728, "length": 7340032, "depth": 0, "present": true, "zero": false, 
"data": true, "offset": OFFSET}]

+=== bitmap command fails to modify image already in use ===
+
+qemu-img: Could not open 'TEST_DIR/t.qcow2': Failed to get "write" lock
+Is another process using the image [TEST_DIR/t.qcow2]?
+
  === Check handling of inconsistent bitmap ===

  image: TEST_DIR/t.IMGFMT



It would be nice to test more than --add. I guess the implementation
is shared but if someone change it the test will protect us.

Reviewed-by: Nir Soffer 

[ANNOUNCE] QEMU 6.1.0-rc0 is now available

[Michael Roth]

Hello,

On behalf of the QEMU Team, I'd like to announce the availability of the
first release candidate for the QEMU 6.1 release.  This release is meant
for testing purposes and should not be used in a production environment.

  http://download.qemu-project.org/qemu-6.1.0-rc0.tar.xz
  http://download.qemu-project.org/qemu-6.1.0-rc0.tar.xz.sig

You can help improve the quality of the QEMU 6.1 release by testing this
release and reporting bugs on Launchpad:

  https://bugs.launchpad.net/qemu/

The release plan, as well a documented known issues for release
candidates, are available at:

  http://wiki.qemu.org/Planning/6.1

Please add entries to the ChangeLog for the 6.1 release below:

  http://wiki.qemu.org/ChangeLog/6.1

Thank you to everyone involved!

Re: [PATCH for-6.2 23/34] target/arm: Rename MVEGenDualAccOpFn to MVEGenLongDualAccOpFn

[Richard Henderson]

On 7/13/21 3:37 AM, Peter Maydell wrote:

The MVEGenDualAccOpFn is a bit misnamed, since it is used for
the "long dual accumulate" operations that use a 64-bit
accumulator. Rename it to MVEGenLongDualAccOpFn so we can
use the former name for the 32-bit accumulator insns.

Signed-off-by: Peter Maydell
---
  target/arm/translate-mve.c | 16 
  1 file changed, 8 insertions(+), 8 deletions(-)


Reviewed-by: Richard Henderson 

r~

Re: [PATCH for-6.2 22/34] target/arm: Implement MVE narrowing moves

[Richard Henderson]

On 7/13/21 3:37 AM, Peter Maydell wrote:

Implement the MVE narrowing move insns VMOVN, VQMOVN and VQMOVUN.
These take a double-width input, narrow it (possibly saturating) and
store the result to either the top or bottom half of the output
element.

Signed-off-by: Peter Maydell
---
  target/arm/helper-mve.h| 20 ++
  target/arm/mve.decode  | 12 ++
  target/arm/mve_helper.c| 78 ++
  target/arm/translate-mve.c | 22 +++
  4 files changed, 132 insertions(+)


Reviewed-by: Richard Henderson 

r~

[PULL 1/3] Hexagon (target/hexagon) remove put_user_*/get_user_*

[Taylor Simpson]

Replace put_user_* with cpu_st*_data_ra
Replace get_user_* with cpu_ld*_data_ra

Suggested-by: Richard Henderson 
Reviewed-by: Richard Henderson 
Signed-off-by: Taylor Simpson 
Message-Id: <1626384156-6248-2-git-send-email-tsimp...@quicinc.com>
---
 target/hexagon/op_helper.c | 39 ++-
 1 file changed, 18 insertions(+), 21 deletions(-)

diff --git a/target/hexagon/op_helper.c b/target/hexagon/op_helper.c
index 4595559..a959dba 100644
--- a/target/hexagon/op_helper.c
+++ b/target/hexagon/op_helper.c
@@ -17,6 +17,7 @@
 
 #include "qemu/osdep.h"
 #include "qemu.h"
+#include "exec/cpu_ldst.h"
 #include "exec/helper-proto.h"
 #include "fpu/softfloat.h"
 #include "cpu.h"
@@ -140,22 +141,22 @@ void HELPER(debug_check_store_width)(CPUHexagonState 
*env, int slot, int check)
 
 void HELPER(commit_store)(CPUHexagonState *env, int slot_num)
 {
-switch (env->mem_log_stores[slot_num].width) {
+uintptr_t ra = GETPC();
+uint8_t width = env->mem_log_stores[slot_num].width;
+target_ulong va = env->mem_log_stores[slot_num].va;
+
+switch (width) {
 case 1:
-put_user_u8(env->mem_log_stores[slot_num].data32,
-env->mem_log_stores[slot_num].va);
+cpu_stb_data_ra(env, va, env->mem_log_stores[slot_num].data32, ra);
 break;
 case 2:
-put_user_u16(env->mem_log_stores[slot_num].data32,
- env->mem_log_stores[slot_num].va);
+cpu_stw_data_ra(env, va, env->mem_log_stores[slot_num].data32, ra);
 break;
 case 4:
-put_user_u32(env->mem_log_stores[slot_num].data32,
- env->mem_log_stores[slot_num].va);
+cpu_stl_data_ra(env, va, env->mem_log_stores[slot_num].data32, ra);
 break;
 case 8:
-put_user_u64(env->mem_log_stores[slot_num].data64,
- env->mem_log_stores[slot_num].va);
+cpu_stq_data_ra(env, va, env->mem_log_stores[slot_num].data64, ra);
 break;
 default:
 g_assert_not_reached();
@@ -393,37 +394,33 @@ static void check_noshuf(CPUHexagonState *env, uint32_t 
slot)
 static uint8_t mem_load1(CPUHexagonState *env, uint32_t slot,
  target_ulong vaddr)
 {
-uint8_t retval;
+uintptr_t ra = GETPC();
 check_noshuf(env, slot);
-get_user_u8(retval, vaddr);
-return retval;
+return cpu_ldub_data_ra(env, vaddr, ra);
 }
 
 static uint16_t mem_load2(CPUHexagonState *env, uint32_t slot,
   target_ulong vaddr)
 {
-uint16_t retval;
+uintptr_t ra = GETPC();
 check_noshuf(env, slot);
-get_user_u16(retval, vaddr);
-return retval;
+return cpu_lduw_data_ra(env, vaddr, ra);
 }
 
 static uint32_t mem_load4(CPUHexagonState *env, uint32_t slot,
   target_ulong vaddr)
 {
-uint32_t retval;
+uintptr_t ra = GETPC();
 check_noshuf(env, slot);
-get_user_u32(retval, vaddr);
-return retval;
+return cpu_ldl_data_ra(env, vaddr, ra);
 }
 
 static uint64_t mem_load8(CPUHexagonState *env, uint32_t slot,
   target_ulong vaddr)
 {
-uint64_t retval;
+uintptr_t ra = GETPC();
 check_noshuf(env, slot);
-get_user_u64(retval, vaddr);
-return retval;
+return cpu_ldq_data_ra(env, vaddr, ra);
 }
 
 /* Floating point */
-- 
2.7.4

[PULL 2/3] target/hexagon: Drop include of qemu.h

[Taylor Simpson]

From: Peter Maydell 

The qemu.h file is a CONFIG_USER_ONLY header; it doesn't appear on
the include path for softmmu builds.  Currently we include it
unconditionally in target/hexagon/op_helper.c.  We used to need it
for the put_user_*() and get_user_*() functions, but now that we have
removed the uses of those from op_helper.c, the only reason it's
still there is that we're implicitly relying on it pulling in some
other headers.

Explicitly include the headers we need for other functions, and drop
the include of qemu.h.

Signed-off-by: Peter Maydell 
Message-Id: <20210717103017.20491-1-peter.mayd...@linaro.org>
Reviewed-by: Alex Bennée 
Reviewed-by: Richard Henderson 
Reviewed-by: Taylor Simpson 
Signed-off-by: Taylor Simpson 
---
 target/hexagon/op_helper.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/target/hexagon/op_helper.c b/target/hexagon/op_helper.c
index a959dba..61d5cde 100644
--- a/target/hexagon/op_helper.c
+++ b/target/hexagon/op_helper.c
@@ -16,7 +16,8 @@
  */
 
 #include "qemu/osdep.h"
-#include "qemu.h"
+#include "qemu/log.h"
+#include "exec/exec-all.h"
 #include "exec/cpu_ldst.h"
 #include "exec/helper-proto.h"
 #include "fpu/softfloat.h"
-- 
2.7.4

[PULL 3/3] linux-test (tests/tcg/multiarch/linux-test.c) add check

[Taylor Simpson]

Add a check that the SIGSEGV handler is called

Reviewed-by: Richard Henderson 
Signed-off-by: Taylor Simpson 
Message-Id: <1626384156-6248-3-git-send-email-tsimp...@quicinc.com>
---
 tests/tcg/multiarch/linux-test.c | 8 
 1 file changed, 8 insertions(+)

diff --git a/tests/tcg/multiarch/linux-test.c b/tests/tcg/multiarch/linux-test.c
index c8c6aed..e2d88f8 100644
--- a/tests/tcg/multiarch/linux-test.c
+++ b/tests/tcg/multiarch/linux-test.c
@@ -439,10 +439,14 @@ static void sig_alarm(int sig)
 alarm_count++;
 }
 
+/* Count the number of times handler is called */
+static int sig_segv_called;
+
 static void sig_segv(int sig, siginfo_t *info, void *puc)
 {
 if (sig != SIGSEGV)
 error("signal");
+sig_segv_called++;
 longjmp(jmp_env, 1);
 }
 
@@ -492,6 +496,10 @@ static void test_signal(void)
 *(volatile uint8_t *)0 = 0;
 }
 
+if (sig_segv_called == 0) {
+error("SIGSEGV handler not called");
+}
+
 act.sa_handler = SIG_DFL;
 sigemptyset(&act.sa_mask);
 act.sa_flags = 0;
-- 
2.7.4

[PULL 0/3] SIGSEGV fixes

[Taylor Simpson]

The following changes since commit 7457b407edd6e8555e4b46488aab2f13959fccf8:

  Merge remote-tracking branch 
'remotes/thuth-gitlab/tags/pull-request-2021-07-19' into staging (2021-07-19 
11:34:08 +0100)

are available in the git repository at:

  https://github.com/quic/qemu tags/pull-hex-20210721

for you to fetch changes up to 953ea3e4b426ee0c8349343c53e3358cfec720f2:

  linux-test (tests/tcg/multiarch/linux-test.c) add check (2021-07-21 15:54:28 
-0500)


The Hexagon target was silently failing the SIGSEGV test because
the signal handler was not called.

Patch 1/3 fixes the Hexagon target
Patch 2/3 drops include qemu.h from target/hexagon/op_helper.c
Patch 3/3 adds a check that the signal handler is called


Peter Maydell (1):
  target/hexagon: Drop include of qemu.h

Taylor Simpson (2):
  Hexagon (target/hexagon) remove put_user_*/get_user_*
  linux-test (tests/tcg/multiarch/linux-test.c) add check

 target/hexagon/op_helper.c   | 42 +++-
 tests/tcg/multiarch/linux-test.c |  8 
 2 files changed, 28 insertions(+), 22 deletions(-)

Re: [PATCH v2 2/5] migration: Make from_dst_file accesses thread-safe

[Eric Blake]

On Wed, Jul 21, 2021 at 03:34:06PM -0400, Peter Xu wrote:
> Accessing from_dst_file is potentially racy in current code base like below:
> 
>   if (s->from_dst_file)
> do_something(s->from_dst_file);
> 
> Because from_dst_file can be reset right after the check in another
> thread (rp_thread).  One example is migrate_fd_cancel().
> 
> Use the same qemu_file_lock to protect it too, just like to_dst_file.
> 
> When it's safe to access without lock, comment it.
> 
> There's one special reference in migration_thread() that can be replaced by
> the newly introduced rp_thread_created flag.
> 
> Reported-by: Dr. David Alan Gilbert 
> Signed-off-by: Peter Xu 
> ---
>  migration/migration.c | 32 +---
>  migration/migration.h |  8 +---
>  migration/ram.c   |  1 +
>  3 files changed, 31 insertions(+), 10 deletions(-)
> 
> diff --git a/migration/migration.c b/migration/migration.c
> index 21b94f75a3..fa70400f98 100644
> --- a/migration/migration.c
> +++ b/migration/migration.c
> @@ -1879,10 +1879,12 @@ static void migrate_fd_cancel(MigrationState *s)
>  QEMUFile *f = migrate_get_current()->to_dst_file;
>  trace_migrate_fd_cancel();
>  
> +qemu_mutex_lock(&s->qemu_file_lock);
>  if (s->rp_state.from_dst_file) {
>  /* shutdown the rp socket, so causing the rp thread to shutdown */
>  qemu_file_shutdown(s->rp_state.from_dst_file);
>  }
> +qemu_mutex_unlock(&s->qemu_file_lock);

Worth using WITH_QEMU_LOCK_GUARD?

> @@ -2827,11 +2845,13 @@ out:
>   * Maybe there is something we can do: it looks like a
>   * network down issue, and we pause for a recovery.
>   */
> -qemu_fclose(rp);
> -ms->rp_state.from_dst_file = NULL;
> +migration_release_from_dst_file(ms);
>  rp = NULL;
>  if (postcopy_pause_return_path_thread(ms)) {
> -/* Reload rp, reset the rest */
> +/*
> + * Reload rp, reset the rest.  Referencing it is save since

s/save/safe/

> + * it's reset only by us above, or when migration completes
> + */
>  rp = ms->rp_state.from_dst_file;
>  ms->rp_state.error = false;
>  goto retry;

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

Re: [PATCH for 6.1 1/2] ui/gtk: add a keyboard fifo to the VTE consoles

[Volker Rümelin]

+static void gd_vc_send_chars(VirtualConsole *vc)
+{
+uint32_t len, avail;
+const uint8_t *buf;
+
+len = qemu_chr_be_can_write(vc->vte.chr);
+avail = fifo8_num_used(&vc->vte.out_fifo);
+if (len > avail) {
+len = avail;
+}
+while (len > 0) {
+uint32_t size;
+
+buf = fifo8_pop_buf(&vc->vte.out_fifo, len, &size);
+qemu_chr_be_write(vc->vte.chr, (uint8_t *)buf, size);
+len -= size;
+avail -= size;
+}
+/*
+ * characters are pending: we send them a bit later (XXX:
+ * horrible, should change char device API)
+ */
+if (avail > 0) {
+timer_mod(vc->vte.kbd_timer,
+  qemu_clock_get_ms(QEMU_CLOCK_VIRTUAL) + 1);
+}

There is ChardevClass->chr_accept_input() which gets called when you can
send more data, so there is no need to use a timer for that.


Oh, I didn't notice this callback function. With this, the retry timer 
and my attempt to quickly slow down the write retries are really not 
necessary.




Typical workflow is to only read data when it can be pushed forward to
the guest, so when the guest stops reading data qemu stops doing so too,
effectively forwarding the stalls.  Which works fine for things like tcp
sockets.  Not so much for user input though.

So, yes, just throw away data is the only option we have here.  Adding a
reasonable-sized fifo makes sense too to cover bulky input, so you can
cut+paste a longish URL even if the guest accepts only a few chars at a
time (16550 fifo is 16 chars IIRC ...).

I would suggest to keep things simple, just throw away what you can't
store in the fifo, I don't see the point taking different actions
depending on how long the stalls are lasting (patch 2/2).


I will send a version 2 patch.

With best regards,
Volker

QEMU question: upstreaming I2C device with unpublished datasheet

[Shengtan Mao]

Hi everyone,
we are hoping to upstream a MAX I2C device to QEMU. The device's datasheet
is not public, and we are contacting the Maxim company to get their
permission. If Maxim is okay with upstreaming the device with an
unpublished datasheet, will this still be an issue with QEMU?

best,
Shengtan

Re: About two-dimensional page translation (e.g., Intel EPT) and shadow page table in Linux QEMU/KVM

[Sean Christopherson]

On Thu, Jul 15, 2021, harry harry wrote:
> Hi Sean,
> 
> Thanks for the explanations. Please see my comments below. Thanks!
> 
> >  When TDP (EPT) is used, the hardware MMU has two parts: the TDP PTEs that
> >  are controlled by KVM, and the IA32 PTEs that are controlled by the guest.
> >  And there's still a KVM MMU for the guest; the KVM MMU in that case knows
> >  how to connfigure the TDP PTEs in hardware _and_ walk the guest IA32 PTEs,
> >  e.g. to handle memory accesses during emulation.
> 
> Sorry, I could not understand why the emulated MMU is still needed
> when TDP (e.g., Intel EPT) is used?
> In particular, in what situations, we need the emulated MMU to
> configure the TDP PTEs in hardware and walk the guest IA32 PTEs?

Ignoring some weird corner cases that blur the lines between emulation and
hardware configuration, the emulated IA32 MMU isn't used to configure TDP PTEs 
in
hardware, it's only used to walk the the guest page tables.

> Why do we need the emulated MMU in these situations?

For emulation of any instruction/flow that starts with a guest virtual address.
On Intel CPUs, that includes quite literally any "full" instruction emulation,
since KVM needs to translate CS:RIP to a guest physical address in order to 
fetch
the guest's code stream.  KVM can't avoid "full" emulation unless the guest is
heavily enlightened, e.g. to avoid string I/O, among many other things.

[PATCH for-6.1?] iotest: Further enhance qemu-img-bitmaps

[Eric Blake]

Add a regression test to make sure we detect attempts to use 'qemu-img
bitmap' to modify an in-use local file.

Suggested-by: Nir Soffer 
Signed-off-by: Eric Blake 
---

Sadly, this missed my bitmaps pull request today.  If there's any
reason to respin that pull request, I'm inclined to add this in, as it
just touches the iotests; otherwise, if it slips to 6.2 it's not too
bad.

 tests/qemu-iotests/tests/qemu-img-bitmaps | 6 ++
 tests/qemu-iotests/tests/qemu-img-bitmaps.out | 5 +
 2 files changed, 11 insertions(+)

diff --git a/tests/qemu-iotests/tests/qemu-img-bitmaps 
b/tests/qemu-iotests/tests/qemu-img-bitmaps
index 7a3fe8c3d37a..3b6fade11735 100755
--- a/tests/qemu-iotests/tests/qemu-img-bitmaps
+++ b/tests/qemu-iotests/tests/qemu-img-bitmaps
@@ -129,6 +129,12 @@ $QEMU_IMG map --output=json --image-opts \
 $QEMU_IMG map --output=json --image-opts \
 "$IMG,x-dirty-bitmap=qemu:dirty-bitmap:b3" | _filter_qemu_img_map

+echo
+echo "=== bitmap command fails to modify image already in use ==="
+echo
+
+$QEMU_IMG bitmap --add "$TEST_IMG" b4 2>&1 | _filter_testdir
+
 nbd_server_stop

 echo
diff --git a/tests/qemu-iotests/tests/qemu-img-bitmaps.out 
b/tests/qemu-iotests/tests/qemu-img-bitmaps.out
index e851f0320ecb..c6e12dd700aa 100644
--- a/tests/qemu-iotests/tests/qemu-img-bitmaps.out
+++ b/tests/qemu-iotests/tests/qemu-img-bitmaps.out
@@ -116,6 +116,11 @@ Format specific information:
 { "start": 2097152, "length": 1048576, "depth": 0, "present": false, "zero": 
false, "data": false},
 { "start": 3145728, "length": 7340032, "depth": 0, "present": true, "zero": 
false, "data": true, "offset": OFFSET}]

+=== bitmap command fails to modify image already in use ===
+
+qemu-img: Could not open 'TEST_DIR/t.qcow2': Failed to get "write" lock
+Is another process using the image [TEST_DIR/t.qcow2]?
+
 === Check handling of inconsistent bitmap ===

 image: TEST_DIR/t.IMGFMT
-- 
2.31.1

[PULL 22/27] target/avr: Implement gdb_adjust_breakpoint

[Richard Henderson]

Ensure at registration that all breakpoints are in
code space, not data space.

Reviewed-by: Philippe Mathieu-Daudé 
Signed-off-by: Richard Henderson 
---
 target/avr/cpu.h   |  1 +
 target/avr/cpu.c   |  1 +
 target/avr/gdbstub.c   | 13 +
 target/avr/translate.c | 14 --
 4 files changed, 15 insertions(+), 14 deletions(-)

diff --git a/target/avr/cpu.h b/target/avr/cpu.h
index d148e8c75a..93e3faa0a9 100644
--- a/target/avr/cpu.h
+++ b/target/avr/cpu.h
@@ -162,6 +162,7 @@ hwaddr avr_cpu_get_phys_page_debug(CPUState *cpu, vaddr 
addr);
 int avr_cpu_gdb_read_register(CPUState *cpu, GByteArray *buf, int reg);
 int avr_cpu_gdb_write_register(CPUState *cpu, uint8_t *buf, int reg);
 int avr_print_insn(bfd_vma addr, disassemble_info *info);
+vaddr avr_cpu_gdb_adjust_breakpoint(CPUState *cpu, vaddr addr);
 
 static inline int avr_feature(CPUAVRState *env, AVRFeature feature)
 {
diff --git a/target/avr/cpu.c b/target/avr/cpu.c
index 57e3fab4a0..ea14175ca5 100644
--- a/target/avr/cpu.c
+++ b/target/avr/cpu.c
@@ -223,6 +223,7 @@ static void avr_cpu_class_init(ObjectClass *oc, void *data)
 cc->disas_set_info = avr_cpu_disas_set_info;
 cc->gdb_read_register = avr_cpu_gdb_read_register;
 cc->gdb_write_register = avr_cpu_gdb_write_register;
+cc->gdb_adjust_breakpoint = avr_cpu_gdb_adjust_breakpoint;
 cc->gdb_num_core_regs = 35;
 cc->gdb_core_xml_file = "avr-cpu.xml";
 cc->tcg_ops = &avr_tcg_ops;
diff --git a/target/avr/gdbstub.c b/target/avr/gdbstub.c
index c28ed67efe..1c1b908c92 100644
--- a/target/avr/gdbstub.c
+++ b/target/avr/gdbstub.c
@@ -82,3 +82,16 @@ int avr_cpu_gdb_write_register(CPUState *cs, uint8_t 
*mem_buf, int n)
 
 return 0;
 }
+
+vaddr avr_cpu_gdb_adjust_breakpoint(CPUState *cpu, vaddr addr)
+{
+/*
+ * This is due to some strange GDB behavior
+ * Let's assume main has address 0x100:
+ * b main   - sets breakpoint at address 0x0100 (code)
+ * b *0x100 - sets breakpoint at address 0x00800100 (data)
+ *
+ * Force all breakpoints into code space.
+ */
+return addr % OFFSET_DATA;
+}
diff --git a/target/avr/translate.c b/target/avr/translate.c
index 8237a03c23..f7202a646b 100644
--- a/target/avr/translate.c
+++ b/target/avr/translate.c
@@ -2958,20 +2958,6 @@ static void avr_tr_translate_insn(DisasContextBase 
*dcbase, CPUState *cs)
 DisasContext *ctx = container_of(dcbase, DisasContext, base);
 TCGLabel *skip_label = NULL;
 
-/*
- * This is due to some strange GDB behavior
- * Let's assume main has address 0x100:
- * b main   - sets breakpoint at address 0x0100 (code)
- * b *0x100 - sets breakpoint at address 0x00800100 (data)
- *
- * The translator driver has already taken care of the code pointer.
- */
-if (!ctx->base.singlestep_enabled &&
-cpu_breakpoint_test(cs, OFFSET_DATA + ctx->base.pc_next, BP_ANY)) {
-gen_breakpoint(ctx);
-return;
-}
-
 /* Conditionally skip the next instruction, if indicated.  */
 if (ctx->skip_cond != TCG_COND_NEVER) {
 skip_label = gen_new_label();
-- 
2.25.1

Re: [PATCH v3 2/4] vhost-user-rng-pci: Add vhost-user-rng-pci implementation

[Alex Bennée]

Mathieu Poirier  writes:

> This patch provides a PCI bus interface to the vhost-user-rng backed.
> The implentation is similar to what was done for vhost-user-i2c-pci and
> vhost-user-fs-pci.
>
> Signed-off-by: Mathieu Poirier 
> ---

> +static void vhost_user_rng_pci_class_init(ObjectClass *klass, void *data)
> +{
> +DeviceClass *dc = DEVICE_CLASS(klass);
> +VirtioPCIClass *k = VIRTIO_PCI_CLASS(klass);
> +PCIDeviceClass *pcidev_k = PCI_DEVICE_CLASS(klass);
> +k->realize = vhost_user_rng_pci_realize;
> +set_bit(DEVICE_CATEGORY_INPUT, dc->categories);
> +device_class_set_props(dc, vhost_user_rng_pci_properties);
> +pcidev_k->vendor_id = PCI_VENDOR_ID_REDHAT_QUMRANET;
> +pcidev_k->device_id = 0; /* Set by virtio-pci based on virtio id */
> +pcidev_k->revision = 0x00;
> +pcidev_k->class_id = PCI_CLASS_COMMUNICATION_OTHER;

I noticed the other RNGs use:

 pcidev_k->class_id = PCI_CLASS_OTHERS;

-- 
Alex Bennée

[PULL 23/27] accel/tcg: Merge tb_find into its only caller

[Richard Henderson]

We are going to want two things:
(1) check for breakpoints will want to break out of the loop here,
(2) cflags can only be calculated with pc in hand.

Tested-by: Mark Cave-Ayland 
Reviewed-by: Alex Bennée 
Signed-off-by: Richard Henderson 
---
 accel/tcg/cpu-exec.c | 83 ++--
 1 file changed, 41 insertions(+), 42 deletions(-)

diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
index 5bb099174f..cde7069eb7 100644
--- a/accel/tcg/cpu-exec.c
+++ b/accel/tcg/cpu-exec.c
@@ -500,41 +500,6 @@ static inline void tb_add_jump(TranslationBlock *tb, int n,
 return;
 }
 
-static inline TranslationBlock *tb_find(CPUState *cpu,
-TranslationBlock *last_tb,
-int tb_exit, uint32_t cflags)
-{
-CPUArchState *env = (CPUArchState *)cpu->env_ptr;
-TranslationBlock *tb;
-target_ulong cs_base, pc;
-uint32_t flags;
-
-cpu_get_tb_cpu_state(env, &pc, &cs_base, &flags);
-
-tb = tb_lookup(cpu, pc, cs_base, flags, cflags);
-if (tb == NULL) {
-mmap_lock();
-tb = tb_gen_code(cpu, pc, cs_base, flags, cflags);
-mmap_unlock();
-/* We add the TB in the virtual pc hash table for the fast lookup */
-qatomic_set(&cpu->tb_jmp_cache[tb_jmp_cache_hash_func(pc)], tb);
-}
-#ifndef CONFIG_USER_ONLY
-/* We don't take care of direct jumps when address mapping changes in
- * system emulation. So it's not safe to make a direct jump to a TB
- * spanning two pages because the mapping for the second page can change.
- */
-if (tb->page_addr[1] != -1) {
-last_tb = NULL;
-}
-#endif
-/* See if we can patch the calling TB. */
-if (last_tb) {
-tb_add_jump(last_tb, tb_exit, tb);
-}
-return tb;
-}
-
 static inline bool cpu_handle_halt(CPUState *cpu)
 {
 if (cpu->halted) {
@@ -868,22 +833,56 @@ int cpu_exec(CPUState *cpu)
 int tb_exit = 0;
 
 while (!cpu_handle_interrupt(cpu, &last_tb)) {
-uint32_t cflags = cpu->cflags_next_tb;
 TranslationBlock *tb;
+target_ulong cs_base, pc;
+uint32_t flags, cflags;
 
-/* When requested, use an exact setting for cflags for the next
-   execution.  This is used for icount, precise smc, and stop-
-   after-access watchpoints.  Since this request should never
-   have CF_INVALID set, -1 is a convenient invalid value that
-   does not require tcg headers for cpu_common_reset.  */
+/*
+ * When requested, use an exact setting for cflags for the next
+ * execution.  This is used for icount, precise smc, and stop-
+ * after-access watchpoints.  Since this request should never
+ * have CF_INVALID set, -1 is a convenient invalid value that
+ * does not require tcg headers for cpu_common_reset.
+ */
+cflags = cpu->cflags_next_tb;
 if (cflags == -1) {
 cflags = curr_cflags(cpu);
 } else {
 cpu->cflags_next_tb = -1;
 }
 
-tb = tb_find(cpu, last_tb, tb_exit, cflags);
+cpu_get_tb_cpu_state(cpu->env_ptr, &pc, &cs_base, &flags);
+
+tb = tb_lookup(cpu, pc, cs_base, flags, cflags);
+if (tb == NULL) {
+mmap_lock();
+tb = tb_gen_code(cpu, pc, cs_base, flags, cflags);
+mmap_unlock();
+/*
+ * We add the TB in the virtual pc hash table
+ * for the fast lookup
+ */
+qatomic_set(&cpu->tb_jmp_cache[tb_jmp_cache_hash_func(pc)], 
tb);
+}
+
+#ifndef CONFIG_USER_ONLY
+/*
+ * We don't take care of direct jumps when address mapping
+ * changes in system emulation.  So it's not safe to make a
+ * direct jump to a TB spanning two pages because the mapping
+ * for the second page can change.
+ */
+if (tb->page_addr[1] != -1) {
+last_tb = NULL;
+}
+#endif
+/* See if we can patch the calling TB. */
+if (last_tb) {
+tb_add_jump(last_tb, tb_exit, tb);
+}
+
 cpu_loop_exec_tb(cpu, tb, &last_tb, &tb_exit);
+
 /* Try to align the host and virtual clocks
if the guest is in advance */
 align_clocks(&sc, cpu);
-- 
2.25.1

Re: [PATCH v2 3/6] python/aqmp-tui: Add AQMP TUI draft

[Niteesh G. S.]

On Wed, Jul 21, 2021 at 12:34 AM John Snow  wrote:

>
>
> On Tue, Jul 13, 2021 at 6:07 PM G S Niteesh Babu 
> wrote:
>
>> Added a draft of AQMP TUI.
>>
>> Implements the follwing basic features:
>> 1) Command transmission/reception.
>> 2) Shows events asynchronously.
>> 3) Shows server status in the bottom status bar.
>>
>> Also added necessary pylint, mypy configurations
>>
>> Signed-off-by: G S Niteesh Babu 
>> ---
>>  python/qemu/aqmp/aqmp_tui.py | 332 +++
>>  python/setup.cfg |  21 ++-
>>  2 files changed, 352 insertions(+), 1 deletion(-)
>>  create mode 100644 python/qemu/aqmp/aqmp_tui.py
>>
>> diff --git a/python/qemu/aqmp/aqmp_tui.py b/python/qemu/aqmp/aqmp_tui.py
>> new file mode 100644
>> index 00..f853efc1f5
>> --- /dev/null
>> +++ b/python/qemu/aqmp/aqmp_tui.py
>> @@ -0,0 +1,332 @@
>> +# Copyright (c) 2021
>> +#
>> +# Authors:
>> +#  Niteesh Babu G S 
>> +#
>> +# This work is licensed under the terms of the GNU GPL, version 2 or
>> +# later.  See the COPYING file in the top-level directory.
>> +
>> +import argparse
>> +import asyncio
>> +import logging
>> +from logging import Handler
>> +import signal
>> +
>> +import urwid
>> +import urwid_readline
>> +
>> +from .error import MultiException
>> +from .protocol import ConnectError
>> +from .qmp_protocol import QMP, ExecInterruptedError, ExecuteError
>> +from .util import create_task, pretty_traceback
>> +
>> +
>> +UPDATE_MSG = 'UPDATE_MSG'
>> +
>> +# Using root logger to enable all loggers under qemu and asyncio
>> +LOGGER = logging.getLogger()
>> +
>> +palette = [
>> +(Token.Punctuation, '', '', '', 'h15,bold', 'g7'),
>> +(Token.Text, '', '', '', '', 'g7'),
>> +(Token.Name.Tag, '', '', '', 'bold,#f88', 'g7'),
>> +(Token.Literal.Number.Integer, '', '', '', '#fa0', 'g7'),
>> +(Token.Literal.String.Double, '', '', '', '#6f6', 'g7'),
>> +(Token.Keyword.Constant, '', '', '', '#6af', 'g7'),
>> +('background', '', 'black', '', '', 'g7'),
>> +]
>> +
>> +
>> +class StatusBar(urwid.Text):
>> +"""
>> +A simple Text widget that currently only shows connection status.
>> +"""
>> +def __init__(self, text=''):
>> +super().__init__(text, align='right')
>> +
>> +
>> +class Editor(urwid_readline.ReadlineEdit):
>> +"""
>> +Support urwid_readline features along with
>> +history support which lacks in urwid_readline
>> +"""
>> +def __init__(self, master):
>> +super().__init__(caption='> ', multiline=True)
>> +self.master = master
>> +self.history = []
>> +self.last_index = -1
>> +self.show_history = False
>> +
>> +def keypress(self, size, key):
>> +# TODO: Add some logic for down key and clean up logic if
>> possible.
>> +# Returning None means the key has been handled by this widget
>> +# which otherwise is propogated to the parent widget to be
>> +# handled
>> +msg = self.get_edit_text()
>> +if key == 'up' and not msg:
>> +# Show the history when 'up arrow' is pressed with no input
>> text.
>> +# NOTE: The show_history logic is necessary because in
>> 'multiline'
>> +# mode (which we use) 'up arrow' is used to move between
>> lines.
>> +self.show_history = True
>> +last_msg = self.history[self.last_index] if self.history
>> else ''
>> +self.set_edit_text(last_msg)
>> +self.edit_pos = len(last_msg)
>> +self.last_index += 1
>> +elif key == 'up' and self.show_history:
>> +if self.last_index < len(self.history):
>> +self.set_edit_text(self.history[self.last_index])
>> +self.edit_pos = len(self.history[self.last_index])
>> +self.last_index += 1
>> +elif key == 'meta enter':
>> +# When using multiline, enter inserts a new line into the
>> editor
>> +# send the input to the server on alt + enter
>> +self.master.cb_send_to_server(msg)
>> +self.history.insert(0, msg)
>> +self.set_edit_text('')
>> +self.last_index = 0
>> +self.show_history = False
>> +else:
>> +self.show_history = False
>> +self.last_index = 0
>> +return super().keypress(size, key)
>> +return None
>> +
>> +
>> +class EditorWidget(urwid.Filler):
>> +"""
>> +Wraps CustomEdit
>> +"""
>> +def __init__(self, master):
>> +super().__init__(Editor(master), valign='top')
>> +
>> +
>> +class HistoryBox(urwid.ListBox):
>> +"""
>> +Shows all the QMP message transmitted/received
>> +"""
>> +def __init__(self, master):
>> +self.master = master
>> +self.history = urwid.SimpleFocusListWalker([])
>> +super().__init__(self.history)
>> +
>> +def add_to_history(self, history):
>> +self.history.append(urwid.Text(history))
>> +if sel

Re: [PATCH v3 3/4] vhost-user-rng: backend: Add RNG vhost-user daemon implementation

[Alex Bennée]

Mathieu Poirier  writes:

> This patch provides the vhost-user backend implementation to work
> in tandem with the vhost-user-rng implementation of the QEMU VMM.
>
> It uses the vhost-user API so that other VMM can re-use the interface
> without having to write the driver again.
>
> Signed-off-by: Mathieu Poirier 

Try the following patch which creates a nested main loop and runs it
until the g_timeout_add fires again.

--8<---cut here---start->8---
tools/virtio/vhost-user-rng: avoid mutex by using nested main loop

As we are blocking anyway all we really need to do is run a main loop
until the timer fires and the data is consumed.

Signed-off-by: Alex Bennée 

1 file changed, 30 insertions(+), 76 deletions(-)
tools/vhost-user-rng/main.c | 106 +---

modified   tools/vhost-user-rng/main.c
@@ -42,13 +42,10 @@
 
 typedef struct {
 VugDev dev;
-struct itimerspec ts;
-timer_t rate_limit_timer;
-pthread_mutex_t rng_mutex;
-pthread_cond_t rng_cond;
 int64_t quota_remaining;
-bool activate_timer;
+guint timer;
 GMainLoop *loop;
+GMainLoop *blocked;
 } VuRNG;
 
 static gboolean print_cap, verbose;
@@ -59,66 +56,26 @@ static gint source_fd, socket_fd = -1;
 static uint32_t period_ms = 1 << 16;
 static uint64_t max_bytes = INT64_MAX;
 
-static void check_rate_limit(union sigval sv)
+static gboolean check_rate_limit(gpointer user_data)
 {
-VuRNG *rng = sv.sival_ptr;
-bool wakeup = false;
+VuRNG *rng = (VuRNG *) user_data;
 
-pthread_mutex_lock(&rng->rng_mutex);
-/*
- * The timer has expired and the guest has used all available
- * entropy, which means function vu_rng_handle_request() is waiting
- * on us.  As such wake it up once we're done here.
- */
-if (rng->quota_remaining == 0) {
-wakeup = true;
+if (rng->blocked) {
+g_info("%s: called while blocked", __func__);
+g_main_loop_quit(rng->blocked);
 }
-
 /*
  * Reset the entropy available to the guest and tell function
  * vu_rng_handle_requests() to start the timer before using it.
  */
 rng->quota_remaining = max_bytes;
-rng->activate_timer = true;
-pthread_mutex_unlock(&rng->rng_mutex);
-
-if (wakeup) {
-pthread_cond_signal(&rng->rng_cond);
-}
-}
-
-static void setup_timer(VuRNG *rng)
-{
-struct sigevent sev;
-int ret;
-
-memset(&rng->ts, 0, sizeof(struct itimerspec));
-rng->ts.it_value.tv_sec = period_ms / 1000;
-rng->ts.it_value.tv_nsec = (period_ms % 1000) * 100;
-
-/*
- * Call function check_rate_limit() as if it was the start of
- * a new thread when the timer expires.
- */
-sev.sigev_notify = SIGEV_THREAD;
-sev.sigev_notify_function = check_rate_limit;
-sev.sigev_value.sival_ptr = rng;
-/* Needs to be NULL if defaults attributes are to be used. */
-sev.sigev_notify_attributes = NULL;
-ret = timer_create(CLOCK_MONOTONIC, &sev, &rng->rate_limit_timer);
-if (ret < 0) {
-fprintf(stderr, "timer_create() failed\n");
-}
-
+return true;
 }
 
-
 /* Virtio helpers */
 static uint64_t rng_get_features(VuDev *dev)
 {
-if (verbose) {
-g_info("%s: replying", __func__);
-}
+g_info("%s: replying", __func__);
 return 0;
 }
 
@@ -137,7 +94,7 @@ static void vu_rng_handle_requests(VuDev *dev, int qidx)
 VuVirtq *vq = vu_get_queue(dev, qidx);
 VuVirtqElement *elem;
 size_t to_read;
-int len, ret;
+int len;
 
 for (;;) {
 /* Get element in the vhost virtqueue */
@@ -149,24 +106,21 @@ static void vu_rng_handle_requests(VuDev *dev, int qidx)
 /* Get the amount of entropy to read from the vhost server */
 to_read = elem->in_sg[0].iov_len;
 
-pthread_mutex_lock(&rng->rng_mutex);
-
 /*
  * We have consumed all entropy available for this time slice.
  * Wait for the timer (check_rate_limit()) to tell us about the
  * start of a new time slice.
  */
 if (rng->quota_remaining == 0) {
-pthread_cond_wait(&rng->rng_cond, &rng->rng_mutex);
-}
-
-/* Start the timer if the last time slice has expired */
-if (rng->activate_timer == true) {
-rng->activate_timer = false;
-ret = timer_settime(rng->rate_limit_timer, 0, &rng->ts, NULL);
-if (ret < 0) {
-fprintf(stderr, "timer_settime() failed\n");
-}
+g_assert(!rng->blocked);
+rng->blocked = g_main_loop_new(g_main_loop_get_context(rng->loop), 
false);
+g_info("attempting to consume %ld bytes but no quota left (%s)",
+   to_read,
+   g_main_loop_is_running(rng->loop) ? "running" : "not 
running");
+g_main_loop_run(rng->blocked);
+g_info("return from blocked loop: %ld", rng->quota_remaining);
+g_main_loop_unref(rn

[PULL 27/27] accel/tcg: Record singlestep_enabled in tb->cflags

[Richard Henderson]

Set CF_SINGLE_STEP when single-stepping is enabled.
This avoids the need to flush all tb's when turning
single-stepping on or off.

Tested-by: Mark Cave-Ayland 
Reviewed-by: Alex Bennée 
Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 include/exec/exec-all.h   | 1 +
 accel/tcg/cpu-exec.c  | 7 ++-
 accel/tcg/translate-all.c | 4 
 accel/tcg/translator.c| 7 +--
 cpu.c | 4 
 5 files changed, 8 insertions(+), 15 deletions(-)

diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
index 6873cce8df..5d1b6d80fb 100644
--- a/include/exec/exec-all.h
+++ b/include/exec/exec-all.h
@@ -497,6 +497,7 @@ struct TranslationBlock {
 #define CF_COUNT_MASK0x01ff
 #define CF_NO_GOTO_TB0x0200 /* Do not chain with goto_tb */
 #define CF_NO_GOTO_PTR   0x0400 /* Do not chain with goto_ptr */
+#define CF_SINGLE_STEP   0x0800 /* gdbstub single-step in effect */
 #define CF_LAST_IO   0x8000 /* Last insn may be an IO access.  */
 #define CF_MEMI_ONLY 0x0001 /* Only instrument memory ops */
 #define CF_USE_ICOUNT0x0002
diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
index 5cc6363f4c..fc895cf51e 100644
--- a/accel/tcg/cpu-exec.c
+++ b/accel/tcg/cpu-exec.c
@@ -150,10 +150,15 @@ uint32_t curr_cflags(CPUState *cpu)
 uint32_t cflags = cpu->tcg_cflags;
 
 /*
+ * Record gdb single-step.  We should be exiting the TB by raising
+ * EXCP_DEBUG, but to simplify other tests, disable chaining too.
+ *
  * For singlestep and -d nochain, suppress goto_tb so that
  * we can log -d cpu,exec after every TB.
  */
-if (singlestep) {
+if (unlikely(cpu->singlestep_enabled)) {
+cflags |= CF_NO_GOTO_TB | CF_NO_GOTO_PTR | CF_SINGLE_STEP | 1;
+} else if (singlestep) {
 cflags |= CF_NO_GOTO_TB | 1;
 } else if (qemu_loglevel_mask(CPU_LOG_TB_NOCHAIN)) {
 cflags |= CF_NO_GOTO_TB;
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index bf82c15aab..bbfcfb698c 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -1432,10 +1432,6 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
 }
 QEMU_BUILD_BUG_ON(CF_COUNT_MASK + 1 != TCG_MAX_INSNS);
 
-if (cpu->singlestep_enabled) {
-max_insns = 1;
-}
-
  buffer_overflow:
 tb = tcg_tb_alloc(tcg_ctx);
 if (unlikely(!tb)) {
diff --git a/accel/tcg/translator.c b/accel/tcg/translator.c
index b45337f3ba..c53a7f8e44 100644
--- a/accel/tcg/translator.c
+++ b/accel/tcg/translator.c
@@ -38,11 +38,6 @@ bool translator_use_goto_tb(DisasContextBase *db, 
target_ulong dest)
 return false;
 }
 
-/* Suppress goto_tb in the case of single-steping.  */
-if (db->singlestep_enabled) {
-return false;
-}
-
 /* Check for the dest on the same page as the start of the TB.  */
 return ((db->pc_first ^ dest) & TARGET_PAGE_MASK) == 0;
 }
@@ -60,7 +55,7 @@ void translator_loop(const TranslatorOps *ops, 
DisasContextBase *db,
 db->is_jmp = DISAS_NEXT;
 db->num_insns = 0;
 db->max_insns = max_insns;
-db->singlestep_enabled = cpu->singlestep_enabled;
+db->singlestep_enabled = cflags & CF_SINGLE_STEP;
 
 ops->init_disas_context(db, cpu);
 tcg_debug_assert(db->is_jmp == DISAS_NEXT);  /* no early exit */
diff --git a/cpu.c b/cpu.c
index d6ae5ae581..e1799a15bc 100644
--- a/cpu.c
+++ b/cpu.c
@@ -326,10 +326,6 @@ void cpu_single_step(CPUState *cpu, int enabled)
 cpu->singlestep_enabled = enabled;
 if (kvm_enabled()) {
 kvm_update_guest_debug(cpu, 0);
-} else {
-/* must flush all the translated code to avoid inconsistencies */
-/* XXX: only flush what is necessary */
-tb_flush(cpu);
 }
 trace_breakpoint_singlestep(cpu->cpu_index, enabled);
 }
-- 
2.25.1

[PULL 20/27] target/i386: Implement debug_check_breakpoint

[Richard Henderson]

Return false for RF set, as we do in i386_tr_breakpoint_check.

Reviewed-by: Alex Bennée 
Signed-off-by: Richard Henderson 
---
 target/i386/tcg/tcg-cpu.c | 12 
 1 file changed, 12 insertions(+)

diff --git a/target/i386/tcg/tcg-cpu.c b/target/i386/tcg/tcg-cpu.c
index e96ec9bbcc..238e3a9395 100644
--- a/target/i386/tcg/tcg-cpu.c
+++ b/target/i386/tcg/tcg-cpu.c
@@ -54,6 +54,17 @@ static void x86_cpu_synchronize_from_tb(CPUState *cs,
 cpu->env.eip = tb->pc - tb->cs_base;
 }
 
+#ifndef CONFIG_USER_ONLY
+static bool x86_debug_check_breakpoint(CPUState *cs)
+{
+X86CPU *cpu = X86_CPU(cs);
+CPUX86State *env = &cpu->env;
+
+/* RF disables all architectural breakpoints. */
+return !(env->eflags & RF_MASK);
+}
+#endif
+
 #include "hw/core/tcg-cpu-ops.h"
 
 static const struct TCGCPUOps x86_tcg_ops = {
@@ -66,6 +77,7 @@ static const struct TCGCPUOps x86_tcg_ops = {
 .tlb_fill = x86_cpu_tlb_fill,
 #ifndef CONFIG_USER_ONLY
 .debug_excp_handler = breakpoint_handler,
+.debug_check_breakpoint = x86_debug_check_breakpoint,
 #endif /* !CONFIG_USER_ONLY */
 };
 
-- 
2.25.1

[PULL 24/27] accel/tcg: Move breakpoint recognition outside translation

[Richard Henderson]

Trigger breakpoints before beginning translation of a TB
that would begin with a BP.  Thus we never generate code
for the BP at all.

Single-step instructions within a page containing a BP so
that we are sure to check each insn for the BP as above.

We no longer need to flush any TBs when changing BPs.

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/286
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/404
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/489
Tested-by: Mark Cave-Ayland 
Reviewed-by: Alex Bennée 
Signed-off-by: Richard Henderson 
---
 accel/tcg/cpu-exec.c   | 91 --
 accel/tcg/translator.c | 24 +--
 cpu.c  | 20 --
 3 files changed, 89 insertions(+), 46 deletions(-)

diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
index cde7069eb7..5cc6363f4c 100644
--- a/accel/tcg/cpu-exec.c
+++ b/accel/tcg/cpu-exec.c
@@ -222,6 +222,76 @@ static inline void log_cpu_exec(target_ulong pc, CPUState 
*cpu,
 }
 }
 
+static bool check_for_breakpoints(CPUState *cpu, target_ulong pc,
+  uint32_t *cflags)
+{
+CPUBreakpoint *bp;
+bool match_page = false;
+
+if (likely(QTAILQ_EMPTY(&cpu->breakpoints))) {
+return false;
+}
+
+/*
+ * Singlestep overrides breakpoints.
+ * This requirement is visible in the record-replay tests, where
+ * we would fail to make forward progress in reverse-continue.
+ *
+ * TODO: gdb singlestep should only override gdb breakpoints,
+ * so that one could (gdb) singlestep into the guest kernel's
+ * architectural breakpoint handler.
+ */
+if (cpu->singlestep_enabled) {
+return false;
+}
+
+QTAILQ_FOREACH(bp, &cpu->breakpoints, entry) {
+/*
+ * If we have an exact pc match, trigger the breakpoint.
+ * Otherwise, note matches within the page.
+ */
+if (pc == bp->pc) {
+bool match_bp = false;
+
+if (bp->flags & BP_GDB) {
+match_bp = true;
+} else if (bp->flags & BP_CPU) {
+#ifdef CONFIG_USER_ONLY
+g_assert_not_reached();
+#else
+CPUClass *cc = CPU_GET_CLASS(cpu);
+assert(cc->tcg_ops->debug_check_breakpoint);
+match_bp = cc->tcg_ops->debug_check_breakpoint(cpu);
+#endif
+}
+
+if (match_bp) {
+cpu->exception_index = EXCP_DEBUG;
+return true;
+}
+} else if (((pc ^ bp->pc) & TARGET_PAGE_MASK) == 0) {
+match_page = true;
+}
+}
+
+/*
+ * Within the same page as a breakpoint, single-step,
+ * returning to helper_lookup_tb_ptr after each insn looking
+ * for the actual breakpoint.
+ *
+ * TODO: Perhaps better to record all of the TBs associated
+ * with a given virtual page that contains a breakpoint, and
+ * then invalidate them when a new overlapping breakpoint is
+ * set on the page.  Non-overlapping TBs would not be
+ * invalidated, nor would any TB need to be invalidated as
+ * breakpoints are removed.
+ */
+if (match_page) {
+*cflags = (*cflags & ~CF_COUNT_MASK) | CF_NO_GOTO_TB | 1;
+}
+return false;
+}
+
 /**
  * helper_lookup_tb_ptr: quick check for next tb
  * @env: current cpu state
@@ -235,11 +305,16 @@ const void *HELPER(lookup_tb_ptr)(CPUArchState *env)
 CPUState *cpu = env_cpu(env);
 TranslationBlock *tb;
 target_ulong cs_base, pc;
-uint32_t flags;
+uint32_t flags, cflags;
 
 cpu_get_tb_cpu_state(env, &pc, &cs_base, &flags);
 
-tb = tb_lookup(cpu, pc, cs_base, flags, curr_cflags(cpu));
+cflags = curr_cflags(cpu);
+if (check_for_breakpoints(cpu, pc, &cflags)) {
+cpu_loop_exit(cpu);
+}
+
+tb = tb_lookup(cpu, pc, cs_base, flags, cflags);
 if (tb == NULL) {
 return tcg_code_gen_epilogue;
 }
@@ -346,6 +421,12 @@ void cpu_exec_step_atomic(CPUState *cpu)
 cflags &= ~CF_PARALLEL;
 /* After 1 insn, return and release the exclusive lock. */
 cflags |= CF_NO_GOTO_TB | CF_NO_GOTO_PTR | 1;
+/*
+ * No need to check_for_breakpoints here.
+ * We only arrive in cpu_exec_step_atomic after beginning execution
+ * of an insn that includes an atomic operation we can't handle.
+ * Any breakpoint for this insn will have been recognized earlier.
+ */
 
 tb = tb_lookup(cpu, pc, cs_base, flags, cflags);
 if (tb == NULL) {
@@ -837,6 +918,8 @@ int cpu_exec(CPUState *cpu)
 target_ulong cs_base, pc;
 uint32_t flags, cflags;
 
+cpu_get_tb_cpu_state(cpu->env_ptr, &pc, &cs_base, &flags);
+
 /*
  * When requested, use an exact setting for cflags for the next
  * execution.  This is used for icount, precise smc, and stop-
@@ -851,7 +934,9 @@ int cpu_exec(CPUState *cpu)

Re: [PATCH v2 0/6] python: AQMP-TUI Prototype

[John Snow]

On Wed, Jul 21, 2021 at 2:09 PM Niteesh G. S.  wrote:

> On Wed, Jul 21, 2021 at 12:39 AM John Snow  wrote:
>
>> On Wed, Jul 14, 2021 at 3:07 PM Niteesh G. S. 
>> wrote:
>>
>> Why not?
>>
> I have already updated the status of this
> https://lists.gnu.org/archive/html/qemu-devel/2021-07/msg04059.html
>

Sorry, I missed this.

Thanks Niteesh, a few general comments that don't relate directly to the
>> code:
>>
>> 1. It would be nice to be able to highlight/copy-paste things out of the
>> history window, I seemingly can't right now.
>>
>> 2. It would be nice if the mouse scroll wheel worked on the history panel.
>>
>> 3. A greeting message like the old qmp-shell might be nice to see. It
>> would be good if it explained how to quit the program (esc, ctrl^c) and
>> send messages (alt+enter).
>>
>> 4. Some control hints or reminder text in the footer might be nice, for
>> how to quit, send a message, etc.
>>
>

> I'll update the status here as I start working on them one by one.
>
>
OK - They don't need to go into this series, these are just some
observations. All of these items seem like good candidates for standalone
follow-up patches to happen in another series that follows this one.


> For the next revision, I may ask you to start looking into making sure
>> that mypy and pylint pass without exemptions. Do the best you can, and get
>> as far as you are able. You can leave the warnings disabled for V3, but I'd
>> like you to start taking a look now so that you know where the trouble
>> spots are.
>>
>

> Sure.
>
>
I'll be on PTO for the next three business days, returning 2021-07-27 -- If
you get blocked on other tasks, try adding mypy type hints using this
downtime.

Thanks again,
--js

[PULL 16/27] accel/tcg: Handle -singlestep in curr_cflags

[Richard Henderson]

Exchange the test in translator_use_goto_tb for CF_NO_GOTO_TB,
and the test in tb_gen_code for setting CF_COUNT_MASK to 1.

Tested-by: Mark Cave-Ayland 
Signed-off-by: Richard Henderson 
Reviewed-by: Peter Maydell 
Message-Id: <20210717221851.2124573-6-richard.hender...@linaro.org>
---
 accel/tcg/cpu-exec.c  | 8 +++-
 accel/tcg/translate-all.c | 2 +-
 accel/tcg/translator.c| 2 +-
 3 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
index 70ea3c7d68..2206c463f5 100644
--- a/accel/tcg/cpu-exec.c
+++ b/accel/tcg/cpu-exec.c
@@ -149,7 +149,13 @@ uint32_t curr_cflags(CPUState *cpu)
 {
 uint32_t cflags = cpu->tcg_cflags;
 
-if (qemu_loglevel_mask(CPU_LOG_TB_NOCHAIN)) {
+/*
+ * For singlestep and -d nochain, suppress goto_tb so that
+ * we can log -d cpu,exec after every TB.
+ */
+if (singlestep) {
+cflags |= CF_NO_GOTO_TB | 1;
+} else if (qemu_loglevel_mask(CPU_LOG_TB_NOCHAIN)) {
 cflags |= CF_NO_GOTO_TB;
 }
 
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index 5cc01d693b..bf82c15aab 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -1432,7 +1432,7 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
 }
 QEMU_BUILD_BUG_ON(CF_COUNT_MASK + 1 != TCG_MAX_INSNS);
 
-if (cpu->singlestep_enabled || singlestep) {
+if (cpu->singlestep_enabled) {
 max_insns = 1;
 }
 
diff --git a/accel/tcg/translator.c b/accel/tcg/translator.c
index 2ea5a74f30..a59eb7c11b 100644
--- a/accel/tcg/translator.c
+++ b/accel/tcg/translator.c
@@ -39,7 +39,7 @@ bool translator_use_goto_tb(DisasContextBase *db, 
target_ulong dest)
 }
 
 /* Suppress goto_tb in the case of single-steping.  */
-if (db->singlestep_enabled || singlestep) {
+if (db->singlestep_enabled) {
 return false;
 }
 
-- 
2.25.1

[PULL 14/27] accel/tcg: Add CF_NO_GOTO_TB and CF_NO_GOTO_PTR

[Richard Henderson]

Move the -d nochain check to bits on tb->cflags.
These will be used for more than -d nochain shortly.

Set bits during curr_cflags, test them in translator_use_goto_tb,
assert we're not doing anything odd in tcg_gen_goto_tb.  The test
in tcg_gen_exit_tb is redundant with the assert for goto_tb_issue_mask.

Tested-by: Mark Cave-Ayland 
Signed-off-by: Richard Henderson 
Reviewed-by: Alex Bennée 
Reviewed-by: Peter Maydell 
Message-Id: <20210717221851.2124573-4-richard.hender...@linaro.org>
---
 include/exec/exec-all.h | 16 +---
 accel/tcg/cpu-exec.c|  8 +++-
 accel/tcg/translator.c  |  5 +
 tcg/tcg-op.c| 28 
 4 files changed, 33 insertions(+), 24 deletions(-)

diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
index ae7603ca75..6873cce8df 100644
--- a/include/exec/exec-all.h
+++ b/include/exec/exec-all.h
@@ -494,13 +494,15 @@ struct TranslationBlock {
 uint32_t cflags;/* compile flags */
 
 /* Note that TCG_MAX_INSNS is 512; we validate this match elsewhere. */
-#define CF_COUNT_MASK  0x01ff
-#define CF_LAST_IO 0x8000 /* Last insn may be an IO access.  */
-#define CF_MEMI_ONLY   0x0001 /* Only instrument memory ops */
-#define CF_USE_ICOUNT  0x0002
-#define CF_INVALID 0x0004 /* TB is stale. Set with @jmp_lock held */
-#define CF_PARALLEL0x0008 /* Generate code for a parallel context */
-#define CF_CLUSTER_MASK 0xff00 /* Top 8 bits are cluster ID */
+#define CF_COUNT_MASK0x01ff
+#define CF_NO_GOTO_TB0x0200 /* Do not chain with goto_tb */
+#define CF_NO_GOTO_PTR   0x0400 /* Do not chain with goto_ptr */
+#define CF_LAST_IO   0x8000 /* Last insn may be an IO access.  */
+#define CF_MEMI_ONLY 0x0001 /* Only instrument memory ops */
+#define CF_USE_ICOUNT0x0002
+#define CF_INVALID   0x0004 /* TB is stale. Set with @jmp_lock held */
+#define CF_PARALLEL  0x0008 /* Generate code for a parallel context */
+#define CF_CLUSTER_MASK  0xff00 /* Top 8 bits are cluster ID */
 #define CF_CLUSTER_SHIFT 24
 
 /* Per-vCPU dynamic tracing state used to generate this TB */
diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
index ef4214d893..d3232d5764 100644
--- a/accel/tcg/cpu-exec.c
+++ b/accel/tcg/cpu-exec.c
@@ -147,7 +147,13 @@ static void init_delay_params(SyncClocks *sc, const 
CPUState *cpu)
 
 uint32_t curr_cflags(CPUState *cpu)
 {
-return cpu->tcg_cflags;
+uint32_t cflags = cpu->tcg_cflags;
+
+if (qemu_loglevel_mask(CPU_LOG_TB_NOCHAIN)) {
+cflags |= CF_NO_GOTO_TB | CF_NO_GOTO_PTR;
+}
+
+return cflags;
 }
 
 /* Might cause an exception, so have a longjmp destination ready */
diff --git a/accel/tcg/translator.c b/accel/tcg/translator.c
index 59804af37b..2ea5a74f30 100644
--- a/accel/tcg/translator.c
+++ b/accel/tcg/translator.c
@@ -33,6 +33,11 @@ void translator_loop_temp_check(DisasContextBase *db)
 
 bool translator_use_goto_tb(DisasContextBase *db, target_ulong dest)
 {
+/* Suppress goto_tb if requested. */
+if (tb_cflags(db->tb) & CF_NO_GOTO_TB) {
+return false;
+}
+
 /* Suppress goto_tb in the case of single-steping.  */
 if (db->singlestep_enabled || singlestep) {
 return false;
diff --git a/tcg/tcg-op.c b/tcg/tcg-op.c
index 75eaa910c9..c754396575 100644
--- a/tcg/tcg-op.c
+++ b/tcg/tcg-op.c
@@ -2723,10 +2723,6 @@ void tcg_gen_exit_tb(const TranslationBlock *tb, 
unsigned idx)
seen this numbered exit before, via tcg_gen_goto_tb.  */
 tcg_debug_assert(tcg_ctx->goto_tb_issue_mask & (1 << idx));
 #endif
-/* When not chaining, exit without indicating a link.  */
-if (qemu_loglevel_mask(CPU_LOG_TB_NOCHAIN)) {
-val = 0;
-}
 } else {
 /* This is an exit via the exitreq label.  */
 tcg_debug_assert(idx == TB_EXIT_REQUESTED);
@@ -2738,6 +2734,8 @@ void tcg_gen_exit_tb(const TranslationBlock *tb, unsigned 
idx)
 
 void tcg_gen_goto_tb(unsigned idx)
 {
+/* We tested CF_NO_GOTO_TB in translator_use_goto_tb. */
+tcg_debug_assert(!(tcg_ctx->tb_cflags & CF_NO_GOTO_TB));
 /* We only support two chained exits.  */
 tcg_debug_assert(idx <= TB_EXIT_IDXMAX);
 #ifdef CONFIG_DEBUG_TCG
@@ -2746,25 +2744,23 @@ void tcg_gen_goto_tb(unsigned idx)
 tcg_ctx->goto_tb_issue_mask |= 1 << idx;
 #endif
 plugin_gen_disable_mem_helpers();
-/* When not chaining, we simply fall through to the "fallback" exit.  */
-if (!qemu_loglevel_mask(CPU_LOG_TB_NOCHAIN)) {
-tcg_gen_op1i(INDEX_op_goto_tb, idx);
-}
+tcg_gen_op1i(INDEX_op_goto_tb, idx);
 }
 
 void tcg_gen_lookup_and_goto_ptr(void)
 {
-if (!qemu_loglevel_mask(CPU_LOG_TB_NOCHAIN)) {
-TCGv_ptr ptr;
+TCGv_ptr ptr;
 
-plugin_gen_disable_mem_helpers();
-ptr = tcg_temp_new_ptr();
-gen_helper_lookup_tb_ptr(ptr, cpu_env);
-tcg_gen_op1i(INDEX_op_goto_ptr, tcgv_ptr_arg(ptr));
-

[PULL 26/27] accel/tcg: Hoist tb_cflags to a local in translator_loop

[Richard Henderson]

The access internal to tb_cflags() is atomic.
Avoid re-reading it as such for the multiple uses.

Tested-by: Mark Cave-Ayland 
Reviewed-by: Alex Bennée 
Reviewed-by: Philippe Mathieu-Daudé 
Signed-off-by: Richard Henderson 
---
 accel/tcg/translator.c | 9 -
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/accel/tcg/translator.c b/accel/tcg/translator.c
index 4f3728c278..b45337f3ba 100644
--- a/accel/tcg/translator.c
+++ b/accel/tcg/translator.c
@@ -50,6 +50,7 @@ bool translator_use_goto_tb(DisasContextBase *db, 
target_ulong dest)
 void translator_loop(const TranslatorOps *ops, DisasContextBase *db,
  CPUState *cpu, TranslationBlock *tb, int max_insns)
 {
+uint32_t cflags = tb_cflags(tb);
 bool plugin_enabled;
 
 /* Initialize DisasContext */
@@ -72,8 +73,7 @@ void translator_loop(const TranslatorOps *ops, 
DisasContextBase *db,
 ops->tb_start(db, cpu);
 tcg_debug_assert(db->is_jmp == DISAS_NEXT);  /* no early exit */
 
-plugin_enabled = plugin_gen_tb_start(cpu, tb,
- tb_cflags(db->tb) & CF_MEMI_ONLY);
+plugin_enabled = plugin_gen_tb_start(cpu, tb, cflags & CF_MEMI_ONLY);
 
 while (true) {
 db->num_insns++;
@@ -88,14 +88,13 @@ void translator_loop(const TranslatorOps *ops, 
DisasContextBase *db,
update db->pc_next and db->is_jmp to indicate what should be
done next -- either exiting this loop or locate the start of
the next instruction.  */
-if (db->num_insns == db->max_insns
-&& (tb_cflags(db->tb) & CF_LAST_IO)) {
+if (db->num_insns == db->max_insns && (cflags & CF_LAST_IO)) {
 /* Accept I/O on the last instruction.  */
 gen_io_start();
 ops->translate_insn(db, cpu);
 } else {
 /* we should only see CF_MEMI_ONLY for io_recompile */
-tcg_debug_assert(!(tb_cflags(db->tb) & CF_MEMI_ONLY));
+tcg_debug_assert(!(cflags & CF_MEMI_ONLY));
 ops->translate_insn(db, cpu);
 }
 
-- 
2.25.1

Re: [PATCH v2 00/24] python: introduce Asynchronous QMP package

[Niteesh G. S.]

On Thu, Jul 22, 2021 at 1:25 AM John Snow  wrote:

> Looping qemu-devel back in: I removed them by accident by not hitting
> reply-all :(
>
> On Wed, Jul 21, 2021 at 2:06 PM Niteesh G. S. 
> wrote:
>
>>
>>
>> On Wed, Jul 21, 2021 at 11:03 PM John Snow  wrote:
>>
>>>
>>>
>>> On Wed, Jul 21, 2021 at 1:04 PM Niteesh G. S. 
>>> wrote:
>>>
 Hello all,

 I recently rebased(incrementally) my TUI on this V2 patch and faced an
 issue.
 https://gitlab.com/niteesh.gs/qemu/-/commits/aqmp-tui-prototype-v3
 I decided to rebase incrementally so that I can address some of the
 comments posted
 in my patch series. While testing out, the initial draft of TUI
 which worked fine in the V1
 version of AQMP failed in this version.

 Disconnecting from a fully connected state doesn't exit cleanly.

 -
 To reproduce the issue:
 1) Initiate a QMP server

>>>
>>> Please provide the command line.
>>>
>> qemu-system-x86_64 -qmp tcp:localhost:1234,server,wait=on
>>
>>>
>>>
 2) Connect the TUI to the server using aqmp-tui localhost:1234
 --log-file log.txt

>>>
>>> The entry point isn't defined yet in your series, so I will assume
>>> "python3 -m qemu.aqmp.aqmp_tui localhost:1234" should work here.
>>>
>> Yup, sorry about that. I realized this later when recreated the venv.
>>
>>>
>>>
 3) Once the TUI is connected and running, press 'Esc' to exit the app.
 This should result
 in the following exception.

 
 Transitioning from 'Runstate.IDLE' to 'Runstate.CONNECTING'.
 Connecting to ('localhost', 1234) ...
 Connected.
 Awaiting greeting ...
 Response: {
   "QMP": {
 .. Skipping
   }
 }
 Negotiating capabilities ...
 Request: {
   "execute": "qmp_capabilities",
 .. Skipping
   }
 }
 Response: {
   "return": {}
 }
 Transitioning from 'Runstate.CONNECTING' to 'Runstate.RUNNING'.
 Transitioning from 'Runstate.RUNNING' to 'Runstate.DISCONNECTING'.
 Scheduling disconnect.
 Draining the outbound queue ...
 Flushing the StreamWriter ...
 Cancelling writer task ...
 Task.Writer: cancelled.
 Task.Writer: exiting.
 Cancelling reader task ...
 Task.Reader: cancelled.
 Task.Reader: exiting.
 Closing StreamWriter.
 Waiting for StreamWriter to close ...
 QMP Disconnected.
 Transitioning from 'Runstate.DISCONNECTING' to 'Runstate.IDLE'.
 _kill_app: Connection lost
 Connection lost
   | Traceback (most recent call last):
   |   File
 "/home/niteesh/development/qemu/python/qemu/aqmp/aqmp_tui.py", line 246, in
 run
   | main_loop.run()
   |   File
 "/home/niteesh/development/qemu/python/.venv/lib/python3.6/site-packages/urwid/main_loop.py",
 line 287, in run
   | self._run()
   |   File
 "/home/niteesh/development/qemu/python/.venv/lib/python3.6/site-packages/urwid/main_loop.py",
 line 385, in _run
   | self.event_loop.run()
   |   File
 "/home/niteesh/development/qemu/python/.venv/lib/python3.6/site-packages/urwid/main_loop.py",
 line 1494, in run
   | reraise(*exc_info)
   |   File
 "/home/niteesh/development/qemu/python/.venv/lib/python3.6/site-packages/urwid/compat.py",
 line 58, in reraise
   | raise value
   |   File
 "/home/niteesh/development/qemu/python/qemu/aqmp/aqmp_tui.py", line 206, in
 _kill_app
   | raise err
   |   File
 "/home/niteesh/development/qemu/python/qemu/aqmp/aqmp_tui.py", line 201, in
 _kill_app
   | await self.disconnect()
   |   File
 "/home/niteesh/development/qemu/python/qemu/aqmp/protocol.py", line 303, in
 disconnect
   | await self._wait_disconnect()
   |   File
 "/home/niteesh/development/qemu/python/qemu/aqmp/protocol.py", line 573, in
 _wait_disconnect
   | await self._dc_task
   |   File
 "/home/niteesh/development/qemu/python/qemu/aqmp/qmp_client.py", line 316,
 in _bh_disconnect
   | await super()._bh_disconnect()
   |   File
 "/home/niteesh/development/qemu/python/qemu/aqmp/protocol.py", line 644, in
 _bh_disconnect
   | await wait_closed(self._writer)
   |   File "/home/niteesh/development/qemu/python/qemu/aqmp/util.py",
 line 137, in wait_closed
   | await flush(writer)
   |   File "/home/niteesh/development/qemu/python/qemu/aqmp/util.py",
 line 49, in flush
   | await writer.drain()
   |   File "/usr/lib/python3.6/asyncio/streams.py", line 339, in drain
   | yield from self._protocol._drain_helper()
   |   File "/usr/lib/python3.6/asyncio/streams.py", line 210, in
 _

[PULL 25/27] accel/tcg: Remove TranslatorOps.breakpoint_check

[Richard Henderson]

The hook is now unused, with breakpoints checked outside translation.

Tested-by: Mark Cave-Ayland 
Reviewed-by: Philippe Mathieu-Daudé 
Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 include/exec/translator.h | 11 ---
 target/arm/helper.h   |  2 --
 target/alpha/translate.c  | 16 
 target/arm/debug_helper.c |  7 ---
 target/arm/translate-a64.c| 25 -
 target/arm/translate.c| 29 -
 target/avr/translate.c| 18 --
 target/cris/translate.c   | 20 
 target/hexagon/translate.c| 17 -
 target/hppa/translate.c   | 11 ---
 target/i386/tcg/translate.c   | 28 
 target/m68k/translate.c   | 18 --
 target/microblaze/translate.c | 18 --
 target/mips/tcg/translate.c   | 19 ---
 target/nios2/translate.c  | 27 ---
 target/openrisc/translate.c   | 17 -
 target/ppc/translate.c| 18 --
 target/riscv/translate.c  | 17 -
 target/rx/translate.c | 14 --
 target/s390x/tcg/translate.c  | 24 
 target/sh4/translate.c| 18 --
 target/sparc/translate.c  | 17 -
 target/tricore/translate.c| 16 
 target/xtensa/translate.c | 17 -
 24 files changed, 424 deletions(-)

diff --git a/include/exec/translator.h b/include/exec/translator.h
index dd9c06d40d..d318803267 100644
--- a/include/exec/translator.h
+++ b/include/exec/translator.h
@@ -89,15 +89,6 @@ typedef struct DisasContextBase {
  * @insn_start:
  *  Emit the tcg_gen_insn_start opcode.
  *
- * @breakpoint_check:
- *  When called, the breakpoint has already been checked to match the PC,
- *  but the target may decide the breakpoint missed the address
- *  (e.g., due to conditions encoded in their flags).  Return true to
- *  indicate that the breakpoint did hit, in which case no more breakpoints
- *  are checked.  If the breakpoint did hit, emit any code required to
- *  signal the exception, and set db->is_jmp as necessary to terminate
- *  the main loop.
- *
  * @translate_insn:
  *  Disassemble one instruction and set db->pc_next for the start
  *  of the following instruction.  Set db->is_jmp as necessary to
@@ -113,8 +104,6 @@ typedef struct TranslatorOps {
 void (*init_disas_context)(DisasContextBase *db, CPUState *cpu);
 void (*tb_start)(DisasContextBase *db, CPUState *cpu);
 void (*insn_start)(DisasContextBase *db, CPUState *cpu);
-bool (*breakpoint_check)(DisasContextBase *db, CPUState *cpu,
- const CPUBreakpoint *bp);
 void (*translate_insn)(DisasContextBase *db, CPUState *cpu);
 void (*tb_stop)(DisasContextBase *db, CPUState *cpu);
 void (*disas_log)(const DisasContextBase *db, CPUState *cpu);
diff --git a/target/arm/helper.h b/target/arm/helper.h
index db87d7d537..248569b0cd 100644
--- a/target/arm/helper.h
+++ b/target/arm/helper.h
@@ -54,8 +54,6 @@ DEF_HELPER_1(yield, void, env)
 DEF_HELPER_1(pre_hvc, void, env)
 DEF_HELPER_2(pre_smc, void, env, i32)
 
-DEF_HELPER_1(check_breakpoints, void, env)
-
 DEF_HELPER_3(cpsr_write, void, env, i32, i32)
 DEF_HELPER_2(cpsr_write_eret, void, env, i32)
 DEF_HELPER_1(cpsr_read, i32, env)
diff --git a/target/alpha/translate.c b/target/alpha/translate.c
index 949ba6ffde..de6c0a8439 100644
--- a/target/alpha/translate.c
+++ b/target/alpha/translate.c
@@ -2967,21 +2967,6 @@ static void alpha_tr_insn_start(DisasContextBase 
*dcbase, CPUState *cpu)
 tcg_gen_insn_start(dcbase->pc_next);
 }
 
-static bool alpha_tr_breakpoint_check(DisasContextBase *dcbase, CPUState *cpu,
-  const CPUBreakpoint *bp)
-{
-DisasContext *ctx = container_of(dcbase, DisasContext, base);
-
-ctx->base.is_jmp = gen_excp(ctx, EXCP_DEBUG, 0);
-
-/* The address covered by the breakpoint must be included in
-   [tb->pc, tb->pc + tb->size) in order to for it to be
-   properly cleared -- thus we increment the PC here so that
-   the logic setting tb->size below does the right thing.  */
-ctx->base.pc_next += 4;
-return true;
-}
-
 static void alpha_tr_translate_insn(DisasContextBase *dcbase, CPUState *cpu)
 {
 DisasContext *ctx = container_of(dcbase, DisasContext, base);
@@ -3040,7 +3025,6 @@ static const TranslatorOps alpha_tr_ops = {
 .init_disas_context = alpha_tr_init_disas_context,
 .tb_start   = alpha_tr_tb_start,
 .insn_start = alpha_tr_insn_start,
-.breakpoint_check   = alpha_tr_breakpoint_check,
 .translate_insn = alpha_tr_translate_insn,
 .tb_stop= alpha_tr_tb_stop,
 .disas_log  = alpha_tr_disas_log,
diff --git a/target/arm/debug_helper.c

[PULL 12/27] accel/tcg: Move curr_cflags into cpu-exec.c

[Richard Henderson]

We will shortly have more than a simple member read here,
with stuff not necessarily exposed to exec/exec-all.h.

Tested-by: Mark Cave-Ayland 
Signed-off-by: Richard Henderson 
Reviewed-by: Peter Maydell 
Reviewed-by: Alex Bennée 
Reviewed-by: Philippe Mathieu-Daudé 
Message-Id: <20210717221851.2124573-3-richard.hender...@linaro.org>
---
 include/exec/exec-all.h | 5 +
 accel/tcg/cpu-exec.c| 5 +
 2 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
index dfe82ed19c..ae7603ca75 100644
--- a/include/exec/exec-all.h
+++ b/include/exec/exec-all.h
@@ -565,10 +565,7 @@ static inline uint32_t tb_cflags(const TranslationBlock 
*tb)
 }
 
 /* current cflags for hashing/comparison */
-static inline uint32_t curr_cflags(CPUState *cpu)
-{
-return cpu->tcg_cflags;
-}
+uint32_t curr_cflags(CPUState *cpu);
 
 /* TranslationBlock invalidate API */
 #if defined(CONFIG_USER_ONLY)
diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
index e22bcb99f7..ef4214d893 100644
--- a/accel/tcg/cpu-exec.c
+++ b/accel/tcg/cpu-exec.c
@@ -145,6 +145,11 @@ static void init_delay_params(SyncClocks *sc, const 
CPUState *cpu)
 }
 #endif /* CONFIG USER ONLY */
 
+uint32_t curr_cflags(CPUState *cpu)
+{
+return cpu->tcg_cflags;
+}
+
 /* Might cause an exception, so have a longjmp destination ready */
 static inline TranslationBlock *tb_lookup(CPUState *cpu, target_ulong pc,
   target_ulong cs_base,
-- 
2.25.1

[PULL 17/27] accel/tcg: Use CF_NO_GOTO_{TB, PTR} in cpu_exec_step_atomic

[Richard Henderson]

Request that the one TB returns immediately, so that
we release the exclusive lock as soon as possible.

Tested-by: Mark Cave-Ayland 
Signed-off-by: Richard Henderson 
Reviewed-by: Peter Maydell 
Reviewed-by: Alex Bennée 
Message-Id: <20210717221851.2124573-7-richard.hender...@linaro.org>
---
 accel/tcg/cpu-exec.c | 11 ---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
index 2206c463f5..5bb099174f 100644
--- a/accel/tcg/cpu-exec.c
+++ b/accel/tcg/cpu-exec.c
@@ -330,8 +330,7 @@ void cpu_exec_step_atomic(CPUState *cpu)
 CPUArchState *env = (CPUArchState *)cpu->env_ptr;
 TranslationBlock *tb;
 target_ulong cs_base, pc;
-uint32_t flags;
-uint32_t cflags = (curr_cflags(cpu) & ~CF_PARALLEL) | 1;
+uint32_t flags, cflags;
 int tb_exit;
 
 if (sigsetjmp(cpu->jmp_env, 0) == 0) {
@@ -341,8 +340,14 @@ void cpu_exec_step_atomic(CPUState *cpu)
 cpu->running = true;
 
 cpu_get_tb_cpu_state(env, &pc, &cs_base, &flags);
-tb = tb_lookup(cpu, pc, cs_base, flags, cflags);
 
+cflags = curr_cflags(cpu);
+/* Execute in a serial context. */
+cflags &= ~CF_PARALLEL;
+/* After 1 insn, return and release the exclusive lock. */
+cflags |= CF_NO_GOTO_TB | CF_NO_GOTO_PTR | 1;
+
+tb = tb_lookup(cpu, pc, cs_base, flags, cflags);
 if (tb == NULL) {
 mmap_lock();
 tb = tb_gen_code(cpu, pc, cs_base, flags, cflags);
-- 
2.25.1

[PULL 18/27] hw/core: Introduce TCGCPUOps.debug_check_breakpoint

[Richard Henderson]

New hook to return true when an architectural breakpoint is
to be recognized and false when it should be suppressed.

First use must wait until other pieces are in place.

Reviewed-by: Alex Bennée 
Reviewed-by: Philippe Mathieu-Daudé 
Signed-off-by: Richard Henderson 
---
 include/hw/core/tcg-cpu-ops.h | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/include/hw/core/tcg-cpu-ops.h b/include/hw/core/tcg-cpu-ops.h
index 72d791438c..eab27d0c03 100644
--- a/include/hw/core/tcg-cpu-ops.h
+++ b/include/hw/core/tcg-cpu-ops.h
@@ -88,6 +88,12 @@ struct TCGCPUOps {
  */
 bool (*debug_check_watchpoint)(CPUState *cpu, CPUWatchpoint *wp);
 
+/**
+ * @debug_check_breakpoint: return true if the architectural
+ * breakpoint whose PC has matched should really fire.
+ */
+bool (*debug_check_breakpoint)(CPUState *cpu);
+
 /**
  * @io_recompile_replay_branch: Callback for cpu_io_recompile.
  *
-- 
2.25.1

[PULL 15/27] accel/tcg: Drop CF_NO_GOTO_PTR from -d nochain

[Richard Henderson]

The purpose of suppressing goto_ptr from -d nochain had been
to return to the main loop so that -d cpu would be recognized.
But we now include -d cpu logging in helper_lookup_tb_ptr so
there is no need to exclude goto_ptr.

Tested-by: Mark Cave-Ayland 
Signed-off-by: Richard Henderson 
Reviewed-by: Peter Maydell 
Message-Id: <20210717221851.2124573-5-richard.hender...@linaro.org>
---
 accel/tcg/cpu-exec.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
index d3232d5764..70ea3c7d68 100644
--- a/accel/tcg/cpu-exec.c
+++ b/accel/tcg/cpu-exec.c
@@ -150,7 +150,7 @@ uint32_t curr_cflags(CPUState *cpu)
 uint32_t cflags = cpu->tcg_cflags;
 
 if (qemu_loglevel_mask(CPU_LOG_TB_NOCHAIN)) {
-cflags |= CF_NO_GOTO_TB | CF_NO_GOTO_PTR;
+cflags |= CF_NO_GOTO_TB;
 }
 
 return cflags;
-- 
2.25.1

[PULL 13/27] target/alpha: Drop goto_tb path in gen_call_pal

[Richard Henderson]

We are certain of a page crossing here, entering the
PALcode image, so the call to use_goto_tb that should
have been here will never succeed.

We are shortly going to add an assert to tcg_gen_goto_tb
that would trigger for this case.

Reviewed-by: Philippe Mathieu-Daudé 
Signed-off-by: Richard Henderson 
---
 target/alpha/translate.c | 15 ++-
 1 file changed, 2 insertions(+), 13 deletions(-)

diff --git a/target/alpha/translate.c b/target/alpha/translate.c
index 103c6326a2..949ba6ffde 100644
--- a/target/alpha/translate.c
+++ b/target/alpha/translate.c
@@ -1207,19 +1207,8 @@ static DisasJumpType gen_call_pal(DisasContext *ctx, int 
palcode)
   ? 0x2000 + (palcode - 0x80) * 64
   : 0x1000 + palcode * 64);
 
-/* Since the destination is running in PALmode, we don't really
-   need the page permissions check.  We'll see the existence of
-   the page when we create the TB, and we'll flush all TBs if
-   we change the PAL base register.  */
-if (!ctx->base.singlestep_enabled) {
-tcg_gen_goto_tb(0);
-tcg_gen_movi_i64(cpu_pc, entry);
-tcg_gen_exit_tb(ctx->base.tb, 0);
-return DISAS_NORETURN;
-} else {
-tcg_gen_movi_i64(cpu_pc, entry);
-return DISAS_PC_UPDATED;
-}
+tcg_gen_movi_i64(cpu_pc, entry);
+return DISAS_PC_UPDATED;
 }
 #endif
 }
-- 
2.25.1

[PULL 21/27] hw/core: Introduce CPUClass.gdb_adjust_breakpoint

[Richard Henderson]

This will allow a breakpoint hack to move out of AVR's translator.

Reviewed-by: Philippe Mathieu-Daudé 
Signed-off-by: Richard Henderson 
---
 include/hw/core/cpu.h |  4 
 cpu.c | 10 ++
 2 files changed, 14 insertions(+)

diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index 4e0ea68efc..bc864564ce 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -103,6 +103,9 @@ struct SysemuCPUOps;
  *   also implement the synchronize_from_tb hook.
  * @gdb_read_register: Callback for letting GDB read a register.
  * @gdb_write_register: Callback for letting GDB write a register.
+ * @gdb_adjust_breakpoint: Callback for adjusting the address of a
+ *   breakpoint.  Used by AVR to handle a gdb mis-feature with
+ *   its Harvard architecture split code and data.
  * @gdb_num_core_regs: Number of core registers accessible to GDB.
  * @gdb_core_xml_file: File name for core registers GDB XML description.
  * @gdb_stop_before_watchpoint: Indicates whether GDB expects the CPU to stop
@@ -137,6 +140,7 @@ struct CPUClass {
 void (*set_pc)(CPUState *cpu, vaddr value);
 int (*gdb_read_register)(CPUState *cpu, GByteArray *buf, int reg);
 int (*gdb_write_register)(CPUState *cpu, uint8_t *buf, int reg);
+vaddr (*gdb_adjust_breakpoint)(CPUState *cpu, vaddr addr);
 
 const char *gdb_core_xml_file;
 gchar * (*gdb_arch_name)(CPUState *cpu);
diff --git a/cpu.c b/cpu.c
index 83059537d7..91d9e38acb 100644
--- a/cpu.c
+++ b/cpu.c
@@ -267,8 +267,13 @@ static void breakpoint_invalidate(CPUState *cpu, 
target_ulong pc)
 int cpu_breakpoint_insert(CPUState *cpu, vaddr pc, int flags,
   CPUBreakpoint **breakpoint)
 {
+CPUClass *cc = CPU_GET_CLASS(cpu);
 CPUBreakpoint *bp;
 
+if (cc->gdb_adjust_breakpoint) {
+pc = cc->gdb_adjust_breakpoint(cpu, pc);
+}
+
 bp = g_malloc(sizeof(*bp));
 
 bp->pc = pc;
@@ -294,8 +299,13 @@ int cpu_breakpoint_insert(CPUState *cpu, vaddr pc, int 
flags,
 /* Remove a specific breakpoint.  */
 int cpu_breakpoint_remove(CPUState *cpu, vaddr pc, int flags)
 {
+CPUClass *cc = CPU_GET_CLASS(cpu);
 CPUBreakpoint *bp;
 
+if (cc->gdb_adjust_breakpoint) {
+pc = cc->gdb_adjust_breakpoint(cpu, pc);
+}
+
 QTAILQ_FOREACH(bp, &cpu->breakpoints, entry) {
 if (bp->pc == pc && bp->flags == flags) {
 cpu_breakpoint_remove_by_ref(cpu, bp);
-- 
2.25.1

[PULL 11/27] accel/tcg: Reduce CF_COUNT_MASK to match TCG_MAX_INSNS

[Richard Henderson]

The space reserved for CF_COUNT_MASK was overly large.
Reduce to free up cflags bits and eliminate an extra test.

Tested-by: Mark Cave-Ayland 
Signed-off-by: Richard Henderson 
Reviewed-by: Alex Bennée 
Reviewed-by: Peter Maydell 
Message-Id: <20210717221851.2124573-2-richard.hender...@linaro.org>
---
 include/exec/exec-all.h   | 4 +++-
 accel/tcg/translate-all.c | 5 ++---
 2 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
index 754f4130c9..dfe82ed19c 100644
--- a/include/exec/exec-all.h
+++ b/include/exec/exec-all.h
@@ -492,7 +492,9 @@ struct TranslationBlock {
 target_ulong cs_base; /* CS base for this block */
 uint32_t flags; /* flags defining in which context the code was generated 
*/
 uint32_t cflags;/* compile flags */
-#define CF_COUNT_MASK  0x7fff
+
+/* Note that TCG_MAX_INSNS is 512; we validate this match elsewhere. */
+#define CF_COUNT_MASK  0x01ff
 #define CF_LAST_IO 0x8000 /* Last insn may be an IO access.  */
 #define CF_MEMI_ONLY   0x0001 /* Only instrument memory ops */
 #define CF_USE_ICOUNT  0x0002
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index 4df26de858..5cc01d693b 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -1428,11 +1428,10 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
 
 max_insns = cflags & CF_COUNT_MASK;
 if (max_insns == 0) {
-max_insns = CF_COUNT_MASK;
-}
-if (max_insns > TCG_MAX_INSNS) {
 max_insns = TCG_MAX_INSNS;
 }
+QEMU_BUILD_BUG_ON(CF_COUNT_MASK + 1 != TCG_MAX_INSNS);
+
 if (cpu->singlestep_enabled || singlestep) {
 max_insns = 1;
 }
-- 
2.25.1

[PULL 19/27] target/arm: Implement debug_check_breakpoint

[Richard Henderson]

Reuse the code at the bottom of helper_check_breakpoints,
which is what we currently call from *_tr_breakpoint_check.

Reviewed-by: Alex Bennée 
Reviewed-by: Philippe Mathieu-Daudé 
Signed-off-by: Richard Henderson 
---
 target/arm/internals.h| 3 +++
 target/arm/cpu.c  | 1 +
 target/arm/cpu_tcg.c  | 1 +
 target/arm/debug_helper.c | 7 +++
 4 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/target/arm/internals.h b/target/arm/internals.h
index 3ba86e8af8..11a72013f5 100644
--- a/target/arm/internals.h
+++ b/target/arm/internals.h
@@ -282,6 +282,9 @@ void hw_breakpoint_update(ARMCPU *cpu, int n);
  */
 void hw_breakpoint_update_all(ARMCPU *cpu);
 
+/* Callback function for checking if a breakpoint should trigger. */
+bool arm_debug_check_breakpoint(CPUState *cs);
+
 /* Callback function for checking if a watchpoint should trigger. */
 bool arm_debug_check_watchpoint(CPUState *cs, CPUWatchpoint *wp);
 
diff --git a/target/arm/cpu.c b/target/arm/cpu.c
index 9cddfd6a44..752b15bb79 100644
--- a/target/arm/cpu.c
+++ b/target/arm/cpu.c
@@ -1984,6 +1984,7 @@ static const struct TCGCPUOps arm_tcg_ops = {
 .do_unaligned_access = arm_cpu_do_unaligned_access,
 .adjust_watchpoint_address = arm_adjust_watchpoint_address,
 .debug_check_watchpoint = arm_debug_check_watchpoint,
+.debug_check_breakpoint = arm_debug_check_breakpoint,
 #endif /* !CONFIG_USER_ONLY */
 };
 #endif /* CONFIG_TCG */
diff --git a/target/arm/cpu_tcg.c b/target/arm/cpu_tcg.c
index d2d97115ea..ed444bf436 100644
--- a/target/arm/cpu_tcg.c
+++ b/target/arm/cpu_tcg.c
@@ -911,6 +911,7 @@ static const struct TCGCPUOps arm_v7m_tcg_ops = {
 .do_unaligned_access = arm_cpu_do_unaligned_access,
 .adjust_watchpoint_address = arm_adjust_watchpoint_address,
 .debug_check_watchpoint = arm_debug_check_watchpoint,
+.debug_check_breakpoint = arm_debug_check_breakpoint,
 #endif /* !CONFIG_USER_ONLY */
 };
 #endif /* CONFIG_TCG */
diff --git a/target/arm/debug_helper.c b/target/arm/debug_helper.c
index 2ff72d47d1..4a0c479527 100644
--- a/target/arm/debug_helper.c
+++ b/target/arm/debug_helper.c
@@ -216,8 +216,9 @@ static bool check_watchpoints(ARMCPU *cpu)
 return false;
 }
 
-static bool check_breakpoints(ARMCPU *cpu)
+bool arm_debug_check_breakpoint(CPUState *cs)
 {
+ARMCPU *cpu = ARM_CPU(cs);
 CPUARMState *env = &cpu->env;
 int n;
 
@@ -240,9 +241,7 @@ static bool check_breakpoints(ARMCPU *cpu)
 
 void HELPER(check_breakpoints)(CPUARMState *env)
 {
-ARMCPU *cpu = env_archcpu(env);
-
-if (check_breakpoints(cpu)) {
+if (arm_debug_check_breakpoint(env_cpu(env))) {
 HELPER(exception_internal(env, EXCP_DEBUG));
 }
 }
-- 
2.25.1

[PULL 10/27] accel/tcg: Push trace info building into atomic_common.c.inc

[Richard Henderson]

Use trace_mem_get_info instead of trace_mem_build_info,
using the TCGMemOpIdx that we already have.  Do this in
the atomic_trace_*_pre function as common subroutines.

Tested-by: Cole Robinson 
Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 accel/tcg/atomic_template.h   | 48 +--
 accel/tcg/atomic_common.c.inc | 37 ++-
 2 files changed, 37 insertions(+), 48 deletions(-)

diff --git a/accel/tcg/atomic_template.h b/accel/tcg/atomic_template.h
index 6ee0158c5f..d89af4cc1e 100644
--- a/accel/tcg/atomic_template.h
+++ b/accel/tcg/atomic_template.h
@@ -77,10 +77,8 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, 
target_ulong addr,
 DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE,
  PAGE_READ | PAGE_WRITE, retaddr);
 DATA_TYPE ret;
-uint16_t info = trace_mem_build_info(SHIFT, false, 0, false,
- ATOMIC_MMU_IDX);
+uint16_t info = atomic_trace_rmw_pre(env, addr, oi);
 
-atomic_trace_rmw_pre(env, addr, info);
 #if DATA_SIZE == 16
 ret = atomic16_cmpxchg(haddr, cmpv, newv);
 #else
@@ -99,10 +97,8 @@ ABI_TYPE ATOMIC_NAME(ld)(CPUArchState *env, target_ulong 
addr,
 DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE,
  PAGE_READ, retaddr);
 DATA_TYPE val;
-uint16_t info = trace_mem_build_info(SHIFT, false, 0, false,
- ATOMIC_MMU_IDX);
+uint16_t info = atomic_trace_ld_pre(env, addr, oi);
 
-atomic_trace_ld_pre(env, addr, info);
 val = atomic16_read(haddr);
 ATOMIC_MMU_CLEANUP;
 atomic_trace_ld_post(env, addr, info);
@@ -114,10 +110,8 @@ void ATOMIC_NAME(st)(CPUArchState *env, target_ulong addr, 
ABI_TYPE val,
 {
 DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE,
  PAGE_WRITE, retaddr);
-uint16_t info = trace_mem_build_info(SHIFT, false, 0, true,
- ATOMIC_MMU_IDX);
+uint16_t info = atomic_trace_st_pre(env, addr, oi);
 
-atomic_trace_st_pre(env, addr, info);
 atomic16_set(haddr, val);
 ATOMIC_MMU_CLEANUP;
 atomic_trace_st_post(env, addr, info);
@@ -130,10 +124,8 @@ ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong 
addr, ABI_TYPE val,
 DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE,
  PAGE_READ | PAGE_WRITE, retaddr);
 DATA_TYPE ret;
-uint16_t info = trace_mem_build_info(SHIFT, false, 0, false,
- ATOMIC_MMU_IDX);
+uint16_t info = atomic_trace_rmw_pre(env, addr, oi);
 
-atomic_trace_rmw_pre(env, addr, info);
 ret = qatomic_xchg__nocheck(haddr, val);
 ATOMIC_MMU_CLEANUP;
 atomic_trace_rmw_post(env, addr, info);
@@ -147,9 +139,7 @@ ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, target_ulong 
addr,   \
 DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE,  \
  PAGE_READ | PAGE_WRITE, retaddr); \
 DATA_TYPE ret;  \
-uint16_t info = trace_mem_build_info(SHIFT, false, 0, false,\
- ATOMIC_MMU_IDX);   \
-atomic_trace_rmw_pre(env, addr, info);  \
+uint16_t info = atomic_trace_rmw_pre(env, addr, oi);\
 ret = qatomic_##X(haddr, val);  \
 ATOMIC_MMU_CLEANUP; \
 atomic_trace_rmw_post(env, addr, info); \
@@ -182,9 +172,7 @@ ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, target_ulong 
addr,   \
 XDATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, \
   PAGE_READ | PAGE_WRITE, retaddr); \
 XDATA_TYPE cmp, old, new, val = xval;   \
-uint16_t info = trace_mem_build_info(SHIFT, false, 0, false,\
- ATOMIC_MMU_IDX);   \
-atomic_trace_rmw_pre(env, addr, info);  \
+uint16_t info = atomic_trace_rmw_pre(env, addr, oi);\
 smp_mb();   \
 cmp = qatomic_read__nocheck(haddr); \
 do {\
@@ -228,10 +216,8 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, 
target_ulong addr,
 DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE,
  PAGE_READ | PAGE_WRITE, retaddr);
 DATA_TYPE ret;
-uint16_t info = trace_mem_build_info(SHIFT, false, MO_BSWAP, false,
- ATOMIC_MMU_IDX);
+uint16_t info = atomic_trace_rmw_pre(env, addr, oi);
 
-atomic_trace_rmw_pre(

[PULL 06/27] accel/tcg: Fold EXTRA_ARGS into atomic_template.h

[Richard Henderson]

All instances of EXTRA_ARGS are now identical.

Tested-by: Cole Robinson 
Reviewed-by: Peter Maydell 
Reviewed-by: Philippe Mathieu-Daudé 
Signed-off-by: Richard Henderson 
---
 accel/tcg/atomic_template.h | 36 
 accel/tcg/cputlb.c  |  1 -
 accel/tcg/user-exec.c   |  1 -
 3 files changed, 20 insertions(+), 18 deletions(-)

diff --git a/accel/tcg/atomic_template.h b/accel/tcg/atomic_template.h
index d347462af5..52fb26a274 100644
--- a/accel/tcg/atomic_template.h
+++ b/accel/tcg/atomic_template.h
@@ -71,7 +71,8 @@
 #endif
 
 ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, target_ulong addr,
-  ABI_TYPE cmpv, ABI_TYPE newv EXTRA_ARGS)
+  ABI_TYPE cmpv, ABI_TYPE newv,
+  TCGMemOpIdx oi, uintptr_t retaddr)
 {
 ATOMIC_MMU_DECLS;
 DATA_TYPE *haddr = ATOMIC_MMU_LOOKUP_RW;
@@ -92,7 +93,8 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, target_ulong 
addr,
 
 #if DATA_SIZE >= 16
 #if HAVE_ATOMIC128
-ABI_TYPE ATOMIC_NAME(ld)(CPUArchState *env, target_ulong addr EXTRA_ARGS)
+ABI_TYPE ATOMIC_NAME(ld)(CPUArchState *env, target_ulong addr,
+ TCGMemOpIdx oi, uintptr_t retaddr)
 {
 ATOMIC_MMU_DECLS;
 DATA_TYPE val, *haddr = ATOMIC_MMU_LOOKUP_R;
@@ -106,8 +108,8 @@ ABI_TYPE ATOMIC_NAME(ld)(CPUArchState *env, target_ulong 
addr EXTRA_ARGS)
 return val;
 }
 
-void ATOMIC_NAME(st)(CPUArchState *env, target_ulong addr,
- ABI_TYPE val EXTRA_ARGS)
+void ATOMIC_NAME(st)(CPUArchState *env, target_ulong addr, ABI_TYPE val,
+ TCGMemOpIdx oi, uintptr_t retaddr)
 {
 ATOMIC_MMU_DECLS;
 DATA_TYPE *haddr = ATOMIC_MMU_LOOKUP_W;
@@ -121,8 +123,8 @@ void ATOMIC_NAME(st)(CPUArchState *env, target_ulong addr,
 }
 #endif
 #else
-ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong addr,
-   ABI_TYPE val EXTRA_ARGS)
+ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong addr, ABI_TYPE val,
+   TCGMemOpIdx oi, uintptr_t retaddr)
 {
 ATOMIC_MMU_DECLS;
 DATA_TYPE *haddr = ATOMIC_MMU_LOOKUP_RW;
@@ -139,7 +141,7 @@ ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong 
addr,
 
 #define GEN_ATOMIC_HELPER(X)\
 ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, target_ulong addr,   \
-ABI_TYPE val EXTRA_ARGS)\
+ABI_TYPE val, TCGMemOpIdx oi, uintptr_t retaddr) \
 {   \
 ATOMIC_MMU_DECLS;   \
 DATA_TYPE *haddr = ATOMIC_MMU_LOOKUP_RW;\
@@ -173,7 +175,7 @@ GEN_ATOMIC_HELPER(xor_fetch)
  */
 #define GEN_ATOMIC_HELPER_FN(X, FN, XDATA_TYPE, RET)\
 ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, target_ulong addr,   \
-ABI_TYPE xval EXTRA_ARGS)   \
+ABI_TYPE xval, TCGMemOpIdx oi, uintptr_t retaddr) \
 {   \
 ATOMIC_MMU_DECLS;   \
 XDATA_TYPE *haddr = ATOMIC_MMU_LOOKUP_RW;   \
@@ -218,7 +220,8 @@ GEN_ATOMIC_HELPER_FN(umax_fetch, MAX,  DATA_TYPE, new)
 #endif
 
 ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, target_ulong addr,
-  ABI_TYPE cmpv, ABI_TYPE newv EXTRA_ARGS)
+  ABI_TYPE cmpv, ABI_TYPE newv,
+  TCGMemOpIdx oi, uintptr_t retaddr)
 {
 ATOMIC_MMU_DECLS;
 DATA_TYPE *haddr = ATOMIC_MMU_LOOKUP_RW;
@@ -239,7 +242,8 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, 
target_ulong addr,
 
 #if DATA_SIZE >= 16
 #if HAVE_ATOMIC128
-ABI_TYPE ATOMIC_NAME(ld)(CPUArchState *env, target_ulong addr EXTRA_ARGS)
+ABI_TYPE ATOMIC_NAME(ld)(CPUArchState *env, target_ulong addr,
+ TCGMemOpIdx oi, uintptr_t retaddr)
 {
 ATOMIC_MMU_DECLS;
 DATA_TYPE val, *haddr = ATOMIC_MMU_LOOKUP_R;
@@ -253,8 +257,8 @@ ABI_TYPE ATOMIC_NAME(ld)(CPUArchState *env, target_ulong 
addr EXTRA_ARGS)
 return BSWAP(val);
 }
 
-void ATOMIC_NAME(st)(CPUArchState *env, target_ulong addr,
- ABI_TYPE val EXTRA_ARGS)
+void ATOMIC_NAME(st)(CPUArchState *env, target_ulong addr, ABI_TYPE val,
+ TCGMemOpIdx oi, uintptr_t retaddr)
 {
 ATOMIC_MMU_DECLS;
 DATA_TYPE *haddr = ATOMIC_MMU_LOOKUP_W;
@@ -270,8 +274,8 @@ void ATOMIC_NAME(st)(CPUArchState *env, target_ulong addr,
 }
 #endif
 #else
-ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong addr,
-   ABI_TYPE val EXTRA_ARGS)
+ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong addr, ABI_TYPE val,
+   TCGMemOpIdx oi, uintptr_t retaddr)
 {
 ATOMIC_MMU_DECLS;
 DATA_TYPE *haddr 

[PULL 09/27] trace: Fold mem-internal.h into mem.h

[Richard Henderson]

Since the last thing that mem.h does is include mem-internal.h,
the symbols are not actually private.

Tested-by: Cole Robinson 
Reviewed-by: Peter Maydell 
Reviewed-by: Philippe Mathieu-Daudé 
Signed-off-by: Richard Henderson 
---
 trace/mem-internal.h | 50 
 trace/mem.h  | 50 ++--
 plugins/core.c   |  2 +-
 3 files changed, 40 insertions(+), 62 deletions(-)
 delete mode 100644 trace/mem-internal.h

diff --git a/trace/mem-internal.h b/trace/mem-internal.h
deleted file mode 100644
index 8b72b678fa..00
--- a/trace/mem-internal.h
+++ /dev/null
@@ -1,50 +0,0 @@
-/*
- * Helper functions for guest memory tracing
- *
- * Copyright (C) 2016 Lluís Vilanova 
- *
- * This work is licensed under the terms of the GNU GPL, version 2 or later.
- * See the COPYING file in the top-level directory.
- */
-
-#ifndef TRACE__MEM_INTERNAL_H
-#define TRACE__MEM_INTERNAL_H
-
-#define TRACE_MEM_SZ_SHIFT_MASK 0xf /* size shift mask */
-#define TRACE_MEM_SE (1ULL << 4)/* sign extended (y/n) */
-#define TRACE_MEM_BE (1ULL << 5)/* big endian (y/n) */
-#define TRACE_MEM_ST (1ULL << 6)/* store (y/n) */
-#define TRACE_MEM_MMU_SHIFT 8   /* mmu idx */
-
-static inline uint16_t trace_mem_build_info(
-int size_shift, bool sign_extend, MemOp endianness,
-bool store, unsigned int mmu_idx)
-{
-uint16_t res;
-
-res = size_shift & TRACE_MEM_SZ_SHIFT_MASK;
-if (sign_extend) {
-res |= TRACE_MEM_SE;
-}
-if (endianness == MO_BE) {
-res |= TRACE_MEM_BE;
-}
-if (store) {
-res |= TRACE_MEM_ST;
-}
-#ifdef CONFIG_SOFTMMU
-res |= mmu_idx << TRACE_MEM_MMU_SHIFT;
-#endif
-return res;
-}
-
-static inline uint16_t trace_mem_get_info(MemOp op,
-  unsigned int mmu_idx,
-  bool store)
-{
-return trace_mem_build_info(op & MO_SIZE, !!(op & MO_SIGN),
-op & MO_BSWAP, store,
-mmu_idx);
-}
-
-#endif /* TRACE__MEM_INTERNAL_H */
diff --git a/trace/mem.h b/trace/mem.h
index 9644f592b4..2f27e7bdf0 100644
--- a/trace/mem.h
+++ b/trace/mem.h
@@ -12,24 +12,52 @@
 
 #include "tcg/tcg.h"
 
-
-/**
- * trace_mem_get_info:
- *
- * Return a value for the 'info' argument in guest memory access traces.
- */
-static uint16_t trace_mem_get_info(MemOp op, unsigned int mmu_idx, bool store);
+#define TRACE_MEM_SZ_SHIFT_MASK 0xf /* size shift mask */
+#define TRACE_MEM_SE (1ULL << 4)/* sign extended (y/n) */
+#define TRACE_MEM_BE (1ULL << 5)/* big endian (y/n) */
+#define TRACE_MEM_ST (1ULL << 6)/* store (y/n) */
+#define TRACE_MEM_MMU_SHIFT 8   /* mmu idx */
 
 /**
  * trace_mem_build_info:
  *
  * Return a value for the 'info' argument in guest memory access traces.
  */
-static uint16_t trace_mem_build_info(int size_shift, bool sign_extend,
- MemOp endianness, bool store,
- unsigned int mmuidx);
+static inline uint16_t trace_mem_build_info(int size_shift, bool sign_extend,
+MemOp endianness, bool store,
+unsigned int mmu_idx)
+{
+uint16_t res;
+
+res = size_shift & TRACE_MEM_SZ_SHIFT_MASK;
+if (sign_extend) {
+res |= TRACE_MEM_SE;
+}
+if (endianness == MO_BE) {
+res |= TRACE_MEM_BE;
+}
+if (store) {
+res |= TRACE_MEM_ST;
+}
+#ifdef CONFIG_SOFTMMU
+res |= mmu_idx << TRACE_MEM_MMU_SHIFT;
+#endif
+return res;
+}
 
 
-#include "trace/mem-internal.h"
+/**
+ * trace_mem_get_info:
+ *
+ * Return a value for the 'info' argument in guest memory access traces.
+ */
+static inline uint16_t trace_mem_get_info(MemOp op,
+  unsigned int mmu_idx,
+  bool store)
+{
+return trace_mem_build_info(op & MO_SIZE, !!(op & MO_SIGN),
+op & MO_BSWAP, store,
+mmu_idx);
+}
 
 #endif /* TRACE__MEM_H */
diff --git a/plugins/core.c b/plugins/core.c
index e1bcdb570d..474db287cb 100644
--- a/plugins/core.c
+++ b/plugins/core.c
@@ -27,7 +27,7 @@
 #include "exec/helper-proto.h"
 #include "tcg/tcg.h"
 #include "tcg/tcg-op.h"
-#include "trace/mem-internal.h" /* mem_info macros */
+#include "trace/mem.h" /* mem_info macros */
 #include "plugin.h"
 #include "qemu/compiler.h"
 
-- 
2.25.1

[PULL 08/27] accel/tcg: Expand ATOMIC_MMU_LOOKUP_*

[Richard Henderson]

Unify the parameters of atomic_mmu_lookup between cputlb.c and
user-exec.c.  Call the function directly, and remove the macros.

Tested-by: Cole Robinson 
Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 accel/tcg/atomic_template.h | 41 +
 accel/tcg/cputlb.c  |  7 +--
 accel/tcg/user-exec.c   | 12 ++-
 3 files changed, 36 insertions(+), 24 deletions(-)

diff --git a/accel/tcg/atomic_template.h b/accel/tcg/atomic_template.h
index ae6b6a03be..6ee0158c5f 100644
--- a/accel/tcg/atomic_template.h
+++ b/accel/tcg/atomic_template.h
@@ -74,7 +74,8 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, target_ulong 
addr,
   ABI_TYPE cmpv, ABI_TYPE newv,
   TCGMemOpIdx oi, uintptr_t retaddr)
 {
-DATA_TYPE *haddr = ATOMIC_MMU_LOOKUP_RW;
+DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE,
+ PAGE_READ | PAGE_WRITE, retaddr);
 DATA_TYPE ret;
 uint16_t info = trace_mem_build_info(SHIFT, false, 0, false,
  ATOMIC_MMU_IDX);
@@ -95,7 +96,9 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, target_ulong 
addr,
 ABI_TYPE ATOMIC_NAME(ld)(CPUArchState *env, target_ulong addr,
  TCGMemOpIdx oi, uintptr_t retaddr)
 {
-DATA_TYPE val, *haddr = ATOMIC_MMU_LOOKUP_R;
+DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE,
+ PAGE_READ, retaddr);
+DATA_TYPE val;
 uint16_t info = trace_mem_build_info(SHIFT, false, 0, false,
  ATOMIC_MMU_IDX);
 
@@ -109,7 +112,8 @@ ABI_TYPE ATOMIC_NAME(ld)(CPUArchState *env, target_ulong 
addr,
 void ATOMIC_NAME(st)(CPUArchState *env, target_ulong addr, ABI_TYPE val,
  TCGMemOpIdx oi, uintptr_t retaddr)
 {
-DATA_TYPE *haddr = ATOMIC_MMU_LOOKUP_W;
+DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE,
+ PAGE_WRITE, retaddr);
 uint16_t info = trace_mem_build_info(SHIFT, false, 0, true,
  ATOMIC_MMU_IDX);
 
@@ -123,7 +127,8 @@ void ATOMIC_NAME(st)(CPUArchState *env, target_ulong addr, 
ABI_TYPE val,
 ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong addr, ABI_TYPE val,
TCGMemOpIdx oi, uintptr_t retaddr)
 {
-DATA_TYPE *haddr = ATOMIC_MMU_LOOKUP_RW;
+DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE,
+ PAGE_READ | PAGE_WRITE, retaddr);
 DATA_TYPE ret;
 uint16_t info = trace_mem_build_info(SHIFT, false, 0, false,
  ATOMIC_MMU_IDX);
@@ -139,7 +144,8 @@ ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong 
addr, ABI_TYPE val,
 ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, target_ulong addr,   \
 ABI_TYPE val, TCGMemOpIdx oi, uintptr_t retaddr) \
 {   \
-DATA_TYPE *haddr = ATOMIC_MMU_LOOKUP_RW;\
+DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE,  \
+ PAGE_READ | PAGE_WRITE, retaddr); \
 DATA_TYPE ret;  \
 uint16_t info = trace_mem_build_info(SHIFT, false, 0, false,\
  ATOMIC_MMU_IDX);   \
@@ -161,7 +167,8 @@ GEN_ATOMIC_HELPER(xor_fetch)
 
 #undef GEN_ATOMIC_HELPER
 
-/* These helpers are, as a whole, full barriers.  Within the helper,
+/*
+ * These helpers are, as a whole, full barriers.  Within the helper,
  * the leading barrier is explicit and the trailing barrier is within
  * cmpxchg primitive.
  *
@@ -172,7 +179,8 @@ GEN_ATOMIC_HELPER(xor_fetch)
 ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, target_ulong addr,   \
 ABI_TYPE xval, TCGMemOpIdx oi, uintptr_t retaddr) \
 {   \
-XDATA_TYPE *haddr = ATOMIC_MMU_LOOKUP_RW;   \
+XDATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, \
+  PAGE_READ | PAGE_WRITE, retaddr); \
 XDATA_TYPE cmp, old, new, val = xval;   \
 uint16_t info = trace_mem_build_info(SHIFT, false, 0, false,\
  ATOMIC_MMU_IDX);   \
@@ -217,7 +225,8 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, 
target_ulong addr,
   ABI_TYPE cmpv, ABI_TYPE newv,
   TCGMemOpIdx oi, uintptr_t retaddr)
 {
-DATA_TYPE *haddr = ATOMIC_MMU_LOOKUP_RW;
+DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE,
+ PAGE_READ | PAGE_WRITE, retaddr);
 DATA_TYPE r

[PULL 05/27] accel/tcg: Standardize atomic helpers on softmmu api

[Richard Henderson]

Reduce the amount of code duplication by always passing
the TCGMemOpIdx argument to helper_atomic_*.  This is not
currently used for user-only, but it's easy to ignore.

Tested-by: Cole Robinson 
Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 accel/tcg/tcg-runtime.h   | 46 ---
 accel/tcg/cputlb.c| 32 
 accel/tcg/user-exec.c | 26 -
 tcg/tcg-op.c  | 51 ++---
 accel/tcg/atomic_common.c.inc | 70 +++
 5 files changed, 82 insertions(+), 143 deletions(-)

diff --git a/accel/tcg/tcg-runtime.h b/accel/tcg/tcg-runtime.h
index 91a5b7e85f..37cbd722bf 100644
--- a/accel/tcg/tcg-runtime.h
+++ b/accel/tcg/tcg-runtime.h
@@ -39,8 +39,6 @@ DEF_HELPER_FLAGS_1(exit_atomic, TCG_CALL_NO_WG, noreturn, env)
 DEF_HELPER_FLAGS_3(memset, TCG_CALL_NO_RWG, ptr, ptr, int, ptr)
 #endif /* IN_HELPER_PROTO */
 
-#ifdef CONFIG_SOFTMMU
-
 DEF_HELPER_FLAGS_5(atomic_cmpxchgb, TCG_CALL_NO_WG,
i32, env, tl, i32, i32, i32)
 DEF_HELPER_FLAGS_5(atomic_cmpxchgw_be, TCG_CALL_NO_WG,
@@ -88,50 +86,6 @@ DEF_HELPER_FLAGS_5(atomic_cmpxchgq_le, TCG_CALL_NO_WG,
TCG_CALL_NO_WG, i32, env, tl, i32, i32)
 #endif /* CONFIG_ATOMIC64 */
 
-#else
-
-DEF_HELPER_FLAGS_4(atomic_cmpxchgb, TCG_CALL_NO_WG, i32, env, tl, i32, i32)
-DEF_HELPER_FLAGS_4(atomic_cmpxchgw_be, TCG_CALL_NO_WG, i32, env, tl, i32, i32)
-DEF_HELPER_FLAGS_4(atomic_cmpxchgw_le, TCG_CALL_NO_WG, i32, env, tl, i32, i32)
-DEF_HELPER_FLAGS_4(atomic_cmpxchgl_be, TCG_CALL_NO_WG, i32, env, tl, i32, i32)
-DEF_HELPER_FLAGS_4(atomic_cmpxchgl_le, TCG_CALL_NO_WG, i32, env, tl, i32, i32)
-#ifdef CONFIG_ATOMIC64
-DEF_HELPER_FLAGS_4(atomic_cmpxchgq_be, TCG_CALL_NO_WG, i64, env, tl, i64, i64)
-DEF_HELPER_FLAGS_4(atomic_cmpxchgq_le, TCG_CALL_NO_WG, i64, env, tl, i64, i64)
-#endif
-
-#ifdef CONFIG_ATOMIC64
-#define GEN_ATOMIC_HELPERS(NAME) \
-DEF_HELPER_FLAGS_3(glue(glue(atomic_, NAME), b), \
-   TCG_CALL_NO_WG, i32, env, tl, i32)\
-DEF_HELPER_FLAGS_3(glue(glue(atomic_, NAME), w_le),  \
-   TCG_CALL_NO_WG, i32, env, tl, i32)\
-DEF_HELPER_FLAGS_3(glue(glue(atomic_, NAME), w_be),  \
-   TCG_CALL_NO_WG, i32, env, tl, i32)\
-DEF_HELPER_FLAGS_3(glue(glue(atomic_, NAME), l_le),  \
-   TCG_CALL_NO_WG, i32, env, tl, i32)\
-DEF_HELPER_FLAGS_3(glue(glue(atomic_, NAME), l_be),  \
-   TCG_CALL_NO_WG, i32, env, tl, i32)\
-DEF_HELPER_FLAGS_3(glue(glue(atomic_, NAME), q_le),  \
-   TCG_CALL_NO_WG, i64, env, tl, i64)\
-DEF_HELPER_FLAGS_3(glue(glue(atomic_, NAME), q_be),  \
-   TCG_CALL_NO_WG, i64, env, tl, i64)
-#else
-#define GEN_ATOMIC_HELPERS(NAME) \
-DEF_HELPER_FLAGS_3(glue(glue(atomic_, NAME), b), \
-   TCG_CALL_NO_WG, i32, env, tl, i32)\
-DEF_HELPER_FLAGS_3(glue(glue(atomic_, NAME), w_le),  \
-   TCG_CALL_NO_WG, i32, env, tl, i32)\
-DEF_HELPER_FLAGS_3(glue(glue(atomic_, NAME), w_be),  \
-   TCG_CALL_NO_WG, i32, env, tl, i32)\
-DEF_HELPER_FLAGS_3(glue(glue(atomic_, NAME), l_le),  \
-   TCG_CALL_NO_WG, i32, env, tl, i32)\
-DEF_HELPER_FLAGS_3(glue(glue(atomic_, NAME), l_be),  \
-   TCG_CALL_NO_WG, i32, env, tl, i32)
-#endif /* CONFIG_ATOMIC64 */
-
-#endif /* CONFIG_SOFTMMU */
-
 GEN_ATOMIC_HELPERS(fetch_add)
 GEN_ATOMIC_HELPERS(fetch_and)
 GEN_ATOMIC_HELPERS(fetch_or)
diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index 63da1cc96f..842cf4b572 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -2725,38 +2725,6 @@ void cpu_stq_le_data(CPUArchState *env, target_ulong 
ptr, uint64_t val)
 #include "atomic_template.h"
 #endif
 
-/* Second set of helpers are directly callable from TCG as helpers.  */
-
-#undef EXTRA_ARGS
-#undef ATOMIC_NAME
-#undef ATOMIC_MMU_LOOKUP_RW
-#undef ATOMIC_MMU_LOOKUP_R
-#undef ATOMIC_MMU_LOOKUP_W
-
-#define EXTRA_ARGS , TCGMemOpIdx oi
-#define ATOMIC_NAME(X) HELPER(glue(glue(atomic_ ## X, SUFFIX), END))
-#define ATOMIC_MMU_LOOKUP_RW \
-atomic_mmu_lookup(env, addr, oi, DATA_SIZE, PAGE_READ | PAGE_WRITE, 
GETPC())
-#define ATOMIC_MMU_LOOKUP_R \
-atomic_mmu_lookup(env, addr, oi, DATA_SIZE, PAGE_READ, GETPC())
-#define ATOMIC_MMU_LOOKUP_W \
-atomic_mmu_lookup(env, addr, oi, DATA_SIZE, PAGE_WRITE, GETPC())
-
-#define DATA_SIZE 1
-#include "atomic_template.h"
-
-#define DATA_SIZE 2
-#include "atomic_template.h"
-
-#define DATA_SIZE 4
-#include "atomic_template.h"
-
-#ifdef CONFIG_ATOMIC64
-#define DATA_SIZE 8
-#include "atomic_template.h"
-#endif
-#undef ATOMIC_MMU_IDX
-
 /* Code access functions.  */
 
 static uint64_t full_ldub_code(CPUArchStat

[PULL 07/27] accel/tcg: Remove ATOMIC_MMU_DECLS

[Richard Henderson]

All definitions are now empty.

Tested-by: Cole Robinson 
Reviewed-by: Peter Maydell 
Reviewed-by: Philippe Mathieu-Daudé 
Signed-off-by: Richard Henderson 
---
 accel/tcg/atomic_template.h | 12 
 accel/tcg/cputlb.c  |  1 -
 accel/tcg/user-exec.c   |  1 -
 3 files changed, 14 deletions(-)

diff --git a/accel/tcg/atomic_template.h b/accel/tcg/atomic_template.h
index 52fb26a274..ae6b6a03be 100644
--- a/accel/tcg/atomic_template.h
+++ b/accel/tcg/atomic_template.h
@@ -74,7 +74,6 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, target_ulong 
addr,
   ABI_TYPE cmpv, ABI_TYPE newv,
   TCGMemOpIdx oi, uintptr_t retaddr)
 {
-ATOMIC_MMU_DECLS;
 DATA_TYPE *haddr = ATOMIC_MMU_LOOKUP_RW;
 DATA_TYPE ret;
 uint16_t info = trace_mem_build_info(SHIFT, false, 0, false,
@@ -96,7 +95,6 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, target_ulong 
addr,
 ABI_TYPE ATOMIC_NAME(ld)(CPUArchState *env, target_ulong addr,
  TCGMemOpIdx oi, uintptr_t retaddr)
 {
-ATOMIC_MMU_DECLS;
 DATA_TYPE val, *haddr = ATOMIC_MMU_LOOKUP_R;
 uint16_t info = trace_mem_build_info(SHIFT, false, 0, false,
  ATOMIC_MMU_IDX);
@@ -111,7 +109,6 @@ ABI_TYPE ATOMIC_NAME(ld)(CPUArchState *env, target_ulong 
addr,
 void ATOMIC_NAME(st)(CPUArchState *env, target_ulong addr, ABI_TYPE val,
  TCGMemOpIdx oi, uintptr_t retaddr)
 {
-ATOMIC_MMU_DECLS;
 DATA_TYPE *haddr = ATOMIC_MMU_LOOKUP_W;
 uint16_t info = trace_mem_build_info(SHIFT, false, 0, true,
  ATOMIC_MMU_IDX);
@@ -126,7 +123,6 @@ void ATOMIC_NAME(st)(CPUArchState *env, target_ulong addr, 
ABI_TYPE val,
 ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong addr, ABI_TYPE val,
TCGMemOpIdx oi, uintptr_t retaddr)
 {
-ATOMIC_MMU_DECLS;
 DATA_TYPE *haddr = ATOMIC_MMU_LOOKUP_RW;
 DATA_TYPE ret;
 uint16_t info = trace_mem_build_info(SHIFT, false, 0, false,
@@ -143,7 +139,6 @@ ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong 
addr, ABI_TYPE val,
 ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, target_ulong addr,   \
 ABI_TYPE val, TCGMemOpIdx oi, uintptr_t retaddr) \
 {   \
-ATOMIC_MMU_DECLS;   \
 DATA_TYPE *haddr = ATOMIC_MMU_LOOKUP_RW;\
 DATA_TYPE ret;  \
 uint16_t info = trace_mem_build_info(SHIFT, false, 0, false,\
@@ -177,7 +172,6 @@ GEN_ATOMIC_HELPER(xor_fetch)
 ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, target_ulong addr,   \
 ABI_TYPE xval, TCGMemOpIdx oi, uintptr_t retaddr) \
 {   \
-ATOMIC_MMU_DECLS;   \
 XDATA_TYPE *haddr = ATOMIC_MMU_LOOKUP_RW;   \
 XDATA_TYPE cmp, old, new, val = xval;   \
 uint16_t info = trace_mem_build_info(SHIFT, false, 0, false,\
@@ -223,7 +217,6 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, 
target_ulong addr,
   ABI_TYPE cmpv, ABI_TYPE newv,
   TCGMemOpIdx oi, uintptr_t retaddr)
 {
-ATOMIC_MMU_DECLS;
 DATA_TYPE *haddr = ATOMIC_MMU_LOOKUP_RW;
 DATA_TYPE ret;
 uint16_t info = trace_mem_build_info(SHIFT, false, MO_BSWAP, false,
@@ -245,7 +238,6 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, 
target_ulong addr,
 ABI_TYPE ATOMIC_NAME(ld)(CPUArchState *env, target_ulong addr,
  TCGMemOpIdx oi, uintptr_t retaddr)
 {
-ATOMIC_MMU_DECLS;
 DATA_TYPE val, *haddr = ATOMIC_MMU_LOOKUP_R;
 uint16_t info = trace_mem_build_info(SHIFT, false, MO_BSWAP, false,
  ATOMIC_MMU_IDX);
@@ -260,7 +252,6 @@ ABI_TYPE ATOMIC_NAME(ld)(CPUArchState *env, target_ulong 
addr,
 void ATOMIC_NAME(st)(CPUArchState *env, target_ulong addr, ABI_TYPE val,
  TCGMemOpIdx oi, uintptr_t retaddr)
 {
-ATOMIC_MMU_DECLS;
 DATA_TYPE *haddr = ATOMIC_MMU_LOOKUP_W;
 uint16_t info = trace_mem_build_info(SHIFT, false, MO_BSWAP, true,
  ATOMIC_MMU_IDX);
@@ -277,7 +268,6 @@ void ATOMIC_NAME(st)(CPUArchState *env, target_ulong addr, 
ABI_TYPE val,
 ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong addr, ABI_TYPE val,
TCGMemOpIdx oi, uintptr_t retaddr)
 {
-ATOMIC_MMU_DECLS;
 DATA_TYPE *haddr = ATOMIC_MMU_LOOKUP_RW;
 ABI_TYPE ret;
 uint16_t info = trace_mem_build_info(SHIFT, false, MO_BSWAP, false,
@@ -294,7 +284,6 @@ ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong 
addr, ABI_TYPE val,
 ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, 

[PULL 04/27] tcg: Rename helper_atomic_*_mmu and provide for user-only

[Richard Henderson]

Always provide the atomic interface using TCGMemOpIdx oi
and uintptr_t retaddr.  Rename from helper_* to cpu_* so
as to (mostly) match the exec/cpu_ldst.h functions, and
to emphasize that they are not callable from TCG directly.

Tested-by: Cole Robinson 
Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 include/tcg/tcg.h | 78 ---
 accel/tcg/cputlb.c|  8 ++--
 accel/tcg/user-exec.c | 59 --
 target/arm/helper-a64.c   |  8 ++--
 target/i386/tcg/mem_helper.c  | 15 +--
 target/m68k/op_helper.c   | 19 +++--
 target/ppc/mem_helper.c   | 16 +++
 target/s390x/tcg/mem_helper.c | 19 -
 8 files changed, 104 insertions(+), 118 deletions(-)

diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h
index 25dd19d6e1..44ccd86f3e 100644
--- a/include/tcg/tcg.h
+++ b/include/tcg/tcg.h
@@ -1341,31 +1341,32 @@ void helper_be_stq_mmu(CPUArchState *env, target_ulong 
addr, uint64_t val,
 # define helper_ret_stl_mmu   helper_le_stl_mmu
 # define helper_ret_stq_mmu   helper_le_stq_mmu
 #endif
+#endif /* CONFIG_SOFTMMU */
 
-uint32_t helper_atomic_cmpxchgb_mmu(CPUArchState *env, target_ulong addr,
+uint32_t cpu_atomic_cmpxchgb_mmu(CPUArchState *env, target_ulong addr,
+ uint32_t cmpv, uint32_t newv,
+ TCGMemOpIdx oi, uintptr_t retaddr);
+uint32_t cpu_atomic_cmpxchgw_le_mmu(CPUArchState *env, target_ulong addr,
 uint32_t cmpv, uint32_t newv,
 TCGMemOpIdx oi, uintptr_t retaddr);
-uint32_t helper_atomic_cmpxchgw_le_mmu(CPUArchState *env, target_ulong addr,
-   uint32_t cmpv, uint32_t newv,
-   TCGMemOpIdx oi, uintptr_t retaddr);
-uint32_t helper_atomic_cmpxchgl_le_mmu(CPUArchState *env, target_ulong addr,
-   uint32_t cmpv, uint32_t newv,
-   TCGMemOpIdx oi, uintptr_t retaddr);
-uint64_t helper_atomic_cmpxchgq_le_mmu(CPUArchState *env, target_ulong addr,
-   uint64_t cmpv, uint64_t newv,
-   TCGMemOpIdx oi, uintptr_t retaddr);
-uint32_t helper_atomic_cmpxchgw_be_mmu(CPUArchState *env, target_ulong addr,
-   uint32_t cmpv, uint32_t newv,
-   TCGMemOpIdx oi, uintptr_t retaddr);
-uint32_t helper_atomic_cmpxchgl_be_mmu(CPUArchState *env, target_ulong addr,
-   uint32_t cmpv, uint32_t newv,
-   TCGMemOpIdx oi, uintptr_t retaddr);
-uint64_t helper_atomic_cmpxchgq_be_mmu(CPUArchState *env, target_ulong addr,
-   uint64_t cmpv, uint64_t newv,
-   TCGMemOpIdx oi, uintptr_t retaddr);
+uint32_t cpu_atomic_cmpxchgl_le_mmu(CPUArchState *env, target_ulong addr,
+uint32_t cmpv, uint32_t newv,
+TCGMemOpIdx oi, uintptr_t retaddr);
+uint64_t cpu_atomic_cmpxchgq_le_mmu(CPUArchState *env, target_ulong addr,
+uint64_t cmpv, uint64_t newv,
+TCGMemOpIdx oi, uintptr_t retaddr);
+uint32_t cpu_atomic_cmpxchgw_be_mmu(CPUArchState *env, target_ulong addr,
+uint32_t cmpv, uint32_t newv,
+TCGMemOpIdx oi, uintptr_t retaddr);
+uint32_t cpu_atomic_cmpxchgl_be_mmu(CPUArchState *env, target_ulong addr,
+uint32_t cmpv, uint32_t newv,
+TCGMemOpIdx oi, uintptr_t retaddr);
+uint64_t cpu_atomic_cmpxchgq_be_mmu(CPUArchState *env, target_ulong addr,
+uint64_t cmpv, uint64_t newv,
+TCGMemOpIdx oi, uintptr_t retaddr);
 
 #define GEN_ATOMIC_HELPER(NAME, TYPE, SUFFIX) \
-TYPE helper_atomic_ ## NAME ## SUFFIX ## _mmu \
+TYPE cpu_atomic_ ## NAME ## SUFFIX ## _mmu\
 (CPUArchState *env, target_ulong addr, TYPE val,  \
  TCGMemOpIdx oi, uintptr_t retaddr);
 
@@ -1411,31 +1412,22 @@ GEN_ATOMIC_HELPER_ALL(xchg)
 
 #undef GEN_ATOMIC_HELPER_ALL
 #undef GEN_ATOMIC_HELPER
-#endif /* CONFIG_SOFTMMU */
 
-/*
- * These aren't really a "proper" helpers because TCG cannot manage Int128.
- * However, use the same format as the others, for use by the backends.
- *
- * The cmpxchg functions are only defined if HAVE_CMPXCHG128;
- * the ld/st functions are only defined if HAVE_ATOMIC128,
- * as defined by .
- */
-Int128 helper_atomic_cmpxchgo_le_mmu(CPUArchState *env, target_ulong addr,
- Int128 cmpv, Int128 newv,
- TCGMemOpIdx oi, uintptr_t retaddr);
-In

[PULL 02/27] qemu/atomic: Remove pre-C11 atomic fallbacks

[Richard Henderson]

We now require c11, so the fallbacks are now dead code

Tested-by: Cole Robinson 
Reviewed-by: Alex Bennée 
Reviewed-by: Peter Maydell 
Signed-off-by: Richard Henderson 
---
 configure |   7 --
 include/qemu/atomic.h | 204 +++---
 2 files changed, 10 insertions(+), 201 deletions(-)

diff --git a/configure b/configure
index 232c54dcc1..b5965b159f 100755
--- a/configure
+++ b/configure
@@ -3991,18 +3991,11 @@ cat > $TMPC << EOF
 int main(void)
 {
   uint64_t x = 0, y = 0;
-#ifdef __ATOMIC_RELAXED
   y = __atomic_load_n(&x, __ATOMIC_RELAXED);
   __atomic_store_n(&x, y, __ATOMIC_RELAXED);
   __atomic_compare_exchange_n(&x, &y, x, 0, __ATOMIC_RELAXED, 
__ATOMIC_RELAXED);
   __atomic_exchange_n(&x, y, __ATOMIC_RELAXED);
   __atomic_fetch_add(&x, y, __ATOMIC_RELAXED);
-#else
-  typedef char is_host64[sizeof(void *) >= sizeof(uint64_t) ? 1 : -1];
-  __sync_lock_test_and_set(&x, y);
-  __sync_val_compare_and_swap(&x, y, 0);
-  __sync_fetch_and_add(&x, y);
-#endif
   return 0;
 }
 EOF
diff --git a/include/qemu/atomic.h b/include/qemu/atomic.h
index a7654d2a33..fe5467d193 100644
--- a/include/qemu/atomic.h
+++ b/include/qemu/atomic.h
@@ -60,8 +60,9 @@
 (unsigned short)1, 
\
   (expr)+0))
 
-#ifdef __ATOMIC_RELAXED
-/* For C11 atomic ops */
+#ifndef __ATOMIC_RELAXED
+#error "Expecting C11 atomic ops"
+#endif
 
 /* Manual memory barriers
  *
@@ -239,193 +240,8 @@
 #define qatomic_xor(ptr, n) \
 ((void) __atomic_fetch_xor(ptr, n, __ATOMIC_SEQ_CST))
 
-#else /* __ATOMIC_RELAXED */
-
-#ifdef __alpha__
-#define smp_read_barrier_depends()   asm volatile("mb":::"memory")
-#endif
-
-#if defined(__i386__) || defined(__x86_64__) || defined(__s390x__)
-
-/*
- * Because of the strongly ordered storage model, wmb() and rmb() are nops
- * here (a compiler barrier only).  QEMU doesn't do accesses to write-combining
- * qemu memory or non-temporal load/stores from C code.
- */
-#define smp_mb_release()   barrier()
-#define smp_mb_acquire()   barrier()
-
-/*
- * __sync_lock_test_and_set() is documented to be an acquire barrier only,
- * but it is a full barrier at the hardware level.  Add a compiler barrier
- * to make it a full barrier also at the compiler level.
- */
-#define qatomic_xchg(ptr, i)(barrier(), __sync_lock_test_and_set(ptr, i))
-
-#elif defined(_ARCH_PPC)
-
-/*
- * We use an eieio() for wmb() on powerpc.  This assumes we don't
- * need to order cacheable and non-cacheable stores with respect to
- * each other.
- *
- * smp_mb has the same problem as on x86 for not-very-new GCC
- * (http://patchwork.ozlabs.org/patch/126184/, Nov 2011).
- */
-#define smp_wmb()  ({ asm volatile("eieio" ::: "memory"); (void)0; })
-#if defined(__powerpc64__)
-#define smp_mb_release()   ({ asm volatile("lwsync" ::: "memory"); (void)0; })
-#define smp_mb_acquire()   ({ asm volatile("lwsync" ::: "memory"); (void)0; })
-#else
-#define smp_mb_release()   ({ asm volatile("sync" ::: "memory"); (void)0; })
-#define smp_mb_acquire()   ({ asm volatile("sync" ::: "memory"); (void)0; })
-#endif
-#define smp_mb()   ({ asm volatile("sync" ::: "memory"); (void)0; })
-
-#endif /* _ARCH_PPC */
-
-/*
- * For (host) platforms we don't have explicit barrier definitions
- * for, we use the gcc __sync_synchronize() primitive to generate a
- * full barrier.  This should be safe on all platforms, though it may
- * be overkill for smp_mb_acquire() and smp_mb_release().
- */
-#ifndef smp_mb
-#define smp_mb()   __sync_synchronize()
-#endif
-
-#ifndef smp_mb_acquire
-#define smp_mb_acquire()   __sync_synchronize()
-#endif
-
-#ifndef smp_mb_release
-#define smp_mb_release()   __sync_synchronize()
-#endif
-
-#ifndef smp_read_barrier_depends
-#define smp_read_barrier_depends()   barrier()
-#endif
-
-#ifndef signal_barrier
-#define signal_barrier()barrier()
-#endif
-
-/* These will only be atomic if the processor does the fetch or store
- * in a single issue memory operation
- */
-#define qatomic_read__nocheck(p)   (*(__typeof__(*(p)) volatile*) (p))
-#define qatomic_set__nocheck(p, i) ((*(__typeof__(*(p)) volatile*) (p)) = (i))
-
-#define qatomic_read(ptr)   qatomic_read__nocheck(ptr)
-#define qatomic_set(ptr, i) qatomic_set__nocheck(ptr,i)
-
-/**
- * qatomic_rcu_read - reads a RCU-protected pointer to a local variable
- * into a RCU read-side critical section. The pointer can later be safely
- * dereferenced within the critical section.
- *
- * This ensures that the pointer copy is invariant thorough the whole critical
- * section.
- *
- * Inserts memory barriers on architectures that require them (currently only
- * Alpha) and documents which pointers are protected by RCU.
- *
- * qatomic_rcu_read also includes a compiler barrier to ensure that
- * value-speculative optimizations (e.g. VSS: Value Speculation
- * Scheduling) does not perform the data read before the pointer read
- * by speculating the value of 

[PULL 03/27] qemu/atomic: Add aligned_{int64,uint64}_t types

[Richard Henderson]

Use it to avoid some clang-12 -Watomic-alignment errors,
forcing some structures to be aligned and as a pointer when
we have ensured that the address is aligned.

Tested-by: Cole Robinson 
Reviewed-by: Peter Maydell 
Reviewed-by: Philippe Mathieu-Daudé 
Signed-off-by: Richard Henderson 
---
 accel/tcg/atomic_template.h |  4 ++--
 include/qemu/atomic.h   | 14 +-
 include/qemu/stats64.h  |  2 +-
 softmmu/timers-state.h  |  2 +-
 linux-user/hppa/cpu_loop.c  |  2 +-
 util/qsp.c  |  4 ++--
 6 files changed, 20 insertions(+), 8 deletions(-)

diff --git a/accel/tcg/atomic_template.h b/accel/tcg/atomic_template.h
index afa8a9daf3..d347462af5 100644
--- a/accel/tcg/atomic_template.h
+++ b/accel/tcg/atomic_template.h
@@ -28,8 +28,8 @@
 # define SHIFT  4
 #elif DATA_SIZE == 8
 # define SUFFIX q
-# define DATA_TYPE  uint64_t
-# define SDATA_TYPE int64_t
+# define DATA_TYPE  aligned_uint64_t
+# define SDATA_TYPE aligned_int64_t
 # define BSWAP  bswap64
 # define SHIFT  3
 #elif DATA_SIZE == 4
diff --git a/include/qemu/atomic.h b/include/qemu/atomic.h
index fe5467d193..112a29910b 100644
--- a/include/qemu/atomic.h
+++ b/include/qemu/atomic.h
@@ -271,7 +271,19 @@
 _oldn;  \
 })
 
-/* Abstractions to access atomically (i.e. "once") i64/u64 variables */
+/*
+ * Abstractions to access atomically (i.e. "once") i64/u64 variables.
+ *
+ * The i386 abi is odd in that by default members are only aligned to
+ * 4 bytes, which means that 8-byte types can wind up mis-aligned.
+ * Clang will then warn about this, and emit a call into libatomic.
+ *
+ * Use of these types in structures when they will be used with atomic
+ * operations can avoid this.
+ */
+typedef int64_t aligned_int64_t __attribute__((aligned(8)));
+typedef uint64_t aligned_uint64_t __attribute__((aligned(8)));
+
 #ifdef CONFIG_ATOMIC64
 /* Use __nocheck because sizeof(void *) might be < sizeof(u64) */
 #define qatomic_read_i64(P) \
diff --git a/include/qemu/stats64.h b/include/qemu/stats64.h
index fdd3d1b8f9..802402254b 100644
--- a/include/qemu/stats64.h
+++ b/include/qemu/stats64.h
@@ -21,7 +21,7 @@
 
 typedef struct Stat64 {
 #ifdef CONFIG_ATOMIC64
-uint64_t value;
+aligned_uint64_t value;
 #else
 uint32_t low, high;
 uint32_t lock;
diff --git a/softmmu/timers-state.h b/softmmu/timers-state.h
index 8c262ce139..94bb7394c5 100644
--- a/softmmu/timers-state.h
+++ b/softmmu/timers-state.h
@@ -47,7 +47,7 @@ typedef struct TimersState {
 int64_t last_delta;
 
 /* Compensate for varying guest execution speed.  */
-int64_t qemu_icount_bias;
+aligned_int64_t qemu_icount_bias;
 
 int64_t vm_clock_warp_start;
 int64_t cpu_clock_offset;
diff --git a/linux-user/hppa/cpu_loop.c b/linux-user/hppa/cpu_loop.c
index 3aaaf3337c..82d8183821 100644
--- a/linux-user/hppa/cpu_loop.c
+++ b/linux-user/hppa/cpu_loop.c
@@ -82,7 +82,7 @@ static abi_ulong hppa_lws(CPUHPPAState *env)
 o64 = *(uint64_t *)g2h(cs, old);
 n64 = *(uint64_t *)g2h(cs, new);
 #ifdef CONFIG_ATOMIC64
-r64 = qatomic_cmpxchg__nocheck((uint64_t *)g2h(cs, addr),
+r64 = qatomic_cmpxchg__nocheck((aligned_uint64_t *)g2h(cs, 
addr),
o64, n64);
 ret = r64 != o64;
 #else
diff --git a/util/qsp.c b/util/qsp.c
index bacc5fa2f6..8562b14a87 100644
--- a/util/qsp.c
+++ b/util/qsp.c
@@ -83,8 +83,8 @@ typedef struct QSPCallSite QSPCallSite;
 struct QSPEntry {
 void *thread_ptr;
 const QSPCallSite *callsite;
-uint64_t n_acqs;
-uint64_t ns;
+aligned_uint64_t n_acqs;
+aligned_uint64_t ns;
 unsigned int n_objs; /* count of coalesced objs; only used for reporting */
 };
 typedef struct QSPEntry QSPEntry;
-- 
2.25.1

[PULL 01/27] qemu/atomic: Use macros for CONFIG_ATOMIC64

[Richard Henderson]

Clang warnings about questionable atomic usage get localized
to the inline function in atomic.h.  By using a macro, we get
the full traceback to the original use that caused the warning.

Tested-by: Cole Robinson 
Reviewed-by: Philippe Mathieu-Daudé 
Signed-off-by: Richard Henderson 
---
 include/qemu/atomic.h | 29 +
 1 file changed, 9 insertions(+), 20 deletions(-)

diff --git a/include/qemu/atomic.h b/include/qemu/atomic.h
index 3ccf84fd46..a7654d2a33 100644
--- a/include/qemu/atomic.h
+++ b/include/qemu/atomic.h
@@ -457,26 +457,15 @@
 
 /* Abstractions to access atomically (i.e. "once") i64/u64 variables */
 #ifdef CONFIG_ATOMIC64
-static inline int64_t qatomic_read_i64(const int64_t *ptr)
-{
-/* use __nocheck because sizeof(void *) might be < sizeof(u64) */
-return qatomic_read__nocheck(ptr);
-}
-
-static inline uint64_t qatomic_read_u64(const uint64_t *ptr)
-{
-return qatomic_read__nocheck(ptr);
-}
-
-static inline void qatomic_set_i64(int64_t *ptr, int64_t val)
-{
-qatomic_set__nocheck(ptr, val);
-}
-
-static inline void qatomic_set_u64(uint64_t *ptr, uint64_t val)
-{
-qatomic_set__nocheck(ptr, val);
-}
+/* Use __nocheck because sizeof(void *) might be < sizeof(u64) */
+#define qatomic_read_i64(P) \
+_Generic(*(P), int64_t: qatomic_read__nocheck(P))
+#define qatomic_read_u64(P) \
+_Generic(*(P), uint64_t: qatomic_read__nocheck(P))
+#define qatomic_set_i64(P, V) \
+_Generic(*(P), int64_t: qatomic_set__nocheck(P, V))
+#define qatomic_set_u64(P, V) \
+_Generic(*(P), uint64_t: qatomic_set__nocheck(P, V))
 
 static inline void qatomic64_init(void)
 {
-- 
2.25.1

[PULL 00/27] tcg patch queue for rc0

[Richard Henderson]

The following changes since commit e77c8b8b8e933414ef07dbed04e02973fccffeb0:

  Update version for v6.1.0-rc0 release (2021-07-21 17:10:15 +0100)

are available in the Git repository at:

  https://gitlab.com/rth7680/qemu.git tags/pull-tcg-20210721

for you to fetch changes up to c2ffd7549b14373e9ca68eccd84fab141ffde646:

  accel/tcg: Record singlestep_enabled in tb->cflags (2021-07-21 07:47:05 -1000)


Atomic build fixes for clang-12
Breakpoint reorg


Richard Henderson (27):
  qemu/atomic: Use macros for CONFIG_ATOMIC64
  qemu/atomic: Remove pre-C11 atomic fallbacks
  qemu/atomic: Add aligned_{int64,uint64}_t types
  tcg: Rename helper_atomic_*_mmu and provide for user-only
  accel/tcg: Standardize atomic helpers on softmmu api
  accel/tcg: Fold EXTRA_ARGS into atomic_template.h
  accel/tcg: Remove ATOMIC_MMU_DECLS
  accel/tcg: Expand ATOMIC_MMU_LOOKUP_*
  trace: Fold mem-internal.h into mem.h
  accel/tcg: Push trace info building into atomic_common.c.inc
  accel/tcg: Reduce CF_COUNT_MASK to match TCG_MAX_INSNS
  accel/tcg: Move curr_cflags into cpu-exec.c
  target/alpha: Drop goto_tb path in gen_call_pal
  accel/tcg: Add CF_NO_GOTO_TB and CF_NO_GOTO_PTR
  accel/tcg: Drop CF_NO_GOTO_PTR from -d nochain
  accel/tcg: Handle -singlestep in curr_cflags
  accel/tcg: Use CF_NO_GOTO_{TB, PTR} in cpu_exec_step_atomic
  hw/core: Introduce TCGCPUOps.debug_check_breakpoint
  target/arm: Implement debug_check_breakpoint
  target/i386: Implement debug_check_breakpoint
  hw/core: Introduce CPUClass.gdb_adjust_breakpoint
  target/avr: Implement gdb_adjust_breakpoint
  accel/tcg: Merge tb_find into its only caller
  accel/tcg: Move breakpoint recognition outside translation
  accel/tcg: Remove TranslatorOps.breakpoint_check
  accel/tcg: Hoist tb_cflags to a local in translator_loop
  accel/tcg: Record singlestep_enabled in tb->cflags

 configure |   7 --
 accel/tcg/atomic_template.h   | 141 +++-
 accel/tcg/tcg-runtime.h   |  46 
 include/exec/exec-all.h   |  24 ++--
 include/exec/translator.h |  11 --
 include/hw/core/cpu.h |   4 +
 include/hw/core/tcg-cpu-ops.h |   6 +
 include/qemu/atomic.h | 247 ++
 include/qemu/stats64.h|   2 +-
 include/tcg/tcg.h |  78 ++---
 softmmu/timers-state.h|   2 +-
 target/arm/helper.h   |   2 -
 target/arm/internals.h|   3 +
 target/avr/cpu.h  |   1 +
 trace/mem-internal.h  |  50 -
 trace/mem.h   |  50 +++--
 accel/tcg/cpu-exec.c  | 205 +++
 accel/tcg/cputlb.c|  49 +
 accel/tcg/translate-all.c |   7 +-
 accel/tcg/translator.c|  39 ++-
 accel/tcg/user-exec.c |  41 +++
 cpu.c |  34 ++
 linux-user/hppa/cpu_loop.c|   2 +-
 plugins/core.c|   2 +-
 target/alpha/translate.c  |  31 +-
 target/arm/cpu.c  |   1 +
 target/arm/cpu_tcg.c  |   1 +
 target/arm/debug_helper.c |  12 +-
 target/arm/helper-a64.c   |   8 +-
 target/arm/translate-a64.c|  25 -
 target/arm/translate.c|  29 -
 target/avr/cpu.c  |   1 +
 target/avr/gdbstub.c  |  13 +++
 target/avr/translate.c|  32 --
 target/cris/translate.c   |  20 
 target/hexagon/translate.c|  17 ---
 target/hppa/translate.c   |  11 --
 target/i386/tcg/mem_helper.c  |  15 +--
 target/i386/tcg/tcg-cpu.c |  12 ++
 target/i386/tcg/translate.c   |  28 -
 target/m68k/op_helper.c   |  19 +---
 target/m68k/translate.c   |  18 ---
 target/microblaze/translate.c |  18 ---
 target/mips/tcg/translate.c   |  19 
 target/nios2/translate.c  |  27 -
 target/openrisc/translate.c   |  17 ---
 target/ppc/mem_helper.c   |  16 +--
 target/ppc/translate.c|  18 ---
 target/riscv/translate.c  |  17 ---
 target/rx/translate.c |  14 ---
 target/s390x/tcg/mem_helper.c |  19 ++--
 target/s390x/tcg/translate.c  |  24 
 target/sh4/translate.c|  18 ---
 target/sparc/translate.c  |  17 ---
 target/tricore/translate.c|  16 ---
 target/xtensa/translate.c |  17 ---
 tcg/tcg-op.c  |  79 --
 util/qsp.c|   4 +-
 accel/tcg/atomic_common.c.inc | 107 --
 59 files changed, 577 insertions(+), 1216 deletions(-)
 delete mode 100644 trace/mem-internal.h

Re: [PATCH v2 00/24] python: introduce Asynchronous QMP package

[John Snow]

Looping qemu-devel back in: I removed them by accident by not hitting
reply-all :(

On Wed, Jul 21, 2021 at 2:06 PM Niteesh G. S.  wrote:

>
>
> On Wed, Jul 21, 2021 at 11:03 PM John Snow  wrote:
>
>>
>>
>> On Wed, Jul 21, 2021 at 1:04 PM Niteesh G. S. 
>> wrote:
>>
>>> Hello all,
>>>
>>> I recently rebased(incrementally) my TUI on this V2 patch and faced an
>>> issue.
>>> https://gitlab.com/niteesh.gs/qemu/-/commits/aqmp-tui-prototype-v3
>>> I decided to rebase incrementally so that I can address some of the
>>> comments posted
>>> in my patch series. While testing out, the initial draft of TUI
>>> which worked fine in the V1
>>> version of AQMP failed in this version.
>>>
>>> Disconnecting from a fully connected state doesn't exit cleanly.
>>>
>>> -
>>> To reproduce the issue:
>>> 1) Initiate a QMP server
>>>
>>
>> Please provide the command line.
>>
> qemu-system-x86_64 -qmp tcp:localhost:1234,server,wait=on
>
>>
>>
>>> 2) Connect the TUI to the server using aqmp-tui localhost:1234
>>> --log-file log.txt
>>>
>>
>> The entry point isn't defined yet in your series, so I will assume
>> "python3 -m qemu.aqmp.aqmp_tui localhost:1234" should work here.
>>
> Yup, sorry about that. I realized this later when recreated the venv.
>
>>
>>
>>> 3) Once the TUI is connected and running, press 'Esc' to exit the app.
>>> This should result
>>> in the following exception.
>>>
>>> 
>>> Transitioning from 'Runstate.IDLE' to 'Runstate.CONNECTING'.
>>> Connecting to ('localhost', 1234) ...
>>> Connected.
>>> Awaiting greeting ...
>>> Response: {
>>>   "QMP": {
>>> .. Skipping
>>>   }
>>> }
>>> Negotiating capabilities ...
>>> Request: {
>>>   "execute": "qmp_capabilities",
>>> .. Skipping
>>>   }
>>> }
>>> Response: {
>>>   "return": {}
>>> }
>>> Transitioning from 'Runstate.CONNECTING' to 'Runstate.RUNNING'.
>>> Transitioning from 'Runstate.RUNNING' to 'Runstate.DISCONNECTING'.
>>> Scheduling disconnect.
>>> Draining the outbound queue ...
>>> Flushing the StreamWriter ...
>>> Cancelling writer task ...
>>> Task.Writer: cancelled.
>>> Task.Writer: exiting.
>>> Cancelling reader task ...
>>> Task.Reader: cancelled.
>>> Task.Reader: exiting.
>>> Closing StreamWriter.
>>> Waiting for StreamWriter to close ...
>>> QMP Disconnected.
>>> Transitioning from 'Runstate.DISCONNECTING' to 'Runstate.IDLE'.
>>> _kill_app: Connection lost
>>> Connection lost
>>>   | Traceback (most recent call last):
>>>   |   File
>>> "/home/niteesh/development/qemu/python/qemu/aqmp/aqmp_tui.py", line 246, in
>>> run
>>>   | main_loop.run()
>>>   |   File
>>> "/home/niteesh/development/qemu/python/.venv/lib/python3.6/site-packages/urwid/main_loop.py",
>>> line 287, in run
>>>   | self._run()
>>>   |   File
>>> "/home/niteesh/development/qemu/python/.venv/lib/python3.6/site-packages/urwid/main_loop.py",
>>> line 385, in _run
>>>   | self.event_loop.run()
>>>   |   File
>>> "/home/niteesh/development/qemu/python/.venv/lib/python3.6/site-packages/urwid/main_loop.py",
>>> line 1494, in run
>>>   | reraise(*exc_info)
>>>   |   File
>>> "/home/niteesh/development/qemu/python/.venv/lib/python3.6/site-packages/urwid/compat.py",
>>> line 58, in reraise
>>>   | raise value
>>>   |   File
>>> "/home/niteesh/development/qemu/python/qemu/aqmp/aqmp_tui.py", line 206, in
>>> _kill_app
>>>   | raise err
>>>   |   File
>>> "/home/niteesh/development/qemu/python/qemu/aqmp/aqmp_tui.py", line 201, in
>>> _kill_app
>>>   | await self.disconnect()
>>>   |   File
>>> "/home/niteesh/development/qemu/python/qemu/aqmp/protocol.py", line 303, in
>>> disconnect
>>>   | await self._wait_disconnect()
>>>   |   File
>>> "/home/niteesh/development/qemu/python/qemu/aqmp/protocol.py", line 573, in
>>> _wait_disconnect
>>>   | await self._dc_task
>>>   |   File
>>> "/home/niteesh/development/qemu/python/qemu/aqmp/qmp_client.py", line 316,
>>> in _bh_disconnect
>>>   | await super()._bh_disconnect()
>>>   |   File
>>> "/home/niteesh/development/qemu/python/qemu/aqmp/protocol.py", line 644, in
>>> _bh_disconnect
>>>   | await wait_closed(self._writer)
>>>   |   File "/home/niteesh/development/qemu/python/qemu/aqmp/util.py",
>>> line 137, in wait_closed
>>>   | await flush(writer)
>>>   |   File "/home/niteesh/development/qemu/python/qemu/aqmp/util.py",
>>> line 49, in flush
>>>   | await writer.drain()
>>>   |   File "/usr/lib/python3.6/asyncio/streams.py", line 339, in drain
>>>   | yield from self._protocol._drain_helper()
>>>   |   File "/usr/lib/python3.6/asyncio/streams.py", line 210, in
>>> _drain_helper
>>>   | raise ConnectionResetError('Connection lost')
>>>   | ConnectionResetError: Connection lost
>>>
>>> ---

[PULL 2/3] qemu-img: Fail fast on convert --bitmaps with inconsistent bitmap

[Eric Blake]

Waiting until the end of the convert operation (a potentially
time-consuming task) to finally detect that we can't copy a bitmap is
bad, comparing to failing fast up front.  Furthermore, this prevents
us from leaving a file behind with a bitmap that is not marked as
inconsistent even though it does not have sane contents.

This fixes the problems exposed in the previous patch to the iotest:
it adds a fast failure up front, and even if we don't fail early, it
ensures that any bitmap we add but do not properly populate is removed
again rather than left behind incomplete.

Signed-off-by: Eric Blake 
Message-Id: <20210709153951.2801666-3-ebl...@redhat.com>
[eblake: add a hint to the warning message, simplify name computation]
Reviewed-by: Nir Soffer 
Reviewed-by: Vladimir Sementsov-Ogievskiy 
---
 qemu-img.c| 29 +--
 tests/qemu-iotests/tests/qemu-img-bitmaps |  3 +-
 tests/qemu-iotests/tests/qemu-img-bitmaps.out | 21 ++
 3 files changed, 30 insertions(+), 23 deletions(-)

diff --git a/qemu-img.c b/qemu-img.c
index 797742a44331..c5496e82e025 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -2101,6 +2101,29 @@ static int convert_do_copy(ImgConvertState *s)
 return s->ret;
 }

+/* Check that bitmaps can be copied, or output an error */
+static int convert_check_bitmaps(BlockDriverState *src)
+{
+BdrvDirtyBitmap *bm;
+
+if (!bdrv_supports_persistent_dirty_bitmap(src)) {
+error_report("Source lacks bitmap support");
+return -1;
+}
+FOR_EACH_DIRTY_BITMAP(src, bm) {
+if (!bdrv_dirty_bitmap_get_persistence(bm)) {
+continue;
+}
+if (bdrv_dirty_bitmap_inconsistent(bm)) {
+error_report("Cannot copy inconsistent bitmap '%s'",
+ bdrv_dirty_bitmap_name(bm));
+error_printf("Try 'qemu-img bitmap --remove' to delete it\n");
+return -1;
+}
+}
+return 0;
+}
+
 static int convert_copy_bitmaps(BlockDriverState *src, BlockDriverState *dst)
 {
 BdrvDirtyBitmap *bm;
@@ -2127,6 +2150,7 @@ static int convert_copy_bitmaps(BlockDriverState *src, 
BlockDriverState *dst)
   &err);
 if (err) {
 error_reportf_err(err, "Failed to populate bitmap %s: ", name);
+qmp_block_dirty_bitmap_remove(dst->node_name, name, NULL);
 return -1;
 }
 }
@@ -2554,9 +2578,8 @@ static int img_convert(int argc, char **argv)
 ret = -1;
 goto out;
 }
-if (!bdrv_supports_persistent_dirty_bitmap(blk_bs(s.src[0]))) {
-error_report("Source lacks bitmap support");
-ret = -1;
+ret = convert_check_bitmaps(blk_bs(s.src[0]));
+if (ret < 0) {
 goto out;
 }
 }
diff --git a/tests/qemu-iotests/tests/qemu-img-bitmaps 
b/tests/qemu-iotests/tests/qemu-img-bitmaps
index 409c4497a303..09c3d395d1eb 100755
--- a/tests/qemu-iotests/tests/qemu-img-bitmaps
+++ b/tests/qemu-iotests/tests/qemu-img-bitmaps
@@ -140,11 +140,10 @@ $QEMU_IO -c abort "$TEST_IMG" 2>/dev/null
 $QEMU_IMG bitmap --add "$TEST_IMG" b4
 $QEMU_IMG bitmap --remove "$TEST_IMG" b1
 _img_info --format-specific | _filter_irrelevant_img_info
+# Proof that we fail fast if bitmaps can't be copied
 echo
 $QEMU_IMG convert --bitmaps -O qcow2 "$TEST_IMG" "$TEST_IMG.copy" &&
 echo "unexpected success"
-# Bug - even though we failed at conversion, we left a file around with
-# a bitmap marked as not corrupt
 TEST_IMG=$TEST_IMG.copy _img_info --format-specific \
 | _filter_irrelevant_img_info

diff --git a/tests/qemu-iotests/tests/qemu-img-bitmaps.out 
b/tests/qemu-iotests/tests/qemu-img-bitmaps.out
index 6824d4112893..d99c279d0c63 100644
--- a/tests/qemu-iotests/tests/qemu-img-bitmaps.out
+++ b/tests/qemu-iotests/tests/qemu-img-bitmaps.out
@@ -144,22 +144,7 @@ Format specific information:
 granularity: 65536
 corrupt: false

-qemu-img: Failed to populate bitmap b0: Bitmap 'b0' is inconsistent and cannot 
be used
-Try block-dirty-bitmap-remove to delete this bitmap from disk
-image: TEST_DIR/t.IMGFMT.copy
-file format: IMGFMT
-virtual size: 10 MiB (10485760 bytes)
-cluster_size: 65536
-Format specific information:
-bitmaps:
-[0]:
-flags:
-name: b0
-granularity: 65536
-[1]:
-flags:
-[0]: auto
-name: b4
-granularity: 65536
-corrupt: false
+qemu-img: Cannot copy inconsistent bitmap 'b0'
+Try 'qemu-img bitmap --remove' to delete it
+qemu-img: Could not open 'TEST_DIR/t.IMGFMT.copy': Could not open 
'TEST_DIR/t.IMGFMT.copy': No such file or directory
 *** done
-- 
2.31.1

[PULL 3/3] qemu-img: Add --skip-broken-bitmaps for 'convert --bitmaps'

[Eric Blake]

The point of 'qemu-img convert --bitmaps' is to be a convenience for
actions that are already possible through a string of smaller
'qemu-img bitmap' sub-commands.  One situation not accounted for
already is that if a source image contains an inconsistent bitmap (for
example, because a qemu process died abruptly before flushing bitmap
state), the user MUST delete those inconsistent bitmaps before
anything else useful can be done with the image.

We don't want to delete inconsistent bitmaps by default: although a
corrupt bitmap is only a loss of optimization rather than a corruption
of user-visible data, it is still nice to require the user to opt in
to the fact that they are aware of the loss of the bitmap.  Still,
requiring the user to check 'qemu-img info' to see whether bitmaps are
consistent, then use 'qemu-img bitmap --remove' to remove offenders,
all before using 'qemu-img convert', is a lot more work than just
adding a knob 'qemu-img convert --bitmaps --skip-broken-bitmaps' which
opts in to skipping the broken bitmaps.

After testing the new option, also demonstrate the way to manually fix
things (either deleting bad bitmaps, or re-creating them as empty) so
that it is possible to convert without the option.

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1946084
Signed-off-by: Eric Blake 
Message-Id: <20210709153951.2801666-4-ebl...@redhat.com>
[eblake: warning message tweak, test enhancements]
Reviewed-by: Vladimir Sementsov-Ogievskiy 
---
 docs/tools/qemu-img.rst   |  8 -
 qemu-img.c| 29 +++
 tests/qemu-iotests/tests/qemu-img-bitmaps | 16 -
 tests/qemu-iotests/tests/qemu-img-bitmaps.out | 35 ++-
 4 files changed, 79 insertions(+), 9 deletions(-)

diff --git a/docs/tools/qemu-img.rst b/docs/tools/qemu-img.rst
index 1d8470eada0e..b7d602a28804 100644
--- a/docs/tools/qemu-img.rst
+++ b/docs/tools/qemu-img.rst
@@ -414,7 +414,7 @@ Command description:
   4
 Error on reading data

-.. option:: convert [--object OBJECTDEF] [--image-opts] [--target-image-opts] 
[--target-is-zero] [--bitmaps] [-U] [-C] [-c] [-p] [-q] [-n] [-f FMT] [-t 
CACHE] [-T SRC_CACHE] [-O OUTPUT_FMT] [-B BACKING_FILE] [-o OPTIONS] [-l 
SNAPSHOT_PARAM] [-S SPARSE_SIZE] [-r RATE_LIMIT] [-m NUM_COROUTINES] [-W] 
FILENAME [FILENAME2 [...]] OUTPUT_FILENAME
+.. option:: convert [--object OBJECTDEF] [--image-opts] [--target-image-opts] 
[--target-is-zero] [--bitmaps [--skip-broken-bitmaps]] [-U] [-C] [-c] [-p] [-q] 
[-n] [-f FMT] [-t CACHE] [-T SRC_CACHE] [-O OUTPUT_FMT] [-B BACKING_FILE] [-o 
OPTIONS] [-l SNAPSHOT_PARAM] [-S SPARSE_SIZE] [-r RATE_LIMIT] [-m 
NUM_COROUTINES] [-W] FILENAME [FILENAME2 [...]] OUTPUT_FILENAME

   Convert the disk image *FILENAME* or a snapshot *SNAPSHOT_PARAM*
   to disk image *OUTPUT_FILENAME* using format *OUTPUT_FMT*. It can
@@ -456,6 +456,12 @@ Command description:
   *NUM_COROUTINES* specifies how many coroutines work in parallel during
   the convert process (defaults to 8).

+  Use of ``--bitmaps`` requests that any persistent bitmaps present in
+  the original are also copied to the destination.  If any bitmap is
+  inconsistent in the source, the conversion will fail unless
+  ``--skip-broken-bitmaps`` is also specified to copy only the
+  consistent bitmaps.
+
 .. option:: create [--object OBJECTDEF] [-q] [-f FMT] [-b BACKING_FILE] [-F 
BACKING_FMT] [-u] [-o OPTIONS] FILENAME [SIZE]

   Create the new disk image *FILENAME* of size *SIZE* and format
diff --git a/qemu-img.c b/qemu-img.c
index c5496e82e025..908fd0cce5b4 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -82,6 +82,7 @@ enum {
 OPTION_MERGE = 274,
 OPTION_BITMAPS = 275,
 OPTION_FORCE = 276,
+OPTION_SKIP_BROKEN = 277,
 };

 typedef enum OutputFormat {
@@ -2102,7 +2103,7 @@ static int convert_do_copy(ImgConvertState *s)
 }

 /* Check that bitmaps can be copied, or output an error */
-static int convert_check_bitmaps(BlockDriverState *src)
+static int convert_check_bitmaps(BlockDriverState *src, bool skip_broken)
 {
 BdrvDirtyBitmap *bm;

@@ -2114,17 +2115,19 @@ static int convert_check_bitmaps(BlockDriverState *src)
 if (!bdrv_dirty_bitmap_get_persistence(bm)) {
 continue;
 }
-if (bdrv_dirty_bitmap_inconsistent(bm)) {
+if (!skip_broken && bdrv_dirty_bitmap_inconsistent(bm)) {
 error_report("Cannot copy inconsistent bitmap '%s'",
  bdrv_dirty_bitmap_name(bm));
-error_printf("Try 'qemu-img bitmap --remove' to delete it\n");
+error_printf("Try --skip-broken-bitmaps, or "
+ "use 'qemu-img bitmap --remove' to delete it\n");
 return -1;
 }
 }
 return 0;
 }

-static int convert_copy_bitmaps(BlockDriverState *src, BlockDriverState *dst)
+static int convert_copy_bitmaps(BlockDriverState *src, BlockDriverState *dst,
+bool skip_broken)
 {
 BdrvD

[PULL 0/3] block bitmaps patches for rc1, 2021-07-21

[Eric Blake]

The following changes since commit e77c8b8b8e933414ef07dbed04e02973fccffeb0:

  Update version for v6.1.0-rc0 release (2021-07-21 17:10:15 +0100)

are available in the Git repository at:

  https://repo.or.cz/qemu/ericb.git tags/pull-bitmaps-2021-07-21

for you to fetch changes up to 955171e4417bf39edb5503e694501e082a757731:

  qemu-img: Add --skip-broken-bitmaps for 'convert --bitmaps' (2021-07-21 
14:14:41 -0500)


block bitmaps patches for 2021-07-21

- fix 'qemu-img convert --bitmaps' handling of qcow2 files with
  inconsistent bitmaps


Eric Blake (3):
  iotests: Improve and rename test 291 to qemu-img-bitmap
  qemu-img: Fail fast on convert --bitmaps with inconsistent bitmap
  qemu-img: Add --skip-broken-bitmaps for 'convert --bitmaps'

 docs/tools/qemu-img.rst|  8 ++-
 block/dirty-bitmap.c   |  2 +-
 qemu-img.c | 50 ++--
 tests/qemu-iotests/{291 => tests/qemu-img-bitmaps} | 34 ++-
 .../{291.out => tests/qemu-img-bitmaps.out}| 67 +-
 5 files changed, 152 insertions(+), 9 deletions(-)
 rename tests/qemu-iotests/{291 => tests/qemu-img-bitmaps} (79%)
 rename tests/qemu-iotests/{291.out => tests/qemu-img-bitmaps.out} (70%)

-- 
2.31.1

[PULL 1/3] iotests: Improve and rename test 291 to qemu-img-bitmap

[Eric Blake]

Enhance the test to demonstrate existing less-than-stellar behavior of
qemu-img with a qcow2 image containing an inconsistent bitmap: we
don't diagnose the problem until after copying the entire image (a
potentially long time), and when we do diagnose the failure, we still
end up leaving an empty bitmap in the destination.  This mess will be
cleaned up in the next patch.

While at it, rename the test now that we support useful iotest names,
and fix a missing newline in the error message thus exposed.

Signed-off-by: Eric Blake 
Message-Id: <20210709153951.2801666-2-ebl...@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy 
Reviewed-by: Nir Soffer 
---
 block/dirty-bitmap.c  |  2 +-
 .../{291 => tests/qemu-img-bitmaps}   | 21 +++-
 .../{291.out => tests/qemu-img-bitmaps.out}   | 49 ++-
 3 files changed, 69 insertions(+), 3 deletions(-)
 rename tests/qemu-iotests/{291 => tests/qemu-img-bitmaps} (87%)
 rename tests/qemu-iotests/{291.out => tests/qemu-img-bitmaps.out} (76%)

diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 68d295d6e3ed..0ef46163e3ea 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -193,7 +193,7 @@ int bdrv_dirty_bitmap_check(const BdrvDirtyBitmap *bitmap, 
uint32_t flags,
 error_setg(errp, "Bitmap '%s' is inconsistent and cannot be used",
bitmap->name);
 error_append_hint(errp, "Try block-dirty-bitmap-remove to delete"
-  " this bitmap from disk");
+  " this bitmap from disk\n");
 return -1;
 }

diff --git a/tests/qemu-iotests/291 b/tests/qemu-iotests/tests/qemu-img-bitmaps
similarity index 87%
rename from tests/qemu-iotests/291
rename to tests/qemu-iotests/tests/qemu-img-bitmaps
index 20efb080a6c0..409c4497a303 100755
--- a/tests/qemu-iotests/291
+++ b/tests/qemu-iotests/tests/qemu-img-bitmaps
@@ -3,7 +3,7 @@
 #
 # Test qemu-img bitmap handling
 #
-# Copyright (C) 2018-2020 Red Hat, Inc.
+# Copyright (C) 2018-2021 Red Hat, Inc.
 #
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -27,11 +27,13 @@ status=1 # failure is the default!
 _cleanup()
 {
 _cleanup_test_img
+_rm_test_img "$TEST_IMG.copy"
 nbd_server_stop
 }
 trap "_cleanup; exit \$status" 0 1 2 3 15

 # get standard environment, filters and checks
+cd ..
 . ./common.rc
 . ./common.filter
 . ./common.nbd
@@ -129,6 +131,23 @@ $QEMU_IMG map --output=json --image-opts \

 nbd_server_stop

+echo
+echo "=== Check handling of inconsistent bitmap ==="
+echo
+
+# Prepare image with corrupted bitmap
+$QEMU_IO -c abort "$TEST_IMG" 2>/dev/null
+$QEMU_IMG bitmap --add "$TEST_IMG" b4
+$QEMU_IMG bitmap --remove "$TEST_IMG" b1
+_img_info --format-specific | _filter_irrelevant_img_info
+echo
+$QEMU_IMG convert --bitmaps -O qcow2 "$TEST_IMG" "$TEST_IMG.copy" &&
+echo "unexpected success"
+# Bug - even though we failed at conversion, we left a file around with
+# a bitmap marked as not corrupt
+TEST_IMG=$TEST_IMG.copy _img_info --format-specific \
+| _filter_irrelevant_img_info
+
 # success, all done
 echo '*** done'
 rm -f $seq.full
diff --git a/tests/qemu-iotests/291.out 
b/tests/qemu-iotests/tests/qemu-img-bitmaps.out
similarity index 76%
rename from tests/qemu-iotests/291.out
rename to tests/qemu-iotests/tests/qemu-img-bitmaps.out
index 018d6b103f87..6824d4112893 100644
--- a/tests/qemu-iotests/291.out
+++ b/tests/qemu-iotests/tests/qemu-img-bitmaps.out
@@ -1,4 +1,4 @@
-QA output created by 291
+QA output created by qemu-img-bitmaps

 === Initial image setup ===

@@ -115,4 +115,51 @@ Format specific information:
 [{ "start": 0, "length": 2097152, "depth": 0, "present": true, "zero": false, 
"data": true, "offset": OFFSET},
 { "start": 2097152, "length": 1048576, "depth": 0, "present": false, "zero": 
false, "data": false},
 { "start": 3145728, "length": 7340032, "depth": 0, "present": true, "zero": 
false, "data": true, "offset": OFFSET}]
+
+=== Check handling of inconsistent bitmap ===
+
+image: TEST_DIR/t.IMGFMT
+file format: IMGFMT
+virtual size: 10 MiB (10485760 bytes)
+cluster_size: 65536
+backing file: TEST_DIR/t.IMGFMT.base
+backing file format: IMGFMT
+Format specific information:
+bitmaps:
+[0]:
+flags:
+[0]: in-use
+[1]: auto
+name: b2
+granularity: 65536
+[1]:
+flags:
+[0]: in-use
+name: b0
+granularity: 65536
+[2]:
+flags:
+[0]: auto
+name: b4
+granularity: 65536
+corrupt: false
+
+qemu-img: Failed to populate bitmap b0: Bitmap 'b0' is inconsistent and cannot 
be used
+Try block-dirty-bitmap-remove to delete this bitmap from disk
+image: TEST_DIR/t.IMGFMT.copy
+file format: IMGFMT
+virtual size: 10 MiB (10485760 bytes)
+cluster_size: 65536
+Format

[PATCH v2 5/5] migration: Move the yank unregister of channel_close out

[Peter Xu]

It's efficient, but hackish to call yank unregister calls in channel_close(),
especially it'll be hard to debug when qemu crashed with some yank function
leaked.

Remove that hack, but instead explicitly unregister yank functions at the
places where needed, they are:

  (on src)
  - migrate_fd_cleanup
  - postcopy_pause

  (on dst)
  - migration_incoming_state_destroy
  - postcopy_pause_incoming

Signed-off-by: Peter Xu 
---
 migration/migration.c |  5 +
 migration/qemu-file-channel.c |  3 ---
 migration/savevm.c|  7 +++
 migration/yank_functions.c| 14 ++
 migration/yank_functions.h|  1 +
 5 files changed, 27 insertions(+), 3 deletions(-)

diff --git a/migration/migration.c b/migration/migration.c
index fa70400f98..bfeb65b8f7 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -59,6 +59,7 @@
 #include "multifd.h"
 #include "qemu/yank.h"
 #include "sysemu/cpus.h"
+#include "yank_functions.h"
 
 #define MAX_THROTTLE  (128 << 20)  /* Migration transfer speed throttling 
*/
 
@@ -273,6 +274,7 @@ void migration_incoming_state_destroy(void)
 }
 
 if (mis->from_src_file) {
+migration_ioc_unregister_yank_from_file(mis->from_src_file);
 qemu_fclose(mis->from_src_file);
 mis->from_src_file = NULL;
 }
@@ -1811,6 +1813,7 @@ static void migrate_fd_cleanup(MigrationState *s)
  * Close the file handle without the lock to make sure the
  * critical section won't block for long.
  */
+migration_ioc_unregister_yank_from_file(tmp);
 qemu_fclose(tmp);
 }
 
@@ -3352,6 +3355,8 @@ static MigThrError postcopy_pause(MigrationState *s)
 
 /* Current channel is possibly broken. Release it. */
 assert(s->to_dst_file);
+/* Unregister yank for current channel */
+migration_ioc_unregister_yank_from_file(s->to_dst_file);
 qemu_mutex_lock(&s->qemu_file_lock);
 file = s->to_dst_file;
 s->to_dst_file = NULL;
diff --git a/migration/qemu-file-channel.c b/migration/qemu-file-channel.c
index 2f8b1fcd46..bb5a5752df 100644
--- a/migration/qemu-file-channel.c
+++ b/migration/qemu-file-channel.c
@@ -107,9 +107,6 @@ static int channel_close(void *opaque, Error **errp)
 int ret;
 QIOChannel *ioc = QIO_CHANNEL(opaque);
 ret = qio_channel_close(ioc, errp);
-if (OBJECT(ioc)->ref == 1) {
-migration_ioc_unregister_yank(ioc);
-}
 object_unref(OBJECT(ioc));
 return ret;
 }
diff --git a/migration/savevm.c b/migration/savevm.c
index 96b5e5d639..7b7b64bd13 100644
--- a/migration/savevm.c
+++ b/migration/savevm.c
@@ -65,6 +65,7 @@
 #include "qemu/bitmap.h"
 #include "net/announce.h"
 #include "qemu/yank.h"
+#include "yank_functions.h"
 
 const unsigned int postcopy_ram_discard_version;
 
@@ -2568,6 +2569,12 @@ static bool 
postcopy_pause_incoming(MigrationIncomingState *mis)
 /* Clear the triggered bit to allow one recovery */
 mis->postcopy_recover_triggered = false;
 
+/*
+ * Unregister yank with either from/to src would work, since ioc behind it
+ * is the same
+ */
+migration_ioc_unregister_yank_from_file(mis->from_src_file);
+
 assert(mis->from_src_file);
 qemu_file_shutdown(mis->from_src_file);
 qemu_fclose(mis->from_src_file);
diff --git a/migration/yank_functions.c b/migration/yank_functions.c
index 23697173ae..8c08aef14a 100644
--- a/migration/yank_functions.c
+++ b/migration/yank_functions.c
@@ -14,6 +14,7 @@
 #include "qemu/yank.h"
 #include "io/channel-socket.h"
 #include "io/channel-tls.h"
+#include "qemu-file.h"
 
 void migration_yank_iochannel(void *opaque)
 {
@@ -46,3 +47,16 @@ void migration_ioc_unregister_yank(QIOChannel *ioc)
  QIO_CHANNEL(ioc));
 }
 }
+
+void migration_ioc_unregister_yank_from_file(QEMUFile *file)
+{
+QIOChannel *ioc = qemu_file_get_ioc(file);
+
+if (ioc) {
+/*
+ * For migration qemufiles, we'll always reach here.  Though we'll skip
+ * calls from e.g. savevm/loadvm as they don't use yank.
+ */
+migration_ioc_unregister_yank(ioc);
+}
+}
diff --git a/migration/yank_functions.h b/migration/yank_functions.h
index 74c7f18c91..a7577955ed 100644
--- a/migration/yank_functions.h
+++ b/migration/yank_functions.h
@@ -17,3 +17,4 @@
 void migration_yank_iochannel(void *opaque);
 void migration_ioc_register_yank(QIOChannel *ioc);
 void migration_ioc_unregister_yank(QIOChannel *ioc);
+void migration_ioc_unregister_yank_from_file(QEMUFile *file);
-- 
2.31.1

[PATCH v2 4/5] migration: Teach QEMUFile to be QIOChannel-aware

[Peter Xu]

migration uses QIOChannel typed qemufiles.  In follow up patches, we'll need
the capability to identify this fact, so that we can get the backing QIOChannel
from a QEMUFile.

We can also define types for QEMUFile but so far since we only need to be able
to identify QIOChannel, introduce a boolean which is simpler.

Introduce another helper qemu_file_get_ioc() to return the ioc backend of a
qemufile if has_ioc is set.

No functional change.

Signed-off-by: Peter Xu 
---
 migration/qemu-file-channel.c |  4 ++--
 migration/qemu-file.c | 17 -
 migration/qemu-file.h |  4 +++-
 migration/ram.c   |  2 +-
 migration/savevm.c|  4 ++--
 5 files changed, 24 insertions(+), 7 deletions(-)

diff --git a/migration/qemu-file-channel.c b/migration/qemu-file-channel.c
index 867a5ed0c3..2f8b1fcd46 100644
--- a/migration/qemu-file-channel.c
+++ b/migration/qemu-file-channel.c
@@ -187,11 +187,11 @@ static const QEMUFileOps channel_output_ops = {
 QEMUFile *qemu_fopen_channel_input(QIOChannel *ioc)
 {
 object_ref(OBJECT(ioc));
-return qemu_fopen_ops(ioc, &channel_input_ops);
+return qemu_fopen_ops(ioc, &channel_input_ops, true);
 }
 
 QEMUFile *qemu_fopen_channel_output(QIOChannel *ioc)
 {
 object_ref(OBJECT(ioc));
-return qemu_fopen_ops(ioc, &channel_output_ops);
+return qemu_fopen_ops(ioc, &channel_output_ops, true);
 }
diff --git a/migration/qemu-file.c b/migration/qemu-file.c
index 1eacf9e831..6338d8e2ff 100644
--- a/migration/qemu-file.c
+++ b/migration/qemu-file.c
@@ -55,6 +55,8 @@ struct QEMUFile {
 Error *last_error_obj;
 /* has the file has been shutdown */
 bool shutdown;
+/* Whether opaque points to a QIOChannel */
+bool has_ioc;
 };
 
 /*
@@ -101,7 +103,7 @@ bool qemu_file_mode_is_not_valid(const char *mode)
 return false;
 }
 
-QEMUFile *qemu_fopen_ops(void *opaque, const QEMUFileOps *ops)
+QEMUFile *qemu_fopen_ops(void *opaque, const QEMUFileOps *ops, bool has_ioc)
 {
 QEMUFile *f;
 
@@ -109,6 +111,7 @@ QEMUFile *qemu_fopen_ops(void *opaque, const QEMUFileOps 
*ops)
 
 f->opaque = opaque;
 f->ops = ops;
+f->has_ioc = has_ioc;
 return f;
 }
 
@@ -851,3 +854,15 @@ void qemu_file_set_blocking(QEMUFile *f, bool block)
 f->ops->set_blocking(f->opaque, block, NULL);
 }
 }
+
+/*
+ * Return the ioc object if it's a migration channel.  Note: it can return NULL
+ * for callers passing in a non-migration qemufile.  E.g. see qemu_fopen_bdrv()
+ * and its usage in e.g. load_snapshot().  So we need to check against NULL
+ * before using it.  If without the check, migration_incoming_state_destroy()
+ * could fail for load_snapshot().
+ */
+QIOChannel *qemu_file_get_ioc(QEMUFile *file)
+{
+return file->has_ioc ? QIO_CHANNEL(file->opaque) : NULL;
+}
diff --git a/migration/qemu-file.h b/migration/qemu-file.h
index a9b6d6ccb7..3f36d4dc8c 100644
--- a/migration/qemu-file.h
+++ b/migration/qemu-file.h
@@ -27,6 +27,7 @@
 
 #include 
 #include "exec/cpu-common.h"
+#include "io/channel.h"
 
 /* Read a chunk of data from a file at the given position.  The pos argument
  * can be ignored if the file is only be used for streaming.  The number of
@@ -119,7 +120,7 @@ typedef struct QEMUFileHooks {
 QEMURamSaveFunc *save_page;
 } QEMUFileHooks;
 
-QEMUFile *qemu_fopen_ops(void *opaque, const QEMUFileOps *ops);
+QEMUFile *qemu_fopen_ops(void *opaque, const QEMUFileOps *ops, bool has_ioc);
 void qemu_file_set_hooks(QEMUFile *f, const QEMUFileHooks *hooks);
 int qemu_get_fd(QEMUFile *f);
 int qemu_fclose(QEMUFile *f);
@@ -179,5 +180,6 @@ void ram_control_load_hook(QEMUFile *f, uint64_t flags, 
void *data);
 size_t ram_control_save_page(QEMUFile *f, ram_addr_t block_offset,
  ram_addr_t offset, size_t size,
  uint64_t *bytes_sent);
+QIOChannel *qemu_file_get_ioc(QEMUFile *file);
 
 #endif
diff --git a/migration/ram.c b/migration/ram.c
index f728f5072f..08b3cb7a4a 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -550,7 +550,7 @@ static int compress_threads_save_setup(void)
 /* comp_param[i].file is just used as a dummy buffer to save data,
  * set its ops to empty.
  */
-comp_param[i].file = qemu_fopen_ops(NULL, &empty_ops);
+comp_param[i].file = qemu_fopen_ops(NULL, &empty_ops, false);
 comp_param[i].done = true;
 comp_param[i].quit = false;
 qemu_mutex_init(&comp_param[i].mutex);
diff --git a/migration/savevm.c b/migration/savevm.c
index 72848b946c..96b5e5d639 100644
--- a/migration/savevm.c
+++ b/migration/savevm.c
@@ -168,9 +168,9 @@ static const QEMUFileOps bdrv_write_ops = {
 static QEMUFile *qemu_fopen_bdrv(BlockDriverState *bs, int is_writable)
 {
 if (is_writable) {
-return qemu_fopen_ops(bs, &bdrv_write_ops);
+return qemu_fopen_ops(bs, &bdrv_write_ops, false);
 }
-return qemu_fopen_ops(bs, &bdrv_read_ops);
+return qemu_fopen_ops(bs, &bdrv_read

[PATCH v2 0/5] migrations: Fix potential rare race of migration-test after yank

[Peter Xu]

v2:
- Pick r-b for Dave on patch 1/3
- Move migration_file_get_ioc() from patch 5 to patch 4, meanwhile rename it to
  qemu_file_get_ioc(). [Dave]
- Patch 2 "migration: Shutdown src in await_return_path_close_on_source()" is
  replaced by patch "migration: Make from_dst_file accesses thread-safe" [Dave]

Patch 1 fixes a possible race that migration thread can accidentally skip
join() of rp_thread even if the return thread is enabled.  Patch 1 is suspected
to also be the root cause of the recent hard-to-reproduce migration-test
failure here reported by PMM:

https://lore.kernel.org/qemu-devel/YPamXAHwan%2FPPXLf@work-vm/

I didn't reproduce it myself; but after co-debugged with Dave it's suspected
that the race of rp_thread could be the cause.  It's not exposed before because
yank is soo strict on releasing instances, while we're not that strict before,
and didn't join() on rp_thread wasn't so dangerous after all when migration
succeeded before.

Patch 2 fixes another theoretical race on accessing from_dst_file spotted by
Dave.  I don't think there's known issues with it, but may still worth fixing.

Patch 3 should be a cleanup on yank that I think would be nice to have.

Patch 4-5 are further cleanups to remove the ref==1 check in channel_close(),
finally, as I always thought that's a bit hackish.  So I used explicit
unregister of the yank function at necessary places to replace that ref==1 one.

I still think having patch 3-5 altogether would be great, however I think patch
1 should still be the most important to be reviewed.  Also it would be great to
know whether patch 1 could fix the known yank crash.

Please review, thanks.

Peter Xu (5):
  migration: Fix missing join() of rp_thread
  migration: Make from_dst_file accesses thread-safe
  migration: Introduce migration_ioc_[un]register_yank()
  migration: Teach QEMUFile to be QIOChannel-aware
  migration: Move the yank unregister of channel_close out

 migration/channel.c   | 15 ++---
 migration/migration.c | 41 +++---
 migration/migration.h | 15 ++---
 migration/multifd.c   |  8 ++-
 migration/qemu-file-channel.c | 11 ++---
 migration/qemu-file.c | 17 +-
 migration/qemu-file.h |  4 +++-
 migration/ram.c   |  3 ++-
 migration/savevm.c| 11 +++--
 migration/yank_functions.c| 42 +++
 migration/yank_functions.h|  3 +++
 11 files changed, 126 insertions(+), 44 deletions(-)

-- 
2.31.1

[PATCH v2 2/5] migration: Make from_dst_file accesses thread-safe

[Peter Xu]

Accessing from_dst_file is potentially racy in current code base like below:

  if (s->from_dst_file)
do_something(s->from_dst_file);

Because from_dst_file can be reset right after the check in another
thread (rp_thread).  One example is migrate_fd_cancel().

Use the same qemu_file_lock to protect it too, just like to_dst_file.

When it's safe to access without lock, comment it.

There's one special reference in migration_thread() that can be replaced by
the newly introduced rp_thread_created flag.

Reported-by: Dr. David Alan Gilbert 
Signed-off-by: Peter Xu 
---
 migration/migration.c | 32 +---
 migration/migration.h |  8 +---
 migration/ram.c   |  1 +
 3 files changed, 31 insertions(+), 10 deletions(-)

diff --git a/migration/migration.c b/migration/migration.c
index 21b94f75a3..fa70400f98 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -1879,10 +1879,12 @@ static void migrate_fd_cancel(MigrationState *s)
 QEMUFile *f = migrate_get_current()->to_dst_file;
 trace_migrate_fd_cancel();
 
+qemu_mutex_lock(&s->qemu_file_lock);
 if (s->rp_state.from_dst_file) {
 /* shutdown the rp socket, so causing the rp thread to shutdown */
 qemu_file_shutdown(s->rp_state.from_dst_file);
 }
+qemu_mutex_unlock(&s->qemu_file_lock);
 
 do {
 old_state = s->state;
@@ -2686,6 +2688,22 @@ static int migrate_handle_rp_resume_ack(MigrationState 
*s, uint32_t value)
 return 0;
 }
 
+/* Release ms->rp_state.from_dst_file in a safe way */
+static void migration_release_from_dst_file(MigrationState *ms)
+{
+QEMUFile *file = ms->rp_state.from_dst_file;
+
+qemu_mutex_lock(&ms->qemu_file_lock);
+/*
+ * Reset the from_dst_file pointer first before releasing it, as we can't
+ * block within lock section
+ */
+ms->rp_state.from_dst_file = NULL;
+qemu_mutex_unlock(&ms->qemu_file_lock);
+
+qemu_fclose(file);
+}
+
 /*
  * Handles messages sent on the return path towards the source VM
  *
@@ -2827,11 +2845,13 @@ out:
  * Maybe there is something we can do: it looks like a
  * network down issue, and we pause for a recovery.
  */
-qemu_fclose(rp);
-ms->rp_state.from_dst_file = NULL;
+migration_release_from_dst_file(ms);
 rp = NULL;
 if (postcopy_pause_return_path_thread(ms)) {
-/* Reload rp, reset the rest */
+/*
+ * Reload rp, reset the rest.  Referencing it is save since
+ * it's reset only by us above, or when migration completes
+ */
 rp = ms->rp_state.from_dst_file;
 ms->rp_state.error = false;
 goto retry;
@@ -2843,8 +2863,7 @@ out:
 }
 
 trace_source_return_path_thread_end();
-ms->rp_state.from_dst_file = NULL;
-qemu_fclose(rp);
+migration_release_from_dst_file(ms);
 rcu_unregister_thread();
 return NULL;
 }
@@ -2852,7 +2871,6 @@ out:
 static int open_return_path_on_source(MigrationState *ms,
   bool create_thread)
 {
-
 ms->rp_state.from_dst_file = qemu_file_get_return_path(ms->to_dst_file);
 if (!ms->rp_state.from_dst_file) {
 return -1;
@@ -3746,7 +3764,7 @@ static void *migration_thread(void *opaque)
  * If we opened the return path, we need to make sure dst has it
  * opened as well.
  */
-if (s->rp_state.from_dst_file) {
+if (s->rp_state.rp_thread_created) {
 /* Now tell the dest that it should open its end so it can reply */
 qemu_savevm_send_open_return_path(s->to_dst_file);
 
diff --git a/migration/migration.h b/migration/migration.h
index c302879fad..7a5aa8c2fd 100644
--- a/migration/migration.h
+++ b/migration/migration.h
@@ -154,12 +154,13 @@ struct MigrationState {
 QemuThread thread;
 QEMUBH *vm_start_bh;
 QEMUBH *cleanup_bh;
+/* Protected by qemu_file_lock */
 QEMUFile *to_dst_file;
 QIOChannelBuffer *bioc;
 /*
- * Protects to_dst_file pointer.  We need to make sure we won't
- * yield or hang during the critical section, since this lock will
- * be used in OOB command handler.
+ * Protects to_dst_file/from_dst_file pointers.  We need to make sure we
+ * won't yield or hang during the critical section, since this lock will be
+ * used in OOB command handler.
  */
 QemuMutex qemu_file_lock;
 
@@ -192,6 +193,7 @@ struct MigrationState {
 
 /* State related to return path */
 struct {
+/* Protected by qemu_file_lock */
 QEMUFile *from_dst_file;
 QemuThreadrp_thread;
 bool  error;
diff --git a/migration/ram.c b/migration/ram.c
index b5fc454b2f..f728f5072f 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -4012,6 +4012,7 @@ static void ram_dirty_bitmap_reload_notify(MigrationState 
*s)
 int ram_dirty_bitmap_reload(MigrationState 

[PATCH v2 3/5] migration: Introduce migration_ioc_[un]register_yank()

[Peter Xu]

There're plenty of places in migration/* that checks against either socket or
tls typed ioc for yank operations.  Provide two helpers to hide all these
information.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Peter Xu 
---
 migration/channel.c   | 15 ++-
 migration/multifd.c   |  8 ++--
 migration/qemu-file-channel.c |  8 ++--
 migration/yank_functions.c| 28 
 migration/yank_functions.h|  2 ++
 5 files changed, 36 insertions(+), 25 deletions(-)

diff --git a/migration/channel.c b/migration/channel.c
index 01275a9162..c4fc000a1a 100644
--- a/migration/channel.c
+++ b/migration/channel.c
@@ -44,13 +44,7 @@ void migration_channel_process_incoming(QIOChannel *ioc)
  TYPE_QIO_CHANNEL_TLS)) {
 migration_tls_channel_process_incoming(s, ioc, &local_err);
 } else {
-if (object_dynamic_cast(OBJECT(ioc), TYPE_QIO_CHANNEL_SOCKET) ||
-object_dynamic_cast(OBJECT(ioc), TYPE_QIO_CHANNEL_TLS)) {
-yank_register_function(MIGRATION_YANK_INSTANCE,
-   migration_yank_iochannel,
-   QIO_CHANNEL(ioc));
-}
-
+migration_ioc_register_yank(ioc);
 migration_ioc_process_incoming(ioc, &local_err);
 }
 
@@ -94,12 +88,7 @@ void migration_channel_connect(MigrationState *s,
 } else {
 QEMUFile *f = qemu_fopen_channel_output(ioc);
 
-if (object_dynamic_cast(OBJECT(ioc), TYPE_QIO_CHANNEL_SOCKET) ||
-object_dynamic_cast(OBJECT(ioc), TYPE_QIO_CHANNEL_TLS)) {
-yank_register_function(MIGRATION_YANK_INSTANCE,
-   migration_yank_iochannel,
-   QIO_CHANNEL(ioc));
-}
+migration_ioc_register_yank(ioc);
 
 qemu_mutex_lock(&s->qemu_file_lock);
 s->to_dst_file = f;
diff --git a/migration/multifd.c b/migration/multifd.c
index ab41590e71..377da78f5b 100644
--- a/migration/multifd.c
+++ b/migration/multifd.c
@@ -987,12 +987,8 @@ int multifd_load_cleanup(Error **errp)
 for (i = 0; i < migrate_multifd_channels(); i++) {
 MultiFDRecvParams *p = &multifd_recv_state->params[i];
 
-if ((object_dynamic_cast(OBJECT(p->c), TYPE_QIO_CHANNEL_SOCKET) ||
- object_dynamic_cast(OBJECT(p->c), TYPE_QIO_CHANNEL_TLS))
-&& OBJECT(p->c)->ref == 1) {
-yank_unregister_function(MIGRATION_YANK_INSTANCE,
- migration_yank_iochannel,
- QIO_CHANNEL(p->c));
+if (OBJECT(p->c)->ref == 1) {
+migration_ioc_unregister_yank(p->c);
 }
 
 object_unref(OBJECT(p->c));
diff --git a/migration/qemu-file-channel.c b/migration/qemu-file-channel.c
index fad340ea7a..867a5ed0c3 100644
--- a/migration/qemu-file-channel.c
+++ b/migration/qemu-file-channel.c
@@ -107,12 +107,8 @@ static int channel_close(void *opaque, Error **errp)
 int ret;
 QIOChannel *ioc = QIO_CHANNEL(opaque);
 ret = qio_channel_close(ioc, errp);
-if ((object_dynamic_cast(OBJECT(ioc), TYPE_QIO_CHANNEL_SOCKET) ||
- object_dynamic_cast(OBJECT(ioc), TYPE_QIO_CHANNEL_TLS))
-&& OBJECT(ioc)->ref == 1) {
-yank_unregister_function(MIGRATION_YANK_INSTANCE,
- migration_yank_iochannel,
- QIO_CHANNEL(ioc));
+if (OBJECT(ioc)->ref == 1) {
+migration_ioc_unregister_yank(ioc);
 }
 object_unref(OBJECT(ioc));
 return ret;
diff --git a/migration/yank_functions.c b/migration/yank_functions.c
index 96c90e17dc..23697173ae 100644
--- a/migration/yank_functions.c
+++ b/migration/yank_functions.c
@@ -11,6 +11,9 @@
 #include "qapi/error.h"
 #include "io/channel.h"
 #include "yank_functions.h"
+#include "qemu/yank.h"
+#include "io/channel-socket.h"
+#include "io/channel-tls.h"
 
 void migration_yank_iochannel(void *opaque)
 {
@@ -18,3 +21,28 @@ void migration_yank_iochannel(void *opaque)
 
 qio_channel_shutdown(ioc, QIO_CHANNEL_SHUTDOWN_BOTH, NULL);
 }
+
+/* Return whether yank is supported on this ioc */
+static bool migration_ioc_yank_supported(QIOChannel *ioc)
+{
+return object_dynamic_cast(OBJECT(ioc), TYPE_QIO_CHANNEL_SOCKET) ||
+object_dynamic_cast(OBJECT(ioc), TYPE_QIO_CHANNEL_TLS);
+}
+
+void migration_ioc_register_yank(QIOChannel *ioc)
+{
+if (migration_ioc_yank_supported(ioc)) {
+yank_register_function(MIGRATION_YANK_INSTANCE,
+   migration_yank_iochannel,
+   QIO_CHANNEL(ioc));
+}
+}
+
+void migration_ioc_unregister_yank(QIOChannel *ioc)
+{
+if (migration_ioc_yank_supported(ioc)) {
+yank_unregister_function(MIGRATION_YANK_INSTANCE,
+ migration_yank_iochannel,
+ QIO_CHANNEL(ioc));
+

[PATCH v2 1/5] migration: Fix missing join() of rp_thread

[Peter Xu]

It's possible that the migration thread skip the join() of the rp_thread in
below race and crash on src right at finishing migration:

   migration_thread rp_thread
    -
migration_completion()
(before rp_thread quits)
from_dst_file=NULL
[thread got scheduled out]
  s->rp_state.from_dst_file==NULL
(skip join() of rp_thread)
migrate_fd_cleanup()
  qemu_fclose(s->to_dst_file)
  yank_unregister_instance()
assert(yank_find_entry())  <--- crash

It could mostly happen with postcopy, but that shouldn't be required, e.g., I
think it could also trigger with MIGRATION_CAPABILITY_RETURN_PATH set.

It's suspected that above race could be the root cause of a recent (but rare)
migration-test break reported by either Dave or PMM:

https://lore.kernel.org/qemu-devel/YPamXAHwan%2FPPXLf@work-vm/

The issue is: from_dst_file is reset in the rp_thread, so if the thread reset
it to NULL fast enough then the migration thread will assume there's no
rp_thread at all.

This could potentially cause more severe issue (e.g. crash) after the yank code.

Fix it by using a boolean to keep "whether we've created rp_thread".

Cc: Dr. David Alan Gilbert 
Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Peter Xu 
---
 migration/migration.c | 4 +++-
 migration/migration.h | 7 +++
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/migration/migration.c b/migration/migration.c
index 2d306582eb..21b94f75a3 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -2867,6 +2867,7 @@ static int open_return_path_on_source(MigrationState *ms,
 
 qemu_thread_create(&ms->rp_state.rp_thread, "return path",
source_return_path_thread, ms, QEMU_THREAD_JOINABLE);
+ms->rp_state.rp_thread_created = true;
 
 trace_open_return_path_on_source_continue();
 
@@ -2891,6 +2892,7 @@ static int 
await_return_path_close_on_source(MigrationState *ms)
 }
 trace_await_return_path_close_on_source_joining();
 qemu_thread_join(&ms->rp_state.rp_thread);
+ms->rp_state.rp_thread_created = false;
 trace_await_return_path_close_on_source_close();
 return ms->rp_state.error;
 }
@@ -3170,7 +3172,7 @@ static void migration_completion(MigrationState *s)
  * it will wait for the destination to send it's status in
  * a SHUT command).
  */
-if (s->rp_state.from_dst_file) {
+if (s->rp_state.rp_thread_created) {
 int rp_error;
 trace_migration_return_path_end_before();
 rp_error = await_return_path_close_on_source(s);
diff --git a/migration/migration.h b/migration/migration.h
index 2ebb740dfa..c302879fad 100644
--- a/migration/migration.h
+++ b/migration/migration.h
@@ -195,6 +195,13 @@ struct MigrationState {
 QEMUFile *from_dst_file;
 QemuThreadrp_thread;
 bool  error;
+/*
+ * We can also check non-zero of rp_thread, but there's no "official"
+ * way to do this, so this bool makes it slightly more elegant.
+ * Checking from_dst_file for this is racy because from_dst_file will
+ * be cleared in the rp_thread!
+ */
+bool  rp_thread_created;
 QemuSemaphore rp_sem;
 } rp_state;
 
-- 
2.31.1

RE: vulkan support in qemu with virgil

[Kasireddy, Vivek]

Hi Tomeu,
 
> On Wed, Jul 21, 2021 at 03:09:21PM +0200, Tomeu Vizoso wrote:
> > Hi all,
> >
> > At Collabora we have started looking at Vulkan support in QEMU with Virgil.
> >
> > We have seen the work that Vivek has submitted to support the new virtio-gpu
> > BLOB API (thanks!) and have identified a few holes that are still needed for
> > Vulkan support.
> >
> > We would like to know if anybody else is working on Vulkan support or on
> > common tasks such as host-side blobs, CONTEXT_INIT, CROSS_DEVICE,
> > HOST_VISIBLE, venus capsets,
[Kasireddy, Vivek] Nope; I am not aware of anyone working on these features for 
Qemu. Our
focus is mainly on Passthrough (and Headless) GPUs for which Virgl is not 
needed. However,
I am assuming you are already collaborating with CrossVM folks on the 
implementation of
these features in Virtio-gpu Linux Guest driver?

>> a new DisplayChangeListenerOps implementation,
[Kasireddy, Vivek] Could you please elaborate on this? Which backend are you 
planning to
add? I am working on adding a plain and simple Wayland UI backend for Qemu to 
reduce the
GPU Blits:
https://lists.nongnu.org/archive/html/qemu-devel/2021-06/msg06482.html

The above version does not have keyboard and mouse support added but I do have 
a version
which does and it is more or less working. However, the only problem I have run 
into with this
backend is:
https://gitlab.freedesktop.org/wayland/weston/-/issues/514

Not sure if you'd run into a similar issue with your use-case or interested in 
the Wayland UI
backend but I am currently working on trying to come up with a plan to decouple 
OUT_FENCE
signalling from atomic pageflip completion event as suggested by Weston 
upstream (Pekka) to
fix the above issue.

Thanks,
Vivek

> 
> I'm not aware of anyone working on this in qemu specifically.
> The crosvm guys are working on it, not sure what the status is.
> 
> take care,
>   Gerd

Re: [PATCH v7 0/2] support dirtyrate measurement with dirty bitmap

[Peter Xu]

On Tue, Jul 20, 2021 at 11:19:15PM +0800, huang...@chinatelecom.cn wrote:
> From: Hyman Huang(黄勇) 
> 
> v7:
> - do not take bql every time we need when do dirtyrate measuring.
>   try more things once we take bql so that overhead can be reduced.

Reviewed-by: Peter Xu 

-- 
Peter Xu

Re: [PATCH 2/5] migration: Shutdown src in await_return_path_close_on_source()

[Peter Xu]

On Wed, Jul 21, 2021 at 07:00:24PM +0100, Dr. David Alan Gilbert wrote:
> Yes, I think that's best; but try to do as little as possible with the
> lock held.

Yep; non-blocking for the OOB thread.  Will do.

-- 
Peter Xu

Re: [PATCH v2 0/6] python: AQMP-TUI Prototype

[Niteesh G. S.]

On Wed, Jul 21, 2021 at 12:39 AM John Snow  wrote:

>
>
> On Wed, Jul 14, 2021 at 3:07 PM Niteesh G. S. 
> wrote:
>
>> Hello all,
>>
>> UPDATE:  The pipelines have run and all tests passed #336491916
>> Usually, the pipelines start running as soon as I push my code. But this
>> time they took longer to start and there was no sign of starting. This is
>> my
>> first experience with pipelines so I assumed I messed up something from
>> my side.
>>
>> Thanks,
>> Niteesh.
>>
>> On Wed, Jul 14, 2021 at 3:37 AM G S Niteesh Babu 
>> wrote:
>>
>>> GitLab:
>>> https://gitlab.com/niteesh.gs/qemu/-/commits/aqmp-tui-prototype-v1/
>>> Based-on
>>> :
>>> <20210701041313.1696009-1-js...@redhat.com>
>>>  [PATCH 00/20] python: introduce Asynchronous QMP package
>>>
>>> Updates in V2:
>>> 1) Moved loop related initialization to 'run' function in 'App' class
>>> 2) Added a module logger with support in TUI log messages.
>>> 3) Corrected usage of logging.info and logging.debug
>>> 4) Added an option in setup.cfg to silent pylint regarding duplicate-code
>>> 4) Modified the arguments list to the TUI
>>>
>>> NOTE: I am not able to get the pipelines running after the v2 changes.
>>> I was only able to test the changes locally using *make check*.
>>>
>>>
> Why not?
>
I have already updated the status of this
https://lists.gnu.org/archive/html/qemu-devel/2021-07/msg04059.html
All pipelines have run and all passed.

>
>
>> This patch series introduces AQMP-TUI prototype. This prototype has been
>>> helpfull in letting us try out different ideas and giving some insights
>>> into things that we had to take care of in the upcoming TUI. It was also
>>> helpfull in finding out bugs in the AQMP library.
>>>
>>> The intent for this patch series is to get comments on the architectural
>>> design of the prototype. These comments will lay down the foundation for
>>> the upcoming TUI.
>>>
>>> G S Niteesh Babu (6):
>>>   python: disable pylint errors for aqmp-tui
>>>   python: Add dependencies for AQMP TUI
>>>   python/aqmp-tui: Add AQMP TUI draft
>>>   python: add optional pygments dependency
>>>   python/aqmp-tui: add syntax highlighting
>>>   python: add entry point for aqmp-tui
>>>
>>>  python/Pipfile.lock  |  20 ++
>>>  python/qemu/aqmp/aqmp_tui.py | 342 +++
>>>  python/setup.cfg |  36 +++-
>>>  3 files changed, 397 insertions(+), 1 deletion(-)
>>>  create mode 100644 python/qemu/aqmp/aqmp_tui.py
>>>
>>>
> Thanks Niteesh, a few general comments that don't relate directly to the
> code:
>
> 1. It would be nice to be able to highlight/copy-paste things out of the
> history window, I seemingly can't right now.
>
> 2. It would be nice if the mouse scroll wheel worked on the history panel.
>
> 3. A greeting message like the old qmp-shell might be nice to see. It
> would be good if it explained how to quit the program (esc, ctrl^c) and
> send messages (alt+enter).
>
> 4. Some control hints or reminder text in the footer might be nice, for
> how to quit, send a message, etc.
>
I'll update the status here as I start working on them one by one.

For the next revision, I may ask you to start looking into making sure that
> mypy and pylint pass without exemptions. Do the best you can, and get as
> far as you are able. You can leave the warnings disabled for V3, but I'd
> like you to start taking a look now so that you know where the trouble
> spots are.
>
Sure.



> Thanks!
> --js
>

Re: [PATCH 2/5] migration: Shutdown src in await_return_path_close_on_source()

[Dr. David Alan Gilbert]

* Peter Xu (pet...@redhat.com) wrote:
> On Wed, Jul 21, 2021 at 10:55:00AM +0100, Dr. David Alan Gilbert wrote:
> > * Peter Xu (pet...@redhat.com) wrote:
> > > We have a logic in await_return_path_close_on_source() that we will 
> > > explicitly
> > > shutdown the socket when migration encounters errors.  However it could 
> > > be racy
> > > because from_dst_file could have been reset right after checking it but 
> > > before
> > > passing it to qemu_file_shutdown() by the rp_thread.
> > > 
> > > Fix it by shutdown() on the src file instead.  Since they must be a pair 
> > > of
> > > qemu files, shutdown on either of them will work the same.
> > > 
> > > Since at it, drop the check for from_dst_file directly, which makes the
> > > behavior even more predictable.
> > 
> > So while the existing code maybe racy, I'm not sure that this change
> > keeps the semantics; the channel may well have dup()'d the fd's for the
> > two directions, and I'm not convinced that a shutdown() on one will
> > necessarily impact the other; and if the shutdown doesn't happen the
> > rp_thread might not exit, and we might block on the koin.
> 
> dup() seems fine as long as the backend file/socket is the same; but I get the
> point here, e.g., potentially from/to channels can indeed be different ones.
> 
> > 
> > Why don't we solve this a different way - how about we move the:
> > ms->rp_state.from_dst_file = NULL;
> > qemu_fclose(rp);
> > 
> > out of the source_return_path_thread and put it in
> > await_return_path_close_on_source, immediately after the join?
> > Then we *know* that the the rp thread isn't messing with it.
> 
> Yes that looks working for this special case of when rp_thread quits.
> 
> It's just that it'll make things a bit more complicated, previously
> from_dst_file is only reset in rp_thread (for either paused or completed
> migration), now we handle "migration completes" a bit different.
> 
> This also reminded me that maybe we can also use the qemu_file_lock mutex as 
> we
> use to try protect access to to_dst_file, because I think from_dst_file is
> potentially racy in the same way.  Even if we moved the reset to migration
> thread, we still have e.g. migrate_fd_cancel() calling:
> 
> if (s->rp_state.from_dst_file) {
> qemu_file_shutdown(s->rp_state.from_dst_file);
> }
> 
> I think that is also racy too when running in the main thread, as either the
> migration thread or rp_thread could have reset it right after the check.
> 
> So.. maybe I start to use the qemu_file_lock to cover from_dst_file cases too?
> Then I'll cover at leasd both migrate_fd_cancel() and this case.

Yes, I think that's best; but try to do as little as possible with the
lock held.

Dave

> Thanks,
> 
> -- 
> Peter Xu
> 
-- 
Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [PATCH for-6.2 03/23] target/avr: Drop checks for singlestep_enabled

[Philippe Mathieu-Daudé]

+Michael/Alex/Pavel

On 7/21/21 8:41 AM, Richard Henderson wrote:
> GDB single-stepping is now handled generically.
> 
> Signed-off-by: Richard Henderson 
> ---
>  target/avr/translate.c | 19 ---
>  1 file changed, 4 insertions(+), 15 deletions(-)
> 
> diff --git a/target/avr/translate.c b/target/avr/translate.c
> index e08b83..0403470dd8 100644
> --- a/target/avr/translate.c
> +++ b/target/avr/translate.c
> @@ -1089,11 +1089,7 @@ static void gen_goto_tb(DisasContext *ctx, int n, 
> target_ulong dest)
>  tcg_gen_exit_tb(tb, n);
>  } else {
>  tcg_gen_movi_i32(cpu_pc, dest);
> -if (ctx->base.singlestep_enabled) {
> -gen_helper_debug(cpu_env);
> -} else {
> -tcg_gen_lookup_and_goto_ptr();
> -}
> +tcg_gen_lookup_and_goto_ptr();
>  }
>  ctx->base.is_jmp = DISAS_NORETURN;
>  }
> @@ -3011,17 +3007,10 @@ static void avr_tr_tb_stop(DisasContextBase *dcbase, 
> CPUState *cs)
>  tcg_gen_movi_tl(cpu_pc, ctx->npc);
>  /* fall through */
>  case DISAS_LOOKUP:
> -if (!ctx->base.singlestep_enabled) {
> -tcg_gen_lookup_and_goto_ptr();
> -break;
> -}
> -/* fall through */
> +tcg_gen_lookup_and_goto_ptr();
> +break;
>  case DISAS_EXIT:
> -if (ctx->base.singlestep_enabled) {
> -gen_helper_debug(cpu_env);
> -} else {
> -tcg_gen_exit_tb(NULL, 0);
> -}
> +tcg_gen_exit_tb(NULL, 0);
>  break;
>  default:
>  g_assert_not_reached();
> 

Reviewed-by: Philippe Mathieu-Daudé 

Not related to this patch, but looking at the last
gen_helper_debug() use:

/*
 *  The BREAK instruction is used by the On-chip Debug system, and is
 *  normally not used in the application software. When the BREAK
instruction is
 *  executed, the AVR CPU is set in the Stopped Mode. This gives the On-chip
 *  Debugger access to internal resources.  If any Lock bits are set, or
either
 *  the JTAGEN or OCDEN Fuses are unprogrammed, the CPU will treat the BREAK
 *  instruction as a NOP and will not enter the Stopped mode.  This
instruction
 *  is not available in all devices. Refer to the device specific
instruction
 *  set summary.
 */
static bool trans_BREAK(DisasContext *ctx, arg_BREAK *a)
{
if (!avr_have_feature(ctx, AVR_FEATURE_BREAK)) {
return true;
}

#ifdef BREAKPOINT_ON_BREAK
tcg_gen_movi_tl(cpu_pc, ctx->npc - 1);
gen_helper_debug(cpu_env);
ctx->base.is_jmp = DISAS_EXIT;
#else
/* NOP */
#endif

return true;
}

Shouldn't we have a generic 'bool gdbstub_is_attached()' in
"exec/gdbstub.h", then use it in replay_gdb_attached() and
trans_BREAK() instead of this BREAKPOINT_ON_BREAK build-time
definitions?