Re: [Qemu-devel] [PATCH 10/11] qtest.py: Few pylint/style fixes

[Lukáš Doktor]

Dne 20.7.2017 v 20:42 Eduardo Habkost napsal(a):
> On Thu, Jul 20, 2017 at 06:28:14PM +0200, Lukáš Doktor wrote:
> [...]
>> @@ -83,8 +80,11 @@ class QEMUQtestMachine(qemu.QEMUMachine):
>>   socket_scm_helper=None):
>>  if name is None:
>>  name = "qemu-%d" % os.getpid()
>> -super(QEMUQtestMachine, self).__init__(binary, args, name=name, 
>> test_dir=test_dir,
>> -   
>> socket_scm_helper=socket_scm_helper)
>> +scm_helper = socket_scm_helper
> 
> Why is this necessary?
> 
to avoid > 80 chars line. It should be optimized-out by the python compiler so 
it should not slow down the execution. Alternative solution is to use:

super(QEMUQtestMachine,
  self.__init__(...)

which looks IMO uglier, but I can use that in v2, should that be your preferred 
style.

Lukáš

>> +super(QEMUQtestMachine, self).__init__(binary, args, name=name,
>> +   test_dir=test_dir,
>> +   socket_scm_helper=scm_helper)
>> +self._qtest = None
>>  self._qtest_path = os.path.join(test_dir, name + "-qtest.sock")
>>  
>>  def _base_args(self):
>> -- 
>> 2.9.4
>>
> 




signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH 09/11] qmp.py: Avoid overriding a builtin object

[Lukáš Doktor]

Dne 20.7.2017 v 20:38 Eduardo Habkost napsal(a):
> On Thu, Jul 20, 2017 at 06:28:13PM +0200, Lukáš Doktor wrote:
>> The "id" is a builtin method to get object's identity and should not be
>> overridden. This might bring some issues in case someone was directly
>> calling "cmd(..., id=id)" but I haven't found such usage on brief search
>> for "cmd\(.*id=".
>>
>> Signed-off-by: Lukáš Doktor 
>> ---
>>  scripts/qmp/qmp.py | 8 
>>  1 file changed, 4 insertions(+), 4 deletions(-)
>>
>> diff --git a/scripts/qmp/qmp.py b/scripts/qmp/qmp.py
>> index a14b001..c3e0206 100644
>> --- a/scripts/qmp/qmp.py
>> +++ b/scripts/qmp/qmp.py
>> @@ -177,19 +177,19 @@ class QEMUMonitorProtocol(object):
>>  print >>sys.stderr, "QMP:<<< %s" % resp
>>  return resp
>>  
>> -def cmd(self, name, args=None, id=None):
>> +def cmd(self, name, args=None, cmd_id=None):
>>  """
>>  Build a QMP command and send it to the QMP Monitor.
>>  
>>  @param name: command name (string)
>>  @param args: command arguments (dict)
>> -@param id: command id (dict, list, string or int)
>> +@param cmd_id: command id (dict, list, string or int)
>>  """
>>  qmp_cmd = {'execute': name}
>>  if args:
>>  qmp_cmd['arguments'] = args
>> -if id:
>> -qmp_cmd['id'] = id
>> +if cmd_id:
>> +qmp_cmd['cmd_id'] = cmd_id
> 
> The member sent through the monitor should still be called "id".
> i.e.:
> 
> qmp_cmd['id'] = cmd_id
> 
Oups, sorry, automatic rename changed it and I forgot to fix this one back. 
I'll address this in v2. The main problem with this patch is it could break 
named arguments (`cmd(..., id=id)` calls) so I'm not sure it's worth including. 
But as mentioned in commit message I grepped full sources so we better fix this 
before someone starts using it.

Lukáš

> 
>>  return self.cmd_obj(qmp_cmd)
>>  
>>  def command(self, cmd, **kwds):
>> -- 
>> 2.9.4
>>
> 




signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH 07/11] qmp.py: Use object-based class for QEMUMonitorProtocol

[Lukáš Doktor]

Dne 20.7.2017 v 20:35 Eduardo Habkost napsal(a):
> On Thu, Jul 20, 2017 at 06:28:11PM +0200, Lukáš Doktor wrote:
>> There is no need to define QEMUMonitorProtocol as old-style class.
>>
>> Signed-off-by: Lukáš Doktor 
>> ---
>>  scripts/qmp/qmp.py | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/scripts/qmp/qmp.py b/scripts/qmp/qmp.py
>> index bb4d614..68f3420 100644
>> --- a/scripts/qmp/qmp.py
>> +++ b/scripts/qmp/qmp.py
>> @@ -30,7 +30,7 @@ class QMPTimeoutError(QMPError):
>>  pass
>>  
>>  
>> -class QEMUMonitorProtocol:
>> +class QEMUMonitorProtocol(object):
> 
> I don't fully understand the consequences of changing to
> new-style classes when using old-style SuperClass.__init__()
> calls in the __init__().  Should we change QMPShell.__init__() to
> use super()?
> 
The consequence is it becomes a proper object with full object model and less 
workarounds. It also consumes a bit more memory but are the only available mode 
in py3.

As for the old-style superclass, it works, but the correct approach is to 
replace it with `super` call. I'll address that in the v2 (I only checked for 
`.py` files but there are many python sources in qemu tree without the proper 
extension. I still need to get used to this.).

Lukáš

> 
>>  
>>  #: Socket's error class
>>  error = socket.error
>> -- 
>> 2.9.4
>>
> 




signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH qemu] Revert "elf-loader: warn about invalid endianness"

[Philippe Mathieu-Daudé]

Hi Alexey,

On 07/21/2017 01:19 AM, Alexey Kardashevskiy wrote:

This reverts c8e1158cf611 "elf-loader: warn about invalid endianness"
as it produces a useless message every time an LE kernel image is
passed via -kernel on a ppc64-pseries machine. The pseries machine
already checks for ELF_LOAD_WRONG_ENDIAN and tries with big_endian=0.

Signed-off-by: Alexey Kardashevskiy 
---
  hw/core/loader.c | 1 -
  1 file changed, 1 deletion(-)

diff --git a/hw/core/loader.c b/hw/core/loader.c
index c17ace0a2e..e5e8cbb638 100644
--- a/hw/core/loader.c
+++ b/hw/core/loader.c
@@ -480,7 +480,6 @@ int load_elf_ram(const char *filename,
  }
  
  if (target_data_order != e_ident[EI_DATA]) {

-fprintf(stderr, "%s: wrong endianness\n", filename);
  ret = ELF_LOAD_WRONG_ENDIAN;
  goto fail;
  }



I submitted this patch because I spent some time debugging while QEMU 
was failing silently using a MIPS kernel image which used to work, after 
realizing I was in an incorrect build_dir using qemu-system-mipsel to 
load a big endian image and felt stupid [1]. This dumb error can happen 
to other people so I added this warning here.
I was not aware of the ELF_LOAD_WRONG_ENDIAN related code, and at least 
the MIPS arch is not using it.
As I can see in MAINTAINERS, sPAPR is "Supported" meaning "Someone is 
actually paid to look after this", while there is no such paid person 
for the MIPS part.
It seems each arch had a different way to load images and 
hw/core/loader.c was an effort to merge common code but mostly 
"Supported" arch are using it.
While your revert does fixes your sPAPR warning issue, looking at the 
problem roots I think the correct fix is to improve the MIPS port and 
eventually the less loved archs to unify the loader.c calls and avoid 
such problems.
I don't object reverting this patch for 2.10 and improve the loader.c 
usage during 2.11 cycle, I only wonder if this is another 
corporate/hobbyist conflict of interest with corporate crushing on 
hobbyist instead of helping, motivating contribution improving common 
code usage.


Cc'ed MIPS and loader.c maintainers (both "Maintained" and not "Supported").

Phil.

[1] http://lists.nongnu.org/archive/html/qemu-devel/2017-06/msg05926.html

Re: [Qemu-devel] [PATCH 5/5] qtest: Document calling conventions

[Markus Armbruster]

Eric Blake  writes:

> On 07/20/2017 03:37 PM, Eric Blake wrote:
 + * @fmt...: QMP message to send to qemu; only recognizes formats
 + * understood by json-lexer.c
   *
   * Sends a QMP message to QEMU and consumes the response.
   */
>>>
>>> These formats are chosen so that gcc can help us check them.  Please add
>>> GCC_FMT_ATTR().  Precedence: qobject_from_jsonf().
>> 
>> Will do.  (This patch was originally part of my larger series trying to
>> get rid of qobject_from_jsonf() - I may still succeed at that, but
>> separating the easy stuff from the controversial means that the easy
>> stuff needs tweaking).
>
> Bleargh.  It's not that simple.
>
> With printf-style, hmp("literal") and hmp("%s", "literal") produce the
> same results.
>
> But with json-lexer style, %s MODIFIES its input.

Your assertion "MODIFIES its input" confused me briefly, because I read
it as "side effect on the argument string".  That would be bonkers.
What you mean is it doesn't emit the argument string unmodified.

> The original qmp("{'execute':\"foo\"}") sends a JSON object
>  {'execute':"foo"}
> but counterpart qmp("%s", "{'execute':'foo'}") sends a JSON string with
> added \ escaping:
>  "{'execute':\"foo\"}"
>
> So adding the format immediately causes lots of warnings, such as:
>
>   CC  tests/vhost-user-test.o
> tests/vhost-user-test.c: In function ‘test_migrate’:
> tests/vhost-user-test.c:668:5: error: format not a string literal and no
> format arguments [-Werror=format-security]
>  rsp = qmp(cmd);
>  ^~~

cmd = g_strdup_printf("{ 'execute': 'migrate',"
  "'arguments': { 'uri': '%s' } }",
  uri);
rsp = qmp(cmd);
g_free(cmd);
g_assert(qdict_haskey(rsp, "return"));
QDECREF(rsp);

For this to work, @uri must not contain funny characters.

Shouldn't we simply use the built-in quoting here?

rsp = qmp("{ 'execute': 'migrate', 'arguments': { 'uri': %s } }", uri);
g_assert(qdict_haskey(rsp, "return"));
QDECREF(rsp);

>
>   CC  tests/boot-order-test.o
> tests/boot-order-test.c: In function ‘test_a_boot_order’:
> tests/boot-order-test.c:46:26: error: zero-length gnu_printf format
> string [-Werror=format-zero-length]
>  qmp_discard_response("");   /* HACK: wait for event */
>   ^~

Should use qmp_eventwait().  Doesn't because it predates it.

> but we CAN'T rewrite those to qmp("%s", command).

Got more troublemakers?

> Now you can sort-of understand why my larger series wanted to get rid of
> qobject_from_jsonf(), since our use of GCC_FMT_ATTR() there is a lie?

I don't think it's a lie.  qobject_from_jsonf() satisfies the attribute
format(printf, ...) contract: it fetches varargs exactly like printf().
What it does with them is of no concern to this attribute.

Re: [Qemu-devel] [PATCH 05/11] qemu.py: Use custom exceptions rather than Exception

[Lukáš Doktor]

Dne 20.7.2017 v 20:27 Eduardo Habkost napsal(a):
> On Thu, Jul 20, 2017 at 06:28:09PM +0200, Lukáš Doktor wrote:
>> The naked Exception should not be widely used. It makes sense to be a
>> bit more specific and use better-suited custom exceptions. As a benefit
>> we can store the full reply in the exception in case someone needs it
>> when catching the exception.
>>
>> Signed-off-by: Lukáš Doktor 
>> ---
>>  scripts/qemu.py | 17 +++--
>>  1 file changed, 15 insertions(+), 2 deletions(-)
>>
>> diff --git a/scripts/qemu.py b/scripts/qemu.py
>> index 5948e19..2bd522f 100644
>> --- a/scripts/qemu.py
>> +++ b/scripts/qemu.py
>> @@ -19,6 +19,19 @@ import subprocess
>>  import qmp.qmp
>>  
>>  
>> +class MonitorResponseError(qmp.qmp.QMPError):
>> +'''
>> +Represents erroneous QMP monitor reply
>> +'''
>> +def __init__(self, reply):
>> +try:
>> +desc = reply["error"]["desc"]
>> +except KeyError:
>> +desc = reply
>> +super(MonitorResponseError, self).__init__(desc)
>> +self.reply = reply
> 
> This would require every user of self.reply to first check if
> it's a string or dictionary.  All because of the "Monitor is
> closed" case below:
> 
I haven't used it for the `Monitor is closed` exception, so it's just to be 
able to store really broken reply safely. Anyway people can check whether the 
reply is a dict, or I can add `is_valid_reply` property which would check for 
`[error][desc]` presence (which is a bit more precise than just plain dict type 
checking).

>> +
>> +
>>  class QEMUMachine(object):
>>  '''A QEMU VM'''
>>  
>> @@ -197,9 +210,9 @@ class QEMUMachine(object):
>>  '''
>>  reply = self.qmp(cmd, conv_keys, **args)
>>  if reply is None:
>> -raise Exception("Monitor is closed")
>> +raise qmp.qmp.QMPError("Monitor is closed")
> 
> Should we really use the same exception type for this, if it's
> not really a QMP monitor error reply, and won't even have a real
> QMP reply in self.reply?
> 
I wasn't sure but the same exception can be caused by other failures when 
obtaining replies so I think in case the monitor is closed we might expect the 
same exception. Would importing it in the top level of this module to become 
`qemu.QMPError` work for you better, or would you prefer IOError, RuntimeError 
or another custom exception?

Lukáš

> 
>>  if "error" in reply:
>> -raise Exception(reply["error"]["desc"])
>> +raise MonitorResponseError(reply)
>>  return reply["return"]
>>  
>>  def get_qmp_event(self, wait=False):
>> -- 
>> 2.9.4
>>
> 




signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v2 1/8] usb-ccid: Add support to dump all USB packets

[Gerd Hoffmann]

> index bef1f03c42..624dc2c447 100644
> --- a/hw/usb/dev-smartcard-reader.c
> +++ b/hw/usb/dev-smartcard-reader.c
> @@ -54,9 +54,25 @@ do { \
>  #define D_INFO 2
>  #define D_MORE_INFO 3
>  #define D_VERBOSE 4
> +#define D_TRACE 5
> +#define D_REMOTEIO 10

Considered converting all DPRINTFs into tracepoints instead?

Then you can turn then on/off individually as needed.

>  #define CCID_DEV_NAME "usb-ccid"
>  #define USB_CCID_DEV(obj) OBJECT_CHECK(USBCCIDState, (obj),
> CCID_DEV_NAME)
> +
> +static void usb_packet_dump(int lvl, const char *dir, uint8_t *buf,
> size_t len)
> +{
> +int i;
> +if (lvl < D_TRACE) {
> +return;
> +}
> +printf("usb-ccid: usb packet(%s/%zd):", dir, len);
> +for (i = 0; i < len; ++i) {
> +printf(" %02x", buf[i]);
> +}
> +printf("\n");
> +}

There is qemu_hexdump() ...

cheers,
  Gerd

Re: [Qemu-devel] [PATCH qemu] Revert "elf-loader: warn about invalid endianness"

[Philippe Mathieu-Daudé]

Hi Jason, Michael,

On 07/21/2017 01:55 AM, no-re...@patchew.org wrote:

This series failed automatic build test. Please find the testing commands and
their output below. If you have docker installed, you can probably reproduce it
locally.

[...]

GTester: last random seed: R02Sd7d1d0ad279b9a1e69acb248301b1ab6
Vhost user backend fails to broadcast fake RARP
make: *** [check-tests/test-replication] Error 1

Any idea about this error?

Re: [Qemu-devel] [PATCH v5 17/17] migration: Flush receive queue

[Peter Xu]

On Mon, Jul 17, 2017 at 03:42:38PM +0200, Juan Quintela wrote:
> Each time that we sync the bitmap, it is a possiblity that we receive
> a page that is being processed by a different thread.  We fix this
> problem just making sure that we wait for all receiving threads to
> finish its work before we procedeed with the next stage.
> 
> We are low on page flags, so we use a combination that is not valid to
> emit that message:  MULTIFD_PAGE and COMPRESSED.

Btw, would it be possible that we introduce a new QEMU_VM_COMMAND for
this flush operation?  Like: MIG_CMD_MULTIFD_FLUSH?

-- 
Peter Xu

[Qemu-devel] [PATCH v4 43/43] tcg: enable multiple TCG contexts in softmmu

[Emilio G. Cota]

This enables parallel TCG code generation. However, we do not take
advantage of it yet since tb_lock is still held during tb_gen_code.

In user-mode we use a single TCG context; see the documentation
added to tcg_region_init for the rationale.

Note that targets do not need any conversion: targets initialize a
TCGContext (e.g. defining TCG globals), and after this initialization
has finished, the context is cloned by the vCPU threads, each of
them keeping a separate copy.

TCG threads claim one entry in tcg_ctxs[] by atomically increasing
n_tcg_ctxs. Do not be too annoyed by the subsequent atomic_read's
of that variable and tcg_ctxs; they are there just to play nice with
analysis tools such as thread sanitizer.

Note that we do not allocate an array of contexts (we allocate
an array of pointers instead) because when tcg_context_init
is called, we do not know yet how many contexts we'll use since
the bool behind qemu_tcg_mttcg_enabled() isn't set yet.

Previous patches folded some TCG globals into TCGContext. The non-const
globals remaining are only set at init time, i.e. before the TCG
threads are spawned. Here is a list of these set-at-init-time globals
under tcg/:

Only written by tcg_context_init:
- indirect_reg_alloc_order
- tcg_op_defs
Only written by tcg_target_init (called from tcg_context_init):
- tcg_target_available_regs
- tcg_target_call_clobber_regs
- arm: arm_arch, use_idiv_instructions
- i386: have_cmov, have_bmi1, have_bmi2, have_lzcnt,
have_movbe, have_popcnt
- mips: use_movnz_instructions, use_mips32_instructions,
use_mips32r2_instructions, got_sigill (tcg_target_detect_isa)
- ppc: have_isa_2_06, have_isa_3_00, tb_ret_addr
- s390: tb_ret_addr, s390_facilities
- sparc: qemu_ld_trampoline, qemu_st_trampoline (build_trampolines),
 use_vis3_instructions

Only written by tcg_prologue_init:
- 'struct jit_code_entry one_entry'
- aarch64: tb_ret_addr
- arm: tb_ret_addr
- i386: tb_ret_addr, guest_base_flags
- ia64: tb_ret_addr
- mips: tb_ret_addr, bswap32_addr, bswap32u_addr, bswap64_addr

Reviewed-by: Richard Henderson 
Signed-off-by: Emilio G. Cota 
---
 tcg/tcg.h |   7 ++-
 accel/tcg/translate-all.c |   2 +-
 cpus.c|   2 +
 linux-user/syscall.c  |   1 +
 tcg/tcg.c | 137 +++---
 5 files changed, 136 insertions(+), 13 deletions(-)

diff --git a/tcg/tcg.h b/tcg/tcg.h
index 7413aa0..aaf447e 100644
--- a/tcg/tcg.h
+++ b/tcg/tcg.h
@@ -733,7 +733,7 @@ struct TCGContext {
 };
 
 extern TCGContext tcg_init_ctx;
-extern TCGContext *tcg_ctx;
+extern __thread TCGContext *tcg_ctx;
 
 static inline void tcg_set_insn_param(int op_idx, int arg, TCGArg v)
 {
@@ -755,7 +755,7 @@ static inline bool tcg_op_buf_full(void)
 
 /* pool based memory allocation */
 
-/* tb_lock must be held for tcg_malloc_internal. */
+/* user-mode: tb_lock must be held for tcg_malloc_internal. */
 void *tcg_malloc_internal(TCGContext *s, int size);
 void tcg_pool_reset(TCGContext *s);
 TranslationBlock *tcg_tb_alloc(TCGContext *s);
@@ -766,7 +766,7 @@ void tcg_region_reset_all(void);
 size_t tcg_code_size(void);
 size_t tcg_code_capacity(void);
 
-/* Called with tb_lock held.  */
+/* user-mode: Called with tb_lock held.  */
 static inline void *tcg_malloc(int size)
 {
 TCGContext *s = tcg_ctx;
@@ -783,6 +783,7 @@ static inline void *tcg_malloc(int size)
 }
 
 void tcg_context_init(TCGContext *s);
+void tcg_register_thread(void);
 void tcg_prologue_init(TCGContext *s);
 void tcg_func_start(TCGContext *s);
 
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index 9be584b..db3d42c 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -154,7 +154,7 @@ static void *l1_map[V_L1_MAX_SIZE];
 
 /* code generation context */
 TCGContext tcg_init_ctx;
-TCGContext *tcg_ctx;
+__thread TCGContext *tcg_ctx;
 TBContext tb_ctx;
 bool parallel_cpus;
 
diff --git a/cpus.c b/cpus.c
index 6022d40..74ddd49 100644
--- a/cpus.c
+++ b/cpus.c
@@ -1307,6 +1307,7 @@ static void *qemu_tcg_rr_cpu_thread_fn(void *arg)
 CPUState *cpu = arg;
 
 rcu_register_thread();
+tcg_register_thread();
 
 qemu_mutex_lock_iothread();
 qemu_thread_get_self(cpu->thread);
@@ -1454,6 +1455,7 @@ static void *qemu_tcg_cpu_thread_fn(void *arg)
 g_assert(!use_icount);
 
 rcu_register_thread();
+tcg_register_thread();
 
 qemu_mutex_lock_iothread();
 qemu_thread_get_self(cpu->thread);
diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index 003943b..bbf7913 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -6214,6 +6214,7 @@ static void *clone_func(void *arg)
 TaskState *ts;
 
 rcu_register_thread();
+tcg_register_thread();
 env = info->env;
 cpu = ENV_GET_CPU(env);
 thread_cpu = cpu;
diff --git a/tcg/tcg.c b/tcg/tcg.c
index c77654f..9a0ed00 100644
--- a/tcg/tcg.c
+++ b/tcg/tcg.c
@@ -58,6 +58,7 @@
 
 #include "elf.h"
 #include "exec/log.h"
+#include "

[Qemu-devel] [PATCH v4 11/43] tcg: define CF_PARALLEL and use it for TB hashing along with CF_COUNT_MASK

[Emilio G. Cota]

This will enable us to decouple code translation from the value
of parallel_cpus at any given time. It will also help us minimize
TB flushes when generating code via EXCP_ATOMIC.

Note that the declaration of parallel_cpus is brought to exec-all.h
to be able to define there the "curr_cflags" inline.

Signed-off-by: Emilio G. Cota 
---
 include/exec/exec-all.h   | 20 +++-
 include/exec/tb-hash-xx.h |  9 ++---
 include/exec/tb-hash.h|  4 ++--
 include/exec/tb-lookup.h  |  6 +++---
 tcg/tcg.h |  1 -
 accel/tcg/cpu-exec.c  | 45 +++--
 accel/tcg/translate-all.c | 13 +
 exec.c|  2 +-
 tcg/tcg-runtime.c |  2 +-
 tests/qht-bench.c |  2 +-
 10 files changed, 65 insertions(+), 39 deletions(-)

diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
index 8cbd90b..f6a928d 100644
--- a/include/exec/exec-all.h
+++ b/include/exec/exec-all.h
@@ -353,6 +353,9 @@ struct TranslationBlock {
 #define CF_USE_ICOUNT  0x2
 #define CF_IGNORE_ICOUNT 0x4 /* Do not generate icount code */
 #define CF_INVALID 0x8 /* TB is stale. Setters must acquire tb_lock */
+#define CF_PARALLEL0x10 /* Generate code for a parallel context */
+/* cflags' mask for hashing/comparison */
+#define CF_HASH_MASK (CF_COUNT_MASK | CF_PARALLEL)
 
 /* Per-vCPU dynamic tracing state used to generate this TB */
 uint32_t trace_vcpu_dstate;
@@ -396,11 +399,26 @@ struct TranslationBlock {
 uintptr_t jmp_list_first;
 };
 
+extern bool parallel_cpus;
+
+/* Hide the atomic_read to make code a little easier on the eyes */
+static inline uint32_t tb_cflags(const TranslationBlock *tb)
+{
+return atomic_read(&tb->cflags);
+}
+
+/* current cflags for hashing/comparison */
+static inline uint32_t curr_cflags(void)
+{
+return parallel_cpus ? CF_PARALLEL : 0;
+}
+
 void tb_free(TranslationBlock *tb);
 void tb_flush(CPUState *cpu);
 void tb_phys_invalidate(TranslationBlock *tb, tb_page_addr_t page_addr);
 TranslationBlock *tb_htable_lookup(CPUState *cpu, target_ulong pc,
-   target_ulong cs_base, uint32_t flags);
+   target_ulong cs_base, uint32_t flags,
+   uint32_t cf_mask);
 
 #if defined(USE_DIRECT_JUMP)
 
diff --git a/include/exec/tb-hash-xx.h b/include/exec/tb-hash-xx.h
index 6cd3022..747a9a6 100644
--- a/include/exec/tb-hash-xx.h
+++ b/include/exec/tb-hash-xx.h
@@ -48,8 +48,8 @@
  * xxhash32, customized for input variables that are not guaranteed to be
  * contiguous in memory.
  */
-static inline
-uint32_t tb_hash_func6(uint64_t a0, uint64_t b0, uint32_t e, uint32_t f)
+static inline uint32_t
+tb_hash_func7(uint64_t a0, uint64_t b0, uint32_t e, uint32_t f, uint32_t g)
 {
 uint32_t v1 = TB_HASH_XX_SEED + PRIME32_1 + PRIME32_2;
 uint32_t v2 = TB_HASH_XX_SEED + PRIME32_2;
@@ -78,7 +78,7 @@ uint32_t tb_hash_func6(uint64_t a0, uint64_t b0, uint32_t e, 
uint32_t f)
 v4 *= PRIME32_1;
 
 h32 = rol32(v1, 1) + rol32(v2, 7) + rol32(v3, 12) + rol32(v4, 18);
-h32 += 24;
+h32 += 28;
 
 h32 += e * PRIME32_3;
 h32  = rol32(h32, 17) * PRIME32_4;
@@ -86,6 +86,9 @@ uint32_t tb_hash_func6(uint64_t a0, uint64_t b0, uint32_t e, 
uint32_t f)
 h32 += f * PRIME32_3;
 h32  = rol32(h32, 17) * PRIME32_4;
 
+h32 += g * PRIME32_3;
+h32  = rol32(h32, 17) * PRIME32_4;
+
 h32 ^= h32 >> 15;
 h32 *= PRIME32_2;
 h32 ^= h32 >> 13;
diff --git a/include/exec/tb-hash.h b/include/exec/tb-hash.h
index 17b5ee0..0526c4f 100644
--- a/include/exec/tb-hash.h
+++ b/include/exec/tb-hash.h
@@ -59,9 +59,9 @@ static inline unsigned int 
tb_jmp_cache_hash_func(target_ulong pc)
 
 static inline
 uint32_t tb_hash_func(tb_page_addr_t phys_pc, target_ulong pc, uint32_t flags,
-  uint32_t trace_vcpu_dstate)
+  uint32_t cf_mask, uint32_t trace_vcpu_dstate)
 {
-return tb_hash_func6(phys_pc, pc, flags, trace_vcpu_dstate);
+return tb_hash_func7(phys_pc, pc, flags, cf_mask, trace_vcpu_dstate);
 }
 
 #endif
diff --git a/include/exec/tb-lookup.h b/include/exec/tb-lookup.h
index 436b6d5..2961385 100644
--- a/include/exec/tb-lookup.h
+++ b/include/exec/tb-lookup.h
@@ -21,7 +21,7 @@
 /* Might cause an exception, so have a longjmp destination ready */
 static inline TranslationBlock *
 tb_lookup__cpu_state(CPUState *cpu, target_ulong *pc, target_ulong *cs_base,
- uint32_t *flags)
+ uint32_t *flags, uint32_t cf_mask)
 {
 CPUArchState *env = (CPUArchState *)cpu->env_ptr;
 TranslationBlock *tb;
@@ -35,10 +35,10 @@ tb_lookup__cpu_state(CPUState *cpu, target_ulong *pc, 
target_ulong *cs_base,
tb->cs_base == *cs_base &&
tb->flags == *flags &&
tb->trace_vcpu_dstate == *cpu->trace_dstate &&
-   !(atomic_read(&tb->cflags) & CF_INVALID))) {
+   (t

[Qemu-devel] [PATCH v4 20/43] tcg: check CF_PARALLEL instead of parallel_cpus

[Emilio G. Cota]

Thereby decoupling the resulting translated code from the current state
of the system.

The tb->cflags field is not passed to tcg generation functions. So
we add a field to TCGContext, storing there a copy of tb->cflags.

Most architectures have <= 32 registers, which results in a 4-byte hole
in TCGContext. Use this hole for the new field.

Reviewed-by: Richard Henderson 
Signed-off-by: Emilio G. Cota 
---
 tcg/tcg.h |  1 +
 accel/tcg/translate-all.c |  1 +
 tcg/tcg-op.c  | 10 +-
 3 files changed, 7 insertions(+), 5 deletions(-)

diff --git a/tcg/tcg.h b/tcg/tcg.h
index 96872f8..ef1760a 100644
--- a/tcg/tcg.h
+++ b/tcg/tcg.h
@@ -656,6 +656,7 @@ struct TCGContext {
 uintptr_t *tb_jmp_target_addr; /* tb->jmp_target_addr if !USE_DIRECT_JUMP 
*/
 
 TCGRegSet reserved_regs;
+uint32_t tb_cflags; /* cflags of the current TB */
 intptr_t current_frame_offset;
 intptr_t frame_start;
 intptr_t frame_end;
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index c1ce38f..227b566 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -1271,6 +1271,7 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
 tb->flags = flags;
 tb->cflags = cflags;
 tb->trace_vcpu_dstate = *cpu->trace_dstate;
+tcg_ctx.tb_cflags = cflags;
 
 #ifdef CONFIG_PROFILER
 tcg_ctx.tb_count1++; /* includes aborted translations because of
diff --git a/tcg/tcg-op.c b/tcg/tcg-op.c
index 205d07f..5580789 100644
--- a/tcg/tcg-op.c
+++ b/tcg/tcg-op.c
@@ -150,7 +150,7 @@ void tcg_gen_op6(TCGContext *ctx, TCGOpcode opc, TCGArg a1, 
TCGArg a2,
 
 void tcg_gen_mb(TCGBar mb_type)
 {
-if (parallel_cpus) {
+if (tcg_ctx.tb_cflags & CF_PARALLEL) {
 tcg_gen_op1(&tcg_ctx, INDEX_op_mb, mb_type);
 }
 }
@@ -2794,7 +2794,7 @@ void tcg_gen_atomic_cmpxchg_i32(TCGv_i32 retv, TCGv addr, 
TCGv_i32 cmpv,
 {
 memop = tcg_canonicalize_memop(memop, 0, 0);
 
-if (!parallel_cpus) {
+if (!(tcg_ctx.tb_cflags & CF_PARALLEL)) {
 TCGv_i32 t1 = tcg_temp_new_i32();
 TCGv_i32 t2 = tcg_temp_new_i32();
 
@@ -2838,7 +2838,7 @@ void tcg_gen_atomic_cmpxchg_i64(TCGv_i64 retv, TCGv addr, 
TCGv_i64 cmpv,
 {
 memop = tcg_canonicalize_memop(memop, 1, 0);
 
-if (!parallel_cpus) {
+if (!(tcg_ctx.tb_cflags & CF_PARALLEL)) {
 TCGv_i64 t1 = tcg_temp_new_i64();
 TCGv_i64 t2 = tcg_temp_new_i64();
 
@@ -3015,7 +3015,7 @@ static void * const table_##NAME[16] = {  
  \
 void tcg_gen_atomic_##NAME##_i32\
 (TCGv_i32 ret, TCGv addr, TCGv_i32 val, TCGArg idx, TCGMemOp memop) \
 {   \
-if (parallel_cpus) {\
+if (tcg_ctx.tb_cflags & CF_PARALLEL) {  \
 do_atomic_op_i32(ret, addr, val, idx, memop, table_##NAME); \
 } else {\
 do_nonatomic_op_i32(ret, addr, val, idx, memop, NEW,\
@@ -3025,7 +3025,7 @@ void tcg_gen_atomic_##NAME##_i32  
  \
 void tcg_gen_atomic_##NAME##_i64\
 (TCGv_i64 ret, TCGv addr, TCGv_i64 val, TCGArg idx, TCGMemOp memop) \
 {   \
-if (parallel_cpus) {\
+if (tcg_ctx.tb_cflags & CF_PARALLEL) {  \
 do_atomic_op_i64(ret, addr, val, idx, memop, table_##NAME); \
 } else {\
 do_nonatomic_op_i64(ret, addr, val, idx, memop, NEW,\
-- 
2.7.4

[Qemu-devel] [PATCH v4 00/43] tcg: support for multiple TCG contexts

[Emilio G. Cota]

v3:
  https://lists.gnu.org/archive/html/qemu-devel/2017-07/msg06353.html

To ease review/testing, you can pull this series from:
  https://github.com/cota/qemu/tree/multi-tcg-v4
  [ head commit: 1d50a9f24e ]

In this iteration I'm sending only the few patches that contain changes
from v3; they are highlighted in the change list below.

Changes from v3:
- Add R-b tags
- Rebase on top of current master (25d0233c1)
- [PATCH 11] tb->cflags hashing: add CF_COUNT_MASK to CF_HASH_MASK
  - [PATCH 20] tcg_ctx.cf_parallel: convert to tcg_ctx.tb_cflags (Richard:
kept your R-b)
- [PATCH 35] TCG optimizer: allocate using tcg_malloc (Richard: dropped
  your R-b)
- [PATCH 42] TCG regions:
  - expand first and last regions to fully use the buffer
- use tcg_region_bounds as suggested by Richard
- substitute regions.n_full with region.agg_full_size since now
  not all regions are of the same size
  - add region.stride and .start_aligned fields
- Only use qemu_real_host_page_size once in the file; guard size
  is derived as 'region.stride - region.size'
- [PATCH 43] TCG __thread: (Richard: kept your R-b)
  - do not assume CONFIG_SOFTMMU == !CONFIG_USER_ONLY
  - use atomic_read/set on tcg_ctxs[] as well
  - Remove unnecessary clearing of tcg_ctx->prof in tcg_register_thread()

To be done after this series:
- Get rid of tb_lock, or at least push it down so that we take advantage of
  multiple TCG contexts in MTTCG. (I'm doing this in my testing, but doing
  it well will require another patch series.)

Improvements that were suggested during this series' development:
- Perhaps look at arranging fields such that all the fields that are
  "shared" between the contexts are up front, and use the qemu standard
  'memcpy(new, old, offsetof(TCGContext, end_common_fields));' trick, and
  zero the rest.
- Order tb->[*] comparisons by likelihood of mismatch.
- Get rid of parallel_cpus from from cpu_exec_step_atomic -- I'm not sure
  whether just removing it is safe, since we call curr_cflags from several
  places.
- Add CF_NOCHAIN flag, that tcg-op can check via tcg_ctx->tb_cflags.
- Expand CF_HASH_MASK to compare all bits in cflags. Would require cleaning
  CF_IGNORE_ICOUNT up.
- Perhaps parse -accel=tcg command-line arguments before TCG is initialized,
  so that those arguments can be used during TCG initialization.

Thanks,

Emilio

[Qemu-devel] [PATCH v4 42/43] tcg: introduce regions to split code_gen_buffer

[Emilio G. Cota]

This is groundwork for supporting multiple TCG contexts.

The naive solution here is to split code_gen_buffer statically
among the TCG threads; this however results in poor utilization
if translation needs are different across TCG threads.

What we do here is to add an extra layer of indirection, assigning
regions that act just like pages do in virtual memory allocation.
(BTW if you are wondering about the chosen naming, I did not want
to use blocks or pages because those are already heavily used in QEMU).

We use a global lock to serialize allocations as well as statistics
reporting (we now export the size of the used code_gen_buffer with
tcg_code_size()). Note that for the allocator we could just use
a counter and atomic_inc; however, that would complicate the gathering
of tcg_code_size()-like stats. So given that the region operations are
not a fast path, a lock seems the most reasonable choice.

The effectiveness of this approach is clear after seeing some numbers.
I used the bootup+shutdown of debian-arm with '-tb-size 80' as a benchmark.
Note that I'm evaluating this after enabling per-thread TCG (which
is done by a subsequent commit).

* -smp 1, 1 region (entire buffer):
qemu: flush code_size=83885014 nb_tbs=154739 avg_tb_size=357
qemu: flush code_size=83884902 nb_tbs=153136 avg_tb_size=363
qemu: flush code_size=83885014 nb_tbs=152777 avg_tb_size=364
qemu: flush code_size=83884950 nb_tbs=150057 avg_tb_size=373
qemu: flush code_size=83884998 nb_tbs=150234 avg_tb_size=373
qemu: flush code_size=83885014 nb_tbs=154009 avg_tb_size=360
qemu: flush code_size=83885014 nb_tbs=151007 avg_tb_size=370
qemu: flush code_size=83885014 nb_tbs=151816 avg_tb_size=367

That is, 8 flushes.

* -smp 8, 32 regions (80/32 MB per region) [i.e. this patch]:

qemu: flush code_size=76328008 nb_tbs=141040 avg_tb_size=356
qemu: flush code_size=75366534 nb_tbs=138000 avg_tb_size=361
qemu: flush code_size=76864546 nb_tbs=140653 avg_tb_size=361
qemu: flush code_size=76309084 nb_tbs=135945 avg_tb_size=375
qemu: flush code_size=74581856 nb_tbs=132909 avg_tb_size=375
qemu: flush code_size=73927256 nb_tbs=135616 avg_tb_size=360
qemu: flush code_size=78629426 nb_tbs=142896 avg_tb_size=365
qemu: flush code_size=76667052 nb_tbs=138508 avg_tb_size=368

Again, 8 flushes. Note how buffer utilization is not 100%, but it
is close. Smaller region sizes would yield higher utilization,
but we want region allocation to be rare (it acquires a lock), so
we do not want to go too small.

* -smp 8, static partitioning of 8 regions (10 MB per region):
qemu: flush code_size=21936504 nb_tbs=40570 avg_tb_size=354
qemu: flush code_size=11472174 nb_tbs=20633 avg_tb_size=370
qemu: flush code_size=11603976 nb_tbs=21059 avg_tb_size=365
qemu: flush code_size=23254872 nb_tbs=41243 avg_tb_size=377
qemu: flush code_size=28289496 nb_tbs=52057 avg_tb_size=358
qemu: flush code_size=43605160 nb_tbs=78896 avg_tb_size=367
qemu: flush code_size=45166552 nb_tbs=82158 avg_tb_size=364
qemu: flush code_size=63289640 nb_tbs=116494 avg_tb_size=358
qemu: flush code_size=51389960 nb_tbs=93937 avg_tb_size=362
qemu: flush code_size=59665928 nb_tbs=107063 avg_tb_size=372
qemu: flush code_size=38380824 nb_tbs=68597 avg_tb_size=374
qemu: flush code_size=44884568 nb_tbs=79901 avg_tb_size=376
qemu: flush code_size=50782632 nb_tbs=90681 avg_tb_size=374
qemu: flush code_size=3984 nb_tbs=71433 avg_tb_size=372
qemu: flush code_size=64708840 nb_tbs=119052 avg_tb_size=359
qemu: flush code_size=49830008 nb_tbs=90992 avg_tb_size=362
qemu: flush code_size=68372408 nb_tbs=123442 avg_tb_size=368
qemu: flush code_size=3360 nb_tbs=59514 avg_tb_size=378
qemu: flush code_size=44748344 nb_tbs=80974 avg_tb_size=367
qemu: flush code_size=37104248 nb_tbs=67609 avg_tb_size=364

That is, 20 flushes. Note how a static partitioning approach uses
the code buffer poorly, leading to many unnecessary flushes.

Signed-off-by: Emilio G. Cota 
---
 tcg/tcg.h |   6 ++
 accel/tcg/translate-all.c |  63 +
 bsd-user/main.c   |   1 +
 cpus.c|  12 +++
 linux-user/main.c |   1 +
 tcg/tcg.c | 222 +-
 6 files changed, 260 insertions(+), 45 deletions(-)

diff --git a/tcg/tcg.h b/tcg/tcg.h
index 9c01fe6..7413aa0 100644
--- a/tcg/tcg.h
+++ b/tcg/tcg.h
@@ -760,6 +760,12 @@ void *tcg_malloc_internal(TCGContext *s, int size);
 void tcg_pool_reset(TCGContext *s);
 TranslationBlock *tcg_tb_alloc(TCGContext *s);
 
+void tcg_region_init(void);
+void tcg_region_reset_all(void);
+
+size_t tcg_code_size(void);
+size_t tcg_code_capacity(void);
+
 /* Called with tb_lock held.  */
 static inline void *tcg_malloc(int size)
 {
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index 1a98f1a..9be584b 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/tra

[Qemu-devel] [PATCH v4 35/43] tcg: allocate optimizer temps with tcg_malloc

[Emilio G. Cota]

Groundwork for supporting multiple TCG contexts.

While at it, also allocate temps_used directly as a bitmap of the
required size, instead of using a bitmap of TCG_MAX_TEMPS via
TCGTempSet.

Performance-wise we lose about 1.12% in a translation-heavy workload
such as booting+shutting down debian-arm:

Performance counter stats for 'taskset -c 0 arm-softmmu/qemu-system-arm \
-machine type=virt -nographic -smp 1 -m 4096 \
-netdev user,id=unet,hostfwd=tcp::-:22 \
-device virtio-net-device,netdev=unet \
-drive file=die-on-boot.qcow2,id=myblock,index=0,if=none \
-device virtio-blk-device,drive=myblock \
-kernel kernel.img -append console=ttyAMA0 root=/dev/vda1 \
-name arm,debug-threads=on -smp 1' (10 runs):

 exec time (s)  Relative slowdown wrt original (%)
---
 original 20.213321616  0.
 tcg_malloc   20.441130078   1.1270214
 TCGContext   20.477846517   1.3086662
 g_malloc 20.780527895   2.8061013

The other two alternatives shown in the table are:
- TCGContext: embed temps[TCG_MAX_TEMPS] and TCGTempSet used_temps
  in TCGContext. This is simple enough but it isn't faster than using
  tcg_malloc; moreover, it wastes memory.
- g_malloc: allocate/deallocate both temps and used_temps every time
  tcg_optimize is executed.

Suggested-by: Richard Henderson 
Signed-off-by: Emilio G. Cota 
---
 tcg/optimize.c | 306 ++---
 1 file changed, 161 insertions(+), 145 deletions(-)

diff --git a/tcg/optimize.c b/tcg/optimize.c
index adfc56c..ce422e9 100644
--- a/tcg/optimize.c
+++ b/tcg/optimize.c
@@ -40,21 +40,18 @@ struct tcg_temp_info {
 tcg_target_ulong mask;
 };
 
-static struct tcg_temp_info temps[TCG_MAX_TEMPS];
-static TCGTempSet temps_used;
-
-static inline bool temp_is_const(TCGArg arg)
+static inline bool temp_is_const(const struct tcg_temp_info *temps, TCGArg arg)
 {
 return temps[arg].is_const;
 }
 
-static inline bool temp_is_copy(TCGArg arg)
+static inline bool temp_is_copy(const struct tcg_temp_info *temps, TCGArg arg)
 {
 return temps[arg].next_copy != arg;
 }
 
 /* Reset TEMP's state, possibly removing the temp for the list of copies.  */
-static void reset_temp(TCGArg temp)
+static void reset_temp(struct tcg_temp_info *temps, TCGArg temp)
 {
 temps[temps[temp].next_copy].prev_copy = temps[temp].prev_copy;
 temps[temps[temp].prev_copy].next_copy = temps[temp].next_copy;
@@ -64,21 +61,16 @@ static void reset_temp(TCGArg temp)
 temps[temp].mask = -1;
 }
 
-/* Reset all temporaries, given that there are NB_TEMPS of them.  */
-static void reset_all_temps(int nb_temps)
-{
-bitmap_zero(temps_used.l, nb_temps);
-}
-
 /* Initialize and activate a temporary.  */
-static void init_temp_info(TCGArg temp)
+static void init_temp_info(struct tcg_temp_info *temps,
+   unsigned long *temps_used, TCGArg temp)
 {
-if (!test_bit(temp, temps_used.l)) {
+if (!test_bit(temp, temps_used)) {
 temps[temp].next_copy = temp;
 temps[temp].prev_copy = temp;
 temps[temp].is_const = false;
 temps[temp].mask = -1;
-set_bit(temp, temps_used.l);
+set_bit(temp, temps_used);
 }
 }
 
@@ -116,7 +108,8 @@ static TCGOpcode op_to_movi(TCGOpcode op)
 }
 }
 
-static TCGArg find_better_copy(TCGContext *s, TCGArg temp)
+static TCGArg find_better_copy(TCGContext *s, const struct tcg_temp_info 
*temps,
+   TCGArg temp)
 {
 TCGArg i;
 
@@ -145,7 +138,8 @@ static TCGArg find_better_copy(TCGContext *s, TCGArg temp)
 return temp;
 }
 
-static bool temps_are_copies(TCGArg arg1, TCGArg arg2)
+static bool temps_are_copies(const struct tcg_temp_info *temps, TCGArg arg1,
+ TCGArg arg2)
 {
 TCGArg i;
 
@@ -153,7 +147,7 @@ static bool temps_are_copies(TCGArg arg1, TCGArg arg2)
 return true;
 }
 
-if (!temp_is_copy(arg1) || !temp_is_copy(arg2)) {
+if (!temp_is_copy(temps, arg1) || !temp_is_copy(temps, arg2)) {
 return false;
 }
 
@@ -166,15 +160,15 @@ static bool temps_are_copies(TCGArg arg1, TCGArg arg2)
 return false;
 }
 
-static void tcg_opt_gen_movi(TCGContext *s, TCGOp *op, TCGArg *args,
- TCGArg dst, TCGArg val)
+static void tcg_opt_gen_movi(TCGContext *s, struct tcg_temp_info *temps,
+ TCGOp *op, TCGArg *args, TCGArg dst, TCGArg val)
 {
 TCGOpcode new_op = op_to_movi(op->opc);
 tcg_target_ulong mask;
 
 op->opc = new_op;
 
-reset_temp(dst);
+reset_temp(temps, dst);
 temps[dst].is_const = true;
 temps[dst].val = val;
 mask = val;
@@ -188,10 +182,10 @@ static void tcg_opt_gen_movi(TCGContext *s, TCGOp *op, 
TCGArg *args,
 args[1] = val;
 }
 
-static void tcg_opt_

[Qemu-devel] [PATCH] target/arm: fix TCG temp leak in aarch64 rev16

[Emilio G. Cota]

On Wed, Jul 19, 2017 at 13:34:47 -1000, Richard Henderson wrote:
> It is much shorter to reverse all 4 half-words in parallel
> than extract, reverse, and deposit each in turn.
> 
> Suggested-by: Aurelien Jarno 
> Signed-off-by: Richard Henderson 
> ---
>  target/arm/translate-a64.c | 24 ++--
>  1 file changed, 6 insertions(+), 18 deletions(-)
> 
> diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c
> index 3fa39023ca..5bb0f8ef22 100644
> --- a/target/arm/translate-a64.c
> +++ b/target/arm/translate-a64.c
> @@ -4043,25 +4043,13 @@ static void handle_rev16(DisasContext *s, unsigned 
> int sf,
>  TCGv_i64 tcg_rd = cpu_reg(s, rd);
>  TCGv_i64 tcg_tmp = tcg_temp_new_i64();
>  TCGv_i64 tcg_rn = read_cpu_reg(s, rn, sf);
> +TCGv_i64 mask = tcg_const_i64(sf ? 0x00ff00ff00ff00ffull : 0x00ff00ff);
>  
> -tcg_gen_andi_i64(tcg_tmp, tcg_rn, 0x);
> -tcg_gen_bswap16_i64(tcg_rd, tcg_tmp);
> -
> -tcg_gen_shri_i64(tcg_tmp, tcg_rn, 16);
> -tcg_gen_andi_i64(tcg_tmp, tcg_tmp, 0x);
> -tcg_gen_bswap16_i64(tcg_tmp, tcg_tmp);
> -tcg_gen_deposit_i64(tcg_rd, tcg_rd, tcg_tmp, 16, 16);
> -
> -if (sf) {
> -tcg_gen_shri_i64(tcg_tmp, tcg_rn, 32);
> -tcg_gen_andi_i64(tcg_tmp, tcg_tmp, 0x);
> -tcg_gen_bswap16_i64(tcg_tmp, tcg_tmp);
> -tcg_gen_deposit_i64(tcg_rd, tcg_rd, tcg_tmp, 32, 16);
> -
> -tcg_gen_shri_i64(tcg_tmp, tcg_rn, 48);
> -tcg_gen_bswap16_i64(tcg_tmp, tcg_tmp);
> -tcg_gen_deposit_i64(tcg_rd, tcg_rd, tcg_tmp, 48, 16);
> -}
> +tcg_gen_shri_i64(tcg_tmp, tcg_rn, 8);
> +tcg_gen_and_i64(tcg_rd, tcg_rn, mask);
> +tcg_gen_and_i64(tcg_tmp, tcg_tmp, mask);
> +tcg_gen_shli_i64(tcg_rd, tcg_rd, 8);
> +tcg_gen_or_i64(tcg_rd, tcg_rd, tcg_tmp);
>  
>  tcg_temp_free_i64(tcg_tmp);

const leak! patch below -- cut with `git am --scissors'.

Emilio

---8<---

Signed-off-by: Emilio G. Cota 
---
 target/arm/translate-a64.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c
index 883e9df..58ed4c6 100644
--- a/target/arm/translate-a64.c
+++ b/target/arm/translate-a64.c
@@ -4044,20 +4044,21 @@ static void handle_rev16(DisasContext *s, unsigned int 
sf,
 TCGv_i64 tcg_tmp = tcg_temp_new_i64();
 TCGv_i64 tcg_rn = read_cpu_reg(s, rn, sf);
 TCGv_i64 mask = tcg_const_i64(sf ? 0x00ff00ff00ff00ffull : 0x00ff00ff);
 
 tcg_gen_shri_i64(tcg_tmp, tcg_rn, 8);
 tcg_gen_and_i64(tcg_rd, tcg_rn, mask);
 tcg_gen_and_i64(tcg_tmp, tcg_tmp, mask);
 tcg_gen_shli_i64(tcg_rd, tcg_rd, 8);
 tcg_gen_or_i64(tcg_rd, tcg_rd, tcg_tmp);
 
+tcg_temp_free_i64(mask);
 tcg_temp_free_i64(tcg_tmp);
 }
 
 /* C3.5.7 Data-processing (1 source)
  *   31  30  29  28 21 20 16 1510 95 40
  * ++---+---+-+-++--+--+
  * | sf | 1 | S | 1 1 0 1 0 1 1 0 | opcode2 | opcode |  Rn  |  Rd  |
  * ++---+---+-+-++--+--+
  */
 static void disas_data_proc_1src(DisasContext *s, uint32_t insn)
-- 
2.7.4

[Qemu-devel] [PATCH] vhost-user: fix watcher need be removed when vhost-user hotplug

[w00273186]

From: Yunjian Wang 

"nc" is freed after hotplug vhost-user, but the watcher don't be removed.
The QEMU crash when the watcher access the "nc" on socket disconnect.

Program received signal SIGSEGV, Segmentation fault.
#0  object_get_class (obj=obj@entry=0x2) at qom/object.c:750
#1  0x7f9bb4180da1 in qemu_chr_fe_disconnect (be=) at 
chardev/char-fe.c:372
#2  0x7f9bb40d1100 in net_vhost_user_watch (chan=, 
cond=, opaque=) at net/vhost-user.c:188
#3  0x7f9baf97f99a in g_main_context_dispatch () from 
/usr/lib64/libglib-2.0.so.0
#4  0x7f9bb41d7ebc in glib_pollfds_poll () at util/main-loop.c:213
#5  os_host_main_loop_wait (timeout=) at util/main-loop.c:261
#6  main_loop_wait (nonblocking=nonblocking@entry=0) at util/main-loop.c:515
#7  0x7f9bb3e266a7 in main_loop () at vl.c:1917
#8  main (argc=, argv=, envp=) 
at vl.c:4786

Signed-off-by: Yunjian Wang 
---
 net/vhost-user.c | 4 
 1 file changed, 4 insertions(+)

diff --git a/net/vhost-user.c b/net/vhost-user.c
index 36f32a2..c23927c 100644
--- a/net/vhost-user.c
+++ b/net/vhost-user.c
@@ -151,6 +151,10 @@ static void vhost_user_cleanup(NetClientState *nc)
 s->vhost_net = NULL;
 }
 if (nc->queue_index == 0) {
+if (s->watch) {
+g_source_remove(s->watch);
+s->watch = 0;
+}
 qemu_chr_fe_deinit(&s->chr, true);
 }
 
-- 
1.8.3.1

Re: [Qemu-devel] [PATCH v3 35/43] tcg: dynamically allocate optimizer temps

[Emilio G. Cota]

On Thu, Jul 20, 2017 at 14:02:53 -1000, Richard Henderson wrote:
> On 07/20/2017 01:53 PM, Emilio G. Cota wrote:
> >BTW, is there any chance that the pool will be initialized before we copy
> >tcg_init_ctx? That'd mean the main thread has performed translation, which
> >seems unlikely to me. But should then we bother clearing the TCGProfile
> >counters after we copy tcg_init_ctx? I don't see how without translation
> >counters would be !0.
> 
> I wouldn't think so.  This cpu setup should be happening very early.

OK. I've removed the clearing of prof in v4.

> We could perhaps look at arranging fields such that all the fields that are
> "shared" between the contexts are up front, and use the qemu standard
> 
>   memcpy(new, old, offsetof(TCGContext, end_common_fields));
> 
> trick, and zero the rest.

It'll be much faster if you do this because you're familiar with all
the fields in there (I'm not); I've added this to the "to do later"
list in v4's cover letter so that we do not forget.

v4 coming up.

E.

Re: [Qemu-devel] [PATCH qemu] Revert "elf-loader: warn about invalid endianness"

[no-reply]

Hi,

This series failed automatic build test. Please find the testing commands and
their output below. If you have docker installed, you can probably reproduce it
locally.

Type: series
Subject: [Qemu-devel] [PATCH qemu] Revert "elf-loader: warn about invalid 
endianness"
Message-id: 20170721041952.45950-1-...@ozlabs.ru

=== TEST SCRIPT BEGIN ===
#!/bin/bash
set -e
git submodule update --init dtc
# Let docker tests dump environment info
export SHOW_ENV=1
export J=8
time make docker-test-quick@centos6
time make docker-test-build@min-glib
time make docker-test-mingw@fedora
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
1ec97a9 Revert "elf-loader: warn about invalid endianness"

=== OUTPUT BEGIN ===
Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc'
Cloning into '/var/tmp/patchew-tester-tmp-dkcfo8gx/src/dtc'...
Submodule path 'dtc': checked out '558cd81bdd432769b59bff01240c44f82cfb1a9d'
  BUILD   centos6
make[1]: Entering directory '/var/tmp/patchew-tester-tmp-dkcfo8gx/src'
  ARCHIVE qemu.tgz
  ARCHIVE dtc.tgz
  COPYRUNNER
RUN test-quick in qemu:centos6 
Packages installed:
SDL-devel-1.2.14-7.el6_7.1.x86_64
bison-2.4.1-5.el6.x86_64
ccache-3.1.6-2.el6.x86_64
epel-release-6-8.noarch
flex-2.5.35-9.el6.x86_64
gcc-4.4.7-18.el6.x86_64
git-1.7.1-8.el6.x86_64
glib2-devel-2.28.8-9.el6.x86_64
libfdt-devel-1.4.0-1.el6.x86_64
make-3.81-23.el6.x86_64
package g++ is not installed
pixman-devel-0.32.8-1.el6.x86_64
tar-1.23-15.el6_8.x86_64
zlib-devel-1.2.3-29.el6.x86_64

Environment variables:
PACKAGES=libfdt-devel ccache tar git make gcc g++ flex bison zlib-devel 
glib2-devel SDL-devel pixman-devel epel-release
HOSTNAME=847e13737f8c
TERM=xterm
MAKEFLAGS= -j8
HISTSIZE=1000
J=8
USER=root
CCACHE_DIR=/var/tmp/ccache
EXTRA_CONFIGURE_OPTS=
V=
SHOW_ENV=1
MAIL=/var/spool/mail/root
PATH=/usr/lib/ccache:/usr/lib64/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/
LANG=en_US.UTF-8
TARGET_LIST=
HISTCONTROL=ignoredups
SHLVL=1
HOME=/root
TEST_DIR=/tmp/qemu-test
LOGNAME=root
LESSOPEN=||/usr/bin/lesspipe.sh %s
FEATURES= dtc
DEBUG=
G_BROKEN_FILENAMES=1
CCACHE_HASHDIR=
_=/usr/bin/env

Configure options:
--enable-werror --target-list=x86_64-softmmu,aarch64-softmmu 
--prefix=/var/tmp/qemu-build/install
No C++ compiler available; disabling C++ specific optional code
Install prefix/var/tmp/qemu-build/install
BIOS directory/var/tmp/qemu-build/install/share/qemu
binary directory  /var/tmp/qemu-build/install/bin
library directory /var/tmp/qemu-build/install/lib
module directory  /var/tmp/qemu-build/install/lib/qemu
libexec directory /var/tmp/qemu-build/install/libexec
include directory /var/tmp/qemu-build/install/include
config directory  /var/tmp/qemu-build/install/etc
local state directory   /var/tmp/qemu-build/install/var
Manual directory  /var/tmp/qemu-build/install/share/man
ELF interp prefix /usr/gnemul/qemu-%M
Source path   /tmp/qemu-test/src
C compilercc
Host C compiler   cc
C++ compiler  
Objective-C compiler cc
ARFLAGS   rv
CFLAGS-O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -g 
QEMU_CFLAGS   -I/usr/include/pixman-1   -I$(SRC_PATH)/dtc/libfdt -pthread 
-I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include   -fPIE -DPIE -m64 -mcx16 
-D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes 
-Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes 
-fno-strict-aliasing -fno-common -fwrapv  -Wendif-labels 
-Wno-missing-include-dirs -Wempty-body -Wnested-externs -Wformat-security 
-Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration 
-Wold-style-definition -Wtype-limits -fstack-protector-all
LDFLAGS   -Wl,--warn-common -Wl,-z,relro -Wl,-z,now -pie -m64 -g 
make  make
install   install
pythonpython -B
smbd  /usr/sbin/smbd
module supportno
host CPU  x86_64
host big endian   no
target list   x86_64-softmmu aarch64-softmmu
gprof enabled no
sparse enabledno
strip binariesyes
profiler  no
static build  no
pixmansystem
SDL support   yes (1.2.14)
GTK support   no 
GTK GL supportno
VTE support   no 
TLS priority  NORMAL
GNUTLS supportno
GNUTLS rndno
libgcrypt no
libgcrypt kdf no
nettleno 
nettle kdfno
libtasn1  no
curses supportno
virgl support no
curl support  no
mingw32 support   no
Audio drivers oss
Block whitelist (rw) 
Block whitelist (ro) 
VirtFS supportno
VNC support   yes
VNC SASL support  no
VNC JPEG support  no
VNC PNG support   no
xen support   no
brlapi supportno
bluez  supportno
Documentation no
PIE   yes
vde support   no
netmap supportno
Linux AIO support no
ATTR/XATTR support yes
Install blobs yes
KVM support   yes
HAX support   no
TCG support   yes
TCG debug enabled no

[Qemu-devel] [FIX PATCH v2] spapr: Fix QEMU abort during memory unplug

[Bharata B Rao]

Commit 0cffce56 (hw/ppc/spapr.c: adding pending_dimm_unplugs to
sPAPRMachineState) introduced a new way to track pending LMBs of DIMM
device that is marked for removal. Since this commit we can hit the
assert in spapr_pending_dimm_unplugs_add() in the following situation:

- DIMM device removal fails as the guest doesn't allow the removal.
- Subsequent attempt to remove the same DIMM would hit the assert
  as the corresponding sPAPRDIMMState is still part of the
  pending_dimm_unplugs list.

Fix this by removing the assert and conditionally adding the
sPAPRDIMMState to pending_dimm_unplugs list only when it is not
already present.

Fixes: 0cffce56ae3501c5783d779f97993ce478acf856
Signed-off-by: Bharata B Rao 
---
Changes in v2:
- sPAPRDIMMState is now allocated within spapr_pending_dimm_unplugs_add()
  itself (David Gibson)
- spapr_recover_pending_dimm_state() should never return a NULL sPAPRDIMMState,
  added an assert for the same.

 hw/ppc/spapr.c | 37 +
 1 file changed, 21 insertions(+), 16 deletions(-)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 1cb09e7..2465b27 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -2850,11 +2850,25 @@ static sPAPRDIMMState 
*spapr_pending_dimm_unplugs_find(sPAPRMachineState *s,
 return dimm_state;
 }
 
-static void spapr_pending_dimm_unplugs_add(sPAPRMachineState *spapr,
-   sPAPRDIMMState *dimm_state)
+static sPAPRDIMMState *spapr_pending_dimm_unplugs_add(sPAPRMachineState *spapr,
+  uint32_t nr_lmbs,
+  PCDIMMDevice *dimm)
 {
-g_assert(!spapr_pending_dimm_unplugs_find(spapr, dimm_state->dimm));
-QTAILQ_INSERT_HEAD(&spapr->pending_dimm_unplugs, dimm_state, next);
+sPAPRDIMMState *ds = NULL;
+
+/*
+ * If this request is for a DIMM whose removal had failed earlier
+ * (due to guest's refusal to remove the LMBs), we would have this
+ * dimm already in the pending_dimm_unplugs list. In that
+ * case don't add again.
+ */
+if (!spapr_pending_dimm_unplugs_find(spapr, dimm)) {
+ds = g_malloc0(sizeof(sPAPRDIMMState));
+ds->nr_lmbs = nr_lmbs;
+ds->dimm = dimm;
+QTAILQ_INSERT_HEAD(&spapr->pending_dimm_unplugs, ds, next);
+}
+return ds;
 }
 
 static void spapr_pending_dimm_unplugs_remove(sPAPRMachineState *spapr,
@@ -2875,7 +2889,6 @@ static sPAPRDIMMState 
*spapr_recover_pending_dimm_state(sPAPRMachineState *ms,
 uint32_t avail_lmbs = 0;
 uint64_t addr_start, addr;
 int i;
-sPAPRDIMMState *ds;
 
 addr_start = object_property_get_int(OBJECT(dimm), PC_DIMM_ADDR_PROP,
  &error_abort);
@@ -2891,11 +2904,7 @@ static sPAPRDIMMState 
*spapr_recover_pending_dimm_state(sPAPRMachineState *ms,
 addr += SPAPR_MEMORY_BLOCK_SIZE;
 }
 
-ds = g_malloc0(sizeof(sPAPRDIMMState));
-ds->nr_lmbs = avail_lmbs;
-ds->dimm = dimm;
-spapr_pending_dimm_unplugs_add(ms, ds);
-return ds;
+return spapr_pending_dimm_unplugs_add(ms, avail_lmbs, dimm);
 }
 
 /* Callback to be called during DRC release. */
@@ -2911,6 +2920,7 @@ void spapr_lmb_release(DeviceState *dev)
  * during the unplug process. In this case recover it. */
 if (ds == NULL) {
 ds = spapr_recover_pending_dimm_state(spapr, PC_DIMM(dev));
+g_assert(ds);
 /* The DRC being examined by the caller at least must be counted */
 g_assert(ds->nr_lmbs);
 }
@@ -2942,18 +2952,13 @@ static void spapr_memory_unplug_request(HotplugHandler 
*hotplug_dev,
 uint64_t addr_start, addr;
 int i;
 sPAPRDRConnector *drc;
-sPAPRDIMMState *ds;
-
 addr_start = object_property_get_uint(OBJECT(dimm), PC_DIMM_ADDR_PROP,
  &local_err);
 if (local_err) {
 goto out;
 }
 
-ds = g_malloc0(sizeof(sPAPRDIMMState));
-ds->nr_lmbs = nr_lmbs;
-ds->dimm = dimm;
-spapr_pending_dimm_unplugs_add(spapr, ds);
+spapr_pending_dimm_unplugs_add(spapr, nr_lmbs, dimm);
 
 addr = addr_start;
 for (i = 0; i < nr_lmbs; i++) {
-- 
2.7.4

Re: [Qemu-devel] [PATCH 0/3] build configuration query tool and conditional (qemu-io)test skip

[no-reply]

Hi,

This series seems to have some coding style problems. See output below for
more information:

Type: series
Subject: [Qemu-devel] [PATCH 0/3] build configuration query tool and 
conditional (qemu-io)test skip
Message-id: 20170721034730.25612-1-cr...@redhat.com

=== TEST SCRIPT BEGIN ===
#!/bin/bash

BASE=base
n=1
total=$(git log --oneline $BASE.. | wc -l)
failed=0

git config --local diff.renamelimit 0
git config --local diff.renames True

commits="$(git log --format=%H --reverse $BASE..)"
for c in $commits; do
echo "Checking PATCH $n/$total: $(git log -n 1 --format=%s $c)..."
if ! git show $c --format=email | ./scripts/checkpatch.pl --mailback -; then
failed=1
echo
fi
n=$((n+1))
done

exit $failed
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 * [new tag] patchew/20170721041952.45950-1-...@ozlabs.ru -> 
patchew/20170721041952.45950-1-...@ozlabs.ru
Switched to a new branch 'test'
8e1715d qemu-iotests: require CONFIG_LINUX_AIO for test 087
2ba2d91 qemu-iotests: add _require_feature() function
9041490 scripts: introduce buildconf.py

=== OUTPUT BEGIN ===
Checking PATCH 1/3: scripts: introduce buildconf.py...
ERROR: code indent should never use tabs
#107: FILE: scripts/buildconf.py:59:
+^I@echo $({conf})$

WARNING: line over 80 characters
#277: FILE: scripts/buildconf.py:229:
+  'set to "y".  This causes this tool to be 
silent '

WARNING: line over 80 characters
#278: FILE: scripts/buildconf.py:230:
+  'and return only a status code of either 0 
(if '

WARNING: line over 80 characters
#279: FILE: scripts/buildconf.py:231:
+  'configuration is set) or non-zero 
otherwise.'))

WARNING: line over 80 characters
#281: FILE: scripts/buildconf.py:233:
+help=('Do not attempt to use a default target if 
one '

WARNING: line over 80 characters
#282: FILE: scripts/buildconf.py:234:
+  'was not explicitly given in the command 
line'))

total: 1 errors, 5 warnings, 278 lines checked

Your patch has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

Checking PATCH 2/3: qemu-iotests: add _require_feature() function...
Checking PATCH 3/3: qemu-iotests: require CONFIG_LINUX_AIO for test 087...
=== OUTPUT END ===

Test command exited with code: 1


---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-de...@freelists.org

Re: [Qemu-devel] Unified Datagram Socket Transport

[Anton Ivanov]

On 21/07/17 04:55, Jason Wang wrote:



On 2017年07月21日 03:12, anton.iva...@cambridgegreys.com wrote:

Hi all,

This addresses comments so far except Eric's suggestion to use
InetSocketAddressBase. If I understand correctly its intended use,
it will not be of help for protocols which have no port (raw
sockets - GRE, L2TPv3, etc).

It also includes a port of the original socket.c transport to
the new UDST backend. The relevant code is ifdef-ed so there
should be no effect on other systems.


This looks sub-optimal. If you want to do this, I would rather suggest 
you just extend the socket dgram backend like what udst did now.


Apologies, do you mean extend it further to handle the tcp form?

That does not work at present. Sure, you can receive tcp using recvmmsg, 
but you cannot use it to handle a tcp based variable length encaps where 
frame lengths are set in a header. That can be done only be sequential 
read() or recv() to read the frame length first, then the frame.


I can in theory add that to a unified socket backend, but this is 
completely different tx and rx logic - so it will become suboptimal (and 
quite ugly). It will need different tx and rx functions and appropriate 
initialization to select them. I'd rather keep it to datagram only - 
what says on the tin.






I think that this is would be the appropriate place to stop in this
iteration. I would prefer to have this polished, before I start
looking at sendmmsg and bulk send or some of the more unpleasant
encapsulations like geneve.


Pay attention we're softfreeze now. So the feature is for 2.11, if it 
looks good, I can only queue it for 2.11.


OK. Understood.



Btw, looks like not all comments of v1 were addressed.


I will go through the comments one more time. I realized I may have 
missed converting malloc to a g_memdup in a couple places.




Thanks


Best Regards,

A.





A.







--
Anton R. Ivanov

Cambridge Greys Limited, England and Wales company No 10273661
http://www.cambridgegreys.com/

[Qemu-devel] [PATCH qemu] Revert "elf-loader: warn about invalid endianness"

[Alexey Kardashevskiy]

This reverts c8e1158cf611 "elf-loader: warn about invalid endianness"
as it produces a useless message every time an LE kernel image is
passed via -kernel on a ppc64-pseries machine. The pseries machine
already checks for ELF_LOAD_WRONG_ENDIAN and tries with big_endian=0.

Signed-off-by: Alexey Kardashevskiy 
---
 hw/core/loader.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/hw/core/loader.c b/hw/core/loader.c
index c17ace0a2e..e5e8cbb638 100644
--- a/hw/core/loader.c
+++ b/hw/core/loader.c
@@ -480,7 +480,6 @@ int load_elf_ram(const char *filename,
 }
 
 if (target_data_order != e_ident[EI_DATA]) {
-fprintf(stderr, "%s: wrong endianness\n", filename);
 ret = ELF_LOAD_WRONG_ENDIAN;
 goto fail;
 }
-- 
2.11.0

Re: [Qemu-devel] unrecognized option --object

[Fam Zheng]

On Thu, 07/20 19:06, 陳培泓 wrote:
> I build qemu from the https://github.com/qemu/qemu, and it didn't show any
> errors.
> 
> I tried qemu-nbd --object secret,id=sec0...
> but it shows unrecognized option '--object '
> 
> how to solve it?

-object it is. Thanks.

Fam

Re: [Qemu-devel] Unified Datagram Socket Transport

[Jason Wang]

On 2017年07月21日 03:12, anton.iva...@cambridgegreys.com wrote:

Hi all,

This addresses comments so far except Eric's suggestion to use
InetSocketAddressBase. If I understand correctly its intended use,
it will not be of help for protocols which have no port (raw
sockets - GRE, L2TPv3, etc).

It also includes a port of the original socket.c transport to
the new UDST backend. The relevant code is ifdef-ed so there
should be no effect on other systems.


This looks sub-optimal. If you want to do this, I would rather suggest 
you just extend the socket dgram backend like what udst did now.




I think that this is would be the appropriate place to stop in this
iteration. I would prefer to have this polished, before I start
looking at sendmmsg and bulk send or some of the more unpleasant
encapsulations like geneve.


Pay attention we're softfreeze now. So the feature is for 2.11, if it 
looks good, I can only queue it for 2.11.


Btw, looks like not all comments of v1 were addressed.

Thanks



A.

[Qemu-devel] [PATCH 1/3] scripts: introduce buildconf.py

[Cleber Rosa]

scripts/buildconf.py is a command line utility (but also can be used
as a Python module) that introspects the build configuration.

It uses the generated host level config-host.mak to obtain the general
build configuration, and optionally, also target specific
config-target.mak and config-devices.mak.  It does not attempt to
implement a Makefile parser, but instead relies on "make" itself to
parse those files and output the queried variable.

It requires a build tree that has been both configured and built.  By
default, for convenience, it will selected a default target, which can
be displayed and overriden.  A few examples follow.

To get the TLS priority (a host level configuration), one would run:

 $ ./scripts/buildconf.py CONFIG_TLS_PRIORITY
 NORMAL

To get a configuration from the default target devices:

 $ ./scripts/buildconf.py CONFIG_PARALLEL
 y

If one is not interested in the actual value, but whether a given
feature is enabled, the '-c|--check' option can be used:

 $ ./scripts/buildconf.py -c CONFIG_PARALLEL; echo $?
 0

And for checking a target different than the default one:

 $ ./scripts/buildconf.py -c CONFIG_PARALLEL arm-softmmu; echo $?
 255

Signed-off-by: Cleber Rosa 
---
 scripts/buildconf.py | 278 +++
 1 file changed, 278 insertions(+)
 create mode 100755 scripts/buildconf.py

diff --git a/scripts/buildconf.py b/scripts/buildconf.py
new file mode 100755
index 000..0eee6d7
--- /dev/null
+++ b/scripts/buildconf.py
@@ -0,0 +1,278 @@
+#!/usr/bin/env python
+#
+# QEMU build configuration introspection utilty
+#
+# Copyright (C) 2017 Red Hat Inc.
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library; if not, see .
+#
+# Authors:
+#  Cleber Rosa 
+
+"""
+QEMU's build configuration is recorded in both config-host.mak and
+config-host.h, as is specific target configuration expressed in
+/config-{target,devices}.{mak,h}.
+
+This module relies on the .mak files, as they contain a bit more
+information than the .h files.
+
+While it would be possible to write a simple Makefile parser capable to
+handle the variable assignments, this would impose limitations on this
+script and introduce breakages if the build scripts start using
+functionality not expected here.
+
+The approach chosen was one that is definitely slower at runtime but
+is more reliable.  Temporary Makefiles that include config-host.mak,
+and optionally the target specific config-target.mak and
+config-devices.mak files, and print the desired configuration to
+stdout.  As long as the basic premises of a global config-host.mak,
+and target specific config-target.mak and config-devices.mak is kept,
+this tool should be able to keep up with any style or feature chances.
+"""
+
+from __future__ import print_function
+
+import optparse
+import os
+import subprocess
+import sys
+import tempfile
+
+
+TEMPLATE = """
+include {build_prefix}/config-host.mak
+{target_specific}
+
+all:
+   @echo $({conf})
+"""
+
+TARGET_TEMPLATE = """
+include {build_prefix}/{target}/config-target.mak
+include {build_prefix}/{target}/config-devices.mak
+"""
+
+
+class InvalidTarget(Exception):
+"""
+Target chosen is not present in the current build tree configuration
+"""
+
+
+def get_build_root():
+"""
+Returns the absolute location of the root of the build tree
+
+This has been tested from build(only) trees, and works fine when
+it's executed as a command line tool.
+
+If this is used as a Python module, it will really depend on how
+the module is imported.  If the build tree "scripts" directory is
+added to the import path, this will work properly.  If the current
+working directory is the "scripts" directory itself, and no
+explicit import path is added, it will only work when building
+from the source tree, and will *not* work when the build tree is
+different from the source tree.
+
+A longer explanation on this caveat: the Python import
+implementation will look for a matching module in the current
+working directory.  Python's current working directory,
+os.getcwd(), is really like getcwd(3), and not like
+os.getenv("PWD").  Because the scripts directory is linked to the
+source tree, os.getcwd() returns the source tree location instead.
+"""
+return os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+
+
+def is

[Qemu-devel] [PATCH 2/3] qemu-iotests: add _require_feature() function

[Cleber Rosa]

With the previously introduced buildconf.py script, it's possible
to determine if a feature needed by a test is present or not.  This
adds a thin layer on top of scripts/buildconf.py, and allows tests
to be skipped when a feature is required.

The naming of the function, while different in tense from the
_supported_* family of functions, was chosen to match the style
of _require_command(), which seems pretty similar.

Signed-off-by: Cleber Rosa 
---
 tests/qemu-iotests/check | 2 ++
 tests/qemu-iotests/common.rc | 7 +++
 2 files changed, 9 insertions(+)

diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check
index 2a55ec9..c0f4004 100755
--- a/tests/qemu-iotests/check
+++ b/tests/qemu-iotests/check
@@ -60,6 +60,8 @@ fi
 
 build_root="$build_iotests/../.."
 
+export BUILDCONF="$build_root/scripts/buildconf.py"
+
 if [ -x "$build_iotests/socket_scm_helper" ]
 then
 export SOCKET_SCM_HELPER="$build_iotests/socket_scm_helper"
diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
index 2548e58..19b3111 100644
--- a/tests/qemu-iotests/common.rc
+++ b/tests/qemu-iotests/common.rc
@@ -470,6 +470,13 @@ _require_command()
 [ -x "$c" ] || _notrun "$1 utility required, skipped this test"
 }
 
+# tests whether a given configure time feature is enabled
+#
+_require_feature()
+{
+$BUILDCONF -c -n $1 || _notrun "feature not enabled: $1"
+}
+
 _full_imgfmt_details()
 {
 if [ -n "$IMGOPTS" ]; then
-- 
2.9.4

[Qemu-devel] [PATCH 3/3] qemu-iotests: require CONFIG_LINUX_AIO for test 087

[Cleber Rosa]

One of the "sub-"tests of test 087 requires CONFIG_LINUX_AIO.

As a PoC/RFC, this goes the easy route and skips the test as a whole
when that feature is missing.  Other approaches include splitting
the test and adding extra filtering.

Signed-off-by: Cleber Rosa 
---
 tests/qemu-iotests/087 | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tests/qemu-iotests/087 b/tests/qemu-iotests/087
index f8e4903..a2fb7de 100755
--- a/tests/qemu-iotests/087
+++ b/tests/qemu-iotests/087
@@ -34,6 +34,7 @@ status=1  # failure is the default!
 _supported_fmt qcow2
 _supported_proto file
 _supported_os Linux
+_require_feature CONFIG_LINUX_AIO
 
 function do_run_qemu()
 {
-- 
2.9.4

[Qemu-devel] [PATCH 0/3] build configuration query tool and conditional (qemu-io)test skip

[Cleber Rosa]

This is a follow up to a previous discussion about reported failures when
running some qemu-iotests.  Turns out the failures were due to missing
libraries, which in turn, reflected on the host build configuration.

This series introduces a tool that can check both host and target level
build configurations.  On top of that, it adds a function to to be used
on qemu-iotests.  Finally, as an example, it sets a test to be skipped
if the required feature is not enable on the host build configuration.

Cleber Rosa (3):
  scripts: introduce buildconf.py
  qemu-iotests: add _require_feature() function
  qemu-iotests: require CONFIG_LINUX_AIO for test 087

 scripts/buildconf.py | 278 +++
 tests/qemu-iotests/087   |   1 +
 tests/qemu-iotests/check |   2 +
 tests/qemu-iotests/common.rc |   7 ++
 4 files changed, 288 insertions(+)

Re: [Qemu-devel] Status and RFC of patchew testings on QEMU

[Fam Zheng]

On Mon, 07/17 11:02, Daniel P. Berrange wrote:
> I'd like to see a web page that provides a list of all mail threads that
> the test system has queued, with status of which jobs and running, and
> once completed, provides the full logs.

The index page already exists (no running jobs information, though):

http://patchew.org/QEMU/

Fam

Re: [Qemu-devel] [PATCH v7] qga: Add support network interface statistics in guest-network-get-interfaces command

[Fam Zheng]

On Fri, 07/21 09:20, ZhiPeng Lu wrote:
> we can get the network interface statistics inside a virtual machine by
> guest-network-get-interfaces command. it is very useful for us to monitor
> and analyze network traffic.
> 
> Signed-off-by: ZhiPeng Lu 

In the future, please add a '---' line between commit message and revision
history:

---

So that the latter won't get into the git history when maintainers apply it.

Fam

> 
> v1->v2:
>  - correct some spelling mistake and add the stats data to the
>guest-network-get-interfaces command instead of adding a new command.
> v2-v3:
>  - optimize function implementation
> v3->v4:
>  - modify compile error
> v4->v5:
>  - rename some temporary variables and add str_trim_off function for
>calculating the space num in front of the string in guest_get_network_stats
> v5->v6:
>  - use g_strchug instead of str_trim_off implemented by myself
> v6->v7:
>  - add implementation for windows
> ---
>  qga/commands-posix.c | 72 
> +++-
>  qga/commands-win32.c | 47 ++
>  qga/qapi-schema.json | 38 ++-
>  3 files changed, 155 insertions(+), 2 deletions(-)
> 
> diff --git a/qga/commands-posix.c b/qga/commands-posix.c
> index d8e4122..b65dd8e 100644
> --- a/qga/commands-posix.c
> +++ b/qga/commands-posix.c
> @@ -1639,6 +1639,65 @@ guest_find_interface(GuestNetworkInterfaceList *head,
>  return head;
>  }
>  
> +static int guest_get_network_stats(const char *name,
> +   GuestNetworkInterfaceStat *stats)
> +{
> +int name_len;
> +char const *devinfo = "/proc/net/dev";
> +FILE *fp;
> +char *line = NULL, *colon;
> +size_t n;
> +fp = fopen(devinfo, "r");
> +if (!fp) {
> +return -1;
> +}
> +name_len = strlen(name);
> +while (getline(&line, &n, fp) != -1) {
> +long long dummy;
> +long long rx_bytes;
> +long long rx_packets;
> +long long rx_errs;
> +long long rx_dropped;
> +long long tx_bytes;
> +long long tx_packets;
> +long long tx_errs;
> +long long tx_dropped;
> +char *trim_line;
> +trim_line = g_strchug(line);
> +if (trim_line[0] == '\0') {
> +continue;
> +}
> +colon = strchr(trim_line, ':');
> +if (!colon) {
> +continue;
> +}
> +if (colon - name_len  == trim_line &&
> +   strncmp(trim_line, name, name_len) == 0) {
> +if (sscanf(colon + 1,
> +"%lld %lld %lld %lld %lld %lld %lld %lld %lld %lld %lld %lld 
> %lld %lld %lld %lld",
> +  &rx_bytes, &rx_packets, &rx_errs, &rx_dropped,
> +  &dummy, &dummy, &dummy, &dummy,
> +  &tx_bytes, &tx_packets, &tx_errs, &tx_dropped,
> +  &dummy, &dummy, &dummy, &dummy) != 16) {
> +continue;
> +}
> +stats->rx_bytes = rx_bytes;
> +stats->rx_packets = rx_packets;
> +stats->rx_errs = rx_errs;
> +stats->rx_dropped = rx_dropped;
> +stats->tx_bytes = tx_bytes;
> +stats->tx_packets = tx_packets;
> +stats->tx_errs = tx_errs;
> +stats->tx_dropped = tx_dropped;
> +fclose(fp);
> +return 0;
> +}
> +}
> +fclose(fp);
> +g_debug("/proc/net/dev: Interface not found");
> +return -1;
> +}
> +
>  /*
>   * Build information about guest interfaces
>   */
> @@ -1655,6 +1714,7 @@ GuestNetworkInterfaceList 
> *qmp_guest_network_get_interfaces(Error **errp)
>  for (ifa = ifap; ifa; ifa = ifa->ifa_next) {
>  GuestNetworkInterfaceList *info;
>  GuestIpAddressList **address_list = NULL, *address_item = NULL;
> +GuestNetworkInterfaceStat  *interface_stat = NULL;
>  char addr4[INET_ADDRSTRLEN];
>  char addr6[INET6_ADDRSTRLEN];
>  int sock;
> @@ -1774,7 +1834,17 @@ GuestNetworkInterfaceList 
> *qmp_guest_network_get_interfaces(Error **errp)
>  
>  info->value->has_ip_addresses = true;
>  
> -
> +if (!info->value->has_statistics) {
> +interface_stat = g_malloc0(sizeof(*interface_stat));
> +if (guest_get_network_stats(info->value->name,
> +interface_stat) == -1) {
> +info->value->has_statistics = false;
> +g_free(interface_stat);
> +} else {
> +info->value->statistics = interface_stat;
> +info->value->has_statistics = true;
> +}
> +}
>  }
>  
>  freeifaddrs(ifap);
> diff --git a/qga/commands-win32.c b/qga/commands-win32.c
> index 6f16457..433453d 100644
> --- a/qga/commands-win32.c
> +++ b/qga/commands-win32.c
> @@ -1152,6 +1152,42 @@ out:
>  }
>  #endif
>  
> +static DWORD get_interface_index(const char *guid)
> +{
> +ULONG index;
> +DWORD status;
> +wchar_t wbu

[Qemu-devel] [PULL 11/14] target/mips: Add segmentation control registers

[Yongbok Kim]

From: James Hogan 

The optional segmentation control registers CP0_SegCtl0, CP0_SegCtl1 &
CP0_SegCtl2 control the behaviour and required privilege of the legacy
virtual memory segments.

Add them to the CP0 interface so they can be read and written when
CP0_Config3.SC=1, and initialise them to describe the standard legacy
layout so they can be used in future patches regardless of whether they
are exposed to the guest.

Signed-off-by: James Hogan 
Cc: Yongbok Kim 
Cc: Aurelien Jarno 
Reviewed-by: Yongbok Kim 
Signed-off-by: Yongbok Kim 
---
 target/mips/cpu.h   | 30 +
 target/mips/helper.h|  3 ++
 target/mips/machine.c   |  7 ++--
 target/mips/op_helper.c | 24 ++
 target/mips/translate.c | 88 +
 5 files changed, 150 insertions(+), 2 deletions(-)

diff --git a/target/mips/cpu.h b/target/mips/cpu.h
index c24b1f6..74f6a5b 100644
--- a/target/mips/cpu.h
+++ b/target/mips/cpu.h
@@ -306,6 +306,36 @@ struct CPUMIPSState {
 #define CP0PG_XIE 30
 #define CP0PG_ELPA 29
 #define CP0PG_IEC 27
+target_ulong CP0_SegCtl0;
+target_ulong CP0_SegCtl1;
+target_ulong CP0_SegCtl2;
+#define CP0SC_PA9
+#define CP0SC_PA_MASK   (0x7FULL << CP0SC_PA)
+#define CP0SC_PA_1GMASK (0x7EULL << CP0SC_PA)
+#define CP0SC_AM4
+#define CP0SC_AM_MASK   (0x7ULL << CP0SC_AM)
+#define CP0SC_AM_UK 0ULL
+#define CP0SC_AM_MK 1ULL
+#define CP0SC_AM_MSK2ULL
+#define CP0SC_AM_MUSK   3ULL
+#define CP0SC_AM_MUSUK  4ULL
+#define CP0SC_AM_USK5ULL
+#define CP0SC_AM_UUSK   7ULL
+#define CP0SC_EU3
+#define CP0SC_EU_MASK   (1ULL << CP0SC_EU)
+#define CP0SC_C 0
+#define CP0SC_C_MASK(0x7ULL << CP0SC_C)
+#define CP0SC_MASK  (CP0SC_C_MASK | CP0SC_EU_MASK | CP0SC_AM_MASK | \
+ CP0SC_PA_MASK)
+#define CP0SC_1GMASK(CP0SC_C_MASK | CP0SC_EU_MASK | CP0SC_AM_MASK | \
+ CP0SC_PA_1GMASK)
+#define CP0SC0_MASK (CP0SC_MASK | (CP0SC_MASK << 16))
+#define CP0SC1_XAM  59
+#define CP0SC1_XAM_MASK (0x7ULL << CP0SC1_XAM)
+#define CP0SC1_MASK (CP0SC_MASK | (CP0SC_MASK << 16) | CP0SC1_XAM_MASK)
+#define CP0SC2_XR   56
+#define CP0SC2_XR_MASK  (0xFFULL << CP0SC2_XR)
+#define CP0SC2_MASK (CP0SC_1GMASK | (CP0SC_1GMASK << 16) | CP0SC2_XR_MASK)
 int32_t CP0_Wired;
 int32_t CP0_SRSConf0_rw_bitmask;
 int32_t CP0_SRSConf0;
diff --git a/target/mips/helper.h b/target/mips/helper.h
index 60efa01..5f49234 100644
--- a/target/mips/helper.h
+++ b/target/mips/helper.h
@@ -115,6 +115,9 @@ DEF_HELPER_2(mtc0_entrylo1, void, env, tl)
 DEF_HELPER_2(mtc0_context, void, env, tl)
 DEF_HELPER_2(mtc0_pagemask, void, env, tl)
 DEF_HELPER_2(mtc0_pagegrain, void, env, tl)
+DEF_HELPER_2(mtc0_segctl0, void, env, tl)
+DEF_HELPER_2(mtc0_segctl1, void, env, tl)
+DEF_HELPER_2(mtc0_segctl2, void, env, tl)
 DEF_HELPER_2(mtc0_wired, void, env, tl)
 DEF_HELPER_2(mtc0_srsconf0, void, env, tl)
 DEF_HELPER_2(mtc0_srsconf1, void, env, tl)
diff --git a/target/mips/machine.c b/target/mips/machine.c
index 91e31a7..898825d 100644
--- a/target/mips/machine.c
+++ b/target/mips/machine.c
@@ -211,8 +211,8 @@ const VMStateDescription vmstate_tlb = {
 
 const VMStateDescription vmstate_mips_cpu = {
 .name = "cpu",
-.version_id = 9,
-.minimum_version_id = 9,
+.version_id = 10,
+.minimum_version_id = 10,
 .post_load = cpu_post_load,
 .fields = (VMStateField[]) {
 /* Active TC */
@@ -252,6 +252,9 @@ const VMStateDescription vmstate_mips_cpu = {
 VMSTATE_UINTTL(env.CP0_Context, MIPSCPU),
 VMSTATE_INT32(env.CP0_PageMask, MIPSCPU),
 VMSTATE_INT32(env.CP0_PageGrain, MIPSCPU),
+VMSTATE_UINTTL(env.CP0_SegCtl0, MIPSCPU),
+VMSTATE_UINTTL(env.CP0_SegCtl1, MIPSCPU),
+VMSTATE_UINTTL(env.CP0_SegCtl2, MIPSCPU),
 VMSTATE_INT32(env.CP0_Wired, MIPSCPU),
 VMSTATE_INT32(env.CP0_SRSConf0, MIPSCPU),
 VMSTATE_INT32(env.CP0_SRSConf1, MIPSCPU),
diff --git a/target/mips/op_helper.c b/target/mips/op_helper.c
index c52a407..526f8e4 100644
--- a/target/mips/op_helper.c
+++ b/target/mips/op_helper.c
@@ -1322,6 +1322,30 @@ void helper_mtc0_pagegrain(CPUMIPSState *env, 
target_ulong arg1)
 restore_pamask(env);
 }
 
+void helper_mtc0_segctl0(CPUMIPSState *env, target_ulong arg1)
+{
+CPUState *cs = CPU(mips_env_get_cpu(env));
+
+env->CP0_SegCtl0 = arg1 & CP0SC0_MASK;
+tlb_flush(cs);
+}
+
+void helper_mtc0_segctl1(CPUMIPSState *env, target_ulong arg1)
+{
+CPUState *cs = CPU(mips_env_get_cpu(env));
+
+env->CP0_SegCtl1 = arg1 & CP0SC1_MASK;
+tlb_flush(cs);
+}
+
+void helper_mtc0_segctl2(CPUMIPSState *env, target_ulong arg1)
+{
+CPUState *cs = CPU(mips_env_get_cpu(env));
+
+env->CP0_SegCtl2 = arg1 & CP0SC2_MASK;
+tlb_flush(cs);
+}
+
 void helper_mtc0_wired(CPUMIPSState *env, target_ulong arg1)
 {
 if (env->insn_flags & ISA_MIPS32R6) {
diff --git a/target/mips/translate.c b/target/mips/translat

[Qemu-devel] [PULL 10/14] target/mips: Add an MMU mode for ERL

[Yongbok Kim]

From: James Hogan 

The segmentation control feature allows a legacy memory segment to
become unmapped uncached at error level (according to CP0_Status.ERL),
and in fact the user segment is already treated in this way by QEMU.

Add a new MMU mode for this state so that QEMU's mappings don't persist
between ERL=0 and ERL=1.

Signed-off-by: James Hogan 
Reviewed-by: Yongbok Kim 
Cc: Aurelien Jarno 
[yongbok@imgtec.com:
  cosmetic changes]
Signed-off-by: Yongbok Kim 
---
 target/mips/cpu.h   | 17 +
 target/mips/op_helper.c | 10 ++
 2 files changed, 23 insertions(+), 4 deletions(-)

diff --git a/target/mips/cpu.h b/target/mips/cpu.h
index 3cf1676..c24b1f6 100644
--- a/target/mips/cpu.h
+++ b/target/mips/cpu.h
@@ -134,7 +134,7 @@ struct CPUMIPSFPUContext {
 #define FP_UNIMPLEMENTED  32
 };
 
-#define NB_MMU_MODES 3
+#define NB_MMU_MODES 4
 #define TARGET_INSN_START_EXTRA_WORDS 2
 
 typedef struct CPUMIPSMVPContext CPUMIPSMVPContext;
@@ -551,7 +551,7 @@ struct CPUMIPSState {
 #define EXCP_INST_NOTAVAIL 0x2 /* No valid instruction word for BadInstr */
 uint32_t hflags;/* CPU State */
 /* TMASK defines different execution modes */
-#define MIPS_HFLAG_TMASK  0xF5807FF
+#define MIPS_HFLAG_TMASK  0x1F5807FF
 #define MIPS_HFLAG_MODE   0x7 /* execution modes*/
 /* The KSU flags must be the lowest bits in hflags. The flag order
must be the same as defined for CP0 Status. This allows to use
@@ -601,6 +601,7 @@ struct CPUMIPSState {
 #define MIPS_HFLAG_FRE   0x200 /* FRE enabled */
 #define MIPS_HFLAG_ELPA  0x400
 #define MIPS_HFLAG_ITC_CACHE  0x800 /* CACHE instr. operates on ITC tag */
+#define MIPS_HFLAG_ERL   0x1000 /* error level flag */
 target_ulong btarget;/* Jump / branch target   */
 target_ulong bcond;  /* Branch condition (if needed)   */
 
@@ -698,11 +699,16 @@ extern uint32_t cpu_rddsp(uint32_t mask_num, CPUMIPSState 
*env);
 #define MMU_MODE0_SUFFIX _kernel
 #define MMU_MODE1_SUFFIX _super
 #define MMU_MODE2_SUFFIX _user
+#define MMU_MODE3_SUFFIX _error
 #define MMU_USER_IDX 2
 
 static inline int hflags_mmu_index(uint32_t hflags)
 {
-return hflags & MIPS_HFLAG_KSU;
+if (hflags & MIPS_HFLAG_ERL) {
+return 3; /* ERL */
+} else {
+return hflags & MIPS_HFLAG_KSU;
+}
 }
 
 static inline int cpu_mmu_index (CPUMIPSState *env, bool ifetch)
@@ -971,7 +977,10 @@ static inline void compute_hflags(CPUMIPSState *env)
  MIPS_HFLAG_F64 | MIPS_HFLAG_FPU | MIPS_HFLAG_KSU |
  MIPS_HFLAG_AWRAP | MIPS_HFLAG_DSP | MIPS_HFLAG_DSPR2 |
  MIPS_HFLAG_SBRI | MIPS_HFLAG_MSA | MIPS_HFLAG_FRE |
- MIPS_HFLAG_ELPA);
+ MIPS_HFLAG_ELPA | MIPS_HFLAG_ERL);
+if (env->CP0_Status & (1 << CP0St_ERL)) {
+env->hflags |= MIPS_HFLAG_ERL;
+}
 if (!(env->CP0_Status & (1 << CP0St_EXL)) &&
 !(env->CP0_Status & (1 << CP0St_ERL)) &&
 !(env->hflags & MIPS_HFLAG_DM)) {
diff --git a/target/mips/op_helper.c b/target/mips/op_helper.c
index da1817e..c52a407 100644
--- a/target/mips/op_helper.c
+++ b/target/mips/op_helper.c
@@ -67,6 +67,7 @@ static inline type do_##name(CPUMIPSState *env, target_ulong 
addr,  \
 case 1: return (type) cpu_##insn##_super_ra(env, addr, retaddr);\
 default:\
 case 2: return (type) cpu_##insn##_user_ra(env, addr, retaddr); \
+case 3: return (type) cpu_##insn##_error_ra(env, addr, retaddr);\
 }   \
 }
 #endif
@@ -94,6 +95,9 @@ static inline void do_##name(CPUMIPSState *env, target_ulong 
addr,  \
 case 1: cpu_##insn##_super_ra(env, addr, val, retaddr); break;  \
 default:\
 case 2: cpu_##insn##_user_ra(env, addr, val, retaddr); break;   \
+case 3: \
+cpu_##insn##_error_ra(env, addr, val, retaddr); \
+break;  \
 }   \
 }
 #endif
@@ -1451,6 +1455,9 @@ void helper_mtc0_status(CPUMIPSState *env, target_ulong 
arg1)
 val, val & env->CP0_Cause & CP0Ca_IP_mask,
 env->CP0_Cause);
 switch (cpu_mmu_index(env, false)) {
+case 3:
+qemu_log(", ERL\n");
+break;
 case MIPS_HFLAG_UM: qemu_log(", UM\n"); break;
 case MIPS_HFLAG_SM: qemu_log(", SM\n"); break;
 case MIPS_HFLAG_KM: qemu_log("\n"); break;
@@ -2245,6 +2252,9 @@ static void debug_post_eret(CPUMIPSState *env)
 if (env->hflags & MIPS_HFLAG_DM)
 qemu_log(" DEPC " TARGET_FMT_lx, env->CP0_DEPC);
 switch (cpu_mmu_index(en

[Qemu-devel] [PULL 13/14] target/mips: Add EVA support to P5600

[Yongbok Kim]

From: James Hogan 

Add the Enhanced Virtual Addressing (EVA) feature to the P5600 core
configuration, along with the related Segmentation Control (SC) feature
and writable CP0_EBase.WG bit.

This allows it to run Malta EVA kernels.

Signed-off-by: James Hogan 
Reviewed-by: Yongbok Kim 
Cc: Aurelien Jarno 
Signed-off-by: Yongbok Kim 
---
 target/mips/translate_init.c | 14 --
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/target/mips/translate_init.c b/target/mips/translate_init.c
index 8f8196e..741b390 100644
--- a/target/mips/translate_init.c
+++ b/target/mips/translate_init.c
@@ -421,9 +421,9 @@ static const mips_def_t mips_defs[] =
 },
 {
 /* FIXME:
- * Config3: CMGCR, SC, PW, VZ, CTXTC, CDMM, TL
+ * Config3: CMGCR, PW, VZ, CTXTC, CDMM, TL
  * Config4: MMUExtDef
- * Config5: EVA, MRP
+ * Config5: MRP
  * FIR(FCR0): Has2008
  * */
 .name = "P5600",
@@ -436,13 +436,14 @@ static const mips_def_t mips_defs[] =
(1 << CP0C1_PC) | (1 << CP0C1_FP),
 .CP0_Config2 = MIPS_CONFIG2,
 .CP0_Config3 = MIPS_CONFIG3 | (1U << CP0C3_M) | (1 << CP0C3_MSAP) |
-   (1 << CP0C3_BP) | (1 << CP0C3_BI) | (1 << CP0C3_ULRI) |
-   (1 << CP0C3_RXI) | (1 << CP0C3_LPA) | (1 << CP0C3_VInt),
+   (1 << CP0C3_BP) | (1 << CP0C3_BI) | (1 << CP0C3_SC) |
+   (1 << CP0C3_ULRI) | (1 << CP0C3_RXI) | (1 << CP0C3_LPA) 
|
+   (1 << CP0C3_VInt),
 .CP0_Config4 = MIPS_CONFIG4 | (1U << CP0C4_M) | (2 << CP0C4_IE) |
(0x1c << CP0C4_KScrExist),
 .CP0_Config4_rw_bitmask = 0,
-.CP0_Config5 = MIPS_CONFIG5 | (1 << CP0C5_MVH) | (1 << CP0C5_LLB) |
-   (1 << CP0C5_MRP),
+.CP0_Config5 = MIPS_CONFIG5 | (1 << CP0C5_EVA) | (1 << CP0C5_MVH) |
+   (1 << CP0C5_LLB) | (1 << CP0C5_MRP),
 .CP0_Config5_rw_bitmask = (1 << CP0C5_K) | (1 << CP0C5_CV) |
   (1 << CP0C5_MSAEn) | (1 << CP0C5_UFE) |
   (1 << CP0C5_FRE) | (1 << CP0C5_UFR),
@@ -453,6 +454,7 @@ static const mips_def_t mips_defs[] =
 .CP0_Status_rw_bitmask = 0x3C68FF1F,
 .CP0_PageGrain_rw_bitmask = (1U << CP0PG_RIE) | (1 << CP0PG_XIE) |
 (1 << CP0PG_ELPA) | (1 << CP0PG_IEC),
+.CP0_EBaseWG_rw_bitmask = (1 << CP0EBase_WG),
 .CP1_fcr0 = (1 << FCR0_FREP) | (1 << FCR0_UFRP) | (1 << FCR0_HAS2008) |
 (1 << FCR0_F64) | (1 << FCR0_L) | (1 << FCR0_W) |
 (1 << FCR0_D) | (1 << FCR0_S) | (0x03 << FCR0_PRID),
-- 
2.7.4

[Qemu-devel] [PULL 14/14] target/mips: Enable CP0_EBase.WG on MIPS64 CPUs

[Yongbok Kim]

From: James Hogan 

Enable the CP0_EBase.WG (write gate) on the I6400 and MIPS64R2-generic
CPUs. This allows 64-bit guests to run KVM itself, which uses
CP0_EBase.WG to point CP0_EBase at XKPhys.

Signed-off-by: James Hogan 
Cc: Yongbok Kim 
Cc: Aurelien Jarno 
Reviewed-by: Yongbok Kim 
Signed-off-by: Yongbok Kim 
---
 target/mips/translate_init.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/target/mips/translate_init.c b/target/mips/translate_init.c
index 741b390..255d25b 100644
--- a/target/mips/translate_init.c
+++ b/target/mips/translate_init.c
@@ -640,6 +640,7 @@ static const mips_def_t mips_defs[] =
 .SYNCI_Step = 32,
 .CCRes = 2,
 .CP0_Status_rw_bitmask = 0x36FB,
+.CP0_EBaseWG_rw_bitmask = (1 << CP0EBase_WG),
 .CP1_fcr0 = (1 << FCR0_F64) | (1 << FCR0_3D) | (1 << FCR0_PS) |
 (1 << FCR0_L) | (1 << FCR0_W) | (1 << FCR0_D) |
 (1 << FCR0_S) | (0x00 << FCR0_PRID) | (0x0 << FCR0_REV),
@@ -723,6 +724,7 @@ static const mips_def_t mips_defs[] =
 .CP0_PageGrain = (1 << CP0PG_IEC) | (1 << CP0PG_XIE) |
  (1U << CP0PG_RIE),
 .CP0_PageGrain_rw_bitmask = (1 << CP0PG_ELPA),
+.CP0_EBaseWG_rw_bitmask = (1 << CP0EBase_WG),
 .CP1_fcr0 = (1 << FCR0_FREP) | (1 << FCR0_HAS2008) | (1 << FCR0_F64) |
 (1 << FCR0_L) | (1 << FCR0_W) | (1 << FCR0_D) |
 (1 << FCR0_S) | (0x03 << FCR0_PRID) | (0x0 << FCR0_REV),
-- 
2.7.4

Re: [Qemu-devel] [PATCH v5 17/17] migration: Flush receive queue

[Peter Xu]

On Mon, Jul 17, 2017 at 03:42:38PM +0200, Juan Quintela wrote:
> Each time that we sync the bitmap, it is a possiblity that we receive
> a page that is being processed by a different thread.  We fix this
> problem just making sure that we wait for all receiving threads to
> finish its work before we procedeed with the next stage.
> 
> We are low on page flags, so we use a combination that is not valid to
> emit that message:  MULTIFD_PAGE and COMPRESSED.
> 
> I tried to make a migration command for it, but it don't work because
> we sync the bitmap sometimes when we have already sent the beggining
> of the section, so I just added a new page flag.
> 
> Signed-off-by: Juan Quintela 
> ---
>  migration/ram.c | 57 
> -
>  1 file changed, 56 insertions(+), 1 deletion(-)
> 
> diff --git a/migration/ram.c b/migration/ram.c
> index c78b286..bffe204 100644
> --- a/migration/ram.c
> +++ b/migration/ram.c
> @@ -71,6 +71,12 @@
>  #define RAM_SAVE_FLAG_COMPRESS_PAGE0x100
>  #define RAM_SAVE_FLAG_MULTIFD_PAGE 0x200
>  
> +/* We are getting low on pages flags, so we start using combinations
> +   When we need to flush a page, we sent it as
> +   RAM_SAVE_FLAG_MULTIFD_PAGE | RAM_SAVE_FLAG_COMPRESS_PAGE
> +   We don't allow that combination
> +*/
> +
>  static inline bool is_zero_range(uint8_t *p, uint64_t size)
>  {
>  return buffer_is_zero(p, size);
> @@ -193,6 +199,9 @@ struct RAMState {
>  uint64_t iterations_prev;
>  /* Iterations since start */
>  uint64_t iterations;
> +/* Indicates if we have synced the bitmap and we need to assure that
> +   target has processeed all previous pages */
> +bool multifd_needs_flush;
>  /* protects modification of the bitmap */
>  uint64_t migration_dirty_pages;
>  /* number of dirty bits in the bitmap */
> @@ -363,7 +372,6 @@ static void compress_threads_save_setup(void)
>  
>  /* Multiple fd's */
>  
> -
>  typedef struct {
>  int num;
>  int size;
> @@ -595,9 +603,11 @@ struct MultiFDRecvParams {
>  QIOChannel *c;
>  QemuSemaphore ready;
>  QemuSemaphore sem;
> +QemuCond cond_sync;
>  QemuMutex mutex;
>  /* proteced by param mutex */
>  bool quit;
> +bool sync;
>  multifd_pages_t pages;
>  bool done;
>  };
> @@ -637,6 +647,7 @@ void multifd_load_cleanup(void)
>  qemu_thread_join(&p->thread);
>  qemu_mutex_destroy(&p->mutex);
>  qemu_sem_destroy(&p->sem);
> +qemu_cond_destroy(&p->cond_sync);
>  socket_recv_channel_destroy(p->c);
>  g_free(p);
>  multifd_recv_state->params[i] = NULL;
> @@ -675,6 +686,10 @@ static void *multifd_recv_thread(void *opaque)
>  return NULL;
>  }
>  p->done = true;
> +if (p->sync) {
> +qemu_cond_signal(&p->cond_sync);
> +p->sync = false;
> +}

Could we use the same p->ready for this purpose? They looks similar:
all we want to do is to let the main thread know "worker thread has
finished receiving the last piece and becomes idle again", right?

>  qemu_mutex_unlock(&p->mutex);
>  qemu_sem_post(&p->ready);
>  continue;
> @@ -724,9 +739,11 @@ gboolean multifd_new_channel(QIOChannel *ioc)
>  qemu_mutex_init(&p->mutex);
>  qemu_sem_init(&p->sem, 0);
>  qemu_sem_init(&p->ready, 0);
> +qemu_cond_init(&p->cond_sync);
>  p->quit = false;
>  p->id = id;
>  p->done = false;
> +p->sync = false;
>  multifd_init_group(&p->pages);
>  p->c = ioc;
>  atomic_set(&multifd_recv_state->params[id], p);
> @@ -792,6 +809,27 @@ static void multifd_recv_page(uint8_t *address, uint16_t 
> fd_num)
>  qemu_sem_post(&p->sem);
>  }
>  
> +static int multifd_flush(void)
> +{
> +int i, thread_count;
> +
> +if (!migrate_use_multifd()) {
> +return 0;
> +}
> +thread_count = migrate_multifd_threads();
> +for (i = 0; i < thread_count; i++) {
> +MultiFDRecvParams *p = multifd_recv_state->params[i];
> +
> +qemu_mutex_lock(&p->mutex);
> +while (!p->done) {
> +p->sync = true;
> +qemu_cond_wait(&p->cond_sync, &p->mutex);

(similar comment like above)

> +}
> +qemu_mutex_unlock(&p->mutex);
> +}
> +return 0;
> +}
> +
>  /**
>   * save_page_header: write page header to wire
>   *
> @@ -809,6 +847,12 @@ static size_t save_page_header(RAMState *rs, QEMUFile 
> *f,  RAMBlock *block,
>  {
>  size_t size, len;
>  
> +if (rs->multifd_needs_flush &&
> +(offset & RAM_SAVE_FLAG_MULTIFD_PAGE)) {

If multifd_needs_flush is only for multifd, then we may skip this
check, but it looks more like an assertion:

if (rs->multifd_needs_flush) {
assert(offset & RAM_SAVE_FLAG_MULTIFD_PAGE);
offset |= RAM_SAVE_FLAG_ZERO;
}

(Dave mentioned about unaligned flag used in commit message and here:
 ZERO is use

[Qemu-devel] [PULL 03/14] target/mips: Weaken TLB flush on UX, SX, KX, ASID changes

[Yongbok Kim]

From: James Hogan 

There is no need to invalidate any shadow TLB entries when the ASID
changes or when access to one of the 64-bit segments has been disabled,
since doing so doesn't reveal to software whether any TLB entries have
been evicted into the shadow half of the TLB.

Therefore weaken the tlb flushes in these cases to only flush the QEMU
TLB.

Signed-off-by: James Hogan 
Cc: Yongbok Kim 
Cc: Aurelien Jarno 
Tested-by: Yongbok Kim 
Signed-off-by: Yongbok Kim 
---
 target/mips/helper.c| 2 +-
 target/mips/op_helper.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/target/mips/helper.c b/target/mips/helper.c
index 166f0d1..11d6a86 100644
--- a/target/mips/helper.c
+++ b/target/mips/helper.c
@@ -290,7 +290,7 @@ void cpu_mips_store_status(CPUMIPSState *env, target_ulong 
val)
 #if defined(TARGET_MIPS64)
 if ((env->CP0_Status ^ old) & (old & (7 << CP0St_UX))) {
 /* Access to at least one of the 64-bit segments has been disabled */
-cpu_mips_tlb_flush(env);
+tlb_flush(CPU(mips_env_get_cpu(env)));
 }
 #endif
 if (env->CP0_Config3 & (1 << CP0C3_MT)) {
diff --git a/target/mips/op_helper.c b/target/mips/op_helper.c
index 6393eff..091afd5 100644
--- a/target/mips/op_helper.c
+++ b/target/mips/op_helper.c
@@ -1416,7 +1416,7 @@ void helper_mtc0_entryhi(CPUMIPSState *env, target_ulong 
arg1)
 /* If the ASID changes, flush qemu's TLB.  */
 if ((old & env->CP0_EntryHi_ASID_mask) !=
 (val & env->CP0_EntryHi_ASID_mask)) {
-cpu_mips_tlb_flush(env);
+tlb_flush(CPU(mips_env_get_cpu(env)));
 }
 }
 
-- 
2.7.4

[Qemu-devel] [PULL 06/14] target/mips: Decode MIPS32 EVA load & store instructions

[Yongbok Kim]

From: James Hogan 

Implement decoding of MIPS32 EVA loads and stores. These access the user
address space from kernel mode when implemented, so for each instruction
we need to check that EVA is available from Config5.EVA & check for
sufficient COP0 privilege (with the new check_eva()), and then override
the mem_idx used for the operation.

Unfortunately some Loongson 2E instructions use overlapping encodings,
so we must be careful not to prevent those from being decoded when EVA
is absent.

Signed-off-by: James Hogan 
Cc: Yongbok Kim 
Cc: Aurelien Jarno 
Reviewed-by: Yongbok Kim 
Signed-off-by: Yongbok Kim 
---
 target/mips/translate.c | 106 
 1 file changed, 106 insertions(+)

diff --git a/target/mips/translate.c b/target/mips/translate.c
index 38887a1..93fb8f3 100644
--- a/target/mips/translate.c
+++ b/target/mips/translate.c
@@ -427,6 +427,24 @@ enum {
 OPC_EXTR_W_DSP = 0x38 | OPC_SPECIAL3,
 OPC_DEXTR_W_DSP= 0x3C | OPC_SPECIAL3,
 
+/* EVA */
+OPC_LWLE   = 0x19 | OPC_SPECIAL3,
+OPC_LWRE   = 0x1A | OPC_SPECIAL3,
+OPC_CACHEE = 0x1B | OPC_SPECIAL3,
+OPC_SBE= 0x1C | OPC_SPECIAL3,
+OPC_SHE= 0x1D | OPC_SPECIAL3,
+OPC_SCE= 0x1E | OPC_SPECIAL3,
+OPC_SWE= 0x1F | OPC_SPECIAL3,
+OPC_SWLE   = 0x21 | OPC_SPECIAL3,
+OPC_SWRE   = 0x22 | OPC_SPECIAL3,
+OPC_PREFE  = 0x23 | OPC_SPECIAL3,
+OPC_LBUE   = 0x28 | OPC_SPECIAL3,
+OPC_LHUE   = 0x29 | OPC_SPECIAL3,
+OPC_LBE= 0x2C | OPC_SPECIAL3,
+OPC_LHE= 0x2D | OPC_SPECIAL3,
+OPC_LLE= 0x2E | OPC_SPECIAL3,
+OPC_LWE= 0x2F | OPC_SPECIAL3,
+
 /* R6 */
 R6_OPC_PREF= 0x35 | OPC_SPECIAL3,
 R6_OPC_CACHE   = 0x25 | OPC_SPECIAL3,
@@ -1431,6 +1449,7 @@ typedef struct DisasContext {
 bool bp;
 uint64_t PAMask;
 bool mvh;
+bool eva;
 int CP0_LLAddr_shift;
 bool ps;
 bool vp;
@@ -2216,29 +2235,47 @@ static void gen_ld(DisasContext *ctx, uint32_t opc,
 tcg_gen_qemu_ld_tl(t0, t0, mem_idx, MO_TESL);
 gen_store_gpr(t0, rt);
 break;
+case OPC_LWE:
+mem_idx = MIPS_HFLAG_UM;
+/* fall through */
 case OPC_LW:
 tcg_gen_qemu_ld_tl(t0, t0, mem_idx, MO_TESL |
ctx->default_tcg_memop_mask);
 gen_store_gpr(t0, rt);
 break;
+case OPC_LHE:
+mem_idx = MIPS_HFLAG_UM;
+/* fall through */
 case OPC_LH:
 tcg_gen_qemu_ld_tl(t0, t0, mem_idx, MO_TESW |
ctx->default_tcg_memop_mask);
 gen_store_gpr(t0, rt);
 break;
+case OPC_LHUE:
+mem_idx = MIPS_HFLAG_UM;
+/* fall through */
 case OPC_LHU:
 tcg_gen_qemu_ld_tl(t0, t0, mem_idx, MO_TEUW |
ctx->default_tcg_memop_mask);
 gen_store_gpr(t0, rt);
 break;
+case OPC_LBE:
+mem_idx = MIPS_HFLAG_UM;
+/* fall through */
 case OPC_LB:
 tcg_gen_qemu_ld_tl(t0, t0, mem_idx, MO_SB);
 gen_store_gpr(t0, rt);
 break;
+case OPC_LBUE:
+mem_idx = MIPS_HFLAG_UM;
+/* fall through */
 case OPC_LBU:
 tcg_gen_qemu_ld_tl(t0, t0, mem_idx, MO_UB);
 gen_store_gpr(t0, rt);
 break;
+case OPC_LWLE:
+mem_idx = MIPS_HFLAG_UM;
+/* fall through */
 case OPC_LWL:
 t1 = tcg_temp_new();
 /* Do a byte access to possibly trigger a page
@@ -2262,6 +2299,9 @@ static void gen_ld(DisasContext *ctx, uint32_t opc,
 tcg_gen_ext32s_tl(t0, t0);
 gen_store_gpr(t0, rt);
 break;
+case OPC_LWRE:
+mem_idx = MIPS_HFLAG_UM;
+/* fall through */
 case OPC_LWR:
 t1 = tcg_temp_new();
 /* Do a byte access to possibly trigger a page
@@ -2286,6 +2326,9 @@ static void gen_ld(DisasContext *ctx, uint32_t opc,
 tcg_gen_ext32s_tl(t0, t0);
 gen_store_gpr(t0, rt);
 break;
+case OPC_LLE:
+mem_idx = MIPS_HFLAG_UM;
+/* fall through */
 case OPC_LL:
 case R6_OPC_LL:
 op_ld_ll(t0, t0, mem_idx, ctx);
@@ -2318,20 +2361,35 @@ static void gen_st (DisasContext *ctx, uint32_t opc, 
int rt,
 gen_helper_0e2i(sdr, t1, t0, mem_idx);
 break;
 #endif
+case OPC_SWE:
+mem_idx = MIPS_HFLAG_UM;
+/* fall through */
 case OPC_SW:
 tcg_gen_qemu_st_tl(t1, t0, mem_idx, MO_TEUL |
ctx->default_tcg_memop_mask);
 break;
+case OPC_SHE:
+mem_idx = MIPS_HFLAG_UM;
+/* fall through */
 case OPC_SH:
 tcg_gen_qemu_st_tl(t1, t0, mem_idx, MO_TEUW |
ctx->default_tcg_memop_mask);
 break;
+case OPC_SBE:
+mem_idx = MIPS_HFLAG_UM;
+/* fall through */
 case OPC_SB:
 

[Qemu-devel] [PULL 12/14] target/mips: Implement segmentation control

[Yongbok Kim]

From: James Hogan 

Implement the optional segmentation control feature in the virtual to
physical address translation code.

The fixed legacy segment and xkphys handling is replaced with a dynamic
layout based on the segmentation control registers (which should be set
up even when the feature is not exposed to the guest).

Signed-off-by: James Hogan 
Cc: Yongbok Kim 
Cc: Aurelien Jarno 
Reviewed-by: Yongbok Kim 
[yongbok@imgtec.com:
  cosmetic changes]
Signed-off-by: Yongbok Kim 
---
 target/mips/helper.c | 191 ---
 1 file changed, 152 insertions(+), 39 deletions(-)

diff --git a/target/mips/helper.c b/target/mips/helper.c
index 7805e5c..a2b79e8 100644
--- a/target/mips/helper.c
+++ b/target/mips/helper.c
@@ -107,15 +107,107 @@ int r4k_map_address (CPUMIPSState *env, hwaddr 
*physical, int *prot,
 return TLBRET_NOMATCH;
 }
 
+static int is_seg_am_mapped(unsigned int am, bool eu, int mmu_idx)
+{
+/*
+ * Interpret access control mode and mmu_idx.
+ *   AdE? TLB?
+ *  AM  K S U E  K S U E
+ * UK0  0 1 1 0  0 - - 0
+ * MK1  0 1 1 0  1 - - !eu
+ * MSK   2  0 0 1 0  1 1 - !eu
+ * MUSK  3  0 0 0 0  1 1 1 !eu
+ * MUSUK 4  0 0 0 0  0 1 1 0
+ * USK   5  0 0 1 0  0 0 - 0
+ * - 6  - - - -  - - - -
+ * UUSK  7  0 0 0 0  0 0 0 0
+ */
+int32_t adetlb_mask;
+
+switch (mmu_idx) {
+case 3 /* ERL */:
+/* If EU is set, always unmapped */
+if (eu) {
+return 0;
+}
+/* fall through */
+case MIPS_HFLAG_KM:
+/* Never AdE, TLB mapped if AM={1,2,3} */
+adetlb_mask = 0x7000;
+goto check_tlb;
+
+case MIPS_HFLAG_SM:
+/* AdE if AM={0,1}, TLB mapped if AM={2,3,4} */
+adetlb_mask = 0xc038;
+goto check_ade;
+
+case MIPS_HFLAG_UM:
+/* AdE if AM={0,1,2,5}, TLB mapped if AM={3,4} */
+adetlb_mask = 0xe418;
+/* fall through */
+check_ade:
+/* does this AM cause AdE in current execution mode */
+if ((adetlb_mask << am) < 0) {
+return TLBRET_BADADDR;
+}
+adetlb_mask <<= 8;
+/* fall through */
+check_tlb:
+/* is this AM mapped in current execution mode */
+return ((adetlb_mask << am) < 0);
+default:
+assert(0);
+return TLBRET_BADADDR;
+};
+}
+
+static int get_seg_physical_address(CPUMIPSState *env, hwaddr *physical,
+int *prot, target_ulong real_address,
+int rw, int access_type, int mmu_idx,
+unsigned int am, bool eu,
+target_ulong segmask,
+hwaddr physical_base)
+{
+int mapped = is_seg_am_mapped(am, eu, mmu_idx);
+
+if (mapped < 0) {
+/* is_seg_am_mapped can report TLBRET_BADADDR */
+return mapped;
+} else if (mapped) {
+/* The segment is TLB mapped */
+return env->tlb->map_address(env, physical, prot, real_address, rw,
+ access_type);
+} else {
+/* The segment is unmapped */
+*physical = physical_base | (real_address & segmask);
+*prot = PAGE_READ | PAGE_WRITE;
+return TLBRET_MATCH;
+}
+}
+
+static int get_segctl_physical_address(CPUMIPSState *env, hwaddr *physical,
+   int *prot, target_ulong real_address,
+   int rw, int access_type, int mmu_idx,
+   uint16_t segctl, target_ulong segmask)
+{
+unsigned int am = (segctl & CP0SC_AM_MASK) >> CP0SC_AM;
+bool eu = (segctl >> CP0SC_EU) & 1;
+hwaddr pa = ((hwaddr)segctl & CP0SC_PA_MASK) << 20;
+
+return get_seg_physical_address(env, physical, prot, real_address, rw,
+access_type, mmu_idx, am, eu, segmask,
+pa & ~(hwaddr)segmask);
+}
+
 static int get_physical_address (CPUMIPSState *env, hwaddr *physical,
 int *prot, target_ulong real_address,
 int rw, int access_type, int mmu_idx)
 {
 /* User mode can only access useg/xuseg */
+#if defined(TARGET_MIPS64)
 int user_mode = mmu_idx == MIPS_HFLAG_UM;
 int supervisor_mode = mmu_idx == MIPS_HFLAG_SM;
 int kernel_mode = !user_mode && !supervisor_mode;
-#if defined(TARGET_MIPS64)
 int UX = (env->CP0_Status & (1 << CP0St_UX)) != 0;
 int SX = (env->CP0_Status & (1 << CP0St_SX)) != 0;
 int KX = (env->CP0_Status & (1 << CP0St_KX)) != 0;
@@ -148,12 +240,16 @@ static int get_physical_address (CPUMIPSState *env, 
hwaddr *physical,
 
 if (address <= USEG_LIMIT) {
 /* useg */
-if (env->CP0_Status & (1 << CP0St_ERL)) {
-*physical = address & 0x;
-*prot = PAGE

[Qemu-devel] [PULL 09/14] target/mips: Abstract mmu_idx from hflags

[Yongbok Kim]

From: James Hogan 

The MIPS mmu_idx is sometimes calculated from hflags without an env
pointer available as cpu_mmu_index() requires.

Create a common hflags_mmu_index() for the purpose of this calculation
which can operate on any hflags, not just with an env pointer, and
update cpu_mmu_index() itself and gen_intermediate_code() to use it.

Also update debug_post_eret() and helper_mtc0_status() to log the MMU
mode with the status change (SM, UM, or nothing for kernel mode) based
on cpu_mmu_index() rather than directly testing hflags.

This will also allow the logic to be more easily updated when a new MMU
mode is added.

Signed-off-by: James Hogan 
Reviewed-by: Yongbok Kim 
Cc: Aurelien Jarno 
Signed-off-by: Yongbok Kim 
---
 target/mips/cpu.h   | 8 +++-
 target/mips/op_helper.c | 4 ++--
 target/mips/translate.c | 2 +-
 3 files changed, 10 insertions(+), 4 deletions(-)

diff --git a/target/mips/cpu.h b/target/mips/cpu.h
index 2b699a0..3cf1676 100644
--- a/target/mips/cpu.h
+++ b/target/mips/cpu.h
@@ -699,9 +699,15 @@ extern uint32_t cpu_rddsp(uint32_t mask_num, CPUMIPSState 
*env);
 #define MMU_MODE1_SUFFIX _super
 #define MMU_MODE2_SUFFIX _user
 #define MMU_USER_IDX 2
+
+static inline int hflags_mmu_index(uint32_t hflags)
+{
+return hflags & MIPS_HFLAG_KSU;
+}
+
 static inline int cpu_mmu_index (CPUMIPSState *env, bool ifetch)
 {
-return env->hflags & MIPS_HFLAG_KSU;
+return hflags_mmu_index(env->hflags);
 }
 
 static inline bool cpu_mips_hw_interrupts_enabled(CPUMIPSState *env)
diff --git a/target/mips/op_helper.c b/target/mips/op_helper.c
index 3b560d9..da1817e 100644
--- a/target/mips/op_helper.c
+++ b/target/mips/op_helper.c
@@ -1450,7 +1450,7 @@ void helper_mtc0_status(CPUMIPSState *env, target_ulong 
arg1)
 old, old & env->CP0_Cause & CP0Ca_IP_mask,
 val, val & env->CP0_Cause & CP0Ca_IP_mask,
 env->CP0_Cause);
-switch (env->hflags & MIPS_HFLAG_KSU) {
+switch (cpu_mmu_index(env, false)) {
 case MIPS_HFLAG_UM: qemu_log(", UM\n"); break;
 case MIPS_HFLAG_SM: qemu_log(", SM\n"); break;
 case MIPS_HFLAG_KM: qemu_log("\n"); break;
@@ -2244,7 +2244,7 @@ static void debug_post_eret(CPUMIPSState *env)
 qemu_log(" ErrorEPC " TARGET_FMT_lx, env->CP0_ErrorEPC);
 if (env->hflags & MIPS_HFLAG_DM)
 qemu_log(" DEPC " TARGET_FMT_lx, env->CP0_DEPC);
-switch (env->hflags & MIPS_HFLAG_KSU) {
+switch (cpu_mmu_index(env, false)) {
 case MIPS_HFLAG_UM: qemu_log(", UM\n"); break;
 case MIPS_HFLAG_SM: qemu_log(", SM\n"); break;
 case MIPS_HFLAG_KM: qemu_log("\n"); break;
diff --git a/target/mips/translate.c b/target/mips/translate.c
index 9787919..4fb8217 100644
--- a/target/mips/translate.c
+++ b/target/mips/translate.c
@@ -20157,7 +20157,7 @@ void gen_intermediate_code(CPUState *cs, struct 
TranslationBlock *tb)
 #ifdef CONFIG_USER_ONLY
 ctx.mem_idx = MIPS_HFLAG_UM;
 #else
-ctx.mem_idx = ctx.hflags & MIPS_HFLAG_KSU;
+ctx.mem_idx = hflags_mmu_index(ctx.hflags);
 #endif
 ctx.default_tcg_memop_mask = (ctx.insn_flags & ISA_MIPS32R6) ?
  MO_UNALN : MO_ALIGN;
-- 
2.7.4

[Qemu-devel] [PULL 08/14] target/mips: Check memory permissions with mem_idx

[Yongbok Kim]

From: James Hogan 

When performing virtual to physical address translation, check the
required privilege level based on the mem_idx rather than the mode in
the hflags. This will allow EVA loads & stores to operate safely only on
user memory from kernel mode.

For the cases where the mmu_idx doesn't need to be overridden
(mips_cpu_get_phys_page_debug() and cpu_mips_translate_address()), we
calculate the required mmu_idx using cpu_mmu_index(). Note that this
only tests the MIPS_HFLAG_KSU bits rather than MIPS_HFLAG_MODE, so we
don't test the debug mode hflag MIPS_HFLAG_DM any longer. This should be
fine as get_physical_address() only compares against MIPS_HFLAG_UM and
MIPS_HFLAG_SM, neither of which should get set by compute_hflags() when
MIPS_HFLAG_DM is set.

Signed-off-by: James Hogan 
Reviewed-by: Yongbok Kim 
Cc: Aurelien Jarno 
Signed-off-by: Yongbok Kim 
---
 target/mips/helper.c | 17 +
 1 file changed, 9 insertions(+), 8 deletions(-)

diff --git a/target/mips/helper.c b/target/mips/helper.c
index 5b765cd..7805e5c 100644
--- a/target/mips/helper.c
+++ b/target/mips/helper.c
@@ -109,11 +109,11 @@ int r4k_map_address (CPUMIPSState *env, hwaddr *physical, 
int *prot,
 
 static int get_physical_address (CPUMIPSState *env, hwaddr *physical,
 int *prot, target_ulong real_address,
-int rw, int access_type)
+int rw, int access_type, int mmu_idx)
 {
 /* User mode can only access useg/xuseg */
-int user_mode = (env->hflags & MIPS_HFLAG_MODE) == MIPS_HFLAG_UM;
-int supervisor_mode = (env->hflags & MIPS_HFLAG_MODE) == MIPS_HFLAG_SM;
+int user_mode = mmu_idx == MIPS_HFLAG_UM;
+int supervisor_mode = mmu_idx == MIPS_HFLAG_SM;
 int kernel_mode = !user_mode && !supervisor_mode;
 #if defined(TARGET_MIPS64)
 int UX = (env->CP0_Status & (1 << CP0St_UX)) != 0;
@@ -413,11 +413,12 @@ static void raise_mmu_exception(CPUMIPSState *env, 
target_ulong address,
 hwaddr mips_cpu_get_phys_page_debug(CPUState *cs, vaddr addr)
 {
 MIPSCPU *cpu = MIPS_CPU(cs);
+CPUMIPSState *env = &cpu->env;
 hwaddr phys_addr;
 int prot;
 
-if (get_physical_address(&cpu->env, &phys_addr, &prot, addr, 0,
- ACCESS_INT) != 0) {
+if (get_physical_address(env, &phys_addr, &prot, addr, 0, ACCESS_INT,
+ cpu_mmu_index(env, false)) != 0) {
 return -1;
 }
 return phys_addr;
@@ -449,7 +450,7 @@ int mips_cpu_handle_mmu_fault(CPUState *cs, vaddr address, 
int rw,
correctly */
 access_type = ACCESS_INT;
 ret = get_physical_address(env, &physical, &prot,
-   address, rw, access_type);
+   address, rw, access_type, mmu_idx);
 switch (ret) {
 case TLBRET_MATCH:
 qemu_log_mask(CPU_LOG_MMU,
@@ -487,8 +488,8 @@ hwaddr cpu_mips_translate_address(CPUMIPSState *env, 
target_ulong address, int r
 
 /* data access */
 access_type = ACCESS_INT;
-ret = get_physical_address(env, &physical, &prot,
-   address, rw, access_type);
+ret = get_physical_address(env, &physical, &prot, address, rw, access_type,
+   cpu_mmu_index(env, false));
 if (ret != TLBRET_MATCH) {
 raise_mmu_exception(env, address, rw, ret);
 return -1LL;
-- 
2.7.4

[Qemu-devel] [PULL 05/14] target/mips: Prepare loads/stores for EVA

[Yongbok Kim]

From: James Hogan 

EVA load and store instructions access the user mode address map, so
they need to use mem_idx of MIPS_HFLAG_UM. Update the various utility
functions to allow mem_idx to be more easily overridden from the
decoding logic.

Specifically we add a mem_idx argument to the op_ld/st_* helpers used
for atomics, and a mem_idx local variable to gen_ld(), gen_st(), and
gen_st_cond().

Signed-off-by: James Hogan 
Reviewed-by: Yongbok Kim 
Cc: Yongbok Kim 
Cc: Aurelien Jarno 
Signed-off-by: Yongbok Kim 
---
 target/mips/translate.c | 77 +++--
 1 file changed, 42 insertions(+), 35 deletions(-)

diff --git a/target/mips/translate.c b/target/mips/translate.c
index 76dcc5e..38887a1 100644
--- a/target/mips/translate.c
+++ b/target/mips/translate.c
@@ -2029,7 +2029,8 @@ FOP_CONDNS(s, FMT_S, 32, gen_store_fpr32(ctx, fp0, fd))
 /* load/store instructions. */
 #ifdef CONFIG_USER_ONLY
 #define OP_LD_ATOMIC(insn,fname)   \
-static inline void op_ld_##insn(TCGv ret, TCGv arg1, DisasContext *ctx)\
+static inline void op_ld_##insn(TCGv ret, TCGv arg1, int mem_idx,  \
+DisasContext *ctx) \
 {  \
 TCGv t0 = tcg_temp_new();  \
 tcg_gen_mov_tl(t0, arg1);  \
@@ -2040,9 +2041,10 @@ static inline void op_ld_##insn(TCGv ret, TCGv arg1, 
DisasContext *ctx)\
 }
 #else
 #define OP_LD_ATOMIC(insn,fname)   \
-static inline void op_ld_##insn(TCGv ret, TCGv arg1, DisasContext *ctx)\
+static inline void op_ld_##insn(TCGv ret, TCGv arg1, int mem_idx,  \
+DisasContext *ctx) \
 {  \
-gen_helper_1e1i(insn, ret, arg1, ctx->mem_idx);\
+gen_helper_1e1i(insn, ret, arg1, mem_idx); \
 }
 #endif
 OP_LD_ATOMIC(ll,ld32s);
@@ -2053,7 +2055,8 @@ OP_LD_ATOMIC(lld,ld64);
 
 #ifdef CONFIG_USER_ONLY
 #define OP_ST_ATOMIC(insn,fname,ldname,almask)   \
-static inline void op_st_##insn(TCGv arg1, TCGv arg2, int rt, DisasContext 
*ctx) \
+static inline void op_st_##insn(TCGv arg1, TCGv arg2, int rt, int mem_idx,   \
+DisasContext *ctx)   \
 {\
 TCGv t0 = tcg_temp_new();\
 TCGLabel *l1 = gen_new_label();  \
@@ -2077,10 +2080,11 @@ static inline void op_st_##insn(TCGv arg1, TCGv arg2, 
int rt, DisasContext *ctx)
 }
 #else
 #define OP_ST_ATOMIC(insn,fname,ldname,almask)   \
-static inline void op_st_##insn(TCGv arg1, TCGv arg2, int rt, DisasContext 
*ctx) \
+static inline void op_st_##insn(TCGv arg1, TCGv arg2, int rt, int mem_idx,   \
+DisasContext *ctx)   \
 {\
 TCGv t0 = tcg_temp_new();\
-gen_helper_1e2i(insn, t0, arg1, arg2, ctx->mem_idx); \
+gen_helper_1e2i(insn, t0, arg1, arg2, mem_idx);  \
 gen_store_gpr(t0, rt);   \
 tcg_temp_free(t0);   \
 }
@@ -2123,6 +2127,7 @@ static void gen_ld(DisasContext *ctx, uint32_t opc,
int rt, int base, int16_t offset)
 {
 TCGv t0, t1, t2;
+int mem_idx = ctx->mem_idx;
 
 if (rt == 0 && ctx->insn_flags & (INSN_LOONGSON2E | INSN_LOONGSON2F)) {
 /* Loongson CPU uses a load to zero register for prefetch.
@@ -2137,32 +2142,32 @@ static void gen_ld(DisasContext *ctx, uint32_t opc,
 switch (opc) {
 #if defined(TARGET_MIPS64)
 case OPC_LWU:
-tcg_gen_qemu_ld_tl(t0, t0, ctx->mem_idx, MO_TEUL |
+tcg_gen_qemu_ld_tl(t0, t0, mem_idx, MO_TEUL |
ctx->default_tcg_memop_mask);
 gen_store_gpr(t0, rt);
 break;
 case OPC_LD:
-tcg_gen_qemu_ld_tl(t0, t0, ctx->mem_idx, MO_TEQ |
+tcg_gen_qemu_ld_tl(t0, t0, mem_idx, MO_TEQ |
ctx->default_tcg_memop_mask);
 gen_store_gpr(t0, rt);
 break;
 case OPC_LLD:
 case R6_OPC_LLD:
-op_ld_lld(t0, t0, ctx);
+op_ld_lld(t0, t0, mem_idx, ctx);
 gen_store_gpr(t0, rt);
 break;
 case OPC_LDL:
 t1 = tcg_temp_new();
 /* Do a byte access to possibly trigger a page
fault with the unaligned address.  */
- 

[Qemu-devel] [PULL 02/14] target/mips: Fix TLBWI shadow flush for EHINV, XI, RI

[Yongbok Kim]

From: James Hogan 

Writing specific TLB entries with TLBWI flushes shadow TLB entries
unless an existing entry is having its access permissions upgraded. This
is necessary as software would from then on expect the previous mapping
in that entry to no longer be in effect (even if QEMU has quietly
evicted it to the shadow TLB on a TLBWR).

However it won't do this if only EHINV, XI, or RI bits have been set,
even if that results in a reduction of permissions, so add the necessary
checks to invoke the flush when these bits are set.

Fixes: 2fb58b73746e ("target-mips: add RI and XI fields to TLB entry")
Fixes: 9456c2fbcd82 ("target-mips: add TLBINV support")
Signed-off-by: James Hogan 
Cc: Yongbok Kim 
Cc: Aurelien Jarno 
Tested-by: Yongbok Kim 
[yongbok@imgtec.com:
  cosmetic changes]
Signed-off-by: Yongbok Kim 
---
 target/mips/op_helper.c | 12 ++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/target/mips/op_helper.c b/target/mips/op_helper.c
index e5f3ea4..6393eff 100644
--- a/target/mips/op_helper.c
+++ b/target/mips/op_helper.c
@@ -2029,7 +2029,7 @@ void r4k_helper_tlbwi(CPUMIPSState *env)
 int idx;
 target_ulong VPN;
 uint16_t ASID;
-bool G, V0, D0, V1, D1;
+bool EHINV, G, V0, D0, V1, D1, XI0, XI1, RI0, RI1;
 
 idx = (env->CP0_Index & ~0x8000) % env->tlb->nb_tlb;
 tlb = &env->tlb->mmu.r4k.tlb[idx];
@@ -2038,17 +2038,25 @@ void r4k_helper_tlbwi(CPUMIPSState *env)
 VPN &= env->SEGMask;
 #endif
 ASID = env->CP0_EntryHi & env->CP0_EntryHi_ASID_mask;
+EHINV = (env->CP0_EntryHi & (1 << CP0EnHi_EHINV)) != 0;
 G = env->CP0_EntryLo0 & env->CP0_EntryLo1 & 1;
 V0 = (env->CP0_EntryLo0 & 2) != 0;
 D0 = (env->CP0_EntryLo0 & 4) != 0;
+XI0 = (env->CP0_EntryLo0 >> CP0EnLo_XI) &1;
+RI0 = (env->CP0_EntryLo0 >> CP0EnLo_RI) &1;
 V1 = (env->CP0_EntryLo1 & 2) != 0;
 D1 = (env->CP0_EntryLo1 & 4) != 0;
+XI1 = (env->CP0_EntryLo1 >> CP0EnLo_XI) &1;
+RI1 = (env->CP0_EntryLo1 >> CP0EnLo_RI) &1;
 
 /* Discard cached TLB entries, unless tlbwi is just upgrading access
permissions on the current entry. */
 if (tlb->VPN != VPN || tlb->ASID != ASID || tlb->G != G ||
+(!tlb->EHINV && EHINV) ||
 (tlb->V0 && !V0) || (tlb->D0 && !D0) ||
-(tlb->V1 && !V1) || (tlb->D1 && !D1)) {
+(!tlb->XI0 && XI0) || (!tlb->RI0 && RI0) ||
+(tlb->V1 && !V1) || (tlb->D1 && !D1) ||
+(!tlb->XI1 && XI1) || (!tlb->RI1 && RI1)) {
 r4k_mips_tlb_flush_extra(env, env->tlb->nb_tlb);
 }
 
-- 
2.7.4

[Qemu-devel] [PULL 04/14] target/mips: Add CP0_Ebase.WG (write gate) support

[Yongbok Kim]

From: James Hogan 

Add support for the CP0_EBase.WG bit, which allows upper bits to be
written (bits 31:30 on MIPS32, or bits 63:30 on MIPS64), along with the
CP0_Config5.CV bit to control whether the exception vector for Cache
Error exceptions is forced into KSeg1.

This is necessary on MIPS32 to support Segmentation Control and Enhanced
Virtual Addressing (EVA) extensions (where KSeg1 addresses may not
represent an unmapped uncached segment).

It is also useful on MIPS64 to allow the exception base to reside in
XKPhys, and possibly out of range of KSEG0 and KSEG1.

Signed-off-by: James Hogan 
Cc: Yongbok Kim 
Cc: Aurelien Jarno 
Reviewed-by: Yongbok Kim 
[yongbok@imgtec.com:
  minor changes]
Signed-off-by: Yongbok Kim 
---
 target/mips/cpu.h|  5 -
 target/mips/helper.c | 14 --
 target/mips/machine.c|  6 +++---
 target/mips/op_helper.c  | 12 ++--
 target/mips/translate.c  |  8 +---
 target/mips/translate_init.c |  1 +
 6 files changed, 31 insertions(+), 15 deletions(-)

diff --git a/target/mips/cpu.h b/target/mips/cpu.h
index 4a4747a..2b699a0 100644
--- a/target/mips/cpu.h
+++ b/target/mips/cpu.h
@@ -399,7 +399,9 @@ struct CPUMIPSState {
 #define CP0Ca_EC2
 target_ulong CP0_EPC;
 int32_t CP0_PRid;
-int32_t CP0_EBase;
+target_ulong CP0_EBase;
+target_ulong CP0_EBaseWG_rw_bitmask;
+#define CP0EBase_WG 11
 target_ulong CP0_CMGCRBase;
 int32_t CP0_Config0;
 #define CP0C0_M31
@@ -447,6 +449,7 @@ struct CPUMIPSState {
 #define CP0C3_MSAP  28
 #define CP0C3_BP 27
 #define CP0C3_BI 26
+#define CP0C3_SC 25
 #define CP0C3_IPLW 21
 #define CP0C3_MMAR 18
 #define CP0C3_MCU  17
diff --git a/target/mips/helper.c b/target/mips/helper.c
index 11d6a86..5b765cd 100644
--- a/target/mips/helper.c
+++ b/target/mips/helper.c
@@ -831,11 +831,7 @@ void mips_cpu_do_interrupt(CPUState *cs)
 goto set_EPC;
 case EXCP_CACHE:
 cause = 30;
-if (env->CP0_Status & (1 << CP0St_BEV)) {
-offset = 0x100;
-} else {
-offset = 0x2100;
-}
+offset = 0x100;
  set_EPC:
 if (!(env->CP0_Status & (1 << CP0St_EXL))) {
 env->CP0_EPC = exception_resume_pc(env);
@@ -861,9 +857,15 @@ void mips_cpu_do_interrupt(CPUState *cs)
 env->hflags &= ~MIPS_HFLAG_BMASK;
 if (env->CP0_Status & (1 << CP0St_BEV)) {
 env->active_tc.PC = env->exception_base + 0x200;
+} else if (cause == 30 && !(env->CP0_Config3 & (1 << CP0C3_SC) &&
+env->CP0_Config5 & (1 << CP0C5_CV))) {
+/* Force KSeg1 for cache errors */
+env->active_tc.PC = (int32_t)KSEG1_BASE |
+(env->CP0_EBase & 0x1000);
 } else {
-env->active_tc.PC = (int32_t)(env->CP0_EBase & ~0x3ff);
+env->active_tc.PC = env->CP0_EBase & ~0xfff;
 }
+
 env->active_tc.PC += offset;
 set_hflags_for_handler(env);
 env->CP0_Cause = (env->CP0_Cause & ~(0x1f << CP0Ca_EC)) | (cause << 
CP0Ca_EC);
diff --git a/target/mips/machine.c b/target/mips/machine.c
index 38c8fe9..91e31a7 100644
--- a/target/mips/machine.c
+++ b/target/mips/machine.c
@@ -211,8 +211,8 @@ const VMStateDescription vmstate_tlb = {
 
 const VMStateDescription vmstate_mips_cpu = {
 .name = "cpu",
-.version_id = 8,
-.minimum_version_id = 8,
+.version_id = 9,
+.minimum_version_id = 9,
 .post_load = cpu_post_load,
 .fields = (VMStateField[]) {
 /* Active TC */
@@ -272,7 +272,7 @@ const VMStateDescription vmstate_mips_cpu = {
 VMSTATE_INT32(env.CP0_Cause, MIPSCPU),
 VMSTATE_UINTTL(env.CP0_EPC, MIPSCPU),
 VMSTATE_INT32(env.CP0_PRid, MIPSCPU),
-VMSTATE_INT32(env.CP0_EBase, MIPSCPU),
+VMSTATE_UINTTL(env.CP0_EBase, MIPSCPU),
 VMSTATE_INT32(env.CP0_Config0, MIPSCPU),
 VMSTATE_INT32(env.CP0_Config1, MIPSCPU),
 VMSTATE_INT32(env.CP0_Config2, MIPSCPU),
diff --git a/target/mips/op_helper.c b/target/mips/op_helper.c
index 091afd5..3b560d9 100644
--- a/target/mips/op_helper.c
+++ b/target/mips/op_helper.c
@@ -1515,14 +1515,22 @@ target_ulong helper_mftc0_ebase(CPUMIPSState *env)
 
 void helper_mtc0_ebase(CPUMIPSState *env, target_ulong arg1)
 {
-env->CP0_EBase = (env->CP0_EBase & ~0x3000) | (arg1 & 0x3000);
+target_ulong mask = 0x3000 | env->CP0_EBaseWG_rw_bitmask;
+if (arg1 & env->CP0_EBaseWG_rw_bitmask) {
+mask |= ~0x3FFF;
+}
+env->CP0_EBase = (env->CP0_EBase & ~mask) | (arg1 & mask);
 }
 
 void helper_mttc0_ebase(CPUMIPSState *env, target_ulong arg1)
 {
 int other_tc = env->CP0_VPEControl & (0xff << CP0VPECo_TargTC);
 CPUMIPSState *other = mips_cpu_map_tc(env, &other_tc);
-other->CP0_EBase = (other->CP0_EBase & ~0x3000) | (arg1 & 0x3000);
+target_ulong mask = 0x3000 | env->CP0_EBaseWG_rw_bitmask;
+if (arg1 & env->CP0_EB

[Qemu-devel] [PULL 00/14] target-mips queue

[Yongbok Kim]

The following changes since commit 25d0233c1ac6cd14a15fcc834f1de3b179037b1d:

  Merge remote-tracking branch 'remotes/kraxel/tags/ui-20170720-pull-request' 
into staging (2017-07-20 16:40:01 +0100)

are available in the git repository at:

  git://github.com/yongbok/upstream-qemu.git tags/mips-20170721

for you to fetch changes up to bad63a8008a0aaefcd00542c89bee01623d7c9de:

  target/mips: Enable CP0_EBase.WG on MIPS64 CPUs (2017-07-21 03:23:44 +0100)


MIPS patches 2017-07-21

Changes:
* Add Enhanced Virtual Addressing (EVA) support




James Hogan (14):
  target/mips: Fix MIPS64 MFC0 UserLocal on BE host
  target/mips: Fix TLBWI shadow flush for EHINV,XI,RI
  target/mips: Weaken TLB flush on UX,SX,KX,ASID changes
  target/mips: Add CP0_Ebase.WG (write gate) support
  target/mips: Prepare loads/stores for EVA
  target/mips: Decode MIPS32 EVA load & store instructions
  target/mips: Decode microMIPS EVA load & store instructions
  target/mips: Check memory permissions with mem_idx
  target/mips: Abstract mmu_idx from hflags
  target/mips: Add an MMU mode for ERL
  target/mips: Add segmentation control registers
  target/mips: Implement segmentation control
  target/mips: Add EVA support to P5600
  target/mips: Enable CP0_EBase.WG on MIPS64 CPUs

 target/mips/cpu.h|  58 ++-
 target/mips/helper.c | 224 ++--
 target/mips/helper.h |   3 +
 target/mips/machine.c|   9 +-
 target/mips/op_helper.c  |  64 ++-
 target/mips/translate.c  | 405 ++-
 target/mips/translate_init.c |  17 +-
 7 files changed, 660 insertions(+), 120 deletions(-)

-- 
2.7.4

[Qemu-devel] [PULL 07/14] target/mips: Decode microMIPS EVA load & store instructions

[Yongbok Kim]

From: James Hogan 

Implement decoding of microMIPS EVA load and store instruction groups in
the POOL31C pool. These use the same gen_ld(), gen_st(), gen_st_cond()
helpers as the MIPS32 decoding, passing the equivalent MIPS32 opcodes as
opc.

Signed-off-by: James Hogan 
Cc: Yongbok Kim 
Cc: Aurelien Jarno 
Reviewed-by: Yongbok Kim 
Signed-off-by: Yongbok Kim 
---
 target/mips/translate.c | 119 ++--
 1 file changed, 115 insertions(+), 4 deletions(-)

diff --git a/target/mips/translate.c b/target/mips/translate.c
index 93fb8f3..9787919 100644
--- a/target/mips/translate.c
+++ b/target/mips/translate.c
@@ -12519,19 +12519,45 @@ enum {
 LWR = 0x1,
 SWR = 0x9,
 PREF = 0x2,
-/* 0xa is reserved */
+ST_EVA = 0xa,
 LL = 0x3,
 SC = 0xb,
 LDL = 0x4,
 SDL = 0xc,
 LDR = 0x5,
 SDR = 0xd,
-/* 0x6 is reserved */
+LD_EVA = 0x6,
 LWU = 0xe,
 LLD = 0x7,
 SCD = 0xf
 };
 
+/* POOL32C LD-EVA encoding of minor opcode field (bits 11..9) */
+
+enum {
+LBUE = 0x0,
+LHUE = 0x1,
+LWLE = 0x2,
+LWRE = 0x3,
+LBE = 0x4,
+LHE = 0x5,
+LLE = 0x6,
+LWE = 0x7,
+};
+
+/* POOL32C ST-EVA encoding of minor opcode field (bits 11..9) */
+
+enum {
+SWLE = 0x0,
+SWRE = 0x1,
+PREFE = 0x2,
+CACHEE = 0x3,
+SBE = 0x4,
+SHE = 0x5,
+SCE = 0x6,
+SWE = 0x7,
+};
+
 /* POOL32F encoding of minor opcode field (bits 5..0) */
 
 enum {
@@ -13832,7 +13858,7 @@ static void decode_micromips32_opc(CPUMIPSState *env, 
DisasContext *ctx)
 uint16_t insn;
 int rt, rs, rd, rr;
 int16_t imm;
-uint32_t op, minor, mips32_op;
+uint32_t op, minor, minor2, mips32_op;
 uint32_t cond, fmt, cc;
 
 insn = cpu_lduw_code(env, ctx->pc + 2);
@@ -14777,7 +14803,7 @@ static void decode_micromips32_opc(CPUMIPSState *env, 
DisasContext *ctx)
 gen_ld(ctx, mips32_op, rt, rs, offset);
 break;
 do_st_lr:
-gen_st(ctx, mips32_op, rt, rs, SIMM(ctx->opcode, 0, 12));
+gen_st(ctx, mips32_op, rt, rs, offset);
 break;
 case SC:
 gen_st_cond(ctx, OPC_SC, rt, rs, offset);
@@ -14789,6 +14815,91 @@ static void decode_micromips32_opc(CPUMIPSState *env, 
DisasContext *ctx)
 gen_st_cond(ctx, OPC_SCD, rt, rs, offset);
 break;
 #endif
+case LD_EVA:
+if (!ctx->eva) {
+MIPS_INVAL("pool32c ld-eva");
+generate_exception_end(ctx, EXCP_RI);
+break;
+}
+check_cp0_enabled(ctx);
+
+minor2 = (ctx->opcode >> 9) & 0x7;
+offset = sextract32(ctx->opcode, 0, 9);
+switch (minor2) {
+case LBUE:
+mips32_op = OPC_LBUE;
+goto do_ld_lr;
+case LHUE:
+mips32_op = OPC_LHUE;
+goto do_ld_lr;
+case LWLE:
+check_insn_opc_removed(ctx, ISA_MIPS32R6);
+mips32_op = OPC_LWLE;
+goto do_ld_lr;
+case LWRE:
+check_insn_opc_removed(ctx, ISA_MIPS32R6);
+mips32_op = OPC_LWRE;
+goto do_ld_lr;
+case LBE:
+mips32_op = OPC_LBE;
+goto do_ld_lr;
+case LHE:
+mips32_op = OPC_LHE;
+goto do_ld_lr;
+case LLE:
+mips32_op = OPC_LLE;
+goto do_ld_lr;
+case LWE:
+mips32_op = OPC_LWE;
+goto do_ld_lr;
+};
+break;
+case ST_EVA:
+if (!ctx->eva) {
+MIPS_INVAL("pool32c st-eva");
+generate_exception_end(ctx, EXCP_RI);
+break;
+}
+check_cp0_enabled(ctx);
+
+minor2 = (ctx->opcode >> 9) & 0x7;
+offset = sextract32(ctx->opcode, 0, 9);
+switch (minor2) {
+case SWLE:
+check_insn_opc_removed(ctx, ISA_MIPS32R6);
+mips32_op = OPC_SWLE;
+goto do_st_lr;
+case SWRE:
+check_insn_opc_removed(ctx, ISA_MIPS32R6);
+mips32_op = OPC_SWRE;
+goto do_st_lr;
+case PREFE:
+/* Treat as no-op */
+if ((ctx->insn_flags & ISA_MIPS32R6) && (rt >= 24)) {
+/* hint codes 24-31 are reserved and signal RI */
+generate_exception(ctx, EXCP_RI);
+}
+break;
+case CACHEE:
+/* Treat as no-op */
+if (ctx->hflags & MIPS_HFLAG_ITC_CACHE) {
+gen_cache_operation(ctx, rt, rs, offset);
+}
+break;
+case SBE:
+mips32_op = OPC_SBE;
+goto do_st_lr;
+case SHE:
+mips32_op 

[Qemu-devel] [PULL 01/14] target/mips: Fix MIPS64 MFC0 UserLocal on BE host

[Yongbok Kim]

From: James Hogan 

Using MFC0 to read CP0_UserLocal uses tcg_gen_ld32s_tl, however
CP0_UserLocal is a target_ulong. On a big endian host with a MIPS64
target this reads and sign extends the more significant half of the
64-bit register.

Fix this by using ld_tl to load the whole target_ulong and ext32s_tl to
sign extend it, as done for various other target_ulong COP0 registers.

Fixes: d279279e2b5c ("target-mips: implement UserLocal Register")
Signed-off-by: James Hogan 
Cc: Yongbok Kim 
Cc: Aurelien Jarno 
Cc: Petar Jovanovic 
Reviewed-by: Yongbok Kim 
Signed-off-by: Yongbok Kim 
---
 target/mips/translate.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/target/mips/translate.c b/target/mips/translate.c
index 1fd18e9..db6e5b5 100644
--- a/target/mips/translate.c
+++ b/target/mips/translate.c
@@ -5144,8 +5144,9 @@ static void gen_mfc0(DisasContext *ctx, TCGv arg, int 
reg, int sel)
 goto cp0_unimplemented;
 case 2:
 CP0_CHECK(ctx->ulri);
-tcg_gen_ld32s_tl(arg, cpu_env,
- offsetof(CPUMIPSState, active_tc.CP0_UserLocal));
+tcg_gen_ld_tl(arg, cpu_env,
+  offsetof(CPUMIPSState, active_tc.CP0_UserLocal));
+tcg_gen_ext32s_tl(arg, arg);
 rn = "UserLocal";
 break;
 default:
-- 
2.7.4

Re: [Qemu-devel] [PATCH v7] qga: Add support network interface statistics in guest-network-get-interfaces command

[no-reply]

Hi,

This series failed automatic build test. Please find the testing commands and
their output below. If you have docker installed, you can probably reproduce it
locally.

Type: series
Subject: [Qemu-devel] [PATCH v7] qga: Add support network interface statistics 
in guest-network-get-interfaces command
Message-id: 1500600049-5954-1-git-send-email-lu.zhip...@zte.com.cn

=== TEST SCRIPT BEGIN ===
#!/bin/bash
set -e
git submodule update --init dtc
# Let docker tests dump environment info
export SHOW_ENV=1
export J=8
time make docker-test-quick@centos6
time make docker-test-build@min-glib
time make docker-test-mingw@fedora
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 * [new tag] 
patchew/1500600049-5954-1-git-send-email-lu.zhip...@zte.com.cn -> 
patchew/1500600049-5954-1-git-send-email-lu.zhip...@zte.com.cn
Switched to a new branch 'test'
6c36436 qga: Add support network interface statistics in 
guest-network-get-interfaces command

=== OUTPUT BEGIN ===
Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc'
Cloning into '/var/tmp/patchew-tester-tmp-ixdnmwcn/src/dtc'...
Submodule path 'dtc': checked out '558cd81bdd432769b59bff01240c44f82cfb1a9d'
  BUILD   centos6
make[1]: Entering directory '/var/tmp/patchew-tester-tmp-ixdnmwcn/src'
  ARCHIVE qemu.tgz
  ARCHIVE dtc.tgz
  COPYRUNNER
RUN test-quick in qemu:centos6 
Packages installed:
SDL-devel-1.2.14-7.el6_7.1.x86_64
bison-2.4.1-5.el6.x86_64
ccache-3.1.6-2.el6.x86_64
epel-release-6-8.noarch
flex-2.5.35-9.el6.x86_64
gcc-4.4.7-18.el6.x86_64
git-1.7.1-8.el6.x86_64
glib2-devel-2.28.8-9.el6.x86_64
libfdt-devel-1.4.0-1.el6.x86_64
make-3.81-23.el6.x86_64
package g++ is not installed
pixman-devel-0.32.8-1.el6.x86_64
tar-1.23-15.el6_8.x86_64
zlib-devel-1.2.3-29.el6.x86_64

Environment variables:
PACKAGES=libfdt-devel ccache tar git make gcc g++ flex bison zlib-devel 
glib2-devel SDL-devel pixman-devel epel-release
HOSTNAME=7d2901ae5c87
TERM=xterm
MAKEFLAGS= -j8
HISTSIZE=1000
J=8
USER=root
CCACHE_DIR=/var/tmp/ccache
EXTRA_CONFIGURE_OPTS=
V=
SHOW_ENV=1
MAIL=/var/spool/mail/root
PATH=/usr/lib/ccache:/usr/lib64/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/
LANG=en_US.UTF-8
TARGET_LIST=
HISTCONTROL=ignoredups
SHLVL=1
HOME=/root
TEST_DIR=/tmp/qemu-test
LOGNAME=root
LESSOPEN=||/usr/bin/lesspipe.sh %s
FEATURES= dtc
DEBUG=
G_BROKEN_FILENAMES=1
CCACHE_HASHDIR=
_=/usr/bin/env

Configure options:
--enable-werror --target-list=x86_64-softmmu,aarch64-softmmu 
--prefix=/var/tmp/qemu-build/install
No C++ compiler available; disabling C++ specific optional code
Install prefix/var/tmp/qemu-build/install
BIOS directory/var/tmp/qemu-build/install/share/qemu
binary directory  /var/tmp/qemu-build/install/bin
library directory /var/tmp/qemu-build/install/lib
module directory  /var/tmp/qemu-build/install/lib/qemu
libexec directory /var/tmp/qemu-build/install/libexec
include directory /var/tmp/qemu-build/install/include
config directory  /var/tmp/qemu-build/install/etc
local state directory   /var/tmp/qemu-build/install/var
Manual directory  /var/tmp/qemu-build/install/share/man
ELF interp prefix /usr/gnemul/qemu-%M
Source path   /tmp/qemu-test/src
C compilercc
Host C compiler   cc
C++ compiler  
Objective-C compiler cc
ARFLAGS   rv
CFLAGS-O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -g 
QEMU_CFLAGS   -I/usr/include/pixman-1   -I$(SRC_PATH)/dtc/libfdt -pthread 
-I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include   -fPIE -DPIE -m64 -mcx16 
-D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes 
-Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes 
-fno-strict-aliasing -fno-common -fwrapv  -Wendif-labels 
-Wno-missing-include-dirs -Wempty-body -Wnested-externs -Wformat-security 
-Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration 
-Wold-style-definition -Wtype-limits -fstack-protector-all
LDFLAGS   -Wl,--warn-common -Wl,-z,relro -Wl,-z,now -pie -m64 -g 
make  make
install   install
pythonpython -B
smbd  /usr/sbin/smbd
module supportno
host CPU  x86_64
host big endian   no
target list   x86_64-softmmu aarch64-softmmu
gprof enabled no
sparse enabledno
strip binariesyes
profiler  no
static build  no
pixmansystem
SDL support   yes (1.2.14)
GTK support   no 
GTK GL supportno
VTE support   no 
TLS priority  NORMAL
GNUTLS supportno
GNUTLS rndno
libgcrypt no
libgcrypt kdf no
nettleno 
nettle kdfno
libtasn1  no
curses supportno
virgl support no
curl support  no
mingw32 support   no
Audio drivers oss
Block whitelist (rw) 
Block whitelist (ro) 
VirtFS supportno
VNC support   yes
VNC SASL support  no
VNC JPEG support  no
VNC PNG support   no
xen support 

Re: [Qemu-devel] [PATCH] qga: Add support network interface statistics in guest-network-get-interfaces command

[no-reply]

Hi,

This series failed automatic build test. Please find the testing commands and
their output below. If you have docker installed, you can probably reproduce it
locally.

Type: series
Subject: [Qemu-devel] [PATCH] qga: Add support network interface statistics in 
guest-network-get-interfaces command
Message-id: 1500600016-5906-1-git-send-email-lu.zhip...@zte.com.cn

=== TEST SCRIPT BEGIN ===
#!/bin/bash
set -e
git submodule update --init dtc
# Let docker tests dump environment info
export SHOW_ENV=1
export J=8
time make docker-test-quick@centos6
time make docker-test-build@min-glib
time make docker-test-mingw@fedora
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 - [tag update]  
patchew/1500568290-7966-1-git-send-email-peter.mayd...@linaro.org -> 
patchew/1500568290-7966-1-git-send-email-peter.mayd...@linaro.org
 * [new tag] 
patchew/1500600016-5906-1-git-send-email-lu.zhip...@zte.com.cn -> 
patchew/1500600016-5906-1-git-send-email-lu.zhip...@zte.com.cn
Switched to a new branch 'test'
d6d30ab qga: Add support network interface statistics in 
guest-network-get-interfaces command

=== OUTPUT BEGIN ===
Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc'
Cloning into '/var/tmp/patchew-tester-tmp-oap8j5kc/src/dtc'...
Submodule path 'dtc': checked out '558cd81bdd432769b59bff01240c44f82cfb1a9d'
  BUILD   centos6
make[1]: Entering directory '/var/tmp/patchew-tester-tmp-oap8j5kc/src'
  ARCHIVE qemu.tgz
  ARCHIVE dtc.tgz
  COPYRUNNER
RUN test-quick in qemu:centos6 
Packages installed:
SDL-devel-1.2.14-7.el6_7.1.x86_64
bison-2.4.1-5.el6.x86_64
ccache-3.1.6-2.el6.x86_64
epel-release-6-8.noarch
flex-2.5.35-9.el6.x86_64
gcc-4.4.7-18.el6.x86_64
git-1.7.1-8.el6.x86_64
glib2-devel-2.28.8-9.el6.x86_64
libfdt-devel-1.4.0-1.el6.x86_64
make-3.81-23.el6.x86_64
package g++ is not installed
pixman-devel-0.32.8-1.el6.x86_64
tar-1.23-15.el6_8.x86_64
zlib-devel-1.2.3-29.el6.x86_64

Environment variables:
PACKAGES=libfdt-devel ccache tar git make gcc g++ flex bison zlib-devel 
glib2-devel SDL-devel pixman-devel epel-release
HOSTNAME=177a07d232b5
TERM=xterm
MAKEFLAGS= -j8
HISTSIZE=1000
J=8
USER=root
CCACHE_DIR=/var/tmp/ccache
EXTRA_CONFIGURE_OPTS=
V=
SHOW_ENV=1
MAIL=/var/spool/mail/root
PATH=/usr/lib/ccache:/usr/lib64/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/
LANG=en_US.UTF-8
TARGET_LIST=
HISTCONTROL=ignoredups
SHLVL=1
HOME=/root
TEST_DIR=/tmp/qemu-test
LOGNAME=root
LESSOPEN=||/usr/bin/lesspipe.sh %s
FEATURES= dtc
DEBUG=
G_BROKEN_FILENAMES=1
CCACHE_HASHDIR=
_=/usr/bin/env

Configure options:
--enable-werror --target-list=x86_64-softmmu,aarch64-softmmu 
--prefix=/var/tmp/qemu-build/install
No C++ compiler available; disabling C++ specific optional code
Install prefix/var/tmp/qemu-build/install
BIOS directory/var/tmp/qemu-build/install/share/qemu
binary directory  /var/tmp/qemu-build/install/bin
library directory /var/tmp/qemu-build/install/lib
module directory  /var/tmp/qemu-build/install/lib/qemu
libexec directory /var/tmp/qemu-build/install/libexec
include directory /var/tmp/qemu-build/install/include
config directory  /var/tmp/qemu-build/install/etc
local state directory   /var/tmp/qemu-build/install/var
Manual directory  /var/tmp/qemu-build/install/share/man
ELF interp prefix /usr/gnemul/qemu-%M
Source path   /tmp/qemu-test/src
C compilercc
Host C compiler   cc
C++ compiler  
Objective-C compiler cc
ARFLAGS   rv
CFLAGS-O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -g 
QEMU_CFLAGS   -I/usr/include/pixman-1   -I$(SRC_PATH)/dtc/libfdt -pthread 
-I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include   -fPIE -DPIE -m64 -mcx16 
-D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes 
-Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes 
-fno-strict-aliasing -fno-common -fwrapv  -Wendif-labels 
-Wno-missing-include-dirs -Wempty-body -Wnested-externs -Wformat-security 
-Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration 
-Wold-style-definition -Wtype-limits -fstack-protector-all
LDFLAGS   -Wl,--warn-common -Wl,-z,relro -Wl,-z,now -pie -m64 -g 
make  make
install   install
pythonpython -B
smbd  /usr/sbin/smbd
module supportno
host CPU  x86_64
host big endian   no
target list   x86_64-softmmu aarch64-softmmu
gprof enabled no
sparse enabledno
strip binariesyes
profiler  no
static build  no
pixmansystem
SDL support   yes (1.2.14)
GTK support   no 
GTK GL supportno
VTE support   no 
TLS priority  NORMAL
GNUTLS supportno
GNUTLS rndno
libgcrypt no
libgcrypt kdf no
nettleno 
nettle kdfno
libtasn1  no
curses supportno
virgl support no
curl support  no
mingw32 support   no
Audio drivers oss
Block white

Re: [Qemu-devel] Disable image locking for snapshot drive?

[Fam Zheng]

On Thu, 07/20 21:49, Andrew Baumann via Qemu-devel wrote:
> > From: Fam Zheng [mailto:f...@redhat.com]
> > Sent: Wednesday, 19 July 2017 23:53
> > 
> > On Tue, 07/18 16:19, Andrew Baumann wrote:
> > > > From: Eric Blake [mailto:ebl...@redhat.com]
> > > > Sent: Tuesday, 18 July 2017 8:07
> > > > On 07/17/2017 07:33 PM, John Snow wrote:
> > > > > On 07/17/2017 07:30 PM, Andrew Baumann via Qemu-devel wrote:
> > > > >> I'm running a recent Linux build of qemu on Windows Subsystem for
> > Linux
> > > > (WSL) which doesn't appear to implement file locking:
> > > > >>
> > > > >> $ qemu-system-aarch64 ... -drive file=test.vhdx,if=none,id=hd0 -
> > device
> > > > virtio-blk-pci,drive=hd0
> > > > >> qemu-system-aarch64: -drive file=test.vhdx,if=none,id=hd0: Failed to
> > > > unlock byte 100
> > > >
> > > > Does WSL implement fcntl(F_SETLK) but not fcntl(F_OFD_SETLK)?
> > >
> > > Yes, this appears to be the case (there's also one report that it's 
> > > broken):
> > > https://github.com/Microsoft/BashOnWindows/issues/1927
> > 
> > What does fcntl(F_OFD_SETLK) return? If it is -ENOTSUP, we can probably
> > detect
> > that and disable locking. Can you try the patch pasted in the end?
> 
> It returns -EINVAL. Your patch works, if I change the error code. Maybe
> testing for either ENOTSUP or EINVAL, and only allowing the fallback if the
> user didn't force locking on, would be a reasonable compromise?

I'm less comfortable with treating -EINVAL as "not supported".

A better fallback simply disabling is probably F_SETLK. Anyway I think a message
to stderr should be printed like the "#ifndef F_OFD_SETLK" case.

I'll work on a formal patch which does that.

Thanks,
Fam

> 
> Cheers,
> Andrew
> 
> > ---
> > 
> > diff --git a/block/file-posix.c b/block/file-posix.c
> > index cfbb236f6f..0be5bbbd53 100644
> > --- a/block/file-posix.c
> > +++ b/block/file-posix.c
> > @@ -493,6 +493,12 @@ static int raw_open_common(BlockDriverState *bs,
> > QDict *options,
> >  }
> >  s->fd = fd;
> > 
> > +if (s->use_lock) {
> > +int ret0 = qemu_unlock_fd(fd, 0, 0);
> > +if (ret0 == -ENOTSUP) {
> > +s->use_lock = false;
> > +}
> > +}
> >  s->lock_fd = -1;
> >  if (s->use_lock) {
> >  fd = qemu_open(filename, s->open_flags);
> 

Re: [Qemu-devel] [FIX PATCH v1] spapr: Fix QEMU abort during memory unplug

[David Gibson]

On Thu, Jul 20, 2017 at 09:41:19AM +0530, Bharata B Rao wrote:
> Commit 0cffce56 (hw/ppc/spapr.c: adding pending_dimm_unplugs to
> sPAPRMachineState) introduced a new way to track pending LMBs of DIMM
> device that is marked for removal. Since this commit we can hit the
> assert in spapr_pending_dimm_unplugs_add() in the following situation:
> 
> - DIMM device removal fails as the guest doesn't allow the removal.
> - Subsequent attempt to remove the same DIMM would hit the assert
>   as the corresponding sPAPRDIMMState is still part of the
>   pending_dimm_unplugs list.
> 
> Fix this by removing the assert and conditionally adding the
> sPAPRDIMMState to pending_dimm_unplugs list only when it is not
> already present.
> 
> Fixes: 0cffce56ae3501c5783d779f97993ce478acf856
> Signed-off-by: Bharata B Rao 
> ---
> Changes in v1:
> - Added comment (David Gibson)
> - Ensured we free sPAPRDIMMState when corresonding entry already
>   exists (Daniel Henrique Barboza)
> 
> Daniel had shown another alternative, we can switch over to that
> if preferred.
> 
>  hw/ppc/spapr.c | 13 +++--
>  1 file changed, 11 insertions(+), 2 deletions(-)
> 
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index 1cb09e7..c6091e2 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -2853,8 +2853,17 @@ static sPAPRDIMMState 
> *spapr_pending_dimm_unplugs_find(sPAPRMachineState *s,
>  static void spapr_pending_dimm_unplugs_add(sPAPRMachineState *spapr,
> sPAPRDIMMState *dimm_state)
>  {
> -g_assert(!spapr_pending_dimm_unplugs_find(spapr, dimm_state->dimm));
> -QTAILQ_INSERT_HEAD(&spapr->pending_dimm_unplugs, dimm_state, next);
> +/*
> + * If this request is for a DIMM whose removal had failed earlier
> + * (due to guest's refusal to remove the LMBs), we would have this
> + * dimm_state already in the pending_dimm_unplugs list. In that
> + * case don't add again.
> + */
> +if (!spapr_pending_dimm_unplugs_find(spapr, dimm_state->dimm)) {
> +QTAILQ_INSERT_HEAD(&spapr->pending_dimm_unplugs, dimm_state, next);
> +} else {
> +g_free(dimm_state);

This is dangerous.  You're freeing a pointer passed in by the caller
under conditions that the caller can't know, so it can't know if it
has a valid pointer afterwards or not.

It so happens that we don't use the pointer again from the caller in
spapr_memory_unplug_request() and I suspect this situation can't
occur for the call in spapr_recover_pending_dimm_state().  Still,
that's way more subtle that I'm comfortable with.

I think the way to handle this is to change unplugs_add() so that it
takes the relevant fields of the DIMMState rather than a pre-allocated
structure.  It will then return either an existing state if available
or a newly allocated and indexed one otherwise.

> +}
>  }
>  
>  static void spapr_pending_dimm_unplugs_remove(sPAPRMachineState *spapr,

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [Qemu-devel] Can I mount encrypt qcow2?

[陳培泓]

I re-build qemu from the https://github.com/qemu/qemu, and it didn't show
any errors.

I tried the cmd you suggested below:

> qemu-nbd --object secret,id=sec0,file=passwd.txt,format=raw \
>-c /dev/nbd0 \
>--image-opts driver=qcow2,file.filename=
> demo.qcow2,encrypt.format=luks,encrypt.key-secret=sec0

but it shows unrecognized option '--object '
why?I missed to install something?

2017-07-20 17:12 GMT+08:00 Daniel P. Berrange :

> On Thu, Jul 20, 2017 at 05:07:49PM +0800, 陳培泓 wrote:
> > oh~ I don't know can expose the LUKS encryption. I'm sure the older(AES)
> > can't be mounted by qemu-nbd.
>
> It can be mounted, with current git master (all the commands I show
> below are for git master btw).
>
> You should, however, *never* use the old AES format any more. It is
> broken by design and not considered secure.
>
> > If I encrypt by the command you recommended:
> >
> > > qemu-nbd --object secret,id=sec0,file=passwd.txt,format=raw \
> > >  --image-opts driver=qcow2,file.filename=
> > > demo.qcow2,encrypt.format=luks,encrypt.key-secret=sec0
>
> This *is* exposing the encrypted file -  not creating it. If you
> want to connect to a host nbd device then you use the command
> above, with the -c arg
>
> $ qemu-nbd --object secret,id=sec0,file=passwd.txt,format=raw \
>-c /dev/nbd0 \
>--image-opts driver=qcow2,file.filename=
> demo.qcow2,encrypt.format=luks,encrypt.key-secret=sec0
>
>
> If you have a legacy AES qcow2 file the syntax is very similar
>
> $ qemu-nbd --object secret,id=sec0,file=passwd.txt,format=raw \
>-c /dev/nbd0 \
>--image-opts driver=qcow2,file.filename=
> demo.qcow2,encrypt.format=aes,encrypt.key-secret=sec0
>
> Note we just changed the encrypt.format parameter there.
>
>
> To actually create an encrypted file in the first place you need the
> qemu-img command
>
> $ qemu-img create --object secret,id=sec0,file=passwd.txt,format=raw \
>-f qcow2 -o encrypt.format=luks,encrypt.key-secret=sec0 \
>demo.qcow2 1G
>
>
> Regards,
> Daniel
> --
> |: https://berrange.com  -o-https://www.flickr.com/photos/
> dberrange :|
> |: https://libvirt.org -o-
> https://fstop138.berrange.com :|
> |: https://entangle-photo.org-o-https://www.instagram.com/
> dberrange :|
>

[Qemu-devel] [PATCH v7] qga: Add support network interface statistics in guest-network-get-interfaces command

[ZhiPeng Lu]

we can get the network interface statistics inside a virtual machine by
guest-network-get-interfaces command. it is very useful for us to monitor
and analyze network traffic.

Signed-off-by: ZhiPeng Lu 

v1->v2:
 - correct some spelling mistake and add the stats data to the
   guest-network-get-interfaces command instead of adding a new command.
v2-v3:
 - optimize function implementation
v3->v4:
 - modify compile error
v4->v5:
 - rename some temporary variables and add str_trim_off function for
   calculating the space num in front of the string in guest_get_network_stats
v5->v6:
 - use g_strchug instead of str_trim_off implemented by myself
v6->v7:
 - add implementation for windows
---
 qga/commands-posix.c | 72 +++-
 qga/commands-win32.c | 47 ++
 qga/qapi-schema.json | 38 ++-
 3 files changed, 155 insertions(+), 2 deletions(-)

diff --git a/qga/commands-posix.c b/qga/commands-posix.c
index d8e4122..b65dd8e 100644
--- a/qga/commands-posix.c
+++ b/qga/commands-posix.c
@@ -1639,6 +1639,65 @@ guest_find_interface(GuestNetworkInterfaceList *head,
 return head;
 }
 
+static int guest_get_network_stats(const char *name,
+   GuestNetworkInterfaceStat *stats)
+{
+int name_len;
+char const *devinfo = "/proc/net/dev";
+FILE *fp;
+char *line = NULL, *colon;
+size_t n;
+fp = fopen(devinfo, "r");
+if (!fp) {
+return -1;
+}
+name_len = strlen(name);
+while (getline(&line, &n, fp) != -1) {
+long long dummy;
+long long rx_bytes;
+long long rx_packets;
+long long rx_errs;
+long long rx_dropped;
+long long tx_bytes;
+long long tx_packets;
+long long tx_errs;
+long long tx_dropped;
+char *trim_line;
+trim_line = g_strchug(line);
+if (trim_line[0] == '\0') {
+continue;
+}
+colon = strchr(trim_line, ':');
+if (!colon) {
+continue;
+}
+if (colon - name_len  == trim_line &&
+   strncmp(trim_line, name, name_len) == 0) {
+if (sscanf(colon + 1,
+"%lld %lld %lld %lld %lld %lld %lld %lld %lld %lld %lld %lld 
%lld %lld %lld %lld",
+  &rx_bytes, &rx_packets, &rx_errs, &rx_dropped,
+  &dummy, &dummy, &dummy, &dummy,
+  &tx_bytes, &tx_packets, &tx_errs, &tx_dropped,
+  &dummy, &dummy, &dummy, &dummy) != 16) {
+continue;
+}
+stats->rx_bytes = rx_bytes;
+stats->rx_packets = rx_packets;
+stats->rx_errs = rx_errs;
+stats->rx_dropped = rx_dropped;
+stats->tx_bytes = tx_bytes;
+stats->tx_packets = tx_packets;
+stats->tx_errs = tx_errs;
+stats->tx_dropped = tx_dropped;
+fclose(fp);
+return 0;
+}
+}
+fclose(fp);
+g_debug("/proc/net/dev: Interface not found");
+return -1;
+}
+
 /*
  * Build information about guest interfaces
  */
@@ -1655,6 +1714,7 @@ GuestNetworkInterfaceList 
*qmp_guest_network_get_interfaces(Error **errp)
 for (ifa = ifap; ifa; ifa = ifa->ifa_next) {
 GuestNetworkInterfaceList *info;
 GuestIpAddressList **address_list = NULL, *address_item = NULL;
+GuestNetworkInterfaceStat  *interface_stat = NULL;
 char addr4[INET_ADDRSTRLEN];
 char addr6[INET6_ADDRSTRLEN];
 int sock;
@@ -1774,7 +1834,17 @@ GuestNetworkInterfaceList 
*qmp_guest_network_get_interfaces(Error **errp)
 
 info->value->has_ip_addresses = true;
 
-
+if (!info->value->has_statistics) {
+interface_stat = g_malloc0(sizeof(*interface_stat));
+if (guest_get_network_stats(info->value->name,
+interface_stat) == -1) {
+info->value->has_statistics = false;
+g_free(interface_stat);
+} else {
+info->value->statistics = interface_stat;
+info->value->has_statistics = true;
+}
+}
 }
 
 freeifaddrs(ifap);
diff --git a/qga/commands-win32.c b/qga/commands-win32.c
index 6f16457..433453d 100644
--- a/qga/commands-win32.c
+++ b/qga/commands-win32.c
@@ -1152,6 +1152,42 @@ out:
 }
 #endif
 
+static DWORD get_interface_index(const char *guid)
+{
+ULONG index;
+DWORD status;
+wchar_t wbuf[512];
+snwprintf(wbuf, sizeof(wbuf), L"\\device\\tcpip_%s", guid);
+wbuf[sizeof(wbuf) - 1] = 0;
+status = GetAdapterIndex (wbuf, &index);
+if (status != NO_ERROR) {
+return (DWORD)~0;
+} else {
+return index;
+}
+}
+static int guest_get_network_stats(const char *name,
+   GuestNetworkInterfaceStat *stats)
+{
+DWORD IfIndex = 0;
+MIB_IFROW aMib_ifrow;
+memset(&aMib_ifrow, 0, sizeof(aMib_ifrow));
+IfIndex = get_int

[Qemu-devel] [PATCH] qga: Add support network interface statistics in guest-network-get-interfaces command

[ZhiPeng Lu]

we can get the network interface statistics inside a virtual machine by
guest-network-get-interfaces command. it is very useful for us to monitor
and analyze network traffic.

Signed-off-by: ZhiPeng Lu 

v1->v2:
 - correct some spelling mistake and add the stats data to the
   guest-network-get-interfaces command instead of adding a new command.
v2-v3:
 - optimize function implementation
v3->v4:
 - modify compile error
v4->v5:
 - rename some temporary variables and add str_trim_off function for
   calculating the space num in front of the string in guest_get_network_stats
v5->v6:
 - use g_strchug instead of str_trim_off implemented by myself
v6->v7:
 - add implementation for windows
---
 qga/commands-posix.c | 72 +++-
 qga/commands-win32.c | 47 ++
 qga/qapi-schema.json | 38 ++-
 3 files changed, 155 insertions(+), 2 deletions(-)

diff --git a/qga/commands-posix.c b/qga/commands-posix.c
index d8e4122..b65dd8e 100644
--- a/qga/commands-posix.c
+++ b/qga/commands-posix.c
@@ -1639,6 +1639,65 @@ guest_find_interface(GuestNetworkInterfaceList *head,
 return head;
 }
 
+static int guest_get_network_stats(const char *name,
+   GuestNetworkInterfaceStat *stats)
+{
+int name_len;
+char const *devinfo = "/proc/net/dev";
+FILE *fp;
+char *line = NULL, *colon;
+size_t n;
+fp = fopen(devinfo, "r");
+if (!fp) {
+return -1;
+}
+name_len = strlen(name);
+while (getline(&line, &n, fp) != -1) {
+long long dummy;
+long long rx_bytes;
+long long rx_packets;
+long long rx_errs;
+long long rx_dropped;
+long long tx_bytes;
+long long tx_packets;
+long long tx_errs;
+long long tx_dropped;
+char *trim_line;
+trim_line = g_strchug(line);
+if (trim_line[0] == '\0') {
+continue;
+}
+colon = strchr(trim_line, ':');
+if (!colon) {
+continue;
+}
+if (colon - name_len  == trim_line &&
+   strncmp(trim_line, name, name_len) == 0) {
+if (sscanf(colon + 1,
+"%lld %lld %lld %lld %lld %lld %lld %lld %lld %lld %lld %lld 
%lld %lld %lld %lld",
+  &rx_bytes, &rx_packets, &rx_errs, &rx_dropped,
+  &dummy, &dummy, &dummy, &dummy,
+  &tx_bytes, &tx_packets, &tx_errs, &tx_dropped,
+  &dummy, &dummy, &dummy, &dummy) != 16) {
+continue;
+}
+stats->rx_bytes = rx_bytes;
+stats->rx_packets = rx_packets;
+stats->rx_errs = rx_errs;
+stats->rx_dropped = rx_dropped;
+stats->tx_bytes = tx_bytes;
+stats->tx_packets = tx_packets;
+stats->tx_errs = tx_errs;
+stats->tx_dropped = tx_dropped;
+fclose(fp);
+return 0;
+}
+}
+fclose(fp);
+g_debug("/proc/net/dev: Interface not found");
+return -1;
+}
+
 /*
  * Build information about guest interfaces
  */
@@ -1655,6 +1714,7 @@ GuestNetworkInterfaceList 
*qmp_guest_network_get_interfaces(Error **errp)
 for (ifa = ifap; ifa; ifa = ifa->ifa_next) {
 GuestNetworkInterfaceList *info;
 GuestIpAddressList **address_list = NULL, *address_item = NULL;
+GuestNetworkInterfaceStat  *interface_stat = NULL;
 char addr4[INET_ADDRSTRLEN];
 char addr6[INET6_ADDRSTRLEN];
 int sock;
@@ -1774,7 +1834,17 @@ GuestNetworkInterfaceList 
*qmp_guest_network_get_interfaces(Error **errp)
 
 info->value->has_ip_addresses = true;
 
-
+if (!info->value->has_statistics) {
+interface_stat = g_malloc0(sizeof(*interface_stat));
+if (guest_get_network_stats(info->value->name,
+interface_stat) == -1) {
+info->value->has_statistics = false;
+g_free(interface_stat);
+} else {
+info->value->statistics = interface_stat;
+info->value->has_statistics = true;
+}
+}
 }
 
 freeifaddrs(ifap);
diff --git a/qga/commands-win32.c b/qga/commands-win32.c
index 6f16457..433453d 100644
--- a/qga/commands-win32.c
+++ b/qga/commands-win32.c
@@ -1152,6 +1152,42 @@ out:
 }
 #endif
 
+static DWORD get_interface_index(const char *guid)
+{
+ULONG index;
+DWORD status;
+wchar_t wbuf[512];
+snwprintf(wbuf, sizeof(wbuf), L"\\device\\tcpip_%s", guid);
+wbuf[sizeof(wbuf) - 1] = 0;
+status = GetAdapterIndex (wbuf, &index);
+if (status != NO_ERROR) {
+return (DWORD)~0;
+} else {
+return index;
+}
+}
+static int guest_get_network_stats(const char *name,
+   GuestNetworkInterfaceStat *stats)
+{
+DWORD IfIndex = 0;
+MIB_IFROW aMib_ifrow;
+memset(&aMib_ifrow, 0, sizeof(aMib_ifrow));
+IfIndex = get_int

[Qemu-devel] 答复: Re: [PATCH] vhost: fix a migration failed because ofvhost region merge

[peng.hao2]

* Michael S. Tsirkin (m...@redhat.com) wrote:





>> On Wed, Jul 19, 2017 at 03:24:27PM +0200, Igor Mammedov wrote:
> > > On Wed, 19 Jul 2017 12:46:13 +0100
> > > "Dr. David Alan Gilbert"  wrote:
> > > 
> > > > * Igor Mammedov (imamm...@redhat.com) wrote:
> > > > > On Wed, 19 Jul 2017 23:17:32 +0800
> > > > > Peng Hao  wrote:
> > > > >   
> > > > > > When a guest that has several hotplugged dimms is migrated, in
> > > > > > destination host it will fail to resume. Because vhost regions of
> > > > > > several dimms in source host are merged and in the restore stage
> > > > > > in destination host it computes whether more than vhost slot limit
> > > > > > before merging vhost regions of several dimms.  
> > > > > could you provide a bit more detailed description of the problem
> > > > > including command line+used device_add commands on source and
> > > > > command line on destination?  
> > > > 
> > > > (ccing in Marc Andre and Maxime)
> > > > 
> > > > Hmm, I'd like to understade the situation where you get merging between
> > > > RAMBlocks that complicates some stuff for postcopy.
> > > and probably inconsistent merging breaks vhost as well
> > > 
> > > merging might happen if regions are adjacent or overlap
> > > but for that to happen merged regions must have equal
> > > distance between their GPA:HVA pairs, so that following
> > > translation would work:
> > > 
> > > if gva in regionX[gva_start, len, hva_start]
> > >hva = hva_start + gva - gva_start
> > > 
> > > while GVA of regions is under QEMU control and deterministic
> > > HVA is not, so in migration case merging might happen on source
> > > side but not on destination, resulting in different memory maps.
> > > 
> > > Maybe Michael might know details why migration works in vhost usecase,
> > > but I don't see vhost sending any vmstate data.
> > 
>> We aren't merging ramblocks at all.
>> When we are passing blocks A and B to vhost, if we see that
>> 
>> hvaB=hvaA + lenA
>> gpaB=gpaA + lenA
>> 
>> then we can improve performance a bit by passing a single
>> chunk to vhost: hvaA,gpaA,lena+lenB

>OK, but that means that a region can incorporate multiple
>RAMBlocks though? Hmm that's not fun on postcopy.

>> so it does not affect migration normally.

>Well, why? What's required - if the region sizes/lengths/orders
>are different on the source and destination does it matter - if
>it does then that means we have a problem, since that heuristic
>is non-deterministic.
Yes,I encounter the different  region creation orders  on the source and 
destination

if you add dimms on the source before migration. so I just distinguish 
according to 

vm state.

>Dave


> 
> > 
> > > 
> > > > > 
> > > > > Signed-off-by: Peng Hao 
> > > > > Signed-off-by: Wang Yechao 
> > > > > ---
> > > > >  hw/mem/pc-dimm.c | 2 +-
> > > > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > > > > 
> > > > > diff --git a/hw/mem/pc-dimm.c b/hw/mem/pc-dimm.c
> > > > > index ea67b46..bb0fa08 100644
> > > > > --- a/hw/mem/pc-dimm.c
> > > > > +++ b/hw/mem/pc-dimm.c
> > > > > @@ -101,7 +101,7 @@ void pc_dimm_memory_plug(DeviceState *dev, 
> > > > > MemoryHotplugState *hpms,
> > > > >  goto out
> > > > >  }
> > > > >  
> > > > > -if (!vhost_has_free_slot()) {
> > > > > +if (!vhost_has_free_slot() && runstate_is_running()) {
> > > > >  error_setg(&local_err, "a used vhost backend has no free"
> > > > > " memory slots left")
> > > > >  goto out  
> > > 
> > > Even this produces the wrong error message in this case,
> > > it also makes me think if the existing code should undo a lot of
> > > the object_property_set's that happen.
> > > 
> > > Dave
> > > > 
> > > >   
> > > --
> > > Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK
--
Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

Re: [Qemu-devel] [PATCH for-2.10] Use qemu_tolower() and qemu_toupper(), not tolower() and toupper()

[David Gibson]

On Thu, Jul 20, 2017 at 05:31:30PM +0100, Peter Maydell wrote:
> On NetBSD, where tolower() and toupper() are implemented using an
> array lookup, the compiler warns if you pass a plain 'char'
> to these functions:
> 
> gdbstub.c:914:13: warning: array subscript has type 'char'
> 
> This reflects the fact that toupper() and tolower() give
> undefined behaviour if they are passed a value that isn't
> a valid 'unsigned char' or EOF.
> 
> We have qemu_tolower() and qemu_toupper() to avoid this problem;
> use them.
> 
> (The use in scsi-generic.c does not trigger the warning because
> it passes a uint8_t; we switch it anyway, for consistency.)
> 
> Signed-off-by: Peter Maydell 

ppc parts

Acked-by: David Gibson 

> ---
>  gdbstub.c  | 2 +-
>  hw/s390x/s390-virtio-ccw.c | 2 +-
>  hw/scsi/scsi-generic.c | 2 +-
>  target/ppc/monitor.c   | 4 ++--
>  4 files changed, 5 insertions(+), 5 deletions(-)
> 
> diff --git a/gdbstub.c b/gdbstub.c
> index f936ddd..2a94030 100644
> --- a/gdbstub.c
> +++ b/gdbstub.c
> @@ -911,7 +911,7 @@ static int gdb_handle_vcont(GDBState *s, const char *p)
>  
>  cur_action = *p++;
>  if (cur_action == 'C' || cur_action == 'S') {
> -cur_action = tolower(cur_action);
> +cur_action = qemu_tolower(cur_action);
>  res = qemu_strtoul(p + 1, &p, 16, &tmp);
>  if (res) {
>  goto out;
> diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
> index ce3921e..1c7af39 100644
> --- a/hw/s390x/s390-virtio-ccw.c
> +++ b/hw/s390x/s390-virtio-ccw.c
> @@ -318,7 +318,7 @@ static void machine_set_loadparm(Object *obj, const char 
> *val, Error **errp)
>  int i;
>  
>  for (i = 0; i < sizeof(ms->loadparm) && val[i]; i++) {
> -uint8_t c = toupper(val[i]); /* mimic HMC */
> +uint8_t c = qemu_toupper(val[i]); /* mimic HMC */
>  
>  if (('A' <= c && c <= 'Z') || ('0' <= c && c <= '9') || (c == '.') ||
>  (c == ' ')) {
> diff --git a/hw/scsi/scsi-generic.c b/hw/scsi/scsi-generic.c
> index a55ff87..7e1cbab 100644
> --- a/hw/scsi/scsi-generic.c
> +++ b/hw/scsi/scsi-generic.c
> @@ -406,7 +406,7 @@ static int read_naa_id(const uint8_t *p, uint64_t *p_wwn)
>  }
>  *p_wwn = 0;
>  for (i = 8; i < 24; i++) {
> -char c = toupper(p[i]);
> +char c = qemu_toupper(p[i]);
>  c -= (c >= '0' && c <= '9' ? '0' : 'A' - 10);
>  *p_wwn = (*p_wwn << 4) | c;
>  }
> diff --git a/target/ppc/monitor.c b/target/ppc/monitor.c
> index b8f30e9..1491511 100644
> --- a/target/ppc/monitor.c
> +++ b/target/ppc/monitor.c
> @@ -115,14 +115,14 @@ int target_get_monitor_def(CPUState *cs, const char 
> *name, uint64_t *pval)
>  CPUPPCState *env = &cpu->env;
>  
>  /* General purpose registers */
> -if ((tolower(name[0]) == 'r') &&
> +if ((qemu_tolower(name[0]) == 'r') &&
>  ppc_cpu_get_reg_num(name + 1, ARRAY_SIZE(env->gpr), ®num)) {
>  *pval = env->gpr[regnum];
>  return 0;
>  }
>  
>  /* Floating point registers */
> -if ((tolower(name[0]) == 'f') &&
> +if ((qemu_tolower(name[0]) == 'f') &&
>  ppc_cpu_get_reg_num(name + 1, ARRAY_SIZE(env->fpr), ®num)) {
>  *pval = env->fpr[regnum];
>  return 0;

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [Qemu-devel] [PATCH v3 42/43] tcg: introduce regions to split code_gen_buffer

[Richard Henderson]

On 07/20/2017 01:23 PM, Emilio G. Cota wrote:

That's a nice helper -- will do it this way.

For v4, should I send all patches again, or just the handful of patches
that are changing from v3?


I don't mind if you just send the changed patches.
Just be sure to reference the tree in the cover letter.


r~

Re: [Qemu-devel] [PATCH v3 35/43] tcg: dynamically allocate optimizer temps

[Richard Henderson]

On 07/20/2017 01:53 PM, Emilio G. Cota wrote:

BTW, is there any chance that the pool will be initialized before we copy
tcg_init_ctx? That'd mean the main thread has performed translation, which
seems unlikely to me. But should then we bother clearing the TCGProfile
counters after we copy tcg_init_ctx? I don't see how without translation
counters would be !0.


I wouldn't think so.  This cpu setup should be happening very early.

We could perhaps look at arranging fields such that all the fields that are 
"shared" between the contexts are up front, and use the qemu standard


  memcpy(new, old, offsetof(TCGContext, end_common_fields));

trick, and zero the rest.


r~

Re: [Qemu-devel] Commit 77af8a2b95b79699de650965d5228772743efe84 breaks Windows 2000 support

[Programmingkid]

> On Jul 20, 2017, at 3:29 PM, Phil Dennis-Jordan  wrote:
> 
> On Thu, Jul 20, 2017 at 6:40 PM, Programmingkid
>  wrote:
>> I noticed that Windows 2000 does not boot up in QEMU recently. After 
>> bisecting the issue I found the offending commit:
> 
> Ouch. I reckon we have 2 options for fixing this:
> 
> 1. Export two FADTs, one ACPI 1.0, one ACPI 2.0. The latter would need
> to be pointed to by an XSDT, which Qemu currently doesn't implement at
> all as far as I'm aware. Any ideas on how SeaBIOS or OVMF would handle
> this? Any likely other OS regressions?
> 
> 2. Select FADT version with an option. This one is definitely safe,
> but adds yet another option.
> 
> Thoughts?

Option 1 sounds good to me. 

Re: [Qemu-devel] [PATCH v3 35/43] tcg: dynamically allocate optimizer temps

[Emilio G. Cota]

On Wed, Jul 19, 2017 at 21:39:35 -1000, Richard Henderson wrote:
> On 07/19/2017 05:09 PM, Emilio G. Cota wrote:
> >Groundwork for supporting multiple TCG contexts.
> >That is, 2.70% slowdown.
> 
> That's disappointing.  How about using tcg_malloc?
> 
> Maximum allocation is sizeof(tcg_temp_info) * TCG_MAX_TEMPS = 12288, which
> is less than TCG_POOL_CHUNK_SIZE, so we'll retain the allocation in the pool
> across translations.

 exec time (s)  Relative slowdown wrt original (%)
---
 original 20.213321616  0.
 tcg_malloc   20.441130078   1.1270214
 TCGContext   20.477846517   1.3086662
 g_malloc 20.780527895   2.8061013

So will go with tcg_malloc.

BTW, is there any chance that the pool will be initialized before we copy
tcg_init_ctx? That'd mean the main thread has performed translation, which
seems unlikely to me. But should then we bother clearing the TCGProfile
counters after we copy tcg_init_ctx? I don't see how without translation
counters would be !0.

E.

Re: [Qemu-devel] [PATCH v3 42/43] tcg: introduce regions to split code_gen_buffer

[Emilio G. Cota]

On Thu, Jul 20, 2017 at 11:22:10 -1000, Richard Henderson wrote:
> >Perhaps we should then enlarge both the first and last regions so that we
> >fully use the buffer.
> 
> I really like the idea.  That's a lot of space recovered for 64k page hosts.
> 
> I do think we can make the computation clearer.  How about

(snip)
> 
> static inline void tcg_region_bounds(TCGContext *s, size_t curr_region,
>  void **pstart, void **pend)
> {
>   void *start, *end;
> 
>   /* ??? Maybe store "aligned" precomputed.  */
>   start = QEMU_ALIGN_PTR_UP(region.start, qemu_real_host_page_size);
>   /* ??? Maybe store "stride" precomputed.  */
>   start += curr_region * (region.size + qemu_real_host_page_size);
>   end = start + region.size;
> 
>   if (curr_region == 0) {
> start = region.start;
>   }
>   if (curr_region == region.n - 1) {
> end = region.end;
>   }
> 
>   *pstart = start;
>   *pend = end;
> }

That's a nice helper -- will do it this way.

For v4, should I send all patches again, or just the handful of patches
that are changing from v3?

E.

Re: [Qemu-devel] [Bug 1196727] Re: SLIRP on Windows 7 64-bit host or is it me?

[Kenneth Salerno via Qemu-devel]

Hi, you can close this ticket. I can't remember what I did to get it
working.

Sent from Yahoo Mail on Android 
 
  On Thu, Jul 20, 2017 at 8:16 AM, Thomas Huth<1196...@bugs.launchpad.net> 
wrote:   Triaging old bug tickets ... can you still reproduce this problem with
the latest version of QEMU (currently v2.9.0)?

** Changed in: qemu
      Status: New => Incomplete

-- 
You received this bug notification because you are subscribed to the bug
report.
https://bugs.launchpad.net/bugs/1196727

Title:
  SLIRP on Windows 7 64-bit host or is it me?

Status in QEMU:
  Incomplete

Bug description:
  Version: 1.5.1 and tried latest in Git, compiled for x86_64 Windows 64-bit
        Host: Windows 7 64-bit
      Guest: FreeBSD 9.1 i386, RHEL 6.4 x86_64, SLES 11.2 x86_64, OpenSUSE 12.3 
ppc64, Fedora 18 ppc64
  libiconv: 1.14
          glib: 2.28.8
  gettext: 0.18.1.1
  pixman: 0.30.0
    libSDL: 1.2.14
    Driver: virtio-net-pci
      Emu: full (non-KVM)

  I'm new to Windows 7 64-bit as a host for QEMU (previously I was
  running QEMU on Windows XP with no issues) so it could be me, now on
  Windows 7 SLIRP only works for me connecting internally from the host
  to the guest via SLIRP redirect, but any outbound requests from the
  guest to the Internet are failing with the following:

  if_start...
  m_get...
  m = 61f7bd40
  ip_input...
  m = 61f7bd40
  m_len = 48
  tcp_input...
  m = 61f7bd40  iphlen = 20  inso = 0
  tcp_fconnect...
  so = 33e140
  connect()ing, addr.sin_port=80, addr.sin_addr.s_addr=206.190.36.45
  tcp fconnect errno = 10035-Unknown error
  icmp_error...
  msrc = 61f7bd40
  msrc_len = 48
  10.0.2.5 to 206.190.36.45
  m_get...
  m = 61f7b6c0
  ip_output...
  so = 0
  m0 = 61f7b6c0
  if_output...
  so = 0
  ifm = 61f7b6c0
  if_start...
  arp_table_search...
  ip = 0x502000a
  found hw addr = 52:54:00:12:34:56
  m_free...
  m = 61f7b6c0
  tcp_close...
  tp = 377840
  m_free...
  m = 0
  m_free...
  m = 61f7bd40

  Am I doing something wrong with my Windows host configuration or is
  this a bug in SLIRP only on W64 and not W32?

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1196727/+subscriptions

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1196727

Title:
  SLIRP on Windows 7 64-bit host or is it me?

Status in QEMU:
  Incomplete

Bug description:
   Version: 1.5.1 and tried latest in Git, compiled for x86_64 Windows 64-bit
Host: Windows 7 64-bit
  Guest: FreeBSD 9.1 i386, RHEL 6.4 x86_64, SLES 11.2 x86_64, OpenSUSE 12.3 
ppc64, Fedora 18 ppc64
   libiconv: 1.14
  glib: 2.28.8
  gettext: 0.18.1.1
   pixman: 0.30.0
 libSDL: 1.2.14
 Driver: virtio-net-pci
   Emu: full (non-KVM)

  I'm new to Windows 7 64-bit as a host for QEMU (previously I was
  running QEMU on Windows XP with no issues) so it could be me, now on
  Windows 7 SLIRP only works for me connecting internally from the host
  to the guest via SLIRP redirect, but any outbound requests from the
  guest to the Internet are failing with the following:

  if_start...
  m_get...
   m = 61f7bd40
  ip_input...
   m = 61f7bd40
   m_len = 48
  tcp_input...
   m = 61f7bd40  iphlen = 20  inso = 0
  tcp_fconnect...
   so = 33e140
   connect()ing, addr.sin_port=80, addr.sin_addr.s_addr=206.190.36.45
   tcp fconnect errno = 10035-Unknown error
  icmp_error...
   msrc = 61f7bd40
   msrc_len = 48
   10.0.2.5 to 206.190.36.45
  m_get...
   m = 61f7b6c0
  ip_output...
   so = 0
   m0 = 61f7b6c0
  if_output...
   so = 0
   ifm = 61f7b6c0
  if_start...
  arp_table_search...
   ip = 0x502000a
   found hw addr = 52:54:00:12:34:56
  m_free...
   m = 61f7b6c0
  tcp_close...
   tp = 377840
  m_free...
   m = 0
  m_free...
   m = 61f7bd40

  Am I doing something wrong with my Windows host configuration or is
  this a bug in SLIRP only on W64 and not W32?

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1196727/+subscriptions

Re: [Qemu-devel] [PULL 0/8] target/alpha cleanups

[Paolo Bonzini]

I am planning to refresh the series for 2.11.

Paolo

Il 20/lug/2017 11:07, "Peter Maydell"  ha scritto:

> On 19 July 2017 at 22:52, Richard Henderson  wrote:
> > On 07/19/2017 06:57 AM, Peter Maydell wrote:
> >>
> >> On 19 July 2017 at 05:45, Richard Henderson  wrote:
> >>>
> >>> The new title holder for perf top is helper_lookup_tb_ptr.
> >>> Those targets that have a complicated cpu_get_tb_cpu_state
> >>> function are going to regret that.
> >>
> >>
> >> Yeah, Paolo's pointed out (and had some patches for)
> >> ARM's rather complicated cpu_get_tb_cpu_state(). My
> >> issue with his suggested fixes was that they were
> >> pretty fragile in terms of not having any guarantee
> >> that the change always produced the right tb cpu
> >> state flags answer...
> >
> >
> > Oh?  I must have missed seeing this one.
> > A quick patchwork search doesn't pull it up;
> > do either of you have a link?
>
> It was back in September:
> https://lists.gnu.org/archive/html/qemu-devel/2016-09/msg03133.html
>
> thanks
> -- PMM
>
>

Re: [Qemu-devel] [PATCH for-2.10? v2 0/4] random qapi cleanups

[no-reply]

Hi,

This series failed automatic build test. Please find the testing commands and
their output below. If you have docker installed, you can probably reproduce it
locally.

Type: series
Subject: [Qemu-devel] [PATCH for-2.10? v2 0/4] random qapi cleanups
Message-id: 20170720214008.28494-1-ebl...@redhat.com

=== TEST SCRIPT BEGIN ===
#!/bin/bash
set -e
git submodule update --init dtc
# Let docker tests dump environment info
export SHOW_ENV=1
export J=8
time make docker-test-quick@centos6
time make docker-test-build@min-glib
time make docker-test-mingw@fedora
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 * [new tag] patchew/20170720214008.28494-1-ebl...@redhat.com -> 
patchew/20170720214008.28494-1-ebl...@redhat.com
Switched to a new branch 'test'
8efff66 qtest: Document calling conventions
b4f8e98 qtest: Avoid passing raw strings through hmp()
0354364 qapi: Visitor documentation tweak
940d218 tests: Enhance qobject output to cover partial visit

=== OUTPUT BEGIN ===
Submodule 'dtc' (git://git.qemu-project.org/dtc.git) registered for path 'dtc'
Cloning into '/var/tmp/patchew-tester-tmp-6z5wd27i/src/dtc'...
Submodule path 'dtc': checked out '558cd81bdd432769b59bff01240c44f82cfb1a9d'
  BUILD   centos6
make[1]: Entering directory '/var/tmp/patchew-tester-tmp-6z5wd27i/src'
  ARCHIVE qemu.tgz
  ARCHIVE dtc.tgz
  COPYRUNNER
RUN test-quick in qemu:centos6 
Packages installed:
SDL-devel-1.2.14-7.el6_7.1.x86_64
bison-2.4.1-5.el6.x86_64
ccache-3.1.6-2.el6.x86_64
epel-release-6-8.noarch
flex-2.5.35-9.el6.x86_64
gcc-4.4.7-18.el6.x86_64
git-1.7.1-8.el6.x86_64
glib2-devel-2.28.8-9.el6.x86_64
libfdt-devel-1.4.0-1.el6.x86_64
make-3.81-23.el6.x86_64
package g++ is not installed
pixman-devel-0.32.8-1.el6.x86_64
tar-1.23-15.el6_8.x86_64
zlib-devel-1.2.3-29.el6.x86_64

Environment variables:
PACKAGES=libfdt-devel ccache tar git make gcc g++ flex bison zlib-devel 
glib2-devel SDL-devel pixman-devel epel-release
HOSTNAME=330dfab631af
TERM=xterm
MAKEFLAGS= -j8
HISTSIZE=1000
J=8
USER=root
CCACHE_DIR=/var/tmp/ccache
EXTRA_CONFIGURE_OPTS=
V=
SHOW_ENV=1
MAIL=/var/spool/mail/root
PATH=/usr/lib/ccache:/usr/lib64/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/
LANG=en_US.UTF-8
TARGET_LIST=
HISTCONTROL=ignoredups
SHLVL=1
HOME=/root
TEST_DIR=/tmp/qemu-test
LOGNAME=root
LESSOPEN=||/usr/bin/lesspipe.sh %s
FEATURES= dtc
DEBUG=
G_BROKEN_FILENAMES=1
CCACHE_HASHDIR=
_=/usr/bin/env

Configure options:
--enable-werror --target-list=x86_64-softmmu,aarch64-softmmu 
--prefix=/var/tmp/qemu-build/install
No C++ compiler available; disabling C++ specific optional code
Install prefix/var/tmp/qemu-build/install
BIOS directory/var/tmp/qemu-build/install/share/qemu
binary directory  /var/tmp/qemu-build/install/bin
library directory /var/tmp/qemu-build/install/lib
module directory  /var/tmp/qemu-build/install/lib/qemu
libexec directory /var/tmp/qemu-build/install/libexec
include directory /var/tmp/qemu-build/install/include
config directory  /var/tmp/qemu-build/install/etc
local state directory   /var/tmp/qemu-build/install/var
Manual directory  /var/tmp/qemu-build/install/share/man
ELF interp prefix /usr/gnemul/qemu-%M
Source path   /tmp/qemu-test/src
C compilercc
Host C compiler   cc
C++ compiler  
Objective-C compiler cc
ARFLAGS   rv
CFLAGS-O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -g 
QEMU_CFLAGS   -I/usr/include/pixman-1   -I$(SRC_PATH)/dtc/libfdt -pthread 
-I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include   -fPIE -DPIE -m64 -mcx16 
-D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes 
-Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes 
-fno-strict-aliasing -fno-common -fwrapv  -Wendif-labels 
-Wno-missing-include-dirs -Wempty-body -Wnested-externs -Wformat-security 
-Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration 
-Wold-style-definition -Wtype-limits -fstack-protector-all
LDFLAGS   -Wl,--warn-common -Wl,-z,relro -Wl,-z,now -pie -m64 -g 
make  make
install   install
pythonpython -B
smbd  /usr/sbin/smbd
module supportno
host CPU  x86_64
host big endian   no
target list   x86_64-softmmu aarch64-softmmu
gprof enabled no
sparse enabledno
strip binariesyes
profiler  no
static build  no
pixmansystem
SDL support   yes (1.2.14)
GTK support   no 
GTK GL supportno
VTE support   no 
TLS priority  NORMAL
GNUTLS supportno
GNUTLS rndno
libgcrypt no
libgcrypt kdf no
nettleno 
nettle kdfno
libtasn1  no
curses supportno
virgl support no
curl support  no
mingw32 support   no
Audio drivers oss
Block whitelist (rw) 
Block whitelist (ro) 
VirtFS supportno
VNC support   yes
VNC SASL support  no
VNC JPEG support  no
VNC PNG support   no
xen 

Re: [Qemu-devel] Disable image locking for snapshot drive?

[Andrew Baumann via Qemu-devel]

> From: Fam Zheng [mailto:f...@redhat.com]
> Sent: Wednesday, 19 July 2017 23:53
> 
> On Tue, 07/18 16:19, Andrew Baumann wrote:
> > > From: Eric Blake [mailto:ebl...@redhat.com]
> > > Sent: Tuesday, 18 July 2017 8:07
> > > On 07/17/2017 07:33 PM, John Snow wrote:
> > > > On 07/17/2017 07:30 PM, Andrew Baumann via Qemu-devel wrote:
> > > >> I'm running a recent Linux build of qemu on Windows Subsystem for
> Linux
> > > (WSL) which doesn't appear to implement file locking:
> > > >>
> > > >> $ qemu-system-aarch64 ... -drive file=test.vhdx,if=none,id=hd0 -
> device
> > > virtio-blk-pci,drive=hd0
> > > >> qemu-system-aarch64: -drive file=test.vhdx,if=none,id=hd0: Failed to
> > > unlock byte 100
> > >
> > > Does WSL implement fcntl(F_SETLK) but not fcntl(F_OFD_SETLK)?
> >
> > Yes, this appears to be the case (there's also one report that it's broken):
> > https://github.com/Microsoft/BashOnWindows/issues/1927
> 
> What does fcntl(F_OFD_SETLK) return? If it is -ENOTSUP, we can probably
> detect
> that and disable locking. Can you try the patch pasted in the end?

It returns -EINVAL. Your patch works, if I change the error code. Maybe testing 
for either ENOTSUP or EINVAL, and only allowing the fallback if the user didn't 
force locking on, would be a reasonable compromise?

Cheers,
Andrew

> ---
> 
> diff --git a/block/file-posix.c b/block/file-posix.c
> index cfbb236f6f..0be5bbbd53 100644
> --- a/block/file-posix.c
> +++ b/block/file-posix.c
> @@ -493,6 +493,12 @@ static int raw_open_common(BlockDriverState *bs,
> QDict *options,
>  }
>  s->fd = fd;
> 
> +if (s->use_lock) {
> +int ret0 = qemu_unlock_fd(fd, 0, 0);
> +if (ret0 == -ENOTSUP) {
> +s->use_lock = false;
> +}
> +}
>  s->lock_fd = -1;
>  if (s->use_lock) {
>  fd = qemu_open(filename, s->open_flags);

Re: [Qemu-devel] [PATCH for-2.10] Use qemu_tolower() and qemu_toupper(), not tolower() and toupper()

[Peter Maydell]

On 20 July 2017 at 22:29, Eric Blake  wrote:
> On 07/20/2017 04:03 PM, Peter Maydell wrote:
>> https://ftp.netbsd.org/pub/NetBSD/NetBSD-current/src/sys/sys/ctype_inline.h
>> The implementation is
>>   #define toupper(c) ((int)((_toupper_tab_ + 1)[(c)]))
>>
>> (where I assume _toupper_tab_ is the obvious array.)
>
> In fact, if NetBSD's implementation is anything like Cygwin's, then it
> is a safe bet that _toupper_tab_ is a symbol that lands in the middle of
> a larger 384-byte array, such that _toupper_tab_[-1] (for (signed
> char)'\xfe') happens to resolve to a valid image address, and will have
> the same contents as _toupper_tab_[255] (for (unsigned char)'\xfe'),
> precisely to cater to so many clueless programs have forgotten about
> signed char widening to negative values (even though the standard says
> the behavior is undefined, quality-of-implementation demands that you
> try to do the sane thing anyway).

Nope, looks like they just implement as _toupper_tab_[0] is for
EOF, and indexes >0 are for the characters, if I've found the
right bit of their libc:
https://ftp.netbsd.org/pub/NetBSD/NetBSD-current/src/lib/libc/locale/rune.c

thanks
-- PMM

[Qemu-devel] [PATCH v2 4/4] qtest: Document calling conventions

[Eric Blake]

We have two flavors of vararg usage in qtest; make it clear that
qmp() has different semantics than hmp(), and let the compiler
enforce that hmp() is used correctly. However, qmp() (and friends)
only accept a subset of printf flags look-alikes (namely, those
that our JSON parser understands), and what is worse, qmp("true")
(the JSON keyword 'true') is different from qmp("%s", "true")
(the JSON string '"true"'), so marking those as printf-like would
produce more harm from bogus warnings than it helps (we may have
made a mistake in previously marking qobject_from_jsonf(), but
this patch is not addressing that).

Signed-off-by: Eric Blake 

---
v2: several comment tweaks, explain why qmp() can't be marked
---
 tests/libqtest.h | 22 ++
 1 file changed, 14 insertions(+), 8 deletions(-)

diff --git a/tests/libqtest.h b/tests/libqtest.h
index 38bc1e9953..ae57282130 100644
--- a/tests/libqtest.h
+++ b/tests/libqtest.h
@@ -50,7 +50,8 @@ void qtest_quit(QTestState *s);
 /**
  * qtest_qmp_discard_response:
  * @s: #QTestState instance to operate on.
- * @fmt...: QMP message to send to qemu
+ * @fmt...: QMP message to send to qemu; formats arguments through
+ * json-lexer.c (only understands '%((l|ll|I64)?d|[ipsf])').
  *
  * Sends a QMP message to QEMU and consumes the response.
  */
@@ -59,7 +60,8 @@ void qtest_qmp_discard_response(QTestState *s, const char 
*fmt, ...);
 /**
  * qtest_qmp:
  * @s: #QTestState instance to operate on.
- * @fmt...: QMP message to send to qemu
+ * @fmt...: QMP message to send to qemu; formats arguments through
+ * json-lexer.c (only understands '%((l|ll|I64)?d|[ipsf])').
  *
  * Sends a QMP message to QEMU and returns the response.
  */
@@ -68,7 +70,8 @@ QDict *qtest_qmp(QTestState *s, const char *fmt, ...);
 /**
  * qtest_async_qmp:
  * @s: #QTestState instance to operate on.
- * @fmt...: QMP message to send to qemu
+ * @fmt...: QMP message to send to qemu; formats arguments through
+ * json-lexer.c (only understands '%((l|ll|I64)?d|[ipsf])').
  *
  * Sends a QMP message to QEMU and leaves the response in the stream.
  */
@@ -134,7 +137,7 @@ QDict *qtest_qmp_eventwait_ref(QTestState *s, const char 
*event);
 /**
  * qtest_hmp:
  * @s: #QTestState instance to operate on.
- * @fmt...: HMP command to send to QEMU
+ * @fmt...: HMP command to send to QEMU, formats arguments like vsprintf().
  *
  * Send HMP command to QEMU via QMP's human-monitor-command.
  * QMP events are discarded.
@@ -535,7 +538,8 @@ static inline void qtest_end(void)

 /**
  * qmp:
- * @fmt...: QMP message to send to qemu
+ * @fmt...: QMP message to send to qemu; formats arguments through
+ * json-lexer.c (only understands '%((l|ll|I64)?d|[ipsf])').
  *
  * Sends a QMP message to QEMU and returns the response.
  */
@@ -543,7 +547,8 @@ QDict *qmp(const char *fmt, ...);

 /**
  * qmp_async:
- * @fmt...: QMP message to send to qemu
+ * @fmt...: QMP message to send to qemu; formats arguments through
+ * json-lexer.c (only understands '%((l|ll|I64)?d|[ipsf])').
  *
  * Sends a QMP message to QEMU and leaves the response in the stream.
  */
@@ -551,7 +556,8 @@ void qmp_async(const char *fmt, ...);

 /**
  * qmp_discard_response:
- * @fmt...: QMP message to send to qemu
+ * @fmt...: QMP message to send to qemu; formats arguments through
+ * json-lexer.c (only understands '%((l|ll|I64)?d|[ipsf])').
  *
  * Sends a QMP message to QEMU and consumes the response.
  */
@@ -592,7 +598,7 @@ static inline QDict *qmp_eventwait_ref(const char *event)

 /**
  * hmp:
- * @fmt...: HMP command to send to QEMU
+ * @fmt...: HMP command to send to QEMU, formats arguments like vsprintf().
  *
  * Send HMP command to QEMU via QMP's human-monitor-command.
  *
-- 
2.13.3

[Qemu-devel] [PATCH v2 2/4] qapi: Visitor documentation tweak

[Eric Blake]

Make it clear that the name parameter to visit_start_struct()
has the same semantics as for visit_start_int().

Signed-off-by: Eric Blake 
Reviewed-by: Markus Armbruster 

---
v2: tweak wording to avoid overloaded 'OBJ'
---
 include/qapi/visitor.h | 14 +++---
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/include/qapi/visitor.h b/include/qapi/visitor.h
index 74768aabda..8243fa271f 100644
--- a/include/qapi/visitor.h
+++ b/include/qapi/visitor.h
@@ -60,13 +60,13 @@
  * visitors are declared here; the remaining visitors are generated in
  * qapi-visit.h.
  *
- * The @name parameter of visit_type_FOO() describes the relation
- * between this QAPI value and its parent container.  When visiting
- * the root of a tree, @name is ignored; when visiting a member of an
- * object, @name is the key associated with the value; when visiting a
- * member of a list, @name is NULL; and when visiting the member of an
- * alternate, @name should equal the name used for visiting the
- * alternate.
+ * The @name parameter of visit_type_FOO() and visit_start_BAR()
+ * describes the relation between this QAPI value and its parent
+ * container.  When visiting the root of a tree, @name is ignored;
+ * when visiting a member of an object, @name is the key associated
+ * with the value; when visiting a member of a list, @name is NULL;
+ * and when visiting the member of an alternate, @name should equal
+ * the name used for visiting the alternate.
  *
  * The visit_type_FOO() functions expect a non-null @obj argument;
  * they allocate *@obj during input visits, leave it unchanged on
-- 
2.13.3

[Qemu-devel] [PATCH for-2.10? v2 0/4] random qapi cleanups

[Eric Blake]

Doc and testsuite changes only, so safe for softfreeze if we like
it (but no real harm if it slips to 2.11).

Since v1:
- drop changes that conflict with Markus' work-in-progress on virtual
alternates
- tweak comments per review from Markus

Eric Blake (4):
  tests: Enhance qobject output to cover partial visit
  qapi: Visitor documentation tweak
  qtest: Avoid passing raw strings through hmp()
  qtest: Document calling conventions

 include/qapi/visitor.h  | 14 ++---
 tests/libqtest.h| 22 ---
 tests/test-hmp.c|  4 ++--
 tests/test-qobject-output-visitor.c | 42 -
 4 files changed, 64 insertions(+), 18 deletions(-)

-- 
2.13.3

[Qemu-devel] [PATCH v2 3/4] qtest: Avoid passing raw strings through hmp()

[Eric Blake]

The next patch will add __attribute__((__format__)) to hmp(), which
in turn causes gcc to warn about non-literal format strings.  Rather
than risk an arbitrary string containing % being mis-handled, always
pass variable strings along with a %s format.  It also makes it
easier to prove correctness locally, rather than auditing all the
source strings.

Signed-off-by: Eric Blake 
Reviewed-by: Markus Armbruster 
---
 tests/test-hmp.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/test-hmp.c b/tests/test-hmp.c
index d77b3c8710..0af066487c 100644
--- a/tests/test-hmp.c
+++ b/tests/test-hmp.c
@@ -80,7 +80,7 @@ static void test_commands(void)
 if (verbose) {
 fprintf(stderr, "\t%s\n", hmp_cmds[i]);
 }
-response = hmp(hmp_cmds[i]);
+response = hmp("%s", hmp_cmds[i]);
 g_free(response);
 }

@@ -103,7 +103,7 @@ static void test_info_commands(void)
 if (verbose) {
 fprintf(stderr, "\t%s\n", info);
 }
-resp = hmp(info);
+resp = hmp("%s", info);
 g_free(resp);
 /* And move forward to the next line */
 info = strchr(endp + 1, '\n');
-- 
2.13.3

[Qemu-devel] [PATCH v2 1/4] tests: Enhance qobject output to cover partial visit

[Eric Blake]

Add a test that proves (at least when run under valgrind) that
we are correctly handling allocated memory even when a visit
is aborted in the middle for whatever other reason.

See commit f24582d "qapi: fix double free in
qmp_output_visitor_cleanup()" for a fix that was lacking
testsuite exposure prior to this patch.

Signed-off-by: Eric Blake 

---
v2: tweak partial alternate visit
---
 tests/test-qobject-output-visitor.c | 42 -
 1 file changed, 41 insertions(+), 1 deletion(-)

diff --git a/tests/test-qobject-output-visitor.c 
b/tests/test-qobject-output-visitor.c
index 749c54065f..ce2fb8f368 100644
--- a/tests/test-qobject-output-visitor.c
+++ b/tests/test-qobject-output-visitor.c
@@ -1,7 +1,7 @@
 /*
  * QObject Output Visitor unit-tests.
  *
- * Copyright (C) 2011-2016 Red Hat Inc.
+ * Copyright (C) 2011-2017 Red Hat Inc.
  *
  * Authors:
  *  Luiz Capitulino 
@@ -251,6 +251,44 @@ static void 
test_visitor_out_struct_errors(TestOutputVisitorData *data,
 }


+static void test_visitor_out_partial_visit(TestOutputVisitorData *data,
+   const void *unused)
+{
+/* Various checks that a mid-visit abort doesn't leak or double-free. */
+const char *str = "hi";
+Error *err = NULL;
+UserDefAlternate uda = {
+.type = QTYPE_QDICT,
+.u.udfu = { .integer = 1,
+.string = (char *) "bye",
+.enum1 = -1 } /* intentionally bad */
+};
+UserDefAlternate *obj = &uda;
+
+/* Abort within a nested object with no data members */
+visit_start_struct(data->ov, NULL, NULL, 0, &error_abort);
+visit_start_struct(data->ov, "nested", NULL, 0, &error_abort);
+visitor_reset(data);
+
+/* Abort in the middle of a list of strings */
+visit_start_list(data->ov, "list", NULL, 0, &error_abort);
+visit_type_str(data->ov, NULL, (char **)&str, &error_abort);
+visit_type_str(data->ov, NULL, (char **)&str, &error_abort);
+visitor_reset(data);
+
+/*
+ * Abort in the middle of an alternate, due to bad "enum1"
+ * discriminator value.  Since alternates don't support virtual
+ * visits, we perform a real one, relying on our knowledge that
+ * visit_type_UserDefAlternate() calls visit_start_alternate()
+ * under the hood.
+ */
+visit_type_UserDefAlternate(data->ov, NULL, &obj, &err);
+error_free_or_abort(&err);
+visitor_reset(data);
+}
+
+
 static void test_visitor_out_list(TestOutputVisitorData *data,
   const void *unused)
 {
@@ -815,6 +853,8 @@ int main(int argc, char **argv)
 &out_visitor_data, test_visitor_out_struct_nested);
 output_visitor_test_add("/visitor/output/struct-errors",
 &out_visitor_data, test_visitor_out_struct_errors);
+output_visitor_test_add("/visitor/output/partial-visit",
+&out_visitor_data, test_visitor_out_partial_visit);
 output_visitor_test_add("/visitor/output/list",
 &out_visitor_data, test_visitor_out_list);
 output_visitor_test_add("/visitor/output/any",
-- 
2.13.3

Re: [Qemu-devel] [RFC] RFC on Backup tool

[Ishani]

- On Jul 17, 2017, at 8:02 PM, stefanha stefa...@redhat.com wrote:

> On Sun, Jul 16, 2017 at 02:13:21AM +0530, Ishani Chugh wrote:
>> +def write_config(self):
>> +"""
>> +Writes configuration to ini file.
>> +"""
>> +with open(self.config_file, 'w') as config_file:
>> +self.config.write(config_file)
> 
> Please update the config file atomically.  That means the file must be
> consistent at all points in time.  Things to consider:
> 
> 1. open(self.config_file, 'w') truncates the destination file.  If the
>   program crashes or is paused before self.config.write() then the
>   config file will appear empty (its contents are lost)!
> 
> 2. Data is written to the file at the end of the 'with' statement
>   (because that flushes the buffer and closes the file), but there is
>   no guarantee that data is safely stored on disk.  Please use
>   https://docs.python.org/3/library/os.html#os.fsync to prevent data
>   loss in case of power failure.
> 
> The usual pattern is:
> 1. Create a temporary file next to the file you wish to modify.  "Next
>   to" is important because if you create it on another file system
>   (like tmpfs) it may not be possible to atomically replace the old
>   file.
> 2. Write data
> 3. file.flush() + os.fsync() to safely store data in the new file
> 4. os.rename() to atomically replace the old file
> 
> For an example, see:
> https://stackoverflow.com/questions/2333872/atomic-writing-to-file-with-python

Thanks. I will update in next revision.

>> +
>> +def get_socket_path(self, socket_path, tcp):
>> +"""
>> +Return Socket address in form of string or tuple
> 
> Please rename this function get_socket_address().  The return value is
> an 'address', not a 'path'.

Will update in next revision.

>> +"""
>> +if tcp is False:
> 
> PEP8 says:
> 
>  Don't compare boolean values to True or False using == .
> 
>  Yes:   if greeting:
>  No:if greeting == True:
>  Worse: if greeting is True:
> 
> So this should be:
> 
>  if not tcp:
> 
Will update in next revision.
>> +return os.path.abspath(socket_path)
>> +return (socket_path.split(':')[0], int(socket_path.split(':')[1]))
> 
> The str.split(delim, maxsplit) argument can be used but feel free to
> keep your version if you prefer:
> 
>  return tuple(socket_path.split(':', 1))
>
 
Thanks for suggestion. It is a much cleaner method.

>> +
>> +def __full_backup(self, guest_name):
>> +"""
>> +Performs full backup of guest
>> +"""
>> +if guest_name not in self.config.sections():
>> +print ("Cannot find specified guest")
>> +return
>> +if self.is_guest_running(guest_name, self.config[guest_name]['qmp'],
>> + self.config[guest_name]['tcp']) is False:
> 
> This is clearer if self.is_guest_running() looks up the necessary
> information in self.config[] itself:
> 
>  if not self.is_guest_running(guest_name):
>  ...
> 
> Since is_guest_running() is a method it has access to self.config[] and
> there's no need for every caller to fetch 'qmp'/'tcp'.

I agree. I am using the same method to check if a guest is running before adding
an entry to config file when user gives command to add guest. It is possible to 
first add entries in config file and then check if the guest is running and 
delete
the entry from config file if guest is found not running. 
 
>> +return
>> +connection = QEMUMonitorProtocol(
>> + self.get_socket_path(
>> + self.config[guest_name]['qmp'],
>> + 
>> self.config[guest_name]['tcp']))
>> +connection.connect()
>> +cmd = {"execute": "transaction", "arguments": {"actions": []}}
>> +for key in self.config[guest_name]:
>> +if key.startswith("drive_"):
>> +drive = key[key.index('_')+1:]
> 
> len('drive_') is a little clearer than key.index('_')+1.

Thanks. Will update in next revision.

>> +target = self.config[guest_name][key]
>> +sub_cmd = {"type": "drive-backup", "data": {"device": drive,
>> +"target": 
>> target,
>> +"sync": "full"}}
>> +cmd['arguments']['actions'].append(sub_cmd)
>> +print (connection.cmd_obj(cmd))
> 
> The non-RFC version of this patch should not print raw qmp.py objects.
> That's useful for debugging but not user-friendly.  Either there could
> be no output and a 0 exit code (indicating success), or there could be a
> message like:
> 
>  Starting full backup of drives:
>   * drive0
>   * drive1
>  Backup complete!

Thanks. Will update in next revision.

>> +
>> +def __drive_add(self, drive_id, guest_name, target=None):
>> +"""
>> +  

Re: [Qemu-devel] [PATCH for-2.10] Use qemu_tolower() and qemu_toupper(), not tolower() and toupper()

[Eric Blake]

On 07/20/2017 04:03 PM, Peter Maydell wrote:
> On 20 July 2017 at 19:26, Richard Henderson  wrote:
>> On 07/20/2017 06:31 AM, Peter Maydell wrote:
>>>
>>> gdbstub.c:914:13: warning: array subscript has type 'char'
>>>
>>> This reflects the fact that toupper() and tolower() give
>>> undefined behaviour if they are passed a value that isn't
>>> a valid 'unsigned char' or EOF.
>>
>>
>> Not saying we shouldn't use qemu_tolower etc, but this statement is not true
>> at all.  Officially, the argument to toupper and tolower is type int.
> 
> (I was going to quote POSIX here but I see Eric has done so already.)
> 
>> This sounds like a bug in NetBSD -- though it may not even be that, as they
>> may have done something clever and put the symbol in the middle of the data.
>> A trick that worked before compiler warnings got smarter.
> 
> https://ftp.netbsd.org/pub/NetBSD/NetBSD-current/src/sys/sys/ctype_inline.h
> The implementation is
>   #define toupper(c) ((int)((_toupper_tab_ + 1)[(c)]))
> 
> (where I assume _toupper_tab_ is the obvious array.)

In fact, if NetBSD's implementation is anything like Cygwin's, then it
is a safe bet that _toupper_tab_ is a symbol that lands in the middle of
a larger 384-byte array, such that _toupper_tab_[-1] (for (signed
char)'\xfe') happens to resolve to a valid image address, and will have
the same contents as _toupper_tab_[255] (for (unsigned char)'\xfe'),
precisely to cater to so many clueless programs have forgotten about
signed char widening to negative values (even though the standard says
the behavior is undefined, quality-of-implementation demands that you
try to do the sane thing anyway).

The table is of course only MOSTLY symmetric; there are some single-byte
locales where isspace((unsigned char)'\xff') is true but isspace(EOF) is
false (in using the magic-middle-of-array locations, _tab_[0] cannot
always match _tab_[256] in all locales).  And it is wraparound where
'\xff' collides with -1 that explains WHY C99 documents the range to be
ints corresponding to unsigned char.

Side note: NetBSD has a bug in their ctype.h.
toupper(INT64_C(0x10001)) is supposed to be the same as toupper(1),
since POSIX requires toupper() to be available as a function (and
permits a macro as well).  A function with an int parameter must
gracefully truncate a 64-bit input down to 32-bits, so the macro must do
the same; but the NetBSD macro uses all 64 bits to dereference into
garbage memory locations.  Fortunately, no one runs into the bug in real
life because no one really tries to pass 64-bit numbers to ctype macros.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v3 42/43] tcg: introduce regions to split code_gen_buffer

[Richard Henderson]

On 07/20/2017 10:50 AM, Emilio G. Cota wrote:

On Wed, Jul 19, 2017 at 22:04:50 -1000, Richard Henderson wrote:

On 07/19/2017 05:09 PM, Emilio G. Cota wrote:

+/* We do not yet support multiple TCG contexts, so use one region for now 
*/
+n_regions = 1;
+
+/* start on a page-aligned address */
+buf = QEMU_ALIGN_PTR_UP(buf, qemu_real_host_page_size);
+g_assert(buf < tcg_init_ctx.code_gen_buffer + size);
+
+/* discard that initial portion */
+size -= buf - tcg_init_ctx.code_gen_buffer;


It seems pointless wasting most of a page after the prologue when n_regions
== 1.  We don't really need to start on a page boundary in that case.


+/* make region_size a multiple of page_size */
+region_size = size / n_regions;
+region_size = QEMU_ALIGN_DOWN(region_size, qemu_real_host_page_size);


This division can result in a number of pages at the end of the region being
unused.  Is it worthwhile freeing them?  Or marking them mprotect_none along
with the last guard page?


Perhaps we should then enlarge both the first and last regions so that we
fully use the buffer.


I really like the idea.  That's a lot of space recovered for 64k page hosts.

I do think we can make the computation clearer.  How about



  static void tcg_region_assign(TCGContext *s, size_t curr_region)
  {
+void *aligned = QEMU_ALIGN_PTR_UP(region.start, qemu_real_host_page_size);
  void *buf;
+size_t size = region.size;
  
-buf = region.buf + curr_region * (region.size + qemu_real_host_page_size);

+/* The beginning of the first region might not be page-aligned */
+if (curr_region == 0) {
+buf = region.start;
+size += aligned - buf;
+} else {
+buf = aligned + curr_region * (region.size +  
qemu_real_host_page_size);
+}
+/* the last region might be larger than region.size */
+if (curr_region == region.n - 1) {
+void *aligned_end = buf + size;
+
+size += region.end - qemu_real_host_page_size - aligned_end;
+}
  s->code_gen_buffer = buf;
  s->code_gen_ptr = buf;
-s->code_gen_buffer_size = region.size;
-s->code_gen_highwater = buf + region.size - TCG_HIGHWATER;
+s->code_gen_buffer_size = size;
+s->code_gen_highwater = buf + size - TCG_HIGHWATER;
  }


static inline void tcg_region_bounds(TCGContext *s, size_t curr_region,
 void **pstart, void **pend)
{
  void *start, *end;

  /* ??? Maybe store "aligned" precomputed.  */
  start = QEMU_ALIGN_PTR_UP(region.start, qemu_real_host_page_size);
  /* ??? Maybe store "stride" precomputed.  */
  start += curr_region * (region.size + qemu_real_host_page_size);
  end = start + region.size;

  if (curr_region == 0) {
start = region.start;
  }
  if (curr_region == region.n - 1) {
end = region.end;
  }

  *pstart = start;
  *pend = end;
}

static void tcg_region_assign(TCGContext *s, size_t curr_region)
{
  void *start, *end;

  tcg_region_bounds(s, curr_region, &start, &end);

  s->code_gen_buffer = start;
  s->code_gen_ptr = start;
  s->code_gen_buffer_size = end - start;
  s->code_gen_highwater = end - TCG_HIGHWATER;
}


@@ -409,44 +424,50 @@ static size_t tcg_n_regions(void)
  void tcg_region_init(void)
  {
  void *buf = tcg_init_ctx.code_gen_buffer;
+void *aligned;
  size_t size = tcg_init_ctx.code_gen_buffer_size;
+size_t page_size = qemu_real_host_page_size;
  size_t region_size;
  size_t n_regions;
  size_t i;
+int rc;
  
  n_regions = tcg_n_regions();
  
-/* start on a page-aligned address */

-buf = QEMU_ALIGN_PTR_UP(buf, qemu_real_host_page_size);
-g_assert(buf < tcg_init_ctx.code_gen_buffer + size);
-
-/* discard that initial portion */
-size -= buf - tcg_init_ctx.code_gen_buffer;
-
-/* make region_size a multiple of page_size */
-region_size = size / n_regions;
-region_size = QEMU_ALIGN_DOWN(region_size, qemu_real_host_page_size);
+/* The first region will be 'aligned - buf' bytes larger than the others */
+aligned = QEMU_ALIGN_PTR_UP(buf, page_size);
+g_assert(aligned < tcg_init_ctx.code_gen_buffer + size);
+/*
+ * Make region_size a multiple of page_size, using aligned as the start.
+ * As a result of this we might end up with a few extra pages at the end of
+ * the buffer; we will assign those to the last region.
+ */
+region_size = (size - (aligned - buf)) / n_regions;
+region_size = QEMU_ALIGN_DOWN(region_size, page_size);
  
  /* A region must have at least 2 pages; one code, one guard */

-g_assert(region_size >= 2 * qemu_real_host_page_size);
+g_assert(region_size >= 2 * page_size);
  
  /* init the region struct */

  qemu_mutex_init(®ion.lock);
  region.n = n_regions;
-region.buf = buf;
+region.start = buf;
+/* page-align the end, since its last page will be a guard page */
+region.end = QEMU_ALIGN_PTR_DOWN(buf + size, page_size);
  /* do not count the g

Re: [Qemu-devel] [PATCH for-2.10] util/oslib-posix.c: Avoid warning on NetBSD

[Peter Maydell]

On 20 July 2017 at 22:10, Eric Blake  wrote:
> On 07/20/2017 03:53 PM, Peter Maydell wrote:
>> On 20 July 2017 at 19:26, Eric Blake  wrote:
>>> On 07/20/2017 11:32 AM, Peter Maydell wrote:
 On NetBSD the compiler warns:
 util/oslib-posix.c: In function 'sigaction_invoke':
 util/oslib-posix.c:589:5: warning: missing braces around initializer 
 [-Wmissing-braces]
  siginfo_t si = { 0 };
  ^
>>>
>>> Uggh. That is a broken compiler.  C99 declares that 'anything = {0}' is
>>> supposed to be a valid way to zero-initialize anything.
>>
>> Looks like maybe it was https://gcc.gnu.org/bugzilla/show_bug.cgi?id=53119
>
> Would it be better to add a configure check for broken -Wmissing-braces
> (where we would then add -Wno-missing-braces to the (outdated) NetBSD
> compiler)?

There's also a pointer to a clang bug which is not marked as fixed,
and even in gcc it's down as not fixed in 4.8 or 4.9 if I'm reading
the bug comments right.

> But then again, we already rely on gcc/clang's extension of foo={}
> (which is valid for C++, what a shame that the two languages picked
> different universal-zero initializers), so it's probably less churn to
> just fix the few outliers to also rely on the extension than it is to
> bloat configure just to permanently work around the broken compiler.

This is AFAICT the only instance that needs fixing, because the
compiler only warns if the first element in the struct isn't
a simple data type that can be initialized with '= 0'. If NetBSD
had happened to use a similar definition to Linux (which happens
to start with "int si_signo;") we wouldn't have had to change this
initializer. (The fact that so many still-prevalent gcc versions
behave like this is probably why the only case the NetBSD compile
found is one that is an initializer of a system-header-defined
struct where NetBSD has taken a somewhat odd approach.)

(Personally if I were the C standards folks I'd fix it by
blessing the gcc-extension that makes "{}" a valid zero
initializer and bringing it into line with C++.)

thanks
-- PMM

Re: [Qemu-devel] [RFC] RFC on Backup tool

[Ishani]

- On Jul 19, 2017, at 2:59 AM, jsnow js...@redhat.com wrote:

> On 07/17/2017 03:37 PM, Ishani wrote:
>> - On Jul 17, 2017, at 12:48 PM, Fam Zheng f...@redhat.com wrote:
>>> On Sun, 07/16 02:13, Ishani Chugh wrote:
> 
> [...]
> 
>>> Only full backup is implemented in this patch, is the plan to add 
>>> incremental
>>> backup on top?  I'm curious because you have target file path set during 
>>> drive
>>> add, but in the incremental case, it should be possible that each backup 
>>> creates
>>> a new target file that chains up with earlier ones, so I think the target 
>>> file
>>> should be an option for "backup" command as well.
>> 
>> Yes. Incremental backup is to be added. I am still in learning phase with
>> respect to incremental backups. I will modify the arguments and workflow
>> accordingly.
>> 
> 
> You may consider solidifying the backup target *pattern* during drive
> add as an alternative, such as:
> 
> .../path/to/backup/%VM%/%DRIVE%/%%-%mm%-%dd%.qcow2
> 
> Or some such scheme. Simple numerals work well, too:
> 
> myvm/sda/incr.0.qcow2
> myvm/sda/incr.1.qcow2
> 
> Simple numerals offer the benefit that it is easier to reconstruct the
> chain if you lose your metadata in the python script.
> 
> Also consider that even for non-incremental backups, we want full
> backups made subsequently to not, in general, overwrite the previous
> full backup, so the TARGET is more of a "living entity" than a fixed
> thing, even in the simple case.

Okay. Thats a great suggestion. But, it might not work when the entire chain
of backups is not present at the same location. Can a case like this arise?

 It is intended as a
 reference implementation for management stack and backup developers
 to see QEMU's backup features in action. The tool writes details of
 guest in a configuration file and the data is retrieved from the file
 while creating a backup. The usage is as follows:
 Add a guest
 python qemu-backup.py guest add --guest  --qmp  
 [--tcp]

 Add a drive for backup in a specified guest
 python qemu-backup.py drive add --guest  --id  
 [--target
 ]

 Create backup of the added drives:
 python qemu-backup.py backup --guest 

 List all guest configs in configuration file:
 python qemu-backup.py guest list
>>>
>>> Please put these examples into the help output of the command, this way 
>>> users
>>> can read it as well.
>> 
>> I have created a manpage documentation for the tool.
>> http://lists.nongnu.org/archive/html/qemu-devel/2017-07/msg05241.html
>> It is to be updated continuously as the development progresses.
>> 
> 
> You can include this (or a similar link) in future cover letters for the
> benefit of reviewers.

Okay. Will update in next revision.


 I will be obliged by any feedback.
>>>
>>> Thanks for doing this!
>>>
> 
> Yes, thank you :)
> 

Thanks for review. :)


 Signed-off-by: Ishani Chugh 
 ---
  contrib/backup/qemu-backup.py | 244 
 ++
  1 file changed, 244 insertions(+)
  create mode 100644 contrib/backup/qemu-backup.py

 diff --git a/contrib/backup/qemu-backup.py b/contrib/backup/qemu-backup.py
 new file mode 100644
 index 000..9c3dc53
 --- /dev/null
 +++ b/contrib/backup/qemu-backup.py
 @@ -0,0 +1,244 @@
 +#!/usr/bin/python
 +# -*- coding: utf-8 -*-
>>>
>>> You need a copyright header here (the default choice for QEMU is GPLv2 but 
>>> there
>>> is no strict restrictions for scripts). See examples in other *.py files.
>> 
>> Thanks. Will update in next revision.
>>  
> 
> Yes, up to you. Files without a copyright default to GPLv2 in the QEMU
> project, but if you feel strongly one way or another you can argue for
> that license in upstream review.
> 
> (For instance, documentation and other non-code documents can sometimes
> be better served by different licenses.)
> 
> We tend to avoid the implicit copyright when possible, so including an
> explicit GPLv2 license is preferable to declare intent.
> 
> QEMU as a whole is GPLv2, so this is a good license to use if you don't
> have any strong feelings on the matter, but please take a moment to read
> the implications of the license as a new contributor to our project:
> 
> https://tldrlegal.com/license/gnu-general-public-license-v2
> 
> And the full, legal text:
> 
> https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html
> 

I will go through the licenses and update in next revision. Thanks.

 +"""
 +This file is an implementation of backup tool
 +"""
 +from argparse import ArgumentParser
 +import os
 +import errno
 +from socket import error as socket_error
 +import configparser
>>>
>>> Python2 has ConfigParser while python3 has configparser. Please be specific
>>> about the python compatibility level of this script - my system (Fedora) has
>>> python2 as /usr/bin/python, so the shebang and your example co

Re: [Qemu-devel] [PATCH for-2.10] util/oslib-posix.c: Avoid warning on NetBSD

[Eric Blake]

On 07/20/2017 03:53 PM, Peter Maydell wrote:
> On 20 July 2017 at 19:26, Eric Blake  wrote:
>> On 07/20/2017 11:32 AM, Peter Maydell wrote:
>>> On NetBSD the compiler warns:
>>> util/oslib-posix.c: In function 'sigaction_invoke':
>>> util/oslib-posix.c:589:5: warning: missing braces around initializer 
>>> [-Wmissing-braces]
>>>  siginfo_t si = { 0 };
>>>  ^
>>
>> Uggh. That is a broken compiler.  C99 declares that 'anything = {0}' is
>> supposed to be a valid way to zero-initialize anything.
> 
> Looks like maybe it was https://gcc.gnu.org/bugzilla/show_bug.cgi?id=53119

Would it be better to add a configure check for broken -Wmissing-braces
(where we would then add -Wno-missing-braces to the (outdated) NetBSD
compiler)?

But then again, we already rely on gcc/clang's extension of foo={}
(which is valid for C++, what a shame that the two languages picked
different universal-zero initializers), so it's probably less churn to
just fix the few outliers to also rely on the extension than it is to
bloat configure just to permanently work around the broken compiler.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] Build failure on git master

[Peter Maydell]

On 20 July 2017 at 20:29, Mark Cave-Ayland
 wrote:
> On 19/07/17 18:09, Mark Cave-Ayland wrote:
>
>> Hi all,
>>
>> I see the following build failure here on git master with gcc 4.7.2:
>>
>> cc -I/home/build/src/qemu/git/qemu/block -Iblock
>> -I/home/build/src/qemu/git/qemu/tcg
>> -I/home/build/src/qemu/git/qemu/tcg/i386
>> -I/home/build/src/qemu/git/qemu/linux-headers
>> -I/home/build/src/qemu/git/qemu/linux-headers -I.
>> -I/home/build/src/qemu/git/qemu
>> -I/home/build/src/qemu/git/qemu/accel/tcg
>> -I/home/build/src/qemu/git/qemu/include -I/usr/include/pixman-1
>> -I/home/build/src/qemu/git/qemu/dtc/libfdt -Werror -pthread
>> -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include
>> -m64 -mcx16 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE
>> -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings
>> -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv
>> -Wendif-labels -Wno-missing-include-dirs -Wempty-body -Wnested-externs
>> -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers
>> -Wold-style-declaration -Wold-style-definition -Wtype-limits
>> -fstack-protector-all -I/usr/include/p11-kit-1
>> -I/usr/include/libpng12   -I/home/build/src/qemu/git/qemu/tests -MMD -MP
>> -MT block/vpc.o -MF block/vpc.d -O2 -U_FORTIFY_SOURCE
>> -D_FORTIFY_SOURCE=2 -g   -c -o block/vpc.o block/vpc.c
>> block/vpc.c: In function ‘vpc_co_pwritev’:
>> block/vpc.c:652:9: error: ‘ret’ may be used uninitialized in this
>> function [-Werror=maybe-uninitialized]
>> cc1: all warnings being treated as errors
>> make: *** [block/vpc.o] Error 1
>>
>> At a quick glance it looks as if this is being caused by commit cfc87e00
>> "block/vpc.c: Handle write failures in get_image_offset()".
>
> The fix is seemingly quite simple:
>
> diff --git a/block/vpc.c b/block/vpc.c
> index 10e6519..574879b 100644
> --- a/block/vpc.c
> +++ b/block/vpc.c
> @@ -649,7 +649,7 @@ vpc_co_pwritev(BlockDriverState *bs, uint64_t
> offset, uint64_t bytes,
>  int64_t image_offset;
>  int64_t n_bytes;
>  int64_t bytes_done = 0;
> -int ret;
> +int ret = 0;
>  VHDFooter *footer =  (VHDFooter *) s->footer_buf;

Yeah, I think 4.7.2 isn't smart enough to realize this is
a false positive (and that compiler is getting old enough
that it's not in the set I usually test with).

Do you want to send a proper patch?

thanks
-- PMM

Re: [Qemu-devel] [PATCH for-2.10] Use qemu_tolower() and qemu_toupper(), not tolower() and toupper()

[Peter Maydell]

On 20 July 2017 at 19:26, Richard Henderson  wrote:
> On 07/20/2017 06:31 AM, Peter Maydell wrote:
>>
>> gdbstub.c:914:13: warning: array subscript has type 'char'
>>
>> This reflects the fact that toupper() and tolower() give
>> undefined behaviour if they are passed a value that isn't
>> a valid 'unsigned char' or EOF.
>
>
> Not saying we shouldn't use qemu_tolower etc, but this statement is not true
> at all.  Officially, the argument to toupper and tolower is type int.

(I was going to quote POSIX here but I see Eric has done so already.)

> This sounds like a bug in NetBSD -- though it may not even be that, as they
> may have done something clever and put the symbol in the middle of the data.
> A trick that worked before compiler warnings got smarter.

https://ftp.netbsd.org/pub/NetBSD/NetBSD-current/src/sys/sys/ctype_inline.h
The implementation is
  #define toupper(c) ((int)((_toupper_tab_ + 1)[(c)]))

(where I assume _toupper_tab_ is the obvious array.)

thanks
-- PMM

Re: [Qemu-devel] [PATCH v6] Add manpage for QEMU Backup Tool

[Ishani]

- On Jul 20, 2017, at 6:16 PM, stefanha stefa...@redhat.com wrote:

> On Tue, Jul 18, 2017 at 12:45:36AM +0530, Ishani Chugh wrote:
> 
> This looks good.  I think it makes sense to include this patch with the
> patch series that adds the qemu-backup command.  That way the command
> and its man page will be merged together.

Thanks. I will include the patch for qemu-backup in same patch series.
 
>> +@item qemu-backup guest add --guest guestname --qmp socketpath [--tcp]
> 
> For consistency I suggest following QEMU chardev syntax for the socket
> path.  This also allows you to drop --tcp.
> 
> UNIX domain sockets look like this:
> 
>  unix:/path/to/socket
> 
> TCP addresses look like this:
> 
>   tcp:127.0.0.1:1234
Okay. Will fix it in next revision.Thanks.

Re: [Qemu-devel] [PATCH for-2.10] util/oslib-posix.c: Avoid warning on NetBSD

[Peter Maydell]

On 20 July 2017 at 19:26, Eric Blake  wrote:
> On 07/20/2017 11:32 AM, Peter Maydell wrote:
>> On NetBSD the compiler warns:
>> util/oslib-posix.c: In function 'sigaction_invoke':
>> util/oslib-posix.c:589:5: warning: missing braces around initializer 
>> [-Wmissing-braces]
>>  siginfo_t si = { 0 };
>>  ^
>
> Uggh. That is a broken compiler.  C99 declares that 'anything = {0}' is
> supposed to be a valid way to zero-initialize anything.

Looks like maybe it was https://gcc.gnu.org/bugzilla/show_bug.cgi?id=53119

thanks
-- PMM

Re: [Qemu-devel] [PATCH 5/5] qtest: Document calling conventions

[Eric Blake]

On 07/20/2017 03:37 PM, Eric Blake wrote:
>>> + * @fmt...: QMP message to send to qemu; only recognizes formats
>>> + * understood by json-lexer.c
>>>   *
>>>   * Sends a QMP message to QEMU and consumes the response.
>>>   */
>>
>> These formats are chosen so that gcc can help us check them.  Please add
>> GCC_FMT_ATTR().  Precedence: qobject_from_jsonf().
> 
> Will do.  (This patch was originally part of my larger series trying to
> get rid of qobject_from_jsonf() - I may still succeed at that, but
> separating the easy stuff from the controversial means that the easy
> stuff needs tweaking).

Bleargh.  It's not that simple.

With printf-style, hmp("literal") and hmp("%s", "literal") produce the
same results.

But with json-lexer style, %s MODIFIES its input.
The original qmp("{'execute':\"foo\"}") sends a JSON object
 {'execute':"foo"}
but counterpart qmp("%s", "{'execute':'foo'}") sends a JSON string with
added \ escaping:
 "{'execute':\"foo\"}"

So adding the format immediately causes lots of warnings, such as:

  CC  tests/vhost-user-test.o
tests/vhost-user-test.c: In function ‘test_migrate’:
tests/vhost-user-test.c:668:5: error: format not a string literal and no
format arguments [-Werror=format-security]
 rsp = qmp(cmd);
 ^~~

  CC  tests/boot-order-test.o
tests/boot-order-test.c: In function ‘test_a_boot_order’:
tests/boot-order-test.c:46:26: error: zero-length gnu_printf format
string [-Werror=format-zero-length]
 qmp_discard_response("");   /* HACK: wait for event */
  ^~

but we CAN'T rewrite those to qmp("%s", command).

Now you can sort-of understand why my larger series wanted to get rid of
qobject_from_jsonf(), since our use of GCC_FMT_ATTR() there is a lie?

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v3 42/43] tcg: introduce regions to split code_gen_buffer

[Emilio G. Cota]

On Wed, Jul 19, 2017 at 22:04:50 -1000, Richard Henderson wrote:
> On 07/19/2017 05:09 PM, Emilio G. Cota wrote:
> >+/* We do not yet support multiple TCG contexts, so use one region for 
> >now */
> >+n_regions = 1;
> >+
> >+/* start on a page-aligned address */
> >+buf = QEMU_ALIGN_PTR_UP(buf, qemu_real_host_page_size);
> >+g_assert(buf < tcg_init_ctx.code_gen_buffer + size);
> >+
> >+/* discard that initial portion */
> >+size -= buf - tcg_init_ctx.code_gen_buffer;
> 
> It seems pointless wasting most of a page after the prologue when n_regions
> == 1.  We don't really need to start on a page boundary in that case.
> 
> >+/* make region_size a multiple of page_size */
> >+region_size = size / n_regions;
> >+region_size = QEMU_ALIGN_DOWN(region_size, qemu_real_host_page_size);
> 
> This division can result in a number of pages at the end of the region being
> unused.  Is it worthwhile freeing them?  Or marking them mprotect_none along
> with the last guard page?

Perhaps we should then enlarge both the first and last regions so that we
fully use the buffer.

What do you think of the below? It's a delta over the v3 patch.

Emilio

---8<---

diff --git a/tcg/tcg.c b/tcg/tcg.c
index a5c01be..8bf8bca 100644
--- a/tcg/tcg.c
+++ b/tcg/tcg.c
@@ -129,7 +129,8 @@ static unsigned int n_tcg_ctxs;
  */
 struct tcg_region_state {
 QemuMutex lock;
-void *buf;  /* set at init time */
+void *start;/* set at init time */
+void *end;  /* set at init time */
 size_t n;   /* set at init time */
 size_t size;/* size of one region; set at init time */
 size_t current; /* protected by the lock */
@@ -277,13 +278,27 @@ TCGLabel *gen_new_label(void)
 
 static void tcg_region_assign(TCGContext *s, size_t curr_region)
 {
+void *aligned = QEMU_ALIGN_PTR_UP(region.start, qemu_real_host_page_size);
 void *buf;
+size_t size = region.size;
 
-buf = region.buf + curr_region * (region.size + qemu_real_host_page_size);
+/* The beginning of the first region might not be page-aligned */
+if (curr_region == 0) {
+buf = region.start;
+size += aligned - buf;
+} else {
+buf = aligned + curr_region * (region.size +  
qemu_real_host_page_size);
+}
+/* the last region might be larger than region.size */
+if (curr_region == region.n - 1) {
+void *aligned_end = buf + size;
+
+size += region.end - qemu_real_host_page_size - aligned_end;
+}
 s->code_gen_buffer = buf;
 s->code_gen_ptr = buf;
-s->code_gen_buffer_size = region.size;
-s->code_gen_highwater = buf + region.size - TCG_HIGHWATER;
+s->code_gen_buffer_size = size;
+s->code_gen_highwater = buf + size - TCG_HIGHWATER;
 }
 
 static bool tcg_region_alloc__locked(TCGContext *s)
@@ -409,44 +424,50 @@ static size_t tcg_n_regions(void)
 void tcg_region_init(void)
 {
 void *buf = tcg_init_ctx.code_gen_buffer;
+void *aligned;
 size_t size = tcg_init_ctx.code_gen_buffer_size;
+size_t page_size = qemu_real_host_page_size;
 size_t region_size;
 size_t n_regions;
 size_t i;
+int rc;
 
 n_regions = tcg_n_regions();
 
-/* start on a page-aligned address */
-buf = QEMU_ALIGN_PTR_UP(buf, qemu_real_host_page_size);
-g_assert(buf < tcg_init_ctx.code_gen_buffer + size);
-
-/* discard that initial portion */
-size -= buf - tcg_init_ctx.code_gen_buffer;
-
-/* make region_size a multiple of page_size */
-region_size = size / n_regions;
-region_size = QEMU_ALIGN_DOWN(region_size, qemu_real_host_page_size);
+/* The first region will be 'aligned - buf' bytes larger than the others */
+aligned = QEMU_ALIGN_PTR_UP(buf, page_size);
+g_assert(aligned < tcg_init_ctx.code_gen_buffer + size);
+/*
+ * Make region_size a multiple of page_size, using aligned as the start.
+ * As a result of this we might end up with a few extra pages at the end of
+ * the buffer; we will assign those to the last region.
+ */
+region_size = (size - (aligned - buf)) / n_regions;
+region_size = QEMU_ALIGN_DOWN(region_size, page_size);
 
 /* A region must have at least 2 pages; one code, one guard */
-g_assert(region_size >= 2 * qemu_real_host_page_size);
+g_assert(region_size >= 2 * page_size);
 
 /* init the region struct */
 qemu_mutex_init(®ion.lock);
 region.n = n_regions;
-region.buf = buf;
+region.start = buf;
+/* page-align the end, since its last page will be a guard page */
+region.end = QEMU_ALIGN_PTR_DOWN(buf + size, page_size);
 /* do not count the guard page in region.size */
-region.size = region_size - qemu_real_host_page_size;
+region.size = region_size - page_size;
 
-/* set guard pages */
-for (i = 0; i < region.n; i++) {
+/* set guard pages for the first n-1 regions */
+for (i = 0; i < region.n - 1; i++) {
 void *guard;
-int rc;
 
-   

Re: [Qemu-devel] [PATCH 5/5] qtest: Document calling conventions

[Eric Blake]

On 07/20/2017 05:10 AM, Markus Armbruster wrote:
> Eric Blake  writes:
> 
>> We have two flavors of vararg usage in qtest; make it clear that
>> qmp() has different semantics than hmp(), and let the compiler
>> enforce that hmp() is used correctly. Since qmp() only accepts
>> a subset of printf flags (namely, those that our JSON parser
>> understands), I figured that it is probably not worth adding a
>> format attribute to qmp() at this time.
>>
>> Signed-off-by: Eric Blake 
>> ---
>>  tests/libqtest.h | 23 ++-
>>  1 file changed, 14 insertions(+), 9 deletions(-)
>>
>> diff --git a/tests/libqtest.h b/tests/libqtest.h
>> index 38bc1e9..762ed13 100644
>> --- a/tests/libqtest.h
>> +++ b/tests/libqtest.h
>> @@ -50,7 +50,8 @@ void qtest_quit(QTestState *s);
>>  /**
>>   * qtest_qmp_discard_response:
>>   * @s: #QTestState instance to operate on.
>> - * @fmt...: QMP message to send to qemu
>> + * @fmt...: QMP message to send to qemu; only recognizes formats
>> + * understood by json-lexer.c
>>   *
>>   * Sends a QMP message to QEMU and consumes the response.
>>   */
> 
> These formats are chosen so that gcc can help us check them.  Please add
> GCC_FMT_ATTR().  Precedence: qobject_from_jsonf().

Will do.  (This patch was originally part of my larger series trying to
get rid of qobject_from_jsonf() - I may still succeed at that, but
separating the easy stuff from the controversial means that the easy
stuff needs tweaking).

> 
> Where are the "formats understood by json-lexer.c" documented?

Near the top of qobject/json-lexer.c:

 * Extension for vararg handling in JSON construction:
 *
 * %((l|ll|I64)?d|[ipsf])

Note that %i differs from %d (the former produces true/false, while the
latter produces "42" and friends - but it happens to "work" with gcc's
-Wformat checking, thanks to var-arg type promotion rules).

I could just spell it out directly (in fact, in the above-mentioned
larger series, I still kept qtest_qmp() able to understand %s, but
dropped all the other formats like %ld and %p).

>> + * @fmt...: HMP command to send to QEMU, passed through sprintf()
> 
> Not actually through sprintf(), but I get what you mean.  I like to
> document such things as "Format arguments like vsprintf()."

Works for me.

>> @@ -592,13 +597,13 @@ static inline QDict *qmp_eventwait_ref(const char 
>> *event)
>>
>>  /**
>>   * hmp:
>> - * @fmt...: HMP command to send to QEMU
>> + * @fmt...: HMP command to send to QEMU, passed through printf()
> 
> Here, you claim printf().  Typo?

Or inconsistent copy-and-paste.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH 3/5] qapi: Visitor documentation tweak

[Eric Blake]

On 07/20/2017 05:00 AM, Markus Armbruster wrote:
> Eric Blake  writes:
> 
>> Make it clear that the name parameter to visit_start_struct()
>> has the same semantics as for visit_start_int().
>>
>> Signed-off-by: Eric Blake 
>> ---

>> + * The @name parameter of visit_type_FOO() and visit_start_OBJECT()
> 
> Our terminology is horrible:
> 
> JSON object / QDict   / QAPI complex type (struct or union)
> JSON array  / QList   / QAPI array or list (used interchangeably)
> JSON value  / QObject / QAPI I'm-not-even-sure-what
> 
> Because of this mess, nobody knows what you mean when you say OBJECT.
> 
> visit_start_BAR()?

Works for me.

> Reviewed-by: Markus Armbruster 

Since you suggested it, I'll keep your R-b when I send v2.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH 2/5] tests: Enhance qobject output to cover partial visit

[Eric Blake]

On 07/20/2017 04:52 AM, Markus Armbruster wrote:
> Eric Blake  writes:
> 
>> Add a test that proves (at least when run under valgrind) that
>> we are correctly handling allocated memory even when a visit
>> is aborted in the middle for whatever other reason.
>>
>> See commit f24582d "qapi: fix double free in
>> qmp_output_visitor_cleanup()" for a fix that was lacking
>> testsuite exposure prior to this patch.
>>

>> +
>> +/*
>> + * Abort in the middle of an alternate. Alternates can't be
>> + * virtually visited, so we get to inline the first half of
>> + * visit_type_UserDefAlternate().
>> + */
> 
> Not exactly inline.  Perhaps:
> 
>/*
> * Abort in the middle of an alternate.  Since alternates don't
> * support virtual visits, we perform a real one, similar to what
> * visit_type_UserDefAlternate() would do.
> */

Sounds reasonable, if we go with it.

> 
> Hmm, what would visit_type_UserDefAlternate() do for @uda?  Could we
> simply call it here and be done?

Sounds even better; I'll do that for v2.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH 1/5] qapi: Further enhance visitor virtual walk doc example

[Eric Blake]

On 07/20/2017 04:05 AM, Markus Armbruster wrote:
> Eric Blake  writes:
> 
>> Markus pointed out that the example given for virtual walks did
>> not discuss how to do a virtual walk of an alternate type.  It
>> turns out that for output, we don't need to visit an alternate
>> (just directly visit the type that we want); and for input,
>> visit_start_alternate() is not currently wired up for alternates
>> (it requires a QAPI type, rather than permitting NULL for a
>> virtual walk).  Also, the example was never updated in commit
>> 3b098d5 about where visit_complete() would fit in.  Improve the
>> description and example to give more details along these lines.
>>
>> Signed-off-by: Eric Blake 
> 
> This clashes with some unfinished work I have on alternates.  If I can
> finish it quickly, we can compare and decide whether we still need this.

It sounds like your stuff is 2.11 material; at this point, I'm trying to
decide which of my patches are still worth 2.10 softfreeze material.
While doc fixes are safe, I'm also fine stating that we don't want
churn, so I'll remove this part of my series from 2.10 consideration,
and like you say, we'll compare against your work in 2.11 later.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v3 1/3] qemu.py: fix is_running()

[Amador Pahim]

On Thu, Jul 20, 2017 at 7:49 PM, Eduardo Habkost  wrote:
> On Thu, Jul 20, 2017 at 05:09:11PM +0200, Markus Armbruster wrote:
>> Amador Pahim  writes:
>>
>> > On Thu, Jul 20, 2017 at 1:49 PM, Markus Armbruster  
>> > wrote:
>> >> Amador Pahim  writes:
>> >>
>> >>> Current implementation is broken. It does not really test if the child
>> >>> process is running.
>> >>
>> >> What usage exactly is broken by this?  Got a reproducer for me?
>> >
>> > Problem is that 'returncode' is not set without a calling
>> > poll()/wait()/communicate(), so it's only useful to test if the
>> > process is running after such calls. But if we use 'poll()' instead,
>> > it will, according to the docs, "Check if child process has
>> > terminated. Set and return returncode attribute."
>> >
>> > Reproducer is:
>> >
>> >  >>> import subprocess
>> >  >>> devnull = open('/dev/null', 'rb')
>> >  >>> p = subprocess.Popen(['qemu-system-x86_64', '-broken'],
>> > stdin=devnull, stdout=devnull, stderr=devnull, shell=False)
>> >  >>> print p.returncode
>> >  None
>> >  >>> print p.poll()
>> >  1
>> >  >>> print p.returncode
>> >  1
>> >
>> >>> The Popen.returncode will only be set after by a poll(), wait() or
>> >>> communicate(). If the Popen fails to launch a VM, the Popen.returncode
>> >>> will not turn to None by itself.
>> >>
>> >> Hmm.  What is the value of .returncode then?
>> >
>> > returncode starts with None and becomes the process exit code when the
>> > process is over and one of that three methods is called (poll(),
>> > wait() or communicate()).
>> >
>> > There's an error in my description though. The correct would be: "The
>> > Popen.returncode will only be set after a call to poll(), wait() or
>> > communicate(). If the Popen fails to launch a VM, the Popen.returncode
>> > will not turn from None to the actual return code by itself."
>>
>> Suggest to add ", and is_running() continues to report True".
>>
>> >>> Instead of using Popen.returncode, let's use Popen.poll(), which
>> >>> actually checks if child process has terminated.
>> >>>
>> >>> Signed-off-by: Amador Pahim 
>> >>> Reviewed-by: Eduardo Habkost 
>> >>> Reviewed-by: Fam Zheng 
>> >>> ---
>> >>>  scripts/qemu.py | 2 +-
>> >>>  1 file changed, 1 insertion(+), 1 deletion(-)
>> >>>
>> >>> diff --git a/scripts/qemu.py b/scripts/qemu.py
>> >>> index 880e3e8219..f0fade32bd 100644
>> >>> --- a/scripts/qemu.py
>> >>> +++ b/scripts/qemu.py
>> >>> @@ -86,7 +86,7 @@ class QEMUMachine(object):
>> >>>  raise
>> >>>
>> >>>  def is_running(self):
>> >>> -return self._popen and (self._popen.returncode is None)
>> >>> +return self._popen and (self._popen.poll() is None)
>> >>>
>
> After re-reading shutdown(), I think this is _not_ OK: if
> is_running() return False before we call .wait(), we will never
> load the log file or run _post_shutdown() if QEMU exits between
> the launch() and shutdown() calls.

Yes, I just noticed that while cleaning up the code.

>
> Yes, it's fragile.
>
> The root problem on both launch() and shutdown() seems to be
> coupling the external "is QEMU running?" state with the internal
> "did we load the log file and ran _post_shutdown() already?"
> state.
>
> I see two possible approaches for this:
>
> 1) Benefit from the fact that the internal Popen state will not
>change under our feet unless we explicitly call
>poll()/wait()/etc, and keep the existing code.  (Not my
>favorite option)
>
> 2) Rewrite the code so that we don't depend on the subtle Popen
>internal state rules, and track our own internal state in
>a QEMUMachine attribute.  e.g.:

+1 for this approach. I'm working on something similar, thanks for the
detailed "e.g." code here.

>
> def _handle_shutdown(self):
> '''Load log file and call _post_shutdown() hook if necessary'''
> # Must be called only after QEMU actually exited.
> assert not self.is_running()
> if self._shutdown_pending:
> if self.exitcode() < 0:
> sys.stderr.write('qemu received signal %i: %s\n' % 
> (-exitcode, ' '.join(self._args)))
> self._load_io_log()
> self._post_shutdown()
> self._shutdown_pending = False
>
> def _terminate(self):
> '''Terminate QEMU if it's still running'''
> if self.is_running():
> try:
> self._qmp.cmd('quit')
> self._qmp.close()
> except:
> self._popen.kill()
> self._popen.wait()
>
> def _launch(self):
> '''Launch the VM and establish a QMP connection'''
> devnull = open('/dev/null', 'rb')
> qemulog = open(self._qemu_log_path, 'wb')
> self._shutdown_pending = True
> self._pre_launch()
> args = self._wrapper + [self._binary] + self._base_args() + self._args
> self._popen = subprocess.Popen(args, stdin=devnull, stdout=qemulog,
>stderr=subprocess.STDOUT, 

Re: [Qemu-devel] [RFC PATCH for 2.11 00/23] Implementing FP16 for ARMv8.2 using SoftFloat2a and 3c

[Aurelien Jarno]

On 2017-07-20 16:04, Alex Bennée wrote:
> I then implement a few random functions using the softfloat3c code. To
> keep things clean the helper functions are all in advsimd_helper.c. As
> the two implementations are sitting side by side we need to copy the
> exception flags back and forth to ensure the virtual FPU state is
> correct between operations implemented by both libraries. This can
> obviously be removed is an architecture fully transitions to a later
> library.

softfloat3a uses a global FPU state. As it is thread local, that should
not be a problem as long as we have one thread per VCPU (which is the
case with MTTCG). However, how do you plan to handle the switch from one
FPU state to another for the architectures using multiple FPU contexts?
That concerns at least arm, i386, mips and ppc.

Regards,
Aurelien

-- 
Aurelien Jarno  GPG: 4096R/1DDD8C9B
aurel...@aurel32.net http://www.aurel32.net

Re: [Qemu-devel] [RFC PATCH for 2.11 15/23] target/arm/translate-a64.c: AdvSIMD scalar 2 register misc decode

[Richard Henderson]

On 07/20/2017 05:04 AM, Alex Bennée wrote:

+for (pass = 0; pass < elements; pass++) {
+TCGv_i32 tcg_op1 = tcg_temp_new_i32();
+TCGv_i32 tcg_res = tcg_temp_new_i32();
+
+read_vec_element_i32(s, tcg_op1, rn, pass, MO_16);
+
+switch (fpopcode) {
+default:
+fprintf(stderr,"%s: insn %#04x fpop %#2x\n", __func__, insn, 
fpopcode);
+unsupported_encoding(s, insn);
+}


Just noticing that we ought not issue N copies of the unsupported_encoding 
exception.  Return afterwards seems in order.  Is this a pattern that you 
copied that wants fixing elsewhere?



r~

Re: [Qemu-devel] [RFC PATCH for 2.11 14/23] target/arm/translate-a64.c: add ARMv8.2 fadd scalar half-precision

[Richard Henderson]

On 07/20/2017 05:04 AM, Alex Bennée wrote:

This brings in the initial decode skeleton and the helpers for a
scalar half-precision fadd using SoftFloat3c.

Signed-off-by: Alex Bennée
---
  target/arm/advsimd_helper.c | 15 
  target/arm/translate-a64.c  | 60 +
  2 files changed, 75 insertions(+)


... and there it is.  Squash this with previous?  Anyway, do something so that 
they're bisectable.



r~

Re: [Qemu-devel] [RFC PATCH for 2.11 13/23] target/arm/translate-a64.c: add FP16 FADD to AdvSIMD 3 Same

[Richard Henderson]

On 07/20/2017 05:04 AM, Alex Bennée wrote:

Signed-off-by: Alex Bennée
---
  target/arm/helper-a64.h| 1 +
  target/arm/translate-a64.c | 3 +++
  2 files changed, 4 insertions(+)


Implementation of the helper wound up somewhere else.


r~

Re: [Qemu-devel] [RFC PATCH for 2.11 12/23] target/arm/translate-a64.c: add FP16 FAGCT to AdvSIMD 3 Same

[Richard Henderson]

On 07/20/2017 05:04 AM, Alex Bennée wrote:

+static softfloat_flags softfloat_mapping_table[] = {
+{ float_flag_inexact  , softfloat_flag_inexact },
+{ float_flag_underflow, softfloat_flag_underflow },
+{ float_flag_overflow , softfloat_flag_overflow },
+{ float_flag_invalid  , softfloat_flag_invalid },
+};
+/* FIXME: 2a has no infinite flag */
+
+static uint8_t sync_softfloat_flags_from_2a(float_status *flags2a)
+{
+uint8_t flags = flags2a->float_exception_flags;
+int i;
+if (flags) {
+for (i = 0; i < ARRAY_SIZE(softfloat_mapping_table); i++) {
+struct softfloat_flags *map = &softfloat_mapping_table[i];
+if (flags & map->float2a_flag) {
+softfloat_raiseFlags(map->float3c_flag);
+}
+}
+} else {
+softfloat_exceptionFlags = 0;
+}
+
+return softfloat_exceptionFlags;
+}
+


For conversions like this IMO it's better to make the compiler do the work. 
C.f. target/alpha/fpu_helper.c and CONVERT_BIT.


BTW, I think these TLS variables that softfloat3a are using are going to be a 
real problem.  It's one thing to do it temporarily like you are here, 
coordinating between 2a and 3c, but later, when the front end is fully 
converted?  That's just nonsense.


Is it going to be worthwhile to significantly hack up the incoming sources?

If so, then we might do something like this:  Given a 3c function foo,

  T foo_st(T, T, float3a_status *) { ... }
  T foo(T x, T y) { return foo_st(x, y, &tls_status); }

And of course pack all of the relevant state into a struct then

  #define softfloat_exceptionFlags tls_status.exceptionflags

etc, instead of having individual tls variables.  This feels like something 
that we could push back upstream, assuming that upstream is active.



r~

Re: [Qemu-devel] Build failure on git master

[Mark Cave-Ayland]

On 19/07/17 18:09, Mark Cave-Ayland wrote:

> Hi all,
> 
> I see the following build failure here on git master with gcc 4.7.2:
> 
> cc -I/home/build/src/qemu/git/qemu/block -Iblock
> -I/home/build/src/qemu/git/qemu/tcg
> -I/home/build/src/qemu/git/qemu/tcg/i386
> -I/home/build/src/qemu/git/qemu/linux-headers
> -I/home/build/src/qemu/git/qemu/linux-headers -I.
> -I/home/build/src/qemu/git/qemu
> -I/home/build/src/qemu/git/qemu/accel/tcg
> -I/home/build/src/qemu/git/qemu/include -I/usr/include/pixman-1
> -I/home/build/src/qemu/git/qemu/dtc/libfdt -Werror -pthread
> -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include
> -m64 -mcx16 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE
> -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings
> -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv
> -Wendif-labels -Wno-missing-include-dirs -Wempty-body -Wnested-externs
> -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers
> -Wold-style-declaration -Wold-style-definition -Wtype-limits
> -fstack-protector-all -I/usr/include/p11-kit-1
> -I/usr/include/libpng12   -I/home/build/src/qemu/git/qemu/tests -MMD -MP
> -MT block/vpc.o -MF block/vpc.d -O2 -U_FORTIFY_SOURCE
> -D_FORTIFY_SOURCE=2 -g   -c -o block/vpc.o block/vpc.c
> block/vpc.c: In function ‘vpc_co_pwritev’:
> block/vpc.c:652:9: error: ‘ret’ may be used uninitialized in this
> function [-Werror=maybe-uninitialized]
> cc1: all warnings being treated as errors
> make: *** [block/vpc.o] Error 1
> 
> At a quick glance it looks as if this is being caused by commit cfc87e00
> "block/vpc.c: Handle write failures in get_image_offset()".

The fix is seemingly quite simple:

diff --git a/block/vpc.c b/block/vpc.c
index 10e6519..574879b 100644
--- a/block/vpc.c
+++ b/block/vpc.c
@@ -649,7 +649,7 @@ vpc_co_pwritev(BlockDriverState *bs, uint64_t
offset, uint64_t bytes,
 int64_t image_offset;
 int64_t n_bytes;
 int64_t bytes_done = 0;
-int ret;
+int ret = 0;
 VHDFooter *footer =  (VHDFooter *) s->footer_buf;
 QEMUIOVector local_qiov;


ATB,

Mark.

Re: [Qemu-devel] Commit 77af8a2b95b79699de650965d5228772743efe84 breaks Windows 2000 support

[Phil Dennis-Jordan]

On Thu, Jul 20, 2017 at 6:40 PM, Programmingkid
 wrote:
> I noticed that Windows 2000 does not boot up in QEMU recently. After 
> bisecting the issue I found the offending commit:

Ouch. I reckon we have 2 options for fixing this:

1. Export two FADTs, one ACPI 1.0, one ACPI 2.0. The latter would need
to be pointed to by an XSDT, which Qemu currently doesn't implement at
all as far as I'm aware. Any ideas on how SeaBIOS or OVMF would handle
this? Any likely other OS regressions?

2. Select FADT version with an option. This one is definitely safe,
but adds yet another option.

Thoughts?


> commit 77af8a2b95b79699de650965d5228772743efe84
> Author: Phil Dennis-Jordan 
> Date:   Wed Mar 15 19:20:26 2017 +1300
>
> hw/i386: Use Rev3 FADT (ACPI 2.0) instead of Rev1 to improve guest OS 
> support.
>
> This updates the FADT generated for x86/64 machine types from Revision 1 
> to 3. (Based on ACPI standard 2.0 instead of 1.0) The intention is to expose 
> the reset register information to guest operating systems which require it, 
> specifically OS X/macOS. Revision 1 FADTs do not contain the fields relating 
> to the reset register.
>
> The new layout and contents remains backwards-compatible with operating 
> systems which only support ACPI 1.0, as the existing fields are not modified 
> by this change, as the 64-bit and 32-bit variants are allowed to co-exist 
> according to the ACPI 2.0 standard. No regressions became apparent in tests 
> with a range of Windows (XP-10) and Linux versions.
>
> The BIOS tables test suite's FADT checksum test has also been updated to 
> reflect the new FADT layout and content.
>
> Signed-off-by: Phil Dennis-Jordan 
> Message-Id: <1489558827-28971-2-git-send-email-p...@philjordan.eu>
> Signed-off-by: Paolo Bonzini 
>
> :04 04 40063761c0b86f87e798e03ea48eff9ea0753425 
> 6d2a94150cf1eafb16f0ccf6325281415fef64a6 M  hw
> :04 04 fe3f1480a91b76fea238c765f0725e715932d96d 
> 68f9368d8d78fd3267f609b603f97e8a74bdf528 M  include
> :04 04 895e961b0a160100aa95b2f557cfe6b87a7d9bff 
> 8ed08cef10fddee7814e38ad62be11371592a75a M  tests
>
>

Re: [Qemu-devel] [PATCH V4 08/10] block/qcow2: start using the compress format extension

[Eric Blake]

On 07/20/2017 11:30 AM, Peter Lieven wrote:

>> The new code is now unconditionally initializing with -15 instead of
>> -12.  Does that matter, or does decompression work regardless of window
>> size used at creation, as long as the initialized size at decompression
>> is at least as large?  On the other hand, I guess that means if someone
>> compresses with a large window, and then I initialize the decompressor
>> with a small window, my decompression will fail?  That's why knowing the
>> minimum window size should be part of the spec, whether or not we make
>> it a tunable.
> 
> The decompression is supposed to fail if you compress with 15 and
> decompress with 12. In fact it doesn't.

Actually, I think (this is my guess here, not actual researched fact)
that the decompression error is possible ONLY if compression produced a
symbol that actually required more than 12 bits of memory - to get that
large of a symbol, you need to compress a lot of bits.  For our default
cluster of 64k, it might very well be that you rarely, if ever,
encounter a single cluster that compresses differently under window size
15 than it did under window size 12 (other than perhaps the speed at
which compression took place), because there simply wasn't enough
content to reach the point where you needed a symbol in the compression
stream using more than 12 bits.  So in that case, compressing under 15
and decompressing under 12 doesn't hit the error.  But as you get larger
cluster sizes (2M clusters), or perhaps if you pass particularly nasty
sequences of input to compression (I'm not sure what sequences would
have the right properties), then you do indeed result in a compression
stream that starts to encounter symbols exceeding the window size.

But if my guess is right, then don't read the docs as "decompression
will fail", but rather as "decompression may fail" if you set the
decompress window smaller than the compression window.

> I would like to avoid the windowBits in the qcow2 header as it makes
> 
> the code to read and write it more complicated. If you don't like the change
> 
> of the windowBits we can even stick with 12. If someone wants fast compression
> 
> he will likely not use zlib at all and use lzo.

Also, note that historically, 'compress -b N' has allowed tuning the
window size; current POSIX states that compress only has to support
windows from 9 to 14, but permits implementations to use up to 16 (and
future POSIX is considering improving the compress utility to require
support for 16 as the window size, https://posix.rhansen.org/p/bug1041).
 I don't know why gzip didn't expose a '-b N' windowsize parameter the
way compress did, but it sounds like the same thing.

> 
> I just changed the windowBits to 15 as it increases speed and improves 
> compression.

Does windowBits 16 make any difference?

> 
> (likely at the cost of memory during compression/decompression)
> 
> 
> Peter
> 
> 

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.   +1-919-301-3266
Virtualization:  qemu.org | libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [RFC PATCH for 2.11 05/23] softfloat3c: initial build machinery

[Richard Henderson]

On 07/20/2017 05:04 AM, Alex Bennée wrote:

+# so they can still be linked when needed. We build these files surpressing so 
of the normal CFLAGS.


"surpressing so" -> "suppressing some"

Do we gain any confidence for our still supported but less tested 32-bit hosts 
(all of which do support a 64-bit type) by dropping the FAST_INT64 distinction?



-#ifdef __GNUC_STDC_INLINE__
-#define INLINE inline
-#else
-#define INLINE extern inline


Why are you removing this?



r~

Re: [Qemu-devel] [PATCH for-2.10] Use qemu_tolower() and qemu_toupper(), not tolower() and toupper()

[Richard Henderson]

On 07/20/2017 09:04 AM, Eric Blake wrote:

And I found this in C11:

7.4 Character handling 
1 The header  declares several functions useful for classifying
and mapping
characters. 198) In all cases the argument is an int, the value of which
shall be
representable as an unsigned char or shall equal the value of the macro
EOF. If the
argument has any other value, the behavior is undefined.



Thanks.  I should have looked up there.


r~

[Qemu-devel] [PATCH v2 4/5] Raw Backend for UDST

[anton . ivanov]

From: Anton Ivanov 

Raw Socket Backend for Universal Datagram Socket Transport

Signed-off-by: Anton Ivanov 
---
 net/Makefile.objs |   2 +-
 net/clients.h |   3 ++
 net/net.c |   1 +
 net/raw.c | 123 ++
 qapi-schema.json  |  20 -
 qemu-options.hx   |  32 ++
 6 files changed, 178 insertions(+), 3 deletions(-)
 create mode 100644 net/raw.c

diff --git a/net/Makefile.objs b/net/Makefile.objs
index 919bc3d78f..457297b5ed 100644
--- a/net/Makefile.objs
+++ b/net/Makefile.objs
@@ -2,7 +2,7 @@ common-obj-y = net.o queue.o checksum.o util.o hub.o
 common-obj-y += socket.o
 common-obj-y += dump.o
 common-obj-y += eth.o
-common-obj-$(CONFIG_UDST) += udst.o l2tpv3.o gre.o
+common-obj-$(CONFIG_UDST) += udst.o l2tpv3.o gre.o raw.o
 common-obj-$(CONFIG_POSIX) += vhost-user.o
 common-obj-$(CONFIG_SLIRP) += slirp.o
 common-obj-$(CONFIG_VDE) += vde.o
diff --git a/net/clients.h b/net/clients.h
index 8f8a59aee3..98d8ae59b7 100644
--- a/net/clients.h
+++ b/net/clients.h
@@ -53,6 +53,9 @@ int net_init_l2tpv3(const Netdev *netdev, const char *name,
 int net_init_gre(const Netdev *netdev, const char *name,
 NetClientState *peer, Error **errp);
 
+int net_init_raw(const Netdev *netdev, const char *name,
+NetClientState *peer, Error **errp);
+
 #ifdef CONFIG_VDE
 int net_init_vde(const Netdev *netdev, const char *name,
  NetClientState *peer, Error **errp);
diff --git a/net/net.c b/net/net.c
index 6163a8a3af..8eb0aa2bee 100644
--- a/net/net.c
+++ b/net/net.c
@@ -963,6 +963,7 @@ static int (* const 
net_client_init_fun[NET_CLIENT_DRIVER__MAX])(
 #ifdef CONFIG_UDST
 [NET_CLIENT_DRIVER_L2TPV3] = net_init_l2tpv3,
 [NET_CLIENT_DRIVER_GRE] = net_init_gre,
+[NET_CLIENT_DRIVER_RAW] = net_init_raw,
 #endif
 };
 
diff --git a/net/raw.c b/net/raw.c
new file mode 100644
index 00..8f73248095
--- /dev/null
+++ b/net/raw.c
@@ -0,0 +1,123 @@
+/*
+ * QEMU System Emulator
+ *
+ * Copyright (c) 2015-2017 Cambridge Greys Limited
+ * Copyright (c) 2003-2008 Fabrice Bellard
+ * Copyright (c) 2012-2014 Cisco Systems
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to 
deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 
FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#include "qemu/osdep.h"
+#include 
+#include 
+#include 
+#include 
+#include "net/net.h"
+#include 
+#include 
+#include 
+#include "clients.h"
+#include "qemu-common.h"
+#include "qemu/error-report.h"
+#include "qapi/error.h"
+#include "qemu/option.h"
+#include "qemu/sockets.h"
+#include "qemu/iov.h"
+#include "qemu/main-loop.h"
+#include "udst.h"
+
+static int noop(void *us, uint8_t *buf)
+{
+return 0;
+}
+
+int net_init_raw(const Netdev *netdev,
+const char *name,
+NetClientState *peer, Error **errp)
+{
+
+const NetdevRawOptions *raw;
+NetUdstState *s;
+NetClientState *nc;
+
+int fd = -1;
+int err;
+
+struct ifreq ifr;
+struct sockaddr_ll sock;
+
+
+nc = qemu_new_udst_net_client(name, peer);
+
+s = DO_UPCAST(NetUdstState, nc, nc);
+
+fd = socket(AF_PACKET, SOCK_RAW, htons(ETH_P_ALL));
+if (fd == -1) {
+err = -errno;
+error_report("raw_open : raw socket creation failed, errno = %d", 
-err);
+goto outerr;
+}
+
+
+s->dgram_dst = NULL;
+s->dst_size = 0;
+
+assert(netdev->type == NET_CLIENT_DRIVER_RAW);
+raw = &netdev->u.raw;
+
+memset(&ifr, 0, sizeof(struct ifreq));
+strncpy((char *) &ifr.ifr_name, raw->ifname, sizeof(ifr.ifr_name) - 1);
+
+if (ioctl(fd, SIOCGIFINDEX, (void *) &ifr) < 0) {
+err = -errno;
+error_report("SIOCGIFINDEX, failed to get raw interface index for %s",
+raw->ifname);
+goto outerr;
+}
+
+sock.sll_family = AF_PACKET;
+sock.sll_protocol = htons(ETH_P_ALL);
+sock.sll_ifindex = ifr.ifr_ifindex;
+
+if (bind(fd, (struct sockaddr *) &sock, sizeof(struct sockaddr_ll)) < 0

[Qemu-devel] [PATCH v2 5/5] Migrate Datagram operation in socket transport to UDST

[anton . ivanov]

From: Anton Ivanov 

Migrate datagram operation to UDST if UDST is available.

Signed-off-by: Anton Ivanov 
---
 net/socket.c | 123 ---
 1 file changed, 118 insertions(+), 5 deletions(-)

diff --git a/net/socket.c b/net/socket.c
index f85ef7d61b..0523fe9ac1 100644
--- a/net/socket.c
+++ b/net/socket.c
@@ -33,6 +33,9 @@
 #include "qemu/sockets.h"
 #include "qemu/iov.h"
 #include "qemu/main-loop.h"
+#ifdef CONFIG_UDST
+#include "udst.h"
+#endif
 
 typedef struct NetSocketState {
 NetClientState nc;
@@ -49,6 +52,14 @@ typedef struct NetSocketState {
 static void net_socket_accept(void *opaque);
 static void net_socket_writable(void *opaque);
 
+#ifdef CONFIG_UDST
+static int noop(void *us, uint8_t *buf)
+{
+return 0;
+}
+#endif
+
+
 static void net_socket_update_fd_handler(NetSocketState *s)
 {
 qemu_set_fd_handler(s->fd,
@@ -113,6 +124,8 @@ static ssize_t net_socket_receive(NetClientState *nc, const 
uint8_t *buf, size_t
 return size;
 }
 
+#ifndef CONFIG_UDST
+
 static ssize_t net_socket_receive_dgram(NetClientState *nc, const uint8_t 
*buf, size_t size)
 {
 NetSocketState *s = DO_UPCAST(NetSocketState, nc, nc);
@@ -130,6 +143,7 @@ static ssize_t net_socket_receive_dgram(NetClientState *nc, 
const uint8_t *buf,
 }
 return ret;
 }
+#endif
 
 static void net_socket_send_completed(NetClientState *nc, ssize_t len)
 {
@@ -189,6 +203,8 @@ static void net_socket_send(void *opaque)
 }
 }
 
+#ifndef CONFIG_UDST
+
 static void net_socket_send_dgram(void *opaque)
 {
 NetSocketState *s = opaque;
@@ -208,6 +224,7 @@ static void net_socket_send_dgram(void *opaque)
 net_socket_read_poll(s, false);
 }
 }
+#endif
 
 static int net_socket_mcast_create(struct sockaddr_in *mcastaddr, struct 
in_addr *localaddr)
 {
@@ -309,7 +326,85 @@ static void net_socket_cleanup(NetClientState *nc)
 s->listen_fd = -1;
 }
 }
+#ifdef CONFIG_UDST
+static NetUdstState *net_socket_fd_init_dgram(NetClientState *peer,
+const char *model,
+const char *name,
+int fd, int is_connected)
+{
+struct sockaddr_in saddr;
+int newfd;
+socklen_t saddr_len = sizeof(saddr);
+NetClientState *nc;
+NetUdstState *s;
 
+/* fd passed: multicast: "learn" dgram_dst address from bound address and 
save it
+ * Because this may be "shared" socket from a "master" process, datagrams 
would be recv()
+ * by ONLY ONE process: we must "clone" this dgram socket --jjo
+ */
+
+if (is_connected) {
+if (getsockname(fd, (struct sockaddr *) &saddr, &saddr_len) == 0) {
+/* must be bound */
+if (saddr.sin_addr.s_addr == 0) {
+fprintf(stderr, "qemu: error: init_dgram: fd=%d unbound, "
+"cannot setup multicast dst addr\n", fd);
+goto err;
+}
+/* clone dgram socket */
+newfd = net_socket_mcast_create(&saddr, NULL);
+if (newfd < 0) {
+/* error already reported by net_socket_mcast_create() */
+goto err;
+}
+/* clone newfd to fd, close newfd */
+dup2(newfd, fd);
+close(newfd);
+
+} else {
+fprintf(stderr,
+"qemu: error: init_dgram: fd=%d failed getsockname(): 
%s\n",
+fd, strerror(errno));
+goto err;
+}
+}
+
+fprintf(stderr,
+"qemu: init udst for fd=%d\n",
+fd);
+nc = qemu_new_udst_net_client(name, peer);
+
+s = DO_UPCAST(NetUdstState, nc, nc);
+
+s->offset = 0;
+
+qemu_net_finalize_udst_init(s,
+&noop,
+NULL,
+fd);
+
+/* mcast: save bound address as dst */
+if (is_connected) {
+s->dgram_dst = g_memdup(&saddr, sizeof(struct sockaddr_in));
+s->dst_size = sizeof(struct sockaddr_in);
+snprintf(nc->info_str, sizeof(nc->info_str),
+ "socket: fd=%d (cloned mcast=%s:%d)",
+ fd, inet_ntoa(saddr.sin_addr), ntohs(saddr.sin_port));
+} else {
+/* This will be overwritten later if we have a dst */
+s->dgram_dst = NULL;
+s->dst_size = 0;
+snprintf(nc->info_str, sizeof(nc->info_str),
+ "socket: fd=%d", fd);
+}
+
+return s;
+
+err:
+closesocket(fd);
+return NULL;
+}
+#else
 static NetClientInfo net_dgram_socket_info = {
 .type = NET_CLIENT_DRIVER_SOCKET,
 .size = sizeof(NetSocketState),
@@ -386,6 +481,7 @@ err:
 closesocket(fd);
 return NULL;
 }
+#endif
 
 static void net_socket_connect(void *opaque)
 {
@@ -430,7 +526,7 @@ static NetSocketState 
*net_socket_fd_init_stream(NetClientState *peer,
 return s;
 }
 
-static NetSocketState *net_socket_fd_init(NetClientState *peer,
+static void *net_socket_fd_i