Re: [PATCH v3 1/2] meson: convert pc-bios/keymaps/Makefile
I think it should just add build_by_default: false with a comment
explaining why.
Paolo
Il lun 24 ago 2020, 07:23 Gerd Hoffmann ha scritto:
> > --- a/pc-bios/meson.build
> > +++ b/pc-bios/meson.build
> > @@ -25,3 +25,4 @@ if 'DECOMPRESS_EDK2_BLOBS' in config_host
> > endif
> >
> > subdir('descriptors')
> > +subdir('keymaps')
>
> Hmm, this hooks up the keymaps update to the default build.
> Not a good plan. The output is not static, but defaults on
> the libxkbcommon version you have installed. So I end up with
> a dirty tree now after each build.
>
> I guess we should either return to the traditional behavior of not
> building keymaps by default and have a "make keymaps" or similar for
> keymap updates. Or write the generated maps to the build tree not
> the source tree.
>
> take care,
> Gerd
>
>
Re: [PATCH v3 1/2] meson: convert pc-bios/keymaps/Makefile
> --- a/pc-bios/meson.build
> +++ b/pc-bios/meson.build
> @@ -25,3 +25,4 @@ if 'DECOMPRESS_EDK2_BLOBS' in config_host
> endif
>
> subdir('descriptors')
> +subdir('keymaps')
Hmm, this hooks up the keymaps update to the default build.
Not a good plan. The output is not static, but defaults on
the libxkbcommon version you have installed. So I end up with
a dirty tree now after each build.
I guess we should either return to the traditional behavior of not
building keymaps by default and have a "make keymaps" or similar for
keymap updates. Or write the generated maps to the build tree not
the source tree.
take care,
Gerd
Re: [PATCH v6 2/4] copy-on-read: add filter append/drop functions
On 23.08.2020 22:35, Andrey Shinkevich wrote:
On 19.08.2020 13:21, Vladimir Sementsov-Ogievskiy wrote:
19.08.2020 00:24, Andrey Shinkevich wrote:
Provide API for the COR-filter insertion/removal.
Also, drop the filter child permissions for an inactive state when the
filter node is being removed.
Signed-off-by: Andrey Shinkevich
---
block/copy-on-read.c | 103
+++
block/copy-on-read.h | 36 ++
2 files changed, 139 insertions(+)
create mode 100644 block/copy-on-read.h
diff --git a/block/copy-on-read.c b/block/copy-on-read.c
index cb03e0f..150d9b7 100644
--- a/block/copy-on-read.c
+++ b/block/copy-on-read.c
@@ -23,11 +23,21 @@
...
+BlockDriverState *bdrv_cor_filter_append(BlockDriverState *bs,
+ const char *filter_node_name,
+ Error **errp)
+{
+ BlockDriverState *cor_filter_bs;
+ BDRVStateCOR *state;
+ Error *local_err = NULL;
+
+ cor_filter_bs = create_filter_node(bs, filter_node_name, errp);
+ if (cor_filter_bs == NULL) {
+ error_prepend(errp, "Could not create filter node: ");
+ return NULL;
+ }
+
+ if (!filter_node_name) {
+ cor_filter_bs->implicit = true;
+ }
+
+ bdrv_drained_begin(bs);
+ bdrv_replace_node(bs, cor_filter_bs, &local_err);
+ bdrv_drained_end(bs);
+
+ if (local_err) {
+ bdrv_unref(cor_filter_bs);
+ error_propagate(errp, local_err);
+ return NULL;
+ }
+
+ state = cor_filter_bs->opaque;
+ state->active = true;
hmm stop. it's already active, after create_filter_node, so you don't
need to set it again, isn't it?
I will remove the extra assignment from the cor_open() for consistancy.
Andrey
No, I won't. It is wrong to do that. COR-operations wouldn't work.
Andrey
+
+ return cor_filter_bs;
+}
+
...
unknown fs driver type 'virtiofs'
Folks The instructions posted on http://blog.vmsplice.net/2020/04/virtio-fs-has-landed-in-qemu-50.html are simple and I followed them. I've updated my Debian Buster QEMU and LIBVIRT packages to all the newest buster-backports versions, and am running on: Linux ps01ubx 5.7.0-0.bpo.2-amd64 #1 SMP Debian 5.7.10-1~bpo10+1 (2020-07-30) x86_64 GNU/Linux lsmod|grep virtio virtiofs 32768 0 virtio_ring 36864 1 virtiofs virtio 16384 1 virtiofs fuse 139264 4 virtiofs dpkg -l gir1.2-libvirt-glib-1.0:amd64 ipxe-qemu libvirglrenderer0:amd64 libvirt-clients libvirt-daemon libvirt-daemon-system libvirt-glib-1.0-0:amd64 libvirt0:amd64 python3-libvirt qemu qemu-block-extra qemu-system-common qemu-system-data qemu-system-gui:amd64 qemu-system-x86 qemu-utils Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Desc +++-=--- ii gir1.2-libvirt-glib-1.0:amd64 1.0.0-1 amd64 GObj ii ipxe-qemu 1.0.0+git-20190125.36a4c85-1 all PXE ii libvirglrenderer0:amd64 0.7.0-2 amd64 virt ii libvirt-clients 5.0.0-4+deb10u1 amd64 Prog ii libvirt-daemon 5.0.0-4+deb10u1 amd64 Virt ii libvirt-daemon-system 5.0.0-4+deb10u1 amd64 Libv ii libvirt-glib-1.0-0:amd64 1.0.0-1 amd64 libv ii libvirt0:amd64 5.0.0-4+deb10u1 amd64 libr ii python3-libvirt 5.0.0-1 amd64 libv ii qemu 1:5.0-14~bpo10+1 amd64 fast ii qemu-block-extra 1:5.0-14~bpo10+1 amd64 extr ii qemu-system-common 1:5.0-14~bpo10+1 amd64 QEMU ii qemu-system-data 1:5.0-14~bpo10+1 all QEMU ii qemu-system-gui:amd64 1:5.0-14~bpo10+1 amd64 QEMU ii qemu-system-x86 1:5.0-14~bpo10+1 amd64 QEMU ii qemu-utils 1:5.0-14~bpo10+1 amd64 QEMU Whenever I add a filesystem to the vm xml (virsh edit vm) virt-manager spits out an error message. Because virt-manager doesn't have a "virtiofs" line in the drop-down choices I first choose "PATH" and then try to change the XML. The result is similar, whether I use virt-manager or virsh edit vm: virtio-manager replies: unknown fs driver type 'virtiofs' virsh edit vm does not let me save the file. It replies: error: XML document failed to validate against schema: Unable to validate doc against /usr/share/libvirt/schemas/domain.rng Extra element devices in interleave Element domain failed to validate content Failed. Try again? [y,n,i,f,?]: What am I doing wrong? Regards Pedro Serrano
[PATCH 2/2] coroutine: take exactly one batch from global pool at a time
This patch replace the global coroutine queue with a lock-free stack of which
the elements are coroutine queues. Threads can put coroutine queues into the
stack or take queues from it and each coroutine queue has exactly
POOL_BATCH_SIZE coroutines. Note that the stack is not strictly LIFO, but it's
enough for buffer pool.
Coroutines will be put into thread-local pools first while release. Now the
fast pathes of both allocation and release are atomic-free, and there won't
be too many coroutines remain in a single thread since POOL_BATCH_SIZE has been
reduced to 16.
In practice, I've run a VM with two block devices binding to two different
iothreads, and run fio with iodepth 128 on each device. It maintains around
400 coroutines and has about 1% chance of calling to `qemu_coroutine_new`
without this patch. And with this patch, it maintains no more than 273
coroutines and doesn't call `qemu_coroutine_new` after initial allocations.
Signed-off-by: wanghonghao
---
util/qemu-coroutine.c | 63 ---
1 file changed, 42 insertions(+), 21 deletions(-)
diff --git a/util/qemu-coroutine.c b/util/qemu-coroutine.c
index c3caa6c770..070d492edc 100644
--- a/util/qemu-coroutine.c
+++ b/util/qemu-coroutine.c
@@ -21,13 +21,14 @@
#include "block/aio.h"
enum {
-POOL_BATCH_SIZE = 64,
+POOL_BATCH_SIZE = 16,
+POOL_MAX_BATCHES = 32,
};
-/** Free list to speed up creation */
-static QSLIST_HEAD(, Coroutine) release_pool = QSLIST_HEAD_INITIALIZER(pool);
-static unsigned int release_pool_size;
-static __thread QSLIST_HEAD(, Coroutine) alloc_pool =
QSLIST_HEAD_INITIALIZER(pool);
+/** Free stack to speed up creation */
+static QSLIST_HEAD(, Coroutine) pool[POOL_MAX_BATCHES];
+static int pool_top;
+static __thread QSLIST_HEAD(, Coroutine) alloc_pool;
static __thread unsigned int alloc_pool_size;
static __thread Notifier coroutine_pool_cleanup_notifier;
@@ -49,20 +50,26 @@ Coroutine *qemu_coroutine_create(CoroutineEntry *entry,
void *opaque)
if (CONFIG_COROUTINE_POOL) {
co = QSLIST_FIRST(&alloc_pool);
if (!co) {
-if (release_pool_size > POOL_BATCH_SIZE) {
-/* Slow path; a good place to register the destructor, too. */
-if (!coroutine_pool_cleanup_notifier.notify) {
-coroutine_pool_cleanup_notifier.notify =
coroutine_pool_cleanup;
-qemu_thread_atexit_add(&coroutine_pool_cleanup_notifier);
+int top;
+
+/* Slow path; a good place to register the destructor, too. */
+if (!coroutine_pool_cleanup_notifier.notify) {
+coroutine_pool_cleanup_notifier.notify =
coroutine_pool_cleanup;
+qemu_thread_atexit_add(&coroutine_pool_cleanup_notifier);
+}
+
+while ((top = atomic_read(&pool_top)) > 0) {
+if (atomic_cmpxchg(&pool_top, top, top - 1) != top) {
+continue;
}
-/* This is not exact; there could be a little skew between
- * release_pool_size and the actual size of release_pool. But
- * it is just a heuristic, it does not need to be perfect.
- */
-alloc_pool_size = atomic_xchg(&release_pool_size, 0);
-QSLIST_MOVE_ATOMIC(&alloc_pool, &release_pool);
+QSLIST_MOVE_ATOMIC(&alloc_pool, &pool[top - 1]);
co = QSLIST_FIRST(&alloc_pool);
+
+if (co) {
+alloc_pool_size = POOL_BATCH_SIZE;
+break;
+}
}
}
if (co) {
@@ -86,16 +93,30 @@ static void coroutine_delete(Coroutine *co)
co->caller = NULL;
if (CONFIG_COROUTINE_POOL) {
-if (release_pool_size < POOL_BATCH_SIZE * 2) {
-QSLIST_INSERT_HEAD_ATOMIC(&release_pool, co, pool_next);
-atomic_inc(&release_pool_size);
-return;
-}
+int top, value, old;
+
if (alloc_pool_size < POOL_BATCH_SIZE) {
QSLIST_INSERT_HEAD(&alloc_pool, co, pool_next);
alloc_pool_size++;
return;
}
+
+for (top = atomic_read(&pool_top); top < POOL_MAX_BATCHES; top++) {
+QSLIST_REPLACE_ATOMIC(&pool[top], &alloc_pool);
+if (!QSLIST_EMPTY(&alloc_pool)) {
+continue;
+}
+
+value = top + 1;
+
+do {
+old = atomic_cmpxchg(&pool_top, top, value);
+} while (old != top && (top = old) < value);
+
+QSLIST_INSERT_HEAD(&alloc_pool, co, pool_next);
+alloc_pool_size = 1;
+return;
+}
}
qemu_coroutine_delete(co);
--
2.24.3 (Apple Git-128)
[PATCH 1/2] QSLIST: add atomic replace operation
Replace a queue with another atomicly. It's useful when we need to transfer
queues between threads.
Signed-off-by: wanghonghao
---
include/qemu/queue.h | 4
1 file changed, 4 insertions(+)
diff --git a/include/qemu/queue.h b/include/qemu/queue.h
index 456a5b01ee..a3ff544193 100644
--- a/include/qemu/queue.h
+++ b/include/qemu/queue.h
@@ -226,6 +226,10 @@ struct {
\
(dest)->slh_first = atomic_xchg(&(src)->slh_first, NULL);\
} while (/*CONSTCOND*/0)
+#define QSLIST_REPLACE_ATOMIC(dest, src) do { \
+(src)->slh_first = atomic_xchg(&(dest)->slh_first, (src)->slh_first); \
+} while (/*CONSTCOND*/0)
+
#define QSLIST_REMOVE_HEAD(head, field) do { \
typeof((head)->slh_first) elm = (head)->slh_first; \
(head)->slh_first = elm->field.sle_next; \
--
2.24.3 (Apple Git-128)
RE: [PATCH 07/18] hw/sd: sd: Fix incorrect populated function switch status data structure
Hi Bin, > -Original Message- > From: Bin Meng > Sent: Friday, August 21, 2020 3:38 PM > To: Sai Pavan Boddu > Cc: Philippe Mathieu-Daudé ; Alistair Francis > ; Bastian Koppelmann paderborn.de>; Palmer Dabbelt ; Sagar > Karandikar ; qemu-devel@nongnu.org; qemu- > ri...@nongnu.org; Bin Meng ; qemu- > bl...@nongnu.org; Sai Pavan Boddu > Subject: Re: [PATCH 07/18] hw/sd: sd: Fix incorrect populated function switch > status data structure > > Hi Sai, > > On Fri, Aug 21, 2020 at 6:04 PM Sai Pavan Boddu > wrote: > > > > Hi Philippe, > > > > First two patch of SD look good. Tested them over zynqmp and versal > > platforms. > > > > Thanks for testing. > > Can I add your Tested-by tag? [Sai Pavan Boddu] Sure. Thanks. Regards, Sai Pavan > > Regards, > Bin
[PATCH v4 08/12] migration/dirtyrate: skip sampling ramblock with size below MIN_RAMBLOCK_SIZE
In order to sample real RAM, skip ramblock with size below MIN_RAMBLOCK_SIZE
which is set as 128M.
Signed-off-by: Chuan Zheng
---
migration/dirtyrate.c | 24
migration/dirtyrate.h | 10 ++
2 files changed, 34 insertions(+)
diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c
index 050270d..bd398b7 100644
--- a/migration/dirtyrate.c
+++ b/migration/dirtyrate.c
@@ -173,6 +173,24 @@ alloc_ramblock_dirty_info(int *block_index,
return block_dinfo;
}
+static int skip_sample_ramblock(RAMBlock *block)
+{
+int64_t ramblock_size;
+
+/* ramblock size in MB */
+ramblock_size = qemu_ram_get_used_length(block) >> DIRTYRATE_PAGE_SHIFT_MB;
+
+/*
+ * Consider ramblock with size larger than 128M is what we
+ * want to sample.
+ */
+if (ramblock_size < MIN_RAMBLOCK_SIZE) {
+return -1;
+}
+
+return 0;
+}
+
static int record_ramblock_hash_info(struct RamblockDirtyInfo **block_dinfo,
struct DirtyRateConfig config,
int *block_index)
@@ -183,6 +201,9 @@ static int record_ramblock_hash_info(struct
RamblockDirtyInfo **block_dinfo,
int index = 0;
RAMBLOCK_FOREACH_MIGRATABLE(block) {
+if (skip_sample_ramblock(block) < 0) {
+continue;
+}
dinfo = alloc_ramblock_dirty_info(&index, dinfo);
if (dinfo == NULL) {
return -1;
@@ -249,6 +270,9 @@ static int compare_page_hash_info(struct RamblockDirtyInfo
*info,
RAMBlock *block = NULL;
RAMBLOCK_FOREACH_MIGRATABLE(block) {
+if (skip_sample_ramblock(block) < 0) {
+continue;
+}
block_dinfo = NULL;
if (!find_page_matched(block, info, block_index + 1, &block_dinfo)) {
continue;
diff --git a/migration/dirtyrate.h b/migration/dirtyrate.h
index 5050add..41bc264 100644
--- a/migration/dirtyrate.h
+++ b/migration/dirtyrate.h
@@ -35,10 +35,20 @@
#define DIRTYRATE_PAGE_SHIFT_KB 12
/*
+ * Sample page size MB shift
+ */
+#define DIRTYRATE_PAGE_SHIFT_MB 20
+
+/*
* Sample page size 1G shift
*/
#define DIRTYRATE_PAGE_SHIFT_GB 30
+/*
+ * minimum ramblock size to sampled
+ */
+#define MIN_RAMBLOCK_SIZE 128
+
/* Take 1s as default for calculation duration */
#define DEFAULT_FETCH_DIRTYRATE_TIME_SEC 1
--
1.8.3.1
[PATCH v4 12/12] migration/dirtyrate: Add trace_calls to make it easier to debug
Add trace_calls to make it easier to debug
Signed-off-by: Chuan Zheng
---
migration/dirtyrate.c | 7 +++
migration/trace-events | 8
2 files changed, 15 insertions(+)
diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c
index 08c46d3..3513ef3 100644
--- a/migration/dirtyrate.c
+++ b/migration/dirtyrate.c
@@ -23,6 +23,7 @@
#include "qapi/qapi-commands-migration.h"
#include "migration.h"
#include "ram.h"
+#include "trace.h"
#include "dirtyrate.h"
static int CalculatingState = DIRTY_RATE_STATUS_UNSTARTED;
@@ -55,6 +56,7 @@ static int64_t get_sample_page_period(int64_t sec)
static int dirtyrate_set_state(int *state, int old_state, int new_state)
{
assert(new_state < DIRTY_RATE_STATUS__MAX);
+trace_dirtyrate_set_state(DirtyRateStatus_str(new_state));
if (atomic_cmpxchg(state, old_state, new_state) == old_state) {
return 0;
} else {
@@ -78,6 +80,7 @@ static struct DirtyRateInfo *query_dirty_rate_info(void)
* Only support query once for each calculation,
* reset as DIRTY_RATE_STATUS_UNSTARTED after query
*/
+trace_query_dirty_rate_info(DirtyRateStatus_str(CalculatingState));
(void)dirtyrate_set_state(&CalculatingState, CalculatingState,
DIRTY_RATE_STATUS_UNSTARTED);
@@ -129,6 +132,7 @@ static uint32_t get_ramblock_vfn_hash(struct
RamblockDirtyInfo *info,
crc = crc32(0, iov_array.iov_base, iov_array.iov_len);
+trace_get_ramblock_vfn_hash(info->idstr, vfn, crc);
return crc;
}
@@ -246,6 +250,7 @@ static int skip_sample_ramblock(RAMBlock *block)
* want to sample.
*/
if (ramblock_size < MIN_RAMBLOCK_SIZE) {
+trace_skip_sample_ramblock(block->idstr, ramblock_size);
return -1;
}
@@ -292,6 +297,7 @@ static int calc_page_dirty_rate(struct RamblockDirtyInfo
*info)
for (i = 0; i < info->sample_pages_count; i++) {
crc = get_ramblock_vfn_hash(info, info->sample_page_vfn[i]);
if (crc != info->hash_result[i]) {
+trace_calc_page_dirty_rate(info->idstr, crc, info->hash_result[i]);
info->sample_dirty_count++;
}
}
@@ -317,6 +323,7 @@ static bool find_page_matched(RAMBlock *block, struct
RamblockDirtyInfo *infos,
if (infos[i].ramblock_addr != qemu_ram_get_host_addr(block) ||
infos[i].ramblock_pages !=
(qemu_ram_get_used_length(block) >> DIRTYRATE_PAGE_SHIFT_KB)) {
+trace_find_page_matched(block->idstr);
return false;
}
diff --git a/migration/trace-events b/migration/trace-events
index 4ab0a50..34569b9 100644
--- a/migration/trace-events
+++ b/migration/trace-events
@@ -312,3 +312,11 @@ dirty_bitmap_load_bits_zeroes(void) ""
dirty_bitmap_load_header(uint32_t flags) "flags 0x%x"
dirty_bitmap_load_enter(void) ""
dirty_bitmap_load_success(void) ""
+
+# dirtyrate.c
+dirtyrate_set_state(const char *new_state) "new state %s"
+query_dirty_rate_info(const char *new_state) "current state %s"
+get_ramblock_vfn_hash(const char *idstr, uint64_t vfn, uint32_t crc) "ramblock
name: %s, vfn: %"PRIu64 ", crc: %" PRIu32
+calc_page_dirty_rate(const char *idstr, uint32_t new_crc, uint32_t old_crc)
"ramblock name: %s, new crc: %" PRIu32 ", old crc: %" PRIu32
+skip_sample_ramblock(const char *idstr, int64_t ramblock_size) "ramblock name:
%s, ramblock size: %" PRIu64
+find_page_matched(const char *idstr) "ramblock %s addr or size changed"
--
1.8.3.1
[PATCH v4 07/12] migration/dirtyrate: Compare page hash results for recorded sampled page
Compare page hash results for recorded sampled page.
Signed-off-by: Chuan Zheng
Signed-off-by: YanYing Zhuang
---
migration/dirtyrate.c | 64 +++
1 file changed, 64 insertions(+)
diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c
index 66de426..050270d 100644
--- a/migration/dirtyrate.c
+++ b/migration/dirtyrate.c
@@ -202,6 +202,70 @@ static int record_ramblock_hash_info(struct
RamblockDirtyInfo **block_dinfo,
return 0;
}
+static int calc_page_dirty_rate(struct RamblockDirtyInfo *info)
+{
+uint32_t crc;
+int i;
+
+for (i = 0; i < info->sample_pages_count; i++) {
+crc = get_ramblock_vfn_hash(info, info->sample_page_vfn[i]);
+if (crc != info->hash_result[i]) {
+info->sample_dirty_count++;
+}
+}
+
+return 0;
+}
+
+static bool find_page_matched(RAMBlock *block, struct RamblockDirtyInfo *infos,
+ int count, struct RamblockDirtyInfo **matched)
+{
+int i;
+
+for (i = 0; i < count; i++) {
+if (!strcmp(infos[i].idstr, qemu_ram_get_idstr(block))) {
+break;
+}
+}
+
+if (i == count) {
+return false;
+}
+
+if (infos[i].ramblock_addr != qemu_ram_get_host_addr(block) ||
+infos[i].ramblock_pages !=
+(qemu_ram_get_used_length(block) >> DIRTYRATE_PAGE_SHIFT_KB)) {
+return false;
+}
+
+*matched = &infos[i];
+return true;
+}
+
+static int compare_page_hash_info(struct RamblockDirtyInfo *info,
+ int block_index)
+{
+struct RamblockDirtyInfo *block_dinfo = NULL;
+RAMBlock *block = NULL;
+
+RAMBLOCK_FOREACH_MIGRATABLE(block) {
+block_dinfo = NULL;
+if (!find_page_matched(block, info, block_index + 1, &block_dinfo)) {
+continue;
+}
+if (calc_page_dirty_rate(block_dinfo) < 0) {
+return -1;
+}
+update_dirtyrate_stat(block_dinfo);
+}
+
+if (!DirtyStat.total_sample_count) {
+return -1;
+}
+
+return 0;
+}
+
static void calculate_dirtyrate(struct DirtyRateConfig config)
{
/* todo */
--
1.8.3.1
[PATCH v4 00/12] *** A Method for evaluating dirty page rate ***
v3 -> v4:
use crc32 to get hash result instead of md5
add DirtyRateStatus to denote calculation status
add some trace_calls to make it easier to debug
fix some comments accroding to review
v2 -> v3:
fix size_t compile warning
fix codestyle checked by checkpatch.pl
v1 -> v2:
use g_rand_new() to generate rand_buf
move RAMBLOCK_FOREACH_MIGRATABLE into migration/ram.h
add skip_sample_ramblock to filter sampled ramblock
fix multi-numa vm coredump when query dirtyrate
rename qapi interface and rename some structures and functions
succeed to compile by appling each patch
add test for migrating vm
Sometimes it is neccessary to evaluate dirty page rate before migration.
Users could decide whether to proceed migration based on the evaluation
in case of vm performance loss due to heavy workload.
Unlikey simulating dirtylog sync which could do harm on runnning vm,
we provide a sample-hash method to compare hash results for samping page.
In this way, it would have hardly no impact on vm performance.
Evaluate the dirtypage rate both on running and migration vm.
The VM specifications for migration are as follows:
- VM use 4-K page;
- the number of VCPU is 32;
- the total memory is 32Gigabit;
- use 'mempress' tool to pressurize VM(mempress 4096 1024);
- migration bandwidth is 1GB/s
+++
| | running | migrating
|
+++
| no mempress | 4MB/s | 8MB/s (migrated success)
|
---
| mempress 4096 1024 | 1060MB/s | 456MB/s ~ 1142MB/s (cpu throttle
triggered) |
+++
| mempress 4096 4096 | 4114MB/s | 688MB/s ~ 4132MB/s (cpu throttle
triggered) |
+++
Test dirtyrate by qmp command like this:
1. virsh qemu-monitor-command [vmname] '{"execute":"calc-dirty-rate",
"arguments": {"calc-time": [sleep-time]}}';
2. sleep specific time which is a bit larger than sleep-time
3. virsh qemu-monitor-command [vmname] '{"execute":"query-dirty-rate"}'
Further test dirtyrate by libvirt api like this:
virsh getdirtyrate [vmname] [sleep-time]
Chuan Zheng (12):
migration/dirtyrate: add DirtyRateStatus to denote calculation status
migration/dirtyrate: Add RamlockDirtyInfo to store sampled page info
migration/dirtyrate: Add dirtyrate statistics series functions
migration/dirtyrate: move RAMBLOCK_FOREACH_MIGRATABLE into ram.h
migration/dirtyrate: Record hash results for each sampled page
migration/dirtyrate: Compare page hash results for recorded sampled
page
migration/dirtyrate: skip sampling ramblock with size below
MIN_RAMBLOCK_SIZE
migration/dirtyrate: Implement get_sample_page_period() and
block_sample_page_period()
migration/dirtyrate: Implement calculate_dirtyrate() function
migration/dirtyrate: Implement
qmp_cal_dirty_rate()/qmp_get_dirty_rate() function
migration/dirtyrate: Add trace_calls to make it easier to debug
migration/dirtyrate: setup up query-dirtyrate framwork
migration/Makefile.objs | 1 +
migration/dirtyrate.c | 432
migration/dirtyrate.h | 87 ++
migration/ram.c | 11 +-
migration/ram.h | 10 ++
migration/trace-events | 8 +
qapi/migration.json | 61 +++
7 files changed, 600 insertions(+), 10 deletions(-)
create mode 100644 migration/dirtyrate.c
create mode 100644 migration/dirtyrate.h
--
1.8.3.1
[PATCH v4 06/12] migration/dirtyrate: Record hash results for each sampled page
Record hash results for each sampled page, crc32 is taken to calculate
hash results for each sampled 4K-page.
Signed-off-by: Chuan Zheng
Signed-off-by: YanYing Zhuang
---
migration/dirtyrate.c | 136 ++
migration/dirtyrate.h | 15 ++
2 files changed, 151 insertions(+)
diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c
index f6a94d8..66de426 100644
--- a/migration/dirtyrate.c
+++ b/migration/dirtyrate.c
@@ -10,6 +10,7 @@
* See the COPYING file in the top-level directory.
*/
+#include
#include "qemu/osdep.h"
#include "qapi/error.h"
#include "crypto/hash.h"
@@ -66,6 +67,141 @@ static void update_dirtyrate(uint64_t msec)
DirtyStat.dirty_rate = dirtyrate;
}
+/*
+ * get hash result for the sampled memory with length of 4K byte in ramblock,
+ * which starts from ramblock base address.
+ */
+static uint32_t get_ramblock_vfn_hash(struct RamblockDirtyInfo *info,
+ uint64_t vfn)
+{
+struct iovec iov_array;
+uint32_t crc;
+
+iov_array.iov_base = info->ramblock_addr +
+ vfn * DIRTYRATE_SAMPLE_PAGE_SIZE;
+iov_array.iov_len = DIRTYRATE_SAMPLE_PAGE_SIZE;
+
+crc = crc32(0, iov_array.iov_base, iov_array.iov_len);
+
+return crc;
+}
+
+static int save_ramblock_hash(struct RamblockDirtyInfo *info)
+{
+unsigned int sample_pages_count;
+int i;
+int ret = -1;
+GRand *rand = g_rand_new();
+
+sample_pages_count = info->sample_pages_count;
+
+/* ramblock size less than one page, return success to skip this ramblock
*/
+if (unlikely(info->ramblock_pages == 0 || sample_pages_count == 0)) {
+ret = 0;
+goto out;
+}
+
+info->hash_result = g_try_malloc0_n(sample_pages_count,
+sizeof(uint32_t));
+if (!info->hash_result) {
+ret = -1;
+goto out;
+}
+
+info->sample_page_vfn = g_try_malloc0_n(sample_pages_count,
+sizeof(uint64_t));
+if (!info->sample_page_vfn) {
+g_free(info->hash_result);
+ret = -1;
+goto out;
+}
+
+for (i = 0; i < sample_pages_count; i++) {
+info->sample_page_vfn[i] = g_rand_int_range(rand, 0,
+info->ramblock_pages - 1);
+info->hash_result[i] = get_ramblock_vfn_hash(info,
+ info->sample_page_vfn[i]);
+}
+ret = 0;
+
+out:
+g_rand_free(rand);
+return ret;
+}
+
+static void get_ramblock_dirty_info(RAMBlock *block,
+struct RamblockDirtyInfo *info,
+struct DirtyRateConfig *config)
+{
+uint64_t sample_pages_per_gigabytes = config->sample_pages_per_gigabytes;
+
+/* Right shift 30 bits to calc block size in GB */
+info->sample_pages_count = (qemu_ram_get_used_length(block) *
+sample_pages_per_gigabytes) >>
+DIRTYRATE_PAGE_SHIFT_GB;
+
+/* Right shift 12 bits to calc page count in 4KB */
+info->ramblock_pages = qemu_ram_get_used_length(block) >>
+ DIRTYRATE_PAGE_SHIFT_KB;
+info->ramblock_addr = qemu_ram_get_host_addr(block);
+strcpy(info->idstr, qemu_ram_get_idstr(block));
+}
+
+static struct RamblockDirtyInfo *
+alloc_ramblock_dirty_info(int *block_index,
+ struct RamblockDirtyInfo *block_dinfo)
+{
+struct RamblockDirtyInfo *info = NULL;
+int index = *block_index;
+
+if (!block_dinfo) {
+index = 0;
+block_dinfo = g_try_new(struct RamblockDirtyInfo, 1);
+} else {
+index++;
+block_dinfo = g_try_realloc(block_dinfo, (index + 1) *
+sizeof(struct RamblockDirtyInfo));
+}
+if (!block_dinfo) {
+return NULL;
+}
+
+info = &block_dinfo[index];
+*block_index = index;
+memset(info, 0, sizeof(struct RamblockDirtyInfo));
+
+return block_dinfo;
+}
+
+static int record_ramblock_hash_info(struct RamblockDirtyInfo **block_dinfo,
+ struct DirtyRateConfig config,
+ int *block_index)
+{
+struct RamblockDirtyInfo *info = NULL;
+struct RamblockDirtyInfo *dinfo = NULL;
+RAMBlock *block = NULL;
+int index = 0;
+
+RAMBLOCK_FOREACH_MIGRATABLE(block) {
+dinfo = alloc_ramblock_dirty_info(&index, dinfo);
+if (dinfo == NULL) {
+return -1;
+}
+info = &dinfo[index];
+get_ramblock_dirty_info(block, info, &config);
+if (save_ramblock_hash(info) < 0) {
+*block_dinfo = dinfo;
+*block_index = index;
+return -1;
+}
+}
+
+*block_dinfo = dinfo;
+*block_index = index;
+
+return 0;
+}
+
static void calculate_dirtyrate(struct DirtyRateConfig co
[PATCH v4 09/12] migration/dirtyrate: Implement get_sample_page_period() and block_sample_page_period()
Implement get_sample_page_period() and set_sample_page_period() to
sleep specific time between sample actions.
Signed-off-by: Chuan Zheng
---
migration/dirtyrate.c | 24
migration/dirtyrate.h | 2 ++
2 files changed, 26 insertions(+)
diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c
index bd398b7..d1c0a78 100644
--- a/migration/dirtyrate.c
+++ b/migration/dirtyrate.c
@@ -28,6 +28,30 @@
static int CalculatingState = DIRTY_RATE_STATUS_UNSTARTED;
static struct DirtyRateStat DirtyStat;
+static int64_t set_sample_page_period(int64_t msec, int64_t initial_time)
+{
+int64_t current_time;
+
+current_time = qemu_clock_get_ms(QEMU_CLOCK_REALTIME);
+if ((current_time - initial_time) >= msec) {
+msec = current_time - initial_time;
+} else {
+g_usleep((msec + initial_time - current_time) * 1000);
+}
+
+return msec;
+}
+
+static int64_t get_sample_page_period(int64_t sec)
+{
+if (sec <= MIN_FETCH_DIRTYRATE_TIME_SEC ||
+sec > MAX_FETCH_DIRTYRATE_TIME_SEC) {
+sec = DEFAULT_FETCH_DIRTYRATE_TIME_SEC;
+}
+
+return sec;
+}
+
static int dirtyrate_set_state(int *state, int old_state, int new_state)
{
assert(new_state < DIRTY_RATE_STATUS__MAX);
diff --git a/migration/dirtyrate.h b/migration/dirtyrate.h
index 41bc264..50a5636 100644
--- a/migration/dirtyrate.h
+++ b/migration/dirtyrate.h
@@ -51,6 +51,8 @@
/* Take 1s as default for calculation duration */
#define DEFAULT_FETCH_DIRTYRATE_TIME_SEC 1
+#define MIN_FETCH_DIRTYRATE_TIME_SEC 0
+#define MAX_FETCH_DIRTYRATE_TIME_SEC 60
struct DirtyRateConfig {
uint64_t sample_pages_per_gigabytes; /* sample pages per GB */
--
1.8.3.1
[PATCH v4 10/12] migration/dirtyrate: Implement calculate_dirtyrate() function
Implement calculate_dirtyrate() function.
Signed-off-by: Chuan Zheng
Signed-off-by: YanYing Zhuang
---
migration/dirtyrate.c | 45 +++--
1 file changed, 43 insertions(+), 2 deletions(-)
diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c
index d1c0a78..9f52f5f 100644
--- a/migration/dirtyrate.c
+++ b/migration/dirtyrate.c
@@ -171,6 +171,21 @@ static void get_ramblock_dirty_info(RAMBlock *block,
strcpy(info->idstr, qemu_ram_get_idstr(block));
}
+static void free_ramblock_dirty_info(struct RamblockDirtyInfo *infos, int
count)
+{
+int i;
+
+if (!infos) {
+return;
+}
+
+for (i = 0; i < count; i++) {
+g_free(infos[i].sample_page_vfn);
+g_free(infos[i].hash_result);
+}
+g_free(infos);
+}
+
static struct RamblockDirtyInfo *
alloc_ramblock_dirty_info(int *block_index,
struct RamblockDirtyInfo *block_dinfo)
@@ -316,8 +331,34 @@ static int compare_page_hash_info(struct RamblockDirtyInfo
*info,
static void calculate_dirtyrate(struct DirtyRateConfig config)
{
-/* todo */
-return;
+struct RamblockDirtyInfo *block_dinfo = NULL;
+int block_index = 0;
+int64_t msec = 0;
+int64_t initial_time;
+
+rcu_register_thread();
+reset_dirtyrate_stat();
+initial_time = qemu_clock_get_ms(QEMU_CLOCK_REALTIME);
+rcu_read_lock();
+if (record_ramblock_hash_info(&block_dinfo, config, &block_index) < 0) {
+goto out;
+}
+rcu_read_unlock();
+
+msec = config.sample_period_seconds * 1000;
+msec = set_sample_page_period(msec, initial_time);
+
+rcu_read_lock();
+if (compare_page_hash_info(block_dinfo, block_index) < 0) {
+goto out;
+}
+
+update_dirtyrate(msec);
+
+out:
+rcu_read_unlock();
+free_ramblock_dirty_info(block_dinfo, block_index + 1);
+rcu_unregister_thread();
}
void *get_dirtyrate_thread(void *arg)
--
1.8.3.1
[PATCH v4 04/12] migration/dirtyrate: Add dirtyrate statistics series functions
Add dirtyrate statistics to record/update dirtyrate info.
Signed-off-by: Chuan Zheng
---
migration/dirtyrate.c | 29 +
migration/dirtyrate.h | 10 ++
2 files changed, 39 insertions(+)
diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c
index 91987c5..0d7163f 100644
--- a/migration/dirtyrate.c
+++ b/migration/dirtyrate.c
@@ -24,6 +24,7 @@
#include "dirtyrate.h"
static int CalculatingState = DIRTY_RATE_STATUS_UNSTARTED;
+static struct DirtyRateStat DirtyStat;
static int dirtyrate_set_state(int *state, int old_state, int new_state)
{
@@ -35,6 +36,34 @@ static int dirtyrate_set_state(int *state, int old_state,
int new_state)
}
}
+static void reset_dirtyrate_stat(void)
+{
+DirtyStat.total_dirty_samples = 0;
+DirtyStat.total_sample_count = 0;
+DirtyStat.total_block_mem_MB = 0;
+DirtyStat.dirty_rate = 0;
+}
+
+static void update_dirtyrate_stat(struct RamblockDirtyInfo *info)
+{
+DirtyStat.total_dirty_samples += info->sample_dirty_count;
+DirtyStat.total_sample_count += info->sample_pages_count;
+/* size of 4K pages in MB */
+DirtyStat.total_block_mem_MB += info->ramblock_pages / 256;
+}
+
+static void update_dirtyrate(uint64_t msec)
+{
+uint64_t dirtyrate;
+uint64_t total_dirty_samples = DirtyStat.total_dirty_samples;
+uint64_t total_sample_count = DirtyStat.total_sample_count;
+uint64_t total_block_mem_MB = DirtyStat.total_block_mem_MB;
+
+dirtyrate = total_dirty_samples * total_block_mem_MB *
+ 1000 / (total_sample_count * msec);
+
+DirtyStat.dirty_rate = dirtyrate;
+}
static void calculate_dirtyrate(struct DirtyRateConfig config)
{
diff --git a/migration/dirtyrate.h b/migration/dirtyrate.h
index 7da..9db269d 100644
--- a/migration/dirtyrate.h
+++ b/migration/dirtyrate.h
@@ -45,6 +45,16 @@ struct RamblockDirtyInfo {
uint32_t *hash_result; /* array of hash result for sampled pages */
};
+/*
+ * Store calculation statistics for each measure.
+ */
+struct DirtyRateStat {
+uint64_t total_dirty_samples; /* total dirty sampled page */
+uint64_t total_sample_count; /* total sampled pages */
+uint64_t total_block_mem_MB; /* size of total sampled pages in MB */
+int64_t dirty_rate; /* dirty rate in MB/s */
+};
+
void *get_dirtyrate_thread(void *arg);
#endif
--
1.8.3.1
[PATCH v4 02/12] migration/dirtyrate: add DirtyRateStatus to denote calculation status
add DirtyRateStatus to denote calculating status.
Signed-off-by: Chuan Zheng
---
migration/dirtyrate.c | 22 ++
qapi/migration.json | 17 +
2 files changed, 39 insertions(+)
diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c
index 366f4e9..91987c5 100644
--- a/migration/dirtyrate.c
+++ b/migration/dirtyrate.c
@@ -23,6 +23,19 @@
#include "migration.h"
#include "dirtyrate.h"
+static int CalculatingState = DIRTY_RATE_STATUS_UNSTARTED;
+
+static int dirtyrate_set_state(int *state, int old_state, int new_state)
+{
+assert(new_state < DIRTY_RATE_STATUS__MAX);
+if (atomic_cmpxchg(state, old_state, new_state) == old_state) {
+return 0;
+} else {
+return -1;
+}
+}
+
+
static void calculate_dirtyrate(struct DirtyRateConfig config)
{
/* todo */
@@ -32,8 +45,17 @@ static void calculate_dirtyrate(struct DirtyRateConfig
config)
void *get_dirtyrate_thread(void *arg)
{
struct DirtyRateConfig config = *(struct DirtyRateConfig *)arg;
+int ret;
+
+ret = dirtyrate_set_state(&CalculatingState, DIRTY_RATE_STATUS_UNSTARTED,
+ DIRTY_RATE_STATUS_MEASURING);
+if (ret == -1) {
+return NULL;
+}
calculate_dirtyrate(config);
+ret = dirtyrate_set_state(&CalculatingState, DIRTY_RATE_STATUS_MEASURING,
+ DIRTY_RATE_STATUS_MEASURED);
return NULL;
}
diff --git a/qapi/migration.json b/qapi/migration.json
index d500055..d1a7b2d 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -1621,3 +1621,20 @@
##
{ 'event': 'UNPLUG_PRIMARY',
'data': { 'device-id': 'str' } }
+
+##
+# @DirtyRateStatus:
+#
+# An enumeration of dirtyrate status.
+#
+# @unstarted: query-dirtyrate thread is not initial.
+#
+# @measuring: query-dirtyrate thread is created and start to measure.
+#
+# @measured: query-dirtyrate thread is end, we can get result.
+#
+# Since: 5.2
+#
+##
+{ 'enum': 'DirtyRateStatus',
+ 'data': [ 'unstarted', 'measuring', 'measured'] }
--
1.8.3.1
[PATCH v4 11/12] migration/dirtyrate: Implement qmp_cal_dirty_rate()/qmp_get_dirty_rate() function
Implement qmp_cal_dirty_rate()/qmp_get_dirty_rate() function which could be
called
Signed-off-by: Chuan Zheng
---
migration/dirtyrate.c | 45 +
qapi/migration.json | 44
2 files changed, 89 insertions(+)
diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c
index 9f52f5f..08c46d3 100644
--- a/migration/dirtyrate.c
+++ b/migration/dirtyrate.c
@@ -62,6 +62,28 @@ static int dirtyrate_set_state(int *state, int old_state,
int new_state)
}
}
+static struct DirtyRateInfo *query_dirty_rate_info(void)
+{
+int64_t dirty_rate = DirtyStat.dirty_rate;
+struct DirtyRateInfo *info = g_malloc0(sizeof(DirtyRateInfo));
+
+if (CalculatingState == DIRTY_RATE_STATUS_MEASURED) {
+info->dirty_rate = dirty_rate;
+} else {
+info->dirty_rate = -1;
+}
+
+info->status = CalculatingState;
+/*
+ * Only support query once for each calculation,
+ * reset as DIRTY_RATE_STATUS_UNSTARTED after query
+ */
+(void)dirtyrate_set_state(&CalculatingState, CalculatingState,
+ DIRTY_RATE_STATUS_UNSTARTED);
+
+return info;
+}
+
static void reset_dirtyrate_stat(void)
{
DirtyStat.total_dirty_samples = 0;
@@ -378,3 +400,26 @@ void *get_dirtyrate_thread(void *arg)
DIRTY_RATE_STATUS_MEASURED);
return NULL;
}
+
+void qmp_calc_dirty_rate(int64_t calc_time, Error **errp)
+{
+static struct DirtyRateConfig config;
+QemuThread thread;
+
+/*
+ * We don't begin calculating thread only when it's in calculating status.
+ */
+if (CalculatingState == DIRTY_RATE_STATUS_MEASURING) {
+return;
+}
+
+config.sample_period_seconds = get_sample_page_period(calc_time);
+config.sample_pages_per_gigabytes = DIRTYRATE_DEFAULT_SAMPLE_PAGES;
+qemu_thread_create(&thread, "get_dirtyrate", get_dirtyrate_thread,
+ (void *)&config, QEMU_THREAD_DETACHED);
+}
+
+struct DirtyRateInfo *qmp_query_dirty_rate(Error **errp)
+{
+return query_dirty_rate_info();
+}
diff --git a/qapi/migration.json b/qapi/migration.json
index d1a7b2d..97c5aba 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -1638,3 +1638,47 @@
##
{ 'enum': 'DirtyRateStatus',
'data': [ 'unstarted', 'measuring', 'measured'] }
+
+##
+# @DirtyRateInfo:
+#
+# Information about current dirty page rate of vm.
+#
+# @dirty-rate: @dirtyrate describing the dirty page rate of vm
+# in units of MB/s.
+# If this field return '-1', it means querying is not
+# start or not complete.
+#
+# @status: status containing dirtyrate query status includes
+# 'unstarted' or 'measuring' or 'measured'
+#
+# Since: 5.2
+#
+##
+{ 'struct': 'DirtyRateInfo',
+ 'data': {'dirty-rate': 'int64',
+ 'status': 'DirtyRateStatus'} }
+
+##
+# @calc-dirty-rate:
+#
+# start calculating dirty page rate for vm
+#
+# @calc-time: time in units of second for sample dirty pages
+#
+# Since: 5.2
+#
+# Example:
+# {"command": "cal-dirty-rate", "data": {"calc-time": 1} }
+#
+##
+{ 'command': 'calc-dirty-rate', 'data': {'calc-time': 'int64'} }
+
+##
+# @query-dirty-rate:
+#
+# query dirty page rate in units of MB/s for vm
+#
+# Since: 5.2
+##
+{ 'command': 'query-dirty-rate', 'returns': 'DirtyRateInfo' }
--
1.8.3.1
[PATCH v4 01/12] migration/dirtyrate: setup up query-dirtyrate framwork
Add get_dirtyrate_thread() functions to setup query-dirtyrate
framework.
Signed-off-by: Chuan Zheng
Signed-off-by: YanYing Zhuang
---
migration/Makefile.objs | 1 +
migration/dirtyrate.c | 39 +++
migration/dirtyrate.h | 32
3 files changed, 72 insertions(+)
create mode 100644 migration/dirtyrate.c
create mode 100644 migration/dirtyrate.h
diff --git a/migration/Makefile.objs b/migration/Makefile.objs
index 0fc619e..12ae98c 100644
--- a/migration/Makefile.objs
+++ b/migration/Makefile.objs
@@ -6,6 +6,7 @@ common-obj-y += qemu-file.o global_state.o
common-obj-y += qemu-file-channel.o
common-obj-y += xbzrle.o postcopy-ram.o
common-obj-y += qjson.o
+common-obj-y += dirtyrate.o
common-obj-y += block-dirty-bitmap.o
common-obj-y += multifd.o
common-obj-y += multifd-zlib.o
diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c
new file mode 100644
index 000..366f4e9
--- /dev/null
+++ b/migration/dirtyrate.c
@@ -0,0 +1,39 @@
+/*
+ * Dirtyrate implement code
+ *
+ * Copyright (c) 2017-2020 HUAWEI TECHNOLOGIES CO.,LTD.
+ *
+ * Authors:
+ * Chuan Zheng
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "crypto/hash.h"
+#include "crypto/random.h"
+#include "qemu/config-file.h"
+#include "exec/memory.h"
+#include "exec/ramblock.h"
+#include "exec/target_page.h"
+#include "qemu/rcu_queue.h"
+#include "qapi/qapi-commands-migration.h"
+#include "migration.h"
+#include "dirtyrate.h"
+
+static void calculate_dirtyrate(struct DirtyRateConfig config)
+{
+/* todo */
+return;
+}
+
+void *get_dirtyrate_thread(void *arg)
+{
+struct DirtyRateConfig config = *(struct DirtyRateConfig *)arg;
+
+calculate_dirtyrate(config);
+
+return NULL;
+}
diff --git a/migration/dirtyrate.h b/migration/dirtyrate.h
new file mode 100644
index 000..33669b7
--- /dev/null
+++ b/migration/dirtyrate.h
@@ -0,0 +1,32 @@
+/*
+ * Dirtyrate common functions
+ *
+ * Copyright (c) 2020 HUAWEI TECHNOLOGIES CO., LTD.
+ *
+ * Authors:
+ * Chuan Zheng
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#ifndef QEMU_MIGRATION_DIRTYRATE_H
+#define QEMU_MIGRATION_DIRTYRATE_H
+
+/*
+ * Sample 512 pages per GB as default.
+ * TODO: Make it configurable.
+ */
+#define DIRTYRATE_DEFAULT_SAMPLE_PAGES512
+
+/* Take 1s as default for calculation duration */
+#define DEFAULT_FETCH_DIRTYRATE_TIME_SEC 1
+
+struct DirtyRateConfig {
+uint64_t sample_pages_per_gigabytes; /* sample pages per GB */
+int64_t sample_period_seconds; /* time duration between two sampling */
+};
+
+void *get_dirtyrate_thread(void *arg);
+#endif
+
--
1.8.3.1
[PATCH v4 03/12] migration/dirtyrate: Add RamlockDirtyInfo to store sampled page info
Add RamlockDirtyInfo to store sampled page info of each ramblock.
Signed-off-by: Chuan Zheng
---
migration/dirtyrate.h | 18 ++
1 file changed, 18 insertions(+)
diff --git a/migration/dirtyrate.h b/migration/dirtyrate.h
index 33669b7..7da 100644
--- a/migration/dirtyrate.h
+++ b/migration/dirtyrate.h
@@ -19,6 +19,11 @@
*/
#define DIRTYRATE_DEFAULT_SAMPLE_PAGES512
+/*
+ * Record ramblock idstr
+ */
+#define RAMBLOCK_INFO_MAX_LEN 256
+
/* Take 1s as default for calculation duration */
#define DEFAULT_FETCH_DIRTYRATE_TIME_SEC 1
@@ -27,6 +32,19 @@ struct DirtyRateConfig {
int64_t sample_period_seconds; /* time duration between two sampling */
};
+/*
+ * Store dirtypage info for each ramblock.
+ */
+struct RamblockDirtyInfo {
+char idstr[RAMBLOCK_INFO_MAX_LEN]; /* idstr for each ramblock */
+uint8_t *ramblock_addr; /* base address of ramblock we measure */
+uint64_t ramblock_pages; /* ramblock size in 4K-page */
+uint64_t *sample_page_vfn; /* relative offset address for sampled page */
+uint64_t sample_pages_count; /* count of sampled pages */
+uint64_t sample_dirty_count; /* cout of dirty pages we measure */
+uint32_t *hash_result; /* array of hash result for sampled pages */
+};
+
void *get_dirtyrate_thread(void *arg);
#endif
--
1.8.3.1
[PATCH v4 05/12] migration/dirtyrate: move RAMBLOCK_FOREACH_MIGRATABLE into ram.h
RAMBLOCK_FOREACH_MIGRATABLE is need in dirtyrate measure,
move the existing definition up into migration/ram.h
Signed-off-by: Chuan Zheng
---
migration/dirtyrate.c | 1 +
migration/ram.c | 11 +--
migration/ram.h | 10 ++
3 files changed, 12 insertions(+), 10 deletions(-)
diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c
index 0d7163f..f6a94d8 100644
--- a/migration/dirtyrate.c
+++ b/migration/dirtyrate.c
@@ -21,6 +21,7 @@
#include "qemu/rcu_queue.h"
#include "qapi/qapi-commands-migration.h"
#include "migration.h"
+#include "ram.h"
#include "dirtyrate.h"
static int CalculatingState = DIRTY_RATE_STATUS_UNSTARTED;
diff --git a/migration/ram.c b/migration/ram.c
index 76d4fee..37ef0da 100644
--- a/migration/ram.c
+++ b/migration/ram.c
@@ -158,21 +158,12 @@ out:
return ret;
}
-static bool ramblock_is_ignored(RAMBlock *block)
+bool ramblock_is_ignored(RAMBlock *block)
{
return !qemu_ram_is_migratable(block) ||
(migrate_ignore_shared() && qemu_ram_is_shared(block));
}
-/* Should be holding either ram_list.mutex, or the RCU lock. */
-#define RAMBLOCK_FOREACH_NOT_IGNORED(block)\
-INTERNAL_RAMBLOCK_FOREACH(block) \
-if (ramblock_is_ignored(block)) {} else
-
-#define RAMBLOCK_FOREACH_MIGRATABLE(block) \
-INTERNAL_RAMBLOCK_FOREACH(block) \
-if (!qemu_ram_is_migratable(block)) {} else
-
#undef RAMBLOCK_FOREACH
int foreach_not_ignored_block(RAMBlockIterFunc func, void *opaque)
diff --git a/migration/ram.h b/migration/ram.h
index 2eeaacf..011e854 100644
--- a/migration/ram.h
+++ b/migration/ram.h
@@ -37,6 +37,16 @@ extern MigrationStats ram_counters;
extern XBZRLECacheStats xbzrle_counters;
extern CompressionStats compression_counters;
+bool ramblock_is_ignored(RAMBlock *block);
+/* Should be holding either ram_list.mutex, or the RCU lock. */
+#define RAMBLOCK_FOREACH_NOT_IGNORED(block)\
+INTERNAL_RAMBLOCK_FOREACH(block) \
+if (ramblock_is_ignored(block)) {} else
+
+#define RAMBLOCK_FOREACH_MIGRATABLE(block) \
+INTERNAL_RAMBLOCK_FOREACH(block) \
+if (!qemu_ram_is_migratable(block)) {} else
+
int xbzrle_cache_resize(int64_t new_size, Error **errp);
uint64_t ram_bytes_remaining(void);
uint64_t ram_bytes_total(void);
--
1.8.3.1
RE: [PULL v2 00/24] target/xtensa updates for 5.2
> -Original Message- > From: Philippe Mathieu-Daudé On > Behalf Of Philippe Mathieu-Daudé > Sent: Saturday, August 22, 2020 4:21 AM > To: Max Filippov ; qemu-devel@nongnu.org > Cc: Peter Maydell ; Richard Henderson > ; Taylor Simpson > Subject: Re: [PULL v2 00/24] target/xtensa updates for 5.2 > > > I'm asking because IIRC we have been reluctant to accept the hexagon > port due to its big generated imported data (beside the licensing > issues). > I wasn't aware of any licensing issues with the Hexagon port. Could you elaborate? Thanks, Taylor
[PATCH RFC v2 01/10] qemu/: fix some comment spelling errors
I found that there are many spelling errors in the comments of qemu,
so I used the spellcheck tool to check the spelling errors
and finally found some spelling errors in the folder.
Signed-off-by: zhaolichang
---
Changelog | 2 +-
accel/tcg/user-exec.c | 2 +-
audio/audio.c | 2 +-
block.c | 2 +-
configure | 2 +-
fsdev/virtfs-proxy-helper.c | 2 +-
hmp-commands.hx | 2 +-
libdecnumber/decNumber.c| 2 +-
qemu-img.c | 2 +-
qobject/qdict.c | 2 +-
rules.mak | 2 +-
scsi/pr-manager-helper.c| 2 +-
12 files changed, 12 insertions(+), 12 deletions(-)
diff --git a/Changelog b/Changelog
index 4a90bb9..f7e178c 100644
--- a/Changelog
+++ b/Changelog
@@ -241,7 +241,7 @@ version 0.8.0:
version 0.7.2:
- x86_64 fixes (Win2000 and Linux 2.6 boot in 32 bit)
- - merge self modifying code handling in dirty ram page mecanism.
+ - merge self modifying code handling in dirty ram page mechanism.
- MIPS fixes (Ralf Baechle)
- better user net performances
diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c
index d8b027f..d1b3073 100644
--- a/accel/tcg/user-exec.c
+++ b/accel/tcg/user-exec.c
@@ -88,7 +88,7 @@ static inline int handle_cpu_signal(uintptr_t pc, siginfo_t
*info,
* use that value directly. Within cpu_restore_state_from_tb, we
* assume PC comes from GETPC(), as used by the helper functions,
* so we adjust the address by -GETPC_ADJ to form an address that
- * is within the call insn, so that the address does not accidentially
+ * is within the call insn, so that the address does not accidentally
* match the beginning of the next guest insn. However, when the
* pc comes from the signal frame it points to the actual faulting
* host memory insn and not the return from a call insn.
diff --git a/audio/audio.c b/audio/audio.c
index ce8c6de..1a68cfa 100644
--- a/audio/audio.c
+++ b/audio/audio.c
@@ -1674,7 +1674,7 @@ static AudioState *audio_init(Audiodev *dev, const char
*name)
head = audio_handle_legacy_opts();
/*
* In case of legacy initialization, all Audiodevs in the list will
have
- * the same configuration (except the driver), so it does't matter
which
+ * the same configuration (except the driver), so it doesn't matter
which
* one we chose. We need an Audiodev to set up AudioState before we
can
* init a driver. Also note that dev at this point is still in the
* list.
diff --git a/block.c b/block.c
index d9ac0e0..ba03eaf 100644
--- a/block.c
+++ b/block.c
@@ -2602,7 +2602,7 @@ static void bdrv_replace_child_noperm(BdrvChild *child,
/*
* Updates @child to change its reference to point to @new_bs, including
- * checking and applying the necessary permisson updates both to the old node
+ * checking and applying the necessary permission updates both to the old node
* and to @new_bs.
*
* NULL is passed as @new_bs for removing the reference before freeing @child.
diff --git a/configure b/configure
index 2acc4d1..e34ef0c 100644
--- a/configure
+++ b/configure
@@ -3592,7 +3592,7 @@ EOF
xfs="yes"
else
if test "$xfs" = "yes" ; then
- feature_not_found "xfs" "Instal xfsprogs/xfslibs devel"
+ feature_not_found "xfs" "Install xfsprogs/xfslibs devel"
fi
xfs=no
fi
diff --git a/fsdev/virtfs-proxy-helper.c b/fsdev/virtfs-proxy-helper.c
index de061a8..15c0e79 100644
--- a/fsdev/virtfs-proxy-helper.c
+++ b/fsdev/virtfs-proxy-helper.c
@@ -518,7 +518,7 @@ static void statfs_to_prstatfs(ProxyStatFS *pr_stfs, struct
statfs *stfs)
/*
* Gets stat/statfs information and packs in out_iovec structure
- * on success returns number of bytes packed in out_iovec struture
+ * on success returns number of bytes packed in out_iovec structure
* otherwise returns -errno
*/
static int do_stat(int type, struct iovec *iovec, struct iovec *out_iovec)
diff --git a/hmp-commands.hx b/hmp-commands.hx
index 60f395c..27c4bbe 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -1267,7 +1267,7 @@ ERST
},
SRST
``drive_backup``
- Start a point-in-time copy of a block device to a specificed target.
+ Start a point-in-time copy of a block device to a specified target.
ERST
{
diff --git a/libdecnumber/decNumber.c b/libdecnumber/decNumber.c
index 8c19702..1ffe458 100644
--- a/libdecnumber/decNumber.c
+++ b/libdecnumber/decNumber.c
@@ -5626,7 +5626,7 @@ static const uShort LNnn[90] = {
/*would certainly save at least one if it were made ten times */
/*bigger, too (for truncated fractions 0.100 through 0.999). */
/*However, for most practical evaluations, at least four or five */
-/*iterations will be neede -- so this would only speed up by */
+/*iterations will be needed -- so this would only speed up by */
/*20-25% and that probably does not justify increas
Re: [PULL 00/40] ppc-for-5.2 queue 20200818
On Sun, Aug 23, 2020 at 02:54:26PM +0100, Peter Maydell wrote: > On Tue, 18 Aug 2020 at 05:19, David Gibson > wrote: > > > > The following changes since commit d0ed6a69d399ae193959225cdeaa9382746c91cc: > > > > Update version for v5.1.0 release (2020-08-11 17:07:03 +0100) > > > > are available in the Git repository at: > > > > git://github.com/dgibson/qemu.git tags/ppc-for-5.2-20200818 > > > > for you to fetch changes up to 3110f0ee19ccdb50adff3dfa1321039f69efddcd: > > > > spapr/xive: Use xive_source_esb_len() (2020-08-14 13:35:45 +1000) > > > > > > ppc patch queue 2020-08-18 > > > > Here's my first pull request for qemu-5.2, which has quite a few > > accumulated things. Highlights are: > > > > * Preliminary support for POWER10 (Power ISA 3.1) instruction emulation > > * Add documentation on the (very confusing) pseries NUMA configuration > > * Fix some bugs handling edge cases with XICS, XIVE and kernel_irqchip > > * Fix icount for a number of POWER registers > > * Many cleanups to error handling in XIVE code > > * Validate size of -prom-env data > > Hi -- it looks like you've updated the tag but I haven't seen > a new cover letter. Do you want me to apply it? Um.. I've updated the branch, but the tag should still be the same. I'd suggest merging the tag, since that's what I did my testing run on. -- David Gibson| I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson signature.asc Description: PGP signature
[Fwd] Issue 25164 in oss-fuzz: qemu: Fuzzing build failure
Hi Paolo,
Our oss-fuzz builds started failing, after the meson merge. I think I
tracked down the issues:
1.) Looking at the build-log here:
https://oss-fuzz-build-logs.storage.googleapis.com/log-d43d402c-1ce5-4422-b3db-ccbf83a862a0.txt
The error happens at link-time. Re-running the build with V=1:
"/usr/bin/ld" ...
--whole-archive /usr/local/lib/clang/12.0.0/.../libclang_rt.asan-x86_64.a \
--start-group . -T /src/qemu/tests/qtest/fuzz/fork_fuzz.ld \
-wrap qtest_inb -wrap qtest_inw . --end-group .
I compared this against what we had before meson, and noticed that the
start-group/end-group are new. Removing those manually from the link
command, I can link successfully. It looks like these linker groups are
something that meson controls. Is there something we can tweak to make
the fork_fuzz.ld linker script apply to all the objects?
2.) 77afc75f69 ("oss-fuzz/build: remove LIB_FUZZING_ENGINE")
On oss-fuzz, we cannot explicitly specify fsanitize=fuzzer: We have to
leverage the $CC $CXX $CFLAGS $CXXFLAGS $LIB_FUZZING_ENGINE from
oss-fuzz. That was the reason for the "make CONFIG_FUZZ CFLAGS" trickery
in the original build script.
Details:
https://google.github.io/oss-fuzz/getting-started/new-project-guide/#Requirements
To work around this, I think we can create separate configure options
--enable-oss-fuzz and --oss-fuzz-cflags and CONFIG_OSS_FUZZ. In meson we
create a new "source-set" specific_oss_fuzz_ss which is identical to
specific_fuzz_ss, except it does not depend on "-fsanitize=fuzzer",
which is specified in tests/qtest/fuzz/meson.build
I've been working on patches to do (2) but I don't know how to fix (1).
Do you have any ideas?
-Alex
- Forwarded message from ClusterFuzz-External via monorail
-
Date: Sun, 23 Aug 2020 03:10:14 -0700
From: ClusterFuzz-External via monorail
To: alx...@bu.edu
Subject: Issue 25164 in oss-fuzz: qemu: Fuzzing build failure
Status: New
Owner:
CC: b...@redhat.com, stefa...@redhat.com, alx...@bu.edu, pbonz...@redhat.com,
darren.k...@oracle.com
Labels: Proj-qemu
Type: Build-Failure
New issue 25164 by ClusterFuzz-External: qemu: Fuzzing build failure
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=25164
The last 2 builds for qemu have been failing.
Build log:
https://oss-fuzz-build-logs.storage.googleapis.com/log-d43d402c-1ce5-4422-b3db-ccbf83a862a0.txt
Build type: fuzzing
To reproduce locally, please see:
https://google.github.io/oss-fuzz/advanced-topics/reproducing#reproducing-build-failures
This bug tracker is not being monitored by OSS-Fuzz team. If you have any
questions, please create an issue at
https://github.com/google/oss-fuzz/issues/new.
**This bug will be automatically closed within a day once it is fixed.**
--
You received this message because:
1. You were specifically CC'd on the issue
You may adjust your notification preferences at:
https://bugs.chromium.org/hosting/settings
Reply to this email to add a comment.
- End forwarded message -
Re: [PATCH 0/1] qcow2: Skip copy-on-write when allocating a zero cluster
On Fri, Aug 21, 2020 at 08:59:44AM -0400, Brian Foster wrote: > On Fri, Aug 21, 2020 at 01:42:52PM +0200, Alberto Garcia wrote: > > On Fri 21 Aug 2020 01:05:06 PM CEST, Brian Foster > > wrote: > > And yes, (4) is a bit slower than (1) in my tests. On ext4 I get 10% > > more IOPS. > > > > I just ran the tests with aio=native and with a raw image instead of > > qcow2, here are the results: > > > > qcow2: > > |--+-+| > > | preallocation| aio=threads | aio=native | > > |--+-+| > > | off |8139 | 7649 | > > | off (w/o ZERO_RANGE) |2965 | 2779 | > > | metadata |7768 | 8265 | > > | falloc |7742 | 7956 | > > | full | 41389 | 56668 | > > |--+-+| > > > > So this seems like Dave's suggestion to use native aio produced more > predictable results with full file prealloc being a bit faster than per > cluster prealloc. Not sure why that isn't the case with aio=threads. I That will the context switch overhead with aio=threads becoming a performance limiting factor at higher IOPS. The "full" workload there is probably running at 80-120k context switches/s while the aio=native if probably under 10k ctxsw/s because it doesn't switch threads for every IO that has to be submitted/completed. For all the other results, I'd consider the difference to be noise - it's just not significant enough to draw any conclusions from at all. FWIW, the other thing that aio=native gives us is plugging across batch IO submission. This allows bio merging before dispatch and that can greatly increase performance of AIO when the IO being submitted has some mergable submissions. That's not the case for pure random IO like this, but there are relatively few pure random IO workloads out there... :P > was wondering if perhaps the threading affects something indirectly like > the qcow2 metadata allocation itself, but I guess that would be > inconsistent with ext4 showing a notable jump from (1) to (4) (assuming > the previous ext4 numbers were with aio=threads). > > raw: > > |---+-+| > > | preallocation | aio=threads | aio=native | > > |---+-+| > > | off |7647 | 7928 | > > | falloc|7662 | 7856 | > > | full | 45224 | 58627 | > > |---+-+| > > > > A qcow2 file with preallocation=metadata is more or less similar to a > > sparse raw file (and the numbers are indeed similar). > > > > preallocation=off on qcow2 does not have an equivalent on raw files. > > > > It sounds like preallocation=off for qcow2 would be roughly equivalent > to a raw file with a 64k extent size hint (on XFS). Yes, the effect should be close to identical, the only difference is that qcow2 adds new clusters to the end of the file (i.e. the file itself is not sparse), while the extent size hint will just add 64kB extents into the file around the write offset. That demonstrates the other behavioural advantage that extent size hints have is they avoid needing to extend the file, which is yet another way to serialise concurrent IO and create IO pipeline stalls... Cheers, Dave. -- Dave Chinner da...@fromorbit.com
[REPORT] Nightly Performance Tests - Sunday, August 23, 2020
Host CPU : Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz Host Memory : 15.49 GB Start Time (UTC) : 2020-08-23 21:30:02 End Time (UTC) : 2020-08-23 22:02:16 Execution Time : 0:32:14.028460 Status : FAILURE ERROR LOGS 2020-08-23T21:30:03.149828 - Verifying executables of 8 benchmarks for 17 targets 2020-08-23T21:30:03.256787 - Verifying results of reference version v5.1.0 2020-08-23T21:30:03.522675 - Checking out master 2020-08-23T21:30:07.340980 - Pulling the latest changes from QEMU master error: RPC failed; curl 56 GnuTLS recv error (-54): Error in the pull function. fatal: The remote end hung up unexpectedly fatal: protocol error: bad pack header 2020-08-23T21:43:32.047625 - Trial 1/10: Failed to pull QEMU ... retrying again in a minute! fatal: unable to access 'https://git.qemu.org/git/qemu.git/': Could not resolve host: git.qemu.org 2020-08-23T21:44:52.183570 - Trial 2/10: Failed to pull QEMU ... retrying again in a minute! fatal: unable to access 'https://git.qemu.org/git/qemu.git/': Could not resolve host: git.qemu.org 2020-08-23T21:46:12.285963 - Trial 3/10: Failed to pull QEMU ... retrying again in a minute! fatal: unable to access 'https://git.qemu.org/git/qemu.git/': Could not resolve host: git.qemu.org 2020-08-23T21:47:32.368841 - Trial 4/10: Failed to pull QEMU ... retrying again in a minute! fatal: unable to access 'https://git.qemu.org/git/qemu.git/': Could not resolve host: git.qemu.org 2020-08-23T21:48:52.473741 - Trial 5/10: Failed to pull QEMU ... retrying again in a minute! fatal: unable to access 'https://git.qemu.org/git/qemu.git/': Could not resolve host: git.qemu.org 2020-08-23T21:50:12.584987 - Trial 6/10: Failed to pull QEMU ... retrying again in a minute! fatal: unable to access 'https://git.qemu.org/git/qemu.git/': Could not resolve host: git.qemu.org 2020-08-23T21:51:32.688082 - Trial 7/10: Failed to pull QEMU ... retrying again in a minute! fatal: unable to access 'https://git.qemu.org/git/qemu.git/': GnuTLS recv error (-110): The TLS connection was non-properly terminated. 2020-08-23T21:53:03.538585 - Trial 8/10: Failed to pull QEMU ... retrying again in a minute! fatal: unable to access 'https://git.qemu.org/git/qemu.git/': Could not resolve host: git.qemu.org 2020-08-23T21:54:23.661134 - Trial 9/10: Failed to pull QEMU ... retrying again in a minute! fatal: unable to access 'https://git.qemu.org/git/qemu.git/': GnuTLS recv error (-54): Error in the pull function. 2020-08-23T22:02:16.671699 - Trial 10/10: Failed to pull QEMU
Re: [PATCH 0/1] qcow2: Skip copy-on-write when allocating a zero cluster
On Fri, Aug 21, 2020 at 02:12:32PM +0200, Alberto Garcia wrote: > On Fri 21 Aug 2020 01:42:52 PM CEST, Alberto Garcia wrote: > > On Fri 21 Aug 2020 01:05:06 PM CEST, Brian Foster > > wrote: > >>> > 1) off: for every write request QEMU initializes the cluster (64KB) > >>> > with fallocate(ZERO_RANGE) and then writes the 4KB of data. > >>> > > >>> > 2) off w/o ZERO_RANGE: QEMU writes the 4KB of data and fills the rest > >>> > of the cluster with zeroes. > >>> > > >>> > 3) metadata: all clusters were allocated when the image was created > >>> > but they are sparse, QEMU only writes the 4KB of data. > >>> > > >>> > 4) falloc: all clusters were allocated with fallocate() when the image > >>> > was created, QEMU only writes 4KB of data. > >>> > > >>> > 5) full: all clusters were allocated by writing zeroes to all of them > >>> > when the image was created, QEMU only writes 4KB of data. > >>> > > >>> > As I said in a previous message I'm not familiar with xfs, but the > >>> > parts that I don't understand are > >>> > > >>> >- Why is (4) slower than (1)? > >>> > >>> Because fallocate() is a full IO serialisation barrier at the > >>> filesystem level. If you do: > >>> > >>> fallocate(whole file) > >>> > >>> > >>> > >>> . > >>> > >>> The IO can run concurrent and does not serialise against anything in > >>> the filesysetm except unwritten extent conversions at IO completion > >>> (see answer to next question!) > >>> > >>> However, if you just use (4) you get: > >>> > >>> falloc(64k) > >>> > >>> > >>> <4k io> > >>> > >>> falloc(64k) > >>> > >>> > >>> <4k IO completes, converts 4k to written> > >>> > >>> <4k io> > >>> falloc(64k) > >>> > >>> > >>> <4k IO completes, converts 4k to written> > >>> > >>> <4k io> > >>> > >>> > >> > >> Option 4 is described above as initial file preallocation whereas > >> option 1 is per 64k cluster prealloc. Prealloc mode mixup aside, Berto > >> is reporting that the initial file preallocation mode is slower than > >> the per cluster prealloc mode. Berto, am I following that right? > > After looking more closely at the data I can see that there is a peak of > ~30K IOPS during the first 5 or 6 seconds and then it suddenly drops to > ~7K for the rest of the test. How big is the filesystem, how big is the log? (xfs_info output, please!) In general, there are three typical causes of this. The first is typical of the initial burst of allocations running on an empty journal, then allocation transactions getting throttling back to the speed at which metadata can be flushed once the journal fills up. If you have a small filesystem and a default sized log, this is quite likely to happen. The second is that have large logs and you are running on hardware where device cache flushes and FUA writes hammer overall device performance. Hence when the CIL initially fills up and starts flushing (journal writes are pre-flush + FUA so do both) device performance goes way down because now it has to write it's cached data to physical media rather than just cache it in volatile device RAM. IOWs, journal writes end up forcing all volatile data to stable media and so that can slow the device down. ALso, cache flushes might not be queued commands, hence journal writes will also create IO pipeline stalls... The third is the hardware capability. Consumer hardware is designed to have extremely fast bursty behaviour, but then steady state performance is much lower (think "SLC" burst caches in TLC SSDs). I have isome consumer SSDs here that can sustain 400MB/s random 4kB write for about 10-15s, then they drop to about 50MB/s once the burst buffer is full. OTOH, I have enterprise SSDs that will sustain a _much_ higher rate of random 4kB writes indefinitely than the consumer SSDs burst at. However, most consumer workloads don't move this sort of data around, so this sort of design tradeoff is fine for that market (Benchmarketing 101 stuff :). IOWs, this behaviour could be filesystem config, it could be cache flush behaviour, it could simply be storage device design capability. Or it could be a combination of all three things. Watching a set of fast sampling metrics that tell you what the device and filesytem are doing in real time (e.g. I use PCP for this and visualise ithe behaviour in real time via pmchart) gives a lot of insight into exactly what is changing during transient workload changes liek starting a benchmark... > I was running fio with --ramp_time=5 which ignores the first 5 seconds > of data in order to let performance settle, but if I remove that I can > see the effect more clearly. I can observe it with raw files (in 'off' > and 'prealloc' modes) and qcow2 files in 'prealloc' mode. With qcow2 and > preallocation=off the performance is stable during the whole test. What does "preallocation=off" mean again? Is that using fallocate(ZERO_RANGE) prior to the data write rather than prealloc
Re: [PATCH] configure: avoid compiling qemu-keymap by default
Hi Laurent, there are two ways to do this in Meson without having to add more special casing in configure. The simplest is to build qemu-keymap by default only if have_tools. This is a one-liner adding the "build_by_default: have_tools" argument. The second is to move the detection of xkbcommon to Meson, and putting it under "if have_system || have_tools", that is where meson.build has the qemu-keymap executable you do xkbcommon = not_found if have_system || have_tools xkbcommon = ... endif if xkbcommon.found() qemu_keymap = ... endif and CONFIG_XKBCOMMON can also be replaced by xkbcommon.found(). This is a bit more complicated but all the parts are explained in docs/devel/build-system.rst. Feel free to pick the simplest of the two, I just explained both just in case. When I am back I can also explain you on IRC the way I translated the syscall_nr.h generator. Thanks, Paolo Il dom 23 ago 2020, 22:32 ha scritto: > qemu-keymap is not needed with linux-user, so disable it by default if > tools are disabled (tools are disabled by default with linux-user). > > Avoid this error with statically linked binaries: > > Linking target qemu-keymap > /usr/bin/ld: cannot find -lxkbcommon > > Signed-off-by: Laurent Vivier > --- > configure | 5 + > 1 file changed, 5 insertions(+) > > diff --git a/configure b/configure > index d9ca87fbbb52..2cab3330d010 100755 > --- a/configure > +++ b/configure > @@ -3448,6 +3448,11 @@ fi > > ## > # xkbcommon probe > +if test -z "$xkbcommon"; then > + if test "$want_tools" = "no"; then > +xkbcommon=no > + fi > +fi > if test "$xkbcommon" != "no" ; then >if $pkg_config xkbcommon --exists; then > xkbcommon_cflags=$($pkg_config xkbcommon --cflags) > -- > 2.26.2 > > > >
Re: odd meson failure: Unknown variable "exe_name"
It's a rebase issue from the introduction of fuzzing binaries. I will send a fix. Paolo Il dom 23 ago 2020, 12:54 Peter Maydell ha scritto: > On Sun, 23 Aug 2020 at 11:45, Peter Maydell > wrote: > > > > On my x86-64 Linux box, one of my local build trees turned out not > > to work: > > > > $ rm -rf build/x86 && mkdir build/x86 && (cd build/x86 && > > '../../configure' > > > '--target-list=arm-softmmu,aarch64-softmmu,arm-linux-user,aarch64-linux-user' > > '--enable-debug' '--cc=ccache gcc' '--audio-drv-list=pa' > > '--with-pkgversion=pm215' '--enable-trace-backends=log,dtrace' > > '--enable-docs') > > > > fails with > > [...] > > Program keycodemapdb/tools/keymap-gen found: YES > > Program scripts/decodetree.py found: YES > > Program ../scripts/modules/module_block.py found: YES > > Program nm found: YES > > Program scripts/undefsym.sh found: YES > > Program scripts/feature_to_c.sh found: YES > > > > ../../meson.build:1030:14: ERROR: Unknown variable "exe_name". > > > > A full log can be found at > > > /home/petmay01/linaro/qemu-from-laptop/qemu/build/x86/meson-logs/meson-log.txt > > > > ERROR: meson setup failed > > > > > > This is the same box that's worked fine for merge testing, so > > presumably something about the particular set of configure > > options is tripping it up. > > Dropping the '--enable-trace-backends=...' option lets it pass, > so that's the area where the problem is. > > -- PMM > >
[PATCH v3 0/8] linux-user: Adding support for a group of btrfs ioctls
This series covers support for following btrfs ioctls *BTRFS_SUBVOL_CREATE *BTRFS_IOC_ADD_DEV *BTRFS_SUBVOL_SETFLAGS *BTRFS_IOC_RM_DEV *BTRFS_SUBVOL_GETFLAGS *BTRFS_IOC_DEV_INFO *BTRFS_GET_SUBVOL_INFO *BTRFS_IOC_GET_DEV_STATS *BTRFS_IOC_SNAP_CREATE *BTRFS_IOC_GET_FEATURES *BTRFS_IOC_SNAP_DESTROY*BTRFS_IOC_SET_FEATURES *BTRFS_IOC_SCAN_DEV*BTRFS_IOC_GET_SUPPORTED_FEATURES *BTRFS_IOC_DEFAULT_SUBVOL *BTRFS_IOC_QUOTA_RESCAN *BTRFS_IOC_GET_SUBVOL_ROOTREF *BTRFS_IOC_QUOTA_RESCAN_WAIT *BTRFS_IOC_QUOTA_CTL *BTRFS_IOC_SCRUB *BTRFS_IOC_QGROUP_CREATE *BTRFS_IOC_SCRUB_CANCEL *BTRFS_IOC_QGROUP_ASSIGN *BTRFS_IOC_SCRUB_PROGRESS *BTRFS_IOC_INO_PATHS *BTRFS_IOC_QGROUP_LIMIT *BTRFS_IOC_LOGICAL_INO *BTRFS_IOC_QUOTA_RESCAN_STATUS *BTRFS_IOC_LOGICAL_INO_V2 *BTRFS_IOC_INO_LOOKUP_USER *BTRFS_IOC_INO_LOOKUP The functionalities of individual ioctls were described in this series patch commit messages. Since all of these ioctls are added in kernel version 3.9, their definitions in file 'linux-user/ioctls.h' are enwrapped in an #ifdef directive. Testing method: Mini test programs were written for these ioctls. These test programs can be found on a repositort which is located on the link: https://github.com/bozutaf/btrfs-tests These test programs were compiled (sometimes using cross compilers) for following architectures: * Intel 64-bit (little endian) * Power pc 32-bit (big endian) * Power pc 64-bit (big endian) The corresponding native programs were executed without using QEMU on an intel x86_64 host. All applicable compiled programs were in turn executed through QEMU and the results obtained were the same ones gotten for native execution. v2: * Merged two series in one 8 patch series * Changed target ioctl definitions from IOR/IOW/IOWR to IORU/IOWU/IOWRU * Fixed some thunk struct definitions v3: * Added some checks in fine 'syscall_types.h' to see whether the ioctls that use the defined thunk types are present. This is done to support systems that have older versions of 'btrfs.h' file or if the file is not present at all. This is neccesary as to not cause build errors as some structures depend on values which are defined in 'btrfs.h'. Filip Bozuta (8): linux-user: Add support for a group of btrfs ioctls used for subvolumes linux-user: Add support for a group of btrfs ioctls used for snapshots linux-user: Add support for btrfs ioctls used to manipulate with devices linux-user: Add support for btrfs ioctls used to get/set features linux-user: Add support for a group of btrfs inode ioctls linux-user: Add support for two btrfs ioctls used for subvolume linux-user: Add support for btrfs ioctls used to manage quota linux-user: Add support for btrfs ioctls used to scrub a filesystem configure | 9 ++ linux-user/ioctls.h| 124 linux-user/syscall.c | 3 + linux-user/syscall_defs.h | 37 + linux-user/syscall_types.h | 163 + 5 files changed, 336 insertions(+) -- 2.25.1
[PATCH] linux-user: Add support for a group of '_V2' btrfs ioctls
This patch introduces functionality for following btrfs ioctls:
BTRFS_IOC_SUBVOL_CREATE_V2 - Adding a new btrfs subvolume
Create a new btrfs subvolume (same as with BTRFS_IOC_SUBVOL_CREATE).
The third ioctl's argument is a pointer to a following type:
struct btrfs_ioctl_vol_args_v2 {
__s64 fd;
__u64 transid;
__u64 flags;
union {
struct {
__u64 size;
struct btrfs_qgroup_inherit __user *qgroup_inherit;
};
__u64 unused[4];
};
union {
char name[BTRFS_SUBVOL_NAME_MAX + 1];
__u64 devid;
__u64 subvolid; /* added in kernel version 5.8 */
};
};
When calling this ioctl, the 'name' field should be filled with
the aproppriate value that contains the name of the subvolume that
is to be created. The flags field can take values that are
'BTRFS_SUBVOL_RDONLY' or 'BTRFS_SUBVOL_QGROUP_INHERIT'. If the
latter is specified, the field 'qgroup_inherit' should be filled
with aproppriate values of the quota group in which the newly
created subvolume is to be added. The definition of
'struct btrfs_qgroup_inherit' can be found at:
https://elixir.bootlin.com/linux/latest/source/include/uapi/linux/btrfs.h#L90
BTRFS_IOC_SNAP_CREATE_V2 - Adding a btrfs snapshot for a subvolume
Create a new btrfs snapshot for a specified subvolume (same as with
BTRFS_IOC_SNAP_CREATE). The ioctl's third argument is a pointer to
the above mentioned 'struct btrfs_ioctl_vol_args_v2'. Before calling
this ioctl, field 'fd' should be filled with the aproppriate file
descriptor value for the btrfs subvolume for which the snapshot is
to be created. Also, the 'name' field should be filled with the
aproppriate value that represents the name of the snapshot that is
to be created. The 'flags' field takes the same values as in case
of 'BTRFS_IOC_SUBVOL_CREATE_V2' and represents the same functionality.
BTRFS_IOC_RM_DEV_V2 - Removing a btrfs device
Delete a btrfs device (same as with BTRFS_IOC_RM_DEV). The ioctl's third
argument is a pointer to the above mentioned 'struct
btrfs_ioctl_vol_args_v2'.
Before calling this ioctl, either the 'name' or the 'devid' field should
be filled with the name or id of the device that is to be removed. Also, the
flags field should be filled either with 0 or 'BTRFS_DEVICE_SPEC_BY_ID'
depending on if the device is gonna be specified via name or id.
BTRFS_IOC_SNAP_DESTROY_V2 - Removing a btrfs snapshot
Remove a btrfs snapshot (same as with BTRFS_IOC_SNAP_DESTROY). The ioctl's
third argument is a pointer to the above mentioned 'struct
btrfs_ioctl_vol_args_v2'.
Before calling this ioctl, either the 'name' or the 'subvolid' field should
be filled with the name or id of the snapshot that is to be removed. Also,
the
flags field should be filled either with 0 or 'BTRFS_SUBVOL_SPEC_BY_ID'
depending on if the snapshot is gonna be specified via name or id.
Implementation notes:
Since the third argument of the implemented ioctl's is a
structure that contains unions, a special converting function
'target_to_host_btrfs_ioctl_vol_args_v2' was defined in 'syscall.c'.
This function is called instead of 'thunk_convert()' to convert
the values of the third argument from target to host. All of
the ioctls in this patch are of type 'IOW' which is why a converting
function from host to target is not required. Also, a separate printing
function was defined in file 'strace.c' that is called instead of
'thunk_print()' to print the contents of the third argument.
Signed-off-by: Filip Bozuta
Based-on: <20200811164553.27713-2-filip.boz...@syrmia.com>
Based-on: <20200723210233.349690-4-filip.boz...@syrmia.com>
---
linux-user/ioctls.h| 16 +++
linux-user/qemu.h | 5 ++
linux-user/strace.c| 95 ++
linux-user/syscall.c | 77 ++
linux-user/syscall_defs.h | 28 +++
linux-user/syscall_types.h | 5 ++
thunk.c| 2 +-
7 files changed, 227 insertions(+), 1 deletion(-)
diff --git a/linux-user/ioctls.h b/linux-user/ioctls.h
index e3bfe78774..a29ce6b69d 100644
--- a/linux-user/ioctls.h
+++ b/linux-user/ioctls.h
@@ -209,6 +209,14 @@
#ifdef BTRFS_IOC_DEFAULT_SUBVOL
IOCTL(BTRFS_IOC_DEFAULT_SUBVOL, IOC_W, MK_PTR(TYPE_ULONGLONG))
#endif
+#ifdef BTRFS_IOC_SNAP_CREATE_V2
+ IOCTL(BTRFS_IOC_SNAP_CREATE_V2, IOC_W,
+ MK_PTR(MK_STRUCT(STRUCT_btrfs_ioctl_vol_args_v2)))
+#endif
+#ifdef BTRFS_IOC_SUBVOL_CREATE_V2
+ IOCTL(BTRFS_IOC_SUBVOL_CREATE_V2, IOC_W,
+ MK_PTR(MK_STRUCT(STRUCT_btrfs_ioctl_vol_args_v2)))
+#endif
#ifdef BTRFS_IOC_SUBVOL_GETFLAGS
IOCTL(BTRFS_IOC_SUBVOL_GETFLAGS, IOC_R, MK_PTR(TYPE_ULONGLONG))
#endif
@@ -281,6 +289,10 @@
[PATCH v3 8/8] linux-user: Add support for btrfs ioctls used to scrub a filesystem
This patch implements functionality for following ioctls:
BTRFS_IOC_SCRUB - Starting a btrfs filesystem scrub
Start a btrfs filesystem scrub. The third ioctls argument
is a pointer to a following type:
struct btrfs_ioctl_scrub_args {
__u64 devid;/* in */
__u64 start;/* in */
__u64 end; /* in */
__u64 flags;/* in */
struct btrfs_scrub_progress progress; /* out */
/* pad to 1k */
__u64 unused[(1024-32-sizeof(struct btrfs_scrub_progress))/8];
};
Before calling this ioctl, field 'devid' should be filled
with value that represents the device id of the btrfs filesystem
for which the scrub is to be started.
BTRFS_IOC_SCRUB_CANCEL - Canceling scrub of a btrfs filesystem
Cancel a btrfs filesystem scrub if it is running. The third
ioctls argument is ignored.
BTRFS_IOC_SCRUB_PROGRESS - Getting status of a running scrub
Read the status of a running btrfs filesystem scrub. The third
ioctls argument is a pointer to the above mentioned
'struct btrfs_ioctl_scrub_args'. Similarly as with 'BTRFS_IOC_SCRUB',
the 'devid' field should be filled with value that represents the
id of the btrfs device for which the scrub has started. The status
of a running scrub is returned in the field 'progress' which is
of type 'struct btrfs_scrub_progress' and its definition can be
found at:
https://elixir.bootlin.com/linux/latest/source/include/uapi/linux/btrfs.h#L150
Implementation nots:
Ioctls in this patch use type 'struct btrfs_ioctl_scrub_args' as their
third argument. That is the reason why an aproppriate thunk type
definition is added in file 'syscall_types.h'.
Signed-off-by: Filip Bozuta
---
linux-user/ioctls.h| 11 +++
linux-user/syscall_defs.h | 3 +++
linux-user/syscall_types.h | 29 +
3 files changed, 43 insertions(+)
diff --git a/linux-user/ioctls.h b/linux-user/ioctls.h
index 882bfb3e88..e3bfe78774 100644
--- a/linux-user/ioctls.h
+++ b/linux-user/ioctls.h
@@ -215,6 +215,17 @@
#ifdef BTRFS_IOC_SUBVOL_SETFLAGS
IOCTL(BTRFS_IOC_SUBVOL_SETFLAGS, IOC_W, MK_PTR(TYPE_ULONGLONG))
#endif
+#ifdef BTRFS_IOC_SCRUB
+ IOCTL(BTRFS_IOC_SCRUB, IOC_RW,
+ MK_PTR(MK_STRUCT(STRUCT_btrfs_ioctl_scrub_args)))
+#endif
+#ifdef BTRFS_IOC_SCRUB_CANCEL
+ IOCTL(BTRFS_IOC_SCRUB_CANCEL, 0, TYPE_NULL)
+#endif
+#ifdef BTRFS_IOC_SCRUB_PROGRESS
+ IOCTL(BTRFS_IOC_SCRUB_PROGRESS, IOC_RW,
+ MK_PTR(MK_STRUCT(STRUCT_btrfs_ioctl_scrub_args)))
+#endif
#ifdef BTRFS_IOC_DEV_INFO
IOCTL(BTRFS_IOC_DEV_INFO, IOC_RW,
MK_PTR(MK_STRUCT(STRUCT_btrfs_ioctl_dev_info_args)))
diff --git a/linux-user/syscall_defs.h b/linux-user/syscall_defs.h
index 10a7f91016..969377d622 100644
--- a/linux-user/syscall_defs.h
+++ b/linux-user/syscall_defs.h
@@ -982,6 +982,9 @@ struct target_rtc_pll_info {
abi_ullong)
#define TARGET_BTRFS_IOC_SUBVOL_SETFLAGSTARGET_IOW(BTRFS_IOCTL_MAGIC,
26,\
abi_ullong)
+#define TARGET_BTRFS_IOC_SCRUB
TARGET_IOWRU(BTRFS_IOCTL_MAGIC, 27)
+#define TARGET_BTRFS_IOC_SCRUB_CANCEL TARGET_IO(BTRFS_IOCTL_MAGIC,
28)
+#define TARGET_BTRFS_IOC_SCRUB_PROGRESS
TARGET_IOWRU(BTRFS_IOCTL_MAGIC, 29)
#define TARGET_BTRFS_IOC_DEV_INFO
TARGET_IOWRU(BTRFS_IOCTL_MAGIC, 30)
#define TARGET_BTRFS_IOC_INO_PATHS
TARGET_IOWRU(BTRFS_IOCTL_MAGIC, 35)
#define TARGET_BTRFS_IOC_LOGICAL_INO
TARGET_IOWRU(BTRFS_IOCTL_MAGIC, 36)
diff --git a/linux-user/syscall_types.h b/linux-user/syscall_types.h
index d9b7106a69..789723cfb9 100644
--- a/linux-user/syscall_types.h
+++ b/linux-user/syscall_types.h
@@ -421,6 +421,35 @@ STRUCT(btrfs_ioctl_ino_lookup_user_args,
MK_ARRAY(TYPE_CHAR, BTRFS_INO_LOOKUP_USER_PATH_MAX)) /* path */
#endif
+#if defined(BTRFS_IOC_SCRUB) || defined(BTRFS_IOC_SCRUB_PROGRESS)
+STRUCT(btrfs_scrub_progress,
+ TYPE_ULONGLONG, /* data_extents_scrubbed */
+ TYPE_ULONGLONG, /* tree_extents_scrubbed */
+ TYPE_ULONGLONG, /* data_bytes_scrubbed */
+ TYPE_ULONGLONG, /* tree_bytes_scrubbed */
+ TYPE_ULONGLONG, /* read_errors */
+ TYPE_ULONGLONG, /* csum_errors */
+ TYPE_ULONGLONG, /* verify_errors */
+ TYPE_ULONGLONG, /* no_csum */
+ TYPE_ULONGLONG, /* csum_discards */
+ TYPE_ULONGLONG, /* super_errors */
+ TYPE_ULONGLONG, /* malloc_errors */
+ TYPE_ULONGLONG, /* uncorrectable_errors */
+ TYPE_ULONGLONG, /* corrected_er */
+ TYPE_ULONGLONG, /* last_physical */
+ TYPE_ULONGLONG) /* unverified_errors */
+
+STRUCT(btrfs_ioctl_scrub_args,
+ TYPE_ULONGLONG, /* devid */
+ TYPE_ULONGLONG, /* start */
+ TYPE_ULONGL
[PATCH v3 4/8] linux-user: Add support for btrfs ioctls used to get/set features
This patch implements functionality for following ioctls:
BTRFS_IOC_GET_FEATURES - Getting feature flags
Read feature flags for a btrfs filesystem. The feature flags
are returned inside the ioctl's third argument which represents
a pointer to a following structure type:
struct btrfs_ioctl_feature_flags {
__u64 compat_flags;
__u64 compat_ro_flags;
__u64 incompat_flags;
};
All of the structure field represent bit masks that can be composed
of values which can be found on:
https://elixir.bootlin.com/linux/latest/source/fs/btrfs/ctree.h#L282
BTRFS_IOC_SET_FEATURES - Setting feature flags
Set and clear feature flags for a btrfs filesystem. The feature flags
are set using the ioctl's third argument which represents a
'struct btrfs_ioctl_feature_flags[2]' array. The first element of the
array represent flags which are to be cleared and the second element of
the array represent flags which are to be set. The second element has the
priority over the first, which means that if there are matching flags
in the elements, they will be set in the filesystem. If the flag values
in the third argument aren't correctly set to be composed of the available
predefined flag values, errno ENOPERM ("Operation not permitted") is
returned.
BTRFS_IOC_GET_SUPPORTED_FEATURES - Getting supported feature flags
Read supported feature flags for a btrfs filesystem. The supported
feature flags are read using the ioctl's third argument which represents
a 'struct btrfs_ioctl_feature_flags[3]' array. The first element of this
array represents all of the supported flags in the btrfs filesystem.
The second element represents flags that can be safely set and third element
represent flags that can be safely clearead.
Implementation notes:
All of the implemented ioctls use 'struct btrfs_ioctl_feature_flags' as
third argument. That is the reason why a corresponding defintion was added
in file 'linux-user/syscall_types.h'.
Signed-off-by: Filip Bozuta
---
linux-user/ioctls.h| 12
linux-user/syscall_defs.h | 3 +++
linux-user/syscall_types.h | 5 +
3 files changed, 20 insertions(+)
diff --git a/linux-user/ioctls.h b/linux-user/ioctls.h
index 55a6cbeca5..50fae1e33b 100644
--- a/linux-user/ioctls.h
+++ b/linux-user/ioctls.h
@@ -216,6 +216,18 @@
IOCTL(BTRFS_IOC_GET_DEV_STATS, IOC_RW,
MK_PTR(MK_STRUCT(STRUCT_btrfs_ioctl_get_dev_stats)))
#endif
+#ifdef BTRFS_IOC_GET_FEATURES
+ IOCTL(BTRFS_IOC_GET_FEATURES, IOC_R,
+ MK_PTR(MK_STRUCT(STRUCT_btrfs_ioctl_feature_flags)))
+#endif
+#ifdef BTRFS_IOC_SET_FEATURES
+ IOCTL(BTRFS_IOC_SET_FEATURES, IOC_W,
+ MK_PTR(MK_ARRAY(MK_STRUCT(STRUCT_btrfs_ioctl_feature_flags), 2)))
+#endif
+#ifdef BTRFS_IOC_GET_SUPPORTED_FEATURES
+ IOCTL(BTRFS_IOC_GET_SUPPORTED_FEATURES, IOC_R,
+ MK_PTR(MK_ARRAY(MK_STRUCT(STRUCT_btrfs_ioctl_feature_flags), 3)))
+#endif
#ifdef BTRFS_IOC_GET_SUBVOL_INFO
IOCTL(BTRFS_IOC_GET_SUBVOL_INFO, IOC_R,
MK_PTR(MK_STRUCT(STRUCT_btrfs_ioctl_get_subvol_info_args)))
diff --git a/linux-user/syscall_defs.h b/linux-user/syscall_defs.h
index 4c7bfa0391..44811fdcbb 100644
--- a/linux-user/syscall_defs.h
+++ b/linux-user/syscall_defs.h
@@ -981,6 +981,9 @@ struct target_rtc_pll_info {
abi_ullong)
#define TARGET_BTRFS_IOC_DEV_INFO
TARGET_IOWRU(BTRFS_IOCTL_MAGIC, 30)
#define TARGET_BTRFS_IOC_GET_DEV_STATS
TARGET_IOWRU(BTRFS_IOCTL_MAGIC, 52)
+#define TARGET_BTRFS_IOC_GET_FEATURES TARGET_IORU(BTRFS_IOCTL_MAGIC,
57)
+#define TARGET_BTRFS_IOC_SET_FEATURES TARGET_IOWU(BTRFS_IOCTL_MAGIC,
57)
+#define TARGET_BTRFS_IOC_GET_SUPPORTED_FEATURES TARGET_IORU(BTRFS_IOCTL_MAGIC,
57)
#define TARGET_BTRFS_IOC_GET_SUBVOL_INFOTARGET_IORU(BTRFS_IOCTL_MAGIC,
60)
/* usb ioctls */
diff --git a/linux-user/syscall_types.h b/linux-user/syscall_types.h
index ea6898979b..925054cfd4 100644
--- a/linux-user/syscall_types.h
+++ b/linux-user/syscall_types.h
@@ -409,6 +409,11 @@ STRUCT(btrfs_ioctl_get_dev_stats,
128 - 2 - BTRFS_DEV_STAT_VALUES_MAX)) /* unused */
#endif
+STRUCT(btrfs_ioctl_feature_flags,
+ TYPE_ULONGLONG, /* compat_flags */
+ TYPE_ULONGLONG, /* compat_ro_flags */
+ TYPE_ULONGLONG) /* incompat_flags */
+
STRUCT(rtc_time,
TYPE_INT, /* tm_sec */
TYPE_INT, /* tm_min */
--
2.25.1
[PATCH v3 7/8] linux-user: Add support for btrfs ioctls used to manage quota
This patch implements functionality for following ioctls:
BTRFS_IOC_QUOTA_CTL - Enabling/Disabling quota support
Enable or disable quota support for a btrfs filesystem. Quota
support is enabled or disabled using the ioctls third argument
which represents a pointer to a following type:
struct btrfs_ioctl_quota_ctl_args {
__u64 cmd;
__u64 status;
};
Before calling this ioctl, the 'cmd' field should be filled
with one of the values 'BTRFS_QUOTA_CTL_ENABLE' (enabling quota)
'BTRFS_QUOTA_CTL_DISABLE' (disabling quota).
BTRFS_IOC_QGROUP_CREATE - Creating/Removing a subvolume quota group
Create or remove a subvolume quota group. The subvolume quota
group is created or removed using the ioctl's third argument which
represents a pointer to a following type:
struct btrfs_ioctl_qgroup_create_args {
__u64 create;
__u64 qgroupid;
};
Before calling this ioctl, the 'create' field should be filled
with the aproppriate value depending on if the user wants to
create or remove a quota group (0 for removing, everything else
for creating). Also, the 'qgroupid' field should be filled with
the value for the quota group id that is to be created.
BTRFS_IOC_QGROUP_ASSIGN - Asigning or removing a quota group as child group
Asign or remove a quota group as child quota group of another
group in the btrfs filesystem. The asignment is done using the
ioctl's third argument which represents a pointert to a following type:
struct btrfs_ioctl_qgroup_assign_args {
__u64 assign;
__u64 src;
__u64 dst;
};
Before calling this ioctl, the 'assign' field should be filled with
the aproppriate value depending on if the user wants to asign or remove
a quota group as a child quota group of another group (0 for removing,
everythin else for asigning). Also, the 'src' and 'dst' fields should
be filled with the aproppriate quota group id values depending on which
quota group needs to asigned or removed as child quota group of another
group ('src' gets asigned or removed as child group of 'dst').
BTRFS_IOC_QGROUP_LIMIT - Limiting the size of a quota group
Limit the size of a quota group. The size of the quota group is limited
with the ioctls third argument which represents a pointer to a following
type:
struct btrfs_ioctl_qgroup_limit_args {
__u64 qgroupid;
struct btrfs_qgroup_limit lim;
};
Before calling this ioctl, the 'qgroup' id field should be filled with
aproppriate value of the quota group id for which the size is to be
limited. The second field is of following type:
struct btrfs_qgroup_limit {
__u64 flags;
__u64 max_rfer;
__u64 max_excl;
__u64 rsv_rfer;
__u64 rsv_excl;
};
The 'max_rfer' field should be filled with the size to which the quota
group should be limited. The 'flags' field can be used for passing
additional options and can have values which can be found on:
https://elixir.bootlin.com/linux/latest/source/include/uapi/linux/btrfs.h#L67
BTRFS_IOC_QUOTA_RESCAN_STATUS - Checking status of running rescan operation
Check status of a running rescan operation. The status is checked using
the ioctl's third argument which represents a pointer to a following type:
struct btrfs_ioctl_quota_rescan_args {
__u64 flags;
__u64 progress;
__u64 reserved[6];
};
If there is a rescan operation running, 'flags' field is set to 1, and
'progress' field is set to aproppriate value which represents the progress
of the operation.
BTRFS_IOC_QUOTA_RESCAN - Starting a rescan operation
Start ar rescan operation to Trash all quota groups and scan the metadata
again with the current config. Before calling this ioctl,
BTRFS_IOC_QUOTA_RESCAN_STATUS sould be run to check if there is already a
rescan operation runing. After that ioctl call, the received
'struct btrfs_ioctl_quota_rescan_args' should be than passed as this ioctls
third argument.
BTRFS_IOC_QUOTA_RESCAN_WAIT - Waiting for a rescan operation to finish
Wait until a rescan operation is finished (if there is a rescan operation
running). The third ioctls argument is ignored.
Implementation notes:
Almost all of the ioctls in this patch use structure types as third
arguments.
That is the reason why aproppriate thunk definitions were added in file
'syscall_types.h'.
Signed-off-by: Filip Bozuta
---
linux-user/ioctls.h| 27 +++
linux-user/syscall_defs.h | 7 +++
linux-user/syscall_types.h | 29 +
3 files changed, 63 insertions(+)
diff --git a/linux-user/ioctls.h b/linux-user/ioctls.h
index 728880b9f3..882bfb3e88 100644
--- a/linux-user/ioctls.h
+++ b/linux-user/ioctls.h
@@ -227,6 +227,33 @@
IOCTL(BTRFS_IOC_LOGICAL_INO, IOC_RW
[PATCH v3 2/8] linux-user: Add support for a group of btrfs ioctls used for snapshots
This patch implements functionality for following ioctls:
BTRFS_IOC_SNAP_CREATE - Creating a subvolume snapshot
Create a snapshot of a btrfs subvolume. The snapshot is created using the
ioctl's third argument that is a pointer to a 'struct btrfs_ioctl_vol_args'
(which was mentioned in the previous patch). Before calling this ioctl,
the fields of the structure should be filled with aproppriate values for
the file descriptor and path of the subvolume for which the snapshot is to
be created.
BTRFS_IOC_SNAP_DESTROY - Removing a subvolume snapshot
Delete a snapshot of a btrfs subvolume. The snapshot is deleted using the
ioctl's third argument that is a pointer to a 'struct btrfs_ioctl_vol_args'
(which was mentioned in the previous patch). Before calling this ioctl,
the fields of the structure should be filled with aproppriate values for
the file descriptor and path of the subvolume for which the snapshot is to
be deleted.
Implementation notes:
Since the thunk type 'struct btrfs_ioctl_vol_args' is defined in the
previous patch, the implementation for these ioctls was straightforward.
Signed-off-by: Filip Bozuta
---
linux-user/ioctls.h| 8
linux-user/syscall_defs.h | 2 ++
linux-user/syscall_types.h | 3 ++-
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/linux-user/ioctls.h b/linux-user/ioctls.h
index 12d1444224..f33a99f8b6 100644
--- a/linux-user/ioctls.h
+++ b/linux-user/ioctls.h
@@ -174,10 +174,18 @@
IOCTL(FS_IOC32_GETVERSION, IOC_R, MK_PTR(TYPE_INT))
IOCTL(FS_IOC32_SETVERSION, IOC_W, MK_PTR(TYPE_INT))
+#ifdef BTRFS_IOC_SNAP_CREATE
+ IOCTL(BTRFS_IOC_SNAP_CREATE, IOC_W,
+ MK_PTR(MK_STRUCT(STRUCT_btrfs_ioctl_vol_args)))
+#endif
#ifdef BTRFS_IOC_SUBVOL_CREATE
IOCTL(BTRFS_IOC_SUBVOL_CREATE, IOC_W,
MK_PTR(MK_STRUCT(STRUCT_btrfs_ioctl_vol_args)))
#endif
+#ifdef BTRFS_IOC_SNAP_DESTROY
+ IOCTL(BTRFS_IOC_SNAP_DESTROY, IOC_W,
+ MK_PTR(MK_STRUCT(STRUCT_btrfs_ioctl_vol_args)))
+#endif
#ifdef BTRFS_IOC_SUBVOL_GETFLAGS
IOCTL(BTRFS_IOC_SUBVOL_GETFLAGS, IOC_R, MK_PTR(TYPE_ULONGLONG))
#endif
diff --git a/linux-user/syscall_defs.h b/linux-user/syscall_defs.h
index 2757956dfa..a945e12547 100644
--- a/linux-user/syscall_defs.h
+++ b/linux-user/syscall_defs.h
@@ -968,7 +968,9 @@ struct target_rtc_pll_info {
#define TARGET_FS_IOC32_SETVERSION TARGET_IOW('v', 2, int)
/* btrfs ioctls */
+#define TARGET_BTRFS_IOC_SNAP_CREATETARGET_IOWU(BTRFS_IOCTL_MAGIC,
1)
#define TARGET_BTRFS_IOC_SUBVOL_CREATE TARGET_IOWU(BTRFS_IOCTL_MAGIC,
14)
+#define TARGET_BTRFS_IOC_SNAP_DESTROY TARGET_IOWU(BTRFS_IOCTL_MAGIC,
15)
#define TARGET_BTRFS_IOC_SUBVOL_GETFLAGSTARGET_IOR(BTRFS_IOCTL_MAGIC,
25,\
abi_ullong)
#define TARGET_BTRFS_IOC_SUBVOL_SETFLAGSTARGET_IOW(BTRFS_IOCTL_MAGIC,
26,\
diff --git a/linux-user/syscall_types.h b/linux-user/syscall_types.h
index db61dbc1b3..038cd7869b 100644
--- a/linux-user/syscall_types.h
+++ b/linux-user/syscall_types.h
@@ -354,7 +354,8 @@ STRUCT(blkpg_partition,
MK_ARRAY(TYPE_CHAR, BLKPG_DEVNAMELTH), /* devname */
MK_ARRAY(TYPE_CHAR, BLKPG_VOLNAMELTH)) /* volname */
-#ifdef BTRFS_IOC_SUBVOL_CREATE
+#if defined(BTRFS_IOC_SUBVOL_CREATE) || defined(BTRFS_IOC_SNAP_CREATE) || \
+defined(BTRFS_IOC_SNAP_DESTROY)
STRUCT(btrfs_ioctl_vol_args,
TYPE_LONGLONG, /* fd */
MK_ARRAY(TYPE_CHAR, BTRFS_PATH_NAME_MAX + 1)) /* name */
--
2.25.1
[PATCH v3 3/8] linux-user: Add support for btrfs ioctls used to manipulate with devices
This patch implements functionality for following ioctls:
BTRFS_IOC_SCAN_DEV - Scanning device for a btrfs filesystem
Scan a device for a btrfs filesystem. The device that is to
be scanned is passed in the ioctl's third argument which
represents a pointer to a 'struct ioc_vol_args' (which was
mentioned in a previous patch). Before calling this ioctl,
the name field of this structure should be filled with the
aproppriate name value which represents a path for the device.
If the device contains a btrfs filesystem, the ioctl returns 0,
otherwise a negative value is returned.
BTRFS_IOC_ADD_DEV - Adding a device to a btrfs filesystem
Add a device to a btrfs filesystem. The device that is to be
added is passed in the ioctl's third argument which represents
a pointer to a 'struct ioc_vol_args' (which was mentioned in
a previous patch). Before calling this ioctl, the name field of
this structure should be filled with the aproppriate name value
which represents a path for the device.
BTRFS_IOC_RM_DEV - Removing a device from a btrfs filesystem
Remove a device from a btrfs filesystem. The device that is to be
removed is passed in the ioctl's third argument which represents
a pointer to a 'struct ioc_vol_args' (which was mentioned in
a previous patch). Before calling this ioctl, the name field of
this structure should be filled with the aproppriate name value
which represents a path for the device.
BTRFS_IOC_DEV_INFO - Getting information about a device
Obtain information for device in a btrfs filesystem. The information
is gathered in the ioctl's third argument which represents a pointer
to a following structure type:
struct btrfs_ioctl_dev_info_args {
__u64 devid;/* in/out */
__u8 uuid[BTRFS_UUID_SIZE]; /* in/out */
__u64 bytes_used; /* out */
__u64 total_bytes; /* out */
__u64 unused[379]; /* pad to 4k */
__u8 path[BTRFS_DEVICE_PATH_NAME_MAX]; /* out */
};
Before calling this ioctl, field "devid" should be set with the id value
for the device for which the information is to be obtained. If this field
is not aproppriately set, the errno ENODEV ("No such device") is returned.
BTRFS_IOC_GET_DEV_STATS - Getting device statistics
Obtain stats informatin for device in a btrfs filesystem. The information
is gathered in the ioctl's third argument which represents a pointer to
a following structure type:
struct btrfs_ioctl_get_dev_stats {
__u64 devid;/* in */
__u64 nr_items; /* in/out */
__u64 flags;/* in/out */
/* out values: */
__u64 values[BTRFS_DEV_STAT_VALUES_MAX];
/*
* This pads the struct to 1032 bytes. It was originally meant to pad to
* 1024 bytes, but when adding the flags field, the padding calculation
* was not adjusted.
*/
__u64 unused[128 - 2 - BTRFS_DEV_STAT_VALUES_MAX];
};
Before calling this ioctl, field "devid" should be set with the id value
for the device for which the information is to be obtained. If this field
is not aproppriately set, the errno ENODEV ("No such device") is returned.
BTRFS_IOC_FORGET_DEV - Remove unmounted devices
Search and remove all stale devices (devices which are not mounted).
The third ioctl argument is a pointer to a 'struct btrfs_ioctl_vol_args'.
The ioctl call will release all unmounted devices which match the path
which is specified in the "name" field of the structure. If an empty
path ("") is specified, all unmounted devices will be released.
Implementation notes:
Ioctls BTRFS_IOC_DEV_INFO and BTRFS_IOC_GET_DEV_STATS use types
'struct btrfs_ioctl_dev_info_args' and ' struct btrfs_ioctl_get_dev_stats'
as third argument types. That is the reason why corresponding structure
definitions were added in file 'linux-user/syscall_types.h'.
Since the thunk type for 'struct ioc_vol_args' was already added in a
previous patch, the rest of the implementation was straightforward.
Signed-off-by: Filip Bozuta
---
linux-user/ioctls.h| 24
linux-user/syscall_defs.h | 6 ++
linux-user/syscall_types.h | 24 +++-
3 files changed, 53 insertions(+), 1 deletion(-)
diff --git a/linux-user/ioctls.h b/linux-user/ioctls.h
index f33a99f8b6..55a6cbeca5 100644
--- a/linux-user/ioctls.h
+++ b/linux-user/ioctls.h
@@ -178,6 +178,22 @@
IOCTL(BTRFS_IOC_SNAP_CREATE, IOC_W,
MK_PTR(MK_STRUCT(STRUCT_btrfs_ioctl_vol_args)))
#endif
+#ifdef BTRFS_IOC_SCAN_DEV
+ IOCTL(BTRFS_IOC_SCAN_DEV, IOC_W,
+ MK_PTR(MK_STRUCT(STRUCT_btrfs_ioctl_vol_args)))
+#endif
+#ifdef BTRFS_IOC_FORGET_DEV
+ IOCTL(BTRFS_
[PATCH v3 5/8] linux-user: Add support for a group of btrfs inode ioctls
This patch implements functionality of following ioctls:
BTRFS_IOC_INO_LOOKUP - Reading tree root id and path
Read tree root id and path for a given file or directory.
The name and tree root id are returned in an ioctl's third
argument that represents a pointer to a following type:
struct btrfs_ioctl_ino_lookup_args {
__u64 treeid;
__u64 objectid;
char name[BTRFS_INO_LOOKUP_PATH_MAX];
};
Before calling this ioctl, field 'objectid' should be filled
with the object id value for which the tree id and path are
to be read. Value 'BTRFS_FIRST_FREE_OBJECTID' represents the
object id for the first available btrfs object (directory or
file).
BTRFS_IOC_INO_PATHS - Reading paths to all files
Read path to all files with a certain inode number. The paths
are returned in the ioctl's third argument which represents
a pointer to a following type:
struct btrfs_ioctl_ino_path_args {
__u64 inum; /* in */
__u64 size; /* in */
__u64 reserved[4];
/* struct btrfs_data_container *fspath; out */
__u64 fspath; /* out */
};
Before calling this ioctl, the 'inum' and 'size' field should
be filled with the aproppriate inode number and size of the
directory where file paths should be looked for. For now, the
paths are returned in an '__u64' (unsigned long long) value
'fspath'.
BTRFS_IOC_LOGICAL_INO - Reading inode numbers
Read inode numbers for files on a certain logical adress. The
inode numbers are returned in the ioctl's third argument which
represents a pointer to a following type:
struct btrfs_ioctl_logical_ino_args {
__u64 logical;/* in */
__u64 size; /* in */
__u64 reserved[3];/* must be 0 for now */
__u64 flags; /* in, v2 only */
/* struct btrfs_data_container *inodes;out */
__u64 inodes;
};
Before calling this ioctl, the 'logical' and 'size' field should
be filled with the aproppriate logical adress and size of where
the inode numbers of files should be looked for. For now, the
inode numbers are returned in an '__u64' (unsigned long long)
value 'inodes'.
BTRFS_IOC_LOGICAL_INO_V2 - Reading inode numbers
Same as the above mentioned ioctl except that it allows passing
a flags 'BTRFS_LOGICAL_INO_ARGS_IGNORE_OFFSET'.
BTRFS_IOC_INO_LOOKUP_USER - Reading subvolume name and path
Read name and path of a subvolume. The tree root id and
path are read in an ioctl's third argument which represents a
pointer to a following type:
struct btrfs_ioctl_ino_lookup_user_args {
/* in, inode number containing the subvolume of 'subvolid' */
__u64 dirid;
/* in */
__u64 treeid;
/* out, name of the subvolume of 'treeid' */
char name[BTRFS_VOL_NAME_MAX + 1];
/*
* out, constructed path from the directory with which the ioctl is
* called to dirid
*/
char path[BTRFS_INO_LOOKUP_USER_PATH_MAX];
};
Before calling this ioctl, the 'dirid' and 'treeid' field should
be filled with aproppriate values which represent the inode number
of the directory that contains the subvolume and treeid of the
subvolume.
Implementation notes:
All of the ioctls in this patch use structure types as third arguments.
That is the reason why aproppriate thunk definitions were added in file
'syscall_types.h'.
Signed-off-by: Filip Bozuta
---
linux-user/ioctls.h| 20
linux-user/syscall_defs.h | 5 +
linux-user/syscall_types.h | 32
3 files changed, 57 insertions(+)
diff --git a/linux-user/ioctls.h b/linux-user/ioctls.h
index 50fae1e33b..169f98f7a3 100644
--- a/linux-user/ioctls.h
+++ b/linux-user/ioctls.h
@@ -202,6 +202,10 @@
IOCTL(BTRFS_IOC_SNAP_DESTROY, IOC_W,
MK_PTR(MK_STRUCT(STRUCT_btrfs_ioctl_vol_args)))
#endif
+#ifdef BTRFS_IOC_INO_LOOKUP
+ IOCTL(BTRFS_IOC_INO_LOOKUP, IOC_RW,
+ MK_PTR(MK_STRUCT(STRUCT_btrfs_ioctl_ino_lookup_args)))
+#endif
#ifdef BTRFS_IOC_SUBVOL_GETFLAGS
IOCTL(BTRFS_IOC_SUBVOL_GETFLAGS, IOC_R, MK_PTR(TYPE_ULONGLONG))
#endif
@@ -212,6 +216,14 @@
IOCTL(BTRFS_IOC_DEV_INFO, IOC_RW,
MK_PTR(MK_STRUCT(STRUCT_btrfs_ioctl_dev_info_args)))
#endif
+#ifdef BTRFS_IOC_INO_PATHS
+ IOCTL(BTRFS_IOC_INO_PATHS, IOC_RW,
+ MK_PTR(MK_STRUCT(STRUCT_btrfs_ioctl_ino_path_args)))
+#endif
+#ifdef BTRFS_IOC_LOGICAL_INO
+ IOCTL(BTRFS_IOC_LOGICAL_INO, IOC_RW,
+ MK_PTR(MK_STRUCT(STRUCT_btrfs_ioctl_logical_ino_a
[PATCH v3 6/8] linux-user: Add support for two btrfs ioctls used for subvolume
This patch implements functionality for following ioctl:
BTRFS_IOC_DEFAULT_SUBVOL - Setting a default subvolume
Set a default subvolume for a btrfs filesystem. The third
ioctl's argument is a '__u64' (unsigned long long) which
represents the id of a subvolume that is to be set as
the default.
BTRFS_IOC_GET_SUBVOL_ROOTREF - Getting tree and directory id of subvolumes
Read tree and directory id of subvolumes from a btrfs
filesystem. The tree and directory id's are returned in the
ioctl's third argument which represents a pointer to a
following type:
struct btrfs_ioctl_get_subvol_rootref_args {
/* in/out, minimum id of rootref's treeid to be searched */
__u64 min_treeid;
/* out */
struct {
__u64 treeid;
__u64 dirid;
} rootref[BTRFS_MAX_ROOTREF_BUFFER_NUM];
/* out, number of found items */
__u8 num_items;
__u8 align[7];
};
Before calling this ioctl, 'min_treeid' field should be filled
with value that represent the minimum value for the tree id.
Implementation notes:
Ioctl BTRFS_IOC_GET_SUBVOL_ROOTREF uses the above mentioned structure
type as third argument. That is the reason why a aproppriate thunk
structure definition is added in file 'syscall_types.h'.
Signed-off-by: Filip Bozuta
---
linux-user/ioctls.h| 7 +++
linux-user/syscall_defs.h | 3 +++
linux-user/syscall_types.h | 13 +
3 files changed, 23 insertions(+)
diff --git a/linux-user/ioctls.h b/linux-user/ioctls.h
index 169f98f7a3..728880b9f3 100644
--- a/linux-user/ioctls.h
+++ b/linux-user/ioctls.h
@@ -206,6 +206,9 @@
IOCTL(BTRFS_IOC_INO_LOOKUP, IOC_RW,
MK_PTR(MK_STRUCT(STRUCT_btrfs_ioctl_ino_lookup_args)))
#endif
+#ifdef BTRFS_IOC_DEFAULT_SUBVOL
+ IOCTL(BTRFS_IOC_DEFAULT_SUBVOL, IOC_W, MK_PTR(TYPE_ULONGLONG))
+#endif
#ifdef BTRFS_IOC_SUBVOL_GETFLAGS
IOCTL(BTRFS_IOC_SUBVOL_GETFLAGS, IOC_R, MK_PTR(TYPE_ULONGLONG))
#endif
@@ -248,6 +251,10 @@
IOCTL(BTRFS_IOC_GET_SUBVOL_INFO, IOC_R,
MK_PTR(MK_STRUCT(STRUCT_btrfs_ioctl_get_subvol_info_args)))
#endif
+#ifdef BTRFS_IOC_GET_SUBVOL_ROOTREF
+ IOCTL(BTRFS_IOC_GET_SUBVOL_ROOTREF, IOC_RW,
+ MK_PTR(MK_STRUCT(STRUCT_btrfs_ioctl_get_subvol_rootref_args)))
+#endif
#ifdef BTRFS_IOC_INO_LOOKUP_USER
IOCTL(BTRFS_IOC_INO_LOOKUP_USER, IOC_RW,
MK_PTR(MK_STRUCT(STRUCT_btrfs_ioctl_ino_lookup_user_args)))
diff --git a/linux-user/syscall_defs.h b/linux-user/syscall_defs.h
index a6a9454b85..1d0bfb5479 100644
--- a/linux-user/syscall_defs.h
+++ b/linux-user/syscall_defs.h
@@ -976,6 +976,8 @@ struct target_rtc_pll_info {
#define TARGET_BTRFS_IOC_SUBVOL_CREATE TARGET_IOWU(BTRFS_IOCTL_MAGIC,
14)
#define TARGET_BTRFS_IOC_SNAP_DESTROY TARGET_IOWU(BTRFS_IOCTL_MAGIC,
15)
#define TARGET_BTRFS_IOC_INO_LOOKUP
TARGET_IOWRU(BTRFS_IOCTL_MAGIC, 18)
+#define TARGET_BTRFS_IOC_DEFAULT_SUBVOL TARGET_IOW(BTRFS_IOCTL_MAGIC,
19,\
+ abi_ullong)
#define TARGET_BTRFS_IOC_SUBVOL_GETFLAGSTARGET_IOR(BTRFS_IOCTL_MAGIC,
25,\
abi_ullong)
#define TARGET_BTRFS_IOC_SUBVOL_SETFLAGSTARGET_IOW(BTRFS_IOCTL_MAGIC,
26,\
@@ -989,6 +991,7 @@ struct target_rtc_pll_info {
#define TARGET_BTRFS_IOC_GET_SUPPORTED_FEATURES TARGET_IORU(BTRFS_IOCTL_MAGIC,
57)
#define TARGET_BTRFS_IOC_LOGICAL_INO_V2
TARGET_IOWRU(BTRFS_IOCTL_MAGIC, 59)
#define TARGET_BTRFS_IOC_GET_SUBVOL_INFOTARGET_IORU(BTRFS_IOCTL_MAGIC,
60)
+#define TARGET_BTRFS_IOC_GET_SUBVOL_ROOTREF
TARGET_IOWRU(BTRFS_IOCTL_MAGIC, 61)
#define TARGET_BTRFS_IOC_INO_LOOKUP_USER
TARGET_IOWRU(BTRFS_IOCTL_MAGIC, 62)
/* usb ioctls */
diff --git a/linux-user/syscall_types.h b/linux-user/syscall_types.h
index b84e14235f..5566d101c2 100644
--- a/linux-user/syscall_types.h
+++ b/linux-user/syscall_types.h
@@ -431,6 +431,19 @@ STRUCT(btrfs_ioctl_dev_info_args,
MK_ARRAY(TYPE_CHAR, BTRFS_DEVICE_PATH_NAME_MAX)) /* path */
#endif
+#ifdef BTRFS_IOC_GET_SUBVOL_ROOTREF
+STRUCT(rootref,
+ TYPE_ULONGLONG, /* treeid */
+ TYPE_ULONGLONG) /* dirid */
+
+STRUCT(btrfs_ioctl_get_subvol_rootref_args,
+ TYPE_ULONGLONG, /* min_treeid */
+ MK_ARRAY(MK_STRUCT(STRUCT_rootref),
+BTRFS_MAX_ROOTREF_BUFFER_NUM), /* rootref */
+ TYPE_CHAR, /* num_items */
+ MK_ARRAY(TYPE_CHAR, 7)) /* align */
+#endif
+
#ifdef BTRFS_IOC_GET_DEV_STATS
STRUCT(btrfs_ioctl_get_dev_stats,
TYPE_ULONGLONG, /* devid */
--
2.25.1
[PATCH v3 1/8] linux-user: Add support for a group of btrfs ioctls used for subvolumes
This patch implements functionality of following ioctls:
BTRFS_IOC_SUBVOL_CREATE - Creating a btrfs subvolume
Create a btrfs subvolume. The subvolume is created using the ioctl's
third argument which represents a pointer to a following structure
type:
struct btrfs_ioctl_vol_args {
__s64 fd;
char name[BTRFS_PATH_NAME_MAX + 1];
};
Before calling this ioctl, the fields of this structure should be filled
with aproppriate values. The fd field represents the file descriptor
value of the subvolume and the name field represents the subvolume
path.
BTRFS_IOC_SUBVOL_GETFLAGS - Getting subvolume flags
Read the flags of the btrfs subvolume. The flags are read using
the ioctl's third argument that is a pointer of __u64 (unsigned long).
The third argument represents a bit mask that can be composed of following
values:
BTRFS_SUBVOL_RDONLY (1ULL << 1)
BTRFS_SUBVOL_QGROUP_INHERIT (1ULL << 2)
BTRFS_DEVICE_SPEC_BY_ID (1ULL << 3)
BTRFS_SUBVOL_SPEC_BY_ID (1ULL << 4)
BTRFS_IOC_SUBVOL_SETFLAGS - Setting subvolume flags
Set the flags of the btrfs subvolume. The flags are set using the
ioctl's third argument that is a pointer of __u64 (unsigned long).
The third argument represents a bit mask that can be composed of same
values as in the case of previous ioctl (BTRFS_IOC_SUBVOL_GETFLAGS).
BTRFS_IOC_SUBVOL_GETINFO - Getting subvolume information
Read information about the subvolume. The subvolume information is
returned in the ioctl's third argument which represents a pointer to
a following structure type:
struct btrfs_ioctl_get_subvol_info_args {
/* Id of this subvolume */
__u64 treeid;
/* Name of this subvolume, used to get the real name at mount point */
char name[BTRFS_VOL_NAME_MAX + 1];
/*
* Id of the subvolume which contains this subvolume.
* Zero for top-level subvolume or a deleted subvolume.
*/
__u64 parent_id;
/*
* Inode number of the directory which contains this subvolume.
* Zero for top-level subvolume or a deleted subvolume
*/
__u64 dirid;
/* Latest transaction id of this subvolume */
__u64 generation;
/* Flags of this subvolume */
__u64 flags;
/* UUID of this subvolume */
__u8 uuid[BTRFS_UUID_SIZE];
/*
* UUID of the subvolume of which this subvolume is a snapshot.
* All zero for a non-snapshot subvolume.
*/
__u8 parent_uuid[BTRFS_UUID_SIZE];
/*
* UUID of the subvolume from which this subvolume was received.
* All zero for non-received subvolume.
*/
__u8 received_uuid[BTRFS_UUID_SIZE];
/* Transaction id indicating when change/create/send/receive happened */
__u64 ctransid;
__u64 otransid;
__u64 stransid;
__u64 rtransid;
/* Time corresponding to c/o/s/rtransid */
struct btrfs_ioctl_timespec ctime;
struct btrfs_ioctl_timespec otime;
struct btrfs_ioctl_timespec stime;
struct btrfs_ioctl_timespec rtime;
/* Must be zero */
__u64 reserved[8];
};
All of the fields of this structure are filled after the ioctl call.
Implementation notes:
Ioctls BTRFS_IOC_SUBVOL_CREATE and BTRFS_IOC_SUBVOL_GETINFO have structure
types as third arguments. That is the reason why a corresponding definition
are added in file 'linux-user/syscall_types.h'.
The line '#include ' is added in file 'linux-user/syscall.c'
to
recognise preprocessor definitions for these ioctls. Since the file
"linux/btrfs.h"
was added in the kernel version 3.9, it is enwrapped in an #ifdef statement
with parameter CONFIG_BTRFS which is defined in 'configure' if the
header file is present.
Signed-off-by: Filip Bozuta
---
configure | 9 +
linux-user/ioctls.h| 15 +++
linux-user/syscall.c | 3 +++
linux-user/syscall_defs.h | 8
linux-user/syscall_types.h | 32
5 files changed, 67 insertions(+)
diff --git a/configure b/configure
index 4e5fe33211..a0747fe3ab 100755
--- a/configure
+++ b/configure
@@ -4954,6 +4954,12 @@ if check_include sys/kcov.h ; then
kcov=yes
fi
+# check for btrfs filesystem support (kernel must be 3.9+)
+btrfs=no
+if check_include linux/btrfs.h ; then
+btrfs=yes
+fi
+
# If we're making warnings fatal, apply this to Sphinx runs as well
sphinx_werror=""
if test "$werror" = "yes"; then
@@ -6938,6 +6944,9 @@ fi
if test "$kcov" = "yes" ; then
echo "CONFIG_KCOV=y" >> $config_host_mak
fi
+if test "$btrfs" = "yes" ; then
+ echo "CONFIG_BTRFS=y" >> $config_host_mak
+fi
if test "$inotify" = "yes" ; then
echo "CONFIG_INOTIFY=y" >> $config_host_mak
fi
diff --git a/linux-user/ioctls.h b/linux-user/ioc
Re: [PATCH v6 4/4] block: apply COR-filter to block-stream jobs
On 19.08.2020 13:46, Vladimir Sementsov-Ogievskiy wrote:
19.08.2020 00:24, Andrey Shinkevich wrote:
The patch completes the series with the COR-filter insertion to any
block-stream operation. It also makes changes to the iotests 030.
The test case 'test_stream_parallel' was deleted due to multiple
errors.
...
Signed-off-by: Andrey Shinkevich
---
block/stream.c | 76
--
tests/qemu-iotests/030 | 50 +++---
tests/qemu-iotests/030.out | 4 +--
3 files changed, 61 insertions(+), 69 deletions(-)
diff --git a/block/stream.c b/block/stream.c
index 8bf6b6d..0b11979 100644
--- a/block/stream.c
+++ b/block/stream.c
@@ -19,6 +19,7 @@
#include "qapi/qmp/qerror.h"
#include "qemu/ratelimit.h"
#include "sysemu/block-backend.h"
+#include "block/copy-on-read.h"
enum {
/*
@@ -33,8 +34,11 @@ typedef struct StreamBlockJob {
BlockJob common;
BlockDriverState *base_overlay; /* COW overlay (stream from
this) */
BlockDriverState *above_base; /* Node directly above the base */
+ BlockDriverState *cor_filter_bs;
+ BlockDriverState *target_bs;
BlockdevOnError on_error;
char *backing_file_str;
+ char *base_fmt;
bool bs_read_only;
bool chain_frozen;
} StreamBlockJob;
@@ -53,34 +57,26 @@ static void stream_abort(Job *job)
StreamBlockJob *s = container_of(job, StreamBlockJob, common.job);
if (s->chain_frozen) {
- BlockJob *bjob = &s->common;
- bdrv_unfreeze_backing_chain(blk_bs(bjob->blk), s->above_base);
+ bdrv_unfreeze_backing_chain(s->cor_filter_bs, s->above_base);
}
}
static int stream_prepare(Job *job)
{
StreamBlockJob *s = container_of(job, StreamBlockJob, common.job);
- BlockJob *bjob = &s->common;
- BlockDriverState *bs = blk_bs(bjob->blk);
+ BlockDriverState *bs = s->target_bs;
BlockDriverState *unfiltered_bs = bdrv_skip_filters(bs);
BlockDriverState *base = bdrv_filter_or_cow_bs(s->above_base);
Error *local_err = NULL;
int ret = 0;
- bdrv_unfreeze_backing_chain(bs, s->above_base);
+ bdrv_unfreeze_backing_chain(s->cor_filter_bs, s->above_base);
s->chain_frozen = false;
if (bdrv_cow_child(unfiltered_bs)) {
- const char *base_id = NULL, *base_fmt = NULL;
- if (base) {
- base_id = s->backing_file_str;
- if (base->drv) {
- base_fmt = base->drv->format_name;
- }
- }
bdrv_set_backing_hd(unfiltered_bs, base, &local_err);
- ret = bdrv_change_backing_file(unfiltered_bs, base_id,
base_fmt);
+ ret = bdrv_change_backing_file(unfiltered_bs,
s->backing_file_str,
+ s->base_fmt);
if (local_err) {
error_report_err(local_err);
return -EPERM;
@@ -94,7 +90,9 @@ static void stream_clean(Job *job)
{
StreamBlockJob *s = container_of(job, StreamBlockJob, common.job);
BlockJob *bjob = &s->common;
- BlockDriverState *bs = blk_bs(bjob->blk);
+ BlockDriverState *bs = s->target_bs;
+
+ bdrv_cor_filter_drop(s->cor_filter_bs);
/* Reopen the image back in read-only mode if necessary */
if (s->bs_read_only) {
@@ -104,13 +102,14 @@ static void stream_clean(Job *job)
}
g_free(s->backing_file_str);
+ g_free(s->base_fmt);
}
static int coroutine_fn stream_run(Job *job, Error **errp)
{
StreamBlockJob *s = container_of(job, StreamBlockJob, common.job);
BlockBackend *blk = s->common.blk;
- BlockDriverState *bs = blk_bs(blk);
+ BlockDriverState *bs = s->target_bs;
BlockDriverState *unfiltered_bs = bdrv_skip_filters(bs);
bool enable_cor = !bdrv_cow_child(s->base_overlay);
int64_t len;
@@ -231,6 +230,12 @@ void stream_start(const char *job_id,
BlockDriverState *bs,
int basic_flags = BLK_PERM_CONSISTENT_READ |
BLK_PERM_WRITE_UNCHANGED;
BlockDriverState *base_overlay = bdrv_find_overlay(bs, base);
BlockDriverState *above_base;
+ BlockDriverState *cor_filter_bs = NULL;
+ char *base_fmt = NULL;
+
+ if (base && base->drv) {
+ base_fmt = g_strdup(base->drv->format_name);
+ }
if (!base_overlay) {
error_setg(errp, "'%s' is not in the backing chain of '%s'",
@@ -264,17 +269,36 @@ void stream_start(const char *job_id,
BlockDriverState *bs,
}
}
- /* Prevent concurrent jobs trying to modify the graph
structure here, we
- * already have our own plans. Also don't allow resize as the
image size is
- * queried only at the job start and then cached. */
- s = block_job_create(job_id, &stream_job_driver, NULL, bs,
- basic_flags | BLK_PERM_GRAPH_MOD,
- basic_flags | BLK_PERM_WRITE,
+ cor_filter_bs = bdrv_cor_filter_append(bs, filter_node_name, errp);
+ if (cor_filter_bs == NULL) {
+ goto fail
Re: [PATCH v6 2/4] copy-on-read: add filter append/drop functions
On 19.08.2020 13:21, Vladimir Sementsov-Ogievskiy wrote:
19.08.2020 00:24, Andrey Shinkevich wrote:
Provide API for the COR-filter insertion/removal.
Also, drop the filter child permissions for an inactive state when the
filter node is being removed.
Signed-off-by: Andrey Shinkevich
---
block/copy-on-read.c | 103
+++
block/copy-on-read.h | 36 ++
2 files changed, 139 insertions(+)
create mode 100644 block/copy-on-read.h
diff --git a/block/copy-on-read.c b/block/copy-on-read.c
index cb03e0f..150d9b7 100644
--- a/block/copy-on-read.c
+++ b/block/copy-on-read.c
@@ -23,11 +23,21 @@
#include "qemu/osdep.h"
#include "block/block_int.h"
#include "qemu/module.h"
+#include "qapi/error.h"
+#include "qapi/qmp/qdict.h"
+#include "block/copy-on-read.h"
+
+
+typedef struct BDRVStateCOR {
+ bool active;
+} BDRVStateCOR;
static int cor_open(BlockDriverState *bs, QDict *options, int
flags,
Error **errp)
{
+ BDRVStateCOR *state = bs->opaque;
+
bs->file = bdrv_open_child(NULL, options, "file", bs,
&child_of_bds,
BDRV_CHILD_FILTERED |
BDRV_CHILD_PRIMARY,
false, errp);
@@ -42,6 +52,8 @@ static int cor_open(BlockDriverState *bs, QDict
*options, int flags,
((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) &
bs->file->bs->supported_zero_flags);
+ state->active = true;
We don't need to call bdrv_child_refresh_perms() here, as permissions
will be
updated soon, when the filter node get its parent, yes?
Let's add at least a comment on this.
+
return 0;
}
@@ -57,6 +69,17 @@ static void cor_child_perm(BlockDriverState *bs,
BdrvChild *c,
uint64_t perm, uint64_t shared,
uint64_t *nperm, uint64_t *nshared)
{
+ BDRVStateCOR *s = bs->opaque;
+
+ if (!s->active) {
+ /*
+ * While the filter is being removed
+ */
+ *nperm = 0;
+ *nshared = BLK_PERM_ALL;
+ return;
+ }
+
*nperm = perm & PERM_PASSTHROUGH;
*nshared = (shared & PERM_PASSTHROUGH) | PERM_UNCHANGED;
@@ -135,6 +158,7 @@ static void cor_lock_medium(BlockDriverState
*bs, bool locked)
static BlockDriver bdrv_copy_on_read = {
.format_name = "copy-on-read",
+ .instance_size = sizeof(BDRVStateCOR),
.bdrv_open = cor_open,
.bdrv_child_perm = cor_child_perm,
@@ -159,4 +183,83 @@ static void bdrv_copy_on_read_init(void)
bdrv_register(&bdrv_copy_on_read);
}
+
+static BlockDriverState *create_filter_node(BlockDriverState *bs,
+ const char
*filter_node_name,
+ Error **errp)
+{
+ QDict *opts = qdict_new();
+
+ qdict_put_str(opts, "driver", "copy-on-read");
+ qdict_put_str(opts, "file", bdrv_get_node_name(bs));
+ if (filter_node_name) {
+ qdict_put_str(opts, "node-name", filter_node_name);
+ }
+
+ return bdrv_open(NULL, NULL, opts, BDRV_O_RDWR, errp);
+}
+
+
+BlockDriverState *bdrv_cor_filter_append(BlockDriverState *bs,
+ const char *filter_node_name,
+ Error **errp)
+{
+ BlockDriverState *cor_filter_bs;
+ BDRVStateCOR *state;
+ Error *local_err = NULL;
+
+ cor_filter_bs = create_filter_node(bs, filter_node_name, errp);
+ if (cor_filter_bs == NULL) {
+ error_prepend(errp, "Could not create filter node: ");
+ return NULL;
+ }
+
+ if (!filter_node_name) {
+ cor_filter_bs->implicit = true;
+ }
+
+ bdrv_drained_begin(bs);
+ bdrv_replace_node(bs, cor_filter_bs, &local_err);
+ bdrv_drained_end(bs);
+
+ if (local_err) {
+ bdrv_unref(cor_filter_bs);
+ error_propagate(errp, local_err);
+ return NULL;
+ }
+
+ state = cor_filter_bs->opaque;
+ state->active = true;
hmm stop. it's already active, after create_filter_node, so you don't
need to set it again, isn't it?
I will remove the extra assignment from the cor_open() for consistancy.
Andrey
+
+ return cor_filter_bs;
+}
+
+
+void bdrv_cor_filter_drop(BlockDriverState *cor_filter_bs)
+{
+ BdrvChild *child;
+ BlockDriverState *bs;
+ BDRVStateCOR *s = cor_filter_bs->opaque;
+
+ child = bdrv_filter_child(cor_filter_bs);
+ if (!child) {
+ return;
+ }
+ bs = child->bs;
+
+ /* Retain the BDS until we complete the graph change. */
+ bdrv_ref(bs);
+ /* Hold a guest back from writing while permissions are being
reset. */
+ bdrv_drained_begin(bs);
+ /* Drop permissions before the graph change. */
+ s->active = false;
+ bdrv_child_refresh_perms(cor_filter_bs, child, &error_abort);
+ bdrv_replace_node(cor_filter_bs, bs, &error_
Re: [PATCH v6 3/4] qapi: add filter-node-name to block-stream
On 19.08.2020 13:29, Vladimir Sementsov-Ogievskiy wrote:
19.08.2020 00:24, Andrey Shinkevich wrote:
Provide the possibility to pass the 'filter-node-name' parameter to the
block-stream job as it is done for the commit block job. That will be
needed for further iotests implementations.
and for users as well. I think, the last sentence may be dropped.
Signed-off-by: Andrey Shinkevich
---
block/monitor/block-hmp-cmds.c | 4 ++--
block/stream.c | 4 +++-
blockdev.c | 8 +++-
include/block/block_int.h | 7 ++-
qapi/block-core.json | 6 ++
5 files changed, 24 insertions(+), 5 deletions(-)
diff --git a/block/monitor/block-hmp-cmds.c
b/block/monitor/block-hmp-cmds.c
index 4d3db5e..4e66775 100644
--- a/block/monitor/block-hmp-cmds.c
+++ b/block/monitor/block-hmp-cmds.c
@@ -507,8 +507,8 @@ void hmp_block_stream(Monitor *mon, const QDict
*qdict)
qmp_block_stream(true, device, device, base != NULL, base,
false, NULL,
false, NULL, qdict_haskey(qdict, "speed"),
speed, true,
- BLOCKDEV_ON_ERROR_REPORT, false, false, false,
false,
- &error);
+ BLOCKDEV_ON_ERROR_REPORT, false, NULL, false,
false, false,
+ false, &error);
hmp_handle_error(mon, error);
}
diff --git a/block/stream.c b/block/stream.c
index b9c1141..8bf6b6d 100644
--- a/block/stream.c
+++ b/block/stream.c
@@ -221,7 +221,9 @@ static const BlockJobDriver stream_job_driver = {
void stream_start(const char *job_id, BlockDriverState *bs,
BlockDriverState *base, const char
*backing_file_str,
int creation_flags, int64_t speed,
- BlockdevOnError on_error, Error **errp)
+ BlockdevOnError on_error,
+ const char *filter_node_name,
+ Error **errp)
{
StreamBlockJob *s;
BlockDriverState *iter;
diff --git a/blockdev.c b/blockdev.c
index 237fffb..800ecb3 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -2476,6 +2476,7 @@ void qmp_block_stream(bool has_job_id, const
char *job_id, const char *device,
bool has_backing_file, const char *backing_file,
bool has_speed, int64_t speed,
bool has_on_error, BlockdevOnError on_error,
+ bool has_filter_node_name, const char
*filter_node_name,
bool has_auto_finalize, bool auto_finalize,
bool has_auto_dismiss, bool auto_dismiss,
Error **errp)
@@ -2491,6 +2492,10 @@ void qmp_block_stream(bool has_job_id, const
char *job_id, const char *device,
on_error = BLOCKDEV_ON_ERROR_REPORT;
}
+ if (!has_filter_node_name) {
+ filter_node_name = NULL;
+ }
this works automatically, you don't need to initialize it by hand
In the current impementation of QEMU, it may look like an excess. I will
remove it. But the qmp_block_commit() has that check.
Andrey
+
bs = bdrv_lookup_bs(device, device, errp);
if (!bs) {
return;
@@ -2558,7 +2563,8 @@ void qmp_block_stream(bool has_job_id, const
char *job_id, const char *device,
}
stream_start(has_job_id ? job_id : NULL, bs, base_bs, base_name,
- job_flags, has_speed ? speed : 0, on_error,
&local_err);
+ job_flags, has_speed ? speed : 0, on_error,
+ filter_node_name, &local_err);
if (local_err) {
error_propagate(errp, local_err);
goto out;
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 465a601..3efde33 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -1122,6 +1122,9 @@ int is_windows_drive(const char *filename);
* See @BlockJobCreateFlags
* @speed: The maximum speed, in bytes per second, or 0 for unlimited.
* @on_error: The action to take upon error.
+ * @filter_node_name: The node name that should be assigned to the
filter
+ * driver that the commit job inserts into the graph above @bs. NULL
means
+ * that a node name should be autogenerated.
* @errp: Error object.
*
* Start a streaming operation on @bs. Clusters that are unallocated
@@ -1134,7 +1137,9 @@ int is_windows_drive(const char *filename);
void stream_start(const char *job_id, BlockDriverState *bs,
BlockDriverState *base, const char
*backing_file_str,
int creation_flags, int64_t speed,
- BlockdevOnError on_error, Error **errp);
+ BlockdevOnError on_error,
+ const char *filter_node_name,
+ Error **errp);
/**
* commit_start:
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 0b8ccd3..1db6ce1 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -2524,6 +2524,11 @@
# 'stop' and '
Re: [PATCH 04/12] elf2dmp/pdb: Plug memleak in pdb_init_from_file
On Fri, 14 Aug 2020 12:02:33 -0400
Pan Nengyuan wrote:
> Missing g_error_free in pdb_init_from_file() error path. Fix that.
>
> Reported-by: Euler Robot
> Signed-off-by: Pan Nengyuan
> ---
> Cc: Viktor Prutyanov
> ---
> contrib/elf2dmp/pdb.c | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/contrib/elf2dmp/pdb.c b/contrib/elf2dmp/pdb.c
> index a5bd40c99d..b3a6547068 100644
> --- a/contrib/elf2dmp/pdb.c
> +++ b/contrib/elf2dmp/pdb.c
> @@ -285,6 +285,7 @@ int pdb_init_from_file(const char *name, struct
> pdb_reader *reader) reader->gmf = g_mapped_file_new(name, TRUE,
> &gerr); if (gerr) {
> eprintf("Failed to map PDB file \'%s\'\n", name);
> +g_error_free(gerr);
> return 1;
> }
>
Reviewed-by: Viktor Prutyanov
--
Viktor Prutyanov
Re: [PATCH 03/12] elf2dmp/qemu_elf: Plug memleak in QEMU_Elf_init
On Fri, 14 Aug 2020 12:02:32 -0400
Pan Nengyuan wrote:
> Missing g_error_free in QEMU_Elf_init() error path. Fix that.
>
> Reported-by: Euler Robot
> Signed-off-by: Pan Nengyuan
> ---
> Cc: Viktor Prutyanov
> ---
> contrib/elf2dmp/qemu_elf.c | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/contrib/elf2dmp/qemu_elf.c b/contrib/elf2dmp/qemu_elf.c
> index 0db7816586..b601b6d7ba 100644
> --- a/contrib/elf2dmp/qemu_elf.c
> +++ b/contrib/elf2dmp/qemu_elf.c
> @@ -126,6 +126,7 @@ int QEMU_Elf_init(QEMU_Elf *qe, const char
> *filename) qe->gmf = g_mapped_file_new(filename, TRUE, &gerr);
> if (gerr) {
> eprintf("Failed to map ELF dump file \'%s\'\n", filename);
> +g_error_free(gerr);
> return 1;
> }
>
Reviewed-by: Viktor Prutyanov
--
Viktor Prutyanov
Re: [PULL 0/6] Linux user for 5.2 patches
On Sun, 23 Aug 2020 at 16:00, Laurent Vivier wrote: > > The following changes since commit d7df0ceee0fd2e512cd214a9074ebeeb40da3099: > > Merge remote-tracking branch 'remotes/philmd-gitlab/tags/sd-next-20200821' = > into staging (2020-08-22 23:53:08 +0100) > > are available in the Git repository at: > > git://github.com/vivier/qemu.git tags/linux-user-for-5.2-pull-request > > for you to fetch changes up to b3a3af70c377a3e67d43f3be39a333228487b50c: > > linux-user: Fix 'utimensat()' implementation (2020-08-23 16:57:58 +0200) > > > Add clock_getres_time64, timer_gettime64, timer_settime64, > timerfd_gettime64, timerfd_settime64 > Some fixes (page protection, print_fdset, timerspec, itimerspec) > > Applied, thanks. Please update the changelog at https://wiki.qemu.org/ChangeLog/5.2 for any user-visible changes. -- PMM
Re: [PATCH] scripts/qemu-version.sh: Add missing space before ']'
Hi
On Sun, Aug 23, 2020 at 2:26 PM Thomas Huth wrote:
>
> When configure has been run with --with-pkgversion=xyz, the shell complains
> about a missing ']' in this script.
>
> Fixes: 2c273f32d3 ("meson: generate qemu-version.h")
> Signed-off-by: Thomas Huth
oops, my bad
Reviewed-by: Marc-André Lureau
> ---
> scripts/qemu-version.sh | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/scripts/qemu-version.sh b/scripts/qemu-version.sh
> index 4847385e42..03128c56a2 100755
> --- a/scripts/qemu-version.sh
> +++ b/scripts/qemu-version.sh
> @@ -6,7 +6,7 @@ dir="$1"
> pkgversion="$2"
> version="$3"
>
> -if [ -z "$pkgversion"]; then
> +if [ -z "$pkgversion" ]; then
> cd "$dir"
> if [ -e .git ]; then
> pkgversion=$(git describe --match 'v*' --dirty | echo "")
> --
> 2.18.2
>
Re: [PATCH-for-5.2] exec: Remove MemoryRegion::global_locking field
On Sat, 22 Aug 2020 at 16:13, Philippe Mathieu-Daudé wrote:
>
> +Robert
>
> On 8/7/20 12:16 PM, Paolo Bonzini wrote:
> > On 07/08/20 12:02, Stefan Hajnoczi wrote:
> >> On Thu, Aug 06, 2020 at 05:07:26PM +0200, Philippe Mathieu-Daudé wrote:
> >>> Last uses of memory_region_clear_global_locking() have been
> >>> removed in commit 7070e085d4 ("acpi: mark PMTIMER as unlocked")
> >>> and commit 08565552f7 ("cputlb: Move NOTDIRTY handling from I/O
> >>> path to TLB path").
> >>> Remove memory_region_clear_global_locking() and the now unused
> >>> 'global_locking' field in MemoryRegion.
> >>>
> >>> Reported-by: Alexander Bulekov
> >>> Suggested-by: Stefan Hajnoczi
> >>> Signed-off-by: Philippe Mathieu-Daudé
> >>> ---
> >>> include/exec/memory.h | 14 --
> >>> accel/tcg/cputlb.c| 4 ++--
> >>> exec.c| 2 +-
> >>> softmmu/memory.c | 6 --
> >>> 4 files changed, 3 insertions(+), 23 deletions(-)
> >>
> >> It can be added back in later, if necessary. For now let's drop the dead
> >> code.
> >>
> >> Reviewed-by: Stefan Hajnoczi
> >>
> >
> > I expect it will come back since Linaro is working on BQL-free interrupt
> > handling for TCG, but no objections.
>
> Robert, any comment on this patch?
>
> Thanks,
>
> Phil.
>
Phil, Thanks for the heads up on this !
No objections to removing this. We can easily re-add it if/when needed.
Thanks,
-Rob
[PULL 4/6] linux-user: Modify 'target_to_host/host_to_target_itimerspec()'
From: Filip Bozuta
Functions 'target_to_host_itimerspec()' and 'host_to_target_itimerspec()'
are used to convert values of type 'struct itimerspec' between target and
host. This type has 'struct timespec' as its fields. That is the reason
why this patch introduces a little modification to the converting functions
to be implemented using already existing functions that convert 'struct
timespec':
'target_to_host_timespec()' and 'host_to_target_timespec()'. This makes the
code of 'target_to_host_itimerspec()' and 'host_to_target_itimerspec()' more
clean and readable.
Signed-off-by: Filip Bozuta
Reviewed-by: Laurent Vivier
Message-Id: <20200722153421.295411-2-filip.boz...@syrmia.com>
Signed-off-by: Laurent Vivier
---
linux-user/syscall.c | 46 ++--
1 file changed, 19 insertions(+), 27 deletions(-)
diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index 945fc252791c..aea1160804a2 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -1229,7 +1229,9 @@ static inline abi_long copy_to_user_timeval64(abi_ulong
target_tv_addr,
defined(TARGET_NR_nanosleep) || defined(TARGET_NR_clock_settime) || \
defined(TARGET_NR_utimensat) || defined(TARGET_NR_mq_timedsend) || \
defined(TARGET_NR_mq_timedreceive) || defined(TARGET_NR_ipc) || \
-defined(TARGET_NR_semop) || defined(TARGET_NR_semtimedop)
+defined(TARGET_NR_semop) || defined(TARGET_NR_semtimedop) || \
+defined(TARGET_NR_timer_settime) || \
+(defined(TARGET_NR_timerfd_settime) && defined(CONFIG_TIMERFD))
static inline abi_long target_to_host_timespec(struct timespec *host_ts,
abi_ulong target_addr)
{
@@ -6783,46 +6785,36 @@ static inline abi_long target_ftruncate64(void
*cpu_env, abi_long arg1,
#if defined(TARGET_NR_timer_settime) || \
(defined(TARGET_NR_timerfd_settime) && defined(CONFIG_TIMERFD))
-static inline abi_long target_to_host_itimerspec(struct itimerspec
*host_itspec,
+static inline abi_long target_to_host_itimerspec(struct itimerspec *host_its,
abi_ulong target_addr)
{
-struct target_itimerspec *target_itspec;
-
-if (!lock_user_struct(VERIFY_READ, target_itspec, target_addr, 1)) {
+if (target_to_host_timespec(&host_its->it_interval, target_addr +
+offsetof(struct target_itimerspec,
+ it_interval)) ||
+target_to_host_timespec(&host_its->it_value, target_addr +
+offsetof(struct target_itimerspec,
+ it_value))) {
return -TARGET_EFAULT;
}
-host_itspec->it_interval.tv_sec =
-tswapal(target_itspec->it_interval.tv_sec);
-host_itspec->it_interval.tv_nsec =
-tswapal(target_itspec->it_interval.tv_nsec);
-host_itspec->it_value.tv_sec = tswapal(target_itspec->it_value.tv_sec);
-host_itspec->it_value.tv_nsec = tswapal(target_itspec->it_value.tv_nsec);
-
-unlock_user_struct(target_itspec, target_addr, 1);
return 0;
}
#endif
#if ((defined(TARGET_NR_timerfd_gettime) || \
defined(TARGET_NR_timerfd_settime)) && defined(CONFIG_TIMERFD)) || \
-defined(TARGET_NR_timer_gettime) || defined(TARGET_NR_timer_settime)
+ defined(TARGET_NR_timer_gettime) || defined(TARGET_NR_timer_settime)
static inline abi_long host_to_target_itimerspec(abi_ulong target_addr,
- struct itimerspec *host_its)
-{
-struct target_itimerspec *target_itspec;
-
-if (!lock_user_struct(VERIFY_WRITE, target_itspec, target_addr, 0)) {
+ struct itimerspec *host_its)
+{
+if (host_to_target_timespec(target_addr + offsetof(struct
target_itimerspec,
+ it_interval),
+&host_its->it_interval) ||
+host_to_target_timespec(target_addr + offsetof(struct
target_itimerspec,
+ it_value),
+&host_its->it_value)) {
return -TARGET_EFAULT;
}
-
-target_itspec->it_interval.tv_sec = tswapal(host_its->it_interval.tv_sec);
-target_itspec->it_interval.tv_nsec =
tswapal(host_its->it_interval.tv_nsec);
-
-target_itspec->it_value.tv_sec = tswapal(host_its->it_value.tv_sec);
-target_itspec->it_value.tv_nsec = tswapal(host_its->it_value.tv_nsec);
-
-unlock_user_struct(target_itspec, target_addr, 0);
return 0;
}
#endif
--
2.26.2
[PULL 6/6] linux-user: Fix 'utimensat()' implementation
From: Filip Bozuta
Implementation of syscall 'utimensat()' in 'syscall.c' uses functions
target_to_host/host_to_target_timespec() to convert values of
'struct timespec' between host and target. However, the implementation
doesn't check whether the conversion succeeds and thus can cause an
inappropriate error or succeed unappropriately instead of setting errno
EFAULT ('Bad address') which is supposed to be set in these cases.
This was confirmed with the LTP test for utimensat ('testcases/utimensat')
which fails for test cases when the errno EFAULT is expected. After changes
from this patch, the test passes for all test cases.
Signed-off-by: Filip Bozuta
Reviewed-by: Laurent Vivier
Message-Id: <2020083101.6636-1-filip.boz...@syrmia.com>
Signed-off-by: Laurent Vivier
---
linux-user/syscall.c | 9 +++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index bbb61a59c72f..b4a7b605f3d4 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -11919,8 +11919,13 @@ static abi_long do_syscall1(void *cpu_env, int num,
abi_long arg1,
if (!arg3) {
tsp = NULL;
} else {
-target_to_host_timespec(ts, arg3);
-target_to_host_timespec(ts+1, arg3+sizeof(struct
target_timespec));
+if (target_to_host_timespec(ts, arg3)) {
+return -TARGET_EFAULT;
+}
+if (target_to_host_timespec(ts + 1, arg3 +
+sizeof(struct target_timespec))) {
+return -TARGET_EFAULT;
+}
tsp = ts;
}
if (!arg2)
--
2.26.2
[PULL 5/6] linux-user: Add support for a group of 2038 safe syscalls
From: Filip Bozuta
This patch implements functionality for following time64 syscalls:
*clock_getres_time64
This a year 2038 safe variant of syscall:
int clock_getres(clockid_t clockid, struct timespec *res)
--finding the resoultion of a specified clock--
man page: https://man7.org/linux/man-pages/man2/clock_getres.2.html
*timer_gettime64
*timer_settime64
These are year 2038 safe variants of syscalls:
int timer_settime(timer_t timerid, int flags,
const struct itimerspec *new_value,
struct itimerspec *old_value)
int timer_gettime(timer_t timerid, struct itimerspec *curr_value)
--arming/dissarming and fetching state of POSIX per-process timer--
man page: https://man7.org/linux/man-pages/man2/timer_settime.2.html
*timerfd_gettime64
*timerfd_settime64
These are year 2038 safe variants of syscalls:
int timerfd_settime(int fd, int flags,
const struct itimerspec *new_value,
struct itimerspec *old_value)
int timerfd_gettime(int fd, struct itimerspec *curr_value)
--timers that notify via file descriptor--
man page: https://man7.org/linux/man-pages/man2/timerfd_settime.2.html
Implementation notes:
Syscall 'clock_getres_time64' was implemented similarly to
'clock_getres()'.
The only difference was that for the conversion of 'struct timespec' from
host to target, function 'host_to_target_timespec64()' was used instead of
'host_to_target_timespec()'.
For other syscalls, new functions 'host_to_target_itimerspec64()' and
'target_to_host_itimerspec64()' were added to convert the value of the
'struct itimerspec' from host to target and vice versa. A new type
'struct target__kernel_itimerspec' was added in 'syscall_defs.h'. This
type was defined with fields which are of the already defined type
'struct target_timespec'. This new 'struct target__kernel_itimerspec'
type is used in these new converting functions. These new functions were
defined similarly to 'host_to_target_itimerspec()' and
'target_to_host_itimerspec()'
the only difference being that 'target_to_host_timespec64()' and
'host_to_target_timespec64()' were used.
Signed-off-by: Filip Bozuta
Reviewed-by: Laurent Vivier
Message-Id: <20200722153421.295411-3-filip.boz...@syrmia.com>
Signed-off-by: Laurent Vivier
---
linux-user/syscall.c | 139 +-
linux-user/syscall_defs.h | 5 ++
2 files changed, 143 insertions(+), 1 deletion(-)
diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index aea1160804a2..bbb61a59c72f 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -1247,7 +1247,9 @@ static inline abi_long target_to_host_timespec(struct
timespec *host_ts,
}
#endif
-#if defined(TARGET_NR_clock_settime64) || defined(TARGET_NR_futex_time64)
+#if defined(TARGET_NR_clock_settime64) || defined(TARGET_NR_futex_time64) || \
+defined(TARGET_NR_timer_settime64) || \
+(defined(TARGET_NR_timerfd_settime64) && defined(CONFIG_TIMERFD))
static inline abi_long target_to_host_timespec64(struct timespec *host_ts,
abi_ulong target_addr)
{
@@ -6801,6 +6803,24 @@ static inline abi_long target_to_host_itimerspec(struct
itimerspec *host_its,
}
#endif
+#if defined(TARGET_NR_timer_settime64) || \
+(defined(TARGET_NR_timerfd_settime64) && defined(CONFIG_TIMERFD))
+static inline abi_long target_to_host_itimerspec64(struct itimerspec *host_its,
+ abi_ulong target_addr)
+{
+if (target_to_host_timespec64(&host_its->it_interval, target_addr +
+ offsetof(struct target__kernel_itimerspec,
+ it_interval)) ||
+target_to_host_timespec64(&host_its->it_value, target_addr +
+ offsetof(struct target__kernel_itimerspec,
+ it_value))) {
+return -TARGET_EFAULT;
+}
+
+return 0;
+}
+#endif
+
#if ((defined(TARGET_NR_timerfd_gettime) || \
defined(TARGET_NR_timerfd_settime)) && defined(CONFIG_TIMERFD)) || \
defined(TARGET_NR_timer_gettime) || defined(TARGET_NR_timer_settime)
@@ -6819,6 +6839,26 @@ static inline abi_long
host_to_target_itimerspec(abi_ulong target_addr,
}
#endif
+#if ((defined(TARGET_NR_timerfd_gettime64) || \
+ defined(TARGET_NR_timerfd_settime64)) && defined(CONFIG_TIMERFD)) || \
+ defined(TARGET_NR_timer_gettime64) || defined(TARGET_NR_timer_settime64)
+static inline abi_long host_to_target_itimerspec64(abi_ulong target_addr,
+ struct itimerspec *host_its)
+{
+if (host_to_target_timespec64(target_addr +
+ offsetof(struct target__kernel_itimerspec,
+
[PULL 0/6] Linux user for 5.2 patches
The following changes since commit d7df0ceee0fd2e512cd214a9074ebeeb40da3099: Merge remote-tracking branch 'remotes/philmd-gitlab/tags/sd-next-20200821' = into staging (2020-08-22 23:53:08 +0100) are available in the Git repository at: git://github.com/vivier/qemu.git tags/linux-user-for-5.2-pull-request for you to fetch changes up to b3a3af70c377a3e67d43f3be39a333228487b50c: linux-user: Fix 'utimensat()' implementation (2020-08-23 16:57:58 +0200) Add clock_getres_time64, timer_gettime64, timer_settime64, timerfd_gettime64, timerfd_settime64 Some fixes (page protection, print_fdset, timerspec, itimerspec) Filip Bozuta (4): linux-user: Fix "print_fdset()" in "strace.c" to not print ", " after last value linux-user: Modify 'target_to_host/host_to_target_itimerspec()' linux-user: Add support for a group of 2038 safe syscalls linux-user: Fix 'utimensat()' implementation Richard Henderson (2): linux-user: Validate mmap/mprotect prot value linux-user: Adjust guest page protection for the host linux-user/mmap.c | 110 +++--- linux-user/strace.c | 8 +- linux-user/syscall.c | 190 -- linux-user/syscall_defs.h | 5 + 4 files changed, 250 insertions(+), 63 deletions(-) --=20 2.26.2
[PULL 2/6] linux-user: Validate mmap/mprotect prot value
From: Richard Henderson
The kernel will return -EINVAL for bits set in the prot argument
that are unknown or invalid. Previously we were simply cropping
out the bits that we care about.
Introduce validate_prot_to_pageflags to perform this check in a
single place between the two syscalls. Differentiate between
the target and host versions of prot. Compute the qemu internal
page_flags value at the same time.
Signed-off-by: Richard Henderson
Reviewed-by: Peter Maydell
Message-Id: <20200519185645.3915-2-richard.hender...@linaro.org>
Signed-off-by: Laurent Vivier
---
linux-user/mmap.c | 106 +++---
1 file changed, 73 insertions(+), 33 deletions(-)
diff --git a/linux-user/mmap.c b/linux-user/mmap.c
index 0019447892e0..46c7eeba9bd2 100644
--- a/linux-user/mmap.c
+++ b/linux-user/mmap.c
@@ -59,64 +59,96 @@ void mmap_fork_end(int child)
pthread_mutex_unlock(&mmap_mutex);
}
+/*
+ * Validate target prot bitmask.
+ * Return the prot bitmask for the host in *HOST_PROT.
+ * Return 0 if the target prot bitmask is invalid, otherwise
+ * the internal qemu page_flags (which will include PAGE_VALID).
+ */
+static int validate_prot_to_pageflags(int *host_prot, int prot)
+{
+int valid = PROT_READ | PROT_WRITE | PROT_EXEC | TARGET_PROT_SEM;
+int page_flags = (prot & PAGE_BITS) | PAGE_VALID;
+
+/*
+ * For the host, we need not pass anything except read/write/exec.
+ * While PROT_SEM is allowed by all hosts, it is also ignored, so
+ * don't bother transforming guest bit to host bit. Any other
+ * target-specific prot bits will not be understood by the host
+ * and will need to be encoded into page_flags for qemu emulation.
+ */
+*host_prot = prot & (PROT_READ | PROT_WRITE | PROT_EXEC);
+
+return prot & ~valid ? 0 : page_flags;
+}
+
/* NOTE: all the constants are the HOST ones, but addresses are target. */
-int target_mprotect(abi_ulong start, abi_ulong len, int prot)
+int target_mprotect(abi_ulong start, abi_ulong len, int target_prot)
{
abi_ulong end, host_start, host_end, addr;
-int prot1, ret;
+int prot1, ret, page_flags, host_prot;
-trace_target_mprotect(start, len, prot);
+trace_target_mprotect(start, len, target_prot);
-if ((start & ~TARGET_PAGE_MASK) != 0)
+if ((start & ~TARGET_PAGE_MASK) != 0) {
return -TARGET_EINVAL;
+}
+page_flags = validate_prot_to_pageflags(&host_prot, target_prot);
+if (!page_flags) {
+return -TARGET_EINVAL;
+}
len = TARGET_PAGE_ALIGN(len);
end = start + len;
if (!guest_range_valid(start, len)) {
return -TARGET_ENOMEM;
}
-prot &= PROT_READ | PROT_WRITE | PROT_EXEC;
-if (len == 0)
+if (len == 0) {
return 0;
+}
mmap_lock();
host_start = start & qemu_host_page_mask;
host_end = HOST_PAGE_ALIGN(end);
if (start > host_start) {
/* handle host page containing start */
-prot1 = prot;
-for(addr = host_start; addr < start; addr += TARGET_PAGE_SIZE) {
+prot1 = host_prot;
+for (addr = host_start; addr < start; addr += TARGET_PAGE_SIZE) {
prot1 |= page_get_flags(addr);
}
if (host_end == host_start + qemu_host_page_size) {
-for(addr = end; addr < host_end; addr += TARGET_PAGE_SIZE) {
+for (addr = end; addr < host_end; addr += TARGET_PAGE_SIZE) {
prot1 |= page_get_flags(addr);
}
end = host_end;
}
-ret = mprotect(g2h(host_start), qemu_host_page_size, prot1 &
PAGE_BITS);
-if (ret != 0)
+ret = mprotect(g2h(host_start), qemu_host_page_size,
+ prot1 & PAGE_BITS);
+if (ret != 0) {
goto error;
+}
host_start += qemu_host_page_size;
}
if (end < host_end) {
-prot1 = prot;
-for(addr = end; addr < host_end; addr += TARGET_PAGE_SIZE) {
+prot1 = host_prot;
+for (addr = end; addr < host_end; addr += TARGET_PAGE_SIZE) {
prot1 |= page_get_flags(addr);
}
-ret = mprotect(g2h(host_end - qemu_host_page_size),
qemu_host_page_size,
- prot1 & PAGE_BITS);
-if (ret != 0)
+ret = mprotect(g2h(host_end - qemu_host_page_size),
+ qemu_host_page_size, prot1 & PAGE_BITS);
+if (ret != 0) {
goto error;
+}
host_end -= qemu_host_page_size;
}
/* handle the pages in the middle */
if (host_start < host_end) {
-ret = mprotect(g2h(host_start), host_end - host_start, prot);
-if (ret != 0)
+ret = mprotect(g2h(host_start), host_end - host_start, host_prot);
+if (ret != 0) {
goto error;
+}
}
-page_set_flags(start, start + len, prot | PAGE_VALID);
+page_set_flags(start, start + len, page_flags);
mmap_unlock();
return 0;
error
[PULL 1/6] linux-user: Fix "print_fdset()" in "strace.c" to not print ", " after last value
From: Filip Bozuta
Function "print_fdset()" in "strace.c" is used to print the file descriptor
values in "print__newselect()" which prints arguments of syscall _newselect().
Until changes from this patch, this function was printing "," even after the
last value of the fd_set argument. This was changed in this patch by removing
this unnecessary "," after the last fd value and thus improving the estetics of
the _newselect() "-strace" print.
Implementation notes:
The printing fix was made possible by using an existing function
"get_comma()"
which returns a "," or an empty string "" based on its argument (0 for ","
and
other for "").
Signed-off-by: Filip Bozuta
Reviewed-by: Laurent Vivier
Message-Id: <20200702160915.9517-1-filip.boz...@syrmia.com>
Signed-off-by: Laurent Vivier
---
linux-user/strace.c | 8 ++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/linux-user/strace.c b/linux-user/strace.c
index 13981341b327..5e380486433c 100644
--- a/linux-user/strace.c
+++ b/linux-user/strace.c
@@ -541,6 +541,7 @@ static void
print_fdset(int n, abi_ulong target_fds_addr)
{
int i;
+int first = 1;
qemu_log("[");
if( target_fds_addr ) {
@@ -555,9 +556,12 @@ print_fdset(int n, abi_ulong target_fds_addr)
return;
for (i=n; i>=0; i--) {
-if ((tswapal(target_fds[i / TARGET_ABI_BITS]) >> (i &
(TARGET_ABI_BITS - 1))) & 1)
-qemu_log("%d,", i);
+if ((tswapal(target_fds[i / TARGET_ABI_BITS]) >>
+(i & (TARGET_ABI_BITS - 1))) & 1) {
+qemu_log("%s%d", get_comma(first), i);
+first = 0;
}
+}
unlock_user(target_fds, target_fds_addr, 0);
}
qemu_log("]");
--
2.26.2
[PULL 3/6] linux-user: Adjust guest page protection for the host
From: Richard Henderson Executable guest pages are never directly executed by the host, but do need to be readable for translation. Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Peter Maydell Message-Id: <20200519185645.3915-3-richard.hender...@linaro.org> Signed-off-by: Laurent Vivier --- linux-user/mmap.c | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/linux-user/mmap.c b/linux-user/mmap.c index 46c7eeba9bd2..f2615634201d 100644 --- a/linux-user/mmap.c +++ b/linux-user/mmap.c @@ -76,8 +76,12 @@ static int validate_prot_to_pageflags(int *host_prot, int prot) * don't bother transforming guest bit to host bit. Any other * target-specific prot bits will not be understood by the host * and will need to be encoded into page_flags for qemu emulation. + * + * Pages that are executable by the guest will never be executed + * by the host, but the host will need to be able to read them. */ -*host_prot = prot & (PROT_READ | PROT_WRITE | PROT_EXEC); +*host_prot = (prot & (PROT_READ | PROT_WRITE)) + | (prot & PROT_EXEC ? PROT_READ : 0); return prot & ~valid ? 0 : page_flags; } -- 2.26.2
Re: [PULL] nvme updates
On Wed, 19 Aug 2020 at 20:23, Keith Busch wrote: > > We're trying our first nvme pull request from a dedicated development > tree containing various fixes, cleanups, spec compliance, and welcoming > Klaus Jensen to maintaining the emulated nvme device development. > > The following changes since commit d0ed6a69d399ae193959225cdeaa9382746c91cc: > > Update version for v5.1.0 release (2020-08-11 17:07:03 +0100) > > are available in the Git repository at: > > git://git.infradead.org/qemu-nvme.git nvme-next Hi; it looks like this isn't a gpg-signed tag? error: remotes/nvme/nvme-next: cannot verify a non-tag object of type commit. thanks -- PMM
[PATCH] configure: avoid compiling qemu-keymap by default
qemu-keymap is not needed with linux-user, so disable it by default if tools are disabled (tools are disabled by default with linux-user). Avoid this error with statically linked binaries: Linking target qemu-keymap /usr/bin/ld: cannot find -lxkbcommon Signed-off-by: Laurent Vivier --- configure | 5 + 1 file changed, 5 insertions(+) diff --git a/configure b/configure index d9ca87fbbb52..2cab3330d010 100755 --- a/configure +++ b/configure @@ -3448,6 +3448,11 @@ fi ## # xkbcommon probe +if test -z "$xkbcommon"; then + if test "$want_tools" = "no"; then +xkbcommon=no + fi +fi if test "$xkbcommon" != "no" ; then if $pkg_config xkbcommon --exists; then xkbcommon_cflags=$($pkg_config xkbcommon --cflags) -- 2.26.2
Re: [PULL 00/40] ppc-for-5.2 queue 20200818
On Tue, 18 Aug 2020 at 05:19, David Gibson wrote: > > The following changes since commit d0ed6a69d399ae193959225cdeaa9382746c91cc: > > Update version for v5.1.0 release (2020-08-11 17:07:03 +0100) > > are available in the Git repository at: > > git://github.com/dgibson/qemu.git tags/ppc-for-5.2-20200818 > > for you to fetch changes up to 3110f0ee19ccdb50adff3dfa1321039f69efddcd: > > spapr/xive: Use xive_source_esb_len() (2020-08-14 13:35:45 +1000) > > > ppc patch queue 2020-08-18 > > Here's my first pull request for qemu-5.2, which has quite a few > accumulated things. Highlights are: > > * Preliminary support for POWER10 (Power ISA 3.1) instruction emulation > * Add documentation on the (very confusing) pseries NUMA configuration > * Fix some bugs handling edge cases with XICS, XIVE and kernel_irqchip > * Fix icount for a number of POWER registers > * Many cleanups to error handling in XIVE code > * Validate size of -prom-env data Hi -- it looks like you've updated the tag but I haven't seen a new cover letter. Do you want me to apply it? thanks -- PMM
Re: [PULL v3 00/20] riscv-to-apply queue
On Sat, 22 Aug 2020 at 06:53, Alistair Francis wrote: > > The following changes since commit f86d9a093dada59bde5582c7ec320487c4b8: > > Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' > into staging (2020-08-21 17:26:52 +0100) > > are available in the Git repository at: > > g...@github.com:alistair23/qemu.git tags/pull-riscv-to-apply-20200821-1 > > for you to fetch changes up to 01c41d15de13104774d08e951db24815c8cffc79: > > hw/intc: ibex_plic: Honour source priorities (2020-08-21 22:37:55 -0700) > > > The first RISC-V PR for the 5.2 window. > > This includes: > - NaNBox fixes > - Vector extension improvements > - a L2 cache controller > - PMP fixes > - Upgrade to OpenSBI v0.8 and the generic platform > - Fixes for the Ibex PLIC > Applied, thanks. Please update the changelog at https://wiki.qemu.org/ChangeLog/5.2 for any user-visible changes. -- PMM
Re: [PULL 00/14] Linux user for 5.2 patches
Le 21/08/2020 à 21:08, Laurent Vivier a écrit : > Le 21/08/2020 à 18:23, Peter Maydell a écrit : >> On Thu, 13 Aug 2020 at 07:51, Laurent Vivier wrote: >>> >>> The following changes since commit d0ed6a69d399ae193959225cdeaa9382746c91cc: >>> >>> Update version for v5.1.0 release (2020-08-11 17:07:03 +0100) >>> >>> are available in the Git repository at: >>> >>> git://github.com/vivier/qemu.git tags/linux-user-for-5.2-pull-request >>> >>> for you to fetch changes up to 04275cad60c8f99e0dd7f56aecda68ceac926da8: >>> >>> linux-user: Fix 'utimensat()' implementation (2020-08-12 10:09:58 +0200) >>> >>> >>> Add btrfs ioctls >>> Add clock_getres_time64, timer_gettime64, timer_settime64, >>> timerfd_gettime64, timerfd_settime64 >>> Some fixes (page protection, print_fdset, timerspec, itimerspec) >>> >>> >> >> Fails to compile: >> >> ../../linux-user/syscall_types.h:407:28: error: >> ‘BTRFS_INO_LOOKUP_USER_PATH_MAX’ undeclared here (not in a function); >> did you mean ‘BTRFS_INO_LOOKUP_PATH_MAX’? >> MK_ARRAY(TYPE_CHAR, BTRFS_INO_LOOKUP_USER_PATH_MAX)) /* path */ >> ^ >> >> ../../linux-user/syscall_types.h:451:17: error: >> ‘BTRFS_MAX_ROOTREF_BUFFER_NUM’ undeclared here (not in a function) >> BTRFS_MAX_ROOTREF_BUFFER_NUM), /* rootref */ >> ^ >> >> On PPC, even more errors, relating to not having >> BTRFS_PATH_NAME_MAX, PTRFS_VOL_NAME_MAX, etc. >> >> Not sure if this was a semantic conflict with the meson >> conversion, or just an assumption of a newer btrfs.h >> than some systems have. > > Well, for me master is also broken (F32, linux-user only, static): > > Compiling C object libqemuutil.a.p/util_compatfd.c.o > ../../../Projects/qemu/util/compatfd.c: In function 'qemu_signalfd': > ../../../Projects/qemu/util/compatfd.c:103:19: error: 'SYS_signalfd' > undeclared (first use in this function); did you mean 'SYS_signalfd4'? > 103 | ret = syscall(SYS_signalfd, -1, mask, _NSIG / 8); > | ^~~~ > | SYS_signalfd4 > ../../../Projects/qemu/util/compatfd.c:103:19: note: each undeclared > identifier is reported only once for each function it appears in My build env was dirty and I had an asm/unistd.h that was not valid. Fixed now. Thanks, Laurent
Re: [PATCH v2] linux-user: detect mismatched ELF ABI in qemu-mips[n32][el]
Le 23/08/2020 à 12:17, Carlo Marcelo Arenas Belón a écrit :
> MIPS provides 2 ILP32 ABIs, and therefore 4 possible qemu-mips binaries
> with 2 pairs using the same endianess and bitness.
>
> This could lead to an O32 image loading in the N32 binary or vice versa
> and in cryptic errors (if lucky that the CPU doesn't match the FPU used)
> like :
>
> qemu: Unexpected FPU mode (o32 ELF loaded to qemu-mipsn32[el])
> ELF binary's NaN mode not supported by CPU(n32 -> qemu-mips[el])
>
> Add an ABI check macro that could be used while checking the ELF header
> that relies in the ABI2 flag to identify n32 binaries and abort instead
> early with a more descriptive error :
>
> Invalid ELF image for this architecture
>
> Signed-off-by: Carlo Marcelo Arenas Belón
> ---
> Changes since v1:
> - Use the provided definition from include/elf.h (per Laurent)
> - Abort instead of warning (per Laurent, not using a custom error though)
> - Expand the check to all other combinations (per Aleksandar)
>
> linux-user/elfload.c | 11 +++
> 1 file changed, 11 insertions(+)
>
> diff --git a/linux-user/elfload.c b/linux-user/elfload.c
> index fe9dfe795d..69936dcd45 100644
> --- a/linux-user/elfload.c
> +++ b/linux-user/elfload.c
> @@ -918,6 +918,12 @@ static void elf_core_copy_regs(target_elf_gregset_t
> *regs, const CPUPPCState *en
>
> #define elf_check_arch(x) ((x) == EM_MIPS || (x) == EM_NANOMIPS)
>
> +#ifdef TARGET_ABI_MIPSN32
> +#define elf_check_abi(x) ((x) & EF_MIPS_ABI2)
> +#else
> +#define elf_check_abi(x) (!((x) & EF_MIPS_ABI2))
> +#endif
> +
> static inline void init_thread(struct target_pt_regs *regs,
> struct image_info *infop)
> {
> @@ -1487,6 +1493,10 @@ static void elf_core_copy_regs(target_elf_gregset_t
> *regs,
> #define elf_check_arch(x) ((x) == ELF_ARCH)
> #endif
>
> +#ifndef elf_check_abi
> +#define elf_check_abi(x) (1)
> +#endif
> +
> #ifndef ELF_HWCAP
> #define ELF_HWCAP 0
> #endif
> @@ -1644,6 +1654,7 @@ static bool elf_check_ident(struct elfhdr *ehdr)
> static bool elf_check_ehdr(struct elfhdr *ehdr)
> {
> return (elf_check_arch(ehdr->e_machine)
> +&& elf_check_abi(ehdr->e_flags)
> && ehdr->e_ehsize == sizeof(struct elfhdr)
> && ehdr->e_phentsize == sizeof(struct elf_phdr)
> && (ehdr->e_type == ET_EXEC || ehdr->e_type == ET_DYN));
>
Reviewed-by: Laurent Vivier
[PATCH v2 7/7] gitlab-ci: Add cross-compiling build tests
Now that we can use all our QEMU test containers in the gitlab-CI, we can easily add some jobs that test cross-compilation for various architectures. There is just only small ugliness: Since the shared runners on gitlab.com are single-threaded, we have to split each compilation job into two parts (--disable-user and --disable-system), and exclude some additional targets, to avoid that the jobs are running too long and hitting the timeout of 1 h. Signed-off-by: Thomas Huth --- .gitlab-ci.d/crossbuilds.yml | 113 +++ .gitlab-ci.yml | 1 + MAINTAINERS | 1 + 3 files changed, 115 insertions(+) create mode 100644 .gitlab-ci.d/crossbuilds.yml diff --git a/.gitlab-ci.d/crossbuilds.yml b/.gitlab-ci.d/crossbuilds.yml new file mode 100644 index 00..4ec7226b5c --- /dev/null +++ b/.gitlab-ci.d/crossbuilds.yml @@ -0,0 +1,113 @@ + +.cross_system_build_job_template: &cross_system_build_job_definition + stage: build + image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest + script: +- mkdir build +- cd build +- PKG_CONFIG_PATH=$PKG_CONFIG_PATH + ../configure --enable-werror $QEMU_CONFIGURE_OPTS --disable-user +--target-list-exclude="aarch64-softmmu i386-softmmu microblaze-softmmu + mips-softmmu mipsel-softmmu mips64-softmmu ppc64-softmmu sh4-softmmu + xtensa-softmmu" +- make -j$(expr $(nproc) + 1) all check-build + +.cross_user_build_job_template: &cross_user_build_job_definition + stage: build + image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest + script: +- mkdir build +- cd build +- PKG_CONFIG_PATH=$PKG_CONFIG_PATH + ../configure --enable-werror $QEMU_CONFIGURE_OPTS --disable-system +- make -j$(expr $(nproc) + 1) all check-build + +cross-armel-system: + <<: *cross_system_build_job_definition + variables: +IMAGE: debian-armel-cross + +cross-armel-user: + <<: *cross_user_build_job_definition + variables: +IMAGE: debian-armel-cross + +cross-armhf-system: + <<: *cross_system_build_job_definition + variables: +IMAGE: debian-armhf-cross + +cross-armhf-user: + <<: *cross_user_build_job_definition + variables: +IMAGE: debian-armhf-cross + +cross-arm64-system: + <<: *cross_system_build_job_definition + variables: +IMAGE: debian-arm64-cross + +cross-arm64-user: + <<: *cross_user_build_job_definition + variables: +IMAGE: debian-arm64-cross + +cross-mips-system: + <<: *cross_system_build_job_definition + variables: +IMAGE: debian-mips-cross + +cross-mips-user: + <<: *cross_user_build_job_definition + variables: +IMAGE: debian-mips-cross + +cross-mipsel-system: + <<: *cross_system_build_job_definition + variables: +IMAGE: debian-mipsel-cross + +cross-mipsel-user: + <<: *cross_user_build_job_definition + variables: +IMAGE: debian-mipsel-cross + +cross-mips64el-system: + <<: *cross_system_build_job_definition + variables: +IMAGE: debian-mips64el-cross + +cross-mips64el-user: + <<: *cross_user_build_job_definition + variables: +IMAGE: debian-mips64el-cross + +cross-ppc64el-system: + <<: *cross_system_build_job_definition + variables: +IMAGE: debian-ppc64el-cross + +cross-ppc64el-user: + <<: *cross_user_build_job_definition + variables: +IMAGE: debian-ppc64el-cross + +cross-s390x-system: + <<: *cross_system_build_job_definition + variables: +IMAGE: debian-s390x-cross + +cross-s390x-user: + <<: *cross_user_build_job_definition + variables: +IMAGE: debian-s390x-cross + +cross-win32-system: + <<: *cross_system_build_job_definition + variables: +IMAGE: debian-win32-cross + +cross-win64-system: + <<: *cross_system_build_job_definition + variables: +IMAGE: debian-win64-cross diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index b7967b9a13..8a4b67ecca 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -18,6 +18,7 @@ include: - local: '/.gitlab-ci.d/edk2.yml' - local: '/.gitlab-ci.d/opensbi.yml' - local: '/.gitlab-ci.d/containers.yml' + - local: '/.gitlab-ci.d/crossbuilds.yml' .native_build_job_template: &native_build_job_definition stage: build diff --git a/MAINTAINERS b/MAINTAINERS index 0886eb3d2b..2731f7a594 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3067,6 +3067,7 @@ M: Alex Bennée R: Wainer dos Santos Moschetta S: Maintained F: .gitlab-ci.yml +F: .gitlab-ci.d/crossbuilds.yml Guest Test Compilation Support M: Alex Bennée -- 2.18.2
[PATCH v2 1/7] configure: Add system = 'linux' for meson when cross-compiling
Meson needs the "system = xyz" line when cross-compiling. We are already adding a "system = 'windows'" for the MinGW cross-compilation case here, so let's add a "system = 'linux'" now for Linux hosts, too. Signed-off-by: Thomas Huth --- configure | 3 +++ 1 file changed, 3 insertions(+) diff --git a/configure b/configure index d9ca87fbbb..01204ba0b5 100755 --- a/configure +++ b/configure @@ -8208,6 +8208,9 @@ if test -n "$cross_prefix"; then ?:*) pre_prefix=/ ;; esac fi +if test "$linux" = "yes" ; then +echo "system = 'linux'" >> $cross +fi case "$ARCH" in i386|x86_64) echo "cpu_family = 'x86'" >> $cross -- 2.18.2
[PATCH v2 6/7] configure: Allow automatic WHPX detection
The whpx variable is currently initialized to "no" which causes the WHPX check to skip the detection unless the user specified --enable-whpx. Since the detection code should be able to figure it out correctly, let's initialized the variable to "" on MinGW-builds for proper auto-detection instead. Message-Id: <20200804170055.2851-11-th...@redhat.com> Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Stefan Weil Signed-off-by: Thomas Huth --- configure | 1 + 1 file changed, 1 insertion(+) diff --git a/configure b/configure index 01204ba0b5..2ca3c1d8e9 100755 --- a/configure +++ b/configure @@ -849,6 +849,7 @@ case $targetos in MINGW32*) mingw32="yes" hax="yes" + whpx="" vhost_user="no" audio_possible_drivers="dsound sdl" if check_include dsound.h; then -- 2.18.2
[PATCH v2 3/7] tests/Makefile: test-image-locking needs CONFIG_POSIX
test-image-locking.c uses the qemu_lock_fd_test() function which is only available on POSIX-like systems. Reviewed-by: John Snow Message-Id: <20200804170055.2851-4-th...@redhat.com> Signed-off-by: Thomas Huth --- tests/Makefile.include | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/Makefile.include b/tests/Makefile.include index 9ac8f5b86a..497f1f21ff 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -87,7 +87,9 @@ check-unit-$(CONFIG_BLOCK) += tests/test-blockjob$(EXESUF) check-unit-$(CONFIG_BLOCK) += tests/test-blockjob-txn$(EXESUF) check-unit-$(CONFIG_BLOCK) += tests/test-block-backend$(EXESUF) check-unit-$(CONFIG_BLOCK) += tests/test-block-iothread$(EXESUF) +ifeq ($(CONFIG_POSIX),y) check-unit-$(CONFIG_BLOCK) += tests/test-image-locking$(EXESUF) +endif check-unit-y += tests/test-x86-cpuid$(EXESUF) # all code tested by test-x86-cpuid is inside topology.h ifeq ($(CONFIG_SOFTMMU),y) -- 2.18.2
[PATCH v2 4/7] tests/Makefile: test-replication needs CONFIG_POSIX
test-replication uses sigaction() and friends which are only available on POSIX-like systems. Reviewed-by: Philippe Mathieu-Daudé Message-Id: <20200804170055.2851-5-th...@redhat.com> Signed-off-by: Thomas Huth --- tests/Makefile.include | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/tests/Makefile.include b/tests/Makefile.include index 497f1f21ff..a9746c2ecb 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -156,11 +156,13 @@ check-unit-$(CONFIG_BLOCK) += tests/test-crypto-afsplit$(EXESUF) check-unit-$(call land,$(CONFIG_BLOCK),$(CONFIG_QEMU_PRIVATE_XTS)) += tests/test-crypto-xts$(EXESUF) check-unit-$(CONFIG_BLOCK) += tests/test-crypto-block$(EXESUF) check-unit-y += tests/test-logging$(EXESUF) -check-unit-$(call land,$(CONFIG_BLOCK),$(CONFIG_REPLICATION)) += tests/test-replication$(EXESUF) check-unit-$(CONFIG_SOFTMMU) += tests/test-bufferiszero$(EXESUF) check-unit-y += tests/test-uuid$(EXESUF) check-unit-y += tests/ptimer-test$(EXESUF) check-unit-y += tests/test-qapi-util$(EXESUF) +ifeq ($(CONFIG_BLOCK)$(CONFIG_REPLICATION)$(CONFIG_POSIX),yyy) +check-unit-y += tests/test-replication$(EXESUF) +endif generated-files-y += tests/test-qapi-types.h generated-files-y += tests/include/test-qapi-types-sub-module.h -- 2.18.2
[PATCH v2 0/7] Run cross-compilation build tests in the gitlab-CI
Now that we can use all our QEMU build containers in the gitlab-CI, we can also run the cross-compilation jobs there. Of course, some problems have to be fixed first, so this is taken care of in the first four patches. The following two patches make sure that we can also enable WHPX builds with our debian-win64-cross container, so that we can compile-test this accelerator code now, too. The last patch then finally enables the cross-compilation jobs in the CI. v2: - Dropped patches that are not necessary anymore - Added the first two patches to fix problems with the new meson build system Thomas Huth (7): configure: Add system = 'linux' for meson when cross-compiling tests/docker: Install python3-setuptools in the debian9-mxe containers tests/Makefile: test-image-locking needs CONFIG_POSIX tests/Makefile: test-replication needs CONFIG_POSIX dockerfiles/debian-win64-cross: Download WHPX MinGW headers configure: Allow automatic WHPX detection gitlab-ci: Add cross-compiling build tests .gitlab-ci.d/crossbuilds.yml | 113 ++ .gitlab-ci.yml| 1 + MAINTAINERS | 1 + configure | 4 + tests/Makefile.include| 6 +- .../dockerfiles/debian-win64-cross.docker | 9 +- tests/docker/dockerfiles/debian9-mxe.docker | 2 +- 7 files changed, 133 insertions(+), 3 deletions(-) create mode 100644 .gitlab-ci.d/crossbuilds.yml -- 2.18.2
[PATCH v2 2/7] tests/docker: Install python3-setuptools in the debian9-mxe containers
The python setuptools are a requirement for meson, so we need to install this additional package now. Signed-off-by: Thomas Huth --- tests/docker/dockerfiles/debian9-mxe.docker | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/docker/dockerfiles/debian9-mxe.docker b/tests/docker/dockerfiles/debian9-mxe.docker index ae2c222a6f..7865c821f4 100644 --- a/tests/docker/dockerfiles/debian9-mxe.docker +++ b/tests/docker/dockerfiles/debian9-mxe.docker @@ -8,7 +8,7 @@ FROM qemu/debian9 MAINTAINER Philippe Mathieu-Daudé RUN DEBIAN_FRONTEND=noninteractive eatmydata \ -apt install -y --no-install-recommends gnupg dirmngr +apt install -y --no-install-recommends gnupg dirmngr python3-setuptools # Add the foreign architecture we want and install dependencies RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys C6BF758A33A3A276 && \ -- 2.18.2
[PATCH v2 5/7] dockerfiles/debian-win64-cross: Download WHPX MinGW headers
To compile-test the WHPX accelerator, we need to download these system headers first (they are unfortunately not part of any released and packaged MinGW toolchain yet). Idea taken from another patch by Stefan Weil. Message-Id: <20200804170055.2851-12-th...@redhat.com> Signed-off-by: Thomas Huth --- tests/docker/dockerfiles/debian-win64-cross.docker | 9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/tests/docker/dockerfiles/debian-win64-cross.docker b/tests/docker/dockerfiles/debian-win64-cross.docker index 2fc9cfcbc6..4cc4a3f365 100644 --- a/tests/docker/dockerfiles/debian-win64-cross.docker +++ b/tests/docker/dockerfiles/debian-win64-cross.docker @@ -32,7 +32,14 @@ RUN apt-get update && \ mxe-$TARGET-w64-mingw32.shared-sdl2 \ mxe-$TARGET-w64-mingw32.shared-sdl2-mixer \ mxe-$TARGET-w64-mingw32.shared-sdl2-gfx \ -mxe-$TARGET-w64-mingw32.shared-zlib +mxe-$TARGET-w64-mingw32.shared-zlib \ +curl && \ +curl -s -S -o /usr/lib/mxe/usr/x86_64-w64-mingw32.shared/include/WinHvEmulation.h \ + "https://sourceforge.net/p/mingw-w64/mingw-w64/ci/master/tree/mingw-w64-headers/include/winhvemulation.h?format=raw"; && \ +curl -s -S -o /usr/lib/mxe/usr/x86_64-w64-mingw32.shared/include/WinHvPlatform.h \ + "https://sourceforge.net/p/mingw-w64/mingw-w64/ci/master/tree/mingw-w64-headers/include/winhvplatform.h?format=raw"; && \ +curl -s -S -o /usr/lib/mxe/usr/x86_64-w64-mingw32.shared/include/winhvplatformdefs.h \ + "https://sourceforge.net/p/mingw-w64/mingw-w64/ci/master/tree/mingw-w64-headers/include/winhvplatformdefs.h?format=raw"; # Specify the cross prefix for this image (see tests/docker/common.rc) ENV QEMU_CONFIGURE_OPTS --cross-prefix=x86_64-w64-mingw32.shared- -- 2.18.2
Re: [PATCH] meson: Build qemu-nbd on macOS again
Hi
On Sun, Aug 23, 2020 at 12:32 PM Thomas Huth wrote:
>
> Before switching to the meson build system, we used to compile qemu-nbd
> for macOS, too, which is especially important for running the iotests
> there. Commit b7c70bf2c5 disabled it by accident, since it did not take
> into consideration that the $bsd variable in the configure script was
> also set to "yes" on macOS. Fix it by enabling qemu-nbd on all systems
> but Windows now instead (which was likely the original intention of the
> old code in the configure script).
>
> Fixes: b7c70bf2c5 ("meson: qemu-{img,io,nbd}")
> Signed-off-by: Thomas Huth
lgtm
Reviewed-by: Marc-André Lureau
> ---
> meson.build | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meson.build b/meson.build
> index df5bf728b5..d79f849768 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -1075,7 +1075,7 @@ if have_tools
>qemu_io = executable('qemu-io', files('qemu-io.c'),
> dependencies: [block, qemuutil], install: true)
>qemu_block_tools += [qemu_img, qemu_io]
> - if targetos == 'linux' or targetos == 'sunos' or targetos.endswith('bsd')
> + if targetos != 'windows'
> qemu_nbd = executable('qemu-nbd', files('qemu-nbd.c'),
> dependencies: [block, qemuutil], install: true)
> qemu_block_tools += [qemu_nbd]
> --
> 2.18.2
>
Re: [PULL 0/6] Meson build system fixes
On Fri, 21 Aug 2020 at 22:18, Paolo Bonzini wrote: > > meson fixes: > > * --disable-tools --enable-system build > * s390 no-TCG build > * fdmon-io_uring > * 'shift' error message in version_ge() > > > Marc-André Lureau (1): > meson: convert pc-bios/keymaps/Makefile > > Paolo Bonzini (2): > target/s390x: fix meson.build issue > keymaps: update I have noticed that now if I do a build starting from a source tree with this keymaps update, the build seems to cause the keymap files to be modified: e104462:bionic:qemu$ git status On branch master Your branch is up-to-date with 'origin/master'. Changes not staged for commit: (use "git add ..." to update what will be committed) (use "git checkout -- ..." to discard changes in working directory) modified: pc-bios/keymaps/ar modified: pc-bios/keymaps/bepo modified: pc-bios/keymaps/cz modified: pc-bios/keymaps/da modified: pc-bios/keymaps/de modified: pc-bios/keymaps/de-ch modified: pc-bios/keymaps/en-gb modified: pc-bios/keymaps/en-us modified: pc-bios/keymaps/es modified: pc-bios/keymaps/et modified: pc-bios/keymaps/fi modified: pc-bios/keymaps/fo modified: pc-bios/keymaps/fr modified: pc-bios/keymaps/fr-be modified: pc-bios/keymaps/fr-ca modified: pc-bios/keymaps/fr-ch modified: pc-bios/keymaps/hr modified: pc-bios/keymaps/hu modified: pc-bios/keymaps/is modified: pc-bios/keymaps/it modified: pc-bios/keymaps/ja modified: pc-bios/keymaps/lt modified: pc-bios/keymaps/lv modified: pc-bios/keymaps/mk modified: pc-bios/keymaps/nl modified: pc-bios/keymaps/no modified: pc-bios/keymaps/pl modified: pc-bios/keymaps/pt modified: pc-bios/keymaps/pt-br modified: pc-bios/keymaps/ru modified: pc-bios/keymaps/th modified: pc-bios/keymaps/tr no changes added to commit (use "git add" and/or "git commit -a") The changes seem to be like this: --- a/pc-bios/keymaps/ar +++ b/pc-bios/keymaps/ar @@ -776,244 +776,6 @@ XF86AudioMedia 0xed # evdev 247 (0xf7): no evdev -> QKeyCode mapping (xkb keysym XF86RFKill) -# evdev 248 (0xf8): no evdev -> QKeyCode mapping (xkb keysym NoSymbol) - -# evdev 249 (0xf9): no evdev -> QKeyCode mapping (xkb keysym NoSymbol) - -# evdev 250 (0xfa): no evdev -> QKeyCode mapping (xkb keysym NoSymbol) ... -# evdev 363 (0x16b): no evdev -> QKeyCode mapping (xkb keysym NoSymbol) - -# evdev 364 (0x16c): no evdev -> QKeyCode mapping (xkb keysym XF86Favorites) - -# evdev 365 (0x16d): no evdev -> QKeyCode mapping (xkb keysym NoSymbol) - -# evdev 366 (0x16e): no evdev -> QKeyCode mapping (xkb keysym XF86Keyboard) Why is a build (which is now by definition in a build tree) causing my source tree to be changed, and can we make it stop? thanks -- PMM
Re: odd meson failure: Unknown variable "exe_name"
On Sun, 23 Aug 2020 at 11:45, Peter Maydell wrote: > > On my x86-64 Linux box, one of my local build trees turned out not > to work: > > $ rm -rf build/x86 && mkdir build/x86 && (cd build/x86 && > '../../configure' > '--target-list=arm-softmmu,aarch64-softmmu,arm-linux-user,aarch64-linux-user' > '--enable-debug' '--cc=ccache gcc' '--audio-drv-list=pa' > '--with-pkgversion=pm215' '--enable-trace-backends=log,dtrace' > '--enable-docs') > > fails with > [...] > Program keycodemapdb/tools/keymap-gen found: YES > Program scripts/decodetree.py found: YES > Program ../scripts/modules/module_block.py found: YES > Program nm found: YES > Program scripts/undefsym.sh found: YES > Program scripts/feature_to_c.sh found: YES > > ../../meson.build:1030:14: ERROR: Unknown variable "exe_name". > > A full log can be found at > /home/petmay01/linaro/qemu-from-laptop/qemu/build/x86/meson-logs/meson-log.txt > > ERROR: meson setup failed > > > This is the same box that's worked fine for merge testing, so > presumably something about the particular set of configure > options is tripping it up. Dropping the '--enable-trace-backends=...' option lets it pass, so that's the area where the problem is. -- PMM
odd meson failure: Unknown variable "exe_name"
On my x86-64 Linux box, one of my local build trees turned out not to work: $ rm -rf build/x86 && mkdir build/x86 && (cd build/x86 && '../../configure' '--target-list=arm-softmmu,aarch64-softmmu,arm-linux-user,aarch64-linux-user' '--enable-debug' '--cc=ccache gcc' '--audio-drv-list=pa' '--with-pkgversion=pm215' '--enable-trace-backends=log,dtrace' '--enable-docs') fails with [...] Program keycodemapdb/tools/keymap-gen found: YES Program scripts/decodetree.py found: YES Program ../scripts/modules/module_block.py found: YES Program nm found: YES Program scripts/undefsym.sh found: YES Program scripts/feature_to_c.sh found: YES ../../meson.build:1030:14: ERROR: Unknown variable "exe_name". A full log can be found at /home/petmay01/linaro/qemu-from-laptop/qemu/build/x86/meson-logs/meson-log.txt ERROR: meson setup failed This is the same box that's worked fine for merge testing, so presumably something about the particular set of configure options is tripping it up. thanks -- PMM
Re: [PULL 00/23] SD/MMC patches for 2020-08-21
On Fri, 21 Aug 2020 at 18:29, Philippe Mathieu-Daudé wrote: > > The following changes since commit d6f83a72a7db94a3ede9f5cc4fb39f9c8e89f954: > > Merge remote-tracking branch 'remotes/philmd-gitlab/tags/acceptance-testing= > -20200812' into staging (2020-08-21 14:51:43 +0100) > > are available in the Git repository at: > > https://gitlab.com/philmd/qemu.git tags/sd-next-20200821 > > for you to fetch changes up to 6d2d4069c47e23b9e3913f9c8204fd0edcb99fb3: > > hw/sd: Correct the maximum size of a Standard Capacity SD Memory Card (2020= > -08-21 16:49:22 +0200) > > > SD/MMC patches > > - Convert legacy SD host controller to the SDBus API > - Move legacy API to a separate "sdcard_legacy.h" header > - Introduce methods to access multiple bytes on SDBus data lines > - Fix 'switch function' group location > - Fix SDSC maximum card size (2GB) > > CI jobs result: > https://gitlab.com/philmd/qemu/-/pipelines/180605963 Applied, thanks. Please update the changelog at https://wiki.qemu.org/ChangeLog/5.2 for any user-visible changes. -- PMM
Re: [PATCH] linux-user: warn if trying to use qemu-mipsn32[el] with non n32 ELF
The differences of bit width and endianness are already being taken care of by the current code. The differences in ABI for ILP32 are the only ones missing and so the new version[1] (sorry, not a series since it makes sense as a single change anyway) should be enough to fix any inconsistencies. With this change, all linux-user versions will show the same consistent error when asked to load an incorrect (wrong architecture, endianness or bit width) binary Carlo [1] https://patchwork.ozlabs.org/project/qemu-devel/patch/20200823101703.18451-1-care...@gmail.com/
[PATCH] scripts/qemu-version.sh: Add missing space before ']'
When configure has been run with --with-pkgversion=xyz, the shell complains
about a missing ']' in this script.
Fixes: 2c273f32d3 ("meson: generate qemu-version.h")
Signed-off-by: Thomas Huth
---
scripts/qemu-version.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/qemu-version.sh b/scripts/qemu-version.sh
index 4847385e42..03128c56a2 100755
--- a/scripts/qemu-version.sh
+++ b/scripts/qemu-version.sh
@@ -6,7 +6,7 @@ dir="$1"
pkgversion="$2"
version="$3"
-if [ -z "$pkgversion"]; then
+if [ -z "$pkgversion" ]; then
cd "$dir"
if [ -e .git ]; then
pkgversion=$(git describe --match 'v*' --dirty | echo "")
--
2.18.2
[PATCH v2] linux-user: detect mismatched ELF ABI in qemu-mips[n32][el]
MIPS provides 2 ILP32 ABIs, and therefore 4 possible qemu-mips binaries
with 2 pairs using the same endianess and bitness.
This could lead to an O32 image loading in the N32 binary or vice versa
and in cryptic errors (if lucky that the CPU doesn't match the FPU used)
like :
qemu: Unexpected FPU mode (o32 ELF loaded to qemu-mipsn32[el])
ELF binary's NaN mode not supported by CPU(n32 -> qemu-mips[el])
Add an ABI check macro that could be used while checking the ELF header
that relies in the ABI2 flag to identify n32 binaries and abort instead
early with a more descriptive error :
Invalid ELF image for this architecture
Signed-off-by: Carlo Marcelo Arenas Belón
---
Changes since v1:
- Use the provided definition from include/elf.h (per Laurent)
- Abort instead of warning (per Laurent, not using a custom error though)
- Expand the check to all other combinations (per Aleksandar)
linux-user/elfload.c | 11 +++
1 file changed, 11 insertions(+)
diff --git a/linux-user/elfload.c b/linux-user/elfload.c
index fe9dfe795d..69936dcd45 100644
--- a/linux-user/elfload.c
+++ b/linux-user/elfload.c
@@ -918,6 +918,12 @@ static void elf_core_copy_regs(target_elf_gregset_t *regs,
const CPUPPCState *en
#define elf_check_arch(x) ((x) == EM_MIPS || (x) == EM_NANOMIPS)
+#ifdef TARGET_ABI_MIPSN32
+#define elf_check_abi(x) ((x) & EF_MIPS_ABI2)
+#else
+#define elf_check_abi(x) (!((x) & EF_MIPS_ABI2))
+#endif
+
static inline void init_thread(struct target_pt_regs *regs,
struct image_info *infop)
{
@@ -1487,6 +1493,10 @@ static void elf_core_copy_regs(target_elf_gregset_t
*regs,
#define elf_check_arch(x) ((x) == ELF_ARCH)
#endif
+#ifndef elf_check_abi
+#define elf_check_abi(x) (1)
+#endif
+
#ifndef ELF_HWCAP
#define ELF_HWCAP 0
#endif
@@ -1644,6 +1654,7 @@ static bool elf_check_ident(struct elfhdr *ehdr)
static bool elf_check_ehdr(struct elfhdr *ehdr)
{
return (elf_check_arch(ehdr->e_machine)
+&& elf_check_abi(ehdr->e_flags)
&& ehdr->e_ehsize == sizeof(struct elfhdr)
&& ehdr->e_phentsize == sizeof(struct elf_phdr)
&& (ehdr->e_type == ET_EXEC || ehdr->e_type == ET_DYN));
--
2.28.0.424.gade71fd49b
[PATCH] meson: Don't make object files for dtrace on macOS
dtrace on macOS uses unresolved symbols with a special prefix to define
probes [1], only headers should be generated for USDT (dtrace(1)). But
it doesn't support backwards compatible no-op -G flag [2] and implicit
build rules fail.
1. https://markmail.org/message/6grq2ygr5nwdwsnb
2. https://markmail.org/message/5xrxt2w5m42nojkz
Cc: Daniel P. Berrangé
Cc: Cameron Esfahani
Signed-off-by: Roman Bolshakov
---
trace/meson.build | 13 -
1 file changed, 8 insertions(+), 5 deletions(-)
This is an update of the previous patch that is no longer valid due to
conversion of the build system to meson:
https://lists.gnu.org/archive/html/qemu-devel/2020-08/msg02493.html
diff --git a/trace/meson.build b/trace/meson.build
index 56e870848e..1c1fb31a61 100644
--- a/trace/meson.build
+++ b/trace/meson.build
@@ -39,12 +39,15 @@ foreach dir : [ '.' ] + trace_events_subdirs
output: fmt.format('trace-dtrace', 'h'),
input: trace_dtrace,
command: [ 'dtrace', '-o', '@OUTPUT@',
'-h', '-s', '@INPUT@' ])
-trace_dtrace_o = custom_target(fmt.format('trace-dtrace', 'o'),
- output: fmt.format('trace-dtrace', 'o'),
- input: trace_dtrace,
- command: [ 'dtrace', '-o', '@OUTPUT@',
'-G', '-s', '@INPUT@' ])
+trace_ss.add(trace_dtrace_h)
+if host_machine.system() != 'darwin'
+ trace_dtrace_o = custom_target(fmt.format('trace-dtrace', 'o'),
+ output: fmt.format('trace-dtrace', 'o'),
+ input: trace_dtrace,
+ command: [ 'dtrace', '-o', '@OUTPUT@',
'-G', '-s', '@INPUT@' ])
+ trace_ss.add(trace_dtrace_o)
+endif
-trace_ss.add(trace_dtrace_h, trace_dtrace_o)
genh += trace_dtrace_h
endif
endforeach
--
2.24.3 (Apple Git-128)
[PATCH] meson: Build qemu-nbd on macOS again
Before switching to the meson build system, we used to compile qemu-nbd
for macOS, too, which is especially important for running the iotests
there. Commit b7c70bf2c5 disabled it by accident, since it did not take
into consideration that the $bsd variable in the configure script was
also set to "yes" on macOS. Fix it by enabling qemu-nbd on all systems
but Windows now instead (which was likely the original intention of the
old code in the configure script).
Fixes: b7c70bf2c5 ("meson: qemu-{img,io,nbd}")
Signed-off-by: Thomas Huth
---
meson.build | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meson.build b/meson.build
index df5bf728b5..d79f849768 100644
--- a/meson.build
+++ b/meson.build
@@ -1075,7 +1075,7 @@ if have_tools
qemu_io = executable('qemu-io', files('qemu-io.c'),
dependencies: [block, qemuutil], install: true)
qemu_block_tools += [qemu_img, qemu_io]
- if targetos == 'linux' or targetos == 'sunos' or targetos.endswith('bsd')
+ if targetos != 'windows'
qemu_nbd = executable('qemu-nbd', files('qemu-nbd.c'),
dependencies: [block, qemuutil], install: true)
qemu_block_tools += [qemu_nbd]
--
2.18.2
Re: [PULL 00/14] Linux user for 5.2 patches
Le 21/08/2020 à 21:08, Laurent Vivier a écrit : > Le 21/08/2020 à 18:23, Peter Maydell a écrit : >> On Thu, 13 Aug 2020 at 07:51, Laurent Vivier wrote: >>> >>> The following changes since commit d0ed6a69d399ae193959225cdeaa9382746c91cc: >>> >>> Update version for v5.1.0 release (2020-08-11 17:07:03 +0100) >>> >>> are available in the Git repository at: >>> >>> git://github.com/vivier/qemu.git tags/linux-user-for-5.2-pull-request >>> >>> for you to fetch changes up to 04275cad60c8f99e0dd7f56aecda68ceac926da8: >>> >>> linux-user: Fix 'utimensat()' implementation (2020-08-12 10:09:58 +0200) >>> >>> >>> Add btrfs ioctls >>> Add clock_getres_time64, timer_gettime64, timer_settime64, >>> timerfd_gettime64, timerfd_settime64 >>> Some fixes (page protection, print_fdset, timerspec, itimerspec) >>> >>> >> >> Fails to compile: >> >> ../../linux-user/syscall_types.h:407:28: error: >> ‘BTRFS_INO_LOOKUP_USER_PATH_MAX’ undeclared here (not in a function); >> did you mean ‘BTRFS_INO_LOOKUP_PATH_MAX’? >> MK_ARRAY(TYPE_CHAR, BTRFS_INO_LOOKUP_USER_PATH_MAX)) /* path */ >> ^ >> >> ../../linux-user/syscall_types.h:451:17: error: >> ‘BTRFS_MAX_ROOTREF_BUFFER_NUM’ undeclared here (not in a function) >> BTRFS_MAX_ROOTREF_BUFFER_NUM), /* rootref */ >> ^ >> >> On PPC, even more errors, relating to not having >> BTRFS_PATH_NAME_MAX, PTRFS_VOL_NAME_MAX, etc. >> >> Not sure if this was a semantic conflict with the meson >> conversion, or just an assumption of a newer btrfs.h >> than some systems have. > > Well, for me master is also broken (F32, linux-user only, static): > > Compiling C object libqemuutil.a.p/util_compatfd.c.o > ../../../Projects/qemu/util/compatfd.c: In function 'qemu_signalfd': > ../../../Projects/qemu/util/compatfd.c:103:19: error: 'SYS_signalfd' > undeclared (first use in this function); did you mean 'SYS_signalfd4'? > 103 | ret = syscall(SYS_signalfd, -1, mask, _NSIG / 8); > | ^~~~ > | SYS_signalfd4 > ../../../Projects/qemu/util/compatfd.c:103:19: note: each undeclared > identifier is reported only once for each function it appears in "--static" seems broken too: $ '.../configure' '--disable-system' '--enable-linux-user' '--disable-tools' '--static' '--disable-slirp' '--enable-capstone=git' C++ compiler c++ does not work with C compiler cc Disabling C++ specific optional code cross containers docker NOTE: guest cross-compilers enabled: cc The Meson build system Version: 0.55.0 ... gprof enabled: NO sparse enabled: NO strip binaries: YES profiler: NO static build: NO SDL support: NO SDL image support: NO GTK support: YES ... Thanks, Laurent
Re: [PATCH v2] CODING_STYLE.rst: flesh out our naming conventions.
On 10/08/2020 12.51, Alex Bennée wrote: Mention a few of the more common naming conventions we follow in the code base including common variable names and function prefix and suffix examples. Signed-off-by: Alex Bennée --- v2 - punctuation fixes suggested by Cornelia - re-worded section on qemu_ prefix - expanded on _locked suffix --- CODING_STYLE.rst | 30 -- 1 file changed, 28 insertions(+), 2 deletions(-) diff --git a/CODING_STYLE.rst b/CODING_STYLE.rst index 427699e0e42..e7ae44aed7f 100644 --- a/CODING_STYLE.rst +++ b/CODING_STYLE.rst @@ -109,8 +109,34 @@ names are lower_case_with_underscores_ending_with_a_t, like the POSIX uint64_t and family. Note that this last convention contradicts POSIX and is therefore likely to be changed. -When wrapping standard library functions, use the prefix ``qemu_`` to alert -readers that they are seeing a wrapped version; otherwise avoid this prefix. +Variable Naming Conventions +--- + +A number of short naming conventions exist for variables that use +common QEMU types. For example, the architecture independent CPUState +this is often held as a ``cs`` pointer variable, whereas the concrete +CPUArchState us usually held in a pointer called ``env``. + +Likewise, in device emulation code the common DeviceState is usually +called ``dev`` with the actual status structure often uses the terse +``s`` or maybe ``foodev``. Please let's not recommend single-letter variables like 's' here. This is a really bad idea, since they are hard to "grep" and can be confused quite easily. I think it would be best to simply drop that last part of the sentence (starting with "with the actual..."). Thomas
Re: [PATCH v6 4/8] ppc/e500: Use start-powered-off CPUState property
On Sat, Aug 22, 2020 at 10:59:56AM +0200, Cédric Le Goater wrote: > Hello, > > On 8/19/20 6:43 PM, Thiago Jung Bauermann wrote: > > Instead of setting CPUState::halted to 1 in ppce500_cpu_reset_sec(), use > > the start-powered-off property which makes cpu_common_reset() initialize it > > to 1 in common code. > > > > Also change creation of CPU object from cpu_create() to object_new() and > > qdev_realize_and_unref() because cpu_create() realizes the CPU and it's not > > possible to set a property after the object is realized. > > > > Reviewed-by: Philippe Mathieu-Daudé > > Signed-off-by: Thiago Jung Bauermann > > > This is breaking make check : > > tests/qtest/libqtest.c:175: kill_qemu() detected QEMU death from signal > 11 (Segmentation fault) (core dumped) > ERROR boot-serial-test - too few tests run (expected 7, got 0) > make: *** > [/home/legoater/work/qemu/qemu-powernv-5.2.git/tests/Makefile.include:650: > check-qtest-ppc64] Error 1 > make: *** Waiting for unfinished jobs > > > gdb --args build/ppc64-softmmu/qemu-system-ppc64 -display none -M > ppce500 > ... > Thread 1 "qemu-system-ppc" received signal SIGSEGV, Segmentation fault. > 0x5596ebf2 in ppce500_init (machine=0x567aa6e0) > at /home/legoater/work/qemu/qemu-powernv-5.2.git/hw/ppc/e500.c:880 > 880 irqs[i].irq[OPENPIC_OUTPUT_INT] = > input[PPCE500_INPUT_INT]; > > > AFAIUI, 'input is not initialized since the CPU is not yet > realized. Sigh. For future reference, Thiago, running an all-targets make check is pretty much a minimum bar to test before posting. -- David Gibson| I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson signature.asc Description: PGP signature
