[Qemu-devel] [PATCH qemu v5 04/18] memory: Move AddressSpaceDispatch from AddressSpace to FlatView

2017-09-21 Thread Alexey Kardashevskiy
off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- Changes: v4: * since FlatView::rcu is used now to dispose FV, call_rcu() in address_space_update_topology() is replaced with direct call to flatview_unref() --- include/exec/memory-internal.h | 12 +++- include/exec/memory.h

[Qemu-devel] [PATCH qemu v5 13/18] memory: Do not allocate FlatView in address_space_init

2017-09-21 Thread Alexey Kardashevskiy
This creates a new AS object without any FlatView as memory_region_transaction_commit() may want to reuse the empty FV. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- I do not really want to add view!=NULL into flatview_ref()/flatview_unref() as this is quite special case whe

[Qemu-devel] [PATCH qemu v5 02/18] memory: Open code FlatView rendering

2017-09-21 Thread Alexey Kardashevskiy
() if there is no registered listeners; this should improve starting time. This should cause no behavioural change. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- Changes: v4: * address_space_update_topology_pass() is called now under if (!QTAILQ_EMPTY(>listeners)) v3: * inlined &am

[Qemu-devel] [PATCH qemu v5 08/18] memory: Rename mem_begin/mem_commit/mem_add helpers

2017-09-21 Thread Alexey Kardashevskiy
This renames some helpers to reflect better what they do. This should cause no behavioural change. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- Changes: v3: * s/flatview_mem_add/flatview_add_to_dispatch/ * s/phys_page_compact_all/address_space_dispatch_compac

[Qemu-devel] [PATCH qemu v5 11/18] memory: Move address_space_update_ioeventfds

2017-09-21 Thread Alexey Kardashevskiy
So it is called (twice) from the same function. This is to make the next patches a bit simpler. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- memory.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/memory.c b/memory.c index eb730cd619..9868d0f57c

[Qemu-devel] [PATCH qemu v5 06/18] memory: Switch memory from using AddressSpace to FlatView

2017-09-21 Thread Alexey Kardashevskiy
[]; } subpage_t; struct MemoryRegionSection { MemoryRegion *mr; -AddressSpace *address_space; +FlatView *fv; hwaddr offset_within_region; Int128 size; hwaddr offset_within_address_space; bool readonly; }; This should cause no behavioural change. Signed-off-by: Alexey

[Qemu-devel] [PATCH qemu v5 05/18] memory: Remove AddressSpace pointer from AddressSpaceDispatch

2017-09-21 Thread Alexey Kardashevskiy
AS in ASD is only used to pass AS from mem_begin() to register_subpage() to store it in MemoryRegionSection, we can do this directly now. This should cause no behavioural change. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- exec.c | 15 +++ 1 file changed, 7 inse

[Qemu-devel] [PATCH qemu v5 03/18] memory: Move FlatView allocation to a helper

2017-09-21 Thread Alexey Kardashevskiy
This moves a FlatView allocation and initialization to a helper. While we are nere, replace g_new with g_new0 to not to bother if we add new fields in the future. This should cause no behavioural change. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- Changes: v3: * s/flatview

Re: [Qemu-devel] [PATCH qemu v4 09/18] memory: Store physical root MR in FlatView

2017-09-21 Thread Alexey Kardashevskiy
On 21/09/17 15:22, Alexey Kardashevskiy wrote: > On 21/09/17 10:02, Alexey Kardashevskiy wrote: >> On 21/09/17 03:15, Paolo Bonzini wrote: >>> On 20/09/2017 13:46, Alexey Kardashevskiy wrote: >>>> Address spaces get to keep a root MR (alias or not) but Flat

Re: [Qemu-devel] [PATCH qemu v4 09/18] memory: Store physical root MR in FlatView

2017-09-20 Thread Alexey Kardashevskiy
On 21/09/17 10:02, Alexey Kardashevskiy wrote: > On 21/09/17 03:15, Paolo Bonzini wrote: >> On 20/09/2017 13:46, Alexey Kardashevskiy wrote: >>> Address spaces get to keep a root MR (alias or not) but FlatView stores >>> the actual MR as this is going to be used

Re: [Qemu-devel] [PATCH qemu v4 09/18] memory: Store physical root MR in FlatView

2017-09-20 Thread Alexey Kardashevskiy
On 21/09/17 03:15, Paolo Bonzini wrote: > On 20/09/2017 13:46, Alexey Kardashevskiy wrote: >> Address spaces get to keep a root MR (alias or not) but FlatView stores >> the actual MR as this is going to be used later on to decide whether to >> share a particular FlatView or

Re: [Qemu-devel] [PATCH qemu v4 18/18] memory: Give memory_region_transaction_commit a hint

2017-09-20 Thread Alexey Kardashevskiy
On 21/09/17 03:14, Paolo Bonzini wrote: > On 20/09/2017 13:46, Alexey Kardashevskiy wrote: >> This extends memory_region_transaction_commit() to receive a MR as >> if it is a root MR or its topmost parent is, then we can only rebuild >> its FlatView and update it for a

Re: [Qemu-devel] [PATCH qemu v4 12/18] memory: Share FlatView's and dispatch trees between address spaces

2017-09-20 Thread Alexey Kardashevskiy
On 21/09/17 03:18, Paolo Bonzini wrote: > On 20/09/2017 13:46, Alexey Kardashevskiy wrote: >> +QTAILQ_FOREACH(as, _spaces, address_spaces_link) { >> +MemoryRegion *physmr = memory_region_get_flatview_root(as->root); >> +FlatView *new_view = g_hash

Re: [Qemu-devel] [PATCH qemu v4 15/18] memory: Share special empty FlatView

2017-09-20 Thread Alexey Kardashevskiy
On 21/09/17 03:13, Paolo Bonzini wrote: > On 20/09/2017 13:46, Alexey Kardashevskiy wrote: >> This shares an cached empty FlatView among address spaces. The empty >> FV is used every time when a root MR renders into a FV without memory >> sections which happens wh

[Qemu-devel] [PATCH qemu v4 15/18] memory: Share special empty FlatView

2017-09-20 Thread Alexey Kardashevskiy
ry FVs which are created (and then released) to make sure that they are empty and can be replaced with @empty_view. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- memory.c | 14 ++ 1 file changed, 14 insertions(+) diff --git a/memory.c b/memory.c index 4add0fd030..92b13

[Qemu-devel] [PATCH qemu v4 09/18] memory: Store physical root MR in FlatView

2017-09-20 Thread Alexey Kardashevskiy
Address spaces get to keep a root MR (alias or not) but FlatView stores the actual MR as this is going to be used later on to decide whether to share a particular FlatView or not. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- Changes: v4: * s/memory_region_unalias_

[Qemu-devel] [PATCH qemu v4 13/18] memory: Do not allocate FlatView in address_space_init

2017-09-20 Thread Alexey Kardashevskiy
This creates a new AS object without any FlatView as memory_region_transaction_commit() may want to reuse the empty FV. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- I do not really want to add view!=NULL into flatview_ref()/flatview_unref() as this is quite special case whe

[Qemu-devel] [PATCH qemu v4 12/18] memory: Share FlatView's and dispatch trees between address spaces

2017-09-20 Thread Alexey Kardashevskiy
address_space_init_shareable() as FlatViews are shared elsewhere and keeping as::ref_count correct seems an unnecessary and useless complication. This should cause no change and memory use or boot time yet. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- Changes: v4: * used g_hash_table_ne

[Qemu-devel] [PATCH qemu v4 04/18] memory: Move AddressSpaceDispatch from AddressSpace to FlatView

2017-09-20 Thread Alexey Kardashevskiy
off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- Changes: v4: * since FlatView::rcu is used now to dispose FV, call_rcu() in address_space_update_topology() is replaced with direct call to flatview_unref() --- include/exec/memory-internal.h | 12 +++- include/exec/memory.h

[Qemu-devel] [PATCH qemu v4 16/18] memory: Get rid of address_space_init_shareable

2017-09-20 Thread Alexey Kardashevskiy
Since FlatViews are shared now and ASes not, this gets rid of address_space_init_shareable(). This should cause no behavioural change. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- Changes: v3: * now removes @malloced and @ref_count, used to be in the previos patch --- includ

[Qemu-devel] [PATCH qemu v4 14/18] memory: Add flat views to HMP "info mtree"

2017-09-20 Thread Alexey Kardashevskiy
This adds a new "-d" switch to "info mtree" to print dispatch tree internals. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- Changes: v3: * reimplemented as there is no more global FlatView list --- Example: QEMU 2.10.50 monitor - type 'help' for more inform

[Qemu-devel] [PATCH qemu v4 11/18] memory: Move address_space_update_ioeventfds

2017-09-20 Thread Alexey Kardashevskiy
So it is called (twice) from the same function. This is to make the next patches a bit simpler. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- memory.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/memory.c b/memory.c index eb730cd619..9868d0f57c

[Qemu-devel] [PATCH qemu v4 06/18] memory: Switch memory from using AddressSpace to FlatView

2017-09-20 Thread Alexey Kardashevskiy
[]; } subpage_t; struct MemoryRegionSection { MemoryRegion *mr; -AddressSpace *address_space; +FlatView *fv; hwaddr offset_within_region; Int128 size; hwaddr offset_within_address_space; bool readonly; }; This should cause no behavioural change. Signed-off-by: Alexey

[Qemu-devel] [PATCH qemu v4 10/18] memory: Alloc dispatch tree where topology is generared

2017-09-20 Thread Alexey Kardashevskiy
This is to make next patches simpler. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- memory.c | 18 +- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/memory.c b/memory.c index 98b0b95e6e..eb730cd619 100644 --- a/memory.c +++ b/memory.c @@ -742,6

[Qemu-devel] [PATCH qemu v4 17/18] memory: Create FlatView directly

2017-09-20 Thread Alexey Kardashevskiy
unt of temporary dispatch trees (~37 -> ~30), the total memory footprint goes down (18G -> 17G). Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- memory.c | 18 +++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/memory.c b/memory.c i

[Qemu-devel] [PATCH qemu v4 05/18] memory: Remove AddressSpace pointer from AddressSpaceDispatch

2017-09-20 Thread Alexey Kardashevskiy
AS in ASD is only used to pass AS from mem_begin() to register_subpage() to store it in MemoryRegionSection, we can do this directly now. This should cause no behavioural change. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- exec.c | 15 +++ 1 file changed, 7 inse

[Qemu-devel] [PATCH qemu v4 18/18] memory: Give memory_region_transaction_commit a hint

2017-09-20 Thread Alexey Kardashevskiy
. memory_region_set_enabled() is a special case here, it does not use a hint when MR is being disabled. On POWER8 with 255 CPUs, 255 virtio-net, 40 PCI bridges guest this brings down the boot time from 20s to 12s, the total memory footprint goes down (17G -> 8G). Signed-off-by: Alexey Kardashevskiy <a...@ozl

[Qemu-devel] [PATCH qemu v4 03/18] memory: Move FlatView allocation to a helper

2017-09-20 Thread Alexey Kardashevskiy
This moves a FlatView allocation and initialization to a helper. While we are nere, replace g_new with g_new0 to not to bother if we add new fields in the future. This should cause no behavioural change. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- Changes: v3: * s/flatview

[Qemu-devel] [PATCH qemu v4 07/18] memory: Cleanup after switching to FlatView

2017-09-20 Thread Alexey Kardashevskiy
We store AddressSpaceDispatch* in FlatView anyway so there is no need to carry it from mem_add() to register_subpage/register_multipage. This should cause no behavioural change. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- exec.c | 16 1 file changed, 8 inse

[Qemu-devel] [PATCH qemu v4 00/18] memory: Reduce memory use

2017-09-20 Thread Alexey Kardashevskiy
rework This is based on sha1 c51700273a Peter Maydell "Merge remote-tracking branch 'remotes/cohuck/tags/s390x-20170919-v2' into staging". Please comment. Thanks. Alexey Kardashevskiy (18): exec: Explicitly export target AS from address_space_translate_internal memory:

[Qemu-devel] [PATCH qemu v4 08/18] memory: Rename mem_begin/mem_commit/mem_add helpers

2017-09-20 Thread Alexey Kardashevskiy
This renames some helpers to reflect better what they do. This should cause no behavioural change. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- Changes: v3: * s/flatview_mem_add/flatview_add_to_dispatch/ * s/phys_page_compact_all/address_space_dispatch_compac

[Qemu-devel] [PATCH qemu v4 02/18] memory: Open code FlatView rendering

2017-09-20 Thread Alexey Kardashevskiy
() if there is no registered listeners; this should improve starting time. This should cause no behavioural change. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- Changes: v4: * address_space_update_topology_pass() is called now under if (!QTAILQ_EMPTY(>listeners)) v3: * inlined &am

[Qemu-devel] [PATCH qemu v4 01/18] exec: Explicitly export target AS from address_space_translate_internal

2017-09-20 Thread Alexey Kardashevskiy
This adds an AS** parameter to address_space_do_translate() to make it easier for the next patch to share FlatViews. This should cause no behavioural change. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- exec.c | 11 +++ 1 file changed, 7 insertions(+), 4 deletions(-)

Re: [Qemu-devel] [PATCH qemu v3 01/13] memory: Postpone flatview and dispatch tree building till all devices are added

2017-09-19 Thread Alexey Kardashevskiy
On 19/09/17 20:47, Paolo Bonzini wrote: > On 19/09/2017 12:39, Alexey Kardashevskiy wrote: >>> No, there's still >>> >>> as->dispatch = NULL; >>> memory_region_update_pending |= root->enabled; >>> memory_region_transaction_co

Re: [Qemu-devel] [PATCH qemu v3 01/13] memory: Postpone flatview and dispatch tree building till all devices are added

2017-09-19 Thread Alexey Kardashevskiy
On 19/09/17 17:42, Paolo Bonzini wrote: > On 19/09/2017 08:57, Alexey Kardashevskiy wrote: >>>> 2) also, after patch 2 you know that the address space has no listeners >>>> here, so the begin/commit isn't really needed. Instead you can use the >>>&g

Re: [Qemu-devel] [PATCH qemu v3 01/13] memory: Postpone flatview and dispatch tree building till all devices are added

2017-09-19 Thread Alexey Kardashevskiy
On 19/09/17 17:40, Paolo Bonzini wrote: > On 19/09/2017 08:57, Alexey Kardashevskiy wrote: >>>> 3) you can consult the list (or hash table :)) of live FlatViews (which >>>> means you keep it live after memory_region_transaction_commit ends, and >>>> onl

Re: [Qemu-devel] [PATCH qemu v3 01/13] memory: Postpone flatview and dispatch tree building till all devices are added

2017-09-19 Thread Alexey Kardashevskiy
On 19/09/17 12:09, Alexey Kardashevskiy wrote: > On 19/09/17 00:54, Paolo Bonzini wrote: >> On 18/09/2017 12:16, Alexey Kardashevskiy wrote: >>> Most devices use at least one address space and every time a new address >>> space is added, flat views and dispatch trees

Re: [Qemu-devel] [PATCH qemu v3 01/13] memory: Postpone flatview and dispatch tree building till all devices are added

2017-09-18 Thread Alexey Kardashevskiy
On 19/09/17 00:54, Paolo Bonzini wrote: > On 18/09/2017 12:16, Alexey Kardashevskiy wrote: >> Most devices use at least one address space and every time a new address >> space is added, flat views and dispatch trees are rebuild for all address >> spaces. This is not a pro

Re: [Qemu-devel] [PATCH qemu v3 00/13] memory: Reduce memory use

2017-09-18 Thread Alexey Kardashevskiy
On 19/09/17 00:59, Paolo Bonzini wrote: > On 18/09/2017 12:16, Alexey Kardashevskiy wrote: >> This was inspired by https://bugzilla.redhat.com/show_bug.cgi?id=1481593 >> Previous versions: >> v1: https://lists.gnu.org/archive/html/qemu-devel/2017-09/msg01559.html >>

Re: [Qemu-devel] [PATCH qemu v3 02/13] exec: Explicitely export target AS from address_space_translate_internal

2017-09-18 Thread Alexey Kardashevskiy
On 19/09/17 00:28, Eric Blake wrote: > On 09/18/2017 05:16 AM, Alexey Kardashevskiy wrote: > > Subject line: s/Explicitely/Explicitly/ > >> This is not so mechanical change in order to move to shared FlatViews >> so make it a separate patch. > > Hard to un

Re: [Qemu-devel] [PATCH qemu v3 07/13] memory: Switch memory from using AddressSpace to FlatView

2017-09-18 Thread Alexey Kardashevskiy
On 18/09/17 20:17, Alexey Kardashevskiy wrote: > FlatView's will be shared between AddressSpace's and subpage_t > and MemoryRegionSection cannot store AS anymore, hence this change. > > In particular, for: > > typedef struct subpage_t { > MemoryRegion iomem

[Qemu-devel] [PATCH qemu v3 05/13] memory: Move AddressSpaceDispatch from AddressSpace to FlatView

2017-09-18 Thread Alexey Kardashevskiy
ack and set to an AS atomically. flatview_destroy() is executed under RCU instead of address_space_dispatch_free() now. This makes mem_begin/mem_commit to work with ASD and mem_add with FV as later on mem_add will be taking FV as an argument anyway. Signed-off-by: Alexey Kardashevskiy <a...@ozl

[Qemu-devel] [PATCH qemu v3 02/13] exec: Explicitely export target AS from address_space_translate_internal

2017-09-18 Thread Alexey Kardashevskiy
This is not so mechanical change in order to move to shared FlatViews so make it a separate patch. The first argument of address_space_do_translate() will become a FlatView, however since address_space_get_iotlb_entry() still wants AS, hence this change. Signed-off-by: Alexey Kardashevskiy

[Qemu-devel] [PATCH qemu v3 12/13] memory: Get rid of address_space_init_shareable

2017-09-18 Thread Alexey Kardashevskiy
Since FlatViews are shared now and ASes not, this gets rid of address_space_init_shareable(). Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- Changes: v3: * now removes @malloced and @ref_count, used to be in the previos patch --- include/exec/memory.h

[Qemu-devel] [PATCH qemu v3 09/13] memory: Rename mem_begin/mem_commit/mem_add helpers

2017-09-18 Thread Alexey Kardashevskiy
This renames some helper to reflect better what they do. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- Changes: v3: * s/flatview_mem_add/flatview_add_to_dispatch/ * s/phys_page_compact_all/address_space_dispatch_compact/ * s/address_space_dispatch_alloc/address_space_dispat

[Qemu-devel] [PATCH qemu v3 10/13] memory: Store physical root MR in FlatView

2017-09-18 Thread Alexey Kardashevskiy
Address spaces get to keep a root MR (alias or not) but FlatView stores the actual MR as this is going to be used later on to decide whether to share a particular FlatView or not. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- memory.c | 27 ++- 1 file c

[Qemu-devel] [PATCH qemu v3 11/13] memory: Share FlatView's and dispatch trees between address spaces

2017-09-18 Thread Alexey Kardashevskiy
This allows sharing flat views between address spaces when the same root memory region is used when creating a new address space. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- Changes: v3: * got rid of global and per-AS lists of FlatView objects --- I could also make all

[Qemu-devel] [PATCH qemu v3 13/13] memory: Add flat views to HMP "info mtree"

2017-09-18 Thread Alexey Kardashevskiy
This adds a new switch to "info mtree" to print dispatch tree internals. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- Changes: v3: * reimplemented as there is no more global FlatView list --- Example: aik@fstn1-p1:~$ echo "info mtree -f -d" | nc localhost

[Qemu-devel] [PATCH qemu v3 08/13] memory: Cleanup after switching to FlatView

2017-09-18 Thread Alexey Kardashevskiy
We store AddressSpaceDispatch* in FlatView anyway so there is no need to carry it from mem_add() to register_subpage/register_multipage. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- exec.c | 16 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/ex

[Qemu-devel] [PATCH qemu v3 07/13] memory: Switch memory from using AddressSpace to FlatView

2017-09-18 Thread Alexey Kardashevskiy
[]; } subpage_t; struct MemoryRegionSection { MemoryRegion *mr; -AddressSpace *address_space; +FlatView *fv; hwaddr offset_within_region; Int128 size; hwaddr offset_within_address_space; bool readonly; }; Signed-off-by: Alexey Kardashevskiy <a...@ozlabs

[Qemu-devel] [PATCH qemu v3 04/13] memory: Move FlatView allocation to a helper

2017-09-18 Thread Alexey Kardashevskiy
This moves a FlatView allocation and initialization to a helper. While we are nere, replace g_new with g_new0 to not to bother if we add new fields in the future. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- Changes: v3: * s/flatview_alloc/flatview_new/ --- memory.

[Qemu-devel] [PATCH qemu v3 00/13] memory: Reduce memory use

2017-09-18 Thread Alexey Kardashevskiy
aging". Please comment. Thanks. Alexey Kardashevskiy (13): memory: Postpone flatview and dispatch tree building till all devices are added exec: Explicitely export target AS from address_space_translate_internal memory: Open code FlatView rendering memory: Move FlatView

[Qemu-devel] [PATCH qemu v3 03/13] memory: Open code FlatView rendering

2017-09-18 Thread Alexey Kardashevskiy
We are going to share FlatView's between AddressSpace's and per-AS memory listeners won't suite the purpose anymore so open code the dispatch tree rendering. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- Changes: v3: * inlined & simplified address_space_update_flatview --

[Qemu-devel] [PATCH qemu v3 06/13] memory: Remove AddressSpace pointer from AddressSpaceDispatch

2017-09-18 Thread Alexey Kardashevskiy
AS in ASD is only used to pass AS from mem_begin() to register_subpage() to store it in MemoryRegionSection, we can do this directly now. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- exec.c | 15 +++ 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/ex

[Qemu-devel] [PATCH qemu v3 01/13] memory: Postpone flatview and dispatch tree building till all devices are added

2017-09-18 Thread Alexey Kardashevskiy
)allocations. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- Changes: v2: * wrapped qemu_run_machine_init_done_notifiers() as well --- vl.c | 9 + 1 file changed, 9 insertions(+) diff --git a/vl.c b/vl.c index 9e62e92aea..e4f2ece590 100644 --- a/vl.c +++ b/vl.c @@ -4

Re: [Qemu-devel] [PATCH qemu v2 11/13] memory: Share FlatView's and dispatch trees between address spaces

2017-09-15 Thread Alexey Kardashevskiy
On 15/09/17 19:25, Paolo Bonzini wrote: > On 15/09/2017 10:40, Alexey Kardashevskiy wrote: >> + >> +static bool flatview_can_share(FlatView *old_view, FlatView *new_view) >> +{ >> +MemoryRegion *old_root = memory_region_unalias_entire(old_view->root);

[Qemu-devel] [PATCH qemu v2 01/13] memory: Postpone flatview and dispatch tree building till all devices are added

2017-09-15 Thread Alexey Kardashevskiy
)allocations. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- Changes: v2: * wrapped qemu_run_machine_init_done_notifiers() as well --- vl.c | 9 + 1 file changed, 9 insertions(+) diff --git a/vl.c b/vl.c index fb1f05b937..ee3cc7db48 100644 --- a/vl.c +++ b/vl.c @@ -4

[Qemu-devel] [PATCH qemu v2 05/13] memory: Move AddressSpaceDispatch from AddressSpace to FlatView

2017-09-15 Thread Alexey Kardashevskiy
ack and set to an AS atomically. flatview_destroy() is executed under RCU instead of address_space_dispatch_free() now. This makes mem_begin/mem_commit to work with ASD and mem_add with FV as later on mem_add will be taking FV as an argument anyway. Signed-off-by: Alexey Kardashevskiy <a...@ozl

[Qemu-devel] [PATCH qemu v2 07/13] memory: Switch memory from using AddressSpace to FlatView

2017-09-15 Thread Alexey Kardashevskiy
[]; } subpage_t; struct MemoryRegionSection { MemoryRegion *mr; -AddressSpace *address_space; +FlatView *fv; hwaddr offset_within_region; Int128 size; hwaddr offset_within_address_space; bool readonly; }; Signed-off-by: Alexey Kardashevskiy <a...@ozlabs

[Qemu-devel] [PATCH qemu v2 00/13] memory: Reduce memory use

2017-09-15 Thread Alexey Kardashevskiy
: v2: * total rework This is based on sha1 3dabde1128 Peter Maydell "Merge remote-tracking branch 'remotes/dgilbert/tags/pull-hmp-20170914' into staging". Please comment. Thanks. Alexey Kardashevskiy (13): memory: Postpone flatview and dispatch tree building till all devices

[Qemu-devel] [PATCH qemu v2 11/13] memory: Share FlatView's and dispatch trees between address spaces

2017-09-15 Thread Alexey Kardashevskiy
This allows sharing flat views between address spaces when the same root memory region is used when creating a new address space. This adds a global list of flat views and a list of attached address spaces per a flat view. Each address space references a flat view. Signed-off-by: Alexey

[Qemu-devel] [PATCH qemu v2 02/13] exec: Explicitely export target AS from address_space_translate_internal

2017-09-15 Thread Alexey Kardashevskiy
This is not so mechanical change in order to move to shared FlatViews so make it a separate patch. The first argument of address_space_do_translate() will become a FlatView, however since address_space_get_iotlb_entry() still wants AS, hence this change. Signed-off-by: Alexey Kardashevskiy

[Qemu-devel] [PATCH qemu v2 12/13] memory: Get rid of address_space_init_shareable

2017-09-15 Thread Alexey Kardashevskiy
Since FlatViews are shared now and ASes not, this gets rid of address_space_init_shareable(). Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- Squash to the previous one? It is a separate patch to make review easier, mostly. --- include/exec/memory.h | 17 - incl

[Qemu-devel] [PATCH qemu v2 10/13] memory: Move root MR from AddressSpace to FlatView

2017-09-15 Thread Alexey Kardashevskiy
Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- include/exec/memory.h | 2 +- hw/pci/pci.c | 3 ++- memory.c | 30 -- 3 files changed, 19 insertions(+), 16 deletions(-) diff --git a/include/exec/memory.h b/include/exec/memory.h

[Qemu-devel] [PATCH qemu v2 13/13] memory: Add flat views to HMP "info mtree"

2017-09-15 Thread Alexey Kardashevskiy
This adds a new switch to "info mtree" to print dispatch tree internals. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- Example: aik@fstn1-p1:~$ echo "info mtree -f -d" | nc localhost 3 QEMU 2.9.94 monitor - type 'help' for more information (qemu) in

[Qemu-devel] [PATCH qemu v2 09/13] memory: Rename mem_begin/mem_commit/mem_add helpers

2017-09-15 Thread Alexey Kardashevskiy
This renames some helper to reflect better what they do. This drops AS in address_space_update_flatview() as it is not used anymore and renames to flatview_render_new(). Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- include/exec/memory-internal.h | 6 +++---

[Qemu-devel] [PATCH qemu v2 06/13] memory: Remove AddressSpace pointer from AddressSpaceDispatch

2017-09-15 Thread Alexey Kardashevskiy
AS in ASD is only used to pass AS from mem_begin() to register_subpage() to store it in MemoryRegionSection, we can do this directly now. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- exec.c | 15 +++ 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/ex

[Qemu-devel] [PATCH qemu v2 03/13] memory: Open code FlatView rendering

2017-09-15 Thread Alexey Kardashevskiy
We are going to share FlatView's between AddressSpace's and per-AS memory listeners won't suite the purpose anymore so open code the dispatch tree rendering. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- include/exec/memory-internal.h | 6 +++-- include/exec/memory.h

[Qemu-devel] [PATCH qemu v2 04/13] memory: Move FlatView allocation to a helper

2017-09-15 Thread Alexey Kardashevskiy
This moves a FlatView allocation and initialization to a helper. While we are nere, replace g_new with g_new0 to not to bother if we add new fields in the future. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- memory.c | 16 1 file changed, 8 insertions(+), 8 del

[Qemu-devel] [PATCH qemu v2 08/13] memory: Cleanup after switching to FlatView

2017-09-15 Thread Alexey Kardashevskiy
We store AddressSpaceDispatch* in FlatView anyway so there is no need to carry it from mem_add() to register_subpage/register_multipage. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- exec.c | 16 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/ex

[Qemu-devel] [PATCH qemu v2] pci: Initialize pci_dev->name before use

2017-09-15 Thread Alexey Kardashevskiy
This moves pci_dev->name initialization earlier so pci_dev->bus_master_as could get a name instead of an empty string. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> Reviewed-by: Philippe Mathieu-Daudé <f4...@amsat.org> --- Changes: v2: * fixed mistype in the commit log *

Re: [Qemu-devel] [Qemu-ppc] [PATCH 2/4] ppc: add CPU IRQ state to PPC VMStateDescription

2017-09-12 Thread Alexey Kardashevskiy
On 13/09/17 02:46, Mark Cave-Ayland wrote: > On 12/09/17 17:41, Mark Cave-Ayland wrote: > >> The commit message mentions that prior to the conversion some CPU state >> was missing but it doesn't mention anything about dropping existing >> fields as part of the conversion process so I suspect that

Re: [Qemu-devel] [RFC PATCH qemu 3/4] memory: Share flat views and dispatch trees between address spaces

2017-09-12 Thread Alexey Kardashevskiy
On 12/09/17 17:12, Paolo Bonzini wrote: > On 12/09/2017 07:55, Alexey Kardashevskiy wrote: >> On 12/09/17 01:30, Paolo Bonzini wrote: >>> On 11/09/2017 14:08, Alexey Kardashevskiy wrote: >>>>> Ok, this makes sense. Maybe it should be a flatview rather t

Re: [Qemu-devel] [RFC PATCH qemu 3/4] memory: Share flat views and dispatch trees between address spaces

2017-09-11 Thread Alexey Kardashevskiy
On 12/09/17 01:30, Paolo Bonzini wrote: > On 11/09/2017 14:08, Alexey Kardashevskiy wrote: >>> Ok, this makes sense. Maybe it should be a flatview rather than an >>> AddressSpaceDispatch (a FlatView is essentially a list of >>> MemoryRegionSections; attaching t

[Qemu-devel] [PATCH qemu] vfio, spapr: Fix levels calculation

2017-09-11 Thread Alexey Kardashevskiy
;128GB guests with standard 64K pages as this requires blocks with zone order 9 and the popular limit of CONFIG_FORCE_MAX_ZONEORDER=9 means that only blocks up to order 8 are allowed. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- hw/vfio/spapr.c | 2 +- 1 file changed, 1 insertion(+),

Re: [Qemu-devel] [RFC PATCH qemu 3/4] memory: Share flat views and dispatch trees between address spaces

2017-09-11 Thread Alexey Kardashevskiy
On 11/09/17 19:37, Paolo Bonzini wrote: > On 11/09/2017 11:06, Alexey Kardashevskiy wrote: >> On 11/09/17 17:40, Paolo Bonzini wrote: >>> On 07/09/2017 11:20, Alexey Kardashevskiy wrote: >>>> >>>> /* Accessed via RCU. */ >>>>

Re: [Qemu-devel] [RFC PATCH qemu 3/4] memory: Share flat views and dispatch trees between address spaces

2017-09-11 Thread Alexey Kardashevskiy
On 11/09/17 17:40, Paolo Bonzini wrote: > On 07/09/2017 11:20, Alexey Kardashevskiy wrote: >> >> /* Accessed via RCU. */ >> struct FlatView *current_map; >> >> int ioeventfd_nb; >> struct MemoryRegionIoeventfd *ioeventfds; >&g

[Qemu-devel] [PATCH qemu] xhci: Avoid DMA when ERSTBA is set to zero

2017-09-11 Thread Alexey Kardashevskiy
Segment Table Size Register (ERSTSZ). Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- On pseries, the SLOF firmware initializes XHCI and sets non-zero value to ERSTBA. Then, it jumps to the guest and the guest requests the SLOF to quiesce devices, that includes XHCI. SLOF remov

Re: [Qemu-devel] [RFC PATCH qemu 2/4] memory: Prepare for shared flat views

2017-09-10 Thread Alexey Kardashevskiy
On 09/09/17 17:18, David Gibson wrote: > On Thu, Sep 07, 2017 at 07:20:08PM +1000, Alexey Kardashevskiy wrote: >> We are going to share flat views and dispatch trees between address >> spaces. This moves bits around but no change in behaviour is expected >> here. The following

Re: [Qemu-devel] [RFC PATCH qemu 1/4] memory: Postpone flatview and dispatch tree building till all devices are added

2017-09-08 Thread Alexey Kardashevskiy
On 08/09/17 00:30, Peter Maydell wrote: > On 7 September 2017 at 15:27, Alexey Kardashevskiy <a...@ozlabs.ru> wrote: >> On 07/09/17 19:30, Peter Maydell wrote: >>> What happens if something in a device realize function tries >>> to do a read from an AddressSpa

Re: [Qemu-devel] [RFC PATCH qemu 0/4] memory: Reduce memory use

2017-09-07 Thread Alexey Kardashevskiy
On 08/09/17 12:08, Alexey Kardashevskiy wrote: > On 08/09/17 00:54, Dr. David Alan Gilbert wrote: >> * Alexey Kardashevskiy (a...@ozlabs.ru) wrote: >>> On 07/09/17 19:51, Dr. David Alan Gilbert wrote: >>>> * Alexey Kardashevskiy (a...@ozlabs.ru) wrote: &g

Re: [Qemu-devel] [RFC PATCH qemu 0/4] memory: Reduce memory use

2017-09-07 Thread Alexey Kardashevskiy
On 08/09/17 00:54, Dr. David Alan Gilbert wrote: > * Alexey Kardashevskiy (a...@ozlabs.ru) wrote: >> On 07/09/17 19:51, Dr. David Alan Gilbert wrote: >>> * Alexey Kardashevskiy (a...@ozlabs.ru) wrote: >>>> This was inspired by https://bugzilla.redhat.com/show_bug

Re: [Qemu-devel] [RFC PATCH qemu 3/4] memory: Share flat views and dispatch trees between address spaces

2017-09-07 Thread Alexey Kardashevskiy
On 08/09/17 06:53, Philippe Mathieu-Daudé wrote: > On 09/07/2017 06:20 AM, Alexey Kardashevskiy wrote: >> This allows sharing flat views between address spaces when the same root >> memory region is used when creating a new address space. >> >> This adds a global li

Re: [Qemu-devel] [RFC PATCH qemu 0/4] memory: Reduce memory use

2017-09-07 Thread Alexey Kardashevskiy
On 07/09/17 19:51, Dr. David Alan Gilbert wrote: > * Alexey Kardashevskiy (a...@ozlabs.ru) wrote: >> This was inspired by https://bugzilla.redhat.com/show_bug.cgi?id=1481593 >> >> What happens ithere is that every virtio block device creates 2 address >> spaces - for

Re: [Qemu-devel] [RFC PATCH qemu 1/4] memory: Postpone flatview and dispatch tree building till all devices are added

2017-09-07 Thread Alexey Kardashevskiy
On 07/09/17 19:30, Peter Maydell wrote: > On 7 September 2017 at 10:20, Alexey Kardashevskiy <a...@ozlabs.ru> wrote: >> Most devices use at least one address space and every time a new address >> space is added, flat views and dispatch trees are rebuild for

[Qemu-devel] [RFC PATCH qemu 2/4] memory: Prepare for shared flat views

2017-09-07 Thread Alexey Kardashevskiy
address_space_get_iotlb_entry() work. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- include/exec/memory.h | 79 +++--- exec.c | 128 +++- hw/alpha/typhoon.c | 2 +- hw/dma/rc4030.c | 4 +- h

[Qemu-devel] [RFC PATCH qemu 0/4] memory: Reduce memory use

2017-09-07 Thread Alexey Kardashevskiy
one provides visible outcome. There are still things to polish and double check the use of RCU, I'd like to get any feedback before proceeding - is this going the right way or way too ugly? This is based on sha1 1ab5eb4efb Peter Maydell "Update version for v2.10.0 release". Please comment. T

[Qemu-devel] [RFC PATCH qemu 3/4] memory: Share flat views and dispatch trees between address spaces

2017-09-07 Thread Alexey Kardashevskiy
tree building instead of doing so via a memory listener. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- This was suggested by Paolo in https://lists.gnu.org/archive/html/qemu-devel/2017-08/msg05011.html I am not putting "Suggested-by" as I want to make sure that th

[Qemu-devel] [RFC PATCH qemu 1/4] memory: Postpone flatview and dispatch tree building till all devices are added

2017-09-07 Thread Alexey Kardashevskiy
)allocations. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- vl.c | 4 1 file changed, 4 insertions(+) diff --git a/vl.c b/vl.c index 8e247cc2a2..3c39cc8b3a 100644 --- a/vl.c +++ b/vl.c @@ -4655,12 +4655,16 @@ int main(int argc, char **argv, char **envp) igd_gfx_pa

[Qemu-devel] [RFC PATCH qemu 4/4] memory: Add flat views to HMP "info mtree"

2017-09-07 Thread Alexey Kardashevskiy
This adds a new switch to "info mtree" to print dispatch tree internals. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- Example: aik@fstn1-p1:~$ echo "info mtree -f -d" | nc localhost 3 QEMU 2.9.94 monitor - type 'help' for more information (qemu) in

Re: [Qemu-devel] [PATCH] qcow2: allocate cluster_cache/cluster_data on demand

2017-08-30 Thread Alexey Kardashevskiy
On 31/08/17 03:20, Stefan Hajnoczi wrote: > On Tue, Aug 22, 2017 at 02:56:00PM +1000, Alexey Kardashevskiy wrote: >> On 19/08/17 12:46, Alexey Kardashevskiy wrote: >>> On 19/08/17 01:18, Eric Blake wrote: >>>> On 08/18/2017 08:31 AM, Stefan Hajnoczi wrote: >>&

Re: [Qemu-devel] [RFC PATCH qemu] exec: Destroy dispatch immediately

2017-08-29 Thread Alexey Kardashevskiy
On 25/08/17 18:53, Paolo Bonzini wrote: > On 25/08/2017 10:31, Alexey Kardashevskiy wrote: >> Otherwise old dispatch holds way too much memory before RCU gets >> a chance to free old dispatches. >> >> Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru>

Re: [Qemu-devel] [PATCH qemu v4 2/3] vfio/spapr: Add a notifier for PPC64 HV/PR KVM about new group attached to LIOBN

2017-08-27 Thread Alexey Kardashevskiy
On 25/08/17 16:21, David Gibson wrote: > On Thu, Jul 20, 2017 at 05:22:30PM +1000, Alexey Kardashevskiy wrote: >> This implements a notification for a new IOMMU group attached to >> sPAPR's logical IO bus (LIOBN) to enable in-kernel TCE acceleration. >

Re: [Qemu-devel] [RFC PATCH qemu] exec: Destroy dispatch immediately

2017-08-25 Thread Alexey Kardashevskiy
On 25/08/17 19:01, Paolo Bonzini wrote: > On 25/08/2017 10:31, Alexey Kardashevskiy wrote: >> Each address_space_init() updates topology for _every_ address space. > > And finally, does this patch help with the above? > > diff --git a/memory.c b/memory.c > index

[Qemu-devel] [RFC PATCH qemu] exec: Destroy dispatch immediately

2017-08-25 Thread Alexey Kardashevskiy
Otherwise old dispatch holds way too much memory before RCU gets a chance to free old dispatches. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- This is a follow-up to the "Memory use with >100 virtio devices" thread. I assume this is a dirty hack (which fixes

[Qemu-devel] [PATCH qemu] pci: Initialize pci_dev->name before use

2017-08-24 Thread Alexey Kardashevskiy
This moves pci_dev->name initialization earlier so pci_dev->bus_master_as could get a name instead of empry string. Signed-off-by: Alexey Kardashevskiy <a...@ozlabs.ru> --- hw/pci/pci.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/pci/pci.c b/hw/pci

Re: [Qemu-devel] Memory use with >100 virtio devices

2017-08-24 Thread Alexey Kardashevskiy
On 21/08/17 15:50, Alexey Kardashevskiy wrote: > On 21/08/17 14:31, David Gibson wrote: >> On Fri, Aug 18, 2017 at 02:18:53PM +0100, Stefan Hajnoczi wrote: >>> On Fri, Aug 18, 2017 at 03:39:20PM +1000, Alexey Kardashevskiy wrote: >>>> ==94451=

Re: [Qemu-devel] [PATCH] qcow2: allocate cluster_cache/cluster_data on demand

2017-08-21 Thread Alexey Kardashevskiy
On 19/08/17 12:46, Alexey Kardashevskiy wrote: > On 19/08/17 01:18, Eric Blake wrote: >> On 08/18/2017 08:31 AM, Stefan Hajnoczi wrote: >>> Most qcow2 files are uncompressed so it is wasteful to allocate (32 + 1) >>> * cluster_size + 512 bytes upfront. Allocate

Re: [Qemu-devel] Memory use with >100 virtio devices

2017-08-20 Thread Alexey Kardashevskiy
On 21/08/17 14:31, David Gibson wrote: > On Fri, Aug 18, 2017 at 02:18:53PM +0100, Stefan Hajnoczi wrote: >> On Fri, Aug 18, 2017 at 03:39:20PM +1000, Alexey Kardashevskiy wrote: >>> ==94451== 4 of 10 >>> ==94451== max-live:

Re: [Qemu-devel] [PATCH] qcow2: allocate cluster_cache/cluster_data on demand

2017-08-19 Thread Alexey Kardashevskiy
On 19/08/17 12:53, Alexey Kardashevskiy wrote: > On 19/08/17 12:46, Alexey Kardashevskiy wrote: >> On 19/08/17 01:18, Eric Blake wrote: >>> On 08/18/2017 08:31 AM, Stefan Hajnoczi wrote: >>>> Most qcow2 files are uncompressed so it is wasteful to allocate (32 + 1) &

<    4   5   6   7   8   9   10   11   12   13   >