Re: [virtio-dev] Re: guest / host buffer sharing ...

2019-12-12 Thread David Stevens
> > > Without buffer sharing support the driver importing a virtio-gpu dma-buf > > > can send the buffer scatter list to the host. So both virtio-gpu and > > > the other device would actually access the same guest pages, but they > > > are not aware that the buffer is shared between devices. > > >

[virtio-dev] [PATCH RFC v4 13/13] virtio-mem: Drop slab objects when unplug continues to fail

2019-12-12 Thread David Hildenbrand
Start dropping slab objects after 30 minutes and repeat every 30 minutes in case we can't unplug more memory using alloc_contig_range(). Log messages and make it configurable. Enable dropping slab objects as default (especially, reclaimable slab objects that are not movable). In the future, we mig

[virtio-dev] [PATCH RFC v4 11/13] mm/vmscan: Move count_vm_event(DROP_SLAB) into drop_slab()

2019-12-12 Thread David Hildenbrand
Let's count within the function itself, so every invocation (of future users) will be counted. Cc: Alexander Viro Cc: Andrew Morton Cc: linux-fsde...@vger.kernel.org Signed-off-by: David Hildenbrand --- fs/drop_caches.c | 4 +--- mm/vmscan.c | 1 + 2 files changed, 2 insertions(+), 3 dele

[virtio-dev] [PATCH RFC v4 09/13] virtio-mem: Offline and remove completely unplugged memory blocks

2019-12-12 Thread David Hildenbrand
Let's offline+remove memory blocks once all subblocks are unplugged. We can use the new Linux MM interface for that. As no memory is in use anymore, this shouldn't take a long time and shouldn't fail. There might be corner cases where the offlining could still fail (especially, if another notifier

[virtio-dev] [PATCH RFC v4 12/13] mm/vmscan: Export drop_slab() and drop_slab_node()

2019-12-12 Thread David Hildenbrand
We already have a way to trigger reclaiming of all reclaimable slab objects from user space (echo 2 > /proc/sys/vm/drop_caches). Let's allow drivers to also trigger this when they really want to make progress and know what they are doing. virtio-mem wants to use these functions when it failed to u

[virtio-dev] [PATCH RFC v4 10/13] virtio-mem: Better retry handling

2019-12-12 Thread David Hildenbrand
Let's start with a retry interval of 30 seconds and double the time until we reach 30 minutes, in case we keep getting errors. Reset the retry interval in case we succeeded. Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Oscar Salvador Cc: Michal Hocko Cc: Igor Mammedov Cc: Dave Young Cc: Andre

[virtio-dev] [PATCH RFC v4 08/13] mm/memory_hotplug: Introduce offline_and_remove_memory()

2019-12-12 Thread David Hildenbrand
virtio-mem wants to offline and remove a memory block once it unplugged all subblocks (e.g., using alloc_contig_range()). Let's provide an interface to do that from a driver. virtio-mem already supports to offline partially unplugged memory blocks. Offlining a fully unplugged memory block will not

[virtio-dev] [PATCH RFC v4 03/13] virtio-mem: Paravirtualized memory hotunplug part 1

2019-12-12 Thread David Hildenbrand
Unplugging subblocks of memory blocks that are offline is easy. All we have to do is watch out for concurrent onlining acticity. Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Oscar Salvador Cc: Michal Hocko Cc: Igor Mammedov Cc: Dave Young Cc: Andrew Morton Cc: Dan Williams Cc: Pavel Tatashi

[virtio-dev] [PATCH RFC v4 02/13] virtio-mem: Paravirtualized memory hotplug

2019-12-12 Thread David Hildenbrand
Each virtio-mem device owns exactly one memory region. It is responsible for adding/removing memory from that memory region on request. When the device driver starts up, the requested amount of memory is queried and then plugged to Linux. On request, further memory can be plugged or unplugged. Thi

[virtio-dev] [PATCH RFC v4 07/13] virtio-mem: Allow to offline partially unplugged memory blocks

2019-12-12 Thread David Hildenbrand
Dropping the reference count of PageOffline() pages allows offlining code to skip them. However, we also have to convert PG_reserved to another flag - let's use PG_dirty - so has_unmovable_pages() will properly handle them. PG_reserved pages get detected as unmovable right away. We need the flag t

[virtio-dev] [PATCH RFC v4 06/13] mm: Allow to offline unmovable PageOffline() pages via MEM_GOING_OFFLINE

2019-12-12 Thread David Hildenbrand
virtio-mem wants to allow to offline memory blocks of which some parts were unplugged (allocated via alloc_contig_range()), especially, to later offline and remove completely unplugged memory blocks. The important part is that PageOffline() has to remain set until the section is offline, so these p

[virtio-dev] [PATCH RFC v4 04/13] mm: Export alloc_contig_range() / free_contig_range()

2019-12-12 Thread David Hildenbrand
A virtio-mem device wants to allocate memory from the memory region it manages in order to unplug it in the hypervisor - similar to a balloon driver. Also, it might want to plug previously unplugged (allocated) memory and give it back to Linux. alloc_contig_range() / free_contig_range() seem to be

[virtio-dev] [PATCH RFC v4 05/13] virtio-mem: Paravirtualized memory hotunplug part 2

2019-12-12 Thread David Hildenbrand
We can use alloc_contig_range() to try to unplug subblocks. Unplugged blocks will be marked PG_offline, however, don't have the PG_reserved flag set. This way, we can differentiate these allocated subblocks from subblocks that were never onlined and handle them properly in virtio_mem_fake_online().

[virtio-dev] [PATCH RFC v4 01/13] ACPI: NUMA: export pxm_to_node

2019-12-12 Thread David Hildenbrand
Will be needed by virtio-mem to identify the node from a pxm. Cc: "Rafael J. Wysocki" Cc: Len Brown Cc: linux-a...@vger.kernel.org Signed-off-by: David Hildenbrand --- drivers/acpi/numa/srat.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/acpi/numa/srat.c b/drivers/acpi/numa/srat

[virtio-dev] [PATCH RFC v4 00/13] virtio-mem: paravirtualized memory

2019-12-12 Thread David Hildenbrand
This series is based on latest linux-next. The patches are located at: https://github.com/davidhildenbrand/linux.git virtio-mem-rfc-v4 The basic idea of virtio-mem is to provide a flexible, cross-architecture memory hot(un)plug solution that avoids many limitations imposed by existing technolo

Re: [virtio-dev] Re: guest / host buffer sharing ...

2019-12-12 Thread Gerd Hoffmann
On Thu, Dec 12, 2019 at 09:26:32PM +0900, David Stevens wrote: > > > > Second I think it is a bad idea > > > > from the security point of view. When explicitly exporting buffers it > > > > is easy to restrict access to the actual exports. > > > > > > Restricting access to actual exports could perh

Re: [virtio-dev] Re: guest / host buffer sharing ...

2019-12-12 Thread David Stevens
> > > Second I think it is a bad idea > > > from the security point of view. When explicitly exporting buffers it > > > is easy to restrict access to the actual exports. > > > > Restricting access to actual exports could perhaps help catch bugs. > > However, I don't think it provides any security

Re: [virtio-dev] [RFC RESEND] virtio-video: Add virtio video device specification

2019-12-12 Thread Dmitry Sepp
Hi Keiichi, Thank you for your comment. What do you thing about selection/crop rectangles? Should this be defined as a must have? Or we just assume the device always sticks to the stream resolution. Regards, Dmitry. On Donnerstag, 12. Dezember 2019 06:39:11 CET Keiichi Watanabe wrote: > On Tue

Re: [virtio-dev] Re: guest / host buffer sharing ...

2019-12-12 Thread Gerd Hoffmann
Hi, > > First the addressing is non-trivial, especially with the "transport > > specific device address" in the tuple. > > There is complexity here, but I think it would also be present in the > buffer sharing device case. With a buffer sharing device, the same > identifying information would n