Re: [PATCH] Add virtio gpu driver.

2016-05-30 Thread Daniel Vetter
On Tue, May 31, 2016 at 8:29 AM, Gerd Hoffmann  wrote:
>>  but it also means that
>> atomic userspace can't use hotspots before we add properties to fbs. And
>> doing that is a bit tricky since drm_framebuffer objects are meant to be
>> invariant - this assumption is deeply in-grained into the code all over
>> the place, everything just compares pointers when semantically it means to
>> compare the entire fb (including backing storage pointer/offsets and
>> everything).
>
> Hmm, the hotspot location for a given cursor image is invariant too, so
> I don't think that would be a big issue.
>
> I'd expect userspace define a bunch of cursors, then switch between them
> (instead of defining a single cursor, then constantly updating it).

Agreed, conceptually it would be a really nice fit to put the hotspot
into the invariant drm_framebuffer. I just meant to say that you need
to add a bit of new uapi to make it happen, since we can't reuse our
existing get/set_prop functions (since those only change props after
the object is created). And we also can't use our existing ioctls to
enumerate properties of an object (since chicken-egg: you want the
list of properties before creating a new framebuffer, so can't use
that framebuffer to enumerate them).

But really not a big concern.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization


Re: [PATCH v3 7/7] [wip] virtio-gpu: add page flip support

2016-05-30 Thread Daniel Vetter
On Tue, May 31, 2016 at 8:18 AM, Gerd Hoffmann  wrote:
>> > https://lists.freedesktop.org/archives/dri-devel/2016-May/108772.html
>>
>> Hm, smells more like virtio isn't too happy with the default ordering of
>> the commit operation. The default is:
>>
>> - Disable any crtc/encoders that need to be disabled/change.
>> - Bash new plane setup into hw.
>> - Enable all crtcs/encoders that need to be enabled/have changed.
>>
>> There's two problems:
>> - some hw gets real grumpy if you bash in plane state without the crtc
>>   state yet matching.
>> - if you do runtime pm nothing is enabled and the writes get lost at best,
>>   or hang your interconnect at worst.
>>
>> That's why you can overwrite atomic_commit_tail, and use something more
>> sensible. See for example what it looks like for rockchip. I have a gut
>> feeling that should also take care of your troubles. Using the old crtc is
>> definitely not what you want.
>
>> Another option is that virtio isn't happy about bashing in plane state for
>> disabled crtc. Again helpers have you covered, look at the active_only
>> parameter for drm_atomic_helper_commit_planes().
>
> virtio-gpu is a bit simplified compared to real hardware, so there isn't
> really separate plane/crtc state.
>
> Right now the virtual outputs are linked to drm_crtc.  To apply any
> changes I need to lookup the crtc to figure which virtual output should
> be updated.
>
> So, setting active_only should make sure I have a valid crtc pointer on
> plane updates, right?  It probably also skips the disable + enable crtc
> steps on commit?  What happens when outputs are disabled?
>
> Maybe it makes sense to link our virtual outputs to (primary) planes not
> crtcs?

Nah, I just misunderstood your patch. If it's all about finding the
corresponding crtc, then you're all good. I thought there was some
other reason (like the virtual hw getting upset about certain things).
I'll pick up your patch into my nonblocking atomic branch to make sure
virtio isn't accidentally broken. btw can you pls drop an ack or r-b
onto my virtio conversion? I already added your tested-by.

Thanks, Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization


Re: [PATCH] Add virtio gpu driver.

2016-05-30 Thread Gerd Hoffmann
  Hi,

> > Why attach the hotspot to the plane?  Wouldn't it make more sense to
> > make it a framebuffer property?
> 
> We don't have properties on the framebuffer. I guess you /could/ just add
> it internally to struct drm_framebuffer, and not bother exposing to
> userspace. I guess that would be a lot simpler,

Yes.  I can simply stick the hotspot into drm_framebuffer in
drm_mode_cursor_universal() and pick up the values in the driver's plane
update function.

>  but it also means that
> atomic userspace can't use hotspots before we add properties to fbs. And
> doing that is a bit tricky since drm_framebuffer objects are meant to be
> invariant - this assumption is deeply in-grained into the code all over
> the place, everything just compares pointers when semantically it means to
> compare the entire fb (including backing storage pointer/offsets and
> everything).

Hmm, the hotspot location for a given cursor image is invariant too, so
I don't think that would be a big issue.

I'd expect userspace define a bunch of cursors, then switch between them
(instead of defining a single cursor, then constantly updating it).

cheers,
  Gerd

___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization


Re: [PATCH v3 7/7] [wip] virtio-gpu: add page flip support

2016-05-30 Thread Gerd Hoffmann
  Hi,

> > https://lists.freedesktop.org/archives/dri-devel/2016-May/108772.html
> 
> Hm, smells more like virtio isn't too happy with the default ordering of
> the commit operation. The default is:
> 
> - Disable any crtc/encoders that need to be disabled/change.
> - Bash new plane setup into hw.
> - Enable all crtcs/encoders that need to be enabled/have changed.
> 
> There's two problems:
> - some hw gets real grumpy if you bash in plane state without the crtc
>   state yet matching.
> - if you do runtime pm nothing is enabled and the writes get lost at best,
>   or hang your interconnect at worst.
> 
> That's why you can overwrite atomic_commit_tail, and use something more
> sensible. See for example what it looks like for rockchip. I have a gut
> feeling that should also take care of your troubles. Using the old crtc is
> definitely not what you want.

> Another option is that virtio isn't happy about bashing in plane state for
> disabled crtc. Again helpers have you covered, look at the active_only
> parameter for drm_atomic_helper_commit_planes().

virtio-gpu is a bit simplified compared to real hardware, so there isn't
really separate plane/crtc state.

Right now the virtual outputs are linked to drm_crtc.  To apply any
changes I need to lookup the crtc to figure which virtual output should
be updated.

So, setting active_only should make sure I have a valid crtc pointer on
plane updates, right?  It probably also skips the disable + enable crtc
steps on commit?  What happens when outputs are disabled?

Maybe it makes sense to link our virtual outputs to (primary) planes not
crtcs?

cheers,
  Gerd

___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization


Re: [PATCH V2 2/2] vhost_net: conditionally enable tx polling

2016-05-30 Thread Jason Wang



On 2016年05月30日 23:55, Michael S. Tsirkin wrote:

On Mon, May 30, 2016 at 02:47:54AM -0400, Jason Wang wrote:

We always poll tx for socket, this is sub optimal since:

- it will be only used when we exceed the sndbuf of the socket.
- since we use two independent polls for tx and vq, this will slightly
   increase the waitqueue traversing time and more important, vhost
   could not benefit from commit
   9e641bdcfa4ef4d6e2fbaa59c1be0ad5d1551fd5 ("net-tun: restructure
   tun_do_read for better sleep/wakeup efficiency") even if we've
   stopped rx polling during handle_rx since tx poll were still left in
   the waitqueue.

Why is this an issue?
sock_def_write_space only wakes up when queue is half empty,
not on each packet.
 if ((atomic_read(&sk->sk_wmem_alloc) << 1) <= sk->sk_sndbuf)

I suspect the issue is with your previous patch,
it now pokes at the spinlock on data path
where it used not to.

Is that right?


The problem is not tx wake up but still rx wake up. Patch 1 removes rx 
poll, but still left tx poll. So in sock_def_readable(), 
skwq_has_sleeper() returns true, we still need to traverse waitqueue and 
touch spinlocks. With this patch, unless a heavy tx load, tx poll were 
disabled, sock_def_readable() can return finish very soon.






Fix this by conditionally enable tx polling only when -EAGAIN were
met.

Test shows about 8% improvement on guest rx pps.

Before: ~135
After:  ~146

Signed-off-by: Jason Wang 
---
  drivers/vhost/net.c | 3 +++
  1 file changed, 3 insertions(+)

diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
index e91603b..5a05fa0 100644
--- a/drivers/vhost/net.c
+++ b/drivers/vhost/net.c
@@ -378,6 +378,7 @@ static void handle_tx(struct vhost_net *net)
goto out;
  
  	vhost_disable_notify(&net->dev, vq);

+   vhost_net_disable_vq(net, vq);
  
  	hdr_size = nvq->vhost_hlen;

zcopy = nvq->ubufs;
@@ -459,6 +460,8 @@ static void handle_tx(struct vhost_net *net)
% UIO_MAXIOV;
}
vhost_discard_vq_desc(vq, 1);
+   if (err == -EAGAIN)
+   vhost_net_enable_vq(net, vq);
break;
}
if (err != len)
--
1.8.3.1


___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH V2 1/2] vhost_net: stop polling socket during rx processing

2016-05-30 Thread Jason Wang



On 2016年05月30日 23:47, Michael S. Tsirkin wrote:

On Mon, May 30, 2016 at 02:47:53AM -0400, Jason Wang wrote:

We don't stop rx polling socket during rx processing, this will lead
unnecessary wakeups from under layer net devices (E.g
sock_def_readable() form tun). Rx will be slowed down in this
way. This patch avoids this by stop polling socket during rx
processing. A small drawback is that this introduces some overheads in
light load case because of the extra start/stop polling, but single
netperf TCP_RR does not notice any change. In a super heavy load case,
e.g using pktgen to inject packet to guest, we get about ~8.8%
improvement on pps:

before: ~124 pkt/s
after:  ~135 pkt/s

Signed-off-by: Jason Wang 
---
  drivers/vhost/net.c | 56 +++--
  1 file changed, 29 insertions(+), 27 deletions(-)

diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
index 10ff494..e91603b 100644
--- a/drivers/vhost/net.c
+++ b/drivers/vhost/net.c
@@ -301,6 +301,32 @@ static bool vhost_can_busy_poll(struct vhost_dev *dev,
   !vhost_has_work(dev);
  }
  
+static void vhost_net_disable_vq(struct vhost_net *n,

+struct vhost_virtqueue *vq)
+{
+   struct vhost_net_virtqueue *nvq =
+   container_of(vq, struct vhost_net_virtqueue, vq);
+   struct vhost_poll *poll = n->poll + (nvq - n->vqs);
+   if (!vq->private_data)
+   return;
+   vhost_poll_stop(poll);
+}
+
+static int vhost_net_enable_vq(struct vhost_net *n,
+   struct vhost_virtqueue *vq)
+{
+   struct vhost_net_virtqueue *nvq =
+   container_of(vq, struct vhost_net_virtqueue, vq);
+   struct vhost_poll *poll = n->poll + (nvq - n->vqs);
+   struct socket *sock;
+
+   sock = vq->private_data;
+   if (!sock)
+   return 0;
+
+   return vhost_poll_start(poll, sock->file);
+}
+
  static int vhost_net_tx_get_vq_desc(struct vhost_net *net,
struct vhost_virtqueue *vq,
struct iovec iov[], unsigned int iov_size,

BTW we might want to rename these functions, name no longer
reflects function ...


Do you mean adding something reflect busy polling in the name? Then the 
name may be too long or have suggestion on the name?






@@ -627,6 +653,7 @@ static void handle_rx(struct vhost_net *net)
if (!sock)
goto out;
vhost_disable_notify(&net->dev, vq);
+   vhost_net_disable_vq(net, vq);
  
  	vhost_hlen = nvq->vhost_hlen;

sock_hlen = nvq->sock_hlen;
@@ -715,9 +742,10 @@ static void handle_rx(struct vhost_net *net)
total_len += vhost_len;
if (unlikely(total_len >= VHOST_NET_WEIGHT)) {
vhost_poll_queue(&vq->poll);
-   break;
+   goto out;
}
}
+   vhost_net_enable_vq(net, vq);

OK so if sock is readable but RX VQ is empty, this will
immediately schedule another round of handle_rx and so ad
infinitum,

Looks like a bug.


Yes it is, will change the above headcount check to:

/* OK, now we need to know about added descriptors. */
if (!headcount) {
if (unlikely(vhost_enable_notify(&net->dev, vq))) {
/* They have slipped one in as we were
 * doing that: check again. */
vhost_disable_notify(&net->dev, vq);
continue;
}
/* Nothing new?  Wait for eventfd to tell us
 * they refilled. */
goto out;
}






  out:
mutex_unlock(&vq->mutex);
  }
@@ -796,32 +824,6 @@ static int vhost_net_open(struct inode *inode, struct file 
*f)
return 0;
  }
  
-static void vhost_net_disable_vq(struct vhost_net *n,

-struct vhost_virtqueue *vq)
-{
-   struct vhost_net_virtqueue *nvq =
-   container_of(vq, struct vhost_net_virtqueue, vq);
-   struct vhost_poll *poll = n->poll + (nvq - n->vqs);
-   if (!vq->private_data)
-   return;
-   vhost_poll_stop(poll);
-}
-
-static int vhost_net_enable_vq(struct vhost_net *n,
-   struct vhost_virtqueue *vq)
-{
-   struct vhost_net_virtqueue *nvq =
-   container_of(vq, struct vhost_net_virtqueue, vq);
-   struct vhost_poll *poll = n->poll + (nvq - n->vqs);
-   struct socket *sock;
-
-   sock = vq->private_data;
-   if (!sock)
-   return 0;
-
-   return vhost_poll_start(poll, sock->file);
-}
-
  static struct socket *vhost_net_stop_vq(struct vhost_net *n,
struct vhost_virtqueue *vq)
  {
--
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to maj

[PATCH v6v3 02/12] mm: migrate: support non-lru movable page migration

2016-05-30 Thread Minchan Kim
Per Vlastimi's review comment.

Thanks for the detail review, Vlastimi!
If you have another concern, feel free to say.
After I resolve all thing, I will send v7 rebased on recent mmotm.

>From b14aab2d3ac0702c7b2eec36409d74406d43 Mon Sep 17 00:00:00 2001
From: Minchan Kim 
Date: Fri, 8 Apr 2016 10:34:49 +0900
Subject: [PATCH] mm: migrate: support non-lru movable page migration

We have allowed migration for only LRU pages until now and it was
enough to make high-order pages. But recently, embedded system(e.g.,
webOS, android) uses lots of non-movable pages(e.g., zram, GPU memory)
so we have seen several reports about troubles of small high-order
allocation. For fixing the problem, there were several efforts
(e,g,. enhance compaction algorithm, SLUB fallback to 0-order page,
reserved memory, vmalloc and so on) but if there are lots of
non-movable pages in system, their solutions are void in the long run.

So, this patch is to support facility to change non-movable pages
with movable. For the feature, this patch introduces functions related
to migration to address_space_operations as well as some page flags.

If a driver want to make own pages movable, it should define three functions
which are function pointers of struct address_space_operations.

1. bool (*isolate_page) (struct page *page, isolate_mode_t mode);

What VM expects on isolate_page function of driver is to return *true*
if driver isolates page successfully. On returing true, VM marks the page
as PG_isolated so concurrent isolation in several CPUs skip the page
for isolation. If a driver cannot isolate the page, it should return *false*.

Once page is successfully isolated, VM uses page.lru fields so driver
shouldn't expect to preserve values in that fields.

2. int (*migratepage) (struct address_space *mapping,
struct page *newpage, struct page *oldpage, enum migrate_mode);

After isolation, VM calls migratepage of driver with isolated page.
The function of migratepage is to move content of the old page to new page
and set up fields of struct page newpage. Keep in mind that you should
indicate to the VM the oldpage is no longer movable via __ClearPageMovable()
under page_lock if you migrated the oldpage successfully and returns 0.
If driver cannot migrate the page at the moment, driver can return -EAGAIN.
On -EAGAIN, VM will retry page migration in a short time because VM interprets
-EAGAIN as "temporal migration failure". On returning any error except -EAGAIN,
VM will give up the page migration without retrying in this time.

Driver shouldn't touch page.lru field VM using in the functions.

3. void (*putback_page)(struct page *);

If migration fails on isolated page, VM should return the isolated page
to the driver so VM calls driver's putback_page with migration failed page.
In this function, driver should put the isolated page back to the own data
structure.

4. non-lru movable page flags

There are two page flags for supporting non-lru movable page.

* PG_movable

Driver should use the below function to make page movable under page_lock.

void __SetPageMovable(struct page *page, struct address_space *mapping)

It needs argument of address_space for registering migration family functions
which will be called by VM. Exactly speaking, PG_movable is not a real flag of
struct page. Rather than, VM reuses page->mapping's lower bits to represent it.

#define PAGE_MAPPING_MOVABLE 0x2
page->mapping = page->mapping | PAGE_MAPPING_MOVABLE;

so driver shouldn't access page->mapping directly. Instead, driver should
use page_mapping which mask off the low two bits of page->mapping so it can get
right struct address_space.

For testing of non-lru movable page, VM supports __PageMovable function.
However, it doesn't guarantee to identify non-lru movable page because
page->mapping field is unified with other variables in struct page.
As well, if driver releases the page after isolation by VM, page->mapping
doesn't have stable value although it has PAGE_MAPPING_MOVABLE
(Look at __ClearPageMovable). But __PageMovable is cheap to catch whether
page is LRU or non-lru movable once the page has been isolated. Because
LRU pages never can have PAGE_MAPPING_MOVABLE in page->mapping. It is also
good for just peeking to test non-lru movable pages before more expensive
checking with lock_page in pfn scanning to select victim.

For guaranteeing non-lru movable page, VM provides PageMovable function.
Unlike __PageMovable, PageMovable functions validates page->mapping and
mapping->a_ops->isolate_page under lock_page. The lock_page prevents sudden
destroying of page->mapping.

Driver using __SetPageMovable should clear the flag via __ClearMovablePage
under page_lock before the releasing the page.

* PG_isolated

To prevent concurrent isolation among several CPUs, VM marks isolated page
as PG_isolated under lock_page. So if a CPU encounters PG_isolated non-lru
movable page, it can skip it. Driver doesn't need to manipulate the fla

Re: PATCH v6v2 02/12] mm: migrate: support non-lru movable page migration

2016-05-30 Thread Minchan Kim
On Mon, May 30, 2016 at 11:36:07AM +0200, Vlastimil Babka wrote:
> On 05/30/2016 03:39 AM, Minchan Kim wrote:
> >After isolation, VM calls migratepage of driver with isolated page.
> >The function of migratepage is to move content of the old page to new page
> >and set up fields of struct page newpage. Keep in mind that you should
> >clear PG_movable of oldpage via __ClearPageMovable under page_lock if you
> >migrated the oldpage successfully and returns 0.
> 
> This "clear PG_movable" is one of the reasons I was confused about
> what __ClearPageMovable() really does. There's no actual
> "PG_movable" page flag and the function doesn't clear even the
> actual mapping flag :) Also same thing in the Documentation/ part.
> 
> Something like "... you should indicate to the VM that the oldpage
> is no longer movable via __ClearPageMovable() ..."?

It's better. I will change it.

> 
> >--- a/mm/compaction.c
> >+++ b/mm/compaction.c
> >@@ -81,6 +81,39 @@ static inline bool migrate_async_suitable(int migratetype)
> >
> > #ifdef CONFIG_COMPACTION
> >
> >+int PageMovable(struct page *page)
> >+{
> >+struct address_space *mapping;
> >+
> >+VM_BUG_ON_PAGE(!PageLocked(page), page);
> >+if (!__PageMovable(page))
> >+return 0;
> >+
> >+mapping = page_mapping(page);
> >+if (mapping && mapping->a_ops && mapping->a_ops->isolate_page)
> >+return 1;
> >+
> >+return 0;
> >+}
> >+EXPORT_SYMBOL(PageMovable);
> >+
> >+void __SetPageMovable(struct page *page, struct address_space *mapping)
> >+{
> >+VM_BUG_ON_PAGE(!PageLocked(page), page);
> >+VM_BUG_ON_PAGE((unsigned long)mapping & PAGE_MAPPING_MOVABLE, page);
> >+page->mapping = (void *)((unsigned long)mapping | PAGE_MAPPING_MOVABLE);
> >+}
> >+EXPORT_SYMBOL(__SetPageMovable);
> >+
> >+void __ClearPageMovable(struct page *page)
> >+{
> >+VM_BUG_ON_PAGE(!PageLocked(page), page);
> >+VM_BUG_ON_PAGE(!PageMovable(page), page);
> >+page->mapping = (void *)((unsigned long)page->mapping &
> >+PAGE_MAPPING_MOVABLE);
> >+}
> >+EXPORT_SYMBOL(__ClearPageMovable);
> 
> The second confusing thing is that the function is named
> __ClearPageMovable(), but what it really clears is the mapping
> pointer,
> which is not at all the opposite of what __SetPageMovable() does.
> 
> I know it's explained in the documentation, but it also deserves a
> comment here so it doesn't confuse everyone who looks at it.
> Even better would be a less confusing name for the function, but I
> can't offer one right now.

To me, __ClearPageMovable naming is suitable for user POV.
It effectively makes the page unmovable. The confusion is just caused
by the implementation and I don't prefer exported API depends on the
implementation. So I want to add just comment.

I didn't add comment above the function because I don't want to export
internal implementation to the user. I think they don't need to know it.

index a7df2ae71f2a..d1d2063b4fd9 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -108,6 +108,11 @@ void __ClearPageMovable(struct page *page)
 {
VM_BUG_ON_PAGE(!PageLocked(page), page);
VM_BUG_ON_PAGE(!PageMovable(page), page);
+   /*
+* Clear registered address_space val with keeping PAGE_MAPPING_MOVABLE
+* flag so that VM can catch up released page by driver after isolation.
+* With it, VM migration doesn't try to put it back.
+*/
page->mapping = (void *)((unsigned long)page->mapping &
PAGE_MAPPING_MOVABLE);

> 
> >diff --git a/mm/util.c b/mm/util.c
> >index 917e0e3d0f8e..b756ee36f7f0 100644
> >--- a/mm/util.c
> >+++ b/mm/util.c
> >@@ -399,10 +399,12 @@ struct address_space *page_mapping(struct page *page)
> > }
> >
> > mapping = page->mapping;
> 
> I'd probably use READ_ONCE() here to be safe. Not all callers are
> under page lock?

I don't understand. Yeah, all caller are not under page lock but at least,
new user of movable pages should call it under page_lock.
Yeah, I will write the rule down in document.
In this case, what kinds of problem do you see?

> 
> >-if ((unsigned long)mapping & PAGE_MAPPING_FLAGS)
> >+if ((unsigned long)mapping & PAGE_MAPPING_ANON)
> > return NULL;
> >-return mapping;
> >+
> >+return (void *)((unsigned long)mapping & ~PAGE_MAPPING_FLAGS);
> > }
> >+EXPORT_SYMBOL(page_mapping);
> >
> > /* Slow path of page_mapcount() for compound pages */
> > int __page_mapcount(struct page *page)
> >
> 
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization


Re: [PATCH V2 2/2] vhost_net: conditionally enable tx polling

2016-05-30 Thread Michael S. Tsirkin
On Mon, May 30, 2016 at 02:47:54AM -0400, Jason Wang wrote:
> We always poll tx for socket, this is sub optimal since:
> 
> - it will be only used when we exceed the sndbuf of the socket.
> - since we use two independent polls for tx and vq, this will slightly
>   increase the waitqueue traversing time and more important, vhost
>   could not benefit from commit
>   9e641bdcfa4ef4d6e2fbaa59c1be0ad5d1551fd5 ("net-tun: restructure
>   tun_do_read for better sleep/wakeup efficiency") even if we've
>   stopped rx polling during handle_rx since tx poll were still left in
>   the waitqueue.

Why is this an issue?
sock_def_write_space only wakes up when queue is half empty,
not on each packet.
if ((atomic_read(&sk->sk_wmem_alloc) << 1) <= sk->sk_sndbuf)

I suspect the issue is with your previous patch,
it now pokes at the spinlock on data path
where it used not to.

Is that right?


> 
> Fix this by conditionally enable tx polling only when -EAGAIN were
> met.
> 
> Test shows about 8% improvement on guest rx pps.
> 
> Before: ~135
> After:  ~146
> 
> Signed-off-by: Jason Wang 
> ---
>  drivers/vhost/net.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
> index e91603b..5a05fa0 100644
> --- a/drivers/vhost/net.c
> +++ b/drivers/vhost/net.c
> @@ -378,6 +378,7 @@ static void handle_tx(struct vhost_net *net)
>   goto out;
>  
>   vhost_disable_notify(&net->dev, vq);
> + vhost_net_disable_vq(net, vq);
>  
>   hdr_size = nvq->vhost_hlen;
>   zcopy = nvq->ubufs;
> @@ -459,6 +460,8 @@ static void handle_tx(struct vhost_net *net)
>   % UIO_MAXIOV;
>   }
>   vhost_discard_vq_desc(vq, 1);
> + if (err == -EAGAIN)
> + vhost_net_enable_vq(net, vq);
>   break;
>   }
>   if (err != len)
> -- 
> 1.8.3.1
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization


Re: [PATCH V2 1/2] vhost_net: stop polling socket during rx processing

2016-05-30 Thread Michael S. Tsirkin
On Mon, May 30, 2016 at 02:47:53AM -0400, Jason Wang wrote:
> We don't stop rx polling socket during rx processing, this will lead
> unnecessary wakeups from under layer net devices (E.g
> sock_def_readable() form tun). Rx will be slowed down in this
> way. This patch avoids this by stop polling socket during rx
> processing. A small drawback is that this introduces some overheads in
> light load case because of the extra start/stop polling, but single
> netperf TCP_RR does not notice any change. In a super heavy load case,
> e.g using pktgen to inject packet to guest, we get about ~8.8%
> improvement on pps:
> 
> before: ~124 pkt/s
> after:  ~135 pkt/s
> 
> Signed-off-by: Jason Wang 
> ---
>  drivers/vhost/net.c | 56 
> +++--
>  1 file changed, 29 insertions(+), 27 deletions(-)
> 
> diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
> index 10ff494..e91603b 100644
> --- a/drivers/vhost/net.c
> +++ b/drivers/vhost/net.c
> @@ -301,6 +301,32 @@ static bool vhost_can_busy_poll(struct vhost_dev *dev,
>  !vhost_has_work(dev);
>  }
>  
> +static void vhost_net_disable_vq(struct vhost_net *n,
> +  struct vhost_virtqueue *vq)
> +{
> + struct vhost_net_virtqueue *nvq =
> + container_of(vq, struct vhost_net_virtqueue, vq);
> + struct vhost_poll *poll = n->poll + (nvq - n->vqs);
> + if (!vq->private_data)
> + return;
> + vhost_poll_stop(poll);
> +}
> +
> +static int vhost_net_enable_vq(struct vhost_net *n,
> + struct vhost_virtqueue *vq)
> +{
> + struct vhost_net_virtqueue *nvq =
> + container_of(vq, struct vhost_net_virtqueue, vq);
> + struct vhost_poll *poll = n->poll + (nvq - n->vqs);
> + struct socket *sock;
> +
> + sock = vq->private_data;
> + if (!sock)
> + return 0;
> +
> + return vhost_poll_start(poll, sock->file);
> +}
> +
>  static int vhost_net_tx_get_vq_desc(struct vhost_net *net,
>   struct vhost_virtqueue *vq,
>   struct iovec iov[], unsigned int iov_size,

BTW we might want to rename these functions, name no longer
reflects function ...


> @@ -627,6 +653,7 @@ static void handle_rx(struct vhost_net *net)
>   if (!sock)
>   goto out;
>   vhost_disable_notify(&net->dev, vq);
> + vhost_net_disable_vq(net, vq);
>  
>   vhost_hlen = nvq->vhost_hlen;
>   sock_hlen = nvq->sock_hlen;
> @@ -715,9 +742,10 @@ static void handle_rx(struct vhost_net *net)
>   total_len += vhost_len;
>   if (unlikely(total_len >= VHOST_NET_WEIGHT)) {
>   vhost_poll_queue(&vq->poll);
> - break;
> + goto out;
>   }
>   }
> + vhost_net_enable_vq(net, vq);

OK so if sock is readable but RX VQ is empty, this will
immediately schedule another round of handle_rx and so ad
infinitum,

Looks like a bug.


>  out:
>   mutex_unlock(&vq->mutex);
>  }
> @@ -796,32 +824,6 @@ static int vhost_net_open(struct inode *inode, struct 
> file *f)
>   return 0;
>  }
>  
> -static void vhost_net_disable_vq(struct vhost_net *n,
> -  struct vhost_virtqueue *vq)
> -{
> - struct vhost_net_virtqueue *nvq =
> - container_of(vq, struct vhost_net_virtqueue, vq);
> - struct vhost_poll *poll = n->poll + (nvq - n->vqs);
> - if (!vq->private_data)
> - return;
> - vhost_poll_stop(poll);
> -}
> -
> -static int vhost_net_enable_vq(struct vhost_net *n,
> - struct vhost_virtqueue *vq)
> -{
> - struct vhost_net_virtqueue *nvq =
> - container_of(vq, struct vhost_net_virtqueue, vq);
> - struct vhost_poll *poll = n->poll + (nvq - n->vqs);
> - struct socket *sock;
> -
> - sock = vq->private_data;
> - if (!sock)
> - return 0;
> -
> - return vhost_poll_start(poll, sock->file);
> -}
> -
>  static struct socket *vhost_net_stop_vq(struct vhost_net *n,
>   struct vhost_virtqueue *vq)
>  {
> -- 
> 1.8.3.1
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization


Re: [PATCH] virtio-gpu: fix output lookup

2016-05-30 Thread Daniel Vetter
On Mon, May 30, 2016 at 02:03:26PM +0200, Gerd Hoffmann wrote:
> Needed for multihead setups where we can have disabled
> outputs and therefore plane->crtc can be NULL.
> 
> Signed-off-by: Gerd Hoffmann 

See my reply in the other thread, but I think you have a more fundamental
issue here. I dropped two suggestions worth trying, both need your own
atomic_commit_tail (on top of my branch at least):
- set active_only to true in commit_planes()
- move commit_planes after commit_modeset_enables().

Cheers, Daniel
> ---
>  drivers/gpu/drm/virtio/virtgpu_plane.c | 8 +++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/gpu/drm/virtio/virtgpu_plane.c 
> b/drivers/gpu/drm/virtio/virtgpu_plane.c
> index 70b44a2..e50674b 100644
> --- a/drivers/gpu/drm/virtio/virtgpu_plane.c
> +++ b/drivers/gpu/drm/virtio/virtgpu_plane.c
> @@ -63,11 +63,17 @@ static void virtio_gpu_plane_atomic_update(struct 
> drm_plane *plane,
>  {
>   struct drm_device *dev = plane->dev;
>   struct virtio_gpu_device *vgdev = dev->dev_private;
> - struct virtio_gpu_output *output = 
> drm_crtc_to_virtio_gpu_output(plane->crtc);
> + struct virtio_gpu_output *output = NULL;
>   struct virtio_gpu_framebuffer *vgfb;
>   struct virtio_gpu_object *bo;
>   uint32_t handle;
>  
> + if (plane->state->crtc)
> + output = drm_crtc_to_virtio_gpu_output(plane->state->crtc);
> + if (old_state->crtc)
> + output = drm_crtc_to_virtio_gpu_output(old_state->crtc);
> + WARN_ON(!output);
> +
>   if (plane->state->fb) {
>   vgfb = to_virtio_gpu_framebuffer(plane->state->fb);
>   bo = gem_to_virtio_gpu_obj(vgfb->obj);
> -- 
> 1.8.3.1
> 
> ___
> dri-devel mailing list
> dri-de...@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel

-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization


Re: [PATCH] Add virtio gpu driver.

2016-05-30 Thread Daniel Vetter
On Mon, May 30, 2016 at 03:50:36PM +0200, Gerd Hoffmann wrote:
>   Hi,
> 
> > - add a small core function to registerr HOT_X/HOT_Y for a (cursor) plane,
> >   e.g. drm_plane_register_hotspot(). That should allocate the properties
> >   (if they don't exist yet) and then attach those props to the cursor. We
> >   don't want those props everywhere, but only on drivers that support/need
> >   them, aka virtual hw.
> 
> Hmm, why is this special to virtual hw?
> 
> > if (crtc->cursor) {
> > -   ret = drm_mode_cursor_universal(crtc, req, file_priv);
> > +   if (drm_core_check_feature(DRIVER_ATOMIC))
> > +   ret = drm_mode_cursor_atomic(crtc, req, file_priv);
> > +   else
> > +   ret = drm_mode_cursor_universal(crtc, req, file_priv);
> > goto out;
> 
> >   drm_mode_cursor_atomic would simply be a fusing of
> >   drm_mode_cursor_universal + drm_atomic_helper_update_plane (dump all the
> >   intermediate variables and store directly in the plane state), with the
> >   addition of also storing hot_x/y into the plane state.
> 
> Hmm, that'll either make drm_mode_cursor_atomic a big cut+pasted
> function, or need quite some refactoring to move common code into
> functions callable from both drm_mode_cursor_atomic
> +drm_mode_cursor_universal ...
> 
> Why attach the hotspot to the plane?  Wouldn't it make more sense to
> make it a framebuffer property?

We don't have properties on the framebuffer. I guess you /could/ just add
it internally to struct drm_framebuffer, and not bother exposing to
userspace. I guess that would be a lot simpler, but it also means that
atomic userspace can't use hotspots before we add properties to fbs. And
doing that is a bit tricky since drm_framebuffer objects are meant to be
invariant - this assumption is deeply in-grained into the code all over
the place, everything just compares pointers when semantically it means to
compare the entire fb (including backing storage pointer/offsets and
everything).

So would be a bit more work to wire up for atomic userspace, but indeed a
lot less work to implement. I'm totally happy if you go with that tradeoff
;-)

Cheers, Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization


Re: [PATCH v3 7/7] [wip] virtio-gpu: add page flip support

2016-05-30 Thread Daniel Vetter
On Mon, May 30, 2016 at 02:06:50PM +0200, Gerd Hoffmann wrote:
>   Hi,
> 
> > > But I'll take you up on the implied offer to help out and test ;-)
> > 
> > git://people.freedesktop.org/~danvet/drm stuff
> 
> Tried that branch.
> 
> > Would be really awesome if you could test this on virtio. Note that the
> > new nonblocking helpers require that your atomic backend gets the drm
> > event handling right. So if there's a bug in that logic then you'll see
> > lots of dmesg noise about waits timing out (after 10s of waiting). From a
> > quick inspection it should work though.
> 
> No timeouts.  Yay!
> 
> But it seems crtcs can be (temporarely) disabled now, so we might have
> to pick up the crtc from old_state in virtio_gpu_plane_atomic_update to
> figure which virtual output needs to be turned off.  Ran into this last
> week already.  Happened with multihead setups only, but the same patch
> fixes this one too ;)
> 
> https://lists.freedesktop.org/archives/dri-devel/2016-May/108772.html

Hm, smells more like virtio isn't too happy with the default ordering of
the commit operation. The default is:

- Disable any crtc/encoders that need to be disabled/change.
- Bash new plane setup into hw.
- Enable all crtcs/encoders that need to be enabled/have changed.

There's two problems:
- some hw gets real grumpy if you bash in plane state without the crtc
  state yet matching.
- if you do runtime pm nothing is enabled and the writes get lost at best,
  or hang your interconnect at worst.

That's why you can overwrite atomic_commit_tail, and use something more
sensible. See for example what it looks like for rockchip. I have a gut
feeling that should also take care of your troubles. Using the old crtc is
definitely not what you want.

Another option is that virtio isn't happy about bashing in plane state for
disabled crtc. Again helpers have you covered, look at the active_only
parameter for drm_atomic_helper_commit_planes().

Aside, if you wonder why these defaults: They match what the crtc helpers
are doing, but they are a bit surprising indeed.

Cheers, Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization


Re: [PATCH] Add virtio gpu driver.

2016-05-30 Thread Gerd Hoffmann
  Hi,

> - add a small core function to registerr HOT_X/HOT_Y for a (cursor) plane,
>   e.g. drm_plane_register_hotspot(). That should allocate the properties
>   (if they don't exist yet) and then attach those props to the cursor. We
>   don't want those props everywhere, but only on drivers that support/need
>   them, aka virtual hw.

Hmm, why is this special to virtual hw?

>   if (crtc->cursor) {
> - ret = drm_mode_cursor_universal(crtc, req, file_priv);
> + if (drm_core_check_feature(DRIVER_ATOMIC))
> + ret = drm_mode_cursor_atomic(crtc, req, file_priv);
> + else
> + ret = drm_mode_cursor_universal(crtc, req, file_priv);
>   goto out;

>   drm_mode_cursor_atomic would simply be a fusing of
>   drm_mode_cursor_universal + drm_atomic_helper_update_plane (dump all the
>   intermediate variables and store directly in the plane state), with the
>   addition of also storing hot_x/y into the plane state.

Hmm, that'll either make drm_mode_cursor_atomic a big cut+pasted
function, or need quite some refactoring to move common code into
functions callable from both drm_mode_cursor_atomic
+drm_mode_cursor_universal ...

Why attach the hotspot to the plane?  Wouldn't it make more sense to
make it a framebuffer property?

cheers,
  Gerd

___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization


Re: [PATCH v6 03/12] mm: balloon: use general non-lru movable page feature

2016-05-30 Thread Vlastimil Babka

On 05/20/2016 04:23 PM, Minchan Kim wrote:

Now, VM has a feature to migrate non-lru movable pages so
balloon doesn't need custom migration hooks in migrate.c
and compaction.c. Instead, this patch implements
page->mapping->a_ops->{isolate|migrate|putback} functions.

With that, we could remove hooks for ballooning in general
migration functions and make balloon compaction simple.

Cc: virtualization@lists.linux-foundation.org
Cc: Rafael Aquini 
Cc: Konstantin Khlebnikov 
Signed-off-by: Gioh Kim 
Signed-off-by: Minchan Kim 


Except for the inode/pseudofs stuff which I'm not familiar with,

Acked-by: Vlastimil Babka 

___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization


Re: [PATCH v3 7/7] [wip] virtio-gpu: add page flip support

2016-05-30 Thread Gerd Hoffmann
  Hi,

> > But I'll take you up on the implied offer to help out and test ;-)
> 
> git://people.freedesktop.org/~danvet/drm stuff

Tried that branch.

> Would be really awesome if you could test this on virtio. Note that the
> new nonblocking helpers require that your atomic backend gets the drm
> event handling right. So if there's a bug in that logic then you'll see
> lots of dmesg noise about waits timing out (after 10s of waiting). From a
> quick inspection it should work though.

No timeouts.  Yay!

But it seems crtcs can be (temporarely) disabled now, so we might have
to pick up the crtc from old_state in virtio_gpu_plane_atomic_update to
figure which virtual output needs to be turned off.  Ran into this last
week already.  Happened with multihead setups only, but the same patch
fixes this one too ;)

https://lists.freedesktop.org/archives/dri-devel/2016-May/108772.html

cheers,
  Gerd

___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization


[PATCH] virtio-gpu: fix output lookup

2016-05-30 Thread Gerd Hoffmann
Needed for multihead setups where we can have disabled
outputs and therefore plane->crtc can be NULL.

Signed-off-by: Gerd Hoffmann 
---
 drivers/gpu/drm/virtio/virtgpu_plane.c | 8 +++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/virtio/virtgpu_plane.c 
b/drivers/gpu/drm/virtio/virtgpu_plane.c
index 70b44a2..e50674b 100644
--- a/drivers/gpu/drm/virtio/virtgpu_plane.c
+++ b/drivers/gpu/drm/virtio/virtgpu_plane.c
@@ -63,11 +63,17 @@ static void virtio_gpu_plane_atomic_update(struct drm_plane 
*plane,
 {
struct drm_device *dev = plane->dev;
struct virtio_gpu_device *vgdev = dev->dev_private;
-   struct virtio_gpu_output *output = 
drm_crtc_to_virtio_gpu_output(plane->crtc);
+   struct virtio_gpu_output *output = NULL;
struct virtio_gpu_framebuffer *vgfb;
struct virtio_gpu_object *bo;
uint32_t handle;
 
+   if (plane->state->crtc)
+   output = drm_crtc_to_virtio_gpu_output(plane->state->crtc);
+   if (old_state->crtc)
+   output = drm_crtc_to_virtio_gpu_output(old_state->crtc);
+   WARN_ON(!output);
+
if (plane->state->fb) {
vgfb = to_virtio_gpu_framebuffer(plane->state->fb);
bo = gem_to_virtio_gpu_obj(vgfb->obj);
-- 
1.8.3.1

___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization


Re: PATCH v6v2 02/12] mm: migrate: support non-lru movable page migration

2016-05-30 Thread Vlastimil Babka

On 05/30/2016 03:39 AM, Minchan Kim wrote:

After isolation, VM calls migratepage of driver with isolated page.
The function of migratepage is to move content of the old page to new page
and set up fields of struct page newpage. Keep in mind that you should
clear PG_movable of oldpage via __ClearPageMovable under page_lock if you
migrated the oldpage successfully and returns 0.


This "clear PG_movable" is one of the reasons I was confused about what 
__ClearPageMovable() really does. There's no actual "PG_movable" page 
flag and the function doesn't clear even the actual mapping flag :) Also 
same thing in the Documentation/ part.


Something like "... you should indicate to the VM that the oldpage is no 
longer movable via __ClearPageMovable() ..."?



--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -81,6 +81,39 @@ static inline bool migrate_async_suitable(int migratetype)

 #ifdef CONFIG_COMPACTION

+int PageMovable(struct page *page)
+{
+   struct address_space *mapping;
+
+   VM_BUG_ON_PAGE(!PageLocked(page), page);
+   if (!__PageMovable(page))
+   return 0;
+
+   mapping = page_mapping(page);
+   if (mapping && mapping->a_ops && mapping->a_ops->isolate_page)
+   return 1;
+
+   return 0;
+}
+EXPORT_SYMBOL(PageMovable);
+
+void __SetPageMovable(struct page *page, struct address_space *mapping)
+{
+   VM_BUG_ON_PAGE(!PageLocked(page), page);
+   VM_BUG_ON_PAGE((unsigned long)mapping & PAGE_MAPPING_MOVABLE, page);
+   page->mapping = (void *)((unsigned long)mapping | PAGE_MAPPING_MOVABLE);
+}
+EXPORT_SYMBOL(__SetPageMovable);
+
+void __ClearPageMovable(struct page *page)
+{
+   VM_BUG_ON_PAGE(!PageLocked(page), page);
+   VM_BUG_ON_PAGE(!PageMovable(page), page);
+   page->mapping = (void *)((unsigned long)page->mapping &
+   PAGE_MAPPING_MOVABLE);
+}
+EXPORT_SYMBOL(__ClearPageMovable);


The second confusing thing is that the function is named 
__ClearPageMovable(), but what it really clears is the mapping pointer,

which is not at all the opposite of what __SetPageMovable() does.

I know it's explained in the documentation, but it also deserves a 
comment here so it doesn't confuse everyone who looks at it.
Even better would be a less confusing name for the function, but I can't 
offer one right now.



diff --git a/mm/util.c b/mm/util.c
index 917e0e3d0f8e..b756ee36f7f0 100644
--- a/mm/util.c
+++ b/mm/util.c
@@ -399,10 +399,12 @@ struct address_space *page_mapping(struct page *page)
}

mapping = page->mapping;


I'd probably use READ_ONCE() here to be safe. Not all callers are under 
page lock?



-   if ((unsigned long)mapping & PAGE_MAPPING_FLAGS)
+   if ((unsigned long)mapping & PAGE_MAPPING_ANON)
return NULL;
-   return mapping;
+
+   return (void *)((unsigned long)mapping & ~PAGE_MAPPING_FLAGS);
 }
+EXPORT_SYMBOL(page_mapping);

 /* Slow path of page_mapcount() for compound pages */
 int __page_mapcount(struct page *page)



___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization


Re: [PATCH v6 02/12] mm: migrate: support non-lru movable page migration

2016-05-30 Thread Vlastimil Babka

On 05/30/2016 03:33 AM, Minchan Kim wrote:




+   page->mapping = (void *)((unsigned long)page->mapping &
+   PAGE_MAPPING_MOVABLE);


This should be negated to clear... use ~PAGE_MAPPING_MOVABLE ?


No.

The intention is to clear only mapping value but PAGE_MAPPING_MOVABLE
flag. So, any new migration trial will be failed because PageMovable
checks page's mapping value but ongoing migraion handling can catch
whether it's movable page or not with the type bit.


Oh, OK, I got that wrong. I'll point out in the reply to the v6v2 what 
misled me :)




So this effectively prevents movable compound pages from being
migrated. Are you sure no users of this functionality are going to
have compound pages? I assumed that they could, and so made the code
like this, with the is_lru variable (which is redundant after your
change).


This implementation at the moment disables effectively non-lru compound
page migration but I'm not a god so I can't make sure no one doesn't want
it in future. If someone want it, we can support it then because this work
doesn't prevent it by design.


Oh well. As long as the balloon pages or zsmalloc don't already use 
compound pages...




I thouht PageCompound check right before isolate_movable_page in
isolate_migratepages_block will filter it out mostly but yeah
it is racy without zone->lru_lock so it could reach to isolate_movable_page.
However, PageMovable check in there investigates mapping, mapping->a_ops,
and a_ops->isolate_page to verify whether it's movable page or not.

I thought it's sufficient to filter THP page.


I guess, yeah.



[...]


@@ -755,33 +844,69 @@ static int move_to_new_page(struct page *newpage, struct 
page *page,
enum migrate_mode mode)
{
struct address_space *mapping;
-   int rc;
+   int rc = -EAGAIN;
+   bool is_lru = !__PageMovable(page);

VM_BUG_ON_PAGE(!PageLocked(page), page);
VM_BUG_ON_PAGE(!PageLocked(newpage), newpage);

mapping = page_mapping(page);
-   if (!mapping)
-   rc = migrate_page(mapping, newpage, page, mode);
-   else if (mapping->a_ops->migratepage)
-   /*
-* Most pages have a mapping and most filesystems provide a
-* migratepage callback. Anonymous pages are part of swap
-* space which also has its own migratepage callback. This
-* is the most common path for page migration.
-*/
-   rc = mapping->a_ops->migratepage(mapping, newpage, page, mode);
-   else
-   rc = fallback_migrate_page(mapping, newpage, page, mode);
+   /*
+* In case of non-lru page, it could be released after
+* isolation step. In that case, we shouldn't try
+* fallback migration which is designed for LRU pages.
+*/


Hmm but is_lru was determined from !__PageMovable() above, also well
after the isolation step. So if the driver already released it, we
wouldn't detect it? And this function is all under same page lock,
so if __PageMovable was true above, so will be PageMovable below?


You are missing what I mentioned above.
We should keep the type bit to catch what you are saying(i.e., driver
already released).

__PageMovable just checks PAGE_MAPPING_MOVABLE flag and PageMovable
checks page->mapping valid while __ClearPageMovable reset only
valid vaule of mapping, not PAGE_MAPPING_MOVABLE flag.

I wrote it down in Documentation/vm/page_migration.

"For testing of non-lru movable page, VM supports __PageMovable function.
However, it doesn't guarantee to identify non-lru movable page because
page->mapping field is unified with other variables in struct page.
As well, if driver releases the page after isolation by VM, page->mapping
doesn't have stable value although it has PAGE_MAPPING_MOVABLE
(Look at __ClearPageMovable). But __PageMovable is cheap to catch whether
page is LRU or non-lru movable once the page has been isolated. Because
LRU pages never can have PAGE_MAPPING_MOVABLE in page->mapping. It is also
good for just peeking to test non-lru movable pages before more expensive
checking with lock_page in pfn scanning to select victim.

For guaranteeing non-lru movable page, VM provides PageMovable function.
Unlike __PageMovable, PageMovable functions validates page->mapping and
mapping->a_ops->isolate_page under lock_page. The lock_page prevents sudden
destroying of page->mapping.

Driver using __SetPageMovable should clear the flag via __ClearMovablePage
under page_lock before the releasing the page."


Right, I get it now.



+   if (!((unsigned long)page->mapping & PAGE_MAPPING_FLAGS))
page->mapping = NULL;


The two lines above make little sense to me without a comment.


I folded this.

@@ -901,7 +901,12 @@ static int move_to_new_page(struct page *newpage, struct 
page *page,
__ClearPageIsolated(page);
}

-   

Re: [PATCH v3 7/7] [wip] virtio-gpu: add page flip support

2016-05-30 Thread Daniel Vetter
On Fri, May 27, 2016 at 09:50:27AM +0200, Daniel Vetter wrote:
> On Fri, May 27, 2016 at 09:46:03AM +0200, Gerd Hoffmann wrote:
> > On Mi, 2016-05-25 at 18:37 +0200, Daniel Vetter wrote:
> > > On Fri, Oct 2, 2015 at 1:58 PM, Gerd Hoffmann  wrote:
> > > > Signed-off-by: Gerd Hoffmann 
> > > 
> > > So I entirely missed this, but this isn't really how to implement
> > > page_flip for an atomic driver. Working on some stuff and will hack up
> > > a likely totally broken patch, but should be enough as guideline.
> > > -Daniel
> > 
> > Hmm, no patch in my inbox yet.  Care to send it over or give some hints
> > what is wrong with this?
> 
> You should use drm_atomic_helper_page_flip as .page_flip hook instead of
> rolling your own, when you have a brand-new atomic driver. If that helper
> doesn't work that means you have a bug in your driver somewhere.
> 
> And indeed virtio just plugs in drm_atomic_helper_commit, which thus far
> doesnt' do nonblocking commits, which means the page flip stuff won't
> work. I'm working on generic nonblocking atomic to fix this up. Currently
> still wip and buggy though, hence no patches.
> 
> But I'll take you up on the implied offer to help out and test ;-)

Ok, patches are now on the list and also in my fdo repo at:

https://cgit.freedesktop.org/~danvet/drm/log/

git://people.freedesktop.org/~danvet/drm stuff

Would be really awesome if you could test this on virtio. Note that the
new nonblocking helpers require that your atomic backend gets the drm
event handling right. So if there's a bug in that logic then you'll see
lots of dmesg noise about waits timing out (after 10s of waiting). From a
quick inspection it should work though.

Upshot is that once you get normal modeset to work with those new helpers,
then nonblocking should Just Work, and there's no need anymore for
additional testing. Or at least shouldn't.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization