Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-27 Thread Jinhao Fan
After digging through the source code, I found event_notifier_cleanup() only closes the eventfd, but does not de-register the event from QEMU’s event loop. event_notifier_set_handler() actually interacts with the event loop. It is a wrapper around aio_set_event_notifier(), which is again a wrapper

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-27 Thread Klaus Jensen
On Jul 27 16:16, Jinhao Fan wrote: > at 3:06 PM, Klaus Jensen wrote: > > > On Jul 26 14:08, Klaus Jensen wrote: > >> Alright. Forget about the iommu, that was just a coincidence. > >> > >> This patch seems to fix it. I guess it is the > >> event_notifier_set_handler(..., NULL) that does the tric

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-27 Thread Jinhao Fan
at 3:06 PM, Klaus Jensen wrote: > On Jul 26 14:08, Klaus Jensen wrote: >> Alright. Forget about the iommu, that was just a coincidence. >> >> This patch seems to fix it. I guess it is the >> event_notifier_set_handler(..., NULL) that does the trick, but I'd like >> to understand why ;) >> >> >

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-27 Thread Klaus Jensen
On Jul 26 14:08, Klaus Jensen wrote: > > Alright. Forget about the iommu, that was just a coincidence. > > This patch seems to fix it. I guess it is the > event_notifier_set_handler(..., NULL) that does the trick, but I'd like > to understand why ;) > > > diff --git i/hw/nvme/ctrl.c w/hw/nvme/c

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-26 Thread Stefan Hajnoczi
On Tue, 5 Jul 2022 at 10:25, Jinhao Fan wrote: > > Add property "ioeventfd" which is enabled by default. When this is > enabled, updates on the doorbell registers will cause KVM to signal > an event to the QEMU main loop to handle the doorbell updates. > Therefore, instead of letting the vcpu thre

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-26 Thread Klaus Jensen
On Jul 26 13:32, Klaus Jensen wrote: > On Jul 26 13:24, Klaus Jensen wrote: > > On Jul 26 12:09, Klaus Jensen wrote: > > > On Jul 26 11:19, Klaus Jensen wrote: > > > > On Jul 26 15:55, Jinhao Fan wrote: > > > > > at 3:41 PM, Klaus Jensen wrote: > > > > > > > > > > > On Jul 26 15:35, Jinhao Fan wr

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-26 Thread Klaus Jensen
On Jul 26 13:24, Klaus Jensen wrote: > On Jul 26 12:09, Klaus Jensen wrote: > > On Jul 26 11:19, Klaus Jensen wrote: > > > On Jul 26 15:55, Jinhao Fan wrote: > > > > at 3:41 PM, Klaus Jensen wrote: > > > > > > > > > On Jul 26 15:35, Jinhao Fan wrote: > > > > >> at 4:55 AM, Klaus Jensen wrote: >

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-26 Thread Klaus Jensen
On Jul 26 12:09, Klaus Jensen wrote: > On Jul 26 11:19, Klaus Jensen wrote: > > On Jul 26 15:55, Jinhao Fan wrote: > > > at 3:41 PM, Klaus Jensen wrote: > > > > > > > On Jul 26 15:35, Jinhao Fan wrote: > > > >> at 4:55 AM, Klaus Jensen wrote: > > > >> > > > >>> We have a regression following th

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-26 Thread Klaus Jensen
On Jul 26 11:19, Klaus Jensen wrote: > On Jul 26 15:55, Jinhao Fan wrote: > > at 3:41 PM, Klaus Jensen wrote: > > > > > On Jul 26 15:35, Jinhao Fan wrote: > > >> at 4:55 AM, Klaus Jensen wrote: > > >> > > >>> We have a regression following this patch that we need to address. > > >>> > > >>> Wi

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-26 Thread Klaus Jensen
On Jul 26 15:55, Jinhao Fan wrote: > at 3:41 PM, Klaus Jensen wrote: > > > On Jul 26 15:35, Jinhao Fan wrote: > >> at 4:55 AM, Klaus Jensen wrote: > >> > >>> We have a regression following this patch that we need to address. > >>> > >>> With this patch, issuing a reset on the device (`nvme res

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-26 Thread Jinhao Fan
at 3:41 PM, Klaus Jensen wrote: > On Jul 26 15:35, Jinhao Fan wrote: >> at 4:55 AM, Klaus Jensen wrote: >> >>> We have a regression following this patch that we need to address. >>> >>> With this patch, issuing a reset on the device (`nvme reset /dev/nvme0` >>> will do the trick) causes QEMU t

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-26 Thread Klaus Jensen
On Jul 26 15:35, Jinhao Fan wrote: > at 4:55 AM, Klaus Jensen wrote: > > > > > We have a regression following this patch that we need to address. > > > > With this patch, issuing a reset on the device (`nvme reset /dev/nvme0` > > will do the trick) causes QEMU to hog my host cpu at 100%. > > >

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-26 Thread Jinhao Fan
at 4:55 AM, Klaus Jensen wrote: > > We have a regression following this patch that we need to address. > > With this patch, issuing a reset on the device (`nvme reset /dev/nvme0` > will do the trick) causes QEMU to hog my host cpu at 100%. > > I'm still not sure what causes this. The trace out

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-25 Thread Klaus Jensen
On Jul 5 22:24, Jinhao Fan wrote: > Add property "ioeventfd" which is enabled by default. When this is > enabled, updates on the doorbell registers will cause KVM to signal > an event to the QEMU main loop to handle the doorbell updates. > Therefore, instead of letting the vcpu thread run both gue

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-13 Thread Klaus Jensen
On Jul 12 14:23, Klaus Jensen wrote: > On Jul 9 11:06, Jinhao Fan wrote: > > at 10:24 PM, Jinhao Fan wrote: > > > > > @@ -5793,6 +5891,7 @@ static uint16_t nvme_dbbuf_config(NvmeCtrl *n, > > > const NvmeRequest *req) > > > uint64_t dbs_addr = le64_to_cpu(req->cmd.dptr.prp1); > > > uint6

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-12 Thread Klaus Jensen
On Jul 9 11:06, Jinhao Fan wrote: > at 10:24 PM, Jinhao Fan wrote: > > > @@ -5793,6 +5891,7 @@ static uint16_t nvme_dbbuf_config(NvmeCtrl *n, const > > NvmeRequest *req) > > uint64_t dbs_addr = le64_to_cpu(req->cmd.dptr.prp1); > > uint64_t eis_addr = le64_to_cpu(req->cmd.dptr.prp2); > >

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-08 Thread Jinhao Fan
at 10:24 PM, Jinhao Fan wrote: > @@ -5793,6 +5891,7 @@ static uint16_t nvme_dbbuf_config(NvmeCtrl *n, const > NvmeRequest *req) > uint64_t dbs_addr = le64_to_cpu(req->cmd.dptr.prp1); > uint64_t eis_addr = le64_to_cpu(req->cmd.dptr.prp2); > int i; > +int ret; > I just noticed th

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-07 Thread Jinhao Fan
at 1:51 PM, Klaus Jensen wrote: > On Jul 6 19:34, Jinhao Fan wrote: >> at 2:43 AM, Keith Busch wrote: >> >>> On Tue, Jul 05, 2022 at 07:11:36PM +0200, Klaus Jensen wrote: On Jul 5 22:24, Jinhao Fan wrote: > @@ -1374,7 +1374,14 @@ static void nvme_enqueue_req_completion(NvmeCQueue >>

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-06 Thread Klaus Jensen
On Jul 6 19:34, Jinhao Fan wrote: > at 2:43 AM, Keith Busch wrote: > > > On Tue, Jul 05, 2022 at 07:11:36PM +0200, Klaus Jensen wrote: > >> On Jul 5 22:24, Jinhao Fan wrote: > >>> @@ -1374,7 +1374,14 @@ static void nvme_enqueue_req_completion(NvmeCQueue > >>> *cq, NvmeRequest *req) > >>> > >>

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-06 Thread Jinhao Fan
at 2:43 AM, Keith Busch wrote: > On Tue, Jul 05, 2022 at 07:11:36PM +0200, Klaus Jensen wrote: >> On Jul 5 22:24, Jinhao Fan wrote: >>> @@ -1374,7 +1374,14 @@ static void nvme_enqueue_req_completion(NvmeCQueue >>> *cq, NvmeRequest *req) >>> >>> QTAILQ_REMOVE(&req->sq->out_req_list, req, en

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-06 Thread Jinhao Fan
at 1:11 AM, Klaus Jensen wrote: > On Jul 5 22:24, Jinhao Fan wrote: >> Add property "ioeventfd" which is enabled by default. When this is >> enabled, updates on the doorbell registers will cause KVM to signal >> an event to the QEMU main loop to handle the doorbell updates. >> Therefore, instead

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-05 Thread Keith Busch
On Tue, Jul 05, 2022 at 07:11:36PM +0200, Klaus Jensen wrote: > On Jul 5 22:24, Jinhao Fan wrote: > > @@ -1374,7 +1374,14 @@ static void nvme_enqueue_req_completion(NvmeCQueue > > *cq, NvmeRequest *req) > > > > QTAILQ_REMOVE(&req->sq->out_req_list, req, entry); > > QTAILQ_INSERT_TAIL(

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-05 Thread Klaus Jensen
On Jul 5 22:24, Jinhao Fan wrote: > Add property "ioeventfd" which is enabled by default. When this is > enabled, updates on the doorbell registers will cause KVM to signal > an event to the QEMU main loop to handle the doorbell updates. > Therefore, instead of letting the vcpu thread run both gue