Re: [PATCH] hw/nvme: Add helper functions for qid-db conversion
On Wed, Aug 03, 2022 at 09:46:05AM +0800, Jinhao Fan wrote: > at 4:54 PM, Klaus Jensen wrote: > > > I am unsure if the compiler will transform that division into the shift > > if it can infer that the divisor is a power of two (it most likely > > will be able to). > > > > But I see no reason to have a potential division here when we can do > > without and to me it is just as readable when you know the definition of > > DSTRD is `2 ^ (2 + DSTRD)`. > > OK. I will send a new patch with shifts instead of divisions. BTW, why do we > want to avoid divisions? Integer division is at least an order of magnitude more CPU cycles than a shift. Some archs are worse than others, but historically we go out of the way to avoid them in a hot path, so shifting is a more familiar coding pattern. Compilers typically implement division as a shift if you're dividing by a a power of two integer constant expression (ICE). This example here isn't an ICE, but it is a shifted constant power-of-two. I wrote up a simple test to see what my compiler does with that, and it looks like gcc will properly optimize it, but only if compiled with '-O3'.
Re: [PATCH] hw/nvme: Add helper functions for qid-db conversion
at 4:54 PM, Klaus Jensen wrote: > I am unsure if the compiler will transform that division into the shift > if it can infer that the divisor is a power of two (it most likely > will be able to). > > But I see no reason to have a potential division here when we can do > without and to me it is just as readable when you know the definition of > DSTRD is `2 ^ (2 + DSTRD)`. OK. I will send a new patch with shifts instead of divisions. BTW, why do we want to avoid divisions?
Re: [PATCH] hw/nvme: Add helper functions for qid-db conversion
On Aug 2 10:54, Klaus Jensen wrote: > On Aug 2 16:31, Jinhao Fan wrote: > > at 2:02 PM, Klaus Jensen wrote: > > > > > On Jul 28 16:07, Jinhao Fan wrote: > > >> With the introduction of shadow doorbell and ioeventfd, we need to do > > >> frequent conversion between qid and its doorbell offset. The original > > >> hard-coded calculation is confusing and error-prone. Add several helper > > >> functions to do this task. > > >> > > >> Signed-off-by: Jinhao Fan > > >> --- > > >> hw/nvme/ctrl.c | 61 -- > > >> 1 file changed, 39 insertions(+), 22 deletions(-) > > >> > > >> diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c > > >> index 533ad14e7a..6116c0e660 100644 > > >> --- a/hw/nvme/ctrl.c > > >> +++ b/hw/nvme/ctrl.c > > >> @@ -487,6 +487,29 @@ static int nvme_check_cqid(NvmeCtrl *n, uint16_t > > >> cqid) > > >> { > > >> return cqid < n->conf_ioqpairs + 1 && n->cq[cqid] != NULL ? 0 : -1; > > >> } > > >> +static inline bool nvme_db_offset_is_cq(NvmeCtrl *n, hwaddr offset) > > >> +{ > > >> +hwaddr stride = 4 << NVME_CAP_DSTRD(ldq_le_p(&n->bar.cap)); > > >> +return (offset / stride) & 1; > > >> +} > > > > > > This can be changed morphed into `(offset >> (2 + dstrd)) & 1` if I am not > > > mistaken. > > > > > > > Yes. But my current code looks more readable to me. Is it necessary to > > change to `(offset >> (2 + dstrd)) & 1`. > > > > I am unsure if the compiler will transform that division into the shift > if it can infer that the divisor is a power of two (it most likely > will be able to). > > But I see no reason to have a potential division here when we can do > without and to me it is just as readable when you know the definition of > DSTRD is `2 ^ (2 + DSTRD)`. > > > >> + > > >> +static inline uint16_t nvme_db_offset_to_qid(NvmeCtrl *n, hwaddr offset) > > >> +{ > > >> +hwaddr stride = 4 << NVME_CAP_DSTRD(ldq_le_p(&n->bar.cap)); > > >> +return offset / (2 * stride); > > >> +} > > > > > > Same, should be able to do `offset >> (2 * dstrd + 1)`, no? > > > > Same as above. > > > I meant `offset >> (2 + dstrd + 1)` ('+', not '*') like above of course. signature.asc Description: PGP signature
Re: [PATCH] hw/nvme: Add helper functions for qid-db conversion
at 2:02 PM, Klaus Jensen wrote: > On Jul 28 16:07, Jinhao Fan wrote: >> With the introduction of shadow doorbell and ioeventfd, we need to do >> frequent conversion between qid and its doorbell offset. The original >> hard-coded calculation is confusing and error-prone. Add several helper >> functions to do this task. >> >> Signed-off-by: Jinhao Fan >> --- >> hw/nvme/ctrl.c | 61 -- >> 1 file changed, 39 insertions(+), 22 deletions(-) >> >> diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c >> index 533ad14e7a..6116c0e660 100644 >> --- a/hw/nvme/ctrl.c >> +++ b/hw/nvme/ctrl.c >> @@ -487,6 +487,29 @@ static int nvme_check_cqid(NvmeCtrl *n, uint16_t cqid) >> { >> return cqid < n->conf_ioqpairs + 1 && n->cq[cqid] != NULL ? 0 : -1; >> } >> +static inline bool nvme_db_offset_is_cq(NvmeCtrl *n, hwaddr offset) >> +{ >> +hwaddr stride = 4 << NVME_CAP_DSTRD(ldq_le_p(&n->bar.cap)); >> +return (offset / stride) & 1; >> +} > > This can be changed morphed into `(offset >> (2 + dstrd)) & 1` if I am not > mistaken. > Yes. But my current code looks more readable to me. Is it necessary to change to `(offset >> (2 + dstrd)) & 1`. >> + >> +static inline uint16_t nvme_db_offset_to_qid(NvmeCtrl *n, hwaddr offset) >> +{ >> +hwaddr stride = 4 << NVME_CAP_DSTRD(ldq_le_p(&n->bar.cap)); >> +return offset / (2 * stride); >> +} > > Same, should be able to do `offset >> (2 * dstrd + 1)`, no? Same as above.
Re: [PATCH] hw/nvme: Add helper functions for qid-db conversion
On Aug 2 16:31, Jinhao Fan wrote: > at 2:02 PM, Klaus Jensen wrote: > > > On Jul 28 16:07, Jinhao Fan wrote: > >> With the introduction of shadow doorbell and ioeventfd, we need to do > >> frequent conversion between qid and its doorbell offset. The original > >> hard-coded calculation is confusing and error-prone. Add several helper > >> functions to do this task. > >> > >> Signed-off-by: Jinhao Fan > >> --- > >> hw/nvme/ctrl.c | 61 -- > >> 1 file changed, 39 insertions(+), 22 deletions(-) > >> > >> diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c > >> index 533ad14e7a..6116c0e660 100644 > >> --- a/hw/nvme/ctrl.c > >> +++ b/hw/nvme/ctrl.c > >> @@ -487,6 +487,29 @@ static int nvme_check_cqid(NvmeCtrl *n, uint16_t cqid) > >> { > >> return cqid < n->conf_ioqpairs + 1 && n->cq[cqid] != NULL ? 0 : -1; > >> } > >> +static inline bool nvme_db_offset_is_cq(NvmeCtrl *n, hwaddr offset) > >> +{ > >> +hwaddr stride = 4 << NVME_CAP_DSTRD(ldq_le_p(&n->bar.cap)); > >> +return (offset / stride) & 1; > >> +} > > > > This can be changed morphed into `(offset >> (2 + dstrd)) & 1` if I am not > > mistaken. > > > > Yes. But my current code looks more readable to me. Is it necessary to > change to `(offset >> (2 + dstrd)) & 1`. > I am unsure if the compiler will transform that division into the shift if it can infer that the divisor is a power of two (it most likely will be able to). But I see no reason to have a potential division here when we can do without and to me it is just as readable when you know the definition of DSTRD is `2 ^ (2 + DSTRD)`. > >> + > >> +static inline uint16_t nvme_db_offset_to_qid(NvmeCtrl *n, hwaddr offset) > >> +{ > >> +hwaddr stride = 4 << NVME_CAP_DSTRD(ldq_le_p(&n->bar.cap)); > >> +return offset / (2 * stride); > >> +} > > > > Same, should be able to do `offset >> (2 * dstrd + 1)`, no? > > Same as above. > -- One of us - No more doubt, silence or taboo about mental illness. signature.asc Description: PGP signature
Re: [PATCH] hw/nvme: Add helper functions for qid-db conversion
On Jul 28 16:07, Jinhao Fan wrote: > With the introduction of shadow doorbell and ioeventfd, we need to do > frequent conversion between qid and its doorbell offset. The original > hard-coded calculation is confusing and error-prone. Add several helper > functions to do this task. > > Signed-off-by: Jinhao Fan > --- > hw/nvme/ctrl.c | 61 -- > 1 file changed, 39 insertions(+), 22 deletions(-) > > diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c > index 533ad14e7a..6116c0e660 100644 > --- a/hw/nvme/ctrl.c > +++ b/hw/nvme/ctrl.c > @@ -487,6 +487,29 @@ static int nvme_check_cqid(NvmeCtrl *n, uint16_t cqid) > { > return cqid < n->conf_ioqpairs + 1 && n->cq[cqid] != NULL ? 0 : -1; > } > +static inline bool nvme_db_offset_is_cq(NvmeCtrl *n, hwaddr offset) > +{ > +hwaddr stride = 4 << NVME_CAP_DSTRD(ldq_le_p(&n->bar.cap)); > +return (offset / stride) & 1; > +} This can be changed morphed into `(offset >> (2 + dstrd)) & 1` if I am not mistaken. > + > +static inline uint16_t nvme_db_offset_to_qid(NvmeCtrl *n, hwaddr offset) > +{ > +hwaddr stride = 4 << NVME_CAP_DSTRD(ldq_le_p(&n->bar.cap)); > +return offset / (2 * stride); > +} Same, should be able to do `offset >> (2 * dstrd + 1)`, no? > + > +static inline hwaddr nvme_cqid_to_db_offset(NvmeCtrl *n, uint16_t cqid) > +{ > +hwaddr stride = 4 << NVME_CAP_DSTRD(ldq_le_p(&n->bar.cap)); > +return stride * (cqid * 2 + 1); > +} > + > +static inline hwaddr nvme_sqid_to_db_offset(NvmeCtrl *n, uint16_t sqid) > +{ > +hwaddr stride = 4 << NVME_CAP_DSTRD(ldq_le_p(&n->bar.cap)); > +return stride * sqid * 2; > +} > > static void nvme_inc_cq_tail(NvmeCQueue *cq) > { > @@ -4256,7 +4279,7 @@ static void nvme_cq_notifier(EventNotifier *e) > static int nvme_init_cq_ioeventfd(NvmeCQueue *cq) > { > NvmeCtrl *n = cq->ctrl; > -uint16_t offset = (cq->cqid << 3) + (1 << 2); > +uint16_t offset = nvme_cqid_to_db_offset(n, cq->cqid); > int ret; > > ret = event_notifier_init(&cq->notifier, 0); > @@ -4283,7 +4306,7 @@ static void nvme_sq_notifier(EventNotifier *e) > static int nvme_init_sq_ioeventfd(NvmeSQueue *sq) > { > NvmeCtrl *n = sq->ctrl; > -uint16_t offset = sq->sqid << 3; > +uint16_t offset = nvme_sqid_to_db_offset(n, sq->sqid); > int ret; > > ret = event_notifier_init(&sq->notifier, 0); > @@ -4300,7 +4323,7 @@ static int nvme_init_sq_ioeventfd(NvmeSQueue *sq) > > static void nvme_free_sq(NvmeSQueue *sq, NvmeCtrl *n) > { > -uint16_t offset = sq->sqid << 3; > +uint16_t offset = nvme_sqid_to_db_offset(n, sq->sqid); > > n->sq[sq->sqid] = NULL; > timer_free(sq->timer); > @@ -4379,8 +4402,8 @@ static void nvme_init_sq(NvmeSQueue *sq, NvmeCtrl *n, > uint64_t dma_addr, > sq->timer = timer_new_ns(QEMU_CLOCK_VIRTUAL, nvme_process_sq, sq); > > if (n->dbbuf_enabled) { > -sq->db_addr = n->dbbuf_dbs + (sqid << 3); > -sq->ei_addr = n->dbbuf_eis + (sqid << 3); > +sq->db_addr = n->dbbuf_dbs + nvme_sqid_to_db_offset(n, sqid); > +sq->ei_addr = n->dbbuf_eis + nvme_sqid_to_db_offset(n, sqid); > > if (n->params.ioeventfd && sq->sqid != 0) { > if (!nvme_init_sq_ioeventfd(sq)) { > @@ -4690,8 +4713,8 @@ static uint16_t nvme_get_log(NvmeCtrl *n, NvmeRequest > *req) > > static void nvme_free_cq(NvmeCQueue *cq, NvmeCtrl *n) > { > -uint16_t offset = (cq->cqid << 3) + (1 << 2); > - > +uint16_t offset = nvme_cqid_to_db_offset(n, cq->cqid); > + > n->cq[cq->cqid] = NULL; > timer_free(cq->timer); > if (cq->ioeventfd_enabled) { > @@ -4755,8 +4778,8 @@ static void nvme_init_cq(NvmeCQueue *cq, NvmeCtrl *n, > uint64_t dma_addr, > QTAILQ_INIT(&cq->req_list); > QTAILQ_INIT(&cq->sq_list); > if (n->dbbuf_enabled) { > -cq->db_addr = n->dbbuf_dbs + (cqid << 3) + (1 << 2); > -cq->ei_addr = n->dbbuf_eis + (cqid << 3) + (1 << 2); > +cq->db_addr = n->dbbuf_dbs + nvme_cqid_to_db_offset(n, cqid); > +cq->ei_addr = n->dbbuf_eis + nvme_cqid_to_db_offset(n, cqid); > > if (n->params.ioeventfd && cqid != 0) { > if (!nvme_init_cq_ioeventfd(cq)) { > @@ -6128,13 +6151,8 @@ static uint16_t nvme_dbbuf_config(NvmeCtrl *n, const > NvmeRequest *req) > NvmeCQueue *cq = n->cq[i]; > > if (sq) { > -/* > - * CAP.DSTRD is 0, so offset of ith sq db_addr is (i<<3) > - * nvme_process_db() uses this hard-coded way to calculate > - * doorbell offsets. Be consistent with that here. > - */ > -sq->db_addr = dbs_addr + (i << 3); > -sq->ei_addr = eis_addr + (i << 3); > +sq->db_addr = dbs_addr + nvme_sqid_to_db_offset(n, i); > +sq->ei_addr = eis_addr + nvme_sqid_to_db_offset(n, i); > pci_dma_write(&n->parent_obj, sq->db_addr, &sq->tail, > sizeof(sq->tail)); > > @@
Re: [PATCH] hw/nvme: Add helper functions for qid-db conversion
at 4:07 PM, Jinhao Fan wrote: > With the introduction of shadow doorbell and ioeventfd, we need to do > frequent conversion between qid and its doorbell offset. The original > hard-coded calculation is confusing and error-prone. Add several helper > functions to do this task. > > Signed-off-by: Jinhao Fan > --- > hw/nvme/ctrl.c | 61 -- > 1 file changed, 39 insertions(+), 22 deletions(-) > > diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c > index 533ad14e7a..6116c0e660 100644 > --- a/hw/nvme/ctrl.c > +++ b/hw/nvme/ctrl.c > @@ -487,6 +487,29 @@ static int nvme_check_cqid(NvmeCtrl *n, uint16_t cqid) > { > return cqid < n->conf_ioqpairs + 1 && n->cq[cqid] != NULL ? 0 : -1; > } > +static inline bool nvme_db_offset_is_cq(NvmeCtrl *n, hwaddr offset) > +{ > +hwaddr stride = 4 << NVME_CAP_DSTRD(ldq_le_p(&n->bar.cap)); > +return (offset / stride) & 1; > +} > + > +static inline uint16_t nvme_db_offset_to_qid(NvmeCtrl *n, hwaddr offset) > +{ > +hwaddr stride = 4 << NVME_CAP_DSTRD(ldq_le_p(&n->bar.cap)); > +return offset / (2 * stride); > +} > + > +static inline hwaddr nvme_cqid_to_db_offset(NvmeCtrl *n, uint16_t cqid) > +{ > +hwaddr stride = 4 << NVME_CAP_DSTRD(ldq_le_p(&n->bar.cap)); > +return stride * (cqid * 2 + 1); > +} > + > +static inline hwaddr nvme_sqid_to_db_offset(NvmeCtrl *n, uint16_t sqid) > +{ > +hwaddr stride = 4 << NVME_CAP_DSTRD(ldq_le_p(&n->bar.cap)); > +return stride * sqid * 2; > +} > > static void nvme_inc_cq_tail(NvmeCQueue *cq) > { > @@ -4256,7 +4279,7 @@ static void nvme_cq_notifier(EventNotifier *e) > static int nvme_init_cq_ioeventfd(NvmeCQueue *cq) > { > NvmeCtrl *n = cq->ctrl; > -uint16_t offset = (cq->cqid << 3) + (1 << 2); > +uint16_t offset = nvme_cqid_to_db_offset(n, cq->cqid); > int ret; > > ret = event_notifier_init(&cq->notifier, 0); > @@ -4283,7 +4306,7 @@ static void nvme_sq_notifier(EventNotifier *e) > static int nvme_init_sq_ioeventfd(NvmeSQueue *sq) > { > NvmeCtrl *n = sq->ctrl; > -uint16_t offset = sq->sqid << 3; > +uint16_t offset = nvme_sqid_to_db_offset(n, sq->sqid); > int ret; > > ret = event_notifier_init(&sq->notifier, 0); > @@ -4300,7 +4323,7 @@ static int nvme_init_sq_ioeventfd(NvmeSQueue *sq) > > static void nvme_free_sq(NvmeSQueue *sq, NvmeCtrl *n) > { > -uint16_t offset = sq->sqid << 3; > +uint16_t offset = nvme_sqid_to_db_offset(n, sq->sqid); > > n->sq[sq->sqid] = NULL; > timer_free(sq->timer); > @@ -4379,8 +4402,8 @@ static void nvme_init_sq(NvmeSQueue *sq, NvmeCtrl *n, > uint64_t dma_addr, > sq->timer = timer_new_ns(QEMU_CLOCK_VIRTUAL, nvme_process_sq, sq); > > if (n->dbbuf_enabled) { > -sq->db_addr = n->dbbuf_dbs + (sqid << 3); > -sq->ei_addr = n->dbbuf_eis + (sqid << 3); > +sq->db_addr = n->dbbuf_dbs + nvme_sqid_to_db_offset(n, sqid); > +sq->ei_addr = n->dbbuf_eis + nvme_sqid_to_db_offset(n, sqid); > > if (n->params.ioeventfd && sq->sqid != 0) { > if (!nvme_init_sq_ioeventfd(sq)) { > @@ -4690,8 +4713,8 @@ static uint16_t nvme_get_log(NvmeCtrl *n, NvmeRequest > *req) > > static void nvme_free_cq(NvmeCQueue *cq, NvmeCtrl *n) > { > -uint16_t offset = (cq->cqid << 3) + (1 << 2); > - > +uint16_t offset = nvme_cqid_to_db_offset(n, cq->cqid); > + > n->cq[cq->cqid] = NULL; > timer_free(cq->timer); > if (cq->ioeventfd_enabled) { > @@ -4755,8 +4778,8 @@ static void nvme_init_cq(NvmeCQueue *cq, NvmeCtrl *n, > uint64_t dma_addr, > QTAILQ_INIT(&cq->req_list); > QTAILQ_INIT(&cq->sq_list); > if (n->dbbuf_enabled) { > -cq->db_addr = n->dbbuf_dbs + (cqid << 3) + (1 << 2); > -cq->ei_addr = n->dbbuf_eis + (cqid << 3) + (1 << 2); > +cq->db_addr = n->dbbuf_dbs + nvme_cqid_to_db_offset(n, cqid); > +cq->ei_addr = n->dbbuf_eis + nvme_cqid_to_db_offset(n, cqid); > > if (n->params.ioeventfd && cqid != 0) { > if (!nvme_init_cq_ioeventfd(cq)) { > @@ -6128,13 +6151,8 @@ static uint16_t nvme_dbbuf_config(NvmeCtrl *n, const > NvmeRequest *req) > NvmeCQueue *cq = n->cq[i]; > > if (sq) { > -/* > - * CAP.DSTRD is 0, so offset of ith sq db_addr is (i<<3) > - * nvme_process_db() uses this hard-coded way to calculate > - * doorbell offsets. Be consistent with that here. > - */ > -sq->db_addr = dbs_addr + (i << 3); > -sq->ei_addr = eis_addr + (i << 3); > +sq->db_addr = dbs_addr + nvme_sqid_to_db_offset(n, i); > +sq->ei_addr = eis_addr + nvme_sqid_to_db_offset(n, i); > pci_dma_write(&n->parent_obj, sq->db_addr, &sq->tail, > sizeof(sq->tail)); > > @@ -6146,9 +6164,8 @@ static uint16_t nvme_dbbuf_config(NvmeCtrl *n, const > NvmeRequest *req) > } > > if (cq) { > -/* CAP.DSTRD is 0, so offset of ith cq db_addr is (i<<3)+(1<
[PATCH] hw/nvme: Add helper functions for qid-db conversion
With the introduction of shadow doorbell and ioeventfd, we need to do frequent conversion between qid and its doorbell offset. The original hard-coded calculation is confusing and error-prone. Add several helper functions to do this task. Signed-off-by: Jinhao Fan --- hw/nvme/ctrl.c | 61 -- 1 file changed, 39 insertions(+), 22 deletions(-) diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c index 533ad14e7a..6116c0e660 100644 --- a/hw/nvme/ctrl.c +++ b/hw/nvme/ctrl.c @@ -487,6 +487,29 @@ static int nvme_check_cqid(NvmeCtrl *n, uint16_t cqid) { return cqid < n->conf_ioqpairs + 1 && n->cq[cqid] != NULL ? 0 : -1; } +static inline bool nvme_db_offset_is_cq(NvmeCtrl *n, hwaddr offset) +{ +hwaddr stride = 4 << NVME_CAP_DSTRD(ldq_le_p(&n->bar.cap)); +return (offset / stride) & 1; +} + +static inline uint16_t nvme_db_offset_to_qid(NvmeCtrl *n, hwaddr offset) +{ +hwaddr stride = 4 << NVME_CAP_DSTRD(ldq_le_p(&n->bar.cap)); +return offset / (2 * stride); +} + +static inline hwaddr nvme_cqid_to_db_offset(NvmeCtrl *n, uint16_t cqid) +{ +hwaddr stride = 4 << NVME_CAP_DSTRD(ldq_le_p(&n->bar.cap)); +return stride * (cqid * 2 + 1); +} + +static inline hwaddr nvme_sqid_to_db_offset(NvmeCtrl *n, uint16_t sqid) +{ +hwaddr stride = 4 << NVME_CAP_DSTRD(ldq_le_p(&n->bar.cap)); +return stride * sqid * 2; +} static void nvme_inc_cq_tail(NvmeCQueue *cq) { @@ -4256,7 +4279,7 @@ static void nvme_cq_notifier(EventNotifier *e) static int nvme_init_cq_ioeventfd(NvmeCQueue *cq) { NvmeCtrl *n = cq->ctrl; -uint16_t offset = (cq->cqid << 3) + (1 << 2); +uint16_t offset = nvme_cqid_to_db_offset(n, cq->cqid); int ret; ret = event_notifier_init(&cq->notifier, 0); @@ -4283,7 +4306,7 @@ static void nvme_sq_notifier(EventNotifier *e) static int nvme_init_sq_ioeventfd(NvmeSQueue *sq) { NvmeCtrl *n = sq->ctrl; -uint16_t offset = sq->sqid << 3; +uint16_t offset = nvme_sqid_to_db_offset(n, sq->sqid); int ret; ret = event_notifier_init(&sq->notifier, 0); @@ -4300,7 +4323,7 @@ static int nvme_init_sq_ioeventfd(NvmeSQueue *sq) static void nvme_free_sq(NvmeSQueue *sq, NvmeCtrl *n) { -uint16_t offset = sq->sqid << 3; +uint16_t offset = nvme_sqid_to_db_offset(n, sq->sqid); n->sq[sq->sqid] = NULL; timer_free(sq->timer); @@ -4379,8 +4402,8 @@ static void nvme_init_sq(NvmeSQueue *sq, NvmeCtrl *n, uint64_t dma_addr, sq->timer = timer_new_ns(QEMU_CLOCK_VIRTUAL, nvme_process_sq, sq); if (n->dbbuf_enabled) { -sq->db_addr = n->dbbuf_dbs + (sqid << 3); -sq->ei_addr = n->dbbuf_eis + (sqid << 3); +sq->db_addr = n->dbbuf_dbs + nvme_sqid_to_db_offset(n, sqid); +sq->ei_addr = n->dbbuf_eis + nvme_sqid_to_db_offset(n, sqid); if (n->params.ioeventfd && sq->sqid != 0) { if (!nvme_init_sq_ioeventfd(sq)) { @@ -4690,8 +4713,8 @@ static uint16_t nvme_get_log(NvmeCtrl *n, NvmeRequest *req) static void nvme_free_cq(NvmeCQueue *cq, NvmeCtrl *n) { -uint16_t offset = (cq->cqid << 3) + (1 << 2); - +uint16_t offset = nvme_cqid_to_db_offset(n, cq->cqid); + n->cq[cq->cqid] = NULL; timer_free(cq->timer); if (cq->ioeventfd_enabled) { @@ -4755,8 +4778,8 @@ static void nvme_init_cq(NvmeCQueue *cq, NvmeCtrl *n, uint64_t dma_addr, QTAILQ_INIT(&cq->req_list); QTAILQ_INIT(&cq->sq_list); if (n->dbbuf_enabled) { -cq->db_addr = n->dbbuf_dbs + (cqid << 3) + (1 << 2); -cq->ei_addr = n->dbbuf_eis + (cqid << 3) + (1 << 2); +cq->db_addr = n->dbbuf_dbs + nvme_cqid_to_db_offset(n, cqid); +cq->ei_addr = n->dbbuf_eis + nvme_cqid_to_db_offset(n, cqid); if (n->params.ioeventfd && cqid != 0) { if (!nvme_init_cq_ioeventfd(cq)) { @@ -6128,13 +6151,8 @@ static uint16_t nvme_dbbuf_config(NvmeCtrl *n, const NvmeRequest *req) NvmeCQueue *cq = n->cq[i]; if (sq) { -/* - * CAP.DSTRD is 0, so offset of ith sq db_addr is (i<<3) - * nvme_process_db() uses this hard-coded way to calculate - * doorbell offsets. Be consistent with that here. - */ -sq->db_addr = dbs_addr + (i << 3); -sq->ei_addr = eis_addr + (i << 3); +sq->db_addr = dbs_addr + nvme_sqid_to_db_offset(n, i); +sq->ei_addr = eis_addr + nvme_sqid_to_db_offset(n, i); pci_dma_write(&n->parent_obj, sq->db_addr, &sq->tail, sizeof(sq->tail)); @@ -6146,9 +6164,8 @@ static uint16_t nvme_dbbuf_config(NvmeCtrl *n, const NvmeRequest *req) } if (cq) { -/* CAP.DSTRD is 0, so offset of ith cq db_addr is (i<<3)+(1<<2) */ -cq->db_addr = dbs_addr + (i << 3) + (1 << 2); -cq->ei_addr = eis_addr + (i << 3) + (1 << 2); +cq->db_addr = dbs_addr + nvme_cqid_to_db_offset(n, i); +cq->ei_addr = eis_addr + nvme_cqi