Re: [Qemu-devel] [PATCH v3 1/1] coroutine-lock: do not touch coroutine after another one has been entered

On Thu, Jun 1, 2017 at 6:08 PM, Roman Pen wrote: [cut] > Thread 12 (Thread 0x7f57f2ffd700 (LWP 56426)): > #0 raise () from /lib/x86_64-linux-gnu/libc.so.6 > #1 abort () from /lib/x86_64-linux-gnu/libc.so.6 > #2 qemu_coroutine_enter (co=0x7f5804009af0) at

Re: [Qemu-devel] [PATCH v2 1/1] coroutine-lock: do not touch coroutine after another one has been entered

On Thu, Jun 1, 2017 at 3:15 PM, Stefan Hajnoczi <stefa...@redhat.com> wrote: > On Wed, May 31, 2017 at 03:23:25PM +0200, Roman Penyaev wrote: >> On Wed, May 31, 2017 at 3:06 PM, Stefan Hajnoczi <stefa...@gmail.com> wrote: >> > On Tue, May 30, 2017 at 12:07:36PM +020

Re: [Qemu-devel] [PATCH v2 1/1] coroutine-lock: do not touch coroutine after another one has been entered

On Tue, May 30, 2017 at 1:35 PM, Fam Zheng wrote: [cut] > Reviewed-by: Fam Zheng Do I need to resend the patch with 'reviewed-by' line? Or it is already queued? -- Roman

Re: [Qemu-devel] [PATCH v2 1/1] coroutine-lock: do not touch coroutine after another one has been entered

On Wed, May 31, 2017 at 3:06 PM, Stefan Hajnoczi wrote: > On Tue, May 30, 2017 at 12:07:36PM +0200, Roman Pen wrote: >> diff --git a/util/qemu-coroutine-lock.c b/util/qemu-coroutine-lock.c >> index 6328eed26bc6..d589d8c66d5e 100644 >> --- a/util/qemu-coroutine-lock.c >> +++

Re: [Qemu-devel] [PATCH v2 1/1] coroutine-lock: do not touch coroutine after another one has been entered

On Wed, May 31, 2017 at 3:22 PM, Paolo Bonzini wrote: > > > On 31/05/2017 15:06, Stefan Hajnoczi wrote: >>> + >>> +while ((next = QSIMPLEQ_FIRST(_queue_wakeup))) { >>> +QSIMPLEQ_REMOVE_HEAD(_queue_wakeup, co_queue_next); >>> qemu_coroutine_enter(next);

Re: [Qemu-devel] [PATCH 1/1] coroutine-lock: do not touch coroutine after another one has been entered

On Tue, May 30, 2017 at 11:35 AM, Paolo Bonzini wrote: > > > On 30/05/2017 10:53, Roman Pen wrote: >> The fix is obvious: use temporal queue and do not touch coroutine after >> first qemu_coroutine_enter() is invoked. >> >> The issue is quite rare and happens every ~12 hours

Re: [Qemu-devel] [PATCH 1/1] scripts/qemugdb/coroutine.py: get pthread pointer from '(gdb) thread info $id' output

On Tue, Mar 14, 2017 at 11:21 AM, Paolo Bonzini wrote: > > > On 14/03/2017 11:08, Roman Pen wrote: >> For sure under corefile debugging it is not possible to invoke >> any syscalls, like arch_prctl(), so avoid doing that. That will >> simplify the script. > > Is the issue

Re: [Qemu-devel] [PATCH 3/3] linux-aio: fix re-entrant completion processing

On Wed, Sep 28, 2016 at 11:34 AM, Fam Zheng <f...@redhat.com> wrote: > On Wed, 09/28 11:14, Roman Penyaev wrote: >> On Wed, Sep 28, 2016 at 5:01 AM, Fam Zheng <f...@redhat.com> wrote: >> > On Tue, 09/27 19:55, Roman Penyaev wrote: >> >> > The bug

Re: [Qemu-devel] [PATCH 3/3] linux-aio: fix re-entrant completion processing

On Wed, Sep 28, 2016 at 5:01 AM, Fam Zheng <f...@redhat.com> wrote: > On Tue, 09/27 19:55, Roman Penyaev wrote: >> > The bug is 100% deterministic. Just boot up a guest with -drive >> > format=qcow2,aio=native. >> >> It turns out to be that everythin

Re: [Qemu-devel] [PATCH 3/3] linux-aio: fix re-entrant completion processing

On Tue, Sep 27, 2016 at 5:25 PM, Stefan Hajnoczi <stefa...@redhat.com> wrote: > On Tue, Sep 27, 2016 at 04:29:55PM +0200, Roman Penyaev wrote: >> On Tue, Sep 27, 2016 at 4:06 PM, Stefan Hajnoczi <stefa...@redhat.com> wrote: >> > Commit 0ed93d84edabc7656f5c998ae1a346f

Re: [Qemu-devel] [PATCH 3/3] linux-aio: fix re-entrant completion processing

Hey Stefan, On Tue, Sep 27, 2016 at 4:06 PM, Stefan Hajnoczi wrote: > Commit 0ed93d84edabc7656f5c998ae1a346fe8b94ca54 ("linux-aio: process > completions from ioq_submit()") added an optimization that processes > completions each time ioq_submit() returns with requests in

Re: [Qemu-devel] [PATCH 3/3] linux-aio: process completions from ioq_submit()

On Tue, Jul 19, 2016 at 1:47 PM, Paolo Bonzini <pbonz...@redhat.com> wrote: > > > On 19/07/2016 13:44, Roman Penyaev wrote: >>>> >> >>>> >> Can this leave I/O stuck if in_queue > 0 && in_flight == 0 after the >>>&g

Re: [Qemu-devel] [PATCH 3/3] linux-aio: process completions from ioq_submit()

On Tue, Jul 19, 2016 at 1:18 PM, Roman Penyaev <roman.peny...@profitbricks.com> wrote: > On Tue, Jul 19, 2016 at 12:36 PM, Paolo Bonzini <pbonz...@redhat.com> wrote: >> >> >> On 19/07/2016 12:25, Roman Pen wrote: >>> if (laiocb->co) { >&

Re: [Qemu-devel] [PATCH 3/3] linux-aio: process completions from ioq_submit()

On Tue, Jul 19, 2016 at 12:36 PM, Paolo Bonzini wrote: > > > On 19/07/2016 12:25, Roman Pen wrote: >> if (laiocb->co) { >> -qemu_coroutine_enter(laiocb->co, NULL); >> +if (laiocb->co == qemu_coroutine_self()) { >> +laiocb->self_completed =

Re: [Qemu-devel] [PATCH V2 1/1] linux-aio: prevent submitting more than MAX_EVENTS

On Fri, Jul 15, 2016 at 11:18 AM, Roman Penyaev <roman.peny...@profitbricks.com> wrote: > On Wed, Jul 13, 2016 at 1:45 PM, Kevin Wolf <kw...@redhat.com> wrote: >> Am 13.07.2016 um 13:33 hat Roman Penyaev geschrieben: >>> Just to be sure that we are on the same page:

Re: [Qemu-devel] [PATCH] linux-aio: keep processing events if MAX_EVENTS reached

On Fri, Jul 15, 2016 at 4:18 PM, Stefan Hajnoczi <stefa...@gmail.com> wrote: > On Tue, Jul 12, 2016 at 04:12:42PM +0200, Roman Penyaev wrote: >> On Tue, Jun 28, 2016 at 11:42 AM, Stefan Hajnoczi <stefa...@gmail.com> wrote: >> > On Mon, Jun 27, 2016 at 08:36:1

Re: [Qemu-devel] [PATCH V2 1/1] linux-aio: prevent submitting more than MAX_EVENTS

On Fri, Jul 15, 2016 at 12:17 PM, Roman Penyaev <roman.peny...@profitbricks.com> wrote: > On Fri, Jul 15, 2016 at 11:58 AM, Paolo Bonzini <pbonz...@redhat.com> wrote: >> >> >> On 15/07/2016 11:18, Roman Penyaev wrote: >>> Those 3 red spike

Re: [Qemu-devel] [PATCH V2 1/1] linux-aio: prevent submitting more than MAX_EVENTS

On Fri, Jul 15, 2016 at 11:58 AM, Paolo Bonzini <pbonz...@redhat.com> wrote: > > > On 15/07/2016 11:18, Roman Penyaev wrote: >> Those 3 red spikes and a blue hill is what we have to focus on. The >> blue hill at the right corner of the chart means that almos

Re: [Qemu-devel] [PATCH V2 1/1] linux-aio: prevent submitting more than MAX_EVENTS

On Wed, Jul 13, 2016 at 1:45 PM, Kevin Wolf <kw...@redhat.com> wrote: > Am 13.07.2016 um 13:33 hat Roman Penyaev geschrieben: >> Just to be sure that we are on the same page: >> >> 1. We have this commit "linux-aio: Cancel BH if not needed" which >> >&

Re: [Qemu-devel] [PATCH V2 1/1] linux-aio: prevent submitting more than MAX_EVENTS

On Wed, Jul 13, 2016 at 1:45 PM, Kevin Wolf <kw...@redhat.com> wrote: > Am 13.07.2016 um 13:33 hat Roman Penyaev geschrieben: >> Just to be sure that we are on the same page: >> >> 1. We have this commit "linux-aio: Cancel BH if not needed" which >> >&

Re: [Qemu-devel] [PATCH V2 1/1] linux-aio: prevent submitting more than MAX_EVENTS

On Wed, Jul 13, 2016 at 2:22 PM, Eric Blake wrote: > On 07/13/2016 01:57 AM, Roman Pen wrote: >> v1..v2: >> >> o comment tweaks. >> o fix QEMU coding style. > > The above comments should be delayed... > >> >> Invoking io_setup(MAX_EVENTS) we ask kernel to create ring buffer

Re: [Qemu-devel] [PATCH V2 1/1] linux-aio: prevent submitting more than MAX_EVENTS

On Wed, Jul 13, 2016 at 12:31 PM, Paolo Bonzini wrote: > > > On 13/07/2016 09:57, Roman Pen wrote: >> v1..v2: >> >> o comment tweaks. >> o fix QEMU coding style. >> >> Invoking io_setup(MAX_EVENTS) we ask kernel to create ring buffer for us >> with specified number of

Re: [Qemu-devel] [PATCH 1/1] linux-aio: prevent submitting more than MAX_EVENTS

On Wed, Jul 13, 2016 at 9:43 AM, Paolo Bonzini wrote: > > > On 12/07/2016 19:51, Roman Pen wrote: >> +if (s->io_q.in_flight >= MAX_EVENTS) >> +break; >> QSIMPLEQ_FOREACH(aiocb, >io_q.pending, next) { >> iocbs[len++] = >iocb; >> -

Re: [Qemu-devel] [PATCH] linux-aio: keep processing events if MAX_EVENTS reached

On Tue, Jul 12, 2016 at 4:59 PM, Paolo Bonzini <pbonz...@redhat.com> wrote: > > > On 12/07/2016 16:12, Roman Penyaev wrote: >> But what is the most important thing here is, that reverting >> "linux-aio: Cancel BH if not needed" brings these numbers: >&g

Re: [Qemu-devel] [PATCH] linux-aio: keep processing events if MAX_EVENTS reached

On Tue, Jun 28, 2016 at 11:42 AM, Stefan Hajnoczi <stefa...@gmail.com> wrote: > On Mon, Jun 27, 2016 at 08:36:19PM +0200, Roman Penyaev wrote: >> On Jun 27, 2016 6:37 PM, "Stefan Hajnoczi" <stefa...@redhat.com> wrote: >> > >> > Commit ccb9dc10129954

Re: [Qemu-devel] [PATCH] linux-aio: keep processing events if MAX_EVENTS reached

2) fetches MAX_EVENTS events then it's > necessary to call it again. Failure to do so can lead to hung I/O > because the eventfd has already been cleared and the completion BH will > not run again. > > This patch fixes the hang by calling io_getevents(2) again if the events > array was

Re: [Qemu-devel] [PATCH v3 0/7] virtio-blk: multiqueue support

Hi, Stefan. On Mon, Jun 20, 2016 at 12:36 PM, Stefan Hajnoczi wrote: > On Tue, Jun 07, 2016 at 05:28:24PM +0100, Stefan Hajnoczi wrote: >> v3: >> * Drop Patch 1 to batch guest notify for non-dataplane >> >>The Linux AIO completion BH and the virtio-blk batch notify BH

Re: [Qemu-devel] [PATCH v2 0/8] virtio-blk: multiqueue support

Hi, On Sat, Jun 4, 2016 at 12:26 AM, Stefan Hajnoczi wrote: > On Thu, Jun 02, 2016 at 05:19:41PM -0700, Stefan Hajnoczi wrote: >> On Mon, May 30, 2016 at 06:25:58PM -0700, Stefan Hajnoczi wrote: >> > v2: >> > * Simplify s->rq live migration [Paolo] >> > * Use more

Re: [Qemu-devel] [RFC] virtio-blk: simple multithreaded MQ implementation for bdrv_raw

On Mon, May 30, 2016 at 8:40 AM, Alexandre DERUMIER wrote: > Hi, > >>>To avoid any locks in qemu backend and not to introduce thread safety >>>into qemu block-layer I open same backend device several times, one >>>device per one MQ. e.g. the following is the stack for a

Re: [Qemu-devel] [RFC] virtio-blk: simple multithreaded MQ implementation for bdrv_raw

On Sat, May 28, 2016 at 12:27 AM, Stefan Hajnoczi wrote: > On Fri, May 27, 2016 at 01:55:04PM +0200, Roman Pen wrote: >> Hello, all. >> >> This is RFC because mostly this patch is a quick attempt to get true >> multithreaded multiqueue support for a block device with native