[Qemu-devel] What is the best commit for record-replay?

2017-03-23 Thread Igor R
Hi,

I'm trying to use the deterministic record/replay feature, and I would
like to know which commit I should take to get it work.
In RC0 it seems to be broken. I tried pre-MTTCG commit 2421f381dc, as
mentioned here:
http://lists.nongnu.org/archive/html/qemu-devel/2017-03/msg02657.html
with this patch:
http://lists.nongnu.org/archive/html/qemu-devel/2017-02/msg01316.html
My command line is:
./qemu-system-i386 -net none  -icount
shift=7,rr=replay,rrfile=replay.bin -drive
file=MyFedora386.qcow,if=none,id=img-direct -drive
driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
ide-hd,drive=img-blkreplay -monitor stdio

The replay advances for a while, but gets stuck in about 10-15 sec,
and it looks like it encounters deadlock trying to acquire rcu lock.

Is there a working commit of RR?


Thanks.



Re: [Qemu-devel] What is the best commit for record-replay?

2017-09-18 Thread Aleksandr Bezzubikov
2017-05-02 15:42 GMT+03:00 Igor R :
> I'm trying to use the deterministic record/replay feature, and I would
> like to know which commit I should take to get it work.
> In RC0 it seems to be broken. I tried pre-MTTCG commit 2421f381dc, as
>>>
 Can you retry with the latest rc? There were some fixes regarding rr since 
 rc0.
>>>
>>>
>>> I've taken 2.9 release, and RR does not seem to work there.
>>> I recorded the boot process of x86 Fedora-21 linux and the replay got
>>> stuck almost immediately.
>>
>> What's your command line?
>>
>> Does it get stuck at the same place each time?
>>
>> Can you boot fine with icount but without record/replay?
>
> Here is the exact scenario:
> - Get 2.9 from git, configure it as follows: "./configure
> --target-list=i386-softmmu --enable-sdl" and  make.
> - Download 
> https://people.debian.org/~aurel32/qemu/i386/debian_squeeze_i386_standard.qcow2
> - Run qemu with the following command line, until login prompt:
> -icount shift=7,rr=record,rrfile=replay.bin -drive
> file=debian_squeeze_i386_standard.qcow2,if=none,id=img-direct -drive
> driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
> ide-hd,drive=img-blkreplay -monitor stdio
> - Replay: -icount shift=7,rr=replay,rrfile=replay.bin -drive
> file=debian_squeeze_i386_standard.qcow2,if=none,id=img-direct -drive
> driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
> ide-hd,drive=img-blkreplay -monitor stdio
>
> Every time I attempt to replay, QEMU gets stuck at the same EIP, at a
> very early stage.
>
>
>> Can you boot fine with icount but without record/replay?
>
> Yes. I can also enable icount and recording - it also boots fine. The
> problem with the replay.

Hi guys,
Maybe the thread is a bit outdated, but the problem is still relevant.
I've just tried to record and replay WinXP boot process, and I've encountered
exactly the same problem as described above - record is fine, replay
gets stuck early. I use current master.
And I've discovered the second problem - recording makes initial snapshot,
but it doesn't seem to be saved to the disk - replay can't see it.

Hope you've already found the solution (as the last post was on 2 May)
and it's just got missed the mailing list.

>



-- 
Aleksandr Bezzubikov



Re: [Qemu-devel] What is the best commit for record-replay?

2017-09-18 Thread Aleksandr Bezzubikov
[+CC Pavel Dovgaluk, me]

2017-09-18 15:02 GMT+03:00 Aleksandr Bezzubikov :
> 2017-05-02 15:42 GMT+03:00 Igor R :
>> I'm trying to use the deterministic record/replay feature, and I would
>> like to know which commit I should take to get it work.
>> In RC0 it seems to be broken. I tried pre-MTTCG commit 2421f381dc, as

> Can you retry with the latest rc? There were some fixes regarding rr 
> since rc0.


 I've taken 2.9 release, and RR does not seem to work there.
 I recorded the boot process of x86 Fedora-21 linux and the replay got
 stuck almost immediately.
>>>
>>> What's your command line?
>>>
>>> Does it get stuck at the same place each time?
>>>
>>> Can you boot fine with icount but without record/replay?
>>
>> Here is the exact scenario:
>> - Get 2.9 from git, configure it as follows: "./configure
>> --target-list=i386-softmmu --enable-sdl" and  make.
>> - Download 
>> https://people.debian.org/~aurel32/qemu/i386/debian_squeeze_i386_standard.qcow2
>> - Run qemu with the following command line, until login prompt:
>> -icount shift=7,rr=record,rrfile=replay.bin -drive
>> file=debian_squeeze_i386_standard.qcow2,if=none,id=img-direct -drive
>> driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
>> ide-hd,drive=img-blkreplay -monitor stdio
>> - Replay: -icount shift=7,rr=replay,rrfile=replay.bin -drive
>> file=debian_squeeze_i386_standard.qcow2,if=none,id=img-direct -drive
>> driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
>> ide-hd,drive=img-blkreplay -monitor stdio
>>
>> Every time I attempt to replay, QEMU gets stuck at the same EIP, at a
>> very early stage.
>>
>>
>>> Can you boot fine with icount but without record/replay?
>>
>> Yes. I can also enable icount and recording - it also boots fine. The
>> problem with the replay.
>
> Hi guys,
> Maybe the thread is a bit outdated, but the problem is still relevant.
> I've just tried to record and replay WinXP boot process, and I've encountered
> exactly the same problem as described above - record is fine, replay
> gets stuck early. I use current master.
> And I've discovered the second problem - recording makes initial snapshot,
> but it doesn't seem to be saved to the disk - replay can't see it.
>
> Hope you've already found the solution (as the last post was on 2 May)
> and it's just got missed the mailing list.
>
>>
>
>
>
> --
> Aleksandr Bezzubikov



-- 
Aleksandr Bezzubikov



Re: [Qemu-devel] What is the best commit for record-replay?

2017-09-19 Thread Pavel Dovgalyuk
> From: Aleksandr Bezzubikov [mailto:zuban...@gmail.com]
> 2017-09-18 15:02 GMT+03:00 Aleksandr Bezzubikov :
> > 2017-05-02 15:42 GMT+03:00 Igor R :
> >> I'm trying to use the deterministic record/replay feature, and I would
> >> like to know which commit I should take to get it work.
> >> In RC0 it seems to be broken. I tried pre-MTTCG commit 2421f381dc, as
> 
> > Can you retry with the latest rc? There were some fixes regarding rr 
> > since rc0.
> 
> 
>  I've taken 2.9 release, and RR does not seem to work there.
>  I recorded the boot process of x86 Fedora-21 linux and the replay got
>  stuck almost immediately.
> >>>
> >>> What's your command line?
> >>>
> >>> Does it get stuck at the same place each time?
> >>>
> >>> Can you boot fine with icount but without record/replay?
> >>
> >> Here is the exact scenario:
> >> - Get 2.9 from git, configure it as follows: "./configure
> >> --target-list=i386-softmmu --enable-sdl" and  make.
> >> - Download 
> >> https://people.debian.org/~aurel32/qemu/i386/debian_squeeze_i386_standard.qcow2
> >> - Run qemu with the following command line, until login prompt:
> >> -icount shift=7,rr=record,rrfile=replay.bin -drive
> >> file=debian_squeeze_i386_standard.qcow2,if=none,id=img-direct -drive
> >> driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
> >> ide-hd,drive=img-blkreplay -monitor stdio
> >> - Replay: -icount shift=7,rr=replay,rrfile=replay.bin -drive
> >> file=debian_squeeze_i386_standard.qcow2,if=none,id=img-direct -drive
> >> driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
> >> ide-hd,drive=img-blkreplay -monitor stdio
> >>
> >> Every time I attempt to replay, QEMU gets stuck at the same EIP, at a
> >> very early stage.
> >>
> >>
> >>> Can you boot fine with icount but without record/replay?
> >>
> >> Yes. I can also enable icount and recording - it also boots fine. The
> >> problem with the replay.
> >
> > Hi guys,
> > Maybe the thread is a bit outdated, but the problem is still relevant.
> > I've just tried to record and replay WinXP boot process, and I've 
> > encountered
> > exactly the same problem as described above - record is fine, replay
> > gets stuck early. I use current master.

Maybe this commit will work: cfb2d02be9413d45b30ed6d8e38800250b6b4b48

> > And I've discovered the second problem - recording makes initial snapshot,
> > but it doesn't seem to be saved to the disk - replay can't see it.

It is ok, because there is a mode where snapshot is created and loaded.

> >
> > Hope you've already found the solution (as the last post was on 2 May)
> > and it's just got missed the mailing list.

As I know, RR is still broken in the current version.
It was caused by the MTTCG implementation.
Alex Bennee tried to fix RR back. Alex, have you found any solution?

We also trying to find a way to fix RR. It seems, that we will reinvent BQL for 
RR.

Pavel Dovgalyuk




Re: [Qemu-devel] What is the best commit for record-replay?

2017-09-19 Thread Alex Bennée

Pavel Dovgalyuk  writes:

>> From: Aleksandr Bezzubikov [mailto:zuban...@gmail.com]
>> 2017-09-18 15:02 GMT+03:00 Aleksandr Bezzubikov :
>> > 2017-05-02 15:42 GMT+03:00 Igor R :
>> >> I'm trying to use the deterministic record/replay feature, and I would
>> >> like to know which commit I should take to get it work.
>> >> In RC0 it seems to be broken. I tried pre-MTTCG commit 2421f381dc, as
>> 
>> > Can you retry with the latest rc? There were some fixes regarding rr 
>> > since rc0.
>> 
>> 
>>  I've taken 2.9 release, and RR does not seem to work there.
>>  I recorded the boot process of x86 Fedora-21 linux and the replay got
>>  stuck almost immediately.
>> >>>
>> >>> What's your command line?
>> >>>
>> >>> Does it get stuck at the same place each time?
>> >>>
>> >>> Can you boot fine with icount but without record/replay?
>> >>
>> >> Here is the exact scenario:
>> >> - Get 2.9 from git, configure it as follows: "./configure
>> >> --target-list=i386-softmmu --enable-sdl" and  make.
>> >> - Download 
>> >> https://people.debian.org/~aurel32/qemu/i386/debian_squeeze_i386_standard.qcow2
>> >> - Run qemu with the following command line, until login prompt:
>> >> -icount shift=7,rr=record,rrfile=replay.bin -drive
>> >> file=debian_squeeze_i386_standard.qcow2,if=none,id=img-direct -drive
>> >> driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
>> >> ide-hd,drive=img-blkreplay -monitor stdio
>> >> - Replay: -icount shift=7,rr=replay,rrfile=replay.bin -drive
>> >> file=debian_squeeze_i386_standard.qcow2,if=none,id=img-direct -drive
>> >> driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
>> >> ide-hd,drive=img-blkreplay -monitor stdio
>> >>
>> >> Every time I attempt to replay, QEMU gets stuck at the same EIP, at a
>> >> very early stage.
>> >>
>> >>
>> >>> Can you boot fine with icount but without record/replay?
>> >>
>> >> Yes. I can also enable icount and recording - it also boots fine. The
>> >> problem with the replay.
>> >
>> > Hi guys,
>> > Maybe the thread is a bit outdated, but the problem is still relevant.
>> > I've just tried to record and replay WinXP boot process, and I've 
>> > encountered
>> > exactly the same problem as described above - record is fine, replay
>> > gets stuck early. I use current master.
>
> Maybe this commit will work: cfb2d02be9413d45b30ed6d8e38800250b6b4b48
>
>> > And I've discovered the second problem - recording makes initial snapshot,
>> > but it doesn't seem to be saved to the disk - replay can't see it.
>
> It is ok, because there is a mode where snapshot is created and loaded.
>
>> >
>> > Hope you've already found the solution (as the last post was on 2 May)
>> > and it's just got missed the mailing list.
>
> As I know, RR is still broken in the current version.
> It was caused by the MTTCG implementation.
> Alex Bennee tried to fix RR back. Alex, have you found any solution?
>
> We also trying to find a way to fix RR. It seems, that we will reinvent BQL 
> for RR.

I think the method outlined in my RFC is the way to go, essentially the
RR mutex taking over for the what the BQL did. The RFC patch hadn't
hoisted the mutex for the additional devices so I'm just re-basing now
and I'll see if I can make the changes for Igor's test case.

--
Alex Bennée



Re: [Qemu-devel] What is the best commit for record-replay?

2017-09-19 Thread Aleksandr Bezzubikov
2017-09-19 12:30 GMT+03:00 Alex Bennée :
>
> Pavel Dovgalyuk  writes:
>
>>> From: Aleksandr Bezzubikov [mailto:zuban...@gmail.com]
>>> 2017-09-18 15:02 GMT+03:00 Aleksandr Bezzubikov :
>>> > 2017-05-02 15:42 GMT+03:00 Igor R :
>>> >> I'm trying to use the deterministic record/replay feature, and I 
>>> >> would
>>> >> like to know which commit I should take to get it work.
>>> >> In RC0 it seems to be broken. I tried pre-MTTCG commit 2421f381dc, as
>>> 
>>> > Can you retry with the latest rc? There were some fixes regarding rr 
>>> > since rc0.
>>> 
>>> 
>>>  I've taken 2.9 release, and RR does not seem to work there.
>>>  I recorded the boot process of x86 Fedora-21 linux and the replay got
>>>  stuck almost immediately.
>>> >>>
>>> >>> What's your command line?
>>> >>>
>>> >>> Does it get stuck at the same place each time?
>>> >>>
>>> >>> Can you boot fine with icount but without record/replay?
>>> >>
>>> >> Here is the exact scenario:
>>> >> - Get 2.9 from git, configure it as follows: "./configure
>>> >> --target-list=i386-softmmu --enable-sdl" and  make.
>>> >> - Download 
>>> >> https://people.debian.org/~aurel32/qemu/i386/debian_squeeze_i386_standard.qcow2
>>> >> - Run qemu with the following command line, until login prompt:
>>> >> -icount shift=7,rr=record,rrfile=replay.bin -drive
>>> >> file=debian_squeeze_i386_standard.qcow2,if=none,id=img-direct -drive
>>> >> driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
>>> >> ide-hd,drive=img-blkreplay -monitor stdio
>>> >> - Replay: -icount shift=7,rr=replay,rrfile=replay.bin -drive
>>> >> file=debian_squeeze_i386_standard.qcow2,if=none,id=img-direct -drive
>>> >> driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
>>> >> ide-hd,drive=img-blkreplay -monitor stdio
>>> >>
>>> >> Every time I attempt to replay, QEMU gets stuck at the same EIP, at a
>>> >> very early stage.
>>> >>
>>> >>
>>> >>> Can you boot fine with icount but without record/replay?
>>> >>
>>> >> Yes. I can also enable icount and recording - it also boots fine. The
>>> >> problem with the replay.
>>> >
>>> > Hi guys,
>>> > Maybe the thread is a bit outdated, but the problem is still relevant.
>>> > I've just tried to record and replay WinXP boot process, and I've 
>>> > encountered
>>> > exactly the same problem as described above - record is fine, replay
>>> > gets stuck early. I use current master.
>>
>> Maybe this commit will work: cfb2d02be9413d45b30ed6d8e38800250b6b4b48

Unfortunately this one doesn't work either. It seems we need just an
all-in-one fix
for the current implementation to make it work.

>>
>>> > And I've discovered the second problem - recording makes initial snapshot,
>>> > but it doesn't seem to be saved to the disk - replay can't see it.
>>
>> It is ok, because there is a mode where snapshot is created and loaded.

So it shouldn't work properly when I use 'rrsnapshot=' for both
record and replay?
Then how can I enable this mode?

>>
>>> >
>>> > Hope you've already found the solution (as the last post was on 2 May)
>>> > and it's just got missed the mailing list.
>>
>> As I know, RR is still broken in the current version.
>> It was caused by the MTTCG implementation.
>> Alex Bennee tried to fix RR back. Alex, have you found any solution?
>>
>> We also trying to find a way to fix RR. It seems, that we will reinvent BQL 
>> for RR.
>
> I think the method outlined in my RFC is the way to go, essentially the
> RR mutex taking over for the what the BQL did. The RFC patch hadn't
> hoisted the mutex for the additional devices so I'm just re-basing now
> and I'll see if I can make the changes for Igor's test case.
>
> --
> Alex Bennée



-- 
Aleksandr Bezzubikov



Re: [Qemu-devel] What is the best commit for record-replay?

2017-09-19 Thread Alex Bennée

Aleksandr Bezzubikov  writes:

> 2017-09-19 12:30 GMT+03:00 Alex Bennée :
>>
>> Pavel Dovgalyuk  writes:
>>
 From: Aleksandr Bezzubikov [mailto:zuban...@gmail.com]
 2017-09-18 15:02 GMT+03:00 Aleksandr Bezzubikov :
 > 2017-05-02 15:42 GMT+03:00 Igor R :
 >> I'm trying to use the deterministic record/replay feature, and I 
 >> would
 >> like to know which commit I should take to get it work.
 >> In RC0 it seems to be broken. I tried pre-MTTCG commit 2421f381dc, 
 >> as
 
 > Can you retry with the latest rc? There were some fixes regarding rr 
 > since rc0.
 
 
  I've taken 2.9 release, and RR does not seem to work there.
  I recorded the boot process of x86 Fedora-21 linux and the replay got
  stuck almost immediately.
 >>>
 >>> What's your command line?
 >>>
 >>> Does it get stuck at the same place each time?
 >>>
 >>> Can you boot fine with icount but without record/replay?
 >>
 >> Here is the exact scenario:
 >> - Get 2.9 from git, configure it as follows: "./configure
 >> --target-list=i386-softmmu --enable-sdl" and  make.
 >> - Download 
 >> https://people.debian.org/~aurel32/qemu/i386/debian_squeeze_i386_standard.qcow2
 >> - Run qemu with the following command line, until login prompt:
 >> -icount shift=7,rr=record,rrfile=replay.bin -drive
 >> file=debian_squeeze_i386_standard.qcow2,if=none,id=img-direct -drive
 >> driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
 >> ide-hd,drive=img-blkreplay -monitor stdio
 >> - Replay: -icount shift=7,rr=replay,rrfile=replay.bin -drive
 >> file=debian_squeeze_i386_standard.qcow2,if=none,id=img-direct -drive
 >> driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
 >> ide-hd,drive=img-blkreplay -monitor stdio
 >>
 >> Every time I attempt to replay, QEMU gets stuck at the same EIP, at a
 >> very early stage.
 >>
 >>
 >>> Can you boot fine with icount but without record/replay?
 >>
 >> Yes. I can also enable icount and recording - it also boots fine. The
 >> problem with the replay.
 >
 > Hi guys,
 > Maybe the thread is a bit outdated, but the problem is still relevant.
 > I've just tried to record and replay WinXP boot process, and I've 
 > encountered
 > exactly the same problem as described above - record is fine, replay
 > gets stuck early. I use current master.
>>>
>>> Maybe this commit will work: cfb2d02be9413d45b30ed6d8e38800250b6b4b48
>
> Unfortunately this one doesn't work either. It seems we need just an
> all-in-one fix
> for the current implementation to make it work.
>
>>>
 > And I've discovered the second problem - recording makes initial 
 > snapshot,
 > but it doesn't seem to be saved to the disk - replay can't see it.
>>>
>>> It is ok, because there is a mode where snapshot is created and loaded.
>
> So it shouldn't work properly when I use 'rrsnapshot=' for both
> record and replay?
> Then how can I enable this mode?
>
>>>
 >
 > Hope you've already found the solution (as the last post was on 2 May)
 > and it's just got missed the mailing list.
>>>
>>> As I know, RR is still broken in the current version.
>>> It was caused by the MTTCG implementation.
>>> Alex Bennee tried to fix RR back. Alex, have you found any solution?
>>>
>>> We also trying to find a way to fix RR. It seems, that we will reinvent BQL 
>>> for RR.
>>
>> I think the method outlined in my RFC is the way to go, essentially the
>> RR mutex taking over for the what the BQL did. The RFC patch hadn't
>> hoisted the mutex for the additional devices so I'm just re-basing now
>> and I'll see if I can make the changes for Igor's test case.
>>
>> --
>> Alex Bennée

Could you try:

  https://github.com/stsquad/qemu/tree/bql-and-replay-locks-v2

And report back?

--
Alex Bennée



Re: [Qemu-devel] What is the best commit for record-replay?

2017-09-19 Thread Alexander Bezzubikov

Alex Bennée писал 2017-09-19 17:25:

Aleksandr Bezzubikov  writes:


2017-09-19 12:30 GMT+03:00 Alex Bennée :


Pavel Dovgalyuk  writes:


From: Aleksandr Bezzubikov [mailto:zuban...@gmail.com]
2017-09-18 15:02 GMT+03:00 Aleksandr Bezzubikov 
:

> 2017-05-02 15:42 GMT+03:00 Igor R :
>> I'm trying to use the deterministic record/replay feature, and I would
>> like to know which commit I should take to get it work.
>> In RC0 it seems to be broken. I tried pre-MTTCG commit 2421f381dc, as

> Can you retry with the latest rc? There were some fixes regarding rr 
since rc0.


 I've taken 2.9 release, and RR does not seem to work there.
 I recorded the boot process of x86 Fedora-21 linux and the replay got
 stuck almost immediately.
>>>
>>> What's your command line?
>>>
>>> Does it get stuck at the same place each time?
>>>
>>> Can you boot fine with icount but without record/replay?
>>
>> Here is the exact scenario:
>> - Get 2.9 from git, configure it as follows: "./configure
>> --target-list=i386-softmmu --enable-sdl" and  make.
>> - Download 
https://people.debian.org/~aurel32/qemu/i386/debian_squeeze_i386_standard.qcow2
>> - Run qemu with the following command line, until login prompt:
>> -icount shift=7,rr=record,rrfile=replay.bin -drive
>> file=debian_squeeze_i386_standard.qcow2,if=none,id=img-direct -drive
>> driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
>> ide-hd,drive=img-blkreplay -monitor stdio
>> - Replay: -icount shift=7,rr=replay,rrfile=replay.bin -drive
>> file=debian_squeeze_i386_standard.qcow2,if=none,id=img-direct -drive
>> driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
>> ide-hd,drive=img-blkreplay -monitor stdio
>>
>> Every time I attempt to replay, QEMU gets stuck at the same EIP, at a
>> very early stage.
>>
>>
>>> Can you boot fine with icount but without record/replay?
>>
>> Yes. I can also enable icount and recording - it also boots fine. The
>> problem with the replay.
>
> Hi guys,
> Maybe the thread is a bit outdated, but the problem is still relevant.
> I've just tried to record and replay WinXP boot process, and I've encountered
> exactly the same problem as described above - record is fine, replay
> gets stuck early. I use current master.


Maybe this commit will work: 
cfb2d02be9413d45b30ed6d8e38800250b6b4b48


Unfortunately this one doesn't work either. It seems we need just an
all-in-one fix
for the current implementation to make it work.




> And I've discovered the second problem - recording makes initial snapshot,
> but it doesn't seem to be saved to the disk - replay can't see it.


It is ok, because there is a mode where snapshot is created and 
loaded.


So it shouldn't work properly when I use 'rrsnapshot=' for both
record and replay?
Then how can I enable this mode?




>
> Hope you've already found the solution (as the last post was on 2 May)
> and it's just got missed the mailing list.


As I know, RR is still broken in the current version.
It was caused by the MTTCG implementation.
Alex Bennee tried to fix RR back. Alex, have you found any solution?

We also trying to find a way to fix RR. It seems, that we will 
reinvent BQL for RR.


I think the method outlined in my RFC is the way to go, essentially 
the

RR mutex taking over for the what the BQL did. The RFC patch hadn't
hoisted the mutex for the additional devices so I'm just re-basing 
now

and I'll see if I can make the changes for Igor's test case.

--
Alex Bennée


Could you try:

  https://github.com/stsquad/qemu/tree/bql-and-replay-locks-v2

And report back?


I've encountered 2 new things:
1) Significant performance regression during recording
2) Sometimes I can get through BIOS to the OS boot process itself
   Previously it got stuck in BIOS, the last block I had (I mean with -d 
in_asm) was


IN:
0x7ef6:  pushfl
0x7ef8:  pushal
0x7efa:  mov$0xe,%ah
0x7efc:  xor%bx,%bx
0x7efe:  int$0x10

   So I couldn't even get to the boot process of the OS itself.
   Now it passes to the OS unpredictably, and still not very far. 
Anyway, it doesn't

   reach the point I stopped recording.

So we have a little progress here.




--
Alex Bennée


--
Aleksandr Bezzubikov



Re: [Qemu-devel] What is the best commit for record-replay?

2017-09-19 Thread Pavel Dovgalyuk
> From: Alex Bennée [mailto:alex.ben...@linaro.org]
> >
> >>>
>  >
>  > Hope you've already found the solution (as the last post was on 2 May)
>  > and it's just got missed the mailing list.
> >>>
> >>> As I know, RR is still broken in the current version.
> >>> It was caused by the MTTCG implementation.
> >>> Alex Bennee tried to fix RR back. Alex, have you found any solution?
> >>>
> >>> We also trying to find a way to fix RR. It seems, that we will reinvent 
> >>> BQL for RR.
> >>
> >> I think the method outlined in my RFC is the way to go, essentially the
> >> RR mutex taking over for the what the BQL did. The RFC patch hadn't
> >> hoisted the mutex for the additional devices so I'm just re-basing now
> >> and I'll see if I can make the changes for Igor's test case.
> >>
> >> --
> >> Alex Bennée
> 
> Could you try:
> 
>   https://github.com/stsquad/qemu/tree/bql-and-replay-locks-v2
> 
> And report back?

Most of the code look reasonable.
Isn't better to lock before acting with icount in the following function?
 
static void prepare_icount_for_run(CPUState *cpu)
{
if (use_icount) {
int insns_left;

/* These should always be cleared by process_icount_data after
 * each vCPU execution. However u16.high can be raised
 * asynchronously by cpu_exit/cpu_interrupt/tcg_handle_interrupt
 */
g_assert(cpu->icount_decr.u16.low == 0);
g_assert(cpu->icount_extra == 0);

cpu->icount_budget = tcg_get_icount_limit();
insns_left = MIN(0x, cpu->icount_budget);
cpu->icount_decr.u16.low = insns_left;
cpu->icount_extra = cpu->icount_budget - insns_left;

if (replay_mode != REPLAY_MODE_NONE) {
replay_mutex_lock();
}
}
}



Pavel Dovgalyuk




Re: [Qemu-devel] What is the best commit for record-replay?

2017-04-25 Thread Igor R
>> Hi,
>>
>> I'm trying to use the deterministic record/replay feature, and I would
>> like to know which commit I should take to get it work.
>> In RC0 it seems to be broken. I tried pre-MTTCG commit 2421f381dc, as

> Can you retry with the latest rc? There were some fixes regarding rr since 
> rc0.


I've taken 2.9 release, and RR does not seem to work there.
I recorded the boot process of x86 Fedora-21 linux and the replay got
stuck almost immediately.



Re: [Qemu-devel] What is the best commit for record-replay?

2017-04-26 Thread Alex Bennée

Igor R  writes:

>>> Hi,
>>>
>>> I'm trying to use the deterministic record/replay feature, and I would
>>> like to know which commit I should take to get it work.
>>> In RC0 it seems to be broken. I tried pre-MTTCG commit 2421f381dc, as
>
>> Can you retry with the latest rc? There were some fixes regarding rr since 
>> rc0.
>
>
> I've taken 2.9 release, and RR does not seem to work there.
> I recorded the boot process of x86 Fedora-21 linux and the replay got
> stuck almost immediately.

What's your command line?

Does it get stuck at the same place each time?

Can you boot fine with icount but without record/replay?

--
Alex Bennée



Re: [Qemu-devel] What is the best commit for record-replay?

2017-05-02 Thread Igor R
 I'm trying to use the deterministic record/replay feature, and I would
 like to know which commit I should take to get it work.
 In RC0 it seems to be broken. I tried pre-MTTCG commit 2421f381dc, as
>>
>>> Can you retry with the latest rc? There were some fixes regarding rr since 
>>> rc0.
>>
>>
>> I've taken 2.9 release, and RR does not seem to work there.
>> I recorded the boot process of x86 Fedora-21 linux and the replay got
>> stuck almost immediately.
>
> What's your command line?
>
> Does it get stuck at the same place each time?
>
> Can you boot fine with icount but without record/replay?

Here is the exact scenario:
- Get 2.9 from git, configure it as follows: "./configure
--target-list=i386-softmmu --enable-sdl" and  make.
- Download 
https://people.debian.org/~aurel32/qemu/i386/debian_squeeze_i386_standard.qcow2
- Run qemu with the following command line, until login prompt:
-icount shift=7,rr=record,rrfile=replay.bin -drive
file=debian_squeeze_i386_standard.qcow2,if=none,id=img-direct -drive
driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
ide-hd,drive=img-blkreplay -monitor stdio
- Replay: -icount shift=7,rr=replay,rrfile=replay.bin -drive
file=debian_squeeze_i386_standard.qcow2,if=none,id=img-direct -drive
driver=blkreplay,if=none,image=img-direct,id=img-blkreplay -device
ide-hd,drive=img-blkreplay -monitor stdio

Every time I attempt to replay, QEMU gets stuck at the same EIP, at a
very early stage.


> Can you boot fine with icount but without record/replay?

Yes. I can also enable icount and recording - it also boots fine. The
problem with the replay.



Re: [Qemu-devel] What is the best commit for record-replay?

2017-10-31 Thread Pavel Dovgalyuk
> From: Alex Bennée [mailto:alex.ben...@linaro.org]
> Aleksandr Bezzubikov  writes:
> 
> > 2017-09-19 12:30 GMT+03:00 Alex Bennée :
> >>
> >>> As I know, RR is still broken in the current version.
> >>> It was caused by the MTTCG implementation.
> >>> Alex Bennee tried to fix RR back. Alex, have you found any solution?
> >>>
> >>> We also trying to find a way to fix RR. It seems, that we will reinvent 
> >>> BQL for RR.
> >>
> >> I think the method outlined in my RFC is the way to go, essentially the
> >> RR mutex taking over for the what the BQL did. The RFC patch hadn't
> >> hoisted the mutex for the additional devices so I'm just re-basing now
> >> and I'll see if I can make the changes for Igor's test case.
> >>
> >> --
> >> Alex Bennée
> 
> Could you try:
> 
>   https://github.com/stsquad/qemu/tree/bql-and-replay-locks-v2
> 
> And report back?

Please check the new series of the patches.
It includes patches from Alex and some fixes that made them working.

Pavel Dovgalyuk




Re: [Qemu-devel] What is the best commit for record-replay?

2017-10-31 Thread Alex Bennée

Pavel Dovgalyuk  writes:

>> From: Alex Bennée [mailto:alex.ben...@linaro.org]
>> Aleksandr Bezzubikov  writes:
>>
>> > 2017-09-19 12:30 GMT+03:00 Alex Bennée :
>> >>
>> >>> As I know, RR is still broken in the current version.
>> >>> It was caused by the MTTCG implementation.
>> >>> Alex Bennee tried to fix RR back. Alex, have you found any solution?
>> >>>
>> >>> We also trying to find a way to fix RR. It seems, that we will reinvent 
>> >>> BQL for RR.
>> >>
>> >> I think the method outlined in my RFC is the way to go, essentially the
>> >> RR mutex taking over for the what the BQL did. The RFC patch hadn't
>> >> hoisted the mutex for the additional devices so I'm just re-basing now
>> >> and I'll see if I can make the changes for Igor's test case.
>> >>
>> >> --
>> >> Alex Bennée
>>
>> Could you try:
>>
>>   https://github.com/stsquad/qemu/tree/bql-and-replay-locks-v2
>>
>> And report back?
>
> Please check the new series of the patches.
> It includes patches from Alex and some fixes that made them working.

For reference that is:

  Subject: [RFC PATCH 00/26] replay additions
  From: Pavel Dovgalyuk 
  Date: Tue, 31 Oct 2017 14:24:57 +0300
  Message-ID: <20171031112457.10516.8971.stgit@pasha-VirtualBox>

--
Alex Bennée



Re: [Qemu-devel] What is the best commit for record-replay?

2017-10-31 Thread Pavel Dovgalyuk
> From: Alex Bennée [mailto:alex.ben...@linaro.org]
> Pavel Dovgalyuk  writes:
> 
> >> From: Alex Bennée [mailto:alex.ben...@linaro.org]
> >> Aleksandr Bezzubikov  writes:
> >>
> >> > 2017-09-19 12:30 GMT+03:00 Alex Bennée :
> >> >>
> >> >>> As I know, RR is still broken in the current version.
> >> >>> It was caused by the MTTCG implementation.
> >> >>> Alex Bennee tried to fix RR back. Alex, have you found any solution?
> >> >>>
> >> >>> We also trying to find a way to fix RR. It seems, that we will 
> >> >>> reinvent BQL for RR.
> >> >>
> >> >> I think the method outlined in my RFC is the way to go, essentially the
> >> >> RR mutex taking over for the what the BQL did. The RFC patch hadn't
> >> >> hoisted the mutex for the additional devices so I'm just re-basing now
> >> >> and I'll see if I can make the changes for Igor's test case.
> >> >>
> >> >> --
> >> >> Alex Bennée
> >>
> >> Could you try:
> >>
> >>   https://github.com/stsquad/qemu/tree/bql-and-replay-locks-v2
> >>
> >> And report back?
> >
> > Please check the new series of the patches.
> > It includes patches from Alex and some fixes that made them working.
> 
> For reference that is:
> 
>   Subject: [RFC PATCH 00/26] replay additions
>   From: Pavel Dovgalyuk 
>   Date: Tue, 31 Oct 2017 14:24:57 +0300
>   Message-ID: <20171031112457.10516.8971.stgit@pasha-VirtualBox>

That's right.

Pavel Dovgalyuk




Re: [Qemu-devel] What is the best commit for record-replay?

2017-04-08 Thread Pranith Kumar
On Thu, Mar 23, 2017 at 4:05 AM, Igor R  wrote:
> Hi,
>
> I'm trying to use the deterministic record/replay feature, and I would
> like to know which commit I should take to get it work.
> In RC0 it seems to be broken. I tried pre-MTTCG commit 2421f381dc, as

Can you retry with the latest rc? There were some fixes regarding rr since rc0.

Thanks,
--
Pranith