Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-15 Thread Thomas Gleixner
On Tue, Dec 15 2020 at 07:59, Marcelo Tosatti wrote: > On Fri, Dec 11, 2020 at 10:59:59PM +0100, Paolo Bonzini wrote: >> So it's still true that the time advances during live migration brownout; >> 0.1 seconds is just the final part of the live migration process. But for >> _live_ migration there

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-15 Thread Andy Lutomirski
On Tue, Dec 15, 2020 at 3:35 AM Marcelo Tosatti wrote: > > On Fri, Dec 11, 2020 at 10:59:59PM +0100, Paolo Bonzini wrote: > > On 11/12/20 22:04, Thomas Gleixner wrote: > > > > Its 100ms off with migration, and can be reduced further (customers > > > > complained about 5 seconds but seem happy

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-15 Thread Marcelo Tosatti
On Fri, Dec 11, 2020 at 10:59:59PM +0100, Paolo Bonzini wrote: > On 11/12/20 22:04, Thomas Gleixner wrote: > > > Its 100ms off with migration, and can be reduced further (customers > > > complained about 5 seconds but seem happy with 0.1ms). > > What is 100ms? Guaranteed maximum migration time? >

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-12 Thread Thomas Gleixner
On Fri, Dec 11 2020 at 22:59, Paolo Bonzini wrote: > On 11/12/20 22:04, Thomas Gleixner wrote: > The nanosecond and TSC times are sent as part of the paused-VM state at > the very end of the live migration process. > > So it's still true that the time advances during live migration > brownout;

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-11 Thread Paolo Bonzini
On 11/12/20 22:04, Thomas Gleixner wrote: Its 100ms off with migration, and can be reduced further (customers complained about 5 seconds but seem happy with 0.1ms). What is 100ms? Guaranteed maximum migration time? I suppose it's the length between the time from KVM_GET_CLOCK and

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-11 Thread Thomas Gleixner
On Fri, Dec 11 2020 at 11:18, Marcelo Tosatti wrote: > On Fri, Dec 11, 2020 at 02:30:34PM +0100, Thomas Gleixner wrote: > Unless you notify applications to invalidate their time reads, > i can't see a way to fix this. This is just wrong. Suspend/resume handles that fine and the system is

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-11 Thread Marcelo Tosatti
On Fri, Dec 11, 2020 at 02:30:34PM +0100, Thomas Gleixner wrote: > On Thu, Dec 10 2020 at 21:27, Marcelo Tosatti wrote: > > On Thu, Dec 10, 2020 at 10:48:10PM +0100, Thomas Gleixner wrote: > >> You really all live in a seperate universe creating your own rules how > >> things which other people

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-11 Thread Paolo Bonzini
On 11/12/20 01:27, Marcelo Tosatti wrote: This features first, correctness later frenzy is insane and it better stops now before you pile even more crap on the existing steaming pile of insanities. FWIW I agree, in fact I wish I knew exactly where to start simplifying it; the timekeeping code

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-11 Thread Thomas Gleixner
On Thu, Dec 10 2020 at 21:27, Marcelo Tosatti wrote: > On Thu, Dec 10, 2020 at 10:48:10PM +0100, Thomas Gleixner wrote: >> You really all live in a seperate universe creating your own rules how >> things which other people work hard on to get it correct can be screwed >> over. > > 1. T =

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-10 Thread Marcelo Tosatti
On Thu, Dec 10, 2020 at 10:48:10PM +0100, Thomas Gleixner wrote: > On Thu, Dec 10 2020 at 12:26, Marcelo Tosatti wrote: > > On Wed, Dec 09, 2020 at 09:58:23PM +0100, Thomas Gleixner wrote: > >> Marcelo, > >> > >> On Wed, Dec 09 2020 at 13:34, Marcelo Tosatti wrote: > >> > On Tue, Dec 08, 2020 at

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-10 Thread Thomas Gleixner
On Thu, Dec 10 2020 at 15:19, Andy Lutomirski wrote: >> On Dec 10, 2020, at 2:28 PM, Thomas Gleixner wrote: >> Can we please focus on real problems instead of making up new ones? >> >> Correctness of time is a real problem despite the believe of virt folks >> that it can be ignored or duct taped

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-10 Thread Andy Lutomirski
> On Dec 9, 2020, at 2:14 AM, Thomas Gleixner wrote: > > But what's more problematic is the basic requirement that time all over > the place has to be consistent. > > On machines which use DISTORTED_REALTIME everything _IS_ consistent > within the distorted universe they created. It's still

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-10 Thread Andy Lutomirski
> On Dec 10, 2020, at 2:28 PM, Thomas Gleixner wrote: > > On Thu, Dec 10 2020 at 14:01, Andy Lutomirski wrote: >>> On Thu, Dec 10, 2020 at 1:25 PM Thomas Gleixner wrote: >>> I'm still convinced that a notification about 'we take a nap' will be >>> more robust, less complex and more trivial

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-10 Thread Thomas Gleixner
On Thu, Dec 10 2020 at 12:26, Marcelo Tosatti wrote: > On Wed, Dec 09, 2020 at 09:58:23PM +0100, Thomas Gleixner wrote: >> Marcelo, >> >> On Wed, Dec 09 2020 at 13:34, Marcelo Tosatti wrote: >> > On Tue, Dec 08, 2020 at 10:33:15PM +0100, Thomas Gleixner wrote: >> >> On Tue, Dec 08 2020 at 15:11,

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-10 Thread Thomas Gleixner
On Thu, Dec 10 2020 at 14:01, Andy Lutomirski wrote: > On Thu, Dec 10, 2020 at 1:25 PM Thomas Gleixner wrote: >> I'm still convinced that a notification about 'we take a nap' will be >> more robust, less complex and more trivial to backport. > > What do you have in mind? Suppose the host kernel

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-10 Thread Andy Lutomirski
On Thu, Dec 10, 2020 at 1:25 PM Thomas Gleixner wrote: > > Andy, > > > I'm still convinced that a notification about 'we take a nap' will be > more robust, less complex and more trivial to backport. What do you have in mind? Suppose the host kernel sends the guest an interrupt on all vCPUs

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-10 Thread Thomas Gleixner
Andy, On Thu, Dec 10 2020 at 07:16, Andy Lutomirski wrote: >> On Dec 10, 2020, at 6:52 AM, Maxim Levitsky wrote: >> On Thu, 2020-12-10 at 12:48 +0100, Paolo Bonzini wrote: On 08/12/20 22:20, Thomas Gleixner wrote: So now life migration comes a long time after timekeeping had set the

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-10 Thread Thomas Gleixner
On Thu, Dec 10 2020 at 14:01, Peter Zijlstra wrote: > On Thu, Dec 10, 2020 at 01:22:02PM +0100, Paolo Bonzini wrote: >> On 10/12/20 13:14, Peter Zijlstra wrote: >> > On Thu, Dec 10, 2020 at 12:42:36PM +0100, Paolo Bonzini wrote: >> > > On 07/12/20 18:41, Thomas Gleixner wrote: >> > > > Right this

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-10 Thread Maxim Levitsky
On Thu, 2020-12-10 at 12:48 +0100, Paolo Bonzini wrote: > On 08/12/20 18:08, Maxim Levitsky wrote: > > > Even if you support TSCADJUST and let the guest write to it does not > > > change the per guest offset at all. TSCADJUST is per [v]CPU and adds on > > > top: > > > > > > tscvcpu =

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-10 Thread Oliver Upton
On Thu, Dec 10, 2020 at 12:05 PM Paolo Bonzini wrote: > > On 10/12/20 18:59, Oliver Upton wrote: > > However, I don't believe we can assume the guest's TSCs to be synchronized, > > even if sane guests will never touch them. In this case, I think a per-vCPU > > ioctl is still warranted, allowing

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-10 Thread Paolo Bonzini
On 10/12/20 18:59, Oliver Upton wrote: However, I don't believe we can assume the guest's TSCs to be synchronized, even if sane guests will never touch them. In this case, I think a per-vCPU ioctl is still warranted, allowing userspace to get at the guest CPU adjust component of Thomas' equation

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-10 Thread Oliver Upton
On Thu, Dec 10, 2020 at 9:16 AM Andy Lutomirski wrote: > > > > > On Dec 10, 2020, at 6:52 AM, Maxim Levitsky wrote: > > > > On Thu, 2020-12-10 at 12:48 +0100, Paolo Bonzini wrote: > >>> On 08/12/20 22:20, Thomas Gleixner wrote: > >>> So now life migration comes a long time after timekeeping had

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-10 Thread Marcelo Tosatti
On Wed, Dec 09, 2020 at 09:58:23PM +0100, Thomas Gleixner wrote: > Marcelo, > > On Wed, Dec 09 2020 at 13:34, Marcelo Tosatti wrote: > > On Tue, Dec 08, 2020 at 10:33:15PM +0100, Thomas Gleixner wrote: > >> On Tue, Dec 08 2020 at 15:11, Marcelo Tosatti wrote: > >> > max_cycles overflow. Sent a

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-10 Thread Andy Lutomirski
> On Dec 10, 2020, at 6:52 AM, Maxim Levitsky wrote: > > On Thu, 2020-12-10 at 12:48 +0100, Paolo Bonzini wrote: >>> On 08/12/20 22:20, Thomas Gleixner wrote: >>> So now life migration comes a long time after timekeeping had set the >>> limits and just because it's virt it expects that

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-10 Thread Maxim Levitsky
On Thu, 2020-12-10 at 12:48 +0100, Paolo Bonzini wrote: > On 08/12/20 22:20, Thomas Gleixner wrote: > > So now life migration comes a long time after timekeeping had set the > > limits and just because it's virt it expects that everything works and it > > just can ignore these limits. > > > >

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-10 Thread Peter Zijlstra
On Thu, Dec 10, 2020 at 01:22:02PM +0100, Paolo Bonzini wrote: > On 10/12/20 13:14, Peter Zijlstra wrote: > > On Thu, Dec 10, 2020 at 12:42:36PM +0100, Paolo Bonzini wrote: > > > On 07/12/20 18:41, Thomas Gleixner wrote: > > > > Right this happens still occasionally, but for quite some time this

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-10 Thread Paolo Bonzini
On 10/12/20 13:14, Peter Zijlstra wrote: On Thu, Dec 10, 2020 at 12:42:36PM +0100, Paolo Bonzini wrote: On 07/12/20 18:41, Thomas Gleixner wrote: Right this happens still occasionally, but for quite some time this is 100% firmware sillyness and not a fundamental property of the hardware

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-10 Thread Peter Zijlstra
On Thu, Dec 10, 2020 at 12:42:36PM +0100, Paolo Bonzini wrote: > On 07/12/20 18:41, Thomas Gleixner wrote: > > Right this happens still occasionally, but for quite some time this is > > 100% firmware sillyness and not a fundamental property of the hardware > > anymore. > > It's still a

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-10 Thread Paolo Bonzini
On 08/12/20 18:08, Maxim Levitsky wrote: Even if you support TSCADJUST and let the guest write to it does not change the per guest offset at all. TSCADJUST is per [v]CPU and adds on top: tscvcpu = tsc_host + guest_offset + TSC_ADJUST Scaling is just orthogonal and does not change any of

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-10 Thread Paolo Bonzini
On 08/12/20 22:20, Thomas Gleixner wrote: So now life migration comes a long time after timekeeping had set the limits and just because it's virt it expects that everything works and it just can ignore these limits. TBH. That's not any different than SMM or hard/firmware taking the machine out

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-10 Thread Paolo Bonzini
On 07/12/20 18:41, Thomas Gleixner wrote: Right this happens still occasionally, but for quite some time this is 100% firmware sillyness and not a fundamental property of the hardware anymore. It's still a fundamental property of old hardware. Last time I tried to kill support for processors

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-09 Thread Thomas Gleixner
Marcelo, On Wed, Dec 09 2020 at 13:34, Marcelo Tosatti wrote: > On Tue, Dec 08, 2020 at 10:33:15PM +0100, Thomas Gleixner wrote: >> On Tue, Dec 08 2020 at 15:11, Marcelo Tosatti wrote: >> > max_cycles overflow. Sent a message to Maxim describing it. >> >> Truly helpful. Why the hell did you not

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-09 Thread Marcelo Tosatti
On Tue, Dec 08, 2020 at 10:33:15PM +0100, Thomas Gleixner wrote: > On Tue, Dec 08 2020 at 15:11, Marcelo Tosatti wrote: > > On Tue, Dec 08, 2020 at 05:02:07PM +0100, Thomas Gleixner wrote: > >> On Tue, Dec 08 2020 at 16:50, Maxim Levitsky wrote: > >> > On Mon, 2020-12-07 at 20:29 -0300, Marcelo

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-09 Thread Thomas Gleixner
Andy, On Tue, Dec 08 2020 at 20:08, Andy Lutomirski wrote: > On Tue, Dec 8, 2020 at 4:19 PM Thomas Gleixner wrote: >> On Tue, Dec 08 2020 at 12:32, Andy Lutomirski wrote: >> all the way through the end and then come up with a real proposal which >> solves all of the issues mentioned there. > >

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-08 Thread Andy Lutomirski
On Tue, Dec 8, 2020 at 4:19 PM Thomas Gleixner wrote: > > On Tue, Dec 08 2020 at 12:32, Andy Lutomirski wrote: > >> On Dec 8, 2020, at 11:25 AM, Thomas Gleixner wrote: > >> One issue here is that guests might want to run their own NTP/PTP. One > >> reason to do that is that some people prefer

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-08 Thread Thomas Gleixner
On Tue, Dec 08 2020 at 12:32, Andy Lutomirski wrote: >> On Dec 8, 2020, at 11:25 AM, Thomas Gleixner wrote: >> One issue here is that guests might want to run their own NTP/PTP. One >> reason to do that is that some people prefer the leap second smearing >> NTP servers. > > I would hope that

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-08 Thread Thomas Gleixner
On Tue, Dec 08 2020 at 15:12, Marcelo Tosatti wrote: > On Tue, Dec 08, 2020 at 06:25:13PM +0200, Maxim Levitsky wrote: >> On Tue, 2020-12-08 at 17:02 +0100, Thomas Gleixner wrote: >> The "bug" is that if VMM moves a hardware time counter (tsc or anything >> else) >> forward by large enough value

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-08 Thread Thomas Gleixner
On Tue, Dec 08 2020 at 15:11, Marcelo Tosatti wrote: > On Tue, Dec 08, 2020 at 05:02:07PM +0100, Thomas Gleixner wrote: >> On Tue, Dec 08 2020 at 16:50, Maxim Levitsky wrote: >> > On Mon, 2020-12-07 at 20:29 -0300, Marcelo Tosatti wrote: >> >> > +This ioctl allows to reconstruct the guest's

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-08 Thread Thomas Gleixner
On Tue, Dec 08 2020 at 09:33, Andy Lutomirski wrote: > On Tue, Dec 8, 2020 at 8:26 AM Maxim Levitsky wrote: > > IIRC we introduced mul_u64_u32_shift() for roughly this reason,but we > don't seem to be using it in the relevant code paths. We should be > able to use the same basic math with wider

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-08 Thread Thomas Gleixner
On Tue, Dec 08 2020 at 18:25, Maxim Levitsky wrote: > On Tue, 2020-12-08 at 17:02 +0100, Thomas Gleixner wrote: >> For one I have no idea which bug you are talking about and if the bug is >> caused by the VMM then why would you "fix" it in the guest kernel. > > The "bug" is that if VMM moves a

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-08 Thread Andy Lutomirski
> On Dec 8, 2020, at 11:25 AM, Thomas Gleixner wrote: > > On Tue, Dec 08 2020 at 09:43, Andy Lutomirski wrote: >> On Tue, Dec 8, 2020 at 6:23 AM Marcelo Tosatti wrote: >> It looks like it tries to accomplish the right goal, but in a rather >> roundabout way. The host knows how to convert

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-08 Thread Thomas Gleixner
On Tue, Dec 08 2020 at 09:43, Andy Lutomirski wrote: > On Tue, Dec 8, 2020 at 6:23 AM Marcelo Tosatti wrote: > It looks like it tries to accomplish the right goal, but in a rather > roundabout way. The host knows how to convert from TSC to > CLOCK_REALTIME, and ptp_kvm.c exposes this data to the

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-08 Thread Marcelo Tosatti
On Tue, Dec 08, 2020 at 06:25:13PM +0200, Maxim Levitsky wrote: > On Tue, 2020-12-08 at 17:02 +0100, Thomas Gleixner wrote: > > On Tue, Dec 08 2020 at 16:50, Maxim Levitsky wrote: > > > On Mon, 2020-12-07 at 20:29 -0300, Marcelo Tosatti wrote: > > > > > +This ioctl allows to reconstruct the

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-08 Thread Marcelo Tosatti
On Tue, Dec 08, 2020 at 05:02:07PM +0100, Thomas Gleixner wrote: > On Tue, Dec 08 2020 at 16:50, Maxim Levitsky wrote: > > On Mon, 2020-12-07 at 20:29 -0300, Marcelo Tosatti wrote: > >> > +This ioctl allows to reconstruct the guest's IA32_TSC and TSC_ADJUST > >> > value > >> > +from the state

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-08 Thread Andy Lutomirski
On Tue, Dec 8, 2020 at 6:23 AM Marcelo Tosatti wrote: > > On Mon, Dec 07, 2020 at 10:04:45AM -0800, Andy Lutomirski wrote: > > > > > > I do have a feature request, though: IMO it would be quite nifty if the new > > kvmclock structure could also expose NTP corrections. In other words, if > > you

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-08 Thread Marcelo Tosatti
On Tue, Dec 08, 2020 at 04:50:53PM +0200, Maxim Levitsky wrote: > On Mon, 2020-12-07 at 20:29 -0300, Marcelo Tosatti wrote: > > On Thu, Dec 03, 2020 at 07:11:16PM +0200, Maxim Levitsky wrote: > > > These two new ioctls allow to more precisly capture and > > > restore guest's TSC state. > > > > >

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-08 Thread Andy Lutomirski
On Tue, Dec 8, 2020 at 8:26 AM Maxim Levitsky wrote: > > On Tue, 2020-12-08 at 17:02 +0100, Thomas Gleixner wrote: > > On Tue, Dec 08 2020 at 16:50, Maxim Levitsky wrote: > > > On Mon, 2020-12-07 at 20:29 -0300, Marcelo Tosatti wrote: > > > > > +This ioctl allows to reconstruct the guest's

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-08 Thread Maxim Levitsky
On Tue, 2020-12-08 at 09:58 -0600, Oliver Upton wrote: > +cc Sean's new handle > > On Tue, Dec 8, 2020 at 9:57 AM Oliver Upton wrote: > > On Tue, Dec 8, 2020 at 5:13 AM Maxim Levitsky wrote: > > > On Mon, 2020-12-07 at 11:29 -0600, Oliver Upton wrote: > > > > On Thu, Dec 3, 2020 at 11:12 AM

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-08 Thread Maxim Levitsky
On Tue, 2020-12-08 at 17:40 +0100, Thomas Gleixner wrote: > On Tue, Dec 08 2020 at 13:13, Maxim Levitsky wrote: > > On Mon, 2020-12-07 at 11:29 -0600, Oliver Upton wrote: > > > How would a VMM maintain the phase relationship between guest TSCs > > > using these ioctls? > > > > By using the

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-08 Thread Thomas Gleixner
On Tue, Dec 08 2020 at 13:13, Maxim Levitsky wrote: > On Mon, 2020-12-07 at 11:29 -0600, Oliver Upton wrote: >> >> How would a VMM maintain the phase relationship between guest TSCs >> using these ioctls? > > By using the nanosecond timestamp. > > While I did made it optional in the V2 it was

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-08 Thread Maxim Levitsky
On Tue, 2020-12-08 at 17:02 +0100, Thomas Gleixner wrote: > On Tue, Dec 08 2020 at 16:50, Maxim Levitsky wrote: > > On Mon, 2020-12-07 at 20:29 -0300, Marcelo Tosatti wrote: > > > > +This ioctl allows to reconstruct the guest's IA32_TSC and TSC_ADJUST > > > > value > > > > +from the state

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-08 Thread Thomas Gleixner
On Tue, Dec 08 2020 at 16:50, Maxim Levitsky wrote: > On Mon, 2020-12-07 at 20:29 -0300, Marcelo Tosatti wrote: >> > +This ioctl allows to reconstruct the guest's IA32_TSC and TSC_ADJUST value >> > +from the state obtained in the past by KVM_GET_TSC_STATE on the same vCPU. >> > + >> > +If

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-08 Thread Oliver Upton
+cc Sean's new handle On Tue, Dec 8, 2020 at 9:57 AM Oliver Upton wrote: > > On Tue, Dec 8, 2020 at 5:13 AM Maxim Levitsky wrote: > > > > On Mon, 2020-12-07 at 11:29 -0600, Oliver Upton wrote: > > > On Thu, Dec 3, 2020 at 11:12 AM Maxim Levitsky > > > wrote: > > > > These two new ioctls allow

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-08 Thread Oliver Upton
On Tue, Dec 8, 2020 at 5:13 AM Maxim Levitsky wrote: > > On Mon, 2020-12-07 at 11:29 -0600, Oliver Upton wrote: > > On Thu, Dec 3, 2020 at 11:12 AM Maxim Levitsky wrote: > > > These two new ioctls allow to more precisly capture and > > > restore guest's TSC state. > > > > > > Both ioctls are

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-08 Thread Maxim Levitsky
On Mon, 2020-12-07 at 20:29 -0300, Marcelo Tosatti wrote: > On Thu, Dec 03, 2020 at 07:11:16PM +0200, Maxim Levitsky wrote: > > These two new ioctls allow to more precisly capture and > > restore guest's TSC state. > > > > Both ioctls are meant to be used to accurately migrate guest TSC > > even

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-08 Thread Marcelo Tosatti
On Mon, Dec 07, 2020 at 10:04:45AM -0800, Andy Lutomirski wrote: > > > On Dec 7, 2020, at 9:00 AM, Maxim Levitsky wrote: > > > > On Mon, 2020-12-07 at 08:53 -0800, Andy Lutomirski wrote: > On Dec 7, 2020, at 8:38 AM, Thomas Gleixner wrote: > >>> > >>> On Mon, Dec 07 2020 at 14:16,

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-08 Thread Marcelo Tosatti
On Thu, Dec 03, 2020 at 07:11:16PM +0200, Maxim Levitsky wrote: > These two new ioctls allow to more precisly capture and > restore guest's TSC state. > > Both ioctls are meant to be used to accurately migrate guest TSC > even when there is a significant downtime during the migration. > >

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-08 Thread Marcelo Tosatti
On Sun, Dec 06, 2020 at 05:19:16PM +0100, Thomas Gleixner wrote: > On Thu, Dec 03 2020 at 19:11, Maxim Levitsky wrote: > > + case KVM_SET_TSC_STATE: { > > + struct kvm_tsc_state __user *user_tsc_state = argp; > > + struct kvm_tsc_state tsc_state; > > + u64 host_tsc,

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-08 Thread Maxim Levitsky
On Mon, 2020-12-07 at 10:04 -0800, Andy Lutomirski wrote: > > On Dec 7, 2020, at 9:00 AM, Maxim Levitsky wrote: > > > > On Mon, 2020-12-07 at 08:53 -0800, Andy Lutomirski wrote: > > > > > On Dec 7, 2020, at 8:38 AM, Thomas Gleixner > > > > > wrote: > > > > > > > > On Mon, Dec 07 2020 at

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-08 Thread Maxim Levitsky
On Mon, 2020-12-07 at 11:29 -0600, Oliver Upton wrote: > On Thu, Dec 3, 2020 at 11:12 AM Maxim Levitsky wrote: > > These two new ioctls allow to more precisly capture and > > restore guest's TSC state. > > > > Both ioctls are meant to be used to accurately migrate guest TSC > > even when there

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-08 Thread Peter Zijlstra
On Mon, Dec 07, 2020 at 06:41:41PM +0100, Thomas Gleixner wrote: > Right this happens still occasionally, but for quite some time this is > 100% firmware sillyness and not a fundamental property of the hardware > anymore. Ever since Nehalem (2008) TSC is synchronized on <= 2 sockets, and any

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-08 Thread Peter Zijlstra
On Mon, Dec 07, 2020 at 05:38:36PM +0100, Thomas Gleixner wrote: > For anything halfways modern the write to TSC is reflected in TSC_ADJUST > which means you get the precise offset. IIRC this is true for everything that has TSC_ADJUST.

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-07 Thread Andy Lutomirski
> On Dec 7, 2020, at 9:00 AM, Maxim Levitsky wrote: > > On Mon, 2020-12-07 at 08:53 -0800, Andy Lutomirski wrote: On Dec 7, 2020, at 8:38 AM, Thomas Gleixner wrote: >>> >>> On Mon, Dec 07 2020 at 14:16, Maxim Levitsky wrote: > On Sun, 2020-12-06 at 17:19 +0100, Thomas Gleixner

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-07 Thread Thomas Gleixner
On Mon, Dec 07 2020 at 14:16, Vitaly Kuznetsov wrote: > Maxim Levitsky writes: >> But other than that I don't mind making TSC offset global per VM thing. >> Paulo, what do you think about this? >> > > Not Paolo here but personally I'd very much prefer we go this route but > unsynchronized TSCs

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-07 Thread Oliver Upton
On Thu, Dec 3, 2020 at 11:12 AM Maxim Levitsky wrote: > > These two new ioctls allow to more precisly capture and > restore guest's TSC state. > > Both ioctls are meant to be used to accurately migrate guest TSC > even when there is a significant downtime during the migration. > > Suggested-by:

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-07 Thread Maxim Levitsky
On Mon, 2020-12-07 at 08:53 -0800, Andy Lutomirski wrote: > > On Dec 7, 2020, at 8:38 AM, Thomas Gleixner wrote: > > > > On Mon, Dec 07 2020 at 14:16, Maxim Levitsky wrote: > > > > On Sun, 2020-12-06 at 17:19 +0100, Thomas Gleixner wrote: > > > > From a timekeeping POV and the guests

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-07 Thread Andy Lutomirski
> On Dec 7, 2020, at 8:38 AM, Thomas Gleixner wrote: > > On Mon, Dec 07 2020 at 14:16, Maxim Levitsky wrote: >>> On Sun, 2020-12-06 at 17:19 +0100, Thomas Gleixner wrote: >>> From a timekeeping POV and the guests expectation of TSC this is >>> fundamentally wrong: >>> >>> tscguest =

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-07 Thread Thomas Gleixner
On Mon, Dec 07 2020 at 14:16, Maxim Levitsky wrote: > On Sun, 2020-12-06 at 17:19 +0100, Thomas Gleixner wrote: >> From a timekeeping POV and the guests expectation of TSC this is >> fundamentally wrong: >> >> tscguest = scaled(hosttsc) + offset >> >> The TSC has to be viewed systemwide

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-07 Thread Vitaly Kuznetsov
Maxim Levitsky writes: > > But other than that I don't mind making TSC offset global per VM thing. > Paulo, what do you think about this? > Not Paolo here but personally I'd very much prefer we go this route but unsynchronized TSCs are, unfortunately, still a thing: I was observing it on an AMD

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-07 Thread Maxim Levitsky
On Sun, 2020-12-06 at 17:19 +0100, Thomas Gleixner wrote: > On Thu, Dec 03 2020 at 19:11, Maxim Levitsky wrote: > > + case KVM_SET_TSC_STATE: { > > + struct kvm_tsc_state __user *user_tsc_state = argp; > > + struct kvm_tsc_state tsc_state; > > + u64 host_tsc,

Re: [PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-06 Thread Thomas Gleixner
On Thu, Dec 03 2020 at 19:11, Maxim Levitsky wrote: > + case KVM_SET_TSC_STATE: { > + struct kvm_tsc_state __user *user_tsc_state = argp; > + struct kvm_tsc_state tsc_state; > + u64 host_tsc, wall_nsec; > + > + u64 new_guest_tsc,

[PATCH v2 1/3] KVM: x86: implement KVM_{GET|SET}_TSC_STATE

2020-12-03 Thread Maxim Levitsky
These two new ioctls allow to more precisly capture and restore guest's TSC state. Both ioctls are meant to be used to accurately migrate guest TSC even when there is a significant downtime during the migration. Suggested-by: Paolo Bonzini Signed-off-by: Maxim Levitsky ---