Jenkins.impala.io back up

Everything went as planned and Jenkins is now upgraded. - Zach

Heads Up - Jenkins shut down.

I am going to upgrade jenkins.impala.io to address a security advisory; no GVOs will run for a bit. Hopefully I should be done in 30 minutes or less. - Zach

Re: Review Request 64317: SENTRY-2085: Keep sentry exceptions contained within Sentry

ive/TestUserManagement.java 02ac514 Diff: https://reviews.apache.org/r/64317/diff/4/ Changes: https://reviews.apache.org/r/64317/diff/3-4/ Testing --- Running JUnit tests with mvn install. Thanks, Zachary Amsden

Re: Review Request 64317: SENTRY-2085: Keep sentry exceptions contained within Sentry

/diff/3/ Changes: https://reviews.apache.org/r/64317/diff/2-3/ Testing --- Running JUnit tests with mvn install. Thanks, Zachary Amsden

Re: Review Request 64317: SENTRY-2085: Keep sentry exceptions contained within Sentry

317/diff/2/?file=1908499#file1908499line838> > > > > This is not the appropriate error that should be logged. > > > > SemanticException is not not right exception, please throw appropriate > > exception and handle it. > > Zachary Amsden wrote: > I'm

Re: Review Request 64317: SENTRY-2085: Keep sentry exceptions contained within Sentry

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64317/#review194177 --- On Dec. 5, 2017, 12:55 a.m., Zachary Amsden wrote: > > -

Re: Review Request 64317: SENTRY-2085: Keep sentry exceptions contained within Sentry

org/r/64317/diff/2/?file=1908513#file1908513line74> > > > > We don't need this Ditto. - Zachary --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64317/#review1

Re: Review Request 64317: SENTRY-2085: Keep sentry exceptions contained within Sentry

bleHierarchy = ImmutableList.of(server1, uri); Assert.assertTrue(authorizableHierarchy.toString(), authzProvider.hasAccess(user1, authorizableHierarchy, actions, ActiveRoleSet.ALL)); } ``` - Zachary --- This is an automatically ge

Re: Review Request 64317: SENTRY-2085: Keep sentry exceptions contained within Sentry

://reviews.apache.org/r/64317/diff/2/ Changes: https://reviews.apache.org/r/64317/diff/1-2/ Testing --- Running JUnit tests with mvn install. Thanks, Zachary Amsden

Re: Review Request 64317: SENTRY-2085: Keep sentry exceptions contained within Sentry

e/hive/TestUserManagement.java Line 383 (original) <https://reviews.apache.org/r/64317/#comment271086> Missing the same change for sentry-tests-hive - Zachary Amsden On Dec. 4, 2017, 11:30 p.m., Zachary Amsden wrote: > > --- > Thi

Review Request 64317: SENTRY-2085: Keep sentry exceptions contained within Sentry

/sentry/tests/e2e/hive/TestUserManagement.java 02ac51454a13c0c1c61bb8684872e4815bd88b97 Diff: https://reviews.apache.org/r/64317/diff/1/ Testing --- Running JUnit tests with mvn install. Thanks, Zachary Amsden

IMPALA-5607

In discussion about IMPALA-5607, it came up that implementing this as described is a compatibility breaking change. There are a couple of required changes: 1) Return data type for date_part and EXTRACT FROM must be promoted to BIGINT. 2) MILLISECONDS now will include seconds part in the

Re: [VOTE] Graduate to a TLP

+1 On Wed, Oct 18, 2017 at 11:35 AM, David Knupp wrote: > +1 > >> Daniel Hecht >> October 18, 2017 at 11:32 AM >> +1 >> Anuj Phadke >> October 18, 2017 at 9:23 AM >> +1 >> >> On Wed, Oct 18, 2017 at 9:17 AM Joe

Re: Build broken by b66af0357e - IMPALA-5854: Update external hadoop versions

If this fails for you, wiping out toolchain/cdh_components and restarting the build should fix the problem. If not, ping me! - Zach On Fri, Sep 1, 2017 at 10:19 AM, Tim Armstrong wrote: > It works ok for me. Maybe you need to re-source impala-config.sh or >

Re: Making a new development environment from scratch

Nice! I had to hack around this as well to get a different Java version set up. It may be worth making the Java version a configurable parameter. - Zach On Fri, Aug 11, 2017 at 10:16 AM, Jim Apple wrote: > bin/bootstrap_development.sh now no longer references the chef

Re: IMPALA-3738 advice requested

I would recommend care when messing with the DCHECK_EQ macros and user-defined types, I've been burned by this many times. This may have been fixed with the new version of glog-0.3.4-p2 toolchain update, the scary old comment "TODO(hamaji): Figure out a way to fix" and "#if 1 using ::operator<<;

Can a committer carry a +2

Last GVO run passed with flying colors: https://gerrit.cloudera.org/#/c/6935/ Thanks in advance, - Zach

Re: connection refuseLeap status not available

This is similar to what I hit with NTP the other day after a restart. I tried a number of things, and I think the only thing that worked was waiting for NTP to sync. Pitfalls: ntpdate requires a host on the command line, and doesn't read the configuration file. There was some circumstantial

Re: Bootstrapping a development environment on Ubuntu 16.04

FWIW, I've tried 16.04 XFCE / Ubuntu studio variants on two different HW machines now, was able to build impala on one and load data, but both machines flaked out with weird graphics related failures after kernel upgrades resulting in a black screen on boot. I was able to debug one of them enough

Could someone kick off a GVO for this?

https://gerrit.cloudera.org/#/c/6335/

Nested namespaces in C++ code

So I've recently come upon a situation where it would be useful to have nested namespaces. The reason is that there are a lot of static functions in a class which I would like to benchmark. Writing the same code to run the benchmark test over and over is very tedious and is much more efficiently

Verification run for 6389

Assuming things are good with the build, could someone kick off a verification run for https://gerrit.cloudera.org/#/c/6389/ Thanks, - Zach

Re: Impala JIRA migration to https://issues.apache.org/jira is complete

As one of those affected, what should I do? I already created a new account as zamsden, but all of my issues are assigned to zamsden_impala_ad21. Should I reassign and transfer things over or am I forever doomed to be zamsden_impala_ad21? Thanks, - Zach On Mon, Mar 13, 2017 at 2:24 PM, Jim

GVO request

Looking to start a verification run for https://gerrit.cloudera.org/#/c/6275/ Thanks in advance, - Zach

Re: status-benchmark.cc compilation time

, Henry Robinson <he...@cloudera.com> wrote: > Did you run . bin/set-classpath.sh before running expr-benchmark? > > On 21 February 2017 at 11:30, Zachary Amsden <zams...@cloudera.com> wrote: > > > Unfortunately some of the benchmarks have actually bit-rotted. F

Re: status-benchmark.cc compilation time

Unfortunately some of the benchmarks have actually bit-rotted. For example, expr-benchmark compiles but immediately throws JNI exceptions. On Tue, Feb 21, 2017 at 10:55 AM, Marcel Kornacker wrote: > I'm also in favor of not compiling it on the standard commandline. > >

Re: If FE tests hang on your machine, try restarting the minicluster

There was a brief connectivity interruption today with an external NTP pool that broke Kudu for me, could be the same issue. On Fri, Feb 17, 2017 at 11:56 AM, Lars Volker wrote: > Hi All, > > The frontend tests seemed to hang on my local dev machine. Running jstack > on the

Re: Preferred syntax for warning about ignored Status returns

fter the function for functions > with prototypes but must go earlier in the line for functions without > prototypes. > > Did I get that right? > > On Mon, Jan 9, 2017 at 10:11 AM, Zachary Amsden <zams...@cloudera.com> > wrote: > > Maybe I'm just being dense but I coul

Re: Preferred syntax for warning about ignored Status returns

com> wrote: > That's applying it to the type definition. At the type use: > > https://godbolt.org/g/RMYVW7 > > On Mon, Jan 9, 2017 at 9:56 AM, Zachary Amsden <zams...@cloudera.com> > wrote: > > GCC doesn't catch this when optimization is enabled and the re

Re: [PATCH 1/3] L1 TSC handling

change, of one line in kvm_arch_vcpu_load, is made redundant by a different patch sent by Zachary Amsden (and not yet applied): kvm_arch_vcpu_load() should not read the guest TSC, and if it didn't, of course we didn't have to change the call of kvm_get_msr() to read_l1_tsc(). Signed-off-by: Nadav

Re: [PATCH 2/3] Fix nested VMX TSC emulation

Comments inline. Sorry for top-posting. Gmail is not my normal mode of LKML processing, but hey. On Tue, Aug 2, 2011 at 5:54 AM, Nadav Har'El n...@il.ibm.com wrote: This patch fixes two corner cases in nested (L2) handling of TSC-related issues: 1. Somewhat suprisingly, according to the

Re: [PATCH 3/3] Fix TSC MSR read in nested SVM

Caution: this requires more care. Pretty sure this breaks userspace suspend at the cost of supporting a not-so-reasonable hardware feature. On Tue, Aug 2, 2011 at 5:55 AM, Nadav Har'El n...@il.ibm.com wrote: When the TSC MSR is read by an L2 guest (when L1 allowed this MSR to be read without

Re: Nested VMX - L1 hangs on running L2

, Jul 29, 2011, Zachary Amsden wrote about Re: Nested VMX - L1 hangs on running L2: ... In that case, you need to distinguish between reads of the TSC MSR by the guest and reads by the host (as done internally to track drift and ... Unfortunately, the layering currently doesn't seem to allow

Re: Nested VMX - L1 hangs on running L2

2011/7/27 Nadav Har'El n...@math.technion.ac.il: On Wed, Jul 20, 2011, Zachary Amsden wrote about Re: Nested VMX - L1 hangs on running L2: No, both patches are wrong. kvm_get_msr(vcpu, MSR_IA32_TSC, tsc) should always return the L1 TSC, regardless of the setting of any MSR bitmap

Re: KVM: Remaining body of TSC emulation work

On 06/21/2011 01:50 AM, Jan Kiszka wrote: On 2011-06-21 01:59, Zachary Amsden wrote: In-Reply-To: This is the remaining bulk of work I have related to TSC emulation. In summary, I believe this fixes all known issues with TSC. A few rather subtle issues are cleaned up, S4 suspend is fixed

Fwd: [KVM TSC emulation 1/9] Infrastructure for software and hardware based TSC rate scaling

Original Message Subject: [KVM TSC emulation 1/9] Infrastructure for software and hardware based TSC rate scaling Date: Mon, 20 Jun 2011 16:59:29 -0700 From: Zachary Amsden zams...@redhat.com To: Avi Kivity a...@redhat.com, Marcelo Tosatti mtosa...@redhat.com, Glauber

Fwd: [KVM TSC emulation 2/9] Improve TSC offset matching

Original Message Subject:[KVM TSC emulation 2/9] Improve TSC offset matching Date: Mon, 20 Jun 2011 16:59:30 -0700 From: Zachary Amsden zams...@redhat.com To: Avi Kivity a...@redhat.com, Marcelo Tosatti mtosa...@redhat.com, Glauber Costa glom...@redhat.com, Frank

Fwd: [KVM TSC emulation 3/9] Leave TSC synchronization window open with each new sync

Original Message Subject: [KVM TSC emulation 3/9] Leave TSC synchronization window open with each new sync Date: Mon, 20 Jun 2011 16:59:31 -0700 From: Zachary Amsden zams...@redhat.com To: Avi Kivity a...@redhat.com, Marcelo Tosatti mtosa...@redhat.com, Glauber Costa

Fwd: [KVM TSC emulation 4/9] Fix last_guest_tsc / tsc_offset semantics

Original Message Subject:[KVM TSC emulation 4/9] Fix last_guest_tsc / tsc_offset semantics Date: Mon, 20 Jun 2011 16:59:32 -0700 From: Zachary Amsden zams...@redhat.com To: Avi Kivity a...@redhat.com, Marcelo Tosatti mtosa...@redhat.com, Glauber Costa glom

Fwd: [KVM TSC emulation 5/9] Add last_host_tsc tracking back to KVM

Original Message Subject:[KVM TSC emulation 5/9] Add last_host_tsc tracking back to KVM Date: Mon, 20 Jun 2011 16:59:33 -0700 From: Zachary Amsden zams...@redhat.com To: Avi Kivity a...@redhat.com, Marcelo Tosatti mtosa...@redhat.com, Glauber Costa glom

Fwd: [KVM TSC emulation 6/9] Allow adjust_tsc_offset to be in host or guest cycles

Original Message Subject: [KVM TSC emulation 6/9] Allow adjust_tsc_offset to be in host or guest cycles Date: Mon, 20 Jun 2011 16:59:34 -0700 From: Zachary Amsden zams...@redhat.com To: Avi Kivity a...@redhat.com, Marcelo Tosatti mtosa...@redhat.com, Glauber Costa

Fwd: [KVM TSC emulation 7/9] Don't mark TSC unstable due to S4 suspend

Original Message Subject:[KVM TSC emulation 7/9] Don't mark TSC unstable due to S4 suspend Date: Mon, 20 Jun 2011 16:59:35 -0700 From: Zachary Amsden zams...@redhat.com To: Avi Kivity a...@redhat.com, Marcelo Tosatti mtosa...@redhat.com, Glauber Costa glom

Fwd: [KVM TSC emulation 8/9] Track TSC synchronization in generations

Original Message Subject:[KVM TSC emulation 8/9] Track TSC synchronization in generations Date: Mon, 20 Jun 2011 16:59:36 -0700 From: Zachary Amsden zams...@redhat.com To: Avi Kivity a...@redhat.com, Marcelo Tosatti mtosa...@redhat.com, Glauber Costa glom

Fwd: [KVM TSC emulation 9/9] Add software TSC emulation

Original Message Subject:[KVM TSC emulation 9/9] Add software TSC emulation Date: Mon, 20 Jun 2011 16:59:37 -0700 From: Zachary Amsden zams...@redhat.com To: Avi Kivity a...@redhat.com, Marcelo Tosatti mtosa...@redhat.com, Glauber Costa glom...@redhat.com, Frank

[PATCH x86 kvm] Fix register corruption in pvclock_scale_delta

for 128-bit multiply Signed-off-by: Zachary Amsden zams...@redhat.com diff --git a/arch/x86/include/asm/pvclock.h b/arch/x86/include/asm/pvclock.h index 31d84ac..a518c0a 100644 --- a/arch/x86/include/asm/pvclock.h +++ b/arch/x86/include/asm/pvclock.h @@ -22,6 +22,8 @@ static inline u64

Re: [Qemu-devel] [PATCH v4 5/5] hpet 'driftfix': add code in hpet_timer() to compensate delayed callbacks and coalesced interrupts

On 05/09/2011 12:03 AM, Ulrich Obergfell wrote: Loss of periodic timer interrupts caused by delayed callbacks and by interrupt coalescing is compensated by gradually injecting additional interrupts during subsequent timer intervals, starting at a rate of one additional interrupt per interval.

Re: [Qemu-devel] [PATCH v4 5/5] hpet 'driftfix': add code in hpet_timer() to compensate delayed callbacks and coalesced interrupts

On 05/09/2011 12:03 AM, Ulrich Obergfell wrote: Loss of periodic timer interrupts caused by delayed callbacks and by interrupt coalescing is compensated by gradually injecting additional interrupts during subsequent timer intervals, starting at a rate of one additional interrupt per interval.

Re: [PATCH 27/30] nVMX: Additional TSC-offset handling

On 05/08/2011 01:29 AM, Nadav Har'El wrote: In the unlikely case that L1 does not capture MSR_IA32_TSC, L0 needs to emulate this MSR write by L2 by modifying vmcs02.tsc_offset. We also need to set vmcs12.tsc_offset, for this change to survive the next nested entry (see prepare_vmcs02()).

Re: 2.6.32 guest with paravirt clock enabled hangs on 2.6.37.6 host (w qemu-kvm-0.13.0)

On 05/08/2011 12:06 PM, Nikola Ciprich wrote: OK, I see.. the problem is, that I'm trying to hunt down bug causing hangs when 2.6.32 guests try to run tcpdump - this seems to be reproducible even on latest 2.6.32.x, and seems like it depends on kvm-clock.. So I was thinking about bisecting

Re: 2.6.32 guest with paravirt clock enabled hangs on 2.6.37.6 host (w qemu-kvm-0.13.0)

On 05/09/2011 11:25 AM, Nikola Ciprich wrote: The guest, because latest kernels do not suffer this problem, so I'd like to find fix so it can be pushed to -stable (we're using 2.6.32.x) host is currently 2.6.37 (and i'm currently testing 2.6.38 as well) n. That's a pretty wide range to be

Re: Bug in KVM clock backwards compensation

On 04/29/2011 01:40 AM, Joerg Roedel wrote: On Thu, Apr 28, 2011 at 08:00:57PM -0700, Zachary Amsden wrote: On 04/28/2011 01:20 PM, Joerg Roedel wrote: This code checks how many guest tsc cycles have passed since this vCPU was de-scheduled last time (and before it is running

Bug in KVM clock backwards compensation

So I've been going over the new code changes to the TSC related code and I don't like one particular set of changes. In particular, here: kvm_x86_ops-vcpu_load(vcpu, cpu); if (unlikely(vcpu-cpu != cpu) || check_tsc_unstable()) { /* Make sure TSC doesn't go

Re: Bug in KVM clock backwards compensation

On 04/28/2011 12:06 AM, Jan Kiszka wrote: On 2011-04-28 08:59, Zachary Amsden wrote: So I've been going over the new code changes to the TSC related code and I don't like one particular set of changes. In particular, here: kvm_x86_ops-vcpu_load(vcpu, cpu); if (unlikely

Re: Bug in KVM clock backwards compensation

On 04/28/2011 12:13 AM, Roedel, Joerg wrote: On Thu, Apr 28, 2011 at 02:59:57AM -0400, Zachary Amsden wrote: So I've been going over the new code changes to the TSC related code and I don't like one particular set of changes. In particular, here: kvm_x86_ops-vcpu_load(vcpu, cpu

Re: Bug in KVM clock backwards compensation

On 04/28/2011 12:22 AM, Roedel, Joerg wrote: On Thu, Apr 28, 2011 at 03:06:01AM -0400, Jan Kiszka wrote: And /me still wonders (like I did when this first popped up) if the proper place of determining TSC stability really have to be KVM. If the Linux core fails to detect some instability

Re: Bug in KVM clock backwards compensation

On 04/28/2011 01:20 PM, Joerg Roedel wrote: On Thu, Apr 28, 2011 at 11:34:44AM -0700, Zachary Amsden wrote: On 04/28/2011 12:13 AM, Roedel, Joerg wrote: I see it different. This code wants to check if the _guest_ tsc moves forwared (or at least not backwards). So it is fully

Re: [PATCH 3/6] KVM: X86: Make tsc_delta calculation a function of guest tsc

On 2011-04-19 08:46, Roedel, Joerg wrote: On Mon, Apr 18, 2011 at 08:02:35PM -0400, Zachary Amsden wrote: On Sat, Apr 16, 2011 at 06:09:17PM +0200, Jan Kiszka wrote: On 2011-03-25 09:44, Joerg Roedel wrote: + tsc_delta = !vcpu

Re: [PATCH 3/6] KVM: X86: Make tsc_delta calculation a function of guest tsc

On Sat, Apr 16, 2011 at 06:09:17PM +0200, Jan Kiszka wrote: On 2011-03-25 09:44, Joerg Roedel wrote: + tsc_delta = !vcpu-arch.last_guest_tsc ? 0 : +tsc - vcpu-arch.last_guest_tsc; This patch appears to cause troubles to Linux

Re: [PATCHv2] fix regression caused by e48672fa25e879f7ae21785c7efd187738139593

On 04/11/2011 12:12 PM, Nikola Ciprich wrote: Hello Zachary, what is the current status, are You going to post this patch to Avi? I'd like to see one (or both) in stable eventually, I think it's good candidate.. BR nik I think for upstream the newer patch is the way to go, but I would

Re: [PATCHv2] fix regression caused by e48672fa25e879f7ae21785c7efd187738139593

On 03/09/2011 05:36 PM, Nikola Ciprich wrote: commit 387b9f97750444728962b236987fbe8ee8cc4f8c moved kvm_request_guest_time_update(vcpu), breaking 32bit SMP guests using kvm-clock. Fix this by moving (new) clock update function to proper place. Signed-off-by: Nikola

Re: [PATCH] add missing guest clock update removed by e48672fa25e879f7ae21785c7efd187738139593

On 03/09/2011 02:30 PM, Nikola Ciprich wrote: Can you try moving the kvm_make_request() inside the if conditional and see if it that also fixes it? yes, changing to: if (unlikely(vcpu-cpu != cpu) || check_tsc_unstable()) { kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu); /*

Re: [PATCHv2] fix regression caused by e48672fa25e879f7ae21785c7efd187738139593

On 03/09/2011 05:36 PM, Nikola Ciprich wrote: commit 387b9f97750444728962b236987fbe8ee8cc4f8c moved kvm_request_guest_time_update(vcpu), breaking 32bit SMP guests using kvm-clock. Fix this by moving (new) clock update function to proper place. Signed-off-by: Nikola

Re: [PATCH] add missing guest clock update removed by e48672fa25e879f7ae21785c7efd187738139593

On 03/07/2011 05:18 AM, Nikola Ciprich wrote: e48672fa25e879f7ae21785c7efd187738139593 removed kvm_request_guest_time_update(vcpu); this breaks 32bit SMP guests using virtio-clock. thus add unconditional call to kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu) to fix the problem. Signed-off-by:

Re: regression - 2.6.36 - 2.6.37 - kvm - 32bit SMP guests don't boot

On 03/05/2011 02:21 AM, Nikola Ciprich wrote: Can you try this patch to see if it fixes the problem? You haven't read my replies, did you? ;-) kvm_request_guest_time_update seems to have been removed, and kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu) seems to be used instead, adding it

Re: regression - 2.6.36 - 2.6.37 - kvm - 32bit SMP guests don't boot

On 03/03/2011 05:01 PM, Nikola Ciprich wrote: That sounds like a kernel which will be vulnerable to broken KVM clock on 32-bit. There's a kernel side fix that is needed, but why the server side change triggers the problem needs more investigation. OK, it's important for me that I can fix

Re: regression - 2.6.36 - 2.6.37 - kvm - 32bit SMP guests don't boot

On 03/04/2011 02:09 PM, Glauber Costa wrote: On Fri, 2011-03-04 at 19:27 +0100, Nikola Ciprich wrote: Hello Zachary, You don't see any messages about TSC being unstable or switching clocksource after loading the KVM module? And you are not suspending the host or anything?

Re: regression - 2.6.36 - 2.6.37 - kvm - 32bit SMP guests don't boot

On 03/04/2011 05:36 PM, Nikola Ciprich wrote: I think although the long term plan is to just do this update once in your case (stable tsc), this update is needed. Why don't you send a patch to re-include it ? Yes, I'll gladly submit patch, one question, is this OK to just add calling

Re: regression - 2.6.36 - 2.6.37 - kvm - 32bit SMP guests don't boot

On 03/04/2011 05:36 PM, Nikola Ciprich wrote: I think although the long term plan is to just do this update once in your case (stable tsc), this update is needed. Why don't you send a patch to re-include it ? Yes, I'll gladly submit patch, one question, is this OK to just add calling

Re: regression - 2.6.36 - 2.6.37 - kvm - 32bit SMP guests don't boot

On 03/03/2011 02:06 AM, Nikola Ciprich wrote: No worries. What mess? twice sending the same mail, nevermind :) I have two things you can try: first is running a single VCPU guest, if you have not done so already. yup, UP guest is fine, just SMP doesn't work. Second is

Re: regression - 2.6.36 - 2.6.37 - kvm - 32bit SMP guests don't boot

On 03/03/2011 04:06 PM, Nikola Ciprich wrote: What is the exact kernel version you are using in the guest. It's latest centos (2.6.18-194.32.1.el5), so I guess there are a lot of fixes, but it's possible the kvm-clock is broken in it. I can't influence what kernel is used there (at least

Re: regression - 2.6.36 - 2.6.37 - kvm - 32bit SMP guests don't boot

(resend, sorry for the mess) No worries. What mess? I have two things you can try: first is running a single VCPU guest, if you have not done so already. Second is adding the bootparameter clocksource=acpi_pm to your guest kernel. If either of those fixes the problem, it very well

Re: regression - 2.6.36 - 2.6.37 - kvm - 32bit SMP guests don't boot

On 02/27/2011 12:20 PM, Nikola Ciprich wrote: I was not aware of the thread. Please cc me directly, or add a keyword I track - timekeeping, TSC.. Hello Zachary, thanks for Your time looking at this! That change alone may not bisect well; without further fixes on top of it, you may

Re: regression - 2.6.36 - 2.6.37 - kvm - 32bit SMP guests don't boot

On 02/28/2011 09:32 AM, Nikola Ciprich wrote: Does the bug you are hitting manifest on both Intel and AMD platforms? I don't have any AMD box here, I'll try this out at my home box. Further, do the systems you are hitting this on have stable or unstable TSCs? how do I find

Re: regression - 2.6.36 - 2.6.37 - kvm - 32bit SMP guests don't boot

On Mon, Feb 28, 2011 at 10:17:24AM -0500, Zachary Amsden wrote: On 02/28/2011 09:32 AM, Nikola Ciprich wrote: Does the bug you are hitting manifest on both Intel and AMD platforms? I don't have any AMD box here, I'll try this out at my home box. Further, do

Re: regression - 2.6.36 - 2.6.37 - kvm - 32bit SMP guests don't boot

c285545f813d7b0ce989fd34e42ad1fe785dc65d Author: Zachary Amsden zams...@redhat.com Date: Sat Sep 18 14:38:15 2010 -1000 KVM: x86: TSC catchup mode Negate the effects of AN TYM spell while kvm thread is preempted by tracking conversion factor to the highest TSC rate and catching the TSC up when it has

Re: [PATCH 0/6] KVM support for TSC scaling

On 02/21/2011 12:28 PM, Roedel, Joerg wrote: On Sun, Feb 13, 2011 at 10:19:19AM -0500, Avi Kivity wrote: On 02/09/2011 07:29 PM, Joerg Roedel wrote: Hi Avi, Marcelo, here is the patch-set to implement the TSC-scaling feature of upcoming AMD CPUs. When this feature is supported the

Re: [PATCH 5/6] KVM: X86: Delegate tsc-offset calculation to architecture code

On 02/09/2011 12:29 PM, Joerg Roedel wrote: With TSC scaling in SVM the tsc-offset needs to be calculated differently. This patch propagates this calculation into the architecture specific modules so that this complexity can be handled there. Signed-off-by: Joerg Roedeljoerg.roe...@amd.com ---

Re: [PATCH] KVM: x86: Convert tsc_write_lock to raw_spinlock

On 02/07/2011 06:35 AM, Jan Kiszka wrote: On 2011-02-04 22:03, Zachary Amsden wrote: On 02/04/2011 04:49 AM, Jan Kiszka wrote: Code under this lock requires non-preemptibility. Ensure this also over -rt by converting it to raw spinlock. Oh dear, I had forgotten about

Re: [PATCH] KVM: x86: Convert tsc_write_lock to raw_spinlock

On 02/07/2011 10:00 AM, Jan Kiszka wrote: On 2011-02-07 15:11, Zachary Amsden wrote: On 02/07/2011 06:35 AM, Jan Kiszka wrote: On 2011-02-04 22:03, Zachary Amsden wrote: On 02/04/2011 04:49 AM, Jan Kiszka wrote: Code under this lock requires non-preemptibility

Re: [PATCH] KVM: x86: Convert tsc_write_lock to raw_spinlock

On 02/04/2011 04:49 AM, Jan Kiszka wrote: Code under this lock requires non-preemptibility. Ensure this also over -rt by converting it to raw spinlock. Oh dear, I had forgotten about that. I believe kvm_lock might have the same assumption in a few places regarding clock. -- To

Re: [KVM TSC trapping / migration 2/2] Add TSC KHZ MSR

On 01/14/2011 06:00 AM, Juan Quintela wrote: Marcelo Tosattimtosa...@redhat.com wrote: On Fri, Jan 07, 2011 at 10:44:20AM -1000, Zachary Amsden wrote: On 01/07/2011 12:48 AM, Marcelo Tosatti wrote: On Thu, Jan 06, 2011 at 12:10:45AM -1000, Zachary Amsden wrote

Re: [KVM TSC trapping / migration 1/2] Add TSC trapping for SVM and VMX

On 01/07/2011 01:23 AM, Marcelo Tosatti wrote: On Thu, Jan 06, 2011 at 12:10:44AM -1000, Zachary Amsden wrote: Reasons to trap the TSC are numerous, but we want to avoid it as much as possible for performance reasons. We provide two conservative modes via modules parameters and userspace

Re: [KVM TSC trapping / migration 2/2] Add TSC KHZ MSR

On 01/07/2011 12:48 AM, Marcelo Tosatti wrote: On Thu, Jan 06, 2011 at 12:10:45AM -1000, Zachary Amsden wrote: Use an MSR to allow soft migration to hosts which do not support TSC trapping. Rather than make this a required element of any migration protocol, we allow the TSC rate

KVM TSC trapping

On top of my last patchset, I now implement TSC trapping and a flexible migration scheme for maintaining stable TSC across migration. Since it is administratively configured, it can be selectively enabled only for VMs which require it. In particular, VMs which use KVM clock probably do not want

Re: [KVM TSC trapping / migration 2/2] Add TSC KHZ MSR

On 01/06/2011 12:34 AM, Alexander Graf wrote: Am 06.01.2011 um 11:10 schrieb Zachary Amsdenzams...@redhat.com: Use an MSR to allow soft migration to hosts which do not support TSC trapping. Rather than make this a required element of any migration protocol, we allow the TSC rate to be

Re: [KVM TSC trapping / migration 1/2] Add TSC trapping for SVM and VMX

On 01/06/2011 12:41 AM, Alexander Graf wrote: Am 06.01.2011 um 11:10 schrieb Zachary Amsdenzams...@redhat.com: Reasons to trap the TSC are numerous, but we want to avoid it as much as possible for performance reasons. We provide two conservative modes via modules parameters and userspace

Re: [KVM TSC trapping / migration 1/2] Add TSC trapping for SVM and VMX

On 01/06/2011 01:32 AM, Avi Kivity wrote: On 01/06/2011 12:10 PM, Zachary Amsden wrote: Reasons to trap the TSC are numerous, but we want to avoid it as much as possible for performance reasons. We provide two conservative modes via modules parameters and userspace hinting. First, the module

Re: [KVM TSC trapping / migration 1/2] Add TSC trapping for SVM and VMX

On 01/06/2011 01:38 AM, Alexander Graf wrote: On 06.01.2011, at 12:30, Zachary Amsden wrote: On 01/06/2011 12:41 AM, Alexander Graf wrote: Am 06.01.2011 um 11:10 schrieb Zachary Amsdenzams...@redhat.com: Reasons to trap the TSC are numerous, but we want to avoid

Re: [KVM TSC trapping / migration 2/2] Add TSC KHZ MSR

On 01/06/2011 01:40 AM, Alexander Graf wrote: On 06.01.2011, at 12:27, Zachary Amsden wrote: On 01/06/2011 12:34 AM, Alexander Graf wrote: Am 06.01.2011 um 11:10 schrieb Zachary Amsdenzams...@redhat.com: Use an MSR to allow soft migration to hosts which do not support TSC

Re: [KVM TSC trapping / migration 1/2] Add TSC trapping for SVM and VMX

On 01/06/2011 12:38 PM, Alexander Graf wrote: snip Sure, I'm not saying your patch is bad or goes in the wrong direction. I'd just think it'd be awesome to have an easy way for the guest OS to know that something as crucial as TSC reading speed got changed, hopefully even TSC frequency.

Re: [KVM Clock Synchronization 4/4] Add master clock for KVM clock

On 01/04/2011 08:20 AM, Marcelo Tosatti wrote: On Tue, Dec 28, 2010 at 07:38:20PM -1000, Zachary Amsden wrote: On systems with synchronized TSCs, we still have VCPU individual KVM clocks, each with their own computed offset. As this all happens at different times, the computed KVM clock

Re: [KVM Clock Synchronization 2/4] Keep TSC synchronized across host suspend

On 01/04/2011 05:36 AM, Marcelo Tosatti wrote: On Tue, Dec 28, 2010 at 07:38:18PM -1000, Zachary Amsden wrote: During a host suspend, TSC may go backwards, which KVM interprets as an unstable TSC. Technically, KVM should not be marking the TSC unstable, which causes the TSC clocksource

KVM clock synchronization

A further set of improvements to KVM clock. Now it actually can stay synchronized on SMP systems with a stable TSC, does not get destabilized by host suspend, and is resistant to migration. I did look a bit into the second to last remaining migration issue; that is, we read the TSCs for all

[KVM Clock Synchronization 1/4] Make cyc_to_nsec conversions more reliable

Rather than use the host CPU TSC rate, which can change, compute cycle to nanosecond conversion in the guest TSC rate, which is fixed. This allows the math for write compensation detection to be made more reliable. Signed-off-by: Zachary Amsden zams...@redhat.com --- arch/x86/kvm/x86.c | 58

[KVM Clock Synchronization 2/4] Keep TSC synchronized across host suspend

During a host suspend, TSC may go backwards, which KVM interprets as an unstable TSC. Technically, KVM should not be marking the TSC unstable, which causes the TSC clocksource to go bad, but should be adjusting the TSC offsets in such a case. Signed-off-by: Zachary Amsden zams...@redhat.com

[KVM Clock Synchronization 4/4] Add master clock for KVM clock

compare to ensure it does not happen. This change should remove that requirement. Signed-off-by: Zachary Amsden zams...@redhat.com --- arch/x86/include/asm/kvm_host.h |1 + arch/x86/kvm/x86.c | 42 ++- 2 files changed, 42 insertions(+), 1

Re: [Qemu-devel] [RFC][PATCH 0/3] Let RTC follow backward jumps of host clock immediately

On 12/17/2010 04:58 AM, Jan Kiszka wrote: By default, we base the mc146818 RTC on the host clock (CLOCK_REALTIME). This works fine if only the frequency of the host clock is tuned (e.g. by NTP) or if it is set to a future time. However, if the host is tuned backward, e.g. because NTP obtained

Re: vread in kvm_clock

On 12/19/2010 08:16 PM, Avi Kivity wrote: On 12/20/2010 03:15 AM, Zachary Amsden wrote: On 12/19/2010 05:27 AM, Avi Kivity wrote: On 12/17/2010 07:43 PM, Zachary Amsden wrote: On 12/15/2010 10:16 AM, Julien Desfossez wrote: Hi, I'm currently working with the kvm clocksource and I'm

Re: vread in kvm_clock

On 12/19/2010 05:27 AM, Avi Kivity wrote: On 12/17/2010 07:43 PM, Zachary Amsden wrote: On 12/15/2010 10:16 AM, Julien Desfossez wrote: Hi, I'm currently working with the kvm clocksource and I'm wondering if we could implement the vread function for this clock source when we are running

Re: vread in kvm_clock

On 12/15/2010 10:16 AM, Julien Desfossez wrote: Hi, I'm currently working with the kvm clocksource and I'm wondering if we could implement the vread function for this clock source when we are running on a host with constant_tsc. If I understand correctly the hv_clock structure is per_cpu

  1   2   3   4   5   6   7   8   9   10   >