Re: [Xen-devel] [V2 PATCH 7/9] x86/hvm: pkeys, add pkeys support for guest_walk_tables

2015-12-02 Thread Han, Huaitong
On Tue, 2015-12-01 at 20:30 +, Andrew Cooper wrote:
> > +#include 
> 
> I can see why you need xstate.h, but I why do you need i387.h ?
Use vcpu_save_fpu functions.

> > +
> >  if ( pse2M )
> >  {
> >  /* Special case: this guest VA is in a PSE superpage, so
> > there's
> > @@ -330,6 +390,11 @@ guest_walk_tables(struct vcpu *v, struct
> > p2m_domain *p2m,
> >  goto out;
> >  }
> >  rc |= ((gflags & mflags) ^ mflags);
> > +#if GUEST_PAGING_LEVELS >= 4
> > +pkeys = guest_l1e_get_pkeys(gw->l1e);
> > +if (leaf_pte_pkeys_check(v, pfec, gflags, pkeys))
> > +rc |= _PAGE_PKEY_BIT;
> > +#endif
> 
> As I identified in v1, the fact that you do not modify the callers of
> guest_walk_tables() proves that this change is buggy.  You must
> modify
> the callers to cope with the new error of _PAGE_PKEY_BIT.
_PAGE_PKEY_BIT just a flag to tell
 hap_p2m_ga_to_gfn(GUEST_PAGING_LEVELS) there is a page fault, PK is
real physical fault which is included in pfec, the funtion of
_PAGE_PKEY_BIT is that hap_p2m_ga_to_gfn returns INVALID_GFN, you can
find the pfec modification on PATCH 9/9.

> 
> ~Andrew
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Skylake: VT-d and other error messages

2015-12-02 Thread Shuai Ruan
On Sat, Nov 28, 2015 at 03:46:05PM -0500, Eric Shelton wrote:

hi eric:

I test this on my skylake machine. Result is follow
> Not enabling x2APIC (upon firmware request)
I got the message too.
> ...
> mwait-idle: does not run on family 6 model 94
I got the message too.
> ...
> [VT-D] iommu.c:875: iommu_fault_status: Primary Pending Fault
> [VT-D] INTR-REMAP: Request device [:f0:1f.0] fault index 0, iommu reg =
> 82c000201000
> (on motherboard 1) [VT-D] INTR-REMAP: reason 22 - Present field in the IRTE
> entry is clear
> (on motherboard 2) [VT-D] INTR-REMAP: reason 25 - Blocked a compatibility
> format interrupt request
> 
The error above do not happen on my skylake machine.

Thanks 

Shuai
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Skylake: VT-d and other error messages

2015-12-02 Thread Shuai Ruan
On Wed, Dec 02, 2015 at 04:47:17PM +0800, Shuai Ruan wrote:
> On Sat, Nov 28, 2015 at 03:46:05PM -0500, Eric Shelton wrote:
> 
> hi eric:
> 
> I test this on my skylake machine. Result is follow
> > Not enabling x2APIC (upon firmware request)
> I got the message too.
According to Andrew's comments. When disable an option called "X2APIC
Opt Out" in bios , the message will disappear.
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v2 0/3] VPMU fixes

2015-12-02 Thread Dietmar Hahn
Am Mittwoch 02 Dezember 2015, 02:20:49 schrieb Tian, Kevin:
> > From: Boris Ostrovsky [mailto:boris.ostrov...@oracle.com]
> > Sent: Wednesday, December 02, 2015 12:50 AM
> > 
> > * Limit VPMU support to PMU versions 2, 3 and 4 (emulated at version 3 
> > level)
> > * Always implement family 6 VPMU quirk.
> >   ==>  Intel folks: is the quirk needed for all family 6 processors or can 
> > we
> > limit it to certain models?
> 
> Let me confirm this information internally. btw could you provide a link
> where you find out the original quirk information?

http://old-list-archives.xenproject.org/xen-devel/2010-11/msg01157.html
Dietmar.

> 
> Thanks
> Kevin
> 
> 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
> 

-- 
Company details: http://ts.fujitsu.com/imprint.html

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] build: fix clean rule to cover objects in unvisited subdirs

2015-12-02 Thread Ian Campbell
On Tue, 2015-12-01 at 09:44 -0700, Jan Beulich wrote:
> > > > On 01.12.15 at 17:34,  wrote:
> > > On Dec 1, 2015, at 10:07 AM, Jan Beulich  wrote:
> > > 
> > > For one build run, yes. But then you can (a) build individual object
> > > files and (b) as mentioned above change configuration (implying
> > > that you know what you're doing). Also you could, using the
> > > example above, do a kexec=y build, then a kexec=n one, then
> > > notice you needed to clean in between, so you then clean using
> > > kexec=n and build again with that option, but cleaning again
> > > would still leave the kexec files around.
> > > 
> > > And btw., we have a similar issue already when you switch
> > > between arches (no cleaning happens cross-arch).
> > 
> > OK, so you are working on a different assumption than I was. I was
> > treating the clean rule as needing to be run when you are wanting to
> > explicitly rebuild all object files needed for the current build 
> > configuration 
> > (i.e., only cleaning files that would be linked into the current
> > hypervisor 
> > build).
> > It sounds like you are expecting the clean rule to clean out all object
> > files no matter whether they are part of the current build
> > configuration
> > or not. 
> > 
> > Working on that assumption, it seems like running a:
> > find . -name “*.o” -type f -delete 
> > from the xen/ directory would accomplish that and would be less
> > fragile than trying to grab various different variables and munge
> > them to try to grab all possible .o files specified by the system.
> > Plus,
> > the find command would likely execute quicker. 
> > 
> > Does something like that seem acceptable?
> 
> I can't see an immediate reason why it would not be, as long
> as it's clear that this won't eliminate the need to recurse into
> the subdirectories. But I'd certainly recommend to wait for
> other feedback (namely by other hypervisor maintainers)
> before you go that route.
> 
> Also please note that -delete is not a standard primary, so
> would need replacing.
> 
> Also the same global approach could then perhaps be used to
> remove all the .*.d files.

I can't think of a good reason not to do both of these.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Hotplugged devices in Xen 4.5 and domain reboot

2015-12-02 Thread Ian Campbell
On Tue, 2015-12-01 at 17:21 +, Wei Liu wrote:
> 
> > In dom0 we have next issue:
> > /libxl: error: libxl_device.c:283:libxl__device_disk_set_backend: Disk
> > vdev=xvda10 failed to stat: /dev/sda1: No such file or directory//-
> > /this issue occurs due to missing /dev/sda1 device (all hardware are
> > placed in DomD domain).
> 
> Looks like the path that is stat'ed is not correct.

The path is within the driver domain, while the stat is in the toolstack
domain, so it cannot possibly be correct.

It is a bug if the toolstack is trying to do that stat when toolstack-domid 
!= driver-domain-domid.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Hotplugged devices in Xen 4.5 and domain reboot

2015-12-02 Thread Ian Campbell
On Tue, 2015-12-01 at 18:52 +0100, Roger Pau Monné wrote:
> El 01/12/15 a les 17.48, Iurii Mykhalskyi ha escrit:
> > > Does something like the following work? If not, could you paste the
> > > error when running it with -vvv.
> > > 
> > > xl block-attach DomU
> > > format=raw,vdev=hdc,access=rw,backend=DomD,target=/path/to/dev
> > In dom0 we have next issue:
> > /libxl: error: libxl_device.c:283:libxl__device_disk_set_backend: Disk
> > vdev=xvda10 failed to stat: /dev/sda1: No such file or directory//-
> > /this issue occurs due to missing /dev/sda1 device (all hardware are
> > placed in DomD domain).
> 
> I'm not sure how can you get to this path, the libxl chunk in 
> stable-4.5 is:
> 
> 271 if (disk->format == LIBXL_DISK_FORMAT_EMPTY) {
> 272 if (!disk->is_cdrom) {
> 273 LOG(ERROR, "Disk vdev=%s is empty but not cdrom", disk-
> >vdev);
> 274 return ERROR_INVAL;
> 275 }
> 276 memset(&a.stab, 0, sizeof(a.stab));
> 277 } else if ((disk->backend == LIBXL_DISK_BACKEND_UNKNOWN ||
> 278 disk->backend == LIBXL_DISK_BACKEND_PHY) &&
> 279disk->backend_domid == LIBXL_TOOLSTACK_DOMID &&
> 280!disk->script) {
> 281 if (stat(disk->pdev_path, &a.stab)) {
> 282 LOGE(ERROR, "Disk vdev=%s failed to stat: %s",
> 283 disk->vdev, disk->pdev_path);
> 284 return ERROR_INVAL;
> 285 }
> 286 }
> 
> So it seems that block-attach is ignoring the 'backend=foo' field in 
> the disk configuration?
> 
> Can you paste the full output of the execution with -vvv?

Also a dummy attach will print the parsed json of the requested spec, e.g.:

# xl -N block-attach 0 
format=raw,vdev=hdc,access=rw,backend=DomD,target=/path/to/dev
disk: {
"backend_domname": "DomD",
"pdev_path": "/path/to/dev",
"vdev": "hdc",
"format": "raw",
"readwrite": 1
}

I'm not sure if -vvv on a proper attach will do the same in 4.5, so having
the output of both would be useful.

Ian.
> 
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Hotplugged devices in Xen 4.5 and domain reboot

2015-12-02 Thread Ian Campbell
On Tue, 2015-12-01 at 15:03 -0600, Doug Goldstein wrote:
> This works, but might have problems in Xen 4.5. If you're using running
> on Linux 3.14 or newer then you will have a problem. You need to
> backport commit 9c89dc95201ffed5fead17b35754bf9440fdbdc0 if you're using
> the C based xenstore and 7d418eab3b6dbdeec84bf73af301dca54368547a if
> you're using the Ocaml based xenstore.

What's the underlying problem in Linux which this commit happens to
workaround?

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] x86/time: Don't use EFI's GetTime call by default

2015-12-02 Thread Jan Beulich
>>> On 01.12.15 at 20:26,  wrote:
> On 01/12/15 17:26, Jan Beulich wrote:
> On 01.12.15 at 17:57,  wrote:
>>> When EFI is used, don't use EFI's GetTime() to get the time, because it
>>> is broken on many platforms. From Linux commit 7efe665903d0 ("rtc:
>>> Disable EFI rtc for x86"):
>>> "Disable it explicitly for x86 so that we don't give users false
>>> hope that this driver will work - it won't, and your machine is likely
>>> to crash."
>>>
>>> Signed-off-by: Ross Lagerwall 
>> NAK, since being conceptually wrong (and both of my systems work
>> fine). Vendors should get their firmware fixed, and by not using
>> runtime service functions we would give them even less reason to
>> do so. Until then we have "efi=no-rs".
> 
> This is completely unreasonable.
> 
> It is not conceptually wrong.

I'm sorry, but no. Otherwise you mean to state that specifications
don't even need to be written, since if people don't play by them,
workarounds will get implemented anyway.

>  GetTime() is very well known completely
> broken, especially after ExitBootServices(), to the point that every
> other EFI implementation (including windows) completely avoids it.

I'm not sure about the order of things. I started working with EFI on
IA64, where using runtime services is a must. Windows started
using EFI on IA64 too, so I'm pretty convinced they used GetTime()
there. Whether they didn't on x86 because x86 implementations
were broken, or whether x86 implementations end up broken
because Windows never used those functions is simply an unknown.

> The fact that your two systems don't crash immediately is curious, but
> they are not a representative of systems in general.  Not a single
> broadwell or skylake platform I have access to boots in EFI mode if
> GetTime() is used (which include 4 different manufactures).

The original x86 implementation we used almost 15 years back
(on top of systems not even supporting EFI, since there simply were
none back then) didn't have this problem either.

> Vendors will not fix their firmware.  Disabling all runtime services is
> not a reasonable alternative.

Then we should see about adding support for "efi=no-time".

> This is a firmware bug just like many others and needs to be worked
> around by default like others.

Except that you propose to work around it even when there is nothing
to be worked around. I wouldn't mind switching on the workaround
for known broken systems.

> Anything else is actively damaging to the Xen community.  People just
> get frustrated when it doesn't work (especially if the problem has been
> identify and a fix rejected upstream) and will move elsewhere instead.

I'd leave specifying (or switching on by source patching) default
workarounds to distros then. They know what their customers need,
or what systems they expect their stuff to be run on.

> Any situation where a command line override is required to make Xen boot
> is a bug in Xen and should be fixed.  This is why we have __init time
> quirks. it doesn't matter if we have some truly horrendous workarounds;
> Xen needs to be able to boot by default wherever possible.

But requiring command line overrides for other things that are
considered to not work by default is fine? I'm thinking of XenServer's
universal disabling of XSAVE here...

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCHv6] 01/28] build: import Kbuild/Kconfig from Linux 4.2

2015-12-02 Thread Jan Beulich
>>> On 01.12.15 at 21:04,  wrote:
> On 11/30/15 11:19 AM, Ian Campbell wrote:
>> On Mon, 2015-11-30 at 11:00 -0600, Doug Goldstein wrote:
>>> Since there is a request to have KEXEC and the UARTs
>>> configurable by the user
>> 
>> Who asked for this?
>> 
>> I have quite a strong preference for not adding _any_ new[*] user
>> configurable options in this first pass, since I think those need to be
>> considered quite carefully whereas this first series should be largely
>> about the mechanics of introducing Kconfig files.
>> 
>> Ian.
>> 
>> [*] i.e. anything which is not already controllable by the current .config
>> driven thing.
>> 
> 
> The ARM UARTs are the take away I had from conversations with Julien
> Grall and reading past comments on the ML how people can change the ARM
> UARTs. Obviously if that's not desired I can drop that. I originally had
> them enabled as they are in config/arm{32,64}.mk and changed them to be
> user configurable later in the series.
> 
> As far as KEXEC goes, its a user configurable option now in Rules.mk.
> You can build "make kexec=n" and it will disable it. I chose that one as
> an original example in v1 of how a user configurable option would look
> in this scheme.

And note how Ian said "_any_ new".

In fact as a follow-up we should convert other command line driven
overrides to config options (debug, perfc, bigmem, shadow) imo.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCHv6] 01/28] build: import Kbuild/Kconfig from Linux 4.2

2015-12-02 Thread Ian Campbell
On Tue, 2015-12-01 at 14:04 -0600, Doug Goldstein wrote:
> On 11/30/15 11:19 AM, Ian Campbell wrote:
> > On Mon, 2015-11-30 at 11:00 -0600, Doug Goldstein wrote:
> > > Since there is a request to have KEXEC and the UARTs
> > > configurable by the user
> > 
> > Who asked for this?
> > 
> > I have quite a strong preference for not adding _any_ new[*] user
> > configurable options in this first pass, since I think those need to be
> > considered quite carefully whereas this first series should be largely
> > about the mechanics of introducing Kconfig files.
> > 
> > Ian.
> > 
> > [*] i.e. anything which is not already controllable by the current
> > .config
> > driven thing.
> > 
> 
> The ARM UARTs are the take away I had from conversations with Julien
> Grall

AIUI all Julien was asking for was that the same set of UARTs should be
enabled for arm32 or arm64 both before and after this series, not that they
should be user configurable (in this first pass at least).

TBH I think this series is getting bogged down in trying to do much all at
the same time, switching to Kconfig, making things newly user-selectable,
arranging for out of tree builds, some of which are either (potentially)
controversial or simply result in apparently unnecessary changes if the
reviewer is only expecting a subset of those changes (especially those only
expecting the first), leading to a few RTTs of back and forth with
reviewers.

I would encourage you to par this first series back to the simplest
possible "switch to Kconfig, retaining the exact same set of user facing
options as today" and avoid feature creep from people requesting new and
exciting things which the switch to Kconfig makes possible.

That way we can make progress on a mostly mechanical switch to Kconfig
without continuously getting side tracked on questions like whether this or
that should be user selectable or not.

All of the "feature-creep" (which I don't mean in the pejorative sense)
things can easily be done in a follow up series.

>  and reading past comments on the ML how people can change the ARM
> UARTs. Obviously if that's not desired I can drop that. I originally had
> them enabled as they are in config/arm{32,64}.mk and changed them to be
> user configurable later in the series.
> 
> As far as KEXEC goes, its a user configurable option now in Rules.mk.
> You can build "make kexec=n" and it will disable it. I chose that one as
> an original example in v1 of how a user configurable option would look
> in this scheme.

I don't have a problem with converting existing user-configurable options
into Kconfig in the first pass, that seems natural/justifiable enough to
me, and doing it in the final patch as you have done seems sensible.

Treading very carefully around the creeping-featuritis trap (:-)), it does
seem that if one of the user options from xen/Rules.mk is going to be
converted then they all ought to be. Perhaps that's a topic for a followup
series though.

Ian.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Skylake: VT-d and other error messages

2015-12-02 Thread Andrew Cooper
On 02/12/15 02:51, Eric Shelton wrote:
> ASRock Z170 Extreme 4 with BIOS version 2.30 was motherboard #1
> Gigabyte GA-Z170XP-SLI with BIOS version F5 was motherboard #2
>
> As I initially reported, both were reporting the error involving f0:1f.0
>
> Not surprisingly, there are no f0:xx.y devices.  The closest I suppose is:
> 00:1f.0 ISA Bridge: Intel Corporation Sunrise Point-H LPC Controller
> (rev 31)
>
> At the moment, I am back on vanilla 4.6.0, and no such error messages
> are being reported via 'xl dmesg'.  I am guessing reporting of this
> error was added post-4.6.
>
> Eric

Nothing obvious comes to mind which would have this effect.

Can you provide `lspci -tv` from one of the affected systems?

If the platform is generating interrupts from a non-existent device,
something is quite broken.

~Andrew
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Build problems with xen 4.7

2015-12-02 Thread Jan Beulich
>>> On 01.12.15 at 19:47,  wrote:
> On Tue, 1 Dec 2015, Jan Beulich wrote:
>> I.e. there must be something different in how make gets invoked or
>> the environment set up.
> 
> It happens if CFLAGS is set to anything as a environment variable, eg.
> export CFLAGS=" "
> make dist-xen

Ugly, but I think it could be worked around (if we really want to
support CFLAGS coming in from the command line), along the lines
of

ABC_ORIG := $(ABC)
unexport ABC
MAKE := ABC=$(ABC_ORIG) $(MAKE)
ABC += xyz

all:
@echo $@: 'ABC=$(ABC)' "ABC=$$ABC"
$(MAKE) test

test:
@echo $@: 'ABC=$(ABC)' "ABC=$$ABC"

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] linux 4.4 Regression: 100% cpu usage on idle pv guest under Xen with single vcpu.

2015-12-02 Thread Sander Eikelenboom

On 2015-12-02 00:41, Boris Ostrovsky wrote:

On 12/01/2015 06:30 PM, Sander Eikelenboom wrote:

On 2015-12-02 00:19, Boris Ostrovsky wrote:

On 12/01/2015 06:00 PM, Sander Eikelenboom wrote:

On 2015-12-01 23:47, Boris Ostrovsky wrote:

On 11/30/2015 05:55 PM, Sander Eikelenboom wrote:

On 2015-11-30 23:54, Boris Ostrovsky wrote:

On 11/30/2015 04:46 PM, Sander Eikelenboom wrote:

On 2015-11-30 22:45, Konrad Rzeszutek Wilk wrote:
On Sat, Nov 28, 2015 at 04:47:43PM +0100, Sander Eikelenboom 
wrote:

Hi all,

I have just tested a 4.4-rc2 kernel (current linus tree) + the 
tip tree

pulled on top.

Running this kernel under Xen on PV-guests with multiple vcpus 
goes well (on

idle < 10% cpu usage),
but a guest with only a single vcpu doesn't idle at all, it 
seems a kworker

thread is stuck:
root   569 98.0  0.0  0 0 ?R 16:02 12:47
[kworker/0:1]

Running a 4.3 kernel works fine with a single vpcu, bisecting 
would probably
quite painful since there were some breakages this merge 
window with respect

to Xen pv-guests.

There are some differences in the diff's from booting a 4.3, 
4.4-single,

4.4-multi cpu boot:


Boris has been tracking a bunch of them. I am attaching the 
latest set of

patches I've to carry on top of v4.4-rc3.


Hi Konrad,

i will test those, see if it fixes all my issues and report back


They shouldn't help you ;-( (and I just saw a message from you 
confirming this)


The first one fixes a 32-bit bug (on bare metal too). The second 
fixes

a fatal bug for 32-bit PV guests. The other two are code
improvements/cleanup.


One of these patches also fixes a bug i was having with a 
pci-passthrough device in
a HVM that wasn't working (depending on which dom0-kernel i was 
using (4.3 or 4.4)),

but didn't report yet.

Fingers crossed but i think this pv-guest single vcpu issue is the 
last i'm troubled by for now ;)


I could not reproduce this, including with your kernel config file.


Hmm that's unpleasant :-\

Hmm other strange thing is it doesn't seem to affect dom0 (which is 
also a PV guest), but only unprivileged ones
All unprivileged pv-guests seem to have the irq issue, but only with 
a single vcpu i see to get the stuck kworker thread that got my 
attention, with a 2 vcpu that doesn't seem to happen, but you still 
get the dmesg output and warnings about hvc)


Could it be that:

arch/x86/include/asm/i8259.h
static inline int nr_legacy_irqs(void)
{
return legacy_pic->nr_legacy_irqs;
}

returns something different in some circumstances ?


It should return 16 pre-8c058b0b9c34d8c8d7912880956543769323e2d8 and 
0

after that commit.

This is the last number that you see in
NR_IRQS:4352 nr_irqs:48 0
line.

I think you should be able to safely revert both
b4ff8389ed14b849354b59ce9b360bdefcdbf99c and
8c058b0b9c34d8c8d7912880956543769323e2d8 and see if it makes any
difference.


-boris



That was already underway compiling :)

And it does reveal that reverting both fixes the issue, no stuck 
kworker thread .. and no:
   genirq: Flags mismatch irq 8.  (hvc_console) vs.  
(rtc0)

   hvc_open: request_irq failed with rc -16.



Let me try it again tomorrow. Can you post your guest config file, Xen
version and host HW (Intel or AMD)? 'xl info' maybe?

-boris


Hi Boris,

A fresh new day .. a fresh new thought.
If i look at the /proc/interrupts from a broken and a kernel with both 
commits the
thing that catches the eye is irq8, just as the dmesg message was 
telling.


In my PV guest rtc0 now seems to try and take irq8 that was already 
assigned to HVC ?
Sounds like some assumptions around the legacy range are broken 
somewhere.


What is the benefit of not just reserving the legacy range ?

Attached the /proc/interrupts from both boots.

--
Sander






What i did get was an conflict reverting 
b4ff8389ed14b849354b59ce9b360bdefcdbf99c:
arch/arm64/include/asm/irq.h, although that shouldn't matter because 
we are on x86 and not on arm.


-- Sander




-- Sander



-boris


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel   CPU0   
 16: 315536  xen-percpu-virq  timer0
 17:  0  xen-percpu-ipi   spinlock0
 18:  0  xen-percpu-ipi   resched0
 19:  0  xen-percpu-ipi   callfunc0
 20:  0  xen-percpu-virq  debug0
 21:  0  xen-percpu-ipi   callfuncsingle0
 22:  0  xen-percpu-ipi   irqwork0
 23:346   xen-dyn-event xenbus
 24:134   xen-dyn-event hvc_console
 25:  11464   xen-dyn-event blkif
 26:  28710   xen-dyn-event eth0-q0-tx
 27:  40136   xen-dyn-event eth0-q0-rx
NMI:  0   Non-maskable interrupts
LOC:  0   Local timer interrupts
SPU:  0   Spurious interrupts
PMI:  0   Performance monitoring interrupts
IWI:  0   IRQ work interrupts
RTR:  0   APIC ICR read retries
RES:  0   Rescheduling in

[Xen-devel] [xen-4.6-testing test] 65283: regressions - FAIL

2015-12-02 Thread osstest service owner
flight 65283 xen-4.6-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/65283/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 16 
guest-localmigrate/x10 fail in 65210 REGR. vs. 63449

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-xl-rtds 15 guest-start.2  fail in 65227 pass in 65283
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 13 guest-localmigrate 
fail in 65241 pass in 65283
 test-armhf-armhf-xl-rtds 11 guest-startfail in 65256 pass in 65283
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 15 guest-localmigrate.2 
fail pass in 65210
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 13 guest-localmigrate 
fail pass in 65227
 test-amd64-i386-rumpuserxen-i386 15 
rumpuserxen-demo-xenstorels/xenstorels.repeat fail pass in 65241
 test-amd64-amd64-rumpuserxen-amd64 15 
rumpuserxen-demo-xenstorels/xenstorels.repeat fail pass in 65256

Regressions which are regarded as allowable (not blocking):
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 15 guest-localmigrate.2 
fail in 65227 like 63379
 test-armhf-armhf-xl-rtds 16 guest-start/debian.repeatfail   like 63449

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stop fail never pass
 test-armhf-armhf-libvirt-raw  9 debian-di-installfail   never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-armhf-armhf-libvirt-qcow2  9 debian-di-installfail never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-vhd   9 debian-di-installfail   never pass
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-amd64-i386-libvirt-xsm  12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 guest-saverestorefail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-xsm  12 migrate-support-checkfail   never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop fail never pass
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop  fail never pass
 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop  fail never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  78833c04250416f1870c458309d3ac0e5cf915fd
baseline version:
 xen  40d7a7454835c2f7c639c78f6c09e7b6f0e4a4e2

Last test of basis63449  2015-11-01 10:09:20 Z   31 days
Failing since 64055  2015-11-10 11:39:11 Z   21 days   20 attempts
Testing same since64935  2015-11-20 02:51:37 Z   12 days   14 attempts


People who touched revisions under test:
  Andrew Cooper 
  Ian Campbell 
  Ian Jackson 
  Jan Beulich 
  Kevin Tian 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  

Re: [Xen-devel] Regression: Xen guest with 5G of RAM on 32bit fail to boot

2015-12-02 Thread Paolo Bonzini


On 01/12/2015 18:53, Anthony PERARD wrote:
> The problem is in qemu_ram_alloc_internal() where 'size' and 'maxsize' are
> now been truncate to 32bit, due to 'qemu_host_page_size' been an uintptr_t
> in the HOST_PAGE_ALIGN macro.

Isn't it qemu_host_page_mask that causes the problem?

This should also work, as it causes qemu_host_page_mask to sign-extend:

diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
index f9998b9..87a4145 100644
--- a/include/exec/cpu-all.h
+++ b/include/exec/cpu-all.h
@@ -174,11 +174,10 @@ extern unsigned long reserved_va;
 #define TARGET_PAGE_MASK ~(TARGET_PAGE_SIZE - 1)
 #define TARGET_PAGE_ALIGN(addr) (((addr) + TARGET_PAGE_SIZE - 1) & 
TARGET_PAGE_MASK)
 
-/* ??? These should be the larger of uintptr_t and target_ulong.  */
 extern uintptr_t qemu_real_host_page_size;
-extern uintptr_t qemu_real_host_page_mask;
+extern intptr_t qemu_real_host_page_mask;
 extern uintptr_t qemu_host_page_size;
-extern uintptr_t qemu_host_page_mask;
+extern intptr_t qemu_host_page_mask;
 
 #define HOST_PAGE_ALIGN(addr) (((addr) + qemu_host_page_size - 1) & 
qemu_host_page_mask)
 #define REAL_HOST_PAGE_ALIGN(addr) (((addr) + qemu_real_host_page_size - 1) & \
diff --git a/translate-all.c b/translate-all.c
index a940bd2..7a15109 100644
--- a/translate-all.c
+++ b/translate-all.c
@@ -118,7 +118,7 @@ typedef struct PageDesc {
 #define V_L1_SHIFT (L1_MAP_ADDR_SPACE_BITS - TARGET_PAGE_BITS - V_L1_BITS)
 
 uintptr_t qemu_host_page_size;
-uintptr_t qemu_host_page_mask;
+intptr_t qemu_host_page_mask;
 
 /* The bottom level has pointers to PageDesc */
 static void *l1_map[V_L1_SIZE];
@@ -326,14 +326,14 @@ void page_size_init(void)
 /* NOTE: we can always suppose that qemu_host_page_size >=
TARGET_PAGE_SIZE */
 qemu_real_host_page_size = getpagesize();
-qemu_real_host_page_mask = ~(qemu_real_host_page_size - 1);
+qemu_real_host_page_mask = -(intptr_t)qemu_real_host_page_size;
 if (qemu_host_page_size == 0) {
 qemu_host_page_size = qemu_real_host_page_size;
 }
 if (qemu_host_page_size < TARGET_PAGE_SIZE) {
 qemu_host_page_size = TARGET_PAGE_SIZE;
 }
-qemu_host_page_mask = ~(qemu_host_page_size - 1);
+qemu_host_page_mask = -(intptr_t)qemu_host_page_size;
 }
 
 static void page_init(void)
diff --git a/translate-common.c b/translate-common.c
index 619feb4..171222d 100644
--- a/translate-common.c
+++ b/translate-common.c
@@ -21,7 +21,7 @@
 #include "qom/cpu.h"
 
 uintptr_t qemu_real_host_page_size;
-uintptr_t qemu_real_host_page_mask;
+intptr_t qemu_real_host_page_mask;
 
 #ifndef CONFIG_USER_ONLY
 /* mask must never be zero, except for A20 change call */

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] xen-pciback: fix up cleanup path when alloc fails

2015-12-02 Thread David Vrabel
On 26/11/15 20:32, Doug Goldstein wrote:
> When allocating a pciback device fails, avoid the possibility of a
> use after free.

We should not require clearing drvdata for correctness.  We should
ensure we retain drvdata for as long as it is needed.

I note that pcistub_device_release() has:

kfree(dev_data);
pci_set_drvdata(dev, NULL);

/* Clean-up the device */
xen_pcibk_config_free_dyn_fields(dev);
xen_pcibk_config_free_dev(dev);

Which should (at a minimum) be reordered to move the kfree(dev_data) to
after the calls that require it

David

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [xen-unstable test] 65141: regressions - FAIL

2015-12-02 Thread Ian Campbell
On Mon, 2015-11-30 at 05:35 +, Hu, Robert wrote:

Also adding Vt-x maintainers (Kevin and Jun) for their help/input, I'm not
sure if there is a dedicated nested-vmx maintainer. This failure has been
blocking the xen-unstable push gate for a week now so it really does need
looking into.

Also CC the arch X86 maintainers, for-their-info.

> > -Original Message-
> > From: Ian Jackson [mailto:ian.jack...@eu.citrix.com]
> > Sent: Friday, November 27, 2015 11:54 PM
> > To: Hu, Robert 
> > Cc: xen-de...@lists.xensource.com; osstest service owner
> > 
> > Subject: Re: [xen-unstable test] 65141: regressions - FAIL
> > 
> > osstest service owner writes ("[xen-unstable test] 65141: regressions -
> > FAIL"):
> > > flight 65141 xen-unstable real [real]
> > > http://logs.test-lab.xenproject.org/osstest/logs/65141/
> > > 
> > > Regressions :-(
> > > 
> > > Tests which did not succeed and are blocking,
> > > including tests which could not be run:
> > >  test-amd64-amd64-qemuu-nested-intel 16 debian-hvm-install/l1/l2 fail
> > REGR. vs. 65114
> > 
> > Hi, Robert.  I hope you don't mind me asking you about these Nested
> > HVM tests in osstest production; if there's someone else we should
> > contact please let us know.
> [Hu, Robert] 
> 
> I'm trying to look for...
> But can I get the bad commit first?

It is in the original mail report, which is in the ML archives:
http://lists.xenproject.org/archives/html/xen-devel/2015-11/msg03222.html

version targeted for testing:
 xen  b1d398b67781140d1c6efd05778d0ad4103b2a32
baseline version:
 xen  713b7e4ef2aa4ec3ae697cde9c81d5a57548f9b1

You can also find it via the logs at
 http://logs.test-lab.xenproject.org/osstest/logs/65141
by clicking the header of one of the build-$arch jobs e.g. 
http://logs.test-lab.xenproject.org/osstest/logs/65141/build-amd64/info.html

and looking at the various tree_* and revision_* in the "Test control
variables" table, e.g.:
revision_xenb1d398b67781140d1c6efd05778d0ad4103b2a32
definition

> > Anyway: it seems this test failed just now.
> > 
> > osstest thinks it is a regression, but I think it is more likely that
> > this test either exhibits a heisenbug, or that there is some problem
> > which is specific to the particular host.

FWIW looking at the different branches in
 
http://logs.test-lab.xenproject.org/osstest/results/history/test-amd64-amd64-qemuu-nested-intel/

While xen-unstable has started failing on godello1 xen-4.6-testing does
seem to be passing there.

The history for the previous job name (before splitting into -intel and
-amd) also shows a pass on godello1:

http://logs.test-lab.xenproject.org/osstest/results/history/test-amd64-amd64-qemuu-nested/xen-unstable.html

Of course this is a new test so there isn't very much historical data to
draw conclusions from.

> > We'd appreciate it if you and your colleagues could take a look at
> > this and analyse the failure.
> > 
> > In the meantime the osstest bisector will try to start work on it and
> > I will report what it discovers.

According to 
http://osstest.test-lab.xenproject.org/~osstest/pub/results/bisect/xen-unstable/test-amd64-amd64-qemuu-nested-intel.debian-hvm-install--l1--l2.html
it was unable to reproduce a baseline, probably because it didn't have
enough historical data. 

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCHv6] 01/28] build: import Kbuild/Kconfig from Linux 4.2

2015-12-02 Thread Ian Campbell
On Tue, 2015-12-01 at 04:00 -0700, Jan Beulich wrote:
> But not forking doesn't mean importing the whole directory. Some
> Makefile adjustments will be necessary anyway, so removing some
> of the frontends wouldn't make things worse. We indeed should
> avoid editing files we import, but I don't see any bad in skipping
> some of the files.

It is much easier to resync based on entire directories (my preference, if
possible) than to fiddle around picking up individual files, deciding if a
"new" file is actually new or just excluded last time for some reason etc.

> Reasons I'd like to avoid importing everything:
> - no introduction of new build dependencies (one of the frontends
> being written in C++ is the mildest form, requiring HOSTCXX to be
> set),

Only if someone wants to use it. I see no harm in having such a frontend
optionally available to those who are willing to meet its prerequisites in
their build environment, that certainly doesn't mean it has to work for
everyone nor that we should add a C++ compiler and QT environment to the
standard set of Xen build deps.

I believe this is the policy in the Linux tree too.

> - limiting the amount of new code that needs maintaining (no
> matter how simple a re-import, it still is work, and hence the less
> frequently we need to do this, the better; I assume you agree
> that the likelihood for changes that we want/need to pull in
> grows with the size of the code, and with what I propose the
> import size would shrink by almost 50%), unless you or Doug
> volunteer to look after this code on a regular basis,

I disagree with the (implied) conclusion here that you would somehow be
personally on the hook for these regular updates if we would import only
50% of this kconfig code base, nor that there would therefore be some sort
of additional personal burden on you if we take the whole thing. We should
be arranging that the maintenance burden is ~0 by rejecting diversion and
making the reimport process as brain-dead as possible.

Nonetheless if you don't want this to formally come under the remit of "THE
REST" then I'd happy to see an entry for xen/tools/kconfig in the
MAINTAINERS listing whoever wants to step up (Doug, Ian and/or myself).

But I honestly don't think this code is going to require much maintenance
at all on our end, apart from the very occasional reimporting of the whole
thing from Linux when we notice some major missing feature we care about,
which is the case that Ian and I are arguing we should optimise for.

And having put aside suggestions such as renaming Kconfig to Xconfig
throughout I also don't foresee making very many large changes to this code
base at all, there's simply no reason to do so, at least none which can't
be pushed back on. At worst I would expect to see generic Kconfig feature
requests which should redirected upstream and the results reimported.

I think this is all backed up by the fact that after this initial import
the remainder of this series consists of:

$ git fetch https://github.com/cardoe/xen kconfig_v6
From https://github.com/cardoe/xen
 * branchkconfig_v6 -> FETCH_HEAD
$ git diff --stat a28f2c4~1 FETCH_HEAD -- xen/scripts/kconfig
 xen/scripts/kconfig/.gitignore |  3 +++
 xen/scripts/kconfig/Makefile   | 77 
+
 2 files changed, 80 insertions(+)

(The first of which seems like it ought to be fixed upstream instead and it
might even be possible to avoid the latter and therefore avoid the
consequential renaming of the upstream Makefile => Makefile.linux by using
xen/scripts/Makefile.kconfig somehow).

> - avoiding code (scripts) that seem completely pointless in our
> environment (e.g. streamline_config.pl).

I think the overhead of a few extra files of marginal usefulness is far
outweighed by being able to just resync a whole directory.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] Seeking input for 2016 Advisory Board budget (needed by Friday)

2015-12-02 Thread Lars Kurth
Hi everyone,
I am putting together the Advisory Board budget for 2016. Besides items that 
are operational and spending on a new HW for the Test Lab, are there any items 
which I should add that help the developer community. Things that have been 
raised in the past, were
* Easier hosting of personal git repos on a git hosting service
* Development of code standards checking tools
* Other tools (e.g. fixing up maintainers.pkl and similar)
Feel free to let me know some ideas, with if possible links to some suppliers. 
I will need to have a draft budget together by next Tuesday. So I do need input 
by Friday.
Cheers
Lars
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Regression: Xen guest with 5G of RAM on 32bit fail to boot

2015-12-02 Thread Dr. David Alan Gilbert
* Paolo Bonzini (pbonz...@redhat.com) wrote:
> 
> 
> On 01/12/2015 18:53, Anthony PERARD wrote:
> > The problem is in qemu_ram_alloc_internal() where 'size' and 'maxsize' are
> > now been truncate to 32bit, due to 'qemu_host_page_size' been an uintptr_t
> > in the HOST_PAGE_ALIGN macro.
> 
> Isn't it qemu_host_page_mask that causes the problem?
> 
> This should also work, as it causes qemu_host_page_mask to sign-extend:

I've just posted a set that just flips them to ram_addr_t (and removes the 10 
year
old warning that questions whether the type is right - which it obviously 
wasn't);
see '[For 2.5?? PATCH 1/1] qemu_{real_}host_page_[size|mask] change types to
 ram_addr_t'

Anthony: I'd appreciate if you could give it a Xen test.

Dave

> diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
> index f9998b9..87a4145 100644
> --- a/include/exec/cpu-all.h
> +++ b/include/exec/cpu-all.h
> @@ -174,11 +174,10 @@ extern unsigned long reserved_va;
>  #define TARGET_PAGE_MASK ~(TARGET_PAGE_SIZE - 1)
>  #define TARGET_PAGE_ALIGN(addr) (((addr) + TARGET_PAGE_SIZE - 1) & 
> TARGET_PAGE_MASK)
>  
> -/* ??? These should be the larger of uintptr_t and target_ulong.  */
>  extern uintptr_t qemu_real_host_page_size;
> -extern uintptr_t qemu_real_host_page_mask;
> +extern intptr_t qemu_real_host_page_mask;
>  extern uintptr_t qemu_host_page_size;
> -extern uintptr_t qemu_host_page_mask;
> +extern intptr_t qemu_host_page_mask;
>  
>  #define HOST_PAGE_ALIGN(addr) (((addr) + qemu_host_page_size - 1) & 
> qemu_host_page_mask)
>  #define REAL_HOST_PAGE_ALIGN(addr) (((addr) + qemu_real_host_page_size - 1) 
> & \
> diff --git a/translate-all.c b/translate-all.c
> index a940bd2..7a15109 100644
> --- a/translate-all.c
> +++ b/translate-all.c
> @@ -118,7 +118,7 @@ typedef struct PageDesc {
>  #define V_L1_SHIFT (L1_MAP_ADDR_SPACE_BITS - TARGET_PAGE_BITS - V_L1_BITS)
>  
>  uintptr_t qemu_host_page_size;
> -uintptr_t qemu_host_page_mask;
> +intptr_t qemu_host_page_mask;
>  
>  /* The bottom level has pointers to PageDesc */
>  static void *l1_map[V_L1_SIZE];
> @@ -326,14 +326,14 @@ void page_size_init(void)
>  /* NOTE: we can always suppose that qemu_host_page_size >=
> TARGET_PAGE_SIZE */
>  qemu_real_host_page_size = getpagesize();
> -qemu_real_host_page_mask = ~(qemu_real_host_page_size - 1);
> +qemu_real_host_page_mask = -(intptr_t)qemu_real_host_page_size;
>  if (qemu_host_page_size == 0) {
>  qemu_host_page_size = qemu_real_host_page_size;
>  }
>  if (qemu_host_page_size < TARGET_PAGE_SIZE) {
>  qemu_host_page_size = TARGET_PAGE_SIZE;
>  }
> -qemu_host_page_mask = ~(qemu_host_page_size - 1);
> +qemu_host_page_mask = -(intptr_t)qemu_host_page_size;
>  }
>  
>  static void page_init(void)
> diff --git a/translate-common.c b/translate-common.c
> index 619feb4..171222d 100644
> --- a/translate-common.c
> +++ b/translate-common.c
> @@ -21,7 +21,7 @@
>  #include "qom/cpu.h"
>  
>  uintptr_t qemu_real_host_page_size;
> -uintptr_t qemu_real_host_page_mask;
> +intptr_t qemu_real_host_page_mask;
>  
>  #ifndef CONFIG_USER_ONLY
>  /* mask must never be zero, except for A20 change call */
--
Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Skylake: VT-d and other error messages

2015-12-02 Thread Jan Beulich
>>> On 02.12.15 at 03:51,  wrote:
> ASRock Z170 Extreme 4 with BIOS version 2.30 was motherboard #1
> Gigabyte GA-Z170XP-SLI with BIOS version F5 was motherboard #2
> 
> As I initially reported, both were reporting the error involving f0:1f.0
> 
> Not surprisingly, there are no f0:xx.y devices.  The closest I suppose is:
> 00:1f.0 ISA Bridge: Intel Corporation Sunrise Point-H LPC Controller (rev
> 31)

Did you also check "iommu=verbose" output for occurrences of this
ID (e.g. representing an IO-APIC or HPET)?

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [xen-unstable test] 65141: regressions - FAIL

2015-12-02 Thread Jan Beulich
>>> On 02.12.15 at 11:34,  wrote:
> On Mon, 2015-11-30 at 05:35 +, Hu, Robert wrote:
> 
> Also adding Vt-x maintainers (Kevin and Jun) for their help/input, I'm not
> sure if there is a dedicated nested-vmx maintainer. This failure has been
> blocking the xen-unstable push gate for a week now so it really does need
> looking into.
> 
> Also CC the arch X86 maintainers, for-their-info.

I was actually waiting for the bisector to point at something. Also
looking at the history this might be machine specific (the sole
initial pass was on italia0, all failures are on godello1). Should new
tests perhaps be capable of causing regressions only once they
passed on every host they may (usefully) run on?

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] xen 4.5.0 rtds scheduler perform poorly with 2vms

2015-12-02 Thread Wei Liu
On Tue, Dec 01, 2015 at 11:54:14PM -0600, Meng Xu wrote:
> Hi Lars and Dario,
> 
> 2015-12-01 4:11 GMT-06:00 Lars Kurth :
> >
> > I wonder whether we need to add some health warnings and recommended 
> > background reading to http://wiki.xenproject.org/wiki/RTDS-Based-Scheduler
> 
> 
> Maybe we could add some health warning and add a link to this discussion?
> Misconfiguration of the system will usually cause performance
> degradation, even for the other schedulers, such as ARINC653, credit,
> credit2.
> What I'm thinking is how much expert information we should expose to
> users. Sometimes, exposing too much information may not be so helpful.
> Sometimes, more information just  cause more confusion.
> 
> What do you guys think which type of information we should include?
> 

Wiki and xl manpage ?

Wei.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [xen-unstable test] 65141: regressions - FAIL

2015-12-02 Thread Andrew Cooper
On 02/12/15 10:52, Jan Beulich wrote:
 On 02.12.15 at 11:34,  wrote:
>> On Mon, 2015-11-30 at 05:35 +, Hu, Robert wrote:
>>
>> Also adding Vt-x maintainers (Kevin and Jun) for their help/input, I'm not
>> sure if there is a dedicated nested-vmx maintainer. This failure has been
>> blocking the xen-unstable push gate for a week now so it really does need
>> looking into.
>>
>> Also CC the arch X86 maintainers, for-their-info.
> I was actually waiting for the bisector to point at something. Also
> looking at the history this might be machine specific (the sole
> initial pass was on italia0, all failures are on godello1). Should new
> tests perhaps be capable of causing regressions only once they
> passed on every host they may (usefully) run on?

As a general improvement, OSSTest should not make new tests blocking by
default.  They should need to be proved stable (5 consecutive passes?)
before they become blocking, to prevent a new test spuriously passing
and subsequently blocking pushes.

During this time, the author of the new test has the onus to ensure test
stability; either modifications to the test, or bugfixes to master.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] xen 4.5.0 rtds scheduler perform poorly with 2vms

2015-12-02 Thread Lars Kurth

> On 2 Dec 2015, at 05:54, Meng Xu  wrote:
> 
> Hi Lars and Dario,
> 
> 2015-12-01 4:11 GMT-06:00 Lars Kurth :
>> 
>> I wonder whether we need to add some health warnings and recommended 
>> background reading to http://wiki.xenproject.org/wiki/RTDS-Based-Scheduler
> 
> 
> Maybe we could add some health warning and add a link to this discussion?
> Misconfiguration of the system will usually cause performance
> degradation, even for the other schedulers, such as ARINC653, credit,
> credit2.

That is the minimum IMHO for the wiki and xl.

> What I'm thinking is how much expert information we should expose to
> users. Sometimes, exposing too much information may not be so helpful.
> Sometimes, more information just  cause more confusion.

Maybe just link to some of the theory on real-time schedulers (at least on the 
wiki). Maybe also to a couple of links to xen-devel@ threads like this one.

> What do you guys think which type of information we should include?

I am not the expert, but I get the point on too much information. 

Lars
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] xen 4.5.0 rtds scheduler perform poorly with 2vms

2015-12-02 Thread Ian Campbell
On Tue, 2015-12-01 at 23:54 -0600, Meng Xu wrote:
> Hi Lars and Dario,
> 
> 2015-12-01 4:11 GMT-06:00 Lars Kurth :
> > 
> > I wonder whether we need to add some health warnings and recommended
> > background reading to http://wiki.xenproject.org/wiki/RTDS-Based-Schedu
> > ler
> 
> 
> Maybe we could add some health warning and add a link to this discussion?
> Misconfiguration of the system will usually cause performance
> degradation, even for the other schedulers, such as ARINC653, credit,
> credit2.
>
> What I'm thinking is how much expert information we should expose to
> users. Sometimes, exposing too much information may not be so helpful.
> Sometimes, more information just  cause more confusion.
> 
> What do you guys think which type of information we should include?

I think there is an important distinction between credit2/credit and RT
schedulers such as arinc/rtds etc, which is that the former should just
work out of the box with no tweaking at all whereas the latter in general
need some sort of "intelligent input/configuration" to even begin using
them and have GIGO properties wrt their parameters.

(That's not to say you can't tweak credit* etc and break it if you want, but 
one typically doesn't need to start doing so just to get something running at 
all).

And AIUI the "intelligent input/configuration" requires some amount of 
background in RT scheduling, else you can get pathological results and think 
the scheduler and/or Xen is broken.

So I think some sort of warning that the RT schedulers do not "just work" and 
require one to understand the properties of your workloads and the schedulers 
and to feed them non-garbage inputs would be a useful to people who might 
otherwise expect to just "xl create" (maybe with some random inputs to the 
required settings) and have a useful result.

Having given that warning I don't think some links to some relevant background 
RT stuff would be too much info, neither would the inclusion of some specifics 
about the specific algorithm. After all that background and info is critical to 
being able to run a system using those schedulers, isn't it?

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [V2 PATCH 1/9] x86/hvm: pkeys, add pkeys support for cpuid handling

2015-12-02 Thread Jan Beulich
>>> On 01.12.15 at 21:03,  wrote:
> On 27/11/15 09:51, Huaitong Han wrote:
>> --- a/xen/arch/x86/hvm/hvm.c
>> +++ b/xen/arch/x86/hvm/hvm.c
>> @@ -4582,6 +4582,18 @@ void hvm_cpuid(unsigned int input, unsigned int *eax, 
>> unsigned int *ebx,
>>  /* Don't expose INVPCID to non-hap hvm. */
>>  if ( (count == 0) && !hap_enabled(d) )
>>  *ebx &= ~cpufeat_mask(X86_FEATURE_INVPCID);
>> +
>> +/* X86_FEATURE_PKU is not yet implemented for shadow paging
>> + *
>> + * Hypervisor gets guest pkru value from XSAVE state, because
>> + * Hypervisor CR4 without X86_CR4_PKE disables RDPKRU instruction.
>> + */
>> +if ( (count == 0) && (!hap_enabled(d) || !cpu_has_xsave) )
>> +*ecx &= ~cpufeat_mask(X86_FEATURE_PKU);
>> +
>> +if ( (count == 0) && cpu_has_pku )
>> +*ecx |= (v->arch.hvm_vcpu.guest_cr[4] & X86_CR4_PKE) ?
>> + cpufeat_mask(X86_FEATURE_OSPKE) : 0;
> 
> This is still buggy.  cpu_has_pku has no relevance to whether OSPKE
> becomes visible.
> 
> Visibility of OSPKE is determined solely by v->arch.hvm_vcpu.guest_cr[4]
> & X86_CR4_PKE and nothing else.

Actually I wouldn't mind guarding against the case where the CR4 flag
is wrongly set for whatever reason, but that ought to check the PKU
bit in *ecx, not the host flag. Same applies to the cpu_has_xsave
check - this too should check the guest flag, not the host one.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [xen-unstable test] 65141: regressions - FAIL

2015-12-02 Thread Ian Campbell
On Wed, 2015-12-02 at 03:52 -0700, Jan Beulich wrote:
> > > > On 02.12.15 at 11:34,  wrote:
> > On Mon, 2015-11-30 at 05:35 +, Hu, Robert wrote:
> > 
> > Also adding Vt-x maintainers (Kevin and Jun) for their help/input, I'm
> > not
> > sure if there is a dedicated nested-vmx maintainer. This failure has
> > been
> > blocking the xen-unstable push gate for a week now so it really does
> > need
> > looking into.
> > 
> > Also CC the arch X86 maintainers, for-their-info.
> 
> I was actually waiting for the bisector to point at something. Also
> looking at the history this might be machine specific (the sole
> initial pass was on italia0, all failures are on godello1). Should new
> tests perhaps be capable of causing regressions only once they
> passed on every host they may (usefully) run on?

We renamed the test (to add an -intel/-amd suffix and run on an appropriate
host) so

 
http://logs.test-lab.xenproject.org/osstest/results/history/test-amd64-amd64-qemuu-nested/xen-unstable.html

is also relevant, rimava is an amd machine (which lead to the rename/split.

That older named test also shows a pass on godello1. That initial pass on
godello1 might have been a fluke, but the current run of seven failures
seems unlikely to be a fluke in the opposite direction.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [V2 PATCH 7/9] x86/hvm: pkeys, add pkeys support for guest_walk_tables

2015-12-02 Thread Jan Beulich
>>> On 02.12.15 at 09:23,  wrote:
> On Tue, 2015-12-01 at 20:30 +, Andrew Cooper wrote:
>> > +#include 
>> 
>> I can see why you need xstate.h, but I why do you need i387.h ?
> Use vcpu_save_fpu functions.

Which is bogus by itself: That function isn't meant to be used for
purposes like the one you have, e.g. due to its side effect of
clearing ->fpu_dirtied. You really ought to be using a lower level
function here (and I don't think the corresponding struct vcpu
should get altered in any way).

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [V2 PATCH 7/9] x86/hvm: pkeys, add pkeys support for guest_walk_tables

2015-12-02 Thread Jan Beulich
>>> On 27.11.15 at 10:52,  wrote:
> @@ -90,6 +92,53 @@ static uint32_t set_ad_bits(void *guest_p, void *walk_p, 
> int set_dirty)
>  return 0;
>  }
>  
> +#if GUEST_PAGING_LEVELS >= 4
> +uint32_t leaf_pte_pkeys_check(struct vcpu *vcpu, uint32_t pfec,
> +uint32_t pte_access, uint32_t pte_pkeys)
> +{
> +bool_t pkru_ad, pkru_wd;
> +bool_t ff, wf, uf, rsvdf, pkuf;
> +unsigned int pkru = 0;
> +
> +uf = pfec & PFEC_user_mode;
> +wf = pfec & PFEC_write_access;
> +rsvdf = pfec & PFEC_reserved_bit;
> +ff = pfec & PFEC_insn_fetch;
> +pkuf = pfec & PFEC_prot_key;

None of these operations yield valid bool_t values. All of them
should be using !!(), and readability would imo benefit if you
made the expressions initializers instead of assignments.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [V2 PATCH 1/9] x86/hvm: pkeys, add pkeys support for cpuid handling

2015-12-02 Thread Jan Beulich
>>> On 27.11.15 at 10:51,  wrote:
> --- a/tools/libxc/xc_cpuid_x86.c
> +++ b/tools/libxc/xc_cpuid_x86.c
> @@ -421,9 +421,11 @@ static void xc_cpuid_hvm_policy(xc_interface *xch,
>  bitmaskof(X86_FEATURE_ADX)  |
>  bitmaskof(X86_FEATURE_SMAP) |
>  bitmaskof(X86_FEATURE_FSGSBASE));
> +regs[2] &= bitmaskof(X86_FEATURE_PKU);
>  } else
> -regs[1] = 0;
> -regs[0] = regs[2] = regs[3] = 0;
> +regs[1] = regs[2] = 0;
> +
> +regs[0] = regs[3] = 0;
>  break;

This is not a valid change until the hypervisor side implementation is
complete.

Also you should Cc tools maintainers on tools changes.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] xen 4.5.0 rtds scheduler perform poorly with 2vms

2015-12-02 Thread Dario Faggioli
On Wed, 2015-12-02 at 11:03 +, Lars Kurth wrote:
> > On 2 Dec 2015, at 05:54, Meng Xu  wrote:
> > 
> > Maybe we could add some health warning and add a link to this
> > discussion?
> > Misconfiguration of the system will usually cause performance
> > degradation, even for the other schedulers, such as ARINC653,
> > credit,
> > credit2.
> 
> That is the minimum IMHO for the wiki and xl.
> 
I'm not so sure. Perhaps just a quick hint, in xl manpage, I'd say.

> > What I'm thinking is how much expert information we should expose
> > to
> > users. Sometimes, exposing too much information may not be so
> > helpful.
> > Sometimes, more information just  cause more confusion.
> 
> Maybe just link to some of the theory on real-time schedulers (at
> least on the wiki). Maybe also to a couple of links to xen-devel@
> threads like this one.
> 
We can reference the thread on the wiki, and we can put a few sentences
about all this there. However, trying to link "some relevant theory" is
unpractical, as it won't be easy to define what both "some" and
"relevant" mean. :-)

I'd be in favour (and ca do so) of adding a reference to the RT-Xen
website, and maybe to one of (the original?) RT-Xen paper. But nothing
more.

Regards,
Dario
-- 
<> (Raistlin Majere)
-
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)



signature.asc
Description: This is a digitally signed message part
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [V2 PATCH 2/9] x86/hvm: pkeys, add the flag to enable Memory Protection Keys

2015-12-02 Thread Jan Beulich
>>> On 27.11.15 at 10:51,  wrote:
> @@ -1307,6 +1311,9 @@ void __init noreturn __start_xen(unsigned long mbi_p)
>  if ( cpu_has_smap )
>  set_in_cr4(X86_CR4_SMAP);
>  
> +if ( !opt_pku )
> +setup_clear_cpu_cap(X86_FEATURE_PKU);
> +
>  if ( cpu_has_fsgsbase )
>  set_in_cr4(X86_CR4_FSGSBASE);

I don't think this is a good place to do this. It really belongs in
cpu/common.c, e.g. next to the xsave disabling. In no case
does it belong in the middle of a series of CR4 updates.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [V2 PATCH 5/9] x86/hvm: pkeys, add functions to get pkeys value from PTE

2015-12-02 Thread Jan Beulich
>>> On 27.11.15 at 10:51,  wrote:
> --- a/xen/include/asm-x86/x86_64/page.h
> +++ b/xen/include/asm-x86/x86_64/page.h
> @@ -134,6 +134,25 @@ typedef l4_pgentry_t root_pgentry_t;
>  #define get_pte_flags(x) (((int)((x) >> 40) & ~0xFFF) | ((int)(x) & 0xFFF))
>  #define put_pte_flags(x) (((intpte_t)((x) & ~0xFFF) << 40) | ((x) & 0xFFF))
>  
> +/*
> + * Protection keys define a new 4-bit protection key field
> + * (PKEY) in bits 62:59 of leaf entries of the page tables.
> + * This corresponds to bit 22:19 of a 24-bit flags.
> + *
> + * Notice: Bit 22 is used by _PAGE_GNTTAB which is visible to PV guests,
> + * so Protection keys must be disabled on PV guests.
> + */
> +#define _PAGE_PKEY_BIT0 (1u<<19)   /* Protection Keys, bit 1/4 */
> +#define _PAGE_PKEY_BIT1 (1u<<20)   /* Protection Keys, bit 2/4 */
> +#define _PAGE_PKEY_BIT2 (1u<<21)   /* Protection Keys, bit 3/4 */
> +#define _PAGE_PKEY_BIT3 (1u<<22)   /* Protection Keys, bit 4/4 */
> +
> +/* The order of mask _PAGE_PKEY_BIT0 is 19 */
> +#define get_pte_pkeys(x) ((int)(get_pte_flags(x) >> 19) & 0xF)

Pointless cast.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] xen 4.5.0 rtds scheduler perform poorly with 2vms

2015-12-02 Thread Dario Faggioli
On Wed, 2015-12-02 at 11:01 +, Ian Campbell wrote:
> On Tue, 2015-12-01 at 23:54 -0600, Meng Xu wrote:
> > 
> > What do you guys think which type of information we should include?
> 
> I think there is an important distinction between credit2/credit and
> RT
> schedulers such as arinc/rtds etc, which is that the former should
> just
> work out of the box with no tweaking at all whereas the latter in
> general
> need some sort of "intelligent input/configuration" to even begin
> using
> them and have GIGO properties wrt their parameters.
> 
Exactly.

This is particularly true for the "issues" raised in this thread. In
fact, this has all been about 'schedulability'. Well, if you aim at
'schedulability', you (ought to) have the necessary RT background to
figure out what it takes to get there.


> And AIUI the "intelligent input/configuration" requires some amount
> of background in RT scheduling, else you can get pathological results
> and think the scheduler and/or Xen is broken.
> 
Yep.

> So I think some sort of warning that the RT schedulers do not "just
> work" and require one to understand the properties of your workloads
> and the schedulers and to feed them non-garbage inputs would be a
> useful to people who might otherwise expect to just "xl create"
> (maybe with some random inputs to the required settings) and have a
> useful result.
> 
Yeah, I guess we can add something like that. I will send a patch.

> Having given that warning I don't think some links to some relevant
> background RT stuff would be too much info, neither would the
> inclusion of some specifics about the specific algorithm. After all
> that background and info is critical to being able to run a system
> using those schedulers, isn't it?
> 
True, but I'd much rather avoid turning either xl manpage and/or our
wiki in a collection of references to real-time scheduling papers! As
said in the other mail, I really would try to limit all this, both in
terms of numbers and of complexity of the references themselves.

Regards,
Dario
-- 
<> (Raistlin Majere)
-
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)



signature.asc
Description: This is a digitally signed message part
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [V2 PATCH 7/9] x86/hvm: pkeys, add pkeys support for guest_walk_tables

2015-12-02 Thread Jan Beulich
>>> On 27.11.15 at 10:52,  wrote:
> @@ -90,6 +92,53 @@ static uint32_t set_ad_bits(void *guest_p, void *walk_p, 
> int set_dirty)
>  return 0;
>  }
>  
> +#if GUEST_PAGING_LEVELS >= 4
> +uint32_t leaf_pte_pkeys_check(struct vcpu *vcpu, uint32_t pfec,
> +uint32_t pte_access, uint32_t pte_pkeys)
> +{
> +bool_t pkru_ad, pkru_wd;
> +bool_t ff, wf, uf, rsvdf, pkuf;
> +unsigned int pkru = 0;

Pointless initializer.

> +uf = pfec & PFEC_user_mode;
> +wf = pfec & PFEC_write_access;
> +rsvdf = pfec & PFEC_reserved_bit;
> +ff = pfec & PFEC_insn_fetch;
> +pkuf = pfec & PFEC_prot_key;
> +
> +if ( !cpu_has_xsave || !pkuf || is_pv_vcpu(vcpu) )
> +return 0;
> +
> +vcpu_save_fpu(vcpu);
> +pkru = *(unsigned int*)get_xsave_addr(vcpu->arch.xsave_area, 
> XSTATE_PKRU);

I.e. you really want to _only_ save that one component.

> @@ -106,6 +155,7 @@ guest_walk_tables(struct vcpu *v, struct p2m_domain *p2m,
>  #if GUEST_PAGING_LEVELS >= 4 /* 64-bit only... */
>  guest_l3e_t *l3p = NULL;
>  guest_l4e_t *l4p;
> +uint32_t pkeys;

No reason I can see to use a fixed width type here - unsigned int
ought to be fine.

> @@ -199,6 +250,9 @@ guest_walk_tables(struct vcpu *v, struct p2m_domain *p2m,
>  
>  pse1G = (gflags & _PAGE_PSE) && guest_supports_1G_superpages(v); 
>  
> +if (pse1G && leaf_pte_pkeys_check(v, pfec, gflags, pkeys))

Coding style (more further down).

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [V2 PATCH 8/9] x86/hvm: pkeys, add xstate support for pkeys

2015-12-02 Thread Jan Beulich
>>> On 02.12.15 at 08:20,  wrote:
>> Does this even compile?  There is already
>> 
>> static void *get_xsave_addr(void *xsave, unsigned int xfeature_idx)
>> 
>> higher in the same file.
>> 
>> That function should be augmented to take a struct xsave_struct
>> *xsave,
>> look at whether the representation is compressed or not, and use the
>> appropriate offset array.
>> 
> Just because I have pulled staging branch when "static void
> *get_xsave_addr(void *xsave, unsigned int xfeature_idx)" is not added.

Yet you use the function on patch 7 already...

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [V2 PATCH 9/9] x86/hvm: pkeys, add pkeys support for gva2gfn funcitons

2015-12-02 Thread Jan Beulich
 >>> On 27.11.15 at 10:52,  wrote:
> --- a/xen/arch/x86/hvm/hvm.c
> +++ b/xen/arch/x86/hvm/hvm.c
> @@ -4304,7 +4304,8 @@ static enum hvm_copy_result __hvm_clear(paddr_t addr, 
> int size)
>  p2m_type_t p2mt;
>  char *p;
>  int count, todo = size;
> -uint32_t pfec = PFEC_page_present | PFEC_write_access;
> +uint32_t pfec = PFEC_page_present | PFEC_write_access |
> +hvm_pku_enabled(curr) ? PFEC_prot_key : 0;
>  
>  /*
>   * XXX Disable for 4.1.0: PV-on-HVM drivers will do grant-table ops
> @@ -4405,7 +4406,8 @@ enum hvm_copy_result hvm_copy_to_guest_virt(
>  {
>  return __hvm_copy(buf, vaddr, size,
>HVMCOPY_to_guest | HVMCOPY_fault | HVMCOPY_virt,
> -  PFEC_page_present | PFEC_write_access | pfec);
> +  PFEC_page_present | PFEC_write_access | pfec |
> +  hvm_pku_enabled(current) ? PFEC_prot_key : 0);
>  }
>  
>  enum hvm_copy_result hvm_copy_from_guest_virt(
> @@ -4413,7 +4415,8 @@ enum hvm_copy_result hvm_copy_from_guest_virt(
>  {
>  return __hvm_copy(buf, vaddr, size,
>HVMCOPY_from_guest | HVMCOPY_fault | HVMCOPY_virt,
> -  PFEC_page_present | pfec);
> +  PFEC_page_present | pfec |
> +  hvm_pku_enabled(current) ? PFEC_prot_key : 0);
>  }
>  
>  enum hvm_copy_result hvm_fetch_from_guest_virt(
> @@ -4431,7 +4434,8 @@ enum hvm_copy_result hvm_copy_to_guest_virt_nofault(
>  {
>  return __hvm_copy(buf, vaddr, size,
>HVMCOPY_to_guest | HVMCOPY_no_fault | HVMCOPY_virt,
> -  PFEC_page_present | PFEC_write_access | pfec);
> +  PFEC_page_present | PFEC_write_access | pfec |
> +  hvm_pku_enabled(current) ? PFEC_prot_key : 0);
>  }
>  
>  enum hvm_copy_result hvm_copy_from_guest_virt_nofault(
> @@ -4439,7 +4443,8 @@ enum hvm_copy_result hvm_copy_from_guest_virt_nofault(
>  {
>  return __hvm_copy(buf, vaddr, size,
>HVMCOPY_from_guest | HVMCOPY_no_fault | HVMCOPY_virt,
> -  PFEC_page_present | pfec);
> +  PFEC_page_present | pfec |
> +  hvm_pku_enabled(current) ? PFEC_prot_key : 0);
>  }
>  
>  enum hvm_copy_result hvm_fetch_from_guest_virt_nofault(

Was this patch tested at all? The lack of parentheses in all the
changes you make result - afaict - in PFEC_prot_key to be
unconditionally passed to __hvm_copy(), which can't be right.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v2] libxc: try to find last used pfn when migrating

2015-12-02 Thread Wei Liu
On Wed, Dec 02, 2015 at 08:42:17AM +0100, Juergen Gross wrote:
> For migration the last used pfn of a guest is needed to size the
> logdirty bitmap and as an upper bound of the page loop. Unfortunately
> there are pv-kernels advertising a much higher maximum pfn as they
> are really using in order to support memory hotplug. This will lead
> to allocation of much more memory in Xen tools during migration as
> really needed.
> 
> Try to find the last used guest pfn of a pv-domu by scanning the p2m
> tree from the last entry towards it's start and search for an entry
> not being invalid.
> 
> Normally the mid pages of the p2m tree containing all invalid entries
> are being reused, so we can just scan the top page for identical
> entries and skip them but the first one.
> 
> Signed-off-by: Juergen Gross 

Reviewed-by: Wei Liu 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCHv3] 1/3] libxc: prefer using privcmd character device

2015-12-02 Thread Wei Liu
On Tue, Dec 01, 2015 at 01:27:53PM -0600, Doug Goldstein wrote:
> Prefer using the character device over the proc file if the character
> device exists. This follows similar conversions of xenbus to avoid
> issues with FMODE_ATOMIC_POS added in Linux 3.14 and newer.
> 
> CC: Ian Jackson 
> CC: Stefano Stabellini 
> CC: Ian Campbell 
> CC: Wei Liu 
> Signed-off-by: Doug Goldstein 

Acked-by: Wei Liu 

> ---
>  tools/libxc/xc_linux_osdep.c | 8 +++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/libxc/xc_linux_osdep.c b/tools/libxc/xc_linux_osdep.c
> index 76c55ff..c3a3a14 100644
> --- a/tools/libxc/xc_linux_osdep.c
> +++ b/tools/libxc/xc_linux_osdep.c
> @@ -46,7 +46,13 @@
>  static xc_osdep_handle linux_privcmd_open(xc_interface *xch)
>  {
>  int flags, saved_errno;
> -int fd = open("/proc/xen/privcmd", O_RDWR);
> +int fd = open("/dev/xen/privcmd", O_RDWR); /* prefer this newer 
> interface */
> +
> +if ( fd == -1 && ( errno == ENOENT || errno == ENXIO || errno == ENODEV 
> ))
> +{
> +/* Fallback to /proc/xen/privcmd */
> +fd = open("/proc/xen/privcmd", O_RDWR);
> +}
>  
>  if ( fd == -1 )
>  {
> -- 
> 2.4.10
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH 1/2] libxc: replace INVALID_P2M_ENTRY by INVALID_PFN

2015-12-02 Thread Wei Liu
On Tue, Dec 01, 2015 at 06:14:53PM +0100, Juergen Gross wrote:
> INVALID_P2M_ENTRY is defined as (xen_pfn_t)-1 and is often used
> according to it's type for an invalid pfn. Change the name of the
> macro to INVALID_PFN.
> 
> Signed-off-by: Juergen Gross 

Acked-by: Wei Liu 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH 2/2] libxc: do proper return code checking of allocator in domain builder

2015-12-02 Thread Wei Liu
On Tue, Dec 01, 2015 at 06:14:54PM +0100, Juergen Gross wrote:
> Signed-off-by: Juergen Gross 

Acked-by: Wei Liu 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Regression: Xen guest with 5G of RAM on 32bit fail to boot

2015-12-02 Thread Paolo Bonzini


On 02/12/2015 11:30, Paolo Bonzini wrote:
> diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
> index f9998b9..87a4145 100644
> --- a/include/exec/cpu-all.h
> +++ b/include/exec/cpu-all.h
> @@ -174,11 +174,10 @@ extern unsigned long reserved_va;
>  #define TARGET_PAGE_MASK ~(TARGET_PAGE_SIZE - 1)
>  #define TARGET_PAGE_ALIGN(addr) (((addr) + TARGET_PAGE_SIZE - 1) & 
> TARGET_PAGE_MASK)
>  
> -/* ??? These should be the larger of uintptr_t and target_ulong.  */
>  extern uintptr_t qemu_real_host_page_size;
> -extern uintptr_t qemu_real_host_page_mask;
> +extern intptr_t qemu_real_host_page_mask;
>  extern uintptr_t qemu_host_page_size;
> -extern uintptr_t qemu_host_page_mask;
> +extern intptr_t qemu_host_page_mask;
>  
>  #define HOST_PAGE_ALIGN(addr) (((addr) + qemu_host_page_size - 1) & 
> qemu_host_page_mask)
>  #define REAL_HOST_PAGE_ALIGN(addr) (((addr) + qemu_real_host_page_size - 1) 
> & \
> diff --git a/translate-all.c b/translate-all.c
> index a940bd2..7a15109 100644
> --- a/translate-all.c
> +++ b/translate-all.c
> @@ -118,7 +118,7 @@ typedef struct PageDesc {
>  #define V_L1_SHIFT (L1_MAP_ADDR_SPACE_BITS - TARGET_PAGE_BITS - V_L1_BITS)
>  
>  uintptr_t qemu_host_page_size;
> -uintptr_t qemu_host_page_mask;
> +intptr_t qemu_host_page_mask;
>  
>  /* The bottom level has pointers to PageDesc */
>  static void *l1_map[V_L1_SIZE];
> @@ -326,14 +326,14 @@ void page_size_init(void)
>  /* NOTE: we can always suppose that qemu_host_page_size >=
> TARGET_PAGE_SIZE */
>  qemu_real_host_page_size = getpagesize();
> -qemu_real_host_page_mask = ~(qemu_real_host_page_size - 1);
> +qemu_real_host_page_mask = -(intptr_t)qemu_real_host_page_size;
>  if (qemu_host_page_size == 0) {
>  qemu_host_page_size = qemu_real_host_page_size;
>  }
>  if (qemu_host_page_size < TARGET_PAGE_SIZE) {
>  qemu_host_page_size = TARGET_PAGE_SIZE;
>  }
> -qemu_host_page_mask = ~(qemu_host_page_size - 1);
> +qemu_host_page_mask = -(intptr_t)qemu_host_page_size;
>  }
>  
>  static void page_init(void)
> diff --git a/translate-common.c b/translate-common.c
> index 619feb4..171222d 100644
> --- a/translate-common.c
> +++ b/translate-common.c
> @@ -21,7 +21,7 @@
>  #include "qom/cpu.h"
>  
>  uintptr_t qemu_real_host_page_size;
> -uintptr_t qemu_real_host_page_mask;
> +intptr_t qemu_real_host_page_mask;
>  
>  #ifndef CONFIG_USER_ONLY
>  /* mask must never be zero, except for A20 change call */
> 
> 

Ok, I tested this by adding

+ assert(HOST_PAGE_ALIGN(0x123456700ll) == 0x123457000ll);
+ assert(REAL_HOST_PAGE_ALIGN(0x123456700ll) == 0x123457000ll);

and doing a 32-bit x86_64-linux-user build.  Since Dave's patch does not
compile for user-mode emulation (ram_addr_t is a softmmu concept), I'm
queuing my patch for 2.5.

Paolo

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Regression: Xen guest with 5G of RAM on 32bit fail to boot

2015-12-02 Thread Dr. David Alan Gilbert
* Paolo Bonzini (pbonz...@redhat.com) wrote:
> 
> 
> On 02/12/2015 11:30, Paolo Bonzini wrote:
> > diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
> > index f9998b9..87a4145 100644
> > --- a/include/exec/cpu-all.h
> > +++ b/include/exec/cpu-all.h
> > @@ -174,11 +174,10 @@ extern unsigned long reserved_va;
> >  #define TARGET_PAGE_MASK ~(TARGET_PAGE_SIZE - 1)
> >  #define TARGET_PAGE_ALIGN(addr) (((addr) + TARGET_PAGE_SIZE - 1) & 
> > TARGET_PAGE_MASK)
> >  
> > -/* ??? These should be the larger of uintptr_t and target_ulong.  */
> >  extern uintptr_t qemu_real_host_page_size;
> > -extern uintptr_t qemu_real_host_page_mask;
> > +extern intptr_t qemu_real_host_page_mask;
> >  extern uintptr_t qemu_host_page_size;
> > -extern uintptr_t qemu_host_page_mask;
> > +extern intptr_t qemu_host_page_mask;
> >  
> >  #define HOST_PAGE_ALIGN(addr) (((addr) + qemu_host_page_size - 1) & 
> > qemu_host_page_mask)
> >  #define REAL_HOST_PAGE_ALIGN(addr) (((addr) + qemu_real_host_page_size - 
> > 1) & \
> > diff --git a/translate-all.c b/translate-all.c
> > index a940bd2..7a15109 100644
> > --- a/translate-all.c
> > +++ b/translate-all.c
> > @@ -118,7 +118,7 @@ typedef struct PageDesc {
> >  #define V_L1_SHIFT (L1_MAP_ADDR_SPACE_BITS - TARGET_PAGE_BITS - V_L1_BITS)
> >  
> >  uintptr_t qemu_host_page_size;
> > -uintptr_t qemu_host_page_mask;
> > +intptr_t qemu_host_page_mask;
> >  
> >  /* The bottom level has pointers to PageDesc */
> >  static void *l1_map[V_L1_SIZE];
> > @@ -326,14 +326,14 @@ void page_size_init(void)
> >  /* NOTE: we can always suppose that qemu_host_page_size >=
> > TARGET_PAGE_SIZE */
> >  qemu_real_host_page_size = getpagesize();
> > -qemu_real_host_page_mask = ~(qemu_real_host_page_size - 1);
> > +qemu_real_host_page_mask = -(intptr_t)qemu_real_host_page_size;
> >  if (qemu_host_page_size == 0) {
> >  qemu_host_page_size = qemu_real_host_page_size;
> >  }
> >  if (qemu_host_page_size < TARGET_PAGE_SIZE) {
> >  qemu_host_page_size = TARGET_PAGE_SIZE;
> >  }
> > -qemu_host_page_mask = ~(qemu_host_page_size - 1);
> > +qemu_host_page_mask = -(intptr_t)qemu_host_page_size;
> >  }
> >  
> >  static void page_init(void)
> > diff --git a/translate-common.c b/translate-common.c
> > index 619feb4..171222d 100644
> > --- a/translate-common.c
> > +++ b/translate-common.c
> > @@ -21,7 +21,7 @@
> >  #include "qom/cpu.h"
> >  
> >  uintptr_t qemu_real_host_page_size;
> > -uintptr_t qemu_real_host_page_mask;
> > +intptr_t qemu_real_host_page_mask;
> >  
> >  #ifndef CONFIG_USER_ONLY
> >  /* mask must never be zero, except for A20 change call */
> > 
> > 
> 
> Ok, I tested this by adding
> 
> + assert(HOST_PAGE_ALIGN(0x123456700ll) == 0x123457000ll);
> + assert(REAL_HOST_PAGE_ALIGN(0x123456700ll) == 0x123457000ll);
> 
> and doing a 32-bit x86_64-linux-user build.  Since Dave's patch does not
> compile for user-mode emulation (ram_addr_t is a softmmu concept), I'm
> queuing my patch for 2.5.

Hmm yes OK; my alternate was just making ram_addr_t being always defined.
I'm not sure we have any other type that's more suitable than ram_addr_t
but I guess the intptr_t will work for the mask.

Dave

> 
> Paolo
--
Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Regression: Xen guest with 5G of RAM on 32bit fail to boot

2015-12-02 Thread Juan Quintela
Paolo Bonzini  wrote:
> On 02/12/2015 11:30, Paolo Bonzini wrote:
>> diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
>> index f9998b9..87a4145 100644
>> --- a/include/exec/cpu-all.h
>> +++ b/include/exec/cpu-all.h
>> @@ -174,11 +174,10 @@ extern unsigned long reserved_va;
>>  #define TARGET_PAGE_MASK ~(TARGET_PAGE_SIZE - 1)
>>  #define TARGET_PAGE_ALIGN(addr) (((addr) + TARGET_PAGE_SIZE - 1) & 
>> TARGET_PAGE_MASK)
>>  
>> -/* ??? These should be the larger of uintptr_t and target_ulong.  */
>>  extern uintptr_t qemu_real_host_page_size;
>> -extern uintptr_t qemu_real_host_page_mask;
>> +extern intptr_t qemu_real_host_page_mask;
>>  extern uintptr_t qemu_host_page_size;
>> -extern uintptr_t qemu_host_page_mask;
>> +extern intptr_t qemu_host_page_mask;
>>  
>>  #define HOST_PAGE_ALIGN(addr) (((addr) + qemu_host_page_size - 1) & 
>> qemu_host_page_mask)
>>  #define REAL_HOST_PAGE_ALIGN(addr) (((addr) + qemu_real_host_page_size - 1) 
>> & \
>> diff --git a/translate-all.c b/translate-all.c
>> index a940bd2..7a15109 100644
>> --- a/translate-all.c
>> +++ b/translate-all.c
>> @@ -118,7 +118,7 @@ typedef struct PageDesc {
>>  #define V_L1_SHIFT (L1_MAP_ADDR_SPACE_BITS - TARGET_PAGE_BITS - V_L1_BITS)
>>  
>>  uintptr_t qemu_host_page_size;
>> -uintptr_t qemu_host_page_mask;
>> +intptr_t qemu_host_page_mask;
>>  
>>  /* The bottom level has pointers to PageDesc */
>>  static void *l1_map[V_L1_SIZE];
>> @@ -326,14 +326,14 @@ void page_size_init(void)
>>  /* NOTE: we can always suppose that qemu_host_page_size >=
>> TARGET_PAGE_SIZE */
>>  qemu_real_host_page_size = getpagesize();
>> -qemu_real_host_page_mask = ~(qemu_real_host_page_size - 1);
>> +qemu_real_host_page_mask = -(intptr_t)qemu_real_host_page_size;
>>  if (qemu_host_page_size == 0) {
>>  qemu_host_page_size = qemu_real_host_page_size;
>>  }
>>  if (qemu_host_page_size < TARGET_PAGE_SIZE) {
>>  qemu_host_page_size = TARGET_PAGE_SIZE;
>>  }
>> -qemu_host_page_mask = ~(qemu_host_page_size - 1);
>> +qemu_host_page_mask = -(intptr_t)qemu_host_page_size;
>>  }
>>  
>>  static void page_init(void)
>> diff --git a/translate-common.c b/translate-common.c
>> index 619feb4..171222d 100644
>> --- a/translate-common.c
>> +++ b/translate-common.c
>> @@ -21,7 +21,7 @@
>>  #include "qom/cpu.h"
>>  
>>  uintptr_t qemu_real_host_page_size;
>> -uintptr_t qemu_real_host_page_mask;
>> +intptr_t qemu_real_host_page_mask;
>>  
>>  #ifndef CONFIG_USER_ONLY
>>  /* mask must never be zero, except for A20 change call */
>> 
>> 
>
> Ok, I tested this by adding
>
> + assert(HOST_PAGE_ALIGN(0x123456700ll) == 0x123457000ll);
> + assert(REAL_HOST_PAGE_ALIGN(0x123456700ll) == 0x123457000ll);
>
> and doing a 32-bit x86_64-linux-user build.  Since Dave's patch does not
> compile for user-mode emulation (ram_addr_t is a softmmu concept), I'm
> queuing my patch for 2.5.
>
> Paolo

Reviewed-by: Juan Quintela 

Dave patch massively broke linux-user.  Going that route can make sense,
but not so late on the cycle.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v2] libxc: try to find last used pfn when migrating

2015-12-02 Thread Andrew Cooper
On 02/12/15 07:42, Juergen Gross wrote:
> diff --git a/tools/libxc/xc_sr_save_x86_hvm.c 
> b/tools/libxc/xc_sr_save_x86_hvm.c
> index cdee774..3c879ed 100644
> --- a/tools/libxc/xc_sr_save_x86_hvm.c
> +++ b/tools/libxc/xc_sr_save_x86_hvm.c
> @@ -135,6 +135,20 @@ static int x86_hvm_normalise_page(struct xc_sr_context 
> *ctx,
>  static int x86_hvm_setup(struct xc_sr_context *ctx)
>  {
>  xc_interface *xch = ctx->xch;
> +xen_pfn_t nr_pfns;
> +
> +if ( xc_domain_nr_gpfns(xch, ctx->domid, &nr_pfns) < 0 )
> +{
> +PERROR("Unable to obtain the guest p2m size");
> +return -1;
> +}
> +if ( nr_pfns > ~XEN_DOMCTL_PFINFO_LTAB_MASK )
> +{
> +PERROR("Cannot save this big a guest");

Strictly speaking to match the moved code, this should set errno = E2BIG.

However, the error handling in libxc is in a dire state, and the error
message is retained, which is the important point.

Entire patch Reviewed-by: Andrew Cooper  with
or without the errno tweaks.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v2 2/2] libxl: implement virDomainInterfaceStats

2015-12-02 Thread Joao Martins


On 12/02/2015 12:45 AM, Jim Fehlig wrote:
> On 11/23/2015 11:57 AM, Joao Martins wrote:
>> Introduce support for domainInterfaceStats API call for querying
>> network interface statistics. Consequently it also enables the
>> use of `virsh domifstat  ` command plus
>> seeing the interfaces names instead of "-" when doing
>> `virsh domiflist `.
>>
>> After successful guest creation we fill the network
>> interfaces names based on domain, device id and append suffix
>> if it's emulated in the following form: vif.[-emu].
> 
> One interesting Xen behavior that has existing for many, many years is that a 
> PV
> nic is implicitly created for each emulated nic specified in the config. The
> guest OS picks which one to use. These days most will use the PV nic, and if
> they are nice, "unplug" the emulated one via the unplug protocol. E.g. an HVM
> guest with
> 
>  
> 
> 
> 
>   
> 
> results in two vif devices on the host
> 
> # ip a | grep vif
> 607: vif519.0-emu:  mtu 1500 qdisc pfifo_fast
> master br0 state UNKNOWN group default qlen 500
> 608: vif519.0:  mtu 1500 qdisc pfifo_fast
> master br0 state UNKNOWN group default qlen 512
> 
> both are connected to the bridge
> 
> # brctl show br0
> bridge namebridge idSTP enabled   interfaces
> br08000.001e676598f5noeth0
>   vif519.0
>   vif519.0-emu
> 
> In this case, the (not nice) guest OS is using the PV nic but did not unplug 
> the
> emulated one. So we have two interfaces, but the virDomainDef only contains 
> one
> 
> # virsh domiflist 519
> Interface  Type   Source Model   MAC
> ---
> vif519.0-emu bridge br0-   00:16:3e:7a:35:ce
> 
> Not a fault of this patch, but we'll need to figure out how to handle the
> implicitly created PV nic. The interesting case is identifying emulated nics
> that have been unplugged by a nice guest, and hence no longer exist in the 
> host
> (e.g. vif519.0-emu in the above example).
> 
Indeed this is an issue I am aware of but wasn't sure on how to handle it. The
way I test an HVM guest on libvirt with a (explicitly created) PV nic was with
"model=netfront" since there wouldn't be an emulated nic. So perhaps we could
differ it if this was specified on HVM guests. If that wasn't the case we would
add the complementary interface along with checking if indeed the net device
exists. On cleanup we would just delete the second interface that would appear
with the same name.

>> We extract the network interfaces info from libxl in
>> libxlDomainStartCallback() and make ifname . On domain
>> cleanup we also clear ifname, in case it was set by libvirt (i.e.
>> being prefixed with "vif"). We also skip these two steps in case the name
>> of the interface was manually inserted by the adminstrator.
>>
>> For getting the interface statistics we resort to virNetInterfaceStats
>> and let libvirt handle the platform specific nits. Note that the latter
>> is not yet supported in FreeBSD.
>>
>> Signed-off-by: Joao Martins 
>> ---
>> Changes since v3:
>>  - Use libxl_device_nic_list() for getting each network interface
>>  devid in DomainStartCallback.
>>  - Improve error reporting by appropriately setting the right error
>>  when no interface is known.
>>  - Do not unlock vm if libxlDomainObjEndJob() returns false
>>  - Set vm->def->net[i]->ifname on DomainStartCallback instead of
>>  DomainStart.
>>  - Change commit message reflecting the changes on the previous
>>  item and mention correct interface names when doing domiflist.
>>
>> Changes since v2:
>>  - Clear ifname if it's autogenerated, since otherwise will persist
>>  on successive domain starts. Change commit message reflecting this
>>  change.
>>
>> Changes since v1:
>>  - Fill .ifname after domain start with generated
>>  name from libxl  based on domain id and devid returned by libxl.
>>  After that path validation don interfaceStats is enterily based
>>  on ifname pretty much like the other drivers.
>>  - Modify commit message reflecting the changes mentioned in
>>  the previous item.
>>  - Bump version to 1.2.22
>> ---
>>  src/libxl/libxl_domain.c | 29 +++
>>  src/libxl/libxl_driver.c | 52 
>> 
>>  2 files changed, 81 insertions(+)
>>
>> diff --git a/src/libxl/libxl_domain.c b/src/libxl/libxl_domain.c
>> index a7267b0..141f241 100644
>> --- a/src/libxl/libxl_domain.c
>> +++ b/src/libxl/libxl_domain.c
>> @@ -728,6 +728,17 @@ libxlDomainCleanup(libxlDriverPrivatePtr driver,
>>  }
>>  }
>>  
>> +if ((vm->def->nnets)) {
>> +ssize_t i;
> 
> size_t
> 
Apologies for the typo.

>> +
>> +for (i = 0; i < vm->def->nnets; i++) {
>> +virDomainNetDefPtr net = vm->def->nets[i];
>> +
>> +if (STRPREFIX(net->ifname, "vif"))
>> +VIR_FREE(net->ifna

Re: [Xen-devel] [PATCH v2 1/2] libxl: rename libxlConsoleCallback

2015-12-02 Thread Joao Martins


On 12/02/2015 12:08 AM, Jim Fehlig wrote:
> On 11/23/2015 11:56 AM, Joao Martins wrote:
>> . to a more generic name i.e. libxlDomainStartCallback,
>> since it will now cover another case other than the console.
>>
>> Signed-off-by: Joao Martins 
>> ---
>>  src/libxl/libxl_domain.c | 4 ++--
>>  1 file changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/src/libxl/libxl_domain.c b/src/libxl/libxl_domain.c
>> index 40dcea1..a7267b0 100644
>> --- a/src/libxl/libxl_domain.c
>> +++ b/src/libxl/libxl_domain.c
>> @@ -854,7 +854,7 @@ libxlDomainFreeMem(libxl_ctx *ctx, libxl_domain_config 
>> *d_config)
>>  }
>>  
>>  static void
>> -libxlConsoleCallback(libxl_ctx *ctx, libxl_event *ev, void *for_callback)
>> +libxlDomainStartCallback(libxl_ctx *ctx, libxl_event *ev, void 
>> *for_callback)
>>  {
>>  virDomainObjPtr vm = for_callback;
>>  size_t i;
>> @@ -988,7 +988,7 @@ libxlDomainStart(libxlDriverPrivatePtr driver, 
>> virDomainObjPtr vm,
>>  virObjectUnlock(vm);
>>  
>>  aop_console_how.for_callback = vm;
>> -aop_console_how.callback = libxlConsoleCallback;
>> +aop_console_how.callback = libxlDomainStartCallback;
> 
> Before pushing, I wanted to change the 'aop_console_how' variable to something
> more generic too, but realized it is the 'const libxl_asyncprogress_how
> *aop_console_how' parameter to libxl_domain_create_{new,restore}. AFAICT, this
> callback is invoked when a console becomes available for the domain. It might 
> be
> possible that network devices have not been created (devid = -1) when the
> callback is invoked. I thought about adding a separate 
> libxlDomainStartCallback
> and registering it with the 'const libxl_asyncop_how *ao_how' parameter, but
> this would change the synchronous behavior of libxlDomainStart and be quite a
> bit more intrusive.
I see, I wasn't aware of that.

> 
> In the end, I think it is best to explicitly call a function that creates the
> ifnames after a successful libxl_domain_create_{new,restore}. See my reply to
> 2/2 for an example of this idea.
OK, Thanks for all the feedback!

Regards,
Joao

> 
> Regards,
> Jim
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [linux-linus test] 65285: regressions - FAIL

2015-12-02 Thread osstest service owner
flight 65285 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/65285/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemut-winxpsp3  6 xen-boot fail REGR. vs. 59254
 test-amd64-i386-qemut-rhel6hvm-intel  6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-xl-qemut-debianhvm-amd64-xsm  6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-freebsd10-i386  6 xen-bootfail REGR. vs. 59254
 test-amd64-i386-rumpuserxen-i386  6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-xl-qemut-debianhvm-amd64  6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-freebsd10-amd64  6 xen-boot   fail REGR. vs. 59254
 test-amd64-i386-xl-qemuu-win7-amd64  6 xen-boot   fail REGR. vs. 59254
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1  6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-xl-xsm6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-xl6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-xl-qemut-win7-amd64  6 xen-boot   fail REGR. vs. 59254
 test-amd64-amd64-xl   6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm  6 xen-boot  fail REGR. vs. 59254
 test-amd64-amd64-xl-qemut-stubdom-debianhvm-amd64-xsm 13 guest-localmigrate 
fail REGR. vs. 59254
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm  6 xen-boot fail REGR. vs. 59254
 test-armhf-armhf-xl-arndale   6 xen-boot  fail REGR. vs. 59254
 test-armhf-armhf-xl   6 xen-boot  fail REGR. vs. 59254
 test-armhf-armhf-xl-cubietruck  6 xen-bootfail REGR. vs. 59254
 test-armhf-armhf-xl-credit2   6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-pair 10 xen-boot/dst_host fail REGR. vs. 59254
 test-amd64-i386-pair  9 xen-boot/src_host fail REGR. vs. 59254
 test-amd64-i386-qemuu-rhel6hvm-amd  6 xen-bootfail REGR. vs. 59254
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 13 guest-localmigrate 
fail REGR. vs. 59254
 test-amd64-i386-xl-qemuu-debianhvm-amd64  6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-xl-qemuu-ovmf-amd64  6 xen-boot   fail REGR. vs. 59254
 test-armhf-armhf-xl-xsm   6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-qemuu-rhel6hvm-intel  6 xen-boot  fail REGR. vs. 59254
 test-armhf-armhf-xl-multivcpu  6 xen-boot fail REGR. vs. 59254
 test-amd64-i386-xl-qemut-winxpsp3-vcpus1  6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-xl-qemuu-winxpsp3  6 xen-boot fail REGR. vs. 59254

Regressions which are regarded as allowable (not blocking):
 test-amd64-i386-libvirt-xsm   6 xen-boot  fail REGR. vs. 59254
 test-amd64-amd64-libvirt-xsm  6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-libvirt   6 xen-boot  fail REGR. vs. 59254
 test-amd64-amd64-rumpuserxen-amd64 15 
rumpuserxen-demo-xenstorels/xenstorels.repeat fail REGR. vs. 59254
 test-armhf-armhf-libvirt-xsm  6 xen-boot  fail REGR. vs. 59254
 test-armhf-armhf-xl-rtds  6 xen-boot  fail REGR. vs. 59254
 test-armhf-armhf-libvirt  6 xen-boot  fail REGR. vs. 59254
 test-amd64-i386-xl-raw6 xen-bootfail baseline untested
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 6 xen-boot fail baseline 
untested
 test-armhf-armhf-xl-vhd   6 xen-bootfail baseline untested
 test-amd64-i386-libvirt-pair 10 xen-boot/dst_host   fail baseline untested
 test-amd64-i386-libvirt-pair  9 xen-boot/src_host   fail baseline untested
 test-amd64-amd64-libvirt-vhd  9 debian-di-install   fail baseline untested
 test-amd64-amd64-i386-pvgrub  6 xen-bootfail baseline untested
 test-armhf-armhf-libvirt-raw  6 xen-bootfail baseline untested
 test-armhf-armhf-libvirt-qcow2  6 xen-boot  fail baseline untested
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stop fail like 59254

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-amd64-amd64-xl-pvh-intel 14 guest-saverestorefail  never pass
 test-amd64-amd64-qemuu-nested-amd  6 xen-boot  fail never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-qemuu-nested-intel 13 xen-boot/l1 fail never pass
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stop fail never pass

version targeted for testing:
 linux2255702db4014d1c69d6037ed7bdad2d2e271985
baseline version:
 linux45820c294fe1b1a9df495d57f40585ef2d069a39

Last test of basis59254  

Re: [Xen-devel] [PATCH] xen/events/fifo: Consume unprocessed events when a CPU dies

2015-12-02 Thread David Vrabel
On 30/06/15 13:26, Boris Ostrovsky wrote:
> On 06/30/2015 05:51 AM, Ross Lagerwall wrote:
>> On 06/29/2015 02:32 PM, Boris Ostrovsky wrote:
>>> On 06/29/2015 06:19 AM, Ross Lagerwall wrote:
 On 06/19/2015 05:06 PM, David Vrabel wrote:
> On 19/06/15 17:02, Boris Ostrovsky wrote:
>> On 06/19/2015 11:15 AM, Ross Lagerwall wrote:
>>> When a CPU is offlined, there may be unprocessed events on a port
>>> for
>>> that CPU.  If the port is subsequently reused on a different CPU, it
>>> could be in an unexpected state with the link bit set, resulting in
>>> interrupts being missed. Fix this by consuming any unprocessed
>>> events
>>> for a particular CPU when that CPU dies.
>>>
>>> Signed-off-by: Ross Lagerwall 
>>> ---
>>>drivers/xen/events/events_fifo.c | 23 ++-
>>>1 file changed, 18 insertions(+), 5 deletions(-)
>>>
>>> diff --git a/drivers/xen/events/events_fifo.c
>>> b/drivers/xen/events/events_fifo.c
>>> index 417415d..1dd0ba12 100644
>>> --- a/drivers/xen/events/events_fifo.c
>>> +++ b/drivers/xen/events/events_fifo.c
>>> @@ -281,7 +281,8 @@ static void handle_irq_for_port(unsigned port)
>>>  static void consume_one_event(unsigned cpu,
>>>  struct evtchn_fifo_control_block
>>> *control_block,
>>> -  unsigned priority, unsigned long *ready)
>>> +  unsigned priority, unsigned long *ready,
>>> +  bool drop)
>>>{
>>>struct evtchn_fifo_queue *q = &per_cpu(cpu_queue, cpu);
>>>uint32_t head;
>>> @@ -313,13 +314,17 @@ static void consume_one_event(unsigned cpu,
>>>if (head == 0)
>>>clear_bit(priority, ready);
>>>-if (evtchn_fifo_is_pending(port) &&
>>> !evtchn_fifo_is_masked(port))
>>> -handle_irq_for_port(port);
>>> +if (evtchn_fifo_is_pending(port) &&
>>> !evtchn_fifo_is_masked(port)) {
>>> +if (unlikely(drop))
>>> +pr_warn("Dropping pending event for port %u\n", port);
>>
>> Maybe pr_info (or pr_notice)?
>
> We want a warning here because we think this shouldn't happen -- if it
> does we actually need to retrigger the event on its new CPU.
>
>> Also, why not do this (testing for unprocessed events) in
>> xen_evtchn_close()?
>
> We can't do anything about them when closing because they may be in
> the
> middle of a queue.
>>>
>>> (Sorry, I missed this)
>>>
>>> Why can't (actually, why doesn't) the cpu that is being offlined drain
>>> its queue?
>>>
>>
>> Where would this be done? I thought using CPU notifiers was the
>> correct way to hook when a CPU goes down without having to stick fifo
>> event channel code in the core Xen code.
> 
> In xen_evtchn_close(). We should be getting there (roughly) as cpu_die()
> -> xen_cpu_die() -> xen_smp_intr_free() -> unbind_from_irqhandler(). In
> fact, this path is taken right before cpu_down() sends CPU_DEAD
> notifications.
> 
> I think cleaning up in xen_evtchn_close() is better because it is
> possible to close event channel for reasons other than CPU going away,
> in which case we also may need to deal with unprocessed events.

Having looked at this further and attempted to do this, draining events
in close is difficult because

a) we can't wait for LINKED to clear when closing since we're holding
the desc spin lock; and deferring the close to a tasklet or work doesn't
work because:

b) rebinding a PIRQ will fail if the close is deferred and not yet
completed and there is no way to ensure the close happens promptly
without changes to core irq code.

c) Xen will be fixed to not reuse ports that are still LINKED.

I'm going to apply Ross's original patch and Cc stable.

David

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH v2] x86: Fixup IRQs when CPUs go down during shutdown

2015-12-02 Thread Ross Lagerwall
Commit fc0c3fa2ad5c ("x86/IO-APIC: fix setup of Xen internally used IRQs
(take 2)") introduced a regression on some hardware where Xen would hang
during shutdown, repeating the following message:
APIC error on CPU0: 08(08), Receive accept error

This appears to be because an interrupt (in this case from the serial
console) destined for a CPU other than the boot CPU is left unhandled so
an APIC error on CPU 0 is generated instead.

To fix this, before taking down the non-boot CPUs, call fixup_irqs()
with a CPU mask of only the boot CPU to reset the IRQ affinities
correctly.

Signed-off-by: Ross Lagerwall 
---
 xen/arch/x86/irq.c| 36 ++--
 xen/arch/x86/smp.c|  8 
 xen/arch/x86/smpboot.c|  3 ++-
 xen/include/asm-x86/irq.h |  5 +++--
 4 files changed, 35 insertions(+), 17 deletions(-)

diff --git a/xen/arch/x86/irq.c b/xen/arch/x86/irq.c
index 5f515a0..28337a1 100644
--- a/xen/arch/x86/irq.c
+++ b/xen/arch/x86/irq.c
@@ -2328,14 +2328,12 @@ static int __init setup_dump_irqs(void)
 }
 __initcall(setup_dump_irqs);
 
-/* A cpu has been removed from cpu_online_mask.  Re-set irq affinities. */
-void fixup_irqs(void)
+/* CPU(s) have been removed from mask.  Re-set irq affinities. */
+void fixup_irqs(const cpumask_t *mask, bool_t verbose)
 {
-unsigned int irq, sp;
+unsigned int irq;
 static int warned;
 struct irq_desc *desc;
-irq_guest_action_t *action;
-struct pending_eoi *peoi;
 
 for ( irq = 0; irq < nr_irqs; irq++ )
 {
@@ -2355,21 +2353,20 @@ void fixup_irqs(void)
 vector = irq_to_vector(irq);
 if ( vector >= FIRST_HIPRIORITY_VECTOR &&
  vector <= LAST_HIPRIORITY_VECTOR )
-cpumask_and(desc->arch.cpu_mask, desc->arch.cpu_mask,
-&cpu_online_map);
+cpumask_and(desc->arch.cpu_mask, desc->arch.cpu_mask, mask);
 
 cpumask_copy(&affinity, desc->affinity);
-if ( !desc->action || cpumask_subset(&affinity, &cpu_online_map) )
+if ( !desc->action || cpumask_subset(&affinity, mask) )
 {
 spin_unlock(&desc->lock);
 continue;
 }
 
-cpumask_and(&affinity, &affinity, &cpu_online_map);
+cpumask_and(&affinity, &affinity, mask);
 if ( cpumask_empty(&affinity) )
 {
 break_affinity = 1;
-cpumask_copy(&affinity, &cpu_online_map);
+cpumask_copy(&affinity, mask);
 }
 
 if ( desc->handler->disable )
@@ -2385,16 +2382,27 @@ void fixup_irqs(void)
 
 spin_unlock(&desc->lock);
 
-if ( break_affinity && set_affinity )
-printk("Broke affinity for irq %i\n", irq);
-else if ( !set_affinity )
-printk("Cannot set affinity for irq %i\n", irq);
+if ( verbose )
+{
+if ( break_affinity && set_affinity )
+printk("Broke affinity for irq %i\n", irq);
+else if ( !set_affinity )
+printk("Cannot set affinity for irq %i\n", irq);
+}
 }
 
 /* That doesn't seem sufficient.  Give it 1ms. */
 local_irq_enable();
 mdelay(1);
 local_irq_disable();
+}
+
+void fixup_eoi(void)
+{
+unsigned int irq, sp;
+struct irq_desc *desc;
+irq_guest_action_t *action;
+struct pending_eoi *peoi;
 
 /* Clean up cpu_eoi_map of every interrupt to exclude this CPU. */
 for ( irq = 0; irq < nr_irqs; irq++ )
diff --git a/xen/arch/x86/smp.c b/xen/arch/x86/smp.c
index 50ff6c2..4446769 100644
--- a/xen/arch/x86/smp.c
+++ b/xen/arch/x86/smp.c
@@ -286,6 +286,7 @@ void __stop_this_cpu(void)
 
 static void stop_this_cpu(void *dummy)
 {
+fixup_eoi();
 __stop_this_cpu();
 for ( ; ; )
 halt();
@@ -298,6 +299,13 @@ static void stop_this_cpu(void *dummy)
 void smp_send_stop(void)
 {
 int timeout = 10;
+cpumask_t online;
+
+cpumask_clear(&online);
+cpumask_set_cpu(smp_processor_id(), &online);
+local_irq_disable();
+fixup_irqs(&online, 0);
+local_irq_enable();
 
 smp_call_function(stop_this_cpu, NULL, 0);
 
diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index 5d48bac..1eb16cb 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -922,7 +922,8 @@ void __cpu_disable(void)
 
 /* It's now safe to remove this processor from the online map */
 cpumask_clear_cpu(cpu, &cpu_online_map);
-fixup_irqs();
+fixup_irqs(&cpu_online_map, 1);
+fixup_eoi();
 
 if ( cpu_disable_scheduler(cpu) )
 BUG();
diff --git a/xen/include/asm-x86/irq.h b/xen/include/asm-x86/irq.h
index fcf37a3..6c8f968 100644
--- a/xen/include/asm-x86/irq.h
+++ b/xen/include/asm-x86/irq.h
@@ -148,8 +148,9 @@ int map_domain_emuirq_pirq(struct domain *d, int pirq, int 
irq);
 int unmap_domain_pirq_emuirq(struct domain *d, int pirq);
 bool_t hvm_domain_use_pirq(const struct domain *, const struct pirq *);
 
-/* A cpu has been removed from cpu_online_mask.  Re-set 

Re: [Xen-devel] Crash in set_cpu_sibling_map() booting Xen 4.6.0 on Fusion

2015-12-02 Thread Jan Beulich
>>> On 24.11.15 at 21:28,  wrote:
> RFC. Boot tested on VMware Fusion, and on a 2-socket Xeon server.

Mind trying this one instead?

Jan

--- unstable.orig/xen/arch/x86/mpparse.c
+++ unstable/xen/arch/x86/mpparse.c
@@ -89,19 +89,14 @@ void __init set_nr_cpu_ids(unsigned int
 
 void __init set_nr_sockets(void)
 {
-/*
- * Count the actual cpus in the socket 0 and use it to calculate nr_sockets
- * so that the latter will be always >= the actual socket number in the
- * system even when APIC IDs from MP table are too sparse.
- */
-unsigned int cpus = bitmap_weight(phys_cpu_present_map.mask,
-  boot_cpu_data.x86_max_cores *
-  boot_cpu_data.x86_num_siblings);
-
-if ( cpus == 0 )
-cpus = 1;
-
-nr_sockets = DIV_ROUND_UP(num_processors + disabled_cpus, cpus);
+   nr_sockets = last_physid(phys_cpu_present_map)
+/ boot_cpu_data.x86_max_cores
+/ boot_cpu_data.x86_num_siblings + 1;
+   if (disabled_cpus)
+   nr_sockets += (disabled_cpus - 1)
+ / boot_cpu_data.x86_max_cores
+ / boot_cpu_data.x86_num_siblings + 1;
+   printk(XENLOG_DEBUG "nr_sockets: %u\n", nr_sockets);
 }
 
 /*
--- unstable.orig/xen/include/asm-x86/mpspec.h
+++ unstable/xen/include/asm-x86/mpspec.h
@@ -43,6 +43,19 @@ typedef struct physid_mask physid_mask_t
 #define physid_isset(physid, map)  test_bit(physid, (map).mask)
 #define physid_test_and_set(physid, map)   test_and_set_bit(physid, 
(map).mask)
 
+#define first_physid(map)  find_first_bit((map).mask, \
+  MAX_APICS)
+#define next_physid(id, map)   find_next_bit((map).mask, \
+ MAX_APICS, (id) + 
1)
+#define last_physid(map) ({ \
+   const unsigned long *mask = (map).mask; \
+   unsigned int id, last = MAX_APICS; \
+   for (id = find_first_bit(mask, MAX_APICS); id < MAX_APICS; \
+id = find_next_bit(mask, MAX_APICS, (id) + 1)) \
+   last = id; \
+   last; \
+})
+
 #define physids_and(dst, src1, src2)   bitmap_and((dst).mask, 
(src1).mask, (src2).mask, MAX_APICS)
 #define physids_or(dst, src1, src2)bitmap_or((dst).mask, 
(src1).mask, (src2).mask, MAX_APICS)
 #define physids_clear(map) bitmap_zero((map).mask, 
MAX_APICS)



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH] x86: __{cpu,dev}initdata drop follow-up

2015-12-02 Thread Jan Beulich
While reviewing those patches I noticed a few types that could do with
tweaking.

Signed-off-by: Jan Beulich 
---
For the AMD MWAIT change it would certainly be nice to get confirmation
on whether just Fam10 doesn't deep-sleep, or whether there's a wider
range (I'm pretty sure modern families don't have such a limitation
anymore).

--- a/xen/arch/x86/cpu/amd.c
+++ b/xen/arch/x86/cpu/amd.c
@@ -332,8 +332,6 @@ static void disable_c1_ramping(void)
}
 }
 
-int force_mwait;
-
 static void disable_c1e(void *unused)
 {
uint64_t msr_content;
@@ -510,7 +508,7 @@ static void __devinit init_amd(struct cp
 amd_get_topology(c);
 
/* Pointless to use MWAIT on Family10 as it does not deep sleep. */
-   if (c->x86 >= 0x10 && !force_mwait)
+   if (c->x86 == 0x10)
__clear_bit(X86_FEATURE_MWAIT, c->x86_capability);
 
if (!cpu_has_amd_erratum(c, AMD_ERRATUM_121))
--- a/xen/arch/x86/cpu/intel_cacheinfo.c
+++ b/xen/arch/x86/cpu/intel_cacheinfo.c
@@ -27,7 +27,7 @@ struct _cache_table
 };
 
 /* all the cache descriptor types we care about (no TLB or trace cache 
entries) */
-static struct _cache_table cache_table[] =
+static const struct _cache_table cache_table[] =
 {
{ 0x06, LVL_1_INST, 8 },/* 4-way set assoc, 32 byte line size */
{ 0x08, LVL_1_INST, 16 },   /* 4-way set assoc, 32 byte line size */
--- a/xen/arch/x86/numa.c
+++ b/xen/arch/x86/numa.c
@@ -49,9 +49,8 @@ cpumask_t node_to_cpumask[MAX_NUMNODES]
 
 nodemask_t __read_mostly node_online_map = { { [0] = 1UL } };
 
-int numa_off = 0;
-
-int acpi_numa;
+bool_t numa_off = 0;
+s8 acpi_numa = 0;
 
 int srat_disabled(void)
 {
--- a/xen/include/asm-x86/acpi.h
+++ b/xen/include/asm-x86/acpi.h
@@ -107,7 +107,7 @@ extern void acpi_reserve_bootmem(void);
 
 #define ARCH_HAS_POWER_INIT1
 
-extern int acpi_numa;
+extern s8 acpi_numa;
 extern int acpi_scan_nodes(u64 start, u64 end);
 #define NR_NODE_MEMBLKS (MAX_NUMNODES*2)
 
--- a/xen/include/asm-x86/numa.h
+++ b/xen/include/asm-x86/numa.h
@@ -30,7 +30,7 @@ extern nodeid_t pxm_to_node(unsigned int
 
 extern void numa_add_cpu(int cpu);
 extern void numa_init_array(void);
-extern int numa_off;
+extern bool_t numa_off;
 
 
 extern int srat_disabled(void);



x86: __{cpu,dev}initdata drop follow-up

While reviewing those patches I noticed a few types that could do with
tweaking.

Signed-off-by: Jan Beulich 
---
For the AMD MWAIT change it would certainly be nice to get confirmation
on whether just Fam10 doesn't deep-sleep, or whether there's a wider
range (I'm pretty sure modern families don't have such a limitation
anymore).

--- a/xen/arch/x86/cpu/amd.c
+++ b/xen/arch/x86/cpu/amd.c
@@ -332,8 +332,6 @@ static void disable_c1_ramping(void)
}
 }
 
-int force_mwait;
-
 static void disable_c1e(void *unused)
 {
uint64_t msr_content;
@@ -510,7 +508,7 @@ static void __devinit init_amd(struct cp
 amd_get_topology(c);
 
/* Pointless to use MWAIT on Family10 as it does not deep sleep. */
-   if (c->x86 >= 0x10 && !force_mwait)
+   if (c->x86 == 0x10)
__clear_bit(X86_FEATURE_MWAIT, c->x86_capability);
 
if (!cpu_has_amd_erratum(c, AMD_ERRATUM_121))
--- a/xen/arch/x86/cpu/intel_cacheinfo.c
+++ b/xen/arch/x86/cpu/intel_cacheinfo.c
@@ -27,7 +27,7 @@ struct _cache_table
 };
 
 /* all the cache descriptor types we care about (no TLB or trace cache 
entries) */
-static struct _cache_table cache_table[] =
+static const struct _cache_table cache_table[] =
 {
{ 0x06, LVL_1_INST, 8 },/* 4-way set assoc, 32 byte line size */
{ 0x08, LVL_1_INST, 16 },   /* 4-way set assoc, 32 byte line size */
--- a/xen/arch/x86/numa.c
+++ b/xen/arch/x86/numa.c
@@ -49,9 +49,8 @@ cpumask_t node_to_cpumask[MAX_NUMNODES]
 
 nodemask_t __read_mostly node_online_map = { { [0] = 1UL } };
 
-int numa_off = 0;
-
-int acpi_numa;
+bool_t numa_off = 0;
+s8 acpi_numa = 0;
 
 int srat_disabled(void)
 {
--- a/xen/include/asm-x86/acpi.h
+++ b/xen/include/asm-x86/acpi.h
@@ -107,7 +107,7 @@ extern void acpi_reserve_bootmem(void);
 
 #define ARCH_HAS_POWER_INIT1
 
-extern int acpi_numa;
+extern s8 acpi_numa;
 extern int acpi_scan_nodes(u64 start, u64 end);
 #define NR_NODE_MEMBLKS (MAX_NUMNODES*2)
 
--- a/xen/include/asm-x86/numa.h
+++ b/xen/include/asm-x86/numa.h
@@ -30,7 +30,7 @@ extern nodeid_t pxm_to_node(unsigned int
 
 extern void numa_add_cpu(int cpu);
 extern void numa_init_array(void);
-extern int numa_off;
+extern bool_t numa_off;
 
 
 extern int srat_disabled(void);
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [xen-unstable test] 65141: regressions - FAIL

2015-12-02 Thread Ian Campbell
On Wed, 2015-12-02 at 10:34 +, Ian Campbell wrote:
> 
[...]
> FWIW looking at the different branches in
>  
> http://logs.test-lab.xenproject.org/osstest/results/history/test-amd64-amd64-qemuu-nested-intel/
> 
> While xen-unstable has started failing on godello1 xen-4.6-testing does
> seem to be passing there.
> 
> The history for the previous job name (before splitting into -intel and
> -amd) also shows a pass on godello1:
> 
> http://logs.test-lab.xenproject.org/osstest/results/history/test-amd64-amd64-qemuu-nested/xen-unstable.html
> 
> Of course this is a new test so there isn't very much historical data to
> draw conclusions from.
> 
> > > We'd appreciate it if you and your colleagues could take a look at
> > > this and analyse the failure.
> > > 
> > > In the meantime the osstest bisector will try to start work on it and
> > > I will report what it discovers.
> 
> According to 
> http://osstest.test-lab.xenproject.org/~osstest/pub/results/bisect/xen-un
> stable/test-amd64-amd64-qemuu-nested-intel.debian-hvm-install--l1
> --l2.html
> it was unable to reproduce a baseline, probably because it didn't have
> enough historical data. 

So I have run an adhoc test of the version of Xen tested by flight 64494,
i.e. the one which passed on godello under the old job name test-amd64-
amd64-qemuu-nested but using the new name and current test harness and it
was successful:

http://osstest.test-lab.xenproject.org/~osstest/pub/logs/65301/

I think that ought to give a baseline for the bisector to work with. I'll
prod it to do so.

Given the number of commits I expect this is going to take a little while
to produce results, given that the regression is already a week old it
would be good if the VT-d maintainers could investigate in parallel.

The changes to xen.git in the region are below in case that flags anything
to anyone. Note that other trees have changed as well though.

Ian.

$ git log --oneline d07f63fa6e70..b1d398b67781 
b1d398b x86: allow disabling the emulated local apic
302a851 x86/vlapic: fixes for HVM code when running without a vlapic
0ce647a x86: suppress bogus log message
4243baf HVM/save: allow the usage of zeroextend and a fixup function
fafa16e HVM/save: pass a size parameter to the HVM compat functions
08f3fad build: fix dependencies for files compiled from their parent directory
bdead3f MAINTAINERS: change the vt-d maintainer
b38d426 x86/viridian: flush remote tlbs by hypercall
713b7e4 public/event_channel.h: correct comment
d380b35 x86/boot: check for not allowed sections before linking
c708181 libxc: expose xsaves/xgetbv1/xsavec to hvm guest
460b9a4 x86/xsaves: enable xsaves/xrstors for hvm guest
da62246 x86/xsaves: enable xsaves/xrstors/xsavec in xen
3ebe992 x86/xsaves: using named operand instead numbered operand in xrstor
d9d610d build: remove .d files from xen/ on a clean
1e8193b console: make printk() line continuation tracking per-CPU
2a91f05 xen/arm: vgic-v3: Make clear that GICD_*SPI_* registers are reserved
c38f9e4 xen/arm: vgic-v3: Don't implement write-only register read as zero
cdae22a xen/arm: vgic-v3: Remove spurious return in GICR_INVALLR
833a693 xen/arm: vgic-v3: Emulate read to GICD_ICACTIVER
5482d13 xen/arm: vgic: Re-order the register emulations to match the memory map
49b6d4c xen/arm: vgic-v3: Remove GICR_MOVALLR and GICR_MOVLPIR
a2b83f9 xen/arm: vgic: Properly emulate the full register
84ce5f4 xen/arm: vgic-v3: Only emulate identification registers required by the 
spec
cf1a142 xen/arm: vgic-v3: Use the correct offset GICR_IGRPMODR0
675b68f xen/arm: vgic-v3: Don't try to emulate IROUTER which do not exist in 
the spec
afbbf2c xen/arm: vgic-v2: Implement correctly ICFGR{0, 1} read-only
c4d6bbd xen/arm: vgic-v3: Support 32-bit access for 64-bit registers
423e9ec xen/arm: vgic: Introduce helpers to extract/update/clear/set vGIC 
register ...
5d495f4 xen/arm: vgic: Optimize the way to store the target vCPU in the rank
e99e162 xen/arm: vgic-v2: Don't ignore a write in ITARGETSR if one field is 0
9f5e16e xen/arm: vgic-v2: Handle correctly byte write in ITARGETSR
bc50de8 xen/arm: vgic-v2: Implement correctly ITARGETSR0 - ITARGETSR7 read-only
ca55006 xen/arm: move ticks conversions function declarations to the header file
6c31176 arm: export platform_op XENPF_settime64
f1e5b11 xen: move wallclock functions from x86 to common
7d596f5 x86/VPMU: return correct fixed PMC count
3acb0d6 public/io/netif.h: tidy up and remove duplicate comments
8267253 public/io/netif.h: add definition of gso_prefix flag
19167b1 public/io/netif.h: document the reality of netif_rx_request/reponse
0c3f246 x86/VPMU: Initialize VPMU's lvtpc vector
c03480c x86/vPMU: document as unsupported
9b43668 Merge branch 'staging' of ssh://xenbits.xen.org/home/xen/git/xen into 
staging
9cf5f0a x86/kexec: hide more kexec infrastructure behind CONFIG_KEXEC
0aa684b x86: drop MAX_APICID
c87303c libxl: fix line wrapping issues introduced by automatic replacement
91cee73 libxl: convert libxl__sprintf(gc) to GCSPRINTF
0a

[Xen-devel] Log Messages ....

2015-12-02 Thread kumara rathnavel
Hello All,

I require logs of the Hypercalls made from Guest to the Hypervisor after
the boot, as xl dmesg provides me with the log messages during the
initialisation.

This would be a very great help!

Thanks in advance !!

Thanking you,
Kumar
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v2] x86: Fixup IRQs when CPUs go down during shutdown

2015-12-02 Thread Andrew Cooper
On 02/12/15 13:46, Ross Lagerwall wrote:
> @@ -298,6 +299,13 @@ static void stop_this_cpu(void *dummy)
>  void smp_send_stop(void)
>  {
>  int timeout = 10;
> +cpumask_t online;
> +
> +cpumask_clear(&online);
> +cpumask_set_cpu(smp_processor_id(), &online);

You can avoid this intermediate variable with cpumask_of(smp_processor_id())

There are a set of pre-canned cpumasks with a single cpu set.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] xen/events/fifo: Consume unprocessed events when a CPU dies

2015-12-02 Thread David Vrabel
On 19/06/15 16:15, Ross Lagerwall wrote:
> When a CPU is offlined, there may be unprocessed events on a port for
> that CPU.  If the port is subsequently reused on a different CPU, it
> could be in an unexpected state with the link bit set, resulting in
> interrupts being missed. Fix this by consuming any unprocessed events
> for a particular CPU when that CPU dies.

Applied to for-linus-4.4, thanks.

David

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] x86: __{cpu,dev}initdata drop follow-up

2015-12-02 Thread Andrew Cooper
On 02/12/15 13:51, Jan Beulich wrote:
> While reviewing those patches I noticed a few types that could do with
> tweaking.
>
> Signed-off-by: Jan Beulich 

Reviewed-by: Andrew Cooper 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v2] x86: Fixup IRQs when CPUs go down during shutdown

2015-12-02 Thread Jan Beulich
>>> On 02.12.15 at 14:46,  wrote:
> Commit fc0c3fa2ad5c ("x86/IO-APIC: fix setup of Xen internally used IRQs
> (take 2)") introduced a regression on some hardware where Xen would hang
> during shutdown, repeating the following message:
> APIC error on CPU0: 08(08), Receive accept error
> 
> This appears to be because an interrupt (in this case from the serial
> console) destined for a CPU other than the boot CPU is left unhandled so
> an APIC error on CPU 0 is generated instead.
> 
> To fix this, before taking down the non-boot CPUs, call fixup_irqs()
> with a CPU mask of only the boot CPU to reset the IRQ affinities
> correctly.
> 
> Signed-off-by: Ross Lagerwall 
> ---

Even though in this case interested people may know, missing info
on changes from previous version here.

> +/* CPU(s) have been removed from mask.  Re-set irq affinities. */
> +void fixup_irqs(const cpumask_t *mask, bool_t verbose)

The comment doesn't match reality. And I wonder whether it
wouldn't be reasonable to imply "verbose" (either from mask
equaling &cpu_online_map, or by introducing SYS_STATE_shutdown
and/or SYS_STATE_reboot).

> @@ -2385,16 +2382,27 @@ void fixup_irqs(void)
>  
>  spin_unlock(&desc->lock);
>  
> -if ( break_affinity && set_affinity )
> -printk("Broke affinity for irq %i\n", irq);
> -else if ( !set_affinity )
> -printk("Cannot set affinity for irq %i\n", irq);
> +if ( verbose )
> +{
> +if ( break_affinity && set_affinity )
> +printk("Broke affinity for irq %i\n", irq);
> +else if ( !set_affinity )
> +printk("Cannot set affinity for irq %i\n", irq);
> +}

How about

if ( !verbose )
   continue;

limiting churn on code?

> --- a/xen/arch/x86/smp.c
> +++ b/xen/arch/x86/smp.c
> @@ -286,6 +286,7 @@ void __stop_this_cpu(void)
>  
>  static void stop_this_cpu(void *dummy)
>  {
> +fixup_eoi();
>  __stop_this_cpu();

Is this really needed during shutdown?

> @@ -298,6 +299,13 @@ static void stop_this_cpu(void *dummy)
>  void smp_send_stop(void)
>  {
>  int timeout = 10;
> +cpumask_t online;
> +
> +cpumask_clear(&online);
> +cpumask_set_cpu(smp_processor_id(), &online);

That's what we have cpumask_of() for.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Log Messages ....

2015-12-02 Thread George Dunlap
On Wed, Dec 2, 2015 at 1:49 PM, kumara rathnavel  wrote:
> Hello All,
>
> I require logs of the Hypercalls made from Guest to the Hypervisor after the
> boot, as xl dmesg provides me with the log messages during the
> initialisation.

One thing you could look at is xentrace and xenalyze.  You can find a
brief introduction here:

https://blog.xenproject.org/2012/09/27/tracing-with-xentrace-and-xenalyze/

You can enable Xen tracing on the Xen command-line using the
"tbuf_size" and "tevt_mask" parameters described here:

http://xenbits.xenproject.org/docs/unstable/misc/xen-command-line.html

 -George

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] linux 4.4 Regression: 100% cpu usage on idle pv guest under Xen with single vcpu.

2015-12-02 Thread David Vrabel
On 28/11/15 15:47, Sander Eikelenboom wrote:
> 
> -rtc_cmos rtc_cmos: hctosys: unable to read the hardware clock
> +hctosys: unable to open rtc device (rtc0)
> 
> -genirq: Flags mismatch irq 8.  (hvc_console) vs.  (rtc0)
> -hvc_open: request_irq failed with rc -16.

I have reproduced this issue.  We really shouldn't have an RTC device in
a PV guest and I think this irq conflict breaks hvc0.

David

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v2 1/3] x86/VPMU: Support only versions 2 through 4 of architectural performance monitoring

2015-12-02 Thread Jan Beulich
>>> On 01.12.15 at 17:50,  wrote:
> @@ -746,6 +746,13 @@ static void core2_vpmu_do_cpuid(unsigned int input,
>  if ( cpu_has(¤t_cpu_data, X86_FEATURE_DSCPL) )
>  *ecx |= cpufeat_mask(X86_FEATURE_DSCPL);
>  }
> +break;
> +
> +case 0xa:
> +/* Since we don't fully emulate version 4 report version 3 */
> +if ( MASK_EXTR(*eax, PMU_VERSION_MASK) == 4 )

Considering that this may not be what physical CPUID returned, I
think you'd better use >= here.

> @@ -955,59 +962,25 @@ int vmx_vpmu_initialise(struct vcpu *v)
>  int __init core2_vpmu_init(void)
>  {
>  u64 caps;
> +unsigned int version = 0;
>  
> -if ( current_cpu_data.x86 != 6 )
> +if ( current_cpu_data.cpuid_level >= 0xa )
> +version = MASK_EXTR(cpuid_eax(0xa), PMU_VERSION_MASK);
> +
> +if ( version == 4 )
> +printk(XENLOG_INFO "VPMU: PMU version 4 is not fully supported. "
> +   "Emulating version 3\n");
> +else if ( (version != 2) && (version != 3) )
>  {
> -printk(XENLOG_WARNING "VPMU: only family 6 is supported\n");
> +printk(XENLOG_WARNING "VPMU: PMU version %u is not supported\n",
> +   version);
>  return -EINVAL;
>  }

I'd appreciate if you would use switch() here.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCHv6] 01/28] build: import Kbuild/Kconfig from Linux 4.2

2015-12-02 Thread Doug Goldstein
On 12/2/15 3:47 AM, Ian Campbell wrote:
> On Tue, 2015-12-01 at 14:04 -0600, Doug Goldstein wrote:
>> On 11/30/15 11:19 AM, Ian Campbell wrote:
>>> On Mon, 2015-11-30 at 11:00 -0600, Doug Goldstein wrote:
 Since there is a request to have KEXEC and the UARTs
 configurable by the user
>>>
>>> Who asked for this?
>>>
>>> I have quite a strong preference for not adding _any_ new[*] user
>>> configurable options in this first pass, since I think those need to be
>>> considered quite carefully whereas this first series should be largely
>>> about the mechanics of introducing Kconfig files.
>>>
>>> Ian.
>>>
>>> [*] i.e. anything which is not already controllable by the current
>>> .config
>>> driven thing.
>>>
>>
>> The ARM UARTs are the take away I had from conversations with Julien
>> Grall
> 
> AIUI all Julien was asking for was that the same set of UARTs should be
> enabled for arm32 or arm64 both before and after this series, not that they
> should be user configurable (in this first pass at least).
> 
> TBH I think this series is getting bogged down in trying to do much all at
> the same time, switching to Kconfig, making things newly user-selectable,
> arranging for out of tree builds, some of which are either (potentially)
> controversial or simply result in apparently unnecessary changes if the
> reviewer is only expecting a subset of those changes (especially those only
> expecting the first), leading to a few RTTs of back and forth with
> reviewers.
> 
> I would encourage you to par this first series back to the simplest
> possible "switch to Kconfig, retaining the exact same set of user facing
> options as today" and avoid feature creep from people requesting new and
> exciting things which the switch to Kconfig makes possible.

That I can do. I always expected that I would have a follow on series
(maybe even multiples) as I was able to tease out some of the
dependencies better. e.g. converting some of include/asm-ARCH/config.h

I would also happily add myself to MAINTAINERS for the Kconfig parts and
would ensure some kind of periodic sync up with Linux upstream. I'm not
looking at doing this conversion as a one and done but sticking around
for the long haul.

> 
> That way we can make progress on a mostly mechanical switch to Kconfig
> without continuously getting side tracked on questions like whether this or
> that should be user selectable or not.
> 
> All of the "feature-creep" (which I don't mean in the pejorative sense)
> things can easily be done in a follow up series.
> 
>>  and reading past comments on the ML how people can change the ARM
>> UARTs. Obviously if that's not desired I can drop that. I originally had
>> them enabled as they are in config/arm{32,64}.mk and changed them to be
>> user configurable later in the series.
>>
>> As far as KEXEC goes, its a user configurable option now in Rules.mk.
>> You can build "make kexec=n" and it will disable it. I chose that one as
>> an original example in v1 of how a user configurable option would look
>> in this scheme.
> 
> I don't have a problem with converting existing user-configurable options
> into Kconfig in the first pass, that seems natural/justifiable enough to
> me, and doing it in the final patch as you have done seems sensible.
> 
> Treading very carefully around the creeping-featuritis trap (:-)), it does
> seem that if one of the user options from xen/Rules.mk is going to be
> converted then they all ought to be. Perhaps that's a topic for a followup
> series though.
> 
> Ian.
> 

Sure thing. I'll break that out. My plan was to convert them all. The
patch that does KEXEC was actually from the original RFC/v1 series where
it was just an example one and I never dropped it in later series
because it was useful for testing. I've got a follow on that converts
all the items in xen/Rules.mk that I have not posted to the list and I
can move the KEXEC into that series.

-- 
Doug Goldstein



signature.asc
Description: OpenPGP digital signature
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCHv6] 01/28] build: import Kbuild/Kconfig from Linux 4.2

2015-12-02 Thread Doug Goldstein
On 12/2/15 3:38 AM, Jan Beulich wrote:
 On 01.12.15 at 21:04,  wrote:
>> On 11/30/15 11:19 AM, Ian Campbell wrote:
>>> On Mon, 2015-11-30 at 11:00 -0600, Doug Goldstein wrote:
 Since there is a request to have KEXEC and the UARTs
 configurable by the user
>>>
>>> Who asked for this?
>>>
>>> I have quite a strong preference for not adding _any_ new[*] user
>>> configurable options in this first pass, since I think those need to be
>>> considered quite carefully whereas this first series should be largely
>>> about the mechanics of introducing Kconfig files.
>>>
>>> Ian.
>>>
>>> [*] i.e. anything which is not already controllable by the current .config
>>> driven thing.
>>>
>>
>> The ARM UARTs are the take away I had from conversations with Julien
>> Grall and reading past comments on the ML how people can change the ARM
>> UARTs. Obviously if that's not desired I can drop that. I originally had
>> them enabled as they are in config/arm{32,64}.mk and changed them to be
>> user configurable later in the series.
>>
>> As far as KEXEC goes, its a user configurable option now in Rules.mk.
>> You can build "make kexec=n" and it will disable it. I chose that one as
>> an original example in v1 of how a user configurable option would look
>> in this scheme.
> 
> And note how Ian said "_any_ new".
> 
> In fact as a follow-up we should convert other command line driven
> overrides to config options (debug, perfc, bigmem, shadow) imo.
> 
> Jan
> 

I've got that series queued up locally with plans to rebase it and post
it once the initial series lands. I'll pull the KEXEC one into that
series and drop it from the original.

-- 
Doug Goldstein



signature.asc
Description: OpenPGP digital signature
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH XEN v5 09/23] tools: Refactor hypercall calling wrappers into libxencall.

2015-12-02 Thread Ian Campbell
On Fri, 2015-11-13 at 15:49 +, Andrew Cooper wrote:
> On 09/11/15 12:00, Ian Campbell wrote:
> > diff --git a/tools/libs/call/linux.c b/tools/libs/call/linux.c
> > new file mode 100644
> > index 000..906ca7e
> > --- /dev/null
> > +++ b/tools/libs/call/linux.c
> > @@ -0,0 +1,132 @@
> > +/*
> > + * This library is free software; you can redistribute it and/or
> > + * modify it under the terms of the GNU Lesser General Public
> > + * License as published by the Free Software Foundation;
> > + * version 2.1 of the License.
> > + *
> > + * This library is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> > + * Lesser General Public License for more details.
> > + *
> > + * You should have received a copy of the GNU Lesser General Public
> > + * License along with this library; If not, see  > censes/>.
> > + *
> > + * Split out from xc_linus_osdep.c:
> > + *
> > + * Copyright 2006 Sun Microsystems, Inc.  All rights reserved.
> > + */
> > +
> > +#include 
> > +#include 
> > +#include 
> > +
> > +#include 
> > +#include 
> > +
> > +#include "private.h"
> > +
> > +int osdep_xencall_open(xencall_handle *xcall)
> > +{
> > +int flags, saved_errno;
> > +int fd = open("/proc/xen/privcmd", O_RDWR);
> 
> I know this is a straight copy, but these new libraries really should
> start with the PVOps interface of /dev/xen/privcmd before falling back
> to the older classic-linux interfaces under /proc

Right, I'll pick this up for (almost, modulo resolving a conflict or two)
free when I rebase over Doug's patch to do this, which I'll be doing before
reposting.

> Ideally, once all these library improvements are done, there should be
> no need to mount xenfs in a Linux PVops domain.
> 
> > +
> > +if ( fd == -1 )
> > +{
> > +PERROR("Could not obtain handle on privileged command
> > interface");
> > +return -1;
> > +}
> > +
> > +/* Although we return the file handle as the 'xc handle' the API
> > +   does not specify / guarentee that this integer is in fact
> > +   a file handle. Thus we must take responsiblity to ensure
> > +   it doesn't propagate (ie leak) outside the process */
> 
> Again, I know this is a straight copy, but it is racy.  The open() call
> needs O_CLOEXEC.
> 
> There might be some issues here to do with older versions of libc.

Or indeed super old versions of Linux (O_CLOEXEC came with 2.6.23 according
to man).

I'm tempted to hedge and use O_CLOEXEC with /dev/xen/privcmd, and leave the
legacy /proc/xen/privcmd path doing it this way, after all we've managed to
get this far...

> > +/* Do not copy the VMA to child process on fork. Avoid the page
> > being COW
> > +on hypercall. */
> > +rc = madvise(p, npages * PAGE_SIZE, MADV_DONTFORK);
> 
> This also seems racy, although it is unclear from the manpages how to
> avoid the race.

pthread_atfork() might be one approach, and libxencall already uses
pthreads (for locking the cache), so it wouldn't be a problem to add more
uses, I don't think.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] linux 4.4 Regression: 100% cpu usage on idle pv guest under Xen with single vcpu.

2015-12-02 Thread David Vrabel
On 28/11/15 15:47, Sander Eikelenboom wrote:
> genirq: Flags mismatch irq 8.  (hvc_console) vs.  (rtc0)

We shouldn't register an rtc_cmos device because its legacy irq
conflicts with the irq needed for hvc0.  For a multi VCPU guest irq 8 is
in use for the pv spinlocks and this gets requested first, preventing
the rtc device from probing.

Does this patch fix it for you?

David
8<
x86: rtc_cmos platform device requires legacy irqs

Adding the rtc platform device when there are no legacy irqs (no
legacy PIC) causes a conflict with other devices that end up using the
same irq number.

In a single VCPU PV guest we should have:

/proc/interrupts:
   CPU0
  0:   4934  xen-percpu-virq  timer0
  1:  0  xen-percpu-ipi   spinlock0
  2:  0  xen-percpu-ipi   resched0
  3:  0  xen-percpu-ipi   callfunc0
  4:  0  xen-percpu-virq  debug0
  5:  0  xen-percpu-ipi   callfuncsingle0
  6:  0  xen-percpu-ipi   irqwork0
  7:321   xen-dyn-event xenbus
  8: 90   xen-dyn-event hvc_console
  ...

But hvc_console cannot get its interrupt because it is already in use
by rtc0 and the console does not work.

  genirq: Flags mismatch irq 8.  (hvc_console) vs.  (rtc0)

The rtc_cmos device requires a particular legacy irq so don't add it
if there are no legacy irqs.

Signed-off-by: David Vrabel 
---
 arch/x86/kernel/rtc.c | 5 +
 1 file changed, 5 insertions(+)

diff --git a/arch/x86/kernel/rtc.c b/arch/x86/kernel/rtc.c
index cd96852..07c70f1 100644
--- a/arch/x86/kernel/rtc.c
+++ b/arch/x86/kernel/rtc.c
@@ -14,6 +14,7 @@
 #include 
 #include 
 #include 
+#include 

 #ifdef CONFIG_X86_32
 /*
@@ -200,6 +201,10 @@ static __init int add_rtc_cmos(void)
}
 #endif

+   /* RTC uses legacy IRQs. */
+   if (!nr_legacy_irqs())
+   return -ENODEV;
+
platform_device_register(&rtc_device);
dev_info(&rtc_device.dev,
 "registered platform RTC device (no PNP device found)\n");
-- 
2.1.4




___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH] xen-pciback: fix up cleanup path when alloc fails

2015-12-02 Thread Doug Goldstein
On 12/2/15 4:35 AM, David Vrabel wrote:
> On 26/11/15 20:32, Doug Goldstein wrote:
>> When allocating a pciback device fails, avoid the possibility of a
>> use after free.
> 
> We should not require clearing drvdata for correctness.  We should
> ensure we retain drvdata for as long as it is needed.
> 
> I note that pcistub_device_release() has:
> 
>   kfree(dev_data);
>   pci_set_drvdata(dev, NULL);
> 
>   /* Clean-up the device */
>   xen_pcibk_config_free_dyn_fields(dev);
>   xen_pcibk_config_free_dev(dev);
> 
> Which should (at a minimum) be reordered to move the kfree(dev_data) to
> after the calls that require it
> 
> David
> 

I apologize but at this point I'm confused at what action I should be
taking. Are you saying NACK to the original patch and suggesting this as
the replacement? Or saying that this should be done in addition to the
original patch?

I created the original patch when looking through the other probe()
calls and seeing that they all did pci_set_drvdata() with memory they
allocated but probe() failed they ensured that pci_set_drvdata() was
cleared. But the behavior in xen-pciback was different. It kfree()'d the
memory that passed to pci_set_drvdata() and never set that pointer to
NULL. Which could possibly result in a use after free. The use after
free doesn't occur today as Konrad pointed out but in the future its
possible should some other code changes occur. It was more of a
defensive coding patch in the end. I had planned on resubmitting the
patch with a reworded commit message after Konrad pointed out there was
currently no use after free and retaining the Reviewed-By since the code
wouldn't change but if that's not what I should be doing I will gladly
go another route.

-- 
Doug Goldstein



signature.asc
Description: OpenPGP digital signature
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Skylake: VT-d and other error messages

2015-12-02 Thread Eric Shelton
I have attached the output from 'xl dmesg' with iommu=debug
apic_verbosity=debug loglvl=all

This is with 4.6.0, with the recent patch enabling mwait-idle on Skylake
applied.  My initial post was running the latest out of xen.git master.  I
may be able to try to newer code again later this week.

Eric


On Wed, Dec 2, 2015 at 5:45 AM, Jan Beulich  wrote:

> >>> On 02.12.15 at 03:51,  wrote:
> > ASRock Z170 Extreme 4 with BIOS version 2.30 was motherboard #1
> > Gigabyte GA-Z170XP-SLI with BIOS version F5 was motherboard #2
> >
> > As I initially reported, both were reporting the error involving f0:1f.0
> >
> > Not surprisingly, there are no f0:xx.y devices.  The closest I suppose
> is:
> > 00:1f.0 ISA Bridge: Intel Corporation Sunrise Point-H LPC Controller (rev
> > 31)
>
> Did you also check "iommu=verbose" output for occurrences of this
> ID (e.g. representing an IO-APIC or HPET)?
>
> Jan
>
>
 Xen 4.6.0-9.fc20
(XEN) Xen version 4.6.0 (user@) (gcc (GCC) 4.8.3 20140911 (Red Hat 4.8.3-7)) 
debug=n Mon Nov 30 22:31:58 UTC 2015
(XEN) Latest ChangeSet: 
(XEN) Bootloader: GRUB 2.00
(XEN) Command line: placeholder iommu=debug apic_verbosity=debug loglvl=all 
console=none dom0_mem=min:1024M dom0_mem=max:4096M
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN)  Found 1 MBR signatures
(XEN)  Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN)   - 0009c800 (usable)
(XEN)  0009c800 - 000a (reserved)
(XEN)  000e - 0010 (reserved)
(XEN)  0010 - 69c3a000 (usable)
(XEN)  69c3a000 - 69c3b000 (ACPI NVS)
(XEN)  69c3b000 - 69c65000 (reserved)
(XEN)  69c65000 - 69cb8000 (usable)
(XEN)  69cb8000 - 6a3bd000 (reserved)
(XEN)  6a3bd000 - 76b99000 (usable)
(XEN)  76b99000 - 777b (reserved)
(XEN)  777b - 77f9a000 (ACPI NVS)
(XEN)  77f9a000 - 77fff000 (ACPI data)
(XEN)  77fff000 - 7800 (usable)
(XEN)  7800 - 7810 (reserved)
(XEN)  e000 - f000 (reserved)
(XEN)  fe00 - fe011000 (reserved)
(XEN)  fec0 - fec01000 (reserved)
(XEN)  fee0 - fee01000 (reserved)
(XEN)  ff00 - 0001 (reserved)
(XEN)  0001 - 00087600 (usable)
(XEN) ACPI: RSDP 000F05B0, 0024 (r2 ALASKA)
(XEN) ACPI: XSDT 77A1F098, 00B4 (r1 ALASKA   A M I   1072009 AMI 10013)
(XEN) ACPI: FACP 77A41B30, 010C (r5 ALASKA   A M I   1072009 AMI 10013)
(XEN) ACPI: DSDT 77A1F1E8, 22943 (r2 ALASKA   A M I   1072009 INTL 20120913)
(XEN) ACPI: FACS 77F99F80, 0040
(XEN) ACPI: APIC 77A41C40, 0084 (r3 ALASKA   A M I   1072009 AMI 10013)
(XEN) ACPI: FPDT 77A41CC8, 0044 (r1 ALASKA   A M I   1072009 AMI 10013)
(XEN) ACPI: FIDT 77A41D10, 009C (r1 ALASKA   A M I   1072009 AMI 10013)
(XEN) ACPI: MCFG 77A41DB0, 003C (r1 ALASKAA M I  1072009 MSFT   97)
(XEN) ACPI: HPET 77A41DF0, 0038 (r1 ALASKAA M I  1072009 AMI.5000B)
(XEN) ACPI: SSDT 77A41E28, 036D (r1 SataRe SataTabl 1000 INTL 20120913)
(XEN) ACPI: LPIT 77A42198, 0094 (r1 INTEL   SKL0 MSFT   5F)
(XEN) ACPI: SSDT 77A42230, 002B (r2 INTEL  UsbCTabl 1000 INTL 20120913)
(XEN) ACPI: DBGP 77A42260, 0034 (r1 INTEL  0 MSFT   5F)
(XEN) ACPI: DBG2 77A42298, 0054 (r0 INTEL  0 MSFT   5F)
(XEN) ACPI: SSDT 77A422F0, 0618 (r2  INTEL xh_rvp080 INTL 20120913)
(XEN) ACPI: AAFT 77A42908, 051A (r1 ALASKA OEMAAFT   1072009 MSFT   97)
(XEN) ACPI: SSDT 77A42E28, 5212 (r2 SaSsdt  SaSsdt  3000 INTL 20120913)
(XEN) ACPI: UEFI 77A48040, 0042 (r10 0)
(XEN) ACPI: SSDT 77A48088, 0E58 (r2 CpuRef  CpuSsdt 3000 INTL 20120913)
(XEN) ACPI: DMAR 77A48EE0, 00A8 (r1 INTEL  SKL 1 INTL1)
(XEN) ACPI: ASF! 77A48F88, 00A5 (r32 INTEL   HCG1 TFSMF4240)
(XEN) System RAM: 32452MB (33230872kB)
(XEN) No NUMA configuration found
(XEN) Faking a node at -00087600
(XEN) Domain heap initialised
(XEN) found SMP MP-table at 000fc6a0
(XEN) DMI 2.8 present.
(XEN) APIC boot state is 'xapic'
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x1808
(XEN) ACPI: v5 SLEEP INFO: control[1:1804], status[1:1800]
(XEN) ACPI: Invalid sleep control/status register data: 0:0x8:0x3 0:0x8:0x3
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1:1804,1:0], pm1x_evt[1:1800,1:0]
(XEN) ACPI: 32/64X FACS address mismatch in FADT - 77f99f80/, 
using 32
(XEN) ACPI: wakeup_vec[77f99f8c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee0
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
(XEN) Processor #0 7:14 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)

Re: [Xen-devel] [PATCH v2] x86: Fixup IRQs when CPUs go down during shutdown

2015-12-02 Thread Ross Lagerwall

On 12/02/2015 02:02 PM, Jan Beulich wrote:

On 02.12.15 at 14:46,  wrote:

Commit fc0c3fa2ad5c ("x86/IO-APIC: fix setup of Xen internally used IRQs
(take 2)") introduced a regression on some hardware where Xen would hang
during shutdown, repeating the following message:
APIC error on CPU0: 08(08), Receive accept error

This appears to be because an interrupt (in this case from the serial
console) destined for a CPU other than the boot CPU is left unhandled so
an APIC error on CPU 0 is generated instead.

To fix this, before taking down the non-boot CPUs, call fixup_irqs()
with a CPU mask of only the boot CPU to reset the IRQ affinities
correctly.

Signed-off-by: Ross Lagerwall 
---


Even though in this case interested people may know, missing info
on changes from previous version here.


+/* CPU(s) have been removed from mask.  Re-set irq affinities. */
+void fixup_irqs(const cpumask_t *mask, bool_t verbose)


The comment doesn't match reality. And I wonder whether it
wouldn't be reasonable to imply "verbose" (either from mask
equaling &cpu_online_map, or by introducing SYS_STATE_shutdown
and/or SYS_STATE_reboot).


I considered introducing a new SYS_STATE but decided that a function 
parameter was clearer rather than implying it from some other global state.





@@ -2385,16 +2382,27 @@ void fixup_irqs(void)

  spin_unlock(&desc->lock);

-if ( break_affinity && set_affinity )
-printk("Broke affinity for irq %i\n", irq);
-else if ( !set_affinity )
-printk("Cannot set affinity for irq %i\n", irq);
+if ( verbose )
+{
+if ( break_affinity && set_affinity )
+printk("Broke affinity for irq %i\n", irq);
+else if ( !set_affinity )
+printk("Cannot set affinity for irq %i\n", irq);
+}


How about

 if ( !verbose )
continue;

limiting churn on code?


OK.




--- a/xen/arch/x86/smp.c
+++ b/xen/arch/x86/smp.c
@@ -286,6 +286,7 @@ void __stop_this_cpu(void)

  static void stop_this_cpu(void *dummy)
  {
+fixup_eoi();
  __stop_this_cpu();


Is this really needed during shutdown?


Possibly not, but I think it's cleaner to do the same as what is used 
for CPU down.





@@ -298,6 +299,13 @@ static void stop_this_cpu(void *dummy)
  void smp_send_stop(void)
  {
  int timeout = 10;
+cpumask_t online;
+
+cpumask_clear(&online);
+cpumask_set_cpu(smp_processor_id(), &online);


That's what we have cpumask_of() for.



OK.

--
Ross Lagerwall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v1 2/2] libxl: re-implement libxl__xs_printf()

2015-12-02 Thread Ian Campbell
On Tue, 2015-12-01 at 13:55 +, Paul Durrant wrote:
> This patch adds a new libxl__xs_vprintf() which actually checks the
> success of the underlying call to xs_write() (logging if it fails) and
> then re-implements libxl__xs_printf() using this (and replacing the
> call to vasprintf() with a call to libxl__vsprintf()).

Is the addition of libxl__xs_vprintf() an end in itself (i.e. do you have
an upcoming use for it) or just an artefact of how you decided to fix
libxl__xs_printf()?

Ian.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v2] libxc: try to find last used pfn when migrating

2015-12-02 Thread Ian Campbell
On Wed, 2015-12-02 at 12:36 +, Andrew Cooper wrote:
> On 02/12/15 07:42, Juergen Gross wrote:
> > diff --git a/tools/libxc/xc_sr_save_x86_hvm.c
> > b/tools/libxc/xc_sr_save_x86_hvm.c
> > index cdee774..3c879ed 100644
> > --- a/tools/libxc/xc_sr_save_x86_hvm.c
> > +++ b/tools/libxc/xc_sr_save_x86_hvm.c
> > @@ -135,6 +135,20 @@ static int x86_hvm_normalise_page(struct
> > xc_sr_context *ctx,
> >  static int x86_hvm_setup(struct xc_sr_context *ctx)
> >  {
> >  xc_interface *xch = ctx->xch;
> > +xen_pfn_t nr_pfns;
> > +
> > +if ( xc_domain_nr_gpfns(xch, ctx->domid, &nr_pfns) < 0 )
> > +{
> > +PERROR("Unable to obtain the guest p2m size");
> > +return -1;
> > +}
> > +if ( nr_pfns > ~XEN_DOMCTL_PFINFO_LTAB_MASK )
> > +{
> > +PERROR("Cannot save this big a guest");
> 
> Strictly speaking to match the moved code, this should set errno = E2BIG.
> 
> However, the error handling in libxc is in a dire state, and the error
> message is retained, which is the important point.
> 
> Entire patch Reviewed-by: Andrew Cooper  with
> or without the errno tweaks.

I could make the errno tweak on commit, if there is agreement.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH v2] build: fix clean to remove all .o and .d files

2015-12-02 Thread Jonathan Creekmore
In commit 8b6ef9c152edceabecc7f90c811cd538a7b7a110, several files in
xen/common/compat were changed to be built using the Makefile in
xen/common, by appending the compat prefix to the object
files. Additionally, the xen/common/compat directory was removed from
the subdirs-y variable, so it is no longer visited by the clean
rule. This resulted in some object files and dependency files being
generated by inclusion into obj-y, but not cleaned because they lived in a
directory that was unvisited by the clean rules.

Since there is a desire for all of the object files and dependency files
to be cleaned, just search for all objects and dependency files and
delete them on clean. The previous method of only tracking with the
$(DEPS) and *.o in the clean rules had the disadvantage that, if the
configuration changed between a build and a clean, some of the
dependencies or objects could get left behind. This method does not have
the same disadvantage.

CC: Ian Campbell 
CC: Ian Jackson 
CC: Jan Beulich 
CC: Keir Fraser 
CC: Tim Deegan 
Signed-off-by: Jonathan Creekmore 
---
 xen/Makefile | 3 ++-
 xen/Rules.mk | 2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/xen/Makefile b/xen/Makefile
index 3a1de99..365c477 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -91,7 +91,8 @@ _clean: delete-unfresh-files
$(MAKE) -f $(BASEDIR)/Rules.mk -C xsm clean
$(MAKE) -f $(BASEDIR)/Rules.mk -C crypto clean
$(MAKE) -f $(BASEDIR)/Rules.mk -C arch/$(TARGET_ARCH) clean
-   rm -f include/asm *.o $(TARGET) $(TARGET).gz $(TARGET).efi 
$(TARGET)-syms *~ core $(DEPS)
+   find . \( -name "*.o" -o -name "*.d" \) -exec rm -f {} \;
+   rm -f include/asm $(TARGET) $(TARGET).gz $(TARGET).efi $(TARGET)-syms 
*~ core
rm -f include/asm-*/asm-offsets.h
rm -f .banner
 
diff --git a/xen/Rules.mk b/xen/Rules.mk
index 02db110..123a4a7 100644
--- a/xen/Rules.mk
+++ b/xen/Rules.mk
@@ -173,7 +173,7 @@ FORCE:
 
 .PHONY: clean
 clean:: $(addprefix _clean_, $(subdir-all))
-   rm -f *.o *~ core $(DEPS)
+   rm -f *~ core
 _clean_%/: FORCE
$(MAKE) -f $(BASEDIR)/Rules.mk -C $* clean
 
-- 
2.6.2


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v2] x86: Fixup IRQs when CPUs go down during shutdown

2015-12-02 Thread Jan Beulich
>>> On 02.12.15 at 16:09,  wrote:
> On 12/02/2015 02:02 PM, Jan Beulich wrote:
> On 02.12.15 at 14:46,  wrote:
>>> --- a/xen/arch/x86/smp.c
>>> +++ b/xen/arch/x86/smp.c
>>> @@ -286,6 +286,7 @@ void __stop_this_cpu(void)
>>>
>>>   static void stop_this_cpu(void *dummy)
>>>   {
>>> +fixup_eoi();
>>>   __stop_this_cpu();
>>
>> Is this really needed during shutdown?
> 
> Possibly not, but I think it's cleaner to do the same as what is used 
> for CPU down.

I'm not convinced. Andrew?

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v2] libxc: try to find last used pfn when migrating

2015-12-02 Thread Juergen Gross
On 02/12/15 16:28, Ian Campbell wrote:
> On Wed, 2015-12-02 at 12:36 +, Andrew Cooper wrote:
>> On 02/12/15 07:42, Juergen Gross wrote:
>>> diff --git a/tools/libxc/xc_sr_save_x86_hvm.c
>>> b/tools/libxc/xc_sr_save_x86_hvm.c
>>> index cdee774..3c879ed 100644
>>> --- a/tools/libxc/xc_sr_save_x86_hvm.c
>>> +++ b/tools/libxc/xc_sr_save_x86_hvm.c
>>> @@ -135,6 +135,20 @@ static int x86_hvm_normalise_page(struct
>>> xc_sr_context *ctx,
>>>  static int x86_hvm_setup(struct xc_sr_context *ctx)
>>>  {
>>>  xc_interface *xch = ctx->xch;
>>> +xen_pfn_t nr_pfns;
>>> +
>>> +if ( xc_domain_nr_gpfns(xch, ctx->domid, &nr_pfns) < 0 )
>>> +{
>>> +PERROR("Unable to obtain the guest p2m size");
>>> +return -1;
>>> +}
>>> +if ( nr_pfns > ~XEN_DOMCTL_PFINFO_LTAB_MASK )
>>> +{
>>> +PERROR("Cannot save this big a guest");
>>
>> Strictly speaking to match the moved code, this should set errno = E2BIG.
>>
>> However, the error handling in libxc is in a dire state, and the error
>> message is retained, which is the important point.
>>
>> Entire patch Reviewed-by: Andrew Cooper  with
>> or without the errno tweaks.
> 
> I could make the errno tweak on commit, if there is agreement.

Sure, go ahead.


Juergen


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v1 2/2] libxl: re-implement libxl__xs_printf()

2015-12-02 Thread Paul Durrant
> -Original Message-
> From: Ian Campbell [mailto:ian.campb...@citrix.com]
> Sent: 02 December 2015 15:21
> To: Paul Durrant; xen-de...@lists.xenproject.org
> Cc: Ian Jackson; Stefano Stabellini; Wei Liu
> Subject: Re: [PATCH v1 2/2] libxl: re-implement libxl__xs_printf()
> 
> On Tue, 2015-12-01 at 13:55 +, Paul Durrant wrote:
> > This patch adds a new libxl__xs_vprintf() which actually checks the
> > success of the underlying call to xs_write() (logging if it fails) and
> > then re-implements libxl__xs_printf() using this (and replacing the
> > call to vasprintf() with a call to libxl__vsprintf()).
> 
> Is the addition of libxl__xs_vprintf() an end in itself (i.e. do you have
> an upcoming use for it) or just an artefact of how you decided to fix
> libxl__xs_printf()?
> 

It's an artefact but seemed sufficiently useful to expose.

  Paul

> Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Skylake: VT-d and other error messages

2015-12-02 Thread Jan Beulich
>>> On 02.12.15 at 16:04,  wrote:
> I have attached the output from 'xl dmesg' with iommu=debug
> apic_verbosity=debug loglvl=all

As of 4.6.0 it actually takes a debug hypervisor to get those
messages logged (which arguably is wrong, and which quite
certainly is wrong for messages accompanying error returns).

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v2] build: fix clean to remove all .o and .d files

2015-12-02 Thread Jan Beulich
>>> On 02.12.15 at 16:29,  wrote:
> --- a/xen/Makefile
> +++ b/xen/Makefile
> @@ -91,7 +91,8 @@ _clean: delete-unfresh-files
>   $(MAKE) -f $(BASEDIR)/Rules.mk -C xsm clean
>   $(MAKE) -f $(BASEDIR)/Rules.mk -C crypto clean
>   $(MAKE) -f $(BASEDIR)/Rules.mk -C arch/$(TARGET_ARCH) clean
> - rm -f include/asm *.o $(TARGET) $(TARGET).gz $(TARGET).efi 
> $(TARGET)-syms *~ core $(DEPS)
> + find . \( -name "*.o" -o -name "*.d" \) -exec rm -f {} \;

If you really meant *.d, then *.[od] would have done. But in fact I
think we want to limit this to ".*.d". Which I could fix up while
committing, but then I'm not sure ...

> --- a/xen/Rules.mk
> +++ b/xen/Rules.mk
> @@ -173,7 +173,7 @@ FORCE:
>  
>  .PHONY: clean
>  clean:: $(addprefix _clean_, $(subdir-all))
> - rm -f *.o *~ core $(DEPS)
> + rm -f *~ core

... this is a good idea, as it's not clear to me whether "clean" actually
works when invoked in sub-trees of xen/.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Crash in set_cpu_sibling_map() booting Xen 4.6.0 on Fusion

2015-12-02 Thread Ed Swierk
I tested on VMware Fusion with 3, 4 and 8 CPUs, and it works in all cases.

(XEN) Xen version 4.6.1-pre ( 4.6.1~pre-1skyport1)
(eswi...@skyportsystems.com) (gcc (Debian 5.2.1-19.1skyport1) 5.2.1
20150930) debug=n Wed Dec  2 07:22:20 PST 2015
(XEN) Bootloader: SYSLINUX 4.05 20140113
(XEN) Command line: xen console=com1,vga com1=115200 no-bootscrub
dom0_mem=2048M,max:2048M loglvl=all cpuinfo=1
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN) Disc information:
(XEN)  Found 1 MBR signatures
(XEN)  Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN)   - 0009f800 (usable)
(XEN)  0009f800 - 000a (reserved)
(XEN)  000dc000 - 0010 (reserved)
(XEN)  0010 - bfef (usable)
(XEN)  bfef - bfeff000 (ACPI data)
(XEN)  bfeff000 - bff0 (ACPI NVS)
(XEN)  bff0 - c000 (usable)
(XEN)  f000 - f800 (reserved)
(XEN)  fec0 - fec1 (reserved)
(XEN)  fee0 - fee01000 (reserved)
(XEN)  fffe - 0001 (reserved)
(XEN)  0001 - 0001c000 (usable)
(XEN) ACPI: RSDP 000F6A10, 0024 (r2 PTLTD )
(XEN) ACPI: XSDT BFEF030B, 0054 (r1 INTEL  440BX 604 VMW   1324272)
(XEN) ACPI: FACP BFEFEE73, 00F4 (r4 INTEL  440BX 604 PTL F4240)
(XEN) ACPI: DSDT BFEF05B1, E8C2 (r1 PTLTD  Custom604 MSFT  301)
(XEN) ACPI: FACS BFEFFFC0, 0040
(XEN) ACPI: BOOT BFEF0589, 0028 (r1 PTLTD  $SBFTBL$  604  LTP1)
(XEN) ACPI: APIC BFEF050F, 007A (r1 PTLTD   APIC604  LTP0)
(XEN) ACPI: MCFG BFEF04D3, 003C (r1 PTLTD  $PCITBL$  604  LTP1)
(XEN) ACPI: SRAT BFEF03C3, 0110 (r2 VMWARE MEMPLUG   604 VMW 1)
(XEN) ACPI: WAET BFEF039B, 0028 (r1 VMWARE VMW WAET  604 VMW 1)
(XEN) System RAM: 6143MB (6291004kB)
(XEN) SRAT: PXM 0 -> APIC 00 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 02 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 04 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 06 -> Node 0
(XEN) SRAT: Node 0 PXM 0 0-a
(XEN) SRAT: Node 0 PXM 0 10-1000
(XEN) SRAT: Node 0 PXM 0 1000-c000
(XEN) SRAT: Node 0 PXM 0 1-1c000
(XEN) NUMA: Allocated memnodemap from 1bdc5 - 1bdc52000
(XEN) NUMA: Using 8 for the hash shift.
(XEN) Domain heap initialised
(XEN) found SMP MP-table at 000f6a80
(XEN) DMI present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x1008
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1:1004,1:0], pm1x_evt[1:1000,1:0]
(XEN) ACPI: wakeup_vec[bfefffcc], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee0
(XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
(XEN) Processor #0 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x02] enabled)
(XEN) Processor #2 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x04] enabled)
(XEN) Processor #4 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x06] enabled)
(XEN) Processor #6 6:6 APIC version 21
(XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x01] address[0xfec0] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 1, version 17, address 0xfec0, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
(XEN) ERST table was not found
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 4 CPUs (0 hotplug CPUs)
(XEN) IRQ limits: 24 GSI, 760 MSI/MSI-X
(XEN) Not enabling x2APIC: depends on iommu_supports_eim.
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7
(XEN) CPU0: No MCE banks present. Machine check support disabled
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Initializing CPU#0
(XEN) Detected 2592.669 MHz processor.
(XEN) Initing memory sharing.
(XEN) alt table 82d0802bd090 -> 82d0802be2c0
(XEN) PCI: MCFG configuration 0: base f000 segment  buses 00 - 7f
(XEN) PCI: MCFG area at f000 reserved in E820
(XEN) PCI: Using MCFG for segment  bus 00-7f
(XEN) I/O virtualisation disabled
(XEN) CPU0: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) nr_sockets: 7
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using new ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) Platform timer is 3.579MHz ACPI PM Timer
(XEN) Allocated console ring of 32 KiB.
(XEN) mwait-idle: MWAIT substates: 0x10
(XEN) mwait-idle: v0.4 model 0x46
(XEN) mwait-idle: lapic_timer_reliable_states 0x
(XEN) VMX: Supported advanced features:
(XEN)  - APIC TPR shadow

Re: [Xen-devel] [PATCH v2] build: fix clean to remove all .o and .d files

2015-12-02 Thread Jonathan Creekmore

> On Dec 2, 2015, at 9:36 AM, Jan Beulich  wrote:
> 
 On 02.12.15 at 16:29,  wrote:
>> --- a/xen/Makefile
>> +++ b/xen/Makefile
>> @@ -91,7 +91,8 @@ _clean: delete-unfresh-files
>>  $(MAKE) -f $(BASEDIR)/Rules.mk -C xsm clean
>>  $(MAKE) -f $(BASEDIR)/Rules.mk -C crypto clean
>>  $(MAKE) -f $(BASEDIR)/Rules.mk -C arch/$(TARGET_ARCH) clean
>> -rm -f include/asm *.o $(TARGET) $(TARGET).gz $(TARGET).efi 
>> $(TARGET)-syms *~ core $(DEPS)
>> +find . \( -name "*.o" -o -name "*.d" \) -exec rm -f {} \;
> 
> If you really meant *.d, then *.[od] would have done. But in fact I
> think we want to limit this to ".*.d". Which I could fix up while
> committing, but then I'm not sure …

Easy enough to change it to “.*.d”.

> 
>> --- a/xen/Rules.mk
>> +++ b/xen/Rules.mk
>> @@ -173,7 +173,7 @@ FORCE:
>> 
>> .PHONY: clean
>> clean:: $(addprefix _clean_, $(subdir-all))
>> -rm -f *.o *~ core $(DEPS)
>> +rm -f *~ core
> 
> ... this is a good idea, as it's not clear to me whether "clean" actually
> works when invoked in sub-trees of xen/.

I can definitely take that bit out; I just didn’t see the point in replicating 
the
removal of the files. Note that, whether I take it out or leave it in, “clean” 
still
will not totally work when invoked in sub-trees of xen/ because *.o is still not
enough to catch all of the object files (which was the whole point of this patch
in the first place).
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Skylake: VT-d and other error messages

2015-12-02 Thread Jan Beulich
>>> On 02.12.15 at 16:37,  wrote:
> On Dec 2, 2015, at 10:28 AM, Jan Beulich  wrote:
> 
> On 02.12.15 at 16:04,  wrote:
>>> I have attached the output from 'xl dmesg' with iommu=debug
>>> apic_verbosity=debug loglvl=all
>> 
>> As of 4.6.0 it actually takes a debug hypervisor to get those
>> messages logged (which arguably is wrong, and which quite
>> certainly is wrong for messages accompanying error returns).
> 
> So when I rebuild the hypervisor, it sounds like I should be doing a debug 
> build?

Yes please.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v2] build: fix clean to remove all .o and .d files

2015-12-02 Thread Jan Beulich
>>> On 02.12.15 at 16:41,  wrote:

>> On Dec 2, 2015, at 9:36 AM, Jan Beulich  wrote:
>> 
> On 02.12.15 at 16:29,  wrote:
>>> --- a/xen/Makefile
>>> +++ b/xen/Makefile
>>> @@ -91,7 +91,8 @@ _clean: delete-unfresh-files
>>> $(MAKE) -f $(BASEDIR)/Rules.mk -C xsm clean
>>> $(MAKE) -f $(BASEDIR)/Rules.mk -C crypto clean
>>> $(MAKE) -f $(BASEDIR)/Rules.mk -C arch/$(TARGET_ARCH) clean
>>> -   rm -f include/asm *.o $(TARGET) $(TARGET).gz $(TARGET).efi 
>>> $(TARGET)-syms 
> *~ core $(DEPS)
>>> +   find . \( -name "*.o" -o -name "*.d" \) -exec rm -f {} \;
>> 
>> If you really meant *.d, then *.[od] would have done. But in fact I
>> think we want to limit this to ".*.d". Which I could fix up while
>> committing, but then I'm not sure …
> 
> Easy enough to change it to “.*.d”.
> 
>> 
>>> --- a/xen/Rules.mk
>>> +++ b/xen/Rules.mk
>>> @@ -173,7 +173,7 @@ FORCE:
>>> 
>>> .PHONY: clean
>>> clean:: $(addprefix _clean_, $(subdir-all))
>>> -   rm -f *.o *~ core $(DEPS)
>>> +   rm -f *~ core
>> 
>> ... this is a good idea, as it's not clear to me whether "clean" actually
>> works when invoked in sub-trees of xen/.
> 
> I can definitely take that bit out; I just didn’t see the point in 
> replicating the
> removal of the files. Note that, whether I take it out or leave it in, 
> “clean” still
> will not totally work when invoked in sub-trees of xen/ because *.o is still 
> not
> enough to catch all of the object files (which was the whole point of this 
> patch
> in the first place).

True, but let's not make matters worse. I'd be fine leaving this
second change in only if we knew "clean" doesn't work at all when
invoked against sub-trees of xen/.

And again - no reason to re-submit, I can easily tweak the patch
upon commit, as long as you're fine with this being done with your
S-o-b in place.

Jan

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v2] build: fix clean to remove all .o and .d files

2015-12-02 Thread Jonathan Creekmore

> On Dec 2, 2015, at 9:46 AM, Jan Beulich  wrote:
> 
 On 02.12.15 at 16:41,  wrote:
> 
>>> On Dec 2, 2015, at 9:36 AM, Jan Beulich  wrote:
>>> 
>> On 02.12.15 at 16:29,  wrote:
 --- a/xen/Makefile
 +++ b/xen/Makefile
 @@ -91,7 +91,8 @@ _clean: delete-unfresh-files
$(MAKE) -f $(BASEDIR)/Rules.mk -C xsm clean
$(MAKE) -f $(BASEDIR)/Rules.mk -C crypto clean
$(MAKE) -f $(BASEDIR)/Rules.mk -C arch/$(TARGET_ARCH) clean
 -  rm -f include/asm *.o $(TARGET) $(TARGET).gz $(TARGET).efi 
 $(TARGET)-syms 
>> *~ core $(DEPS)
 +  find . \( -name "*.o" -o -name "*.d" \) -exec rm -f {} \;
>>> 
>>> If you really meant *.d, then *.[od] would have done. But in fact I
>>> think we want to limit this to ".*.d". Which I could fix up while
>>> committing, but then I'm not sure …
>> 
>> Easy enough to change it to “.*.d”.
>> 
>>> 
 --- a/xen/Rules.mk
 +++ b/xen/Rules.mk
 @@ -173,7 +173,7 @@ FORCE:
 
 .PHONY: clean
 clean:: $(addprefix _clean_, $(subdir-all))
 -  rm -f *.o *~ core $(DEPS)
 +  rm -f *~ core
>>> 
>>> ... this is a good idea, as it's not clear to me whether "clean" actually
>>> works when invoked in sub-trees of xen/.
>> 
>> I can definitely take that bit out; I just didn’t see the point in 
>> replicating the
>> removal of the files. Note that, whether I take it out or leave it in, 
>> “clean” still
>> will not totally work when invoked in sub-trees of xen/ because *.o is still 
>> not
>> enough to catch all of the object files (which was the whole point of this 
>> patch
>> in the first place).
> 
> True, but let's not make matters worse. I'd be fine leaving this
> second change in only if we knew "clean" doesn't work at all when
> invoked against sub-trees of xen/.
> 
> And again - no reason to re-submit, I can easily tweak the patch
> upon commit, as long as you're fine with this being done with your
> S-o-b in place.
> 

Yeah, that is fine.
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Skylake: VT-d and other error messages

2015-12-02 Thread Eric Shelton
On Dec 2, 2015, at 10:28 AM, Jan Beulich  wrote:

 On 02.12.15 at 16:04,  wrote:
>> I have attached the output from 'xl dmesg' with iommu=debug
>> apic_verbosity=debug loglvl=all
> 
> As of 4.6.0 it actually takes a debug hypervisor to get those
> messages logged (which arguably is wrong, and which quite
> certainly is wrong for messages accompanying error returns).

So when I rebuild the hypervisor, it sounds like I should be doing a debug 
build?

Eric
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCHv3] 1/3] libxc: prefer using privcmd character device

2015-12-02 Thread Ian Campbell
On Wed, 2015-12-02 at 11:42 +, Wei Liu wrote:
> On Tue, Dec 01, 2015 at 01:27:53PM -0600, Doug Goldstein wrote:
> > Prefer using the character device over the proc file if the character
> > device exists. This follows similar conversions of xenbus to avoid
> > issues with FMODE_ATOMIC_POS added in Linux 3.14 and newer.
> > 
> > CC: Ian Jackson 
> > CC: Stefano Stabellini 
> > CC: Ian Campbell 
> > CC: Wei Liu 
> > Signed-off-by: Doug Goldstein 
> 
> Acked-by: Wei Liu 

I believe this version also satisfies Ian's comments on v1, so I've applied
all 3 patches.

NB the extra "]" in the subject meant git am produced things like "1/3]
libxc: prefer using privcmd character device". I've fixed that up.

> 
> > ---
> >  tools/libxc/xc_linux_osdep.c | 8 +++-
> >  1 file changed, 7 insertions(+), 1 deletion(-)
> > 
> > diff --git a/tools/libxc/xc_linux_osdep.c
> > b/tools/libxc/xc_linux_osdep.c
> > index 76c55ff..c3a3a14 100644
> > --- a/tools/libxc/xc_linux_osdep.c
> > +++ b/tools/libxc/xc_linux_osdep.c
> > @@ -46,7 +46,13 @@
> >  static xc_osdep_handle linux_privcmd_open(xc_interface *xch)
> >  {
> >  int flags, saved_errno;
> > -int fd = open("/proc/xen/privcmd", O_RDWR);
> > +int fd = open("/dev/xen/privcmd", O_RDWR); /* prefer this newer
> > interface */
> > +
> > +if ( fd == -1 && ( errno == ENOENT || errno == ENXIO || errno ==
> > ENODEV ))
> > +{
> > +/* Fallback to /proc/xen/privcmd */
> > +fd = open("/proc/xen/privcmd", O_RDWR);
> > +}
> >  
> >  if ( fd == -1 )
> >  {

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH 0/2] libxc: domain builder related enhancements

2015-12-02 Thread Ian Campbell
On Tue, 2015-12-01 at 18:14 +0100, Juergen Gross wrote:
> Add some more checks for having got allocated memory in the domain
> builder. Related to this add an INVALID_PFN macro to be able to do
> these checks.
> 
> Juergen Gross (2):
>   libxc: replace INVALID_P2M_ENTRY by INVALID_PFN
>   libxc: do proper return code checking of allocator in domain builder

Both applied with Wei's ack.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH OSSTEST] cr-try-bisect-adhoc: Ensure tmp exists.

2015-12-02 Thread Ian Campbell
Signed-off-by: Ian Campbell 
---
 cr-try-bisect-adhoc | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/cr-try-bisect-adhoc b/cr-try-bisect-adhoc
index 5f8b7e1..f451324 100755
--- a/cr-try-bisect-adhoc
+++ b/cr-try-bisect-adhoc
@@ -27,6 +27,8 @@ nope () { echo "nope: $*"; exit 0; }
 
 if ! test -f job; then nope "no job"; exit 0; fi
 
+mkdir -p tmp
+
 . ./job
 # job should set:
 #   branch=
-- 
2.6.1


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v2] libxc: try to find last used pfn when migrating

2015-12-02 Thread Ian Campbell
On Wed, 2015-12-02 at 16:30 +0100, Juergen Gross wrote:
> On 02/12/15 16:28, Ian Campbell wrote:
> > On Wed, 2015-12-02 at 12:36 +, Andrew Cooper wrote:
> > > On 02/12/15 07:42, Juergen Gross wrote:
> > > > diff --git a/tools/libxc/xc_sr_save_x86_hvm.c
> > > > b/tools/libxc/xc_sr_save_x86_hvm.c
> > > > index cdee774..3c879ed 100644
> > > > --- a/tools/libxc/xc_sr_save_x86_hvm.c
> > > > +++ b/tools/libxc/xc_sr_save_x86_hvm.c
> > > > @@ -135,6 +135,20 @@ static int x86_hvm_normalise_page(struct
> > > > xc_sr_context *ctx,
> > > >  static int x86_hvm_setup(struct xc_sr_context *ctx)
> > > >  {
> > > >  xc_interface *xch = ctx->xch;
> > > > +xen_pfn_t nr_pfns;
> > > > +
> > > > +if ( xc_domain_nr_gpfns(xch, ctx->domid, &nr_pfns) < 0 )
> > > > +{
> > > > +PERROR("Unable to obtain the guest p2m size");
> > > > +return -1;
> > > > +}
> > > > +if ( nr_pfns > ~XEN_DOMCTL_PFINFO_LTAB_MASK )
> > > > +{
> > > > +PERROR("Cannot save this big a guest");
> > > 
> > > Strictly speaking to match the moved code, this should set errno =
> > > E2BIG.
> > > 
> > > However, the error handling in libxc is in a dire state, and the
> > > error
> > > message is retained, which is the important point.
> > > 
> > > Entire patch Reviewed-by: Andrew Cooper 
> > > with
> > > or without the errno tweaks.
> > 
> > I could make the errno tweak on commit, if there is agreement.
> 
> Sure, go ahead.

Will do it in my next commit pass (likely to be tomorrow).

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [PATCH OSSTEST] cr-try-bisect-adhoc: Set laundered_testid so graph URL is correct

2015-12-02 Thread Ian Campbell
Otherwise the testid is missing from the filename, resulting in e.g.
http://osstest.test-lab.xenproject.org/~osstest/pub/results-adhoc/bisect/xen-unstable/test-amd64-amd64-qemuu-nested-intel..svg

Instead of test-amd64-amd64-qemuu-nested-intel.debian-hvm-install-l1-l2.svg

Signed-off-by: Ian Campbell 
---
Untested...
---
 cr-try-bisect-adhoc | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/cr-try-bisect-adhoc b/cr-try-bisect-adhoc
index f451324..6d5d7c4 100755
--- a/cr-try-bisect-adhoc
+++ b/cr-try-bisect-adhoc
@@ -36,6 +36,8 @@ mkdir -p tmp
 #   testid=
 #   bisect= list of args to cs-bisection-step, eg  '--fail-flight='
 
+laundered_testid=${testid//\//--}
+
 . ./cri-bisect
 . ./cri-args-hostlists
 select_branch
-- 
2.6.1


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] Bitergia Review Process Tools Phase 2 (Plan of record, please review)

2015-12-02 Thread Lars Kurth
Hi folks,

I just got off the phone with Bitergia and in the second phase of the study, we 
will do the following.

1) More accurate matching of reviews to commits. This means
- Fix issues with capitalisation and white spaces, slashes, ...
- We do have a sizeable portion of patches that have the prefix corrected on 
check-in, thus make the matching resilient to such changes
- Deal with different git trees (internal): some patches posted to xen-devel 
end up in other repos but xen.git (e.g. osstest.git, mini-os.git, ...)
- Deal with different git trees (external): some patches posted to xen-devel 
end up in qemu, kernel, BSDs, ... these can be ignored, if they can't be 
matched. These tend to have qemu-de...@nongnu.org, various kernel lists, ... in 
the CC list
- If a significant proportion of reviews remain unmatched: try and handle 
spelling mistakes (I think these may be negligible)

2) Document the tools they have such that we can run them ourselves and check 
into one of our trees. We also have the option to run them as hosted service

In addition, we will implement something like 
http://s.bitergia.com/db-openstack-gerrit for the project with a number of 
filter and reporting capabilities. I hope this can help reviewers, 
maintainers/committers, contributors and me. This should help monitor the 
situation going forward and highlight any issues early on. In addition, it 
should also help provide reports that measure the value that reviewers add to 
the project. In addition, contributors should be able to mine the data for 
their own historical performance (which should allow them to create more 
realistic plans).

Specific capabilities
* Add filters (e.g. by company, by person, by other groups of people … on a 
number of axis, such as "submitted by", "reviewed by", ...)
* A subset of the stats we defined in phase 1 will be exposed
* Add custom pre-defined stats, e.g. post ACK-conversations on patches. 
* Add custom filters for 
  - Active reviews (e.g. those with activity in the last 1-2 weeks)
  - Reviews that are almost complete, e.g. 80% acked (the idea is to focus 
review attention onto reviews that are almost done)
  - Old reviews that may be abandoned (for housekeeping)
  - Some filter to select "complex patches" - the idea here is that these 
require more coordination amongst reviewers
  - Some filter related to contributions which are waiting for comments (aka, 
those which may be stuck)
* Add custom reports, e.g. a report that highlights review contributions more 
accurately 

If you have any extra ideas, let me know

Best regards
Lars
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCHv2 2/3] mm: don't free pages until mm locks are released

2015-12-02 Thread Tim Deegan
At 07:25 + on 02 Dec (1449041100), Tian, Kevin wrote:
> > From: David Vrabel [mailto:david.vra...@citrix.com]
> > Sent: Saturday, November 14, 2015 2:50 AM
> > 
> > If a page is freed without translations being invalidated, and the page is
> > subsequently allocated to another domain, a guest with a cached
> > translation will still be able to access the page.
> > 
> > Currently translations are invalidated before releasing the page ref, but
> > while still holding the mm locks.  To allow translations to be invalidated
> > without holding the mm locks, we need to keep a reference to the page
> > for a bit longer in some cases.
> > 
> > [ This seems difficult to a) verify as correct; and b) difficult to get
> > correct in the future.  A better suggestion would be useful.  Perhaps
> > using something like pg->tlbflush_needed mechanism that already exists
> > for pages from PV guests? ]
> 
> Per-page flag looks clean in general, but not an expert here. Tim might
> have a better idea.

I think you can probably use the tlbflush_timestamp stuff as-is for
EPT flushes -- the existing TLB shootdowns already drop all EPT
translations.

Holding the ref until the flush finishes is a common enough idiom but
I can see that having to track lists of them does make it a little clunky. 

Cheers,

Tim.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCHv2 2/3] mm: don't free pages until mm locks are released

2015-12-02 Thread George Dunlap
On 02/12/15 16:23, Tim Deegan wrote:
> At 07:25 + on 02 Dec (1449041100), Tian, Kevin wrote:
>>> From: David Vrabel [mailto:david.vra...@citrix.com]
>>> Sent: Saturday, November 14, 2015 2:50 AM
>>>
>>> If a page is freed without translations being invalidated, and the page is
>>> subsequently allocated to another domain, a guest with a cached
>>> translation will still be able to access the page.
>>>
>>> Currently translations are invalidated before releasing the page ref, but
>>> while still holding the mm locks.  To allow translations to be invalidated
>>> without holding the mm locks, we need to keep a reference to the page
>>> for a bit longer in some cases.
>>>
>>> [ This seems difficult to a) verify as correct; and b) difficult to get
>>> correct in the future.  A better suggestion would be useful.  Perhaps
>>> using something like pg->tlbflush_needed mechanism that already exists
>>> for pages from PV guests? ]
>>
>> Per-page flag looks clean in general, but not an expert here. Tim might
>> have a better idea.
> 
> I think you can probably use the tlbflush_timestamp stuff as-is for
> EPT flushes -- the existing TLB shootdowns already drop all EPT
> translations.

Are you saying that if you do a TLB shootdown you don't need to do an
invept command?  That doesn't sound right.

 -George


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


[Xen-devel] [xen-unstable-smoke test] 65307: tolerable all pass - PUSHED

2015-12-02 Thread osstest service owner
flight 65307 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/65307/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  2c4f313a7e62c7e559a469d4af4c3d03c49afa43
baseline version:
 xen  c8eb0ec277ae387e78d685523e0fee633e46f046

Last test of basis65279  2015-12-01 13:01:14 Z1 days
Testing same since65307  2015-12-02 15:00:48 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  Boris Ostrovsky 
  David Vrabel 
  Doug Goldstein 
  Jan Beulich 
  Kai Huang 
  Kevin Tian 
  Robert Hu 
  Xudong Hao 

jobs:
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=xen-unstable-smoke
+ revision=2c4f313a7e62c7e559a469d4af4c3d03c49afa43
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x '!=' x/home/osstest/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/home/osstest/repos/lock
++ exec with-lock-ex -w /home/osstest/repos/lock ./ap-push xen-unstable-smoke 
2c4f313a7e62c7e559a469d4af4c3d03c49afa43
+ branch=xen-unstable-smoke
+ revision=2c4f313a7e62c7e559a469d4af4c3d03c49afa43
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x/home/osstest/repos/lock '!=' x/home/osstest/repos/lock ']'
+ . ./cri-common
++ . ./cri-getconfig
++ umask 002
+ select_xenbranch
+ case "$branch" in
+ tree=xen
+ xenbranch=xen-unstable-smoke
+ qemuubranch=qemu-upstream-unstable
+ '[' xxen = xlinux ']'
+ linuxbranch=
+ '[' xqemu-upstream-unstable = x ']'
+ select_prevxenbranch
++ ./cri-getprevxenbranch xen-unstable-smoke
+ prevxenbranch=xen-unstable
+ '[' x2c4f313a7e62c7e559a469d4af4c3d03c49afa43 = x ']'
+ : tested/2.6.39.x
+ . ./ap-common
++ : osst...@xenbits.xen.org
+++ getconfig OsstestUpstream
+++ perl -e '
use Osstest;
readglobalconfig();
print $c{"OsstestUpstream"} or die $!;
'
++ :
++ : git://xenbits.xen.org/xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/xen.git
++ : git://xenbits.xen.org/qemu-xen-traditional.git
++ : git://git.kernel.org
++ : git://git.kernel.org/pub/scm/linux/kernel/git
++ : git
++ : git://xenbits.xen.org/libvirt.git
++ : osst...@xenbits.xen.org:/home/xen/git/libvirt.git
++ : git://xenbits.xen.org/libvirt.git
++ : git://xenbits.xen.org/rumpuser-xen.git
++ : git
++ : git://xenbits.xen.org/rumpuser-xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/rumpuser-xen.git
+++ besteffort_repo https://github.com/rumpkernel/rumpkernel-netbsd-src
+++ local repo=https://github.com/rumpkernel/rumpkernel-netbsd-src
+++ cached_repo https://github.com/rumpkernel/rumpkernel-netbsd-src 
'[fetch=try]'
+++ local repo=https://github.com/rumpkernel/rumpkernel-netbsd-src
+++ local 'options=[fetch=try]'
 getconfig GitCacheProxy
 perl -e '
use Osstest;
readglobalconfig();
print $c{"GitCacheProxy"} or die $!;

Re: [Xen-devel] [PATCH OSSTEST] cr-try-bisect-adhoc: Set laundered_testid so graph URL is correct

2015-12-02 Thread Ian Jackson
Ian Campbell writes ("[PATCH OSSTEST] cr-try-bisect-adhoc: Set laundered_testid 
so graph URL is correct"):
> Otherwise the testid is missing from the filename, resulting in e.g.
> http://osstest.test-lab.xenproject.org/~osstest/pub/results-adhoc/bisect/xen-unstable/test-amd64-amd64-qemuu-nested-intel..svg

Acked-by: Ian Jackson 

Looks plausible.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH OSSTEST] cr-try-bisect-adhoc: Ensure tmp exists.

2015-12-02 Thread Ian Jackson
Ian Campbell writes ("[PATCH OSSTEST] cr-try-bisect-adhoc: Ensure tmp exists."):
> Signed-off-by: Ian Campbell 

Acked-by: Ian Jackson 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCHv2 2/3] mm: don't free pages until mm locks are released

2015-12-02 Thread Jan Beulich
>>> On 02.12.15 at 17:30,  wrote:
> On 02/12/15 16:23, Tim Deegan wrote:
>> At 07:25 + on 02 Dec (1449041100), Tian, Kevin wrote:
 From: David Vrabel [mailto:david.vra...@citrix.com]
 Sent: Saturday, November 14, 2015 2:50 AM

 If a page is freed without translations being invalidated, and the page is
 subsequently allocated to another domain, a guest with a cached
 translation will still be able to access the page.

 Currently translations are invalidated before releasing the page ref, but
 while still holding the mm locks.  To allow translations to be invalidated
 without holding the mm locks, we need to keep a reference to the page
 for a bit longer in some cases.

 [ This seems difficult to a) verify as correct; and b) difficult to get
 correct in the future.  A better suggestion would be useful.  Perhaps
 using something like pg->tlbflush_needed mechanism that already exists
 for pages from PV guests? ]
>>>
>>> Per-page flag looks clean in general, but not an expert here. Tim might
>>> have a better idea.
>> 
>> I think you can probably use the tlbflush_timestamp stuff as-is for
>> EPT flushes -- the existing TLB shootdowns already drop all EPT
>> translations.
> 
> Are you saying that if you do a TLB shootdown you don't need to do an
> invept command?  That doesn't sound right.

No. flush_area_local() (i.e. the actual worker function of TLB
shootdown) calls hvm_flush_guest_tlbs() (in turn leading to
hvm_asid_flush_core()).

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCHv2 2/3] mm: don't free pages until mm locks are released

2015-12-02 Thread David Vrabel
On 02/12/15 16:23, Tim Deegan wrote:
> At 07:25 + on 02 Dec (1449041100), Tian, Kevin wrote:
>>> From: David Vrabel [mailto:david.vra...@citrix.com]
>>> Sent: Saturday, November 14, 2015 2:50 AM
>>>
>>> If a page is freed without translations being invalidated, and the page is
>>> subsequently allocated to another domain, a guest with a cached
>>> translation will still be able to access the page.
>>>
>>> Currently translations are invalidated before releasing the page ref, but
>>> while still holding the mm locks.  To allow translations to be invalidated
>>> without holding the mm locks, we need to keep a reference to the page
>>> for a bit longer in some cases.
>>>
>>> [ This seems difficult to a) verify as correct; and b) difficult to get
>>> correct in the future.  A better suggestion would be useful.  Perhaps
>>> using something like pg->tlbflush_needed mechanism that already exists
>>> for pages from PV guests? ]
>>
>> Per-page flag looks clean in general, but not an expert here. Tim might
>> have a better idea.
> 
> I think you can probably use the tlbflush_timestamp stuff as-is for
> EPT flushes -- the existing TLB shootdowns already drop all EPT
> translations.

Regular TLB invalidate instructions (e.g., INVLPG) only invalidate
linear and combined mappings.  guest-physical mappings are not affected
and require an INVEPT.

Section 2.3.3.2 of the Intel SDM vol 3 gives a useful summary.

David

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCHv2 2/3] mm: don't free pages until mm locks are released

2015-12-02 Thread Tim Deegan
At 16:30 + on 02 Dec (1449073841), George Dunlap wrote:
> On 02/12/15 16:23, Tim Deegan wrote:
> > At 07:25 + on 02 Dec (1449041100), Tian, Kevin wrote:
> >>> From: David Vrabel [mailto:david.vra...@citrix.com]
> >>> Sent: Saturday, November 14, 2015 2:50 AM
> >>>
> >>> If a page is freed without translations being invalidated, and the page is
> >>> subsequently allocated to another domain, a guest with a cached
> >>> translation will still be able to access the page.
> >>>
> >>> Currently translations are invalidated before releasing the page ref, but
> >>> while still holding the mm locks.  To allow translations to be invalidated
> >>> without holding the mm locks, we need to keep a reference to the page
> >>> for a bit longer in some cases.
> >>>
> >>> [ This seems difficult to a) verify as correct; and b) difficult to get
> >>> correct in the future.  A better suggestion would be useful.  Perhaps
> >>> using something like pg->tlbflush_needed mechanism that already exists
> >>> for pages from PV guests? ]
> >>
> >> Per-page flag looks clean in general, but not an expert here. Tim might
> >> have a better idea.
> > 
> > I think you can probably use the tlbflush_timestamp stuff as-is for
> > EPT flushes -- the existing TLB shootdowns already drop all EPT
> > translations.
> 
> Are you saying that if you do a TLB shootdown you don't need to do an
> invept command?

Yes, I think so.  flush_area_local() -> hvm_flush_guest_tlbs() ->
hvm_asid_flush_core() should DTRT.

Tim.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] Log Messages ....

2015-12-02 Thread kumara rathnavel
Hello George,

Thanks a lot. One more question if  I connect serial port will I be able to
see what is happening in xenwithout using xen trace 
On 2 Dec 2015 19:34, "George Dunlap"  wrote:

> On Wed, Dec 2, 2015 at 1:49 PM, kumara rathnavel 
> wrote:
> > Hello All,
> >
> > I require logs of the Hypercalls made from Guest to the Hypervisor after
> the
> > boot, as xl dmesg provides me with the log messages during the
> > initialisation.
>
> One thing you could look at is xentrace and xenalyze.  You can find a
> brief introduction here:
>
> https://blog.xenproject.org/2012/09/27/tracing-with-xentrace-and-xenalyze/
>
> You can enable Xen tracing on the Xen command-line using the
> "tbuf_size" and "tevt_mask" parameters described here:
>
> http://xenbits.xenproject.org/docs/unstable/misc/xen-command-line.html
>
>  -George
>
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Re: [Xen-devel] [PATCH v2] libxl: Introduce a template for devices with a controller

2015-12-02 Thread Olaf Hering
On Tue, Dec 01, George Dunlap wrote:

> We have several outstanding patch series which add devices that have
> two levels: a controller and individual devices attached to that
> controller.

Will likely work for pvscsi. Thanks.

Acked-by: Olaf Hering 

Olaf

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


  1   2   >