Re: [Xen-devel] Questions about the new usb hotplug code in libxl and about adding hotplug (with qmp) usbredir tcp channels

[Fabio Fantoni]

Il 28/04/2016 18:33, Martin Cerveny ha scritto:



On Thu, 28 Apr 2016, Fabio Fantoni wrote:


Il 27/04/2016 20:03, Martin Cerveny ha scritto:



On Wed, 27 Apr 2016, Fabio Fantoni wrote:


Il 27/04/2016 13:26, George Dunlap ha scritto:

On 27/04/16 12:02, Fabio Fantoni wrote:
Hi, I took a look at the new pvusb hotplug code in libxl to try 
to add

also hotplug (with qmp) usbredir tcp channels.
Adding usbredir tcp channels at domU start requires for example 
adding

qemu parameters like these: "-chardev
socket,id=charredir4,host=192.168.1.35,port=4 -device
usb-redir,chardev=charredir4,id=redir4".
It is possible to hotplug it with qmp using "chardev-add" and
"device_add" commands.
Looking at old George Dunlap's patches I tested years ago
(http://xenbits.xen.org/gitweb/?p=people/gdunlap/xen.git;a=commitdiff;h=f7a77843e3fcf070c72115be8ed349a3bfe34e60) 
I can understand what they do and I can add similar qmp functions 
for

usbredir tcp too.
But now I see that bigger and different usb hotplug code was 
added, I

looked at these patches:
http://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=bf7628f087b212052a0e9f024044b2790c33f820 

http://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=043910384cb9ea2c781a7dceac238e110a559c10 

and the full current code in xen's staging branch but I didn't 
find qmp

commands for the qemu usb passthrough, I suppose it is missing or
incomplete (though strange), am I wrong?
If that is correct, pvusb drivers are needed for both host and 
domU to
have usb passthrough working but in new windows pv drivers, the 
pvusb

one is missing, so without the "qemu emulated" usb passthrough it
doesn't work at all in similar cases, right?

That's right -- the PVUSB code *only* does PVUSB passthrough. The
interface was designed, however, to be able to add emulated USB on 
top.



How do you think I should proceed to implement hotplug usbredir tcp
channels in libxl?

So I'm not familiar with usbredir tcp channels.  I'm guessing that it
allows you to transmit the USB commands "over the wire", so that you
could connect (say) a keyboard or printer on your local computer 
to the
qemu process running in a remote dom0, and have the USB device 
presented

as an emulated device to the guest?


Yes.
In qemu upstream there are 2 methods of usbredir use, one is 
"dynamic" using spice. I made a quick and easy implementation in 
libxl some time ago at domU's create:
http://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=f5414ee57a17500e650ea11766474b11da940da2 
this also requires an emulated usb<1|2|3> controller, which I added 
with this:
http://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=bba6747189fb9c4cb51f3c99977d224615106c59 
It works fine with both windows and linux domUs and I already use 
it in production but it works ONLY using spice.




Not right. The usbredir is _separated_ protocol from spice 
(http://www.spice-space.org/page/UsbRedir).
I know, I spoked about what is actually implemented in libxl and 
below in my old mail there was already details about the other method.
What I asked are advices on how is good implement the other method 
(upstream qemu's usbredir tcp channels) in libxl (about the other 
code in addition to the qmp commands functions itself that I'm 
writing now in libxl_qmp.c), 2 days ago I only did some fast "manual 
external" tests with Sid and W7 domUs to be sure is it working 
hotplug (with qmp) before start to implement it in libxl.

Sorry for my bad english if it is not understandable what I write.
I am using this protocol successfully with raspberry-pi "remote usb 
server":

- local usb  enumerator and filter
- remote qmp sender to qemu () (chardev-add/device_add) to
  program "tcp listen" port on qemu
  eg. "remotely" do full automation/filtering (hotplug) - and 
usbredir server () connected to prepared port qemu


(this is part of my VDI solution 
https://gridforums.nvidia.com/default/topic/752/grid-vgpu-benchmarks/vdi-click-to-photon-with-raspberry-pi/)


I will be very pleased if paravirtualized-usb channel will be created
with usb redir protocol in mind. There is problems in qemu 
"emulation" speed - when usb is remotely attached qemu takes about 
20% cpu (probably problem of active "pooling" usb devices DomU/Dom0) 
without any traffic on tcp channel.


Have you tried to use an usb3 emulated controller? with the usb3 if I 
remember good this problem is mitigate.
If you add usb controller at domU's create with usb=1 (in domU's xl 
cfg) it add the qemu default than with the old chipset (the only 
currently used by xen) is usb1 controller, with the q35 (not 
supported by xen) the default is usb2 controller instead. Can you use 
usbversion=3 (added in xen 4.4) for add usb3 controller at domU's xl 
create or add nec-usb-xhci device (hotplug with "manual" qmp

Re: [Xen-devel] Questions about the new usb hotplug code in libxl and about adding hotplug (with qmp) usbredir tcp channels

[Fabio Fantoni]

Il 27/04/2016 20:03, Martin Cerveny ha scritto:



On Wed, 27 Apr 2016, Fabio Fantoni wrote:


Il 27/04/2016 13:26, George Dunlap ha scritto:

On 27/04/16 12:02, Fabio Fantoni wrote:

Hi, I took a look at the new pvusb hotplug code in libxl to try to add
also hotplug (with qmp) usbredir tcp channels.
Adding usbredir tcp channels at domU start requires for example adding
qemu parameters like these: "-chardev
socket,id=charredir4,host=192.168.1.35,port=4 -device
usb-redir,chardev=charredir4,id=redir4".
It is possible to hotplug it with qmp using "chardev-add" and
"device_add" commands.
Looking at old George Dunlap's patches I tested years ago
(http://xenbits.xen.org/gitweb/?p=people/gdunlap/xen.git;a=commitdiff;h=f7a77843e3fcf070c72115be8ed349a3bfe34e60) 


I can understand what they do and I can add similar qmp functions for
usbredir tcp too.
But now I see that bigger and different usb hotplug code was added, I
looked at these patches:
http://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=bf7628f087b212052a0e9f024044b2790c33f820 



http://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=043910384cb9ea2c781a7dceac238e110a559c10 



and the full current code in xen's staging branch but I didn't find 
qmp

commands for the qemu usb passthrough, I suppose it is missing or
incomplete (though strange), am I wrong?
If that is correct, pvusb drivers are needed for both host and domU to
have usb passthrough working but in new windows pv drivers, the pvusb
one is missing, so without the "qemu emulated" usb passthrough it
doesn't work at all in similar cases, right?

That's right -- the PVUSB code *only* does PVUSB passthrough. The
interface was designed, however, to be able to add emulated USB on top.


How do you think I should proceed to implement hotplug usbredir tcp
channels in libxl?

So I'm not familiar with usbredir tcp channels.  I'm guessing that it
allows you to transmit the USB commands "over the wire", so that you
could connect (say) a keyboard or printer on your local computer to the
qemu process running in a remote dom0, and have the USB device 
presented

as an emulated device to the guest?


Yes.
In qemu upstream there are 2 methods of usbredir use, one is 
"dynamic" using spice. I made a quick and easy implementation in 
libxl some time ago at domU's create:
http://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=f5414ee57a17500e650ea11766474b11da940da2 

this also requires an emulated usb<1|2|3> controller, which I added 
with this:
http://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=bba6747189fb9c4cb51f3c99977d224615106c59 

It works fine with both windows and linux domUs and I already use it 
in production but it works ONLY using spice.




Not right. The usbredir is _separated_ protocol from spice 
(http://www.spice-space.org/page/UsbRedir).
I know, I spoked about what is actually implemented in libxl and below 
in my old mail there was already details about the other method.
What I asked are advices on how is good implement the other method 
(upstream qemu's usbredir tcp channels) in libxl (about the other code 
in addition to the qmp commands functions itself that I'm writing now in 
libxl_qmp.c), 2 days ago I only did some fast "manual external" tests 
with Sid and W7 domUs to be sure is it working hotplug (with qmp) before 
start to implement it in libxl.

Sorry for my bad english if it is not understandable what I write.
I am using this protocol successfully with raspberry-pi "remote usb 
server":

- local usb  enumerator and filter
- remote qmp sender to qemu () (chardev-add/device_add) to
  program "tcp listen" port on qemu
  eg. "remotely" do full automation/filtering (hotplug) - and usbredir 
server () connected to prepared port qemu


(this is part of my VDI solution 
https://gridforums.nvidia.com/default/topic/752/grid-vgpu-benchmarks/vdi-click-to-photon-with-raspberry-pi/)


I will be very pleased if paravirtualized-usb channel will be created
with usb redir protocol in mind. There is problems in qemu "emulation" 
speed - when usb is remotely attached qemu takes about 20% cpu 
(probably problem of active "pooling" usb devices DomU/Dom0) without 
any traffic on tcp channel.


Have you tried to use an usb3 emulated controller? with the usb3 if I 
remember good this problem is mitigate.
If you add usb controller at domU's create with usb=1 (in domU's xl cfg) 
it add the qemu default than with the old chipset (the only currently 
used by xen) is usb1 controller, with the q35 (not supported by xen) the 
default is usb2 controller instead. Can you use usbversion=3 (added in 
xen 4.4) for add usb3 controller at domU's xl create or add nec-usb-xhci 
device (hotplug with "manual" qmp command).





Thanks, Martin Cerveny


I want to add the other method to make passthrough of client

Re: [Xen-devel] Questions about the new usb hotplug code in libxl and about adding hotplug (with qmp) usbredir tcp channels

[Fabio Fantoni]

Il 27/04/2016 13:26, George Dunlap ha scritto:

On 27/04/16 12:02, Fabio Fantoni wrote:

Hi, I took a look at the new pvusb hotplug code in libxl to try to add
also hotplug (with qmp) usbredir tcp channels.
Adding usbredir tcp channels at domU start requires for example adding
qemu parameters like these: "-chardev
socket,id=charredir4,host=192.168.1.35,port=4 -device
usb-redir,chardev=charredir4,id=redir4".
It is possible to hotplug it with qmp using "chardev-add" and
"device_add" commands.
Looking at old George Dunlap's patches I tested years ago
(http://xenbits.xen.org/gitweb/?p=people/gdunlap/xen.git;a=commitdiff;h=f7a77843e3fcf070c72115be8ed349a3bfe34e60)
I can understand what they do and I can add similar qmp functions for
usbredir tcp too.
But now I see that bigger and different usb hotplug code was added, I
looked at these patches:
http://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=bf7628f087b212052a0e9f024044b2790c33f820

http://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=043910384cb9ea2c781a7dceac238e110a559c10

and the full current code in xen's staging branch but I didn't find qmp
commands for the qemu usb passthrough, I suppose it is missing or
incomplete (though strange), am I wrong?
If that is correct, pvusb drivers are needed for both host and domU to
have usb passthrough working but in new windows pv drivers, the pvusb
one is missing, so without the "qemu emulated" usb passthrough it
doesn't work at all in similar cases, right?

That's right -- the PVUSB code *only* does PVUSB passthrough.  The
interface was designed, however, to be able to add emulated USB on top.


How do you think I should proceed to implement hotplug usbredir tcp
channels in libxl?

So I'm not familiar with usbredir tcp channels.  I'm guessing that it
allows you to transmit the USB commands "over the wire", so that you
could connect (say) a keyboard or printer on your local computer to the
qemu process running in a remote dom0, and have the USB device presented
as an emulated device to the guest?


Yes.
In qemu upstream there are 2 methods of usbredir use, one is "dynamic" 
using spice. I made a quick and easy implementation in libxl some time 
ago at domU's create:

http://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=f5414ee57a17500e650ea11766474b11da940da2
this also requires an emulated usb<1|2|3> controller, which I added with 
this:

http://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=bba6747189fb9c4cb51f3c99977d224615106c59
It works fine with both windows and linux domUs and I already use it in 
production but it works ONLY using spice.
I want to add the other method to make passthrough of client usb devices 
possible without spice too.
Adding it at domU's create is quick and easy, I could do something like 
the first patch posted above which use the usb controller provided by 
the other patch but unfortunately the addition of usbredir tcp channels 
is not "dynamic" as with spice, but it needs a specific target (ip and 
port) running usbredirserver, so usbredir tcp will be nearly always used 
"hotplug" with the domU running.
Hotplug can be done with qmp using "chardev-add" and "device_add" 
commands, and similar commands for remove. Keeping track of current 
usbredir channels can be done with the xenstore, like pvusb does, I suppose.
I think that making it hotplug should be done with care and if possible 
without making it incompatible with others usb passthrough methods.

So I wonder how it should be done in practice.



If so, then:
1. It may be possible using Juergen's qusb work to act as a usbredir tcp
channel <-> pvusb bridge
2. But having the devices presented to Windows guests without pvusbfront
drivers would depend on having emulated USB working first.
Here I don't understand exactly what you mean, usbredir tcp channels are 
created by upstream qemu and use an emulated usb controller that should 
be already present or added before. I don't know how to use a pvusb 
instead and I suppose it will require specific changes to be done in qemu.


I'm working up a draft implementation of the emulated USB just so I can
make sure the current interface is suitable to extend it before it
becomes public (and thus needs to be supported).
The current pvusb code added recently is more bigger/complex and I have 
difficulties to understand it by following all the relevant code paths. 
I'm trying to understand if a usbredir tcp channels hotplug 
implementation can be based on it. I suppose it will be possible after 
adding the full emulated usb passthrough case.

Any advice is appreciated.



I'm not sure how the usbredir tcp channel interacts with emulated
controllers; that's what will determine what the libxl interface will
look like.

Peace,
  -George



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] Questions about the new usb hotplug code in libxl and about adding hotplug (with qmp) usbredir tcp channels

[Fabio Fantoni]

Hi, I took a look at the new pvusb hotplug code in libxl to try to add 
also hotplug (with qmp) usbredir tcp channels.
Adding usbredir tcp channels at domU start requires for example adding 
qemu parameters like these: "-chardev 
socket,id=charredir4,host=192.168.1.35,port=4 -device 
usb-redir,chardev=charredir4,id=redir4".
It is possible to hotplug it with qmp using "chardev-add" and 
"device_add" commands.
Looking at old George Dunlap's patches I tested years ago 
(http://xenbits.xen.org/gitweb/?p=people/gdunlap/xen.git;a=commitdiff;h=f7a77843e3fcf070c72115be8ed349a3bfe34e60) 
I can understand what they do and I can add similar qmp functions for 
usbredir tcp too.
But now I see that bigger and different usb hotplug code was added, I 
looked at these patches:

http://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=bf7628f087b212052a0e9f024044b2790c33f820
http://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=043910384cb9ea2c781a7dceac238e110a559c10
and the full current code in xen's staging branch but I didn't find qmp 
commands for the qemu usb passthrough, I suppose it is missing or 
incomplete (though strange), am I wrong?
If that is correct, pvusb drivers are needed for both host and domU to 
have usb passthrough working but in new windows pv drivers, the pvusb 
one is missing, so without the "qemu emulated" usb passthrough it 
doesn't work at all in similar cases, right?


How do you think I should proceed to implement hotplug usbredir tcp 
channels in libxl?


Thanks for any reply and my bad English.



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] libxl: small indentation fix in libxl_types.idl

[Fabio Fantoni]

Il 24/02/2016 12:51, Wei Liu ha scritto:

On Fri, Feb 19, 2016 at 04:22:28PM +0100, Fabio Fantoni wrote:

Signed-off-by: Fabio Fantoni 

Acked-by: Wei Liu 


ping




---
  tools/libxl/libxl_types.idl | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index 9ad7eba..8eb24e1 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -457,7 +457,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
  ("ioports",  Array(libxl_ioport_range, "num_ioports")),
  ("irqs", Array(uint32, "num_irqs")),
  ("iomem",Array(libxl_iomem_range, "num_iomem")),
-("claim_mode",libxl_defbool),
+("claim_mode",   libxl_defbool),
  ("event_channels",   uint32),
  ("kernel",   string),
  ("cmdline",  string),
--
1.9.1




___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] List of projects for 4.7

[Fabio Fantoni]

Il 18/03/2016 13:07, Wei Liu ha scritto:

Hi all

Today is that last posting day for new features. And we are two weeks
away from the anticipated freeze date.

I've gone through the outstanding patch series on the list and ask for
input from various core community members. I've enumerated a list
here, which covers several areas of this release and can be used as a
guideline.

Important and close patch, new features:
1. xSplice
2. CPUID levelling
3. ARM ACPI
4. COLO HA
5. RTDS per-vcpu parameter setting
6. Event driven RTDS


Hi, what about "Domain snapshot API"? From a fast search I not found 
recent news about it.


Thanks for any reply and sorry for my bad english.



Important bug fixes:
1. Intel VT-d flush issue
2. SMEP / SMAP fix
3. QEMU hotplug script fix

Note, it is possible for bug fixes to go in after freeze.

Nice to have, not very complicated patch sets:
1. IOREQ server P2M type
2. QEMU-based PVUSB backend
3. Oxenstored improvement
4. Load bios via toolstack

I think it would be good to use the remaining time to focus on things
that are close, important or both.

Thanks
Wei.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Xen domUs seem unable to use one qxl memory bar

[Fabio Fantoni]

Il 24/02/2016 16:44, Fabio Fantoni ha scritto:
Today I was trying a newer version of qxl driver on windows 10 domU 
(on dom0 with xen 4.6) and I got a windows blue screen about the 
updated driver.

On latest xl dmesg line I found this error which I suppose is related:
(XEN) memory.c:161:d0v0 Could not allocate order=18 extent: id=53 
memflags=0 (0 of 1)

Checking the few commits added in the latest update I found this one:
https://cgit.freedesktop.org/~teuf/qxl-wddm-dod/commit/?id=5df8c4f318d808a8381a948da3ce15b5d4aa6682 

Here it starts to use the second qxl memory bar which was unused 
before in windows qxl drivers FWIK.
This additional memory bar is not present in stdvga and if I remember 
correctly when I started to try qxl vga some years ago I was not able 
to boot the domUs at all for one or more problem about memory, solved 
by Jan Beulich and/or Anthony Perard.
After such fixes qxl was usable, now I use it even in production 
windows domUs with monitor resolution up to 1920x1080.


Can someone tell me if the second qxl memory bar is really "unusable" 
or something, please?

If yes how can it be solved?


Ping...can someone help me please?
I did other test in a testing system with xen 4.6 from git (based on 
commit 842e19d951c04c99c27a0fa2bca3d1e677a3) and qemu from qemu-xen 
master (based on commit 316a862e5534249a6e6d876b4e203342d3fb870e).
In this case gave me different xl dmesg output (I suppose for debug 
feature of new win pv driver), log in attachments.


If you need more information/tests tell me and I'll post them.

Thanks for any reply and sorry for my bad english.

48D4830245C8B48
(d6) XEN|BUGCHECK: - Parameter[4282] = 894838246C8B482E
(d6) XEN|BUGCHECK: - Parameter[4283] = 8B480200D7B4
(d6) XEN|BUGCHECK: - Parameter[4284] = 08D7848948402474
(d6) XEN|BUGCHECK: - Parameter[4285] = 5F20C4834802
(d6) XEN|BUGCHECK: - Parameter[4286] = 245C8948CCC3
(d6) XEN|BUGCHECK: - Parameter[4287] = 8B4830EC83485708
(d6) XEN|BUGCHECK: - Parameter[4288] = 48DA8B480A8B48C1
(d6) XEN|BUGCHECK: - Parameter[4289] = 010842C60675C985
(d6) XEN|BUGCHECK: - Parameter[4290] = 8D4C000842C66BEB
(d6) XEN|BUGCHECK: - Parameter[4291] = 0001BA502444
(d6) XEN|BUGCHECK: - Parameter[4292] = C085009890FF
(d6) XEN|BUGCHECK: - Parameter[4293] = 485024448B485578
(d6) XEN|BUGCHECK: - Parameter[4294] = 482024448D4C0B8B
(d6) XEN|BUGCHECK: - Parameter[4295] = 850850FF5824548D
(d6) XEN|BUGCHECK: - Parameter[4296] = 448B48FF333C78C0
(d6) XEN|BUGCHECK: - Parameter[4297] = 4858244C8B482024
(d6) XEN|BUGCHECK: - Parameter[4298] = 448D4C0048246483
(d6) XEN|BUGCHECK: - Parameter[4299] = 3D0850FFD78B4824
(d6) XEN|BUGCHECK: - Parameter[4300] = C0850C74C01E0339
(d6) XEN|BUGCHECK: - Parameter[4301] = 0148247C83481578
(d6) XEN|BUGCHECK: - Parameter[4302] = 7201FF83C7FF0B77
(d6) XEN|BUGCHECK: - Parameter[4303] = 48C033010843C6CC
(d6) XEN|BUGCHECK: - Parameter[4304] = 30C4834840245C8B
(d6) XEN|BUGCHECK: - Parameter[4305] = 72013A83C35F
(d6) XEN|BUGCHECK: - Parameter[4306] = 83C3C01E0304B806
(d6) XEN|BUGCHECK: - Parameter[4307] = 0305B8067201047A
(d6) XEN|BUGCHECK: - Parameter[4308] = 000150BA83C3C01E
(d6) XEN|BUGCHECK: - Parameter[4309] = 1E0348B806740100
(d6) XEN|BUGCHECK: - Parameter[4310] = 00FFB90C428BC3C0
(d6) XEN|BUGCHECK: - Parameter[4311] = 3B0A7602F883
(d6) XEN|BUGCHECK: - Parameter[4312] = C01E0306B80674C1
(d6) XEN|BUGCHECK: - Parameter[4313] = 7602F88314428BC3
(d6) XEN|BUGCHECK: - Parameter[4314] = 2C428BEE75C13B04
(d6) XEN|BUGCHECK: - Parameter[4315] = 74C0850A7403F883
(d6) XEN|BUGCHECK: - Parameter[4316] = 33C3C01E0310B806
(d6) XEN|BUGCHECK: - Parameter[4317] = 01047A83C3C0
(d6) XEN|BUGCHECK: - Parameter[4318] = C3C01E0310B80674
(d6) XEN|BUGCHECK: - Parameter[4319] = 047403F88320428B
(d6) XEN|BUGCHECK: - Parameter[4320] = 01247A83EE75C085
(d6) XEN|BUGCHECK: - Parameter[4321] = 15E8831C428BE875
(d6) XEN|BUGCHECK: - Parameter[4322] = 1E031025C01BD8F7
(d6) XEN|BUGCHECK: - Parameter[4323] = 48C48B48C3C0
(d6) XEN|BUGCHECK: - Parameter[4324] = 4810688948085889
(d6) XEN|BUGCHECK: - Parameter[4325] = 4120788948187089
(d6) XEN|BUGCHECK: - Parameter[4326] = FA8B4860EC834856
(d6) XEN|BUGCHECK: - Parameter[4327] = 107F83D98B48D233
(d6) XEN|BUGCHECK: - Parameter[4328] = C00DB80A7304
(d6) XEN|BUGCHECK: - Parameter[4329] = 0001BD0121E9
(d6) XEN|BUGCHECK: - Parameter[4330] = 0160A939
(d6) XEN|BUGCHECK: - Parameter[4331] = 078B010E8F0F
(d6) XEN|BUGCHECK: - Parameter[4332] = 8BC90348800C8D48
(d6) XEN|BUGCHECK: - Parameter[4333] = 844001A0CB84
(d6) XEN|BUGCHECK: - Parameter[4334] = A800F5850FC5
(d6) XEN|BUGCHECK: - Parameter[4335] = 4800E9840F04
(d6) XEN|BUGCHECK: - Parameter[4336] = 8B01A8CB9489
(d6) XEN|BUGCHECK: - Parameter[4337] = C90348800C8D4807
(d6) XEN|BUGCHECK: - Parameter[4338] = 01B0CB948948
(d6) XEN|BUGCHECK: - Parameter[4339] = 1074C5844018478B
(d6) XEN|BUGCHECK: - Parameter[4340] = 8D4805C08348078B
(d6) XEN|BUGCHECK: - Parameter[4341] = C32C8

Re: [Xen-devel] [PATCH v2 3/3] xl: new "loglvl" command

[Fabio Fantoni]

Il 04/03/2016 17:48, Jan Beulich ha scritto:

This is pretty simplistic for now, but I'd rather have someone better
friends with the tools improve it (if desired).

Signed-off-by: Jan Beulich 

--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
@@ -5958,6 +5958,26 @@ int libxl_send_debug_keys(libxl_ctx *ctx
  return 0;
  }
  
+int libxl_log_level(libxl_ctx *ctx, bool set, bool guest,

+int *lower_thresh, int *upper_thresh)
+{
+int ret;
+GC_INIT(ctx);
+if (set) {
+ret = xc_set_log_level(ctx->xch, guest, *lower_thresh, *upper_thresh);
+} else {
+ret = xc_get_log_level(ctx->xch, guest, lower_thresh, upper_thresh);
+}
+if ( ret < 0 ) {
+LOGE(ERROR, "%s %slog level",
+ set ? "setting" : "getting", guest ? "guest " : "");
+GC_FREE;
+return ERROR_FAIL;
+}
+GC_FREE;
+return 0;
+}
+
  libxl_xen_console_reader *
  libxl_xen_console_read_start(libxl_ctx *ctx, int clear)
  {
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -1765,6 +1765,8 @@ int libxl_send_trigger(libxl_ctx *ctx, u
 libxl_trigger trigger, uint32_t vcpuid);
  int libxl_send_sysrq(libxl_ctx *ctx, uint32_t domid, char sysrq);
  int libxl_send_debug_keys(libxl_ctx *ctx, char *keys);
+int libxl_log_level(libxl_ctx *ctx, bool set, bool guest,
+int *lower_thresh, int *upper_thresh);
  
  typedef struct libxl__xen_console_reader libxl_xen_console_reader;
  
--- a/tools/libxl/xl.h

+++ b/tools/libxl/xl.h
@@ -81,6 +81,7 @@ int main_trigger(int argc, char **argv);
  int main_sysrq(int argc, char **argv);
  int main_debug_keys(int argc, char **argv);
  int main_dmesg(int argc, char **argv);
+int main_loglvl(int argc, char **argv);
  int main_top(int argc, char **argv);
  int main_networkattach(int argc, char **argv);
  int main_networklist(int argc, char **argv);
@@ -209,6 +210,8 @@ extern void printf_info_sexp(int domid,
  #define XL_GLOBAL_CONFIG XEN_CONFIG_DIR "/xl.conf"
  #define XL_LOCK_FILE XEN_LOCK_DIR "/xl"
  
+#define ARRAY_SIZE(a) (sizeof(a) / sizeof(a[0]))

+
  #endif /* XL_H */
  
  /*

--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -6469,6 +6469,84 @@ int main_debug_keys(int argc, char **arg
  return 0;
  }
  
+static const struct {

+int level;
+char string[8];
+} loglvls[] = {
+{ 0, "none" },
+{ 1, "error" },
+{ 2, "warning" },
+{ 3, "info" },
+{ 4, "all" },
+{ 4, "debug" },
+};


double "4" for both all and debug seems strange to me, is it right?


+
+static int parse_loglvl(char **parg)
+{
+unsigned int i;
+
+for (i = 0; i < ARRAY_SIZE(loglvls); ++i) {
+size_t l = strlen(loglvls[i].string);
+
+if (!strncmp(*parg, loglvls[i].string, l)) {
+*parg += l;
+return loglvls[i].level;
+}
+}
+
+return -1;
+}
+
+static const char *format_loglvl(int loglvl)
+{
+unsigned int i;
+
+for (i = 0; i < ARRAY_SIZE(loglvls); ++i) {
+if (loglvl == loglvls[i].level)
+return loglvls[i].string;
+}
+
+return "";
+}
+
+int main_loglvl(int argc, char **argv)
+{
+static const struct option opts[] = {
+{"guest", 0, 0, 'g'},
+{"set", 0, 0, 's'},
+COMMON_LONG_OPTS
+};
+int opt, lower_thresh = -1, upper_thresh = -1;
+bool guest = false, set = false;
+
+SWITCH_FOREACH_OPT(opt, "gs:", opts, "loglvl", 0) {
+case 'g':
+guest = true;
+break;
+case 's':
+if (*optarg != '/')
+lower_thresh = parse_loglvl(&optarg);
+if (*optarg == '/') {
+++optarg;
+upper_thresh = parse_loglvl(&optarg);
+}
+set = true;
+break;
+}
+
+if (libxl_log_level(ctx, set, guest, &lower_thresh, &upper_thresh)) {
+fprintf(stderr, "cannot %s %s log level\n",
+set ? "set" : "get", guest ? "guest" : "host");
+return 1;
+}
+
+if (!set)
+printf("%s log levels: %s/%s\n", guest ? "guest" : "host",
+   format_loglvl(lower_thresh), format_loglvl(upper_thresh));
+
+return 0;
+}
+
  int main_dmesg(int argc, char **argv)
  {
  unsigned int clear = 0;
--- a/tools/libxl/xl_cmdtable.c
+++ b/tools/libxl/xl_cmdtable.c
@@ -309,6 +309,13 @@ struct cmd_spec cmd_table[] = {
"[-c]",
"  -cClear dmesg buffer as well as printing it",
  },
+{ "loglvl",
+  &main_loglvl, 0, 1,
+  "Manage Xen log levels",
+  "[-g] [-s=[LOWER][/UPPER]]",
+  "-g, --guest act on guest log level\n"
+  "-s [LOWER][/UPPER], --set=[LOWER][/UPPER]   set new log level\n"
+},
  { "top",
&main_top, 0, 0,
"Monitor a host and the domains in real time",






___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] Xen domUs seem unable to use one qxl memory bar

[Fabio Fantoni]

Today I was trying a newer version of qxl driver on windows 10 domU (on 
dom0 with xen 4.6) and I got a windows blue screen about the updated driver.

On latest xl dmesg line I found this error which I suppose is related:
(XEN) memory.c:161:d0v0 Could not allocate order=18 extent: id=53 
memflags=0 (0 of 1)

Checking the few commits added in the latest update I found this one:
https://cgit.freedesktop.org/~teuf/qxl-wddm-dod/commit/?id=5df8c4f318d808a8381a948da3ce15b5d4aa6682
Here it starts to use the second qxl memory bar which was unused before 
in windows qxl drivers FWIK.
This additional memory bar is not present in stdvga and if I remember 
correctly when I started to try qxl vga some years ago I was not able to 
boot the domUs at all for one or more problem about memory, solved by 
Jan Beulich and/or Anthony Perard.
After such fixes qxl was usable, now I use it even in production windows 
domUs with monitor resolution up to 1920x1080.


Can someone tell me if the second qxl memory bar is really "unusable" or 
something, please?

If yes how can it be solved?

If you need more information/tests tell me and I'll post them.

Thanks for any reply and sorry for my bad english.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] libxl: add basic spice support for pv domUs

[Fabio Fantoni]

Il 16/02/2016 18:44, Wei Liu ha scritto:

On Wed, Jan 13, 2016 at 11:47:55AM +0100, Fabio Fantoni wrote:
[...]

diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
index 3b695bd..04a96ba 100644
--- a/docs/man/xl.cfg.pod.5
+++ b/docs/man/xl.cfg.pod.5
@@ -1607,82 +1607,6 @@ it.
  =back
-=head3 Spice Graphics Support
-
-The following options control the features of SPICE.
-
-=over 4
-
-=item B
-

To be honest I'm a bit confused by this large hunk. Did you notice some
sort of misplacement of this section? Or is it your editor is using
different wrapping setting?

Now is inside hvm specific section but if basic support for the pv domUs
will be added I think should be moved out of hvm section. Or I'm wrong?


Yes, you're right. Could you say so in the commit message?

yes, I'll do it



  =head2 Keymaps
  The keymaps available are defined by the device-model which you are
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index 6b73848..a5cbcfc 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -510,6 +510,19 @@ typedef struct libxl__ctx libxl_ctx;
  #define LIBXL_HAVE_BUILDINFO_USBVERSION 1
  /*
+ * LIBXL_HAVE_BUILDINFO_SPICE
+ *
+ * If this is defined, then the libxl_domain_build_info structure will
+ * contain spice, a libxl_spice_info struct instead of older hvm.spice one
+ * which is now deprecated.
+ *

This reads like hvm.spice is removed but I think hvm.spice still exists.

Yes I keeped hvm.spice (as "duplicate") for compatibility as suggested by
someone long time ago.


[...]

   * If this is defined, libxl_device_* structures containing a backend_domid
diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
index 8770486..a1853dc 100644
--- a/tools/libxl/libxl_create.c
+++ b/tools/libxl/libxl_create.c
@@ -195,6 +195,19 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,
  if (!b_info->event_channels)
  b_info->event_channels = 1023;
+/* If older u.hvm.spice is enabled then propagate it to the top level */
+libxl_defbool_setdefault(&b_info->u.hvm.spice.enable, false);

This is wrong -- don't set u.hvm without checking the guest is actually
hvm guest.

If I remember good without setting always setdefault I saw some cases of xl
create segfault.
I didn't remember exactly cases, was long time ago.


I was not asking you to not set this. Presumably you still need to set
this, just that you can't set it unconditionally. You need to check if
this guest is really a HVM guest.


If I remember good some version ago I did it hvm specific but xl create 
of pv domUs fails for this default not setted





+libxl_defbool_setdefault(&b_info->spice.enable, false);
+if (!libxl_defbool_val(b_info->spice.enable) &&
+libxl_defbool_val(b_info->u.hvm.spice.enable)) {
+b_info->spice = b_info->u.hvm.spice;

Please use libxl_spice_info_copy (an autogenerated function) to do the
copying.  Doing a shallow copy like this is prone to error -- consider
in the future your structure contains pointers to allocated memory,
doing shallow copy will cause double free.

Thank for spotted it.
Can you explain exactly how to do it? I did a fast search but I'm probably
not understanding correctly.

libxl_spice_info_copy(ctx, &b_info->spice, b_info->u.hvm.spice);

Should the code be as the line above?
I also not understand if I must also an _init before the _copy like this:

libxl_spice_info_init(&b_info->spice);

You don't need to call _init because it should already be initialised
at this point.

And yes, you should just use libxl_spice_info_copy to replace the
direct assignment.


So replace "b_info->spice = b_info->u.hvm.spice;" with 
"libxl_spice_info_copy(ctx, &b_info->spice, b_info->u.hvm.spice);" is 
correct?





+}
+
+libxl_defbool_setdefault(&b_info->spice.disable_ticketing, false);
+libxl_defbool_setdefault(&b_info->spice.agent_mouse, true);
+libxl_defbool_setdefault(&b_info->spice.vdagent, false);
+libxl_defbool_setdefault(&b_info->spice.clipboard_sharing, false);
+
  switch (b_info->type) {
  case LIBXL_DOMAIN_TYPE_HVM:
  if (b_info->shadow_memkb == LIBXL_MEMKB_DEFAULT)
@@ -291,18 +304,17 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,
  libxl_defbool_setdefault(&b_info->u.hvm.usb,false);
  libxl_defbool_setdefault(&b_info->u.hvm.xen_platform_pci,   true);
-libxl_defbool_setdefault(&b_info->u.hvm.spice.enable, false);
-if (!libxl_defbool_val(b_info->u.hvm.spice.enable) &&
-(b_info->u.hvm.spice.usbredirection > 0) ){
-b_info->u.hvm.spice.usbredirection = 0;
-LOG(WARN, "spice disabled, disabling usbredirection");
+if (!libxl_defbool_val(b_info->spice.enable) &&a

[Xen-devel] [PATCH] libxl: small indentation fix in libxl_types.idl

[Fabio Fantoni]

Signed-off-by: Fabio Fantoni 
---
 tools/libxl/libxl_types.idl | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index 9ad7eba..8eb24e1 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -457,7 +457,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
 ("ioports",  Array(libxl_ioport_range, "num_ioports")),
 ("irqs", Array(uint32, "num_irqs")),
 ("iomem",Array(libxl_iomem_range, "num_iomem")),
-("claim_mode",  libxl_defbool),
+("claim_mode",   libxl_defbool),
 ("event_channels",   uint32),
 ("kernel",   string),
 ("cmdline",  string),
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Developers for virgl 3d windows guest support

[Fabio Fantoni]

Il 27/01/2016 19:37, Konrad Rzeszutek Wilk ha scritto:

On Tue, Jan 26, 2016 at 05:48:34PM +0100, Fabio Fantoni wrote:

I take a fast look to virgl 3d project even if I not tested it for now. It
seems interesting for adding 2d and 3d hw acceleration support to virtual
machines with a large hw (gpu) choice.
I take a look also to intel gvt-g but it has a very limited cpu support
choice.
I saw that windows guest support with direct3d drivers in virgl is planned
but I nothing was done for now.
I'm approaching a new project (based on xen) and I'm in planning phase now.
My goal would be to have virgl 3d windows guest support (drivers with opengl
and direct3d) and support of remote rendering with spice (we are already
using with qxl for now)

Did you look at the VirtIO drivers and their roadmap?


About virtio-gpu using virgl3d project for 3d hw acceleration support 
and is what I mainly watching for its large gpu choice/support, seems 
any gpu that have a drm driver in host kernel is supported by virgl, or 
I'm wrong?
I saw that they don't have short term plan about windows support: 
https://github.com/YanVugenfirer/kvm-guest-drivers-windows/issues/50


If you mean instead intel igvt-g and seems also nvidia with recent posts 
I saw, seems a good project but has very limited hardware choice.
Intel cpus with integrated gpu that support igvt-g don't seems powerful 
enough for major of servers I need to build (about xeon only workstation 
series seems support it).
About nvidia only grid card seems support it (high cost useful only for 
a minimal user target with intensive 3d usage FWIK)


I'm in evaulating/exploring which project/technology is more suitable 
for my goal. My goal is to run at least 10 windows Desktop vms (office 
and schools use target) on a xen host with 2d/3d hw acceleration support.


I'm wondering what solution is more long-term reliable and approceable 
to have such 2d/3d support and FWIK ity seems virgl is the way to go but 
i really need some advice from some expert out there in order to make 
the right choice.






Are there any developers expert about these things that can advice me if
developing windows drivers (WDDM kernel driver + GL and D3D userspace
pieces) and any other needed pieces of code in virglrender, mesa, and /or
qemu is a good choice?

I have no clue about Windows. But when I started working on Linux it took
me good three months to get comfortable. I would presume the same thing
is for Windows. Albeit you would need also to have understanding of
OpenGL.


If yes, what we would like to know is approx time/effort to have the above
support upstream. Approximately how many persons and how many time would
require to complete the tasks? (my project is going to be financed so we
eventually have resource to schedule working time in the community)

Um, .. Are you saying you need the person/month values so that you can
get the money? Or that you have the money and just need to figure out
if it can be done with the amount you have?


I'm trying to understand if there are some persons able to start and 
close a payed project in order to have such 2d/3d support developed on 
both windows drivers side and whatever virtualGPU solution actually 
available (virgl or other that I'm not aware of).




Thanks for any reply and sorry for my bad english.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] Developers for virgl 3d windows guest support

[Fabio Fantoni]

I take a fast look to virgl 3d project even if I not tested it for now. 
It seems interesting for adding 2d and 3d hw acceleration support to 
virtual machines with a large hw (gpu) choice.
I take a look also to intel gvt-g but it has a very limited cpu support 
choice.
I saw that windows guest support with direct3d drivers in virgl is 
planned but I nothing was done for now.

I'm approaching a new project (based on xen) and I'm in planning phase now.
My goal would be to have virgl 3d windows guest support (drivers with 
opengl and direct3d) and support of remote rendering with spice (we are 
already using with qxl for now)
Are there any developers expert about these things that can advice me if 
developing windows drivers (WDDM kernel driver + GL and D3D userspace 
pieces) and any other needed pieces of code in virglrender, mesa, and 
/or qemu is a good choice?
If yes, what we would like to know is approx time/effort to have the 
above support upstream. Approximately how many persons and how many time 
would require to complete the tasks? (my project is going to be financed 
so we eventually have resource to schedule working time in the community)


Thanks for any reply and sorry for my bad english.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] libxl: add basic spice support for pv domUs

[Fabio Fantoni]

Il 11/01/2016 16:00, Wei Liu ha scritto:

On Tue, Dec 01, 2015 at 05:04:35PM +0100, Fabio Fantoni wrote:

This patch adds basic spice support for pv domUs.
The qemu parameters are the same as the hvm ones and they works.
Therefore xl cfg parameters are the same as the hvm ones except that
features not supported yet by pv domUs (vdagent and usbredirection)
are kept disabled by default.
It also enables vfb and vkb required to have basic spice working.

Signed-off-by: Fabio Fantoni 

---

Notes:
- The vfb part is only a draft and needs to be improved.
- Patch is tested and working, except for the pointer not
visible in some cases with pv domUs but always working.
- I not use the api, test the u.hvm.spice retro-compatibility with
api is needed.

Any feedback is appreciated.

Changes in v8:
- refresh all for xen 4.7

Changes in v7:
- refresh xl_sxp.c

Changes in v6:
- refresh libxl_create.c

Changes in v5:
- libxl_create.c: * don't copy u.hvm.spice in the newer if
the newer is already used
* set default for all spice bool options in any case
* spice features not supported in pv will be disabled and
will show a warning about them if was setted enabled
- xl_cmdimpl.c: parse all spice options out of hvm part
- libxl_dm.c: changed some forgotten u.hvm.spice to spice

Changes in v4:
- added libxl.h changes
- libxl_create.c: added older u.hvm.spice compatibility
copying it in newer one

Changes in v3:
- xl.cfg.pod.5: moved spice out of hvm section and specified
the features for now hvm only.
- libxl_types.idl: added spice struct out of keyedunion hvm only.
- use new generic spice struct instead of hvm only ones.

Changes in v2:
- xl_cmdimpl.c: always set vnc and sdl toplevel parameters in &vfb
with vnc or spice enabled on pv domUs otherwise in some cases it
would fail with error for one bool default value missing.
- libxl_dm.c: do not add -nographic if spice is enabled, even though
-nographic seems buggy in upstream qemu.
---
  docs/man/xl.cfg.pod.5   | 155 ++--
  tools/libxl/libxl.h |  13 
  tools/libxl/libxl_create.c  |  49 +-
  tools/libxl/libxl_dm.c  |  39 ++-
  tools/libxl/libxl_types.idl |   3 +-
  tools/libxl/xl_cmdimpl.c|  51 ---
  tools/libxl/xl_sxp.c|  12 ++--
  7 files changed, 179 insertions(+), 143 deletions(-)

diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
index 3b695bd..04a96ba 100644
--- a/docs/man/xl.cfg.pod.5
+++ b/docs/man/xl.cfg.pod.5
@@ -1607,82 +1607,6 @@ it.
  
  =back
  
-=head3 Spice Graphics Support

-
-The following options control the features of SPICE.
-
-=over 4
-
-=item B
-

To be honest I'm a bit confused by this large hunk. Did you notice some
sort of misplacement of this section? Or is it your editor is using
different wrapping setting?


Now is inside hvm specific section but if basic support for the pv domUs 
will be added I think should be moved out of hvm section. Or I'm wrong?





  =head2 Keymaps
  
  The keymaps available are defined by the device-model which you are

diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index 6b73848..a5cbcfc 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -510,6 +510,19 @@ typedef struct libxl__ctx libxl_ctx;
  #define LIBXL_HAVE_BUILDINFO_USBVERSION 1
  
  /*

+ * LIBXL_HAVE_BUILDINFO_SPICE
+ *
+ * If this is defined, then the libxl_domain_build_info structure will
+ * contain spice, a libxl_spice_info struct instead of older hvm.spice one
+ * which is now deprecated.
+ *

This reads like hvm.spice is removed but I think hvm.spice still exists.


Yes I keeped hvm.spice (as "duplicate") for compatibility as suggested 
by someone long time ago.





+ * If it is set, callers may use spice to specify the spice values.
+ *

May use which one? hvm.spice or the new spice (libxl_spice_info).


This is needed to know if the new one is present and use it, also this 
added after advice of someone, I think should be useful for people that 
use libxl "without xl", like libvirt.
More exactly libxl_spice_info was already present, is the "spice" new 
instead "hvm.spice"

tools/libxl/libxl_types.idl
+("spice",libxl_spice_info),
I should rewrite the description to be more comprensible? If yes what 
exactly I must write?

+ * If this is defined, then the libxl_domain_build_info structure will
+ * contain spice, a libxl_spice_info struct instead of older hvm.spice one
+ * which is now deprecated.
This above (already present) seems comprensible to me but probably my 
english is too bad.






+ * If this is not defined, the spice struct does not exist.
+ */
+#define LIBXL_HAVE_BUILDINFO_SPICE 1
+
+/*
   * LIBXL_HAVE_DEVICE_BACKEND_DOMNAME
   *
   * If this is defined, libxl_device_* structures containing a backend_domid
diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
index 8770486..a1853dc 100644
--- a/tools/libxl/lib

Re: [Xen-devel] VirtIO-GPU 3D OpenGL Hardware Acceleration for VMs and Xen

[Fabio Fantoni]

Il 11/01/2016 10:47, Pasi Kärkkäinen ha scritto:

Hello,

And now all the components listed below are released and available easily.

Dom0:
- Qemu 2.5 has virtio-gpu 3D/OpenGL acceleration support for VMs.

DomU:
- Linux 4.4 kernel has the virtio-gpu drm driver.
- Mesa 11.1 has the virtio-gpu 3D driver (virgl galliumd3 / OpenGL).


Did anyone try this stuff with Xen already? :)


I'm also interested on it but I not tried it for now because it is 
usable only locally (with qemu's gtk or sdl2 ui if I remember good), I'm 
waiting spice support (in development).
I'll should be able to do needed libxl support if nobody will do it 
before but I'm not able to do hypervisor changes if needed (like initial 
problems with qxl on xen)





-- Pasi


On Mon, Nov 16, 2015 at 10:52:41PM +0200, Pasi Kärkkäinen wrote:

Hello,

It seems upstream Linux/Gallium3D/Mesa/Qemu/KVM has recently gained virtualized 
support for 3D/OpenGL hardware acceleration in VMs, allowing using the GPU of 
the host in VMs.

Components:
- Linux 4.4 kernel includes the DRM driver for VirtIO-GPU 3D 
acceleration (needed in the VM).
- Qemu 2.5 (rc0) includes the VirtIO-GPU 3D mode support (needed on the 
host/dom0).
- Gallium3D VirGL driver is included in Mesa git (needed in the VM, 
supports up to OpenGL 3.3 atm).
- On the host/dom0 one needs *any* OpenGL driver (for the host GPU 
obviously), no special requirements there, if I understood correctly.


Has someone looked into this already? How much work would it be to get 
VirtIO-GPU working in Xen HVM guests?


Thanks,

-- Pasi



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [PATCH] libxl: add basic spice support for pv domUs

[Fabio Fantoni]

This patch adds basic spice support for pv domUs.
The qemu parameters are the same as the hvm ones and they works.
Therefore xl cfg parameters are the same as the hvm ones except that
features not supported yet by pv domUs (vdagent and usbredirection)
are kept disabled by default.
It also enables vfb and vkb required to have basic spice working.

Signed-off-by: Fabio Fantoni 

---

Notes:
- The vfb part is only a draft and needs to be improved.
- Patch is tested and working, except for the pointer not
visible in some cases with pv domUs but always working.
- I not use the api, test the u.hvm.spice retro-compatibility with
api is needed.

Any feedback is appreciated.

Changes in v8:
- refresh all for xen 4.7

Changes in v7:
- refresh xl_sxp.c

Changes in v6:
- refresh libxl_create.c

Changes in v5:
- libxl_create.c: * don't copy u.hvm.spice in the newer if
the newer is already used
* set default for all spice bool options in any case
* spice features not supported in pv will be disabled and
will show a warning about them if was setted enabled
- xl_cmdimpl.c: parse all spice options out of hvm part
- libxl_dm.c: changed some forgotten u.hvm.spice to spice

Changes in v4:
- added libxl.h changes
- libxl_create.c: added older u.hvm.spice compatibility
copying it in newer one

Changes in v3:
- xl.cfg.pod.5: moved spice out of hvm section and specified
the features for now hvm only.
- libxl_types.idl: added spice struct out of keyedunion hvm only.
- use new generic spice struct instead of hvm only ones.

Changes in v2:
- xl_cmdimpl.c: always set vnc and sdl toplevel parameters in &vfb
with vnc or spice enabled on pv domUs otherwise in some cases it
would fail with error for one bool default value missing.
- libxl_dm.c: do not add -nographic if spice is enabled, even though
-nographic seems buggy in upstream qemu.
---
 docs/man/xl.cfg.pod.5   | 155 ++--
 tools/libxl/libxl.h |  13 
 tools/libxl/libxl_create.c  |  49 +-
 tools/libxl/libxl_dm.c  |  39 ++-
 tools/libxl/libxl_types.idl |   3 +-
 tools/libxl/xl_cmdimpl.c|  51 ---
 tools/libxl/xl_sxp.c|  12 ++--
 7 files changed, 179 insertions(+), 143 deletions(-)

diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
index 3b695bd..04a96ba 100644
--- a/docs/man/xl.cfg.pod.5
+++ b/docs/man/xl.cfg.pod.5
@@ -1607,82 +1607,6 @@ it.
 
 =back
 
-=head3 Spice Graphics Support
-
-The following options control the features of SPICE.
-
-=over 4
-
-=item B
-
-Allow access to the display via the SPICE protocol.  This enables the
-other SPICE-related settings.
-
-=item B
-
-Specify the interface address to listen on if given, otherwise any
-interface.
-
-=item B
-
-Specify the port to listen on by the SPICE server if the SPICE is
-enabled.
-
-=item B
-
-Specify the secure port to listen on by the SPICE server if the SPICE
-is enabled. At least one of the spiceport or spicetls_port must be
-given if SPICE is enabled.  NB. the options depending on spicetls_port
-have not been supported.
-
-=item B
-
-Enable client connection without password. When disabled, spicepasswd
-must be set. The default is false (0).
-
-=item B
-
-Specify the ticket password which is used by a client for connection.
-
-=item B
-
-Whether SPICE agent is used for client mouse mode. The default is true (1)
-(turn on)
-
-=item B
-
-Enables spice vdagent. The Spice vdagent is an optional component for
-enhancing user experience and performing guest-oriented management
-tasks. Its features includes: client mouse mode (no need to grab mouse
-by client, no mouse lag), automatic adjustment of screen resolution,
-copy and paste (text and image) between client and domU. It also
-requires vdagent service installed on domU o.s. to work. The default is 0.
-
-=item B
-
-Enables Spice clipboard sharing (copy/paste). It requires spicevdagent
-enabled. The default is false (0).
-
-=item B
-
-Enables spice usbredirection. Creates NUMBER usbredirection channels
-for redirection of up to 4 usb devices from spice client to domU's qemu.
-It requires an usb controller and if not defined it will automatically adds
-an usb2 controller. The default is disabled (0).
-
-=item B
-
-Specifies what image compression is to be used by spice (if given), otherwise
-the qemu default will be used. Please see documentations of your current qemu
-version for details.
-
-=item B
-
-Specifies what streaming video setting is to be used by spice (if given),
-otherwise the qemu default will be used.
-
-=back
-
 =head3 Miscellaneous Emulated Hardware
 
 =over 4
@@ -1838,6 +1762,85 @@ xen-qemudepriv-domid$domid or xen-qemudepriv-shared or 
root.
 
 =back
 
+=head2 Spice Graphics Support
+
+The following options control the features of SPICE.
+
+=over 4
+
+=item B
+
+Allow access to the display via the SPICE protocol.  This enables the
+other SPICE-related settings.
+
+=item B
+
+Specify the interface address to listen on if given,

[Xen-devel] Update upstream qemu to 2.4.1 for xen-unstable

[Fabio Fantoni]

I saw that upstream qemu for xen-unstable is still not updated at least 
to latest stable version.
I tested qemu 2.4.0.1 for some weeks on 2 systems backporting this 2 
patches:
spice: surface switch fast path requires same format too (I think is 
good backport it also on future update in xen as it fix a spice regression)
xen/HVM: atomically access pointers in bufioreq handling (I suppose that 
this will be added in xen git too)


I also recently tested 2.4.1 (also backporting same 2 patches above) 
without finding regression for now.


Thanks for any reply and sorry for my bad english.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Questions for patch "libxl: add basic spice support for pv domUs"

[Fabio Fantoni]

Il 18/11/2015 16:16, Konrad Rzeszutek Wilk ha scritto:

On Wed, Nov 18, 2015 at 12:13:06PM +0100, Fabio Fantoni wrote:

Long time ago, I did a libxl patch for add basic spice support for pv domUs.
I did some improvements based on comments I received, if I remember good I
did all except add of vfb parameters (vfb=['spice=1,...'])
Major of advanced spice features can't be added in pv (also now based on
what I know) like vdagent, usbredir, qxl but probably can be useful add it
to make possible use spice for both pv and hvm domUs.
There is also a minor thing present with vnc but not in spice missed in qemu
and I not understand how to add it that is automatic port selection (like
what do vncunused).
If I'm not wrong latest version (v6) is here:
https://github.com/Fantu/Xen/commit/dd51fe9bddcd8f190f7fd2f5d927c4fd3749b2cb

What do you think about this patch?
I'll probably rebased it to actual xen-unstable for 4.7 and I'm asking if
there are important changes that must be done.

Could you provide an example of how to use the spice? I tried it myself
but ran into trouble of running an 'spice' agent (or whatever it is called).

I fear it is just my ignorance - as in *maybe* I was running QEMU
using spice but it was not clear to me if that was the case. Is there
a particular standalone program to connect to a spice agent? (I tried
virt-viewer using 'spice://' but it complained and wasn't sure whether
that was because it wasn't compiled with that or I was doing something
silly).

Thanks!


Any help is appreciated.
Spice is not enable by default on xen build (but it is enabled on mainly 
distros upstream qemu packages), on my systems I add spice (server) and 
spice-protocol and I enable it in upstream qemu builtin in xen with:

--with-extra-qemuu-configure-args="--enable-spice --enable-usb-redir"

My full xen configure is:
./configure --prefix=/usr --disable-blktap1 --disable-qemu-traditional 
--disable-rombios --with-system-seabios=/usr/share/seabios/bios-256k.bin 
--with-extra-qemuu-configure-args="--enable-spice --enable-usb-redir" 
--disable-blktap2 --enable-ovmf


virt-viewer is the best spice client FWIK using for example 'remote-viewer 
spice://:' virt-viewer must be compiled with spice support (that 
require spice-gtk installed)

I use it full features (with vdagent,usbredir,qxl) in production mainly 
with tens of windows 7 domUs with spice-guest-tools 0.100 (or 0.74 in 
older) on monitor 1920x1080
I use it for both thin client and maintenance. Before I used vnc for 
maintenance and rdp (installed in domU) with proprietary usb redirection 
(that are all bugged). Now with spice is only and better.


About pv domUs I use xen's "vkvm" (vnc or spice) only for maintenance 
and I prefer have spice also for them instead vnc if possible (what I'm 
doing with the patch above).
The perfect would be have full features spice also with pv for good very 
low cost systems and good linux desktop domUs (for example for schools) 
but I fear it will never be possible.


Thanks for any reply and sorry for my bad english.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] Questions for patch "libxl: add basic spice support for pv domUs"

[Fabio Fantoni]

Long time ago, I did a libxl patch for add basic spice support for pv domUs.
I did some improvements based on comments I received, if I remember good 
I did all except add of vfb parameters (vfb=['spice=1,...'])
Major of advanced spice features can't be added in pv (also now based on 
what I know) like vdagent, usbredir, qxl but probably can be useful add 
it to make possible use spice for both pv and hvm domUs.
There is also a minor thing present with vnc but not in spice missed in 
qemu and I not understand how to add it that is automatic port selection 
(like what do vncunused).

If I'm not wrong latest version (v6) is here:
https://github.com/Fantu/Xen/commit/dd51fe9bddcd8f190f7fd2f5d927c4fd3749b2cb

What do you think about this patch?
I'll probably rebased it to actual xen-unstable for 4.7 and I'm asking 
if there are important changes that must be done.


Thanks for any reply and sorry for my bad english.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] Config.mk: update OVMF changeset

[Fabio Fantoni]

Il 23/10/2015 14:56, Ian Campbell ha scritto:

On Fri, 2015-10-23 at 13:43 +0100, Stefano Stabellini wrote:

We have no existing stable baseline for that arch, and no testing or
reason to believe that cb9a7eb (the Config.mk version currently
referenced by 4.6) as being any good at all on that platform,
whether we backport a couple of fixes to it or not.

It is true that ovmf arm64 is not in osstest, but I ran the test
manually and I know that cb9a7eb plus the one backport works, which is
just a build fix. In addition the original work for arm64 support was
done far earlier than cb9a7eb.

20% of the patches since then are ARM related and I'm not sure that a quick
smoke test is a high enough bar to add something in a stable point release
(it might suffice for adding to the development branch and subsequently
releasing, after plenty of time and -rc's, test days etc)


I'm not convinced that taking some arbitrary old (although not as old
as I
thought) OVMF tree which we have tested to our satisfaction and
released on
x86, slapping a couple of arm64 backports on it and saying "this is now
a
good and stable thing to use on arm64" makes it good enough to release
as
ovmf arm64 in 4.6.1, encouraging our users to go about using etc.

Far better to be honest about it for now and point arm64 users at a
more
bleeding edge ovmf release outside of our own stable releases and
prepare
to do something better in 4.7.
  
Are you suggesting we don't create an OVMF branch for 4.6 until the

first backport request comes along which we think is appropriate, then
we decide what to do?  I would rather have an OVMF branch for 4.6 now,
even if it is just cb9a7eb with no backports.

I'm not against having an OVMF branch ready for any potential bug fixes
which might crop up in the feature set we released and in future we should
probably create one as a matter of course as part of branching.

What I don't like is adding OVMF/arm64 as a new feature in a point release
with very little of the usual confidence we would have in something we
would add in a 4.6.0, let alone a 4.6.1.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


A recent ovmf patch (already tested) I think is good to backport is:
0f34a051104e2b1b9123d56d48673de4b21bc533 - OvmfPkg: XenPvBlkDxe: handle 
empty cdrom drives
Can be considered please? Without any domU using ovmf and having empty 
cdrom (required for cd hotplug) freeze on boot start.


There is also another important occasional bug (reported and linked also 
by Stefano Stabellini) but without solution for now where is good 
backport any fixes related when will be done.


Thanks for any reply and sorry for my bad english.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [edk2] [PATCH v2] OvmfPkg: XenPvBlkDxe: handle empty cdrom drives

[Fabio Fantoni]

Il 21/10/2015 14:45, Laszlo Ersek ha scritto:

On 10/21/15 13:39, Stefano Stabellini wrote:

Empty cdroms are not going to connect, avoid waiting for the backend to
switch to state 4, which is never going to happen, and return
error instead from XenPvBlockFrontInitialization(). Detect an
empty cdrom by looking at the "params" node on xenstore, which is set to
"" or "aio:" for empty drives by libxl.

Contributed-under: TianoCore Contribution Agreement 1.0
Signed-off-by: Stefano Stabellini 

---
Changes in v2:
- better commit message
- return EFI_DEVICE_ERROR instead of EFI_NO_MEDIA
- check the return status of XenBusIo->XsBackendRead
---
  OvmfPkg/XenPvBlkDxe/BlockFront.c |   15 +++
  1 file changed, 15 insertions(+)

diff --git a/OvmfPkg/XenPvBlkDxe/BlockFront.c b/OvmfPkg/XenPvBlkDxe/BlockFront.c
index 256ac55..d07e980 100644
--- a/OvmfPkg/XenPvBlkDxe/BlockFront.c
+++ b/OvmfPkg/XenPvBlkDxe/BlockFront.c
@@ -169,6 +169,7 @@ XenPvBlockFrontInitialization (
XEN_BLOCK_FRONT_DEVICE *Dev;
XenbusState State;
UINT64 Value;
+  CHAR8 *Params;
  
ASSERT (NodeName != NULL);
  
@@ -186,6 +187,20 @@ XenPvBlockFrontInitialization (

}
FreePool (DeviceType);
  
+  if (Dev->MediaInfo.CdRom) {

+Status = XenBusIo->XsBackendRead (XenBusIo, XST_NIL, "params", 
(VOID**)&Params);
+if (Status != XENSTORE_STATUS_SUCCESS) {
+  DEBUG ((EFI_D_ERROR, "%a: Failed to read params (%d)\n", __FUNCTION__, 
Status));
+  goto Error;
+}
+if (AsciiStrLen (Params) == 0 || AsciiStrCmp (Params, "aio:") == 0) {
+  FreePool (Params);
+  DEBUG ((EFI_D_INFO, "%a: Empty cdrom\n", __FUNCTION__));
+  goto Error;
+}
+FreePool (Params);
+  }
+
Status = XenBusReadUint64 (XenBusIo, "backend-id", FALSE, &Value);
if (Status != XENSTORE_STATUS_SUCCESS || Value > MAX_UINT16) {
  DEBUG ((EFI_D_ERROR, "XenPvBlk: Failed to get backend-id (%d)\n",


Reviewed-by: Laszlo Ersek 

I test-built this for X64 and Ia32, and then committed it to SVN as rev
18651.

Thanks!
Laszlo


Thanks to both.
I tested it with a git cherry-pick on my tested build of some days ago.
With ide disks it boots correctly and xl cd-insert and xl cd-eject are 
working.
With pvdisk don't start, but this time instead freeze on "tianocore 
logo" but freeze with black screen after initial windows boot.
I retried for take full logs as possibile but it booted correctly and 
with xl cd-insert and xl cd-eject was working.
I retried other 2 reboot without problem but at the third windows freeze 
after arrived to login screen.
I suppose that this patch is ok and the problem is another but I not 
understand exactly what.


In attachment full qemu log and xl dmesg about the latest test where 
windows freezed at login screen, rdp and xl shutdown was not working, 
its qemu process was still active and with near 100% cpu usage.

If you need more informations/tests tell me and I'll post them.

Thanks for any reply and sorry for my bad english.

(d24) HVM Loader
(d24) Detected Xen v4.6.0
(d24) Xenbus rings @0xfeffc000, event channel 1
(d24) System requested OVMF
(d24) CPU speed is 2660 MHz
(d24) Relocating guest memory for lowmem MMIO space disabled
(XEN) irq.c:275: Dom24 PCI link 0 changed 0 -> 5
(d24) PCI-ISA link 0 routed to IRQ5
(XEN) irq.c:275: Dom24 PCI link 1 changed 0 -> 10
(d24) PCI-ISA link 1 routed to IRQ10
(XEN) irq.c:275: Dom24 PCI link 2 changed 0 -> 11
(d24) PCI-ISA link 2 routed to IRQ11
(XEN) irq.c:275: Dom24 PCI link 3 changed 0 -> 5
(d24) PCI-ISA link 3 routed to IRQ5
(d24) pci dev 01:3 INTA->IRQ10
(d24) pci dev 02:0 INTA->IRQ11
(d24) pci dev 03:0 INTA->IRQ5
(d24) pci dev 04:0 INTA->IRQ5
(d24) pci dev 05:0 INTA->IRQ10
(d24) pci dev 06:0 INTA->IRQ11
(d24) pci dev 1d:0 INTA->IRQ10
(d24) pci dev 1d:1 INTB->IRQ11
(d24) pci dev 1d:2 INTC->IRQ5
(d24) pci dev 1d:7 INTD->IRQ5
(d24) RAM in high memory; setting high_mem resource base to 10800
(d24) pci dev 05:0 bar 10 size 00400: 0f000
(d24) pci dev 05:0 bar 14 size 00400: 0f400
(d24) pci dev 02:0 bar 14 size 00100: 0f808
(d24) pci dev 06:0 bar 30 size 4: 0f900
(d24) pci dev 05:0 bar 30 size 1: 0f904
(d24) pci dev 03:0 bar 10 size 04000: 0f905
(d24) pci dev 05:0 bar 18 size 02000: 0f9054000
(d24) pci dev 04:0 bar 14 size 01000: 0f9056000
(d24) pci dev 1d:7 bar 10 size 01000: 0f9057000
(d24) pci dev 02:0 bar 10 size 00100: 0c001
(d24) pci dev 06:0 bar 10 size 00100: 0c101
(d24) pci dev 06:0 bar 14 size 00100: 0f9058000
(d24) pci dev 04:0 bar 10 size 00020: 0c201
(d24) pci dev 05:0 bar 1c size 00020: 0c221
(d24) pci dev 1d:0 bar 20 size 00020: 0c241
(d24) pci dev 1d:1 bar 20 size 00020: 0c261
(d24) pci dev 1d:2 bar 20 size 00020: 0c281
(d24) pci dev 01:1 bar 20 size 00010: 0c2a1
(d24) Multiprocessor initialisation:
(d24)  - CPU0 ... 36-bit phys ... fixed MTRRs ... var MTRRs [1/8] ... done.
(d24)  - CPU1 ... 36-bit phys ... fixed

Re: [Xen-devel] [Qemu-devel] Question about xen disk unplug support for ahci missed in qemu

[Fabio Fantoni]

2015-10-19 18:57 GMT+02:00 Stefano Stabellini <
stefano.stabell...@eu.citrix.com>:

> On Mon, 19 Oct 2015, John Snow wrote:
> > On 10/19/2015 07:44 AM, Stefano Stabellini wrote:
> > > On Mon, 19 Oct 2015, Gerd Hoffmann wrote:
> > >>   Hi,
> > >>
> >  I'm trying to follow this discussion as best as I am able, but my
> lack
> >  of experience with Xen prevents me from really participating in a
> >  meaningful way.
> > 
> >  (I see that Laszlo is still discussing some CD-ROM issues with Fabio
> >  which may be of interest to me...)
> > 
> >  At any rate, I won't be authoring any Xen-specific hacks to the AHCI
> >  device, but I do have plans to implement hot-plugging emulation as
> per
> >  the AHCI spec. Perhaps this is sufficient for the Xen layer, but
> someone
> >  else will need to author the appropriate glue code.
> > 
> >  If "real" hot-plugging is not sufficient, we'll need to discuss
> further,
> >  preferably over some RFC patches.
> > >>>
> > >>> That's fine. AHCI hot-plugging would go a long way and once we have
> > >>> that, the rest is easy.
> > >>
> > >> Can we get some more background on this?
> > >>
> > >> IIRC the IDE bits are needed to boot hvm guests, which goes like this:
> > >>
> > >>   (1) boot disk is hooked up using both xenbus and ide.
> > >>   (2) seabios boots using ide.
> > >>   (3) linux kernel activates xenbus, at which point qemu zaps the ide
> > >>   disks to avoid the disk being present twice in the system.
> > >>
> > >> Correct?
> > >>
> > >> Do we really want repeat this exercise for AHCI?  Alot has changed
> since
> > >> this boot hack for ide was added ...
> > >>
> > >> As far I know OVMF has xenbus drivers, so OVMF should already boot xen
> > >> guests just fine without this, correct?
> > >
> > > I agree with you that the current unplug in nasty. Also I don't care
> > > much about AHCI, in fact I don't think we should be spending efforts
> > > into making that scenario work better. I think we should be working on
> > > OVMF instead and fix the bug about empty cdrom drives reported by
> Fabio.
> > >
> >
> > OVMF and AHCI go hand in hand here from my viewpoint. I'm happy to debug
> > any OVMF+SATA/AHCI problems that are reported.
> >
> > Last I saw, Laszlo asked Fabio for some more information on this
> > problem, so I am waiting for that information to start work on that
> issue.
>
> Fabio reported a bug using OVFM+xen_disk, no AHCI involved. OVFM has
> already support for the Xen PV disk protocol, see
> OvmfPkg/XenPvBlkDxe/XenPvBlkDxe.c.
>

I not tried with ahci in ovmf in latest because as you told that is a
missing unplug case in qemu.

I tried with xendisk and ide, the empty cdrom problem is with both, from xl
domU cfg:

',raw,xvdb,ro,cdrom' for xendisk and ',raw,hdb,ro,cdrom' for ide.

Using seabios instead boot correctly, with ovmf not:
http://lists.xen.org/archives/html/xen-devel/2015-10/msg01833.html

If I remember good also in latest test persist also the problem that
ovmf not respect the boot order parameter.
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [Qemu-devel] Question about xen disk unplug support for ahci missed in qemu

[Fabio Fantoni]

Il 19/10/2015 12:18, Stefano Stabellini ha scritto:

On Fri, 16 Oct 2015, John Snow wrote:

On 10/13/2015 01:10 PM, Stefano Stabellini wrote:

On Tue, 13 Oct 2015, John Snow wrote:

On 10/13/2015 11:55 AM, Fabio Fantoni wrote:

I added ahci disk support in libxl and using it for week seems that was
ok, after a reply of Stefano Stabellini seems that xen disk unplug
support only ide disks:
http://git.qemu.org/?p=qemu.git;a=commitdiff;h=679f4f8b178e7c66fbc2f39c905374ee8663d5d8

Today Paul Durrant told me that even if pv disk is ok also with ahci and
the emulated one is offline can be a risk:
http://lists.xenproject.org/archives/html/win-pv-devel/2015-10/msg00021.html


I tried to take a fast look in qemu code but I not understand the needed
thing for add the xen disk unplug support also for ahci, can someone do
it or tell me useful information for do it please?

Thanks for any reply and sorry for my bad english.


I'm not entirely sure what features you need AHCI to support in order
for Xen to be happy.

I'd guess hotplugging, but where I get confused is that IDE disks don't
support hotplugging either, so I guess I'm not sure sure what you need.

Stefano, can you help bridge my Xen knowledge gap?
  
Hi John,


we need something like hw/i386/xen/xen_platform.c:unplug_disks but that
can unplug AHCI disk. And by unplug, I mean "make disappear" like
pci_piix3_xen_ide_unplug does for ide.


I'm trying to follow this discussion as best as I am able, but my lack
of experience with Xen prevents me from really participating in a
meaningful way.

(I see that Laszlo is still discussing some CD-ROM issues with Fabio
which may be of interest to me...)

At any rate, I won't be authoring any Xen-specific hacks to the AHCI
device, but I do have plans to implement hot-plugging emulation as per
the AHCI spec. Perhaps this is sufficient for the Xen layer, but someone
else will need to author the appropriate glue code.

If "real" hot-plugging is not sufficient, we'll need to discuss further,
preferably over some RFC patches.

That's fine. AHCI hot-plugging would go a long way and once we have
that, the rest is easy.

Thanks,

Stefano


Thanks for any improvement you will do.
About AHCI hotplugging will remove bad thing of actual "xen unplug" in 
seabios/ovmf but not the hybrid pv solution right?
Doing without hybrid like what I tried you told me with ovmf probably is 
better solution after solving other bugs but probably someone don't want 
plain remove it.


About empty cdrom not working in ovmf but needed by xl 
cd-insert/cd-eject I suppose that is easly reproducible, I had it in any 
case I tested.
If not and more details are needed tell me and I'll rebuild and test 
following Lazslo instructions.


Thanks for any reply and sorry for my bad english.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [Qemu-devel] Question about xen disk unplug support for ahci missed in qemu

[Fabio Fantoni]

Il 16/10/2015 18:53, Paul Durrant ha scritto:

-Original Message-
From: Kevin Wolf [mailto:kw...@redhat.com]
Sent: 16 October 2015 17:43
To: Paul Durrant
Cc: Fabio Fantoni; Stefano Stabellini; John Snow; Anthony Perard; qemu-
de...@nongnu.org; xen-devel@lists.xen.org; qemu-bl...@nongnu.org
Subject: Re: [Qemu-devel] Question about xen disk unplug support for ahci
missed in qemu

Am 16.10.2015 um 18:20 hat Paul Durrant geschrieben:

-Original Message-
From: Kevin Wolf [mailto:kw...@redhat.com]
Sent: 16 October 2015 17:12
To: Paul Durrant
Cc: Fabio Fantoni; Stefano Stabellini; John Snow; Anthony Perard; qemu-
de...@nongnu.org; xen-devel@lists.xen.org; qemu-bl...@nongnu.org
Subject: Re: [Qemu-devel] Question about xen disk unplug support for

ahci

missed in qemu

Am 16.10.2015 um 17:10 hat Paul Durrant geschrieben:

-Original Message-
From: Kevin Wolf [mailto:kw...@redhat.com]
Sent: 16 October 2015 16:02
To: Paul Durrant
Cc: Fabio Fantoni; Stefano Stabellini; John Snow; Anthony Perard;

qemu-

de...@nongnu.org; xen-devel@lists.xen.org; qemu-

bl...@nongnu.org

Subject: Re: [Qemu-devel] Question about xen disk unplug support

for

ahci

missed in qemu

Am 16.10.2015 um 16:24 hat Paul Durrant geschrieben:

-Original Message-
From: Kevin Wolf [mailto:kw...@redhat.com]
Sent: 16 October 2015 15:04
To: Paul Durrant
Cc: Fabio Fantoni; Stefano Stabellini; John Snow; Anthony Perard;

qemu-

de...@nongnu.org; xen-devel@lists.xen.org; qemu-

bl...@nongnu.org

Subject: Re: [Qemu-devel] Question about xen disk unplug

support

for

ahci

missed in qemu

Am 14.10.2015 um 14:48 hat Paul Durrant geschrieben:

-Original Message-
From: Fabio Fantoni [mailto:fabio.fant...@m2r.biz]
Sent: 14 October 2015 12:12
To: Kevin Wolf; Stefano Stabellini
Cc: John Snow; Anthony Perard; qemu-de...@nongnu.org;

xen-

de...@lists.xen.org; qemu-bl...@nongnu.org; Paul Durrant
Subject: Re: [Qemu-devel] Question about xen disk unplug

support

for

ahci

missed in qemu



Il 14/10/2015 11:47, Kevin Wolf ha scritto:

[ CC qemu-block ]

Am 13.10.2015 um 19:10 hat Stefano Stabellini geschrieben:

On Tue, 13 Oct 2015, John Snow wrote:

On 10/13/2015 11:55 AM, Fabio Fantoni wrote:

I added ahci disk support in libxl and using it for week

seems

that

was

ok, after a reply of Stefano Stabellini seems that xen disk

unplug

support only ide disks:


http://git.qemu.org/?p=qemu.git;a=commitdiff;h=679f4f8b178e7c66fbc2f39

c905374ee8663d5d8

Today Paul Durrant told me that even if pv disk is ok also

with

ahci

and

the emulated one is offline can be a risk:
http://lists.xenproject.org/archives/html/win-pv-

devel/2015-

10/msg00021.html


I tried to take a fast look in qemu code but I not

understand

the

needed

thing for add the xen disk unplug support also for ahci,

can

someone do

it or tell me useful information for do it please?

Thanks for any reply and sorry for my bad english.


I'm not entirely sure what features you need AHCI to

support

in

order

for Xen to be happy.

I'd guess hotplugging, but where I get confused is that IDE

disks

don't

support hotplugging either, so I guess I'm not sure sure

what

you

need.

Stefano, can you help bridge my Xen knowledge gap?

Hi John,

we need something like

hw/i386/xen/xen_platform.c:unplug_disks

but

that

can unplug AHCI disk. And by unplug, I mean "make

disappear"

like

pci_piix3_xen_ide_unplug does for ide.

Maybe this would be the right time to stop the craziness

with

your

hybrid IDE/xendisk setup. It's a horrible thing that would

never

happen

on real hardware.

Unfortunately, it's going to be difficult to remove such 'craziness'

when

you

don't know a priori whether the VM has PV drivers or not.

Why wouldn't you know that beforehand? I mean, even on real

hardware

you
can have different disk interfaces (IDE, AHCI, SCSI) and you install
the exact driver that your hardware needs. You just do the same

thing on

VM: If your hardware is PV, you install a PV driver. If your

hardware is

IDE, you install an IDE driver. Whether it's PV or IDE is something

that

you, the user, decided when configuring the VM, so you definitely

know.

That's not necessarily true. The host admin that provisions the VM

does

not

necessarily know what OS the user of that VM will install. The admin

may

just

be providing a generic VM with an emulated CD drive that the user

can

point

at any ISO they want.

So, as a host admin, if you provide a VM with only PV backends and

your

user is trying to boot an OS with no PV drivers they are not going to be
happy, so you provide emulated devices. Then, at some point later,

when

the user installs PV drivers, there really should be some way for those

drivers

to start up without any need to contact the host admin and have the

VM

reconfigured.

Why only IDE and xendisk then? Maybe I have an OS that works great

with

AHCI, or virtio-blk, 

Re: [Xen-devel] Regression: qemu crash of hvm domUs with spice (backtrace included) - patch backport propose

[Fabio Fantoni]

Il 09/10/2015 09:56, Fabio Fantoni ha scritto:

Il 08/10/2015 17:58, Andreas Kinzler ha scritto:

Is this still current? I made an interesting observation:

I had no problems with SPICE and vanilla Xen 4.5.1 when using it on 
Gentoo with glibc 2.19/gcc 4.6.4.
Segfaults started when I switched to glibc 2.20/gcc 4.9.3 - I did not 
change Xen source code at all.
All this might be related to: 
https://lists.nongnu.org/archive/html/qemu-devel/2015-06/msg02764.html


Andreas


Thanks for your mail.
The problem I had seems different, I not found the exactly cause but I 
solved using newer qemu (2.2 from unstable - now 4.6) with xen 4.5.1.

Big distro I saw already use newer qemu and should be ok.
I still using glibc < 2.20 in my debian servers, this is probably 
because I not had your problem but I think that backport the patch you 
linked can be useful for solve a qemu crash case, I'll test it in my 
next build and if I'll not found regression I'll require the backport 
for xen qemu gits.





https://github.com/Fantu/Xen/commits/rebase/m2r-staging
Latest test with regression based on latest stable-4.5, more exactly:
https://github.com/Fantu/Xen/commits/rebase/m2r-testing
Some days ago on same dom0 and domU I tried with latest stable 
version (that I use on only 2 production servers for now but I not 
saw the regression), more exactly:

https://github.com/Fantu/Xen/commits/rebase/m2r-stable-4.5
Dom0 debian 7 with kernel 3.16 from backports, seabios 1.8.1-2 from 
unstable and this xen configure:
./configure --prefix=/usr --disable-blktap1 
--disable-qemu-traditional --disable-rombios 
--with-system-seabios=/usr/share/seabios/bios-256k.bin 
--with-extra-qemuu-configure-args="--enable-spice 
--enable-usb-redir" --disable-blktap2


I suppose that there is unexpected case caused by a backports or 
missed patch/es to backports from unstable.
I not found with a fast look rilevant patch to try to revert, can 
anyone suggest me the more probable point/s for bisect and/or patch 
to revert or I must try full bisect 4.5.0->stable-4.5?








I tried to use xen 4.6 with its qemu plus cherry-pick of this patch:
http://git.qemu.org/?p=qemu.git;a=commit;h=c6e484707f28b3e115e64122a0570f6b3c585489 
- spice-display: fix segfault in qemu_spice_create_update

Used some days without see regression.
Probably is useful apply it to qemu-xen unstable and 4.6 (about older 
versions I don't know and I not tested).

@Stefano Stabellini: can you take a look to it please?

About qemu used in unstable I think is good update to 2.4.0.1 now that 
xen 4.7 devel is started, I think we keep upstream qemu updated with too 
many latency and there is a more high bug risk with xen for major of 
distro that use newer qemu version (following stable version of both).


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [Qemu-devel] Question about xen disk unplug support for ahci missed in qemu

[Fabio Fantoni]

Il 16/10/2015 12:47, Stefano Stabellini ha scritto:

On Fri, 16 Oct 2015, Fabio Fantoni wrote:

Il 16/10/2015 12:13, Anthony PERARD ha scritto:

On Fri, Oct 16, 2015 at 10:32:44AM +0200, Fabio Fantoni wrote:

Il 15/10/2015 20:02, Anthony PERARD ha scritto:

On Thu, Oct 15, 2015 at 06:27:17PM +0200, Fabio Fantoni wrote:

Il 14/10/2015 13:06, Stefano Stabellini ha scritto:

I would suggest Fabio to avoid AHCI disks altogether and just use
OVMF
with PV disks only and Anthony's patch to libxl to avoid creating
any
IDE disks: http://marc.info/?l=xen-devel&m=144482080812353.

Would that work for you?

Thanks for the advice, I tried it:
https://github.com/Fantu/Xen/commits/rebase/m2r-testing-4.6

I installed W10 pro 64 bit with ide disk, installed the win pv drivers
and
after changed to xvdX instead hdX, is the only change needed, right?
Initial boot is ok (ovmf part about pv disks seems ok) but windows
boot
fails with problem with pv drivers.
In attachment full qemu log with xen_platform trace and domU's xl cfg.

Someone have windows domUs with ovmf and pv disks only working? If yes
can
tell me the difference to understand what can be the problem please?

When I worked on the PV disk implementation in OVMF, I was able to boot
a Windows 8 with pv disk only.

I don't have access to the guest configuration I was using, but I think
one
difference would be the viridian setting, I'm pretty sure I did not set
it.


I tried with viridian disabled but did the same thing, looking cdrom as
latest thing before xenbug trace in qemu log I tried also to remove it but
also in this case don't boot correctly, full qemu log in attachment.
I don't know if is a ovmf thing to improve (like what seems in Laszlo and
Kevin mails) or xen winpv drivers unexpected case, have you tried also
with
latest winpv builds? (for exclude regression)

No, I did not tried the latest winpv drivers.

Sorry I can help much more that that. When I install this win8 guest tried
to boot it with pv drivers only, that was more than a year ago. I have not
check if it's still working. (Also I can not try anything more recent,
right now.)


I did many other tests, retrying with ide first boot working but show pv
devices not working, I did another reboot (with ide) and pv devices was
working, after I retried with pv (xvdX) and boot correctly.
After other tests I found that with empty cdrom device (required for xl
cd-insert/cd-eject) boot stop at start (tianocore image), same result with ide
instead.
 From xl cfg:
disk=['/mnt/vm/disks/W10UEFI.disk1.cow-sn1,qcow2,xvda,rw',',raw,xvdb,ro,cdrom']
With seabios domU boot also with empty cdrom.
In qemu log I found only these that can be related:

xen be: qdisk-51728: error: Could not open image: No such file or directory
xen be: qdisk-51728: initialise() failed

And latest xl dmesg line is:

(d1) Invoking OVMF ...

If you need more informations/test tell me and I'll post them.

Are you saying that without any cdrom drives, it works correctly?
Yes, I did also another test to be sure, starting with ide, installing 
windows, the pv drivers, rebooting 2 times (with one at boot of time 
boot with ide only and without net and disks pv drivers working) and 
after rebooting with pv disks (xvdX) works.
With cdrom not empty (with iso) works, with empty not, tried with both 
ide (hdX) and pv (xvdX).

Empty cdrom not working with ovmf I suppose is ovmf bug or inexpected case.
About major of winpv drivers problem at boot I suppose can be solved 
improving ovmf and winpv driver removing bad hybrid thing actually, but 
I have too low knowledge to be sure.
About the problem of pv start after install that requiring at least 2 
reboot can be also a windows 10 problem (only a suppose).


About empty cdrom with ovmf can be solved please?

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [Qemu-devel] Question about xen disk unplug support for ahci missed in qemu

[Fabio Fantoni]

Il 16/10/2015 12:13, Anthony PERARD ha scritto:

On Fri, Oct 16, 2015 at 10:32:44AM +0200, Fabio Fantoni wrote:

Il 15/10/2015 20:02, Anthony PERARD ha scritto:

On Thu, Oct 15, 2015 at 06:27:17PM +0200, Fabio Fantoni wrote:

Il 14/10/2015 13:06, Stefano Stabellini ha scritto:

I would suggest Fabio to avoid AHCI disks altogether and just use OVMF
with PV disks only and Anthony's patch to libxl to avoid creating any
IDE disks: http://marc.info/?l=xen-devel&m=144482080812353.

Would that work for you?

Thanks for the advice, I tried it:
https://github.com/Fantu/Xen/commits/rebase/m2r-testing-4.6

I installed W10 pro 64 bit with ide disk, installed the win pv drivers and
after changed to xvdX instead hdX, is the only change needed, right?
Initial boot is ok (ovmf part about pv disks seems ok) but windows boot
fails with problem with pv drivers.
In attachment full qemu log with xen_platform trace and domU's xl cfg.

Someone have windows domUs with ovmf and pv disks only working? If yes can
tell me the difference to understand what can be the problem please?

When I worked on the PV disk implementation in OVMF, I was able to boot
a Windows 8 with pv disk only.

I don't have access to the guest configuration I was using, but I think one
difference would be the viridian setting, I'm pretty sure I did not set it.


I tried with viridian disabled but did the same thing, looking cdrom as
latest thing before xenbug trace in qemu log I tried also to remove it but
also in this case don't boot correctly, full qemu log in attachment.
I don't know if is a ovmf thing to improve (like what seems in Laszlo and
Kevin mails) or xen winpv drivers unexpected case, have you tried also with
latest winpv builds? (for exclude regression)

No, I did not tried the latest winpv drivers.

Sorry I can help much more that that. When I install this win8 guest tried
to boot it with pv drivers only, that was more than a year ago. I have not
check if it's still working. (Also I can not try anything more recent,
right now.)



I did many other tests, retrying with ide first boot working but show pv 
devices not working, I did another reboot (with ide) and pv devices was 
working, after I retried with pv (xvdX) and boot correctly.
After other tests I found that with empty cdrom device (required for xl 
cd-insert/cd-eject) boot stop at start (tianocore image), same result 
with ide instead.
From xl cfg: 
disk=['/mnt/vm/disks/W10UEFI.disk1.cow-sn1,qcow2,xvda,rw',',raw,xvdb,ro,cdrom']

With seabios domU boot also with empty cdrom.
In qemu log I found only these that can be related:
xen be: qdisk-51728: error: Could not open image: No such file or 
directory

xen be: qdisk-51728: initialise() failed

And latest xl dmesg line is:

(d1) Invoking OVMF ...


If you need more informations/test tell me and I'll post them.

Thanks for any reply and sorry for my bad english.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [Qemu-devel] Question about xen disk unplug support for ahci missed in qemu

[Fabio Fantoni]

Il 15/10/2015 20:02, Anthony PERARD ha scritto:

On Thu, Oct 15, 2015 at 06:27:17PM +0200, Fabio Fantoni wrote:

Il 14/10/2015 13:06, Stefano Stabellini ha scritto:

I would suggest Fabio to avoid AHCI disks altogether and just use OVMF
with PV disks only and Anthony's patch to libxl to avoid creating any
IDE disks: http://marc.info/?l=xen-devel&m=144482080812353.

Would that work for you?

Thanks for the advice, I tried it:
https://github.com/Fantu/Xen/commits/rebase/m2r-testing-4.6

I installed W10 pro 64 bit with ide disk, installed the win pv drivers and
after changed to xvdX instead hdX, is the only change needed, right?
Initial boot is ok (ovmf part about pv disks seems ok) but windows boot
fails with problem with pv drivers.
In attachment full qemu log with xen_platform trace and domU's xl cfg.

Someone have windows domUs with ovmf and pv disks only working? If yes can
tell me the difference to understand what can be the problem please?

When I worked on the PV disk implementation in OVMF, I was able to boot
a Windows 8 with pv disk only.

I don't have access to the guest configuration I was using, but I think one
difference would be the viridian setting, I'm pretty sure I did not set it.

I tried with viridian disabled but did the same thing, looking cdrom as 
latest thing before xenbug trace in qemu log I tried also to remove it 
but also in this case don't boot correctly, full qemu log in attachment.
I don't know if is a ovmf thing to improve (like what seems in Laszlo 
and Kevin mails) or xen winpv drivers unexpected case, have you tried 
also with latest winpv builds? (for exclude regression)


Thanks for any reply and sorry for my bad english.
main_channel_link: add main channel client
main_channel_handle_parsed: net test: latency 17.727000 ms, bitrate 578858111 
bps (552.042113 Mbps)
inputs_connect: inputs channel client create
red_dispatcher_set_cursor_peer: 
xen_platform_log xen platform: XEN|DllInitialize: 8.2.0 (80) (17.09.2015)
xen_platform_log xen platform: XEN|AcpiFindRsdp: 0x000EA020
xen_platform_log xen platform: XEN|SystemGetStartOptions:  TESTSIGNING  
NOEXECUTE=OPTIN  NOVGA
xen_platform_log xen platform: XEN|SystemGetVersionInformation: KERNEL: 10.0 
(BUILD 10240) PLATFORM WIN32_NT (x64)
xen_platform_log xen platform: XEN|SystemGetVersionInformation: SUITES:
xen_platform_log xen platform: XEN|SystemGetVersionInformation: - TERMINAL
xen_platform_log xen platform: XEN|SystemGetVersionInformation: - SINGLEUSERTS
xen_platform_log xen platform: XEN|SystemGetVersionInformation: TYPE: 
WORKSTATION
xen_platform_log xen platform: XEN|SystemGetMemoryInformation: RANGE[0] 
.1000 - .0009efff
xen_platform_log xen platform: XEN|SystemGetMemoryInformation: RANGE[1] 
.0010 - .eed94fff
xen_platform_log xen platform: XEN|SystemGetMemoryInformation: RANGE[2] 
.eee12000 - .efe91fff
xen_platform_log xen platform: XEN|SystemGetMemoryInformation: RANGE[3] 
.efef6000 - .effc
xen_platform_log xen platform: XEN|SystemGetMemoryInformation: RANGE[4] 
.efff - .efff
xen_platform_log xen platform: XEN|SystemGetMemoryInformation: RANGE[5] 
0001. - 0001.07eaafff
xen_platform_log xen platform: XEN|AcpiGetXsdt: 0xFC012BA0
xen_platform_log xen platform: XEN|SystemProcessorInformation: > (0:0)
xen_platform_log xen platform: XEN|SystemProcessorInformation: Manufacturer: 
GenuineIntel
xen_platform_log xen platform: XEN|SystemProcessorInformation: APIC ID: 00
xen_platform_log xen platform: XEN|SystemProcessorInformation: PROCESSOR ID: 00
xen_platform_log xen platform: XEN|SystemProcessorInformation: < (0:0)
xen_platform_log xen platform: XEN|SystemProcessorInformation: > (0:1)
xen_platform_log xen platform: XEN|SystemProcessorInformation: Manufacturer: 
GenuineIntel
xen_platform_log xen platform: XEN|SystemProcessorInformation: APIC ID: 02
xen_platform_log xen platform: XEN|SystemProcessorInformation: PROCESSOR ID: 01
xen_platform_log xen platform: XEN|SystemProcessorInformation: < (0:1)
xen_platform_log xen platform: XEN: HYPERCALL PAGE 0 @ 0001.03dd4000
xen_platform_log xen platform: XENFILT|DriverEntry: 8.2.0 (80) (17.09.2015)
xen_platform_log xen platform: XEN: 4.6.0 (__XEN_INTERFACE_VERSION__ = 00040600)
xen_platform_log xen platform: XENFILT|FdoCreate: E0006B7A7C60 
(ACPI\PNP0A03\0)
xen_platform_log xen platform: XENFILT|PdoCreate: E0006B786A60 
(PCI\VEN_8086&DEV_1237&SUBSYS_11001AF4&REV_02\00)
xen_platform_log xen platform: XENFILT|PdoCreate: E0006B7DBC60 
(PCI\VEN_8086&DEV_7000&SUBSYS_11001AF4&REV_00\08)
xen_platform_log xen platform: XENFILT|PdoCreate: E0006B7DB880 
(PCI\VEN_8086&DEV_7010&SUBSYS_11001AF4&REV_00\09)
xen_platform_log xen platform: XENFILT|PdoCreate: E0006B7DCC60 
(PCI\VEN_5853&DEV_0001&SUBSYS_00015853&REV_01\10)
xen_platform_log xe

Re: [Xen-devel] [Qemu-devel] Question about xen disk unplug support for ahci missed in qemu

[Fabio Fantoni]

Il 14/10/2015 13:06, Stefano Stabellini ha scritto:

On Wed, 14 Oct 2015, Kevin Wolf wrote:

[ CC qemu-block ]

Am 13.10.2015 um 19:10 hat Stefano Stabellini geschrieben:

On Tue, 13 Oct 2015, John Snow wrote:

On 10/13/2015 11:55 AM, Fabio Fantoni wrote:

I added ahci disk support in libxl and using it for week seems that was
ok, after a reply of Stefano Stabellini seems that xen disk unplug
support only ide disks:
http://git.qemu.org/?p=qemu.git;a=commitdiff;h=679f4f8b178e7c66fbc2f39c905374ee8663d5d8

Today Paul Durrant told me that even if pv disk is ok also with ahci and
the emulated one is offline can be a risk:
http://lists.xenproject.org/archives/html/win-pv-devel/2015-10/msg00021.html


I tried to take a fast look in qemu code but I not understand the needed
thing for add the xen disk unplug support also for ahci, can someone do
it or tell me useful information for do it please?

Thanks for any reply and sorry for my bad english.


I'm not entirely sure what features you need AHCI to support in order
for Xen to be happy.

I'd guess hotplugging, but where I get confused is that IDE disks don't
support hotplugging either, so I guess I'm not sure sure what you need.

Stefano, can you help bridge my Xen knowledge gap?
  
Hi John,


we need something like hw/i386/xen/xen_platform.c:unplug_disks but that
can unplug AHCI disk. And by unplug, I mean "make disappear" like
pci_piix3_xen_ide_unplug does for ide.

Maybe this would be the right time to stop the craziness with your
hybrid IDE/xendisk setup. It's a horrible thing that would never happen
on real hardware.

I would be quite happy to stop (or even get rid of) the craziness.



Can't you just teach SeaBIOS how to deal with your PV disks and then
only add that to your VM and forget about IDE/AHCI? I mean, that's how
it's done for virtio-blk, and it doesn't involve any insanities like
ripping out non-hotpluggable devices.

Teaching SeaBIOS to deal with PV disks can be done, in fact we already
support PV disks in OVMF. It is possible to boot Windows with OVMF
without any IDE disks (patch pending for libxl to create a VM without
emulated IDE disks).

However we have to be honest that implementing PV disk support in
SeaBIOS is a different magnitude of effort compared to implementing AHCI
"unplug".

I would suggest Fabio to avoid AHCI disks altogether and just use OVMF
with PV disks only and Anthony's patch to libxl to avoid creating any
IDE disks: http://marc.info/?l=xen-devel&m=144482080812353.

Would that work for you?


Thanks for the advice, I tried it:
https://github.com/Fantu/Xen/commits/rebase/m2r-testing-4.6

I installed W10 pro 64 bit with ide disk, installed the win pv drivers 
and after changed to xvdX instead hdX, is the only change needed, right?
Initial boot is ok (ovmf part about pv disks seems ok) but windows boot 
fails with problem with pv drivers.

In attachment full qemu log with xen_platform trace and domU's xl cfg.

Someone have windows domUs with ovmf and pv disks only working? If yes 
can tell me the difference to understand what can be the problem please?






Hm... How does a reboot of a machine that had its IDE already removed
actually work? Do you restart the qemu process on reboot?

Restart QEMU, yes.


main_channel_link: add main channel client
main_channel_handle_parsed: net test: latency 8.258000 ms, bitrate 727789623 
bps (694.074271 Mbps)
inputs_connect: inputs channel client create
red_dispatcher_set_cursor_peer: 
xen_platform_log xen platform: XEN|DllInitialize: 8.2.0 (80) (17.09.2015)
xen_platform_log xen platform: XEN|AcpiFindRsdp: 0x000EA020
xen_platform_log xen platform: XEN|SystemGetStartOptions:  TESTSIGNING  
NOEXECUTE=OPTIN  NOVGA
xen_platform_log xen platform: XEN|SystemGetVersionInformation: KERNEL: 10.0 
(BUILD 10240) PLATFORM WIN32_NT (x64)
xen_platform_log xen platform: XEN|SystemGetVersionInformation: SUITES:
xen_platform_log xen platform: XEN|SystemGetVersionInformation: - TERMINAL
xen_platform_log xen platform: XEN|SystemGetVersionInformation: - SINGLEUSERTS
xen_platform_log xen platform: XEN|SystemGetVersionInformation: TYPE: 
WORKSTATION
xen_platform_log xen platform: XEN|SystemGetMemoryInformation: RANGE[0] 
.1000 - .0009efff
xen_platform_log xen platform: XEN|SystemGetMemoryInformation: RANGE[1] 
.0010 - .eed94fff
xen_platform_log xen platform: XEN|SystemGetMemoryInformation: RANGE[2] 
.eee12000 - .efe91fff
xen_platform_log xen platform: XEN|SystemGetMemoryInformation: RANGE[3] 
.efef6000 - .effc
xen_platform_log xen platform: XEN|SystemGetMemoryInformation: RANGE[4] 
.efff - .efff
xen_platform_log xen platform: XEN|SystemGetMemoryInformation: RANGE[5] 
0001. - 0001.07eaafff
xen_platform_log xen platform: XEN|AcpiGetXsdt: 0xFC012BA0
xen_platform_log xen platform: XEN|SystemProcessorInforma

Re: [Xen-devel] scripts/git-checkout.sh fails in some cases

[Fabio Fantoni]

Il 15/10/2015 16:22, Wei Liu ha scritto:

On Thu, Oct 15, 2015 at 03:40:36PM +0200, Fabio Fantoni wrote:

I already saw scripts/git-checkout.sh fails many times changing repo and/or
tag for my tests, for example today:

GIT=git
/mnt/vm/source/xen/Xen/tools/firmware/../../scripts/git-checkout.sh
git://xenbits.xen.org/osstest/ovmf.git xen-tested-master ovmf-dir
Cloning into 'ovmf-dir-remote.tmp'...
remote: Counting objects: 203149, done.
remote: Compressing objects: 100% (49944/49944), done.
remote: Total 203149 (delta 153028), reused 196420 (delta 146807)
Ricezione degli oggetti: 100% (203149/203149), 194.95 MiB | 527.00 KiB/s,
done.
Risoluzione dei delta: 100% (153028/153028), done.
Checking connectivity... fatto.
warning: l'HEAD remoto si riferisce ad un ref inesistente, impossibile
eseguire il checkout.
fatal: Cannot update paths and switch to branch 'dummy' at the same time.
Did you intend to checkout 'xen-tested-master' which can not be resolved
as commit?
Makefile:23: set di istruzioni per l'obiettivo "ovmf-dir" non riuscito
make[3]: *** [ovmf-dir] Errore 128

I not understand why use a dummy branch even if directory is a "temp."
I tried a very fast change and worked:

#!/bin/sh

if test $# -lt 3; then
echo "Usage: $0   "
exit 1
fi

TREE=$1
TAG=$2
DIR=$3

set -e

if test \! -d $DIR-remote; then
rm -rf $DIR-remote $DIR-remote.tmp
mkdir -p $DIR-remote.tmp; rmdir $DIR-remote.tmp
if test "$TAG" ; then
$GIT clone -b $TAG $TREE $DIR-remote.tmp
else
$GIT clone $TREE $DIR-remote.tmp
fi
mv $DIR-remote.tmp $DIR-remote
fi
rm -f $DIR
ln -sf $DIR-remote $DIR

But probably a better improvement/solution is needed for a good script
working with any branch (its head) or tag.

Thanks for any reply and sorry for my bad english.

It's just because your local copy doesn't have xen-tested-master --
there is no such local branch.

We either put a commit id or tag in Config.mk. If you want to put a
branch in Config.mk in your case, use origin/xen-tested-master.

I don't think it is a problem in git-checkout.sh because I don't think
we should make it "smart" enough to second guess what the user wants.

Wei.

Thanks for your reply, I'll try with origin next tests.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] scripts/git-checkout.sh fails in some cases

[Fabio Fantoni]

I already saw scripts/git-checkout.sh fails many times changing repo 
and/or tag for my tests, for example today:
GIT=git 
/mnt/vm/source/xen/Xen/tools/firmware/../../scripts/git-checkout.sh 
git://xenbits.xen.org/osstest/ovmf.git xen-tested-master ovmf-dir

Cloning into 'ovmf-dir-remote.tmp'...
remote: Counting objects: 203149, done.
remote: Compressing objects: 100% (49944/49944), done.
remote: Total 203149 (delta 153028), reused 196420 (delta 146807)
Ricezione degli oggetti: 100% (203149/203149), 194.95 MiB | 527.00 
KiB/s, done.

Risoluzione dei delta: 100% (153028/153028), done.
Checking connectivity... fatto.
warning: l'HEAD remoto si riferisce ad un ref inesistente, impossibile 
eseguire il checkout.

fatal: Cannot update paths and switch to branch 'dummy' at the same time.
Did you intend to checkout 'xen-tested-master' which can not be 
resolved as commit?

Makefile:23: set di istruzioni per l'obiettivo "ovmf-dir" non riuscito
make[3]: *** [ovmf-dir] Errore 128


I not understand why use a dummy branch even if directory is a "temp."
I tried a very fast change and worked:

#!/bin/sh

if test $# -lt 3; then
echo "Usage: $0   "
exit 1
fi

TREE=$1
TAG=$2
DIR=$3

set -e

if test \! -d $DIR-remote; then
rm -rf $DIR-remote $DIR-remote.tmp
mkdir -p $DIR-remote.tmp; rmdir $DIR-remote.tmp
if test "$TAG" ; then
$GIT clone -b $TAG $TREE $DIR-remote.tmp
else
$GIT clone $TREE $DIR-remote.tmp
fi
mv $DIR-remote.tmp $DIR-remote
fi
rm -f $DIR
ln -sf $DIR-remote $DIR


But probably a better improvement/solution is needed for a good script 
working with any branch (its head) or tag.


Thanks for any reply and sorry for my bad english.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [Qemu-devel] Question about xen disk unplug support for ahci missed in qemu

[Fabio Fantoni]

Il 14/10/2015 11:47, Kevin Wolf ha scritto:

[ CC qemu-block ]

Am 13.10.2015 um 19:10 hat Stefano Stabellini geschrieben:

On Tue, 13 Oct 2015, John Snow wrote:

On 10/13/2015 11:55 AM, Fabio Fantoni wrote:

I added ahci disk support in libxl and using it for week seems that was
ok, after a reply of Stefano Stabellini seems that xen disk unplug
support only ide disks:
http://git.qemu.org/?p=qemu.git;a=commitdiff;h=679f4f8b178e7c66fbc2f39c905374ee8663d5d8

Today Paul Durrant told me that even if pv disk is ok also with ahci and
the emulated one is offline can be a risk:
http://lists.xenproject.org/archives/html/win-pv-devel/2015-10/msg00021.html


I tried to take a fast look in qemu code but I not understand the needed
thing for add the xen disk unplug support also for ahci, can someone do
it or tell me useful information for do it please?

Thanks for any reply and sorry for my bad english.


I'm not entirely sure what features you need AHCI to support in order
for Xen to be happy.

I'd guess hotplugging, but where I get confused is that IDE disks don't
support hotplugging either, so I guess I'm not sure sure what you need.

Stefano, can you help bridge my Xen knowledge gap?
  
Hi John,


we need something like hw/i386/xen/xen_platform.c:unplug_disks but that
can unplug AHCI disk. And by unplug, I mean "make disappear" like
pci_piix3_xen_ide_unplug does for ide.

Maybe this would be the right time to stop the craziness with your
hybrid IDE/xendisk setup. It's a horrible thing that would never happen
on real hardware.

Can't you just teach SeaBIOS how to deal with your PV disks and then
only add that to your VM and forget about IDE/AHCI? I mean, that's how
it's done for virtio-blk, and it doesn't involve any insanities like
ripping out non-hotpluggable devices.

Hm... How does a reboot of a machine that had its IDE already removed
actually work? Do you restart the qemu process on reboot?

Kevin
I thinkthat would be a good idea, unfortunately I'm not able to do that 
and I do not know if the developers of xen would agree to such modification.


If will be done, for example having new disk type "xenpv" require the pv 
drivers installed, with linux having drivers included should not be a 
problem but on windows yes FWIK.
Like virtio driver should be installed before (or adding them in windows 
install), I don't know if will be ok specifying them in install for with 
xen driver, another possibility can be start domU with ide disk type, 
install the xen pv drivers and reboot selecting xenpv disk type instead.
Doing it FWIK require not only xen and qemu changes but also pv drivers 
change, I added on cc also Paul Durrant for see what think about.
With actual unplug based on my tests in some years I had many problem 
with windows domUs (with both old and new pv drivers) resulting in 
"windows boot blue screen error", I suppose that changing/improving 
unplug method can decrease them, is it correct?


About reboot xen do different from kvm recreating full domU each time 
(and new qemu process), I don't know if is possible and good change the 
method.


Thanks for any reply and sorry for my bad english.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] Question about xen disk unplug support for ahci missed in qemu

[Fabio Fantoni]

I added ahci disk support in libxl and using it for week seems that was 
ok, after a reply of Stefano Stabellini seems that xen disk unplug 
support only ide disks:

http://git.qemu.org/?p=qemu.git;a=commitdiff;h=679f4f8b178e7c66fbc2f39c905374ee8663d5d8
Today Paul Durrant told me that even if pv disk is ok also with ahci and 
the emulated one is offline can be a risk:

http://lists.xenproject.org/archives/html/win-pv-devel/2015-10/msg00021.html

I tried to take a fast look in qemu code but I not understand the needed 
thing for add the xen disk unplug support also for ahci, can someone do 
it or tell me useful information for do it please?


Thanks for any reply and sorry for my bad english.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] xen 4.6.0 on jessie dom0 unable to start domUs and questions about systemd and build deps

[Fabio Fantoni]

Il 11/10/2015 17:43, Fabio Fantoni ha scritto:

Il 11/10/2015 16:54, Wei Liu ha scritto:

On Sat, Oct 10, 2015 at 03:08:54PM +0200, Fabio Fantoni wrote:
[...]
there was an error in generated config/Tools.mk I solved manually 
(replacing

@debug@ with y), already reported here:
http://lists.xen.org/archives/html/xen-devel/2015-10/msg01184.html


That one is a real bug and I've posted patch for it with you CC'ed.


Thanks, now I'll test it to be sure is all ok with it.
For the others see below.




I enabled xen services with systemd:
systemctl enable xencommons
systemctl enable xendomains
I missed or did wrong something?

And used this grub entry:

menuentry 'Jessie con Linux 3.16.0-4-amd64 e XEN 4.6 - RAID' --class
debian --class gnu-linux --class gnu --class os {
set root='(lvm/RAID-ROOT)'
echo'Caricamento Hypervisor Xen 4.6...'
multiboot   /boot/xen.gz placeholder dom0_mem=2G,max:2G

Make sure xen.gz is pointing to xen 4.6 file.


Yes or will not start and is the only build I did on full clean system.




echo'Caricamento Linux 3.16.0-4-amd64...'
module  /boot/vmlinuz-3.16.0-4-amd64 placeholder
root=/dev/mapper/RAID-ROOT ro quiet
echo'Caricamento ramdisk iniziale...'
module  --nounzip /boot/initrd.img-3.16.0-4-amd64
}

I tried to remove swiotlb=65762 from kernel parameters used for year as
workaround that I not remember good, should be still neded?


Not sure.


Is there someone can reply me about it please?




DomU fails to start:

[...]

deregister slotnum=3
libxl: debug: libxl_device.c:937:device_backend_callback: calling
device_backend_cleanup
libxl: debug: libxl_event.c:691:libxl__ev_xswatch_deregister: watch
w=0x1828820: deregister unregistered
libxl: error: libxl.c:1972:libxl__get_domid: failed to get own domid
(domid)
libxl: error: libxl_device.c:987:device_hotplug: Failed to get domid

I think this is toolstack domain trying to get it's own domain id.
And this leads to other errors.


But there are strange things:

xl list
NameID   Mem VCPUs State   
Time(s)

(null)   0  2048 8 r-

Was xen-init-dom0 service started? I doubt that because there wasn't
/local/domain/0/domid in your xenstore dump.


Yes, you are right:

systemctl status xen-init-dom0
  xen-init-dom0.service - xen-init-dom0, initialise Dom0 
configuration (xenstore nodes, JSON configuration stub)
   Loaded: loaded (/usr/lib/systemd/system/xen-init-dom0.service; 
disabled)

   Active: inactive (dead)


systemd support need improvement or documentation like this page:
http://wiki.xenproject.org/wiki/Compiling_Xen_From_Source

I did this following it:

systemctl enable xencommons
systemctl enable xendomains

But seems don't include/enable all needed deps.

Can someone tell me exactly what is the solution please? the patch for 
it (if needed, I don't know good systemd) and/or things missed to do 
for fully enable it (to add also in docs)




I did a tests and with these commands seems that load all needed for 
boot correctly domUs:

systemctl enable xenstored.service
systemctl enable xenconsoled.service
systemctl enable xen-init-dom0.service
systemctl enable xen-qemu-dom0-disk-backend.service
systemctl enable xendomains.service

These is only a strange thing about xendomains:

- systemctl enable xenstored.service
Created symlink from 
/etc/systemd/system/multi-user.target.wants/xenstored.service to 
/usr/lib/systemd/system/xenstored.service.
Created symlink from 
/etc/systemd/system/sockets.target.wants/xenstored_ro.socket to 
/usr/lib/systemd/system/xenstored_ro.socket.
Created symlink from 
/etc/systemd/system/sockets.target.wants/xenstored.socket to 
/usr/lib/systemd/system/xenstored.socket.

- systemctl enable xenconsoled.service
Created symlink from 
/etc/systemd/system/multi-user.target.wants/xenconsoled.service to 
/usr/lib/systemd/system/xenconsoled.service.

- systemctl enable xen-init-dom0.service
Created symlink from 
/etc/systemd/system/multi-user.target.wants/xen-init-dom0.service to 
/usr/lib/systemd/system/xen-init-dom0.service.

- systemctl enable xen-qemu-dom0-disk-backend.service
Created symlink from 
/etc/systemd/system/multi-user.target.wants/xen-qemu-dom0-disk-backend.service 
to /usr/lib/systemd/system/xen-qemu-dom0-disk-backend.service.

- systemctl enable xendomains.service
Synchronizing state for xendomains.service with sysvinit using 
update-rc.d...

Executing /usr/sbin/update-rc.d xendomains defaults
insserv: warning: current start runlevel(s) (empty) of script 
`xendomains' overrides LSB defaults (2 3 5).
insserv: warning: current stop runlevel(s) (0 1 2 3 5 6) of script 
`xendomains' overrides LSB defaults (0 1 6).

Executing /usr/sbin/update-rc.d xendomains enable
Created symlink from 
/etc/systemd/system/multi-user.target.wants/xendomains.service to 

Re: [Xen-devel] [PATCH] build: don't shadow debug with "@debug@" in tools build

[Fabio Fantoni]

Il 11/10/2015 16:41, Wei Liu ha scritto:

In 16181cbb (tools: Honor Config.mk debug value, rather than setting our
own), configure doesn't set debug variable anymore. There is, however,
one place that was missed. The file config/Tools.mk.in was still
expecting a @debug@ value from configure. After 16181cbb that value
remained "debug := @debug@" all the time because configure didn't
substitute it.

The consequence was that we couldn't get a debug build even if debug was
set to "y" in Config.mk.

Fix this by removing the stray line "debug := @debug@" in Tools.mk.in.

Reported-by: Fabio Fantoni 
Signed-off-by: Wei Liu 


Tested-by: Fabio Fantoni 

Thanks, tried and now seems all ok about debug things on build FWIK.

---
Cc: Fabio Fantoni 
Cc: Ian Campbell 
Cc: Ian Jackson 
Cc: George Dunlap 

Backport candidate for 4.6.
---
  config/Tools.mk.in | 3 ---
  1 file changed, 3 deletions(-)

diff --git a/config/Tools.mk.in b/config/Tools.mk.in
index 9bd5f6c..ccfe137 100644
--- a/config/Tools.mk.in
+++ b/config/Tools.mk.in
@@ -5,9 +5,6 @@ ifeq ($(CONFIG_RUMP),y)
  XEN_OS  := NetBSDRump
  endif
  
-# A debug build of tools?

-debug   := @debug@
-
  # Tools path
  BISON   := @BISON@
  FLEX:= @FLEX@



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] xen 4.6.0 on jessie dom0 unable to start domUs and questions about systemd and build deps

[Fabio Fantoni]

Il 11/10/2015 16:54, Wei Liu ha scritto:

On Sat, Oct 10, 2015 at 03:08:54PM +0200, Fabio Fantoni wrote:
[...]

there was an error in generated config/Tools.mk I solved manually (replacing
@debug@ with y), already reported here:
http://lists.xen.org/archives/html/xen-devel/2015-10/msg01184.html


That one is a real bug and I've posted patch for it with you CC'ed.


Thanks, now I'll test it to be sure is all ok with it.
For the others see below.




I enabled xen services with systemd:
systemctl enable xencommons
systemctl enable xendomains
I missed or did wrong something?

And used this grub entry:

menuentry 'Jessie con Linux 3.16.0-4-amd64 e XEN 4.6 - RAID' --class
debian --class gnu-linux --class gnu --class os {
set root='(lvm/RAID-ROOT)'
echo'Caricamento Hypervisor Xen 4.6...'
multiboot   /boot/xen.gz placeholder dom0_mem=2G,max:2G

Make sure xen.gz is pointing to xen 4.6 file.


Yes or will not start and is the only build I did on full clean system.




echo'Caricamento Linux 3.16.0-4-amd64...'
module  /boot/vmlinuz-3.16.0-4-amd64 placeholder
root=/dev/mapper/RAID-ROOT ro quiet
echo'Caricamento ramdisk iniziale...'
module  --nounzip /boot/initrd.img-3.16.0-4-amd64
}

I tried to remove swiotlb=65762 from kernel parameters used for year as
workaround that I not remember good, should be still neded?


Not sure.


Is there someone can reply me about it please?




DomU fails to start:

[...]

deregister slotnum=3
libxl: debug: libxl_device.c:937:device_backend_callback: calling
device_backend_cleanup
libxl: debug: libxl_event.c:691:libxl__ev_xswatch_deregister: watch
w=0x1828820: deregister unregistered
libxl: error: libxl.c:1972:libxl__get_domid: failed to get own domid
(domid)
libxl: error: libxl_device.c:987:device_hotplug: Failed to get domid

I think this is toolstack domain trying to get it's own domain id.
And this leads to other errors.


But there are strange things:

xl list
NameID   Mem VCPUs State   Time(s)
(null)   0  2048 8 r-

Was xen-init-dom0 service started? I doubt that because there wasn't
/local/domain/0/domid in your xenstore dump.


Yes, you are right:

systemctl status xen-init-dom0
  xen-init-dom0.service - xen-init-dom0, initialise Dom0 configuration 
(xenstore nodes, JSON configuration stub)
   Loaded: loaded (/usr/lib/systemd/system/xen-init-dom0.service; 
disabled)

   Active: inactive (dead)


systemd support need improvement or documentation like this page:
http://wiki.xenproject.org/wiki/Compiling_Xen_From_Source

I did this following it:

systemctl enable xencommons
systemctl enable xendomains

But seems don't include/enable all needed deps.

Can someone tell me exactly what is the solution please? the patch for 
it (if needed, I don't know good systemd) and/or things missed to do for 
fully enable it (to add also in docs)






15.3

and:

xenstore-ls
tool = ""
xenstored = ""
local = ""
domain = ""
  0 = ""
   device-model = ""
0 = ""
 state = "running"
   backend = ""
vbd = ""
 1 = ""
  768 = ""
   frontend = "/local/domain/1/device/vbd/768"
   params = "/mnt/vm/disks/W10.disk1.xm"
   script = "/etc/xen/scripts/block"
   frontend-id = "1"
   online = "1"
   removable = "0"
   bootable = "1"
   state = "6"
   dev = "hda"
   type = "phy"
   mode = "w"
   device-type = "disk"
   discard-enable = "1"
  832 = ""
   frontend = "/local/domain/1/device/vbd/832"
   params = "/mnt/vm/iso/Windows10pro64bit.iso"
   script = "/etc/xen/scripts/block"
   frontend-id = "1"
   online = "1"
   removable = "1"
   bootable = "1"
   state = "6"
   dev = "hdb"
   type = "phy"
   mode = "r"
   device-type = "cdrom"
   discard-enable = "0"
vm = ""
libxl = ""

data about a domU not started correctly...

I not understand what is exactly is wrong.

Checking xen services I found a xendriverdomain service added recently but
not in systemd:
http://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=2909ad4c96ba5527b35ac87c089116cd0320a5e7
is this needed but missed in systemd support?


That's irrelevant to your problem because you weren't using driver
domain.

Wei.



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] xen 4.6.0 on jessie dom0 unable to start domUs and questions about systemd and build deps

[Fabio Fantoni]

Hi, I built xen 4.6.0 from source on jessie 64 bit system.
I cleaned some deps from too many of previous systems, built
successufull xen and upstream qemu with these dependencies:
aptitude install build-essential git-core patch python-dev uuid-dev
gettext libncurses5-dev libyajl-dev libaio-dev pkg-config libglib2.0-dev
libssl-dev libpixman-1-dev bridge-utils wget bcc iasl bin86 texinfo
libc6-dev-i386 e2fslibs-dev gcc-multilib gcc liblzma-dev zlib1g-dev
libc6-dev libsystemd-daemon-dev
# Installed seabios package from unstable
# Additional qemu deps
aptitude install libcurl4-openssl-dev libncurses5-dev libjpeg-dev
# additional spice and usbredir deps (latest spice rebuild from unstable
instead)
aptitude install libspice-protocol-dev libspice-server-dev
libusbredirparser-dev libusb-1.0-0-dev

I have dubt about some deps not installed because not blocked by
configure but that I remember was used in past, for example:
bison flex binutils graphviz gawk iproute
these or other deps are required even if configure don't check/stop and
built successfull?

I did these changes in Config.mk:
debug ?= y
PYTHON_PREFIX_ARG ?=

configure options used:
./configure --prefix=/usr --disable-blktap1 --disable-qemu-traditional
--disable-rombios --with-system-seabios=/usr/share/seabios/bios-256k.bin
--with-extra-qemuu-configure-args="--enable-spice --enable-usb-redir"
--disable-blktap2 -enable-systemd

there was an error in generated config/Tools.mk I solved manually
(replacing @debug@ with y), already reported here:
http://lists.xen.org/archives/html/xen-devel/2015-10/msg01184.html

I enabled xen services with systemd:
systemctl enable xencommons
systemctl enable xendomains
I missed or did wrong something?

And used this grub entry:

menuentry 'Jessie con Linux 3.16.0-4-amd64 e XEN 4.6 - RAID' --class
debian --class gnu-linux --class gnu --class os {
set root='(lvm/RAID-ROOT)'
echo'Caricamento Hypervisor Xen 4.6...'
multiboot   /boot/xen.gz placeholder dom0_mem=2G,max:2G
echo'Caricamento Linux 3.16.0-4-amd64...'
module  /boot/vmlinuz-3.16.0-4-amd64 placeholder
root=/dev/mapper/RAID-ROOT ro quiet
echo'Caricamento ramdisk iniziale...'
module  --nounzip /boot/initrd.img-3.16.0-4-amd64
}

I tried to remove swiotlb=65762 from kernel parameters used for year as
workaround that I not remember good, should be still neded?


DomU fails to start:

xl -vvv create /etc/xen/W10.cfg
Parsing config from /etc/xen/W10.cfg
libxl: debug: libxl_create.c:1557:do_domain_create: ao 0x1825260:
create: how=(nil) callback=(nil) poller=0x18252f0
libxl: debug: libxl_device.c:269:libxl__device_disk_set_backend: Disk
vdev=hda spec.backend=unknown
libxl: debug: libxl_device.c:298:libxl__device_disk_set_backend: Disk
vdev=hda, using backend phy
libxl: debug: libxl_device.c:269:libxl__device_disk_set_backend: Disk
vdev=hdb spec.backend=unknown
libxl: debug: libxl_device.c:298:libxl__device_disk_set_backend: Disk
vdev=hdb, using backend phy
libxl: debug: libxl_create.c:945:initiate_domain_create: running
bootloader
libxl: debug: libxl_bootloader.c:324:libxl__bootloader_run: not a PV
domain, skipping bootloader
libxl: debug: libxl_event.c:691:libxl__ev_xswatch_deregister: watch
w=0x1826d28: deregister unregistered
libxl: detail: libxl_dom.c:239:hvm_set_viridian_features: base group
enabled
libxl: detail: libxl_dom.c:239:hvm_set_viridian_features: freq group
enabled
libxl: detail: libxl_dom.c:239:hvm_set_viridian_features:
time_ref_count group enabled
xc: detail: elf_parse_binary: phdr: paddr=0x10 memsz=0x5bbc4
xc: detail: elf_parse_binary: memory: 0x10 -> 0x15bbc4
xc: detail: VIRTUAL MEMORY ARRANGEMENT:
xc: detail:   Loader:   0010->0015bbc4
xc: detail:   Modules:  ->
xc: detail:   TOTAL:->f800
xc: detail:   ENTRY:00100630
xc: detail: PHYSICAL MEMORY ALLOCATION:
xc: detail:   4KB PAGES: 0x0200
xc: detail:   2MB PAGES: 0x03bf
xc: detail:   1GB PAGES: 0x0002
xc: detail: elf_load_binary: phdr 0 at 0x7f0f7a73e000 -> 0x7f0f7a790151
domainbuilder: detail: xc_dom_gnttab_hvm_seed: called, pfn=0x108000
libxl: debug: libxl_device.c:269:libxl__device_disk_set_backend: Disk
vdev=hda spec.backend=phy
libxl: debug: libxl_event.c:639:libxl__ev_xswatch_register: watch
w=0x1828820 wpath=/local/domain/0/backend/vbd/1/768/state token=3/0:
register slotnum=3
libxl: debug: libxl_device.c:269:libxl__device_disk_set_backend: Disk
vdev=hdb spec.backend=phy
libxl: debug: libxl_device.c:269:libxl__device_disk_set_backend: Disk
vdev=hdb spec.backend=phy
libxl: debug: libxl_event.c:639:libxl__ev_xswatch_register: watch
w=0x1829a30 wpath=/local/domain/0/backend/vbd/1/832/state token=2/1:
register slotnum=2
libxl: debug: libxl_create.c:1580:do_domain_create: ao 0x1825260:
inprogress: poller=0x18252f0, flags=i
libxl: debug: libxl_event.c:576:watchfd_callback: watch w=0x1828820
wpath=/l

[Xen-devel] Regression: debug not setted correctly in config/Tools.mk of xen 4.6

[Fabio Fantoni]

In latest weeks building xen 4.6 with debug enable qemu trace was 
missed, today building xen 4.6.0 I found the cause.
Even if debug ?= y in Config.mk, config/Tools.mk is not generate 
correctly, have this line:

debug   := @debug@

I not found the exact cause for doing the patch that fix it, can someone 
help me please?


If can be useful, in my installation I change these 2 line in Config.mk 
(enable debug and fix python prefix, if I remember good pygrub on debian 
was not working without):

debug ?= y
PYTHON_PREFIX_ARG ?=

configure options used:
./configure --prefix=/usr --disable-blktap1 --disable-qemu-traditional 
--disable-rombios --with-system-seabios=/usr/share/seabios/bios-256k.bin 
--with-extra-qemuu-configure-args="--enable-spice --enable-usb-redir" 
--disable-blktap2 -enable-systemd

in attachment the full Tools.mk generated.


Thanks for any reply and sorry for my bad english.
-include $(XEN_ROOT)/config/Paths.mk

CONFIG_RUMP := n
ifeq ($(CONFIG_RUMP),y)
XEN_OS  := NetBSDRump
endif

# A debug build of tools?
debug   := @debug@

# Tools path
BISON   := 
FLEX:= 
PYTHON  := python
PYTHON_PATH := /usr/bin/python
PY_NOOPT_CFLAGS := -O1
PERL:= /usr/bin/perl
CURL_CONFIG := 
XML2_CONFIG := 
BASH:= /bin/bash
XGETTTEXT   := /usr/bin/xgettext
AS86:= /usr/bin/as86
LD86:= /usr/bin/ld86
BCC := /usr/bin/bcc
IASL:= /usr/bin/iasl
AWK := /usr/bin/awk
FETCHER := /usr/bin/wget -c -O
SEABIOS_PATH:= /usr/share/seabios/bios-256k.bin
OVMF_PATH   := 

# Extra folder for libs/includes
PREPEND_INCLUDES:= 
PREPEND_LIB := 
APPEND_INCLUDES := 
APPEND_LIB  := 

PTHREAD_CFLAGS  := -pthread
PTHREAD_LDFLAGS := -pthread
PTHREAD_LIBS:= 

PTYFUNCS_LIBS   := -lutil

LIBNL3_LIBS := 
LIBNL3_CFLAGS   := 
XEN_TOOLS_RPATH := n

# Download GIT repositories via HTTP or GIT's own protocol?
# GIT's protocol is faster and more robust, when it works at all (firewalls
# may block it). We make it the default, but if your GIT repository downloads
# fail or hang, please specify GIT_HTTP=y in your environment.
GIT_HTTP:= n

# Optional components
XENSTAT_XENTOP  := y
OCAML_TOOLS := n
FLASK_POLICY:= n
CONFIG_OVMF := n
CONFIG_ROMBIOS  := n
CONFIG_SEABIOS  := y
CONFIG_QEMU_TRAD:= n
CONFIG_QEMU_XEN := y
CONFIG_BLKTAP2  := n
CONFIG_QEMUU_EXTRA_ARGS:= --enable-spice --enable-usb-redir
CONFIG_REMUS_NETBUF := n

CONFIG_SYSTEMD  := y
SYSTEMD_CFLAGS  := 
SYSTEMD_LIBS:= -lsystemd 
XEN_SYSTEMD_DIR := $(prefix)/lib/systemd/system/
XEN_SYSTEMD_MODULES_LOAD := $(prefix)/lib/modules-load.d/

LINUX_BACKEND_MODULES := xen-evtchn xen-gntdev xen-gntalloc xen-blkback 
xen-netback xen-pciback evtchn gntdev netbk blkbk xen-scsibk usbbk pciback 
xen-acpi-processor blktap2

#System options
ZLIB:=  -DHAVE_LZMA -llzma
CONFIG_LIBICONV := n
CONFIG_GCRYPT   := n
EXTFS_LIBS  := -lext2fs
CURSES_LIBS := -lncurses
TINFO_LIBS  := -ltinfo
ARGP_LDFLAGS:= 

FILE_OFFSET_BITS:= 
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Regression: qemu crash of hvm domUs with spice (backtrace included)

[Fabio Fantoni]

Il 08/10/2015 17:58, Andreas Kinzler ha scritto:

Is this still current? I made an interesting observation:

I had no problems with SPICE and vanilla Xen 4.5.1 when using it on 
Gentoo with glibc 2.19/gcc 4.6.4.
Segfaults started when I switched to glibc 2.20/gcc 4.9.3 - I did not 
change Xen source code at all.
All this might be related to: 
https://lists.nongnu.org/archive/html/qemu-devel/2015-06/msg02764.html


Andreas


Thanks for your mail.
The problem I had seems different, I not found the exactly cause but I 
solved using newer qemu (2.2 from unstable - now 4.6) with xen 4.5.1.

Big distro I saw already use newer qemu and should be ok.
I still using glibc < 2.20 in my debian servers, this is probably 
because I not had your problem but I think that backport the patch you 
linked can be useful for solve a qemu crash case, I'll test it in my 
next build and if I'll not found regression I'll require the backport 
for xen qemu gits.





https://github.com/Fantu/Xen/commits/rebase/m2r-staging
Latest test with regression based on latest stable-4.5, more exactly:
https://github.com/Fantu/Xen/commits/rebase/m2r-testing
Some days ago on same dom0 and domU I tried with latest stable 
version (that I use on only 2 production servers for now but I not 
saw the regression), more exactly:

https://github.com/Fantu/Xen/commits/rebase/m2r-stable-4.5
Dom0 debian 7 with kernel 3.16 from backports, seabios 1.8.1-2 from 
unstable and this xen configure:
./configure --prefix=/usr --disable-blktap1 
--disable-qemu-traditional --disable-rombios 
--with-system-seabios=/usr/share/seabios/bios-256k.bin 
--with-extra-qemuu-configure-args="--enable-spice --enable-usb-redir" 
--disable-blktap2


I suppose that there is unexpected case caused by a backports or 
missed patch/es to backports from unstable.
I not found with a fast look rilevant patch to try to revert, can 
anyone suggest me the more probable point/s for bisect and/or patch 
to revert or I must try full bisect 4.5.0->stable-4.5?







___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Improve or change devel mailing-list for improve efficiency and spending less time, is this possible?

[Fabio Fantoni]

Il 31/08/2015 21:38, Russell Pavlicek ha scritto:

Please forgive the top-post.  I am stuck with an interface which does not 
facilitate inline replies (as insane as that may sound).


Russell has evaluated some off-the shelf tooling that would allow bridging
the gap: unfortunately there is nothing good out there, which works well
in practice. Google groups, which was designed with list support in it
also kind of sucks. However, I believe we still have a project with a
contractor ongoing to try and bridge xen-users with
http://xenproject.org/help/questions-and-answers.html (such that they
become mirrors of each other) - I will let Russell give an update. If that
project gets to the point that the bridge works, that nobody is badly
impacted, maybe we can consider extending it to other lists.

Lars is correct that the choices are sparse and poor when it comes to skinning existing email 
lists as forums.  We have a stalled, but on-going, project to skin the xen-users mailing list 
on the Q&A forum on XenProject.org.  Most of the basic functionality exists in the 
extension we are trying to use, but functions (emailing certain parties who are interacting 
via the website) do not work as advertised.  The contractor was following up with the 
extension provider, but atop-to-bottom reorganization of the contracting company has put the 
effort on hold for the past several weeks until they reboot their services division 
(supposedly, "next month"; we'll see).

We were approached by a startup last year which had promising forum-skinning 
technology, but they were both closed source and expensive.  I think that 
company may have failed already.

There are a handful of options of integrated forum & mailing list (Discourse 
being one of the most prominent), but they require recreating the mailing list 
under the auspices of the tool. So the xen-devel mailing list as we know it would 
have to be abandoned and/or migrated to the new tool.  That represents a 
potentially large disturbance to existing users, which isn't desirable.

I haven't reassessed the solution landscape in a few months, so maybe it is 
time to check again.  But I am not hopeful that we will find a good result.

BTW, if you know someone looking for their next FOSS project, feel free to 
suggest transparent skinning of an email list with a forum interface.  It is a 
real niche with no good players.


I still not found a good and "all-in-one" solution but I saw this open 
source project: patchwork http://jk.ozlabs.org/projects/patchwork/
Seems interesting, is integrated with mailing list, now seems with 
"basic features" but probably in future it could become great.
I same that many open source project already use it, has someone here 
already tried it in other projects? If yes what do you think?




Russ Pavlicek
Xen Project Evangelist, Citrix Systems
Home Office: +1-301-829-5327
Mobile: +1-301-814-1143
UK VoIP: +44 1223 852 894


From: Lars Kurth
Sent: Monday, August 31, 2015 1:32 PM
To: Wei Liu; Fabio Fantoni; Russell Pavlicek
Cc: xen devel
Subject: Re: [Xen-devel] Improve or change devel mailing-list for improve 
efficiency and spending less time, is this possible?

Adding Russell,


On 31/08/2015 15:16, "Wei Liu"  wrote:


CC Lars.

On Mon, Aug 31, 2015 at 03:46:30PM +0200, Fabio Fantoni wrote:

I saw some discussions about the developer cycle, patch review and how
to
improve.

Based on what I saw an important thing for improve efficiency and
spending
less time is the way of communication, actual seems only
mailing-list/mails.

It seems to me that currently has some limits and in many cases takes a
long
time for some specific searches,trouble finding patches that need
attention
etc...

A good solution I think would be a forum specific for developing with
these
features:

Fabio, of course the downside is that some developers are quite happy with
mailing lists. There is also a generational gap: developers that grew up
with lists prefer lists, developers that grew up with forums prefer forums
(the same is true for people starting in FOSS with github, which exposes
forum like functionality). This is a well known problem in many FOSS
communities.
http://programmers.stackexchange.com/questions/71148/why-do-programmers-sti
ll-use-mailing-lists is covering many topics of that debate (without
getting religious about it).

We did also have a discussion at the 2014 developer meeting, where we
discussed the high volume of traffic on xen-devel. At the time we decided
not to split the list and use better tagging (arm, x86, tools, ...).

Russell has evaluated some off-the shelf tooling that would allow bridging
the gap: unfortunately there is nothing good out there, which works well
in practice. Google groups, which was designed with list support in it
also kind of sucks. However, I believe we still have a project with a
contractor ongoing to try and

Re: [Xen-devel] Oldest supported Xen version in upstream QEMU (Was: Re: [Minios-devel] [PATCH v2 0/15+5+5] Begin to disentangle libxenctrl and provide some stable libraries)

[Fabio Fantoni]

Il 24/09/2015 09:15, Ian Campbell ha scritto:

On Wed, 2015-09-23 at 18:36 +0100, Stefano Stabellini wrote:

On Wed, 23 Sep 2015, Ian Campbell wrote:

On Tue, 2015-09-22 at 22:31 +0100, Stefano Stabellini wrote:

The oldest Xen version I build-test for every pull request is Xen 4.0.0,

I setup a build trees for 4.0 thru 4.6 yesterday to test this, what a
pain 4.1 and 4.0 are to build with a modern gcc! (Mostly newer compiler
warnings and mostly, but not all, fixes which I could just backport
from newer Xen, the exceptions were a couple of things which were
removed before they needed to be fixed)


I think it is very reasonable to remove anything older than that.
I am OK with removing Xen 4.0.0 too, but I would like a warning to be
sent ahead of time to qemu-devel to see if anybody complains.

There is not much point in removing <=3.4 support and keeping 4.0, since
4.0.0 was the last one which used a plain int as a handle, anything older
than 4.0.0 is trivial if 4.0.0 is supported.

One approach I am considering in order to keep 4.0.0 support and earlier
was to turn the "int fd" for <=4.0 into a pointer by having the open
wrapper do malloc(sizeof int) and the using wrappers do xc_foo(*handle).

This way all the different variants take pointers and we have less hoops to
jump through to typedef everything in the correct way for each variant.

If you would rather avoid doing that then I think dropping 4.0.0 support
would be the way to go and I'll send a mail to qemu-devel.
  
I would rather drop 4.0 support.

Supporting 4.0 didn't turn out quite as ugly as I had feared.


Even if seems that major of distro use newer qemu version instead older 
with xen seems that keep updated also xen to latest stable version (or 
at least previous) and the cases where can uses next qemu version with 
very old xen I think is at least very rare.
Another thing is that upstream qemu on older xen not had good support 
(at least for hvm domUs) and when I started to use it some years ago 
seems that was with too very few users.

There are also some problem, for example this:
http://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=2e814a017155b885e4d4b5a88dc05e7367a9722a
without hvm domUs with emulated vga don't start with qemu>=1.4, applied 
to xen 4.3 and backported only in stable-4.2
Therefore excluded recent unexpected that I not know for a working 
support of hvm domUsat least xen 4.2 is needed.
In my cases I had saw good hvm domUs support with upstream qemu for 
production use only starting from xen 4.3.
I think is good know if older xen is really used with newer (more 
exactly next for these changes) qemu instead wasting time supporting and 
testing also many older xen versions.

In any cases thanks to all for any xen and qemu improvements.

Sorry for my bad english.



So before I send an email to qemu-devel to propose dropping 4.0 what do
you think of the following which handles the evtchn case, there is a
similar patch for gnttab and a (yet to be written) patch for the
foreign memory mapping case.

The relevant bit for this discussion is the 4.0.0 definition of
xenevtchn_open in xen_common.h, the rest is just adjusting it to use
the API of the new library (for reasons explained in the commit
message).

commit d97f6bb5045685d766d85b8cd004ce007fe29120
Author: Ian Campbell 
Date:   Wed Sep 23 17:30:15 2015 +0100

 xen: Switch to libxenevtchn interface for compat shims.
 
 In Xen 4.7 we are refactoring parts libxenctrl into a number of

 separate libraries which will provide backward and forward API and ABI
 compatiblity.
 
 One such library will be libxenevtchn which provides access to event

 channels.
 
 In preparation for this switch the compatibility layer in xen_common.h

 (which support building with older versions of Xen) to use what will
 be the new library API. This means that the evtchn shim will disappear
 for versions of Xen which include libxenevtchn.
 
 To simplify things for the <= 4.0.0 support we wrap the int fd in a

 malloc(sizeof int) such that the handle is always a pointer. This
 leads to less typedef headaches and the need for
 XC_HANDLER_INITIAL_VALUE etc for these interfaces.
 
 Build tested with 4.1 and 4.5.
 
 Note that this patch does not add any support for actually using

 libxenevtchn, it just adjusts the existing shims.
 
 Note that xc_evtchn_alloc_unbound functionality remains in libxenctrl,

 since that functionality is not exposed by /dev/xen/evtchn.
 
 Signed-off-by: Ian Campbell 


diff --git a/hw/xen/xen_backend.c b/hw/xen/xen_backend.c
index b2cb22b..1fd8e01 100644
--- a/hw/xen/xen_backend.c
+++ b/hw/xen/xen_backend.c
@@ -243,19 +243,19 @@ static struct XenDevice *xen_be_get_xendev(const char 
*type, int dom, int dev,
  xendev->debug  = debug;
  xendev->local_port = -1;
  
-xendev->evtchndev = xen_xc_evtchn_open(NULL, 0);

-if (xendev->evtchndev == XC_HANDLE

[Xen-devel] DomU's qemu crashed with "Bad ram offset" using xen stable-4.6

[Fabio Fantoni]

Dom0: Squeeze 64 bit with kernel 4.1.6 and xen stable-4.6 based on 
commit 70d63e48077f8fee8eda6d8d95eeda52a34d9077

DomU W7 pro 64 bit with new pv drivers, xl cfg in attachment.

Qemu crash on windows boot with this line in log:

Bad ram offset 805b1000


xl dmesg about the domU:

(d4) HVM Loader
(d4) Detected Xen v4.6.0-rc
(d4) Xenbus rings @0xfeffc000, event channel 1
(d4) System requested SeaBIOS
(d4) CPU speed is 2660 MHz
(d4) Relocating guest memory for lowmem MMIO space disabled
(XEN) irq.c:275: Dom4 PCI link 0 changed 0 -> 5
(d4) PCI-ISA link 0 routed to IRQ5
(XEN) irq.c:275: Dom4 PCI link 1 changed 0 -> 10
(d4) PCI-ISA link 1 routed to IRQ10
(XEN) irq.c:275: Dom4 PCI link 2 changed 0 -> 11
(d4) PCI-ISA link 2 routed to IRQ11
(XEN) irq.c:275: Dom4 PCI link 3 changed 0 -> 5
(d4) PCI-ISA link 3 routed to IRQ5
(d4) pci dev 01:3 INTA->IRQ10
(d4) pci dev 02:0 INTA->IRQ11
(d4) pci dev 03:0 INTA->IRQ5
(d4) pci dev 04:0 INTA->IRQ5
(d4) pci dev 05:0 INTA->IRQ10
(d4) pci dev 06:0 INTA->IRQ11
(d4) pci dev 07:0 INTA->IRQ5
(d4) No RAM in high memory; setting high_mem resource base to 1
(d4) pci dev 05:0 bar 10 size 00400: 0f000
(d4) pci dev 05:0 bar 14 size 00400: 0f400
(d4) pci dev 02:0 bar 14 size 00100: 0f808
(d4) pci dev 06:0 bar 30 size 4: 0f900
(d4) pci dev 05:0 bar 30 size 1: 0f904
(d4) pci dev 03:0 bar 10 size 04000: 0f905
(d4) pci dev 05:0 bar 18 size 02000: 0f9054000
(d4) pci dev 04:0 bar 14 size 01000: 0f9056000
(d4) pci dev 07:0 bar 24 size 01000: 0f9057000
(d4) pci dev 02:0 bar 10 size 00100: 0c001
(d4) pci dev 06:0 bar 10 size 00100: 0c101
(d4) pci dev 06:0 bar 14 size 00100: 0f9058000
(d4) pci dev 04:0 bar 10 size 00020: 0c201
(d4) pci dev 05:0 bar 1c size 00020: 0c221
(d4) pci dev 07:0 bar 20 size 00020: 0c241
(d4) pci dev 01:1 bar 20 size 00010: 0c261
(d4) Multiprocessor initialisation:
(d4)  - CPU0 ... 36-bit phys ... fixed MTRRs ... var MTRRs [1/8] ... done.
(d4)  - CPU1 ... 36-bit phys ... fixed MTRRs ... var MTRRs [1/8] ... done.
(d4) Writing SMBIOS tables ...
(d4) Loading SeaBIOS ...
(d4) Creating MP tables ...
(d4) Loading ACPI ...
(d4) S3 disabled
(d4) S4 disabled
(d4) vm86 TSS at fc00a100
(d4) BIOS map:
(d4)  1-100d3: Scratch space
(d4)  c-f: Main BIOS
(d4) E820 table:
(d4)  [00]: : - :000a: RAM
(d4)  HOLE: :000a - :000c
(d4)  [01]: :000c - :0010: RESERVED
(d4)  [02]: :0010 - :7800: RAM
(d4)  HOLE: :7800 - :fc00
(d4)  [03]: :fc00 - 0001:: RESERVED
(d4) Invoking SeaBIOS ...
(d4) SeaBIOS (version Debian-1.8.2-1)
(d4)
(d4) Found Xen hypervisor signature at 4100
(d4) Running on QEMU (i440fx)
(d4) xen: copy e820...
(d4) Relocating init from 0x000df450 to 0x77faf480 (size 68288)
(d4) CPU Mhz=2661
(d4) Found 10 PCI devices (max PCI bus is 00)
(d4) Allocated Xen hypercall page at 77fff000
(d4) Detected Xen v4.6.0-rc
(d4) xen: copy BIOS tables...
(d4) Copying SMBIOS entry point from 0x00010010 to 0x000f6780
(d4) Copying MPTABLE from 0xfc001160/fc001170 to 0x000f6680
(d4) Copying PIR from 0x00010030 to 0x000f6600
(d4) Copying ACPI RSDP from 0x000100b0 to 0x000f65d0
(d4) Using pmtimer, ioport 0xb008
(d4) Scan for VGA option rom
(d4) Running option rom at c000:0003
(XEN) stdvga.c:147:d4v0 entering stdvga and caching modes
(d4) pmm call arg1=0
(d4) Turning on vga text mode console
(XEN) stdvga.c:520:d4v0 leaving caching mode
(d4) SeaBIOS (version Debian-1.8.2-1)
(d4) Machine UUID 295a3edd-3e4a-4b40-b3a1-22da17f487d5
(d4) All threads complete.
(d4) Found 0 lpt ports
(d4) Found 0 serial ports
(d4) ATA controller 1 at 1f0/3f4/0 (irq 14 dev 9)
(d4) ATA controller 2 at 170/374/0 (irq 15 dev 9)
(d4) AHCI controller at 07.0, iobase f9057000, irq 5
(d4) Searching bootorder for: /pci@i0cf8/*@7/drive@0/disk@0
(d4) AHCI/0: registering: "AHCI/0: QEMU HARDDISK ATA-7 Hard-Disk 
(5 MiBytes)"

(d4) DVD/CD [ata0-1: QEMU DVD-ROM ATAPI-4 DVD/CD]
(d4) Searching bootorder for: /pci@i0cf8/*@1,1/drive@0/disk@1
(d4) PS2 keyboard initialized
(d4) All threads complete.
(d4) Scan for option roms
(d4) Running option rom at c980:0003
(d4) pmm call arg1=1
(d4) pmm call arg1=0
(d4) pmm call arg1=1
(d4) pmm call arg1=0
(d4) Searching bootorder for: /pci@i0cf8/*@6
(d4)
(d4) Press F12 for boot menu.
(d4)
(d4) Searching bootorder for: HALT
(d4) drive 0x000f6540: PCHS=16383/16/63 translation=lba 
LCHS=1024/255/63 s=10240

(d4)
(d4) Space available for UMB: ca800-ee800, f5fa0-f64e0
(d4) Returned 253952 bytes of ZoneHigh
(d4) e820 map has 6 items:
(d4)   0:  - 0009fc00 = 1 RAM
(d4)   1: 0009fc00 - 000a = 2 RESERVED
(d4)   2: 000f - 0010 = 2 RESERVED
(d4)   3: 0010 - 77ffe000 = 1 RAM
(d4)   4: 77ffe000 - 7800 = 2 RESERVED
(d4)   5: 0

Re: [Xen-devel] [PATCH RFC v2 0/4] HVM x86 deprivileged mode operations

[Fabio Fantoni]

Il 03/09/2015 18:01, Ben Catterall ha scritto:

Hi all,

I have made requested changes and reworked the patch series based on the
comments recieved. Thank you to all of the contributors to those discussions!
The next step will be to provide an example of usage of this system which
will follow in another patch.

The main changes from v1 are:
  - No longer copying the privileged Xen stack but instead change the
interrupt/exception, syscall and sysenter pointers to be below the current
execution point.
  - AMD SVM support
  - Stop IST copying onto the privileged stack
  - Watchdog timer to kill a long running deprivileged domain
  - Support for crashing a domain whilst performing a deprivileged operation
  - .text section is now aliased
  - Assembly updates
  - Updated deprivileged context switching code to fix bugs
  - Moved processor-specific code to processor-specific files
  - Reduction of user stack sizes
  - Updates to interfaces and an is_hvm_deprivileged_mode() style test
  - Small bug fixes
  - Revised performance tests

Many thanks in advance,
Ben

The aim of this work is to create a proof-of-concept to establish if it is
feasible to move certain Xen operations into a deprivileged context to mitigate
the impact of a bug or compromise in such areas. An example would be x86_emulate
or virtual device emulation which is not done in QEMU for performance reasons.


Sorry for my stupid questions:
Is there a test with benchmark using qemu instead for know how is 
different? Qemu seems that emulate also some istructions cases that xen 
hypervisor doesn't for now, or I'm wrong?


Is there any possible hardware technology or set of instructions for 
improve the operations also deprivileged or transition from Xen is 
obliged to control even mappings memory access?
Is there any possible future hardware technology or set of instructions 
for take needed informations from hypervisor for executing directly all 
needed checks, them if ok and any possible exceptions/protections or 
delegate this to xen for each instruction with a tremendous impact on 
the efficiency can not be improved?
If I said only absurd things because of my knowledge too low about sorry 
for having wasted your time.


Thanks for any reply and sorry for my bad english.



Performance testing
---
Performance testing indicates that the overhead for this deprivileged mode
depend heavily upon the processor. This overhead is the cost of moving into
deprivileged mode and then fully back out of deprivileged mode. The conclusions
are that the overheads are not negligible and that operations using this
mechanism would benefit from being long running or be high risk components. It
will need to be evaluated on a case-by-case basis.

I performed 10 writes to a single I/O port on an Intel 2.2GHz Xeon
E5-2407 0 processor and an AMD Opteron 2376. This was done from a python script
within the HVM guest using time.time() and running Debian Jessie. Each write was
trapped to cause a vmexit and the time for each write was calculated. The port
operation is bypassed so that no portio is actually performed. Thus, the
differences in the measurements below can be taken as the pure overhead. These
experiments were repeated. Note that only the host and this HVM guest were
running (both Debian Jessie) during the experiments.

Intel Intel 2.2GHz Xeon E5-2407 0 processor:

1.55e-06 seconds was the average time for performing the write without the
  deprivileged code running.

5.75e-06 seconds was the average time for performing the write with the
  deprivileged code running.

So approximately 351% overhead

AMD Opteron 2376:
-
1.74e-06 seconds was the average time for performing the write without the
  deprivileged code running.
3.10e-06 seconds was the average time for performing the write with an entry and
  exit from deprvileged mode.

So approximately 178% overhead.

Signed-off-by: Ben Catterall 


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Is: Xen4.6.0-rc2+ (origin/smoked) dom0 crashes. Was:Re: Emulation error and calltrace about domUs qemu process testing latest xen from source (git master)

[Fabio Fantoni]

Il 01/09/2015 16:32, Konrad Rzeszutek Wilk ha scritto:

On Tue, Sep 01, 2015 at 03:05:22PM +0200, Fabio Fantoni wrote:

Dom0: wheezy 64 bit with 3.2 kernel from official repository, xen based on
commit 801ab48e5556cb54f67e3cb57f077f47e8663ced.

What previous version of Xen booted with this version of dom0?


The previous xen build was 4.6.0-rc2 but seems that this problem don't 
happen always, rebooting the dom0 and doing other test is not crashed.
I suppose that is a problem that was already present and happen 
occasionally with some specific operation cases, but I don't have 
enought knowledge to debug depth and try solve :(


If you need more informations and tests tell me and I'll post them.

Thanks for any reply and sorry for my bad english.




Thanks.

Booting windows 7 64 bit domU with new pv drivers, domU hang, dom0 show a
calltrace and was a over 30 of load.
The only particular thing is that I added udev files from older xen version
to make possible boot correctly windows domUs with new pv drivers (except
with custom vifname).

 From xl dmesg:
(XEN) mm.c:870:d0v4 pg_owner 0 l1e_owner 0, but real_pg_owner 1
(XEN) mm.c:941:d0v4 Error getting mfn 2c3fd3 (pfn 75dd3) from L1 entry
8002c3fd3767 for l1e_owner=0, pg_owner=0
(XEN) mm.c:5074:d0v4 ptwr_emulate: could not get_page_from_l1e()

Here the calltrace taken from log:
Sep  1 14:25:41 testVS01OU kernel: [  355.798759] BUG: unable to handle
kernel paging request at 880003008da8
Sep  1 14:25:41 testVS01OU kernel: [  355.798892] IP: []
ptep_set_access_flags+0x21/0x45
Sep  1 14:25:41 testVS01OU kernel: [  355.798983] PGD 1606067 PUD 160a067
PMD 3d39067 PTE 801003008065
Sep  1 14:25:41 testVS01OU kernel: [  355.799180] Oops: 0003 [#1] SMP
Sep  1 14:25:41 testVS01OU kernel: [  355.799298] CPU 4
Sep  1 14:25:41 testVS01OU kernel: [  355.799338] Modules linked in:
xt_physdev iptable_filter ip_tables x_tables tun xen_pciback xen_netback
xen_blkback xen_gntalloc xen_gntdev xen_evtchn xenfs ib_iser rdma_cm ib_addr
iw_cm ib_cm ib_sa ib_mad ib_core iscsi_tcp libiscsi_tcp libiscsi
scsi_transport_iscsi nfsd nfs nfs_acl auth_rpcgss fscache lockd sunrpc
bridge stp loop coretemp iTCO_wdt snd_pcm snd_page_alloc snd_timer
i7core_edac edac_core snd soundcore crc32c_intel dcdbas pcspkr joydev evdev
iTCO_vendor_support acpi_power_meter wmi button processor thermal_sys ext4
crc16 jbd2 mbcache dm_mod sd_mod crc_t10dif sg sr_mod cdrom usbhid hid
ata_generic ehci_hcd ata_piix usbcore mpt2sas libata usb_common raid_class
scsi_transport_sas scsi_mod bnx2 [last unloaded: scsi_wait_scan]
Sep  1 14:25:41 testVS01OU kernel: [  355.802032]
Sep  1 14:25:41 testVS01OU kernel: [  355.802068] Pid: 3795, comm:
qemu-system-i38 Not tainted 3.2.0-4-amd64 #1 Debian 3.2.68-1+deb7u3 Dell
Inc. PowerEdge T310/02P9X9
Sep  1 14:25:41 testVS01OU kernel: [  355.802210] RIP:
e030:[]  []
ptep_set_access_flags+0x21/0x45
Sep  1 14:25:41 testVS01OU kernel: [  355.802319] RSP: e02b:880073a33a48
EFLAGS: 00010202
Sep  1 14:25:41 testVS01OU kernel: [  355.802374] RAX: 8002c3fd3701 RBX:
880072801348 RCX: 8002c3fd3767
Sep  1 14:25:41 testVS01OU kernel: [  355.802432] RDX: 880003008da8 RSI:
7ffba1fb5000 RDI: 880072801348
Sep  1 14:25:41 testVS01OU kernel: [  355.802491] RBP: 7ffba1fb5000 R08:
0001 R09: ea0a81f0
Sep  1 14:25:41 testVS01OU kernel: [  355.802549] R10: 880002f2cdc0 R11:
880002f2cdc0 R12: 0001
Sep  1 14:25:41 testVS01OU kernel: [  355.802608] R13: ea0a81c0 R14:
0702 R15: 8800034ee878
Sep  1 14:25:41 testVS01OU kernel: [  355.802668] FS: 7ffbc45eb700()
GS:88007f28() knlGS:
Sep  1 14:25:41 testVS01OU kernel: [  355.802744] CS:  e033 DS:  ES:
 CR0: 8005003b
Sep  1 14:25:41 testVS01OU kernel: [  355.802800] CR2: 880003008da8 CR3:
0375e000 CR4: 2660
Sep  1 14:25:41 testVS01OU kernel: [  355.802859] DR0:  DR1:
 DR2: 
Sep  1 14:25:41 testVS01OU kernel: [  355.802919] DR3:  DR6:
0ff0 DR7: 0400
Sep  1 14:25:41 testVS01OU kernel: [  355.802978] Process qemu-system-i38
(pid: 3795, threadinfo 880073a32000, task 88000303d180)
Sep  1 14:25:41 testVS01OU kernel: [  355.803059] Stack:
Sep  1 14:25:41 testVS01OU kernel: [  355.803109] 00043f6eb067
880072801348 880072801348 880003008da8
Sep  1 14:25:41 testVS01OU kernel: [  355.803305] 7ffba1fb5000
810d2050  00043f6eb067
Sep  1 14:25:41 testVS01OU kernel: [  355.803501] 00043f6eb067
ea0a81f0 8002c3fd3727 8800030c20c0
Sep  1 14:25:41 testVS01OU kernel: [  355.803697] Call Trace:
Sep  1 14:25:41 testVS01OU kernel: [  355.803750] [] ?
handle_pte_fault+0x762/0x7b2
Sep  1 14:25:41 testVS01OU kernel: [  355.803808] [] ?
pmd_val+0x7/0x8
Sep  1 14:25:41 testVS01OU kernel: [  355.803863] [] ?
pte_offset_kernel+0x16/0x

[Xen-devel] Emulation error and calltrace about domUs qemu process testing latest xen from source (git master)

[Fabio Fantoni]

Dom0: wheezy 64 bit with 3.2 kernel from official repository, xen based 
on commit 801ab48e5556cb54f67e3cb57f077f47e8663ced.
Booting windows 7 64 bit domU with new pv drivers, domU hang, dom0 show 
a calltrace and was a over 30 of load.
The only particular thing is that I added udev files from older xen 
version to make possible boot correctly windows domUs with new pv 
drivers (except with custom vifname).


From xl dmesg:
(XEN) mm.c:870:d0v4 pg_owner 0 l1e_owner 0, but real_pg_owner 1
(XEN) mm.c:941:d0v4 Error getting mfn 2c3fd3 (pfn 75dd3) from L1 entry 
8002c3fd3767 for l1e_owner=0, pg_owner=0

(XEN) mm.c:5074:d0v4 ptwr_emulate: could not get_page_from_l1e()

Here the calltrace taken from log:
Sep  1 14:25:41 testVS01OU kernel: [  355.798759] BUG: unable to handle 
kernel paging request at 880003008da8
Sep  1 14:25:41 testVS01OU kernel: [  355.798892] IP: 
[] ptep_set_access_flags+0x21/0x45
Sep  1 14:25:41 testVS01OU kernel: [  355.798983] PGD 1606067 PUD 
160a067 PMD 3d39067 PTE 801003008065

Sep  1 14:25:41 testVS01OU kernel: [  355.799180] Oops: 0003 [#1] SMP
Sep  1 14:25:41 testVS01OU kernel: [  355.799298] CPU 4
Sep  1 14:25:41 testVS01OU kernel: [  355.799338] Modules linked in: 
xt_physdev iptable_filter ip_tables x_tables tun xen_pciback xen_netback 
xen_blkback xen_gntalloc xen_gntdev xen_evtchn xenfs ib_iser rdma_cm 
ib_addr iw_cm ib_cm ib_sa ib_mad ib_core iscsi_tcp libiscsi_tcp libiscsi 
scsi_transport_iscsi nfsd nfs nfs_acl auth_rpcgss fscache lockd sunrpc 
bridge stp loop coretemp iTCO_wdt snd_pcm snd_page_alloc snd_timer 
i7core_edac edac_core snd soundcore crc32c_intel dcdbas pcspkr joydev 
evdev iTCO_vendor_support acpi_power_meter wmi button processor 
thermal_sys ext4 crc16 jbd2 mbcache dm_mod sd_mod crc_t10dif sg sr_mod 
cdrom usbhid hid ata_generic ehci_hcd ata_piix usbcore mpt2sas libata 
usb_common raid_class scsi_transport_sas scsi_mod bnx2 [last unloaded: 
scsi_wait_scan]

Sep  1 14:25:41 testVS01OU kernel: [  355.802032]
Sep  1 14:25:41 testVS01OU kernel: [  355.802068] Pid: 3795, comm: 
qemu-system-i38 Not tainted 3.2.0-4-amd64 #1 Debian 3.2.68-1+deb7u3 Dell 
Inc. PowerEdge T310/02P9X9
Sep  1 14:25:41 testVS01OU kernel: [  355.802210] RIP: 
e030:[]  [] 
ptep_set_access_flags+0x21/0x45
Sep  1 14:25:41 testVS01OU kernel: [  355.802319] RSP: 
e02b:880073a33a48  EFLAGS: 00010202
Sep  1 14:25:41 testVS01OU kernel: [  355.802374] RAX: 8002c3fd3701 
RBX: 880072801348 RCX: 8002c3fd3767
Sep  1 14:25:41 testVS01OU kernel: [  355.802432] RDX: 880003008da8 
RSI: 7ffba1fb5000 RDI: 880072801348
Sep  1 14:25:41 testVS01OU kernel: [  355.802491] RBP: 7ffba1fb5000 
R08: 0001 R09: ea0a81f0
Sep  1 14:25:41 testVS01OU kernel: [  355.802549] R10: 880002f2cdc0 
R11: 880002f2cdc0 R12: 0001
Sep  1 14:25:41 testVS01OU kernel: [  355.802608] R13: ea0a81c0 
R14: 0702 R15: 8800034ee878
Sep  1 14:25:41 testVS01OU kernel: [  355.802668] FS: 
7ffbc45eb700() GS:88007f28() knlGS:
Sep  1 14:25:41 testVS01OU kernel: [  355.802744] CS:  e033 DS:  ES: 
 CR0: 8005003b
Sep  1 14:25:41 testVS01OU kernel: [  355.802800] CR2: 880003008da8 
CR3: 0375e000 CR4: 2660
Sep  1 14:25:41 testVS01OU kernel: [  355.802859] DR0:  
DR1:  DR2: 
Sep  1 14:25:41 testVS01OU kernel: [  355.802919] DR3:  
DR6: 0ff0 DR7: 0400
Sep  1 14:25:41 testVS01OU kernel: [  355.802978] Process 
qemu-system-i38 (pid: 3795, threadinfo 880073a32000, task 
88000303d180)

Sep  1 14:25:41 testVS01OU kernel: [  355.803059] Stack:
Sep  1 14:25:41 testVS01OU kernel: [  355.803109] 00043f6eb067 
880072801348 880072801348 880003008da8
Sep  1 14:25:41 testVS01OU kernel: [  355.803305] 7ffba1fb5000 
810d2050  00043f6eb067
Sep  1 14:25:41 testVS01OU kernel: [  355.803501] 00043f6eb067 
ea0a81f0 8002c3fd3727 8800030c20c0

Sep  1 14:25:41 testVS01OU kernel: [  355.803697] Call Trace:
Sep  1 14:25:41 testVS01OU kernel: [  355.803750] [] ? 
handle_pte_fault+0x762/0x7b2
Sep  1 14:25:41 testVS01OU kernel: [  355.803808] [] ? 
pmd_val+0x7/0x8
Sep  1 14:25:41 testVS01OU kernel: [  355.803863] [] ? 
pte_offset_kernel+0x16/0x35
Sep  1 14:25:41 testVS01OU kernel: [  355.803920] [] ? 
do_page_fault+0x320/0x345
Sep  1 14:25:41 testVS01OU kernel: [  355.803979] [] ? 
xen_write_msr_safe+0x73/0xb9
Sep  1 14:25:41 testVS01OU kernel: [  355.804036] [] ? 
xen_end_context_switch+0xe/0x1c
Sep  1 14:25:41 testVS01OU kernel: [  355.804094] [] ? 
xen_mc_issue.constprop.23+0x31/0x49
Sep  1 14:25:41 testVS01OU kernel: [  355.804153] [] ? 
paravirt_write_msr+0xb/0xe
Sep  1 14:25:41 testVS01OU kernel: [  355.804211] [] ? 
__switch_to+0x18e/0x265
Sep  1 14:25:41 testVS01OU kernel: [  355.804268] [] ? 
pvclock_clocksource_read+0x42/0xb2
Sep  1 14:25

Re: [Xen-devel] [linux-3.14 bisection] complete test-amd64-i386-xl-qcow2

[Fabio Fantoni]

Il 01/09/2015 11:19, Ian Campbell ha scritto:

On Wed, 2015-08-26 at 20:02 +, osstest service owner wrote:

branch xen-unstable
xen branch xen-unstable
job test-amd64-i386-xl-qcow2
test guest-saverestore

Tree: linux 
git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: qemu git://xenbits.xen.org/staging/qemu-xen-unstable.git
Tree: qemuu git://xenbits.xen.org/staging/qemu-upstream-unstable.git
Tree: xen git://xenbits.xen.org/xen.git

*** Found and reproduced problem changeset ***

   Bug is in tree:  linux 
git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
   Bug introduced:  9e6c072a69d87100808d16279d60e9f857291340
   Bug not present: b60d3eee854b881b3f7a478c8b622cbef72d4c4e


   commit 9e6c072a69d87100808d16279d60e9f857291340
   Author: Marek Marczykowski-Górecki 
   Date:   Fri Jun 26 03:28:24 2015 +0200
   
   xen/gntdevt: Fix race condition in gntdev_release()

I'm not sure what to make of this.

The qcow2 test is one of the only ones I'd expect to be exercising gntdev
(most tests use LVM+blkback), which explains why this particular commit is
apparently seeing issues due to this particular change.

 From these ...

http://logs.test-lab.xenproject.org/osstest/results/history/test-amd64-i386-xl-qcow2/linux-3.14.html
http://logs.test-lab.xenproject.org/osstest/results/history/test-amd64-i386-xl-qcow2/linux-3.16.html
http://logs.test-lab.xenproject.org/osstest/results/history/test-amd64-i386-xl-qcow2/linux-3.18.html
http://logs.test-lab.xenproject.org/osstest/results/history/test-amd64-i386-xl-qcow2/linux-4.1.html
http://logs.test-lab.xenproject.org/osstest/results/history/test-amd64-i386-xl-qcow2/linux-linus.html

... it seems only 3.14 is hit, I wonder why?

Maybe the other stable trees haven't gotten this yet (or osstest hasn't
gotten to a revision with it) but linux-linus certainly will and it appears
to be fine there. Perhaps a mis-applied/behaving backport or a missing
associated patch?

Flight 60952 on linux-linus was on huxelrebe0, contained 30b03d05e0746 and
passed. That is the same h/w as huxelrebe1 which is failing (sticky) on
3.14. Also 60785 on linux-4.1 was on huxelrebe1 itself and contains the
backport, and was fine. So I conclude it is probably not a h/w specific
issue.

Ian.


Hi, recently I tested qcow2 overlay on raw base as domUs disks (qcow2 
format in xl already specified) and I found that disk was always 
corrupted (without nothing about in logs).
I tried with diffent versions of upstream qemu (2.2 and 2.4) and dom0 
kernel (3.2 from wheezy repo and 3.16 from wheezy-backports), maybe also 
related to this?


Thanks for any reply and sorry for my bad english.

   
   commit 30b03d05e07467b8c6ec683ea96b5bffcbcd3931 upstream.
   
   While gntdev_release() is called the MMU notifier is still registered

   and can traverse priv->maps list even if no pages are mapped (which is
   the case -- gntdev_release() is called after all). But
   gntdev_release() will clear that list, so make sure that only one of
   those things happens at the same time.
   
   Signed-off-by: Marek Marczykowski-Górecki 

   Signed-off-by: David Vrabel 
   Signed-off-by: Greg Kroah-Hartman 


For bisection revision-tuple graph see:

http://logs.test-lab.xenproject.org/osstest/results/bisect/linux-3.14/test-amd64-i386-xl-qcow2.guest-saverestore.html
Revision IDs in each graph node refer, respectively, to the Trees above.


Searching for failure / basis pass:
  60823 fail [host=huxelrebe1] / 60666 [host=chardonnay1] 60655 [host=elbling1] 
60549 [host=chardonnay0] template as basis? using template as basis.
Failure / basis pass flights: 60823 / 60666
(tree with no url: ovmf)
(tree with no url: seabios)
Tree: linux 
git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: qemu git://xenbits.xen.org/staging/qemu-xen-unstable.git
Tree: qemuu git://xenbits.xen.org/staging/qemu-upstream-unstable.git
Tree: xen git://xenbits.xen.org/xen.git
Latest 318ff69ca4c275bae4b875b87df5bdbd7988486a 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
7f057440b31da38196e3398fd1b618fc36ad97d6 
bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 
145a8004a7d659668d5a3b0ad9868d7678b24822
Basis pass 9b8b905951bde404f20a7bd4b37a5134f3484569 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
7f057440b31da38196e3398fd1b618fc36ad97d6 
bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 
201eac83831d94ba2e9a63a7eed4c128633fafb1
Generating revisions with ./adhoc-revtuple-generator  
git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git#9b8b905951bde404f20a7bd4b37a5134f3484569-318ff69ca4c275bae4b875b87df5bdbd7988486a
 
git://xenbits.xen.org/osstest/linux-firmware.git#c530a75c1e6a472b0eb9558310b518f0dfcd8860-c530a75c1e6a472b0eb9558310b518f0dfcd8860
 
git://xenbits.xen.org/sta

[Xen-devel] Improve or change devel mailing-list for improve efficiency and spending less time, is this possible?

[Fabio Fantoni]

I saw some discussions about the developer cycle, patch review and how 
to improve.


Based on what I saw an important thing for improve efficiency and 
spending less time is the way of communication, actual seems only 
mailing-list/mails.


It seems to me that currently has some limits and in many cases takes a 
long time for some specific searches,trouble finding patches that need 
attention etc...


A good solution I think would be a forum specific for developing with 
these features:
- light even if with many specific features useful for developing and 
compatible with all browser and devices
- have all as forum discussion that is easier to search, access to all 
post etc... now the search is only via web for month (losing link in 
different months) and with mail (if storing thousands of emails per month)
- possibility to include group or single users (for example maintainers) 
to a discussion (also notify with mails based on specific destination 
users settings), better that send mails, faster for search, will avoid 
also problem of mails not received, mail address changed etc...
- good patch "integration", possibility to include/see/download patches, 
raw vision (like the patch sent now with mail with git), add of special 
tags etc... This is probably the more difficult thing to do because need 
to find the best way to manage well the patches, including series and 
ultimately have a better solution to track, review, comment, etc...
- optionally automatic include maintainer for patch posted, probably 
after loading it on server scanning with a similar to 
scripts/get_maintainer.pl
- advanced searching including if is a normal discussion or patch, if 
the user is present on notify list, selection of all tags, for example 
for search in patch that are fix or features, is the discussion/patch is 
without reply or with reply older that specified range (faster search of 
patch without review, or without recent reply), is closed (about patch 
if merged) or not



I did a fast internet search about without find a similar solution, the 
only near thing is probably forum with some specific extension (if exist).
There are some solutions with part of these goals but lacking in some 
important, for example github.


What do you think about this?


Thanks for any reply and sorry for my bad english.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] xen 4.6 regression: xl create fails if domU have custom vifname

[Fabio Fantoni]

Il 27/08/2015 16:46, Wei Liu ha scritto:

On Thu, Aug 27, 2015 at 04:41:46PM +0200, Fabio Fantoni wrote:

Today trying xen 4.6.0-rc2 with all things needed for future production
server I found a regression: xl create fails if domU have custom vifname.
xl create test.cfg
Parsing config from test.cfg
libxl: error: libxl_exec.c:118:libxl_report_child_exitstatus:
/etc/xen/scripts/vif-bridge add [14581] exited with error status 1
libxl: error: libxl_device.c:1085:device_hotplug_child_death_cb: script: ip
link set vif14.0-emu name frc-0-0-emu failed

This is the culprit.

Did you do a clean install?

You need to remove udev files in /etc. I should have mentioned this in
testing instructions.

Wei.


Sorry for my mistake I didn't remember to check /etc because the testing 
deb (with make debball) delete /etc files only if specified as 
considered config files.

Removing the udev file solves the problem.
Thanks.




libxl: error: libxl_create.c:1380:domcreate_attach_vtpms: unable to add nic
devices
libxl: error: libxl_exec.c:118:libxl_report_child_exitstatus:
/etc/xen/scripts/block remove [14615] exited with error status 1
libxl: error: libxl_device.c:1085:device_hotplug_child_death_cb: script:
/etc/xen/scripts/block failed; error detected.
libxl: error: libxl.c:1580:libxl__destroy_domid: non-existant domain 14
libxl: error: libxl.c:1538:domain_destroy_callback: unable to destroy guest
with domid 14
libxl: error: libxl.c:1465:domain_destroy_cb: destruction of domain 14
failed

 From domU xl cfg:
vif=['bridge=xenbr0,mac=00:16:3e:dd:e3:f1,vifname=frc-0-0']

Same domU removing only vifname is working.

On production server with xen 4.4.2 or 4.5.1 domUs with vifname are working.

I did a fast search without found the exact bug/regression cause.

If you need more informations/tests tell me and I'll post them.

Thanks for any reply and sorry for my bad english.



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] xen 4.6 regression: xl create fails if domU have custom vifname

[Fabio Fantoni]

Today trying xen 4.6.0-rc2 with all things needed for future production 
server I found a regression: xl create fails if domU have custom vifname.

xl create test.cfg
Parsing config from test.cfg
libxl: error: libxl_exec.c:118:libxl_report_child_exitstatus: 
/etc/xen/scripts/vif-bridge add [14581] exited with error status 1
libxl: error: libxl_device.c:1085:device_hotplug_child_death_cb: script: 
ip link set vif14.0-emu name frc-0-0-emu failed
libxl: error: libxl_create.c:1380:domcreate_attach_vtpms: unable to add 
nic devices
libxl: error: libxl_exec.c:118:libxl_report_child_exitstatus: 
/etc/xen/scripts/block remove [14615] exited with error status 1
libxl: error: libxl_device.c:1085:device_hotplug_child_death_cb: script: 
/etc/xen/scripts/block failed; error detected.

libxl: error: libxl.c:1580:libxl__destroy_domid: non-existant domain 14
libxl: error: libxl.c:1538:domain_destroy_callback: unable to destroy 
guest with domid 14
libxl: error: libxl.c:1465:domain_destroy_cb: destruction of domain 14 
failed


From domU xl cfg:
vif=['bridge=xenbr0,mac=00:16:3e:dd:e3:f1,vifname=frc-0-0']

Same domU removing only vifname is working.

On production server with xen 4.4.2 or 4.5.1 domUs with vifname are working.

I did a fast search without found the exact bug/regression cause.

If you need more informations/tests tell me and I'll post them.

Thanks for any reply and sorry for my bad english.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [xen 4.6 retrospective] More public/easy to find information about the release schedule

[Fabio Fantoni]

Il 10/08/2015 11:06, Lars Kurth ha scritto:

On 10 Aug 2015, at 09:33, Wei Liu  wrote:

On Fri, Aug 07, 2015 at 05:36:57PM +0200, Roger Pau Monné wrote:

= Issue / Observation =

The information about the release schedule is not clearly published
anywhere apart from the mailing lists, which makes it hard for
non-developers (or even for developers) given that the mailing list
traffic for xen-devel is high.

This is not entirely true: see 
http://wiki.xenproject.org/wiki/Xen_Project_Hypervisor_Roadmap/4.6
Hi, I take a look to the wiki page, can be good mention also the "add of 
ahci disk controller support for hvm domUs"? I saw that is missed in 
features list.

However, I think https://www.freebsd.org/releng/ and also the odd mail on 
announce@ would make sense

Lars
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] Status of HVM improvements, support of recent technologies and request for some tips

[Fabio Fantoni]

Some years ago the support/improvement of hvm domUsseemed neglected, 
recently seems to be good.


Qemu upstream when I started using early versions from the 1.2 was 
neglected but now it seems early to be quite used, tested and 
considered, thanks to all those who are working to improve support for xen.


New windows pv driversseems to be well alignedwith several improvements.
I found many bugs at the beginning but thanks to Paul Durrant have been 
resolved quickly.

Will be an official xen project build withsigned drivers out there?

OVMF (eufi firmware for hvm domUs) support some years ago was bad but 
now with contribution of Anthony Perard and Wei Lui seems usable with 
windows and linux domUs.
On my latest fast tests I found that boot option is not working or not 
supported for now as I reported in another mail.

Should I report it to ovmf upstream?
In boot tests I saw also floppy test that should be not present based on 
qemu settings: is it only a boot test or is there a bug about it?
I'll test it further and I'll compare with seabios about some small 
strange things I found.


Actuallywhere a system binaries are specified they will not update 
automatically hvmloader built-in seabios and ovmf firmware but require 
xen rebuild (probably major for production system from distro build).
I saw in wiki project "Improvements to firmware handling HVM guests" 
that include also it, is there any news about?


Another probably good thing should be unified eufi and bios support in 
hvmloader.
I saw in wiki project "OVMF Compatibility Support Module support in 
Xen", is there any news about?



Another interesting thing is Q35 support.
Now we are still using very old emulated hardware based on I440FX emulation.
Q35 adds emulation for the ICH9 host chipset, PCI-E bus by default, ahci 
disk controller by default and other things probably useful on recent 
systems.
I did a fast tests with it on xen 2 years ago but I was blocked by disk 
not visible that was require new qemu parameters and that had a bug about.
With a basic ahci support in libxl (included in 4.6) and another patch 
to convert other disk cases (I'll keep it tested and I'll repost for 
4.7) seems that now is possible use new qemu parameters for disks, and 
I'll now retry Q35 on xen.
I'll probably start to write some libxl patches about and I have some 
questions:


Have someone already start to do it but without posting the patches for now?
About xl parameter for chipset selection what is the correct parameter 
to add? device_model_machine="q35|i440fx" (default="i440fx") can be good?
I did a fast look to in libxl_dm.c and I don't know what to do for q35 
case of xen_platform_pci=0. Actually "-machine xenfv" is used but it 
seems only for old chipset case. Any idea/advice?
AHCI disks controller is the default built-in in q35 instead of ide one 
for old chipset. Is it correct to force hdtype="ahci" with q35 and add a 
warning if user selects hdtype="ide"?
If I remember well, is old pci bridge still needed 
("-device","i82801b11-bridge") for emulated vgas support along other 
things or did something changed in the meanwhile?
About hvmloader also this time I'll remove an assert (if I remember 
good) to do q35 tests but a better change is needed and I don't know about.

Any help is appreciated.


Thanks for any reply and sorry for my bad english.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] unable to boot from cdrom using OVMF in latest xen-unstable

[Fabio Fantoni]

Il 29/07/2015 13:20, Wei Liu ha scritto:

On Wed, Jul 29, 2015 at 01:15:24PM +0200, Fabio Fantoni wrote:
[...]

(d14) BIOS map:
(d14)  ffe0-: Main BIOS
(d14) E820 table:
(d14)  [00]: : - :000a: RAM
(d14)  HOLE: :000a - :000f
(d14)  [01]: :000f - :0010: RESERVED
(d14)  [02]: :0010 - :7eead000: RAM
(d14)  HOLE: :7eead000 - :fc00
(d14)  [03]: :fc00 - 0001:: RESERVED
(d14) Invoking OVMF ...
(XEN) irq.c:276: Dom14 PCI link 0 changed 5 -> 11
(XEN) irq.c:276: Dom14 PCI link 1 changed 10 -> 11
(XEN) irq.c:276: Dom14 PCI link 2 changed 11 -> 10
(XEN) irq.c:276: Dom14 PCI link 3 changed 5 -> 10

The strange thing is that I not found lines "Boot from" similar to domUs
with seabios instead, for example a w10 domU, seabios and boot='dc' and with
empty cdrom have also these lines:

Right. That's because OVMF doesn't write to the correct port. Sigh.

It requires recompiling OVMF to make it write to serial console, which
is probably too much trouble for you.

Another thing to try is to use VNC to connect to your DomU to see if
there is anything useful there.

Wei.
I use spice instead, output should be same even if quality and features 
is better.
I remembered that ovmf from debian packages is compiled without debug, I 
recompiled with ovmf included in xen-unstable instead.

After on first test I saw that boot correctly from cdrom.
I did other tests after remember that with serial='pty' I need to look 
qemu log, see the port and do a cat on it (sorry for lost time 
previously for my bad memory).
Without debug is too fast on boot is not possible but with this with 
debug yes.
With ide (default) and boot='c' do strange things, boot from cdrom and 
domU's serial output is:

cat /dev/pts/5
3h
3h
With hdtype='ahci' printed the output correctly but what did in boot is 
still strange:

cat /dev/pts/4
Boot Failed. EFI Floppy
Boot Failed. EFI Floppy 1
Booting in insecure mode
System BootOrder not found.  Initializing defaults.
device path: 
"Acpi(PNP0A03,0)/Pci(2|0)/VenHw(3D3CA290-B9A5-11E3-B75D-B8AC6F7D65E6)/HD(Part1,SigF5D503AE-FD24-4E28-8D1B-8D0A2F97F0BC)/\EFI\fedora\shim.efi"
Creating boot entry "Boot0005" with label "Fedora" for file 
"\EFI\fedora\shim.efi"

Booting in insecure mode
based on qemu configuration floppy should be not present and about the 
boot order the strange thing is "System BootOrder not found. 
Initializing defaults."


if you need more informations/tests tell me and I'll post them.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] unable to boot from cdrom using OVMF in latest xen-unstable

[Fabio Fantoni]

Il 29/07/2015 13:04, Wei Liu ha scritto:

On Wed, Jul 29, 2015 at 12:59:04PM +0200, Fabio Fantoni wrote:

Il 29/07/2015 12:44, Wei Liu ha scritto:

On Wed, Jul 29, 2015 at 12:42:41PM +0200, Fabio Fantoni wrote:

Il 29/07/2015 11:58, Wei Liu ha scritto:

On Wed, Jul 29, 2015 at 11:31:30AM +0200, Fabio Fantoni wrote:

Today I retried ovmf after long time from previous, I tried old fedora 20
uefi domUs and still boot correctly but trying to boot from cdrom using
boot='dc' or boot='d' don't boot from cdrom (seems didn't try) but boot from
harddisk (even with boot='d').
After booting fedora I checked the cdrom and see it correctly, the problem
seems only about boot order.
xen is build from staging git based on commit
44313ab77f3e3c5b566ea4f23b0e32bfd5eafa29 and ovmf is from debian package
0~20150106.5c2d456b-1.

If you need more informations/tests tell me and I'll post them.

Thanks for any reply and sorry for my bad english.

OK, the commit you tested contains the fix to booting OVMF.

Do you have serial log?

Wei.

You mean ovmf serial log? probably I did it in past but I not remember how
and with a fast search I not found (probably was some years ago when I
started to try ovmf and initially with xen domUs was not working at all)
I found only serial="pty" commented in my tests domUs cfg probably related
but I don't remember how to see/store the output, can you tell me it please?

I meant Xen serial log. And you probably need to increase your log level
in Xen command line.

Wei.

Sorry for my stupid question, what xen serial log you mean? serial of dom0
for debug domU's internal boot seems very strange to me.


Because firmware (in this case, OVMF) writes something to console as
guest output.


this is my actual dom0 grub entry is:

menuentry 'RAID - Debian 7.8 (wheezy) con Linux 3.16.0-0.bpo.4-amd64 e XEN
4.6-unstable' --class debian --class gnu-linux --class gnu --class os
--class xen {
set fallback="1"
set root='(RAID-ROOT)'
echo'Caricamento Hypervisor Xen 4.6-unstable...'
multiboot   /boot/xen.gz placeholder dom0_mem=2G,max:3G
echo'Caricamento Linux 3.16.0-0.bpo.4-amd64 ...'
module  /boot/vmlinuz-3.16.0-0.bpo.4-amd64 placeholder
root=/dev/mapper/RAID-ROOT ro swiotlb=65762 quiet
echo'Caricamento ramdisk iniziale...'
module  /boot/initrd.img-3.16.0-0.bpo.4-amd64
}

this is an old entry I used for debug dom0 boot problem:

menuentry 'Wheezy con Linux 3.2.0-4-amd64 e XEN 4.4 - RAID - Debug su
Seriale' --class debian --class gnu-linux --class gnu --class os {
set root='(RAID-ROOT)'
echo'Caricamento Hypervisor Xen 4.4...'
multiboot/boot/xen-4.4-amd64.gz placeholder dom0_mem=2G,max:2G
swiotlb=65762 loglvl=all guest_loglvl=all sync_console console_to_ring
com2=19200,8n1 console=com2
echo'Caricamento Linux 3.2.0-4-amd64...'
linux/boot/vmlinuz-3.2.0-4-amd64 placeholder
root=/dev/mapper/RAID-ROOT ro console=hvc0 earlyprintk=xen nomodeset
echo'Caricamento ramdisk iniziale...'
initrd/boot/initrd.img-3.2.0-4-amd64
}

About dom0 you mean add only "loglvl=all guest_loglvl=all" or also other
thing?
And about domU?


I think your configuration is alright.

Actually you can use xl dmesg to get the log. The interesting part is
hvmloader information.

Wei.


Thanks for any reply.



Here the xl dmesg about fedora with ovmf:

(d14) HVM Loader
(d14) Detected Xen v4.6-unstable
(d14) Xenbus rings @0xfeffc000, event channel 1
(d14) System requested OVMF
(d14) CPU speed is 2660 MHz
(d14) Relocating guest memory for lowmem MMIO space disabled
(XEN) irq.c:276: Dom14 PCI link 0 changed 0 -> 5
(d14) PCI-ISA link 0 routed to IRQ5
(XEN) irq.c:276: Dom14 PCI link 1 changed 0 -> 10
(d14) PCI-ISA link 1 routed to IRQ10
(XEN) irq.c:276: Dom14 PCI link 2 changed 0 -> 11
(d14) PCI-ISA link 2 routed to IRQ11
(XEN) irq.c:276: Dom14 PCI link 3 changed 0 -> 5
(d14) PCI-ISA link 3 routed to IRQ5
(d14) pci dev 01:3 INTA->IRQ10
(d14) pci dev 02:0 INTA->IRQ11
(d14) pci dev 03:0 INTA->IRQ5
(d14) pci dev 04:0 INTA->IRQ5
(d14) pci dev 06:0 INTA->IRQ11
(d14) pci dev 1d:0 INTA->IRQ10
(d14) pci dev 1d:1 INTB->IRQ11
(d14) pci dev 1d:2 INTC->IRQ5
(d14) pci dev 1d:7 INTD->IRQ5
(d14) No RAM in high memory; setting high_mem resource base to 1
(d14) pci dev 02:0 bar 14 size 00100: 0f008
(d14) pci dev 05:0 bar 10 size 00100: 0f108
(d14) pci dev 06:0 bar 30 size 4: 0f200
(d14) pci dev 05:0 bar 30 size 1: 0f204
(d14) pci dev 03:0 bar 10 size 04000: 0f205
(d14) pci dev 04:0 bar 14 size 01000: 0f2054000
(d14) pci dev 05:0 bar 18 size 01000: 0f2055000
(d14) pci dev 1d:7 bar 10 size 01000: 0f2056000
(d14) pci dev 02:0 bar 10 size 00100: 0c001
(d

Re: [Xen-devel] unable to boot from cdrom using OVMF in latest xen-unstable

[Fabio Fantoni]

Il 29/07/2015 12:44, Wei Liu ha scritto:

On Wed, Jul 29, 2015 at 12:42:41PM +0200, Fabio Fantoni wrote:

Il 29/07/2015 11:58, Wei Liu ha scritto:

On Wed, Jul 29, 2015 at 11:31:30AM +0200, Fabio Fantoni wrote:

Today I retried ovmf after long time from previous, I tried old fedora 20
uefi domUs and still boot correctly but trying to boot from cdrom using
boot='dc' or boot='d' don't boot from cdrom (seems didn't try) but boot from
harddisk (even with boot='d').
After booting fedora I checked the cdrom and see it correctly, the problem
seems only about boot order.
xen is build from staging git based on commit
44313ab77f3e3c5b566ea4f23b0e32bfd5eafa29 and ovmf is from debian package
0~20150106.5c2d456b-1.

If you need more informations/tests tell me and I'll post them.

Thanks for any reply and sorry for my bad english.

OK, the commit you tested contains the fix to booting OVMF.

Do you have serial log?

Wei.

You mean ovmf serial log? probably I did it in past but I not remember how
and with a fast search I not found (probably was some years ago when I
started to try ovmf and initially with xen domUs was not working at all)
I found only serial="pty" commented in my tests domUs cfg probably related
but I don't remember how to see/store the output, can you tell me it please?

I meant Xen serial log. And you probably need to increase your log level
in Xen command line.

Wei.
Sorry for my stupid question, what xen serial log you mean? serial of 
dom0 for debug domU's internal boot seems very strange to me.


this is my actual dom0 grub entry is:
menuentry 'RAID - Debian 7.8 (wheezy) con Linux 3.16.0-0.bpo.4-amd64 e 
XEN 4.6-unstable' --class debian --class gnu-linux --class gnu --class 
os --class xen {

set fallback="1"
set root='(RAID-ROOT)'
echo'Caricamento Hypervisor Xen 4.6-unstable...'
multiboot   /boot/xen.gz placeholder dom0_mem=2G,max:3G
echo'Caricamento Linux 3.16.0-0.bpo.4-amd64 ...'
module  /boot/vmlinuz-3.16.0-0.bpo.4-amd64 placeholder 
root=/dev/mapper/RAID-ROOT ro swiotlb=65762 quiet

echo'Caricamento ramdisk iniziale...'
module  /boot/initrd.img-3.16.0-0.bpo.4-amd64
}


this is an old entry I used for debug dom0 boot problem:
menuentry 'Wheezy con Linux 3.2.0-4-amd64 e XEN 4.4 - RAID - Debug su 
Seriale' --class debian --class gnu-linux --class gnu --class os {

set root='(RAID-ROOT)'
echo'Caricamento Hypervisor Xen 4.4...'
multiboot/boot/xen-4.4-amd64.gz placeholder dom0_mem=2G,max:2G 
swiotlb=65762 loglvl=all guest_loglvl=all sync_console console_to_ring 
com2=19200,8n1 console=com2

echo'Caricamento Linux 3.2.0-4-amd64...'
linux/boot/vmlinuz-3.2.0-4-amd64 placeholder 
root=/dev/mapper/RAID-ROOT ro console=hvc0 earlyprintk=xen nomodeset

echo'Caricamento ramdisk iniziale...'
initrd/boot/initrd.img-3.2.0-4-amd64
}


About dom0 you mean add only "loglvl=all guest_loglvl=all" or also other 
thing?

And about domU?

Thanks for any reply.



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] unable to boot from cdrom using OVMF in latest xen-unstable

[Fabio Fantoni]

Il 29/07/2015 11:58, Wei Liu ha scritto:

On Wed, Jul 29, 2015 at 11:31:30AM +0200, Fabio Fantoni wrote:

Today I retried ovmf after long time from previous, I tried old fedora 20
uefi domUs and still boot correctly but trying to boot from cdrom using
boot='dc' or boot='d' don't boot from cdrom (seems didn't try) but boot from
harddisk (even with boot='d').
After booting fedora I checked the cdrom and see it correctly, the problem
seems only about boot order.
xen is build from staging git based on commit
44313ab77f3e3c5b566ea4f23b0e32bfd5eafa29 and ovmf is from debian package
0~20150106.5c2d456b-1.

If you need more informations/tests tell me and I'll post them.

Thanks for any reply and sorry for my bad english.

OK, the commit you tested contains the fix to booting OVMF.

Do you have serial log?

Wei.
You mean ovmf serial log? probably I did it in past but I not remember 
how and with a fast search I not found (probably was some years ago when 
I started to try ovmf and initially with xen domUs was not working at all)
I found only serial="pty" commented in my tests domUs cfg probably 
related but I don't remember how to see/store the output, can you tell 
me it please?


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] unable to boot from cdrom using OVMF in latest xen-unstable

[Fabio Fantoni]

Il 29/07/2015 11:40, Wei Liu ha scritto:

On Wed, Jul 29, 2015 at 11:31:30AM +0200, Fabio Fantoni wrote:

Today I retried ovmf after long time from previous, I tried old fedora 20
uefi domUs and still boot correctly but trying to boot from cdrom using
boot='dc' or boot='d' don't boot from cdrom (seems didn't try) but boot from
harddisk (even with boot='d').
After booting fedora I checked the cdrom and see it correctly, the problem
seems only about boot order.
xen is build from staging git based on commit
44313ab77f3e3c5b566ea4f23b0e32bfd5eafa29 and ovmf is from debian package
0~20150106.5c2d456b-1.

If you need more informations/tests tell me and I'll post them.

Thanks for any reply and sorry for my bad english.

What has changed in your setup? I guess it's only Xen? Do you use the
same OVMF image all the time?

If the only thing changed is Xen, I would suggest you bisect the
hypervisor. There were a bunch of changes in emulation code there.
If you don't have a base line version, I will try to come up with one
for you.

Wei.
Thanks for reply, I did other tests and I found that having empty 
harddrive boot from cdrom correctly also with omvf, the problem seems 
that not cosider boot option at all.
I did also few tests about ovmf and if I remember good is the first time 
I try to boot from cdrom on domUs with harddrive not empty, other time I 
did it only for system install, then is probable that the is present 
also in older xen and ovmf versions.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v5 0/6] libxl: xs_restrict QEMU

[Fabio Fantoni]

Il 29/07/2015 11:21, Stefano Stabellini ha scritto:

On Mon, 27 Jul 2015, Fabio Fantoni wrote:

Il 23/07/2015 19:26, Stefano Stabellini ha scritto:

Hi all,

this patch series changes libxl to start QEMU as device model with the
new xsrestrict option (http://marc.info/?l=xen-devel&m=143341692707358).
It also starts a second QEMU to provide PV backends in userspace (qdisk)
to HVM guests.

Hi, I'm interested to test this serie.
xen patch "run QEMU as non-root" and qemu patch linked above are the only
prerequisite or other are needed?

Yes and thank you for testing!



I saw that second patch is marked as [WIP], is it usable or I must wait to
have it complete before test this serie?

It is usable, but it is not recommended why to do it in libxl.


Thanks for reply, what do you mean about "but it is not recommended why 
to do it in libxl"?






Thanks for any reply and sorry for my bad english.



Changes in v5:
- improve commit messages with security details

Changes in v4:
- update xenstore-paths.markdown
- add error message in case count > MAX_PHYSMAP_ENTRIES
- add a note to xenstore-paths.markdown about the possible change in
privilege level
- only change permissions if xsrestrict is supported

Changes in v3:
- use LIBXL_TOOLSTACK_DOMID instead of 0 in the commit message
- update commit message with more info on why it is safe
- add a limit on the number of physmap entries to save and restore
- add emulator_ids
- mark patch #3 as WIP
- use LIBXL_TOOLSTACK_DOMID instead of 0 in the commit message
- change xs path to include the emulator_id
- change qdisk-backend-pid path on xenstore
- use dcs->dmss.pvqemu to spawn the second QEMU
- keep track of the rc of both QEMUs before proceeding


Stefano Stabellini (6):
libxl: do not add a vkb backend to hvm guests
[WIP] libxl: xsrestrict QEMU
libxl: allow /local/domain/$LIBXL_TOOLSTACK_DOMID/device-model/$DOMID
to be written by $DOMID
libxl: change xs path for QEMU
libxl: change qdisk-backend-pid path on xenstore
libxl: spawns two QEMUs for HVM guests

   docs/misc/xenstore-paths.markdown |   30 --
   tools/libxl/libxl.c   |2 +-
   tools/libxl/libxl_create.c|   58 +--
   tools/libxl/libxl_device.c|2 +-
   tools/libxl/libxl_dm.c|  115
+
   tools/libxl/libxl_dom.c   |   19 --
   tools/libxl/libxl_internal.c  |   19 --
   tools/libxl/libxl_internal.h  |   15 -
   tools/libxl/libxl_pci.c   |   14 ++---
   tools/libxl/libxl_utils.c |   10 
   10 files changed, 225 insertions(+), 59 deletions(-)

Cheers,

Stefano

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] unable to boot from cdrom using OVMF in latest xen-unstable

[Fabio Fantoni]

Today I retried ovmf after long time from previous, I tried old fedora 
20 uefi domUs and still boot correctly but trying to boot from cdrom 
using boot='dc' or boot='d' don't boot from cdrom (seems didn't try) but 
boot from harddisk (even with boot='d').
After booting fedora I checked the cdrom and see it correctly, the 
problem seems only about boot order.
xen is build from staging git based on commit 
44313ab77f3e3c5b566ea4f23b0e32bfd5eafa29 and ovmf is from debian package 
0~20150106.5c2d456b-1.


If you need more informations/tests tell me and I'll post them.

Thanks for any reply and sorry for my bad english.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v7] run QEMU as non-root

[Fabio Fantoni]

Il 23/07/2015 19:08, Stefano Stabellini ha scritto:

Try to use "xen-qemudepriv-domid$domid" first, then
"xen-qemudepriv-shared" and root if everything else fails.

The uids need to be manually created by the user or, more likely, by the
xen package maintainer.

Expose a device_model_user setting in libxl_domain_build_info, so that
opinionated callers, such as libvirt, can set any user they like. Do not
fall back to root if device_model_user is set.

QEMU is going to setuid and setgid to the user ID and the group ID of
the specified user, soon after initialization, before starting to deal
with any guest IO.

To actually secure QEMU when running in Dom0, we need at least to
deprivilege the privcmd and xenstore interfaces, this is just the first
step in that direction.

Signed-off-by: Stefano Stabellini 


Thanks for this patch, now I'll test it.
I think can be good add also domU's xl cfg parameter for set custom user 
to use instead make possible only in libxl from external tools, is 
possible to add it?
For example in my case I use xl and I want run domU's qemu with user not 
shared but I want create only one user for each effective domU and not 
thousand of users.

Another comment below...



---
Changes in v7:
- do not fall back to root if the user explicitly set
b_info->device_model_user.

Changes in v6:
- add device_model_user to libxl_domain_build_info
- improve doc
- improve wording in commit message

Changes in v5:
- improve wording in doc
- fix wording in warning message
- fix example in doc
- drop xen-qemudepriv-$domname

Changes in v4:
- rename qemu-deprivilege to qemu-deprivilege.txt
- add a note about qemu-deprivilege.txt to INSTALL
- instead of xen-qemudepriv-base + $domid, try xen-qemudepriv-domid$domid
- introduce libxl__dm_runas_helper to make the code nicer

Changes in v3:
- clarify doc
- handle errno == ERANGE
---
  INSTALL|7 +
  docs/misc/qemu-deprivilege.txt |   31 ++
  tools/libxl/libxl.h|6 +
  tools/libxl/libxl_dm.c |   55 
  tools/libxl/libxl_internal.h   |4 +++
  tools/libxl/libxl_types.idl|1 +
  6 files changed, 104 insertions(+)
  create mode 100644 docs/misc/qemu-deprivilege.txt

diff --git a/INSTALL b/INSTALL
index a0f2e7b..fe83c20 100644
--- a/INSTALL
+++ b/INSTALL
@@ -297,6 +297,13 @@ systemctl enable xendomains.service
  systemctl enable xen-watchdog.service
  
  
+QEMU Deprivilege

+
+It is recommended to run QEMU as non-root.
+See docs/misc/qemu-deprivilege.txt for an explanation on what you need
+to do at installation time to run QEMU as a dedicated user.
+
+
  History of options
  ==
  
diff --git a/docs/misc/qemu-deprivilege.txt b/docs/misc/qemu-deprivilege.txt

new file mode 100644
index 000..12eb104
--- /dev/null
+++ b/docs/misc/qemu-deprivilege.txt
@@ -0,0 +1,31 @@
+For security reasons, libxl tries to pass a non-root username to QEMU as
+argument. During initialization QEMU calls setuid and setgid with the
+user ID and the group ID of the user passed as argument.
+Libxl looks for the following users in this order:
+
+1) a user named "xen-qemuuser-domid$domid",
+Where $domid is the domid of the domain being created.
+This requires the reservation of 65535 uids from xen-qemuuser-domid1
+to xen-qemuuser-domid65535. To use this mechanism, you might want to
+create a large number of users at installation time. For example:
+
+for ((i=1; i<65536; i++))
+do
+adduser --no-create-home --system xen-qemuuser-domid$i
+done
+
+You might want to consider passing --group to adduser to create a new
+group for each new user.
+
+
+2) a user named "xen-qemuuser-shared"
+As a fall back if both 1) and 2) fail, libxl will use a single user for
+all QEMU instances. The user is named xen-qemuuser-shared. This is
+less secure but still better than running QEMU as root. Using this is as
+simple as creating just one more user on your host:
+
+adduser --no-create-home --system xen-qemuuser-shared
+
+
+3) root
+As a last resort, libxl will start QEMU as root.
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index efc0617..3f4283f 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -192,6 +192,12 @@
   * is not present, instead of ERROR_INVAL.
   */
  #define LIBXL_HAVE_ERROR_DOMAIN_NOTFOUND 1
+
+/* libxl_domain_build_info has device_model_user to specify the user to
+ * run the device model with. See docs/misc/qemu-deprivilege.txt.
+ */
+#define LIBXL_HAVE_DEVICE_MODEL_USER 1
+
  /*
   * libxl ABI compatibility
   *
diff --git a/tools/libxl/libxl_dm.c b/tools/libxl/libxl_dm.c
index 0c6408d..24c43df 100644
--- a/tools/libxl/libxl_dm.c
+++ b/tools/libxl/libxl_dm.c
@@ -19,6 +19,8 @@
  
  #include "libxl_internal.h"

  #include 
+#include 
+#include 
  
  static const char *libxl_tapif_script(libxl__gc *gc)

  {
@@ -418,6 +420,33 @@ static char *dm_spice_options(libxl__gc *gc,
  return opt;
  }
  
+/* return 1

Re: [Xen-devel] [PATCH v5 0/6] libxl: xs_restrict QEMU

[Fabio Fantoni]

Il 23/07/2015 19:26, Stefano Stabellini ha scritto:

Hi all,

this patch series changes libxl to start QEMU as device model with the
new xsrestrict option (http://marc.info/?l=xen-devel&m=143341692707358).
It also starts a second QEMU to provide PV backends in userspace (qdisk)
to HVM guests.


Hi, I'm interested to test this serie.
xen patch "run QEMU as non-root" and qemu patch linked above are the 
only prerequisite or other are needed?
I saw that second patch is marked as [WIP], is it usable or I must wait 
to have it complete before test this serie?


Thanks for any reply and sorry for my bad english.




Changes in v5:
- improve commit messages with security details

Changes in v4:
- update xenstore-paths.markdown
- add error message in case count > MAX_PHYSMAP_ENTRIES
- add a note to xenstore-paths.markdown about the possible change in
privilege level
- only change permissions if xsrestrict is supported

Changes in v3:
- use LIBXL_TOOLSTACK_DOMID instead of 0 in the commit message
- update commit message with more info on why it is safe
- add a limit on the number of physmap entries to save and restore
- add emulator_ids
- mark patch #3 as WIP
- use LIBXL_TOOLSTACK_DOMID instead of 0 in the commit message
- change xs path to include the emulator_id
- change qdisk-backend-pid path on xenstore
- use dcs->dmss.pvqemu to spawn the second QEMU
- keep track of the rc of both QEMUs before proceeding


Stefano Stabellini (6):
   libxl: do not add a vkb backend to hvm guests
   [WIP] libxl: xsrestrict QEMU
   libxl: allow /local/domain/$LIBXL_TOOLSTACK_DOMID/device-model/$DOMID to 
be written by $DOMID
   libxl: change xs path for QEMU
   libxl: change qdisk-backend-pid path on xenstore
   libxl: spawns two QEMUs for HVM guests

  docs/misc/xenstore-paths.markdown |   30 --
  tools/libxl/libxl.c   |2 +-
  tools/libxl/libxl_create.c|   58 +--
  tools/libxl/libxl_device.c|2 +-
  tools/libxl/libxl_dm.c|  115 +
  tools/libxl/libxl_dom.c   |   19 --
  tools/libxl/libxl_internal.c  |   19 --
  tools/libxl/libxl_internal.h  |   15 -
  tools/libxl/libxl_pci.c   |   14 ++---
  tools/libxl/libxl_utils.c |   10 
  10 files changed, 225 insertions(+), 59 deletions(-)

Cheers,

Stefano

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v5 1/6] libxl: do not add a vkb backend to hvm guests

[Fabio Fantoni]

Il 24/07/2015 12:21, Stefano Stabellini ha scritto:

On Fri, 24 Jul 2015, Paul Durrant wrote:

-Original Message-
From: xen-devel-boun...@lists.xen.org [mailto:xen-devel-
boun...@lists.xen.org] On Behalf Of Stefano Stabellini
Sent: 23 July 2015 18:28
To: xen-de...@lists.xensource.com
Cc: Wei Liu; Ian Jackson; Ian Campbell; Stefano Stabellini
Subject: [Xen-devel] [PATCH v5 1/6] libxl: do not add a vkb backend to hvm
guests

When QEMU restricts its xenstore connection, it cannot provide PV
backends. A separate QEMU instance is required to provide PV backends in
userspace, such as qdisk. With two separate instances, it is not
possible to take advantage of vkb for mouse and keyboard, as the QEMU
that emulates the graphic card (the device model), would be separate
from the QEMU running the vkb backend (PV QEMU).

Removing this functionality is acceptable, because is only useful for
power saving when usb emulation is off, letting QEMU sleep for longer
periods of time.  However usb emulation is on by default, and how to
take advantage of this configuration has never been documented.


I don't think I agree. Turning off USB emulation for HVM guests (particularly 
Windows) has been shown to be highly advantageous in performance and 
scalability terms, and we have a prototype HID driver (not yet part of the 
XenProject driver set, but hopefully soon will be) which uses vkb.

I would appreciate if this kind of comments were made at v1 or v2, not
v5 of a series :-)


I know that turning USB emulation off is a big win, but nobody is really
doing it. The reason is that we didn't properly documented how to do it.
As you say, not even the Xen Project Windows PV drivers take advantage
of vkb yet, even though they might soon. I still think that removing vkb
cannot be considered a regression.

If it comes to a choice, I think that securing QEMU is more important
that turning USB emulation off and the two are fundamentally
incompatible.

Even if we run two QEMUs, one for emulation, one for the backends, the
vkb backend would need to be running in the same QEMU that offers vga
emulation because of the cursor rendering. It is a no go.

We could expose vkb but only together with vfb, so the vkb cursor would
only affect the vfb screen and not the vga screen. I don't know if you
would like that solution, but in any case it could still be introduced
after this work as it is independent.

Another possibility would be to add an option to request a vkb backend
for HVM guests in the config file. If the option is selected QEMU would
run as root. I wouldn't dare to recommend it to anybody.


Time ago I tried this patch (without others of this serie) and if I 
remember good I tried also windows domUs (with new winpv drivers) 
without found regressions.
But from a fast look to all patches the thing with more probable 
regression can be the "qemu split" and restriction instead this, today I 
don't have enought time but probably next week I'll try complete serie 
on both linux and windows domUs and including all features that may risk 
regressions FWIK.






Signed-off-by: Stefano Stabellini 
Acked-by: Ian Campbell 
---
  tools/libxl/libxl_create.c |5 -
  1 file changed, 5 deletions(-)

diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
index f0da7dc..a74b340 100644
--- a/tools/libxl/libxl_create.c
+++ b/tools/libxl/libxl_create.c
@@ -1275,17 +1275,12 @@ static void domcreate_launch_dm(libxl__egc
*egc, libxl__multidev *multidev,
  {
  libxl__device_console console;
  libxl__device device;
-libxl_device_vkb vkb;

  init_console_info(gc, &console, 0);
  console.backend_domid = state->console_domid;
  libxl__device_console_add(gc, domid, &console, state, &device);
  libxl__device_console_dispose(&console);

-libxl_device_vkb_init(&vkb);
-libxl__device_vkb_add(gc, domid, &vkb);
-libxl_device_vkb_dispose(&vkb);
-
  dcs->dmss.dm.guest_domid = domid;
  if (libxl_defbool_val(d_config->b_info.device_model_stubdomain))
  libxl__spawn_stub_dm(egc, &dcs->dmss);
--
1.7.10.4





___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Stable backport request for new OVMF version

[Fabio Fantoni]

Il 23/07/2015 16:51, Jan Beulich ha scritto:

On 23.07.15 at 16:43,  wrote:

On Thu, 2015-07-23 at 08:26 -0600, Jan Beulich wrote:

On 23.07.15 at 15:56,  wrote:

The version of OVMF in 4.5 (and presumably earlier) doesn't build
with
the gcc in modern Linux distros (I noticed it with 4.9 from
Debian/Jessie).

I think we should update Config.mk:OVMF_UPSTREAM_REVISION to
cb9a7ebabcd6b8a49dc0854b2f9592d732b5afbd in all stable branches.
That
version is the one currently in master.

To be honest I didn't even know we support use of OVMF in any
released version (nor in unstable). When did this change?

It was present in 4.3. I don't think we build it by default since its
toolchain is a bit picky, but we turn it on and test it in osstest from
4.4 onwards.

You can ask for it with bios="ovmf" in your hvm guest's config.

Well, that means it can be used by the adventurous, but it doesn't
mean we consider it supported.

Jan




Even if used only by the adventurous left it unable to build with newer 
gcc don't seems very good, even if the adventurous should be able to 
change Config.mk to use update ovmf git that don't fail the build.



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2] xen/HVM: atomically access pointers in bufioreq handling

[Fabio Fantoni]

Il 23/07/2015 08:59, Jan Beulich ha scritto:

The number of slots per page being 511 (i.e. not a power of two) means
that the (32-bit) read and write indexes going beyond 2^32 will likely
disturb operation. The hypervisor side gets I/O req server creation
extended so we can indicate that we're using suitable atomic accesses
where needed, allowing it to atomically canonicalize both pointers when
both have gone through at least one cycle.

The Xen side counterpart (which is not a functional prereq to this
change, albeit a build one) went in already (commit b7007bc6f9).


If I understand good a xen 4.6 is needed to build qemu with this patch, 
if yes probably is good add also a change to configure of require xen 
>=4.6, instead qemu will fail to build if xen version if lower.




Signed-off-by: Jan Beulich 
---
v2: Adjust description.

--- a/xen-hvm.c
+++ b/xen-hvm.c
@@ -957,19 +957,30 @@ static void handle_ioreq(XenIOState *sta
  
  static int handle_buffered_iopage(XenIOState *state)

  {
+buffered_iopage_t *buf_page = state->buffered_io_page;
  buf_ioreq_t *buf_req = NULL;
  ioreq_t req;
  int qw;
  
-if (!state->buffered_io_page) {

+if (!buf_page) {
  return 0;
  }
  
  memset(&req, 0x00, sizeof(req));
  
-while (state->buffered_io_page->read_pointer != state->buffered_io_page->write_pointer) {

-buf_req = &state->buffered_io_page->buf_ioreq[
-state->buffered_io_page->read_pointer % IOREQ_BUFFER_SLOT_NUM];
+for (;;) {
+uint32_t rdptr = buf_page->read_pointer, wrptr;
+
+xen_rmb();
+wrptr = buf_page->write_pointer;
+xen_rmb();
+if (rdptr != buf_page->read_pointer) {
+continue;
+}
+if (rdptr == wrptr) {
+break;
+}
+buf_req = &buf_page->buf_ioreq[rdptr % IOREQ_BUFFER_SLOT_NUM];
  req.size = 1UL << buf_req->size;
  req.count = 1;
  req.addr = buf_req->addr;
@@ -981,15 +992,14 @@ static int handle_buffered_iopage(XenIOS
  req.data_is_ptr = 0;
  qw = (req.size == 8);
  if (qw) {
-buf_req = &state->buffered_io_page->buf_ioreq[
-(state->buffered_io_page->read_pointer + 1) % 
IOREQ_BUFFER_SLOT_NUM];
+buf_req = &buf_page->buf_ioreq[(rdptr + 1) %
+   IOREQ_BUFFER_SLOT_NUM];
  req.data |= ((uint64_t)buf_req->data) << 32;
  }
  
  handle_ioreq(state, &req);
  
-xen_mb();

-state->buffered_io_page->read_pointer += qw ? 2 : 1;
+atomic_add(&buf_page->read_pointer, qw + 1);
  }
  
  return req.count;

--- unstable.orig/qemu/upstream/include/hw/xen/xen_common.h 2015-03-31 
15:58:04.0 +0200
+++ unstable/qemu/upstream/include/hw/xen/xen_common.h  2015-06-15 
08:24:28.0 +0200
@@ -370,7 +370,8 @@ static inline void xen_unmap_pcidev(XenX
  static inline int xen_create_ioreq_server(XenXC xc, domid_t dom,
ioservid_t *ioservid)
  {
-int rc = xc_hvm_create_ioreq_server(xc, dom, 1, ioservid);
+int rc = xc_hvm_create_ioreq_server(xc, dom, HVM_IOREQSRV_BUFIOREQ_ATOMIC,
+ioservid);
  
  if (rc == 0) {

  trace_xen_ioreq_server_create(*ioservid);



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] RFC on deprivileged x86 hypervisor device models

[Fabio Fantoni]

Il 17/07/2015 12:09, Ben Catterall ha scritto:

Hi all,

I'm working on an x86 proof-of-concept series to evaluate if it is 
feasible to move device models currently running in the hypervisor and 
x86 emulation code for HVM guests into a deprivileged context.


I've put together the following document as I have been considering 
several different ways this could be achieved and was hoping to get 
feedback from maintainers before I go ahead.


Have you already take a look to this patch serie?
http://lists.xen.org/archives/html/xen-devel/2015-07/msg00108.html



Many thanks in advance,
Ben

Context
---
The aim is to run device models, which are already running inside the 
hypervisor (e.g. x86 emulate), in deprivileged user mode for HVM 
guests, using suitably mapped page tables. A simple hypercall 
convention is needed to pass data between these two modes of operation 
and a mechanism to move between them.


This is intended as a proof-of-concept, with the aim of determining if 
this idea is feasible within performance constraints.


Motivation
--
The motivation for moving the device models and x86 emulation code 
into ring 3 is to mitigate a system compromise due a bug in any of 
these systems. These systems are currently part of the hypervisor and, 
consequently, a bug in any of these could allow an attacker to gain 
control (or perform a DOS) of Xen and/or guests.



Moving between privilege levels

The general process is to determine if we need to run a device model 
(or similar) and then, if so, switch into deprivileged mode. The 
operation is performed by deprivileged code which calls into the 
hypervisor as and when needed. After the operation completes, we 
return to the hypervisor.


If deprivileged mode needs to make any hypervisor requests, it can do 
these using a syscall interface, possibly placing an operation code 
into a register to indicate the operation. This would allow it to get 
data to/from the hypervisor.


I am currently considering three different methods regarding the 
context switch and would be grateful of any feedback.


Method One
--
This method works by building on top of the QEMU emulation path code. 
This currently operates as a state machine, using flags to determine 
the current emulation state. These states are used to determine which 
code paths to take when calling out of and into the hypervisor before 
and after emulation.


The intention would be to add new states and then follow the same 
process as existing code does except, rather than blocking the vcpu, 
we switch into deprivileged mode and process the request on the 
current vcpu. This is different to QEMU which blocks the current vcpu 
so additional code is needed to support this context switch. There may 
be other code paths which have not been written in this way which 
would require rewriting.


When moving into deprivileged mode, we need to be careful to ensure 
that when we leave, we can redo the call into the hypervisor after the 
device model completes without causing problems. Thus, we need to be 
_certain_ that the same call path is followed on the re-entry and that 
the system's state can handle this. This may mean undoing operations 
such as memory allocations.



Method Two
--
At the point of detecting the need to perform a deprivileged 
operation, we take a copy of the current stack from the current stack 
position up to the point where the guest entered Xen and save it. 
Subsequently, we move the stack pointer back.


This effectively gives us a clean stack as though we had just entered 
Xen. We then put the deprivileged context onto this new stack and 
enter deprivileged mode.


Upon returning, we restore the previous stack with the guest's and 
Xen's context then jump to the saved rip and continue execution. Xen 
will then perform the necessary processing, determining if the 
operation was successful or not.


We are effectively context switching out Xen for deprivileged code and 
then bringing Xen back in once we're done.


As Xen is non-preemptive, the Xen stack won't be updated whilst we're 
in deprivileged mode. If it may be updated (I'm speculating here), 
e.g. an interrupt, then we can pause deprivileged mode by hooking the 
interrupt and restoring the Xen stack, then handle the interrupt and 
finally go back to deprivileged mode.


Problem: If the device model or emulator edit the saved guest 
registers and these are touched by Xen on the return path after 
finishing servicing the deprivileged operation, then the guest will 
use these values not those the deprivileged mode provided.


This is not a problem if the code doesn't do this. If it does, we 
could give higher precedence to deprivileged changes. So, deprivileged 
mode pushes the changes into the hypervisor which caches them and 
then, just before guest context is restored, makes those changes, thus 
discarding any Xen made.




Method Three

A per vcpu stack is

Re: [Xen-devel] SSE instruction emulation issues

[Fabio Fantoni]

Il 16/07/2015 16:54, Wang, Zhi A ha scritto:

Hi Fantoni:
If the emulation is successfully done without problem, maybe the instruction 
emulator in xen inject an incorrect fault into HVM, so that Xorg got killed. 
Can you also show the binary opcode with asm code? It looks only asm code in 
the gdb log.


Thanks for reply, I tried to add some gdprintk on probable fail about 
movaps, more exactly before these lines:

4005 fail_if(vex.pfx & VEX_PREFIX_SCALAR_MASK);
4037 fail_if((vex.opcx != vex_0f) ||
4050 generate_exception_if((b >= 0x28) &&
But xl dmesg showed nothing.

Gdb data more complete that this (full backtrace, registers, latest 
instructions and threads backtrace) I not found, see attachment.


This is the command used:
gdb -batch-silent -ex "run" -ex "set logging overwrite on" -ex "set 
logging file /root/X-gdb.log" -ex "set logging on" -ex "set pagination 
off" -ex "handle SIG33 pass nostop noprint" -ex "echo Full 
backtrace:\n" -ex "backtrace full" -ex "echo \n\nRegisters:\n" -ex 
"info registers" -ex "echo \n\nCurrent instructions:\n" -ex "x/16i 
\$pc" -ex "echo \n\nThreads backtrace:\n" -ex "thread apply all 
backtrace" -ex "set logging off" -ex "quit" --args "/usr/bin/X.bak" "$@"


Do you mean that I need something more complete to replace this about 
instructions? x/16i $pc


Another thing, on fedora 21 (where xorg crash is sid instead) where qemu 
is a 100% cpu (Xorg of domU) I saw that after long time arrive to view 
correct login page also with qxl, probably was about 20 minutes instead 
of 3-4 seconds max, pratically seems working (probably after latest one 
or more of many x86/hvm patches) but with performance 1/1000 or lesser 
that normal.
Probably is something similar to windows some years ago where before an 
old Jan Beulich patch performance (based on screen output) was 1/10 or 
lesser.


Sorry if I had too low knowledge to find exact problem or at least all 
useful data :(




-Original Message-
From: Fabio Fantoni [mailto:fabio.fant...@m2r.biz]
Sent: Thursday, July 16, 2015 8:15 PM
To: Wang, Zhi A; Jan Beulich
Cc: Andrew Cooper; Paul Durrant; xen-devel
Subject: Re: SSE instruction emulation issues

Il 15/07/2015 16:35, Wang, Zhi A ha scritto:

You can put the MMIO emulation failed output message in the email like what I 
did, that will help to cook a patch for instruction emulator. Only gdb log is 
not enough as xen-developer has to know the exact opcode. I also found that not 
all forms of one SSE instruction was supported, for example an instruction may 
support move data from xmm register to mem, or move from xmm register to xmm 
registers, maybe only one form is supported in the instruction emulator.

Thanks for your reply, unfortunately don't give directly "MMIO emulation failed" but on 
"better" cases only software crash inside domUs, on other only qemu at 100% cpu and in 
domUs black screen ecc...
There are also many other cases of "strange" low performance probably related 
to MMIO operations.
I started to find it some years ago on hvm desktop domUs, derisory/hardly 
visible on older windows like xp or linux DE with fewer effects and lower 
screen resolution of client used and increasing using newer system, DE and 
bigger resolutions.
Using only thing like rdp inside domU's S.O. the problem is minimized, using 
client for connecting directly to domUs like spice is increased.
Seems increased also using rdp but using softwares that use "more graphic 
operations" (even if not 3D).
Trying similar thing on kvm instead using same qemu upstream version similar 
domU configuration ecc.. is better.
About more specific problem with qxl vga on xen was unable to works at all 
until 2 memory fixes in libxl/qemu did by anthony perard 3-4 years ago is I 
remember good, after on windows domUs was working but with very bad 
performance, after Jan beulich a fix about x86/hvm 2-3 years performances 
became decent and other x86/hvm changes in latest 1-2 years they made the 
performance closer to kvm for most of the vm windows.
About linux domUs qxl seems correctly working on suse as dom0 and domUs as 
reported by a suse developer but I not found the specific suse workaround, may 
be also other cases where is working out-of-box.
Xen is already better for many features and find/solves this problem probably 
make it very good also in all recent desktop necessity (except particular 3d 
acceleration that require phisical vga or similar).

About cases where I taken backtrace of domU's X crash I suppose I must find 
possible fails case without output in xl dmesg and add output to find what is 
the exact case, right?
I should add gdprintk(XENLOG_INFO,"...\n"); in any fail_if() and
generate_exceptio

Re: [Xen-devel] SSE instruction emulation issues

[Fabio Fantoni]

Il 15/07/2015 16:35, Wang, Zhi A ha scritto:

You can put the MMIO emulation failed output message in the email like what I 
did, that will help to cook a patch for instruction emulator. Only gdb log is 
not enough as xen-developer has to know the exact opcode. I also found that not 
all forms of one SSE instruction was supported, for example an instruction may 
support move data from xmm register to mem, or move from xmm register to xmm 
registers, maybe only one form is supported in the instruction emulator.


Thanks for your reply, unfortunately don't give directly "MMIO emulation 
failed" but on "better" cases only software crash inside domUs, on other 
only qemu at 100% cpu and in domUs black screen ecc...
There are also many other cases of "strange" low performance probably 
related to MMIO operations.
I started to find it some years ago on hvm desktop domUs, 
derisory/hardly visible on older windows like xp or linux DE with fewer 
effects and lower screen resolution of client used and increasing using 
newer system, DE and bigger resolutions.
Using only thing like rdp inside domU's S.O. the problem is minimized, 
using client for connecting directly to domUs like spice is increased.
Seems increased also using rdp but using softwares that use "more 
graphic operations" (even if not 3D).
Trying similar thing on kvm instead using same qemu upstream version 
similar domU configuration ecc.. is better.
About more specific problem with qxl vga on xen was unable to works at 
all until 2 memory fixes in libxl/qemu did by anthony perard 3-4 years 
ago is I remember good, after on windows domUs was working but with very 
bad performance, after Jan beulich a fix about x86/hvm 2-3 years 
performances became decent and other x86/hvm changes in latest 1-2 years 
they made the performance closer to kvm for most of the vm windows.
About linux domUs qxl seems correctly working on suse as dom0 and domUs 
as reported by a suse developer but I not found the specific suse 
workaround, may be also other cases where is working out-of-box.
Xen is already better for many features and find/solves this problem 
probably make it very good also in all recent desktop necessity (except 
particular 3d acceleration that require phisical vga or similar).


About cases where I taken backtrace of domU's X crash I suppose I must 
find possible fails case without output in xl dmesg and add output to 
find what is the exact case, right?
I should add gdprintk(XENLOG_INFO,"...\n"); in any fail_if() and 
generate_exception_if() that can be related to movaps for example based 
on backtrace, or is this wrong or stupid?



Thanks for any reply and sorry for my bad english.



Thanks,
Zhi.

-Original Message-
From: Fabio Fantoni [mailto:fabio.fant...@m2r.biz]
Sent: Wednesday, July 15, 2015 9:56 PM
To: Jan Beulich
Cc: Andrew Cooper; Paul Durrant; Wang, Zhi A; xen-devel
Subject: Re: SSE instruction emulation issues

Il 15/07/2015 13:35, Jan Beulich ha scritto:

On 15.07.15 at 13:13,  wrote:

Il 10/07/2015 14:16, Jan Beulich ha scritto:

On 10.07.15 at 14:00,  wrote:

On 09/07/15 20:32, Zhi Wang wrote:

   We found that MOVD instruction are used by some windows
driver during developing XenGT, and also we found this one:

(XEN) MMIO emulation failed: d7v1 64bit @ 0010:f8000294e273 ->
66 0f e7 00 48 83 c0 10 45 3 b cb 73 f0 45 85 c9

Disassembly:
  0:66 0f e7 00  movntdq %xmm0,(%rax)
  4:48 83 c0 10  add$0x10,%rax
  8:45 3b cb cmp%r11d,%r9d
  b:73 f0jae0xfffd
  d:45 85 c9 test   %r9d,%r9d

The x86 instruction emulator does appear to have a decode for this
instruction.  This failure suggests that the implementation is buggy.

To start with diagnosing, add a test case to
tools/tests/x86_emulator/test_x86_emulator.c

Considering that we already test MOVDQU, the emulation of which
shares code with MOVNTDQ (which only differs in aspects not of
interest to the emulator) I'm not sure this will turn up anything
interesting. Perhaps an even easier step would be to simply run the
emulator test on the machine where the issue is seen? We're playing
some prefix byte tricks there... Otoh failure to execute the
constructed instruction would bring down the hypervisor.

I also have a problem with mmio as I already reported many times but
I

And to be honest, I don't see the value in re-stating this every once
in a while without providing any new information.


don't know if it is the same as the one reported by the intel
developer about xengt, I have it in linux hvm domUs with qxl.

Looks different - their's was about MOVD (which we clearly don't
support right now) while yours looks to be about MOVAPS.


Today with the latest xen update from git staging (with the addiction
of the xengt patch that add support of

Re: [Xen-devel] Regression in OSSTest Windows install test case

[Fabio Fantoni]

Il 15/07/2015 16:57, Wei Liu ha scritto:

On Wed, Jul 15, 2015 at 03:55:24PM +0100, Paul Durrant wrote:
[...]

Here is the config I use (slightly modified from the one in OSSTest). No
setting with regard to video card, so it should be the cirrus.


Cirrus driver is not available for windows 7, support of it is near 
abandonment in upstream if I remember good what I read one year ago (or 
more) in qemu-devel, is better use stdvga instead in windows >=vista.
A had also strange graphic and performance problem with cirrus on 
windows 7 2-3 years ago, since then I have not set the cirrus on vm 
windows 7.

Try to vga="stdvga"



---
name= 'win'
memory = 512

 Wow. That's ambitious for Windows 7!


I notice that too. But the installation seems to finish in a reasonable
time. :-P


Years ago on my tests system with few ram I tried to decrease ram as 
possible but I saw that with windows 7 (even if 32 bit) with < 1 gb made 
occasional problems that wasted my tests.
About performance for run tests (after windows install even if basic) 2 
gb of ram at least is needed for don't waste large amount of time on 
massive numbers of tests.
I saw many occasional windows tests fail on "[Xen-devel] [xen-unstable 
test] : regressions - FAIL" that I was unable to reproduce doing 
same tests (windows install, shutdown, save/restore) and that blocked 
push for days even if I not saw "real bug/regression" was found.
If these are only caused by "bad" xl cfg values rare to find on real 
users domUs I think is bad and should be improved.




Wei.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] SSE instruction emulation issues

[Fabio Fantoni]

Il 15/07/2015 13:35, Jan Beulich ha scritto:

On 15.07.15 at 13:13,  wrote:

Il 10/07/2015 14:16, Jan Beulich ha scritto:

On 10.07.15 at 14:00,  wrote:

On 09/07/15 20:32, Zhi Wang wrote:

  We found that MOVD instruction are used by some windows driver
during developing XenGT, and also we found this one:

(XEN) MMIO emulation failed: d7v1 64bit @ 0010:f8000294e273 -> 66
0f e7 00 48 83 c0 10 45 3
b cb 73 f0 45 85 c9

Disassembly:
 0:66 0f e7 00  movntdq %xmm0,(%rax)
 4:48 83 c0 10  add$0x10,%rax
 8:45 3b cb cmp%r11d,%r9d
 b:73 f0jae0xfffd
 d:45 85 c9 test   %r9d,%r9d

The x86 instruction emulator does appear to have a decode for this
instruction.  This failure suggests that the implementation is buggy.

To start with diagnosing, add a test case to
tools/tests/x86_emulator/test_x86_emulator.c

Considering that we already test MOVDQU, the emulation of which
shares code with MOVNTDQ (which only differs in aspects not of
interest to the emulator) I'm not sure this will turn up anything
interesting. Perhaps an even easier step would be to simply run
the emulator test on the machine where the issue is seen? We're
playing some prefix byte tricks there... Otoh failure to execute
the constructed instruction would bring down the hypervisor.

I also have a problem with mmio as I already reported many times but I

And to be honest, I don't see the value in re-stating this every once
in a while without providing any new information.


don't know if it is the same as the one reported by the intel developer
about xengt, I have it in linux hvm domUs with qxl.

Looks different - their's was about MOVD (which we clearly don't
support right now) while yours looks to be about MOVAPS.


Today with the latest xen update from git staging (with the addiction of
the xengt patch that add support of emulating SSE2 instruction MOVD) I
had a different domU's Xorg backtrace containing also a "error: Cannot
access memory at address":

Sadly a gdb backtrace is nothing I can see use extract useful information
from. Iirc Paul had already asked you to instrument the involved code
paths (considering that the x86 insn emulator supports MOVAPS as used
by the failing code) to figure out where in the whole involved stack the
failure actually originates.

Jan



Thanks for your reply, as I wrote the other times I don't know a better 
debug method about particular things like this (x86 instructions 
emulation) and I'm asking what I should do.
If you mean to look at the code involved, search the part about the 
problem, think how can go wrong or unexpected, add debug output if 
needed, try quick changes to it ecc... I can do it with simpler software 
and I did something similar with libxl but I don't know how to do the 
same for code like xen/arch/x86/x86_emulate/x86_emulate.c. I already 
took a look at it but I didn't find "MOVAPS" in comments like many others.
If the problem is located in something like libxl where there are 
instructions that I know or that are intuitive I can imagine what the 
software is supposed to do and I can do quick targeted tests or changes, 
but on thing like x86 emulation I can't (or at least not before knowing 
all instructions and essential data about it).
Is this what you mean and is that the only way to collect useful data or 
to solve the problem?
If so, I suppose that for any change in xen/arch/x86/x86_emulate and 
similar I can't simply make the change, do a make, make install and test 
it immediatly like libxl/xl but I have to rebuild full xen, install it 
and reboot dom0, is it right?
Can you post a link with a quick reference about x86 emulation and/or 
instruction sets like sse2 which can help me learn what to do or an 
extensive knowledge on the subject is required in this case?
What kind of logging instruction for debug can I use? Are they visible 
with xl dmesg or I must do something different and more specific in this 
case?


Thanks for any reply and sorry for my bad english.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] Request backport to xen qemu-upstream gits of a fix that solves occasional qemu crash using spice

[Fabio Fantoni]

On production system using xen 4.5 I had occasional qemu crash using 
spice, from qemu log:
qemu-system-i386: spice-qemu-char.c:173: spice_chr_add_watch: 
Assertion `cond == G_IO_OUT' failed.

It was solved upstream with this patch:
http://git.qemu.org/?p=qemu.git;a=commitdiff;h=f7a8beb5e6a13dc924895244777d9ef08b23b367
I already applied it on 2 my systems with xen>=4.5 without find regressions.
It is a small patch that affect only users that use spice solving an 
occasional qemu using spice, so I think is good backport it to 
qemu-upstream-unstable.git and qemu-upstream-4.5-testing.git.
On qemu-upstream-4.4-testing.git and older is not needed because they 
have older qemu without patch that introduced the regression ("serial: 
poll the serial console with G_IO_HUP").


Thanks for any reply and sorry for my bad english.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v7 00/15] x86/hvm: I/O emulation cleanup and fix | Full Backtrace of domU's X crash caused by SSE2 istruction in attachment

[Fabio Fantoni]

Il 10/07/2015 13:00, Jan Beulich ha scritto:

On 10.07.15 at 12:51,  wrote:

Il 10/07/2015 12:20, Jan Beulich ha scritto:

Other than MOVD, MOVAPS is already being supported by the
insn emulator.


Then why do you think MOVAPS fails in my test?

No idea. That's what Paul asked you to narrow down.
Can you be more specific about instrumenting xen because I don't know 
what to do, other then using gdb on domU xorg. Any hint would be 
appreciated.

Sse2 was introduced in cpus 11 years ago, so I think it would be useful
to add support for all missing instructions to avoid more cases like
this (and it seems to improve performance too, comparing the same task
executed with/without sse2).

The reason we didn't add any more of the SSE insns so far is that
we don't expect them to be used for accessing MMIO. Once we
learn they're needed, we'll add emulation for them, but you
realize this is a significant task?

Jan

Sorry for this, since the end of 2011 I'm trying desperately to improve 
recent desktop domUs graphic performance (windows >xp), recent linux DE 
(gnome 3, unity ecc...), bigger resolutions (7-8 years ago the standard 
was 1024x768 now it's around 1920x1080)
FWIK this seems necessary for what I'm trying to achieve (and to anyone 
other who want to do the same since I post all my patches), it seems 
also needed for xengt (in development) and there may be others.


A temp. solution at least to have qxl fully working on any linux domUs 
can be to find the workaround/fix that make it work in SUSE and apply it 
upstream

From my mail of some days ago:
I also took a fast look at suse kernel patches 
(https://github.com/openSUSE/kernel-source/tree/SLE12-SP1) where qxl 
is also working on linux domUs (other things seems already similar 
based on what Jim Fehlig told me) but I didn't find a possible 
fix/workaround for it to try. Can someone tell me about possible 
patches I should try please?


Thanks for any reply.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v7 00/15] x86/hvm: I/O emulation cleanup and fix | Full Backtrace of domU's X crash caused by SSE2 istruction in attachment

[Fabio Fantoni]

Il 10/07/2015 12:20, Jan Beulich ha scritto:

On 10.07.15 at 12:09,  wrote:

Il 10/07/2015 11:54, Fabio Fantoni ha scritto:

Il 10/07/2015 11:31, Paul Durrant ha scritto:

-Original Message-
From: Fabio Fantoni [mailto:fabio.fant...@m2r.biz]
Sent: 10 July 2015 10:28
To: Paul Durrant; xen-devel@lists.xen.org
Cc: Jan Beulich; Andrew Cooper
Subject: Re: [Xen-devel] [PATCH v7 00/15] x86/hvm: I/O emulation
cleanup
and fix | Full Backtrace of domU's X crash caused by SSE2 istruction in
attachment

Il 09/07/2015 15:10, Paul Durrant ha scritto:

This patch series re-works much of the code involved in emulation
of port
and memory mapped I/O for HVM guests.

The code has become very convoluted and, at least by inspection,
certain
emulations will apparently malfunction.

The series is broken down into 15 patches (which are also available in
my xenbits repo:

http://xenbits.xen.org/gitweb/?p=people/pauldu/xen.git

on the emulation34 branch).

Yesterday I retried with this version and seems that you fixed
something
that make possible atleast debug in the domU.

I taken gdb data of X crash inside Sid hvm domU:
#0  sse2_blt (src_bits=, dst_bits=,
src_stride=, dst_stride=,
src_bpp=, src_x=, src_y=0, dest_x=0,
dest_y=0, width=1024, height=, dst_bpp=,
imp=) at ../../pixman/pixman-sse2.c:4773

Latest istruction:
=> 0x7f70360ef8eb :movaps %xmm0,-0x10(%rsi)

Full log in attachment.

If you need more informations/tests tell me and I'll post them.


I imagine you got a GP fault due to handle_mmio() returning
X86EMUL_UNHANDLEABLE, but that's only a guess.
I suggest you try to instrument Xen a little to find out why.

Thanks for reply, sorry but I not understand exactly what I must do.
Can you detail please?

I take a look in xen/arch/x86/x86_emulate/x86_emulate.c and the
istruction seems like this other found by xengt developers:
https://github.com/01org/XenGT-Preview-xen/commit/f2bad31f80f698a452c37cb39841
da8e4f69350f

Other than MOVD, MOVAPS is already being supported by the
insn emulator.

Jan


Thanks for your reply.
Then why do you think MOVAPS fails in my test? If you need it, here is 
the full gdb output:

http://lists.xen.org/archives/html/xen-devel/2015-07/txtpTIgy5CpzU.txt
If you need more informations/tests tell me and I'll post them.
Sse2 was introduced in cpus 11 years ago, so I think it would be useful 
to add support for all missing instructions to avoid more cases like 
this (and it seems to improve performance too, comparing the same task 
executed with/without sse2).
I'm going to try updated xen staging with MOVD support patch added, any 
other useful information for other tests/debug is appreciated.


Thanks for any reply and sorry for my bad english.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v7 00/15] x86/hvm: I/O emulation cleanup and fix | Full Backtrace of domU's X crash caused by SSE2 istruction in attachment

[Fabio Fantoni]

Il 10/07/2015 11:54, Fabio Fantoni ha scritto:

Il 10/07/2015 11:31, Paul Durrant ha scritto:

-Original Message-
From: Fabio Fantoni [mailto:fabio.fant...@m2r.biz]
Sent: 10 July 2015 10:28
To: Paul Durrant; xen-devel@lists.xen.org
Cc: Jan Beulich; Andrew Cooper
Subject: Re: [Xen-devel] [PATCH v7 00/15] x86/hvm: I/O emulation 
cleanup

and fix | Full Backtrace of domU's X crash caused by SSE2 istruction in
attachment

Il 09/07/2015 15:10, Paul Durrant ha scritto:
This patch series re-works much of the code involved in emulation 
of port

and memory mapped I/O for HVM guests.

The code has become very convoluted and, at least by inspection, 
certain

emulations will apparently malfunction.

The series is broken down into 15 patches (which are also available in
my xenbits repo:

http://xenbits.xen.org/gitweb/?p=people/pauldu/xen.git

on the emulation34 branch).
Yesterday I retried with this version and seems that you fixed 
something

that make possible atleast debug in the domU.

I taken gdb data of X crash inside Sid hvm domU:
#0  sse2_blt (src_bits=, dst_bits=,
src_stride=, dst_stride=,
src_bpp=, src_x=, src_y=0, dest_x=0,
dest_y=0, width=1024, height=, dst_bpp=,
imp=) at ../../pixman/pixman-sse2.c:4773

Latest istruction:
=> 0x7f70360ef8eb :movaps %xmm0,-0x10(%rsi)

Full log in attachment.

If you need more informations/tests tell me and I'll post them.

I imagine you got a GP fault due to handle_mmio() returning 
X86EMUL_UNHANDLEABLE, but that's only a guess.

I suggest you try to instrument Xen a little to find out why.
Thanks for reply, sorry but I not understand exactly what I must do. 
Can you detail please?


I take a look in xen/arch/x86/x86_emulate/x86_emulate.c and the 
istruction seems like this other found by xengt developers:

https://github.com/01org/XenGT-Preview-xen/commit/f2bad31f80f698a452c37cb39841da8e4f69350f



   Paul


Thanks for any reply and sorry for my bad english.


Previous changelog
--

v4:
   - Removed previous patch (make sure translated MMIO reads or
 writes fall within a page) and rebased rest of series.
   - Address Jan's comments on patch #1

v3:
   - Addressed comments from Jan
   - Re-ordered series to bring a couple of more trivial patches to 
the

 front
   - Backport to XenServer (4.5) now passing automated tests
   - Tested on unstable with QEMU upstream and trad, with and without
 HAP (to force shadow emulation)

v2:
   - Removed bogus assertion from patch #15
   - Re-worked patch #17 after basic testing of back-port onto 
XenServer


Subsequent changes are logged in the individual patch files (thanks
to David Vrabel for that).

Testing
---

v6 of the series was been back-ported to staging-4.5 and then dropped
onto the XenServer (Dundee) patch queue. All automated branch-safety

tests

pass.

v7 has just been compile tested since changes were largely 
cosmetic. It

will be back-ported in the near future.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel





___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v7 00/15] x86/hvm: I/O emulation cleanup and fix | Full Backtrace of domU's X crash caused by SSE2 istruction in attachment

[Fabio Fantoni]

Il 10/07/2015 11:31, Paul Durrant ha scritto:

-Original Message-
From: Fabio Fantoni [mailto:fabio.fant...@m2r.biz]
Sent: 10 July 2015 10:28
To: Paul Durrant; xen-devel@lists.xen.org
Cc: Jan Beulich; Andrew Cooper
Subject: Re: [Xen-devel] [PATCH v7 00/15] x86/hvm: I/O emulation cleanup
and fix | Full Backtrace of domU's X crash caused by SSE2 istruction in
attachment

Il 09/07/2015 15:10, Paul Durrant ha scritto:

This patch series re-works much of the code involved in emulation of port
and memory mapped I/O for HVM guests.

The code has become very convoluted and, at least by inspection, certain
emulations will apparently malfunction.

The series is broken down into 15 patches (which are also available in
my xenbits repo:

http://xenbits.xen.org/gitweb/?p=people/pauldu/xen.git

on the emulation34 branch).

Yesterday I retried with this version and seems that you fixed something
that make possible atleast debug in the domU.

I taken gdb data of X crash inside Sid hvm domU:
#0  sse2_blt (src_bits=, dst_bits=,
src_stride=, dst_stride=,
src_bpp=, src_x=, src_y=0, dest_x=0,
dest_y=0, width=1024, height=, dst_bpp=,
imp=) at ../../pixman/pixman-sse2.c:4773

Latest istruction:
=> 0x7f70360ef8eb :movaps %xmm0,-0x10(%rsi)

Full log in attachment.

If you need more informations/tests tell me and I'll post them.


I imagine you got a GP fault due to handle_mmio() returning 
X86EMUL_UNHANDLEABLE, but that's only a guess.
I suggest you try to instrument Xen a little to find out why.
Thanks for reply, sorry but I not understand exactly what I must do. Can 
you detail please?


   Paul


Thanks for any reply and sorry for my bad english.


Previous changelog
--

v4:
   - Removed previous patch (make sure translated MMIO reads or
 writes fall within a page) and rebased rest of series.
   - Address Jan's comments on patch #1

v3:
   - Addressed comments from Jan
   - Re-ordered series to bring a couple of more trivial patches to the
 front
   - Backport to XenServer (4.5) now passing automated tests
   - Tested on unstable with QEMU upstream and trad, with and without
 HAP (to force shadow emulation)

v2:
   - Removed bogus assertion from patch #15
   - Re-worked patch #17 after basic testing of back-port onto XenServer

Subsequent changes are logged in the individual patch files (thanks
to David Vrabel for that).

Testing
---

v6 of the series was been back-ported to staging-4.5 and then dropped
onto the XenServer (Dundee) patch queue. All automated branch-safety

tests

pass.

v7 has just been compile tested since changes were largely cosmetic. It
will be back-ported in the near future.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v7 00/15] x86/hvm: I/O emulation cleanup and fix | Full Backtrace of domU's X crash caused by SSE2 istruction in attachment

[Fabio Fantoni]

Il 09/07/2015 15:10, Paul Durrant ha scritto:

This patch series re-works much of the code involved in emulation of port
and memory mapped I/O for HVM guests.

The code has become very convoluted and, at least by inspection, certain
emulations will apparently malfunction.

The series is broken down into 15 patches (which are also available in
my xenbits repo: http://xenbits.xen.org/gitweb/?p=people/pauldu/xen.git
on the emulation34 branch).


Yesterday I retried with this version and seems that you fixed something 
that make possible atleast debug in the domU.


I taken gdb data of X crash inside Sid hvm domU:
#0  sse2_blt (src_bits=, dst_bits=, 
src_stride=, dst_stride=, 
src_bpp=, src_x=, src_y=0, dest_x=0, 
dest_y=0, width=1024, height=, dst_bpp=, 
imp=) at ../../pixman/pixman-sse2.c:4773


Latest istruction:
=> 0x7f70360ef8eb :movaps %xmm0,-0x10(%rsi)

Full log in attachment.

If you need more informations/tests tell me and I'll post them.

Thanks for any reply and sorry for my bad english.



Previous changelog
--

v4:
  - Removed previous patch (make sure translated MMIO reads or
writes fall within a page) and rebased rest of series.
  - Address Jan's comments on patch #1

v3:
  - Addressed comments from Jan
  - Re-ordered series to bring a couple of more trivial patches to the
front
  - Backport to XenServer (4.5) now passing automated tests
  - Tested on unstable with QEMU upstream and trad, with and without
HAP (to force shadow emulation)

v2:
  - Removed bogus assertion from patch #15
  - Re-worked patch #17 after basic testing of back-port onto XenServer

Subsequent changes are logged in the individual patch files (thanks
to David Vrabel for that).

Testing
---

v6 of the series was been back-ported to staging-4.5 and then dropped
onto the XenServer (Dundee) patch queue. All automated branch-safety tests
pass.

v7 has just been compile tested since changes were largely cosmetic. It
will be back-ported in the near future.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel


Full backtrace:
#0  sse2_blt (src_bits=, dst_bits=, 
src_stride=, dst_stride=, src_bpp=, src_x=, src_y=0, dest_x=0, dest_y=0, width=1024, 
height=, dst_bpp=, imp=) at 
../../pixman/pixman-sse2.c:4773
w = 4160
s = 0x7f702985e530 "\300\300\300"
d = 0x7f70226d3fd0 "\300\300\300"
src_bytes = 0x7f702985d130 
"\277\277\277\261\277\277\277\260\277\277\277\257\277\277\277\256\277\277\277\257\277\277\277\261\300\300\300\261\300\300\300\260\277\277\277\261\277\277\277\261\277\277\277\260\277\277\277\257\277\277\277\262\277\277\277\261\277\277\277\262\300\300\300\261\300\300\300\261\300\300\300\261\277\277\277\257\277\277\277\261\277\277\277\261\277\277\277\261\277\277\277\260\300\300\300\262\277\277\277\261\277\277\277\260\277\277\277\260\277\277\277\260\277\277\277\261\277\277\277\261\277\277\277\261\277\277\277\261\277\277\277\261\277\277\277\260\277\277\277\261\277\277\277\261\277\277\277\261\277\277\277\260\277\277\277\261\277\277\277\261\277\277\277\261\277\277\277\260\277\277\277\262\277\277\277\261\277\277\277\261\277\277\277\260\277\277\277\261\277\277\277\261\277\277\277\261\277\277\277\261"...
dst_bytes = 
byte_width = 4096
#1  0x7f70360efa39 in sse2_blt (height=768, width=1024, dest_y=0, 
dest_x=, src_y=, src_x=, 
dst_bpp=, src_bpp=, dst_stride=, 
src_stride=, dst_bits=, src_bits=, 
imp=) at ../../pixman/pixman-sse2.c:4822
No locals.
#2  sse2_composite_copy_area (imp=, info=) at 
../../pixman/pixman-sse2.c:4815
op = 
src_image = 
mask_image = 
dest_image = 
src_x = 
src_y = 
mask_x = 
mask_y = 
dest_x = 
dest_y = 0
width = 1024
height = 768
#3  0x7f7036067711 in pixman_image_composite32 (op=op@entry=PIXMAN_OP_SRC, 
src=, mask=mask@entry=0x0, dest=, 
src_x=src_x@entry=0, src_y=src_y@entry=0, mask_x=0, mask_y=0, dest_x=0, 
dest_y=0, width=1024, height=768) at ../../pixman/pixman.c:707
src_format = 
mask_format = 0
dest_format = PIXMAN_a8r8g8b8
region = {extents = {x1 = 0, y1 = 0, x2 = 1024, y2 = 768}, data = 0x0}
extents = {x1 = 0, y1 = 0, x2 = 1024, y2 = 768}
imp = 0x7f7038e848a0
func = 0x7f70360ef9d0 
info = {op = PIXMAN_OP_SRC, src_image = 0x7f7039267e90, mask_image = 
0x0, dest_image = 0x7f70392603d0, src_x = 0, src_y = 0, mask_x = 0, mask_y = 0, 
dest_x = 0, dest_y = 0, width = 1024, height = 768, src_flags = 42420863, 
mask_flags = 8194, dest_flags = 34032255}
pbox = 0x7fff9ba9e060
n = 0
#4  0x7f70360677e3 in pixman_image_composite (op=op@entry=PIXMAN_OP_SRC, 
src=, mask=mask@entry=0x0, dest=, 
src_x=src_x@entry=0, src_y=src_y@entry=0, mask_x=, 
mask_y=, dest_x=, dest_y=, 
width=, height=) at ../../pixman/pixman.c:730
No locals.
#5  0x7f7030e61ca8 in download_box_no_update (s

Re: [Xen-devel] [PATCH v4] libxl: Add AHCI support for upstream qemu

[Fabio Fantoni]

Il 09/07/2015 12:26, Ian Campbell ha scritto:

On Wed, 2015-07-08 at 16:31 +0200, Fabio Fantoni wrote:

Usage:
hdtype=ide|ahci (default=ide)

If hdtype=ahci adds ich9 disk controller in ahci mode and uses it with
upstream qemu to emulate disks instead of ide.
It doesn't support cdroms which still using ide (cdroms will use
"-device ide-cd" as new qemu parameter)
Ahci requires new qemu parameter but for now other emulated disks cases
remains with old ones (I did it in other patch, not needed by this one)
I did it as libxl parameter disabled by default to avoid possible
problems:
- with save/restore/migration (restoring with ahci a domU that was with
ide instead)
- windows < 8 without pv drivers (a registry key change is needed for
AHCI<->IDE change FWIK to avoid possible blue screen)
- windows XP or older that many not support ahci by default.
Setting AHCI with libxl parameter and default to disabled seems the best
solution.
AHCI increase hvm domUs boot performance. On linux hvm domU I saw up to
only 20% of the previous total boot time, whereas boot time decrease a
lot on W7 domUs for most of boots I have done. Small difference in boot
time compared to ide mode on W8 and newer (probably other xen
improvements or fixes are needed not ahci related)

Signed-off-by: Fabio Fantoni 

---

Changes in v4:
- libxl.h: LIBXL_HAVE_AHCI->LIBXL_HAVE_HVM_AHCI

In retrospect this should have been LIBXL_HAVE_HVM_HDTYPE. Since the
above was my suggestion if you agree I can make that simple change as I
commit.


Yes you can change it directly as this change not risk introduce bug, 
thanks.
Is also Stabellini Acked-by (he did in v2) needed for qemu parameters 
part (not changed in v3 and v4)?

http://lists.xen.org/archives/html/xen-devel/2015-06/msg03551.html
I not added in latest posted version because I don't know if can be 
still valid.




Acked-by: Ian Campbell 

Ian.




___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [PATCH v4] libxl: Add AHCI support for upstream qemu

[Fabio Fantoni]

Usage:
hdtype=ide|ahci (default=ide)

If hdtype=ahci adds ich9 disk controller in ahci mode and uses it with
upstream qemu to emulate disks instead of ide.
It doesn't support cdroms which still using ide (cdroms will use
"-device ide-cd" as new qemu parameter)
Ahci requires new qemu parameter but for now other emulated disks cases
remains with old ones (I did it in other patch, not needed by this one)
I did it as libxl parameter disabled by default to avoid possible
problems:
- with save/restore/migration (restoring with ahci a domU that was with
ide instead)
- windows < 8 without pv drivers (a registry key change is needed for
AHCI<->IDE change FWIK to avoid possible blue screen)
- windows XP or older that many not support ahci by default.
Setting AHCI with libxl parameter and default to disabled seems the best
solution.
AHCI increase hvm domUs boot performance. On linux hvm domU I saw up to
only 20% of the previous total boot time, whereas boot time decrease a
lot on W7 domUs for most of boots I have done. Small difference in boot
time compared to ide mode on W8 and newer (probably other xen
improvements or fixes are needed not ahci related)

Signed-off-by: Fabio Fantoni 

---

Changes in v4:
- libxl.h: LIBXL_HAVE_AHCI->LIBXL_HAVE_HVM_AHCI
- xl_cmdimpl.c: improved using libxl_hdtype_from_string

Changes in v3:
- changed ahci=0|1 to hdtype=ide|ahci
- small doc entry and code improvements as suggested by Ian Campbell

Changes in v2:
- libxl_dm.c: small code style fix
- added vbd-interface.txt changes
---
 docs/man/xl.cfg.pod.5   | 10 ++
 docs/misc/vbd-interface.txt |  6 --
 tools/libxl/libxl.h | 10 ++
 tools/libxl/libxl_create.c  |  3 +++
 tools/libxl/libxl_dm.c  | 11 ++-
 tools/libxl/libxl_types.idl |  6 ++
 tools/libxl/xl_cmdimpl.c|  7 +++
 7 files changed, 50 insertions(+), 3 deletions(-)

diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
index a3e0e2e..ef7d558 100644
--- a/docs/man/xl.cfg.pod.5
+++ b/docs/man/xl.cfg.pod.5
@@ -904,6 +904,16 @@ default is B.
 
 =back
 
+=item B
+
+Select the hd disk type (ide|ahci).
+If hdtype=ahci adds ich9 disk controller in AHCI mode and uses it with
+upstream qemu to emulate disks instead of IDE. It decreases boot time
+but may not be supported by default in Windows xp and older Windows.
+The default is ide.
+
+=back
+
 =head3 Paging
 
 The following options control the mechanisms used to virtualise guest
diff --git a/docs/misc/vbd-interface.txt b/docs/misc/vbd-interface.txt
index f873db0..1c996bf 100644
--- a/docs/misc/vbd-interface.txt
+++ b/docs/misc/vbd-interface.txt
@@ -3,18 +3,20 @@ Xen guest interface
 
 A Xen guest can be provided with block devices.  These are always
 provided as Xen VBDs; for HVM guests they may also be provided as
-emulated IDE or SCSI disks.
+emulated IDE, AHCI or SCSI disks.
 
 The abstract interface involves specifying, for each block device:
 
  * Nominal disk type: Xen virtual disk (aka xvd*, the default); SCSI
-   (sd*); IDE (hd*).
+   (sd*); IDE or AHCI (hd*).
 
For HVM guests, each whole-disk hd* and and sd* device is made
available _both_ via emulated IDE resp. SCSI controller, _and_ as a
Xen VBD.  The HVM guest is entitled to assume that the IDE or SCSI
disks available via the emulated IDE controller target the same
underlying devices as the corresponding Xen VBD (ie, multipath).
+   In hd* case with hdtype=ahci, disk will be AHCI via emulated
+   ich9 disk controller.
 
For PV guests every device is made available to the guest only as a
Xen VBD.  For these domains the type is advisory, for use by the
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index a1c5d15..3a4edba 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -596,6 +596,16 @@ typedef struct libxl__ctx libxl_ctx;
 #define LIBXL_HAVE_SPICE_STREAMINGVIDEO 1
 
 /*
+ * LIBXL_HAVE_HVM_AHCI
+ *
+ * If defined, then the u.hvm structure will contain a enum type:
+ * hdtype. This value defines if ahci support is present.
+ *
+ * If this is not defined, the ahci support is ignored.
+ */
+#define LIBXL_HAVE_HVM_AHCI 1
+
+/*
  * LIBXL_HAVE_DOMAIN_CREATE_RESTORE_PARAMS 1
  *
  * If this is defined, libxl_domain_create_restore()'s API has changed to
diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
index f366a09..290303b 100644
--- a/tools/libxl/libxl_create.c
+++ b/tools/libxl/libxl_create.c
@@ -250,6 +250,9 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,
 if (!b_info->u.hvm.vga.kind)
 b_info->u.hvm.vga.kind = LIBXL_VGA_INTERFACE_TYPE_CIRRUS;
 
+if (!b_info->u.hvm.hdtype)
+b_info->u.hvm.hdtype = LIBXL_HDTYPE_IDE;
+
 switch (b_info->device_model_version) {
 case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN_TRADITIONAL:
 switch (b_info->u.hvm.vga.kind) {
diff --git a/tools/libxl/libxl_dm.c b/tools/libxl/libxl_dm.c
index 317a8eb..ad434

Re: [Xen-devel] [PATCH v5 00/16] x86/hvm: I/O emulation cleanup and fix

[Fabio Fantoni]

Il 30/06/2015 16:48, Fabio Fantoni ha scritto:

Il 30/06/2015 15:05, Paul Durrant ha scritto:
This patch series re-works much of the code involved in emulation of 
port

and memory mapped I/O for HVM guests.

The code has become very convoluted and, at least by inspection, certain
emulations will apparently malfunction.

The series is broken down into 16 patches (which are also available in
my xenbits repo: http://xenbits.xen.org/gitweb/?p=people/pauldu/xen.git
on the emulation32 branch).

Previous changelog
--

v2:
  - Removed bogus assertion from patch #15
  - Re-worked patch #17 after basic testing of back-port onto XenServer

v3:
  - Addressed comments from Jan
  - Re-ordered series to bring a couple of more trivial patches to the
front
  - Backport to XenServer (4.5) now passing automated tests
  - Tested on unstable with QEMU upstream and trad, with and without
HAP (to force shadow emulation)

v4:
  - Removed previous patch (make sure translated MMIO reads or
writes fall within a page) and rebased rest of series.
  - Address Jan's comments on patch #1

Changelog (now per-patch)
-

0001-x86-hvm-make-sure-emulation-is-retried-if-domain-is-.patch

This is a fix for an issue on staging reported by Don Slutz

0002-x86-hvm-remove-multiple-open-coded-chunking-loops.patch

v5: Addressed further comments from Jan

0003-x86-hvm-change-hvm_mmio_read_t-and-hvm_mmio_write_t-.patch

v5: New patch to tidy up types

0004-x86-hvm-restrict-port-numbers-to-uint16_t-and-sizes-.patch

v5: New patch to tidy up more types

0005-x86-hvm-unify-internal-portio-and-mmio-intercepts.patch

v5: Addressed further comments from Jan and simplified implementation
 by passing ioreq_t to accept() function

0006-x86-hvm-add-length-to-mmio-check-op.patch

v5: Simplified by leaving mmio_check() implementation alone and
 calling to check last byte if first-byte check passes

0007-x86-hvm-unify-dpci-portio-intercept-with-standard-po.patch

v5: Addressed further comments from Jan

0008-x86-hvm-unify-stdvga-mmio-intercept-with-standard-mm.patch

v5: Fixed semantic problems pointed out by Jan

0009-x86-hvm-limit-reps-to-avoid-the-need-to-handle-retry.patch

v5: Addressed further comments from Jan

0010-x86-hvm-only-call-hvm_io_assist-from-hvm_wait_for_io.patch

v5: Added Jan's acked-by

0011-x86-hvm-split-I-O-completion-handling-from-state-mod.patch

v5: Confirmed call to msix_write_completion() is in the correct place.

0012-x86-hvm-remove-HVMIO_dispatched-I-O-state.patch

v5: Added some extra comments to the commit

0013-x86-hvm-remove-hvm_io_state-enumeration.patch

v5: Added Jan's acked-by

0014-x86-hvm-use-ioreq_t-to-track-in-flight-state.patch

v5: Added missing hunk with call to handle_pio()

0015-x86-hvm-always-re-emulate-I-O-from-a-buffer.patch

v5: Added Jan's acked-by

0016-x86-hvm-track-large-memory-mapped-accesses-by-buffer.patch

v5: Fixed to cache up three distict I/O emulations per instruction

Testing
---

The series was been back-ported to staging-4.5 and then dropped onto the
XenServer (Dundee) patch queue. All automated branch-safety tests pass.

The series as-is has been manually tested with a Windows 7 (32-bit) VM
using upstream QEMU.




Thanks for your work.
I did some very fast tests, no regression found, on my linux domUs qxl 
is still not working and I'm unable to debug it.
@Jim Fehlig: can you try if qxl is still working at least on suse 
dom0/domU after this serie please?


Can someone tell me how to debug the qxl problem now that qemu don't 
crash anymore but remain at 100% cpu without nothing about in dom0 
logs (if exists) please?


Thanks for any reply and sorry for my bad english.


I don't have knowledge about x86 emulation but I'm trying desperately to 
find the cause of such problems which persists despite this serie of 
patches.
In latest xengt xen patches I saw this patch: "vgt: add support of 
emulating SSE2 instruction MOVD"

https://github.com/01org/XenGT-Preview-xen/commit/f2bad31f80f698a452c37cb39841da8e4f69350f
This xengt patch is still based on xen 4.5 instead, and x86_emulate.c is 
different but I noticed some strange things looking at it and also 
comparing it with upstream staging...


xengt add this:

case 0x7e: /* movd xmm,mm/mm32 */
this seems still missing in upstream staging, should it be added? Should 
I try the patch with latest upstream xen or does it need other changes?

There is also this:

ea.bytes = (b == 0x7e ? 4: 16);

An sse2 istruction of 4 byte seems strange to me, is it right?


In upstream I saw this:

case 0xe7: /* movntq mm,m64 */
/* {,v}movntdq xmm,m128 */
/* vmovntdq ymm,m256 */
 fail_if(ea.type != OP_MEM);
 fail_if(vex.pfx == vex_f3);
 /* fall through */


In the last qemu crash I had with qxl on linux I got this:

#0  __memset_sse2 () at ../sysdeps/x86_64/multiarch/../memset.S:908
Latest istruction:

Re: [Xen-devel] [PATCH v3] libxl: Add AHCI support for upstream qemu

[Fabio Fantoni]

Il 26/06/2015 13:40, Fabio Fantoni ha scritto:

Usage:
hdtype=ide|ahci (default=ide)

If hdtype=ahci adds ich9 disk controller in ahci mode and uses it with
upstream qemu to emulate disks instead of ide.
It doesn't support cdroms which still using ide (cdroms will use
"-device ide-cd" as new qemu parameter)
Ahci requires new qemu parameter but for now other emulated disks cases
remains with old ones (I did it in other patch, not needed by this one)
I did it as libxl parameter disabled by default to avoid possible
problems:
- with save/restore/migration (restoring with ahci a domU that was with
ide instead)
- windows < 8 without pv drivers (a registry key change is needed for
AHCI<->IDE change FWIK to avoid possible blue screen)
- windows XP or older that many not support ahci by default.
Setting AHCI with libxl parameter and default to disabled seems the best
solution.
AHCI increase hvm domUs boot performance. On linux hvm domU I saw up to
only 20% of the previous total boot time, whereas boot time decrease a
lot on W7 domUs for most of boots I have done. Small difference in boot
time compared to ide mode on W8 and newer (probably other xen
improvements or fixes are needed not ahci related)

Signed-off-by: Fabio Fantoni 

---

Changes in v3:
- changed ahci=0|1 to hdtype=ide|ahci
- small doc entry and code improvements as suggested by Ian Campbell


Ping...
Features freeze of xen 4.6 is near, can someone review this new version 
please?




Changes in v2:
- libxl_dm.c: small code style fix
- added vbd-interface.txt changes
---
  docs/man/xl.cfg.pod.5   | 10 ++
  docs/misc/vbd-interface.txt |  6 --
  tools/libxl/libxl.h | 10 ++
  tools/libxl/libxl_create.c  |  3 +++
  tools/libxl/libxl_dm.c  | 11 ++-
  tools/libxl/libxl_types.idl |  6 ++
  tools/libxl/xl_cmdimpl.c| 11 +++
  7 files changed, 54 insertions(+), 3 deletions(-)

diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
index a3e0e2e..ef7d558 100644
--- a/docs/man/xl.cfg.pod.5
+++ b/docs/man/xl.cfg.pod.5
@@ -904,6 +904,16 @@ default is B.
  
  =back
  
+=item B

+
+Select the hd disk type (ide|ahci).
+If hdtype=ahci adds ich9 disk controller in AHCI mode and uses it with
+upstream qemu to emulate disks instead of IDE. It decreases boot time
+but may not be supported by default in Windows xp and older Windows.
+The default is ide.
+
+=back
+
  =head3 Paging
  
  The following options control the mechanisms used to virtualise guest

diff --git a/docs/misc/vbd-interface.txt b/docs/misc/vbd-interface.txt
index f873db0..1c996bf 100644
--- a/docs/misc/vbd-interface.txt
+++ b/docs/misc/vbd-interface.txt
@@ -3,18 +3,20 @@ Xen guest interface
  
  A Xen guest can be provided with block devices.  These are always

  provided as Xen VBDs; for HVM guests they may also be provided as
-emulated IDE or SCSI disks.
+emulated IDE, AHCI or SCSI disks.
  
  The abstract interface involves specifying, for each block device:
  
   * Nominal disk type: Xen virtual disk (aka xvd*, the default); SCSI

-   (sd*); IDE (hd*).
+   (sd*); IDE or AHCI (hd*).
  
 For HVM guests, each whole-disk hd* and and sd* device is made

 available _both_ via emulated IDE resp. SCSI controller, _and_ as a
 Xen VBD.  The HVM guest is entitled to assume that the IDE or SCSI
 disks available via the emulated IDE controller target the same
 underlying devices as the corresponding Xen VBD (ie, multipath).
+   In hd* case with hdtype=ahci, disk will be AHCI via emulated
+   ich9 disk controller.
  
 For PV guests every device is made available to the guest only as a

 Xen VBD.  For these domains the type is advisory, for use by the
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index 0a7913b..6b79ce9 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -596,6 +596,16 @@ typedef struct libxl__ctx libxl_ctx;
  #define LIBXL_HAVE_SPICE_STREAMINGVIDEO 1
  
  /*

+ * LIBXL_HAVE_AHCI
+ *
+ * If defined, then the u.hvm structure will contain a enum type:
+ * hdtype. This value defines if ahci support is present.
+ *
+ * If this is not defined, the ahci support is ignored.
+ */
+#define LIBXL_HAVE_AHCI 1
+
+/*
   * LIBXL_HAVE_DOMAIN_CREATE_RESTORE_PARAMS 1
   *
   * If this is defined, libxl_domain_create_restore()'s API has changed to
diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
index 86384d2..0ff7046 100644
--- a/tools/libxl/libxl_create.c
+++ b/tools/libxl/libxl_create.c
@@ -250,6 +250,9 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,
  if (!b_info->u.hvm.vga.kind)
  b_info->u.hvm.vga.kind = LIBXL_VGA_INTERFACE_TYPE_CIRRUS;
  
+if (!b_info->u.hvm.hdtype)

+b_info->u.hvm.hdtype = LIBXL_HDTYPE_IDE;
+
  switch (b_info->device_model_version) {
  case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN_TRADITIONAL:
  switch (b_info->u.hvm.vga.kind) {
diff --git a/

Re: [Xen-devel] [PATCH v3 2/6] libxl: do not add a vkb backend to hvm guests

[Fabio Fantoni]

Il 01/07/2015 12:55, Roger Pau Monné ha scritto:

El 01/07/15 a les 12.29, Stefano Stabellini ha escrit:

Given that xen-fbfront is likely to go away for HVM guests, I wouldn't
be opposed to stop the driver initialization in Linux on x86/HVM. Unless
Roger's work on HVMlite is going to need xen-fbfront again, but in that
case we'll be able to distinguish a regular HVM guest from an HVMlite
guest, I think.

I haven't get to that point yet, but yes, it seems useful for HVMlite
guests because we won't have an emulated VGA card any more.


If you don't want have emulated vga you can already do it fast in xl 
cfg: vga="none"
Note: emulated vga is used by spice and vnc of hvm domUs, without 
connecting with them you'll have black screen.




HVMlite guest can be easily identified because they set the device model
to none:

device_model_version == LIBXL_DEVICE_MODEL_VERSION_NONE

For HVMlite.

Roger.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v5] run QEMU as non-root

[Fabio Fantoni]

Il 01/07/2015 02:04, Jim Fehlig ha scritto:

On 06/30/2015 07:55 AM, Stefano Stabellini wrote:

Try to use "xen-qemudepriv-domid$domid" first, then
"xen-qemudepriv-shared" and root if everything else fails.

The uids need to be manually created by the user or, more likely, by the
xen package maintainer.

To actually secure QEMU when running in Dom0, we need at least to
deprivilege the privcmd and xenstore interfaces, this is just the first
step in that direction.

Signed-off-by: Stefano Stabellini 

---
Changes in v5:
- improve wording in doc
- fix wording in warning message
- fix example in doc
- drop xen-qemudepriv-$domname

Changes in v4:
- rename qemu-deprivilege to qemu-deprivilege.txt
- add a note about qemu-deprivilege.txt to INSTALL
- instead of xen-qemudepriv-base + $domid, try 
xen-qemudepriv-domid$domid

- introduce libxl__dm_runas_helper to make the code nicer

Changes in v3:
- clarify doc
- handle errno == ERANGE
---
  INSTALL|7 ++
  docs/misc/qemu-deprivilege.txt |   26 +
  tools/libxl/libxl_dm.c |   50 


  tools/libxl/libxl_internal.h   |4 
  4 files changed, 87 insertions(+)
  create mode 100644 docs/misc/qemu-deprivilege.txt

diff --git a/INSTALL b/INSTALL
index a0f2e7b..fe83c20 100644
--- a/INSTALL
+++ b/INSTALL
@@ -297,6 +297,13 @@ systemctl enable xendomains.service
  systemctl enable xen-watchdog.service
+QEMU Deprivilege
+
+It is recommended to run QEMU as non-root.
+See docs/misc/qemu-deprivilege.txt for an explanation on what you need
+to do at installation time to run QEMU as a dedicated user.
+
+
  History of options
  ==
  diff --git a/docs/misc/qemu-deprivilege.txt 
b/docs/misc/qemu-deprivilege.txt

new file mode 100644
index 000..783874b
--- /dev/null
+++ b/docs/misc/qemu-deprivilege.txt
@@ -0,0 +1,26 @@
+For security reasons, libxl tries to create QEMU as non-root.
+Libxl looks for the following users in this order:
+
+1) a user named "xen-qemuuser-domid$domid",
+Where $domid is the domid of the domain being created.
+This requires the reservation of 65535 uids from xen-qemuuser-domid1
+to xen-qemuuser-domid65535. To use this mechanism, you might want to
+create a large number of users at installation time. For example:
+
+for ((i=1; i<65536; i++))
+do
+adduser --system xen-qemuuser-domid$i
+done
+
+
+2) a user named "xen-qemuuser-shared"
+As a fall back if both 1) and 2) fail, libxl will use a single user for
+all QEMU instances. The user is named xen-qemuuser-shared. This is
+less secure but still better than running QEMU as root. Using this 
is as

+simple as creating just one more user on your host:
+
+adduser --system xen-qemuuser-shared
+
+
+3) root
+As a last resort, libxl will start QEMU as root.


The more I think about it, the more I feel libxl is the wrong place 
for this policy. As mentioned earlier [0], libvirt allows apps to 
control the user:group policy. It is already supported by the qemu 
driver. It could be used by the libxl driver to inform libxl that the 
emulator (and other binaries?) it spawns should be in the context of 
the specified user:group.


Regards,
Jim


In the patch I saw this:

+if (user) {
+flexarray_append(dm_args, "-runas");
+flexarray_append(dm_args, user);
+}

So seems that already use qemu parameter for it.

What you mean is to add also the possibility to specify user for use it 
from libvirt and similar instead doing a different thing to support the 
xen specific one? If yes I also think can be a good idea. In that case 
can you explain a better way to do it? Probably Stabellini didn't done 
it because don't know other similar implementation already present in 
other systems that use qemu and how are good and used.





[0] 
http://lists.xenproject.org/archives/html/xen-devel/2015-05/msg02139.html



diff --git a/tools/libxl/libxl_dm.c b/tools/libxl/libxl_dm.c
index 0c6408d..66d638d 100644
--- a/tools/libxl/libxl_dm.c
+++ b/tools/libxl/libxl_dm.c
@@ -19,6 +19,8 @@
#include "libxl_internal.h"
  #include 
+#include 
+#include 
static const char *libxl_tapif_script(libxl__gc *gc)
  {
@@ -418,6 +420,33 @@ static char *dm_spice_options(libxl__gc *gc,
  return opt;
  }
  +/* return 1 if the user was found, 0 if it was not, -1 on error */
+static int libxl__dm_runas_helper(libxl__gc *gc, char *username)
+{
+struct passwd pwd, *user = NULL;
+char *buf = NULL;
+long buf_size;
+
+buf_size = sysconf(_SC_GETPW_R_SIZE_MAX);
+if (buf_size < 0) {
+LOGE(ERROR, "sysconf(_SC_GETPW_R_SIZE_MAX) returned error %ld",
+buf_size);
+return -1;
+}
+
+retry:
+buf = libxl__realloc(gc, buf, buf_size);
+errno = 0;
+getpwnam_r(username, &pwd, buf, buf_size, &user);
+if (user != NULL)
+return 1;
+if (errno == ERANGE) {
+buf_size += 128;
+goto retry;
+}
+retur

Re: [Xen-devel] [PATCH v5 00/16] x86/hvm: I/O emulation cleanup and fix

[Fabio Fantoni]

Il 30/06/2015 15:05, Paul Durrant ha scritto:

This patch series re-works much of the code involved in emulation of port
and memory mapped I/O for HVM guests.

The code has become very convoluted and, at least by inspection, certain
emulations will apparently malfunction.

The series is broken down into 16 patches (which are also available in
my xenbits repo: http://xenbits.xen.org/gitweb/?p=people/pauldu/xen.git
on the emulation32 branch).

Previous changelog
--

v2:
  - Removed bogus assertion from patch #15
  - Re-worked patch #17 after basic testing of back-port onto XenServer

v3:
  - Addressed comments from Jan
  - Re-ordered series to bring a couple of more trivial patches to the
front
  - Backport to XenServer (4.5) now passing automated tests
  - Tested on unstable with QEMU upstream and trad, with and without
HAP (to force shadow emulation)

v4:
  - Removed previous patch (make sure translated MMIO reads or
writes fall within a page) and rebased rest of series.
  - Address Jan's comments on patch #1

Changelog (now per-patch)
-

0001-x86-hvm-make-sure-emulation-is-retried-if-domain-is-.patch

This is a fix for an issue on staging reported by Don Slutz

0002-x86-hvm-remove-multiple-open-coded-chunking-loops.patch

v5: Addressed further comments from Jan

0003-x86-hvm-change-hvm_mmio_read_t-and-hvm_mmio_write_t-.patch

v5: New patch to tidy up types

0004-x86-hvm-restrict-port-numbers-to-uint16_t-and-sizes-.patch

v5: New patch to tidy up more types

0005-x86-hvm-unify-internal-portio-and-mmio-intercepts.patch

v5: Addressed further comments from Jan and simplified implementation
 by passing ioreq_t to accept() function

0006-x86-hvm-add-length-to-mmio-check-op.patch

v5: Simplified by leaving mmio_check() implementation alone and
 calling to check last byte if first-byte check passes

0007-x86-hvm-unify-dpci-portio-intercept-with-standard-po.patch

v5: Addressed further comments from Jan

0008-x86-hvm-unify-stdvga-mmio-intercept-with-standard-mm.patch

v5: Fixed semantic problems pointed out by Jan

0009-x86-hvm-limit-reps-to-avoid-the-need-to-handle-retry.patch

v5: Addressed further comments from Jan

0010-x86-hvm-only-call-hvm_io_assist-from-hvm_wait_for_io.patch

v5: Added Jan's acked-by

0011-x86-hvm-split-I-O-completion-handling-from-state-mod.patch

v5: Confirmed call to msix_write_completion() is in the correct place.

0012-x86-hvm-remove-HVMIO_dispatched-I-O-state.patch

v5: Added some extra comments to the commit

0013-x86-hvm-remove-hvm_io_state-enumeration.patch

v5: Added Jan's acked-by

0014-x86-hvm-use-ioreq_t-to-track-in-flight-state.patch

v5: Added missing hunk with call to handle_pio()

0015-x86-hvm-always-re-emulate-I-O-from-a-buffer.patch

v5: Added Jan's acked-by

0016-x86-hvm-track-large-memory-mapped-accesses-by-buffer.patch

v5: Fixed to cache up three distict I/O emulations per instruction

Testing
---

The series was been back-ported to staging-4.5 and then dropped onto the
XenServer (Dundee) patch queue. All automated branch-safety tests pass.

The series as-is has been manually tested with a Windows 7 (32-bit) VM
using upstream QEMU.




Thanks for your work.
I did some very fast tests, no regression found, on my linux domUs qxl 
is still not working and I'm unable to debug it.
@Jim Fehlig: can you try if qxl is still working at least on suse 
dom0/domU after this serie please?


Can someone tell me how to debug the qxl problem now that qemu don't 
crash anymore but remain at 100% cpu without nothing about in dom0 logs 
(if exists) please?


Thanks for any reply and sorry for my bad english.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [PATCH v3] libxl: Add AHCI support for upstream qemu

[Fabio Fantoni]

Usage:
hdtype=ide|ahci (default=ide)

If hdtype=ahci adds ich9 disk controller in ahci mode and uses it with
upstream qemu to emulate disks instead of ide.
It doesn't support cdroms which still using ide (cdroms will use
"-device ide-cd" as new qemu parameter)
Ahci requires new qemu parameter but for now other emulated disks cases
remains with old ones (I did it in other patch, not needed by this one)
I did it as libxl parameter disabled by default to avoid possible
problems:
- with save/restore/migration (restoring with ahci a domU that was with
ide instead)
- windows < 8 without pv drivers (a registry key change is needed for
AHCI<->IDE change FWIK to avoid possible blue screen)
- windows XP or older that many not support ahci by default.
Setting AHCI with libxl parameter and default to disabled seems the best
solution.
AHCI increase hvm domUs boot performance. On linux hvm domU I saw up to
only 20% of the previous total boot time, whereas boot time decrease a
lot on W7 domUs for most of boots I have done. Small difference in boot
time compared to ide mode on W8 and newer (probably other xen
improvements or fixes are needed not ahci related)

Signed-off-by: Fabio Fantoni 

---

Changes in v3:
- changed ahci=0|1 to hdtype=ide|ahci
- small doc entry and code improvements as suggested by Ian Campbell

Changes in v2:
- libxl_dm.c: small code style fix
- added vbd-interface.txt changes
---
 docs/man/xl.cfg.pod.5   | 10 ++
 docs/misc/vbd-interface.txt |  6 --
 tools/libxl/libxl.h | 10 ++
 tools/libxl/libxl_create.c  |  3 +++
 tools/libxl/libxl_dm.c  | 11 ++-
 tools/libxl/libxl_types.idl |  6 ++
 tools/libxl/xl_cmdimpl.c| 11 +++
 7 files changed, 54 insertions(+), 3 deletions(-)

diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
index a3e0e2e..ef7d558 100644
--- a/docs/man/xl.cfg.pod.5
+++ b/docs/man/xl.cfg.pod.5
@@ -904,6 +904,16 @@ default is B.
 
 =back
 
+=item B
+
+Select the hd disk type (ide|ahci).
+If hdtype=ahci adds ich9 disk controller in AHCI mode and uses it with
+upstream qemu to emulate disks instead of IDE. It decreases boot time
+but may not be supported by default in Windows xp and older Windows.
+The default is ide.
+
+=back
+
 =head3 Paging
 
 The following options control the mechanisms used to virtualise guest
diff --git a/docs/misc/vbd-interface.txt b/docs/misc/vbd-interface.txt
index f873db0..1c996bf 100644
--- a/docs/misc/vbd-interface.txt
+++ b/docs/misc/vbd-interface.txt
@@ -3,18 +3,20 @@ Xen guest interface
 
 A Xen guest can be provided with block devices.  These are always
 provided as Xen VBDs; for HVM guests they may also be provided as
-emulated IDE or SCSI disks.
+emulated IDE, AHCI or SCSI disks.
 
 The abstract interface involves specifying, for each block device:
 
  * Nominal disk type: Xen virtual disk (aka xvd*, the default); SCSI
-   (sd*); IDE (hd*).
+   (sd*); IDE or AHCI (hd*).
 
For HVM guests, each whole-disk hd* and and sd* device is made
available _both_ via emulated IDE resp. SCSI controller, _and_ as a
Xen VBD.  The HVM guest is entitled to assume that the IDE or SCSI
disks available via the emulated IDE controller target the same
underlying devices as the corresponding Xen VBD (ie, multipath).
+   In hd* case with hdtype=ahci, disk will be AHCI via emulated
+   ich9 disk controller.
 
For PV guests every device is made available to the guest only as a
Xen VBD.  For these domains the type is advisory, for use by the
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index 0a7913b..6b79ce9 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -596,6 +596,16 @@ typedef struct libxl__ctx libxl_ctx;
 #define LIBXL_HAVE_SPICE_STREAMINGVIDEO 1
 
 /*
+ * LIBXL_HAVE_AHCI
+ *
+ * If defined, then the u.hvm structure will contain a enum type:
+ * hdtype. This value defines if ahci support is present.
+ *
+ * If this is not defined, the ahci support is ignored.
+ */
+#define LIBXL_HAVE_AHCI 1
+
+/*
  * LIBXL_HAVE_DOMAIN_CREATE_RESTORE_PARAMS 1
  *
  * If this is defined, libxl_domain_create_restore()'s API has changed to
diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
index 86384d2..0ff7046 100644
--- a/tools/libxl/libxl_create.c
+++ b/tools/libxl/libxl_create.c
@@ -250,6 +250,9 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,
 if (!b_info->u.hvm.vga.kind)
 b_info->u.hvm.vga.kind = LIBXL_VGA_INTERFACE_TYPE_CIRRUS;
 
+if (!b_info->u.hvm.hdtype)
+b_info->u.hvm.hdtype = LIBXL_HDTYPE_IDE;
+
 switch (b_info->device_model_version) {
 case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN_TRADITIONAL:
 switch (b_info->u.hvm.vga.kind) {
diff --git a/tools/libxl/libxl_dm.c b/tools/libxl/libxl_dm.c
index 33f9ce6..7eba4e6 100644
--- a/tools/libxl/libxl_dm.c
+++ b/tools/libxl/libxl_dm.c
@@ -818,6 +818,8 @@ static int libxl__build_device_mode

Re: [Xen-devel] [PATCH v2] libxl: Add AHCI support for upstream qemu

[Fabio Fantoni]

Il 25/06/2015 14:18, Ian Campbell ha scritto:

On Thu, 2015-06-25 at 12:58 +0100, Stefano Stabellini wrote:

On Thu, 25 Jun 2015, Fabio Fantoni wrote:

Il 25/06/2015 12:21, Ian Campbell ha scritto:

On Tue, 2015-06-23 at 11:15 +0200, Fabio Fantoni wrote:

Usage:
ahci=0|1 (default=0)

I think a global rather than per disk option is OK (I can't think why a
user would want to mix and match) but maybe we should consider using an
enum (with values ide and ahci, defaulting to ide in libxl) so that we
can add support for whatever fancy new disk controller everyone is using
in 5 years time?

ahci was added 4 years ago in qemu and I don't know of newer similar
tecnology,

It maybe hasn't been invented yet (although Malcolm suggested nvme,
which is plausible).


  in the case of enum probably shold be more generic for include more

future possibility or I'm wrong? in that case what can be the name?
@stabellini and other developer: any advice about this?

I don't know of any other block technologies that would use "hd" as
block device names. Virtio-blk uses "vd", so it couldn't be confused.
However for the sake of being future proof, it might make sense to
introduce an enum, maybe something like "hdtype"?

enum hdtype {
 ide,
 ahci,
}

then in the config file:

hdtype=ahci

That's the sort of thing I was imagining, yes.

Ian.



Good, I'll do now in v3

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2] libxl: Add AHCI support for upstream qemu

[Fabio Fantoni]

Il 25/06/2015 12:21, Ian Campbell ha scritto:

On Tue, 2015-06-23 at 11:15 +0200, Fabio Fantoni wrote:

Usage:
ahci=0|1 (default=0)

I think a global rather than per disk option is OK (I can't think why a
user would want to mix and match) but maybe we should consider using an
enum (with values ide and ahci, defaulting to ide in libxl) so that we
can add support for whatever fancy new disk controller everyone is using
in 5 years time?


ahci was added 4 years ago in qemu and I don't know of newer similar 
tecnology, in the case of enum probably shold be more generic for 
include more future possibility or I'm wrong? in that case what can be 
the name?

@stabellini and other developer: any advice about this?




If enabled adds ich9 disk controller in ahci mode and uses it with
upstream qemu to emulate disks instead of ide.
It doesn't support cdroms which still using ide (cdroms will use
"-device ide-cd" as new qemu parameter)

I don't follow this reference to "will use" and a new qemu parameter,
there seems to be nothing corresponding in this patch AFAICT.


Is a "FAQ" for explain why cdrom is still ide and ahci use "new" qemu 
parameters (added in qemu 5 or 6 years ago but still not fully used in 
xen... :( )





Ahci requires new qemu parameter but for now other emulated disks cases
remains with old ones (I did it in other patch, not needed by this one)

You can drop the reference to the other patch I think.


I thinked also in this case that can be useful as "FAQ".




I did it as libxl parameter disabled by default to avoid possible
problems:
- with save/restore/migration (restoring with ahci a domU that was with
ide instead)
- windows < 8 without pv drivers (a registry key change is needed for
AHCI<->IDE change FWIK to avoid possible blue screen)

What is "FWIK"?


FWIK= for what I know
on physical computer I see that change ide<->ahci in bios without change 
in windows do a blue screen on boot, on xen domUs with pv driver don't 
do but without probably will do the same. In any case is not a xen problem.





- windows XP or older that many not support ahci by default.
Setting AHCI with libxl parameter and default to disabled seems the best
solution.
AHCI increase hvm domUs boot performance. On linux hvm domU I saw up to
only 20% of the previous total boot time, whereas boot time decrease a
lot on W7 domUs for most of boots I have done. Small difference in boot
time compared to ide mode on W8 and newer (probably other xen
improvements or fixes are needed not ahci related)

Signed-off-by: Fabio Fantoni 

---

Changes in v2:
- libxl_dm.c: small code style fix
- added vbd-interface.txt changes
---
  docs/man/xl.cfg.pod.5   |  9 +
  docs/misc/vbd-interface.txt |  5 +++--
  tools/libxl/libxl.h | 10 ++
  tools/libxl/libxl_create.c  |  1 +
  tools/libxl/libxl_dm.c  | 10 +-
  tools/libxl/libxl_types.idl |  1 +
  tools/libxl/xl_cmdimpl.c|  1 +
  7 files changed, 34 insertions(+), 3 deletions(-)

diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
index a3e0e2e..7e16123 100644
--- a/docs/man/xl.cfg.pod.5
+++ b/docs/man/xl.cfg.pod.5
@@ -904,6 +904,15 @@ default is B.
  
  =back
  
+=item B

"=item B" please.


I'll do in v3




+If enabled adds ich9 disk controller in ahci mode and uses it with
+upstream qemu to emulate disks instead of ide. It decrease boot time but

"decreases"


I'll do in v3




+may be not supported by default in windows xp and older windows.
+The default is disabled (0).

"may not be supported".

I think AHCI and IDE should be capitalised in the text (not the option
name). As should "Windows XP" and "Windows"


I'll do in v3




+
+=back
+
  =head3 Paging
  
  The following options control the mechanisms used to virtualise guest

diff --git a/docs/misc/vbd-interface.txt b/docs/misc/vbd-interface.txt
index f873db0..afb6846 100644
--- a/docs/misc/vbd-interface.txt
+++ b/docs/misc/vbd-interface.txt
@@ -3,18 +3,19 @@ Xen guest interface
  
  A Xen guest can be provided with block devices.  These are always

  provided as Xen VBDs; for HVM guests they may also be provided as
-emulated IDE or SCSI disks.
+emulated IDE, AHCI or SCSI disks.
  
  The abstract interface involves specifying, for each block device:
  
   * Nominal disk type: Xen virtual disk (aka xvd*, the default); SCSI

-   (sd*); IDE (hd*).
+   (sd*); IDE or AHCI (hd*).
  
 For HVM guests, each whole-disk hd* and and sd* device is made

 available _both_ via emulated IDE resp. SCSI controller, _and_ as a
 Xen VBD.  The HVM guest is entitled to assume that the IDE or SCSI
 disks available via the emulated IDE controller target the same
 underlying devices as the corresponding Xen VBD (ie, multipath).
+   In hd* case with ahci=1, disk will be AHCI via emulated ich9 controller.
  
 For PV gu

Re: [Xen-devel] [PATCH v3 00/18] x86/hvm: I/O emulation cleanup and fix

[Fabio Fantoni]

Il 23/06/2015 12:39, Paul Durrant ha scritto:

This patch series re-works much of the code involved in emulation of port
and memory mapped I/O for HVM guests.

The code has become very convoluted and, at least by inspection, certain
emulations will apparently malfunction.

The series is broken down into 18 patches (which are also available in
my xenbits repo: http://xenbits.xen.org/gitweb/?p=people/pauldu/xen.git
on the emulation25 branch) as follows:

0001-x86-hvm-simplify-hvmemul_do_io.patch
0002-x86-hvm-remove-hvm_io_pending-check-in-hvmemul_do_io.patch
0003-x86-hvm-remove-extraneous-parameter-from-hvmtrace_io.patch
0004-x86-hvm-make-sure-translated-MMIO-reads-or-writes-fa.patch
0005-x86-hvm-remove-multiple-open-coded-chunking-loops.patch
0006-x86-hvm-re-name-struct-hvm_mmio_handler-to-hvm_mmio_.patch
0007-x86-hvm-unify-internal-portio-and-mmio-intercepts.patch
0008-x86-hvm-add-length-to-mmio-check-op.patch
0009-x86-hvm-unify-dpci-portio-intercept-with-standard-po.patch
0010-x86-hvm-unify-stdvga-mmio-intercept-with-standard-mm.patch
0011-x86-hvm-revert-82ed8716b-fix-direct-PCI-port-I-O-emu.patch
0012-x86-hvm-only-call-hvm_io_assist-from-hvm_wait_for_io.patch
0013-x86-hvm-split-I-O-completion-handling-from-state-mod.patch
0014-x86-hvm-remove-HVMIO_dispatched-I-O-state.patch
0015-x86-hvm-remove-hvm_io_state-enumeration.patch
0016-x86-hvm-use-ioreq_t-to-track-in-flight-state.patch
0017-x86-hvm-always-re-emulate-I-O-from-a-buffer.patch
0018-x86-hvm-track-large-memory-mapped-accesses-by-buffer.patch

v2:
  - Removed bogus assertion from patch 16
  - Re-worked patch 17 after basic testing of back-port onto XenServer

v3:
  - Addressed comments from Jan
  - Re-ordered series to bring a couple of more trivial patches to the
front
  - Backport to XenServer (4.5) now passing automated tests
  - Tested on unstable with QEMU upstream and trad, with and without
HAP (to force shadow emulation)




Thanks, I tested this serie emulation25 branch of your git.
I not found regression for now but trying qxl linux domUs that before 
caused qemu crash on sse2 operations now qemu didn't crashed but screen 
showed nothing and qemu process of domU with cpu at 100% and I'm unable 
to found details in dom0 about the problem, seems similar to 
xen-unstable of one year ago.
Someone can tell me a way to know if sse2 istructions are now fully 
working (including operations with videoram) and/or how to debug this 
problem without nothing that crashes where take datas with gdb?


Thanks for any reply and sorry for my bad english.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2] libxl: Add AHCI support for upstream qemu

[Fabio Fantoni]

Usage:
ahci=0|1 (default=0)

If enabled adds ich9 disk controller in ahci mode and uses it with
upstream qemu to emulate disks instead of ide.
It doesn't support cdroms which still using ide (cdroms will use
"-device ide-cd" as new qemu parameter)
Ahci requires new qemu parameter but for now other emulated disks cases
remains with old ones (I did it in other patch, not needed by this one)
I did it as libxl parameter disabled by default to avoid possible
problems:
- with save/restore/migration (restoring with ahci a domU that was with
ide instead)
- windows < 8 without pv drivers (a registry key change is needed for
AHCI<->IDE change FWIK to avoid possible blue screen)
- windows XP or older that many not support ahci by default.
Setting AHCI with libxl parameter and default to disabled seems the best
solution.
AHCI increase hvm domUs boot performance. On linux hvm domU I saw up to
only 20% of the previous total boot time, whereas boot time decrease a
lot on W7 domUs for most of boots I have done. Small difference in boot
time compared to ide mode on W8 and newer (probably other xen
improvements or fixes are needed not ahci related)

Signed-off-by: Fabio Fantoni 

---

Changes in v2:
- libxl_dm.c: small code style fix
- added vbd-interface.txt changes
---
 docs/man/xl.cfg.pod.5   |  9 +
 docs/misc/vbd-interface.txt |  5 +++--
 tools/libxl/libxl.h | 10 ++
 tools/libxl/libxl_create.c  |  1 +
 tools/libxl/libxl_dm.c  | 10 +-
 tools/libxl/libxl_types.idl |  1 +
 tools/libxl/xl_cmdimpl.c|  1 +
 7 files changed, 34 insertions(+), 3 deletions(-)

diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
index a3e0e2e..7e16123 100644
--- a/docs/man/xl.cfg.pod.5
+++ b/docs/man/xl.cfg.pod.5
@@ -904,6 +904,15 @@ default is B.
 
 =back
 
+=item B
+
+If enabled adds ich9 disk controller in ahci mode and uses it with
+upstream qemu to emulate disks instead of ide. It decrease boot time but
+may be not supported by default in windows xp and older windows.
+The default is disabled (0).
+
+=back
+
 =head3 Paging
 
 The following options control the mechanisms used to virtualise guest
diff --git a/docs/misc/vbd-interface.txt b/docs/misc/vbd-interface.txt
index f873db0..afb6846 100644
--- a/docs/misc/vbd-interface.txt
+++ b/docs/misc/vbd-interface.txt
@@ -3,18 +3,19 @@ Xen guest interface
 
 A Xen guest can be provided with block devices.  These are always
 provided as Xen VBDs; for HVM guests they may also be provided as
-emulated IDE or SCSI disks.
+emulated IDE, AHCI or SCSI disks.
 
 The abstract interface involves specifying, for each block device:
 
  * Nominal disk type: Xen virtual disk (aka xvd*, the default); SCSI
-   (sd*); IDE (hd*).
+   (sd*); IDE or AHCI (hd*).
 
For HVM guests, each whole-disk hd* and and sd* device is made
available _both_ via emulated IDE resp. SCSI controller, _and_ as a
Xen VBD.  The HVM guest is entitled to assume that the IDE or SCSI
disks available via the emulated IDE controller target the same
underlying devices as the corresponding Xen VBD (ie, multipath).
+   In hd* case with ahci=1, disk will be AHCI via emulated ich9 controller.
 
For PV guests every device is made available to the guest only as a
Xen VBD.  For these domains the type is advisory, for use by the
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index 0a7913b..6a3677d 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -596,6 +596,16 @@ typedef struct libxl__ctx libxl_ctx;
 #define LIBXL_HAVE_SPICE_STREAMINGVIDEO 1
 
 /*
+ * LIBXL_HAVE_AHCI
+ *
+ * If defined, then the u.hvm structure will contain a boolean type:
+ * ahci. This value defines if ahci support is present.
+ *
+ * If this is not defined, the ahci support is ignored.
+ */
+#define LIBXL_HAVE_AHCI 1
+
+/*
  * LIBXL_HAVE_DOMAIN_CREATE_RESTORE_PARAMS 1
  *
  * If this is defined, libxl_domain_create_restore()'s API has changed to
diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
index 86384d2..8ca2481 100644
--- a/tools/libxl/libxl_create.c
+++ b/tools/libxl/libxl_create.c
@@ -331,6 +331,7 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,
 libxl_defbool_setdefault(&b_info->u.hvm.nested_hvm, false);
 libxl_defbool_setdefault(&b_info->u.hvm.usb,false);
 libxl_defbool_setdefault(&b_info->u.hvm.xen_platform_pci,   true);
+libxl_defbool_setdefault(&b_info->u.hvm.ahci,   false);
 
 libxl_defbool_setdefault(&b_info->u.hvm.spice.enable, false);
 if (!libxl_defbool_val(b_info->u.hvm.spice.enable) &&
diff --git a/tools/libxl/libxl_dm.c b/tools/libxl/libxl_dm.c
index 33f9ce6..9216028 100644
--- a/tools/libxl/libxl_dm.c
+++ b/tools/libxl/libxl_dm.c
@@ -818,6 +818,8 @@ static int libxl__build_device_model_args_new(libxl__gc *gc,
 flexarray_append(dm_args, libxl__sprintf(gc, "%&quo

Re: [Xen-devel] [PATCH] libxl: Add AHCI support for upstream qemu

[Fabio Fantoni]

Il 22/06/2015 17:32, Stefano Stabellini ha scritto:

On Mon, 22 Jun 2015, Fabio Fantoni wrote:

Usage:
ahci=0|1 (default=0)

If enabled adds ich9 disk controller in ahci mode and uses it with
upstream qemu to emulate disks instead of ide.
It doesn't support cdroms which still using ide (cdroms will use
"-device ide-cd" as new qemu parameter)
Ahci requires new qemu parameter but for now other emulated disks cases
remains with old ones (I did it in other patch, not needed by this one)
I did it as libxl parameter disabled by default to avoid possible
problems:
- with save/restore/migration (restoring with ahci a domU that was with
ide instead)
- windows < 8 without pv drivers (a registry key change is needed for
AHCI<->IDE change FWIK to avoid possible blue screen)
- windows XP or older that many not support ahci by default.
Setting AHCI with libxl parameter and default to disabled seems the best
solution.
AHCI increase hvm domUs boot performance. On linux hvm domU I saw up to
only 20% of the previous total boot time, whereas boot time decrease a
lot on W7 domUs for most of boots I have done. Small difference in boot
time compared to ide mode on W8 and newer (probably other xen
improvements or fixes are needed not ahci related)

Signed-off-by: Fabio Fantoni 

I am OK with this patch going in Xen 4.6.



  docs/man/xl.cfg.pod.5   |  9 +
  tools/libxl/libxl.h | 10 ++
  tools/libxl/libxl_create.c  |  1 +
  tools/libxl/libxl_dm.c  | 10 +-
  tools/libxl/libxl_types.idl |  1 +
  tools/libxl/xl_cmdimpl.c|  1 +
  6 files changed, 31 insertions(+), 1 deletion(-)

diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
index a3e0e2e..7e16123 100644
--- a/docs/man/xl.cfg.pod.5
+++ b/docs/man/xl.cfg.pod.5
@@ -904,6 +904,15 @@ default is B.
  
  =back
  
+=item B

+
+If enabled adds ich9 disk controller in ahci mode and uses it with
+upstream qemu to emulate disks instead of ide. It decrease boot time but
+may be not supported by default in windows xp and older windows.
+The default is disabled (0).
+
+=back
+
  =head3 Paging
  
  The following options control the mechanisms used to virtualise guest

diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index 0a7913b..6a3677d 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -596,6 +596,16 @@ typedef struct libxl__ctx libxl_ctx;
  #define LIBXL_HAVE_SPICE_STREAMINGVIDEO 1
  
  /*

+ * LIBXL_HAVE_AHCI
+ *
+ * If defined, then the u.hvm structure will contain a boolean type:
+ * ahci. This value defines if ahci support is present.
+ *
+ * If this is not defined, the ahci support is ignored.
+ */
+#define LIBXL_HAVE_AHCI 1
+
+/*
   * LIBXL_HAVE_DOMAIN_CREATE_RESTORE_PARAMS 1
   *
   * If this is defined, libxl_domain_create_restore()'s API has changed to
diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
index 86384d2..8ca2481 100644
--- a/tools/libxl/libxl_create.c
+++ b/tools/libxl/libxl_create.c
@@ -331,6 +331,7 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,
  libxl_defbool_setdefault(&b_info->u.hvm.nested_hvm, false);
  libxl_defbool_setdefault(&b_info->u.hvm.usb,false);
  libxl_defbool_setdefault(&b_info->u.hvm.xen_platform_pci,   true);
+libxl_defbool_setdefault(&b_info->u.hvm.ahci,   false);
  
  libxl_defbool_setdefault(&b_info->u.hvm.spice.enable, false);

  if (!libxl_defbool_val(b_info->u.hvm.spice.enable) &&
diff --git a/tools/libxl/libxl_dm.c b/tools/libxl/libxl_dm.c
index 33f9ce6..93f191a 100644
--- a/tools/libxl/libxl_dm.c
+++ b/tools/libxl/libxl_dm.c
@@ -818,6 +818,8 @@ static int libxl__build_device_model_args_new(libxl__gc *gc,
  flexarray_append(dm_args, libxl__sprintf(gc, "%"PRId64, ram_size));
  
  if (b_info->type == LIBXL_DOMAIN_TYPE_HVM) {

+if (libxl_defbool_val(b_info->u.hvm.ahci))
+flexarray_append_pair(dm_args, "-device", "ahci,id=ahci0");
  for (i = 0; i < num_disks; i++) {
  int disk, part;
  int dev_number =
@@ -872,7 +874,13 @@ static int libxl__build_device_model_args_new(libxl__gc 
*gc,
  drive = libxl__sprintf
  (gc, 
"file=%s,if=scsi,bus=0,unit=%d,format=%s,cache=writeback",
   pdev_path, disk, format);
-else if (disk < 4)
+else if (disk < 6 && libxl_defbool_val(b_info->u.hvm.ahci)){

One space missing.

This means that we can now have disks up to hdf. I think you should
update the in-code comment above. I think you should also update
docs/txt/misc/vbd-interface.txt, explaining that hd* mean AHCI if ahci=1
is specified.


Thanks for reply, I'll do in v2.





+flexarray_vappend(dm_args, "-drive",
+
GCSPRINTF(&quo

[Xen-devel] [PATCH] libxl: Add AHCI support for upstream qemu

[Fabio Fantoni]

Usage:
ahci=0|1 (default=0)

If enabled adds ich9 disk controller in ahci mode and uses it with
upstream qemu to emulate disks instead of ide.
It doesn't support cdroms which still using ide (cdroms will use
"-device ide-cd" as new qemu parameter)
Ahci requires new qemu parameter but for now other emulated disks cases
remains with old ones (I did it in other patch, not needed by this one)
I did it as libxl parameter disabled by default to avoid possible
problems:
- with save/restore/migration (restoring with ahci a domU that was with
ide instead)
- windows < 8 without pv drivers (a registry key change is needed for
AHCI<->IDE change FWIK to avoid possible blue screen)
- windows XP or older that many not support ahci by default.
Setting AHCI with libxl parameter and default to disabled seems the best
solution.
AHCI increase hvm domUs boot performance. On linux hvm domU I saw up to
only 20% of the previous total boot time, whereas boot time decrease a
lot on W7 domUs for most of boots I have done. Small difference in boot
time compared to ide mode on W8 and newer (probably other xen
improvements or fixes are needed not ahci related)

Signed-off-by: Fabio Fantoni 
---
 docs/man/xl.cfg.pod.5   |  9 +
 tools/libxl/libxl.h | 10 ++
 tools/libxl/libxl_create.c  |  1 +
 tools/libxl/libxl_dm.c  | 10 +-
 tools/libxl/libxl_types.idl |  1 +
 tools/libxl/xl_cmdimpl.c|  1 +
 6 files changed, 31 insertions(+), 1 deletion(-)

diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
index a3e0e2e..7e16123 100644
--- a/docs/man/xl.cfg.pod.5
+++ b/docs/man/xl.cfg.pod.5
@@ -904,6 +904,15 @@ default is B.
 
 =back
 
+=item B
+
+If enabled adds ich9 disk controller in ahci mode and uses it with
+upstream qemu to emulate disks instead of ide. It decrease boot time but
+may be not supported by default in windows xp and older windows.
+The default is disabled (0).
+
+=back
+
 =head3 Paging
 
 The following options control the mechanisms used to virtualise guest
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index 0a7913b..6a3677d 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -596,6 +596,16 @@ typedef struct libxl__ctx libxl_ctx;
 #define LIBXL_HAVE_SPICE_STREAMINGVIDEO 1
 
 /*
+ * LIBXL_HAVE_AHCI
+ *
+ * If defined, then the u.hvm structure will contain a boolean type:
+ * ahci. This value defines if ahci support is present.
+ *
+ * If this is not defined, the ahci support is ignored.
+ */
+#define LIBXL_HAVE_AHCI 1
+
+/*
  * LIBXL_HAVE_DOMAIN_CREATE_RESTORE_PARAMS 1
  *
  * If this is defined, libxl_domain_create_restore()'s API has changed to
diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
index 86384d2..8ca2481 100644
--- a/tools/libxl/libxl_create.c
+++ b/tools/libxl/libxl_create.c
@@ -331,6 +331,7 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,
 libxl_defbool_setdefault(&b_info->u.hvm.nested_hvm, false);
 libxl_defbool_setdefault(&b_info->u.hvm.usb,false);
 libxl_defbool_setdefault(&b_info->u.hvm.xen_platform_pci,   true);
+libxl_defbool_setdefault(&b_info->u.hvm.ahci,   false);
 
 libxl_defbool_setdefault(&b_info->u.hvm.spice.enable, false);
 if (!libxl_defbool_val(b_info->u.hvm.spice.enable) &&
diff --git a/tools/libxl/libxl_dm.c b/tools/libxl/libxl_dm.c
index 33f9ce6..93f191a 100644
--- a/tools/libxl/libxl_dm.c
+++ b/tools/libxl/libxl_dm.c
@@ -818,6 +818,8 @@ static int libxl__build_device_model_args_new(libxl__gc *gc,
 flexarray_append(dm_args, libxl__sprintf(gc, "%"PRId64, ram_size));
 
 if (b_info->type == LIBXL_DOMAIN_TYPE_HVM) {
+if (libxl_defbool_val(b_info->u.hvm.ahci))
+flexarray_append_pair(dm_args, "-device", "ahci,id=ahci0");
 for (i = 0; i < num_disks; i++) {
 int disk, part;
 int dev_number =
@@ -872,7 +874,13 @@ static int libxl__build_device_model_args_new(libxl__gc 
*gc,
 drive = libxl__sprintf
 (gc, 
"file=%s,if=scsi,bus=0,unit=%d,format=%s,cache=writeback",
  pdev_path, disk, format);
-else if (disk < 4)
+else if (disk < 6 && libxl_defbool_val(b_info->u.hvm.ahci)){
+flexarray_vappend(dm_args, "-drive",
+
GCSPRINTF("file=%s,if=none,id=ahcidisk-%d,format=%s,cache=writeback",
+pdev_path, disk, format), "-device", 
GCSPRINTF("ide-hd,bus=ahci0.%d,unit=0,drive=ahcidisk-%d",
+disk, disk), NULL);
+continue;
+}else if (disk < 4)
 drive = libxl__sprintf
 (gc, 
"file=%s,if=ide,index=%d,media=disk,format=%s,cache=writeback

Re: [Xen-devel] [PATCH v2][RFC] libxl: Add AHCI support for upstream qemu

[Fabio Fantoni]

Il 22/06/2015 12:34, George Dunlap ha scritto:

On Fri, Jun 19, 2015 at 12:18 PM, Fabio Fantoni
 wrote:> Il 11/06/2015 12:28, Fabio Fantoni ha
scritto:

Il 11/06/2015 12:06, Zir Blazer ha scritto:

Since I'm not a developer I may be peeking my nose a bit too far, but
based on what I know, I think that enabling AHCI by default would be a
compatibility suicide. I'm not sure about Linux and Windows Vista/7/8+, but
at least for Windows XP based VMs, it would be a terrible idea.


Also use windows xp without security updates (support ended one year ago)
is a "suicide".

I already did this patch considering windows domU problems (I'm using
mainly them for now), ahci used with option (ahci=0|1) instead replace and
default is disabled.
I tried it with different windows (excluding xp...abandoned)
I also tried with new winpv drivers
(http://www.xenproject.org/developers/teams/windows-pv-drivers.html)

With this patch applied ahci will be not used and will be used only
setting ahci=1, is it a good idea or is there problem also in this case?


I did many other tests in different linux hvm domUs (fedora and ubuntu) and
windows (7, 8.1, 10) without found problems.
Is this patch acceptable for xen 4.6?

Well maybe I missed something, but:

1. The most recent version of this patch (v2) has RFC in the title;
this is a specific request *not* to apply this patch.

2. The most recent version of this patch has the following in the
changelog: "NOTES: This patch is a only a fast draft for testing."
That also sounds like you're asking people not to apply the patch.

3. After reading the changelog, many people were still unclear what
the purpose of the patch is.  You answered their questions by e-mail,
but that information needs to be in the changelog.

So you need to resend the patch 1) with RFC removed from the title;
and 2) with a proper changelog that doesn't say "just a draft", but
that 3) explains what the purpose of the change for people reading
through the revision history.

  -George


Thanks for reply, I did RFC initially more than 1 month ago when I was 
less certainty requiring comment and experts review.

Is RFC=request for comments or I'm wrong?
Now after many tests on many systems and keep using it for weeks without 
problem seems ok.

I'll post v3 including missing doc, libxl.h entry and improving changelog.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2][RFC] libxl: Add AHCI support for upstream qemu

[Fabio Fantoni]

Il 11/06/2015 12:28, Fabio Fantoni ha scritto:

Il 11/06/2015 12:06, Zir Blazer ha scritto:
Since I'm not a developer I may be peeking my nose a bit too far, but 
based on what I know, I think that enabling AHCI by default would be 
a compatibility suicide. I'm not sure about Linux and Windows 
Vista/7/8+, but at least for Windows XP based VMs, it would be a 
terrible idea.


Also use windows xp without security updates (support ended one year 
ago) is a "suicide".


I already did this patch considering windows domU problems (I'm using 
mainly them for now), ahci used with option (ahci=0|1) instead replace 
and default is disabled.

I tried it with different windows (excluding xp...abandoned)
I also tried with new winpv drivers 
(http://www.xenproject.org/developers/teams/windows-pv-drivers.html)


With this patch applied ahci will be not used and will be used only 
setting ahci=1, is it a good idea or is there problem also in this case?




I did many other tests in different linux hvm domUs (fedora and ubuntu) 
and windows (7, 8.1, 10) without found problems.

Is this patch acceptable for xen 4.6?



Thanks for any reply and sorry for my bad english.




Back during WXP years, the vanilla install ISO didn't had any AHCI 
Drivers integrated at all. If you set the SATA Controller to AHCI, 
the WXP installation wouldn't detect any Hard Disk at all. If you 
really wanted to use AHCI during its early days, you would need a 
Floppy Disk with the AHCI Drivers of your SATA Controller 
(Chipset/South Bridge or third party controller), that you could use 
by pressing the F6 key right after booting the install CD:

http://www.pc-tips-and-tricks.com/images/Install-windows-xp-02.jpg

If you didn't had a Floppy Disk Driver or were lazy, the typical 
choice was to set the SATA Controller to IDE and install WXP, 
skipping anything AHCI related.


At some point an application called nLite appeared, which allowed you 
to make your custom Windows XP ISO. You could easily modify a lot of 
default options, integrate hotfixes, and even Drivers, and after you 
were happy with your changes, you could then make your custom ISO and 
burn it to a CD (Or from a USB Flash Drive with other tools to make 
it booteable). So, with nLite, it was rather easy to integrated the 
AHCI Drivers of your SATA Controller to the install ISO. This way, if 
you were doing a fresh install, reinstall or repair, there was no 
excuse to not use AHCI.
However, since for VM usage AHCI was not available at all until QEMU 
implemented it recently, by default you will be always using IDE, so 
there was no actual need to make a customized WXP install ISO with 
AHCI Drivers that you were never going to use. Besides, Windows XP 
doesn't even install nor have ready any AHCI Drivers if you're in IDE 
mode, so if you're going to switch to use AHCI by default, every 
existing WXP VM would greet you with a BSOD after upgrading Xen.



So, with this proposal you would have two issues: First, doing a new 
install of Windows XP on a VM , since by default it shouldn't detect 
any HD at all in AHCI. Second, that existing WXP VMs will BSOD on 
boot. For as long as you can switch to IDE mode everything should 
still be working as always, but it would mean that at the very least 
you would have to modify the DomU config file to replace default AHCI 
with old IDE.


Regarding using Windows XP with AHCI:
First, as stated before, if you want to install it on a new VM, you 
need the AHCI Drivers. Since it seems that at some point support for 
Floppy Disk got removed from Xen (I tried recently to use a Floppy 
Disk image with fda= as stated somewhere on the wiki, but it did 
nothing. Nor I found any recent documentation claiming that you can 
mount a Floppy Disk image at all), the only choice is to make a 
custom ISO with nLite with the AHCI Drivers integrated in it. Since 
the emulated Southbridge is an Intel ICH9, I believe that the Drivers 
that should be needed are these:
http://www.win-raid.com/t22f23-Guide-Integration-of-Intels-AHCI-RAID-drivers-into-a-Windows-XP-W-k-W-k-CD.html 


Check for the a) and c) options for WXP/2003 in 32 and 64 Bits variants.

Also, it is important to note that the plain ICH9 Southbridge 
according to Intel DOES NOT OFFICIALLY HAVE AHCI SUPPORT, ICH9R does.

http://en.wikipedia.org/wiki/I/O_Controller_Hub#ICH9

The previous Drivers are claimed to have been modified to force 
support for ICH9 since AHCI is there, just that the original Drivers 
doesn't want to use it. Typical artificial market segmentation 
strategies. Still, if QEMU is emulating the plain ICH9 and you can't 
get Windows to see drives in AHCI mode after integrating the AHCI 
Drivers, that may be why.



Second, it is possible to switch from IDE to AHCI on WXP without 
reinstalling, but doing it is rather hacky and instructions are very 
Chipset and vendor dependant, which may (Or not) include editing

Re: [Xen-devel] [PATCH v2 00/17] x86/hvm: I/O emulation cleanup and fix

[Fabio Fantoni]

Il 11/06/2015 17:42, Paul Durrant ha scritto:

This patch series re-works much of the code involved in emulation of port
and memory mapped I/O for HVM guests.

The code has become very convoluted and, at least by inspection, certain
emulations will apparently malfunction.

The series is broken down into 17 patches (which are also available in
my xenbits repo: http://xenbits.xen.org/gitweb/?p=people/pauldu/xen.git
on the emulation22 branch) as follows:

0001-x86-hvm-simplify-hvmemul_do_io.patch
0002-x86-hvm-re-name-struct-hvm_mmio_handler-to-hvm_mmio_.patch
0003-x86-hvm-unify-internal-portio-and-mmio-intercepts.patch
0004-x86-hvm-unify-dpci-portio-intercept-with-standard-po.patch
0005-x86-hvm-unify-stdvga-mmio-intercept-with-standard-mm.patch
0006-x86-hvm-revert-82ed8716b-fix-direct-PCI-port-I-O-emu.patch
0007-x86-hvm-only-call-hvm_io_assist-from-hvm_wait_for_io.patch
0008-x86-hvm-split-I-O-completion-handling-from-state-mod.patch
0009-x86-hvm-remove-hvm_io_pending-check-in-hvmemul_do_io.patch
0010-x86-hvm-remove-HVMIO_dispatched-I-O-state.patch
0011-x86-hvm-remove-hvm_io_state-enumeration.patch
0012-x86-hvm-use-ioreq_t-to-track-in-flight-state.patch
0013-x86-hvm-only-acquire-RAM-pages-for-emulation-when-we.patch
0014-x86-hvm-remove-extraneous-parameter-from-hvmtrace_io.patch
0015-x86-hvm-make-sure-translated-MMIO-reads-or-writes-fa.patch
0016-x86-hvm-remove-multiple-open-coded-chunking-loops.patch
0017-x86-hvm-track-large-memory-mapped-accesses-by-buffer.patch

v2:
  - Removed bogus assertion from patch 16
  - Re-worked patch 17 after basic testing of back-port onto XenServer


Thanks, I confirm that this new version solves the critical regression 
that causes dom0 insta-reboot.


I tested them on windows and linux hvm domUs, I had strange results...
On windows 7 64 bit sp1 domU (with new winpv drivers) seems there are a 
small performance regression (or probably "only" latency increased) but 
I not found warning/error or something userful in logs, I'm not sure is 
real regression of your patches, if needed I'll retry without.
On linux domU (fedora 21) qxl that had sse2 instructions problem still 
not working, same of latest 2 years not always qemu crash with a gdb 
showing sse2 instruction problem but xorg crash with EQ overflowing 
and/or domU remain with 100% cpu used by xorg qemu process at 100% cpu.
I posted the backtrace similar time ago and a qemu/spice developer told 
that was a conseguence of other problem if I remember good.
How can I be sure that sse2 or similar istructions (with videoram) are 
now working correctly?


If you need more informations/tests tell me and I'll post them.

Thanks for any reply and sorry for my bad english.



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2][RFC] libxl: Add AHCI support for upstream qemu

[Fabio Fantoni]

einstalling-windows-xp.php

But should these not work, chances are that you will need to add a rescue ISO 
to your DomU config file to repair the broken install, since I'm not 100% sure 
if you can even get it to boot in Safe Mode if it BSODs at boot.


I think that's all what I could figure out. If you're using the GPLPV Drivers I 
don't know if there are any changes regarding running in AHCI mode.





Date: Mon, 8 Jun 2015 16:17:39 +0200
From: fabio.fant...@m2r.biz
To: stefano.stabell...@eu.citrix.com
CC: xen-de...@lists.xensource.com; wei.l...@citrix.com; 
ian.campb...@citrix.com; ian.jack...@eu.citrix.com; paul.durr...@citrix.com; 
anthony.per...@citrix.com
Subject: Re: [Xen-devel] [PATCH v2][RFC] libxl: Add AHCI support for upstream 
qemu

Il 08/06/2015 16:00, Stefano Stabellini ha scritto:

On Tue, 19 May 2015, Fabio Fantoni wrote:

Il 19/05/2015 12:40, Wei Liu ha scritto:

On Mon, May 18, 2015 at 07:22:01PM +0200, Fabio Fantoni wrote:

Il 18/05/2015 17:53, Wei Liu ha scritto:

On Thu, May 14, 2015 at 01:11:13PM +0200, Fabio Fantoni wrote:

Usage:
ahci=0|1 (default=0)

If enabled adds ich9 disk controller in ahci mode and uses it with
upstream qemu to emulate disks instead of ide.

Is ICH9 available in our default setup?

Why do we not always enable AHCI?

ahci seems require ich9 controller (default in q35 but not in older used
by
xen), I think that change it ide->ahci automatically for all cases may
causes problems, probably in save/restore and more probably in windows <=7
without pv where change ide<->ahci require a registry change for not have
blue screen.


I guess we now have a QEMU with Q35. But migrating from previous
previous versions will indeed need to be taken care of.

This patch don't add q35 cipset support but only ich9 ahci disk controller in
older and only cipset supported in xen.

This is pretty nice actually, thanks for the patch!



Time ago I tried to add q35 support in xen but I had some problems I was
unable to solves and nobody helped me.
Q35 have newer ich9 ahci disk controller as default, on older cipset must be
added (done in this patch).
qemu, kvm ovmf already support q35, xen need changes at least in hvmloader
(that I'm probably unable to do) and libxl (that I started time ago without
good result), Add ahci support and switch all qemu parameters in libxl to new
-device is a good start also for future q35 support (are both part needed
based on my old q35 tests on xen)

Yeah, going q35 opens a whole new can of worms. I think that adding the
ich9 chipset to the existing machine is a good compromise.



It doesn't support cdroms which still use ide (cdroms will use
"-device
ide-cd" as new qemu parameters)
Ahci requires new qemu parameters but for now other emulated disks
cases
remains with old ones because automatic bus selection seems bugged in
qemu using new parameters. (I'll retry)


Buggy as in? Have you reported to QEMU upstream?

Already reported long time ago in xen-devel and qemu-devel, the only reply
from qemu-devel was to use fixed bus and is what I did in v2 of this
patch.
Can someone tell me if can be a problem fixed bus?


I don't have enough knowledge on this. Maybe Anthony and Stefano have
more insight.

Let me get this straight: cdrom still works but goes via ide. In
addition AHCI disks can be attached. Seems good to me.

Have you tested Windows?

If this works with Windows too, I would expose an ahci option
(default=off) in the config file. If ahci=on, then all disks could be
exposed via ahci. Does this seem sensible? In a couple of releases, if
it works well, we could make ahci=on the default.

I tested both linux and windows, mainly windows domUs (7 and 8.1).
Windows < 8 FWIK needs a registry key change for switch ide<->ahci
without bluescreen (this is the reason because I did ahci=0|1 instead
change it by default) but trying with windows 7 with pv is possible
change without bluescreen, probably the problem is only without pv but
ahci option default disabled resolve it (for the users don't set ahci=0
on xen upgrade process).
Based on what I know and all tests I did this patch seems the best way
for add ahci support but I think that a experts (like you) review is better.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel





___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 00/17] x86/hvm: I/O emulation cleanup and fix

[Fabio Fantoni]

Il 10/06/2015 11:13, Fabio Fantoni ha scritto:

Il 09/06/2015 17:21, Paul Durrant ha scritto:

-Original Message-
From: Fabio Fantoni [mailto:fabio.fant...@m2r.biz]
Sent: 09 June 2015 15:44
To: Paul Durrant; xen-de...@lists.xenproject.org
Subject: Re: [Xen-devel] [PATCH 00/17] x86/hvm: I/O emulation 
cleanup and

fix

Il 08/06/2015 16:33, Paul Durrant ha scritto:
This patch series re-works much of the code involved in emulation 
of port

and memory mapped I/O for HVM guests.

The code has become very convoluted and, at least by inspection, 
certain

emulations will apparently malfunction.

The series is broken down into 17 patches (which are also available in
my xenbits repo:

http://xenbits.xen.org/gitweb/?p=people/pauldu/xen.git

on the emulation18 branch) as follows:

Big thanks for your work.
I tested them taking the patches from emulation18 branch.
I tried xl create of a window 7 domU but during xl create dom0
istant-reboot and there isn't anything about the problem in the logs
(kern.log, syslog, qemu log, xl log ecc...)
I tried also with linux (hvm) domU and with stdvga instead qxl I had
setted but same result.

For debug the problem I must enable all xen debug in grub entry and
redirect output to serial, similar to debug of dom0 boot problem or I
must do different thing?

Having serial is very useful in these cases. I've been debugging with 
a 32-bit Win 7 VM so it would be useful if you could bisect bit. 
There are natural boundaries in the series you could try first:


Apply everything up to and including 'x86/hvm: unify stdvga mmio 
intercept with standard mmio intercept'
Then try everything up to and including ' x86/hvm: remove extraneous 
parameter from hvmtrace_io_assist()'

Then, if you get that far, try the rest one at a time.

   Paul


Found the patch that cause dom0 insta-reboot:
 x86/hvm: remove multiple open coded 'chunking' loops in 
hvmemul_read/write()


In attachments also xl -vvv create output until the crash if can be 
useful, I not found useful information in logs after reboot :(


If you need more informations/tests tell me and I'll post them.



Thanks for reply.
I tried to boot with full debug enabled and with serial on lan but 
server insta-reboot without show error or useful output on redirected 
output or monitor.
I used same parameters of other debug I did long time ago if I 
remember good, here the grub2 entry:
menuentry 'Wheezy con Linux 3.16.0-0.bpo.4-amd64 e XEN - RAID - Debug 
su Seriale' --class debian --class gnu-linux --class gnu --class os {

set root='(RAID-ROOT)'
echo'Caricamento Hypervisor Xen...'
multiboot/boot/xen.gz placeholder dom0_mem=2G,max:3G 
swiotlb=65762 loglvl=all guest_loglvl=all sync_console 
console_to_ring com2=19200,8n1 console=com2

echo'Caricamento Linux 3.16.0-0.bpo.4-amd64...'
linux/boot/vmlinuz-3.16.0-0.bpo.4-amd64 placeholder 
root=/dev/mapper/RAID-ROOT ro console=hvc0 earlyprintk=xen nomodeset

echo'Caricamento ramdisk iniziale...'
initrd/boot/initrd.img-3.16.0-0.bpo.4-amd64
}

Here the entry without debug working:


menuentry 'RAID - Debian 7.8 (wheezy) con Linux 3.16.0-0.bpo.4-amd64 
e XEN 4.6-unstable' --class debian --class gnu-linux --class gnu 
--class os --class xen {

set fallback="1"
set root='(RAID-ROOT)'
echo'Caricamento Hypervisor Xen 4.6-unstable...'
multiboot/boot/xen.gz placeholder dom0_mem=2G,max:3G
echo'Caricamento Linux 3.16.0-0.bpo.4-amd64 ...'
module/boot/vmlinuz-3.16.0-0.bpo.4-amd64 placeholder 
root=/dev/mapper/RAID-ROOT ro swiotlb=65762 quiet

echo'Caricamento ramdisk iniziale...'
module/boot/initrd.img-3.16.0-0.bpo.4-amd64
}

I did something wrong?

Now I'll try to bisect the patch serie following your advice.




Thanks for any reply and sorry for my bad english.


x86/hvm: simplify hvmemul_do_io()
x86/hvm: re-name struct hvm_mmio_handler to hvm_mmio_ops
x86/hvm: unify internal portio and mmio intercepts
x86/hvm: unify dpci portio intercept with standard portio intercept
x86/hvm: unify stdvga mmio intercept with standard mmio intercept
x86/hvm: revert 82ed8716b "fix direct PCI port I/O emulation retry...
x86/hvm: only call hvm_io_assist() from hvm_wait_for_io()
x86/hvm: split I/O completion handling from state model
x86/hvm: remove hvm_io_pending() check in hvmemul_do_io()
x86/hvm: remove HVMIO_dispatched I/O state
x86/hvm: remove hvm_io_state enumeration
x86/hvm: use ioreq_t to track in-flight state
x86/hvm: only acquire RAM pages for emulation when we need to
x86/hvm: remove extraneous parameter from hvmtrace_io_assist()
x86/hvm: make sure translated MMIO reads or writes fall within a page
x86/hvm: remove multiple open coded 'chunking' loops
x86/hvm: track large memory mapped accesses by linear address


_

Re: [Xen-devel] [PATCH 00/17] x86/hvm: I/O emulation cleanup and fix

[Fabio Fantoni]

Il 09/06/2015 17:21, Paul Durrant ha scritto:

-Original Message-
From: Fabio Fantoni [mailto:fabio.fant...@m2r.biz]
Sent: 09 June 2015 15:44
To: Paul Durrant; xen-de...@lists.xenproject.org
Subject: Re: [Xen-devel] [PATCH 00/17] x86/hvm: I/O emulation cleanup and
fix

Il 08/06/2015 16:33, Paul Durrant ha scritto:

This patch series re-works much of the code involved in emulation of port
and memory mapped I/O for HVM guests.

The code has become very convoluted and, at least by inspection, certain
emulations will apparently malfunction.

The series is broken down into 17 patches (which are also available in
my xenbits repo:

http://xenbits.xen.org/gitweb/?p=people/pauldu/xen.git

on the emulation18 branch) as follows:

Big thanks for your work.
I tested them taking the patches from emulation18 branch.
I tried xl create of a window 7 domU but during xl create dom0
istant-reboot and there isn't anything about the problem in the logs
(kern.log, syslog, qemu log, xl log ecc...)
I tried also with linux (hvm) domU and with stdvga instead qxl I had
setted but same result.

For debug the problem I must enable all xen debug in grub entry and
redirect output to serial, similar to debug of dom0 boot problem or I
must do different thing?


Having serial is very useful in these cases. I've been debugging with a 32-bit 
Win 7 VM so it would be useful if you could bisect bit. There are natural 
boundaries in the series you could try first:

Apply everything up to and including 'x86/hvm: unify stdvga mmio intercept with 
standard mmio intercept'
Then try everything up to and including ' x86/hvm: remove extraneous parameter 
from hvmtrace_io_assist()'
Then, if you get that far, try the rest one at a time.

   Paul


Thanks for reply.
I tried to boot with full debug enabled and with serial on lan but 
server insta-reboot without show error or useful output on redirected 
output or monitor.
I used same parameters of other debug I did long time ago if I remember 
good, here the grub2 entry:
menuentry 'Wheezy con Linux 3.16.0-0.bpo.4-amd64 e XEN - RAID - Debug 
su Seriale' --class debian --class gnu-linux --class gnu --class os {

set root='(RAID-ROOT)'
echo'Caricamento Hypervisor Xen...'
multiboot/boot/xen.gz placeholder dom0_mem=2G,max:3G 
swiotlb=65762 loglvl=all guest_loglvl=all sync_console console_to_ring 
com2=19200,8n1 console=com2

echo'Caricamento Linux 3.16.0-0.bpo.4-amd64...'
linux/boot/vmlinuz-3.16.0-0.bpo.4-amd64 placeholder 
root=/dev/mapper/RAID-ROOT ro console=hvc0 earlyprintk=xen nomodeset

echo'Caricamento ramdisk iniziale...'
initrd/boot/initrd.img-3.16.0-0.bpo.4-amd64
}

Here the entry without debug working:


menuentry 'RAID - Debian 7.8 (wheezy) con Linux 3.16.0-0.bpo.4-amd64 e 
XEN 4.6-unstable' --class debian --class gnu-linux --class gnu --class 
os --class xen {

set fallback="1"
set root='(RAID-ROOT)'
echo'Caricamento Hypervisor Xen 4.6-unstable...'
multiboot/boot/xen.gz placeholder dom0_mem=2G,max:3G
echo'Caricamento Linux 3.16.0-0.bpo.4-amd64 ...'
module/boot/vmlinuz-3.16.0-0.bpo.4-amd64 placeholder 
root=/dev/mapper/RAID-ROOT ro swiotlb=65762 quiet

echo'Caricamento ramdisk iniziale...'
module/boot/initrd.img-3.16.0-0.bpo.4-amd64
}

I did something wrong?

Now I'll try to bisect the patch serie following your advice.




Thanks for any reply and sorry for my bad english.


x86/hvm: simplify hvmemul_do_io()
x86/hvm: re-name struct hvm_mmio_handler to hvm_mmio_ops
x86/hvm: unify internal portio and mmio intercepts
x86/hvm: unify dpci portio intercept with standard portio intercept
x86/hvm: unify stdvga mmio intercept with standard mmio intercept
x86/hvm: revert 82ed8716b "fix direct PCI port I/O emulation retry...
x86/hvm: only call hvm_io_assist() from hvm_wait_for_io()
x86/hvm: split I/O completion handling from state model
x86/hvm: remove hvm_io_pending() check in hvmemul_do_io()
x86/hvm: remove HVMIO_dispatched I/O state
x86/hvm: remove hvm_io_state enumeration
x86/hvm: use ioreq_t to track in-flight state
x86/hvm: only acquire RAM pages for emulation when we need to
x86/hvm: remove extraneous parameter from hvmtrace_io_assist()
x86/hvm: make sure translated MMIO reads or writes fall within a page
x86/hvm: remove multiple open coded 'chunking' loops
x86/hvm: track large memory mapped accesses by linear address


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 00/17] x86/hvm: I/O emulation cleanup and fix

[Fabio Fantoni]

Il 08/06/2015 16:33, Paul Durrant ha scritto:

This patch series re-works much of the code involved in emulation of port
and memory mapped I/O for HVM guests.

The code has become very convoluted and, at least by inspection, certain
emulations will apparently malfunction.

The series is broken down into 17 patches (which are also available in
my xenbits repo: http://xenbits.xen.org/gitweb/?p=people/pauldu/xen.git
on the emulation18 branch) as follows:


Big thanks for your work.
I tested them taking the patches from emulation18 branch.
I tried xl create of a window 7 domU but during xl create dom0 
istant-reboot and there isn't anything about the problem in the logs 
(kern.log, syslog, qemu log, xl log ecc...)
I tried also with linux (hvm) domU and with stdvga instead qxl I had 
setted but same result.


For debug the problem I must enable all xen debug in grub entry and 
redirect output to serial, similar to debug of dom0 boot problem or I 
must do different thing?


Thanks for any reply and sorry for my bad english.



x86/hvm: simplify hvmemul_do_io()
x86/hvm: re-name struct hvm_mmio_handler to hvm_mmio_ops
x86/hvm: unify internal portio and mmio intercepts
x86/hvm: unify dpci portio intercept with standard portio intercept
x86/hvm: unify stdvga mmio intercept with standard mmio intercept
x86/hvm: revert 82ed8716b "fix direct PCI port I/O emulation retry...
x86/hvm: only call hvm_io_assist() from hvm_wait_for_io()
x86/hvm: split I/O completion handling from state model
x86/hvm: remove hvm_io_pending() check in hvmemul_do_io()
x86/hvm: remove HVMIO_dispatched I/O state
x86/hvm: remove hvm_io_state enumeration
x86/hvm: use ioreq_t to track in-flight state
x86/hvm: only acquire RAM pages for emulation when we need to
x86/hvm: remove extraneous parameter from hvmtrace_io_assist()
x86/hvm: make sure translated MMIO reads or writes fall within a page
x86/hvm: remove multiple open coded 'chunking' loops
x86/hvm: track large memory mapped accesses by linear address


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] xen-unstable build fails with XEN_DUMP_DIR undeclader in xl_cmdimpl.c

[Fabio Fantoni]

Il 09/06/2015 12:54, Fabio Fantoni ha scritto:

Il 09/06/2015 09:02, Olaf Hering ha scritto:

On Mon, Jun 08, Fabio Fantoni wrote:


I saw that config/Paths.mk contains:
XEN_DUMP_DIR := /var/lib/xen/dump

But build fails with:

xl_cmdimpl.c: In function âhandle_domain_deathâ:
xl_cmdimpl.c:2330:33: error: âXEN_DUMP_DIRâ undeclared (first use 
in this

function)
xl_cmdimpl.c:2330:33: note: each undeclared identifier is reported 
only

once for each function it appears in
xl_cmdimpl.c:2330:46: error: expected â)â before string constant

With a fast look in code I not found the right cause.

The logic in the Makefile is supposed to regenerate _paths.h if anything
within that file changes. Appearently that logic does not work properly.

As suggested a 'make clean' or 'rm tools/*/_*.h' should work around such
bug.

Olaf


'make clean' or 'rm tools/*/_*.h' as workaround didn't works, problem 
still happen.


I already use git reset --hard and git git clean -f -d -x before any 
test that clean all except other git repos (seabios, ovmf and qemu).


I missed to tell I use also a change I did for workaround in Config.mk:
PYTHON_PREFIX_ARG ?= --prefix="$(prefix)" to PYTHON_PREFIX_ARG ?=

Probably it in combination with one or more ./configure options that 
have a regression.


I'm testing with this: 
https://github.com/Fantu/Xen/commits/rebase/m2r-staging and was 
working until https://github.com/Fantu/Xen/commits/rebase/m2r-next
Is simply a rebase and only the first patch had a conflict for this 
commit: 7f5f57b466b026d19cacfe7fc5771895d05a053b


Problem found, I not saw other Config.mk changes during conflict on 
rebase, I'm stupid, sorry.




The regression should be between upstream commit 
f48218fd2d35e274ef58caee889aecd6610c8cb6 and 
ecdae1cfaa7f6123decaa1b9d7205c3ff726b941



Without problem tools/libxl/_paths.h is correctly generated:

#define sbindir "/usr/sbin"
#define bindir "/usr/bin"
#define LIBEXEC "/usr/lib/xen"
#define LIBEXEC_BIN "/usr/lib/xen/bin"
#define libdir "/usr/lib"
#define SHAREDIR "/usr/share"
#define XENFIRMWAREDIR "/usr/lib/xen/boot"
#define XEN_CONFIG_DIR "/etc/xen"
#define XEN_SCRIPT_DIR "/etc/xen/scripts"
#define XEN_LOCK_DIR "/var/lock"
#define XEN_RUN_DIR "/var/run/xen"
#define XEN_PAGING_DIR "/var/lib/xen/xenpaging"
#define XEN_DUMP_DIR "/var/lib/xen/dump"


With the problem instead:

#define SBINDIR ""
#define BINDIR ""
#define LIBEXEC "/usr/lib/xen"
#define LIBEXEC_BIN "/usr/lib/xen/bin"
#define LIBDIR ""
#define SHAREDIR "/usr/share"
#define XENFIRMWAREDIR "/usr/lib/xen/boot"
#define XEN_CONFIG_DIR "/etc/xen"
#define XEN_SCRIPT_DIR "/etc/xen/scripts"
#define XEN_LOCK_DIR "/var/lock"
#define XEN_RUN_DIR "/var/run/xen"
#define XEN_PAGING_DIR "/var/lib/xen/xenpaging"


I'm still trying various combinations but I have a problem to found 
the exactly parameters that cause the problem, some restult are 
strange...I'm going crazy :(


During testing I had also this:

ovmf.c: In function âovmf_loadâ:
ovmf.c:100:21: error: âovmfâ undeclared (first use in this function)
ovmf.c:100:21: note: each undeclared identifier is reported only once 
for each function it appears in

ovmf.c: At top level:
ovmf.c:154:14: error: âovmfâ undeclared here (not in a function)
Probably because I tried before with build ovmf and after with 
prebuild from system that cause an inexpected case


This and probably another (that I not remember exactly) are the only 
upstream minor problems caused by rare unexpected case.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] xen-unstable build fails with XEN_DUMP_DIR undeclader in xl_cmdimpl.c

[Fabio Fantoni]

Il 09/06/2015 09:02, Olaf Hering ha scritto:

On Mon, Jun 08, Fabio Fantoni wrote:


I saw that config/Paths.mk contains:
XEN_DUMP_DIR := /var/lib/xen/dump

But build fails with:

xl_cmdimpl.c: In function âhandle_domain_deathâ:
xl_cmdimpl.c:2330:33: error: âXEN_DUMP_DIRâ undeclared (first use in this
function)
xl_cmdimpl.c:2330:33: note: each undeclared identifier is reported only
once for each function it appears in
xl_cmdimpl.c:2330:46: error: expected â)â before string constant

With a fast look in code I not found the right cause.

The logic in the Makefile is supposed to regenerate _paths.h if anything
within that file changes. Appearently that logic does not work properly.

As suggested a 'make clean' or 'rm tools/*/_*.h' should work around such
bug.

Olaf


'make clean' or 'rm tools/*/_*.h' as workaround didn't works, problem 
still happen.


I already use git reset --hard and git git clean -f -d -x before any 
test that clean all except other git repos (seabios, ovmf and qemu).


I missed to tell I use also a change I did for workaround in Config.mk:
PYTHON_PREFIX_ARG ?= --prefix="$(prefix)" to PYTHON_PREFIX_ARG ?=

Probably it in combination with one or more ./configure options that 
have a regression.


I'm testing with this: 
https://github.com/Fantu/Xen/commits/rebase/m2r-staging and was working 
until https://github.com/Fantu/Xen/commits/rebase/m2r-next
Is simply a rebase and only the first patch had a conflict for this 
commit: 7f5f57b466b026d19cacfe7fc5771895d05a053b


The regression should be between upstream commit 
f48218fd2d35e274ef58caee889aecd6610c8cb6 and 
ecdae1cfaa7f6123decaa1b9d7205c3ff726b941



Without problem tools/libxl/_paths.h is correctly generated:

#define sbindir "/usr/sbin"
#define bindir "/usr/bin"
#define LIBEXEC "/usr/lib/xen"
#define LIBEXEC_BIN "/usr/lib/xen/bin"
#define libdir "/usr/lib"
#define SHAREDIR "/usr/share"
#define XENFIRMWAREDIR "/usr/lib/xen/boot"
#define XEN_CONFIG_DIR "/etc/xen"
#define XEN_SCRIPT_DIR "/etc/xen/scripts"
#define XEN_LOCK_DIR "/var/lock"
#define XEN_RUN_DIR "/var/run/xen"
#define XEN_PAGING_DIR "/var/lib/xen/xenpaging"
#define XEN_DUMP_DIR "/var/lib/xen/dump"


With the problem instead:

#define SBINDIR ""
#define BINDIR ""
#define LIBEXEC "/usr/lib/xen"
#define LIBEXEC_BIN "/usr/lib/xen/bin"
#define LIBDIR ""
#define SHAREDIR "/usr/share"
#define XENFIRMWAREDIR "/usr/lib/xen/boot"
#define XEN_CONFIG_DIR "/etc/xen"
#define XEN_SCRIPT_DIR "/etc/xen/scripts"
#define XEN_LOCK_DIR "/var/lock"
#define XEN_RUN_DIR "/var/run/xen"
#define XEN_PAGING_DIR "/var/lib/xen/xenpaging"


I'm still trying various combinations but I have a problem to found the 
exactly parameters that cause the problem, some restult are 
strange...I'm going crazy :(


During testing I had also this:

ovmf.c: In function âovmf_loadâ:
ovmf.c:100:21: error: âovmfâ undeclared (first use in this function)
ovmf.c:100:21: note: each undeclared identifier is reported only once 
for each function it appears in

ovmf.c: At top level:
ovmf.c:154:14: error: âovmfâ undeclared here (not in a function)
Probably because I tried before with build ovmf and after with prebuild 
from system that cause an inexpected case.





___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] xen-unstable build fails with XEN_DUMP_DIR undeclader in xl_cmdimpl.c

[Fabio Fantoni]

Il 08/06/2015 15:56, Wei Liu ha scritto:

On Mon, Jun 08, 2015 at 03:49:24PM +0200, Fabio Fantoni wrote:

Il 08/06/2015 15:30, Wei Liu ha scritto:

On Mon, Jun 08, 2015 at 02:36:09PM +0200, Fabio Fantoni wrote:

Il 08/06/2015 13:28, Wei Liu ha scritto:

On Mon, Jun 08, 2015 at 01:25:05PM +0200, Fabio Fantoni wrote:

I saw that config/Paths.mk contains:
XEN_DUMP_DIR := /var/lib/xen/dump

But build fails with:

xl_cmdimpl.c: In function âhandle_domain_deathâ:
xl_cmdimpl.c:2330:33: error: âXEN_DUMP_DIRâ undeclared (first use in this
function)
xl_cmdimpl.c:2330:33: note: each undeclared identifier is reported only
once for each function it appears in
xl_cmdimpl.c:2330:46: error: expected â)â before string constant

With a fast look in code I not found the right cause.


Thanks for any reply and sorry for my bad english.

Our testing system doesn't complain about this.

Maybe you have some stale files lying around? Try make distclean then
make again?

Wei.

Thanks for your reply.

I add some my patches not related:
https://github.com/Fantu/Xen/tree/rebase/m2r-staging all already tested for
at least some days.

I tried another clean build without my patches (git reset --hard ... git
clean -f -d -x) but still fails.

I did this operations:
./configure --prefix=/usr --disable-blktap1 --disable-qemu-traditional
--disable-rombios --with-system-seabios=/usr/share/seabios/bios-256k.bin
--with-extra-qemuu-configure-args="--enable-spice --enable-usb-redir"
--disable-blktap2 --enable-ovmf --with-system-ovmf=/usr/share/ovmf/OVMF.fd
make debball

Have you checked if libxl/_paths.h is generated?

I don't think I spot anything suspicious in commit log and your build
rune.

Wei.

I checked tools/libxl/_paths.h, generated with many lines but XEN_DUMP_DIR
is missed.

#define SBINDIR ""
#define BINDIR ""
#define LIBEXEC "/usr/lib/xen"
#define LIBEXEC_BIN "/usr/lib/xen/bin"
#define LIBDIR ""
#define SHAREDIR "/usr/share"
#define XENFIRMWAREDIR "/usr/lib/xen/boot"
#define XEN_CONFIG_DIR "/etc/xen"
#define XEN_SCRIPT_DIR "/etc/xen/scripts"
#define XEN_LOCK_DIR "/var/lock"
#define XEN_RUN_DIR "/var/run/xen"
#define XEN_PAGING_DIR "/var/lib/xen/xenpaging"

config/Paths.mk instead contain it correctly.
I checked the commit 
http://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=bf36d0792032a0245e4a17a64c8b9eb9e056f282
but not add something about libxl/_paths.h and in it there aren't all path
present in config.
I suppose that something is missed or there inexpected case that cause
partial libxl/_paths.h generation (seems more probable the first case).

I just tried and it worked. This is the rune I used (run in top level
directory):

./configure --prefix=/usr --disable-blktap1 --disable-qemu-traditional 
--disable-rombios --disable-blktpa2 --enable-ovmf

I don't have customised built seabios and ovmf so I omitted those.

The commit I use is ecdae1cfaa7f6123decaa1b9d7205c3ff726b941.

Could you verify you see the same error with the exact rune and commit I
used?

Wei.


I tried changing the configure options (the your above with blktap2 type 
fix) works.

I'll try to found what exactly cause the inexpected case.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2][RFC] libxl: Add AHCI support for upstream qemu

[Fabio Fantoni]

Il 08/06/2015 16:00, Stefano Stabellini ha scritto:

On Tue, 19 May 2015, Fabio Fantoni wrote:

Il 19/05/2015 12:40, Wei Liu ha scritto:

On Mon, May 18, 2015 at 07:22:01PM +0200, Fabio Fantoni wrote:

Il 18/05/2015 17:53, Wei Liu ha scritto:

On Thu, May 14, 2015 at 01:11:13PM +0200, Fabio Fantoni wrote:

Usage:
ahci=0|1 (default=0)

If enabled adds ich9 disk controller in ahci mode and uses it with
upstream qemu to emulate disks instead of ide.

Is ICH9 available in our default setup?

Why do we not always enable AHCI?

ahci seems require ich9 controller (default in q35 but not in older used
by
xen), I think that change it ide->ahci automatically for all cases may
causes problems, probably in save/restore and more probably in windows <=7
without pv where change ide<->ahci require a registry change for not have
blue screen.


I guess we now have a QEMU with Q35. But migrating from previous
previous versions will indeed need to be taken care of.

This patch don't add q35 cipset support but only ich9 ahci disk controller in
older and only cipset supported in xen.

This is pretty nice actually, thanks for the patch!



Time ago I tried to add q35 support in xen but I had some problems I was
unable to solves and nobody helped me.
Q35 have newer ich9 ahci disk controller as default, on older cipset must be
added (done in this patch).
qemu, kvm ovmf already support q35, xen need changes at least in hvmloader
(that I'm probably unable to do) and libxl (that I started time ago without
good result), Add ahci support and switch all qemu parameters in libxl to new
-device is a good start also for future q35 support (are both part needed
based on my old q35 tests on xen)

Yeah, going q35 opens a whole new can of worms. I think that adding the
ich9 chipset to the existing machine is a good compromise.



It doesn't support cdroms which still use ide (cdroms will use
"-device
ide-cd" as new qemu parameters)
Ahci requires new qemu parameters but for now other emulated disks
cases
remains with old ones because automatic bus selection seems bugged in
qemu using new parameters. (I'll retry)


Buggy as in? Have you reported to QEMU upstream?

Already reported long time ago in xen-devel and qemu-devel, the only reply
from qemu-devel was to use fixed bus and is what I did in v2 of this
patch.
Can someone tell me if can be a problem fixed bus?


I don't have enough knowledge on this. Maybe Anthony and Stefano have
more insight.

Let me get this straight: cdrom still works but goes via ide. In
addition AHCI disks can be attached. Seems good to me.

Have you tested Windows?

If this works with Windows too, I would expose an ahci option
(default=off) in the config file. If ahci=on, then all disks could be
exposed via ahci. Does this seem sensible? In a couple of releases, if
it works well, we could make ahci=on the default.


I tested both linux and windows, mainly windows domUs (7 and 8.1).
Windows < 8 FWIK needs a registry key change for switch ide<->ahci 
without bluescreen (this is the reason because I did ahci=0|1 instead 
change it by default) but trying with windows 7 with pv is possible 
change without bluescreen, probably the problem is only without pv but 
ahci option default disabled resolve it (for the users don't set ahci=0 
on xen upgrade process).
Based on what I know and all tests I did this patch seems the best way 
for add ahci support but I think that a experts (like you) review is better.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] xen-unstable build fails with XEN_DUMP_DIR undeclader in xl_cmdimpl.c

[Fabio Fantoni]

Il 08/06/2015 15:30, Wei Liu ha scritto:

On Mon, Jun 08, 2015 at 02:36:09PM +0200, Fabio Fantoni wrote:

Il 08/06/2015 13:28, Wei Liu ha scritto:

On Mon, Jun 08, 2015 at 01:25:05PM +0200, Fabio Fantoni wrote:

I saw that config/Paths.mk contains:
XEN_DUMP_DIR := /var/lib/xen/dump

But build fails with:

xl_cmdimpl.c: In function âhandle_domain_deathâ:
xl_cmdimpl.c:2330:33: error: âXEN_DUMP_DIRâ undeclared (first use in this
function)
xl_cmdimpl.c:2330:33: note: each undeclared identifier is reported only
once for each function it appears in
xl_cmdimpl.c:2330:46: error: expected â)â before string constant

With a fast look in code I not found the right cause.


Thanks for any reply and sorry for my bad english.

Our testing system doesn't complain about this.

Maybe you have some stale files lying around? Try make distclean then
make again?

Wei.

Thanks for your reply.

I add some my patches not related:
https://github.com/Fantu/Xen/tree/rebase/m2r-staging all already tested for
at least some days.

I tried another clean build without my patches (git reset --hard ... git
clean -f -d -x) but still fails.

I did this operations:
./configure --prefix=/usr --disable-blktap1 --disable-qemu-traditional
--disable-rombios --with-system-seabios=/usr/share/seabios/bios-256k.bin
--with-extra-qemuu-configure-args="--enable-spice --enable-usb-redir"
--disable-blktap2 --enable-ovmf --with-system-ovmf=/usr/share/ovmf/OVMF.fd
make debball

Have you checked if libxl/_paths.h is generated?

I don't think I spot anything suspicious in commit log and your build
rune.

Wei.


I checked tools/libxl/_paths.h, generated with many lines but 
XEN_DUMP_DIR is missed.

#define SBINDIR ""
#define BINDIR ""
#define LIBEXEC "/usr/lib/xen"
#define LIBEXEC_BIN "/usr/lib/xen/bin"
#define LIBDIR ""
#define SHAREDIR "/usr/share"
#define XENFIRMWAREDIR "/usr/lib/xen/boot"
#define XEN_CONFIG_DIR "/etc/xen"
#define XEN_SCRIPT_DIR "/etc/xen/scripts"
#define XEN_LOCK_DIR "/var/lock"
#define XEN_RUN_DIR "/var/run/xen"
#define XEN_PAGING_DIR "/var/lib/xen/xenpaging"

config/Paths.mk instead contain it correctly.
I checked the commit 
http://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=bf36d0792032a0245e4a17a64c8b9eb9e056f282 
but not add something about libxl/_paths.h and in it there aren't all 
path present in config.
I suppose that something is missed or there inexpected case that cause 
partial libxl/_paths.h generation (seems more probable the first case).


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel