Out of interest
Have you removed apparmor or placed all Libvirt apparmor profies into complain
mode?
If not you will get permission denied errors.
You can confirm by checking dmesg to see if you have any permission denied
messages from apparmor
Or run aa-status and see if the the Libvirt profie is in enforce/complain mode.
The /tmp/qemu.orig file is just a file we write the original qemu command to
for debugging. It is not needed
But all uses should be able to read/write to /tmp.
We wrap the qemu/kvm binary with a script that on Ubuntu can be found here
/usr/bin/kvm
If you comment out echo "qemu ${args[@]}" > /tmp/qemu.orig in this script it
will silence that warning.
https://github.com/openstack/networking-ovs-dpdk/blob/master/devstack/libs/ovs-dpdk#L104
I may remove this from our wrapper script as we most never use it for debugging
anymore however in the past it was
Useful to compare the original qemu command line and the update qemu command
line.
I don’t know if I have mentioned this before but we also have a Ubuntu version
of our getting start guide that should merge shortly
https://review.openstack.org/#/c/243190/6/doc/source/getstarted/ubuntu.rst
Regards
Sean.
From: Prathyusha Guduri [mailto:[email protected]]
Sent: Tuesday, November 24, 2015 12:42 PM
To: Mooney, Sean K
Cc: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [networking-ovs-dpdk]
Hi All,
I also found another error while launching an instance.
libvirtError: internal error: process exited while connecting to monitor:
/usr/bin/kvm-spice: line 42: /tmp/qemu.orig: Permission denied
I dont want to change any permissions manually and again face the dependency
issues. So kindly help
Thanks,
Prathyusha
On Tue, Nov 24, 2015 at 4:02 PM, Prathyusha Guduri
<[email protected]<mailto:[email protected]>> wrote:
Hi Sean,
Thanks for you kind help.
I did the following.
# apt-get install ubuntu-cloud-keyring
# echo "deb
http://ubuntu-cloud.archive.canonical.com/ubuntu<http://www.google.com/url?q=http%3A%2F%2Fubuntu-cloud.archive.canonical.com%2Fubuntu&sa=D&sntz=1&usg=AFQjCNGlCfoplf1zSbILNxCSVK23zRxa2A>"
\
"trusty-updates/kilo main" > /etc/apt/sources.list.d/cloudarchive-kilo.list
# apt-get update && apt-get dist-upgrade
and then uninstalled the libvirt and qemu that were installed manually and then
ran stack.sh after cleaning and unstacking.
Now fortunately libvirt and qemu satisfy minimum requirements.
$ virsh --version
1.2.12
$ kvm --version
/usr/bin/kvm: line 42: /tmp/qemu.orig: Permission denied
QEMU emulator version 2.2.0 (Debian 1:2.2+dfsg-5expubuntu9.3~cloud0), Copyright
(c) 2003-2008 Fabrice Bellard
Am using an ubuntu 14.04 system
$ uname -a
Linux ubuntu-Precision-Tower-5810 3.13.0-24-generic #46-Ubuntu SMP Thu Apr 10
19:11:08 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
After stack.sh which was successful, tried creating a new instance - which gave
an ERROR again.
$ nova list
+--------------------------------------+----------------+--------+------------+-------------+-------------------------------------------------------+
| ID | Name | Status | Task State |
Power State | Networks |
+--------------------------------------+----------------+--------+------------+-------------+-------------------------------------------------------+
| 31a7e160-d04c-4216-91cf-30ce86c2b1fa | demo-instance1 | ERROR | - |
NOSTATE | private=10.0.0.3, fd34:f4c5:412:0:f816:3eff:fea4:b9fe |
$ sudo service ovs-dpdk status
sourcing config
ovs alive
VHOST_CONFIG: bind to /var/run/openvswitch/vhufb8052e5-d3
2015-11-24T10:23:25Z|00126|dpdk|INFO|Socket /var/run/openvswitch/vhufb8052e5-d3
created for vhost-user port vhufb8052e5-d3
2015-11-24T10:23:25Z|00004|dpif_netdev(pmd18)|INFO|Core 2 processing port
'vhufb8052e5-d3'
2015-11-24T10:23:25Z|00002|dpif_netdev(pmd19)|INFO|Core 8 processing port
'dpdk0'
2015-11-24T10:23:25Z|00127|bridge|INFO|bridge br-int: added interface
vhufb8052e5-d3 on port 6
2015-11-24T10:23:25Z|00005|dpif_netdev(pmd18)|INFO|Core 2 processing port
'dpdk0'
2015-11-24T10:23:26Z|00128|connmgr|INFO|br-int<->unix: 1 flow_mods in the last
0 s (1 deletes)
2015-11-24T10:23:26Z|00129|ofp_util|INFO|normalization changed ofp_match,
details:
2015-11-24T10:23:26Z|00130|ofp_util|INFO| pre: in_port=5,nw_proto=58,tp_src=136
2015-11-24T10:23:26Z|00131|ofp_util|INFO|post: in_port=5
2015-11-24T10:23:26Z|00132|connmgr|INFO|br-int<->unix: 1 flow_mods in the last
0 s (1 deletes)
2015-11-24T10:23:26Z|00133|connmgr|INFO|br-int<->unix: 1 flow_mods in the last
0 s (1 deletes)
2015-11-24T10:23:29Z|00134|bridge|WARN|could not open network device
vhufb8052e5-d3 (No such device)
VHOST_CONFIG: socket created, fd:52
VHOST_CONFIG: bind to /var/run/openvswitch/vhufb8052e5-d3
2015-11-24T10:23:29Z|00135|dpdk|INFO|Socket /var/run/openvswitch/vhufb8052e5-d3
created for vhost-user port vhufb8052e5-d3
2015-11-24T10:23:29Z|00006|dpif_netdev(pmd18)|INFO|Core 2 processing port
'vhufb8052e5-d3'
2015-11-24T10:23:29Z|00003|dpif_netdev(pmd19)|INFO|Core 8 processing port
'dpdk0'
2015-11-24T10:23:29Z|00136|bridge|INFO|bridge br-int: added interface
vhufb8052e5-d3 on port 7
2015-11-24T10:23:30Z|00007|dpif_netdev(pmd18)|INFO|Core 2 processing port
'dpdk0'
0
I understand that ovs-dpdk is running. The error log of n-cpu.log is
2015-11-24 15:47:27.957 ^[[01;31mERROR nova.compute.manager
[^[[01;36mreq-24fc3f16-ccd5-4e2d-b583-60ade23bc1ed ^[[00;36mNone None^[[01;31m]
^[[01;35m^[[01;31mNo compute node record for host
ubuntu-Precision-Tower-5810^[[00m
2015-11-24 15:47:27.964 ^[[01;33mWARNING nova.compute.monitors
[^[[01;36mreq-24fc3f16-ccd5-4e2d-b583-60ade23bc1ed ^[[00;36mNone None^[[01;33m]
^[[01;35m^[[01;33mExcluding nova.compute.monitors.cpu monitor virt_driver. Not
in the list of enabled monitors (CONF.compute_monitors).^[[00m
2015-11-24 15:47:27.964 ^[[00;36mINFO nova.compute.resource_tracker
[^[[01;36mreq-24fc3f16-ccd5-4e2d-b583-60ade23bc1ed ^[[00;36mNone None^[[00;36m]
^[[01;35m^[[00;36mAuditing locally available compute resources for node
ubuntu-Precision-Tower-5810^[[00m
2015-11-24 15:47:28.087 ^[[00;32mDEBUG nova.compute.resource_tracker
[^[[01;36mreq-24fc3f16-ccd5-4e2d-b583-60ade23bc1ed ^[[00;36mNone None^[[00;32m]
^[[01;35m^[[00;32mHypervisor: free VCPUs: 9^[[00m ^[[00;33mfrom (pid=7880)
_report_hypervisor_resource_view
/opt/stack/nova/nova/compute/resource_tracker.py:588^[[00m
I dont understand where am I missing out now.
On Mon, Nov 23, 2015 at 7:50 PM, Mooney, Sean K
<[email protected]<mailto:[email protected]>> wrote:
Hi
qemu version was 2.0.0 does not support mapping hugepage as shared.
as a result the dpdk implementation of vhost-user cannot function with this
version.
Similarly libvirt 1.2.2 has no knowledge of vhost-user.
If you are on fedora 21 then the virt-preview repo packages the required
Libvirt and qemu.
On Ubuntu the kilo cloud archive also packages version that meet the minimum
versions.
Looking at the log I would agree that this error is probably related to
manually installing Libvirt and
not adding the appropriate libvirtd configuration options.
Devstack does not have native support for installing a vhost-user compatible
Libvirt or qemu.
when using our networking-ovs-dpdk plugin we ask you to enable the virt-prevew
or cloud archive
Before stacking so that when devestack install Libvirt/qemu it get compatible
versions.
for centos as we have been unable to find an equivalent we explicitly install
Libvirt/qemu.
In general we don’t provide support for installing the required version
Libvirt/qemu in
the networking-ovs-dpdk devstack plugin.
What os are you currently using? ubuntu?
in your case enabling the kilo cloud archive, uinstalling Libvirt/qemu and then
restacking
should provide the appropriate packages.
In liberty we use that standard neutron openvswich agent binary and set the
agent type to DPDK OVS Agent to enable
the ovsdpdk ml2 driver to manage the node. In mitaka the standard ovs neutron
agent and standard openvswitch ml2 drivers will be used instead.
To confirm if ovs-dpdk is running you can use the following command
sudo service ovs-dpdk status
this will check if ovs-dpdk is running by reading the pid files and checking if
the ovsdb and ovs-vswitchd
process are running and tail the last 20ish lines of the vswitchd log file.
note this is not a systemd service so the systemd equivalent command will not
work.
Alternative you can user ps and grep
ps aux | grep ovs
regards
sean.
From: Prathyusha Guduri
[mailto:[email protected]<mailto:[email protected]>]
Sent: Monday, November 23, 2015 9:16 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [networking-ovs-dpdk]
Hi Sean,
Thanks for your help before. It would be great if you look into another issue
too.
Am able to run stack.sh successfully and all services are up. But,
libvirt version was 1.2.2 and qemu version was 2.0.0
To satisfy the minimum requirement of qemu- version >=2.1 and libvirt-version
>= 1.2.10
I manually installed qemu and libvirt from respective sources.
Now
$ kvm --version
/usr/bin/kvm: line 42: /tmp/qemu.orig: Permission denied
QEMU emulator version 2.1.3, Copyright (c) 2003-2008 Fabrice Bellard
$ virsh --version
1.2.10
So basic requirement is satisfied.
Before creating an instance ran the below command,
$ nova flavor-key m1.tiny set "hw:mem_page_size=large"
Now created an instance
$ nova boot --flavor m1.tiny --image cirros-0.3.4-x86_64-uec --nic
net-id=445e2dc5-221b-48ea-aea4-d04dee12fc7f --security-group default
demo-instance1
It gives the ERROR :
2015-11-23 13:19:59.654 ERROR nova.virt.libvirt.host
[req-2d9d060d-1934-4e9e-af1c-010e177bea11 None None] Connection to libvirt
failed: error from service: CheckAuthorization: Did not receive a reply.
Possible causes include: the remote application did not send a reply, the
message bus security policy blocked the reply, the reply timeout expired, or
the network connection was broken.
2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host Traceback (most recent
call last):
2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host File
"/opt/stack/nova/nova/virt/libvirt/host.py", line 527, in get_connection
2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host conn =
self._get_connection()
2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host File
"/opt/stack/nova/nova/virt/libvirt/host.py", line 514, in _get_connection
2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host wrapped_conn =
self._get_new_connection()
2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host File
"/opt/stack/nova/nova/virt/libvirt/host.py", line 466, in _get_new_connection
2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host wrapped_conn =
self._connect(self._uri, self._read_only)
2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host File
"/opt/stack/nova/nova/virt/libvirt/host.py", line 320, in _connect
2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host libvirt.openAuth, uri,
auth, flags)
2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host File
"/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 141, in
proxy_call
2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host rv = execute(f, *args,
**kwargs)
2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host File
"/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 122, in execute
2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host six.reraise(c, e, tb)
2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host File
"/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 80, in tworker
2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host rv = meth(*args,
**kwargs)
2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host File
"/usr/local/lib/python2.7/dist-packages/libvirt.py", line 105, in openAuth
2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host if ret is None:raise
libvirtError('virConnectOpenAuth() failed')
2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host libvirtError: error from
service: CheckAuthorization: Did not receive a reply. Possible causes include:
the remote application did not send a reply, the message bus security policy
blocked the reply, the reply timeout expired, or the network connection was
broken.
2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host
Traceback (most recent call last):
I suspect this is because, I manually installed libvirt and qemu. My doubt is
why devstack is not installing a correct version when it is supposed to. why a
version less than min requirement is being installed???
Now that because am installing manually, there might be a problem with groups -
devstack creates some group and installs but manual installation doesn't bother
about that groups.
Can you please suggest a way on how do avoid that???
Also, I just want to make sure that the agent running is neutron-openvswitch
only. No ovsdpdk agent running.
$ ps -Al | grep neutron
0 S 1000 8882 8859 3 80 0 - 49946 ep_pol pts/34 00:02:24
neutron-openvsw
But
$ neutron agent-list
3385a430-5738-43cb-b853-059add5ab602 | DPDK OVS Agent |
ubuntu-Precision-Tower-5810 | :-) | True | neutron-openvswitch-agent
So this implies that dpdk agent is running right??? I remember reading in
launchpad bugs that ovsdpdk agent is removed and that now openvswitch takes
care of everything. Just wanted to confirm that my setup has ovs-dpdk running.
Regards,
Prathyusha
On Wed, Nov 18, 2015 at 7:23 PM, James Page
<[email protected]<mailto:[email protected]>> wrote:
Hi Sean
On Wed, Nov 18, 2015 at 12:30 PM, Mooney, Sean K
<[email protected]<mailto:[email protected]>> wrote:
Hi james
Yes we are planning on testing the packaged release to see if it is compatible
with our ml2 driver and the
Changes we are submitting upstream. If it is we will add a use binary flag to
our devstack plugin to skip the
Compilation step and use that instead on 15.10 or 14.04 cloud-archive:liberty
Excellent.
As part of your packaging did ye fix pciutils to correctly report the unused
drivers when an interface is bound
The dpdk driver? Also does it support both igb_uio and/or vfio-pci drivers for
dpdk interface?
Re pcituils, we've not done any work in that area - can you give an example of
what you would expect?
The dpdk package supports both driver types in /etc/dpdk/interfaces - when you
declare an adapter for use, you get to specify the module you want to use as
well; we're relying the in-tree kernel drivers (uio-pci-generic and vfio-pci)
right now.
Anyway yes I hope to check it out and seeing what ye have done. When ovs-dpdk
starts getting packaged in more operating systems
We will probably swap our default to the binary install though we will keep the
source install option as it allows us to work on new features
Before they are packaged and to have better performance.
That sounds sensible; re 'better performance' - yeah we do have to baseline the
optimizations at compile time right now (ssse3 only right now) , but I really
hope that does change so that we can move to a runtime CPU feature detection
model, allowing the best possible performance through the packages we have in
Ubuntu (or any other distribution for that matter).
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
[email protected]?subject:unsubscribe<http://[email protected]?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
