Re: [RFCv2 00/12] Introduce host-side virtio queue and CAIF Virtio.
On Fri, Dec 21, 2012 at 7:11 AM, Rusty Russell ru...@rustcorp.com.au wrote: Michael S. Tsirkin m...@redhat.com writes: On Wed, Dec 05, 2012 at 03:36:58PM +0100, Sjur Brændeland wrote: Feedback on this patch-set is appreciated, particularly on structure and code-reuse between vhost.c and the host-side virtio-queue. I'd also like some suggestions on how to handle the build configuration better - currently there are some unnecessary build dependencies. Rusty seems to disagree but one of the concerns people have about vhost is security; so I value getting as much static checking as we can. This discards __user annotations so this doesn't work for me. Sometimes, when we generalize code, we lose some type safety. Callbacks take void *, for example. And it happens *all the time* with const. We don't create a set of parallel const-safe routines. Extracting common code where it can be shared provides better, not worse security, because more people will read it. I've never audited the vhost code, for example. We already have a 'struct vring', we should just use that. I meant to do more work on this, but I've run out of time :( I was thinking of a drivers/virtio/virtio_ring.c, and in that we'd put the wrappers for vhost_net/blk, and for CAIF, etc. We could at least have a variant which did the __user etc thing, even if you have to pass in getdesc(). getu16: get_user or assignment. getdesc: copy (and check, translate) the descriptor. getmem/putmem: memcpy or copy_to/from_user. (Completely untested, of course...) Thoughts? Any thoughts on this Michael? Regards, Sjur ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[PATCH V3 1/2] virtio-net: fix the set affinity bug when CPU IDs are not consecutive
As Michael mentioned, set affinity and select queue will not work very well when CPU IDs are not consecutive, this can happen with hot unplug. Fix this bug by traversal the online CPUs, and create a per cpu variable to find the mapping from CPU to the preferable virtual-queue. Cc: Rusty Russell ru...@rustcorp.com.au Cc: Michael S. Tsirkin m...@redhat.com Cc: Jason Wang jasow...@redhat.com Cc: Eric Dumazet erdnet...@gmail.com Cc: virtualization@lists.linux-foundation.org Cc: net...@vger.kernel.org Signed-off-by: Wanlong Gao gaowanl...@cn.fujitsu.com --- drivers/net/virtio_net.c | 39 +-- 1 file changed, 29 insertions(+), 10 deletions(-) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index a6fcf15..a77f86c 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -41,6 +41,8 @@ module_param(gso, bool, 0444); #define VIRTNET_SEND_COMMAND_SG_MAX2 #define VIRTNET_DRIVER_VERSION 1.0.0 +DEFINE_PER_CPU(int, vq_index) = -1; + struct virtnet_stats { struct u64_stats_sync tx_syncp; struct u64_stats_sync rx_syncp; @@ -1016,6 +1018,7 @@ static int virtnet_vlan_rx_kill_vid(struct net_device *dev, u16 vid) static void virtnet_set_affinity(struct virtnet_info *vi, bool set) { int i; + int cpu; /* In multiqueue mode, when the number of cpu is equal to the number of * queue pairs, we let the queue pairs to be private to one cpu by @@ -1029,16 +1032,29 @@ static void virtnet_set_affinity(struct virtnet_info *vi, bool set) return; } - for (i = 0; i vi-max_queue_pairs; i++) { - int cpu = set ? i : -1; - virtqueue_set_affinity(vi-rq[i].vq, cpu); - virtqueue_set_affinity(vi-sq[i].vq, cpu); - } + if (set) { + i = 0; + for_each_online_cpu(cpu) { + virtqueue_set_affinity(vi-rq[i].vq, cpu); + virtqueue_set_affinity(vi-sq[i].vq, cpu); + per_cpu(vq_index, cpu) = i; + i++; + if (i = vi-max_queue_pairs) + break; + } - if (set) vi-affinity_hint_set = true; - else + } else { + for(i = 0; i vi-max_queue_pairs; i++) { + virtqueue_set_affinity(vi-rq[i].vq, -1); + virtqueue_set_affinity(vi-sq[i].vq, -1); + } + + for_each_online_cpu(cpu) + per_cpu(vq_index, cpu) = -1; + vi-affinity_hint_set = false; + } } static void virtnet_get_ringparam(struct net_device *dev, @@ -1127,12 +1143,15 @@ static int virtnet_change_mtu(struct net_device *dev, int new_mtu) /* To avoid contending a lock hold by a vcpu who would exit to host, select the * txq based on the processor id. - * TODO: handle cpu hotplug. */ static u16 virtnet_select_queue(struct net_device *dev, struct sk_buff *skb) { - int txq = skb_rx_queue_recorded(skb) ? skb_get_rx_queue(skb) : - smp_processor_id(); + int txq = 0; + + if (skb_rx_queue_recorded(skb)) + txq = skb_get_rx_queue(skb); + else if ((txq = per_cpu(vq_index, smp_processor_id())) == -1) + txq = 0; while (unlikely(txq = dev-real_num_tx_queues)) txq -= dev-real_num_tx_queues; -- 1.8.1 ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[PATCH V3 2/2] virtio-net: reset virtqueue affinity when doing cpu hotplug
Add a cpu notifier to virtio-net, so that we can reset the virtqueue affinity if the cpu hotplug happens. It improve the performance through enabling or disabling the virtqueue affinity after doing cpu hotplug. Cc: Rusty Russell ru...@rustcorp.com.au Cc: Michael S. Tsirkin m...@redhat.com Cc: Jason Wang jasow...@redhat.com Cc: Eric Dumazet erdnet...@gmail.com Cc: virtualization@lists.linux-foundation.org Cc: net...@vger.kernel.org Signed-off-by: Wanlong Gao gaowanl...@cn.fujitsu.com --- drivers/net/virtio_net.c | 30 ++ 1 file changed, 30 insertions(+) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index a77f86c..31a25db 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -26,6 +26,7 @@ #include linux/scatterlist.h #include linux/if_vlan.h #include linux/slab.h +#include linux/cpu.h static int napi_weight = 128; module_param(napi_weight, int, 0444); @@ -125,6 +126,9 @@ struct virtnet_info { /* Does the affinity hint is set for virtqueues? */ bool affinity_hint_set; + + /* CPU hot plug notifier */ + struct notifier_block nb; }; struct skb_vnet_hdr { @@ -1057,6 +1061,23 @@ static void virtnet_set_affinity(struct virtnet_info *vi, bool set) } } +static int virtnet_cpu_callback(struct notifier_block *nfb, + unsigned long action, void *hcpu) +{ + struct virtnet_info *vi = container_of(nfb, struct virtnet_info, nb); + switch(action) { + case CPU_ONLINE: + case CPU_ONLINE_FROZEN: + case CPU_DEAD: + case CPU_DEAD_FROZEN: + virtnet_set_affinity(vi, true); + break; + default: + break; + } + return NOTIFY_OK; +} + static void virtnet_get_ringparam(struct net_device *dev, struct ethtool_ringparam *ring) { @@ -1518,6 +1539,13 @@ static int virtnet_probe(struct virtio_device *vdev) } } + vi-nb.notifier_call = virtnet_cpu_callback; + err = register_hotcpu_notifier(vi-nb); + if (err) { + pr_debug(virtio_net: registering cpu notifier failed\n); + goto free_recv_bufs; + } + /* Assume link up if device can't report link status, otherwise get link status from config. */ if (virtio_has_feature(vi-vdev, VIRTIO_NET_F_STATUS)) { @@ -1562,6 +1590,8 @@ static void virtnet_remove(struct virtio_device *vdev) { struct virtnet_info *vi = vdev-priv; + unregister_hotcpu_notifier(vi-nb); + /* Prevent config work handler from accessing the device. */ mutex_lock(vi-config_lock); vi-config_enable = false; -- 1.8.1 ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH V3 1/2] virtio-net: fix the set affinity bug when CPU IDs are not consecutive
On 01/08/2013 06:07 PM, Wanlong Gao wrote: As Michael mentioned, set affinity and select queue will not work very well when CPU IDs are not consecutive, this can happen with hot unplug. Fix this bug by traversal the online CPUs, and create a per cpu variable to find the mapping from CPU to the preferable virtual-queue. Cc: Rusty Russell ru...@rustcorp.com.au Cc: Michael S. Tsirkin m...@redhat.com Cc: Jason Wang jasow...@redhat.com Cc: Eric Dumazet erdnet...@gmail.com Cc: virtualization@lists.linux-foundation.org Cc: net...@vger.kernel.org Signed-off-by: Wanlong Gao gaowanl...@cn.fujitsu.com --- drivers/net/virtio_net.c | 39 +-- 1 file changed, 29 insertions(+), 10 deletions(-) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index a6fcf15..a77f86c 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -41,6 +41,8 @@ module_param(gso, bool, 0444); #define VIRTNET_SEND_COMMAND_SG_MAX2 #define VIRTNET_DRIVER_VERSION 1.0.0 +DEFINE_PER_CPU(int, vq_index) = -1; + I think this should not be a global one, consider we may have more than one virtio-net cards with different max queues. struct virtnet_stats { struct u64_stats_sync tx_syncp; struct u64_stats_sync rx_syncp; @@ -1016,6 +1018,7 @@ static int virtnet_vlan_rx_kill_vid(struct net_device *dev, u16 vid) static void virtnet_set_affinity(struct virtnet_info *vi, bool set) { int i; + int cpu; /* In multiqueue mode, when the number of cpu is equal to the number of * queue pairs, we let the queue pairs to be private to one cpu by @@ -1029,16 +1032,29 @@ static void virtnet_set_affinity(struct virtnet_info *vi, bool set) return; } - for (i = 0; i vi-max_queue_pairs; i++) { - int cpu = set ? i : -1; - virtqueue_set_affinity(vi-rq[i].vq, cpu); - virtqueue_set_affinity(vi-sq[i].vq, cpu); - } + if (set) { + i = 0; + for_each_online_cpu(cpu) { + virtqueue_set_affinity(vi-rq[i].vq, cpu); + virtqueue_set_affinity(vi-sq[i].vq, cpu); + per_cpu(vq_index, cpu) = i; + i++; + if (i = vi-max_queue_pairs) + break; Can this happen? we check only set when the number are equal. + } - if (set) vi-affinity_hint_set = true; - else + } else { + for(i = 0; i vi-max_queue_pairs; i++) { + virtqueue_set_affinity(vi-rq[i].vq, -1); + virtqueue_set_affinity(vi-sq[i].vq, -1); + } + + for_each_online_cpu(cpu) + per_cpu(vq_index, cpu) = -1; + This looks suboptimal since it may leads only txq zero is used. vi-affinity_hint_set = false; + } } static void virtnet_get_ringparam(struct net_device *dev, @@ -1127,12 +1143,15 @@ static int virtnet_change_mtu(struct net_device *dev, int new_mtu) /* To avoid contending a lock hold by a vcpu who would exit to host, select the * txq based on the processor id. - * TODO: handle cpu hotplug. */ static u16 virtnet_select_queue(struct net_device *dev, struct sk_buff *skb) { - int txq = skb_rx_queue_recorded(skb) ? skb_get_rx_queue(skb) : - smp_processor_id(); + int txq = 0; + + if (skb_rx_queue_recorded(skb)) + txq = skb_get_rx_queue(skb); + else if ((txq = per_cpu(vq_index, smp_processor_id())) == -1) + txq = 0; while (unlikely(txq = dev-real_num_tx_queues)) txq -= dev-real_num_tx_queues; ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH V3 1/2] virtio-net: fix the set affinity bug when CPU IDs are not consecutive
Wanlong Gao gaowanl...@cn.fujitsu.com writes: */ static u16 virtnet_select_queue(struct net_device *dev, struct sk_buff *skb) { - int txq = skb_rx_queue_recorded(skb) ? skb_get_rx_queue(skb) : - smp_processor_id(); + int txq = 0; + + if (skb_rx_queue_recorded(skb)) + txq = skb_get_rx_queue(skb); + else if ((txq = per_cpu(vq_index, smp_processor_id())) == -1) + txq = 0; You should use __get_cpu_var() instead of smp_processor_id() here, ie: else if ((txq = __get_cpu_var(vq_index)) == -1) And AFAICT, no reason to initialize txq to 0 to start with. So: int txq; if (skb_rx_queue_recorded(skb)) txq = skb_get_rx_queue(skb); else { txq = __get_cpu_var(vq_index); if (txq == -1) txq = 0; } Now, just to confirm, I assume this can happen even if we use vq_index, right, because of races with virtnet_set_channels? while (unlikely(txq = dev-real_num_tx_queues)) txq -= dev-real_num_tx_queues; Thanks, Rusty. ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [RFCv2 00/12] Introduce host-side virtio queue and CAIF Virtio.
Sjur Brændeland sjurb...@gmail.com writes: On Fri, Dec 21, 2012 at 7:11 AM, Rusty Russell ru...@rustcorp.com.au wrote: Michael S. Tsirkin m...@redhat.com writes: On Wed, Dec 05, 2012 at 03:36:58PM +0100, Sjur Brændeland wrote: Feedback on this patch-set is appreciated, particularly on structure and code-reuse between vhost.c and the host-side virtio-queue. I'd also like some suggestions on how to handle the build configuration better - currently there are some unnecessary build dependencies. Rusty seems to disagree but one of the concerns people have about vhost is security; so I value getting as much static checking as we can. This discards __user annotations so this doesn't work for me. Sometimes, when we generalize code, we lose some type safety. Callbacks take void *, for example. And it happens *all the time* with const. We don't create a set of parallel const-safe routines. Extracting common code where it can be shared provides better, not worse security, because more people will read it. I've never audited the vhost code, for example. We already have a 'struct vring', we should just use that. I meant to do more work on this, but I've run out of time :( I was thinking of a drivers/virtio/virtio_ring.c, and in that we'd put the wrappers for vhost_net/blk, and for CAIF, etc. We could at least have a variant which did the __user etc thing, even if you have to pass in getdesc(). getu16: get_user or assignment. getdesc: copy (and check, translate) the descriptor. getmem/putmem: memcpy or copy_to/from_user. (Completely untested, of course...) Thoughts? Any thoughts on this Michael? I'm actually testing code this time, and it's mutated a little. It basically involves moving much of vring.c into a virtio_host.c: the parts which actually touch the ring. Then it provides accessors for vring.c to use which are __user-safe (all casts are inside virtio_host.c). I should have something to post by end of today, my time... Thanks, Rusty. ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[PATCH 00/12] VMCI for Linux upstreaming
* * * This series of VMCI linux upstreaming patches include latest udpate from VMware to address Greg's and all other's code review comments. Summary of changes: - Rebase our linux kernel tree from v3.5 to v3.7. - Fix all checkpatch warnings and errors. Fix some checkpatch with -strict errors. This addresses Greg's comment: On 15 Nov 2012 15:47:14 -0800 Re: [PATCH 07/12] VMCI: queue pairs implementation. chckpatch sanity checking. - Fix vmci_get_contextid naming error. - Fix host side queue pair creation bug. - Fix resource hash table lookup issue. - Remove __devinit and __devexit device driver annotations. - Remove ASSERT/BUG_ON for vmci and vsock source codes. This addresses Greg's comment on On 15 Nov 2012 15:47:14 -0800 Re: [PATCH 02/12] VMCI: datagram implementation. Remove all BUG_ON(), asserts. - use standard Linux kernel ioctl interface. This addresses Greg's comment on On Thu, 15 Nov 2012 16:01:18 -0800 Re: [PATCH 12/12] VMCI: Some header and config files. use linux in-kernel IO macros for VMCI. - Add vmci ioctl code entry in ioctl-number.txt. - Remove printk from header file. Align properly for macro PCI_VENDOR_ID_VMWARE. Remove #define for VMCI_DRIVER_VERSION_STRING. Remove #define for MODULE_NAME. Remove #define for pr_fmt and #define ASSERT(x). Remove inline function VMCI_MAKE_HANDLE and vmci_make_handle which return a structure on stack. use macro instead and use C99 initialization. Remove #define for VMCI_HANDLE_TO_CONTEXT_ID and VMCI_HANDLE_TO_RESOURCE_ID Change macro to inline function for VMCI_HANDLE_EQUAL. Remove u32 typedefs for vmci_id, vmci_event, vmci_privilege_flags. Use C99 style initialization for struct VMCI_INVALID_HANDLE. Use inline function instead of macro for VMCI_HANDLE_INVALID. No change for vmw_vmci_api.h location (under include/linux/ directory). No change for macros VMCI_CONTEXT_IS_VM, VMCI_HOST_CONTEXT_ID, VMCI_RESERVED_CID_LIMIT, VMCI_HYPERVISOR_CONTEXT_ID and VMCI_WELL_KNOWN_CONTEXT_ID. Still in header file include/linux/vmw_vmci_defs.h This addresses Greg's several comments on Thu, 15 Nov 2012 16:01:18 -0800 Re: [PATCH 12/12] VMCI: Some header and config files. * * * In an effort to improve the out-of-the-box experience with Linux kernels for VMware users, VMware is working on readying the Virtual Machine Communication Interface (vmw_vmci) and VMCI Sockets (vmw_vsock) kernel modules for inclusion in the Linux kernel. The purpose of this post is to acquire feedback on the vmw_vmci kernel module. The vmw_vsock kernel module will be presented in a later post. * * * VMCI allows virtual machines to communicate with host kernel modules and the VMware hypervisors. User level applications both in a virtual machine and on the host can use vmw_vmci through VMCI Sockets, a socket address family designed to be compatible with UDP and TCP at the interface level. Today, VMCI and VMCI Sockets are used by the VMware shared folders (HGFS) and various VMware Tools components inside the guest for zero-config, network-less access to VMware host services. In addition to this, VMware's users are using VMCI Sockets for various applications, where network access of the virtual machine is restricted or non-existent. Examples of this are VMs communicating with device proxies for proprietary hardware running as host applications and automated testing of applications running within virtual machines. In a virtual machine, VMCI is exposed as a regular PCI device. The primary communication mechanisms supported are a point-to-point bidirectional transport based on a pair of memory-mapped queues, and asynchronous notifications in the form of datagrams and doorbells. These features are available to kernel level components such as HGFS and VMCI Sockets through the VMCI kernel API. In addition to this, the VMCI kernel API provides support for receiving events related to the state of the VMCI communication channels, and the virtual machine itself. Outside the virtual machine, the host side support of the VMCI kernel module makes the same VMCI kernel API available to VMCI endpoints on the host. In addition to this, the host side manages each VMCI device in a virtual machine through a context object. This context object serves to identify the virtual machine for communication, and to track the resource consumption of the given VMCI device. Both operations related to communication between the virtual machine and the host kernel, and those related to the management of the VMCI device state in the host kernel, are invoked by the user level component of the hypervisor through a set of ioctls on the
[PATCH 01/12] VMCI: context implementation.
VMCI Context code maintains state for vmci and allows the driver to communicate with multiple VMs. Signed-off-by: George Zhang georgezh...@vmware.com Acked-by: Andy king ack...@vmware.com Acked-by: Dmitry Torokhov d...@vmware.com --- drivers/misc/vmw_vmci/vmci_context.c | 1214 ++ drivers/misc/vmw_vmci/vmci_context.h | 182 + 2 files changed, 1396 insertions(+), 0 deletions(-) create mode 100644 drivers/misc/vmw_vmci/vmci_context.c create mode 100644 drivers/misc/vmw_vmci/vmci_context.h diff --git a/drivers/misc/vmw_vmci/vmci_context.c b/drivers/misc/vmw_vmci/vmci_context.c new file mode 100644 index 000..f866a4b --- /dev/null +++ b/drivers/misc/vmw_vmci/vmci_context.c @@ -0,0 +1,1214 @@ +/* + * VMware VMCI Driver + * + * Copyright (C) 2012 VMware, Inc. All rights reserved. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License as published by the + * Free Software Foundation version 2 and no later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY + * or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * for more details. + */ + +#include linux/vmw_vmci_defs.h +#include linux/vmw_vmci_api.h +#include linux/highmem.h +#include linux/kernel.h +#include linux/module.h +#include linux/sched.h +#include linux/slab.h + +#include vmci_queue_pair.h +#include vmci_datagram.h +#include vmci_doorbell.h +#include vmci_context.h +#include vmci_driver.h +#include vmci_event.h + +/* + * List of current VMCI contexts. Contexts can be added by + * vmci_ctx_create() and removed via vmci_ctx_destroy(). + * These, along with context lookup, are protected by the + * list structure's lock. + */ +static struct { + struct list_head head; + spinlock_t lock; /* Spinlock for context list operations */ +} ctx_list = { + .head = LIST_HEAD_INIT(ctx_list.head), + .lock = __SPIN_LOCK_UNLOCKED(ctx_list.lock), +}; + +/* Used by contexts that did not set up notify flag pointers */ +static bool ctx_dummy_notify; + +static void ctx_signal_notify(struct vmci_ctx *context) +{ + *context-notify = true; +} + +static void ctx_clear_notify(struct vmci_ctx *context) +{ + *context-notify = false; +} + +/* + * If nothing requires the attention of the guest, clears both + * notify flag and call. + */ +static void ctx_clear_notify_call(struct vmci_ctx *context) +{ + if (context-pending_datagrams == 0 + vmci_handle_arr_get_size(context-pending_doorbell_array) == 0) + ctx_clear_notify(context); +} + +/* + * Sets the context's notify flag iff datagrams are pending for this + * context. Called from vmci_setup_notify(). + */ +void vmci_ctx_check_signal_notify(struct vmci_ctx *context) +{ + spin_lock(context-lock); + if (context-pending_datagrams) + ctx_signal_notify(context); + spin_unlock(context-lock); +} + +/* + * Allocates and initializes a VMCI context. + */ +struct vmci_ctx *vmci_ctx_create(u32 cid, u32 priv_flags, +uintptr_t event_hnd, +int user_version, +const struct cred *cred) +{ + struct vmci_ctx *context; + int error; + + if (cid == VMCI_INVALID_ID) { + pr_devel(Invalid context ID for VMCI context\n); + error = -EINVAL; + goto err_out; + } + + if (priv_flags ~VMCI_PRIVILEGE_ALL_FLAGS) { + pr_devel(Invalid flag (flags=0x%x) for VMCI context\n, +priv_flags); + error = -EINVAL; + goto err_out; + } + + if (user_version == 0) { + pr_devel(Invalid suer_version %d\n, user_version); + error = -EINVAL; + goto err_out; + } + + context = kzalloc(sizeof(*context), GFP_KERNEL); + if (!context) { + pr_warn(Failed to allocate memory for VMCI context\n); + error = -EINVAL; + goto err_out; + } + + kref_init(context-kref); + spin_lock_init(context-lock); + INIT_LIST_HEAD(context-list_item); + INIT_LIST_HEAD(context-datagram_queue); + INIT_LIST_HEAD(context-notifier_list); + + /* Initialize host-specific VMCI context. */ + init_waitqueue_head(context-host_context.wait_queue); + + context-queue_pair_array = vmci_handle_arr_create(0); + if (!context-queue_pair_array) { + error = -ENOMEM; + goto err_free_ctx; + } + + context-doorbell_array = vmci_handle_arr_create(0); + if (!context-doorbell_array) { + error = -ENOMEM; + goto err_free_qp_array; + } + + context-pending_doorbell_array = vmci_handle_arr_create(0); + if
[PATCH 02/12] VMCI: datagram implementation.
VMCI datagram Implements datagrams to allow data to be sent between host and guest. Signed-off-by: George Zhang georgezh...@vmware.com Acked-by: Andy king ack...@vmware.com Acked-by: Dmitry Torokhov d...@vmware.com --- drivers/misc/vmw_vmci/vmci_datagram.c | 500 + drivers/misc/vmw_vmci/vmci_datagram.h | 52 +++ 2 files changed, 552 insertions(+), 0 deletions(-) create mode 100644 drivers/misc/vmw_vmci/vmci_datagram.c create mode 100644 drivers/misc/vmw_vmci/vmci_datagram.h diff --git a/drivers/misc/vmw_vmci/vmci_datagram.c b/drivers/misc/vmw_vmci/vmci_datagram.c new file mode 100644 index 000..ed5c433 --- /dev/null +++ b/drivers/misc/vmw_vmci/vmci_datagram.c @@ -0,0 +1,500 @@ +/* + * VMware VMCI Driver + * + * Copyright (C) 2012 VMware, Inc. All rights reserved. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License as published by the + * Free Software Foundation version 2 and no later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY + * or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * for more details. + */ + +#include linux/vmw_vmci_defs.h +#include linux/vmw_vmci_api.h +#include linux/module.h +#include linux/sched.h +#include linux/slab.h +#include linux/bug.h + +#include vmci_datagram.h +#include vmci_resource.h +#include vmci_context.h +#include vmci_driver.h +#include vmci_event.h +#include vmci_route.h + +/* + * struct datagram_entry describes the datagram entity. It is used for datagram + * entities created only on the host. + */ +struct datagram_entry { + struct vmci_resource resource; + u32 flags; + bool run_delayed; + vmci_datagram_recv_cb recv_cb; + void *client_data; + u32 priv_flags; +}; + +struct delayed_datagram_info { + struct datagram_entry *entry; + struct vmci_datagram msg; + struct work_struct work; + bool in_dg_host_queue; +}; + +/* Number of in-flight host-host datagrams */ +static atomic_t delayed_dg_host_queue_size = ATOMIC_INIT(0); + +/* + * Create a datagram entry given a handle pointer. + */ +static int dg_create_handle(u32 resource_id, + u32 flags, + u32 priv_flags, + vmci_datagram_recv_cb recv_cb, + void *client_data, struct vmci_handle *out_handle) +{ + int result; + u32 context_id; + struct vmci_handle handle; + struct datagram_entry *entry; + + if ((flags VMCI_FLAG_WELLKNOWN_DG_HND) != 0) + return VMCI_ERROR_INVALID_ARGS; + + if ((flags VMCI_FLAG_ANYCID_DG_HND) != 0) { + context_id = VMCI_INVALID_ID; + } else { + context_id = vmci_get_context_id(); + if (context_id == VMCI_INVALID_ID) + return VMCI_ERROR_NO_RESOURCES; + } + + handle = vmci_make_handle(context_id, resource_id); + + entry = kmalloc(sizeof(*entry), GFP_KERNEL); + if (!entry) { + pr_warn(Failed allocating memory for datagram entry\n); + return VMCI_ERROR_NO_MEM; + } + + entry-run_delayed = (flags VMCI_FLAG_DG_DELAYED_CB) ? true : false; + entry-flags = flags; + entry-recv_cb = recv_cb; + entry-client_data = client_data; + entry-priv_flags = priv_flags; + + /* Make datagram resource live. */ + result = vmci_resource_add(entry-resource, + VMCI_RESOURCE_TYPE_DATAGRAM, + handle); + if (result != VMCI_SUCCESS) { + pr_warn(Failed to add new resource (handle=0x%x:0x%x), error: %d\n, + handle.context, handle.resource, result); + kfree(entry); + return result; + } + + *out_handle = vmci_resource_handle(entry-resource); + return VMCI_SUCCESS; +} + +/* + * Internal utility function with the same purpose as + * vmci_datagram_get_priv_flags that also takes a context_id. + */ +static int vmci_datagram_get_priv_flags(u32 context_id, + struct vmci_handle handle, + u32 *priv_flags) +{ + if (context_id == VMCI_INVALID_ID) + return VMCI_ERROR_INVALID_ARGS; + + if (context_id == VMCI_HOST_CONTEXT_ID) { + struct datagram_entry *src_entry; + struct vmci_resource *resource; + + resource = vmci_resource_by_handle(handle, + VMCI_RESOURCE_TYPE_DATAGRAM); + if (!resource) + return VMCI_ERROR_INVALID_ARGS; + + src_entry = container_of(resource, struct datagram_entry, +
[PATCH 03/12] VMCI: doorbell implementation.
VMCI doorbell code allows for notifcations between host and guest. Signed-off-by: George Zhang georgezh...@vmware.com Acked-by: Andy king ack...@vmware.com Acked-by: Dmitry Torokhov d...@vmware.com --- drivers/misc/vmw_vmci/vmci_doorbell.c | 604 + drivers/misc/vmw_vmci/vmci_doorbell.h | 51 +++ 2 files changed, 655 insertions(+), 0 deletions(-) create mode 100644 drivers/misc/vmw_vmci/vmci_doorbell.c create mode 100644 drivers/misc/vmw_vmci/vmci_doorbell.h diff --git a/drivers/misc/vmw_vmci/vmci_doorbell.c b/drivers/misc/vmw_vmci/vmci_doorbell.c new file mode 100644 index 000..c3e8397 --- /dev/null +++ b/drivers/misc/vmw_vmci/vmci_doorbell.c @@ -0,0 +1,604 @@ +/* + * VMware VMCI Driver + * + * Copyright (C) 2012 VMware, Inc. All rights reserved. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License as published by the + * Free Software Foundation version 2 and no later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY + * or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * for more details. + */ + +#include linux/vmw_vmci_defs.h +#include linux/vmw_vmci_api.h +#include linux/completion.h +#include linux/hash.h +#include linux/kernel.h +#include linux/list.h +#include linux/module.h +#include linux/sched.h +#include linux/slab.h + +#include vmci_datagram.h +#include vmci_doorbell.h +#include vmci_resource.h +#include vmci_driver.h +#include vmci_route.h + + +#define VMCI_DOORBELL_INDEX_BITS 6 +#define VMCI_DOORBELL_INDEX_TABLE_SIZE (1 VMCI_DOORBELL_INDEX_BITS) +#define VMCI_DOORBELL_HASH(_idx) hash_32(_idx, VMCI_DOORBELL_INDEX_BITS) + +/* + * DoorbellEntry describes the a doorbell notification handle allocated by the + * host. + */ +struct dbell_entry { + struct vmci_resource resource; + struct hlist_node node; + struct work_struct work; + vmci_callback notify_cb; + void *client_data; + u32 idx; + u32 priv_flags; + bool run_delayed; + atomic_t active;/* Only used by guest personality */ +}; + +/* The VMCI index table keeps track of currently registered doorbells. */ +struct dbell_index_table { + spinlock_t lock;/* Index table lock */ + struct hlist_head entries[VMCI_DOORBELL_INDEX_TABLE_SIZE]; +}; + +static struct dbell_index_table vmci_doorbell_it = { + .lock = __SPIN_LOCK_UNLOCKED(vmci_doorbell_it.lock), +}; + +/* + * The max_notify_idx is one larger than the currently known bitmap index in + * use, and is used to determine how much of the bitmap needs to be scanned. + */ +static u32 max_notify_idx; + +/* + * The notify_idx_count is used for determining whether there are free entries + * within the bitmap (if notify_idx_count + 1 max_notify_idx). + */ +static u32 notify_idx_count; + +/* + * The last_notify_idx_reserved is used to track the last index handed out - in + * the case where multiple handles share a notification index, we hand out + * indexes round robin based on last_notify_idx_reserved. + */ +static u32 last_notify_idx_reserved; + +/* This is a one entry cache used to by the index allocation. */ +static u32 last_notify_idx_released = PAGE_SIZE; + + +/* + * Utility function that retrieves the privilege flags associated + * with a given doorbell handle. For guest endpoints, the + * privileges are determined by the context ID, but for host + * endpoints privileges are associated with the complete + * handle. Hypervisor endpoints are not yet supported. + */ +int vmci_dbell_get_priv_flags(struct vmci_handle handle, u32 *priv_flags) +{ + if (priv_flags == NULL || handle.context == VMCI_INVALID_ID) + return VMCI_ERROR_INVALID_ARGS; + + if (handle.context == VMCI_HOST_CONTEXT_ID) { + struct dbell_entry *entry; + struct vmci_resource *resource; + + resource = vmci_resource_by_handle(handle, + VMCI_RESOURCE_TYPE_DOORBELL); + if (!resource) + return VMCI_ERROR_NOT_FOUND; + + entry = container_of(resource, struct dbell_entry, resource); + *priv_flags = entry-priv_flags; + vmci_resource_put(resource); + } else if (handle.context == VMCI_HYPERVISOR_CONTEXT_ID) { + /* +* Hypervisor endpoints for notifications are not +* supported (yet). +*/ + return VMCI_ERROR_INVALID_ARGS; + } else { + *priv_flags = vmci_context_get_priv_flags(handle.context); + } + + return VMCI_SUCCESS; +} + +/* + * Find doorbell entry by bitmap index. + */ +static struct dbell_entry *dbell_index_table_find(u32 idx) +{ + u32 bucket = VMCI_DOORBELL_HASH(idx); +
[PATCH 04/12] VMCI: device driver implementaton.
VMCI driver code implementes both the host and guest personalities of the VMCI driver. Signed-off-by: George Zhang georgezh...@vmware.com Acked-by: Andy king ack...@vmware.com Acked-by: Dmitry Torokhov d...@vmware.com --- drivers/misc/vmw_vmci/vmci_driver.c | 117 +++ drivers/misc/vmw_vmci/vmci_driver.h | 50 +++ 2 files changed, 167 insertions(+), 0 deletions(-) create mode 100644 drivers/misc/vmw_vmci/vmci_driver.c create mode 100644 drivers/misc/vmw_vmci/vmci_driver.h diff --git a/drivers/misc/vmw_vmci/vmci_driver.c b/drivers/misc/vmw_vmci/vmci_driver.c new file mode 100644 index 000..7b3fce2 --- /dev/null +++ b/drivers/misc/vmw_vmci/vmci_driver.c @@ -0,0 +1,117 @@ +/* + * VMware VMCI Driver + * + * Copyright (C) 2012 VMware, Inc. All rights reserved. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License as published by the + * Free Software Foundation version 2 and no later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY + * or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * for more details. + */ + +#include linux/vmw_vmci_defs.h +#include linux/vmw_vmci_api.h +#include linux/atomic.h +#include linux/kernel.h +#include linux/module.h +#include linux/init.h + +#include vmci_driver.h +#include vmci_event.h + +static bool vmci_disable_host; +module_param_named(disable_host, vmci_disable_host, bool, 0); +MODULE_PARM_DESC(disable_host, +Disable driver host personality (default=enabled)); + +static bool vmci_disable_guest; +module_param_named(disable_guest, vmci_disable_guest, bool, 0); +MODULE_PARM_DESC(disable_guest, +Disable driver guest personality (default=enabled)); + +static bool vmci_guest_personality_initialized; +static bool vmci_host_personality_initialized; + +/* + * vmci_get_context_id() - Gets the current context ID. + * + * Returns the current context ID. Note that since this is accessed only + * from code running in the host, this always returns the host context ID. + */ +u32 vmci_get_context_id(void) +{ + if (vmci_guest_code_active()) + return vmci_get_vm_context_id(); + else if (vmci_host_code_active()) + return VMCI_HOST_CONTEXT_ID; + + return VMCI_INVALID_ID; +} +EXPORT_SYMBOL_GPL(vmci_get_context_id); + +static int __init vmci_drv_init(void) +{ + int vmci_err; + int error; + + vmci_err = vmci_event_init(); + if (vmci_err VMCI_SUCCESS) { + pr_err(Failed to initialize VMCIEvent (result=%d)\n, + vmci_err); + return -EINVAL; + } + + if (!vmci_disable_guest) { + error = vmci_guest_init(); + if (error) { + pr_warn(Failed to initialize guest personality (err=%d)\n, + error); + } else { + vmci_guest_personality_initialized = true; + pr_info(Guest personality initialized and is %s\n, + vmci_guest_code_active() ? + active : inactive); + } + } + + if (!vmci_disable_host) { + error = vmci_host_init(); + if (error) { + pr_warn(Unable to initialize host personality (err=%d)\n, + error); + } else { + vmci_host_personality_initialized = true; + pr_info(Initialized host personality\n); + } + } + + if (!vmci_guest_personality_initialized + !vmci_host_personality_initialized) { + vmci_event_exit(); + return -ENODEV; + } + + return 0; +} +module_init(vmci_drv_init); + +static void __exit vmci_drv_exit(void) +{ + if (vmci_guest_personality_initialized) + vmci_guest_exit(); + + if (vmci_host_personality_initialized) + vmci_host_exit(); + + vmci_event_exit(); +} +module_exit(vmci_drv_exit); + +MODULE_AUTHOR(VMware, Inc.); +MODULE_DESCRIPTION(VMware Virtual Machine Communication Interface.); +MODULE_VERSION(1.0.0.0-k); +MODULE_LICENSE(GPL v2); diff --git a/drivers/misc/vmw_vmci/vmci_driver.h b/drivers/misc/vmw_vmci/vmci_driver.h new file mode 100644 index 000..f69156a --- /dev/null +++ b/drivers/misc/vmw_vmci/vmci_driver.h @@ -0,0 +1,50 @@ +/* + * VMware VMCI Driver + * + * Copyright (C) 2012 VMware, Inc. All rights reserved. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License as published by the + * Free Software Foundation version 2 and no later version. + * + * This program is distributed in the hope that
[PATCH 06/12] VMCI: handle array implementation.
VMCI handle code adds support for dynamic arrays that will grow if they need to. Signed-off-by: George Zhang georgezh...@vmware.com Acked-by: Andy king ack...@vmware.com Acked-by: Dmitry Torokhov d...@vmware.com --- drivers/misc/vmw_vmci/vmci_handle_array.c | 142 + drivers/misc/vmw_vmci/vmci_handle_array.h | 52 +++ 2 files changed, 194 insertions(+), 0 deletions(-) create mode 100644 drivers/misc/vmw_vmci/vmci_handle_array.c create mode 100644 drivers/misc/vmw_vmci/vmci_handle_array.h diff --git a/drivers/misc/vmw_vmci/vmci_handle_array.c b/drivers/misc/vmw_vmci/vmci_handle_array.c new file mode 100644 index 000..344973a --- /dev/null +++ b/drivers/misc/vmw_vmci/vmci_handle_array.c @@ -0,0 +1,142 @@ +/* + * VMware VMCI Driver + * + * Copyright (C) 2012 VMware, Inc. All rights reserved. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License as published by the + * Free Software Foundation version 2 and no later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY + * or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * for more details. + */ + +#include linux/slab.h +#include vmci_handle_array.h + +static size_t handle_arr_calc_size(size_t capacity) +{ + return sizeof(struct vmci_handle_arr) + + capacity * sizeof(struct vmci_handle); +} + +struct vmci_handle_arr *vmci_handle_arr_create(size_t capacity) +{ + struct vmci_handle_arr *array; + + if (capacity == 0) + capacity = VMCI_HANDLE_ARRAY_DEFAULT_SIZE; + + array = kmalloc(handle_arr_calc_size(capacity), GFP_ATOMIC); + if (!array) + return NULL; + + array-capacity = capacity; + array-size = 0; + + return array; +} + +void vmci_handle_arr_destroy(struct vmci_handle_arr *array) +{ + kfree(array); +} + +void vmci_handle_arr_append_entry(struct vmci_handle_arr **array_ptr, + struct vmci_handle handle) +{ + struct vmci_handle_arr *array = *array_ptr; + + if (unlikely(array-size = array-capacity)) { + /* reallocate. */ + struct vmci_handle_arr *new_array; + size_t new_capacity = array-capacity * VMCI_ARR_CAP_MULT; + size_t new_size = handle_arr_calc_size(new_capacity); + + new_array = krealloc(array, new_size, GFP_ATOMIC); + if (!new_array) + return; + + new_array-capacity = new_capacity; + *array_ptr = array = new_array; + } + + array-entries[array-size] = handle; + array-size++; +} + +/* + * Handle that was removed, VMCI_INVALID_HANDLE if entry not found. + */ +struct vmci_handle vmci_handle_arr_remove_entry(struct vmci_handle_arr *array, + struct vmci_handle entry_handle) +{ + struct vmci_handle handle = VMCI_INVALID_HANDLE; + size_t i; + + for (i = 0; i array-size; i++) { + if (vmci_handle_is_equal(array-entries[i], entry_handle)) { + handle = array-entries[i]; + array-size--; + array-entries[i] = array-entries[array-size]; + array-entries[array-size] = VMCI_INVALID_HANDLE; + break; + } + } + + return handle; +} + +/* + * Handle that was removed, VMCI_INVALID_HANDLE if array was empty. + */ +struct vmci_handle vmci_handle_arr_remove_tail(struct vmci_handle_arr *array) +{ + struct vmci_handle handle = VMCI_INVALID_HANDLE; + + if (array-size) { + array-size--; + handle = array-entries[array-size]; + array-entries[array-size] = VMCI_INVALID_HANDLE; + } + + return handle; +} + +/* + * Handle at given index, VMCI_INVALID_HANDLE if invalid index. + */ +struct vmci_handle +vmci_handle_arr_get_entry(const struct vmci_handle_arr *array, size_t index) +{ + if (unlikely(index = array-size)) + return VMCI_INVALID_HANDLE; + + return array-entries[index]; +} + +bool vmci_handle_arr_has_entry(const struct vmci_handle_arr *array, + struct vmci_handle entry_handle) +{ + size_t i; + + for (i = 0; i array-size; i++) + if (vmci_handle_is_equal(array-entries[i], entry_handle)) + return true; + + return false; +} + +/* + * NULL if the array is empty. Otherwise, a pointer to the array + * of VMCI handles in the handle array. + */ +struct vmci_handle *vmci_handle_arr_get_handles(struct vmci_handle_arr *array) +{ + if (array-size) + return array-entries; + + return NULL; +} diff --git a/drivers/misc/vmw_vmci/vmci_handle_array.h
[PATCH 08/12] VMCI: resource object implementation.
VMCI resource tracks all used resources within the vmci code. Signed-off-by: George Zhang georgezh...@vmware.com Acked-by: Andy king ack...@vmware.com Acked-by: Dmitry Torokhov d...@vmware.com --- drivers/misc/vmw_vmci/vmci_resource.c | 229 + drivers/misc/vmw_vmci/vmci_resource.h | 59 + 2 files changed, 288 insertions(+), 0 deletions(-) create mode 100644 drivers/misc/vmw_vmci/vmci_resource.c create mode 100644 drivers/misc/vmw_vmci/vmci_resource.h diff --git a/drivers/misc/vmw_vmci/vmci_resource.c b/drivers/misc/vmw_vmci/vmci_resource.c new file mode 100644 index 000..a196f84 --- /dev/null +++ b/drivers/misc/vmw_vmci/vmci_resource.c @@ -0,0 +1,229 @@ +/* + * VMware VMCI Driver + * + * Copyright (C) 2012 VMware, Inc. All rights reserved. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License as published by the + * Free Software Foundation version 2 and no later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY + * or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * for more details. + */ + +#include linux/vmw_vmci_defs.h +#include linux/hash.h +#include linux/types.h +#include linux/rculist.h + +#include vmci_resource.h +#include vmci_driver.h + + +#define VMCI_RESOURCE_HASH_BITS 7 +#define VMCI_RESOURCE_HASH_BUCKETS (1 VMCI_RESOURCE_HASH_BITS) + +struct vmci_hash_table { + spinlock_t lock; + struct hlist_head entries[VMCI_RESOURCE_HASH_BUCKETS]; +}; + +static struct vmci_hash_table vmci_resource_table = { + .lock = __SPIN_LOCK_UNLOCKED(vmci_resource_table.lock), +}; + +static unsigned int vmci_resource_hash(struct vmci_handle handle) +{ + return hash_32(handle.resource, VMCI_RESOURCE_HASH_BITS); +} + +/* + * Gets a resource (if one exists) matching given handle from the hash table. + */ +static struct vmci_resource *vmci_resource_lookup(struct vmci_handle handle, + enum vmci_resource_type type) +{ + struct vmci_resource *r, *resource = NULL; + struct hlist_node *node; + unsigned int idx = vmci_resource_hash(handle); + + rcu_read_lock(); + hlist_for_each_entry_rcu(r, node, +vmci_resource_table.entries[idx], node) { + u32 cid = r-handle.context; + u32 rid = r-handle.resource; + + if (r-type == type + rid == handle.resource + (cid == handle.context || cid == VMCI_INVALID_ID)) { + resource = r; + break; + } + } + rcu_read_unlock(); + + return resource; +} + +/* + * Find an unused resource ID and return it. The first + * VMCI_RESERVED_RESOURCE_ID_MAX are reserved so we start from + * its value + 1. + * Returns VMCI resource id on success, VMCI_INVALID_ID on failure. + */ +static u32 vmci_resource_find_id(u32 context_id, +enum vmci_resource_type resource_type) +{ + static u32 resource_id = VMCI_RESERVED_RESOURCE_ID_MAX + 1; + u32 old_rid = resource_id; + u32 current_rid; + + /* +* Generate a unique resource ID. Keep on trying until we wrap around +* in the RID space. +*/ + do { + struct vmci_handle handle; + + current_rid = resource_id; + resource_id++; + if (unlikely(resource_id == VMCI_INVALID_ID)) { + /* Skip the reserved rids. */ + resource_id = VMCI_RESERVED_RESOURCE_ID_MAX + 1; + } + + handle = vmci_make_handle(context_id, current_rid); + if (!vmci_resource_lookup(handle, resource_type)) + return current_rid; + } while (resource_id != old_rid); + + return VMCI_INVALID_ID; +} + + +int vmci_resource_add(struct vmci_resource *resource, + enum vmci_resource_type resource_type, + struct vmci_handle handle) + +{ + unsigned int idx; + int result; + + spin_lock(vmci_resource_table.lock); + + if (handle.resource == VMCI_INVALID_ID) { + handle.resource = vmci_resource_find_id(handle.context, + resource_type); + if (handle.resource == VMCI_INVALID_ID) { + result = VMCI_ERROR_NO_HANDLE; + goto out; + } + } else if (vmci_resource_lookup(handle, resource_type)) { + result = VMCI_ERROR_ALREADY_EXISTS; + goto out; + } + + resource-handle = handle; + resource-type = resource_type; + INIT_HLIST_NODE(resource-node); + kref_init(resource-kref); +
[PATCH 10/12] VMCI: guest side driver implementation.
VMCI guest side driver code implementation. Signed-off-by: George Zhang georgezh...@vmware.com Acked-by: Andy king ack...@vmware.com Acked-by: Dmitry Torokhov d...@vmware.com --- drivers/misc/vmw_vmci/vmci_guest.c | 759 1 files changed, 759 insertions(+), 0 deletions(-) create mode 100644 drivers/misc/vmw_vmci/vmci_guest.c diff --git a/drivers/misc/vmw_vmci/vmci_guest.c b/drivers/misc/vmw_vmci/vmci_guest.c new file mode 100644 index 000..d302c89 --- /dev/null +++ b/drivers/misc/vmw_vmci/vmci_guest.c @@ -0,0 +1,759 @@ +/* + * VMware VMCI Driver + * + * Copyright (C) 2012 VMware, Inc. All rights reserved. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License as published by the + * Free Software Foundation version 2 and no later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY + * or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * for more details. + */ + +#include linux/vmw_vmci_defs.h +#include linux/vmw_vmci_api.h +#include linux/moduleparam.h +#include linux/interrupt.h +#include linux/highmem.h +#include linux/kernel.h +#include linux/module.h +#include linux/sched.h +#include linux/init.h +#include linux/pci.h +#include linux/smp.h +#include linux/io.h + +#include vmci_datagram.h +#include vmci_doorbell.h +#include vmci_context.h +#include vmci_driver.h +#include vmci_event.h + +#define PCI_VENDOR_ID_VMWARE 0x15AD +#define PCI_DEVICE_ID_VMWARE_VMCI 0x0740 + +#define VMCI_UTIL_NUM_RESOURCES 1 + +static bool vmci_disable_msi; +module_param_named(disable_msi, vmci_disable_msi, bool, 0); +MODULE_PARM_DESC(disable_msi, Disable MSI use in driver - (default=0)); + +static bool vmci_disable_msix; +module_param_named(disable_msix, vmci_disable_msix, bool, 0); +MODULE_PARM_DESC(disable_msix, Disable MSI-X use in driver - (default=0)); + +static u32 ctx_update_sub_id = VMCI_INVALID_ID; +static u32 vm_context_id = VMCI_INVALID_ID; + +struct vmci_guest_device { + struct device *dev; /* PCI device we are attached to */ + void __iomem *iobase; + + unsigned int irq; + unsigned int intr_type; + bool exclusive_vectors; + struct msix_entry msix_entries[VMCI_MAX_INTRS]; + + struct tasklet_struct datagram_tasklet; + struct tasklet_struct bm_tasklet; + + void *data_buffer; + void *notification_bitmap; +}; + +/* vmci_dev singleton device and supporting data*/ +static struct vmci_guest_device *vmci_dev_g; +static DEFINE_SPINLOCK(vmci_dev_spinlock); + +static atomic_t vmci_num_guest_devices = ATOMIC_INIT(0); + +bool vmci_guest_code_active(void) +{ + return atomic_read(vmci_num_guest_devices) != 0; +} + +u32 vmci_get_vm_context_id(void) +{ + if (vm_context_id == VMCI_INVALID_ID) { + u32 result; + struct vmci_datagram get_cid_msg; + get_cid_msg.dst = + vmci_make_handle(VMCI_HYPERVISOR_CONTEXT_ID, +VMCI_GET_CONTEXT_ID); + get_cid_msg.src = VMCI_ANON_SRC_HANDLE; + get_cid_msg.payload_size = 0; + result = vmci_send_datagram(get_cid_msg); + if (result = 0) + vm_context_id = result; + } + return vm_context_id; +} + +/* + * VM to hypervisor call mechanism. We use the standard VMware naming + * convention since shared code is calling this function as well. + */ +int vmci_send_datagram(struct vmci_datagram *dg) +{ + unsigned long flags; + int result; + + /* Check args. */ + if (dg == NULL) + return VMCI_ERROR_INVALID_ARGS; + + /* +* Need to acquire spinlock on the device because the datagram +* data may be spread over multiple pages and the monitor may +* interleave device user rpc calls from multiple +* VCPUs. Acquiring the spinlock precludes that +* possibility. Disabling interrupts to avoid incoming +* datagrams during a rep out and possibly landing up in +* this function. +*/ + spin_lock_irqsave(vmci_dev_spinlock, flags); + + if (vmci_dev_g) { + iowrite8_rep(vmci_dev_g-iobase + VMCI_DATA_OUT_ADDR, +dg, VMCI_DG_SIZE(dg)); + result = ioread32(vmci_dev_g-iobase + VMCI_RESULT_LOW_ADDR); + } else { + result = VMCI_ERROR_UNAVAILABLE; + } + + spin_unlock_irqrestore(vmci_dev_spinlock, flags); + + return result; +} +EXPORT_SYMBOL_GPL(vmci_send_datagram); + +/* + * Gets called with the new context id if updated or resumed. + * Context id. + */ +static void vmci_guest_cid_update(u32 sub_id, + const struct vmci_event_data *event_data, +
[PATCH 11/12] VMCI: host side driver implementation.
VMCI host side driver code implementation. Signed-off-by: George Zhang georgezh...@vmware.com Acked-by: Andy king ack...@vmware.com Acked-by: Dmitry Torokhov d...@vmware.com --- drivers/misc/vmw_vmci/vmci_host.c | 1042 + 1 files changed, 1042 insertions(+), 0 deletions(-) create mode 100644 drivers/misc/vmw_vmci/vmci_host.c diff --git a/drivers/misc/vmw_vmci/vmci_host.c b/drivers/misc/vmw_vmci/vmci_host.c new file mode 100644 index 000..16e7f54 --- /dev/null +++ b/drivers/misc/vmw_vmci/vmci_host.c @@ -0,0 +1,1042 @@ +/* + * VMware VMCI Driver + * + * Copyright (C) 2012 VMware, Inc. All rights reserved. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License as published by the + * Free Software Foundation version 2 and no later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY + * or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * for more details. + */ + +#include linux/vmw_vmci_defs.h +#include linux/vmw_vmci_api.h +#include linux/moduleparam.h +#include linux/miscdevice.h +#include linux/interrupt.h +#include linux/highmem.h +#include linux/atomic.h +#include linux/kernel.h +#include linux/module.h +#include linux/mutex.h +#include linux/sched.h +#include linux/file.h +#include linux/init.h +#include linux/poll.h +#include linux/pci.h +#include linux/smp.h +#include linux/fs.h +#include linux/io.h + +#include vmci_handle_array.h +#include vmci_queue_pair.h +#include vmci_datagram.h +#include vmci_doorbell.h +#include vmci_resource.h +#include vmci_context.h +#include vmci_driver.h +#include vmci_event.h + +#define VMCI_UTIL_NUM_RESOURCES 1 + +enum { + VMCI_NOTIFY_RESOURCE_QUEUE_PAIR = 0, + VMCI_NOTIFY_RESOURCE_DOOR_BELL = 1, +}; + +enum { + VMCI_NOTIFY_RESOURCE_ACTION_NOTIFY = 0, + VMCI_NOTIFY_RESOURCE_ACTION_CREATE = 1, + VMCI_NOTIFY_RESOURCE_ACTION_DESTROY = 2, +}; + +/* + * VMCI driver initialization. This block can also be used to + * pass initial group membership etc. + */ +struct vmci_init_blk { + u32 cid; + u32 flags; +}; + +/* VMCIqueue_pairAllocInfo_VMToVM */ +struct vmci_qp_alloc_info_vmvm { + struct vmci_handle handle; + u32 peer; + u32 flags; + u64 produce_size; + u64 consume_size; + u64 produce_page_file;/* User VA. */ + u64 consume_page_file;/* User VA. */ + u64 produce_page_file_size; /* Size of the file name array. */ + u64 consume_page_file_size; /* Size of the file name array. */ + s32 result; + u32 _pad; +}; + +/* VMCISetNotifyInfo: Used to pass notify flag's address to the host driver. */ +struct vmci_set_notify_info { + u64 notify_uva; + s32 result; + u32 _pad; +}; + +/* + * Per-instance host state + */ +struct vmci_host_dev { + struct vmci_ctx *context; + int user_version; + enum vmci_obj_type ct_type; + struct mutex lock; /* Mutex lock for vmci context access */ +}; + +static struct vmci_ctx *host_context; +static bool vmci_host_device_initialized; +static atomic_t vmci_host_active_users = ATOMIC_INIT(0); + +/* + * Determines whether the VMCI host personality is + * available. Since the core functionality of the host driver is + * always present, all guests could possibly use the host + * personality. However, to minimize the deviation from the + * pre-unified driver state of affairs, we only consider the host + * device active if there is no active guest device or if there + * are VMX'en with active VMCI contexts using the host device. + */ +bool vmci_host_code_active(void) +{ + return vmci_host_device_initialized + (!vmci_guest_code_active() || +atomic_read(vmci_host_active_users) 0); +} + +/* + * Called on open of /dev/vmci. + */ +static int vmci_host_open(struct inode *inode, struct file *filp) +{ + struct vmci_host_dev *vmci_host_dev; + + vmci_host_dev = kzalloc(sizeof(struct vmci_host_dev), GFP_KERNEL); + if (vmci_host_dev == NULL) + return -ENOMEM; + + vmci_host_dev-ct_type = VMCIOBJ_NOT_SET; + mutex_init(vmci_host_dev-lock); + filp-private_data = vmci_host_dev; + + return 0; +} + +/* + * Called on close of /dev/vmci, most often when the process + * exits. + */ +static int vmci_host_close(struct inode *inode, struct file *filp) +{ + struct vmci_host_dev *vmci_host_dev = filp-private_data; + + if (vmci_host_dev-ct_type == VMCIOBJ_CONTEXT) { + vmci_ctx_destroy(vmci_host_dev-context); + vmci_host_dev-context = NULL; + + /* +* The number of active contexts is used to track whether any +* VMX'en are using the host personality. It is incremented when +* a context is created
[PATCH 12/12] VMCI: Some header and config files.
VMCI head config patch Adds all the necessary files to enable building of the VMCI module with the Linux Makefiles and Kconfig systems. Also adds the header files used for building modules against the driver. Signed-off-by: George Zhang georgezh...@vmware.com Acked-by: Andy king ack...@vmware.com Acked-by: Dmitry Torokhov d...@vmware.com --- drivers/misc/Kconfig |1 drivers/misc/Makefile |2 drivers/misc/vmw_vmci/Kconfig | 16 + drivers/misc/vmw_vmci/Makefile |4 include/linux/vmw_vmci_api.h | 82 include/linux/vmw_vmci_defs.h | 880 6 files changed, 985 insertions(+), 0 deletions(-) create mode 100644 drivers/misc/vmw_vmci/Kconfig create mode 100644 drivers/misc/vmw_vmci/Makefile create mode 100644 include/linux/vmw_vmci_api.h create mode 100644 include/linux/vmw_vmci_defs.h diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig index b151b7c..264e647 100644 --- a/drivers/misc/Kconfig +++ b/drivers/misc/Kconfig @@ -507,4 +507,5 @@ source drivers/misc/lis3lv02d/Kconfig source drivers/misc/carma/Kconfig source drivers/misc/altera-stapl/Kconfig source drivers/misc/mei/Kconfig +source drivers/misc/vmw_vmci/Kconfig endmenu diff --git a/drivers/misc/Makefile b/drivers/misc/Makefile index 2129377..5c99726 100644 --- a/drivers/misc/Makefile +++ b/drivers/misc/Makefile @@ -49,3 +49,5 @@ obj-y += carma/ obj-$(CONFIG_USB_SWITCH_FSA9480) += fsa9480.o obj-$(CONFIG_ALTERA_STAPL) +=altera-stapl/ obj-$(CONFIG_INTEL_MEI)+= mei/ +obj-$(CONFIG_MAX8997_MUIC) += max8997-muic.o +obj-$(CONFIG_VMWARE_VMCI) += vmw_vmci/ diff --git a/drivers/misc/vmw_vmci/Kconfig b/drivers/misc/vmw_vmci/Kconfig new file mode 100644 index 000..55015e7 --- /dev/null +++ b/drivers/misc/vmw_vmci/Kconfig @@ -0,0 +1,16 @@ +# +# VMware VMCI device +# + +config VMWARE_VMCI + tristate VMware VMCI Driver + depends on X86 + help + This is VMware's Virtual Machine Communication Interface. It enables + high-speed communication between host and guest in a virtual + environment via the VMCI virtual device. + + If unsure, say N. + + To compile this driver as a module, choose M here: the + module will be called vmw_vmci. diff --git a/drivers/misc/vmw_vmci/Makefile b/drivers/misc/vmw_vmci/Makefile new file mode 100644 index 000..4da9893 --- /dev/null +++ b/drivers/misc/vmw_vmci/Makefile @@ -0,0 +1,4 @@ +obj-$(CONFIG_VMWARE_VMCI) += vmw_vmci.o +vmw_vmci-y += vmci_context.o vmci_datagram.o vmci_doorbell.o \ + vmci_driver.o vmci_event.o vmci_guest.o vmci_handle_array.o \ + vmci_host.o vmci_queue_pair.o vmci_resource.o vmci_route.o diff --git a/include/linux/vmw_vmci_api.h b/include/linux/vmw_vmci_api.h new file mode 100644 index 000..023430e --- /dev/null +++ b/include/linux/vmw_vmci_api.h @@ -0,0 +1,82 @@ +/* + * VMware VMCI Driver + * + * Copyright (C) 2012 VMware, Inc. All rights reserved. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License as published by the + * Free Software Foundation version 2 and no later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY + * or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * for more details. + */ + +#ifndef __VMW_VMCI_API_H__ +#define __VMW_VMCI_API_H__ + +#include linux/uidgid.h +#include linux/vmw_vmci_defs.h + +#undef VMCI_KERNEL_API_VERSION +#define VMCI_KERNEL_API_VERSION_1 1 +#define VMCI_KERNEL_API_VERSION_2 2 +#define VMCI_KERNEL_API_VERSION VMCI_KERNEL_API_VERSION_2 + +typedef void (vmci_device_shutdown_fn) (void *device_registration, + void *user_data); + +int vmci_datagram_create_handle(u32 resource_id, u32 flags, + vmci_datagram_recv_cb recv_cb, + void *client_data, + struct vmci_handle *out_handle); +int vmci_datagram_create_handle_priv(u32 resource_id, u32 flags, u32 priv_flags, +vmci_datagram_recv_cb recv_cb, +void *client_data, +struct vmci_handle *out_handle); +int vmci_datagram_destroy_handle(struct vmci_handle handle); +int vmci_datagram_send(struct vmci_datagram *msg); +int vmci_doorbell_create(struct vmci_handle *handle, u32 flags, +u32 priv_flags, +vmci_callback notify_cb, void *client_data); +int vmci_doorbell_destroy(struct vmci_handle handle); +int vmci_doorbell_notify(struct vmci_handle handle, u32 priv_flags); +u32 vmci_get_context_id(void); +bool vmci_is_context_owner(u32 context_id, kuid_t uid); + +int vmci_event_subscribe(u32 event, +
[PATCH 0/6] VSOCK for Linux upstreaming
* * * This series of VSOCK linux upstreaming patches include latest udpate from VMware to address Greg's and all other's code review comments. Summary of changes: - Rebase our linux kernel tree from v3.5 to v3.7. - Fix all checkpatch warnings and errors. Fix some checkpatch with -strict errors. This addresses Greg's comment: On 15 Nov 2012 15:47:14 -0800 Re: [PATCH 07/12] VMCI: queue pairs implementation. chckpatch sanity checking. - Fix vmci_get_contextid naming error. - Remove ASSERT/BUG_ON for vmci and vsock source codes. This addresses ddress Greg's comment on On 15 Nov 2012 15:47:14 -0800 Re: [PATCH 02/12] VMCI: datagram implementation. Remove all BUG_ON(), asserts. - use standard Linux kernel ioctl interface. This addresses Greg's comment on On Thu, 15 Nov 2012 16:01:18 -0800 Re: [PATCH 12/12] VMCI: Some header and config files. use linux in-kernel IO macros for VMCI. - Add vmci ioctl code entry in ioctl-number.txt. - Remove printk from header file. Align properly for macro PCI_VENDOR_ID_VMWARE. Remove #define for VMCI_DRIVER_VERSION_STRING. Remove #define for MODULE_NAME. Remove #define for pr_fmt and #define ASSERT(x). Remove inline function VMCI_MAKE_HANDLE and vmci_make_handle which return a structure on stack. use macro instead and use C99 initialization. Remove #define for VMCI_HANDLE_TO_CONTEXT_ID and VMCI_HANDLE_TO_RESOURCE_ID Change macro to inline function for VMCI_HANDLE_EQUAL. Remove u32 typedefs for vmci_id, vmci_event, vmci_privilege_flags. Use C99 style initialization for struct VMCI_INVALID_HANDLE. Use inline function instead of macro for VMCI_HANDLE_INVALID. No change for vmw_vmci_api.h location (under include/linux/ directory). No change for macros VMCI_CONTEXT_IS_VM, VMCI_HOST_CONTEXT_ID, VMCI_RESERVED_CID_LIMIT, VMCI_HYPERVISOR_CONTEXT_ID and VMCI_WELL_KNOWN_CONTEXT_ID. Still in header file include/linux/vmw_vmci_defs.h This addresses Greg's several comments on Thu, 15 Nov 2012 16:01:18 -0800 Re: [PATCH 12/12] VMCI: Some header and config files. * * * In an effort to improve the out-of-the-box experience with Linux kernels for VMware users, VMware is working on readying the Virtual Machine Communication Interface (vmw_vmci) and VMCI Sockets (VSOCK) (vmw_vsock) kernel modules for inclusion in the Linux kernel. The purpose of this post is to acquire feedback on the vmw_vsock kernel module. The vmw_vmci kernel module has been presented in an early post. * * * VMCI Sockets allows virtual machines to communicate with host kernel modules and the VMware hypervisors. VMCI Sockets kernel module has dependency on VMCI kernel module. User level applications both in a virtual machine and on the host can use vmw_vmci through VMCI Sockets API which facilitates fast and efficient communication between guest virtual machines and their host. A socket address family designed to be compatible with UDP and TCP at the interface level. Today, VMCI and VMCI Sockets are used by the VMware shared folders (HGFS) and various VMware Tools components inside the guest for zero-config, network-less access to VMware host services. In addition to this, VMware's users are using VMCI Sockets for various applications, where network access of the virtual machine is restricted or non-existent. Examples of this are VMs communicating with device proxies for proprietary hardware running as host applications and automated testing of applications running within virtual machines. The VMware VMCI Sockets are similar to other socket types, like Berkeley UNIX socket interface. The VMCI sockets module supports both connection-oriented stream sockets like TCP, and connectionless datagram sockets like UDP. The VSOCK protocol family is defined as AF_VSOCK and the socket operations split for SOCK_DGRAM and SOCK_STREAM. For additional information about the use of VMCI and in particular VMCI Sockets, please refer to the VMCI Socket Programming Guide available at https://www.vmware.com/support/developer/vmci-sdk/. --- George Zhang (6): VSOCK: vsock protocol implementation. VSOCK: vsock address implementaion. VSOCK: notification implementation. VSOCK: statistics implementation. VSOCK: utility functions. VSOCK: header and config files. Documentation/ioctl/ioctl-number.txt |1 include/linux/socket.h |4 net/Kconfig |1 net/Makefile |1 net/vmw_vsock/Kconfig| 14 net/vmw_vsock/Makefile |4 net/vmw_vsock/af_vsock.c | 3279 ++
[PATCH 2/6] VSOCK: vsock address implementaion.
VSOCK linux address code implementation. Signed-off-by: George Zhang georgezh...@vmware.com Acked-by: Andy king ack...@vmware.com Acked-by: Dmitry Torokhov d...@vmware.com --- net/vmw_vsock/vsock_addr.c | 116 net/vmw_vsock/vsock_addr.h | 34 + 2 files changed, 150 insertions(+), 0 deletions(-) create mode 100644 net/vmw_vsock/vsock_addr.c create mode 100644 net/vmw_vsock/vsock_addr.h diff --git a/net/vmw_vsock/vsock_addr.c b/net/vmw_vsock/vsock_addr.c new file mode 100644 index 000..9564576 --- /dev/null +++ b/net/vmw_vsock/vsock_addr.c @@ -0,0 +1,116 @@ +/* + * VMware vSockets Driver + * + * Copyright (C) 2007-2012 VMware, Inc. All rights reserved. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License as published by the Free + * Software Foundation version 2 and no later version. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + */ + +#include linux/types.h +#include linux/socket.h +#include linux/stddef.h +#include net/sock.h + +#include vsock_common.h + +void vsock_addr_init(struct sockaddr_vm *addr, u32 cid, u32 port) +{ + memset(addr, 0, sizeof(*addr)); + addr-svm_family = AF_VSOCK; + addr-svm_cid = cid; + addr-svm_port = port; +} + +int vsock_addr_validate(const struct sockaddr_vm *addr) +{ + if (!addr) + return -EFAULT; + + if (addr-svm_family != AF_VSOCK) + return -EAFNOSUPPORT; + + if (addr-svm_zero[0] != 0) + return -EINVAL; + + return 0; +} + +bool vsock_addr_bound(const struct sockaddr_vm *addr) +{ + return addr-svm_port != VMADDR_PORT_ANY; +} + +void vsock_addr_unbind(struct sockaddr_vm *addr) +{ + vsock_addr_init(addr, VMADDR_CID_ANY, VMADDR_PORT_ANY); +} + +bool vsock_addr_equals_addr(const struct sockaddr_vm *addr, + const struct sockaddr_vm *other) +{ + return addr-svm_cid == other-svm_cid + addr-svm_port == other-svm_port; +} + +bool vsock_addr_equals_addr_any(const struct sockaddr_vm *addr, + const struct sockaddr_vm *other) +{ + return (addr-svm_cid == VMADDR_CID_ANY || + other-svm_cid == VMADDR_CID_ANY || + addr-svm_cid == other-svm_cid) + addr-svm_port == other-svm_port; +} + +bool vsock_addr_equals_handle_port(const struct sockaddr_vm *addr, + struct vmci_handle handle, u32 port) +{ + return addr-svm_cid == handle.context addr-svm_port == port; +} + +int vsock_addr_cast(const struct sockaddr *addr, + size_t len, struct sockaddr_vm **out_addr) +{ + if (len sizeof(**out_addr)) + return -EFAULT; + + *out_addr = (struct sockaddr_vm *)addr; + return vsock_addr_validate(*out_addr); +} + +bool vsock_addr_socket_context_stream(u32 cid) +{ + static const u32 non_socket_contexts[] = { + VMCI_HYPERVISOR_CONTEXT_ID, + VMCI_WELL_KNOWN_CONTEXT_ID, + }; + int i; + + BUILD_BUG_ON(sizeof(cid) != sizeof(*non_socket_contexts)); + + for (i = 0; i ARRAY_SIZE(non_socket_contexts); i++) { + if (cid == non_socket_contexts[i]) + return false; + + } + + return true; +} + +bool vsock_addr_socket_context_dgram(u32 cid, u32 rid) +{ + if (cid == VMCI_HYPERVISOR_CONTEXT_ID) { + /* Registrations of PBRPC Servers do not modify VMX/Hypervisor +* state and are allowed. +*/ + return rid == VMCI_UNITY_PBRPC_REGISTER; + } + + return true; +} diff --git a/net/vmw_vsock/vsock_addr.h b/net/vmw_vsock/vsock_addr.h new file mode 100644 index 000..65c4bcf --- /dev/null +++ b/net/vmw_vsock/vsock_addr.h @@ -0,0 +1,34 @@ +/* + * VMware vSockets Driver + * + * Copyright (C) 2007-2012 VMware, Inc. All rights reserved. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License as published by the Free + * Software Foundation version 2 and no later version. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + */ + +#ifndef _VSOCK_ADDR_H_ +#define _VSOCK_ADDR_H_ + +void vsock_addr_init(struct sockaddr_vm *addr, u32 cid, u32 port); +int vsock_addr_validate(const struct sockaddr_vm *addr); +bool vsock_addr_bound(const struct sockaddr_vm *addr); +void vsock_addr_unbind(struct sockaddr_vm *addr); +bool
[PATCH 3/6] VSOCK: notification implementation.
VSOCK control notifications for VMCI Stream Sockets protocol. Signed-off-by: George Zhang georgezh...@vmware.com Acked-by: Andy king ack...@vmware.com Acked-by: Dmitry Torokhov d...@vmware.com --- net/vmw_vsock/notify.c | 675 net/vmw_vsock/notify.h | 124 + 2 files changed, 799 insertions(+), 0 deletions(-) create mode 100644 net/vmw_vsock/notify.c create mode 100644 net/vmw_vsock/notify.h diff --git a/net/vmw_vsock/notify.c b/net/vmw_vsock/notify.c new file mode 100644 index 000..23bad8b --- /dev/null +++ b/net/vmw_vsock/notify.c @@ -0,0 +1,675 @@ +/* + * VMware vSockets Driver + * + * Copyright (C) 2009-2012 VMware, Inc. All rights reserved. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License as published by the Free + * Software Foundation version 2 and no later version. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + */ + +#include linux/types.h +#include linux/socket.h +#include linux/stddef.h +#include net/sock.h + +#include notify.h +#include af_vsock.h + +#define PKT_FIELD(vsk, field_name) ((vsk)-notify.pkt.field_name) + +#define VSOCK_MAX_DGRAM_RESENDS 10 + +static bool vsock_vmci_notify_waiting_write(struct vsock_vmci_sock *vsk) +{ +#if defined(VSOCK_OPTIMIZATION_WAITING_NOTIFY) + bool retval; + u64 notify_limit; + + if (!PKT_FIELD(vsk, peer_waiting_write)) + return false; + +#ifdef VSOCK_OPTIMIZATION_FLOW_CONTROL + /* When the sender blocks, we take that as a sign that the sender is +* faster than the receiver. To reduce the transmit rate of the sender, +* we delay the sending of the read notification by decreasing the +* write_notify_window. The notification is delayed until the number of +* bytes used in the queue drops below the write_notify_window. +*/ + + if (!PKT_FIELD(vsk, peer_waiting_write_detected)) { + PKT_FIELD(vsk, peer_waiting_write_detected) = true; + if (PKT_FIELD(vsk, write_notify_window) PAGE_SIZE) { + PKT_FIELD(vsk, write_notify_window) = + PKT_FIELD(vsk, write_notify_min_window); + } else { + PKT_FIELD(vsk, write_notify_window) -= PAGE_SIZE; + if (PKT_FIELD(vsk, write_notify_window) + PKT_FIELD(vsk, write_notify_min_window)) + PKT_FIELD(vsk, write_notify_window) = + PKT_FIELD(vsk, write_notify_min_window); + + } + } + notify_limit = vsk-consume_size - PKT_FIELD(vsk, write_notify_window); +#else + notify_limit = 0; +#endif + + /* For now we ignore the wait information and just see if the free +* space exceeds the notify limit. Note that improving this function +* to be more intelligent will not require a protocol change and will +* retain compatibility between endpoints with mixed versions of this +* function. +* +* The notify_limit is used to delay notifications in the case where +* flow control is enabled. Below the test is expressed in terms of +* free space in the queue: if free_space ConsumeSize - +* write_notify_window then notify An alternate way of expressing this +* is to rewrite the expression to use the data ready in the receive +* queue: if write_notify_window bufferReady then notify as +* free_space == ConsumeSize - bufferReady. +*/ + retval = vmci_qpair_consume_free_space(vsk-qpair) notify_limit; +#ifdef VSOCK_OPTIMIZATION_FLOW_CONTROL + if (retval) { + /* +* Once we notify the peer, we reset the detected flag so the +* next wait will again cause a decrease in the window size. +*/ + + PKT_FIELD(vsk, peer_waiting_write_detected) = false; + } +#endif + return retval; +#else + return true; +#endif +} + +static bool vsock_vmci_notify_waiting_read(struct vsock_vmci_sock *vsk) +{ +#if defined(VSOCK_OPTIMIZATION_WAITING_NOTIFY) + if (!PKT_FIELD(vsk, peer_waiting_read)) + return false; + + /* For now we ignore the wait information and just see if there is any +* data for our peer to read. Note that improving this function to be +* more intelligent will not require a protocol change and will retain +* compatibility between endpoints with mixed versions of this +* function. +*/ + return vmci_qpair_produce_buf_ready(vsk-qpair) 0; +#else + return true; +#endif +} + +static void
[PATCH 4/6] VSOCK: statistics implementation.
VSOCK stats for VMCI Stream Sockets protocol. Signed-off-by: George Zhang georgezh...@vmware.com Acked-by: Andy king ack...@vmware.com Acked-by: Dmitry Torokhov d...@vmware.com --- net/vmw_vsock/stats.c | 30 ++ net/vmw_vsock/stats.h | 150 + 2 files changed, 180 insertions(+), 0 deletions(-) create mode 100644 net/vmw_vsock/stats.c create mode 100644 net/vmw_vsock/stats.h diff --git a/net/vmw_vsock/stats.c b/net/vmw_vsock/stats.c new file mode 100644 index 000..49e09bb --- /dev/null +++ b/net/vmw_vsock/stats.c @@ -0,0 +1,30 @@ +/* + * VMware vSockets Driver + * + * Copyright (C) 2009-2012 VMware, Inc. All rights reserved. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License as published by the Free + * Software Foundation version 2 and no later version. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + */ + +#include linux/types.h +#include linux/socket.h +#include linux/stddef.h +#include net/sock.h + +#include af_vsock.h +#include stats.h + +#ifdef VSOCK_GATHER_STATISTICS +u64 vsock_stats_ctl_pkt_count[VSOCK_PACKET_TYPE_MAX]; +u64 vsock_stats_consume_queue_hist[VSOCK_NUM_QUEUE_LEVEL_BUCKETS]; +u64 vsock_stats_produce_queue_hist[VSOCK_NUM_QUEUE_LEVEL_BUCKETS]; +atomic64_t vsock_stats_consume_total; +atomic64_t vsock_stats_produce_total; +#endif diff --git a/net/vmw_vsock/stats.h b/net/vmw_vsock/stats.h new file mode 100644 index 000..07e2bf0 --- /dev/null +++ b/net/vmw_vsock/stats.h @@ -0,0 +1,150 @@ +/* + * VMware vSockets Driver + * + * Copyright (C) 2009-2012 VMware, Inc. All rights reserved. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License as published by the Free + * Software Foundation version 2 and no later version. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + */ + +#ifndef __STATS_H__ +#define __STATS_H__ + +#include linux/types.h + +#include vsock_common.h +#include vsock_packet.h + +/* Define VSOCK_GATHER_STATISTICS to turn on statistics gathering. Currently + * this consists of 3 types of stats: 1. The number of control datagram + * messages sent. 2. The level of queuepair fullness (in 10% buckets) whenever + * data is about to be enqueued or dequeued from the queuepair. 3. The total + * number of bytes enqueued/dequeued. + */ + +#ifdef VSOCK_GATHER_STATISTICS + +#define VSOCK_NUM_QUEUE_LEVEL_BUCKETS 10 +extern u64 vsock_stats_ctl_pkt_count[VSOCK_PACKET_TYPE_MAX]; +extern u64 vsock_stats_consume_queue_hist[VSOCK_NUM_QUEUE_LEVEL_BUCKETS]; +extern u64 vsock_stats_produce_queue_hist[VSOCK_NUM_QUEUE_LEVEL_BUCKETS]; +extern atomic64_t vsock_stats_consume_total; +extern atomic64_t vsock_stats_produce_total; + +#define VSOCK_STATS_STREAM_CONSUME_HIST(vsk) \ + vsock_vmci_stats_update_queue_bucket_count((vsk)-qpair,\ + (vsk)-consume_size,\ + vmci_qpair_consume_buf_ready((vsk)-qpair), \ + vsock_stats_consume_queue_hist) +#define VSOCK_STATS_STREAM_PRODUCE_HIST(vsk) \ + vsock_vmci_stats_update_queue_bucket_count((vsk)-qpair,\ + (vsk)-produce_size,\ + vmci_qpair_produce_buf_ready((vsk)-qpair), \ + vsock_stats_produce_queue_hist) +#define VSOCK_STATS_CTLPKT_LOG(pkt_type) \ + (++vsock_stats_ctl_pkt_count[pkt_type]) +#define VSOCK_STATS_STREAM_CONSUME(bytes) \ + atomic64_add(vsock_stats_consume_total, bytes) +#define VSOCK_STATS_STREAM_PRODUCE(bytes) \ + atomic64_add(vsock_stats_produce_total, bytes) +#define VSOCK_STATS_CTLPKT_DUMP_ALL() vsock_vmci_stats_ctl_pkt_dump_all() +#define VSOCK_STATS_HIST_DUMP_ALL() vsock_vmci_stats_hist_dump_all() +#define VSOCK_STATS_TOTALS_DUMP_ALL() vsock_vmci_stats_totals_dump_all() +#define VSOCK_STATS_RESET() vsock_vmci_stats_reset() + +static inline void +vsock_vmci_stats_update_queue_bucket_count(vmci_qpair *qpair, + u64 queue_size, + u64 data_ready, + u64 queue_hist[]) +{ + u64 bucket = 0; + u32 remainder = 0; + + /* We can't do 64 / 64 = 64 bit divides on linux because it requires a +* libgcc which is not linked into the kernel module. Since this
[PATCH 5/6] VSOCK: utility functions.
VSOCK utility functions for Linux VSocket module. Signed-off-by: George Zhang georgezh...@vmware.com Acked-by: Andy king ack...@vmware.com Acked-by: Dmitry Torokhov d...@vmware.com --- net/vmw_vsock/util.c | 345 ++ net/vmw_vsock/util.h | 187 +++ 2 files changed, 532 insertions(+), 0 deletions(-) create mode 100644 net/vmw_vsock/util.c create mode 100644 net/vmw_vsock/util.h diff --git a/net/vmw_vsock/util.c b/net/vmw_vsock/util.c new file mode 100644 index 000..b77cafa --- /dev/null +++ b/net/vmw_vsock/util.c @@ -0,0 +1,345 @@ +/* + * VMware vSockets Driver + * + * Copyright (C) 2007-2012 VMware, Inc. All rights reserved. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License as published by the Free + * Software Foundation version 2 and no later version. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + */ + +#include linux/types.h +#include linux/list.h +#include linux/socket.h +#include linux/stddef.h +#include net/sock.h + +#include af_vsock.h +#include util.h + +struct list_head vsock_bind_table[VSOCK_HASH_SIZE + 1]; +struct list_head vsock_connected_table[VSOCK_HASH_SIZE]; + +DEFINE_SPINLOCK(vsock_table_lock); + +void vsock_vmci_log_pkt(char const *function, u32 line, + struct vsock_packet *pkt) +{ + char buf[256]; + char *cur = buf; + int left = sizeof(buf); + int written = 0; + char *type_strings[] = { + [VSOCK_PACKET_TYPE_INVALID] = INVALID, + [VSOCK_PACKET_TYPE_REQUEST] = REQUEST, + [VSOCK_PACKET_TYPE_NEGOTIATE] = NEGOTIATE, + [VSOCK_PACKET_TYPE_OFFER] = OFFER, + [VSOCK_PACKET_TYPE_ATTACH] = ATTACH, + [VSOCK_PACKET_TYPE_WROTE] = WROTE, + [VSOCK_PACKET_TYPE_READ] = READ, + [VSOCK_PACKET_TYPE_RST] = RST, + [VSOCK_PACKET_TYPE_SHUTDOWN] = SHUTDOWN, + [VSOCK_PACKET_TYPE_WAITING_WRITE] = WAITING_WRITE, + [VSOCK_PACKET_TYPE_WAITING_READ] = WAITING_READ, + [VSOCK_PACKET_TYPE_REQUEST2] = REQUEST2, + [VSOCK_PACKET_TYPE_NEGOTIATE2] = NEGOTIATE2, + }; + + written = snprintf(cur, left, PKT: %u:%u - %u:%u, + pkt-dg.src.context, pkt-src_port, + pkt-dg.dst.context, pkt-dst_port); + if (written = left) + goto error; + + left -= written; + cur += written; + + switch (pkt-type) { + case VSOCK_PACKET_TYPE_REQUEST: + case VSOCK_PACKET_TYPE_NEGOTIATE: + written = snprintf(cur, left, , %s, size = % FMT64 u, + type_strings[pkt-type], pkt-u.size); + break; + + case VSOCK_PACKET_TYPE_OFFER: + case VSOCK_PACKET_TYPE_ATTACH: + written = snprintf(cur, left, , %s, handle = %u:%u, + type_strings[pkt-type], + pkt-u.handle.context, + pkt-u.handle.resource); + break; + + case VSOCK_PACKET_TYPE_WROTE: + case VSOCK_PACKET_TYPE_READ: + case VSOCK_PACKET_TYPE_RST: + written = snprintf(cur, left, , %s, type_strings[pkt-type]); + break; + case VSOCK_PACKET_TYPE_SHUTDOWN: { + bool recv; + bool send; + + recv = pkt-u.mode RCV_SHUTDOWN; + send = pkt-u.mode SEND_SHUTDOWN; + written = snprintf(cur, left, , %s, mode = %c%c, + type_strings[pkt-type], + recv ? 'R' : ' ', send ? 'S' : ' '); + } + break; + + case VSOCK_PACKET_TYPE_WAITING_WRITE: + case VSOCK_PACKET_TYPE_WAITING_READ: + written = snprintf(cur, left, + , %s, generation = % FMT64 u, offset = % FMT64 u, + type_strings[pkt-type], + pkt-u.wait.generation, pkt-u.wait.offset); + + break; + + case VSOCK_PACKET_TYPE_REQUEST2: + case VSOCK_PACKET_TYPE_NEGOTIATE2: + written = snprintf(cur, left, + , %s, size = % FMT64 u, proto = %u, + type_strings[pkt-type], pkt-u.size, + pkt-proto); + break; + + default: + written = snprintf(cur, left, , unrecognized type); + } + + if (written = left) + goto error; + + left -= written; + cur += written; + + written = snprintf(cur, left, [%s:%u]\n,
[PATCH 6/6] VSOCK: header and config files.
VSOCK header files, Makefiles and Kconfig systems for Linux VSocket module. Signed-off-by: George Zhang georgezh...@vmware.com Acked-by: Andy king ack...@vmware.com Acked-by: Dmitry Torokhov d...@vmware.com --- Documentation/ioctl/ioctl-number.txt |1 include/linux/socket.h |4 net/Kconfig |1 net/Makefile |1 net/vmw_vsock/Kconfig| 14 + net/vmw_vsock/Makefile |4 net/vmw_vsock/notify_qstate.c| 408 ++ net/vmw_vsock/vmci_sockets.h | 270 +++ net/vmw_vsock/vmci_sockets_packet.h | 79 +++ net/vmw_vsock/vsock_common.h | 103 + net/vmw_vsock/vsock_packet.h | 92 net/vmw_vsock/vsock_version.h| 22 ++ 12 files changed, 998 insertions(+), 1 deletions(-) create mode 100644 net/vmw_vsock/Kconfig create mode 100644 net/vmw_vsock/Makefile create mode 100644 net/vmw_vsock/notify_qstate.c create mode 100644 net/vmw_vsock/vmci_sockets.h create mode 100644 net/vmw_vsock/vmci_sockets_packet.h create mode 100644 net/vmw_vsock/vsock_common.h create mode 100644 net/vmw_vsock/vsock_packet.h create mode 100644 net/vmw_vsock/vsock_version.h diff --git a/Documentation/ioctl/ioctl-number.txt b/Documentation/ioctl/ioctl-number.txt index 2152b0e..df2b341 100644 --- a/Documentation/ioctl/ioctl-number.txt +++ b/Documentation/ioctl/ioctl-number.txt @@ -70,6 +70,7 @@ Code Seq#(hex) Include FileComments 0x03 all linux/hdreg.h 0x04 D2-DC linux/umsdos_fs.h Dead since 2.6.11, but don't reuse these. 0x06 all linux/lp.h +0x07 9F-D0 linux/vmw_vmci_defs.h mailto:georgezh...@vmware.com 0x09 all linux/raid/md_u.h 0x10 00-0F drivers/char/s390/vmcp.h 0x12 all linux/fs.h diff --git a/include/linux/socket.h b/include/linux/socket.h index 9a546ff..2c57d63 100644 --- a/include/linux/socket.h +++ b/include/linux/socket.h @@ -178,7 +178,8 @@ struct ucred { #define AF_CAIF37 /* CAIF sockets */ #define AF_ALG 38 /* Algorithm sockets*/ #define AF_NFC 39 /* NFC sockets */ -#define AF_MAX 40 /* For now.. */ +#define AF_VSOCK 40 /* VMCI sockets */ +#define AF_MAX 41 /* For now.. */ /* Protocol families, same as address families. */ #define PF_UNSPEC AF_UNSPEC @@ -221,6 +222,7 @@ struct ucred { #define PF_CAIFAF_CAIF #define PF_ALG AF_ALG #define PF_NFC AF_NFC +#define PF_VSOCK AF_VSOCK #define PF_MAX AF_MAX /* Maximum queue length specifiable by listen. */ diff --git a/net/Kconfig b/net/Kconfig index 30b48f5..f143ac3 100644 --- a/net/Kconfig +++ b/net/Kconfig @@ -218,6 +218,7 @@ source net/dcb/Kconfig source net/dns_resolver/Kconfig source net/batman-adv/Kconfig source net/openvswitch/Kconfig +source net/vmw_vsock/Kconfig config RPS boolean diff --git a/net/Makefile b/net/Makefile index 4f4ee08..cae59f4 100644 --- a/net/Makefile +++ b/net/Makefile @@ -70,3 +70,4 @@ obj-$(CONFIG_CEPH_LIB)+= ceph/ obj-$(CONFIG_BATMAN_ADV) += batman-adv/ obj-$(CONFIG_NFC) += nfc/ obj-$(CONFIG_OPENVSWITCH) += openvswitch/ +obj-$(CONFIG_VMWARE_VSOCK) += vmw_vsock/ diff --git a/net/vmw_vsock/Kconfig b/net/vmw_vsock/Kconfig new file mode 100644 index 000..95e2568 --- /dev/null +++ b/net/vmw_vsock/Kconfig @@ -0,0 +1,14 @@ +# +# Vsock protocol +# + +config VMWARE_VSOCK + tristate Virtual Socket protocol + depends on VMWARE_VMCI + help + Virtual Socket Protocol is a socket protocol similar to TCP/IP + allowing comunication between Virtual Machines and VMware + hypervisor. + + To compile this driver as a module, choose M here: the module + will be called vsock. If unsure, say N. diff --git a/net/vmw_vsock/Makefile b/net/vmw_vsock/Makefile new file mode 100644 index 000..4e940fe --- /dev/null +++ b/net/vmw_vsock/Makefile @@ -0,0 +1,4 @@ +obj-$(CONFIG_VMWARE_VSOCK) += vmw_vsock.o + +vmw_vsock-y += af_vsock.o notify.o notify_qstate.o stats.o util.o \ + vsock_addr.o diff --git a/net/vmw_vsock/notify_qstate.c b/net/vmw_vsock/notify_qstate.c new file mode 100644 index 000..1132ae4 --- /dev/null +++ b/net/vmw_vsock/notify_qstate.c @@ -0,0 +1,408 @@ +/* + * VMware vSockets Driver + * + * Copyright (C) 2009-2012 VMware, Inc. All rights reserved. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License as published by the Free + * Software Foundation version 2 and no later version. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A
Re: [PATCH 07/12] VMCI: queue pairs implementation.
On Tue, Jan 08, 2013 at 03:54:54PM -0800, George Zhang wrote: VMCI queue pairs allow for bi-directional ordered communication between host and guests. You should wrap your commit lines at 72 characters, like git asks you to :) +/* Guest device port I/O. */ +struct PPNSet { + u64 num_produce_pages; + u64 num_consume_pages; + u32 *produce_ppns; + u32 *consume_ppns; + bool initialized; +}; I know this is a private structure to the driver, so it's not that big of a deal at all, but the naming for this is a bit odd (mixed case.) Not a show stopper at all, but if you had run checkpatch.pl on it, it would have warned you about this. thanks, greg k-h ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH 00/12] VMCI for Linux upstreaming
On Tue, Jan 08, 2013 at 03:52:33PM -0800, George Zhang wrote: * * * This series of VMCI linux upstreaming patches include latest udpate from VMware to address Greg's and all other's code review comments. Nice work, thanks for the changes you've made over time, and for your persistence. I can't see anything else to complain about, so I've applied this to my tree and it will show up in linux-next tomorrow or so. thanks, greg k-h ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH 0/6] VSOCK for Linux upstreaming
On Tue, Jan 08, 2013 at 03:59:08PM -0800, George Zhang wrote: * * * This series of VSOCK linux upstreaming patches include latest udpate from VMware to address Greg's and all other's code review comments. Dave, you acked these patches a while ago, and now that I've taken the VMCI patches that these depend on through my char-misc tree, can I take these as well? thanks, greg k-h ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [Pv-drivers] [PATCH 07/12] VMCI: queue pairs implementation.
Hi Greg, On Tuesday, January 08, 2013 04:15:39 PM Greg KH wrote: On Tue, Jan 08, 2013 at 03:54:54PM -0800, George Zhang wrote: +/* Guest device port I/O. */ +struct PPNSet { + u64 num_produce_pages; + u64 num_consume_pages; + u32 *produce_ppns; + u32 *consume_ppns; + bool initialized; +}; I know this is a private structure to the driver, so it's not that big of a deal at all, but the naming for this is a bit odd (mixed case.) Not a show stopper at all, but if you had run checkpatch.pl on it, it would have warned you about this. Surprisingly it does not: [dtor@dtor-ws vmci]$ ./scripts/checkpatch.pl -f drivers/misc/vmw_vmci/vmci_queue_pair.h total: 0 errors, 0 warnings, 191 lines checked drivers/misc/vmw_vmci/vmci_queue_pair.h has no obvious style problems and is ready for submission. Also silent on the patch itself... We'll send a followup patch anyway. Thanks, Dmitry ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [Pv-drivers] [PATCH 07/12] VMCI: queue pairs implementation.
On Tue, Jan 08, 2013 at 04:28:51PM -0800, Dmitry Torokhov wrote: Hi Greg, On Tuesday, January 08, 2013 04:15:39 PM Greg KH wrote: On Tue, Jan 08, 2013 at 03:54:54PM -0800, George Zhang wrote: +/* Guest device port I/O. */ +struct PPNSet { + u64 num_produce_pages; + u64 num_consume_pages; + u32 *produce_ppns; + u32 *consume_ppns; + bool initialized; +}; I know this is a private structure to the driver, so it's not that big of a deal at all, but the naming for this is a bit odd (mixed case.) Not a show stopper at all, but if you had run checkpatch.pl on it, it would have warned you about this. Surprisingly it does not: [dtor@dtor-ws vmci]$ ./scripts/checkpatch.pl -f drivers/misc/vmw_vmci/vmci_queue_pair.h total: 0 errors, 0 warnings, 191 lines checked drivers/misc/vmw_vmci/vmci_queue_pair.h has no obvious style problems and is ready for submission. Also silent on the patch itself... You might want to upgrade your version of checkpatch. As of 3.8-rc2, this is what I get: $ ./scripts/checkpatch.pl -f drivers/misc/vmw_vmci/vmci_queue_pair.h WARNING: Avoid CamelCase: PPNSet #28: FILE: misc/vmw_vmci/vmci_queue_pair.h:28: +struct PPNSet { total: 0 errors, 1 warnings, 191 lines checked drivers/misc/vmw_vmci/vmci_queue_pair.h has style problems, please review. If any of these errors are false positives, please report them to the maintainer, see CHECKPATCH in MAINTAINERS. We'll send a followup patch anyway. Great. greg k-h ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH 0/6] VSOCK for Linux upstreaming
From: Greg KH gre...@linuxfoundation.org Date: Tue, 8 Jan 2013 16:21:10 -0800 On Tue, Jan 08, 2013 at 03:59:08PM -0800, George Zhang wrote: * * * This series of VSOCK linux upstreaming patches include latest udpate from VMware to address Greg's and all other's code review comments. Dave, you acked these patches a while ago, Really? I'd like to see where I did that. Instead, what I remember doing was deferring to the feedback these folks received, stating that ideas that the virtio people had mentioned should be considered instead. http://marc.info/?l=linux-netdevm=135301515818462w=2 So definitely NACK this code and any infrastructure you've merged which essentialy depends upon it. ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [Pv-drivers] [PATCH 0/6] VSOCK for Linux upstreaming
On Tuesday, January 08, 2013 05:30:56 PM David Miller wrote: From: Greg KH gre...@linuxfoundation.org Date: Tue, 8 Jan 2013 16:21:10 -0800 On Tue, Jan 08, 2013 at 03:59:08PM -0800, George Zhang wrote: * * * This series of VSOCK linux upstreaming patches include latest udpate from VMware to address Greg's and all other's code review comments. Dave, you acked these patches a while ago, Really? I'd like to see where I did that. Instead, what I remember doing was deferring to the feedback these folks received, stating that ideas that the virtio people had mentioned should be considered instead. http://marc.info/?l=linux-netdevm=135301515818462w=2 I believe Andy replied to Anthony's AF_VMCHANNEL post and the differences between the proposed solutions. So definitely NACK this code and any infrastructure you've merged which essentialy depends upon it. No, there is no infrastructure that depends on VSOCK, as VSOCK is built on top of VMCI, not the other way around. Thanks, Dmitry ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [Pv-drivers] [PATCH 0/6] VSOCK for Linux upstreaming
From: Dmitry Torokhov d...@vmware.com Date: Tue, 08 Jan 2013 17:41:44 -0800 On Tuesday, January 08, 2013 05:30:56 PM David Miller wrote: From: Greg KH gre...@linuxfoundation.org Date: Tue, 8 Jan 2013 16:21:10 -0800 On Tue, Jan 08, 2013 at 03:59:08PM -0800, George Zhang wrote: * * * This series of VSOCK linux upstreaming patches include latest udpate from VMware to address Greg's and all other's code review comments. Dave, you acked these patches a while ago, Really? I'd like to see where I did that. Instead, what I remember doing was deferring to the feedback these folks received, stating that ideas that the virtio people had mentioned should be considered instead. http://marc.info/?l=linux-netdevm=135301515818462w=2 I believe Andy replied to Anthony's AF_VMCHANNEL post and the differences between the proposed solutions. I'd much rather see a hypervisor neutral solution than a hypervisor specific one which this certainly is. ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [Pv-drivers] [PATCH 0/6] VSOCK for Linux upstreaming
On Tue, Jan 08, 2013 at 05:46:01PM -0800, David Miller wrote: From: Dmitry Torokhov d...@vmware.com Date: Tue, 08 Jan 2013 17:41:44 -0800 On Tuesday, January 08, 2013 05:30:56 PM David Miller wrote: From: Greg KH gre...@linuxfoundation.org Date: Tue, 8 Jan 2013 16:21:10 -0800 On Tue, Jan 08, 2013 at 03:59:08PM -0800, George Zhang wrote: * * * This series of VSOCK linux upstreaming patches include latest udpate from VMware to address Greg's and all other's code review comments. Dave, you acked these patches a while ago, Really? I'd like to see where I did that. Instead, what I remember doing was deferring to the feedback these folks received, stating that ideas that the virtio people had mentioned should be considered instead. http://marc.info/?l=linux-netdevm=135301515818462w=2 I believe Andy replied to Anthony's AF_VMCHANNEL post and the differences between the proposed solutions. I'd much rather see a hypervisor neutral solution than a hypervisor specific one which this certainly is. Objectively speaking neither solution is hypervisor neutral as there are hypervisors that implement either VMCI or virtio or something else entirely. Our position is that VSOCK feature set is more complete and that it should be possible to use transports other than VMCI for VSOCK traffic, should interested parties implement them, and on this basis we ask to include VSOCK. Thanks, Dmitry ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH V3 1/2] virtio-net: fix the set affinity bug when CPU IDs are not consecutive
On 01/09/2013 07:31 AM, Rusty Russell wrote: Wanlong Gao gaowanl...@cn.fujitsu.com writes: */ static u16 virtnet_select_queue(struct net_device *dev, struct sk_buff *skb) { -int txq = skb_rx_queue_recorded(skb) ? skb_get_rx_queue(skb) : - smp_processor_id(); +int txq = 0; + +if (skb_rx_queue_recorded(skb)) +txq = skb_get_rx_queue(skb); +else if ((txq = per_cpu(vq_index, smp_processor_id())) == -1) +txq = 0; You should use __get_cpu_var() instead of smp_processor_id() here, ie: else if ((txq = __get_cpu_var(vq_index)) == -1) And AFAICT, no reason to initialize txq to 0 to start with. So: int txq; if (skb_rx_queue_recorded(skb)) txq = skb_get_rx_queue(skb); else { txq = __get_cpu_var(vq_index); if (txq == -1) txq = 0; } Got it, thank you. Now, just to confirm, I assume this can happen even if we use vq_index, right, because of races with virtnet_set_channels? I still can't understand this race, could you explain more? thank you. Regards, Wanlong Gao while (unlikely(txq = dev-real_num_tx_queues)) txq -= dev-real_num_tx_queues; Thanks, Rusty. ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH V3 1/2] virtio-net: fix the set affinity bug when CPU IDs are not consecutive
On 01/08/2013 06:26 PM, Jason Wang wrote: On 01/08/2013 06:07 PM, Wanlong Gao wrote: As Michael mentioned, set affinity and select queue will not work very well when CPU IDs are not consecutive, this can happen with hot unplug. Fix this bug by traversal the online CPUs, and create a per cpu variable to find the mapping from CPU to the preferable virtual-queue. Cc: Rusty Russell ru...@rustcorp.com.au Cc: Michael S. Tsirkin m...@redhat.com Cc: Jason Wang jasow...@redhat.com Cc: Eric Dumazet erdnet...@gmail.com Cc: virtualization@lists.linux-foundation.org Cc: net...@vger.kernel.org Signed-off-by: Wanlong Gao gaowanl...@cn.fujitsu.com --- drivers/net/virtio_net.c | 39 +-- 1 file changed, 29 insertions(+), 10 deletions(-) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index a6fcf15..a77f86c 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -41,6 +41,8 @@ module_param(gso, bool, 0444); #define VIRTNET_SEND_COMMAND_SG_MAX2 #define VIRTNET_DRIVER_VERSION 1.0.0 +DEFINE_PER_CPU(int, vq_index) = -1; + I think this should not be a global one, consider we may have more than one virtio-net cards with different max queues. Yes, would you move this into virtio_info? struct virtnet_stats { struct u64_stats_sync tx_syncp; struct u64_stats_sync rx_syncp; @@ -1016,6 +1018,7 @@ static int virtnet_vlan_rx_kill_vid(struct net_device *dev, u16 vid) static void virtnet_set_affinity(struct virtnet_info *vi, bool set) { int i; +int cpu; /* In multiqueue mode, when the number of cpu is equal to the number of * queue pairs, we let the queue pairs to be private to one cpu by @@ -1029,16 +1032,29 @@ static void virtnet_set_affinity(struct virtnet_info *vi, bool set) return; } -for (i = 0; i vi-max_queue_pairs; i++) { -int cpu = set ? i : -1; -virtqueue_set_affinity(vi-rq[i].vq, cpu); -virtqueue_set_affinity(vi-sq[i].vq, cpu); -} +if (set) { +i = 0; +for_each_online_cpu(cpu) { +virtqueue_set_affinity(vi-rq[i].vq, cpu); +virtqueue_set_affinity(vi-sq[i].vq, cpu); +per_cpu(vq_index, cpu) = i; +i++; +if (i = vi-max_queue_pairs) +break; Can this happen? we check only set when the number are equal. will remove. +} -if (set) vi-affinity_hint_set = true; -else +} else { +for(i = 0; i vi-max_queue_pairs; i++) { +virtqueue_set_affinity(vi-rq[i].vq, -1); +virtqueue_set_affinity(vi-sq[i].vq, -1); +} + +for_each_online_cpu(cpu) +per_cpu(vq_index, cpu) = -1; + This looks suboptimal since it may leads only txq zero is used. So, which value is best for txq when we don't set affinity? just remain to smp_processor_id()? Thanks, Wanlong Gao vi-affinity_hint_set = false; +} } static void virtnet_get_ringparam(struct net_device *dev, @@ -1127,12 +1143,15 @@ static int virtnet_change_mtu(struct net_device *dev, int new_mtu) /* To avoid contending a lock hold by a vcpu who would exit to host, select the * txq based on the processor id. - * TODO: handle cpu hotplug. */ static u16 virtnet_select_queue(struct net_device *dev, struct sk_buff *skb) { -int txq = skb_rx_queue_recorded(skb) ? skb_get_rx_queue(skb) : - smp_processor_id(); +int txq = 0; + +if (skb_rx_queue_recorded(skb)) +txq = skb_get_rx_queue(skb); +else if ((txq = per_cpu(vq_index, smp_processor_id())) == -1) +txq = 0; while (unlikely(txq = dev-real_num_tx_queues)) txq -= dev-real_num_tx_queues; ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
Re: [PATCH V3 1/2] virtio-net: fix the set affinity bug when CPU IDs are not consecutive
On 01/09/2013 09:52 AM, Wanlong Gao wrote: On 01/08/2013 06:26 PM, Jason Wang wrote: On 01/08/2013 06:07 PM, Wanlong Gao wrote: As Michael mentioned, set affinity and select queue will not work very well when CPU IDs are not consecutive, this can happen with hot unplug. Fix this bug by traversal the online CPUs, and create a per cpu variable to find the mapping from CPU to the preferable virtual-queue. Cc: Rusty Russell ru...@rustcorp.com.au Cc: Michael S. Tsirkin m...@redhat.com Cc: Jason Wang jasow...@redhat.com Cc: Eric Dumazet erdnet...@gmail.com Cc: virtualization@lists.linux-foundation.org Cc: net...@vger.kernel.org Signed-off-by: Wanlong Gao gaowanl...@cn.fujitsu.com --- drivers/net/virtio_net.c | 39 +-- 1 file changed, 29 insertions(+), 10 deletions(-) diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c index a6fcf15..a77f86c 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -41,6 +41,8 @@ module_param(gso, bool, 0444); #define VIRTNET_SEND_COMMAND_SG_MAX2 #define VIRTNET_DRIVER_VERSION 1.0.0 +DEFINE_PER_CPU(int, vq_index) = -1; + I think this should not be a global one, consider we may have more than one virtio-net cards with different max queues. Yes, would you move this into virtio_info? Yes, I think it's better. struct virtnet_stats { struct u64_stats_sync tx_syncp; struct u64_stats_sync rx_syncp; @@ -1016,6 +1018,7 @@ static int virtnet_vlan_rx_kill_vid(struct net_device *dev, u16 vid) static void virtnet_set_affinity(struct virtnet_info *vi, bool set) { int i; + int cpu; /* In multiqueue mode, when the number of cpu is equal to the number of * queue pairs, we let the queue pairs to be private to one cpu by @@ -1029,16 +1032,29 @@ static void virtnet_set_affinity(struct virtnet_info *vi, bool set) return; } - for (i = 0; i vi-max_queue_pairs; i++) { - int cpu = set ? i : -1; - virtqueue_set_affinity(vi-rq[i].vq, cpu); - virtqueue_set_affinity(vi-sq[i].vq, cpu); - } + if (set) { + i = 0; + for_each_online_cpu(cpu) { + virtqueue_set_affinity(vi-rq[i].vq, cpu); + virtqueue_set_affinity(vi-sq[i].vq, cpu); + per_cpu(vq_index, cpu) = i; + i++; + if (i = vi-max_queue_pairs) + break; Can this happen? we check only set when the number are equal. will remove. + } - if (set) vi-affinity_hint_set = true; - else + } else { + for(i = 0; i vi-max_queue_pairs; i++) { + virtqueue_set_affinity(vi-rq[i].vq, -1); + virtqueue_set_affinity(vi-sq[i].vq, -1); + } + + for_each_online_cpu(cpu) + per_cpu(vq_index, cpu) = -1; + This looks suboptimal since it may leads only txq zero is used. So, which value is best for txq when we don't set affinity? just remain to smp_processor_id()? The value which will let us use all queues are ok. How about this? i = 0; for_each_online_cpu(cpu) per_cpu(vq_index, cpu) = ++i % vi-curr_queues; Thanks, Wanlong Gao vi-affinity_hint_set = false; + } } static void virtnet_get_ringparam(struct net_device *dev, @@ -1127,12 +1143,15 @@ static int virtnet_change_mtu(struct net_device *dev, int new_mtu) /* To avoid contending a lock hold by a vcpu who would exit to host, select the * txq based on the processor id. - * TODO: handle cpu hotplug. */ static u16 virtnet_select_queue(struct net_device *dev, struct sk_buff *skb) { - int txq = skb_rx_queue_recorded(skb) ? skb_get_rx_queue(skb) : - smp_processor_id(); + int txq = 0; + + if (skb_rx_queue_recorded(skb)) + txq = skb_get_rx_queue(skb); + else if ((txq = per_cpu(vq_index, smp_processor_id())) == -1) + txq = 0; while (unlikely(txq = dev-real_num_tx_queues)) txq -= dev-real_num_tx_queues; ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization
[PATCH] virtio-spec: fix two typos
From: Amos Kong ak...@redhat.com VIRTIO_NET_F_VTRL_VQ - VIRTIO_NET_F_CTRL_VQ VIRTIO_NET_CTRL_MQ is defined to 4 in kernel code Signed-off-by: Amos Kong ak...@redhat.com --- virtio-spec.lyx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/virtio-spec.lyx b/virtio-spec.lyx index 78ec5d0..1ba9992 100644 --- a/virtio-spec.lyx +++ b/virtio-spec.lyx @@ -5156,7 +5156,7 @@ Control Virtqueue \end_layout \begin_layout Standard -The driver uses the control virtqueue (if VIRTIO_NET_F_VTRL_VQ is negotiated) +The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is negotiated) to send commands to manipulate various features of the device which would not easily map into the configuration space. \end_layout @@ -5501,7 +5501,7 @@ struct virtio_net_ctrl_mq { \change_inserted 1986246365 1353594263 -#define VIRTIO_NET_CTRL_MQ1 +#define VIRTIO_NET_CTRL_MQ4 \end_layout \begin_layout Plain Layout -- 1.7.11.7 ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization