Re: [PATCH RFC 0/3] Support standard SRIOV configuration for IB VFs

2015-05-27 Thread Or Gerlitz
On 5/27/2015 12:11 AM, Jason Gunthorpe wrote: On Tue, May 26, 2015 at 04:32:58PM -0400, Doug Ledford wrote: - ifcfg/udev/networkmanager: So what happens when I do ip link add link ib0 name ib0.1 type ipoib And get two IPoIB interfaces with the same GUID? I doubt any sane user

Re: [PATCH RFC 0/3] Support standard SRIOV configuration for IB VFs

2015-05-27 Thread Jason Gunthorpe
On Wed, May 27, 2015 at 01:53:11PM -0400, Doug Ledford wrote: This basic idea is exactly why I think we should stick with the 20 byte LLADDR for ILFA_VF_MAC. It gives a route for the PF to tell the VF what QPN to use for IPoIB (if we ever see HW support to implement that) If we use 8

Re: [PATCH RFC 0/3] Support standard SRIOV configuration for IB VFs

2015-05-27 Thread Doug Ledford
On Wed, 2015-05-27 at 11:11 -0600, Jason Gunthorpe wrote: On Wed, May 27, 2015 at 10:14:06AM -0400, Doug Ledford wrote: Because the QPN is part of the LLADDR IB can create two interfaces on the same physical port that are completely separated by hardware. Read Haggi's email, he explains

Re: [PATCH RFC 0/3] Support standard SRIOV configuration for IB VFs

2015-05-27 Thread Doug Ledford
On Tue, 2015-05-26 at 15:11 -0600, Jason Gunthorpe wrote: On Tue, May 26, 2015 at 04:32:58PM -0400, Doug Ledford wrote: Not so much ethernet world as netdevice world. The iproute2 program is used to configure any and all netdevices, ethernet or otherwise. Right now, we can abuse it to

Re: [PATCH RFC 0/3] Support standard SRIOV configuration for IB VFs

2015-05-27 Thread Jason Gunthorpe
On Wed, May 27, 2015 at 10:14:06AM -0400, Doug Ledford wrote: Because the QPN is part of the LLADDR IB can create two interfaces on the same physical port that are completely separated by hardware. Read Haggi's email, he explains how they plan to use this to create interfaces that can be

Re: [PATCH RFC 0/3] Support standard SRIOV configuration for IB VFs

2015-05-27 Thread Jason Gunthorpe
On Thu, May 28, 2015 at 12:58:59AM +0300, Or Gerlitz wrote: On Wed, May 27, 2015 at 8:53 PM, Doug Ledford dledf...@redhat.com wrote: On Wed, 2015-05-27 at 11:11 -0600, Jason Gunthorpe wrote: Well, let's just be clear: netlink/iproute2 screwed the pooch on their implementation of this

Re: [PATCH RFC 0/3] Support standard SRIOV configuration for IB VFs

2015-05-27 Thread Or Gerlitz
On Wed, May 27, 2015 at 8:53 PM, Doug Ledford dledf...@redhat.com wrote: On Wed, 2015-05-27 at 11:11 -0600, Jason Gunthorpe wrote: Well, let's just be clear: netlink/iproute2 screwed the pooch on their implementation of this stuff. Any solution that doesn't include fixing this up in some way

Re: [PATCH RFC 0/3] Support standard SRIOV configuration for IB VFs

2015-05-26 Thread Jason Gunthorpe
On Tue, May 26, 2015 at 06:33:15AM +0300, Or Gerlitz wrote: This allows generic code to work with the LLADDR - for instance 'ip link set vf mac' should have checked the size of the IFLA_ADDRESS and demanded that the address argument is the same number of bytes. It is very broken the

Re: [PATCH RFC 0/3] Support standard SRIOV configuration for IB VFs

2015-05-26 Thread Jason Gunthorpe
On Tue, May 26, 2015 at 12:53:46PM -0400, Doug Ledford wrote: I perfectly understand what you are trying to do. I care about the design and consistency of netlink - and that means there is one LLADDR definition for a net device, and every single netlink message that touches a LLADDR

Re: [PATCH RFC 0/3] Support standard SRIOV configuration for IB VFs

2015-05-26 Thread Doug Ledford
On Tue, 2015-05-26 at 12:13 -0600, Jason Gunthorpe wrote: On Tue, May 26, 2015 at 12:53:46PM -0400, Doug Ledford wrote: Yes, it is ugly that we are using a IPoIB netdev to configure the RDMA device. If you want to suggest that we use the RDMA Netlink interface to do this, I had thought

Re: [PATCH RFC 0/3] Support standard SRIOV configuration for IB VFs

2015-05-26 Thread Jason Gunthorpe
On Tue, May 26, 2015 at 04:32:58PM -0400, Doug Ledford wrote: Not so much ethernet world as netdevice world. The iproute2 program is used to configure any and all netdevices, ethernet or otherwise. Right now, we can abuse it to do the same here, but it uses the netdevice ndo_ ops, not

Re: [PATCH RFC 0/3] Support standard SRIOV configuration for IB VFs

2015-05-26 Thread Doug Ledford
On Tue, 2015-05-26 at 00:50 +0300, Or Gerlitz wrote: On Tue, May 26, 2015 at 12:14 AM, Jason Gunthorpe jguntho...@obsidianresearch.com wrote: On Mon, May 25, 2015 at 11:04:41PM +0300, Or Gerlitz wrote: OK, so rewinding a bit, the IB VF [1] identity is their 8 bytes port GUID, and as

Re: [PATCH RFC 0/3] Support standard SRIOV configuration for IB VFs

2015-05-26 Thread Doug Ledford
On Mon, 2015-05-25 at 16:32 -0600, Jason Gunthorpe wrote: On Tue, May 26, 2015 at 12:50:52AM +0300, Or Gerlitz wrote: On Tue, May 26, 2015 at 12:14 AM, Jason Gunthorpe jguntho...@obsidianresearch.com wrote: On Mon, May 25, 2015 at 11:04:41PM +0300, Or Gerlitz wrote: OK, so rewinding

Re: [PATCH RFC 0/3] Support standard SRIOV configuration for IB VFs

2015-05-25 Thread Jason Gunthorpe
On Mon, May 25, 2015 at 11:04:41PM +0300, Or Gerlitz wrote: OK, so rewinding a bit, the IB VF [1] identity is their 8 bytes port GUID, and as Jason noted the user/kernel API allows to deliver up to 32 bytes between user and kernel under the set_vf_mac flow (do_setvfinfo() in

Re: [PATCH RFC 0/3] Support standard SRIOV configuration for IB VFs

2015-05-25 Thread Or Gerlitz
On Fri, May 22, 2015 at 12:11 AM, Doug Ledford dledf...@redhat.com wrote: On Thu, 2015-05-21 at 22:55 +0300, Or Gerlitz wrote: On Thu, May 21, 2015 at 7:40 PM, Doug Ledford dledf...@redhat.com wrote: The MAC/GUID mapping isn't the only thing that has to be faked here Exactly nothing is

Re: [PATCH RFC 0/3] Support standard SRIOV configuration for IB VFs

2015-05-25 Thread Or Gerlitz
On Tue, May 26, 2015 at 12:14 AM, Jason Gunthorpe jguntho...@obsidianresearch.com wrote: On Mon, May 25, 2015 at 11:04:41PM +0300, Or Gerlitz wrote: OK, so rewinding a bit, the IB VF [1] identity is their 8 bytes port GUID, and as Jason noted the user/kernel API allows to deliver up to 32

Re: [PATCH RFC 0/3] Support standard SRIOV configuration for IB VFs

2015-05-25 Thread Jason Gunthorpe
On Tue, May 26, 2015 at 12:50:52AM +0300, Or Gerlitz wrote: On Tue, May 26, 2015 at 12:14 AM, Jason Gunthorpe jguntho...@obsidianresearch.com wrote: On Mon, May 25, 2015 at 11:04:41PM +0300, Or Gerlitz wrote: OK, so rewinding a bit, the IB VF [1] identity is their 8 bytes port GUID, and

Re: [PATCH RFC 0/3] Support standard SRIOV configuration for IB VFs

2015-05-25 Thread Or Gerlitz
On Tue, May 26, 2015 at 1:32 AM, Jason Gunthorpe jguntho...@obsidianresearch.com wrote: On Tue, May 26, 2015 at 12:50:52AM +0300, Or Gerlitz wrote: On Tue, May 26, 2015 at 12:14 AM, Jason Gunthorpe jguntho...@obsidianresearch.com wrote: On Mon, May 25, 2015 at 11:04:41PM +0300, Or Gerlitz

Re: [PATCH RFC 0/3] Support standard SRIOV configuration for IB VFs

2015-05-21 Thread Or Gerlitz
On Thu, May 21, 2015 at 7:40 PM, Doug Ledford dledf...@redhat.com wrote: The MAC/GUID mapping isn't the only thing that has to be faked here Exactly nothing is faked here, Virtualization systems such as open-stack provision unique 48 bit mac values to VMs, and it's perfectly legitimate and

Re: [PATCH RFC 0/3] Support standard SRIOV configuration for IB VFs

2015-05-21 Thread Jason Gunthorpe
On Thu, May 21, 2015 at 05:11:48PM -0400, Doug Ledford wrote: fffe in the middle 2 bytes, and then when we call get vf_info, we get a MAC back that removes those 2 bytes and generates an inconsistency between what we *think* our constructed guid should be and what the set guid actually is.

Re: [PATCH RFC 0/3] Support standard SRIOV configuration for IB VFs

2015-05-21 Thread Doug Ledford
On Thu, 2015-05-21 at 22:55 +0300, Or Gerlitz wrote: On Thu, May 21, 2015 at 7:40 PM, Doug Ledford dledf...@redhat.com wrote: The MAC/GUID mapping isn't the only thing that has to be faked here Exactly nothing is faked here, Virtualization systems such as open-stack provision unique 48

[PATCH RFC 0/3] Support standard SRIOV configuration for IB VFs

2015-05-21 Thread Or Gerlitz
Standard configuration of SRIOV VFs through the host is done over the following chain of calls: libvirt -- netlink -- PF netdevice -- where the PF netdevice exports the ndo_set_vf_ calls. When this comes to IB/IPoIB we should normalize this into the verbs framework so we further go: PF IPoIB --

Re: [PATCH RFC 0/3] Support standard SRIOV configuration for IB VFs

2015-05-21 Thread Doug Ledford
On Thu, 2015-05-21 at 19:24 +0300, Or Gerlitz wrote: Standard configuration of SRIOV VFs through the host is done over the following chain of calls: libvirt -- netlink -- PF netdevice -- where the PF netdevice exports the ndo_set_vf_ calls. When this comes to IB/IPoIB we should normalize