Re: [PATCH 10/11] IB: only keep a single key in struct ib_mr

2016-01-06 Thread Jason Gunthorpe
On Wed, Jan 06, 2016 at 05:10:06PM +0200, Sagi Grimberg wrote: > > >>>ULPs are *already* using the same registrations for both local and > >>>remote access. > >> > >>Where? Out of tree? > > > >I haven't found anything in-tree for sure. > > We have that in iSER. > > iSCSI allows a FirstBurst func

Re: [PATCH 10/11] IB: only keep a single key in struct ib_mr

2016-01-05 Thread Jason Gunthorpe
On Fri, Dec 25, 2015 at 06:46:07PM +0200, Liran Liss wrote: > > From: Jason Gunthorpe > > > > >fill mr->key by the lkey or rkey based on that and everything will > > > >work fine. > > > > > > But the ULP *can* register a memory buffer with

Re: [PATCH for-next 2/2] IB/core: Use hop-limit from IP stack for RoCE

2016-01-03 Thread Jason Gunthorpe
On Sun, Jan 03, 2016 at 03:59:11PM +0200, Matan Barak wrote: > @@ -434,6 +434,7 @@ int ib_init_ah_from_wc(struct ib_device *device, u8 > port_num, > int ret; > enum rdma_network_type net_type = RDMA_NETWORK_IB; > enum ib_gid_type gid_type = IB_GID_TYPE_IB; > + int hoplimit =

Re: [PATCH 10/11] IB: only keep a single key in struct ib_mr

2015-12-22 Thread Jason Gunthorpe
On Tue, Dec 22, 2015 at 06:58:14PM +0200, Sagi Grimberg wrote: > > >The ULP decides if this MR is going to be used as a lkey or rkey > >by passing IB_REG_LKEY or IB_REG_RKEY. The HCA driver will then > >fill mr->key by the lkey or rkey based on that and everything will > >work fine. > > But the

Re: [PATCH libibverbs 0/3] Add cross-channel support

2015-12-20 Thread Jason Gunthorpe
On Sun, Dec 20, 2015 at 01:22:41PM +0200, Leon Romanovsky wrote: > From: Leon Romanovsky > > This patchset adds supplementary part of cross-channel support [1] > to libibverbs. All of this needs some kind of man page update And considering this is a non-standard feature it had better be a reall

Re: [PATCH rdma-next 1/6] IB/core: Save the device attributes on the device structure

2015-12-18 Thread Jason Gunthorpe
On Fri, Dec 18, 2015 at 10:08:41AM +0200, Or Gerlitz wrote: > On 12/17/2015 7:41 PM, Jason Gunthorpe wrote: > >On Thu, Dec 17, 2015 at 03:44:19PM +0200, Sagi Grimberg wrote: > >>>+ ret = ib_query_device(device, &device->attrs); > >>>+ if (ret) { >

Re: [PATCH rdma-next 1/6] IB/core: Save the device attributes on the device structure

2015-12-17 Thread Jason Gunthorpe
On Thu, Dec 17, 2015 at 03:44:19PM +0200, Sagi Grimberg wrote: > > >+ret = ib_query_device(device, &device->attrs); > >+if (ret) { > >+printk(KERN_WARNING "Couldn't query the device attributes\n"); > >+goto out; > >+} > >+ > > I thought we're all for removing t

Re: [PATCH for-next V2 00/11] Add RoCE v2 support

2015-12-16 Thread Jason Gunthorpe
On Wed, Dec 16, 2015 at 03:39:16PM -0500, Doug Ledford wrote: > These patches add the concept of duplicate GIDs that are differentiated > by their RoCE version (also called network type). and by vlan, and smac, and ... Basically everything network unique about a namespace has to be encapsulted in

Re: [PATCH] svc_rdma: use local_dma_lkey

2015-12-16 Thread Jason Gunthorpe
On Wed, Dec 16, 2015 at 04:11:04PM +0100, Christoph Hellwig wrote: > We now alwasy have a per-PD local_dma_lkey available. Make use of that > fact in svc_rdma and stop registering our own MR. > > Signed-off-by: Christoph Hellwig Reviewed-by: Jason Gunthorpe > +++ b/net/

Re: [PATCH RESEND] infiniband:core:Add needed error path in cm_init_av_by_path

2015-12-16 Thread Jason Gunthorpe
On Wed, Dec 16, 2015 at 11:26:39AM +0100, Michael Wang wrote: > > On 12/15/2015 06:30 PM, Jason Gunthorpe wrote: > > On Tue, Dec 15, 2015 at 05:38:34PM +0100, Michael Wang wrote: > >> The hop_limit is only suggest that the package allowed to be > >> routed, not hav

Re: [PATCH for-next V2 00/11] Add RoCE v2 support

2015-12-16 Thread Jason Gunthorpe
On Wed, Dec 16, 2015 at 08:56:01AM +0200, Moni Shoua wrote: > I can't object to that but I really would like to get an example of a > security risk. How can anyone give you an example when nobody knows exactly how mlx hardware works in this area? >From an kapi prespective, the security design is

Re: [PATCH for-next V2 00/11] Add RoCE v2 support

2015-12-16 Thread Jason Gunthorpe
On Wed, Dec 16, 2015 at 09:57:02AM +, Liran Liss wrote: > Currently, namespaces are not supported for RoCE. IMHO, we should not be accepting rocev2 without at least basic namespace support too, since it is fairly trivial to do based on the work that is already done for verbs. An obvious missin

Re: [PATCH for-next V2 00/11] Add RoCE v2 support

2015-12-15 Thread Jason Gunthorpe
On Tue, Dec 15, 2015 at 04:45:21PM -0500, Doug Ledford wrote: > In particular, Liran piped up with this comment: > > "Also, I don't want to do any route resolution on the Rx path. A UD QP > completion just reports the details of the packet it received. > > Conceptually, an incoming packet may no

Re: [PATCH 3/3] IB core: Display 64 bit counters from the extended set

2015-12-15 Thread Jason Gunthorpe
On Tue, Dec 15, 2015 at 03:01:03PM -0500, Hal Rosenstock wrote: > > I would agree, from my observation, that the two main byte counters > > are always available. > > The extended packet counts work but I thought there was a PMA with one > of the extended byte counts wired to 0. Can't remember >

Re: [PATCH 3/3] IB core: Display 64 bit counters from the extended set

2015-12-15 Thread Jason Gunthorpe
On Tue, Dec 15, 2015 at 01:51:35PM -0600, Christoph Lameter wrote: > On Mon, 14 Dec 2015, Hal Rosenstock wrote: > > > > Mellanox should really confirm this for their hardware matrix. > > > > I am trying to get definitive answer to this. > > I was told today on a conf call with a couple of Mellano

Re: [PATCH RESEND] infiniband:core:Add needed error path in cm_init_av_by_path

2015-12-15 Thread Jason Gunthorpe
On Tue, Dec 15, 2015 at 05:38:34PM +0100, Michael Wang wrote: > The hop_limit is only suggest that the package allowed to be > routed, not have to, correct? If the hop limit is >= 2 (?) then the GRH is mandatory. The SM will return this information in the PathRecord if it determines a GRH is requi

Re: [PATCH 3/3] IB core: Display 64 bit counters from the extended set

2015-12-11 Thread Jason Gunthorpe
On Fri, Dec 11, 2015 at 07:23:13PM -0500, ira.weiny wrote: > On Fri, Dec 11, 2015 at 05:00:47PM -0700, Jason Gunthorpe wrote: > > > > FWIW, I also hate the sysfs counters that reflect the PMA, these would > > be much better are free running, wrapping, non-resetting counters

Re: [PATCH 3/3] IB core: Display 64 bit counters from the extended set

2015-12-11 Thread Jason Gunthorpe
> qib, mlx4 are fine. mlx5 should be as well I would think (I don't have that > hardware.) I have no specifics to add, but I keep running into systems, even today, where the 64 bit counters don't work. The MAD might be there, but several counters are wired to 0. Not sure exactly which HW though.

Re: device attr cleanup (was: Handle mlx4 max_sge_rd correctly)

2015-12-10 Thread Jason Gunthorpe
On Thu, Dec 10, 2015 at 11:56:50PM -0500, Doug Ledford wrote: > Looking at struct netdevice, it has the sort of organization I would > call reasonable. Things like struct tx_stats is a struct, even though > it's embedded in the parent struct and not a pointer and there is > exactly and only one c

Re: [PATCH 02/37] IB/rdmavt: Consolidate dma ops in rdmavt.

2015-12-10 Thread Jason Gunthorpe
On Thu, Dec 10, 2015 at 04:46:24PM -0800, Christoph Hellwig wrote: > On Thu, Dec 10, 2015 at 05:29:50PM -0700, Jason Gunthorpe wrote: > > Hrm.. sizeof(void *) > sizeof(dma_addr_t) seemed pretty obscure to me, > > here is the original discussion: > > > > https:

Re: [PATCH 02/37] IB/rdmavt: Consolidate dma ops in rdmavt.

2015-12-10 Thread Jason Gunthorpe
On Thu, Dec 10, 2015 at 03:41:12PM -0800, Christoph Hellwig wrote: > On Thu, Dec 10, 2015 at 10:44:13AM -0700, Jason Gunthorpe wrote: > > > the signature of the function in struct ib_dma_mapping_ops. > > > > It is supposed to be a dma_addr_t 'cookie' not a u6

Re: device attr cleanup (was: Handle mlx4 max_sge_rd correctly)

2015-12-10 Thread Jason Gunthorpe
On Thu, Dec 10, 2015 at 11:07:37AM -0500, Chuck Lever wrote: > Invasive IB core changes like this clean up are especially > burdensome for me because NFS/RDMA changes do not normally > go through Doug's tree, so it takes extra co-ordination. The ARM folks do this sort of stuff on a regular basis

Re: [PATCH 02/37] IB/rdmavt: Consolidate dma ops in rdmavt.

2015-12-10 Thread Jason Gunthorpe
On Thu, Dec 10, 2015 at 11:17:09AM -0500, Dennis Dalessandro wrote: > On Tue, Dec 08, 2015 at 08:08:21AM +0200, Leon Romanovsky wrote: > >On Mon, Dec 07, 2015 at 03:43:06PM -0500, Dennis Dalessandro wrote: > >>+ > >>+#define BAD_DMA_ADDRESS ((u64)0) Just use DMA_ERROR_CODE. > >>+static u64 rvt_dm

Re: [PATCH for-next V2 05/11] IB/core: Add rdma_network_type to wc

2015-12-10 Thread Jason Gunthorpe
On Thu, Dec 10, 2015 at 09:58:51AM +0200, Moni Shoua wrote: > > creating a horrible API, or requiring all vendors to implement a > > network type flag. > > > Actually you haven't suggested anything yet besides a name to the function. > I already said that calculating gid_index from wc and grh requ

Re: [PATCH 1/1] Ibacm: default pkey for partitioned fabrics

2015-12-09 Thread Jason Gunthorpe
On Wed, Dec 09, 2015 at 05:13:49PM +, Hefty, Sean wrote: > Example: Compute nodes are assigned pkeys 0x8000 and 0x7fff. A node > running the job scheduler has pkeys 0x and 0x8000 (maybe it's > also the backup SA). Ibacm would need to select pkey 0x8000 for > communication. I've also se

Re: [PATCH for-next V2 05/11] IB/core: Add rdma_network_type to wc

2015-12-09 Thread Jason Gunthorpe
On Wed, Dec 09, 2015 at 09:38:21AM +, Liran Liss wrote: > > But, it is not part of our verbs API, and I'd *strongly* encourage other > > vendors and future hardware to simply return the gid index that the > > hardware matched instead of requiring the software to try and guess after > > the fac

Re: [PATCH for-next V2 05/11] IB/core: Add rdma_network_type to wc

2015-12-09 Thread Jason Gunthorpe
On Wed, Dec 09, 2015 at 11:34:10AM +0200, Moni Shoua wrote: > > Eh? I think you've missed the point, there is no net device when > > looking at a wc. > > > > Look, here is a concrete direction: > > > > Replace all the crap in > > ib_init_ah_from_wc/get_sgid_index_from_eth/rdma_addr_find_dmac_by_grh

Re: [RFC contig pages support 1/2] IB: Supports contiguous memory operations

2015-12-09 Thread Jason Gunthorpe
On Wed, Dec 09, 2015 at 10:00:02AM +, Shachar Raindel wrote: > > Yes please. > Note that other HW vendors are developing similar solutions, see for > example: > http://www.slideshare.net/linaroorg/hkg15106-replacing-cmem-meeting-tis-soc-shared-buffer-allocation-management-and-address-translati

Re: [PATCH 1/1] Ibacm: default pkey for partitioned fabrics

2015-12-09 Thread Jason Gunthorpe
On Wed, Dec 09, 2015 at 01:07:14PM +, Wan, Kaike wrote: > > > + /* Determine the default pkey index for SA access first. > > > + * Order of preference: 0x, 0x7fff, first pkey. > > > > No, IBA says that only the default pkey should be used to talk to the SA, > > every port needs 0x7FFF

Re: [PATCH 1/1] Ibacm: default pkey for partitioned fabrics

2015-12-09 Thread Jason Gunthorpe
On Wed, Dec 09, 2015 at 07:51:46AM -0500, Hal Rosenstock wrote: > > ipoib always uses the 0 pkey index to create the default ipoib > > interface. (see eg, update_parent_pkey) > > This is beyond IBA spec and is currently a linux convention for IPoIB. > IMO it should be changed to search for this pk

Re: [PATCH 05/10] IB/qib: Use rdmavt lid defines in qib

2015-12-08 Thread Jason Gunthorpe
On Tue, Dec 08, 2015 at 06:19:41PM -0500, Dennis Dalessandro wrote: > So what is the consensus here? Should we leave it alone for now and > potentially go back and deal with this separately? Just define the new one > as LE and use it, even though it doesn't match the rest? Something else > entir

Re: device attr cleanup (was: Handle mlx4 max_sge_rd correctly)

2015-12-08 Thread Jason Gunthorpe
On Tue, Dec 08, 2015 at 03:02:44PM -0800, Christoph Hellwig wrote: > On Tue, Dec 08, 2015 at 03:59:40PM -0700, Jason Gunthorpe wrote: > > Or, can we please stop this bikeshedding. Christoph's patch is done, > > well tested and does a lot more clean up that this hacky three lin

Re: device attr cleanup (was: Handle mlx4 max_sge_rd correctly)

2015-12-08 Thread Jason Gunthorpe
On Wed, Dec 09, 2015 at 12:47:55AM +0200, Or Gerlitz wrote: > On Wed, Dec 9, 2015 at 12:04 AM, Doug Ledford wrote: > > Makes sense. > > thanks. > > > Show me what you are talking about (either a link to Ira's > > patch you are referring to or your own patch). > > The patch is three liner to add

Re: [PATCH for-next V2 05/11] IB/core: Add rdma_network_type to wc

2015-12-08 Thread Jason Gunthorpe
On Tue, Dec 08, 2015 at 09:23:18AM +0200, Moni Shoua wrote: > >> Now, when same gid value can be present in multiple entries you need > >> an extra parameter to get distinguish between them - that would be > >> the netwrok_type > > > > Eh? You are only worried about duplicates between rocev1 mode

Re: [PATCH 1/1] Ibacm: default pkey for partitioned fabrics

2015-12-08 Thread Jason Gunthorpe
On Tue, Dec 08, 2015 at 12:33:02PM -0500, kaike@intel.com wrote: > From: Kaike Wan > > In an insecure IB fabric, the default pkey in a port is 0x, where each > node is allowed to talk to any other node in the fabric, including the SA > node. However, in a secure fabric, to limit member ac

Re: [PATCH 05/10] IB/qib: Use rdmavt lid defines in qib

2015-12-08 Thread Jason Gunthorpe
On Tue, Dec 08, 2015 at 02:24:55PM -0500, ira.weiny wrote: > On Mon, Dec 07, 2015 at 01:54:39PM -0700, Jason Gunthorpe wrote: > > On Mon, Dec 07, 2015 at 03:49:12PM -0500, Dennis Dalessandro wrote: > > > /* A multicast address requires a GRH (see ch. 8.4.1). */ > &g

Re: [PATCH for-next 1/2] net/mlx5_core: Configure HW to support atomic request in host endianness

2015-12-08 Thread Jason Gunthorpe
On Tue, Dec 08, 2015 at 08:08:02PM +0200, eran ben elisha wrote: > On Tue, Dec 8, 2015 at 7:17 PM, Jason Gunthorpe > wrote: > > On Tue, Dec 08, 2015 at 06:54:15PM +0200, Eran Ben Elisha wrote: > >> HW is capable of 2 requestor endianness modes for standard 8 Bytes > >

Re: [PATCH for-next 1/2] net/mlx5_core: Configure HW to support atomic request in host endianness

2015-12-08 Thread Jason Gunthorpe
On Tue, Dec 08, 2015 at 06:54:15PM +0200, Eran Ben Elisha wrote: > HW is capable of 2 requestor endianness modes for standard 8 Bytes > atomic: BE (0x0) and host endianness (0x1). Read the supported modes > from hca atomic capabilities and configure HW to host endianness mode if > supported. Uh, i

Re: [RFC contig pages support 1/2] IB: Supports contiguous memory operations

2015-12-08 Thread Jason Gunthorpe
On Tue, Dec 08, 2015 at 07:18:52AM -0800, Christoph Hellwig wrote: > There is absolutely nothing IB specific here. If you want to support > anonymous mmaps to allocate large contiguous pages work with the MM > folks on providing that in a generic fashion. Yes please. We already have huge page mm

Re: [PATCH 06/37] IB/rdmavt: Add query and modify device stubs

2015-12-07 Thread Jason Gunthorpe
On Mon, Dec 07, 2015 at 09:26:11PM +, Hefty, Sean wrote: > > +static int rvt_query_device(struct ib_device *ibdev, > > + struct ib_device_attr *props, > > + struct ib_udata *uhw) > > +{ > > + /* > > +* Return rvt_dev_info.props contents > > +

Re: [PATCH 05/10] IB/qib: Use rdmavt lid defines in qib

2015-12-07 Thread Jason Gunthorpe
On Mon, Dec 07, 2015 at 03:49:12PM -0500, Dennis Dalessandro wrote: > /* A multicast address requires a GRH (see ch. 8.4.1). */ > - if (ah_attr->dlid >= QIB_MULTICAST_LID_BASE && > - ah_attr->dlid != QIB_PERMISSIVE_LID && > + if (ah_attr->dlid >= be16_to_cpu(IB_MULTICAST_LID_B

Re: [PATCH for-next V2 05/11] IB/core: Add rdma_network_type to wc

2015-12-07 Thread Jason Gunthorpe
On Mon, Dec 07, 2015 at 08:34:43PM +0200, Moni Shoua wrote: > Well, just tell me how you want to discover gid_index when you poll > the WC out of the CQ. Hey, I'm not desiging this rocev2 stuff, this is something the rocev2 community needs to sort out. > Like I said, the sgid itself is in the GRH

Re: [PATCH for-next V2 05/11] IB/core: Add rdma_network_type to wc

2015-12-07 Thread Jason Gunthorpe
On Mon, Dec 07, 2015 at 08:37:41AM +0200, Moni Shoua wrote: > On Mon, Dec 7, 2015 at 8:34 AM, Jason Gunthorpe > wrote: > > On Mon, Dec 07, 2015 at 08:15:40AM +0200, Moni Shoua wrote: > > > >> What you have though is the sgid taken from the GRH that is scattered >

Re: [PATCH for-next V2 05/11] IB/core: Add rdma_network_type to wc

2015-12-06 Thread Jason Gunthorpe
On Mon, Dec 07, 2015 at 08:15:40AM +0200, Moni Shoua wrote: > What you have though is the sgid taken from the GRH that is scattered > to the first 40/20 bytes of the receive WQE. This is not enough to > determine the network type. It is enough to discover the sgid index which will tell you the t

Re: [PATCH for-next V2 05/11] IB/core: Add rdma_network_type to wc

2015-12-06 Thread Jason Gunthorpe
On Thu, Dec 03, 2015 at 04:20:50PM +, Liran Liss wrote: > > From: linux-rdma-ow...@vger.kernel.org [mailto:linux-rdma- > > > Subject: Re: [PATCH for-next V2 05/11] IB/core: Add rdma_network_type to > > wc > > > > Bloating the WC with a field that's not really useful for the ULPs seems > > pr

Re: [PATCH for-next V2 05/11] IB/core: Add rdma_network_type to wc

2015-12-03 Thread Jason Gunthorpe
On Thu, Dec 03, 2015 at 03:47:12PM +0200, Matan Barak wrote: > From: Somnath Kotur > > Providers should tell IB core the wc's network type. > This is used in order to search for the proper GID in the > GID table. When using HCAs that can't provide this info, > IB core tries to deep examine the pa

Re: stalled again

2015-12-02 Thread Jason Gunthorpe
On Wed, Dec 02, 2015 at 04:29:27AM -0800, Christoph Hellwig wrote: > Maybe we need more co-maintainers if the two current maintainers can't > handle the workload? Doug is the only maintainer. The idea of co-maintainers was rejected by some parties. Don't be confused, the other 'M' people in the M

Re: [PATCH v2 04/17] staging/rdma/hfi1: Fix qp.h comments

2015-12-01 Thread Jason Gunthorpe
On Tue, Dec 01, 2015 at 03:38:13PM -0500, Jubin John wrote: > From: Kaike Wan > > This patch fixes a few incorrect header file comments in qp.h FWIW, within drivers/infiniband we've been moving these comments to the implementation, not the function prototype. Jason -- To unsubscribe from this l

Re: Future of FMR support, was: Re: [PATCH v1 5/9] xprtrdma: Add ro_unmap_sync method for FMR

2015-11-25 Thread Jason Gunthorpe
On Wed, Nov 25, 2015 at 09:09:20AM -0800, santosh shilimkar wrote: > >I'd say drop the current iWarp transport if it's not testable. The > >only real difference between IB and iWarp is the needed to create > >a MR for the RDMA READ sink, and we're much better of adding that into > >the current IB

Re: [PATCH for-next V1 9/9] IB/cma: Join and leave multicast groups with IGMP

2015-11-25 Thread Jason Gunthorpe
On Wed, Nov 25, 2015 at 10:31:15AM +0200, Moni Shoua wrote: > On Tue, Nov 24, 2015 at 8:15 PM, Jason Gunthorpe > wrote: > > On Tue, Nov 24, 2015 at 11:41:10AM +0200, Moni Shoua wrote: > >> On Mon, Nov 23, 2015 at 11:25 PM, Jason Gunthorpe > >> wrote: > >&g

Re: [PATCH for-next V1 5/9] IB/core: Add rdma_network_type to wc

2015-11-25 Thread Jason Gunthorpe
On Wed, Nov 25, 2015 at 04:18:25PM +0200, Matan Barak wrote: > On Wed, Nov 25, 2015 at 8:55 AM, Jason Gunthorpe > wrote: > > On Tue, Nov 24, 2015 at 09:07:41PM +0200, Matan Barak wrote: > > > >> IMHO, the user is entitles to choose any valid sgid_index for the > >

Re: [PATCH for-next V1 5/9] IB/core: Add rdma_network_type to wc

2015-11-24 Thread Jason Gunthorpe
On Tue, Nov 24, 2015 at 09:07:41PM +0200, Matan Barak wrote: > IMHO, the user is entitles to choose any valid sgid_index for the > interface. Anything he chooses guaranteed to be valid (from security > perspective) No, the namespace patches will have to limit the sgid_indexes that can be used wit

Re: [PATCH 2/9] IB: add a proper completion queue abstraction

2015-11-24 Thread Jason Gunthorpe
On Tue, Nov 24, 2015 at 06:47:14PM -0800, Caitlin Bestler wrote: > Acknowledge packet for the last packet of the request has been > committed to the wire (including the appropriate fields for RDMA > READ response). > The target side cannot generate a response before it rece

Re: [PATCH] IB/ipoib: CSUM support in connected mode

2015-11-24 Thread Jason Gunthorpe
On Tue, Nov 24, 2015 at 10:45:14AM +0100, Yann Droneaud wrote: > > Same as my question above, if peer supports this feature do you see > > anything wrong? > > If peer is going to forward this packet to a different network, which > is not IPoIB based, the checksum will be checked and the packet wi

Re: [PATCH v1 3/9] xprtrdma: Introduce ro_unmap_sync method

2015-11-24 Thread Jason Gunthorpe
On Tue, Nov 24, 2015 at 09:39:33AM -0500, Chuck Lever wrote: > >> Do we really want to go down that road? It seems like we've decided > >> in general that while the protocol specs say MR must be unmapped before > >> proceeding with the data that is painful enough to ignore this > >> requirement.

Re: [PATCH for-next 03/10] IB/iser: Don't register memory for all immediatedata writes

2015-11-24 Thread Jason Gunthorpe
On Tue, Nov 24, 2015 at 12:04:36PM +0200, Sagi Grimberg wrote: > Jason and Or, > > > > >I'm with Or on this, this is really goofy looking. > > > >This routine probably should not be setting the rkey at all, it makes > >no sense to have a routine that returns a lkey and a rkey. Those are > >always

Re: [PATCH for-next V1 9/9] IB/cma: Join and leave multicast groups with IGMP

2015-11-24 Thread Jason Gunthorpe
On Tue, Nov 24, 2015 at 11:41:10AM +0200, Moni Shoua wrote: > On Mon, Nov 23, 2015 at 11:25 PM, Jason Gunthorpe > wrote: > > On Thu, Oct 15, 2015 at 07:07:12PM +0300, Matan Barak wrote: > >> diff --git a/include/rdma/ib_sa.h b/include/rdma/ib_sa.h > >> index 0a40

Re: [PATCH for-next V1 5/9] IB/core: Add rdma_network_type to wc

2015-11-24 Thread Jason Gunthorpe
On Tue, Nov 24, 2015 at 03:47:51PM +0200, Matan Barak wrote: > > It isn't just the hop limit that has to come from the route entry, all > > the source information of the path comes from there. Ie the gid table > > should accept the route entry directly and spit out the sgid_index. > > > > The respo

Re: [PATCH for-next V1 3/9] IB/core: Add gid attributes to sysfs

2015-11-24 Thread Jason Gunthorpe
On Tue, Nov 24, 2015 at 03:49:28PM +0200, Matan Barak wrote: > Of course that if there is no such documentation, I can add a new file > for the sysfs ABI defined in this patch. That is probably needed, our old stuff predates this new process. Jason -- To unsubscribe from this list: send the line

Re: [PATCH 2/9] IB: add a proper completion queue abstraction

2015-11-24 Thread Jason Gunthorpe
On Tue, Nov 24, 2015 at 10:08:39AM -0700, c...@asomi.com wrote: >>> My recollection of the IB verbs is that they were unlikely to have >>> overlooked something like that. If it did slip through then there >>> should be an errata. >>verbs reflects the wire protocol and the wire proto

Re: [PATCH v1 3/9] xprtrdma: Introduce ro_unmap_sync method

2015-11-23 Thread Jason Gunthorpe
On Mon, Nov 23, 2015 at 10:45:56PM -0800, Christoph Hellwig wrote: > On Mon, Nov 23, 2015 at 05:14:14PM -0500, Chuck Lever wrote: > > In the current xprtrdma implementation, some memreg strategies > > implement ro_unmap synchronously (the MR is knocked down before the > > method returns) and some a

Re: Future of FMR support, was: Re: [PATCH v1 5/9] xprtrdma: Add ro_unmap_sync method for FMR

2015-11-23 Thread Jason Gunthorpe
On Mon, Nov 23, 2015 at 10:52:26PM -0800, Christoph Hellwig wrote: > > So at lest for 4.5 we're unlikely to be able to get rid of it alone > due to the RDS issue. We'll then need performance numbers for mlx4, > and figure out how much we care about mthca. mthca is unfortunately very popular in t

Re: [PATCH 2/9] IB: add a proper completion queue abstraction

2015-11-23 Thread Jason Gunthorpe
On Mon, Nov 23, 2015 at 06:35:28PM -0800, Caitlin Bestler wrote: > Is it possible for an IB HCA to transmit a response on a QP and not > in that packet or a previous packet acknowledge something that it > has delivered to the user? AFAIK, the rules of ack coalescing do not interact with the send

Re: [PATCH 2/9] IB: add a proper completion queue abstraction

2015-11-23 Thread Jason Gunthorpe
On Mon, Nov 23, 2015 at 07:34:53PM -0500, Tom Talpey wrote: > Been there, seen that. Bluescreened on it, mysteriously. Yes, me too :( Jason -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http:

Re: [PATCH 2/9] IB: add a proper completion queue abstraction

2015-11-23 Thread Jason Gunthorpe
On Mon, Nov 23, 2015 at 03:30:42PM -0800, Caitlin Bestler wrote: >The receive completion can be safely assumed to indicate transmit >completion over a reliable connection unless your peer has gone >completely bonkers and is replying to a command that it did not >receive. Perhaps iW

Re: [PATCH 2/9] IB: add a proper completion queue abstraction

2015-11-23 Thread Jason Gunthorpe
On Mon, Nov 23, 2015 at 02:33:05PM -0800, Bart Van Assche wrote: > On 11/23/2015 02:18 PM, Jason Gunthorpe wrote: > >On Mon, Nov 23, 2015 at 01:54:05PM -0800, Bart Van Assche wrote: > >What I don't see is how SRP handles things when the > >sendq fills up, ie the

Re: [PATCH 2/9] IB: add a proper completion queue abstraction

2015-11-23 Thread Jason Gunthorpe
On Mon, Nov 23, 2015 at 01:54:05PM -0800, Bart Van Assche wrote: > Not really ... Please have a look at the SRP initiator source code. What the > SRP initiator does is to poll the send queue before sending a new > SCSI I see that. What I don't see is how SRP handles things when the sendq fills up

Re: [PATCH 2/9] IB: add a proper completion queue abstraction

2015-11-23 Thread Jason Gunthorpe
On Mon, Nov 23, 2015 at 01:04:25PM -0800, Bart Van Assche wrote: > Considerable time ago the send queue in the SRP initiator driver was > modified from signaled to non-signaled to reduce the number of interrupts > triggered by the SRP initiator driver. The SRP initiator driver polls the > send que

Re: [PATCH for-next V1 9/9] IB/cma: Join and leave multicast groups with IGMP

2015-11-23 Thread Jason Gunthorpe
On Thu, Oct 15, 2015 at 07:07:12PM +0300, Matan Barak wrote: > diff --git a/include/rdma/ib_sa.h b/include/rdma/ib_sa.h > index 0a40ed2..5bea0e8 100644 > +++ b/include/rdma/ib_sa.h > @@ -206,6 +206,9 @@ struct ib_sa_mcmember_rec { > u8 scope; > u8 join_state; >

Re: [PATCH for-next V1 7/9] IB/cma: Add configfs for rdma_cm

2015-11-23 Thread Jason Gunthorpe
On Thu, Oct 15, 2015 at 07:07:10PM +0300, Matan Barak wrote: > Users would like to control the behaviour of rdma_cm. > For example, old applications which don't set the > required RoCE gid type could be executed on RoCE V2 > network types. In order to support this configuration, > we implement a co

Re: [PATCH for-next V1 3/9] IB/core: Add gid attributes to sysfs

2015-11-23 Thread Jason Gunthorpe
On Thu, Oct 15, 2015 at 07:07:06PM +0300, Matan Barak wrote: > This patch set adds attributes of net device and gid type to each GID > in the GID table. Users that use verbs directly need to specify > the GID index. Since the same GID could have different types or > associated net devices, users sh

Re: [PATCH for-next V1 5/9] IB/core: Add rdma_network_type to wc

2015-11-23 Thread Jason Gunthorpe
> + /* Use the hint from IP Stack to select GID Type */ > + network_gid_type = ib_network_to_gid_type(addr->dev_addr.network); > + if (addr->dev_addr.network != RDMA_NETWORK_IB) { > + route->path_rec->gid_type = network_gid_type; > + /* TODO: get the hoplimit fro

Re: [PATCH 2/9] IB: add a proper completion queue abstraction

2015-11-23 Thread Jason Gunthorpe
On Sat, Nov 14, 2015 at 08:13:44AM +0100, Christoph Hellwig wrote: > On Fri, Nov 13, 2015 at 03:06:36PM -0700, Jason Gunthorpe wrote: > > Looking at that thread and then at the patch a bit more.. > > > > +void ib_process_cq_direct(struct ib_cq *cq) > > [..] > >

Re: [PATCH 2/9] IB: add a proper completion queue abstraction

2015-11-23 Thread Jason Gunthorpe
On Sat, Nov 14, 2015 at 08:08:49AM +0100, Christoph Hellwig wrote: > On Fri, Nov 13, 2015 at 11:25:13AM -0700, Jason Gunthorpe wrote: > > For instance, like this, not fulling draining the cq and then doing: > > > > > + completed = __ib_process_cq(cq, budget); > &

Re: [PATCH 10/11] IB: only keep a single key in struct ib_mr

2015-11-23 Thread Jason Gunthorpe
spots in other API places. Anyhow, for the core stuff: Reviewed-by: Jason Gunthorpe Jason -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 11/11] IB: provide better access flags for fast registrations

2015-11-23 Thread Jason Gunthorpe
On Sun, Nov 22, 2015 at 06:46:49PM +0100, Christoph Hellwig wrote: > Instead of the confusing IB spec values provide a flags argument that > describes: > > a) the operation we perform the memory registration for, and > b) if we want to access it for read or write purposes. > > This helps to a

Re: [PATCH] IB/cma: Add a missing rcu_read_unlock()

2015-11-23 Thread Jason Gunthorpe
c2). > Cc: Haggai Eran > Cc: stable > drivers/infiniband/core/cma.c | 5 + > 1 file changed, 1 insertion(+), 4 deletions(-) Yep Reviewed-by: Jason Gunthorpe You say sparse detected this? Mellanox/Or folks - I thought you ran all the common static stuff on your patches? Jason -- T

Re: [PATCH] IB/ipoib: CSUM support in connected mode

2015-11-23 Thread Jason Gunthorpe
On Wed, Nov 18, 2015 at 10:27:41PM +0200, Yuval Shaia wrote: > > You need private-data exchange to negotiate the feature. > > > > The feature should be a per-packet csum status header. > > > > When sending a skb that is already fully csumed the receiver sets > > CHECKSUM_UNNECESSARY. > > > > Whe

Re: [PATCH for-next 03/10] IB/iser: Don't register memory for all immediatedata writes

2015-11-23 Thread Jason Gunthorpe
On Tue, Nov 17, 2015 at 11:41:39AM +0200, Sagi Grimberg wrote: > > >On 11/16/2015 6:37 PM, Sagi Grimberg wrote: > >>+++ b/drivers/infiniband/ulp/iser/iser_memory.c > >>@@ -250,7 +250,7 @@ iser_reg_dma(struct iser_device *device, struct > >>iser_data_buf *mem, > >> struct scatterlist *sg = mem

Re: garbage collect old memory registration code

2015-11-23 Thread Jason Gunthorpe
good to me. Thanks for doing this. Reviewed-by: Jason Gunthorpe (for core bits) Driver changes look pretty much trivial too me. Jason -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/9] IB: add a proper completion queue abstraction

2015-11-13 Thread Jason Gunthorpe
On Fri, Nov 13, 2015 at 11:57:56AM -0800, Bart Van Assche wrote: > I think this is the conversation you are referring to: "About a shortcoming > of the verbs API" (http://thread.gmane.org/gmane.linux.drivers.rdma/5028). > That conversation occurred five years ago, which means that you have an > ex

PGP Keysigning at SC|15?

2015-11-13 Thread Jason Gunthorpe
Hello, To those attending SC|15, I would like to try and meet up to do a PGP keysigning. I am starting a new key with modern crypto and would like to get a few signatures for it - https://quartz.obsidianresearch.com/~jgg/transition.txt I'll be around booth #278 and can meet up other interested p

Re: [PATCH 2/9] IB: add a proper completion queue abstraction

2015-11-13 Thread Jason Gunthorpe
On Fri, Nov 13, 2015 at 02:46:43PM +0100, Christoph Hellwig wrote: > This adds an abstraction that allows ULP to simply pass a completion > object and completion callback with each submitted WR and let the RDMA > core handle the nitty gritty details of how to handle completion > interrupts and poll

Re: [PATCH] IB/mad: In validate_mad, validate CM method and attribute

2015-11-12 Thread Jason Gunthorpe
On Thu, Nov 12, 2015 at 08:30:55PM +, Hefty, Sean wrote: > > > + /* CM attributes other than ClassPortInfo only use Send method > > */ > > > + if (mad_hdr->mgmt_class == IB_MGMT_CLASS_CM) { > > > + if (mad_hdr->attr_id != IB_MGMT_CLASSPORTINFO_ATTR_ID) { > > > +

Re: Fwd: [PATCH] svcrdma: Do not send XDR roundup bytes for a write chunk

2015-11-12 Thread Jason Gunthorpe
On Thu, Nov 12, 2015 at 09:47:13AM -0500, Chuck Lever wrote: > I wish git send-mail had an address book. I’m really tired > of misspelling the to/cc addresses on patches. It does: CONFIGURATION sendemail.aliasesfile To avoid typing long email addresses, point this to one or more

Re: [PATCH RFC 2/3] svcrdma: Use device rdma_read_access_flags

2015-11-11 Thread Jason Gunthorpe
On Wed, Nov 11, 2015 at 11:25:06AM +0200, Sagi Grimberg wrote: > For RDMA READs, a HCA will perform the memory scatters when on the RX > path, when receiving the read responses containing the data. This means > that the HCA needs to perform a lookup of the relevant scatter entries > upon each read

Re: [PATCH RFC 2/3] svcrdma: Use device rdma_read_access_flags

2015-11-11 Thread Jason Gunthorpe
On Wed, Nov 11, 2015 at 12:02:25AM -0800, Christoph Hellwig wrote: > > No need for the else, !rdma_cap_needs_rdma_read_mr means pd->local_dma_lkey > > is okay > > to use. > > What would happen if someone tried to use NFS on usnic without this? Honestly, I have no idea how usnic fits into the ker

Re: [PATCH RFC 2/3] svcrdma: Use device rdma_read_access_flags

2015-11-10 Thread Jason Gunthorpe
On Tue, Nov 10, 2015 at 06:01:01PM -0500, Chuck Lever wrote: > The current mechanism of statically choosing either FRMR or > local DMA depending on the device is probably not adequate. > Maybe we could post all the Read WRs via a single chain? Or > stick with FRMR when the amount of data to read is

Re: [PATCH RFC 2/3] svcrdma: Use device rdma_read_access_flags

2015-11-10 Thread Jason Gunthorpe
On Tue, Nov 10, 2015 at 04:00:43PM -0500, Tom Talpey wrote: > Hmm, agreed, but it must still be acceptable to perform a registration > instead of using the local_dma_lkey. As mentioned earlier, there are > scatter limits and other implications for all-physical addressing that > an upper layer may

Re: [PATCH v1 ib-next 2/3] IB/core: IB/core: Allow legacy verbs through extended interfaces

2015-11-10 Thread Jason Gunthorpe
On Tue, Nov 10, 2015 at 09:57:13PM +0200, Eli Cohen wrote: > Yes, we can do this but I think this should be the subject for another > patch, agree? Sure > Regarding using stabs, it may be nice but I don't think performance is > the issue here. Most verbs implementations involve relatively long i

Re: [PATCH ib-next 2/3] IB/core: IB/core: Allow legacy verbs through extended interfaces

2015-11-10 Thread Jason Gunthorpe
On Tue, Nov 10, 2015 at 10:02:26PM +0200, Eli Cohen wrote: > On Tue, Nov 10, 2015 at 11:23:35AM -0800, Bart Van Assche wrote: > > > > How about using ENOTSUPP ? > > > > $ PAGER= git grep 'define ENOTSUPP' include > > include/linux/errno.h:#define ENOTSUPP 524 /* Operation is not supported */ > >

Re: [PATCH RFC 1/3] IB/core: Expose a device attribute for rdma_read access flags

2015-11-10 Thread Jason Gunthorpe
On Tue, Nov 10, 2015 at 05:41:47AM -0800, Christoph Hellwig wrote: > FYI, this is the API I'd aim for (only SRP and no HW driver converted > yet): > n = ib_map_mr_sg(desc->mr, state->sg, state->sg_nents, > - dev->mr_page_size); > + dev->mr_page_size,

Re: [PATCH RFC 2/3] svcrdma: Use device rdma_read_access_flags

2015-11-10 Thread Jason Gunthorpe
On Tue, Nov 10, 2015 at 04:04:32AM -0800, Christoph Hellwig wrote: > On Tue, Nov 10, 2015 at 01:46:40PM +0200, Sagi Grimberg wrote: > > > > > > On 10/11/2015 13:41, Christoph Hellwig wrote: > > >Oh, and while we're at it. Can someone explain why we're even > > >using rdma_read_chunk_frmr for IB?

Re: [PATCH] IB: start documenting device capabilities

2015-11-10 Thread Jason Gunthorpe
On Tue, Nov 10, 2015 at 01:19:43PM +0100, Christoph Hellwig wrote: > Just IB_DEVICE_LOCAL_DMA_LKEY and IB_DEVICE_MEM_MGT_EXTENSIONS for now > as I'm most familar with those. > > Signed-off-by: Christoph Hellwig Looks right to me Reviewed-By: Jason Gunthorpe Jason -- To unsu

Re: [PATCH v1 ib-next 2/3] IB/core: IB/core: Allow legacy verbs through extended interfaces

2015-11-10 Thread Jason Gunthorpe
On Tue, Nov 10, 2015 at 08:00:09PM +0200, Eli Cohen wrote: > When an extended verbs is an extension to a legacy verb, the original > functionality is preserved. Hence we do not require each hardware driver > to set the extended capability. This will allow to use the extended verb > in its simple fo

Re: [PATCH RFC 1/3] IB/core: Expose a device attribute for rdma_read access flags

2015-11-10 Thread Jason Gunthorpe
On Tue, Nov 10, 2015 at 12:51:10PM +0100, Yann Droneaud wrote: > Why were those hw providers not modified to > enforce IB_ACCESS_REMOTE_WRITE when needed, instead of asking users to > set it for them ? iWarp hardware simply cannot do it it all. Jason -- To unsubscribe from this list: send the lin

Re: [PATCH RFC 1/3] IB/core: Expose a device attribute for rdma_read access flags

2015-11-10 Thread Jason Gunthorpe
On Tue, Nov 10, 2015 at 12:44:13PM +0200, Sagi Grimberg wrote: > Different devices may require different access permissions > for rdma reads e.g. for Infiniband devices, local write access > suffices while iWARP devices require remote write permissions as > well. > > This situation generates extra

Re: [PATCH ib-next 2/3] IB/core: IB/core: Allow legacy verbs through extended interfaces

2015-11-10 Thread Jason Gunthorpe
On Tue, Nov 10, 2015 at 12:25:41PM +0200, Matan Barak wrote: > The kernel will do the above fallback if the command is a legacy > command wrapped in an extended command (i.e - no extra flags). > If an application uses one of the extended values, and fall back to > legacy verb on ENOSYS, it'll behav

Re: [PATCH ib-next 2/3] IB/core: IB/core: Allow legacy verbs through extended interfaces

2015-11-10 Thread Jason Gunthorpe
On Tue, Nov 10, 2015 at 05:40:26PM +0200, Eli Cohen wrote: > On Tue, Nov 10, 2015 at 05:23:10PM +0200, Eli Cohen wrote: > > > > > > Call it ENOTSUP then: > > > > > >ENOTSUP Operation not supported (POSIX.1) > > > > > > Same value on Linux. > > > > Yes, that's better. I will rese

Re: [PATCH ib-next 2/3] IB/core: IB/core: Allow legacy verbs through extended interfaces

2015-11-09 Thread Jason Gunthorpe
On Tue, Nov 10, 2015 at 01:24:31AM +0200, Eli Cohen wrote: > > Also, when the driver tests the ex flags for support it should be > > returning EOPNOTSUPP or such not EINVAL.. Return codes for the ex > > stuff could stand a good sanity audit. > > > > #define EOPNOTSUPP 95 /* Operation no

  1   2   3   4   5   6   7   8   9   10   >