RE: APM support for IPoIB
> Perhaps working on getting that merged upstream would be a better first step. We have it on our plans to submit RDMA-CM APM to upstream. Since we are in holidays season here it will take us some time to come back with plan. In any case as Bang wrote this is not related to IPoIB-CM, that requires a different implementation, and any one is welcome to contribute here :-) Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH for-next V2 00/22] Add SRIOV support for IB interfaces
many thanks Jack Tziporet -Original Message- From: linux-rdma-ow...@vger.kernel.org [mailto:linux-rdma-ow...@vger.kernel.org] On Behalf Of Jack Morgenstein Sent: Friday, August 03, 2012 11:41 AM To: rol...@kernel.org Cc: linux-rdma@vger.kernel.org; Or Gerlitz; Jack Morgenstein; Dotan Barak; Tziporet Koren; Yevgeny Petrilin; Liran Liss Subject: [PATCH for-next V2 00/22] Add SRIOV support for IB interfaces This patch set adds SRIOV support for IB interfaces. Patches 1-4 are "precondition" patches. Patches 5-22 actually implement the feature. NOTE: Patch 18 depends on patch IB/mlx4: fix possible deadlock with sm_lock spinlock (a separately-submitted patch) being applied previously to the for-next git. This patch set introduces Infiniband SRIOV support for ConnectX2 and ConnectX3 devices. Each function presents itself as an independent vHCA (virtual HCA) to the host while a single HCA is observable by the network, which is unaware of the vHCAs. No changes are required by the IB subsystem, ULPs, and apps to support SRIOV, and vHCAs are interoperable with any existing (non-virtualized) IB deployments. We term this model for SRIOV implementation the shared-port model. Sharing the same physical port(s) among multiple vHCAs is achieved as follows: 1. Each vHCA port presents its own virtual GID table. Currently, the virtual GID table comprises a single entry (at index 0) that maps to a unique index in the physical GID table. The vHCA of the PF maps to physical GID index 0. To obtain GIDs for other vHCAs, alias GUIDs are requested from the SM. These are GUIDs which the SM places, per port, in the port's guid table after the 0'th slot (which is read-only and determined by the FW). The host admin can assign GIDs to vHCAs using a sysfs interface (see below). 2. Each vHCA port presents its own virtual PKey table. The virtual PKey table is a mapping of selected indexes of the physical pkey table. The host admin can control which pkey indexes are mapped to which virtual indexes using a sysfs interface (see below). Note that the physical PKey table may contain both full and partial memberships of the same PKey to allow different membership types in different virtual tables. 3. Each vHCA port has its own virtual port state. A vHCA port is up if the following conditions apply: - The physical port is up - The virtual GID table contains the GIDs requested by the host admin - The SM has acknowledged the requested GIDs since the last time that the physical port came up 4. Other port attributes are shared, e.g., GID prefix, LID, SM LID, LMC mask. 5. Special QPs are para-virtualized. vHCAs are not given direct access to QP0/1. Rather, these QPs are operated by a special context hosted by the PF, which mediates access to/from vHCAs. This is done by opening a “tunnel” per vHCA port per QP0/1. A tunnel comprises a pair of UD QPs: a “Tunnel QP” in the PF-context and a “Proxy QP” in the vHCA. All vHCA MAD traffic must pass through the corresponding tunnel. vHCA QPs cannot be assigned to VL15 and are denied of the well-known QKey. QP0 access is restricted to the PF vHCA. VF vHCAs also have (virtual) QP0’s, but they never receive any SMPs and all SMPs sent are discarded. QP1 traffic is allowed for all vHCAs, but special care is required to bridge the gap between the host and network views. Specifically: - Transaction IDs are mapped to guarantee uniqueness among vHCAs - CM para-virtualization o Incoming requests are steered to the correct vHCA according to the embedded GID o Local communication IDs are mapped to ensure uniqueness among vHCAs - Multicast para-virtualization o The PF context aggregates membership state from all vHCAs o The SA is contacted only when the aggregate membership changes o If the aggregate does not change, the PF context will provide the requesting vHCA with the proper response Incoming MADs are steered according to: - the DGID If a GRH is present - the mapped transaction ID for response MADs - the embedded GID in CM requests - the remote communication ID in other CM messages To allow the host admin to control the virtual GID and PKey tables of vHCAs, a new sysfs ‘iov’ sub-tree has been added under the PF infiniband device. Details on this mechanism can be found in the change log of: IB/mlx4: Add iov directory in sysfs under the ib device Some Limitations 1. FMRs are not currently supported on slaves. This will be corrected in a future submission. 2. RoCE is not currently supported on slaves. This will be corrected in a future submission. Changes for V2 -- 1. Patches already accepted by Roland in the V1 set are removed from V2. The following (precondition) patches submitted in V1 were accepted: 1: net/mlx4_core: Pass an invalid PCI id number to VFs 3: IB/mlx4: Add debug printouts 6: IB/sa: Add GuidInfoRecord query su
RE: [PATCH][MINOR] mstflint/hca_self_test.ofed: Handle lspci output for ConnectX3 (on RHEL 6.2)
Lloyd
Oren just had a new baby
I hope he will handle it fast when he will return from vacation
Tziporet
-Original Message-
From: linux-rdma-ow...@vger.kernel.org
[mailto:linux-rdma-ow...@vger.kernel.org] On Behalf Of Lloyd Brown
Sent: Tuesday, July 17, 2012 7:12 PM
To: linux-rdma@vger.kernel.org
Subject: Re: [PATCH][MINOR] mstflint/hca_self_test.ofed: Handle lspci output
for ConnectX3 (on RHEL 6.2)
Just wondering. Did I send this to the wrong place? Am I working off the
wrong repo? I expected such a simple patch to fly through, but I haven't heard
anything. If anyone can tell me what I did wrong, I'd be happy to fix it.
Lloyd Brown
Systems Administrator
Fulton Supercomputing Lab
Brigham Young University
http://marylou.byu.edu
On 07/12/2012 08:39 AM, Lloyd Brown wrote:
> Not sure whether the difference is the hardware, or the OS revision,
> but the lspci output looks like this example, and wasn't being caught:
>
> 02:00.0 Network controller [0207]: Mellanox Technologies MT27500
> Family [ConnectX-3]
>
> Signed-off-by: Lloyd Brown
> ---
> hca_self_test.ofed |4 ++--
> 1 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/hca_self_test.ofed b/hca_self_test.ofed index
> c3e3f22..3c5360d 100755
> --- a/hca_self_test.ofed
> +++ b/hca_self_test.ofed
> @@ -110,7 +110,7 @@ MEM_CON_LSPCI_NAME="Memory controller"
> MEL_LSPCI_NAME_EXP="${HCA_LSPCI_NAME}|${VPI_LSPCI_NAME}|${NIC_LSPCI_NAME}"
>
> # HCA/PCI check
> -NUM_IB_DEV=`lspci 2> /dev/null | grep -E "(${MEL_LSPCI_NAME_EXP}):
> Mellanox Technolog" | wc -l`
> +NUM_IB_DEV=`lspci 2> /dev/null | grep -E
> +"(${MEL_LSPCI_NAME_EXP})(\s\[[0-9]+\])?: Mellanox Technolog" | wc -l`
> NUM_MEM_CON=`lspci 2> /dev/null | grep "${MEM_CON_LSPCI_NAME}:
> Mellanox Technolog" | wc -l`
>
> let "NUM_HCAS=$NUM_IB_DEV + $NUM_MEM_CON"
> @@ -211,7 +211,7 @@ function get_curr_ca_pci_info () {
> loop_cnt=$1
>
> if [ "${g_lspci_was_ran}" == "" ]; then
> -lspci 2> /dev/null | grep -E
> "(${MEL_LSPCI_NAME_EXP}|${MEM_CON_LSPCI_NAME}): Mellanox Technolog" >
> ${G_LSPCI_OUTPUT_FILE}
> +lspci 2> /dev/null | grep -E
> + "(${MEL_LSPCI_NAME_EXP}|${MEM_CON_LSPCI_NAME})(\s\[[0-9]+\])?:
> + Mellanox Technolog" > ${G_LSPCI_OUTPUT_FILE}
> g_lspci_was_ran=1
> fi
> current_pci_info=$(cat ${G_LSPCI_OUTPUT_FILE} | head -$(expr
> $loop_cnt + 1) | tail -1)
>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the
body of a message to majord...@vger.kernel.org More majordomo info at
http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [GIT PULL] please pull infiniband.git
Hi Roland Can you updated us on status of review of our mlx4 SRIOV IB submission? Thanks Tziporet -Original Message- From: linux-rdma-ow...@vger.kernel.org [mailto:linux-rdma-ow...@vger.kernel.org] On Behalf Of Roland Dreier Sent: Sunday, June 24, 2012 5:11 AM To: Linus Torvalds Cc: linux-rdma@vger.kernel.org; linux-ker...@vger.kernel.org Subject: [GIT PULL] please pull infiniband.git Hi Linus, Please pull from git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git tags/rdma-fixes InfiniBand/RDMA fixes for 3.5-rc: - Fixes to new ocrdma driver - Typo in test in CMA Dan Carpenter (1): RDMA/ocrdma: Fix off by one in ocrdma_query_gid() Mahesh Vardhamanaiah (2): RDMA/ocrdma: Correct reported max queue sizes RDMA/ocrdma: Correct queue SGE calculation Parav Pandit (2): RDMA/ocrdma: Fixed GID table for vlan and events RDMA/ocrdma: Fixed RQ error CQE polling Roland Dreier (1): Merge branches 'cma' and 'ocrdma' into for-linus Sean Hefty (1): RDMA/cma: QP type check on received REQs should be AND not OR drivers/infiniband/core/cma.c |2 +- drivers/infiniband/hw/ocrdma/ocrdma.h |1 + drivers/infiniband/hw/ocrdma/ocrdma_hw.c| 18 drivers/infiniband/hw/ocrdma/ocrdma_main.c | 63 +++ drivers/infiniband/hw/ocrdma/ocrdma_sli.h |5 ++- drivers/infiniband/hw/ocrdma/ocrdma_verbs.c | 12 ++--- 6 files changed, 55 insertions(+), 46 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [GIT PULL] please pull infiniband.git
>> What about the IPoIB patches (e.g. use Interrupt moderation) > I have no idea what patch that might be. We've had the modify_cq call in > ipoib for years now, with moderation parameters set through ethtool. http://www.spinics.net/lists/linux-rdma/msg09142.html and http://www.spinics.net/lists/linux-rdma/msg09143.html Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [GIT PULL] please pull infiniband.git
Roland What about the IPoIB patches (e.g. use Interrupt moderation) Regards, Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[RFC] – Proposal for new process for OFED releases
We propose a new process for the OFED releases starting from next OFED release: - OFED content will be the relevant kernel.org modules and user space released packages - OFED will offer only backports to the distros (no fixes) - OFED package will be used for easy installation of all packages in a friendly manner The main goals of this change: 1. Ensure OFED and the upstream kernel are the same 2. Provide customers a way to use the new features in latest kernels on existing distros 3. OFED qualification will contribute to the stability of the upstream code We think that at this point of the RDMA technology maturity this is the right way to go. In this way OFED is not conflicting with the kernel or the distros, and still provide a valuable value for early adopters of new features. Versions: We suggest that the OFED version will be the same as kernel.org For example, for kernel 3.2 the OFED release would be OFED-3.2. This would make it easy for people to associate the OFED code with the corresponding kernel.org code. Some open questions that we should consider: - How to handle experimental features? - Need to follow up kernel stable releases if bug fixes are relevant to OFA modules - Should we have a release for every kernel release (I think yes) - What should we do with modules like SDP that are not in kernel? Comments and responses are welcome The decision will be taken in the next EWG meeting. Tziporet & Woody -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: Kernel and User Libraries Patch
> I work DataDirect Networks Inc. and have been working with SRP target driver > using OFED kernel and user libraries. > I had to patch some kernel and user libraries to export a verb > ‘ib_modify_port’ to user space as SRP target driver requires it. > Wanted to check if this patch can be applied to the OFED Code for future > releases and if yes, what would be the procedure to follow ? Hi Pavan The process is to submit the patches to the linux-rdma list (linux-rdma@vger.kernel.org) Since its related to SRP you should mail it to SRP maintainer David Dillow After this is accepted by the kernel you can submit the patches to OFED with the relevant backport patches Regards, Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [ANNOUNCE] libmthca 1.0.5 and libmlx4 1.0.2 are released
>> What about the new CX3 device ID patch? > Did I ever get a final version of the libmthca and kernel patches? CX3 is using libmlx4 and not libmthca. Vlad sent a patch on 1/19/11: http://www.spinics.net/lists/linux-rdma/msg07237.html Is contain adding several device IDs including CX3 Do you wish separate patch on each device? Regards, Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [ANNOUNCE] libmthca 1.0.5 and libmlx4 1.0.2 are released
Roland What about the new CX3 device ID patch? Also - when will the RoCE user space patches be included in the library? Thanks Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [RFC] XRC upstream merge reboot
> > For MPI, I would expect an xrcd to be associated with a single job instance. > > So did I, but they said that this was not the case, and they were very > > pleased > > with the final (more complicated implementation-wise) interface. > > We need to get them involved in this discussion ASAP. > Tziporet, who should be the MPI contacts for this thread? Yevgeny K. can contact the Open MPI people Sayantan can be the contact for MVAPICH Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] please pull infiniband.git
On 1/17/2011 11:55 PM, Roland Dreier wrote: Linus, please pull from master.kernel.org:/pub/scm/linux/kernel/git/roland/infiniband.git for-linus This tree is also available from kernel.org mirrors at: git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git for-linus This will get the last batch of InfiniBand/RDMA patches for the merge window. The biggest thing here is Tejun's conversion of the RDMA stack away from flush_scheduled_work(); the remaining stuff is mostly driver fixes, with an SRP compile warning fix and a vzalloc() conversion patch thrown in. Roland, What about the new mlx4 device IDs? We wish to have them in for CX3 so that the new kernel will support it Thanks, Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Does any load balancing mode work for IPoIB bonding?
On 1/13/2011 8:08 PM, Chris Worley wrote: It looks like active-backup is the only bonding mode that works. Is that correct? Yes this is teh only mode supported by IPoIB Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: SR-IOV support in MLX CX2
On 1/10/2011 4:03 PM, Wei Lin Guay wrote: Hi all, Is Mellanox CX2 HCA supports SR-IOV? I'm currently using CentOS 5.5 with Xen 4.0.1 but open for any recommendation on different linux dist. The first blocking issue that I have is my CX2 doesn't have the SR-IOV extended capability in the PCI config dump as mentioned in the specification. What are the additional firmware/driver that is required in order to have the SR-IOV support in CX2? If SR-IOV is currently not supported, how can I enable PCI-bypass for CX2 in Xen? CX2 is capable of supporting SRIOV, however we have not yet released FW and driver for this This is under work now and I hope we will start submitting patches to the kernel soon ... Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ewg] IPoIB to Ethernet routing performance
On 12/28/2010 5:30 PM, Reeted wrote: You and Richard seem to have good experience of infiniband in virtualized environments. May I ask one thing? We were thinking about buying some Mellanox Connectx-2 for use with SR-IOV (hardware virtualization for PCI bypass, supposedly supported by connectx-2) in KVM (also supposedly supports SR-IOV and PCI bypass). Do you have info if this will work, in KVM or other hypervisors? I asked in KVM mailing list but they have not tried this card (which is the only SR-IOV card among Infiniband ones, so they have not tried infiniband). We are working on enabling SRIOV on ConnectX2 cards Once we will have it working with KVM we will submit the patches to the linux-rdma list Should be in few months - but don't ask how much is few :-) Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: static_rate broken in mlx4?
On 12/6/2010 10:00 PM, Jason Gunthorpe wrote: Does this work? Has anyone seen it work? I'm testing it on a pair of mlx4 MT25418 and I'm trying to get down to 2.5Gbps and I can't see any effect from twiddling the static_rate AH attribute for an RC QP. The value seems to be working, I put 30 in to a ibv_modify_qp with IBV_QP_AV and 2 comes back (the max, it looks like..) Static rate is not implemented on ConnectX Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: root owned writable files under /sys
Yevgeny is going to RD next week Will see if someone can work on this here before he returns Tziporet -Original Message- From: Or Gerlitz [mailto:ogerl...@voltaire.com] Sent: Thursday, November 18, 2010 3:03 PM To: Jack Morgenstein; Yevgeny Petrilin Cc: Sumeet Lahorani; Tziporet Koren; linux-rdma@vger.kernel.org Subject: Re: root owned writable files under /sys Jack Morgenstein wrote: > Port trigger belongs to Yevgeny P. Please interact with him regarding this. I know, the thing is that I failed to get any response from him on the matter of pushing it upstream for long time, and this is IB related patch, so I tried to pull you on that... Yevgeny, could you push that patch any time soon? Or. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: Support for Memory windows.
> Is there any plans to implement it in a near future? No Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Support for Memory windows.
On 11/15/2010 5:06 PM, Romulo Goncalves wrote:
Hello,
For my PhD project I have been using RDMA over inifiniband.
Until now I have used memory regions, but now I would like to also define
memory windows.
The steps taken are the following ones:
Register a memory region.
Alloc a window
bind the window to the registered memory region.
The memory region registration worked as expected, however, the call of the
function to alloc the memory window did not.
It seems there is not an implementation for the functions to alloc, bind, and
dealloc a memory window.
I looked into verbs.h and I found the reference to the functions in the
context_ops:
struct ibv_context_ops {
struct ibv_mw * (*alloc_mw)(struct ibv_pd *pd, enum ibv_mw_type type);
int (*bind_mw)(struct ibv_qp *qp, struct ibv_mw
*mw, struct ibv_mw_bind *mw_bind);
int (*dealloc_mw)(struct ibv_mw *mw);
}
However, I do not see their declaration or implementation anywhere else.
Could be the version of my openfabrics does not support memory windows?
Memory Windows have not been implemented
Tziporet
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: ib receive completion error
On 11/9/2010 9:22 PM, Usha Srinivasan wrote: > Hello, > Can someone from Mellanox tell me what the vendor error 0x32 means? I am > getting this error for wc.opcode 128 (IB_WC_RECV) wc.status 4 > (IB_WC_LOC_PROT_ERR). This error means corrupted MPT or PD violation Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: ib0: failed send event
On 11/4/2010 10:45 PM, Craig Prescott wrote: Hi, We have a new-ish node whose IPoIB access was interrupted last night; last message in the syslog was: ib0: failed send event (status=2, wrid=74 vend_err 67) The node has a Mellanox ConnectX HCA (MHQH19-XTC). I understand the status corresponds to IB_WC_LOC_QP_OP_ERR. What does a vendor error code of 67 mean? Thanks, Craig Prescott -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Error is: Bus error reading WQE. Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: FW: [PATCHv10 02/12] ib_core: IBoE CMA device binding
On 10/5/2010 7:57 PM, Roland Dreier wrote: > Sorry, I don't have much to say. I am working on many things at once, > IBoE is one of them. I hope to get it in for 2.6.37. Thanks, I know you are a very busy man, it's just that this is very important for us. Please let us know the moment you need anything from us so will not miss the submission window. Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: FW: [PATCHv10 02/12] ib_core: IBoE CMA device binding
On 9/26/2010 6:46 PM, Tziporet Koren wrote: Roland, Following Eli's update below, we have not received any further comments on this matter. Please confirm that this is in line with what you had requested. Also, having not heard any other concerns with the rest of the iboe v10 patchset, we assume it is in good shape for upstream integration in the upcoming 2.6.37. If there is anything further that you would like us to modify please let us know and we will do it immediately. Can we get an answer? Ignoring mails is not the way to behave in the Linux community Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Maximum size for memory registration (ibv_reg_mr)
On 9/25/2010 12:57 AM, Roland Dreier wrote: All Mellanox adapters have some limit on the amount of memory they can register. You can mess around with module parameters to adjust it. You can use mlx4_core module parameter (log_mtts_per_seg) to enlarge number of MTTs per segment. This enable to register more memory with the same number of segments. In OFED 1.5.2 the max value was enlarged to 7 Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: FW: [PATCHv10 02/12] ib_core: IBoE CMA device binding
On Thu, Sep 02, 2010 at 01:10:17PM -0700, Roland Dreier wrote: Please just resend this, thanks. Here's the new code. I added a function that tries to find a match for a given port, istead of ib_find_cached_gid() which searches all the ports. I tested this on a machine with two cards with different configurations and it works fine. Roland, Following Eli's update below, we have not received any further comments on this matter. Please confirm that this is in line with what you had requested. Also, having not heard any other concerns with the rest of the iboe v10 patchset, we assume it is in good shape for upstream integration in the upcoming 2.6.37. If there is anything further that you would like us to modify please let us know and we will do it immediately. Thanks, Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: libibverbs: question on future releases
On 9/16/2010 6:28 PM, Roland Dreier wrote: I don't have definite plans, and I do plan to include your patch. I guess I should do it soon... Will the new libibverbs include the changes needed for IBoE too? Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: CQ overrun with ib_send_bw
On 8/13/2010 10:21 PM, Ralph Campbell wrote: > On Fri, 2010-08-13 at 12:14 -0700, Hefty, Sean wrote: >>> I know there is a bug with "ib_send_bw -b" (bi-directional) >>> since it doesn't create a CQ that is large enough for all the >>> posted sends *and* receives. I have tried several times to get the >>> following patch applied but I never got a reply and nothing was >>> done. >> >> Who's the maintainer of these tests? > > I believe it is: > > Ido Shamai > > git://git.openfabrics.org/~shamoya/perftest.git > > Yes Ido is the maintainer, however he is on vacation till Sep. I add Amir that may help for now Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: InfiniBand/RDMA merger plans for 2.6.36
> It's not personal -- it was a combination of when things were ready and > when I had time to work on things, and unfortunately it all lined up so > I wasn't able to get anything big queued up this cycle. Can you confirm it will be in 2.6.37? Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: InfiniBand/RDMA merge plans for 2.6.36
On 8/5/2010 2:33 AM, Roland Dreier wrote: Hi Roland, > > Here are a few topics I'm tracking that are not ready in time for the > 2.6.36 window and will need to wait for 2.6.37 at least: > > - XRC. While I think we made significant progress here, the fact is >that this is not ready to merge at the beginning of the merge >window, and so we'll need to keep working on it and wait for the >next merge window. I think this is just blocked on me at the >moment. Jack did all the changes you asked for and submitting the V3 patches on 2010/05/19, which was long enough before the window. What is the reason it will not be part of 2.6.36 kernel? > > - IBoE. Same as XRC; we made significant progress (and I opened an >iboe branch to track this), and I think we have finally gotten the >user-kernel interface nailed down yet, but it's just too late. Eli sent the user space patch that was the last technical debate last week. He will also send a new patch-set in day or two. This feature was under a lot of technical review from the community. Please consider including it in 2.6.36 after all. In general - none of our new features going to be included and I wish to understand the reason behind it (I hope it's not personal :-) regards Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [RFC PATCH] libmthca: Add support for the reg_io_mr verb.
Hi Tom, What is the purpose of this? Is there a reason you did it only for mthca and not mlx4? Tziporet
Re: [ewg] https://bugs.openfabrics.org/show_bug.cgi?id=2074
On 7/10/2010 1:54 AM, Sunil Joshi wrote: Hi , As per the comment in the following bugzilla bug, Can someone please help in getting things resolved. I assigned it to Vlad, I hope he can help you Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: root owned writable files under /sys
On 7/7/2010 8:42 AM, Or Gerlitz wrote: > > Jack, Tziporet > > Can you clarify the status of the upstream kernel mlx4 multi-protocol > support? looking on Linus git, I see one commit, > 7ff93f8b7ecbc36e7ffc5c11a61643821c1bfee5 "mlx4_core: Multiple port type > support" dated to Oct 2008, wheres ofed ships couple of patches touching this > area, e.g adding the above sysfs entries. So what is the extra functionality > introduced or bug/s fixed by those patches? any reason not to push them > upstream? > Jack is on vacation and will be back in 2 weeks I will ask him to look at this when he is back Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: mlx4 pci device table
On 6/28/2010 4:11 PM, Or Gerlitz wrote: > Roland Dreier wrote: >> I think the current upstream location is correct. This matches the practice >> of eg iw_cxgb3 as well as cxgb3i, bnx2i >> etc. This does have the >> disadvantage that mlx4_en and mlx4_ib are not auto-loaded by PCI hotplug, >> but so it goes. > okay. Still, its too bad that ofed ships patches that do things the > other way around vs upstream. Yevgeny, if you have reasoning in place to > do things the other way, why not submit upstream? > Or This patch was submitted to upstream but was not accepted. however without it PXE boot cannot work since only mlx4_core is loaded and not the Eth driver Any other suggestion how to solve it will be accepted. Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ewg] libibverbs without an HCA
On 6/1/2010 2:39 PM, Albert Strasheim wrote: Hello all Having just reviewed the various on-host shared memory mechanisms available in Linux and being quite unimpressed, I have the following question: Has anyone considered making a kernel and userspace driver so that libibverbs can be used on a single machine without any Infiniband hardware? The Verbs API seems like a very nice way to handle shared memory between processes where the number of shared buffers can vary in size and quantity. Regards Albert These kind of questions should be addressed by the Linux RDMA List (already CC) Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] mlx4_en: show device's port used
On 5/26/2010 12:32 AM, Roland Dreier wrote: > I don't think there are many devices out there which have more than > one port. ?? http://developer.intel.com/network/connectivity/solutions/gigabit.htm http://www.broadcom.com/products/Ethernet-Controllers/Enterprise-Server/BCM5704C etc But they have different PCI function for each port and we have 2 ports on same PCI device Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ewg] [PATCH] mlx4_core: request MSIX vectors as much as there CPU cores
On 5/5/2010 7:32 PM, Jason Gunthorpe wrote: On Wed, May 05, 2010 at 07:54:54AM -0700, Roland Dreier wrote: > The current code requires num_possible_cpus() + 1 MSIX vectors. However, > num_possible_cpus() stands for the max number of supported CPUs by the kernel. I believe this is wrong -- num_possible_cpus() is the maximum number of CPUs for the running system, including hotplug. FWIW, I've seen some systems running RH kernels that blow up here because they run out of interrupt numbers if you have two IB cards in them. Mainline kernels have a reworked IRQ number allocator and don't have a problem, so maybe this is an ofed only patch? This is not related to OFED We found it in performance work of our EN (10G) driver Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [GIT PULL] please pull infiniband.git
On 3/2/2010 9:56 AM, Roland Dreier wrote: Linus, please pull from master.kernel.org:/pub/scm/linux/kernel/git/roland/infiniband.git for-linus This tree is also available from kernel.org mirrors at: git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git for-linus This will get my current pile for the 2.6.34 merge window: What about the pending patches for: - XRC - IBoE - SRIOV support for mlx4 Thanks Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH V2 2/2] mlx4/IB: Add support for enhanced atomic operations
On 2/26/2010 1:44 AM, Roland Dreier wrote: Is there no firmware dependency here? If this really should be set unconditionally, then please just roll it into the other flags that get set for all devices. There is no FW dependency here and it's working from first FW release of ConnectX. We have added it to the driver just now since there was no request before and now RDS wish to use it. Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[ANNOUNCE] OFED 1.5.1 rc2 release is available
OFED 1.5.1-rc2 is available Notes: The tarball is available on: http://www.openfabrics.org/downloads/OFED/ofed-1.5.1/OFED-1.5.1-rc2.tgz To get BUILD_ID run ofed_info Please report any issues in bugzilla https://bugs.openfabrics.org/ for OFED 1.5.1 Vladimir & Tziporet Supported Platforms and Operating Systems - o CPU architectures: - x86_64 - x86 - ppc64 - ia64 o Linux Operating Systems: - RedHat EL4 up72.6.9-78.ELsmp - RedHat EL4 up82.6.9-89.ELsmp - RedHat EL5 up32.6.18-128.el5 - RedHat EL5 up42.6.18-164.el5 - SLES10 SP22.6.16.60-0.21-smp - SLES10 SP32.6.16.60-0.54-smp - SLES112.6.27.19-5-default - OEL 4 up7 2.6.9-78.ELsmp - OEL 4 up8 2.6.9-89.ELsmp - CentOS5.3 2.6.18-128.el5 - CentOS5.4 2.6.18-164.el5 - Fedora Core12 2.6.31.5-127.fc12* - OpenSuSE 11.2 2.6.31.5-0.1-default * - kernel.org2.6.29, 2.6.30, 2.6.31 and 2.6.32* * Minimal QA for these versions Main changes from 1.5.1-rc1: === 1. Updated packages: - DAPL (compat-dapl-1.2.16, dapl-2.0.27) - ibutils: ibutils-1.5.2 - ib-bonding: ib-bonding-0.9.0-42 - libnes: libnes-1.0.0-0.1.g9b5f558 - libsdp: libsdp-1.1.100-0.1.g920ea31 - management: updated to the latest build - iSER: Added RHEL5.4 support - MPI: mvapich-1.2.0-3635 mvapich2-1.4-3.20100222svn3774 2. Bug fixes commit bb30f640fd973d48686e1663afbab67b23fa1c09 Merge: 03a9f17 329fbc0 Author: Vladimir Sokolovsky Date: Wed Feb 24 08:47:38 2010 +0200 Merge branch 'code-drop/20100223' of git://git.openfabrics.org/~agrover/ofed_1_5/linux-2.6 into ofed_kernel_1_5 commit 03a9f17215dde8a49c8da22be88a8e1dc69ba754 Merge: cff66e5 19d717c Author: Vladimir Sokolovsky Date: Wed Feb 24 08:40:15 2010 +0200 Merge branch 'ofed_kernel_1_5' of ssh://sofa.openfabrics.org/home/ctung/scm/ofed-1.5 into ofed_kernel_1_5 commit 329fbc005ecf6307377158aec7a7ce54c70777a9 Author: Tina Yang Date: Fri Feb 19 16:53:00 2010 -0800 RDS: Fix locking in rds_send_drop_to() Signed-off-by: Tina Yang Signed-off-by: Andy Grover commit 02955868562b143e4f81929275793bfa6c5b0ce7 Author: Andy Grover Date: Tue Feb 16 15:41:15 2010 -0800 RDS: Turn down alarming reconnect messages RDS's error messages when a connection goes down are a little extreme. A connection may go down, and it will be re-established, and everything is fine. This patch links these messages through rdsdebug(), instead of to printk directly. Signed-off-by: Andy Grover commit 3e7b1a2efceab7a4090fe8cbf5b8c2e2c67e7529 Author: Andy Grover Date: Tue Feb 16 13:54:12 2010 -0800 RDS: Workaround for in-use MRs on close causing crash if a machine is shut down without closing sockets properly, and freeing all MRs, then a BUG_ON will bring it down. This patch changes these to WARN_ONs -- leaking MRs is not fatal (although not ideal, and there is more work to do here for a proper fix.) Signed-off-by: Andy Grover commit 19d717c18ee40ca830bfed01839913329c548ed1 Author: Chien Tung Date: Tue Feb 23 12:20:54 2010 -0600 RDMA/nes: add kernel_patches/fixes/nes_0027_kr.patch add support for KR device id 0x0110. nes_init_phy clean up. Deprecate NES_PHY_TYPE_IRIS. Signed-off-by: Chien Tung commit cff66e5d6f3cb6dbd3ae68b7cbfd017a89833fa5 Merge: 464df47 51dbf00 Author: Vladimir Sokolovsky Date: Tue Feb 23 17:38:33 2010 +0200 Merge remote branch 'amirv/ofed_kernel_1_5' into ofed_kernel_1_5 commit 51dbf00d8b656eaa89b95db86faa42f2106109c8 Author: Amir Vadai Date: Tue Feb 23 10:02:59 2010 +0200 sdp: Prevent kernel crash if device init fails (plus bonus fix) If sdp_add_device() fails, there is no client data stored in the IB device, leading to a kernel crash when a connection is being established. Fix this by rejecting connections when the device is not initialized. Also, fix a bad goto target in an error case early in sdp_init_qp(). Signed-off-by: Joachim Fenkes Signed-off-by: Amir Vadai commit 464df477da4cb5996fdc39b95340b63797ed210b Author: Eli Cohen Date: Tue Feb 23 14:23:55 2010 +0200 mlx4: Fix bug in SL for IBoE SL is mapped to 3 bits in IBoE. However, we need to save all 4 bits so that consumers that query the SL will read the value they set. Signed-off-by: Eli Cohen commit cae3a274c31755ca933d639c07ac433583c03213 Author: Eli Cohen Date: Mon Feb 22 18:27:16 2010 +02
OFED 1.5.1 roadmap
Hi I have updated OFED 1.5.1 roadmap on the web: http://www.openfabrics.org/txt/woody/roadmap.txt Schedule: - - RC1 - Feb 11, 2010 - done - RC2 - Feb 25, 2010 - RC3 - Mar 4, 2010 - GA - Mar 11, 2010 Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ewg] MLX4 Strangeness
On 2/15/2010 10:24 PM, Tom Tucker wrote: Hello, I am seeing some very strange behavior on my MLX4 adapters running 2.7 firmware and the latest OFED 1.5.1. Two systems are involved and each have dual ported MTHCA DDR adapter and MLX4 adapters. The scenario starts with NFSRDMA stress testing between the two systems running bonnie++ and iozone concurrently. The test completes and there is no issue. Then 6 minutes pass and the server "times out" the connection and shuts down the RC connection to the client. From this point on, using the RDMA CM, a new RC QP can be brought up and moved to RTS, however, the first RDMA_SEND to the NFS SERVER system fails with IB_WC_RETRY_EXC_ERR. I have confirmed: - that "arp" completed successfully and the neighbor entries are populated on both the client and server - that the QP are in the RTS state on both the client and server - that there are RECV WR posted to the RQ on the server and they did not error out - that no RECV WR completed successfully or in error on the server - that there are SEND WR posted to the QP on the client - the client side SEND_WR fails with error 12 as mentioned above I have also confirmed the following with a different application (i.e. rping): server# rping -s client# rping -c -a 192.168.80.129 fails with the exact same error, i.e. client# rping -c -a 192.168.80.129 cq completion failed status 12 wait for RDMA_WRITE_ADV state 10 client DISCONNECT EVENT... However, if I run rping the other way, it works fine, that is, client# rping -s server# rping -c -a 192.168.80.135 It runs without error until I stop it. Does anyone have any ideas on how I might debug this? Tom What is the vendor syndrome error when you get a completion with error? Does the issue occurs only on the ConnectX cards (mlx4) or also on the InfiniHost cards (mthca) Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: ibutils license problems + a few other minor issues
On 2/11/2010 7:11 PM, Doug Ledford wrote: The ibutils tarball as found for download on the openfabrics web site has some licensing issues that need resolved ASAP. The tarball in its current form is probably illegal to distribute until the items are fixed. So, here's what popped up in review (done by Jay Fenlason): The COPYING files sprinkled around in the tarball are not consistent. Some of them contain a copy of the GPLv2, some of them contain a statement that the package is licensed under either GPLv2 in the file COPYING or the OFA BSD license included here in. Of course, it doesn't work to reference a COPYING file from within the COPYING file you are referencing and say it has the information you aren't including here. I would suggest that the proper place for the statement of GPLv2 or BSD is in possibly a LICENSE file with the COPYING file being the GPLv2 as stated. It also seems redundant to have multiple COPYING files and possibly also multiple LICENSE files in a single tarball, so cleanup of unnecessary files seems in order here. Hi Doug Thanks for providing the info (I guess it was in this way from the first days we submitted the code to OFA) Yevgeny will fix this soon Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ewg] [PATCH] [for-2.6.33] rdma/cm: disallow loopback address for iwarp devices
On 2/8/2010 8:02 AM, Sean Hefty wrote: Since iWarp devices are not guaranteed to support loopback connections, prevent rdma_bind_addr from associating the loopback address with an iWarp device. Signed-off-by: Sean Hefty Steve Have you tested this patch? When accepted to kernel can you prepare a patch for OFED 1.5.1 under fixes Thanks Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ewg] bug 1918 - openmpi broken due to rdma-cm changes
On 2/7/2010 6:39 PM, Steve Wise wrote: If ofed-1.5.1 is based on 2.6.33 then it will get this patch automatically (assuming it goes upstream and makes 2.6.33). Or we can pull it in as a kernel_patches/fixes/ patch. OFED 1.5.1 is not based on 2.6.33, but on 2.6.30, so we need the patch under fixes. Steve - can you prepare such a patch? Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ewg] bug 1918 - openmpi broken due to rdma-cm changes
On 2/5/2010 6:52 PM, Sean Hefty wrote: BTW: Was this change an artifact of rebasing ofed-1.5.1 on a new kernel version? apparently Sorry to jump late on this thread OFED 1.5.1 was not rebased on a new kernel - its still based on 2.6.30. But many time we take patches that were accepted by the kernel to OFED. These patches where pushed to OFED by David Wilder from IBM, since loopback support of CMA was important for them. Therefore I add David to the thread too. Steve/Jeff - how come no one tested iWARP with the new kernel patches when Sean submitted them? Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ewg] bug 1918 - openmpi broken due to rdma-cm changes
On 2/7/2010 3:22 AM, Steve Wise wrote: Good catch, I'll update the patch and submit for 2.6.33 on Monday. NOTE: This doesn't solve our IB/openmpi regression for ofed-1.5.1. If this patch will be accepted to the kernel 2.6.33 we can take it too Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/2] Add support for enhanced atomic operations
On 2/2/2010 5:54 PM, Hal Rosenstock wrote: Will Mellanox be adding this option to the IB spec rather than keep it as vendor proprietary ? Yes, it's in our plans Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 0/4] Current XRC queue
> I finally found time to finish going over the base XRC patches. I fixed > the "userspace can destroy an XRCD handle while still having QPs/SRQs > referring to it" problem with a reference count, rather than the more > complicated method used by the original patches. > > I think the next step would be for someone (probably not me if we want > this to happen soon) to port the rest of the mlx4 XRC patch set on top > of this, ideally trying to factor out any reference-counting-ish code > away from the device-specific driver and into common code. Then we > should finally be able to merge this and move on to the IBoE changes. Thanks Roland Jack from Mellanox will take it from here Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: mode connected infiniband
On 1/14/2010 7:22 PM, Roland Dreier wrote: and we should probably change the upstream kernel default to "yes" as well. I agree IPoIB CM is very stable now and has a better performance Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Documentation for libiverbs?
On 1/13/2010 12:06 PM, Werther Pirani wrote: Hello, I've been redirected to this list from OpenFabrics' web site, so let me apologise right away if this is not the right place for this question. We are writing an application that requires high throughput and low latency. More specifically, we need to take advantage of features like RDMA, possibly through the ibv_ API in libiverbs, but the documentation available is really too sketchy to be of any help (and even the OpenFabrics' web site isn't of any use in this respect). At the moment all I have to go by is the RDMA_RC_example in the "RDMA Ware Network Programming User's Manual" from Mellanox, but it's really not quite enough. Since even searching the web isn't turning out any documentation, I was wondering if anyone here has any to share. You can look at the ping-pong examples that come with libibverbs. http://www.openfabrics.org/downloads/libibverbs/ Also you can look at the performance tests (ib_write_lat, ib_write_bw, etc). http://www.openfabrics.org/downloads/perftest/ Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: OpenFabrics server is not responding
Vlad Can you explain what you saw with other eth that are responding? Tziporet -Original Message- From: Woodruff, Robert J [mailto:robert.j.woodr...@intel.com] Sent: Wednesday, December 23, 2009 6:01 PM To: Tziporet Koren; Jeff Squyres; bec...@nas.nasa.gov; Bill Boas Cc: 'Jeffrey Scott'; linux-rdma@vger.kernel.org Subject: RE: OpenFabrics server is not responding Tziporet wrote, >Bill - do you know who is the administrator that can fix this >We cannot access git and also web site and ewg mailing list does not >work I cannot access it either. Does anyone know where it is physically located ? woody -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: OpenFabrics server is not responding
Bill - do you know who is the administrator that can fix this We cannot access git and also web site and ewg mailing list does not work Thanks Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ANNOUNCE] OFED 1.5 rc4 release is available
On 12/16/2009 6:39 PM, Moni Shoua wrote: Limitations: - SLES10 SP3 on IA64 is not supported yet In what way is it unsupported? Our QA say that OFED-1.5 can be compiled on this platform. OK - so it is supported Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: InfiniBand/RDMA merge plans for 2.6.33
On 12/14/2009 8:37 PM, Roland Dreier wrote: - mlx4 SR-IOV support. Again, main problem was my lack of time. I agree in principle with this stuff, just want to be careful that we don't turn the mlx4 driver into an unmaintainable mess of "if (sriov) something; else something_else" all over. Roland, You have not sent any comments to our patches that were sent few weeks ago on time for 2.6.32 inclusion, and now I am surprised you do not accept them for 2.6.32. I think we still have time to work together and fix your concerns on mlx4 driver. Can you send more concrete comments so we can fix them? Since we have a HW that supports SRIOV and many people are interested in this new technology for KVM thus it is important that we drive it now Thanks Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OFED-1.x SRP's user manual
Pawel Dziekonski wrote: On Mon, 16 Nov 2009 at 05:39:45PM +0100, Pawel Dziekonski wrote: Hello, where do I find SRP user manual? There is no manual in OFED distro, I have also checked main website and wiki. Manual is mentioned in OFED/docs/SRPT_README.txt. Hello again. I got no answers - does the manual exist at all? P You can look at the UM on Mellanox web: http://www.mellanox.com/related-docs/prod_software/Mellanox_OFED_Linux_user_manual_1_40_1.pdf look at Chapter 7 SRP Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RDMAoE verbs questions
Jason Gunthorpe wrote: On Tue, Nov 24, 2009 at 06:23:15PM -0500, Jeff Squyres wrote: 2. I am somewhat confused by the overloading of the term "transport". It appears that a device will have ibv_device.transport_type==IBV_TRANSPORT_IB for both IB and RDMAOE devices. The only way to tell the difference is to examine the new ibv_port_attr.transport field to see if it is RDMA_TRANSPORT_IB or RDMA_TRANSPORT_RDMAOE. I haven't seen these patches but this seems poor to me. I think any app that isn't using rdmacm will need patching and support for RDMAOE (certainly all mine will). libibverbs shouldn't overload the existing transport_type checks for something that is not 100% compatible with IB. Good catch - I agree that the ABI should be 100% backward compatible, and we will fix this. We can add a sysfs option to query the transport type, or add another verb Note that application does not need to query the transport type, but we thought it can be good to know also from debug perspective. Thus I think sysfs is the best place. Opinions? Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH RESEND] ib/iser: re-write SG handling for rdma logic
Roland Dreier wrote: Actually I started merging stuff without sending the "applied" emails yet. So far I have this in for-next: What about the new RDMAoE patches? Thanks Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: OFED 1.5 complains about freeing invalid memtype
Bart Van Assche wrote: Hello, While /etc/init.d/openibd runs the following kernel messages were generated: mstmread:2578 freeing invalid memtype fe60-fe61 mstmread:2578 freeing invalid memtype fe61-fe70 mstmread:2581 freeing invalid memtype fe60-fe61 mstmread:2581 freeing invalid memtype fe61-fe70 mstmread:2584 freeing invalid memtype fe60-fe61 mstmread:2584 freeing invalid memtype fe61-fe70 mstmread:2587 freeing invalid memtype fe60-fe61 mstmread:2587 freeing invalid memtype fe61-fe70 mstmread:2626 freeing invalid memtype fea0-fea1 mstmread:2626 freeing invalid memtype fea1-feb0 mstmread:2629 freeing invalid memtype fea0-fea1 mstmread:2629 freeing invalid memtype fea1-feb0 mstmread:2632 freeing invalid memtype fea0-fea1 mstmread:2632 freeing invalid memtype fea1-feb0 mstmread:2635 freeing invalid memtype fea0-fea1 mstmread:2635 freeing invalid memtype fea1-feb0 This occurred with a 2.6.30.9 kernel with kernel debugging enabled, OFED-1.5-20091027-0600 and mlx4 hardware. Is this a known issue ? This happened from mstmread utility Are you using it? In any case we will check it in Mellanox Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Multicast joins failing on 1.5-rc1? (OFED BACKPORT BUG)
Jason Gunthorpe wrote: Correct. We need someone to pick up the above patch for the backports. I don't know who that is (someone please speak up?) Vlad will take it Please open a bug in bugzilla too Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ewg] Re: Possible process deadlock in RMPP flow
Sean Hefty wrote: I can't find anything off in the code for this. Eventually it was a FW issue that is fixed in our new 2.7.0 release Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ewg] Re: Possible process deadlock in RMPP flow
Or Gerlitz wrote: Eli Cohen wrote: Thanks Or. This one is already in OFED 1.4.2 but apparently this is a different problem. Once I have information whether the patch Roland posted fixed it I will update the list. Eli, did you find a commit that fixes the problem you reported on? Or. Not yet :-( Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ofa-general] InfiniBand/RDMA merge plans for 2.6.32
Roland Dreier wrote: > What about RDMAoE? > Patches were sent few weeks ago and it seems you ignore them. Sorry, I should have mentioned that. Yes, I have been ignoring the patches -- I want to get through XRC first, We can help with XRC if this will expedite the RDMAoE. Will it? Tziporet -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
