Re: Write Packets to InfiniBand HCA
On 1/4/2012 11:23 PM, Greg I Kerr wrote: Yes I want to do this with IB. I wasn't aware that I could use a UD QP to write my own packets to the wire. Is this what you're suggesting I can use a UD QP for? You'll have to further clarify the my own packets part of your sentence... IB have some layers which come into play in various elements (software/hardware of HCAs/switches) during the life cycle of the packet. Specifically each IB packet would have an IB L2 header called LRH, IB also has L3 header named GRH which is optional unless you need multicast (or IBoE). The UD thing I mentioned is IB's L4 - the transport. The most common transports are UD and RC. So if you go UD the packet will have L2 (LRH) and L4 UD headers (called BTH plus DTH) which on top of you add the payload. Or. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: Write Packets to InfiniBand HCA
Greg, I don't think the iPath interface is documented outside the headers, which are included as part of the QLogic PSM package. I believe that package is included with stock OFED. iPath is basically the hardware interface at the very bottom of the QLogic software stack, it wasn't intended for normal use. -Original Message- From: Roland Dreier [mailto:rol...@purestorage.com] Sent: Tuesday, January 03, 2012 6:41 PM To: Greg I Kerr Cc: Mike Heinz; linux-rdma@vger.kernel.org Subject: Re: Write Packets to InfiniBand HCA It is possible with ConnectX (cf MLX QPs in the kernel driver). However I don't know what documentation is available and some hacking would be needed to use this for something more general than sending MADs on special QPs. On Tue, Jan 3, 2012 at 8:43 AM, Greg I Kerr ker...@husky.neu.edu wrote: Yes I should have mentioned that I am using a Mellanox Connect-X adapter. Do you know where I can find documentation for QLogic's iPath? A quick google search didn't seem to turn anything up. Thanks for the information. - Greg Kerr On Tue, Jan 3, 2012 at 9:53 AM, Mike Heinz michael.he...@qlogic.com wrote: That would depend on which HCA you are using. I know that you can use QLogic's iPath interface to do what you want, but I don't think it is possible through the verbs interface available through stock OFED. -Original Message- From: linux-rdma-ow...@vger.kernel.org [mailto:linux-rdma-ow...@vger.kernel.org] On Behalf Of Greg I Kerr Sent: Monday, January 02, 2012 9:50 PM To: linux-rdma@vger.kernel.org Subject: Write Packets to InfiniBand HCA Hi, Does anyone know if it is or isn't possible to put the HCA in some kind of raw mode where I can compose a packet in software and write it to the card? This is obviously in comparison to calling ibv_post_send and having a work request converted into a packet. Thanks, Greg Kerr -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html This message and any attached documents contain information from QLogic Corporation or its wholly-owned subsidiaries that may be confidential. If you are not the intended recipient, you may not read, copy, distribute, or use this information. If you have received this transmission in error, please notify the sender immediately by reply e-mail and then delete this message. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html This message and any attached documents contain information from QLogic Corporation or its wholly-owned subsidiaries that may be confidential. If you are not the intended recipient, you may not read, copy, distribute, or use this information. If you have received this transmission in error, please notify the sender immediately by reply e-mail and then delete this message. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Write Packets to InfiniBand HCA
Roland Dreier rol...@purestorage.com wrote: It is possible with ConnectX (cf MLX QPs in the kernel driver). However I don't know what documentation is available and some hacking would be needed to use this for something more general than sending MADs on special QPs. Greg, RAW Ethernet QPs (soon to be re-submitted upstream) which to be implemented within the ConnnetX / mlx4 driver as MLX transport QPs would allow you to do that - over Ethernet. If you're asking on IB, I would suggest using UD QPs which are available today from user space, do you have any issue with them? Or. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Write Packets to InfiniBand HCA
Thanks for the help everyone. Greg, RAW Ethernet QPs (soon to be re-submitted upstream) which to be implemented within the ConnnetX / mlx4 driver as MLX transport QPs would allow you to do that - over Ethernet. If you're asking on IB, I would suggest using UD QPs which are available today from user space, do you have any issue with them? Yes I want to do this with IB. I wasn't aware that I could use a UD QP to write my own packets to the wire. Is this what you're suggesting I can use a UD QP for? Thanks, Greg On Wed, Jan 4, 2012 at 4:03 PM, Or Gerlitz or.gerl...@gmail.com wrote: Roland Dreier rol...@purestorage.com wrote: It is possible with ConnectX (cf MLX QPs in the kernel driver). However I don't know what documentation is available and some hacking would be needed to use this for something more general than sending MADs on special QPs. Or. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: Write Packets to InfiniBand HCA
That would depend on which HCA you are using. I know that you can use QLogic's iPath interface to do what you want, but I don't think it is possible through the verbs interface available through stock OFED. -Original Message- From: linux-rdma-ow...@vger.kernel.org [mailto:linux-rdma-ow...@vger.kernel.org] On Behalf Of Greg I Kerr Sent: Monday, January 02, 2012 9:50 PM To: linux-rdma@vger.kernel.org Subject: Write Packets to InfiniBand HCA Hi, Does anyone know if it is or isn't possible to put the HCA in some kind of raw mode where I can compose a packet in software and write it to the card? This is obviously in comparison to calling ibv_post_send and having a work request converted into a packet. Thanks, Greg Kerr -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html This message and any attached documents contain information from QLogic Corporation or its wholly-owned subsidiaries that may be confidential. If you are not the intended recipient, you may not read, copy, distribute, or use this information. If you have received this transmission in error, please notify the sender immediately by reply e-mail and then delete this message. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Write Packets to InfiniBand HCA
Yes I should have mentioned that I am using a Mellanox Connect-X adapter. Do you know where I can find documentation for QLogic's iPath? A quick google search didn't seem to turn anything up. Thanks for the information. - Greg Kerr On Tue, Jan 3, 2012 at 9:53 AM, Mike Heinz michael.he...@qlogic.com wrote: That would depend on which HCA you are using. I know that you can use QLogic's iPath interface to do what you want, but I don't think it is possible through the verbs interface available through stock OFED. -Original Message- From: linux-rdma-ow...@vger.kernel.org [mailto:linux-rdma-ow...@vger.kernel.org] On Behalf Of Greg I Kerr Sent: Monday, January 02, 2012 9:50 PM To: linux-rdma@vger.kernel.org Subject: Write Packets to InfiniBand HCA Hi, Does anyone know if it is or isn't possible to put the HCA in some kind of raw mode where I can compose a packet in software and write it to the card? This is obviously in comparison to calling ibv_post_send and having a work request converted into a packet. Thanks, Greg Kerr -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html This message and any attached documents contain information from QLogic Corporation or its wholly-owned subsidiaries that may be confidential. If you are not the intended recipient, you may not read, copy, distribute, or use this information. If you have received this transmission in error, please notify the sender immediately by reply e-mail and then delete this message. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Write Packets to InfiniBand HCA
It is possible with ConnectX (cf MLX QPs in the kernel driver). However I don't know what documentation is available and some hacking would be needed to use this for something more general than sending MADs on special QPs. On Tue, Jan 3, 2012 at 8:43 AM, Greg I Kerr ker...@husky.neu.edu wrote: Yes I should have mentioned that I am using a Mellanox Connect-X adapter. Do you know where I can find documentation for QLogic's iPath? A quick google search didn't seem to turn anything up. Thanks for the information. - Greg Kerr On Tue, Jan 3, 2012 at 9:53 AM, Mike Heinz michael.he...@qlogic.com wrote: That would depend on which HCA you are using. I know that you can use QLogic's iPath interface to do what you want, but I don't think it is possible through the verbs interface available through stock OFED. -Original Message- From: linux-rdma-ow...@vger.kernel.org [mailto:linux-rdma-ow...@vger.kernel.org] On Behalf Of Greg I Kerr Sent: Monday, January 02, 2012 9:50 PM To: linux-rdma@vger.kernel.org Subject: Write Packets to InfiniBand HCA Hi, Does anyone know if it is or isn't possible to put the HCA in some kind of raw mode where I can compose a packet in software and write it to the card? This is obviously in comparison to calling ibv_post_send and having a work request converted into a packet. Thanks, Greg Kerr -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html This message and any attached documents contain information from QLogic Corporation or its wholly-owned subsidiaries that may be confidential. If you are not the intended recipient, you may not read, copy, distribute, or use this information. If you have received this transmission in error, please notify the sender immediately by reply e-mail and then delete this message. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Write Packets to InfiniBand HCA
Hi, Does anyone know if it is or isn't possible to put the HCA in some kind of raw mode where I can compose a packet in software and write it to the card? This is obviously in comparison to calling ibv_post_send and having a work request converted into a packet. Thanks, Greg Kerr -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
