Re: [openib-general] Re: Questions about libibat, ib_uat, and ib_a
On Wed, 2005-11-02 at 01:02, Pradeep Satyanarayana wrote: > [EMAIL PROTECTED] wrote on 10/18/2005 03:40:47 PM: > > > > > > > > > > > On Mon, 2005-10-18 at 10:07, Kevin Reilly wrote: > > >On Mon, 2005-10-17 at 10:07, Hal Rosenstock wrote: > > >> > Should this code work, because it seems that out_dev is a > kernel > > >> > address (platform: PPC64) which cannot accessed by a userspace > > >> > program. Via GDB I can see that rt has the following content: > > >> > > > >> > The address is rt->out_dev = 0xc000cffaa800 which looks > like a > > >> > kernel address. > > >> > > >> Yes, this is a bug which has been previously pointed out on the > list and > > >> not fixed. > > > > > Can some one point me to the previous discussions on this list (search > did not yield any results)? There were various posts from Heiko J Schick <[EMAIL PROTECTED]> on 10/17 and subsequent ones from Kevin. > The problem is because of a copy_to_user (in uat.c) between struct > ib_at_ib_route > which are different between user and kernel space causing this crash. > What was the rationale of putting a pointer to struct ibv_device in > the user space version of > ib_at_ib_route? The out_dev field in user space is not really used as > far as I could see. > > > >The fix for this involves an ABI change: it should return the GID > of the > > >outgoing IB device. > > > > > Would a simple solution like adding a device_name field to both the > ib_at_ib_route structures > be acceptable? The out_dev field could be used as a "reserved" field > in user space and not be used. > That should not break anything as far as I can see. Ideally, the current out_dev field should be removed and all consumers should be converted over to the new structures/interfaces. Guess the question also is whether people want this by name, GID, or both ? -- Hal > > >-- Hal > > Pradeep > [EMAIL PROTECTED] > > __ > > ___ > openib-general mailing list > openib-general@openib.org > http://openib.org/mailman/listinfo/openib-general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Re: Questions about libibat, ib_uat, and ib_a
[EMAIL PROTECTED] wrote on 10/18/2005 03:40:47 PM: > > > > > On Mon, 2005-10-18 at 10:07, Kevin Reilly wrote: > >On Mon, 2005-10-17 at 10:07, Hal Rosenstock wrote: > >> > Should this code work, because it seems that out_dev is a kernel > >> > address (platform: PPC64) which cannot accessed by a userspace > >> > program. Via GDB I can see that rt has the following content: > >> > > >> > The address is rt->out_dev = 0xc000cffaa800 which looks like a > >> > kernel address. > >> > >> Yes, this is a bug which has been previously pointed out on the list and > >> not fixed. > > Can some one point me to the previous discussions on this list (search did not yield any results)? The problem is because of a copy_to_user (in uat.c) between struct ib_at_ib_route which are different between user and kernel space causing this crash. What was the rationale of putting a pointer to struct ibv_device in the user space version of ib_at_ib_route? The out_dev field in user space is not really used as far as I could see. > >The fix for this involves an ABI change: it should return the GID of the > >outgoing IB device. > > Would a simple solution like adding a device_name field to both the ib_at_ib_route structures be acceptable? The out_dev field could be used as a "reserved" field in user space and not be used. That should not break anything as far as I can see. > >-- Hal Pradeep [EMAIL PROTECTED]___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Re: Questions about libibat, ib_uat, and ib_a
Kevin Reilly wrote: Thanks Sean, I think the rdma_resolve_addr() does what we want. Translate a local IP to a ib_device structure that i can use in the ibverbs. What we want to do is pretty simple and we won't need to create a connection. Based on your description, I think that rdma_bind_addr() may work better. The bind call works synchronously, whereas, resolve is an asynchronous operation. The difference is that bind translates a local address only, and resolve translates remote and an optional local address. Can we have a discussion on the timeframe for this? These are already working in the kernel. A userspace implementation should be available in 1-2 weeks. I finished coding the necessary userspace support kernel module on Friday of last week, and am now starting on the userspace library. - Sean ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] Re: Questions about libibat, ib_uat, and ib_a
>Will the CMA have that the same function ib_at_route_by_ip() that we are >using in libibat? The CMA will resolve IB routes based on source/destination TCP/IP addresses if that is what you are looking for. It will then establish connections based on those routes. You may want to look at rdma_cm.h in the include/rdma directory to ensure that it meets your needs. - Sean ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Re: Questions about libibat, ib_uat, and ib_a
Will the CMA have that the same function ib_at_route_by_ip() that we are
using in libibat?
/**
* ib_at_route_by_ip - asynchronously resolve ip address to ib route
* @dst_ip: destination ip
* @src_ip: source ip - optional
* @tos: ip type of service
* @flags: ib_at_route_flags
* @ib_route: out structure
* @async_comp: asynchronous callback structure - optional
* @req_id: pointer for request ID
*
* Resolve the specified dst_ip to a &struct ib_route structure.
* src_ip can be provide to force specific output interface.
* flags can be used to select resolving method; currently IB-ARP or ATS.
*
* See ib_at_completion structure documentation for asynchronous
* operation details.
*/
int ib_at_route_by_ip(uint32_t dst_ip, uint32_t src_ip, int tos, uint16_t
flags,
struct ib_at_ib_route *ib_route,
struct ib_at_completion *async_comp, uint64_t
*req_id);
Kevin J. Reilly
STSM, HPC Architecture
-Federation/HPS Chief Engineer
-HPC interconnect architect
(office) 845-433-7976 (tieline) 8-293-7976
Sean Hefty
<[EMAIL PROTECTED]
ntel.com> To
Pradeep
10/20/2005 12:47 Satyanarayana/Beaverton/[EMAIL
PROTECTED]
PM cc
Hal Rosenstock <[EMAIL PROTECTED]>,
Kevin
Reilly/Poughkeepsie/[EMAIL PROTECTED],
[EMAIL PROTECTED],
"openib-general@openib.org"
Subject
Re: [openib-general] Re: Questions
about libibat, ib_uat, and ib_a
Pradeep Satyanarayana wrote:
> Is there a ballpark estimate (or a plan) of when CMA willl be ready?
> Estimates like by end of Q4 2005
> or end of Q1 2006 will help us make some decisions if we should submit a
> patch for this bug or wait
> for CMA.
The kernel CMA is ready today. An additional change will be required at
some
point once the iWarp Emulation Protocol is defined, but that will be minor.
Work on the user CMA should begin by the end of this week. I estimate that
it
will take about 4 weeks to complete.
- Sean
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Re: Questions about libibat, ib_uat, and ib_a
Pradeep Satyanarayana wrote: Is there a ballpark estimate (or a plan) of when CMA willl be ready? Estimates like by end of Q4 2005 or end of Q1 2006 will help us make some decisions if we should submit a patch for this bug or wait for CMA. The kernel CMA is ready today. An additional change will be required at some point once the iWarp Emulation Protocol is defined, but that will be minor. Work on the user CMA should begin by the end of this week. I estimate that it will take about 4 weeks to complete. - Sean ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Re: Questions about libibat, ib_uat, and ib_a
On Thu, 2005-10-20 at 12:35, Pradeep Satyanarayana wrote: > Is there a ballpark estimate (or a plan) of when CMA willl be ready? > Estimates like by end of Q4 2005 > or end of Q1 2006 will help us make some decisions if we should submit > a patch for this bug or wait > for CMA. CMA is ready now. It's up to Sean to say what he thinks for user CMA availability (for planning purposes). -- Hal ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Re: Questions about libibat, ib_uat, and ib_a
[EMAIL PROTECTED] wrote on 10/19/2005 01:37:14 PM: > On Tue, 2005-10-18 at 18:40, Kevin Reilly wrote: > > > > > > On Mon, 2005-10-18 at 10:07, Kevin Reilly wrote: > > >On Mon, 2005-10-17 at 10:07, Hal Rosenstock wrote: > > >> > Should this code work, because it seems that out_dev is a kernel > > >> > address (platform: PPC64) which cannot accessed by a userspace > > >> > program. Via GDB I can see that rt has the following content: > > >> > > > >> > The address is rt->out_dev = 0xc000cffaa800 which looks like a > > >> > kernel address. > > >> > > >> Yes, this is a bug which has been previously pointed out on the list and > > >> not fixed. > > > > > >The fix for this involves an ABI change: it should return the GID of the > > >outgoing IB device. > > > > > >-- Hal > > > > Should we (IBM) work on submitting a patch for this? > > That's up to you. > > > Returning the GID or the device_name would be good fix. > > Yes, either of these could be made to work. > > > I guess our reluctance is that we've heard the this address translation > > library function might be depreciated for another interface? > > Yes, that has been my reluctance as well. It appears AT is likely to be > superceeded by CMA. > Is there a ballpark estimate (or a plan) of when CMA willl be ready? Estimates like by end of Q4 2005 or end of Q1 2006 will help us make some decisions if we should submit a patch for this bug or wait for CMA. Pradeep [EMAIL PROTECTED]___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Re: Questions about libibat, ib_uat, and ib_a
On Tue, 2005-10-18 at 18:40, Kevin Reilly wrote: > > > On Mon, 2005-10-18 at 10:07, Kevin Reilly wrote: > >On Mon, 2005-10-17 at 10:07, Hal Rosenstock wrote: > >> > Should this code work, because it seems that out_dev is a kernel > >> > address (platform: PPC64) which cannot accessed by a userspace > >> > program. Via GDB I can see that rt has the following content: > >> > > >> > The address is rt->out_dev = 0xc000cffaa800 which looks like a > >> > kernel address. > >> > >> Yes, this is a bug which has been previously pointed out on the list and > >> not fixed. > > > >The fix for this involves an ABI change: it should return the GID of the > >outgoing IB device. > > > >-- Hal > > Should we (IBM) work on submitting a patch for this? That's up to you. > Returning the GID or the device_name would be good fix. Yes, either of these could be made to work. > I guess our reluctance is that we've heard the this address translation > library function might be depreciated for another interface? Yes, that has been my reluctance as well. It appears AT is likely to be superceeded by CMA. > Having neither leaves us without a method to translate healthy > "heartbeat-able" IP interfaces to HCAs where we can run things over verbs. As of now, there is no user space CMA although work is likely to commence on this shortly. You should make sure the APIs suit your needs. -- Hal > Kevin J. Reilly > STSM, HPC Architecture > -Federation/HPS Chief Engineer > -HPC interconnect architect > (office) 845-433-7976 (tieline) 8-293-7976 > > ___ > openib-general mailing list > openib-general@openib.org > http://openib.org/mailman/listinfo/openib-general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Re: Questions about libibat, ib_uat, and ib_a
On Mon, 2005-10-17 at 10:07, Hal Rosenstock wrote: > > Should this code work, because it seems that out_dev is a kernel > > address (platform: PPC64) which cannot accessed by a userspace > > program. Via GDB I can see that rt has the following content: > > > > The address is rt->out_dev = 0xc000cffaa800 which looks like a > > kernel address. > > Yes, this is a bug which has been previously pointed out on the list and > not fixed. The fix for this involves an ABI change: it should return the GID of the outgoing IB device. -- Hal ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
