[openib-general] [MailServer Notification]To Recipient virus found and action taken.
ScanMail for Microsoft Exchange has detected virus-infected attachment(s). Sender = [EMAIL PROTECTED] Recipient(s) = openib-general@openib.org Subject = [openib-general] Members Support Scanning time = 10/7/2005 2:04:42 AM Engine/Pattern = 7.510-1002/2.879.00 Action on virus found: The attachment ykj.zip contains WORM_MYTOB.EI virus. ScanMail has Deleted it. Warning to recipient. ScanMail has detected a virus. 10/7/2005 ykj.zip/Deleted openib-general@openib.org [EMAIL PROTECTED] [openib-general] Members Support ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] Timeline of IPoIB performance
I'm seeing an IPoIB netperf performance drop off, up to 90 MB/s, when using kernels newer than 2.6.11. This doesn't appear to be an OpenIB IPoIB issue since the in-kernel and a recent svn3687 snapshot both have the same performance (464 MB/s) with 2.6.11. I used the same kernel config file as a starting point for each of these kernel builds. Have there been any changes in Linux that would explain these results? All benchmarks are with RHEL4 x86_64 with HCA FW v4.7.0 dual EM64T 3.2 GHz PCIe IB HCA (memfull) Kernel OpenIBmsi_x netperf (MB/s) 2.6.14-rc3 in-kernel1 374 2.6.13.2svn3627 1 386 2.6.13.2in-kernel1 394 2.6.12 in-kernel1 406 2.6.11 in-kernel1 464 2.6.11 svn3687 1 464 2.6.9-11.ELsmp svn3513 1 425 (Woody's results, 3.6Ghz EM64T) Thanks, - Matt ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] Your order# 1266.
You've seen it on 60 Minutes and read the BBC News report -- now find out just what everyone is talking about. # Suppress your appetite and feel full and satisfied all day long # Increase your energy levels # Lose excess weight # Increase your metabolism # Burn body fat # Burn calories # Attack obesity And more.. http://htupreulx.info/ # Suitable for vegetarians and vegans # MAINTAIN your weight loss # Make losing weight a sure guarantee # Look your best during the summer months http://htupreulx.info/ Regards, Dr. Jimmie Fleming ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Latest build test results
On Thu, 2005-10-06 at 14:11, Nishanth Aravamudan wrote:
On 06.10.2005 [13:25:35 -0400], Hal Rosenstock wrote:
On Thu, 2005-10-06 at 13:11, Nishanth Aravamudan wrote:
On 06.10.2005 [19:40:40 +0300], Dan Bar Dov wrote:
I've fixed the 2.6.14-rc3 compilation warnings with iSER on x86 in
version 3682.
Great! Thanks.
I'm re-running the tests (due to a subtle flaw in my PATH, my cronjobs
weren't running) now and will post the latest results.
You might also want to apply
https://openib.org/svn/gen2/trunk/src/linux-kernel/patches/linux-2.6.14-rc3-fib-frontend.diff
to get rid of the AT and SDP warnings.
I already submitted several jobs for 2.6.14-rc3-git6, but I'll redo the
gen2 ones with that patch, thanks.
Here are the results from 2.6.14-rc3-git6 + gen2 3683
Looks like x86 is broken in the current svn tree.
x86 and ppc64 mainline is fine with both =y and =m
ppc64 + gen2 =y
drivers/infiniband/ulp/srp/ib_srp.c: In function `srp_process_rsp':
drivers/infiniband/ulp/srp/ib_srp.c:650: warning: long long unsigned int
format, u64 arg (arg 2)
drivers/infiniband/core/at.c:1547: warning: initialization from incompatible
pointer type
drivers/infiniband/ulp/sdp/sdp_link.c:752: warning: initialization from
incompatible pointer type
ppc64 + gen2 =m
same as above, plus
*** Warning: .ip_dev_find [drivers/infiniband/ulp/sdp/ib_sdp.ko] undefined!
*** Warning: .ip_dev_find [drivers/infiniband/core/ib_at.ko] undefined!
x86 + gen2 =y *FAILED*
What gcc version are you using ?
drivers/infiniband/ulp/iser/iser_conn.c: In function `iser_adaptor_release':
drivers/infiniband/ulp/iser/iser_conn.c:195: parse error before `)'
drivers/infiniband/ulp/iser/iser_conn.c:203: parse error before `)'
drivers/infiniband/ulp/iser/iser_conn.c:206: parse error before `)'
drivers/infiniband/ulp/iser/iser_conn.c: In function `iser_conn_establish':
drivers/infiniband/ulp/iser/iser_conn.c:284: parse error before `)'
drivers/infiniband/ulp/iser/iser_conn.c: In function
`iser_post_receive_control':
drivers/infiniband/ulp/iser/iser_conn.c:861: parse error before `)'
drivers/infiniband/ulp/iser/iser_conn.c:873: parse error before `)'
drivers/infiniband/ulp/iser/iser_initiator.c: In function `iser_reg_rdma_mem':
drivers/infiniband/ulp/iser/iser_initiator.c:125: parse error before `)'
drivers/infiniband/ulp/iser/iser_initiator.c:130: parse error before `)'
drivers/infiniband/ulp/iser/iser_initiator.c:141: parse error before `)'
drivers/infiniband/ulp/iser/iser_initiator.c:153: parse error before `)'
drivers/infiniband/ulp/iser/iser_mod.c: In function `init_module':
drivers/infiniband/ulp/iser/iser_mod.c:154: parse error before `)'
drivers/infiniband/ulp/iser/iser_mod.c: In function `cleanup_module':
drivers/infiniband/ulp/iser/iser_mod.c:243: parse error before `)'
drivers/infiniband/ulp/iser/iser_lkdapl.c: In function
`iser_create_ia_pz_evd':
drivers/infiniband/ulp/iser/iser_lkdapl.c:147: parse error before `)'
drivers/infiniband/ulp/iser/iser_lkdapl.c: In function `iser_consume_events':
drivers/infiniband/ulp/iser/iser_lkdapl.c:691: parse error before `)'
drivers/infiniband/ulp/iser/iser_lkdapl.c: In function
`iser_event_handler_thread':
drivers/infiniband/ulp/iser/iser_lkdapl.c:731: parse error before `)'
drivers/infiniband/ulp/iser/iser_lkdapl.c:749: parse error before `)'
drivers/infiniband/ulp/iser/iser_lkdapl.c: In function
`iser_handle_conn_event':
drivers/infiniband/ulp/iser/iser_lkdapl.c:776: parse error before `)'
drivers/infiniband/ulp/iser/iser_lkdapl.c:779: parse error before `)'
drivers/infiniband/ulp/iser/iser_lkdapl.c:782: parse error before `)'
drivers/infiniband/ulp/iser/iser_lkdapl.c:785: parse error before `)'
drivers/infiniband/ulp/iser/iser_lkdapl.c:788: parse error before `)'
drivers/infiniband/ulp/iser/iser_lkdapl.c:791: parse error before `)'
drivers/infiniband/ulp/iser/iser_lkdapl.c:794: parse error before `)'
drivers/infiniband/ulp/iser/iser_lkdapl.c:797: parse error before `)'
drivers/infiniband/ulp/iser/iser_lkdapl.c:800: parse error before `)'
drivers/infiniband/ulp/iser/iser_lkdapl.c: In function
`iser_handle_single_kdapl_event':
drivers/infiniband/ulp/iser/iser_lkdapl.c:1025: parse error before `)'
x86 + gen2 =m *FAILED*
same as above
Can you try this patch and see if it eliminates the iser errors ?
Thanks.
-- Hal
Signed-off-by: Hal Rosenstock [EMAIL PROTECTED]
Index: iser.h
===
--- iser.h (revision 3691)
+++ iser.h (working copy)
@@ -334,7 +334,7 @@ extern int iser_debug_level;
do {\
if (iser_debug_level 0) \
printk(KERN_DEBUG PFX %s: fmt,\
- __func__, ## arg); \
+ __func__ , ## arg); \
} while (0)
#define iser_err(fmt,
Re: [openib-general] Timeline of IPoIB performance
Hi Matt, On Fri, 2005-10-07 at 04:06, Matt Leininger wrote: I'm seeing an IPoIB netperf performance drop off, up to 90 MB/s, when using kernels newer than 2.6.11. This doesn't appear to be an OpenIB IPoIB issue since the in-kernel and a recent svn3687 snapshot both have the same performance (464 MB/s) with 2.6.11. I used the same kernel config file as a starting point for each of these kernel builds. Have there been any changes in Linux that would explain these results? All benchmarks are with RHEL4 x86_64 with HCA FW v4.7.0 dual EM64T 3.2 GHz PCIe IB HCA (memfull) Kernel OpenIBmsi_x netperf (MB/s) 2.6.14-rc3 in-kernel1 374 2.6.13.2svn3627 1 386 2.6.13.2in-kernel1 394 2.6.12 in-kernel1 406 2.6.11 in-kernel1 464 2.6.11 svn3687 1 464 2.6.9-11.ELsmp svn3513 1 425 (Woody's results, 3.6Ghz EM64T) There was already the following thread on netdev that I found: TCP Network performance degade from 2.4.18 to 2.6.10 http://marc.theaimsgroup.com/?l=linux-netdevm=112792558832125w=2 I think you should (cross)post this to netdev. -- Hal ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Latest build test results
On 07.10.2005 [08:21:19 -0400], Hal Rosenstock wrote: On Thu, 2005-10-06 at 14:11, Nishanth Aravamudan wrote: On 06.10.2005 [13:25:35 -0400], Hal Rosenstock wrote: On Thu, 2005-10-06 at 13:11, Nishanth Aravamudan wrote: On 06.10.2005 [19:40:40 +0300], Dan Bar Dov wrote: I've fixed the 2.6.14-rc3 compilation warnings with iSER on x86 in version 3682. Great! Thanks. I'm re-running the tests (due to a subtle flaw in my PATH, my cronjobs weren't running) now and will post the latest results. You might also want to apply https://openib.org/svn/gen2/trunk/src/linux-kernel/patches/linux-2.6.14-rc3-fib-frontend.diff to get rid of the AT and SDP warnings. I already submitted several jobs for 2.6.14-rc3-git6, but I'll redo the gen2 ones with that patch, thanks. Here are the results from 2.6.14-rc3-git6 + gen2 3683 Looks like x86 is broken in the current svn tree. x86 and ppc64 mainline is fine with both =y and =m ppc64 + gen2 =y drivers/infiniband/ulp/srp/ib_srp.c: In function `srp_process_rsp': drivers/infiniband/ulp/srp/ib_srp.c:650: warning: long long unsigned int format, u64 arg (arg 2) drivers/infiniband/core/at.c:1547: warning: initialization from incompatible pointer type drivers/infiniband/ulp/sdp/sdp_link.c:752: warning: initialization from incompatible pointer type ppc64 + gen2 =m same as above, plus *** Warning: .ip_dev_find [drivers/infiniband/ulp/sdp/ib_sdp.ko] undefined! *** Warning: .ip_dev_find [drivers/infiniband/core/ib_at.ko] undefined! x86 + gen2 =y *FAILED* What gcc version are you using ? I believe the build systems on all the automated machines are 2.95: Reading specs from /usr/lib/gcc-lib/i386-linux/2.95.4/specs gcc version 2.95.4 20011002 (Debian prerelease) drivers/infiniband/ulp/iser/iser_conn.c: In function `iser_adaptor_release': drivers/infiniband/ulp/iser/iser_conn.c:195: parse error before `)' drivers/infiniband/ulp/iser/iser_conn.c:203: parse error before `)' drivers/infiniband/ulp/iser/iser_conn.c:206: parse error before `)' drivers/infiniband/ulp/iser/iser_conn.c: In function `iser_conn_establish': drivers/infiniband/ulp/iser/iser_conn.c:284: parse error before `)' drivers/infiniband/ulp/iser/iser_conn.c: In function `iser_post_receive_control': drivers/infiniband/ulp/iser/iser_conn.c:861: parse error before `)' drivers/infiniband/ulp/iser/iser_conn.c:873: parse error before `)' drivers/infiniband/ulp/iser/iser_initiator.c: In function `iser_reg_rdma_mem': drivers/infiniband/ulp/iser/iser_initiator.c:125: parse error before `)' drivers/infiniband/ulp/iser/iser_initiator.c:130: parse error before `)' drivers/infiniband/ulp/iser/iser_initiator.c:141: parse error before `)' drivers/infiniband/ulp/iser/iser_initiator.c:153: parse error before `)' drivers/infiniband/ulp/iser/iser_mod.c: In function `init_module': drivers/infiniband/ulp/iser/iser_mod.c:154: parse error before `)' drivers/infiniband/ulp/iser/iser_mod.c: In function `cleanup_module': drivers/infiniband/ulp/iser/iser_mod.c:243: parse error before `)' drivers/infiniband/ulp/iser/iser_lkdapl.c: In function `iser_create_ia_pz_evd': drivers/infiniband/ulp/iser/iser_lkdapl.c:147: parse error before `)' drivers/infiniband/ulp/iser/iser_lkdapl.c: In function `iser_consume_events': drivers/infiniband/ulp/iser/iser_lkdapl.c:691: parse error before `)' drivers/infiniband/ulp/iser/iser_lkdapl.c: In function `iser_event_handler_thread': drivers/infiniband/ulp/iser/iser_lkdapl.c:731: parse error before `)' drivers/infiniband/ulp/iser/iser_lkdapl.c:749: parse error before `)' drivers/infiniband/ulp/iser/iser_lkdapl.c: In function `iser_handle_conn_event': drivers/infiniband/ulp/iser/iser_lkdapl.c:776: parse error before `)' drivers/infiniband/ulp/iser/iser_lkdapl.c:779: parse error before `)' drivers/infiniband/ulp/iser/iser_lkdapl.c:782: parse error before `)' drivers/infiniband/ulp/iser/iser_lkdapl.c:785: parse error before `)' drivers/infiniband/ulp/iser/iser_lkdapl.c:788: parse error before `)' drivers/infiniband/ulp/iser/iser_lkdapl.c:791: parse error before `)' drivers/infiniband/ulp/iser/iser_lkdapl.c:794: parse error before `)' drivers/infiniband/ulp/iser/iser_lkdapl.c:797: parse error before `)' drivers/infiniband/ulp/iser/iser_lkdapl.c:800: parse error before `)' drivers/infiniband/ulp/iser/iser_lkdapl.c: In function `iser_handle_single_kdapl_event': drivers/infiniband/ulp/iser/iser_lkdapl.c:1025: parse error before `)' x86 + gen2 =m *FAILED* same as above Can you try this patch and see if it eliminates the iser errors ? Thanks. Will try it in a bit. Thanks, Nish ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit
Re: [openib-general] Latest build test results
On Thu, 2005-10-06 at 15:26, Hal Rosenstock wrote:
On Thu, 2005-10-06 at 15:20, Nishanth Aravamudan wrote:
On 06.10.2005 [13:25:35 -0400], Hal Rosenstock wrote:
On Thu, 2005-10-06 at 13:11, Nishanth Aravamudan wrote:
On 06.10.2005 [19:40:40 +0300], Dan Bar Dov wrote:
I've fixed the 2.6.14-rc3 compilation warnings with iSER on x86 in
version 3682.
Great! Thanks.
I'm re-running the tests (due to a subtle flaw in my PATH, my cronjobs
weren't running) now and will post the latest results.
You might also want to apply
https://openib.org/svn/gen2/trunk/src/linux-kernel/patches/linux-2.6.14-rc3-fib-frontend.diff
to get rid of the AT and SDP warnings.
This patch does remove the warning regarding undefined symbols during
modpost, but does not remove the warnings
drivers/infiniband/core/at.c:1547: warning: initialization from
incompatible pointer type
drivers/infiniband/ulp/sdp/sdp_link.c:752: warning: initialization from
incompatible pointer type
Right. Roland reported a change to struct packet_type in 2.6.14. I'll
work on a patch for this too. Thanks.
Can you try this patch for the above 2 warnings ? If it works, I check
it into the patches directory. Thanks.
-- Hal
Update arp_recv functions to latest 2.6.14 netdevice.h API for struct
packet_type
Signed-off-by: Hal Rosenstock [EMAIL PROTECTED]
Index: core/at.c
===
--- core/at.c (revision 3691)
+++ core/at.c (working copy)
@@ -1258,7 +1258,7 @@ static void ib_at_arp_work(void *data)
}
static int ib_at_arp_recv(struct sk_buff *skb, struct net_device *dev,
-struct packet_type *pt)
+ struct packet_type *pt, struct net_device *orig_dev)
{
struct arp_work *work;
struct arphdr *arp_hdr;
Index: ulp/sdp/sdp_link.c
===
--- ulp/sdp/sdp_link.c (revision 3691)
+++ ulp/sdp/sdp_link.c (working copy)
@@ -716,7 +716,7 @@ done:
* sdp_link_arp_recv - receive all ARP packets
*/
static int sdp_link_arp_recv(struct sk_buff *skb, struct net_device *dev,
-struct packet_type *pt)
+struct packet_type *pt, struct net_device
*orig_dev)
{
struct sdp_work *work;
struct arphdr *arp_hdr;
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] IBM eHCA testing..
I have two IBM eHCA cards installed and it appears that OpenSM is happily talking to the firmware and bringing up the links. So now I'm looking at the install instructions for the ehca2_EHCA2_0025.tgz code drop, and wondering what (if any) issues there are with a 2.6.13 kernel, or later OpenIB svn drops. Is there a later code drop I can get ahold of? Is the nr_ports issue something in the driver? I wound up connecting to the lower port in the Openpower720 machine.. do you know if that's port 1 or 2? ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Latest build test results
On 07.10.2005 [09:48:56 -0400], Hal Rosenstock wrote: On Thu, 2005-10-06 at 15:26, Hal Rosenstock wrote: On Thu, 2005-10-06 at 15:20, Nishanth Aravamudan wrote: On 06.10.2005 [13:25:35 -0400], Hal Rosenstock wrote: On Thu, 2005-10-06 at 13:11, Nishanth Aravamudan wrote: On 06.10.2005 [19:40:40 +0300], Dan Bar Dov wrote: I've fixed the 2.6.14-rc3 compilation warnings with iSER on x86 in version 3682. Great! Thanks. I'm re-running the tests (due to a subtle flaw in my PATH, my cronjobs weren't running) now and will post the latest results. You might also want to apply https://openib.org/svn/gen2/trunk/src/linux-kernel/patches/linux-2.6.14-rc3-fib-frontend.diff to get rid of the AT and SDP warnings. This patch does remove the warning regarding undefined symbols during modpost, but does not remove the warnings drivers/infiniband/core/at.c:1547: warning: initialization from incompatible pointer type drivers/infiniband/ulp/sdp/sdp_link.c:752: warning: initialization from incompatible pointer type Right. Roland reported a change to struct packet_type in 2.6.14. I'll work on a patch for this too. Thanks. Can you try this patch for the above 2 warnings ? If it works, I check it into the patches directory. Thanks. Will try this along with the other patch you sent after I return from class (about 2 hours). Thanks, Nish ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [MailServer Notification]To Recipient virus found and action taken.
ScanMail for Microsoft Exchange has detected virus-infected attachment(s). Sender = [EMAIL PROTECTED] Recipient(s) = openib-general@openib.org Subject = [openib-general] Members Support Scanning time = 10/7/2005 8:48:13 AM Engine/Pattern = 7.510-1002/2.879.00 Action on virus found: The attachment ykj.zip contains WORM_MYTOB.EI virus. ScanMail has Deleted it. Warning to recipient. ScanMail has detected a virus. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Timeline of IPoIB performance
Matt, I have seen the same thing. I just didn't relate it to the Kernel. My IPoIB performance is down to ~340MB/sec with 2.6.12.1 and svn 3040. With 2.6.13 and svn 3490 the peak is 402MB/sec. At 02:06 AM 10/7/2005, Matt Leininger wrote: I'm seeing an IPoIB netperf performance drop off, up to 90 MB/s, when using kernels newer than 2.6.11. This doesn't appear to be an OpenIB IPoIB issue since the in-kernel and a recent svn3687 snapshot both have the same performance (464 MB/s) with 2.6.11. I used the same kernel config file as a starting point for each of these kernel builds. Have there been any changes in Linux that would explain these results? All benchmarks are with RHEL4 x86_64 with HCA FW v4.7.0 dual EM64T 3.2 GHz PCIe IB HCA (memfull) Kernel OpenIBmsi_x netperf (MB/s) 2.6.14-rc3 in-kernel1 374 2.6.13.2svn3627 1 386 2.6.13.2in-kernel1 394 2.6.12 in-kernel1 406 2.6.11 in-kernel1 464 2.6.11 svn3687 1 464 2.6.9-11.ELsmp svn3513 1 425 (Woody's results, 3.6Ghz EM64T) Thanks, - Matt ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Timeline of IPoIB performance
Hmm, looks like something in the network stack must have changed. 2.6.12 in-kernel1 406 2.6.11 in-kernel1 464 This looks like the biggest dropoff. I can think of two things that would be interesting to do if you or anyone else has time. First, taking profiles of netperf runs between these two kernels and comparing might be enlightening. Also, it would be useful to pin down when the regression happened, so running the same test with 2.6.12-rc1 through 2.6.12-rc6 would be a good thing. - R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] IBM eHCA testing..
I believe the lower port is port 1. I will defer to the EHCA team as regards to issues with 2.6.13 (if any). We have minimally used both ports on p570. So, my guess is that should work on a Openpower720. Pradeep [EMAIL PROTECTED] [EMAIL PROTECTED] wrote on 10/07/2005 07:12:07 AM: I have two IBM eHCA cards installed and it appears that OpenSM is happily talking to the firmware and bringing up the links. So now I'm looking at the install instructions for the ehca2_EHCA2_0025.tgz code drop, and wondering what (if any) issues there are with a 2.6.13 kernel, or later OpenIB svn drops. Is there a later code drop I can get ahold of? Is the nr_ports issue something in the driver? I wound up connecting to the lower port in the Openpower720 machine.. do you know if that's port 1 or 2? ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] [RFC] IB address translation using ARP
At 06:38 AM 9/30/2005, Caitlin Bestler wrote: -Original Message- From: [EMAIL PROTECTED] [ mailto:[EMAIL PROTECTED]] On Behalf Of Roland Dreier Sent: Thursday, September 29, 2005 6:50 PM To: Sean Hefty Cc: Openib Subject: Re: [openib-general] [RFC] IB address translation using ARP Sean Can you explain how RDMA works in this case? This is simply Sean performing IP routing, and not IB routing, correct? Are you Sean referring to a protocol running on top of IP or IB directly? Sean Is the router establishing a second reliable connection on Sean the backend? Does it simply translate headers as packets Sean pass through in this case? I think the usage model is the following: you have some magic device that has an IB port on one side and something else on the other side. Think of something like a gateway that talks SDP on the IB side and TCP/IP on the other side. You configure your IPoIB routing so that this magic device is the next hop for talking to hosts on the IP network on the other side. Now someone tries to make an SDP connection to an IP address on the other side of the magic device. Routing tables + ARP give it the GID of the IB port of this magic device. It connects to the magic device and run SDP to talk to the magic device, and the magic device magically splices this into a TCP connection to the real destination. Or the same idea for an NFS/RDMA - NFS/UDP gateway, etc. Those examples are all basically application level gateways. As such they would have no transport or connection setup implications. The application level gateway simply offers a service on network X that it fulfills on network Y. But as far as network X is concerned the gateway IS the server. It must be viewed as such. The cross over point between the two domains represents independent management domains, trust domains, reliable delivery domains, etc. I do not believe it is possible to construct a transport layer gateway that bridges RDMA between IB and iWARP while appearing to be a normal RDMA endpoint on both networks. Higher level gateways will be possible for many applications, but I don't see how that relates to connection establishment. That would require having an end-to-end reliable connection, complete with flow control semantics, that bridged the two networks by some method other than encapsulation or tunneling. We took steps to insure that both IB and iWARP could transmit packets in the main data path very efficiently between the two interconnects but it was never envisioned that a connection was truly end-to-end transparent across the gateway component. I think most of the architects would not support such an effort to define such a beast. There are many issues in attempting such an offering. Just examine all of the problems with the existing iSCSI to FC solutions; they ignore a number of customer issues and hence have been relegated in many customer minds as TTM, play toys not ready for prime time. This is one of the many reasons why iSCSI has not taken off as the hype portrayed. It would be best to define a CM architecture that enabled communication between like endpoints and avoid the gateway dilemma. Let the gateway provider work out such issues as there are many requirements already on each side of these interconnects. Mike ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] [RFC] IB address translation using ARP
At 06:24 AM 9/30/2005, Yaron Haviv wrote: -Original Message- From: Roland Dreier [ mailto:[EMAIL PROTECTED]] Sent: Thursday, September 29, 2005 9:50 PM To: Sean Hefty Cc: Yaron Haviv; Openib Subject: Re: [openib-general] [RFC] IB address translation using ARP I think the usage model is the following: you have some magic device that has an IB port on one side and something else on the other side. Think of something like a gateway that talks SDP on the IB side and TCP/IP on the other side. Also applicable to two IB ports, e.g. forwarding SDP traffic from one IB partition to SDP on another partition (may even be the same port with two P_Keys), and doing some load-balancing or traffic management in between, overall there are many use cases for that. While I can envision how an endpoint could communicate with another in separate partitions, doing so really violates the spirit of the partitioning where endpoints must be in the same partition in order to see one another and communicate. Attempting to create an intermediary who has insights into both and then somehow is able to communicate how to find one another using some proprietary (can't be through standards that I can think of) method, seems like way too much complexity to be worth it. Mike ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] IBM eHCA testing..
Hi, Troy, There is INSTALL file in the EHCA driver package. In OpenPower 720 port 1 is at the top, port 2 is at the bottom. In P570, port1 is at the bottom, port2 is at the top. Thanks Shirley Ma IBM Linux Technology Center 15300 SW Koll Parkway Beaverton, OR 97006-6063 Phone(Fax): (503) 578-7638___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] [RFC] IB address translation using ARP
It would be best to define a CM architecture that enabled communication between like endpoints and avoid the gateway dilemma. Let the gateway provider work out such issues as there are many requirements already on each side of these interconnects. I've given this some more thought since the original postings and agree with you. It doesn't seem right to me to have the CM establish a connection to something that is not the specified destination, under the assumption that whatever is being connected to is a gateway. I think it would be better for the application to determine that the actual destination is on a different subnet, locate the gateway, and issue a connection request to the gateway. - Sean ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH] udapl: PPC64 cpuinfo change
On Thu, Oct 06, 2005 at 11:01:21PM -0500, Troy Benjegerdes wrote: Oh boy is there some reason 'gettimeofday' does not work? In general, it doesn't work as well. Trying to infer timebase/clock/rtsc frequency is going to be a mess. Using cycle counters is quite portable today and provides accurate results (with caveats on it's use). I'm open to using the next best thing once it's clear the cycle counters do NOT work. Think cpus that dynamically change frequency.. Laptops do now.. how long before something with infiniband does and breaks this code horribly? (think embedded systems) I don't buy this argument. Most of the tests load the CPU and it essentially runs at a fixed frequency. A better argument is how to benchmark under virtualized environment. I think that is totally broken today regardless of what method one uses to measure time. There are a couple of implementations of gettimeofday fully in userspace that hide the details and still read the high-res hardware counters. Google for 'vDSO gettimeofday'. Well, I'm sure Michael is open to patches on this for userspace/perftest stuff and like wise for James Lentini for uDAPL. grant ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH] [ADDR] address translation module for CMA
The following patch adds a simple IP to IB address translation module
using ARP. It is based off AT and SDP, but kept as simple as possible.
I would like to merge this back into the trunk, and apply other changes
there.
Signed-off-by: Sean Hefty [EMAIL PROTECTED]
Index: include/rdma/ib_addr.h
===
--- include/rdma/ib_addr.h (revision 0)
+++ include/rdma/ib_addr.h (revision 0)
@@ -0,0 +1,72 @@
+/*
+ * Copyright (c) 2005 Voltaire Inc. All rights reserved.
+ * Copyright (c) 2005 Intel Corporation. All rights reserved.
+ *
+ * This Software is licensed under one of the following licenses:
+ *
+ * 1) under the terms of the Common Public License 1.0 a copy of which is
+ *available from the Open Source Initiative, see
+ *http://www.opensource.org/licenses/cpl.php.
+ *
+ * 2) under the terms of the The BSD License a copy of which is
+ *available from the Open Source Initiative, see
+ *http://www.opensource.org/licenses/bsd-license.php.
+ *
+ * 3) under the terms of the GNU General Public License (GPL) Version 2 a
+ *copy of which is available from the Open Source Initiative, see
+ *http://www.opensource.org/licenses/gpl-license.php.
+ *
+ * Licensee has the right to choose one of the above licenses.
+ *
+ * Redistributions of source code must retain the above copyright
+ * notice and one of the license notices.
+ *
+ * Redistributions in binary form must reproduce both the above copyright
+ * notice, one of the license notices in the documentation
+ * and/or other materials provided with the distribution.
+ *
+ */
+
+#if !defined(IB_ADDR_H)
+#define IB_ADDR_H
+
+#include linux/socket.h
+#include rdma/ib_verbs.h
+
+struct ib_addr {
+ union ib_gidsgid;
+ union ib_giddgid;
+ u16 pkey;
+};
+
+/**
+ * ib_translate_addr - Translate a local IP address to an Infiniband GID and
+ * PKey.
+ */
+int ib_translate_addr(struct sockaddr *addr, union ib_gid *gid, u16 *pkey);
+
+/**
+ * ib_resolve_addr - Resolve source and destination IP addresses to
+ * Infiniband network addresses.
+ * @src_addr: An optional source address to use in the resolution. If a
+ * source address is not provided, a usable address will be returned via
+ * the callback.
+ * @dst_addr: The destination address to resolve.
+ * @addr: A reference to a data location that will receive the resolved
+ * addresses. The data location must remain valid until the callback has
+ * been invoked.
+ * @timeout_ms: Amount of time to wait for the address resolution to complete.
+ * @callback: Call invoked once address resolution has completed, timed out,
+ * or been canceled. A status of 0 indicates success.
+ * @context: User-specified context associated with the call.
+ */
+int ib_resolve_addr(struct sockaddr *src_addr, struct sockaddr *dst_addr,
+ struct ib_addr *addr, int timeout_ms,
+ void (*callback)(int status, struct sockaddr *src_addr,
+struct ib_addr *addr, void *context),
+ void *context);
+
+void ib_addr_cancel(struct ib_addr *addr);
+
+#endif /* IB_ADDR_H */
+
Index: core/addr.c
===
--- core/addr.c (revision 0)
+++ core/addr.c (revision 0)
@@ -0,0 +1,351 @@
+/*
+ * Copyright (c) 2005 Voltaire Inc. All rights reserved.
+ * Copyright (c) 2002-2005, Network Appliance, Inc. All rights reserved.
+ * Copyright (c) 1999-2005, Mellanox Technologies, Inc. All rights reserved.
+ * Copyright (c) 2005 Intel Corporation. All rights reserved.
+ *
+ * This Software is licensed under one of the following licenses:
+ *
+ * 1) under the terms of the Common Public License 1.0 a copy of which is
+ *available from the Open Source Initiative, see
+ *http://www.opensource.org/licenses/cpl.php.
+ *
+ * 2) under the terms of the The BSD License a copy of which is
+ *available from the Open Source Initiative, see
+ *http://www.opensource.org/licenses/bsd-license.php.
+ *
+ * 3) under the terms of the GNU General Public License (GPL) Version 2 a
+ *copy of which is available from the Open Source Initiative, see
+ *http://www.opensource.org/licenses/gpl-license.php.
+ *
+ * Licensee has the right to choose one of the above licenses.
+ *
+ * Redistributions of source code must retain the above copyright
+ * notice and one of the license notices.
+ *
+ * Redistributions in binary form must reproduce both the above copyright
+ * notice, one of the license notices in the documentation
+ * and/or other materials provided with the distribution.
+ */
+#include linux/in.h
+#include linux/in6.h
+#include linux/inetdevice.h
+#include linux/workqueue.h
+#include net/arp.h
+#include net/neighbour.h
+#include net/route.h
+#include rdma/ib_addr.h
+
+MODULE_AUTHOR(Sean Hefty);
+MODULE_DESCRIPTION(IB Address Translation);
+MODULE_LICENSE(Dual BSD/GPL);
+
+struct
[openib-general] Questions about mad_test
I am hoping some one will be able to help me out with a few answers saving me some debug time, or having to expend effort on something that is already known. I was trying to execute mad_test and found that it errors out. For some reason it does not like the DR Path that I gave it. 1. I ran ibnetdiscover and got the set of LIDs that I use is DR Path. Is that correct way to go about it? It always errors out with something like: hop 0 != 0 or hop 1 != dev_port 2. Also there is an expectation of there being a device /dev/infiniband/mthca0/ports/1/mad (using all defaults in this case) -is that correct? Any specific major and minor numbers I must use? 3. Anything else that I am missing? I am using this from trunk 3675 on 2.6.13 kernel. Thanks in advance for all the help! Pradeep [EMAIL PROTECTED]___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH] [CMA] RDMA CM abstraction module
The following patch adds in a basic RDMA connection management abstraction.
It is functional, but needs additional work for handling device removal, plus
several missing features.
I'd like to merge this back into the trunk, and continue working on it from
there.
This depends on the ib_addr module.
Signed-off-by: Sean Hefty [EMAIL PROTECTED]
Index: include/rdma/rdma_cm.h
===
--- include/rdma/rdma_cm.h (revision 0)
+++ include/rdma/rdma_cm.h (revision 0)
@@ -0,0 +1,201 @@
+/*
+ * Copyright (c) 2005 Voltaire Inc. All rights reserved.
+ * Copyright (c) 2005 Intel Corporation. All rights reserved.
+ *
+ * This Software is licensed under one of the following licenses:
+ *
+ * 1) under the terms of the Common Public License 1.0 a copy of which is
+ *available from the Open Source Initiative, see
+ *http://www.opensource.org/licenses/cpl.php.
+ *
+ * 2) under the terms of the The BSD License a copy of which is
+ *available from the Open Source Initiative, see
+ *http://www.opensource.org/licenses/bsd-license.php.
+ *
+ * 3) under the terms of the GNU General Public License (GPL) Version 2 a
+ *copy of which is available from the Open Source Initiative, see
+ *http://www.opensource.org/licenses/gpl-license.php.
+ *
+ * Licensee has the right to choose one of the above licenses.
+ *
+ * Redistributions of source code must retain the above copyright
+ * notice and one of the license notices.
+ *
+ * Redistributions in binary form must reproduce both the above copyright
+ * notice, one of the license notices in the documentation
+ * and/or other materials provided with the distribution.
+ *
+ */
+
+#if !defined(RDMA_CM_H)
+#define RDMA_CM_H
+
+#include linux/socket.h
+#include rdma/ib_addr.h
+#include rdma/ib_sa.h
+
+/*
+ * Upon receiving a device removal event, users must destroy the associated
+ * RDMA identifier and release all resources allocated with the device.
+ */
+enum rdma_event_type {
+ RDMA_EVENT_ADDR_RESOLVED,
+ RDMA_EVENT_ADDR_ERROR,
+ RDMA_EVENT_ROUTE_RESOLVED,
+ RDMA_EVENT_ROUTE_ERROR,
+ RDMA_EVENT_CONNECT_REQUEST,
+ RDMA_EVENT_CONNECT_ERROR,
+ RDMA_EVENT_UNREACHABLE,
+ RDMA_EVENT_REJECTED,
+ RDMA_EVENT_ESTABLISHED,
+ RDMA_EVENT_DISCONNECTED,
+ RDMA_EVENT_DEVICE_REMOVAL,
+};
+
+struct rdma_addr {
+ struct sockaddr src_addr;
+ struct sockaddr dst_addr;
+ union {
+ struct ib_addr ibaddr;
+ } addr;
+};
+
+struct rdma_route {
+ struct rdma_addr addr;
+ struct ib_sa_path_rec *path_rec;
+ int num_paths;
+};
+
+struct rdma_event {
+ enum rdma_event_type event;
+ int status;
+ void*private_data;
+ u8 private_data_len;
+};
+
+struct rdma_id;
+
+/**
+ * rdma_event_handler - Callback used to report user events.
+ *
+ * Notes: Users may not call rdma_destroy_id from this callback to destroy
+ * the passed in id, or a corresponding listen id. Returning a
+ * non-zero value from the callback will destroy the corresponding id.
+ */
+typedef int (*rdma_event_handler)(struct rdma_id *id, struct rdma_event
*event);
+
+struct rdma_id {
+ struct ib_device*device;
+ void*context;
+ struct ib_qp*qp;
+ rdma_event_handler event_handler;
+ struct rdma_routeroute;
+};
+
+struct rdma_id* rdma_create_id(rdma_event_handler event_handler, void
*context);
+
+void rdma_destroy_id(struct rdma_id *id);
+
+/**
+ * rdma_bind_addr - Bind an RDMA identifier to a source address and
+ * associated RDMA device, if needed.
+ *
+ * @id: RDMA identifier.
+ * @addr: Local address information. Wildcard values are permitted.
+ *
+ * This associates a source address with the RDMA identifier before calling
+ * rdma_listen. If a specific local address is given, the RDMA identifier will
+ * be bound to a local RDMA device.
+ */
+int rdma_bind_addr(struct rdma_id *id, struct sockaddr *addr);
+
+/**
+ * rdma_resolve_addr - Resolve destination and optional source addresses
+ * from IP addresses to an RDMA address. If successful, the specified
+ * rdma_id will be bound to a local device.
+ *
+ * @id: RDMA identifier.
+ * @src_addr: Source address information. This parameter may be NULL.
+ * @dst_addr: Destination address information.
+ * @timeout_ms: Time to wait for resolution to complete.
+ */
+int rdma_resolve_addr(struct rdma_id *id, struct sockaddr *src_addr,
+ struct sockaddr *dst_addr, int timeout_ms);
+
+/**
+ * rdma_resolve_route - Resolve the RDMA address bound to the RDMA identifier
+ * into route information needed to establish a connection.
+ *
+ * This is called on the client side of a connection, but its use is optional.
+ * Users must have first called rdma_bind_addr to resolve a dst_addr
+ * into an RDMA
RE: [openib-general] [RFC] IB address translation using ARP
From: Michael Krause [mailto:[EMAIL PROTECTED] Sent: Friday, October 07, 2005 12:29 PM To: Yaron Haviv Cc: Openib Subject: RE: [openib-general] [RFC] IB address translation using ARP At 06:24 AM 9/30/2005, Yaron Haviv wrote: -Original Message- From: Roland Dreier [ mailto:[EMAIL PROTECTED] Sent: Thursday, September 29, 2005 9:50 PM To: Sean Hefty Cc: Yaron Haviv; Openib Subject: Re: [openib-general] [RFC] IB address translation using ARP I think the usage model is the following: you have some magic device that has an IB port on one side and something else on the other side. Think of something like a gateway that talks SDP on the IB side and TCP/IP on the other side. Also applicable to two IB ports, e.g. forwarding SDP traffic from one IB partition to SDP on another partition (may even be the same port with two P_Keys), and doing some load-balancing or traffic management in between, overall there are many use cases for that. While I can envision how an endpoint could communicate with another in separate partitions, doing so really violates the spirit of the partitioning where endpoints must be in the same partition in order to see one another and communicate. Mike, This is exactly the same case as two IPoIB interfaces over same port with two partitions configured with IP routing between them, or a layer 7 proxy that connects two network segments I don’t see anything wrong with such a model Attempting to create an intermediary who has insights into both and then somehow is able to communicate how to find one another using some proprietary (can't be through standards that I can think of) method, seems like way too much complexity to be worth it. Assuming the ULPs on both sides are standards, how the proxy is built and how it functions is application dependent just like people do proxies for XML which don’t need to obey to any standard beside be transparent to both sides. OpenIB should not block the ability to provide gateway/proxy functionality, or routing traffic beyond a single IP addressing hop. This is just matching IB to capabilities already available in iWarp. Yaron ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] [RFC] IB address translation using ARP
-Original Message- From: [EMAIL PROTECTED] [mailto:openib-general- [EMAIL PROTECTED] On Behalf Of Sean Hefty Sent: Friday, October 07, 2005 12:40 PM To: 'Michael Krause'; Caitlin Bestler Cc: Openib Subject: RE: [openib-general] [RFC] IB address translation using ARP It would be best to define a CM architecture that enabled communication between like endpoints and avoid the gateway dilemma. Let the gateway provider work out such issues as there are many requirements already on each side of these interconnects. I've given this some more thought since the original postings and agree with you. It doesn't seem right to me to have the CM establish a connection to something that is not the specified destination, under the assumption that whatever is being connected to is a gateway. I think it would be better for the application to determine that the actual destination is on a different subnet, locate the gateway, and issue a connection request to the gateway. - Sean Sean, I believe this is exactly how it is been proposed The gateway is the endpoint in IB, and the IB CM request is done against the gateway, the gateway may decide to create its own connection on the other side based on IB headers or Private data or even application data (depend on the type of the gateway), this just requires that traffic targeted to a certain IP range/subnet/non-local will end up in the gateway without the need to specify address by address individually (just like its done in IP) Yaron ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib- general ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [RFC] IB address translation using ARP
Yaron Haviv wrote: Sean, I believe this is exactly how it is been proposed The gateway is the endpoint in IB, and the IB CM request is done against the gateway, the gateway may decide to create its own connection on the Yes - I agree with that. I'm referring to the RDMA connection manager, versus the IB connection manager. targeted to a certain IP range/subnet/non-local will end up in the gateway without the need to specify address by address individually (just like its done in IP) IP is connectionless, so I'm not sure how to relate from IP to the RDMA CM. With TCP, the connection is to the actual endpoint, not the IP router. This seems more similar to an application requesting a connection to a proxy server. - Sean ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] Re: Questions about mad_test
Hi Pradeep, On Fri, 2005-10-07 at 15:19, Pradeep Satyanarayana wrote: I am hoping some one will be able to help me out with a few answers saving me some debug time, or having to expend effort on something that is already known. I was trying to execute mad_test and found that it errors out. What is your command invocation ? Can you send the output of ibnetdiscover ? For some reason it does not like the DR Path that I gave it. 1. I ran ibnetdiscover and got the set of LIDs that I use is DR Path. Is that correct way to go about it? It always errors out with something like: hop 0 != 0 or hop 1 != dev_port It's telling you the DR path you specified is invalid. LIDs go direct and are hardware forwarded (via LID routing). DR is uses a list of next hop (switch) ports (and not LIDs) and is firmware or software forwarded usually although that is more an implementation than architectural. See IBA 1.2 14.2.2 p.797 on for more on DR SMPs (MADs). 2. Also there is an expectation of there being a device /dev/infiniband/mthca0/ports/1/mad (using all defaults in this case) -is that correct? Any specific major and minor numbers I must use? No. It just accesses those and some /sys/class/infiniband infiniband_mad files. 3. Anything else that I am missing? I am using this from trunk 3675 on 2.6.13 kernel. Thanks in advance for all the help! Pradeep [EMAIL PROTECTED] ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [RFC] IB address translation using ARP
On Fri, 2005-10-07 at 16:10, Sean Hefty wrote: Yaron Haviv wrote: Sean, I believe this is exactly how it is been proposed The gateway is the endpoint in IB, and the IB CM request is done against the gateway, the gateway may decide to create its own connection on the Yes - I agree with that. I'm referring to the RDMA connection manager, versus the IB connection manager. targeted to a certain IP range/subnet/non-local will end up in the gateway without the need to specify address by address individually (just like its done in IP) IP is connectionless, so I'm not sure how to relate from IP to the RDMA CM. IP is connectionless but has been implemented on top of connection oriented link layers which may gateway to other connection oriented link layers or non connection oriented link layers. I think it is analagous to that. -- Hal With TCP, the connection is to the actual endpoint, not the IP router. This seems more similar to an application requesting a connection to a proxy server. - Sean ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [RFC] IB address translation using ARP
Hal Rosenstock wrote: IP is connectionless, so I'm not sure how to relate from IP to the RDMA CM. IP is connectionless but has been implemented on top of connection oriented link layers which may gateway to other connection oriented link layers or non connection oriented link layers. I think it is analagous to that. I didn't think that IP was even being run in this case. Aren't we talking about an application level gateway? If the RDMA CM ran a protocol that ensured that data sent from the source reached the actual destination, then this would make more sense to me. But the protocol is coming from the client. I just don't think that the RDMA CM should connect to a gateway under the assumption that a client is running a protocol that operates this way. If the source and destination were both running iWarp, then wouldn't a connection be established to the actual destination, and not a gateway? - Sean ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [RFC] IB address translation using ARP
On Fri, 2005-10-07 at 17:02, Sean Hefty wrote: Hal Rosenstock wrote: IP is connectionless, so I'm not sure how to relate from IP to the RDMA CM. IP is connectionless but has been implemented on top of connection oriented link layers which may gateway to other connection oriented link layers or non connection oriented link layers. I think it is analagous to that. I didn't think that IP was even being run in this case. Aren't we talking about an application level gateway? Yes. If the RDMA CM ran a protocol that ensured that data sent from the source reached the actual destination, then this would make more sense to me. But the protocol is coming from the client. Wouldn't the gateway/host reject or drop the connection if it couldn't do what was required ? I just don't think that the RDMA CM should connect to a gateway under the assumption that a client is running a protocol that operates this way. If the source and destination were both running iWarp, then wouldn't a connection be established to the actual destination, and not a gateway? Would it shortcut the connection across IP subnets or go through a gateway in that case ? -- Hal ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [RFC] IB address translation using ARP
Hal Rosenstock wrote: If the RDMA CM ran a protocol that ensured that data sent from the source reached the actual destination, then this would make more sense to me. But the protocol is coming from the client. Wouldn't the gateway/host reject or drop the connection if it couldn't do what was required ? I would assume so, and maybe that's sufficient. The one problem that I see if this feature weren't in the RDMA CM is that clients may need to be transport aware. (Assuming that an iWarp connection would go directly to the destination.) - Sean ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [RFC] IB address translation using ARP
On Fri, 2005-10-07 at 17:30, Sean Hefty wrote: Hal Rosenstock wrote: If the RDMA CM ran a protocol that ensured that data sent from the source reached the actual destination, then this would make more sense to me. But the protocol is coming from the client. Wouldn't the gateway/host reject or drop the connection if it couldn't do what was required ? I would assume so, and maybe that's sufficient. The one problem that I see if this feature weren't in the RDMA CM is that clients may need to be transport aware. (Assuming that an iWarp connection would go directly to the destination.) Would an iWARP connection jump across IP subnets ? It would need to determine that it could do this (ala NHRP with ATM). Also, could there be other RDMA networks between them (like IB) ? -- Hal ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [RFC] IB address translation using ARP
Hal Rosenstock wrote: Would an iWARP connection jump across IP subnets ? It would need to determine that it could do this (ala NHRP with ATM). Also, could there be other RDMA networks between them (like IB) ? if iWarp is on top of TCP, I don't think that it would care about IP subnets. - Sean ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [RFC] IB address translation using ARP
On Fri, 2005-10-07 at 19:57, Sean Hefty wrote: Hal Rosenstock wrote: Would an iWARP connection jump across IP subnets ? It would need to determine that it could do this (ala NHRP with ATM). Also, could there be other RDMA networks between them (like IB) ? if iWarp is on top of TCP, I don't think that it would care about IP subnets. I think iWARP can be on top of TCP or SCTP. But why wouldn't it care ? Doesn't a routing decision still need to be made at the IP layer ? Doesn't the IP next hop need to be determined (e.g. gateway when the destination is off the local IP subnet) ? Is there something that precludes iWARP from working across IP subnets ? -- Hal ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Timeline of IPoIB performance
I wonder if this BIC bug has anything to do with it: http://lkml.org/lkml/2005/10/7/230 ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] IBM eHCA testing..
On Fri, Oct 07, 2005 at 09:33:27AM -0700, Shirley Ma wrote: Hi, Troy, There is INSTALL file in the EHCA driver package. In OpenPower 720 port 1 is at the top, port 2 is at the bottom. In P570, port1 is at the bottom, port2 is at the top. Okay, I guess I should read more carefully ;) What is the issue with needing to use port 1? Can that be fixed in the driver, or does that need a firmware update? ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Timeline of IPoIB performance
I'm adding netdev to this thread to see if they can help. I'm seeing an IPoIB (IP over InfiniBand) netperf performance drop off, of up to 90 MB/s, when using kernels newer than 2.6.11. This doesn't appear to be an OpenIB IPoIB issue since the older in-kernel IB for 2.6.11 and a recent svn3687 snapshot both have the same performance (464 MB/s) with 2.6.11. I used the same kernel config file as a starting point for each of these kernel builds. Have there been any changes in Linux that would explain these results? Here is the hardware setup and netperf results using 'netperf -f -M -c -C -H IPoIB_ADDRESS All benchmarks are with RHEL4 x86_64 with HCA FW v4.7.0 dual EM64T 3.2 GHz PCIe IB HCA (memfull) Kernel OpenIBmsi_x netperf (MB/s) 2.6.14-rc3 in-kernel1 374 2.6.13.2svn3627 1 386 2.6.13.2in-kernel1 394 2.6.12.5-lustre in-kernel1 399 2.6.12.5in-kernel1 402 2.6.12 in-kernel1 406 2.6.12-rc6 in-kernel1 407 2.6.12-rc5 in-kernel1 405 2.6.12-rc4 in-kernel1 470 2.6.12-rc3 in-kernel1 466 2.6.12-rc2 in-kernel1 469 2.6.12-rc1 in-kernel1 466 2.6.11 in-kernel1 464 2.6.11 svn3687 1 464 2.6.9-11.ELsmp svn3513 1 425 (Woody's results, 3.6Ghz EM64T) Thanks, - Matt ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] fu-ka.jpg
$B!!(B $B!!:NiCW$7$^$9!#!j$J%a!%k$GIT2w$G$7$?$i?=$7LuM-$j$^$;$s!#(B $B!!5.J}$KD@\!L$*M6$$%a!%kMzNr!M$,0l7oJ]N1Cf$K$J$C$F$*$j$^$9!#(B $B%3%A%i$+$i%a!%kFbMF$r%3%T!$7$FG[?.$9$k;v$HCW$7$^$7$F!$43N(B $BG'$Ne!JV;v!!(Bhttp://www.alladdin-master.com?return1 $B$r$*4j$$CW$7$^$9!#(B $B#Iw9a(B $B$5$s#(B $BK\J8(B: $B!V$O$8$a$^$7$F(B^^$B6a=j$NJ}$rC5$7$F$F!$$J$?$rR2p$5$l$?$N$GJV(B $B;v$r=P$7$F$_$^$7$?!#2qR$r-$a2H;vjEA$$$r$7$F#3%v7n$,N)$A$^(B $B$7$?!#$O$C$-$j$$$C$F!d$7$$$G$9!#+J,$O6/[EMAIL PROTECTED];W$C$F$$(B $B$^$7$?$,!Bg4V0c$$$G$7$?!#(B $B7k:'$H$+$=3$H$O!A4A39M$($F(B $B$$$^$;$s!#(B [EMAIL PROTECTED]$3$Nd$7$5$rK:$l$5$;$FM_$7$/$FEPO?$7$^$7$?!#(B $B;d+?H7P:QE*$K$OMM5$,$$j$^$9$N$G!(B $B$$kDxEY(B(20$BK|0L$+$J!!!(B $BP(B)$B$ONO$K$J$C$F$$2$k$3$H$,$G$-$k$H;W$$$^$9!#$G$-$l$PAa4|$,(B $B$N$G!D@\%a!%k$G$-$^$;$s$+!)(B $B!!;d$N%%I%l%9$O(Bfu-ka*e*cco@ hotmail.com$B59$7$/!JV;vBT$C$F$^(B $B$9$Mv!W(B $B(%W%i%$%P%7!J]8n$N0Y!K\?M$N%a%C%;!%8$K$F!Z(B*$B![$N3NG'$r$*(B $B4j$$$7$^$9!#(B [EMAIL PROTECTED]pJs$r3NG'$9$k$K$O%3%A%i$N%Z!%8$K$F4JC1$JjB3$-(B ($BL5NA(B)$B$r$*4j$$$7$^$9!#$=$N8e!4JC1$J-dG'Z$r:Q$^$;$k$H!99$K(B $B!o(B10,000$B1_(B($BAjEv%]%$%s%H(B)$B$^$G40A4L5NA$G$*;[EMAIL PROTECTED](B $B!o(B0$B1_$G$J$s$H!!!(B $B!!(Bhttp://www.alladdin-master.com?return1 $B(2q0w$NJ}$OF~2q(B24$B;[EMAIL PROTECTED]@\OMm$NL5$+$C$?(B $Bl9g$Or7oL5$/[EMAIL PROTECTED];$FD:$-$^$9!#(B $B($3$N%a!%k$r3+Iu$7$F(B2$B;~4V0JFb$KEPO?$5$l$k$H$B$r%W%l%%s%HCW$7$^$9!#(B $B$*j?t$G$9$,G[?.5qH]$NJ}$O%3%A%i$^$G(B [EMAIL PROTECTED] Please inform the following address if this mail is unnecessary. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
