[openib-general] $B2q$C$F$/$l$l$P:9$7>e$2$^$9!#(B
$B!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g(B $B!c(BNO.I don't veceive your mail$B!d"M!!([EMAIL PROTECTED] $B!c:#8e!"l9g$O!d"M!!([EMAIL PROTECTED] $B!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g(B ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] $BO"[EMAIL PROTECTED]"v(B
$B0l=o$K5o$F$/[EMAIL PROTECTED]http://www.00-love5.com/?f $BA4$FL5NA$G;H$($k$N$G$*;n$72<$5$$(B /_/_/_/_/_/_/_/_/_/_/_/_/_/ I don't veceive your mail [EMAIL PROTECTED] $B%a!<%kITMW(B [EMAIL PROTECTED] /_/_/_/_/_/_/_/_/_/_/_/_/_/ ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] uDAPL open HCA problem
LEI> Any idea what is going on? Nope, sorry. Looks like something is going wrong inside ib_at. - R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] uDAPL open HCA problem
Hi, Thank you very much for your reply. Now the open HCA problem comes back :-( Here is the log message: [EMAIL PROTECTED] mpiexec -n 2 ./a.out DAPL: NOT Setting Loopback dapl_ib_init: ib_thread_init(12016) dapl_ia_open (ib0, 8, 0x7ff28668, 0xd9da48) open_hca: mthca0 - 0xdb3390 ib_thread(12016,0x40200960): ENTER: pipe 8 at 4 open_hca: Found dev mthca0 0002c902004002e8 open_hca: GID subnet fe80 id 0002c902004002e9 ips_by_gid: RET 0 at_rec 0x7ff283d0 -> id 2861 dapli_at_event_cb() ip_comp_handler: rec 0x7ff283d0 ->id 2861 id 2861 num -22 3afa6000 ip_comp_handler: resolution err -22 retry 1 ip_comp_handler: ips_by_gid 0 rec 0x7ff283d0->id 2862 dapli_at_event_cb() ip_comp_handler: rec 0x7ff283d0 ->id 2862 id 2862 num -22 0 ip_comp_handler: resolution err -22 retry 2 ip_comp_handler: ips_by_gid 0 rec 0x7ff283d0->id 2863 dapli_at_event_cb() ip_comp_handler: rec 0x7ff283d0 ->id 2863 id 2863 num -22 0 ip_comp_handler: resolution err -22 retry 3 ip_comp_handler: ips_by_gid 0 rec 0x7ff283d0->id 2864 dapli_at_event_cb() ip_comp_handler: rec 0x7ff283d0 ->id 2864 id 2864 num -22 0 ip_comp_handler: resolution err -22 retry 4 ip_comp_handler: ERR: at_rec 0x7ff283d0, id 2864 num -22 open_hca: ERR ib_at_ips_by_gid for mthca0 dapls_ib_open_hca failed 4 dapl_ia_open () returns 0x4 DAPL: Stopped (dapl_fini) dapl_ib_release: ib_thread_destroy(12016) ib_thread_destroy: waiting for ib_thread ib_thread(12016) EXIT [rdma_udapl_priv.c:640] error(262144): Cannot open IA DAPL: NOT Setting Loopback dapl_ib_init: ib_thread_init(11337) dapl_ia_open (ib0, 8, 0x7fa8d618, 0xd9da48) open_hca: mthca0 - 0xdb3390 ib_thread(11337,0x40800960): ENTER: pipe 8 at 4 open_hca: Found dev mthca0 0002c90200400314 open_hca: GID subnet fe80 id 0002c90200400315 ips_by_gid: RET 0 at_rec 0x7fa8d380 -> id 4627 dapli_at_event_cb() ip_comp_handler: rec 0x7fa8d380 ->id 4627 id 4627 num -22 3c66c000 ip_comp_handler: resolution err -22 retry 1 ip_comp_handler: ips_by_gid 0 rec 0x7fa8d380->id 4628 dapli_at_event_cb() ip_comp_handler: rec 0x7fa8d380 ->id 4628 id 4628 num -22 0 ip_comp_handler: resolution err -22 retry 2 [rdma_udapl_priv.c:640] error(262144): Cannot open IA ip_comp_handler: ips_by_gid 0 rec 0x7fa8d380->id 4629 dapli_at_event_cb() ip_comp_handler: rec 0x7fa8d380 ->id 4629 id 4629 num -22 0 ip_comp_handler: resolution err -22 retry 3 ip_comp_handler: ips_by_gid 0 rec 0x7fa8d380->id 4630 dapli_at_event_cb() ip_comp_handler: rec 0x7fa8d380 ->id 4630 id 4630 num -22 0 ip_comp_handler: resolution err -22 retry 4 ip_comp_handler: ERR: at_rec 0x7fa8d380, id 4630 num -22 open_hca: ERR ib_at_ips_by_gid for mthca0 dapls_ib_open_hca failed 4 dapl_ia_open () returns 0x4 DAPL: Stopped (dapl_fini) dapl_ib_release: ib_thread_destroy(11337) ib_thread_destroy: waiting for ib_thread ib_thread(11337) EXIT ib_thread_destroy(12016) exit rank 0 in job 421 ro0_33361 caused collective abort of all ranks exit status of rank 0: return code 1 Any idea what is going on? Thanks. Lei - Original Message - From: Roland Dreier <[EMAIL PROTECTED]> Date: Friday, October 21, 2005 7:48 pm Subject: Re: [openib-general] uDAPL open HCA problem >LEI> Hi, I'm from the same lab as Sayantan. Thanks for your >LEI> suggestion. Currently we could not reproduce the problem, >LEI> however, we meet another problem. When I try to tear > down a >LEI> connection between two nodes I often get some messages like >LEI> this: > >LEI> [ 0] 005e0406 [ 4] [ 8] [ c] >LEI> [10] 05f9 [14] [18] 0008 [1c] fe10 > > That's OK, it's just showing that you polled a "work request flushed" > status from a completion queue. The latest version of libmthca should > no longer print these messages. > > - R. > ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] uDAPL open HCA problem
LEI> Hi, I'm from the same lab as Sayantan. Thanks for your LEI> suggestion. Currently we could not reproduce the problem, LEI> however, we meet another problem. When I try to tear down a LEI> connection between two nodes I often get some messages like LEI> this: LEI> [ 0] 005e0406 [ 4] [ 8] [ c] LEI> [10] 05f9 [14] [18] 0008 [1c] fe10 That's OK, it's just showing that you polled a "work request flushed" status from a completion queue. The latest version of libmthca should no longer print these messages. - R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] uDAPL open HCA problem
Hi, I'm from the same lab as Sayantan. Thanks for your suggestion. Currently we could not reproduce the problem, however, we meet another problem. When I try to tear down a connection between two nodes I often get some messages like this: [ 0] 005e0406 [ 4] [ 8] [ c] [10] 05f9 [14] [18] 0008 [1c] fe10 The program can run and exit though. After using the debug option as you suggested I got the following log. It starts from the point where I start to free the resources and disconnect the nodes: dapl_lmr_free (0x76f3b0) dapl_lmr_free (0x76f4e0) dapl_lmr_free (0x76f650) dapli_cq_event_cb(0x5c40c0) dapli_cm_event() dapli_cm_event: EVENT=0x7 ID=0x76fa70 CTX=0x76fb00 passive_cb: conn 0x76fb00 id 7797360 event 7 dapli_async_event_cb(0x5c40c0) dapl_lmr_free (0x76fee0) dapl_lmr_free (0x7a9150) dapl_lmr_free (0x7a9280) dapl_lmr_free (0x7a93b0) dapl_lmr_free (0x7a94e0) dapl_lmr_free (0x7a9610) dapl_lmr_free (0x7a9740) dapl_lmr_free (0x7a9870) dapl_lmr_free (0x7a99a0) dapl_lmr_free (0x7a9ad0) dapl_ep_disconnect (0x69b070, 1) disconnect(ep 0x69b070, conn 0x76f7a0, id 7797184 flags 1) dapl_ep_disconnect () returns 0x0 dapli_cq_event_cb(0x5c4410) dapli_cm_event() dapli_cm_event: EVENT=0x8 ID=0x76f9c0 CTX=0x76f7a0 active_cb: conn 0x76f7a0 id 7797184 event 8 dapli_async_event_cb(0x5c4410) dapl_evd_wait (0x5c89b0, -1, 1, 0x7febf7c0, 0x7febf7bc) dapl_evd_wait: EVD 0x5c89b0, CQ (nil) dapl_evd_wait (0x5c89b0, -1, 1, 0x7f9a9b50, 0x7f9a9b4c) dapl_evd_wait: EVD 0x5c89b0, CQ (nil) dapli_cq_event_cb(0x5c4410) dapli_cm_event() dapli_cm_event: EVENT=0x9 ID=0x76f9c0 CTX=0x76f7a0 active_cb: conn 0x76f7a0 id 7797184 event 9 --> dapl_evd_connection_callback: ctxt: 0x69b070 event: 1 cm_handle 0x76f7a0 dapls_ib_get_dat_event: event(passive) ib=0x1 dat=0x4005 disconnect(ep 0x69b070, conn 0x76f7a0, id 7797184 flags 0) destroy_cm_id: conn 0x76f7a0 id 7797184 modify_qp: qp 0x69b3a0, state 6 qp_num 0x2c0406 dapli_evd_post_event: Called with event # 4005 dapl_evd_connection_callback () returns active_cb: DESTROY conn 0x76f7a0 id 7797184 dapli_async_event_cb(0x5c4410) dapl_evd_wait () returns 0x0 dapli_cq_event_cb(0x5c40c0) dapli_cm_event() dapli_cm_event: EVENT=0x9 ID=0x76fa70 CTX=0x76fb00 passive_cb: conn 0x76fb00 id 7797360 event 9 --> dapl_cr_callback! context: 0x5c8b20 event: 1 cm_handle 0x76fb00 dapls_ib_get_dat_event: event(passive) ib=0x1 dat=0x4005 disconnect(ep 0x69b070, conn 0x76fb00, id 7797360 flags 0) destroy_cm_id: conn 0x76fb00 id 7797360 modify_qp: qp 0x69b3a0, state 6 qp_num 0x4e0406 dapli_evd_post_event: Called with event # 4005 dapl_evd_wait () returns 0x0 dapli_async_event_cb(0x5c40c0) dapli_cq_event_cb(0x5c40c0) dapli_cm_event() dapli_cm_event: EVENT=0x7 ID=0x76f910 CTX=0x7a9120 passive_cb: conn 0x7a9120 id 7797008 event 7 dapli_async_event_cb(0x5c40c0) dapl_ep_disconnect (0x69bd20, 1) disconnect(ep 0x69bd20, conn 0x76fa50, id 7797872 flags 1) dapl_ep_disconnect () returns 0x0 dapl_evd_wait (0x5ccb00, -1, 1, 0x7febf7c0, 0x7febf7b8) dapl_evd_wait: EVD 0x5ccb00, CQ (nil) dapli_cq_event_cb(0x5c4410) dapli_cm_event() dapli_cm_event: EVENT=0x8 ID=0x76fc70 CTX=0x76fa50 active_cb: conn 0x76fa50 id 7797872 event 8 dapli_async_event_cb(0x5c4410) dapl_evd_wait (0x5ccb00, -1, 1, 0x7f9a9b50, 0x7f9a9b48) dapl_evd_wait: EVD 0x5ccb00, CQ (nil) dapli_cq_event_cb(0x5c4410) dapli_cm_event() dapli_cm_event: EVENT=0x9 ID=0x76fc70 CTX=0x76fa50 active_cb: conn 0x76fa50 id 7797872 event 9 --> dapl_evd_connection_callback: ctxt: 0x69bd20 event: 1 cm_handle 0x76fa50 dapli_cq_event_cb(0x5c40c0) dapli_cm_event() dapli_cm_event: EVENT=0x9 ID=0x76f910 CTX=0x7a9120 passive_cb: conn 0x7a9120 id 7797008 event 9 --> dapl_cr_callback! context: 0x5ccc70 event: 1 cm_handle 0x7a9120 dapls_ib_get_dat_event: event(passive) ib=0x1 dat=0x4005 disconnect(ep 0x69bd20, conn 0x7a9120, id 7797008 flags 0) destroy_cm_id: conn 0x7a9120 id 7797008 modify_qp: qp 0x76f220, state 6 qp_num 0x4e0407 dapli_evd_post_event: Called with event # 4005 dapl_evd_wait () returns 0x0 dapl_ep_free (0x69b070) dapl_ep_disconnect (0x69b070, 0) dapl_ep_disconnect () returns 0x0 dapl_ep_free: Free EP: b, ep 0x69b070 qp_state 1 qp_handle 69b3a0 qp_free: ep_ptr 0x69b070 qp 0x69b3a0 modify_qp: qp 0x69b3a0, state 6 qp_num 0x4e0406 dapli_async_event_cb(0x5c40c0) dapls_ib_get_dat_event: event(passive) ib=0x1 dat=0x4005 disconnect(ep 0x69bd20, conn 0x76fa50, id 7797872 flags 0) destroy_cm_id: conn 0x76fa50 id 7797872 modify_qp: qp 0x76f220, state 6 qp_num 0x2c0407 dapli_evd_post_event: Called with event # 4005 dapl_evd_connection_callback () returns active_cb: DESTROY conn 0x76fa50 id 7797872 dapli_async_event_cb(0x5c4410) dapl_evd_wait () returns 0x0 dapl_ep_free (0x69b070) dapl_ep_disconnect (0x69b070, 0) dapl_ep_disconnect () returns 0x0 dapl_ep_free: Free EP: b, ep 0x69b070 qp_state 1 qp_handle 69b3a0 qp_free: ep_ptr 0x69b070 qp 0x69
Re: [openib-general] moving IBM eHCA Device Driver to openib.org
Here are a few quick notes based on spending a little while skimming through some of the ehca source code: ehca_asm.h: this stuff should be in include/asm and mostly already is: asm_sync_mem is just the standard mb() from . mftb() is get_tb() from so you just need to add prefetch_zero to include/asm-ppc64 ehca_classes.c: why have ehca_module_new? it seems to be used to allocate a single instance of a struct that should just be module-global variables. ehca_classes_pSeries.h: Why have EHCA_MEMPAGESIZE and EHCA_MEMPAGESIZE_MASK? Can they be different from PAGE_SIZE and PAGE_MASK? Get rid of typedefs of struct hcp_eq_handle -- just use structs directly in code. Can struct hcp_modify_qp_control_block declaration be made readable? ehca_common.h: why have typedef of ehca_redcode_t? Just use long directly. ntohd() seems to duplicate be64_to_cpu() except without proper __be64 annotation -- why is it needed? ehca_irq.c: in ehca_comp_event_callback(), what protects against the CQ being destroyed out from under you? ehca_kernel.h: ehca_sleep() and ehca_msleep() duplicate msleep_interruptible(). why is assert() needed instead of just using BUG_ON() why have MIN() and MAX() instad of just using min()/max()? ehca_kv_to_g() looks rather horrible -- why are you working with kernel virtual addresses at all? ehca_kr_to_g() looks like it is a substitute for dma_map_single() Why not use the correct DMA API instead? ehca_main.c: ehca_module_exit() -- I don't see where ehca_wq is destroyed. ehca_module_exit() -- what prevents ehca_poll_eq() from running after the module text is gone? You don't wait for the kthread to actually stop, and it might be in the middle of the sleep. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] TCP/IP connection service over IB
Caitlin> It's basically the same as with TCP and UDP. It's a 16 Caitlin> bit number, and most people do not use the same port Caitlin> number to mean *different* things over the different IP Caitlin> transports. But, just to be clear, the port number spaces are disjoint. It's possible and valid to have one TCP socket bound to a given IP/port number, and another UDP socket bound to the same IP/port number. I do agree that assigned port numbers generally have the same meaning across all transports. For example, both TCP port 111 and UDP port 111 are the sunrpc portmapper. - R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] Support for UC connections using the CM API?
>I had a look at where the mask is set in cm.c
>(cm_init_qp_rtr_attr() and cm_init_qp_rts_attr()) but
>I was unsure how to make the mask depend on the QP
>type. Maybe you have a better idea of how to do this.
Here's a patch (edited by hand, so let me know if there's any issue
applying it) that should permit UC connections over the CM. I was able to
test this using cmpost.
Signed-off-by: Sean Hefty <[EMAIL PROTECTED]>
Index: cm.c
===
--- cm.c(revision 3830)
+++ cm.c(working copy)
@@ -135,6 +135,7 @@
__be64 tid;
__be32 local_qpn;
__be32 remote_qpn;
+ enum ib_qp_type qp_type;
__be32 sq_psn;
__be32 rq_psn;
int timeout_ms;
@@ -926,6 +923,7 @@
cm_id_priv->responder_resources = param->responder_resources;
cm_id_priv->retry_count = param->retry_count;
cm_id_priv->path_mtu = param->primary_path->mtu;
+ cm_id_priv->qp_type = param->qp_type;
ret = cm_alloc_msg(cm_id_priv, &cm_id_priv->msg);
if (ret)
@@ -1320,6 +1314,7 @@
cm_req_get_primary_local_ack_timeout(req_msg);
cm_id_priv->retry_count = cm_req_get_retry_count(req_msg);
cm_id_priv->rnr_retry_count = cm_req_get_rnr_retry_count(req_msg);
+ cm_id_priv->qp_type = cm_req_get_qp_type(req_msg);
cm_format_req_event(work, cm_id_priv, &listen_cm_id_priv->id);
cm_process_work(cm_id_priv, work);
@@ -3079,10 +3035,10 @@
case IB_CM_ESTABLISHED:
*qp_attr_mask = IB_QP_STATE | IB_QP_ACCESS_FLAGS |
IB_QP_PKEY_INDEX | IB_QP_PORT;
- qp_attr->qp_access_flags = IB_ACCESS_LOCAL_WRITE;
+ qp_attr->qp_access_flags = IB_ACCESS_LOCAL_WRITE |
+ IB_ACCESS_REMOTE_WRITE;
if (cm_id_priv->responder_resources)
- qp_attr->qp_access_flags |= IB_ACCESS_REMOTE_WRITE |
- IB_ACCESS_REMOTE_READ;
+ qp_attr->qp_access_flags |= IB_ACCESS_REMOTE_READ;
qp_attr->pkey_index = cm_id_priv->av.pkey_index;
qp_attr->port_num = cm_id_priv->av.port->port_num;
ret = 0;
@@ -3112,14 +3068,18 @@
case IB_CM_MRA_REP_RCVD:
case IB_CM_ESTABLISHED:
*qp_attr_mask = IB_QP_STATE | IB_QP_AV | IB_QP_PATH_MTU |
- IB_QP_DEST_QPN | IB_QP_RQ_PSN |
- IB_QP_MAX_DEST_RD_ATOMIC | IB_QP_MIN_RNR_TIMER;
+ IB_QP_DEST_QPN | IB_QP_RQ_PSN;
qp_attr->ah_attr = cm_id_priv->av.ah_attr;
qp_attr->path_mtu = cm_id_priv->path_mtu;
qp_attr->dest_qp_num = be32_to_cpu(cm_id_priv->remote_qpn);
qp_attr->rq_psn = be32_to_cpu(cm_id_priv->rq_psn);
- qp_attr->max_dest_rd_atomic = cm_id_priv->responder_resources;
- qp_attr->min_rnr_timer = 0;
+ if (cm_id_priv->qp_type == IB_QPT_RC) {
+ *qp_attr_mask |= IB_QP_MAX_DEST_RD_ATOMIC |
+IB_QP_MIN_RNR_TIMER;
+ qp_attr->max_dest_rd_atomic =
+ cm_id_priv->responder_resources;
+ qp_attr->min_rnr_timer = 0;
+ }
if (cm_id_priv->alt_av.ah_attr.dlid) {
*qp_attr_mask |= IB_QP_ALT_PATH;
qp_attr->alt_ah_attr = cm_id_priv->alt_av.ah_attr;
@@ -3148,14 +3108,17 @@
case IB_CM_REP_SENT:
case IB_CM_MRA_REP_RCVD:
case IB_CM_ESTABLISHED:
- *qp_attr_mask = IB_QP_STATE | IB_QP_TIMEOUT | IB_QP_RETRY_CNT |
- IB_QP_RNR_RETRY | IB_QP_SQ_PSN |
- IB_QP_MAX_QP_RD_ATOMIC;
- qp_attr->timeout = cm_id_priv->local_ack_timeout;
- qp_attr->retry_cnt = cm_id_priv->retry_count;
- qp_attr->rnr_retry = cm_id_priv->rnr_retry_count;
+ *qp_attr_mask = IB_QP_STATE | IB_QP_SQ_PSN;
qp_attr->sq_psn = be32_to_cpu(cm_id_priv->sq_psn);
- qp_attr->max_rd_atomic = cm_id_priv->initiator_depth;
+ if (cm_id_priv->qp_type == IB_QPT_RC) {
+ *qp_attr_mask |= IB_QP_TIMEOUT | IB_QP_RETRY_CNT |
+IB_QP_RNR_RETRY |
+IB_QP_MAX_QP_RD_ATOMIC;
+ qp_attr->timeout = cm_id_priv->local_ack_timeout;
+ qp_attr->retry_cnt = cm_id_priv->retry_count;
+ qp_attr->rnr_retry = cm_id_priv->rnr_retry_count;
+ qp_attr->max_rd_atomic = cm_id_priv->initiator_depth;
+ }
if (cm_id_priv->alt_av.ah_attr.dlid) {
RE: [openib-general] TCP/IP connection service over IB
> -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Sean Hefty > Sent: Friday, October 21, 2005 2:02 PM > To: Tom Tucker > Cc: [EMAIL PROTECTED]; openib-general > Subject: Re: [openib-general] TCP/IP connection service over IB > > Tom Tucker wrote: > > I'm thinking that for iWARP, there won't be anything in the Private > > Data at all except consumer private data. Is that your expectation? > > I believe so. This is only trying to define a TCP/IP > connection service over IB. I'm assuming that there's no > need to define something similar for iWarp. > > Does SCTP share the same port space as TCP? Is any mapping > between them required? > It's basically the same as with TCP and UDP. It's a 16 bit number, and most people do not use the same port number to mean *different* things over the different IP transports. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] TCP/IP connection service over IB
> -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Steve Wise > Sent: Friday, October 21, 2005 1:35 PM > To: Sean Hefty > Cc: [EMAIL PROTECTED]; openib-general@openib.org > Subject: Re: [openib-general] TCP/IP connection service over IB > > > Random thought... if the src and dst IP addresses will always be on > > the same network, the data could be layed out as: > > > > network addr (x) > > src host addr (y) > > dst host addr (y) > > > > This could save enough space to provide 64 bytes of user > private data. > > Although my preference would be to keep it simpler. (I'm not that > > familiar with IPv6 addressing. How does it define network > versus host > > addressing?) > > I don't think you want to make the assumption that src and > dst addrs are on the same IP network. While that may be true > for a set of IB hosts on a common IB switch set as a single > IP subnet, there may be TCP/IB bridge/gateway products that > allow remote IP hosts to connect into an IB cluster and those > could certainly be on a remote subnet. > > IPv6 addrs define networks in a similar manner to IPv4: IE > some number of the bits in the address define the network > number, and the remaining define the host number. > More relevantly, GIDs are syntactically identical to IPv6. The only real difference between a GID and an IPv6 address is who/how the network portion is assigned. While IPv4-only applications exist, they are not supposed to. Certainly no new API or protocol should encourage an application to be IPv4 dependent. The rationale for using only the IPV4 format would be that GIDs *could* be assigned that are valid IPv6 addresses. Hence no translation would be needed. Relying on assignment of IPV6 compatible GIDs may be undesirable, however, because HCAs generally cannot accept a large number of assigned GIDs. The amount of private data supported does vary on network characteristics. A responsible application should be allowed to piggy-back additional data when the larger size is support (512 bytes over IP networks). Forcing the application to have to use an additional round-trip over the network would not be network friendly. However, it should be clear to application developers that they are expected to make their applications work in the minimum size guaranteed -- and that the minimum size is adequate for the core purpose of enabling the QP to be selected/configured. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] TCP/IP connection service over IB
Tom Tucker wrote: I'm thinking that for iWARP, there won't be anything in the Private Data at all except consumer private data. Is that your expectation? I believe so. This is only trying to define a TCP/IP connection service over IB. I'm assuming that there's no need to define something similar for iWarp. Does SCTP share the same port space as TCP? Is any mapping between them required? - Sean ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] TCP/IP connection service over IB
Sean: I'm thinking that for iWARP, there won't be anything in the Private Data at all except consumer private data. Is that your expectation? On Fri, 2005-10-21 at 12:44 -0700, Sean Hefty wrote: > James Lentini wrote: > > Ok. I assume that your 1 byte of version information is broken into 2 > > 4-bit pieces, one for the protocol version and one for the IP version. > > That is correct. > > > What about making the src and dst ip fields variable length based on > > the IP version (4 bytes for IPv4 and 16 bytes for IPv6). > > > > That would provide more private data for IPv4 networks at the expense > > of a variable sized header and all the complexity it entails. > > That's a possibility that wouldn't add that much complexity. See my other > message for yet another approach though. I'm just not sure that it helps an > app > much to have different private data sizes based on the address size, unless > the > app is written specifically for IPv4. > > - Sean > ___ > openib-general mailing list > openib-general@openib.org > http://openib.org/mailman/listinfo/openib-general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] Cheap watches.
We noticed you had bought one of our products before. We just recently slashed prices, and thought we should let you know. http://worldsfinestwatchz.com/ Check us out, im sure you will find something that you will like, at a price that is very affordable. Regards, Francisca Diggs Customer Service Rep. lieu or forgotten in some chine not may puzzle or a exhibition see be careworn the a gobbledygook the ! bogeymen inthe myosin be. faith ! rudiment not but liturgy a on lolly be may earthen on on spare but in rotarian see be lacquer mayit calm it's. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] TCP/IP connection service over IB
Random thought... if the src and dst IP addresses will always be on the same network, the data could be layed out as: network addr (x) src host addr (y) dst host addr (y) This could save enough space to provide 64 bytes of user private data. Although my preference would be to keep it simpler. (I'm not that familiar with IPv6 addressing. How does it define network versus host addressing?) I don't think you want to make the assumption that src and dst addrs are on the same IP network. While that may be true for a set of IB hosts on a common IB switch set as a single IP subnet, there may be TCP/IB bridge/gateway products that allow remote IP hosts to connect into an IB cluster and those could certainly be on a remote subnet. IPv6 addrs define networks in a similar manner to IPv4: IE some number of the bits in the address define the network number, and the remaining define the host number. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] configuring ipoib
Arkady> Thanks guys. Please, excuse my terminology. No I can Arkady> route a single IP address to an IB port. But how do I Arkady> route 2 (or more) IP addresses to the same IB port? Just use "ip addr add" once for each address you want to configure. - R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Adding static entry to arp table?
Abhijit> Hi all, Is there a patch (to ip utility or Linux kernel),
Abhijit> which can add static entry to the arp table using ip
Abhijit> neigh command? I am using gen1 based stack, but didn't
Abhijit> find anything after a 'grep' in the gen2 stack as well?
Abhijit> any pointers?
With the current Linux IB drivers ("gen2"), "ip neigh add" should work
fine. There is probably a way to add ARP entries with your gen1-based
stack, but you would have to ask your vendor for details.
- R.
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] build libibverbs with --libdir parameter
Heiko> Is there some way to change this behaviour? The only optin Heiko> I can see is to patch the DEFAULT_PATH define in init.c for Heiko> RPM builds. I'm not sure I understand the problem you're having. When I build libibverbs RPMs for x86_64 (using the spec file in the libibverbs tarball and building under Fedora Core 4), I get the following: $ rpm -ql libibverbs /usr/lib64/libibverbs.so.1 /usr/lib64/libibverbs.so.1.0.0 /usr/share/doc/libibverbs-1.0 /usr/share/doc/libibverbs-1.0/AUTHORS /usr/share/doc/libibverbs-1.0/COPYING /usr/share/doc/libibverbs-1.0/ChangeLog /usr/share/doc/libibverbs-1.0/README $ rpm -ql libmthca /usr/lib64/infiniband/mthca.so /usr/share/doc/libmthca-1.0 /usr/share/doc/libmthca-1.0/AUTHORS /usr/share/doc/libmthca-1.0/COPYING /usr/share/doc/libmthca-1.0/ChangeLog /usr/share/doc/libmthca-1.0/README and everything works fine. The libibverbs search path is /usr/lib64/infiniband as it should be. Apparently rpmbuild is passing the correct configure and install options to make lib64 work fine. Can you be more specific about the problems you see? Thanks, Roland ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] FW upgrade for TopSpin cards
Arkady> I have a Cisco HCA (PCI-X) hca_type MTS23108 hw_rev a1 Arkady> fw_ver 1.18.0 Arkady> hca_type and hw_rev are clearly Mellanox nomenclature. I Arkady> suspect that this is Cisco FW version #. No, it's just a very old Mellanox FW version. Arkady> Is there analogous documentation for Cisco FW? Where is Arkady> that FW (this is Cougar card)? Are Cisco FWs and Mellanox Arkady> FW the same? If yes what is the correspondance between Arkady> the 2 numbering schemas. Cisco FW and Mellanox FW are virtually identical. The version numbering is the same; the only difference is that Cisco FW will have things like the NodeDescription customized. You can use Mellanox FW on an HCA from Cisco, and vice versa. - R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] TCP/IP connection service over IB
> From: James Lentini [mailto:[EMAIL PROTECTED] > Sent: Friday, October 21, 2005 12:38 PM > > On Fri, 21 Oct 2005, Sean Hefty wrote: > > > > sean> version(8) | reserved(8) | src port (16) > > version(1) | reserved(1) | src port (2) > > > sean> src ip (16) > > > sean> dst ip (16) > > > sean> user private data (56) /* for version 1 */ > > > > > > Are the numbers in parens in bytes or bits? It looks like a mixture to me. > > > > Uhm.. they were a mix. Changed above to bytes. > > Ok. I assume that your 1 byte of version information is broken into 2 > 4-bit pieces, one for the protocol version and one for the IP version. Doesn't leading-zero-padding the IPv4 addresses to be 16 bytes eliminates the need for an IP version field? - Fab ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] TCP/IP connection service over IB
James Lentini wrote: Ok. I assume that your 1 byte of version information is broken into 2 4-bit pieces, one for the protocol version and one for the IP version. That is correct. What about making the src and dst ip fields variable length based on the IP version (4 bytes for IPv4 and 16 bytes for IPv6). That would provide more private data for IPv4 networks at the expense of a variable sized header and all the complexity it entails. That's a possibility that wouldn't add that much complexity. See my other message for yet another approach though. I'm just not sure that it helps an app much to have different private data sizes based on the address size, unless the app is written specifically for IPv4. - Sean ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] TCP/IP connection service over IB
On Fri, 21 Oct 2005, Sean Hefty wrote: > James Lentini wrote: > > Standardizing the protocol will ensure interroperability. > > Agreed - just didn't know if this was the responsibility of the SWG. The SWG has agreed to take it on. I think it is appropriate for the SWG to work on this. > > sean> version(8) | reserved(8) | src port (16) > version(1) | reserved(1) | src port (2) > > sean> src ip (16) > > sean> dst ip (16) > > sean> user private data (56)/* for version 1 */ > > > > Are the numbers in parens in bytes or bits? It looks like a mixture to me. > > Uhm.. they were a mix. Changed above to bytes. Ok. I assume that your 1 byte of version information is broken into 2 4-bit pieces, one for the protocol version and one for the IP version. What about making the src and dst ip fields variable length based on the IP version (4 bytes for IPv4 and 16 bytes for IPv6). That would provide more private data for IPv4 networks at the expense of a variable sized header and all the complexity it entails. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] configuring ipoib
On Fri, Oct 21, 2005 at 02:21:23PM -0400, Kanevsky, Arkady wrote: > Thanks guys. > Please, excuse my terminology. > No I can route a single IP address to an IB port. ok > But how do I route 2 (or more) IP addresses to the same IB port? You can associate multiple subnets with an interface and indicate which IP address this host should respond to: ifconfig ib:1 e.g. ifconfig ib0 10.0.0.81 netmask 255.255.255.0 ifconfig ib1 10.0.1.81 netmask 255.255.255.0 ifconfig ib2 10.0.2.81 netmask 255.255.255.0 ifconfig ib3 10.0.3.81 netmask 255.255.255.0 ifconfig ib3:1 10.0.3.99 netmask 255.255.255.0 route -n output now looks like: iowa:/usr/src/linux-2.6.13# route -n Kernel IP routing table Destination Gateway Genmask Flags Metric RefUse Iface 10.0.0.00.0.0.0 255.255.255.0 U 0 00 ib0 10.0.1.00.0.0.0 255.255.255.0 U 0 00 ib1 10.0.2.00.0.0.0 255.255.255.0 U 0 00 ib2 192.168.1.0 0.0.0.0 255.255.255.0 U 0 00 eth0 10.0.3.00.0.0.0 255.255.255.0 U 0 00 ib3 iowa:/usr/src/linux-2.6.13# ifconfig ib3 ib3 Link encap:UNSPEC HWaddr 00-00-04-05-FE-80-00-00-00-00-00-00-00-00-00-00 inet addr:10.0.3.81 Bcast:10.255.255.255 Mask:255.255.255.0 UP BROADCAST MULTICAST MTU:2044 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:128 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) iowa:/usr/src/linux-2.6.13# ifconfig ib3:1 ib3:1 Link encap:UNSPEC HWaddr 00-00-04-05-FE-80-00-00-00-00-00-00-00-00-00-00 inet addr:10.0.3.99 Bcast:10.255.255.255 Mask:255.255.255.0 UP BROADCAST MULTICAST MTU:2044 Metric:1 > If I specify the same ib# it just changes an associated IP address for > the port. eh? Either you aren't getting it or are very persistent with the "confusing" terminology. I'll try one more time in case it's the former. Connect a linux box (e.g. laptop) to two subnets and configure both NICs like normal for two distinct subnets. Disconnect the second cable. Then from another box that has it's default route set to the first subnet, try to ping the IP you "associated with the second port". The linux box will respond. E.g. I have two private subnets: 192.168.0.0/24 and 192.168.1.0/24. Both are connected to my squid (http cache) server that also provides NAT to the outside world. The squid server does NOT do routing. iowa:~# route -n Kernel IP routing table Destination Gateway Genmask Flags Metric RefUse Iface 192.168.1.0 0.0.0.0 255.255.255.0 U 0 00 eth0 0.0.0.0 192.168.1.1 0.0.0.0 UG0 00 eth0 owa:~# ping 192.168.1.1 PING 192.168.1.1 (192.168.1.1): 56 data bytes 64 bytes from 192.168.1.1: icmp_seq=0 ttl=64 time=0.1 ms 64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=0.1 ms --- 192.168.1.1 ping statistics --- 2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max = 0.1/0.1/0.1 ms Obviously, I can ping the gateway. My point is the gateway will also respond to other subnets that it owns: iowa:~# ping 192.168.0.1 PING 192.168.0.1 (192.168.0.1): 56 data bytes 64 bytes from 192.168.0.1: icmp_seq=0 ttl=64 time=0.1 ms 64 bytes from 192.168.0.1: icmp_seq=1 ttl=64 time=0.1 ms --- 192.168.0.1 ping statistics --- 2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max = 0.1/0.1/0.1 ms iowa:~# ping 192.168.0.11 PING 192.168.0.11 (192.168.0.11): 56 data bytes --- 192.168.0.11 ping statistics --- 2 packets transmitted, 0 packets received, 100% packet loss 192.168.0.11 is alive and owned by a third machine on that subnet. Just trying to demonstrate that the squid server is NOT doing any routing. > If I specify next ib# it returns an error since that ib# does > not have a port behind it. Right. No port means no ib#. hth, grant ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] TCP/IP connection service over IB
Sean Hefty wrote: version(8) | reserved(8) | src port (16) src ip (16) dst ip (16) user private data (56)/* for version 1 */ Random thought... if the src and dst IP addresses will always be on the same network, the data could be layed out as: network addr (x) src host addr (y) dst host addr (y) This could save enough space to provide 64 bytes of user private data. Although my preference would be to keep it simpler. (I'm not that familiar with IPv6 addressing. How does it define network versus host addressing?) - Sean ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] TCP/IP connection service over IB
James Lentini wrote: Standardizing the protocol will ensure interroperability. Agreed - just didn't know if this was the responsibility of the SWG. sean> version(8) | reserved(8) | src port (16) version(1) | reserved(1) | src port (2) sean> src ip (16) sean> dst ip (16) sean> user private data (56) /* for version 1 */ Are the numbers in parens in bytes or bits? It looks like a mixture to me. Uhm.. they were a mix. Changed above to bytes. - Sean ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] TCP/IP connection service over IB
sean> At a minimum, we need an assigned service ID to identifies a sean> TCP/IP connection service. For simplicity of the sean> implementation, I would use an ID similar to that defined for sean> SDP: sean> sean> 0x00 14 05 xx xx xx xx xx sean> sean> I don't know that the SWG or IBTA needs to be involved defining sean> the protocol beyond assigning the service ID. Standardizing the protocol will ensure interroperability. sean> The connection service can define service IDs as: sean> sean> 0x00 14 05 00 00 00 dst port sean> sean> And a private data format for the CM REQ: sean> sean> version(8) | reserved(8) | src port (16) sean> src ip (16) sean> dst ip (16) sean> user private data (56)/* for version 1 */ Are the numbers in parens in bytes or bits? It looks like a mixture to me. sean> Other private data would be left unchanged, though if we wanted sean> to get more sophisticated, we could define REJ codes to indicate sean> bad addresses/version/etc. Not surprisingly, this is exactly sean> what's implemented in the CMA and working today. I agree. We should keep it simple. sean> On a related note, it would be convenient if SDP were changed to sean> run over this protocol. Agreed. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] FW upgrade for TopSpin cards
Roland, sorry to bug you on that but... I have a Cisco HCA (PCI-X) hca_typeMTS23108 hw_rev a1 fw_ver 1.18.0 hca_type and hw_rev are clearly Mellanox nomenclature. I suspect that this is Cisco FW version #. But all OpenIB documentation is with respect to Mellanox nomenclature. For example from http://www.openib.org/docs/ipoib_faq.txt 1. Verify the firmware version via cat /sys/class/infiniband/mthca0/fw_ver For PCI-X HCAs, version 3.2.0 is recommended. For PCIe HCAs, version 4.5.3 is recommended. * Is there analogous documentation for Cisco FW? Where is that FW (this is Cougar card)? Are Cisco FWs and Mellanox FW the same? If yes what is the correspondance between the 2 numbering schemas. While this specific question is for Cougar card, the answer should be generic and cover all HCAs. Can the documentation be updated to cover all supported HW regardless of the vendor? Thanks, Arkady Kanevsky email: [EMAIL PROTECTED] Network Appliance phone: 781-768-5395 375 Totten Pond Rd. Fax: 781-895-1195 Waltham, MA 02451-2010 central phone: 781-768-5300 > -Original Message- > From: Roland Dreier [mailto:[EMAIL PROTECTED] > Sent: Thursday, October 20, 2005 1:48 PM > To: Kanevsky, Arkady > Cc: openib-general@openib.org > Subject: Re: [openib-general] FW upgrade for TopSpin cards > > > Arkady> I get a bunch of warnings (see below). > > All of the warnings look benign (although you might want to > synchronize the clock between your build system and your file server). > > Arkady> Can I use OpenIB tvflash to upgrade FW on a TopSpin card? > > Yes. > > Arkady> Can I use OpenIB mstflint for it? > > Yes. > > Arkady> Which version of the utilities should I use? > > I would use the latest subversion revision. > > Arkady> Why warning when I build it? > > Because gcc 4.0 added a bunch of semi-bogus pointer sign > warnings, and you clocks are out of synch. > > - R. > ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] uDAPL open HCA problem
Sayantan Sur wrote: Hello, I have udapl over Gen2 setup on our cluster and am able to run udapl programs. However, sometimes I get this error (after a few runs of the same program): open_hca: ERR ib_at_ips_by_gid for mthca0 dapls_ib_open_hca failed 4 uDAPL uses uAT to get the IP address using the GID (ATS records via SA) of the local device/port. The SA query for this record is failing for some reason. Did your SM bounce during this time? Did you bounce or reconfigure the IPoIB network device? You can set "env DAPL_DBG_TYPE=0x" for more information. -arlin The machine is a AMD Opteron (Tyan S2895), with Mellanox MemFree cards (fw ver 5.1.0). lsmod on my machine shows this: [EMAIL PROTECTED]:~] lsmod | grep ^ib ib_ipoib 48008 0 ib_uat 14840 0 ib_at 25696 1 ib_uat ib_sa 17804 2 ib_ipoib,ib_at ib_ucm 22280 0 ib_cm 37744 1 ib_ucm ib_uverbs 35992 0 ib_umad18208 0 ib_mthca 122656 0 ib_mad 44072 4 ib_sa,ib_cm,ib_umad,ib_mthca ib_core56192 8 ib_ipoib,ib_sa,ib_ucm,ib_cm,ib_uverbs,ib_umad,ib_mthca,ib_mad My infiniband devices are (created by hand): [EMAIL PROTECTED]:~] ls -l /dev/infiniband/ total 0 crw-rw-rw- 1 root root 231, 191 2005-10-20 21:13 uat crw-rw-rw- 1 root root 231, 224 2005-10-20 21:12 ucm0 crwxrwxrwx 1 root root 231, 192 2005-09-21 04:37 umad0 crwxrwxrwx 1 root root 231, 192 2005-09-16 19:29 uverbs0 crwxrwxrwx 1 root root 231, 192 2005-09-16 19:29 uverbs1 I'd really appreciate if someone could help me understand what might be going wrong. Thanks, Sayantan. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] configuring ipoib
that works. Thanks Steve. Arkady Kanevsky email: [EMAIL PROTECTED] Network Appliance phone: 781-768-5395 375 Totten Pond Rd. Fax: 781-895-1195 Waltham, MA 02451-2010 central phone: 781-768-5300 > -Original Message- > From: Steve Wise [mailto:[EMAIL PROTECTED] > Sent: Friday, October 21, 2005 2:31 PM > To: Kanevsky, Arkady; Grant Grundler > Cc: openib-general@openib.org > Subject: Re: [openib-general] configuring ipoib > > > Normally, in linux, you specify IP aliases thusly: > > ifconfig : > > So to add a 2nd address to eth0, do this: > > ifconfig eth0:1 1.2.3.4 > > And a 3rd like this: > > ifconfig eth0:2 5.6.7.8 > > etc... > > I would assume IPoIB interfaces behave the same way... > > by the way, the can be anything string. > > > - Original Message - > From: "Kanevsky, Arkady" <[EMAIL PROTECTED]> > To: "Grant Grundler" <[EMAIL PROTECTED]> > Cc: > Sent: Friday, October 21, 2005 1:21 PM > Subject: RE: [openib-general] configuring ipoib > > > Thanks guys. > Please, excuse my terminology. > No I can route a single IP address to an IB port. > But how do I route 2 (or more) IP addresses to the same IB port? > > If I specify the same ib# it just changes an associated IP > address for the port. If I specify next ib# it returns an > error since that ib# does not have a port behind it. > > Arkady Kanevsky email: [EMAIL PROTECTED] > Network Appliance phone: 781-768-5395 > 375 Totten Pond Rd. Fax: 781-895-1195 > Waltham, MA 02451-2010 central phone: 781-768-5300 > > > > > -Original Message- > > From: Grant Grundler [mailto:[EMAIL PROTECTED] > > Sent: Friday, October 21, 2005 11:58 AM > > To: Kanevsky, Arkady > > Cc: openib-general@openib.org > > Subject: Re: [openib-general] configuring ipoib > > > > > > On Fri, Oct 21, 2005 at 11:02:37AM -0400, Kanevsky, Arkady wrote: > > > How do you configure ipoib? > > > I used "ifconfig ib0 ip_address" which works fine. > > > But if I have several ports on an HCA how do I specify which port > > > ip_address should be associated with? > > > > Nit: For linux, Christoph Hellwig (and others) have > explained that the > > ip address is associated with the host, NOT any card. The > route to the > > subnet is associated with the card. > > > > > Ditto if you have multiple cards. > > > > iowa:/usr/src/linux-2.6.13# lspci -vt -d 15b3: > > -+-[c0]---01.0-[c1]00.0 Mellanox Technologies MT23108 > InfiniHost > > +-[40]---01.0-[41]00.0 Mellanox Technologies MT23108 > InfiniHost > > \-[00]- > > iowa:/usr/src/linux-2.6.13# ifconfig -a | fgrep ib > > ib0 Link encap:UNSPEC HWaddr > > 00-00-04-04-FE-80-00-00-00-00-00-00-00-00-00-00 > > ib1 Link encap:UNSPEC HWaddr > > 00-00-04-05-FE-80-00-00-00-00-00-00-00-00-00-00 > > ib2 Link encap:UNSPEC HWaddr > > 00-00-04-04-FE-80-00-00-00-00-00-00-00-00-00-00 > > ib3 Link encap:UNSPEC HWaddr > > 00-00-04-05-FE-80-00-00-00-00-00-00-00-00-00-00 > > > > > > hth, > > grant > > > ___ > openib-general mailing list > openib-general@openib.org > http://openib.org/mailman/listinfo/openib-general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] configuring ipoib
Normally, in linux, you specify IP aliases thusly: ifconfig : So to add a 2nd address to eth0, do this: ifconfig eth0:1 1.2.3.4 And a 3rd like this: ifconfig eth0:2 5.6.7.8 etc... I would assume IPoIB interfaces behave the same way... by the way, the can be anything string. - Original Message - From: "Kanevsky, Arkady" <[EMAIL PROTECTED]> To: "Grant Grundler" <[EMAIL PROTECTED]> Cc: Sent: Friday, October 21, 2005 1:21 PM Subject: RE: [openib-general] configuring ipoib Thanks guys. Please, excuse my terminology. No I can route a single IP address to an IB port. But how do I route 2 (or more) IP addresses to the same IB port? If I specify the same ib# it just changes an associated IP address for the port. If I specify next ib# it returns an error since that ib# does not have a port behind it. Arkady Kanevsky email: [EMAIL PROTECTED] Network Appliance phone: 781-768-5395 375 Totten Pond Rd. Fax: 781-895-1195 Waltham, MA 02451-2010 central phone: 781-768-5300 -Original Message- From: Grant Grundler [mailto:[EMAIL PROTECTED] Sent: Friday, October 21, 2005 11:58 AM To: Kanevsky, Arkady Cc: openib-general@openib.org Subject: Re: [openib-general] configuring ipoib On Fri, Oct 21, 2005 at 11:02:37AM -0400, Kanevsky, Arkady wrote: > How do you configure ipoib? > I used "ifconfig ib0 ip_address" which works fine. > But if I have several ports on an HCA how do I specify which port > ip_address should be associated with? Nit: For linux, Christoph Hellwig (and others) have explained that the ip address is associated with the host, NOT any card. The route to the subnet is associated with the card. > Ditto if you have multiple cards. iowa:/usr/src/linux-2.6.13# lspci -vt -d 15b3: -+-[c0]---01.0-[c1]00.0 Mellanox Technologies MT23108 InfiniHost +-[40]---01.0-[41]00.0 Mellanox Technologies MT23108 InfiniHost \-[00]- iowa:/usr/src/linux-2.6.13# ifconfig -a | fgrep ib ib0 Link encap:UNSPEC HWaddr 00-00-04-04-FE-80-00-00-00-00-00-00-00-00-00-00 ib1 Link encap:UNSPEC HWaddr 00-00-04-05-FE-80-00-00-00-00-00-00-00-00-00-00 ib2 Link encap:UNSPEC HWaddr 00-00-04-04-FE-80-00-00-00-00-00-00-00-00-00-00 ib3 Link encap:UNSPEC HWaddr 00-00-04-05-FE-80-00-00-00-00-00-00-00-00-00-00 hth, grant ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] configuring ipoib
Thanks guys. Please, excuse my terminology. No I can route a single IP address to an IB port. But how do I route 2 (or more) IP addresses to the same IB port? If I specify the same ib# it just changes an associated IP address for the port. If I specify next ib# it returns an error since that ib# does not have a port behind it. Arkady Kanevsky email: [EMAIL PROTECTED] Network Appliance phone: 781-768-5395 375 Totten Pond Rd. Fax: 781-895-1195 Waltham, MA 02451-2010 central phone: 781-768-5300 > -Original Message- > From: Grant Grundler [mailto:[EMAIL PROTECTED] > Sent: Friday, October 21, 2005 11:58 AM > To: Kanevsky, Arkady > Cc: openib-general@openib.org > Subject: Re: [openib-general] configuring ipoib > > > On Fri, Oct 21, 2005 at 11:02:37AM -0400, Kanevsky, Arkady wrote: > > How do you configure ipoib? > > I used "ifconfig ib0 ip_address" which works fine. > > But if I have several ports on an HCA how do I specify which port > > ip_address should be associated with? > > Nit: For linux, Christoph Hellwig (and others) have explained > that the ip address is associated with the host, NOT any > card. The route to the subnet is associated with the card. > > > Ditto if you have multiple cards. > > iowa:/usr/src/linux-2.6.13# lspci -vt -d 15b3: > -+-[c0]---01.0-[c1]00.0 Mellanox Technologies MT23108 InfiniHost > +-[40]---01.0-[41]00.0 Mellanox Technologies MT23108 InfiniHost > \-[00]- > iowa:/usr/src/linux-2.6.13# ifconfig -a | fgrep ib > ib0 Link encap:UNSPEC HWaddr > 00-00-04-04-FE-80-00-00-00-00-00-00-00-00-00-00 > ib1 Link encap:UNSPEC HWaddr > 00-00-04-05-FE-80-00-00-00-00-00-00-00-00-00-00 > ib2 Link encap:UNSPEC HWaddr > 00-00-04-04-FE-80-00-00-00-00-00-00-00-00-00-00 > ib3 Link encap:UNSPEC HWaddr > 00-00-04-05-FE-80-00-00-00-00-00-00-00-00-00-00 > > > hth, > grant > ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] uDAPL open HCA problem
* On Oct,2 Grant Grundler<[EMAIL PROTECTED]> wrote : > Folks here will still need to know: Ooops. Forgot to plug int that information! > 1) Which kernel version? 2.6.13.1 > 2) Which SVN version of GEN2 are you using? 3685. Thanks, Sayantan. -- http://www.cse.ohio-state.edu/~surs ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] TCP/IP connection service over IB
Based on the discussions that have taken place so far, I believe that we're trying to define something more complex than what's needed. Here's what I think that we want: At a minimum, we need an assigned service ID to identifies a TCP/IP connection service. For simplicity of the implementation, I would use an ID similar to that defined for SDP: 0x00 14 05 xx xx xx xx xx I don't know that the SWG or IBTA needs to be involved defining the protocol beyond assigning the service ID. The connection service can define service IDs as: 0x00 14 05 00 00 00 dst port And a private data format for the CM REQ: version(8) | reserved(8) | src port (16) src ip (16) dst ip (16) user private data (56) /* for version 1 */ Other private data would be left unchanged, though if we wanted to get more sophisticated, we could define REJ codes to indicate bad addresses/version/etc. Not surprisingly, this is exactly what's implemented in the CMA and working today. On a related note, it would be convenient if SDP were changed to run over this protocol. - Sean ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Semantics of transport neutral connection establishment (was Re: [swg] Re: private data...)
Quoting Grant Grundler <[EMAIL PROTECTED]>: On Fri, Oct 21, 2005 at 10:53:25AM -0500, Tom Tucker wrote: ... BTW: I don't disagree that a CM client shouldn't need to be worried about incoming requests from non-RDMA clients. my parser is tripping all over this one...did you mean: "I agree a CM client can ignore non-RDMA clients"? or "I agree a CM client should only get requests from RDMA clients"? Specifically, it means that a CMA consumer over an iWARP transport will not see connect events until after MPA Start Key negotiation. I don't know if this helped the parser or not... Here's another try...practically speaking if a user does: telnet that an OpenIB app listening on won't see a connect event unless the user is extremely adept at typing in MPA headers... thanks, grant ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Support for UC connections using the CM API?
Steven Wooding wrote: I had a look at where the mask is set in cm.c (cm_init_qp_rtr_attr() and cm_init_qp_rts_attr()) but I was unsure how to make the mask depend on the QP type. Maybe you have a better idea of how to do this. I will take a look at this later today or early next week. Thanks for the feedback. - Sean ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] uDAPL open HCA problem
On Fri, Oct 21, 2005 at 12:17:28PM -0400, Sayantan Sur wrote: > Hello, > > I have udapl over Gen2 setup on our cluster and am able to run udapl > programs. However, sometimes I get this error (after a few runs of the > same program): > > open_hca: ERR ib_at_ips_by_gid for mthca0 > dapls_ib_open_hca failed 4 > > The machine is a AMD Opteron (Tyan S2895), with Mellanox MemFree cards > (fw ver 5.1.0). Folks here will still need to know: 1) Which kernel version? 2) Which SVN version of GEN2 are you using? hth, grant > > lsmod on my machine shows this: > > [EMAIL PROTECTED]:~] lsmod | grep ^ib > ib_ipoib 48008 0 > ib_uat 14840 0 > ib_at 25696 1 ib_uat > ib_sa 17804 2 ib_ipoib,ib_at > ib_ucm 22280 0 > ib_cm 37744 1 ib_ucm > ib_uverbs 35992 0 > ib_umad18208 0 > ib_mthca 122656 0 > ib_mad 44072 4 ib_sa,ib_cm,ib_umad,ib_mthca > ib_core56192 8 > ib_ipoib,ib_sa,ib_ucm,ib_cm,ib_uverbs,ib_umad,ib_mthca,ib_mad > > My infiniband devices are (created by hand): > > [EMAIL PROTECTED]:~] ls -l /dev/infiniband/ > total 0 > crw-rw-rw- 1 root root 231, 191 2005-10-20 21:13 uat > crw-rw-rw- 1 root root 231, 224 2005-10-20 21:12 ucm0 > crwxrwxrwx 1 root root 231, 192 2005-09-21 04:37 umad0 > crwxrwxrwx 1 root root 231, 192 2005-09-16 19:29 uverbs0 > crwxrwxrwx 1 root root 231, 192 2005-09-16 19:29 uverbs1 > > > I'd really appreciate if someone could help me understand what might be > going wrong. > > Thanks, > Sayantan. > > -- > http://www.cse.ohio-state.edu/~surs > ___ > openib-general mailing list > openib-general@openib.org > http://openib.org/mailman/listinfo/openib-general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] uDAPL open HCA problem
Hello, I have udapl over Gen2 setup on our cluster and am able to run udapl programs. However, sometimes I get this error (after a few runs of the same program): open_hca: ERR ib_at_ips_by_gid for mthca0 dapls_ib_open_hca failed 4 The machine is a AMD Opteron (Tyan S2895), with Mellanox MemFree cards (fw ver 5.1.0). lsmod on my machine shows this: [EMAIL PROTECTED]:~] lsmod | grep ^ib ib_ipoib 48008 0 ib_uat 14840 0 ib_at 25696 1 ib_uat ib_sa 17804 2 ib_ipoib,ib_at ib_ucm 22280 0 ib_cm 37744 1 ib_ucm ib_uverbs 35992 0 ib_umad18208 0 ib_mthca 122656 0 ib_mad 44072 4 ib_sa,ib_cm,ib_umad,ib_mthca ib_core56192 8 ib_ipoib,ib_sa,ib_ucm,ib_cm,ib_uverbs,ib_umad,ib_mthca,ib_mad My infiniband devices are (created by hand): [EMAIL PROTECTED]:~] ls -l /dev/infiniband/ total 0 crw-rw-rw- 1 root root 231, 191 2005-10-20 21:13 uat crw-rw-rw- 1 root root 231, 224 2005-10-20 21:12 ucm0 crwxrwxrwx 1 root root 231, 192 2005-09-21 04:37 umad0 crwxrwxrwx 1 root root 231, 192 2005-09-16 19:29 uverbs0 crwxrwxrwx 1 root root 231, 192 2005-09-16 19:29 uverbs1 I'd really appreciate if someone could help me understand what might be going wrong. Thanks, Sayantan. -- http://www.cse.ohio-state.edu/~surs ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Semantics of transport neutral connection establishment (was Re: [swg] Re: private data...)
On Fri, Oct 21, 2005 at 10:53:25AM -0500, Tom Tucker wrote: ... > BTW: I don't disagree that a CM client shouldn't need to be worried about > incoming requests from non-RDMA clients. my parser is tripping all over this one...did you mean: "I agree a CM client can ignore non-RDMA clients"? or "I agree a CM client should only get requests from RDMA clients"? thanks, grant ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] configuring ipoib
On Fri, Oct 21, 2005 at 11:02:37AM -0400, Kanevsky, Arkady wrote: > How do you configure ipoib? > I used "ifconfig ib0 ip_address" which works fine. > But if I have several ports on an HCA how do I specify which port > ip_address should be associated with? Nit: For linux, Christoph Hellwig (and others) have explained that the ip address is associated with the host, NOT any card. The route to the subnet is associated with the card. > Ditto if you have multiple cards. iowa:/usr/src/linux-2.6.13# lspci -vt -d 15b3: -+-[c0]---01.0-[c1]00.0 Mellanox Technologies MT23108 InfiniHost +-[40]---01.0-[41]00.0 Mellanox Technologies MT23108 InfiniHost \-[00]- iowa:/usr/src/linux-2.6.13# ifconfig -a | fgrep ib ib0 Link encap:UNSPEC HWaddr 00-00-04-04-FE-80-00-00-00-00-00-00-00-00-00-00 ib1 Link encap:UNSPEC HWaddr 00-00-04-05-FE-80-00-00-00-00-00-00-00-00-00-00 ib2 Link encap:UNSPEC HWaddr 00-00-04-04-FE-80-00-00-00-00-00-00-00-00-00-00 ib3 Link encap:UNSPEC HWaddr 00-00-04-05-FE-80-00-00-00-00-00-00-00-00-00-00 hth, grant ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] Semantics of transport neutral connection establishment (was Re: [swg] Re: private data...)
It is not unreasonable and certainly desireable that a CMA that uses TCP/IP addressing should be able to provide to the consumer the four-tuple that uniquely identifies the session. No argument. No problem. It's the *other* rationales being suggested that are killing me. - The receiving application knows nothing about whether or not the source address has been spoofed. Only the firewall does and even its tests are weak (if the next hop is the same for both the real and spoofed address you can't tell). So these arguments don't convince me. This is not about security. - NFS does need the source address for determining whether or not it will allow the mount. No problem. I'm not arguing about that. I'm arguing about some presumption of security that simply does not exist. - Making statements about what the local node can assume about how the remote node prepared the data (kernel vs. user) will get us absolutely slapped by the Linux crowd. I'm gonna duck -- you're on your own ;-) - In general, it seems that this Private Data protocol is getting overloaded with requirements that extend beyond identiciation of the 4-tuple. I am suggesting that OpenIB apps should exchange this kind of info in a different way instead of establishing a Private Data protocol generic to all ULP. BTW: I don't disagree that a CM client shouldn't need to be worried about incoming requests from non-RDMA clients. Quoting Caitlin Bestler <[EMAIL PROTECTED]>: [caitlin] Comments inline [/caitlin] -Original Message- From: Tom Tucker [mailto:[EMAIL PROTECTED] Sent: Thu 10/20/2005 9:26 PM To: Caitlin Bestler Cc: Sean Hefty; Fab Tillier; [EMAIL PROTECTED]; openib-general@openib.org Subject: Re: [openib-general] Semantics of transport neutral connection establishment (was Re: [swg] Re: private data...) I think this is a useful discussion, however, I would point out that some of the information being exchanged doesn't have to be in the private data. It could be exchanged in an untagged send/recv after the connection is established; which has the benefit of allowing the application to use an arbitrarily large chunk of data to authenticate and authorize the remote peer instead of trying to boil the ocean in 64B of data. It might be better to only solve the core problem ... which I think is identifying the service and QP configuration. On Thu, 2005-10-20 at 14:58 -0700, Caitlin Bestler wrote: I believe a review of what the implementer of a transport neutral daemon for an RDMA protocol would be expecting from a Connection Management service: -- It expects that it can listen for connection requests on a specific 16-bit port number (with traditional TCP port number semantics) on either a specific IP Address or for all IP Addresses associated with the network device. I would expand this to say "...or on all interfaces that have an IP address". From my reading, it seems to me that the current CMA does this. -- It will receive connection requests that were initiated by active peers that wish to establish a reliable connection for the purpose of exchanging RDMA messges. This Connection Request will identify: 1) The remote IP Address of the active peer. This will beauthenticated in the sense that the address is known to have more meaning than just being a value made up by a remote user-mode peer. If it is a lie then privileged software is complicit in the lie. The address may be even more authenticated than that. Are you saying that it should not be possible for a user mode peer to masquerade as another host? If this is what you're saying, then I don't think it is any more secure done in the kernel than in user mode because the remote peer has no way of knowing where the data was prepared. I think that if authentication is the purpose of the remote address, don't bother. If the active peer needs to be authenticated, do it after connection establishment when you can exchange signatures of sufficient size to be useful. Am I missing something here? [caitlin] I'm merely noting what a TCP daemon is able to assume today as part of IP connection setup. The remote IP address supplied may be heavily authenticated (the local network actively prevents IP spoofing by checking routes, etc.) or next to worthless (the only guarantee si that the forger had root access on some machine). Regardless of whether this level of authentication is advisable, it is *exactly* the assumption that many servers make today. This includes many NFS configurations. When the network is isolated from external connections, and the network administrator is confident that they control "root" for all machines within the local network this can be a signifigant level of defense. Within a corporate intranet, for example, this may be the mechanism to ensure that marketing does not examine internla engineering documents. I wouldn't recommend it for protecting HR files, but it can be quite adequate for many purp
Re: [openib-general] configuring ipoib
On Fri, 2005-10-21 at 11:02, Kanevsky, Arkady wrote: > How do you configure ipoib? > I used "ifconfig ib0 ip_address" which works fine. > But if I have several ports on an HCA how do I specify which port > ip_address should be associated with? > Ditto if you have multiple cards. HCA port 0 = ib0 HCA port 1 = ib1 I think additional HCAs would be: ib2/ib3, etc. -- Hal > Thanks, > > > Arkady Kanevsky email: [EMAIL PROTECTED] > > Network Appliance phone: 781-768-5395 > > 375 Totten Pond Rd. Fax: 781-895-1195 > > Waltham, MA 02451-2010 central phone: 781-768-5300 > > > > > > > __ > > ___ > openib-general mailing list > openib-general@openib.org > http://openib.org/mailman/listinfo/openib-general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Adding static entry to arp table?
I've used it for Ethernet MAC addresses. I haven't tried it for non-ARPHRD_ETHERNET types. Quoting Abhijit Gadgil <[EMAIL PROTECTED]>: There is no need for the API, I am using the 'ip neigh' command, because the 'arp -s' has got some issues with the hardware address length. Does 'arp -s' actually work? -abhijit On Fri, 2005-10-21 at 19:50, [EMAIL PROTECTED] wrote: Do you need it to be an API, or can you just use the arp -s command? Quoting Abhijit Gadgil <[EMAIL PROTECTED]>: > Hi all, > > Is there a patch (to ip utility or Linux kernel), which can add static > entry to the arp table using ip neigh command? I am using gen1 based > stack, but didn't find anything after a 'grep' in the gen2 stack as > well? any pointers? > > Thanks and regards. > > -abhijit > > > ___ > openib-general mailing list > openib-general@openib.org > http://openib.org/mailman/listinfo/openib-general > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] configuring ipoib
Title: Message How do you configure ipoib? I used "ifconfig ib0 ip_address" which works fine. But if I have several ports on an HCA how do I specify which port ip_address should be associated with? Ditto if you have multiple cards. Thanks, Arkady Kanevsky email: [EMAIL PROTECTED] Network Appliance phone: 781-768-5395 375 Totten Pond Rd. Fax: 781-895-1195 Waltham, MA 02451-2010 central phone: 781-768-5300 ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Re: rdma_bind_addr question
I think we can simplify this without changing the format of how the IPoIB mac addr is stored. assume the IPoIB netdevs have type ARPHRD_INFINIBAND, and iWARP netdevs are type ARPHRD_ETHERNET. The CMA would search only the IWARP openib devices and compare the 6 byte MAC address if the netdev type is ETHERNET, and search only the IB openib devices matching on the full 20 byte IPoIB mac addr. Doing this removes the possibility that a 6 byte ethernet mac addr accidentally matches the first 6 bytes of the IPoIB mac addr... Steve. - Original Message - From: "Tom Tucker" <[EMAIL PROTECTED]> To: "Sean Hefty" <[EMAIL PROTECTED]> Cc: Sent: Thursday, October 20, 2005 11:41 PM Subject: [openib-general] Re: rdma_bind_addr question On Thu, 2005-10-20 at 13:09 -0700, Sean Hefty wrote: Tom Tucker wrote: > BTW -- I think this means that we need an ARPHRD_IWARP type. It may be that the CMA can simply look in its local device list for an ib_device that has a given MAC address. After pondering this, I think you're correct. There is one issue, however. Currently, the GID is stored beginning at the fourth byte of the dev_addr for IBoIB, but the Ethernet MAC address begins at byte 0. Is it possible to move this 4B quantity to follow the GID? If so, we could pad the dev_addr for iWARP devices with zeroes and use the exact same code to search the cma device table. If the device is found, it already has a type in the ib_device structure to distinguish between IB and iWARP devices. If the caller gave us an IP address for a dumb Ethernet device, we would go looking for it in the cma device list and simply not find it. It would still fail, just later, and would avoid a new ARPHRD type. I don't know the detail of how iWarp will work with this. Will iWarp need a call similar to ib_translate_addr() to translate an IP address into a MAC address? Are the MAC addresses stored with the ib_device somehow? - Sean ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Adding static entry to arp table?
There is no need for the API, I am using the 'ip neigh' command, because the 'arp -s' has got some issues with the hardware address length. Does 'arp -s' actually work? -abhijit On Fri, 2005-10-21 at 19:50, [EMAIL PROTECTED] wrote: > Do you need it to be an API, or can you just use the arp -s command? > > Quoting Abhijit Gadgil <[EMAIL PROTECTED]>: > > > Hi all, > > > > Is there a patch (to ip utility or Linux kernel), which can add static > > entry to the arp table using ip neigh command? I am using gen1 based > > stack, but didn't find anything after a 'grep' in the gen2 stack as > > well? any pointers? > > > > Thanks and regards. > > > > -abhijit > > > > > > ___ > > openib-general mailing list > > openib-general@openib.org > > http://openib.org/mailman/listinfo/openib-general > > > > To unsubscribe, please visit > > http://openib.org/mailman/listinfo/openib-general > > > > > ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] $B$9$0$KEEOC$b$i$($^$;$s$+!)(B
$BF|K\:GBg5i$NBg?M$N=P2q$$!*!*(B $B-!!Z40A4L5NA![$*$3$:$+$$(BGET!! $B-"!Z=P2q$$J]>[EMAIL PROTECTED]@dBP$K2q$($k!*!*(B $B40A4(/(.(/(.(/(B $BL5NA(5(1(0(1!s!!"+EvA3$G$9!#(B $B!|(BPC$B!&7HBS%5%$%H$HO"F0%7%9%F%`(B $B!|D>%"%I!&D>EE8r49<+M3(B $B$^$:$OEEOC$9$k"M(B http://awg.webchu.com/sweet-s/?gyakuen $B$5$!!":[EMAIL PROTECTED]&=P$9$k$N$O$"$J$?$NM&5$$H7hCG$N$_!#(B $B0lDL$N%a!<%k$G$"$J$?$N1?L?$rJQ$($h$&!*(B $BH~?M7O!"2D0&$$7O!"$*;P7O$,B?$$$7!"!z!T9b3[=jF@http://awg.webchu.com/sweet-s/?gyakuen $B!a!a!a!a!a!a!a!a!a!a!a!a!a!a!a!a!a!a!a(B NO.I don't veceive your mail [EMAIL PROTECTED] $B:#8e!"l9g$O(B [EMAIL PROTECTED] $B!a!a!a!a!a!a!a!a!a!a!a!a!a!a!a!a!a!a!a(B ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Adding static entry to arp table?
Do you need it to be an API, or can you just use the arp -s command? Quoting Abhijit Gadgil <[EMAIL PROTECTED]>: Hi all, Is there a patch (to ip utility or Linux kernel), which can add static entry to the arp table using ip neigh command? I am using gen1 based stack, but didn't find anything after a 'grep' in the gen2 stack as well? any pointers? Thanks and regards. -abhijit ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] Semantics of transport neutral connection establishment (was Re: [swg] Re: private data...)
[caitlin] Comments inline [/caitlin] -Original Message- From: Tom Tucker [mailto:[EMAIL PROTECTED] Sent: Thu 10/20/2005 9:26 PM To: Caitlin Bestler Cc: Sean Hefty; Fab Tillier; [EMAIL PROTECTED]; openib-general@openib.org Subject: Re: [openib-general] Semantics of transport neutral connection establishment (was Re: [swg] Re: private data...) I think this is a useful discussion, however, I would point out that some of the information being exchanged doesn't have to be in the private data. It could be exchanged in an untagged send/recv after the connection is established; which has the benefit of allowing the application to use an arbitrarily large chunk of data to authenticate and authorize the remote peer instead of trying to boil the ocean in 64B of data. It might be better to only solve the core problem ... which I think is identifying the service and QP configuration. On Thu, 2005-10-20 at 14:58 -0700, Caitlin Bestler wrote: > I believe a review of what the implementer of a transport neutral > daemon for an RDMA protocol would be expecting from a Connection > Management service: > > -- It expects that it can listen for connection requests on a specific >16-bit port number (with traditional TCP port number semantics) on >either a specific IP Address or for all IP Addresses associated with >the network device. I would expand this to say "...or on all interfaces that have an IP address". From my reading, it seems to me that the current CMA does this. > -- It will receive connection requests that were initiated by active peers >that wish to establish a reliable connection for the purpose of >exchanging RDMA messges. > >This Connection Request will identify: > 1) The remote IP Address of the active peer. > This will beauthenticated > in the sense that the address is known to have more meaning than > just being a value made up by a remote user-mode peer. If it is a > lie then privileged software is complicit in the lie. The address may be > even more authenticated than that. Are you saying that it should not be possible for a user mode peer to masquerade as another host? If this is what you're saying, then I don't think it is any more secure done in the kernel than in user mode because the remote peer has no way of knowing where the data was prepared. I think that if authentication is the purpose of the remote address, don't bother. If the active peer needs to be authenticated, do it after connection establishment when you can exchange signatures of sufficient size to be useful. Am I missing something here? [caitlin] I'm merely noting what a TCP daemon is able to assume today as part of IP connection setup. The remote IP address supplied may be heavily authenticated (the local network actively prevents IP spoofing by checking routes, etc.) or next to worthless (the only guarantee si that the forger had root access on some machine). Regardless of whether this level of authentication is advisable, it is *exactly* the assumption that many servers make today. This includes many NFS configurations. When the network is isolated from external connections, and the network administrator is confident that they control "root" for all machines within the local network this can be a signifigant level of defense. Within a corporate intranet, for example, this may be the mechanism to ensure that marketing does not examine internla engineering documents. I wouldn't recommend it for protecting HR files, but it can be quite adequate for many purposes. More importantly, if this guarantee is not provided then an explicit warning should be made. For example, unless the CM header itself is mariked as having IP data in it there is no way to know that a user mode application simply hasn't made up an IP address and submitted as part of a normal CM requests private data. [/caitlin] > 2) The destination IP address that the active peer requested. That > is, if the network device supports multiple addresses concurrently (as with > a > web farm) the connection request will identify *which* address was > specified by the remote active peer. ... and port. I agree this is needed because it is part of the "local service signature" aka service id aka port number/ip address" [caitlin] But the port was implicit in the listen. There is no need for it to be part of the Connection Request reported to the listener. If there is a desire to conserve Service IDs then the wire protocol could certainly include the TCP port in the CM Private Data. But that would be transparent to the application. The transparency to the applicaion is my concern. I'll confess to not seeing any great need to avoid allocating 64K Service IDs out of a 64-bit range -- but that's an issue for the IB developers to work out. [/caitlin] > 3) Private Data supplied by the remote peer to establish its > identity, IMHO, private data is useless (or at least
[openib-general] $B!y=EMW!y(B
$B(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(B $B(.(#(#(B $BG.$$$4MWK>$K$*1~$($7$F$D$$$KL5NA%]%$%s%HBgI}(BUP$B!*(B $B(.(.(#(B $BATG/$NJ}$K$b$4G<[EMAIL PROTECTED]:$1$kM%NI=d$j2q$$%5%$%H(B $B(.(.(.!!(BSWEETS $B!A:#:G$bG.$$!*AG?M!&?M:J!&5U1g!&=w;R9;@8(B^^$B;D=kCf$KBg9??e!*!A(B $B(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(B $B!]!&!]!&!]!&!]!&!]!&!]!&!]!&!]!&!]!&!]!&!]!&!]!&!]!&!]!&!]!&!]!&!]!&!]!&!]!&!]!&(B $B(.(.(.!!!V(B1$B1_!WJ,L5NA%]%$%s%HB#Dh%-%c%s%Z!<%sx$7=k$$F|!9$,B3$-$^$9$,!"$*Hh$l$G$O$"$j$^$;$s$+!)(B $B!VL~$5$l$?$$!W(B [EMAIL PROTECTED](BH$B$J=P2q$$$,L5$$!W(B $B!V$*6b$,L5$$!W(B $BEy!9$*G:$_$G$O$"$j$^$;$s$+!)(B $B$?$@:#!"[EMAIL PROTECTED];o9-9pBgNL7G:\Cf$K$D$-!"6/NO$K$"$J$?$NM_K>$rK~$?$9=P2q$$$r;Y1gCW$7$^$9!*(B $B"(!V%5%/%i!W(B $B!V6Hl$rDs6!$7$F$*$j$^$9!#(B $B"(A4$F%*!<%W%s$K$7$F$*$j$^$9$N$G0B?4$7$F$*3Z$7$_2<$5$$!#(B $B:#$9$0EPO?$7$FD:$$$?J}$K$O!";O$a$K(B1$B1_J,$N%]%$%s%H$rL5NA$G:9$7>e$2$F$*$j$^$9"v(B $BEPO?$O$3$A$i"*(B http://www.00-love5.com/?0yen $B!y(B*$B!&(B*$B!y(B*$B!&(B*$B!y(B*$B!&(B*$B!y(B*$B!&(B*$B!y(B*$B!&(B*$B!y(B*$B!&(B*$B!y(B*$B!&(B*$B!y(B*$B!&!y(B*$B!&(B*$B!y(B*$B!&(B*$B!y(B $B!}L5NA%]%$%s%H$GAjEvM7$Y$^$9$N$G@'Hs$*;n$72<$5$$"v(B $B!}$[$H$s$I$NJ}$,L5NA%]%$%s%HFb$G!"[EMAIL PROTECTED](BGET$B$7$F$^$9!*!*(B $B!};HMQ$7$F$_$F!V$3$l$O!*!W$H;W$C$FD:$$$?J}$N$_M-NA$X$*?J$_2<$5$$!*(B $B;n$7$F$_$k!*"*(B http://www.00-love5.com/?0yen $B!]!&!]!&!]!&!]!&!]!&!]!&!]!&!]!&!]!&!]!&!]!&!]!&!]!&!]!&!]!&!]!&!]!&!]!&!]!&!]!&(B $B$b$7!"5.J}$,AGE($J=P2q$$$r5a$a$F$$$i$C$7$c$i$J$1$l$P$*OM$S?=$7>e$2$^$9!#(B $B$*http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] Adding static entry to arp table?
Hi all, Is there a patch (to ip utility or Linux kernel), which can add static entry to the arp table using ip neigh command? I am using gen1 based stack, but didn't find anything after a 'grep' in the gen2 stack as well? any pointers? Thanks and regards. -abhijit ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] $B%;%l%V$N7hDjHG!*(B
$B!g!g!g!g!y!g!g!g!g!g!y!g!g!g!g!g!g!g!g!g!y!g!g!g!g!g!y!g!g!g!g!g(B $B!c(BNO.I don't veceive your mail$B!d"M!!([EMAIL PROTECTED] $B!c:#8e!"l9g$O!d"M!!([EMAIL PROTECTED] $B!g!g!g!g!y!g!g!g!g!g!y!g!g!g!g!g!g!g!g!g!y!g!g!g!g!g!y!g!g!g!g!g(B ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] $B%(%C%A$G2T$0(B
$B"!!!:#2s$N!Z>R2pNA![!ZF~2qHqMQ![$OA4$FL5NA$G$9!#EPO?8eH/@8$9$k;[EMAIL PROTECTED];$s!#(B $B"!!!DL>o!Z(B2,000$B1_J,![$NL5NA%]%$%s%H$r"(!Z(B10,000$B1_J,![$HCW$7$^$9!#(B $B"!!!5U1g=u4uK>[EMAIL PROTECTED]:GDc(B3$BK|1_0J>e$,3NDj$5$l$F$$$kJ}$N$_$4>R2pCW$7$^$9!#(B $B"!!!0lH/[EMAIL PROTECTED])$J$i$J$/$F$b!":G?7>pJs$r?o;~99?78e>R2p$5$;$FD:$-$^$9!#(B $B"!!!pJs0lMw$r4QMw$G$-$^$9!#(B $B!c$*;n$7!d$4F~2q$NJ}$O"M(B http://1191.jp/buzz/index.html $B"(=EMW"((B $B!|0lK|1_L5NA(BP$B$G0l%v7n$[$IMxMQ2DG=$G$9!#!JM>M5$G$9!#!K(B $B!|>e5-!Z%Z!<%8![$,I=<($5$l$J$+$C$?>l9g$O!L8"Mx=*N;!M$H$J$C$F$*$j$^$9$N$G!"0lHLF~2q%Z!<%8!Z![$r$4MxMQ2<$5$$!#(B $B$=$NBe$o$j$K5.J}MM$N!L5U!oFCJL8"Mx!M<[EMAIL PROTECTED]"Mx$,0\9T!W$7$F$7$^$$$^$9$N$GM=$a$4N;>52<$5$$!#(B http://1191.jp/buzz/ $B!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g!g(B I don't veceive your mail [EMAIL PROTECTED] $B%a!<%k$Nhttp://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
