Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey
Sean Hefty wrote: Note that since the HCA validates the pkey in the in coming packet, no matter what the IB SW would do, partial members of a partition can't talk to each other. So the approach taken by the core/ipoib code was to just ignore the MSb in places where the code looks for the pkey --index-- and use the full member pkey when forming MGIDs. This seems fine to me. My concern is that ib_find_cached_pkey() returns an index to a pkey that wasn't the one in the search. Can this lead to a QP being configured in such a way that communication with a remote QP would silently fail? My understanding is that when an IPoIB broadcast domain contains both partial and full members (*) attempts to communicate between two partial members would silently fail, does this silence is something you think we should work to change? (*) eg when you have bunch or clients and a server or bunch of servers and you don't want to allow --clients-- to communicate among themselves) I'm not against this patch, but I want to make sure that I understand the issues, so we're not creating a work-around solution. The patch is against the librdmacm, yet there's nothing that I see in the librdmacm that makes me think it's behaving incorrectly. My thinking is that if in the end of this thread we are willing to move forward without changing ib_find_cached_pkey() then this patch should be merged. Or. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] anyone have OFED 1.2 alpha1 compiling on ppc64
Quoting Scott Weitzenkamp (sweitzen) [EMAIL PROTECTED]: Subject: anyone have OFED 1.2 alpha1 compiling on ppc64 I tried both RHEL4 and SLES10 usinstall.sh, and get this. I filed bug 379, anyone else tried ppc64? Scott, could pls you upload the kernel sources and .config files to staging? If you do, we'll be able to add these to mightly cross-build environment. -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] Airtist Telecharger vos MP3 sans DRM a partir de 0, 2�
Emailin english : Si vous avez des difficultees pour visualiser cette page , cliquezici Tlchargezvos musiques en mp3 sans DRM Rejoignez vos artistes dans la communaut! Bonjour, Airtist est le nouveau site Internet communautaire de tlchargement musical. Sur Airtist vous pouvez dcouvrir et tlcharger de la musique et rejoindre la communaut d'internautes et d'artistes: http://www.airtist.com Avec Airtist tlchargez vos musiques en mp3 sans DRM Accdez une communaut musicale avec des artistes indpendants de tous styles musicaux et de tous pays, tlchargez vos musiques avec 2 modes: Payant prix variable dtermin par l'artiste ( partir de 0,20euro le titre) Gratuit et Lgal base d'annonces publicitaires (bientt disponible) Libert des titres que vous tlchargez, compatibles avec tous les lecteurs et baladeurs mp3. Rejoignez vos artistes dans la communaut Crez une page web personnelle Airtist: Blog, albums photos, votes, commentaires, messagerie, rseaux d'amis, musiques, concours, etc. Solidarit avec les associations A chaque tlchargement d'une musique 1 centime est revers l'association ou uvre caritatives de ton choix comme Sol en Si, Fondation Nicolas Hulot, Restos du Coeur etc. Tu es artiste ou tu fais parti d'un groupe ? Tu fais de la musique et tu souhaites la distribuer sur Airtist et tre pay? Inscription artiste Airtist.com c'est plus de 5000 musiques tlcharger dans tous les styles musicaux, artistes connus, indpendants et autoproduits et de tous pays. Vous pouvez tlcharger les albums de Henri Salvador,Anas, Bloc Party, Nosfell, Yann Tiersen Shannon Wright,Higelin et de centaines d'autres artistes connus et indpendantsde la scne montante franaise et internationale. Airtist est une plateforme de tlchargement lgale en partenariat avec la SACEM. Pour vous desabonner de cette liste, cliquez sur desinscription powered by eoxiamail v 2.10.4; ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] ofa_1_2_kernel 20070222-0200 daily build status
This email was generated automatically, please do not reply Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-core-mod --with-addr_trans-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.12 Passed on i686 with linux-2.6.19 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.19 Passed on powerpc with linux-2.6.19 Passed on powerpc with linux-2.6.18 Passed on x86_64 with linux-2.6.18 Passed on powerpc with linux-2.6.17 Passed on x86_64 with linux-2.6.13 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.12 Passed on x86_64 with linux-2.6.15 Passed on ia64 with linux-2.6.15 Passed on x86_64 with linux-2.6.14 Passed on ppc64 with linux-2.6.18 Passed on powerpc with linux-2.6.12 Passed on powerpc with linux-2.6.14 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on ppc64 with linux-2.6.12 Passed on powerpc with linux-2.6.16 Passed on ppc64 with linux-2.6.19 Passed on ppc64 with linux-2.6.16 Passed on ia64 with linux-2.6.19 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.14 Passed on powerpc with linux-2.6.15 Passed on powerpc with linux-2.6.13 Passed on ppc64 with linux-2.6.14 Passed on ia64 with linux-2.6.12 Passed on ia64 with linux-2.6.16 Passed on ppc64 with linux-2.6.13 Passed on ppc64 with linux-2.6.15 Passed on ppc64 with linux-2.6.17 Passed on ia64 with linux-2.6.13 Passed on ia64 with linux-2.6.18 Passed on x86_64 with linux-2.6.9-42.ELsmp Failed: Build failed on ia64 with linux-2.6.16.21-0.8-default Log: /home/vlad/tmp/ofa_1_2_kernel-20070222-0200_linux-2.6.16.21-0.8-default_ia64_check/include/rdma/ib_verbs.h:1590: error: implicit declaration of function âsg_dma_lenâ /home/vlad/tmp/ofa_1_2_kernel-20070222-0200_linux-2.6.16.21-0.8-default_ia64_check/drivers/infiniband/core/addr.c: At top level: /home/vlad/tmp/ofa_1_2_kernel-20070222-0200_linux-2.6.16.21-0.8-default_ia64_check/drivers/infiniband/core/addr.c:61: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_2_kernel-20070222-0200_linux-2.6.16.21-0.8-default_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_kernel-20070222-0200_linux-2.6.16.21-0.8-default_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_kernel-20070222-0200_linux-2.6.16.21-0.8-default_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_kernel-20070222-0200_linux-2.6.16.21-0.8-default_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.16.21-0.8-default' make: *** [kernel] Error 2 -- Build failed on x86_64 with linux-2.6.9-22.ELsmp Log: /home/vlad/tmp/ofa_1_2_kernel-20070222-0200_linux-2.6.9-22.ELsmp_x86_64_check/drivers/net/cxgb3/vsc8211.c:167: error: âADVERTISE_PAUSE_CAPâ undeclared (first use in this function) /home/vlad/tmp/ofa_1_2_kernel-20070222-0200_linux-2.6.9-22.ELsmp_x86_64_check/drivers/net/cxgb3/vsc8211.c:167: error: (Each undeclared identifier is reported only once /home/vlad/tmp/ofa_1_2_kernel-20070222-0200_linux-2.6.9-22.ELsmp_x86_64_check/drivers/net/cxgb3/vsc8211.c:167: error: for each function it appears in.) /home/vlad/tmp/ofa_1_2_kernel-20070222-0200_linux-2.6.9-22.ELsmp_x86_64_check/drivers/net/cxgb3/vsc8211.c:170: error: âADVERTISE_PAUSE_ASYMâ undeclared (first use in this function) make[3]: *** [/home/vlad/tmp/ofa_1_2_kernel-20070222-0200_linux-2.6.9-22.ELsmp_x86_64_check/drivers/net/cxgb3/vsc8211.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_2_kernel-20070222-0200_linux-2.6.9-22.ELsmp_x86_64_check/drivers/net/cxgb3] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_kernel-20070222-0200_linux-2.6.9-22.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-22.ELsmp' make: *** [kernel] Error 2 -- Build failed on x86_64 with linux-2.6.9-34.ELsmp Log: /home/vlad/tmp/ofa_1_2_kernel-20070222-0200_linux-2.6.9-34.ELsmp_x86_64_check/drivers/net/cxgb3/cxgb3_offload.c: In function âadd_adapterâ: /home/vlad/tmp/ofa_1_2_kernel-20070222-0200_linux-2.6.9-34.ELsmp_x86_64_check/drivers/net/cxgb3/cxgb3_offload.c:1062: error: âadapter_list_lockâ undeclared (first use in this function) /home/vlad/tmp/ofa_1_2_kernel-20070222-0200_linux-2.6.9-34.ELsmp_x86_64_check/drivers/net/cxgb3/cxgb3_offload.c: In function âremove_adapterâ: /home/vlad/tmp/ofa_1_2_kernel-20070222-0200_linux-2.6.9-34.ELsmp_x86_64_check/drivers/net/cxgb3/cxgb3_offload.c:1069
[openib-general] ofa_1_2_kernel 20070222-0251 daily build status
This email was generated automatically, please do not reply Common build parameters: --with-ipoib-mod --with-sdp-mod --with-srp-mod --with-user_mad-mod --with-user_access-mod --with-mthca-mod --with-core-mod --with-addr_trans-mod --with-cxgb3-mod Passed: Passed on i686 with 2.6.15-23-server Passed on i686 with linux-2.6.14 Passed on i686 with linux-2.6.17 Passed on i686 with linux-2.6.18 Passed on i686 with linux-2.6.19 Passed on i686 with linux-2.6.13 Passed on i686 with linux-2.6.16 Passed on i686 with linux-2.6.15 Passed on i686 with linux-2.6.12 Passed on x86_64 with linux-2.6.20 Passed on x86_64 with linux-2.6.19 Passed on powerpc with linux-2.6.19 Passed on powerpc with linux-2.6.18 Passed on x86_64 with linux-2.6.18 Passed on x86_64 with linux-2.6.16 Passed on x86_64 with linux-2.6.17 Passed on x86_64 with linux-2.6.14 Passed on x86_64 with linux-2.6.12 Passed on powerpc with linux-2.6.17 Passed on x86_64 with linux-2.6.13 Passed on x86_64 with linux-2.6.15 Passed on ia64 with linux-2.6.13 Passed on ppc64 with linux-2.6.12 Passed on ia64 with linux-2.6.18 Passed on powerpc with linux-2.6.16 Passed on powerpc with linux-2.6.14 Passed on ppc64 with linux-2.6.14 Passed on powerpc with linux-2.6.12 Passed on ia64 with linux-2.6.15 Passed on powerpc with linux-2.6.13 Passed on ia64 with linux-2.6.16 Passed on ppc64 with linux-2.6.19 Passed on powerpc with linux-2.6.15 Passed on ppc64 with linux-2.6.18 Passed on x86_64 with linux-2.6.9-42.ELsmp Passed on ppc64 with linux-2.6.17 Passed on x86_64 with linux-2.6.16.21-0.8-smp Passed on ia64 with linux-2.6.12 Passed on ppc64 with linux-2.6.15 Passed on ia64 with linux-2.6.19 Passed on ppc64 with linux-2.6.13 Passed on ia64 with linux-2.6.17 Passed on ia64 with linux-2.6.14 Passed on ppc64 with linux-2.6.16 Failed: Build failed on ia64 with linux-2.6.16.21-0.8-default Log: /home/vlad/tmp/ofa_1_2_kernel-20070222-0251_linux-2.6.16.21-0.8-default_ia64_check/include/rdma/ib_verbs.h:1590: error: implicit declaration of function 'sg_dma_len' /home/vlad/tmp/ofa_1_2_kernel-20070222-0251_linux-2.6.16.21-0.8-default_ia64_check/drivers/infiniband/core/addr.c: At top level: /home/vlad/tmp/ofa_1_2_kernel-20070222-0251_linux-2.6.16.21-0.8-default_ia64_check/drivers/infiniband/core/addr.c:61: warning: initialization from incompatible pointer type make[4]: *** [/home/vlad/tmp/ofa_1_2_kernel-20070222-0251_linux-2.6.16.21-0.8-default_ia64_check/drivers/infiniband/core/addr.o] Error 1 make[3]: *** [/home/vlad/tmp/ofa_1_2_kernel-20070222-0251_linux-2.6.16.21-0.8-default_ia64_check/drivers/infiniband/core] Error 2 make[2]: *** [/home/vlad/tmp/ofa_1_2_kernel-20070222-0251_linux-2.6.16.21-0.8-default_ia64_check/drivers/infiniband] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_kernel-20070222-0251_linux-2.6.16.21-0.8-default_ia64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/ia64/linux-2.6.16.21-0.8-default' make: *** [kernel] Error 2 -- Build failed on x86_64 with linux-2.6.9-34.ELsmp Log: /home/vlad/tmp/ofa_1_2_kernel-20070222-0251_linux-2.6.9-34.ELsmp_x86_64_check/drivers/net/cxgb3/cxgb3_offload.c: In function 'add_adapter': /home/vlad/tmp/ofa_1_2_kernel-20070222-0251_linux-2.6.9-34.ELsmp_x86_64_check/drivers/net/cxgb3/cxgb3_offload.c:1062: error: 'adapter_list_lock' undeclared (first use in this function) /home/vlad/tmp/ofa_1_2_kernel-20070222-0251_linux-2.6.9-34.ELsmp_x86_64_check/drivers/net/cxgb3/cxgb3_offload.c: In function 'remove_adapter': /home/vlad/tmp/ofa_1_2_kernel-20070222-0251_linux-2.6.9-34.ELsmp_x86_64_check/drivers/net/cxgb3/cxgb3_offload.c:1069: error: 'adapter_list_lock' undeclared (first use in this function) make[3]: *** [/home/vlad/tmp/ofa_1_2_kernel-20070222-0251_linux-2.6.9-34.ELsmp_x86_64_check/drivers/net/cxgb3/cxgb3_offload.o] Error 1 make[2]: *** [/home/vlad/tmp/ofa_1_2_kernel-20070222-0251_linux-2.6.9-34.ELsmp_x86_64_check/drivers/net/cxgb3] Error 2 make[1]: *** [_module_/home/vlad/tmp/ofa_1_2_kernel-20070222-0251_linux-2.6.9-34.ELsmp_x86_64_check] Error 2 make[1]: Leaving directory `/home/vlad/kernel.org/x86_64/linux-2.6.9-34.ELsmp' make: *** [kernel] Error 2 -- Build failed on x86_64 with linux-2.6.9-22.ELsmp Log: /home/vlad/tmp/ofa_1_2_kernel-20070222-0251_linux-2.6.9-22.ELsmp_x86_64_check/drivers/net/cxgb3/vsc8211.c:167: error: 'ADVERTISE_PAUSE_CAP' undeclared (first use in this function) /home/vlad/tmp/ofa_1_2_kernel-20070222-0251_linux-2.6.9-22.ELsmp_x86_64_check/drivers/net/cxgb3/vsc8211.c:167: error: (Each undeclared identifier is reported only once /home/vlad/tmp/ofa_1_2_kernel-20070222-0251_linux-2.6.9-22.ELsmp_x86_64_check/drivers/net/cxgb3/vsc8211.c:167: error: for each function it appears in.) /home/vlad/tmp/ofa_1_2_kernel-20070222-0251_linux-2.6.9-22.ELsmp_x86_64_check/drivers/net/cxgb3/vsc8211.c:170: error
Re: [openib-general] [PATCH 1/2] ofed_1_2 Fix copyrights in the cxgb3 driver.
On Thu, 2007-02-15 at 13:59 -0600, Steve Wise wrote: Fix copyrights in the cxgb3 driver. Remove the Open Grid Computing copyright. It shouldn't be there. Signed-off-by: Steve Wise [EMAIL PROTECTED] --- Applied. -- Vladimir Sokolovsky [EMAIL PROTECTED] Mellanox Technologies Ltd. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH 2/2] ofed_1_2 Fix copyrights in the iw_cxgb3 driver.
On Thu, 2007-02-15 at 14:00 -0600, Steve Wise wrote: Fix copyrights in the iw_cxgb3 driver. Remove the Open Grid Computing copyright. It shouldn't be there. Signed-off-by: Steve Wise [EMAIL PROTECTED] --- Applied. -- Vladimir Sokolovsky [EMAIL PROTECTED] Mellanox Technologies Ltd. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [ewg] anyone have OFED 1.2 alpha1 compiling on ppc64
On Thu, 2007-02-22 at 02:11, Scott Weitzenkamp (sweitzen) wrote: I tried both RHEL4 and SLES10 usinstall.sh, and get this. I filed bug 379, anyone else tried ppc64? gcc -DHAVE_CONFIG_H -I. -I. -I. -I./include/infiniband -I./../libibcommon/incl\ ude/infiniband -Wall -m64 -g -O2 -MT libibumad_la-umad.lo -MD -MP -MF .deps/lib\ ibumad_la-umad.Tpo -c src/umad.c -fPIC -DPIC -o .libs/libibumad_la-umad.o In file included from src/umad.c:50: ./include/infiniband/umad.h:37:31: infiniband/common.h: No such file or directo\ ry src/umad.c: In function `port_alloc': src/umad.c:94: warning: implicit declaration of function `IBWARN' src/umad.c: In function `get_port': src/umad.c:160: warning: implicit declaration of function `snprintf' src/umad.c:163: warning: implicit declaration of function `sys_read_uint' src/umad.c:177: warning: implicit declaration of function `sys_read_uint64' src/umad.c:182: warning: implicit declaration of function `sys_read_gid' src/umad.c: In function `get_ca': src/umad.c:354: warning: implicit declaration of function `sys_read_string' src/umad.c:363: warning: implicit declaration of function `sys_read_guid' make[3]: *** [libibumad_la-umad.lo] Error 1 make[3]: Leaving directory `/var/tmp/OFEDRPM/BUILD/ofa_user-1.2/src/userspace/m\ anagement/libibumad' make[2]: *** [all-recursive] Error 1 make[2]: Leaving directory `/var/tmp/OFEDRPM/BUILD/ofa_user-1.2/src/userspace/m\ anagement/libibumad' make[1]: *** [all] Error 2 make[1]: Leaving directory `/var/tmp/OFEDRPM/BUILD/ofa_user-1.2/src/userspace/m\ anagement/libibumad' make: *** [subdirs] Error 1 make: Leaving directory `/var/tmp/OFEDRPM/BUILD/ofa_user-1.2/src/userspace/mana\ gement' That missing header (common.h) is in libibcommon. Somehow, libibcommon is not installed. libibumad depends on libibcommon. Is this a build/install script issue with OFED 1.2 ? Vlad ? -- Hal Scott Weitzenkamp SQA and Release Manager Server Virtualization Business Unit Cisco Systems __ ___ ewg mailing list [EMAIL PROTECTED] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH ofed_1_2] iw_cxgb3: Stop the EP Timer on BAD CLOSE.
On Wed, 2007-02-21 at 14:46 -0600, Steve Wise wrote: Stop the ep timer in ec_status() if the status indicates a bad close. Signed-off-by: Steve Wise [EMAIL PROTECTED] --- drivers/infiniband/hw/cxgb3/iwch_cm.c |1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/drivers/infiniband/hw/cxgb3/iwch_cm.c b/drivers/infiniband/hw/cxgb3/iwch_cm.c index e5442e3..d00e5dd 100644 --- a/drivers/infiniband/hw/cxgb3/iwch_cm.c +++ b/drivers/infiniband/hw/cxgb3/iwch_cm.c @@ -1635,6 +1635,7 @@ static int ec_status(struct t3cdev *tdev printk(KERN_ERR MOD %s BAD CLOSE - Aborting tid %u\n, __FUNCTION__, ep-hwtid); + stop_ep_timer(ep); attrs.next_state = IWCH_QP_STATE_ERROR; iwch_modify_qp(ep-com.qp-rhp, ep-com.qp, IWCH_QP_ATTR_NEXT_STATE, Applied. -- Vladimir Sokolovsky [EMAIL PROTECTED] Mellanox Technologies Ltd. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] libibverbs: can't compile more than once due to man3 symbolic links
The code below was just added to libibverbs/Makefile.am install-data-hook: cd $(DESTDIR)$(mandir)/man3 \ $(LN_S) ibv_get_async_event.3 ibv_ack_async_event.3 \ This creates a problem when re-compiling/re-installing libibverbs -- the ln -s ( = $(LN_S) ) fails because the symbolic links still exist in the man/man3 directory. I rummaged around the libtool documentation, and there is no pre-defined macro which does ln -fs (which would just overwrite the current links). Any ideas on how to fix this problem cleanly (i.e., without violating spirit of libtool/automake)? - Jack ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] libibverbs: can't compile more than once due to man3 symbolic links
Is there a reason not to use man_MANS = ibv_get_async_event.3 ? On Feb 22, 2007, at 7:32 AM, Jack Morgenstein wrote: The code below was just added to libibverbs/Makefile.am install-data-hook: cd $(DESTDIR)$(mandir)/man3 \ $(LN_S) ibv_get_async_event.3 ibv_ack_async_event.3 \ This creates a problem when re-compiling/re-installing libibverbs -- the ln -s ( = $(LN_S) ) fails because the symbolic links still exist in the man/man3 directory. I rummaged around the libtool documentation, and there is no pre- defined macro which does ln -fs (which would just overwrite the current links). Any ideas on how to fix this problem cleanly (i.e., without violating spirit of libtool/automake)? - Jack ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/ openib-general -- Jeff Squyres Server Virtualization Business Unit Cisco Systems ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] libibverbs: can't compile more than once due to man3 symbolic links
Quoting Jack Morgenstein [EMAIL PROTECTED]: Subject: libibverbs: can't compile more than once due to man3 symbolic links The code below was just added to libibverbs/Makefile.am install-data-hook: cd $(DESTDIR)$(mandir)/man3 \ $(LN_S) ibv_get_async_event.3 ibv_ack_async_event.3 \ This creates a problem when re-compiling/re-installing libibverbs -- the ln -s ( = $(LN_S) ) fails because the symbolic links still exist in the man/man3 directory. I rummaged around the libtool documentation, and there is no pre-defined macro which does ln -fs (which would just overwrite the current links). Any ideas on how to fix this problem cleanly (i.e., without violating spirit of libtool/automake)? Probably just add $(RM) ibv_ack_async_event.3 \ -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] libibverbs: can't compile more than once due to man3 symbolic links
Blah -- disregard; I read the mail too quickly and didn't look at the actual Makefile.am to see what you were really asking. FWIW, the install app, by default, removes things before copying in the new target. So putting a manual rm -f in here, while klunky, has precedent and will make it work. On Feb 22, 2007, at 7:38 AM, Jeff Squyres wrote: Is there a reason not to use man_MANS = ibv_get_async_event.3 ? On Feb 22, 2007, at 7:32 AM, Jack Morgenstein wrote: The code below was just added to libibverbs/Makefile.am install-data-hook: cd $(DESTDIR)$(mandir)/man3 \ $(LN_S) ibv_get_async_event.3 ibv_ack_async_event.3 \ This creates a problem when re-compiling/re-installing libibverbs -- the ln -s ( = $(LN_S) ) fails because the symbolic links still exist in the man/man3 directory. I rummaged around the libtool documentation, and there is no pre- defined macro which does ln -fs (which would just overwrite the current links). Any ideas on how to fix this problem cleanly (i.e., without violating spirit of libtool/automake)? - Jack ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/ openib-general -- Jeff Squyres Server Virtualization Business Unit Cisco Systems ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/ openib-general -- Jeff Squyres Server Virtualization Business Unit Cisco Systems ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH] libibverbs: can't compile more than once due to man3 symbolic links
The following patch removes manpage symbolic links so that they may be relinked in the install. Suggested by Michael Tsirkin. Signed-off-by: Jack Morgenstein [EMAIL PROTECTED] diff --git a/Makefile.am b/Makefile.am index 5d2383e..455041e 100644 --- a/Makefile.am +++ b/Makefile.am @@ -70,6 +70,18 @@ dist-hook: libibverbs.spec install-data-hook: cd $(DESTDIR)$(mandir)/man3 \ + $(RM) ibv_ack_async_event.3 \ + $(RM) ibv_ack_cq_events.3 \ + $(RM) ibv_close_device.3 \ + $(RM) ibv_dealloc_pd.3 \ + $(RM) ibv_dereg_mr.3 \ + $(RM) ibv_destroy_ah.3 \ + $(RM) ibv_destroy_comp_channel.3 \ + $(RM) ibv_destroy_cq.3 \ + $(RM) ibv_destroy_qp.3 \ + $(RM) ibv_destroy_srq.3 \ + $(RM) ibv_detach_mcast.3 \ + $(RM) ibv_free_device_list.3 \ $(LN_S) ibv_get_async_event.3 ibv_ack_async_event.3 \ $(LN_S) ibv_get_cq_event.3 ibv_ack_cq_events.3 \ $(LN_S) ibv_open_device.3 ibv_close_device.3 \ ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH 0/7] cxgb3 - Chelsio T3 1G/10G driver updates
Divy, Do these need to be pulled into OFED 1.2 as well? Steve. On Thu, 2007-02-22 at 03:58 -0800, Divy Le Ray wrote: Jeff, I'm sending a series of incremental patches updating the cxgb3 driver. These patches are built against Linus'git tree. Cheers, Divy - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey
On Thu, 2007-02-22 at 02:28, Or Gerlitz wrote: Hal Rosenstock wrote: On Wed, 2007-02-21 at 15:45, Or Gerlitz wrote: On 21 Feb 2007 08:20:23 -0500, Hal Rosenstock [EMAIL PROTECTED] wrote: If the IPoIB spec does not allow both partial and full members of a partition to share a broadcast domain (eg the IPv4 broadcast group associated with the full membership pkey) or any other multicast group, burn it (or at least the relevant section). I was referring to the IB spec, not an IPoIB RFC. Can you provide a pointer? See MCMemberRecord:P_Key description in table 210 (p. 908). The OpenIB code supposed to work and as done with the RDMA CM header, the implementation should not wait for spec to be written or changed. Really ? Maybe I'm mistaken but I didn't think that OpenIB/OpenFabrics wanted to issue code which is not IBA spec compliant. The code resides in the Linux kernel, period. Linux is not under the control of this or that organization, period, period. Linux uses an hierarchic maintainship structure where Roland, Sean and yourself are listed as the maintainers, which means you are able to promote and/or block this or that agenda, go for it! OpenIB claims IBA compliance (currently mostly v1.2) and is there any good reason that we shouldn't continue to adhere to this ? -- Hal Or. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey
On Thu, 2007-02-22 at 03:04, Or Gerlitz wrote: Sean Hefty wrote: Note that since the HCA validates the pkey in the in coming packet, no matter what the IB SW would do, partial members of a partition can't talk to each other. So the approach taken by the core/ipoib code was to just ignore the MSb in places where the code looks for the pkey --index-- and use the full member pkey when forming MGIDs. This seems fine to me. My concern is that ib_find_cached_pkey() returns an index to a pkey that wasn't the one in the search. Can this lead to a QP being configured in such a way that communication with a remote QP would silently fail? My understanding is that when an IPoIB broadcast domain contains both partial and full members (*) attempts to communicate between two partial members would silently fail, An IB multicast group _cannot_ have partial members so this never should get far enough to where two limited members would be unable to communicate. -- Hal does this silence is something you think we should work to change? (*) eg when you have bunch or clients and a server or bunch of servers and you don't want to allow --clients-- to communicate among themselves) I'm not against this patch, but I want to make sure that I understand the issues, so we're not creating a work-around solution. The patch is against the librdmacm, yet there's nothing that I see in the librdmacm that makes me think it's behaving incorrectly. My thinking is that if in the end of this thread we are willing to move forward without changing ib_find_cached_pkey() then this patch should be merged. Or. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] librdmacm examples not working under OFED 1.2 alpha
I have tested RH4 U4 and to some extent also RH5 beta and see the following: under RH4 U4 - rping: addr and route resolution passing, client getting reject on conn req - udaddy: working fine on both UDP and IPOIB port spaces - mckey: not applicable on RH4 U4 till my patch with ip_ib_mc_map is merged under both udaddy and rping librdmacm report: librdmacm: couldn't read ABI version. librdmacm: assuming: 4 under RH5 = basically, the same: rping does not work, udaddy works on both port spaces. Also was able to check mckey and it works fine on both port spaces. The ABI error print is not seen. The rping client/server logs are below, Or. rping client [EMAIL PROTECTED] ~]# rping -c -v -d -a 193.168.80.175 ipaddr (193.168.80.175) librdmacm: couldn't read ABI version. librdmacm: assuming: 4 created cm_id 0x505f10 cma_event type 0 cma_id 0x505f10 (parent) cma_event type 2 cma_id 0x505f10 (parent) rdma_resolve_addr - rdma_resolve_route successful created pd 0x507830 created channel 0x506260 created cq 0x507880 created qp 0x507990 rping_setup_buffers called on cb 0x505010 allocated registered buffers... cq_thread started. cq completion failed status 5 wait for CONNECTED state 10 connect error -1 rping_free_buffers called on cb 0x505010 cma_event type 8 cma_id 0x505f10 (parent) cma event 8, error 0 rping server === [EMAIL PROTECTED] ~]# rping -s -d -v -S 100 -C 100 verbose size 100 count 100 librdmacm: couldn't read ABI version. librdmacm: assuming: 4 created cm_id 0x505f00 rdma_bind_addr successful rdma_listen ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] librdmacm examples not working under OFED 1.2 alpha
What device? On Thu, 2007-02-22 at 17:07 +0200, Or Gerlitz wrote: I have tested RH4 U4 and to some extent also RH5 beta and see the following: under RH4 U4 - rping: addr and route resolution passing, client getting reject on conn req - udaddy: working fine on both UDP and IPOIB port spaces - mckey: not applicable on RH4 U4 till my patch with ip_ib_mc_map is merged under both udaddy and rping librdmacm report: librdmacm: couldn't read ABI version. librdmacm: assuming: 4 under RH5 = basically, the same: rping does not work, udaddy works on both port spaces. Also was able to check mckey and it works fine on both port spaces. The ABI error print is not seen. The rping client/server logs are below, Or. rping client [EMAIL PROTECTED] ~]# rping -c -v -d -a 193.168.80.175 ipaddr (193.168.80.175) librdmacm: couldn't read ABI version. librdmacm: assuming: 4 created cm_id 0x505f10 cma_event type 0 cma_id 0x505f10 (parent) cma_event type 2 cma_id 0x505f10 (parent) rdma_resolve_addr - rdma_resolve_route successful created pd 0x507830 created channel 0x506260 created cq 0x507880 created qp 0x507990 rping_setup_buffers called on cb 0x505010 allocated registered buffers... cq_thread started. cq completion failed status 5 wait for CONNECTED state 10 connect error -1 rping_free_buffers called on cb 0x505010 cma_event type 8 cma_id 0x505f10 (parent) cma event 8, error 0 rping server === [EMAIL PROTECTED] ~]# rping -s -d -v -S 100 -C 100 verbose size 100 count 100 librdmacm: couldn't read ABI version. librdmacm: assuming: 4 created cm_id 0x505f00 rdma_bind_addr successful rdma_listen ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] librdmacm examples not working under OFED 1.2 alpha
Steve Wise wrote: What device? mthca ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] librdmacm examples not working under OFED 1.2 alpha
librdmacm: couldn't read ABI version. librdmacm: assuming: 4 I think there was a kernel patch from Woody to address this. Woody? -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] OFED-1.2-20070221-1741.tgz package is available
On Wed, 2007-02-21 at 16:40 -0600, Tom Tucker wrote: Bug: 355 (problems building modules that depend on OFED 1.2 modules) In order to build kernel modules depending on OFED's modules you need to take Modules.symvers file from prefix/src/openib/Modules.symvers (part of kernel-ib-devel RPM) and copy this to modules subdir and then compile your module. Won't this blow away all the version information for the non-IB symbols? See Documentation/kbuild/modules.txt (under kernel sources): --- 7.3 Symbols from another external module ... Use an extra Module.symvers file When an external module is built, a Module.symvers file is generated containing all exported symbols which are not defined in the kernel. To get access to symbols from module 'bar', one can copy the Module.symvers file from the compilation of the 'bar' module to the directory where the 'foo' module is built. During the module build, kbuild will read the Module.symvers file in the directory of the external module and when the build is finished, a new Module.symvers file is created containing the sum of all symbols defined and not part of the kernel. -- Vladimir Sokolovsky [EMAIL PROTECTED] Mellanox Technologies Ltd. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] Address List Change Now Scheduled for Wednesday, 2/28/2007
The list will now be migrated on Wednesday, 2/28/2007. List address: [EMAIL PROTECTED] Updated change-date: Wednesday, 2/28/2007 Michael -Original Message- From: Jeff Squyres [mailto:[EMAIL PROTECTED] Sent: Wednesday, February 21, 2007 8:09 AM To: Michael S. Tsirkin Cc: OpenFabrics General; Lee, Michael Paichi Subject: Re: Address List Change for Friday, 2/23/2007 On Feb 21, 2007, at 10:34 AM, Michael S. Tsirkin wrote: Can you look at the other lists that have migrated for examples? (e.g., ewg) If I look at other lists, there's no guarantee the rule will catch the actual message. Can't you just paste in the new address of the list in your existing rules? I must be missing something. It may be complex to send an actual example message *before* the list moves. In this case, maybe the migration can be done in the middle of the week? I'll let Michael Lee answer; we're currently driving off his goodwill and his schedule. I guess I didn't see why this was complex -- if a few mails get misplaced over the weekend because cutting-n-pasting the new e-mail address into existing rules somehow didn't work, is there a huge problem? -- Jeff Squyres Server Virtualization Business Unit Cisco Systems ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [ewg] anyone have OFED 1.2 alpha1 compiling on ppc64
That missing header (common.h) is in libibcommon. Somehow, libibcommon is not installed. libibumad depends on libibcommon. Is this a build/install script issue with OFED 1.2 ? Vlad ? -- Hal I tried install.sh again, this time telling it to build libibcommon instead of relying on dependencies, and get this: + install -m 0755 /var/tmp/OFED/usr/local/ofed/bin32/mread /var/tmp/OFED/usr/lo\ cal/ofed/bin install: cannot stat `/var/tmp/OFED/usr/local/ofed/bin32/mread': No such file o\ r directory I believe mread has been renamed to mstread. # ls /var/tmp/OFED/usr/local/ofed/bin32 mstflint mstmread mstmwrite mstregdump mstvpd Scott ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey
My understanding is that when an IPoIB broadcast domain contains both partial and full members (*) attempts to communicate between two partial members would silently fail, does this silence is something you think we should work to change? I'm looking at this from a different view than just ipoib multicast groups. For example, can two users of the ib_cm successfully establish a connection, but not actually be able to transfer data between each other? This seems possible, though unlikely. This is the type of silent failure I'm referring to. Without this patch, two clients that try to connect using the librdmacm will fail. That failure is reported to the user. With this patch, the connection would be created, but I don't think that it guarantees that communication can actually occur. I don't want to mask a configuration issue. My thinking is that if in the end of this thread we are willing to move forward without changing ib_find_cached_pkey() then this patch should be merged. I'm still unsure about where the cause of this problem lies. It may be that the kernel rdma_cm or rdma_ucm needs to change if we decide the ib_find_cached_pkey is correct. - Sean ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey
An IB multicast group _cannot_ have partial members so this never should get far enough to where two limited members would be unable to communicate. Can someone help my understanding here? Is ipoib joining a multicast group using the full membership PKey, even if the node that it joins from only has the limited membership PKey configured? And the code in ib_find_cached_pkey helps enable this? - Sean ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH v2] libibverbs: can't compile more than once due to man3 symbolic links
Missed 2 lines in the patch. Below is the correct patch: --- The following patch removes manpage symbolic links so that they may be relinked in the install. Suggested by Michael Tsirkin. Signed-off-by: Jack Morgenstein [EMAIL PROTECTED] diff --git a/Makefile.am b/Makefile.am index 5d2383e..705b184 100644 --- a/Makefile.am +++ b/Makefile.am @@ -70,6 +70,20 @@ dist-hook: libibverbs.spec install-data-hook: cd $(DESTDIR)$(mandir)/man3 \ + $(RM) ibv_ack_async_event.3 \ + $(RM) ibv_ack_cq_events.3 \ + $(RM) ibv_close_device.3 \ + $(RM) ibv_dealloc_pd.3 \ + $(RM) ibv_dereg_mr.3 \ + $(RM) ibv_destroy_ah.3 \ + $(RM) ibv_destroy_comp_channel.3 \ + $(RM) ibv_destroy_cq.3 \ + $(RM) ibv_destroy_qp.3 \ + $(RM) ibv_destroy_srq.3 \ + $(RM) ibv_detach_mcast.3 \ + $(RM) ibv_free_device_list.3 \ + $(RM) ibv_init_ah_from_wc.3 \ + $(RM) mult_to_ibv_rate.3 \ $(LN_S) ibv_get_async_event.3 ibv_ack_async_event.3 \ $(LN_S) ibv_get_cq_event.3 ibv_ack_cq_events.3 \ $(LN_S) ibv_open_device.3 ibv_close_device.3 \ ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] mthca adjust_key()
Could anyone tell me why this routine in mthca is necessary? There
aren't any comments to explain it; I'm wondering if this is a workaround
for Sinai of some kind?
static inline u32 adjust_key(struct mthca_dev *dev, u32 key)
{
if (dev-mthca_flags MTHCA_FLAG_SINAI_OPT)
return ((key 20) 0x80) | (key 0x7f);
else
return key;
}
Thanks in advance,
Jesse
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH v2] libibverbs: can't compile more than once due to man3 symbolic links
Thanks, I applied this and pushed it out. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [RFC/BUG] DMA vs. CQ race
A first-cut at a patch was sent out, some very reasonable objections were raised, and the thread fizzled out. Sorry, I meant to respond again, but I never got around to it. The biggest concern with the earlier patch seemed to be backward compatibility. There was a stab at addressing that in http://tinyurl.com/2x3s52, but no commentary. (Too ugly for words?) I think you went off into the weeds there, but I'll respond to that earlier email in detail. Any suggestions as to how to proceed? Should I just code something up in order to have a concrete target to discuss? Or are there any new thoughts based on the previous emails? I actually have a vague plan for a somewhat cleaner way to get this fix. For a variety of reasons, I am planning on changing the way the kernel handles memory registration so that low-level drivers have more control over what happens. This would allow us to folow Gleb's suggestion to use register MR to create and map the kernel's buffer and avoid some of the error path ugliness. So I would prefer to map the coherent memory that way. However this will take a while to come to fruition, since it is kind of a background task for me. How severe is this issue? In other words, when you produced the problem, was it a synthetic test, or a workload that someone might actually want to run? - R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] mthca adjust_key()
Could anyone tell me why this routine in mthca is necessary? There
aren't any comments to explain it; I'm wondering if this is a workaround
for Sinai of some kind?
static inline u32 adjust_key(struct mthca_dev *dev, u32 key)
{
if (dev-mthca_flags MTHCA_FLAG_SINAI_OPT)
return ((key 20) 0x80) | (key 0x7f);
else
return key;
}
It's a performance optimization for Sinai.
- R.
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [RFC/BUG] DMA vs. CQ race
On Thu, Feb 22, 2007 at 10:34:16AM -0800, Roland Dreier wrote: I actually have a vague plan for a somewhat cleaner way to get this fix. For a variety of reasons, I am planning on changing the way the kernel handles memory registration so that low-level drivers have more control over what happens. This would allow us to folow Gleb's suggestion to use register MR to create and map the kernel's buffer and avoid some of the error path ugliness. So I would prefer to map the coherent memory that way. OK, I look forward to seeing what you have in mind. However this will take a while to come to fruition, since it is kind of a background task for me. How severe is this issue? In other words, when you produced the problem, was it a synthetic test, or a workload that someone might actually want to run? We found this accidentally, running a normal MPI job, on a normally sized machine (i.e., tens, not hundreds of processors.) It appears to be more easily produced that we'd expected, and we consider it to be a severe problem. -- Arthur ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [RFC/BUG] libibverbs: DMA vs. CQ race
Assuming that something along the lines of the previous patch is used, we need to address userspace/kernel compatibility. The existing abi versioning doesn't seem to be exactly what we want to use, though, because we want to change a verb's semantics to work around a bug. (Changing the abi_version may be an inevitable result, though.) How about adding semantic flags to the mthca_* commands (mthca_create_cq, etc.)? Userspace could read the contents of a new sysfs file which, if found, would indicate the flags that the kernel understands. Then it could pass the flags, if it chooses, to get the kernel to use the desired semantics. This is not really the design philosophy that we've used in defining the user-kernel interfaces for IB verbs. Rather than having complexity in the kernel to handle both old and new ways of doing things, the way we've used to handle cases like this is the following: - specify new fixed ABI (in this case, mthca abi_version 2) - update library to handle old and new ABI (in this case, update libmthca to use mthca kernel abi 1 or 2 depending on what it detects at runtime) - update kernel to implement new ABI, and remove old ABI from kernel (in this case, update kernel mthca driver to abi_version 2) The net effect of this is that updated userspace works fine with any kernel, but updating the kernel will require updating userspace libraries too. However the important point is that once userspace is updated, it's still possible to boot into old kernels and have things work without downgrading userspace. If we really wanted to export some flags from mthca back to libmthca, I guess it would be possible to bump the abi version and add a flags field to the response to the alloc_ucontext command, but in this case I don't see a reason to worry about it. - R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [Bug 384] New: OFED 1.2 alpha1 ib-bonding won't compile on RHEL4 U3 ppc64
https://bugs.openfabrics.org/show_bug.cgi?id=384 Summary: OFED 1.2 alpha1 ib-bonding won't compile on RHEL4 U3 ppc64 Product: OpenFabrics Linux Version: 1.2alpha1 Platform: PPC64 OS/Version: RHEL 4 Status: NEW Severity: normal Priority: P3 Component: IPoIB AssignedTo: [EMAIL PROTECTED] ReportedBy: [EMAIL PROTECTED] + cd linux/drivers/net/bonding/ ++ pwd + make -C /lib/modules/2.6.9-34.EL/build M=/var/tmp/OFEDRPM/BUILD/ib-bonding-0.\ 9.0/linux/drivers/net/bonding make: Entering directory `/usr/src/kernels/2.6.9-34.EL-ppc64' LD /var/tmp/OFEDRPM/BUILD/ib-bonding-0.9.0/linux/drivers/net/bonding/bui\ lt-in.o CC [M] /var/tmp/OFEDRPM/BUILD/ib-bonding-0.9.0/linux/drivers/net/bonding/bon\ d_main.o In file included from /var/tmp/OFEDRPM/BUILD/ib-bonding-0.9.0/linux/drivers/net\ /bonding/bond_main.c:78: /var/tmp/OFEDRPM/BUILD/ib-bonding-0.9.0/linux/drivers/net/bonding/bonding.h: In\ function `bond_set_slave_inactive_flags': /var/tmp/OFEDRPM/BUILD/ib-bonding-0.9.0/linux/drivers/net/bonding/bonding.h:260\ : error: `IFF_SLAVE_INACTIVE' undeclared (first use in this function) /var/tmp/OFEDRPM/BUILD/ib-bonding-0.9.0/linux/drivers/net/bonding/bonding.h:260\ : error: (Each undeclared identifier is reported only once /var/tmp/OFEDRPM/BUILD/ib-bonding-0.9.0/linux/drivers/net/bonding/bonding.h:260\ : error: for each function it appears in.) /var/tmp/OFEDRPM/BUILD/ib-bonding-0.9.0/linux/drivers/net/bonding/bonding.h:262\ : error: `IFF_SLAVE_NEEDARP' undeclared (first use in this function) -- Configure bugmail: https://bugs.openfabrics.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [RFC/BUG] DMA vs. CQ race
We found this accidentally, running a normal MPI job, on a normally sized machine (i.e., tens, not hundreds of processors.) It appears to be more easily produced that we'd expected, and we consider it to be a severe problem. Hmm, OK. Then I will do my best to make sure we get a fix for this into 2.6.22. - R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [ewg] anyone have OFED 1.2 alpha1 compiling on ppc64
Hi Scott, Try OFED-1.2-20070221-1741. This issue was fixed in this package. Regards, Vladimir -Original Message- From: Scott Weitzenkamp (sweitzen) [mailto:[EMAIL PROTECTED] Sent: Thursday, February 22, 2007 7:36 PM To: Hal Rosenstock Cc: [EMAIL PROTECTED]; OPENIB; Vladimir Sokolovsky Subject: RE: [ewg] anyone have OFED 1.2 alpha1 compiling on ppc64 That missing header (common.h) is in libibcommon. Somehow, libibcommon is not installed. libibumad depends on libibcommon. Is this a build/install script issue with OFED 1.2 ? Vlad ? -- Hal I tried install.sh again, this time telling it to build libibcommon instead of relying on dependencies, and get this: + install -m 0755 /var/tmp/OFED/usr/local/ofed/bin32/mread /var/tmp/OFED/usr/lo\ cal/ofed/bin install: cannot stat `/var/tmp/OFED/usr/local/ofed/bin32/mread': No such file o\ r directory I believe mread has been renamed to mstread. # ls /var/tmp/OFED/usr/local/ofed/bin32 mstflint mstmread mstmwrite mstregdump mstvpd Scott ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey
On 2/22/07, Sean Hefty [EMAIL PROTECTED] wrote: An IB multicast group _cannot_ have partial members so this never should get far enough to where two limited members would be unable to communicate. Can someone help my understanding here? Is ipoib joining a multicast group using the full membership PKey, even if the node that it joins from only has the limited membership PKey configured? And the code in ib_find_cached_pkey helps enable this? Yep. The ipoib create_child function Or-s 0x8000 to the device pkey which was provided by the user. Now, IPoIB uses the device pkey when forming MGIDs and when doing modify qp to init. Indeed the way ib_find_cached_pkey() is implemented, make the latter use trivial. Or. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey
On 2/22/07, Sean Hefty [EMAIL PROTECTED] wrote: My understanding is that when an IPoIB broadcast domain contains both partial and full members (*) attempts to communicate between two partial members would silently fail, does this silence is something you think we should work to change? I'm looking at this from a different view than just ipoib multicast groups. For example, can two users of the ib_cm successfully establish a connection, but not actually be able to transfer data between each other? This seems possible, though unlikely. This is the type of silent failure I'm referring to. I don't think this is possible since the active CM uses the pkey index of the pkey provided in REQ.path to send the REQ mad, same for the passive CM - it uses the index in its table of REQ.path.pkey. So if the CMs are able to talk over QP1 using this pkey index the CM consumers can talk over their RC (REQ) / UD (SIDR REQ) QPs. And both the CM and its consumers would use the same index - the one returned from the ib_get_cached_pkey Or. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH for-2.6.21] IPoIB/cm: improve small message bandwidth
Quoting Roland Dreier [EMAIL PROTECTED]: Subject: Re: [PATCH for-2.6.21] IPoIB/cm: improve small message bandwidth Thanks, queued for 2.6.21. With this patch I see small-packet latency down almost all the way back to what datagram mode gives -- on a pair of fast woodcrest systems I see latencies for netpipe tcp 1 byte messages like datagram 13.xx original CM 17.xx patched CM 14.xx so there is still a measurable difference but it is much less now. Hmm. An old system I tried here has a much higher latency, but does not seem to exhibit latency difference between datagram and CM. 1. Is there something special you do when you run the benchmark (msi, taskset, ...)? 2. On a wild guess that the issue here is higher interrupt rate with CM, is there a chance you could test the following patch posted by me earlier? http://www.mail-archive.com/openib-general@openib.org/msg29290.html Thanks, -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH for-2.6.21] IPoIB/cm: improve small message bandwidth
1. Is there something special you do when you run the benchmark (msi, taskset, ...)? Yes, I am using MSI-X, and I pin the interrupt handler to one CPU (CPU#0 in my particular case). Then I use taskset to pin the NPtcp process to a CPU in a different package (CPU#2 in my system). BTW with these same systems, I am getting up to ~1150 MB/sec of throughput with DDR mem-free Arbel, as measured with NPtcp. 2. On a wild guess that the issue here is higher interrupt rate with CM, is there a chance you could test the following patch posted by me earlier? http://www.mail-archive.com/openib-general@openib.org/msg29290.html OK, I'll try that when I get a chance. - R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] anyone have OFED 1.2 alpha1 compiling on ppc64
How do I upload sources? -Original Message- From: Michael S. Tsirkin [mailto:[EMAIL PROTECTED] Sent: Thursday, February 22, 2007 1:00 AM To: Scott Weitzenkamp (sweitzen) Cc: [EMAIL PROTECTED]; OPENIB Subject: Re: anyone have OFED 1.2 alpha1 compiling on ppc64 Quoting Scott Weitzenkamp (sweitzen) [EMAIL PROTECTED]: Subject: anyone have OFED 1.2 alpha1 compiling on ppc64 I tried both RHEL4 and SLES10 usinstall.sh, and get this. I filed bug 379, anyone else tried ppc64? Scott, could pls you upload the kernel sources and .config files to staging? If you do, we'll be able to add these to mightly cross-build environment. -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH for-2.6.21] IPoIB/cm: improve small message bandwidth
OK, I applied the following patch (I had to change one line of your
patch to get it to apply because the small-message changed the context
so one chunk didn't apply).
Anyway I don't see any difference in small message latency or large
message throughput. (Actually latency seems slightly worse but I
think the change is within my normal variability so I'm don't think
the difference is significant)
diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h
b/drivers/infiniband/ulp/ipoib/ipoib.h
index 2594db2..20d7ad4 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib.h
+++ b/drivers/infiniband/ulp/ipoib/ipoib.h
@@ -98,9 +98,9 @@ enum {
#defineIPOIB_OP_RECV (1ul 31)
#ifdef CONFIG_INFINIBAND_IPOIB_CM
-#defineIPOIB_CM_OP_SRQ (1ul 30)
+#defineIPOIB_OP_CM (1ul 30)
#else
-#defineIPOIB_CM_OP_SRQ (0)
+#defineIPOIB_OP_CM (0)
#endif
/* structs */
@@ -143,7 +143,6 @@ struct ipoib_cm_rx {
struct ipoib_cm_tx {
struct ib_cm_id *id;
- struct ib_cq*cq;
struct ib_qp*qp;
struct list_head list;
struct net_device *dev;
@@ -232,6 +231,7 @@ struct ipoib_dev_priv {
unsigned tx_tail;
struct ib_sgetx_sge;
struct ib_send_wrtx_wr;
+ unsigned tx_outstanding;
struct ib_wc ibwc[IPOIB_NUM_WC];
@@ -438,6 +438,7 @@ void ipoib_cm_destroy_tx(struct ipoib_cm_tx *tx);
void ipoib_cm_skb_too_long(struct net_device* dev, struct sk_buff *skb,
unsigned int mtu);
void ipoib_cm_handle_rx_wc(struct net_device *dev, struct ib_wc *wc);
+void ipoib_cm_handle_tx_wc(struct net_device *dev, struct ib_wc *wc);
#else
struct ipoib_cm_tx;
@@ -526,6 +527,9 @@ static inline void ipoib_cm_handle_rx_wc(struct net_device
*dev, struct ib_wc *w
{
}
+static inline void ipoib_cm_handle_tx_wc(struct net_device *dev, struct ib_wc
*wc)
+{
+}
#endif
#ifdef CONFIG_INFINIBAND_IPOIB_DEBUG
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c
b/drivers/infiniband/ulp/ipoib/ipoib_cm.c
index 3484e8b..9515ef6 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c
@@ -82,7 +82,7 @@ static int ipoib_cm_post_receive(struct net_device *dev, int
id)
struct ib_recv_wr *bad_wr;
int i, ret;
- priv-cm.rx_wr.wr_id = id | IPOIB_CM_OP_SRQ;
+ priv-cm.rx_wr.wr_id = id | IPOIB_OP_CM | IPOIB_OP_RECV;
for (i = 0; i IPOIB_CM_RX_SG; ++i)
priv-cm.rx_sge[i].addr = priv-cm.srq_ring[id].mapping[i];
@@ -344,7 +344,7 @@ static void skb_put_frags(struct sk_buff *skb, unsigned int
hdr_space,
void ipoib_cm_handle_rx_wc(struct net_device *dev, struct ib_wc *wc)
{
struct ipoib_dev_priv *priv = netdev_priv(dev);
- unsigned int wr_id = wc-wr_id ~IPOIB_CM_OP_SRQ;
+ unsigned int wr_id = wc-wr_id ~(IPOIB_OP_CM | IPOIB_OP_RECV);
struct sk_buff *skb, *newskb;
struct ipoib_cm_rx *p;
unsigned long flags;
@@ -436,7 +436,7 @@ static inline int post_send(struct ipoib_dev_priv *priv,
priv-tx_sge.addr = addr;
priv-tx_sge.length = len;
- priv-tx_wr.wr_id = wr_id;
+ priv-tx_wr.wr_id = wr_id | IPOIB_OP_CM;
return ib_post_send(tx-qp, priv-tx_wr, bad_wr);
}
@@ -487,20 +487,19 @@ void ipoib_cm_send(struct net_device *dev, struct sk_buff
*skb, struct ipoib_cm_
dev-trans_start = jiffies;
++tx-tx_head;
- if (tx-tx_head - tx-tx_tail == ipoib_sendq_size) {
+ if (++priv-tx_outstanding == ipoib_sendq_size) {
ipoib_dbg(priv, TX ring 0x%x full, stopping kernel net
queue\n,
tx-qp-qp_num);
netif_stop_queue(dev);
- set_bit(IPOIB_FLAG_NETIF_STOPPED, tx-flags);
}
}
}
-static void ipoib_cm_handle_tx_wc(struct net_device *dev, struct ipoib_cm_tx
*tx,
- struct ib_wc *wc)
+void ipoib_cm_handle_tx_wc(struct net_device *dev, struct ib_wc *wc)
{
struct ipoib_dev_priv *priv = netdev_priv(dev);
- unsigned int wr_id = wc-wr_id;
+ struct ipoib_cm_tx *tx = wc-qp-qp_context;
+ unsigned int wr_id = wc-wr_id ~IPOIB_OP_CM;
struct ipoib_tx_buf *tx_req;
unsigned long flags;
@@ -525,11 +524,10 @@ static void ipoib_cm_handle_tx_wc(struct net_device *dev,
struct ipoib_cm_tx *tx
spin_lock_irqsave(priv-tx_lock, flags);
++tx-tx_tail;
- if (unlikely(test_bit(IPOIB_FLAG_NETIF_STOPPED, tx-flags))
- tx-tx_head - tx-tx_tail = ipoib_sendq_size 1) {
- clear_bit(IPOIB_FLAG_NETIF_STOPPED, tx-flags);
+ if (unlikely(--priv-tx_outstanding == ipoib_sendq_size 1)
+ netif_queue_stopped(dev)
+ test_bit(IPOIB_FLAG_ADMIN_UP, priv-flags))
Re: [openib-general] anyone have OFED 1.2 alpha1 compiling on ppc64
Don't you have an account at ssh.openfabrics.org? If yes, just put kernel sources and the .config under your home directory Quoting r. Scott Weitzenkamp (sweitzen) [EMAIL PROTECTED]: Subject: Re: anyone have OFED 1.2 alpha1 compiling on ppc64 How do I upload sources? -Original Message- From: Michael S. Tsirkin [mailto:[EMAIL PROTECTED] Sent: Thursday, February 22, 2007 1:00 AM To: Scott Weitzenkamp (sweitzen) Cc: [EMAIL PROTECTED]; OPENIB Subject: Re: anyone have OFED 1.2 alpha1 compiling on ppc64 Quoting Scott Weitzenkamp (sweitzen) [EMAIL PROTECTED]: Subject: anyone have OFED 1.2 alpha1 compiling on ppc64 I tried both RHEL4 and SLES10 usinstall.sh, and get this. I filed bug 379, anyone else tried ppc64? Scott, could pls you upload the kernel sources and .config files to staging? If you do, we'll be able to add these to mightly cross-build environment. -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH for-2.6.21] IPoIB/cm: improve small message bandwidth
Quoting Roland Dreier [EMAIL PROTECTED]: Subject: Re: [PATCH for-2.6.21] IPoIB/cm: improve small message bandwidth OK, I applied the following patch (I had to change one line of your patch to get it to apply because the small-message changed the context so one chunk didn't apply). Anyway I don't see any difference in small message latency or large message throughput. (Actually latency seems slightly worse but I think the change is within my normal variability so I'm don't think the difference is significant) OK, thanks for testing this. I need to spend more time on reproducing this issue, and profiling. I'll add this to my todo list. -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] anyone have OFED 1.2 alpha1 compiling on ppc64
Don't you have an account at ssh.openfabrics.org? Can an admin please give me an account? Scott ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] anyone have OFED 1.2 alpha1 compiling on ppc64
Quoting Scott Weitzenkamp (sweitzen) [EMAIL PROTECTED]: Subject: RE: anyone have OFED 1.2 alpha1 compiling on ppc64 Don't you have an account at ssh.openfabrics.org? Can an admin please give me an account? I'm not an admin but I think you want to post your ssh public key. -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH] librdmacm: fix bug causing failure to work with partial membership pkey
Can someone help my understanding here? Is ipoib joining a multicast group using the full membership PKey, even if the node that it joins from only has the limited membership PKey configured? And the code in ib_find_cached_pkey helps enable this? Yep. The ipoib create_child function Or-s 0x8000 to the device pkey which was provided by the user. Now, IPoIB uses the device pkey when forming MGIDs and when doing modify qp to init. Indeed the way ib_find_cached_pkey() is implemented, make the latter use trivial. Doesn't this allow ipoib to join a multicast group for which it may not be able to communicate with all members? For the broadcast group, this seems like an error to me. Can ipoib work in such a configuration? If all nodes were assigned a partial membership PKey, none of them could communicate, but no errors would be generated anywhere. Joining a multicast group requires specifying the full membership PKey. I don't see anything in the spec that explicitly prohibits joining the group from a node with only a partial membership PKey, but at first glance, this seems like a subnet configuration issue. Is there some use of this I'm overlooking? - Sean ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] IPOIB NAPI
An API idea: how about instead testing missed_events, we add a flag: IB_CQ_TEST (or a longer name IB_CQ_REPORT_MISSED_EVENTS?) and change ib_req_notify_cq to return int which will keep the missed_events value, only if this flag is set? This has 2 advatages - Less churn updating all users to new API - they just ignore return value - and still almost no overhead for them as they don't set IB_CQ_TEST - For all users we have to push less values on stack - note compiler can't get rid of them as we are calling function through a pointer - For users that do missed_events = ib_req_notify_cq(priv-cq, IB_CQ_NEXT_COMP | IB_CQ_TEST) we get the result in register. Yes, I like this. So ib_req_notify_cq() gets a return value that is negative if an error occurred, 0 if everything is fine, or positive if a missed event might have happened. I think I prefer the longer name IB_CQ_REPORT_MISSED_EVENTS -- at least there's a chance at guessing what it means even if you don't read the documentation. By the way, how about extending the userspace API in a similiar fashion? missed_events = ibv_req_notify_cq(priv-cq, IBV_CQ_NEXT_COMP | IBV_CQ_REPORT_MISSED_EVENTS) -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] IPOIB NAPI
By the way, how about extending the userspace API in a similiar fashion? missed_events = ibv_req_notify_cq(priv-cq, IBV_CQ_NEXT_COMP | IBV_CQ_REPORT_MISSED_EVENTS) It would require a kernel-user ABI bump. Is it worth it? - R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] ipoib the partial pkey, was: librdmacm: fix bug causing failure to work with partial membership pkey
Doesn't this allow ipoib to join a multicast group for which it may not be able to communicate with all members? For the broadcast group, this seems like an error to me. Can ipoib work in such a configuration? If all nodes were assigned a partial membership PKey, none of them could communicate, but no errors would be generated anywhere. I looked into this more... RFC 4391 states (middle of page 5): For a node to join a partition, one of its ports must be assigned the relevant P_Key by the SM [RFC4392]. Jumping to RFC 4392 (top of page 4): at the time of creating an IB multicast group, multiple values such as the P_Key, Q_Key, Service Level, Hop Limit, Flow ID, TClass, MTU, etc. have to be specified. These values should be such that all potential members of the IB multicast group are able to communicate with one another when using them. and page 14: Note that this IB_join to the broadcast group is a FullMember join. If any of the ports or the switches linking the port to the rest of the IPoIB subnet cannot support the parameters (e.g., path MTU or P_Key) associated with the broadcast group, then the IB_join request will fail and the requesting port will not become part of the IPoIB subnet My initial interpretation of these statements lead me to believe that pkey check in ib_find_cached_pkey should not mask out the upper bit, which would prevent ipoib from joining a multicast group until it has been configured with the full membership pkey for the broadcast group. Does this seem reasonable? - Sean ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] IPOIB NAPI
Quoting Roland Dreier [EMAIL PROTECTED]: Subject: Re: IPOIB NAPI By the way, how about extending the userspace API in a similiar fashion? missed_events = ibv_req_notify_cq(priv-cq, IBV_CQ_NEXT_COMP | IBV_CQ_REPORT_MISSED_EVENTS) It would require a kernel-user ABI bump. Is it worth it? I hear some people asking for it: I imagine reasons are same as NAPI - race-free, clean API to switch from polling to event mode - rather than a minor optimization. -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH] libmthca: optimize calls to htonl with constant parameter
GCC seems to be unable to propogate constants across calls to htonl.
So it turns out to be worth the while to replace htonl with
a hand-written macro in case of constant parameter.
Signed-off-by: Michael S. Tsirkin [EMAIL PROTECTED]
Signed-off-by: Ishai Rabinovitz [EMAIL PROTECTED]
---
Roland, I'm looking at micro-optimizing libmthca/mthca some more.
The following optimization is minor, but it seems quite safe.
What do you think? Tested with gcc 4.0.3.
diff --git a/src/cq.c b/src/cq.c
index 0aeb7a9..9428f74 100644
--- a/src/cq.c
+++ b/src/cq.c
@@ -275,7 +275,7 @@ static int handle_error_cqe(struct mthca_cq *cq,
* doorbell count field. In that case we always free the CQE.
*/
if (mthca_is_memfree(cq-ibv_cq.context) ||
- !(new_wqe htonl(0x3f)) || (!cqe-db_cnt dbd))
+ !(new_wqe CONSTANT_HTONL(0x3f)) || (!cqe-db_cnt dbd))
return 0;
cqe-db_cnt = htons(ntohs(cqe-db_cnt) - dbd);
diff --git a/src/mthca.h b/src/mthca.h
index 1f31bc3..798029f 100644
--- a/src/mthca.h
+++ b/src/mthca.h
@@ -112,6 +112,20 @@ enum {
MTHCA_OPCODE_INVALID= 0xff
};
+/* GCC does not seem to be able to do constant propogation
+ * across htonl/ntohl calls */
+#if __BYTE_ORDER == __LITTLE_ENDIAN
+#define CONSTANT_HTONL(x)\
+ unsigned)x) 24) | \
+unsigned)x) 8) 0xff00) | \
+unsigned)x) 8) 0xff) | \
+(((unsigned)x) 24))
+#elif __BYTE_ORDER == __BIG_ENDIAN
+#define CONSTANT_HTONL(x) (x)
+#else
+#define CONSTANT_HTONL(x) htonl(x)
+#endif
+
struct mthca_ah_page;
struct mthca_device {
diff --git a/src/qp.c b/src/qp.c
index f2483e9..85d3385 100644
--- a/src/qp.c
+++ b/src/qp.c
@@ -138,10 +138,10 @@ int mthca_tavor_post_send(struct ibv_qp *ibqp, struct
ibv_send_wr *wr,
((struct mthca_next_seg *) wqe)-ee_nds = 0;
((struct mthca_next_seg *) wqe)-flags =
((wr-send_flags IBV_SEND_SIGNALED) ?
-htonl(MTHCA_NEXT_CQ_UPDATE) : 0) |
+CONSTANT_HTONL(MTHCA_NEXT_CQ_UPDATE) : 0) |
((wr-send_flags IBV_SEND_SOLICITED) ?
-htonl(MTHCA_NEXT_SOLICIT) : 0) |
- htonl(1);
+CONSTANT_HTONL(MTHCA_NEXT_SOLICIT) : 0) |
+ CONSTANT_HTONL(1);
if (wr-opcode == IBV_WR_SEND_WITH_IMM ||
wr-opcode == IBV_WR_RDMA_WRITE_WITH_IMM)
((struct mthca_next_seg *) wqe)-imm = wr-imm_data;
@@ -359,9 +359,9 @@ int mthca_tavor_post_recv(struct ibv_qp *ibqp, struct
ibv_recv_wr *wr,
((struct mthca_next_seg *) wqe)-nda_op = 0;
((struct mthca_next_seg *) wqe)-ee_nds =
- htonl(MTHCA_NEXT_DBD);
+ CONSTANT_HTONL(MTHCA_NEXT_DBD);
((struct mthca_next_seg *) wqe)-flags =
- htonl(MTHCA_NEXT_CQ_UPDATE);
+ CONSTANT_HTONL(MTHCA_NEXT_CQ_UPDATE);
wqe += sizeof (struct mthca_next_seg);
size = sizeof (struct mthca_next_seg) / 16;
@@ -505,10 +505,10 @@ int mthca_arbel_post_send(struct ibv_qp *ibqp, struct
ibv_send_wr *wr,
((struct mthca_next_seg *) wqe)-flags =
((wr-send_flags IBV_SEND_SIGNALED) ?
-htonl(MTHCA_NEXT_CQ_UPDATE) : 0) |
+CONSTANT_HTONL(MTHCA_NEXT_CQ_UPDATE) : 0) |
((wr-send_flags IBV_SEND_SOLICITED) ?
-htonl(MTHCA_NEXT_SOLICIT) : 0) |
- htonl(1);
+CONSTANT_HTONL(MTHCA_NEXT_SOLICIT) : 0) |
+ CONSTANT_HTONL(1);
if (wr-opcode == IBV_WR_SEND_WITH_IMM ||
wr-opcode == IBV_WR_RDMA_WRITE_WITH_IMM)
((struct mthca_next_seg *) wqe)-imm = wr-imm_data;
@@ -750,7 +750,7 @@ int mthca_arbel_post_recv(struct ibv_qp *ibqp, struct
ibv_recv_wr *wr,
if (i qp-rq.max_gs) {
((struct mthca_data_seg *) wqe)-byte_count = 0;
- ((struct mthca_data_seg *) wqe)-lkey =
htonl(MTHCA_INVAL_LKEY);
+ ((struct mthca_data_seg *) wqe)-lkey =
CONSTANT_HTONL(MTHCA_INVAL_LKEY);
((struct mthca_data_seg *) wqe)-addr = 0;
}
@@ -872,7 +872,7 @@ int mthca_alloc_qp_buf(struct ibv_pd *pd, struct ibv_qp_cap
*cap,
for (scatter = (void *) (next + 1);
(void *) scatter (void *) next + (1
qp-rq.wqe_shift);
++scatter)
- scatter-lkey = htonl(MTHCA_INVAL_LKEY);
+ scatter-lkey =
CONSTANT_HTONL(MTHCA_INVAL_LKEY);
}
Re: [openib-general] [PATCH 0/7] cxgb3 - Chelsio T3 1G/10G driver updates
Steve Wise wrote: Divy, Do these need to be pulled into OFED 1.2 as well? Hi Steve, Yes, I believe so. Cheers, Divy Steve. On Thu, 2007-02-22 at 03:58 -0800, Divy Le Ray wrote: Jeff, I'm sending a series of incremental patches updating the cxgb3 driver. These patches are built against Linus'git tree. Cheers, Divy - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH] 2.6.21-rc1: please pull rdma-dev.git for-roland
Roland,
Please consider the following minor fixes for 2.6.21:
rdma_cm: remove unused node_guid from cma_device structure.
ib_cm: remove ca_guid from cm_device structure.
rdma_cm: request reversible paths only.
ib_core: Set hop limit in ib_init_ah_from_wc correctly.
The patches are in git.openfabrics.org/~shefty/rdma-dev.git,
for-roland branch, which is based on 2.6.21-rc1.
Signed-off-by: Sean Hefty [EMAIL PROTECTED]
---
commit 28e218621d36cf9da42f07af08775769eb289fc0
Author: Sean Hefty [EMAIL PROTECTED]
Date: Thu Feb 22 11:37:44 2007 -0800
rdma_cm: remove unused node_guid from cma_device structure.
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index bb27ce9..d441815 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -77,7 +77,6 @@ static int next_port;
struct cma_device {
struct list_headlist;
struct ib_device*device;
- __be64 node_guid;
struct completion comp;
atomic_trefcount;
struct list_headid_list;
@@ -2674,7 +2673,6 @@ static void cma_add_one(struct ib_device *device)
return;
cma_dev-device = device;
- cma_dev-node_guid = device-node_guid;
init_completion(cma_dev-comp);
atomic_set(cma_dev-refcount, 1);
commit 6de97f2a3373357d720b1653dfc0aac6d40b7506
Author: Sean Hefty [EMAIL PROTECTED]
Date: Thu Feb 22 11:37:38 2007 -0800
ib_cm: remove ca_guid from cm_device structure.
The cm_device references an ib_device, which contains the node_guid.
diff --git a/drivers/infiniband/core/cm.c b/drivers/infiniband/core/cm.c
index d446998..842cd0b 100644
--- a/drivers/infiniband/core/cm.c
+++ b/drivers/infiniband/core/cm.c
@@ -88,7 +88,6 @@ struct cm_port {
struct cm_device {
struct list_head list;
struct ib_device *device;
- __be64 ca_guid;
struct cm_port port[0];
};
@@ -739,8 +738,8 @@ retest:
ib_cancel_mad(cm_id_priv-av.port-mad_agent, cm_id_priv-msg);
spin_unlock_irqrestore(cm_id_priv-lock, flags);
ib_send_cm_rej(cm_id, IB_CM_REJ_TIMEOUT,
- cm_id_priv-av.port-cm_dev-ca_guid,
- sizeof cm_id_priv-av.port-cm_dev-ca_guid,
+ cm_id_priv-id.device-node_guid,
+ sizeof cm_id_priv-id.device-node_guid,
NULL, 0);
break;
case IB_CM_REQ_RCVD:
@@ -883,7 +882,7 @@ static void cm_format_req(struct cm_req_msg *req_msg,
req_msg-local_comm_id = cm_id_priv-id.local_id;
req_msg-service_id = param-service_id;
- req_msg-local_ca_guid = cm_id_priv-av.port-cm_dev-ca_guid;
+ req_msg-local_ca_guid = cm_id_priv-id.device-node_guid;
cm_req_set_local_qpn(req_msg, cpu_to_be32(param-qp_num));
cm_req_set_resp_res(req_msg, param-responder_resources);
cm_req_set_init_depth(req_msg, param-initiator_depth);
@@ -1442,7 +1441,7 @@ static void cm_format_rep(struct cm_rep_msg *rep_msg,
cm_rep_set_flow_ctrl(rep_msg, param-flow_control);
cm_rep_set_rnr_retry_count(rep_msg, param-rnr_retry_count);
cm_rep_set_srq(rep_msg, param-srq);
- rep_msg-local_ca_guid = cm_id_priv-av.port-cm_dev-ca_guid;
+ rep_msg-local_ca_guid = cm_id_priv-id.device-node_guid;
if (param-private_data param-private_data_len)
memcpy(rep_msg-private_data, param-private_data,
@@ -3385,7 +3384,6 @@ static void cm_add_one(struct ib_device *device)
return;
cm_dev-device = device;
- cm_dev-ca_guid = device-node_guid;
set_bit(IB_MGMT_METHOD_SEND, reg_req.method_mask);
for (i = 1; i = device-phys_port_cnt; i++) {
commit 87680047dd09ca4a4e8ec575dad215c92cf45ed3
Author: Sean Hefty [EMAIL PROTECTED]
Date: Wed Feb 21 16:40:44 2007 -0800
rdma_cm: request reversible paths only
The rdma_cm requires that path records be reversible. Set the reversible
bit when issuing an path record query.
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index f8d69b3..bb27ce9 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -1492,11 +1492,13 @@ static int cma_query_ib_route(struct rdma_id_private
*id_priv, int
timeout_ms,
ib_addr_get_dgid(addr, path_rec.dgid);
path_rec.pkey = cpu_to_be16(ib_addr_get_pkey(addr));
path_rec.numb_path = 1;
+ path_rec.reversible = 1;
id_priv-query_id = ib_sa_path_rec_get(sa_client, id_priv-id.device,
id_priv-id.port_num, path_rec,
IB_SA_PATH_REC_DGID | IB_SA_PATH_REC_SGID |
- IB_SA_PATH_REC_PKEY | IB_SA_PATH_REC_NUMB_PATH,
+ IB_SA_PATH_REC_PKEY | IB_SA_PATH_REC_NUMB_PATH |
+
Re: [openib-general] [PATCH] 2.6.21-rc1: please pull rdma-dev.git for-roland
These all look fine, I'll queue them up. Signed-off-by: Sean Hefty [EMAIL PROTECTED] I notice that the actual patches you committed don't have your sign-off in the git changelog. I assume this is a mistake so I'll add it back in... ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH] 2.6.21-rc1: please pull rdma-dev.git for-roland
I notice that the actual patches you committed don't have your sign-off in the git changelog. I assume this is a mistake so I'll add it back in... which means I can't just pull your branch. But that's OK, still doing git format-patch, edit patches, git am is pretty easy. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH] 2.6.21-rc1: please pull rdma-dev.git for-roland
The patches are in git.openfabrics.org/~shefty/rdma-dev.git, for-roland branch, which is based on 2.6.21-rc1. One other request: please include a URL that I can just copy and paste, so I don't actually have to read and parse complete sentences. Something like: the patches are in git://git.openfabrics.org/~shefty/rdma-dev.git for-roland - R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH] 2.6.21-rc1: please pull rdma-dev.git for-roland
Anyway, all 4 queued up in my for-2.6.21 branch ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH] 2.6.21-rc1: please pull rdma-dev.git for-roland
the patches are in git://git.openfabrics.org/~shefty/rdma-dev.git for-roland I will do that in the future. And yes, the sign off line was just a mistake. Thanks for fixing that. - Sean ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH] libmthca: optimize calls to htonl with constant parameter
GCC seems to be unable to propogate constants across calls to htonl.
So it turns out to be worth the while to replace htonl with
a hand-written macro in case of constant parameter.
I'm wondering why this helps you. On my system (which has Debian's
old glibc 2.3.6, certainly nothing particularly fancy), I see in
my netinet/in.h:
/* Get machine dependent optimized versions of byte swapping functions.
*/
#include bits/byteswap.h
#ifdef __OPTIMIZE__
/* We can optimize calls to the conversion functions. Either nothing
has
to be done or we are using directly the byte-swapping functions which
often can be inlined. */
# if __BYTE_ORDER == __BIG_ENDIAN
//...
# else
# if __BYTE_ORDER == __LITTLE_ENDIAN
# define ntohl(x) __bswap_32 (x)
and so on (and gcc defines __OPTIMIZE__ if you pass it any -O flag
including -Os). And in bits/byteswap.h I have
/* Swap bytes in 32 bit value. */
#define __bswap_constant_32(x) \
x) 0xff00) 24) | (((x) 0x00ff) 8) |
\
(((x) 0xff00) 8) | (((x) 0x00ff) 24))
and variations of __bswap_32() that look roughly like
# define __bswap_32(x) \
(__extension__
\
({ register unsigned int __v, __x = (x);
\
if (__builtin_constant_p (__x))
\
__v = __bswap_constant_32 (__x);
\
else
\
and so on. (The point of all this being that for constants, htonl()
should expand to roughly the same thing as your CONSTANT_HTONL() --
the only difference is that you don't have the for the 24 and
24 parts, which I guess just has the potential to bite us if someone
did something like CONSTANT_HTONL(1L) on a 64-bit system).
As a quick test I compiled the code
#include netinet/in.h
enum {
Y = 5
};
uint32_t foo(uint32_t x)
{
return x | htonl(Y);
}
with gcc -c -O and the disassembly of foo() looks like
foo:
0: 89 f8 mov%edi,%eax
2: 0d 00 00 00 05 or $0x500,%eax
7: c3 retq
and so everything works exactly the way we would want. (32-bit i386
also just does or with a constant too).
In fact for libmthca I just checked that the preprocessor output of
places like the following (which your patch converts)
((wr-send_flags IBV_SEND_SIGNALED) ?
htonl(MTHCA_NEXT_CQ_UPDATE) : 0) |
is
((wr-send_flags IBV_SEND_SIGNALED) ?
(__extension__ ({ register unsigned int __v, __x = (MTHCA_NEXT_CQ_UPDATE);
if (__builtin_constant_p (__x)) __v = __x) 0xff00) 24) | (((__x)
0x00ff) 8) | (((__x) 0xff00) 8) | (((__x) 0x00ff)
24)); else __asm__ (bswap %0 : =r (__v) : 0 (__x)); __v; })) : 0) |
And if I compare the generated assembly for libmthca with and without
your patch (on both x86-64 and i386), I don't see any significant
difference (the size is exactly the same, I just see things like the
compiler using eax and edx in the opposite order and trivial things
like that).
So what is different in your setup that causes this patch to make a
difference for you?
(BTW, one thing I did notice while looking at the i386 assembly is
that one micro-optimization that might make sense to use something
like __attribute__((regparm(3))) for internal function calls within
libibverbs and libmthca on i386, since otherwise we waste instructions
pushing stuff on the stack for no reason other than compliance with
the crufty old i386 ABI. Something like a FASTCALL macro in
infiniband/arch.h perhaps... if anyone really cares about 32-bit
i386 performance any more)
- R.
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH] libmthca: optimize calls to htonl with constant parameter
On Thu, Feb 22, 2007 at 10:09:28PM -0800, Roland Dreier wrote: (BTW, one thing I did notice while looking at the i386 assembly is that one micro-optimization that might make sense to use something like __attribute__((regparm(3))) for internal function calls within libibverbs and libmthca on i386, since otherwise we waste instructions pushing stuff on the stack for no reason other than compliance with the crufty old i386 ABI. Something like a FASTCALL macro in infiniband/arch.h perhaps... if anyone really cares about 32-bit i386 performance any more) Newer gccs have the -fwhole-program --combine options that address this and more. One of the things that happens is that all internal functions are made 'static' and all compilation units are optimized in one go. gcc will optimize calling convention and alot of other things for static functions. That should provide an across the board micro-improvement even on x86-64. Jason ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
