[openib-general] Re: tog Phar amacy
http://www.lovinsiter.com SA C VXV om I AaI m bALn A a iL I a G eIUxR nSMA $$$ $$ 12 3 $ 1 3 ,,, 1 , , 18 7,4 3 295223___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] respect CFLAGS in OSM
On Thu, 2006-01-19 at 16:50, Pete Wyckoff wrote: I do something like: CFLAGS=-g ./configure ... to build a debug tree from openib svn. Some places override this CFLAGS setting, though, applying optimization even though I explicitly do not want it. This patch fixes that. These apply to OSM below gen2/trunk/src/userspace/. Thanks. Applied. Signed-off-by: Pete Wyckoff [EMAIL PROTECTED] ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] Re: Userspace testing results (many kernels, many svn trees)
Quoting r. Nishanth Aravamudan [EMAIL PROTECTED]: Subject: Re: Userspace testing results (many kernels,many svn trees) On 09.01.2006 [08:33:01 +0200], Michael S. Tsirkin wrote: Quoting r. Nishanth Aravamudan [EMAIL PROTECTED]: Just like with the results I posted earlier, all the perftest results are seriously wrong for 32-bit clients (with both 32-bit and 64-bit servers). I am not sure who else to notify beyond the general list (is there a corresponding MAINTAINERS files like in the kernel proper for the OpenIB code?) That would be me - sorry about the delay, I'll take a look at this. Thanks a lot, Nishanth! This work is very much appreciated. No worries, hope the problem is not too hard to fix :) OK, I'm going to concentrate on rdma_lat/rdma_bw for now. # file ./rdma_bw ./rdma_bw: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.2.5, dynamically linked (uses shared libs), not stripped # ./rdma_bw swlab155 local address: LID 0xd9, QPN 0x24040a, PSN 0x98e7f4 RKey 0xe0003f VAddr 0x00556db000 remote address: LID 0xd9, QPN 0x240406, PSN 0x952ea0, RKey 0xe00033 VAddr 0x00556db000 Bandwidth peak (#0 to #999): 879.956 MB/sec Bandwidth average: 879.954 MB/sec Service Demand peak (#0 to #999): 3773 cycles/KB Service Demand Avg : 37 cycles/KB Seems like I cant reproduce the problem. Which distribution and CPU architecture is this, again? -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] respect CFLAGS in OSM
Hi Pete, I have looked again at this patch and what it is changing. My understanding is that you found the -g -O2 CLFAGS (provided through the specific target CFLAGS) unneeded. You also think they will interfere with settings you might want to provide from the command line. I have just double checked what I new to be the rule for autoconf: If the user provides CFLAGS or LDFLAGS from the command like - they are appended to the compile or link flags. The impact on gcc is that the later settings - i.e. those provided by the user take precedence over the flags provided at the beginning of the command line. So the patch below is actually not needed. Just to convince you I attach some gcc traces showing that -O0 -O2 acts like -O2 and -O2 -O0 acts like -O0. Bottom line I would like to keep the code as it is without any change such that default installation will use the -O2 mode. Eitan Zahavi Design Technology Director Mellanox Technologies LTD Tel:+972-4-9097208 Fax:+972-4-9593245 P.O. Box 586 Yokneam 20692 ISRAEL swlab25:/home/eitan/SW/work/examples/usr/libexec/gcc/i386-redhat-linux/ 4.0.0/cc1 getHostName.c getHostName.c -auxbase getHostName -O0 -O2 -version -o /tmp/ccet3OkS.s GNU C version 4.0.0 20050519 (Red Hat 4.0.0-8) (i386-redhat-linux) compiled by GNU C version 4.0.0 20050519 (Red Hat 4.0.0-8). GGC heuristics: --param ggc-min-expand=99 --param ggc-min-heapsize=129317 options passed: -auxbase -O0 -O2 options enabled: -falign-loops -fargument-alias -fbranch-count-reg -fcaller-saves -fcommon -fcprop-registers -fcrossjumping -fcse-follow-jumps -fcse-skip-blocks -fdefer-pop -fdelete-null-pointer-checks -feliminate-unused-debug-types -fexpensive-optimizations -fforce-mem -ffunction-cse -fgcse -fgcse-lm -fguess-branch-probability -fident -fif-conversion -fif-conversion2 -fivopts -fkeep-static-consts -fleading-underscore -floop-optimize -floop-optimize2 -fmath-errno -fmerge-constants -foptimize-register-move -foptimize-sibling-calls -fpcc-struct-return -fpeephole -fpeephole2 -fregmove -freorder-blocks -freorder-functions -frerun-cse-after-loop -frerun-loop-opt -fsched-interblock -fsched-spec -fsched-stalled-insns-dep -fsplit-ivs-in-unroller -fstrength-reduce -fstrict-aliasing -fthread-jumps -ftrapping-math -ftree-ccp -ftree-ch -ftree-copyrename -ftree-dce -ftree-dominator-opts -ftree-dse -ftree-fre -ftree-loop-im -ftree-loop-ivcanon -ftree-loop-optimize -ftree-lrs -ftree-pre -ftree-sra -ftree-ter -funit-at-a-time -fvar-tracking -fzero-initialized-in-bss -m80387 -mhard-float -mno-soft-float -mieee-fp -mfp-ret-in-387 -mno-red-zone -mtls-direct-seg-refs -mtune=i386 -march=i386 swlab25:/home/eitan/SW/work/examples/usr/libexec/gcc/i386-redhat-linux/ 4.0.0/cc1 getHostName.c getHostName.c -auxbase getHostName -O2 -O0 -version -o /tmp/ccet3OkS.s GNU C version 4.0.0 20050519 (Red Hat 4.0.0-8) (i386-redhat-linux) compiled by GNU C version 4.0.0 20050519 (Red Hat 4.0.0-8). GGC heuristics: --param ggc-min-expand=99 --param ggc-min-heapsize=129317 options passed: -auxbase -O2 -O0 options enabled: -falign-loops -fargument-alias -fbranch-count-reg -fcommon -feliminate-unused-debug-types -ffunction-cse -fgcse-lm -fident -fivopts -fkeep-static-consts -fleading-underscore -floop-optimize2 -fmath-errno -fpcc-struct-return -fpeephole -fsched-interblock -fsched-spec -fsched-stalled-insns-dep -fsplit-ivs-in-unroller -ftrapping-math -ftree-loop-im -ftree-loop-ivcanon -ftree-loop-optimize -funit-at-a-time -fvar-tracking -fzero-initialized-in-bss -m80387 -mhard-float -mno-soft-float -mieee-fp -mfp-ret-in-387 -mno-red-zone -mtls-direct-seg-refs -mtune=i386 -march=i386 -Original Message- From: [EMAIL PROTECTED] [mailto:openib-general- [EMAIL PROTECTED] On Behalf Of Pete Wyckoff Sent: Thursday, January 19, 2006 11:50 PM To: openib-general@openib.org Subject: [openib-general] respect CFLAGS in OSM I do something like: CFLAGS=-g ./configure ... to build a debug tree from openib svn. Some places override this CFLAGS setting, though, applying optimization even though I explicitly do not want it. This patch fixes that. These apply to OSM below gen2/trunk/src/userspace/. Signed-off-by: Pete Wyckoff [EMAIL PROTECTED] Index: management/osm/libvendor/Makefile.am === --- management/osm/libvendor/Makefile.am (revision 5098) +++ management/osm/libvendor/Makefile.am (working copy) @@ -3,8 +3,6 @@ if DEBUG DBGFLAGS = -ggdb -D_DEBUG_ -else -DBGFLAGS = -g -O2 endif INCLUDES = $(OSMV_INCLUDES) Index: management/osm/complib/Makefile.am === --- management/osm/complib/Makefile.am(revision 5098) +++ management/osm/complib/Makefile.am(working copy) @@ -5,8 +5,6 @@ if DEBUG DBGFLAGS = -ggdb -D_DEBUG_ -else -DBGFLAGS = -g -O2 endif libosmcomp_la_CFLAGS = -Wall $(DBGFLAGS)
RE: [openib-general] respect CFLAGS in ibis/ibdm, fix missing file warning
Hi Pete, Please see my response to the similar patch for OpenSM. I think you can still apply your optimization flags simply by using the CFLAGS at the command line. Please double check and let me know. Thanks Eitan Zahavi Design Technology Director Mellanox Technologies LTD Tel:+972-4-9097208 Fax:+972-4-9593245 P.O. Box 586 Yokneam 20692 ISRAEL -Original Message- From: [EMAIL PROTECTED] [mailto:openib-general- [EMAIL PROTECTED] On Behalf Of Pete Wyckoff Sent: Thursday, January 19, 2006 11:53 PM To: openib-general@openib.org Subject: [openib-general] respect CFLAGS in ibis/ibdm,fix missing file warning Avoid overriding CFLAGS in ibis and ibdm. These apply to ibis and ibdm below gen2/utils/src/linux-user/. The third chunk below avoids a configure warning for a file osm_build_id.h that appears nowhere in my source or build tree. Signed-off-by: Pete Wyckoff [EMAIL PROTECTED] Index: ibis/src/Makefile.am === --- ibis/src/Makefile.am (revision 5098) +++ ibis/src/Makefile.am (working copy) @@ -38,7 +38,7 @@ if DEBUG DBG = -O0 -g -Wall -Werror else -DBG = -O2 -Wall +DBG = -Wall endif AM_CFLAGS = $(TCL_CPPFLAGS) $(OSM_CFLAGS) $(DBG) -fno-strict-aliasing - fPIC Index: ibdm/datamodel/Makefile.am === --- ibdm/datamodel/Makefile.am(revision 5098) +++ ibdm/datamodel/Makefile.am(working copy) @@ -60,8 +60,6 @@ # Support debug mode through config variable if DEBUG DBG = -O0 -g -else -DBG = -O2 endif # We have a special mode where we know the package will be eventually moved Index: ibis/config/osm.m4 === --- ibis/config/osm.m4(revision 5098) +++ ibis/config/osm.m4(working copy) @@ -156,6 +156,8 @@ AM_CONDITIONAL(OSM_VENDOR_SIM, test $OSM_VENDOR = sim) AM_CONDITIONAL(OSM_BUILD_OPENIB, test $OSM_BUILD = openib) +if test -f $osm_include_dir/opensm/osm_build_id.h; then + dnl validate the defined path - so the build id header is there AC_CHECK_FILE($osm_include_dir/opensm/osm_build_id.h,, AC_MSG_ERROR([OSM: could not find $with_osm/include/opensm/osm_build_id.h])) @@ -168,6 +170,9 @@ else osm_debug_flags= fi +else + osm_debug_flags= +fi OSM_CFLAGS=-I$osm_include_dir $osm_extra_includes $osm_debug_flags $osm_vendor_sel -D_XOPEN_SOURCE=600 -D_BSD_SOURCE=1 ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] respect CFLAGS in OSM
On Mon, 2006-01-23 at 08:27, Eitan Zahavi wrote: Hi Pete, I have looked again at this patch and what it is changing. My understanding is that you found the -g -O2 CLFAGS (provided through the specific target CFLAGS) unneeded. You also think they will interfere with settings you might want to provide from the command line. I have just double checked what I new to be the rule for autoconf: If the user provides CFLAGS or LDFLAGS from the command like - they are appended to the compile or link flags. The impact on gcc is that the later settings - i.e. those provided by the user take precedence over the flags provided at the beginning of the command line. So the patch below is actually not needed. Just to convince you I attach some gcc traces showing that -O0 -O2 acts like -O2 and -O2 -O0 acts like -O0. Yes, it does override. What about the -g setting ? Should that stay or go ? -- Hal Bottom line I would like to keep the code as it is without any change such that default installation will use the -O2 mode. Eitan Zahavi Design Technology Director Mellanox Technologies LTD Tel:+972-4-9097208 Fax:+972-4-9593245 P.O. Box 586 Yokneam 20692 ISRAEL swlab25:/home/eitan/SW/work/examples/usr/libexec/gcc/i386-redhat-linux/ 4.0.0/cc1 getHostName.c getHostName.c -auxbase getHostName -O0 -O2 -version -o /tmp/ccet3OkS.s GNU C version 4.0.0 20050519 (Red Hat 4.0.0-8) (i386-redhat-linux) compiled by GNU C version 4.0.0 20050519 (Red Hat 4.0.0-8). GGC heuristics: --param ggc-min-expand=99 --param ggc-min-heapsize=129317 options passed: -auxbase -O0 -O2 options enabled: -falign-loops -fargument-alias -fbranch-count-reg -fcaller-saves -fcommon -fcprop-registers -fcrossjumping -fcse-follow-jumps -fcse-skip-blocks -fdefer-pop -fdelete-null-pointer-checks -feliminate-unused-debug-types -fexpensive-optimizations -fforce-mem -ffunction-cse -fgcse -fgcse-lm -fguess-branch-probability -fident -fif-conversion -fif-conversion2 -fivopts -fkeep-static-consts -fleading-underscore -floop-optimize -floop-optimize2 -fmath-errno -fmerge-constants -foptimize-register-move -foptimize-sibling-calls -fpcc-struct-return -fpeephole -fpeephole2 -fregmove -freorder-blocks -freorder-functions -frerun-cse-after-loop -frerun-loop-opt -fsched-interblock -fsched-spec -fsched-stalled-insns-dep -fsplit-ivs-in-unroller -fstrength-reduce -fstrict-aliasing -fthread-jumps -ftrapping-math -ftree-ccp -ftree-ch -ftree-copyrename -ftree-dce -ftree-dominator-opts -ftree-dse -ftree-fre -ftree-loop-im -ftree-loop-ivcanon -ftree-loop-optimize -ftree-lrs -ftree-pre -ftree-sra -ftree-ter -funit-at-a-time -fvar-tracking -fzero-initialized-in-bss -m80387 -mhard-float -mno-soft-float -mieee-fp -mfp-ret-in-387 -mno-red-zone -mtls-direct-seg-refs -mtune=i386 -march=i386 swlab25:/home/eitan/SW/work/examples/usr/libexec/gcc/i386-redhat-linux/ 4.0.0/cc1 getHostName.c getHostName.c -auxbase getHostName -O2 -O0 -version -o /tmp/ccet3OkS.s GNU C version 4.0.0 20050519 (Red Hat 4.0.0-8) (i386-redhat-linux) compiled by GNU C version 4.0.0 20050519 (Red Hat 4.0.0-8). GGC heuristics: --param ggc-min-expand=99 --param ggc-min-heapsize=129317 options passed: -auxbase -O2 -O0 options enabled: -falign-loops -fargument-alias -fbranch-count-reg -fcommon -feliminate-unused-debug-types -ffunction-cse -fgcse-lm -fident -fivopts -fkeep-static-consts -fleading-underscore -floop-optimize2 -fmath-errno -fpcc-struct-return -fpeephole -fsched-interblock -fsched-spec -fsched-stalled-insns-dep -fsplit-ivs-in-unroller -ftrapping-math -ftree-loop-im -ftree-loop-ivcanon -ftree-loop-optimize -funit-at-a-time -fvar-tracking -fzero-initialized-in-bss -m80387 -mhard-float -mno-soft-float -mieee-fp -mfp-ret-in-387 -mno-red-zone -mtls-direct-seg-refs -mtune=i386 -march=i386 -Original Message- From: [EMAIL PROTECTED] [mailto:openib-general- [EMAIL PROTECTED] On Behalf Of Pete Wyckoff Sent: Thursday, January 19, 2006 11:50 PM To: openib-general@openib.org Subject: [openib-general] respect CFLAGS in OSM I do something like: CFLAGS=-g ./configure ... to build a debug tree from openib svn. Some places override this CFLAGS setting, though, applying optimization even though I explicitly do not want it. This patch fixes that. These apply to OSM below gen2/trunk/src/userspace/. Signed-off-by: Pete Wyckoff [EMAIL PROTECTED] Index: management/osm/libvendor/Makefile.am === --- management/osm/libvendor/Makefile.am(revision 5098) +++ management/osm/libvendor/Makefile.am(working copy) @@ -3,8 +3,6 @@ if DEBUG DBGFLAGS = -ggdb -D_DEBUG_ -else -DBGFLAGS = -g -O2 endif INCLUDES = $(OSMV_INCLUDES) Index: management/osm/complib/Makefile.am === ---
RE: [openib-general] respect CFLAGS in OSM
Just to convince you I attach some gcc traces showing that -O0 -O2 acts like -O2 and -O2 -O0 acts like -O0. Yes, it does override. What about the -g setting ? Should that stay or go ? [EZ] If -g is the problem we could easily remove the -g . However I would recommend having OpenSM compile -O2 by default and not rely on the user to provide that. -- Hal Bottom line I would like to keep the code as it is without any change such that default installation will use the -O2 mode. Eitan Zahavi Design Technology Director Mellanox Technologies LTD Tel:+972-4-9097208 Fax:+972-4-9593245 P.O. Box 586 Yokneam 20692 ISRAEL swlab25:/home/eitan/SW/work/examples/usr/libexec/gcc/i386-redhat-linux/ 4.0.0/cc1 getHostName.c getHostName.c -auxbase getHostName -O0 -O2 -version -o /tmp/ccet3OkS.s GNU C version 4.0.0 20050519 (Red Hat 4.0.0-8) (i386-redhat-linux) compiled by GNU C version 4.0.0 20050519 (Red Hat 4.0.0-8). GGC heuristics: --param ggc-min-expand=99 --param ggc-min-heapsize=129317 options passed: -auxbase -O0 -O2 options enabled: -falign-loops -fargument-alias -fbranch-count-reg -fcaller-saves -fcommon -fcprop-registers -fcrossjumping -fcse-follow-jumps -fcse-skip-blocks -fdefer-pop -fdelete-null-pointer-checks -feliminate-unused-debug-types -fexpensive-optimizations -fforce-mem -ffunction-cse -fgcse -fgcse-lm -fguess-branch-probability -fident -fif-conversion -fif-conversion2 -fivopts -fkeep-static-consts -fleading-underscore -floop-optimize -floop-optimize2 -fmath-errno -fmerge-constants -foptimize-register-move -foptimize-sibling-calls -fpcc-struct-return -fpeephole -fpeephole2 -fregmove -freorder-blocks -freorder-functions -frerun-cse-after-loop -frerun-loop-opt -fsched-interblock -fsched-spec -fsched-stalled-insns-dep -fsplit-ivs-in-unroller -fstrength-reduce -fstrict-aliasing -fthread-jumps -ftrapping-math -ftree-ccp -ftree-ch -ftree-copyrename -ftree-dce -ftree-dominator-opts -ftree-dse -ftree-fre -ftree-loop-im -ftree-loop-ivcanon -ftree-loop-optimize -ftree-lrs -ftree-pre -ftree-sra -ftree-ter -funit-at-a-time -fvar-tracking -fzero-initialized-in-bss -m80387 -mhard-float -mno-soft-float -mieee-fp -mfp-ret-in-387 -mno-red-zone -mtls-direct-seg-refs -mtune=i386 -march=i386 swlab25:/home/eitan/SW/work/examples/usr/libexec/gcc/i386-redhat-linux/ 4.0.0/cc1 getHostName.c getHostName.c -auxbase getHostName -O2 -O0 -version -o /tmp/ccet3OkS.s GNU C version 4.0.0 20050519 (Red Hat 4.0.0-8) (i386-redhat-linux) compiled by GNU C version 4.0.0 20050519 (Red Hat 4.0.0-8). GGC heuristics: --param ggc-min-expand=99 --param ggc-min-heapsize=129317 options passed: -auxbase -O2 -O0 options enabled: -falign-loops -fargument-alias -fbranch-count-reg -fcommon -feliminate-unused-debug-types -ffunction-cse -fgcse-lm -fident -fivopts -fkeep-static-consts -fleading-underscore -floop-optimize2 -fmath-errno -fpcc-struct-return -fpeephole -fsched-interblock -fsched-spec -fsched-stalled-insns-dep -fsplit-ivs-in-unroller -ftrapping-math -ftree-loop-im -ftree-loop-ivcanon -ftree-loop-optimize -funit-at-a-time -fvar-tracking -fzero-initialized-in-bss -m80387 -mhard-float -mno-soft-float -mieee-fp -mfp-ret-in-387 -mno-red-zone -mtls-direct-seg-refs -mtune=i386 -march=i386 -Original Message- From: [EMAIL PROTECTED] [mailto:openib-general- [EMAIL PROTECTED] On Behalf Of Pete Wyckoff Sent: Thursday, January 19, 2006 11:50 PM To: openib-general@openib.org Subject: [openib-general] respect CFLAGS in OSM I do something like: CFLAGS=-g ./configure ... to build a debug tree from openib svn. Some places override this CFLAGS setting, though, applying optimization even though I explicitly do not want it. This patch fixes that. These apply to OSM below gen2/trunk/src/userspace/. Signed-off-by: Pete Wyckoff [EMAIL PROTECTED] Index: management/osm/libvendor/Makefile.am === --- management/osm/libvendor/Makefile.am (revision 5098) +++ management/osm/libvendor/Makefile.am (working copy) @@ -3,8 +3,6 @@ if DEBUG DBGFLAGS = -ggdb -D_DEBUG_ -else -DBGFLAGS = -g -O2 endif INCLUDES = $(OSMV_INCLUDES) Index: management/osm/complib/Makefile.am === --- management/osm/complib/Makefile.am(revision 5098) +++ management/osm/complib/Makefile.am(working copy) @@ -5,8 +5,6 @@ if DEBUG DBGFLAGS = -ggdb -D_DEBUG_ -else -DBGFLAGS = -g -O2 endif libosmcomp_la_CFLAGS = -Wall $(DBGFLAGS) -D_XOPEN_SOURCE=600 - D_BSD_SOURCE=1 Index: management/osm/opensm/Makefile.am === --- management/osm/opensm/Makefile.am (revision 5098) +++
RE: [openib-general] respect CFLAGS in OSM
On Mon, 2006-01-23 at 08:39, Hal Rosenstock wrote: On Mon, 2006-01-23 at 08:27, Eitan Zahavi wrote: Hi Pete, I have looked again at this patch and what it is changing. My understanding is that you found the -g -O2 CLFAGS (provided through the specific target CFLAGS) unneeded. You also think they will interfere with settings you might want to provide from the command line. I have just double checked what I new to be the rule for autoconf: If the user provides CFLAGS or LDFLAGS from the command like - they are appended to the compile or link flags. The impact on gcc is that the later settings - i.e. those provided by the user take precedence over the flags provided at the beginning of the command line. So the patch below is actually not needed. Just to convince you I attach some gcc traces showing that -O0 -O2 acts like -O2 and -O2 -O0 acts like -O0. Yes, it does override. Actually I'm not sure about this. I too see both but can't tell which was honored and didn't see the specific text in gcc to indicate the precedence here. Is it first option or last option of the same or something else ? -- Hal What about the -g setting ? Should that stay or go ? -- Hal Bottom line I would like to keep the code as it is without any change such that default installation will use the -O2 mode. Eitan Zahavi Design Technology Director Mellanox Technologies LTD Tel:+972-4-9097208 Fax:+972-4-9593245 P.O. Box 586 Yokneam 20692 ISRAEL swlab25:/home/eitan/SW/work/examples/usr/libexec/gcc/i386-redhat-linux/ 4.0.0/cc1 getHostName.c getHostName.c -auxbase getHostName -O0 -O2 -version -o /tmp/ccet3OkS.s GNU C version 4.0.0 20050519 (Red Hat 4.0.0-8) (i386-redhat-linux) compiled by GNU C version 4.0.0 20050519 (Red Hat 4.0.0-8). GGC heuristics: --param ggc-min-expand=99 --param ggc-min-heapsize=129317 options passed: -auxbase -O0 -O2 options enabled: -falign-loops -fargument-alias -fbranch-count-reg -fcaller-saves -fcommon -fcprop-registers -fcrossjumping -fcse-follow-jumps -fcse-skip-blocks -fdefer-pop -fdelete-null-pointer-checks -feliminate-unused-debug-types -fexpensive-optimizations -fforce-mem -ffunction-cse -fgcse -fgcse-lm -fguess-branch-probability -fident -fif-conversion -fif-conversion2 -fivopts -fkeep-static-consts -fleading-underscore -floop-optimize -floop-optimize2 -fmath-errno -fmerge-constants -foptimize-register-move -foptimize-sibling-calls -fpcc-struct-return -fpeephole -fpeephole2 -fregmove -freorder-blocks -freorder-functions -frerun-cse-after-loop -frerun-loop-opt -fsched-interblock -fsched-spec -fsched-stalled-insns-dep -fsplit-ivs-in-unroller -fstrength-reduce -fstrict-aliasing -fthread-jumps -ftrapping-math -ftree-ccp -ftree-ch -ftree-copyrename -ftree-dce -ftree-dominator-opts -ftree-dse -ftree-fre -ftree-loop-im -ftree-loop-ivcanon -ftree-loop-optimize -ftree-lrs -ftree-pre -ftree-sra -ftree-ter -funit-at-a-time -fvar-tracking -fzero-initialized-in-bss -m80387 -mhard-float -mno-soft-float -mieee-fp -mfp-ret-in-387 -mno-red-zone -mtls-direct-seg-refs -mtune=i386 -march=i386 swlab25:/home/eitan/SW/work/examples/usr/libexec/gcc/i386-redhat-linux/ 4.0.0/cc1 getHostName.c getHostName.c -auxbase getHostName -O2 -O0 -version -o /tmp/ccet3OkS.s GNU C version 4.0.0 20050519 (Red Hat 4.0.0-8) (i386-redhat-linux) compiled by GNU C version 4.0.0 20050519 (Red Hat 4.0.0-8). GGC heuristics: --param ggc-min-expand=99 --param ggc-min-heapsize=129317 options passed: -auxbase -O2 -O0 options enabled: -falign-loops -fargument-alias -fbranch-count-reg -fcommon -feliminate-unused-debug-types -ffunction-cse -fgcse-lm -fident -fivopts -fkeep-static-consts -fleading-underscore -floop-optimize2 -fmath-errno -fpcc-struct-return -fpeephole -fsched-interblock -fsched-spec -fsched-stalled-insns-dep -fsplit-ivs-in-unroller -ftrapping-math -ftree-loop-im -ftree-loop-ivcanon -ftree-loop-optimize -funit-at-a-time -fvar-tracking -fzero-initialized-in-bss -m80387 -mhard-float -mno-soft-float -mieee-fp -mfp-ret-in-387 -mno-red-zone -mtls-direct-seg-refs -mtune=i386 -march=i386 -Original Message- From: [EMAIL PROTECTED] [mailto:openib-general- [EMAIL PROTECTED] On Behalf Of Pete Wyckoff Sent: Thursday, January 19, 2006 11:50 PM To: openib-general@openib.org Subject: [openib-general] respect CFLAGS in OSM I do something like: CFLAGS=-g ./configure ... to build a debug tree from openib svn. Some places override this CFLAGS setting, though, applying optimization even though I explicitly do not want it. This patch fixes that. These apply to OSM below gen2/trunk/src/userspace/. Signed-off-by: Pete Wyckoff [EMAIL PROTECTED] Index: management/osm/libvendor/Makefile.am === ---
RE: [openib-general] respect CFLAGS in OSM
Actually I'm not sure about this. I too see both but can't tell which was honored and didn't see the specific text in gcc to indicate the precedence here. Is it first option or last option of the same or something else ? [EZ] This is exactly why I have added the traces from running gcc with the two options. If you look just below you will see the verbose report from cc1 showing that the last option wins by showing the exact list of optimization options used by the compiler. I have tested that on multiple gcc versions. (The command line itself is reported when you run gcc -v. Then you need to remove the -quite flag to get the correct level of verbosity. ) swlab25:/home/eitan/SW/work/examples/usr/libexec/gcc/i386-redhat-linux/ 4.0.0/cc1 getHostName.c getHostName.c -auxbase getHostName -O0 -O2 -version -o /tmp/ccet3OkS.s GNU C version 4.0.0 20050519 (Red Hat 4.0.0-8) (i386-redhat-linux) compiled by GNU C version 4.0.0 20050519 (Red Hat 4.0.0-8). GGC heuristics: --param ggc-min-expand=99 --param ggc-min-heapsize=129317 options passed: -auxbase -O0 -O2 options enabled: -falign-loops -fargument-alias -fbranch-count-reg -fcaller-saves -fcommon -fcprop-registers -fcrossjumping -fcse-follow-jumps -fcse-skip-blocks -fdefer-pop -fdelete-null-pointer-checks -feliminate-unused-debug-types -fexpensive-optimizations -fforce-mem -ffunction-cse -fgcse -fgcse-lm -fguess-branch-probability -fident -fif-conversion -fif-conversion2 -fivopts -fkeep-static-consts -fleading-underscore -floop-optimize -floop-optimize2 -fmath-errno -fmerge-constants -foptimize-register-move -foptimize-sibling-calls -fpcc-struct-return -fpeephole -fpeephole2 -fregmove -freorder-blocks -freorder-functions -frerun-cse-after-loop -frerun-loop-opt -fsched-interblock -fsched-spec -fsched-stalled-insns-dep -fsplit-ivs-in-unroller -fstrength-reduce -fstrict-aliasing -fthread-jumps -ftrapping-math -ftree-ccp -ftree-ch -ftree-copyrename -ftree-dce -ftree-dominator-opts -ftree-dse -ftree-fre -ftree-loop-im -ftree-loop-ivcanon -ftree-loop-optimize -ftree-lrs -ftree-pre -ftree-sra -ftree-ter -funit-at-a-time -fvar-tracking -fzero-initialized-in-bss -m80387 -mhard-float -mno-soft-float -mieee-fp -mfp-ret-in-387 -mno-red-zone -mtls-direct-seg-refs -mtune=i386 -march=i386 swlab25:/home/eitan/SW/work/examples/usr/libexec/gcc/i386-redhat-linux/ 4.0.0/cc1 getHostName.c getHostName.c -auxbase getHostName -O2 -O0 -version -o /tmp/ccet3OkS.s GNU C version 4.0.0 20050519 (Red Hat 4.0.0-8) (i386-redhat-linux) compiled by GNU C version 4.0.0 20050519 (Red Hat 4.0.0-8). GGC heuristics: --param ggc-min-expand=99 --param ggc-min-heapsize=129317 options passed: -auxbase -O2 -O0 options enabled: -falign-loops -fargument-alias -fbranch-count-reg -fcommon -feliminate-unused-debug-types -ffunction-cse -fgcse-lm -fident -fivopts -fkeep-static-consts -fleading-underscore -floop-optimize2 -fmath-errno -fpcc-struct-return -fpeephole -fsched-interblock -fsched-spec -fsched-stalled-insns-dep -fsplit-ivs-in-unroller -ftrapping-math -ftree-loop-im -ftree-loop-ivcanon -ftree-loop-optimize -funit-at-a-time -fvar-tracking -fzero-initialized-in-bss -m80387 -mhard-float -mno-soft-float -mieee-fp -mfp-ret-in-387 -mno-red-zone -mtls-direct-seg-refs -mtune=i386 -march=i386 ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] Re: [PATCH] Opensm - running with console option
On Sun, 2006-01-22 at 07:29, Yael Kalka wrote: Hi Hal, I've noticed that when running opensm with --console, it exits with a message that option `-console' requires an argument. But I saw in the main.c that such argument isn't used. The following patch removes this dependency. Thanks, Yael Thanks. Applied. Signed-off-by: Yael Kalka [EMAIL PROTECTED] ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] respect CFLAGS in OSM
On Mon, 2006-01-23 at 09:10, Eitan Zahavi wrote: Actually I'm not sure about this. I too see both but can't tell which was honored and didn't see the specific text in gcc to indicate the precedence here. Is it first option or last option of the same or something else ? [EZ] This is exactly why I have added the traces from running gcc with the two options. If you look just below you will see the verbose report from cc1 showing that the last option wins by showing the exact list of optimization options used by the compiler. I have tested that on multiple gcc versions. (The command line itself is reported when you run gcc -v. Then you need to remove the -quite flag to get the correct level of verbosity. ) Got it. It's all the extra -f options that are enabled by -O2. I will revert the patch to the OSM makefiles. -- Hal swlab25:/home/eitan/SW/work/examples/usr/libexec/gcc/i386-redhat-linux/ 4.0.0/cc1 getHostName.c getHostName.c -auxbase getHostName -O0 -O2 -version -o /tmp/ccet3OkS.s GNU C version 4.0.0 20050519 (Red Hat 4.0.0-8) (i386-redhat-linux) compiled by GNU C version 4.0.0 20050519 (Red Hat 4.0.0-8). GGC heuristics: --param ggc-min-expand=99 --param ggc-min-heapsize=129317 options passed: -auxbase -O0 -O2 options enabled: -falign-loops -fargument-alias -fbranch-count-reg -fcaller-saves -fcommon -fcprop-registers -fcrossjumping -fcse-follow-jumps -fcse-skip-blocks -fdefer-pop -fdelete-null-pointer-checks -feliminate-unused-debug-types -fexpensive-optimizations -fforce-mem -ffunction-cse -fgcse -fgcse-lm -fguess-branch-probability -fident -fif-conversion -fif-conversion2 -fivopts -fkeep-static-consts -fleading-underscore -floop-optimize -floop-optimize2 -fmath-errno -fmerge-constants -foptimize-register-move -foptimize-sibling-calls -fpcc-struct-return -fpeephole -fpeephole2 -fregmove -freorder-blocks -freorder-functions -frerun-cse-after-loop -frerun-loop-opt -fsched-interblock -fsched-spec -fsched-stalled-insns-dep -fsplit-ivs-in-unroller -fstrength-reduce -fstrict-aliasing -fthread-jumps -ftrapping-math -ftree-ccp -ftree-ch -ftree-copyrename -ftree-dce -ftree-dominator-opts -ftree-dse -ftree-fre -ftree-loop-im -ftree-loop-ivcanon -ftree-loop-optimize -ftree-lrs -ftree-pre -ftree-sra -ftree-ter -funit-at-a-time -fvar-tracking -fzero-initialized-in-bss -m80387 -mhard-float -mno-soft-float -mieee-fp -mfp-ret-in-387 -mno-red-zone -mtls-direct-seg-refs -mtune=i386 -march=i386 swlab25:/home/eitan/SW/work/examples/usr/libexec/gcc/i386-redhat-linux/ 4.0.0/cc1 getHostName.c getHostName.c -auxbase getHostName -O2 -O0 -version -o /tmp/ccet3OkS.s GNU C version 4.0.0 20050519 (Red Hat 4.0.0-8) (i386-redhat-linux) compiled by GNU C version 4.0.0 20050519 (Red Hat 4.0.0-8). GGC heuristics: --param ggc-min-expand=99 --param ggc-min-heapsize=129317 options passed: -auxbase -O2 -O0 options enabled: -falign-loops -fargument-alias -fbranch-count-reg -fcommon -feliminate-unused-debug-types -ffunction-cse -fgcse-lm -fident -fivopts -fkeep-static-consts -fleading-underscore -floop-optimize2 -fmath-errno -fpcc-struct-return -fpeephole -fsched-interblock -fsched-spec -fsched-stalled-insns-dep -fsplit-ivs-in-unroller -ftrapping-math -ftree-loop-im -ftree-loop-ivcanon -ftree-loop-optimize -funit-at-a-time -fvar-tracking -fzero-initialized-in-bss -m80387 -mhard-float -mno-soft-float -mieee-fp -mfp-ret-in-387 -mno-red-zone -mtls-direct-seg-refs -mtune=i386 -march=i386 ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] respect CFLAGS in OSM
On Mon, 2006-01-23 at 09:10, Eitan Zahavi wrote: Actually I'm not sure about this. I too see both but can't tell which was honored and didn't see the specific text in gcc to indicate the precedence here. Is it first option or last option of the same or something else ? [EZ] This is exactly why I have added the traces from running gcc with the two options. If you look just below you will see the verbose report from cc1 showing that the last option wins by showing the exact list of optimization options used by the compiler. I have tested that on multiple gcc versions. (The command line itself is reported when you run gcc -v. Then you need to remove the -quite flag to get the correct level of verbosity. ) Got it. It's all the extra -f options that are enabled by -O2. I will revert the patch to the OSM makefiles. [EZ] Let's wait for Pete's response. -- Hal swlab25:/home/eitan/SW/work/examples/usr/libexec/gcc/i386-redhat-linux/ 4.0.0/cc1 getHostName.c getHostName.c -auxbase getHostName -O0 -O2 -version -o /tmp/ccet3OkS.s GNU C version 4.0.0 20050519 (Red Hat 4.0.0-8) (i386-redhat-linux) compiled by GNU C version 4.0.0 20050519 (Red Hat 4.0.0-8). GGC heuristics: --param ggc-min-expand=99 --param ggc-min-heapsize=129317 options passed: -auxbase -O0 -O2 options enabled: -falign-loops -fargument-alias -fbranch-count-reg -fcaller-saves -fcommon -fcprop-registers -fcrossjumping -fcse-follow-jumps -fcse-skip-blocks -fdefer-pop -fdelete-null-pointer-checks -feliminate-unused-debug-types -fexpensive-optimizations -fforce-mem -ffunction-cse -fgcse -fgcse-lm -fguess-branch-probability -fident -fif-conversion -fif-conversion2 -fivopts -fkeep-static-consts -fleading-underscore -floop-optimize -floop-optimize2 -fmath-errno -fmerge-constants -foptimize-register-move -foptimize-sibling-calls -fpcc-struct-return -fpeephole -fpeephole2 -fregmove -freorder-blocks -freorder-functions -frerun-cse-after-loop -frerun-loop-opt -fsched-interblock -fsched-spec -fsched-stalled-insns-dep -fsplit-ivs-in-unroller -fstrength-reduce -fstrict-aliasing -fthread-jumps -ftrapping-math -ftree-ccp -ftree-ch -ftree-copyrename -ftree-dce -ftree-dominator-opts -ftree-dse -ftree-fre -ftree-loop-im -ftree-loop-ivcanon -ftree-loop-optimize -ftree-lrs -ftree-pre -ftree-sra -ftree-ter -funit-at-a-time -fvar-tracking -fzero-initialized-in-bss -m80387 -mhard-float -mno-soft-float -mieee-fp -mfp-ret-in-387 -mno-red-zone -mtls-direct-seg-refs -mtune=i386 -march=i386 swlab25:/home/eitan/SW/work/examples/usr/libexec/gcc/i386-redhat-linux/ 4.0.0/cc1 getHostName.c getHostName.c -auxbase getHostName -O2 -O0 -version -o /tmp/ccet3OkS.s GNU C version 4.0.0 20050519 (Red Hat 4.0.0-8) (i386-redhat-linux) compiled by GNU C version 4.0.0 20050519 (Red Hat 4.0.0-8). GGC heuristics: --param ggc-min-expand=99 --param ggc-min-heapsize=129317 options passed: -auxbase -O2 -O0 options enabled: -falign-loops -fargument-alias -fbranch-count-reg -fcommon -feliminate-unused-debug-types -ffunction-cse -fgcse-lm -fident -fivopts -fkeep-static-consts -fleading-underscore -floop-optimize2 -fmath-errno -fpcc-struct-return -fpeephole -fsched-interblock -fsched-spec -fsched-stalled-insns-dep -fsplit-ivs-in-unroller -ftrapping-math -ftree-loop-im -ftree-loop-ivcanon -ftree-loop-optimize -funit-at-a-time -fvar-tracking -fzero-initialized-in-bss -m80387 -mhard-float -mno-soft-float -mieee-fp -mfp-ret-in-387 -mno-red-zone -mtls-direct-seg-refs -mtune=i386 -march=i386 ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH] devinfo multiple device support
For devinfo, it makes sense to report all available devices
if the user did not specify a specific one.
---
Make ibv_devinfo list all IB devices, rather than the first device, by default.
Signed-off-by: Dotan Barak [EMAIL PROTECTED]
Signed-off-by: Michael S. Tsirkin [EMAIL PROTECTED]
Index: last_stable/src/userspace/libibverbs/examples/devinfo.c
===
--- last_stable.orig/src/userspace/libibverbs/examples/devinfo.c
2006-01-22 13:19:22.0 +0200
+++ last_stable/src/userspace/libibverbs/examples/devinfo.c 2006-01-22
13:18:23.0 +0200
@@ -403,7 +403,10 @@
fprintf(stderr, No IB devices found\n);
return -1;
}
- ret |= print_hca_cap(*dev_list, ib_port);
+ while (*dev_list) {
+ ret |= print_hca_cap(*dev_list, ib_port);
+ ++dev_list;
+ }
}
if (ib_devname)
--
MST
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] Re: [PATCH] devinfo multiple device support
thanks, applied. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH] RFC: mthca handling of signals
We have run into the following problem: if a task receives
a signal while in the process of e.g. destroying a resource
(which could be because the relevant file was closed)
mthca could bail out from trying to take a command
interface semaphore without performing the appropriate
command to tell hardware that the resource is being destroyed.
As a result we see messages like
ib_mthca :04:00.0: HW2SW_CQ failed (-4)
In this case, hardware could access the resource after the memory has been
freed, possibly causing memory corruption.
A simple solution is to replace down_interruptible by down in command
interface activation.
A more elegant, but much bigger, change would involve making
resource allocation command interruptible, while keeping
resource cleanup commands uninterruptible.
---
Its not safe to cancel a command upon a signal.
Signed-off-by: Michael S. Tsirkin [EMAIL PROTECTED]
Index: openib/drivers/infiniband/hw/mthca/mthca_cmd.c
===
--- openib/drivers/infiniband/hw/mthca/mthca_cmd.c (revision 5121)
+++ openib/drivers/infiniband/hw/mthca/mthca_cmd.c (working copy)
@@ -199,8 +199,7 @@ static int mthca_cmd_post(struct mthca_d
{
int err = 0;
- if (down_interruptible(dev-cmd.hcr_sem))
- return -EINTR;
+ down(dev-cmd.hcr_sem);
if (event) {
unsigned long end = jiffies + GO_BIT_TIMEOUT;
@@ -255,8 +254,7 @@ static int mthca_cmd_poll(struct mthca_d
int err = 0;
unsigned long end;
- if (down_interruptible(dev-cmd.poll_sem))
- return -EINTR;
+ down(dev-cmd.poll_sem);
err = mthca_cmd_post(dev, in_param,
out_param ? *out_param : 0,
@@ -333,8 +331,7 @@ static int mthca_cmd_wait(struct mthca_d
int err = 0;
struct mthca_cmd_context *context;
- if (down_interruptible(dev-cmd.event_sem))
- return -EINTR;
+ down(dev-cmd.event_sem);
spin_lock(dev-cmd.context_lock);
BUG_ON(dev-cmd.free_head 0);
--
MST
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [git pull] InfiniBand fixes for 2.6.16
Linus, please pull from
master.kernel.org:/pub/scm/linux/kernel/git/roland/infiniband.git for-linus
This tree is also available from kernel.org mirrors at:
git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git
for-linus
The pull will get the following changes:
Michael S. Tsirkin:
IPoIB: Make sure path is fully initialized before using it
IB/uverbs: Flush scheduled work before unloading module
IB/sa_query: Flush scheduled work before unloading module
IPoIB: Lock accesses to multicast packet queues
IB/mthca: Use correct GID in MADs sent on port 2
drivers/infiniband/core/sa_query.c |2 ++
drivers/infiniband/core/uverbs_main.c |1 +
drivers/infiniband/hw/mthca/mthca_av.c |2 +-
drivers/infiniband/ulp/ipoib/ipoib_main.c |4 ++--
drivers/infiniband/ulp/ipoib/ipoib_multicast.c | 25 +---
5 files changed, 28 insertions(+), 6 deletions(-)
diff --git a/drivers/infiniband/core/sa_query.c
b/drivers/infiniband/core/sa_query.c
index acda7d6..501cc05 100644
--- a/drivers/infiniband/core/sa_query.c
+++ b/drivers/infiniband/core/sa_query.c
@@ -956,6 +956,8 @@ static void ib_sa_remove_one(struct ib_d
ib_unregister_event_handler(sa_dev-event_handler);
+ flush_scheduled_work();
+
for (i = 0; i = sa_dev-end_port - sa_dev-start_port; ++i) {
ib_unregister_mad_agent(sa_dev-port[i].agent);
kref_put(sa_dev-port[i].sm_ah-ref, free_sm_ah);
diff --git a/drivers/infiniband/core/uverbs_main.c
b/drivers/infiniband/core/uverbs_main.c
index 96ea79b..903f85a 100644
--- a/drivers/infiniband/core/uverbs_main.c
+++ b/drivers/infiniband/core/uverbs_main.c
@@ -902,6 +902,7 @@ static void __exit ib_uverbs_cleanup(voi
unregister_filesystem(uverbs_event_fs);
class_destroy(uverbs_class);
unregister_chrdev_region(IB_UVERBS_BASE_DEV, IB_UVERBS_MAX_DEVICES);
+ flush_scheduled_work();
idr_destroy(ib_uverbs_pd_idr);
idr_destroy(ib_uverbs_mr_idr);
idr_destroy(ib_uverbs_mw_idr);
diff --git a/drivers/infiniband/hw/mthca/mthca_av.c
b/drivers/infiniband/hw/mthca/mthca_av.c
index a14eed0..a19e0ed 100644
--- a/drivers/infiniband/hw/mthca/mthca_av.c
+++ b/drivers/infiniband/hw/mthca/mthca_av.c
@@ -184,7 +184,7 @@ int mthca_read_ah(struct mthca_dev *dev,
ah-av-sl_tclass_flowlabel cpu_to_be32(0xf);
ib_get_cached_gid(dev-ib_dev,
be32_to_cpu(ah-av-port_pd) 24,
- ah-av-gid_index,
+ ah-av-gid_index % dev-limits.gid_table_len,
header-grh.source_gid);
memcpy(header-grh.destination_gid.raw,
ah-av-dgid, 16);
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c
b/drivers/infiniband/ulp/ipoib/ipoib_main.c
index fd3f5c8..c3b5f79 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_main.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c
@@ -505,7 +505,7 @@ static void neigh_add_path(struct sk_buf
list_add_tail(neigh-list, path-neigh_list);
- if (path-pathrec.dlid) {
+ if (path-ah) {
kref_get(path-ah-ref);
neigh-ah = path-ah;
@@ -591,7 +591,7 @@ static void unicast_arp_send(struct sk_b
return;
}
- if (path-pathrec.dlid) {
+ if (path-ah) {
ipoib_dbg(priv, Send unicast ARP to %04x\n,
be16_to_cpu(path-pathrec.dlid));
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c
b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c
index 98039da..ccaa0c3 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c
@@ -97,6 +97,7 @@ static void ipoib_mcast_free(struct ipoi
struct ipoib_dev_priv *priv = netdev_priv(dev);
struct ipoib_neigh *neigh, *tmp;
unsigned long flags;
+ int tx_dropped = 0;
ipoib_dbg_mcast(netdev_priv(dev),
deleting multicast group IPOIB_GID_FMT \n,
@@ -123,8 +124,14 @@ static void ipoib_mcast_free(struct ipoi
if (mcast-ah)
ipoib_put_ah(mcast-ah);
- while (!skb_queue_empty(mcast-pkt_queue))
+ while (!skb_queue_empty(mcast-pkt_queue)) {
+ ++tx_dropped;
dev_kfree_skb_any(skb_dequeue(mcast-pkt_queue));
+ }
+
+ spin_lock_irqsave(priv-tx_lock, flags);
+ priv-stats.tx_dropped += tx_dropped;
+ spin_unlock_irqrestore(priv-tx_lock, flags);
kfree(mcast);
}
@@ -276,8 +283,10 @@ static int ipoib_mcast_join_finish(struc
}
/* actually send any queued packets */
+ spin_lock_irq(priv-tx_lock);
while (!skb_queue_empty(mcast-pkt_queue)) {
struct sk_buff *skb = skb_dequeue(mcast-pkt_queue);
+
Re: [openib-general] respect CFLAGS in OSM
[EMAIL PROTECTED] wrote on Mon, 23 Jan 2006 15:27 +0200: I have looked again at this patch and what it is changing. My understanding is that you found the -g -O2 CLFAGS (provided through the specific target CFLAGS) unneeded. You also think they will interfere with settings you might want to provide from the command line. I have just double checked what I new to be the rule for autoconf: If the user provides CFLAGS or LDFLAGS from the command like - they are appended to the compile or link flags. The impact on gcc is that the later settings - i.e. those provided by the user take precedence over the flags provided at the beginning of the command line. So the patch below is actually not needed. Just to convince you I attach some gcc traces showing that -O0 -O2 acts like -O2 and -O2 -O0 acts like -O0. Bottom line I would like to keep the code as it is without any change such that default installation will use the -O2 mode. Yes, later -O options do override previous ones. I didn't think of explicitly disabling optimization with -O0 in my build script. The implication is that you want me to do the following to compile a debugging version of osm: CFLAGS=-g -O0 ./configure ... rather than what I expected to work: CFLAGS=-g ./configure ... I can deal with that, and it's not a big enough concern to me now that you've pointed out this work-around. Feel free to ignore my suggestion, then: it's your code. My surprise comes from having become accustomed to autoconf-based programs that always use the user-specified CFLAGS exactly if given. AC_PROG_CC sets it to -g -O2 by default for gcc only if no CFLAGS was set by the user. If I don't want -g or don't want -O2, usually it is easy to make that happen without editing files. -- Pete ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH] uar size != 8M
There are some cards around that have UAR size different from 8M.
As a sanity check, compare it against the device reported limit instead.
Signed-off-by: Michael S. Tsirkin [EMAIL PROTECTED]
Index: linux-2.6.15/drivers/infiniband/hw/mthca/mthca_main.c
===
--- linux-2.6.15.orig/drivers/infiniband/hw/mthca/mthca_main.c 2006-01-12
12:05:35.0 +0200
+++ linux-2.6.15/drivers/infiniband/hw/mthca/mthca_main.c 2006-01-23
21:06:47.0 +0200
@@ -155,6 +155,13 @@ static int __devinit mthca_dev_lim(struc
return -ENODEV;
}
+ if (dev_lim-uar_size pci_resource_len(mdev-pdev, 2)) {
+ mthca_err(mdev, HCA reported UAR size of 0x%x bigger than
+ PCI resource 2 size of 0x%lx, aborting.\n,
+ dev_lim-uar_size, pci_resource_len(mdev-pdev, 2));
+ return -ENODEV;
+ }
+
mdev-limits.num_ports = dev_lim-num_ports;
mdev-limits.vl_cap = dev_lim-max_vl;
mdev-limits.mtu_cap= dev_lim-max_mtu;
@@ -976,8 +983,7 @@ static int __devinit mthca_init_one(stru
err = -ENODEV;
goto err_disable_pdev;
}
- if (!(pci_resource_flags(pdev, 2) IORESOURCE_MEM) ||
- pci_resource_len(pdev, 2) != 1 23) {
+ if (!(pci_resource_flags(pdev, 2) IORESOURCE_MEM)) {
dev_err(pdev-dev, Missing UAR, aborting.\n);
err = -ENODEV;
goto err_disable_pdev;
--
MST
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] Re: Repost [PATCH 1 of 3] move destructor to struct neigh_parms
Quoting r. Michael S. Tsirkin [EMAIL PROTECTED]: Move destructor from neigh_ops (which is shared between devices) to neigh_parms which is not, so that multiple drivers can set it safely. I posted this on netdev and lkml on Wednesday, but there was no response. I guess this patch is non-controverisal since no one else in kernel uses the destructor. Do you think we can push this into mainline? -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] Re: Userspace testing results (many kernels, many svn trees)
On 23.01.2006 [15:19:59 +0200], Michael S. Tsirkin wrote: Quoting r. Nishanth Aravamudan [EMAIL PROTECTED]: Subject: Re: Userspace testing results (many kernels,many svn trees) On 09.01.2006 [08:33:01 +0200], Michael S. Tsirkin wrote: Quoting r. Nishanth Aravamudan [EMAIL PROTECTED]: Just like with the results I posted earlier, all the perftest results are seriously wrong for 32-bit clients (with both 32-bit and 64-bit servers). I am not sure who else to notify beyond the general list (is there a corresponding MAINTAINERS files like in the kernel proper for the OpenIB code?) That would be me - sorry about the delay, I'll take a look at this. Thanks a lot, Nishanth! This work is very much appreciated. No worries, hope the problem is not too hard to fix :) OK, I'm going to concentrate on rdma_lat/rdma_bw for now. # file ./rdma_bw ./rdma_bw: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for GNU/Linux 2.2.5, dynamically linked (uses shared libs), not stripped # ./rdma_bw swlab155 local address: LID 0xd9, QPN 0x24040a, PSN 0x98e7f4 RKey 0xe0003f VAddr 0x00556db000 remote address: LID 0xd9, QPN 0x240406, PSN 0x952ea0, RKey 0xe00033 VAddr 0x00556db000 Bandwidth peak (#0 to #999): 879.956 MB/sec Bandwidth average: 879.954 MB/sec Service Demand peak (#0 to #999): 3773 cycles/KB Service Demand Avg : 37 cycles/KB Seems like I cant reproduce the problem. Which distribution and CPU architecture is this, again? The machines are running ppc64 kernels. They are both SLES9-SP2 userspace setups, with the infiniband components being updated to the current svn. Bah, I just noticed that perftests doesn't even build right now (svn 5130). on ppc32 machines: /usr/local/autobench/var/tmp/gen2-trunk/userspace/perftest patching file Makefile gcc -m32 -m32 -I/usr/local/autobench/var/tmp/out/ppc32/include -Wall -O2 -g -D_GNU_SOURCE -m32 -L/lib -L/usr/local/autobench/var/tmp/out/ppc32/lib rdma_lat.c get_clock.c -libverbs -o rdma_lat In file included from rdma_lat.c:57: get_clock.h:50:27: operator '||' has no right operand get_clock.h:73:2: warning: #warning get_cycles not implemented for this architecture: attempt asm/timex.h rdma_lat.c:517: error: parse error before get_median rdma_lat.c:517: error: parse error before cycles_t rdma_lat.c:518: warning: return type defaults to `int' rdma_lat.c: In function `get_median': rdma_lat.c:519: error: `n' undeclared (first use in this function) rdma_lat.c:519: error: (Each undeclared identifier is reported only once rdma_lat.c:519: error: for each function it appears in.) rdma_lat.c:520: error: `delta' undeclared (first use in this function) rdma_lat.c: In function `cycles_compare': rdma_lat.c:527: error: syntax error before '*' token rdma_lat.c:528: error: syntax error before '*' token rdma_lat.c:529: error: `a' undeclared (first use in this function) rdma_lat.c:529: error: `b' undeclared (first use in this function) rdma_lat.c: At top level: rdma_lat.c:535: error: parse error before cycles_t rdma_lat.c: In function `print_report': rdma_lat.c:538: error: `cycles_t' undeclared (first use in this function) rdma_lat.c:538: error: parse error before median rdma_lat.c:541: error: `delta' undeclared (first use in this function) rdma_lat.c:541: error: `iters' undeclared (first use in this function) rdma_lat.c:549: error: `tstamp' undeclared (first use in this function) rdma_lat.c:552: error: `options' undeclared (first use in this function) rdma_lat.c:574: error: `median' undeclared (first use in this function) rdma_lat.c: In function `main': rdma_lat.c:605: error: `cycles_t' undeclared (first use in this function) rdma_lat.c:605: error: `tstamp' undeclared (first use in this function) rdma_lat.c:747: warning: implicit declaration of function `get_cycles' make: *** [rdma_lat] Error 1 on ppc64 machines: gcc -m64 -m64 -I/usr/local/autobench/var/tmp/out/ppc64/include -Wall -O2 -g -D_GNU_SOURCE -m64 -L/lib64 -L/usr/local/autobench/var/tmp/out/ppc64/lib rdma_lat.c get_clock.c -libverbs -o rdma_lat In file included from rdma_lat.c:57: get_clock.h:50:27: operator '||' has no right operand get_clock.h:73:2: warning: #warning get_cycles not implemented for this architecture: attempt asm/timex.h make: *** [rdma_lat] Error 1 Once we get that sorted out, I can re-verify that perftest is/isn't broken still. Thanks, Nish ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Re: ipoib_mcast_send.patch
Quoting r. Roland Dreier [EMAIL PROTECTED]:
Also should we count dropped packets here?
Right. And since it seems that we cant get by with just one bit,
here's the original patch again, with dropped packet counter fixed.
---
Fix the following race scenario:
Device is up.
Port event or set mcast list triggers ipoib_mcast_stop_thread,
this cancels the query and waits on mcast done completion.
Completion is called and done is set.
Meanwhile, ipoib_mcast_send arrives and starts a new query,
re-initializing done.
Further, there's an additional issue that I saw in testing:
ipoib_mcast_send may get called when priv-broadcast is NULL
(e.g. if the device was downed and then upped internally because
of a port event).
If this happends and the sendonly join request gets completed before
priv-broadcast is set, we get an oops
Do not send multicasts if mcast thread is stopped or if
priv-broadcast is not set.
Signed-off-by: Michael S. Tsirkin [EMAIL PROTECTED]
Index: linux-2.6.15/drivers/infiniband/ulp/ipoib/ipoib_multicast.c
===
--- linux-2.6.15.orig/drivers/infiniband/ulp/ipoib/ipoib_multicast.c
2006-01-23 21:24:10.0 +0200
+++ linux-2.6.15/drivers/infiniband/ulp/ipoib/ipoib_multicast.c 2006-01-23
21:25:19.0 +0200
@@ -600,6 +600,10 @@ int ipoib_mcast_start_thread(struct net_
queue_work(ipoib_workqueue, priv-mcast_task);
mutex_unlock(mcast_mutex);
+ spin_lock_irq(priv-lock);
+ set_bit(IPOIB_MCAST_STARTED, priv-flags);
+ spin_unlock_irq(priv-lock);
+
return 0;
}
@@ -610,6 +614,10 @@ int ipoib_mcast_stop_thread(struct net_d
ipoib_dbg_mcast(priv, stopping multicast thread\n);
+ spin_lock_irq(priv-lock);
+ clear_bit(IPOIB_MCAST_STARTED, priv-flags);
+ spin_unlock_irq(priv-lock);
+
mutex_lock(mcast_mutex);
clear_bit(IPOIB_MCAST_RUN, priv-flags);
cancel_delayed_work(priv-mcast_task);
@@ -692,6 +700,12 @@ void ipoib_mcast_send(struct net_device
*/
spin_lock(priv-lock);
+ if (!test_bit(IPOIB_MCAST_STARTED, priv-flags) || !priv-broadcast) {
+ ++priv-stats.tx_dropped;
+ dev_kfree_skb_any(skb);
+ goto unlock;
+ }
+
mcast = __ipoib_mcast_find(dev, mgid);
if (!mcast) {
/* Let's create a new send only group now */
@@ -753,6 +767,7 @@ out:
ipoib_send(dev, skb, mcast-ah, IB_MULTICAST_QPN);
}
+unlock:
spin_unlock(priv-lock);
}
Index: linux-2.6.15/drivers/infiniband/ulp/ipoib/ipoib.h
===
--- linux-2.6.15.orig/drivers/infiniband/ulp/ipoib/ipoib.h 2006-01-23
21:24:10.0 +0200
+++ linux-2.6.15/drivers/infiniband/ulp/ipoib/ipoib.h 2006-01-23
21:24:46.0 +0200
@@ -85,6 +85,7 @@ enum {
IPOIB_FLAG_SUBINTERFACE = 4,
IPOIB_MCAST_RUN = 5,
IPOIB_STOP_REAPER = 6,
+ IPOIB_MCAST_STARTED = 7,
IPOIB_MAX_BACKOFF_SECONDS = 16,
--
MST
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Re: Userspace testing results (many kernels, many svn trees)
Quoting r. Nishanth Aravamudan [EMAIL PROTECTED]: Bah, I just noticed that perftests doesn't even build right now (svn 5130). Fixed, thanks. -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Re: Userspace testing results (many kernels, many svn trees)
On 23.01.2006 [18:46:42 +0200], Michael S. Tsirkin wrote: Quoting r. Nishanth Aravamudan [EMAIL PROTECTED]: Bah, I just noticed that perftests doesn't even build right now (svn 5130). Fixed, thanks. Great! What svn revision is it fixed in? 5159 is running the tests now, but I can cancel and resubmit if it was fixed after that. Thanks, Nish ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] Re: Userspace testing results (many kernels, many svn trees)
Quoting Nishanth Aravamudan [EMAIL PROTECTED]: The machines are running ppc64 kernels. BTW, does this configuration (ppc32 on ppc64) support the mftb instruction? -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] Re: Re: Userspace testing results (many kernels, many svn trees)
Quoting r. Nishanth Aravamudan [EMAIL PROTECTED]: Subject: Re: Re: Userspace testing results (many kernels,many svn trees) On 23.01.2006 [18:46:42 +0200], Michael S. Tsirkin wrote: Quoting r. Nishanth Aravamudan [EMAIL PROTECTED]: Bah, I just noticed that perftests doesn't even build right now (svn 5130). Fixed, thanks. Great! What svn revision is it fixed in? 5159 is running the tests now, but I can cancel and resubmit if it was fixed after that. Thanks, Nish This one: r5162 | mst | 2006-01-23 18:51:34 +0200 (Mon, 23 Jan 2006) | 3 lines typo fix -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] Re: Userspace testing results (many kernels, many svn trees)
On 23.01.2006 [18:53:19 +0200], Michael S. Tsirkin wrote: Quoting Nishanth Aravamudan [EMAIL PROTECTED]: The machines are running ppc64 kernels. BTW, does this configuration (ppc32 on ppc64) support the mftb instruction? Good question :) How would I go about checking? Thanks, Nish ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] Re: Re: Userspace testing results (many kernels, many svn trees)
On 23.01.2006 [18:53:58 +0200], Michael S. Tsirkin wrote: Quoting r. Nishanth Aravamudan [EMAIL PROTECTED]: Subject: Re: Re: Userspace testing results (many kernels,many svn trees) On 23.01.2006 [18:46:42 +0200], Michael S. Tsirkin wrote: Quoting r. Nishanth Aravamudan [EMAIL PROTECTED]: Bah, I just noticed that perftests doesn't even build right now (svn 5130). Fixed, thanks. Great! What svn revision is it fixed in? 5159 is running the tests now, but I can cancel and resubmit if it was fixed after that. Thanks, Nish This one: r5162 | mst | 2006-01-23 18:51:34 +0200 (Mon, 23 Jan 2006) | 3 lines typo fix Ok, running the tests again with 5162 -- will take some time to get to a set of sizes (32/64) that we are interested in, but will post again once they finish. Thanks, Nish ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] Re: Repost [PATCH 1 of 3] move destructor to struct neigh_parms
Michael I posted this on netdev and lkml on Wednesday, but there Michael was no response. I guess this patch is non-controverisal Michael since no one else in kernel uses the destructor. Michael Do you think we can push this into mainline? Let me resend to Dave Miller directly and then it should be OK. - R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Re: Userspace testing results (many kernels, many svn trees)
Michael BTW, does this configuration (ppc32 on ppc64) support the Michael mftb instruction? I have some ppc64 machines around -- let me do some perftest checking. - R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] Re: Userspace testing results (many kernels, many svn trees)
Quoting r. Nishanth Aravamudan [EMAIL PROTECTED]: Subject: Re: Userspace testing results (many kernels,many svn trees) On 23.01.2006 [18:53:19 +0200], Michael S. Tsirkin wrote: Quoting Nishanth Aravamudan [EMAIL PROTECTED]: The machines are running ppc64 kernels. BTW, does this configuration (ppc32 on ppc64) support the mftb instruction? Good question :) How would I go about checking? Thanks, Nish This seems to imply we are ok http://www-128.ibm.com/developerworks/eserver/articles/archguide.html Apple used to have some good ppc documentation but I couldnt locate it at the moment. -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [ANNOUNCE] libibverbs 1.0-rc5 released
I just tagged a 1.0-rc5 release of libibverbs and pushed it out to the relevant channels, which means that it should appear on http://openib.org/downloads/ shortly. Once the Fedora project build system is fixed, I will also kick off builds for Fedora Extras, so binary packages for Fedora 4 and 5 will appear on standard Fedora mirrors in a few days Changes since 1.0-rc4 include: - Change from dlist-based ibv_get_devices() API to simpler array-based ibv_get_device_list() API. This breaks source compatibility with 1.0-rc4. - Add Sean Hefty's user/kernel marshalling functions. - Lots of fixes for pingpong examples. See the ChangeLog in the package for full details. I think we're getting close to a full 1.0 release with a frozen API and ABI. My plans for getting there are the following: - As of now, the only changes to the API that I will accept are pure additions that do not break source compatibility with existing consumer or provider code. Any exceptions to this will have to be extremely well justified. - In about 3-4 weeks, I'll release 1.0-rc6. This release will add support for resizing CQs and (if the code appears in time) query QP and query SRQ support from Dotan Barak at Mellanox. - Once 1.0-rc6 is out, I'll consider the API and ABI absolutely frozen. In other words, any binary applications or device provider libraries built against 1.0-rc6 will continue to work aganst any later release in the 1.0 series. This means that any changes that affect source or binary compatibility with existing consumer or provider code will have to have end-of-the-world type consequences for me to accept them. - Bug fixes are of course always welcome at any point in the development cycle. - Depending on how testing feedback looks, 3-4 weeks after 1.0-rc6, I'll release either a full 1.0, or if we're unlucky, 1.0-rc7. - Once 1.0 is out, I'll continue to accept changes that don't affect compatibility, and continue to do 1.0.1, 1.0.2, etc. releases as necessary. - When API or ABI breaking changes become necessary, I'll kick off a 1.1 release series and we can go nuts again. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] Re: Repost [PATCH 1 of 3] move destructor tostruct neigh_parms
Quoting r. Roland Dreier [EMAIL PROTECTED]: Subject: Re: Repost [PATCH 1 of 3] move destructor tostruct neigh_parms Michael I posted this on netdev and lkml on Wednesday, but there Michael was no response. I guess this patch is non-controverisal Michael since no one else in kernel uses the destructor. Michael Do you think we can push this into mainline? Let me resend to Dave Miller directly and then it should be OK. - R. Ok. I guess we'll want a trunk patch with an ifdef, to use until 2.6.16 is released? -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] ANNOUNCE: mstflint update
Hi! I have updated mstflint tool with code from mellanox MFT 1.0.1 package. mstflint is a stand-alone firmware burning tool for Mellanox manufactured HCA cards. Some success has been reported with cards from Topspin/Cisco. See the README file under src/userspace/mstflint for more info. Changes: * bug fixes * more flash types supported * new flag -skip_is to allow safe firmware updates even if new firmware includes initial sector update (by updating all firmware except the IS) * portability cleanups: use stdio instead of low level file descriptor operations This code has been tested on x86 and x86_64, on PCI-X and PCI-Express cards. I'd appreciate feedback and testing on other platforms. You can help by testing this tool even if you dont have IB cards, or if you dont want to burn firmware, as described below. Thanks, MST --- To build: cd src/userspace/mstflint make --- Ways to test the tool (without accessing device): # ./mstflint -i ~/fw-25218-5_1_0-rc22-lion-cub.bin q Image type: Failsafe I.S. Version:1 Chip Revision: A0 GUID Des:Node Port1Port2Sys image GUIDs: 0002c9000100d050 0002c9000100d051 0002c9000100d052 0002c9000100d050 Board ID: (MT_014001) VSD: PSID:MT_014001 # ./mstflint -i ~/fw-25218-5_1_0-rc22-lion-cub.bin v Failsafe image: Invariant /0x0028-0x095f (0x000938)/ (BOOT2) - OK Primary Image /0x0001-0x00010107 (0x000108)/ (Pointer Sector)- OK /0x00030028-0x000308af (0x000888)/ (BOOT2) - OK /0x000308b0-0x00035ae7 (0x005238)/ (BOOT2) - OK /0x00035ae8-0x000380cf (0x0025e8)/ (Configuration) - OK /0x000380d0-0x00038103 (0x34)/ (GUID) - OK /0x00038104-0x00046b87 (0x00ea84)/ (DDR) - OK /0x00046b88-0x0004fb1b (0x008f94)/ (DDR) - OK /0x0004fb1c-0x00067ca3 (0x018188)/ (DDR) - OK /0x00067ca4-0x0007fc53 (0x017fb0)/ (DDR) - OK /0x0007fc54-0x0008245b (0x002808)/ (DDR) - OK /0x0008245c-0x000839c3 (0x001568)/ (DDR) - OK /0x000839c4-0x000839d7 (0x14)/ (Configuration) - OK /0x000839d8-0x00083a1b (0x44)/ (Jump addresses) - OK /0x00083a1c-0x0008e1cb (0x00a7b0)/ (EMT Service) - OK Secondary Image /0x0002-0x00020107 (0x000108)/ (Pointer Sector)- OK /0x00090028-0x000908af (0x000888)/ (BOOT2) - OK /0x000908b0-0x00095ae7 (0x005238)/ (BOOT2) - OK /0x00095ae8-0x000980cf (0x0025e8)/ (Configuration) - OK /0x000980d0-0x00098103 (0x34)/ (GUID) - OK /0x00098104-0x000a6b87 (0x00ea84)/ (DDR) - OK /0x000a6b88-0x000afb1b (0x008f94)/ (DDR) - OK /0x000afb1c-0x000c7ca3 (0x018188)/ (DDR) - OK /0x000c7ca4-0x000dfc53 (0x017fb0)/ (DDR) - OK /0x000dfc54-0x000e245b (0x002808)/ (DDR) - OK /0x000e245c-0x000e39c3 (0x001568)/ (DDR) - OK /0x000e39c4-0x000e39d7 (0x14)/ (Configuration) - OK /0x000e39d8-0x000e3a1b (0x44)/ (Jump addresses) - OK /0x000e3a1c-0x000ee1cb (0x00a7b0)/ (EMT Service) - OK FW Image verification succeeded. Image is OK. More ways to test the tool (read flash but do not burn anything to the device): # ./mstflint -d /sys/class/infiniband/mthca0/device/resource0 q Image type: Failsafe I.S. Version:1 Chip Revision: A0 GUID Des:Node Port1Port2Sys image GUIDs: 0a66b492c901 0a66b4910002c901 0a66b4920002c901 0a66b4930002c901 Board ID: (MT_00A001) VSD: PSID:MT_00A001 # ./mstflint -d /sys/class/infiniband/mthca0/device/resource0 v Failsafe image: Invariant /0x0028-0x095f (0x000938)/ (BOOT2) - OK Primary Image /0x0001-0x00010107 (0x000108)/ (Pointer Sector)- OK /0x00030028-0x000308af (0x000888)/ (BOOT2) - OK /0x000308b0-0x00034baf (0x004300)/ (BOOT2) - OK /0x00034bb0-0x00035a93 (0x000ee4)/ (Configuration) - OK /0x00035a94-0x00035ac7 (0x34)/ (GUID) - OK /0x00035ac8-0x0003e2bb (0x0087f4)/ (DDR) - OK /0x0003e2bc-0x0004c2e3 (0x00e028)/ (DDR) - OK /0x0004c2e4-0x0004f213 (0x002f30)/ (DDR) - OK /0x0004f214-0x00050907 (0x0016f4)/ (DDR) - OK /0x00050908-0x000693e7 (0x018ae0)/ (DDR) - OK /0x000693e8-0x0007d307 (0x013f20)/ (DDR) - OK /0x0007d308-0x0007d31b (0x14)/ (Configuration) - OK /0x0007d31c-0x0007d35f (0x44)/ (Jump addresses) - OK
Re: [openib-general] Re: Userspace testing results (many kernels, many svn trees)
PPC Architecture defines a 64-bit Time Base register (2-32bit regs in 32 bit mode). Below link is the documentation. http://www-128.ibm.com/developerworks/eserver/articles/archguide.html Book II: PowerPC Virtual Environment Architecture Page 30 Thanks Shirley Ma IBM Linux Technology Center 15300 SW Koll Parkway Beaverton, OR 97006-6063 Phone(Fax): (503) 578-7638___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Re: Userspace testing results (many kernels, many svn trees)
Quoting r. Shirley Ma [EMAIL PROTECTED]: Subject: Re: [openib-general] Re: Userspace testing results (many kernels,?many svn trees) PPC Architecture defines a 64-bit Time Base register (2-32bit regs in 32 bit mode). Below link is the documentation. http://www-128.ibm.com/developerworks/eserver/articles/archguide.html Book II: PowerPC Virtual Environment Architecture Page 30 Thanks Shirley Ma IBM Linux Technology Center So am I right when I say that mftb returns a 32 bit register on ppc32 and a 64 bit register on ppc64? If true, it seems that this line typedef unsigned long long cycles_t; should be replaced by typedef unsigned long cycles_t; -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Re: Repost [PATCH 1 of 3] move destructor to struct neigh_parms
You can resend the patch to Dave Miller (network maintainer) and ask him for inputs directly. Thanks Shirley Ma IBM Linux Technology Center 15300 SW Koll Parkway Beaverton, OR 97006-6063 Phone(Fax): (503) 578-7638___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Re: Userspace testing results (many kernels, many svn trees)
If true, it seems that this line typedef unsigned long long cycles_t; should be replaced by typedef unsigned long cycles_t; Yes. Thanks Shirley Ma IBM Linux Technology Center 15300 SW Koll Parkway Beaverton, OR 97006-6063 Phone(Fax): (503) 578-7638 ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] Re: Re: Userspace testing results (many kernels, many svn trees)
Quoting r. Shirley Ma [EMAIL PROTECTED]: Subject: Re: Re: Userspace testing results (many kernels,?many svn trees) If true, it seems that this line typedef unsigned long long cycles_t; should be replaced by typedef unsigned long cycles_t; Yes. OK, I did it this way. # svn ci get_clock.h Sendingget_clock.h Transmitting file data . Committed revision 5163. You might want to try this rev out. -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Re: Userspace testing results (many kernels, many svn trees)
I read the documentation again, I think I was wrong. It returns 64 bit always. Thanks Shirley Ma IBM Linux Technology Center 15300 SW Koll Parkway Beaverton, OR 97006-6063 Phone(Fax): (503) 578-7638___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Re: Userspace testing results (many kernels, many svn trees)
Quoting r. Shirley Ma [EMAIL PROTECTED]: Subject: Re: [openib-general] Re: Userspace testing results (many kernels,?many svn trees) I read the documentation again, I think I was wrong. It returns 64 bit always. Thanks Shirley Ma IBM Linux Technology Center 15300 SW Koll Parkway Beaverton, OR 97006-6063 Phone(Fax): (503) 578-7638 I reverted it, if this change is needed you know what to do. -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] Re: [PATCH] CMA and iWARP
[EMAIL PROTECTED] wrote: Sean I haven't seen anyone object to merging the other changes. Sean Roland, Hal - any opinion? I don't see much urgency in merging it now. When svn diverges from what's upstream in the kernel, it makes my life harder because I have to figure out which patches belong upstream and sometimes merge things by hand (when they hit the divergent regions). Also I can't say I'm thrilled by adding + struct iw_cm_verbs*iwcm; to struct ib_device -- we still really haven't answered the issue of how iWARP connections interact with the host network stack, we've just pushed it off into low-level driver code where we can't see it. My understanding is that this is a partial answer, not a deferred one. What is fully addressed is how the iWARP device accepts or initiates a connection that will start the transition to MPA mode immediately. In that mode, there is no integration with the host network TCP layer required -- only co-ordination with the host network IP layer. Linking to the net device does that. Deferred entry into MPA mode is indeed a complex issue, which is why Tom did not propose an immediate solution. The initial model is IB compatible, deals with the needs of virtually all RDMA applications, and is very simple. Locking down this first step, to enable transport neutral application development, is important. It should not have to wait for the entire stack integration problem to be solved. The latter is a complex issue that includes things like compliance with netfilter rules that need to apply to *all* connections with IP semantics, including SDP. That complexity will take some time to work out properly. Meanwhile we can establish a very workable baseline. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] Re: Re: Userspace testing results (many kernels, many svn trees)
On 23.01.2006 [19:55:05 +0200], Michael S. Tsirkin wrote: Quoting r. Shirley Ma [EMAIL PROTECTED]: Subject: Re: Re: Userspace testing results (many kernels,?many svn trees) If true, it seems that this line typedef unsigned long long cycles_t; should be replaced by typedef unsigned long cycles_t; Yes. OK, I did it this way. # svn ci get_clock.h Sendingget_clock.h Transmitting file data . Committed revision 5163. You might want to try this rev out. heh, ok. I'm going to let the 5162 version of a 32/32 setup finish, then run 5163. Thanks, Nish ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH] Add -m option to ping pong programs to set path MTU
Looks fine to me. On Fri, 2006-01-20 at 16:48 -0800, Roland Dreier wrote: Thanks, I applied my own version of this. Please make sure that the svn tree still works for you. (This patch was the one that pushed me over the edge with code duplication in the pingpong examples, so I put the MTU switch statement into a new pingpong.c file...) - R. -- Ralph Campbell [EMAIL PROTECTED] ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Re: [PATCH] CMA and iWARP
Steve Wise wrote: Is it possible to just go ahead and push the core iwarp stuff into kernel.org? I don't think that makes sense or would be accepted. This would be asking for changes to the kernel code that nothing else in the kernel would use. - Sean ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] Re: [PATCH] CMA and iWARP
[EMAIL PROTECTED] wrote: Steve Wise wrote: Is it possible to just go ahead and push the core iwarp stuff into kernel.org? I don't think that makes sense or would be accepted. This would be asking for changes to the kernel code that nothing else in the kernel would use. But there are solid precedents for creating an extensible/flexible interface even if there is only a single user of that interface currently. For example the scsi_transport_iscsi defines an very general interface to an iscsi transport layer even though iscsi_tcp is the only instantiation of that transport. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] Re: [PATCH] CMA and iWARP
On Mon, 2006-01-23 at 10:50 -0800, Caitlin Bestler wrote: [EMAIL PROTECTED] wrote: Steve Wise wrote: Is it possible to just go ahead and push the core iwarp stuff into kernel.org? I don't think that makes sense or would be accepted. This would be asking for changes to the kernel code that nothing else in the kernel would use. But there are solid precedents for creating an extensible/flexible interface even if there is only a single user of that interface currently. For example the scsi_transport_iscsi defines an very general interface to an iscsi transport layer even though iscsi_tcp is the only instantiation of that transport. And the name change for the include directory from include/infiniband to include/rmda was also a precursor to full RDMA support for more than just IB. But I'm no expert on what the criteria is for pushing openib change sets into kernel.org, so I can only voice my opinion (and ask lots of questions :)... Roland, can you help clarify this? Also, what _is_ the criteria in your mind for pulling iwarp support into: 1) openib trunk, and 2) kernel.org? Thanks, Steve. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Re: Userspace testing results (many kernels, many svn trees)
Shirley I read the documentation again, I think I was wrong. It Shirley returns 64 bit always. I don't think that's correct. How could mftb return a 64-bit value on a CPU with 32-bit registers? The best thing to look at is the definition of get_tb() in asm-powerpc/time.h in the kernel tree. I think that should give something that always works. - R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Re: [PATCH] CMA and iWARP
Steve Roland, can you help clarify this? Steve Also, what _is_ the criteria in your mind for pulling iwarp Steve support into: 1) openib trunk, and 2) kernel.org? It's a judgement call that involves everyone. One thing that would really help me feel more comfortable would be seeing a driver for some other piece of hardware (NetEffect?). I don't want to push a design based on one piece of hardware and then find out that all the new hardware wants something else. Maybe hearing Chelsio's opinion would be helpful as well, as they have been dealing with similar issues from a different direction. - R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Re: [PATCH] CMA and iWARP
Caitlin But there are solid precedents for creating an Caitlin extensible/flexible interface even if there is only a Caitlin single user of that interface currently. For example the Caitlin scsi_transport_iscsi defines an very general interface to Caitlin an iscsi transport layer even though iscsi_tcp is the Caitlin only instantiation of that transport. Sure, but we're talking about creating an interface with zero users. And even when there is one user, we want to avoid premature and incorrect abstraction. - R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Re: [PATCH] CMA and iWARP
Caitlin What is fully addressed is how the iWARP device accepts Caitlin or initiates a connection that will start the transition Caitlin to MPA mode immediately. Caitlin In that mode, there is no integration with the host Caitlin network TCP layer required -- only co-ordination with the Caitlin host network IP layer. Linking to the net device does Caitlin that. I disagree. The proposed interface says that accepting a connection is completely the responsibility of the low-level driver, with no hooks for central policy, bypassing packet filtering, etc, etc. That's possibly a legitimate decision but it is a pretty major decision as well. - R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] Re: [PATCH] CMA and iWARP
Tom I agree there are more elegant approaches, however, the Tom design criteria was to minimize changes to ib_verbs and the Tom risk of IB functional regression. I think this approach Tom accomplishes that goal. What would the more elegant approach be? I don't think minimizing changes is really the dimension to optimize on. The luxury of Linux development is that we can choose the best solution, even if it means breaking the world (although of course the costs of churn in terms of risk and effort do need to be weighed). - R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Re: [PATCH] CMA and iWARP
On Mon, 2006-01-23 at 11:07 -0800, Roland Dreier wrote: Steve Roland, can you help clarify this? Steve Also, what _is_ the criteria in your mind for pulling iwarp Steve support into: 1) openib trunk, and 2) kernel.org? It's a judgement call that involves everyone. One thing that would really help me feel more comfortable would be seeing a driver for some other piece of hardware (NetEffect?). I don't want to push a design based on one piece of hardware and then find out that all the new hardware wants something else. Maybe hearing Chelsio's opinion would be helpful as well, as they have been dealing with similar issues from a different direction. I think these are very good points. I'll see if I can't get some other folks to weigh in on this. - R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] Re: [PATCH] CMA and iWARP
On Mon, 2006-01-23 at 11:11 -0800, Roland Dreier wrote: Tom I agree there are more elegant approaches, however, the Tom design criteria was to minimize changes to ib_verbs and the Tom risk of IB functional regression. I think this approach Tom accomplishes that goal. What would the more elegant approach be? Phase III I don't think minimizing changes is really the dimension to optimize on. The luxury of Linux development is that we can choose the best solution, even if it means breaking the world (although of course the costs of churn in terms of risk and effort do need to be weighed). The discussions from back at IDF advocated a phased approach. From my recollection: Phase I - iWARP device driver that mapped RNIC events and DTO to IB events and DTOs. Very small change required to core in the form of a new node type. [done] Phase II - Transport independent connection management. This milestone was to begin merge with trunk since it required more significant core changes. [done] Phase III - Transport neutral naming, pluggable transports, etc... Sonoma is a great place to dig into these discussions. Phases I and II are complete in the branch, were demonstrated at SC'05, and have now been submitted as a patch to the trunk. - R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] Re: [PATCH] CMA and iWARP
Tom Tucker wrote: Phases I and II are complete in the branch, were demonstrated at SC'05, and have now been submitted as a patch to the trunk. To add to this, the patch makes fairly minor changes (a couple to only a few lines) to ib_verbs, ib_mad, ib_addr, and ib_cm. Some work was done to ib_addr and rdma_cm to simplify iWarp support. The largest changes in the patch were adding iWarp specific files, and modifications to the rdma_cm. I wanted to defer merging the iWarp changes into the rdma_cm until we could get feedback from the kernel developers first, which we've now had the chance to receive. - Sean ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] Re: Re: Userspace testing results (many kernels, many svn trees)
Quoting r. Nishanth Aravamudan [EMAIL PROTECTED]: Subject: Re: Re: Userspace testing results (many kernels,?many svn trees) On 23.01.2006 [19:55:05 +0200], Michael S. Tsirkin wrote: Quoting r. Shirley Ma [EMAIL PROTECTED]: Subject: Re: Re: Userspace testing results (many kernels,?many svn trees) If true, it seems that this line typedef unsigned long long cycles_t; should be replaced by typedef unsigned long cycles_t; Yes. OK, I did it this way. # svn ci get_clock.h Sendingget_clock.h Transmitting file data . Committed revision 5163. You might want to try this rev out. heh, ok. I'm going to let the 5162 version of a 32/32 setup finish, then run 5163. Well, Shirley Ma here said its a mistake, so I reverted the change - no need to re-run it. -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] Re: Re: Userspace testing results (many kernels, many svn trees)
On 23.01.2006 [10:24:12 -0800], Nishanth Aravamudan wrote: On 23.01.2006 [19:55:05 +0200], Michael S. Tsirkin wrote: Quoting r. Shirley Ma [EMAIL PROTECTED]: Subject: Re: Re: Userspace testing results (many kernels,?many svn trees) If true, it seems that this line typedef unsigned long long cycles_t; should be replaced by typedef unsigned long cycles_t; Yes. OK, I did it this way. # svn ci get_clock.h Sendingget_clock.h Transmitting file data . Committed revision 5163. You might want to try this rev out. heh, ok. I'm going to let the 5162 version of a 32/32 setup finish, then run 5163. Looks like 5162/5163 is fine building wise. Here is what I got for rdma_lat for a 32-bit server and 32-bit client: loading libehca local address: LID 0x0d QPN 0x140406 PSN 0x253f3e RKey 0x2340032 VAddr 0x0010019001 remote address: LID 0x08 QPN 0x140406 PSN 0xa79d77 RKey 0x2340032 VAddr 0x0010019001 Latency typical: 3.25746e+09 usec Latency best : 3.19975e+09 usec Latency worst : 4.21767e+10 usec and for rdma_bw: loading libehca local address: LID 0x0d, QPN 0x150406, PSN 0x1b3ee5 RKey 0x23a0032 VAddr 0x00f7fce000 remote address: LID 0x08, QPN 0x150406, PSN 0x2fa0a9, RKey 0x23a0032 VAddr 0x00f7fb8000 Bandwidth peak (#0 to #999): 4.3446e-07 MB/sec Bandwidth average: 4.3446e-07 MB/sec Service Demand peak (#0 to #999): 17301 cycles/KB Service Demand Avg : 0 cycles/KB So it's still present... Thanks, Nish ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] Re: Re: Userspace testing results (many kernels, many svn trees)
Quoting r. Nishanth Aravamudan [EMAIL PROTECTED]: Subject: Re: Re: Userspace testing results (many kernels,many svn trees) On 23.01.2006 [10:24:12 -0800], Nishanth Aravamudan wrote: On 23.01.2006 [19:55:05 +0200], Michael S. Tsirkin wrote: Quoting r. Shirley Ma [EMAIL PROTECTED]: Subject: Re: Re: Userspace testing results (many kernels,?many svn trees) If true, it seems that this line typedef unsigned long long cycles_t; should be replaced by typedef unsigned long cycles_t; Yes. OK, I did it this way. # svn ci get_clock.h Sendingget_clock.h Transmitting file data . Committed revision 5163. You might want to try this rev out. heh, ok. I'm going to let the 5162 version of a 32/32 setup finish, then run 5163. Looks like 5162/5163 is fine building wise. Here is what I got for rdma_lat for a 32-bit server and 32-bit client: loading libehca local address: LID 0x0d QPN 0x140406 PSN 0x253f3e RKey 0x2340032 VAddr 0x0010019001 remote address: LID 0x08 QPN 0x140406 PSN 0xa79d77 RKey 0x2340032 VAddr 0x0010019001 Latency typical: 3.25746e+09 usec Latency best : 3.19975e+09 usec Latency worst : 4.21767e+10 usec and for rdma_bw: loading libehca local address: LID 0x0d, QPN 0x150406, PSN 0x1b3ee5 RKey 0x23a0032 VAddr 0x00f7fce000 remote address: LID 0x08, QPN 0x150406, PSN 0x2fa0a9, RKey 0x23a0032 VAddr 0x00f7fb8000 Bandwidth peak (#0 to #999): 4.3446e-07 MB/sec Bandwidth average: 4.3446e-07 MB/sec Service Demand peak (#0 to #999): 17301 cycles/KB Service Demand Avg : 0 cycles/KB So it's still present... Thanks, Nish Hmm. Could you please try running e.g. rdma_lat with -H to get all the results? -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] Re: Re: Userspace testing results (many kernels, many svn trees)
On 23.01.2006 [23:22:45 +0200], Michael S. Tsirkin wrote: Quoting r. Nishanth Aravamudan [EMAIL PROTECTED]: Subject: Re: Re: Userspace testing results (many kernels,many svn trees) On 23.01.2006 [10:24:12 -0800], Nishanth Aravamudan wrote: On 23.01.2006 [19:55:05 +0200], Michael S. Tsirkin wrote: Quoting r. Shirley Ma [EMAIL PROTECTED]: Subject: Re: Re: Userspace testing results (many kernels,?many svn trees) If true, it seems that this line typedef unsigned long long cycles_t; should be replaced by typedef unsigned long cycles_t; Yes. OK, I did it this way. # svn ci get_clock.h Sendingget_clock.h Transmitting file data . Committed revision 5163. You might want to try this rev out. heh, ok. I'm going to let the 5162 version of a 32/32 setup finish, then run 5163. Looks like 5162/5163 is fine building wise. Here is what I got for rdma_lat for a 32-bit server and 32-bit client: loading libehca local address: LID 0x0d QPN 0x140406 PSN 0x253f3e RKey 0x2340032 VAddr 0x0010019001 remote address: LID 0x08 QPN 0x140406 PSN 0xa79d77 RKey 0x2340032 VAddr 0x0010019001 Latency typical: 3.25746e+09 usec Latency best : 3.19975e+09 usec Latency worst : 4.21767e+10 usec and for rdma_bw: loading libehca local address: LID 0x0d, QPN 0x150406, PSN 0x1b3ee5 RKey 0x23a0032 VAddr 0x00f7fce000 remote address: LID 0x08, QPN 0x150406, PSN 0x2fa0a9, RKey 0x23a0032 VAddr 0x00f7fb8000 Bandwidth peak (#0 to #999): 4.3446e-07 MB/sec Bandwidth average: 4.3446e-07 MB/sec Service Demand peak (#0 to #999): 17301 cycles/KB Service Demand Avg : 0 cycles/KB So it's still present... Thanks, Nish Hmm. Could you please try running e.g. rdma_lat with -H to get all the results? Sure, let me modify the script I use to run the jobs and resubmit the job. Thanks, Nish ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH] net: Move destructor from neigh-ops to neigh_params
This is a resend of a patch written by Michael S. Tsirkin
[EMAIL PROTECTED]. I'd like to get an ACK or NAK of it from Dave
and other networking people, so that we can either merge it upstream
or try a different approach. There definitely is a problem with
neighbour destructors that IP-over-IB is running into.
It would be good to know what the design was behind putting the
destructor method in neigh-ops in the first place.
Dave, if you want to merge this directly, that's fine. Or I'm fine
with merging this through the IB tree if you'd prefer (if you want me
to do that, let me know if you think it's 2.6.16 material).
Thanks,
Roland
struct neigh_ops currently has a destructor field, which no in-kernel
drivers outside of infiniband use. The infiniband/ulp/ipoib in-tree
driver stashes some info in the neighbour structure (the results of
the second-stage lookup from ARP results to real link-level path), and
it uses neigh-ops-destructor to get a callback so it can clean up
this extra info when a neighbour is freed. We've run into problems
with this: since the destructor is in an ops field that is shared
between neighbours that may belong to different net devices, there's
no way to set/clear it safely.
The following patch moves this field to neigh_parms where it can be
safely set, together with its twin neigh_setup. Two additional
patches in the patch series update ipoib to use this new interface.
Signed-off-by: Michael S. Tsirkin [EMAIL PROTECTED]
Signed-off-by: Roland Dreier [EMAIL PROTECTED]
---
diff --git a/include/net/neighbour.h b/include/net/neighbour.h
index 6fa9ae1..b0666d6 100644
--- a/include/net/neighbour.h
+++ b/include/net/neighbour.h
@@ -68,6 +68,7 @@ struct neigh_parms
struct net_device *dev;
struct neigh_parms *next;
int (*neigh_setup)(struct neighbour *);
+ void(*neigh_destructor)(struct neighbour *);
struct neigh_table *tbl;
void*sysctl_table;
@@ -145,7 +146,6 @@ struct neighbour
struct neigh_ops
{
int family;
- void(*destructor)(struct neighbour *);
void(*solicit)(struct neighbour *, struct sk_buff*);
void(*error_report)(struct neighbour *, struct
sk_buff*);
int (*output)(struct sk_buff*);
diff --git a/net/core/neighbour.c b/net/core/neighbour.c
index e68700f..3489e23 100644
--- a/net/core/neighbour.c
+++ b/net/core/neighbour.c
@@ -586,8 +586,8 @@ void neigh_destroy(struct neighbour *nei
kfree(hh);
}
- if (neigh-ops neigh-ops-destructor)
- (neigh-ops-destructor)(neigh);
+ if (neigh-parms-neigh_destructor)
+ (neigh-parms-neigh_destructor)(neigh);
skb_queue_purge(neigh-arp_queue);
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c
b/drivers/infiniband/ulp/ipoib/ipoib_main.c
index fd3f5c8..9588124 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_main.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c
@@ -247,7 +247,6 @@ static void path_free(struct net_device
if (neigh-ah)
ipoib_put_ah(neigh-ah);
*to_ipoib_neigh(neigh-neighbour) = NULL;
- neigh-neighbour-ops-destructor = NULL;
kfree(neigh);
}
@@ -530,7 +529,6 @@ static void neigh_add_path(struct sk_buf
err:
*to_ipoib_neigh(skb-dst-neighbour) = NULL;
list_del(neigh-list);
- neigh-neighbour-ops-destructor = NULL;
kfree(neigh);
++priv-stats.tx_dropped;
@@ -769,21 +767,9 @@ static void ipoib_neigh_destructor(struc
ipoib_put_ah(ah);
}
-static int ipoib_neigh_setup(struct neighbour *neigh)
-{
- /*
-* Is this kosher? I can't find anybody in the kernel that
-* sets neigh-destructor, so we should be able to set it here
-* without trouble.
-*/
- neigh-ops-destructor = ipoib_neigh_destructor;
-
- return 0;
-}
-
static int ipoib_neigh_setup_dev(struct net_device *dev, struct neigh_parms
*parms)
{
- parms-neigh_setup = ipoib_neigh_setup;
+ parms-neigh_destructor = ipoib_neigh_destructor;
return 0;
}
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] Re: [PATCH] RFC: mthca handling of signals
This is a good catch, and I think this is the right solution. One question: does the mutex use in mthca_multicast_detach() suffer from the same problem? It seems that we might call to detach from a multicast group during process exit with a signal pending, and end up failing there when we shouldn't. - R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Re: [PATCH] CMA and iWARP
On Mon, Jan 23, 2006 at 11:10:15AM -0800, Roland Dreier wrote: I disagree. The proposed interface says that accepting a connection is completely the responsibility of the low-level driver, with no hooks for central policy, bypassing packet filtering, etc, etc. That's possibly a legitimate decision but it is a pretty major decision as well. Yes, but we need to start somewhere. Until someone submits a driver that does all the things you mention, it makes sense to move forward with what has been proposed to date. Isn't there a chance the proposed interface is the right approach anyway? Or does RDMA infrastructure need to directly meddle with NETFILTER? grant ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Re: [PATCH] RFC: mthca handling of signals
Quoting r. Roland Dreier [EMAIL PROTECTED]: Subject: [openib-general] Re: [PATCH] RFC: mthca handling of signals This is a good catch, and I think this is the right solution. One question: does the mutex use in mthca_multicast_detach() suffer from the same problem? It seems that we might call to detach from a multicast group during process exit with a signal pending, and end up failing there when we shouldn't. Looks like it's the same problem, although I didnt see it failing here. -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] Re: [PATCH] net: Move destructor from neigh-ops to neigh_params
From: Roland Dreier [EMAIL PROTECTED] Date: Mon, 23 Jan 2006 13:27:32 -0800 I'd like to get an ACK or NAK of it from Dave Dave is in New Zealand at linux.conf.au, don't expect him to be too active for at least a week... ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH] RFC: net: Neighbour, Route and Redirect Notifiers
Enclosed is a patch for notifying registered modules
of neighbour updates, route changes, and redirect events.
I'm submitting this first to this list for commentary since
I am presuming that OpenIB needs this functionality.
Something like this will be needed for iWARP support.
The issue is that for an IP network, the next hop to
the remote peer can change dynamically. This patch provides
a mechanism for the provider/core to be notified so that it can
change the next hop hardware mac in the RNIC. There are various
ways it can change, thus the different events. It could be
argued that route change events are not needed. I included it
here so that the argument can be had.
Also PathMTU change needs to be added. The code that does
this, however, is nasty and scattered about. I'll submit it
after this has been looked at if people agree that this is a
good general approach.
Thanks,
Signed-off-by: Tom Tucker [EMAIL PROTECTED]
diff -u -r -x '.*' --new-file linux-2.6.14.5/include/net/netevent.h
linux-2.6.14.5.tom/include/net/netevent.h
--- linux-2.6.14.5/include/net/netevent.h 1969-12-31 18:00:00.0
-0600
+++ linux-2.6.14.5.tom/include/net/netevent.h 2006-01-23 15:41:21.0
-0600
@@ -0,0 +1,40 @@
+#ifndef _NET_EVENT_H
+#define _NET_EVENT_H
+
+/*
+ * Generic netevent notifiers
+ *
+ * Authors:
+ * Tom Tucker [EMAIL PROTECTED]
+ *
+ * Changes:
+ */
+
+#ifdef __KERNEL__
+
+#include net/dst.h
+
+struct netevent_redirect {
+ struct dst_entry *old;
+ struct dst_entry *new;
+};
+
+struct netevent_route_change {
+int event;
+struct fib_info *fib_info;
+};
+
+enum netevent_notif_type {
+ NETEVENT_NEIGH_UPDATE = 1, /* arg is * struct neighbour */
+ NETEVENT_ROUTE_UPDATE, /* arg is * struct netevent_route_change */
+ NETEVENT_REDIRECT, /* arg is * struct netevent_redirect */
+};
+
+extern int register_netevent_notifier(struct notifier_block *nb);
+extern int unregister_netevent_notifier(struct notifier_block *nb);
+extern int call_netevent_notifiers(unsigned long val, void *v);
+
+#endif
+#endif
+
+
diff -u -r -x '.*' --new-file linux-2.6.14.5/net/core/Makefile
linux-2.6.14.5.tom/net/core/Makefile
--- linux-2.6.14.5/net/core/Makefile2005-12-26 18:26:33.0 -0600
+++ linux-2.6.14.5.tom/net/core/Makefile2006-01-16 13:41:42.0
-0600
@@ -7,7 +7,7 @@
obj-$(CONFIG_SYSCTL) += sysctl_net_core.o
-obj-y += dev.o ethtool.o dev_mcast.o dst.o \
+obj-y += dev.o ethtool.o dev_mcast.o dst.o netevent.o \
neighbour.o rtnetlink.o utils.o link_watch.o filter.o
obj-$(CONFIG_XFRM) += flow.o
diff -u -r -x '.*' --new-file linux-2.6.14.5/net/core/neighbour.c
linux-2.6.14.5.tom/net/core/neighbour.c
--- linux-2.6.14.5/net/core/neighbour.c 2005-12-26 18:26:33.0 -0600
+++ linux-2.6.14.5.tom/net/core/neighbour.c 2006-01-16 13:41:42.0
-0600
@@ -30,9 +30,11 @@
#include net/neighbour.h
#include net/dst.h
#include net/sock.h
+#include net/netevent.h
#include linux/rtnetlink.h
#include linux/random.h
#include linux/string.h
+#include linux/notifier.h
#define NEIGH_DEBUG 1
@@ -756,6 +758,7 @@
NEIGH_PRINTK2(neigh %p is suspected.\n, neigh);
neigh-nud_state = NUD_STALE;
neigh_suspect(neigh);
+ call_netevent_notifiers(NETEVENT_NEIGH_UPDATE, neigh);
}
} else if (state NUD_DELAY) {
if (time_before_eq(now,
@@ -763,6 +766,7 @@
NEIGH_PRINTK2(neigh %p is now reachable.\n, neigh);
neigh-nud_state = NUD_REACHABLE;
neigh_connect(neigh);
+ call_netevent_notifiers(NETEVENT_NEIGH_UPDATE, neigh);
next = neigh-confirmed + neigh-parms-reachable_time;
} else {
NEIGH_PRINTK2(neigh %p is probed.\n, neigh);
@@ -781,6 +785,7 @@
neigh-nud_state = NUD_FAILED;
notify = 1;
+ call_netevent_notifiers(NETEVENT_NEIGH_UPDATE, neigh);
NEIGH_CACHE_STAT_INC(neigh-tbl, res_failed);
NEIGH_PRINTK2(neigh %p is failed.\n, neigh);
@@ -1051,6 +1056,9 @@
(neigh-flags | NTF_ROUTER) :
(neigh-flags ~NTF_ROUTER);
}
+
+ call_netevent_notifiers(NETEVENT_NEIGH_UPDATE, neigh);
+
write_unlock_bh(neigh-lock);
#ifdef CONFIG_ARPD
if (notify neigh-parms-app_probes)
diff -u -r -x '.*' --new-file linux-2.6.14.5/net/core/netevent.c
linux-2.6.14.5.tom/net/core/netevent.c
--- linux-2.6.14.5/net/core/netevent.c 1969-12-31 18:00:00.0 -0600
+++ linux-2.6.14.5.tom/net/core/netevent.c 2006-01-16 13:50:25.0
-0600
@@ -0,0 +1,69 @@
+/*
+ * Network event notifiers
+ *
+ * Authors:
+ * Tom Tucker
Re: [openib-general] Re: Re: Userspace testing results (many kernels, many svn trees)
Quoting r. Nishanth Aravamudan [EMAIL PROTECTED]: Looks like 5162/5163 is fine building wise. Here is what I got for rdma_lat for a 32-bit server and 32-bit client: loading libehca local address: LID 0x0d QPN 0x140406 PSN 0x253f3e RKey 0x2340032 VAddr 0x0010019001 remote address: LID 0x08 QPN 0x140406 PSN 0xa79d77 RKey 0x2340032 VAddr 0x0010019001 Latency typical: 3.25746e+09 usec Latency best : 3.19975e+09 usec Latency worst : 4.21767e+10 usec and for rdma_bw: loading libehca local address: LID 0x0d, QPN 0x150406, PSN 0x1b3ee5 RKey 0x23a0032 VAddr 0x00f7fce000 remote address: LID 0x08, QPN 0x150406, PSN 0x2fa0a9, RKey 0x23a0032 VAddr 0x00f7fb8000 Bandwidth peak (#0 to #999): 4.3446e-07 MB/sec Bandwidth average: 4.3446e-07 MB/sec Service Demand peak (#0 to #999): 17301 cycles/KB Service Demand Avg : 0 cycles/KB So it's still present... I have just uploaded a simple utility which I called clock_test which measures a clock once a second: this way you'll know whether mtfb is measuring time properly. Update to the latest bits and give it a try. -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] Re: [PATCH] RFC: mthca handling of signals
Quoting r. Roland Dreier [EMAIL PROTECTED]: Subject: Re: [PATCH] RFC: mthca handling of signals This is a good catch, and I think this is the right solution. One question: does the mutex use in mthca_multicast_detach() suffer from the same problem? It seems that we might call to detach from a multicast group during process exit with a signal pending, and end up failing there when we shouldn't. - R. While this isnt a probem, we probably want to change the one in mcast_attach as well, for consistency. -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] Re: Re: Userspace testing results (many kernels, many svn trees)
On 23.01.2006 [23:22:45 +0200], Michael S. Tsirkin wrote: Quoting r. Nishanth Aravamudan [EMAIL PROTECTED]: Subject: Re: Re: Userspace testing results (many kernels,many svn trees) On 23.01.2006 [10:24:12 -0800], Nishanth Aravamudan wrote: On 23.01.2006 [19:55:05 +0200], Michael S. Tsirkin wrote: Quoting r. Shirley Ma [EMAIL PROTECTED]: Subject: Re: Re: Userspace testing results (many kernels,?many svn trees) If true, it seems that this line typedef unsigned long long cycles_t; should be replaced by typedef unsigned long cycles_t; Yes. OK, I did it this way. # svn ci get_clock.h Sendingget_clock.h Transmitting file data . Committed revision 5163. You might want to try this rev out. heh, ok. I'm going to let the 5162 version of a 32/32 setup finish, then run 5163. Looks like 5162/5163 is fine building wise. Here is what I got for rdma_lat for a 32-bit server and 32-bit client: loading libehca local address: LID 0x0d QPN 0x140406 PSN 0x253f3e RKey 0x2340032 VAddr 0x0010019001 remote address: LID 0x08 QPN 0x140406 PSN 0xa79d77 RKey 0x2340032 VAddr 0x0010019001 Latency typical: 3.25746e+09 usec Latency best : 3.19975e+09 usec Latency worst : 4.21767e+10 usec and for rdma_bw: loading libehca local address: LID 0x0d, QPN 0x150406, PSN 0x1b3ee5 RKey 0x23a0032 VAddr 0x00f7fce000 remote address: LID 0x08, QPN 0x150406, PSN 0x2fa0a9, RKey 0x23a0032 VAddr 0x00f7fb8000 Bandwidth peak (#0 to #999): 4.3446e-07 MB/sec Bandwidth average: 4.3446e-07 MB/sec Service Demand peak (#0 to #999): 17301 cycles/KB Service Demand Avg : 0 cycles/KB So it's still present... Thanks, Nish Hmm. Could you please try running e.g. rdma_lat with -H to get all the results? rdma_lat -H: loading libehca local address: LID 0x0d QPN 0x140406 PSN 0x304b04 RKey 0x2340032 VAddr 0x0010019001 remote address: LID 0x08 QPN 0x140406 PSN 0xc99ca RKey 0x2340032 VAddr 0x0010019001 #, usec 1, 3.20378e+09 2, 3.20646e+09 3, 3.21451e+09 4, 3.21586e+09 5, 3.2172e+09 6, 3.21854e+09 7, 3.21854e+09 8, 3.21988e+09 9, 3.21988e+09 10, 3.22123e+09 11, 3.22123e+09 12, 3.22123e+09 13, 3.22123e+09 14, 3.22123e+09 15, 3.22123e+09 16, 3.22257e+09 17, 3.22257e+09 18, 3.22257e+09 19, 3.22257e+09 20, 3.22391e+09 21, 3.22391e+09 22, 3.22391e+09 23, 3.22391e+09 24, 3.22391e+09 25, 3.22391e+09 26, 3.22391e+09 27, 3.22391e+09 28, 3.22391e+09 29, 3.22391e+09 30, 3.22391e+09 31, 3.22525e+09 32, 3.22525e+09 33, 3.22525e+09 34, 3.22525e+09 35, 3.22525e+09 36, 3.22525e+09 37, 3.22525e+09 38, 3.22659e+09 39, 3.22659e+09 40, 3.22659e+09 41, 3.22659e+09 42, 3.22659e+09 43, 3.22659e+09 44, 3.22659e+09 45, 3.22659e+09 46, 3.22794e+09 47, 3.22794e+09 48, 3.22794e+09 49, 3.22794e+09 50, 3.22794e+09 51, 3.22794e+09 52, 3.22794e+09 53, 3.22794e+09 54, 3.22794e+09 55, 3.22794e+09 56, 3.22794e+09 57, 3.22794e+09 58, 3.22794e+09 59, 3.22794e+09 60, 3.22794e+09 61, 3.22928e+09 62, 3.22928e+09 63, 3.22928e+09 64, 3.22928e+09 65, 3.22928e+09 66, 3.22928e+09 67, 3.22928e+09 68, 3.22928e+09 69, 3.22928e+09 70, 3.22928e+09 71, 3.22928e+09 72, 3.23062e+09 73, 3.23062e+09 74, 3.23062e+09 75, 3.23196e+09 76, 3.23196e+09 77, 3.23196e+09 78, 3.23196e+09 79, 3.23196e+09 80, 3.23196e+09 81, 3.23196e+09 82, 3.23196e+09 83, 3.23196e+09 84, 3.23196e+09 85, 3.23196e+09 86, 3.23196e+09 87, 3.23331e+09 88, 3.23331e+09 89, 3.23331e+09 90, 3.23331e+09 91, 3.23331e+09 92, 3.23331e+09 93, 3.23331e+09 94, 3.23465e+09 95, 3.23465e+09 96, 3.23465e+09 97, 3.23465e+09 98, 3.23465e+09 99, 3.23465e+09 100, 3.23465e+09 101, 3.23465e+09 102, 3.23465e+09 103, 3.23465e+09 104, 3.23465e+09 105, 3.23465e+09 106, 3.23465e+09 107, 3.23599e+09 108, 3.23599e+09 109, 3.23599e+09 110, 3.23599e+09 111, 3.23599e+09 112, 3.23599e+09 113, 3.23599e+09 114, 3.23599e+09 115, 3.23599e+09 116, 3.23599e+09 117, 3.23599e+09 118, 3.23599e+09 119, 3.23599e+09 120, 3.23599e+09 121, 3.23599e+09 122, 3.23599e+09 123, 3.23733e+09 124, 3.23733e+09 125, 3.23733e+09 126, 3.23733e+09 127, 3.23733e+09 128, 3.23733e+09 129, 3.23733e+09 130, 3.23733e+09 131, 3.23733e+09 132, 3.23733e+09 133, 3.23733e+09 134, 3.23733e+09 135, 3.23733e+09 136, 3.23733e+09 137, 3.23733e+09 138, 3.23733e+09 139, 3.23733e+09 140, 3.23733e+09 141, 3.23867e+09 142, 3.23867e+09 143, 3.23867e+09 144, 3.23867e+09 145, 3.23867e+09 146, 3.23867e+09 147, 3.23867e+09 148, 3.23867e+09 149, 3.23867e+09 150, 3.23867e+09 151, 3.23867e+09 152, 3.23867e+09 153, 3.23867e+09 154, 3.23867e+09 155, 3.23867e+09 156, 3.23867e+09 157, 3.23867e+09 158, 3.23867e+09 159, 3.24002e+09 160, 3.24002e+09 161, 3.24002e+09 162, 3.24002e+09 163, 3.24002e+09 164, 3.24002e+09 165, 3.24002e+09 166, 3.24002e+09 167, 3.24002e+09 168, 3.24002e+09 169, 3.24002e+09 170, 3.24002e+09 171, 3.24002e+09 172, 3.24002e+09 173, 3.24002e+09 174, 3.24002e+09 175, 3.24002e+09 176,
[openib-general] Re: Re: Userspace testing results (many kernels, many svn trees)
Quoting r. Nishanth Aravamudan [EMAIL PROTECTED]: Subject: Re: Re: Userspace testing results (many kernels,many svn trees) On 23.01.2006 [23:22:45 +0200], Michael S. Tsirkin wrote: Quoting r. Nishanth Aravamudan [EMAIL PROTECTED]: Subject: Re: Re: Userspace testing results (many kernels,many svn trees) On 23.01.2006 [10:24:12 -0800], Nishanth Aravamudan wrote: On 23.01.2006 [19:55:05 +0200], Michael S. Tsirkin wrote: Quoting r. Shirley Ma [EMAIL PROTECTED]: Subject: Re: Re: Userspace testing results (many kernels,?many svn trees) If true, it seems that this line typedef unsigned long long cycles_t; should be replaced by typedef unsigned long cycles_t; Yes. OK, I did it this way. # svn ci get_clock.h Sendingget_clock.h Transmitting file data . Committed revision 5163. You might want to try this rev out. heh, ok. I'm going to let the 5162 version of a 32/32 setup finish, then run 5163. Looks like 5162/5163 is fine building wise. Here is what I got for rdma_lat for a 32-bit server and 32-bit client: loading libehca local address: LID 0x0d QPN 0x140406 PSN 0x253f3e RKey 0x2340032 VAddr 0x0010019001 remote address: LID 0x08 QPN 0x140406 PSN 0xa79d77 RKey 0x2340032 VAddr 0x0010019001 Latency typical: 3.25746e+09 usec Latency best : 3.19975e+09 usec Latency worst : 4.21767e+10 usec and for rdma_bw: loading libehca local address: LID 0x0d, QPN 0x150406, PSN 0x1b3ee5 RKey 0x23a0032 VAddr 0x00f7fce000 remote address: LID 0x08, QPN 0x150406, PSN 0x2fa0a9, RKey 0x23a0032 VAddr 0x00f7fb8000 Bandwidth peak (#0 to #999): 4.3446e-07 MB/sec Bandwidth average: 4.3446e-07 MB/sec Service Demand peak (#0 to #999): 17301 cycles/KB Service Demand Avg : 0 cycles/KB So it's still present... Thanks, Nish Hmm. Could you please try running e.g. rdma_lat with -H to get all the results? rdma_lat -H: loading libehca local address: LID 0x0d QPN 0x140406 PSN 0x304b04 RKey 0x2340032 VAddr 0x0010019001 remote address: LID 0x08 QPN 0x140406 PSN 0xc99ca RKey 0x2340032 VAddr 0x0010019001 #, usec Latency typical: 3.25746e+09 usec Latency best : 3.20378e+09 usec Latency worst : 4.0715e+10 usec Could the high/low bits be swapped? What happends if you change cycles_t from long long to long? Could you try running the clock_test utility? -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Re: Re: Userspace testing results (many kernels, many svn trees)
On 24.01.2006 [01:01:32 +0200], Michael S. Tsirkin wrote: Quoting r. Nishanth Aravamudan [EMAIL PROTECTED]: Looks like 5162/5163 is fine building wise. Here is what I got for rdma_lat for a 32-bit server and 32-bit client: loading libehca local address: LID 0x0d QPN 0x140406 PSN 0x253f3e RKey 0x2340032 VAddr 0x0010019001 remote address: LID 0x08 QPN 0x140406 PSN 0xa79d77 RKey 0x2340032 VAddr 0x0010019001 Latency typical: 3.25746e+09 usec Latency best : 3.19975e+09 usec Latency worst : 4.21767e+10 usec and for rdma_bw: loading libehca local address: LID 0x0d, QPN 0x150406, PSN 0x1b3ee5 RKey 0x23a0032 VAddr 0x00f7fce000 remote address: LID 0x08, QPN 0x150406, PSN 0x2fa0a9, RKey 0x23a0032 VAddr 0x00f7fb8000 Bandwidth peak (#0 to #999): 4.3446e-07 MB/sec Bandwidth average: 4.3446e-07 MB/sec Service Demand peak (#0 to #999): 17301 cycles/KB Service Demand Avg : 0 cycles/KB So it's still present... I have just uploaded a simple utility which I called clock_test which measures a clock once a second: this way you'll know whether mtfb is measuring time properly. Will it get built by running make in the perftest directory? Any special usage I should know about? Thanks, Nish ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Re: Re: Userspace testing results (many kernels, many svn trees)
Quoting r. Nishanth Aravamudan [EMAIL PROTECTED]: I have just uploaded a simple utility which I called clock_test which measures a clock once a second: this way you'll know whether mtfb is measuring time properly. Will it get built by running make in the perftest directory? Yes. Any special usage I should know about? Look at its source, you'll see. You just run it for a while and it will print out the time tkaen from mtfb each second. Kill it with CRTL-C. -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] Re: Re: Userspace testing results (many kernels, many svn trees)
On 24.01.2006 [01:39:07 +0200], Michael S. Tsirkin wrote: Quoting r. Nishanth Aravamudan [EMAIL PROTECTED]: Subject: Re: Re: Userspace testing results (many kernels,many svn trees) On 23.01.2006 [23:22:45 +0200], Michael S. Tsirkin wrote: Quoting r. Nishanth Aravamudan [EMAIL PROTECTED]: Subject: Re: Re: Userspace testing results (many kernels,many svn trees) On 23.01.2006 [10:24:12 -0800], Nishanth Aravamudan wrote: On 23.01.2006 [19:55:05 +0200], Michael S. Tsirkin wrote: Quoting r. Shirley Ma [EMAIL PROTECTED]: Subject: Re: Re: Userspace testing results (many kernels,?many svn trees) If true, it seems that this line typedef unsigned long long cycles_t; should be replaced by typedef unsigned long cycles_t; Yes. OK, I did it this way. # svn ci get_clock.h Sendingget_clock.h Transmitting file data . Committed revision 5163. You might want to try this rev out. heh, ok. I'm going to let the 5162 version of a 32/32 setup finish, then run 5163. Looks like 5162/5163 is fine building wise. Here is what I got for rdma_lat for a 32-bit server and 32-bit client: loading libehca local address: LID 0x0d QPN 0x140406 PSN 0x253f3e RKey 0x2340032 VAddr 0x0010019001 remote address: LID 0x08 QPN 0x140406 PSN 0xa79d77 RKey 0x2340032 VAddr 0x0010019001 Latency typical: 3.25746e+09 usec Latency best : 3.19975e+09 usec Latency worst : 4.21767e+10 usec and for rdma_bw: loading libehca local address: LID 0x0d, QPN 0x150406, PSN 0x1b3ee5 RKey 0x23a0032 VAddr 0x00f7fce000 remote address: LID 0x08, QPN 0x150406, PSN 0x2fa0a9, RKey 0x23a0032 VAddr 0x00f7fb8000 Bandwidth peak (#0 to #999): 4.3446e-07 MB/sec Bandwidth average: 4.3446e-07 MB/sec Service Demand peak (#0 to #999): 17301 cycles/KB Service Demand Avg : 0 cycles/KB So it's still present... Thanks, Nish Hmm. Could you please try running e.g. rdma_lat with -H to get all the results? rdma_lat -H: loading libehca local address: LID 0x0d QPN 0x140406 PSN 0x304b04 RKey 0x2340032 VAddr 0x0010019001 remote address: LID 0x08 QPN 0x140406 PSN 0xc99ca RKey 0x2340032 VAddr 0x0010019001 #, usec Latency typical: 3.25746e+09 usec Latency best : 3.20378e+09 usec Latency worst : 4.0715e+10 usec Could the high/low bits be swapped? I was thinking that might be it, but wasn't sure. What happends if you change cycles_t from long long to long? Could you try running the clock_test utility? I'll try the latter first (I found the usage from the file and it is built by make). Could you send me a patch to do the former, in case I need to? It can be a patch that applies directly in the perftest directory. Thanks, Nish ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Re: [PATCH] CMA and iWARP
Yes, but we need to start somewhere. Until someone submits a driver that does all the things you mention, it makes sense to move forward with what has been proposed to date. I agree with this, and overall I am very much in favor of getting iWARP support all the way upstream. The reason I want to take time to make sure that we have the right code before we merge it is that I get the feeling that there may be elements of a) using the IB tree to get changes upstream that would be vetoed on netdev and b) trying to get openib and the kernel community to accept code just so a vendor can meet a product marketing deadline. BTW, upon reflection, the best idea for moving this forward might be to push the Ammasso driver along with the rest of the iWARP patches, so that there's some more context for review. Just because a vendor is out of business is no reason for Linux not to have a driver for a piece of hardware. - R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] RE: [RFC] DAT 2.0 immediate data proposal
Arkady, Response inline From: Kanevsky, Arkady [mailto:[EMAIL PROTECTED] Sent: Tuesday, January 17, 2006 7:16 AM To: Davis, Arlin R; Lentini, James Cc: [EMAIL PROTECTED]; openib-general@openib.org Subject: RE: [RFC] DAT 2.0 immediate data proposal Arlin, a few things need to be addressed. 1. correlation with local and remote invalidate This potentially effects both DAT_DTOs and post operations How does this differ from normal sends or writes? 2. Need a precise defintion for CONFIRM_FLAG definition in a transport independent fashion. What guarantees DAT Provider provides on successful local completion? Remote end guarantee? My understanding what you are trying to do is create 2 models one IB and one for iWARP. So for IB Consumers will use CONFIRM_FLAG and for iWARP IMMED_FLAG. Provider will indicate in Provider_attr which model it supports. The issue I have with it is that I do not see a model that Consumer can use to create a transport independent code. It looks like Immed_flag can be made transport independent. But with sender specifying the behavior a protocol extension is needed for IB. IB will always deliver Immediate data in the header not a payload and remote Provider can control how it is delivered to a Consumer. But this means that there is no need for DTO_flags for Send side. Instead it can be used for Recv side or controlled purely by Provider. Maybe we need to just go back to one model and always deliver via the event? With the post_recv_immed requirements, other transports have a mechanism to emulate and create the necessary resources on the recv side to place idata and copy to event when operation is completed. Would this work for iWARP? Two different models for receiving idata should be avoided if at all possible. 3. Need to define error behavior. for new operations, async errors, EP behavior. I will work on updating the draft. post_send_immed will look much like post_send and post_rdma_write_immed will look a lot like post_rdma_write with some additional errors based on the post receive buffer requirement. 4. Need to define DAT_Provider attributes for immediate data and dto_flags behavior 5. Does Solicited_wait completion_flag value now applicable for RDMA_write for immediate data? yes, applicable to send, send_immed, and write_immed 6. Is dto_completion_data xfer_length include immediate_data size or not? no 7. what memory privilages needed for a recv buffer for immediate data? Based on the operation write_immed would require write privileges and send_immed would require recv privileges. 8. SRQ interaction? Good question. all post_recv_immed or all post_recv? 9. What happens of buffer for recv operation NOT recv_immed is matched for incomming recv/rdma_write op? The rules should be: Can receive a send, send_immed, or write_immed with recv_immed. Cannot receive send_immed or write_immed on a recv. However, I am not sure how you would enforce this on IB (DTO error on the receiving side?) since the idata is delivered via CQ and does not require a special receive post descriptor. 10. Change dat_ep_post_write_immed to dat_ep_post_rdma_write_immed to be consistent with current terminology. Ok 11. Need to cleanup operation description to make it clear that Send|RDMA_write and immediate data part is a single atomic operation. The current followed by language is misleading. Make it explicit that there is a single local DTO completion and single remote DTO completion. Ok, I will clean that up 12. Is your intension that post_recv_immed can ONLY except immediate data and is not capable to recv any message? No, the intention is to extend the post_recv to handle 32bit idata which may arrive with or without other send or rdma_write data. Does it make more sense to add a dto_flags to the existing post_recv? 13. size should be num_segments for dat_ep_post_recv_immed() ok ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] RE: [RFC] DAT 2.0 immediate data proposal
Maybe we need to just go back to one model and always deliver via the event? With the post_recv_immed requirements, other transports have a mechanism to emulate and create the necessary resources on the recv side to place idata and copy to event when operation is completed. Would this work for iWARP? Two different models for receiving idata should be avoided if at all possible. Always delivering by the event is not feasible for an iWARP vendor. If you are working over RDMAC verbs then the work completion is no longer accessible by the time the Work Completion is reaped. So copying from the receive buffer to the event does not work since the location of the receive buffer is now known only to the application. The same problem exists in the opposite direction for InfiniBand HCAs using standard verbs. They cannot copy from the CQE to the receive buffer. So the user is stuck checking a flag or the event type to know where their data is. This is not terribly user friendly, but it is the best that can be offered if we want to enable this optimization. The need to check the flag does reduce the value of the optimization though. 6. Is dto_completion_data xfer_length include immediate_data size or not? no Then how does the receiver know how much data there is? Even if an iWarp Provider attempts to optimize immediate placement into the CQ, it will end up setting the xfer_length whenever the packet is received out of order. So it is far simpler for the application to simply know that the data will be in the buffer, and that the xfer_length will be set. It doesn't need to worry about whether they were set by the cq_poll verb or by the hardware. 11. Need to cleanup operation description to make it clear that Send|RDMA_write and immediate data part is a single atomic operation. The current followed by language is misleading. Make it explicit that there is a single local DTO completion and single remote DTO completion. Ok, I will clean that up The best mapping available over RDMAC-compliant firmware for an iWARP NIC would be to post two operations (RDMA Write followed by a short Send). That would require additional spacein the send and completion queues since a completion for the write can only be suppressed for a successful completion. Whether these extra slots were required would be an IA attribute. And the requirement is that nothing for that QP can come between the iWARP Write and the Send. How the provider does that is up to it. Options include locking over both posts and a composite work request. Anyone working over existing RDMAC-compliant verbs will have to use the first approach. 12. Is your intension that post_recv_immed can ONLY except immediate data and is not capable to recv any message? No, the intention is to extend the post_recv to handle 32bit idata which may arrive with or without other send or rdma_write data. Does it make more sense to add a dto_flags to the existing post_recv? How does this map to iWARP? When the data can be sent as an immediate OR as data, then when received it can be placed into the receive buffer or even potentially directly into the CQ when everything aligns just right. But an iWARP sender has to place the immediate value as the first four bytes of a Send message. There is no other mapping than makes sense. Shoving the rest of the message up is complex, as is using the last four bytes of the message since the last four bytes *could* cross a DDP Segment boundary, and would require the user to provide a buffer that was 4 bytes larger. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] Re: [PATCH] CMA and iWARP
On Mon, Jan 23, 2006 at 03:53:19PM -0800, Roland Dreier wrote: Yes, but we need to start somewhere. Until someone submits a driver that does all the things you mention, it makes sense to move forward with what has been proposed to date. I agree with this, and overall I am very much in favor of getting iWARP support all the way upstream. *nod* BTW, this is a message that needs to be repeated regularly until iWARP support *is* upstream. The opposite perception is still lingering in some places because of discussions from 1 and 2 years ago. The reason I want to take time to make sure that we have the right code before we merge it is that I get the feeling that there may be elements of a) using the IB tree to get changes upstream that would be vetoed on netdev Yeah, that has happened before. And I expect netdev folks might strongly object (if they haven't already) to some sideband method of managing TCP/IP config when TCP/IP is exclusively running on an RNIC (TOE with RDMA front-end). IMHO, that's seems like the hardest to fix issue so everyone is happy. Most of the other details can be negotiated. and b) trying to get openib and the kernel community to accept code just so a vendor can meet a product marketing deadline. TTM via kernel.org? BWHAHAHA! :^) Sorry, I can't take that serious. :) BTW, upon reflection, the best idea for moving this forward might be to push the Ammasso driver along with the rest of the iWARP patches, so that there's some more context for review. Just because a vendor is out of business is no reason for Linux not to have a driver for a piece of hardware. Exactly. says the co-maintainer of the parisc-linux port. :) thanks, grant ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] RE: [RFC] DAT 2.0 immediate data proposal
Arlin, comments inline. Arkady Kanevsky email: [EMAIL PROTECTED] Network Appliance Inc. phone: 781-768-5395 1601 Trapelo Rd. - Suite 16.Fax: 781-895-1195 Waltham, MA 02451 central phone: 781-768-5300 From: Davis, Arlin R [mailto:[EMAIL PROTECTED] Sent: Monday, January 23, 2006 7:15 PMTo: Kanevsky, Arkady; Lentini, JamesCc: openib-general@openib.org; [EMAIL PROTECTED]Subject: RE: [RFC] DAT 2.0 immediate data proposal Arkady, Response inline From: Kanevsky, Arkady [mailto:[EMAIL PROTECTED] Sent: Tuesday, January 17, 2006 7:16 AMTo: Davis, Arlin R; Lentini, JamesCc: [EMAIL PROTECTED]; openib-general@openib.orgSubject: RE: [RFC] DAT 2.0 immediate data proposal Arlin, a few things need to be addressed. 1. correlation with local and remote invalidate This potentially effects both DAT_DTOs and post operations How does this differ from normal sends or writes?[AK]We had added a new Send_with_Invalidate. The completion also states whether RMR was invalidated and which one. But the text for interaction is added through out the completion and post operations. See the latest draft of uDAPL and kDAPL 1.3 specs on the DAT reflector. 2. Need a precise defintion for CONFIRM_FLAG definition in a transport independent fashion. What guarantees DAT Provider "provides" on successful local completion? Remote end guarantee? My understanding what you are trying to do is create 2 models one IB and one for iWARP. So for IB Consumers will use CONFIRM_FLAG and for iWARP IMMED_FLAG. Provider will indicate in Provider_attr which model it supports. The issue I have with it is that I do not see a model that Consumer can use to create a transport independent code. It looks like Immed_flag can be made transport independent. But with "sender" specifying the behavior a protocol extension is needed for IB. IB will always deliver Immediate data in the header not a payload and remote Provider can control how it is delivered to a Consumer. But this means that there is no need for DTO_flags for Send side. Instead it can be used for Recv side or controlled purely by Provider. Maybe we need to just go back to one model and always deliver via the event? With the post_recv_immed requirements, other transports have a mechanism to emulate and create the necessary resources on the recv side to place idata and copy to event when operation is completed. Would this work for iWARP? Two different models for receiving idata should be avoided if at all possible.[AK]Caitlin already responded to this. 3. Need to define error behavior. for new operations, async errors, EP behavior. I will work on updating the draft. post_send_immed will look much like post_send and post_rdma_write_immed will look a lot like post_rdma_write with some additional errors based on the post receive buffer requirement.[AK]Also consider if youwantto addremote invalidate to the new operation. 4. Need to define DAT_Provider attributes for immediate data and dto_flags behavior 5. Does Solicited_wait completion_flag value now applicable for RDMA_write for immediate data? yes, applicable to send, send_immed, and write_immed 6. Is dto_completion_data xfer_length include immediate_data size or not? no[AK]It can work both ways. Either we include4 extra bytes for immediate dataor not. Consumerjust have to know.The real data alwaysstarts at 4 byte boundary into the buffer is immediate data is returned inline. We need to state how immediate data is positioned if it is smaller than 4 bytes. 7. what memory privilages needed for a recv buffer for immediate data? Based on the operation write_immed would require write privileges and send_immed would require recv privileges. 8. SRQ interaction? Good question. all post_recv_immed or all post_recv?[AK]Will this work for the user model? Not supporting handling immediate recv and regular recv with potential immediate data onone SRQ. 9. What happens of buffer for recv operation NOT recv_immed is matched for incomming recv/rdma_write op? The rules should be: Can receive a send, send_immed, or write_immed with recv_immed. Cannot receive send_immed or write_immed on a recv. However, I am not sure how you would enforce this on IB (DTO error on the receiving side?) since the idata is delivered via CQ and does not require a special receive post descriptor.[AK]We can make thisProvider attribute. Or we can state that if immed data is return in event then there is no error for recv. 10. Change dat_ep_post_write_immed to dat_ep_post_rdma_write_immed to be consistent with current
Re: [openib-general] Re: Re: Userspace testing results (many kernels, many svn trees)
Michael Could the high/low bits be swapped? What happends if you
Michael change cycles_t from long long to long? Could you try
Michael running the clock_test utility?
What seems to be happening is that mftb is giving the low 32 bits of
the timebase (as expected on ppc32). Since your get_cycles() is
returning a long long, those 32 bits get put in the most significant
32 bits of the return value, and the low 32 bits are garbage (ppc is
big endian).
If I compile clock_test for ppc32, I see that get_cycles() compiles to:
164c get_cycles:
164c: 7c 6c 42 e6 mftbr3
1650: 4e 80 00 20 blr
For comparison, a function like
unsigned long long blah(void) { return 0x10002ull; }
compiles to
blah:
0: 38 60 00 01 li r3,1
4: 38 80 00 02 li r4,2
8: 4e 80 00 20 blr
In other words the convention on ppc32 is that unsigned long long
return values have the high 32 bits in r3 and the low 32 bits in r4.
I think you want to use something like
typedef unsigned long long cycles_t;
static inline cycles_t get_cycles()
{
unsigned long low, hi, hi2;
do {
asm volatile (mftbu %0 : =r (hi));
asm volatile (mftb %0 : =r (low));
asm volatile (mftbu %0 : =r (hi2));
} while (hi != hi2);
return ((unsigned long long) hi 32) | low;
}
for ppc32. However, this is not quite enough to make things work on
all powerpc systems, because the timebase does not necessarily run at
the same speed as the CPU. For example, on an IBM JS20 blade,
clock_test prints
1 sec = 6536.8 usec
1 sec = 6537.05 usec
(both as a 32-bit and 64-bit executable) because, as /proc/cpuinfo shows:
processor : 0
cpu : PPC970FX, altivec supported
clock : 2194.624509MHz
revision: 3.0
processor : 1
cpu : PPC970FX, altivec supported
clock : 2194.624509MHz
revision: 3.0
timebase: 14318000
machine : CHRP IBM,8842-P2C
the timebase runs at about 14.3 MHz, or approx 153 times slower than
the CPU clock.
I'm not sure how you want to fix this in perftest.
- R.
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH] RFC: AMSO1100 iWARP Driver
Tom, thanks for posting this. After thinking this over, I really
think the amso1100 driver belongs upstream. If Linux is shipping
ARCnet and Sound Blaster CD-ROM support, then you've got a long wait
until your card is obsolete enough to forget about. And having a real
iWARP driver just makes things a lot easier to justify and understand,
although I would still like to get buy-in from people like NetEffect
and Chelsio.
Anyway, some easy comments from a quick skim:
+/*
+ * WARNING: If you change this file, also bump CC_IVN_BASE
+ * in common/include/clustercore/cc_ivn.h.
+ */
Uh, where's clustercore/cc_ivn.h?
+typedef enum {
...
+} cc_event_id_t;
+typedef enum {
...
+} cc_resource_indicator_t;
typedefs that create foo_t are strongly deprecated in the kernel.
Just do
enum cc_event_id {
...
};
and use enum cc_event_id everywhere.
+switch (mq_index) {
+case (0):
no need for parentheses here. and can the magic (0), (1), (2) be
given names that say what they mean?
+struct c2_mq_shared volatile *shared;
volatile declarations are almost always a bug... use proper locking or
memory barriers to say what you mean instead.
+/*
+ * Now read back shared-armed to make the PCI
+ * write synchronous. This is necessary for
+ * correct cq notification semantics.
+ */
+{
+volatile char c;
+c = shared-armed;
+}
If you're reading across PCI you should be using readb().
+qp-adapter_handle = reply-qp_handle;
+qp-state = IB_QPS_RESET;
+qp-send_sgl_depth = qp_attrs-cap.max_send_sge;
+qp-rdma_write_sgl_depth = qp_attrs-cap.max_send_sge;
+qp-recv_sgl_depth = qp_attrs-cap.max_recv_sge;
whitespace damage alert
+#define assert(expr) \
+if(!(expr)) { \
+printk(KERN_ERR PFX Assertion failed! %s, %s, %s, line %d\n,\
+ #expr, __FILE__, __FUNCTION__, __LINE__); \
+}
probably just use BUG_ON() -- then you get a traceback too.
+struct c2_adapter_pci_regs {
+char reg_magic[8];
+u32 version;
+u32 ivn;
+u32 pci_window_size;
+u32 q0_q_size;
Indent with tabs not 4 spaces (lots of other places too)
+static inline u32 c2_read32(const void __iomem *addr)
+{
+return readl(addr);
+}
Any reason for not using readl() directly (and similarly for all the
other c2_readxx c2_writexx funcs)?
- R.
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH] RFC: AMSO1100 iWARP Driver
Roland:
Thanks for the comments. I'll take a pass through given your review and
repost.
Thanks, again,
Tom
On Mon, 2006-01-23 at 22:07 -0800, Roland Dreier wrote:
Tom, thanks for posting this. After thinking this over, I really
think the amso1100 driver belongs upstream. If Linux is shipping
ARCnet and Sound Blaster CD-ROM support, then you've got a long wait
until your card is obsolete enough to forget about. And having a real
iWARP driver just makes things a lot easier to justify and understand,
although I would still like to get buy-in from people like NetEffect
and Chelsio.
Anyway, some easy comments from a quick skim:
+/*
+ * WARNING: If you change this file, also bump CC_IVN_BASE
+ * in common/include/clustercore/cc_ivn.h.
+ */
Uh, where's clustercore/cc_ivn.h?
time-warp comment.
+typedef enum {
...
+} cc_event_id_t;
+typedef enum {
...
+} cc_resource_indicator_t;
typedefs that create foo_t are strongly deprecated in the kernel.
Just do
enum cc_event_id {
...
};
got it.
and use enum cc_event_id everywhere.
+ switch (mq_index) {
+ case (0):
no need for parentheses here. and can the magic (0), (1), (2) be
given names that say what they mean?
yeah - I noticed this too.
+ struct c2_mq_shared volatile *shared;
volatile declarations are almost always a bug... use proper locking or
memory barriers to say what you mean instead.
agreed. wmb()
+ /*
+ * Now read back shared-armed to make the PCI
+ * write synchronous. This is necessary for
+ * correct cq notification semantics.
+ */
+ {
+ volatile char c;
+ c = shared-armed;
+ }
If you're reading across PCI you should be using readb().
agreed.
+ qp-adapter_handle = reply-qp_handle;
+ qp-state = IB_QPS_RESET;
+qp-send_sgl_depth = qp_attrs-cap.max_send_sge;
+qp-rdma_write_sgl_depth = qp_attrs-cap.max_send_sge;
+qp-recv_sgl_depth = qp_attrs-cap.max_recv_sge;
whitespace damage alert
it's all over... vi vs. emacs wars.
+#define assert(expr) \
+if(!(expr)) { \
+printk(KERN_ERR PFX Assertion failed! %s, %s, %s, line %d\n,\
+ #expr, __FILE__, __FUNCTION__, __LINE__); \
+}
probably just use BUG_ON() -- then you get a traceback too.
yep.
+struct c2_adapter_pci_regs {
+char reg_magic[8];
+u32 version;
+u32 ivn;
+u32 pci_window_size;
+u32 q0_q_size;
Indent with tabs not 4 spaces (lots of other places too)
+static inline u32 c2_read32(const void __iomem *addr)
+{
+ return readl(addr);
+}
Any reason for not using readl() directly (and similarly for all the
other c2_readxx c2_writexx funcs)?
we used to share firmware/device driver code... These could be reduced
to their Linux equivalent readl, etc
- R.
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
