>>> I'm not sure the standard OpenIB NetPIPE runs can reproduce this >>> type of workload. However, we have developed a working PVFS2- >>> NetPIPE module which can reproduce this problem on occassion, if >>> there is interest in further testing this on your end, I can make >>> it available. > Yes. Please send it to me. I'd like to test it. Is it a user space > appl.? > I want to see if we could reach the limit of mappings mentioned above.
The netpipe code is available with mercurial by: hg clone http://source.scl.ameslab.gov/hg/netpipe3-pvfs-dev Once you have pvfs2-1.5.1 installed, you should be able to do 'make pvfs' in the netpipe3-pvfs-dev directory and build NPpvfs. The command line arguments I used to reproduce this were: ./NPpvfs -d $PVFS_FILE_PATH -l 32768 -u 268435456 -n 100 -o $NETPIPE_OUTPUT_FILE This is the dmesg log: PU0001 000e0091:ehca_hcall_7arg_7ret HCAD_ERROR opcode=160 ret=fffffffffffffff7 arg1=1000000003000004 arg2=5 arg3=4000f830000 arg4=10000 arg5=e0000000000000 arg6=eb6b6920 arg7=0 out1=0 out2=0 out3=0 out4=0 out5=0 out6=0 out7=0 PU0001 00090454:ehca_reg_mr HCAD_ERROR hipz_alloc_mr failed, h_ret=fffffffffffffff7 hca_hndl=1000000003000004 PU0001 00090478:ehca_reg_mr <<< ret=ffffffea shca=c0000000e796b000 e_mr=c0000000ce865e80 iova_start=000004000f830000 size=10000 acl=7 e_pd=c0000000eb6b6920 pginfo=c0000000dfcb3a70 num_pages=10 num_4k=10 PU0001 00090176:ehca_reg_user_mr <<< rc=ffffffffffffffea pd=c0000000eb6b6920 region=c0000000ce861dd0 mr_access_flags=7 udata=c0000000dfcb3ba0 _______________________________________________ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general