[ewg] [ANNOUNCE] dapl-2.0.23 release
New release for uDAPL v2 (2.0.23) available at: http://www.openfabrics.org/downloads/dapl Latest Packages (see ChangeLog for recent changes): md5sum: f58d6dd903cee271d71b0eb6fa33984e compat-dapl-1.2.14.tar.gz md5sum: dc1f31a4bc28aedcb725264d65db7e51 dapl-2.0.23.tar.gz Vlad, please pull into OFED 1.5 using package spec files and installing following order (v2 first): dapl-2.0.23-1 dapl-utils-2.0.23-1 dapl-devel-2.0.23-1 dapl-debuginfo-2.0.23-1 compat-dapl-1.2.14-1 compat-dapl-devel-1.2.14-1 Summary of changes since last release: v2 - cma: cannot reuse the cm_id and qp for new connection, must reallocate a new one. v2 - scm, cma: update DAPL cm protocol revision with latest address/port changes v2 - ucm: modify IB address format to align better with sockaddr_in6 v2 - Add definition for getpid similar to that used by the other dtest apps. v2 - WinOF provides a common implementation of gettimeofday that should v2 - the completion manager was updated to provide an abstraction that v2 - dtestcm: remove IB verb definitions v2 - dtest, dtestx: remove IB verb definitions v2 - scm: tighten up socket options to insure similiar behavior on Windows and Linux. v2 - cma: improve serialization of destroy and event processing v2 - scm: improve serialization of destroy and state changes v2 - common: no cleanup/release code for timer thread v2 - scm, cma: dapli_thread doesn't always get teminated on library close. v2 - ucm: tighten up locking with CM processing, state changes v2 - ucm: For UD type QP's, return CR p_data with CONN_EST event on passive side. v2 - ucm: cleanup extra cr/lf v2 - ucm: fix issues with UD QP's. v2 - winof: Convert windows version of dapl and dat libaries to use private heaps. v2 - dtest, dtestx: modifications for UD QP testing with ucm provider. v2 - scm, ucm: UD QP support was broken when porting to common openib code base. v2 - cma: cleanup warning with unused local variable, ret, in disconnect v2 - cma: remove debug message after rdma_disconnect failure v2 - scm: socket errno check needs O/S dependent wrapper v2 - dapltest: update script files for WinOF v2 - cma: conditional check for new rdma_cm definition. For more details see the following: http://www.openfabrics.org/downloads/dapl documentation/uDAPL_v20.pdf for v2 API specification documentation/transition_to_dat20_120406.pdf for details on porting to 2.0 documentation/DAT_IB_Extensions.pdf for IB extension details. documentation/DAT_IW_Extensions.pdf for iWARP extension details. documentation/uDAPL_ofed_testing_bkm.pdf for latest uDAPL OFED BKM, updated 8-22-2008 uDAT/uDAPL source git tree is located at git://git.openfabrics.org/~ardavis/scm/dapl.git Thanks, -arlin ___ ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] [ANNOUNCE] dapl-1.2.6 and dapl-2.0.8 release
New release for uDAPL v1 (1.2.6) and v2 (2.0.8) is available at: http://www.openfabrics.org/downloads/dapl md5sum: 752ae54a93b4883c88b41241f52db4ab dapl-1.2.6.tar.gz md5sum: a48f9da59318c395bcc6ad170226764a dapl-2.0.8.tar.gz Vlad, please pull into OFED 1.3.1 using package spec files and installing: dapl-1.2.6-1 dapl-devel-1.2.6-1 dapl-2.0.8-1 dapl-utils-2.0.8-1 dapl-devel-2.0.8-1 dapl-debuginfo-2.0.8-1 tags: dapl-1.2.6-1, dapl-2.0.8-1 Summary of changes since last release: v2 - add private data exchange with reject v1,v2 - better error reporting in non-debug builds v1,v2 - update only OFA entries in dat.conf, cooperate with non-ofa providers v1,v2 - support for zero byte operations, iov==NULL v1,v2 - multi-transport support for inline data and private data differences v1,v2 - fix memory leaks and other reported bugs since OFED 1.3 Thanks, -arlin ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] [PATCH][ofed 1.3 docs] uDAPL release notes updated for OFED 1.3
Signed-off by: Arlin Davis [EMAIL PROTECTED] --- diff --git a/uDAPL_release_notes.txt b/uDAPL_release_notes.txt index 33b684a..84c20d7 100644 --- a/uDAPL_release_notes.txt +++ b/uDAPL_release_notes.txt @@ -1,15 +1,102 @@ - Release Notes for - Gamma 3.3 and OFED 1.3 DAPL Release + OFED 1.3 DAPL Release February 2008 -DAPL GAMMA 3.3/OFED 1.3 RELEASE NOTES +OFED 1.3 RELEASE NOTES This release of the DAPL reference implementation is timed to coincide with OFED release 1.3 of the Open Fabrics (www.openfabrics.org) software stack. + +NEW SINCE OFED 1.2 + +* New Features + 1. Add v2.0 library support for new 2.0 API Specification + 2. Separate v1.2 library release to co-exist with v2.0 libraries. + 3. New dat.conf with both 1.2 and 2.0 support + 4. New v2.0 dtestx utilities to test IB extensions + +* Bug Fixes + + v1.2 and v2.0 + - uDAT: static/dynamic registry parsing fixes + - uDAPL: provider fixes for dat_psp_create_any + - dtest/dapltest: change default provider names to sync with dat.conf + - openib_cma: issues with destroy_cm_id and init/resp exchange + - dapltest: use gettimeofday instead of get_cycles for better portability + - dapltest: endian issue with mem_handle, mem_address + - dapltest fix to include inet_ntoa definitions + - fix build problems on 32-bit and 64-bit PowerPC + - cleanup packaging + + v2.0 + - set default config options to match spec file, --enable-debug --enable-ext-type=ib + - use unique devel target names, libdat2.so, /usr/include/dat2 + - dtestx fix memory leak, freeaddrinfo after getaddrinfo + - Fix for IB extended DTO cookie deallocation on inbound rdma_Write_immed + - WinOF: Update OFED code base to include WinOF changes, work from same code base + - WinOF: add DAT_API definition, __stdcall for windows, nothing for linux + - dtest: add dat_evd_query to check correct size + - openib_cma: add macro to convert SID to PORT + - dtest: endian support for exchanging RMR info + - openib_cma: lower default settings, inline and RDMA init/resp + - openib_cma: missing ia_query for max_iov_segments_per_rdma_write + + v1.2 + - openib_cma: turn down dbg noise level on rejects + - dtest: typo in memset + + +BUILD: v1 and v2 uDAPL source install/build instructions (redhat example): + +# cd to distribution SRPMS directory + cd /tmp/OFED-1.3/SRPMS +rpm -i dapl-1.2*.rpm +rpm -i dapl-2.0*.rpm +cd /usr/src/redhat/SOURCES +tar zxf dapl-1.2*.tgz +tar zxf dapl-2.0*.tgz + + # NON_DEBUG build example for x86_64, using OFED targets + + ./configure --prefix /usr --sysconf=/etc --libdir /usr/lib64 +LDFLAGS=-L/usr/lib64 CPPFLAGS=-I/usr/include + + # build and install + + make + make install + + # DEBUG build example for x86_64, using OFED targets + + ./configure --enable-debug --prefix /usr --sysconf=/etc --libdir /usr/lib64 +LDFLAGS=-L/usr/lib64 CPPFLAGS=-I/usr/include + + # build and install + + make + make install + + # DEBUG messages: set environment variable DAPL_DBG_TYPE, default + mapping is 0x0003 + + DAPL_DBG_TYPE_ERR = 0x0001, + DAPL_DBG_TYPE_WARN = 0x0002, + DAPL_DBG_TYPE_EVD = 0x0004, + DAPL_DBG_TYPE_CM= 0x0008, + DAPL_DBG_TYPE_EP= 0x0010, + DAPL_DBG_TYPE_UTIL = 0x0020, + DAPL_DBG_TYPE_CALLBACK = 0x0040, + DAPL_DBG_TYPE_DTO_COMP_ERR= 0x0080, + DAPL_DBG_TYPE_API = 0x0100, + DAPL_DBG_TYPE_RTN = 0x0200, + DAPL_DBG_TYPE_EXCEPTION = 0x0400, + DAPL_DBG_TYPE_SRQ = 0x0800, + DAPL_DBG_TYPE_CNTR = 0x1000 + + NEW SINCE Gamma 3.2 and OFED 1.1 * New Features ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
RE: [ewg] build_ofa_user.sh for dual dapl?
Is there an updated build_ofa_user.sh that works for the two versions of dapl? I am not sure what Vlad uses to pull and build all the OFED library packages. For uDAPL you can get the latest v1 and v2 packages from http://www.openfabrics.org//downloads/dapl/ dapl-1.2.4.tar.gz dapl-2.0.6.tar.gz and use appropriate rpmbuild options to build RPMS. -arlin ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] [PATCH] DAPL v2 - fix package to build against and target /dat2/include
Fix OFED v2 package to build against and target /dat2/include directory. Signed-off by: Arlin Davis [EMAIL PROTECTED] diff --git a/Makefile.am b/Makefile.am index c4cb1bf..60b3db6 100755 --- a/Makefile.am +++ b/Makefile.am @@ -183,17 +183,17 @@ dapl_udapl_libdaplofa_la_LDFLAGS = -version-info 2:0:0 $(daplofa_version_script) libdatincludedir = $(includedir)/dat2 -libdatinclude_HEADERS = dat/include/dat/dat.h \ - dat/include/dat/dat_error.h \ - dat/include/dat/dat_platform_specific.h \ - dat/include/dat/dat_redirection.h \ - dat/include/dat/dat_registry.h \ - dat/include/dat/dat_vendor_specific.h \ - dat/include/dat/udat_config.h \ - dat/include/dat/udat.h \ - dat/include/dat/udat_redirection.h \ - dat/include/dat/udat_vendor_specific.h \ - dat/include/dat/dat_ib_extensions.h +libdatinclude_HEADERS = dat/include/dat2/dat.h \ + dat/include/dat2/dat_error.h \ + dat/include/dat2/dat_platform_specific.h \ + dat/include/dat2/dat_redirection.h \ + dat/include/dat2/dat_registry.h \ + dat/include/dat2/dat_vendor_specific.h \ + dat/include/dat2/udat_config.h \ + dat/include/dat2/udat.h \ + dat/include/dat2/udat_redirection.h \ + dat/include/dat2/udat_vendor_specific.h \ + dat/include/dat2/dat_ib_extensions.h man_MANS = man/dtest.1 man/dapltest.1 @@ -203,16 +203,16 @@ EXTRA_DIST = dat/common/dat_dictionary.h \ dat/common/dat_sr.h \ dat/udat/udat_sr_parser.h \ dat/udat/linux/dat_osd.h \ -dat/include/dat/dat.h \ -dat/include/dat/dat_error.h \ -dat/include/dat/dat_platform_specific.h \ -dat/include/dat/dat_redirection.h \ -dat/include/dat/dat_registry.h \ -dat/include/dat/dat_vendor_specific.h \ -dat/include/dat/udat_config.h \ -dat/include/dat/udat.h \ -dat/include/dat/udat_redirection.h \ -dat/include/dat/udat_vendor_specific.h \ +dat/include/dat2/dat.h \ +dat/include/dat2/dat_error.h \ +dat/include/dat2/dat_platform_specific.h \ +dat/include/dat2/dat_redirection.h \ +dat/include/dat2/dat_registry.h \ +dat/include/dat2/dat_vendor_specific.h \ +dat/include/dat2/udat_config.h \ +dat/include/dat2/udat.h \ +dat/include/dat2/udat_redirection.h \ +dat/include/dat2/udat_vendor_specific.h \ dapl/common/dapl_adapter_util.h \ dapl/common/dapl_cno_util.h \ dapl/common/dapl_cookie.h \ diff --git a/dapl/ibal/dapl_ibal_dto.h b/dapl/ibal/dapl_ibal_dto.h index be5686f..283fd91 100644 --- a/dapl/ibal/dapl_ibal_dto.h +++ b/dapl/ibal/dapl_ibal_dto.h @@ -52,7 +52,7 @@ #include dapl_ibal_util.h #ifdef DAT_EXTENSIONS -#include dat/dat_ib_extensions.h +#include dat2/dat_ib_extensions.h #endif extern DAT_RETURN diff --git a/dapl/include/dapl.h b/dapl/include/dapl.h index 49a3f49..42db81e 100755 --- a/dapl/include/dapl.h +++ b/dapl/include/dapl.h @@ -45,11 +45,11 @@ #define _DAPL_H_ #if defined(__KERNEL__) -#include dat/kdat.h +#include dat2/kdat.h #else -#include dat/udat.h +#include dat2/udat.h #endif /* defined(__KERNEL__) */ -#include dat/dat_registry.h +#include dat2/dat_registry.h #include dapl_osd.h #include dapl_debug.h diff --git a/dapl/openib_cma/dapl_ib_dto.h b/dapl/openib_cma/dapl_ib_dto.h index 09cef14..cea989b 100644 --- a/dapl/openib_cma/dapl_ib_dto.h +++ b/dapl/openib_cma/dapl_ib_dto.h @@ -51,7 +51,7 @@ #include dapl_ib_util.h #ifdef DAT_EXTENSIONS -#include dat/dat_ib_extensions.h +#include dat2/dat_ib_extensions.h #endif #defineDEFAULT_DS_ENTRIES 8 diff --git a/dapl/udapl/dapl_init.c b/dapl/udapl/dapl_init.c index 94a4619..ce92f9f 100644 --- a/dapl/udapl/dapl_init.c +++ b/dapl/udapl/dapl_init.c @@ -37,7 +37,7 @@ **/ #include dapl.h -#include dat/dat_registry.h /* Provider API function prototypes */ +#include dat2/dat_registry.h /* Provider API function prototypes */ #include dapl_hca_util.h #include dapl_init.h #include dapl_provider.h diff --git a/dapl/udapl/linux/dapl_osd.h b/dapl/udapl/linux/dapl_osd.h index e932ae0..caf971f 100644 --- a/dapl/udapl/linux/dapl_osd.h +++ b/dapl/udapl/linux/dapl_osd.h @@ -54,7 +54,7 @@ #endif -#include dat/udat.h +#include dat2/udat.h #include assert.h #include errno.h #include pthread.h diff --git a/dat/common/dat_api.c b/dat/common/dat_api.c index a381f05
[ewg] [ANNOUCE] dapl 2.0.6 release
New release for dapl 2.0 available on the OFA download page and in my git tree. - changes to correctly build against and target /dat2/include directory. md5sum: c0ad3d8b5df252ff5ce47f616a462f4edapl-2.0.6.tar.gz Vlad, please pull new v2 release into OFED 1.3 RC4 and install the following dapl packages: dapl-1.2.4-1 dapl-devel-1.2.4-1 dapl-2.0.6-1 dapl-utils-2.0.6-1 dapl-devel-2.0.6-1 dapl-debuginfo-2.0.6-1 See blocked::http://www.openfabrics.org/downloads/dapl/README.html http://www.openfabrics.org/downloads/dapl/README.html for details. -arlin ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
RE: [ewg] Re: [ofa-general] [ANNOUCE] dapl 2.0.5 release [PATCH] install.pl
Hi Arlin,
I am not sure that I will be available to do this till OFED-1.3-rc4.
If you can send me the patch to install.pl script I will apply this to
my git tree.
git://git.openfabrics.org/~vlad/ofed_1_3_scripts master
Change install script to include v1 dapl-devel package.
Signed-off by: Arlin Davis [EMAIL PROTECTED]
diff --git a/install.pl b/install.pl
index 9be3013..354e896 100755
--- a/install.pl
+++ b/install.pl
@@ -295,7 +295,7 @@ my @user_packages = (libibverbs, libibverbs-devel,
libibverbs-devel-static
librdmacm, librdmacm-utils, librdmacm-devel,
librdmacm-debuginfo,
libsdp, libsdp-devel, libsdp-debuginfo,
opensm, opensm-libs, opensm-devel,
opensm-debuginfo, opensm-static,
- dapl-v1, dapl-v2, dapl-devel, dapl-devel-static,
dapl-utils,
dapl-debuginfo,
+ dapl-v1, dapl-v1-devel, dapl-v2, dapl-devel,
dapl-devel-static,
dapl-utils, dapl-debuginfo,
perftest, mstflint, tvflash,
qlvnictools, sdpnetstat, srptools, rds-tools,
ibutils, infiniband-diags, qperf, qperf-debuginfo,
@@ -309,7 +309,7 @@ my @basic_user_packages = (libibverbs,
libibverbs-utils, libmthca, libmlx
my @hpc_kernel_packages = (kernel-ib, ib-bonding);
my @hpc_kernel_modules = (@basic_kernel_modules);
my @hpc_user_packages = (@basic_user_packages, librdmacm,
-librdmacm-utils, dapl-v1, dapl-v2, dapl-devel,
dapl-devel-static,
dapl-utils,
+librdmacm-utils, dapl-v1, dapl-v1-devel,
dapl-v2, dapl-devel,
dapl-devel-static, dapl-utils,
infiniband-diags, ibutils, qperf, @mpi_packages);
# all_packages is required to save ordered (following dependencies) list of
@@ -932,6 +932,13 @@ my %packages_info = (
dist_req_inst = [], ofa_req_build = [libibverbs,
libibverbs-devel, librdmacm,
librdmacm-devel],
ofa_req_inst = [libibverbs, librdmacm],
install32 = 1, exception = 0, configure_options = '' },
+'dapl-v1-devel' =
+{ name = dapl-devel, parent = dapl-v1,
+selected = 0, installed = 0, rpm_exist = 0, rpm_exist32 = 0,
+available = 1, mode = user, dist_req_build = [],
+dist_req_inst = [], ofa_req_build =
[libibverbs,libibverbs-devel, librdmacm,
librdmacm-devel],
+ofa_req_inst = [dapl-v1],
+install32 = 1, exception = 0, configure_options = '' },
'dapl-v2' =
{ name = dapl, parent = dapl-v2,
selected = 0, installed = 0, rpm_exist = 0, rpm_exist32 = 0,
@@ -3120,7 +3127,7 @@ sub install_rpm_32
my $release = $main_packages{$packages_info{$name}{'parent'}}{'release'};
if ( $name =~ m/dapl-v/ ) {
-$name = dapl;
+$name = $packages_info{$name}{'name'};
}
$package = $RPMS/$name-$version-$release.$target_cpu32.rpm;
@@ -3181,7 +3188,7 @@ sub install_rpm
if ( $name =~ m/dapl-v/ ) {
$tmp_name = $name;
-$name = dapl;
+$name = $packages_info{$name}{'name'};
}
if ($name eq $packages_info{'open-iscsi-generic'}{'name'}) {
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] [ANNOUCE] dapl 2.0.5 release
There is new release for dapl 2.0 available on the OFA download page and in my git tree. Changes to allow both v1 and v2 development packages to be installed on the same system. v2 libdat.so has been renamed to libdat2.so. md5sum: 010459e421a5c194438d58b1ccf1c6d0 dapl-2.0.5.tar.gz Vlad, please pull new v2 release into OFED 1.3 RC3 and install the following packages: Note: please make sure dapl-1.2.4-devel is added to list. dapl-1.2.4-1 dapl-devel-1.2.4-1 dapl-2.0.5-1 dapl-utils-2.0.5-1 dapl-devel-2.0.5-1 dapl-debuginfo-2.0.5-1 See BLOCKED::http://www.openfabrics.org/downloads/dapl/README.html http://www.openfabrics.org/downloads/dapl/README.html for details. -arlin ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
RE: [ewg] OFED meeting agenda for today (Jan 28) meeting
Vlad, Why is dapl v1 and v2 libraries not setup for --build32 option? (see bug 824) Are there some issues needing resolved? -arlin _ From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Tziporet Koren Sent: Monday, January 28, 2008 7:50 AM To: EWG Cc: OpenFabrics General Subject: [ewg] OFED meeting agenda for today (Jan 28) meeting This is the agenda for the meeting today: 1. OFED 1.3 readiness toward RC3 this week - all 2. Decide regarding IBM request to change IPoIB to support CM without SRQ (see http://lists.openfabrics.org/pipermail/ewg/2008-January/005507.html) 3. Review high priority bugs: 846 criticalRHEL 5 [EMAIL PROTECTED] SDP crash on RHEL5 ppc64 running netserver 859 criticalSLES 10 [EMAIL PROTECTED] Bonding configuration on Sles10 sp1 is not loaded consistently 863 criticalRHEL 4 [EMAIL PROTECTED] ib-bonding won't compile for RHEL4 U6 874 criticalOther [EMAIL PROTECTED] Intel MPI (IMB test) hangs intermittently on the qlogic HCA 760 major All [EMAIL PROTECTED] UDP performance on Rx is lower than Tx 761 major Other [EMAIL PROTECTED] Poor and jittery UDP performance at small messages 869 major SLES 10 [EMAIL PROTECTED] mstflint won't biuld on SLES10 x86 736 major Other [EMAIL PROTECTED] IBV_WC_RETRY_EXC_ERR errors with local rdma_reads 767 major Other [EMAIL PROTECTED] Non backport Kernels that don't build in genalloc cause compile errors for cxgb3 Tziporet ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ofa-general] RE: [ewg] OFED meeting agenda for today (Jan 28) meeting
Arlin Davis wrote: Vlad, Why is dapl v1 and v2 libraries not setup for --build32 option? (see bug 824) Are there some issues needing resolved? I enabled the 32-bit build for uDAPL and the build works fine but we have some issues with the RPM installation due to ordering. Does the install.pl script handle ofa_pre_inst settings when both 32bit and 64bit libraries are being installed? Install dapl-v1 RPM: Running rpm -iv /tmp/OFED-1.3-rc2/RPMS/redhat-release-5Server-5.1.0.2/dapl-1.2.3-0.84.ofed20071128.x86_64.rpm Install dapl-v2 RPM: Running rpm -iv /tmp/OFED-1.3-rc2/RPMS/redhat-release-5Server-5.1.0.2/dapl-2.0.3-0.217.ofed20080115.x86_64.rpm Install dapl-devel RPM: Running rpm -iv /tmp/OFED-1.3-rc2/RPMS/redhat-release-5Server-5.1.0.2/dapl-devel-2.0.3-0.217.ofed20080115.x86_64.r pm Running rpm -iv /tmp/OFED-1.3-rc2/RPMS/redhat-release-5Server-5.1.0.2/dapl-devel-2.0.3-0.217.ofed20080115.i686.rpm Failed to install dapl-devel RPM See /tmp/OFED.8978.logs/dapl-devel.rpminstall.log error: Failed dependencies: libdaplofa.so.2 is needed by dapl-devel-2.0.3-0.217.ofed20080115.i686 libdat.so.2 is needed by dapl-devel-2.0.3-0.217.ofed20080115.i686 I also noticed that dapl-devel is not part of HPC selection. Please add. Thanks, -arlin ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [ofa-general] how to use Intel MPI with dapl2?
Scott Weitzenkamp (sweitzen) wrote: How did you configure your servers to run Intel MPI with v2 libraries? I only installed the DAPL 2.0 libs. Did you happen to see the following message (I_MPI_DEBUG=50) before failover to sockets? I_MPI: [0] I_MPI_dat_ia_openv_wrap(): DAPL version compatibility requirement check failed; required DAPL 1.2, provided DAPL 2.0 -arlin ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [ofa-general] how to use Intel MPI with dapl2?
Scott Weitzenkamp (sweitzen) wrote: I am using Intel MPI 3.1 build 26 with OFED 1.3 beta2 on RHEL4 x86_64. Intel MPI works as before with dapl1, but I see very slow performance with dapl2. Are there extra command-line params I need to use dapl2? Intel MPI 3.1.26 does not support uDAPL v2. How did you configure your servers to run Intel MPI with v2 libraries? Did you put the ofa-v2-ib0 entry at the top of the /etc/dat.conf file or specify a v2 device via -genv I_MPI_DEVICE rdssm:ofa-v2-ib0? I didn't see a device option on your command line. Anyway, uDAPL v2 support is not there yet. $ /data/software/qa/MPI/intel_mpi/intelmpi-3.1-`uname -m`/bin/mpiexec -genv I_MPI_DEBUG 3 -n 2 osu_latency.x *[1] MPI startup(): DAPL provider NULL on rank 0:svbu-qa1850-1 differs from NUL L string on rank 1:svbu-qa1850-2 Looks like it failed over to sockets device because of errors with the RDMA device specified in dat.conf. Something looks weird with your dat.conf configuration because Intel MPI is showing NULL as the provider name (first column in dat.conf). You can set -genv I_MPI_FAILOVER disable to disable failover and immediately bail after any RDMA device error. -arlin ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [ofa-general] OFED 1.2.5 - GA release
Michael S. Tsirkin wrote: Can anyone tell me how to get ib_local_sa.ko installed with OFED 1.2.5? We cannot move to OFED 1.2.5 without SA caching options. ib_local_sa was merged with ib_sa in 1.2.5. There are no extra modules to load. Michael, thanks for the heads up. Sure would be nice if major changes like this would be mentioned somewhere in release notes. :-) -arlin ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [ofa-general] OFED 1.2.5 - GA release
How can I build/install OFED 1.2.5 with ib_local_sa.ko? It seems to build but does not install and I need SA caching options. Can anyone tell me how to get ib_local_sa.ko installed with OFED 1.2.5? We cannot move to OFED 1.2.5 without SA caching options. Thanks, -arlin ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [ofa-general] OFED 1.2.5 - GA release
Tziporet Koren wrote: I am happy to announce on OFED 1.2.5 GA release. Vlad, How can I build/install OFED 1.2.5 with ib_local_sa.ko? It seems to build but does not install and I need SA caching options. -arlin ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] OFED 1.2.5 issue with MSI ??
I just noticed when setting up MSI support that ib_mthca fails with the following: This is on 2 different Intel platforms that worked with OFED 1.2: ib_mthca: Mellanox InfiniBand HCA driver v0.08 (February 14, 2006) ib_mthca: Initializing :0a:00.0 ACPI: PCI interrupt :0a:00.0[A] - GSI 16 (level, low) - IRQ 169 PCI: Setting latency timer of device :0a:00.0 to 64 ib_mthca :0a:00.0: NOP command failed to generate interrupt (IRQ 233), aborting. ib_mthca :0a:00.0: Try again with MSI/MSI-X disabled. ib_mthca: probe of :0a:00.0 failed with error -16 lspci -v v 0a:00.0 InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor compatibility mode) (rev a0) Subsystem: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor compatibility mode) Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- Status: Cap+ 66Mhz- UDF- FastB2B- ParErr- DEVSEL=fast TAbort- TAbort- MAbort- SERR- PERR- Latency: 0, Cache Line Size 10 Interrupt: pin A routed to IRQ 169 Region 0: Memory at c130 (64-bit, non-prefetchable) [size=1M] Region 2: Memory at c000 (64-bit, prefetchable) [size=8M] Region 4: Memory at b800 (64-bit, prefetchable) [size=128M] Capabilities: [40] Power Management version 2 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-) Status: D0 PME-Enable- DSel=0 DScale=0 PME- Capabilities: [48] Vital Product Data Capabilities: [90] Message Signalled Interrupts: 64bit+ Queue=0/5 Enable- Address: 7000 Data: Capabilities: [84] MSI-X: Enable- Mask- TabSize=32 Vector table: BAR=0 offset=00082000 PBA: BAR=0 offset=00082200 Capabilities: [60] Express Endpoint IRQ 0 Device: Supported: MaxPayload 128 bytes, PhantFunc 0, ExtTag- Device: Latency L0s 64ns, L1 unlimited Device: AtnBtn- AtnInd- PwrInd- Device: Errors: Correctable+ Non-Fatal+ Fatal+ Unsupported- Device: RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- Device: MaxPayload 128 bytes, MaxReadReq 512 bytes Link: Supported Speed 2.5Gb/s, Width x8, ASPM L0s, Port 8 Link: Latency L0s unlimited, L1 unlimited Link: ASPM Disabled RCB 64 bytes CommClk- ExtSynch- Link: Speed 2.5Gb/s, Width x8 /etc/modprobe.conf options ib_mthca msi=1 Anyone else see this issue? -arlin ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] Re: [ofa-general] RE: OFA website edits
Jeff Squyres wrote: Heh -- I didn't notice these links until Sean moved them up to the top of the text. Yes, we should definitely link to the MPI project home sites; we have lots of our own information there, separate downloads, etc. ] Are these the links we want? MVAPICH - http://mvapich.cse.ohio-state.edu/ OpenMPI - http://www.open-mpi.org/ ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [ofa-general] RE: OFA website edits
I would like to propose adding project directories under http://www.openfabrics.org/downloads/ where appropriate and give maintainers access. For example: Jeff, please add the following directories with maintainer access as follow (or grant access at a maintainer group level): http://www.openfabrics.org/downloads/verbs (rdreier) http://www.openfabrics.org/downloads/rdmacm (shefty) http://www.openfabrics.org/downloads/dapl (ardavis) http://www.openfabrics.org/downloads/sdp (eitan) http://www.openfabrics.org/downloads/utils (eitan) http://www.openfabrics.org/downloads/management (sashak) http://www.openfabrics.org/downloads/OFED (vlad) http://www.openfabrics.org/downloads/archives (vlad) http://www.openfabrics.org/downloads/WinOF (ssmith) (Stan Smith will need an account) http://www.openfabrics.org/downloads/hw/mthca (rdreir) http://www.openfabrics.org/downloads/hw/mlx4 (rdreir) http://www.openfabrics.org/downloads/hw/ehca (raisch) http://www.openfabrics.org/downloads/hw/ipath (ralphc) http://www.openfabrics.org/downloads/hw/cxgb3 (ralphc) http://www.openfabrics.org/downloads/mpi/mvapich (pasha) http://www.openfabrics.org/downloads/mpi/mvapich2 (rowland) http://www.openfabrics.org/downloads/mpi/openmpi (jsquyres) Let us know when these directories are created and the maintainers, who want to expose their packages via the webpage, will create a README that details the contents of the directory along with WEB_README that provides a short description for the webpage. Will this format allow you to auto configure the download webpage sufficiently? The idea is to only add links/descriptions to those project sub-directories with WEB_README files present. Please advise if something on the list is wrong or we missed a project. Thanks, -arlin ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
