Hello, Thanks for the answer! Yes, the problem was the memory locking for the users. The root does not have this problem.
Best regards, Boyan On Sunday 28 June 2009 17:16:32 Tziporet Koren wrote: > Boyan Lazarov wrote: > > Hello, > > > > We have installed OFED 1.4.1 on Scientific Linux 53 (RH derivative). > > When I run the osu_* tests (OpenMPI) as root it seems that everything is > > running fine and the mpi is using openib for communications. However if > > I run the test as an ordinary user I am getting the following: > > > > WARNING: There was an error initializing an OpenFabrics device. > > > > and openmpi is using tcp for communications - the speed is much lower. > > > > Anyone with similar problem and any suggestions? > > Can you send output of /var/log/messages? > Also it could be that you have this problem: > 2. Memory registration by the user is limited according to administrator > setting. See "Pinning (Locking) User Memory Pages" in OFED_tips.txt for > system configuration. > > =========================================================================== >==== 3. Pinning (Locking) User Memory Pages > =========================================================================== >==== > > Memory locking is managed by the kernel on a per user basis. Regular > users (as > opposed to root) have a limited number of pages which they may pin, where > the limit is pre-set by the administrator. Registering memory for IB verbs > requires pinning memory, thus an application cannot register more memory > than > it is allowed to pin. > > The user can change the system per-process memory lock limit by adding > the following two lines to the file /etc/security/limits.conf: > > * soft memlock <number> > * hard memlock <number> > > where <number> denotes the number of KBytes that may be locked by a > user process. > > The above change to /etc/security/limits.conf will allow any user > process in the > system to lock up to <number> KBytes of memory. > > On some systems, it may be possible to use "unlimited" for the size to > disable > these limits entirely. > > Note: The file /etc/security/limits.conf contains further documentation. > > Tziporet _______________________________________________ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
