Re: [Lustre-discuss] Rx failures

2010-02-11 Thread Bernd Schubert
On Thursday 11 February 2010, Ulrich Sibiller wrote: > Ulrich Sibiller schrieb: > > Feb 10 13:33:24 hpc9master02 kernel: LustreError: > > 4475:0:(lib-move.c:2436:LNetPut()) Error sending PUT to > > 12345-192.168.60@o2ib: -113 > > > > Feb 2 16:08:19 hpc9oss1 kernel: Lustre: > > 7937:0:(o2iblnd_

Re: [Lustre-discuss] Lustre 1.8.1 QDR Support

2010-02-11 Thread Jagga Soorma
Yet more information. Looks like the switch thinks that this could be set to 10Gbps (QDR): hpc116:/mnt/SLES11x86_64 # iblinkinfo.pl -R | grep -i reshpc116 1 34[ ] ==( 4X 5.0 Gbps Active / LinkUp)==> 201[ ] "hpc116 HCA-1" ( Could be 10.0 Gbps) -J On Thu, Feb 11, 2010 at 1:

Re: [Lustre-discuss] Lustre 1.8.1 QDR Support

2010-02-11 Thread Jagga Soorma
More information: hpc116:/mnt/SLES11x86_64 # lspci | grep -i mellanox 10:00.0 InfiniBand: Mellanox Technologies MT26428 [ConnectX IB QDR, PCIe 2.0 5GT/s] (rev a0) hpc116:/mnt/SLES11x86_64 # ibstatus Infiniband device 'mlx4_0' port 1 status: default gid: fe80::::0002:c903:0006:

[Lustre-discuss] Lustre 1.8.1 QDR Support

2010-02-11 Thread Jagga Soorma
Hi Guys, Wanted to give a bit more information. So for some reason the transfer rates on my ib interfaces are autonegotiating at 20Gb/s (4X DDR). However, these are QDR HCA's. Here is the hardware that I have: HP IB 4X QDR PCI-e G2 Dual Port HCA HP 3M 4X DDR/QDR QSFP IB Cu Cables Qlogic 12200

Re: [Lustre-discuss] lctl ping between 1.8.1 1.8.2 protocol error

2010-02-11 Thread Isaac Huang
On Thu, Feb 11, 2010 at 03:33:33PM +0100, Sebastian Reitenbach wrote: > Hi, > > in my test system I installed Lustre 1.8.2 from source on a opensuse 10.2 > i386 > (2.6.18.8-0.13-xenpaelustre) as a client. Other clients and the servers are > running 1.8.1 on SLES 11 x86_64 (2.6.27.39-0.3-xen-lus

[Lustre-discuss] Another Infiniband Question

2010-02-11 Thread Jagga Soorma
I have a QDR ib switch that should support up to 40Gbps. After installing the kernel-ib and lustre client rpms on my SuSe nodes I see the following: hpc102:~ # ibstatus mlx4_0:1 Infiniband device 'mlx4_0' port 1 status: default gid: fe80::::0002:c903:0006:de19 base lid:

[Lustre-discuss] lctl ping between 1.8.1 1.8.2 protocol error

2010-02-11 Thread Sebastian Reitenbach
Hi, in my test system I installed Lustre 1.8.2 from source on a opensuse 10.2 i386 (2.6.18.8-0.13-xenpaelustre) as a client. Other clients and the servers are running 1.8.1 on SLES 11 x86_64 (2.6.27.39-0.3-xen-lustre) lctl ping to and from the client ends with a protocol error. The 1.8.2 client

Re: [Lustre-discuss] Infiniband VS 1GiG Transfer rates. Confused

2010-02-11 Thread Peter Kjellstrom
On Thursday 11 February 2010, Erik Froese wrote: > Jagga, > I think this is more a function of scp than than ib in general. Have > you tried using the HPN-SSH patches? > http://www.psc.edu/networking/projects/hpn-ssh/ > > You could also try using apache to serve the iso over HTTP to see if > SCP is

Re: [Lustre-discuss] Rx failures

2010-02-11 Thread Ulrich Sibiller
Ulrich Sibiller schrieb: > Feb 10 13:33:24 hpc9master02 kernel: LustreError: > 4475:0:(lib-move.c:2436:LNetPut()) Error sending > PUT to 12345-192.168.60@o2ib: -113 > Feb 2 16:08:19 hpc9oss1 kernel: Lustre: > 7937:0:(o2iblnd_cb.c:2220:kiblnd_passive_connect()) Conn > stale 192.168.60@o