I have found that the 400EX is a SDR card, while the 400EX-D is a DDR card. I don't know if is a firmware problem.
http://sup.xenya.si/sup/info/voltaire/HCAwebfinal.pdf HCA 400Ex/Ex-F • Dual port 4X (10 Gbps) InfiniBand PCI-Express low profile host channel adapter HCA 400Ex-D • Dual port 4X DDR (20 Gbps) InfiniBand PCI-Express low profile host channel adapter HCA 400 • Dual port 4X (10 Gbps) InfiniBand PCI/PCI-X low profile host channel adapter ----- Messaggio originale ----- Da: "Michael Rasmussen" <m...@miras.org> A: "Fabrizio Cuseo" <f.cu...@panservice.it>, "pve-user" <pve-user@pve.proxmox.com> Inviato: Martedì, 7 luglio 2015 12:01:03 Oggetto: Re: [PVE-User] Infiniband Mellanox cards and DDR AFAIK this card is a SDR card so DDR is not possible. On July 7, 2015 11:46:41 AM CEST, Fabrizio Cuseo <f.cu...@panservice.it> wrote: Hallo. I am testing a Mellanox card (IBM Voltaire HCA 400Ex-D) with a cisco 4x DDR infiniband switch. The problem I have is that the cards (that seems to be 4X DDR) have a 10Gbit (SDR) link. --- 03:00.0 InfiniBand: Mellanox Technologies MT25208 [InfiniHost III Ex] (rev 20) --- root@pve1:/etc/apt/sources.list.d# ibstat CA 'mthca0' CA type: MT25208 Number of ports: 2 Firmware version: 5.3.0 Hardware version: 20 Node GUID: 0x0008f104039814d0 System image GUID: 0x0008f104039814d3 Port 1: State: Active Physical state: LinkUp Rate: 10 Base lid: 6 LMC: 0 SM lid: 2 Capability mask: 0x02510a68 Port GUID: 0x0008f104039814d1 Link laye r: InfiniBand -------- root@pve1:/etc/apt/sources.list.d# ibportstate 6 0 CA PortInfo: # Port info: Lid 6 port 0 LinkState:.......................Active PhysLinkState:...................LinkUp Lid:.............................6 SMLid:...........................2 LMC:.............................0 LinkWidthSupported:..............1X or 4X LinkWidthEnabled:................1X or 4X LinkWidthActive:.................4X LinkSpeedSupported:..............2.5 Gbps LinkSpeedEnabled:................2.5 Gbps LinkSpeedActive:.................2.5 Gbps Mkey:............................<not displayed> MkeyLeasePeriod:.................15 ProtectBits:.....................0 ----- If I try to set the speed at DDR, I have this error: root@pve1:/etc/apt/sources.list.d# ibportstate 6 0 speed 2 Initial CA PortInfo: # Port info: Lid 6 port 0 LinkState:.......................Active PhysLinkState:...................LinkUp Lid:.............................6 SMLid:...........................2 LMC:.............................0 LinkWidthSupported:..............1X or 4X LinkWidthEnabled:................1X or 4X LinkWidthActive:.................4X LinkSpeedSupported:..............2.5 Gbps LinkSpeedEnabled:................2.5 Gbps LinkSpeedActive:.................2.5 Gbps Mkey:............................<not displayed> MkeyLeasePeriod:.................15 ProtectBits:.....................0 ibportstate: iberror: failed: smp set portinfo failed ---- The switch reported this card in infiniband topology: MT25218 InfiniHostEx Mellanox Technologies Someone have tested the cards ? Fabrizio Cuseo -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. ---- This mail was virus scanned and spam checked before delivery. This mail is also DKIM signed. See header dkim-signature. -- --- Fabrizio Cuseo - mailto:f.cu...@panservice.it Direzione Generale - Panservice InterNetWorking Servizi Professionali per Internet ed il Networking Panservice e' associata AIIP - RIPE Local Registry Phone: +39 0773 410020 - Fax: +39 0773 470219 http://www.panservice.it mailto:i...@panservice.it Numero verde nazionale: 800 901492 _______________________________________________ pve-user mailing list pve-user@pve.proxmox.com http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user