Re: [OMPI users] mlx4 error - looking for guidance

2009-03-05 Thread Pavel Shamis (Pasha)
The fw version 2.3.0 is too old. I recommend you to upgrade to the latest version (2.6.0) from Mellanox website http://www.mellanox.com/content/pages.php?pg=firmware_table_ConnectXIB Thanks, Pasha Jeff Layton wrote: Oops. I ran it on the head node and not the compute node. Here is the output

Re: [OMPI users] mlx4 error - looking for guidance

2009-03-05 Thread Jeff Layton
Oops. I ran it on the head node and not the compute node. Here is the output from a compute node: hca_id: mlx4_0 fw_ver: 2.3.000 node_guid: 0018:8b90:97fe:1b6d sys_image_guid: 0018:8b90:97fe:1b70 vendor_id:

Re: [OMPI users] mlx4 error - looking for guidance

2009-03-05 Thread Pavel Shamis (Pasha)
Do you have the same HCA adapter type on all of your machines ? In the error log I see mlx4 error message , and mlx4 is connectX driver, but ibv_devinfo show some older hca. Pasha Jeff Layton wrote: Pasha, Here you go... :) Thanks for looking at this. Jeff hca_id: mthca0 fw_ver:

Re: [OMPI users] mlx4 error - looking for guidance

2009-03-05 Thread Jeff Layton
Pasha, Here you go... :) Thanks for looking at this. Jeff hca_id: mthca0 fw_ver: 4.8.200 node_guid: 0003:ba00:0100:38ac sys_image_guid: 0003:ba00:0100:38af vendor_id: 0x02c9 vend

Re: [OMPI users] mlx4 error - looking for guidance

2009-03-05 Thread Pavel Shamis (Pasha)
Jeff, Can you please provide more information about you HCA type (ibv_devinfo -v). Do you see this error immediate during startup, or you get it during your run ? Thanks, Pasha Jeff Layton wrote: Evening everyone, I'm running a CFD code on IB and I've encountered an error I'm not sure about

[OMPI users] mlx4 error - looking for guidance

2009-03-04 Thread Jeff Layton
Evening everyone, I'm running a CFD code on IB and I've encountered an error I'm not sure about and I'm looking for some guidance on where to start looking. Here's the error: mlx4: local QP operation err (QPN 260092, WQE index 9a9e, vendor syndrome 6f, opcode = 5e) [0,1,6][btl_openib_compon