[ewg] strange value of IB port rate

2013-02-20 Thread Mahesh Chaudhari 
Hi All,
i have Mellanox dual port IB card installed in m/c
when i do :

 "cat /sys/class/infiniband/mthca0/ports/1/rate "
it shows 20 Gb/sec (4X DDR)

while on port2 

                " cat /sys/class/infiniband/mthca0/ports/2/rate "
it shows 8.5 Gb/sec (4X)

when i looked at source code, i found a equation to calculate rate :

                       rate = 25 * ib_width_enum_to_int(attr.active_width) * 
attr.active_speed;
                       

                       return sprintf(buf, "%d%s Gb/sec (%dX%s)\n",
   rate / 10, rate % 10 ? ".5" : "",
   ib_width_enum_to_int(attr.active_width), speed);



                                where   ib_width_enum_to_int(attr.active_width) 
=  1 | 4 |8 |12 | -1(error)
                                         attr.active_speed = 1 | 2 | 4 


    i am wondering, how could it possible to get such an odd value (8.5 Gbps) 


/usr/ofed/sbin/ibstatus utility also shows :


Infiniband device 'mthca0' port 1 status:
    default gid:     fe80::::001a:4bff:ff0c:96e5
    base lid:     0x6
    sm lid:         0x1
    state:         4: ACTIVE
    phys state:     5: LinkUp
    rate:         20 Gb/sec (4X DDR)
    link_layer:     InfiniBand

Infiniband device 'mthca0' port 2 status:
    default gid:     fe80::::001a:4bff:ff0c:96e6
    base lid:     0x0
    sm lid:         0x0
    state:         1: DOWN
    phys state:     2: Polling
    rate:         8.5 Gb/sec (4X)
    link_layer:     InfiniBand





Any Clue ???
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

Re: [ewg] strange value of IB port rate

2013-02-20 Thread Hal Rosenstock 
On 2/20/2013 6:51 AM, Mahesh Chaudhari wrote:
> Hi All,
> i have Mellanox dual port IB card installed in m/c
> when i do :
>  "cat /sys/class/infiniband/mthca0/ports/1/rate "
> it shows 20 Gb/sec (4X DDR)
> 
> while on port2
> " cat /sys/class/infiniband/mthca0/ports/2/rate "
> it shows 8.5 Gb/sec (4X)
> 
> when i looked at source code, i found a equation to calculate rate :
> 
>rate = 25 *
> ib_width_enum_to_int(attr.active_width) * attr.active_speed;
>   
>return sprintf(buf, "%d%s Gb/sec (%dX%s)\n",
>rate / 10, rate % 10 ? ".5" : "",
>ib_width_enum_to_int(attr.active_width), speed);
> 
> 
> where  
> ib_width_enum_to_int(attr.active_width) =  1 | 4 |8 |12 | -1(error)
>  attr.active_speed = 1 | 2 | 4 
> 
> 
> i am wondering, how could it possible to get such an odd value (8.5
> Gbps)
> 
> /usr/ofed/sbin/ibstatus utility also shows :
> 
> Infiniband device 'mthca0' port 1 status:
> default gid: fe80::::001a:4bff:ff0c:96e5
> base lid: 0x6
> sm lid: 0x1
> state: 4: ACTIVE
> phys state: 5: LinkUp
> rate: 20 Gb/sec (4X DDR)
> link_layer: InfiniBand
> 
> Infiniband device 'mthca0' port 2 status:
> default gid: fe80::::001a:4bff:ff0c:96e6
> base lid: 0x0
> sm lid: 0x0
> state: 1: DOWN
> phys state: 2: Polling
> rate: 8.5 Gb/sec (4X)
> link_layer: InfiniBand
> 
> 
> 
> 
> Any Clue ???

In the case of DOWN ports, rate is meaningless and should be ignored.

-- Hal

> 
> 
> ___
> ewg mailing list
> ewg@lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

Re: [ewg] strange value of IB port rate

2013-02-20 Thread Or Gerlitz 

On 20/02/2013 13:54, Hal Rosenstock wrote:

In the case of DOWN ports, rate is meaningless and should be ignored.
should be fixed by 0559d8dc13a1cd68b5e64c0b61659f36c7b5c89f "IB/core: 
Don't return EINVAL from sysfs rate attribute for invalid speeds"

___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

Re: [ewg] strange value of IB port rate

2013-02-20 Thread Mahesh Chaudhari 
We have a cluster of 16 nodes. And rest all 15 nodes shows the same value i.e. 
10 Gb/sec (4X) except this node 




 From: Hal Rosenstock 
To: Mahesh Chaudhari  
Cc: "ewg@lists.openfabrics.org"  
Sent: Wednesday, 20 February 2013 5:24 PM
Subject: Re: [ewg] strange value of IB port rate
 
On 2/20/2013 6:51 AM, Mahesh Chaudhari wrote:
> Hi All,
> i have Mellanox dual port IB card installed in m/c
> when i do :
>              "cat /sys/class/infiniband/mthca0/ports/1/rate "
> it shows 20 Gb/sec (4X DDR)
> 
> while on port2
>                 " cat /sys/class/infiniband/mthca0/ports/2/rate "
> it shows 8.5 Gb/sec (4X)
> 
> when i looked at source code, i found a equation to calculate rate :
> 
>                        rate = 25 *
> ib_width_enum_to_int(attr.active_width) * attr.active_speed;
>                      
>                        return sprintf(buf, "%d%s Gb/sec (%dX%s)\n",
>                        rate / 10, rate % 10 ? ".5" : "",
>                        ib_width_enum_to_int(attr.active_width), speed);
> 
> 
>                                 where  
> ib_width_enum_to_int(attr.active_width) =  1 | 4 |8 |12 | -1(error)
>                                              attr.active_speed = 1 | 2 | 4 
> 
> 
>     i am wondering, how could it possible to get such an odd value (8.5
> Gbps)
> 
> /usr/ofed/sbin/ibstatus utility also shows :
> 
> Infiniband device 'mthca0' port 1 status:
>     default gid:     fe80::::001a:4bff:ff0c:96e5
>     base lid:     0x6
>     sm lid:         0x1
>     state:         4: ACTIVE
>     phys state:     5: LinkUp
>     rate:         20 Gb/sec (4X DDR)
>     link_layer:     InfiniBand
> 
> Infiniband device 'mthca0' port 2 status:
>     default gid:     fe80::::001a:4bff:ff0c:96e6
>     base lid:     0x0
>     sm lid:         0x0
>     state:         1: DOWN
>     phys state:     2: Polling
>     rate:         8.5 Gb/sec (4X)
>     link_layer:     InfiniBand
> 
> 
> 
> 
> Any Clue ???

In the case of DOWN ports, rate is meaningless and should be ignored.

-- Hal

> 
> 
> ___
> ewg mailing list
> ewg@lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg

Re: [ewg] strange value of IB port rate

2013-02-20 Thread Hal Rosenstock 
On 2/20/2013 7:41 AM, Mahesh Chaudhari wrote:
> We have a cluster of 16 nodes. And rest all 15 nodes shows the same
> value i.e. 10 Gb/sec (4X) except this node

What's different about this node ? Same HCAs ? Same firmware ? Same
kernel ? Same user space packages ?

IB spec says:
C14-24.2.1: If PortInfo:Portstate=Down, then
• a SubnGet(PortInfo) shall produce valid data for PortInfo:PortState
and PortInfo:PortPhysicalState; whether any other component has
valid data is vendor-dependent

It relies on those vendor dependent values. Some additional checking
could be introduced into those utilities to avoid this confusion but
they're just largely pretty printers for the returned information.

-- Hal

> 
> 
> *From:* Hal Rosenstock 
> *To:* Mahesh Chaudhari 
> *Cc:* "ewg@lists.openfabrics.org" 
> *Sent:* Wednesday, 20 February 2013 5:24 PM
> *Subject:* Re: [ewg] strange value of IB port rate
> 
> On 2/20/2013 6:51 AM, Mahesh Chaudhari wrote:
>> Hi All,
>> i have Mellanox dual port IB card installed in m/c
>> when i do :
>>  "cat /sys/class/infiniband/mthca0/ports/1/rate "
>> it shows 20 Gb/sec (4X DDR)
>>
>> while on port2
>>" cat /sys/class/infiniband/mthca0/ports/2/rate "
>> it shows 8.5 Gb/sec (4X)
>>
>> when i looked at source code, i found a equation to calculate rate :
>>
>>rate = 25 *
>> ib_width_enum_to_int(attr.active_width) * attr.active_speed;
>> 
>>return sprintf(buf, "%d%s Gb/sec (%dX%s)\n",
>>rate / 10, rate % 10 ? ".5" : "",
>>ib_width_enum_to_int(attr.active_width), speed);
>>
>>
>>where 
>> ib_width_enum_to_int(attr.active_width) =  1 | 4 |8 |12 | -1(error)
>>  attr.active_speed = 1 | 2
> | 4
>>
>>
>>i am wondering, how could it possible to get such an odd value (8.5
>> Gbps)
>>
>> /usr/ofed/sbin/ibstatus utility also shows :
>>
>> Infiniband device 'mthca0' port 1 status:
>>default gid:fe80::::001a:4bff:ff0c:96e5
>>base lid:0x6
>>sm lid:0x1
>>state:4: ACTIVE
>>phys state:5: LinkUp
>>rate:20 Gb/sec (4X DDR)
>>link_layer:InfiniBand
>>
>> Infiniband device 'mthca0' port 2 status:
>>default gid:fe80::::001a:4bff:ff0c:96e6
>>base lid:0x0
>>sm lid:0x0
>>state:1: DOWN
>>phys state:2: Polling
>>rate:8.5 Gb/sec (4X)
>>link_layer:InfiniBand
>>
>>
>>
>>
>> Any Clue ???
> 
> In the case of DOWN ports, rate is meaningless and should be ignored.
> 
> -- Hal
> 
>>
>>
>> ___
>> ewg mailing list
>> ewg@lists.openfabrics.org 
>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
> 
> 
> 

___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg