Create bond on Infiniband ports

2013-05-28 Thread Alex Liptsin
Hi.

I use FreeBSD 9.1 with OFED compiled on it.

There is a Mellanox adapter:

[root@qa-h-vrt-030-006 ~]# pciconf -lv |grep mlx4 -A 3
mlx4_core0@pci0:0:5:0:  class=0x028000 card=0x005015b3 chip=0x100315b3 rev=0x00 
hdr=0x00
vendor = 'Mellanox Technologies'
device = 'MT27500 Family [ConnectX-3]'
class  = network

I want to create Bond on the two ports (ib0 and ib1) of this device:

[root@qa-h-vrt-030-006 ~]# ifconfig
em0: flags=8843 metric 0 mtu 1500

options=209b
ether 00:50:56:23:1e:06
inet6 fe80::250:56ff:fe23:1e06%em0 prefixlen 64 scopeid 0x2
inet 10.195.30.6 netmask 0x broadcast 10.195.255.255
nd6 options=23
media: Ethernet autoselect (1000baseT )
status: active
lo0: flags=8049 metric 0 mtu 16384
options=63
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
inet 127.0.0.1 netmask 0xff00
nd6 options=23
ib0: flags=8043 metric 0 mtu 65520
options=80018
lladdr 80.0.0.48.fe.80.0.0.0.0.0.0.0.2.c9.0.1.0.d0.51
inet 11.195.30.1 netmask 0x broadcast 11.195.255.255
inet6 fe80::250:56ff:fe23:1e06%ib0 prefixlen 64 scopeid 0x4
nd6 options=23
ib1: flags=8043 metric 0 mtu 65520
options=80018
lladdr 80.0.0.49.fe.80.0.0.0.0.0.0.0.2.c9.0.1.0.d0.52
inet 12.195.30.1 netmask 0x broadcast 12.195.255.255
inet6 fe80::250:56ff:fe23:1e06%ib1 prefixlen 64 scopeid 0x5
nd6 options=23


1.  Is there "bond" in FreeBSD or its "Lagg"?

2. I tried to create this way, but failed:



[root@qa-h-vrt-030-006 ~]# ifconfig lagg0 create

root@qa-h-vrt-031-005 conf]# ifconfig lagg0 laggproto failover laggport ib0 
laggport ib1

ifconfig: SIOCSLAGGPORT: Protocol not supported

Any ideas?
Is it supported on Infiniband ports?


Regards,
Alex Liptsin


___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"


Re: Create bond on Infiniband ports

2013-05-28 Thread Garrett Cooper
On Tue, May 28, 2013 at 4:35 AM, Alex Liptsin  wrote:

> Hi.
>
> I use FreeBSD 9.1 with OFED compiled on it.
>
> There is a Mellanox adapter:
>
> [root@qa-h-vrt-030-006 ~]# pciconf -lv |grep mlx4 -A 3
> mlx4_core0@pci0:0:5:0:  class=0x028000 card=0x005015b3 chip=0x100315b3
> rev=0x00 hdr=0x00
> vendor = 'Mellanox Technologies'
> device = 'MT27500 Family [ConnectX-3]'
> class  = network
>
> I want to create Bond on the two ports (ib0 and ib1) of this device:
>
> [root@qa-h-vrt-030-006 ~]# ifconfig
> em0: flags=8843 metric 0 mtu 1500
>
> options=209b
> ether 00:50:56:23:1e:06
> inet6 fe80::250:56ff:fe23:1e06%em0 prefixlen 64 scopeid 0x2
> inet 10.195.30.6 netmask 0x broadcast 10.195.255.255
> nd6 options=23
> media: Ethernet autoselect (1000baseT )
> status: active
> lo0: flags=8049 metric 0 mtu 16384
> options=63
> inet6 ::1 prefixlen 128
> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
> inet 127.0.0.1 netmask 0xff00
> nd6 options=23
> ib0: flags=8043 metric 0 mtu 65520
> options=80018
> lladdr 80.0.0.48.fe.80.0.0.0.0.0.0.0.2.c9.0.1.0.d0.51
> inet 11.195.30.1 netmask 0x broadcast 11.195.255.255
> inet6 fe80::250:56ff:fe23:1e06%ib0 prefixlen 64 scopeid 0x4
> nd6 options=23
> ib1: flags=8043 metric 0 mtu 65520
> options=80018
> lladdr 80.0.0.49.fe.80.0.0.0.0.0.0.0.2.c9.0.1.0.d0.52
> inet 12.195.30.1 netmask 0x broadcast 12.195.255.255
> inet6 fe80::250:56ff:fe23:1e06%ib1 prefixlen 64 scopeid 0x5
> nd6 options=23
>
>
> 1.  Is there "bond" in FreeBSD or its "Lagg"?
>

It's lagg. Setting up bonded interfaces is different (Anthony C. or someone
else can provide more details about this). failover mode via lagg is
completely different from failover mode with IB IIRC.


> 2. I tried to create this way, but failed:
>
> [root@qa-h-vrt-030-006 ~]# ifconfig lagg0 create
>
> root@qa-h-vrt-031-005 conf]# ifconfig lagg0 laggproto failover laggport
> ib0 laggport ib1
>
> ifconfig: SIOCSLAGGPORT: Protocol not supported
>
> Any ideas?
> Is it supported on Infiniband ports?
>

The media setting pieces are look to only be supported with mlx4 in en mode:

1076 static int mlx4_en_ioctl(struct ifnet *dev, u_long command, caddr_t
data)
1077 {
...
1115 case SIOCGIFMEDIA:
1116 error = ifmedia_ioctl(dev, ifr, &priv->media, command);
1117 break;

If I remember correctly, IB ports with mlx4 default to ib mode, not en mode
(it also helps to have the right drivers loaded for this).

Cheers,
-Garrett
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"