Adarsh, Sorry for the delay. I'm not an LACP protocol expert, but looking at your logs I don't see ql1 on either node receiving a lacpdu response. Are you certain that link is working?
-Andrew On Jul 10, 2012, at 1:53 PM, Adarsh Joshi wrote: > Andrew, > > Here are the logs with LACP_DEBUG defined in ieee802.3ad_lacp.c, > > after typing > > Ifconfig lagg0 create > ifconfig lagg0 laggproto lacp laggport ql0 laggport ql1 192.168.100.1 netmask > 255.255.255.0 > > I compiled it as a standalone driver by the way. > > System 1: > > # ifconfig -v lagg0 > lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 > options=13b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,TSO4> > ether 00:0e:1e:08:05:20 > inet 192.168.100.1 netmask 0xffffff00 broadcast 192.168.100.255 > nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> > media: Ethernet autoselect > status: active > groups: lagg > laggproto lacp > lag id: [(8000,00-0E-1E-08-05-20,01D3,0000,0000), > (8000,00-0E-1E-04-2C-F0,0213,0000,0000)] > laggport: ql1 flags=18<COLLECTING,DISTRIBUTING> state=7D > [(8000,00-0E-1E-08-05-20,01D3,8000,000D), > (FFFF,00-00-00-00-00-00,0000,FFFF,0000)] > laggport: ql0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> state=3D > [(8000,00-0E-1E-08-05-20,01D3,8000,000C), > (8000,00-0E-1E-04-2C-F0,0213,8000,000E)] > > > System 2: > > # ifconfig -v lagg0 > lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 > options=13b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,TSO4> > ether 00:0e:1e:04:2c:f0 > inet 192.168.100.2 netmask 0xffffff00 broadcast 192.168.100.255 > nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> > media: Ethernet autoselect > status: active > groups: lagg > laggproto lacp > lag id: [(8000,00-0E-1E-04-2C-F0,0213,0000,0000), > (FFFF,00-00-00-00-00-00,0000,0000,0000)] > laggport: ql1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> state=7D > [(8000,00-0E-1E-04-2C-F0,0213,8000,000F), > (FFFF,00-00-00-00-00-00,0000,FFFF,0000)] > laggport: ql0 flags=18<COLLECTING,DISTRIBUTING> state=3D > [(8000,00-0E-1E-04-2C-F0,0213,8000,000E), > (8000,00-0E-1E-08-05-20,01D3,8000,000C)] > > > System 1 logs : > > Jul 10 10:38:49 bsd-14 kernel: lacp_attach[738] : lacp attached > Jul 10 10:38:49 bsd-14 kernel: lacp_attach[740] : lacp_defined > Jul 10 10:38:49 bsd-14 kernel: lagg0: link state changed to UP > Jul 10 10:38:49 bsd-14 kernel: ql0: media changed 0x0 -> 0x100033, ether = 1, > fdx = 1, link = 1 > Jul 10 10:38:49 bsd-14 kernel: ql0: -> UNSELECTED > Jul 10 10:38:49 bsd-14 kernel: ql1: media changed 0x0 -> 0x100033, ether = 1, > fdx = 1, link = 1 > Jul 10 10:38:49 bsd-14 kernel: ql1: -> UNSELECTED > Jul 10 10:38:49 bsd-14 kernel: lacp_select_tx_port: no active aggregator > Jul 10 10:38:50 bsd-14 kernel: ql1: port > lagid=[(8000,00-0E-1E-08-05-20,01D3,8000,000D),(0000,00-00-00-00-00-00,0000,0000,0000)] > Jul 10 10:38:50 bsd-14 kernel: ql1: aggregator created > Jul 10 10:38:50 bsd-14 kernel: ql1: aggregator > lagid=[(8000,00-0E-1E-08-05-20,01D3,0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)] > Jul 10 10:38:50 bsd-14 kernel: ql1: mux_state 0 -> 1 > Jul 10 10:38:50 bsd-14 kernel: ql0: port > lagid=[(8000,00-0E-1E-08-05-20,01D3,8000,000C),(0000,00-00-00-00-00-00,0000,0000,0000)] > Jul 10 10:38:50 bsd-14 kernel: ql0: aggregator created > Jul 10 10:38:50 bsd-14 kernel: ql0: aggregator > lagid=[(8000,00-0E-1E-08-05-20,01D3,0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)] > Jul 10 10:38:50 bsd-14 kernel: ql0: mux_state 0 -> 1 > Jul 10 10:38:51 bsd-14 kernel: ql1: lacpdu transmit > Jul 10 10:38:51 bsd-14 kernel: actor=(8000,00-0E-1E-08-05-20,01D3,8000,000D) > Jul 10 10:38:51 bsd-14 kernel: actor.state=85<ACTIVITY,AGGREGATION,EXPIRED> > Jul 10 10:38:51 bsd-14 kernel: partner=(0000,00-00-00-00-00-00,0000,0000,0000) > Jul 10 10:38:51 bsd-14 kernel: partner.state=2<TIMEOUT> > Jul 10 10:38:51 bsd-14 kernel: maxdelay=0 > Jul 10 10:38:51 bsd-14 kernel: ql0: lacpdu transmit > Jul 10 10:38:51 bsd-14 kernel: actor=(8000,00-0E-1E-08-05-20,01D3,8000,000C) > Jul 10 10:38:51 bsd-14 kernel: actor.state=85<ACTIVITY,AGGREGATION,EXPIRED> > Jul 10 10:38:51 bsd-14 kernel: partner=(0000,00-00-00-00-00-00,0000,0000,0000) > Jul 10 10:38:51 bsd-14 kernel: partner.state=2<TIMEOUT> > Jul 10 10:38:51 bsd-14 kernel: maxdelay=0 > Jul 10 10:38:51 bsd-14 kernel: ql0: lacpdu receive > Jul 10 10:38:51 bsd-14 kernel: actor=(8000,00-0E-1E-04-2C-F0,0213,8000,000E) > Jul 10 10:38:51 bsd-14 kernel: actor.state=5<ACTIVITY,AGGREGATION> > Jul 10 10:38:51 bsd-14 kernel: partner=(8000,00-0E-1E-08-05-20,01D3,8000,000C) > Jul 10 10:38:51 bsd-14 kernel: partner.state=85<ACTIVITY,AGGREGATION,EXPIRED> > Jul 10 10:38:51 bsd-14 kernel: maxdelay=0 > Jul 10 10:38:51 bsd-14 kernel: ql0: old pstate 2<TIMEOUT> > Jul 10 10:38:51 bsd-14 kernel: ql0: new pstate 5<ACTIVITY,AGGREGATION> > Jul 10 10:38:51 bsd-14 kernel: ql0: partner timeout changed > Jul 10 10:38:51 bsd-14 kernel: ql0: lacpdu receive > Jul 10 10:38:51 bsd-14 kernel: actor=(8000,00-0E-1E-04-2C-F0,0213,8000,000E) > Jul 10 10:38:51 bsd-14 kernel: actor.state=5<ACTIVITY,AGGREGATION> > Jul 10 10:38:51 bsd-14 kernel: partner=(8000,00-0E-1E-08-05-20,01D3,8000,000C) > Jul 10 10:38:51 bsd-14 kernel: partner.state=85<ACTIVITY,AGGREGATION,EXPIRED> > Jul 10 10:38:51 bsd-14 kernel: maxdelay=0 > Jul 10 10:38:52 bsd-14 kernel: ql1: lacp_sm_rx_timer: EXPIRED -> DEFAULTED > Jul 10 10:38:52 bsd-14 kernel: ql1: partner timeout changed > Jul 10 10:38:52 bsd-14 kernel: ql1: lacp_sm_mux_timer: aggregator > [(8000,00-0E-1E-08-05-20,01D3,0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)], > pending 1 -> 0 > Jul 10 10:38:52 bsd-14 kernel: ql1: collecting disabled > Jul 10 10:38:52 bsd-14 kernel: lacp_aggregator_delref: > lagid=[(8000,00-0E-1E-08-05-20,01D3,0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)], > refcnt 1 -> 0 > Jul 10 10:38:52 bsd-14 kernel: ql1: mux_state 1 -> 0 > Jul 10 10:38:52 bsd-14 kernel: ql1: lacpdu transmit > Jul 10 10:38:52 bsd-14 kernel: actor=(8000,00-0E-1E-08-05-20,01D3,8000,000D) > Jul 10 10:38:52 bsd-14 kernel: actor.state=45<ACTIVITY,AGGREGATION,DEFAULTED> > Jul 10 10:38:52 bsd-14 kernel: partner=(FFFF,00-00-00-00-00-00,0000,FFFF,0000) > Jul 10 10:38:52 bsd-14 kernel: > partner.state=3c<AGGREGATION,SYNC,COLLECTING,DISTRIBUTING> > Jul 10 10:38:52 bsd-14 kernel: maxdelay=0 > Jul 10 10:38:52 bsd-14 kernel: ql0: lacp_sm_mux_timer: aggregator > [(8000,00-0E-1E-08-05-20,01D3,0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)], > pending 1 -> 0 > Jul 10 10:38:52 bsd-14 kernel: ql0: collecting disabled > Jul 10 10:38:52 bsd-14 kernel: lacp_aggregator_delref: > lagid=[(8000,00-0E-1E-08-05-20,01D3,0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)], > refcnt 1 -> 0 > Jul 10 10:38:52 bsd-14 kernel: ql0: mux_state 1 -> 0 > Jul 10 10:38:52 bsd-14 kernel: ql0: lacpdu transmit > Jul 10 10:38:52 bsd-14 kernel: actor=(8000,00-0E-1E-08-05-20,01D3,8000,000C) > Jul 10 10:38:52 bsd-14 kernel: actor.state=5<ACTIVITY,AGGREGATION> > Jul 10 10:38:52 bsd-14 kernel: partner=(8000,00-0E-1E-04-2C-F0,0213,8000,000E) > Jul 10 10:38:52 bsd-14 kernel: partner.state=5<ACTIVITY,AGGREGATION> > Jul 10 10:38:52 bsd-14 kernel: maxdelay=0 > Jul 10 10:38:53 bsd-14 kernel: ql1: port > lagid=[(8000,00-0E-1E-08-05-20,01D3,8000,000D),(FFFF,00-00-00-00-00-00,0000,FFFF,0000)] > Jul 10 10:38:53 bsd-14 kernel: ql1: aggregator created > Jul 10 10:38:53 bsd-14 kernel: ql1: aggregator > lagid=[(8000,00-0E-1E-08-05-20,01D3,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000,0000)] > Jul 10 10:38:53 bsd-14 kernel: ql1: mux_state 0 -> 1 > Jul 10 10:38:53 bsd-14 kernel: ql0: port > lagid=[(8000,00-0E-1E-08-05-20,01D3,8000,000C),(8000,00-0E-1E-04-2C-F0,0213,8000,000E)] > Jul 10 10:38:53 bsd-14 kernel: ql0: aggregator created > Jul 10 10:38:53 bsd-14 kernel: ql0: aggregator > lagid=[(8000,00-0E-1E-08-05-20,01D3,0000,0000),(8000,00-0E-1E-04-2C-F0,0213,0000,0000)] > Jul 10 10:38:53 bsd-14 kernel: ql0: mux_state 0 -> 1 > Jul 10 10:38:54 bsd-14 kernel: ql0: lacpdu receive > Jul 10 10:38:54 bsd-14 kernel: actor=(8000,00-0E-1E-04-2C-F0,0213,8000,000E) > Jul 10 10:38:54 bsd-14 kernel: actor.state=d<ACTIVITY,AGGREGATION,SYNC> > Jul 10 10:38:54 bsd-14 kernel: partner=(8000,00-0E-1E-08-05-20,01D3,8000,000C) > Jul 10 10:38:54 bsd-14 kernel: partner.state=5<ACTIVITY,AGGREGATION> > Jul 10 10:38:54 bsd-14 kernel: maxdelay=0 > Jul 10 10:38:54 bsd-14 kernel: ql0: old pstate 5<ACTIVITY,AGGREGATION> > Jul 10 10:38:54 bsd-14 kernel: ql0: new pstate d<ACTIVITY,AGGREGATION,SYNC> > Jul 10 10:38:55 bsd-14 kernel: ql1: lacp_sm_mux_timer: aggregator > [(8000,00-0E-1E-08-05-20,01D3,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000,0000)], > pending 1 -> 0 > Jul 10 10:38:55 bsd-14 kernel: ql1: collecting disabled > Jul 10 10:38:55 bsd-14 kernel: ql1: mux_state 1 -> 2 > Jul 10 10:38:55 bsd-14 kernel: ql1: collecting enabled > Jul 10 10:38:55 bsd-14 kernel: ql1: mux_state 2 -> 3 > Jul 10 10:38:55 bsd-14 kernel: ql1: enable distributing on aggregator > [(8000,00-0E-1E-08-05-20,01D3,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000,0000)], > nports 0 -> 1 > Jul 10 10:38:55 bsd-14 kernel: lacp_select_active_aggregator: > Jul 10 10:38:55 bsd-14 kernel: > [(8000,00-0E-1E-08-05-20,01D3,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000,0000)], > speed=10000000000, nports=1 > Jul 10 10:38:55 bsd-14 kernel: active aggregator changed > Jul 10 10:38:55 bsd-14 kernel: old (none) > Jul 10 10:38:55 bsd-14 kernel: new > [(8000,00-0E-1E-08-05-20,01D3,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000,0000)] > Jul 10 10:38:55 bsd-14 kernel: Set table 1 with 1 ports > Jul 10 10:38:55 bsd-14 kernel: lacp_suppress_distributing > Jul 10 10:38:55 bsd-14 kernel: ql1: marker transmit, port=13, > sys=00:0e:1e:08:05:20, id=1 > Jul 10 10:38:55 bsd-14 kernel: ql0: marker transmit, port=12, > sys=00:0e:1e:08:05:20, id=1 > Jul 10 10:38:55 bsd-14 kernel: ql1: mux_state 3 -> 4 > Jul 10 10:38:55 bsd-14 kernel: ql1: lacpdu transmit > Jul 10 10:38:55 bsd-14 kernel: ql0: marker response, port=12, > sys=00:0e:1e:08:05:20, id=1 > Jul 10 10:38:55 bsd-14 kernel: actor=(8000,00-0E-1E-08-05-20,01D3,8000,000D) > Jul 10 10:38:55 bsd-14 kernel: > actor.state=7d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING,DEFAULTED> > Jul 10 10:38:55 bsd-14 kernel: partner=(FFFF,00-00-00-00-00-00,0000,FFFF,0000) > Jul 10 10:38:55 bsd-14 kernel: > partner.state=3c<AGGREGATION,SYNC,COLLECTING,DISTRIBUTING> > Jul 10 10:38:55 bsd-14 kernel: maxdelay=0 > Jul 10 10:38:55 bsd-14 kernel: ql0: lacp_sm_mux_timer: aggregator > [(8000,00-0E-1E-08-05-20,01D3,0000,0000),(8000,00-0E-1E-04-2C-F0,0213,0000,0000)], > pending 1 -> 0 > Jul 10 10:38:55 bsd-14 kernel: ql0: collecting disabled > Jul 10 10:38:55 bsd-14 kernel: ql0: mux_state 1 -> 2 > Jul 10 10:38:55 bsd-14 kernel: ql0: collecting enabled > Jul 10 10:38:55 bsd-14 kernel: ql0: mux_state 2 -> 3 > Jul 10 10:38:55 bsd-14 kernel: ql0: lacpdu transmit > Jul 10 10:38:55 bsd-14 kernel: actor=(8000,00-0E-1E-08-05-20,01D3,8000,000C) > Jul 10 10:38:55 bsd-14 kernel: > actor.state=1d<ACTIVITY,AGGREGATION,SYNC,COLLECTING> > Jul 10 10:38:55 bsd-14 kernel: partner=(8000,00-0E-1E-04-2C-F0,0213,8000,000E) > Jul 10 10:38:55 bsd-14 kernel: partner.state=d<ACTIVITY,AGGREGATION,SYNC> > Jul 10 10:38:55 bsd-14 kernel: maxdelay=0 > Jul 10 10:38:55 bsd-14 kernel: ql0: lacpdu receive > Jul 10 10:38:55 bsd-14 kernel: actor=(8000,00-0E-1E-04-2C-F0,0213,8000,000E) > Jul 10 10:38:55 bsd-14 kernel: > actor.state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING> > Jul 10 10:38:55 bsd-14 kernel: partner=(8000,00-0E-1E-08-05-20,01D3,8000,000C) > Jul 10 10:38:55 bsd-14 kernel: > partner.state=1d<ACTIVITY,AGGREGATION,SYNC,COLLECTING> > Jul 10 10:38:55 bsd-14 kernel: maxdelay=0 > Jul 10 10:38:55 bsd-14 kernel: ql0: old pstate d<ACTIVITY,AGGREGATION,SYNC> > Jul 10 10:38:55 bsd-14 kernel: ql0: new pstate > 3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING> > Jul 10 10:38:56 bsd-14 kernel: ql0: enable distributing on aggregator > [(8000,00-0E-1E-08-05-20,01D3,0000,0000),(8000,00-0E-1E-04-2C-F0,0213,0000,0000)], > nports 0 -> 1 > Jul 10 10:38:56 bsd-14 kernel: lacp_select_active_aggregator: > Jul 10 10:38:56 bsd-14 kernel: > [(8000,00-0E-1E-08-05-20,01D3,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000,0000)], > speed=10000000000, nports=1 > Jul 10 10:38:56 bsd-14 kernel: > [(8000,00-0E-1E-08-05-20,01D3,0000,0000),(8000,00-0E-1E-04-2C-F0,0213,0000,0000)], > speed=10000000000, nports=1 > Jul 10 10:38:56 bsd-14 kernel: active aggregator changed > Jul 10 10:38:56 bsd-14 kernel: old > [(8000,00-0E-1E-08-05-20,01D3,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000,0000)] > Jul 10 10:38:56 bsd-14 kernel: new > [(8000,00-0E-1E-08-05-20,01D3,0000,0000),(8000,00-0E-1E-04-2C-F0,0213,0000,0000)] > Jul 10 10:38:56 bsd-14 kernel: Set table 0 with 1 ports > Jul 10 10:38:56 bsd-14 kernel: lacp_suppress_distributing > Jul 10 10:38:56 bsd-14 kernel: ql1: marker transmit, port=13, > sys=00:0e:1e:08:05:20, id=2 > Jul 10 10:38:56 bsd-14 kernel: ql0: marker transmit, port=12, > sys=00:0e:1e:08:05:20, id=2 > Jul 10 10:38:56 bsd-14 kernel: ql0: mux_state 3 -> 4 > Jul 10 10:38:56 bsd-14 kernel: ql0: marker response, port=12, > sys=00:0e:1e:08:05:20, id=2 > Jul 10 10:38:59 bsd-14 kernel: lacp_transit_expire > ^C > > > > > System 2 logs : > > Jul 10 02:38:24 bsd-15 kernel: lacp_attach[738] : lacp attached > Jul 10 02:38:24 bsd-15 kernel: lacp_attach[740] : lacp_defined > Jul 10 02:38:24 bsd-15 kernel: lagg0: link state changed to UP > Jul 10 02:38:24 bsd-15 kernel: ql0: media changed 0x0 -> 0x100033, ether = 1, > fdx = 1, link = 1 > Jul 10 02:38:24 bsd-15 kernel: ql0: -> UNSELECTED > Jul 10 02:38:24 bsd-15 kernel: ql1: media changed 0x0 -> 0x100033, ether = 1, > fdx = 1, link = 1 > Jul 10 02:38:24 bsd-15 kernel: ql1: -> UNSELECTED > Jul 10 02:38:24 bsd-15 kernel: lacp_select_tx_port: no active aggregator > Jul 10 02:38:25 bsd-15 kernel: ql1: port > lagid=[(8000,00-0E-1E-04-2C-F0,0213,8000,000F),(0000,00-00-00-00-00-00,0000,0000,0000)] > Jul 10 02:38:25 bsd-15 kernel: ql1: aggregator created > Jul 10 02:38:25 bsd-15 kernel: ql1: aggregator > lagid=[(8000,00-0E-1E-04-2C-F0,0213,0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)] > Jul 10 02:38:25 bsd-15 kernel: ql1: mux_state 0 -> 1 > Jul 10 02:38:25 bsd-15 kernel: ql0: port > lagid=[(8000,00-0E-1E-04-2C-F0,0213,8000,000E),(0000,00-00-00-00-00-00,0000,0000,0000)] > Jul 10 02:38:25 bsd-15 kernel: ql0: aggregator created > Jul 10 02:38:25 bsd-15 kernel: ql0: aggregator > lagid=[(8000,00-0E-1E-04-2C-F0,0213,0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)] > Jul 10 02:38:25 bsd-15 kernel: ql0: mux_state 0 -> 1 > Jul 10 02:38:26 bsd-15 kernel: ql0: lacpdu receive > Jul 10 02:38:26 bsd-15 kernel: actor=(8000,00-0E-1E-08-05-20,01D3,8000,000C) > Jul 10 02:38:26 bsd-15 kernel: actor.state=85<ACTIVITY,AGGREGATION,EXPIRED> > Jul 10 02:38:26 bsd-15 kernel: partner=(0000,00-00-00-00-00-00,0000,0000,0000) > Jul 10 02:38:26 bsd-15 kernel: partner.state=2<TIMEOUT> > Jul 10 02:38:26 bsd-15 kernel: maxdelay=0 > Jul 10 02:38:26 bsd-15 kernel: ql0: lacp_sm_rx_update_ntt: assert ntt > Jul 10 02:38:26 bsd-15 kernel: ql0: old pstate 2<TIMEOUT> > Jul 10 02:38:26 bsd-15 kernel: ql0: new pstate > 85<ACTIVITY,AGGREGATION,EXPIRED> > Jul 10 02:38:26 bsd-15 kernel: ql0: partner timeout changed > Jul 10 02:38:26 bsd-15 kernel: ql0: lacpdu transmit > Jul 10 02:38:26 bsd-15 kernel: actor=(8000,00-0E-1E-04-2C-F0,0213,8000,000E) > Jul 10 02:38:26 bsd-15 kernel: actor.state=5<ACTIVITY,AGGREGATION> > Jul 10 02:38:26 bsd-15 kernel: partner=(8000,00-0E-1E-08-05-20,01D3,8000,000C) > Jul 10 02:38:26 bsd-15 kernel: partner.state=85<ACTIVITY,AGGREGATION,EXPIRED> > Jul 10 02:38:26 bsd-15 kernel: maxdelay=0 > Jul 10 02:38:26 bsd-15 kernel: ql1: lacpdu transmit > Jul 10 02:38:26 bsd-15 kernel: actor=(8000,00-0E-1E-04-2C-F0,0213,8000,000F) > Jul 10 02:38:26 bsd-15 kernel: actor.state=85<ACTIVITY,AGGREGATION,EXPIRED> > Jul 10 02:38:26 bsd-15 kernel: partner=(0000,00-00-00-00-00-00,0000,0000,0000) > Jul 10 02:38:26 bsd-15 kernel: partner.state=2<TIMEOUT> > Jul 10 02:38:26 bsd-15 kernel: maxdelay=0 > Jul 10 02:38:26 bsd-15 kernel: ql0: collecting disabled > Jul 10 02:38:26 bsd-15 kernel: lacp_aggregator_delref: > lagid=[(8000,00-0E-1E-04-2C-F0,0213, > 0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)], refcnt 1 -> 0 > > Jul 10 02:38:26 bsd-15 kernel: ql0: mux_state 1 -> 0 > Jul 10 02:38:26 bsd-15 kernel: ql0: lacpdu transmit > Jul 10 02:38:26 bsd-15 kernel: actor=(8000,00-0E-1E-04-2C-F0,0213,8000,000E) > Jul 10 02:38:26 bsd-15 kernel: actor.state=5<ACTIVITY,AGGREGATION> > Jul 10 02:38:26 bsd-15 kernel: partner=(8000,00-0E-1E-08-05-20,01D3,8000,000C) > Jul 10 02:38:26 bsd-15 kernel: partner.state=85<ACTIVITY,AGGREGATION,EXPIRED> > Jul 10 02:38:26 bsd-15 kernel: maxdelay=0 > Jul 10 02:38:27 bsd-15 kernel: ql0: lacpdu receive > Jul 10 02:38:27 bsd-15 kernel: actor=(8000,00-0E-1E-08-05-20,01D3,8000,000C) > Jul 10 02:38:27 bsd-15 kernel: actor.state=5<ACTIVITY,AGGREGATION> > Jul 10 02:38:27 bsd-15 kernel: partner=(8000,00-0E-1E-04-2C-F0,0213,8000,000E) > Jul 10 02:38:27 bsd-15 kernel: partner.state=5<ACTIVITY,AGGREGATION> > Jul 10 02:38:27 bsd-15 kernel: maxdelay=0 > Jul 10 02:38:27 bsd-15 kernel: ql0: old pstate > 85<ACTIVITY,AGGREGATION,EXPIRED> > Jul 10 02:38:27 bsd-15 kernel: ql0: new pstate 5<ACTIVITY,AGGREGATION> > Jul 10 02:38:27 bsd-15 kernel: ql1: lacp_sm_rx_timer: EXPIRED -> DEFAULTED > Jul 10 02:38:27 bsd-15 kernel: ql1: partner timeout changed > Jul 10 02:38:27 bsd-15 kernel: ql1: lacp_sm_mux_timer: aggregator > [(8000,00-0E-1E-04-2C-F0, > 0213,0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)], pending 1 -> 0 > Jul 10 02:38:27 bsd-15 kernel: ql1: collecting disabled > Jul 10 02:38:27 bsd-15 kernel: lacp_aggregator_delref: > lagid=[(8000,00-0E-1E-04-2C-F0,0213, > 0000,0000),(0000,00-00-00-00-00-00,0000,0000,0000)], refcnt 1 -> 0 > > Jul 10 02:38:27 bsd-15 kernel: ql1: mux_state 1 -> 0 > Jul 10 02:38:27 bsd-15 kernel: ql1: lacpdu transmit > Jul 10 02:38:27 bsd-15 kernel: actor=(8000,00-0E-1E-04-2C-F0,0213,8000,000F) > Jul 10 02:38:27 bsd-15 kernel: actor.state=45<ACTIVITY,AGGREGATION,DEFAULTED> > Jul 10 02:38:27 bsd-15 kernel: partner=(FFFF,00-00-00-00-00-00,0000,FFFF,0000) > Jul 10 02:38:27 bsd-15 kernel: > partner.state=3c<AGGREGATION,SYNC,COLLECTING,DISTRIBUTING> > Jul 10 02:38:27 bsd-15 kernel: maxdelay=0 > Jul 10 02:38:27 bsd-15 kernel: ql0: port > lagid=[(8000,00-0E-1E-04-2C-F0,0213,8000,000E),(8000,00-0E-1E-08-05-20,01D3,8000,000C)] > Jul 10 02:38:27 bsd-15 kernel: ql0: aggregator created > Jul 10 02:38:27 bsd-15 kernel: ql0: aggregator > lagid=[(8000,00-0E-1E-04-2C-F0,0213,0000,0000),(8000,00-0E-1E-08-05-20,01D3,0000,0000)] > Jul 10 02:38:27 bsd-15 kernel: ql0: mux_state 0 -> 1 > Jul 10 02:38:28 bsd-15 kernel: ql1: port > lagid=[(8000,00-0E-1E-04-2C-F0,0213,8000,000F),(FFFF,00-00-00-00-00-00,0000,FFFF,0000)] > Jul 10 02:38:28 bsd-15 kernel: ql1: aggregator created > Jul 10 02:38:28 bsd-15 kernel: ql1: aggregator > lagid=[(8000,00-0E-1E-04-2C-F0,0213,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000,0000)] > Jul 10 02:38:28 bsd-15 kernel: ql1: mux_state 0 -> 1 > Jul 10 02:38:29 bsd-15 kernel: ql0: lacp_sm_mux_timer: aggregator > [(8000,00-0E-1E-04-2C-F0, > 0213,0000,0000),(8000,00-0E-1E-08-05-20,01D3,0000,0000)], pending 1 -> 0 > Jul 10 02:38:29 bsd-15 kernel: ql0: collecting disabled > Jul 10 02:38:29 bsd-15 kernel: ql0: mux_state 1 -> 2 > Jul 10 02:38:29 bsd-15 kernel: ql0: lacpdu transmit > Jul 10 02:38:29 bsd-15 kernel: actor=(8000,00-0E-1E-04-2C-F0,0213,8000,000E) > Jul 10 02:38:29 bsd-15 kernel: actor.state=d<ACTIVITY,AGGREGATION,SYNC> > Jul 10 02:38:29 bsd-15 kernel: partner=(8000,00-0E-1E-08-05-20,01D3,8000,000C) > Jul 10 02:38:29 bsd-15 kernel: partner.state=5<ACTIVITY,AGGREGATION> > Jul 10 02:38:29 bsd-15 kernel: maxdelay=0 > Jul 10 02:38:30 bsd-15 kernel: ql0: lacpdu receive > Jul 10 02:38:30 bsd-15 kernel: actor=(8000,00-0E-1E-08-05-20,01D3,8000,000C) > Jul 10 02:38:30 bsd-15 kernel: > actor.state=1d<ACTIVITY,AGGREGATION,SYNC,COLLECTING> > Jul 10 02:38:30 bsd-15 kernel: partner=(8000,00-0E-1E-04-2C-F0,0213,8000,000E) > Jul 10 02:38:30 bsd-15 kernel: partner.state=d<ACTIVITY,AGGREGATION,SYNC> > Jul 10 02:38:30 bsd-15 kernel: maxdelay=0 > Jul 10 02:38:30 bsd-15 kernel: ql0: old pstate 5<ACTIVITY,AGGREGATION> > Jul 10 02:38:30 bsd-15 kernel: ql0: new pstate > 1d<ACTIVITY,AGGREGATION,SYNC,COLLECTING> > Jul 10 02:38:30 bsd-15 kernel: ql1: lacp_sm_mux_timer: aggregator > [(8000,00-0E-1E-04-2C-F0, > 0213,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000,0000)], pending 1 -> 0 > Jul 10 02:38:30 bsd-15 kernel: ql1: collecting disabled > Jul 10 02:38:30 bsd-15 kernel: ql1: mux_state 1 -> 2 > Jul 10 02:38:30 bsd-15 kernel: ql1: collecting enabled > Jul 10 02:38:30 bsd-15 kernel: ql1: mux_state 2 -> 3 > Jul 10 02:38:30 bsd-15 kernel: ql1: enable distributing on aggregator > [(8000,00-0E-1E-04-2C-F0,0213,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000,0000)], > nports 0 -> 1 > Jul 10 02:38:30 bsd-15 kernel: lacp_select_active_aggregator: > Jul 10 02:38:30 bsd-15 kernel: > [(8000,00-0E-1E-04-2C-F0,0213,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000,0000)], > speed=10000000000, nports=1 > Jul 10 02:38:30 bsd-15 kernel: active aggregator changed > Jul 10 02:38:30 bsd-15 kernel: old (none) > Jul 10 02:38:30 bsd-15 kernel: new > [(8000,00-0E-1E-04-2C-F0,0213,0000,0000),(FFFF,00-00-00 > 00-00-00,0000,0000,0000)] > Jul 10 02:38:30 bsd-15 kernel: Set table 1 with 1 ports > Jul 10 02:38:30 bsd-15 kernel: lacp_suppress_distributing > Jul 10 02:38:30 bsd-15 kernel: ql1: marker transmit, port=15, > sys=00:0e:1e:04:2c:f0, id=1 > Jul 10 02:38:30 bsd-15 kernel: ql0: marker transmit, port=14, > sys=00:0e:1e:04:2c:f0, id=1 > Jul 10 02:38:30 bsd-15 kernel: ql1: mux_state 3 -> 4 > Jul 10 02:38:30 bsd-15 kernel: ql1: lacpdu transmit > Jul 10 02:38:30 bsd-15 kernel: ql0: marker response, port=14, > sys=00:0e:1e:04:2c:f0, id=1 > Jul 10 02:38:30 bsd-15 kernel: actor=(8000,00-0E-1E-04-2C-F0,0213,8000,000F) > Jul 10 02:38:30 bsd-15 kernel: > actor.state=7d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING,DEFAULTED> > Jul 10 02:38:30 bsd-15 kernel: partner=(FFFF,00-00-00-00-00-00,0000,FFFF,0000) > Jul 10 02:38:30 bsd-15 kernel: > partner.state=3c<AGGREGATION,SYNC,COLLECTING,DISTRIBUTING> > Jul 10 02:38:30 bsd-15 kernel: maxdelay=0 > Jul 10 02:38:30 bsd-15 kernel: ql0: collecting enabled > Jul 10 02:38:30 bsd-15 kernel: ql0: mux_state 2 -> 3 > Jul 10 02:38:30 bsd-15 kernel: ql0: enable distributing on aggregator > [(8000,00-0E-1E-04-2C-F0,0213,0000,0000),(8000,00-0E-1E-08-05-20,01D3,0000,0000)], > nports 0 -> 1 > Jul 10 02:38:30 bsd-15 kernel: lacp_select_active_aggregator: > Jul 10 02:38:30 bsd-15 kernel: > [(8000,00-0E-1E-04-2C-F0,0213,0000,0000),(8000,00-0E-1E-08-05-20,01D3,0000,0000)], > speed=10000000000, nports=1 > Jul 10 02:38:30 bsd-15 kernel: > [(8000,00-0E-1E-04-2C-F0,0213,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000,0000)], > speed=10000000000, nports=1 > Jul 10 02:38:30 bsd-15 kernel: active aggregator not changed > Jul 10 02:38:30 bsd-15 kernel: new > [(8000,00-0E-1E-04-2C-F0,0213,0000,0000),(FFFF,00-00-00-00-00-00,0000,0000,0000)] > Jul 10 02:38:30 bsd-15 kernel: ql0: mux_state 3 -> 4 > Jul 10 02:38:30 bsd-15 kernel: ql0: lacpdu transmit > Jul 10 02:38:30 bsd-15 kernel: actor=(8000,00-0E-1E-04-2C-F0,0213,8000,000E) > Jul 10 02:38:30 bsd-15 kernel: > actor.state=3d<ACTIVITY,AGGREGATION,SYNC,COLLECTING,DISTRIBUTING> > Jul 10 02:38:30 bsd-15 kernel: partner=(8000,00-0E-1E-08-05-20,01D3,8000,000C) > Jul 10 02:38:30 bsd-15 kernel: > partner.state=1d<ACTIVITY,AGGREGATION,SYNC,COLLECTING> > Jul 10 02:38:30 bsd-15 kernel: maxdelay=0 > Jul 10 02:38:33 bsd-15 kernel: lacp_transit_expire > ^C > > Let me know if you need more info. > > Thanks > Adarsh > > > -----Original Message----- > From: owner-freebsd-...@freebsd.org [mailto:owner-freebsd-...@freebsd.org] On > Behalf Of Adarsh Joshi > Sent: Tuesday, July 10, 2012 10:08 AM > To: Andrew Boyer > Cc: freebsd-net@freebsd.org > Subject: RE: lacp lagg port flags do not show correctly resulting in poor > traffic distribution/performance > > Andrew, > > Thanks for the reply. > > The reason for my suspicion on the portflags is thus (extracted from the > ifconfig output in my previous mail): > > System 1: > Laggport: ql1 flags = 18 state = 7D > Laggport: ql0 flags = 1c state = 3D > > System 2: > Laggport: ql1 flags = 1c state = 7D > Laggport: ql0 flags = 18 state = 3D > > I should have explained my setup to you before. Here it is. > Both the ql0 interfaces of the 2 systems are connected using a single cable > and ql1 interfaces of the 2 systems are connected using a single cable. > > System 1 System 2 > ql0 <=======> ql0 > ql1 <=======> ql1 > > With this setup, I don't think it is possible for ports ql0 to talk to their > partners (each other) and ql1 ports not getting a response from their partner > and still get the lagg configuration I have posted. > > I thought the portflags are dependent on the LACP state. But I see different > flags for the same LACP state (For the state 7D, ql1 on system 1 shows flags > = 18 and ql1 on system 2 shows flags = 1c). > > Or is my understanding totally wrong? > > I will send the LACP_DEBUG logs within the hour. > > Thanks > Adarsh > > From: Andrew Boyer [mailto:abo...@averesystems.com] > Sent: Tuesday, July 10, 2012 5:57 AM > To: Adarsh Joshi > Cc: freebsd-net@freebsd.org > Subject: Re: lacp lagg port flags do not show correctly resulting in poor > traffic distribution/performance > > > On Jul 9, 2012, at 8:38 PM, Adarsh Joshi wrote: > > > Hi, > > I am trying to configure lacp lagg interfaces with 2 systems connected back > to back as follows: > > Ifconfig lagg0 create > Ifconfig lagg0 laggproto lacp laggport ql0 laggport ql1 192.168.100.1 netmask > 255.255.255.0 > > Sometimes, the lag interface comes up correctly but sometimes the laggport > flags do not show properly. Instead of 1c<ACTIVE,COLLECTING,DISTRIBUTING>, it > shows values of 18. I have seen similar issues reported on various forums > with no solution. > Looking at the lagg driver code and reading the standard, I thought the > laggport flags ( defined in if_lagg.h) are based on the LACP_STATE_BITS in > file ieee8023ad_lacp.h. But the following ifconfig -v output does not make > any sense to me. > > My concern is that when all the interfaces show flags as 1c, the traffic is > distributed across both the interfaces uniformly and I get aggregated > throughput. If not, the traffic flows only on 1 interface. > > Is this a bug? How do I solve this? Or am I doing something wrong? > > I am using Free-BSD 9.0 release. > > System 1: > # ifconfig -v lagg0 > lag id: [(8000,00-0E-1E-08-05-20,0213,0000,0000), > (8000,00-0E-1E-04-2C-F0,0213,0000,0000)] > laggport: ql1 flags=18<COLLECTING,DISTRIBUTING> state=7D > [(8000,00-0E-1E-08-05-20,0213,8000,000F), > (FFFF,00-00-00-00-00-00,0000,FFFF,0000)] > laggport: ql0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> state=3D > [(8000,00-0E-1E-08-05-20,0213,8000,000E), > (8000,00-0E-1E-04-2C-F0,0213,8000,000E)] > > System 2: > > # ifconfig -v lagg0 > lag id: [(8000,00-0E-1E-04-2C-F0,0213,0000,0000), > (FFFF,00-00-00-00-00-00,0000,0000,0000)] > laggport: ql1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING> state=7D > [(8000,00-0E-1E-04-2C-F0,0213,8000,000F), > (FFFF,00-00-00-00-00-00,0000,FFFF,0000)] > laggport: ql0 flags=18<COLLECTING,DISTRIBUTING> state=3D > [(8000,00-0E-1E-04-2C-F0,0213,8000,000E), > (8000,00-0E-1E-08-05-20,0213,8000,000E)] > > > thanks > Adarsh > > I don't think you have a port flags problem per se; the flags are correctly > displaying the state of the lagg. Your problem is that your systems aren't > negotiating the correct lagg configuration. Each tuple after the laggport > represents the [(actor state),(partner state)]. Ports ql0 have been able to > talk to their partners (each other). Neither ql1 port has seen a response > from a partner, though. > > You could try restarting the state machine on one box with 'ifconfig lagg0 > laggproto lacp'. To see the negotiation you'll need to rebuild your kernel > with '#define LACP_DEBUG 1' added to the top of sys/net/ieee802.3ad_lacp.c. > Or upgrade to a newer stable snapshot that has the net.lacp_debug sysctl and > turn it on. > > Or just turn off LACP. What does it get you in this configuration? > > Hope this helps, > Andrew > > -------------------------------------------------- > Andrew Boyer abo...@averesystems.com<mailto:abo...@averesystems.com> > > > > > > ________________________________ > This message and any attached documents contain information from QLogic > Corporation or its wholly-owned subsidiaries that may be confidential. If you > are not the intended recipient, you may not read, copy, distribute, or use > this information. If you have received this transmission in error, please > notify the sender immediately by reply e-mail and then delete this message. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org" > > > This message and any attached documents contain information from QLogic > Corporation or its wholly-owned subsidiaries that may be confidential. If you > are not the intended recipient, you may not read, copy, distribute, or use > this information. If you have received this transmission in error, please > notify the sender immediately by reply e-mail and then delete this message. > -------------------------------------------------- Andrew Boyer abo...@averesystems.com _______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
