This is really cool. I’m going to check into this and see what I come up with ☺ Thanks for the input! I love what LACP does in theory, I know in some practices it’s not optimal and not everyone’s first choice. It is however the recommended method from DW and the only one they will support. :-P
A couple of things that continue to interest me is that the 48 port that is unmanageable. Assuming it is malfunctioning, by DEFAULT HP Procurve switches have dynamic LACP across all ports, preventing network loops automatically but also making it cumbersome to troubleshoot this particular issue. Knowing that, and the possibility of something running FUBAR on one of the switches, I think it’s quite possible that the LACP that was originally configured on the malfunctioning switch could be the culprit for the inbalance. I am programming a 24-port Procurve that we have as a spare to handle the trunk to test the theory. The older Procurves have lots of options in regards to LACP (their console is very much like a Cisco) so I may be able to setup something similar to what you’re suggesting. I’ll update as I find out more. Thanks for the discussion and I love the progression, getting lots of ideas from the conversation. ☺ Gino, unfortunately because of the Horizon Compact I don’t believe we have that option, at least not to get the full 800Mb/s (theoretical) from the radios ☹ It’s been awhile since I went through the manual for the setup, but I recall there were some negative aspects to linking the radios through port 2 or something along those lines. And for Bill, since we’re running a large bridged network currently (but so happy we’re changing it soon!!) we’re not running any routers at the base of our towers. I agree totally if we were segmented that OSPF in combination with a few other protocols would handle the job and give us some other troubleshooting capabilities not available to us in our current config. Even packet captures would be cumbersome ☹ -Tim From: Af [mailto:af-boun...@afmug.com] On Behalf Of Mark Radabaugh via Af Sent: Friday, October 24, 2014 10:10 AM To: af@afmug.com Subject: Re: [AFMUG] Dragonwave latency issues Tim, Obviously you want to fix the radio issue but I can give you a little advice on LACP. The switch (or router) doing LACP can take various headers into account when doing the 'hashing' to decide how to send the flows. If LACP is what you are stuck with (Gino suggested a layer 1 method) and you have control over the devices doing LACP take a look at the LACP Hashing options. Typical options are MAC SRC/DST, MAC Src/Dst + L3 port, or some other options. If most of your traffic is flowing between two interfaces (a router at each end) the MAC src/dst hashing sends everything over one of the links. You want to find a hashing method that has more randomness to it. In at least one case where we use that we had to resort to not using the LACP functionality the radio manufacturer provided and use the better LACP options in our Juniper and Cisco switches in order to make the traffic balance on the interfaces. Mark On 10/24/14, 12:38 PM, Timothy D. McNabb via Af wrote: We have now been able to trace to the heart of the problem, though unfortunately still attempting to determine the cause. To give you a rough background, we have a main link that “bonds” 2 Dragonwaves together to form one unit (using the dual radio mount). The DW manual states for this to work when doubling the throughput, it is required to use LACP trunking. We have this in place and it has been working fine up until recently. The trunk is still active (no network loop) however one radio is working more than the other, eventually saturating one of the two links and causing the latency. The second set of radios aren’t performing in terms of actual traffic (signal is ok) but at most they’ve moved 200Mb/s even though they are licensed for 400Mbs. The link with the LACP has more or less load-balanced itself in the past, however they are now performing very asymmetrical at this point with over a 200Mb/s difference. I understand that LACP does not actively load balance but that’s what has been observed in the past. I suspect the issue is being caused by one of the switches doing the trunking (we just discovered it is unmanageable but still operating, again no network loop). It’s either that or the one leg of the link itself with the low bandwidth usage is not working properly despite indications otherwise. We’ll investigate our theories but always looking for additional input ☺ -Tim From: Af [mailto:af-boun...@afmug.com] On Behalf Of Josh Reynolds via Af Sent: Thursday, October 23, 2014 1:47 PM To: af@afmug.com<mailto:af@afmug.com> Subject: Re: [AFMUG] Dragonwave latency issues You can do a bit more with metro-e style NIDS... those have the layer2 tools to do proper testing. Josh Reynolds, Chief Information Officer SPITwSPOTS, www.spitwspots.com<http://www.spitwspots.com> On 10/23/2014 12:37 PM, Eric Kuhnke via Af wrote: it's going to be really hard to do any meaningful diagnostics with a layer 2 switch on each end, not routers... you could be seeing a broadcast flood of some type. On Thu, Oct 23, 2014 at 1:10 PM, Timothy D. McNabb via Af <af@afmug.com<mailto:af@afmug.com>> wrote: Unfortunately there is no QoS and flow control is off on the switches ☹ Dragonwave was contacted as well. No determination yet though. -Tim From: Af [mailto:af-boun...@afmug.com<mailto:af-boun...@afmug.com>] On Behalf Of Peter Kranz via Af Sent: Wednesday, October 22, 2014 1:36 PM To: af@afmug.com<mailto:af@afmug.com> Subject: Re: [AFMUG] Dragonwave latency issues Backpressure from the switches in terms of flow-control can show as latency on dragonwave links. Disable any QOS features on the dragonwave if you are using them. Email dragonwave support Peter Kranz Founder/CEO - Unwired Ltd www.UnwiredLtd.com<http://www.unwiredltd.com/> Desk: 510-868-1614 x100 Mobile: 510-207-0000 pkr...@unwiredltd.com<mailto:pkr...@unwiredltd.com> From: Af [mailto:af-boun...@afmug.com] On Behalf Of Timothy D. McNabb via Af Sent: Wednesday, October 22, 2014 1:15 PM To: af@afmug.com<mailto:af@afmug.com> Subject: Re: [AFMUG] Dragonwave latency issues No routers between, just switches. -Tim From: Af [mailto:af-boun...@afmug.com] On Behalf Of Paul Conlin via Af Sent: Wednesday, October 22, 2014 12:22 PM To: af@afmug.com<mailto:af@afmug.com> Subject: Re: [AFMUG] Dragonwave latency issues Could the routers at each end be the limiting factor? What is their CPU utilization when the link is loaded? What happens to latency if you stress the link at 200 Mbps with a speed test? Those radios should be able to do close to 400 Mbps all day long with no latency. PC Blaze Broadband From: Af [mailto:af-boun...@afmug.com] On Behalf Of Joshua Heide via Af Sent: Wednesday, October 22, 2014 3:06 PM To: af@afmug.com<mailto:af@afmug.com> Subject: Re: [AFMUG] Dragonwave latency issues Yes it’s a horizon compact Bandwidth of the unit is 400mbs Bandwidth usage between 150-200mbs during peak hours. No QOS Yes during non-peak hours its sits at 1ms SNR 35.00 dB From our prtg graphs this issues has started end of September and latency has gotten worse during peak times as we have deployed more 450 gear to that tower. I currently have HAAM enabled on the link and it stays at 256qam unless we have some bad weather. Josh Heide Velociter Wireless (office) 209-838-1221 (fax) 209-838-1800 www.velociter.net<http://www.velociter.net/> From: Af [mailto:af-boun...@afmug.com] On Behalf Of Bill Prince via Af Sent: Wednesday, October 22, 2014 11:47 AM To: af@afmug.com<mailto:af@afmug.com> Subject: Re: [AFMUG] Dragonwave latency issues So it's a Horizon Compact? What is the total bandwidth, and what percentage are you using? Have you set up any QOS? 180 ms sounds like a lot; especially when ours are typically less than 1 ms. -38 is right in the game. What are the other parameters besides signal level? bp On 10/22/2014 11:18 AM, Joshua Heide via Af wrote: We have a dragonwave that has latency issues that coincide with traffic peak times. As our traffic peaks so does that latency at 180ms. Any ideas that could cause this? Signal is -38 Current HAAM Mode hc50_364_256qam Thanks, Josh Heide Velociter Wireless (office) 209-838-1221 (fax) 209-838-1800 www.velociter.net<http://www.velociter.net/> -- Mark Radabaugh Amplex m...@amplex.net<mailto:m...@amplex.net> 419.837.5015 x 1021