First, I wanted to give thanks to David for helping me track down this issue and for providing insight into the workings of the FWSM.
To recap the issue I was seeing the majority of outbound traffic from the FWSM was exiting on the 3rd and 6th port of the ether-channel while the inbound traffic to the FWSM was pretty much equally distributed across the six links. Some more info on our setup… Behind this FWSM we have several high profile web properties so the majority of the traffic is http. Also, everything behind the FWSM is NATed with static NAT and we were using src-dst-ip for the ether-channel load-balancing algorithm on the 6500's. Generally traffic on the FWSM will exit the same port it was received on with several exceptions 1) Traffic inspected by the CP 2) Fragmented traffic 3) Packets forwarded between the NP's We were able to rule out the inspected and fragmented traffic pretty easily which just left option 3. The command 'show np 1 stats | inc blade' displays the number of packets that are forwarded from one NP to the other. When we issued this command twice with a 5 second pause in between we saw a significant increase in the counters. Here is an example of what happens when a client makes a connection to one of our sites 1) Packet comes from client destined to the server. 2) The SUP hashes the packet based on the src and dst IP and sends it out port 1 (NP 1) of the FWSM 3) Connection is created on NP1 4) IP header in the packet is NATed by the FWSM and sent out to destination server 5) Server replies back to the client 6) SUP hashes the packet again, but since one of the IPs has been NATed by the FWSM the packet is now hashed to port 5 (NP2) 7) FWSM receives the packet, but since the connection for this flow resides on NP1 it forwards the packet from NP2 to NP1 8) once the packet is processed by NP1 it is forwarded out port 3 (if we reversed it and NP2 did the processing it would have been forwarded out port 6) So the reason we saw the majority of traffic egress ports 3 and 6 was because the ether-channel hash and the static NAT. By changing the ether-channel load-balancing algorithm to src-dst-port we were able to get equal distribution of traffic out of the FWSM. On Wed, Oct 28, 2009 at 3:51 AM, <nm...@guesswho.com> wrote: > David, > > It appears that I might have misunderstood the original question since it > was only pertaining to traffic from the FWSM. My apologies. > > Thanks, > > Nick > > > > > > *From:* David White, Jr. (dwhitejr) [mailto:dwhit...@cisco.com] > *Sent:* Tuesday, October 27, 2009 10:32 PM > *To:* Nicholas Maio > *Cc:* j4b...@gmail.com; cisco-nsp@puck.nether.net > > *Subject:* Re: [c-nsp] FWSM traffic distribution across internal > etherchannel > > > > Hi Nick, > > Changing the SUP's load-balancing algorithm (which is what is described in > the link provided) only affects the traffic that egresses the switch and > ingresses the FWSM. It does not impact the packet distribution in the > reverse direction (egress the FWSM and ingress on the switch). > > I didn't indicate that I would need to know the traffic profile to > determine the correct SUP load-balancing algorithm, but rather to explain > why ports 3 and 6 were mainly utilized for traffic egressing the FWSM - > which was Jack's original question. > > Sincerely, > > David. > > nm...@guesswho.com wrote: > > David, > > The section named "Customizing the FWSM Internal Interface" in the following > page > > http://www.cisco.com/en/US/docs/security/fwsm/fwsm31/configuration/guide/switch_f.html > > would be helpful. > > > > As you stated you would need to know the traffic profile to detemine the > correct algorithm but why would you say that there aren't any commands to > change this? The command is not run in the fwsm but rather the switch/router. > > Nick > > > > > > ________________________________________ > > From: cisco-nsp-boun...@puck.nether.net [cisco-nsp-boun...@puck.nether.net] > On Behalf Of David White, Jr. (dwhitejr) [dwhit...@cisco.com] > > Sent: Tuesday, October 27, 2009 8:29 PM > > To: jack b > > Cc: cisco-nsp@puck.nether.net > > Subject: Re: [c-nsp] FWSM traffic distribution across internal etherchannel > > > > Hi Jack, > > > > Yes, it is most likely that this is normal. There are no CLI commands > > on the FWSM to adjust this. I would have to understand your traffic > > profile along with your config to tell you why the given profile is > > almost exclusively utilizing ports 3 and 6. > > > > Sincerely, > > > > David. > > > > > > jack b wrote: > > > > I have a FWSM running 2.3(4)11 in slot 4 of a 6509. I have noticed that I am > > getting unequal traffic distribution on the links that make up the ether > > channel bundle between the FWSM and 6509. > > > > Here is a snapshot of the traffic distribution > > > > 4/1 in 28.99mbps out 458.10mbps > > 4/2 in 12.37mbps out 248.31mbps > > 4/3 in 960.86mbps out 294.95mbps > > 4/4 in 34.07mbps out 505.22mbps > > 4/5 in 15.08mbps out 243.10mbps > > 4/6 in 950.63mbps out 262.68mbps > > > > In is traffic from the FWSM to the switch and out is traffic from the switch > > to the FWSM. > > > > Is this normal operation, or is there a way to distribute the traffic from > > the FWSM to the switch more evenly? > > _______________________________________________ > > cisco-nsp mailing list cisco-nsp@puck.nether.net > > https://puck.nether.net/mailman/listinfo/cisco-nsp > > archive at http://puck.nether.net/pipermail/cisco-nsp/ > > > > > > > > _______________________________________________ > > cisco-nsp mailing list cisco-nsp@puck.nether.net > > https://puck.nether.net/mailman/listinfo/cisco-nsp > > archive at http://puck.nether.net/pipermail/cisco-nsp/ > > > > _______________________________________________ cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp archive at http://puck.nether.net/pipermail/cisco-nsp/
