Based on my testing, the bandwidth values configured in the CBWFQ policy-map definitely affect the bandwidth allocated for each class. If I upload something via ssh and via http at the same time, changing the bandwidth values for each drastically affects the actual speeds I'm getting.
These things I know: - the shaper is working - applying the CBWFQ policy-map directly to the interface sans shaper does nothing (no reserved bandwidth for anything) - when the CBWFQ poilcy-map is nested inside the shaper policy-map, shaping works, and traffic is balanced properly Now, as to how the internals work of feeding queues and display vs actual router actions, I have no idea. =D Keller Giacomarro [email protected] On Sat, Jul 14, 2012 at 5:50 AM, Kim Pedersen <[email protected]> wrote: > My understanding of this phenomenon (and i havent seen any other good > explanation of it) is that its an internal counter thing. I dont > actually think your software queues fill up. And i think you could > verify this with an actual file transfer. > > Kim > > On Sat, Jul 14, 2012 at 12:45 PM, Keller Giacomarro <[email protected]> > wrote: > > Okay, got it. > > > > One last question! If the shaping is already done by the time the real > > software queues are hit, what causes the software queues to stack up? > Why > > don't they get bypassed like they do if I apply the policy-map directly > to > > the physical interface? There is nothing in the tx_ring, so why do the > > CBWFQ queues increase? > > > > Thanks again. > > > > Keller Giacomarro > > [email protected] > > > > > > > > On Sat, Jul 14, 2012 at 5:41 AM, Kim Pedersen <[email protected]> > wrote: > >> > >> Hi, > >> > >> The "real" software queues is after the shaper. > >> The shaper releases packets to a conformed rate to the software > >> queues, which the scheduler then uses (according to your > >> bandwidth/LLQ) to release to the TX ring. > >> > >> > >> Kim > >> > >> > >> On Sat, Jul 14, 2012 at 12:37 PM, Keller Giacomarro <[email protected] > > > >> wrote: > >> > Great diagram, that does help. My question is -- with nested > >> > shaper/CBWFQ > >> > policy-maps, where do the CBWFQ queues fit into the diagram? Before > or > >> > after the shaper? Or integrated somehow? > >> > > >> > My guess with the show commands is that when the router looks up the > >> > shaper > >> > queue depth, it is different than the split moment later when it looks > >> > up > >> > the CBWFQ queue depth. I've seen this before in show commands (try a > >> > 'show > >> > ip route' with RIP routes, and notice that as you page the route ages > >> > change!), just wanted to be sure my understanding wasn't the one at > >> > fault! > >> > > >> > Keller Giacomarro > >> > [email protected] > >> > > >> > > >> > > >> > On Sat, Jul 14, 2012 at 5:32 AM, Kim Pedersen <[email protected]> > >> > wrote: > >> >> > >> >> Hi, > >> >> > >> >> First of, take _all_ QoS counters and their relationship with a > grain > >> >> of > >> >> salt. > >> >> > >> >> Getting a clear indication of how things are working internally by > >> >> just using the counters will only mislead you. > >> >> A visualized example might clear it up for you: > >> >> > >> >> http://wiki.nil.com/Traffic_shaping_in_Cisco_IOS > >> >> > >> >> The bottom picture explains it. > >> >> > >> >> I dont think your understanding is wrong, i just think the counters > >> >> (and how they are increased internally) misleads you. > >> >> > >> >> Kim > >> >> > >> >> On Sat, Jul 14, 2012 at 12:20 PM, Keller Giacomarro > >> >> <[email protected]> > >> >> wrote: > >> >> > Hi Kim, > >> >> > > >> >> > Thanks for the input, trying to wrap my head around what you're > >> >> > saying. > >> >> > > >> >> > If the shaper is putting packets into shaping queues to wait their > >> >> > turn > >> >> > to > >> >> > meet CIR, what causes the CBWFQ queues to fill up? There is no > >> >> > backpressure > >> >> > to tell the class-based queues to stack up -- they should be able > to > >> >> > go > >> >> > straight to the tx_ring. If it works that way, I would expect to > see > >> >> > the > >> >> > shaper working (overall traffic being throttled) but the > class-based > >> >> > software queues seeing no depth at all. > >> >> > > >> >> > To clarify, I'm not doing per-class shaping. I'm shaping the > entire > >> >> > interface, then feeding class-based queues. This is what I'm > >> >> > doing... > >> >> > > >> >> > policy-map pm-shaper > >> >> > class class-default > >> >> > shaper average percent 100 > >> >> > service-policy pm-cbwfq > >> >> > ! > >> >> > policy-map pm-cbwfq > >> >> > class cm-ssh > >> >> > bandwidth percent 5 > >> >> > class cm-http > >> >> > bandwidth percent 20 > >> >> > class class-default > >> >> > fair-queue > >> >> > > >> >> > Thanks again, looking forward to hearing back. > >> >> > > >> >> > Keller Giacomarro > >> >> > [email protected] > >> >> > > >> >> > > >> >> > > >> >> > On Sat, Jul 14, 2012 at 5:12 AM, Kim Pedersen <[email protected]> > >> >> > wrote: > >> >> >> > >> >> >> Hi, > >> >> >> > >> >> >> For the most part you are right. > >> >> >> > >> >> >> However, an important piece of the puzzle is the fact that > shaping > >> >> >> queues are not the same as your regular software queues. > >> >> >> > >> >> >> If shaping is in effect (ie, the current packet needs to wait in > >> >> >> line), it gets put into one of your defined _shaping_ queues. From > >> >> >> the > >> >> >> shaping queue(s) packets are then released to the software queues > >> >> >> and > >> >> >> finally to your TX ring. > >> >> >> > >> >> >> Kim > >> >> >> > >> >> >> On Sat, Jul 14, 2012 at 12:07 PM, Keller Giacomarro > >> >> >> <[email protected]> > >> >> >> wrote: > >> >> >> > I am trying to wrap my head around how QoS queueing actually > >> >> >> > functions. > >> >> >> > If > >> >> >> > you're able, please confirm or debunk my understanding! > >> >> >> > > >> >> >> > Your 'normal' QoS setup involves a CBWFQ policy-map applied > >> >> >> > outbound > >> >> >> > on > >> >> >> > a > >> >> >> > WAN interface. As the tx_ring fills up, packets are queued into > >> >> >> > the > >> >> >> > CBWFQ > >> >> >> > policy-map queues (one per class-map), and are dequeued as > normal. > >> >> >> > The > >> >> >> > tx_ring filling up is the trigger for filling queues. Okay, > >> >> >> > simple > >> >> >> > so > >> >> >> > far. > >> >> >> > > >> >> >> > The complication comes when you have an interface with a rate > >> >> >> > limit > >> >> >> > higher > >> >> >> > than your CIR (like a home Internet connection via 100Mbps > >> >> >> > ethernet > >> >> >> > with > >> >> >> > a > >> >> >> > CIR of 512Kbps). If CBWFQ is applied directly to the interface, > >> >> >> > even > >> >> >> > if > >> >> >> > the bandwidth is set, the tx_ring clears faster than the WAN > >> >> >> > circuit > >> >> >> > will > >> >> >> > take the data, and the software queues are bypassed entirely. > In > >> >> >> > this > >> >> >> > situation, applying a CBWFQ policy-map directly to the > interface, > >> >> >> > even > >> >> >> > setting the bandwidth command, does absolutely nothing. > >> >> >> > > >> >> >> > Here's where I get fuzzier. The solution to this is to put > >> >> >> > something > >> >> >> > else > >> >> >> > between the CBWFQ policy-map and the tx_ring: a shaper via > nested > >> >> >> > policy > >> >> >> > maps. The shaper is configured to the correct CIR. As the > shaper > >> >> >> > sees > >> >> >> > that the interface is transmitting too fast, it begins to fill > up > >> >> >> > the > >> >> >> > CBWFQ > >> >> >> > policy-map queues instead of transmitting. In this way, the > >> >> >> > physical > >> >> >> > interface is faster than the CIR but we still create the > necessary > >> >> >> > 'backpressure' to fill up the software queues. > >> >> >> > > >> >> >> > Two things muck with my understanding: > >> >> >> > In https://supportforums.cisco.com/thread/2132501 , a Cisco > >> >> >> > employee > >> >> >> > says > >> >> >> > that the shaper uses WFQ (or HQF in the newest releases) to > >> >> >> > de-queue > >> >> >> > the > >> >> >> > CBWFQ queues. Why is the shaper implementing any dequeueing > >> >> >> > strategy > >> >> >> > at > >> >> >> > all? Shouldn't the CBWFQ policy-map be handling that (such as > >> >> >> > policy > >> >> >> > queues going first, etc)? And how can it possibly do that > without > >> >> >> > full > >> >> >> > flow information? > >> >> >> > > >> >> >> > The other issue is that the show commands on my router support > my > >> >> >> > understanding...almost. If I'm moving a lot of ssh data > upstream > >> >> >> > (via > >> >> >> > scp), I can see the shaper queue fill and the CBWFQ queue fill, > >> >> >> > makes > >> >> >> > sense. Most of the time their values are the same. However, > >> >> >> > they > >> >> >> > do > >> >> >> > on occasion differ by a number or two. Show command artifact, > or > >> >> >> > an > >> >> >> > indication that I have no idea what I'm talking about? > >> >> >> > > >> >> >> > gateway#show policy-map int f0/1 > >> >> >> > FastEthernet0/1 > >> >> >> > > >> >> >> > Service-policy output: pm-wan-out-shaper > >> >> >> > > >> >> >> > Class-map: class-default (match-any) > >> >> >> > 600317 packets, 143960388 bytes > >> >> >> > 30 second offered rate 631000 bps, drop rate 14000 bps > >> >> >> > Match: any > >> >> >> > Traffic Shaping > >> >> >> > Target/Average Byte Sustain Excess Interval > >> >> >> > Increment > >> >> >> > Rate Limit bits/int bits/int (ms) > >> >> >> > (bytes) > >> >> >> > 600000/600000 937 3750 3750 6 > >> >> >> > 468 > >> >> >> > > >> >> >> > Adapt *Queue* Packets Bytes Packets Bytes > >> >> >> > Shaping > >> >> >> > Active *Depth* Delayed Delayed > >> >> >> > Active > >> >> >> > - *3* 598394 141284492 90698 > 94225802 > >> >> >> > yes > >> >> >> > > >> >> >> > Class-map: cm-ssh (match-all) > >> >> >> > 70238 packets, 105192011 bytes > >> >> >> > 30 second offered rate 621000 bps, drop rate 14000 bps > >> >> >> > Match: protocol ssh > >> >> >> > Queueing > >> >> >> > Output Queue: Conversation 74 > >> >> >> > Bandwidth 5 (%) > >> >> >> > Bandwidth 30 (kbps) > >> >> >> > (pkts matched/bytes matched) 62464/93671468 > >> >> >> > *(depth/total drops/no-buffer drops) 2*/1816/3 > >> >> >> > exponential weight: 9 > >> >> >> > mean queue depth: 1 > >> >> >> > > >> >> >> > Appreciate your input -- hopefully this helps someone else too, > as > >> >> >> > none > >> >> >> > of > >> >> >> > the standard study resources I've read have adequately explained > >> >> >> > how > >> >> >> > this > >> >> >> > works! > >> >> >> > > >> >> >> > Keller Giacomarro > >> >> >> > [email protected] > >> >> >> > _______________________________________________ > >> >> >> > For more information regarding industry leading CCIE Lab > training, > >> >> >> > please visit www.ipexpert.com > >> >> >> > > >> >> >> > Are you a CCNP or CCIE and looking for a job? Check out > >> >> >> > www.PlatinumPlacement.com > >> >> >> > > >> >> >> > http://onlinestudylist.com/mailman/listinfo/ccie_rs > >> >> >> > >> >> >> > >> >> >> > >> >> >> -- > >> >> >> // Freedom Matters > >> >> >> // CCIE #29189 > >> >> >> // www.packet-forwarding.net > >> >> > > >> >> > > >> > > >> > > >> > >> > >> > >> -- > >> // Freedom Matters > >> // CCIE #29189 > >> // www.packet-forwarding.net > > > > > > > > -- > // Freedom Matters > // CCIE #29189 > // www.packet-forwarding.net > _______________________________________________ For more information regarding industry leading CCIE Lab training, please visit www.ipexpert.com Are you a CCNP or CCIE and looking for a job? Check out www.PlatinumPlacement.com http://onlinestudylist.com/mailman/listinfo/ccie_rs
