My understanding of this has always been that the policy is hierarchical, so you have your desired queuing policy in the "child" and the parent policy is just a shaper for class default. If you think about that, the parent shaper is now in control of how much traffic goes out the interface and the tx_ring doesn't really come into play. In fact, when you enable CBWFQ, the tx_ring is automatically set to 2 packets just to minimize any potential for packets to stack up in the tx_ring rather than the software queues.
So here's what happens: A packet is forwarded to the interface for egress, and is put through the parent policy and the shaper that is applied to class-default. If the shaper mechanism is ready to send that packet (sufficient tokens in the bucket for this time interval, etc.), the packet is transmitted out of the shaper. The tx_ring is really pretty irrelevant at this point, it's just the interface between software and hardware queues. If the shaper determines that it can't send that packet due to the shaper parameters, it's now experiencing congestion. At this point, the child policy kicks in and applies the LLQ, bandwidth reservations, and/or policers that are configured for the child policy. So the child applies to the traffic that has congested within the parent shaper. Note that until pretty recent IOS code (15.1 or 2 maybe? And IOS-XE), you couldn't do a shaper-in-shaper, only a policer. I've also always understood the hierarchical shaper policy to be purely FIFO, so it's interesting that someone claims theres fair-queueing at work there, that would screw with things I would think. Lastly, I concur with the policy counters being "rough." I have to assume that different counters are polled at different moments or the queues are just so dynamic that as you suggested values change during the output. Do remember we're talking about things that happen over intervals of milliseconds, here. I often have to caution customers of this as they will otherwise start getting confused/concerned when various counters don't all add up and they start asking where the difference is going. Hope that helps, Bob Sent from my iPad On Jul 14, 2012, at 9:27 AM, Moataz Mamdouh <[email protected]> wrote: > Hi > as per the diagram the software queues is the same as the shaping queue > so for example if some voice traffic need to be sent and the shaping is > active , this traffic is going to fill the LLQ ( SW queue) > > so why it's needed to configure bandwidth command under the LLQ class map > the packet are going to de-queued once there are free tokens in the bucket > and sent with rate equal to the interface bandwidth > > i hope that someone can add more on this > > > Moataz > > > ________________________________ > From: Kim Pedersen <[email protected]> > To: Keller Giacomarro <[email protected]> > Cc: CCIE IPExpert OSL <[email protected]> > Sent: Saturday, 14 July 2012, 12:41 > Subject: Re: [OSL | CCIE_RS] Shapers, CBWFQ, and tx_ring, oh my! > > Hi, > > The "real" software queues is after the shaper. > The shaper releases packets to a conformed rate to the software > queues, which the scheduler then uses (according to your > bandwidth/LLQ) to release to the TX ring. > > > Kim > > > On Sat, Jul 14, 2012 at 12:37 PM, Keller Giacomarro <[email protected]> > wrote: >> Great diagram, that does help. My question is -- with nested shaper/CBWFQ >> policy-maps, where do the CBWFQ queues fit into the diagram? Before or >> after the shaper? Or integrated somehow? >> >> My guess with the show commands is that when the router looks up the shaper >> queue depth, it is different than the split moment later when it looks up >> the CBWFQ queue depth. I've seen this before in show commands (try a 'show >> ip route' with RIP routes, and notice that as you page the route ages >> change!), just wanted to be sure my understanding wasn't the one at fault! >> >> Keller Giacomarro >> [email protected] >> >> >> >> On Sat, Jul 14, 2012 at 5:32 AM, Kim Pedersen <[email protected]> wrote: >>> >>> Hi, >>> >>> First of, take _all_ QoS counters and their relationship with a grain of >>> salt. >>> >>> Getting a clear indication of how things are working internally by >>> just using the counters will only mislead you. >>> A visualized example might clear it up for you: >>> >>> http://wiki.nil.com/Traffic_shaping_in_Cisco_IOS >>> >>> The bottom picture explains it. >>> >>> I dont think your understanding is wrong, i just think the counters >>> (and how they are increased internally) misleads you. >>> >>> Kim >>> >>> On Sat, Jul 14, 2012 at 12:20 PM, Keller Giacomarro <[email protected]> >>> wrote: >>>> Hi Kim, >>>> >>>> Thanks for the input, trying to wrap my head around what you're saying. >>>> >>>> If the shaper is putting packets into shaping queues to wait their turn >>>> to >>>> meet CIR, what causes the CBWFQ queues to fill up? There is no >>>> backpressure >>>> to tell the class-based queues to stack up -- they should be able to go >>>> straight to the tx_ring. If it works that way, I would expect to see >>>> the >>>> shaper working (overall traffic being throttled) but the class-based >>>> software queues seeing no depth at all. >>>> >>>> To clarify, I'm not doing per-class shaping. I'm shaping the entire >>>> interface, then feeding class-based queues. This is what I'm doing... >>>> >>>> policy-map pm-shaper >>>> class class-default >>>> shaper average percent 100 >>>> service-policy pm-cbwfq >>>> ! >>>> policy-map pm-cbwfq >>>> class cm-ssh >>>> bandwidth percent 5 >>>> class cm-http >>>> bandwidth percent 20 >>>> class class-default >>>> fair-queue >>>> >>>> Thanks again, looking forward to hearing back. >>>> >>>> Keller Giacomarro >>>> [email protected] >>>> >>>> >>>> >>>> On Sat, Jul 14, 2012 at 5:12 AM, Kim Pedersen <[email protected]> >>>> wrote: >>>>> >>>>> Hi, >>>>> >>>>> For the most part you are right. >>>>> >>>>> However, an important piece of the puzzle is the fact that shaping >>>>> queues are not the same as your regular software queues. >>>>> >>>>> If shaping is in effect (ie, the current packet needs to wait in >>>>> line), it gets put into one of your defined _shaping_ queues. From the >>>>> shaping queue(s) packets are then released to the software queues and >>>>> finally to your TX ring. >>>>> >>>>> Kim >>>>> >>>>> On Sat, Jul 14, 2012 at 12:07 PM, Keller Giacomarro >>>>> <[email protected]> >>>>> wrote: >>>>>> I am trying to wrap my head around how QoS queueing actually >>>>>> functions. >>>>>> If >>>>>> you're able, please confirm or debunk my understanding! >>>>>> >>>>>> Your 'normal' QoS setup involves a CBWFQ policy-map applied outbound >>>>>> on >>>>>> a >>>>>> WAN interface. As the tx_ring fills up, packets are queued into the >>>>>> CBWFQ >>>>>> policy-map queues (one per class-map), and are dequeued as normal. >>>>>> The >>>>>> tx_ring filling up is the trigger for filling queues. Okay, simple >>>>>> so >>>>>> far. >>>>>> >>>>>> The complication comes when you have an interface with a rate limit >>>>>> higher >>>>>> than your CIR (like a home Internet connection via 100Mbps ethernet >>>>>> with >>>>>> a >>>>>> CIR of 512Kbps). If CBWFQ is applied directly to the interface, even >>>>>> if >>>>>> the bandwidth is set, the tx_ring clears faster than the WAN circuit >>>>>> will >>>>>> take the data, and the software queues are bypassed entirely. In >>>>>> this >>>>>> situation, applying a CBWFQ policy-map directly to the interface, >>>>>> even >>>>>> setting the bandwidth command, does absolutely nothing. >>>>>> >>>>>> Here's where I get fuzzier. The solution to this is to put something >>>>>> else >>>>>> between the CBWFQ policy-map and the tx_ring: a shaper via nested >>>>>> policy >>>>>> maps. The shaper is configured to the correct CIR. As the shaper >>>>>> sees >>>>>> that the interface is transmitting too fast, it begins to fill up the >>>>>> CBWFQ >>>>>> policy-map queues instead of transmitting. In this way, the physical >>>>>> interface is faster than the CIR but we still create the necessary >>>>>> 'backpressure' to fill up the software queues. >>>>>> >>>>>> Two things muck with my understanding: >>>>>> In https://supportforums.cisco.com/thread/2132501 , a Cisco employee >>>>>> says >>>>>> that the shaper uses WFQ (or HQF in the newest releases) to de-queue >>>>>> the >>>>>> CBWFQ queues. Why is the shaper implementing any dequeueing strategy >>>>>> at >>>>>> all? Shouldn't the CBWFQ policy-map be handling that (such as policy >>>>>> queues going first, etc)? And how can it possibly do that without >>>>>> full >>>>>> flow information? >>>>>> >>>>>> The other issue is that the show commands on my router support my >>>>>> understanding...almost. If I'm moving a lot of ssh data upstream >>>>>> (via >>>>>> scp), I can see the shaper queue fill and the CBWFQ queue fill, makes >>>>>> sense. Most of the time their values are the same. However, they >>>>>> do >>>>>> on occasion differ by a number or two. Show command artifact, or an >>>>>> indication that I have no idea what I'm talking about? >>>>>> >>>>>> gateway#show policy-map int f0/1 >>>>>> FastEthernet0/1 >>>>>> >>>>>> Service-policy output: pm-wan-out-shaper >>>>>> >>>>>> Class-map: class-default (match-any) >>>>>> 600317 packets, 143960388 bytes >>>>>> 30 second offered rate 631000 bps, drop rate 14000 bps >>>>>> Match: any >>>>>> Traffic Shaping >>>>>> Target/Average Byte Sustain Excess Interval >>>>>> Increment >>>>>> Rate Limit bits/int bits/int (ms) >>>>>> (bytes) >>>>>> 600000/600000 937 3750 3750 6 468 >>>>>> >>>>>> Adapt *Queue* Packets Bytes Packets Bytes >>>>>> Shaping >>>>>> Active *Depth* Delayed Delayed >>>>>> Active >>>>>> - *3* 598394 141284492 90698 94225802 >>>>>> yes >>>>>> >>>>>> Class-map: cm-ssh (match-all) >>>>>> 70238 packets, 105192011 bytes >>>>>> 30 second offered rate 621000 bps, drop rate 14000 bps >>>>>> Match: protocol ssh >>>>>> Queueing >>>>>> Output Queue: Conversation 74 >>>>>> Bandwidth 5 (%) >>>>>> Bandwidth 30 (kbps) >>>>>> (pkts matched/bytes matched) 62464/93671468 >>>>>> *(depth/total drops/no-buffer drops) 2*/1816/3 >>>>>> exponential weight: 9 >>>>>> mean queue depth: 1 >>>>>> >>>>>> Appreciate your input -- hopefully this helps someone else too, as >>>>>> none >>>>>> of >>>>>> the standard study resources I've read have adequately explained how >>>>>> this >>>>>> works! >>>>>> >>>>>> Keller Giacomarro >>>>>> [email protected] >>>>>> _______________________________________________ >>>>>> For more information regarding industry leading CCIE Lab training, >>>>>> please visit www.ipexpert.com >>>>>> >>>>>> Are you a CCNP or CCIE and looking for a job? Check out >>>>>> www.PlatinumPlacement.com >>>>>> >>>>>> http://onlinestudylist.com/mailman/listinfo/ccie_rs >>>>> >>>>> >>>>> >>>>> -- >>>>> // Freedom Matters >>>>> // CCIE #29189 >>>>> // www.packet-forwarding.net >>>> >>>> >> >> > > > > -- > // Freedom Matters > // CCIE #29189 > // www.packet-forwarding.net > _______________________________________________ > For more information regarding industry leading CCIE Lab training, please > visit www.ipexpert.com > > Are you a CCNP or CCIE and looking for a job? Check out > www.PlatinumPlacement.com > > http://onlinestudylist.com/mailman/listinfo/ccie_rs > _______________________________________________ > For more information regarding industry leading CCIE Lab training, please > visit www.ipexpert.com > > Are you a CCNP or CCIE and looking for a job? Check out > www.PlatinumPlacement.com > > http://onlinestudylist.com/mailman/listinfo/ccie_rs _______________________________________________ For more information regarding industry leading CCIE Lab training, please visit www.ipexpert.com Are you a CCNP or CCIE and looking for a job? Check out www.PlatinumPlacement.com http://onlinestudylist.com/mailman/listinfo/ccie_rs
