Re: [LARTC] More on qdiscs
Hi, only a few notes on the theme. You are right with the displacement and bad enqueue byte counters. Maybe it would be better to cound packets at dequeue time only in clasfull qdisc. It also makes better sense because qdosc can also instruct SFQ to drop packet - this drop is then not decremented from HTB/CBQ's stats. HTB uses enqueue time counting because CBQ does it too and at very begining I based my work on CBQ. Seems it is time to change things. Another important info: you really CAN'T drop packets in dequeue routine if you don't want to fool classfull parent. Many logic in CBQ/HTB/ATM/PRIO qdisc is based on the existence of backlog. When you drop in dequeue, parent will think that he itself still has some packets somewhere (in children) and will constantly attempt to find them. And will be confused by the fact that it can't. Enqueue routine can give you confirmation of packet enqueue, drop routine the same. Only dequeue can't say hey there is your skb and by the way two others was dropped. What a pity. SOLUTIONS: One way is to monitor (store) q.qlen variable of all children of classfull qdisc. When I call enqueue/drop/dequeue on it I'll see its discrepancy agains last stored value and will update my counter accordingly. In the same way we could add q.bytelen and then we would be able to do the same for bytes - but this is not neccessary probably as we need to know only bytes dequeued ... There is other way - add parameter to dequeue routine which tells us how many packets we should out qlen decrease by. But I think that former approach is simpler and prone to silent drops (for example by some timer). Don do you plan to implement these corrections for CBQ/PRIO ? I'll do the change for HTB. devik On Sat, 4 May 2002, Don Cohen wrote: I notice one other small problem with my modified version of SFQ. The fact that packets can be dropped at dequeue time is incompatible with the way HTB (and probably CBQ and others modeled on it) keep statistics. When I fill a low rate queue causing packets to expire and be dropped at dequeue I get interesting statistics like this: This is my variant of SFQ qdisc plfq 8016: dev eth1 ... Sent 17468 bytes 166 pkts (dropped 833, overlimits 0) ... qdisc htb 1: dev eth1 r2q 10 default 2 dcache 0 deq_util 1/100 deq_rate 0 trials_per_deq 0 dcache_hits 0 direct_packets 0 Sent 32626 bytes 309 pkts (dropped 690, overlimits 1126) backlog 143p Note that plfq, which is inside of htb, thinks it has dropped more packets than htb has. The reason is that htb considers a packet to be sent when it's enqueued. I notice in retrospect that even with current SFQ, the statistics are not correct. The reason is that it's possible to successfully enqueue packet#1 (at which point HTB adds its length to the number of bytes sent), then fail to enqueue packet#2. But within SFQ this failure might really mean that packet#2 was accepted to be sent later and it displaced packet#1. It happens that this replacement keeps the packet count correct, but not the byte count. One can imagine another version of SFQ that, instead of limiting the number of packets in the queue, limits the number of bytes. In that case the number of packets would not be correct either. I think that HTB (and other classful qdiscs) should count the number of bytes and packets sent when they're dequeued, not when they're enqueued. It's an interesting point that, at the moment, I don't see any way for the classful qdisc to know how many packets and bytes were dropped. However, I think it would be useful to know instead how many were passed to enqueue. Then you'd get a report like Enqueued 32626 bytes 309 pkts, Dequeued 17468 bytes 166 pkts where the difference is the sum of what's been dropped and what's still in the queue. ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: [LARTC] IMQ question
The first line only mark packet. After it leaves iptables it will enter IMQ because of the mark. devik On Sun, 5 May 2002, Ciprian Niculescu wrote: whitch is the logic in this peace of code iptables -t mangle -A PREROUTING -j IMQ iptables -t mangle -A PREROUTING -s 1.2.3.4 -j MARK --set-mark 1 and in the imq0 i have a class that handle the pachets marked with 1 this works, the question is why? beacouse i understand that: the pachet first enter the IMQ table, and in the queue, after that he exit and continue with the next rule in the PREROUTING, witch is the mark. ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
[LARTC] bad idee or?
i want to do: iptables -t mangle -A PREROUTING -j MARK multiples marks with different mark iptables -t mangle -A POSTROUTING ... -j MARK ... multiples marks with different mark iptables -t mangle -A POSTROUTING -j IMQ practicaly i want to prioritise/limit all the trafic in and out, by different complex/wheird rules on an linux router with 4 real interfaces and 3 more virtual, the trafic is at about 5Mbit with the ISP, and spikes of 20Mbit on some local interfaces. so the ideea is too mark the trafic in the PRE, and POST ROUTING table (this is the place where i have all the trafic), and at the end of the ROUTING process to put all the trafic throw IMQ, shape/prio it and the let it go to the real interfaces where i do some ganeral interface limiting so the ideea is good or not :) C ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: [LARTC] bad idee or?
It how it is supposed to work IMHO :) On Sun, 5 May 2002, Ciprian Niculescu wrote: i want to do: iptables -t mangle -A PREROUTING -j MARK multiples marks with different mark so the ideea is too mark the trafic in the PRE, and POST ROUTING table (this is the place where i have all the trafic), and at the end of the ROUTING process to put all the trafic throw IMQ, shape/prio it and the let it go to the real interfaces where i do some ganeral interface limiting so the ideea is good or not :) ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
[LARTC] Re: More on qdiscs - about dangling backlogs
Martin Devera writes: Hi, only a few notes on the theme. You are right with the displacement and bad enqueue byte counters. Maybe it would be better to cound packets at dequeue time only in clasfull qdisc. It also makes better sense because qdosc can also instruct SFQ to drop packet - this drop I don't understand. What other qdisc instructs SFQ to drop? is then not decremented from HTB/CBQ's stats. HTB uses enqueue time counting because CBQ does it too and at very I figured that's why it worked that way. begining I based my work on CBQ. Seems it is time to change things. Another important info: you really CAN'T drop packets in dequeue routine if you don't want to fool classfull parent. Many logic in CBQ/HTB/ATM/PRIO qdisc is based on the existence of backlog. I gather you only care whether this qdisc is waiting to send, not how much is in the queue. When you drop in dequeue, parent will think that he itself still has some packets somewhere (in children) and will constantly attempt to find them. And will be confused by the fact that it can't. Enqueue routine can give you confirmation of packet enqueue, drop routine the same. Only dequeue can't say hey there is your skb and by the way two others was dropped. What a pity. SOLUTIONS: One way is to monitor (store) q.qlen variable of all children of classfull qdisc. When I call enqueue/drop/dequeue on it I'll see its discrepancy agains last stored value and will update my counter This seems like the right solution, except that you shouldn't even need to store a counter. Just use the one in the child qdisc. BTW, I think you are justified in assuming that this counter can only change in two ways: decrease when dequeue is called (but possibly by more than one packet) and increase by 1 when enqueue is called. And I guess also requeue. Which raises another problem, since that's not called by the parent, right? accordingly. In the same way we could add q.bytelen and then we would be able to do the same for bytes - but this is not neccessary probably as we need to know only bytes dequeued ... My impression is that every qdisc is currently supposed to keep track of # packets in the queue, but not necessarily # bytes. Best not to change that, especially since we don't even have any currently proposed use for it. There is other way - add parameter to dequeue routine which tells us how many packets we should out qlen decrease by. But I think that former approach is simpler and prone to silent drops (for example by some timer). I think you're saying the former approach is better, but prone suggests otherwise. I prefer the former approach. It's better for all the other qdiscs out there to keep working without change. There are only a few classful qdiscs that would have to change, and they could still work unchanged with qdiscs that don't do silent drops. Don do you plan to implement these corrections for CBQ/PRIO ? Not any time soon. The last time I tried to read CBQ it was in order to figure out what all those parameters mean, and I didn't get very far. Now that I no longer use it I have less incentive. I'll do the change for HTB. Thanks. But which ones? It sounds like you plan to - count as sent only what's returned from dequeue - ignore the return value of enqueue and check the queue length to see whether it's empty I like those, but also suggested that HTB would be a good place to add code for dropping stale packets. If you're interested in doing that I should mention: Not all packets come with timestamps (e.g., locally generated ones). Solution: at enqueue, if the timestamp is 0, set it to the current time. ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
