Re: [LARTC] Re: how flexible is ingress traffic policing to bandwidth limit?
Greg Stark wrote: Jason Boxman <[EMAIL PROTECTED]> writes: On Tuesday 08 June 2004 23:33, Greg Stark wrote: Well ultimately all shaping works by dropping packets. Merely delaying transmission isn't going to slow down anything in the long run, just increase the pipeline. You can delay and/or drop them after they've arrived just as easily. Though it would have to be before they're ack'd and delivered to the user. That's basically what IMQ does, I'm just saying perhaps that should just work instead of requiring a fake interface. Ultimately, packets from a misbehaving flow can be dropped, but it does not always come to a drop. When you shape on egress, you force applications on the local network to throttle back, believing they're sending as fast as the receiver can receive. As you delay, TCP figures it out. I don't think so. It may look like that's what's happening, but at least for TCP I don't think it works that way. As long as packets aren't being dropped then TCP will just continue to grow the window, interpreting this delay as just a longer pipeline that needs filling. Applications will be slowed down temporarily because it takes time to do this, but they'll eventually be outputting data just as fast as an application without a shaper. Eventually in the egress case either you will run out of buffer and drop, or your queue is big enough for the advertised window (say 32k-64k) and tcp will only add a packet for everyone acked - you decide when the packets go so you have good control. If you drop, the sender reduces it's cwin and only slowly tries to increase until another drop. Only when packets get dropped or are delayed so long that the client retransmits does TCP scale back the transmit window. And only when that happens does the client see any reduced bandwidth. So if your shaper isn't dropping packets it's just evening out the flow of data, not actually affecting the net rate the clients can pump out data. Once you have dropped the rate you release packets does controll the flow and if it's slow the cwin will only grow slowly as it's clocked by acks (I think). Contrast that with ingress, where the packets you want to delay are already on their way. Well on egress the packets are "already on their way" as well, after all. They're just haven't gone as many hops. Even for locally generated traffic the egress qdisc is being run after the data packetized and ready to go. I think what Jason was getting at is that on egress you have total control - it doesn't matter what TCP does. Ingress shaping is harder as packets are already headed for you - in the case of P2P possibly quite alot - it's easier to shape when peering with "real" servers. Though in both cases, the fact that TCP slow (but sort of exponential) start overshoots bandwidth hurts ingress shaping, but not egress. Andy. Hm, I wonder if I want RED or something similar to ensure packets get dropped fast enough instead of filling HTB queues and then dropping. If you're curious about RED, here's a possible example implementation for ingress policing: http://digriz.org.uk/jdg-qos-script/ This is an interesting script. It looks like a successor to wondershaper. But I'm a bit too deep in my own re-implementation of wondershaper now to start over. ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: [LARTC] Re: how flexible is ingress traffic policing to bandwidth limit?
On Wednesday 09 June 2004 16:09, Greg Stark wrote: > Sanjay Arora <[EMAIL PROTECTED]> writes: > > Sorry to interrupt the flow, especially being a newbie, but won´t the > > sender just retransmit the dropped packets at the same rate? > > no. > > > I am not so thorogh with TCP/IP, but is there something in the protocol > > that speeds or slows the transmission. > > yes. > > > Please do explain in the TCP/IP for complete idiots terminology ;-)) > > You could do worse than this: > > http://www.thinkingsecure.com/docs/TCPIP-Illustrated-1/tcp_time.htm#21_0 I have to admit, I am thus far thoroughly enjoying: Jacobson, V., Congestion Avoidance and Control. Proceedings of ACM SIGCOMM '88. August 1988, p. 314-329. It's also available in Postscript format: ftp://ftp.ee.lbl.gov/papers/congavoid.ps.Z For those who don't know how TCP handles congestion, this is a rather excellent read. ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
[LARTC] Re: how flexible is ingress traffic policing to bandwidth limit?
Sanjay Arora <[EMAIL PROTECTED]> writes: > Sorry to interrupt the flow, especially being a newbie, but won´t the > sender just retransmit the dropped packets at the same rate? no. > I am not so thorogh with TCP/IP, but is there something in the protocol that > speeds or slows the transmission. yes. > Please do explain in the TCP/IP for complete idiots terminology ;-)) You could do worse than this: http://www.thinkingsecure.com/docs/TCPIP-Illustrated-1/tcp_time.htm#21_0 -- greg ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: [LARTC] Re: how flexible is ingress traffic policing to bandwidth limit?
On Wed, 2004-06-09 at 09:03, Greg Stark wrote: > Damion de Soto <[EMAIL PROTECTED]> writes: > > > because you can't shape inbound traffic. Shaping works by delaying the > > transmission, and you can't delay packets that haven't arrived yet. Ingress > > policing just drops packets, and hopes the sender will slow down. > Sorry to interrupt the flow, especially being a newbie, but wonÂt the sender just retransmit the dropped packets at the same rate? I am not so thorogh with TCP/IP, but is there something in the protocol that speeds or slows the transmission. Please do explain in the TCP/IP for complete idiots terminology ;-)) With best regards. Sanjay. ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
[LARTC] Re: how flexible is ingress traffic policing to bandwidth limit?
Jason Boxman <[EMAIL PROTECTED]> writes: > On Tuesday 08 June 2004 23:33, Greg Stark wrote: > > > > Well ultimately all shaping works by dropping packets. Merely delaying > > transmission isn't going to slow down anything in the long run, just > > increase the pipeline. You can delay and/or drop them after they've arrived > > just as easily. Though it would have to be before they're ack'd and > > delivered to the user. That's basically what IMQ does, I'm just saying > > perhaps that should just work instead of requiring a fake interface. > > Ultimately, packets from a misbehaving flow can be dropped, but it does not > always come to a drop. When you shape on egress, you force applications on > the local network to throttle back, believing they're sending as fast as the > receiver can receive. As you delay, TCP figures it out. I don't think so. It may look like that's what's happening, but at least for TCP I don't think it works that way. As long as packets aren't being dropped then TCP will just continue to grow the window, interpreting this delay as just a longer pipeline that needs filling. Applications will be slowed down temporarily because it takes time to do this, but they'll eventually be outputting data just as fast as an application without a shaper. Only when packets get dropped or are delayed so long that the client retransmits does TCP scale back the transmit window. And only when that happens does the client see any reduced bandwidth. So if your shaper isn't dropping packets it's just evening out the flow of data, not actually affecting the net rate the clients can pump out data. > Contrast that with ingress, where the packets you want to delay are already > on their way. Well on egress the packets are "already on their way" as well, after all. They're just haven't gone as many hops. Even for locally generated traffic the egress qdisc is being run after the data packetized and ready to go. > > Hm, I wonder if I want RED or something similar to ensure packets get > > dropped fast enough instead of filling HTB queues and then dropping. > > If you're curious about RED, here's a possible example implementation for > ingress policing: > > http://digriz.org.uk/jdg-qos-script/ This is an interesting script. It looks like a successor to wondershaper. But I'm a bit too deep in my own re-implementation of wondershaper now to start over. -- greg ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: [LARTC] Re: how flexible is ingress traffic policing to bandwidth limit?
On Tuesday 08 June 2004 23:33, Greg Stark wrote: > Damion de Soto <[EMAIL PROTECTED]> writes: > > > > because you can't shape inbound traffic. Shaping works by delaying the > > transmission, and you can't delay packets that haven't arrived yet. > > Ingress policing just drops packets, and hopes the sender will slow down. > > Well ultimately all shaping works by dropping packets. Merely delaying > transmission isn't going to slow down anything in the long run, just > increase the pipeline. You can delay and/or drop them after they've arrived > just as easily. Though it would have to be before they're ack'd and > delivered to the user. That's basically what IMQ does, I'm just saying > perhaps that should just work instead of requiring a fake interface. Ultimately, packets from a misbehaving flow can be dropped, but it does not always come to a drop. When you shape on egress, you force applications on the local network to throttle back, believing they're sending as fast as the receiver can receive. As you delay, TCP figures it out. Contrast that with ingress, where the packets you want to delay are already on their way. > Hm, I wonder if I want RED or something similar to ensure packets get > dropped fast enough instead of filling HTB queues and then dropping. If you're curious about RED, here's a possible example implementation for ingress policing: http://digriz.org.uk/jdg-qos-script/ -- Jason Boxman Perl Programmer / *NIX Systems Administrator Shimberg Center for Affordable Housing | University of Florida http://edseek.com/ - Linux and FOSS stuff ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
[LARTC] Re: how flexible is ingress traffic policing to bandwidth limit?
Damion de Soto <[EMAIL PROTECTED]> writes: > Greg, > > For some reason that hadn't occurred to me. That should work just fine. I > > guess I should mark the packets in iptables to avoid throttling traffic from > > gateway itself, or does match see the external ip? > > The only (common) time you need to use iptables to mark traffic, is when > you're using egress qdiscs on your outbound (WAN) interface (to shape upload > speeds) and want to filter based on private (LAN) IP address. Well I'll be doing the opposite here, putting an egress qdisc on my LAN interface limiting the downstream bandwidth. So I would have the equivalent need, to distinguish traffic originating from outside vs traffic originating from the router so I can still scp kernels over to the router without being limited. Hm, though in practice I hardly ever scp things _from_ the router so perhaps I don't care. > > IMQ does seem like a handy tool, but why is there a distinction at all between > > egress and ingress qdiscs at all? Why not just allow people to attach HTB as > > an ingress qdisc directly? > > because you can't shape inbound traffic. Shaping works by delaying the > transmission, and you can't delay packets that haven't arrived yet. Ingress > policing just drops packets, and hopes the sender will slow down. Well ultimately all shaping works by dropping packets. Merely delaying transmission isn't going to slow down anything in the long run, just increase the pipeline. You can delay and/or drop them after they've arrived just as easily. Though it would have to be before they're ack'd and delivered to the user. That's basically what IMQ does, I'm just saying perhaps that should just work instead of requiring a fake interface. Hm, I wonder if I want RED or something similar to ensure packets get dropped fast enough instead of filling HTB queues and then dropping. > > I suppose in an ideal world the best thing would be to receive the packets, > > hand them to user-space, but not mark them as received, ie, not ack them. That > > avoids introducing any loss from the user point of view but still slows the > > flow down. But that seems fiddly and would only work for TCP I guess. > > That's an interesting idea, but yeah... i think it might be a bit hard to > implement, and violate far too many RFCs... My original thought was to just drop an ACK packet, which would clearly not violate any assumptions. But if there's data in the ACK packet that could have negative effects. I suspect this lies well into Bad Idea(tm) realm, I'm just thinking out loud. -- greg ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
[LARTC] Re: how flexible is ingress traffic policing to bandwidth limit?
Greg, For some reason that hadn't occurred to me. That should work just fine. I guess I should mark the packets in iptables to avoid throttling traffic from gateway itself, or does match see the external ip? The only (common) time you need to use iptables to mark traffic, is when you're using egress qdiscs on your outbound (WAN) interface (to shape upload speeds) and want to filter based on private (LAN) IP address. IMQ does seem like a handy tool, but why is there a distinction at all between egress and ingress qdiscs at all? Why not just allow people to attach HTB as an ingress qdisc directly? because you can't shape inbound traffic. Shaping works by delaying the transmission, and you can't delay packets that haven't arrived yet. Ingress policing just drops packets, and hopes the sender will slow down. I suppose in an ideal world the best thing would be to receive the packets, hand them to user-space, but not mark them as received, ie, not ack them. That avoids introducing any loss from the user point of view but still slows the flow down. But that seems fiddly and would only work for TCP I guess. That's an interesting idea, but yeah... i think it might be a bit hard to implement, and violate far too many RFCs... regards, -- ~~~ Damion de Soto - Software Engineer email: [EMAIL PROTECTED] SnapGear - A CyberGuard Company ---ph: +61 7 3435 2809 | Custom Embedded Solutions fax: +61 7 3891 3630 | and Security Appliancesweb: http://www.snapgear.com ~~~ --- Free Embedded Linux Distro at http://www.snapgear.org --- ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
[LARTC] Re: how flexible is ingress traffic policing to bandwidth limit?
Damion de Soto <[EMAIL PROTECTED]> writes: > You can create different ingress policers that only match specific ports, and > give them different priorities, but that still won't work as well as using IMQ, > or if your box is a gateway (and you are only shaping traffic going through it), > then you can use egress queues on the LAN interface. For some reason that hadn't occurred to me. That should work just fine. I guess I should mark the packets in iptables to avoid throttling traffic from gateway itself, or does match see the external ip? IMQ does seem like a handy tool, but why is there a distinction at all between egress and ingress qdiscs at all? Why not just allow people to attach HTB as an ingress qdisc directly? I suppose in an ideal world the best thing would be to receive the packets, hand them to user-space, but not mark them as received, ie, not ack them. That avoids introducing any loss from the user point of view but still slows the flow down. But that seems fiddly and would only work for TCP I guess. -- greg ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
