[LARTC] ESFQ?
Hi again, I was just looking around for ESFQ sources, and I see that the main site is down, and only has kernel 2.6.4 patches. Is ESFQ maintained? If so, where can I find patches for 2.6.10? Thanks, -justin ___ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: [LARTC] HTB + GRED (santa clauss plz help me out)
Dear Andy, I 've been trynig for a long time now to make HTB and GRED to work together. The problem beeing that GRED doesn't use handles (instead it uses DP:1 DP:2 etc) i can not preperly filter traffic to them. Tomas Graf suggested to use the tc_index index of u32 classifier so dear Sant .. i am currently with the following scripts that works! Have you seen this http://www.opalsoft.net/qos/DS-27.htm Though if it works you don't need it :-) I am aware of it and have spend a few hours trying to make it work... without any success. But when i try to add an HTB before the GRED, everything goes to hell. You may need to repeat filters to get HTB to go from root to GRED (well you do with PRIO) eg. from a usenet post. That IS EXCACTLY MY PROBLEM. In the case of ordinary qdiscs i repeat my filters to the new handles (3:1 3:2 etc). In the case of GRED there are no handles to play with and i am unuable to filter traffic into GRED... Antonios ___ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: [LARTC] Scheduler Mechnisms!
Thank you very much, i will try to find these papers which must be very helpful for me. The more means that whether there are other mechanisms not only for Linux. Sorry, i have not make it clear! Sometimes, i wonder whether the qdiscs such as CBQ, RED, GRED ... are belong to the scheduler mechanisms in linux enviroment. For example, In Red, which i can find are enqueue, and dequeue so, if i add a RED qidsc to a class, must i add a scheduler mechanism so that i can decide which packet in the queues will be scheduled and put to the link? Good luck, Best, It depends on what you mean by more. More for Linux? Certainly. HTB3 seems to be a popular mechanism, and you can use IMQ to set up an intermediate device to allow shaping of both inbound and outbound traffic, using one (or more!) scheduling mechanisms in series. (In fact, there are several versions of IMQ out there. I've given links to both the projects that seem to be alive, but there may be more.) There's also ESFQ, but there doesn't seem to be much active work on that. There are forward ports to recent Linux kernels, though. QLinux has a version of H-SFQ for Linux, but again it seems to be getting long in the tooth. Unfortunately, I can't find any forward ports of that. http://luxik.cdi.cz/~devik/qos/htb/ http://www.linuximq.net/ http://pupa.da.ru/imq/ http://www.digriz.org.uk/jdg-qos-script/#qos-2.6 http://kem.p.lodz.pl/~peter/qnet/ http://lass.cs.umass.edu/software/qlinux/ There are a great many systems that I can't find a Linux version of. Cisco routers support something called Class-Based Weighted Fair Queueing (CBWFQ) which seems to be a hybrid of classful and classless scheduling. Cisco also has two versions of ECN, for forwards and backwards propogation. I've listed below a number of papers detailing various QoS schemes. Some of these have been implemented in other OS' (the BSDs tend to get a lot of this stuff implemented quickly for them as part of ALTQ) and some I've never seen an implementation at all. However, the papers should all give enough information to write a version for Linux. Note: ALTQ can be found at: http://www.csl.sony.co.jp/person/kjc/kjc/software.html Please note that this list is not exhaustive. Rather, I got exhausted after trying to find what was out there and what state it was currently in. QoS is a big field, if the number of papers is anything to go by. Linux only touches the fringes of it. If anyone feels particularly bored, or in need of a good ego boost, it would be cool to see if a reasonable selection of these could be introduced into Linux over the 2.7 cycle. EDF (Earliest Deadline First) http://citeseer.ist.psu.edu/13919.html BLUE (an alternative to RED) http://citeseer.ist.psu.edu/feng99blue.html AF PHB (Assured Forwarding Per-Hop Behaviour) http://citeseer.ist.psu.edu/552302.html SFB (Stochastic Fair Blue) http://citeseer.ist.psu.edu/551253.html GREEN (a pro-active variant on the theme of RED) http://citeseer.ist.psu.edu/feng02green.html SMART (Scalable Multipath Aggregated RouTing) http://citeseer.ist.psu.edu/vutukury00smart.html CSFQ (Core Stateless Fair Queueing) http://citeseer.ist.psu.edu/391.html StFQ (Start-Time Fair Queueing) http://citeseer.ist.psu.edu/goyal96starttime.html RFQ (Rainbow Fair Queueing) http://citeseer.ist.psu.edu/cao00rainbow.html PrFQ (Probabalistic Fair Queueing) http://citeseer.ist.psu.edu/anker00prfq.html ERR (Elastic Round Robin) http://citeseer.ist.psu.edu/kanhere02fair.html GFQ (Greedy Fair Queueing) http://citeseer.ist.psu.edu/690207.html PERR (Prioritized Elastic Round Robin) http://citeseer.ist.psu.edu/681127.html AOQ (Anchored Opportunity Queueing) http://citeseer.ist.psu.edu/701742.html BSFQ (Bin Sort Fair Queueing) http://citeseer.ist.psu.edu/622188.html As for the final question on what happens between enqueue and dequeue, there are various diagrams out there which show different aspects of how packets traverse the system. I've included a few links to those I could find: http://open-source.arkoon.net/kernel/kernel_net.png http://ebtables.sourceforge.net/br_fw_ia/PacketFlow.png http://ebtables.sourceforge.net/br_fw_ia/br_fw_ia.html http://www.docum.org/docum.org/kptd/ The last of these is the infamous Kernel Packet Travelling Diagram. Most links to this that I've been able to find are broken. It should be noted that the diagrams all refer to the Linux 2.4 kernel. Linux 2.6 has quite a few QoS changes to it, but I'm unclear as to whether they significantly alter any of the flows. I hope this is of some use. Or, at the very least, is an effective remedy to insomnia. :) Jonathan --- Zhenyu Wu [EMAIL PROTECTED] wrote: Hello, Normally, in addition to such qdisc scheduler mechanisms as FIFO, PQ, WRR, WFQ, are there any more? Then, there is a confusion on scheduler in Linux enviroment: Assume there is a
Re: [LARTC] Scheduler Mechnisms!
I may be wrong on this, but I believe that RED can be attached to any queueing system, including the basic FIFO queues. In a sense, you're still using a scheduling system, when using the default arrangement, it's just a first-come, first-served one. RED is classless and applies to the whole of a queue. What that queue is attached to, if I understand it correctly, isn't important. It can be a class, but it can just as easily be everything going through that device. Again, someone correct me if I'm wrong, but as I understand it, there are four levels to the whole QoS/diffserv concept. One of these levels is the queueing discipline. This can be something like CBQ, WFQ, FIFO, PRIO, or whatever. This is how the data is organized, it does not describe how the data is sent. In the case of something like CBQ, you have a defined set of queues in parallel, with rules as to what packets fall into what queue. On the other hand, queueing schemes such as FIFO are flat. There's a single queue that everything goes through, though there may be different rules for how things get pushed to it. Another level is the scheduling mechanism. This describes how the data is sent, once organized, but does not describe the organization itself. If you've only one queue, then there's really not much to schedule. If you've multiple queues, then it's fairly normal to use round robin or weighted round robin to pick which queue to pull a packet from. Linux' CBQ uses weighted round robin, according to the C file. The next level is the packet dropping mechanism. When queues flood, packets are going to be dropped. There's nowhere to store them. I'm pretty sure the default behaviour is to simply continue accepting packets, but to drop any that expire before being sent or which fall off the end of the queue (if the queue is bounded). RED, GRED, and a whole host of similar mechanisms, try to drop packets in a more controlled manner. However, that is really all they do. Finally, there are mechanisms for damping overly active applications, such as ECN. The idea here is that if you throttle back whatever is generating excess traffic, you don't get the problems assoicated with dealing with it. The default behaviour is to do nothing. When setting up QoS - on Linux or anything else - you basically pick one of each of the four categories to assemble a packet delivery system. Even without QoS, you're doing that, you're just using the defaults in all cases. The mechanisms are still going to be there. The Linux configuration menu does NOT match the above terminology, or the terminology in the source code. Thus, the source code identifies CBQ as a queueing discipline, but the configuration menu calls it a scheduler. The QoS help is also not very helpful, as it mostly tells people to look at the source. However, if you look at the source for CBQ or RED, for example, the explanation is relative to the cited papers, so you then have to go and read those before coming back and doing anything. This is one area I hope is going to get resolved in the reasonably near future. If not, I might have to come up with a patch myself. The very thought of that should send shivers down the spines of any kernel developers out there. Jonathan --- Zhenyu Wu [EMAIL PROTECTED] wrote: Thank you very much, i will try to find these papers which must be very helpful for me. The more means that whether there are other mechanisms not only for Linux. Sorry, i have not make it clear! Sometimes, i wonder whether the qdiscs such as CBQ, RED, GRED ... are belong to the scheduler mechanisms in linux enviroment. For example, In Red, which i can find are enqueue, and dequeue so, if i add a RED qidsc to a class, must i add a scheduler mechanism so that i can decide which packet in the queues will be scheduled and put to the link? Good luck, Best, __ Do you Yahoo!? The all-new My Yahoo! - What will yours do? http://my.yahoo.com ___ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
[LARTC] Suggestion - table of QoS mechanisms
Hi, A thought for the list. As I mentioned in another posting, there are a lot of QoS mechanisms out there. Linux supports some, but not all. Some patchsets add others, but don't work for all kernels. There are also userland implementations, usually sitting in software routers, but there are other packages. Would it be helpful if I worked on a table of what's out there for Linux and in what form? The main drawback of such a list is that while I can tell you if such-and-such an implementation exists, that doesn't mean the implementation is any good, or that the QoS concept is valid. There are plenty of arguments amongst QoS researchers as to whether RED is useful or not, and those are the people most qualified to know the answer. Nor would I be able to verify what kernel patches work well together, so the individual existance of specific mechanisms doesn't mean you can combine them usefully. On the other hand, there doesn't seem to be any easy way for people to find out what does exist, what doesn't exist YET for Linux but could easily be written, or what used to exist but has been dropped for reasons known or unknown. For example, SGI's Scheduled Transfer Protocol, MPLS, WRR and ESFQ are all examples of networking algorithms that are apparently deceased. The Layer 7 packet classifier isn't dead, but doesn't apply cleanly to kernels 2.6.9 or later. Finding these can be fun, too. I've got a copy of the Scheduled Transfer Protocol patches, but that's because I downloaded them while they were still on SGI's FTP site. If they exist anywhere on the Internet today, I haven't the foggiest where. The site for ESFQ is dead, and the only known patches forward-ported to recent kernels is merged into the qnet patch series, making it hard to extract. Any thoughts on this? Jonathan __ Do you Yahoo!? Read only the mail you want - Yahoo! Mail SpamGuard. http://promotions.yahoo.com/new_mail ___ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: [LARTC] Suggestion - table of QoS mechanisms
On Tuesday 04 January 2005 23:24, Jonathan Day wrote: Hi, A thought for the list. As I mentioned in another posting, there are a lot of QoS mechanisms out there. Linux supports some, but not all. Some patchsets add others, but don't work for all kernels. There are also userland implementations, usually sitting in software routers, but there are other packages. Would it be helpful if I worked on a table of what's out there for Linux and in what form? Possibly. I only know of CBQ, HTB, HFSC, SFQ, TBF, PFIFO, PRIO, G/RED for Linux offhand. The main drawback of such a list is that while I can tell you if such-and-such an implementation exists, that doesn't mean the implementation is any good, or that the QoS concept is valid. There are plenty of arguments amongst QoS researchers as to whether RED is useful or not, and those are the people most qualified to know the answer. Nor would I be able to verify what kernel patches work well together, so the individual existance of specific mechanisms doesn't mean you can combine them usefully. Yeah, QoS isn't exactly a plug and play experience. On the other hand, there doesn't seem to be any easy way for people to find out what does exist, what doesn't exist YET for Linux but could easily be written, or what used to exist but has been dropped for reasons known or unknown. I wrote a guide, Practical Guide to Linux Traffic Control[1], which I keep up to date as developments change. I only cover stuff in the mainline kernel for the most part, though. [1] http://trekweb.com/~jasonb/articles/traffic_shaping/ For example, SGI's Scheduled Transfer Protocol, MPLS, WRR and ESFQ are all examples of networking algorithms that are apparently deceased. The Layer 7 packet classifier isn't dead, but doesn't apply cleanly to kernels 2.6.9 or later. Layer 7 does patch against 2.6.9 with an experimental patch available since the beginning of December on the project's SF page. It Works For Me (tm) but I guess it hasn't been tested sufficiently such that it's now available as a stable 2.6.9+ patch. Finding these can be fun, too. I've got a copy of the Scheduled Transfer Protocol patches, but that's because I downloaded them while they were still on SGI's FTP site. If they exist anywhere on the Internet today, I haven't the foggiest where. The site for ESFQ is dead, and the only known patches forward-ported to recent kernels is merged into the qnet patch series, making it hard to extract. That's too bad. I had wanted to include something about ESFQ but never got around to it, since SFQ generally suits my needs. -- Jason Boxman Perl Programmer / *NIX Systems Administrator Shimberg Center for Affordable Housing | University of Florida http://edseek.com/ - Linux and FOSS stuff ___ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: [LARTC] Suggestion - table of QoS mechanisms
The URL for the guide was useful, thanks. Here are a few other QoS systems for Linux: RSVP is provided in the stock kernel. This allows you to reserve a given amount of bandwidth for a specific UDP data stream. It is typically not used in the real world because it doesn't scale well. Too much state information needs to be transmitted and kept track of, to be useful on backbone routers. USAGI is based on KAME, and KAME supports ALTQ. In turn, ALTQ supports HFSC, JoBS, RIO and BLUE for both IPv4 and IPv6. It is NOT clear from the USAGI web page as to whether ALTQ is included in their code. http://www.linux-ipv6.org/ http://www.csl.sony.co.jp/person/kjc/kjc/software.html QLinux supports H-SFQ, but is based on Linux 2.2 and the 2.4 sources don't seem to have ever been released. http://lass.cs.umass.edu/software/qlinux/ DGT2684 (seems to be dead, unless the pseudo-QoS for ATM in the Linux kernel is based on this, but then the code on Sourceforge should be current, you'd have thought) http://sourceforge.net/projects/dgt2684 I'm not altogether sure what SIMA did, but it seems to have been a queueing system of sorts for the 2.2 kernels. http://www.atm.tut.fi/faster/sima/ It's a cheat, but you can route traffic onto and off Network Simulator and therefore use any QoS devices available for that for regular networking. This includes Fair Queueing, Stochastic Fair Queueing and Deficit Round Robin, by default. Many of the ALTQ routines have NS implementations, as well, and I'm sure there are others. NS seems to be popular with protocol researchers. http://www.isi.edu/nsnam/ns/ There's also a QoS Library which provides a useful API for applications. http://www.coverfire.com/lql/ Finally, I also mentioned SGI's STP patch. STP allows you to reserve network resources for a future data stream. As far as I can tell, it is very similar in concept to RSVP, except that it is not UDP-specific and is specifically designed for very high-speed networks, where constructing and destructing connections at the time of use can add excessive latency. By pre-allocating, the connection can all be set up and ready to use when it is actually needed. --- Jason Boxman [EMAIL PROTECTED] wrote: (snip) Possibly. I only know of CBQ, HTB, HFSC, SFQ, TBF, PFIFO, PRIO, G/RED for Linux offhand. (snip) __ Do you Yahoo!? Take Yahoo! Mail with you! Get it on your mobile phone. http://mobile.yahoo.com/maildemo ___ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
[LARTC] Connections direct to the gateway
Hi, I use the fairnat scripts on a linux gateway, on this gateway runs a transparent web proxy, a local web server and some other few applications for the clients in the lan. Using this scripts limits the bandwidth of the connection to the gateway too so for example the proxy become unuseful. It is possibile with some rules or another way to get the full bandwidth and not limit it for the connection from the clients in the lan direct to the gateway ? Bye Thanks for the help. ___ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Re: [LARTC] ESFQ?
Ouch... Is there any other way to do host-based fair sharing (well, other than actually classifying each host :-( )? Thanks, -justin Jonathan Day wrote: To the best of my knowledge, ESFQ for Linux is essentially dead. There's a patchset - QNet - which does port ESFQ to the 2.6.8/2.6.9 kernels, but ESFQ is not split out, so it looks like an all-or-nothing deal. http://kem.p.lodz.pl/~peter/qnet/ I don't know if QNet is still being maintained - the last update on the page refers to October 2004 - and there's nothing to indicate how well the forward ports actually work in practice. A search using Google shows only older ESFQ versions (one for 2.6.0-test11, for example) but nothing newer. There was one posting about ESFQ to the kernel developers mailing list, but I couldn't see any follow-ups to it. Nor does it appear to be in Andrew Morton's patchset (an excellent indicator of interest level and the probability of ending up in the official kernel). Unfortunately, this seems to be fairly common in Linux QoS - too many one-man projects and too few resources too keep them going. --- Justin Schoeman [EMAIL PROTECTED] wrote: Hi again, I was just looking around for ESFQ sources, and I see that the main site is down, and only has kernel 2.6.4 patches. Is ESFQ maintained? If so, where can I find patches for 2.6.10? Thanks, -justin ___ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ __ Do you Yahoo!? Yahoo! Mail - You care about security. So do we. http://promotions.yahoo.com/new_mail ___ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
