RE: [LARTC] TC Hashing Filters
Dear list, After much code crunching and beating my head against the wall (literally), I discovered the faulty code. Thanks mostly to gypsy who played a huge role in helping me discover the problem. My tc filter line had an error in it which completely baffles me because it worked up until 2045 lines were entered. Old command: tc filter add dev $dev protocol ip parent 12: u32 ht 2:$table1: ht 3:$table2: match ip dst $ip/32 flowid 12:$classid New command: tc filter add dev $dev protocol ip parent 12: prio 5 u32 ht 2:$table1: ht 3:$table2: match ip dst $ip/32 flowid 12:$classid Who would have thought that a prio would cause that much trouble? I mean, I know in the documentation it says that it is required for CLASS commands, but since there's nothing on filters, I had no clue. And usually if a command requires a parameter, it will error immediately and not wait for 2045 entries. Also, why the hell would the priority cause a problem like that to begin with?! Weird. Anyways, THANKS EVERYONE! (gypsy, Catalin, and Ed especially) It all works, it's very resource friendly, and all is good again (and I get to keep my job)! The processor is only about 20% in use now. I'm very grateful for the hashing filter suggestion from Ed. Thanks a ton. I will be posting a How To shortly so others can do the same if they ever need to. Thanks again! Adam Towarnyckyj -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of gypsy Sent: Thursday, July 15, 2004 8:37 PM To: [EMAIL PROTECTED]; LARTC Subject: Re: [LARTC] TC Hashing Filters Adam Towarnyckyj wrote: > Any tc filter command I try to > add from here on out gives me the same error even if I try it manually > instead of using the script. Adam, That is a killer. Please read the following and then, when we both have the same information, I'm going to try again to assist you. HINT TO READERS: I hope someone else will help us both because I have obligations that I'm stealing time from that I can ill afford. There is a LARTC mailing list thread dated (about) 24 thru 26 June 2003 between Trevor Warren and Michael Ulitskiy whose Subject is "u32 clarification...limits on 2000>???" Please use your favorite method to find it. Note that there may be kernel issues not mentioned by them; kernels change. I'd like to suggest that you see if anyone involved in that thread will send you a testing script; perhaps you could find a way to start with a working setup and then apply minor changes until either it breaks or it suits you. ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
RE: [LARTC] TC Hashing Filters
gypsy wrote: > Your script can be modified to display the line with the substitutions > and to exit when executing causes an error. What I envision is that > you'll watch 2000+ lines of good stuff scroll by and that the final line > will be your Bad Boy. And the last few lines are what we all are > holding our breath to see . I have done as you suggested (actually, if you noticed in the Action subroutine, all the error reporting from before was still there I just commented it out) and this is what I've come up with. This is the last known good command and then the command that errors, as well as the error message, and the counter I added back in. Performing: /sbin/tc filter add dev eth1 protocol ip parent 1: u32 ht 2:54: match ip dst 216.19.20.84 flowid 1:FFFB 2045 Performing: /sbin/tc filter add dev eth1 protocol ip parent 1: u32 ht 2:2E: match ip dst 216.19.46.46 flowid 1:FFFB WHOOPS: RTNETLINK answers: File exists The error exists on the 2046th command. Any tc filter command I try to add from here on out gives me the same error even if I try it manually instead of using the script. I examined the previous commands that worked fine and there are plenty that dump into the 2:2E: table. > I think the line above should count down from 65535, not up. As you can see, I did this as you suggested. I like it better because it keeps things very separate so there's no confusion. Thanks for the suggestion. > Sorry, that's all I have time for now. Any time you can spare is incredibly helpful. Thank you. Adam Towarnyckyj ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
RE: [LARTC] TC Hashing Filters
gypsy wrote:
>Try setting classid to 0x and decrement rather than increment it.
>
>Pepper the script with
>debug (print or echo) lines
>error traps that exit the script on error.
>
>Is what you posted entire so that if I extract it from the message then
>it should run?
It's really tough for me to convey what I've done so far mainly
because when I get in "Troubleshoot Mode" I really tend to forget what
I've tried and the results of those efforts after I've done them. This
is because if it doesn't work, I rule it out and go on to the next thing
until I find out what the problem is. The reason I say this is because I
didn't really tell you specifically what I've done to troubleshoot. Only
that I found out where the problem was. For this, I apologize. So let me
try to be as specific as possible.
This is what the script does step by step:
1. Connect to the provisioning database (MySQL)
2. Define subroutines
a. SelectSQL - Subroutine for placing all information from a sql
query into a variable.
b. SelectSingleSQL - Subroutine for placing one piece of
information from a sql query into a variable.
c. SimpleSQL - Subroutine for making a sql query
d. Action - Subroutine for performing a system action and
outputting any errors to an array for later use.
3. Remove existing root qdisc and add a new one (clears all information
currently stored).
4. Create transit class and hash table/filter.
5. For creating the individual classes for each rate, we have it connect
to our database and add a class for each rate located in that database.
This is so it can be dynamic in case we need to add new classes down the
road.
6. This is where the script grabs all the accounts from our database by
modem. The modem table holds the rate for each customer. Then the script
compares those modems to the public IPs assigned to that customer and
adds the tc command to limit that IP based on the modem rate.
7. Take all the errors from any "Action" and output them. (This emails
to me directly when there's a problem).
8. There's a bunch of stuff here for promotional rates we're running
that is unimportant to the current problem I'm having.
The error appears at step 6. For each modem in our database, it
checks the IPs assigned to it. For each of those, it runs: tc filter add
dev $dev protocol ip parent 1: u32 ht 2:$table: match ip dst $ip flowid
1:$classid
I added a counter in there and an exit command in the "Action"
subroutine so that when the script errors, it exits and shows me how
many IPs tc has added before it produced an error. The number was 2045
and the error was RTNETLINK answers: File exists.
As for your suggestion about the classid, I'm a bit confused as
to what you mean about decrementing it. Could you be a little more
specific on where this is in the script?
I have attached the script in its entirety so you can see it and
maybe figure out what is wrong. The only problem with running it would
be connecting to the database. If you want, I can put up a mock database
and you can connect to that for testing purposes. The problem with this
is that I'd have to populate it with about 3000 entries for you to see
the error I'm seeing. I have removed my traps and counters so you can
see what the script was originally. I'll comment where the error occurs.
Thank you all once again for your help and time. It is very much
appreciated.
#!/usr/bin/perl
#
# TC Helper Script: Written by Mike Davis & Adam Towarnyckyj
#
# Synchronizes data rates with MySQL server and applies hourly.
#
### Configuration Section ###
$dev = "eth1";
$tc = "/sbin/tc";
$mysql_host = "sql database";
$mysql_db = "databse";
$mysql_user = "user";
$mysql_pass = "password";
### END Configuration Section ###
use POSIX qw(strftime);
# Database connect and define subroutines
use DBI;
$dsn = "DBI:mysql:database=$mysql_db;hostname=$mysql_host";
$dbh = DBI->connect($dsn, $mysql_user, $mysql_pass) || die "Can't
connect to database: " . DBI->errstr;
# Subroutine for placing all information from a sql query into a
variable.
sub SelectSQL {
my($sql) = @_;
my @MATCHES, $hash;
$sth = $dbh->prepare("$sql");
$sth->execute();
while ($hash = $sth->fetchrow_hashref) {
push @MATCHES, $hash;
}
return @MATCHES;
}
sub SelectSingleSQL {
my($sql) = @_;
my($gotit, $return, $hash);
$sth = $dbh->prepare("$sql");
$sth->execute();
while ($hash = $sth->fetchrow_array) {
unless ($gotit) {
$return = $hash;
$gotit++;
} else { warn "got multiple SQL returns when exepecting only
one"; }
}
return $return;
}
sub SimpleSQL {
my($sql) = $_[0];
my $rows_affected;
$rows_affected = $dbh
RE: [LARTC] TC Hashing Filters
First off, no need to be rude. I read the documentation; I just missed
the number limit.
Second, I know it's in hex and that's what I'm using.
Third, the script is rather large. I didn't want to post even a snippet
because it is so big. But, if that's what you require in order to give
me some advice, then here it is:
Action("$tc qdisc del dev $dev root");
Action("$tc qdisc add dev $dev root handle 1:0 cbq bandwidth 200mbit
avpkt 1000");
# Create 'transit class', tc hash tables, and hash filter
Action("$tc class add dev $dev parent 1: classid 1:2 cbq bandwidth
200Mbit rate 200MBit allot 1514 weight 2Mbit prio 8 maxburst 20 avpkt
1000");
Action("$tc filter add dev $dev parent 1: handle 2: protocol ip u32
divisor 256");
Action("$tc filter add dev $dev protocol ip parent 1: u32 match ip dst
0.0.0.0/0 hashkey mask 0x00ff at 16 link 2:");
# Create classes for rate groups
@RATES = SelectSQL("SELECT dsrate FROM dsrate");
my $classid = 3;
foreach $dsrate (@RATES) {
$ds = $$dsrate{dsrate};
if ($ds == "0" || $ds == "1") {
next;
}
Action("$tc class add dev eth1 parent 1: classid 1:$classid cbq
bandwidth 200Mbit rate $$dsrate{dsrate}Kbit allot 1514 prio 5 maxburst
20 avpkt 1000 bounded");
$rates{$ds} = $classid;
$classid++;
}
# Get our list of accounts
@MODEMS = SelectSQL("SELECT mid, dsrate FROM modems");
$z = 0;
# Figure out account IPs and put 'em in!
foreach $modem (@MODEMS) {
if ($$modem{dsrate} == "0" || $$modem{dsrate} == "1") {
next;
}
my @COMPUTERS = SelectSQL("SELECT ipid FROM computers WHERE
mid='$$modem{mid}'");
foreach $computer (@COMPUTERS) {
my $ip = SelectSingleSQL("SELECT ipaddr FROM ips WHERE
ipid='$$computer{ipid}'");
@octets = split(/\./,$ip);
$table = $octets[3];
$table = sprintf("%X", $table);
$classid = $rates{$$modem{dsrate}};
Action("$tc filter add dev $dev protocol ip parent 1:
u32 ht 2:$table: match ip dst $ip flowid 1:$classid");
$z++;
print "$z\n";
}
}
"Action" is a sub that performs a system action and reports errors to
another sub.
"SelectSQL" is a sub that performs a sql query to our database.
COMMENT: Why is it whenever anyone new posts a question to a list even
after reading through the documentation, someone always has to jump down
his throat instead of being helpful and kind? This is a mailing list
which was created so people can ask questions and get a helpful response
from a community. I have been nothing but nice to everyone here and I am
really grateful for all the help that has been provided so far. Instead
of barking at people, simply stating that you require a snippet of code
and providing an answer would be much more helpful and wouldn't cause
people like me to bark back which is surely going to cause an argument
between us.
Thanks again!
Adam Towarnyckyj
-Original Message-
From: gypsy [mailto:[EMAIL PROTECTED]
Sent: Monday, July 12, 2004 6:24 PM
To: [EMAIL PROTECTED]; LARTC
Subject: Re: [LARTC] TC Hashing Filters
Adam Towarnyckyj wrote:
>
> Hey all,
> So I got the script to run and populate everything. I watch as
> the script uses the tc commands to add filters for each IP into their
> appropriate tables. After 2045 entries, it starts to give me a "File
> Exists" error. I've done extensive testing on the script and
everything
COMMENT: Why is it that everyone who has a bad script does not post
(even a snip of) it?
The limit is 65536.
If you don't get the significance of that, it means the numbers are hex
and you're trying to make 'em be decimal. Go read docum.org.
___
LARTC mailing list / [EMAIL PROTECTED]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
RE: [LARTC] TC Hashing Filters
Hey all, So I got the script to run and populate everything. I watch as the script uses the tc commands to add filters for each IP into their appropriate tables. After 2045 entries, it starts to give me a "File Exists" error. I've done extensive testing on the script and everything else to come to this same result. I also looked at how many entries were going into individual tables. It seems there are only 15-20 per table and this isn't a lot. So my question is does the hash filter have a limit on the number of entries? I wouldn't believe so but I keep running into this problem. If this is the case, I guess I'll be looking for another alternative. Thanks! Adam Towarnyckyj ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
RE: [LARTC] TC Hashing Filters
Gideon was an incredible help with all of this. He solved my little problem I was having. It was such a simple solution and I completely missed it. You see, when I changed the src to dst (which is what I'm trying to accomplish because I'm shaping the download rates of our customers, not upload rates) I forgot that in the IP Header, bit 12 is the source address. Since I was shaping incoming traffic, I want the destination IP. Gideon was so kind to point this out to me. I knew it was something very small and obvious... So, here are the completed commands that work great for anyone who is attempting to do this themselves. I will also post to this list with this same subject the link to the How To as soon as I get it created. # Delete current qdisc wiping the slate clean tc qdisc del dev eth1 root # Create root qdisc tc qdisc add dev eth1 root handle 1: cbq bandwidth 200Mbit avpkt 1000 # Create "transit class" tc class add dev eth1 parent 1: classid 1:2 cbq bandwidth 200Mbit rate 200MBit allot 1514 weight 2Mbit prio 8 maxburst 20 avpkt 1000 # Create hash tables (256) tc filter add dev eth1 parent 1: handle 2: protocol ip u32 divisor 256 # Create hash filter to separate traffic going TO 216.19.49.0/24 using the last octet tc filter add dev eth1 protocol ip parent 1: u32 match ip dst 216.19.49.0/24 hashkey mask 0x00ff at 16 link 2: # Create the 128Kbit class for limiting tc class add dev eth1 parent 1: classid 1:3 cbq bandwidth 200Mbit rate 128Kbit allot 1514 prio 5 maxburst 20 avpkt 1000 bounded # Filter per IP into the hash tables tc filter add dev eth1 protocol ip parent 1: u32 ht 2:8c: match ip dst 216.19.49.140 flowid 1:3 (Note: The last filter will need to be done for each IP you want to limit. Also note that you can create multiple classes for different rates you would like to limit. You'll just need to point the last filter's flowid to that class) Thanks, once again, to everyone for their help and the pointing out of my obvious mistakes. If anyone has any questions about how to do this or anything, please feel free to ask. I'll answer to the best of my abilities. Adam Towarnyckyj ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
RE: [LARTC] TC Hashing Filters
Ok, I tried this: tc qdisc add dev eth1 root handle 1: cbq bandwidth 200Mbit avpkt 1000 tc class add dev eth1 parent 1: classid 1:2 cbq bandwidth 200Mbit rate 200MBit allot 1514 weight 2Mbit prio 8 maxburst 20 avpkt 1000 tc filter add dev eth1 parent 1: handle 2: protocol ip u32 divisor 256 tc filter add dev eth1 protocol ip parent 1: u32 match ip src 216.19.49.140/32 hashkey mask 0x00ff at 12 link 2: tc class add dev eth1 parent 1:2 classid 1:3 cbq bandwidth 200Mbit rate 128Kbit allot 1514 prio 5 maxburst 20 avpkt 1000 bounded tc filter add dev eth1 protocol ip parent 1: u32 ht 2:8c: match ip src 216.19.49.140 flowid 1:3 As well as changing the last class parent to 1: Neither of these worked. As for the app, could I take a look at it? I'm sure it could be easily modified to use CBQ and if not, I could modify the box the use HTB. I really don't mind either way. I was having a problem patching for HTB and that's why I'm not currently using it. My boss and I wrote a perl script to pull information from a MySql database (rate and customer IP) and plug it in to TC and that's what I was planning on doing this time around only with a few modifications for the hashing tables. I won't start on that though until I can get this up and running. Anyways, thanks for the help. Adam Towarnyckyj -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Sent: Tuesday, July 06, 2004 12:15 PM To: Adam Towarnyckyj; [EMAIL PROTECTED] Subject: RE: [LARTC] TC Hashing Filters Here the filter parents should be root not 1:2. I think in the howto it says that all filters should be attached to root. I'll get my box up that I have doing this to give you a working example. >tc filter add dev eth1 parent 1:2 handle 2: protocol ip u32 divisor 256 > >tc filter add dev eth1 protocol ip parent 1: u32 match ip src >216.19.49.0/24 hashkey mask 0x00ff at 12 link 2: > >tc class add dev eth1 parent 1:2 classid 1:3 htb rate 64Kbit ceil 64Kbit >burst 0Kbit > >tc filter add dev eth1 protocol ip parent 1:2 u32 ht 2:8c: match ip src >216.19.49.140 flowid 1:3 > >I even tried changing the src to dst and this still does not work. >Gideon, if you are out there somewhere, I'd appreciate it if you could >tell me if this actually worked for you. This is the most frustrating >project I have ever had to deal with because it makes so much sense and >in theory it looks like it should be working just fine. Anyways, thanks >for everyone's help. If anyone has any other ideas, I welcome them. > >Adam Towarnyckyj ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
RE: [LARTC] TC Hashing Filters
Original Message ># Create root qdisc >tc qdisc add dev eth1 root handle 1:0 cbq bandwidth 200Mbit avpkt 1000 > ># Create transit class >tc class add dev eth1 parent 1:0 classid 1:1 cbq bandwidth 200Mbit rate >200Mbit allot 1514 weight 2Mbit prio 8 maxburst 10 avpkt 1000 > ># Create hash table and attach to transit class >tc filter add dev eth1 parent 1:1 handle 2: protocol ip u32 divisor 256 > ># Create filter to hash out last octet and link to hash table 2: >tc filter add dev eth1 protocol ip parent 1:0 prio 5 u32 ht 800:: match >ip src 216.19.49.0/24 hashkey mask 0x00ff at 12 link 2: > ># Create class for 128Kbit limit >tc class add dev eth1 parent 1:1 classid 1:2 cbq bandwidth 200Mbit rate >128kbit allot 1514 prio 5 maxburst 20 avpkt 1000 bounded > ># Create filter for IP I'm limiting >tc filter add dev eth1 protocol ip parent 1:0 prio 5 u32 ht 2:8c match >ip src 216.19.49.140 flowid 1:2 I messed with this a bit more before finally just giving up for today. I looked at Gideon's example he posted to this list in the past (http://mailman.ds9a.nl/pipermail/lartc/2003q2/008516.html) and copied it character by character except I changed the ethernet device it was attached to and the IP addresses he was using. Here's what I came up with: tc qdisc add dev eth1 root handle 1: htb tc class add dev eth1 parent 1: classid 1:2 htb rate 100MBit ceil 100MBit burst 0Kbit tc filter add dev eth1 parent 1:2 handle 2: protocol ip u32 divisor 256 tc filter add dev eth1 protocol ip parent 1: u32 match ip src 216.19.49.0/24 hashkey mask 0x00ff at 12 link 2: tc class add dev eth1 parent 1:2 classid 1:3 htb rate 64Kbit ceil 64Kbit burst 0Kbit tc filter add dev eth1 protocol ip parent 1:2 u32 ht 2:8c: match ip src 216.19.49.140 flowid 1:3 I even tried changing the src to dst and this still does not work. Gideon, if you are out there somewhere, I'd appreciate it if you could tell me if this actually worked for you. This is the most frustrating project I have ever had to deal with because it makes so much sense and in theory it looks like it should be working just fine. Anyways, thanks for everyone's help. If anyone has any other ideas, I welcome them. Adam Towarnyckyj ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
RE: [LARTC] TC Hashing Filters
[EMAIL PROTECTED] wrote: >> # Create filter to hash out last octet and link to hash table 2: >> tc filter add dev eth1 protocol ip parent 1:1 prio 5 u32 ht >>800:: match ip src 216.19.49.0/24 hashkey mask 0x00ff at 12 link 2: > > >I think this needs to be attached to the root. >parent 1:0 Catalin BOIE ([EMAIL PROTECTED]) wrote: >> # Create root qdisc >> tc qdisc add dev eth1 root handle 1:0 cbq bandwidth 200Mbit avpkt 1000 >> # Create a "transit class" >> tc class add dev eth1 parent 1:0 classid 1:1 cbq bandwidth 200Mbit >> rate 200Mbit allot 1514 weight 2Mbit prio 8 maxburst 10 avpkt 1000 >> # Create hash table attached to transit class >> tc filter add dev eth1 parent 1:0 handle 2: protocol ip u32 divisor 256 >> # Create filter to hash out last octet and link to hash table 2: >> tc filter add dev eth1 protocol ip parent 1:1 prio 5 u32 ht >> 800:: match ip src 216.19.49.0/24 hashkey mask 0x00ff at 12 link 2: >> # Create class for 128Kbit limit >> tc class add dev eth1 parent 1:0 classid 1:2 cbq bandwidth > >I think here is parent 1:1, right? > >> 200Mbit rate 128kbit allot 1514 prio 5 maxburst 20 avpkt 1000 bounded >> # Create filter for IP I'm limiting >> tc filter add dev eth1 protocol ip parent 1:0 prio 5 u32 ht 2:8c >> match ip src 216.19.49.140 flowid 1:2 I have tried both of your suggestions and it is still allowing all traffic to pass through. Thank you for the help though. Here is what I have now: # Create root qdisc tc qdisc add dev eth1 root handle 1:0 cbq bandwidth 200Mbit avpkt 1000 # Create transit class tc class add dev eth1 parent 1:0 classid 1:1 cbq bandwidth 200Mbit rate 200Mbit allot 1514 weight 2Mbit prio 8 maxburst 10 avpkt 1000 # Create hash table and attach to transit class tc filter add dev eth1 parent 1:1 handle 2: protocol ip u32 divisor 256 # Create filter to hash out last octet and link to hash table 2: tc filter add dev eth1 protocol ip parent 1:0 prio 5 u32 ht 800:: match ip src 216.19.49.0/24 hashkey mask 0x00ff at 12 link 2: # Create class for 128Kbit limit tc class add dev eth1 parent 1:1 classid 1:2 cbq bandwidth 200Mbit rate 128kbit allot 1514 prio 5 maxburst 20 avpkt 1000 bounded # Create filter for IP I'm limiting tc filter add dev eth1 protocol ip parent 1:0 prio 5 u32 ht 2:8c match ip src 216.19.49.140 flowid 1:2 If anyone can still see anything I'm doing wrong, I'd appreciate it if you pointed it out. I've tried many different scenarios with different parents and they still don't work. And as for the question earlier about if I was doing this for an ISP, yes I am. If you would like some help, and if I can get this to work, I plan on writing a How To and posting it to a website so people can easily set this up themselves. I find the lack of documentation on the subject quite frustrating at times. Thanks all for your help. Adam Towarnyckyj ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
[LARTC] TC Hashing Filters
Hey all, I had asked a question a little while ago about CPU usage being outrageous while using tc and a cbq qdisc. Ed was very kind and offered a few suggestions. One of these was to look at hashing. So after pounding away at it for about a week, I have a general understanding of how it works and have tried to implement it. Unfortunately, and quite obviously since I'm posting here, it does not work for me. The Problem: When implemented, and all the tc commands are entered, I get no errors or anything for that matter. It looks like everything went in smoothly. I do a show and sure enough, it is all there and looks good. However, when I have a computer hooked up on one side of the limiter and the other side hooked up to the internet, I do a download test and get the average 35Mbps that I normally see from our office. The Situation: The first step in troubleshooting this problem involved finding out if tc even limited at all. So I set up a simple class with a 128Kbit rate and cbq root qdisc like I had originally and all worked well. The download test showed 128Kbit/s like it should. I was happy that this part works. It seems that when I install the hash tables and assign it to hash based on the last octet in the IP address, it just allows traffic to pass through without limiting. This is where I run into the trouble. The Commands: Here is what I originally started out with based on the Hashing Filter How To: # Create root qdisc tc qdisc add dev eth1 root handle 1:0 cbq bandwidth 200Mbit avpkt 1000 # Create a "transit class" tc class add dev eth1 parent 1:0 classid 1:1 cbq bandwidth 200Mbit rate 200Mbit allot 1514 weight 2Mbit prio 8 maxburst 10 avpkt 1000 # Create hash table attached to transit class tc filter add dev eth1 parent 1:0 handle 2: protocol ip u32 divisor 256 # Create filter to hash out last octet and link to hash table 2: tc filter add dev eth1 protocol ip parent 1:1 prio 5 u32 ht 800:: match ip src 216.19.49.0/24 hashkey mask 0x00ff at 12 link 2: # Create class for 128Kbit limit tc class add dev eth1 parent 1:0 classid 1:2 cbq bandwidth 200Mbit rate 128kbit allot 1514 prio 5 maxburst 20 avpkt 1000 bounded # Create filter for IP I'm limiting tc filter add dev eth1 protocol ip parent 1:0 prio 5 u32 ht 2:8c match ip src 216.19.49.140 flowid 1:2 Troubleshooting: There are a few things I'd like to point out. First off, I'm only working with one IP address at the moment and the traffic is coming from only one Class C at the moment. I've even tried hard setting the hash ip src to 216.19.49.140/32 and that doesn't work. Second, I've tried attaching each of these to different parents. I tried attaching the hash table to the root qdisc as well as the transit class. I've tried the same for the limiting class and the ip filter. None of this works. Third, I've also tried it without the transit class at all and just doing everything direct off the root qdisc. Not only have I used the Hashing Filter How To but I have also used some suggestions from Gideon who posted to this list in the past. (http://mailman.ds9a.nl/pipermail/lartc/2003q2/008516.html). I tried htb as well and this qdisc didn't work either. Also, I've changed the src to dst and I originally started out using the hash ip src as our entire network of 216.19.0.0/18. If anyone has an understanding of hashing filters, please PLEASE let me know what I'm doing wrong. I've been working on this for three days straight now and in the meantime, our bandwidth limiting is not running which is INCREDIBLY important to my boss. I currently have just a straight list of 5000 users that I do not want to stray too far from (the iptables project Ed referred to requires a complete overhaul of my current implementation). I'm pulling my hair out because according to my understanding of it and based on what others have done this SHOULD be working. Thank you very much for at least taking the time to read this. I truly appreciate it. I'm also hoping that this will spark a thread for future people who run into the same trouble I am having. Thanks. Adam Towarnyckyj ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
[LARTC] Perl Script for pulling information from a mysql database
Here's the script. I would have posted it to a website, but I figure
this'll be better so it is always on the list for people in the future
in case they wanted to see it.
If you have any questions about any of it, please let me know. Mike and
I aren't the cleanest of programmers. Cron the script to run whenever
you need it to. :)
#!/usr/bin/perl
#
# TC Helper Script: Written by Mike Davis & Adam Towarnyckyj
#
# Syncronizes data rates with MySQL server and applies hourly.
#
### Configuration Section ###
$dev = "eth1";
$tc = "/sbin/tc";
$mysql_host = "host";
$mysql_db = "database";
$mysql_user = "user";
$mysql_pass = "password";
### END Configuration Section ###
use POSIX qw(strftime);
use DBI;
# Database connect and define subroutines
$dsn = "DBI:mysql:database=$mysql_db;hostname=$mysql_host";
$dbh = DBI->connect($dsn, $mysql_user, $mysql_pass) || die "Can't
connect to database: " . DBI->errstr;
sub SelectSQL {
my($sql) = @_;
my @MATCHES, $hash;
$sth = $dbh->prepare("$sql");
$sth->execute();
while ($hash = $sth->fetchrow_hashref) {
push @MATCHES, $hash;
}
return @MATCHES;
}
sub SelectSingleSQL {
my($sql) = @_;
my($gotit, $return, $hash);
$sth = $dbh->prepare("$sql");
$sth->execute();
while ($hash = $sth->fetchrow_array) {
unless ($gotit) {
$return = $hash;
$gotit++;
} else { warn "got multiple SQL returns when exepecting only
one"; }
}
return $return;
}
sub SimpleSQL {
my($sql) = $_[0];
my $rows_affected;
$rows_affected = $dbh->do($sql);
return $rows_affected;
}
sub Action {
my($action) = @_;
# print"Performing: $action\n";
$warn=`$action 2>&1`;
if ($warn) {
chomp($warn);
$prepare = "ERROR: $warn. Command was: $action";
#print"WHOOPS: $warn\n";
push @WARNING, $prepare;
}
}
### Ok, now we start having fun. Let's rebuild the tc tree.
# Remove existing tree and add the root.
Action("$tc qdisc del dev $dev root");
Action("$tc qdisc add dev $dev root handle 10:0 cbq bandwidth 200mbit
avpkt 1000");
Action("$tc class add dev $dev parent 10: classid 10:1 cbq bandwidth
200Mbit rate 200Mbit allot 1514 weight 2Mbit prio 8 maxburst 10 avpkt
1000");
# Get our list of accounts
@MODEMS = SelectSQL("SELECT mid, dsrate FROM modems");
# Figure out account IPs and put 'em in!
foreach $modem (@MODEMS) {
if ($$modem{dsrate} == "0" || $$modem{dsrate} == "1") {
next;
}
my @COMPUTERS = SelectSQL("SELECT ipid FROM computers WHERE
mid='$$modem{mid}'");
foreach $computer (@COMPUTERS) {
my $ip = SelectSingleSQL("SELECT ipaddr FROM ips WHERE
ipid='$$computer{ipid}'");
my $rate = $$modem{dsrate} . "kbit";
my $classid = sprintf("%X", $$computer{ipid});
Action("$tc class add dev $dev parent 10: classid
10:$classid cbq bandwidth 200Mbit rate $rate allot 1514 prio 5 maxburst
20 avpkt 1000
bounded");
Action("$tc qdisc add dev $dev parent 10:$classid sfq
quantum 1514 perturb 15");
Action("$tc filter add dev $dev parent 10:0 protocol ip
prio 25 u32 match ip dst $ip flowid 10:$classid");
}
}
if (@WARNING) {
print"WARNING: TCHELPER produced errors! See below:\n @WARNING\n";
}
#
# Cool, everyone is now limited.
# Exit Nice and clean.
$dbh->disconnect;
exit(0);
___
LARTC mailing list / [EMAIL PROTECTED]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
RE: [LARTC] CBQ troubles, processor overload
I'd love to share. Whoever was interested in that, please let me know in an email. Thanks for the suggestions. I really appreciate them and I'll look into it. Adam Towarnyckyj -Original Message- From: Ed Wildgoose [mailto:[EMAIL PROTECTED] Sent: Tuesday, June 22, 2004 12:09 AM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: Re: [LARTC] CBQ troubles, processor overload > The problem I'm running into is that the processor gets >overloaded because of the amount of work tc is doing. I assumed it would >be able to handle the apx 5000 customers we have on it. I have a bridge >set up between two devices that run from the internet to the local >network. This bridge takes up 20% of the CPU when tc is not enabled. >When tc becomes enabled, it finishes off the rest of the CPU and eats >most of the queue as well. > > 5,000 rules is significant. Have a look at the hashing examples in the LARTC howto for some ideas on how to slash bandwidth required. There is also a high performance iptables project kicking around which does much better for large rulesets. Since you don't seem to need anything advanced I would have thought this was a drop in replacement. Have a look at http://www.hipac.org/index.htm - Never used it though, just came across it on google. I think there is another chap who posted a few hours earlier may be really interested in your perl script to read users from the DB and build rules. If you have any kind of traffic accounting I think he would be interested in that as well. Want to share any of that...? Good luck Ed W ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
[LARTC] CBQ troubles, processor overload
Hey there everyone, I appreciated everyone's help so much in the last few posts I decided to post some more! :) We implemented our bandwidth management machine on the network last week and everything looked great. After looking at it today, anything past the machine is losing major packets and the latency is high. I'm using RedHat 9.0 on a Pentium 4 2.8ghz with 512mb of ram. I set up tc to run cbq as the root qdisc. I run a perl script to pull data from our customer database (IP address and customer number) and add it in to TC. Here are the commands I run: /usr/sbin/tc qdisc del dev eth1 root (used to delete the current setup so a fresh start can be made) /usr/sbin/tc qdisc add dev eth1 root handle 10:0 cbq bandwidth 200mbit avpkt 1000 (the root qdisc) /usr/sbin/tc class add dev eth1 parent 10: classid 10:1 cbq bandwidth 200Mbit rate 200Mbit allot 1514 weight 2Mbit prio 8 maxburst 10 avpkt 1000 (the parent class for all our non-customers) /usr/sbin/tc class add dev eth1 parent 10: classid 10:$classid cbq bandwidth 200Mbit rate $rate allot 1514 prio 5 maxburst 20 avpkt 1000 bounded (the parent class for all our customers to split) /usr/sbin/tc qdisc add dev eth1 parent 10:$classid sfq quantum 1514 perturb 15 (used for fair sharing) /usr/sbin/tc filter add dev eth1 parent 10:0 protocol ip prio 25 u32 match ip dst $ip flowid 10:$classid (filter for customer based on IP) The problem I'm running into is that the processor gets overloaded because of the amount of work tc is doing. I assumed it would be able to handle the apx 5000 customers we have on it. I have a bridge set up between two devices that run from the internet to the local network. This bridge takes up 20% of the CPU when tc is not enabled. When tc becomes enabled, it finishes off the rest of the CPU and eats most of the queue as well. I'm wondering if the commands I'm using are being implemented correctly, if I'm using the right setup, and if I'm using the right qdiscs or if there's a better alternative. I stole most of this code from the guy who worked here before me. The commands seemed to work for another project we had but we have a lot more customers now than we did before. I would assume the machine we're running would be able to handle it but I'm not sure. Any help would be greatly appreciated. I understand what you all have to put up with sometimes and you are terrific for helping out those of us in need. Thanks. Adam Towarnyckyj ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
RE: [LARTC] ntop performance
*smacks forehead* RTFM I found a few things that might improve performance but the problem is I lose a lot of logging options. I guess I'm open to suggestions for what you all think is a good program to use for such tasks. Thanks! Adam Towarnyckyj ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
[LARTC] ntop performance
Hey all, I wanted to let you all know the hex thing worked out for me. Thanks again. I had another question though that was sparked by Ionut Gogu's question on monitoring. I setup and installed ntop for this purpose and noticed that we have so much traffic it is killing the machine. We have 1 GB of ram in there and it is not enough. It still overflows to the swap file and eats about half that as well. There are approximately 5000 connections setup over a bridge being controlled by tc. Is there a way to get ntop to run cleaner or is there a better alternative out there that someone knows about? Any suggestions would be greatly appreciated. Thanks! Adam ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
RE: [LARTC] Class ID limits
Thank you very much for your help. I couldn't seem to find that anywhere in the documentation. Maybe I wasn't looking in the right place. Anyways, thanks again. Adam -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Nikolay Datchev Sent: Thursday, June 17, 2004 12:54 AM To: Catalin BOIE Cc: Adam Towarnyckyj; [EMAIL PROTECTED] Subject: Re: [LARTC] Class ID limits > > I actually have a few questions. First, am I right in assuming this or > > is the reason something totally different? And, if I'm right, is there > > any way around the limit other than creating a new qdisc? I'm trying to > > use a number we have set up in our database and sometimes it goes into > > the 5 digit range. Any help would be greatly appreciated. Thanks! > > Class parameters to tc are hexa numbers so you can use from x:1 to x:, > meaning 65535 classes. > And note that you must supply the classid in hex. Try searching google for a small tool like dec2hex, which converts decimal numbers to hex. ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
[LARTC] Class ID limits
Hey everyone, I had a question about the class id's using the TC command. I'm using a command that looks something like this: tc class add dev eth1 parent 10: classid 10:$variable cbq bandwidth 200Mbit rate 512Kbit allot 1514 prio 5 maxburst 20 avpkt 1000 bounded That's just an example. My problem is I'm guessing the $variable is not allowed to be anything over . I'm getting an error when I run that command with anything over 4 digits as a $variable. The error is: RTNETLINK answers: Invalid argument I've also tried just entering that in at the command prompt and using 1 as the variable and that error shows up. I actually have a few questions. First, am I right in assuming this or is the reason something totally different? And, if I'm right, is there any way around the limit other than creating a new qdisc? I'm trying to use a number we have set up in our database and sometimes it goes into the 5 digit range. Any help would be greatly appreciated. Thanks! Adam Towarnyckyj Network Operations CommSpeed AZ, LLC http://www.commspeed.net/ Phone: 928-772- x131 ___ LARTC mailing list / [EMAIL PROTECTED] http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
