Re: [LARTC] Default source address
On Sun, Jul 10, 2005 at 11:19:33AM +0200, Sebastian Spies wrote: > Hi, > I'm using vpnc to connect to a vpn. > The VPN-GW wants me to use him as a default route. So I used > source-routing to differ between my physical ethernet and the tunnel: > ip rule add from w.x.y.z table 20 > ip route add default via tun0 table 20 > Everything works fine, as I have a client to set my inet4-srcaddress. > My question is: Is there a way, to tell linux which is my default > sourceaddress? It should use the primary address of the interface the routing table picks to send the data out of. If you've managed to get multiple primary addresses, it picks the lowest one, numerically, I believe. Otherwise you have to bind your side of the socket to an address, and it should use that. -- Paul "TBBle" Hampson, on an alternate email client. ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Re: RFC - bandwidth optimization idea
On Sun, Jul 10, 2005 at 08:49:13AM +0100, Ed W wrote: > >Wait, you're trying to send more data than the link can take? Then > >send UDP, throttle it at the local end with a drop-oldest qdisc. Then > >you get the effect of 'most recent data is best'. Anything more > >compilcated in terms of priority either needs a custom qdisc, or your > >application needs to not try and send more than the link can take. > The situation described is real and complex. For example I run an email > service which caters for people using satellite phones (1,200 baud on a > good day), but the whole point is that they don't need to change any > settings when they jump on a 10Mbit leased line connection... Ah, I was picturing voice over a low-latency, low-speed link. Now I can understand what you're trying to acheieve. Is that 1200 baud each way? Or do you have to alternate up and down somehow? > This is a total pain to optimise. Ideally I would like an API to be > able to limit the congestion window on the local machine for a > particular connection (which I don't think exists on either windows or > linux?). This way the OS will report that the queue is full quickly to > the local program without buffering up a ton of data. Indeed. For TCP, you could use setsockopt with SO_SNDBUF maybe? However, I'm not sure this is what you want. > The issue in my case is that you have two simultaneous streams in > transit for email, one to receive new mail and one to send mail out. In > the case of the sat phone it's possible to have net buffers which are 20 > secs or so long and so when you send out a status message to say "email > received successfully, send me the next one", it can end up queued > behind a bunch of lower priority data for a VERY long time. Often these > buffers are on the remote ISP end where you have very little control. > This is a serious slowdown on a link which is costing you $1.50/min. Assuming you can send both ways simultaneously, or at least guarantee some traffic time outbound no matter how busy the inbound traffic, you would surely have to pipeline your commands in order to get any kind of efficient use out of a high-latency link like a satellite link. Otherwise you lose 2x round-trip-time of incoming data stream while you request the next item. In this situation, you would want the OS buffers to be as full as possible so the minimal time possible is spent receiving, but using a traffic-shaping solution so that the most important stuff (acks and commands) goes out in preference to whatever else you're sending. eg. If you're doing POP3 and SMTP, you make sure any to-tcp-110 or tcp-ack-only packet is dequeued before any from-tcp-25 packets. You'd also need to jack the receive window right up, or wait for TCP to figure that out for itself. > My main focus has been adjusting the protocol to be less interactive, > but it would be nice to have more operating system support for these > fringe cases This is actually a common case, and often cited as a great big hole in TCP/IP's traffic algorithms. I know, it was a question on the exam. ^_^ -- Paul "TBBle" Hampson, on an alternate email client. ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Re: RFC - bandwidth optimization idea
On Sat, Jul 09, 2005 at 08:25:39AM -0700, Don Cohen wrote: > > From: [EMAIL PROTECTED] (Paul Hampson) > > I believe the general solution to this is to use UDP, and make sure > The scheme I describe wouldn't make a lot of sense for tcp, which > after all specifies congestion control, retransmission, etc. > But UDP still goes through the queuing that I want to optimize. > > your source machine doesn't queue up packets locally (eg. ethernet > > network contention) and let the best-effort nature of UDP deal with > > dropping stuff that gets delayed. > The problem is that the OS is not helpful in avoiding queuing up > packets locally. That's part of what I'm trying to fix. > For instance, a relatively cheap approximation would be to give > the application a way to see how many packets it has in the queue. > Then it could at least delay its decision about what to put into > the queue until the queue was short. Even better would be to > see an estimate of how long it will be before the next packet it > enqueues will be sent - like "your call will be answered in > approximately 4 minutes". > > I'm not sure there's any way to have an 'I changed my mind about > > sending that' interface into your network stack... And generally > > it wouldn't be useful, data spends longer in transit than it does > > in your queues. > That depends on the rate at which the queue is emptied. > If your queue has a rate limit of 10bps then your packets can spend > a long time in the queue. > - There are slow links > (For instance, I recall hearing that submarines have very low rates.) > - The application might be allocated a small part of the bandwidth > shared with other applications. Wait, you're trying to send more data than the link can take? Then send UDP, throttle it at the local end with a drop-oldest qdisc. Then you get the effect of 'most recent data is best'. Anything more compilcated in terms of priority either needs a custom qdisc, or your application needs to not try and send more than the link can take. > It occurs to me that an example where this would be helpful is > transmitting voice data over a low bandwidth link (like a cell phone). > Suppose you know that the actual transit time is .1 sec and you want > the listener to always hear what the speaker was saying .2 sec ago at > the best possible quality. > Suppose the available bandwidth is shared with other applications. > The voice application doesn't know when they will want to send or how > urgent their data might be. Someone else decides that. It just wants > to send the best possible data in the bandwidth allocated to it. I > imagine is continually sampling the input and revising what it > considers to be the most valuable unsent data for the last .1 sec. > Whenever the OS decides it's time to send the next voice packet I want > it to send the latest idea of what's most valuable. I don't want to > have to put data into the queue to wait for times that might depend on > what urgent communication might be required by other applications. You gotta prioritise your data, using TOS or diffserv or something. Set your voice to real-time, so it always gets sent, and the your other applications can use unused packet-times. Use a dropping qdisc for traffic where 'most-recent' is more important than 'all, in order' as described above, and you're set. I have a vauge recollection that this sort of thing is discussed in Tannenbaum's Computer Networks textbook, to do with positional data of satellites or something. (eg. if the positional data is delayed, we write it off, we don't want to delay the data about where we are _now_ in order to know where we were _then_) -- Paul "TBBle" Hampson, on an alternate email client. ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] RFC - bandwidth optimization idea
On Fri, Jul 08, 2005 at 08:55:08PM -0700, Don Cohen wrote: > > I'm interested in all of > - opinions about why this is a good or bad idea > - pointers to similar proposals or products that already exist > - implementation suggestions > This is meant for real time applications that have small available > bandwidth and so they have to consider carefully what's the best way > to use that bandwidth. I imagine that things happen that cause them > to continually reevaluate what's the most important/urgent thing to > send next. I want to make it possible for them to delay the choice > until the OS is actually ready to send that next packet. The reason > they can't do this now is that the OS enqueues packets. Suppose an > application uses udp or tcp to tell the OS to send some data. It then > discovers that data is obsolete. The old data might still be in the > queue to be sent but it's too late to recall it. One way to avoid > that is to always delay telling the OS to send something until the OS > is almost ready to send the next packet from the queue that your data > will enter. But that's not so easy to do, and there's a big penalty > if you wait just a little too long. What I want, at least > conceptually, is that the application maintains its own queue of data > to be sent, ordered by priority. Whenever the OS is ready to send the > next packet for that application, it removes the highest priority > packet (if any) from the queue and sends it. I believe the general solution to this is to use UDP, and make sure your source machine doesn't queue up packets locally (eg. ethernet network contention) and let the best-effort nature of UDP deal with dropping stuff that gets delayed. I'm not sure there's any way to have an 'I changed my mind about sending that' interface into your network stack... And generally it wouldn't be useful, data spends longer in transit than it does in your queues. -- Paul "TBBle" Hampson, on an alternate email client. ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Packet Counting...
On Wed, Mar 16, 2005 at 10:16:32AM -0600, M. A. Imam wrote:
> Sure i would like to try that... Also if you can tell me how accurate it can
> be, i will be greatful.. By accurate i mean like if i will be able to get the
> count for each second also...
I'm not sure it's _that_ accurate, but here it is:
(Unscripted, you need a USAGE table which everything from FORWARD
that you're interested in gets passed through.)
This script is used to create the tables.
#! /usr/bin/perl
for my $i (33..254) {
print "/sbin/iptables -N USAGE_$i\n";
print "/sbin/iptables -A USAGE -d 203.194.23.$i -j USAGE_$i\n";
}
This snippet is part of my RADIUS dial-in script, and adds a link from
the USAGE_nnn table to a table named for the user who is on that IP:
($1 is the IP address, $ACCOUNT_NAME is the account name)
if [ $# -eq 1 -a "x$ACCOUNT_NAME" != "x" -a "x$POOL_NAME" !=
"x\"expired_pool\"" ]; then
CLASS=`echo $1 | /usr/bin/cut -d. -f 4`
SUBNET=`echo $1 | /usr/bin/cut -d. -f 3`
if [ "$SUBNET" = "23" ]; then
TABLE_NAME=`echo $ACCOUNT_NAME`
sudo /sbin/iptables -N USAGE_$TABLE_NAME && sudo /sbin/iptables
-A USAGE_$TABLE_NAME -j ACCEPT || true
sudo /sbin/iptables -F USAGE_$CLASS && sudo /sbin/iptables -A
USAGE_$CLASS -j USAGE_$TABLE_NAME || true
fi
fi
This perl script is run every ten minutes to scrape the usage data.
#! /usr/bin/perl
use strict;
open IPTABLES, "/sbin/iptables -t filter -Z -L -v -x |";
my $table;
my $account;
while () {
$table = $1 if m#^Chain (.*) \(.*\)#;
next unless $table =~ /USAGE_\"(.*)\"/;
$account = $1;
next unless m#^\s+\d+\s+(\d+)\s+ACCEPT#;
next if $1 == 0;
print "$account: $1\n";
}
I hope that helps?
--
Paul "TBBle" Hampson, on an alternate email client.
___
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Packet Counting...
On Wed, Mar 16, 2005 at 09:46:35AM -0600, M. A. Imam wrote: > Hi, > How can i count the number of packets on an interface evry 2 or 5 seconds. > and > i want to count only specific packets like only arriving packets from port > 5001 I knocked up something like that using the built-in counters in iptables. It was pretty nasty, and it's just been replaced with netflow. But it _can_ be done, taking advantage of iptables's atomic display/clear command. Which I forget off hand. ^_^ If you want to go this way, I can post the script I'm using to get the data out of iptables. -- Paul "TBBle" Hampson, on an alternate email client. ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] P2P
On Fri, Mar 11, 2005 at 03:09:58PM -0500, Hugo Martinez wrote: > Im getting into tc. How can I control P2P (peer to peer) traffic??? > which filters??? any ideas??? I ended up with a combination of ipp2p and l7-filter, since they each seemed to pick up stuff the other missed. I used them to jump to connmark, and protected them in front with a connmark check so they didn't waste time checking connections I already knew were P2P. And then anything with connmark went to an IMQ, which I was able to rate-shape depending on time of day. I however found that I ran out of memory, and had to turn down the aging of the conntrack list, on the theory that P2P traffic doesn't spend a lot of time completely idle, being more inclined to short-lived but active connections. I did this on Debian Sarge, and it required kernel rebuilding, _and_ a rebuild of the iptables package. I can post up the patches I used, if you like. I haven't updated it in a fair while, so there's a reasonable chance it's an older version of either ipp2p or l7-filter, or the IMQ patch. -- Paul "TBBle" Hampson, on an alternate email client. ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
[LARTC] Please check that this bandwidth-limiting solution makes sense
I've done the follwing, for my ISP where all connections
are either 512k, 1Mbit or 2Mbit, but we sell accounts
as little as 128kb.
This perl script generates the bulk of my rate shaping setup.
The whole /24 subnet gets filters attached, but customers actuall
only range from 17 to 254.
#! /usr/bin/perl
my $i;
my $tc = '/sbin/tc';
print "$tc qdisc add dev eth0.10 root handle 1: htb default 0\n";
print "$tc filter add dev eth0.10 parent 1: prio 5 protocol ip u32\n";
print "$tc filter add dev eth0.10 parent 1: prio 5 handle 2: protocol ip u32 divisor
256\n";
for $i (1 .. 254) {
#for $i (165 .. 165) {
printf "$tc filter add dev eth0.10 protocol ip parent 1: prio 5 u32 match ip
dst 0/0 ht 2:%x: flowid 1:%d\n", $i, $i;
}
print "$tc filter add dev eth0.10 protocol ip parent 1: prio 5 u32 ht 800:: match ip
dst {customer-subnet}/24 hashkey mask 0x00ff at 12 link 2:\n";
Then when each customer dials in, our RADIUS server calls
the following with $1 as their IP address, and $2 as the kbits of
speed they are allowed:
#! /bin/bash
CLASS=`echo $1 | /usr/bin/cut -d. -f 4-`
if [ $# -eq 2 ]; then
sudo /sbin/tc class del dev eth0.10 parent 1: classid 1:$CLASS || true
sudo /sbin/tc class add dev eth0.10 parent 1: classid 1:$CLASS htb rate $2kbit
|| true
fi
The other problem is that some customers are killing the speed for
others using P2P software, I'm considering using the IMQ device
to prioritise and/or rate shape (using l7-filter from
http://l7-filter.sourceforge.net/ if neccessary) them away from
interfering with WWW/SMTP/POP etc support.
I'm also using the ADSL wonder-shaper on the uplink channel,
since I suspect ACKs are being drowned out at peak times.
I'd appreciate any advice or thoughts here, as I've read the
LARTC HOWTO a couple of times, and I _think_ I get it. I'd like
a little more surety than that though.
Thankyou very much!
--
=
Paul "TBBle" Hampson
Bubblesworth Pty Ltd (ABN: 51 095 284 361)
[EMAIL PROTECTED]
On a sidewalk near Portland State
University someone wrote `Trust Jesus', and
someone else wrote `But Cut the Cards'.
-
Random signature generator 3.0 by Paul "TBBle" Hampson
=
___
LARTC mailing list / [EMAIL PROTECTED]
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
