This doesn't really have anything to do with equal-cost-multipath, so
I'm starting a new thread on it. For those who missed my previous posts,
here's the story so far:
I have two ISP's with different filtering rules. Both provide boxes with
10BaseT interfaces and interfaces to their respective network technology
(cable and DSL). They are each connected to an ethernet card in my
gateway. Behind the gateway is a private network which accesses the
two ISP's via IP masquerade. The gateway also runs a number of services
such as FTP, HTTP, and SMTP--I have another machine between the gateway
and the private network that provides security with a much simpler
configuration.
The ISP on eth1 (eth1ISP) rejects packets with source addresses that
are not on its subnet, but otherwise does not restrict traffic in either
direction. The IP address is static so it can be used for incoming
connections, and there is no possiblity of having the IP address change
while connections or masquerading are active.
The ISP on eth2 (eth2ISP) rejects packets with destination TCP port 80,
has no restrictions on source address, and allows all other packets.
The IP address is dynamic so it can only be used for outgoing connections.
The IP address for eth1ISP can be used as a source address here.
I use static subnet routes to unconditionally assign traffic local to
each ISP to that ISP's ethernet interface.
The executive summary is that I want to set up the gateway so that each
ISP is used most effectively. This means a complicated setup, with parts
in user space and parts in kernel space. However, I suspect it will
become a more common case as many people can now get multiple >=1-megabit
Internet feeds to their homes for less than the cost of dedicated
28.8K dialup.
Here's what I want to do:
1. Ping each ISP's gateway with a user-space daemon and adjust the
default route if either gateway or some well-connected external host
behind that gateway becomes unreachable. Leave subnet routes for the
gateways themselves so that the pinging still can happen when one ISP is
abandoned and the default gateway directed to the other. The change in
gateway can be achieved by replacing the default route with a route to
teql0, eth1, or eth2 as appropriate.
2a. Route packets to TCP port 80 only through eth1ISP if eth1ISP is
reachable.
2b. Route packets to TCP port 80 through a transparent proxy to eth2ISP's
proxy server if eth1ISP is not reachable. The transparent proxy will
rewrite the HTTP requests as proxy requests.
Switching from 2a to 2b and back can be handled by the user-space daemon
in #1.
3. Route all other packets to both ISP's network interfaces with
eth1ISP's address as source. For IP masquerade, always use eth1ISP's
address on masqueraded connections that go through a default route.
Connections that go through a static subnet route should always go
through that static subnet route because they're ISP-specific.
#1 is trivial; it's not even a kernel issue. #3 is also trivial, although
actually getting it to work is a bit of a challenge. equal-cost-multipath
doesn't do real load balancing when it works, and doesn't work in all
the cases I expect it to. The teql packet scheduler has some problems
accounting each interface separately--something I'd like to keep track
of because eth2ISP charges by the byte while eth1ISP doesn't.
#2 is not quite so trivial. There's no "route by destination port number"
option in iproute2 and routing by fwmark as shipped in Linux 2.2.9
doesn't do anything useful for me.
The problem with ipchains/fwmark is that ipchains is called in the wrong
place to be useful. The "input" and "forward" chains don't handle packets
that originate on the local machine, including outgoing masqueraded
packets. "output" is called too late to influence routing decisions
about the packet; basically all ipchains can do in the "output" stage
is feed extra information to the packet schedulers or do simple allow/deny
decisions. It can't decide what source address a packet should have,
or which interface it is to be routed on.
I propose adding a chain called "route" that is called before routing
tables are applied to a packet. For the purposes of ipchains, a packet
that is being forwarded would have the interface set to the interface that
the packet arrived on, and would have its original source IP address.
A packet that originates on the local machine from a program would have
the interface name set to "LOCAL" (or perhaps to the empty string?),
and the source address is either what was set by the program with bind(),
or if no local address has been assigned, use 0.0.0.0/32 and port 0.
Such packets could then be marked by ipchains, and the existing fwmark
routing code can be used to route them based on that mark. It should be
possible to redirect into the transparent proxy from this chain as well.
>From the five minutes I just spent looking at the code, it looks
like it's fairly simple to add support for #2a above to ip_queue_xmit()
or ip_route_output(). Another chain would have to be set up in the
ipchains initialization code. I don't think ipchains (the user-space
program) needs to be modified at all for this, but that's just a guess.
#2b above (route outgoing packets to port 80 through transparent proxy)
looks like it will require some extra work as the packets have to end
up being fed back into the input so that the transparent proxy can get
at them. It may be possible to do this without further kernel
modifications by routing to the loopback interface, then setting up
ipchains like this:
# The following code is only used in case 2b above. In case 2a,
# the user-space daemon disables the following rules.
# 10.0.0.0 is the private network that is masquerading.
# Mark packets that would be routed to port 80, so the routing
# code knows there to route them. This relies on the proposed
# 'route' chain above.
ipchains -A route -d ! 10.0.0.0/8 80 -p tcp -m 1
# Route marked packets through loopback, setting a special source
# IP address which we can search for later.
ip route add table 50 0/0 src 127.0.1.1 via 127.0.0.1 dev lo
ip rule add fwmark 1 table 50
# Transparent proxy on the looped-back packets. Note we check
# for our special address.
ipchains -A input -i lo -s 127.0.1.1 -d ! 127.0.0.0/8 80 \
-p tcp -j REDIRECT <proxy server port # here>
# The transparent proxy now forwards the connection to a proxy
# server, after rewriting the HTTP request as a proxy request.
Of course I haven't tested this because it relies on code that doesn't
exist yet. :-/ I also hear that the loopback interface doesn't allow
this sort of thing in Linux 2.2 like it did in 2.0.
What do the gurus here think about this? Have I missed any major
details? Minor ones?
--
Zygo Blaxell, Linux Engineer, Corel Corporation. [EMAIL PROTECTED] (work) or
[EMAIL PROTECTED] (play). Opinions above are my own, not Corel's.
Size of 'diff -Nurw [...] winehq corel' as of Thu Jun 3 13:14:00 EDT 1999
Lines/files: In 20094 / 98, Out 17319 / 152, Both 12093 / 142
-
To unsubscribe from this list: send the line "unsubscribe linux-net" in
the body of a message to [EMAIL PROTECTED]
