Paul B. Henson(hen...@acm.org) on 2017.12.19 17:54:48 -0800: > I've got a box with an LTE cellular modem in it whose purpose is to provide > a backup connection to the Internet if the hardwire service goes down. It's > running OSPF to connect to the rest of the network, and the only time any > traffic should go over the cellular link (which is slower and bandwidth > capped) is if the hardwire interconnection is down, including ideally > traffic generated from the system itself. > > I have that part working, by adding in a local static default route to the > cellular gateway with less priority than the OSPF default route. However, > for testing purposes, I'd like to be able to poke out the cellular link on > an as-needed basis without having to switch the entire box over to using it. > Virtual routing tables looked perfect for this purpose, as I could just > spawn a single process with a different default route, we do something > similar with network name spaces under Linux. > > However, I can't quite get it to work. What I'd really like is to be able to > make a copy of the current system routing table, then change one thing about > it. However, a new rdomain shows up with no routes or interfaces in the > routing table. I can add the new default route pointing out the cellular > link, and get traffic to go out there.
When you create a new routing domain, for example by adding an interface to a routing domain (e.g. ifconfig umb0 rdomain 10), you create a new routing table 10. It will be empty until you add an address on umb0 or, for example add your default route. This routing table will be used to forward packets that are "in that routing domain" (the packet is marked with the rdomain or rather the rtable it will use). How does the packet get marked? Three ways: * with pf, as you have discovered. As the manpage documents, the mark needs to be set before route lookup is done. * when a paket comes in on an interface in rdomain 10, it will stay in rdomain 10 (unless pf changes it). * a packet is generated on the local machine by a process that "is in that routing domain". I.e. processes are also marked with a rdomain. To start a process in a specific rdomain (10), use "route -T 10 exec command", for example route -T 10 exec ping -n ip or even route -T 10 exec ksh Processes spawned by that shell will inherit the rdomain. Note that i used -n in the ping example. DNS resolving using the resolvers in resolv.conf might not work, as long as those resolvers are not reachable in rdomain 10. Hope this helps ... > But I haven't sorted out how to make > all the traffic for my internal network still go through the internal link > rather than get sent out the default route. While ideally all the OSPF > routes would propagate to the other routing domain I tried just adding a > static to the /16 for our internal address space: > > Internet: > Destination Gateway Flags Refs Use Mtu Prio > Iface > default 24.x.x.x UGS 0 6 - 8 umb0 > 10.0/16 10.128.0.21 UGS 0 0 - 8 em0 > > That doesn't work; the documentation says you need to get pf to pass packets > across routing domains. However, it says: > > rtable number > Used to select an alternate routing table for the routing lookup. > Only effective before the route lookup happened, i.e. when > filtering inbound. > > Unfortunately, for traffic originating from the system itself, there isn't > really an "inbound" interface? So I'm not sure what pf rule would make this > work. Is it just not possible, or am I missing something? > > Thanks much. > --
