Re: [CentOS] Multiple Internet facing Nics - Gateway issue
Now where would the proper place be to put a route for load balancing like: ip route add default scope global nexthop via xx.yy.51.46 dev eth2 weight 3 nexthop via aa.bb.166.2 dev eth3 weight 1 Hey Doug, Congrats! Now that we have helped you, it is your turn to help us! :-D Let us know when you get that one working! Cheers, Christopher ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
Doug Coats wrote: Christopher - you have been a great help! My internal network ip is 192.168.4.1 and I need it to access the aa.bb.166.2 interface or eth3. What would the rule look like that I need to add? ip route add 192.168.4.0/24 (??? i don't know your subnet - guessing your netmask here) dev ethx (replace x with the 192.168.4.1 interface) proto kernel scope link src 192.168.4.1 table Cable ip route add 192.168.4.0/24 (??? i don't know your subnet - guessing your netmask here) dev ethx (replace x with the 192.168.4.1 interface) proto kernel scope link src 192.168.4.1 table T1 That should fix things for both interfaces. Or use the command only for the table that corresponds to eth3. cheers, Christopher On Thu, Jul 2, 2009 at 6:51 PM, Christopher Chan christopher.c...@bradbury.edu.hk mailto:christopher.c...@bradbury.edu.hk wrote: As a follow up issue. The only thing that is not working properly is that I can not pull up my website that is hosted on this server from our private network. Do I need iproutes for my other two nics? I have never needed them before. That is because you never redirected routing lookups to the custom tables. You can either add routing entries for your internal network into those two custom tables or you can add two SNAT rules assuming you also use the box as a nat box for the Internet. ___ CentOS mailing list CentOS@centos.org mailto:CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
2009/7/2 John R Pierce pie...@hogranch.com: now, these commands are NOT persistent, and, AFAIK, RHEL has no provision for `ip route` or rule commands, so I end up sticking this stuff in /etc/rc.d/rc.local or something. More detail on this: digging a bit on the network-scripts I found that there are few checks and calls to /sbin/ifup-pre-local, /sbin/ifup-local, /sbin/ifdown-pre-local, /sbin/ifdown-local, so I guess that this is the right place to put ip routes and rules commands. Regards Lorenzo Quatrini ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
Lorenzo Quatrini wrote: More detail on this: digging a bit on the network-scripts I found that there are few checks and calls to /sbin/ifup-pre-local, /sbin/ifup-local, /sbin/ifdown-pre-local, /sbin/ifdown-local, so I guess that this is the right place to put ip routes and rules commands. i dunno, i would sort of assume the -local functions are for the localhost interface (/dev/lo), and the idea of having system specific config files in /sbin/ is somewhat abhorrant. in fact, upon some digging, it looks like you put RULES in /etc/sysconfig/network-scripts/rule-ethX and ROUTES in /etc/sysconfig/network-scripts/route-ethX the lines in the rule-* file are run prefixed by `ip rule add` while the lines in the route-* file are prefixed by `ip route add` on an interface 'up' event, and on the corresponding 'down' event, they are prefixed by ip {route|rule} del so in fact, my 'AFAIK there is no provision' is in fact wrong, at least in CentOS 5.x ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
John R Pierce ha scritto: i dunno, i would sort of assume the -local functions are for the localhost interface (/dev/lo), and the idea of having system specific config files in /sbin/ is somewhat abhorrant. I agree! in fact, upon some digging, it looks like you put RULES in /etc/sysconfig/network-scripts/rule-ethX and ROUTES in /etc/sysconfig/network-scripts/route-ethX the lines in the rule-* file are run prefixed by `ip rule add` while the lines in the route-* file are prefixed by `ip route add` on an interface 'up' event, and on the corresponding 'down' event, they are prefixed by ip {route|rule} del So does anyone have a clue on which is the syntax for creating complex routes and rules using route-* and rule-*? ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
in fact, upon some digging, it looks like you put RULES in /etc/sysconfig/network-scripts/rule-ethX and ROUTES in /etc/sysconfig/network-scripts/route-ethX the lines in the rule-* file are run prefixed by `ip rule add` while the lines in the route-* file are prefixed by `ip route add` on an interface 'up' event, and on the corresponding 'down' event, they are prefixed by ip {route|rule} del So does anyone have a clue on which is the syntax for creating complex routes and rules using route-* and rule-*? Heh. I have not had time to go through the network scripts...last thing I heard about was route-* but that was Centos/RHEL 4. Docs have been lacking on this side of things... ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
Adding the routes and rules to the appropriate files in network-scripts did not work. It gave me a number of errors and did not create the rules after reboot or/and network restart. On Fri, Jul 3, 2009 at 11:12 AM, Chan Chung Hang Christopher christopher.c...@bradbury.edu.hk wrote: in fact, upon some digging, it looks like you put RULES in /etc/sysconfig/network-scripts/rule-ethX and ROUTES in /etc/sysconfig/network-scripts/route-ethX the lines in the rule-* file are run prefixed by `ip rule add` while the lines in the route-* file are prefixed by `ip route add` on an interface 'up' event, and on the corresponding 'down' event, they are prefixed by ip {route|rule} del So does anyone have a clue on which is the syntax for creating complex routes and rules using route-* and rule-*? Heh. I have not had time to go through the network scripts...last thing I heard about was route-* but that was Centos/RHEL 4. Docs have been lacking on this side of things... ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
On Fri, Jul 3, 2009 at 1:40 PM, Doug Coats dcoats...@gmail.com wrote: Adding the routes and rules to the appropriate files in network-scripts did not work. It gave me a number of errors and did not create the rules after reboot or/and network restart. You will have to show the file contents and the actual error messages to get any serious help. did not work is not enough info. I have used route-* on CentOS 5. It can work. -Bob ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
Lorenzo Quatrini wrote: So does anyone have a clue on which is the syntax for creating complex routes and rules using route-* and rule-*? the route-if and rule-if are invoked by ifup-routes (all this in /etc/sysconfig/networking-scripts), and it appears they are read and each line is appeneded to /sbin/ip rule add $line or /sbin/ip route add $line respectively. now, it also appears the routes are processed before the rules, not sure, but this could be a complication? actually, upon further perusal, the ROUTES section is parsed as... if [ -f $file ]; then if egrep -q '^[[:space:]]*ADDRESS[0-9]+=' $file ; then # new format handle_file $file ${1%:*} else # older format { cat $file ; echo ; } | while read line; do if [[ ! $line =~ '^[[:space:]]*(\#.*)?$' ]]; then /sbin/ip route add $line fi done fi where 'older format' is as I described, I'm not quite sure what that egrep is looking for, but it appears the new format its looking for is... ADDRESSn=network NETMASKn=prefix GATEWAYn=next-hop (for n=0,1,2,...) and it constructs a /sbin/ip route add network/prefix via next-hop dev if from each triplet of these... AH, yes, here, this shows it... http://kbase.redhat.com/faq/docs/DOC-8024 now, this makes no provisions for routes that have rule table #s. apparently the rules-if files were added to rhel5 and finding /any/ documentation is challenging. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
Bob you are so right! I made the mistake of not removing ip rule add and ip route add from each of the lines. Which caused the script, which wanted to add those itself, get confused. Now I have removed those and it brings up my routes on reboot and network restart. You will have to show the file contents and the actual error messages to get any serious help. did not work is not enough info. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
So for those of you following this thread. This is what I ended up with that seems to be working. /etc/iproute2/rt_tables # # reserved values # 255 local 254 main 253 default 0 unspec # # local # #1 inr.ruhep 1 Cable 2 T1 /etc/sysconfig/network-scripts/rule-eth2 # Creates Rule Cable from xx.yy.51.46 table Cable # Sets up the routing rule from xx.yy.51.45 to default lookup Cable /etc/sysconfig/network-scripts/rule-eth3 # Creates Rule T1 from aa.bb.166.2 table T1 # Sets up the routing rule from aa.bb.166.2 to default lookup T1 /etc/sysconfig/network-scripts/route-eth2 # Sets up routing for Cable xx.yy.51.44/30 dev eth2 src xx.yy.51.45 table Cable default via xx.yy.51.45 table Cable # Sets up main routing table for Cable xx.yy.51.44/30 dev eth2 src xx.yy.51.45 # Sets up default route default via xx.yy.51.46 # Sets up route for internal network 192.168.4.0/24 dev eth0 proto kernel scope link src 192.168.4.1 table Cable /etc/sysconfig/network-scripts/route-eth3 # Sets up routing for T1 aa.bb.166.0/27 dev eth3 src aa.bb.166.2 table T1 default via aa.bb.166.2 table T1 # Sets up main routing table for T1 aa.bb.166.0/27 dev eth3 src aa.bb.166.2 # Sets up routing for internal network 192.168.4.0/24 dev eth0 proto kernel scope link src 192.168.4.1 table T1 Now where would the proper place be to put a route for load balancing like: ip route add default scope global nexthop via xx.yy.51.46 dev eth2 weight 3 nexthop via aa.bb.166.2 dev eth3 weight 1 ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
As a follow up issue. The only thing that is not working properly is that I can not pull up my website that is hosted on this server from our private network. Do I need iproutes for my other two nics? I have never needed them before. That is because you never redirected routing lookups to the custom tables. You can either add routing entries for your internal network into those two custom tables or you can add two SNAT rules assuming you also use the box as a nat box for the Internet. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
Christopher - you have been a great help! My internal network ip is 192.168.4.1 and I need it to access the aa.bb.166.2 interface or eth3. What would the rule look like that I need to add? On Thu, Jul 2, 2009 at 6:51 PM, Christopher Chan christopher.c...@bradbury.edu.hk wrote: As a follow up issue. The only thing that is not working properly is that I can not pull up my website that is hosted on this server from our private network. Do I need iproutes for my other two nics? I have never needed them before. That is because you never redirected routing lookups to the custom tables. You can either add routing entries for your internal network into those two custom tables or you can add two SNAT rules assuming you also use the box as a nat box for the Internet. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
I am ecstatically confused. After I entered the last two commands my routing is working the way that I need it to. ip rule add from 173.11.51.46 table Cable ip rule add from 67.152.166.2 table T1 The problem is I don't know which actual commands worked. I had just rebooted. The other ip commands were all in the rc.local file so they all ran. Why did the above commands make the Cable and T1 rules show up in the rules list but the following two did not? ip route add 173.11.51.44/30 dev eth2 src 173.11.51.45 table Cable ip route add 67.152.166.0/27 dev eth3 src 67.152.166.2 table T1 Do I need all of the commands? So that what is in rc.local looks like this? ip rule add from 173.11.51.46 table Cable ip route add 173.11.51.44/30 dev eth2 src 173.11.51.45 table Cable ip route add default via 173.11.51.45 table Cable ip route add 173.11.51.44/30 dev eth2 src 173.11.51.45 ip route add default via 173.11.51.46 ip rule add from 173.11.51.45 to default lookup Cable ip rule add from 67.152.166.2 table T1 ip route add 67.152.166.0/27 dev eth3 src 67.152.166.2 table T1 ip route add default via 67.152.166.2 table T1 ip route add 67.152.166.0/27 dev eth3 src 67.152.166.2 ip rule add from 67.152.166.2 to default lookup T1 ip route flush cache ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
John R Pierce ha scritto: now, these commands are NOT persistent, and, AFAIK, RHEL has no provision for `ip route` or rule commands, so I end up sticking this stuff in /etc/rc.d/rc.local or something. I was facing the same problem some time ago... what is the right place to put ip route commands and configuration files? Does someone have a better place other than rc.local? If I put them in rc.local, is there a way to issue a command like service network restart? TIA Lorenzo Quatrini ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
On Thu, Jul 2, 2009 at 10:05 AM, Lorenzo Quatrinilorenzo.quatr...@gmail.com wrote: John R Pierce ha scritto: now, these commands are NOT persistent, and, AFAIK, RHEL has no provision for `ip route` or rule commands, so I end up sticking this stuff in /etc/rc.d/rc.local or something. I was facing the same problem some time ago... what is the right place to put ip route commands and configuration files? Does someone have a better place other than rc.local? If I put them in rc.local, is there a way to issue a command like service network restart? TIA Lorenzo Quatrini ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos Yeah, in /etc/sysconfig/network-scripts/ e.g. /etc/sysconfig/network-scripts/route-eth0 I got something like: aaa.bbb.ccc.0/24 dev eth0 in there ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
Lorenzo Quatrini wrote: John R Pierce ha scritto: now, these commands are NOT persistent, and, AFAIK, RHEL has no provision for `ip route` or rule commands, so I end up sticking this stuff in /etc/rc.d/rc.local or something. I was facing the same problem some time ago... what is the right place to put ip route commands and configuration files? Does someone have a better place other than rc.local? Yes, you can create rule-interfacename file under /etc/sysconfig/network-scripts directory for CentOS 5.x If I put them in rc.local, is there a way to issue a command like service network restart? TIA Lorenzo Quatrini ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos -- CL Martinez carlopmart {at} gmail {d0t} com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
Doug Coats ha scritto: I am ecstatically confused. After I entered the last two commands my routing is working the way that I need it to. ip rule add from 173.11.51.46 table Cable ip rule add from 67.152.166.2 table T1 The problem is I don't know which actual commands worked. I had just rebooted. The other ip commands were all in the rc.local file so they all ran. Why did the above commands make the Cable and T1 rules show up in the rules list but the following two did not? ip route add 173.11.51.44/30 http://173.11.51.44/30 dev eth2 src 173.11.51.45 table Cable ip route add 67.152.166.0/27 http://67.152.166.0/27 dev eth3 src 67.152.166.2 table T1 Do I need all of the commands? So that what is in rc.local looks like this? ip rule add from 173.11.51.46 table Cable ip route add 173.11.51.44/30 http://173.11.51.44/30 dev eth2 src 173.11.51.45 table Cable ip route add default via 173.11.51.45 table Cable ip route add 173.11.51.44/30 http://173.11.51.44/30 dev eth2 src 173.11.51.45 ip route add default via 173.11.51.46 ip rule add from 173.11.51.45 to default lookup Cable ip rule add from 67.152.166.2 table T1 ip route add 67.152.166.0/27 http://67.152.166.0/27 dev eth3 src 67.152.166.2 table T1 ip route add default via 67.152.166.2 table T1 ip route add 67.152.166.0/27 http://67.152.166.0/27 dev eth3 src 67.152.166.2 ip rule add from 67.152.166.2 to default lookup T1 ip route flush cache Don't know if can be applied here, but when I did some test I discovered that if you want to have 2 gateways you have to use scope so... could you try doing this: ip route delete default ip route add default scope global nexthop via 173.11.51.46 weight 10 nexthop via 67.152.166.2 weight 20 for what I can recall that was all that I need to tell the system about the dual route. Of course if you want to load-balance adjust the weight to your needs. Hope this helps Regards Lorenzo Quatrini ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
luc...@lastdot.org ha scritto: On Thu, Jul 2, 2009 at 10:05 AM, Lorenzo Quatrinilorenzo.quatr...@gmail.com wrote: John R Pierce ha scritto: now, these commands are NOT persistent, and, AFAIK, RHEL has no provision for `ip route` or rule commands, so I end up sticking this stuff in /etc/rc.d/rc.local or something. I was facing the same problem some time ago... what is the right place to put ip route commands and configuration files? Does someone have a better place other than rc.local? If I put them in rc.local, is there a way to issue a command like service network restart? Yeah, in /etc/sysconfig/network-scripts/ e.g. /etc/sysconfig/network-scripts/route-eth0 I got something like: aaa.bbb.ccc.0/24 dev eth0 in there Yes, that's the default on CentOS... but can I put iproute2 commands and configuration files there? Where do I put the configuration to have something like ip route add default scope global nexthop via aa.bb.cc.dd weight 10 nexthop via xx.yy.zz.tt 1 weight 20 ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
Doug Coats wrote: I am ecstatically confused. After I entered the last two commands my routing is working the way that I need it to. ip rule add from 173.11.51.46 table Cable ip rule add from 67.152.166.2 table T1 The problem is I don't know which actual commands worked. I had just rebooted. The other ip commands were all in the rc.local file so they all ran. Why did the above commands make the Cable and T1 rules show up in the rules list but the following two did not? The following too are not ip rules. They are ip routing table entries. They will show up if you run 'ip route list table Cable' ip route add 173.11.51.44/30 dev eth2 src 173.11.51.45 table Cable ip route add 67.152.166.0/27 dev eth3 src 67.152.166.2 table T1 Do I need all of the commands? So that what is in rc.local looks like this? Take a look at the ones I posted...they are taken from rc.local since I have not taken to effort to integrate them elsewhere per interface. ip rule add from 173.11.51.46 table Cable ip route add 173.11.51.44/30 dev eth2 src 173.11.51.45 table Cable ip route add default via 173.11.51.45 table Cable ip route add 173.11.51.44/30 dev eth2 src 173.11.51.45 ip route add default via 173.11.51.46 ip rule add from 173.11.51.45 to default lookup Cable ip rule add from 67.152.166.2 table T1 ip route add 67.152.166.0/27 dev eth3 src 67.152.166.2 table T1 ip route add default via 67.152.166.2 table T1 ip route add 67.152.166.0/27 dev eth3 src 67.152.166.2 ip rule add from 67.152.166.2 to default lookup T1 ip route flush cache I find it interesting that you have ip rule commands before the relevant ip route commands for the custom routing table... ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
I was simply trying to follow the example in: http://lartc.org/howto/lartc.rpdb.multiple-links.html That is my introduction and main resource for what I was doing. If there is other resorces that any could suggest beyond the man page I wold be interested to understanding this better. As a follow up issue. The only thing that is not working properly is that I can not pull up my website that is hosted on this server from our private network. Do I need iproutes for my other two nics? I have never needed them before. Thanks again for all your help On Thu, Jul 2, 2009 at 8:06 AM, Chan Chung Hang Christopher christopher.c...@bradbury.edu.hk wrote: Doug Coats wrote: I am ecstatically confused. After I entered the last two commands my routing is working the way that I need it to. ip rule add from 173.11.51.46 table Cable ip rule add from 67.152.166.2 table T1 The problem is I don't know which actual commands worked. I had just rebooted. The other ip commands were all in the rc.local file so they all ran. Why did the above commands make the Cable and T1 rules show up in the rules list but the following two did not? The following too are not ip rules. They are ip routing table entries. They will show up if you run 'ip route list table Cable' ip route add 173.11.51.44/30 dev eth2 src 173.11.51.45 table Cable ip route add 67.152.166.0/27 dev eth3 src 67.152.166.2 table T1 Do I need all of the commands? So that what is in rc.local looks like this? Take a look at the ones I posted...they are taken from rc.local since I have not taken to effort to integrate them elsewhere per interface. ip rule add from 173.11.51.46 table Cable ip route add 173.11.51.44/30 dev eth2 src 173.11.51.45 table Cable ip route add default via 173.11.51.45 table Cable ip route add 173.11.51.44/30 dev eth2 src 173.11.51.45 ip route add default via 173.11.51.46 ip rule add from 173.11.51.45 to default lookup Cable ip rule add from 67.152.166.2 table T1 ip route add 67.152.166.0/27 dev eth3 src 67.152.166.2 table T1 ip route add default via 67.152.166.2 table T1 ip route add 67.152.166.0/27 dev eth3 src 67.152.166.2 ip rule add from 67.152.166.2 to default lookup T1 ip route flush cache I find it interesting that you have ip rule commands before the relevant ip route commands for the custom routing table... ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
[CentOS] Multiple Internet facing Nics - Gateway issue
I have a server with 4 nics. Two are using different internet connections, both with static IP's, and two are connected to our private network. The two internet facing nics seem to be battling over the gateway designation. Which ever I designate as the gateway the other stops responding to incoming traffic. I need both to listen to inbound traffic. One for our main web page and the other for another web server. I found some information of Iproute2 but that did not seem to solve my issues. Here are the config files: ifcfg-eth2 DEVICE=eth2 BOOTPROTO=static HWADDR=00:24:E8:52:92:8E ONBOOT=yes IPADDR=xx.yy.51.45 NETMASK=255.255.255.252 GATEWAY=xx.yy.51.46 ifcfg-eth3 DEVICE=eth3 BOOTPROTO=static HWADDR=00:24:E8:52:92:90 ONBOOT=yes IPADDR=aa.bb.166.2 NETMASK=255.255.255.224 GATEWAY=aa.bb.166.1 network NETWORKING=yes NETWORKING_IPV6=no HOSTNAME=heritage01 GATEWAY=xx.yy.51.46 GATEWAYDEV=eth2 rt_tables # # reserved values # 255 local 254 main 253 default 0 unspec # # local # #1 inr.ruhep 200 Cable 201 T1 iproute2 routes ip route add xx.yy.51.44/30 dev eth2 src xx.yy.51.46 table Cable ip route add default via xx.yy.51.46 ip route add xx.yy.51.44/30 dev eth2 src xx.yy.51.46 ip route add default via xx.yy.51.46 ip rule add from xx.yy.51.46 table Cable ip route add aa.bb.166.0/27 dev eth3 src aa.bb.165.2 table T1 ip route add default via aa.bb.166.1 ip route add aa.bb.166.0/27 dev eth3 src aa.bb.166.2 ip rule add from aa.bb.166.2 table T1 ip route flush cache Other people seem to be using this configuration successfully but I seem to be missing something important. Any help is greatly appreciated!!! Thanks! ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
On Wed, 1 Jul 2009, Doug Coats wrote: I have a server with 4 nics. Two are using different internet connections, both with static IP's, and two are connected to our private network. The two internet facing nics seem to be battling over the gateway designation. Which ever I designate as the gateway the other stops responding to incoming traffic. I need both to listen to inbound traffic. One for our main web page and the other for another web server. I found some information of Iproute2 but that did not seem to solve my issues. Here are the config files: ifcfg-eth2 DEVICE=eth2 BOOTPROTO=static HWADDR=00:24:E8:52:92:8E ONBOOT=yes IPADDR=xx.yy.51.45 NETMASK=255.255.255.252 GATEWAY=xx.yy.51.46 ifcfg-eth3 DEVICE=eth3 BOOTPROTO=static HWADDR=00:24:E8:52:92:90 ONBOOT=yes IPADDR=aa.bb.166.2 NETMASK=255.255.255.224 GATEWAY=aa.bb.166.1 network NETWORKING=yes NETWORKING_IPV6=no HOSTNAME=heritage01 GATEWAY=xx.yy.51.46 GATEWAYDEV=eth2 remove the gateway from /etc/sysconfig/network, reboot and try again. -- James A. Peltier Systems Analyst (FASNet), VIVARIUM Technical Director HPC Coordinator Simon Fraser University - Burnaby Campus Phone : 778-782-6573 Fax : 778-782-3045 E-Mail : jpelt...@sfu.ca Website : http://www.fas.sfu.ca | http://vivarium.cs.sfu.ca http://blogs.sfu.ca/people/jpeltier MSN : subatomic_s...@hotmail.com The point of the HPC scheduler is to keep everyone equally unhappy.___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
Doug Coats schrieb: I have a server with 4 nics. Two are using different internet connections, both with static IP's, and two are connected to our private network. The two internet facing nics seem to be battling over the gateway designation. Which ever I designate as the gateway the other stops responding to incoming traffic. I need both to listen to inbound traffic. One for our main web page and the other for another web server. I found some information of Iproute2 but that did not seem to solve my issues. Here are the config files: ifcfg-eth2 DEVICE=eth2 BOOTPROTO=static HWADDR=00:24:E8:52:92:8E ONBOOT=yes IPADDR=xx.yy.51.45 NETMASK=255.255.255.252 GATEWAY=xx.yy.51.46 ifcfg-eth3 DEVICE=eth3 BOOTPROTO=static HWADDR=00:24:E8:52:92:90 ONBOOT=yes IPADDR=aa.bb.166.2 NETMASK=255.255.255.224 GATEWAY=aa.bb.166.1 network NETWORKING=yes NETWORKING_IPV6=no HOSTNAME=heritage01 GATEWAY=xx.yy.51.46 GATEWAYDEV=eth2 rt_tables # # reserved values # 255 local 254 main 253 default 0 unspec # # local # #1 inr.ruhep 200 Cable 201 T1 iproute2 routes ip route add xx.yy.51.44/30 dev eth2 src xx.yy.51.46 table Cable ip route add default via xx.yy.51.46 ip route add xx.yy.51.44/30 dev eth2 src xx.yy.51.46 ip route add default via xx.yy.51.46 ip rule add from xx.yy.51.46 table Cable ip route add aa.bb.166.0/27 dev eth3 src aa.bb.165.2 table T1 ip route add default via aa.bb.166.1 ip route add aa.bb.166.0/27 dev eth3 src aa.bb.166.2 ip rule add from aa.bb.166.2 table T1 ip route flush cache Other people seem to be using this configuration successfully but I seem to be missing something important. Any help is greatly appreciated!!! Thanks! The GATEWAY parameter within the ifcfg-ethX configuration files tries to set the default gateway. There can be just 1 default gateway for a system. So, the interface which comes up first will set the route and the other has no effect. Remove the GATEWAY entry from all ifcfg-ethX files and place the GATEWAY parameter into /etc/sysconfig/network only. Running different networks you must can set additional routings by route-ethX configuration files inside /etc/sysconfig/network-scripts/. Given your default gateway is set as xx.yy.51.46 for the network configured on eth2, you set to use a gateway aa.bb.166.1 for other destinations by ip route add targetnet/CIDR via aa.bb.166.1 dev eth3 Please see http://www.centos.org/docs/5/html/5.1/Deployment_Guide/s1-networkscripts-static-routes.html You multiple uplink topic is documented in this howto http://lartc.org/howto/lartc.rpdb.multiple-links.html Alexander ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
Thanks for your responses. I have tried some of the configurations that you have mentioned but not all of them. I will try the others tonight when the server is less active. In the meantime I will read the links you sent. Thanks again. This is starting to drive me crazy. On Wed, Jul 1, 2009 at 3:07 PM, Alexander Dalloz ad+li...@uni-x.orgad%2bli...@uni-x.org wrote: Doug Coats schrieb: I have a server with 4 nics. Two are using different internet connections, both with static IP's, and two are connected to our private network. The two internet facing nics seem to be battling over the gateway designation. Which ever I designate as the gateway the other stops responding to incoming traffic. I need both to listen to inbound traffic. One for our main web page and the other for another web server. I found some information of Iproute2 but that did not seem to solve my issues. Here are the config files: ifcfg-eth2 DEVICE=eth2 BOOTPROTO=static HWADDR=00:24:E8:52:92:8E ONBOOT=yes IPADDR=xx.yy.51.45 NETMASK=255.255.255.252 GATEWAY=xx.yy.51.46 ifcfg-eth3 DEVICE=eth3 BOOTPROTO=static HWADDR=00:24:E8:52:92:90 ONBOOT=yes IPADDR=aa.bb.166.2 NETMASK=255.255.255.224 GATEWAY=aa.bb.166.1 network NETWORKING=yes NETWORKING_IPV6=no HOSTNAME=heritage01 GATEWAY=xx.yy.51.46 GATEWAYDEV=eth2 rt_tables # # reserved values # 255 local 254 main 253 default 0 unspec # # local # #1 inr.ruhep 200 Cable 201 T1 iproute2 routes ip route add xx.yy.51.44/30 dev eth2 src xx.yy.51.46 table Cable ip route add default via xx.yy.51.46 ip route add xx.yy.51.44/30 dev eth2 src xx.yy.51.46 ip route add default via xx.yy.51.46 ip rule add from xx.yy.51.46 table Cable ip route add aa.bb.166.0/27 dev eth3 src aa.bb.165.2 table T1 ip route add default via aa.bb.166.1 ip route add aa.bb.166.0/27 dev eth3 src aa.bb.166.2 ip rule add from aa.bb.166.2 table T1 ip route flush cache Other people seem to be using this configuration successfully but I seem to be missing something important. Any help is greatly appreciated!!! Thanks! The GATEWAY parameter within the ifcfg-ethX configuration files tries to set the default gateway. There can be just 1 default gateway for a system. So, the interface which comes up first will set the route and the other has no effect. Remove the GATEWAY entry from all ifcfg-ethX files and place the GATEWAY parameter into /etc/sysconfig/network only. Running different networks you must can set additional routings by route-ethX configuration files inside /etc/sysconfig/network-scripts/. Given your default gateway is set as xx.yy.51.46 for the network configured on eth2, you set to use a gateway aa.bb.166.1 for other destinations by ip route add targetnet/CIDR via aa.bb.166.1 dev eth3 Please see http://www.centos.org/docs/5/html/5.1/Deployment_Guide/s1-networkscripts-static-routes.html You multiple uplink topic is documented in this howto http://lartc.org/howto/lartc.rpdb.multiple-links.html Alexander ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
ip rule add from xx.yy.51.46 table Cable ip rule add from aa.bb.166.2 table T1 Other people seem to be using this configuration successfully but I seem to be missing something important. Change the above rules to: ip rule add from xx.yy.51.46 to default lookup Cable ip rule add from aa.bb.166.2 to default lookup T1 ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
I changed those two rules to no affect. I have also done the above mentioned things. My tcpdump shows connections comming in on eth3 but there is no outbound communication. Example: tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth3, link-type EN10MB (Ethernet), capture size 96 bytes 20:21:11.915624 IP 41.178.204.108.25591 ipaa-bb-166-2.z166-152-67.customer.algx.net.smtp: S 3528290937:3528290937(0) win 65535 mss 1442,nop,wscale 2,nop,nop,sackOK 20:21:14.892106 IP 41.178.204.108.25591 ipaa-bb-166-2.z166-152-67.customer.algx.net.smtp: S 3528290937:3528290937(0) win 65535 mss 1442,nop,wscale 2,nop,nop,sackOK 20:21:16.017705 20:21:16.663301 IP newsletter.publicradio.org.54562 ipaa-bb-166-2.z166-152-67.customer.algx.net.smtp: S 1665702165:1665702165(0) win 5840 mss 1460,sackOK,timestamp 3732071063 0,nop,wscale 3 20:21:17.658835 IP client-201.230.112.209.speedy.net.pe.12095 ipaa-bb-166-2.z166-152-67.customer.algx.net.smtp: S 2778637809:2778637809(0) win 65535 mss 1400,nop,wscale 1,nop,nop,sackOK An abreviated lsof -i4 -n COMMANDPID USER FD TYPE DEVICE SIZE NODE NAME mysqld4004 mysql 10u IPv4 8750 TCP *:mysql (LISTEN) spamd 4056 root5u IPv4 8943 TCP 127.0.0.1:783 (LISTEN) sendmail 4076 root4u IPv4 9094 TCP *:smtp (LISTEN) httpd 4110 root4u IPv4 9117 TCP aa.bb.166.2:http (LISTEN) httpd 4166 apache4u IPv4 9117 TCP aa.bb.166.2:http (LISTEN) httpd 4167 apache4u IPv4 9117 TCP aa.bb.166.2:http (LISTEN) httpd 4168 apache4u IPv4 9117 TCP aa.bb.166.2:http (LISTEN) So it appears to me that the device is listening but it never picks up the phone. Any thoughts? On Wed, Jul 1, 2009 at 7:00 PM, Christopher Chan christopher.c...@bradbury.edu.hk wrote: ip rule add from xx.yy.51.46 table Cable ip rule add from aa.bb.166.2 table T1 Other people seem to be using this configuration successfully but I seem to be missing something important. Change the above rules to: ip rule add from xx.yy.51.46 to default lookup Cable ip rule add from aa.bb.166.2 to default lookup T1 ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
Hello Doug, Here are the iproute2 commands for a multigateway setup I did. ip route add 220.232.217.72/29 dev eth0 proto kernel scope link src 220.232.217.78 table pacific ip route add 10.9.0.0/17 dev eth1 proto kernel scope link src 10.9.1.101 table pacific ip route add default via 220.232.217.73 dev eth0 table pacific ip route add 203.174.45.24/29 dev eth2 proto kernel scope link src 203.174.45.30 table wharf ip route add 10.9.0.0/17 dev eth1 proto kernel scope link src 10.9.1.101 table wharf ip route add default via 203.174.45.25 dev eth2 table wharf ip route add 59.152.193.16/29 dev eth2 proto kernel scope link src 59.152.193.22 table frahw ip route add 10.9.0.0/17 dev eth1 proto kernel scope link src 10.9.1.101 table frahw ip route add default via 59.152.193.17 dev eth2 table wharf ip route add 10.9.0.0/17 dev eth1 proto kernel scope link src 10.9.1.101 table saurer ip route add 10.0.0.0/8 via 10.9.0.1 dev eth1 table saurer ip rule add from 220.232.217.78 to default lookup pacific ip rule add from 203.174.45.30 to default lookup wharf You may also want to check your firewall rules too. I must also say that there are no GATEWAY entries in my ifcfg-ethx. Only one in network. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
Thanks for your responce! If I open up Iptables to accept everything on eth3 and eth2 or turn it off all together I get the same results. I will have to look through your iproute2 commands to see how they match up to mine. Do you see anything different in mine that would indicate the issue? On Wed, Jul 1, 2009 at 9:25 PM, Christopher Chan christopher.c...@bradbury.edu.hk wrote: Hello Doug, Here are the iproute2 commands for a multigateway setup I did. ip route add 220.232.217.72/29 dev eth0 proto kernel scope link src 220.232.217.78 table pacific ip route add 10.9.0.0/17 dev eth1 proto kernel scope link src 10.9.1.101 table pacific ip route add default via 220.232.217.73 dev eth0 table pacific ip route add 203.174.45.24/29 dev eth2 proto kernel scope link src 203.174.45.30 table wharf ip route add 10.9.0.0/17 dev eth1 proto kernel scope link src 10.9.1.101 table wharf ip route add default via 203.174.45.25 dev eth2 table wharf ip route add 59.152.193.16/29 dev eth2 proto kernel scope link src 59.152.193.22 table frahw ip route add 10.9.0.0/17 dev eth1 proto kernel scope link src 10.9.1.101 table frahw ip route add default via 59.152.193.17 dev eth2 table wharf ip route add 10.9.0.0/17 dev eth1 proto kernel scope link src 10.9.1.101 table saurer ip route add 10.0.0.0/8 via 10.9.0.1 dev eth1 table saurer ip rule add from 220.232.217.78 to default lookup pacific ip rule add from 203.174.45.30 to default lookup wharf You may also want to check your firewall rules too. I must also say that there are no GATEWAY entries in my ifcfg-ethx. Only one in network. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
I am not understanding somethings very fundamental to me troubleshooting this issue. How do I clear the iproute2's that I have created? How do I tell if they are in affect? What should they look like when listed if they are configured properly? Thanks for holding my hand in this! ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
Doug Coats wrote: Thanks for your responce! If I open up Iptables to accept everything on eth3 and eth2 or turn it off all together I get the same results. I will have to look through your iproute2 commands to see how they match up to mine. Do you see anything different in mine that would indicate the issue? Well, besides the differences in ip route command parameters nothing sticks out at the moment. What do get when you run 'ip rule list'? Are you doing any NATing on this box too? On Wed, Jul 1, 2009 at 9:25 PM, Christopher Chan christopher.c...@bradbury.edu.hk mailto:christopher.c...@bradbury.edu.hk wrote: Hello Doug, Here are the iproute2 commands for a multigateway setup I did. ip route add 220.232.217.72/29 http://220.232.217.72/29 dev eth0 proto kernel scope link src 220.232.217.78 table pacific ip route add 10.9.0.0/17 http://10.9.0.0/17 dev eth1 proto kernel scope link src 10.9.1.101 table pacific ip route add default via 220.232.217.73 dev eth0 table pacific ip route add 203.174.45.24/29 http://203.174.45.24/29 dev eth2 proto kernel scope link src 203.174.45.30 table wharf ip route add 10.9.0.0/17 http://10.9.0.0/17 dev eth1 proto kernel scope link src 10.9.1.101 table wharf ip route add default via 203.174.45.25 dev eth2 table wharf ip route add 59.152.193.16/29 http://59.152.193.16/29 dev eth2 proto kernel scope link src 59.152.193.22 table frahw ip route add 10.9.0.0/17 http://10.9.0.0/17 dev eth1 proto kernel scope link src 10.9.1.101 table frahw ip route add default via 59.152.193.17 dev eth2 table wharf ip route add 10.9.0.0/17 http://10.9.0.0/17 dev eth1 proto kernel scope link src 10.9.1.101 table saurer ip route add 10.0.0.0/8 http://10.0.0.0/8 via 10.9.0.1 dev eth1 table saurer ip rule add from 220.232.217.78 to default lookup pacific ip rule add from 203.174.45.30 to default lookup wharf You may also want to check your firewall rules too. I must also say that there are no GATEWAY entries in my ifcfg-ethx. Only one in network. ___ CentOS mailing list CentOS@centos.org mailto:CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
I am convinced that my iproute2 tables are all messed upIn fact I found some errors in my initial post. # ip rule list 0: from all lookup 255 32766: from all lookup main 32767: from all lookup default /etc/iproute2/rt_tables # # reserved values # 255 local 254 main 253 default 0 unspec # # local # #1 inr.ruhep 200 Cable 201 T1 So Cable and T1 are not even showing up in the list. When I run my ip commands from the command line I get the following results for each line. # ip route add 173.11.51.44/30 dev eth2 src 173.11.51.45 table Cable # ip route add default via 173.11.51.45 table Cable # ip route add 173.11.51.44/30 dev eth2 src 173.11.51.45 RTNETLINK answers: File exists # ip route add default via 173.11.51.46 RTNETLINK answers: File Exists # ip rule add from 173.11.51.45 to default lookup Cable RTNETLINK answers: Numerical result out of range # ip route add 67.152.166.0/27 dev eth3 src 67.152.166.2 table T1 # ip route add default via 67.152.166.2 table T1 # ip route add 67.152.166.0/27 dev eth3 src 67.152.166.2 RTNETLINK answers: File Exists # ip rule add from 67.152.166.2 to default lookup T1 RTNETLINK answers: Numerical result out of range So the first few lines run fine for each table and then I start getting errors. Are these all the commands that I need to use to set this up? I know that when I first entered them by hand the T1 and Cable showed up in the tables list. Am I going crazy? ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
Doug Coats wrote: I am not understanding somethings very fundamental to me troubleshooting this issue. How do I clear the iproute2's that I have created? They are cleared by being deleted or a reboot takes place. Try 'ip route help' and 'ip rule help' How do I tell if they are in affect? Now that is a good one...ping/telnet tests from another host... What should they look like when listed if they are configured properly? Here is my output from ip rule list 0:from all lookup local 32764:from 203.174.45.30 lookup wharf 32765:from 220.232.217.78 lookup pacific 32766:from all lookup main 32767:from all lookup default Lower numbers mean higher priority. Routing tables: ip route list table wharf: 203.174.45.24/29 dev eth2 proto kernel scope link src 203.174.45.30 10.9.0.0/17 dev eth1 proto kernel scope link src 10.9.1.101 default via 203.174.45.25 dev eth2 ip route list table pacific 220.232.217.72/29 dev eth0 proto kernel scope link src 220.232.217.78 10.9.0.0/17 dev eth1 proto kernel scope link src 10.9.1.101 default via 220.232.217.73 dev eth0 ip route list will give you your main routing table which is the same one that route -n will report ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
Doug Coats wrote: I am convinced that my iproute2 tables are all messed upIn fact I found some errors in my initial post. # ip rule list 0: from all lookup 255 32766: from all lookup main 32767: from all lookup default Boy, did we miss that... /etc/iproute2/rt_tables # # reserved values # 255 local 254 main 253 default 0 unspec # # local # #1 inr.ruhep 200 Cable 201 T1 So Cable and T1 are not even showing up in the list. I use lower values ( 50 ) and I do not use CAPS. Don't know if the CAPS make a difference. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
So if I run the following two commands the rules show up in the list. ip rule add from 173.11.51.46 table Cable ip rule add from 67.152.166.2 table T1 # ip rule list 0: from all lookup 255 32764: from 67.152.166.2 lookup T1 32765: from 173.11.51.46 lookup Cable 32766: from all lookup main 32767: from all lookup default So why does that work but my other command not? ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
Doug Coats wrote: So if I run the following two commands the rules show up in the list. ip rule add from 173.11.51.46 table Cable ip rule add from 67.152.166.2 table T1 # ip rule list 0: from all lookup 255 32764: from 67.152.166.2 lookup T1 32765: from 173.11.51.46 lookup Cable 32766: from all lookup main 32767: from all lookup default So why does that work but my other command not? Hmm...I wonder if they have been changes to iproute2 from RHEL4 to RHEL5... Now that your ip rules are in place...do things work for you? ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
Doug Coats wrote: So if I run the following two commands the rules show up in the list. ip rule add from 173.11.51.46 table Cable ip rule add from 67.152.166.2 table T1 # ip rule list 0: from all lookup 255 32764: from 67.152.166.2 lookup T1 32765: from 173.11.51.46 lookup Cable 32766: from all lookup main 32767: from all lookup default So why does that work but my other command not? Which other commands? The ip route ones? ip route list table T1 ip route list table Cable ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Multiple Internet facing Nics - Gateway issue
Doug Coats wrote: So if I run the following two commands the rules show up in the list. ip rule add from 173.11.51.46 table Cable ip rule add from 67.152.166.2 table T1 # ip rule list 0: from all lookup 255 32764: from 67.152.166.2 lookup T1 32765: from 173.11.51.46 lookup Cable 32766: from all lookup main 32767: from all lookup default So why does that work but my other command not? what other command? the rt_tables file just associates the names T1 and Cable with rule #s, it doesn't actually define the rules, for that you need to use the ip rule add ... commands. me, I make one interface the 'default', and just define ONE extra route for the 'other' network say eth0 is 100.100.16.15/24 with a default gateway of 100.100.16.1, and eth1 is 200.200.18.30/24 with a route of 200.200.18.1 ... I'd setup the system so the global default gateway is the one on eth0 via /etc/sysconfig/network then, I'd define ONE extra rule, and one extra route table entry... ip rule add from 200.200.18.30 table Alt ip route add default via 200.200.18.1 dev eth1 table Alt now, these commands are NOT persistent, and, AFAIK, RHEL has no provision for `ip route` or rule commands, so I end up sticking this stuff in /etc/rc.d/rc.local or something. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos