Re: [pfSense Support] Outbound NAT to Virt. IP issues. Maybe it's the config, maybe it's VMWare ESXi?
On Mon, Dec 22, 2008 at 5:31 PM, Jason Lixfeld jason-lists.pfse...@lixfeld.ca wrote: Hi Dimitri, It is a CARP address, yes and it does in fact match the mask on the WAN interface; they are both /28. After doing some more digging, I figured it out. It was a VMWare thing. I had to set the virtual adapter with a security policy exception to allow promiscuous mode. There seems to be another issue though - it seems as though there is another client out there on the WAN (albeit, on a different VLAN) using a pfSense box, because I see the same MAC address as what my pfSense box is using for my CARP MAC Address. Probably VRRP is what you're seeing. http://en.wikipedia.org/wiki/Common_Address_Redundancy_Protocol#History for history on this. --Bill - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org
RE: [pfSense Support] Outbound NAT to Virt. IP issues. Maybe it's the config, maybe it's VMWare ESXi?
What kind of Virtual IP are you using? If you are using CARP addresses (which is what I'm using), make sure your subnet mask actually matches your WAN interface subnet mask. Dimitri Rodis Integrita Systems LLC -Original Message- From: Jason Lixfeld [mailto:jason-lists.pfse...@lixfeld.ca] Sent: Monday, December 22, 2008 8:04 AM To: support@pfsense.com Subject: [pfSense Support] Outbound NAT to Virt. IP issues. Maybe it's the config, maybe it's VMWare ESXi? Hello, and happy holidays! I have an ESXi server installed with the 1.2.1-RC2 VM upgraded to RC4 up and running. Everything has been working as expected, but then I tried to setup outbound NAT to a virtual IP and everything stopped: I've configured a Virtual IP on the WAN side which is on the same subnet as the WAN interface itself. I have an outbound NAT rule set up to nat all outbound connections to the Virtual IP. I also have the outbound NAT set for Manual Outbound NAT rule generation (Advanced Outbound NAT (AON)). From the WAN side, I see the MAC for both the virtual IP and the physical WAN interface IP but I can't ping the Virtual IP however I can ping the physical WAN interface IP, no problem. As soon as I set outbound NAT to Automatic Outbound NAT rule generation, traffic works again (albeit I still can't ping the virtual IP, but at that point, it's moot). I checked the pfSense firewall rules and verified that it's configured to pass ICMP from any to any on the WAN interface and the LAN interface has a rule to allow IP from any to any, so by all accounts this should be working. I'm not sure if it's something in pfSense that I'm doing wrong, or if it's a VMWare issue. The fact that I can see the MAC Address on the WAN side seems to indicate that ESXi is doing what it's supposed to. I haven't seen any indication that ESXi doesn't want to pass traffic for a virtual MAC address while I've been looking over it's configuration, so I'm at a loss and I'm wondering if anyone has any insight. Just for completeness, here's the ARP table from a 3550 I have on the WAN side to verify it sees the MAC address and ARP, etc. I've also included the ifconfig from the pfSense shell. switchshow arp | i Vlan5 Internet aaa.bbb.ccc.215 - 000b.5f33.6100 ARPA Vlan5 Internet aaa.bbb.ccc.209 0 0013.5f1e.93c0 ARPA Vlan5 Internet aaa.bbb.ccc.211 16 000c.291b.3c6f ARPA Vlan5 Internet aaa.bbb.ccc.210 17 .5e00.0101 ARPA Vlan5 switchshow mac-address-table | i Fa0/1 5.5e00.0101DYNAMIC Fa0/1 5000c.291b.3c6fDYNAMIC Fa0/1 .215 is the 3550 I'm using to verify the WAN side. .209 is the default gateway for the pfSense box that leads to the intermaweb. .210 is the virtual IP. .211 is the physical IP. switchping aaa.bbb.ccc.209 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to aaa.bbb.ccc.209, timeout is 2 seconds: ! Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms switchping aaa.bbb.ccc.211 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to aaa.bbb.ccc.211, timeout is 2 seconds: ! Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms switchping aaa.bbb.ccc.210 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to aaa.bbb.ccc.210, timeout is 2 seconds: . Success rate is 0 percent (0/5) switch # ifconfig le0: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu 1500 options=8VLAN_MTU ether 00:0c:29:1b:3c:65 inet 10.1.11.1 netmask 0xff00 broadcast 10.1.11.255 inet6 fe80::20c:29ff:fe1b:3c65%le0 prefixlen 64 scopeid 0x1 media: Ethernet autoselect status: active le1: flags=8943UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST metric 0 mtu 1500 options=8VLAN_MTU ether 00:0c:29:1b:3c:6f inet6 fe80::20c:29ff:fe1b:3c6f%le1 prefixlen 64 scopeid 0x2 inet aaa.bbb.ccc.211 netmask 0xfff0 broadcast aaa.bbb.ccc.223 media: Ethernet autoselect status: active le2: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu 1500 options=8VLAN_MTU ether 00:0c:29:1b:3c:79 inet 10.255.255.1 netmask 0xff00 broadcast 10.255.255.255 inet6 fe80::20c:29ff:fe1b:3c79%le2 prefixlen 64 scopeid 0x3 media: Ethernet autoselect status: active plip0: flags=108810POINTOPOINT,SIMPLEX,MULTICAST,NEEDSGIANT metric 0 mtu 1500 pfsync0: flags=41UP,RUNNING metric 0 mtu 1460 pfsync: syncdev: lo0 syncpeer: 224.0.0.240 maxupd: 128 lo0: flags=8049UP,LOOPBACK,RUNNING,MULTICAST metric 0 mtu 16384 inet 127.0.0.1 netmask 0xff00 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6 enc0: flags=0 metric 0 mtu 1536 pflog0: flags=100PROMISC metric 0 mtu 33204 tun0: flags=8051UP,POINTOPOINT,RUNNING,MULTICAST metric 0 mtu 1500 inet6 fe80::20c:29ff:fe1b:3c65%tun0 prefixlen 64 scopeid 0x9 inet 192.0.2.1 -- 192.0.2.2 netmask 0x Opened
Re: [pfSense Support] Outbound NAT to Virt. IP issues. Maybe it's the config, maybe it's VMWare ESXi?
Hi Dimitri, It is a CARP address, yes and it does in fact match the mask on the WAN interface; they are both /28. After doing some more digging, I figured it out. It was a VMWare thing. I had to set the virtual adapter with a security policy exception to allow promiscuous mode. There seems to be another issue though - it seems as though there is another client out there on the WAN (albeit, on a different VLAN) using a pfSense box, because I see the same MAC address as what my pfSense box is using for my CARP MAC Address. Is there a way to change the CARP MAC address so I can differentiate my MAC address from this other person's? On 22-Dec-08, at 6:03 PM, Dimitri Rodis wrote: What kind of Virtual IP are you using? If you are using CARP addresses (which is what I'm using), make sure your subnet mask actually matches your WAN interface subnet mask. Dimitri Rodis Integrita Systems LLC -Original Message- From: Jason Lixfeld [mailto:jason-lists.pfse...@lixfeld.ca] Sent: Monday, December 22, 2008 8:04 AM To: support@pfsense.com Subject: [pfSense Support] Outbound NAT to Virt. IP issues. Maybe it's the config, maybe it's VMWare ESXi? Hello, and happy holidays! I have an ESXi server installed with the 1.2.1-RC2 VM upgraded to RC4 up and running. Everything has been working as expected, but then I tried to setup outbound NAT to a virtual IP and everything stopped: I've configured a Virtual IP on the WAN side which is on the same subnet as the WAN interface itself. I have an outbound NAT rule set up to nat all outbound connections to the Virtual IP. I also have the outbound NAT set for Manual Outbound NAT rule generation (Advanced Outbound NAT (AON)). From the WAN side, I see the MAC for both the virtual IP and the physical WAN interface IP but I can't ping the Virtual IP however I can ping the physical WAN interface IP, no problem. As soon as I set outbound NAT to Automatic Outbound NAT rule generation, traffic works again (albeit I still can't ping the virtual IP, but at that point, it's moot). I checked the pfSense firewall rules and verified that it's configured to pass ICMP from any to any on the WAN interface and the LAN interface has a rule to allow IP from any to any, so by all accounts this should be working. I'm not sure if it's something in pfSense that I'm doing wrong, or if it's a VMWare issue. The fact that I can see the MAC Address on the WAN side seems to indicate that ESXi is doing what it's supposed to. I haven't seen any indication that ESXi doesn't want to pass traffic for a virtual MAC address while I've been looking over it's configuration, so I'm at a loss and I'm wondering if anyone has any insight. Just for completeness, here's the ARP table from a 3550 I have on the WAN side to verify it sees the MAC address and ARP, etc. I've also included the ifconfig from the pfSense shell. switchshow arp | i Vlan5 Internet aaa.bbb.ccc.215 - 000b.5f33.6100 ARPA Vlan5 Internet aaa.bbb.ccc.209 0 0013.5f1e.93c0 ARPA Vlan5 Internet aaa.bbb.ccc.211 16 000c.291b.3c6f ARPA Vlan5 Internet aaa.bbb.ccc.210 17 .5e00.0101 ARPA Vlan5 switchshow mac-address-table | i Fa0/1 5.5e00.0101DYNAMIC Fa0/1 5000c.291b.3c6fDYNAMIC Fa0/1 .215 is the 3550 I'm using to verify the WAN side. .209 is the default gateway for the pfSense box that leads to the intermaweb. .210 is the virtual IP. .211 is the physical IP. switchping aaa.bbb.ccc.209 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to aaa.bbb.ccc.209, timeout is 2 seconds: ! Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms switchping aaa.bbb.ccc.211 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to aaa.bbb.ccc.211, timeout is 2 seconds: ! Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms switchping aaa.bbb.ccc.210 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to aaa.bbb.ccc.210, timeout is 2 seconds: . Success rate is 0 percent (0/5) switch # ifconfig le0: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu 1500 options=8VLAN_MTU ether 00:0c:29:1b:3c:65 inet 10.1.11.1 netmask 0xff00 broadcast 10.1.11.255 inet6 fe80::20c:29ff:fe1b:3c65%le0 prefixlen 64 scopeid 0x1 media: Ethernet autoselect status: active le1: flags=8943UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST metric 0 mtu 1500 options=8VLAN_MTU ether 00:0c:29:1b:3c:6f inet6 fe80::20c:29ff:fe1b:3c6f%le1 prefixlen 64 scopeid 0x2 inet aaa.bbb.ccc.211 netmask 0xfff0 broadcast aaa.bbb.ccc.223 media: Ethernet autoselect status: active le2: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu 1500 options=8VLAN_MTU ether 00:0c:29:1b:3c:79 inet 10.255.255.1 netmask 0xff00 broadcast 10.255.255.255 inet6 fe80::20c:29ff:fe1b:3c79%le2 prefixlen 64 scopeid 0x3 media: Ethernet autoselect
Re: [pfSense Support] Outbound NAT to Virt. IP issues. Maybe it's the config, maybe it's VMWare ESXi?
On Mon, Dec 22, 2008 at 6:31 PM, Jason Lixfeld jason-lists.pfse...@lixfeld.ca wrote: Hi Dimitri, It is a CARP address, yes and it does in fact match the mask on the WAN interface; they are both /28. After doing some more digging, I figured it out. It was a VMWare thing. I had to set the virtual adapter with a security policy exception to allow promiscuous mode. There seems to be another issue though - it seems as though there is another client out there on the WAN (albeit, on a different VLAN) using a pfSense box, because I see the same MAC address as what my pfSense box is using for my CARP MAC Address. Is there a way to change the CARP MAC address so I can differentiate my MAC address from this other person's? Use a different VHID. - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org
