[pfSense-discussion] Problems with CARP VIP and layer 3 switch
Hi I have two pfSense machines configured as cluster using carp, they are both connected to a layer 3 switch. There are about 10 different subnets configured on that and each client machine under these subnets use the switch as its default gateway, and then it routes the traffic. 10.10.0.210.10.0.3 --- | pfSense | - | pfSense | --- VIP 10.10.0.1 \/ \ / - | switch | - / \ / \ 10.10.1.0/24 10.10.2.0/24 The problem is that every time a configuration is changed, I can access the VIP with no problem from the same subnet of the pfSense machine (10.10.0.0/24), but for any other subnet the VIP becomes unreachable. Has anyone seen this problem before? If I'm not so clear in my explanation, please let me know. Thanks -- Vinícius Coque - To unsubscribe, e-mail: discussion-unsubscr...@pfsense.com For additional commands, e-mail: discussion-h...@pfsense.com Commercial support available - https://portal.pfsense.org
Re: [pfSense-discussion] Problems with CARP VIP and layer 3 switch
On Wed, Apr 13, 2011 at 10:32 PM, Vinicius Coque wrote: > Hi > > I have two pfSense machines configured as cluster using carp, they are > both connected to a layer 3 switch. There are about 10 different > subnets configured on that and each client machine under these subnets > use the switch as its default gateway, and then it routes the traffic. > > 10.10.0.2 10.10.0.3 > --- > | pfSense | - | pfSense | > --- > VIP 10.10.0.1 > \ / > \ / > - > | switch | > - > / \ > / \ > 10.10.1.0/24 10.10.2.0/24 > > The problem is that every time a configuration is changed, I can > access the VIP with no problem from the same subnet of the pfSense > machine (10.10.0.0/24), but for any other subnet the VIP becomes > unreachable. > Some kind of routing issue it seems. Check the routing table on the firewall when it doesn't work and verify it. - To unsubscribe, e-mail: discussion-unsubscr...@pfsense.com For additional commands, e-mail: discussion-h...@pfsense.com Commercial support available - https://portal.pfsense.org
Re: [pfSense-discussion] Problems with CARP VIP and layer 3 switch
> > Some kind of routing issue it seems. Check the routing table on the > firewall when it doesn't work and verify it. > Hi Chris I don't think it is a routing issue because I can access the VIP and the pfSense lan IP from other subnets. When I change some configuration on cluster just the VIP goes down, while the lan IP of the pfSense boxes (10.10.0.2 and 10.10.0.3) are still available. -- Vinícius Coque - To unsubscribe, e-mail: discussion-unsubscr...@pfsense.com For additional commands, e-mail: discussion-h...@pfsense.com Commercial support available - https://portal.pfsense.org
Re: [pfSense-discussion] Problems with CARP VIP and layer 3 switch
On Thu, Apr 14, 2011 at 5:57 AM, Vinicius Coque wrote: > > I don't think it is a routing issue because I can access the VIP and > the pfSense lan IP from other subnets. When I change some > configuration on cluster just the VIP goes down, while the lan IP of > the pfSense boxes (10.10.0.2 and 10.10.0.3) are still available. > What does the CARP status show, and what do the logs show for CARP? - To unsubscribe, e-mail: discussion-unsubscr...@pfsense.com For additional commands, e-mail: discussion-h...@pfsense.com Commercial support available - https://portal.pfsense.org
Re: [pfSense-discussion] Problems with CARP VIP and layer 3 switch
> > What does the CARP status show, and what do the logs show for CARP? > CARP Status pfSense master: vip1 172.16.0.39 MASTER pfSense backup: vip1 172.16.0.39 BACKUP System logs: pfSense master: Apr 15 17:08:08 utm-teste1 syslogd: kernel boot file is /boot/kernel/kernel Apr 15 20:08:32 utm-teste1 check_reload_status: syncing firewall Apr 15 17:08:32 utm-teste1 php: : Beginning XMLRPC sync to https://10.10.0.2:5081. Apr 15 17:08:33 utm-teste1 php: : XMLRPC sync successfully completed with https://10.10.0.2:5081. Apr 15 17:08:33 utm-teste1 php: : Beginning XMLRPC sync to https://10.10.0.2:5081. Apr 15 17:08:33 utm-teste1 php: : XMLRPC sync successfully completed with https://10.10.0.2:5081. Apr 15 17:08:35 utm-teste1 php: : Filter sync successfully completed with https://10.10.0.2:5081. pfSense backup: Apr 15 17:08:12 utm-teste2 syslogd: kernel boot file is /boot/kernel/kernel Apr 15 17:08:32 utm-teste2 check_reload_status: syncing firewall Apr 15 17:08:32 utm-teste2 kernel: vip1: link state changed to DOWN Apr 15 17:08:32 utm-teste2 kernel: vip1: INIT -> MASTER (preempting) Apr 15 17:08:32 utm-teste2 kernel: vip1: link state changed to UP Apr 15 17:08:32 utm-teste2 kernel: vip1: MASTER -> BACKUP (more frequent advertisement received) Apr 15 17:08:32 utm-teste2 kernel: vip1: link state changed to DOWN Apr 15 17:08:32 utm-teste2 check_reload_status: syncing firewall Apr 15 17:08:33 utm-teste2 kernel: vip1: link state changed to DOWN Apr 15 17:08:33 utm-teste2 kernel: vip1: INIT -> MASTER (preempting) Apr 15 17:08:33 utm-teste2 kernel: vip1: link state changed to UP Apr 15 17:08:33 utm-teste2 check_reload_status: reloading filter Apr 15 17:08:33 utm-teste2 php: /xmlrpc.php: ROUTING: change default route to 172.16.0.36 Apr 15 17:08:33 utm-teste2 apinger: Exiting on signal 15. Apr 15 17:08:33 utm-teste2 kernel: vip1: MASTER -> BACKUP (more frequent advertisement received) Apr 15 17:08:33 utm-teste2 kernel: vip1: link state changed to DOWN Apr 15 17:08:33 utm-teste2 php: : ROUTING: change default route to 172.16.0.36 Apr 15 17:08:34 utm-teste2 apinger: Starting Alarm Pinger, apinger(30466) Apr 15 17:08:34 utm-teste2 php: /xmlrpc.php: Resyncing OpenVPN instances. Apr 15 17:08:34 utm-teste2 dnsmasq[13434]: exiting on receipt of SIGTERM Apr 15 17:08:35 utm-teste2 dnsmasq[31870]: started, version 2.55 cachesize 1 Apr 15 17:08:35 utm-teste2 dnsmasq[31870]: compile time options: IPv6 GNU-getopt no-DBus I18N DHCP TFTP Apr 15 17:08:35 utm-teste2 dnsmasq[31870]: reading /etc/resolv.conf Apr 15 17:08:35 utm-teste2 dnsmasq[31870]: using nameserver 8.8.4.4#53 Apr 15 17:08:35 utm-teste2 dnsmasq[31870]: using nameserver 8.8.8.8#53 Apr 15 17:08:35 utm-teste2 dnsmasq[31870]: read /etc/hosts - 2 addresses - To unsubscribe, e-mail: discussion-unsubscr...@pfsense.com For additional commands, e-mail: discussion-h...@pfsense.com Commercial support available - https://portal.pfsense.org
Re: [pfSense-discussion] Problems with CARP VIP and layer 3 switch
On Fri, Apr 15, 2011 at 4:14 PM, Vinicius Coque wrote: >> >> What does the CARP status show, and what do the logs show for CARP? >> > > > CARP Status > pfSense master: > > vip1 172.16.0.39 MASTER > > pfSense backup: > > vip1 172.16.0.39 BACKUP > > > System logs: > > pfSense master: > > Apr 15 17:08:08 utm-teste1 syslogd: kernel boot file is /boot/kernel/kernel > Apr 15 20:08:32 utm-teste1 check_reload_status: syncing firewall > Apr 15 17:08:32 utm-teste1 php: : Beginning XMLRPC sync to > https://10.10.0.2:5081. > Apr 15 17:08:33 utm-teste1 php: : XMLRPC sync successfully completed > with https://10.10.0.2:5081. > Apr 15 17:08:33 utm-teste1 php: : Beginning XMLRPC sync to > https://10.10.0.2:5081. > Apr 15 17:08:33 utm-teste1 php: : XMLRPC sync successfully completed > with https://10.10.0.2:5081. > Apr 15 17:08:35 utm-teste1 php: : Filter sync successfully completed > with https://10.10.0.2:5081. > > pfSense backup: > > Apr 15 17:08:12 utm-teste2 syslogd: kernel boot file is /boot/kernel/kernel > Apr 15 17:08:32 utm-teste2 check_reload_status: syncing firewall > Apr 15 17:08:32 utm-teste2 kernel: vip1: link state changed to DOWN > Apr 15 17:08:32 utm-teste2 kernel: vip1: INIT -> MASTER (preempting) > Apr 15 17:08:32 utm-teste2 kernel: vip1: link state changed to UP > Apr 15 17:08:32 utm-teste2 kernel: vip1: MASTER -> BACKUP (more > frequent advertisement received) That looks like a consequence of: http://redmine.pfsense.org/issues/1433 plus something on your switch(es). The MAC will move in the switch's CAM table from the primary's port to the secondary's when the secondary switches from master to backup even though it's for a fraction of a second, but should immediately move back on the switch when the master picks back up. There's something on the switch that isn't behaving correctly for MACs that quickly change ports, which is ultimately the actual problem, though that CARP switch shouldn't happen during a config change which exacerbates the issue. - To unsubscribe, e-mail: discussion-unsubscr...@pfsense.com For additional commands, e-mail: discussion-h...@pfsense.com Commercial support available - https://portal.pfsense.org
Re: [pfSense-discussion] Problems with CARP VIP and layer 3 switch
On Fri, Apr 15, 2011 at 7:31 PM, Chris Buechler wrote: > On Fri, Apr 15, 2011 at 4:14 PM, Vinicius Coque wrote: >>> >>> What does the CARP status show, and what do the logs show for CARP? >>> >> >> >> CARP Status >> pfSense master: >> >> vip1 172.16.0.39 MASTER >> >> pfSense backup: >> >> vip1 172.16.0.39 BACKUP >> >> >> System logs: >> >> pfSense master: >> >> Apr 15 17:08:08 utm-teste1 syslogd: kernel boot file is /boot/kernel/kernel >> Apr 15 20:08:32 utm-teste1 check_reload_status: syncing firewall >> Apr 15 17:08:32 utm-teste1 php: : Beginning XMLRPC sync to >> https://10.10.0.2:5081. >> Apr 15 17:08:33 utm-teste1 php: : XMLRPC sync successfully completed >> with https://10.10.0.2:5081. >> Apr 15 17:08:33 utm-teste1 php: : Beginning XMLRPC sync to >> https://10.10.0.2:5081. >> Apr 15 17:08:33 utm-teste1 php: : XMLRPC sync successfully completed >> with https://10.10.0.2:5081. >> Apr 15 17:08:35 utm-teste1 php: : Filter sync successfully completed >> with https://10.10.0.2:5081. >> >> pfSense backup: >> >> Apr 15 17:08:12 utm-teste2 syslogd: kernel boot file is /boot/kernel/kernel >> Apr 15 17:08:32 utm-teste2 check_reload_status: syncing firewall >> Apr 15 17:08:32 utm-teste2 kernel: vip1: link state changed to DOWN >> Apr 15 17:08:32 utm-teste2 kernel: vip1: INIT -> MASTER (preempting) >> Apr 15 17:08:32 utm-teste2 kernel: vip1: link state changed to UP >> Apr 15 17:08:32 utm-teste2 kernel: vip1: MASTER -> BACKUP (more >> frequent advertisement received) > > That looks like a consequence of: > http://redmine.pfsense.org/issues/1433 > > plus something on your switch(es). The MAC will move in the switch's > CAM table from the primary's port to the secondary's when the > secondary switches from master to backup even though it's for a > fraction of a second, but should immediately move back on the switch > when the master picks back up. There's something on the switch that > isn't behaving correctly for MACs that quickly change ports, which is > ultimately the actual problem, though that CARP switch shouldn't > happen during a config change which exacerbates the issue. > > - > To unsubscribe, e-mail: discussion-unsubscr...@pfsense.com > For additional commands, e-mail: discussion-h...@pfsense.com > > Commercial support available - https://portal.pfsense.org > > Now I understand the problem. I'll keep track of the bug on redmine. Thanks for helping Chris. -- Vinícius Coque - To unsubscribe, e-mail: discussion-unsubscr...@pfsense.com For additional commands, e-mail: discussion-h...@pfsense.com Commercial support available - https://portal.pfsense.org
Re: [pfSense-discussion] Problems with CARP VIP and layer 3 switch
On Sun, Apr 17, 2011 at 10:25 PM, Vinicius Coque wrote: > > Now I understand the problem. I'll keep track of the bug on redmine. > I would definitely check the problem on the switch too as in a CARP setup it shouldn't have problems with MACs that switch between ports quickly. That bug in and of itself isn't the problem, the nature of CARP means that switch issue will potentially cause other issues for you in the future. - To unsubscribe, e-mail: discussion-unsubscr...@pfsense.com For additional commands, e-mail: discussion-h...@pfsense.com Commercial support available - https://portal.pfsense.org
Re: [pfSense-discussion] Problems with CARP VIP and layer 3 switch
On Sun, Apr 17, 2011 at 11:49 PM, Chris Buechler wrote: > On Sun, Apr 17, 2011 at 10:25 PM, Vinicius Coque wrote: >> >> Now I understand the problem. I'll keep track of the bug on redmine. >> > > I would definitely check the problem on the switch too as in a CARP > setup it shouldn't have problems with MACs that switch between ports > quickly. That bug in and of itself isn't the problem, the nature of > CARP means that switch issue will potentially cause other issues for > you in the future. > > - > To unsubscribe, e-mail: discussion-unsubscr...@pfsense.com > For additional commands, e-mail: discussion-h...@pfsense.com > > Commercial support available - https://portal.pfsense.org > > My client really needs the cluster working, so I have to find a solution for that. Now you gave me more information about the problem, I'll check the switch and the CARP setup and see what I can get. If something works for me I'll inform you. - To unsubscribe, e-mail: discussion-unsubscr...@pfsense.com For additional commands, e-mail: discussion-h...@pfsense.com Commercial support available - https://portal.pfsense.org
RE: [pfSense-discussion] Problems with CARP VIP and layer 3 switch
> From: Vinicius Coque [mailto:vco...@gmail.com] > Sent: Monday, April 18, 2011 08:01 > > On Sun, Apr 17, 2011 at 11:49 PM, Chris Buechler wrote: > > On Sun, Apr 17, 2011 at 10:25 PM, Vinicius Coque wrote: > >> > >> Now I understand the problem. I'll keep track of the bug on redmine. > > > > I would definitely check the problem on the switch too as in a CARP > > setup it shouldn't have problems with MACs that switch between ports > > quickly. That bug in and of itself isn't the problem, the nature of > > CARP means that switch issue will potentially cause other issues for > > you in the future. > > My client really needs the cluster working, so I have to find a solution for > that. Now you gave me more information about the problem, I'll check > the switch and the CARP setup and see what I can get. If something > works for me I'll inform you. Can you tell us what model of switch(es) is(are) involved here? There are some specific configurations that can cause issues, others on the list may be able to make suggestions. -Adam Thompson athom...@athompso.net - To unsubscribe, e-mail: discussion-unsubscr...@pfsense.com For additional commands, e-mail: discussion-h...@pfsense.com Commercial support available - https://portal.pfsense.org
Re: [pfSense-discussion] Problems with CARP VIP and layer 3 switch
On Mon, Apr 18, 2011 at 10:32 AM, Adam Thompson wrote: >> From: Vinicius Coque [mailto:vco...@gmail.com] >> Sent: Monday, April 18, 2011 08:01 >> >> On Sun, Apr 17, 2011 at 11:49 PM, Chris Buechler > wrote: >> > On Sun, Apr 17, 2011 at 10:25 PM, Vinicius Coque > wrote: >> >> >> >> Now I understand the problem. I'll keep track of the bug on > redmine. >> > >> > I would definitely check the problem on the switch too as in a CARP >> > setup it shouldn't have problems with MACs that switch between ports >> > quickly. That bug in and of itself isn't the problem, the nature of >> > CARP means that switch issue will potentially cause other issues for >> > you in the future. >> >> My client really needs the cluster working, so I have to find a > solution for >> that. Now you gave me more information about the problem, I'll check >> the switch and the CARP setup and see what I can get. If something >> works for me I'll inform you. > > > Can you tell us what model of switch(es) is(are) involved here? There > are some specific configurations that can cause issues, others on the > list may be able to make suggestions. > > -Adam Thompson > athom...@athompso.net > > > > > - > To unsubscribe, e-mail: discussion-unsubscr...@pfsense.com > For additional commands, e-mail: discussion-h...@pfsense.com > > Commercial support available - https://portal.pfsense.org > > Hi Adam We are using two switches HP E5500-24G -- Vinícius Coque - To unsubscribe, e-mail: discussion-unsubscr...@pfsense.com For additional commands, e-mail: discussion-h...@pfsense.com Commercial support available - https://portal.pfsense.org
Re: [pfSense-discussion] Problems with CARP VIP and layer 3 switch
On 11-04-18 09:47 AM, Vinicius Coque wrote: On Mon, Apr 18, 2011 at 10:32 AM, Adam Thompson wrote: From: Vinicius Coque [mailto:vco...@gmail.com] Sent: Monday, April 18, 2011 08:01 On Sun, Apr 17, 2011 at 11:49 PM, Chris Buechler wrote: On Sun, Apr 17, 2011 at 10:25 PM, Vinicius Coque wrote: Now I understand the problem. I'll keep track of the bug on redmine. I would definitely check the problem on the switch too as in a CARP setup it shouldn't have problems with MACs that switch between ports quickly. That bug in and of itself isn't the problem, the nature of CARP means that switch issue will potentially cause other issues for you in the future. My client really needs the cluster working, so I have to find a solution for that. Now you gave me more information about the problem, I'll check the switch and the CARP setup and see what I can get. If something works for me I'll inform you. Can you tell us what model of switch(es) is(are) involved here? There are some specific configurations that can cause issues, others on the list may be able to make suggestions. -Adam Thompson athom...@athompso.net - To unsubscribe, e-mail: discussion-unsubscr...@pfsense.com For additional commands, e-mail: discussion-h...@pfsense.com Commercial support available - https://portal.pfsense.org Hi Adam We are using two switches HP E5500-24G -- Vinícius Coque These switches should be able to do not only carp but cook for you and clean your house -) Do quick test. Determine on which port of the switch VIP's MAC is located where you are running without problem, then introduce the problem and watch where this MAC is now, does it correspond to where you see Active VIP? At the same time I would run tcpdump on both hosts to see who is advertising as MASTER. Evgeny. - To unsubscribe, e-mail: discussion-unsubscr...@pfsense.com For additional commands, e-mail: discussion-h...@pfsense.com Commercial support available - https://portal.pfsense.org
