Final email in this thread, for correctness. The initial request disappearing and the firewalls staying demoted "forever" are independent issues. A new request for bulk transfer is sent after 2h+. Due to bulk transfer performance the transfers never finish. I see on average 3kpps of pfsync on this cluster, looking at pfsync this is what I find:
12:02:45.778145 10.240.252.78 > 10.240.252.77: PFSYNCv6 len 36 act UPD ST REQ count 1 id: 0000000000000000 creatorid: 00000000 12:02:45.778153 10.240.252.77 > 10.240.252.78: PFSYNCv6 len 1412 act BULK UPD STAT count 1 creatorid: b33d7f45 age: 00:00:00 status: start 14:16:09.689102 10.240.252.78 > 10.240.252.77: PFSYNCv6 len 1264 act UPD ST REQ count 1 id: 0000000000000000 creatorid: 00000000 14:16:09.689114 10.240.252.77 > 10.240.252.78: PFSYNCv6 len 124 act BULK UPD STAT count 1 creatorid: b33d7f45 age: 00:00:00 status: start 16:29:33.604110 10.240.252.78 > 10.240.252.77: PFSYNCv6 len 36 act UPD ST REQ count 1 id: 0000000000000000 creatorid: 00000000 16:29:33.604120 10.240.252.77 > 10.240.252.78: PFSYNCv6 len 544 act BULK UPD STAT count 1 creatorid: b33d7f45 age: 00:00:00 status: start 18:42:57.518630 10.240.252.78 > 10.240.252.77: PFSYNCv6 len 124 act UPD ST REQ count 1 id: 0000000000000000 creatorid: 00000000 18:42:57.518634 10.240.252.77 > 10.240.252.78: PFSYNCv6 len 1384 act BULK UPD STAT count 1 creatorid: b33d7f45 age: 00:00:00 status: start 20:56:21.433270 10.240.252.78 > 10.240.252.77: PFSYNCv6 len 208 act UPD ST REQ count 1 id: 0000000000000000 creatorid: 00000000 20:56:21.433283 10.240.252.77 > 10.240.252.78: PFSYNCv6 len 628 act BULK UPD STAT count 1 creatorid: b33d7f45 age: 00:00:00 status: start 23:09:45.347531 10.240.252.78 > 10.240.252.77: PFSYNCv6 len 36 act UPD ST REQ count 1 id: 0000000000000000 creatorid: 00000000 23:09:45.347534 10.240.252.77 > 10.240.252.78: PFSYNCv6 len 292 act BULK UPD STAT count 1 creatorid: b33d7f45 age: 00:00:00 status: start 01:23:09.262083 10.240.252.78 > 10.240.252.77: PFSYNCv6 len 36 act UPD ST REQ count 1 id: 0000000000000000 creatorid: 00000000 01:23:09.262093 10.240.252.77 > 10.240.252.78: PFSYNCv6 len 712 act BULK UPD STAT count 1 creatorid: b33d7f45 age: 00:00:00 status: start 03:36:33.176294 10.240.252.78 > 10.240.252.77: PFSYNCv6 len 616 act UPD ST REQ count 1 id: 0000000000000000 creatorid: 00000000 03:36:33.176300 10.240.252.77 > 10.240.252.78: PFSYNCv6 len 628 act BULK UPD STAT count 1 creatorid: b33d7f45 age: 00:00:00 status: start 05:49:57.090125 10.240.252.78 > 10.240.252.77: PFSYNCv6 len 124 act UPD ST REQ count 1 id: 0000000000000000 creatorid: 00000000 05:49:57.090130 10.240.252.77 > 10.240.252.78: PFSYNCv6 len 1132 act BULK UPD STAT count 1 creatorid: b33d7f45 age: 00:00:00 status: start /T On Tue, Sep 2, 2014 at 12:07 PM, Tony Sarendal <t...@polarcap.org> wrote: > As Chuck pointed out this has nothing to do with pfsense or freebsd. > > While I dig deeper I'm running with the following config to get around the > problem: > pf1.swe1# cat /etc/hostname.pfsync0 > ! sleep 10 > ! ifconfig $1 syncdev vlan44 syncpeer 10.240.252.77 up > > pf1.swe1# > > I see the request for the bulk transfer now, and the bulk transfer > starting. > Although bulk transfer performance looks like a problem, but that is for > another thread. > > /T > > > > On Sat, Aug 30, 2014 at 9:31 PM, System Administrator <ad...@bitwise.net> > wrote: > >> And what does OP's message have to do with pfSense ??? (especially >> since he's clearly indicating currently supported OpenBSD versions 5.4 >> and 5.5 near the bottom...) >> >> On 30 Aug 2014 at 14:22, Chuck Burns wrote: >> >> > On Saturday, August 30, 2014 8:27:24 AM Tony Sarendal wrote: >> > > Good morning, >> > > >> > > I'm having issues with pfsync on trunk interfaces, although I suspect >> > > it to >> > <snip> >> > > Running on pfsync on trunk(4) that initial request never shows up, and >> > > the bulk update never starts/finishes. I would like to run pfsync on >> > > trunk(4) lacp link, but as it looks now I have firewalls with carp >> > > demote counter 33 forever. >> > <snip> >> > >> > pfSense is FreeBSD-based. not OpenBSD-based... >> > >> > different versions of pf between OpenBSD and FreeBSD >> > >> > -- >> > Chuck Burns >> > Audemus Jura Nostra Defendere