Pfsync states and reply-to
Hello, Does pfsync currently support failover of a pf established 'reply-to'/'route-to' states when Carp failover happens? I`ve noticed that states created with pf rules that contains 'reply-to'/'route-to', after switching to backup server are using default route/routing table instead of routes that were supplied in 'reply-to'/'route-to' pf rules. I know you were working around it from: http://marc.info/?l=openbsd-tech&m=128940646028112&w=2 http://marc.info/?l=openbsd-tech&m=126403343102126&w=2 Do you plan to include syncing 'reply-to'/'route-to' information in pf states in near future? Regards. Dariusz Binkul
Block with reply-to
Fellow users, do I understand correctly that RST replies to packets blocked with pf cannot be arbitrarily routed? pf.conf(5) says that "(...) reply-to is useful only in rules that create state". Since 'block' and 'match' rules seem to (understandably) not create state entries, there is no apparent way to direct TCP-RST (and/or ICMP unreachable) replies to a route of traffic being blocked. In my environment they all go through default gateway. Is there something that I'm missing or is it a bug or a feature (should I use route(8) tables instead, perhaps)? Thanks,
Re: Pfsync bulk for 6 states takes 13 minutes
Hello, I reproduced the problem on: OpenBSD 5.4 (GENERIC.MP) #41: Tue Jul 30 15:30:02 MDT 2013 pfsync takes as much time as whith OpenBSD 5.3. If you have any ideas why is that happening, please let me know. I have a test env with physical access prepared specially to solve this case. Remote access for OpenBSD developers is an option too. Aug 5 09:56:43 pfw1 /bsd: root on wd0a (89609dda54f2ae25.a) swap on wd0b dump on wd0b Aug 5 09:56:43 pfw1 /bsd: carp: carp10 demoted group carp by 1 to 129 (carpdev) Aug 5 09:56:43 pfw1 /bsd: carp: pfsync0 demoted group carp by 32 to 161 (pfsync init) Aug 5 09:56:43 pfw1 /bsd: carp: pfsync0 demoted group pfsync by 32 to 32 (pfsync init) Aug 5 09:56:43 pfw1 /bsd: carp: pfsync0 demoted group carp by 1 to 162 (pfsync bulk start) Aug 5 09:56:43 pfw1 /bsd: carp: pfsync0 demoted group pfsync by 1 to 33 (pfsync bulk start) Aug 5 09:56:43 pfw1 savecore: no core dump Aug 5 09:56:45 pfw1 /bsd: carp10: state transition: INIT -> BACKUP Aug 5 09:56:45 pfw1 /bsd: carp: carp10 demoted group carp by -1 to 161 (carpdev) Aug 5 09:57:00 pfw1 /bsd: carp10: state transition: BACKUP -> MASTER Aug 5 09:57:00 pfw1 /bsd: carp10: state transition: MASTER -> BACKUP Aug 5 10:00:01 pfw1 newsyslog[9921]: logfile turned over Aug 5 10:00:01 pfw1 newsyslog[9921]: logfile turned over Aug 5 10:00:01 pfw1 syslogd: restart Aug 5 10:10:06 pfw1 /bsd: carp: pfsync0 demoted group carp by -1 to 32 (pfsync bulk done) Aug 5 10:10:06 pfw1 /bsd: carp: pfsync0 demoted group pfsync by -1 to 32 (pfsync bulk done) Aug 5 10:10:06 pfw1 /bsd: carp: pfsync0 demoted group carp by -32 to 0 (pfsync init) Aug 5 10:10:06 pfw1 /bsd: carp: pfsync0 demoted group pfsync by -32 to 0 (pfsync init) Aug 5 10:10:08 pfw1 /bsd: carp10: state transition: BACKUP -> MASTER Regards -- Dariusz Binkul 2013/8/2 Kenneth R Westerback > On Fri, Aug 02, 2013 at 12:04:24PM +0200, Dariusz Binkul wrote: > > Hello, > > > > I have 2 openbsd systems (OpenBSD 5.3 (GENERIC.MP) #62: Tue Mar 12 > 18:21:20 > > MDT 2013) in active-backup configuration. > > During boot of OS, pfsync bulk takes 13 minutes to sync with master > server > > no matter how many states there are to sync. > > > > I've reproduced this problem in my testing environment. Even with no > > traffic (only 6 states in PF State Table) pfsync bulk took 13 minutes. > > As you have a test setup, you might get more immediate attention > if you reproduced the problem on the latest 5.4 snapshot. > > Ken
Pfsync bulk for 6 states takes 13 minutes
Hello, I have 2 openbsd systems (OpenBSD 5.3 (GENERIC.MP) #62: Tue Mar 12 18:21:20 MDT 2013) in active-backup configuration. During boot of OS, pfsync bulk takes 13 minutes to sync with master server no matter how many states there are to sync. I've reproduced this problem in my testing environment. Even with no traffic (only 6 states in PF State Table) pfsync bulk took 13 minutes. Logs from operation: Aug 2 09:47:02 pfw1 /bsd: root on wd0a (89609dda54f2ae25.a) swap on wd0b dump on wd0b Aug 2 09:47:02 pfw1 /bsd: carp: carp10 demoted group carp by 1 to 129 (carpdev) Aug 2 09:47:02 pfw1 /bsd: carp: pfsync0 demoted group carp by 32 to 161 (pfsync init) Aug 2 09:47:02 pfw1 /bsd: carp: pfsync0 demoted group pfsync by 32 to 32 (pfsync init) Aug 2 09:47:02 pfw1 /bsd: carp: pfsync0 demoted group carp by 1 to 162 (pfsync bulk start) Aug 2 09:47:02 pfw1 /bsd: carp: pfsync0 demoted group pfsync by 1 to 33 (pfsync bulk start) Aug 2 09:47:01 pfw1 savecore: no core dump Aug 2 09:47:06 pfw1 /bsd: carp10: state transition: INIT -> BACKUP Aug 2 09:47:06 pfw1 /bsd: carp: carp10 demoted group carp by -1 to 33 (carpdev) Aug 2 10:00:01 pfw1 newsyslog[16429]: logfile turned over tail: /var/log/messages has been replaced, reopening. Aug 2 10:00:01 pfw1 newsyslog[16429]: logfile turned over Aug 2 10:00:01 pfw1 syslogd: restart Aug 2 10:00:24 pfw1 /bsd: carp: pfsync0 demoted group carp by -1 to 32 (pfsync bulk done) Aug 2 10:00:24 pfw1 /bsd: carp: pfsync0 demoted group pfsync by -1 to 32 (pfsync bulk done) Aug 2 10:00:24 pfw1 /bsd: carp: pfsync0 demoted group carp by -32 to 0 (pfsync init) Aug 2 10:00:24 pfw1 /bsd: carp: pfsync0 demoted group pfsync by -32 to 0 (pfsync init) my pf.conf: set state-policy if-bound set block-policy drop set limit states 40 set limit src-nodes 2 set skip on lo pass quick log proto pfsync keep state (no-sync) pass quick log proto carp pass quick log block quick log I`ve started to commented out line by line from pf.conf and I`ve noticed that pfsync bulk time depends on "limit states" value. So when I changed it to default (1 states) pfsync bulk took 16 seconds. When it is set to 1 it takes 4 minutes to perform pfsync bulk. 1 states: Aug 2 11:30:52 pfw1 /bsd: carp: pfsync0 demoted group carp by 32 to 161 (pfsync init) Aug 2 11:30:52 pfw1 /bsd: carp: pfsync0 demoted group pfsync by 32 to 32 (pfsync init) Aug 2 11:30:52 pfw1 /bsd: carp: pfsync0 demoted group carp by 1 to 162 (pfsync bulk start) Aug 2 11:30:52 pfw1 /bsd: carp: pfsync0 demoted group pfsync by 1 to 33 (pfsync bulk start) Aug 2 11:30:51 pfw1 savecore: no core dump Aug 2 11:30:53 pfw1 /bsd: carp10: state transition: INIT -> BACKUP Aug 2 11:30:53 pfw1 /bsd: carp: carp10 demoted group carp by -1 to 161 (carpdev) Aug 2 11:31:08 pfw1 /bsd: carp10: state transition: BACKUP -> MASTER Aug 2 11:31:08 pfw1 /bsd: carp10: state transition: MASTER -> BACKUP Aug 2 11:31:14 pfw1 /bsd: carp: pfsync0 demoted group carp by -1 to 32 (pfsync bulk done) Aug 2 11:31:14 pfw1 /bsd: carp: pfsync0 demoted group pfsync by -1 to 32 (pfsync bulk done) Aug 2 11:31:14 pfw1 /bsd: carp: pfsync0 demoted group carp by -32 to 0 (pfsync init) Aug 2 11:31:14 pfw1 /bsd: carp: pfsync0 demoted group pfsync by -32 to 0 (pfsync init) 10 states: Aug 2 11:35:38 pfw1 /bsd: carp: pfsync0 demoted group carp by 32 to 161 (pfsync init) Aug 2 11:35:38 pfw1 /bsd: carp: pfsync0 demoted group pfsync by 32 to 32 (pfsync init) Aug 2 11:35:38 pfw1 /bsd: carp: pfsync0 demoted group carp by 1 to 162 (pfsync bulk start) Aug 2 11:35:38 pfw1 /bsd: carp: pfsync0 demoted group pfsync by 1 to 33 (pfsync bulk start) Aug 2 11:35:37 pfw1 savecore: no core dump Aug 2 11:35:39 pfw1 /bsd: carp10: state transition: INIT -> BACKUP Aug 2 11:35:39 pfw1 /bsd: carp: carp10 demoted group carp by -1 to 161 (carpdev) Aug 2 11:39:00 pfw1 /bsd: carp: pfsync0 demoted group carp by -1 to 32 (pfsync bulk done) Aug 2 11:39:00 pfw1 /bsd: carp: pfsync0 demoted group pfsync by -1 to 32 (pfsync bulk done) Aug 2 11:39:00 pfw1 /bsd: carp: pfsync0 demoted group carp by -32 to 0 (pfsync init) Aug 2 11:39:00 pfw1 /bsd: carp: pfsync0 demoted group pfsync by -32 to 0 (pfsync init) Why is that happening? My testing environment: Machine: 2x Sun Fire X2200 M2 with Quad Core Processor Network cables: machine1 nfe0 - crossover cabel - nfe0 machine2 (pfsync) machine1 nfe1 - crossover cabel - nfe1 machine2 (carp) machine1 bge0 - my lan connection, cable to switch machine2 bge0 - my lan connection, cable to switch (same switch that the one line above. Network configuration Machine1 /etc/hostname.nfe0 inet 172.31.255.9 255.255.255.252 NONE description "LAN" -inet6 up /etc/hostname.nfe1 up description WAN -inet6 /etc/hostname.carp10 carpdev nfe1 vhid 3 pass sosecret advskew 0 advbase 5 -inet6 inet 10.10.5.1 255.255.255.0 /etc/hostname.pfsync0 up syncdev nfe0 /etc/hostname.bge0 inet 192.168.50.58 255.255.255.0 NONE description "LAN WAW" -inet6 up !route add -inet
