Re: pf sync
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Fri, 19 Nov 2004 13:20:11 -0800 Sean <[EMAIL PROTECTED]> wrote: > Linux had been around for quite awhile in 1998. I remember using > trying in vain to get my Diamond graphics adaptec to work with X in > Redhat 5.2... Never used RH, I was using SuSE 5, and wordperfect back then, now http://linux.corel.com doesnt exist so I use oo.org. - -- Ed. Debian 3. OpenBSD 3.5. Two things came out of berkeley: BSD and LSD. Don't think this a coincidence. Can't cross chasm in small jumps PGP KeyID 04EDACDA A0F3 44E9 C367 C6C1 C891 4C71 69AF 3CF5 04ED ACDA -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.5 (GNU/Linux) iD8DBQFBnni4aa889QTtrNoRAixRAJ9t3OrxqTwFj6olFtDkwhhjGKQ3kACbBZ4/ rW8k8JXfVVWhNg8+MWI1IM0= =NkCY -END PGP SIGNATURE-
Re: problem load balancing incoming connections
On Fri, Nov 19, 2004 at 06:14:59PM -0300, Emilio Lucena wrote: > This has done the trick. At least as far as incoming connections to the > internal webserver are concerned. However, there remains a problem. > Traffic from the Internet to the firewall itself is still not working as > desired. Somehow, the default route is still being used for outgoing > packets in this case. For example, if I ping the IP of the second > interface (not the one with the default route), the echo reply packet is > sent out through the default interface. Even if I use a reply-to rule > like: > > pass in log-all on $ext_if2 reply-to ($ext_if2 $gw_if2) \ > net proto icmp all icmp-type $icmp_types keep state > > the result is not the expected. See the output of tcpdump: > > Nov 19 17:12:01.557537 rule 11/0(match): pass in on rl0: 201.9.164.54 > > 200.177.74.138: icmp: echo request > Nov 19 17:12:01.557633 rule 11/0(match): pass out on tun0: 200.177.74.138 > > 201.9.164.54: icmp: echo reply > Nov 19 17:12:01.557679 rule 11/0(match): pass out on rl0: 200.177.74.138 > > 201.9.164.54: icmp: echo reply > > Yes ... the packet is duplicated on the other interface, and in fact it > gets lost or discarded, because the pinging host reports packet loss. The tcpdump on pflog has a simple explanation. Remember how pf first sees the reply trying to go out on tun0, where it matches the reply-to state, gets routed out rl0, matches the same state again and gets passed out on rl0? The first filter pass, on tun0, sends the packet to pflog due to your 'log-all' option. This logging happens before the reply-to option is processed, and isn't suppressed just because the packet later gets routed by pf. That doesn't mean the echo reply was sent out on tun0. pflog doesn't really tell you what packets were sent out on an interface, it tells you whether a packet passed the state table or ruleset on an interface. Repeat the test and also run tcpdump on rl0 and tun0 themselves. This tells you what packets are really sent out on these interfaces. The rule looks fine, the pflog tcpdump confirms the reply-to state is working. Maybe the packet loss has another explanation, things should work like this. > 1. Fault-tolerance. A very simple one. Something that avoided the packets > going to the wrong interface (when it goes down) would be sufficient. Do > you think "ifstated" could be used for that purpose (I am thinking about > using some anchors to have the ruleset updated when something bad > happens)? Yes, that will work for new outgoing connections from you. When both links are up, you round-robin connection out through both. When one goes down, you remove the address from the table, and the round-robin rule will use only the working address. Incoming connections will simply cease to arrive on the non-working interface, so there's nothing to handle there. Ongoing connections on the interface that just died will stall and have to be re-established, there is no way to change your address during a connection, obviously. > Is this still the recommended setup for 3.5 and 3.6? If so, how could one > add queue information for 2 interfaces with different bandwidth limits? I guess you can use 'route-to' on the 'pass in on $int_if' rules, so things better ressemble the reply-to rules. I don't remember why I recommended re-routing from the default route interface, to be honest ;) Packets will get refiltered on the external interface, you can use two separate rules to pass outgoing connections there using separate queues. You'll have two 'altq on $ext_if1/2' rules, and can setup separate queues on each interface. Shouldn't be a problem. > Also IP1 and IP2 will probably be from the same network. Do you see this > as potential problem (as far as PF is concerned)? I can't see any problems there. Daniel
Re: problem load balancing incoming connections
Daniel,
First of all, thanks a lot for your reply and useful suggestions.
On Fri, 19 Nov 2004, Daniel Hartmeier wrote:
> 2) If the rule contains a reply-to option, the packet is routed out
> the specified interface.
>
> New (since 3.5) is the little detail that the packet, now
> attempting to pass out the 'right' interface, is again filtered
> by pf! This was necessary for certain setups so the packet gets
> a chance to create a new state there. In your specific case, it
> isn't necessary, but harmful.
It may well be needed for my final setup. The ruleset I attached was a
test one, based on the PF guide example in the "Address Polls and Load
Balancing" section, with the addition of some rules suggested in your
message.
> > pass out on $ext_if1 route-to ($ext_if2 $gw_if2) from $ext_if2 to any
> > pass out on $ext_if2 route-to ($ext_if1 $gw_if1) from $ext_if1 to any
>
> I don't understand the purpose of these rules, BTW. The rules
> contradict the comment (did you intentionally swap $ext_if 1 and
> to in the route-to arguments?).
This comes from the PF guide example as well. I guess it is intended to
re-route packets that go out through the wrong interface.
>pass in on $ext_if1 from any to $intweb \
> tag from_ext_if1 keep state
>pass in on $ext_if2 from any to $intweb \
> tag from_ext_if2 keep state
>
> d) use 'reply-to' on dedicated 'pass out on $int_if keep state', based
> on the tags added, like
>
>pass out on $int_if reply-to ($ext_if1 gw_if1) \
> from any to $intweb tagged from_ext_if1 keep state
>pass out on $int_if reply-to ($ext_if2 gw_if2) \
> from any to $intweb tagged from_ext_if2 keep state
>
This has done the trick. At least as far as incoming connections to the
internal webserver are concerned. However, there remains a problem.
Traffic from the Internet to the firewall itself is still not working as
desired. Somehow, the default route is still being used for outgoing
packets in this case. For example, if I ping the IP of the second
interface (not the one with the default route), the echo reply packet is
sent out through the default interface. Even if I use a reply-to rule
like:
pass in log-all on $ext_if2 reply-to ($ext_if2 $gw_if2) \
net proto icmp all icmp-type $icmp_types keep state
the result is not the expected. See the output of tcpdump:
Nov 19 17:12:01.557537 rule 11/0(match): pass in on rl0: 201.9.164.54 >
200.177.74.138: icmp: echo request
Nov 19 17:12:01.557633 rule 11/0(match): pass out on tun0: 200.177.74.138 >
201.9.164.54: icmp: echo reply
Nov 19 17:12:01.557679 rule 11/0(match): pass out on rl0: 200.177.74.138 >
201.9.164.54: icmp: echo reply
Yes ... the packet is duplicated on the other interface, and in fact it
gets lost or discarded, because the pinging host reports packet loss.
> Make sure you have 'keep state' on _all_ 'pass' rules, and that you have
> 'flags S/SA' on all 'pass proto tcp keep state' rules. Your current
> ruleset violates both of these principles, further complicating things.
>
> Daniel
>
I was really desperate to get anything to work, so I decided to start with
a very simple ruleset. In reality, my final setup will have to approach a
series of other points:
1. Fault-tolerance. A very simple one. Something that avoided the packets
going to the wrong interface (when it goes down) would be sufficient. Do
you think "ifstated" could be used for that purpose (I am thinking about
using some anchors to have the ruleset updated when something bad
happens)?
2. ACK priorization. Again, following the recommendations in your site,
I will need rules with queue specifications. That reminds me of the
second suggestion in your message dated from last year, where you say:
"don't use route-to on internal interface, but let all connections get
routed to the external interface with the default route, and use route-to
THERE"
with a rule like:
pass out on $ext_if1 route-to { $ext_if1, $ext_if2} \
from any to any keep state
Is this still the recommended setup for 3.5 and 3.6? If so, how could one
add queue information for 2 interfaces with different bandwidth limits?
3. Most likely, I will get an IP address for each interface and the same
next-hop from my DSL ISP. So for instance, If I have 2 dsl conncetions, I
will get something like:
IP1 -> IPnh
IP2 -> IPnh
Also IP1 and IP2 will probably be from the same network. Do you see this
as potential problem (as far as PF is concerned)?
Thanks a million for any additional help.
Regards,
ebl
Re: problem load balancing incoming connections
Oops,
I forgot to attach the new ruleset.
ext_if1="rl0"
gw_if1="200.177.74.1"
gw_if2="200.164.195.8"
ext_if2="tun0"
int_if="sis0"
lan_net=$int_if:network
tcp_INservices = "{ 22 }"
icmp_types = "echoreq"
priv_nets = "{ 127.0.0.0/8, 192.168.0.0/16, 172.16.0.0/12, 10.0.0.0/8 }"
intweb = "192.168.1.10"
# logging and pfstat
set loginterface $int_if
# scrub
scrub in all
# nat/rdr
nat on $ext_if1 from $int_if:network to any -> ($ext_if1)
nat on $ext_if2 from $int_if:network to any -> ($ext_if2)
# internal web server
rdr on {$ext_if1,$ext_if2} proto tcp from any to any port 80 -> $intweb
# filter rules
# default deny
block in log from any to any
block out log from any to any
# loopback
pass quick on lo0 all
# Internal services on the LAN
pass in log on $ext_if1 from any to $intweb tag from_ef1 keep state
pass in log on $ext_if2 from any to $intweb tag from_ef2 keep state
# packets for the internal webserver
pass out log on $int_if reply-to ($ext_if1 $gw_if1) \
from any to $intweb tagged from_ef1 keep state
pass out log on $int_if reply-to ($ext_if2 $gw_if2) \
from any to $intweb tagged from_ef2 keep state
# from internal net to external services
pass in log on $int_if from $int_if:network to any keep state
# from firewall to internal network
pass out log on $int_if from $int_if to $int_if:network keep state
# firewall services
pass in log on $ext_if1 inet proto tcp from any to ($ext_if1) \
port $tcp_INservices flags S/SA keep state
pass in log on $ext_if2 inet proto tcp from any to ($ext_if2) \
port $tcp_INservices flags S/SA keep state
# icmp
pass in log-all inet proto icmp all icmp-type $icmp_types keep state label icmp
#pass in log-all on $ext_if1 reply-to ($ext_if1 $gw_if1) inet proto icmp all
icmp-type $icmp_types keep state label icmp-if1
#pass in log-all on $ext_if2 reply-to ($ext_if2 $gw_if2) inet proto icmp all
icmp-type $icmp_types keep state label icmp-if2
# outside services
pass out on $ext_if1 proto tcp all flags S/SA keep state
pass out on $ext_if2 proto tcp all flags S/SA keep state
pass out on $ext_if1 proto {udp icmp} all keep state
pass out on $ext_if2 proto {udp icmp} all keep state
Re: pf sync
Thomas Althoff wrote: OpenBSD is still new (6 months) experiance for me, kind of funny since i started my "i love unix" carrier back in 1998 with Zenix. Many years before Linux, AIX, Tru64 and the gang. Linux had been around for quite awhile in 1998. I remember using trying in vain to get my Diamond graphics adaptec to work with X in Redhat 5.2... Sean
RE: pf sync
> Thanks, Ill check out the how to as the above does not make much sense to the untrained reader, but I am sure it will when I have been through the appropriate materials. Feel free to ask more questions ..i'm listning. OpenBSD is still new (6 months) experiance for me, kind of funny since i started my "i love unix" carrier back in 1998 with Zenix. Many years before Linux, AIX, Tru64 and the gang. When i first was told that OpenBSD with pf/carp is "THE" soloution i was looking for was everything just "block hole" for me. It looks hard in the begining to work with carp and pfsync, it's simple when all the bits and pieces are in place. My first "i love OpenBSD" experiance started when i was reading the diffrent man(8) pages, compared with the other ~10 Unix flavours i have "walk thru" is OpenBSD the first one with good man pages. -Thomas ps. Zenix was not i nice animal to work with, 1998 was still the "IBM compatible" days.
Re: pf sync
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Fri, 19 Nov 2004 01:07:34 +0100 "Thomas Althoff" <[EMAIL PROTECTED]> wrote: > http://www.countersiege.com/doc/pfsync-carp/ > > Mini HOWTO (everything is mentioned on the page above) > > In pf.conf: Allow pfsync traffic > In /etc/sysctl.conf: enable carp > tell pfsync to use a specific NI in /etc/hostname.pfsync > Establish your diffrent carp groups in /etc/hostname.carp[0-255] > > I'm using 3 boxes with 19 diffrent carp interfaces failing over for > various resons, i'm not using the carp round-robin stuff. Thanks, Ill check out the how to as the above does not make much sense to the untrained reader, but I am sure it will when I have been through the appropriate materials. Thanks. - -- Ed. Debian 3. OpenBSD 3.5. Two things came out of berkeley: BSD and LSD. Don't think this a coincidence. Can't cross chasm in small jumps PGP KeyID 04EDACDA A0F3 44E9 C367 C6C1 C891 4C71 69AF 3CF5 04ED ACDA -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.5 (GNU/Linux) iD8DBQFBnkdhaa889QTtrNoRAjvJAJ48ldLHpQHkm+X8FTWzG6scOlFSBwCg8WMm 89cJRoZ7d9ajQOBD+DlM3fg= =W1u7 -END PGP SIGNATURE-
Re: States
Sorry, redirected to pf@ by accident. -J. On Nov 19, 2004, at 6:51 AM, Jason Dixon wrote: On Nov 19, 2004, at 6:32 AM, Sergi Toledo wrote: Hi I've been looking for the maximum number of states that pf is able to handle, but I can't find the correct .c or .h file. Which one is it? I suppose these states are stored in memory. Am I wrong? Thanks in advance Sergio There is no hard limit in the source. They are limited only by your available memory, but can be capped using "set limit states" in pf.conf. The general rule is 1k states per 1MB of memory. -- Jason Dixon DixonGroup Consulting http://www.dixongroup.net -- Jason Dixon DixonGroup Consulting http://www.dixongroup.net
Re: States
On Nov 19, 2004, at 6:32 AM, Sergi Toledo wrote: Hi I've been looking for the maximum number of states that pf is able to handle, but I can't find the correct .c or .h file. Which one is it? I suppose these states are stored in memory. Am I wrong? Thanks in advance Sergio There is no hard limit in the source. They are limited only by your available memory, but can be capped using "set limit states" in pf.conf. The general rule is 1k states per 1MB of memory. -- Jason Dixon DixonGroup Consulting http://www.dixongroup.net
Re: altq question
It wasnt very clear to me that i had to apply the queues for INcoming
connections... and right now yes, this really makes the difference.
On Thu, 18 Nov 2004 22:26:06 +0200, colin <[EMAIL PROTECTED]> wrote:
> First of all I want to say Hi to everybody because this is my first thread.
>
> Im rather new to altq and pf but basically i want to limit the http
> traffic from one of my LANs machines.
> Ive tried all the schedulers but i havent noticed any differences between
> them.
>
> Here is a sample of what ive done using cbq:
>
> altq on $EXT_IF cbq bandwidth 256Kb queue {http, other}
>
> queue http bandwidth 6Kb priority 0 cbq (red)
> queue std bandwidth 90% cbq(default)
>
> pass out on $EXT_IF proto tcp from $LAN_IP to any port = 80 \
>flags S/SA modulate state queue http
>
> After ive applied these rules from LAN_IP i was able to browse web
> pages even with 10.24 Kb/s
>
> pfctl -vvsq
>
> [ measured:4.4 packets/s, 9.48Kb/s ]
> [ measured:4.3 packets/s, 10.24Kb/s ]
>
> If this in not correct please tell me what do i miss and what should i
> do to limit that http traffic?
>
> ps Im using OpenBSD 3.6
>
