Re: pf shape download
Hi,
I already write that I wanted to do dynamic shaping.
Here my test rule output from pftop, system is 4.9 :
0 Pass In Q lo0 K00
0 inet6 from any to ::1/128 flags S/SA
1 Pass In Q lo0 K00
0 inet6 from any to fe80::1/128 flags S/SA
2 Pass Out Q lo0 K00
0 inet6 from any to ::1/128 flags S/SA
3 Pass Out Q lo0 K00
0 inet6 from any to fe80::1/128 flags S/SA
4 Pass In Q lo0 K00
0 inet from any to 127.0.0.1/32 flags S/SA
5 Pass Out Q lo0 K00
0 inet from any to 127.0.0.1/32 flags S/SA
6 Pass Out Q K00
0 from to any flags S/SA
7 Pass Out Q K00
0 inet from 192.168.3.0/24 to any flags S/SA
8 Pass In Q K3 234
3 from to any flags S/SA
9 Pass In Q K 9311132
49 inet from 192.168.3.0/24 to any flags S/SA
10 Pass In Q ext_if udp K00
0 inet from 10.5.16.255/32 port = 698 to any
11 Pass In K 3114 287664
35 all flags S/SA
12 Pass OutK 2790 234360
9 all flags S/SA
13 Pass Out em0 K 13539103
21 from to any flags S/SA queue second
Now, will downloading, if I add a address to
with pfctl, it won't shape it until I stop de download and
restart it. After adding the address to the table, I clear
state for this address
pfctl -t second -T add 10.254.200.2
pfctl -k 10.254.200.2
Even try
pfctl -F all -f /etc/pf.conf
without result, it will stay on default
Once I stop and restart my download, it will pass
trough second but that not what I need, I wanted to
shape automaticly those who take too much
bandwith.
When restarted my download and I pass trough second,
if I delete the address from the tab and clear the state
again, it will change to the default queue.
pfctl -t second -T del 10.254.200.2
pfctl -k 10.254.200.2
But if I try to shape 10.254.200.2 again by adding it to
second tab, I must restart my download again.
Is it normal or a behaviure ?
Le 2011-09-07 17:25, Michel Blais a icrit :
Hi all,
thanks for your help and tips.
I have do some testing when I add some free time.
I finally got it working by creating the queue on my internal
if (now em1 instead of re1)
altq on $int_if hfsc bandwidth 97Mb qlimit 500 queue { main, second }
queue main on $int_if bandwidth 1Mb qlimit 250 priority 4
hfsc(upperlimit 97Mb default)
queue second on $int_if bandwidth 1Mb qlimit 250 priority 0
hfsc(upperlimit 1Mb)
and using the following rules
pass out on $ext_if from $my_ip queue depri
2 things I don't understand :
1 - pass out on external if = traffic going out on WAN
this should be upload then
download should be pass in on external if or
out on internal if, right ?
Why must I use a rule on upload to shape download ?
Also, on the bob exemple of queue faq :
http://openbsd.org/faq/pf/queueing.html
Queue is on external if (just like me) but rules
pass out on dc0 to $bob queue bob_in
is also out on internal if.
Why must I do my out rules on external if ?
Is the FAQ wrong ?
2 - I can't use match to transfert traffic in a queue ?
If I use
match out on $ext_if from $my_ip queue depri
instead of
pass out on $ext_if from $my_ip queue depri
then it doesn't work anymore
From what I understand from match rule, it should always
be apply like a quick rule without altering pass or block rule.
Am I wrong ? That would be perfect for my queue rules
because queue will change dynamically. I know I could do
my shaping rules without match but I will have more
exception to take care of.
Also, I can see on a mail from william.dun...@gmail.com
subject : Re: match queue ignored
After further experimentation, I found out the following:
"match queue" overrides:
- a previous "match queue" assignment
- the default queue
Was it add on 5.0 ?
I'm using 4.9
My rule set that work fine from pftop output
after a speed test :
RULE ACTION DIR LOG Q IF PRK PKTSBYTES
STATES MAX INFO
0 Pass In Q lo0 K00
0 inet6 from any to ::1/128 flags S/SA
1 Pass In Q lo0 K00
0 inet6 from any to fe80::1/128 flags S/SA
2 Pass Out Q lo0 K00
0 inet6 from any to ::1/128 flags S/SA
3 Pass Out Q lo0 K00
0 inet6 from any to fe80::1/128 flags S/SA
4 Pass
Re: pf shape download
Hi all,
thanks for your help and tips.
I have do some testing when I add some free time.
I finally got it working by creating the queue on my internal
if (now em1 instead of re1)
altq on $int_if hfsc bandwidth 97Mb qlimit 500 queue { main, second }
queue main on $int_if bandwidth 1Mb qlimit 250 priority 4
hfsc(upperlimit 97Mb default)
queue second on $int_if bandwidth 1Mb qlimit 250 priority 0
hfsc(upperlimit 1Mb)
and using the following rules
pass out on $ext_if from $my_ip queue depri
2 things I don't understand :
1 - pass out on external if = traffic going out on WAN
this should be upload then
download should be pass in on external if or
out on internal if, right ?
Why must I use a rule on upload to shape download ?
Also, on the bob exemple of queue faq :
http://openbsd.org/faq/pf/queueing.html
Queue is on external if (just like me) but rules
pass out on dc0 to $bob queue bob_in
is also out on internal if.
Why must I do my out rules on external if ?
Is the FAQ wrong ?
2 - I can't use match to transfert traffic in a queue ?
If I use
match out on $ext_if from $my_ip queue depri
instead of
pass out on $ext_if from $my_ip queue depri
then it doesn't work anymore
From what I understand from match rule, it should always
be apply like a quick rule without altering pass or block rule.
Am I wrong ? That would be perfect for my queue rules
because queue will change dynamically. I know I could do
my shaping rules without match but I will have more
exception to take care of.
Also, I can see on a mail from william.dun...@gmail.com
subject : Re: match queue ignored
After further experimentation, I found out the following:
"match queue" overrides:
- a previous "match queue" assignment
- the default queue
Was it add on 5.0 ?
I'm using 4.9
My rule set that work fine from pftop output
after a speed test :
RULE ACTION DIR LOG Q IF PRK PKTSBYTES STATES
MAX INFO
0 Pass In Q lo0 K00
0 inet6 from any to ::1/128 flags S/SA
1 Pass In Q lo0 K00
0 inet6 from any to fe80::1/128 flags S/SA
2 Pass Out Q lo0 K00
0 inet6 from any to ::1/128 flags S/SA
3 Pass Out Q lo0 K00
0 inet6 from any to fe80::1/128 flags S/SA
4 Pass In Q lo0 K00
0 inet from any to 127.0.0.1/32 flags S/SA
5 Pass Out Q lo0 K00
0 inet from any to 127.0.0.1/32 flags S/SA
6 Pass Out Q K00
0 from to any flags S/SA
7 Pass Out Q K00
0 inet from 192.168.3.0/24 to any flags S/SA
8 Pass In Q K3 571
2 from to any flags S/SA
9 Pass In Q K6 354
6 inet from 192.168.3.0/24 to any flags S/SA
10 Pass Inext_if K00
0 all flags S/SA
11 Pass Out ext_if K00
0 all flags S/SA
12 Pass Inint_if K00
0 all flags S/SA
13 Pass Out int_if K00
0 all flags S/SA
14 Pass Out em0 K 7070 6572775
24 inet from 10.254.200.2/32 to any flags S/SA queue depri
PFtop -v rules output after a other speed test
RULE ACTION DIR LOG Q IF PRK PKTSBYTES STATES
MAX INFO
0 Pass In Q lo0 K00
0 inet6 from any to ::1/128 flags S/SA
1 Pass In Q lo0 K00
0 inet6 from any to fe80::1/128 flags S/SA
2 Pass Out Q lo0 K00
0 inet6 from any to ::1/128 flags S/SA
3 Pass Out Q lo0 K00
0 inet6 from any to fe80::1/128 flags S/SA
4 Pass In Q lo0 K00
0 inet from any to 127.0.0.1/32 flags S/SA
5 Pass Out Q lo0 K00
0 inet from any to 127.0.0.1/32 flags S/SA
6 Pass Out Q K 20 1120
0 from to any flags S/SA
7 Pass Out Q K00
0 inet from 192.168.3.0/24 to any flags S/SA
8 Pass In Q K1 78
1 from to any flags S/SA
9 Pass In Q K4 384
3 inet from 192.168.3.0/24 to any flags S/SA
10 Pass Inext_if K00
0 all flags S/
Re: pf shape download
Hi David (and thank to all the others for you reply), I didn't have time to work on it but will have some time this week. I think my main problem was from my Windows 7 laptop that look like to block traffic until it "understand" that traffic can pass. A exemple we often see with Windows 7 is when the are a internet problem, when the problem is fix, you can ping external network and domain name but until Windows 7 remove the yellow triangle on the network adapter, browser (we are using IE, Firefox & Chrome) won't be able to go on the internet. Even with our old firewall (base on iptables), it was the same. It taked some minutes for Windows user to apply rule change. Exemple, I forward the port 80 to a server with a alert to contact us, the contact us then we erase the rule and apply it but the user will still be forwarded for several minutes. So will doing my test, if I apply a ruleset like, "pass in" instead of "block in", often, traffic was still block. It's really hard to test ruleset in this condition. For sure, I was able one time to make the traffic pass on my second queue but it was after a long time working on something else. When I came back to it and looked at pftop, I saw the traffic on the second queue. If I remember well, my ruleset was block in block out pass in on re0 to 10.254.200.2 queue second pass out on re0 to 10.254.200.2 queue second pass in on re1 to 10.254.200.2 queue second pass out on re1 to 10.254.200.2 queue second pass in on re0 from 10.254.200.2 queue second pass out on re0 from 10.254.200.2 queue second pass in on re1 from 10.254.200.2 queue second pass out on re1 from 10.254.200.2 queue second I will do more test and write back to the mailling list. Michel Le 2011-08-22 18:40, David Newman a icrit : Did you have any luck getting this working? Thanks! dn On 8/16/11 8:20 AM, Michel Blais wrote: Hi, I'm having a problem to shape download with PF. I have 2 HFSC queue (main and second) created on my internal NIC. Main is my default queue. If I try to match download traffic to the second queue, it still go trought the main queue. The IP I want to download trought the second queue for my test unit is 10.254.200.2 $ext_if=re0 $int_if=re1 My rule to foward traffic to second queue is : match out on $int_if from any to 10.254.200.2 I also try with pass instead of match Look fine if I check the bob exemple in this faq : http://www.openbsd.org/faq/pf/queueing.html#example1 pfctl -vvsq still show traffic on main queue : queue main on re1 bandwidth 1Mb priority 2 qlimit 100 hfsc( red default upperlimit 97Mb ) [ pkts: 24701 bytes: 37333295 dropped pkts: 0 bytes: 0 ] [ qlength: 0/100 ] [ measured: 236.4 packets/s, 2.86Mb/s ] queue second on re1 bandwidth 1Mb priority 0 qlimit 250 hfsc( red upperlimit 97Mb ) [ pkts: 0 bytes: 0 dropped pkts: 0 bytes: 0 ] [ qlength: 0/250 ] [ measured: 0.0 packets/s, 0 b/s ] pftop -v rules show me that the rule don't match 12 Pass out re1 K 0 0 0 inet from any to 10.254.200.2/32flags S/SA queue second I can see my download with tcpdump : # tcpdump -i re1 host 10.254.200.2 ... 10:49:19.802505 10.254.200.2.49266> hammurabi.acc.umu.se.www: . ack 832200 win 64240 (DF) 10:49:19.802716 hammurabi.acc.umu.se.www> 10.254.200.2.49266: . 832200:833660(1460) ack 1 win 6564 (DF) 10:49:19.802911 hammurabi.acc.umu.se.www> 10.254.200.2.49266: . 833660:835120(1460) ack 1 win 6564 (DF) 10:49:19.803040 hammurabi.acc.umu.se.www> 10.254.200.2.49266: . 835120:836580(1460) ack 1 win 6564 (DF) 10:49:19.803211 10.254.200.2.49266> hammurabi.acc.umu.se.www: . ack 836580 win 64240 (DF) 10:49:19.803248 hammurabi.acc.umu.se.www> 10.254.200.2.49266: . 836580:838040(1460) ack 1 win 6564 (DF) 10:49:19.803252 hammurabi.acc.umu.se.www> 10.254.200.2.49266: . 838040:839500(1460) ack 1 win 6564 (DF) 10:49:19.803367 hammurabi.acc.umu.se.www> 10.254.200.2.49266: . 839500:840960(1460) ack 1 win 6564 (DF) ... I have pass days on this with OpenBSD 4.9 and FreeBSD 8.2 without result. I even tryed every 8 possible rules at the same time and pfctl was still showing traffic trought the main queue on : match in on re0 from any to 10.254.200.2 queue second match in on re1 from any to 10.254.200.2 queue second match out on re0 from any to 10.254.200.2 queue second match out on re0 from any to 10.254.200.2 queue second match in on re0 from 10.254.200.2 to any queue second match in on re1 from 10.254.200.2 to any queue second match out on re0 from 10.254.200.2 to any queue second match out on re0 from 10.254.200.2 to any queue second in this case, pftop was showing that it match out on re0 from 10.254.200.2 to any match on re1 from 10.254.200.2 to any it look like only upload rule match Can somebody help me on this ? Thanks Michel P.S : I have a VoIP queue that I will add after that will need the realtime option, that why I'm using HFSC. -- Michel Blais Administrate
Re: pf shape download
On Tue, 16 Aug 2011 11:20:45 -0400 Michel Blais wrote > Hi, > > I'm having a problem to shape download with PF. I have 2 HFSC queue > (main and second) created on my internal NIC. Main is my default > queue. If I try to match download traffic to the second queue, it still > go trought the main queue. > > The IP I want to download trought the second queue for my test > unit is 10.254.200.2 > $ext_if=re0 > $int_if=re1 > > My rule to foward traffic to second queue is : > match out on $int_if from any to 10.254.200.2 > I also try with pass instead of match > > Look fine if I check the bob exemple in this faq : > http://www.openbsd.org/faq/pf/queueing.html#example1 > > pfctl -vvsq still show traffic on main queue : > > queue main on re1 bandwidth 1Mb priority 2 qlimit 100 hfsc( red default > upperlimit 97Mb ) >[ pkts: 24701 bytes: 37333295 dropped pkts: 0 > bytes: 0 ] >[ qlength: 0/100 ] >[ measured: 236.4 packets/s, 2.86Mb/s ] > queue second on re1 bandwidth 1Mb priority 0 qlimit 250 hfsc( red > upperlimit 97Mb ) >[ pkts: 0 bytes: 0 dropped pkts: 0 > bytes: 0 ] >[ qlength: 0/250 ] >[ measured: 0.0 packets/s, 0 b/s ] > > pftop -v rules show me that the rule don't match > 12 Pass out re1 K 0 0 0 inet from any to 10.254.200.2/32flags > S/SA queue second > > I can see my download with tcpdump : > # tcpdump -i re1 host 10.254.200.2 > ... > 10:49:19.802505 10.254.200.2.49266 > hammurabi.acc.umu.se.www: . ack > 832200 win 64240 (DF) > 10:49:19.802716 hammurabi.acc.umu.se.www > 10.254.200.2.49266: . > 832200:833660(1460) ack 1 win 6564 (DF) > 10:49:19.802911 hammurabi.acc.umu.se.www > 10.254.200.2.49266: . > 833660:835120(1460) ack 1 win 6564 (DF) > 10:49:19.803040 hammurabi.acc.umu.se.www > 10.254.200.2.49266: . > 835120:836580(1460) ack 1 win 6564 (DF) > 10:49:19.803211 10.254.200.2.49266 > hammurabi.acc.umu.se.www: . ack > 836580 win 64240 (DF) > 10:49:19.803248 hammurabi.acc.umu.se.www > 10.254.200.2.49266: . > 836580:838040(1460) ack 1 win 6564 (DF) > 10:49:19.803252 hammurabi.acc.umu.se.www > 10.254.200.2.49266: . > 838040:839500(1460) ack 1 win 6564 (DF) > 10:49:19.803367 hammurabi.acc.umu.se.www > 10.254.200.2.49266: . > 839500:840960(1460) ack 1 win 6564 (DF) > ... > > I have pass days on this with OpenBSD 4.9 and > FreeBSD 8.2 without result. > > I even tryed every 8 possible rules at the same time and > pfctl was still showing traffic trought the main queue on : > > match in on re0 from any to 10.254.200.2 queue second > match in on re1 from any to 10.254.200.2 queue second > match out on re0 from any to 10.254.200.2 queue second > match out on re0 from any to 10.254.200.2 queue second > match in on re0 from 10.254.200.2 to any queue second > match in on re1 from 10.254.200.2 to any queue second > match out on re0 from 10.254.200.2 to any queue second > match out on re0 from 10.254.200.2 to any queue second > > in this case, pftop was showing that it > match out on re0 from 10.254.200.2 to any > match on re1 from 10.254.200.2 to any > it look like only upload rule match > > Can somebody help me on this ? > > Thanks > > Michel > > P.S : I have a VoIP queue that I will add after that will need the > realtime option, that why I'm using HFSC. I don't know what is your complete ruleset. with "match(pass) out on $int_if from any to 10.254.200.2" pf will match your main (default) queue. Try this pass in quick on $int_if from 10.254.200.2 to any queue second pass in quick on $int_if from to any queue main
Re: pf shape download
It would be easier to look for what's wrong if you include the whole ruleset On 2011-08-16, Michel Blais wrote: > > I'm having a problem to shape download with PF. I have 2 HFSC queue > (main and second) created on my internal NIC. Main is my default > queue. If I try to match download traffic to the second queue, it still > go trought the main queue. > > The IP I want to download trought the second queue for my test > unit is 10.254.200.2 > $ext_if=re0 > $int_if=re1 > > My rule to foward traffic to second queue is : > match out on $int_if from any to 10.254.200.2 > I also try with pass instead of match > > Look fine if I check the bob exemple in this faq : > http://www.openbsd.org/faq/pf/queueing.html#example1 > > pfctl -vvsq still show traffic on main queue : > > queue main on re1 bandwidth 1Mb priority 2 qlimit 100 hfsc( red default > upperlimit 97Mb ) >[ pkts: 24701 bytes: 37333295 dropped pkts: 0 > bytes: 0 ] >[ qlength: 0/100 ] >[ measured: 236.4 packets/s, 2.86Mb/s ] > queue second on re1 bandwidth 1Mb priority 0 qlimit 250 hfsc( red > upperlimit 97Mb ) >[ pkts: 0 bytes: 0 dropped pkts: 0 > bytes: 0 ] >[ qlength: 0/250 ] >[ measured: 0.0 packets/s, 0 b/s ] > > pftop -v rules show me that the rule don't match > 12 Pass out re1 K 0 0 0 inet from any to 10.254.200.2/32flags > S/SA queue second > > I can see my download with tcpdump : > # tcpdump -i re1 host 10.254.200.2 > ... > 10:49:19.802505 10.254.200.2.49266 > hammurabi.acc.umu.se.www: . ack > 832200 win 64240 (DF) > 10:49:19.802716 hammurabi.acc.umu.se.www > 10.254.200.2.49266: . > 832200:833660(1460) ack 1 win 6564 (DF) > 10:49:19.802911 hammurabi.acc.umu.se.www > 10.254.200.2.49266: . > 833660:835120(1460) ack 1 win 6564 (DF) > 10:49:19.803040 hammurabi.acc.umu.se.www > 10.254.200.2.49266: . > 835120:836580(1460) ack 1 win 6564 (DF) > 10:49:19.803211 10.254.200.2.49266 > hammurabi.acc.umu.se.www: . ack > 836580 win 64240 (DF) > 10:49:19.803248 hammurabi.acc.umu.se.www > 10.254.200.2.49266: . > 836580:838040(1460) ack 1 win 6564 (DF) > 10:49:19.803252 hammurabi.acc.umu.se.www > 10.254.200.2.49266: . > 838040:839500(1460) ack 1 win 6564 (DF) > 10:49:19.803367 hammurabi.acc.umu.se.www > 10.254.200.2.49266: . > 839500:840960(1460) ack 1 win 6564 (DF) > ... > > I have pass days on this with OpenBSD 4.9 and > FreeBSD 8.2 without result. > > I even tryed every 8 possible rules at the same time and > pfctl was still showing traffic trought the main queue on : > > match in on re0 from any to 10.254.200.2 queue second > match in on re1 from any to 10.254.200.2 queue second > match out on re0 from any to 10.254.200.2 queue second > match out on re0 from any to 10.254.200.2 queue second > match in on re0 from 10.254.200.2 to any queue second > match in on re1 from 10.254.200.2 to any queue second > match out on re0 from 10.254.200.2 to any queue second > match out on re0 from 10.254.200.2 to any queue second > > in this case, pftop was showing that it > match out on re0 from 10.254.200.2 to any > match on re1 from 10.254.200.2 to any > it look like only upload rule match > > Can somebody help me on this ? > > Thanks > > Michel > > P.S : I have a VoIP queue that I will add after that will need the > realtime option, that why I'm using HFSC.
pf shape download
Hi, I'm having a problem to shape download with PF. I have 2 HFSC queue (main and second) created on my internal NIC. Main is my default queue. If I try to match download traffic to the second queue, it still go trought the main queue. The IP I want to download trought the second queue for my test unit is 10.254.200.2 $ext_if=re0 $int_if=re1 My rule to foward traffic to second queue is : match out on $int_if from any to 10.254.200.2 I also try with pass instead of match Look fine if I check the bob exemple in this faq : http://www.openbsd.org/faq/pf/queueing.html#example1 pfctl -vvsq still show traffic on main queue : queue main on re1 bandwidth 1Mb priority 2 qlimit 100 hfsc( red default upperlimit 97Mb ) [ pkts: 24701 bytes: 37333295 dropped pkts: 0 bytes: 0 ] [ qlength: 0/100 ] [ measured: 236.4 packets/s, 2.86Mb/s ] queue second on re1 bandwidth 1Mb priority 0 qlimit 250 hfsc( red upperlimit 97Mb ) [ pkts: 0 bytes: 0 dropped pkts: 0 bytes: 0 ] [ qlength: 0/250 ] [ measured: 0.0 packets/s, 0 b/s ] pftop -v rules show me that the rule don't match 12 Pass out re1 K 0 0 0 inet from any to 10.254.200.2/32flags S/SA queue second I can see my download with tcpdump : # tcpdump -i re1 host 10.254.200.2 ... 10:49:19.802505 10.254.200.2.49266 > hammurabi.acc.umu.se.www: . ack 832200 win 64240 (DF) 10:49:19.802716 hammurabi.acc.umu.se.www > 10.254.200.2.49266: . 832200:833660(1460) ack 1 win 6564 (DF) 10:49:19.802911 hammurabi.acc.umu.se.www > 10.254.200.2.49266: . 833660:835120(1460) ack 1 win 6564 (DF) 10:49:19.803040 hammurabi.acc.umu.se.www > 10.254.200.2.49266: . 835120:836580(1460) ack 1 win 6564 (DF) 10:49:19.803211 10.254.200.2.49266 > hammurabi.acc.umu.se.www: . ack 836580 win 64240 (DF) 10:49:19.803248 hammurabi.acc.umu.se.www > 10.254.200.2.49266: . 836580:838040(1460) ack 1 win 6564 (DF) 10:49:19.803252 hammurabi.acc.umu.se.www > 10.254.200.2.49266: . 838040:839500(1460) ack 1 win 6564 (DF) 10:49:19.803367 hammurabi.acc.umu.se.www > 10.254.200.2.49266: . 839500:840960(1460) ack 1 win 6564 (DF) ... I have pass days on this with OpenBSD 4.9 and FreeBSD 8.2 without result. I even tryed every 8 possible rules at the same time and pfctl was still showing traffic trought the main queue on : match in on re0 from any to 10.254.200.2 queue second match in on re1 from any to 10.254.200.2 queue second match out on re0 from any to 10.254.200.2 queue second match out on re0 from any to 10.254.200.2 queue second match in on re0 from 10.254.200.2 to any queue second match in on re1 from 10.254.200.2 to any queue second match out on re0 from 10.254.200.2 to any queue second match out on re0 from 10.254.200.2 to any queue second in this case, pftop was showing that it match out on re0 from 10.254.200.2 to any match on re1 from 10.254.200.2 to any it look like only upload rule match Can somebody help me on this ? Thanks Michel P.S : I have a VoIP queue that I will add after that will need the realtime option, that why I'm using HFSC.
