Re: traffic shaping on RELENG_10 ?

[Ian Smith]

On Thu, 17 Dec 2015 14:24:34 -0500, Kenneth D. Merry wrote:
 > On Wed, Dec 16, 2015 at 15:20:26 -0500, Mike Tancsa wrote:
 > > I need to do some simple traffic shaping, but whenever I try and use
 > > altq, I dont seem to have any luck-- I mean zero.
 > 
 > It took me a while to get ALTQ working.  More below..
 > 
 > >  I like the management of pf via pf.conf, but miss the simplicity of
 > > dummynet and ipfw.  Has anyone used pf and ipfw together in RELENG10 ?
 > > Any tips / caveats ? Or am I better off re-writing the pf rules into
 > > ipfw and be done with it.
 > 
 > I'm using ipfw for firewall and in-kernel NAT, and ALTQ (configured via pf)
 > for traffic shaping.

I also greatly appreciate your post on this, Ken.

I've been wondering about doing the opposite, after over 15 years using 
ipfw and over 12 years using dummynet, ie trying pf but wanting to use 
dummynet for shaping, as I've always found it advantageous to do shaping 
inbound as well as outbound, which pf doesn't do, like tc in linux where 
I'd had to do some real work getting tc to shape inbound by treating it 
as outbound traffic on the inside interface - which does work fine in a 
rather internet-cafe environment requiring judicious cat-herding :)

Does anyone know if it's possible to use ipfw just for dummynet shaping, 
using pf rules for filtering, in terms of the ordering of pfil(9) hooks?

 > One thing that was non-obvious with the in-kernel NAT versus the userland
 > natd(8) was that you have to set 'sysctl net.inet.ip.fw.one_pass=0'.
 > Otherwise the rules after the NAT step in the ipfw configuration are
 > skipped.

Also needed when using dummynet - or netgraph, or reass; it's rather too
heavily overloaded, and I've always thought its default of 1 to be the 
opposite of what most people want .. way too late to change it though.

 > In-kernel NAT:
 > 
 > ${fwcmd} nat 123 config log if ${oif} same_ports unreg_only
 > 
 > # This is necessary with the in-kernel NAT to re-inject packets once they've
 > # gone through the NAT.
 > sysctl net.inet.ip.fw.one_pass=0

   ${fwcmd} disable one_pass# accomplishes the same thing.

 > ${fwcmd} enable altq

cheers, Ian
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: traffic shaping on RELENG_10 ?

[Kenneth D. Merry]

On Thu, Dec 17, 2015 at 14:36:22 -0500, Mike Tancsa wrote:
> On 12/17/2015 2:24 PM, Kenneth D. Merry wrote:
> > It took me a while to get ALTQ working.  More below..
> 
> Thank you very much for the detail!  Are you running any patches ? I
> seem to recall over the years the pfsense folks with patches to pf to
> get it to play well.  However, I dont know if thats relevant to
> RELENG_10 or not.

No patches, just stable/10.

> I wonder if there is a similar issue with the re driver and altq. I
> tried to get it to work on the old zoo and never once was able to shape
> a packet :(

I'm not sure.  It uses if_start, not if_transmit, so in theory it would
work.

> In my case, I just want to prevent a process from eating up bandwidth by
> accident so dummynet does the trick just fine.  I am going to try this
> new box with em nics and see if I can get altq to work.  It would be
> nice if its in one config file and I dont have to have multiple firewall
> systems.

The em(4) driver does work for ALTQ.  And I'm sure ALTQ will work fine
with pf.

We (and by that I mean someone other than me) should fix things so that
it is easy to use ALTQ with a multi-queue NIC.  i.e. an easy to follow
netgraph recipe that will setup ng_iface on top of your NIC and use ALTQ
on that.  Or another solution would be to modify ALTQ so that it'll work
with NICs that use if_transmit one way or another.

> Just curious, since you had altq working, why did you not want to just
> use pf for your firewall rules and natting ?

Inertia.  I've had the same basic firewall setup for a long time, and
didn't want to figure out the right way to re-write it in pf.

Ken
-- 
Kenneth Merry
k...@freebsd.org
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: traffic shaping on RELENG_10 ?

[Mike Tancsa]

On 12/17/2015 2:24 PM, Kenneth D. Merry wrote:
> It took me a while to get ALTQ working.  More below..

Thank you very much for the detail!  Are you running any patches ? I
seem to recall over the years the pfsense folks with patches to pf to
get it to play well.  However, I dont know if thats relevant to
RELENG_10 or not.
I wonder if there is a similar issue with the re driver and altq. I
tried to get it to work on the old zoo and never once was able to shape
a packet :(
In my case, I just want to prevent a process from eating up bandwidth by
accident so dummynet does the trick just fine.  I am going to try this
new box with em nics and see if I can get altq to work.  It would be
nice if its in one config file and I dont have to have multiple firewall
systems.

Just curious, since you had altq working, why did you not want to just
use pf for your firewall rules and natting ?

---Mike



-- 
---
Mike Tancsa, tel +1 519 651 3400
Sentex Communications, m...@sentex.net
Providing Internet services since 1994 www.sentex.net
Cambridge, Ontario Canada   http://www.tancsa.com/
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: traffic shaping on RELENG_10 ?

[Kenneth D. Merry]

On Wed, Dec 16, 2015 at 15:20:26 -0500, Mike Tancsa wrote:
> I need to do some simple traffic shaping, but whenever I try and use
> altq, I dont seem to have any luck-- I mean zero.

It took me a while to get ALTQ working.  More below..

>  I like the management of pf via pf.conf, but miss the simplicity of
> dummynet and ipfw.  Has anyone used pf and ipfw together in RELENG10 ?
> Any tips / caveats ? Or am I better off re-writing the pf rules into
> ipfw and be done with it.

I'm using ipfw for firewall and in-kernel NAT, and ALTQ (configured via pf)
for traffic shaping.

Step 1 was getting an ethernet card that supported ALTQ.  I tried enabling
IGB_LEGACY_TX in the igb driver, but it led to panics.  I tried configuring
ng_iface to provide the single queue that ALTQ needs, but wasn't able to
figure it out.  It ended up being easier to buy a $22 ethernet card that
uses the em(4) driver.

Hopefully this will help someone in a similar situation reading the list
archives...

This mostly works for me.  I have 10Mb upstream.  One of the keys was
getting outbound DNS traffic to have a real-time priority (it's in the
"other" category below) so that web browsers didn't time out when there was
heavy upstream traffic.  (e.g. uploading gigabytes of a photo library to
my iCloud Photo Library...)

One thing that was non-obvious with the in-kernel NAT versus the userland
natd(8) was that you have to set 'sysctl net.inet.ip.fw.one_pass=0'.
Otherwise the rules after the NAT step in the ipfw configuration are
skipped.

Using the in-kernel NAT was necessary to allow tagging packets from certain
internal hosts so that they could be routed to different ALTQ rules after
the NAT step.  The tags are lost if you use the userland natd.

I use ipfw count rules to tag packets, and then make decisions about
whether to drop them later.

Although the priorities are specified, I believe the hfsc schedule does not
use them.  So pay no attention to that.  And I don't currently have any
hosts tagged in the lpweb (low priority web) category.  I'm not sure that
would be productive in any case without a change in the ALTQ setup to limit
the bandwidth for that queue or something like that.  And, even then, if I
have a large upload going on, I want it to go as fast as possible when it
isn't contending with other traffic.

Here's my pf.conf:

altq on em0 hfsc bandwidth 9600Kb queue { lpweb, mail, web, hphost, ssh, icmp, 
femtocell, other }
queue lpweb bandwidth 20% priority 0 hfsc (upperlimit 99%)
queue mail bandwidth 10% priority 1 hfsc (upperlimit 99%)
queue web bandwidth 20% priority 2 hfsc (upperlimit 99%)
queue hphost bandwidth 10% priority 3 hfsc (upperlimit 90% realtime 20%)
queue other bandwidth 20% priority 4 hfsc (default upperlimit 99% realtime 20%)
queue icmp bandwidth 5% priority 5 hfsc (upperlimit 15%)
queue ssh bandwidth 10% priority 6 hfsc (upperlimit 99% realtime 15%)
queue femtocell bandwidth 5% priority 7 hfsc (upperlimit 50% realtime 20%)

And here are some snippets from my firewall script.  Obviously this isn't
complete:

fwcmd="ipfw"
pfcmd="pfctl"
oif="em0"
iif="igb0"
oip="my_external_ip_address"

Here is where I tag outbound packets from certain hosts.  The tags will
follow the packets through the in-kernel NAT so that I can then put them on
the correct ALTQ on the outside interface:

# Streaming boxes
${fwcmd} add count tag 10 ip from 10.0.0.0/24{30-39} to any via ${iif}

# Mark packets coming from the Verizon Femtocell
${fwcmd} add count tag 30 all from 10.0.0.44 to any via ${iif}

In-kernel NAT:

${fwcmd} nat 123 config log if ${oif} same_ports unreg_only

# This is necessary with the in-kernel NAT to re-inject packets once they've
# gone through the NAT.
sysctl net.inet.ip.fw.one_pass=0

# ALTQ queues have to be setup via pf
${pfcmd} -f $PF_FILE

# Use count rules to tag the traffic in different queues.

# Femtocell and similar super-high priority, but lower bandwidth 
# machines.
${fwcmd} add count altq femtocell ip from any to any tagged 30-39

# Tags 10-19 are high priority machines.
${fwcmd} add count altq hphost ip from any to any tagged 10-19

# ${WORK} VPN server.  ssh connections to work run over the VPN.
${fwcmd} add count altq ssh ip from any to ${WORK_VPN_SERVER} dst-port 443

# Low priority machines.  This would primarily be needed if there
# were a machine taking up a lot of bandwidth.
${fwcmd} add count altq lpweb ip from any to any dst-port 443 tagged 20-29

# IMAP, outbound mail
${fwcmd} add count altq mail ip from ${oip} 25,143,993 to any 
${fwcmd} add count altq mail ip from ${oip} to any dst-port 25

# Normal web access.
${fwcmd} add count altq web ip from any to any dst-port 80,443

# SSH
${fwcmd} add count altq ssh ip from any to any dst-port 22

# ICMP
${fwcmd} add count altq icmp icmp from any to any

# Explicitly add DNS to the other queue.
${fwcmd} add count altq other udp from any to any dst-port 53

# Anything else, especially DNS.  This is the default.
${fwcmd} add count altq other ip from a

Re: traffic shaping on RELENG_10 ?

[Mike Tancsa]

On 12/16/2015 4:27 PM, Karl Denninger wrote:
>> Protect some boxes via pf, and then limit one box from eating up all the
>> bandwidth when doing a backups and syncs.
>> I would like to keep pf for firewall rules, and I would like to use
>> ipfw+DUMMYNET for bandwidth shaping
>>
>>
>>  ---Mike
>>
> I use this...
>ipfw pipe 1 config bw 3000Kbit/s
>ipfw add 1700 pipe 1 tcp from {wherever} to any
> 
> (and so on you can add any sort of flow into the pipe, and it will
> honor that limit)
> 
> Works very well.
> 

Thanks, I have indeed used dummynet with great success. What I am trying
to find out is can I still use pf for all the firewall rules, and then
just use ipfw+dummynet for traffic shaping safely.

---Mike



-- 
---
Mike Tancsa, tel +1 519 651 3400
Sentex Communications, m...@sentex.net
Providing Internet services since 1994 www.sentex.net
Cambridge, Ontario Canada   http://www.tancsa.com/
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: traffic shaping on RELENG_10 ?

[Karl Denninger]

On 12/16/2015 15:02, Mike Tancsa wrote:
> On 12/16/2015 3:22 PM, Karl Denninger wrote:
>> On 12/16/2015 14:20, Mike Tancsa wrote:
>>> I need to do some simple traffic shaping, but whenever I try and use
>>> altq, I dont seem to have any luck-- I mean zero.
>>>
>>>  I like the management of pf via pf.conf, but miss the simplicity of
>>> dummynet and ipfw.  Has anyone used pf and ipfw together in RELENG10 ?
>>> Any tips / caveats ? Or am I better off re-writing the pf rules into
>>> ipfw and be done with it.
>>>
>>> ---Mike
>> What are you trying to do?
>>
>> I use rate-shaping in ipfw without problems and have been for quite some
>> time.
> Protect some boxes via pf, and then limit one box from eating up all the
> bandwidth when doing a backups and syncs.
> I would like to keep pf for firewall rules, and I would like to use
> ipfw+DUMMYNET for bandwidth shaping
>
>
>   ---Mike
>
I use this...
   ipfw pipe 1 config bw 3000Kbit/s
   ipfw add 1700 pipe 1 tcp from {wherever} to any

(and so on you can add any sort of flow into the pipe, and it will
honor that limit)

Works very well.

-- 
Karl Denninger
k...@denninger.net 
/The Market Ticker/
/[S/MIME encrypted email preferred]/


smime.p7s
Description: S/MIME Cryptographic Signature

Re: traffic shaping on RELENG_10 ?

[Mike Tancsa]

On 12/16/2015 3:22 PM, Karl Denninger wrote:
> 
> On 12/16/2015 14:20, Mike Tancsa wrote:
>> I need to do some simple traffic shaping, but whenever I try and use
>> altq, I dont seem to have any luck-- I mean zero.
>>
>>  I like the management of pf via pf.conf, but miss the simplicity of
>> dummynet and ipfw.  Has anyone used pf and ipfw together in RELENG10 ?
>> Any tips / caveats ? Or am I better off re-writing the pf rules into
>> ipfw and be done with it.
>>
>>  ---Mike
> What are you trying to do?
> 
> I use rate-shaping in ipfw without problems and have been for quite some
> time.

Protect some boxes via pf, and then limit one box from eating up all the
bandwidth when doing a backups and syncs.
I would like to keep pf for firewall rules, and I would like to use
ipfw+DUMMYNET for bandwidth shaping


---Mike




-- 
---
Mike Tancsa, tel +1 519 651 3400
Sentex Communications, m...@sentex.net
Providing Internet services since 1994 www.sentex.net
Cambridge, Ontario Canada   http://www.tancsa.com/
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: traffic shaping on RELENG_10 ?

[Karl Denninger]

On 12/16/2015 14:20, Mike Tancsa wrote:
> I need to do some simple traffic shaping, but whenever I try and use
> altq, I dont seem to have any luck-- I mean zero.
>
>  I like the management of pf via pf.conf, but miss the simplicity of
> dummynet and ipfw.  Has anyone used pf and ipfw together in RELENG10 ?
> Any tips / caveats ? Or am I better off re-writing the pf rules into
> ipfw and be done with it.
>
>   ---Mike
What are you trying to do?

I use rate-shaping in ipfw without problems and have been for quite some
time.

-- 
Karl Denninger
k...@denninger.net 
/The Market Ticker/
/[S/MIME encrypted email preferred]/


smime.p7s
Description: S/MIME Cryptographic Signature

Re: Traffic shaping

[Christoph Moench-Tegeder]

## Peter Jeremy ([EMAIL PROTECTED]):

> ipfw/dummynet pipes only handle a single flow direction:  If you
> have something like
>   ipfw NUMBER pipe 1 ip from any to any via ifX
> then both incoming and outgoing traffic share the pipe and you are
> limiting the combined uplink and downlink traffic - which probably
> isn't what you want.

Yes, I see.
My last experience with things like this suffered a little from a
slightly underfunctional implementation of bandwidth control (and
using dummynet/FreeBSD was not an option). I only had control
over the outgoing queue on the external interface, so I tried
slowing down slow start by limiting the acks from teh destination
to the source of the transfer based on calculations with packet
sizes and -rates (which seemed to be more than a little unreliable).
[Perhaps I could construct a scenario with some more interfaces,
where limiting the outgoing rates would be easier, but that would
be too far fetched for now].
So my problem in short: How much do I have to limit acks in the
direction with the lower bandwidth in order to control the
usage of the other direction? This is a little away from the
original thread, but I just would like to know (might come in
handy some time and perhaps I can "ptimize" my old "solution"
a little with other people's experience).

Regards,
cmt

-- 
Spare Space

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-stable" in the body of the message

Re: Traffic shaping

[Peter Jeremy]

On 2002-Oct-06 16:19:08 +0200, Christoph Moench-Tegeder <[EMAIL PROTECTED]> 
wrote:
>## Daniel O'Connor ([EMAIL PROTECTED]):
>
>> Actually, if you limit incoming TCP it will adapt to the correct speed.
>> I do this at home without hassle (except the latency in games goes up
>> from ~40 to ~100 but it is still acceptable)
>
>How much do you have to limit TCP for the desired effect? I never
>tried shaping on asymmetric lines, and the traffic ratio for a
>single TCP bulk transfer (1500 (or little less in case of PPPoE,
>PPTP, etc.) bytes incoming vs. 40 bytes outgoing) does not match
>the up/down-ratio of his line (1:6) by any means.

ipfw/dummynet pipes only handle a single flow direction:  If you
have something like
  ipfw NUMBER pipe 1 ip from any to any via ifX
then both incoming and outgoing traffic share the pipe and you are
limiting the combined uplink and downlink traffic - which probably
isn't what you want.

Instead, you need two pipes with uplink traffic in one and downlink
traffic in the other.  Since the pipes are independent, you can set
the uplink and downlink limits to suit your ADSL link:
  ipfw pipe 1 config bw UPLINK kbps
  ipfw NUMBER pipe 1 ip from any to any out xmit ifX
  ipfw pipe 2 config bw DOWNLINK kbps
  ipfw NUMBER pipe 2 ip from any to any in recv ifX

Peter

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-stable" in the body of the message

Re: Traffic shaping

[Christoph Moench-Tegeder]

## Daniel O'Connor ([EMAIL PROTECTED]):

> Actually, if you limit incoming TCP it will adapt to the correct speed.
> I do this at home without hassle (except the latency in games goes up
> from ~40 to ~100 but it is still acceptable)

How much do you have to limit TCP for the desired effect? I never
tried shaping on asymmetric lines, and the traffic ratio for a
single TCP bulk transfer (1500 (or little less in case of PPPoE,
PPTP, etc.) bytes incoming vs. 40 bytes outgoing) does not match
the up/down-ratio of his line (1:6) by any means.

Regards,
cmt

-- 
Spare Space

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-stable" in the body of the message

Re: Traffic shaping

[Daniel O'Connor]

On Sun, 2002-10-06 at 22:56, Christoph Moench-Tegeder wrote:
> Limit the maximum usage of your line to about 90% to 95%. This prevents
> usage of the queue in your modem. However, this will not work if
> queueing on the remote side is the problem (bulk transfers like downloads
> come to mind); you are out of luck if the queue on the remote side is too
> long.

Actually, if you limit incoming TCP it will adapt to the correct speed.
I do this at home without hassle (except the latency in games goes up
from ~40 to ~100 but it is still acceptable)

-- 
Daniel O'Connor software and network engineer
for Genesis Software - http://www.gsoft.com.au
"The nice thing about standards is that there
are so many of them to choose from."
  -- Andrew Tanenbaum
GPG Fingerprint - 9A8C 569F 685A D928 5140  AE4B 319B 41F4 5D17 FDD5


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-stable" in the body of the message

Re: Traffic shaping

[Christoph Moench-Tegeder]

## Maarten de Vries ([EMAIL PROTECTED]):

> I could do with some hints regarding traffic shaping. My homenetwork is
> hooked up to an assymetrical (1536/256) ADSL line, using a FreeBSD 4.6
> system which provides the usual NAT/Gateway/Firewall services. I'm using
> dummynet to control the amount of bandwidth used by certain hosts on the
> network, which all works fine. However, I still suffer from latency when the
> maximum up- and/or downstream of the line is being utilized. If I understand
> it well, this happens because the buffer in the ADSL modem fills up
> completely at those occasions. Is there anyway to prevent this from
> happening?

Limit the maximum usage of your line to about 90% to 95%. This prevents
usage of the queue in your modem. However, this will not work if
queueing on the remote side is the problem (bulk transfers like downloads
come to mind); you are out of luck if the queue on the remote side is too
long.

Regards,
cmt

-- 
Spare Space

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-stable" in the body of the message

Re: Traffic Shaping?

[Scott M. Nolde]

As much as I hate to toot my own horn, I'll do it.  I've just published a
script for doing WF2Q+ traffic shaping at http://bsdvault.net.  This
script may be overkill for what you need, but it could help you in other
ways and serve as an example of how to do implement traffic shaping.

It's worth a read and i'm open to comments.  Please post comments at
bsdvault's site.

- Scott

Lasse Laursen([EMAIL PROTECTED])@2002.09.16 21:02:14 +:
> Hi all,
> 
> Setup summary:
> 
> We have a server that runs some free hosting - we would like to limit the
> max bandwidth usage to 40 MBit.
> 
> We have one network card in the machine and we would like to put a limit on
> the usage of bandwidth. I have had a look at mod_throttle but according to
> the documentation it simply denies connections to the webserver (Apache)
> once the bandwidth limit has been reached.
> 
> Is there any way to easy shape the traffic (we need as little downtime as
> possible.)?
> 
> Could you please reply to my email address as well as the list - :-)
> 
> Thanks in advance.
> 
> Regards
> --
> Lasse Laursen <[EMAIL PROTECTED]> - Systems Developer
> NetGroup A/S, St. Kongensgade 40H, DK-1264 K?benhavn K, Denmark
> Phone: +45 3370 1526 - Fax: +45 3313 0066 - Web: www.netgroup.dk
> 
> - We don't surf the net, we make the waves.
> 
> 
> 
> To Unsubscribe: send mail to [EMAIL PROTECTED]
> with "unsubscribe freebsd-stable" in the body of the message

-- 
Scott Nolde
GPG Key 0xD869AB48

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-stable" in the body of the message