Re: PF inadequacy: queue download
On Wednesday 03 May 2006 15:15, Trevor Talbot wrote: > On Tuesday, May 2, 2006, at 19:52 US/Pacific, Lars Hansson wrote: > > On Wednesday 03 May 2006 00:15, Karl O. Pinc wrote: > >> On 05/02/2006 02:22:33 AM, Lars Hansson wrote: > >>> The majority of users/developers has a separate firewall and then > >>> "download queing" is just a matter of doing it on the inside > >>> interface. > >> > >> To be fair, this only works if you've a single "inside interface". > > > > No, it works with multiple. > > He means when you need a single limit shared among multiple internal > clients -- and they're on different interfaces. Search the list > archives for details on his network; he's posted about it before. Ah right, I remember that now. --- Lars
Re: PF inadequacy: queue download
On Tuesday, May 2, 2006, at 19:52 US/Pacific, Lars Hansson wrote: On Wednesday 03 May 2006 00:15, Karl O. Pinc wrote: On 05/02/2006 02:22:33 AM, Lars Hansson wrote: The majority of users/developers has a separate firewall and then "download queing" is just a matter of doing it on the inside interface. To be fair, this only works if you've a single "inside interface". No, it works with multiple. He means when you need a single limit shared among multiple internal clients -- and they're on different interfaces. Search the list archives for details on his network; he's posted about it before.
Re: PF inadequacy: queue download
On Wednesday 03 May 2006 00:15, Karl O. Pinc wrote: > On 05/02/2006 02:22:33 AM, Lars Hansson wrote: > > The majority of users/developers has a separate firewall and then > > "download > > queing" is just a matter of doing it on the inside interface. > > To be fair, this only works if you've a single "inside interface". No, it works with multiple. --- Lars Hansson
Re: PF inadequacy: queue download
On 05/02/2006 02:22:33 AM, Lars Hansson wrote: The majority of users/developers has a separate firewall and then "download queing" is just a matter of doing it on the inside interface. To be fair, this only works if you've a single "inside interface". Karl <[EMAIL PROTECTED]> Free Software: "You don't pay back, you pay forward." -- Robert A. Heinlein
Re: PF inadequacy: queue download
> I'll summarize again for you. pick one: > > 1) submit a diff > 2) pay a developer to do it > 3) get over it Get over what? This is a suggestion, a feature request. As Travis H. said: > Well that's a way of looking at it. Alternately, some coders may wake > up one day and wonder what they should code. Maybe all their itches > are being scratched, but they still want to code something. Maybe > they don't know which direction will be of the most benefit to the > community at large. I know I personally have gotten (and implemented) > ideas from others. > note that "continue whining on the mailing lists and annoy developers > enough so that they eventually might unsubscribe" is not on the list. If you classify a feature request, and justifications and defences for that feature request, as whining, then just ignore this thread. Why threaten to unsubscribe? After a post like that I couldn't care less. Kestas
Re: PF inadequacy: queue download
On Tue, May 02, 2006 at 02:32:31AM -0700, [EMAIL PROTECTED] wrote: > I'm not demanding anyone do anything, I'm not trolling, I just want to > get this acknowledged as an area for potential development. Why > everyone's so resistant to this is beyond me. That this is the only > extra feature I'd like to have in pf I think reflects pretty well on > pf. No problem. I, dhartmei@, in my role as OpenBSD and pf developer[1], hereby acknowledge inbound queueing in pf/altq as an area of potential development. Several users, on several occasions, have shown interest in this feature, and so far no strong technical reason has been named why it should not be added. I vouch that, if and when development in that area has occurred, I'll do my best to review, test, and reach a consensus to commit that work. I'm not resistant to this feature, I just don't want to implement it myself. I'd rather do laundry and watch a movie. That doesn't mean I discourage anyone else from doing it. Go ahead, knock yourself out. It's not a trivial task, you'll earn respect. Is that official enough? Can we move on now? Daniel [1] http://www.openbsd.org/cgi-bin/cvsweb/src/sys/net/pf.c?rev=1.1&content-type=text/x-cvsweb-markup
Re: PF inadequacy: queue download
What if your firewall box has ssh access on the external interface and you want to make sure no-one accessing sshd can hog up the bandwidth; you can't do this with pf. What if you're using OpenBSD as a desktop computer, you might want to allow certain applications different bandwidth allowances; you can't do this with pf (without an extra box). What if you've got an OpenBSD multi user sshd machine: you want to allow all users internet access, but you want to make sure they can all have an equal share of the download bandwidth; you can't do this with pf (without an extra box, and some sort of very messy identd+tables hack). As far as firewalls go I'm a big fan of pf, actually I just finished writing a fairly detailed walkthrough of my pf.conf ( http://kuliukas.com/guide.html ), which is why I don't want to have to use/recommend something else if I need this functionality; I think download shaping would be the icing on the cake. I'm not demanding anyone do anything, I'm not trolling, I just want to get this acknowledged as an area for potential development. Why everyone's so resistant to this is beyond me. That this is the only extra feature I'd like to have in pf I think reflects pretty well on pf. Kestas
Re: PF inadequacy: queue download
* [EMAIL PROTECTED] <[EMAIL PROTECTED]> [2006-05-01 02:50]: > I don't think time spent developing PF or ALTQ could be better spent > developing something other than download queueing. it's nice that you think so. now, let me tell you some news: it does not matter what you think. what matters is what we think, the ones that write the code. when we see a clean, well written diff that implements this and makes sense, we might incorporate that. maybe you could get one of us to code that if you fund him to do that (let me tell you beforehands that you're looking at a 4 digit number for sure). endless whining here will make sure we'll never implement it unless we have a reallyt urgent need. since we hadn't had that in the past, what, 4 years, it's kinda unlikely that changes. I'll summarize again for you. pick one: 1) submit a diff 2) pay a developer to do it 3) get over it note that "continue whining on the mailing lists and annoy developers enough so that they eventually might unsubscribe" is not on the list. -- Henning Brauer, [EMAIL PROTECTED], [EMAIL PROTECTED] BS Web Services, http://bsws.de OpenBSD-based Webhosting, Mail Services, Managed Servers, ...
Re: PF inadequacy: queue download
[EMAIL PROTECTED] writes: > it's good to hear from someone who isn't pretending to be a/speak for > the developers. Kestas, several core PF developers have responded to your original message and various follow-ups, essentially trying to elicit some sort of fact-based reasoning why this feature should be developed by them. Your response to them has been essentially "Some other product has this" mixed in with some repetions of "I want this", combined with a liberal dose of name-calling. Neither of these approaches are terribly useful as tools of persuasion. Discussions of this general mold seem to pop up with alarming frequency, so I will outline some of the smarter ways to get developers' or more experienced users' attention - * "Some other product has this, why can't PF do this" Well, in most cases it is likely that PF can indeed do what you want, but possibly in a slightly different way than what you would do with the other product. If you are able to explain to other list readers what it is you want to achieve, some helpful user is likely to point you in the right direction. Offering explanations of what you want to achieve is always a lot more useful than quoting a chunk of some other product's configuration file claiming that PF is deficient if its users can't parse product foo's configuration file. * "I want this! I'm sure PF does not have this" It is possible that you have indeed found a useful feature which could usefully be implemented. If you have identified a missing feature which you feel is essential to your network, then clearly something needs to be done. Making the case for a new feature to be implemented takes a bit of work (offering up some code for review usually helps), most important is making a well reasoned case that this is a) actually a useful feature and b) the feature belongs in the base system. It is quite possible that your project satisfies condition a, but not condition b, which means that it is better off as a separate program to be maintained outside the base system. Examples of "a, but not b" features which became widely used programs maintained outside the base system are pftop and expiretable. * "You're a bunch of mindless moron zealots! I want to talk to the real developers!" The likelihood that the stranger on the PF mailing list telling you that your favorite missing feature is not worth implementing is indeed a real developer with commit access to the source tree is far from negligible. If your explanation of the obvious goodness of your wished-for feature does not convince, you should consider the possibility that a) your idea isn't actually that obviously good or b) you need to work a bit more on that explanation. Abuse and name-calling never helps your case, ever. -- Peter N. M. Hansteen, member of the first RFC 1149 implementation team http://www.blug.linux.no/rfc1149/ http://www.datadok.no/ http://www.nuug.no/ "First, we kill all the spammers" The Usenet Bard, "Twice-forwarded tales" 20:11:56 delilah spamd[26905]: 146.151.48.74: disconnected after 36099 seconds.
Re: PF inadequacy: queue download
On Tuesday 02 May 2006 09:29, [EMAIL PROTECTED] wrote: >Why the resistance? >The other two major firewalls iptables and IPFW can do > it, why can't PF? Because it's not deemed a really urgent, or even wanted, feature, obviously. The majority of users/developers has a separate firewall and then "download queing" is just a matter of doing it on the inside interface. Most of the previous questions about "download queuing" has been from people who didnt know that you could achieve the this by queuing on the inside interface, not by people who wanted to shape traffic to services the firewall box itself. --- Lars Hansson
Re: PF inadequacy: queue download
> Firewalls should firewall, not serve services. Why not? This isn't a corporate HQ where the box comes under heavy load, it's my home firewall/gateway/file server/development box; there's no reason it can't perform all those roles (other than pf being unable to shape download traffic). > I'm sure I'm part of a large majority of list members who would be > thrilled to see this topic end. While there's no harm in asking for a > feature expansion, and a discussion about the technical feasibility of > it, when it devolves into accusations of "zealot speak", it's time to > move on. There was one accusation of zealot speak, because the comment made no sense whatsoever, and he has still not justified it. For some reason most people who are replying to this thread seem to be looking for any reason why this feature request should be ignored; it's not necessary, it's not needed, pay $2 if you want it done, someone mentioned "zealot speak", download shaping isn't needed on firewalls, etc.. Why the resistance? The other two major firewalls iptables and IPFW can do it, why can't PF?
Re: PF inadequacy: queue download
Thanks Travis, it's good to hear from someone who isn't pretending to be a/speak for the developers. Kestas
Re: PF inadequacy: queue download
[In a message on 01 May 2006 01:51:35 PDT, [EMAIL PROTECTED] wrote:] >This works adequetly (How could it be "better"? Sounds like zealot >speak to me.) if the boxes only function is NAT, but if it also runs >external services queueing inbound traffic on the internal interface >doesn't work, because external services get priority. Is it a firewall or a dessert topping? Firewalls should firewall, not serve services. If you can't afford one box to firewall, and another to provide services, well, you're in a fix. I got a Sun IPX I'll give you if you pay shipping (anything to end this topic) -- Hardly sucks any juice, too. I'm sure I'm part of a large majority of list members who would be thrilled to see this topic end. While there's no harm in asking for a feature expansion, and a discussion about the technical feasibility of it, when it devolves into accusations of "zealot speak", it's time to move on. Sean PS You're all buying your CDs, right? Once I'm employed again, I will be.
Re: PF inadequacy: queue download
[EMAIL PROTECTED] wrote: > > works just as good as it possibly could if pf had a "download" queue > > mechanism, if not better. > > This works adequetly (How could it be "better"? Sounds like zealot > speak to me. to answer that, i believe there's no room for discussion there, then. > if the boxes only function is NAT, but if it also runs > external services queueing inbound traffic if the machine's only function is NAT it does not run external services. if i have an external service which i want to have jurisdiction over the input traffic on, i run it behind the LAN side. if i have an external service that i don't care about getting download traffic control over, i run it where it makes the most sense to me. perhaps you don't have the same liberty of convenience. -- jared [ openbsd 3.9-current GENERIC ( mar 15 ) // i386 ]
Re: PF inadequacy: queue download
On 5/1/06, Can Erkin Acar <[EMAIL PROTECTED]> wrote: On Sun, Apr 30, 2006 at 08:22:51AM -0700, [EMAIL PROTECTED] wrote: > I don't think time spent developing PF or ALTQ could be better spent > developing something other than download queueing. Everyone here seems > to agree it's PF's worst deficiency. I don't know about "worst deficiency", but it sounds like it could be useful. This might surprise you but OpenBSD does not run on requests, or polls or democracy. If a developer feels that such a feature is intresting/important and have resources to spare, than the feature will be implemented. Well that's a way of looking at it. Alternately, some coders may wake up one day and wonder what they should code. Maybe all their itches are being scratched, but they still want to code something. Maybe they don't know which direction will be of the most benefit to the community at large. I know I personally have gotten (and implemented) ideas from others. For example, just today I pushed out a new distribution of pf_dns_lookup. This program will resolve a set of IPs or hosts, and either print them out (to be redirected to a file) or stuff them into a table. This is to follow the IPs of DDNS clients. I got the idea from a discussion about how to handle dynamic DNS on this very list. -- "Curiousity killed the cat, but for a while I was a suspect" -- Steven Wright Security Guru for Hire http://www.lightconsulting.com/~travis/ -><- GPG fingerprint: 9D3F 395A DAC5 5CCC 9066 151D 0A6B 4098 0C55 1484
Re: PF inadequacy: queue download
On Sun, Apr 30, 2006 at 08:22:51AM -0700, [EMAIL PROTECTED] wrote: > I don't think time spent developing PF or ALTQ could be better spent > developing something other than download queueing. Everyone here seems > to agree it's PF's worst deficiency. Intresting definition for "Everyone". It seems the definition does not include the developers. If you can not live without "download queueing" you can always: 1. Write and submit a patch 2. Fund a developer to do the work for you. 3. Go run something else. I guess you would get bored after a while and choose #3. Feel free to surprise me though. Note that "unending trolling on the mailing lists" > I'm thinking perhaps there's some messy hack I can come up with using > virtual interfaces, does anyone have any ideas? Not that I know of. > > and there are 'lots of people' that care deeply about this > I said lots of people *here*, and there aren't close to 40 here, and > $500 is very steep for an extra box which has to do so little, and this > wouldn't require a whole hackathon to code. I'm not sure why you're > being so resistant to a feature request. This might surprise you but OpenBSD does not run on requests, or polls or democracy. If a developer feels that such a feature is intresting/important and have resources to spare, than the feature will be implemented. The best way to get results is to 'shut up and code'. > > By all means, try it. There are more readers (and developers) on the misc@ > > and tech@ lists, so I'd start there. > I posted in misc, but no-one was biting, and I've just sent one to > tech. > > > Stock altq could put token buckets on input interfaces for rate > > limiting purposes, referred to as the "traffic conditioner" (CDNR). > > That capability was removed when the classifier was merged with pf. > > > > Old messages that might be relevant: > > http://www.benzedrine.cx/pf/msg02871.html > > http://www.benzedrine.cx/pf/msg07159.html (bottom) > Interesting, I wonder how difficult it would be to get the > functionality back. Any ideas on why it was dropped in the first place? Input condititioners are different from queues. You would *not* have the same amount of control over the traffic as you would have when using a separate box. Can
Re: PF inadequacy: queue download
> works just as good as it possibly could if pf had a "download" queue > mechanism, if not better. This works adequetly (How could it be "better"? Sounds like zealot speak to me.) if the boxes only function is NAT, but if it also runs external services queueing inbound traffic on the internal interface doesn't work, because external services get priority. Kestas
Re: PF inadequacy: queue download
On Sat, Apr 29, 2006 at 05:10:40PM +0200, Stanislaw Halik wrote: > > I can speak for myself - I can't afford both the hardware and the > electricity bill for a separate machine. Maybe downstream limiting isn't > very robust, but IMO is the biggest thing pf/altq lacks. i queue the incoming downstream on the outbound-towards-my-LAN side. works just as good as it possibly could if pf had a "download" queue mechanism, if not better. if you are in a colo situation and you can't afford a little soekris ( or somethin' ) to drop in between and do this -- well... kudos on finding so inexpensive a colo that you can afford that but not a little 1 time ~250 USD or so for the investment on soekris/whatever -- jared [ openbsd 3.9-current GENERIC ( mar 15 ) // i386 ]
Re: PF inadequacy: queue download
I don't think time spent developing PF or ALTQ could be better spent developing something other than download queueing. Everyone here seems to agree it's PF's worst deficiency. I'm thinking perhaps there's some messy hack I can come up with using virtual interfaces, does anyone have any ideas? > and there are 'lots of people' that care deeply about this I said lots of people *here*, and there aren't close to 40 here, and $500 is very steep for an extra box which has to do so little, and this wouldn't require a whole hackathon to code. I'm not sure why you're being so resistant to a feature request. > By all means, try it. There are more readers (and developers) on the misc@ > and tech@ lists, so I'd start there. I posted in misc, but no-one was biting, and I've just sent one to tech. > Stock altq could put token buckets on input interfaces for rate > limiting purposes, referred to as the "traffic conditioner" (CDNR). > That capability was removed when the classifier was merged with pf. > > Old messages that might be relevant: > http://www.benzedrine.cx/pf/msg02871.html > http://www.benzedrine.cx/pf/msg07159.html (bottom) Interesting, I wonder how difficult it would be to get the functionality back. Any ideas on why it was dropped in the first place? Kestas
Re: PF inadequacy: queue download
On Sat, Apr 29, 2006 at 04:26:19PM -0700, [EMAIL PROTECTED] wrote: > I'd have a look at this problem myself, but I'm not good with C. I was > hoping there was some sort of todo list I could petition this to be > added too, because lots of people here seem to agree this is pf's (and > ALTQ's) worst problem. I'm not sure if this thread is a statistically relevant measurement. If the main reason why the workaround with the additional machine is problematic is hardware and electricity cost, and there are 'lots of people' that care deeply about this, and each of them would save, say, $500 if the feature were implemented, it shouldn't be hard to collect $20k (two man-months, a reasonable estimate, I'd say) and send that to Theo with the request for implementation during a hackathon. Or, to turn the argument around, if the entire support for the feature is five people offering lollipops and eternal gratitute, developer time is probably better spent elsewhere. By all means, try it. There are more readers (and developers) on the misc@ and tech@ lists, so I'd start there. Daniel
Re: PF inadequacy: queue download
On Saturday, Apr 29, 2006, at 08:58 US/Pacific, Daniel Hartmeier wrote: On Sat, Apr 29, 2006 at 05:10:40PM +0200, Stanislaw Halik wrote: I can speak for myself - I can't afford both the hardware and the electricity bill for a separate machine. Maybe downstream limiting isn't very robust, but IMO is the biggest thing pf/altq lacks. What I tried to express in the last paragraph of the referenced mail was that it's not pf that's lacking anything, but altq. There simply are no input queues with altq. If you added those queues to altq, you might not even have to change a single line in pf to get them used. Stock altq could put token buckets on input interfaces for rate limiting purposes, referred to as the "traffic conditioner" (CDNR). That capability was removed when the classifier was merged with pf. Old messages that might be relevant: http://www.benzedrine.cx/pf/msg02871.html http://www.benzedrine.cx/pf/msg07159.html (bottom)
Re: PF inadequacy: queue download
> Question's of the form "why hasn't this been done" sound kind of weird > to me, almost as if they were rhetorical. Assuming you haven't donated > blood in the last month, "why haven't you"? Are you really interested in > the petty excuses humans have for not doing things? :) I was hoping you'd tell me something along the lines of "Because we already have, you just need to do XYZ". Unfortunately I also can't afford an extra machine to get the basic functionality of download queueing. > If you added those queues to altq, you might not even have to change a single > line in pf to get them used. Well you can only specifiy the total bandwidth for a certain interface, and all queues which belong to an interface are one-way. You'd have to have a way to specify download bandwidth on an interface and have a seperate set of child queues for download. Then you'd have to have a way of sending incoming and outgoing packets belonging to a certain state to two queues. It's doable, but it certainly isn't an internal ALTQ problem. I read ALTQ and pf were now 'married' (in Theo's words), so I'd think the problem would require changes to both pf and ALTQ. I'd have a look at this problem myself, but I'm not good with C. I was hoping there was some sort of todo list I could petition this to be added too, because lots of people here seem to agree this is pf's (and ALTQ's) worst problem. If there's some sort of bounty system with OpenBSD I'd be happy to set a bounty so get this done faster. Kestas
Re: PF inadequacy: queue download
On 04/29/2006 10:58:39 AM, Daniel Hartmeier wrote: What I tried to express in the last paragraph of the referenced mail was that it's not pf that's lacking anything, but altq. While there are now ties between pf and altq (pf classifying packets for altq, and pfctl setting up queues), that doesn't mean the core of altq is maintained by the people who maintain pf. It's in separate source files, and if you compare the respective CVS histories, you'll see the difference. Where would be the proper forum to discuss input queueing then? Karl <[EMAIL PROTECTED]> Free Software: "You don't pay back, you pay forward." -- Robert A. Heinlein
Re: PF inadequacy: queue download
On Sat, Apr 29, 2006 at 05:10:40PM +0200, Stanislaw Halik wrote: > I can speak for myself - I can't afford both the hardware and the > electricity bill for a separate machine. Maybe downstream limiting isn't > very robust, but IMO is the biggest thing pf/altq lacks. What I tried to express in the last paragraph of the referenced mail was that it's not pf that's lacking anything, but altq. There simply are no input queues with altq. If you added those queues to altq, you might not even have to change a single line in pf to get them used. While there are now ties between pf and altq (pf classifying packets for altq, and pfctl setting up queues), that doesn't mean the core of altq is maintained by the people who maintain pf. It's in separate source files, and if you compare the respective CVS histories, you'll see the difference. Question's of the form "why hasn't this been done" sound kind of weird to me, almost as if they were rhetorical. Assuming you haven't donated blood in the last month, "why haven't you"? Are you really interested in the petty excuses humans have for not doing things? :) Daniel
Re: PF inadequacy: queue download
On Sat, Apr 29, 2006, Daniel Hartmeier wrote: >> I know this is possible because IPFW with dummynet doesn't have any >> problems. If everyone loves PF because of its elegance why can't it do >> something as simple as queue download traffic? > http://lists.freebsd.org/mailman/htdig/freebsd-pf/2005-November/001657.html | But it wouldn't change anything, because the congestion is upstream of | your ALTQ box. You can drop as many packets as you like after you | received them, that doesn't free up any bandwidth on your downlink. It surely has its caveats, but has its use, too. I'm stuck with ipfw on one last machine, because I can't limit bulk TCP traffic with pf. Of course, downstream limiting will never throttle DoS attacks, ICMP or 'dumb' UDP traffic with no acknowledgements, but works just fine for everything else. Even on ipfw's contemptible 'dummynet'. | If you want to do this with ALTQ, you can do so by limiting outgoing | packets on the "other" interface, assuming the box is forwarding all | packets between two interfaces. I can speak for myself - I can't afford both the hardware and the electricity bill for a separate machine. Maybe downstream limiting isn't very robust, but IMO is the biggest thing pf/altq lacks. -- sh pgpapWeAMPhkf.pgp Description: PGP signature
Re: PF inadequacy: queue download
On 04/29/2006 08:05:47 AM, [EMAIL PROTECTED] wrote: "Note that queueing is only useful for packets in the outbound direction. But this is wrong. It's not too late to queue it; by queueing it and dropping some packets of inbound traffic the sending host slows down the speed at which it sends. It's a question of what's "useful". Not useful against a malicious agent, but useful for traffic shaping regardless. I'd promised some benchmarks to demonstrate this quite some time ago, but just have not been able to get to it. In the meantime I've had to set up an entirely separate box, a bridge with only 2 interfaces, just to get inbound queueing. :-( I'm doing VOIP, and in practice inbound queueing definately works. However, I've still flakey voip issues that I haven't worked out and so have not been willing to report anything. To support inbound queueing for voip in a really elegent fashion you probably want additional semantics in the queue so you can always have enough free bandwidth to handle another call and don't have to just throttle data way back just in case there's lots of voip traffic. But that's yet another issue Karl <[EMAIL PROTECTED]> Free Software: "You don't pay back, you pay forward." -- Robert A. Heinlein
Re: PF inadequacy: queue download
On Sat, Apr 29, 2006 at 06:05:47AM -0700, [EMAIL PROTECTED] wrote: > I know this is possible because IPFW with dummynet doesn't have any > problems. If everyone loves PF because of its elegance why can't it do > something as simple as queue download traffic? http://lists.freebsd.org/mailman/htdig/freebsd-pf/2005-November/001657.html Daniel
