Re: PF or BPF
On Tue, 14 Feb 2006, Otto Moerbeek wrote: SNIP > Something like: "Even a thread started by Dave might eventually--if > the topic has sufficiently departed from the original subject--lead to > a (small) improvement to OpenBSD?" > > -Otto Exactly!
Re: PF or BPF
On Tue, 14 Feb 2006, Tony Sterrett wrote: > > > I'm not sure I'd do it in that way. I'm thinking if BPF provided stateful > > > inspection is would be > > > more useful. > > > > Asking for stateful inspection in bpf(4) is like wanting a carburettor > > for a pushbike. You might be able to shoehorn it in there, but it won't > > be pretty, will ruin its simplicity and probably won't be much use. > > Yeah this would be something in addition to BPF and not to alter BPF. I like > the simple > functionary but I think it would be hard to management complex rule(s). The > language is > a little clunky. Just think is doing something when you have to check protocol > #, source and dst address > and TCP flags. I guess the fact that BPF branches only forward does both > simplify and limit its scope. Having only forward jumps is an essential (security) feature of bpf. This makes it possible to validate the bpf code to ensure that it terminates. After all, bpf let's a user inject code into the kernel. You want to be 100% sure it doesn't do nasty things. See sys/net/bpf_filter:bpf_validate(). -Otto
Re: PF or BPF
I'm not sure I'd do it in that way. I'm thinking if BPF provided stateful inspection is would be more useful. Asking for stateful inspection in bpf(4) is like wanting a carburettor for a pushbike. You might be able to shoehorn it in there, but it won't be pretty, will ruin its simplicity and probably won't be much use. Yeah this would be something in addition to BPF and not to alter BPF. I like the simple functionary but I think it would be hard to management complex rule (s). The language is a little clunky. Just think is doing something when you have to check protocol #, source and dst address and TCP flags. I guess the fact that BPF branches only forward does both simplify and limit its scope. FFPF is a different approach, and they (rightly) didn't use bpf(4) as their base implementation. Some of their ideas look pretty good, but if you are interested in pursuing them the you had probably best do it in parallel to the existing bpf(4) infrastructure. -d I'm at the survey stage. I know about a number of efforts which apply BPF-like technology to lots of applications. As you say, FFPF has some neat ideas, and it is efficient (context switching, number of copies) , more scalability (BPF is a little clunky no loops) and able to handle more complex situations. Its even has backward compatibility of BPF. However, It doesn't support BSD that as far as I know, I hadn't looked that closely for that reason. Might in interesting if its no overly dependent some linux kernel feature. Respectfully, Tony Sterrett [EMAIL PROTECTED] Consultant in Open Source Software, featuring OpenBSD and Linux. www.sterrett.net (858) 433-1467 San Diego (408) 705-2135 San Jose
Re: PF or BPF
On Tue, 14 Feb 2006, Diana Eichert wrote: > I was thinking there should have been something in the commit message > about Dave contributing to this fix. The entire xargs discussion wouldn't > have occurred if I hadn't used "find" in my reply to Dave regarding "PF > or BPF". > > http://marc.theaimsgroup.com/?l=openbsd-cvs&m=113991945111836&w=2 > Log message: > Document that -0 is an non-standard extension (sad but true). > ok millert@ > > g.day > > diana Something like: "Even a thread started by Dave might eventually--if the topic has sufficiently departed from the original subject--lead to a (small) improvement to OpenBSD?" -Otto
Re: PF or BPF
> I was thinking there should have been something in the commit message > about Dave contributing to this fix. The entire xargs discussion wouldn't > have occurred if I hadn't used "find" in my reply to Dave regarding "PF > or BPF". Something like ``please do not feed the Feustel again''? Miod
Re: PF or BPF
I was thinking there should have been something in the commit message about Dave contributing to this fix. The entire xargs discussion wouldn't have occurred if I hadn't used "find" in my reply to Dave regarding "PF or BPF". http://marc.theaimsgroup.com/?l=openbsd-cvs&m=113991945111836&w=2 Log message: Document that -0 is an non-standard extension (sad but true). ok millert@ g.day diana
Re: xargs PF or BPF
On Tue, 14 Feb 2006, Ray Lai wrote: > On Tue, Feb 14, 2006 at 11:39:45AM +0100, Otto Moerbeek wrote: > > On Tue, 14 Feb 2006, Michael Schmidt wrote: > > > > > Matthias Kilian wrote: > > > > > > > And watch out for silly file names containing whitespace. > > > > > > > > BTW: if this is a contest on creative use of find(1) and other > > > > standard tools: > > > > > > > > $ find . -type f | sed '[EMAIL PROTECTED]@grep -l -- foo @' | sh > > > > > > > > Yes, this isn't robust against whitespace, either PLUS it's > > > > inefficient. But in some cases the find ... | sed ... | sh pattern > > > > is quite useful. > > > > > > > > > > Sometime ago I have had the same problem with spaces in filenames and > > > dealing > > > with them as xargs parameters. There I have used (here only as an > > > example): > > > > > > find . -print | grep -i ' ' | xargs -I {} ls -ald {} > > > > > > FYI, that has been on a non-OpenBSD system. > > > I4m not at my OpenBSD system at the moment, so I can4t check whether > > > OpenBSD > > > xargs supports the shown options. Maybe someone may test it. > > > > > > One may check this at a directory with space-containing filenames. > > > Without the "-I {}" and "{}" parts you get funny output. > > > > > > > Well, -print0 in find and xargs -0 are designed to deal with that. > > Sadly these are not in POSIX (which is not documented correctly in the > > xargs case). > > Does this diff fix it? (I also added a comma after the last -R.) I already committed a similar diff, -Otto > > -Ray- > > Index: xargs.1 > === > RCS file: /cvs/src/usr.bin/xargs/xargs.1,v > retrieving revision 1.15 > diff -u -r1.15 xargs.1 > --- xargs.1 12 Sep 2005 09:44:59 - 1.15 > +++ xargs.1 14 Feb 2006 13:37:48 - > @@ -316,7 +316,8 @@ > .St -p1003.2 > compliant. > The > -.Fl J , o , P , R > +.Fl 0 , J , o , P , > +.Fl R , > and > .Fl r > options are non-standard
Re: xargs PF or BPF
On Tue, Feb 14, 2006 at 11:39:45AM +0100, Otto Moerbeek wrote: > On Tue, 14 Feb 2006, Michael Schmidt wrote: > > > Matthias Kilian wrote: > > > > > And watch out for silly file names containing whitespace. > > > > > > BTW: if this is a contest on creative use of find(1) and other > > > standard tools: > > > > > > $ find . -type f | sed '[EMAIL PROTECTED]@grep -l -- foo @' | sh > > > > > > Yes, this isn't robust against whitespace, either PLUS it's > > > inefficient. But in some cases the find ... | sed ... | sh pattern > > > is quite useful. > > > > > > > Sometime ago I have had the same problem with spaces in filenames and > > dealing > > with them as xargs parameters. There I have used (here only as an example): > > > > find . -print | grep -i ' ' | xargs -I {} ls -ald {} > > > > FYI, that has been on a non-OpenBSD system. > > I4m not at my OpenBSD system at the moment, so I can4t check whether OpenBSD > > xargs supports the shown options. Maybe someone may test it. > > > > One may check this at a directory with space-containing filenames. > > Without the "-I {}" and "{}" parts you get funny output. > > > > Well, -print0 in find and xargs -0 are designed to deal with that. > Sadly these are not in POSIX (which is not documented correctly in the > xargs case). Does this diff fix it? (I also added a comma after the last -R.) -Ray- Index: xargs.1 === RCS file: /cvs/src/usr.bin/xargs/xargs.1,v retrieving revision 1.15 diff -u -r1.15 xargs.1 --- xargs.1 12 Sep 2005 09:44:59 - 1.15 +++ xargs.1 14 Feb 2006 13:37:48 - @@ -316,7 +316,8 @@ .St -p1003.2 compliant. The -.Fl J , o , P , R +.Fl 0 , J , o , P , +.Fl R , and .Fl r options are non-standard
Re: xargs PF or BPF
On Tue, 14 Feb 2006, Michael Schmidt wrote: > Matthias Kilian wrote: > > > And watch out for silly file names containing whitespace. > > > > BTW: if this is a contest on creative use of find(1) and other > > standard tools: > > > > $ find . -type f | sed '[EMAIL PROTECTED]@grep -l -- foo @' | sh > > > > Yes, this isn't robust against whitespace, either PLUS it's > > inefficient. But in some cases the find ... | sed ... | sh pattern > > is quite useful. > > > > Sometime ago I have had the same problem with spaces in filenames and dealing > with them as xargs parameters. There I have used (here only as an example): > > find . -print | grep -i ' ' | xargs -I {} ls -ald {} > > FYI, that has been on a non-OpenBSD system. > I4m not at my OpenBSD system at the moment, so I can4t check whether OpenBSD > xargs supports the shown options. Maybe someone may test it. > > One may check this at a directory with space-containing filenames. > Without the "-I {}" and "{}" parts you get funny output. > Well, -print0 in find and xargs -0 are designed to deal with that. Sadly these are not in POSIX (which is not documented correctly in the xargs case). -Otto
Re: xargs PF or BPF
Matthias Kilian wrote: And watch out for silly file names containing whitespace. BTW: if this is a contest on creative use of find(1) and other standard tools: $ find . -type f | sed '[EMAIL PROTECTED]@grep -l -- foo @' | sh Yes, this isn't robust against whitespace, either PLUS it's inefficient. But in some cases the find ... | sed ... | sh pattern is quite useful. Sometime ago I have had the same problem with spaces in filenames and dealing with them as xargs parameters. There I have used (here only as an example): find . -print | grep -i ' ' | xargs -I {} ls -ald {} FYI, that has been on a non-OpenBSD system. I4m not at my OpenBSD system at the moment, so I can4t check whether OpenBSD xargs supports the shown options. Maybe someone may test it. One may check this at a directory with space-containing filenames. Without the "-I {}" and "{}" parts you get funny output. Have a nice day Michael -- Michael Schmidt MIRRORS: DJGPP ftp://ftp.fh-koblenz.de/pub/DJGPP/ Ghostscript ftp://ftp.fh-koblenz.de/pub/Ghostscript/
Re: PF or BPF
On Mon, 13 Feb 2006, Tony Sterrett wrote: > I'm not sure I'd do it in that way. I'm thinking if BPF provided stateful > inspection is would be > more useful. Asking for stateful inspection in bpf(4) is like wanting a carburettor for a pushbike. You might be able to shoehorn it in there, but it won't be pretty, will ruin its simplicity and probably won't be much use. FFPF is a different approach, and they (rightly) didn't use bpf(4) as their base implementation. Some of their ideas look pretty good, but if you are interested in pursuing them the you had probably best do it in parallel to the existing bpf(4) infrastructure. -d
Re: xargs PF or BPF
On Mon, Feb 13, 2006 at 06:32:53PM -0800, Ted Unangst wrote: > > > find ./ -type f -print | xargs -i rm -f > > > Instead of > > > rm -f $(find ./ -type f -print) > > > > Because that will fail when there are too many arguments, and will > > probably break on filenames with spaces (use xargs -0 for these). > > with either version, hilarity is sure to ensue when you discover an > accidentally created -r file. always use --. And watch out for silly file names containing whitespace. BTW: if this is a contest on creative use of find(1) and other standard tools: $ find . -type f | sed '[EMAIL PROTECTED]@grep -l -- foo @' | sh Yes, this isn't robust against whitespace, either PLUS it's inefficient. But in some cases the find ... | sed ... | sh pattern is quite useful. Ciao, Kili
Re: PF or BPF
On Feb 13, 2006, at 5:16 PM, Ted Unangst wrote: On 2/13/06, Tony Sterrett <[EMAIL PROTECTED]> wrote: I'm looking at the tradeoff of porting bpf with states from linux to OpenBSD from linux. Daniel Hartmeier in Design and Performance of the "OpenBSD Stateful Packet Filter (pf)" says that pf is more efficient than bpf, so it may be pointless. On the other hand having this facility would increase the richness of our toolkit. what you want to do is add the ability to attach a bpf filter to a pf rule, though i wouldn't try to wedge this functionality into pf.conf's grammar. i still haven't come up with much reason why you need bpf to express a rule pattern that can't be done with pf. I'm not sure I'd do it in that way. I'm thinking if BPF provided stateful inspection is would be more useful. There are a few BPF enhancement projects like FFPF: Fairly Fast Packet Filters Vrije Universiteit Amsterdam, The Netherlands. that add stateful packet inspection. I think this only runs in linux and it uses linux kernel hooks and therefore would need porting. So the notion was to port. My reasoning for stateful BPF is mainly in the sprit of providing a rich toolset. Provide primitives not solutions. It will just provided an additional means of packet inspection. Also there is a lot of research that uses BPF in various ways packet classifier, packet switching for grid, NIDS, reconfigurable networks, etc. Some of this may be useful in the future to a creative designer in ways unknown. Respectfully, Tony Sterrett [EMAIL PROTECTED] Consultant in Open Source Software, featuring OpenBSD and Linux. www.sterrett.net (858) 433-1467 San Diego (408) 705-2135 San Jose
Re: PF or BPF
On Mon, 13 Feb 2006, Jason Crawford wrote: SNIP > He couldn't even figure out how to find the applications that use bpf, > so I think figuring out all the features in a utility might be out of > his grasp... > > Jason hence my original suggestion, minus my "|" miscue of course.
Re: xargs PF or BPF
On 2/13/06, Andrew Pinski <[EMAIL PROTECTED]> wrote: > > On Feb 13, 2006, at 10:00 PM, Jason Crawford wrote: > >> > >> Time to write your own program in C instead if the time to invoke > >> rm is taking too much time. > > > > No point, xargs does what I need it to do, and is much more efficient > > than having find execute rm itself. The fewer times you call execve(2) > > the better. > > > One execve is enough? Then rewriting the script into C will save > all execve. Do you think rm(1) does anything special, > other than remove(3)? You misunderstand. find calls execve(2) for every file it finds matching the criteria, where as xargs will only execve(2) once either the pipe is closed or it hits the max args. That's A LOT fewer execve(2) calls. And rm calls unlink(2), as remove(3) would waste even more resources. Jason
Re: xargs PF or BPF
On Mon, 13 Feb 2006, Andrew Pinski wrote: > Time to write your own program in C instead if the time to invoke > rm is taking too much time. rm *is* a small program written in C. You need to consider how the tools actually invoke it - think about it for a while. -d
Re: xargs PF or BPF
On Feb 13, 2006, at 10:00 PM, Jason Crawford wrote: Time to write your own program in C instead if the time to invoke rm is taking too much time. No point, xargs does what I need it to do, and is much more efficient than having find execute rm itself. The fewer times you call execve(2) the better. One execve is enough? Then rewriting the script into C will save all execve. Do you think rm(1) does anything special, other than remove(3)? -- Pinski
Re: xargs PF or BPF
On Mon, 13 Feb 2006, Andrew Pinski wrote: > > On Feb 13, 2006, at 9:24 PM, Damien Miller wrote: > > Because that will fail when there are too many arguments, and will > > probably break on filenames with spaces (use xargs -0 for these). > > Why not use -exec in find? > > find . -type f -name ttt -exec rm {}\; this conversation is going in circles. Look back a couple of messages in this very thread
Re: xargs PF or BPF
On 2/13/06, Andrew Pinski <[EMAIL PROTECTED]> wrote: > > On Feb 13, 2006, at 9:53 PM, Jason Crawford wrote: > > > On 2/13/06, Andrew Pinski <[EMAIL PROTECTED]> wrote: > >> On Feb 13, 2006, at 9:24 PM, Damien Miller wrote: > >>> Because that will fail when there are too many arguments, and will > >>> probably break on filenames with spaces (use xargs -0 for these). > >> > >> Why not use -exec in find? > >> > >> find . -type f -name ttt -exec rm {}\; > > > > Because as stated many times on this list already (originally to > > correct me), that will execute rm for each file, while piping to xargs > > will only run rm once xargs stops getting input, or when it hits max > > command line length, in which case it will execute another rm based on > > input from the pipe. > > Time to write your own program in C instead if the time to invoke > rm is taking too much time. No point, xargs does what I need it to do, and is much more efficient than having find execute rm itself. The fewer times you call execve(2) the better. Jason
Re: xargs PF or BPF
On 2/13/06, Andrew Pinski <[EMAIL PROTECTED]> wrote: > On Feb 13, 2006, at 9:24 PM, Damien Miller wrote: > > Because that will fail when there are too many arguments, and will > > probably break on filenames with spaces (use xargs -0 for these). > > Why not use -exec in find? > > find . -type f -name ttt -exec rm {}\; Because as stated many times on this list already (originally to correct me), that will execute rm for each file, while piping to xargs will only run rm once xargs stops getting input, or when it hits max command line length, in which case it will execute another rm based on input from the pipe. Jason
Re: xargs PF or BPF
On Feb 13, 2006, at 9:53 PM, Jason Crawford wrote: On 2/13/06, Andrew Pinski <[EMAIL PROTECTED]> wrote: On Feb 13, 2006, at 9:24 PM, Damien Miller wrote: Because that will fail when there are too many arguments, and will probably break on filenames with spaces (use xargs -0 for these). Why not use -exec in find? find . -type f -name ttt -exec rm {}\; Because as stated many times on this list already (originally to correct me), that will execute rm for each file, while piping to xargs will only run rm once xargs stops getting input, or when it hits max command line length, in which case it will execute another rm based on input from the pipe. Time to write your own program in C instead if the time to invoke rm is taking too much time. -- Pinski
Re: PF or BPF
On 2/13/06, Stuart Henderson <[EMAIL PROTECTED]> wrote: > On 2006/02/13 17:28, Jason Crawford wrote: > > Well in the case of /usr/src, I think you must MIGHT hit the maximum > > argument length for the shell by using xargs > > I haven't seen xargs do the wrong thing here. Embedded spaces annoy, > but that's what -print0 (to find) and -0 (to xargs) are for. I almost > always use xargs here, to the extent I have to look up how to do a > 'find -exec' most times that I want to use it. I guess I'm used to older behavior I've seen on other non-OpenBSD systems. Thanks for the corrections from everyone. Like someone has previously stated, you learn something new from some of these threads that were previously thought useless. > > That and well, explaining xargs to Dave > > will end up leading to another 20+ mail thread > > I think an actual utility that doesn't need programming skills to > experiment with it might be easier than explaining Berkeley Packet > Filter vs. Packet Filter. I know most of us know what BPF is, > but googling around from a beginner's point of view I'm still not > quite sure how I learnt about it. There's a paper at > http://www.tcpdump.org/papers/bpf-usenix93.pdf (section 2, 'the > network tap', for example) but I know I haven't read that before. > > Learning xargs and find (not to mention regular expressions, > shell syntax - for/while/..., and so on) are probably more useful > to general sysadmin tasks than learning what BPF is, though.. > (even learning how to use tcpdump is probably more generally > useful than learning about BPF - and let's pre-empt one possible > path down that avenue: root being able to see certain passwords > with 'tcpdump -s1500 -X' is not a security hole, it's just a > demonstration of why some protocols should be buried). He couldn't even figure out how to find the applications that use bpf, so I think figuring out all the features in a utility might be out of his grasp... Jason
Re: PF or BPF
On Monday 13 February 2006 21:25, Damien Miller wrote: > On Mon, 13 Feb 2006, Dave Feustel wrote: > > > Marco, > > > > I would like to add that I appreciate the work you and the rest of the > > crew are doing to develop OpenBSD. > > Please show your appreciation by educating yourself using the available > manpages (which represent a huge amount of work) before asking questions > whose answers can easily be found with a little reading. > > -d Roger. Wilco. -- Lose, v., experience a loss, get rid of, "lose the weight" Loose, adj., not tight, let go, free, "loose clothing"
Re: xargs PF or BPF
On 2/13/06, Damien Miller <[EMAIL PROTECTED]> wrote: > > Why so many people is using xargs ? > > > > I mean for instance why bother use xargs AND a pipe to do somthing like this > > : > > > > find ./ -type f -print | xargs -i rm -f > > Instead of > > rm -f $(find ./ -type f -print) > > Because that will fail when there are too many arguments, and will > probably break on filenames with spaces (use xargs -0 for these). with either version, hilarity is sure to ensue when you discover an accidentally created -r file. always use --.
Re: xargs PF or BPF
On Feb 13, 2006, at 9:24 PM, Damien Miller wrote: Because that will fail when there are too many arguments, and will probably break on filenames with spaces (use xargs -0 for these). Why not use -exec in find? find . -type f -name ttt -exec rm {}\; -- Pinski
Re: PF or BPF
On Mon, 13 Feb 2006, Dave Feustel wrote: > Marco, > > I would like to add that I appreciate the work you and the rest of the > crew are doing to develop OpenBSD. Please show your appreciation by educating yourself using the available manpages (which represent a huge amount of work) before asking questions whose answers can easily be found with a little reading. -d
Re: xargs PF or BPF
On Tue, 14 Feb 2006, noob lenoobie wrote: > On Mon, 13 Feb 2006, Matthias Kilian wrote: > >(b) pipeing to xargs(1) may be faster. > > Why so many people is using xargs ? > > I mean for instance why bother use xargs AND a pipe to do somthing like this > : > > find ./ -type f -print | xargs -i rm -f > Instead of > rm -f $(find ./ -type f -print) Because that will fail when there are too many arguments, and will probably break on filenames with spaces (use xargs -0 for these). -d
Re: PF or BPF
Hi Dave, On 2006.02.14, at 12:53 PM, Dave Feustel wrote: Marco, I would like to add that I appreciate the work you and the rest of the crew are doing to develop OpenBSD. It might be best in the future to first outline what you've done to research your questions and then ask the question. Otherwise people will think you've done nothing and the thread will start with flames. Leaving us hanging with the X vulnerabilities and providing no details is not a good way to start with that either. Persisting with that behaviour after people have asked for details would only fan the flames. If you gave details in the first post, people here qualified to answer could have put that to rest quickly and you would not be building up a reputation. Bye for now, Shane J Pearsonshanejp netspace net au ->|
Re: xargs PF or BPF
On Mon, 13 Feb 2006, Matthias Kilian wrote: >(b) pipeing to xargs(1) may be faster. Why so many people is using xargs ? I mean for instance why bother use xargs AND a pipe to do somthing like this : find ./ -type f -print | xargs -i rm -f Instead of rm -f $(find ./ -type f -print) ? Richard.
Re: PF or BPF
Marco, I would like to add that I appreciate the work you and the rest of the crew are doing to develop OpenBSD. On Monday 13 February 2006 19:36, Marco Peereboom wrote: > http://www.oxide.org/cvs/tedu.html > Commit Statistics: > > Total: 864 > src: 834 (96.528%) > ports: 6 (0.694%) > www: 24 (2.778%) > Total Days: 1095 > Average per day: 0.789 > Oldest: Tue Jan 28 16:00:45 MST 2003 > Newest: Sat Jan 28 00:27:38 MST 2006
Re: PF or BPF
dereck wrote: The responses here are totally out of line. So was his last comment in http://groups.google.com/group/lucky.openbsd.misc/msg/942c4c6d5bc26fca
Re: PF or BPF
On Monday 13 February 2006 19:36, Marco Peereboom wrote: > Time for you to start using Linux, Windows or OSX. > OpenBSD is clearly not fulfilling your needs Your psychic abilities are failing you again. > and the lists are unfriendly. So What? > http://www.oxide.org/cvs/tedu.html > Commit Statistics: > > Total: 864 > src: 834 (96.528%) > ports: 6 (0.694%) > www: 24 (2.778%) > Total Days: 1095 > Average per day: 0.789 > Oldest: Tue Jan 28 16:00:45 MST 2003 > Newest: Sat Jan 28 00:27:38 MST 2006 > > Dave Feustel: > Commit Statistics: > > Total: 0 So What? -- Lose, v., experience a loss, get rid of, "lose the weight" Loose, adj., not tight, let go, free, "loose clothing"
Re: PF or BPF
On 2/13/06, Tony Sterrett <[EMAIL PROTECTED]> wrote: > I'm looking at the tradeoff of porting bpf with states from linux to > OpenBSD from linux. Daniel Hartmeier in Design and Performance of > the "OpenBSD Stateful Packet Filter (pf)" says that pf is more > efficient than bpf, so it may be pointless. On the other hand having > this facility would increase the richness of our toolkit. what you want to do is add the ability to attach a bpf filter to a pf rule, though i wouldn't try to wedge this functionality into pf.conf's grammar. i still haven't come up with much reason why you need bpf to express a rule pattern that can't be done with pf.
Re: PF or BPF
Hey, BPF is a really cool pseudo device (software that's access like a device, you'll see it in /dev). It is programmed with a assembly like load/store instruction set. This is a very efficient way of filtering incoming packets. It used by tcpdump, pcap and ppp. Its neat but it doesn't know about state which PF does. BPF is used (or can be used ) on a lot of systems but its is a BSD Native as the named indicates. There has been some research on adding state and performance enhancement in projects named BPF+ and DBPF. I'm looking at the tradeoff of porting bpf with states from linux to OpenBSD from linux. Daniel Hartmeier in Design and Performance of the "OpenBSD Stateful Packet Filter (pf)" says that pf is more efficient than bpf, so it may be pointless. On the other hand having this facility would increase the richness of our toolkit. BTW tcpdump can be used to produce a bpf program and pcap is a convent way of using BPF. Here's an example. mail# tcpdump -d host 192.142.1.1 and dst port 80 (000) ldh [12] (001) jeq #0x800 jt 2jf 15 (002) ld [26] (003) jeq #0xc08e0101 jt 6jf 4 (004) ld [30] (005) jeq #0xc08e0101 jt 6jf 15 (006) ldb [23] (007) jeq #0x6 jt 9jf 8 (008) jeq #0x11jt 9jf 15 (009) ldh [20] (010) jset #0x1fff jt 15 jf 11 (011) ldxb 4*([14]&0xf) (012) ldh [x + 16] (013) jeq #0x50jt 14 jf 15 (014) ret #96 (015) ret #0 mail# Respectfully, Tony Sterrett [EMAIL PROTECTED] Consultant in Open Source Software, featuring OpenBSD and Linux. www.sterrett.net (858) 433-1467 San Diego (408) 705-2135 San Jose On Feb 13, 2006, at 3:17 PM, Ray Lai wrote: On Mon, Feb 13, 2006 at 05:28:22PM -0500, Jason Crawford wrote: Well in the case of /usr/src, I think you must MIGHT hit the maximum argument length for the shell by using xargs, unless you did it inside of each directory in /usr/src. That and well, explaining xargs to Dave will end up leading to another 20+ mail thread xargs runs the command multiple times when it hits this limit: [EMAIL PROTECTED] jot 0 | xargs echo | sed 's/ .*//' 1 5001 10001 15001 20001 25001 ^C The manual tries to describe this behavior: Any arguments specified on the command line are given to the utility upon each invocation, followed by some number of the arguments read from stan- dard input. The utility is repeatedly executed one or more times until standard input is exhausted. -Ray-
Re: PF or BPF
Time for you to start using Linux, Windows or OSX. OpenBSD is clearly not fulfilling your needs and the lists are unfriendly. http://www.oxide.org/cvs/tedu.html Commit Statistics: Total: 864 src: 834 (96.528%) ports: 6 (0.694%) www: 24 (2.778%) Total Days: 1095 Average per day: 0.789 Oldest: Tue Jan 28 16:00:45 MST 2003 Newest: Sat Jan 28 00:27:38 MST 2006 Dave Feustel: Commit Statistics: Total: 0 On Mon, Feb 13, 2006 at 01:00:36PM -0500, Dave Feustel wrote: > On Monday 13 February 2006 12:45, Ted Unangst wrote: > > On 2/13/06, Dave Feustel <[EMAIL PROTECTED]> wrote: > > > What can BPF do that PF can not? > > > > different things. > > OK, I'll bite. Such as? > (this might be a loong, drawnout thread, but I've got time :-)) > > -- > Lose, v., experience a loss, get rid of, "lose the weight" > Loose, adj., not tight, let go, free, "loose clothing"
Re: PF or BPF
On 2006/02/13 17:28, Jason Crawford wrote: > Well in the case of /usr/src, I think you must MIGHT hit the maximum > argument length for the shell by using xargs I haven't seen xargs do the wrong thing here. Embedded spaces annoy, but that's what -print0 (to find) and -0 (to xargs) are for. I almost always use xargs here, to the extent I have to look up how to do a 'find -exec' most times that I want to use it. > That and well, explaining xargs to Dave > will end up leading to another 20+ mail thread I think an actual utility that doesn't need programming skills to experiment with it might be easier than explaining Berkeley Packet Filter vs. Packet Filter. I know most of us know what BPF is, but googling around from a beginner's point of view I'm still not quite sure how I learnt about it. There's a paper at http://www.tcpdump.org/papers/bpf-usenix93.pdf (section 2, 'the network tap', for example) but I know I haven't read that before. Learning xargs and find (not to mention regular expressions, shell syntax - for/while/..., and so on) are probably more useful to general sysadmin tasks than learning what BPF is, though.. (even learning how to use tcpdump is probably more generally useful than learning about BPF - and let's pre-empt one possible path down that avenue: root being able to see certain passwords with 'tcpdump -s1500 -X' is not a security hole, it's just a demonstration of why some protocols should be buried).
Re: xargs PF or BPF
On 2006-02-13 18:10:53 -0500, Tim Donahue wrote: > As done by xargs? > > grep foo 1 > > grep foo 2 > > grep foo 3 Any arguments specified on the command line are given to the utility upon each invocation, followed by some number of the arguments read from stan- dard input. The utility is repeatedly executed until standard input is exhausted. > Wouldn't for a small list -exec be faster as it is a single invocation of the > grep vs multiple invocations of grep for xargs. There won't be more invocations than necessary. > IIRC, the reason xargs exists is to get around limits posed by the number of > args an application like rm or grep can take when passed a large list of > arguments like, say, a recursive find for old files in your spam > quarantine :-). That is _one_ reason. Best Martin -- http://www.tm.oneiros.de
Re: PF or BPF
On Mon, Feb 13, 2006 at 05:28:22PM -0500, Jason Crawford wrote: > Well in the case of /usr/src, I think you must MIGHT hit the maximum > argument length for the shell by using xargs, unless you did it inside > of each directory in /usr/src. That and well, explaining xargs to Dave > will end up leading to another 20+ mail thread xargs runs the command multiple times when it hits this limit: [EMAIL PROTECTED] jot 0 | xargs echo | sed 's/ .*//' 1 5001 10001 15001 20001 25001 ^C The manual tries to describe this behavior: Any arguments specified on the command line are given to the utility upon each invocation, followed by some number of the arguments read from stan- dard input. The utility is repeatedly executed one or more times until standard input is exhausted. -Ray-
Re: xargs PF or BPF
On Monday 13 February 2006 17:13, Stuart Henderson wrote: > On 2006/02/13 16:53, Jason Crawford wrote: > > On 2/13/06, Matthias Kilian <[EMAIL PROTECTED]> wrote: > > > On Mon, Feb 13, 2006 at 02:03:27PM -0700, Diana Eichert wrote: > > > > find /usr/src -name "*.[c|h]" -exec grep 'bpf.h' /dev/null {} \; > > it's in quotes, this is handled by find, not the shell. > > > > (b) pipeing to xargs(1) may be faster. > > > > why? > As done by find -exec, correct? > grep foo 1 2 3 4 5 6 7 ... > > vs. > As done by xargs? > grep foo 1 > grep foo 2 > grep foo 3 > ... Wouldn't for a small list -exec be faster as it is a single invocation of the grep vs multiple invocations of grep for xargs. IIRC, the reason xargs exists is to get around limits posed by the number of args an application like rm or grep can take when passed a large list of arguments like, say, a recursive find for old files in your spam quarantine :-). Tim Donahue
Re: PF or BPF
On Mon, 13 Feb 2006, Greg Thomas wrote: SNIP > > > > (b) pipeing to xargs(1) may be faster. > > > why? > > > > grep foo 1 2 3 4 5 6 7 ... > > > > vs. > > > > grep foo 1 > > grep foo 2 > > grep foo 3 > > grep foo 4 > > grep foo 5 > > grep foo 6 > > grep foo 7 > > One of the nice things about misc is that even if a discussion starts > out poorly I still end up learning something. > > Greg glad I could help. ;-) diana
Re: PF or BPF
On 2/13/06, Stuart Henderson <[EMAIL PROTECTED]> wrote: > On 2006/02/13 16:53, Jason Crawford wrote: > > On 2/13/06, Matthias Kilian <[EMAIL PROTECTED]> wrote: > > > On Mon, Feb 13, 2006 at 02:03:27PM -0700, Diana Eichert wrote: > > > > find /usr/src -name "*.[c|h]" -exec grep 'bpf.h' /dev/null {} \; > > >^(a) ^(b) > > > > > > (a) I doubt there are any file names ending in a pipe symbol in /usr/src. > > man ksh > > it's in quotes, this is handled by find, not the shell. Right, my mistake. > > > (b) pipeing to xargs(1) may be faster. > > why? > > grep foo 1 2 3 4 5 6 7 ... > > vs. > > grep foo 1 > grep foo 2 > grep foo 3 > grep foo 4 > grep foo 5 > grep foo 6 > grep foo 7 Well in the case of /usr/src, I think you must MIGHT hit the maximum argument length for the shell by using xargs, unless you did it inside of each directory in /usr/src. That and well, explaining xargs to Dave will end up leading to another 20+ mail thread Jason
Re: PF or BPF
> > > find /usr/src -name "*.[c|h]" -exec grep 'bpf.h' /dev/null {} \; > >^(a) ^(b) > > > > (a) I doubt there are any file names ending in a pipe symbol in /usr/src. > man ksh The point being made is that '*.[ch]' is what you want. | does not mean "or" in a character class in this situation. > > (b) pipeing to xargs(1) may be faster. > why? Because without it find starts a grep process on every found file individually. Using xargs will execute one grep process on many files at once: find /usr/src -name '*.[ch]' | xargs grep -L 'bpf.h' There is a new + feature in some versions of find that precludes using xargs, but I don't think it's in OpenBSD's find.
Re: PF or BPF
On 2/13/06, Stuart Henderson <[EMAIL PROTECTED]> wrote: > On 2006/02/13 16:53, Jason Crawford wrote: > > On 2/13/06, Matthias Kilian <[EMAIL PROTECTED]> wrote: > > > On Mon, Feb 13, 2006 at 02:03:27PM -0700, Diana Eichert wrote: > > > > find /usr/src -name "*.[c|h]" -exec grep 'bpf.h' /dev/null {} \; > > >^(a) ^(b) > > > > > > (a) I doubt there are any file names ending in a pipe symbol in /usr/src. > > man ksh > > it's in quotes, this is handled by find, not the shell. > > > > (b) pipeing to xargs(1) may be faster. > > why? > > grep foo 1 2 3 4 5 6 7 ... > > vs. > > grep foo 1 > grep foo 2 > grep foo 3 > grep foo 4 > grep foo 5 > grep foo 6 > grep foo 7 One of the nice things about misc is that even if a discussion starts out poorly I still end up learning something. Greg
Re: PF or BPF
On Mon, 13 Feb 2006, Stuart Henderson wrote: > > > (a) I doubt there are any file names ending in a pipe symbol in /usr/src. > > man ksh > > it's in quotes, this is handled by find, not the shell. > > > > (b) pipeing to xargs(1) may be faster. > > why? > > grep foo 1 2 3 4 5 6 7 ... > > vs. > > grep foo 1 > grep foo 2 > grep foo 3 > grep foo 4 > grep foo 5 > grep foo 6 > grep foo 7 > ... Cool, my miscue distracts everyone from Dave, sure glad I could help. diana Past hissy-fits are not a predictor of future hissy-fits. Nick Holland(06 Dec 2005)
Re: PF or BPF
On 2006/02/13 16:53, Jason Crawford wrote: > On 2/13/06, Matthias Kilian <[EMAIL PROTECTED]> wrote: > > On Mon, Feb 13, 2006 at 02:03:27PM -0700, Diana Eichert wrote: > > > find /usr/src -name "*.[c|h]" -exec grep 'bpf.h' /dev/null {} \; > >^(a) ^(b) > > > > (a) I doubt there are any file names ending in a pipe symbol in /usr/src. > man ksh it's in quotes, this is handled by find, not the shell. > > (b) pipeing to xargs(1) may be faster. > why? grep foo 1 2 3 4 5 6 7 ... vs. grep foo 1 grep foo 2 grep foo 3 grep foo 4 grep foo 5 grep foo 6 grep foo 7 ...
Re: PF or BPF
On 2/13/06, Matthias Kilian <[EMAIL PROTECTED]> wrote: > On Mon, Feb 13, 2006 at 02:03:27PM -0700, Diana Eichert wrote: > > find /usr/src -name "*.[c|h]" -exec grep 'bpf.h' /dev/null {} \; >^(a) ^(b) > > (a) I doubt there are any file names ending in a pipe symbol in /usr/src. man ksh > (b) pipeing to xargs(1) may be faster. why? Jason
Re: PF or BPF
On Mon, 13 Feb 2006, Matthias Kilian wrote: > On Mon, Feb 13, 2006 at 02:03:27PM -0700, Diana Eichert wrote: > > find /usr/src -name "*.[c|h]" -exec grep 'bpf.h' /dev/null {} \; >^(a) ^(b) > > (a) I doubt there are any file names ending in a pipe symbol in /usr/src. > (b) pipeing to xargs(1) may be faster. > > (SCNR) > > Ciao, > Kili a. Yep, you're right, I've spent to much time building RegEx's lately. b. I knew there had to be someone to pipe in with xargs. You want to explain xargs to Dave you go for it.
Re: PF or BPF
On Mon, Feb 13, 2006 at 02:03:27PM -0700, Diana Eichert wrote: > find /usr/src -name "*.[c|h]" -exec grep 'bpf.h' /dev/null {} \; ^(a) ^(b) (a) I doubt there are any file names ending in a pipe symbol in /usr/src. (b) pipeing to xargs(1) may be faster. (SCNR) Ciao, Kili
Re: PF or BPF
On 2/13/06, Dave Feustel <[EMAIL PROTECTED]> wrote: > What OpenBSD programs use bpf. Oh c'mon Dave, use the tools that are given to you. find /usr/src -name "*.[c|h]" -exec grep 'bpf.h' /dev/null {} \; will find files that include references to bpf. Your comments re: Ted are sad. I can't believe some of the questions you ask sometimes. diana
Re: PF or BPF
On 2/13/06, Dave Feustel <[EMAIL PROTECTED]> wrote: > On Monday 13 February 2006 14:52, Jason Crawford wrote: > > You cannot learn all there is to know about bpf and how to effectively > > use it in 10 minutes, so you, personally, do NOT need to use bpf at > > all. It's what the other utilities like pf and tcpdump use to do what > > they do. The utilities are nice user friendly wrappers to the bpf > > interfaces, and someone with your experience (lack there of?) should > > probably not be touching bpf directly. bpf is very powerful and very > > useful, but you really need to understand a lot more than what you > > have grasped so far to use bpf effectively. > > Well, one thing is for certain, the caustic responders to this thread aren't > psychic. > > So let's try a r e a l s i m p l e q u e s t i o n : > > What OpenBSD programs use bpf. > > Please don't try to figure out why I am asking the question. > Just answer it or go do something else that won't upset you. You're right, none of the responders are psychic, which is why if you don't include some information, the responses may be inaccurate. Reading the man page (and some unix common sense) will easily answer that for you. 1) you have all the source code 2) the man page says what exact include file bpf has for it's ioctl interface and 3) you can use find and/or grep to search text files. It's really not hard, just try to actually think. While you may get upset about this kind of stuff, I have much better and more important things to worry about. Trust me, nothing on an internet mailing list is that important to me. Jason
Re: PF or BPF
On Mon, 13 Feb 2006, Ted Unangst wrote: > On 2/13/06, Dave Feustel <[EMAIL PROTECTED]> wrote: > > What OpenBSD programs use bpf. > > tcpdump. And there's more: $ cd /usr/src $ grep -lr bpf.h bin sbin usr.bin usr.sbin libexec will give you a nice list. -Otto
Re: PF or BPF
On 2/13/06, Dave Feustel <[EMAIL PROTECTED]> wrote: > > Well, one thing is for certain, the caustic responders to this thread aren't > psychic. > > So let's try a r e a l s i m p l e q u e s t i o n : > > What OpenBSD programs use bpf. > > Please don't try to figure out why I am asking the question. > Just answer it or go do something else that won't upset you. Here's something really simple: try answering questions on your own.
Re: PF or BPF
On Mon, Feb 13, 2006 at 03:29:09PM -0500, Dave Feustel wrote: > So let's try a r e a l s i m p l e q u e s t i o n : > > What OpenBSD programs use bpf. I used this command, Dave: find /usr/src -name "*.c" -exec grep bpf {} /dev/null \; And discovered this list: libpcap dhclient pflogd sysctl dhcpd dhcrelay hostapd mopd pppd pppoe rarpd rbootd tcpdump You could have done this, Dave.
Re: PF or BPF
On 2/13/06, Dave Feustel <[EMAIL PROTECTED]> wrote: > What OpenBSD programs use bpf. tcpdump.
Re: PF or BPF
On Monday 13 February 2006 14:52, Jason Crawford wrote: > You cannot learn all there is to know about bpf and how to effectively > use it in 10 minutes, so you, personally, do NOT need to use bpf at > all. It's what the other utilities like pf and tcpdump use to do what > they do. The utilities are nice user friendly wrappers to the bpf > interfaces, and someone with your experience (lack there of?) should > probably not be touching bpf directly. bpf is very powerful and very > useful, but you really need to understand a lot more than what you > have grasped so far to use bpf effectively. Well, one thing is for certain, the caustic responders to this thread aren't psychic. So let's try a r e a l s i m p l e q u e s t i o n : What OpenBSD programs use bpf. Please don't try to figure out why I am asking the question. Just answer it or go do something else that won't upset you. Thanks, Dave Feustel -- Lose, v., experience a loss, get rid of, "lose the weight" Loose, adj., not tight, let go, free, "loose clothing"
Re: PF or BPF
On 2/13/06, Dave Feustel <[EMAIL PROTECTED]> wrote: > On Monday 13 February 2006 13:51, dereck wrote: > > This is getting ridiculous! The guy said he was under > > attack.(!) What is the point of a _misc_ list anyway? > > He's not clogging the dev list! > > > > The responses here are totally out of line. Haven't > > any of you guys EVER had a desperate situation before? > > Dereck, > > Thanks for the support. However, my situation is not desparate. > By refusing to answer a question to which he indicated he had an > answer, Ted has left all of us hanging as to whether he *really* > knows what the differences are between the capabilities of pf and bpf. > *I* could certainly not testify that Ted actually knows the answer to > that question as he claims to. :-) If he can code rthreads, I think it's pretty safe to say he understands the differences between bpf and pf, those seem like some really inflammatory remarks to me. If you bother to take some time to read the manuals instead of expecting to be spoon fed the information on the mailing list, then you'll learn a lot more, as well as not get flamed by others on the list. Ted has much better things to do (like make rthreads kick even more ass) than to answer silly questions by a user who is too lazy to read. > > (BTW, I had read the bpf man page and, frankly, I couldn't make > any sense out of it on first reading. I started getting a better idea > of bpf by the time I started reading the freebsd bpf man page, > but then I started wondering "why bother with bpf? How do I > even use it?". It must have a useful purpose or it wouldn't be in OpenBSD.) You cannot learn all there is to know about bpf and how to effectively use it in 10 minutes, so you, personally, do NOT need to use bpf at all. It's what the other utilities like pf and tcpdump use to do what they do. The utilities are nice user friendly wrappers to the bpf interfaces, and someone with your experience (lack there of?) should probably not be touching bpf directly. bpf is very powerful and very useful, but you really need to understand a lot more than what you have grasped so far to use bpf effectively. Jason
Re: PF or BPF
This is getting ridiculous! The guy said he was under attack.(!) What is the point of a _misc_ list anyway? He's not clogging the dev list! The responses here are totally out of line. Haven't any of you guys EVER had a desperate situation before? Sheesh. --- Ted Unangst <[EMAIL PROTECTED]> wrote: > On 2/13/06, Dave Feustel <[EMAIL PROTECTED]> > wrote: > > On Monday 13 February 2006 12:45, Ted Unangst > wrote: > > > On 2/13/06, Dave Feustel > <[EMAIL PROTECTED]> wrote: > > > > What can BPF do that PF can not? > > > > > > different things. > > > > OK, I'll bite. Such as? > > no, if you can't read a man page, you aren't > qualified to read my emails either.
Re: PF or BPF
On 2/13/06, Dave Feustel <[EMAIL PROTECTED]> wrote: > > Dereck, > > Thanks for the support. However, my situation is not desparate. > By refusing to answer a question to which he indicated he had an > answer, Ted has left all of us hanging as to whether he *really* > knows what the differences are between the capabilities of pf and bpf. You're a complete tool. If I said what Ted said then that would be possible. I think everyone here realizes that Ted knows the differences. > *I* could certainly not testify that Ted actually knows the answer to > that question as he claims to. :-) > > (BTW, I had read the bpf man page and, frankly, I couldn't make > any sense out of it on first reading. Did it sound anything like pf? Did you look anywhere other than the man page before posting your question? Greg
Re: PF or BPF
On Monday 13 February 2006 13:51, dereck wrote: > This is getting ridiculous! The guy said he was under > attack.(!) What is the point of a _misc_ list anyway? > He's not clogging the dev list! > > The responses here are totally out of line. Haven't > any of you guys EVER had a desperate situation before? Dereck, Thanks for the support. However, my situation is not desparate. By refusing to answer a question to which he indicated he had an answer, Ted has left all of us hanging as to whether he *really* knows what the differences are between the capabilities of pf and bpf. *I* could certainly not testify that Ted actually knows the answer to that question as he claims to. :-) (BTW, I had read the bpf man page and, frankly, I couldn't make any sense out of it on first reading. I started getting a better idea of bpf by the time I started reading the freebsd bpf man page, but then I started wondering "why bother with bpf? How do I even use it?". It must have a useful purpose or it wouldn't be in OpenBSD.) Maybe someone else can jump in here. Dave -- Lose, v., experience a loss, get rid of, "lose the weight" Loose, adj., not tight, let go, free, "loose clothing"
Re: PF or BPF
On 2006/02/13 13:00, Dave Feustel wrote: > On Monday 13 February 2006 12:45, Ted Unangst wrote: > > On 2/13/06, Dave Feustel <[EMAIL PROTECTED]> wrote: > > > What can BPF do that PF can not? > > > > different things. > > OK, I'll bite. Such as? > (this might be a loong, drawnout thread, but I've got time :-)) PF is for firewalling/natting etc. BPF is for reading and injecting packets on a network interface (used by tcpdump etc, amongst others). I'm sure there must be some books suitable for learning this type of thing.
Re: PF or BPF
Dave Feustel wrote: What can BPF do that PF can not? Thanks, Dave Feustel One is a packet sniffer, one is a firewall. However, you are not qualified to operate such tools. Please disconnect your keyboard from your PC.
Re: PF or BPF
On 2/13/06, Dave Feustel <[EMAIL PROTECTED]> wrote: > On Monday 13 February 2006 12:45, Ted Unangst wrote: > > On 2/13/06, Dave Feustel <[EMAIL PROTECTED]> wrote: > > > What can BPF do that PF can not? > > > > different things. > > OK, I'll bite. Such as? > (this might be a loong, drawnout thread, but I've got time :-)) man 4 bpf The Berkeley Packet Filter provides a raw interface to data link layers in a protocol-independent fashion. man 8 pfctl Packet filtering restricts the types of packets that pass through network interfaces entering or leaving the host based on filter rules as de- scribed in pf.conf(5). There, thread over. -- Jon Simola Systems Administrator ABC Communications
Re: PF or BPF
On 2/13/06, Dave Feustel <[EMAIL PROTECTED]> wrote: > On Monday 13 February 2006 12:45, Ted Unangst wrote: > > On 2/13/06, Dave Feustel <[EMAIL PROTECTED]> wrote: > > > What can BPF do that PF can not? > > > > different things. > > OK, I'll bite. Such as? no, if you can't read a man page, you aren't qualified to read my emails either.
Re: PF or BPF
On 2/13/06, Dave Feustel <[EMAIL PROTECTED]> wrote: > What can BPF do that PF can not? > Your questions keep getting better and better. Just curious as to whether you've heard of Google? 1. Make an /etc/bpf.conf and see what happens. Oh, wait, I don't see a reference to a config file in man bpf. Hmmm, maybe bpf doesn't do anything related to pf. or 2. Google for pf vs bpf and see that they can work together but do very dissimilar duties. Greg
Re: PF or BPF
On Monday 13 February 2006 12:45, Ted Unangst wrote: > On 2/13/06, Dave Feustel <[EMAIL PROTECTED]> wrote: > > What can BPF do that PF can not? > > different things. OK, I'll bite. Such as? (this might be a loong, drawnout thread, but I've got time :-)) -- Lose, v., experience a loss, get rid of, "lose the weight" Loose, adj., not tight, let go, free, "loose clothing"
Re: PF or BPF
On 2/13/06, Dave Feustel <[EMAIL PROTECTED]> wrote: > What can BPF do that PF can not? different things.
PF or BPF
What can BPF do that PF can not? Thanks, Dave Feustel -- Lose, v., experience a loss, get rid of, "lose the weight" Loose, adj., not tight, let go, free, "loose clothing"