Re: issue with pf syntax parser
On Wed, Sep 9, 2015 at 4:21 AM, Joseph A Borg wrote: > so you can have something like this, which is confusing to me but it might be > readable to somebody else: > > passin on $DMZ_if > \ > inet proto { > tcp udp > } > \ > to port $keep_tcp_out Yeah, that's a good example of something which should probably be just one line without redundant whitespace. Thanks, -- Raul
Re: issue with pf syntax parser
just tested this. how nice. makes the configuration file infinitely more readable without having to hassle over extra whitespace (which is not meaningful). just for the record, it seems, in lists, the parser treats newline as a space so you can have something like this, which is confusing to me but it might be readable to somebody else: passin on $DMZ_if \ inet proto { tcp udp } \ to port $keep_tcp_out > On 05 Sep 2015, at 18:48, Benny Lofgren wrote: > > On 2015-09-04 14:30, Joseph A Borg wrote: >> I have something like this in pf.conf: >> >> services = "{ >> ssh, >> \ >> http, https, 8000, >> 8080, 8088, \ >> smtp, pop3, pop3s, >> imap, imaps, \ >> submission, 465, >> \ >> domain, ntp >> \ >> }" >> >> if there’s white space after the back slash the parser barfs by not creating >> the macro and then raising an error when it’s first used. >> >> I would assume this to be an inconvenience for the user as it’s not always >> possible to check whitespace after the backslash > > > Everyone who commented so far seem to have missed the obvious - you > don't NEED to escape the newline in this case. The parser handles this > case just fine without them: > > paddan:/etc# cat /tmp/tstpf.conf > services = "{ >ssh, >http, https, 8000, 8080, 8088, >smtp, pop3, pop3s, imap, imaps, >submission, 465, >domain, ntp >}" > > block in proto tcp from any to any port $services > > paddan:/etc# pfctl -f /tmp/tstpf.conf > paddan:/etc# pfctl -s rules > block drop in proto tcp from any to any port = 22 > block drop in proto tcp from any to any port = 80 > block drop in proto tcp from any to any port = 443 > block drop in proto tcp from any to any port = 8000 > block drop in proto tcp from any to any port = 8080 > block drop in proto tcp from any to any port = 8088 > block drop in proto tcp from any to any port = 25 > block drop in proto tcp from any to any port = 110 > block drop in proto tcp from any to any port = 995 > block drop in proto tcp from any to any port = 143 > block drop in proto tcp from any to any port = 993 > block drop in proto tcp from any to any port = 587 > block drop in proto tcp from any to any port = 465 > block drop in proto tcp from any to any port = 53 > block drop in proto tcp from any to any port = 123 > paddan:/etc# _ > > > Regards, > /Benny
Re: issue with pf syntax parser
On 2015-09-04 14:30, Joseph A Borg wrote: > I have something like this in pf.conf: > > services = "{ > ssh, > \ > http, https, 8000, > 8080, 8088, \ > smtp, pop3, pop3s, > imap, imaps, \ > submission, 465, > \ > domain, ntp > \ > }" > > if there’s white space after the back slash the parser barfs by not creating > the macro and then raising an error when it’s first used. > > I would assume this to be an inconvenience for the user as it’s not always > possible to check whitespace after the backslash Everyone who commented so far seem to have missed the obvious - you don't NEED to escape the newline in this case. The parser handles this case just fine without them: paddan:/etc# cat /tmp/tstpf.conf services = "{ ssh, http, https, 8000, 8080, 8088, smtp, pop3, pop3s, imap, imaps, submission, 465, domain, ntp }" block in proto tcp from any to any port $services paddan:/etc# pfctl -f /tmp/tstpf.conf paddan:/etc# pfctl -s rules block drop in proto tcp from any to any port = 22 block drop in proto tcp from any to any port = 80 block drop in proto tcp from any to any port = 443 block drop in proto tcp from any to any port = 8000 block drop in proto tcp from any to any port = 8080 block drop in proto tcp from any to any port = 8088 block drop in proto tcp from any to any port = 25 block drop in proto tcp from any to any port = 110 block drop in proto tcp from any to any port = 995 block drop in proto tcp from any to any port = 143 block drop in proto tcp from any to any port = 993 block drop in proto tcp from any to any port = 587 block drop in proto tcp from any to any port = 465 block drop in proto tcp from any to any port = 53 block drop in proto tcp from any to any port = 123 paddan:/etc# _ Regards, /Benny
Re: issue with pf syntax parser
On Fri, Sep 04, 2015 at 07:43:35PM +0200, Joseph A Borg wrote: > this is all very fascinating. Is it possible to contemplate a pre-filter that > chomps out trailing whitespace and comments? Would this overly complicate the > parsing process and introduce security issues? Nah... you'll loose all line number info. And it is ugly, -Otto > > I???m asking because this might improve readability, usability and security > for less gifted users like me??? > > > > On 04 Sep 2015, at 19:37, Otto Moerbeek wrote: > > > > On Fri, Sep 04, 2015 at 05:51:54PM +0300, Kimmo Paasiala wrote: > > > >> On Fri, Sep 4, 2015 at 4:02 PM, Joseph A Borg wrote: > >>> maybe the syntax error should point to the line where there are extra > >>> characters after the escape? > >>> > >>> > >> > >> That would require making the backslash a lexical token in the > >> pf.conf(5) syntax. Now it's just a simple escape character that gets > >> eaten and forgotten by the lexical analyzer that splits the input into > >> tokens to be parsed by the syntax parser. > >> > >> -Kimmo > > > > It's a historical accident that the pf grammar is line oriented. I > > once started a litttle project to modify the pf yacc grammer to treat > > end-of-lines as regular whitespace and came a long way, but something > > got in between and the diff got lost > > > > -Otto
Re: issue with pf syntax parser
On Fri, Sep 04, 2015 at 05:49:34PM -0600, Theo de Raadt wrote: > > > --- /usr/share/man/man5/pf.conf.5 Wed Mar 5 16:22:58 2014 > > > +++ var1/man5/pf.conf.5 Thu Sep 3 16:19:21 2015 > > > @@ -63,7 +63,7 @@ A method for detecting a host's operating system. > > > Some example rulesets. > > > .El > > > .Pp > > > -The current line can be extended over multiple lines using a backslash > > > +The current line can be extended over multiple lines using a > > > backslash-newline > > > .Pq Sq \e . > > > Comments can be put anywhere in the file using a hash mark > > > .Pq Sq # , > > I would recommend: > > The current line can be extended over multiple lines using a backslash > > without trailing white space. > > > > I agree with later posts that nothing needs to be done to the parser. I > > have experienced the same error and wished it would give me the line > > number, but somehow I managed to find the mistake. > > The sh man page, for instance, does not mention the problem with spaces > either: > > A backslash in the input line causes the shell to prompt for further > input. > > Yet millions of people have gotten by with reading and writing shell > scripts. > > The same situation in for C. > > So maybe we should wait until the exact same "improvement proposals" > arrive for the shell, the C language, and all the other programs that > behave in the same way? > > Or the OP should admit they simply don't know unix. Yes, unix is > something you have to learn. There is not different from anything > else in life. I think it is pretty outrageous to suggest a few extra > words (term I use is "over-documentation") would have avoided this grief > in the first place. That is rewriting history; I am certain the OP > only is complaining after the fact. > > This is not OpenBSD folk being mean. Very few people learn to ride a > bicicle without a scrape or two. Stop whining, and get back on the > bike. > > The documentation is sufficient. > > Yep, and all I have in learning OpenBSD is scrapes and scratches, and bloody bruises. But, it is the way to learn!, besides like mentioned earlier (estimate 99.9+) of BSD users are already used to the parser, or the way it behaves in other nix process's. No change needed would be my 2 cents. :)
Re: issue with pf syntax parser
> > --- /usr/share/man/man5/pf.conf.5 Wed Mar 5 16:22:58 2014 > > +++ var1/man5/pf.conf.5 Thu Sep 3 16:19:21 2015 > > @@ -63,7 +63,7 @@ A method for detecting a host's operating system. > > Some example rulesets. > > .El > > .Pp > > -The current line can be extended over multiple lines using a backslash > > +The current line can be extended over multiple lines using a > > backslash-newline > > .Pq Sq \e . > > Comments can be put anywhere in the file using a hash mark > > .Pq Sq # , > I would recommend: > The current line can be extended over multiple lines using a backslash > without trailing white space. > > I agree with later posts that nothing needs to be done to the parser. I > have experienced the same error and wished it would give me the line > number, but somehow I managed to find the mistake. The sh man page, for instance, does not mention the problem with spaces either: A backslash in the input line causes the shell to prompt for further input. Yet millions of people have gotten by with reading and writing shell scripts. The same situation in for C. So maybe we should wait until the exact same "improvement proposals" arrive for the shell, the C language, and all the other programs that behave in the same way? Or the OP should admit they simply don't know unix. Yes, unix is something you have to learn. There is not different from anything else in life. I think it is pretty outrageous to suggest a few extra words (term I use is "over-documentation") would have avoided this grief in the first place. That is rewriting history; I am certain the OP only is complaining after the fact. This is not OpenBSD folk being mean. Very few people learn to ride a bicicle without a scrape or two. Stop whining, and get back on the bike. The documentation is sufficient.
Re: issue with pf syntax parser
On 09/04/15 09:26, Raul Miller wrote: As a general rule, whenever a syntax error is not obvious, the real error happened before that point... But looking at my copy of `man pf.conf`, I don't see any mention of backslash in the section on Grammar. The only sentence using the word backslash is the one at the begining which states "The current line can be extended over multiple lines using a backslash (`\')." though reading down through example uses of backslash it is also used with \n presumably to denote a newline character for printf - all of which suggests that pf.conf inherits backslash behavior from the C compiler tradition. And, indeed, reading `man cc` and searching specifically for the word "backslash" I get a more thorough treatment of the rules which are probably relevant for pf.conf (or at least it explictly mentions "backslash-newline" - it doesn't mention the \n convention either - for that, I guess I'd go with `man printf`). Thinking it over, perhaps `man pf.conf` should have that introductory line changed to read "The current line can be extended over multiple lines using a backslash-newline (`\')." Or, for real emphasis, the parenthetical comment could be (`\' at the end of the line)." In other words, for /usr/share/man/man5/pf.conf.5 either: --- /usr/share/man/man5/pf.conf.5 Wed Mar 5 16:22:58 2014 +++ var1/man5/pf.conf.5 Thu Sep 3 16:19:21 2015 @@ -63,7 +63,7 @@ A method for detecting a host's operating system. Some example rulesets. .El .Pp -The current line can be extended over multiple lines using a backslash +The current line can be extended over multiple lines using a backslash-newline .Pq Sq \e . Comments can be put anywhere in the file using a hash mark .Pq Sq # , I would recommend: The current line can be extended over multiple lines using a backslash without trailing white space. I agree with later posts that nothing needs to be done to the parser. I have experienced the same error and wished it would give me the line number, but somehow I managed to find the mistake. or --- /usr/share/man/man5/pf.conf.5 Wed Mar 5 16:22:58 2014 +++ var2/man5/pf.conf.5 Thu Sep 3 16:33:39 2015 @@ -63,8 +63,8 @@ A method for detecting a host's operating system. Some example rulesets. .El .Pp -The current line can be extended over multiple lines using a backslash -.Pq Sq \e . +The current line can be extended over multiple lines using a backslash-newline +.Pq So \e Sc at the end of the line . Comments can be put anywhere in the file using a hash mark .Pq Sq # , and extend to the end of the current line. Or I guess you could use a real newline, something like this: --- /usr/share/man/man5/pf.conf.5 Wed Mar 5 16:22:58 2014 +++ var3/man5/pf.conf.5 Thu Sep 3 16:36:27 2015 @@ -63,8 +63,10 @@ A method for detecting a host's operating system. Some example rulesets. .El .Pp -The current line can be extended over multiple lines using a backslash -.Pq Sq \e . +The current line can be extended over multiple lines using a backslash-newline +.Po So \e +.br +.Sc Pc . Comments can be put anywhere in the file using a hash mark .Pq Sq # , and extend to the end of the current line. (If you have set up the edited file like I have in my diffs, you can see that last modified man page using man -M `pwd`/var3 pf.conf). It's a bit of an odd concept, though, so maybe it is worth documenting. Thanks,
Re: issue with pf syntax parser
On 09/04/15 19:43, Joseph A Borg wrote: > this is all very fascinating. Is it possible to contemplate a pre-filter that > chomps out trailing whitespace and comments? Would this overly complicate the > parsing process and introduce security issues? Backslash at end of line meaning join with next line is very *nix, as is unintentional trailing whitespace fscking it up. Changing this will add confusion to the big masses and likely hurt more people than it will help. Don't expect anyone to pick this idea up. /Alexander > > I’m asking because this might improve readability, usability and security for > less gifted users like me… > > >> On 04 Sep 2015, at 19:37, Otto Moerbeek wrote: >> >> On Fri, Sep 04, 2015 at 05:51:54PM +0300, Kimmo Paasiala wrote: >> >>> On Fri, Sep 4, 2015 at 4:02 PM, Joseph A Borg wrote: maybe the syntax error should point to the line where there are extra characters after the escape? >>> >>> That would require making the backslash a lexical token in the >>> pf.conf(5) syntax. Now it's just a simple escape character that gets >>> eaten and forgotten by the lexical analyzer that splits the input into >>> tokens to be parsed by the syntax parser. >>> >>> -Kimmo >> >> It's a historical accident that the pf grammar is line oriented. I >> once started a litttle project to modify the pf yacc grammer to treat >> end-of-lines as regular whitespace and came a long way, but something >> got in between and the diff got lost >> >> -Otto
Re: issue with pf syntax parser
On Fri, Sep 4, 2015 at 1:43 PM, Joseph A Borg wrote: > I’m asking because this might improve readability, usability and security for > less gifted users like me… Or it might not? Seriously, how could we even tell the difference? I mean, I get as frustrated as anyone when I'm trying to read documentation and the issue I want to read about is not documented where I expected it to be. But it's even more frustrating when something gets broken to "fix" someone else's comprehension problems. Good documentation, good explanations, and maybe a little experimentation are good fixes for comprehension problems. And sometimes with a lot of work good documentation can be woven into the user interface or application interface. But adding a whole new layer of complexity doesn't really improve anyone's comprehension - even if it is a lot of fun to do. Here, what you've got is a whole bunch of systems (not just pf.conf) where \ at the end of a line includes the next line in the current line. That's kind of an awkward thing, but honestly so is anything other multi-line mechanism. But another approach is to just use really long lines (also awkward, but easily comprehended.) Thanks, -- Raul
Re: issue with pf syntax parser
this is all very fascinating. Is it possible to contemplate a pre-filter that chomps out trailing whitespace and comments? Would this overly complicate the parsing process and introduce security issues? I’m asking because this might improve readability, usability and security for less gifted users like me… > On 04 Sep 2015, at 19:37, Otto Moerbeek wrote: > > On Fri, Sep 04, 2015 at 05:51:54PM +0300, Kimmo Paasiala wrote: > >> On Fri, Sep 4, 2015 at 4:02 PM, Joseph A Borg wrote: >>> maybe the syntax error should point to the line where there are extra >>> characters after the escape? >>> >>> >> >> That would require making the backslash a lexical token in the >> pf.conf(5) syntax. Now it's just a simple escape character that gets >> eaten and forgotten by the lexical analyzer that splits the input into >> tokens to be parsed by the syntax parser. >> >> -Kimmo > > It's a historical accident that the pf grammar is line oriented. I > once started a litttle project to modify the pf yacc grammer to treat > end-of-lines as regular whitespace and came a long way, but something > got in between and the diff got lost > > -Otto
Re: issue with pf syntax parser
On Fri, Sep 04, 2015 at 05:51:54PM +0300, Kimmo Paasiala wrote: > On Fri, Sep 4, 2015 at 4:02 PM, Joseph A Borg wrote: > > maybe the syntax error should point to the line where there are extra > > characters after the escape? > > > > > > That would require making the backslash a lexical token in the > pf.conf(5) syntax. Now it's just a simple escape character that gets > eaten and forgotten by the lexical analyzer that splits the input into > tokens to be parsed by the syntax parser. > > -Kimmo It's a historical accident that the pf grammar is line oriented. I once started a litttle project to modify the pf yacc grammer to treat end-of-lines as regular whitespace and came a long way, but something got in between and the diff got lost -Otto
Re: issue with pf syntax parser
On Fri, Sep 4, 2015 at 4:02 PM, Joseph A Borg wrote: > maybe the syntax error should point to the line where there are extra > characters after the escape? > > That would require making the backslash a lexical token in the pf.conf(5) syntax. Now it's just a simple escape character that gets eaten and forgotten by the lexical analyzer that splits the input into tokens to be parsed by the syntax parser. -Kimmo
Re: issue with pf syntax parser
As a general rule, whenever a syntax error is not obvious, the real error happened before that point... But looking at my copy of `man pf.conf`, I don't see any mention of backslash in the section on Grammar. The only sentence using the word backslash is the one at the begining which states "The current line can be extended over multiple lines using a backslash (`\')." though reading down through example uses of backslash it is also used with \n presumably to denote a newline character for printf - all of which suggests that pf.conf inherits backslash behavior from the C compiler tradition. And, indeed, reading `man cc` and searching specifically for the word "backslash" I get a more thorough treatment of the rules which are probably relevant for pf.conf (or at least it explictly mentions "backslash-newline" - it doesn't mention the \n convention either - for that, I guess I'd go with `man printf`). Thinking it over, perhaps `man pf.conf` should have that introductory line changed to read "The current line can be extended over multiple lines using a backslash-newline (`\')." Or, for real emphasis, the parenthetical comment could be (`\' at the end of the line)." In other words, for /usr/share/man/man5/pf.conf.5 either: --- /usr/share/man/man5/pf.conf.5 Wed Mar 5 16:22:58 2014 +++ var1/man5/pf.conf.5 Thu Sep 3 16:19:21 2015 @@ -63,7 +63,7 @@ A method for detecting a host's operating system. Some example rulesets. .El .Pp -The current line can be extended over multiple lines using a backslash +The current line can be extended over multiple lines using a backslash-newline .Pq Sq \e . Comments can be put anywhere in the file using a hash mark .Pq Sq # , or --- /usr/share/man/man5/pf.conf.5 Wed Mar 5 16:22:58 2014 +++ var2/man5/pf.conf.5 Thu Sep 3 16:33:39 2015 @@ -63,8 +63,8 @@ A method for detecting a host's operating system. Some example rulesets. .El .Pp -The current line can be extended over multiple lines using a backslash -.Pq Sq \e . +The current line can be extended over multiple lines using a backslash-newline +.Pq So \e Sc at the end of the line . Comments can be put anywhere in the file using a hash mark .Pq Sq # , and extend to the end of the current line. Or I guess you could use a real newline, something like this: --- /usr/share/man/man5/pf.conf.5 Wed Mar 5 16:22:58 2014 +++ var3/man5/pf.conf.5 Thu Sep 3 16:36:27 2015 @@ -63,8 +63,10 @@ A method for detecting a host's operating system. Some example rulesets. .El .Pp -The current line can be extended over multiple lines using a backslash -.Pq Sq \e . +The current line can be extended over multiple lines using a backslash-newline +.Po So \e +.br +.Sc Pc . Comments can be put anywhere in the file using a hash mark .Pq Sq # , and extend to the end of the current line. (If you have set up the edited file like I have in my diffs, you can see that last modified man page using man -M `pwd`/var3 pf.conf). It's a bit of an odd concept, though, so maybe it is worth documenting. Thanks, -- Raul On Fri, Sep 4, 2015 at 9:02 AM, Joseph A Borg wrote: > maybe the syntax error should point to the line where there are extra > characters after the escape? > > >> On 04 Sep 2015, at 14:53, Peter Hessler wrote: >> >> Backslash says "ignore the following charecter". You are using it to >> ignore the newline. >> >> If you ignore the space instead, the newline then matters. >> >> This is not a bug, this is 100% by design. >> >> You'll need to ensure there are no trailing spaces after a backslash >> (and we do recommend removing trailing spaces in general, but those are >> not as damaging) >> >> >> On 2015 Sep 04 (Fri) at 14:30:05 +0200 (+0200), Joseph A Borg wrote: >> :I have something like this in pf.conf: >> : >> :services = "{ >> : ssh, >> \ >> : http, https, 8000, >> 8080, 8088, \ >> : smtp, pop3, pop3s, >> imap, imaps, \ >> : submission, 465, >> \ >> : domain, ntp >> \ >> : }" >> : >> :if there???s white space after the back slash the parser barfs by not >> creating the macro and then raising an error when it???s first used. >> : >> :I would assume this to be an inconvenience for the user as it???s not >> always possible to check whitespace after the backslash >> : >> :regards >> : >> >> -- >> "First things first -- but not necessarily in that order" >> -- The Doctor, "Doctor Who"
Re: issue with pf syntax parser
maybe the syntax error should point to the line where there are extra characters after the escape? > On 04 Sep 2015, at 14:53, Peter Hessler wrote: > > Backslash says "ignore the following charecter". You are using it to > ignore the newline. > > If you ignore the space instead, the newline then matters. > > This is not a bug, this is 100% by design. > > You'll need to ensure there are no trailing spaces after a backslash > (and we do recommend removing trailing spaces in general, but those are > not as damaging) > > > On 2015 Sep 04 (Fri) at 14:30:05 +0200 (+0200), Joseph A Borg wrote: > :I have something like this in pf.conf: > : > :services = "{ > : ssh, > \ > : http, https, 8000, > 8080, 8088, \ > : smtp, pop3, pop3s, > imap, imaps, \ > : submission, 465, > \ > : domain, ntp > \ > : }" > : > :if there???s white space after the back slash the parser barfs by not > creating the macro and then raising an error when it???s first used. > : > :I would assume this to be an inconvenience for the user as it???s not always > possible to check whitespace after the backslash > : > :regards > : > > -- > "First things first -- but not necessarily in that order" > -- The Doctor, "Doctor Who"
Re: issue with pf syntax parser
ok got it. Treat it as an escape character. At least a note somewhere might help the nincompoops like me. is it possible to make it more resilient though? maybe use a different escape character that would ignore all trailing whitespace? heck! having a comment after the escape would be great for inline documentation would this count as unnecessary feature creep? > On 04 Sep 2015, at 14:53, Peter Hessler wrote: > > Backslash says "ignore the following charecter". You are using it to > ignore the newline. > > If you ignore the space instead, the newline then matters. > > This is not a bug, this is 100% by design. > > You'll need to ensure there are no trailing spaces after a backslash > (and we do recommend removing trailing spaces in general, but those are > not as damaging) > > > On 2015 Sep 04 (Fri) at 14:30:05 +0200 (+0200), Joseph A Borg wrote: > :I have something like this in pf.conf: > : > :services = "{ > : ssh, > \ > : http, https, 8000, > 8080, 8088, \ > : smtp, pop3, pop3s, > imap, imaps, \ > : submission, 465, > \ > : domain, ntp > \ > : }" > : > :if there???s white space after the back slash the parser barfs by not > creating the macro and then raising an error when it???s first used. > : > :I would assume this to be an inconvenience for the user as it???s not always > possible to check whitespace after the backslash > : > :regards > : > > -- > "First things first -- but not necessarily in that order" > -- The Doctor, "Doctor Who"
Re: issue with pf syntax parser
Backslash says "ignore the following charecter". You are using it to ignore the newline. If you ignore the space instead, the newline then matters. This is not a bug, this is 100% by design. You'll need to ensure there are no trailing spaces after a backslash (and we do recommend removing trailing spaces in general, but those are not as damaging) On 2015 Sep 04 (Fri) at 14:30:05 +0200 (+0200), Joseph A Borg wrote: :I have something like this in pf.conf: : :services = "{ : ssh, \ : http, https, 8000, 8080, 8088, \ : smtp, pop3, pop3s, imap, imaps, \ : submission, 465, \ : domain, ntp \ : }" : :if there???s white space after the back slash the parser barfs by not creating the macro and then raising an error when it???s first used. : :I would assume this to be an inconvenience for the user as it???s not always possible to check whitespace after the backslash : :regards : -- "First things first -- but not necessarily in that order" -- The Doctor, "Doctor Who"
issue with pf syntax parser
I have something like this in pf.conf: services= "{ ssh, \ http, https, 8000, 8080, 8088, \ smtp, pop3, pop3s, imap, imaps, \ submission, 465, \ domain, ntp \ }" if there’s white space after the back slash the parser barfs by not creating the macro and then raising an error when it’s first used. I would assume this to be an inconvenience for the user as it’s not always possible to check whitespace after the backslash regards