Recursive macro expansion problems
I am running into some trouble with recursive macro expansion, here is the pf.conf that shows what I am seeing. --- pf.conf --- # I am having some trouble with pfctl and recursive macros. I was # hoping to use nested lists, and that is where I noticed these issues. # I sure wish nested lists worked. hosts = '"192.168.1.0/24" "192.168.2.0/24"' ports = "http https" host_list = "{" $hosts "}" port_list = "{" $ports "}" single_host = "192.168.1.1" single_port = "http" # A macro that is nested but does not contain {} works end_01 = "proto tcp from {" $hosts "} to any port {" $ports "}" pass $end_01 # And rules containg macros that have {} work pass proto tcp from $host_list to any port $port_list # This is to show that rules work with unquoted macros at the end # as well as the same syntax as the next rule that fails end_02 = "proto tcp from " $single_host " to any port " $single_port pass $end_02 # If a macro is recursive and contains {} it doesn't work end_03 = "proto tcp from " $host_list " to any port " $port_list pass $end_03 --- pf.conf --- I am not sure if this is expected or if I am quoting incorrectly or if something is actually broken. I did read through the archives and see some suggestions on quoting CIDR notation, but I didn't see anything about {} in recursive macros. Does anyone have any idea why the end_03 macro gives a "syntax error"? l8rZ, -- andrew - ICQ# 253198 - JID: [EMAIL PROTECTED] BOFH excuse of the day: global warming
Recursive macro expansion problems
Hello, I am trying to reproduce the example below, given in the FAQ: host1 = "192.168.1.1" host2 = "192.168.1.2" all_hosts = "{" $host1 $host2 "}" This sample works with a host type macro but network type macro, like following lines, do not work. int_lan = "172.16.1.0/24" rel_int_lan = "172.16.2.0/24" rel_ext_lan = "172.16.3.0/24" lans = "{" $int_lan $rel_int_lan $rel_ext_lan "}" I obtain this error: # pfctl -nf /etc/pf.conf /etc/pf.conf:48: syntax error /etc/pf.conf:83: macro 'lans' not defined Could you give me the correct syntax? Thanks Xavier
Re: Recursive macro expansion problems
On Tue, May 23, 2006 at 03:31:46PM -0700, andrew fresh wrote: > host_list = "{" $hosts "}" > port_list = "{" $ports "}" Try adding q_host_list = '"{' $hosts '}"' q_port_list = '"{' $ports '}"' then replace > end_03 = "proto tcp from " $host_list " to any port " $port_list with end_03 = "proto tcp from " $q_host_list " to any port " $q_port_list The rule is that when a macro is used to define another macro, it should contain quotes (as the right-hand-side of a macro definition is a concatenation of strings), while a macro used in a rule definition should not. Like pass from "{ 10.1.2.3 10.2.3.4 }" to any is not a host list, but a single string, interpreted as a host name. And macro = { 10.1.2.3 10.2.3.4 } is not a valid macro definition, because the right-hand-side is not a string (or a sequence thereof), but interpreted as tokens. It's neither like shell variable expansion, nor like C #defines. But something else entirely. I keep hearing it's supposed to be like that :) Daniel
Re: Recursive macro expansion problems
On Wed, May 24, 2006 at 01:55:34AM +0200, Daniel Hartmeier wrote: > On Tue, May 23, 2006 at 03:31:46PM -0700, andrew fresh wrote: > > > host_list = "{" $hosts "}" > > port_list = "{" $ports "}" > > Try adding > > q_host_list = '"{' $hosts '}"' > q_port_list = '"{' $ports '}"' > > then replace > > > end_03 = "proto tcp from " $host_list " to any port " $port_list > > with > > end_03 = "proto tcp from " $q_host_list " to any port " $q_port_list That does work. Thank you! Having two different macros depending on if I am using them in another macro or in a rule is not as handy as I had hoped, but it works. I will probably end up putting the {} in the secondary macro (as in end_01 in my original mail). I really wish I could do something like: macro = "{ item_1 item_2 }" and that would allow me to do pass ... { item_3 $macro } and pass ... $macro in the same ruleset and it would just work. Oh well, maybe someday :-) PF is great anyway. I am sure I will learn all the quirks I need eventually. l8rZ, -- andrew - ICQ# 253198 - JID: [EMAIL PROTECTED] Hey! It compiles! Ship it!
Re: Recursive macro expansion problems
On 5/24/06, Daniel Hartmeier <[EMAIL PROTECTED]> wrote: On Tue, May 23, 2006 at 03:31:46PM -0700, andrew fresh wrote: > host_list = "{" $hosts "}" > port_list = "{" $ports "}" Try adding q_host_list = '"{' $hosts '}"' q_port_list = '"{' $ports '}"' Thankyou s much for the clarification Danny :-) The PF FAQ at http://www.openbsd.org/faq/pf/macros.html shows double quotes inside {} -- Macros can be defined recursively. Since macros are not expanded within quotes the following syntax must be used: host1 = "192.168.1.1" host2 = "192.168.1.2" all_hosts = "{" $host1 $host2 "}" is that an error ? Thankyou so mcuh Kind Regards Siju
Re: Recursive macro expansion problems
On Fri, May 26, 2006 at 11:09:51AM +0530, Siju George wrote: >host1 = "192.168.1.1" >host2 = "192.168.1.2" >all_hosts = "{" $host1 $host2 "}" > > > is that an error ? No, it's correct if you want to use $all_hosts in a rule like pass from $all_hosts to any But it won't work as right-hand-side of another macro definition, like from_all_hosts = "from " $all_hosts " to any" pass $from_all_hosts Daniel
Re: Recursive macro expansion problems
Not a pf expert, but I think you may want to look at tables instead. kr, Jan On Fri, Jan 19, 2007 at 07:20:11PM +0100, Xavier ROUX wrote: > Hello, > > I am trying to reproduce the example below, given in the FAQ: > > host1 = "192.168.1.1" > host2 = "192.168.1.2" > all_hosts = "{" $host1 $host2 "}" > > This sample works with a host type macro but network type macro, like > following lines, do not work. > > int_lan = "172.16.1.0/24" > rel_int_lan = "172.16.2.0/24" > rel_ext_lan = "172.16.3.0/24" > > lans = "{" $int_lan $rel_int_lan $rel_ext_lan "}" > > I obtain this error: > > # pfctl -nf /etc/pf.conf > /etc/pf.conf:48: syntax error > /etc/pf.conf:83: macro 'lans' not defined > > Could you give me the correct syntax? > > Thanks > Xavier -- UNIX isn't dead - It just smells funny [EMAIL PROTECTED] "Baldric, you wouldn't recognize a subtle plan if it painted itself pur- ple and danced naked on a harpsicord singing 'subtle plans are here a- gain'."
Re: Recursive macro expansion problems
On Fri, Jan 19, 2007 at 07:20:11PM +0100, Xavier ROUX wrote: > int_lan = "172.16.1.0/24" > rel_int_lan = "172.16.2.0/24" > rel_ext_lan = "172.16.3.0/24" > > lans = "{" $int_lan $rel_int_lan $rel_ext_lan "}" > > I obtain this error: > > # pfctl -nf /etc/pf.conf > /etc/pf.conf:48: syntax error > /etc/pf.conf:83: macro 'lans' not defined > > Could you give me the correct syntax? Basically, it can't be done, due to the way the parser deals with macros and strings. I wonder if we should make a difference between double quotes "" and single quotes '', like shells do. Right now, both are the same, there's no difference between using "" or '' for strings. We could try to make them behave differently, so that macros get expanded inside "" but not inside '', similar to $ echo "$SHELL" /bin/ksh $ echo '$SHELL' $SHELL Then you could simply write lans = "{ $int_lan $rel_int_lan $rel_ext_lan }" No idea how many rulesets that breaks, nor how much effort it takes to implement it. But this question pops up regularly, and it's frustrating not to be able to provide a good explanation. Daniel
Re: Recursive macro expansion problems
On Fri, 19 Jan 2007, Daniel Hartmeier wrote: > On Fri, Jan 19, 2007 at 07:20:11PM +0100, Xavier ROUX wrote: > > > int_lan = "172.16.1.0/24" > > rel_int_lan = "172.16.2.0/24" > > rel_ext_lan = "172.16.3.0/24" > > > > lans = "{" $int_lan $rel_int_lan $rel_ext_lan "}" > > > > I obtain this error: > > > > # pfctl -nf /etc/pf.conf > > /etc/pf.conf:48: syntax error > > /etc/pf.conf:83: macro 'lans' not defined > > > > Could you give me the correct syntax? > > Basically, it can't be done, due to the way the parser deals with macros > and strings. I just updated this diff that was more then 4 years old. It never was pretty, and still isn't, but it _does_ get the job done, without "{", "'{'", etc. tricks [EMAIL PROTECTED] $ cat test int_lan = "172.16.1.0/24" rel_int_lan = "172.16.2.0/24" rel_ext_lan = "172.16.3.0/24" lans = "$int_lan $rel_int_lan $rel_ext_lan" pass in from { $lans } [EMAIL PROTECTED] $ obj/pfctl -nvvf test int_lan = "172.16.1.0/24" rel_int_lan = "172.16.2.0/24" rel_ext_lan = "172.16.3.0/24" lans = "$int_lan $rel_int_lan $rel_ext_lan" @0 pass in inet from 172.16.1.0/24 to any flags S/SA keep state @1 pass in inet from 172.16.2.0/24 to any flags S/SA keep state @2 pass in inet from 172.16.3.0/24 to any flags S/SA keep state -- Cam Index: parse.y === RCS file: /cvs/src/sbin/pfctl/parse.y,v retrieving revision 1.516 diff -u -p -u -r1.516 parse.y --- parse.y 7 Nov 2006 01:12:01 - 1.516 +++ parse.y 19 Jan 2007 21:31:00 - @@ -5228,22 +5228,18 @@ parse_rules(FILE *input, struct pfctl *x int symset(const char *nam, const char *val, int persist) { - struct sym *sym; + struct sym *osym, *sym; + size_t rlen; + const char *i; + char*n, *p0, *p, *r, *tok; - for (sym = TAILQ_FIRST(&symhead); sym && strcmp(nam, sym->nam); - sym = TAILQ_NEXT(sym, entries)) - ; /* nothing */ - - if (sym != NULL) { - if (sym->persist == 1) - return (0); - else { - free(sym->nam); - free(sym->val); - TAILQ_REMOVE(&symhead, sym, entries); - free(sym); - } + TAILQ_FOREACH(osym, &symhead, entries) { + if (strcmp(nam, osym->nam) == 0) + break; } + if (osym != NULL && osym->persist == 1) + return (0); + if ((sym = calloc(1, sizeof(*sym))) == NULL) return (-1); @@ -5258,10 +5254,76 @@ symset(const char *nam, const char *val, free(sym); return (-1); } + + p0 = p = malloc(strlen(val) + 1); + if (p0 == NULL) { + free(sym->nam); + free(sym); + return (-1); + } + + for (i = val; *i != '\0'; i++) { + if (*i != '$') { + /* Just copy. */ + *p++ = *i; + } else { + /* Found a macro: copy it and try to expand. */ + i++; + *p++ = '\0'; + + tok = p; + while (isalnum(*i) || *i == '_') + *p++ = *i++; + i--; + if (tok == p) { + yyerror("spurious $"); + goto error; + } + *p = '\0'; + p = tok - 1;/* rewind */ + + r = symget(tok); + if (r == NULL) { + yyerror("macro '%s' not defined", tok); + goto error; + } + /* +* Reallocate space to be able to hold: +* 1. what we have so far (p0), +* 2. the expanded macro (r), and +* 3. what's left of the source string (i). +*/ + rlen = strlen(r); + n = realloc(p0, (p - p0) + rlen + strlen(i) + 1); + if (n == NULL) + goto error; + /* Move pointers into new storage. */ + p = n + (p - p0); + p0 = n; + /* Concatenate expanded macro. */ + if (strlcpy(p, r, rlen + 1) != rlen)
Re: Recursive macro expansion problems
On Fri, Jan 19, 2007 at 07:20:11PM +0100, Xavier ROUX wrote: > I am trying to reproduce the example below, given in the FAQ: > host1 = "192.168.1.1" > host2 = "192.168.1.2" > all_hosts = "{" $host1 $host2 "}" > I obtain this error: > # pfctl -nf /etc/pf.conf > /etc/pf.conf:48: syntax error > /etc/pf.conf:83: macro 'lans' not defined > Could you give me the correct syntax? openbsd 3.8 mazzy[3.8]# cat /var/tmp/123 A="'10/8'" B="'192/8'" C="{" $A $B "}" pass in from $C mazzy[3.8]# cat /var/tmp/123 | pfctl -vnf - A = "'10/8'" B = "'192/8'" C = "{ 10/8 192/8 }" pass in inet from 10.0.0.0/8 to any pass in inet from 192.0.0.0/8 to any -- Dmitry
Re: Recursive macro expansion problems
On Mon, Jan 22, 2007 at 09:50:27AM +0700, Dmitry Medvedev wrote: > openbsd 3.8 > mazzy[3.8]# cat /var/tmp/123 > A="'10/8'" > B="'192/8'" Are you just using single quotes to get around the fact that double-quotes are symmetric and thus can't be nested? We should do something about that, but I'm not sure what, since the paired punctuation marks seem to all have significance. I think the same thing happened in bourne shell, and so in POSIX now they have $() instead of backticks. Any ideas on what would be the most obvious way to allow nested quotes? In Perl you can specify a delimiter of your choosing; specifically, qq{} means quote the stuff in curly braces, and qq() parenthesis, qq|| means vertical bars, etc. (Specifically, q is qoute, qq is double-quote, qx is quote and execute, etc.) This is handy, since with fixed strings you can pick something not in the string... -- ``Unthinking respect for authority is the greatest enemy of truth.'' -- Albert Einstein -><- http://www.subspacefield.org/~travis/> pgpBYH8DY7ct9.pgp Description: PGP signature