Re: [racket-dev] When is it safe to not rename a runtime value in a macro?
Investigated a bit today. I wrote a function that optimizes kernel syntax, making these kinds of replacements: (let-values ([(x:id) y:id]) body) -> (let-syntaxes+values ([(x) (make-rename-transformer #'y)]) () body) and then fully expanding the code again. It also does simple beta reduction, by changing ((lambda (x ...) body ...) e ...) into a `let-values'. I've managed to shave 1/4 the runtime off simple things like this: (let ([y 4]) ((lambda (x) (let ([z y]) (+ x z))) 5)) It doesn't correctly handle variables that get mutated, but having it search their lexical scope for `set!' should do it. I think. It hasn't been too bad so far. Neil ⊥ On 08/25/2012 11:03 AM, Matthias Felleisen wrote: It's the wrong way to go. Let's investigate more before we jump to conclusions. On Aug 25, 2012, at 12:54 PM, Neil Toronto wrote: Then I suppose the obvious thing to do is move the optimizations up to the macro level? :) It's also for debugging, by the way. Having to mentally or manually substitute bindings to understand fully expanded code annoys me and wastes time. Neil ⊥ On 08/25/2012 10:24 AM, Robby Findler wrote: This is definitely a macro writer bill of rights situation. Too bad that TR's optimizer cannot take advantage of all these kinds of things that are already happening in the optimizer one level down. Robby On Sat, Aug 25, 2012 at 11:19 AM, Neil Toronto wrote: A lot of macros start by renaming some syntax that's assumed to eventually represent a runtime value, like the `or' macro does: (syntax->datum (expand-syntax #'(or #t #f))) '(let-values (((or-part) '#t)) (if or-part or-part '#f)) But it's not always a good thing, particularly when the output will be processed by another macro, and parts will be tested using an `identifier=?' function. For example, my `inline-sort' takes `<' as an argument. So (inline-sort < a b) expands to (let ([lt? <] [temp1 a] [temp2 b]) (if (lt? temp1 temp2) (values temp1 temp2) (values temp2 temp1))) Well, it did at first. When I discovered that Typed Racket's optimizer would never detect that `lt?' is actually `<' and replace it with `unsafe-fl<', I altered the macro to detect when the `<' is an identifier, and not rename it with a `let'. I know that I should assert my rights under the Macro Bill. But I like to program defensively. So what's safe to not rename? I think the following are: symbol number string bytes null character regexp prefab struct key Also, should I ever have to intern syntax that's one of these kinds of things? Neil ⊥ _ Racket Developers list: http://lists.racket-lang.org/dev _ Racket Developers list: http://lists.racket-lang.org/dev _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] When is it safe to not rename a runtime value in a macro?
On Aug 25, 2012, at 3:56 PM, Ryan Culpepper wrote: > In other words, a macro's subexpressions should be used linearly (as in > "linear types"). Putting an expression in the macro's result in an expression > context counts as a use. (But putting it in the result in a quoted context > doesn't.) Calling 'local-expand' or 'syntax-local-expand-expression' on it > also counts as a use, but it gives back another linearly-restricted > expression. (The same applies to definitions; I should really say all "forms" > should be used linearly.) Are you going to write up a tool and/or a macro-linear-type system that checks such things? ICFP is waiting -- Matthias smime.p7s Description: S/MIME cryptographic signature _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] When is it safe to not rename a runtime value in a macro?
7 hours ago, Matthias Felleisen wrote: > > It's the wrong way to go. Let's investigate more before we jump to > conclusions. Would it work if let-bound (etc) ids could have some property tracing them to the expressions they were bound to? -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] When is it safe to not rename a runtime value in a macro?
6 hours ago, Ryan Culpepper wrote: > > In Racket, literal data carry lexical information just like > identifiers. When a literal datum is used as an expression, the > macro expander synthesizes a '#%datum' identifier that determines > what to do with the literal. The Racket '#%datum' macro just expands > into a 'quote' expression if the datum is not a keyword. [And it's not just an academic exerice: I have an LC-like language in my class where everything is an identifier. (It makes a nice point about really having nothing except for lambdas, applications, and identifiers.)] -- ((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay: http://barzilay.org/ Maze is Life! _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] When is it safe to not rename a runtime value in a macro?
On 08/25/2012 01:57 PM, Neil Toronto wrote: On 08/25/2012 11:33 AM, Ryan Culpepper wrote: On 08/25/2012 01:08 PM, Neil Toronto wrote: A number can expand to an arbitrary expression? How? And what do you mean by "the '#%datum' macro associated with them"? Applied to them? > (let-syntax ([#%datum (lambda (stx) #'(printf "hello\n"))]) 5) hello In Racket, literal data carry lexical information just like identifiers. When a literal datum is used as an expression, the macro expander synthesizes a '#%datum' identifier that determines what to do with the literal. The Racket '#%datum' macro just expands into a 'quote' expression if the datum is not a keyword. The implicit '#%app' syntax works similarly, except it takes its lexical context from the pair that represents the application. Holy heck I had no idea. That's awesome and scary. So this is what I have now: (define skip-ids (syntax->list #'(+ - * / < > <= >= = min max))) (define (skip-binding? e-stx) (let ([e-stx (local-expand e-stx 'expression #f)]) (and (identifier? e-stx) (findf (λ (skip-id) (free-identifier=? skip-id e-stx)) skip-ids I would have called it safe before today, but I knew fewer awesome and scary things. Does it look safe to you? In this case you can get rid of the 'local-expand', since you're only looking for references to particular variables. If you do keep the 'local-expand', you should use the result in the code your macro produces so that the expression isn't expanded multiple times. For example: (define-syntax (m stx) (syntax-case stx () [(m e stuff ...) (let ([ee (local-expand #'e 'expression #f)]) (if (and (identifier? ee) (for/or ([skip-id skip-ids]) (free-identifier=? ee skip-id))) #`(real-m #,ee stuff ...) #`(let ([tmp #,ee]) (real-m tmp stuff ...])) Depending on exactly what you're doing, you might want an empty stop list instead of #f. Or you might want to use 'syntax-local-expand-expression' instead so that the expander doesn't have to retraverse the expanded expression. -- In other words, a macro's subexpressions should be used linearly (as in "linear types"). Putting an expression in the macro's result in an expression context counts as a use. (But putting it in the result in a quoted context doesn't.) Calling 'local-expand' or 'syntax-local-expand-expression' on it also counts as a use, but it gives back another linearly-restricted expression. (The same applies to definitions; I should really say all "forms" should be used linearly.) Ryan _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] When is it safe to not rename a runtime value in a macro?
On 08/25/2012 11:33 AM, Ryan Culpepper wrote: On 08/25/2012 01:08 PM, Neil Toronto wrote: A number can expand to an arbitrary expression? How? And what do you mean by "the '#%datum' macro associated with them"? Applied to them? > (let-syntax ([#%datum (lambda (stx) #'(printf "hello\n"))]) 5) hello In Racket, literal data carry lexical information just like identifiers. When a literal datum is used as an expression, the macro expander synthesizes a '#%datum' identifier that determines what to do with the literal. The Racket '#%datum' macro just expands into a 'quote' expression if the datum is not a keyword. The implicit '#%app' syntax works similarly, except it takes its lexical context from the pair that represents the application. Holy heck I had no idea. That's awesome and scary. So this is what I have now: (define skip-ids (syntax->list #'(+ - * / < > <= >= = min max))) (define (skip-binding? e-stx) (let ([e-stx (local-expand e-stx 'expression #f)]) (and (identifier? e-stx) (findf (λ (skip-id) (free-identifier=? skip-id e-stx)) skip-ids I would have called it safe before today, but I knew fewer awesome and scary things. Does it look safe to you? Neil ⊥ _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] When is it safe to not rename a runtime value in a macro?
On 08/25/2012 01:08 PM, Neil Toronto wrote: On 08/25/2012 10:53 AM, Ryan Culpepper wrote: On 08/25/2012 12:19 PM, Neil Toronto wrote: I've reordered these a bit: number string bytes character regexp In other words, "literal data". But did you check that the '#%datum' macro associated with them has the standard meaning? If not, they could expand into arbitrary expressions (possibly with side effects)! A number can expand to an arbitrary expression? How? And what do you mean by "the '#%datum' macro associated with them"? Applied to them? > (let-syntax ([#%datum (lambda (stx) #'(printf "hello\n"))]) 5) hello In Racket, literal data carry lexical information just like identifiers. When a literal datum is used as an expression, the macro expander synthesizes a '#%datum' identifier that determines what to do with the literal. The Racket '#%datum' macro just expands into a 'quote' expression if the datum is not a keyword. The implicit '#%app' syntax works similarly, except it takes its lexical context from the pair that represents the application. Ryan _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] When is it safe to not rename a runtime value in a macro?
It's the wrong way to go. Let's investigate more before we jump to conclusions. On Aug 25, 2012, at 12:54 PM, Neil Toronto wrote: > Then I suppose the obvious thing to do is move the optimizations up to the > macro level? :) > > It's also for debugging, by the way. Having to mentally or manually > substitute bindings to understand fully expanded code annoys me and wastes > time. > > Neil ⊥ > > On 08/25/2012 10:24 AM, Robby Findler wrote: >> This is definitely a macro writer bill of rights situation. Too bad >> that TR's optimizer cannot take advantage of all these kinds of things >> that are already happening in the optimizer one level down. >> >> Robby >> >> On Sat, Aug 25, 2012 at 11:19 AM, Neil Toronto >> wrote: >>> A lot of macros start by renaming some syntax that's assumed to eventually >>> represent a runtime value, like the `or' macro does: >>> (syntax->datum (expand-syntax #'(or #t #f))) >>> '(let-values (((or-part) '#t)) (if or-part or-part '#f)) >>> >>> But it's not always a good thing, particularly when the output will be >>> processed by another macro, and parts will be tested using an `identifier=?' >>> function. For example, my `inline-sort' takes `<' as an argument. So >>> >>> (inline-sort < a b) >>> >>> expands to >>> >>> (let ([lt? <] [temp1 a] [temp2 b]) >>> (if (lt? temp1 temp2) (values temp1 temp2) (values temp2 temp1))) >>> >>> Well, it did at first. When I discovered that Typed Racket's optimizer would >>> never detect that `lt?' is actually `<' and replace it with `unsafe-fl<', I >>> altered the macro to detect when the `<' is an identifier, and not rename it >>> with a `let'. >>> >>> I know that I should assert my rights under the Macro Bill. But I like to >>> program defensively. So what's safe to not rename? I think the following >>> are: >>> >>> symbol >>> number >>> string >>> bytes >>> null >>> character >>> regexp >>> prefab struct key >>> >>> Also, should I ever have to intern syntax that's one of these kinds of >>> things? >>> >>> Neil ⊥ >>> >>> _ >>> Racket Developers list: >>> http://lists.racket-lang.org/dev > > _ > Racket Developers list: > http://lists.racket-lang.org/dev smime.p7s Description: S/MIME cryptographic signature _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] When is it safe to not rename a runtime value in a macro?
On 08/25/2012 10:53 AM, Ryan Culpepper wrote: On 08/25/2012 12:19 PM, Neil Toronto wrote: I've reordered these a bit: number string bytes character regexp In other words, "literal data". But did you check that the '#%datum' macro associated with them has the standard meaning? If not, they could expand into arbitrary expressions (possibly with side effects)! A number can expand to an arbitrary expression? How? And what do you mean by "the '#%datum' macro associated with them"? Applied to them? symbol Do you mean identifier, as in "it's just a variable reference"? But it could be an identifier macro. Or it could be a variable that other parts of the code could concurrently mutate. For example, suppose 'match' expanded less cautiously than it actually does: (match x [(cons a b) a]) => (if (pair? x) (unsafe-car x) (match-error )) Now suppose that x is a module-level variable that some other thread mutates between the 'pair?' check and the 'unsafe-car'. Right. Typed Racket would actually catch these at compile time, but mutation would still break my macros. *sigh* If you must do such things, the safest way in general is to fully local-expand the expression you want to analyze; that gives you something with a known grammar. Now it's feasible to identify literal data: it's any 'quote' expression. You don't have to worry about identifier macros, but the 'set!' danger remains for any variables that you don't completely control the scope of. In your specific case, it would also be safe to check whether the comparison expression is an identifier free-identifier=? to one of a fixed list of known (immutable) variables; if so, it's safe to duplicate. I think I'll have to go that route. Thanks for the explanations! Neil ⊥ _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] When is it safe to not rename a runtime value in a macro?
Then I suppose the obvious thing to do is move the optimizations up to the macro level? :) It's also for debugging, by the way. Having to mentally or manually substitute bindings to understand fully expanded code annoys me and wastes time. Neil ⊥ On 08/25/2012 10:24 AM, Robby Findler wrote: This is definitely a macro writer bill of rights situation. Too bad that TR's optimizer cannot take advantage of all these kinds of things that are already happening in the optimizer one level down. Robby On Sat, Aug 25, 2012 at 11:19 AM, Neil Toronto wrote: A lot of macros start by renaming some syntax that's assumed to eventually represent a runtime value, like the `or' macro does: (syntax->datum (expand-syntax #'(or #t #f))) '(let-values (((or-part) '#t)) (if or-part or-part '#f)) But it's not always a good thing, particularly when the output will be processed by another macro, and parts will be tested using an `identifier=?' function. For example, my `inline-sort' takes `<' as an argument. So (inline-sort < a b) expands to (let ([lt? <] [temp1 a] [temp2 b]) (if (lt? temp1 temp2) (values temp1 temp2) (values temp2 temp1))) Well, it did at first. When I discovered that Typed Racket's optimizer would never detect that `lt?' is actually `<' and replace it with `unsafe-fl<', I altered the macro to detect when the `<' is an identifier, and not rename it with a `let'. I know that I should assert my rights under the Macro Bill. But I like to program defensively. So what's safe to not rename? I think the following are: symbol number string bytes null character regexp prefab struct key Also, should I ever have to intern syntax that's one of these kinds of things? Neil ⊥ _ Racket Developers list: http://lists.racket-lang.org/dev _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] When is it safe to not rename a runtime value in a macro?
On 08/25/2012 12:19 PM, Neil Toronto wrote: A lot of macros start by renaming some syntax that's assumed to eventually represent a runtime value, like the `or' macro does: > (syntax->datum (expand-syntax #'(or #t #f))) '(let-values (((or-part) '#t)) (if or-part or-part '#f)) But it's not always a good thing, particularly when the output will be processed by another macro, and parts will be tested using an `identifier=?' function. For example, my `inline-sort' takes `<' as an argument. So (inline-sort < a b) expands to (let ([lt? <] [temp1 a] [temp2 b]) (if (lt? temp1 temp2) (values temp1 temp2) (values temp2 temp1))) Well, it did at first. When I discovered that Typed Racket's optimizer would never detect that `lt?' is actually `<' and replace it with `unsafe-fl<', I altered the macro to detect when the `<' is an identifier, and not rename it with a `let'. Is it feasible to make the TR optimizer smarter about this? That would be the best solution. I know that I should assert my rights under the Macro Bill. But I like to program defensively. So what's safe to not rename? I think the following are: I've reordered these a bit: number string bytes character regexp In other words, "literal data". But did you check that the '#%datum' macro associated with them has the standard meaning? If not, they could expand into arbitrary expressions (possibly with side effects)! symbol Do you mean identifier, as in "it's just a variable reference"? But it could be an identifier macro. Or it could be a variable that other parts of the code could concurrently mutate. For example, suppose 'match' expanded less cautiously than it actually does: (match x [(cons a b) a]) => (if (pair? x) (unsafe-car x) (match-error )) Now suppose that x is a module-level variable that some other thread mutates between the 'pair?' check and the 'unsafe-car'. null prefab struct key I don't know what you mean by these in this context. Maybe 'quote' expressions containing them? -- If you must do such things, the safest way in general is to fully local-expand the expression you want to analyze; that gives you something with a known grammar. Now it's feasible to identify literal data: it's any 'quote' expression. You don't have to worry about identifier macros, but the 'set!' danger remains for any variables that you don't completely control the scope of. In your specific case, it would also be safe to check whether the comparison expression is an identifier free-identifier=? to one of a fixed list of known (immutable) variables; if so, it's safe to duplicate. Also, should I ever have to intern syntax that's one of these kinds of things? Do you mean intern the data? IIUC, yes: the reader interns literal data, but macros are free to introduce non-interned literal data. (It will probably get interned again when the compiled form is loaded from zo, but there's a period during the expansion of the enclosing module when it won't be interned.) Ryan _ Racket Developers list: http://lists.racket-lang.org/dev
Re: [racket-dev] When is it safe to not rename a runtime value in a macro?
This is definitely a macro writer bill of rights situation. Too bad that TR's optimizer cannot take advantage of all these kinds of things that are already happening in the optimizer one level down. Robby On Sat, Aug 25, 2012 at 11:19 AM, Neil Toronto wrote: > A lot of macros start by renaming some syntax that's assumed to eventually > represent a runtime value, like the `or' macro does: > >> (syntax->datum (expand-syntax #'(or #t #f))) > '(let-values (((or-part) '#t)) (if or-part or-part '#f)) > > But it's not always a good thing, particularly when the output will be > processed by another macro, and parts will be tested using an `identifier=?' > function. For example, my `inline-sort' takes `<' as an argument. So > > (inline-sort < a b) > > expands to > > (let ([lt? <] [temp1 a] [temp2 b]) > (if (lt? temp1 temp2) (values temp1 temp2) (values temp2 temp1))) > > Well, it did at first. When I discovered that Typed Racket's optimizer would > never detect that `lt?' is actually `<' and replace it with `unsafe-fl<', I > altered the macro to detect when the `<' is an identifier, and not rename it > with a `let'. > > I know that I should assert my rights under the Macro Bill. But I like to > program defensively. So what's safe to not rename? I think the following > are: > > symbol > number > string > bytes > null > character > regexp > prefab struct key > > Also, should I ever have to intern syntax that's one of these kinds of > things? > > Neil ⊥ > > _ > Racket Developers list: > http://lists.racket-lang.org/dev _ Racket Developers list: http://lists.racket-lang.org/dev
[racket-dev] When is it safe to not rename a runtime value in a macro?
A lot of macros start by renaming some syntax that's assumed to eventually represent a runtime value, like the `or' macro does: > (syntax->datum (expand-syntax #'(or #t #f))) '(let-values (((or-part) '#t)) (if or-part or-part '#f)) But it's not always a good thing, particularly when the output will be processed by another macro, and parts will be tested using an `identifier=?' function. For example, my `inline-sort' takes `<' as an argument. So (inline-sort < a b) expands to (let ([lt? <] [temp1 a] [temp2 b]) (if (lt? temp1 temp2) (values temp1 temp2) (values temp2 temp1))) Well, it did at first. When I discovered that Typed Racket's optimizer would never detect that `lt?' is actually `<' and replace it with `unsafe-fl<', I altered the macro to detect when the `<' is an identifier, and not rename it with a `let'. I know that I should assert my rights under the Macro Bill. But I like to program defensively. So what's safe to not rename? I think the following are: symbol number string bytes null character regexp prefab struct key Also, should I ever have to intern syntax that's one of these kinds of things? Neil ⊥ _ Racket Developers list: http://lists.racket-lang.org/dev
