Re: [PATCH] local-eval, local-compile, and the-environment (v3)
David Kastrup d...@gnu.org writes: (define current-module (let ((top-level (the-environment))) (lambda () (eval '(the-environment) top-level Some more notes about the above code (changing `eval' == `local-eval'): * (local-eval '(the-environment) environment) is a no-op: it always returns the same environment that was passed in, so there's no point in doing it. * Of course `top-level' is a constant, and would be even if it were within the (lambda () ...) because it captures no lexicals. It is always in the same module, i.e. the module containing the above definition. This same constant value is always returned by (current-module). * Also note that the real `current-module' simply accesses a fluid, which can also be set by `set-current-module'. (Fluids are a scheme analogue to dynamically-scoped variables in Lisp). Conceptually, it is variable that is explicitly set by the user. It has no relation to the code that is currently executing. Rather, it is used during compilation (or within a REPL) to keep track of where the user would like top-level definitions to go. Mark
Re: [PATCH] local-eval, local-compile, and the-environment (v3)
I wrote: * Also note that the real `current-module' simply accesses a fluid, which can also be set by `set-current-module'. (Fluids are a scheme analogue to dynamically-scoped variables in Lisp). Conceptually, it is variable that is explicitly set by the user. It has no relation to the code that is currently executing. Rather, it is used during compilation (or within a REPL) to keep track of where the user would like top-level definitions to go. That last bit was not quite right, let me try again: The `current-module' is used during compilation, within a REPL, or by primitive-eval, to keep track of which module the user would like top-level forms to be compiled in. It is set at compile time by constructs such as `define-module' or (eval-when (compile) (set-current-module module)), and this module is baked into every identifier at compile time before macro processing takes place. Macros in general might slice and dice and mix together fragments of code from many different modules. When this jumble of macro-expanded code is evaluated, (current-module) is no longer relevant to it. Instead, each top-level variable reference uses the module that was baked into its identifier before macro expansion. Mark
Re: [PATCH] local-eval, local-compile, and the-environment (v3)
Mark H Weaver m...@netris.org writes: David Kastrup d...@gnu.org writes: (define current-module (let ((top-level (the-environment))) (lambda () (eval '(the-environment) top-level Some more notes about the above code (changing `eval' == `local-eval'): * (local-eval '(the-environment) environment) is a no-op: it always returns the same environment that was passed in, so there's no point in doing it. I think this was based on the the current module is part of the environment, but local-eval does not change (current-module) mantra which I interpreted in a confused manner. The interesting thing is that it is the _lexically_ current module that is part of the environment, and inside of local-eval, this may well differ from the _actually_ current module as given by (current-module). -- David Kastrup
Re: [PATCH] local-eval, local-compile, and the-environment (v3)
Mark H Weaver m...@netris.org writes: Here's the third version of my simple `local-eval' patch. Notable changes from last time: * Pattern variables are now captured properly. * `the-environment' now works as advertised within macro definitions. * Added doc strings for `local-eval' and `local-compile'. I am open to reimplementing this in a different way for 2.0.5, along the lines suggested by Andy. I'd like to capture all bindings, not just the ones reachable by symbols. I'd like to support _all_ lexical bindings, including local syntax transformers. I'm also warming to the idea of standardizing on variable objects as a way to represent mutable lexicals. However, there's no time to do this for 2.0.4. That job depends on other big jobs, notably a major overhaul of the evaluator. Nonetheless, I think it is very important to include this simple implementation in 2.0.4. This is a BUG FIX, the bug being that `local-eval' was removed from underneath Lilypond's feet. A partial implementation (that almost certainly does everything they need) is _far_ better than none at all. Lilypond can only depend on `local-eval' if installations of Guile without it are quite rare. If we can get this in 2.0.4, there's hope that we can make Guile 2.0.[0-3] rare. Please consider it. This implementation is quite robust. I am not altogether comfortable with pushing a temporary fix for the sake of LilyPond when we'll get another behavioral change with the next version. LilyPond would be mostly left in the rain for older versions unless we made that implementation a fallback within our own sources. However, there would be no point in adopting a fallback implementation that would not also work with 2.0.0-2.0.3. As far as I understand, this implementation could be pulled into a separate file and used for bridging the 2.0.0-2.0.3 gap. If it is solid enough to be maintained as 2.0.4, it would likely be a reasonably good bet. Of course, a version that would be tighter integrated with the compiler and optimizer would seem like a more powerful prospect, but it also sounds that it would need more thorough maintenance to not fall into bit rot. I am not sure that the reasons for not permitting definition context in local-eval are not of somewhat more theoretical than practical nature, and in particular it sounds to me like I'm also warming to the idea of standardizing on variable objects as a way to represent mutable lexicals could suggest that the strength of even the theoretical reasons might get eroded further. Of course, _calling_ mutual recursive functions in execution before the second part has been defined will remain unfeasible. -- David Kastrup
Re: [PATCH] local-eval, local-compile, and the-environment (v3)
David Kastrup d...@gnu.org writes: I am not altogether comfortable with pushing a temporary fix for the sake of LilyPond when we'll get another behavioral change with the next version. But there would _not_ be a behavioral change in the next version. It would only be a change in the internal implementation. As far as I understand, this implementation could be pulled into a separate file and used for bridging the 2.0.0-2.0.3 gap. Unfortunately, any implementation of `local-eval' requires changes to psyntax.scm, which is not something that you could reasonably do as an external package. I am not sure that the reasons for not permitting definition context in local-eval are not of somewhat more theoretical than practical nature, There's at least one practical reason not to allow it, namely that it is _impossible_ to implement. Consider this: (let ((x 1)) (define (get-x) x) (the-environment)) If we allow (the-environment) to add definitions to the implicit `letrec', then (get-x) cannot know which binding of `x' to use. In fact, it cannot lookup _any_ bindings, because absolutely any identifier (even syntactic keywords) could be redefined within this implicit `letrec'. This means that it's impossible to compile or evaluate anything within the body of the outer `let', which means it's not even possible to evaluate (the-environment) itself. So there's no way to run the code above if we allow (the-environment) to add definitions. Mark
Re: [PATCH] local-eval, local-compile, and the-environment (v3)
Mark H Weaver m...@netris.org writes: David Kastrup d...@gnu.org writes: Mark H Weaver m...@netris.org writes: David Kastrup d...@gnu.org writes: I am not sure that the reasons for not permitting definition context in local-eval are not of somewhat more theoretical than practical nature, There's at least one practical reason not to allow it, namely that it is _impossible_ to implement. Consider this: (let ((x 1)) (define (get-x) x) (the-environment)) If we allow (the-environment) to add definitions to the implicit `letrec', Then just let's not allow it. Consider the body of local-eval to be wrapped inside of an implicit (begin ...). I think you mean an implicit (let () ...). If that's what you want, then you can do it yourself, and the result will be less likely to confuse. What is confusing here? Obviously, definitions made in the scope of local-eval can't retroactively affect the environment where the-environment was called. And now instead of continuing with the unfriendly For this reason, they are prohibited. we continue with You may consider the local-eval body as being wrapped inside of an implicit (let () ...). I repeat: is there any valid construct under the currently proposed local-eval code that would change its behavior? Is there any currently valid construct that would change its behavior? If not, you _gain_ functionality, and the resulting semantics are still straightforward to explain as far as I can see. I don't understand this at all. Change what behavior? The behavior of local-eval given arbitrary expressions. Are there any expressions currently valid that would change their behavior if the body of local-eval were wrapped in an implicit (let () ...)? What functionality do you gain? The ability to make definitions valid in the local-eval without having to write (let () ...). Again: any drawback that is more substantial than I just don't want you to do that? -- David Kastrup
Re: [PATCH] local-eval, local-compile, and the-environment (v3)
David Kastrup d...@gnu.org writes: Mark H Weaver m...@netris.org writes: David Kastrup d...@gnu.org writes: Mark H Weaver m...@netris.org writes: David Kastrup d...@gnu.org writes: I am not sure that the reasons for not permitting definition context in local-eval are not of somewhat more theoretical than practical nature, There's at least one practical reason not to allow it, namely that it is _impossible_ to implement. Consider this: (let ((x 1)) (define (get-x) x) (the-environment)) If we allow (the-environment) to add definitions to the implicit `letrec', Then just let's not allow it. Consider the body of local-eval to be wrapped inside of an implicit (begin ...). I think you mean an implicit (let () ...). If that's what you want, then you can do it yourself, and the result will be less likely to confuse. What is confusing here? Obviously, definitions made in the scope of local-eval can't retroactively affect the environment where the-environment was called. And now instead of continuing with the unfriendly For this reason, they are prohibited. we continue with You may consider the local-eval body as being wrapped inside of an implicit (let () ...). If you want to include local definitions, then you'll need to wrap the form passed to `local-eval' within (begin ...) anyway. If you're doing that, why not just wrap them in (let () ...) instead? Is it really more work to type #`(let () #,@forms) than #`(begin #,@forms) ? I would like for users to be able to use the simple mental model that the local expression is simply put in place of (the-environment). I'd also like to achieve the following source-level equivalence: expr == (local-eval 'expr (the-environment)) You are suggesting that we wrap the expression within a (let () ...), for the dubious benefit of allowing you to wrap the local forms in (begin ...) instead of (let () ...). I don't see any compelling benefit to this. On the other hand, I see less elegant semantics and potential confusion among users, who might reasonably expect the definitions in their (begin ...) to be added to the implicit `letrec', as would happen if the (begin ...) were put in place of (the-environment). Mark
Re: [PATCH] local-eval, local-compile, and the-environment (v3)
David Kastrup d...@gnu.org writes: You are suggesting that we wrap the expression within a (let () ...), for the dubious benefit of allowing you to wrap the local forms in (begin ...) instead of (let () ...). Are there even situations where you could put definitions after begin? How are they different from the situation where you can't? I don't see any compelling benefit to this. On the other hand, I see less elegant semantics and potential confusion among users, who might reasonably expect the definitions in their (begin ...) to be added to the implicit `letrec', as would happen if the (begin ...) were put in place of (the-environment). begin can start with definitions, but not always? But (let () ...) can? Scheme is weird. `begin' is indeed quite weird in Scheme. Its meaning depends on the context where it is found: * In expression context, it is like `progn' in Lisp. It evaluates the expressions in order from left to right, and returns the value(s) of the last expression. * At the top-level, or within an internal body where local definitions are allowed (i.e. if every form above it in the internal body is a definition), it is a _splicing_ operator: it acts as if the contents of the begin were spliced into the surrounding context. This is mainly for the benefit of macros, so that they may expand into multiple definitions. For example: (let ((x 1)) (define (get-x) x) (begin (define x 2) (get-x))) is equivalent to: (let ((x 1)) (define (get-x) x) (define x 2) (get-x)) which is equivalent to: (let ((x 1)) (letrec ((get-x (lambda () x)) (x 2)) (get-x))) and therefore the result is 2, because the reference to `x' within (get-x) refers to the inner binding of `x'. Mark
Re: [PATCH] local-eval, local-compile, and the-environment (v3)
Mark H Weaver m...@netris.org writes: David Kastrup d...@gnu.org writes: You are suggesting that we wrap the expression within a (let () ...), for the dubious benefit of allowing you to wrap the local forms in (begin ...) instead of (let () ...). Are there even situations where you could put definitions after begin? How are they different from the situation where you can't? I don't see any compelling benefit to this. On the other hand, I see less elegant semantics and potential confusion among users, who might reasonably expect the definitions in their (begin ...) to be added to the implicit `letrec', as would happen if the (begin ...) were put in place of (the-environment). begin can start with definitions, but not always? But (let () ...) can? Scheme is weird. `begin' is indeed quite weird in Scheme. Its meaning depends on the context where it is found: * In expression context, it is like `progn' in Lisp. It evaluates the expressions in order from left to right, and returns the value(s) of the last expression. * At the top-level, or within an internal body where local definitions are allowed (i.e. if every form above it in the internal body is a definition), it is a _splicing_ operator: it acts as if the contents of the begin were spliced into the surrounding context. This is mainly for the benefit of macros, so that they may expand into multiple definitions. Well, what if begin were able to splice definitions into the original lexical context of the-environment? I think it would be strange if a continuation taken in that context would get to see new definitions. And if a continuation there does not get to see it, it would be strange if successive calls of local-eval on the same environment got to see it. So there is not much of a sensible option except an implicit (let () ...) wrapper for the sake of splicing begin, or bugging out for new definitions. In the light of local-eval evaluating a _form_ rather than a _body_ (stupid of me to forget), and seeing the weird semantics of begin, I agree that an implicit (let () ...) wrapper for a single form does not really appear to add significant gains. It would be marginally convenient and marginally confusing. So one might as well forget about it. -- David Kastrup
Re: [PATCH] local-eval, local-compile, and the-environment (v3)
David Kastrup d...@gnu.org writes: In the light of local-eval evaluating a _form_ rather than a _body_ (stupid of me to forget), and seeing the weird semantics of begin, I agree that an implicit (let () ...) wrapper for a single form does not really appear to add significant gains. It would be marginally convenient and marginally confusing. So one might as well forget about it. Except possibly for symmetry: how does plain eval handle it? guile (let ((x 2)) (eval '(begin (define x 4) x) (current-module))) 4 guile x 4 guile Right through to the top. And we couldn't do that in local-eval. But it also has no qualms just because previously evaluated forms would have used a previous definition of x. But that's because of top-level. Eval is always top-level, and define acts like set! there. What if the-environment had been taken at top-level (basically just carrying the information of (current-module))? Should local-eval then behave accordingly? If so, could we not just fold eval and local-eval into one function? And one could then define (define current-module (let ((top-level (the-environment))) (lambda () (eval '(the-environment) top-level if the-environment just returns the current module when at top level? Basically, I can accept your reasoning. This is just putting out a bit of brainstorming. -- David Kastrup
Re: [PATCH] local-eval, local-compile, and the-environment (v3)
Mark H Weaver m...@netris.org writes: What if the-environment had been taken at top-level (basically just carrying the information of (current-module))? Should local-eval then behave accordingly? If so, could we not just fold eval and local-eval into one function? This is a good question. Unfortunately, there is a non-trivial difference in the semantics of `eval' vs `local-eval'. `eval' temporarily sets the (current-module) to its second argument during evaluation of the expression. `local-eval' does not do this, and as I recall that was something you felt strongly about (and I agree). I don't think I would feel as strong about it if the argument was a module, implicating top-level. It is probably not all that easy to get the-environment at top level. And one could then define (define current-module (let ((top-level (the-environment))) (lambda () (eval '(the-environment) top-level if the-environment just returns the current module when at top level? I see what you're getting at, but this would be a bad idea, because it still makes sense to have `module' be an independent type. It might come handy to have local-eval not balk upon getting a module, but without letting it set the current module. At times, this functionality might come in handy. It would also be clear then why eval would have to balk on every non-module (== non-top-level) environment. Also, in the code above, `top-level' would not actually be a top-level environment, because (the-environment) is not a top-level form. If you put (define x 5) in place of (the-environment), it would not set a top-level variable; it would produce an error. I am not saying that the actual code would need to work. It was more intended as a show of equivalences. -- David Kastrup
Re: [PATCH] local-eval, local-compile, and the-environment (v3)
David Kastrup d...@gnu.org writes: It might come handy to have local-eval not balk upon getting a module, but without letting it set the current module. Agreed, and my implementation of `local-eval' does exactly that. Mark
