Re: [BUG] Eval sets incorrect runtime metainformation
On Wed 26 Jun 2024 11:36, Maxime Devos writes: > IIRC, the question wasn’t about debugging in general, it was about > source locations in particular. Surely program-sources (or, in this > case, procedure-source maybe?) (why are the procedures in this family > even named program-whatever, this prevents doing the same for > interpreted code later) could be adjusted to also work for ‘eval’. For > example, ‘eval’ could set the ‘source’ (*) procedure property when a > closure is made. I think it's really valuable to imagine how things should be but if you are going to argue they should be different, you should first try to understand how they are. `program-sources` is a mapping from bytecode offsets to source locations. For compiled procedures we can make this mapping because each bytecode position has a single source. For interpreted procedures, what you end up getting is the bytecode-to-source mapping *for eval*, not for the code being interpreted. Is it a great thing that there is a debugging (I use the term on purpose to mean all kinds of run-time reflection etc) difference between eval and compile? No, of course not. I would rather there not be a difference and not have to document something that is at best extraneous. There are differing pressures on eval: for bootstrap times (and macro expansion time) you want it to have the least amount of overhead possible, whereas for debugging you want to attach meta-data that isn't strictly needed at run-time. Attaching that meta-data has memory and time overheads. If we are looking to get the source location *just of the interpreted closure* -- that is possible; see eval.scm:581, there you would attach some other properties. You would have to define a different debugging interface that looks for source location information in a way different from program-sources. For me it's not worth it but I encourage you to experiment with (ice-9 eval); it's just another Scheme program. (You would need to take a different approach to memoization, in order to pass through source location information.) Andy
Re: [BUG] Eval sets incorrect runtime metainformation
Hello, On Tue 25 Jun 2024 17:07, Andrew Tropin writes: > (use-modules (system vm program) > (ice-9 eval-string)) > > (eval-string "(define (test-fn) 'hey)" > #:file "hello.scm" > #:line 1 > #:column 1 > #:compile? #f) > > (format #t "~a\n" (program-sources test-fn)) > ;; ((0 ice-9/eval.scm 329 . 13) (12 ice-9/eval.scm 330 . 21) (44 > ice-9/eval.scm 330 . 15)) What you are seeing here is that in general the debugging experience is different for interpreted and compiled procedures. For example, you will not be able to set a breakpoint in interpreted code, because the code for the closure that is part of `eval` corresponds to potentially many different functions. program-sources will only work usefully on compiled procedures. https://www.gnu.org/software/guile/manual/html_node/Compiled-Procedures.html. I would suggest that if you are working on a rich IDE, that you pass #:compile? #t. Nothing else will work as you like. That said, the evaluator does attach so-called "meta-data" information to procedures, such as the procedure name. https://www.gnu.org/software/guile/manual/html_node/Procedure-Properties.html. If you know that you are making a procedure you can insert some meta-data for use by your run-time, in an initial vector alist. See https://www.gnu.org/software/guile/manual/html_node/Procedure-Properties.html. But that's limited and doesn't take macros, etc into account. Regards, Andy
Re: what to do about gnulib
Hi Bruno!
Thanks for your kind answer and tips :)
On Mon 24 Jun 2024 19:09, Bruno Haible writes:
> I can reproduce a problem by
> 0. using a git checkout of guile ('main' branch, not 'master' branch),
> 1. using a GNU gettext version 0.22.5, 0.21, 0.20.2, or 0.19.8.1,
> 2. removing the gnulib-local/m4/clock_time.m4.diff file, which no longer
> applies,
> 3. running
> $ $GNULIB_SRCDIR/gnulib-tool --update
> 4. running
> $ ./autogen.sh
> (which is what the HACKING file recommends).
>
> It fails with
>
> configure:12833: error: possibly undefined macro: gl_PTHREADLIB
> If this token and others are legitimate, please use m4_pattern_allow.
> See the Autoconf documentation.
> configure:12943: error: possibly undefined macro: gl_WEAK_SYMBOLS
> configure:27584: error: possibly undefined macro: gl_TYPE_WINT_T_PREREQ
> autoreconf: error: /usr/bin/autoconf failed with exit status: 1
>
> Following your analysis, I locally apply the workaround from the Gnulib
> documentation section "3.11 Caveat: gettextize and autopoint users",
Thank you! It is embarrassing but I did not know this manual existed.
I will use this workaround.
Question, do you think it would be reasonable for autopoint to avoid
overwriting newer files? It seems like the sort of problem we could
avoid, but who knows.
> 373 | unreachable (scm_jit_state *j)
Ah interesting, will fix.
> The way Guile handles versioning of Gnulib-imported files is fine.
That's great to hear. In that case, no change planned to how we do
things.
> Note, though, that the imported po/ infrastructure will still be from 2010,
> due to this line in configure.ac:
> AM_GNU_GETTEXT_VERSION([0.18.1])
> It would be reasonable to bump this version specification to
> AM_GNU_GETTEXT_VERSION([0.19.8.1])
> (from 2016) or
> AM_GNU_GETTEXT_VERSION([0.20.2])
> (from 2020) or
> AM_GNU_GETTEXT_VERSION([0.21.1])
> (from 2022).
>
> Note also that when upgrading to a newer Gnulib, you now have another choice
> than Gnulib's 'master' branch: the 'stable-202401' branch. See the Gnulib
> documentation [2], section "1.6.1 Stable Branches".
Thanks!!
Best regards,
Andy
fyi: deleted `master` branch
Hi, Following the conventional switch to name the main development branch `main` instead of `master`, I have deleted the `master` branch which wasn't being automatically kept in sync and could confuse people. Thanks to Bruno Haible for his detailed notes reproducing my problems updating gnulib :) Andy
what to do about gnulib
Hi, I am not sure what to do about gnulib. To recap, gnulib provides compile-time shims for "foreign" or out-of-date target systems that allows Guile to program to a single POSIX + GNU API and not worry too much about what the system actually offers. It is never installed; instead it includes the `gnulib-tool` utility to update the in-tree copy of gnulib, for a selected set of modules. So far, so good. However I was unable to update gnulib before the most recent release. The reason is essentially the problem described in this issue: https://sourceware.org/bugzilla/show_bug.cgi?id=30051 To wit, running "autoreconf -vif" invokes the "autopoint" tool supplied by installed gettext, which copies over .m4 files from installed gettext, but these files are older than the ones that are already "vendored" in-tree by gnulib. Other parts of gnulib depend on the newer gnulib-supplied macros which were stompled over; autoconf thus fails to run. So, I can't update gnulib right now. Not a great situation. The options, as I understand them: 1. Some projects (e.g. poke) seem to import the whole of gnulib as a git submodule, and then run "gnulib-tool --update" from that submodule as part of their bootstrap. In this way the stompled files are restored. However I do not want git submodules in Guile; they add additional steps for every time you change to a different HEAD, and I know from experience that I can't rely on myself to perform them all, much less any user with a bug report. Do not want. 2. Same as (1), but subtree merge. This is what we do with lightening and what I would like to do with Whippet, if that project works out. But gnulib is big; a checkout is 200 MB of data + 130 MB git repo. ** Note, for (1) and (2), if you wanted to preserve the ability to bootstrap from a tarball, you'd have to include gnulib in the tarball. Of course you could argue that if you are not gnulib-tool --update'ing, are you really bootstrapping? I don't know the answer. *** 3. Fix autopoint to not overwrite newer m4 files with its copies. I don't know? 4. Fix installed gettext to not define gl_ macros ? That would make it so that it won't stomple on a local gnulib copy. I don't know though. 5. Update to whatever version of gnulib is the latest without this change? 6. Something else? Stop using gnulib? I am not a gnulib wizard and would appreciate any thoughts :) Cc'ing Bruno as he is the expert. Really I would like to avoid gnulib-tool --update in Guile's bootstrap and just keep things like it is. Of course I could be being unreasonable. Cheers, Andy
GNU Guile 3.0.10 released
We are pleased to announce GNU Guile release 3.0.10, the latest in the 3.0 stable release series. Guile 3.0.10 is a bug-fix release which also adds optimizations, support for a new WebAssembly back-end, new custom port facilities, a new optional front-end syntax, and more. See the NEWS extract at the end of the mail for full details. Here are the compressed sources: ftp://ftp.gnu.org/gnu/guile/guile-3.0.10.tar.gz (9.3MB) ftp://ftp.gnu.org/gnu/guile/guile-3.0.10.tar.lz (5.3MB) ftp://ftp.gnu.org/gnu/guile/guile-3.0.10.tar.xz (5.6MB) Here are the GPG detached signatures: ftp://ftp.gnu.org/gnu/guile/guile-3.0.10.tar.gz.sig ftp://ftp.gnu.org/gnu/guile/guile-3.0.10.tar.lz.sig ftp://ftp.gnu.org/gnu/guile/guile-3.0.10.tar.xz.sig Use a mirror for higher download bandwidth: https://www.gnu.org/order/ftp.html Here are the SHA1 and SHA256 checksums: ca52ff876b8552b6f8f70231cc33ef7cbcc4a95f guile-3.0.10.tar.gz Lb28l1mLL68xATVk77SOT+1EEx0o6ZbCar6KWyO1bCo guile-3.0.10.tar.gz 105a99a0771d7371502b529b762abe18d729930e guile-3.0.10.tar.lz 8Nl8zPUG5LYQTFFjm9G3v2dADgs0AfCy9rcTZTIEAyc guile-3.0.10.tar.lz e95e37f681dae692de52242929cbc7092eef3eb4 guile-3.0.10.tar.xz vXFoUX/VJjM0RtT3q4FlJ5JWNAlPvTcyLhfiuNjnY4g guile-3.0.10.tar.xz The SHA256 checksum is base64 encoded, instead of the hexadecimal encoding that most checksum tools default to. Use a .sig file to verify that the corresponding file (without the .sig suffix) is intact. First, be sure to download both the .sig file and the corresponding tarball. Then, run a command like this: gpg --verify guile-3.0.10.tar.gz.sig The signature should match the fingerprint of the following key: pub rsa4096 2017-08-09 [SC] 4FD4 D288 D445 934E 0A14 F9A5 A880 3732 E443 6885 uid Andy Wingo uid Andy Wingo uid Andy Wingo If that command fails because you don't have the required public key, or that public key has expired, try the following commands to retrieve or refresh it, and then rerun the 'gpg --verify' command. gpg --recv-keys 4FD4D288D445934E0A14F9A5A8803732E4436885 As a last resort to find the key, you can try the official GNU keyring: wget -q https://ftp.gnu.org/gnu/gnu-keyring.gpg gpg --keyring gnu-keyring.gpg --verify guile-3.0.10.tar.gz.sig This release was bootstrapped with the following tools: Autoconf 2.71 Automake 1.16.5 Libtool 2.4.7 Gnulib v0.1-4855-g8f4538a53d Makeinfo 7.1 An extract from NEWS follows. * Notable changes ** Beginnings of support for alternate back-ends A number of adaptations and additions of as-yet unstable interfaces have been made to allow third-party projects such as the Hoot Guile-to-WebAssembly whole-program compiler (https://spritely.institute/hoot/) to use the Guile front-end and optimizer. Depending on how Hoot development goes, we may consider adding first-class support for WebAssembly as a compilation target; discussion is welcome on the guile-devel mailing list. ** `define` in all bodies `define` adds a definition to the body in which it appears, as if each non-tail definition or expression in that body were a binding in a surrounding `letrec*` clause. However, in some places, using `define` would result in the annoying error "definition in expression context, where definitions are not allowed", which could be fixed by explicitly adding a surrounding binding contour, for example an empty `let`. This was because there was no implicit surrounding lexical binding contour for the body of `when` and `unless`, for `cond` clauses, `case` clauses, `and-let*` bodies, and `with-fluids`. But no more; now these contexts now create a binding contour, allowing the use of `define`. ** Two bug fixes of note regarding scoping of top-level variables Previously, a reference to a top-level variable in a module other than the current module would be silently rewritten to reference the current module, if the variable was unbound in its original module. This was a hack from the early days of when we extended psyntax to know about the module system, and is now fixed to properly use the scope of the introduced binding instead of the scope of the macro use site. Also, embarrassingly, sometimes macro-introduced top-level variables would use the same generated name. This is because of the strategy discussed in the manual, "Hygiene and the Top-Level"; sometimes the hashes would collide, for example if two definitions were the same in the beginning and only differed long into the end. This has been fixed to ensure name uniqueness. * New interfaces and functionality ** R6RS custom textual ports Guile finally supports this venerable R6RS interface; see "Custom Ports" in the manual for more. These ports are suspendable (see "Non-Blocking I/O"). Also new in this release, custom binary ports are now suspendable as well. ** New "soft port" interface Instead of using R6RS custom textual ports, we recommend
GNU Guile 3.0.8 released
[A resend, as original mail accidentally had two From headers; apologies for any duplicates!] We are delighted to announce GNU Guile release 3.0.8, the latest in the 3.0 stable release series. The Guile 3.0.8 release mixes maintenance and optimizations that were landed since the previous 3.0.7 stable release in May 2021. See the NEWS extract at the end of the mail for full details. Compared to the previous stable series (2.2.x), Guile 3.0 adds support for just-in-time native code generation, speeding up all Guile programs. The Guile web page is located at http://gnu.org/software/guile/, and among other things, it contains a copy of the Guile manual and pointers to more resources. Guile is an implementation of the Scheme programming language, packaged for use in a wide variety of environments. In addition to implementing the R5RS, R6RS, and R7RS Scheme standards, Guile includes full access to POSIX system calls, networking support, multiple threads, dynamic linking, a foreign function call interface, powerful string processing, and HTTP client and server implementations. Guile can run interactively, as a script interpreter, and as a Scheme compiler to VM bytecode. It is also packaged as a library so that applications can easily incorporate a complete Scheme interpreter/VM. An application can use Guile as an extension language, a clean and powerful configuration language, or as multi-purpose "glue" to connect primitives provided by the application. It is easy to call Scheme code from C code and vice versa. Applications can add new functions, data types, control structures, and even syntax to Guile, to create a domain-specific language tailored to the task at hand. Guile 3.0.8 can be installed in parallel with Guile 2.2.x; see http://www.gnu.org/software/guile/manual/html_node/Parallel-Installations.html. A more detailed NEWS summary follows these details on how to get the Guile sources. Here are the compressed sources: http://ftp.gnu.org/gnu/guile/guile-3.0.8.tar.lz (6MB) http://ftp.gnu.org/gnu/guile/guile-3.0.8.tar.xz (6MB) http://ftp.gnu.org/gnu/guile/guile-3.0.8.tar.gz (10MB) Here are the GPG detached signatures[*]: http://ftp.gnu.org/gnu/guile/guile-3.0.8.tar.lz.sig http://ftp.gnu.org/gnu/guile/guile-3.0.8.tar.xz.sig http://ftp.gnu.org/gnu/guile/guile-3.0.8.tar.gz.sig Use a mirror for higher download bandwidth: http://www.gnu.org/order/ftp.html Here are the SHA256 checksums: 96f5263f89dda2d757fe1129e489e672f5a0b7540ee6710ec7ef352120d8af52 guile-3.0.8.tar.lz daa7060a56f2804e9b74c8d7e7fe8beed12b43aab2789a38585183fcc17b8a13 guile-3.0.8.tar.xz f25ae0c26e911af1b5005292d4f56621879f74d6958b30741cf67d8b6feb2016 guile-3.0.8.tar.gz [*] Use a .sig file to verify that the corresponding file (without the .sig suffix) is intact. First, be sure to download both the .sig file and the corresponding tarball. Then, run a command like this: gpg --verify guile-3.0.8.tar.gz.sig If that command fails because you don't have the required public key, then run this command to import it: gpg --keyserver keys.gnupg.net --recv-keys 4FD4D288D445934E0A14F9A5A8803732E4436885 and rerun the 'gpg --verify' command. This release was bootstrapped with the following tools: Autoconf 2.71 Automake 1.16.2 Libtool 2.4.6 Gnulib v0.1-4855-g8f4538a53d Makeinfo 6.7 An extract from NEWS follows. Changes in 3.0.8 (since 3.0.7): * Notable changes ** Cross-module inlining Although historically Guile has treated modules as glorified hash tables, most modules are actually _declarative_ -- they just define functions and variables and provide them for other modules to use, and don't manipulate modules as first-class objects. See "Declarative Modules" in the manual, for more discussion. Since version 3.0.0, Guile has taken advantage of declarative semantics to allow a top-level definition to be inlined within its uses in the same compilation unit, provided the binding is never assigned and defined exactly once. Guile 3.0.8 extends this to allow some exported declarative definitions to be inlined into other modules. This facility is mostly transparent to the user and is enabled at the default -O2 optimization level. "Small" definitions are available for cross-module inlining (-Oinlinable-exports, included at -O2). The actual inlining decision is performed by Guile's partial evaluation pass (the -Ocross-module-inlining modifier to -Opeval, included at -O2 also), subject to effort and size growth counters. Note however that as with macros, when a definition changes in module A, a separately compiled module B that uses that definition doesn't automatically get recompiled. This is a limitation in Guile that we would like to fix. As another limitation, cross-module inlining is only available for imports from modules which have already been compiled at -O2 (or otherwise with -Oinlinable-exports). When determining whether to enable this facility by default, we weighed the usabil
GNU Guile 3.0.8 released
We are delighted to announce GNU Guile release 3.0.8, the latest in the 3.0 stable release series. The Guile 3.0.8 release mixes maintenance and optimizations that were landed since the previous 3.0.7 stable release in May 2021. See the NEWS extract at the end of the mail for full details. Compared to the previous stable series (2.2.x), Guile 3.0 adds support for just-in-time native code generation, speeding up all Guile programs. The Guile web page is located at http://gnu.org/software/guile/, and among other things, it contains a copy of the Guile manual and pointers to more resources. Guile is an implementation of the Scheme programming language, packaged for use in a wide variety of environments. In addition to implementing the R5RS, R6RS, and R7RS Scheme standards, Guile includes full access to POSIX system calls, networking support, multiple threads, dynamic linking, a foreign function call interface, powerful string processing, and HTTP client and server implementations. Guile can run interactively, as a script interpreter, and as a Scheme compiler to VM bytecode. It is also packaged as a library so that applications can easily incorporate a complete Scheme interpreter/VM. An application can use Guile as an extension language, a clean and powerful configuration language, or as multi-purpose "glue" to connect primitives provided by the application. It is easy to call Scheme code from C code and vice versa. Applications can add new functions, data types, control structures, and even syntax to Guile, to create a domain-specific language tailored to the task at hand. Guile 3.0.8 can be installed in parallel with Guile 2.2.x; see http://www.gnu.org/software/guile/manual/html_node/Parallel-Installations.html. A more detailed NEWS summary follows these details on how to get the Guile sources. Here are the compressed sources: http://ftp.gnu.org/gnu/guile/guile-3.0.8.tar.lz (6MB) http://ftp.gnu.org/gnu/guile/guile-3.0.8.tar.xz (6MB) http://ftp.gnu.org/gnu/guile/guile-3.0.8.tar.gz (10MB) Here are the GPG detached signatures[*]: http://ftp.gnu.org/gnu/guile/guile-3.0.8.tar.lz.sig http://ftp.gnu.org/gnu/guile/guile-3.0.8.tar.xz.sig http://ftp.gnu.org/gnu/guile/guile-3.0.8.tar.gz.sig Use a mirror for higher download bandwidth: http://www.gnu.org/order/ftp.html Here are the SHA256 checksums: 96f5263f89dda2d757fe1129e489e672f5a0b7540ee6710ec7ef352120d8af52 guile-3.0.8.tar.lz daa7060a56f2804e9b74c8d7e7fe8beed12b43aab2789a38585183fcc17b8a13 guile-3.0.8.tar.xz f25ae0c26e911af1b5005292d4f56621879f74d6958b30741cf67d8b6feb2016 guile-3.0.8.tar.gz [*] Use a .sig file to verify that the corresponding file (without the .sig suffix) is intact. First, be sure to download both the .sig file and the corresponding tarball. Then, run a command like this: gpg --verify guile-3.0.8.tar.gz.sig If that command fails because you don't have the required public key, then run this command to import it: gpg --keyserver keys.gnupg.net --recv-keys 4FD4D288D445934E0A14F9A5A8803732E4436885 and rerun the 'gpg --verify' command. This release was bootstrapped with the following tools: Autoconf 2.71 Automake 1.16.2 Libtool 2.4.6 Gnulib v0.1-4855-g8f4538a53d Makeinfo 6.7 An extract from NEWS follows. Changes in 3.0.8 (since 3.0.7): * Notable changes ** Cross-module inlining Although historically Guile has treated modules as glorified hash tables, most modules are actually _declarative_ -- they just define functions and variables and provide them for other modules to use, and don't manipulate modules as first-class objects. See "Declarative Modules" in the manual, for more discussion. Since version 3.0.0, Guile has taken advantage of declarative semantics to allow a top-level definition to be inlined within its uses in the same compilation unit, provided the binding is never assigned and defined exactly once. Guile 3.0.8 extends this to allow some exported declarative definitions to be inlined into other modules. This facility is mostly transparent to the user and is enabled at the default -O2 optimization level. "Small" definitions are available for cross-module inlining (-Oinlinable-exports, included at -O2). The actual inlining decision is performed by Guile's partial evaluation pass (the -Ocross-module-inlining modifier to -Opeval, included at -O2 also), subject to effort and size growth counters. Note however that as with macros, when a definition changes in module A, a separately compiled module B that uses that definition doesn't automatically get recompiled. This is a limitation in Guile that we would like to fix. As another limitation, cross-module inlining is only available for imports from modules which have already been compiled at -O2 (or otherwise with -Oinlinable-exports). When determining whether to enable this facility by default, we weighed the usability problems of stale inlined bindings against the benefit of allowing module boundaries to no
thoughts on targetting the web
Hi :) A brain-dump tonight. I was thinking about what it would mean for Guile to target the web. The goal would be for Guile to be a useful language for programming client-side interaction on web pages. Now, there are a number of takes for Schemes of various stripes to target JS. I haven't made a full survey but my starting point is that in a Guile context, we basically want the Guile Scheme language, which is to say: 1. Tail calls. The solution must include proper tail calls. 2. Multiple values. The solution should have the same semantics that Guile has for multiple-value returns, including implicit truncation. 3. Varargs. The solution should allow for rest arguments, keyword arguments, and the like. 4. Delimited continuations. The solution should allow non-local exit from any function, and should allow for capturing stack slices to partial continuations, and should allow those continuations to be instantiated multiple times if desired. We should be able to target fibers to the web. Otherwise, why bother? 5. Garbage collection. *We should re-use the host GC*. Although it would be possible to manage a heap in linear memory, that has retention problems due to cycles between the Guile heap and the JS heap. 6. Numbers. We should support Scheme's unbounded integers. Requires bignums in the limit case, but thankfully JS has bignums now. We will still be able to unbox in many cases though. 7. Target existing web. We shouldn't depend on some future WebAssembly or JS proposal -- the solution should work in the here and now and only get better as features are added to the web platform. >From a UX perspective, I would expect we would generally want whole-program compilation with aggressive DCE / tree-shaking, producing a single JS or WebAssembly artifact at build-time. But that's a later issue. I have thought about this off and on over the years but in the end was flummoxed about how to meet all requirements. However recently I think I have come up with a solution for most of these: (1) In JS, tail calls are part of ES2015, but not implemented in all browsers. In WebAssembly, they are a future plan, but hard for various reasons. So in practice the solution for today's web is to use a universal trampoline and make all calls tail calls -- i.e. call all functions via: while true: call(pop()) Whenever we can target proper tail calls, this will only get faster. (2) Neither JS nor WebAssembly have the truncation semantics we want. Therefore we will return values via an explicit array, and then the continuation will be responsible for consuming those values and binding any needed variables. (3) Although JS does have varargs support, WebAssembly does not. But we can actually use the same solution here as we use for (1) and (2) -- pass arguments on an explicit array + nvalues, and relying on the function to parse them appropriately. In this way we can get Guile keyword arguments. This also makes the type of Scheme functions uniform, which is important in WebAssembly contexts. (4) This is the interesting bit! As hinted in (1), we will transform the program such that all calls are tail calls. This is a form of minimal CPS conversion -- minimal in the sense that functions are broken up into the minimum number of pieces to ensure the all-tail-calls property. Non-tail calls transform to tail calls by saving their continuation's state on a stack, which is the same as stack allocation for these continuations. The continuation of the call pops the saved state from the stack. Because the stack is explicit, we can manipulate it as data: slice it to capture a delimited continuation, drop values to make a non-local exit, splat a saved slice back on to compose a captured delimited continuation with the current continuation, and so on. Therefore we have the necessary primitives to implement delimited continuations as a library. (5) Scheme needs a representation that can store any value. In JS this is easy because JS is untyped. For WebAssembly, I think I would lean on externref for this purpose, which effectively boxes all values. There are no fixnums in the current WebAssembly spec, so this is suboptimal, and we have to call out to JS to evaluate type predicates and access fields. But, support for structured GC-managed types is coming to WebAssembly, which will speed up object access. (6) The easy solution here is to make JS numbers, which are doubles at heart, represent flonums, and use JS bignums for Scheme integers. Fine. (7) This principle has been taken into account in (1)-(6). Now, a note on the transformation described in (4), which I call "tailification". The first step of tailification computes the set of
heads-up: development branch is now "main"
Hi, Just a heads-up that the main Guile development branch is now named "main". If you have a recent git and a fairly conventional setup, it will be sufficient to switch your working copy by doing: git fetch origin git checkout main Then you can continue to "git pull" as usual. One note, the default branch that you get when you clone is currently still "master". This should be resolved soon. In the meanwhile I will keep "master" sync'd with "main". Andy
GNU Guile 3.0.7 released
We are humbled to announce GNU Guile release 3.0.7, the latest in the 3.0 stable release series. This is a fix-up release. Compared to Guile 3.0.6, Guile 3.0.7 fixes a number of bugs, many of which were introduced in the 3.0 series. Compared to the previous stable series (2.2.x), Guile 3.0 adds support for just-in-time native code generation, speeding up all Guile programs. See the NEWS extract at the end of the mail for full details. The Guile web page is located at http://gnu.org/software/guile/, and among other things, it contains a copy of the Guile manual and pointers to more resources. Guile is an implementation of the Scheme programming language, packaged for use in a wide variety of environments. In addition to implementing the R5RS, R6RS, and R7RS Scheme standards, Guile includes full access to POSIX system calls, networking support, multiple threads, dynamic linking, a foreign function call interface, powerful string processing, and HTTP client and server implementations. Guile can run interactively, as a script interpreter, and as a Scheme compiler to VM bytecode. It is also packaged as a library so that applications can easily incorporate a complete Scheme interpreter/VM. An application can use Guile as an extension language, a clean and powerful configuration language, or as multi-purpose "glue" to connect primitives provided by the application. It is easy to call Scheme code from C code and vice versa. Applications can add new functions, data types, control structures, and even syntax to Guile, to create a domain-specific language tailored to the task at hand. Guile 3.0.7 can be installed in parallel with Guile 2.2.x; see http://www.gnu.org/software/guile/manual/html_node/Parallel-Installations.html. A more detailed NEWS summary follows these details on how to get the Guile sources. Here are the compressed sources: http://ftp.gnu.org/gnu/guile/guile-3.0.7.tar.lz (10MB) http://ftp.gnu.org/gnu/guile/guile-3.0.7.tar.xz (13MB) http://ftp.gnu.org/gnu/guile/guile-3.0.7.tar.gz (21MB) Here are the GPG detached signatures[*]: http://ftp.gnu.org/gnu/guile/guile-3.0.7.tar.lz.sig http://ftp.gnu.org/gnu/guile/guile-3.0.7.tar.xz.sig http://ftp.gnu.org/gnu/guile/guile-3.0.7.tar.gz.sig Use a mirror for higher download bandwidth: http://www.gnu.org/order/ftp.html Here are the SHA256 checksums: f02166205ced31651d27bd037f947e199a442545ca73f913907c69469ddd7b54 guile-3.0.7.tar.lz f57d86c70620271bfceb7a9be0c81744a033f08adc7ceba832c9917ab3e691b7 guile-3.0.7.tar.xz c7935b7a29e42443f6a35d35cf20ffa7d028c399303f872cd1219598a83656ae guile-3.0.7.tar.gz [*] Use a .sig file to verify that the corresponding file (without the .sig suffix) is intact. First, be sure to download both the .sig file and the corresponding tarball. Then, run a command like this: gpg --verify guile-3.0.7.tar.gz.sig If that command fails because you don't have the required public key, then run this command to import it: gpg --keyserver keys.gnupg.net --recv-keys 4FD4D288D445934E0A14F9A5A8803732E4436885 and rerun the 'gpg --verify' command. This release was bootstrapped with the following tools: Autoconf 2.71 Automake 1.16.2 Libtool 2.4.6 Gnulib v0.1-4551-ga3a946f670 Makeinfo 6.7 An extract from NEWS follows. Changes in 3.0.7 (since 3.0.6) * New interfaces and functionality ** More O_* POSIX constants are now defined in Scheme Guile now defines constants such as `O_NOFOLLOW', `O_CLOEXEC', `O_TMPFILE', and more on platforms that support them. These may be passed as arguments to procedures such as `open' and `open-fdes'. * Bug fixes ** Fix bugs introduced in 3.0.6 with Scheme `read` re-write ** Fix deadlock after `primitive-fork' (#41948) ** Fix duplicates handlers for interfaces that use interfaces (#43025) ** Fix compile-psyntax.scm for (language tree-il canonicalize) removal ** Fix prompt compilation bug (#48098) ** Fix R7RS include-library-declarations, cond-expand (#40252) ** Fix --enable-mini-gmp on FreeBSD and other targets ** Fix excessive compile times for vectors >16k elements long ** Fix use of literal tree-il as source language (#45131) ** Fix SRFI-64 test-end to not remove globally-installed test runner
GNU Guile 3.0.6 released
We are pleased to announce GNU Guile release 3.0.6, the latest in the
3.0 stable release series.
Compared to the previous release in the 3.0 series, Guile 3.0.6 improves
source-location information for compiled code, removes the dependency on
libltdl, fixes some important bugs, adds an optional bundled "mini-gmp"
library, as well as the usual set of minor optimizations and bug fixes.
Compared to the previous stable series (2.2.x), Guile 3.0 adds support
for just-in-time native code generation, speeding up all Guile programs.
See the NEWS extract at the end of the mail for full details.
The Guile web page is located at http://gnu.org/software/guile/, and
among other things, it contains a copy of the Guile manual and pointers
to more resources.
Guile is an implementation of the Scheme programming language, packaged
for use in a wide variety of environments. In addition to implementing
the R5RS, R6RS, and R7RS Scheme standards, Guile includes full access to
POSIX system calls, networking support, multiple threads, dynamic
linking, a foreign function call interface, powerful string processing,
and HTTP client and server implementations.
Guile can run interactively, as a script interpreter, and as a Scheme
compiler to VM bytecode. It is also packaged as a library so that
applications can easily incorporate a complete Scheme interpreter/VM.
An application can use Guile as an extension language, a clean and
powerful configuration language, or as multi-purpose "glue" to connect
primitives provided by the application. It is easy to call Scheme code
from C code and vice versa. Applications can add new functions, data
types, control structures, and even syntax to Guile, to create a
domain-specific language tailored to the task at hand.
Guile 3.0.6 can be installed in parallel with Guile 2.2.x; see
http://www.gnu.org/software/guile/manual/html_node/Parallel-Installations.html.
A more detailed NEWS summary follows these details on how to get the
Guile sources.
Here are the compressed sources:
http://ftp.gnu.org/gnu/guile/guile-3.0.6.tar.lz (10MB)
http://ftp.gnu.org/gnu/guile/guile-3.0.6.tar.xz (13MB)
http://ftp.gnu.org/gnu/guile/guile-3.0.6.tar.gz (21MB)
Here are the GPG detached signatures[*]:
http://ftp.gnu.org/gnu/guile/guile-3.0.6.tar.lz.sig
http://ftp.gnu.org/gnu/guile/guile-3.0.6.tar.xz.sig
http://ftp.gnu.org/gnu/guile/guile-3.0.6.tar.gz.sig
Use a mirror for higher download bandwidth:
http://www.gnu.org/order/ftp.html
Here are the SHA256 checksums:
2e5b9e3e56a967a58ae591053a65c1851875bf5e06c60caab409d5647cff4975
guile-3.0.6.tar.lz
e2bd83c2077d721356e7579ca33097a13a20e2b7eda6c2362ee1166fbc845d28
guile-3.0.6.tar.xz
000fc43c1b0a5cfbd85b67e01afd58e847bd1f279e3439bb7db37282b0459f56
guile-3.0.6.tar.gz
[*] Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact. First, be sure to download both the .sig file
and the corresponding tarball. Then, run a command like this:
gpg --verify guile-3.0.6.tar.gz.sig
If that command fails because you don't have the required public key,
then run this command to import it:
gpg --keyserver keys.gnupg.net --recv-keys
4FD4D288D445934E0A14F9A5A8803732E4436885
and rerun the 'gpg --verify' command.
This release was bootstrapped with the following tools:
Autoconf 2.71
Automake 1.16.2
Libtool 2.4.6
Gnulib v0.1-4551-ga3a946f670
Makeinfo 6.7
An extract from NEWS follows.
Changes in 3.0.6 (since 3.0.5)
* Notable changes
** Reimplement dynamic library loading ("dlopening") without libltdl
Guile used to load dynamic libraries with libltdl, a library provided by
the Libtool project.
Libltdl provided some compatibility benefits when loading shared
libraries made with older toolchains on older operating systems.
However, no system from the last 10 years or so appears to need such a
thick compatibility layer.
Besides being an unmaintained dependency of limited utility, libltdl
also has the negative aspect that in its search for libraries to load,
it could swallow useful errors for libraries that are found but not
loadable, instead showing just errors for search path candidates that
are not found.
Guile now implements dynamic library loading directly in terms of the
standard "dlopen" interface, providing a limited shim for platforms with
similar functionality exposed under different names (MinGW).
This change has a few practical impacts to Guile users. There is a new
library search path variable, `GUILE_EXTENSIONS_PATH'. Also, errors when
loading a library fails now have better errors. And Guile no longer has
a libltdl dependency.
Although Guile no longer uses libltdl, for backwards compatibility Guile
still adds `LTDL_LIBRARY_PATH' to the loadable library search path, and
includes ad-hoc logic to support uninstalled dynamically loadable
libraries via also adding the ".libs" subdirectories of
`LTDL_LIBRARY_PATH' elements. See "Foreign Libraries" in the
documentation for a full discuss
Re: Preventing file descriptor leak to execl'd processes
On Sat 06 Mar 2021 17:55, Marius Bakke writes: > $ ls -l /proc/self/fd > lrwx-- 1 marius marius 64 Mar 6 17:41 0 -> /dev/pts/18 > lrwx-- 1 marius marius 64 Mar 6 17:41 1 -> /dev/pts/18 > lrwx-- 1 marius marius 64 Mar 6 17:41 2 -> /dev/pts/18 > lr-x-- 1 marius marius 64 Mar 6 17:41 3 -> /proc/9940/fd > > $ /tmp/test-shell -c 'ls -l /proc/self/fd' > lrwx-- 1 marius marius 64 Mar 6 17:41 0 -> /dev/pts/18 > lrwx-- 1 marius marius 64 Mar 6 17:41 1 -> /dev/pts/18 > lrwx-- 1 marius marius 64 Mar 6 17:41 2 -> /dev/pts/18 > lr-x-- 1 marius marius 64 Mar 6 17:41 3 -> /proc/9951/fd > lr-x-- 1 marius marius 64 Mar 6 17:41 7 -> /tmp/test-shell > > I've managed to work around it by setting FD_CLOEXEC on it: > > (port-for-each (lambda (port) >(let ((name (port-filename port)) > (self (car (program-arguments > (when (and name (string=? name self)) >(fcntl port F_SETFD (logior FD_CLOEXEC >(fcntl port F_GETFD))) > > But it seems heavy-handed. Is there an easier way to access the "script > port"? Perhaps Guile itself should make it FD_CLOEXEC by default? I think Guile itself should make the load port FD_CLOEXEC / O_CLOEXEC. More broadly there are a number of file descriptors in Guile that should be cloexec but aren't yet. Want to make a patch for the load port ? :) Andy
Re: Some non-standard O_* flags are missing
Heya :) On Tue 09 Mar 2021 22:47, Maxime Devos writes: > On Tue, 2021-03-09 at 21:36 +0100, Andy Wingo wrote: >> Hi :) Sure, would be happy to accept a patch for these. > See <https://debbugs.gnu.org/cgi/bugreport.cgi?bug=46220> > and messages above. I'll look at updating NEWS and the manual > later. Great! I agree it's weird that O_NOTRANS is defined but 0 on GNU/Linux. Perhaps better not to define it if it's zero? Dunno tho. >> It's adding >> more definitions to the base environment, which is usually a negative, >> but we'll have to find some kind of module solution for all of these >> flags at some point. > > Would defining O_* in a new module (ice-9 open-flags) > (or the existing module (ice-9 posix) maybe?) be acceptable? > Or alternatively, a syntax (open-flag SYMBOL) that expands to > the flag's value -- Let's punt for now :) >> If you do send a patch, please update the manual >> and NEWS also. > > The manual only documents O_RDONLY, O_WRONLY, O_RDWR, > O_APPEND and O_CREAT currently, and points the reader > at glibc's manual for additional flags. Ah yes. Probably the flags (possibly) supported by Guile should be in the manual. Document them if you have a moment? :) Otherwise just document the new ones. Andy
Re: Linking to libraries in the ELF?
On Fri 29 Jan 2021 17:20, Maxime Devos writes: > I've got this crazy idea a few days ago, about the FFI interface. > When a program written in C is compiled with some shared libraries, > these libraries are referred to in special [insert ELF terminology > here] of the binary. I wonder if it would be reasonable and feasible > to do something similar for compiles guile modules? These sections of executables or shared libraries are actually *interpreted* by the system dynamic library loader, ld.so. Guile has its own loader, independent of ld.so, and so can implement this -- or do more flexible things. In any case we probably shouldn't tie ourselves to the particular implementation of a particular loader. Of course, dynamic-link was... not a great interface. But with the recent changes in git I think it can be fine (see "Foreign Libraries" in the manual). For the FFI the bigger problem IMO is generating faster foreign procedure bindings -- i.e. farther on the compilation-vs-interpretation scale. Cheers, Andy
Re: [PATCH 1/1] scm_set_source_properties_x: optimize if only name, line, and/or col
On Sun 17 Jan 2021 23:55, Bruce Korb writes:
> [Can I] swap out this code for something better now? I've disliked
> this code for years.
>
>> static SCM
>> ag_scm_c_eval_string_from_file_line(
>> char const * pzExpr, char const * pzFile, int line)
Since Guile 2.0.1 you can
(use-modules (ice-9 eval-string))
(eval-string str #:file file #:line line)
>From C:
SCM_KEYWORD (k_file, "file");
SCM_KEYWORD (k_line, "line");
...
SCM ans =
scm_call_5 (scm_c_public_ref ("ice-9 eval-string", "eval-string"),
scm_from_utf8_string (str),
k_file, scm_from utf8_string (file),
k_line, scm_from_int (line));
Regards,
Andy
Re: Some non-standard O_* flags are missing
Hi :) Sure, would be happy to accept a patch for these. It's adding more definitions to the base environment, which is usually a negative, but we'll have to find some kind of module solution for all of these flags at some point. If you do send a patch, please update the manual and NEWS also. Cheers, Andy On Sun 31 Jan 2021 22:13, Maxime Devos writes: > Hi guilers, > > I noticed the following open flags are not defined: > O_NOFOLLOW, O_TMPFILE, O_IGNORE_CTTY, O_NOLINK, > O_SHLOCK, O_EXLOCK, O_ASYNC, O_NOATIME. > > Some of these are Hurd-specific, Linux-specific > and BSD-specific. I'm particularily interested > in O_NOFOLLOW, O_TMPFILE, O_IGNORE_CTTY, O_NOLINK > and O_NOATIME, the others don't matter for me, > though they may be useful for others. > > Could extra O_* flags be exported to Guile (in libguile/filesys.c) > on systems where they are defined? > > Greetings, > Maxime
Re: [PATCH] Support cross compilation
Hi :) Which library is this for? It doesn't appear to be guile itself. Andy On Sun 14 Feb 2021 10:16, Christopher Baines writes: > With these changes, I was able to cross-compile guile-lib to the GNU > Hurd, and use part of the library at least. > > * configure.ac: Set GUILE_TARGET when cross compiling. > * am/guile.mk: Pass GUILE_TARGET to guild compile. > --- > am/guile.mk | 2 +- > configure.ac | 4 > 2 files changed, 5 insertions(+), 1 deletion(-) > > diff --git a/am/guile.mk b/am/guile.mk > index 3adfc4f..f08e58d 100644 > --- a/am/guile.mk > +++ b/am/guile.mk > @@ -47,4 +47,4 @@ GUILE_WARNINGS = -Wunbound-variable -Warity-mismatch > -Wformat > SUFFIXES = .scm .go > .scm.go: > $(AM_V_GEN)$(top_builddir)/pre-inst-env \ > - guild compile $(GUILE_WARNINGS) -o "$@" "$<" > + guild compile $(GUILE_TARGET) $(GUILE_WARNINGS) -o "$@" "$<" > diff --git a/configure.ac b/configure.ac > index 07be121..0aa812f 100644 > --- a/configure.ac > +++ b/configure.ac > @@ -67,6 +67,10 @@ GUILE_GLOBAL_SITE_DIR > GUILE_SITE_CCACHE_DIR > GUILE_FLAGS > > +if test \"$cross_compiling\" != no; then > + GUILE_TARGET=\"--target=$host_alias\" > + AC_SUBST([GUILE_TARGET]) > +fi > > AC_ARG_WITH( >[guile-site],
GNU Guile 3.0.5 released
We are delighted to announce GNU Guile release 3.0.5, the latest in the 3.0 stable release series. Compared to the previous release in the 3.0 series, Guile 3.0.5 can compile chained "if" expressions into the equivalent of what a C compiler does with "switch". It also adds some new warning passes. Compared to the previous stable series (2.2.x), Guile 3.0 adds support for just-in-time native code generation, speeding up all Guile programs. See the NEWS extract at the end of the mail for full details. The Guile web page is located at http://gnu.org/software/guile/, and among other things, it contains a copy of the Guile manual and pointers to more resources. Guile is an implementation of the Scheme programming language, packaged for use in a wide variety of environments. In addition to implementing the R5RS, R6RS, and R7RS Scheme standards, Guile includes full access to POSIX system calls, networking support, multiple threads, dynamic linking, a foreign function call interface, powerful string processing, and HTTP client and server implementations. Guile can run interactively, as a script interpreter, and as a Scheme compiler to VM bytecode. It is also packaged as a library so that applications can easily incorporate a complete Scheme interpreter/VM. An application can use Guile as an extension language, a clean and powerful configuration language, or as multi-purpose "glue" to connect primitives provided by the application. It is easy to call Scheme code from C code and vice versa. Applications can add new functions, data types, control structures, and even syntax to Guile, to create a domain-specific language tailored to the task at hand. Guile 3.0.5 can be installed in parallel with Guile 2.2.x; see http://www.gnu.org/software/guile/manual/html_node/Parallel-Installations.html. A more detailed NEWS summary follows these details on how to get the Guile sources. Here are the compressed sources: http://ftp.gnu.org/gnu/guile/guile-3.0.5.tar.lz (10MB) http://ftp.gnu.org/gnu/guile/guile-3.0.5.tar.xz (12MB) http://ftp.gnu.org/gnu/guile/guile-3.0.5.tar.gz (21MB) Here are the GPG detached signatures[*]: http://ftp.gnu.org/gnu/guile/guile-3.0.5.tar.lz.sig http://ftp.gnu.org/gnu/guile/guile-3.0.5.tar.xz.sig http://ftp.gnu.org/gnu/guile/guile-3.0.5.tar.gz.sig Use a mirror for higher download bandwidth: http://www.gnu.org/order/ftp.html Here are the SHA256 checksums: a484eeffbd4d655b0c05b1382df8d40f1e561f7e71b963065762f6a6a497c675 guile-3.0.5.tar.lz 2d76fb023d2366126a5fac04704f9bd843846b80cccba6da5d752318b03350f1 guile-3.0.5.tar.xz 222046009a20b432ffa7c11b8d5a1d9ad0d8627be05cc1e8af612bc54ba2ea85 guile-3.0.5.tar.gz [*] Use a .sig file to verify that the corresponding file (without the .sig suffix) is intact. First, be sure to download both the .sig file and the corresponding tarball. Then, run a command like this: gpg --verify guile-3.0.5.tar.gz.sig If that command fails because you don't have the required public key, then run this command to import it: gpg --keyserver keys.gnupg.net --recv-keys 4FD4D288D445934E0A14F9A5A8803732E4436885 and rerun the 'gpg --verify' command. This release was bootstrapped with the following tools: Autoconf 2.69 Automake 1.16.2 Libtool 2.4.6 Gnulib v0.1-1157-gb03f418 Makeinfo 6.7 An extract from NEWS follows. Changes in 3.0.5 (since 3.0.4) * New interfaces and functionality ** O(1) compilation of `case' and related expressions Guile now optimizes chains of eq? comparisons to constants, resulting in O(1) dispatch time, regardless of the length of the chain. This optimization is also unlocked in many cases for `match' expressions with many similar clauses whose first differentiator are constants. ** New (ice-9 copy-tree) module This module includes the `copy-tree' procedure that was previously implemented in C and present in the default `(guile)' module. See "Copying" in the manual. ** New warning: use-before-definition This analysis, enabled at `-W1', issues warnings for programs that use top-level variables before they are defined. ** New warning: non-idempotent-definition This analysis, enabled at `-W1', issues warnings for programs that whose use of a variable is ambiguous. For example, in the program: (define saved-add +) (define + error) The intention would seem to be to "save" the value of the base `+' procedure, then override it locally. However if this program is ever loaded twice, then the second time it is loaded, `+' will be taken from the local binding instead of the import. Users that want this kind of behavior should either use lexical bindings instead of top-level bindings, or otherwise rename important clobbered bindings via modules. * New deprecations ** `copy-tree' in the default environment, and `scm_copy_tree' from C Import the `(ice-9 copy-tree)' module instead. ** `unbound-variable-analysis`, `macro-use-before-definition-analysis` These bindings from `(language tree-il
Re: The size of ‘.go’ files
Hi :) On Tue 09 Jun 2020 18:09, Ludovic Courtès writes: > Andy Wingo skribis: > >> The guile.arities section starts with a sorted array of fixed-size >> headers, then is followed by a sequence of ULEB128 references to local >> variable names, including non-arguments. The size is a bit perplexing, >> I agree. I can think of a number of ways to encode that section >> differently but we'd need to understand a bit more about it and why the >> baseline compiler is significantly different. > > ‘.guile.arities’ size should be proportional to the number of > procedures, right? Additionally, if there are only/mostly thunks, the > string table for argument names should be small if not empty. For N > thunks, I would expect roughly N 28-byte headers + NxM UL128, say 100 > bytes per thunk; there’s 1000 of them, so we should be ~100,000 bytes. > This is roughly what we get observe with the baseline compiler. Yes but that doesn't mean that you can directly compare baseline to CPS -- CPS has many more intermediate names than baseline for non-argument locals, all of which end up getting entries in the arities section. Andy
Re: GNU Guile 3.0.3 released
On Tue 23 Jun 2020 11:36, Chris Vine writes: > On Tue, 23 Jun 2020 10:05:51 +0200 > Ludovic Courtès wrote: >> Hi Chris, >> >> Chris Vine skribis: >> >> > On Sun, 21 Jun 2020 23:04:03 +0200 >> > Ludovic Courtès wrote: >> >> We are delighted to announce GNU Guile release 3.0.3, the third bug-fix >> >> release of the new 3.0 stable series. This release represents 170 >> >> commits by 17 people since version 3.0.2. See the NEWS excerpt that >> >> follows for full details. >> > [snip] >> > >> > This has a libguile so ABI jump from libguile-3.0.so.1 to >> > libguile-3.0.so.3, which breaks my binaries linked to libguile. Is that >> > normal for a micro update in the stable release series and if so can >> > there be some warning in the announcement? >> >> Yes, it means you need to relink those binaries. >> >> I was hesitant about the SONAME: the ABI jump was unnecessary unless in >> ‘--disable-deprecated’ builds. I erred on the side of cautiousness: >> >> >> https://git.savannah.gnu.org/cgit/guile.git/commit/?id=5d052c87bd8f0fd894e67f0bebd4fa6f6160d83c > > Hi, > > Ah right. There must have been two SO breaks between guile-3.0.2 and > guile-3.0.3. > > It's a nuisance having SO bumps on micro releases and I wonder if that > could be included in the announcement so that you don't first notice it > when stuff fails to run? I think I agree with Chris. The intention is certainly to have a stable ABI within a stable series, so 3.0.3 should have the same CURRENT. It's certainly correct that a --disable-deprecated 3.0.3 build has a different ABI than 3.0.2, and if that were what we were looking at, we would indeed need the CURRENT version bump; but I think the premise is wrong: we do *not* have a stable ABI in --disable-deprecated builds, and we never have. Otherwise we wouldn't ever be able to deprecate anything within a stable series. WDYT about a quick 3.0.4 that restores the CURRENT ? Cheers, Andy
Re: The size of ‘.go’ files
Hi :)
A few points of information :)
On Fri 05 Jun 2020 22:50, Ludovic Courtès writes:
> [Sorting] the ELF sections of a .go file by size; for ‘python-xyz.go’,
> I get this:
>
> $13 = ((".rtl-text" . 3417108)
> (".guile.arities" . 1358536)
> (".data" . 586912)
> (".rodata" . 361599)
> (".symtab" . 117000)
> (".debug_line" . 97342)
> (".debug_info" . 54519)
> (".guile.frame-maps" . 47114)
> ("" . 1344)
> (".guile.arities.strtab" . 681)
> ("" . 232)
> (".shstrtab" . 229)
> (".dynamic" . 112)
> (".debug_str" . 87)
> (".strtab" . 75)
> (".debug_abbrev" . 65)
> (".guile.docstrs.strtab" . 1)
> ("" . 0)
> (".guile.procprops" . 0)
> (".guile.docstrs" . 0)
> (".debug_loc" . 0))
>
> More than half of those 6 MiB is code, and more than 1 MiB is
> “.guile.arities” (info "(guile) Object File Format"), which is
> surprisingly large; presumably the file only contains thunks (the
> ‘thunked’ fields of ).
The guile.arities section starts with a sorted array of fixed-size
headers, then is followed by a sequence of ULEB128 references to local
variable names, including non-arguments. The size is a bit perplexing,
I agree. I can think of a number of ways to encode that section
differently but we'd need to understand a bit more about it and why the
baseline compiler is significantly different.
> Stripping the .debug_* sections (if that works) clearly wouldn’t help.
I believe that it should eventually be possible to strip guile.arities,
fwiw.
> So I guess we could generate less code (reduce ‘.rtl-text’), perhaps by
> tweaking ‘define-record-type*’, but I have little hope there.
Hehe :) As you mention later:
> With 3.0.3-to-be and -O1, python-xyz.go weighs in at 3.4 MiB instead of
> 5.9 MiB! Here’s the section size distribution:
>
> $4 = ((".rtl-text" . 2101168)
> (".data" . 586392)
> (".rodata" . 360703)
> (".guile.arities" . 193106)
> (".symtab" . 117000)
> (".debug_line" . 76685)
> (".debug_info" . 53513)
> ("" . 1280)
> (".guile.arities.strtab" . 517)
> ("" . 232)
> (".shstrtab" . 211)
> (".dynamic" . 96)
> (".debug_str" . 87)
> (".strtab" . 75)
> (".debug_abbrev" . 56)
> (".guile.docstrs.strtab" . 1)
> ("" . 0)
> (".guile.procprops" . 0)
> (".guile.docstrs" . 0)
> (".debug_loc" . 0))
> scheme@(guile-user)> (stat:size (stat go))
> $5 = 3519323
>
> “.rtl-text” is 38% smaller and “.guile.arities” is almost a tenth of
> what it was.
The difference in the text are the new baseline intrinsics,
e.g. $vector-ref. It goes in the opposite direction from instruction
explosion, which sought to (1) make the JIT compiler easier by
decomposing compound operations into their atomic parts, (2) make the
optimizer learn more information from flow rather than type-checking
side effects, and (3) allow the optimizer to eliminate / hoist / move
the component pieces of macro-operations.
However in the baseline compiler (2) and (3) aren't possible because
there is no optimizer on that level, and therefore the result is
actually a lose -- 10 micro-ops cost more than 1 macro-op because of
stack traffic overhead, which isn't currently mitigated by the JIT (1).
So instruction explosion is residual code explosion, which should pay
off in theory, but not for the baseline compiler. So I added new
intrinsics for e.g. $vector-ref et al. Thus the smaller code size.
I am not sure what causes the significantly different .guile.arities
size!
> Something’s going on here! Thoughts?
There are more possibilities for making code size smaller, e.g. having
two equivalent encodings for bytecode, where one is smaller:
https://webkit.org/blog/9329/a-new-bytecode-format-for-javascriptcore/
Or it could be that if we could do register allocation for a
target-dependent fixed set of registers in bytecode already, that could
decrease minimum instruction size, making more instructions fit into
single 32-bit words. Would be nice if the JIT could rely on the
bytecode compiler to already have done register allocation, and reify
corresponding debug information. Just a thought though, and not really
appropriate to the baseline compiler.
Cheers,
Andy
Re: [PATCH] Avoid malloc/free/dynwind for POSIX subrs that take strings
On Sun 17 May 2020 23:46, Ludovic Courtès writes:
> The libunistring functions can take a pre-allocated buffer, but they
> always malloc a fresh one if needed. So the best we could do is have a
> ‘scm_to_stringn’ variant that takes a buffer, but it’s not guaranteed
> that it’ll actually be used. All in all, it seems the added complexity
> is not warranted. The worst case of ‘scm_locale_string_data’ is also
> rare enough.
>
> Thoughts?
Interesting. Probably we want to make a public
scm_to_{,locale_,utf8_}gc_string{,n} API and use that as a fallback.
GC-managed character buffers are less error-prone and probably just as
fast.
We can mostly avoid the double-copy by inline conversions, as we do with
UTF-8. For narrow strings scm_to_gc_stringn can always run iconv in a
mode that just calculates output byte size; surely equivalent
functionality is available from unistring, also.
Andy
Re: [PATCH] Add tree-il optimizations for equal? on char and number literals
On Wed 13 May 2020 23:16, Linus Björnstam writes: > On the latest master equal? was not reduced to eq? on chars in the repl > for things like ,opt (define (a b) (equal? b #\b)). This turned out to be that I had broken the REPL command. The optimizer was working fine though. Fixed now :) Cheers, Andy
Re: [PATCH] Add tree-il optimizations for equal? on char and number literals
Hi :) On Wed 13 May 2020 13:20, Linus Björnstam writes: > Hi there! > > Aleix and I noticed that equal? has a lot higher overhead than eqv? on > chars, which means using (ice-9 match) for chars was suboptimal. This > patch fixes that. I think we can be a little more simple here. Scheme doesn't specify what (eqv? #\a x) is, but in Guile it is equivalent to (eq? #\a x), and our compiler should be free to turn the portable eqv? invocation into eq?. But as the comment on line 416 says, we should really do this in peval and not in the expander. So you nerd-sniped me ;) I just pushed a patch that did this. While looking, I found this: > + (make-conditional src (make-primcall src prim (list a b)) > + (make-primcall src prim (cons b rest)) > + (make-const src #f)) This was in the original code but is wrong: if "b" has a side-effect, it will happen twice. I have fixed it in git. Thanks for the debugging and patch! Andy
Re: [PATCH] Avoid malloc/free/dynwind for POSIX subrs that take strings
Hi :) On Wed 29 Apr 2020 22:22, Ludovic Courtès writes: > As discussed on IRC, the patch below arranges so that subrs that take > strings and pass them to syscall wrappers can avoid the > malloc/free/dynwind overhead. This gives a 10% speedup on a tight loop > that calls these subrs: Neat optimization. Since it's internal, no problem from me. My concern is only about calcifying aspects of our C / GC interface in API. > On IRC, Andy mentioned concerns that the SCM could disappear and thus, > our the internal pointer returned by ‘scm_locale_string_data’ wouldn’t > be enough to prevent the stringbuf from being GC’d. I would suggest doing the whole optimization and being sure to do scm_remember_upto_here_1 on the SCM value at use sites. Andy
Re: [PATCH, v2] Fix build on platforms where the stack grows upwards
On Tue 10 Mar 2020 23:58, John David Anglin writes: > On 2020-02-08 9:07 a.m., Ludovic Courtès wrote: >> John Paul Adrian Glaubitz skribis: >> >>> * libguile/continuations.c (scm_dynthrow): Fix missing mra >>>parameter to grow_stack for SCM_STACK_GROWS_UP. >> Applied, thanks! > I believe that this change was also applied to the 2.2 stable branch > (v2.2.7). There is no mra > parameter on 2.2. Indeed; fixed. Thanks for the fix and the note, and thanks Ludo for getting the fix into 3.0.1 :) Andy
a hack to do -- jitmaps
Hey :) I thought of a thing that I don't have time to implement right now: perf jitmap support in Guile. Basically, the "perf" Linux tool is a widely-available instruction-level and microarchitectural profiler. It's great: you run "perf record guile foo.scm", and then you run "perf report", and you get instruction-level info on the profile. You can grab call graphs, see performance counters, all kinds of things. Good stuff. With run-time code generation though, perf needs some help from the program generating code. The most basic step is what's known as "perf maps", placed in /tmp/perf-$PID.map. These are super-basic and just identify code ranges with functions. I just pushed a patch doing this. The more useful thing is what's known as "jitmaps". See https://lwn.net/Articles/633846/, and https://chromium.googlesource.com/v8/v8/+/refs/heads/master/src/diagnostics/perf-jit.cc. It's totally underdocumented, but it actually saves the JIT code to a file, which is needed to have instruction-level profiling. It also adds support for unwinding frames and source info. Anyway if anyone wants to take this task in the next few weeks, LMK :) Andy
Re: Segfault while building on 64-bit Cygwin
Aah, you all are amazing -- thank you!! Applied and merged. Cheers, Andy On Mon 17 Feb 2020 20:27, Charles Stanhope writes: > On 2/16/20, Charles Stanhope wrote: >> On 2/16/20, Mike Gran wrote: >>> >>> I can confirm that Charles's patch, plus another one line patch >>> to define CPU_SETSIZE, is enough to get Guile 3.0.x to build and run >>> on my box. All tests pass except strptime in French, and the absence >>> of crypt. This is a 64-bit build. >> >> Mike, thanks for going further with the Guile build. The CPU_SETSIZE >> issue was what was hanging me up from compiling before Andy's comment >> got me to look at lightening. I assumed I had some configuration, >> package, or compiler issue. Good to know there's a simple fix. >> >> Just a further warning to anyone watching, that patch I posted is a >> real hack job just to test my theory of the cause of the segfault. I >> would expect it to fail when you have fewer than four arguments in a >> JITed function call. I wouldn't try doing much else with that Guile >> build besides run the tests. :) > > I had a little bit more time to look into the lightening > implementation last night. I've attached a patch that is less horrible > and more correct than my previous one. It reserves the stack space > regardless of the number of parameters and appears to work. But I'm > new to the lightening code base, so I'm not convinced it is the > correct solution. It's just the solution I was left with after my time > ran out. I wanted to post this patch as a replacement to the prior one > in case people did want to do more testing with Guile 3.0 on Cygwin > x64. > > With that, I will let more experienced people come up with the > appropriate solution. Happy hacking, everybody! > > -- > Charles > > diff --git a/lightening/x86.c b/lightening/x86.c > index 965191a..bdd26e1 100644 > --- a/lightening/x86.c > +++ b/lightening/x86.c > @@ -328,6 +328,10 @@ reset_abi_arg_iterator(struct abi_arg_iterator *iter, > size_t argc, >memset(iter, 0, sizeof *iter); >iter->argc = argc; >iter->args = args; > +#if __CYGWIN__ && __X64 > + // Reserve slots on the stack for 4 register parameters (8 bytes each). > + iter->stack_size = 32; > +#endif > } > > static void
Re: Segfault while building on 64-bit Cygwin
On Mon 20 Jan 2020 18:22, Mike Gran writes: > On Mon, Jan 20, 2020 at 11:38:35AM -0500, John Cowan wrote: >> Yes, gladly, but I don't know how to get one in this context. Do I need to >> add some flags to the Makefile, and if so, where? (It's a twisty maze of >> passages, all different.) . Note that this *is* a build with JIT enabled; >> when I disable it using the env variable, there are no errors and 3.0.0 >> works fine. >> >> Also, it may take some time, as I have to rebuild my Windows system. > > I also tried building Guile 3.0.0 on Cygwin 3.1.x. The failure comes from > trying to parse compiled .go files. > > The last time that I had this sort of problem, it was because the > O_BINARY flag was dropped or missing when writing .go files, leading > to CR+LF characters in the compiled files. And I diagnosed it by > byte-comparing Linux-compiled .go files with Cygwin-compiled .go > files, and by looking for CR+LF combinations in the compiled .go > files. > > I don't know if that is what is happening here, but, I'll check that > next time I have a chance. Given that John said that compilation went fine with GUILE_JIT_THRESHOLD=-1, I think perhaps this problem may have been fixed in the past. My suspicions are that this issue is an ABI issue with lightening that could perhaps be reproduced by: git co https://gitlab.com/wingo/lightening cd lightening make -C tests test-native Of course any additional confirmation is useful and welcome! Cheers, Andy
Re: CPU and GC cost of bignums
Hi :)
Nice investigation! Perhaps slot-allocation should track live variables
using something that's not bigints, but who knows.
On Wed 05 Feb 2020 17:29, Ludovic Courtès writes:
> /* The next three functions (custom_libgmp_*) are passed to
> mp_set_memory_functions (in GMP) so that memory used by the digits
> themselves is known to the garbage collector. This is needed so
> @@ -237,19 +227,20 @@ finalize_bignum (void *ptr, void *data)
> static void *
> custom_gmp_malloc (size_t alloc_size)
> {
> - return scm_malloc (alloc_size);
> + return scm_gc_malloc (alloc_size, "GMP");
> }
>
> static void *
> custom_gmp_realloc (void *old_ptr, size_t old_size, size_t new_size)
> {
> - return scm_realloc (old_ptr, new_size);
> + return scm_gc_realloc (old_ptr, old_size, new_size, "GMP");
> }
>
> static void
> custom_gmp_free (void *ptr, size_t size)
> {
> - free (ptr);
> + /* Do nothing: all memory allocated by GMP is under GC control and
> + will be freed when needed. */
> }
I think this makes sense to me as a short-term fix. The down-side is
that limbs can alias Scheme objects.
In the long-term I think we should be representing bignums as
pointerless objects whose first word is the tag and a word count,
followed by inline "limbs" (in the sense of
https://gmplib.org/manual/Nomenclature-and-Types.html#Nomenclature-and-Types).
Generally we can use the low-level API to work on these
(https://gmplib.org/manual/Low_002dlevel-Functions.html#Low_002dlevel-Functions),
and if we need to use mpz_t, we can easily create an mpz_t that points
to these values.
Cheers,
Andy
Re: Removal of hppa support
Greets, On Tue 04 Feb 2020 12:02, John Paul Adrian Glaubitz writes: > On 1/27/20 4:46 PM, Andy Wingo wrote: >> William is correct. HPPA support is not gone from Guile; and indeed >> it's good to hear from you :) I wasn't sure there were any IA64 users >> remaining. > > It fails to build from source on Debian hppa, however: Thanks for these reports and apologies for the breakages. Looking forward to the patches. > I fixed the preprocessor conditional, but then I'm running into another > issue: > > continuations.c: In function 'capture_auxiliary_stack': > continuations.c:152:7: warning: implicit declaration of function 'getcontext' > [-Wimplicit-function-declaration] Probably a missing include for ucontext.h. >> If someone would like to write an IA64 backend for Lightening, I would >> be happy to accept it :) The beginnings of one are there in the git >> history. > > Ok. I assume that applies to alpha, hppa, m68k, powerpc*, riscv*, sparc* as > well. Yes indeed! Andy
Re: Removal of hppa support
Hi :) On Sun 26 Jan 2020 07:19, William ML Leslie writes: > On Sun, 26 Jan 2020, 4:46 pm William ML Leslie, > wrote: > On Sun, 26 Jan 2020, 8:20 am John Paul Adrian Glaubitz, > wrote: > > > I noticed that you recently purged hppa support from guile [1]. > > This change does not remove hppa support from guile, only support for > the jit. > > As an ia64 user, i'm a bit sad that we lost jit too - but i don't run > any performance-sensitive guile jobs on that system. > > Maybe /lost/ is the wrong word. We won't be benefitting from the work > the lightning team have done in supporting hacker-friendly > architectures in guile 3. This jit is completely new. William is correct. HPPA support is not gone from Guile; and indeed it's good to hear from you :) I wasn't sure there were any IA64 users remaining. Initially in Guile I planned to use GNU Lightning, in part because of its great platform support. However it turned out to not be the right thing, and reluctantly I ended up doing something that was more like a rewrite than a refactor. In that context I personally don't have the budget to write the IA64 backend. So, Guile 3 still runs on IA64, just without JIT support. If someone would like to write an IA64 backend for Lightening, I would be happy to accept it :) The beginnings of one are there in the git history. Andy
GNU Guile 3.0.0 released
We are delighted to announce GNU Guile release 3.0.0, the first in the new 3.0 stable release series. Compared to the previous stable series (2.2.x), Guile 3.0 adds support for just-in-time native code generation, speeding up all Guile programs. See the NEWS extract at the end of the mail for full details. The Guile web page is located at http://gnu.org/software/guile/, and among other things, it contains a copy of the Guile manual and pointers to more resources. Guile is an implementation of the Scheme programming language, packaged for use in a wide variety of environments. In addition to implementing the R5RS, R6RS, and R7RS Scheme standards, Guile includes full access to POSIX system calls, networking support, multiple threads, dynamic linking, a foreign function call interface, powerful string processing, and HTTP client and server implementations. Guile can run interactively, as a script interpreter, and as a Scheme compiler to VM bytecode. It is also packaged as a library so that applications can easily incorporate a complete Scheme interpreter/VM. An application can use Guile as an extension language, a clean and powerful configuration language, or as multi-purpose "glue" to connect primitives provided by the application. It is easy to call Scheme code from C code and vice versa. Applications can add new functions, data types, control structures, and even syntax to Guile, to create a domain-specific language tailored to the task at hand. Guile 3.0.0 can be installed in parallel with Guile 2.2.x; see http://www.gnu.org/software/guile/manual/html_node/Parallel-Installations.html. A more detailed NEWS summary follows these details on how to get the Guile sources. Here are the compressed sources: http://ftp.gnu.org/gnu/guile/guile-3.0.0.tar.lz (10MB) http://ftp.gnu.org/gnu/guile/guile-3.0.0.tar.xz (12MB) http://ftp.gnu.org/gnu/guile/guile-3.0.0.tar.gz (21MB) Here are the GPG detached signatures[*]: http://ftp.gnu.org/gnu/guile/guile-3.0.0.tar.lz.sig http://ftp.gnu.org/gnu/guile/guile-3.0.0.tar.xz.sig http://ftp.gnu.org/gnu/guile/guile-3.0.0.tar.gz.sig Use a mirror for higher download bandwidth: http://www.gnu.org/order/ftp.html Here are the SHA256 checksums: e28c450d11f7335769f607214f9b79547400881ddbbc9805ccf3ce2121aa97e0 guile-3.0.0.tar.lz c9138d6595a9f69bf9733d0bc2d3b9f3d8b79f35f289006912b3361cb0510c75 guile-3.0.0.tar.xz 049b286849fa9764fac781071c4ec9daef707da51e5050ffb498d7bf6422da2f guile-3.0.0.tar.gz [*] Use a .sig file to verify that the corresponding file (without the .sig suffix) is intact. First, be sure to download both the .sig file and the corresponding tarball. Then, run a command like this: gpg --verify guile-3.0.0.tar.gz.sig If that command fails because you don't have the required public key, then run this command to import it: gpg --keyserver keys.gnupg.net --recv-keys 4FD4D288D445934E0A14F9A5A8803732E4436885 and rerun the 'gpg --verify' command. This release was bootstrapped with the following tools: Autoconf 2.69 Automake 1.16.1 Libtool 2.4.6 Gnulib v0.1-1157-gb03f418 Makeinfo 6.7 An extract from NEWS follows. Changes in 3.0.0 (since the stable 2.2 series): * Notable changes ** Just-in-time code generation Guile programs now run up to 4 times faster, relative to Guile 2.2, thanks to just-in-time (JIT) native code generation. Notably, this brings the performance of "eval" as written in Scheme back to the level of "eval" written in C, as in the days of Guile 1.8. See "Just-In-Time Native Code" in the manual, for more information. JIT compilation will be enabled automatically and transparently. To disable JIT compilation, configure Guile with `--enable-jit=no' or `--disable-jit'. The default is `--enable-jit=auto', which enables the JIT if it is available. See `./configure --help' for more. JIT compilation is enabled by default on x86-64, i686, ARMv7, and AArch64 targets. ** Lower-level bytecode Relative to the virtual machine in Guile 2.2, Guile's VM instruction set is now more low-level. This allows it to express more advanced optimizations, for example type check elision or integer devirtualization, and makes the task of JIT code generation easier. Note that this change can mean that for a given function, the corresponding number of instructions in Guile 3.0 may be higher than Guile 2.2, which can lead to slowdowns when the function is interpreted. We hope that JIT compilation more than makes up for this slight slowdown. ** Interleaved internal definitions and expressions allowed It used to be that internal definitions had to precede all expressions in their bodies. This restriction has been relaxed. If an expression precedes an internal definition, it is treated as if it were a definition of an unreferenced variable. For example, the expression `(foo)' transforms to the equivalent of `(define _ (begin (foo) #f))', if it precedes other definitions. This change improves the readability of Gui
Re: %module-public-interface
On Wed 15 Jan 2020 07:50, dsm...@roadrunner.com writes: > I'm porting some old code to 3.0 and I've come across this: > > (define-module (bobotpp bot)) > > (set-module-uses! %module-public-interface > > (list (module-ref (resolve-module '(guile-user) #f) > > 'the-bot-module))) > > What is the 3.0 equivalent? I'm not sure what the above is trying to do. As mentioned in IRC, (module-public-interface (current-module)). (This change was part of 2.0.) Cheers, Andy
Re: GNU Guile 2.9.9 Released [beta]
On Tue 14 Jan 2020 22:48, Stefan Israelsson Tampe writes: > Strange that I did not dee this error before in the 2.x series > ever. Isn't it so that for procedures define in a (let () ...) the > case you are mentioning happened before but I was on the impression > that no inlining was done for defines on different places in the > module before This is correct, yes. The declarative bindings optimization makes toplevel bindings more like letrec bindings, which exposes them to this other optimization. My point was that since Guile 2.0, procedure identity has not been firmly guaranteed in all cases. Andy
Re: GNU Guile 2.9.9 Released [beta]
On Tue 14 Jan 2020 21:13, Stefan Israelsson Tampe writes: > Okey, here is another case that fails with the patch that prevents identity > misses for toplevels e.g we need similar fixes for anonymous functions. > > (define-module (b) > #:export (q)) > > (define h (make-hash-table)) > (define (method f) > (hash-set! h f 1) > f) > (define q (method (lambda x x))) > > (pk (hash-ref h q)) > > This fails with (#f) > > I solved this in my code by placing the method function in another module. Interestingly, this case is not really related to the declarative bindings optimization, letrectification, or other things. It's the same as: (let ((h (make-hash-table))) (define (method f) (hash-set! h f 1) f) (let* ((q (let ((f (lambda x x))) (method f (pk (hash-ref h q I.e. no top-level bindings are needed. This prints #f in releases as old as 2.0.14 and probably older :) It optimizes as: (let* ((h (make-hash-table)) (q (begin (hash-set! h (lambda x x) 1) (lambda x x (pk (hash-ref h q))) So, not a recent change. Of course we can discuss whether it's the right thing or not! Andy
Re: GNU Guile 2.9.9 Released [beta]
On Tue 14 Jan 2020 17:03, Mikael Djurfeldt writes: > Hmm... it seems like both Stefan and you have interpreted my post > exactly the opposite way compared to how it was meant. :) Hah! My apologies :) > What I wanted to say is that I probably prefer you to *reverse* the > recent patch because I prefer to have good optimization also when > procedures are referenced by value in more than one non-operator > position. I prefer this over having (eq? p p) => #t for the reasons I > stated. I understand this also! However what Stefan is saying echoes what I've heard from other people. There are some cases where eta-converting all lexical procedure references helps nobody -- it makes (eqv? p p) be false in places where many people expect it would be true, without enabling significant optimizations. In that case, the choice is pretty clear. But if there are significant optimizations left on the table, I would hesitate a lot before chasing an ideal of procedure identity in cases where the procedure's behavior cannot possibly differ. Anyway that's not where we are currently, thankfully! Cheers, Andy
Re: GNU Guile 2.9.9 Released [beta]
On Tue 14 Jan 2020 15:47, Stefan Israelsson Tampe writes: > Yes, your patch is indicating when you should use the same identity > e.g. all uses of procedures in a higher order position such as an > argument or a return value. But I looked at your patch, which looks > good but I saw that for operator position you decrease the count. Why? > Also you are free to use one version in argument / return position and > another one in operator position the only limit is to use the same > identity for on operator position. Finally don't you need to count > usage of returning a variable as well? Not sure what the bug is. Do you have a test case that shows the behavior that you think is not good? Andy
Re: GNU Guile 2.9.9 Released [beta]
On Tue 14 Jan 2020 13:18, Mikael Djurfeldt writes: > I probably don't have a clue about what you are talking about (or at > least hope so), but this---the "eq change"---sounds scary to me. > > One of the *strengths* of Scheme is that procedures are first class > citizens. As wonderfully show-cased in e.g. SICP this can be used to > obtain expressive and concise programs, where procedures can occur > many times as values outside operator position. > > I would certainly *not* want to trade in an important optimization > step in those cases to obtain intuitive procedure equality. The risk > is then that you would tend to avoid passing around procedures as > values. Is this true? (eq? '() '()) What about this? (eq? '(a) '(a)) And yet, are datums not first-class values? What does being first-class have to do with it? Does it matter whether it's eq? or eqv? What about: (eq? (lambda () 10) (lambda () 10)) What's the difference? What's the difference in the lambda calculus between "\x.f x" and "f"? What if in a partial evaluator, you see a `(eq? x y)`, and you notice that `x' is bound to a lambda expression? Can you say anything about the value of the expression? Does comparing procedures for equality mean anything at all? https://cs-syd.eu/posts/2016-01-17-function-equality-in-haskell Anyway :) All that is a bit of trolling on my part. What I mean to say is that instincts are tricky when it comes to object identity, equality, equivalence, and especially all of those combined with procedures. The R6RS (what can be more Schemely than a Scheme standard?) makes this clear. All that said, with the recent patch, I believe that Guile 3.0's behavior preserves your intuitions. Bug reports very welcome! Andy
Re: GNU Guile 2.9.9 Released [beta]
On Mon 13 Jan 2020 22:32, Stefan Israelsson Tampe writes: > In current guile (eq? f f) = #f for a procedure f. Try: Note that procedure equality is explicitly unspecified by R6RS. Guile's declarative modules optimization took advantage of this to eta-expand references to declaratively-bound top-level lambda expressions. This unlocks the "well-known" closure optimizations: closure elision, contification, and so on. However, the intention with the eta expansion was really to prevent the (module-add! mod 'foo foo) from making the procedure not-well-known. If that's the only reference to `foo' outside the operator position, procedure identity for `foo' is kept, because it's only accessed outside the module. But then I realized thanks to your mail (and the three or four times that people stumbled against this beforehand) that we can preserve the optimizations and peoples' intuitions about procedure equality if we restrict eta-expansion to those procedures that are only referenced by value in at most a single position. It would be best to implement the eta-expansion after peval; doing it where we do leaves some optimization opportunities on the table. But I have implemented this change in git and it should fix this issue. Comparative benchmark results: https://wingolog.org/pub/guile-2.9.7-vs-guile-2.9.9-with-eq-change-microbenchmarks.png Regards, Andy
Re: GNU Guile 2.9.9 Released [beta]
On Mon 13 Jan 2020 09:39, Andy Wingo writes: > Compared to the previous prerelease (2.9.7), Guile 2.9.8 fixes a number > of bugs. Obviously this was meant to be 2.9.9 versus 2.9.8 :) > Changes since alpha 2.9.8 (since 2.9.7): Here too :)
GNU Guile 2.9.9 Released [beta]
We are pleased to announce GNU Guile release 2.9.9. This is the ninfth and probably final pre-release of what will eventually become the 3.0 release series. Compared to the current stable series (2.2.x), the future Guile 3.0 adds support for just-in-time native code generation, speeding up all Guile programs. See the NEWS extract at the end of the mail for full details. Compared to the previous prerelease (2.9.7), Guile 2.9.8 fixes a number of bugs. The current plan is to make a 3.0.0 final release on 17 January 2020. If there's nothing wrong with this prerelease, 3.0.0 will be essentially identical to 2.9.9. With that in mind, please test and make sure the release works on your platform! Please send any build reports (success or failure) to guile-devel@gnu.org, along with platform details. You can file a bug by sending mail to bug-gu...@gnu.org. The Guile web page is located at http://gnu.org/software/guile/, and among other things, it contains a copy of the Guile manual and pointers to more resources. Guile is an implementation of the Scheme programming language, packaged for use in a wide variety of environments. In addition to implementing the R5RS, R6RS, and R7RS Scheme standards, Guile includes a module system, full access to POSIX system calls, networking support, multiple threads, dynamic linking, a foreign function call interface, powerful string processing, and HTTP client and server implementations. Guile can run interactively, as a script interpreter, and as a Scheme compiler to VM bytecode. It is also packaged as a library so that applications can easily incorporate a complete Scheme interpreter/VM. An application can use Guile as an extension language, a clean and powerful configuration language, or as multi-purpose "glue" to connect primitives provided by the application. It is easy to call Scheme code >From C code and vice versa. Applications can add new functions, data types, control structures, and even syntax to Guile, to create a domain-specific language tailored to the task at hand. Guile 2.9.9 can be installed in parallel with Guile 2.2.x; see http://www.gnu.org/software/guile/manual/html_node/Parallel-Installations.html. A more detailed NEWS summary follows these details on how to get the Guile sources. Here are the compressed sources: http://alpha.gnu.org/gnu/guile/guile-2.9.9.tar.lz (10MB) http://alpha.gnu.org/gnu/guile/guile-2.9.9.tar.xz (12MB) http://alpha.gnu.org/gnu/guile/guile-2.9.9.tar.gz (21MB) Here are the GPG detached signatures[*]: http://alpha.gnu.org/gnu/guile/guile-2.9.9.tar.lz.sig http://alpha.gnu.org/gnu/guile/guile-2.9.9.tar.xz.sig http://alpha.gnu.org/gnu/guile/guile-2.9.9.tar.gz.sig Use a mirror for higher download bandwidth: http://www.gnu.org/order/ftp.html Here are the SHA256 checksums: 59f136e5db36eba070cc5e68784e632dc2beae4b21fd6c7c8ed2c598cc992efc guile-2.9.9.tar.lz bf71920cfa23e59fc6257bee84ef4dfeccf4f03e96bb8205592e09f9dbff2969 guile-2.9.9.tar.xz eafe394cf99d9dd1ab837e6d1b9b2b8d9f0cd13bc34e64ca92456ce1bc2b1925 guile-2.9.9.tar.gz [*] Use a .sig file to verify that the corresponding file (without the .sig suffix) is intact. First, be sure to download both the .sig file and the corresponding tarball. Then, run a command like this: gpg --verify guile-2.9.9.tar.gz.sig If that command fails because you don't have the required public key, then run this command to import it: gpg --keyserver keys.gnupg.net --recv-keys 4FD4D288D445934E0A14F9A5A8803732E4436885 and rerun the 'gpg --verify' command. This release was bootstrapped with the following tools: Autoconf 2.69 Automake 1.16.1 Libtool 2.4.6 Gnulib v0.1-1157-gb03f418 Makeinfo 6.7 An extract from NEWS follows. Changes since alpha 2.9.8 (since 2.9.7): * Notable changes ** `define-module' #:autoload no longer pulls in the whole module One of the ways that a module can use another is "autoloads". For example: (define-module (a) #:autoload (b) (make-b)) In this example, module `(b)' will only be imported when the `make-b' identifier is referenced. However besides the imprecision about when a given binding is actually referenced, this mechanism used to cause the whole imported module to become available, not just the specified bindings. This has now been changed to only import the specified bindings. This is a backward-incompatible change. The fix is to mention all bindings of interest in the autoload clause. Feedback is welcome. ** `guard' no longer unwinds the stack for clause tests SRFI-34, and then R6RS and R7RS, defines a `guard' form that is a shorthand for `with-exception-handler'. The cond-like clauses for the exception handling are specified to run with the continuation of the `guard', while any re-propagation of the exception happens with the continuation of the original `raise'. In practice, this means that one needs full `call-with-continuation' to implement the specified semantics, to be able to unwind the stac
Re: GNU Guile 2.9.8 Released [beta]
On Wed 08 Jan 2020 15:22, Nala Ginrut writes: > In unknown file: >4 (primitive-load-path "artanis/server/server-context" #<…>) > In ice-9/eval.scm: >626:19 3 (_ #) > 155:9 2 (_ #) > In ice-9/boot-9.scm: > 1153:19 1 (_ _ _ _ _ _ _) > 1655:16 0 (raise-exception _ #:continuable? _) > > ice-9/boot-9.scm:1655:16: In procedure raise-exception: > Wrong number of arguments to # ice-9/boot-9.scm:1153:19 (a b c d e f)> > > > Any hint that I can figure out whait's incompatible? Gosh we need to improve this info. Anyway the procedure at boot-9.scm:1153:19 is a record constructor, for a record with 6 fields. Apparently somewhere in (artanis server server-context) is calling it with the wrong number of arguments. What do you use for records? Do you have your own abstraction or do you use R6RS records or something? Perhaps something in artanis relied on the way that R6RS records used to implement single inheritance, as a chain of objects instead of a flat record. Or perhaps the adaptations to R6RS records in Guile introduced a bug. I am interested to know the answer :) Andy
Re: GNU Guile 2.9.8 Released [beta]
On Thu 09 Jan 2020 22:14, Stefan Israelsson Tampe
writes:
> In language/cps/closure-conversion.scm:
> 749:15 5 (_ 2705 _)
> 771:22 4 (lp # …)
> 771:22 3 (lp # …)
> 771:22 2 (lp # …)
> 610:11 1 (allocate-closure _ _ _ _ _ 1)
Nice bug! I wish I had a test case :) Can you try the following patch
and see if it fixes the issue for you?
Cheers,
Andy
diff --git a/module/language/cps/closure-conversion.scm
b/module/language/cps/closure-conversion.scm
index 1452212f0..17a81f674 100644
--- a/module/language/cps/closure-conversion.scm
+++ b/module/language/cps/closure-conversion.scm
@@ -1,6 +1,6 @@
;;; Continuation-passing style (CPS) intermediate language (IL)
-;; Copyright (C) 2013-2019 Free Software Foundation, Inc.
+;; Copyright (C) 2013-2020 Free Software Foundation, Inc.
This library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
@@ -596,6 +596,17 @@ term."
($continue ktag0 src
($primcall 'allocate-words/immediate `(closure . ,(+ nfree 2))
())
+(#(#t 0)
+ (with-cps cps
+ (build-term ($continue k src ($const #f)
+(#(#t 1)
+ ;; A well-known closure of one free variable is replaced
+ ;; at each use with the free variable itself, so we don't
+ ;; need a binding at all; and yet, the continuation
+ ;; expects one value, so give it something. DCE should
+ ;; clean up later.
+ (with-cps cps
+ (build-term ($continue k src ($const #f)
(#(#t 2)
;; Well-known closure with two free variables; the closure is a
;; pair.
@@ -666,17 +677,6 @@ bound to @var{var}, and continue to @var{k}."
(#(#f 0)
(with-cps cps
(build-term ($continue k src ($const-fun kfun)
- (#(#t 0)
- (with-cps cps
- (build-term ($continue k src ($const #f)
- (#(#t 1)
- ;; A well-known closure of one free variable is replaced
- ;; at each use with the free variable itself, so we don't
- ;; need a binding at all; and yet, the continuation
- ;; expects one value, so give it something. DCE should
- ;; clean up later.
- (with-cps cps
- (build-term ($continue k src ($const #f)
(#(well-known? nfree)
;; A bit of a mess, but beta conversion should remove the
;; final $values if possible.
Re: Better HTTPS support in (web client)
On Fri 10 Jan 2020 15:49, Ludovic Courtès writes: > Hello Guilers! > > I’ve pushed a ‘wip-https-client’ branch that contains improvements for > HTTPS support in (web client) that I’d like to be part of Guile 3: > > https://git.savannah.gnu.org/cgit/guile.git/log/?h=wip-https-client Looks nice, sounds like a great thing to merge in! Andy
Re: GNU Guile 2.9.5 Released [beta]
On Tue 07 Jan 2020 00:14, Chris Vine writes: > I wonder if it would surprise the programmer to have the cond > conditionals evaluated in a different dynamic environment from the one > in which the cond consequential is evaluated where there is a > conditional which is true. I entirely agree it's not ideal and can be surprising! I am not sure that there is an "ideal" here though; with-exception-handler is wonderfully expressive but can be verbose, guard is a pleasant abbreviation but how to deal with re-raising from the original context? In the end, "guard" is just a macro over a more general facility. But it's a macro that we expect people to use, and to cover the common case. To that end I think we should make it cheap and avoid rewinding while also preserving the nice characteristic of evaluating cond consequents in the continuation of the "guard" itself. Andy
Re: SHA256 performance with Guile 2.2 vs. Guile 3.0
On Tue 07 Jan 2020 12:08, Ludovic Courtès writes: > Andy Wingo skribis: > >> Concretely I would add a little part of the compiler to the Tree-IL >> phase to serialize a bytecode for the "small" definitions in the module, >> for declarative modules, both public and private (because public >> definitions may alias private definitions). This would be stored as a >> bytevector in an additional field of the module, and the program being >> compiled would be transformed to initialize the "lto" field (placeholder >> name) of the module, so that once the compiled module is loaded, we have >> the inlinable bindings. I think this can be done compatibly. > > OK, sounds great. What are your thoughts about versioning that wire > Tree-IL representation? It would be a little bytecode language, with its own versioning considerations. It would need to have a translation to and from Tree-IL, though not necessarily lossless. It would change only in ABI-compatible ways, using the bytecode version of the Guile doing the compilation as a proxy for what is OK to support. A
Re: GNU Guile 2.9.8 Released [beta]
On Tue 07 Jan 2020 21:00, Stefan Israelsson Tampe writes: > Bump! > > Great, but loading an extension like: > > (catch #t > (lambda () > (throw #t) > (load-extension "libguile-persist" "persist_init") (pk 1)) > (lambda x > (let ((file >(%search-load-path "src/.libs/libguile-persist.so"))) > (if >file >(catch #t > (lambda () >(load-extension file "persist_init")) > (lambda x >(warn > "libguile-persist is not loadable!"))) >(warn > "libguile-persist is not present, did you forget to make it?") > > And then have line like > > (pk 5 serialize-vm-ra) > > Loading this module prints > (5 #f) > > Worked in releases before. Surely this test case can be shortened? All the load-extension bits appear to not be necessary, right? Andy
Re: support for mips32r6
On Fri 30 Aug 2019 10:09, Bruno Haible writes: > A libffcall bug report [1] made me aware of an incompatible change in > the MIPS ISAs. Namely, for jumping to an address given in a register, > starting with mips32r6, the existing 'jr' instruction does not work > any more: it produces an 'illegal instruction'. Thanks for the fix; applied :) Andy
Re: landed r7rs support
On Fri 22 Nov 2019 17:00, Nala Ginrut writes: > On Tue, Nov 19, 2019, 03:34 Amirouche Boubekki > wrote; > > link: > https://www.gnu.org/software/guile/manual/guile.html#Extending-the-Compiler > > In that section, maybe add a note about a web assembly backend? > > +1 > In the past decade, we've gotten some experiences about compiler frontend > around the Guile community. However, Guile is so powerful that we can also > add backend in a elegant way. It's > better to mention it explicitly, and it's deserved to have more information > jn the manual. Done :) Cheers, Andy
Re: GNU Guile 2.9.5 Released [beta]
On Mon 06 Jan 2020 00:26, Chris Vine writes: > I have a 'try' macro which adopts the approach that if an exception > arises, the macro unwinds from the dynamic environment of the code > where the exception arose to the dynamic environment of the call to > 'try', evaluates the cond clauses in that environment, and then if no > cond clause matches re-raises the exception in that environment with > 'raise' (rather than 'raise-continuable'). In other words, it does > stack unwinding in the same way as exception implementations in almost > all other mainstream languages which use exceptions. It would be > trivial to implement this with guile-3.0's with-exception-handler with > its unwind? argument set to true. I am not sure this really matches with this use case: (define (call-with-backtrace thunk) (call/ec (lambda (ret) (with-exception-handler (lambda (exn) (show-backtrace exn) ;; placeholder (ret)) thunk (define (false-on-file-errors thunk) (call/ec (lambda (ret) (with-exception-handler (lambda (exn) (if (file-error? exn) (ret #f) (raise-continuable exn))) thunk (define (foo f) (call-with-backtrace (lambda () (false-on-file-errors f If there's an error while invoking `f' that's not a file error, you want to have remained in the context of the error so you can show a full backtrace. To my mind this is central to the exception handler design. So far so good I think. If I change the implementation of `false-on-file-errors' to be: (define (false-on-file-errors thunk) (guard (exn ((file-error? exn) #f)) (thunk))) I think this change should preserve the not-unwinding environment that `call-with-backtrace' expects. > On the other hand, as you say it does not seem feasible to implement > in guile the R6RS/R7RS requirement to unwind to the environment of the > call to 'guard' when evaluating the cond clauses, and then return to > the environment of the original exception in order to re-raise if no > cond clause matches. It's feasible, just not a good idea IMO. The problem is that call/cc is quite expensive. Additionally that it captures the whole state of the current thread, so a fiber (github.com/wingo/fibers) with a `guard' may error if it is preempted and migrated to a different CPU. > Furthermore such a return is only relevant if the exception is to be > re-raised with 'raise-continuable' instead of 'raise': it is pointless > if the exception is re-raised with 'raise' because with 'raise' you > can never get back there again. FWIW I am not sure how raise-continuable will be used but it's a fairly straightforward thing implementation-wise that doesn't bother me. > I am somewhat influenced by my view of 'raise-continuable'. I don't > like it - how often does anyone use continuable exceptions, which seem > to be a reimplementation of common lisp restarts? I am not sure that they are restarts. A restart to my mind is more like: (define (with-restart name thunk) (let lp () (define tag (make-prompt-tag)) (call-with-prompt tag (lambda () (parameterize ((current-restarts (acons name tag (current-restart (thunk))) (lambda (k) (lp) (define (invoke-restart-by-name name . vals) (match (assoc name (current-restarts)) ((name . tag) (apply abort-to-prompt tag vals So you could invoke a restart within an exception handler but it has nothing to do with whether raise or raise-continuable was used. The continuation captured by the equivalent of common lisp's `restart-case' isn't the continuation that raises the error. Regards, Andy
Re: SHA256 performance with Guile 2.2 vs. Guile 3.0
On Mon 06 Jan 2020 10:47, Ludovic Courtès writes: > Andy Wingo skribis: > >> With cross-module inlining of "small" definitions, I think we would >> solve a lot of this kind of problem. I think we could add this during >> 3.0 and for this reason I would hesitate to apply this patch for 3.0 >> because it changes "fx+" exports to be macros rather than "normal" >> values in the ABI. WDYT? > > I agree that cross-module inlining is the better fix whereas this patch > is the immediate workaround. > > Are you confident that cross-module inlining can happen be added without > introducing incompatibilities over in the 3.0 series? (At first sight > it seems tricky to me, notably because we’d have to store Tree-IL in > object files, which introduces compatibility and thus external > representation versioning considerations.) Concretely I would add a little part of the compiler to the Tree-IL phase to serialize a bytecode for the "small" definitions in the module, for declarative modules, both public and private (because public definitions may alias private definitions). This would be stored as a bytevector in an additional field of the module, and the program being compiled would be transformed to initialize the "lto" field (placeholder name) of the module, so that once the compiled module is loaded, we have the inlinable bindings. I think this can be done compatibly. > If you do, then it’s fine to drop this patch. If conversely > cross-module inlining might take longer, then we can have this patch in > and drop it in 3.2. Your call! (I guess I’m not being that helpful > here. :-)) :) I hesitate to land this patch because we haven't shown that it significantly helps things, it would need to be undone, and it makes the ABI more fragile. So if that's OK let's do nothing :) Cheers, Andy
Re: GNU Guile 2.9.5 Released [beta]
On Sun 01 Dec 2019 21:41, Chris Vine writes: > Is this rewrite, and the new with-exception-handler procedure, an > opportunity to think about standardization of guile's implementation of > the R6RS/R7RS 'guard' form, or at least think about what is wanted for > 'guard'? > > The formal semantics (including specimen implementation) of 'guard' for > R6RS with the corrigendum to §7.1 of the standard library at > http://www.r6rs.org/r6rs-errata.html, and for R7RS without corrigendum > (at §4.2.7 and §7.3, page 72 of the standard), is: > > (i) to evaluate the guard body within a block with its own continuation > (as constructed by call/cc); > > (ii) if an exception is thrown, evaluate the handler (and its cond > clauses) in the dynamic context of the original caller of 'guard' via > that continuation; > > (iii) if no matching cond clause and no else clause is found, return to > the dynamic environment of the original 'raise' and re-raise the > exception with 'raise-continuable', even for non-continuable > exceptions. > > If a fully conforming R6RS/R7RS implementation runs this code: > > (guard (exn [(equal? exn 5) #f]) > (guard (exn [(equal? exn 6) 'never-reached]) > (dynamic-wind > (lambda () (display "in") (newline)) > (lambda () (raise 5)) > (lambda () (display "out") (newline) > > the code evaluates to #f and should print this: > > in > out > in > out > > In chez scheme it does so. In most other implementations (including > guile and racket) it seems to print: > > in > out I really think the standards messed up regarding the specification of "guard": http://scheme-reports.org/mail/scheme-reports/msg03247.html But those ships have sailed and are now lost at sea. Guile currently has two separate implementations of "guard" for SRFI-34 (used by R7RS) and R6RS. It would seem that besides not respecting the specification, the R6RS one is broken, as it expects the "cond" clauses to evaluate to a single value. For SRFI-34 (and R7RS), after the exception refactor, I did a re-write to give a shot at implementing the specified behavior. It works with a caveat: because it uses delimited continuations as the rewind mechanism, and Guile has a limitation that some delimited continuations can't be rewound (if the continuation bounces through C), then re-raising the exception fails because the context can't be rewound. This can cause previously working programs to break! Which makes me think, if call/cc (rather than call-with-prompt / abort-to-prompt) is necessary to implement "guard", we are in a bad place and we should specify something else. I have long thought that the right thing to do is this: we evaluate the "cond" tests in the dynamic environment of the "raise". Then if a test succeeds, we unwind and run the corresponding consequent. That way there's no rewinding. Here's an implementation: (define-syntax guard (syntax-rules (else) ((guard (var (t e e* ...) ...) body body* ...) (let ((tag (make-prompt-tag))) (call-with-prompt tag (lambda () (with-exception-handler (lambda (var) (cond (t (abort-to-prompt tag (lambda () e e* ...))) ...) (raise var)) (lambda () body body* ...))) (lambda (k thunk) (thunk))) Though I think it might be reasonable to use "raise-continuable" instead of "raise" if nothing matches. WDYT? Andy
Re: [PATCH] Add string-replace-substring to (ice-9 string-fun)
On Fri 20 Dec 2019 18:32, lloda writes: > This patch adds string-replace-substring that wingo posted to the > mailing list to (ice-9 strings). This is a commonly used function and a > good implementation isn't trivial, so I think it deserves inclusion. I didn't know that this module existed :) Sure, why not. Please fix the commit message when you push. Andy
Re: [PATCH] Add srfi-171 to guile
Hi :)
Since this is a final SRFI I think there's no problem getting it in.
Some formatting notes follow; since it's your first Guile patch I'm a
bit verbose :) Probably this will miss 3.0.0 but make 3.0.1, FWIW.
On Sun 22 Dec 2019 15:55, Linus Björnstam writes:
> From 7e8d3b22ba5f814c40dbb5ab616a318c0cdc2f3e Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?Linus=20Bj=C3=B6rnstam?=
> Date: Sun, 22 Dec 2019 15:38:34 +0100
> Subject: [PATCH 1/2] Added srfi-171 to guile under the module name (srfi
> srfi-171).
>
> For more info, read the SRFI document:
> https://srfi.schemers.org/srfi-171/srfi-171.html
Needs a note per-file; see other commit log messages. Also please wrap
to 72 characters.
> --- /dev/null
> +++ b/module/srfi/srfi-171.scm
> @@ -0,0 +1,498 @@
> +;
> +;; Copyright 2019 Linus Bj.rnstam
> +;;
I think you've assigned copyright so this can have the standard Guile
copyright block, right?
> +;; This module name is guile-specific. The correct module name is of course
> +;; (srfi 171)
I don't think it's right to say there is a "correct" name. R6RS, R7RS,
and Guile have different naming conventions for SRFI modules and that's
OK.
The style in Guile is generally that block comments like this should be
complete sentences, starting with capital letters and including
terminating punctuation. Generally we do two spaces after periods,
also.
> +(define-module (srfi srfi-171)
> + #:declarative? #t
> + #:use-module (srfi srfi-9)
> + #:use-module ((srfi srfi-43)
> +#:select (vector->list))
> + #:use-module ((srfi srfi-69) #:prefix srfi69:)
> + #:use-module ((rnrs hashtables) #:prefix rnrs:)
> + #:use-module (srfi srfi-171 meta)
> + #:export (rcons reverse-rcons
> + rcount
> + rany
Better to put rcons on its own line so that other exports are also
aligned with the open paren.
> +;; A special value to be used as a placeholder where no value has been set
> and #f
> +;; doesn't cut it. Not exported.
> +
> +(define-record-type
> + (make-nothing)
> + nothing?)
> +(define nothing (make-nothing))
Note that this can be somewhat cheaper as:
(define nothing (list 'nothing))
(define (nothing? x) (eq? x nothing))
> +;; helper function which ensures x is reduced.
Capitalize. FWIW, better done as a docstring:
(define (ensure-reduced x)
"Ensure that @var{x} is reduced."
...)
> +;; helper function that wraps a reduced value twice since reducing functions
> (like list-reduce)
> +;; unwraps them. tconcatenate is a good example: it re-uses it's reducer on
> it's input using list-reduce.
> +;; If that reduction finishes early and returns a reduced value, list-reduce
> would "unreduce"
> +;; that value and try to continue the transducing process.
Capitalize and limit to 80 characters wide.
> +(define (preserving-reduced f)
> + (lambda (a b)
> +(let ((return (f a b)))
> + (if (reduced? return)
> + (reduced return)
> + return
> +
> +
> +
> +
Generally, put one blank line between functions. Two lines can be
between sections. Four is too much :)
> +
> +;; Reducing functions meant to be used at the end at the transducing
> +;; process.;
This is a fairly non-standard comment style, FWIW; consider just
prefixing with ";;;".
> +;; a transducer-friendly cons with the empty list as identity
> +(define rcons
> + (case-lambda
Similar comment regarding docstrings
> +;; Use this as the f in transduce to count the amount of elements passed
> through.
> +;; (transduce (tfilter odd?) tcount (list 1 2 3)) => 2
80 characters, and the example can go in an @example if you like:
(define rcount
(case-lambda
"A transducer that counts the number of elements passing through. \
@example
(transduce (tfilter odd?) tcount (list 1 2 3)) @result{} 2
@end example"
...))
> +(define (make-replacer map)
> + (cond
> + ((list? map)
> +(lambda (x)
> + (let ((replacer? (assoc x map)))
> +(if replacer?
> +(cdr replacer?)
> +x
I generally find this sort of thing better with (ice-9 match):
(match (assoc x map)
((x . replacer) replacer)
(#f x))
> +;; Flattens everything and passes each value through the reducer
> +;; (list-transduce tflatten conj (list 1 2 (list 3 4 '(5 6) 7 8))) => (1 2 3
> 4 5 6 7 8)
80 chars
> +;; I am not sure about the correctness of this. It seems to work.
> +;; we could maybe make it faster?
> +(define (tpartition f)
How could you know about the correctness? Probably a good idea to do
what it takes to be sure and then remove the comment. Regarding speed,
I would remove the comment, if it's slow then people can work on it.
Note that in general comments shouldn't be from a first-person
perspective, because the code will be maintai
Re: GNU Guile 2.9.8 Released [beta]
On Fri 03 Jan 2020 06:34, Nala Ginrut writes: > When I was trying to compile Artanis, the configure threw an error: > > checking for Guile version >= 3.0... configure: error: Guile 3.0 required, > but 2.9.8 found > > > Here's what I put in configure.ac: > GUILE_PKG(2.2 2.3 2.9 3.0) > > My question is "what's the correct config here"? There was a bug in guile.m4 from a few years ago doesn't do the right thing for major version changes. I think you need to update guile.m4 from git, then do: GUILE_PKG([2.2 3.0]) Also note that it's not meaningful to put in alpha releases to GUILE_PKG -- 2.9.7's effective version is 3.0. FWIW I would encourage doing GUILE_PKG([3.0 2.2]) instead, as it's generally the right thing to default to newer Guile release series. Cheers, Andy
Re: Re-exporting a replaced binding
On Fri 03 Jan 2020 19:30, Ludovic Courtès writes: > I’m not sure if this is an intended consequence of > cf08dbdc189f0005cab6f2ec7b23ed9d150ec43d, so I thought I’d share this > example of a practical effect: > > ludo@ribbon /tmp [env]$ cat x.scm > (define-module (x) > #:use-module (srfi srfi-1) > #:re-export (delete)) > ludo@ribbon /tmp [env]$ cat y.scm > (define-module (y) > #:use-module (x)) > > (pk 'delete delete) > ludo@ribbon /tmp [env]$ guile -L . -c '(use-modules (y))' > WARNING: (y): imported module (x) overrides core binding `delete' > > ;;; (delete #) > ludo@ribbon /tmp [env]$ guile --version > guile (GNU Guile) 2.9.8 > > Here ‘delete’ is replaced by srfi-1, but the replaced bit is not > propagated to module (x), even though (x) simply re-exports it. > > Should the #:re-export clause propagate the replace bit, or should > it not? :-) It is a good question :) Before, if you re-exported a #:replace binding, it wasn't possible to have it be exported without the "replace" bit set. After the change it is possible to do either, and the default changes to not replacing. From NEWS: Note to make this change, we had to change the way replacement flags are stored, to being associated with modules instead of individual variable objects. This means that users who #:re-export an imported binding that was already marked as #:replace by another module will now see warnings, as they need to use #:re-export-and-replace instead. The 3.0 behavior differs from 2.2 in this regard, although it's just warnings and not run-time behavior. I am sympathetic to the concern that it can be difficult to make a system that warns/doesn't warn in the same way on 2.2 vs 3.0 but I think the change is the right thing, as the new behavior is more expressive. Because it's a user-visible change it is in NEWS. LMK if you think we need a change here! Andy
Re: SHA256 performance with Guile 2.2 vs. Guile 3.0
On Sat 04 Jan 2020 01:40, Ludovic Courtès writes: > Ludovic Courtès skribis: > >> ludo@ribbon ~/src/guix$ ./pre-inst-env guix environment --pure --ad-hoc >> guile-next guile3.0-hashing -- guile ~/tmp/sha256.scm >> >> ;;; (hash "b33576331465a60b003573541bf3b1c205936a16c407bc69f8419a527bf5c988") >> clock utime stime cutime cstime gctime >> 65.17 89.75 0.45 0.00 0.00 35.63 > > (define fx32xor fxxor) > … >From a speed perspective I think there is one major issue and one minor issue. The major issue is that we don't do cross-module inlining. But now that we have declarative modules, this is a possibility: https://lists.gnu.org/archive/html/guile-devel/2016-03/msg00026.html https://lists.gnu.org/archive/html/guile-devel/2016-03/msg00027.html With cross-module inlining of "small" definitions, I think we would solve a lot of this kind of problem. I think we could add this during 3.0 and for this reason I would hesitate to apply this patch for 3.0 because it changes "fx+" exports to be macros rather than "normal" values in the ABI. WDYT? The minor issue, at least relatively speaking, is that IMO the (rnrs arithmetic fixnums) API is not appropriate for bitwise operations. When you do bitwise operations and you want to ensure that you're within some fixed domain, it's best to do e.g. "(logand x #x)" on operands and results. Guile will optimize this well. The good optimization isn't fixnum vs other kinds of numbers, it's unboxing to raw unsigned integers; and you usually want to exclude negative numbers. fx+ doesn't help with that. Cheers, Andy
GNU Guile 2.9.8 Released [beta]
We are pleased to announce GNU Guile release 2.9.8. This is the eighth and possibly final pre-release of what will eventually become the 3.0 release series. Compared to the current stable series (2.2.x), the future Guile 3.0 adds support for just-in-time native code generation, speeding up all Guile programs. See the NEWS extract at the end of the mail for full details. Compared to the previous prerelease (2.9.7), Guile 2.9.8 fixes a bug in libguile that caused writes to unmapped memory in some circumstances. This problem manifested itself as a failure of Guile to compile itself on some systems, notably Ubuntu 18.04 on x86-64. It also fixes a couple warnings related to SRFI-35. The current plan is to make a 3.0.0 final release on 17 January 2020. We may need another prerelease in the interim. It's a good time to test the prereleases to make sure they work on your platform. Please send any build reports (success or failure) to guile-devel@gnu.org, along with platform details. You can file a bug by sending mail to bug-gu...@gnu.org. The Guile web page is located at http://gnu.org/software/guile/, and among other things, it contains a copy of the Guile manual and pointers to more resources. Guile is an implementation of the Scheme programming language, with support for many SRFIs, packaged for use in a wide variety of environments. In addition to implementing the R5RS Scheme standard, Guile includes a module system, full access to POSIX system calls, networking support, multiple threads, dynamic linking, a foreign function call interface, and powerful string processing. Guile can run interactively, as a script interpreter, and as a Scheme compiler to VM bytecode. It is also packaged as a library so that applications can easily incorporate a complete Scheme interpreter/VM. An application can use Guile as an extension language, a clean and powerful configuration language, or as multi-purpose "glue" to connect primitives provided by the application. It is easy to call Scheme code >From C code and vice versa. Applications can add new functions, data types, control structures, and even syntax to Guile, to create a domain-specific language tailored to the task at hand. Guile 2.9.8 can be installed in parallel with Guile 2.2.x; see http://www.gnu.org/software/guile/manual/html_node/Parallel-Installations.html. A more detailed NEWS summary follows these details on how to get the Guile sources. Here are the compressed sources: http://alpha.gnu.org/gnu/guile/guile-2.9.8.tar.lz (10MB) http://alpha.gnu.org/gnu/guile/guile-2.9.8.tar.xz (12MB) http://alpha.gnu.org/gnu/guile/guile-2.9.8.tar.gz (21MB) Here are the GPG detached signatures[*]: http://alpha.gnu.org/gnu/guile/guile-2.9.8.tar.lz.sig http://alpha.gnu.org/gnu/guile/guile-2.9.8.tar.xz.sig http://alpha.gnu.org/gnu/guile/guile-2.9.8.tar.gz.sig Use a mirror for higher download bandwidth: http://www.gnu.org/order/ftp.html Here are the SHA256 checksums: 3ce11e9dca0f475fa944729d99f33c379fe8962e729bd21a99470249624c71d7 guile-2.9.8.tar.lz 4a8cf663b8bfd435168935c74a8ec434328ffad16230322c64f0ac567dda2c26 guile-2.9.8.tar.xz 31c3d458ff9342db130e27c8d82d2a33912da92845e5ee431b6a125971a823d2 guile-2.9.8.tar.gz [*] Use a .sig file to verify that the corresponding file (without the .sig suffix) is intact. First, be sure to download both the .sig file and the corresponding tarball. Then, run a command like this: gpg --verify guile-2.9.8.tar.gz.sig If that command fails because you don't have the required public key, then run this command to import it: gpg --keyserver keys.gnupg.net --recv-keys 4FD4D288D445934E0A14F9A5A8803732E4436885 and rerun the 'gpg --verify' command. This release was bootstrapped with the following tools: Autoconf 2.69 Automake 1.16.1 Libtool 2.4.6 Gnulib v0.1-1157-gb03f418 Makeinfo 6.5 An extract from NEWS follows. Changes in alpha 2.9.8 (since alpha 2.9.7): * Bug fixes ** Fix bug in which abort_to_prompt used an invalid stack pointer This bug manifested itself as a bootstrap compile error on some systems, notably Ubuntu 18.04 on x86-64, and was due to failing to recalculate a local variable after a possible stack relocation. ** SRFI-35 does a #:re-export-and-replace on `&error' ** SRFI-35 avoids compiler warnings for multiply-defined condition types Changes in alpha 2.9.x (since the stable 2.2 series): * Notable changes ** Just-in-time code generation Guile programs now run up to 4 times faster, relative to Guile 2.2, thanks to just-in-time (JIT) native code generation. Notably, this brings the performance of "eval" as written in Scheme back to the level of "eval" written in C, as in the days of Guile 1.8. See "Just-In-Time Native Code" in the manual, for more information. JIT compilation will be enabled automatically and transparently. To disable JIT compilation, configure Guile with `--enable-jit=no' or `--disable-jit'. The default is `--enable-jit=auto', which enab
Re: Removing the locale warning?
On Sat 14 Dec 2019 16:44, Ludovic Courtès writes: > What about getting rid of the locale warning? > > More generally, I'm in favor of reducing run-time warnings to a bare > minimum, because application users often don’t care about them (plus > they’re not i18n’d), and because application developers cannot silence > them or handle them in a way that is more suitable for the application. > > Thoughts? Funny, I find them really useful as they let me know when my environment isn't correctly configured, so I can then take action. How can we resolve these two use cases? Andy
draft 3.0 release schedule
Hi, I've been trying to put out releases on an every-other-Friday schedule. Due to holidays I will probably bump this next one (2.9.7) up to this Friday. Current plan would then be to do 2.9.8 on 3 January 2020, and then if all goes well 3.0.0 on 17 January 2020. Cheers, Andy
test message; please ignore
Please ignore this test message. Thanks :)
GNU Guile 2.9.6 Released [beta]
We are pleased to announce GNU Guile release 2.9.6. This is the sixth pre-release of what will eventually become the 3.0 release series. Compared to the current stable series (2.2.x), the future Guile 3.0 adds support for just-in-time native code generation, speeding up all Guile programs. See the NEWS extract at the end of the mail for full details. Compared to the previous prerelease (2.9.5), Guile 2.9.6 fixes a number of bugs, adds some optimizations, and adds a guile-3 cond-expand feature. We encourage you to test this release and provide feedback to guile-devel@gnu.org, and to file bugs by sending mail to bug-gu...@gnu.org. The Guile web page is located at http://gnu.org/software/guile/, and among other things, it contains a copy of the Guile manual and pointers to more resources. Guile is an implementation of the Scheme programming language, with support for many SRFIs, packaged for use in a wide variety of environments. In addition to implementing the R5RS Scheme standard, Guile includes a module system, full access to POSIX system calls, networking support, multiple threads, dynamic linking, a foreign function call interface, and powerful string processing. Guile can run interactively, as a script interpreter, and as a Scheme compiler to VM bytecode. It is also packaged as a library so that applications can easily incorporate a complete Scheme interpreter/VM. An application can use Guile as an extension language, a clean and powerful configuration language, or as multi-purpose "glue" to connect primitives provided by the application. It is easy to call Scheme code >From C code and vice versa. Applications can add new functions, data types, control structures, and even syntax to Guile, to create a domain-specific language tailored to the task at hand. Guile 2.9.6 can be installed in parallel with Guile 2.2.x; see http://www.gnu.org/software/guile/manual/html_node/Parallel-Installations.html. A more detailed NEWS summary follows these details on how to get the Guile sources. Here are the compressed sources: http://alpha.gnu.org/gnu/guile/guile-2.9.6.tar.lz (10MB) http://alpha.gnu.org/gnu/guile/guile-2.9.6.tar.xz (12MB) http://alpha.gnu.org/gnu/guile/guile-2.9.6.tar.gz (21MB) Here are the GPG detached signatures[*]: http://alpha.gnu.org/gnu/guile/guile-2.9.6.tar.lz.sig http://alpha.gnu.org/gnu/guile/guile-2.9.6.tar.xz.sig http://alpha.gnu.org/gnu/guile/guile-2.9.6.tar.gz.sig Use a mirror for higher download bandwidth: http://www.gnu.org/order/ftp.html Here are the SHA256 checksums: 615e6cabeb6ada4c1b04e9547ce3796e3c80948abd310113ff50a3ee880deba8 guile-2.9.6.tar.lz 6eede2df10c7aa4c4f46d5eeb714752d196fa5325bdde9a0990d7eb8ca833127 guile-2.9.6.tar.xz cb7dbcfb02ea4d5f697d16e95f82959fa76963556cadab0afef741a82f705cbf guile-2.9.6.tar.gz [*] Use a .sig file to verify that the corresponding file (without the .sig suffix) is intact. First, be sure to download both the .sig file and the corresponding tarball. Then, run a command like this: gpg --verify guile-2.9.6.tar.gz.sig If that command fails because you don't have the required public key, then run this command to import it: gpg --keyserver keys.gnupg.net --recv-keys 4FD4D288D445934E0A14F9A5A8803732E4436885 and rerun the 'gpg --verify' command. This release was bootstrapped with the following tools: Autoconf 2.69 Automake 1.16.1 Libtool 2.4.6 Gnulib v0.1-1157-gb03f418 Makeinfo 6.5 The NEWS extract follows, but as a present for having read down this far, here's a comparison of some microbenchmark results between Guile 2.2.6 and Guile 2.9.6: Happy hacking! Now the NEWS: Changes in alpha 2.9.6 (since alpha 2.9.5): * Notable changes ** Add --r6rs, --r7rs options to `guild compile' This makes compiling R6RS or R7RS code a bit easier. See "R6RS Support" and "R7RS Support" in the manual, for more. ** Add guile-3, guile-3.0 cond-expand features See "SRFI-0" in the manual, for more. ** Add #:re-export-and-replace argument to `define-module' This new keyword specifies a set of bindings to re-export, but also marks them as intended to replace core bindings. See "Creating Guile Modules" in the manual, for full details. Note to make this change, we had to change the way replacement flags are stored, to being associated with modules instead of individual variable objects. This means that users who #:re-export an imported binding that was already marked as #:replace by another module will now see warnings, as they need to use #:re-export-and-replace instead. ** Better optimizations for vector-length et al Sometimes the compiler get confused and think it couldn't hoist a `vector-length' call out of a loop. This has been fixed. * Bug fixes ** Fix range inference on division in unreachable code ** Fix frame-call-representation for callees without closures ** Fix range inference for right-shifts ** Fix port-position documentation ** Fix stack overflow if printing a pre-boot error th
Re: [PATCH 1/2] srfi-34: Replace the 'raise' core binding.
On Tue 26 Nov 2019 14:09, Ludovic Courtès writes: > AFAICS there are two blockers: > > 1. We cannot replace & re-export at the same time. Following discussion on IRC, this is fixed now, with #:re-export-and-replace. Would be nice if #:re-export could know whether a binding is local or re-exported and DTRT but that isn't the case currently. > 2. ‘raise’ takes exactly one argument, whereas ‘raise-exception’ takes > an additional keyword argument. I think ignoring this one is fine FWIW. > Perhaps also we should provide a mechanism similar to GCC attributes to > mark a procedure as throwing, so that the compiler can DTRT? An interesting option :) FWIW the compiler has to also know how to call the callee... Something to think about. Cheers, Andy
Re: For a cheaper ‘bytevector->pointer’
Hi :) On Mon 25 Nov 2019 23:03, Ludovic Courtès writes: > Andy Wingo skribis: > >> Honestly I would prefer not to do this. If I understand correctly, the >> problem is in FFI calls -- you have a bytevector and you want to pass it >> as a pointer. In that case the "right" optimization is to avoid the >> scm_tc7_pointer altogether and instead having an unboxed raw pointer. >> The idioms used in FFI are local enough that a compiler can do this. > > I agree! I have a patch from the 2.0 era (attached), but it doesn’t > work because all the tc3s are already taken. I don’t think this has > changed but I could well be missing something about the tag space. > WDYT? I was actually thinking about raw pointer values -- i.e. not immediate-tagged values. If you think about it these values are generally live only between the bytevector->pointer and the FFI call -- the compiler is capable of safely unboxing values in spaces like that. But this would work better with a more compiler-focussed FFI than with the current "interpreted" FFI. But, immediate pointers would be nice too; nicer, in some ways. See also Mark's fixrat work. >> In the short term, what about allowing bytevectors as arguments >> whereever a pointer is allowed? Perhaps it's bad to expand the domain >> of these functions but it may be the right trade-off. > > So in practice, every time there’s '* in the FFI, it’d accept a > bytevector, right? That was the idea :) > I would prefer immediate pointers if that’s possible, and then one of > the two other solutions. In that case I am not sure what a good solution is. Having to add an additional 2-word internal displacement is a bit unfortunate, if that's the case! Andy
Re: [PATCH 1/2] srfi-34: Replace the 'raise' core binding.
On Mon 25 Nov 2019 17:45, Ludovic Courtès writes: > In Guile 2.x, (srfi srfi-34) would already replace 'raise'. Replacing > avoids a run-time warning about the core binding being overridden. > > * module/srfi/srfi-34.scm (raise): New variable. > Mark it as #:replace instead of #:re-export. > --- > module/srfi/srfi-34.scm | 10 +++--- > 1 file changed, 7 insertions(+), 3 deletions(-) > > diff --git a/module/srfi/srfi-34.scm b/module/srfi/srfi-34.scm > index 0e7ad995d..255bfecb9 100644 > --- a/module/srfi/srfi-34.scm > +++ b/module/srfi/srfi-34.scm > @@ -1,6 +1,6 @@ > ;;; srfi-34.scm --- Exception handling for programs > > -;; Copyright (C) 2003, 2006, 2008, 2010 Free Software Foundation, Inc. > +;; Copyright (C) 2003, 2006, 2008, 2010, 2019 Free Software Foundation, Inc. > ;; > ;; This library is free software; you can redistribute it and/or > ;; modify it under the terms of the GNU Lesser General Public > @@ -27,12 +27,16 @@ > ;;; Code: > > (define-module (srfi srfi-34) > - #:re-export (with-exception-handler > - (raise-exception . raise)) > + #:re-export (with-exception-handler) > + #:replace (raise) >#:export-syntax (guard)) > > (cond-expand-provide (current-module) '(srfi-34)) > > +(define (raise exn) > + "Raise the given exception, invoking the current exception handler on EXN." > + (raise-exception exn)) LGTM but it is better to re-export if possible. The reason is that right now the compiler recognizes "throw" and "error" as not falling through, and this is good for a number of reasons; it would be nice to extend this to raise-exception. We should make it possible to re-export and replace at the same time, IMO. Andy
guile-devel@gnu.org
On Mon 25 Nov 2019 17:45, Ludovic Courtès writes: > This ensures core binding '&error' is silently replaced by the SRFI-35 > variant. Again LGTM but it would be nicest to really re-export the binding instead of making a new definition :) Andy
Re: For a cheaper ‘bytevector->pointer’
On Sun 24 Nov 2019 11:52, Ludovic Courtès writes: > A few days ago David was explaining on #guile how ‘bytevector->pointer’ > was generating too much garbage for his use case. An idea we came up > with was to embed the pointer object in the bytevector. > > The patch below does that but it leads to segfaults because I’m guessing > there’s generated bytecode somewhere that still uses the wrong offset; I > adjusted code that emits ‘pointer-ref/immediate’, what else did I > miss? The compiler :) Bytevector literals are stored statically in the .go files, so the assembler would need to change to emit the new layout. Also, compiled access to bytevectors; see prepare-bytevector-access in (language tree-il compile-cps). > Also, since we disable internal pointers, we’d need to register an > additional displacement, and I’m not sure if this is a good idea. > > Thoughts? Honestly I would prefer not to do this. If I understand correctly, the problem is in FFI calls -- you have a bytevector and you want to pass it as a pointer. In that case the "right" optimization is to avoid the scm_tc7_pointer altogether and instead having an unboxed raw pointer. The idioms used in FFI are local enough that a compiler can do this. More broadly -- the current FFI is an interpreter but it should be a compiler. When a call happens, the code interprets the description of the ABI. Instead, pointer->function should ideally *compile* a trampoline. In an ideal world this compilation can happen ahead-of-time, when the .go file is compiled. In the short term, what about allowing bytevectors as arguments whereever a pointer is allowed? Perhaps it's bad to expand the domain of these functions but it may be the right trade-off. Andy
Re: Mutating public bindings of a declarative module
Hi :) On Sun 24 Nov 2019 18:54, Ludovic Courtès writes: > It seems that if you ‘set!’ a public variable of a declarative module, > the change is visible to all the module users, but it’s not necessarily > visible to procedures within that module, presumably because they use an > inlined or specialized variant of that thing. > > I would have imagined that public bindings are considered mutable and > thus not subject to inlining; OTOH, that would obviously be a loss, so > the current approach makes sense. Right, I understand the frustration. For what it is worth, I think we have the right default for what it means to be a declarative module, but I'm definitely open to having that conversation. > Anyway, it complicates a use case for me. In Guix, we “mock” bindings > like so: > > (define-syntax-rule (mock (module proc replacement) body ...) > "Within BODY, replace the definition of PROC from MODULE with the > definition > given by REPLACEMENT." > (let* ((m (resolve-interface 'module)) >(original (module-ref m 'proc))) > (dynamic-wind > (lambda () (module-set! m 'proc replacement)) > (lambda () body ...) > (lambda () (module-set! m 'proc original) > > and that allows us to write tests that temporarily modify public (or > private!) bindings. > > It seems like this could be addressed by compiling selected modules with > ‘user-modules-declarative?’ set to #false, or by avoiding the above hack > altogether when possible, but I thought I’d share my impressions and > listen to what people think. :-) This works. (Actually the way I would do it is to pass #:declarative? #f in the define-module for the modules in question.) Marking some bindings as not declarative also works (e.g. (set! foo foo)). For me the most robust solution would be to have `mock' verify that the module it's funging isn't declarative. We don't currently have a way to know if an individual module binding is declarative or not (though we could add this). Cheers, Andy
GNU Guile 2.9.5 Released [beta]
We are pleased to announce GNU Guile release 2.9.5. This is the fifth pre-release of what will eventually become the 3.0 release series. Compared to the current stable series (2.2.x), the future Guile 3.0 adds support for just-in-time native code generation, speeding up all Guile programs. See the NEWS extract at the end of the mail for full details. Compared to the previous prerelease (2.9.4), Guile 2.9.5 adds low-level optimizations for converting between unboxed integers and floating-point values, generates faster code at optimization level -O3, adds R7RS support, a --r6rs command-line argument, and a fresh implementation of raise-exception and with-exception-handler. We encourage you to test this release and provide feedback to guile-devel@gnu.org, and to file bugs by sending mail to bug-gu...@gnu.org. The Guile web page is located at http://gnu.org/software/guile/, and among other things, it contains a copy of the Guile manual and pointers to more resources. Guile is an implementation of the Scheme programming language, with support for many SRFIs, packaged for use in a wide variety of environments. In addition to implementing the R5RS Scheme standard, Guile includes a module system, full access to POSIX system calls, networking support, multiple threads, dynamic linking, a foreign function call interface, and powerful string processing. Guile can run interactively, as a script interpreter, and as a Scheme compiler to VM bytecode. It is also packaged as a library so that applications can easily incorporate a complete Scheme interpreter/VM. An application can use Guile as an extension language, a clean and powerful configuration language, or as multi-purpose "glue" to connect primitives provided by the application. It is easy to call Scheme code >From C code and vice versa. Applications can add new functions, data types, control structures, and even syntax to Guile, to create a domain-specific language tailored to the task at hand. Guile 2.9.5 can be installed in parallel with Guile 2.2.x; see http://www.gnu.org/software/guile/manual/html_node/Parallel-Installations.html. A more detailed NEWS summary follows these details on how to get the Guile sources. Here are the compressed sources: http://alpha.gnu.org/gnu/guile/guile-2.9.5.tar.lz (10MB) http://alpha.gnu.org/gnu/guile/guile-2.9.5.tar.xz (12MB) http://alpha.gnu.org/gnu/guile/guile-2.9.5.tar.gz (21MB) Here are the GPG detached signatures[*]: http://alpha.gnu.org/gnu/guile/guile-2.9.5.tar.lz.sig http://alpha.gnu.org/gnu/guile/guile-2.9.5.tar.xz.sig http://alpha.gnu.org/gnu/guile/guile-2.9.5.tar.gz.sig Use a mirror for higher download bandwidth: http://www.gnu.org/order/ftp.html Here are the SHA256 checksums: 0c0097092fc5b0b40c5eb97ce09bf5415b31c5f7241f4cdcb01284f81cb2f70f guile-2.9.5.tar.lz f917cb8578740887df8e0090fdecf3f06aaf60d2331067b88ff5c3bb610d69b5 guile-2.9.5.tar.xz 199c5dbba3ec4322dfd9f1e1af6c19deca77fa4a104c59f26aeea23f7640720d guile-2.9.5.tar.gz [*] Use a .sig file to verify that the corresponding file (without the .sig suffix) is intact. First, be sure to download both the .sig file and the corresponding tarball. Then, run a command like this: gpg --verify guile-2.9.5.tar.gz.sig If that command fails because you don't have the required public key, then run this command to import it: gpg --keyserver keys.gnupg.net --recv-keys 4FD4D288D445934E0A14F9A5A8803732E4436885 and rerun the 'gpg --verify' command. This release was bootstrapped with the following tools: Autoconf 2.69 Automake 1.16.1 Libtool 2.4.6 Gnulib v0.1-1157-gb03f418 Makeinfo 6.5 Changes in beta 2.9.5 (since beta 2.9.4): * Notable changes ** Record unification Guile used to have a number of implementations of structured data types in the form of "records": a core facility, SRFI-9 (records), SRFI-35 (condition types -- a form of records) and R6RS records. These facilities were not compatible, as they all were built in different ways. This had the unfortunate corollary that SRFI-35 conditions were not compatible with R6RS conditions. To fix this problem, we have now added the union of functionality from all of these record types into core records: single-inheritance subtyping, mutable and immutable fields, and so on. See "Records" in the manual, for full details. R6RS records, SRFI-9 records, and the SRFI-35 and R6RS exception types have been accordingly "rebased" on top of core records. ** Reimplementation of exceptions Since Guile's origins 25 years ago, `throw' and `catch' have been the primary exception-handling primitives. However these primitives have two problems. One is that it's hard to handle exceptions in a structured way using `catch'. Few people remember what the corresponding `key' and `args' are that an exception handler would see in response to a call to `error', for example. In practice, this results in more generic catch-all exception handling than one might like. The other pr
Re: guile 3 update, halloween edition
Hi :) On Sat 16 Nov 2019 16:26, Ludovic Courtès writes: > Andy Wingo skribis: > >> On Fri 15 Nov 2019 10:03, Ludovic Courtès writes: >> >>> I guess we could add a specific ‘&type-exception’ exception or similar, >>> which would allow us to improve error reporting (that can come later, of >>> course.) > > What I meant is that type errors are “special” enough to deserve their > own type more specific than the catch-all ‘&assertion-failure’ (just > like there’s already a separate ‘&undefined-variable’, for instance.) Agreed! > Speaking of which, it seems that ‘set-guile-exception-converter!’ is > currently private, but I wonder if the goal was to make it public (it > seems to be unused)? It was private also in the exception conversion work that Mark did, FWIW; it just moved over as-is. Honestly I think that now that exceptions are "primary" we should probably move in the opposite direction: instead of adding more converters from key+args to exception objects, we should encourage exception throwers to switch from "throw" to "raise-exception", and allow library authors to define converters in the other way from exception object to the equivalent arguments for "catch". So I think exposing set-guile-exception-converter! might be the wrong thing at this point. Dunno tho. > For instance, C bindings that currently call ‘throw’ could provide > additional “exception converters” for the benefit of Scheme users > who’d rather use structured exceptions. (That would also give less of > an incentive to provide a C API for all of this.) This is a good point! FWIW Regarding C and migration, I have the impression that probably 90% of exception throwers in C use the helpers from error.h (scm_wrong_num_args and so on), which we can change transparently. A remaining 5% might use scm_error_scm, for which a registry might make sense, and 5% use scm_throw directly. These are just guesses tho. >>> 4. Is ‘&warning’ actually used? Is the goal to make it continuable? >>> That sounds great. >> >> Any exception can be raised in a continuable way. Whether a raise is >> continuable or not depends on the value of the #:continuable? keyword to >> raise-exception. I think that's the intention of &warning but I don't >> really have instincts about how it might be used. Guile defines it >> because it's in R6RS, but how it will be used is an open question :) > > I suppose the intent is to effectively allow users to implement the UI > stuff as a sort of co-routine to support separation of concerns: you > just raise a ‘&warning’ that some other code displays in its preferred > way (console message, popup window, whatever) and eventually calls your > continuation. > > That’s something I’ve been wanting for some time, because right now > we’re able to separate out the UI concern for exception display, but not > for warnings. > > However, it seems that the handler passed to ‘with-exception-handler’ > does not receive the continuation, so is it the case that currently > handlers cannot resume exceptions? (Again not a showstopper IMO but > rather another wishlist item :-)). The handler runs within the continuation of "raise-continuable": (with-exception-handler (lambda (exn) (+ exn 30)) (lambda () (+ 2 (raise-exception 10 #:continuable? #t => 42 However I'm not sure this facility is what you want. Like for example there's lots of false-if-exception / catch #t out there; that's equivalent to: (define-syntax-rule (false-if-exception expr) (let/ec k (with-exception-handler (lambda (exn) (k #f)) (lambda () expr So the exception handler there would intervene and get a first crack at the warning, messing up your intent. To me warnings are like logging, and logging is notoriously difficult to standardize :) If it were me I would make a mechanism for warnings that had a with-warning-handler and I would make sure to raise all warnings via a separate raise-warning procedure or something, independent of exceptions. But that's just me :) Andy
Re: landed r7rs support
Hi, On Sun 17 Nov 2019 16:52, Linus Björnstam writes: > The text about the standards process is probably going to annoy people > since it does not mention the division between r7rs small and large :) I suppose that is fair, as I am annoyed by R7RS :) However, the report published in 2013 does bear the name R7RS and not R7RS-small, so it's not an error to refer to it as such. In any case, it does not seem to me that R7RS-large is similar to the other reports in the RnRS series. To me it is more similar in flavor to the SRFI process. Andy
landed r7rs support
Hey all :) Just a little heads-up that I just landed R7RS support. Thanks to Göran Weinholt for akku-scm (https://gitlab.com/akkuscm/akku-r7rs/) and OKUMURA Yuki for yuni (https://github.com/okuoku/yuni), off of which some of these files were based. (These projects are public domain / CC0). The library syntax for R7RS is a subset of R6RS, so to use R7RS you just (import (scheme base)) and off you go. As with R6RS also, there are some small lexical incompatibilities regarding hex escapes; see "R7RS Incompatibilities" in the manual. Also there is a --r7rs command-line option. Cheers, Andy
Re: guile 3 update, halloween edition
Hey thanks for the review :) On Fri 15 Nov 2019 10:03, Ludovic Courtès writes: > 0. Do I get it right that ‘throw’ and ‘catch’ are not “deprecated” in > the sense of (ice-9 deprecated) and are instead simply not the > “preferred” exception mechanism? Correct. I think we could envision deprecating them in some future but not within the next couple years at least. > 1. I see things like: > > +(define (make-condition type . field+value) > + "Return a new condition of type TYPE with fields initialized as specified > +by FIELD+VALUE, a sequence of field names (symbols) and values." > + (unless (exception-type? type) > +(scm-error 'wrong-type-arg "make-condition" "Not a condition type: ~S" > + (list type) #f)) > > and: > > + (unless (symbol? key) > +(throw 'wrong-type-arg "throw" "Wrong type argument in position ~a: > ~a" > + (list 1 key) (list key))) > > I guess we could add a specific ‘&type-exception’ exception or similar, > which would allow us to improve error reporting (that can come later, of > course.) Yes. So right now Guile is in a bit of a transitional state -- it still signals 99.9% of errors via `throw'. Probably we want to change to have structured exceptions for almost all of these. To preserve compatibility we would probably need to mix in an &exception-with-kind-and-args to all of these exceptions, or otherwise augment `exception-kind' and `exception-args' to synthesize these values when appropriate. > Guix has ‘&location’ error conditions, which I’ve found useful when > combined with other error conditions in cases where location info from > the stack isn’t useful: > > https://git.savannah.gnu.org/cgit/guix.git/tree/guix/utils.scm#n832 > > I wonder if (ice-9 exceptions) should provide something like that. Neat :) Yes sure. I think, any exception type can be added to (ice-9 exceptions) -- there's little "name cost" like there is in boot-9. Which reminds me, I want to make boot-9 do a (resolve-module '(ice-9 exceptions)) so that the more capable make-exception-from-throw always gets installed. > 2. What are you thoughts regarding exposing structured exceptions to C? > I’ve always been frustrated by ‘system-error’ :-). Guix has a hack to > augment ‘system-error’ with information about the offending file name: > > https://git.savannah.gnu.org/cgit/guix.git/tree/guix/ui.scm#n520 > > If the POSIX bindings would emit a structured ‘&system-error’ record, > that’d be pretty cool. I don't know :) Right now raise-exception is marked SCM_INTERNAL. Probably it should be public. There is no public C API for any of this new functionality, as it stands; a TODO. Regarding exception objects, there are two questions: one, how to create exceptions of specific kinds; I suspect scm_make_system_error () would be fine, and probably you want scm_make_exception to be able to mix various exceptions. Second question, do we want to expose accessors too? It can be a lot of API surface and I am a bit wary of it. But, perhaps it is the right thing. I do not know. > 3. I wonder if we could take advantage of the new ‘&message’ exception > to start i18n of error messages. It might be as simple as telling > xgettext to recognize ‘make-exception-with-message’ as a keyword, though > currently there are few calls to ‘make-exception-with-message’ followed > by a literal. Eventually this will get called by `error', I think; Guile's R7RS layer in wip-r7rs defines `error' as being: (define (error message . irritants) (raise-exception (let ((exn (make-exception-with-message message))) (if (null? irritants) exn (make-exception exn (make-exception-with-irritants irritants)) But yes this is definitely something to think about it. > 4. Is ‘&warning’ actually used? Is the goal to make it continuable? > That sounds great. Any exception can be raised in a continuable way. Whether a raise is continuable or not depends on the value of the #:continuable? keyword to raise-exception. I think that's the intention of &warning but I don't really have instincts about how it might be used. Guile defines it because it's in R6RS, but how it will be used is an open question :) > Bah, you give us a present and I reply with an additional wishlist. > ;-) :) I hope that the exceptions work can serve as a foundation for further incremental, compatible improvement. Cheers, Andy
Re: guile 3 update, halloween edition
On Sat 02 Nov 2019 06:20, Mark H Weaver writes: > Andy Wingo writes: > >> So, now the pending task is to somehow get a condition/exception >> hierarchy into Guile core. I will try to mostly push things off to side >> modules but it won't always be possible. There will be bijections >> between a Guile's "throw" arguments and structured exceptions, mostly >> inspired with what Julian did in the R6RS layer already. > > For the record, the bijection between R6RS conditions and Guile's throw > arguments was my work, not Julian's. An honest mistake on my part. My sincere apologies! Warm regards, Andy
Re: guile 3 update, halloween edition
Greets :) On Thu 31 Oct 2019 01:01, Chris Vine writes: > "Condition" is a strange word for describing structured error objects, > I agree. However, I think it would be quite confusing to describe > error objects as exceptions. "Error object" or "error condition object" > seems a reasonable alternative if the bare word "condition" is thought > to be inappropriate. I'm very sympathetic to this argument -- an exception seems like a thing-in-motion, not a thing-at-rest. But perhaps it's just the effect of habit, setting up expectations about what good names are. (After all, plenty of people seem happy with the term "condition"!) Perhaps there is a middle ground of sorts: maybe the manual can comprehensively describe what R6RS refers to as conditions using the term "exception objects". WDYT? Andy
Re: guile 3 update, halloween edition
Hey :) On Thu 31 Oct 2019 15:17, Mikael Djurfeldt writes: > How does the record subtyping relate to GOOPS? I do realize that there > are issues related to keeping bootstrapping lean, but shouldn't record > types and classes share mechanisms? They share the struct layer. Records are simple: their fields are laid out in order, are all unboxed, and can be treated as a simple kind of nominal product type. `match' takes advantage of this. GOOPS is more flexible: it can have different slot allocations, unboxed slots, multiple inheritance, and so on. Because of multiple inheritance its accessors have to do dynamic dispatch; whereas for records, if supertype A allocates slot X to index I, all subtypes will have that slot in the same place. A check whether a record is an instance of A is a simple O(1) check, rather than the CPL search of GOOPS. Exceptions have a kind of multiple inheritance, but it's more about object composition than typing. You can make a compound condition from a heterogeneous collection of other conditions. While you could implement compound conditions with subtyping, it's more straightforward to use composition. I think the current status is close to the sweet spot but your thoughts are welcome :) I would dearly like to be able to subtype records in GOOPS, but having looked at it a few times I haven't found quite the right way to do it. Cheers, Andy
guile 3 update, halloween edition
Hey folks! I wanted to send out an update on Guile 3. Do take a look at https://git.savannah.gnu.org/cgit/guile.git/tree/NEWS to see where we've come; basically the JIT is done, and we're ready to release soonish. However! Here begins a long chain of yak-shaving: I wanted good benchmarks. Generally up to now, we haven't really been doing good incremental benchmarks. Ideally we could see some benchmark results historically, on every commit, and against other Scheme implementations. To a degree it's been possible with https://ecraven.github.io/r7rs-benchmarks/, but those benchmarks have a few problems: (1) They use unsafe optimizations on e.g. Chez and Gambit (2) They are infrequently run (3) They rely on R7RS, adding their own little compat layer for Guile, which isn't optimal. Now, regarding (3), probably Guile should just have its own R7RS layer. And it should be easier to enable R6RS too. So I added an --r6rs option, and started importing some R7RS code from Göran Weinholt (thanks!), with the idea of adding --r7rs. That way we can just benchmark the different implementations, just passing --r7rs or whatever to get the behavior we want. We can reduce the set of other scheme implementations to just the high-performance ones: Gambit, Larceny, Chez, and Racket. However! R7RS, like R6RS and like SRFI-35/SRFI-34, and also like Racket, specifies an error-handling system in terms of "raise" and "with-exception-handler". Guile uses "throw" and "catch". There is a pretty good compatibility layer in Guile's R6RS exceptions/conditions code, but it's not shared by SRFI-35/SRFI-34, and unless we built R7RS in terms of R6RS -- something I prefer not to do; these things should be layered on Guile core directly -- we'd have to duplicate the mechanism. Which, of course, is a bit trash. And when you come to think of it, throw/catch/with-throw-handler is also a bit trash. It is too hard to handle exceptions in Guile; the addition of `print-exception' a few years back improved things, but still, making any sense out of the "args" corresponding to a "key" is a mess. All this made me think -- Guile should probably switch to raise/with-exception-handler and structured exceptions. (Or conditions, or whatever we choose to call them. I find the "condition" name a bit weird but maybe that's just a personal problem.) Racket does this too, for what it's worth, though they have their own historical baggage. But, we need to maintain compatibility with throw/catch, because that's not going anywhere any time soon (if ever). So I hacked a bit and eventually came up with a decent implementation of throw/catch on top of raise/with-exception-handler, and I think it's compatible in all the weird ways that it needs to be. But! Now we have bootstrapping problems; how to get the implementation in boot-9? Exceptions in SRFI-35, R6RS, R7RS, and Racket are these hierarchical things: they form a DAG of subtypes. But core records in Guile aren't subtypeable, so what to do? Well, my thinking was that we needed to sedimentarily deposit down into Guile core those commonalities between the different record implementations in Guile: SRFI-35 conditions, R6RS records, and SRFI-9 records. So core now has the notion of field mutability on the record layer (as opposed to the struct layer), a notion of subtyping, a notion of extensibility, and so on. This is all now in the manual and will be in NEWS. With that, we now have just one implementation of records!!! I am very pleased about this. Now you can use core record introspection facilities on any record in Guile. Cool. This also helped untangle some knots in the R6RS inter-module graph. So, now the pending task is to somehow get a condition/exception hierarchy into Guile core. I will try to mostly push things off to side modules but it won't always be possible. There will be bijections between a Guile's "throw" arguments and structured exceptions, mostly inspired with what Julian did in the R6RS layer already. Thoughts welcome! Also: should these structured error objects be named exceptions or conditions? SRFI-35, R6RS, and R7RS say "conditions", but racket and my heart say "exceptions"; wdyt? Cheers, Andy
conflicts in the gnu project now affect guile
Hello all, In the last few weeks, a conversation among GNU maintainers that has been simmering for years burst into public. For a while it resubmerged into private GNU lists, but now it has resurfaced to affect the Guile project. Just for background information, I wrote about my thoughts here: https://wingolog.org/archives/2019/10/08/thoughts-on-rms-and-gnu The summary is that, like many people in GNU, I have long treated Richard Stallman not as a hero, not as a leader, but rather a "missing stair" (https://en.wikipedia.org/wiki/Missing_stair) that one has to route around. This approach was never very inclusive -- if you don't have much experience in GNU, it's possible to not know about it, and to fall in the hole yourself. On the other hand if you know of RMS but not Guile, you might think that Guile developers support RMS. However, recent events made me realize this approach was not only unfair to newcomers, but unjust as well, as by continuing to work on GNU and not saying anything, I was both lending unmerited prestige to RMS, enabling his creepy behavior towards women, and additionally, enabling his apparent pedophilia-advocacy. Regarding this latter point, I wasn't really aware that this was a view RMS was promoting, but I am ashamed to admit that I had heard rumors that Richard publically advocated sex between adults and teenagers, defended sexual harassers, and questioned the experience of victims of sexual assault, and I preferred not to listen. Looking again, and I think Richard's web site speaks for itself: https://web.archive.org/web/20170612074722/http://stallman.org/archives/2017-mar-jun.html#26_May_2017_(Prudish_ignorantism) https://web.archive.org/web/20180131020215/https://stallman.org/archives/2017-jul-oct.html#29_October_2017_(Pestering_women) https://web.archive.org/web/20180104112431/https://www.stallman.org/archives/2017-nov-feb.html#27_November_2017_(Roy_Moore's_relationships) https://web.archive.org/web/20180509120046/https://stallman.org/archives/2018-mar-jun.html#30_April_2018_(UN_peacekeepers_in_South_Sudan) https://web.archive.org/web/20180911075211/https://www.stallman.org/archives/2018-jul-oct.html#17_July_2018_(The_bullshitter's_flirting) https://web.archive.org/web/20180911075211/https://www.stallman.org/archives/2018-jul-oct.html#21_August_2018_(Age_and_attraction) https://web.archive.org/web/20180924231708/https://stallman.org/archives/2018-jul-oct.html#23_September_2018_(Cody_Wilson) https://web.archive.org/web/20181113161736/https://www.stallman.org/archives/2018-sep-dec.html#6_November_2018_(Sex_according_to_porn) https://web.archive.org/web/20190325024048/https://stallman.org/archives/2019-jan-apr.html#14_February_2019_(Respecting_peoples_right_to_say_no) https://www.stallman.org/archives/2019-may-aug.html#11_June_2019_(Stretching_meaning_of_terms) https://web.archive.org/web/20190801201704/https://stallman.org/archives/2019-may-aug.html#12_June_2019_(Declining_sex_rates) https://web.archive.org/web/20190801201704/https://stallman.org/archives/2019-may-aug.html#30_July_2019_(Al_Franken) https://web.archive.org/web/20190903050208/https://stallman.org/archives/2019-jul-oct.html#27_August_2019_(Me-too_frenzy) Anyway. So far, so GNU. A couple weeks ago I thought it an opportune moment to declare publicly the views that I have long held privately: that I do not consider RMS to be the leader of GNU, and that GNU maintainers and other developers with a stake in the project should organize to fill the void. * * * I pause here to mention that you may not agree with this perspective and that is fine. There are many ways that we can continue to work together while this discussion plays out. Part of the purpose of this mail though is to make it clear that there are differences of opinion and that the GNU project is in flux. * * * Now we get to how this issue affects Guile. Before the RMS/GNU/FSF conversation started, Mark Weaver left Guile, for essentially unrelated reasons. He threatened to leave because he wished to be consulted before I landed mixed definitions and expressions and shipped them in the 2.9.4 release; I responded over email asking to talk about the issues; in response a week later I see that he resigned from maintainership and left the Guile group on Savannah. It was truly a shame for Guile, as Mark is an excellent hacker and has done a lot of good work for Guile. It's true also that, mixed with the sadness, I felt a modicum of relief. It has never been easy to work with Mark. I could toil on Guile for weeks, taking time away from my family, and then wake up to receive a private mail excoriating me for my work. It was also far from the first time he threatened to leave the project if he did not get his way. I have never let the problems between Mark and me into the public sphere though, preferring to preserve his reputation, and it
Maintainership changes: many thanks to Mark!
Hi all, After many years working on Guile and more than 5 years in a maintainer role, Mark Weaver has decided to step down. Taking over from him and remaining as Guile co-maintainers are Ludovic Courtès and Andy Wingo. On behalf of myself and Ludovic and no doubt all Guile users and developers: a heartfelt thanks, Mark, for all of your years of service, and see you around the Guile community! Happy hacking, Andy and Ludovic
GNU Guile 2.9.4 Released [beta]
We are pleased to announce GNU Guile release 2.9.4. This is the fourth pre-release of what will eventually become the 3.0 release series. Compared to the current stable series (2.2.x), the future Guile 3.0 adds support for just-in-time native code generation, speeding up all Guile programs. See the NEWS extract at the end of the mail for full details. Compared to the previous prerelease (2.9.3), Guile 2.9.4 adds support for more unboxed floating-point operations, enables inlining of top-level definitions, improves compilation of `letrec*' and internal definitions, and allows mixed internal definitions and expressions in body contexts (e.g. in a `let'). We encourage you to test this release and provide feedback to guile-devel@gnu.org, and to file bugs by sending mail to bug-gu...@gnu.org. The Guile web page is located at http://gnu.org/software/guile/, and among other things, it contains a copy of the Guile manual and pointers to more resources. Guile is an implementation of the Scheme programming language, with support for many SRFIs, packaged for use in a wide variety of environments. In addition to implementing the R5RS Scheme standard, Guile includes a module system, full access to POSIX system calls, networking support, multiple threads, dynamic linking, a foreign function call interface, and powerful string processing. Guile can run interactively, as a script interpreter, and as a Scheme compiler to VM bytecode. It is also packaged as a library so that applications can easily incorporate a complete Scheme interpreter/VM. An application can use Guile as an extension language, a clean and powerful configuration language, or as multi-purpose "glue" to connect primitives provided by the application. It is easy to call Scheme code >From C code and vice versa. Applications can add new functions, data types, control structures, and even syntax to Guile, to create a domain-specific language tailored to the task at hand. Guile 2.9.4 can be installed in parallel with Guile 2.2.x; see http://www.gnu.org/software/guile/manual/html_node/Parallel-Installations.html. A more detailed NEWS summary follows these details on how to get the Guile sources. Here are the compressed sources: http://alpha.gnu.org/gnu/guile/guile-2.9.4.tar.lz (10MB) http://alpha.gnu.org/gnu/guile/guile-2.9.4.tar.xz (12MB) http://alpha.gnu.org/gnu/guile/guile-2.9.4.tar.gz (21MB) Here are the GPG detached signatures[*]: http://alpha.gnu.org/gnu/guile/guile-2.9.4.tar.lz.sig http://alpha.gnu.org/gnu/guile/guile-2.9.4.tar.xz.sig http://alpha.gnu.org/gnu/guile/guile-2.9.4.tar.gz.sig Use a mirror for higher download bandwidth: http://www.gnu.org/order/ftp.html Here are the SHA256 checksums: e2da91bf20ab7b82ec6913dd9adcf64885f232f2242cdcdc7e612ead10ccf4fa guile-2.9.4.tar.lz 17ecf3c09c9784526c6f27955bf9a74a2adf6f7f16e20b00e9cedf0871dc34d6 guile-2.9.4.tar.xz 4b495513866c52b2ce70997711cd3263914e946200c3541b79ccb82ccca69b9c guile-2.9.4.tar.gz [*] Use a .sig file to verify that the corresponding file (without the .sig suffix) is intact. First, be sure to download both the .sig file and the corresponding tarball. Then, run a command like this: gpg --verify guile-2.9.4.tar.gz.sig If that command fails because you don't have the required public key, then run this command to import it: gpg --keyserver keys.gnupg.net --recv-keys 4FD4D288D445934E0A14F9A5A8803732E4436885 and rerun the 'gpg --verify' command. This release was bootstrapped with the following tools: Autoconf 2.69 Automake 1.16.1 Libtool 2.4.6 Gnulib v0.1-1157-gb03f418 Makeinfo 6.5 Changes in alpha 2.9.4 (since alpha 2.9.3): * Notable changes ** Improved compilation of many floating-point primitives Heavy floating-point computations that use `sqrt', `abs', `floor', `ceiling', `sin', `cos', `tan', `asin', `acos', and `atan' will be faster, as the compiler can unbox inexact rationals (flonums) in more cases. ** Improved compilation of internal `define' Internal definitions -- like the `define' in `(let () (define x 10) ...)' -- are now compiled more optimally, using the algorithm from Ghuloum and Dybvig's "Fixing Letrec (reloaded)" paper. This change makes internal `define' just as efficient as `let', `let*', `letrec', and similar. ** Interleaved internal definitions and expressions allowed It used to be that internal definitions had to precede all expressions in their bodies. This restriction has been relaxed. If an expression precedes an internal definition, it is treated as if it were a definition of an unreferenced variable. For example, the expression `(foo)' transforms to the equivalent of `(define _ (begin (foo) #f))', if it precedes other definitions. This change improves the readability of Guile programs, as it used to be that program indentation tended to increase needlessly to allow nested `let' and `letrec' to re-establish definition contexts after initial expressions, for example for type-checks on proce
GNU Guile 2.9.3 Released [beta]
We are pleased to announce GNU Guile release 2.9.3. This is the third pre-release of what will eventually become the 3.0 release series. Compared to the current stable series (2.2.x), the future Guile 3.0 adds support for just-in-time native code generation, speeding up all Guile programs. See the NEWS extract at the end of the mail for full details. Compared to the previous prerelease (2.9.2), Guile 2.9.3 improves the quality of generated machine code, speeding up some programs by up to 50%. We encourage you to test this release and provide feedback to guile-devel@gnu.org, and to file bugs by sending mail to bug-gu...@gnu.org. The Guile web page is located at http://gnu.org/software/guile/, and among other things, it contains a copy of the Guile manual and pointers to more resources. Guile is an implementation of the Scheme programming language, with support for many SRFIs, packaged for use in a wide variety of environments. In addition to implementing the R5RS Scheme standard, Guile includes a module system, full access to POSIX system calls, networking support, multiple threads, dynamic linking, a foreign function call interface, and powerful string processing. Guile can run interactively, as a script interpreter, and as a Scheme compiler to VM bytecode. It is also packaged as a library so that applications can easily incorporate a complete Scheme interpreter/VM. An application can use Guile as an extension language, a clean and powerful configuration language, or as multi-purpose "glue" to connect primitives provided by the application. It is easy to call Scheme code >From C code and vice versa. Applications can add new functions, data types, control structures, and even syntax to Guile, to create a domain-specific language tailored to the task at hand. Guile 2.9.3 can be installed in parallel with Guile 2.2.x; see http://www.gnu.org/software/guile/manual/html_node/Parallel-Installations.html. A more detailed NEWS summary follows these details on how to get the Guile sources. Here are the compressed sources: http://alpha.gnu.org/gnu/guile/guile-2.9.3.tar.lz (10MB) http://alpha.gnu.org/gnu/guile/guile-2.9.3.tar.xz (12MB) http://alpha.gnu.org/gnu/guile/guile-2.9.3.tar.gz (21MB) Here are the GPG detached signatures[*]: http://alpha.gnu.org/gnu/guile/guile-2.9.3.tar.lz.sig http://alpha.gnu.org/gnu/guile/guile-2.9.3.tar.xz.sig http://alpha.gnu.org/gnu/guile/guile-2.9.3.tar.gz.sig Use a mirror for higher download bandwidth: http://www.gnu.org/order/ftp.html Here are the SHA256 checksums: 24b2ebbcc06e37f6d5d56a6053676d7f3fc54886717aa5ab589383ce8739e685 guile-2.9.3.tar.lz f9cd59233bde1a6b316bfcf82a49fa25d6f7e25b7c0019272afc697f19072991 guile-2.9.3.tar.xz b549d5bc7a00247b89f40e495b0ab96947b9f7935829fb6d20573545c1156535 guile-2.9.3.tar.gz [*] Use a .sig file to verify that the corresponding file (without the .sig suffix) is intact. First, be sure to download both the .sig file and the corresponding tarball. Then, run a command like this: gpg --verify guile-2.9.3.tar.gz.sig If that command fails because you don't have the required public key, then run this command to import it: gpg --keyserver keys.gnupg.net --recv-keys 4FD4D288D445934E0A14F9A5A8803732E4436885 and rerun the 'gpg --verify' command. This release was bootstrapped with the following tools: Autoconf 2.69 Automake 1.16.1 Libtool 2.4.6 Gnulib v0.1-1157-gb03f418 Makeinfo 6.5 Changes in alpha 2.9.3 (since alpha 2.9.2): * Notable changes ** Improved just-in-time machine code generation Guile's JIT compiler emits better code for indirect procedure calls, atomic instructions, numeric comparisons, procedure prologues, well-known closures with no free variables, calls and returns, and allocations. Together these improvements can speed up some benchmarks by up to 50%. Cumulative changes in the alpha 2.9.x series (since the stable 2.2 series): * Notable changes ** Just-in-time code generation Guile programs now run up to 4 times faster, relative to Guile 2.2, thanks to just-in-time (JIT) native code generation. Notably, this brings the performance of "eval" as written in Scheme back to the level of "eval" written in C, as in the days of Guile 1.8. See "Just-In-Time Native Code" in the manual, for more information. JIT compilation will be enabled automatically and transparently. To disable JIT compilation, configure Guile with `--enable-jit=no' or `--disable-jit'. The default is `--enable-jit=auto', which enables the JIT if it is available. See `./configure --help' for more. JIT compilation is enabled by default on x86-64, i686, ARMv7, and AArch64 targets. ** Lower-level bytecode Relative to the virtual machine in Guile 2.2, Guile's VM instruction set is now more low-level. This allows it to express more advanced optimizations, for example type check elision or integer devirtualization, and makes the task of JIT code generation easier. Note that this change can mean th
GNU Guile 2.9.2 Released [beta]
We are pleased to announce GNU Guile release 2.9.2. This is the second pre-release of what will eventually become the 3.0 release series. Compared to the current stable series (2.2.x), the future Guile 3.0 adds support for just-in-time native code generation, speeding up all Guile programs. See the NEWS extract at the end of the mail for full details. Compared to the previous prerelease (2.9.1), Guile 2.9.2 adds native code generation support on the ia32, ARMv7, and AArch64 architectures. We encourage you to test this release and provide feedback to guile-devel@gnu.org, and to file bugs by sending mail to bug-gu...@gnu.org. The Guile web page is located at http://gnu.org/software/guile/, and among other things, it contains a copy of the Guile manual and pointers to more resources. Guile is an implementation of the Scheme programming language, with support for many SRFIs, packaged for use in a wide variety of environments. In addition to implementing the R5RS Scheme standard, Guile includes a module system, full access to POSIX system calls, networking support, multiple threads, dynamic linking, a foreign function call interface, and powerful string processing. Guile can run interactively, as a script interpreter, and as a Scheme compiler to VM bytecode. It is also packaged as a library so that applications can easily incorporate a complete Scheme interpreter/VM. An application can use Guile as an extension language, a clean and powerful configuration language, or as multi-purpose "glue" to connect primitives provided by the application. It is easy to call Scheme code >From C code and vice versa. Applications can add new functions, data types, control structures, and even syntax to Guile, to create a domain-specific language tailored to the task at hand. Guile 2.9.2 can be installed in parallel with Guile 2.2.x; see http://www.gnu.org/software/guile/manual/html_node/Parallel-Installations.html. A more detailed NEWS summary follows these details on how to get the Guile sources. Here are the compressed sources: http://alpha.gnu.org/gnu/guile/guile-2.9.2.tar.lz (10MB) http://alpha.gnu.org/gnu/guile/guile-2.9.2.tar.xz (12MB) http://alpha.gnu.org/gnu/guile/guile-2.9.2.tar.gz (21MB) Here are the GPG detached signatures[*]: http://alpha.gnu.org/gnu/guile/guile-2.9.2.tar.lz.sig http://alpha.gnu.org/gnu/guile/guile-2.9.2.tar.xz.sig http://alpha.gnu.org/gnu/guile/guile-2.9.2.tar.gz.sig Use a mirror for higher download bandwidth: http://www.gnu.org/order/ftp.html Here are the SHA256 checksums: 823dce6c89a993663dbd3eba058cb65daa7a67ef120fcf0295f6ba5682c05fef guile-2.9.2.tar.gz 1a64b2e4d7f0d394df437355662219cbfa13fda5ebde93db4ad11356449b7d6c guile-2.9.2.tar.lz e403be8e0ace0ee1150260288755c64a47cce11732e3f8c5fc9e552e5c4365f0 guile-2.9.2.tar.xz [*] Use a .sig file to verify that the corresponding file (without the .sig suffix) is intact. First, be sure to download both the .sig file and the corresponding tarball. Then, run a command like this: gpg --verify guile-2.9.2.tar.gz.sig If that command fails because you don't have the required public key, then run this command to import it: gpg --keyserver keys.gnupg.net --recv-keys 4FD4D288D445934E0A14F9A5A8803732E4436885 and rerun the 'gpg --verify' command. This release was bootstrapped with the following tools: Autoconf 2.69 Automake 1.16.1 Libtool 2.4.6 Gnulib v0.1-1157-gb03f418 Makeinfo 6.5 Changes in alpha 2.9.2 (since alpha 2.9.1): * Notable changes ** Just-in-time code generation support on ARMv7, ia32, AArch64 This release adds just-in-time (JIT) native code generation for the ia32, ARMv7, and AArch64 platforms, in addition to the x86-64 support already present in 2.9.1. ** Cheaper just-in-time code generation Guile now includes a forked version of GNU Lightning. This "Lightening" effort, spun out as a separate project, aims to build on the back-end support from GNU Lightning, but adapting the API and behavior of the library to match Guile's needs. One of the important points that this project fixes is run-time overhead. With the adoption of Lightening, Guile has lowered its thresholds for when to generate native code at run-time, so that user programs run faster, sooner. For more information, see https://gitlab.com/wingo/lightening. Cumulative changes in the alpha 2.9.x series (since the stable 2.2 series): * Notable changes ** Just-in-time code generation Guile programs now run up to 4 times faster, relative to Guile 2.2, thanks to just-in-time (JIT) native code generation. Notably, this brings the performance of "eval" as written in Scheme back to the level of "eval" written in C, as in the days of Guile 1.8. See "Just-In-Time Native Code" in the manual, for more information. JIT compilation will be enabled automatically and transparently. To disable JIT compilation, configure Guile with `--enable-jit=no' or `--disable-jit'. The default is `--enable-jit=auto', which enables
Re: guile-2.9.1 impressions
Hi! On Thu 06 Dec 2018 06:21, Linas Vepstas writes: > After sending the email below, I scanned the guile-devel archives, > and I see Thomas Morley talking about Lilypond performance. > The example program he offers up caught my eye: nested deep > in a loop is this: > > (eval-string "'(a b c)") In this case I believe Guile 2.9 / 3 should be significantly faster than 2.2, because `eval' is compiled to native code rather than bytecode. My measurements showed it to be on par with the hand-optimized C implementation from 1.8 and before. Depends of course on how much the expander is part of your workload, there are differences relative to Guile 1.8. Anyway, thanks for the note and I just wanted to mention this point. Regarding Scheme -> C++ transitions, there is the possibility that this too could be much faster with Guile 2.9.x given that these calls are now JIT-compiled instead of interpreted. We'll have to see. Cheers, Andy
GNU Guile 2.9.1 Released [beta]
We are pleased to announce GNU Guile release 2.9.1. This is the first pre-release of what will eventually become the 3.0 release series. Compared to the current stable series (2.2.x), Guile 2.9.1 adds support for just-in-time native code generation, speeding up all Guile programs. See the NEWS extract at the end of the mail for full details. We encourage you to test this release and provide feedback to guile-devel@gnu.org, and to file bugs by sending mail to bug-gu...@gnu.org. The Guile web page is located at http://gnu.org/software/guile/, and among other things, it contains a copy of the Guile manual and pointers to more resources. Guile is an implementation of the Scheme programming language, with support for many SRFIs, packaged for use in a wide variety of environments. In addition to implementing the R5RS Scheme standard, Guile includes a module system, full access to POSIX system calls, networking support, multiple threads, dynamic linking, a foreign function call interface, and powerful string processing. Guile can run interactively, as a script interpreter, and as a Scheme compiler to VM bytecode. It is also packaged as a library so that applications can easily incorporate a complete Scheme interpreter/VM. An application can use Guile as an extension language, a clean and powerful configuration language, or as multi-purpose "glue" to connect primitives provided by the application. It is easy to call Scheme code From C code and vice versa. Applications can add new functions, data types, control structures, and even syntax to Guile, to create a domain-specific language tailored to the task at hand. Guile 2.9.1 can be installed in parallel with Guile 2.2.x; see http://www.gnu.org/software/guile/manual/html_node/Parallel-Installations.html. A more detailed NEWS summary follows these details on how to get the Guile sources. Here are the compressed sources: http://alpha.gnu.org/gnu/guile/guile-2.9.1.tar.lz (10.3MB) http://alpha.gnu.org/gnu/guile/guile-2.9.1.tar.xz (12.3MB) http://alpha.gnu.org/gnu/guile/guile-2.9.1.tar.gz (20.8MB) Here are the GPG detached signatures[*]: http://alpha.gnu.org/gnu/guile/guile-2.9.1.tar.lz.sig http://alpha.gnu.org/gnu/guile/guile-2.9.1.tar.xz.sig http://alpha.gnu.org/gnu/guile/guile-2.9.1.tar.gz.sig Use a mirror for higher download bandwidth: http://www.gnu.org/order/ftp.html Here are the SHA256 checksums: 9e1dc7ed34a5581e47dafb920276fbb12c9c318ba432d19cb970c01aa1ab3a09 guile-2.9.1.tar.gz f24e6778e3e45ea0691b591ad7e74fdd0040689915b09ae0e52bd2a80f8e2b33 guile-2.9.1.tar.lz 01be24335d4208af3bbd0d3354d3bb66545f157959bb0c5a7cbb1a8bfd486a45 guile-2.9.1.tar.xz [*] Use a .sig file to verify that the corresponding file (without the .sig suffix) is intact. First, be sure to download both the .sig file and the corresponding tarball. Then, run a command like this: gpg --verify guile-2.9.1.tar.gz.sig If that command fails because you don't have the required public key, then run this command to import it: gpg --keyserver keys.gnupg.net --recv-keys 4FD4D288D445934E0A14F9A5A8803732E4436885 and rerun the 'gpg --verify' command. This release was bootstrapped with the following tools: Autoconf 2.69 Automake 1.16.1 Libtool 2.4.6 Gnulib v0.1-1157-gb03f418 Makeinfo 6.5 Changes in alpha 2.9.1 (since the stable 2.2 series): * Notable changes ** Just-in-time code generation Guile programs now run up to 4 times faster, relative to Guile 2.2, thanks to just-in-time (JIT) native code generation. Notably, this brings the performance of "eval" as written in Scheme back to the level of "eval" written in C, as in the days of Guile 1.8. See "Just-In-Time Native Code" in the manual, for more information. JIT compilation will be enabled automatically and transparently. To disable JIT compilation, configure Guile with `--enable-jit=no' or `--disable-jit'. The default is `--enable-jit=auto', which enables the JIT if it is available. See `./configure --help' for more. In this release, JIT compilation is enabled only on x86-64. In future prereleases support will be added for all architectures supported by GNU lightning. Intrepid users on other platforms can try passing `--enable-jit=yes' to see the state of JIT on their platform. ** Lower-level bytecode Relative to the virtual machine in Guile 2.2, Guile's VM instruction set is now more low-level. This allows it to express more advanced optimizations, for example type check elision or integer devirtualization, and makes the task of JIT code generation easier. Note that this change can mean that for a given function, the corresponding number of instructions in Guile 3.0 may be higher than Guile 2.2, which can lead to slowdowns when the function is interpreted. We hope that JIT compilation more than makes up for this slight slowdown. ** By default, GOOPS classes are not redefinable It used to be that all GOOPS classes were redefinable, at least in theory. This facility wa
Re: guile 3 update, september edition
Greets :) On Mon 17 Sep 2018 11:35, l...@gnu.org (Ludovic Courtès) writes: >> The threshold at which Guile will automatically JIT-compile is set from >> the GUILE_JIT_THRESHOLD environment variable. By default it is 5. >> If you set it to -1, you disable the JIT. If you set it to 0, *all* >> code will be JIT-compiled. The test suite passes at >> GUILE_JIT_THRESHOLD=0, indicating that all features in Guile are >> supported by the JIT. Set the GUILE_JIT_LOG environment variable to 1 >> or 2 to see JIT progress. > > Just to be clear, does GUILE_JIT_THRESHOLD represents the number of > times a given instruction pointer is hit? No. It is an abstract "hotness" counter associated with a function's code. (I say "function's code" because many closures can share the same code and thus the same counter. It's not in the scm_tc7_program object because some procedures don't have these.) All counters start at 0 when Guile starts. A function's counters increment by 30 when a function is called, currently, and 2 on every loop back-edge. I have not attempted to tweak these values yet. >> Using GNU Lightning has been useful but in the long term I don't think >> it's the library that we need, for a few reasons: > > [...] > > It might be that the lightning 1.x branch would be a better fit (it was > exactly as you describe.) I think that’s what Racket was (is?) using. Could be! I will have a look. Cheers, Andy
guile 3 update, september edition
Hi! This is an update on progress towards Guile 3. In our last update, we saw the first bits of generated code: https://lists.gnu.org/archive/html/guile-devel/2018-08/msg5.html Since then, the JIT is now feature-complete. It can JIT-compile *all* code in Guile, including delimited continuations, dynamic-wind, all that. It runs automatically, in response to a function being called a lot. It can also tier up from within hot loops. The threshold at which Guile will automatically JIT-compile is set from the GUILE_JIT_THRESHOLD environment variable. By default it is 5. If you set it to -1, you disable the JIT. If you set it to 0, *all* code will be JIT-compiled. The test suite passes at GUILE_JIT_THRESHOLD=0, indicating that all features in Guile are supported by the JIT. Set the GUILE_JIT_LOG environment variable to 1 or 2 to see JIT progress. For debugging (single-stepping, tracing, breakpoints), Guile will fall back to the bytecode interpreter (the VM), for the thread that has debugging enabled. Once debugging is no longer enabled (no more hooks active), that thread can return to JIT-compiled code. Right now the JIT-compiled code exactly replicates what the bytecode interpreter does: the same stack reads and writes, etc. There is some specialization when a bytecode has immediate operands of course. However the choice to do debugging via the bytecode interpreter -- effectively, to always have bytecode around -- will allow machine code (compiled either just-in-time or ahead-of-time) to do register allocation. JIT will probably do a simple block-local allocation. An AOT compiler is free to do something smarter. As far as I can tell, with the default setting of GUILE_JIT_THRESHOLD=5, JIT does not increase startup latency for any workload, and always increases throughput. More benchmarking is needed though. Using GNU Lightning has been useful but in the long term I don't think it's the library that we need, for a few reasons: * When Lightning does a JIT compilation, it builds a graph of operations, does some minor optimizations, and then emits code. But the graph phase takes time and memory. I think we just need a library that just emits code directly. That would lower the cost of JIT and allow us to lower the default GUILE_JIT_THRESHOLD. * The register allocation phase in Lightning exists essentially for calls. However we have a very restricted set of calls that we need to do, and can do the allocation by hand on each architecture. This (We don't use CPU call instructions for Scheme function calls because we use the VM stack. We might be able to revise this in the future but again Lightning is in the way). Doing it by hand would allow a few benefits: * Hand allocation would free up more temporary registers. Right now lightning reserves all registers used as part of the platform calling convention; they are unavailable to the JIT. * Sometimes when Lightning needs a temporary register, it can clobber one that we're using as part of an internal calling convention. I believe this is fixed for x86-64 but I can't be sure for other architectures! See commit 449ef7d9755b553cb0ad2629bca3bc42c5913e88. * We need to do our own register allocation; having Lightning also do it is a misfeature. * Sometimes we know that we can get better emitted code, but the lightning abstraction doesn't let us do it. We should allow ourselves to punch through that abstraction. The platform-specific Lightning files basically expose most of the API we need. We could consider incrementally punching through lightning.h to reach those files. Something to think about for the future. Finally, as far as performance goes -- we're generally somewhere around 80% faster than 2.2. Sometimes more, sometimes less, always faster though AFAIK. As an example, here's a simple fib.scm: $ cat /tmp/fib.scm (define (fib n) (if (< n 2) 1 (+ (fib (- n 1)) (fib (- n 2) Now let's use eval-in-scheme to print the 35th fibonacci number. For Guile 2.2: $ time /opt/guile-2.2/bin/guile -c \ '(begin (primitive-load "/tmp/fib.scm") (pk (fib 35)))' ;;; (14930352) real 0m9.610s user 0m10.547s sys 0m0.040s But with Guile from the lightning branch, we get: $ time /opt/guile/bin/guile -c \ '(begin (primitive-load "/tmp/fib.scm") (pk (fib 35)))' ;;; (14930352) real 0m5.299s user 0m6.167s sys 0m0.064s Meaning that "eval" in Guile 3 is somewhere around 80% faster than in Guile 2.2 -- because "eval" is now JIT-compiled. (Otherwise it's the same program.) This improves bootstrap times, though Guile 3's compiler will generally make more CPS nodes than Guile 2.2 for the same expression, which takes more time and memory, so the gain isn't earth-shattering. Incidentally, as a comparison, Guile 2.0 (whose
Re: FOSDEM 2019
On Tue 21 Aug 2018 19:57, Ricardo Wurmus writes: > If we base our application on this proposal I think we should make these > changes: > > - mention the work on adding JIT Yes definitely! This will be Guile 3, so we can call it "Just-in-time code generation for Scheme: Speed for free in Guile 3" or something. I'd be happy to give this talk. Andy
Guile 3 update, August edition
Hi! Last dispatch was here: https://lists.gnu.org/archive/html/guile-devel/2018-07/msg00037.html To recap, I merged in GNU lightning and added an extra machine-code return address to frames, but hadn't actually written the JIT yet. Since July, I made it so that all Guile bytecode function entry points start with an "instrument-entry" bytecode that holds a counter. The intention is that when the counter increments beyond a certain value, the function should be automatically JIT-compiled. Associated with the counter is a native-code pointer corresponding to the function. I also added "instrument-loop" bytecodes to all loops, to be able to tier up from within hot loops. With all of this done and some other bytecode tweaks, I was able to move on to the JIT compiler itself. I'm happy to say that I now have a first version. It's about 3500 lines of C, so a bit gnarly. It's architecture-independent, as it uses lightning, and there are lightning backends for about every architecture. Lightning seems OK. Not optimal, but OK, and an OK thing to use for now anyway. I did have to write special cases for 32-bit machines, as Guile's VM supports 64-bit arithmetic, and some-endian-specific code. I probably got some of that wrong; review is very welcome: https://git.savannah.gnu.org/cgit/guile.git/tree/libguile/jit.c?h=lightning If you have fixes and are a committer, please feel free to just commit them directly. If you aren't a committer yet and you spot some fixes, mail the list; you should definitely be a committer if you can do that :) I just got the JIT working today. For the time being, the interface is a public function, %jit-compile. Eventually I will remove this when I have more confidence, relying only on the automatic compilation triggered by function entry and loop iterations. As an example: $ cat foo.scm (use-modules (rnrs bytevectors)) (define (f32v-sum bv) (let lp ((n 0) (sum 0.0)) (if (< n (bytevector-length bv)) (lp (+ n 4) (+ sum (bytevector-ieee-single-native-ref bv n))) sum))) (define ones (make-f32vector #e1e7 1.0)) # The JIT currently doesn't emit hook code. $ meta/guile --no-debug scheme@(guile-user)> (load "foo.scm") scheme@(guile-user)> ,time (f32v-sum ones) $2 = 1.0e7 ;; 0.143017s real time, 0.142986s run time. 0.00s spent in GC. scheme@(guile-user)> (%jit-compile f32v-sum) scheme@(guile-user)> ,time (f32v-sum ones) $3 = 1.0e7 ;; 0.048514s real time, 0.048499s run time. 0.00s spent in GC. In this particular example, the JITted code runs about 3x faster than the interpreted code. The JIT doesn't do register allocation; not sure precisely how to do that. A future topic. For the moment I want to consolidate what we have and once it's all just magically working and everybody's programs are faster, we release Guile 3. Cheers, Andy
guile 3 update, july edition
Hi :)
Just a brief update with Guile 3. Last one was here:
https://lists.gnu.org/archive/html/guile-devel/2018-06/msg00026.html
There is a now a "lightning" branch that has GNU lightning merged in and
built statically into Guile. It causes about 1 MB of overhead in the
-Og libguile-3.0.so, bringing it to 6.1 MB, or 1.36 MB stripped. Seems
OK for now. By way of contrast, libguile-2.2.so is 5.65 MB when built
with -Og, or 1.19 MB stripped.
There's some scaffolding for making JIT code emitters for each
instruction. But then I ran into a problem about how to intermingle JIT
and interpreter returns on the stack. I was hoping to avoid having
separate interpreter and JIT return addresses in a stack frame, to avoid
adding overhead. That didn't work out:
https://lists.gnu.org/archive/html/guile-devel/2018-07/msg00013.html
So, I added a slot to the "overhead" part of stack frames. From
frames.h:
Stack frame layout
--
| ... |
+==+ <- fp + 3 = SCM_FRAME_PREVIOUS_SP (fp)
| Dynamic link |
+--+
| Virtual return address (vRA) |
+--+
| Machine return address (mRA) |
+==+ <- fp
| Local 0 |
+--+
| Local 1 |
+--+
| ... |
+--+
| Local N-1|
\--/ <- sp
The stack grows down.
The calling convention is that a caller prepares a stack frame
consisting of the saved FP, the saved virtual return addres, and the
saved machine return address of the calling function, followed by the
procedure and then the arguments to the call, in order. Thus in the
beginning of a call, the procedure being called is in slot 0, the
first argument is in slot 1, and the SP points to the last argument.
The number of arguments, including the procedure, is thus FP - SP.
That took a while. Anything that changes calling conventions is gnarly.
While I was at it, I changed the return calling convention to expect
return values from slot 0 instead of from slot 1, and made some other
minor changes to instructions related to calls and returns.
The next step will be to add an "enter-function" instruction or
something to function entries. This instruction's only real purpose
will be to increment a counter associated with the function. If the
counter exceeds some threshold, JIT code will be emitted for the
function and the function will tier up. If the enter-function
instruction sees that the function already has JIT code (e.g. emitted
from another thread), then it will tier up directly.
Because enter-function is in the right place to run the apply hook for
debugging, we'll probably move that functionality there, instead of
being inline with the call instructions.
The "enter-function" opcode will take an offset to writable data for the
counter, allocated in the ELF image. This data will have the form:
struct jit_data {
void* mcode;
uint32_t counter;
uint32_t start;
uint32_t end;
}
The mcode pointer indicates the JIT code, if any. It will probably need
to be referenced atomically (maybe release/consume ordering?). The
counter is the counter associated with this function; entering a
function will increment it by some amount. The start and end elements
indicate the bounds of the function, and are offsets into the vcode,
relative to the jit_data struct. These are not writable.
Loops will also have an instruction that increments the counter,
possibly tiering up if needed. The whole function will share one
"struct jit_data".
I am currently thinking that we can make JIT-JIT function calls peek
ahead in the vcode of the callee to find the callee JIT code, if any.
I.e.:
(if (has-tc7? callee %tc7-program)
(let ((vcode (word-ref callee 1)))
(if (= (logand (u32-ref vcode 0) #xff)
%enter-function-opcode)
(let ((mcode ((+ vcode (* (u32-ref vcode 1) 4)
(if (zero? mcode)
(jmp! mcode)
(return!))) ;; return to interpreter
(return!)))
(return!))
It's a dependent memory load on the function-call hot path but it will
predict really well. The upside of this is that there is just one
mutable mcode pointer for a function, for all its closures in all
threads. It also avoids reserving more space on the heap for another
mcode word in program objects.
Loops will tier up ("on-stack replacement") by jumping to an offset in
the mcode corresponding to the vcode for the counter-incrementing
instruction. The offset will be determined by running the JIT compiler
for the function but without actually emitting the code and flushing
icache; the compiler is run in a mode just to determine the mc
Re: guile 3 update, june 2018 edition
Hi :) On Mon 02 Jul 2018 11:28, l...@gnu.org (Ludovic Courtès) writes: > Andy Wingo skribis: > >> My current plan is that the frame overhead will still be two slots: the >> saved previous FP, and the saved return address. Right now the return >> address is always a bytecode address. In the future it will be bytecode >> or native code. Guile will keep a runtime routine marking regions of >> native code so it can know if it needs to if an RA is bytecode or native >> code, for debugging reasons; but in most operation, Guile won't need to >> know. The interpreter will tier up to JIT code through an adapter frame >> that will do impedance matching over virtual<->physical addresses. To >> tier down to the interpreter (e.g. when JIT code calls interpreted >> code), the JIT will simply return to the interpreter, which will pick up >> state from the virtual IP, SP, and FP saved in the VM state. > > What will the “adapter frame” look like? Aah, sadly it won't work like this. Somehow I was thinking of an adapter frame on the C stack. However an adapter frame corresponds to a continuation, so it would have to have the life of a continuation, so it would have to be on the VM stack. I don't think I want adapter frames on the VM stack, so I have to scrap this. More below... >> We do walk the stack from Scheme sometimes, notably when making a >> backtrace. So, we'll make the runtime translate the JIT return >> addresses to virtual return addresses in the frame API. To Scheme, it >> will be as if all things were interpreted. > > Currently you can inspect the locals of a stack frame. Will that be > possible with frames corresponding to native code? (I suppose that’d be > difficult.) Yes, because native code manipulates the VM stack in exactly the same way as bytecode. Eventually we should do register allocation and avoid always writing values to the stack, but that is down the road. >> My current problem is knowing when a callee has JIT code. Say you're in >> JITted function F which calls G. Can you directly jump to G's native >> code, or is G not compiled yet and you need to use the interpreter? I >> haven't solved this yet. "Known calls" that use call-label and similar >> can of course eagerly ensure their callees are JIT-compiled, at >> compilation time. Unknown calls are the problem. I don't know whether >> to consider reserving another word in scm_tc7_program objects for JIT >> code. I have avoided JIT overhead elsewhere and would like to do so >> here as well! > > In the absence of a native code pointer in scm_tc7_program objects, how > will libguile find the native code for a given program? This is a good question and it was not clear to me when I wrote this! I think I have a solution now but it involves memory overhead. Oh well. Firstly, I propose to add a slot to stack frames. Stack frames will now store the saved FP, the virtual return address (vRA), and the machine return address IP (mRA). When in JIT code, a return will check if the mRA is nonzero, and if so jump to that mRA. Otherwise it will return from JIT, and the interpreter should continue. Likewise when doing a function return from the interpreter and the mRA is nonzero, the interpreter should return by entering JIT code to that address. When building an interpreter-only Guile (Guile without JIT) or an AOT-only Guile (doesn't exist currently), we could configure Guile to not reserve this extra stack word. However that would be a different ABI: a .go file built with interpreter-only Guile wouldn't work on Guile-with-JIT, because interpreter-only Guile would think stack frames only need two reserved words, whereas Guile-with-JIT would write three words. To avoid the complication, for 3.0 I think we should just use 3-word frames all the time. So, that's returns. Other kinds of non-local returns like abort-to-prompt, resuming delimited continuations, or calling undelimited continuations would work similarly: the continuation would additionally record an mRA, and resuming would jump there instead, if appropriate. Now, calls. One of the reasons that I wanted to avoid an extra program word was because scm_tc7_program doesn't exist in a one-to-one relationship with code. "Well-known" procedures get compiled by closure optimization to be always called via call-label or tail-call-label -- so some code doesn't have program objects. On the other hand, closures mean that some code has many program objects. So I thought about using side tables indexed by code; or inline "maybe-tier-up-here" instructions, which would reference a code pointer location, that if nonzero, would be the JIT code. However I see now that really we need to optimize for the JI
guile 3 update, june 2018 edition
Hi, Just wanted to give an update on Guile 3 developments. Last note was here: https://lists.gnu.org/archive/html/guile-devel/2018-04/msg4.html The news is that the VM has been completely converted over to call out to the Guile runtime through an "intrinsics" vtable. For some intrinsics, the compiler will emit specialized call-intrinsic opcodes. (There's one of these opcodes for each intrinsic function type.) For others that are a bit more specialized, like the intrinsic used in call-with-prompt, the VM calls out directly to the intrinsic. The upshot is that we're now ready to do JIT compilation. JIT-compiled code will use the intrinsics vtable to embed references to runtime routines. In some future, AOT-compiled code can keep the intrinsics vtable in a register, and call indirectly through that register. My current plan is that the frame overhead will still be two slots: the saved previous FP, and the saved return address. Right now the return address is always a bytecode address. In the future it will be bytecode or native code. Guile will keep a runtime routine marking regions of native code so it can know if it needs to if an RA is bytecode or native code, for debugging reasons; but in most operation, Guile won't need to know. The interpreter will tier up to JIT code through an adapter frame that will do impedance matching over virtual<->physical addresses. To tier down to the interpreter (e.g. when JIT code calls interpreted code), the JIT will simply return to the interpreter, which will pick up state from the virtual IP, SP, and FP saved in the VM state. We do walk the stack from Scheme sometimes, notably when making a backtrace. So, we'll make the runtime translate the JIT return addresses to virtual return addresses in the frame API. To Scheme, it will be as if all things were interpreted. This strategy relies on the JIT being a simple code generator, not an optimizer -- the state of the stack whether JIT or interpreted is the same. We can consider relaxing this in the future. My current problem is knowing when a callee has JIT code. Say you're in JITted function F which calls G. Can you directly jump to G's native code, or is G not compiled yet and you need to use the interpreter? I haven't solved this yet. "Known calls" that use call-label and similar can of course eagerly ensure their callees are JIT-compiled, at compilation time. Unknown calls are the problem. I don't know whether to consider reserving another word in scm_tc7_program objects for JIT code. I have avoided JIT overhead elsewhere and would like to do so here as well! For actual JIT code generation, I think my current plan is to import a copy of GNU lightning into Guile's source, using git-subtree merges. Lightning is fine for our purposes as we only need code generation, not optimization, and it supports lots of architectures: ARM, MIPS, PPC, SPARC, x86 / x86-64, IA64, HPPA, AArch64, S390, and Alpha. Lightning will be built statically into libguile. This has the advantage that we always know the version being used, and we are able to extend lightning without waiting for distros to pick up a new version. Already we will need to extend it to support atomic ops. Subtree merges should allow us to pick up upstream improvements without too much pain. This strategy also allows us to drop lightning in the future if that's the right thing. Basically from the user POV it should be transparent. The whole thing will be behind an --enable-jit / --disable-jit configure option. When it is working we can consider enabling shared lightning usage. Happy hacking, Andy
Re: c99 support
On Sat 23 Jun 2018 22:12, Andy Wingo writes:
> Is there anyone who compiles Guile with a compiler that does not support
> C99? If so, please give platform and compiler.
>
> I think my questions are limited to, in decreasing order of importance:
>
> * Is there any system that we target that doesn't have C99 stdint.h
> and stddef.h ?
>
> * Is there any system that we target that doesn't support C99 inline
> functions?
>
> * C99 mixed decls and statements?
>
> * C99 one-line comments (// foo) ?
>
> * C99 compound literals? ((struct x) { 1, 2 }) ?
>
> * stdbool.h
>
> I would like to use C99 inside Guile, and I want to eventually replace
> scm_t_uint8 with uint8_t.
Thanks all for the responses. It would seem that the first four
features of C99 are OK for all platforms that we target, with the
following caveats:
* We should avoid using C++ keywords (e.g. throw) in Guile API files.
* We might want to avoid mixed decls and statements in inline functions
in Guile API files.
We should probably avoid stdbool.h and compound literals, for C++
reasons.
In Guile 3.0 (master branch), the types "scm_t_uint8" and so on are now
deprecated. My recommendation is that all users switch to use
e.g. "uint8_t", "ptrdiff_t", etc from instead of the
scm_t_uint8, etc definitions that they are now using. The definitions
are compatible on all systems, AFAIU, and on GNU, scm_t_uint8 has long
been a simple typedef for uint8_t.
If you make the change while targetting current Guile (2.2), then you'll
won't have deprecation warnings when 3.0 comes out.
Cheers,
Andy
Re: c99 support
On Sat 23 Jun 2018 21:23, Eli Zaretskii writes:
>> From: Andy Wingo
>> Date: Sat, 23 Jun 2018 21:11:29 +0200
>> Cc: guile-devel@gnu.org
>>
>> Is there anyone who compiles Guile with a compiler that does not support
>> C99? If so, please give platform and compiler.
>
> You mean C99 compiler or C99 C run-time library? Or both?
>
> MS-Windows (MinGW) doesn't have a C99 compliant C library, although
> quite a few of what's needed is present.
Hard to say :) I think my questions are limited to, in decreasing order
of importance:
* Is there any system that we target that doesn't have C99 stdint.h
and stddef.h ?
* Is there any system that we target that doesn't support C99 inline
functions?
* C99 mixed decls and statements?
* C99 one-line comments (// foo) ?
* C99 compound literals? ((struct x) { 1, 2 }) ?
* stdbool.h
I want to use C99 inside Guile, and I want to eventually replace
scm_t_uint8 with uint8_t.
I assume MinGW uses GCC. What version? I see that the version 6 series
is available on mingw.org.
Andy
c99 support
Hi, Is there anyone who compiles Guile with a compiler that does not support C99? If so, please give platform and compiler. Thanks, Andy
Re: tracepoints/breakpoints and native compilation
On Tue 15 May 2018 05:08, Mark H Weaver writes: > I think we should consider deprecating these legacy debug hooks, and > instead to support debugging features similar to GDB, using the same > implementation methods that GDB uses. > > Alternatively, I wonder if it would be feasible to enhance GDB itself to > support debugging native Guile code. Tx for feedback. FWIW GDB will patch the executable code in-place. It does so while the program being debugged is stopped; it uses ptrace for this AFAIU. I guess in Guile we'd set asyncs on all active threads to synchronize. Actually modifying the executable code is pretty gnarly and target-specific though. FWIW JS and Java VMs will often "deoptimize" functions that have breakpoints or tracepoints. Some can avoid deoptimization in some cases, but the mechanism is the same: recompile functions. The hooks that Guile has aren't so legacy, and aren't immutable API -- they showed up in 2.0, changed a bit in 2.2, and if we keep them they will change a bit more in 3.0. Their users are already deep in the engine and can adapt. Regarding GDB, it would certainly be feasible to enhance it to understand Guile's native code. Probably it's all possible to do from Scheme. But for tracepoints and and for when you don't have an appropriate GDB you probably do want to be able to also have in-process debugging capabilities. Cheers, Andy
tracepoints/breakpoints and native compilation
Hi :) A design question for everyone. I am wondering how to support breakpoints, tracepoints, and the like in a Guile with native-code compilation. If you are not familiar with what Guile does currently, see: https://www.gnu.org/software/guile/manual/html_node/Traps.html Basically Guile supports calling out to user-defined procedures when: (1) pushing a new continuation (stack frame) (2) tail-calling a procedure (3) popping a continuation (e.g. on return) (4) non-local return (after abort, or after calling a call/cc continuation) (5) advancing the instruction pointer This last one is obviously expensive. To mitigate this cost, Guile builds the VM twice: once with hooks and once without. When run with --debug, you get the VM with hooks; otherwise, no hooks for you. Note that without hooks, you still get interrupts, so you can do statistical profiling, or interrupt a loop. But unlike hooks, interrupts are placed in the code. Practically speaking, without hooks, what you lose is the ability to set a breakpoint. You also lose the ability to trace a procedure, but because tracing can recursively invoke the right VM, that's not a problem in practice. Generally you only care about hook performance when they are disabled, so what you need to do is minimize overhead in that case. In a bytecode interpreter (as Guile has now), the hooks do add some overhead, but they are very predictable branches, so it's not a big deal. Debug mode is on by default in the Guile REPL and I think it adds some 10 or 20% overhead. However in native-code compilation, hooks are more of a problem. They increase the executable size, as each hook invocation has to be present in the text somewhere. They pollute the branch predictor, as there are many more branches. They increase instruction cache size, even in the best case where the slow branches are all out of line. Boo. So... I have a proposal. Right now in the short term I am going to make a JIT for Guile 3.0 (master branch) bytecode. I am still working on massaging the bytecode into a state where it is very easily JITtable, but that will be soon I hope (weeks) and then, outside circumstances permitting, we can have a JIT within the next 3 months. For JIT compilation, the "hook" problem resolves itself very easily: we can JIT in two modes, one that adds hooks and one that doesn't. We can alter the hook API to allow the VM to know when to re-JIT code; adding a breakpoint doesn't have to re-JIT everything. Or of course we can keep the current API and just re-JIT everything. For the bytecode interpreter, we can keep the two VMs. Also easy. However if we ship native code in the .go files -- what do we do? Three options, I see -- one, ship "regular" code (no hooks) -- fast, but no breakpoints from Guile. Two, ship "debug" code (with hooks). Or three, ship the bytecode also, and a JIT compiler, so that we can re-JIT if needed. The first possibility would mean that some Guile-compiled code is not really debuggable in a nice way. It's not ideal. The second is OK, but it would be slow. However the third option seems to offer a good choice for general-purpose installs. If we we ship the bytecode in .go files by default and the built Guile supports JIT compilation, then maybe we can get all the advantages -- peak performance with native code without embedded hook calls, but still the ability to insert hooks if needed. So I am looking at going with the third option. It also opens the door to potential experiments with trace compilation and optimization. I am currently not looking at adaptive optimization otherwise, but anyway that's a bit far off; we need native compilation first to get good startup time. Thoughts welcome! Andy
Guile 3 update -- more instruction explosion, intrinsics
Hello all :)
This mail is an update on Guile 3 progress. Some previous updates were
here:
https://lists.gnu.org/archive/html/guile-devel/2018-01/msg9.html
https://lists.gnu.org/archive/html/guile-devel/2018-01/msg3.html
I took a break for a while and picked up this work again a couple weeks
ago. I think the reason I stopped was because I got to agonizing about
how to call out to run-time routines from eventual native code. Like,
you want to call out from native code to scm_string_set_x; or some more
specialized routine that's part of the ABI but not the API, like an
internal internal_string_set_x routine. How do you get the pointer to
that code? How do you manage saving the VM state and restoring it? Can
you have custom calling conventions? How do you preserve cross-process
sharing of code pages?
For a while I thought inline caches would be a kind of answer, but after
looking at it for a while I think I am going to punt:
https://wingolog.org/archives/2018/02/07/design-notes-on-inline-caches-in-guile
Guile simply isn't all that polymorphic, and the set of runtime routines
callable from native code compiled by Guile is bounded. Better instead
to simply provide a vtable to native code that contains function
pointers to anything that might need to be callable at run-time.
So that's what I settled on. I call them "intrinsics" (I would call
them "builtins" but that word is used for something else). I started
moving over "macro-instructions" that can't be usefully broken apart,
like string-set!, to be intrinsic calls. AOT-compiled native code will
compile these to vtable calls ("call [reg + offset]"), where reg holds a
pointer to the intrinsics and offset is a fixed offset. JIT-compiled
native code can inline the intrinsic address of course.
I also pushed ahead with instruction explosion for string-length,
string-ref, the atomics, integer/char conversion, make-closure, and
f64->scm. I made a bunch more instructions be intrinsics. The VM is
thus now smaller and has fewer instructions whose implementations
contain internal branches, which means we're getting closer to native
code.
There are still some more instructions to push to intrinsics. (Q: When
should an instruction be an intrinsic rather than "exploded" (lowered to
more primitive instructions and control flow)? A: When the optimizer is
unlikely to be able to elide components of the exploded implementation.)
Then the biggest remaining task is dealing with the call instructions,
which are somewhat large still. Probably they need exploding. Once
that's done I think we can look to implementing a simple template method
JIT. If the performance of that beats 2.2 (as I think it should!), then
it could be a good point to release 3.0 just like that.
Long-term I think it makes sense to do AOT compilation. That also opens
the door for self-hosted adaptive optimization. But, I don't know quite
how to get there and keep our portability story. Do we keep bytecode
forever? Only in some cases? I don't know. I certainly want to
minimize the amount of C that we have to maintain :)
Likewise in the medium term I think we should be actively moving library
code from C to Scheme. With the move to intrinsics in the VM, the VM
itself is relying less and less on libguile, making the whole system
less coupled to libguile. For Guile to prosper in the next 10 years, we
need to be able to retarget it, to WebAssembly and Racket-on-Chez and
PyPy and Graal and a whole host of other things. The compiler is
producing low-level, fairly portable output currently, which is
appropriate to this goal, but we are limited by libguile. So let's be
thinking about how to move much of the remaining 80KLOC of C over to
Scheme. We won't be able to move all of it, but certainly we can move
some of it.
Happy hacking,
Andy
guile 3 update: instruction explosion for bytevectors
Greets, FYI I wrote up a new update on the road to Guile 3; as I seem to be getting closer, it's in blog form: https://wingolog.org/archives/2018/01/17/instruction-explosion-in-guile Cheers, A
