Re: [PATCH] Turn on more documentation
Hello, > The problem is that the auto-generated “Standard Library” section looks > very poor in comparison to the rest of the manual. So we should really > try hard to write good doc by hands for these, and come up with a handy > structure (instead of one node per module, all under “Standard > Library”). I could do that pretty easily, but I'm afraid that documentation that isn't in the same file as the code would tend to be updated less often than documentation in the same file. It might not be a problem though, and I'm happy to try. The other alternative is to make our documentation format more expressive, but that might be a lot of trouble. > However, I believe Thien-Thi’s Guile 1.4 has good doc for at least some > of these modules, and it would be great if it could be incorporated. > > WDYT? That would be ideal, but I was concerned that the documentation wasn't copyrighted by the FSF. Unless Thien-Thi is interested in contributing it, of course. Noah
Re: Psyntax security hole prevents secure sandboxing in Guile
> Can you think of anything else that would need to be fixed, besides this > problem with forgeable syntax-objects? It depends how much of a sandbox you're thinking of, but I'd like to make sure that the untrusted code didn't go into an infinite loop, which means either putting it in a separate process or having a timer that would stop it after a deadline. Also you'd have to make sure that you didn't run any procedure returned by the untrusted code, for the same reason. Also, what if the untrusted code allocated a lot of memory? I suppose you could depend on that all being garbage-collected after it finished, but you'd have to be prepared to handle out-of-memory errors while it was running. It might be easiest to just put it in a separate process, although that would make communication harder. Noah
Re: Patches for module/ice-9/occam-channel.scm
Hello, I talked some more with the maintainers. In general, everything that goes in should have tests and documentation. This module isn't documented to begin with, so you shouldn't have to fix that to contribute (although of course we'd love that :-) ). However, we would like tests for your changes, so we can catch regressions in the future. Take a look in test-suite/tests for examples of how those work. Since these are relatively large changes, you'll also need to do a copyright assignment. We assign the copyright on Guile to the Free Software Foundation because under US law, the owner of a copyrighted work is sometimes the only person with the power to sue over violations of copyright. The FSF wants to be able to sue if people violate the GPL, so we assign copyright. Please see http://www.gnu.org/licenses/why-assign.html for more information, or email ass...@gnu.org to fill out a form. Thanks a lot for contributing, Noah On Sun, May 6, 2012 at 5:33 AM, Daniel Krueger wrote: > Hey, > > I myself don't know much about the foundations of the module itself, I > only know it should be really the same as occam's channels, but I > don't know much about them. I just found the package and thought it > would seems to be useful. And as I tried to use it I found the > incompatibilities and fixed them. So I just know how this > implementation works, but not very much about the background. > > I could do some documentation, it's just that I never documented > something in this info(?) format. I'll try it and send some patches, > you can then look if it's okay ^^ > > - Daniel > > On Fri, May 4, 2012 at 3:17 PM, Andrew Gwozdziewycz wrote: >> On Thu, May 3, 2012 at 11:18 PM, Nala Ginrut wrote: >>> Well, I think it's something like Actors but little different. >>> And I think it would be nice if there's a document for it. >> >> It's a foundational part of actors, more than actors themselves. >> Actors need some sort of "mailbox" to communicate with the outside >> world[0]. Channels provide a type of mailbox. I assume by the name, >> the semantics of the channels provided in occam-channel, closely >> resemble those in the Occam programming language. I believe the >> semantics of Occam channels are based on C.A.R. Hoare's CSP[1]. >> >> [0]: https://en.wikipedia.org/wiki/Actor_model >> [1]: https://en.wikipedia.org/wiki/Communicating_sequential_processes >> >>> On Fri, May 4, 2012 at 10:58 AM, Noah Lavine >>> wrote: >>>> Hello, >>>> >>>> Thanks a lot for submitting them! Micro-commits are exactly what we want. >>>> >>>> These patches raise an interesting issue. I do not use occam-channel, >>>> and in fact I have no idea what it does since it has no documentation >>>> and no tests. Normally I only reply to emails if I think I know >>>> enough to give an intelligent response, but in this case no one else >>>> is replying either. This might be wrong, but it's entirely possible >>>> that no one except you on this list has any way to check that these >>>> are correct. (Or if they do, the people who know are not replying.) >>>> >>>> So I propose this: if no one replies to this email in two days, I will >>>> merge the patches. >>>> >>>> And I hate to ask you for more after you've already contributed, but >>>> if you are able, could you write some documentation or tests for this >>>> module? It could badly use both of those things. >>>> >>>> Thanks a lot, >>>> Noah >>>> >>>> On Mon, Apr 30, 2012 at 8:42 AM, Daniel Krueger >>>> wrote: >>>>> Hi, >>>>> >>>>> I've done some work on (ice-9 occam-channel) and fixed the module >>>>> exports, the alt macro and extended it a little bit. Here are the >>>>> patches, but they are all just micro-commits, I hope that is okay. >>>>> >>>>> - Daniel >>>> >>> >> >> >> >> -- >> http://www.apgwoz.com
Re: SRFI-64 module and SRFI-78 module -- archive file attached
Hello, > I'd felt afraid of guile-devel@gnu.org because I'm just a newbie and a > poor enghlish reader/writer; especially of language problem. That's fine. I am sure that you will get used to it if you contribute a lot. > My goal? Hum... Like other people, I'd googled when I'd needed a test > suite. I'd heard the SRFI 64 and SRFI 78 but not been able to use it > then. So, I'd tried with unit-test in guile-lib. > > unit-test is good but my concern was abnormal case, i.e., error or > exception. unit-test supports assert-exception macro but I feel > insufficient when unexpected exception occur. So, I was back to SRFI 64 > and made it work on Guile. > > I think there are people googling for test suite like me. If Guile > include SRFI 64 in bundle, it would be helpful to them. Thanks for doing that! I have only read SRFI-64 yet, but I have a few comments. First of all, it would be nice you had tests for it. You might be able to use SRFI-64 to test itself, which would be cool. Second, since your changes add compatibility for several Scheme implementations, have you thought about trying to make them part of the reference implementation? You would probably have to email Per Bothner to ask him, but that might be the best way to make your changes available to all of the implementations you are making it for. If not, I hope Guile would like to have it, but I think you are targeting more than just Guile. Noah
Re: [PATCH] Turn on more documentation
Hello, >>> The problem is that the auto-generated “Standard Library” section looks >>> very poor in comparison to the rest of the manual. So we should really >>> try hard to write good doc by hands for these, and come up with a handy >>> structure (instead of one node per module, all under “Standard >>> Library”). >> >> I could do that pretty easily, but I'm afraid that documentation that >> isn't in the same file as the code would tend to be updated less often >> than documentation in the same file. > > All of our manual except “Standard Library” is maintained this way. It > takes discipline to keep up-to-date, but it also leads to a better > manual IMO. Okay, I can write documentation this way too. I have a question about this: what is the distinction between the sections "API Reference" and "Guile Modules"? I need to figure out where documentation for different modules goes. >>> However, I believe Thien-Thi’s Guile 1.4 has good doc for at least some >>> of these modules, and it would be great if it could be incorporated. >>> >>> WDYT? >> >> That would be ideal, but I was concerned that the documentation wasn't >> copyrighted by the FSF. Unless Thien-Thi is interested in contributing >> it, of course. > > Yes, I was hoping he would be reading us, let’s see. ;-) After almost a week, I think I'll plan on writing it myself. If someone else happens to contribute documentation, that'll be great too. Thanks, Noah
Re: Do you recognize these modules?
Hello, > The short answer is that the fork (from 2001 or so) was a reaction > to losing repo write privs. Now that i have regained them, i am > interested in merging back the relevant bits, w/ copyright notice > modified to the proper FSF standards. Details up to you... That sounds great to me! > I see from another post you have decided to rewrite, due to my > delayed response. Sorry about that. I anticipate my ability to > keep up in the foreseeable future to be likewise limited, so if > you're still interested, please let me know what i must do to get > these bits in-tree and useful to you (and others) as a basis for > further maintenance (kind of a one-shot mass-transfer). Luckily, I haven't actually started the rewrite, because I also have limited time to work on this. So I'm very glad to hear that you're interested in transfering them. If you are the only author of the pieces, then I think the transfer is simple - you put the FSF copyright notice at the top, and you're done. (Assuming the code works correctly on Guile 2.0. But the best way to check that is by running tests.) But Andy and Ludo are the real authorities on this. > Thanks for taking care of 1.8, btw -- i am sure i'm not alone in > appreciating this work. I'm happy to. Thanks a lot, Noah
Re: Do you recognize these modules?
>> Luckily, I haven't actually started the rewrite, because I also have >> limited time to work on this. So I'm very glad to hear that you're >> interested in transfering them. If you are the only author of the >> pieces, then I think the transfer is simple - you put the FSF >> copyright notice at the top, and you're done. > > Uh no. You can't just assign copyright to somebody without asking them, > much like you can't just assign parentship to somebody without asking > them. Yes, this was specific to this situation, because I thought that he had already signed a copyright assignment form. I don't mean this as a general statement of when you can make changes to Guile. > Since Thien-Thi Nguyen assigned copyright to GOOPS and general changes > to future changes to GUILE to the FSF in 2000 already, it is more or > less a matter of him checking in the changes or otherwise contributing > them in a manner making clear that the contribution is intentional. > Matching copyright headers are obviously a good indicator for that. Ah, so he would have to either check them in himself or send them as patches to the list, correct? (In addition to the assignment.) Noah
Re: [PATCH] Turn on more documentation
Hello, > From “Organisation of this Manual”: > > *Chapter 6: Guile API Reference* > This part of the manual documents the Guile API in > functionality-based groups with the Scheme and C interfaces > presented side by side. > > *Chapter 7: Guile Modules* > Describes some important modules, distributed as part of the Guile > distribution, that extend the functionality provided by the Guile > Scheme core. > > So I think the idea is for core functionality to be in Chapter 6, and > “peripheral things” to be in Chapter 7. The modules you mention would > fall in the second category, I think. That's certainly enough for this project, but I think in general this distinction is not very clear. How would someone guess what functionality is considered "core" and what functionality is an extension? My first guess would be that things in the (guile) module are core and everything else is an extension, but that is not the case. Does this come from an earlier time when the Guile core was distributed separately from the Guile libraries? Unless there is going to be some other distinction between core and extensions, it would seem more natural to me to document everything by functionality, in the same part of the manual. Some sections would correspond to modules, because modules are also supposed to group things by functionality, but that would not be the rule for how the manual worked. What do you think? Noah
Re: [PATCH] Turn on more documentation
Hello, >> Unless there is going to be some other distinction between core and >> extensions, it would seem more natural to me to document everything by >> functionality, in the same part of the manual. Some sections would >> correspond to modules, because modules are also supposed to group >> things by functionality, but that would not be the rule for how the >> manual worked. What do you think? > > I tend to agree, but then I’m not sure what a better sectioning would be > in practice. Any ideas? I think of our current situation as having three top-level lists of functionality, in API Reference, Guile Modules, and Standard Library. How about just merging them into one list? > Perhaps this could be worked out in ‘master’, so that node names don’t > change for 2.0.x. Yes, I certainly agree that we shouldn't change this for 2.0. Thanks, Noah
Re: Do you recognize these modules?
Hello, > If Noah is OK to help with the actual transfer of the doc bits, > then I guess we’re almost all set. :-) > > 1.4 snarfs comments from .scm files for interpolation in template > files to create the final texinfo, but i'm almost certain you > don't need/want that flow, but would rather have the final > texinfo. Is that correct? That's probably true, but I am curious about how you do this - Guile 1.4 and earlier apparently have a different documentation-snarfing format than Guile 1.6 and later. We just had a conversation about moving away from ours because it doesn't lead to well-written texinfo files, but maybe yours is more flexible. Noah
Re: [PATCH] Turn on more documentation
Hi, On Tue, May 15, 2012 at 5:25 PM, Ludovic Courtès wrote: > Hi, > > Andy Wingo skribis: > >> What if we merge "Guile Modules" and "Standard Library" into one >> "Standard Library" node, and keep "API Reference" as it is? > > Sounds good to me. Perhaps “Standard Library” could have sub-sections, > like “Data Structures”, “Web”, “XML”, “Texinfo”, etc. Yes, that sounds good to me too. However, On Tue, May 15, 2012 at 4:24 PM, Andy Wingo wrote: > For me the difference is that nodes of "API reference" describe > functionality that is often not segregated in modules, whereas "Guile > Modules" has nodes that describe modules. this distinction makes sense to me, but is not actually what is in place. For example, "LALR parsing", "Line Oriented and Delimited Text", "Block reading and writing", and "Local evaluation". How about having "API Reference" be the core module, and "Guile Modules" have everything else? I'm still not sure that's the right distinction though. Noah
Re: Register VM WIP
Hello, >> The register based VMs I've seen ignore this issue by allowing for an >> infinite set of registers. :) > > Indeed, that's the plan :) The first shot at an allocator will look a > lot like the one in (language tree-il analyze). That was a bit surprising to me. Do you mean that the register pool will grow and shrink for each function call? Is that why the stack frames can be fixed-size? Thanks, Noah
Re: Register VM WIP
Hi Mark, You are thinking along very similar lines to how I used to think. But I have a different way to think about it that might make it seem better. In our current VM, we have two stacks: the local-variable stack, which has frames for different function calls and is generally what you'd think of as a stack, and the temporary-variable stack, which is literally a stack in the sense that you only operate on the top of it. The temporary-variable stack makes us do a lot of unnecessary work, because we have to load things from the local-variable stack to the temporary-variable stack. I think what Andy is proposing to do is to get rid of the temporary-variable stack and operate directly on the local-variable stack. We shouldn't think of these registers as being like machine registers, and in fact maybe "registers" is not a good name for these objects. They are really just variables in the topmost stack frame. This should only reduce memory usage, because the local-variable stack stays the same and the temporary-variable stack goes away (some temporaries might move to the local-variable stack, but it can't be more than were on the temporary-variable stack, so that's still a win). The machine I was initially thinking of, and I imagine you were too, is different. I had imagined a machine where the number of registers was limited, ideally to the length of a processor cache line, and was separate from the local-variables stack. In such a machine, the registers are used as a cache for the local variables, and you get to deal with all the register allocation problems that a standard compiler would. That would accomplish the goal of keeping more things in cache. The "registers as cache" idea may result in faster code than the "directly addressing local variables" idea, but it's also more complicated to implement. So it makes sense to me that we would try directly addressing local variables first, and maybe later move to using a fixed-size cache of registers. It also occurs to me that the RTL intermediate language, which is really just a language for directly addressing an arbitrary number of local variables, is a standard compiler intermediate language. So it might be useful to have that around anyway, because we could more easily feed its output into, for instance, GCC. Andy, is this an accurate description of the register VM? And Mark and everyone else, does it seem better when you look at it this way? Noah On Wed, May 16, 2012 at 9:44 AM, Mark H Weaver wrote: > Hi Andy! > > Andy Wingo writes: >> On Wed 16 May 2012 06:23, Mark H Weaver writes: >> >>> It's surprising to me for another reason: in order to make the >>> instructions reasonably compact, only a limited number of bits are >>> available in each instruction to specify which registers to use. >> >> It turns out that being reasonably compact isn't terribly important -- >> more important is the number of opcodes it takes to get something done, >> which translates to the number of dispatches. Have you seen the "direct >> threading" VM implementation strategy? In that case the opcode is not >> an index into a jump table, it's a word that encodes the pointer >> directly. So it's a word wide, just for the opcode. That's what >> JavaScriptCore does, for example. The opcode is a word wide, and each >> operand is a word as well. >> >> The design of the wip-rtl VM is to allow 16M registers (24-bit >> addressing). However many instructions can just address 2**8 registers >> (8-bit addressing) or 2**12 registers (12-bit addressing). We will >> reserve registers 253 to 255 as temporaries. If you have so many >> registers as to need more than that, then you have to shuffle operands >> down into the temporaries. That's the plan, anyway. > > I'm very concerned about this design, for the same reason that I was > concerned about NaN-boxing on 32-bit platforms. Efficient use of memory > is extremely important on modern architectures, because of the vast (and > increasing) disparity between cache speed and RAM speed. If you can fit > the active set into the cache, that often makes a profound difference in > the speed of a program. > > I agree that with VMs, minimizing the number of dispatches is crucial, > but beyond a certain point, having more registers is not going to save > you any dispatches, because they will almost never be used anyway. > 2^12 registers is _far_ beyond that point. > > As I wrote before concerning NaN-boxing, I suspect that the reason these > memory-bloated designs are so successful in the JavaScript world is that > they are specifically optimized for use within a modern web browser, > which is already a memory hog anyway. Therefore, if the language > implementation wastes yet more memory it will hardly be noticed. > > If I were designing this VM, I'd work hard to allow as many loops as > possible to run completely in the cache. That means that three things > have to fit into the cache together: the VM itself, the user loop code, >
Re: Register VM WIP
> Perhaps it needs a different name than "register virtual machine". How about "RTL VM", since it's a virtual machine that interprets RTL? Or maybe "frame-addressed VM", because the operations address objects in the current stack frame? Noah
Error in wip-rtl
Hello, I tried to build the latest wip-rtl today, and hit an error that I don't know how to debug. First, when the build gets to "GEN guile-procedures.texi", I get a segmentation fault. I tried to debug it by doing meta/gdb-uninstalled-guile. When I did that, Guile made it to a command prompt, but when I try to evaluate any sort of expression, even #t, I got "ERROR: unbound variable: compile". What might make that happen? Thanks, Noah
Re: Error in wip-rtl
On Wed, May 23, 2012 at 3:10 AM, Andy Wingo wrote: > On Wed 23 May 2012 04:04, Noah Lavine writes: > >> I tried to build the latest wip-rtl today, and hit an error that I >> don't know how to debug. First, when the build gets to "GEN >> guile-procedures.texi", I get a segmentation fault. >> >> I tried to debug it by doing meta/gdb-uninstalled-guile. When I did >> that, Guile made it to a command prompt, but when I try to evaluate >> any sort of expression, even #t, I got "ERROR: unbound variable: >> compile". >> >> What might make that happen? > > Not sure; I assume you tried a clean build? > > Anyway I have some updates so I would hold off building until later > today. Yes, I usually start with 'make distclean' to avoid these errors. But I tried building again today and it went fine. The error must be gone. Noah
Re: assembler in scheme
On Sat, May 26, 2012 at 10:58 PM, Nala Ginrut wrote: > Well, speaking this. I have a question that, is there any convenient > way to add multi-backend in Guile? It depends on what you mean by that. You can add as many low-level languages as you like, including a GCC interface, machine code, and JVM bytecode. The compiler framework doesn't know the difference between frontends and backends (I think) - it just compiles from one language to another, using the compilers it knows about. When you're actually running Guile, only one of those languages can be executed, and it's the one corresponding to the actual virtual machine running Guile. But you could use the framework to statically compile JVM bytecode just fine, I think. It would be very cool to be able to run Guile on the JVM. I think it would really show the advantages of the compilation framework, because the Guile VM and JVM implementations would use the same intermediate language, with the same optimizations. However, I think that would be a lot of work. Noah
Question About the Variable Allocator
Hello,
I've always been puzzled about part of the variable allocator. In
module/language/tree-il/analyze.scm, we deal with allocations, which
are hash tables that say where in the stack each local variable goes.
The maps are two level, symbol -> {lambda -> location}. The reason
given is that different lambdas could have different variables with
the same symbol.
But if I understand correctly, each variable also gets a gensym, and
the gensyms are globally unique. So wouldn't it be possible to use the
gensyms as keys instead, and only have a single level map? If so, why
don't we do it?
Thanks,
Noah
RTL Question
Hello, I've been playing around with the wip-rtl branch, and I have an error I don't understand. Why does this give me a "bad instruction" error: (use-modules (system vm rtl)) (define prog (assemble-rtl-program 2 '((assert-nargs-ee/locals 1 1) (mov 0 1 (prog 3) ? On a related note, I've been working on a small Tree-IL -> RTL compiler. If anyone else is working on one, perhaps we should work together. If not, I'll hopefully have something that we could use relatively soon. Noah
Re: Separate textual/binary ports vs. mixed ports
I agree that separate binary and textual ports are cleaner, but what about using a port to deal with a mixed binary/textual protocol, like HTTP? I think the cleanest way to deal with that would be to have a port where you first read characters and then read binary data. That doesn't directly address the string-port issue, though. Noah On Tue, Jun 5, 2012 at 5:31 AM, Daniel Krueger wrote: > Hey, > > I think if you only use them seperate there's a clearer distinction. > If you have it mixed you can do some, say hacking, where you see it > works but you can't see anywhere what you're exactly doing, most of it > is hidden in the guile implementation, which interprets > %default-port-conversion-strategy and gives you the coding. In one > case you maybe rely on %default-port-conversion-strategy normally > being UTF-8 and then someone sets it to something else, which could > give some hard to track errors. I think explicity just makes code much > clearer and I think seperating textual and binary ports leads to more > explicity, that's my point of view.. > > WDYT? > > - Daniel >
Re: Growable arrays?
Hello, >> vlist is a data type introduced around guile 2.0. You will find it >> documented in the Guile Reference under Compound Data Types. >> >> They are growable and provide vector-like access performances and >> memory locality. > > Ah, too bad. This needs to run under 1.8 as well. If you need to support older versions of Guile, then you can't use any data structures we add now anyway, correct? So it seems like you will have to implement growable vectors yourself no matter what. If you want to, though, you could look at the implementation of vlists in Guile 2.0, which I think is all in Scheme. See module/ice-9/vlist.scm. Even if we can't fix it, it is still nice to hear about data structures you wish Guile had, so that in a few more versions this might not be a problem for you. Noah
features.h in wip-rtl
Hello, I've hit a problem building a recent wip-rtl. It now includes the file libguile/elf.h, which does "#include ". I'm building Guile on Mac OS X, and I think that doesn't have features.h, so the build breaks. However, I don't see any uses of feature.h in elf.h - in particular, there aren't any ifdef's. Can we take out that inclusion? Thanks, Noah
Re: patching gcc to allow other calling conventions
Hello, > Did you consider starting from GNU/MIT Scheme? It supports only IA32 > and x86_64, I think, but it’s in Scheme, and it’s GNU. Actually, that's an interesting thought in general. I looked at MIT scheme a bit a long time ago, but I believe it uses two intermediate languages, a high-level one similar to Tree-IL and a low-level one that I don't know much about. We might be able to turn Tree-IL into the high-level one and use their compiler infrastructure. Since they're a GNU project, there might not be copyright issues. However, I'm not sure if this has advantages over just building it ourselves. And I don't know if the MIT Scheme developers would like this or not. Noah
Re: patching gcc to allow other calling conventions
Hello, > But the used sbcl derivative although not gnu is either in the public domain > or bsd so we should be able to publish what we are doing. I prefere now to > start working on a simple jit scheme for the fun of it, because it is a good > learning experience and that getting results are a good driver to continueu. A JIT would be a good thing for Guile to have. I worked on a JIT a while ago, and I found that the hard part was not generating machine code or connecting it to the VM - both of which it looks like you've done - but making the JIT engine understand all of the VM instructions. You could just hard-code them all, but then you've got a complete duplicate of vm-engine.c with exactly the same information, but in different syntax. So I thought I wanted some way to generate the JIT from vm-engine.c, but that requires parsing C code. That's a capability I want Guile to have in general, but I haven't made it happen yet. What do you think about this? Would you just want to maintain the JIT engine separately from vm-engine.c, or would you like to automatically generate it? (Note: you could also generate vm-engine.c and your JIT from some third source, but I think we rejected that for being too complicated. It would certainly make the build process more difficult.) Noah > /stefan > > Den 18 jun 2012 02:43 skrev "Noah Lavine" : > >> Hello, >> >> > Did you consider starting from GNU/MIT Scheme? It supports only IA32 >> > and x86_64, I think, but it’s in Scheme, and it’s GNU. >> >> Actually, that's an interesting thought in general. I looked at MIT >> scheme a bit a long time ago, but I believe it uses two intermediate >> languages, a high-level one similar to Tree-IL and a low-level one >> that I don't know much about. We might be able to turn Tree-IL into >> the high-level one and use their compiler infrastructure. Since >> they're a GNU project, there might not be copyright issues. >> >> However, I'm not sure if this has advantages over just building it >> ourselves. And I don't know if the MIT Scheme developers would like >> this or not. >> >> Noah >> >
RTL Question
Hello, Time for my next RTL question - how do I use the toplevel-ref instruction? It looks like I need to make a variable object in the instruction stream, or at a known offset from it. I think I should use either make-non-immediate or the linker to do that, but I don't quite know how. Thanks, Noah
Re: Bug in documentation for eq? ?
Hello, I think you're talking past each other a little bit. Andy is saying that the Scheme standard doesn't specify eq? on numbers. David is saying that an object should always be eq? to itself, no matter what object. I believe David's claim is that Guile should guarantee that a variable is eq? to itself, even though the standard doesn't. I don't know what the right answer is, but I think that's what this discussion should be about. Noah On Wed, Jun 20, 2012 at 9:17 AM, Andy Wingo wrote: > Hi, > > [bunch of examples] >> Which of the above would you consider unspecified? > > As the Scheme standard clearly states, all the ones comparing numbers > with eq?. You find some of them surprising; that is your problem ;) The > answer is to not compare numbers with eq?. > > Regards, > > Andy > -- > http://wingolog.org/ >
Re: wip-rtl, solstice edition
Hello, The new VM looks great. > But after those things are done, we still need to bridge the gap between > Tree-IL and RTL assembly. We will probably have to scrap GLIL, though I > can't tell yet. As I said, I've been working on a compiler from Tree-IL directly to RTL. So far, I have not found that I really wanted another layer in between the RTL and the Tree-IL. It looks to me like the GLIL->assembly compiler spends most of its effort building constant tables, which I believe the RTL assembler does in the rtl branch. So GLIL wouldn't be abstracting over much compared to plain RTL, unless you moved that out of the assembler, or made the GLIL-equivalent higher level in some way. > So, that's the status. Apologies for there being no overview yet; I > will try to write something about that soon. I reckon we are about a > month away from a VM that works well, and has good debugging > information, and two or three months away from a merge to master > (meaning, we compile all of Scheme). At that point we could look to > release the first 2.2 beta release, aiming at a final 2.2 sometime early > next year. This is very exciting! Thanks for working on the new VM. Noah
Re: wip-rtl, solstice edition
Hello, > Seriously though, that seems like a good plan. I wonder what Noah’s > attempts at JITing the 2.0 bytecode would have achieved, though, if we > think of both JIT and the new VM as an “interim solution” before AOT > native compilation. I can't remember the last email I sent about that, but I think I might have dropped the ball here, so let me say what the current status of the project was. The JIT compiler worked fine. You could JIT-compile a function and have Guile automatically run the JITted code. The real problem was writing the JIT compiler - the one I had only supported four instructions, because I just wanted to prove I could integrate it with the rest of Guile. I sent another email recently about different ways to make the JITter understand all of the bytecode, but at the time, I thought I would have to parse the C definition of the VM and generate the JITter from that in order for it to be merged into Guile. I never got over that hurdle. If we are willing to generate both the VM and the JITter from another source, it's possible that I could revive my old JIT branch. However, I'm still not sure if that's the best way there. Having the assembler written in Scheme gives you an easier path towards all-Scheme native compilation, which you'd probably want for an AOT compiler. Noah
Re: Preparing for 2.0.6
I think bug 10623 is fixed in stable-2.0, but I don't know how to close it in the tracker. Thanks, Noah On Thu, Jun 28, 2012 at 4:05 PM, Ludovic Courtès wrote: > Hello! > > Still hesitant between writing a native code back-end for IA64 and > making Guile multiboot-compliant? Hesitate no more! > > Time has come for 2.0.6, so let’s squash bugs! > > One thing I’d like to get in by then is the SRFI-9 “functional record > setters”. Mark: what’s your take on this? > > Optionally, integrating SRFI-64 would be great. > > Other than that, fixing as many bugs as we can. > > Any other ideas? > > Andy: would you like to take care of NEWS? :-) > > I’ll be mostly away from July 9th for some time, so I could push the > tarball by then, or otherwise during the GHM. > > Thanks, > Ludo’. > >
Re: Preparing for 2.0.6
On Thu, Jun 28, 2012 at 10:38 PM, Nala Ginrut wrote: > @Noah: Could you apply the patch to fix ecmascript? > http://lists.gnu.org/archive/html/guile-devel/2012-05/msg1.html I think that's already done! Do you not see it in your tree? Noah
Re: Do you recognize these modules?
Ooops, I think those errors are because I forgot to commit my changes to guile.texi for patches 3 and 4. Here are updated versions. (I believe the second patch from earlier should work.) I will make another patch to clean out the @twerpcommentary lines. Noah On Wed, Jul 11, 2012 at 6:44 AM, Andy Wingo wrote: > On Wed 11 Jul 2012 05:02, Noah Lavine writes: > >> I'm sorry to miss 2.0.6 by only a few days, but I have turned this >> documentation into some patches. Here they are; I think they are ready >> to apply to stable-2.0. > > Cool. I applied the first one. When applying the rest, I got: > > /home/wingo/src/guile/doc/ref//lineio.texi:12: Prev reference to > nonexistent node `Double-Key Hash Tables' (perhaps incorrect sectioning?). > /home/wingo/src/guile/doc/ref//hcons.texi:12: `Eq? Hash Consing' has no > Up field (perhaps incorrect sectioning?). > /home/wingo/src/guile/doc/ref//gap-buffer.texi:12: Next reference to > nonexistent node `Double-Key Hash Tables' (perhaps incorrect sectioning?). > /home/wingo/src/guile/doc/ref//guile.texi:375: Menu reference to > nonexistent node `Double-Key Hash Tables' (perhaps incorrect sectioning?). > /home/wingo/src/guile/doc/ref//hcons.texi:12: warning: unreferenced node > `Eq? Hash Consing'. > makeinfo: Removing output file `guile.info' due to errors; use --force to > preserve. > make[4]: *** [guile.info] Error 1 > > As Thien-Thi suggests, it's probably good to filter out the > @twerpcommentary etcetera in your next patches. > > Cheers, > > Andy > -- > http://wingolog.org/ 0001-Documentation-style-change.patch Description: Binary data 0002-Document-ice-9-string-fun.patch Description: Binary data
How do I call functions in RTL?
Hello, I'm trying to understand the call instruction in RTL, and having some trouble. I'm trying to write the simplest function I can that calls another function. I decided to do the equivalent of (lambda (x) (x)), so I don't have to worry about top-level variable references. So I tried the following in the latest git wip-rtl: (define (const) 3) (define call-arg (assemble-program '((begin-program call-arg) (assert-nargs-ee/local 1 1) (call 1 1 1 ()) (return 1 (call-arg const) If I understand correctly, call-arg is called with the procedure const in register 0. Then it calls const, asking const to put its result in register 1, and finally returns const's result, which is 3. Instead, I got this error: ERROR: In procedure (# . #0#): ERROR: Wrong type to apply: (# . #0#) I didn't understand this error, but after puzzling over the code for a while, I realized that I was calling const from register 1 instead of register 0. So I don't understand why const was even in the error, but I fixed the typo and tried again like this: (define call-arg (assemble-program '((begin-program call-arg) (assert-nargs-ee/locals 1 1) (call 1 1 0 ()) (return 1 (call-arg const) That gave a segmentation fault. So I have three questions: What did my first attempt really do? Why did my second attempt fail? And how do I really use the call instruction? Thanks, Noah
Re: wip-rtl return location
That sounds interesting, but I have a question - why not make the MVRA return address immediately after the call, instead of immediately before it? In the common case when returning to the regular return address, that would eliminate the extra branch (although it's a very small branch anyway). I would guess the reason to put it before is for variable-length instructions, but you could handle that by reserving enough bytes for a jump instruction. So it would look like CALL: call f MVRA: jump mv-handler RA: ... rest of function ... I really don't know if this is better or not. I'm just curious why it isn't arranged like this. Thanks, Noah On Thu, Aug 2, 2012 at 10:29 AM, Andy Wingo wrote: > Hi, > > Some brief thoughts on the wip-rtl branch. Currently it has this > strange "return location" thing, where it specifies the register(s) to > which to return value(s), and the number of expected values and whether > it expects a rest list or not. Problem is, this return location is like > a little program that needs to be interpreted at runtime. Worse, it > seems to assume that return values will have to be passed in memory. > > Instead I'd rather just use Dybvig's suggestion: every call instruction > is preceded by an MV return address. For e.g. (values (f)), calling `f' > would be: > > ... > goto CALL > MVRA: > truncate-and-jump RA > CALL: > call f > RA: > return > > So the overhead of multiple values in the normal single-value case is > one jump per call. When we do native compilation, this cost will be > negligible. OTOH for MV returns, we return to a different address than > the one on the stack, which will cause a branch misprediction (google > "return stack buffers" for more info). Of course this is not relevant > to the interpreter, because all of these branches are indirect, but it > will be in the future, so it's a good idea to think about these things > now. With this design, the caller is responsible for handling MV > returns, not the callee. > > Anyway, MV return will cause a branch misprediction. Oh well. I think > we can live with it. Single-valued returns are the common case, and > they will be predicted correctly. > > So, another thing. The reason for the previous "return location" design > was because I wanted to have just two registers reserved by the > implementation: the instruction pointer and the frame pointer. Wanting > an IP is obvious. It's important to locate frame pointers so that > various pieces of code can walk the stack frames: for example the > delimited continuation code, the backtrace printer, the debugger, etc. > It's possible to just using a stack pointer and use dynamic tables to > find where the frame pointer is, like the x86-64 architecture does (or > -fomit-frame-pointer), but that requires more sophistication on the part > of the runtime, and I don't think we're really ready for that right now. > > As I said, I wanted just the IP and the FP. I didn't want an SP because > it causes so much performance noise in the current VM. But then I > realized that in the RTL VM, it doesn't need to be accessed frequently, > because more values are addressed against the FP, and we're not pushing > and popping temporaries. So we can actually keep it around, and it > might not need to be in a register. It retains its useful > characteristics of allowing variable-sized data to be (temporarily) > allocated on the stack, as in procedure calls or MV returns, and as a > stack delimiter for GC. > > In summary: > > - I will remove the "return location" stuff from wip-rtl; > > - All calls will be mv-calls > > - MV returns will return to 1 instruction before the RA > > - All calls will be preceded by a jump over the MVRA > > - Eventually we can remove the MVRA slot from stack frames, because it > is computable from the RA > > - The stack pointer is back in town! > > Andy > -- > http://wingolog.org/ >
Re: wip-rtl native closure creation
I haven't looked at the RTL program structure, but adding a new field is basically what I did with the non-RTL program structure when I worked on JIT there. However, in that case we could still keep everything under 4 words. I don't know if that will work here. Noah On Mon, Aug 6, 2012 at 5:39 AM, Sjoerd van Leent Privé wrote: > On 06-08-12 11:32, Andy Wingo wrote: >> >> On Sun 05 Aug 2012 17:19, Stefan Israelsson Tampe >> writes: >> >>> Probably it is best to have the first qword / dword in the code to be >>> 0 or the native adress e.g. I propose to add that feature to the >>> rtl-branch. >> >> Good question! Given the different tradeoffs, that seems workable. >> Another possibility would be to use a different TC7 for native >> procedures. After all, the only calls we need to make cheaply are >> native->native and bytecode->bytecode, and the rest can go through a >> general dispatch loop (possibly with inline caching). WDYT? (Also note >> that RTL words are 32 bits wide, which may or may not be sufficient for >> native code pointers.) >> >> Andy > > Wouldn't it be feasible in the future that there might be, because of more > memory, other designs, such as caching, which create much more closures than > current designs? I don't know, but on 64-bit platforms (and perhaps even > architectures with a larger bus), it seems to me that it is necessary to > stick to this bus length. >
Re: Do you recognize these modules?
Any objections if I apply these patches soon? I'd like to get them off of my to do list. Thanks, Noah On Wed, Jul 11, 2012 at 1:59 PM, Noah Lavine wrote: > And here is a patch to remove all of the @twerp* comments. But did you > mean that I should remove them in the same patches that import the > documentation? I thought it was easier to do it separately, but I > could probably do it that way too. > > Noah > > On Wed, Jul 11, 2012 at 11:20 AM, Noah Lavine wrote: >> Ooops, I think those errors are because I forgot to commit my changes >> to guile.texi for patches 3 and 4. Here are updated versions. (I >> believe the second patch from earlier should work.) >> >> I will make another patch to clean out the @twerpcommentary lines. >> >> Noah >> >> On Wed, Jul 11, 2012 at 6:44 AM, Andy Wingo wrote: >>> On Wed 11 Jul 2012 05:02, Noah Lavine writes: >>> >>>> I'm sorry to miss 2.0.6 by only a few days, but I have turned this >>>> documentation into some patches. Here they are; I think they are ready >>>> to apply to stable-2.0. >>> >>> Cool. I applied the first one. When applying the rest, I got: >>> >>> /home/wingo/src/guile/doc/ref//lineio.texi:12: Prev reference to >>> nonexistent node `Double-Key Hash Tables' (perhaps incorrect sectioning?). >>> /home/wingo/src/guile/doc/ref//hcons.texi:12: `Eq? Hash Consing' has no >>> Up field (perhaps incorrect sectioning?). >>> /home/wingo/src/guile/doc/ref//gap-buffer.texi:12: Next reference to >>> nonexistent node `Double-Key Hash Tables' (perhaps incorrect sectioning?). >>> /home/wingo/src/guile/doc/ref//guile.texi:375: Menu reference to >>> nonexistent node `Double-Key Hash Tables' (perhaps incorrect sectioning?). >>> /home/wingo/src/guile/doc/ref//hcons.texi:12: warning: unreferenced >>> node `Eq? Hash Consing'. >>> makeinfo: Removing output file `guile.info' due to errors; use --force >>> to preserve. >>> make[4]: *** [guile.info] Error 1 >>> >>> As Thien-Thi suggests, it's probably good to filter out the >>> @twerpcommentary etcetera in your next patches. >>> >>> Cheers, >>> >>> Andy >>> -- >>> http://wingolog.org/
Re: guildhall packages
Hello, On Sun, Aug 26, 2012 at 5:29 PM, Ludovic Courtès wrote: > Ian Price skribis: >> If you think was a useful email, I can post a periodic update on new >> packages and/or provide an atom feed for them. If not, I'll shut up :) > > I like the updates, keep up the good work! :-) I also appreciate the messages and the hard work. Thank you! Noah
Re: Adding to guile curly-infix (SRFI 105), neoteric- & sweet-expressions
Hello, On Mon, Aug 27, 2012 at 12:30 AM, Alan Manuel Gloria wrote: > However, it leads to an edge case in Guile 2.0 where disabling > autocompilation leads to the module-loading C code path going through > a direct C call to the C implementation of primitive-load, a path that > only triggers if autocompilation disabled (when autocompilation is > enabled, it goes through a hook in the language support for Scheme, > which uses the 'read function we've rebound). Hmm, interesting. That sounds like a bug, but I'd like one of the Guile maintainers to clarify that this isn't intended before I look at it more. Is this triggered when you're trying to load a module full of infix expressions? If so, how do you tell Guile that the module is supposed to use infix expressions instead of s-expressions? > I had also considered adding support for sweet-expressions in a > language module for Guile, but this would require 2.0 (my own personal > needs mean that I need it to work way back in 1.6) , and I couldn't > figure out a way to tell Guile to start the REPL in sweet-expression > language (or any language other than Scheme for that matter). Guile versions before 2.0 don't have support for multiple languages. The big change in the 2.0 release was the move from a direct interpreter to a virtual machine, which made multi-language support practical (and also made Guile much faster). > Sincerely, > AmkG Hoping this is helpful, Noah
Two build problems
Hello, I hit two errors while building recent Guile, one of which I have diagnosed. First problem: At first, I couldn't build lib/striconveh.c. Here's what I think was wrong: Make doesn't know that lib/striconveh.c depends on lib/unitypes.h. This is a problem because lib/unitypes.h is generated from lib/unitypes.in.h, but because Make doesn't know about the dependency, it doesn't bother to generate lib/unitypes.h before building striconveh.c, thus generating an error. Here's some evidence that this is true: when I manually make lib/unitypes.h and then make Guile the regular way, the build succeeds. (I actually decided this by manually following the includes and noticing that lib/unitypes.h doesn't exist, then looking in the Makefile to find out why.) I'm not quite sure what the right fix is, for two reasons. First of all, the dependency isn't direct - lib/striconveh.c includes lib/unistr.h, which then includes lib/unitypes.h. So maybe lib/unistr.h should depend on lib/unitypes.h, and striconveh.c shouldn't be directly involved. Second, I believe lib/ actually comes from gnulib, and I don't know how that's organized. So I don't know if this is a Guile bug or a gnulib bug. Second problem: Since I could fix the first one by hand, I decided to go ahead and build Guile. That's when I discovered that even though lib/unitypes.h exists, it's not being included by lib/unistr.h when building files in the libguile/ directory (even though it works fine when building things in the lib/ directory). Therefore, an include path is messed up somewhere. lib/unistr.h says this: #include "unitypes.h". But when I try to build bytevectors.c, I get lots of errors from lib/unistr.h that seem to be based on a lack of macros from lib/unitypes.h. As an experiment, I added the line "#error" to lib/unitypes.h. As expected, the preprocessor and compiler didn't compain about it, which I think means that they're never looking in lib/unitypes.h when building libguile/bytevectors.c. This is a problem. Unfortunately, I don't know how to fix this one. Defining the macro INCLUDES in libguile/Makefile to -I../lib:./lib doesn't seem to help. Thanks, Noah
Re: Two build problems
I have found the cause of the second problem, but I'm not sure what to do about it. Here's what's happening: libguile/bytevectors.c includes . uniconv.h lives in my system include directory (~/.nix-profile/include, actually). It includes a file called "unitypes.h". That refers to ~/.nix-profile/include/unitypes.h, which does the usual #ifdef UNITYPES_H ... #endif trick to avoid being included more than once. But then on the next line, libguile/bytevectors.c includes . That refers to lib/unistr.h, not the system unistr.h. lib/unistr.h also includes "unitypes.h", but the version of unitypes.h in lib/ is different than my system version. However, the version of unitypes.h in lib/ is never used, because it also has a #ifdef UNITYPES_H ... #endif wrapper. So lib/unistr.h doesn't get some #defines it needs, which causes a build failure. So that's the issue. What is the correct solution? Thanks a lot, Noah On Sat, Sep 15, 2012 at 2:40 PM, Noah Lavine wrote: > Hello, > > I hit two errors while building recent Guile, one of which I have diagnosed. > > First problem: > > At first, I couldn't build lib/striconveh.c. Here's what I think was > wrong: Make doesn't know that lib/striconveh.c depends on > lib/unitypes.h. This is a problem because lib/unitypes.h is generated > from lib/unitypes.in.h, but because Make doesn't know about the > dependency, it doesn't bother to generate lib/unitypes.h before > building striconveh.c, thus generating an error. > > Here's some evidence that this is true: when I manually make > lib/unitypes.h and then make Guile the regular way, the build > succeeds. (I actually decided this by manually following the includes > and noticing that lib/unitypes.h doesn't exist, then looking in the > Makefile to find out why.) > > I'm not quite sure what the right fix is, for two reasons. First of > all, the dependency isn't direct - lib/striconveh.c includes > lib/unistr.h, which then includes lib/unitypes.h. So maybe > lib/unistr.h should depend on lib/unitypes.h, and striconveh.c > shouldn't be directly involved. Second, I believe lib/ actually comes > from gnulib, and I don't know how that's organized. So I don't know if > this is a Guile bug or a gnulib bug. > > Second problem: > > Since I could fix the first one by hand, I decided to go ahead and > build Guile. That's when I discovered that even though lib/unitypes.h > exists, it's not being included by lib/unistr.h when building files in > the libguile/ directory (even though it works fine when building > things in the lib/ directory). Therefore, an include path is messed up > somewhere. > > lib/unistr.h says this: #include "unitypes.h". > > But when I try to build bytevectors.c, I get lots of errors from > lib/unistr.h that seem to be based on a lack of macros from > lib/unitypes.h. As an experiment, I added the line "#error" to > lib/unitypes.h. As expected, the preprocessor and compiler didn't > compain about it, which I think means that they're never looking in > lib/unitypes.h when building libguile/bytevectors.c. This is a > problem. > > Unfortunately, I don't know how to fix this one. Defining the macro > INCLUDES in libguile/Makefile to -I../lib:./lib doesn't seem to help. > > Thanks, > Noah
Re: Two build problems
Guile seems to build fine after "make distclean", so I guess the problem was left-over files. I may also have installed another package in an attempt to work around this, though. I'd rather not try reverting package installs to see if I can break it again. It's an unusual problem - in order to reproduce it, you have to have a system with two distinct library directories, one of which has uniconv.h but not unistr.h, and they have to have different definitions of unitypes.h. I don't know of a good solution, but I don't think it's worth worrying about right now. It should never occur for distro builds of Guile, because they will be packaging all of their libraries together, so they'll only have one version of uniconv.h. Noah On Wed, Sep 26, 2012 at 5:41 PM, Ludovic Courtès wrote: > Hi Noah, > > This vaguely rings a bell... > > Noah Lavine skribis: > >> libguile/bytevectors.c includes . >> >> uniconv.h lives in my system include directory > > [...] > >> But then on the next line, libguile/bytevectors.c includes . >> That refers to lib/unistr.h, not the system unistr.h. > > Could it be that lib/unitstr.h is a left-over from a previous build? > > Can you check whether the problem exists when building from a fresh > checkout? (Hydra is happy, FWIW...) > > Thanks, > Ludo’. > >
More RTL Tests
Hello, I have been working on understanding RTL, and I wrote the following tests. They're mostly to illustrate for myself how calling works in RTL, but they also serve to test it. Any objections if I commit them as part of rtl.test? (with-test-prefix "call" (assert-equal 42 (let ((call ;; (lambda (x) (x)) (assemble-program '((begin-program call) (assert-nargs-ee/locals 1 0) (call 1 0 ()) (return 1) ;; MVRA from call (return 1) ;; RA from call (call (lambda () 42 (assert-equal 6 (let ((call-with-3 ;; (lambda (x) (x 3)) (assemble-program '((begin-program call-with-3) (assert-nargs-ee/locals 1 1) (load-constant 1 3) (call 2 0 (1)) (return 2) ;; MVRA from call (return 2) ;; RA from call (call-with-3 (lambda (x) (* x 2)) (with-test-prefix "tail-call" (assert-equal 3 (let ((call ;; (lambda (x) (x)) (assemble-program '((begin-program call) (assert-nargs-ee/locals 1 0) (tail-call 0 0) (call (lambda () 3 (assert-equal 6 (let ((call-with-3 ;; (lambda (x) (x 3)) (assemble-program '((begin-program call-with-3) (assert-nargs-ee/locals 1 1) (mov 1 0) ;; R1 <- R0 (load-constant 0 3) ;; R0 <- 3 (tail-call 1 1) (call-with-3 (lambda (x) (* x 2)) My next goal is to learn to reference top-level variables. Could anyone tell me how that works, to make it easier? Thanks, Noah
Re: Needed: per-port reader options
Hello, On Tue, Oct 16, 2012 at 5:39 PM, Ludovic Courtès wrote: > I think ‘current-reader’ should remove the need to have a port-to-reader > mapping, no? > > Thanks for looking into this! > > Ludo’. > I might not understand this correctly, but aren't the reader flags only supposed to affect the specific file they're reading from? That's why we need a different set of reader options for each port. Of course, there could be one global reader that has a port-to-reader-options mapping, but that's a lot like a port-to-reader mapping. I think what you're suggesting is making a new reader for each input file, letting that reader set its flags based on directives, and then abandoning that reader when the file is read. Is that right? I'm just trying to understand the issue, Noah
Appending Queues
Hello, I was just working on a project that used (ice-9 q), and I found that I needed to append two queues. I wrote the following functions to do it. What do you think of including them in (ice-9 q)? It's pretty simple, but it seems like a natural part of the queue interface. I've included destructive and non-destructive versions. The only change I was considering is allowing an arbitrary number of arguments, but that's something I can implement if people agree that we want the functions in (ice-9 q). (define (append-qs q r) (cons (append (car q) (car r)) (cdr r))) (define (append-qs! q r) (set-cdr! (cdr q) (car r)) (set-cdr! q (cdr r)) q) Thanks, Noah
Re: Appending Queues
Hello, On Fri, Nov 2, 2012 at 1:20 PM, Ian Price wrote: > > I'm loath to add anything to (ice-9 q) since I find the names, and > the lack of a distinct type, less than satisfactory. > Fair enough. I'd be just as happy to implement a new queue container. I think we need to keep (ice-9 q) around for compatibility, though, so I'm tempted to say we should go ahead and make it as good as it can be. > However, if you are going to add them, I'd recommend sticking with the > q- prefix. So, q-append and q-append!. Keeping the naming consistent > means one less exception to remember. > > Also, I think it might be wise to add a list->q and q->list, with the > obvious simple implementations > You're right. :-) Noah
Re: [PATCH] Add ".guile.sls" and ".sls" to the default %load-extensions
If it's semi-standard, we should probably support it too.
However, a Google search reveals the following other uses of .sls:
- A list of images for a slideshow ("sls" stands for "slideshow script")
- The backup files for some program callled Litespeed
- ScriptLab Scripts (seems to be an image generation program)
- Configuration files for the Salt infrastructure management tool
So we probably shouldn't use it ourselves. (Plus, I personally think it's
uglier than .scm, but that's not a strong argument.)
Noah
On Sat, Nov 3, 2012 at 11:31 PM, Mark H Weaver wrote:
> Hello all,
>
> Any objections to adding ".guile.sls" and ".sls" to Guile's default
> %load-extensions? Ian Price tells me that this naming convention is
> commonly followed for R6RS libraries and implementations, e.g. Racket,
> Mosh, Ikarus, and Ypsilon. It would facilitate easy use of R6RS
> libraries without having to rename the files. I'm inclined to go along.
>
> What do you think?
>
> Mark
>
>
>
Re: thoughts on native code
Hello, I assume "compressed native" is the idea you wrote about in your last email, where we generate native code which is a sequence of function calls to VM operations. I really like that idea. As you said, it uses the instruction cache better. But it also fixes something I was worried about, which is that it's a lot of work to port an assembler to a new architecture, so we might end up not supporting many native architectures. But it seems much easier to make an assembler that only knows how to make call instructions and branches. So we could support compressed native on lots of architectures, and maybe uncompressed native only on some. If you want a quick way to do compressed native with reasonable register allocation, GNU libjit might work. I used it a couple years ago for a JIT project that we never fully implemented. I chose it over GNU Lightning specifically because it did register allocation. It implements a full assembler, not just calls, which could also be nice later. Noah On Sat, Nov 10, 2012 at 5:06 PM, Stefan Israelsson Tampe < stefan.ita...@gmail.com> wrote: > I would like to continue the discussion about native code. > > Some facts are, > For example, consider this > (define (f x) (let loop ((s 0) (i 0)) (if (eq? i x) s (loop (+ s i) (+ i > 1) > > The timings for (f 1) ~ (f 100M) is > > 1) current vm : 2.93s > 2) rtl : 1.67s > 3) compressed native : 1.15s > 4) uncompressed native : 0.54s > > sbcl = compressed nativ + better register allocations (normal optimization > level) : 0.68s > > To note is that for this example the call overhead is close to 5ns per > iteration and meaning that > if we combined 4 with better register handling the potential is to get > this loop to run at 0.2s which means > that the loop has the potential of running 500M iterations in one second > without sacrifying safety and not > have a extraterestial code analyzer. Also to note is that the native code > for the compressed native is smaller then the > rtl code by some factor and if we could make use of registers in a better > way we would end up with even less overhead. > > To note is that compressed native is a very simple mechanism to gain some > speed and also improve on memory > usage in the instruction flow, Also the assembler is very simplistic and > it would not be to much hassle to port a new > instruction format to that environment. Also it's probably possible to > handle the complexity of the code in pure C > for the stubs and by compiling them in a special way make sure they output > a format that can be combined > with the meta information in special registers needed to make the > execution of the compiled scheme effective. > > This study also shows that there is a clear benefit to be able to use the > computers registers, and I think this is the way > you would like the system to behave in the end. sbcl does this rather > nicely and we could look at their way of doing it. > > So, the main question now to you is how to implement the register > allocations? Basic principles of register allocation can be gotten out from > the internet, I'm assure of, but the problem is how to handle the > interaction with the helper stubs. That is > something i'm not sure of yet. > > A simple solution would be to assume that the native code have a set of > available registers r1,...,ri and then force the > compilation of the stubs to treat the just like the registers bp, sp, and > bx. I'm sure that this is possible to configure in gcc. > > So the task for me right now is to find out more how to do this, if you > have any pointers or ideas, please help out. > > Cheers > Stefan > > > > > > > On Sat, Nov 10, 2012 at 3:41 PM, Stefan Israelsson Tampe < > stefan.ita...@gmail.com> wrote: > >> Hi all, >> >> After talking with Mark Weaver about his view on native code, I have been >> pondering how to best model our needs. >> >> I do have a framework now that translates almost all of the rtl vm >> directly to native code and it do shows a speed increase of say 4x compared >> to runing a rtl VM. I can also generate rtl code all the way from guile >> scheme right now so It's pretty easy to generate test cases. The problem >> that Mark point out to is that we need to take care to not blow the >> instructuction cache. This is not seen in these simple examples but we need >> larger code bases to test out what is actually true. What we can note >> though is that I expect the size of the code to blow up with a factor of >> around 10 compared to the instruction feed in the rtl code. >> >> One interesting fact is that SBCL does fairly well by basically using the >> native instruction as the instruction flow to it's VM. For example if it >> can deduce that a + operation works with fixnums it simply compiles that as >> a function call to a general + routine e.g. it will do a long jump to the + >> routine, do the plus, and longjump back essentially dispatching general >> inst
Re: [PATCH]
SRFI-9 seems to be part of R7RS anyway (assuming there are no sudden last-minute changes), so I'd say we'll definitely want to do something like that soon. Noah On Sat, Nov 10, 2012 at 10:30 PM, Ian Price wrote: > > The patch looks fine to me (it had better be, since it is already pushed > :P), however I wonder if it wouldn't be prudent to look to moving > define-record-type into guile proper, and making (use-modules (srfi > srfi-9)) effectively a no-op. > > -- > Ian Price -- shift-reset.com > > "Programming is like pinball. The reward for doing it well is > the opportunity to do it again" - from "The Wizardy Compiled" > >
Re: Adding to the end of the load path
Hello, This is coming late in the discussion, but I'd like to suggest a somewhat different approach. I hope this is helpful. It seems to me that in the end, the module-lookup system may need to be more complex than having regular and suffix lookup paths. For instance, one of the big concerns here was reducing the number of stat() calls. What if we know that some load directories only contain certain modules? We might want a way for the user to say "all the (foo ...) modules live in ~/foo, but you don't have to look for any other modules there". Or what if I want to use a backup version of a module that's also included in the regular Guile distribution, because I haven't ported my code to a new version yet (yes, I should use module versions, but I don't)? There might be more complicated scenarios too. Given that the module-lookup system is fundamentally complicated, I'm going to suggest that we *don't* try to make it all configurable by environment variables. If people want full control of lookups, they can write a site-wide Guile init file or a personal ~/.guile. The regular load-path would still be part of the system, and that would be configurable by an environment variable, so legacy setups would continue to work. However, I'd be happy saying that if you wanted to access all of the functionality, you need to write Scheme code. Let's start by adding Scheme interfaces to the functionality we want, and maybe not worry about environment variables if they're complicated. What do you think? Noah On Thu, Nov 15, 2012 at 5:44 PM, Mark H Weaver wrote: > Hi Andreas, > > Andreas Rottmann writes: > > > l...@gnu.org (Ludovic Courtès) writes: > > > >> I pretty much like Mark’s suggestion of using ‘...’ as a special marker, > >> even though that’s a valid file name. > >> > > Well, there's a workaround -- specifying "./..." as an "escape sequence" > > for "..." if you really need to have a three-dot relative directory in > > the path. > > > >> How would that work for you? > >> > > I would like the approach using separate _SUFFIX variables better, as it > > doesn't have this special case. > > As I wrote earlier, I certainly agree that the _SUFFIX approach is > cleaner. Unfortunately, we need a solution that will work nicely with > earlier versions of Guile. > > > While I don't think the special case > > will be a problem for a user setting the environment variables > > themselves, if you want to set them programatically, you now have to > > consider treat "..." specially, escaping it like mentioned above, to be > > general. > > Note that PATH-style variables are already not general, because they > provide no way to include filenames containing ':' (a colon). > > In general, it's best to avoid setting GUILE_LOAD_PATH programmatically, > because it will affect more than just the instance of Guile you > intended; it will also affect any subprocesses that use Guile. It's > better to use -L which is fully general without any special cases, or to > modify %load-path within the program itself. > > > However, I can live with that, but maybe we can have it both > > ways: > > > > - Add the _SUFFIX environment variables, making it clear in the docs > > that they are supported only from Guile 2.0.7 onward. > > Yes, I agree this is a good idea. > > > - Additonally, add "..." as a special marker, but mention it is just > > provided to support Guile < 2.0.7, and should not be used in code that > > needs to depend on Guile 2.0.7 or newer for other reasons > > (e.g. reliance on another added feature or significant bugfix). > > Again, these environment variables are not specific to any particular > piece of code. They are usually associated with an entire user account. > > > I'm not sure how the deprecation strategy is employed exactly, but we > > could mark the "..." feature as deprecated right away, or at least in > > master, and remove it in 2.2 or 2.4. > > I don't think we can mark it deprecated until versions of Guile older > than 2.0.7 have become very rare, which won't be until at least 2017 > (due to Ubuntu 12.04 LTS), and then it will need to be deprecated for a > couple more years before we can get rid of it entirely. Therefore, I > think it's premature to emphasize the transient nature of the "..." > marker. Like it or not, we'll probably be stuck with it for 7 or 8 > years. > > Does that make sense? > > Regards, > Mark > >
Re: Adding to the end of the load path
And replying to myself, I remembered something else that had been in the back of my mind: a while ago on this list, there was discussion of bundling some modules with libguile, perhaps as static C arrays. This could make it easier to embed Guile, as a program could embed all the modules it needs and not have to worry about load-paths. However, this is another extension to the module-lookup system which really cannot be well-specified by environment variables. This is more evidence for my theory that module-lookup is fundamentally too complicated to be conveniently represented by environment variables. Of course, I would be very happy to be proven wrong, Noah On Thu, Nov 15, 2012 at 7:10 PM, Noah Lavine wrote: > Hello, > > This is coming late in the discussion, but I'd like to suggest a somewhat > different approach. I hope this is helpful. > > It seems to me that in the end, the module-lookup system may need to be > more complex than having regular and suffix lookup paths. For instance, one > of the big concerns here was reducing the number of stat() calls. What if > we know that some load directories only contain certain modules? We might > want a way for the user to say "all the (foo ...) modules live in ~/foo, > but you don't have to look for any other modules there". Or what if I want > to use a backup version of a module that's also included in the regular > Guile distribution, because I haven't ported my code to a new version yet > (yes, I should use module versions, but I don't)? There might be more > complicated scenarios too. > > Given that the module-lookup system is fundamentally complicated, I'm > going to suggest that we *don't* try to make it all configurable by > environment variables. If people want full control of lookups, they can > write a site-wide Guile init file or a personal ~/.guile. The regular > load-path would still be part of the system, and that would be configurable > by an environment variable, so legacy setups would continue to work. > However, I'd be happy saying that if you wanted to access all of the > functionality, you need to write Scheme code. Let's start by adding Scheme > interfaces to the functionality we want, and maybe not worry about > environment variables if they're complicated. > > What do you think? > Noah > > > > On Thu, Nov 15, 2012 at 5:44 PM, Mark H Weaver wrote: > >> Hi Andreas, >> >> Andreas Rottmann writes: >> >> > l...@gnu.org (Ludovic Courtès) writes: >> > >> >> I pretty much like Mark’s suggestion of using ‘...’ as a special >> marker, >> >> even though that’s a valid file name. >> >> >> > Well, there's a workaround -- specifying "./..." as an "escape sequence" >> > for "..." if you really need to have a three-dot relative directory in >> > the path. >> > >> >> How would that work for you? >> >> >> > I would like the approach using separate _SUFFIX variables better, as it >> > doesn't have this special case. >> >> As I wrote earlier, I certainly agree that the _SUFFIX approach is >> cleaner. Unfortunately, we need a solution that will work nicely with >> earlier versions of Guile. >> >> > While I don't think the special case >> > will be a problem for a user setting the environment variables >> > themselves, if you want to set them programatically, you now have to >> > consider treat "..." specially, escaping it like mentioned above, to be >> > general. >> >> Note that PATH-style variables are already not general, because they >> provide no way to include filenames containing ':' (a colon). >> >> In general, it's best to avoid setting GUILE_LOAD_PATH programmatically, >> because it will affect more than just the instance of Guile you >> intended; it will also affect any subprocesses that use Guile. It's >> better to use -L which is fully general without any special cases, or to >> modify %load-path within the program itself. >> >> > However, I can live with that, but maybe we can have it both >> > ways: >> > >> > - Add the _SUFFIX environment variables, making it clear in the docs >> > that they are supported only from Guile 2.0.7 onward. >> >> Yes, I agree this is a good idea. >> >> > - Additonally, add "..." as a special marker, but mention it is just >> > provided to support Guile < 2.0.7, and should not be used in code that >> > needs to depend on Guile 2.0.7 or newer for other reasons >> >
Re: Adding to the end of the load path
On Fri, Nov 16, 2012 at 1:52 PM, Mark H Weaver wrote: > Hi Noah, > Hello, > In general, I think the idea of requiring people to write scheme code to > manipulate %load-path (and other settings) is a fine approach. Maybe > you're right that this is better than adding a bunch of new environment > variables. > However, neither init.scm nor ~/.guile is sufficient for this job. > init.scm is site-wide, and generally only editable by root, and ~/.guile > is only run by interactive REPL sessions. So to do as you suggest, we'd > need to add another user-specific file that is read when initializing > guile, even for non-interactive sessions. > Ah, I didn't realize that was an issue. I was modeling this on what Emacs does, but it doesn't run in batch mode very much. Three configuration files with different meanings seems like a lot, but there's no other way that preserves backwards compatibility. Maybe we could use the $XDG_CONFIG directory for it (not that I like their default value). > Also, note that this still doesn't solve our immediate problem regarding > Guildhall and SRFIs in a backward-compatible way, so we still need to > support the "..." marker for the next 7-8 years, unless someone has a > better suggestion. > I see your point, but leaving it like this doesn't fix anything that isn't already broken; is that right? If so, I think we could justify saying that people need to upgrade to get the new behavior, especially if the complexity of supporting old versions is high. Noah
Re: Program received signal SIGSEGV, Segmentation fault.
Hello, On Fri, Nov 16, 2012 at 6:32 PM, Bruce Korb wrote: > On 11/16/12 13:23, Mark H Weaver wrote: > >> Actually, it was scm_from_utf8_string, since GUILE_VERSION was 25 > > > > Okay, that's the problem. You told Guile that the C string was encoded > > in UTF-8, but actually it was encoded in Latin-1: > > OK, so I tried latin1, too. (replacing scm_from_utf3_string with > scm_from_latin1_string). That also does not work. It replaced the > 0xA9 character with '?'. I am no expert on character encodings, but we've seen errors like this before where it turned out that Guile was attempting to display the character on a terminal which didn't support it, and then the terminal converted it into '?'. Could there have been some change in how Guile displays strings that caused this error? Did it used to show a \-escape sequence? > What it all boils down to is that > I am looking for string handling functions that will handle the > NUL terminated list of bytes and keep its nose out of the contents > of the string. Period. Full stop. > Could you explain what you're trying to do a little more? If you're calling a function that looks at characters on a string object that doesn't contain valid characters, then it will fail. If you have a NUL-terminated list of bytes that contains only characters valid in some encoding, then the scm_from_*_string functions are supposed to wrap it. So do you intend to make a string object and then never look inside? Or are you going to roll your own string-handling starting from byte sequences? The rest of your email suggests not. Thanks, Noah
CPS and RTL
Hello, I've had several conversations on this list about using continuation-passing style in Guile. I recently decided to take the hint and implement it. I've pushed a new branch called wip-rtl-cps that I'd appreciate comments on (but I do not necessarily think that this is the branch that will be merged with master - it's still pretty rough). The branch contains a CPS data structure based on the paper "Compiling with Continuations, Continued", by Andrew Kennedy. It lives in module/language/cps.scm. The branch is based on the RTL branch, and there's a CPS->RTL compiler in module/language/cps/compile-rtl.scm. There are also tests for it in test-suite/tests/cps.test. Here are some notes: - It may have been a mistake to do a CPS->RTL compiler first. Even if we want to use RTL eventually, it probably would have been more sensible to do CPS->GLIL first, so the change would be independent of the switch to RTL. However, the code is already here - we can use it as-is, or change to GLIL, or something else. - There's no Tree-IL->CPS compiler yet, mostly because I'm confident that we can do it, so I figured we could always add it later. - There are three major language features missing from the compiler: top-level variables, closures, and anything to do with the dynamic environment. Those are all pretty major features - that's why I said this was rough. - Right now the register allocation pass operates directly on the CPS. I have been thinking about making some higher-order functions for traversing it, similar to tree-il-fold, but I haven't done it yet. Basically, I have a very rough compiler. I'd like to show it to people now, even though it's not fully formed, so that you can point out any issues, and so that we can see if this is something that could eventually wind up in mainline Guile. I'd rather fix problems now than later on. (And of course I would appreciate help from anyone else who wants to contribute.) Thanks, Noah
Re: Making libunistring optional
Hi Mark, I think you and Ludo may actually be thinking along similar lines. It seems like you're saying that you'd like one tarball that contains only Guile-specific code, and another one with Guile plus some dependencies plus maybe a nice build script. I don't have an opinion on whether to separate the libunistring code from the main Guile repo, but I really really like the idea of bundling dependencies with Guile, and giving the users the option of whether to statically link Guile with them or dynamically link with system libraries. It's obviously not ideal, but I think it's a great idea for making Guile more portable. And I think it's the least bad way to help with dependencies. Noah On Mon, Nov 19, 2012 at 7:32 PM, Mark H Weaver wrote: > Hi Ludovic, > > l...@gnu.org (Ludovic Courtès) writes: > > Here’s an attempt to “reduce the number of dependencies” of Guile. The > > approach, as suggested by Bruno Haible, uses Gnulib’s > > ‘libunistring-optional’ module, along with the 22 (!) unistring modules > > that provide the functionality we need. > > > > When libunistring is available, it is used as before; when it’s not, > > those modules are built and linked into libguile. > > > > I had become convinced that this is a good idea, and a simple way to > > make our user’s lives easier. Yet, this adds 184 source files to the > > tarball (would be more if we included tests), so I’d like to hear > > opinions. > > Yikes! It looks like this imports most of the libunistring source code > into Guile. > > I think this is a bad idea. > > If we simply want to save users the trouble of building and installing > our dependencies, how about creating a script to do the job for them, > analogous to jhbuild for Gnome? Or perhaps distribute a "bundle" that > includes Guile and some key dependencies together in a single tarball, > along with a top-level build system that handles the combined build? > > I'd really prefer to avoid importing large libraries like libunistring > directly into our source repository, or including them in our primary > tarballs. > > Regards, > Mark > >
Re: non-scheme scripts: proposed solutions and their pros/cons
As you say, the only real solution is to do more than one of these things. For instance, I think it's really important to be able to load modules written in other languages. However, this may be language-dependent to a certain extent, because some languages (Python) already have ways to define modules. In those cases we should stick with their conventions, and use our other methods for figuring out what language the file is in. However, if we're using Guile in one language and loading an executable in a different language, then we can't use a command-line argument or a different executable to signal it. The only choices left are heuristics and explicit markers. I think the only reasonable choice is both - use heuristics, and let the user supply a marker if the heuristics are wrong. (In the module case, one can imagine a heuristic based on having a language-specific load path for each language, which might be very effective.) But for using Guile as an interpreter for different languages, a command-line argument or argv[0] switch make a lot of sense. What do you think? Noah On Tue, Nov 20, 2012 at 7:15 AM, Ian Price wrote: > > As promised in the other thread, here is my list. This was really a > response to the even the earlier thread I started, which I > (unfortunately) didn't reply to at the time. > > First off, they important question "why do we need this?". Well, guile > is a multi-language vm in principle, even if Scheme is it's first and > foremost citizen. If guile is to be a full-fledged vm for these other > languages, they need all (or at least most of) the rights and privileges > scheme does. This includes the ability to be run as scripts. > > So, what are the solutions? Well, a few have cropped up, but I'm not > sure that individually any of them constitute a complete > solution. They are: a command-line argument, some marker in the > module, a different executable, and heuristics based on file > extension. > > I shall treat these in order, though I can't pretend I will be > comprehensive. Clarifications and additions welcome. > > > * a command line argument > This was the position I initially proposed, and so had some bias > towards. The idea is very simple, we add a --language extension, the > argument to which would be the language of the file(s) we are trying > to execute. > > ** Pros > - Existing source files in the language do not need to be modified to > use this. > - Requires (in theory) modifying only the command-line parsing > > ** Cons > - Has nothing to say about how e.g. scheme modules interact with elisp > modules. > - #! only gives you one argument, and not all languages are going to > support a \ type solution > > * a file marker > The idea here it to have some way of marking in a file which language > it is written in, with some token like #!javascript, or perhaps > something like #!language javascript > > ** Pros > - works well with current module system > e.g. say we have a file foo/bar.js and that had a #!javascript > marker, then I could (use-modules (foo bar)) and guile would notice > the marker, switch to javascript mode for it. > - we already do this for switching to curly-infix and r6rs reader > modes > > ** Cons > - requires modifying existing source code, I couldn't just import an > existing elisp (or whatever) file and use as is. > - the existing mechanism allows switching reader mode at an arbitrary > part of the file. Not a big con, for #!language, I would simply say > we disallow it in arbitrary places. However, people that extend > guile as a language (i.e. lilypond), might disagree and would want > to take advantage of this. > > * different executable > This was a position posed in response to mine, that I was initially > against, but am somewhat more open to now. The idea is that for every > language $foo, we have a script guile-$foo which invokes that > language. > Technically it need not be a different executable, but one whose > action depends on argv[0] and just performs the appropriate action, > but in many respects these can be treated the same. > > ** Pros > - Existing source files in the language do not need to be modified to > use this, they can change the existing languages symlink. > - We can handle common switches used by the language > - If you go for argv[0], may only require modifying the command-line > parsing. > > ** Cons > - proliferation of names, even if just symlinks > - has nothing to say about cross-language interoperability > > > * heuristics > This always gets proposed, and I never like it, but hey ho. The idea > is to have some simple way to guess what language a file is written > in, for example, with a mapping of file extensions to languages. > > ** Pros > Best case scenario, you do nothing. You type guile foo.js, and it just > works. > - same excutable name > - don't need to modify existing code > > ** Cons > *** guessing on file extension > - guile allows user defined extensions > Need to have a way
Re: Is there any thing to convert string encording?
In that case I believe you want to put the bits you're interested in in a
bytevector, and use utf8->string, utf16->string, or utf32->string.
Noah
On Fri, Nov 23, 2012 at 2:48 AM, nalaginrut wrote:
> In ruby, we can use force_encoding to convert the encoding of a string:
> "abc\u{}".force_encoding("UTF-8")
>
> What's the same thing in Guile?
>
> PS: I know Guile treat all strings with "UTF-8". I don't mean to encode
> UTF-8 string. I need something like "force_encoding".
>
>
>
Re: More RTL Tests
I would definitely like to see something like that appear in the RTL branch. Otherwise it might be very difficult to write RTL code. Rather than sending me a patch, I'd like to know what Andy Wingo's plans were for accessing variables. I imagine he was thinking of something like this too, so the best thing would be to get that patch merged into the regular wip-rtl branch. Noah On Wed, Nov 28, 2012 at 2:13 PM, Stefan Israelsson Tampe < stefan.ita...@gmail.com> wrote: > Hi Noah, > > I have a fix to rtl.scm that add an instruction macro like this > > (toplevel dst sym) > > e.g. > > (toplevel 0 pk) > > to refere to pk in the current module I can send you a patch if you like, > it's very instructing for learning how to > program the rtl machine to your needs. And it also allows you to write rtl > assembler refering to toplevels in a nice way > > /Stefan > > > > On Sun, Oct 14, 2012 at 10:57 PM, Stefan Israelsson Tampe < > stefan.ita...@gmail.com> wrote: > >> >> >> On Sun, Oct 14, 2012 at 9:59 PM, Noah Lavine wrote: >> >>> Hello, >>> >>> I have been working on understanding RTL, and I wrote the following >>> tests. They're mostly to illustrate for myself how calling works in >>> RTL, but they also serve to test it. Any objections if I commit them >>> as part of rtl.test? >>> >>> (with-test-prefix "call" >>> (assert-equal 42 >>> (let ((call ;; (lambda (x) (x)) >>>(assemble-program >>> '((begin-program call) >>> (assert-nargs-ee/locals 1 0) >>> (call 1 0 ()) >>> (return 1) ;; MVRA from call >>> (return 1) ;; RA from call >>> (call (lambda () 42 >>> >>> (assert-equal 6 >>> (let ((call-with-3 ;; (lambda (x) (x 3)) >>>(assemble-program >>> '((begin-program call-with-3) >>> (assert-nargs-ee/locals 1 1) >>> (load-constant 1 3) >>> (call 2 0 (1)) >>> (return 2) ;; MVRA from call >>> (return 2) ;; RA from call >>> (call-with-3 (lambda (x) (* x 2)) >>> >>> (with-test-prefix "tail-call" >>> (assert-equal 3 >>> (let ((call ;; (lambda (x) (x)) >>>(assemble-program >>> '((begin-program call) >>> (assert-nargs-ee/locals 1 0) >>> (tail-call 0 0) >>> (call (lambda () 3 >>> >>> (assert-equal 6 >>> (let ((call-with-3 ;; (lambda (x) (x 3)) >>>(assemble-program >>> '((begin-program call-with-3) >>> (assert-nargs-ee/locals 1 1) >>> (mov 1 0) ;; R1 <- R0 >>> (load-constant 0 3) ;; R0 <- 3 >>> (tail-call 1 1) >>> (call-with-3 (lambda (x) (* x 2)) >>> >>> >> This look good to me >> >> My next goal is to learn to reference top-level variables. Could >>> anyone tell me how that works, to make it easier? >>> >> >> The general way, use a symbol in loc a and a module in loc b and then >> issue >> (resolve c b a) >> (box-ref c c) >> >> To put the symbol into c. The simplest way to check this is to pass b and >> a as arguments >> to the funciton, you may set the toplevel by using box-set! >> >> For reference inside functions you may use >> (toplevel-ref dst var_label mod_label sym_label) >> >> This is a bit complicated and I havenot found that all the infrastructure >> is in place to handle this >> but basically one should need to prepare the code like this if I'm not >> missing anything >> >> (program f) >> (toplevel-ref 1 var mod sym) >> ... >> (label var) >> (store-slot SCM_BOOL_F) >> (label mod) >> (store-slot module) >> (label sym) >> (stire-slot 'symbol) >> >> >> where the stire-slot mechanism is not yet supported but should mean that >> aligned to SCM it stores >> the needed references. >> >> This is what I guess it all means, correct me if i'm wrong! >> >> /Stefan >> >> >> >> >> >> >
Re: Fix reader options for R6RS `get-datum'
Hello, On Mon, Dec 17, 2012 at 2:05 PM, Andreas Rottmann wrote: > I think this is my assumption that you seem to disagree on: by using the > binding of `read' from `(rnrs io simple)', instead of the one provided > by Guile's core, the writer of the code using that binding has declared > that he wishes `read' to adhere to R6RS. I actually disagree with this assumption too, but let me explain why. I think there are some situations where it is right and some where it is wrong. A program will want strict R6RS semantics (as Rotty says) if it is, for instance, reading from a data file in a strictly-defined format. In that case, it might have written its parser assuming an R6RS-compliant `read' function, and it could fail if it gets an extended `read'. However, the R6RS reader would be wrong for a program that wants to, say, read a file of Scheme code as S-expressions and manipulate it. The program would have to be written using R6RS libraries to be portable, but the right thing for it to do in any particular Scheme implementation is to read exactly the same syntax as that implementation. (This is a bit of a contrived example, but I hope you can see my point that some programs don't want strict R6RS semantics as much as they want to integrate with whatever extensions their host implementation has made to `read', and that what they really want is a portable way to access the built-in `read'.) If you believe both of these examples, then programs need to be able to choose their own meaning. Perhaps the solution is a procedure like `strict-r6rs-read-options' for programs that want that, or alternatively `native-read-options' for programs that don't. Regards, Noah
Re: A proper tail version of with-fluids
I think with-fluids could at least be semi-tail-recursive. If you imagine a normal non-tail-recursive implementation, you might get to a point where your continuation is going to set a fluid back to a value, and then the *next* continuation is going to set that fluid back to *another* value. Since these are only set!s and have no side-effects, you could somehow notice that and not do the first set!. So your continuation only needs to have one set! for each fluid, no matter how many times that fluid is changed by with-fluids. But that seems tricky to implement. I haven't looked at how Stefan did it, but I'm impressed. Noah On Thu, Jan 3, 2013 at 6:36 PM, Ludovic Courtès wrote: > Hi Stefan, > > Stefan Israelsson Tampe skribis: > > > (define f (lambda (n) (if (= n 0) (fluid-ref a) (with-fluids ((a n)) (f > (- > > n 1)) > > > > with the modified VM: > > scheme@(guile-user)> (f 1000) > > $2 = 1 > > > > with the old VM, it craches. It works! > > Hmm, I can’t see how ‘with-fluids’ or ‘parameterize’ could be > tail-recursive given that it uses ‘dynamic-wind’. Am I missing > something? > > Ludo’. > > >
Re: How to debug the scheme/guile interpreter?
There's a script in the 'meta' directory of the Guile sources called 'gdb-uninstalled-guile'. I believe it sets up whatever environment variables need setting and then calls gdb for you. I've used it to debug Guile in the past. Thanks a lot for tracking down this bug Noah Lavine On Wed, Jan 9, 2013 at 10:39 PM, Nala Ginrut wrote: > On Wed, 2013-01-09 at 21:59 -0500, Peter Teeson wrote: > > Greetings: > > Mac Pro 4,1 - MacOS 10.7.5 > > I filed bug <http://bugs.gnu.org/13342> and Ludo' invited me to > investigate which I've been doing, > > including reading the other bugs on this problem > http://bugs.gnu.org/10015, http://bugs.gnu.org/10681. > > > > My conclusion at this time, after a number of experiments, is that the > issue is not in the llvm compiler but somewhere in the interpreter. > > I am not familiar with the scheme or guile languages but have been a > (now retired) developer on Mac for 25 years. > > Before that 14 years on IBM mainframes as a systems programmer working > on an APL interpreter. > > > > I would like to try and track down this old issue but don't know how to > debug the scheme/guile interpreter. > > Looked in the guile manual but didn't find anything that helped me. > > > > Please give me some pointers on how to proceed. > > > > TIA > > > > Peter > > hi Peter! > I know nothing about that bug. But if you want to use GDB to debug with > Guile code, you may need 'gdbinit' which is in the toplevel guile src > directory. > Say, 'gdb -x gdbinit', then you can use 'gwrite' to print out the SCM > type vars's value for your debugging. > > > > > >
Re: [PATCH] Colorized REPL
Hello, On Fri, Jan 11, 2013 at 9:33 AM, Ludovic Courtès wrote: > >> Nala Ginrut skribis: > > record-type in r6rs is more convenient I think. > > That’s not the question. ;-) It doesn’t justify pulling in all of R6RS. > This is just a small part of a much larger review, but it should be possible to import (rnrs records syntactic), right? Or maybe even just (rnrs records) (I don't know much about the R6RS library system). I see that (rnrs records syntactic) pulls in a bunch of other R6RS stuff, which perhaps you won't like. But I think there is something wrong with this idea - the point of having libraries is that we can use them. If we can't use anything from R6RS because we don't want to pull it in, then why did we implement it? I know that supporting other peoples' r6rs programs is also a reason, but I think that Guile should be able to use the libraries it itself bundles. And in theory, using RnRS libraries is nice because it promotes portable Scheme code. (I do agree that R6RS is a sort of weird case, because a lot of it is different names for features that Guile already has in another form. I'm not sure if that changes this or not.) Thanks, Noah
Re: Guile Lua
I sent an email about that, but it was only an idea. I thought it would be nice if we could work with the Clisp people. However, I can see some barriers to actually doing that, and I don't intend to work on it any time soon. Noah On Sat, Jan 12, 2013 at 3:43 AM, Nala Ginrut wrote: > On Wed, 2012-11-21 at 16:51 +0100, Stefan Israelsson Tampe wrote: > > Hi, > > > In terms of strategy, I think Guile’s focus should remain primarily > > on > > > Scheme variants, and ELisp. Other language front-ends are of course > > > welcome, but we must keep an eye on what the demand is. > > > > What about common lisp is scheme a lisp or is CL a scheme :-) > > > > IIRC， someone raised the topic that emerge Clisp into Guile in 2011, > but what's the status now? > > > Anyway to support CL I would think that we need to support placing > > properties > > on symbols, e,g. currently a symbol slot is a variable, but to > > effectively support CL I would go for > > /Stefan > > > > > > > > Den 21 nov 2012 14:26 skrev "Ludovic Courtès" : > > Hi! > > > > nalaginrut skribis: > > > > > I switch to lua branch then compiled it and try, seems some > > bugs there, > > > it can't run successfully: > > > ---cut > > > scheme@(guile-user)> ,L lua > > > Happy hacking with Lua! To switch back, type `,L scheme'. > > > lua@(guile-user)> x=1 > > > > Maybe you need a semicolon here? > > > > > And I checked the code, it doen't use Guile inner LALR > > parser. > > > Anybody point me out what is the suggested parser > > implementation? > > > > (system base lalr). > > > > > And is there anyone ever evaluated the efficiency about the > > non-scheme > > > language implemented within Guile? > > > > I don’t think so. Only the Scheme and Emacs Lisp front-end > > are > > reasonably mature, anyway. > > > > > Anyway, this wouldn't be a big problem, since Guile could be > > the > > > future dynamic language compiler collection, it could be > > optimized > > > later. > > > > FWIW, I don’t quite buy the “dynamic language compiler > > collection”. > > Others tried this before (Parrot), with some success in terms > > of > > supported languages, but not much beyond that. > > > > In terms of strategy, I think Guile’s focus should remain > > primarily on > > Scheme variants, and ELisp. Other language front-ends are of > > course > > welcome, but we must keep an eye on what the demand is. > > > > Thanks, > > Ludo’. > > > > > >
Re: [PATCH] Colorized REPL
Hello, On Fri, Jan 11, 2013 at 6:26 PM, Ludovic Courtès wrote: > Hello, > > > I know that supporting other peoples' r6rs programs is also a reason, > but I > > think that Guile should be able to use the libraries it itself > > bundles. > > I agree in general, yes. But when the run-time footprint can be reduced > at little cost, it seems nice to do it. > > > And in theory, using RnRS libraries is nice because it promotes > > portable Scheme code. (I do agree that R6RS is a sort of weird case, > > because a lot of it is different names for features that Guile already > > has in another form. I'm not sure if that changes this or not.) > > Exactly: the problem I have with R6RS is that it basically re-implements > several SRFIs or APIs otherwise available in Guile, sometimes just for > dubious aesthetic reasons–e.g., SRFI-1, SRFI-9, SRFI-3[45]. > Traditionally ice-9 modules have not used them. > Yes, I agree with everything you said here. I'm torn, because I think that in general having more portable Scheme code is good for everyone, and the RnRS standards are the best way to do that, so maybe we should just accept that the most recent 1 or 2 standards will always be loaded. But on the other hand, that doesn't mean that this particular module needs to use them. Noah
Re: Loading a module before and after adding a load path
Hello, I see what you are trying to do, and I agree that it should be possible. I have set up the same situation as you: trying to load a module, failing, adding it to my path, trying again, and failing again. Here's what I've figured out: - resolve-module fails on the module, when I think it should succeed. that's because - try-autoload-module fails on the module, when again I think it should succeed. and that's because - autoload-done-or-in-progress? returns #t for the module, when I think it should return #f. specifically, the module is marked as done, and therefore try-module-autoload won't look for it any more. - the module was marked as autoloaded because that's what try-module-autoload does: it *always* marks its module as autoloaded, even if it wasn't found. As a workaround, remove your module from the list "autoloads-done". (actually, there will be a pair in that list, where the cdr of the pair is the name of the module as a string. remove that.) In the long term, Andy and Ludo, what is the right way to fix this? Should try-module-autoload not mark unfound modules as autoloaded? Or maybe there should be a special function that says "retry this module, starting from the beginning". Maybe it could even be part of reload-module. Best, Noah On Fri, Jan 18, 2013 at 11:15 PM, Diogo F. S. Ramos wrote: > Nala Ginrut writes: > > > Why you need to load the module without giving its path? > > Sorry, I should have added some more context. > > While I'm working with the REPL, sometimes I forget to add the path to a > module before trying to use it. It's a mistake. So, after trying to load > the module with `use-modules', and failing, I remember that I have to > add the path to the module. So I add the path to the module with > `add-to-load-path' and I try again to evaluate > `use-modules'. Unfortunately it does not load it and I'm stuck. To fix > it, I end up restarting the REPL and adding the path _before_ trying to > load it, as it's the right way to do it. > > Here is an example with my typical mistake: > > $ cat /tmp/foo/bar.scm > (define-module (foo bar)) > > (define-public (hello) > (display "hello, world") > (newline)) > $ guile > GNU Guile 2.0.5-deb+1-1 > Copyright (C) 1995-2012 Free Software Foundation, Inc. > > Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'. > This program is free software, and you are welcome to redistribute it > under certain conditions; type `,show c' for details. > > Enter `,help' for help. > scheme@(guile-user)> (use-modules (foo bar)) > While compiling expression: > ERROR: no code for module (foo bar) > scheme@(guile-user)> (add-to-load-path "/tmp") > scheme@(guile-user)> (use-modules (foo bar)) > While compiling expression: > ERROR: no code for module (foo bar) > scheme@(guile-user)> (hello) > ;;; :4:0: warning: possibly unbound variable `hello' > :4:0: In procedure #:4:0 > ()>: > :4:0: In procedure module-lookup: Unbound variable: hello > > Entering a new prompt. Type `,bt' for a backtrace or `,q' to continue. > scheme@(guile-user) [1]> > > The question is: Is there a way to recover from this mistake in my live > REPL session? > >
Re: CPS and RTL
Hello Andy and Mark, Thanks for the review! There has actually been more progress since I pushed that branch. I hit a point in the CPS->RTL stuff where I had trouble because I didn't know how to do things (like mutable variables) in RTL. So I've actually ported the compiler to GLIL in a branch on my computer. I also have a working Tree-IL->CPS compiler for some of Tree-IL (it's not done yet). I thought that might be a better way forward because CPS and RTL are, to a certain extent, separate ideas. I'll push my wip-cps branch, which contains a Tree-IL->CPS compiler and a CPS->GLIL compiler. What I'm working on now is actually how to represent mutable variables in CPS. I think having explicit environment structures would be nice (and fit with Kennedy's paper), but I haven't figured out the details yet. I realize it might be confusing to start with CPS->RTL, then switch to CPS->GLIL, then switch back later when the RTL branch is ready. If you'd rather do it that way, we can skip the CPS->GLIL phase. Some thoughts: * Yes, passes might be good. I had thought of writing some generic control-flow operators like 'compute-fixpoint' and then writing other things in terms of those. * Using tree-il first sounds good to me. * I also think that CPS should have some construct which says 'do these things in any order'. I haven't put one in yet, mostly because the compiler wouldn't take advantage of it anyway. Noah On Thu, Jan 24, 2013 at 5:38 AM, Nala Ginrut wrote: > Are you guys going to use CSP to implement SSA for AOT? > 在 2013-1-24 PM6:36，"Andy Wingo" 写道： > > Hi! >> >> On Thu 24 Jan 2013 10:28, Mark H Weaver writes: >> >> > The problem is that CPS fixes the order in which everything is >> > evaluated, such as the order in which procedure arguments are >> > evaluated, the order in which 'let' or 'letrec' initializers are >> > evaluated, etc. The fact that these orders are unspecified in the >> > direct-style gives the compiler freedom to choose an order that >> > generates the best code, and apparently this freedom can often result >> > in significant gains. Such ordering decisions must be made before the >> > conversion to CPS. >> >> Agreed with the sentiment; however, two points: >> >> * we can have a CPS with let / letrec / * operators that bind a number >> of values in unspecified order, and have a pass later that fixes >> their order. >> >> * code motion passes like CSE depend on effects analysis, and can >> often commute some operations >> >> Anyway, violent agreement! >> >> Cheers, >> >> Andy >> -- >> http://wingolog.org/ >> >>
Re: CPS and RTL
Hello, On Thu, Jan 24, 2013 at 10:12 AM, Andy Wingo wrote: > Hi! > > On Thu 24 Jan 2013 14:50, Noah Lavine writes: > > > Thanks for the review! There has actually been more progress since I > > pushed that branch. I hit a point in the CPS->RTL stuff where I had > > trouble because I didn't know how to do things (like mutable variables) > > in RTL. So I've actually ported the compiler to GLIL in a branch on my > > computer. I also have a working Tree-IL->CPS compiler for some of > > Tree-IL (it's not done yet). > > > > I thought that might be a better way forward because CPS and RTL are, to > > a certain extent, separate ideas. > > Cool, please push so we can see. > Given the rest of your email, maybe I'll move the Tree-IL->CPS compiler back to wip-cps-rtl branch and push that. > Honestly I think RTL and CPS go together very well. CPS is all about > giving a name to everything, but that can be inefficient in a stack VM, > because referencing and updating named variables requires separate push > and pop instructions. RTL makes this easy and cheap. > Yes, I hadn't thought about that. RTL does make sense. > Regarding mutable variables: we probably still need to box them in > general because of call/cc. There are cases in which they can be > unboxed, but I think that store-to-load forwarding with DCE can probably > recover many of those cases. Dunno. I would box them as part of an > assignment conversion pass. > I think I was imagining about the same thing you're thinking. > > I realize it might be confusing to start with CPS->RTL, then switch to > > CPS->GLIL, then switch back later when the RTL branch is ready. If you'd > > rather do it that way, we can skip the CPS->GLIL phase. > > Personally I would prefer to target RTL. But that is a personal opinion > :) > I'm happy to! You've convinced me that it's better. I see that you just implemented toplevel-refs, too, so my problem is solved. Best, Noah
Re: rebased wip-rtl-cps
Thanks! Best, Noah On Wed, Jan 23, 2013 at 11:28 AM, Andy Wingo wrote: > Hi Noah, > > A brief note to let you know that I rebased wip-rtl-cps, as wip-rtl > itself was rebased. > > Cheers, > > Andy > -- > http://wingolog.org/ >
Re: CPS and RTL
The push is complete. If I'm not mistaken, you should now see a wip-rtl-cps branch with a Tree-IL->CPS compiler. The compiler is used in cps.test. Please let me know if I did anything wrong. Noah On Thu, Jan 24, 2013 at 11:03 AM, Noah Lavine wrote: > Hello, > > On Thu, Jan 24, 2013 at 10:12 AM, Andy Wingo wrote: > >> Hi! >> >> On Thu 24 Jan 2013 14:50, Noah Lavine writes: >> >> > Thanks for the review! There has actually been more progress since I >> > pushed that branch. I hit a point in the CPS->RTL stuff where I had >> > trouble because I didn't know how to do things (like mutable variables) >> > in RTL. So I've actually ported the compiler to GLIL in a branch on my >> > computer. I also have a working Tree-IL->CPS compiler for some of >> > Tree-IL (it's not done yet). >> > >> > I thought that might be a better way forward because CPS and RTL are, to >> > a certain extent, separate ideas. >> >> Cool, please push so we can see. >> > > Given the rest of your email, maybe I'll move the Tree-IL->CPS compiler > back to wip-cps-rtl branch and push that. > > >> Honestly I think RTL and CPS go together very well. CPS is all about >> giving a name to everything, but that can be inefficient in a stack VM, >> because referencing and updating named variables requires separate push >> and pop instructions. RTL makes this easy and cheap. >> > > Yes, I hadn't thought about that. RTL does make sense. > > >> Regarding mutable variables: we probably still need to box them in >> general because of call/cc. There are cases in which they can be >> unboxed, but I think that store-to-load forwarding with DCE can probably >> recover many of those cases. Dunno. I would box them as part of an >> assignment conversion pass. >> > > I think I was imagining about the same thing you're thinking. > > >> > I realize it might be confusing to start with CPS->RTL, then switch to >> > CPS->GLIL, then switch back later when the RTL branch is ready. If you'd >> > rather do it that way, we can skip the CPS->GLIL phase. >> >> Personally I would prefer to target RTL. But that is a personal opinion >> :) >> > > I'm happy to! You've convinced me that it's better. I see that you just > implemented toplevel-refs, too, so my problem is solved. > > Best, > Noah > >
Re: Guile API for foreign languages: proposing SCM scm_list_0(void)
I don't know much about language interfaces, but why not have these be
constants exported by libguile.so? Is there any reason for other languages
to have to make their own lists?
Thanks,
Noah
On Thu, Jan 31, 2013 at 6:45 AM, Andy Wingo wrote:
> On Thu 31 Jan 2013 12:27, Andy Wingo writes:
>
> > On Tue 22 Jan 2013 11:55, Andy Wingo writes:
> >
> >> SCM_BOOL_F
> >> SCM_BOOL_T
> >> SCM_ELISP_NIL
> >> SCM_EOF_VAL
> >> SCM_EOL
> >> SCM_UNBOUND
> >> SCM_UNDEFINED
> >> SCM_UNSPECIFIED
> >>
> > Instead, users should just keep a table of what the values of these
> > constants are for a given Guile major series. Defining them as
> > enumerated values doesn't help e.g. an Ada compiler. The particular
> > language should make this list in their own source code format, perhaps
> > generated by a small C program linked to libguile.
>
> Or by Scheme:
>
>(for-each
> (lambda (pair)
>(format #t "static const scm_t_bits my_~A = 0x~X;\n"
> (car pair) (object-address (cdr pair
> `(("false" . #f)
>("true" . #t)
>("nil" . #nil)
>("eof" . ,the-eof-object)
>("eol" . ())
>("unspecified" . ,*unspecified*)))
>
>static const scm_t_bits my_false = 0x4;
>static const scm_t_bits my_true = 0x404;
>static const scm_t_bits my_nil = 0x104;
>static const scm_t_bits my_eof = 0xa04;
>static const scm_t_bits my_eol = 0x304;
>static const scm_t_bits my_unspecified = 0x804;
>
> Unbound and undefined are a little trickier.
>
> Andy
> --
> http://wingolog.org/
>
>
Re: [PATCH] Do not scan for coding declarations in open-file
Hello, On Thu, Jan 31, 2013 at 5:00 PM, Ludovic Courtès wrote: > There are several issues IMO. First, some are subrs, so handling > keyword arguments is going to be painful. Second, keyword arguments are > inelegant IMO compared to: > > (set-port-encoding! port (file-encoding port)) > I don't have much experience in this area, but what about making a Scheme binding for scm_i_scan_for_encoding, and then doing something like (define (open-file filename mode #:encoding enc) (let ((port (open-file filename mode))) (set-port-encoding! port (if (eq? enc 'guess-encoding) (scan-for-encoding port) enc Of course, that doesn't address your second point. Best, Noah
Re: About Guile crypto support
Hello, I was just thinking about this, and I was wondering, can you hash an arbitrary Guile object? And if so, what do you hash? (I mean, algorithms like SHA-1 are defined on sequences of bits, as I understand it. So what collection of bits do you hash?) And is the hash recursive? (I.e. is it an equal?-hash, an eqv?-hash, or an eq?-hash.) If I understand the conversation correctly, the answer is yes, and that you hash the bit representation that Guile uses internally, and it is an equal?-hash. Is that accurate? Thanks, Noah On Mon, Feb 4, 2013 at 9:43 PM, Nala Ginrut wrote: > On Tue, 2013-02-05 at 00:03 +0100, Ludovic Courtès wrote: > > Nala Ginrut skribis: > > > > > As mentioned in another thread about digest algorithm support in Guile, > > > my plan is use part of implementation of libgcrypt and make a wrapper, > > > then put into libguile. > > > > We probably don’t want Guile to depend on libgcrypt. > > > > No, I didn't mean to use libgcrypt directly, I just suggested reuse part > of libgcrypt code(only the common digest algorithm) and make wrapper, > then put the C code in libguile. > > > So, instead, I’d suggest choosing the best of the 10 gcrypt FFI bindings > > already mentioned ;-), and putting it in the guildhall. > > > > If you want to go further, you (or its authors) could submit it for > > inclusion in libgcrypt proper. > > > > The gcrypt-guile project is doing so, I'll help it if I can. > But my original thought is orthogonal with gcrypt-guile, just put some > common digest algorithm in libguile rather than a full-stack crypto-lib. > > My suggest opposed by many guys, since they don't think md5/sha are very > common things. ;-) > But I'm dealing with server & web framework development, so I need > digest so much, and maybe it's no so common. > I'll let this topic alone, till others found they have same > requirements, or just forget about it. ;-D > > > Thanks, > > Ludo’. > > > > > > > >
RTL Cache Cells
Hello, The RTL branch now has infrastructure for toplevel and module refs, which is very exciting. Unfortunately, I can't figure out how to use it. :-( In particular, I've been looking at module/system/vm/rtl.scm, and there are two things I don't understand about cache cells. 1. How do I choose the scope of a cache cell? It seems like making it the name of the lambda I'm in would be fine, but that's not what the examples in rtl.test do. 2. How do different toplevel-refs share cache cells? A comment says that they will, but the emit-cache-cell code doesn't seem to do any checking. Does that still need to be implemented? Oh, and as long as I'm at it, 3. I see that the 'toplevel-ref' instruction in vm-engine.c expects a variable object as its cache, but the 'cached-toplevel-ref' instruction *appears* to be passing it a cache cell. I say "appears" because I see that in intern-constant, cache cells are special-case objects that aren't actually cached. So is a cache cell only a compile-time entity? How do they work, anyway? Thanks in advance, Noah
Re: [ANN] guile-csv 0.0.1 released!
This has been said before, but I think the most important thing is for people who are new to Guile to be able to see a list of "mature, well-maintained" libraries (whatever that means), and tell the difference between those and poorly-maintained or bitrotted libraries. It would also be nice to have a place to store unmaintained code, because people can still use it, but they should be clearly separate. Noah On Fri, Feb 8, 2013 at 11:26 AM, Ludovic Courtès wrote: > Mark H Weaver skribis: > > > I haven't yet looked carefully at this code or its API, so this is no > > judgement on you, but in general, I don't think we should follow the > > model of "Hey, here's the first release of a library I just hacked up. > > Please add it to Guildhall now." That's how we ended up with an ice-9 > > directory that's full of bitrotted implementations of half-baked APIs. > > > > I'd much rather follow the example of Shiro Kawai, who is very cautious > > to experiment with new APIs at length before adding them to Gauche, and > > the result is IMO a beautiful and consistent set of APIs. > > > > Maybe we can find a good compromise position between these two extremes. > > > > What do other people think? > > Well, there can be several repositories. Once there’s one at gnu.org, > it could have a lightweight review process, and host reasonably mature > code (the barrier to entry should be lower than that of Guile proper > IMO, but not too demanding.) > > People are free to setup additional repositories with their own > policies. > > Ludo’. > > >
Re: About Guile crypto support
Mark, I agree with everything you said about dependencies. I think the real solution is something like what you said - sharing code, but bundling. One way to push that farther would be to distribute tarballs that include the complete source of some libraries, and somehow making a combined build system that configures and builds all of them. I would nominate libgc for this, since we keep hitting trouble with new or old libgc versions. Noah On Sat, Feb 9, 2013 at 12:50 PM, Mark H Weaver wrote: > Hi Andy, > > Andy Wingo writes: > > > On Sat 09 Feb 2013 16:12, l...@gnu.org (Ludovic Courtès) writes: > > > >> An issue with the FFI is distros where .la and .so files are only > >> available in the -dev package, because then ‘dynamic-link’ won’t work > >> unless that -dev package is installed (as recently discussed on > >> guile-user.) > > > > I have the feeling that we should implement our own dynamic-link > > function without libltdl. It would eliminate a dependency and allow us > > to use other search path rules, like ones that could deal with this > > case. I think the situation would actually be better on other > > architectures because we wouldn't have to deal with bugs like this one: > > > > http://thread.gmane.org/gmane.lisp.guile.bugs/5269 > > The problems we're having with libltdl are likely affecting many other > projects. Wouldn't it be better to fix these problems in libltdl, to > the benefit of all its users, than for each of its users to duplicate > its functionality within their own projects? > > More generally, I'm concerned with the direction we are being pressured > into by those who complain about the number of dependencies. We ought > to look for better solutions than duplicating library functionality > within Guile's own source code. Imagine if every program did that. > That way lies madness. > > IMO, we ought to look for better solutions for those who complain about > dependencies. One idea is to provide precompiled versions of Guile for > the major platforms (i.e. MinGW, MacOS and possibly also GNU/Linux) with > all dependencies included, for use by libguile-based projects that wish > to provide precompiled bundles for their users. It might also make > sense to provide something along the lines of jhbuild to make the build > job easier for those who want more flexibility. > > What do you think? > > Mark > >
CPS Update
Hello, The wip-rtl-cps branch has been rebased again (on top of wip-rtl). It now includes support for toplevel references and sets, thanks mostly to Andy's work on supporting them in RTL. (Although you shouldn't kick that part of it too hard just yet; I think I know where it will break.) Other highlights include using the top-level variable support to evaluate "(+ 3 4)" correctly. Overall, I think it's coming along well. The parts of Tree-IL that are left to implement are local mutable variables (should be easy after toplevel ones), module references and sets (very similar to top-level ones), closures, and the dynamic environment stuff. Local mutable variables are my next project, and then I think closures. One question I've had is, can we assume that the variables in the (guile) module are immutable? I think the current compiler does this. Otherwise there's no correct way to inline primitive procedures like + unless we have a way to invalidate our compiled code and recompile it (which I would like to have anyway, but that's a different story). Best, Noah
Re: CPS Update
Hello, On Sat, Feb 16, 2013 at 2:39 AM, Stefan Israelsson Tampe < stefan.ita...@gmail.com> wrote: > > Isn't the primitiveness decided by tree-il? you wil get a ( > ...) tree il element and then should know how to emit a > corresponding primitive instruction(s). Oh, you're right. I was thinking about that because I don't run the Tree-IL optimizers when I test it, so I don't get any Tree-IL primitives. Eventually maybe the primitive generation should happen in CPS, but I don't think it has to happen all at once. > Anyway I think that your > conclusion is right, it would not make sense to support overwriting > these primitives dynamically, but > > scheme@(guile-user)> (set! + (lambda x (apply * x))) > scheme@(guile-user)> (+ 2 3) > $1 = 6 > scheme@(guile-user)> (define (f x y) (+ x y)) > scheme@(guile-user)> ,x f > Disassembly of #: > >0(assert-nargs-ee/locals 2) ;; 2 args, 0 locals >2(local-ref 0) ;; `x' >4(local-ref 1) ;; `y' >6(add) >7(return) > > So things are not consistent! > Good example! This looks like a bug to me. I think I read that loading GOOPS turns things like '+ into generic operations - that might be a case where this matters a lot. I have thought a bit about how to fix this. The module system already allows us to be notified whenever a variable changes, so it would be easy to watch all of the variables in (guile) and recompile procedures when they change. I might take a look at this soon. > BTW > If you like I could work on getting define* and lambda* to work, I > have that code in my branch and should be able > to make it work on your branch as well. WDYT? Thanks, but I haven't even gotten far enough to think about this. I'm still working on getting all of the basic features going. After that I would definitely like define* and lambda*. Noah > > /Stefan > > On Sat, Feb 16, 2013 at 1:53 AM, Noah Lavine > wrote: > > Hello, > > > > The wip-rtl-cps branch has been rebased again (on top of wip-rtl). It now > > includes support for toplevel references and sets, thanks mostly to > Andy's > > work on supporting them in RTL. (Although you shouldn't kick that part > of it > > too hard just yet; I think I know where it will break.) Other highlights > > include using the top-level variable support to evaluate "(+ 3 4)" > > correctly. > > > > Overall, I think it's coming along well. The parts of Tree-IL that are > left > > to implement are local mutable variables (should be easy after toplevel > > ones), module references and sets (very similar to top-level ones), > > closures, and the dynamic environment stuff. Local mutable variables are > my > > next project, and then I think closures. > > > > One question I've had is, can we assume that the variables in the (guile) > > module are immutable? I think the current compiler does this. Otherwise > > there's no correct way to inline primitive procedures like + unless we > have > > a way to invalidate our compiled code and recompile it (which I would > like > > to have anyway, but that's a different story). > > > > Best, > > Noah > > >
Re: CPS Update
Hello, On Sat, Feb 16, 2013 at 2:14 PM, Mark H Weaver wrote: > Noah Lavine writes: > > Oh, you're right. I was thinking about that because I don't run the > > Tree-IL optimizers when I test it, so I don't get any Tree-IL > > primitives. > > I think you should be running the existing tree-il passes before you > convert to CPS. As I pointed out earlier, many optimizers will not be > able to work well after conversion to CPS, fundamentally because CPS > loses information about where order of evaluation is unspecified. > > Furthermore, some of the existing passes make important simplifications > to the resulting tree-il. For example, the code in fix-letrec.scm > eliminates all nodes, so that's one less thing for you to have > to implement. Primitives.scm marks primitives explicitly using > tree-il nodes, so that later code doesn't have to make thorny > assumptions about what variables should be considered primitives. Interesting points. I think you're right that Tree-IL should be optimized first. For the specific issue of order of evaluation, I'm hoping to add something (either a special function or a special node type) to CPS to represent continuations that can be evaluated in any order. It seems like evaluation order is a decision that should be made in either the code generator or the register allocator, and those operate on CPS. I also dislike losing information. > > I think I read that loading GOOPS turns things like '+ into generic > > operations - that might be a case where this matters a lot. > > No, because GOOPS has a special mechanism for cases like '+': they are > called "primitive generics". When numerical operations are unable to > handle the argument types provided, they calls 'scm_wta_dispatch_*'. > Therefore, adding new methods to numerical operations does not involve > changing their bindings. This is good because it means that you can > extend numerical operations without slowing them down at all for > built-in numbers. > I see. That's very cool. > > > I have thought a bit about how to fix this. The module system already > > allows us to be notified whenever a variable changes, so it would be > > easy to watch all of the variables in (guile) and recompile procedures > > when they change. I might take a look at this soon. > > This would be nice too, of course, but I warn you that it's a can of > worms. In the general case, it involves on-stack replacement, because > you might need to recompile a procedure that's currently in the middle > of being run, and thus currently has activation records on the stack(s). > You mean if a function modifies another function that called it. Yes, that seems complicated. It's not very important to me to fix it in general, because I don't know of a case where it's useful. I think fixing it without the on-stack replacement would be a reasonable compromise. > Thanks for you work on this, Noah! > Thank you! Noah
RTL Call Issue
Hello, I've hit an interesting issue in the RTL "call" instruction. I find that I can't execute a call instruction where the return frame starts at the same place that the procedure to call is at - it must start past there. That doesn't match my expected behavior, but I'm not sure if my expectations are wrong or there's really a bug. To see it, first define two procedures: (define (return-three) 3) (define (return-five) 5) To reproduce the behavior, try running this RTL program: ((assemble-program '(begin-program foo) (assert-nargs-ee/locals 2 3) (call 1 1 ()) (values) (values))) return-three return-five) I expected that to call return-five (which is at register 1) and then print the contents of its stack, which should contain 5. Instead, it gives an error. However, the following works: ((assemble-program '(begin-program foo) (assert-nargs-ee/locals 2 3) (call 2 1 ()) (values) (values))) return-three return-five) The only difference is in the call instruction - it pushes the new frame on just past the procedure, instead of on top of it. Is that expected? I first ran into this when I was cleaning up the CPS register allocator, and I could just change the register allocator, but I'm not sure if the problem is there or in the VM itself. Also note: the example used a procedure with two arguments that tried to call the second one. If you use a procedure with just one argument and try to call that, the VM will segfault. For example, ((assemble-program '((begin-program foo) (assert-nargs-ee/locals 1 3) (call 0 0 ()) (values) (values))) return-three) => segfault Thanks, Noah
Re: propose deprecation of generalized-vector-*
Hello, On Mon, Feb 18, 2013 at 10:55 AM, Andy Wingo wrote: > Hi, > > On Wed 23 Jan 2013 13:20, Daniel Llorens > writes: > > > In [2]: a = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]]) > > In [4]: a[1] > > Out[4]: array([4, 5, 6]) > > In [5]: a[1, 1] > > Out[5]: 5 > > > > array-ref can be extended very simply to do that. It accumulates on the > > position as it is done now, but if the index list comes up short it > > makes a shared array with the remaining axes instead of giving a rank > > error. So it shouldn't be any slower than array_ref. > > It could make sense, yes. What do others think? What happens for > array-set!? Care to propose a patch? I haven't worked with the array functionality, so I might be missing something, but I don't see why this is natural for array-ref. It breaks the expectation that array-ref always returns an element of the array. It seems to be that it might be better to have some other function that returns a slice of the array, with a one-element array being a special case of its result. (array-ref could even be implemented in terms of this other function.) I think that returning a slice instead of throwing an error would be natural if we automatically mapped scalar operations over arrays. But we don't, so an array really does have to be viewed as a very different type than its components, so this change doesn't make sense to me. I would be happy to be wrong here, but this just jumped out at me as something that would be a surprising change in behavior, and possibly lead to bugs. Does anyone have example code that shows why this makes sense? Best, Noah
Re: propose deprecation of generalized-vector-*
On Mon, Feb 18, 2013 at 11:25 AM, Mike Gran wrote: > From: Noah Lavine > > I haven't worked with the array functionality, so I might be missing > > something, but I don't see why this is natural for array-ref. > > One could imagine a Matlab-like syntax where array-ref has to have > the same number of indices as the underlying array, but, if an > index were replaced with a symbol, it would return a slice. > > if np is [[1, 2, 3], [4, 5, 6], [7, 8, 9]] > (array-ref np 1 1) -> 5 > (array-ref np 1 :) -> [4, 5, 6] > (array-ref np : 1) -> [[2], [5], [8]] > Yes, that would be cool. But I'm not arguing that we shouldn't have some cool array-slicing functions - I'm just saying that they might belong in a separate function, not in array-ref. > Or maybe that's already in Scheme. I'll admit I've never done matrices > in scheme. > As far as I know, there's no standard Scheme way to do arrays. I haven't done arrays or matrices in Scheme either, so don't take what I say too seriously. I would like to do it some time, but it'll probably only happen if I need to do a lot of array programming for work some time. I think the only way to get the interface right is to have someone write some serious array-processing software in Guile, and then basically do whatever they say. :-) Noah
Re: [Guile-commits] GNU Guile branch, wip-rtl-cps, updated. v2.1.0-180-g0d0808a
Hello, Yes, I completely agree with this. I didn't do that immediately because I'm trying to get the infrastructure for the general case working. I plan to implement un-boxing in CPS. The real reason not to do it yet is that the tree-il-CPS compiler can't compile any examples that would actually need boxes. (But it will be able to soon!) Noah On Tue, Feb 19, 2013 at 12:53 AM, Mark H Weaver wrote: > Hi Noah, > > "Noah Lavine" writes: > > commit 0d0808ae3f7390ffb250b9deb6706ad4158cce0e > > Author: Noah Lavine > > Date: Mon Feb 18 14:10:58 2013 -0500 > > > > Make Lambda Arguments Mutable > > > > * module/language/cps.scm: let variable objects come with an > > initialization value. > > * module/language/tree-il/compile-cps.scm: put all lambda arguments > in > > variable boxes, so they are mutable. > > Lambda arguments (and all other lexical variables) should only be put > into boxes if they are 'set!' somewhere within their lexical scope. > This can always be determined at compile time. It is crucial that we > minimize the number of mutable variables, since they inhibit most > optimizations. > > The required analysis is already implemented in tree-il/analyze.scm. > > Regards, >Mark >
Re: [Guile-commits] GNU Guile branch, wip-rtl-cps, updated. v2.1.0-180-g0d0808a
Oh, and thanks a lot for reviewing the CPS stuff! I really appreciate it, and I think it will make the end result a lot better than whatever I could do on my own. Noah On Tue, Feb 19, 2013 at 9:28 AM, Noah Lavine wrote: > Hello, > > Yes, I completely agree with this. I didn't do that immediately because > I'm trying to get the infrastructure for the general case working. I plan > to implement un-boxing in CPS. The real reason not to do it yet is that the > tree-il-CPS compiler can't compile any examples that would actually need > boxes. (But it will be able to soon!) > > Noah > > > On Tue, Feb 19, 2013 at 12:53 AM, Mark H Weaver wrote: > >> Hi Noah, >> >> "Noah Lavine" writes: >> > commit 0d0808ae3f7390ffb250b9deb6706ad4158cce0e >> > Author: Noah Lavine >> > Date: Mon Feb 18 14:10:58 2013 -0500 >> > >> > Make Lambda Arguments Mutable >> > >> > * module/language/cps.scm: let variable objects come with an >> > initialization value. >> > * module/language/tree-il/compile-cps.scm: put all lambda arguments >> in >> > variable boxes, so they are mutable. >> >> Lambda arguments (and all other lexical variables) should only be put >> into boxes if they are 'set!' somewhere within their lexical scope. >> This can always be determined at compile time. It is crucial that we >> minimize the number of mutable variables, since they inhibit most >> optimizations. >> >> The required analysis is already implemented in tree-il/analyze.scm. >> >> Regards, >>Mark >> > >
Re: CPS Update
Hello, On Sat, Feb 16, 2013 at 4:18 PM, Mark H Weaver wrote: > Hi Noah, > > > On Sat, Feb 16, 2013 at 2:14 PM, Mark H Weaver wrote: > [...] > > Noah Lavine writes: > > > > You mean if a function modifies another function that called it. > > There are many other cases. Think multiple threads, coroutines, logic > programming systems, etc. That's why I wrote "stack(s)". Actually, I > should have written "(partial) continuation(s)". There are any number > of ways that an activation record for some procedure you modify could > still be alive somewhere in the heap. The issue can arise even with > simple lazy data structures. I don't think it's something we should > punt on. IMO anyway. > > What do you think? > Yes, you're right. I hadn't thought about those cases. This is a tricky question. But before we continue, are you sure that the right semantics is to modify all of the continuations? In particular, let's say you have a function like this: (define (func x) (+ x 2 (my-special-function x))) And my-special-function captures its continuation, k. Later on, you modify func to be this: (define (func x) (+ x 2)) Now what is the continuation k supposed to do? That continuation doesn't exist in the latest version of func. I think in this case you have to treat it like a closure that is still holding on to the continuation that it was passed (conceptually, at least) when it was called. So it would return to the old version of func. On the other hand, take the same example, but this time redefine "+" instead of "func". Now, does the continuation k call the new definition of +, or the old one? These really are questions for me. I don't know what the correct behavior here is, but I think that if we can answer both of these questions, then we know more or less what the correct thing to do is. Noah
Re: [Guile-commits] GNU Guile branch, wip-rtl-cps, updated. v2.1.0-180-g0d0808a
Hello, On Tue, Feb 19, 2013 at 11:03 AM, Mark H Weaver wrote: > Hi Noah, > > Noah Lavine writes: > > Yes, I completely agree with this. I didn't do that immediately > > because I'm trying to get the infrastructure for the general case > > working. I plan to implement un-boxing in CPS. > > You still seem to be proceeding from the assumption that the conversion > to CPS will happen early, and that all optimizations will happen in CPS. > I continue to think that this is a bad idea, because of the order of > evaluation issue. > > I realize that you intend to extend CPS with some way to express > unspecified evaluation order, but I'm not sure that is a good idea. > The fact that CPS fully specifies evaluation order is not merely an > undesirable flaw to be remedied. It is fundamental to the nature of > CPS form, and an important part of what makes CPS desireable as an IR. > I fear that in trying to get the best of both worlds, you will instead > end up with the worst of both worlds. > > I suspect that the way to get the best of both worlds is to do several > optimizations *before* conversion to CPS. We already have an > increasingly sophisticated set of optimization passes implemented in > tree-il. Those early passes already analyze whether or not lexicals > need to be mutable or not, and make several optimizations that depend on > having this information. > > Do you intend to rewrite all of those passes for CPS? > That's a fair point. I do think that some of those optimizations would work better in CPS, but even if that's true, I don't want to port them all at once. I agree that the goal should be to have a compiler in which some optimizations are done in Tree-IL and some in CPS, and once we have that, we can play with which optimizations happen where. So for now, yes, you are completely correct. However, you might like my other reason better: the Tree-IL->CPS compiler can't compile any cases that really need mutable variable slots, so I had to test mutable variables with examples that really don't need them. Once the Tree-IL->CPS compiler can handle interesting cases, then we can start optimizing the uninteresting ones away. Best, Noah
CPS Organization
Hello, I just pushed a somewhat big commit to the CPS branch. It didn't have much content, but it moved register allocation to its own file. It also tries to move in the direction of making logically separate operations into their own passes, which Andy wanted. I think it's mostly good, but I would be interested in reviews of the interface between allocate-registers-and-labels and the rest of the program. The trouble is, it sets a lot of different properties, and therefore returns six values. I could make labels and registers into a separate pass, but we'd still have one pass returning two values and another pass returning four. Another possibility is packaging everything up into a struct. The reason I didn't do that is that the first thing I would do with that struct is extract all of its elements, so it seemed like extra work for not much benefit. I don't know of any reason we need to be able to store the state of a partially-run compiler right now. The old way of handling this was having a bunch of global variables, but this seems cleaner and more functional. There might also be a fourth way that I haven't thought of, or maybe one of these three ways is clearly best. This isn't a big issue now, but it's going to be important to organize passes well as the compiler grows more and more of them. Thanks, Noah
Re: propose deprecation of generalized-vector-*
Hello, On Wed, Feb 20, 2013 at 8:13 PM, Daniel Llorens wrote: > > On Feb 18, 2013, at 16:55, Andy Wingo wrote: > > > It could make sense, yes. What do others think? What happens for > > array-set!? Care to propose a patch? > > Patch is attached. It looks a bit unwieldy because I am duplicating > scm_array_handle_pos(), and I also had to fix the recent array_ref_1 > optimization, but it shouldn't be slower for the old use. > Thanks for working on this! > ... > > --- > > It was interesting to read the other replies, thank you (plural). It was > disappointing that nobody seems to be using arrays, but not very > surprising. I use them a fair amount, but mostly for interfacing with > C/C++. I do little with them in Guile since 1) the facilities are really > not there or they are inconvenient to use, 2) when I need arrays I need > speed and Guile isn't there either (yet!). > I agree about the speed issue, but I hope it will get better soon. The RTL VM will fix some of it, and native compilation will fix more. > More than getting this patch accepted (as it is or with different function > names), my concern is that what this patch does cannot be done on the user > side except through make-shared-array, which is too slow. So I think > there's a good argument that scm_i_make_array(), SCM_I_ARRAY_V(), > SCM_I_ARRAY_BASE() and SCM_I_ARRAY_DIMS() should be published. > > Now some optional motivation: > > In other languages that have arrays, rank-0 arrays are not differentiated > from the scalar inside. This is the right approach if you want to treat > arrays of any rank as values. Then you can say that a rank-n array is > > --- > a rank-n array of rank-0 cells > a rank-(n-1) array of rank-1 cells > ... > a rank-1 array of rank-(n-1) cells > a rank-0 array of rank-n cells = a rank-n cell. > > Figure 1. > --- > > The extended versions of array-ref and array-set! allow you to use a > rank-n array as any of these. > I'm actually not very enthusiastic about this, not because you shouldn't be able to do this, but because in order to enable the automatic de-ranking, you have to have Guile assume which dimensions you want to map over. That's how C, C++ and Fortran do it because that's how arrays are actually stored in memory, so maybe that is the right way. It just seems too low-level for me - I'd rather see an array-slice function that can split along any dimensions. > Old Guile used to have something called ‘enclosed arrays’, they are > described in the 1.6 manual. This is a mechanism inspired by APL that > allows one to split an array as in Figure 1, so that if you have a function > that operates on rank-k arrays, you can array-map! it on arrays of any rank > >= k. > > Example: > > (define A #((1 2) (3 4) (5 6)) > (define (sqr x) (* x x)) > (define (norm v) (+ (sqr (array-ref v 0)) (sqr (array-ref v 1 > (define out #(0 0 0)) > (array-map! out norm (enclosed-array A 1)) > > out => #(5 25 61), hopefully (I don't have Guile 1.6 here to try). > > In later versions of APL (and most obviously in J) people realized that > this is a bad way to go about things, since the reason you want to split A > in rank-1 cells is ‘norm’. That is, rank-1 is a property of the function > ‘norm’. And you don't need to do any conversions to iterate over the rank-1 > cells of A; you only need to move a pointer on each iteration, but to do > this array-map! should be aware of the rank of the procedure in each of its > arguments. > This gets at the heart of my issue with the array functionality. As far as I can tell, in Guile, there is no way to figure out what the rank of a function is. That's why you have to be explicit about what you're mapping over. I suppose the Common Lisp-y approach would be to make an object property called 'rank', set it for all of the built-in arithmetic functions, and maybe have some way to infer the rank of new functions That might be interesting, but I'm skeptical. > ... > > Finally, to fuel the discussion of what kind of array operations Guile > should provide, here is a list similar things that I have read about. I > imagine there are many more. The software is all open source. > > ... > Thanks a lot for starting the conversation. I would like to see Guile provide enough array functionality for serious scientific computing, and it sounds like you want the same thing. I don't really know what's missing yet, though, because I haven't tried to write a program that would use it. I think the idea of splitting arrays is great. My only concern is making it part of array-ref. I still think that's a really bad idea, because it introduces a new class of errors that are really easy to make - accidentally getting an array when you expected whatever was inside the array. I'm coming at this as a user of Matlab and Fortran. In those languages, this isn't a problem, because operations automatically map over arrays, so having an array where you expected a value doesn't lead to new
Re: [APP][Translate] I've implemented a localization adapter
Thanks for posting it! It's always nice to see people writing Guile programs. You might also be interested in the guile-user list (guile-u...@gnu.org). That's usually where people talk about building things with Guile, and they might be interested in this. This list (guile-devel) is mostly about the internals of Guile. Best, Noah Lavine On Fri, Feb 22, 2013 at 11:49 AM, B.Tag wrote: > *Hi guiler: >I've implemented a localization adapter. > This program can used in games or web app. > > Program code: > > * https://github.com/wackOnline/translates > > > *Welcome comments.. > > This my first guile program. > * > -- > -- > > http://www.boolsir.com > > GMAIL : bb.q...@gmail.com > > Q Q : 289871025 > > >
Re: Programming racket like in guile
That makes sense, but if the promotion ever happens, then there might be a bunch of old code using (compat racket ...) that would need to be converted, and we would have to keep the (compat racket ...) modules around for compatibility with old code. If there's any possibility that we will want to rename it to (language racket ...), I think we should do it right now, in the beginning, so that users of those modules don't have to change. It's fine for language support to be incomplete. That's how (language ecmascript) is right now. Best, Noah On Sat, Feb 23, 2013 at 7:44 AM, Stefan Israelsson Tampe < stefan.ita...@gmail.com> wrote: > On Saturday, February 23, 2013 08:54:09 AM Daniel Hartwig wrote: > > For those parts specific to racket, did you consider the (language > > racket ..) namespace, where an eventual language definition could be > > placed also? > > Hmm, my problem with this is that to cover the racket lang is a > monumental effort because it covers such things like imutable cons > cells a new macrology system, a new module system etc. It would take > me forever to actually complete anything close to #lang > racket. Therefore I prefere to call it a compatibility module. The > idea is to minimize the work needed to port code written in racket to > guile. If we than mange after some significant time to repreoduce > #:lan racket we can of cause promote this module to (language > racket). Does this make sense? > > /Stefan > > >
Re: Programming racket like in guile
Hello, On Fri, Feb 22, 2013 at 9:32 PM, Ian Price wrote: > Daniel Hartwig writes: > > > For those parts specific to racket, did you consider the (language > > racket ..) namespace, where an eventual language definition could be > > placed also? > > That sounds somewhat misleading, since Racket is not really one > language. Yeah, there is #lang racket, but they have others as well. > It's true that racket is both a programming system for many languages and the name of one specific language. I don't think it's too ambiguous to use (language racket ...) for that one specific language. After all, if we don't call it racket, then that language has no name! Unless you want to use (language racket ...) for all of the things supported by Racket, and maybe (language racket racket) for the actual #lang racket language. It seems excessive to me, but I'm not completely sure about that. Best, Noah
Re: propose deprecation of generalized-vector-*
Hello, On Thu, Feb 28, 2013 at 2:10 PM, Daniel Llorens wrote: > > On Feb 22, 2013, at 01:22, Noah Lavine wrote: > > > I agree about the speed issue, but I hope it will get better soon. The > RTL VM will fix some of it, and native compilation will fix more. > > That's on Scheme, but there are also many optimization issues related to > array operations. Temporaries, order of traversal, etc. > Yes, you're right. This will be a long-term project. > > I'm actually not very enthusiastic about this, not because you shouldn't > be able to do this, but because in order to enable the automatic > de-ranking, you have to have Guile assume which dimensions you want to map > over. That's how C, C++ and Fortran do it because that's how arrays are > actually stored in memory, so maybe that is the right way. It just seems > too low-level for me - I'd rather see an array-slice function that can > split along any dimensions. > > enclosed-array also let you pick what axes you wanted for the cell. You > needed to specify those axes every time. That feels /more/ low level to me. > > The memory order of arrays in Guile is absolutely low level detail, > especially since it can change at any time. However the ¿logical? order of > the axes is not. It's simpler to define the looping operation so that the > frame (the axes one loops over) consists of the axes that come first. It > plays well with the rank extension / matching mechanism that I show at the > end and with the view of an array as a list. > Yes, I think you've persuaded me that there is a "natural" order to axes. There should still be an operator that splits in other ways, but I agree that we can shortcut that in many cases. > > This gets at the heart of my issue with the array functionality. As far > as I can tell, in Guile, there is no way to figure out what the rank of a > function is. That's why you have to be explicit about what you're mapping > over. > > > > I suppose the Common Lisp-y approach would be to make an object property > called 'rank', set it for all of the built-in arithmetic functions, and > maybe have some way to infer the rank of new functions That might be > interesting, but I'm skeptical. > > Exactly, this is what is needed. Then you can write array functions that > can be extended for arguments of higher rank without the function itself > having to deal with those extra axes that are none of its concern. > Otherwise you need to give axis indications left and right. I've suffered > this in numpy. This information belongs with the function. > (I'll reply to this below) > > Thanks a lot for starting the conversation. I would like to see Guile > provide enough array functionality for serious scientific computing, and it > sounds like you want the same thing. I don't really know what's missing > yet, though, because I haven't tried to write a program that would use it. > > It's a problem, because one needs at the very least mapping and reductions > to write any kind of numeric program. Guile has absolutely nothing for > array reductions and the mapping is very low level. > A slow array reduction is easy enough to add, but I'm guessing that's not what we need. :-) Perhaps we should have some sort of (ice-9 array) module where we put useful array functions. > > I think the idea of splitting arrays is great. My only concern is making > it part of array-ref. I still think that's a really bad idea, because it > introduces a new class of errors that are really easy to make - > accidentally getting an array when you expected whatever was inside the > array. I'm coming at this as a user of Matlab and Fortran. In those > languages, this isn't a problem, because operations automatically map over > arrays, so having an array where you expected a value doesn't lead to new > errors. But in Scheme, operations *don't* automatically map, so getting an > array could lead to an exception at some point later in a program when > really the error was that you didn't give enough indices to array-ref. > > In my experience the kind of rank errors you describe are unlikely to > happen, because in most programs the ranks of arrays are static. That's a good point. I'm still not convinced, because making array-ref do this means that array-ref can return two fundamentally different types of results - arrays and other objects. This is very different than most functions. But I think this comes down to a more fundamental difference - I still don't think that functions should automatically map over arrays, and you do. If they did automatically map, then I would agree with you about array-ref, b
Re: propose deprecation of generalized-vector-*
After reading through my email, I realize that I left out the most important part - a reason *why* functions shouldn't map over arrays. My reason is that this makes function application a more complicated process. Right now, functions are very simple things in Scheme. If we add automatic mapping behavior, the mental model becomes more complicated. It seems cleaner to leave this in a module that users could load or not as they choose. However, I realize that this argument isn't very persuasive. I am open to other reasons to make this work. Best, Noah On Thu, Feb 28, 2013 at 9:42 PM, Noah Lavine wrote: > Hello, > > > On Thu, Feb 28, 2013 at 2:10 PM, Daniel Llorens > wrote: > >> >> On Feb 22, 2013, at 01:22, Noah Lavine wrote: >> >> > I agree about the speed issue, but I hope it will get better soon. The >> RTL VM will fix some of it, and native compilation will fix more. >> >> That's on Scheme, but there are also many optimization issues related to >> array operations. Temporaries, order of traversal, etc. >> > > Yes, you're right. This will be a long-term project. > > >> > I'm actually not very enthusiastic about this, not because you >> shouldn't be able to do this, but because in order to enable the automatic >> de-ranking, you have to have Guile assume which dimensions you want to map >> over. That's how C, C++ and Fortran do it because that's how arrays are >> actually stored in memory, so maybe that is the right way. It just seems >> too low-level for me - I'd rather see an array-slice function that can >> split along any dimensions. >> >> enclosed-array also let you pick what axes you wanted for the cell. You >> needed to specify those axes every time. That feels /more/ low level to me. >> >> The memory order of arrays in Guile is absolutely low level detail, >> especially since it can change at any time. However the ¿logical? order of >> the axes is not. It's simpler to define the looping operation so that the >> frame (the axes one loops over) consists of the axes that come first. It >> plays well with the rank extension / matching mechanism that I show at the >> end and with the view of an array as a list. >> > > Yes, I think you've persuaded me that there is a "natural" order to axes. > There should still be an operator that splits in other ways, but I agree > that we can shortcut that in many cases. > > >> > This gets at the heart of my issue with the array functionality. As far >> as I can tell, in Guile, there is no way to figure out what the rank of a >> function is. That's why you have to be explicit about what you're mapping >> over. >> > >> > I suppose the Common Lisp-y approach would be to make an object >> property called 'rank', set it for all of the built-in arithmetic >> functions, and maybe have some way to infer the rank of new functions That >> might be interesting, but I'm skeptical. >> >> Exactly, this is what is needed. Then you can write array functions that >> can be extended for arguments of higher rank without the function itself >> having to deal with those extra axes that are none of its concern. >> Otherwise you need to give axis indications left and right. I've suffered >> this in numpy. This information belongs with the function. >> > > (I'll reply to this below) > > >> > Thanks a lot for starting the conversation. I would like to see Guile >> provide enough array functionality for serious scientific computing, and it >> sounds like you want the same thing. I don't really know what's missing >> yet, though, because I haven't tried to write a program that would use it. >> >> It's a problem, because one needs at the very least mapping and >> reductions to write any kind of numeric program. Guile has absolutely >> nothing for array reductions and the mapping is very low level. >> > > A slow array reduction is easy enough to add, but I'm guessing that's not > what we need. :-) Perhaps we should have some sort of (ice-9 array) module > where we put useful array functions. > > >> > I think the idea of splitting arrays is great. My only concern is >> making it part of array-ref. I still think that's a really bad idea, >> because it introduces a new class of errors that are really easy to make - >> accidentally getting an array when you expected whatever was inside the >> array. I'm coming at this as a user of Matlab and Fortran. In those >> languages, this isn't a problem, because operati
Re: propose deprecation of generalized-vector-*
Hello again, On Fri, Mar 1, 2013 at 4:01 AM, Daniel Llorens wrote: > > think this comes down to a more fundamental difference - I still don't > think that functions should automatically map over arrays, and you do. If > they did automatically map, then I would agree with you about array-ref, > because then arrays wouldn't be fundamentally different types from the > objects they contained. > > I actually agree here! I don't want regular scheme functions to have > things done to them around their back, it would be another language. I can > accept why you want array-ref to be strict. Indeed my approach tends to a > confusion between a 2-array of 2-arrays and a 4-array. In guile-ploy you > can see this in collapse-array ---if the verb doesn't provide an output > shape, I make an assumption. I also banish 0-rank arrays. > It seems that I misunderstood you then, and I apologize. I am very excited about the library you are proposing, and I would be happy to help in any way I can (as long as I have time...)! (I'm snipping the rest of your message because it needs more thought than I can give it right now.) > Best regards, > > Daniel > > Best, Noah
Re: Thread-unsafe initialization problems in Guile
I've only read the most recent article you posted, but if I understand
correctly, there is a third option: (3) somehow find a way to generate a
portable memory barrier instruction. Is that currently possible? I'm not
sure that it is. Probably option (2) is best if we can do it.
Noah
On Thu, Feb 28, 2013 at 4:07 PM, Mark H Weaver wrote:
> Here's an article from a reputable source that backs up my claim that
> the lazy-initialization pattern that we use in a few places is not
> thread-safe:
>
> http://www.ibm.com/developerworks/java/library/j-dcl/index.html
>
> Here are the thread-unsafe lazy initializations that I'm aware of:
>
> ./posix.c:1375:make_rw_port = scm_c_private_variable ("ice-9
> popen",
> ./debug.c:217:local_eval_var = scm_c_public_variable ("ice-9
> local-eval", "local-eval");
> ./strports.c:541: eval_string = scm_c_public_lookup ("ice-9
> eval-string", "eval-string");
>
> In each of these cases, we have two options: (1) synchronize on every
> access of the lazily-initialized variable (including reads), or (2)
> abandon lazy initialization.
>
> Thoughts?
> Mark
>
>
Re: Thread-unsafe initialization problems in Guile
Hello,
On Tue, Mar 5, 2013 at 10:24 PM, Mark H Weaver wrote:
> Hi Noah,
>
> Noah Lavine writes:
> > I've only read the most recent article you posted, but if I understand
> > correctly, there is a third option: (3) somehow find a way to generate
> > a portable memory barrier instruction. Is that currently possible?
>
> If we were to do that, we'd have to add memory barriers in two places:
> (1) after writing to the lazily-initialized variable, and (2) before
> reading from it. While memory barriers are somewhat more efficient than
> mutexes, they are still very expensive.
>
I'm not sure I understand the issue, but I think I was imagining something
like
if (variable == SCM_BOOL_F) {
acquire_mutex(var_mutex);
if (variable == SCM_BOOL_F) {
variable = initialize_variable();
memory_barrier();
}
release_mutex(var_mutex);
}
That's really just a normal locking scheme with an added memory barrier to
make sure that all threads see an update after the first thread updates the
variable. Would that work?
> As for portability, C11 is the first C standard to support memory
> barriers. For now, our best bet would probably be to use libatomic_ops,
> which is also used by libgc.
>
That means we already depend on libatomic_ops, which is good. However, I
see that the website for that library recommends using C11 instead. But I
really doubt that this issue is a big enough justification to use either
libatomic_ops or C11.
> > Probably option (2) is best if we can do it.
>
> Agreed. Unfortunately, in these cases option (2) would significantly
> increase the number of modules that need to be loaded at initialization
> time. That's why I reluctantly chose option (1).
>
That makes sense.
Thanks a lot for helping me understand these memory issues.
Noah
Re: What's the plan of Guile on GSoC 2013?
Hi, On Thu, Mar 7, 2013 at 11:29 AM, Ludovic Courtès wrote: > So, ideas? :-) > How about continuing the Emacs-Guile integration? I would be really excited to finally see that happen. Noah
Re: CPS Update
Hello, Another quick update on CPS. I just pushed patches that implement closure conversion and use it to compile closures correctly. I think this is a big step forward for the CPS compiler. Since the last email, I've also implemented local mutable variables, and sequences (as part of that). Somewhat shockingly, I think that's almost every language feature. The other things (dynamic environment, prompts) can actually be done by making closures and calling the functions that implement them - it's just faster to use the RTL instructions. However, I think the primitive procedure generation is robust enough to handle those without too much trouble. The bigger task now is a combination of finding and eliminating bugs and slowly removing the ugly hacks I put in there. I don't think there are too many, but the way the closure conversion handles letrecs and the big case statement in the primitive instruction generator are notable examples. There are also some arbitrary limitations in the code - things like only allowing one argument to a procedure, or one procedure in a letrec. I put those in because I knew I could remove them without fundamentally changing things, and I wanted to sketch out the whole compiler first to get the organization right. Now I need to go through and fix all of that. But overall, I think the CPS compiler is going well. Best, Noah On Fri, Feb 15, 2013 at 7:53 PM, Noah Lavine wrote: > Hello, > > The wip-rtl-cps branch has been rebased again (on top of wip-rtl). It now > includes support for toplevel references and sets, thanks mostly to Andy's > work on supporting them in RTL. (Although you shouldn't kick that part of > it too hard just yet; I think I know where it will break.) Other highlights > include using the top-level variable support to evaluate "(+ 3 4)" > correctly. > > Overall, I think it's coming along well. The parts of Tree-IL that are > left to implement are local mutable variables (should be easy after > toplevel ones), module references and sets (very similar to top-level > ones), closures, and the dynamic environment stuff. Local mutable variables > are my next project, and then I think closures. > > One question I've had is, can we assume that the variables in the (guile) > module are immutable? I think the current compiler does this. Otherwise > there's no correct way to inline primitive procedures like + unless we have > a way to invalidate our compiled code and recompile it (which I would like > to have anyway, but that's a different story). > > Best, > Noah > >
Re: set! semantics
Hello, I believe that keeping the box identity but not the box value is the correct behavior of continuations under the Scheme standard. I agree it's not intuitive, though. I have been thinking that we could eliminate variable boxes for mutable variables when we can prove that the continuation is never captured. I need to understand more about how continuations work, but I don't think we need a new variable type for efficiency reasons. If you want a new type of variable because you want different semantics, that's a different issue. Could you give more of an example? I assume that you're talking about something like this: (loop ;; the macro defines a loop counter (call/cc some-other-function)) So when the continuation of the loop is called more than once, the loop counter is incremented too much. Is that right? Best, Noah On Sun, Mar 17, 2013 at 9:07 AM, Stefan Israelsson Tampe < stefan.ita...@gmail.com> wrote: > Dear all, > > I just noted an issue with a programming style, that I suspect > schemers try to avoid but still is quite common in the lisp community > and actually makes for simpler macros when e.g. writing loop macros. > > The issue here is that one setup local variables with a let and then > modify them using set!. > > The problem is that this code will always be boxed. Now the overhead > of this is almost invisible due to other overhead. What I ask is what > happens when we store the stack at a call/cc setup? What will happen > is that the box identity is stored but not the box value and hence > there will be problems with recurrent calls of a continuation. > > Now I could be pretty fine with this because it's a clean statement to > say that this behavior is included in the set! semantic. And we > actually need keep this semantic to not break backward compability. > > Anyway. It's still pretty useful to have a local-set that will only > set a variable if it safely can set the local variable on the stack > directly I therefore propse for guile to include a new variable > binding construct called local-set! personally I would in most cases > avoid it's use but I would love to have it in my loop macros. > > WDYT > /Stefan > >
Re: Numeric improvements in stable-2.0
Wow, that's great. Thank you! Noah On Wed, Mar 20, 2013 at 2:49 PM, Mark H Weaver wrote: > Hello all, > > I wanted to briefly highlight some of the improvements to numerics that > I've recently pushed to stable-2.0. > > * 'number->string' now reliably outputs enough digits to produce the > same number when read back in. Previously, it badly mishandled > subnormal numbers (printing them as "#.#"), and it also failed to > distinguish between unequal inexact numbers, e.g. with IEEE 64-bit > doubles, the successor of 1.0 is (+ 1.0 (expt 2.0 -52)), which > formerly printed as "1.0" even though it was not '=' to 1.0. Now it > prints as "1.0002". > > These problems had far-reaching implications, since the compiler uses > 'number->string' to serialize numeric constants into .go files, and > that includes constants derived by the partial evaluation pass of the > compiler. E.g. (* 1e-160 1e-160) evaluated to #f at the REPL (because > of the mishandling of subnormals), and a very precise value of PI > written as a constant in source code would be imprecisely serialized > in the .go file. > > * 'exact->inexact' now guarantees correct IEEE rounding, i.e. it > reliably produces the closest representable double, with ties going to > even mantissas. > > * 'sqrt' now produces an exact rational result whenever the input is an > exact square of a rational: > > (sqrt 36/49)=> 6/7 > (sqrt #e1e-100) => > 1/100 > (sqrt #e1e100) => 100 > (sqrt (expt 2 300)) => 1427247692705959881058285969449495136382746624 > (sqrt 153440958974845354020314851471204/16857784036736598738708871009) > => 12387128762342198/129837529384753 > > * 'sqrt' now handles exact rational inputs that are too large or too small > to be represented by an inexact: > > (sqrt (expt 10 401)) => 3.1622776601683794e200 (previously +inf.0) > (sqrt (expt 10 -401)) => 3.1622776601683792e-201 (previously 0.0) > > [as you can see, 'sqrt' still does not do correct IEEE rounding, but > that's a more difficult job that will have to wait until after 2.0.8] > > * 'integer-expt' and 'expt' are now much faster when exponentiating > exact rationals (7-10 times faster in many cases). > > Mark > >
Re: [PATCH] Bindings for ‘sendfile’
Hi, sendfile looks very useful! I've thought for a while that if I had time (which I know I won't) I would make a module called (linux) with bindings for non-POSIX Linux kernel features. What do you think of this idea? If so, what do you think of putting sendfile there and expanding it with other functions as we need them? That would make it very clear when writing a program which functions were portable and which weren't. Best, Noah On Wed, Mar 20, 2013 at 6:21 PM, Ludovic Courtès wrote: > Hi, > > I plan to commit the patch below, which adds bindings for ‘sendfile’. > > Comments? > > Ludo’. > >
Re: Special variables to relax boxing
Hello, I think I understand what Stefan wants here, and I think it should probably be possible. If I understand correctly, the issue is that when a continuation is captured, it stores the *locations* of all of the variables in-scope at that point. If that continuation is invoked many times, each invocation will continue to refer to the same locations. What Stefan wants is a way to capture a continuation-like object that includes the *values* of all of the mutable variables that were in scope, so that when it restarts (even multiple times) the values will always start at the same point. To me, the strongest indicator that this should be possible is that if you took the program with mutable variables and rewrote it by splitting each function in two at each use of set! (as in continuation-passing style), making the mutable variable simply an argument to each continuation, then you would get this behavior with the current semantics. (I believe this is also the same as rewriting everything to use monads to handle mutable state.) Here's an example of what I mean. This function with mutable variables: (lambda () (let ((x 5)) (set! x (compute-1 x)) (set! x (compute-2 x)) x)) becomes (lambda () (let ((k1 (lambda (x) (k2 (compute-2 x (k2 (lambda (x) x))) (k1 (compute-1 x However, this rewriting idea runs into trouble when you try to figure out what happens to mutable variables that have been captured by closures (as Mark said). I don't know what the right solution is here. Noah On Thu, Mar 21, 2013 at 5:35 AM, Stefan Israelsson Tampe < stefan.ita...@gmail.com> wrote: > Ok, This was a first step to get what I would like to have implemented > in the tree il backend (Not for scheme but perhaps something useful > for e.g. emacs-lisp, python etc). > > My main issue with the current setup is that adding undoing and > redoing feature with the help of prompts will fail in 95% of the cases > where the program is written in an imperative style which is not > uncommmon in languages we would like to support in guile. Prompts is > such a cool feature and is probably one of the main selling points of > why one should use their language ontop of guile. So I've just > exploring how to improve on this. > > I think you missundeerstood my posted semantic. It will box if you > set! a variable and the same variable is located in a closure, I think > that is a safe approach. But you are right that it's a very unclear > semantic but my intention was to use this as a step of a better > primitive. > > So over to the semantic I would like to have, To do it currently in > guile you would need to do > something like, > (define-syntax with-guarded-var > (lambda (x) > (syntax-case x () > ((_ var code) > (with-syntax ((guard ...)) > #'(let ((guard (make-fluid))) > (with-fluids ((guard var)) > (dynamic-wind > (lambda () (set! var (fluid-ref guard)) > (lambda () (mark-as-special var) code) > (lambda x (fluid-set! guard var))) > > It might be improved but is really really a horrendous solution to the > semantic. The semantic is pretty close to a fluid variable solution > but it will mix much better with delayed computations and is a big > reason for not using fluids in a more simple try. What I would like to > propose is an idiom in tree-il that basicly looks like > > (with-special (var ...) code ...) > > and what it does is to put onto the stack is the following > > mark > var1 > place1 > var2 > place2 > ... > mark-end > - > and then execute the code just as nothing have happend. > > Then we could add the code to the stack copying part of the prompt cal/cc > etc to > when scanning the stack backwards, if it sees maek-end, then it will > store the value > of var1 in place 1 etc. And when reinstalling the stack it will search > for the mark and > place the value of place1 into the var in var1 and so on. > > > Another solution is to add a new control ideom onto the control stack > where the fluid controls and dynamic wind control lives with a pointer > ti the var1 position and the number of vars. With this solution we can > skip the marks and also improve the performance of the installment and > savings of the stack, the drawback is that we neede to handle that we > install the saved stack perhaps at another adress, but it's doable. (A > good thing using this is that we can reuse this rebasing technique to > improve the speed and scaleup of fluid variables at a tail call > position in many cases) > > I actually use a clumsy version of this semantic in guile-log and I > know that although it is anesoteric feature it means that you can > easilly implement really cool features that I would not dare to write > in e.g. kanren, If you write a prompt based logic implementation It > will make a hughe differ
Re: [PATCH] Bindings for ‘sendfile’
Hello, Yes, you're completely right - making it work on all platforms is much better than what I had proposed. I'm glad you're doing this. Thanks, Noah On Thu, Mar 21, 2013 at 5:15 AM, Ludovic Courtès wrote: > Hi Noah, > > Noah Lavine skribis: > > > I've thought for a while that if I had time (which I know I won't) I > would > > make a module called (linux) with bindings for non-POSIX Linux kernel > > features. What do you think of this idea? If so, what do you think of > > putting sendfile there and expanding it with other functions as we need > > them? > > I’ve thought about it, but ended up with making sendfile work whether or > not the syscall is available (just like glibc does, after all). > > So for this particular case, I’d rather keep it in the global name > space. There’s also the untold argument that even if sendfile(2) is > unavailable, the loop written in C is going to be faster than the > equivalent bytecode. > > FWIW, I plan to integrate the Linux bindings I wrote for “boot-to-Guile” > eventually: > > > http://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/patches/guile-linux-syscalls.patch > > These are not defined in POSIX, but apart from the Linux module > syscalls, they (that is, mount(2) and the networking ioctls) happen to > be supported by glibc on all 3 kernels, and also by other libcs. Thus, > I’d rather keep them in the global name space as well. > > WDYT? > > Thanks, > Ludo’. >
