Re: summer of code ideas
Hello, Noah Lavine noah.b.lav...@gmail.com writes: We talked about having a Scheme-based language that could compile to both plain C and JIT, but decided that would make the VM too complicated, Maybe I forgot to feed that thread, but I think it might be easier to have a high-level representation the VM opcodes as sexps (possibly a subset of Scheme), and then write the C files from that. Though the main difficulty is to find a sufficiently expressive way to write opcodes in. Oh dear. My impression was that we had decided to use a C parser. If not, I think the hardest part of making this work would be making the C output look nice. Libjit and C are such similar languages that making the translator would be quite simple. Well I think Andy preferred parsing C code and generating whatever the JIT needs from that. My intuition is that doing it the other way around may be simpler and possibly more elegant, but I haven’t looked into the details. Anyway, you’re the one doing the actual work, so if you find a solution that works well for everyone, then you’re done. :-) Also, after looking at the MELT project and the Modula-2 frontend, I'd say they might be receptive to the idea of using a high-level language to write parts of GCC. I think there’s no “they”—i.e., there are many hackers, and probably diverging views. In practice, MELT is still in a separate branch and isn’t used internally, though. Technically, there are a few issues with MELT: currently it requires both the source and build tree of GCC because of its tight integration with GCC’s internals (for instance it requires ‘gengtype’.) This is just to illustrate that using GCC as a library may prove to be difficult. Thanks, Ludo’.
Re: summer of code ideas
I've been toying with the idea of an AOT compiler for Guile, and I think that would make a good summer project. I think you'd want to compile from Tree-IL to GCC's GIMPLE format (the highest-level intermediate representation they have, I think). One interesting possibility would be basing it on Stalin, which compiles R4RS to C and is apparently extremely good. It's GPL-licensed, but as far as I know not FSF-copyrighted, so someone would either have to ask the person for a copyright assignment or reimplement things. Either way, there would be a substantial amount of work to be done just getting basic translation to work and integrating with both GCC and Guile, and then of course we'd like it to optimize things. Noah On Mon, Mar 7, 2011 at 2:51 PM, Andy Wingo wi...@pobox.com wrote: Hey all, We need to get together a few Guile-proposed projects for SoC students to hack on. How about we collect them here in this thread. I'll kick it off with one idea: * A CPAN for Guile. My initial thoughts are that you want a combination of APT-like discovery and download of sources of software, and a stowfs-like local storage, allowing you to update a tree of symlinks in $XDG_DATA_DIRS/guile/2.0/ to add packages, roll back changes, etc. See http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html for more on $XDG_DATA_DIRS. C extensions are outside the scope of this project. IMO a CPAN for Guile should not compile C helper libraries. You can start here by doing a survey of what other schemes do: chicken and racket being the obvious ones, and also perhaps taking a look at cabal and other CPAN-alikes developed in the last 10 years. You'd need to define a format for code -- tarballs or something. You need to think about signing too, and the possibility both of dumb servers and smart servers. You'll need to write a web client in Guile, if one hasn't been written yet. In short, a large project for an ambitious and careful hacker! Regards, Andy -- http://wingolog.org/
Re: summer of code ideas
Also, about your CPAN for guile (CGAN?) idea - it seems like with what you said about stowfs that what you're looking for is similar to Nix, but used to install Guile packages. (www.nixos.org) Do you think it would be useful to just compile a version of Nix with a different root directory, make some Guile bindings, and use that? Noah On Mon, Mar 7, 2011 at 2:51 PM, Andy Wingo wi...@pobox.com wrote: Hey all, We need to get together a few Guile-proposed projects for SoC students to hack on. How about we collect them here in this thread. I'll kick it off with one idea: * A CPAN for Guile. My initial thoughts are that you want a combination of APT-like discovery and download of sources of software, and a stowfs-like local storage, allowing you to update a tree of symlinks in $XDG_DATA_DIRS/guile/2.0/ to add packages, roll back changes, etc. See http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html for more on $XDG_DATA_DIRS. C extensions are outside the scope of this project. IMO a CPAN for Guile should not compile C helper libraries. You can start here by doing a survey of what other schemes do: chicken and racket being the obvious ones, and also perhaps taking a look at cabal and other CPAN-alikes developed in the last 10 years. You'd need to define a format for code -- tarballs or something. You need to think about signing too, and the possibility both of dumb servers and smart servers. You'll need to write a web client in Guile, if one hasn't been written yet. In short, a large project for an ambitious and careful hacker! Regards, Andy -- http://wingolog.org/
Re: summer of code ideas
Hey all, We need to get together a few Guile-proposed projects for SoC students to hack on. How about we collect them here in this thread. I'll kick it off with one idea: * A CPAN for Guile. A similar idea. Define a syntax for use-modules to pull modules off the net, something like (use-modules (url://a-url.com library module #:optional a-rev-number)) Update modules on startup. Cache modules and use last cached version if internet access is not available. Create server solution for modules. Use openID or related to control access to Guile module server. Use revision-control-like system to server up proper revision of module. Also, add syntax for procedure that, when called on local system, actually execute on remote system. (use-modules ((url://a-url.com library module #:optional a-rev-number) #:rpc external-func)) -Mike
Re: summer of code ideas
Hello! Actually, a fun project would be to implement Nix storage model, its build primitive (‘derivation’), and assorted tools (garbage collector, etc.) in Guile Scheme. It doesn’t seem unrealistic, though it’s perhaps another project; OTOH you mentioned rollback so it’s not completely irrelevant either. ;-) Another (difficult) question is how that would integrate with the rest of GNU—the GNU Build System, GSRC, ELPA, Stowfs [0], etc. Thanks, Ludo’. [0] http://www.gnu.org/software/hurd/hurd/translator/unionfs.html
Re: summer of code ideas
Hi! Noah Lavine noah.b.lav...@gmail.com writes: I've been toying with the idea of an AOT compiler for Guile, and I think that would make a good summer project. Really? I thought you were toying with JIT. :-) I think you'd want to compile from Tree-IL to GCC's GIMPLE format (the highest-level intermediate representation they have, I think). Actually, I suspect there would be a lot of technical problems to solve, notably because GCC is not a library and because passing it such complex data structures may be far from trivial (see MELT). Besides, for AOT, I was beguiled by Andy’s initial plan of using exclusively Scheme tools such as Sassy. Thanks, Ludo’.
Re: summer of code ideas
On Mon, Mar 07 2011, Mike Gran wrote: [...] A similar idea. Define a syntax for use-modules to pull modules off the net, something like (use-modules (url://a-url.com library module #:optional a-rev-number)) FWIW, i think this is a bad idea. It intermingles two concerns that are othogonal, namely installing a package and using it. I don't want to have to specify in my code where a module comes from. This is how PLaneT currently works, and their experience has been so negative that they're redesigning the whole thing now. jao -- Computer games don't affect kids; I mean if Pac Man affected us as kids, we would all be running around in darkened rooms, munching magic pills, and listening to repetitive electronic music. - Kristian Wilson, Nintendo Inc.
Re: summer of code ideas
Hello, Really? I thought you were toying with JIT. :-) Yes, it's been a while since we talked about that :-). I still want to do it, though. The current status as I see it is that I have a prototype JIT that works, and a plausible way to integrate it into Guile's VM. What needs to happen now is to write all of the opcodes as JIT functions (the prototype had five or so opcodes). Since writing them by hand would cause an absurd amount of code duplication, I wanted to do it automatically. We talked about having a Scheme-based language that could compile to both plain C and JIT, but decided that would make the VM too complicated, so the current idea is to use a C parser to parse the VM code and generate a JIT compiler from that. That's one reason I've been contributing patches to the PEG branch. I also started work on a C parser, which I was planning to publish once the PEG branch was merged (and once the parser was done, of course). So the JIT is actually moving along. I think you'd want to compile from Tree-IL to GCC's GIMPLE format (the highest-level intermediate representation they have, I think). Actually, I suspect there would be a lot of technical problems to solve, notably because GCC is not a library and because passing it such complex data structures may be far from trivial (see MELT). Besides, for AOT, I was beguiled by Andy’s initial plan of using exclusively Scheme tools such as Sassy. Ah, I see. I was tempted by the idea of using GCC's already-excellent optimizer as the backend to whatever Scheme-specific stuff we do. If there's a way to do that, I think it would be worth quite a bit of pain to use that. I notice that GCC now has the ability to use plugins, so if we could write one of those then perhaps it wouldn't be so bad. I also imagine that if we approached them and said that a GSoC student would like to write a frontend for them, they might be willing to help out a bit. I think the goal should be to make the compiler part of a GCC frontend developed with the rest of the compiler, so it would stay up to date if interfaces changed, and also so that parts of it could be reused for other things, leading to GCC developers making our compiler faster. Noah
Re: summer of code ideas
Hi! Noah Lavine noah.b.lav...@gmail.com writes: Yes, it's been a while since we talked about that :-). I still want to do it, though. The current status as I see it is that I have a prototype JIT that works, and a plausible way to integrate it into Guile's VM. What needs to happen now is to write all of the opcodes as JIT functions (the prototype had five or so opcodes). Since writing them by hand would cause an absurd amount of code duplication, I wanted to do it automatically. Yes. We talked about having a Scheme-based language that could compile to both plain C and JIT, but decided that would make the VM too complicated, Maybe I forgot to feed that thread, but I think it might be easier to have a high-level representation the VM opcodes as sexps (possibly a subset of Scheme), and then write the C files from that. Though the main difficulty is to find a sufficiently expressive way to write opcodes in. I notice that GCC now has the ability to use plugins, The plug-in API doesn’t allow plug-ins to add new front-ends (info (gccint) Plugins). Thanks, Ludo’.
Re: summer of code ideas
Hello, We talked about having a Scheme-based language that could compile to both plain C and JIT, but decided that would make the VM too complicated, Maybe I forgot to feed that thread, but I think it might be easier to have a high-level representation the VM opcodes as sexps (possibly a subset of Scheme), and then write the C files from that. Though the main difficulty is to find a sufficiently expressive way to write opcodes in. Oh dear. My impression was that we had decided to use a C parser. If not, I think the hardest part of making this work would be making the C output look nice. Libjit and C are such similar languages that making the translator would be quite simple. I notice that GCC now has the ability to use plugins, The plug-in API doesn’t allow plug-ins to add new front-ends (info (gccint) Plugins). Ah, I see. I read their wiki a bit. It does look like using GCC would be difficult, but I still think it would be worthwhile to do just because they have such a nice compiler already written. It looks like what you need to build a frontend is the ability to manipulate GCC's data structures and the ability to write functions that C code can call back to. I believe the FFI can do both of these things now (not that it wouldn't be a pain to fit everything together). Also, after looking at the MELT project and the Modula-2 frontend, I'd say they might be receptive to the idea of using a high-level language to write parts of GCC. Noah
Re: summer of code ideas
Jose A. Ortega Ruiz j...@gnu.org writes: (use-modules (url://a-url.com library module #:optional a-rev-number)) FWIW, i think this is a bad idea. It intermingles two concerns that are othogonal, namely installing a package and using it. I very strongly agree with jao. Systems like this, e.g. Python Eggs, have been a major headache for distributions to deal with. Debian actually takes the time to disable this automatic downloading and installing functionality from their Python packages, and I'm glad for it. I am very security conscious, and the thought of software being automatically installed on-demand from untrusted sources when I run a program, or maybe even when I first use some particular functionality of that program, is very disconcerting. Maybe not everyone wants this, but as a Debian and gNewSense user, I want my distribution to be an intermediary for most of the software I use. I trust them more than I trust most upstreams to ensure that the software has been somewhat vetted for license issues, security problems, anti-features, etc. I want my distribution to be able to modify the packages as necessary to make them work well together and with the rest of the system. I also want experimental distributions to be able to make significant changes to packages to fit within their new ideas of how the system should be set up. Furthermore, there are many thorny issues involved with package management that are very hard to get right, and most of the new crop of language-specific package systems like this are half-baked at best. For example, how do you ensure security? Debian has a reasonably well thought out system for using digital signatures for this. How will we handle it? Also, how will we handle versioning? It is very hard to do this properly. Sometimes you want the cutting-edge version of something, and sometimes you want stability. It is not enough to simply designate a stable version of each individual package. A stable system requires that all the individual pieces have been tested together as a whole, as is done in Debian and other distros. How will we handle it? I don't mean to be a wet blanket, because I can certainly see the appeal of a system like this, but let's please be careful not to repeat the many mistakes that other similar systems have made. It is a very thorny problem. Regards, Mark
Re: summer of code ideas
From: Mark H Weaver m...@netris.org To: Jose A. Ortega Ruiz j...@gnu.org Cc: guile-devel@gnu.org Sent: Monday, March 7, 2011 5:10 PM Subject: Re: summer of code ideas Jose A. Ortega Ruiz j...@gnu.org writes: (use-modules (url://a-url.com library module #:optional a-rev-number)) FWIW, i think this is a bad idea. It intermingles two concerns that are othogonal, namely installing a package and using it. I very strongly agree with jao. Systems like this, e.g. Python Eggs, have been a major headache for distributions to deal with. Debian actually takes the time to disable this automatic downloading and installing functionality from their Python packages, and I'm glad for it. I am very security conscious, and the thought of software being automatically installed on-demand from untrusted sources when I run a program, or maybe even when I first use some particular functionality of that program, is very disconcerting. I think people are conflating 'an idea' with 'the idea' here. The call was for possible SOC ideas. At my work, we sometimes operate this way: field engineers use client programs that are rather like AJAX webapps. The head office updates a server. It works for us. I didn't suggest that for all time Guile must officially such a system. Sheesh. -Mike
