Re: Troubles with Objcode and Storing JIT Pointers
On Sat 03 Jul 2010 20:28, Noah Lavine writes: > I just sent two patches which start to implement this to > guile-sour...@gnu.org. (The first patch is actually a small > documentation change, which I made only so I could rewrite it with > updated documentation in the second patch.) Would you mind resending to guile-devel? It's better to discuss here :) Re-post the rationale too, please. Thanks! Andy -- http://wingolog.org/
Re: Troubles with Objcode and Storing JIT Pointers
Problems fixed - sorry for the confusion. I will send a new patch to guile-sources which corrects all of these issues. Noah On Sat, Jul 3, 2010 at 2:28 PM, Noah Lavine wrote: > Hello, > > I just sent two patches which start to implement this to > guile-sour...@gnu.org. (The first patch is actually a small > documentation change, which I made only so I could rewrite it with > updated documentation in the second patch.) > > The patch works as you described - SCM objcodes become five-word > objects, and all of the references get updated. I also had to change > all of the files that include static objcodes, which is why there are > changes to continuations.c, control.c, foreign.c, gsubr.c, and smob.c. > > Unfortunately, I've hit a snag that I don't understand - if you try to > build it, you will see that control.c encounters an error in a > function that isn't changed by the patch. I am hoping that someone who > understands Guile's build system more than I do could look at that, > because I don't know what's happening. > > There is one other issue with the patch that needs discussion, which > is what to initialize the extra cells to. Currently they're > initialized to NULL, and a comment describes a possible convention for > storing JIT code with them, but I'm not convinced it's the best way. > In particular, using an SCM symbol like 'no-jitcode for empty pointers > would be more self-documenting, but I wasn't sure if it would work in > statically-generated objcodes. > > Thanks > Noah > > On Tue, Jun 22, 2010 at 3:03 PM, Andy Wingo wrote: >> On Tue 22 Jun 2010 00:23, Noah Lavine writes: >> >>> If I may ask, do you already have patches or a git branch that adds [a >>> native code slot to objcode]? If so, what are you doing about the >>> embedded objcode issue? (Changing the bytecode to leave room for it, >>> or hiding the slot in the C code?) >> >> I do not have a patch, no. Feel free to implement :) Having not >> implemented it, I do not know exactly what's needed, but I would not put >> it in what is the struct scm_objcode, rather in the SCM value that >> includes the objcode. >> >> I don't know if we can somehow fit this in a "double cell" or if we need >> to have a five-word object (see scm_words ()). >> >> Andy >> -- >> http://wingolog.org/ >> >
Re: Troubles with Objcode and Storing JIT Pointers
Hello, I just sent two patches which start to implement this to guile-sour...@gnu.org. (The first patch is actually a small documentation change, which I made only so I could rewrite it with updated documentation in the second patch.) The patch works as you described - SCM objcodes become five-word objects, and all of the references get updated. I also had to change all of the files that include static objcodes, which is why there are changes to continuations.c, control.c, foreign.c, gsubr.c, and smob.c. Unfortunately, I've hit a snag that I don't understand - if you try to build it, you will see that control.c encounters an error in a function that isn't changed by the patch. I am hoping that someone who understands Guile's build system more than I do could look at that, because I don't know what's happening. There is one other issue with the patch that needs discussion, which is what to initialize the extra cells to. Currently they're initialized to NULL, and a comment describes a possible convention for storing JIT code with them, but I'm not convinced it's the best way. In particular, using an SCM symbol like 'no-jitcode for empty pointers would be more self-documenting, but I wasn't sure if it would work in statically-generated objcodes. Thanks Noah On Tue, Jun 22, 2010 at 3:03 PM, Andy Wingo wrote: > On Tue 22 Jun 2010 00:23, Noah Lavine writes: > >> If I may ask, do you already have patches or a git branch that adds [a >> native code slot to objcode]? If so, what are you doing about the >> embedded objcode issue? (Changing the bytecode to leave room for it, >> or hiding the slot in the C code?) > > I do not have a patch, no. Feel free to implement :) Having not > implemented it, I do not know exactly what's needed, but I would not put > it in what is the struct scm_objcode, rather in the SCM value that > includes the objcode. > > I don't know if we can somehow fit this in a "double cell" or if we need > to have a five-word object (see scm_words ()). > > Andy > -- > http://wingolog.org/ >
Re: Troubles with Objcode and Storing JIT Pointers
On Tue 22 Jun 2010 00:23, Noah Lavine writes: > If I may ask, do you already have patches or a git branch that adds [a > native code slot to objcode]? If so, what are you doing about the > embedded objcode issue? (Changing the bytecode to leave room for it, > or hiding the slot in the C code?) I do not have a patch, no. Feel free to implement :) Having not implemented it, I do not know exactly what's needed, but I would not put it in what is the struct scm_objcode, rather in the SCM value that includes the objcode. I don't know if we can somehow fit this in a "double cell" or if we need to have a five-word object (see scm_words ()). Andy -- http://wingolog.org/
Re: Troubles with Objcode and Storing JIT Pointers
Hi Andy, That sounds excellent. If I may ask, do you already have patches or a git branch that adds this slot? If so, what are you doing about the embedded objcode issue? (Changing the bytecode to leave room for it, or hiding the slot in the C code?) If not, why don't I just write that up first and submit it as a separate patch for Guile, and then try to add JIT stuff on top of it? Noah On Mon, Jun 21, 2010 at 4:19 PM, Andy Wingo wrote: > Hi Noah, > > On Mon 21 Jun 2010 20:19, Noah Lavine writes: > >> I don't know where to store the JITed code. > > The 2.2 branch probably will add a slot to objcode objects for native > code. If you want to do this, that's fine. Otherwise if you want 2.0 > compat, use an object property, or equivalently, a weak-key hash table. > > Andy > -- > http://wingolog.org/ >
Re: Troubles with Objcode and Storing JIT Pointers
Hi Noah, On Mon 21 Jun 2010 20:19, Noah Lavine writes: > I don't know where to store the JITed code. The 2.2 branch probably will add a slot to objcode objects for native code. If you want to do this, that's fine. Otherwise if you want 2.0 compat, use an object property, or equivalently, a weak-key hash table. Andy -- http://wingolog.org/
Troubles with Objcode and Storing JIT Pointers
Hello all, I have hit a snag in my attempt to add a JIT backend to Guile: I don't know where to store the JITed code. There was a discussion of this a few weeks ago in which it was decided to try to make a simple and quick JIT engine that would be invisible to Scheme, hoping to eventually make a full AOT compilation engine using GCC. I am working on this JIT engine. What I need in this is some way to stash the JIT code I've made for a procedure in such a way that I can get it back again if I need to run the same procedure again. My current attempt tries to put it in the struct scm_objcode structure as an extra pointer, but the trouble is that scm_objcodes are made directly from bytecode, by just casting the bytecode to a struct scm_objcode. That by itself wouldn't be a terrible problem, because I can rearrange things in memory behind-the-scenes to hide the extra pointer, but objcodes can also be embedded in other objcodes. In order to make this work correctly, I'd either have to scan every bytecode that becomes an objcode, find any embedded objcodes, and add space for the new pointer, or modify the bytecode compiler to leave extra room. Of these two, I think the second option would be best because it would be faster, but then the fact that the objcode structure has an extra pointer is trickling up to Scheme code, and I'm not sure where the abstraction leakage would stop. (Bytecode? Assembly code?) I could fix this by not allowing objcode slices, and instead having some sort of indirection, but that removes some efficiency that might be important. Another option is to store the JITed code with the procedure object, but a procedure is already four machine words, and making it any bigger could mess up the machine cache. Can anyone suggest a solution? Thanks Noah