Re: wip-threads-and-fork
On Sat 03 Mar 2012 22:20, l...@gnu.org (Ludovic Courtès) writes: I’d prefer a solution where libguile-internal threads and locks are suitably handled upon fork (basically what wip-threads-and-fork does), and where users are provided with mechanisms to do the same at their level–which boils down to exposing pthread_atfork. WDYT? I would have preferred this, but I came to the conclusion that this approach is not sound. Did you see that I merged the atfork bits into master? (wip-threads-and-fork also had some CLOEXEC bits that needed more baking). They worked... sorta. They had a few problems: 1) It's impossible to work around the lack of atfork() in libraries that you depend on. For example, wip-threads-and-fork called fork() within the GC alloc lock, to get around the lack of a pthread_atfork() in libgc. But then I submitted a patch to make libgc do this itself: http://thread.gmane.org/gmane.comp.programming.garbage-collection.boehmgc/4940 It's pretty difficult to tell which version of libgc you would have. There is no workaround that is sufficient. 2) POSIX explicitly disclaims the result of calling non-signal-safe primitives after a fork() of a multithreaded program. 3) Nobody cares about these bugs. See e.g. the lack of response at http://sourceware.org/bugzilla/show_bug.cgi?id=13725. Even Bruno didn't reply to the Cc. See point (2). 4) The atfork mechanism imposes a total ordering on locks. This is possible for static locks, but difficult for locks on collectable Scheme objects. 5) Relatedly, just to be able to lock all weak tables at a fork, we had to create a new weak table-of-tables and add the tables to it. This is needless complication and overhead. 6) scm_c_atfork() is a broken interface. Because it hangs its hooks off of one pthread_atfork() invocation, it can cause newer locks to insert themselves in the wrong position relative to pthread_atfork() calls made between when Guile installed the scm_c_atfork handler, and the call to scm_c_atfork. There can be only one pthread_atfork() list, in a correct program. In the end I reverted those patches because they were just complication that didn't solve any fundamental problems. I came to the opinion, having run a threaded, forking program, that we would be much better off if we provided good abstractions to spawn processes, but that expecting fork() to work in a multithreaded program is not realistic. Still, there is one other thing that perhaps we could do to shut down the signal handling thread around a fork(). Dunno, perhaps it is worth looking into. Andy -- http://wingolog.org/
Re: [PATCH] tree-il-scheme improvements
Hi, An answer to your one question: On Sun 04 Mar 2012 00:59, Mark H Weaver m...@netris.org writes: +(define compute-base-name Pretty nasty, but we should continue this conversation in the other thread. What other thread? The one about gensym names and peval. Happy hacking, Andy -- http://wingolog.org/
Re: Non-stack-copying call-with-current-continuation?
Hi David, On Sun 04 Mar 2012 13:01, David Kastrup d...@gnu.org writes: The global symbol space is a different identity space than heap equality, and it never gets garbage collected: the lifetime of a gensym is eternal. This is not true in Guile, where symbols can be garbage collected. I am not dismissing his suggestions. I am saying that I find call/ec a nicer primitive than catch/throw exactly because it does not need a name or symbol to work but has its identity and life-time coupled to an object rather than a symbol. OK. And frankly: the manual talks about prompts being composable and gives an example which seems utterly wrong to me since it does not actually abort a computation but rather half-finishes it. It is unclear what part of the computation will finish and what will complete. That is an interesting point. I guess there are two ways of answering it. One is to note that in Scheme, it's difficult in general to determine whether a computation is finished or will finish, because of call/cc. But you ask about a specific point, here: an abort to a prompt is basically boils down to a longjmp to the prompt's handler. The partial continuation is logically passed as an argument to the handler. I say logically because, if Guile can prove that the continuation is not referenced, then it no continuation is reified. In practice, that means that if the handler is of the form (lambda (k . other-args) FORM), and FORM does not reference `k', then an abort to that prompt does not cause the stack to be captured. We should probably say something in the manual about this. (We should probably also add higher-level interfaces like call/ec and generators, as well.) Regards, Andy -- http://wingolog.org/
Re: Non-stack-copying call-with-current-continuation?
Andy Wingo wi...@pobox.com writes: Hi David, On Sun 04 Mar 2012 13:01, David Kastrup d...@gnu.org writes: The global symbol space is a different identity space than heap equality, and it never gets garbage collected: the lifetime of a gensym is eternal. This is not true in Guile, where symbols can be garbage collected. The symbol name is not garbage collected. That is the difference between gensym and make-symbol. And frankly: the manual talks about prompts being composable and gives an example which seems utterly wrong to me since it does not actually abort a computation but rather half-finishes it. It is unclear what part of the computation will finish and what will complete. That is an interesting point. I guess there are two ways of answering it. One is to note that in Scheme, it's difficult in general to determine whether a computation is finished or will finish, because of call/cc. But you ask about a specific point, here: an abort to a prompt is basically boils down to a longjmp to the prompt's handler. The partial continuation is logically passed as an argument to the handler. But where does the partial continuation start and where does it end? If I am doing a longjmp to the prompt's handler, how can it be that the calling stack frame inside of the thunk that is supposed to be exited can finish a calculation? Where is the difference between (+ 34 (abort-to-prompt 'foo)) and (let ((x (abort-to-prompt 'foo))) (+ 34 x)) ? Why is the first allowed to complete and return a result, and the second (presumably) not? Or _if_ the second is allowed to complete, what does abort in abort-to-prompt even mean? All this does not really make discernible sense to me. Whereas call/ec has rather clear semantics and usage. The one thing that is not self-evident is its behavior in case of misuse, namely when it is asked to do a job only call/cc can. -- David Kastrup
Re: [PATCH] tree-il-scheme improvements
Andy Wingo wi...@pobox.com writes: On Sun 04 Mar 2012 00:59, Mark H Weaver m...@netris.org writes: +(define compute-base-name Pretty nasty, but we should continue this conversation in the other thread. What other thread? The one about gensym names and peval. I don't know of any recent thread about gensym names and peval. Do you mean the thread about gensym names and the unused-variable warning pass, entitled build-lexical-var vs. -Wunused-variable? Thanks, Mark
Google Summer of Code 2012
Hi! I am going trough the list of the ideas for the Google Summer of Code 2011. I am wondering if this list is still valid: http://www.gnu.org/software/soc-projects/ideas-2012.html#guile Otherwise, do you have something to suggest? Thanks! Giuseppe
Re: Non-stack-copying call-with-current-continuation?
Hello, On Sun 04 Mar 2012 14:59, David Kastrup d...@gnu.org writes: Andy Wingo wi...@pobox.com writes: This is not true in Guile, where symbols can be garbage collected. The symbol name is not garbage collected. That is the difference between gensym and make-symbol. Not sure I catch your meaning here... But you ask about a specific point, here: an abort to a prompt is basically boils down to a longjmp to the prompt's handler. The partial continuation is logically passed as an argument to the handler. But where does the partial continuation start and where does it end? It starts at the abort, and is delimited by the prompt. Or you can see it as the other way around. If I am doing a longjmp to the prompt's handler, how can it be that the calling stack frame inside of the thunk that is supposed to be exited can finish a calculation? You longjmp, yes: but after having reified (or not) the continuation. It's the continuation that lets you continue. Where is the difference between (+ 34 (abort-to-prompt 'foo)) and (let ((x (abort-to-prompt 'foo))) (+ 34 x)) ? There is no semantic difference. Why is the first allowed to complete and return a result, and the second (presumably) not? Or _if_ the second is allowed to complete, what does abort in abort-to-prompt even mean? Whether it is allowed to complete or not depends on the prompt's handler, as I mentioned before. Here is an analogy. Consider a prompt as creating a new process, in the UNIX sense. Consider abort as calling abort() and dumping core (or not). Consider calling the partial continuation as taking the core file and running it as a program (as used to be possible in some UNIX systems). Sometimes you enable cores because you are interested in them. Sometimes you just want to let a process quit and return a value. The analogy with reifying continuations or not is clear, I hope. All this does not really make discernible sense to me. Whereas call/ec has rather clear semantics and usage. Please do use call/ec, if you prefer its interface. The one thing that is not self-evident is its behavior in case of misuse, namely when it is asked to do a job only call/cc can. For the implementation based on throw, you would get an error that would be caught by the nearest (catch #t ...). For prompts, you would get an error abort to unknown prompt, which would also be caught by the nearest (catch #t ...), or a catch of `misc-error'. (Guile's error handling is not very systematic right now; something to work on.) Regards, Andy -- http://wingolog.org/
Re: [PATCH] tree-il-scheme improvements
On Sun 04 Mar 2012 16:03, Mark H Weaver m...@netris.org writes: Pretty nasty, but we should continue this conversation in the other thread. What other thread? The one about gensym names and peval. I don't know of any recent thread about gensym names and peval. Do you mean the thread about gensym names and the unused-variable warning pass, entitled build-lexical-var vs. -Wunused-variable? Yes, I think I meant that one :) Cheers, Andy -- http://wingolog.org/
Re: Non-stack-copying call-with-current-continuation?
David Kastrup d...@gnu.org writes: Andy Wingo wi...@pobox.com writes: On Sun 04 Mar 2012 13:01, David Kastrup d...@gnu.org writes: The global symbol space is a different identity space than heap equality, and it never gets garbage collected: the lifetime of a gensym is eternal. This is not true in Guile, where symbols can be garbage collected. The symbol name is not garbage collected. That is the difference between gensym and make-symbol. Integers are plentiful and cheap. Anyway, there's a well-known Efficient Gensym Hack where gensyms are given names lazily. We haven't yet implemented that in Guile, but I hope to add it before 2.2. With that optimization, if you never ask for the name of a gensym (e.g. by printing it) then it is never assigned a name or even a number. Dybvig reports that in Chez Scheme, this optimization made gensym 25 times faster. Also, in Guile 2, prompt tags need not be symbols. Anything that can be compared with 'eq?' will work. And frankly: the manual talks about prompts being composable and gives an example which seems utterly wrong to me since it does not actually abort a computation but rather half-finishes it. It is unclear what part of the computation will finish and what will complete. That is an interesting point. I guess there are two ways of answering it. One is to note that in Scheme, it's difficult in general to determine whether a computation is finished or will finish, because of call/cc. But you ask about a specific point, here: an abort to a prompt is basically boils down to a longjmp to the prompt's handler. The partial continuation is logically passed as an argument to the handler. But where does the partial continuation start and where does it end? A partial continuation captures the stack from the prompt to the abort. If you later call the partial continuation with some values, those values will be returned by 'abort-to-prompt', and the computation will continue until it returns to the prompt, at which point the partial continuation will return the values returned to the prompt. An approximate way to think about it is as follows. In the following: (call-with-prompt 'foo (lambda () (let ((x (abort-to-prompt 'foo))) (+ 34 x))) (lambda (k) k)) The returned partial continuation will be equivalent to the following procedure: (lambda xs (let ((x (apply values xs))) (+ 34 x))) To a first (very rough) approximation, you can imagine that you took the body of the thunk passed to 'call-with-prompt', wrapped it within (lambda xs ...), replaced the 'abort-to-prompt' that was invoked with (apply values xs), and that's sort of what the partial continuation acts like. How does this approximation fall short? First of all, it's not quite right to replace the 'abort-to-prompt' with (apply values xs), because if that 'abort-to-prompt' is called again (e.g. if it's within a loop) then it will abort again. More importantly, when you call a partial continuation, it jumps directly to the 'abort-to-prompt', rather than re-running the outer layers of code. Any mutable local variables bound between prompt and abort will have whatever values they were last set! to, rather than being rebound. If I am doing a longjmp to the prompt's handler, how can it be that the calling stack frame inside of the thunk that is supposed to be exited can finish a calculation? If the compiler cannot prove that the partial continuation is ignored by the handler passed to 'call-with-prompt', then the stack segment between the prompt and abort must be copied. If the partial continuation is later called, that stack segment will be copied back. Where is the difference between (+ 34 (abort-to-prompt 'foo)) and (let ((x (abort-to-prompt 'foo))) (+ 34 x)) ? Why is the first allowed to complete and return a result, and the second (presumably) not? Or _if_ the second is allowed to complete, what does abort in abort-to-prompt even mean? What makes you think that the second would not be allowed to complete? In both of these cases, assuming that what you've written above is the entire body of the thunk passed to 'call-with-prompt', the partial continuation will be equivalent to (lambda (x . rest) (+ 34 x)). All this does not really make discernible sense to me. Whereas call/ec has rather clear semantics and usage. The one thing that is not self-evident is its behavior in case of misuse, namely when it is asked to do a job only call/cc can. I agree that Guile should have 'call/ec' (ideally backed by an efficient VM instruction), and that for many purposes 'call/ec' is more natural. However, it should be noted that prompts are strictly more powerful than 'call/ec', and that 'call/ec' can be implemented easily and efficiently using prompts. For now, you can use this definition: (cond-expand (guile-2 (define (call-with-escape-continuation proc) (let ((tag (make-prompt-tag call/ec))) (call-with-prompt
Re: Non-stack-copying call-with-current-continuation?
Mark H Weaver m...@netris.org writes: David Kastrup d...@gnu.org writes: Andy Wingo wi...@pobox.com writes: On Sun 04 Mar 2012 13:01, David Kastrup d...@gnu.org writes: The global symbol space is a different identity space than heap equality, and it never gets garbage collected: the lifetime of a gensym is eternal. This is not true in Guile, where symbols can be garbage collected. The symbol name is not garbage collected. That is the difference between gensym and make-symbol. Integers are plentiful and cheap. We are not talking about an integer generated statically here. We are talking about integers getting burned through every time a control structure is being used. Not at compilation time, but at runtime. And with today's computers, executing a loop often enough that the integers don't fit into a single Scheme cell anymore and stop being cheap is not really an extraordinary event. Anyway, there's a well-known Efficient Gensym Hack where gensyms are given names lazily. We haven't yet implemented that in Guile, but I hope to add it before 2.2. With that optimization, if you never ask for the name of a gensym (e.g. by printing it) then it is never assigned a name or even a number. That would help. Also, in Guile 2, prompt tags need not be symbols. Anything that can be compared with 'eq?' will work. I suppose there is no compelling technical reason why this could not be made to also work with catch/throw, right? Being able to use something like (list #f) instead of gensym would seriously reduce the cost, both felt as well as actual. -- David Kastrup
Re: Non-stack-copying call-with-current-continuation?
David Kastrup d...@gnu.org writes: Mark H Weaver m...@netris.org writes: David Kastrup d...@gnu.org writes: The symbol name is not garbage collected. That is the difference between gensym and make-symbol. Integers are plentiful and cheap. We are not talking about an integer generated statically here. We are talking about integers getting burned through every time a control structure is being used. Not at compilation time, but at runtime. And with today's computers, executing a loop often enough that the integers don't fit into a single Scheme cell anymore and stop being cheap is not really an extraordinary event. Indeed, this is a good point, and another reason why we need the efficient gensym hack. Also, in Guile 2, prompt tags need not be symbols. Anything that can be compared with 'eq?' will work. I suppose there is no compelling technical reason why this could not be made to also work with catch/throw, right? Being able to use something like (list #f) instead of gensym would seriously reduce the cost, both felt as well as actual. Indeed, I see no reason why catch/throw shouldn't accept non-symbol tags. Looking through the code of Guile 1.8, I see nothing that depends upon the tag being a symbol. Unfortunately, catch/throw have long been documented as requiring a symbol, and raise an error if given a non-symbol, at least as far back as Guile 1.4. However, catch/throw _will_ accept uninterned symbols created with 'make-symbol'. Here's a faster implementation of call/ec that uses (list 'call/ec) to create prompt tags on Guile 2, and (make-symbol call/ec) on earlier versions of Guile: (cond-expand (guile-2 (define (call-with-escape-continuation proc) (let ((tag (list 'call/ec))) (call-with-prompt tag (lambda () (proc (lambda xs (abort-to-prompt tag xs (lambda (k xs) (apply values xs)) (guile (define (call-with-escape-continuation proc) (let ((key (make-symbol call/ec))) (catch key (lambda () (proc (lambda xs (throw key xs (lambda (key xs) (apply values xs))) (define call/ec call-with-escape-continuation) Regards, Mark
Re: Non-stack-copying call-with-current-continuation?
Mark H Weaver m...@netris.org writes: However, catch/throw _will_ accept uninterned symbols created with 'make-symbol'. Personally, I like uninterned symbols much better. They can be a bit confusing in Lisp because they share print names, but one can't exactly say that they do in Guile: guile (make-symbol xxx) #uninterned-symbol xxx b7838a10 guile Which feels a bit like gensym-on-demand, except that the serialization happens by address rather than counting. Here's a faster implementation of call/ec that uses (list 'call/ec) to create prompt tags on Guile 2, and (make-symbol call/ec) on earlier versions of Guile: (cond-expand (guile-2 (define (call-with-escape-continuation proc) (let ((tag (list 'call/ec))) (call-with-prompt tag (lambda () (proc (lambda xs (abort-to-prompt tag xs (lambda (k xs) (apply values xs)) (guile (define (call-with-escape-continuation proc) (let ((key (make-symbol call/ec))) (catch key (lambda () (proc (lambda xs (throw key xs (lambda (key xs) (apply values xs))) (define call/ec call-with-escape-continuation) Given the constraints of current guile-1 and guile-2, I doubt that there is much to take away anymore from this solution. Thanks -- David Kastrup
Re: Google Summer of Code 2012
I do not know if that idea is still valid. However, here are two more to add to that list: - Integration with Emacs. Guile has a very-nearly-complete implementation of Elisp. We'd like to get it to the point that it can actually run Emacs, and see if we can implement GNU's editor better than the standard Elisp interpreter. This project will require converting Emacs' C code to use Guile's object system, and possibly working on an Emacs Lisp compiler implemented in Scheme. - Compilation and speed. Guile has a pretty good compiler right now, but we always want more speed. The student could take this in different directions depending on interest. One idea that could take about a summer is to compile Guile to a register virtual machine instead of the current stack VM. I would love to help with either of these, although I am not sure I know enough to be the only mentor for them. Noah On Sun, Mar 4, 2012 at 10:50 AM, Giuseppe Scrivano gscriv...@gnu.org wrote: Hi! I am going trough the list of the ideas for the Google Summer of Code 2011. I am wondering if this list is still valid: http://www.gnu.org/software/soc-projects/ideas-2012.html#guile Otherwise, do you have something to suggest? Thanks! Giuseppe
