[Chicken-users] code snippet (suggested for the wiki) - and several newcomer questions
Hi all, in this message http://lists.gnu.org/archive/html/chicken-users/2008-08/msg00066.html I posted an snippet, which someone suggested as an explanatory example for the wiki. Looking closer, I'd rather tear it apart in some discussion, before I should go there, since it indeed touches a lot of issues chicken beginners might want to learn about. make-internal-pipe returns two values, an input- and an output-port, connected to each other. (So it's only useful in multithreaded contexts, which is why it's interesting, though simple.) The first version - in the above referenced message - uses only the documented interface. Here I'll discuss a more elaborate version, which uses the currently undocumented interface for read-string and read-line too. First lesson. Pro's and Con's of user level threading. - We shall soon need to read a single character from a given input string buffer and update the buffer index atomically. Within a typical POSIX thread system, we would need to lock some mutex, read the character, update the offset and unlock the mutex. Now let me tell you a secret of success of a lot of games, databases and a personal experience from several years as professional computer scientist: the worst thing you can do with operating systems is actually use them. No matter how careful you are, switching to kernel mode and back is expensive in comparison to register arithmetic. User level thread systems, as chicken provides one, schedule in a way, which could be understood as cooperative under the hood. In chicken, any C expression is never interrupted. [[[ Is this actually true or just my understanding? ]]] To get the character out of the string buffer, we need a C snippet no longer than the one, we would put inside the critical section. No need to use locks at all! (define string-ref++ (foreign-lambda* char ((scheme-pointer buf) ((c-pointer integer) i)) char *p=(char *)buf; return(p[(*i)++]);)) Now this could have been the first trap already. Note that the buf parameter is declared as a scheme-pointer. A beginner would have easily chosen then c-string type instead. After all, it's a string, which is going to be passed in. If we declared the buf parameter as c-string, no visible harm where done. But behind the scene chicken would copy the whole input string into a fresh location, then run the C fragment and leave the string for the garbage collector. -- We better had used locks. Now the usual header: (define make-internal-pipe (let ([make-input-port make-input-port] [make-output-port make-output-port] [make-mutex make-mutex] [make-queue make-queue] [make-condition-variable make-condition-variable] [string-length string-length] [string-ref string-ref] [string-append string-append]) (lambda args (define name (or (and (pair? args) (car args)) 'internal-pipe)) Beginners did ask, why all these [x x]-bindings? We keep a local reference in case the global one becomes redefined. This is a questionable practise. While it's just what the doctor ordered, if you want to provide a definition, which is guaranteed to be immune against overwrites, it's difficult to use. If you would - for example - (use 'utf-8) *before* this unit is initialised, the captured string-ref where already utf-8 aware and no longer what we need here. [[[ Again this is what I understand. Is it true? ]]] Also, if you want, say for debugging, to rely on Scheme's ability to overwrite existing bindings, you are lost. (let-location ((off integer 0)) Here we declare off to be the offset in the string buffer to be an integer value, shall be usable in the form (location off) as the second parameter to string-ref++. Otherwise it's nothing but a new variable. Now the structure and some predicates for convenience. (let ((mutex (make-mutex name)) (condition (make-condition-variable name)) (queue (make-queue)) (buf #f)) (define (eof?) (eq? #!eof buf)) (define (buf-empty?) (or (not buf) (fx= off (string-length buf read-input will either update buf and off or wait for input. We use plain SRFI-18 here. A better version would use the mailbox egg and a make-mailbox instead of make-queue. This would safe both, the condition and the mutex. (read-input!) would become a simple mailbox-receive! on the queue. [[[ correct? ]]] (define (read-input!) (mutex-lock! mutex) (if (buf-empty?) (if (queue-empty? queue) (begin (mutex-unlock! mutex condition) (read-input!)) (begin (set! buf #f) (set! buf (queue-remove! queue)) (set! off 0) (mutex-unlock! mutex))) (mutex-unlock! mutex))) (read!) shall read the current character (define (read!) (if (eof?) buf
[Chicken-users] Race condition in scheduler.scm
Hi all,
in this message
http://lists.gnu.org/archive/html/chicken-users/2008-08/msg00094.html
I posted a patch to scheduler.scm, which fixes a race condition wrt. bad
file descriptors in the waiting queue.
(Attached is a slightly brushed up version.)
I have not seen any replies to this message (which is sort of strange on
this list). Did anyone consider to roll it in? If not, why? Anything
bad about it?
The race originally arose in a somewhat strange setup, which might not
be even repeatable for everyone, since it involved accidentally changing
the scheduling sequence (using more or fewer exception handlers), which
may have different results on different machines.
But the issue as such is easy to recreate, if you make a thread wait
asynchronously on any file descriptor and close that fd from a second
thread. (Something, which may or may not be intentional, but easy to
do.)
thank you
/Jörg
BTW ( why I'm I so desperate about it): I just completed the port of
Askemos (www.askemos.org) to chicken. This works pretty good on my
laptop now. I intend to release it as soon as it's better tested. For
that I need to roll the development environment out the several machines
and I'd really, really love to do so straight from chicken svn without
additional patches. (And promise: Askemos will run without the patch;
but it handles lots of concurrent network connections. It will surely
hit the race after a few minutes.)
Index: scheduler.scm
===
--- scheduler.scm (Revision 11653)
+++ scheduler.scm (Arbeitskopie)
@@ -36,7 +36,7 @@
##sys#update-thread-state-buffer ##sys#restore-thread-state-buffer
##sys#remove-from-ready-queue ##sys#unblock-threads-for-i/o ##sys#force-primordial
##sys#fdset-input-set ##sys#fdset-output-set ##sys#fdset-clear
- ##sys#fdset-select-timeout ##sys#fdset-restore
+ ##sys#fdset-select-timeout ##sys#fdset-restore ##sys#handle-bad-fd!
##sys#clear-i/o-state-for-thread!)
(foreign-declare #EOF
#ifdef HAVE_ERRNO_H
@@ -60,6 +60,7 @@
# include sys/types.h
# include sys/time.h
# include time.h
+# include sys/stat.h
static C_word C_msleep(C_word ms);
C_word C_msleep(C_word ms) {
#ifdef __CYGWIN__
@@ -341,6 +342,25 @@
(##sys#setislot t 13 #f)
(##sys#setslot t 11 (cons fd i/o)) )
+(define-foreign-variable error-bad-file int (errno == EBADF))
+
+(define (##sys#handle-bad-fd! e)
+ (let ((bad ((foreign-lambda*
+ bool ((integer fd))
+ struct stat buf;
+ int i = ( (fstat(fd, buf) == -1 errno == EBADF) ? 1 : 0);
+ return(i);)
+ (car e
+(if (and bad (pair? (cdr e)))
+ (thread-signal!
+ (cadr c)
+ (##sys#make-structure
+ 'condition
+ '(exn i/o) ;; better? '(exn i/o net)
+ (list '(exn . message) bad file descriptor
+ '(exn . arguments) (car e)) )))
+bad))
+
(define (##sys#unblock-threads-for-i/o)
(dbg fd-list: ##sys#fd-list)
(let* ([to? (pair? ##sys#timeout-list)]
@@ -353,8 +373,23 @@
(fxmax 0 (- tmo1 now)) )
0) ) ] ) ; otherwise immediate timeout.
(dbg n fds ready)
-(cond [(eq? -1 n)
- (##sys#force-primordial)]
+(cond [(eq? -1 n)
+ (cond
+ (error-bad-file
+ (set! ##sys#fd-list
+ (let loop ((l ##sys#fd-list))
+ (cond
+ ((null? l) l)
+ ((##sys#handle-bad-fd! (car l))
+ (##sys#fdset-clear (caar l))
+ ;; This is supposed to be a rare case, catch
+ ;; them one by one.
+ ;; (loop (cdr l))
+ (cdr l))
+ (else (cons (car l) (loop (cdr l)))
+ (##sys#fdset-restore)
+ (##sys#unblock-threads-for-i/o))
+ (else (##sys#force-primordial))) ]
[(fx n 0)
(set! ##sys#fd-list
(let loop ([n n] [lst ##sys#fd-list])
___
Chicken-users mailing list
Chicken-users@nongnu.org
http://lists.nongnu.org/mailman/listinfo/chicken-users
[Chicken-users] egg announcement: remote-repl
many people on #chicken and the list have requested a simple remote-repl egg. some people have requested a not-so-simple remote-repl egg, too. :) your requests have been answered. there is a new remote-repl egg as soon as the repo gets updated. it should be plugin-able to any existing code without modification. it doesnt require any knowledge of threads or conditions. its highly configurable: it uses the tcp egg by default, but you can tell it to use any type of sockets or wrappers that you'd like. it supports authentication, if you want it. it supports evaluation in any type of environment; everything is hooked on a per-session/per-server basis. for those of you who dont want or need anything so fancy, the defaults are (like i said) tcp and no auth, so you only NEED to specify a port and hostname. errors are handled nicely, so you dont have to worry about some typo in the repl erroring out your whole process. all of that is handled under the hood, even if you do use the hooks. hope this satisfies the requests. let me know if it doesnt. -elf ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
