Re: [Chicken-users] readline blocks threads
On Wed, Mar 5, 2008 at 11:50 PM, Shawn Rutledge <[EMAIL PROTECTED]> wrote: > So they're using Unix sockets. It's already non-blocking, so F_SETFL > doesn't change the behavior. But I guess I'm being stupid... readline (or the terminal? as you say) is blocking, not dbus. But it's good to know definitely what dbus is using. On Thu, Mar 6, 2008 at 12:53 AM, Elf <[EMAIL PROTECTED]> wrote: > the problem isnt nonblocking status. > the problem is that all the term settings get munged by readline - how else > could it interpret arrows, tabs, do paren bouncing, etc. You mean the same settings that can be changed via stty? I don't understand what effect that has. The usual kind of blocking that can be turned on is that when you are sitting at a shell prompt, keystrokes are not sent one-by-one, but rather it can wait until you hit the enter key and send the whole command all at once; this is a feature that can help with slow serial connections or packet networks. But I don't think that mode is in use much these days, and readline would be turning off that feature, right? because it has to see each keystroke in order to do all the tricks. Or you think it's on the output side, that the threads are blocked because output going from csi to the screen does not pass through until it is flushed? Then maybe a thread which does no output would run fine, but one which displays status messages would block? > i am working on a pure scheme readlineish lib to compensate with > approximately > the same feature set as what is presently there. it should be done shortly. OK I'll look forward to it. So you don't have any philosophical problem with the idea that an egg which does polling should automatically start a thread to do that? > however, instead of coding right now, im going through the almost 500 eggs > to make sure that the licence data is correct (ie, the licence tag and the Doesn't sound like much fun. ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] readline blocks threads
the problem isnt nonblocking status. the problem is that all the term settings get munged by readline - how else could it interpret arrows, tabs, do paren bouncing, etc. i am working on a pure scheme readlineish lib to compensate with approximately the same feature set as what is presently there. it should be done shortly. however, instead of coding right now, im going through the almost 500 eggs to make sure that the licence data is correct (ie, the licence tag and the actual licence match and have the right names, and the docs match the licences, and that all versioned licences have the proper version appended to them (eg LGPL-2.1)) because the daily licence muddle has got to stop. -elf On Wed, 5 Mar 2008, Shawn Rutledge wrote: On Tue, Mar 4, 2008 at 4:22 PM, Graham Fawcett <[EMAIL PROTECTED]> wrote: Just curious, does the nbstdin egg help at all? I know it helps when I'm running a REPL inside a Web app. Thanks for the suggestion. The relevant part of that seems to be this: int val = fcntl(fd, F_GETFL, 0); if(val == -1) return(0); return(fcntl(fd, F_SETFL, val | O_NONBLOCK) != -1); so I tried doing that myself: int ufd, sfd; dbus_connection_get_unix_fd(conn, &ufd); dbus_connection_get_socket(conn, &sfd); printf(\"unix FD %d, socket %d\\n\", ufd, sfd); int val = fcntl(sfd, F_GETFL, 0); printf(\"existing flags: 0x%X\\n\", val); if(val != -1) printf(\"fctntl returns %d\\n\", fcntl(sfd, F_SETFL, val | O_NONBLOCK)); out of which I see unix FD 4, socket 4 existing flags: 0x802 fctntl returns 0 so according to the existing flags gotten from F_GETFL, it's already a non-blocking fd. lsof gives me this csi 15457 rutledge4u unix 0xd935e860292124 socket So they're using Unix sockets. It's already non-blocking, so F_SETFL doesn't change the behavior. ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] readline blocks threads
On Tue, Mar 4, 2008 at 4:22 PM, Graham Fawcett <[EMAIL PROTECTED]> wrote: > Just curious, does the nbstdin egg help at all? I know it helps when > I'm running a REPL inside a Web app. Thanks for the suggestion. The relevant part of that seems to be this: int val = fcntl(fd, F_GETFL, 0); if(val == -1) return(0); return(fcntl(fd, F_SETFL, val | O_NONBLOCK) != -1); so I tried doing that myself: int ufd, sfd; dbus_connection_get_unix_fd(conn, &ufd); dbus_connection_get_socket(conn, &sfd); printf(\"unix FD %d, socket %d\\n\", ufd, sfd); int val = fcntl(sfd, F_GETFL, 0); printf(\"existing flags: 0x%X\\n\", val); if(val != -1) printf(\"fctntl returns %d\\n\", fcntl(sfd, F_SETFL, val | O_NONBLOCK)); out of which I see unix FD 4, socket 4 existing flags: 0x802 fctntl returns 0 so according to the existing flags gotten from F_GETFL, it's already a non-blocking fd. lsof gives me this csi 15457 rutledge4u unix 0xd935e860292124 socket So they're using Unix sockets. It's already non-blocking, so F_SETFL doesn't change the behavior. ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] readline blocks threads
On Tue, Mar 4, 2008 at 10:51 PM, Shawn Rutledge <[EMAIL PROTECTED]> wrote: > On Tue, Mar 4, 2008 at 2:45 PM, Jim Ursetto <[EMAIL PROTECTED]> wrote: > > Did you try using rlwrap? It works fine here. > > > > $ rlwrap csi > > #;1> (use srfi-18) > > #;2> (thread-start! (lambda () (let loop () (printf "loop~%") > (thread-sleep! 1) > > (loop > > # > > #;3> loop > > loop > > loop > > loop > > That's interesting. But I figured it's a bad design for the dbus egg > to only work under some circumstances, and I found one that will not > work if I bury the pollling thread inside the egg. > > Maybe the answer is that srfi-18 is a toy, don't take it too > seriously... or just don't depend on it, because it's up to the user > to choose to use it or not. > > It's a general pattern that I'm looking for: what to do when an egg > needs to do polling and/or callbacks... You can use ##sys#thread-wait-for-i/o! + thread-yield to make a thread block for input/output for a given fd. cheers, felix ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] readline blocks threads
On Tue, Mar 4, 2008 at 5:50 PM, Shawn Rutledge <[EMAIL PROTECTED]> wrote: > On Tue, Mar 4, 2008 at 3:47 PM, Elf <[EMAIL PROTECTED]> wrote: > > there is a way to do this. its just very painful. im working on a > readline-ng > > egg with the changes, as the existing readline wont work. > > OK. I hope I'm not causing too much pain. Just curious, does the nbstdin egg help at all? I know it helps when I'm running a REPL inside a Web app. Graham ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] readline blocks threads
On Tue, Mar 4, 2008 at 3:47 PM, Elf <[EMAIL PROTECTED]> wrote: > there is a way to do this. its just very painful. im working on a > readline-ng > egg with the changes, as the existing readline wont work. OK. I hope I'm not causing too much pain. ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] readline blocks threads
there is a way to do this. its just very painful. im working on a readline-ng egg with the changes, as the existing readline wont work. -elf On Tue, 4 Mar 2008, Shawn Rutledge wrote: On Tue, Mar 4, 2008 at 2:45 PM, Jim Ursetto <[EMAIL PROTECTED]> wrote: Did you try using rlwrap? It works fine here. $ rlwrap csi #;1> (use srfi-18) #;2> (thread-start! (lambda () (let loop () (printf "loop~%") (thread-sleep! 1) (loop # #;3> loop loop loop loop That's interesting. But I figured it's a bad design for the dbus egg to only work under some circumstances, and I found one that will not work if I bury the pollling thread inside the egg. Maybe the answer is that srfi-18 is a toy, don't take it too seriously... or just don't depend on it, because it's up to the user to choose to use it or not. It's a general pattern that I'm looking for: what to do when an egg needs to do polling and/or callbacks... ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] readline blocks threads
On Tue, Mar 4, 2008 at 2:45 PM, Jim Ursetto <[EMAIL PROTECTED]> wrote: > Did you try using rlwrap? It works fine here. > > $ rlwrap csi > #;1> (use srfi-18) > #;2> (thread-start! (lambda () (let loop () (printf "loop~%") (thread-sleep! > 1) > (loop > # > #;3> loop > loop > loop > loop That's interesting. But I figured it's a bad design for the dbus egg to only work under some circumstances, and I found one that will not work if I bury the pollling thread inside the egg. Maybe the answer is that srfi-18 is a toy, don't take it too seriously... or just don't depend on it, because it's up to the user to choose to use it or not. It's a general pattern that I'm looking for: what to do when an egg needs to do polling and/or callbacks... ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] readline blocks threads
Shawn: Did you try using rlwrap? It works fine here. $ rlwrap csi #;1> (use srfi-18) #;2> (thread-start! (lambda () (let loop () (printf "loop~%") (thread-sleep! 1) (loop # #;3> loop loop loop loop On 3/4/08, Shawn Rutledge <[EMAIL PROTECTED]> wrote: > If you do > > (use readline) > (current-input-port (make-gnu-readline-port "csi> ")) > (thread-start! (lambda () (let loop () (printf "loop~%") (loop > > the thread doesn't run in the background anymore, presumably because > readline uses blocking I/O. > > > ___ > Chicken-users mailing list > Chicken-users@nongnu.org > http://lists.nongnu.org/mailman/listinfo/chicken-users > ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] readline blocks threads
sigh. thread-quantum-set! is your friend. ill see about trying to get readline to do what you want, but it will lose some functionality, probably. -elf On Tue, 4 Mar 2008, Shawn Rutledge wrote: On Tue, Mar 4, 2008 at 12:36 PM, Elf <[EMAIL PROTECTED]> wrote: try putting a (thread-yield!) and it works in readline. Yeah you're right. csi> (use srfi-18) ; loading library srfi-18 ... csi> (thread-start! (lambda () (let loop () (printf "loop~%") (thread-yield!) (loop # csi> (thread-yield!) loop csi> (thread-yield!) loop Well I'm working on the dbus egg, and as it turns out there needs to be a polling loop to receive incoming dbus messages. So I was hoping to hide the polling loop inside the egg, but maybe there will be some problems with that. I wanted to make it possible to register a function (assign it to some combination of service/path/interface/method-name) as a callback, so it will be called when the matching message is received, rather than leaving it up to the user of the egg to "pull" the messages out and dispatch them, or ask them to be dispatched. So if you write a multi-threaded app, or call thread-yield! periodically, the threaded callback model could work, but it's unfortunate to require that kind of design, or else expose the necessity of doing the polling yourself. I sure wish Chicken had real threads. (Yes I know it's next to impossible.) The excuse I tell myself for accepting that limitation is that maybe when SMP gives way to grid-style computing, when each core has its own memory, then a multi-process model for computing makes more sense. Fortunately SRFI-18 threads do not block on TCP sockets, so there you go, parallel computing is possible, and I should just use processes rather than threads, keep it coarse-grained, and eat the cost of doing the messaging (as opposed to sharing memory). But that doesn't help in this case, because the dbus egg is just a nice interface to an IPC mechanism, which doesn't inherently have the need to block as long as it's based on sockets (which it might not always be). Well I guess I'll have to have stuff like dbus:peek-next-message dbus:next-message dbus:dispatch-next-message dbus:dispatch-loop and then the user of the egg has the choice: wrap the dispatch-loop call in a thread and callbacks happen automatically as long as nothing else blocks, poll manually but still have callbacks, or poll manually and examine the messages manually too. Anybody have a better idea? ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] readline blocks threads
On Tue, Mar 4, 2008 at 12:36 PM, Elf <[EMAIL PROTECTED]> wrote: > try putting a (thread-yield!) and it works in readline. Yeah you're right. csi> (use srfi-18) ; loading library srfi-18 ... csi> (thread-start! (lambda () (let loop () (printf "loop~%") (thread-yield!) (loop # csi> (thread-yield!) loop csi> (thread-yield!) loop Well I'm working on the dbus egg, and as it turns out there needs to be a polling loop to receive incoming dbus messages. So I was hoping to hide the polling loop inside the egg, but maybe there will be some problems with that. I wanted to make it possible to register a function (assign it to some combination of service/path/interface/method-name) as a callback, so it will be called when the matching message is received, rather than leaving it up to the user of the egg to "pull" the messages out and dispatch them, or ask them to be dispatched. So if you write a multi-threaded app, or call thread-yield! periodically, the threaded callback model could work, but it's unfortunate to require that kind of design, or else expose the necessity of doing the polling yourself. I sure wish Chicken had real threads. (Yes I know it's next to impossible.) The excuse I tell myself for accepting that limitation is that maybe when SMP gives way to grid-style computing, when each core has its own memory, then a multi-process model for computing makes more sense. Fortunately SRFI-18 threads do not block on TCP sockets, so there you go, parallel computing is possible, and I should just use processes rather than threads, keep it coarse-grained, and eat the cost of doing the messaging (as opposed to sharing memory). But that doesn't help in this case, because the dbus egg is just a nice interface to an IPC mechanism, which doesn't inherently have the need to block as long as it's based on sockets (which it might not always be). Well I guess I'll have to have stuff like dbus:peek-next-message dbus:next-message dbus:dispatch-next-message dbus:dispatch-loop and then the user of the egg has the choice: wrap the dispatch-loop call in a thread and callbacks happen automatically as long as nothing else blocks, poll manually but still have callbacks, or poll manually and examine the messages manually too. Anybody have a better idea? ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] readline blocks threads
(unless (eq? (build-platform) 'msvc) (set! ##sys#read-prompt-hook (let ([old ##sys#read-prompt-hook] [thread-yield! thread-yield!] ) (lambda () (when (or (##sys#fudge 12) (##sys#tty-port? ##sys#standard-input)) (old) (##sys#thread-block-for-i/o! ##sys#current-thread 0 #t) (thread-yield! ) ) in srfi-18 is how it handles the special case of the repl. i rather doubt it would work for readline. -elf On Tue, 4 Mar 2008, Jim Ursetto wrote: Unit SRFI-18: "Blocking I/O will block all threads, except for some socket operations (see the section about the tcp unit). An exception is the read-eval-print loop on UNIX platforms: waiting for input will not block other threads, provided the current input port reads input from a console." On 3/4/08, Shawn Rutledge <[EMAIL PROTECTED]> wrote: On Tue, Mar 4, 2008 at 12:05 PM, Elf <[EMAIL PROTECTED]> wrote: um... repl will always block threads. as the docs state, read is blocking except for certain network ports. If you try the example threaded loop without readline, it works. ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] readline blocks threads
Unit SRFI-18: "Blocking I/O will block all threads, except for some socket operations (see the section about the tcp unit). An exception is the read-eval-print loop on UNIX platforms: waiting for input will not block other threads, provided the current input port reads input from a console." On 3/4/08, Shawn Rutledge <[EMAIL PROTECTED]> wrote: > On Tue, Mar 4, 2008 at 12:05 PM, Elf <[EMAIL PROTECTED]> wrote: > > um... repl will always block threads. as the docs state, read is > blocking > > except for certain network ports. > > If you try the example threaded loop without readline, it works. > > > ___ > Chicken-users mailing list > Chicken-users@nongnu.org > http://lists.nongnu.org/mailman/listinfo/chicken-users > ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] readline blocks threads
On Tue, 4 Mar 2008, Shawn Rutledge wrote: On Tue, Mar 4, 2008 at 12:05 PM, Elf <[EMAIL PROTECTED]> wrote: um... repl will always block threads. as the docs state, read is blocking except for certain network ports. If you try the example threaded loop without readline, it works. and then it blocks the input thread. try putting a (thread-yield!) and it works in readline. -elf ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] readline blocks threads
On Tue, Mar 4, 2008 at 12:05 PM, Elf <[EMAIL PROTECTED]> wrote: > um... repl will always block threads. as the docs state, read is blocking > except for certain network ports. If you try the example threaded loop without readline, it works. ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] readline blocks threads
um... repl will always block threads. as the docs state, read is blocking except for certain network ports. -elf On Tue, 4 Mar 2008, Shawn Rutledge wrote: If you do (use readline) (current-input-port (make-gnu-readline-port "csi> ")) (thread-start! (lambda () (let loop () (printf "loop~%") (loop the thread doesn't run in the background anymore, presumably because readline uses blocking I/O. ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
[Chicken-users] readline blocks threads
If you do (use readline) (current-input-port (make-gnu-readline-port "csi> ")) (thread-start! (lambda () (let loop () (printf "loop~%") (loop the thread doesn't run in the background anymore, presumably because readline uses blocking I/O. ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
