Re: [Chicken-users] Asynchronous I/O Egg Release

[Mario Domenech Goulart]

Hi Robert,

On Wed, 29 Jun 2016 19:26:48 -0600 Robert Smiley  wrote:

> I have created an asynchronous I/O egg for chicken. Would it be
> possible to have it added to the egg-locations file in subversion?
>
> Here is the repository
>
> https://github.com/yarnoiser/async-io
>
> And here is the url for the release-info file specifically.
>
> https://github.com/yarnoiser/async-io/blob/master/async-io.realease-info
>
> Hopefully this library will make it easier to read from and write to
> any file descriptor without leaving applications hanging, since all
> srfi-18 threads block when i/o is performed on ports other than tcp
> sockets.
>
> This library's readers use simple parsers to retrieve completed tokens
> from input files as they become ready. It includes a parser for scheme
> expressions as well as messages which consist of a single line.
>
> Hopefully this will be of use.

Today I realized that async-io's test was making the Linux salmonella
machines hang.

strace shows that the test process gets stuck on an endless loop running:

  poll([{fd=4, events=POLLIN}], 1, 0) = 0 (Timeout)

I could reproduce the problem on my system with:

  $ for i in `seq 100`; do csi -s run.scm && echo $i; done

It eventually hangs.

Do you know what could be causing this?

All the best.
Mario
-- 
http://parenteses.org/mario

___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] Asynchronous I/O Egg Release

[Kristian Lein-Mathisen]

Not that I know of, unfortunately. You can do a regex search on
api.call-cc.org and see what you find. But it is my understanding that this
is actually how it's meant to work from core, and so it shouldn't be
necessary in the first place. I am guessing there is a bug ticket somewhere
waiting for someone to fix?

It would be nice to fix this once and for all, perhaps also making stdin
nonblocking so that you don't need parley for that.

K.

On Friday, July 8, 2016, Robert Smiley  wrote:

> In all honesty, I hadn't used the make-input-port procedure before. It
> didn't occur to me to use that to make a nonblocking input port. Your code
> snippet seems to solve the problem quite a bit better than my egg does.
>
> Is open-input-file*/nonblock in a currently released egg?
>
> On Thu, Jul 7, 2016 at 6:04 AM, Kristian Lein-Mathisen <
> kristianl...@gmail.com
> > wrote:
>
>> I don't know how useful this is, but I though I'd throw in a test I use
>> as we've encountered this a few times as well in the posix egg:
>>
>>
>> (use posix srfi-18)
>>
>> (define mythread
>>   (thread-start!
>>(lambda ()
>>  (let loop ()
>>(define start (current-milliseconds))
>>(thread-sleep! 0.1)
>>(define elap (- (current-milliseconds) start))
>>(if (> elap 500) ;; the 0.1 second sleep took > 0.5 seconds!
>>(print "OBS! elap = " elap)
>>(loop))
>>
>> (print "cmd: sleep 1 ; echo hi" (with-input-from-pipe "sleep 1.5 ; echo
>> hi" read-string))
>> (thread-join! mythread)
>>
>> $ csi -version
>> Version 4.9.0.1 (stability/4.9.0) (rev 8b3189b)
>> linux-unix-gnu-x86-64 [ 64bit manyargs dload ptables ]
>> bootstrapped 2014-06-07
>> $ csi -s blocking-io-test.scm
>> cmd: sleep 1 ; echo hihi
>>
>> OBS! elap = 1512.0
>>
>>
>> And this is a code-snippet
>>  
>> we
>> use to solve it.
>>
>>
>> K.
>>
>> On Fri, Jul 1, 2016 at 6:53 PM, Matt Welland > > wrote:
>>
>>>
>>>
>>> On Fri, Jul 1, 2016 at 3:11 AM, Andy Bennett >> > wrote:
>>>
 Hi,

 > And of course, reads of files on the file
 > system never block at all

 A read from a file can block when the operating system needs to go to
 disk for the data. This happens when the buffer empties and it cannot be
 refilled before the next read call.

>>>
>>> I don't know if it applies to this discussion but read blocking can be
>>> quite a pain when a network fileserver such as NFS goes offline. It would
>>> be nice if other threads would continue so that the program could detect
>>> the issue and potentially take appropriate action such as let the user know
>>> *why* the program is hung.
>>>
>>>





 Regards,
 @ndy

 --
 andy...@ashurst.eu.org
 
 http://www.ashurst.eu.org/
 0290 DA75 E982 7D99 A51F  E46A 387A 7695 7EBA 75FF


 ___
 Chicken-users mailing list
 Chicken-users@nongnu.org
 
 https://lists.nongnu.org/mailman/listinfo/chicken-users

>>>
>>>
>>> ___
>>> Chicken-users mailing list
>>> Chicken-users@nongnu.org
>>> 
>>> https://lists.nongnu.org/mailman/listinfo/chicken-users
>>>
>>>
>>
>> ___
>> Chicken-users mailing list
>> Chicken-users@nongnu.org
>> 
>> https://lists.nongnu.org/mailman/listinfo/chicken-users
>>
>>
>
___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] Asynchronous I/O Egg Release

[Robert Smiley]

In all honesty, I hadn't used the make-input-port procedure before. It
didn't occur to me to use that to make a nonblocking input port. Your code
snippet seems to solve the problem quite a bit better than my egg does.

Is open-input-file*/nonblock in a currently released egg?

On Thu, Jul 7, 2016 at 6:04 AM, Kristian Lein-Mathisen <
kristianl...@gmail.com> wrote:

> I don't know how useful this is, but I though I'd throw in a test I use as
> we've encountered this a few times as well in the posix egg:
>
>
> (use posix srfi-18)
>
> (define mythread
>   (thread-start!
>(lambda ()
>  (let loop ()
>(define start (current-milliseconds))
>(thread-sleep! 0.1)
>(define elap (- (current-milliseconds) start))
>(if (> elap 500) ;; the 0.1 second sleep took > 0.5 seconds!
>(print "OBS! elap = " elap)
>(loop))
>
> (print "cmd: sleep 1 ; echo hi" (with-input-from-pipe "sleep 1.5 ; echo
> hi" read-string))
> (thread-join! mythread)
>
> $ csi -version
> Version 4.9.0.1 (stability/4.9.0) (rev 8b3189b)
> linux-unix-gnu-x86-64 [ 64bit manyargs dload ptables ]
> bootstrapped 2014-06-07
> $ csi -s blocking-io-test.scm
> cmd: sleep 1 ; echo hihi
>
> OBS! elap = 1512.0
>
>
> And this is a code-snippet
>  
> we
> use to solve it.
>
>
> K.
>
> On Fri, Jul 1, 2016 at 6:53 PM, Matt Welland 
> wrote:
>
>>
>>
>> On Fri, Jul 1, 2016 at 3:11 AM, Andy Bennett 
>> wrote:
>>
>>> Hi,
>>>
>>> > And of course, reads of files on the file
>>> > system never block at all
>>>
>>> A read from a file can block when the operating system needs to go to
>>> disk for the data. This happens when the buffer empties and it cannot be
>>> refilled before the next read call.
>>>
>>
>> I don't know if it applies to this discussion but read blocking can be
>> quite a pain when a network fileserver such as NFS goes offline. It would
>> be nice if other threads would continue so that the program could detect
>> the issue and potentially take appropriate action such as let the user know
>> *why* the program is hung.
>>
>>
>>>
>>>
>>>
>>>
>>>
>>> Regards,
>>> @ndy
>>>
>>> --
>>> andy...@ashurst.eu.org
>>> http://www.ashurst.eu.org/
>>> 0290 DA75 E982 7D99 A51F  E46A 387A 7695 7EBA 75FF
>>>
>>>
>>> ___
>>> Chicken-users mailing list
>>> Chicken-users@nongnu.org
>>> https://lists.nongnu.org/mailman/listinfo/chicken-users
>>>
>>
>>
>> ___
>> Chicken-users mailing list
>> Chicken-users@nongnu.org
>> https://lists.nongnu.org/mailman/listinfo/chicken-users
>>
>>
>
> ___
> Chicken-users mailing list
> Chicken-users@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/chicken-users
>
>
___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] Asynchronous I/O Egg Release

[Kristian Lein-Mathisen]

I don't know how useful this is, but I though I'd throw in a test I use as
we've encountered this a few times as well in the posix egg:


(use posix srfi-18)

(define mythread
  (thread-start!
   (lambda ()
 (let loop ()
   (define start (current-milliseconds))
   (thread-sleep! 0.1)
   (define elap (- (current-milliseconds) start))
   (if (> elap 500) ;; the 0.1 second sleep took > 0.5 seconds!
   (print "OBS! elap = " elap)
   (loop))

(print "cmd: sleep 1 ; echo hi" (with-input-from-pipe "sleep 1.5 ; echo hi"
read-string))
(thread-join! mythread)

$ csi -version
Version 4.9.0.1 (stability/4.9.0) (rev 8b3189b)
linux-unix-gnu-x86-64 [ 64bit manyargs dload ptables ]
bootstrapped 2014-06-07
$ csi -s blocking-io-test.scm
cmd: sleep 1 ; echo hihi

OBS! elap = 1512.0


And this is a code-snippet
 we
use to solve it.


K.

On Fri, Jul 1, 2016 at 6:53 PM, Matt Welland  wrote:

>
>
> On Fri, Jul 1, 2016 at 3:11 AM, Andy Bennett 
> wrote:
>
>> Hi,
>>
>> > And of course, reads of files on the file
>> > system never block at all
>>
>> A read from a file can block when the operating system needs to go to
>> disk for the data. This happens when the buffer empties and it cannot be
>> refilled before the next read call.
>>
>
> I don't know if it applies to this discussion but read blocking can be
> quite a pain when a network fileserver such as NFS goes offline. It would
> be nice if other threads would continue so that the program could detect
> the issue and potentially take appropriate action such as let the user know
> *why* the program is hung.
>
>
>>
>>
>>
>>
>>
>> Regards,
>> @ndy
>>
>> --
>> andy...@ashurst.eu.org
>> http://www.ashurst.eu.org/
>> 0290 DA75 E982 7D99 A51F  E46A 387A 7695 7EBA 75FF
>>
>>
>> ___
>> Chicken-users mailing list
>> Chicken-users@nongnu.org
>> https://lists.nongnu.org/mailman/listinfo/chicken-users
>>
>
>
> ___
> Chicken-users mailing list
> Chicken-users@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/chicken-users
>
>
___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] Asynchronous I/O Egg Release

[Matt Welland]

On Fri, Jul 1, 2016 at 3:11 AM, Andy Bennett  wrote:

> Hi,
>
> > And of course, reads of files on the file
> > system never block at all
>
> A read from a file can block when the operating system needs to go to
> disk for the data. This happens when the buffer empties and it cannot be
> refilled before the next read call.
>

I don't know if it applies to this discussion but read blocking can be
quite a pain when a network fileserver such as NFS goes offline. It would
be nice if other threads would continue so that the program could detect
the issue and potentially take appropriate action such as let the user know
*why* the program is hung.


>
>
>
>
>
> Regards,
> @ndy
>
> --
> andy...@ashurst.eu.org
> http://www.ashurst.eu.org/
> 0290 DA75 E982 7D99 A51F  E46A 387A 7695 7EBA 75FF
>
>
> ___
> Chicken-users mailing list
> Chicken-users@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/chicken-users
>
___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] Asynchronous I/O Egg Release

[Andy Bennett]

Hi,

> And of course, reads of files on the file
> system never block at all

A read from a file can block when the operating system needs to go to
disk for the data. This happens when the buffer empties and it cannot be
refilled before the next read call.





Regards,
@ndy

-- 
andy...@ashurst.eu.org
http://www.ashurst.eu.org/
0290 DA75 E982 7D99 A51F  E46A 387A 7695 7EBA 75FF


___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] Asynchronous I/O Egg Release

[John Cowan]

Chris Vine scripsit:

> The same effect can be achieved with other ports using
> thread-wait-for-i/o!, which although blocking also yields and only
> resumes when there is something available to read.  I don't have a
> problem with that: you just need to go with it.  And that is what the
> original poster's egg should be using.

Is there some reason why this isn't the default mode?  It seems to me
that what we want is for all standard Scheme operations to block the
thread that invokes them and no other threads, so that the programming
model is synchronous I/O (as RnRS demands) but threads still "work".
Disk operations aren't exactly instantaneous either, and shouldn't
block other threads that may be doing computations.

Providing asynchronous I/O to the user is a completely different thing.
I personally wouldn't want to use it, but I know there are reasons why
some people would.

> This is a general manifestation of one of the problems with "green"
> threads, and one of the reasons I don't like them: any inadvertent
> blocking on system calls jams everything.

Indeed, but it's a problem that the underlying system should hide
completely.  If not, I call it a design defect.

-- 
John Cowan  http://www.ccil.org/~cowanco...@ccil.org
To say that Bilbo's breath was taken away is no description at all.  There are
no words left to express his staggerment, since Men changed the language that
they learned of elves in the days when all the world was wonderful. --The Hobbit

___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] Asynchronous I/O Egg Release

[Chris Vine]

On Thu, 30 Jun 2016 12:14:00 -0600
Robert Smiley  wrote:
[snip]
> As for the library busy waiting, yes I suppose it does. The procedures
> reader-ready? and writer-ready? use file-select with a timeout of 0
> from the posix unit under the hood. thread-wait-for-i/o! or
> file-select could be used instead of these procedures to allow a
> thread or the entire process to not busy wait.

You can't use file-select for this purpose, because it also will jam
all "threads".  Having a timeout of 0 will avoid that but at the
expense of a busy wait.  You can mitigate that by including
thread-yield! in the busy loop, but the real answer is to use
thread-wait-for-i/o!.  That does what you want.

Chris

___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] Asynchronous I/O Egg Release

[Peter Bex]

On Thu, Jun 30, 2016 at 12:14:00PM -0600, Robert Smiley wrote:
> Hi guys. I'm not very familiar with the low level details, but the
> documentation for srfi-18 found here.
> 
> https://wiki.call-cc.org/man/4/Unit%20srfi-18
> 
> Contains this quote
> 
> "Blocking I/O will block all threads, except for some socket operations
> (see the section about the tcp unit). An exception is the read-eval-print
> loop on UNIX platforms: waiting for input will not block other threads,
> provided the current input port reads input from a console."

I guess this isn't entirely correct, given your hanging program.

> When I try running the following code:

[code]

> No matter how long I wait before hitting enter, result printed is always 0.
> Although I'm fairly inexperienced, I believe this means that the thread is
> not running, due to chicken blocking on the call to read-line as it waits
> for my input. The thread is never run, and never increments var.

That's right.  A simple way around this is to wrap the port in Parley,
which will make the port nonblocking:

(use srfi-18 parley)

(current-input-port (make-parley-port old)))

(define var 0)
(define thread (make-thread (lambda () (let loop ()
   (set! var (add1 var))
   (loop)
(thread-start! thread)
(read-line)
(print var)

But yeah, the current situation is still a big mess.  Sorry :(

Cheers,
Peter


signature.asc
Description: Digital signature
___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] Asynchronous I/O Egg Release

[Chris Vine]

On Thu, 30 Jun 2016 19:18:29 +0200
Peter Bex  wrote:
> But AFAIK the manual states that all ports are nonblocking, or am I
> just too confused right now?

All standard R5RS ports are blocking.

What the documentation says is "Blocking I/O will block all threads,
except for some socket operations (see the section about the tcp unit).
An exception is the read-eval-print loop on UNIX platforms: waiting for
input will not block other threads, provided the current input port
reads input from a console."

Socket operations still "block".  However, they yield to another thread
when doing so, and do not become scheduled again until there is
something available to read.

The same effect can be achieved with other ports using
thread-wait-for-i/o!, which although blocking also yields and only
resumes when there is something available to read.  I don't have a
problem with that: you just need to go with it.  And that is what the
original poster's egg should be using.

This is a general manifestation of one of the problems with "green"
threads, and one of the reasons I don't like them: any inadvertent
blocking on system calls jams everything.

Chris

___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] Asynchronous I/O Egg Release

[Robert Smiley]

Hi guys. I'm not very familiar with the low level details, but the
documentation for srfi-18 found here.

https://wiki.call-cc.org/man/4/Unit%20srfi-18

Contains this quote

"Blocking I/O will block all threads, except for some socket operations
(see the section about the tcp unit). An exception is the read-eval-print
loop on UNIX platforms: waiting for input will not block other threads,
provided the current input port reads input from a console."

When I try running the following code:

(use srfi-18)

(define var 0)
(define thread (make-thread (lambda () (let loop ()
   (set! var (add1 var))
   (loop)
(thread-start! thread)
(read-line)
(print var)

No matter how long I wait before hitting enter, result printed is always 0.
Although I'm fairly inexperienced, I believe this means that the thread is
not running, due to chicken blocking on the call to read-line as it waits
for my input. The thread is never run, and never increments var.

As for the library busy waiting, yes I suppose it does. The procedures
reader-ready? and writer-ready? use file-select with a timeout of 0 from
the posix unit under the hood. thread-wait-for-i/o! or file-select could be
used instead of these procedures to allow a thread or the entire process to
not busy wait. I was thinking of use cases such as games, where there is
work to be done, regardless of weather a client is currently communicating,
in which case it might be better to periodically check if input or output
is ready, and if not, do something else. Is there a better way to handle
those cases than busy waiting?

As I said, I'm fairly inexperienced and would appreciate any feedback you
have to offer.

-Robert

On Thu, Jun 30, 2016 at 11:18 AM, Peter Bex  wrote:

> On Thu, Jun 30, 2016 at 04:11:57PM +0100, Chris Vine wrote:
> > On Thu, 30 Jun 2016 16:24:52 +0200
> > > The OP (IIUC) states that this doesn't happen automatically for
> > > anything but the TCP unit's sockets, which is incorrect AFAICT.  Any
> > > port created by the system's core procedures should be nonblocking.
> >
> > I think you are wrong.  On my limited testing it happens with chicken if
> > you use R5RS's open-input-file on any file which is not on the file
> > system, such as "/dev/tty".  open-input-file is on any basis part of
> > the system's core procedures.  And of course, reads of files on the file
> > system never block at all: they just return end-of-file when the end is
> > reached, so the issue doesn't arise with them in the first place.
>
> Dammit, you're right!  I had no idea, sorry for being so dense.
> I've created https://bugs.call-cc.org/ticket/1303 for this.
>
> > > This a reasonably low-level procedure that you should only need to
> > > call when writing your own procedures that use file descriptors.  When
> > > you are using ports, this should be done automatically, in all cases.
> >
> > It appears not.
>
> Indeed, and that is certainly a bug!
>
> > Obviously you can always win if any use other than opening a file on the
> > file system (which never blocks anyway) or opening a socket (for which
> > chicken makes special provision) is considered "a low level use".
> > However, that just means you are agreeing with what I and the original
> > poster said, maybe without realising it.
>
> No, I realise completely.  I was just confused as to what you were
> saying.  For some reason I never ran into this, and this hasn't been
> reported before (by my knowledge).
>
> But AFAIK the manual states that all ports are nonblocking, or am I just
> too confused right now?
>
> Cheers,
> Peter
>
> ___
> Chicken-users mailing list
> Chicken-users@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/chicken-users
>
>
___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] Asynchronous I/O Egg Release

[Peter Bex]

On Thu, Jun 30, 2016 at 04:11:57PM +0100, Chris Vine wrote:
> On Thu, 30 Jun 2016 16:24:52 +0200
> > The OP (IIUC) states that this doesn't happen automatically for
> > anything but the TCP unit's sockets, which is incorrect AFAICT.  Any
> > port created by the system's core procedures should be nonblocking.
> 
> I think you are wrong.  On my limited testing it happens with chicken if
> you use R5RS's open-input-file on any file which is not on the file
> system, such as "/dev/tty".  open-input-file is on any basis part of
> the system's core procedures.  And of course, reads of files on the file
> system never block at all: they just return end-of-file when the end is
> reached, so the issue doesn't arise with them in the first place.

Dammit, you're right!  I had no idea, sorry for being so dense.
I've created https://bugs.call-cc.org/ticket/1303 for this.

> > This a reasonably low-level procedure that you should only need to
> > call when writing your own procedures that use file descriptors.  When
> > you are using ports, this should be done automatically, in all cases.
> 
> It appears not.

Indeed, and that is certainly a bug!

> Obviously you can always win if any use other than opening a file on the
> file system (which never blocks anyway) or opening a socket (for which
> chicken makes special provision) is considered "a low level use".
> However, that just means you are agreeing with what I and the original
> poster said, maybe without realising it.

No, I realise completely.  I was just confused as to what you were
saying.  For some reason I never ran into this, and this hasn't been
reported before (by my knowledge).

But AFAIK the manual states that all ports are nonblocking, or am I just
too confused right now?

Cheers,
Peter


signature.asc
Description: Digital signature
___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] Asynchronous I/O Egg Release

[Chris Vine]

On Thu, 30 Jun 2016 16:24:52 +0200
Peter Bex  wrote:
> On Thu, Jun 30, 2016 at 03:13:17PM +0100, Chris Vine wrote:
> > On Thu, 30 Jun 2016 15:22:41 +0200
> > Peter Bex  wrote:  
> > > On Thu, Jun 30, 2016 at 02:10:41PM +0100, Chris Vine wrote:  
> > > > Then consider it a bug.  Only sockets from the tcp unit operate
> > > > in that way, up to chicken-4.10 at least.  For other
> > > > descriptors you have to use chicken's srfi-18 extension, namely
> > > > thread-wait-for-i/o!.
> > > 
> > > Can you produce an example program which doesn't behave as
> > > expected?  
> > 
> > As who is expecting?  It's in the documentation.  
> 
> Which part of the documentation?  Anything that returns a port should
> return a nonblocking port.
> 
> The OP (IIUC) states that this doesn't happen automatically for
> anything but the TCP unit's sockets, which is incorrect AFAICT.  Any
> port created by the system's core procedures should be nonblocking.

I think you are wrong.  On my limited testing it happens with chicken if
you use R5RS's open-input-file on any file which is not on the file
system, such as "/dev/tty".  open-input-file is on any basis part of
the system's core procedures.  And of course, reads of files on the file
system never block at all: they just return end-of-file when the end is
reached, so the issue doesn't arise with them in the first place.

> > As I said, to make this work you have to use thread-wait-for-i/o!  
> 
> This a reasonably low-level procedure that you should only need to
> call when writing your own procedures that use file descriptors.  When
> you are using ports, this should be done automatically, in all cases.

It appears not.

> > ;;;
> > 
> > (require-extension posix srfi-18)
> > 
> > (let-values (((in out) (create-pipe)))
> >   (let ((port (open-input-file* in)))
> > (thread-start!
> >  (make-thread
> >   (lambda ()
> > (let loop ()
> >   (display (read-char port))
> >   (loop))
> >   (let ((port (open-output-file* out)))
> > (let loop ()
> >   (let ((cur-time (time->seconds (current-time
> > (thread-sleep! (seconds->time (+ cur-time 2)))
> > (write-char #\x port)
> > (flush-output port)
> > (loop)  
> 
> This is considered a low-level use, considering you are creating a
> port manually from a file descriptor.  You're right that it's
> expected that you have to do this here, though one could argue that
> open-input-file* and open-output-file* should really be marking the
> file descriptor as nonblocking, but I think that would be considered
> too intrusive.

Obviously you can always win if any use other than opening a file on the
file system (which never blocks anyway) or opening a socket (for which
chicken makes special provision) is considered "a low level use".
However, that just means you are agreeing with what I and the original
poster said, maybe without realising it.

Chris

___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] Asynchronous I/O Egg Release

[Peter Bex]

On Thu, Jun 30, 2016 at 03:13:17PM +0100, Chris Vine wrote:
> On Thu, 30 Jun 2016 15:22:41 +0200
> Peter Bex  wrote:
> > On Thu, Jun 30, 2016 at 02:10:41PM +0100, Chris Vine wrote:
> > > Then consider it a bug.  Only sockets from the tcp unit operate in
> > > that way, up to chicken-4.10 at least.  For other descriptors you
> > > have to use chicken's srfi-18 extension, namely
> > > thread-wait-for-i/o!.  
> > 
> > Can you produce an example program which doesn't behave as expected?
> 
> As who is expecting?  It's in the documentation.

Which part of the documentation?  Anything that returns a port should
return a nonblocking port.

The OP (IIUC) states that this doesn't happen automatically for anything
but the TCP unit's sockets, which is incorrect AFAICT.  Any port created
by the system's core procedures should be nonblocking.

> As I said, to make this work you have to use thread-wait-for-i/o!

This a reasonably low-level procedure that you should only need to
call when writing your own procedures that use file descriptors.  When
you are using ports, this should be done automatically, in all cases.

> ;;;
> 
> (require-extension posix srfi-18)
> 
> (let-values (((in out) (create-pipe)))
>   (let ((port (open-input-file* in)))
> (thread-start!
>  (make-thread
>   (lambda ()
> (let loop ()
>   (display (read-char port))
>   (loop))
>   (let ((port (open-output-file* out)))
> (let loop ()
>   (let ((cur-time (time->seconds (current-time
> (thread-sleep! (seconds->time (+ cur-time 2)))
> (write-char #\x port)
> (flush-output port)
> (loop)

This is considered a low-level use, considering you are creating a port
manually from a file descriptor.  You're right that it's expected that
you have to do this here, though one could argue that open-input-file*
and open-output-file* should really be marking the file descriptor as
nonblocking, but I think that would be considered too intrusive.

Finally, there's also this bug:
https://bugs.call-cc.org/ticket/1134

Cheers,
Peter


signature.asc
Description: Digital signature
___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] Asynchronous I/O Egg Release

[Chris Vine]

On Thu, 30 Jun 2016 15:22:41 +0200
Peter Bex  wrote:
> On Thu, Jun 30, 2016 at 02:10:41PM +0100, Chris Vine wrote:
> > Then consider it a bug.  Only sockets from the tcp unit operate in
> > that way, up to chicken-4.10 at least.  For other descriptors you
> > have to use chicken's srfi-18 extension, namely
> > thread-wait-for-i/o!.  
> 
> Can you produce an example program which doesn't behave as expected?

As who is expecting?  It's in the documentation.

As I said, to make this work you have to use thread-wait-for-i/o!  If
you take out the thread-sleep! (so the scheduler isn't forced to try to
read at the outset) you will get a certain amount of output because the
scheduler starts writing some characters and only when it schedules the
read does it block (the block occuring after all the characters in the
buffer have been vacated by read-char). If you do the write in the
worker thread and the read in the main thread then the program jams
immediately, with or without the thread sleep.

;;;

(require-extension posix srfi-18)

(let-values (((in out) (create-pipe)))
  (let ((port (open-input-file* in)))
(thread-start!
 (make-thread
  (lambda ()
(let loop ()
  (display (read-char port))
  (loop))
  (let ((port (open-output-file* out)))
(let loop ()
  (let ((cur-time (time->seconds (current-time
(thread-sleep! (seconds->time (+ cur-time 2)))
(write-char #\x port)
(flush-output port)
(loop)

___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] Asynchronous I/O Egg Release

[Peter Bex]

On Thu, Jun 30, 2016 at 02:10:41PM +0100, Chris Vine wrote:
> Then consider it a bug.  Only sockets from the tcp unit operate in that
> way, up to chicken-4.10 at least.  For other descriptors you have to use
> chicken's srfi-18 extension, namely thread-wait-for-i/o!.

Can you produce an example program which doesn't behave as expected?

Cheers,
Peter


signature.asc
Description: Digital signature
___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] Asynchronous I/O Egg Release

[Chris Vine]

On Thu, 30 Jun 2016 08:42:06 +0200
Peter Bex  wrote:
> On Wed, Jun 29, 2016 at 07:26:48PM -0600, Robert Smiley wrote:
> > Hopefully this library will make it easier to read from and write
> > to any file descriptor without leaving applications hanging, since
> > all srfi-18 threads block when i/o is performed on ports other than
> > tcp sockets.  
> 
> Hello Robert,
> 
> Can you elaborate on that?  CHICKEN marks all file descriptors it
> opens as nonblocking, and most eggs attempt to do so too, if
> possible.  The core system comes with a thread scheduler which
> automatically puts threads to sleep when they're reading from or
> writing to an fd that isn't ready.  So, the system is supposed to do
> automatically what you're exposing here manually.  If it doesn't,
> that would be considered a bug.

Then consider it a bug.  Only sockets from the tcp unit operate in that
way, up to chicken-4.10 at least.  For other descriptors you have to use
chicken's srfi-18 extension, namely thread-wait-for-i/o!.

Having said that, this egg (so far as I could understand it because
there were some missing reader-token procedures) seems to involve
looping on non-blocking descriptors in a busy wait for a read request
to be fulfilled, which doesn't seem the best idea.  But maybe I didn't
spot the code which yields on EAGAIN.

Chris

___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] Asynchronous I/O Egg Release

[Peter Bex]

On Wed, Jun 29, 2016 at 07:26:48PM -0600, Robert Smiley wrote:
> Hopefully this library will make it easier to read from and write to any
> file descriptor without leaving applications hanging, since all srfi-18
> threads block when i/o is performed on ports other than tcp sockets.

Hello Robert,

Can you elaborate on that?  CHICKEN marks all file descriptors it opens
as nonblocking, and most eggs attempt to do so too, if possible.  The
core system comes with a thread scheduler which automatically puts
threads to sleep when they're reading from or writing to an fd that
isn't ready.  So, the system is supposed to do automatically what you're
exposing here manually.  If it doesn't, that would be considered a bug.

Cheers,
Peter


signature.asc
Description: Digital signature
___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

Re: [Chicken-users] Asynchronous I/O Egg Release

[Mario Domenech Goulart]

Hello Robert,

On Wed, 29 Jun 2016 19:26:48 -0600 Robert Smiley  wrote:

> I have created an asynchronous I/O egg for chicken. Would it be
> possible to have it added to the egg-locations file in subversion?
>
> Here is the repository
>
> https://github.com/yarnoiser/async-io
>
> And here is the url for the release-info file specifically.
>
> https://github.com/yarnoiser/async-io/blob/master/async-io.realease-info
>
> Hopefully this library will make it easier to read from and write to
> any file descriptor without leaving applications hanging, since all
> srfi-18 threads block when i/o is performed on ports other than tcp
> sockets.
>
> This library's readers use simple parsers to retrieve completed tokens
> from input files as they become ready. It includes a parser for scheme
> expressions as well as messages which consist of a single line.
>
> Hopefully this will be of use.

Many thanks.  Your egg has been added to the coop.

All the best.
Mario
-- 
http://parenteses.org/mario

___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users

[Chicken-users] Asynchronous I/O Egg Release

[Robert Smiley]

Hello Chicken Users!

I have created an asynchronous I/O egg for chicken. Would it be possible to
have it added to the egg-locations file in subversion?

Here is the repository

https://github.com/yarnoiser/async-io


And here is the url for the release-info file specifically.

https://github.com/yarnoiser/async-io/blob/master/async-io.realease-info

Hopefully this library will make it easier to read from and write to any
file descriptor without leaving applications hanging, since all srfi-18
threads block when i/o is performed on ports other than tcp sockets.

This library's readers use simple parsers to retrieve completed tokens from
input files as they become ready. It includes a parser for scheme
expressions as well as messages which consist of a single line.

Hopefully this will be of use.

-Robert
___
Chicken-users mailing list
Chicken-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/chicken-users