[perl #132287] [CONC] stall/block in async heavy code

[via RT]

# New Ticket Created by  Justin DeVuyst 
# Please include the string:  [perl #132287]
# in the subject line of all future correspondence about this issue. 
# <URL: https://rt.perl.org/Ticket/Display.html?id=132287 >


Below is the best golfing I could manage.  I ran it like so for 
bisecting:

while perl6 test.pl6; do :; done

The symptom is that sometimes (90%+ on my box and ungolfed) the program
seems to stall out on the qqx line.  The last thing that is printed is
"before".  But on earlier commits (that actually build) no such 
stallage
happened for a couple minutes run time.




[jdv@new-host-2]$ cat test.pl6
use v6.d.PREVIEW;

my %c = (:0reactor_count);
my $s = supply {
     whenever IO::Socket::Async.listen('0.0.0.0', 5000) -> $c-conn {
         %c{$c-conn}<bar>++;

         emit $c-conn;

         %c{$c-conn}<msgs-supplier> = Supplier.new;

         $c-conn.Supply.lines.tap({
             %c{$c-conn}<msgs-supplier>.emit: $_ });
     }
}

sub start_reactor ($msgs-in) {
     start {
         react whenever $msgs-in -> $msg-in {
             say "before";
             say qqx{echo foo};
             say "after";
             %c<reactor_count>++;
         }
     }
}

start {
     react whenever $s -> $c-conn {
         start {
             start_reactor(%c{$c-conn}<msgs-supplier>.Supply);
         }
     }
}

sleep 1;
my $c = await IO::Socket::Async.connect('0.0.0.0', 5000);
await $c.print("foo\n");
sleep 1;
exit %c<reactor_count> ?? 0 !! 1;
[jdv@new-host-2]$




[jdv@new-host-2]$ git bisect good
80b49320cf854ac68a17cdd216575ee26e380325 is the first bad commit
commit 80b49320cf854ac68a17cdd216575ee26e380325
Author: Jonathan Worthington <jn...@jnthn.net>
Date:   Wed Sep 13 16:08:58 2017 +0200

     Initial re-implementation of thread pool scheduler

     Has separate general and timer queues with separate workers, and 
also
     introduces affinity queues, which are intended for cases where 
events
     will be fed into a `Supply`, and thus there's no point having lots 
of
     threads competing over them only to immediately stumble over each
     other. The separate timer queue helps with timer events being 
delayed,
     for example by a process producing a load of output.

     The new implementation also adds a supervisor, which is where we 
will
     put the smarts on how many threads to have in the pool. For now, it 
is
     already smart enough to start a lot less threads than the previous
     scheduler when they obviously aren't needed, which should help a 
good
     bit with memory consumption, as well as to add more when needed to
     break deadlocks. This area will be further developed in upcoming
     commits. The default maximum number of threads has been raised to 
64,
     now that the behavior of the scheduler is not to innevitably end up
     starting the maximum number of threads even when they don't have 
any
     work to do.

     Finally, a new RAKUDO_SCHEDULER_DEBUG environment variable can be 
set
     to see how many and what kinds of threads are being started.

:040000 040000 249e0613cf30b283f4fc3cbb0003474377f6019d 
3bb01ba4471016067e4b542a2158ee3184cdc85c M      src
:040000 040000 6a4e2506cb269875ff4d40054758a34e24c9024d 
6a827bbc76dc09ba2cb3fda55083ebb3074ae552 M      tools
[jdv@new-host-2]$