[perl #126824] [CONC] await on a Supply .head .Promise never fires (but Supplier.head.Promise does)
On Sat Dec 05 11:13:54 2015, b...@abrij.org wrote:
>
> 16:18 skids I think the Supply work may have made
> $fh.watch.Promise not fire anymore.
> 16:20 watch.tap still works.
> 16:21 jnthn The semantics of Supply.Promise changed
> 16:21 You'll need $fh.watch.head(1).Promise to get the
> original ones
> 16:21 skids Ah that would explain it. jnthn++
> 16:22 Hrm that doesn't seem to do it though.
> 2 more elements. Show/hide.
> 16:34 jnthn Hm, head doesn't seem to be filing the "done"
>
> These work:
>
> $ perl6 -e 'my $f = Supplier.new; my $s = $f.head(1).Supply; my $t =
> $s.tap(-> $ { 42.say }, done => {43.say} ); start { for 0..3 {
> $f.emit(42) }; $f.done(); }; await $s.Promise; 44.say; '
> 42
> 43
> 44
>
> $ perl6 -e 'my $f = Supplier.new; my $s = $f.head(1).Supply; my $t =
> $s.tap(-> $ { 42.say }, done => {43.say} ); start { for 0..3 {
> $f.emit(42) }; }; await $s.Promise; 44.say; '
> 42
> 43
> 44
>
>
> This neither sends a done nor fulfills a .Promise (input echoed into
> file from another terminal):
>
> $ perl6 -e 'my $fh = open("/tmp/foo.txt", :create); my $s =
> $fh.watch.head(1); my $t = $s.tap(-> $ { 42.say }, done => {43.say} );
> await $s.Promise; 44.say'
> 42
> ^C
>
This works reliably now, and so could do with a test. Probably thanks to one of
the many Supply fixes over the last months.
> ...and when the .head is off a Supply instead of a Supplier, a done
> from the original Supply propagates but the Promise never unhooks:
>
> $ perl6 -e 'my $f = Supplier.new; my $s = $f.Supply.head(1); my $t =
> $s.tap(-> $ { 42.say }, done => {43.say} ); start { for 0..3 {
> $f.emit(42) }; $f.done(); }; await $s.Promise; 44.say; '
> 42
> 43
> ^C
>
> $ perl6 -e 'my $f = Supplier.new; my $s = $f.Supply.head(1); my $t =
> $s.tap(-> $ { 42.say }, done => {43.say} ); start { for 0..3 {
> $f.emit(42) }; }; await $s.Promise; 44.say; '
> 42
> ^C
This code is vulnerable to a race condition, thus it sometimes works, sometimes
not. If you add a `sleep 1` before the `await` then it reliably hangs. The
reason is that a Supplier is a live supply; if you miss the events (emit/done)
then they're gone for good. (Note that `.Promise` taps the supply separately
from the `.tap` that is already taking place.) For this to reliably work, the
`Promise` should be obtained prior to setting off the `start` block. This works
reliably, every time:
my $f = Supplier.new;
my $s = $f.Supply.head(1);
my $t = $s.tap(-> $ { 42.say }, done => {43.say} );
my $p = $s.Promise;
start {
for 0..3 {
$f.emit(42)
};
$f.done();
}
await $p;
44.say;
Tagging testneeded for the part that deserves a test.
/jnthn
[perl #127960] [CONC] [PERF] GC related crash on repeated sleepsort (using Channel)
On Fri Apr 22 08:55:46 2016, jns...@gellyfish.co.uk wrote:
> The code:
>
> sub sleep_sort (*@list where .all >= 0) {
>
> my $channel = Channel.new;
>
> await @list.map: -> $delay {
> Promise.start({
> sleep $delay / 1000;
> $channel.send($delay);
> });
>
> };
>
> $channel.close;
>
> return $channel.list;
> }
>
> say sleep_sort(3,2,1,5,4) for (1 ... 1);
>
>
> And gives rise to this back trace:
> #0 0x773e8a98 in raise () from /lib64/libc.so.6
> No symbol table info available.
> #1 0x773ea69a in abort () from /lib64/libc.so.6
> No symbol table info available.
> #2 0x779ac867 in uv_mutex_destroy ()
>from /home/jonathan/.rakudobrew/moar-nom/install/lib/libmoar.so
> No symbol table info available.
> #3 0x77930c9d in gc_free ()
>from /home/jonathan/.rakudobrew/moar-nom/install/lib/libmoar.so
> No symbol table info available.
Looks like the mutex missing unlock bug that I fixed in Rakudo commit
48c2af6d059. Also, I've run the code in question (all 10,000 iterations of it)
a bunch of times and not seen any failures. For good measure, I've also added
it as a stress test in S17-channel/stress.t.
/jnthn
[perl #126774] [SEGV] [CONC] Code aborts while working with supplies
On Thu Dec 03 15:26:45 2015, hanenkamp wrote: > I've been doing some digging on this because it's basically holding up > my ability to test the code I'm working on. It appears that a thread > is being garbage collected, but it's aborting because > pthread_mutex_destroy is returning an error code when libuv attempts > to free up the thread resources. > > I suspect that the version of libuv used by MVM contains an old bug > that is hurting my particular program. I'm continuing to dig, but my C > is pretty rusty. It was a mutex being garbage collected (fine) but that was still locked (not fine). MoarVM now detects when that happens and panics with a more informative error. But - more helpfully - I believe I tracked down and fixed the case where this happens. The test in S17-supply/return-in-tap.t covers it (though I suspect there were quite a few more code paths that could lead to the same problem). Since the bug was filed, many other fixes have also been applied to supplies. Since this looks very much like something I fixed yesterday, but there's no reproduction details for me to verify it, I'll resolve this; feel free to re-open or re-file (with more details if possible) if this is still an issue. Thanks, /jnthn
[perl #125782] [CONC] Uncaught exceptions in start { } blocks get no backtrace when raised via await
On Mon Aug 10 14:20:34 2015, r...@hoelz.ro wrote: > See the attached script. > > The original backtrace of an exception seems to vanish when awaiting a > Promise throws that exception. Now we show both backtraces, and more clearly explain what happened (that we're throwing now because the result of a broken Promise was requested, followed by the original exception that broke the Promise and its location). Test in S17-promise/start.t. /jnthn
[perl #129781] [CONC] [SEGV] when start()ing several times in a loop
So, an update on this one.
The first set of ASAN barf (involving rope flattening) and SEGVs were fixed
last month by 5224878b2869. Now it can run for 10,000 iterations without
trouble. Well, until it can't.
Seems that the Cannot Invoke thingy happens about 1 time in a few hundred
thousand iterations:
jnthn@lviv:~/dev/rakudo$ seq 20 | xargs -Iz ./perl6-m -e 'await (for ^1 {
start ‘/etc/hostname’.IO ~~ :e })'
Cannot invoke this object (REPR: P6opaque; Scalar)
in block at -e line 1
jnthn@lviv:~/dev/rakudo$ seq 20 | xargs -Iz ./perl6-m -e 'await (for ^1 {
start ‘/etc/hostname’.IO ~~ :e })'
jnthn@lviv:~/dev/rakudo$ seq 20 | xargs -Iz ./perl6-m -e 'await (for ^1 {
start ‘/etc/hostname’.IO ~~ :e })'
jnthn@lviv:~/dev/rakudo$ seq 20 | xargs -Iz ./perl6-m -e 'await (for ^1 {
start ‘/etc/hostname’.IO ~~ :e })'
Cannot invoke this object (REPR: P6opaque; Scalar)
in block at -e line 1
Fun debugging awaits...
Finally, the last set of ASAN barfage is about global destruction (as mentioned
in the stack trace). This does want looking in to, but only happens when we
pass the --full-cleanup flag. Thus, it's not related to the original problem,
and won't happen when not running under valgrind (since we just leave cleanup
to the OS in that case, which can no doubt do it far faster than we can).
So, to resolve this issue we should:
1) Hunt down the cause of the rare invoke thing.
2) Add a spectest.
/jnthn
[perl #129787] [CONC] die/CATCH inside a start-block with a channel heisenbugs
On Sun Oct 02 12:52:45 2016, gfldex wrote:
> sub f(){
> my $c = Channel.new;
>
> start {
> for 1..* {
> CATCH { default { note .Str } }
>
> $c.send($_);
> die 'bad';
> }
> }
>
> $c.list
> }
>
> .say for f;
>
> # sometimes it works
> # sometimes it counts to 20 and outputs:
> # Command terminated
> # sometimes it outputs some numbers and 'bad's end then some
> # unicode-garbage
> # sometimes it outputs a few 100s 'bad' and then stalls with one thread at
> # 100% CPU
> # heisenbug rate is about 1/3
> #
> # also it seams to eat up plenty of RAM while it's running without the
> # die/CATCH
This one is yet another case of the long-standing issues with use of handles
across threads. The channel golfs away; you can get this bug with just:
start { for ^100 { note "hi" } }; for ^100 { say "oops" }
/jnthn
[perl #129994] [NATIVECALL][BUG] C functions required to be called once in one thread are hung up
On Tue Nov 01 09:01:04 2016, cookbook_...@yahoo.co.jp wrote:
> I'm creating a Perl 6 bindings for
> MeCab ( http://taku910.github.io/mecab/ )
>
> See the following codes and documentations(translations).
>
> My repository is:
> https://github.com/titsuki/p6-MeCab/tree/for-RT
>
>
> The following subtest in the t/02-lattice.t in the above repository is
> hung up.
>
> subtest {
> my MeCab::Model $model .= new;
> my @texts = (("私","僕") xx 3).flat;
>
> my @actual = (@texts.hyper(:batch(1))\
> .map(
> {
> my MeCab::Tagger $tagger = $model.create-
> tagger;
> my MeCab::Lattice $lattice =
> $model.create-lattice;
> $lattice.sentence($_);
> $lattice.tostr if
> $tagger.parse($lattice);
> $tagger.DESTROY;
> $lattice.DESTROY;
> }
> ).list);
>
> my Str $r1 = ("私\t名詞,代名詞,一般,*,*,*,私,ワタシ,ワタシ\nEOS\n");
> my Str $r2 = ("僕\t名詞,代名詞,一般,*,*,*,僕,ボク,ボク\nEOS\n");
> my @expected = (($r1, $r2) xx 3).flat;
> is @actual, @expected;
> }, "MeCab::Tagger should work in the multithread environment";
> ---
>
>
> In the MeCab documentation
> page(Japanese) https://taku910.github.io/mecab/libmecab.html
> "C++ サンプルコード" (en: C++ sample code) section says that:
> In the multithread environment (MeCab::Tagger, MeCab::Model,
> MeCab::Lattice)
>
> * Call MeCab::createModel() and create the Model object.
>
> * Call model->createTagger and create the Tagger object. The Tagger
> objects share the same model, even though you create the multiple
> models under the one model per thread constraint.
>
> * Call model->createLattice or MeCab::createLattice() and crate the
> Lattice object. The Lattice objects include all local variables for
> the morpheme analyze. Must keep the constraint that you can create
> only one object per thread.
>
> * model->swap(antoher_model) function replaces the Tagger object
> models created from the invocant model by another_model. This
> operation is thread-safe.
>
>
>
>
> I think I properly call the MeCab functions from Perl 6(e.g.
> $model.create-tagger, $model.create-lattice) according to the above
> instructions.
> So I think something is wrong in the NativeCall.
I think the hang was due to long-running native functions called on one thread
blocking GC (and thus progress) in all other threads. That is addressed in
https://github.com/MoarVM/MoarVM/commit/5c659178a07d86fe73b7c506c4ea7b10ad38b0e8
and
https://github.com/MoarVM/MoarVM/commit/f769569b78f099d59b7a17966d9096a0837c4d42
and tested in
https://github.com/rakudo/rakudo/commit/5ba75f445dc74560f9dabceeb4b4dc13f78c786f.
If you still have problems, please re-open this issue. Also, if possible, see
if you can find a smaller reproduction (for example, try just running the code
in a number of start blocks, or threads, instead of using `.hyper`).
Thanks,
/jnthn
[perl #129994] [NATIVECALL][BUG] C functions required to be called once in one thread are hung up
On Tue Nov 01 09:01:04 2016, cookbook_...@yahoo.co.jp wrote:
> I'm creating a Perl 6 bindings for
> MeCab ( http://taku910.github.io/mecab/ )
>
> See the following codes and documentations(translations).
>
> My repository is:
> https://github.com/titsuki/p6-MeCab/tree/for-RT
>
>
> The following subtest in the t/02-lattice.t in the above repository is
> hung up.
>
> subtest {
> my MeCab::Model $model .= new;
> my @texts = (("私","僕") xx 3).flat;
>
> my @actual = (@texts.hyper(:batch(1))\
> .map(
> {
> my MeCab::Tagger $tagger = $model.create-
> tagger;
> my MeCab::Lattice $lattice =
> $model.create-lattice;
> $lattice.sentence($_);
> $lattice.tostr if
> $tagger.parse($lattice);
> $tagger.DESTROY;
> $lattice.DESTROY;
> }
> ).list);
>
> my Str $r1 = ("私\t名詞,代名詞,一般,*,*,*,私,ワタシ,ワタシ\nEOS\n");
> my Str $r2 = ("僕\t名詞,代名詞,一般,*,*,*,僕,ボク,ボク\nEOS\n");
> my @expected = (($r1, $r2) xx 3).flat;
> is @actual, @expected;
> }, "MeCab::Tagger should work in the multithread environment";
> ---
>
>
> In the MeCab documentation
> page(Japanese) https://taku910.github.io/mecab/libmecab.html
> "C++ サンプルコード" (en: C++ sample code) section says that:
> In the multithread environment (MeCab::Tagger, MeCab::Model,
> MeCab::Lattice)
>
> * Call MeCab::createModel() and create the Model object.
>
> * Call model->createTagger and create the Tagger object. The Tagger
> objects share the same model, even though you create the multiple
> models under the one model per thread constraint.
>
> * Call model->createLattice or MeCab::createLattice() and crate the
> Lattice object. The Lattice objects include all local variables for
> the morpheme analyze. Must keep the constraint that you can create
> only one object per thread.
>
> * model->swap(antoher_model) function replaces the Tagger object
> models created from the invocant model by another_model. This
> operation is thread-safe.
>
>
>
>
> I think I properly call the MeCab functions from Perl 6(e.g.
> $model.create-tagger, $model.create-lattice) according to the above
> instructions.
> So I think something is wrong in the NativeCall.
I think the hang was due to long-running native functions called on one thread
blocking GC (and thus progress) in all other threads. That is addressed in
https://github.com/MoarVM/MoarVM/commit/5c659178a07d86fe73b7c506c4ea7b10ad38b0e8
and
https://github.com/MoarVM/MoarVM/commit/f769569b78f099d59b7a17966d9096a0837c4d42
and tested in
https://github.com/rakudo/rakudo/commit/5ba75f445dc74560f9dabceeb4b4dc13f78c786f.
If you still have problems, please re-open this issue. Also, if possible, see
if you can find a smaller reproduction (for example, try just running the code
in a number of start blocks, or threads, instead of using `.hyper`).
Thanks,
/jnthn
