Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-29 Thread Romain Beauxis
![giphy](https://user-images.githubusercontent.com/871060/44796344-6a217880-ab72-11e8-9668-914f4679aee0.gif) -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-29 Thread Gilou
Closed #609. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/savonet/liquidsoap/issues/609#event-1815796144-- Check out the

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-29 Thread Gilou
``` uptime 0d 23h 37m 23s ``` Let's call this a proper fix, and another victory for M. Beauxis   -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-29 Thread Gilou
you're killing me ;) Thanks, I'll let it run a bit longer and as I'm such a daredevil, I added a cue_cut() on my lists and testing that along. I'm running on a CPU craziness (will make an issue for it should I spot it more precisely) as well using replaygain extraction, does it use the same

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-29 Thread Romain Beauxis
Told you that fix was legit ;-) -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-29 Thread Gilou
``` uptime 0d 20h 57m 43s ``` Still streaming.. Maybe I can try going back to pipe() ;) -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-28 Thread Gilou
``` 2018/08/28 16:43:30 >>> LOG START 2018/08/28 16:43:30 [main:3] Liquidsoap 1.3.3+scm (https://github.com/savonet/liquidsoap.git@fd1cebd292824ca9f43d932e17c236c9aa674dc7:20180828:164057) ``` Will follow up.. -- You are receiving this because you are subscribed to this thread.

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-28 Thread Romain Beauxis
Thank you. Let me know. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/savonet/liquidsoap/issues/609#issuecomment-416611300--

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-28 Thread Gilou
so I was running the proper branch, I'll compile it myself to keep track of that.. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-28 Thread Gilou
ah OK.. opam sync the rep on a build directory... so no .git dir to get the info... -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-28 Thread Gilou
there is something fishy with how SVN.ml (sic) isn't working properly here... -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-28 Thread Romain Beauxis
@gilou, can we double check that you're indeed on the branch that I pushed? Just want to make sure, that fix really seems legit.. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-28 Thread Romain Beauxis
Hmm weird. I should be displayed. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-28 Thread Gilou
I hope you are suggesting the format rather than stating what should be displayed currently.. I have: ``` 2018/08/28 15:27:49 [main:3] Liquidsoap 1.3.3+scm 2018/08/28 15:27:49 [main:3] Using: bytes=[distributed with OCaml 4.02 or above] pcre=7.3.4 dtools=0.4.0 duppy=0.7.1 duppy.syntax=0.7.1

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-28 Thread Gilou
that was fast.. log froze at smart_crossfade as well in ~5 minutes, bt included.. [gdb.txt](https://github.com/savonet/liquidsoap/files/2328263/gdb.txt) -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-28 Thread Romain Beauxis
Should be: ``` 2018/08/28 08:03:43 [main:3] Liquidsoap 1.3.3+scm (g...@github.com:savonet/liquidsoap.git@f9970afe6a89a38924084b1439c65685b6a65062:20180826:143248) ``` ;-) -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-28 Thread Gilou
Here we go. Side note, maybe having the rev/branch included in the version might help knowing where we are ☺ (yes, I'll try to do a PR for configure.ac ) -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-28 Thread Romain Beauxis
Nah, can you actually try with the branch `deadlock-investigation` that I just pushed? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-28 Thread Romain Beauxis
Here's what I've seen twice now from your backtrace: ``` Thread 8 (Thread 0x7f80f7fff700 (LWP 10561)): #0 __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135 #1 0x7f8123031023 in __GI___pthread_mutex_lock (mutex=0x55a73a86d8a0) at ../nptl/pthread_mutex_lock.c:78 #2

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-28 Thread Gilou
Do you want me to test without smart_crossfade? I'll probably have to rewrite it at some point to have it deal more agressively with silences at some point so ..  -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-28 Thread Romain Beauxis
![giphy 2](https://user-images.githubusercontent.com/871060/44723380-6b30a800-aa95-11e8-869e-bfca4c914f96.gif) -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-28 Thread Gilou
Browsing through the log, I get a weird feeling.. I have things like: `2018/08/28 11:00:08 [smart_cross_9081:3] Analysis: -nandB / -24.523810dB (3.04s / 3.04s)` So I'm adding a few lines before the freeze: ``` 2018/08/28 11:33:46 [decoder:4] Decoding "/home/ubuntu/demo/chill/08 On the

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-28 Thread Gilou
And.. it failed again  after 20 hours, and it was the whole thing, pipping to stereotool This is what the backtrace is once attached: [gdb.txt](https://github.com/savonet/liquidsoap/files/2327670/gdb.txt) Same symptoms: server still responding though quite braindead: ``` uptime 0d 21h 00m 40s

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-27 Thread Romain Beauxis
![giphy 1](https://user-images.githubusercontent.com/871060/44698132-5546c700-aa44-11e8-820c-ce7d27abb45c.gif) -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-27 Thread Gilou
 10 hours uptime mark! I'll let it run a bit more before opening the champagne, and add a bit more complexity I removed to debunk this. I'll avoid adding cue_cut or other extreme gapless playout ideas I have in mind before the release (and I'll see how I can help with docs as well)... -- You

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-27 Thread Romain Beauxis
Great idea. Your stack trace was helpful too and seems to point to a different direction but this is worth trying. Thks! -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-27 Thread Gilou
Will restart testing on latest code, namely with e10e047 -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-26 Thread Gilou
I'm back on sox for testing... If I only use output.external, it lasted for more than 4 hours which was a first... so I added pipe_metadata from https://gist.github.com/gilou/43133d2619202791e153211471aff6a1#file-functions-liq-L115 but it failed after 10+ hours... Code was: ``` s =

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-24 Thread Romain Beauxis
@gilou, thanks for reporting this. You're on the latest code right? It seems that the streaming thread is blocked but the other treads are still responding. One thing you can try is to get a `gdb` backtrace of the running process: ``` % gdb > attach > thread apply all bt ``` -- You are

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-23 Thread Gilou
Closed #609. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/savonet/liquidsoap/issues/609#event-1804986049-- Check out the

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-23 Thread Gilou
Reopened #609. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/savonet/liquidsoap/issues/609#event-1804986094-- Check out the

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-23 Thread Gilou
aand it froze again, after 5 hours.. Same symptoms, liq is somehow frozen, stills responds on its socket but doesn't actually handle anything, yet it responds to SIGUSR1 (truncates the logfile) and closes on other signals (ABRT, HUP..). -- You are receiving this because you are

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-22 Thread Gilou
and interestingly (or not), liq honored USR1 in that state, because it truncated the log file upon receiving! -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-22 Thread Gilou
if I attach strace to stereotool, it seems to be alive and waiting for data... `[pid 30093] read(0, ` -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-22 Thread Gilou
I removed all outputs but the one streaming the piped stream, as I suspect it's the culprit, it froze again after 1h40, smart_crossfade managed to squeeze one more line: `2018/08/22 12:06:31 [smart_crossfade:3] new >= old + margin, old <= medium and new <= high.` This time I had a socket

[Savonet-users] [savonet/liquidsoap] Liquidsoap freezes on a complex script (#609)

2018-08-22 Thread Gilou
Hi, So running most components from master (cry, dtools, duppy, faad, mad, mm). Thing is, that script is quite complex, but if I remove each part, the bug is really hard to reproduce... All parts have been tested stand alone though. The script combines: - playlist()s with a custom check_next