Re: [racket-users] crash in nightly build
Matthew, More than a week has passed since I updated my installation of Racket. Good news: no crashes since the update. Most probably the bug that you fixed was causing the crash. Best regards, Dmitry On 10/31/2015 06:23 AM, Dmitry Pavlov wrote: Matthew, Before upgrading to the version when you fixed this (or possibly other) crash, I have tried for a long time to catch it under gdb or rr (Sam, thanks for the pointer to rr). Unfortunately, the program has never crashed under gdb or rr. I do not know why. I ran it many times, and it works like a Swiss watch. So I am going to upgrade the nightly build and let you know if the crash happens with it (not under gdb or rr). I hope that the bug you fixed is relevant to the crash, and then I should not see it again. My memory use often is in the interval 256-300 MB. And of course a lot of minor GC-s are happening. I am not sure about the condition with the finalizer: - whether it is necessary for the crash, - whether the crash happens right after the creation of finalizer or in the future, - whether my program actually creates finalizers after initialization. No code is suddenly called for the first time near the time of the crash. There is some DB access and some file system access, though. Anyway, I will install the latest nightly build and see how it goes. Best regards, Dmitry On 10/16/2015 01:16 AM, Matthew Flatt wrote: I've just stumbled across a subtle and difficult-to-provoke GC bug while testing new changes to the GC. The problem was introduced in commit 50df879e or so (September 10), so I think the timing of the change is consistent with your problem. For this bug to be relevant, your program would need to run a minor collection when memory use is in the neighborhood of 256MB to 300MB (which is not uncommon for 64-bit programs). The specific crash I can see also involves allocation a finalizer shortly before the GC, but the creation of a finalizer can be triggered in many ways, such as calling a bit of code for the first time so that it gets JIT-compiled. At Thu, 15 Oct 2015 11:56:21 +0300, Dmitry Pavlov wrote: Matthew, It seems that I have lost my grip on the crash. It has not happened for almost a week---neither with the version that I build myself, nor with the nightly build where I definitely saw it. I did not update Racket during this time. I did not do anything else. I have no idea why it is gone. Sorry. Oops, it just crashed again, 5+ hours of running. I will re-run in under gdb. There is no guarantee, though, that it will crash this time. By the way, I input "handle SIGSEGV nostop noprint" into gdb prior to running, to get rid of gdb's complaints about Racket VM's way of doing things. Am I correct that this should not hide the actual crash? Best regards, Dmitry -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] crash in nightly build
Matthew, Before upgrading to the version when you fixed this (or possibly other) crash, I have tried for a long time to catch it under gdb or rr (Sam, thanks for the pointer to rr). Unfortunately, the program has never crashed under gdb or rr. I do not know why. I ran it many times, and it works like a Swiss watch. So I am going to upgrade the nightly build and let you know if the crash happens with it (not under gdb or rr). I hope that the bug you fixed is relevant to the crash, and then I should not see it again. My memory use often is in the interval 256-300 MB. And of course a lot of minor GC-s are happening. I am not sure about the condition with the finalizer: - whether it is necessary for the crash, - whether the crash happens right after the creation of finalizer or in the future, - whether my program actually creates finalizers after initialization. No code is suddenly called for the first time near the time of the crash. There is some DB access and some file system access, though. Anyway, I will install the latest nightly build and see how it goes. Best regards, Dmitry On 10/16/2015 01:16 AM, Matthew Flatt wrote: I've just stumbled across a subtle and difficult-to-provoke GC bug while testing new changes to the GC. The problem was introduced in commit 50df879e or so (September 10), so I think the timing of the change is consistent with your problem. For this bug to be relevant, your program would need to run a minor collection when memory use is in the neighborhood of 256MB to 300MB (which is not uncommon for 64-bit programs). The specific crash I can see also involves allocation a finalizer shortly before the GC, but the creation of a finalizer can be triggered in many ways, such as calling a bit of code for the first time so that it gets JIT-compiled. At Thu, 15 Oct 2015 11:56:21 +0300, Dmitry Pavlov wrote: Matthew, It seems that I have lost my grip on the crash. It has not happened for almost a week---neither with the version that I build myself, nor with the nightly build where I definitely saw it. I did not update Racket during this time. I did not do anything else. I have no idea why it is gone. Sorry. Oops, it just crashed again, 5+ hours of running. I will re-run in under gdb. There is no guarantee, though, that it will crash this time. By the way, I input "handle SIGSEGV nostop noprint" into gdb prior to running, to get rid of gdb's complaints about Racket VM's way of doing things. Am I correct that this should not hide the actual crash? Best regards, Dmitry -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] crash in nightly build
At Thu, 15 Oct 2015 11:56:21 +0300, Dmitry Pavlov wrote: > Oops, it just crashed again, 5+ hours of running. > I will re-run in under gdb. There is no guarantee, though, > that it will crash this time. > > By the way, I input "handle SIGSEGV nostop noprint" into gdb prior > to running, to get rid of gdb's complaints about Racket VM's way of > doing things. Am I correct that this should not hide the actual crash? Yes, that's correct. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] crash in nightly build
I've just stumbled across a subtle and difficult-to-provoke GC bug while testing new changes to the GC. The problem was introduced in commit 50df879e or so (September 10), so I think the timing of the change is consistent with your problem. For this bug to be relevant, your program would need to run a minor collection when memory use is in the neighborhood of 256MB to 300MB (which is not uncommon for 64-bit programs). The specific crash I can see also involves allocation a finalizer shortly before the GC, but the creation of a finalizer can be triggered in many ways, such as calling a bit of code for the first time so that it gets JIT-compiled. At Thu, 15 Oct 2015 11:56:21 +0300, Dmitry Pavlov wrote: > Matthew, > > > It seems that I have lost my grip on the crash. > > It has not happened for almost a week---neither with the version > > that I build myself, nor with the nightly build where I definitely > > saw it. I did not update Racket during this time. > > I did not do anything else. I have no idea why it is gone. Sorry. > > Oops, it just crashed again, 5+ hours of running. > I will re-run in under gdb. There is no guarantee, though, > that it will crash this time. > > By the way, I input "handle SIGSEGV nostop noprint" into gdb prior > to running, to get rid of gdb's complaints about Racket VM's way of > doing things. Am I correct that this should not hide the actual crash? > > Best regards, > > Dmitry > > -- > You received this message because you are subscribed to the Google Groups > "Racket Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to racket-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] crash in nightly build
At Mon, 5 Oct 2015 08:32:00 -0600, Matthew Flatt wrote: > You could configure with `--enable-strip` That should have been `--disable-strip`. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] crash in nightly build
Matthew, Sorry I am a bit late on this. It is good to know that you keep track on crash reports even as vague as mine. Here is an update: -- My crash remains with the nightly build that I installed on Saturday (Oct 3); -- I do not use places; -- I do not use any new advanced macro facilities (not directly at least); -- On the previous build, I tried fixing AGE_GEN_0_TO_GEN_HALF to 1 as you suggested. It did not provoke faster crash. I have not yet tried your rewind-gc branch. I am going to get you a stack trace. Am I correct that in order to get the stack trace, I have to rebuild Racket with -G and run it in gdb? Here is another interesting thing that I was able to notice. As soon as I purged the "compiled" directories from the code tree, the crashes ceased. I did several consecutive successful runs. I ran just "racket myprogram", without "raco make" or DrRacket. Then I executed "raco make myprogram" -- and the very next run has crashed after ~7h of running. How do you think, does it make sense that stored bytecode can cause the crash or is it just a coincidence? Best regards, Dmitry On 10/03/2015 03:27 PM, Matthew Flatt wrote: I imagine that the "rewind-gc" branch was no help at all. At the time of your post, there were several open crash reports; I've now resolved all of them except yours, and none of the others had to do with GC changes. (One was related to the new macro system, and two were old bugs in places.) If you can get any sort of crash report with a stack trace, that information is often enough for me to either track down the problem or at least say something about what parts of the example may be relevant. Thanks, Matthew At Mon, 21 Sep 2015 00:08:43 +0300, Dmitry Pavlov wrote: Hello, I was just able to discover that my program crashes on the nightly build 6.2.900.17, on Linux. The program has a heavy portion of math, double and extended unsafe ops, and accesses multiple low-level libraries. It has crashed after 4 hours of running, while doing just the same that it had been doing for those 4 hours. SIGSEGV MAPERR si_code 1 fault on addr 0x7f9f0d16a008 The previous version of Racket that I used for a long time (and saw no crashes with it) is 6.2.0.3. Before I go into deeper analysis by trial and error, maybe the developers (I guess it would be Matthew) could give any insight on what has been done since 6.2.0.3 that could result in the crash? How to stress the VM to wait less till the crash occurs? Best regards, Dmitry -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] crash in nightly build
At Mon, 05 Oct 2015 17:16:34 +0300, Dmitry Pavlov wrote: > I am going to get you a stack trace. > Am I correct that in order to get the stack trace, I have to rebuild > Racket with -G and run it in gdb? Racket builds with `-g` by default, but `make install` uses `strip`. You could configure with `--enable-strip`, or just run "build/racket/racket3m" directly. You might even get a useful stack trace running the stripped version, since many function names are preserved for access via dlopen(). > Here is another interesting thing that I was able to notice. > As soon as I purged the "compiled" directories from the code tree, > the crashes ceased. I did several consecutive successful runs. > I ran just "racket myprogram", without "raco make" or DrRacket. > Then I executed "raco make myprogram" -- and the very next run > has crashed after ~7h of running. > > How do you think, does it make sense that stored bytecode can > cause the crash or is it just a coincidence? Yes, that's possible; the bytecode format is one thing that changed with the new expander. The fixed bug related to the new expander had to do with bytecode, but that one was about interrupting a bytecode load. You might check whether the `-d` flag has any effect. That flag disables on-demand bytecode loading. Another possibility is that loading from source merely shifts allocation patterns to mask an unrelated bug, but this directory sound more promising than that, so far. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] crash in nightly build
To try without the recent GC changes, you could build the "rewind-gc" branch of https://github.com/mflatt/racket At Mon, 21 Sep 2015 00:08:43 +0300, Dmitry Pavlov wrote: > Hello, > > I was just able to discover that my program crashes on the > nightly build 6.2.900.17, on Linux. The program has a heavy portion > of math, double and extended unsafe ops, and accesses multiple > low-level libraries. It has crashed after 4 hours of running, > while doing just the same that it had been doing for those > 4 hours. > > SIGSEGV MAPERR si_code 1 fault on addr 0x7f9f0d16a008 > > The previous version of Racket that I used for a long time > (and saw no crashes with it) is 6.2.0.3. > > Before I go into deeper analysis by trial and error, maybe > the developers (I guess it would be Matthew) could give any > insight on what has been done since 6.2.0.3 that could result > in the crash? How to stress the VM to wait less till the crash > occurs? > > > Best regards, > > Dmitry > > -- > You received this message because you are subscribed to the Google Groups > "Racket Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to racket-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
