Re: [Chicken-users] Build Failure (maybe infinite recursion) on x86_64 Linux
On Mar 23, 2009, at 10:46 PM, Alex Shinn wrote: felix winkelmann bunny...@gmail.com writes: It seems that one of the regexen's compiled in the definitions for `absolute-pathname?' or `decompose-pathname' triggers a loop in `nfa-join-transitions!' in the files.scm unit (these are executed at toplevel, outside of the scope of the associated lambdas). Yes, but that would happen at load time, not compile time, so I guess some other macro is using the files unit at compile time, thus loading irregex? The chicken compiler uses Unit files. Anyway, I tried compiling all four of the patterns occurring in those two procedures with both the irregex.scm in Chicken and with the latest upstream, and couldn't trigger an infinite loop. Me too. I'm also puzzled as to how this can be behaving differently between Chicken 3 and 4 when the irregex.scm file is identical. Me too. -- Alex ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users Best Wishes, Kon ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Build Failure (maybe infinite recursion) on x86_64 Linux
Kon Lovett wrote: Alex Shinn wrote: I'm also puzzled as to how this can be behaving differently between Chicken 3 and 4 when the irregex.scm file is identical. Me too. I installed Ubuntu 8.10 x86_64 in a VMWare and tried compiling several Chicken releases, but I couldn't reproduce the bug. Maybe I'd need to automate varying the -:s parameter until it gets stuck, as Taylor has been doing. Or maybe it's something specific to his machine or software stack. Tobia ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Build Failure (maybe infinite recursion) on x86_64 Linux
On Tue, Mar 24, 2009 at 7:46 AM, Kon Lovett klov...@pacbell.net wrote: The chicken compiler uses Unit files. Yes, exactly. I'm also puzzled as to how this can be behaving differently between Chicken 3 and 4 when the irregex.scm file is identical. It might be some glue code in regex.scm. It might be something completely different, of course. They is a inifinite number of ways things can break. Can someone give me temporary access to a 64-bit machine (linux, if possible) so that I can try to reproduce the problem? cheers, felix ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Build Failure (maybe infinite recursion) on x86_64 Linux
Taylor Venable wrote: When building Chicken 3.4.7 and higher (up to 3.5.2) on Ubuntu 8.10 x86_64 the chicken compiler goes into an infinite recursion. Excuse my ignorance, how do I check out a version such as 3.4.7 or 3.5.2? I cannot find the relevant tag or branch. Should I look for a log message and checkout a specific release? If so, in which branch? Tobia ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Build Failure (maybe infinite recursion) on x86_64 Linux
On Mon, Mar 23, 2009 at 09:51:08AM +0100, Tobia Conforto wrote: Taylor Venable wrote: When building Chicken 3.4.7 and higher (up to 3.5.2) on Ubuntu 8.10 x86_64 the chicken compiler goes into an infinite recursion. Excuse my ignorance, how do I check out a version such as 3.4.7 or 3.5.2? I cannot find the relevant tag or branch. Should I look for a log message and checkout a specific release? If so, in which branch? They're in the development snapshots area which is linked from the releases page: http://chicken.wiki.br/dev-snapshots/ -- Taylor Christopher Venable http://real.metasyntax.net:2357/ ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Build Failure (maybe infinite recursion) on x86_64 Linux
On Mon, Mar 23, 2009 at 08:48:50AM +0900, Ivan Raikov wrote: I doubt this is the case, since the regex unit is almost identical between Chicken 3 and Chicken 4. Taylor, can you compile Chicken with the attached regex.scm and see if there is any routine from the regex unit that is called at the point when the build process gets stuck? I can trigger the bug using Chicken 3.4.7 when compiled with the supplied regex.scm file (again I can do this by varying the -:s parameter until it gets stuck). A backtrace from GCC is attached (it continues to repeat through many frames after the output given). It exhibits behaviour similar to that when using the regex.scm that comes with Chicken 3.4.7 normally. The regex.c file which is generated is on my website at: http://real.metasyntax.net:2357/tmp/regex-pcre.c.gz (Probably just my ignorance talking here again, but even though the header on the file says PCRE, there are a lot of references to irregex scattered throughout that file.) Thanks for the continued time to try to solve this, -- Taylor Christopher Venable http://real.metasyntax.net:2357/ (gdb) run utils.scm -quiet -no-trace -optimize-level 2 -include-path . -include-path ./ -explicit-use -output-file utils.c -:s74k -:d The program being debugged has been started already. Start it from the beginning? (y or n) y Starting program: /home/taylor/Desktop/chicken-3.4.7/chicken utils.scm -quiet -no-trace -optimize-level 2 -include-path . -include-path ./ -explicit-use -output-file utils.c -:s74k -:d [debug] application startup... [debug] heap resized to 50 bytes [debug] stack bottom is 0x7fff909c2ae0. [debug] entering toplevel toplevel... [debug] entering toplevel library_toplevel... [debug] entering toplevel eval_toplevel... [debug] entering toplevel data_structures_toplevel... [debug] entering toplevel ports_toplevel... [debug] entering toplevel extras_toplevel... [debug] entering toplevel srfi_69_toplevel... [debug] entering toplevel srfi_1_toplevel... [debug] entering toplevel match_toplevel... [debug] entering toplevel srfi_4_toplevel... [debug] entering toplevel utils_toplevel... [debug] entering toplevel regex_toplevel... [debug] entering toplevel files_toplevel... [debug] resizing heap dynamically from 500k to 1000k ... [debug] resizing heap dynamically from 1000k to 2000k ... [debug] resizing heap dynamically from 2000k to 4000k ... [debug] resizing heap dynamically from 4000k to 8000k ... [debug] resizing heap dynamically from 8000k to 16000k ... [debug] resizing heap dynamically from 16000k to 32000k ... [debug] resizing heap dynamically from 32000k to 64000k ... [debug] resizing heap dynamically from 64000k to 128000k ... [debug] resizing heap dynamically from 128000k to 256000k ... [debug] resizing heap dynamically from 256000k to 512000k ... ^C Program received signal SIGINT, Interrupt. 0x7f9c883adfcf in f_12537 (c=2, t0=140735619528656, t1=14680074) at regex.c:10326 10326 ((C_proc3)(void*)(*((C_word*)t3+1)))(3,t3,t2,((C_word*)t0)[2]);} (gdb) bt #0 0x7f9c883adfcf in f_12537 (c=2, t0=140735619528656, t1=14680074) at regex.c:10326 #1 0x7f9c881677e2 in f_3920 (c=3, t0=140309752394536, t1=140735619528656, t2=140309752479688) at library.c:34139 #2 0x7f9c883aded8 in f_12517 (t0=140735619529008, t1=6) at regex.c:10312 #3 0x7f9c883add3a in f_12551 (c=2, t0=140735619529056, t1=14680074) at regex.c:10287 #4 0x7f9c881677e2 in f_3920 (c=3, t0=140309752394536, t1=140735619529056, t2=140309752479688) at library.c:34139 #5 0x7f9c883adbe6 in f_12555 (c=2, t0=140735619529352, t1=140735619533232) at regex.c:10267 #6 0x7f9c88167a01 in f_3890 (c=3, t0=140309752393464, t1=140735619529352, t2=140309752479688) at library.c:34178 #7 0x7f9c883ad9e5 in f_12360 (t0=140735619529888, t1=6) at regex.c:10251 #8 0x7f9c883ad450 in f_12579 (c=2, t0=140735619529976, t1=140735619533232) at regex.c:10198 #9 0x7f9c88167a01 in f_3890 (c=3, t0=140309752393464, t1=140735619529976, t2=140309752479688) at library.c:34178 #10 0x7f9c883ad2f4 in f_12332 (t0=140735619530240, t1=140735619531152, t2=140309752479688, t3=14) at regex.c:10181 #11 0x7f9c883acfc3 in f_12327 (c=2, t0=140735619530512, t1=14679818) at regex.c:10156 #12 0x7f9c8816794c in f_3900 (c=3, t0=140309752393432, t1=140735619530512, t2=140735619531040) at library.c:34165 #13 0x7f9c883acdfe in f_12324 (c=2, t0=140735619530776, t1=10) at regex.c:10140 #14 0x7f9c88167a01 in f_3890 (c=3, t0=140309752393464, t1=140735619530776, t2=140735619531040) at library.c:34178 #15 0x7f9c883accc1 in f_12123 (t1=140735619531152, t2=140309752479688, t3=140735619531040) at regex.c:10127 #16 0x7f9c883b0994 in f_12126 (t1=140735619531152, t2=140309752479688, t3=140735619533728, t4=29) at regex.c:10829 #17 0x7f9c883aed75 in f_12439 (t0=140735619531280, t1=140309752479688) at regex.c:10468 #18 0x7f9c883aec5d in f_12436 (t0=140735619531488, t1=140309752479688) at regex.c:10454
Re: [Chicken-users] Build Failure (maybe infinite recursion) on x86_64 Linux
On Mon, Mar 23, 2009 at 6:20 PM, Taylor Venable tay...@metasyntax.net wrote: On Mon, Mar 23, 2009 at 08:48:50AM +0900, Ivan Raikov wrote: I doubt this is the case, since the regex unit is almost identical between Chicken 3 and Chicken 4. Taylor, can you compile Chicken with the attached regex.scm and see if there is any routine from the regex unit that is called at the point when the build process gets stuck? I can trigger the bug using Chicken 3.4.7 when compiled with the supplied regex.scm file (again I can do this by varying the -:s parameter until it gets stuck). A backtrace from GCC is attached (it continues to repeat through many frames after the output given). It exhibits behaviour similar to that when using the regex.scm that comes with Chicken 3.4.7 normally. The regex.c file which is generated is on my website at: http://real.metasyntax.net:2357/tmp/regex-pcre.c.gz (Probably just my ignorance talking here again, but even though the header on the file says PCRE, there are a lot of references to irregex scattered throughout that file.) It seems that one of the regexen's compiled in the definitions for `absolute-pathname?' or `decompose-pathname' triggers a loop in `nfa-join-transitions!' in the files.scm unit (these are executed at toplevel, outside of the scope of the associated lambdas). cheers, felix ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Build Failure (maybe infinite recursion) on x86_64 Linux
felix winkelmann bunny...@gmail.com writes: It seems that one of the regexen's compiled in the definitions for `absolute-pathname?' or `decompose-pathname' triggers a loop in `nfa-join-transitions!' in the files.scm unit (these are executed at toplevel, outside of the scope of the associated lambdas). Yes, but that would happen at load time, not compile time, so I guess some other macro is using the files unit at compile time, thus loading irregex? Anyway, I tried compiling all four of the patterns occurring in those two procedures with both the irregex.scm in Chicken and with the latest upstream, and couldn't trigger an infinite loop. I'm also puzzled as to how this can be behaving differently between Chicken 3 and 4 when the irregex.scm file is identical. -- Alex ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Build Failure (maybe infinite recursion) on x86_64 Linux
I doubt this is the case, since the regex unit is almost identical between Chicken 3 and Chicken 4. Taylor, can you compile Chicken with the attached regex.scm and see if there is any routine from the regex unit that is called at the point when the build process gets stuck? Thanks, -Ivan regex.scm - Unit for using the PCRE regex package ; ; Copyright (c) 2000-2007, Felix L. Winkelmann ; Copyright (c) 2008, The Chicken Team ; All rights reserved. ; ; Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following ; conditions are met: ; ; Redistributions of source code must retain the above copyright notice, this list of conditions and the following ; disclaimer. ; Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following ; disclaimer in the documentation and/or other materials provided with the distribution. ; Neither the name of the author nor the names of its contributors may be used to endorse or promote ; products derived from this software without specific prior written permission. ; ; THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS AS IS AND ANY EXPRESS ; OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY ; AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR ; CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR ; CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR ; SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY ; THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR ; OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE ; POSSIBILITY OF SUCH DAMAGE. (cond-expand [chicken-compile-shared] [else (declare (unit regex))] ) (declare (usual-integrations) (disable-interrupts) (generic) (disable-warning var) (bound-to-procedure get-output-string open-output-string string-list list-string string-length string-ref substring make-string string-append reverse list-ref char=? char-alphabetic? char-numeric? char-integer ##sys#size ##sys#error ##sys#fragments-string ##sys#write-char-0 ) (export regexp? regexp string-match string-match-positions string-search string-search-positions string-split-fields string-substitute string-substitute* glob? glob-regexp grep regexp-escape irregex string-irregex sre-irregex string-sre irregex? irregex-match-data? irregex-new-matches irregex-reset-matches! irregex-match-start irregex-match-end irregex-match-substring irregex-match-num-submatches irregex-search irregex-search/matches irregex-match irregex-match-string irregex-replace irregex-replace/all irregex-dfa irregex-dfa/search irregex-dfa/extract irregex-nfa irregex-flags irregex-submatches irregex-lengths irregex-names ) ) (cond-expand [paranoia] [else (declare (no-bound-checks) (no-procedure-checks-for-usual-bindings) ) ] ) (cond-expand [unsafe (eval-when (compile) (define-macro (##sys#check-integer . _) '(##core#undefined)) (define-macro (##sys#check-blob . _) '(##core#undefined)) (define-macro (##sys#check-vector . _) '(##core#undefined)) (define-macro (##sys#check-structure . _) '(##core#undefined)) (define-macro (##sys#check-range . _) '(##core#undefined)) (define-macro (##sys#check-pair . _) '(##core#undefined)) (define-macro (##sys#check-list . _) '(##core#undefined)) (define-macro (##sys#check-symbol . _) '(##core#undefined)) (define-macro (##sys#check-string . _) '(##core#undefined)) (define-macro (##sys#check-char . _) '(##core#undefined)) (define-macro (##sys#check-exact . _) '(##core#undefined)) (define-macro (##sys#check-port . _) '(##core#undefined)) (define-macro (##sys#check-number . _) '(##core#undefined)) ) ] [else (declare (bound-to-procedure ;; Imports ##sys#check-string ##sys#check-list ##sys#check-exact ##sys#check-vector ##sys#check-structure ##sys#check-symbol ##sys#check-blob ##sys#check-integer ) (emit-exports regex.exports) ) ] ) (register-feature! 'regex 'irregex) (include irregex.scm) (define-record regexp x) (define (regexp pat #!optional caseless extended utf8) (make-regexp (apply irregex pat (let ((opts '())) (when caseless (set! opts (cons 'i opts))) (when extended (set! opts (cons 'x opts))) (when utf8 (set! opts (cons 'utf8 opts))) opts))) ) (define (unregexp x) (cond ((regexp? x) (regexp-x x)) ((irregex? x) x) (else (irregex x (define (string-match rx str . rest) (print begin string-match) (let ((rx (unregexp rx))) (and-let* ((m (irregex-match rx str))) (let loop ((i
Re: [Chicken-users] Build Failure (maybe infinite recursion) on x86_64 Linux
On Wed, Mar 18, 2009 at 6:55 PM, Taylor Venable tay...@metasyntax.net wrote: On Wed, Mar 18, 2009 at 10:38:30AM +0100, felix winkelmann wrote: Could you try to build chicken 4? (possibly twice, building the compiler with the compiler built from the bootstrap tarball - so that you are really testing the newest version and not some stale code in the bootstrapping tarball). Chicken 4 builds just fine, and I cannot get the bug to appear there even with a variance of -:s options. That means the irregex integration must be buggy. Time for the chicken-3 maintainers to take over... cheers, felix ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Build Failure (maybe infinite recursion) on x86_64 Linux
Hi! I think you're right: these are two cases: the 10k stack is already too small on a 64-bit machine and will continuously GC. The other case is the one we're looking for. It may be that the irregex integration has a bug. Could you try to build chicken 4? (possibly twice, building the compiler with the compiler built from the bootstrap tarball - so that you are really testing the newest version and not some stale code in the bootstrapping tarball). cheers, felix ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Build Failure (maybe infinite recursion) on x86_64 Linux
On Wed, Mar 18, 2009 at 10:38:30AM +0100, felix winkelmann wrote: Could you try to build chicken 4? (possibly twice, building the compiler with the compiler built from the bootstrap tarball - so that you are really testing the newest version and not some stale code in the bootstrapping tarball). Chicken 4 builds just fine, and I cannot get the bug to appear there even with a variance of -:s options. -- Taylor Christopher Venable http://real.metasyntax.net:2357/ ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Build Failure (maybe infinite recursion) on x86_64 Linux
On Mon, Mar 16, 2009 at 09:36:28PM -0400, Taylor Venable wrote: Since regex.c seems heavily involved in this last case, I've also put it on my website at: http://real.metasyntax.net:2357/tmp/regex.c.gz Looking back at the changes log for the devel releases, I just noticed that the first version that I have this problem in is 3.4.7 which is when irregex replaced PCRE. -- Taylor Christopher Venable http://real.metasyntax.net:2357/ ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Build Failure (maybe infinite recursion) on x86_64 Linux
On Sun, Mar 15, 2009 at 12:31:21PM +0100, felix winkelmann wrote:
On Wed, Mar 11, 2009 at 7:25 PM, Taylor Venable tay...@metasyntax.net wrote:
Well I was able to trigger a hang in my build with the debugging
symbols by setting the nursery stack size to 10k. ?Debug log from GDB
is attached. ?The three entries here were taken from allowing the
program to run for varying periods of time. ?The behaviour I got
spiked the CPU but the memory usage did not increase visibly. ?Hope
this sheds a little more light; if not let me know and I can go back
at it. ?(I notice here that the proc parameter to C_reclaim is the
null pointer, don't know if that's meaningful or not...)
The proc being 0 is ok. Apparently the GC looping: it might be that
some sort of stack-limit check is constantly failing which results in
one GC after the other. It would be cool, if you could step out of
the GC (reclaim) and follow the execution - does it trigger a new
GC right away?
My testing has all been done using Chicken 3.4.7 so line numbers
reference source code from that devel release.
Usually when it hangs, I end up sending it ^C when it's inside mark().
So if I step out to C_reclaim() I can step through from line 2802 (or
sometimes 2805; in runtime.c) to line 3016. This I do in GDB using
the next command. When I hit line 3016 which is
C_longjmp(C_restart, 1);
the program hangs again. If I send it ^C again I'm back in that same
region of code again inside C_reclaim. This happens infinitely, as
far as I can tell.
And tracing a bit further, it looks like C_library_toplevel() is
continuously calling C_reclaim(). The code that looks like (library.c
line 6616):
if(!C_demand(931)){
C_save(t1);
C_reclaim((void*)toplevel_trampoline,NULL);}
keeps getting run; this is what's calling the C_reclaim() over and
over again. In fact, it seems that execution never makes it out of
this loop; just these three lines ad infinitum.
I found that the result of C_demand is based on C_stack_pointer and
C_stack_limit. I was unable to determine the former using GDB, but in
this case C_stack_limit is always 0x7FFF83326C30.
Hope this helps. I've put my library.c on my website, since it seems
to be generated code: http://real.metasyntax.net:2357/tmp/library.c.gz
--
Taylor Christopher Venable
http://real.metasyntax.net:2357/
___
Chicken-users mailing list
Chicken-users@nongnu.org
http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Build Failure (maybe infinite recursion) on x86_64 Linux
On Mon, Mar 16, 2009 at 2:32 PM, Taylor Venable tay...@metasyntax.net wrote:
Usually when it hangs, I end up sending it ^C when it's inside mark().
So if I step out to C_reclaim() I can step through from line 2802 (or
sometimes 2805; in runtime.c) to line 3016. This I do in GDB using
the next command. When I hit line 3016 which is
C_longjmp(C_restart, 1);
the program hangs again. If I send it ^C again I'm back in that same
region of code again inside C_reclaim. This happens infinitely, as
far as I can tell.
Right. The GC seems to loop.
And tracing a bit further, it looks like C_library_toplevel() is
continuously calling C_reclaim(). The code that looks like (library.c
line 6616):
if(!C_demand(931)){
C_save(t1);
C_reclaim((void*)toplevel_trampoline,NULL);}
keeps getting run; this is what's calling the C_reclaim() over and
over again. In fact, it seems that execution never makes it out of
this loop; just these three lines ad infinitum.
I found that the result of C_demand is based on C_stack_pointer and
C_stack_limit. I was unable to determine the former using GDB, but in
this case C_stack_limit is always 0x7FFF83326C30.
Hope this helps. I've put my library.c on my website, since it seems
to be generated code: http://real.metasyntax.net:2357/tmp/library.c.gz
Well done, Taylor. Chicken coalesces all allocations inside a single
generated C procedure and allocates it as one chunk. It seems here
that the stack is already too small to hold the data, even after a minor
GC was invoked. Can you reproduce the hang with larger settings for
-:s...? If you add -:d, you should see messages like
[debug] stack resized to bytes
If you can't reproduce the hang with higher settings, change the makefile
(defaults.make or rules.make and pass -:sgoodvalue to the chicken-boot
invocations and look how far it gets (BTW, what does ./chicken-boot -:d show?)
Then we probably have a precompiled bootstrapping compiler with a too
low nursery setting, even though this used to work. It also may be something
completely different.
cheers,
felix
___
Chicken-users mailing list
Chicken-users@nongnu.org
http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Build Failure (maybe infinite recursion) on x86_64 Linux
On Mon, Mar 16, 2009 at 09:09:41PM +0100, felix winkelmann wrote: Well done, Taylor. Chicken coalesces all allocations inside a single generated C procedure and allocates it as one chunk. It seems here that the stack is already too small to hold the data, even after a minor GC was invoked. Can you reproduce the hang with larger settings for -:s...? If you add -:d, you should see messages like [debug] stack resized to bytes If you can't reproduce the hang with higher settings, change the makefile (defaults.make or rules.make and pass -:sgoodvalue to the chicken-boot invocations and look how far it gets (BTW, what does ./chicken-boot -:d show?) Then we probably have a precompiled bootstrapping compiler with a too low nursery setting, even though this used to work. It also may be something completely different. Yes, now that I've tried these things I think that what I've run into during the build process initially is different from the small stack size that I've been using for this. Or in other words I think there may be two separate problems here, or at least I've figured out how to get the program to hang in two different conditions. First, here is the output with -:d for what proceeds normally. Note that with this build (done with make PLATFORM=linux DEBUG_BUILD=yes bootstrap) the original scenario works. [debug] application startup... [debug] heap resized to 50 bytes [debug] stack bottom is 0x7fffce663a60. [debug] entering toplevel toplevel... [debug] stack resized to 262144 bytes [debug] entering toplevel library_toplevel... [debug] entering toplevel eval_toplevel... [debug] entering toplevel data_structures_toplevel... [debug] entering toplevel ports_toplevel... [debug] entering toplevel extras_toplevel... [debug] entering toplevel srfi_69_toplevel... [debug] entering toplevel srfi_1_toplevel... [debug] entering toplevel match_toplevel... [debug] entering toplevel srfi_4_toplevel... [debug] entering toplevel utils_toplevel... [debug] entering toplevel regex_toplevel... [debug] entering toplevel files_toplevel... [debug] entering toplevel support_toplevel... [debug] resizing heap dynamically from 500k to 1040k ... [debug] entering toplevel compiler_toplevel... [debug] entering toplevel optimizer_toplevel... [debug] entering toplevel driver_toplevel... [debug] entering toplevel platform_toplevel... [debug] entering toplevel backend_toplevel... [debug] resizing heap dynamically from 1040k to 2081k ... [debug] resizing heap dynamically from 2081k to 4163k ... [debug] forcing finalizers... [debug] resizing heap dynamically from 4163k to 2081k ... Now, here's the output when I add -:s10k to that: [debug] application startup... [debug] heap resized to 50 bytes [debug] stack bottom is 0x7fff275a99d0. [debug] entering toplevel toplevel... [debug] entering toplevel library_toplevel... The last line repeats forever until I kill it. There is no marked increase in memory consumption. HOWEVER I did figure out how to reproduce it for higher values of -:s using a script that just stepped through trying increasing values until it hung. There are several that I found; both -:s74k and -:s262k trigger it. The GDB output from the latter case is attached. Note that the backtrace gets larger and larger the longer you let it run, and it repeats (starting at frame #57 - #16). The behaviour in this last case seems different than that of the -:s10k case. The former never grew the heap, but continually called the GC process. But the latter is constantly growing the heap and as a result the memory usage balloons. I couldn't say if they're both reflections of the same problem or if they're different. Since regex.c seems heavily involved in this last case, I've also put it on my website at: http://real.metasyntax.net:2357/tmp/regex.c.gz -- Taylor Christopher Venable http://real.metasyntax.net:2357/ Starting program: /home/taylor/Desktop/chicken-3.4.7-DEBUG/chicken-boot utils.scm -quiet -no-trace -optimize-level 2 -include-path . -include-path ./ -explicit-use -output-file utils.c -:d -:s262k [debug] application startup... [debug] heap resized to 50 bytes [debug] stack bottom is 0x7fff4561ba10. [debug] entering toplevel toplevel... [debug] entering toplevel library_toplevel... [debug] entering toplevel eval_toplevel... [debug] entering toplevel data_structures_toplevel... [debug] entering toplevel ports_toplevel... [debug] entering toplevel extras_toplevel... [debug] entering toplevel srfi_69_toplevel... [debug] entering toplevel srfi_1_toplevel... [debug] entering toplevel match_toplevel... [debug] entering toplevel srfi_4_toplevel... [debug] entering toplevel utils_toplevel... [debug] entering toplevel regex_toplevel... [debug] entering toplevel files_toplevel... [debug] resizing heap dynamically from 500k to 1000k ... [debug] resizing heap dynamically from 1000k to 2000k ... [debug] resizing heap dynamically from 2000k to 4000k ... [debug] resizing heap dynamically from 4000k
Re: [Chicken-users] Build Failure (maybe infinite recursion) on x86_64 Linux
On Wed, Mar 11, 2009 at 7:25 PM, Taylor Venable tay...@metasyntax.net wrote: Well I was able to trigger a hang in my build with the debugging symbols by setting the nursery stack size to 10k. Debug log from GDB is attached. The three entries here were taken from allowing the program to run for varying periods of time. The behaviour I got spiked the CPU but the memory usage did not increase visibly. Hope this sheds a little more light; if not let me know and I can go back at it. (I notice here that the proc parameter to C_reclaim is the null pointer, don't know if that's meaningful or not...) The proc being 0 is ok. Apparently the GC looping: it might be that some sort of stack-limit check is constantly failing which results in one GC after the other. It would be cool, if you could step out of the GC (reclaim) and follow the execution - does it trigger a new GC right away? cheers, felix ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Build Failure (maybe infinite recursion) on x86_64 Linux
On Tue, Mar 10, 2009 at 02:19:56PM +0100, felix winkelmann wrote:
On Mon, Mar 9, 2009 at 3:43 PM, Taylor Venable tay...@metasyntax.net wrote:
The bug becomes more and more phantasmagorical!
When initially compiling chicken-boot with make PLATFORM=linux
DEBUGBUILD=yes bootstrap the bug does not appear.
When using -:d OR -:s500k OR -:s1m the bug does not appear.
When run inside GDB (even without debugging symbols) the bug does not
appear.
When invalid data leaks into the garbage collector (and GC's happen
very frequently in chicken), the effects are always interesting and
effectively
non-deterministic. Giving an extra command-line option will allocate storage,
which will slightly shift the moment GC happens, which will lead to another
interesting effect. Debugging these bugs is very instructive.
Try to trigger a crash or loop inside gdb by using different -:s settings.
Well I was able to trigger a hang in my build with the debugging
symbols by setting the nursery stack size to 10k. Debug log from GDB
is attached. The three entries here were taken from allowing the
program to run for varying periods of time. The behaviour I got
spiked the CPU but the memory usage did not increase visibly. Hope
this sheds a little more light; if not let me know and I can go back
at it. (I notice here that the proc parameter to C_reclaim is the
null pointer, don't know if that's meaningful or not...)
--
Taylor Christopher Venable
http://real.metasyntax.net:2357/
(gdb) run utils.scm -quiet -no-trace -optimize-level 2 -include-path .
-include-path ./ -explicit-use -output-file utils.c -:s10k
Starting program: /home/taylor/Desktop/chicken-3.4.7-DEBUG/chicken-boot
utils.scm -quiet -no-trace -optimize-level 2 -include-path . -include-path ./
-explicit-use -output-file utils.c -:s10k
^C
Program received signal SIGINT, Interrupt.
0x008411a2 in mark (x=0xeb05e8) at runtime.c:3044
3044 val = *x;
(gdb) bt
#0 0x008411a2 in mark (x=0xeb05e8) at runtime.c:3044
#1 0x0084054a in C_reclaim (trampoline=0x5b3699, proc=0x0)
at runtime.c:2806
#2 0x005b378c in C_library_toplevel (c=2, t0=30, t1=139900129233496)
at library.c:6618
#3 0x005b36c9 in toplevel_trampoline (dummy=0x0) at library.c:6242
#4 0x0083d0be in CHICKEN_run (toplevel=0x0) at runtime.c:1362
#5 0x0083b1a5 in CHICKEN_main (argc=14, argv=0x7fff1186fae8,
toplevel=0x403faa) at runtime.c:577
#6 0x00403f78 in main (argc=14, argv=0x7fff1186fae8) at chicken.c:1945
(gdb) run utils.scm -quiet -no-trace -optimize-level 2 -include-path .
-include-path ./ -explicit-use -output-file utils.c -:s10k
The program being debugged has been started already.
Start it from the beginning? (y or n) y
Starting program: /home/taylor/Desktop/chicken-3.4.7-DEBUG/chicken-boot
utils.scm -quiet -no-trace -optimize-level 2 -include-path . -include-path ./
-explicit-use -output-file utils.c -:s10k
^C
Program received signal SIGINT, Interrupt.
0x008411aa in mark (x=0x1ef15f0) at runtime.c:3046
3046 if(C_immediatep(val)) return;
(gdb) bt
#0 0x008411aa in mark (x=0x1ef15f0) at runtime.c:3046
#1 0x0084055a in C_reclaim (trampoline=0x5b3699, proc=0x0)
at runtime.c:2807
#2 0x005b378c in C_library_toplevel (c=2, t0=30, t1=140380082637400)
at library.c:6618
#3 0x005b36c9 in toplevel_trampoline (dummy=0x0) at library.c:6242
#4 0x0083d0be in CHICKEN_run (toplevel=0x0) at runtime.c:1362
#5 0x0083b1a5 in CHICKEN_main (argc=14, argv=0x7fffd0faa228,
toplevel=0x403faa) at runtime.c:577
#6 0x00403f78 in main (argc=14, argv=0x7fffd0faa228) at chicken.c:1945
(gdb) run utils.scm -quiet -no-trace -optimize-level 2 -include-path .
-include-path ./ -explicit-use -output-file utils.c -:s10k
The program being debugged has been started already.
Start it from the beginning? (y or n) y
Starting program: /home/taylor/Desktop/chicken-3.4.7-DEBUG/chicken-boot
utils.scm -quiet -no-trace -optimize-level 2 -include-path . -include-path ./
-explicit-use -output-file utils.c -:s10k
^C
Program received signal SIGINT, Interrupt.
0x00840580 in C_reclaim (trampoline=0x5b3699, proc=0x0)
at runtime.c:2805
2805 for(tinfo = trace_buffer; tinfo trace_buffer_limit; ++tinfo) {
(gdb) bt
#0 0x00840580 in C_reclaim (trampoline=0x5b3699, proc=0x0)
at runtime.c:2805
#1 0x005b378c in C_library_toplevel (c=2, t0=30, t1=140472330148440)
at library.c:6618
#2 0x005b36c9 in toplevel_trampoline (dummy=0x0) at library.c:6242
#3 0x0083d0be in CHICKEN_run (toplevel=0x0) at runtime.c:1362
#4 0x0083b1a5 in CHICKEN_main (argc=14, argv=0x7fff4b5c0838,
toplevel=0x403faa) at runtime.c:577
#5 0x00403f78 in main (argc=14, argv=0x7fff4b5c0838) at chicken.c:1945
___
Chicken-users mailing list
Chicken-users@nongnu.org
Re: [Chicken-users] Build Failure (maybe infinite recursion) on x86_64 Linux
On Mon, Mar 9, 2009 at 3:43 PM, Taylor Venable tay...@metasyntax.net wrote: The bug becomes more and more phantasmagorical! When initially compiling chicken-boot with make PLATFORM=linux DEBUGBUILD=yes bootstrap the bug does not appear. When using -:d OR -:s500k OR -:s1m the bug does not appear. When run inside GDB (even without debugging symbols) the bug does not appear. When invalid data leaks into the garbage collector (and GC's happen very frequently in chicken), the effects are always interesting and effectively non-deterministic. Giving an extra command-line option will allocate storage, which will slightly shift the moment GC happens, which will lead to another interesting effect. Debugging these bugs is very instructive. Try to trigger a crash or loop inside gdb by using different -:s settings. cheers, felix ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Build Failure (maybe infinite recursion) on x86_64 Linux
On Sat, Mar 07, 2009 at 11:46:03PM +0100, felix winkelmann wrote: On Fri, Mar 6, 2009 at 3:11 PM, Taylor Venable tay...@metasyntax.net wrote: I have no direct evidence that it was an infinite recursion. It was just a guess by the symptoms, but admittedly I have little knowledge of Chicken's internals. Sorry for the noise on that. The symptoms are that CPU usage increases to max, memory usage increases steadily, and the process appears to hang. Oh, it may very well be a recursion. I was just asking in case there is anything that points into that direction, to give us a hint at what's going wrong here. Adding -debug p produces no output, even when removing -quiet. When I remove -no-trace and add -debug p it also fails in the same way. But when I remove both -no-trace and -debug p it works. So it fails when -no-trace and/or -debug p is specified, but works OK when neither is specified. Hm... What happens if you add -:d and/or -:s500k (or -:s1m)? It is possible that different stack settings influence the behaviour. Is chicken-boot built with -g ? If not, could you add it and run it under gdb, producing a backtrace by interrupt the program if it hangs? Sorry, but this is likely to need quite a lowlevel approach. The bug becomes more and more phantasmagorical! When initially compiling chicken-boot with make PLATFORM=linux DEBUGBUILD=yes bootstrap the bug does not appear. When using -:d OR -:s500k OR -:s1m the bug does not appear. When run inside GDB (even without debugging symbols) the bug does not appear. -- Taylor Christopher Venable http://real.metasyntax.net:2357/ ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Build Failure (maybe infinite recursion) on x86_64 Linux
On Fri, Mar 6, 2009 at 3:11 PM, Taylor Venable tay...@metasyntax.net wrote: I have no direct evidence that it was an infinite recursion. It was just a guess by the symptoms, but admittedly I have little knowledge of Chicken's internals. Sorry for the noise on that. The symptoms are that CPU usage increases to max, memory usage increases steadily, and the process appears to hang. Oh, it may very well be a recursion. I was just asking in case there is anything that points into that direction, to give us a hint at what's going wrong here. Adding -debug p produces no output, even when removing -quiet. When I remove -no-trace and add -debug p it also fails in the same way. But when I remove both -no-trace and -debug p it works. So it fails when -no-trace and/or -debug p is specified, but works OK when neither is specified. Hm... What happens if you add -:d and/or -:s500k (or -:s1m)? It is possible that different stack settings influence the behaviour. Is chicken-boot built with -g ? If not, could you add it and run it under gdb, producing a backtrace by interrupt the program if it hangs? Sorry, but this is likely to need quite a lowlevel approach. Another option is to give me temporary access to your machine. cheers, felix ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Build Failure (maybe infinite recursion) on x86_64 Linux
On Fri, Mar 06, 2009 at 01:45:10PM +0100, felix winkelmann wrote: On Wed, Mar 4, 2009 at 4:32 PM, Taylor Venable tay...@metasyntax.net wrote: Hi all, When building Chicken 3.4.7 and higher (up to 3.5.2) on Ubuntu 8.10 x86_64 the chicken compiler goes into an infinite recursion. ?CPU usage spikes, memory usage increases boundlessly, and no output is ever produced. ?In Chicken 3.4.7 the file that when compiled triggers this behaviour is utils.scm and for a build process I am using: ? ?make PLATFORM=linux bootstrap ? ?make PLATFORM=linux CHICKEN=./chicken-boot The command that causes the infinite recursion is: ? ?./chicken-boot utils.scm -quiet -no-trace -optimize-level 2 -include-path . -include-path ./ -explicit-use -output-file utils.c If any other information would be helpful to diagnose this, let me know and I can provide it. ?Thanks for any help. Thanks for reporting this. If you say infinite recursion, is there anything that indicates an actual recursion? Or does it just hang? Can you invoke the command above with an additional -debug p and show me the output? I have no direct evidence that it was an infinite recursion. It was just a guess by the symptoms, but admittedly I have little knowledge of Chicken's internals. Sorry for the noise on that. The symptoms are that CPU usage increases to max, memory usage increases steadily, and the process appears to hang. Adding -debug p produces no output, even when removing -quiet. When I remove -no-trace and add -debug p it also fails in the same way. But when I remove both -no-trace and -debug p it works. So it fails when -no-trace and/or -debug p is specified, but works OK when neither is specified. Thanks, -- Taylor Christopher Venable http://real.metasyntax.net:2357/ ___ Chicken-users mailing list Chicken-users@nongnu.org http://lists.nongnu.org/mailman/listinfo/chicken-users
