On Thu, Apr 16, 2015 at 11:07:34AM -0400, Chet Ramey wrote: [...] > I knew that rang a bell somewhere. mt_hash is a function in the bash > malloc library that keeps track of all allocations and deallocations in > a table. It's part of the debugging that is enabled when you build from > the devel code. It's been well-known for a long time that the debugging > code in malloc slows bash down considerably; that's why it's not enabled > as part of bash releases.
Actually, this is the post that motivated me to look into this: (yes, the conclusion is idiotic, but I guess the rest of the post is pretty okay). http://spencertipping.com/posts/2013.0814.bash-is-irrecoverably-broken.html Now, there is some truth to what he says: dualbus@yaqui ...src/gnu/bash % time ./bash -c 'i=0; while ((i++<1000)); do declare a$RANDOM$RANDOM=1; done' ./bash -c 'i=0; while ((i++<1000)); do declare a$RANDOM$RANDOM=1; done' 0.01s user 0.06s system 93% cpu 0.077 total dualbus@yaqui ...src/gnu/bash % time ./bash -c 'i=0; while ((i++<10000)); do declare a$RANDOM$RANDOM=1; done' ./bash -c 'i=0; while ((i++<10000)); do declare a$RANDOM$RANDOM=1; done' 0.16s user 0.48s system 98% cpu 0.643 total dualbus@yaqui ...src/gnu/bash % time ./bash -c 'i=0; while ((i++<100000)); do declare a$RANDOM$RANDOM=1; done' ./bash -c 'i=0; while ((i++<100000)); do declare a$RANDOM$RANDOM=1; done' 15.44s user 6.51s system 99% cpu 21.959 total I built bash like this: CFLAGS='-pg -g -O0' ./configure --silent && make -sj4 DEBUG= MALLOC_DEBUG= To make sure the malloc debugging code doesn't interfere. I got a gprof profile with that last run, which gave: Each sample counts as 0.01 seconds. % cumulative self self total time seconds seconds calls s/call s/call name 71.42 12.07 12.07 1100104 0.00 0.00 hash_search 21.18 15.65 3.58 275435 0.00 0.00 morecore 1.63 15.93 0.28 6800525 0.00 0.00 internal_malloc 0.71 16.05 0.12 6200116 0.00 0.00 internal_free 0.59 16.15 0.10 300001 0.00 0.00 expand_word_internal 0.24 16.19 0.04 6800474 0.00 0.00 sh_xmalloc 0.18 16.22 0.03 7203779 0.00 0.00 is_basic 0.18 16.25 0.03 1932530 0.00 0.00 is_basic 0.18 16.28 0.03 200002 0.00 0.00 subexpr 0.18 16.31 0.03 100008 0.00 0.00 find_special_var 0.15 16.33 0.03 pagealign Notice how it spends most of the time in these two functions. Yeah, it's not mt_* like I said, because I did this a time ago and forgot to take notes. Does this matter much? I don't know. Having 100,000 variables declared does seem like something stupid. Still, it shouldn't have that quadratic increase in performance (I didn't even try for 1,000,000 because it was very slow), because it is a hash table. -- Eduardo Bustamante https://dualbus.me/