Re: [PATCH 1/2] Prevent excessive major gcs by having decent amount of unused heap

2021-01-21 Thread Sven Hartrumpf
Hello.

megane, 2021-01-20 17:48:
> Sven Hartrumpf  writes:
>
>> Hello.
>>
> [snip]
>>> I need some -:hi option (only for the new GC!), otherwise it crashes as 
>>> follows:
>>>
>>>   # nallch.x32 -:a0 -:o -:s4096k 0
>>> [panic] out of memory - cannot allocate next heap segment - execution 
>>> terminated
>>
>> I have experimented some more (with x32 binaries).
>> I am now running my binary with:
>>
>> -:a0 -:hiNNNm -:hm3900m -:o -:s4096k
>>
>> Then, I tried several values NNN for -:hiNNNm.
>> crashes: 2 6 7 8 16
>> ok: 3 4 5
>>
>> I added (set-gc-report! #t) and attach a complete log from a crashing run 
>> with NNN=6.
>>
> I think you're having unlucky combination of limited memory and initial
> heap size. If I understand correctly, you should be able to get the same
> error without the patches too with some -:hi values.

That sounds reasonable.
And I managed to crash the old gc, but only for fewer GC parameter settings
(just good luck).

> You could try seeing if -:hg150 helps the situation.
>
> Btw, how much memory does the machine have?

I have a ulimit -m of 24 GB, but the machines have more RAM than that.
My problems are only for i586 and x32, so 4 GB is the theoretical maximum 
anyway.
For easier comparisons, I switched my testing from x32 to i586 completely.

> Here's a small program and a script to test how much memory you can
> allocate with different initial heap sizes:
>
> ;;; FILE: foo.scm
> (import (chicken blob))
> (define foo '())
> (let l ([tot 0] [s 100])
>   ;; (print "at " tot " " s)
>   (set! foo (cons (make-blob s) foo))
>   (l (+ s tot) (min 1 (inexact->exact (round (* s 1.4))
> ;;; RUN:
>
> ;;; FILE: run.sh
> #!/usr/bin/env bash
> csc5 foo.scm
> for HI in $(seq 16); do
>  echo -n " -:hi${HI}M"
>  (./foo -:g -:hi${HI}M -:hg150 2>&1) | grep -E '(at |resized)' | tail -n 1
>  echo -n "B -:hi${HI}M"
>  (./foo -:g -:hi${HI}M -:hg200 2>&1) | grep -E '(at |resized)' | tail -n 1
> done
>
> If I run this with ulimit -v 1048576 I get maximal heap sizes ranging
> from 509573502 to 1007797006 bytes w/o the patches, and
> 518025406-1024700814 with the patches.

The worst runs for your test program:
old gc  -:hi1M[GC] resized heap to 102207885 bytes
new gc B   -:hi11M[GC] resized heap to  99804232 bytes
The best runs:
old gc B -:hi9M[GC] resized heap to211858427 bytes
new gc B -:hi5M[GC] resized heap to197589146 bytes

As hoped, the numbers for old gc and new gc are similar.
The best runs reach the optimum of 4 GB / 2.
The worst runs are somewhat disappointing, but I was not hit
by this before because I had optimized and/or lucky GC
settings for the old GC, it seems.

Thanks for your help; I know now more about wen to be cautious with
Chicken's GC.

Ciao
Sven



Re: [PATCH 1/2] Prevent excessive major gcs by having decent amount of unused heap

2021-01-20 Thread megane


Sven Hartrumpf  writes:

> Hello.
>
[snip]
>> I need some -:hi option (only for the new GC!), otherwise it crashes as 
>> follows:
>>
>>   # nallch.x32 -:a0 -:o -:s4096k 0
>> [panic] out of memory - cannot allocate next heap segment - execution 
>> terminated
>
> I have experimented some more (with x32 binaries).
> I am now running my binary with:
>
> -:a0 -:hiNNNm -:hm3900m -:o -:s4096k
>
> Then, I tried several values NNN for -:hiNNNm.
> crashes: 2 6 7 8 16
> ok: 3 4 5
>
> I added (set-gc-report! #t) and attach a complete log from a crashing run 
> with NNN=6.
>

I think you're having unlucky combination of limited memory and initial
heap size. If I understand correctly, you should be able to get the same
error without the patches too with some -:hi values.

You could try seeing if -:hg150 helps the situation.

Btw, how much memory does the machine have?

Here's a small program and a script to test how much memory you can
allocate with different initial heap sizes:

;;; FILE: foo.scm
(import (chicken blob))
(define foo '())
(let l ([tot 0] [s 100])
  ;; (print "at " tot " " s)
  (set! foo (cons (make-blob s) foo))
  (l (+ s tot) (min 1 (inexact->exact (round (* s 1.4))
;;; RUN:

;;; FILE: run.sh
#!/usr/bin/env bash
csc5 foo.scm
for HI in $(seq 16); do
 echo -n " -:hi${HI}M"
 (./foo -:g -:hi${HI}M -:hg150 2>&1) | grep -E '(at |resized)' | tail -n 1
 echo -n "B -:hi${HI}M"
 (./foo -:g -:hi${HI}M -:hg200 2>&1) | grep -E '(at |resized)' | tail -n 1
done

If I run this with ulimit -v 1048576 I get maximal heap sizes ranging
from 509573502 to 1007797006 bytes w/o the patches, and
518025406-1024700814 with the patches.



Re: [PATCH 1/2] Prevent excessive major gcs by having decent amount of unused heap

2021-01-18 Thread Sven Hartrumpf
Hello.

I wrote, 2021-01-13 22:24:
> Hi.
> 
> megane wrote, 2021-01-13 12:10:
>>
>> Sven Hartrumpf  writes:
>>
>>> Hi Mario.
>>>
>> [snip]
>>> Run options are:
>>>
>>> -:hi256m -:H -:hs0 -:o -:s4096k
>> The combination of -:hi256m and -:hs0 pretty much guarantees these
>> patches won't help you.
> 
> I feared so, when I was asked to post my options :-)
> I probably added -:hs0 some years ago to avoid heap size yo-yo-ing in a dumb 
> way ...;
> I will not use it for the next experiments.
> I need some -:hi option (only for the new GC!), otherwise it crashes as 
> follows:
> 
>   # nallch.x32 -:a0 -:o -:s4096k 0
> [panic] out of memory - cannot allocate next heap segment - execution 
> terminated

I have experimented some more (with x32 binaries).
I am now running my binary with:

-:a0 -:hiNNNm -:hm3900m -:o -:s4096k

Then, I tried several values NNN for -:hiNNNm.
crashes: 2 6 7 8 16
ok: 3 4 5

I added (set-gc-report! #t) and attach a complete log from a crashing run with 
NNN=6.

Ciao
Sven

> Even -:hm3900m is not helping (although only 715 MB are reported below),
> I must give some -:hi option (any value worked so far, smallest test so far 
> -:hi4m):
> 
>   # nallch.x32 -:a0 -:hi128m -:o -:s4096k 0
> 
> Output from chicken's (time ):
> 
> 733.251s CPU time, 100.732s GC time (major), 1033919871/121748867 mutations 
> (total/tracked), 183/1968864 GCs (major/minor), maximum live heap: 714.99 MiB
> 
>> - The first patch would bump the heap size up if your program constantly
>>   needed, say 255.99MB of memory (so it'd generate 10k of garbage, run major
>>   gc, generate 10k of garbage, run major gc, ...). So, if you've chosen
>>   256m conservatively, taking your input data into account, the patches
>>   don't help.
>> - The second patch is a simple hysteresis control that mitigates rapid
>>   heap size yo-yo-ing. Using -:hs0 prevents that completely.
>>
>> Also, there won't be that much speed-up if the major-gc-time to
>> total-run-time ratio is low to begin with.
> 
> The ratio was at 35 % on older hardware (esp. slower RAM is causing this high
> ratio, I guess) and is now down (on a i9-9900K CPU) to 13.7 %
> as calculated from the above output from (time ).
> 
> Ciao
> Sven
> 


ch-crash.log.zst
Description: Binary data


Re: [PATCH 1/2] Prevent excessive major gcs by having decent amount of unused heap

2021-01-13 Thread Sven Hartrumpf
Hi.

megane wrote, 2021-01-13 12:10:
>
> Sven Hartrumpf  writes:
>
>> Hi Mario.
>>
> [snip]
>> Run options are:
>>
>> -:hi256m -:H -:hs0 -:o -:s4096k
> The combination of -:hi256m and -:hs0 pretty much guarantees these
> patches won't help you.

I feared so, when I was asked to post my options :-)
I probably added -:hs0 some years ago to avoid heap size yo-yo-ing in a dumb 
way ...;
I will not use it for the next experiments.
I need some -:hi option (only for the new GC!), otherwise it crashes as follows:

  # nallch.x32 -:a0 -:o -:s4096k 0
[panic] out of memory - cannot allocate next heap segment - execution terminated

Even -:hm3900m is not helping (although only 715 MB are reported below),
I must give some -:hi option (any value worked so far, smallest test so far 
-:hi4m):

  # nallch.x32 -:a0 -:hi128m -:o -:s4096k 0

Output from chicken's (time ):

733.251s CPU time, 100.732s GC time (major), 1033919871/121748867 mutations 
(total/tracked), 183/1968864 GCs (major/minor), maximum live heap: 714.99 MiB

> - The first patch would bump the heap size up if your program constantly
>   needed, say 255.99MB of memory (so it'd generate 10k of garbage, run major
>   gc, generate 10k of garbage, run major gc, ...). So, if you've chosen
>   256m conservatively, taking your input data into account, the patches
>   don't help.
> - The second patch is a simple hysteresis control that mitigates rapid
>   heap size yo-yo-ing. Using -:hs0 prevents that completely.
>
> Also, there won't be that much speed-up if the major-gc-time to
> total-run-time ratio is low to begin with.

The ratio was at 35 % on older hardware (esp. slower RAM is causing this high
ratio, I guess) and is now down (on a i9-9900K CPU) to 13.7 %
as calculated from the above output from (time ).

Ciao
Sven



Re: [PATCH 1/2] Prevent excessive major gcs by having decent amount of unused heap

2021-01-13 Thread megane


Sven Hartrumpf  writes:

> Hi Mario.
>
[snip]
> Run options are:
>
> -:hi256m -:H -:hs0 -:o -:s4096k

Hi Sven,

The combination of -:hi256m and -:hs0 pretty much guarantees these
patches won't help you.

- The first patch would bump the heap size up if your program constantly
  needed, say 255.99MB of memory (so it'd generate 10k of garbage, run major
  gc, generate 10k of garbage, run major gc, ...). So, if you've chosen
  256m conservatively, taking your input data into account, the patches
  don't help.
- The second patch is a simple hysteresis control that mitigates rapid
  heap size yo-yo-ing. Using -:hs0 prevents that completely.

Also, there won't be that much speed-up if the major-gc-time to
total-run-time ratio is low to begin with.



Re: [PATCH 1/2] Prevent excessive major gcs by having decent amount of unused heap

2021-01-12 Thread Sven Hartrumpf
Hi Mario.

Mario wrote, 2021-01-12 19:17:
> Hi Sven,
>
> On Tue, 12 Jan 2021 13:44:04 +0100 (CET) Sven Hartrumpf  
> wrote:
>
>> Mario wrote, 2021-01-12 12:39:
>>>
>>> On Tue, 12 Jan 2021 11:42:31 +0100 (CET) Sven Hartrumpf  
>>> wrote:
>>>
 Mario, 2021-01-10 10:03:
>
> On Sat, 09 Jan 2021 15:14:31 +0100 (CET) Sven Hartrumpf 
>  wrote:
>
>> Thanks for the patches and the tests!
>>
>>> I ran a relatively extensive set of benchmark configurations comparing
>>> CHICKEN from master against CHICKEN from master plus the patches.
>>
>> I would like to run some real-world benchmarks.
>> They are dynamically linked.
>> Would it be enough to run them with the different libchicken.so.11 
>> versions
>> (before and after GC patch) or should I recompile my programs?
>
> If your binaries are dynamically linked and use a version of CHICKEN
> which is ABI-compatible with the code in master (ABI version 11),
> switching libchicken.so (e.g., via LD_LIBRARY_PATH) without recompiling
> your code should do the trick.
>
> If you can, please let us know about your benchmark results.

 The results for x86-64 show improvements (- 2 % run time);
 for i586 and x32, the results show slowdowns (+ 1-2 % run time):

 GC arch.   avg. runtime in ms

 old gc x86-64  615
 new gc x86-64  601

 old gc x32 505
 new gc x32 509

 old gc i586728
 new gc i586739

 (My benchmark is a semantic natural language parser, which is GC-intensive.
 C compiler is gcc 9.3.0 from Ubuntu 20.10.)

 Maybe the changes are somewhat biased for x86-64 and
 could be adjusted for i586 and x32?
>>>
>>> Thanks for sharing your results.
>>>
>>> To be honest, the differences in your results are so small I'd flag them
>>> as "noise".
>>> Are you sure the right CHICKEN libraries were being used?
>>
>> Yes. I checked it several times and ran all benchmarks at least 3
>> times.
>
> Ok.  Thanks for confirming that.
>
>> For the longer run times, I also guess that this might be a random effect
>> of other GC settings etc.
>
> Could be.  Do you compile/run your program with some non-default options?

Compile options are:

 -d0 -O5

Run options are:

-:hi256m -:H -:hs0 -:o -:s4096k

To be fair, I should probably half -:hi (and -:s ?)
for i586 and x32?

>> If "x86-86" in http://parenteses.org/mario/misc/benchmarks-megane/ means
>> "x86-64",
>
> Indeed.  It was a typo.  Thanks for spotting and reporting it.  I've
> fixed it.
>
>> then you have already checked for two of the three discussed
>> architectures that the GC changes are improving runtime on average.
>> (If you have some time, maybe a comparison with x32 would be interesting.)
>
> I can try that.  Would you recommend any Linux distribution that
> provides an installer with x32 support?  Or any convenient method to get
> a x32 system running?

Ubuntu has a very stable multiarch setup. If you install a 64bit Ubuntu, you 
can add
x32 support as follows (please adjust to the gcc versions you need):

  apt install libc6-dev-x32 libc6-x32 libx32atomic1 libx32gcc1 libx32gcc-7-dev 
libx32gcc-8-dev libx32gcc-9-dev libx32gcc-10-dev libx32stdc++6 libx32z1 
libx32z1-dev pkg-config-x86-64-linux-gnux32

> I'm also gonna run the benchmarks on Ubuntu 20.10 with the same
> toolchain you used to see if the toolchain could be responsible for
> flattening the results.
>
> Some combinations benefit much more than others.  For example,
> http://parenteses.org/mario/misc/benchmarks-megane/x86/stock-vs-megane-gcc.html
> is amazing, but
> http://parenteses.org/mario/misc/benchmarks-megane/x86-64/stock-vs-megane-extra-clang-speed.html
> not so much.
>
> It might also be the case that your application simply doesn't benefit
> from the patches...
>
> Since you asked whether recompilation of programs was necessary, I
> suppose it is not easy for you to call your application with the `time'
> macro wrapping a strategic entrypoint, right (it'd require recompiling
> it)?

No problem, I will do this now.
(I just wanted to avoid the recompilation hours to speed up experiments :-) )

> With that we could have a better idea on how the patches influence
> the runtime aspects of your program (e.g., number of GCs, their duration
> etc.).

Ciao
Sven

> All the best.
> Mario
> --
> http://parenteses.org/mario
>



Re: [PATCH 1/2] Prevent excessive major gcs by having decent amount of unused heap

2021-01-12 Thread Mario Domenech Goulart
Hi Sven,

On Tue, 12 Jan 2021 13:44:04 +0100 (CET) Sven Hartrumpf  
wrote:

> Mario wrote, 2021-01-12 12:39:
>>
>> On Tue, 12 Jan 2021 11:42:31 +0100 (CET) Sven Hartrumpf  
>> wrote:
>>
>>> Mario, 2021-01-10 10:03:

 On Sat, 09 Jan 2021 15:14:31 +0100 (CET) Sven Hartrumpf 
  wrote:

> Thanks for the patches and the tests!
>
>> I ran a relatively extensive set of benchmark configurations comparing
>> CHICKEN from master against CHICKEN from master plus the patches.
>
> I would like to run some real-world benchmarks.
> They are dynamically linked.
> Would it be enough to run them with the different libchicken.so.11 
> versions
> (before and after GC patch) or should I recompile my programs?

 If your binaries are dynamically linked and use a version of CHICKEN
 which is ABI-compatible with the code in master (ABI version 11),
 switching libchicken.so (e.g., via LD_LIBRARY_PATH) without recompiling
 your code should do the trick.

 If you can, please let us know about your benchmark results.
>>>
>>> The results for x86-64 show improvements (- 2 % run time);
>>> for i586 and x32, the results show slowdowns (+ 1-2 % run time):
>>>
>>> GC arch.   avg. runtime in ms
>>>
>>> old gc x86-64  615
>>> new gc x86-64  601
>>>
>>> old gc x32 505
>>> new gc x32 509
>>>
>>> old gc i586728
>>> new gc i586739
>>>
>>> (My benchmark is a semantic natural language parser, which is GC-intensive.
>>> C compiler is gcc 9.3.0 from Ubuntu 20.10.)
>>>
>>> Maybe the changes are somewhat biased for x86-64 and
>>> could be adjusted for i586 and x32?
>>
>> Thanks for sharing your results.
>>
>> To be honest, the differences in your results are so small I'd flag them
>> as "noise".
>> Are you sure the right CHICKEN libraries were being used?
>
> Yes. I checked it several times and ran all benchmarks at least 3
> times.

Ok.  Thanks for confirming that.

> For the longer run times, I also guess that this might be a random effect
> of other GC settings etc.

Could be.  Do you compile/run your program with some non-default options?

> If "x86-86" in http://parenteses.org/mario/misc/benchmarks-megane/ means
> "x86-64",

Indeed.  It was a typo.  Thanks for spotting and reporting it.  I've
fixed it.

> then you have already checked for two of the three discussed
> architectures that the GC changes are improving runtime on average.
> (If you have some time, maybe a comparison with x32 would be interesting.)

I can try that.  Would you recommend any Linux distribution that
provides an installer with x32 support?  Or any convenient method to get
a x32 system running?

I'm also gonna run the benchmarks on Ubuntu 20.10 with the same
toolchain you used to see if the toolchain could be responsible for
flattening the results.

Some combinations benefit much more than others.  For example,
http://parenteses.org/mario/misc/benchmarks-megane/x86/stock-vs-megane-gcc.html
is amazing, but
http://parenteses.org/mario/misc/benchmarks-megane/x86-64/stock-vs-megane-extra-clang-speed.html
not so much.

It might also be the case that your application simply doesn't benefit
from the patches...

Since you asked whether recompilation of programs was necessary, I
suppose it is not easy for you to call your application with the `time'
macro wrapping a strategic entrypoint, right (it'd require recompiling
it)?  With that we could have a better idea on how the patches influence
the runtime aspects of your program (e.g., number of GCs, their duration
etc.).

All the best.
Mario
-- 
http://parenteses.org/mario



Re: [PATCH 1/2] Prevent excessive major gcs by having decent amount of unused heap

2021-01-12 Thread Sven Hartrumpf
Hi Mario.

Mario wrote, 2021-01-12 12:39:
> Hi Sven,
>
> On Tue, 12 Jan 2021 11:42:31 +0100 (CET) Sven Hartrumpf  
> wrote:
>
>> Mario, 2021-01-10 10:03:
>>>
>>> On Sat, 09 Jan 2021 15:14:31 +0100 (CET) Sven Hartrumpf  
>>> wrote:
>>>
 Thanks for the patches and the tests!

> I ran a relatively extensive set of benchmark configurations comparing
> CHICKEN from master against CHICKEN from master plus the patches.

 I would like to run some real-world benchmarks.
 They are dynamically linked.
 Would it be enough to run them with the different libchicken.so.11 versions
 (before and after GC patch) or should I recompile my programs?
>>>
>>> If your binaries are dynamically linked and use a version of CHICKEN
>>> which is ABI-compatible with the code in master (ABI version 11),
>>> switching libchicken.so (e.g., via LD_LIBRARY_PATH) without recompiling
>>> your code should do the trick.
>>>
>>> If you can, please let us know about your benchmark results.
>>
>> The results for x86-64 show improvements (- 2 % run time);
>> for i586 and x32, the results show slowdowns (+ 1-2 % run time):
>>
>> GC arch.   avg. runtime in ms
>>
>> old gc x86-64  615
>> new gc x86-64  601
>>
>> old gc x32 505
>> new gc x32 509
>>
>> old gc i586728
>> new gc i586739
>>
>> (My benchmark is a semantic natural language parser, which is GC-intensive.
>> C compiler is gcc 9.3.0 from Ubuntu 20.10.)
>>
>> Maybe the changes are somewhat biased for x86-64 and
>> could be adjusted for i586 and x32?
>
> Thanks for sharing your results.
>
> To be honest, the differences in your results are so small I'd flag them
> as "noise".
> Are you sure the right CHICKEN libraries were being used?

Yes. I checked it several times and ran all benchmarks at least 3 times.
For the longer run times, I also guess that this might be a random effect
of other GC settings etc.

If "x86-86" in http://parenteses.org/mario/misc/benchmarks-megane/ means
"x86-64", then you have already checked for two of the three discussed
architectures that the GC changes are improving runtime on average.
(If you have some time, maybe a comparison with x32 would be interesting.)

Greetings
Sven



Re: [PATCH 1/2] Prevent excessive major gcs by having decent amount of unused heap

2021-01-12 Thread Mario Domenech Goulart
Hi Sven,

On Tue, 12 Jan 2021 11:42:31 +0100 (CET) Sven Hartrumpf  
wrote:

> Mario, 2021-01-10 10:03:
>>
>> On Sat, 09 Jan 2021 15:14:31 +0100 (CET) Sven Hartrumpf  
>> wrote:
>>
>>> Thanks for the patches and the tests!
>>>
 I ran a relatively extensive set of benchmark configurations comparing
 CHICKEN from master against CHICKEN from master plus the patches.
>>>
>>> I would like to run some real-world benchmarks.
>>> They are dynamically linked.
>>> Would it be enough to run them with the different libchicken.so.11 versions
>>> (before and after GC patch) or should I recompile my programs?
>>
>> If your binaries are dynamically linked and use a version of CHICKEN
>> which is ABI-compatible with the code in master (ABI version 11),
>> switching libchicken.so (e.g., via LD_LIBRARY_PATH) without recompiling
>> your code should do the trick.
>>
>> If you can, please let us know about your benchmark results.
>
> The results for x86-64 show improvements (- 2 % run time);
> for i586 and x32, the results show slowdowns (+ 1-2 % run time):
>
> GC arch.   avg. runtime in ms
>
> old gc x86-64  615
> new gc x86-64  601
>
> old gc x32 505
> new gc x32 509
>
> old gc i586728
> new gc i586739
>
> (My benchmark is a semantic natural language parser, which is GC-intensive.
> C compiler is gcc 9.3.0 from Ubuntu 20.10.)
>
> Maybe the changes are somewhat biased for x86-64 and
> could be adjusted for i586 and x32?

Thanks for sharing your results.

To be honest, the differences in your results are so small I'd flag them
as "noise".  Are you sure the right CHICKEN libraries were being used?

All the best.
Mario
-- 
http://parenteses.org/mario



Re: [PATCH 1/2] Prevent excessive major gcs by having decent amount of unused heap

2021-01-12 Thread Sven Hartrumpf
Mario, 2021-01-10 10:03:
> Hi Sven,
>
> On Sat, 09 Jan 2021 15:14:31 +0100 (CET) Sven Hartrumpf  
> wrote:
>
>> Thanks for the patches and the tests!
>>
>>> I ran a relatively extensive set of benchmark configurations comparing
>>> CHICKEN from master against CHICKEN from master plus the patches.
>>
>> I would like to run some real-world benchmarks.
>> They are dynamically linked.
>> Would it be enough to run them with the different libchicken.so.11 versions
>> (before and after GC patch) or should I recompile my programs?
>
> If your binaries are dynamically linked and use a version of CHICKEN
> which is ABI-compatible with the code in master (ABI version 11),
> switching libchicken.so (e.g., via LD_LIBRARY_PATH) without recompiling
> your code should do the trick.
>
> If you can, please let us know about your benchmark results.

The results for x86-64 show improvements (- 2 % run time);
for i586 and x32, the results show slowdowns (+ 1-2 % run time):

GC arch.   avg. runtime in ms

old gc x86-64  615
new gc x86-64  601

old gc x32 505
new gc x32 509

old gc i586728
new gc i586739

(My benchmark is a semantic natural language parser, which is GC-intensive.
C compiler is gcc 9.3.0 from Ubuntu 20.10.)

Maybe the changes are somewhat biased for x86-64 and
could be adjusted for i586 and x32?

Ciao
Sven



Re: [PATCH 1/2] Prevent excessive major gcs by having decent amount of unused heap

2021-01-10 Thread Mario Domenech Goulart
Hi Sven,

On Sat, 09 Jan 2021 15:14:31 +0100 (CET) Sven Hartrumpf  
wrote:

> Thanks for the patches and the tests!
>
>> I ran a relatively extensive set of benchmark configurations comparing
>> CHICKEN from master against CHICKEN from master plus the patches.
>
> I would like to run some real-world benchmarks.
> They are dynamically linked.
> Would it be enough to run them with the different libchicken.so.11 versions
> (before and after GC patch) or should I recompile my programs?

If your binaries are dynamically linked and use a version of CHICKEN
which is ABI-compatible with the code in master (ABI version 11),
switching libchicken.so (e.g., via LD_LIBRARY_PATH) without recompiling
your code should do the trick.

If you can, please let us know about your benchmark results.

All the best.
Mario
-- 
http://parenteses.org/mario



Re: [PATCH 1/2] Prevent excessive major gcs by having decent amount of unused heap

2021-01-09 Thread Sven Hartrumpf
Hi.

Thanks for the patches and the tests!

> I ran a relatively extensive set of benchmark configurations comparing
> CHICKEN from master against CHICKEN from master plus the patches.

I would like to run some real-world benchmarks.
They are dynamically linked.
Would it be enough to run them with the different libchicken.so.11 versions
(before and after GC patch) or should I recompile my programs?

Greetings
Sven



Re: [PATCH 1/2] Prevent excessive major gcs by having decent amount of unused heap

2021-01-08 Thread Mario Domenech Goulart
Hi,

On Wed, 08 Jan 2020 17:03:25 +0200 megane  wrote:

> The garbage collector currently has couple of less than optimal
> behaviors:
>
> 1. If there's constantly only a small amount of heap left after a major
>collection then there will be constant major collections.
>
> 2. If the used heap space does yo-yo then there will be constant heap
>resizes.
>
>
> These patches are for those two cases.
>
> Benchmarks for the 1st patch:
>   http://paste.call-cc.org/paste?id=02242cbbd2c79456920d8dfdb327b791231eefd2
>   
> http://paste.call-cc.org/paste?id=02242cbbd2c79456920d8dfdb327b791231eefd2#a1
>
>   - note the up-to 2x speedup in some cases
>
> Benchmarks when both patches applied:
>   
> http://paste.call-cc.org/paste?id=02242cbbd2c79456920d8dfdb327b791231eefd2#a2
>
>   - see especially kernwyk-string
>
> Some picks:
>
>   Vanilla master:
>
>   Total run time (CPU time):  5m39.54499s
>   Total time spent in major GCs:  54.041s
>   Total number of major GCs:  39245
>
>   With -:hf4M
>
>   Total run time (CPU time):  4m57.95101s
>   Total time spent in major GCs:  15.862s
>   Total number of major GCs:  4766

To celebrate the one-year anniversary of the submission of these patches
I pushed them.

I ran a relatively extensive set of benchmark configurations comparing
CHICKEN from master against CHICKEN from master plus the patches.
Results are here: http://parenteses.org/mario/misc/benchmarks-megane/

I could not find any significant performance degradation with the
patches.  The only degradations were very small, below the 5% deviance
or large differences in tiny timing values, which I'd consider noise.
On the other hand, the performance increase is quite significant and far
from being noise, as far as I can tell.

Thanks for the patches and for your patience, megane.

All the best.
Mario
-- 
http://parenteses.org/mario