On Sunday, 16 December 2012 at 05:37:57 UTC, SomeDude wrote:
Isn't the memory management completely negligible when compared
to the database access here ?
Here are the details ...
My test run selects and returns 206,085 records with 14 fields
per record.
With all dynamic memory allocations disabled that are used to
create the data structure containing the returned rows, a run
takes 5 seconds. This does not return any data, but it runs
exactly through all the records in the same way but returns to a
temporary stack allocated value of appropriate type.
If I disable the GC before the run and re-enable it immediately
after, it takes 7 seconds. I presume a full 2 seconds are used to
disable and re-enable the GC which seems like a lot of time.
With all dynamic memory allocations enabled that are used to
create the data structure containing the returned rows, a run
takes 28 seconds. In this case, all 206K records are returned in
a dynamically generate list.
If I disable the GC before the run and re-enable it immediately
after, it takes 11 seconds. Since a full 2 seconds are used to
disable and re-enable the GC, then 9 seconds are used, and since
5 seconds are used without memory allocations, the allocations
are using 4 seconds, but I'm doing a lot of allocations.
In my case, the structure is dynamically generated by allocating
each individual field for each record returned, so there's
206,085 records x 14 fields = 2,885,190 allocations being
performed. I can cut the individual allocations down to 206,000
by allocating the full record in one shot, however this is a
stress test designed to work D as hard as possible and compare it
with an identically stressed C++ version.
Both the D and C++ versions perform identically with the GC
disabled and subtracting the 2 seconds from the D version to
remove the time used up by enabling and disabling the GC during
and after the run.
I wonder why 2 seconds are used to disable and enable the GC?
That seems like a very large amount of time. If I select only
5,000 records, the time to disable and enable the GC drops
significantly to negligible levels and it takes the same amount
of time per run with GC disabled & enabled, or with GC left
enabled all the time.
During all tests, I do not run out of free RAM, and at no point
does the memory go to swap.
--rt
