Re: rt/aaA.d Line: 553
On 2/15/20 10:21 AM, Ferhat Kurtulmuş wrote: On Saturday, 15 February 2020 at 14:30:20 UTC, Steven Schveighoffer wrote: On 2/15/20 6:49 AM, Ferhat Kurtulmuş wrote: On Friday, 14 February 2020 at 23:41:45 UTC, Steven I'll note that you are going to leak some memory because you are not freeing deleted buckets when you resize. In the GC version, the GC takes care of those. I appreciate it for reviewing the code and your comments. Speed is good now. I put it on the dub db. I hope I am not violating any copyright. I included name of the original author (Martin Nowak) in the code, and explicitly stated that "betterC port of druntime/blob/master/src/rt/aaA.d". What do you think about this one? I am not free-ing deleted entry in remove method: https://github.com/aferust/bcaa/blob/a37b4ee4455477abc82425f32e9cf45394f4c4a1/source/bcaa.d#L228-L230 but here in resize: https://github.com/aferust/bcaa/blob/a37b4ee4455477abc82425f32e9cf45394f4c4a1/source/bcaa.d#L190-L194 Yes, that is the moment they truly become garbage (in the GC version), so it's where you should free them. -Steve
Re: rt/aaA.d Line: 553
On Saturday, 15 February 2020 at 15:21:08 UTC, Ferhat Kurtulmuş wrote: On Saturday, 15 February 2020 at 14:30:20 UTC, Steven Schveighoffer wrote: On 2/15/20 6:49 AM, Ferhat Kurtulmuş wrote: On Friday, 14 February 2020 at 23:41:45 UTC, Steven Ops I ve made commits. Here are the links up to date https://github.com/aferust/bcaa/blob/3f0f4eaf550a28b80cd95a4f2589b1f3a53fe6c0/source/bcaa.d#L193-L195 https://github.com/aferust/bcaa/blob/3f0f4eaf550a28b80cd95a4f2589b1f3a53fe6c0/source/bcaa.d#L231-L233
Re: rt/aaA.d Line: 553
On Saturday, 15 February 2020 at 14:30:20 UTC, Steven Schveighoffer wrote: On 2/15/20 6:49 AM, Ferhat Kurtulmuş wrote: On Friday, 14 February 2020 at 23:41:45 UTC, Steven I'll note that you are going to leak some memory because you are not freeing deleted buckets when you resize. In the GC version, the GC takes care of those. -Steve I appreciate it for reviewing the code and your comments. Speed is good now. I put it on the dub db. I hope I am not violating any copyright. I included name of the original author (Martin Nowak) in the code, and explicitly stated that "betterC port of druntime/blob/master/src/rt/aaA.d". What do you think about this one? I am not free-ing deleted entry in remove method: https://github.com/aferust/bcaa/blob/a37b4ee4455477abc82425f32e9cf45394f4c4a1/source/bcaa.d#L228-L230 but here in resize: https://github.com/aferust/bcaa/blob/a37b4ee4455477abc82425f32e9cf45394f4c4a1/source/bcaa.d#L190-L194 Thus, deleted buckets will wait until a resize call to free them. I think this is better for speed.
Re: rt/aaA.d Line: 553
On 2/15/20 6:49 AM, Ferhat Kurtulmuş wrote: On Friday, 14 February 2020 at 23:41:45 UTC, Steven Schveighoffer wrote: On 2/14/20 6:36 PM, Ferhat Kurtulmuş wrote: İf ((aa.used + 1)* GROW_DEN > aa.dim * GROW_NUM) aa.grow(ti.key); This call is expensive (it reallocates all the buckets and reinserts all the existing data into the new bucket), it should be avoided if in the end we will not be incrementing used. I am trying to write a gc-free AA based on the original runtime code (mem with malloc and free). My question is that why my version is slower (about 1.5 times slower) than the runtime version? Have you ensured you are using -inline, -O, and -release? This is how druntime is compiled. https://controlc.com/2e58c305 Those are some differences from runtime version: - nogc and no typeid of course. - does not care about postblit of key types (don't need it, assume only basic types are allowed for key type) Postblit should be automatic for your code because you are doing the types directly. - runtime version uses typeid to do some alignments which are not implemented in my version. Is that the reason why my code is slower? No, alignments are important only if you don't have the type, which you do. The runtime uses void pointers and opaque structs, so it has to duplicate what the compiler is already doing for you. Your code looks like a direct port, except for the allocations, so it should be as fast if compiled the same. Your code might even be faster, because it's going to be able to inline more aggressively. There may be issues with cache coherency, but I'm not sure how to find or diagnoce those. I'll note that you are going to leak some memory because you are not freeing deleted buckets when you resize. In the GC version, the GC takes care of those. -Steve
Re: rt/aaA.d Line: 553
On Friday, 14 February 2020 at 23:41:45 UTC, Steven Schveighoffer wrote: On 2/14/20 6:36 PM, Ferhat Kurtulmuş wrote: İf ((aa.used + 1)* GROW_DEN > aa.dim * GROW_NUM) aa.grow(ti.key); This call is expensive (it reallocates all the buckets and reinserts all the existing data into the new bucket), it should be avoided if in the end we will not be incrementing used. -Steve I am trying to write a gc-free AA based on the original runtime code (mem with malloc and free). My question is that why my version is slower (about 1.5 times slower) than the runtime version? https://controlc.com/2e58c305 Those are some differences from runtime version: - nogc and no typeid of course. - does not care about postblit of key types (don't need it, assume only basic types are allowed for key type) - runtime version uses typeid to do some alignments which are not implemented in my version. Is that the reason why my code is slower?
Re: rt/aaA.d Line: 553
On Friday, 14 February 2020 at 23:28:39 UTC, Steven Schveighoffer wrote: On 2/14/20 5:57 PM, Ferhat Kurtulmuş wrote: One thing to learn, you can select a section of lines from github (click on first line, then shift-click on last line), then press the 'y' key, and it will generate a permanent link to those lines. https://github.com/dlang/druntime/blob/a581dc6d1d3bf796fcebd982221d0b3d6bbae437/src/rt/aaA.d#L553-L562 Thank you, I didn't know that.
Re: rt/aaA.d Line: 553
On 2/14/20 6:36 PM, Ferhat Kurtulmuş wrote: İf ((aa.used + 1)* GROW_DEN > aa.dim * GROW_NUM) aa.grow(ti.key); This call is expensive (it reallocates all the buckets and reinserts all the existing data into the new bucket), it should be avoided if in the end we will not be incrementing used. -Steve
Re: rt/aaA.d Line: 553
On Friday, 14 February 2020 at 23:19:31 UTC, Paul Backus wrote: On Friday, 14 February 2020 at 22:57:31 UTC, Ferhat Kurtulmuş wrote: findSlotInsert are called two times. Why not: if (++aa.used * GROW_DEN > aa.dim * GROW_NUM) aa.grow(ti.key); auto p = aa.findSlotInsert(hash); // only one call is enough? if (p.deleted) --aa.deleted; ... If I am not wrong this modification will not corrupt the current state of the hash table? `used` counts both filled and deleted buckets, so it shouldn't be incremented when changing a deleted bucket into a filled bucket. İf ((aa.used + 1)* GROW_DEN > aa.dim * GROW_NUM) aa.grow(ti.key); auto p = aa.findSlotInsert(hash); // only one call is enough? if (p.deleted) --aa.deleted; else ++aa.used;
Re: rt/aaA.d Line: 553
On 2/14/20 5:57 PM, Ferhat Kurtulmuş wrote:
I hesitated to post this on the topic of druntime because probably I
will learn something new if I am totally/stupidly wrong.
There are no stupid questions, and this is the learn forum. So you are
in the right place!
In druntime/blob/master/src/rt/aaA.d Lines 553-562:
One thing to learn, you can select a section of lines from github (click
on first line, then shift-click on last line), then press the 'y' key,
and it will generate a permanent link to those lines.
https://github.com/dlang/druntime/blob/a581dc6d1d3bf796fcebd982221d0b3d6bbae437/src/rt/aaA.d#L553-L562
...
auto p = aa.findSlotInsert(hash);
if (p.deleted)
--aa.deleted;
// check load factor and possibly grow
else if (++aa.used * GROW_DEN > aa.dim * GROW_NUM)
{
aa.grow(ti.key);
p = aa.findSlotInsert(hash);
assert(p.empty);
}
...
findSlotInsert are called two times. Why not:
if (++aa.used * GROW_DEN > aa.dim * GROW_NUM)
aa.grow(ti.key);
auto p = aa.findSlotInsert(hash); // only one call is enough?
if (p.deleted)
--aa.deleted;
...
If I am not wrong this modification will not corrupt the current state
of the hash table?
A cursory reading:
I think the case where you find the insert slot and the p.deleted is
true, you can avoid the grow function.
The grow function is much more expensive than findInsertSlot, so calling
it twice is preferable.
The reason we have the deleted status is because we want to reuse
memory, but we can't free the memory if it had a dangling reference to it.
The only time those deleted nodes turn into garbage is on a resize.
HOWEVER, one case where we can avoid the double search is if there are
no deleted nodes (we keep a count of them). This should be reasonably
often in most cases because you insert nodes and don't remove them, or
you grew the AA and all deleted nodes are purged.
So maybe change to:
Bucket *p;
if(aa.deleted > 0 && (p = aa.findSlotInsert(hash)).deleted)
--aa.deleted;
else // everything else the same
-Steve
Re: rt/aaA.d Line: 553
On Friday, 14 February 2020 at 22:57:31 UTC, Ferhat Kurtulmuş wrote: findSlotInsert are called two times. Why not: if (++aa.used * GROW_DEN > aa.dim * GROW_NUM) aa.grow(ti.key); auto p = aa.findSlotInsert(hash); // only one call is enough? if (p.deleted) --aa.deleted; ... If I am not wrong this modification will not corrupt the current state of the hash table? `used` counts both filled and deleted buckets, so it shouldn't be incremented when changing a deleted bucket into a filled bucket.
