On Sep 4, 9:44 am, "Liu Kejia(Donald)" <[EMAIL PROTECTED]> wrote:
> > the fragmentation using a simple program (just to make sure it's the
> > memory fragmentation issue, instead of something)?
>
> I'll try. The bzip2'ed new/delete trace file generated while inserting
> 2GB data is about 27MB, I have upload a copy
> tohttp://www.kerneldict.org/mediawiki/upload/trace.bz2. You can
> download and examine it if you wish.
>
Got it. Thanks!
> > We're somewhat aware of this and Doug is changing the cell cache
> > (issue 150) compaction algorithm that would take advantage of an arena
> > allocator per cell cache that will both improve performance and memory
> > usage.
>
> Does this mean:
> 1. We allocate a whole block of memory for each CellCache and fill it
> till it's full
> 2. When a CellCache is full, we open a new CellCache and do a minor
> compaction to the old one.
> 3. During the compaction, new inserts are written into the new CellCache.
> 4. The old CellCache can be freed wholly when the compaction is done.
Yeah, that's pretty much it.
> I believe this is what google does to its `memtable'. But I think
> there might be quite a few work to adopt it to hypertable. The current
> implementation in hypertable rely on timestamps heavily (if I haven''t
> misread the code), which is a very different approach, right?
The MVCC stuff and compaction is actually quite orthogonal. The new
code is mostly finished. Doug refactored the timestamp into timestamp
and internal revision/transaction as well to disentangle the timestamp/
revision/transaction semantics. Preliminary tests shows a 10% overall
improvement. Hopefully you guys can review the code when it's pushed
to 'next' or 'master', which is real soon now.
> > On Sep 4, 7:46 am, donald <[EMAIL PROTECTED]> wrote:
> >> Hi All,
>
> >> Recently I've done some test about hypertable range server's memory
> >> usage, on a single server with 16GB physical memory, with a local
> >> broker. I'm using Hypertable 0.9.0.10 with a default hypertable.cfg
> >> file. Here are the results:
>
> >> === Total Memory Usage ===
> >> Fresh start:
> >> Virtual Memory 346,680KB
> >> RSS Memory 10,972KB
>
> >> After loading 2GB data (with variable-length key/values):
> >> Virtual Memory 841,684KB
> >> RSS Memory 502,960KB
> >> Heap Memory 496,088KB
>
> >> After loading 4GB data:
> >> Virtual Memory 1,014,088KB
> >> RSS Memory 676,176KB
> >> Heap Memory 662,468KB
>
> >> After loading 8GB data:
> >> Virtual Memory 1,797,644KB
> >> RSS Memory 1,452,820KB
> >> Heap Memory 1,446,024KB
>
> >> With tcmalloc's HEAPPROFILE function, we can easily figure out
> >> CellCache is the biggest heap user, in fact, the `new statement' in
> >> the following code segment is executed millions of times:
>
> >> int CellCache::add(const ByteString key, const ByteString value,
> >> int64_t real_timestamp) {
> >> [...]
> >> (void)real_timestamp;
>
> >> new_key.ptr = ptr = new uint8_t [total_len];
>
> >> memcpy(ptr, key.ptr, key_len);
> >> [...]
>
> >> So I modified the code to let it print a piece of log whenever a new
> >> or delete statement is executed. And collected the following results:
>
> >> === CellCache Memory Usage ==
> >> After loading 2GB data:
> >> new: 4426014 times
> >> delete: 4341878 times
> >> Actual Memory Usage: 46,728 KB
> >> Memory Pages Used: 16051 = 64,204KB (27% fragments)
> >> External Fragments (by comparing the page numbers and the /proc/pid/
> >> maps file): 496,088KB - 64,204KB = 432,064KB (87% fragments)
>
> >> After loading 4GB data:
> >> new: 8910493 times
> >> delete: 8432404 times
> >> Actual Memory Usage: 226,354KB
> >> Memory Pages Used: 69576 = 278,304KB (19% fragments)
> >> External Fragments: 662,468KB - 278,304 KB = 384,164KB (58% fragments)
>
> >> After loading 8GB data:
> >> new: 17360036 times
> >> delete: 17339789 times
> >> Actual Memory Usage: 66,505 KB
>
> >> We can see the memory usage of CellCache is actually very small, while
> >> memory fragmentation is very serious, which lead to very bad memory
> >> utilization (I have to say, even worse than Java).
>
> >> If I compile with libc malloc instead of tcmalloc, the results are
> >> even worse. I think we should reduce the use of dynamic memory
> >> allocations, otherwise this situation could hardly change, even we
> >> could find better memory allocation algorithms.
>
> >> Donald
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Hypertable Development" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/hypertable-dev?hl=en
-~----------~----~----~----~------~----~------~--~---
