Thanks. If you have any news about this problem, please let me know. Hi, > > This is a known problem and we're working on a fix for it now. Your fix > should work for now, but is not an ideal solution and could potentially > cause worse problems. You should be ok for now, but as soon as we come out > with the fixed version, I advise that you upgrade to it immediately. > > - Doug > > On Wed, Jun 20, 2012 at 9:46 AM, BigQiao <[email protected]> wrote: > >> I think I figure it out. >> The memory is cost by Ranges which load by replay_update() in >> RangeServer.cc >> those Ranges will not be compact because their needs_compaction=false; >> >> So I add one line to RangeServer.cc:3616, RangeServer::replay_load_range >> >> 3612 schema = table_info->get_schema(); >> 3613 >> 3614 range = new Range(m_master_client, schema, range_entity, >> live_table_info.get()); >> 3615 //## >> 3616 range->set_needs_compaction(true); //## Qiao Fix Memory Bug >> 2012-06-20 >> 3617 //## >> 3618 range->recovery_initialize(); >> 3619 >> 3620 table_info->add_range(range); >> >> After replay finish, it will start to compact cellcaches. >> >> >> >> We have set Hypertable.RangeServer.**MemoryLimit=4GB, but rangeserver >>> still run out of memory(almost cost 10GB), and 99% cost by CellCache, >>> >>> I wait it for almost 1 hour, but nothing changed. I belive the >>> compactions stopped. >>> >>> >>> Hi, >>>> >>>> "Application queue PAUSED due to low memory condition" is actually >>>> quite common and usually not a problem, unless you have MANY of them. >>>> These >>>> are usually caused either by low memory (because load is too high) or >>>> because hypertable fails to run compactions. We had a bug in 0.9.5.4 which >>>> did not reset some internal flags correctly, thus stopping compactions, >>>> and >>>> the cellcaches were not flushed to disk. but this was fixed in 0.9.5.6 (or >>>> 0.9.5.5, if i remember correctly). >>>> >>>> There are a couple of options you can use to control memory behavior: >>>> >>>> >>>> --Hypertable.RangeServer.**MemoryLimit arg >>>> ** RangeServer memory limit >>>> --Hypertable.RangeServer.**MemoryLimit.Percentage arg (=60) >>>> ** RangeServer memory limit >>>> specified as >>>> ** percentage of physical RAM >>>> --Hypertable.RangeServer.**LowMemoryLimit.Percentage arg (=10) >>>> ** Amount of memory to free in >>>> low memory >>>> ** condition as percentage of >>>> RangeServer >>>> ** memory limit >>>> --Hypertable.RangeServer.**MemoryLimit.EnsureUnused arg >>>> ** Amount of unused physical >>>> memory >>>> --Hypertable.RangeServer.**MemoryLimit.EnsureUnused.**Percentage arg >>>> ** Amount of unused physical >>>> memory >>>> ** specified as percentage of >>>> physical RAM >>>> >>>> bye >>>> Christoph >>>> 2012/6/19 gcc.lua <[email protected]> >>>> >>>>> Hi,all >>>>> I have use c++ stream pipe to implement map/reduce logic handle >>>>> big data(hypertable/hadoop) >>>>> I had search the old issues like my problem, it's like CellCache >>>>> memmory fragment, >>>>> and fixed use CellCachePool import in 0.9.1.0/0.9.1.1, It be removed >>>>> in latest version 0.9.5.6(or sth. old version), >>>>> where can I commit the Hypertable.RangeServer.log, thanks >>>>> >>>>> server:(16GB Memory) >>>>> >>>>> 20674 line: >>>>> 1340084923 INFO Hypertable.RangeServer : (/home/hadoop/temp/ >>>>> hypertable-0615/src/cc/**Hypertable/RangeServe r/ >>>>> MaintenanceScheduler.cc:255) Memory Allocation: BlockCache=0.00% >>>>> BlockIndex=0.00% BloomFilter=0.00% CellCache=99.51% ShadowCache=0.00% >>>>> QueryCache=0.49% >>>>> >>>>> >>>>> 221688 line: >>>>> 1340093490 INFO Hypertable.RangeServer : (/home/hadoop/temp/ >>>>> hypertable-0615/src/cc/**Hypertable/RangeServe r/RangeServer.cc: >>>>> 3710) Application queue PAUSED due to low memory condition >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Hypertable Development" group. >>>>> To post to this group, send email to >>>>> hypertable-dev@googlegroups.**com<[email protected]> >>>>> . >>>>> To unsubscribe from this group, send email to >>>>> hypertable-dev+unsubscribe@**googlegroups.com<hypertable-dev%[email protected]> >>>>> . >>>>> For more options, visit this group at http://groups.google.com/** >>>>> group/hypertable-dev?hl=en<http://groups.google.com/group/hypertable-dev?hl=en> >>>>> . >>>>> >>>>> >>>> -- >> You received this message because you are subscribed to the Google Groups >> "Hypertable Development" group. >> To view this discussion on the web visit >> https://groups.google.com/d/msg/hypertable-dev/-/oNBjwrL8XIYJ. >> >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]. >> For more options, visit this group at >> http://groups.google.com/group/hypertable-dev?hl=en. >> > > > > -- > Doug Judd > CEO, Hypertable Inc. > >
-- You received this message because you are subscribed to the Google Groups "Hypertable Development" group. To view this discussion on the web visit https://groups.google.com/d/msg/hypertable-dev/-/piG91n1uaVgJ. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/hypertable-dev?hl=en.
