Looks like the default is already 'rows', however we use first 1000 cells to estimate number of rows in the cell store. The estimate could be wrong.
Increase max-approx-items might help: bloomfilter="rows --max-approx-items 10000" 2009/7/17 Luke <[email protected]>: > 2009/7/17 孔令华 <[email protected]>: >> Thanks, Doug & Luke >> >> In my opinion, this kind of performance issue is not in the critical path >> (just in mind, no data support, but I think RPCs and disk-seeking would cost >> much more time). A clear log is very important, specially for normal users. > > We sometime needs to turn on debugging to show a lot more data, we > want to make sure the timing is not affected too much. Note, the > logging system is not for range server only, it can be used for many > other tools that could be cpu intensive. With the current format you > can display it in any timezones easily, which would be important when > you have data centers across the world. > >> BTW，according to our tests, using BloomFilter with the default params costs >> too many mems (a RangeServer with 1600 Ranges cost about 9GB, and about 2GB >> without BloomFilter). And I found that BloomFilter actually need these mems >> instead of fragments. > > Are there a lot of columns in the table? Can you use the > bloomfilter=rows to see how much memory it uses? 7GB at 1% false > positive rate can accommodate 6 billion items! > > Do you want an option to provide fixed amount memory per access group? > >> And at present, BloomFilter affects the whole system, which I think should >> be AccessGroup-wide. Using it with the table schema would be a better >> choice(eg. appoint them when create table). > > You can certainly turn it off globally and use the options at create > table time now. Doug is working on a lazy loading scheme, so that it > uses less memory (bloomfilter and index are not loaded unless needed) > > __Luke > >> 2009/7/18 Luke <[email protected]> >>> >>> You can alias lcat='perl -pe "s/^\d+/localtime($&)/e"' >>> >>> Then you can lcat *.log to see more readable timestamps. Time >>> conversion is very slow, especially it needs grab a global lock for >>> timezones. Storing timestamps in timezone neutral format has a lot of >>> benefits. >>> >>> On Fri, Jul 17, 2009 at 7:45 AM, Phoenix<[email protected]> wrote: >>> > >>> > 1. At present, the time field of the log is very hard to read. It's >>> > unreadable at some point. >>> > eg. 1247799471 INFO Hypertable.Master : >>> > >>> > There're some configurable options that can opt the output string. >>> > eg: >>> > >>> > diff --git a/src/cc/Common/Logger.cc b/src/cc/Common/Logger.cc >>> > index 301a2d1..985f231 100644 >>> > --- a/src/cc/Common/Logger.cc >>> > +++ b/src/cc/Common/Logger.cc >>> > @@ -28,6 +28,7 @@ >>> > #include <log4cpp/Layout.hh> >>> > #include <log4cpp/NDC.hh> >>> > #include <log4cpp/Priority.hh> >>> > +#include <log4cpp/PatternLayout.hh> >>> > >>> > #include "Logger.h" >>> > >>> > @@ -127,8 +128,8 @@ void >>> > Logger::initialize(const String &name, int priority, bool >>> > flush_per_log, >>> > std::ostream &out) { >>> > appender = new FlushableOstreamAppender("default", out, >>> > flush_per_log); >>> > - Logging::Layout* layout = new Logging::BasicLayout(); >>> > - //Logging::Layout* layout = new MicrosecondLayout(); >>> > + Logging::PatternLayout* layout = new Logging::PatternLayout(); >>> > + layout->setConversionPattern("[%d{%Y-%m-%d %H:%M:%S}] %p %c %x: %m >>> > %n"); >>> > appender->setLayout(layout); >>> > logger = &(Logging::Category::getInstance(name)); >>> > logger->addAppender(appender); >>> > >>> > >>> > After this mod, the output string is much better. eg: >>> > [2009-07-17 14:53:57] INFO Hypertable.RangeServer : >>> > >>> > >>> > 2. Besides, log should support rolling. At present, the log files'll >>> > grow very large as time goes. >>> > > >>> > >>> >>> >> >> >> >> >> > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Hypertable Development" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/hypertable-dev?hl=en -~----------~----~----~----~------~----~------~--~---
