Hi Doug,
2009/7/18 Doug Judd <[email protected]> > Comments inline ... > > 2009/7/17 孔令华 <[email protected]> > >> Thanks, Doug & Luke >> >> In my opinion, this kind of performance issue is not in the critical path >> (just in mind, no data support, but I think RPCs and disk-seeking would cost >> much more time). A clear log is very important, specially for normal users. > > > That's a good point. > > >> BTW，according to our tests, using BloomFilter with the default params >> costs too many mems (a RangeServer with 1600 Ranges cost about 9GB, and >> about 2GB without BloomFilter). And I found that BloomFilter actually need >> these mems instead of fragments. >> And at present, BloomFilter affects the whole system, which I think should >> be AccessGroup-wide. Using it with the table schema would be a better >> choice(eg. appoint them when create table). > > > Are you suggesting that we increase the default false positive rate in > order to reduce the size of the bloom filter? Or do you feel they should be > disabled by default? The bloom filters are configureable per-access group. > By default, they are set to "on". > Increasing the rate of disabling it by default would be better. Actually, we used it with the default params for long time until we found that the mem usage is too big. Only using them when user actually understand it would be better as it's big mem usage. > On a related note, I'm about to checkin a change that will cause the bloom > filters (and block indexes) to get dropped from memory under low memory > condition and then paged back in on demand. This should help considerable > in the bulk load situation where the bloom filters are not used. > That's a good change. BTW, allowing RS to load CellStores without BloomFilter data even when it actually has such data would help sth. > > - Doug > > 2009/7/18 Luke <[email protected]> >> >> >>> You can alias lcat='perl -pe "s/^\d+/localtime($&)/e"' >>> >>> Then you can lcat *.log to see more readable timestamps. Time >>> conversion is very slow, especially it needs grab a global lock for >>> timezones. Storing timestamps in timezone neutral format has a lot of >>> benefits. >>> >>> On Fri, Jul 17, 2009 at 7:45 AM, Phoenix<[email protected]> >>> wrote: >>> > >>> > 1. At present, the time field of the log is very hard to read. It's >>> > unreadable at some point. >>> > eg. 1247799471 INFO Hypertable.Master : >>> > >>> > There're some configurable options that can opt the output string. >>> > eg: >>> > >>> > diff --git a/src/cc/Common/Logger.cc b/src/cc/Common/Logger.cc >>> > index 301a2d1..985f231 100644 >>> > --- a/src/cc/Common/Logger.cc >>> > +++ b/src/cc/Common/Logger.cc >>> > @@ -28,6 +28,7 @@ >>> > #include <log4cpp/Layout.hh> >>> > #include <log4cpp/NDC.hh> >>> > #include <log4cpp/Priority.hh> >>> > +#include <log4cpp/PatternLayout.hh> >>> > >>> > #include "Logger.h" >>> > >>> > @@ -127,8 +128,8 @@ void >>> > Logger::initialize(const String &name, int priority, bool >>> > flush_per_log, >>> > std::ostream &out) { >>> > appender = new FlushableOstreamAppender("default", out, >>> > flush_per_log); >>> > - Logging::Layout* layout = new Logging::BasicLayout(); >>> > - //Logging::Layout* layout = new MicrosecondLayout(); >>> > + Logging::PatternLayout* layout = new Logging::PatternLayout(); >>> > + layout->setConversionPattern("[%d{%Y-%m-%d %H:%M:%S}] %p %c %x: %m >>> > %n"); >>> > appender->setLayout(layout); >>> > logger = &(Logging::Category::getInstance(name)); >>> > logger->addAppender(appender); >>> > >>> > >>> > After this mod, the output string is much better. eg: >>> > [2009-07-17 14:53:57] INFO Hypertable.RangeServer : >>> > >>> > >>> > 2. Besides, log should support rolling. At present, the log files'll >>> > grow very large as time goes. >>> > > >>> > >>> >>> >>> >> >> >> > > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Hypertable Development" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/hypertable-dev?hl=en -~----------~----~----~----~------~----~------~--~---
