2009/7/17 孔令华 <[email protected]>: > Thanks, Doug & Luke > > In my opinion, this kind of performance issue is not in the critical path > (just in mind, no data support, but I think RPCs and disk-seeking would cost > much more time). A clear log is very important, specially for normal users.
We sometime needs to turn on debugging to show a lot more data, we want to make sure the timing is not affected too much. Note, the logging system is not for range server only, it can be used for many other tools that could be cpu intensive. With the current format you can display it in any timezones easily, which would be important when you have data centers across the world. > BTW，according to our tests, using BloomFilter with the default params costs > too many mems (a RangeServer with 1600 Ranges cost about 9GB, and about 2GB > without BloomFilter). And I found that BloomFilter actually need these mems > instead of fragments. Are there a lot of columns in the table? Can you use the bloomfilter=rows to see how much memory it uses? 7GB at 1% false positive rate can accommodate 6 billion items! Do you want an option to provide fixed amount memory per access group? > And at present, BloomFilter affects the whole system, which I think should > be AccessGroup-wide. Using it with the table schema would be a better > choice(eg. appoint them when create table). You can certainly turn it off globally and use the options at create table time now. Doug is working on a lazy loading scheme, so that it uses less memory (bloomfilter and index are not loaded unless needed) __Luke > 2009/7/18 Luke <[email protected]> >> >> You can alias lcat='perl -pe "s/^\d+/localtime($&)/e"' >> >> Then you can lcat *.log to see more readable timestamps. Time >> conversion is very slow, especially it needs grab a global lock for >> timezones. Storing timestamps in timezone neutral format has a lot of >> benefits. >> >> On Fri, Jul 17, 2009 at 7:45 AM, Phoenix<[email protected]> wrote: >> > >> > 1. At present, the time field of the log is very hard to read. It's >> > unreadable at some point. >> > eg. 1247799471 INFO Hypertable.Master : >> > >> > There're some configurable options that can opt the output string. >> > eg: >> > >> > diff --git a/src/cc/Common/Logger.cc b/src/cc/Common/Logger.cc >> > index 301a2d1..985f231 100644 >> > --- a/src/cc/Common/Logger.cc >> > +++ b/src/cc/Common/Logger.cc >> > @@ -28,6 +28,7 @@ >> > #include <log4cpp/Layout.hh> >> > #include <log4cpp/NDC.hh> >> > #include <log4cpp/Priority.hh> >> > +#include <log4cpp/PatternLayout.hh> >> > >> > #include "Logger.h" >> > >> > @@ -127,8 +128,8 @@ void >> > Logger::initialize(const String &name, int priority, bool >> > flush_per_log, >> > std::ostream &out) { >> > appender = new FlushableOstreamAppender("default", out, >> > flush_per_log); >> > - Logging::Layout* layout = new Logging::BasicLayout(); >> > - //Logging::Layout* layout = new MicrosecondLayout(); >> > + Logging::PatternLayout* layout = new Logging::PatternLayout(); >> > + layout->setConversionPattern("[%d{%Y-%m-%d %H:%M:%S}] %p %c %x: %m >> > %n"); >> > appender->setLayout(layout); >> > logger = &(Logging::Category::getInstance(name)); >> > logger->addAppender(appender); >> > >> > >> > After this mod, the output string is much better. eg: >> > [2009-07-17 14:53:57] INFO Hypertable.RangeServer : >> > >> > >> > 2. Besides, log should support rolling. At present, the log files'll >> > grow very large as time goes. >> > > >> > >> >> > > > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Hypertable Development" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/hypertable-dev?hl=en -~----------~----~----~----~------~----~------~--~---
