WAL is Write Ahead Log - aka redolog, txlog, etc etc. As HBase does writes, we persist in the log first, then make changes to memory.
Furthermore, the WAL is for the entire regionserver - multiple regions, multiple tables. HFile is one region only. When we need to flush memtable, we are creating new HFiles for reads to read against. On Sat, Dec 19, 2009 at 7:26 PM, Zhenyu Zhong <[email protected]> wrote: > I am very appreciated for such a great documentation. > > I have a question. > > When Memstore is flushed to disk, is it flushed to HFile or WAL? Is Memstore > also handled by RegionServer? > The document also says there is another thread serving the request in > Regionserver which writes the data to HFile in HDFS. > I assume the data in memstore will be flushed to HFile via RS, am I right? > If not, what is the difference between the data on WAL and the data in > HFile? > > How often does HBase truncate the WAL? > > thanks > zhenyu > > > > On Fri, Dec 18, 2009 at 3:42 AM, Wim Van Leuven < > [email protected]> wrote: > >> I think that'll be your call, St.Ack. >> >> -----Original Message----- >> From: [email protected] [mailto:[email protected]] On Behalf Of stack >> Sent: donderdag 17 december 2009 17:30 >> To: [email protected] >> Subject: Re: Hbase store format >> >> It'd be kind of a pain converting it to that horrid xdoc format of >> forrest's. Maybe we should start up an hbase docbook of the architecture >> and bring that along with versions? >> >> St.Ack >> >> >> On Thu, Dec 17, 2009 at 6:47 AM, Lars George <[email protected]> >> wrote: >> >> > Should we have it in the same place and format as your "HBase on Windows" >> > article? I'd say that makes the most sense. >> > >> > >> > On Dec 17, 2009, at 9:36, "Wim Van Leuven" >> <[email protected]> >> > wrote: >> > >> > +1G That way it'll be maintained also. >> >> >> >> -----Original Message----- >> >> From: Ryan Rawson [mailto:[email protected]] >> >> Sent: woensdag 16 december 2009 23:22 >> >> To: [email protected] >> >> Subject: Re: Hbase store format >> >> >> >> I concur, the article is really core documentation right now - as in >> >> it accurately describes the architecture and bits and bytes of the >> >> system. >> >> >> >> It's always gratifying to see your code being diagrammed as your >> >> originally intended/wrote it. >> >> >> >> On Wed, Dec 16, 2009 at 2:00 PM, Wim Van Leuven >> >> <[email protected]> wrote: >> >> >> >>> Or just add the full article to the wiki? Isn't this terrific article >> >>> just >> >>> core architecture stuff for HBase? >> >>> >> >>> -----Original Message----- >> >>> From: Stack [mailto:[email protected]] >> >>> Sent: woensdag 16 december 2009 22:46 >> >>> To: [email protected] >> >>> Subject: Re: Hbase store format >> >>> >> >>> +1 >> >>> >> >>> >> >>> >> >>> On Dec 16, 2009, at 1:39 PM, Lars George <[email protected]> >> wrote: >> >>> >> >>> Maybe we should add this to the article page up on wiki? ;) >> >>>> >> >>>> On Wed, Dec 16, 2009 at 10:12 PM, Andrew Purtell >> >>>> <[email protected]>wrote: >> >>>> >> >>>> >> http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.html >> >>>>> >> >>>>> >> >>>>> >> >>>>> ----- Original Message ---- >> >>>>> >> >>>>>> From: kishore g <[email protected]> >> >>>>>> To: [email protected] >> >>>>>> Sent: Wed, December 16, 2009 1:03:05 PM >> >>>>>> Subject: Hbase store format >> >>>>>> >> >>>>>> Hi, >> >>>>>> >> >>>>>> Whats the current implementation in hbase 0.20.2 for storing data. >> >>>>>> >> >>>>> MapFile, >> >>>>> >> >>>>>> HFile, TFile. >> >>>>>> >> >>>>>> Also any pointers to links explaining the format will be helpful >> >>>>>> >> >>>>>> thanks, >> >>>>>> Kishore G >> >>>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>>> >> >>> >> >>> >> >> >> >> >
