Thank you so much, Jonathan Gray and Jean-Daniel Cryans,
1. We will try to do more research and try to do something for the
AddColumn/DeleteColumn issue.
2. It's my fault, I missed to check the HTableDescriptor metadata.
It's great we can use MAX_FILESIZE and MEMSTORE_FLUSHSIZE to control
different table.
3. Yes, the total memstore heap threshold and the HLog.cleanOldLogs can
flush the old memstores.
My scenario is:
The columnfamilies/stores are defined by time-ranges (such days). So,
the columnfamilies/stores of yesterday will be quiet today.
Thank you again.
Schubert Zhang
On Fri, Jan 21, 2011 at 2:48 AM, Jean-Daniel Cryans <[email protected]>wrote:
> To complement what jgray answered...
>
> >
> > 2. Different memstore/memtable-size and region-size for different table.
> > This feature is useful for different tables for different
> applications.
> > Is there a plan to support it?
>
> Have you taken a look at the table properties? In the shell you can
> tune MAX_FILESIZE and MEMSTORE_FLUSHSIZE to do exactly what you
> described.
>
> >
> > 3. Periodic flush.
> > Current HBase only flush memstore to HFile by memory size thresholds.
> > If a Region-Store is quiet for a long time (such as 2 hours) for a
> > write-dense application, the periodic flush can free more memory.
> > Is there a plan to support it?
>
> If a region isn't written to, and there's a write-heavy job going on,
> then as the HLogs get created it's easy to reach the max number of
> HLogs a RS can get (32 by default) and after that point the memstores
> for the oldest edits will be flushed. If it's really inserting a lot
> of data, I expect that it can takes less than an hour to force flush
> unused memstores. Check out HLog.cleanOldLogs and
> HLog.findMemstoresWithEditsEqualOrOlderThan
>
> It's not really periodic or easily configurable tho.
>
> J-D
>
