On Tue, Dec 3, 2013 at 11:42 AM, Enis Söztutar <enis....@gmail.com> wrote:
> On Mon, Dec 2, 2013 at 10:20 PM, Jonathan Hsieh <j...@cloudera.com> wrote: > > > > Deveraj: > > > Jonathan Hsieh, WAL per region (WALpr) would give you the locality (and > > hence HDFS short > > > circuit) of reads if you were to couple it with the favored nodes. The > > cost is of course more WAL > > > files... In the current situation (no WALpr) it would create quite some > > traffic cross machine, no? > > > > I think we all agree that wal per region isn't efficient on today's > > spinning hard drive world where we are limited to a relatively low budget > > or seeks (though may be better in the future with SSD's). > > > > WALpr makes sense in fully SSD world and if hdfs had journaling for writes. > I don't think anybody > is working on this yet. what do you mean by journaling for writes? do you mean where sync operations update length at the nn on every call? > Full SSD clusters are already in place (pinterest > for example), so I > think having WALpr as a pluggable implementation makes sense. HBase should > work with both > WAL-per-regionserver (or multi) or WAL-per-region. > > > I agree here. > > > > With this in mind, I actually I making the case that we would group the > all > > the regions from RS-A onto the same set of preferred regions servers. > This > > way we only need to have one or two other RS's tailing the RS. > > > > So for example, if region X, Y and Z were on RS-A and its hlog, the > shadow > > region memstores for X, Y, and Z would be assigned to the same one or two > > other RSs. Ideally this would be where the HLog files replicas have > > locality (helped by favored nodes/block affinity). Doing this, we hold > the > > number of readers on the active hlogs to a constant number, do not add > any > > new cross machine traffic (though tailing currently has costs on the NN). > > > > One inefficiency we have is that if there is a single log per RS, we end > up > > reading all the logs to tables that may not have the shadow feature > > enabled. However, with HBase multi-wals coming, one strategy is to shard > > wals to a number on the order of the number of disks on a machine (12-24 > > these days). I think the a wal per namespaces (this could be used to > have > > a wal per table) of the hlog would make sense. This way of shardind the > > hlog would reduce the amount of reading of irrelevant log entries on a > log > > tailing scheme. It would have the added benefit of reducing the log > > splitting work reducing MTTR and allowing for recovery priorities if the > > primaries and shadows also go down. (this is an generalization of the > > separate out the META into a separate log idea). > > > > Jon. > > > > -- > > // Jonathan Hsieh (shay) > > // Software Engineer, Cloudera > > // j...@cloudera.com > > > -- // Jonathan Hsieh (shay) // Software Engineer, Cloudera // j...@cloudera.com
