Reviving the thread. Getting dynamic config right will be a big undertaking, and we want to move ahead with the patch since it is useful.
It seems the best way forward for the patch HBASE-15128 is to remove the setSwitch() call, and instead do set_merge_switch and set_split_switch calls so that we can get this in. Since balancer, and other master janitors already work this way, it should not be controversial. Matteo, is that acceptable? >Ideally we may want to have something committable and rollback-able with versions? One learning we had was that HBase should not be filling the role of Puppet / Chef or other provisioners. Enis On Wed, Feb 3, 2016 at 10:31 PM, Mikhail Antonov <olorinb...@gmail.com> wrote: > Looked through the comments and glanced at the patch. > > I generally think that the patch addressing specific existing problem > shouldn't be blocked on generic big feature like dynamic configuration > framework, so would be good to get something committable sooner rather then > latter. > > It seems the only point of controversy is whether we should have generic > setSwitch kind of call and kind of open up some API for that sort of > dynamic configuration, or rather specific setBalancer(), > forbidSplitsAndMerges(). > > So short term, why not to get a patch without generic kind of switches > (should be pretty easy change?), seems like people would agree on it, > commit it to fix existing issue, and then move on to generalize dynamic > configuration? > > I tend to agree that creating api to store generic switches in ZK while not > abandoning idea of generic dynamic configuration framework looks like > slippery slope. Right now we have well-known N things kept in ZK (I admit I > have contributed to that number), but I wouldn't have unknown X things > there. > > (Speaking of which.. I think most people would like to be able to hotswap > most of configuration properties without having to roll all the servers, > and do that from rpc/shell/some other interface, not just reload conf file. > In that case we would need to decide first on the persistent store for it > and consistency around it. > > Ideally we may want to have something committable and rollback-able with > versions? To cover cases like: I changed the parameter, say rpc timeout, > from command line, now I expect it to persist and survive over the next > cluster restart. But what if I don't like the change I made, I would want > to easily discard the change and say "whatever was in configuration was > good, let's go back to it"?) > > -Mikhail > > On Wed, Feb 3, 2016 at 9:50 PM, 张铎 <palomino...@gmail.com> wrote: > > > My opinion, use conf file as default, and use code to override the config > > in conf file. > > And we need to persist the overridden value somewhere, not zk I'd say. I > > agree with Matteo that we should reduce the dependency of zk. Maybe we > > could introduce an hbase:config table? > > > > And for the dynamic configuration framework, it is difficult to treat all > > configurations in the same way I think. I prefer changing all > > configurations with a shell command(the actual implementation should an > RPC > > call to master maybe), but if we have regionservers with different > > hardwares in cluster, they may have different configurations so modify > conf > > file directly on the machine is a more proper way? I have no idea... > > > > Thanks. > > > > 2016-02-04 11:40 GMT+08:00 Enis Söztutar <e...@hortonworks.com>: > > > > > This came up in the review for HBASE-15128, but we thought that we > should > > > surface this discussion in dev@. > > > > > > The idea for HBASE-15128 is that, we should have a dynamic switch in > > master > > > so that the operator (or HBCK) can disable all region splits or merges. > > > > > > We currently have such dynamic switches for balancer and normalizer > which > > > keep their state in zk. We also have a switch for catalog janitor which > > > keeps its state only in memory (not sure why). For the new switches > Heng > > > Chen implemented a "set_switch" RPC to master, a switch type, a zk > node. > > > The idea is to move balancer, catalog janitor and normalizer switches > to > > > this after this patch. > > > > > > However, Matteo raised a point about this framework being an > alternative > > > way of doing dynamic configuration. The problem is that we do not have > a > > > dynamic conf framework that we can piggyback on, and since balancer and > > > other switches already work this way, it does not make sense to block > the > > > issue. What do you guys think? If we want to implement dynamic conf, > how > > > would we handle things like enabling / disabling balancer from conf and > > or > > > from code? Any good ideas? > > > > > > Enis > > > > > > > > > -- > Thanks, > Michael Antonov >
