On Tue, Nov 15, 2016 at 11:44 AM <tof...@ymail.com.invalid> wrote: > > In the absence of more information, intuition says master carries meta > to avoid a whole class of problems. > Off-hand I think the class of problems we'll eliminate are problems that > are well understood and being constantly dealt with and hardened to this > day (ie puts to a region). > > I think we have to evaluate whether the new pv2 master works with > remote meta updates and the fact that those updates can fail partially or > succeed without theI think failing meta updates need to be dealt with > either way AFAIK eventually procedure state will be stored in HDFS which is > also a distributed system. > > I don't think these are really equivalent. If you encounter a bad DN in the write pipeline, you can construct a new write pipeline with a new set of DNs. If you encounter an error updating meta, which other copy of meta do you try to write to?
Also, I don't really see Pv2 as being able to solve the whole reliability problem here. If it leads to more reliable region assignments that's great. But if a region is assigned, but clients can't see it because meta can't be updated, I don't think that's a big improvement. The region's data is offline from the client perspective until it can actually see the new region location.
