Using this lens, backup / restore should come in 2.0 with client facing APIs properly defined.
On Mon, Nov 14, 2016 at 2:57 PM, Enis Söztutar <enis....@gmail.com> wrote: > Another way to look at whether a feature is a "blocker" or not for 2.0 is a > simple test: > - Can this feature be committed for 2.1, 2.2, etc or not assuming it does > not make into 2.0. > > It is a simple test, but affectively validates whether the feature is > client-visible and have backwards compatibility implications. If it is ok > to bring the feature in for 2.1 or later, it means that the release should > never block on it. Otherwise, we should at least have the API / Client > visible parts of it in 2.0 if not the full thing. > > Looking at the features under discussion through this lens reveals easier > decision points I think. A couple of examples: > - Java async client / C++ Client: It is independent work, can come in 2.1 > or 2.2, etc. Whenever it is completed. > - Offheaping : transparent ti clients, so can come in incrementally in > 2.0, 2.1, 2.2, etc. > > On the other hand, 1.x releases have been going on for >1.5 years, and will > likely go on for another year at least. So the choices (dependency, APIs, > etc) that we make now for 2.0 will likely live for >2 years. In that > respect I would rather be a bit more aggressive in terms of dropping > support for older stuff and updating wherever we can. For example require > Hadoop-2.6+, Java-8, etc. > > Enis > > On Mon, Nov 14, 2016 at 12:15 PM, Mikhail Antonov <olorinb...@gmail.com> > wrote: > > > Great to see progress here. > > > > As I'm reading through the list, here're some high-level questions I > have: > > > > - Regarding the work on new AssignmentManager - any notes on > > perf/stability testing? Are you guys running tips of master branch > through > > ITBLL setup? > > - Anyone (Ram, Anoop?) wants to post a high-level writeup on the > current > > up-to-date state of offheaping? > > - Logical clock - at this point is it more like a nice to have feature > > than "need to be done before 2.0"? What are the features blocked or > > affected by lack thereof? > > > > -Mikhail > > > > On Sun, Nov 13, 2016 at 3:57 PM, Stack <st...@duboce.net> wrote: > > > > > Thanks for the writeup Stephen. > > > > > > See below. > > > > > > On Fri, Nov 11, 2016 at 5:15 PM, Stephen Jiang < > syuanjiang...@gmail.com> > > > wrote: > > > > > > > Hello, fellow HBASE developers, > > > > > > > > We are making progress towards HBASE 2.0 releases. I am using the > > > > following queries to search for on-going HBASE 2.0 feature work items > > > > (project = HBase AND (fixVersion = 2.0.0 OR affectedVersion = 2.0.0) > > AND > > > > resolution is EMPTY AND (issuetype != Bug AND issuetype != Test AND > > > > issuetype != Sub-task) ORDER BY issuetype DESC), at this time, we > have > > > > 247! That is a lot. At some time in near future, we definitely need > > to > > > > trim down the list; otherwise, 2.0 will never be released. > > > > > > > > > > > Agree. Suggest we all do our own review but at a certain stage, it is > up > > to > > > the RMs to make a call on what shape of 2.0 will be. > > > > > > > > > > > > > For now, Matteo and I are tracking some big projects that are > on-going: > > > > > > > > (1). HBASE-14350 the stable Assignment Manager (using Procedure V2) > > > > - This is a blocker to have branch-2 cut. In the past few weeks, we > > made > > > > good progress and majority of implementation is done. The patches > are > > > > under review and testing. Matto is drafting a document for review. > > > > > > > > (2). HBASE-14414 Backup/Restore Phase 3 > > > > - Currently it is blocked by HBASE-14123. The giant HBASE-14123 > > patches > > > > was discussed and reviewed from the community (see > > > > http://www.mail-archive.com/dev@hbase.apache.org/msg41090.html for > the > > > > long > > > > discussion); and all feedback were taken care of in the latest patch. > > > > Currently I marked HBASE-14123 as 2.0 blocker, as without it, the > > further > > > > develpment of backup/restore is blocked - the backup/restore is a key > > > > enterprise feature for 2.0 release. I think HBASE-14123 is ready for > > > > another round of vote. > > > > > > > > > > > IMO, not a blocker since it ancillary function but something we should > > get > > > in. > > > > > > > > > > > > > (3). HBASE-15179 Offheap > > > > - This is another important feature that I think it is MUST for 2.0. > > > Since > > > > stack works closely with Intel developers, he has some insight on > this > > > > project: "Intel are betting on this. Alibaba are using the offheap > read > > > > path and interested in write path too. This work is still very much > up > > in > > > > the air and being worked out as we go (especially the Y! Israel > > inmemory > > > > compaction component). It is a little shakey dependent on mslab > > pooling, > > > > blockcache being on by default, async wal being default, and then > > > dependent > > > > on lots of perf and ITBLL testing." > > > > > > > > > > > The above is from a private, hurried email that does not do the current > > > state justice. > > > > > > Ram and Anoop are the authorities on offheaping and have been keeping > up > > > their state in an associated doc. The lads might be persuaded do a > > summary > > > of current state here. > > > > > > Ditto for inmemory compaction. Our Anastasia and Eshcar are working > hard > > at > > > the moment doing laste pieces and perf testing. Maybe we can a status > > > dumped here in dev. > > > > > > > > > > > > > (4). HBASE-16952 Protobuf3 > > > > - Good news, stack got the majority of work done already. This is a > > MUST > > > > for 2.0 release. Now we only have a small sub-task HBASE-16967 > > > (findbugs) > > > > left. > > > > > > > > > > > To be clear, pb3 is under the offheaping umbrella. Let's add async WAL > to > > > this list, also needed by offheaping. > > > > > > Will be back after taking a look at current 2.0 list. > > > > > > Agree that hbase2 needs to support hadoop3. > > > > > > St.Ack > > > > > > > > > > > > > > > > (5). HBASE-14070 Logical Clock > > > > - This needs to be done before 2.0 release. At this time, seems not > > > making > > > > much progress. > > > > > > > > (6). HBASE-16833 JAVA Async Client > > > > - A lot of progress was made by Duo and his associates. We should be > > in > > > a > > > > good shape in JAVA client when 2.0 release. > > > > > > > > (7). HBASE-14850 C++ Async Client > > > > - Another project that is making good progress. I know some customer > > > wants > > > > this. This is long overdue project. Hopefully we will make those > > > customer > > > > happy in 2.0 release with a good performed C++ client. > > > > > > > > Please let me know other on-going projects that needs to be in HBASE > > 2.0 > > > > release (stack mentioned "logical file system tier", but I am not > sure > > > > whether it would make enough progress to have a realistic chance > making > > > > 2.0). As I said at the beginning of this email, at some point soon, > we > > > > will be more picky and trim down the unfinished features in 2.0. > > > > > > > > Happy Weekend! > > > > Stephen > > > > > > > > > > > > > > > -- > > Thanks, > > Michael Antonov > > >
