Favored nodes version 1 did not see much adoption and was hard to use or unusable, depending on who you ask, so please expect a focus on usability and feature completeness when/if v2 is reviewed for possible inclusion in 2.0. While I am not suggesting this is the case, if the v2 candidate shares the feature incompleteness or usability problems of its predecessor it should not go in.
> On Nov 21, 2016, at 6:01 PM, Thiruvel Thirumoolan > <thiru...@yahoo-inc.com.INVALID> wrote: > > We would like to add the favored node enhancements as part of > https://issues.apache.org/jira/browse/HBASE-15531 to 2.0. Most of the code > should be new, there will be very less code changes to Master/AM etc. > > -Thiruvel > > On Monday, November 14, 2016, 10:06:49 PM PST, Anoop John > <anoop.hb...@gmail.com> wrote:>- Anyone (Ram, Anoop?) wants to post a > high-level writeup on the current > up-to-date state of offheaping? > > Off heaping the read path (HBASE-11425) this is completed some time > back. As per my knowledge at least 2 users back ported this work to > 1.x based version and even running in production. Am leaving it them > for giving details.. > > Write path off heaping HBASE-15179 is the umbrella issue. As u can > see most of the sub tasks are done by now.. Mainly 2 more sub tasks > are yet to get committed. Both are some what bigger sized also.. > Patch is already available for them. We hope it can be completed by > Nov end. > > Some more off heaping work, (in compaction path etc) might be taken up > after write path work. Alibaba guys might be joining with that. Had > some discuss with them.. We will come up with jira and doc for that > before actual work begin. But as Enis said ya all that can go in 2.x > (x>0) :-) > > Just giving some high level status update. Thanks .. > > -Anoop- > > >> On Tue, Nov 15, 2016 at 7:31 AM, Nick Dimiduk <ndimi...@gmail.com> wrote: >> Enis, by your criteria it seems the log4j2 upgrade ticket should be >> considered for 2.0 as well: HBASE-12341. >> >>> On Monday, November 14, 2016, Enis Söztutar <enis....@gmail.com> wrote: >>> >>> Another way to look at whether a feature is a "blocker" or not for 2.0 is a >>> simple test: >>> - Can this feature be committed for 2.1, 2.2, etc or not assuming it does >>> not make into 2.0. >>> >>> It is a simple test, but affectively validates whether the feature is >>> client-visible and have backwards compatibility implications. If it is ok >>> to bring the feature in for 2.1 or later, it means that the release should >>> never block on it. Otherwise, we should at least have the API / Client >>> visible parts of it in 2.0 if not the full thing. >>> >>> Looking at the features under discussion through this lens reveals easier >>> decision points I think. A couple of examples: >>> - Java async client / C++ Client: It is independent work, can come in 2.1 >>> or 2.2, etc. Whenever it is completed. >>> - Offheaping : transparent ti clients, so can come in incrementally in >>> 2.0, 2.1, 2.2, etc. >>> >>> On the other hand, 1.x releases have been going on for >1.5 years, and will >>> likely go on for another year at least. So the choices (dependency, APIs, >>> etc) that we make now for 2.0 will likely live for >2 years. In that >>> respect I would rather be a bit more aggressive in terms of dropping >>> support for older stuff and updating wherever we can. For example require >>> Hadoop-2.6+, Java-8, etc. >>> >>> Enis >>> >>> On Mon, Nov 14, 2016 at 12:15 PM, Mikhail Antonov <olorinb...@gmail.com >>> <javascript:;>> >>> wrote: >>> >>>> Great to see progress here. >>>> >>>> As I'm reading through the list, here're some high-level questions I >>> have: >>>> >>>> - Regarding the work on new AssignmentManager - any notes on >>>> perf/stability testing? Are you guys running tips of master branch >>> through >>>> ITBLL setup? >>>> - Anyone (Ram, Anoop?) wants to post a high-level writeup on the >>> current >>>> up-to-date state of offheaping? >>>> - Logical clock - at this point is it more like a nice to have feature >>>> than "need to be done before 2.0"? What are the features blocked or >>>> affected by lack thereof? >>>> >>>> -Mikhail >>>> >>>> On Sun, Nov 13, 2016 at 3:57 PM, Stack <st...@duboce.net <javascript:;>> >>> wrote: >>>> >>>>> Thanks for the writeup Stephen. >>>>> >>>>> See below. >>>>> >>>>> On Fri, Nov 11, 2016 at 5:15 PM, Stephen Jiang < >>> syuanjiang...@gmail.com <javascript:;>> >>>>> wrote: >>>>> >>>>>> Hello, fellow HBASE developers, >>>>>> >>>>>> We are making progress towards HBASE 2.0 releases. I am using the >>>>>> following queries to search for on-going HBASE 2.0 feature work items >>>>>> (project = HBase AND (fixVersion = 2.0.0 OR affectedVersion = 2.0.0) >>>> AND >>>>>> resolution is EMPTY AND (issuetype != Bug AND issuetype != Test AND >>>>>> issuetype != Sub-task) ORDER BY issuetype DESC), at this time, we >>> have >>>>>> 247! That is a lot. At some time in near future, we definitely need >>>> to >>>>>> trim down the list; otherwise, 2.0 will never be released. >>>>>> >>>>>> >>>>> Agree. Suggest we all do our own review but at a certain stage, it is >>> up >>>> to >>>>> the RMs to make a call on what shape of 2.0 will be. >>>>> >>>>> >>>>> >>>>>> For now, Matteo and I are tracking some big projects that are >>> on-going: >>>>>> >>>>>> (1). HBASE-14350 the stable Assignment Manager (using Procedure V2) >>>>>> - This is a blocker to have branch-2 cut. In the past few weeks, we >>>> made >>>>>> good progress and majority of implementation is done. The patches >>> are >>>>>> under review and testing. Matto is drafting a document for review. >>>>>> >>>>>> (2). HBASE-14414 Backup/Restore Phase 3 >>>>>> - Currently it is blocked by HBASE-14123. The giant HBASE-14123 >>>> patches >>>>>> was discussed and reviewed from the community (see >>>>>> http://www.mail-archive.com/dev@hbase.apache.org/msg41090.html for >>> the >>>>>> long >>>>>> discussion); and all feedback were taken care of in the latest patch. >>>>>> Currently I marked HBASE-14123 as 2.0 blocker, as without it, the >>>> further >>>>>> develpment of backup/restore is blocked - the backup/restore is a key >>>>>> enterprise feature for 2.0 release. I think HBASE-14123 is ready for >>>>>> another round of vote. >>>>>> >>>>>> >>>>> IMO, not a blocker since it ancillary function but something we should >>>> get >>>>> in. >>>>> >>>>> >>>>> >>>>>> (3). HBASE-15179 Offheap >>>>>> - This is another important feature that I think it is MUST for 2.0. >>>>> Since >>>>>> stack works closely with Intel developers, he has some insight on >>> this >>>>>> project: "Intel are betting on this. Alibaba are using the offheap >>> read >>>>>> path and interested in write path too. This work is still very much >>> up >>>> in >>>>>> the air and being worked out as we go (especially the Y! Israel >>>> inmemory >>>>>> compaction component). It is a little shakey dependent on mslab >>>> pooling, >>>>>> blockcache being on by default, async wal being default, and then >>>>> dependent >>>>>> on lots of perf and ITBLL testing." >>>>>> >>>>>> >>>>> The above is from a private, hurried email that does not do the current >>>>> state justice. >>>>> >>>>> Ram and Anoop are the authorities on offheaping and have been keeping >>> up >>>>> their state in an associated doc. The lads might be persuaded do a >>>> summary >>>>> of current state here. >>>>> >>>>> Ditto for inmemory compaction. Our Anastasia and Eshcar are working >>> hard >>>> at >>>>> the moment doing laste pieces and perf testing. Maybe we can a status >>>>> dumped here in dev. >>>>> >>>>> >>>>> >>>>>> (4). HBASE-16952 Protobuf3 >>>>>> - Good news, stack got the majority of work done already. This is a >>>> MUST >>>>>> for 2.0 release. Now we only have a small sub-task HBASE-16967 >>>>> (findbugs) >>>>>> left. >>>>>> >>>>>> >>>>> To be clear, pb3 is under the offheaping umbrella. Let's add async WAL >>> to >>>>> this list, also needed by offheaping. >>>>> >>>>> Will be back after taking a look at current 2.0 list. >>>>> >>>>> Agree that hbase2 needs to support hadoop3. >>>>> >>>>> St.Ack >>>>> >>>>> >>>>> >>>>> >>>>>> (5). HBASE-14070 Logical Clock >>>>>> - This needs to be done before 2.0 release. At this time, seems not >>>>> making >>>>>> much progress. >>>>>> >>>>>> (6). HBASE-16833 JAVA Async Client >>>>>> - A lot of progress was made by Duo and his associates. We should be >>>> in >>>>> a >>>>>> good shape in JAVA client when 2.0 release. >>>>>> >>>>>> (7). HBASE-14850 C++ Async Client >>>>>> - Another project that is making good progress. I know some customer >>>>> wants >>>>>> this. This is long overdue project. Hopefully we will make those >>>>> customer >>>>>> happy in 2.0 release with a good performed C++ client. >>>>>> >>>>>> Please let me know other on-going projects that needs to be in HBASE >>>> 2.0 >>>>>> release (stack mentioned "logical file system tier", but I am not >>> sure >>>>>> whether it would make enough progress to have a realistic chance >>> making >>>>>> 2.0). As I said at the beginning of this email, at some point soon, >>> we >>>>>> will be more picky and trim down the unfinished features in 2.0. >>>>>> >>>>>> Happy Weekend! >>>>>> Stephen >>>>>> >>>>> >>>> >>>> >>>> >>>> -- >>>> Thanks, >>>> Michael Antonov >>>>
