+1 go from my POV.
On 10/31/17 10:07 AM, Stack wrote:
I want to push an alpha-4 today. A few items didn't make it (HBASE-19092).
They need more time. We'll pull them in for beta-1. CP API is basically
done. There may be some changes for beta-1 but hopefully only changes
informed by experience trying to port an existing Coprocessor to hbase2.
Shout if there is anything that needs to make alpha-4.
Thanks,
St.Ack
On Sat, Oct 28, 2017 at 2:48 PM, Josh Elser <[email protected]> wrote:
Yup, that was going to be my plan, Mike!
Making a pass now, and will check back later tonight again. I see others
have already done some work today on this front.
On 10/27/17 11:38 PM, Mike Drob wrote:
Josh - Do you want to kick off a bunch of QA runs? (Do you know how to do
it directly on the jenkins job, so you don't have to bother with JIRA
uploads)
If you're busy, then I can make time tomorrow or Sunday to kick off jobs,
but I want to make sure we're not duplicating effort and jenkins cycles.
On Fri, Oct 27, 2017 at 7:10 PM, Josh Elser <[email protected]> wrote:
My turn to bump ;)
By my take: HBASE-18770 and HBASE-19092 are the only issues that remain
needing some more work. The rest are just awaiting a good QA run.
Unless I hear otherwise, I'll try to keep an eye on things over the
weekend, bump them along as necessary, and get them committed. Would be
great to be able get a vote up on Monday.
On 10/24/17 6:03 PM, Stack wrote:
Chatting with my coworker Mr. Mike Drob, we were batting back and forth
what remains to be done. Surfacing our thoughts here so you all clued
in....Or if you think otherwise, please speak up.
We have ~13 issues to land:
https://issues.apache.org/jira/projects/HBASE/versions/12341594 About
two
are meta-issues that are about process which leaves 11.
Duo and Zheng Hu are to merge the FilterList fixes improvements
(HBASE-19057, HBASE-18410 et al.). These are blocker because some
changed
API/semantic that we need to get out earlier rather than later.
Once the above is merged, HBASE-13346, change of Filter method names to
mention Cell instead of KeyValue can land.
HBASE-199048 needs a review (Anoop will probably do it), removing
IA.Private objects as params to MasterObserver... Hopefully this goes in
soon.
Duo is hard at work on trackers for flush and compaction for CPs
(HBASE-18905). How is HBASE-19033 looking Duo (facility for Tephra)?
I think HBASE-18906 (Phoenix Region#waitFor...) will evaporate after Duo
is
done w/ his work above.
I'm on HBASE-18770 bypass and HBASE-19077 restore some parity after all
the
purges allowing CPs do direct calls against Regions in same Host.
Anoop is on HBASE-19047 (Fixes) and Ram on cleanup of CellUtil.
Another day or two?
St.Ack
On Mon, Oct 23, 2017 at 2:52 PM, Stack <[email protected]> wrote:
On Mon, Oct 23, 2017 at 11:59 AM, Josh Elser <[email protected]> wrote:
+1
I was trying to work on helping out on the outstanding alpha-4 stuff
last
week -- will be continuing to try to do the same this week.
If you need any help, Stack, or if others need reviews where I haven't
noticed on my own: feel free to @mention me.
Thanks for the offer Josh. All items seem assigned and are being
actively
worked on. If you get a moment, reviews by you (or anyone else) helps
move
the process along.
We need to merge in HBASE-18410 branch to pick up Filter improvements.
Then HBASE-13346 can go in.
You are already helping out on HBASE-18906, thanks. Looks like that
will
be addressed by other alpha-4s about to land.
St.Ack
TODOs: https://issues.apache.org/jira/projects/HBASE/versions/12341594
On 10/23/17 12:53 PM, Stack wrote:
(Reviving this thread)
Lets push out alpha-4 this week. Alpha-4 is the release that has the
refactor of the Coprocessor API shutting down access to internals
marked
InterfaceAudience.Private.
The outstanding list is here:
https://issues.apache.org/jira/projects/HBASE/versions/12341594
Please push in anything marked alpha-4 that belongs to you.
If issue, talk out loud on this thread. If you need a review to land
an
item, shout on the issue and here; we'll help you out.
As is, items are coming along nicely I'd say. We need to merge the
filter
branch -- HBASE-18410 -- so APIs are finished for hbase2.
Post alpha-4, we'll have to hunt down our downstreamers and help them
test
on top of alpha-4 so rolling into beta-1, we have confidence our
downstreamers know what to expect (or we discover what we missed
BEFORE
we
beta-1).
Thanks for time,
S
On Fri, Sep 8, 2017 at 2:04 PM, Stack <[email protected]> wrote:
I'll put up an alpha3 RC Monday, probably Monday night. That should
be
time, if we all sprint, for the public-facing API fixes to be done.
I had a bunch of Coprocessor refactor and fixup scheduled for alpha3
but
it is plain that more time is needed (in spite of valiant effort so
far
by
Anoop, Duo, Appy, etc.). Therefore, lets run a 2.0.0-alpha-4 whose
theme is
"Coprocessor Fixup". Hopefully we can put an alpha-4 up by the
following
week.
We should then be ready for beta (beta == no new features, no API
changes,
just fixes).
Thanks,
St.Ack
On Thu, Aug 17, 2017 at 12:35 PM, Stack <[email protected]> wrote:
I put up the hbase-2.0.0-alpha2 release candidate. Please vote on
it.
For hbase-2.0.0-alpha3, the theme is solidifying API. I hope to
get a
release out in the next week or so.
I did a weeding of 2.0.0 issues over the last day. If folks are
interested in helping out, below are the items I think we need done
for
alpha3 (below are at least 'Critical' status, are API possibly
altering
items, and are absent those JIRAs that are making active progress,
i.e. the
HTD/HCD revamp by Chia-Ping Tsai). A project NOT listed that needs
doing is
what Andrew did comparing 1.3. and 1.4 APIs
* HBASE-18622 Mitigate compatibility concerns between branch-1 and
branch-2
This is to do what Andrew did between 1.3 and 1.4 branches only do
it
between branch-1 and branch-2.
* HBASE-10462 Recategorize some of the client facing Public /
Private
interfaces
This one is almost done. It could do with a finish, attention to
the
items in last comment, and then our codebase could do with another
sweep
after the spirit of this issue since a bunch has gone in since the
pass
that was the basis of this issue.
* HBASE-10504 Define Replication Interface
I was going to take a crack at this as part of the revamp forced by
'HBASE-15982 Interface ReplicationEndpoint extends Guava's Service'
but if
anyone else is interested, be my guest.
* HBASE-14996 Some more API cleanup for 2.0
Has a bunch of subtasks, some of which are being worked on. Needs
finishing.
* HBASE-14998 Unify synchronous and asynchronous methods in Admin
and
cleanup
Needs a pass. Small issue I think. Could also look at new
AsyncClient
and
make sure symmetry.
* HBASE-15607 Remove PB references from Admin for 2.0
Predicated on result of an ongoing DISCUSSION thread but needs to
be
done.
Rolling upgrade will have implications for our API. Would be good
to
try
it and figure what needs fixup (as said above, according to trial
by
Sean,
we might not be too bad here):
* HBASE-16060 1.x clients cannot access table state talking to 2.0
cluster
* HBASE-16550 Procedure v2 - Add AM compatibility for 2.x Master
and
1.x
RSs; i.e. support Rolling Upgrade from hbase-1 to -2.
* HBASE-17442 Move most of the replication related classes to
hbase-server package
The above would be good to do generally but it may make for ripples
in
API so would be good to do now.
* HBASE-18106 Redo ProcedureInfo and LockInfo
Balazs is working on this. The idea is that we avoid adding two new
types
to our API, two types that are nought but curtailed, read-only
views
on
internals. Input if you have time appreciated.
* HBASE-18596 A hbase1 cluster should be able to replicate to a
hbase2
cluster; verify
Esteban is looking at this one
* HBASE-9417 SecureBulkLoadEndpoint should be folded in core
* HBASE-17143 Scan improvement
Our Coprocessor Interface needs a tough edit. It exposes
implementations
marked audience Private and returns implementations rather than
Interfaces.
In a few locations, we allow returning an alternate implementation
altogether which is probably something we don't want a CP doing. To
that
end, the following issues started by Duo and Anoop need to be taken
to
the
finish line; ideally they'd have an owner:
* HBASE-18169 Coprocessor fix and cleanup before 2.0.0 release <=
The
umbrella issue.
* HBASE-18298 RegionServerServices Interface cleanup for CP expose
* HBASE-16769 Deprecate/remove PB references from MasterObserver
and
RegionServerObserver
Nice-to-haves:
* HBASE-15284 Make TimeRange constructors IA.Private and remove
unused
TimeRange constructors
* HBASE-10944 Remove all kv.getBuffer() and kv.getRow() references
existing in the code
This is the end of an old long-running project moving up on to Cell
Interface. We think it is done but for a few little items
(deprecate
KV
methods in MR and provide Cell versions instead...)
* HBASE-13271 Table#puts(List<Put>) operation is indeterminate;
needs
fixing
* HBASE-13346 Clean up Filter package for post 1.0
* HBASE-14255 Simplify Cell creation post 1.0
* HBASE-14997
Move compareOp and Comparators out of filter to client package
* HBASE-13740 Stop using Hadoop private interfaces
What about:
* HBASE-18601 Remove Htrace 3.2
As has been noted, the HTrace API is our 'trace' API.
If interested in any of the above and you need a legup, just ask in
the
issue and I'll be by....
Thanks,
St.Ack
On Mon, Aug 14, 2017 at 10:54 AM, Stack <[email protected]> wrote:
Heads-up:
I'm about to put up an hbase-2.0.0-alpha2 Release Candidate. Theme
is
updated dependencies, reliance on relocated popular libs (guava,
netty,
protobuf), purge of checked-in generated src, and
master-carries-no-regions
by default.
alpha3 I hope will follow soon after (end-of-August?). Its theme
will
be
settling the APIs and compatibility (At first blush, we are not
looking too
bad; our Sean ran some tests over weekend that have hbase-1 client
running
against an hbase-2 cluster....). The Coprocessor Interface revamp
should be
done by alpha3 (i.e. returning Interfaces rather than
Implementations, and
our shutdown of CPs accessing classes in hbase marked
InterfaceAudience).
We'll also have purged thirdparty classes from our API; e.g. guava
0.12
Service showing through in our replication API and protobufs in
Admin
Interface. On alpha3, we will have to do a bunch of outreach to
make
sure
our downstreamers are up on what is coming down the pipe.
Beta1 in mid-September?
I encourage you to check out the items marked for hbase2:
https://issues.apache.org/jira/projects/HBASE/versions/12327188
Edit
as
you see appropriate. Punt if you know the JIRA will not get any
attention
in next month or so.
A bunch of issues marked blocker are unassigned. I'll leave them
as
is
another while but I'll boot them soon.
While I have your attention:
+ I think we should leave thrift version at 0.9.3. Moving hbase
thrift
to 0.10.0 will break existing clients. The change is easy enough
if
folks
need to upgrade their hbase thrift. See HBASE-18591.
+ Upgrade from 0.94 is disallowed. You have to get to 1.0 first
(0.98?).
St.Ack
On Wed, Aug 2, 2017 at 9:43 AM, Stack <[email protected]> wrote:
On Tue, Aug 1, 2017 at 2:06 PM, Josh Elser <[email protected]>
wrote:
On 7/31/17 9:00 AM, Stack wrote:
On Mon, Jul 24, 2017 at 12:25 PM, Josh Elser<[email protected]>
wrote:
...
I like the idea of this also hitting 2.0 as it would make the
feature a
bit more "real", but am obviously a little nervous (I have no
reason
to be
nervous though). I am pretty happy with the feature in terms
of
how
much it
is covered via testing.
https://issues.apache.org/jira/browse/HBASE-17748
Sounds good to me. Whats involved? Backport? If so, +1 Josh.
Last think on space quota says that need doc too. See 'Space
Quota' in
here:
https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9i
Eu_ktczrlKHK8N4SZzs/edit#heading=h.wuw3a6jukzo5
Does this little section need an update Josh?
Thanks,
S
Yep, just a couple of cherry-picks. Good test coverage and some
docs
already included for 17748. Happy to put that on my plate if
you're good
with it. I can reasonably assume that no one is against it :)
I think I had knocked out docs for the "phase 1" stuff before we
merged it in from the original feature branch. I'll double check
and update
the gdoc. Perhaps this was just a timing thing.
Thanks Josh,
S
