[jira] [Created] (HBASE-19186) Unify to use bytes to show size in master/rs ui

[Guanghao Zhang (JIRA)]

Guanghao Zhang created HBASE-19186:
--

 Summary: Unify to use bytes to show size in master/rs ui
 Key: HBASE-19186
 URL: https://issues.apache.org/jira/browse/HBASE-19186
 Project: HBase
  Issue Type: Improvement
Reporter: Guanghao Zhang
Assignee: Guanghao Zhang
Priority: Minor


1. 10K ==> 10KB or 10M ==> 10MB or 10G => 10GB
2. remove "in bytes" in description



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: DISCUSSION: Minimum hbase1 version from which you can upgrade to hbase2 (1.2.x?)

[Stack]

On Sat, Nov 4, 2017 at 6:19 PM, Guanghao Zhang  wrote:

> Our internal branch is based on 0.98. And we plan rolling to 2.0. So I will
> take a try for rolling from 0.98 to 2.0. But we take a lot backport to our
> internal branch, like async client, netty rpc client, serial replication,
> throttling, some replication improvements and so on. So our rolling
> experience may not apply to community totally. I will post our rolling
> experience (which can apply to community 0.98 branch) after we rolling to
> 2.0 :-).
>
>
Let me try going from 0.98 then and see what is broke. Would be good if you
fellows could do one step rather than two.
S




> 2017-11-05 2:41 GMT+08:00 Stack :
>
> > On Fri, Nov 3, 2017 at 9:01 PM, Guanghao Zhang 
> wrote:
> >
> > > Can we rolling from 0.98 and 1.1 to 1.2? If this rolling is ok, user
> can
> > > rolling to 2.0 by two steps, 0.98 to 1.2, then 1.2 to 2.0.
> > >
> > >
> > Yes. They could do that. Would be a pain. Might be able to go from 0.98
> to
> > 2.0 though... I've not tried it.
> > St.Ack
> >
> >
> >
> >
> > > 2017-11-04 11:25 GMT+08:00 Nick Dimiduk :
> > >
> > > > 1.2 is good, but are we aware of anything that precludes 1.1? 0.98?
> On
> > > disk
> > > > compatibility (HFile, WAL, AMv2) should be the limiting factor here,
> > > right?
> > > > Wire protocols have been compatible all the while...
> > > >
> > > > On Fri, Nov 3, 2017 at 5:56 PM Zach York <
> zyork.contribut...@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > +1 for having the minimum (supported) hbase1 version be 1.2.x.
> > > > >
> > > > > On Fri, Nov 3, 2017 at 5:30 PM, Stack  wrote:
> > > > >
> > > > > > Over in the adjacent "[DISCUSS] hbase-2.0.0 compatibility
> > > expectations"
> > > > > > thread, we chatted some on what would be the minimum hbase-1.x
> > > version
> > > > > from
> > > > > > which you can upgrade to hbase-2.
> > > > > >
> > > > > > The last statement made on this topic by Sean was that only
> > upgrades
> > > > from
> > > > > > 1.2.x, our current stable offering, or later should be supported.
> > > > > >
> > > > > > There was no dissent.
> > > > > >
> > > > > > We all good w/ this? Speak up if you disagree else 1.2.x becomes
> > the
> > > > > > 'official' minimum.
> > > > > >
> > > > > > NOTES:
> > > > > >
> > > > > > + We need to agree on a minimum so we know what migrations to
> test.
> > > > > > + It might be possible to upgrade from versions before 1.2.x but
> we
> > > (or
> > > > > at
> > > > > > least I -- smile) won't have tried it or run verifications to
> > ensure
> > > > all
> > > > > > made it over (let us know if you successfully migrate from a
> > baseline
> > > > > that
> > > > > > precedes 1.2).
> > > > > > + Hopefully we can avoid requiring Users move to the latest on
> the
> > > 1.2
> > > > > > branch. This shouldn't be necessary doing a stop/start upgrade.
> It
> > > > might
> > > > > be
> > > > > > needed doing a rolling upgrade. Lets see.
> > > > > >
> > > > > > Thanks,
> > > > > > St.Ack
> > > > > >
> > > > >
> > > >
> > >
> >
>

[jira] [Resolved] (HBASE-19167) RowCounter not working on 2.0.0-alpha3

[Chia-Ping Tsai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai resolved HBASE-19167.

Resolution: Duplicate

The error doesn't happen in alpha4, but there are other classes which are not 
found in running RowCounter. see HBASE-19185.

> RowCounter not working on 2.0.0-alpha3
> --
>
> Key: HBASE-19167
> URL: https://issues.apache.org/jira/browse/HBASE-19167
> Project: HBase
>  Issue Type: Bug
>  Components: hadoop2, mapreduce, shell
>Affects Versions: 2.0.0-alpha-3
> Environment: Ubuntu 16.04, Distributed 3-Node Deployment, all running 
> 2.0.0-alpha3
>Reporter: Silvan Heller
>Priority: Blocker
> Fix For: 2.0.0-beta-1
>
>
> Steps to reproduce:
> * Create a table 'w' (or any other name)
> * Execute in shell: ./hbase org.apache.hadoop.hbase.mapreduce.RowCounter w 
> (Where 'w' is my tablename)
> Expected Behavior:
> * I receive how many rows my table has
> Actual Behavior:
> *  Jobs fail because org.apache.commons.lang3.ArrayUtils is not present on 
> the classpath. Detailed Stacktrace below.
> == STACKTRACE
> 2017-11-03 10:26:26,773 INFO  [main] mapreduce.Job: Task Id : 
> attempt_1507127903800_0002_m_01_2, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.commons.lang3.ArrayUtils
>   at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>   at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>   at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
>   at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>   at org.apache.hadoop.hbase.HConstants.(HConstants.java:1144)
>   at 
> org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.toScan(ProtobufUtil.java:1091)
>   at 
> org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.convertStringToScan(TableMapReduceUtil.java:576)
>   at 
> org.apache.hadoop.hbase.mapreduce.TableInputFormat.setConf(TableInputFormat.java:122)
>   at 
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76)
>   at 
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
>   at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:749)
>   at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
>   at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-19185) ClassNotFoundException: com.fasterxml.jackson.*

[Chia-Ping Tsai (JIRA)]

Chia-Ping Tsai created HBASE-19185:
--

 Summary: ClassNotFoundException: com.fasterxml.jackson.*
 Key: HBASE-19185
 URL: https://issues.apache.org/jira/browse/HBASE-19185
 Project: HBase
  Issue Type: Bug
Reporter: Chia-Ping Tsai
Assignee: Chia-Ping Tsai


HBASE-16338 update jackson to 2.9.1 which is different from hadoop-2.7.1. We 
should bring our bedding and towels and not be trying to use jackson 
dependencies from hadoop.






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (HBASE-19131) Add the ClusterStatus hook and cleanup other hooks which can be replaced by ClusterStatus hook

[Chia-Ping Tsai (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chia-Ping Tsai resolved HBASE-19131.

  Resolution: Fixed
Hadoop Flags: Reviewed

Push to branch-1.4, branch-1, branch-2, and master.
Thanks for all suggestions!!!

> Add the ClusterStatus hook and cleanup other hooks which can be replaced by 
> ClusterStatus hook
> --
>
> Key: HBASE-19131
> URL: https://issues.apache.org/jira/browse/HBASE-19131
> Project: HBase
>  Issue Type: Sub-task
>  Components: Coprocessors
>Reporter: Chia-Ping Tsai
>Assignee: Chia-Ping Tsai
> Fix For: 1.4.0, 1.5.0, 2.0.0-beta-1
>
> Attachments: HBASE-19131.branch-1.v0.patch, 
> HBASE-19131.branch-1.v1.patch, HBASE-19131.v0.patch, HBASE-19131.v1.patch, 
> HBASE-19131.v2.patch, HBASE-19131.v3.patch, HBASE-19131.v4.patch
>
>
> This issue will add the hook for {{ClusterStatus}} and cleanup other hooks 
> which can be replaced by {{ClusterStatus}} hook.
> # {{ListDeadServers}} -  introduced by HBASE-18131



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-19184) clean up nightly source artifact test to match expectations from switch to git-archive

[Sean Busbey (JIRA)]

Sean Busbey created HBASE-19184:
---

 Summary: clean up nightly source artifact test to match 
expectations from switch to git-archive
 Key: HBASE-19184
 URL: https://issues.apache.org/jira/browse/HBASE-19184
 Project: HBase
  Issue Type: Bug
  Components: build
Reporter: Sean Busbey
Assignee: Sean Busbey
Priority: Critical


HBASE-19152 started switching our nightly test to match the new recommended way 
of building RC source artifacts to rely on git archive instead of the maven 
assembly plugin.

finish cleanup.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (HBASE-19152) Update refguide 'how to build an RC' and the make_rc.sh script

[Sean Busbey (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-19152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Busbey resolved HBASE-19152.
-
Resolution: Fixed

filed HBASE-19184

> Update refguide 'how to build an RC' and the make_rc.sh script
> --
>
> Key: HBASE-19152
> URL: https://issues.apache.org/jira/browse/HBASE-19152
> Project: HBase
>  Issue Type: Sub-task
>  Components: build
>Reporter: stack
>Assignee: stack
>Priority: Trivial
> Fix For: 2.0.0-beta-1
>
> Attachments: 
> 0001-HBASE-19152-Update-refguide-how-to-build-an-RC-and-t.patch, 
> 0001-HBASE-19152-Update-refguide-how-to-build-an-RC-and-t.patch, 
> HBASE-19152.master.001.patch, HBASE-19152.master.002.patch, 
> HBASE-19152.master.003.patch, HBASE-19152.master.004.patch, 
> HBASE-19152.master.005.patch, HBASE-19152.master.006.patch, 
> HBASE-19152.master.007.patch, HBASE-19152.master.008.patch
>
>
> Update the section on how to build an RC. In particular, note that tags 
> should be signed and talk up the new way of building the src tgz (after a 
> Sean suggestion) including removal of old means src.xml from hbase-assembly. 
> Update the make_rc.sh script so it works for branch-2 builds.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: DISCUSSION: Minimum hbase1 version from which you can upgrade to hbase2 (1.2.x?)

[Guanghao Zhang]

Our internal branch is based on 0.98. And we plan rolling to 2.0. So I will
take a try for rolling from 0.98 to 2.0. But we take a lot backport to our
internal branch, like async client, netty rpc client, serial replication,
throttling, some replication improvements and so on. So our rolling
experience may not apply to community totally. I will post our rolling
experience (which can apply to community 0.98 branch) after we rolling to
2.0 :-).

2017-11-05 2:41 GMT+08:00 Stack :

> On Fri, Nov 3, 2017 at 9:01 PM, Guanghao Zhang  wrote:
>
> > Can we rolling from 0.98 and 1.1 to 1.2? If this rolling is ok, user can
> > rolling to 2.0 by two steps, 0.98 to 1.2, then 1.2 to 2.0.
> >
> >
> Yes. They could do that. Would be a pain. Might be able to go from 0.98 to
> 2.0 though... I've not tried it.
> St.Ack
>
>
>
>
> > 2017-11-04 11:25 GMT+08:00 Nick Dimiduk :
> >
> > > 1.2 is good, but are we aware of anything that precludes 1.1? 0.98? On
> > disk
> > > compatibility (HFile, WAL, AMv2) should be the limiting factor here,
> > right?
> > > Wire protocols have been compatible all the while...
> > >
> > > On Fri, Nov 3, 2017 at 5:56 PM Zach York  >
> > > wrote:
> > >
> > > > +1 for having the minimum (supported) hbase1 version be 1.2.x.
> > > >
> > > > On Fri, Nov 3, 2017 at 5:30 PM, Stack  wrote:
> > > >
> > > > > Over in the adjacent "[DISCUSS] hbase-2.0.0 compatibility
> > expectations"
> > > > > thread, we chatted some on what would be the minimum hbase-1.x
> > version
> > > > from
> > > > > which you can upgrade to hbase-2.
> > > > >
> > > > > The last statement made on this topic by Sean was that only
> upgrades
> > > from
> > > > > 1.2.x, our current stable offering, or later should be supported.
> > > > >
> > > > > There was no dissent.
> > > > >
> > > > > We all good w/ this? Speak up if you disagree else 1.2.x becomes
> the
> > > > > 'official' minimum.
> > > > >
> > > > > NOTES:
> > > > >
> > > > > + We need to agree on a minimum so we know what migrations to test.
> > > > > + It might be possible to upgrade from versions before 1.2.x but we
> > (or
> > > > at
> > > > > least I -- smile) won't have tried it or run verifications to
> ensure
> > > all
> > > > > made it over (let us know if you successfully migrate from a
> baseline
> > > > that
> > > > > precedes 1.2).
> > > > > + Hopefully we can avoid requiring Users move to the latest on the
> > 1.2
> > > > > branch. This shouldn't be necessary doing a stop/start upgrade. It
> > > might
> > > > be
> > > > > needed doing a rolling upgrade. Lets see.
> > > > >
> > > > > Thanks,
> > > > > St.Ack
> > > > >
> > > >
> > >
> >
>

Re: Empty hbase-shaded-client JAR file

[Sean Busbey]

Between the addition of the shaded jars in 1.1.0 and 1.1.1/1.2.0 building the 
shaded jars was moved to the release profile after complaints about how much 
time was added to the build by doing them all the time.

I think all the discussion was on JIRA:

https://issues.apache.org/jira/browse/HBASE-13816

Rick, is there some place we could have called out this detail in docs to save 
you time?



On 2017-11-03 20:49, Mike Drob  wrote: 
> Sean,
> 
> Is this a historical vestige? Why can't we populate the shaded jars always?
> 
> 
> 
> On Fri, Nov 3, 2017 at 7:51 PM, Sean Busbey  wrote:
> 
> > Hi Rick!
> >
> > You have to build with the "release" profile active for the shaded
> > jars to be populated
> >
> > e.g.:
> >
> > mvn -Prelease -DskipTests clean install
> >
> >
> >
> > On Fri, Nov 3, 2017 at 7:16 PM, Rick Kellogg 
> > wrote:
> > > Greetings,
> > >
> > >
> > >
> > > I have been trying to build the latest HBase using the master branch.
> > When
> > > the hbase-sharded-client JAR file is generated, it does not include any
> > > .class files.  Build completes without any errors.   Any ideas what
> > might be
> > > causing this issue?
> > >
> > >
> > >
> > > Attempted Maven Commands:  mvn package -DskipTests=true   and mvn install
> > > -DskipTests=true
> > >
> > >
> > >
> > > Environnent: RHEL 7.3, Maven 3.5.0, Oracle JDK 1.8.0_152.
> > >
> > >
> > >
> > > Thanks,
> > >
> > > Rick Kellogg
> > >
> > >
> > >
> > >
> > >
> >
> 

[jira] [Created] (HBASE-19183) Removed redundant groupId from Maven modules

[Jan Hentschel (JIRA)]

Jan Hentschel created HBASE-19183:
-

 Summary: Removed redundant groupId from Maven modules
 Key: HBASE-19183
 URL: https://issues.apache.org/jira/browse/HBASE-19183
 Project: HBase
  Issue Type: Improvement
Affects Versions: 3.0.0
Reporter: Jan Hentschel
Assignee: Jan Hentschel
Priority: Trivial


Currently the modules hbase-checkstyle and hbase-error-prone define the groupId 
redundantly. Remove the groupId from these POMs.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [VOTE] First hbase-2.0.0-alpha4 Release Candidate is available

[Jean-Marc Spaggiari]

Thanks for your reply Stack. I will get ready to test the next version.
Excited to see 2.0 coming out

2017-11-04 14:44 GMT-04:00 Stack :

> On Sat, Nov 4, 2017 at 1:45 AM, Jean-Marc Spaggiari <
> jean-m...@spaggiari.org
> > wrote:
>
> > Folks, are you able to run the tests?
> >
> > I'm trying to run this:
> >
> > mvn test -P runAllTests -Dsurefire.secondPartThreadCount=2
> > -Dtest.build.data.basedirectory=/ram4G
> >
> >
> > But I'm not able to get it running completely. It always fails somewhere,
> > never in the same place. How are you running the tests, if you are?
> >
> > Else:
> > MD5 and SHA files correct for both Binary and Source
> > *changes.txt file doesn't seems correct. Missing last versions*
> > *checkstyle.html doesn't seems to be accurate. Reports 0 files.*
> > Documetation ok.
> >
> > Will run more tests later when I can get runAllTests pass...
> >
>
> Test fail JMS. We have to fix all for beta-1. I should have said this in
> the announcement. Thanks for giving it a go boss.
> S
>
>
> >
> > 2017-11-03 3:32 GMT-04:00 Yung-An He :
> >
> > > +1 non-binding
> > >
> > > Checked MD5 : OK
> > > Download the bin tarball : OK
> > > Start from bin tar with cluster mode via docker container: OK
> > > Import data with 100 million rows and 10 columns to table via
> ImportTsv:
> > OK
> > > Execute basic command via `hbase shell`: OK
> > >
> > >
> > > ‌
> > >
> > > 2017-11-02 5:02 GMT+08:00 Mike Drob :
> > >
> > > > +1
> > > >
> > > > Verified sigs and hashes.
> > > > Src tar matches git tag.
> > > > Able to build from src tar w/ hadoop 2 & 3
> > > > Able to start from bin tar, verified basic functionality.
> > > >
> > > > Filed HBASE-19151 for some warning messages on shell startup.
> > > > Failed to start with Java9, but didn't expect this to work anyway.
> > > >
> > > > On Wed, Nov 1, 2017 at 1:38 PM, Artem Ervits 
> > > > wrote:
> > > >
> > > > > +1
> > > > >
> > > > > Checked md5
> > > > > ran LTT tool with 10k rows
> > > > > ran PE tool with 6M rows
> > > > > executed counter in hbase shell
> > > > >
> > > > >
> > > > >
> > > > > On Wed, Nov 1, 2017 at 1:18 PM, Ted Yu 
> wrote:
> > > > >
> > > > > > +1
> > > > > >
> > > > > > Checked signatures
> > > > > > Built from source
> > > > > > Ran load test tool with 1 mil rows.
> > > > > >
> > > > > > On Wed, Nov 1, 2017 at 7:17 AM, Stack  wrote:
> > > > > >
> > > > > > > The first release candidate for HBase 2.0.0-alpha4 is up at:
> > > > > > >
> > > > > > >   https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.0-
> > > > alpha4RC0/
> > > > > > >
> > > > > > > Maven artifacts are available from a staging directory here:
> > > > > > >
> > > > > > >   https://repository.apache.org/content/repositories/orgapach
> > > > > ehbase-1178
> > > > > > >
> > > > > > > All was signed with my key at 8ACC93D2 [1]
> > > > > > >
> > > > > > > I tagged the RC as 2.0.0-alpha4RC0
> > > > > > > (5c4b985f89c99cc8b0f8515a4097c811a0848835)
> > > > > > >
> > > > > > > hbase-2.0.0-alpha4 is our fourth alpha release along our march
> > > toward
> > > > > > > hbase-2.0.0. It includes all that was in previous alphas (new
> > > > > assignment
> > > > > > > manager, offheap read/write path, in-memory compactions, etc.),
> > but
> > > > > had a
> > > > > > > focus on "Coprocessor Fixup": We no longer pass Coprocessors
> > > > > > > InterfaceAudience.Private parameters and we cut down on the
> > access
> > > > and
> > > > > > > ability to influence hbase core processing (See [2] on why the
> > > > radical
> > > > > > > changes in Coprocessor Interface). If you are a Coprocessor
> > > developer
> > > > > or
> > > > > > > have Coprocessors to deploy on hbase-2.0.0, we need to hear
> about
> > > > your
> > > > > > > experience now before we make an hbase-2.0.0 beta.
> > > > > > >
> > > > > > > hbase-2.0.0-alpha4 is a rough cut ('alpha'), not-for-production
> > > > preview
> > > > > > of
> > > > > > > what hbase-2.0.0 will look like. It is meant for devs and
> > > > downstreamers
> > > > > > to
> > > > > > > test drive and flag us early if we messed up anything ahead of
> > our
> > > > > > rolling
> > > > > > > GAs.
> > > > > > >
> > > > > > > The list of features addressed in 2.0.0 so far can be found
> here
> > > [3].
> > > > > > There
> > > > > > > are thousands. The list of ~2k+ fixes in 2.0.0 exclusively can
> be
> > > > found
> > > > > > > here [4] (My JIRA JQL foo is a bit dodgy -- forgive me if
> > > mistakes).
> > > > > > >
> > > > > > > I've updated our overview doc. on the state of 2.0.0 [6].
> > > > 2.0.0-beta-1
> > > > > > will
> > > > > > > be our next release. Its theme is the "Finishing up 2.0.0"
> > release.
> > > > > Here
> > > > > > is
> > > > > > > the list of what we have targeted for beta-1 [5]. Check it out.
> > > Shout
> > > > > if
> > > > > > > there is anything missing. We may do a 2.0.0-beta-2 if a need.
> > > We'll
> > > > > see.
> > 

Re: [VOTE] First hbase-2.0.0-alpha4 Release Candidate is available

[Stack]

On Sat, Nov 4, 2017 at 1:45 AM, Jean-Marc Spaggiari  wrote:

> Folks, are you able to run the tests?
>
> I'm trying to run this:
>
> mvn test -P runAllTests -Dsurefire.secondPartThreadCount=2
> -Dtest.build.data.basedirectory=/ram4G
>
>
> But I'm not able to get it running completely. It always fails somewhere,
> never in the same place. How are you running the tests, if you are?
>
> Else:
> MD5 and SHA files correct for both Binary and Source
> *changes.txt file doesn't seems correct. Missing last versions*
> *checkstyle.html doesn't seems to be accurate. Reports 0 files.*
> Documetation ok.
>
> Will run more tests later when I can get runAllTests pass...
>

Test fail JMS. We have to fix all for beta-1. I should have said this in
the announcement. Thanks for giving it a go boss.
S


>
> 2017-11-03 3:32 GMT-04:00 Yung-An He :
>
> > +1 non-binding
> >
> > Checked MD5 : OK
> > Download the bin tarball : OK
> > Start from bin tar with cluster mode via docker container: OK
> > Import data with 100 million rows and 10 columns to table via ImportTsv:
> OK
> > Execute basic command via `hbase shell`: OK
> >
> >
> > ‌
> >
> > 2017-11-02 5:02 GMT+08:00 Mike Drob :
> >
> > > +1
> > >
> > > Verified sigs and hashes.
> > > Src tar matches git tag.
> > > Able to build from src tar w/ hadoop 2 & 3
> > > Able to start from bin tar, verified basic functionality.
> > >
> > > Filed HBASE-19151 for some warning messages on shell startup.
> > > Failed to start with Java9, but didn't expect this to work anyway.
> > >
> > > On Wed, Nov 1, 2017 at 1:38 PM, Artem Ervits 
> > > wrote:
> > >
> > > > +1
> > > >
> > > > Checked md5
> > > > ran LTT tool with 10k rows
> > > > ran PE tool with 6M rows
> > > > executed counter in hbase shell
> > > >
> > > >
> > > >
> > > > On Wed, Nov 1, 2017 at 1:18 PM, Ted Yu  wrote:
> > > >
> > > > > +1
> > > > >
> > > > > Checked signatures
> > > > > Built from source
> > > > > Ran load test tool with 1 mil rows.
> > > > >
> > > > > On Wed, Nov 1, 2017 at 7:17 AM, Stack  wrote:
> > > > >
> > > > > > The first release candidate for HBase 2.0.0-alpha4 is up at:
> > > > > >
> > > > > >   https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.0-
> > > alpha4RC0/
> > > > > >
> > > > > > Maven artifacts are available from a staging directory here:
> > > > > >
> > > > > >   https://repository.apache.org/content/repositories/orgapach
> > > > ehbase-1178
> > > > > >
> > > > > > All was signed with my key at 8ACC93D2 [1]
> > > > > >
> > > > > > I tagged the RC as 2.0.0-alpha4RC0
> > > > > > (5c4b985f89c99cc8b0f8515a4097c811a0848835)
> > > > > >
> > > > > > hbase-2.0.0-alpha4 is our fourth alpha release along our march
> > toward
> > > > > > hbase-2.0.0. It includes all that was in previous alphas (new
> > > > assignment
> > > > > > manager, offheap read/write path, in-memory compactions, etc.),
> but
> > > > had a
> > > > > > focus on "Coprocessor Fixup": We no longer pass Coprocessors
> > > > > > InterfaceAudience.Private parameters and we cut down on the
> access
> > > and
> > > > > > ability to influence hbase core processing (See [2] on why the
> > > radical
> > > > > > changes in Coprocessor Interface). If you are a Coprocessor
> > developer
> > > > or
> > > > > > have Coprocessors to deploy on hbase-2.0.0, we need to hear about
> > > your
> > > > > > experience now before we make an hbase-2.0.0 beta.
> > > > > >
> > > > > > hbase-2.0.0-alpha4 is a rough cut ('alpha'), not-for-production
> > > preview
> > > > > of
> > > > > > what hbase-2.0.0 will look like. It is meant for devs and
> > > downstreamers
> > > > > to
> > > > > > test drive and flag us early if we messed up anything ahead of
> our
> > > > > rolling
> > > > > > GAs.
> > > > > >
> > > > > > The list of features addressed in 2.0.0 so far can be found here
> > [3].
> > > > > There
> > > > > > are thousands. The list of ~2k+ fixes in 2.0.0 exclusively can be
> > > found
> > > > > > here [4] (My JIRA JQL foo is a bit dodgy -- forgive me if
> > mistakes).
> > > > > >
> > > > > > I've updated our overview doc. on the state of 2.0.0 [6].
> > > 2.0.0-beta-1
> > > > > will
> > > > > > be our next release. Its theme is the "Finishing up 2.0.0"
> release.
> > > > Here
> > > > > is
> > > > > > the list of what we have targeted for beta-1 [5]. Check it out.
> > Shout
> > > > if
> > > > > > there is anything missing. We may do a 2.0.0-beta-2 if a need.
> > We'll
> > > > see.
> > > > > >
> > > > > > Please take this alpha for a spin especially if you are a
> > Coprocessor
> > > > > > developer or have a Coprocessor you want to deploy on
> hbase-2.0.0.
> > > > Please
> > > > > > vote on whether it ok to put out this RC as our first alpha (bar
> is
> > > low
> > > > > for
> > > > > > an 'alpha' -- e.g. CHANGES.txt has not been updated). Let the
> VOTE
> > be
> > > > > open
> > > > > > for 72 hours (Saturday)
> > > > > >
> > > > > > 

Re: DISCUSSION: Minimum hbase1 version from which you can upgrade to hbase2 (1.2.x?)

[Stack]

On Fri, Nov 3, 2017 at 9:01 PM, Guanghao Zhang  wrote:

> Can we rolling from 0.98 and 1.1 to 1.2? If this rolling is ok, user can
> rolling to 2.0 by two steps, 0.98 to 1.2, then 1.2 to 2.0.
>
>
Yes. They could do that. Would be a pain. Might be able to go from 0.98 to
2.0 though... I've not tried it.
St.Ack




> 2017-11-04 11:25 GMT+08:00 Nick Dimiduk :
>
> > 1.2 is good, but are we aware of anything that precludes 1.1? 0.98? On
> disk
> > compatibility (HFile, WAL, AMv2) should be the limiting factor here,
> right?
> > Wire protocols have been compatible all the while...
> >
> > On Fri, Nov 3, 2017 at 5:56 PM Zach York 
> > wrote:
> >
> > > +1 for having the minimum (supported) hbase1 version be 1.2.x.
> > >
> > > On Fri, Nov 3, 2017 at 5:30 PM, Stack  wrote:
> > >
> > > > Over in the adjacent "[DISCUSS] hbase-2.0.0 compatibility
> expectations"
> > > > thread, we chatted some on what would be the minimum hbase-1.x
> version
> > > from
> > > > which you can upgrade to hbase-2.
> > > >
> > > > The last statement made on this topic by Sean was that only upgrades
> > from
> > > > 1.2.x, our current stable offering, or later should be supported.
> > > >
> > > > There was no dissent.
> > > >
> > > > We all good w/ this? Speak up if you disagree else 1.2.x becomes the
> > > > 'official' minimum.
> > > >
> > > > NOTES:
> > > >
> > > > + We need to agree on a minimum so we know what migrations to test.
> > > > + It might be possible to upgrade from versions before 1.2.x but we
> (or
> > > at
> > > > least I -- smile) won't have tried it or run verifications to ensure
> > all
> > > > made it over (let us know if you successfully migrate from a baseline
> > > that
> > > > precedes 1.2).
> > > > + Hopefully we can avoid requiring Users move to the latest on the
> 1.2
> > > > branch. This shouldn't be necessary doing a stop/start upgrade. It
> > might
> > > be
> > > > needed doing a rolling upgrade. Lets see.
> > > >
> > > > Thanks,
> > > > St.Ack
> > > >
> > >
> >
>

Re: DISCUSSION: Minimum hbase1 version from which you can upgrade to hbase2 (1.2.x?)

[Stack]

On Fri, Nov 3, 2017 at 8:25 PM, Nick Dimiduk  wrote:

> 1.2 is good, but are we aware of anything that precludes 1.1? 0.98? On disk
> compatibility (HFile, WAL, AMv2) should be the limiting factor here, right?
> Wire protocols have been compatible all the while...
>
>
I don't know of anything (Anyone know of anything in here that could hamper
a migration?). I suppose I could give it a go. I was just trying to get
away with a minimal spread.
Thanks Nick,
S



> On Fri, Nov 3, 2017 at 5:56 PM Zach York 
> wrote:
>
> > +1 for having the minimum (supported) hbase1 version be 1.2.x.
> >
> > On Fri, Nov 3, 2017 at 5:30 PM, Stack  wrote:
> >
> > > Over in the adjacent "[DISCUSS] hbase-2.0.0 compatibility expectations"
> > > thread, we chatted some on what would be the minimum hbase-1.x version
> > from
> > > which you can upgrade to hbase-2.
> > >
> > > The last statement made on this topic by Sean was that only upgrades
> from
> > > 1.2.x, our current stable offering, or later should be supported.
> > >
> > > There was no dissent.
> > >
> > > We all good w/ this? Speak up if you disagree else 1.2.x becomes the
> > > 'official' minimum.
> > >
> > > NOTES:
> > >
> > > + We need to agree on a minimum so we know what migrations to test.
> > > + It might be possible to upgrade from versions before 1.2.x but we (or
> > at
> > > least I -- smile) won't have tried it or run verifications to ensure
> all
> > > made it over (let us know if you successfully migrate from a baseline
> > that
> > > precedes 1.2).
> > > + Hopefully we can avoid requiring Users move to the latest on the 1.2
> > > branch. This shouldn't be necessary doing a stop/start upgrade. It
> might
> > be
> > > needed doing a rolling upgrade. Lets see.
> > >
> > > Thanks,
> > > St.Ack
> > >
> >
>

[jira] [Created] (HBASE-19182) Add deprecation in branch-1 for hbase-prefix-tree so some heads up it removed in hbase2

[stack (JIRA)]

stack created HBASE-19182:
-

 Summary: Add deprecation in branch-1 for hbase-prefix-tree so some 
heads up it removed in hbase2
 Key: HBASE-19182
 URL: https://issues.apache.org/jira/browse/HBASE-19182
 Project: HBase
  Issue Type: Sub-task
Reporter: stack
Assignee: stack
Priority: Major
 Fix For: 2.0.0-beta-1


I removed hbase-prefix-tree w/o doing a deprecation cycle. Let me at least add 
late warning it going away (I don't want it to go away; I would prefer someone 
took up this sweet encoding and carried to the next level).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-19181) LogRollBackupSubprocedure will fail if we use AsyncFSWAL instead of FSHLog

[Duo Zhang (JIRA)]

Duo Zhang created HBASE-19181:
-

 Summary: LogRollBackupSubprocedure will fail if we use AsyncFSWAL 
instead of FSHLog
 Key: HBASE-19181
 URL: https://issues.apache.org/jira/browse/HBASE-19181
 Project: HBase
  Issue Type: Bug
  Components: backup
Reporter: Duo Zhang
Priority: Major


In the RSRollLogTask it will cast a WAL to FSHLog.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [VOTE] First hbase-2.0.0-alpha4 Release Candidate is available

[Jean-Marc Spaggiari]

Folks, are you able to run the tests?

I'm trying to run this:

mvn test -P runAllTests -Dsurefire.secondPartThreadCount=2
-Dtest.build.data.basedirectory=/ram4G


But I'm not able to get it running completely. It always fails somewhere,
never in the same place. How are you running the tests, if you are?

Else:
MD5 and SHA files correct for both Binary and Source
*changes.txt file doesn't seems correct. Missing last versions*
*checkstyle.html doesn't seems to be accurate. Reports 0 files.*
Documetation ok.

Will run more tests later when I can get runAllTests pass...

2017-11-03 3:32 GMT-04:00 Yung-An He :

> +1 non-binding
>
> Checked MD5 : OK
> Download the bin tarball : OK
> Start from bin tar with cluster mode via docker container: OK
> Import data with 100 million rows and 10 columns to table via ImportTsv: OK
> Execute basic command via `hbase shell`: OK
>
>
> ‌
>
> 2017-11-02 5:02 GMT+08:00 Mike Drob :
>
> > +1
> >
> > Verified sigs and hashes.
> > Src tar matches git tag.
> > Able to build from src tar w/ hadoop 2 & 3
> > Able to start from bin tar, verified basic functionality.
> >
> > Filed HBASE-19151 for some warning messages on shell startup.
> > Failed to start with Java9, but didn't expect this to work anyway.
> >
> > On Wed, Nov 1, 2017 at 1:38 PM, Artem Ervits 
> > wrote:
> >
> > > +1
> > >
> > > Checked md5
> > > ran LTT tool with 10k rows
> > > ran PE tool with 6M rows
> > > executed counter in hbase shell
> > >
> > >
> > >
> > > On Wed, Nov 1, 2017 at 1:18 PM, Ted Yu  wrote:
> > >
> > > > +1
> > > >
> > > > Checked signatures
> > > > Built from source
> > > > Ran load test tool with 1 mil rows.
> > > >
> > > > On Wed, Nov 1, 2017 at 7:17 AM, Stack  wrote:
> > > >
> > > > > The first release candidate for HBase 2.0.0-alpha4 is up at:
> > > > >
> > > > >   https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.0-
> > alpha4RC0/
> > > > >
> > > > > Maven artifacts are available from a staging directory here:
> > > > >
> > > > >   https://repository.apache.org/content/repositories/orgapach
> > > ehbase-1178
> > > > >
> > > > > All was signed with my key at 8ACC93D2 [1]
> > > > >
> > > > > I tagged the RC as 2.0.0-alpha4RC0
> > > > > (5c4b985f89c99cc8b0f8515a4097c811a0848835)
> > > > >
> > > > > hbase-2.0.0-alpha4 is our fourth alpha release along our march
> toward
> > > > > hbase-2.0.0. It includes all that was in previous alphas (new
> > > assignment
> > > > > manager, offheap read/write path, in-memory compactions, etc.), but
> > > had a
> > > > > focus on "Coprocessor Fixup": We no longer pass Coprocessors
> > > > > InterfaceAudience.Private parameters and we cut down on the access
> > and
> > > > > ability to influence hbase core processing (See [2] on why the
> > radical
> > > > > changes in Coprocessor Interface). If you are a Coprocessor
> developer
> > > or
> > > > > have Coprocessors to deploy on hbase-2.0.0, we need to hear about
> > your
> > > > > experience now before we make an hbase-2.0.0 beta.
> > > > >
> > > > > hbase-2.0.0-alpha4 is a rough cut ('alpha'), not-for-production
> > preview
> > > > of
> > > > > what hbase-2.0.0 will look like. It is meant for devs and
> > downstreamers
> > > > to
> > > > > test drive and flag us early if we messed up anything ahead of our
> > > > rolling
> > > > > GAs.
> > > > >
> > > > > The list of features addressed in 2.0.0 so far can be found here
> [3].
> > > > There
> > > > > are thousands. The list of ~2k+ fixes in 2.0.0 exclusively can be
> > found
> > > > > here [4] (My JIRA JQL foo is a bit dodgy -- forgive me if
> mistakes).
> > > > >
> > > > > I've updated our overview doc. on the state of 2.0.0 [6].
> > 2.0.0-beta-1
> > > > will
> > > > > be our next release. Its theme is the "Finishing up 2.0.0" release.
> > > Here
> > > > is
> > > > > the list of what we have targeted for beta-1 [5]. Check it out.
> Shout
> > > if
> > > > > there is anything missing. We may do a 2.0.0-beta-2 if a need.
> We'll
> > > see.
> > > > >
> > > > > Please take this alpha for a spin especially if you are a
> Coprocessor
> > > > > developer or have a Coprocessor you want to deploy on hbase-2.0.0.
> > > Please
> > > > > vote on whether it ok to put out this RC as our first alpha (bar is
> > low
> > > > for
> > > > > an 'alpha' -- e.g. CHANGES.txt has not been updated). Let the VOTE
> be
> > > > open
> > > > > for 72 hours (Saturday)
> > > > >
> > > > > Thanks,
> > > > > Your 2.0.0 Release Manager
> > > > >
> > > > > 1. http://pgp.mit.edu/pks/lookup?op=get=0x9816C7FC8ACC93D2
> > > > > 2. Why CPs are Incompatible:
> > > > > https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9iEu_
> > > > > ktczrlKHK8N4SZzs/edit#heading=h.9k7mjbauv0wj
> > > > > 3. https://goo.gl/scYjJr
> > > > > 4. https://goo.gl/tMHkYS
> > > > > 5. https://issues.apache.org/jira/projects/HBASE/versions/12340861
> > > > > 6.
> > > > >