Re: Want to join slack channel for Hbase

2020-04-16 Thread Reid Chan 
Invitation sent to subharaj.ma...@gmail.com.

--

Best regards,
R.C




From: Debraj Manna 
Sent: 17 April 2020 12:58
To: dev@hbase.apache.org
Subject: Want to join slack channel for Hbase

Hi

I want to join slack channel for Hbase. Can someone send me an invite to
join the channel?

Thanks,

Want to join slack channel for Hbase

2020-04-16 Thread Debraj Manna 
Hi

I want to join slack channel for Hbase. Can someone send me an invite to
join the channel?

Thanks,

Re: [DISCUSS] Arrange Events for 10-year Anniversary

2020-04-16 Thread Sally Khudairi 
Thanks so much, Andrew!

Best,
Sally

- - - 
Vice President Marketing & Publicity
Vice President Sponsor Relations
The Apache Software Foundation

Tel +1 617 921 8656 | s...@apache.org


On Thu, Apr 16, 2020, at 12:09, Andrew Purtell wrote:
> We graduated to TLP on April 21, 2010. Confirmed. 
> 
> https://whimsy.apache.org/board/minutes/HBase.html#2010-04-21

> 
>> On Apr 15, 2020, at 12:58 PM, Sally Khudairi  wrote:
>> Thank you, Yu. Hello, everyone --my apologies for seemingly ignoring you. 
>> 
>> Misty Linville contacted me earlier about this, but I've been snagged with 
>> some urgent business, thus my delayed response.
>> 
>> I'll be happy to support your anniversary, and can publish a Foundation 
>> announcement, similar to https://s.apache.org/ApacheSVN20 but likely with 
>> fewer/shorter testimonials. We can incorporate some from the suggested 
>> bullet point below into this document.
>> 
>> If the community would like to pursue this, we'll need to coordinate with 
>> the PMC. If you can please confirm the date of the anniversary, I'd 
>> appreciate it.
>> 
>> Warmly,
>> Sally
>> 
>> + copying press@ to keep everyone in the loop
>> 
>> - - -
>> Vice President Marketing & Publicity
>> Vice President Sponsor Relations
>> The Apache Software Foundation
>> 
>> Tel +1 617 921 8656 | s...@apache.org
>> 
>> 
>> On Wed, Apr 15, 2020, at 08:49, Yu Li wrote:
>>> Dear all,
>>> 
>>> Since our project has reached its 10th birthday, and 10 years is definitely 
>>> a great milestone, I propose to arrange some special (virtual) events for 
>>> celebration. What comes into my mind include:
>>> 
>>> * Open threads to collect voices from our dev/user mailing list, like "what 
>>> do you want to say to HBase for its 10th birthday" (as well as our twitter 
>>> accounts maybe, if any)
>>> 
>>> * Arrange some online interviews to both PMC members and our customers. 
>>> Some of us have been in this project all the way and there must be some 
>>> good stories to tell, as well as expectations for the future.
>>> 
>>> * Join the Apache Feathercast as suggested in another thread.
>>> 
>>> * Form a blogpost to include all above events as an official celebration.
>>> 
>>> What do you think? Any other good ideas? Looking forward to more voices 
>>> (smile).
>>> 
>>> Best Regards,
>>> Yu

Build failed in Jenkins: HBase-Nightly-ARM #81

2020-04-16 Thread Apache Jenkins Server 
See 


Changes:

[stack] HBASE-24175 [Flakey Tests] TestSecureExportSnapshot

[github] HBASE-24197 TestHttpServer.testBindAddress failure with latest jetty

[github] HBASE-24170 Remove hadoop-2.0 profile (#1495)

[github] HBASE-24194 : Refactor anonymous inner classes of BufferedEncodedSeeker

[github] HBASE-24186: RegionMover ignores replicationId (#1512)

[github] HBASE-24195 : Admin.getRegionServers() should return live servers exc…

[stack] HBASE-24158 [Flakey Tests] TestAsyncTableGetMultiThreaded


--
[...truncated 421.84 KB...]
[INFO] 
[INFO] --- maven-install-plugin:2.5.2:install 
(default-install) @ hbase-client-project ---
[INFO] Installing 

 to 
/home/jenkins/.m2/repository/org/apache/hbase/hbase-client-project/3.0.0-SNAPSHOT/hbase-client-project-3.0.0-SNAPSHOT.jar
[INFO] Installing 

 to 
/home/jenkins/.m2/repository/org/apache/hbase/hbase-client-project/3.0.0-SNAPSHOT/hbase-client-project-3.0.0-SNAPSHOT.pom
[INFO] Installing 

 to 
/home/jenkins/.m2/repository/org/apache/hbase/hbase-client-project/3.0.0-SNAPSHOT/hbase-client-project-3.0.0-SNAPSHOT-tests.jar
[INFO] 
[INFO] < 
org.apache.hbase:hbase-shaded-client-project >
[INFO] Building Apache HBase - Exemplar for hbase-shaded-client 
archetype 3.0.0-SNAPSHOT [40/41]
[INFO] [ jar 
]-
[INFO] 
[INFO] --- maven-clean-plugin:3.1.0:clean 
(default-clean) @ hbase-shaded-client-project ---
[INFO] Deleting 

[INFO] 
[INFO] --- build-helper-maven-plugin:3.0.0:bsh-property 
(negate-license-bundles-property) @ 
hbase-shaded-client-project ---
[INFO] 
[INFO] --- 
build-helper-maven-plugin:3.0.0:regex-property 
(create-license-file-path-property) @ 
hbase-shaded-client-project ---
[INFO] No match to regex '\\' found in 
'
 The initial value 
'
 is left as-is...
[INFO] 
[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce 
(enforce-maven-version) @ hbase-shaded-client-project ---
[INFO] 
[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce 
(hadoop-profile-min-maven-min-java-banned-xerces) @ 
hbase-shaded-client-project ---
[INFO] 
[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce 
(banned-jsr305) @ hbase-shaded-client-project ---
[INFO] 
[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce 
(banned-scala) @ hbase-shaded-client-project ---
[INFO] 
[INFO] --- 
buildnumber-maven-plugin:1.4:create-timestamp (default) @ 
hbase-shaded-client-project ---
[INFO] 
[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce 
(banned-illegal-imports) @ hbase-shaded-client-project ---
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.6.0:process 
(process-resource-bundles) @ hbase-shaded-client-project 
---
[INFO] Preparing remote bundle 
org.apache:apache-jar-resource-bundle:1.4
[INFO] Copying 3 resources from 1 bundle.
[INFO] 
[INFO] --- maven-resources-plugin:3.1.0:resources 
(default-resources) @ hbase-shaded-client-project ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce 
(check-aggregate-license) @ hbase-shaded-client-project ---
[INFO] Skipping Rule Enforcement.
[INFO]

[jira] [Created] (HBASE-24202) Nightly CI's USE_YETUS_PRERELEASE feature is broken

2020-04-16 Thread Nick Dimiduk (Jira) 
Nick Dimiduk created HBASE-24202:


 Summary: Nightly CI's USE_YETUS_PRERELEASE feature is broken
 Key: HBASE-24202
 URL: https://issues.apache.org/jira/browse/HBASE-24202
 Project: HBase
  Issue Type: Task
  Components: build
Reporter: Nick Dimiduk


I tried kicking off a 
[build|https://builds.apache.org/blue/organizations/jenkins/HBase%20Nightly/detail/master/1701/pipeline/]
 with the latest Yetus today, only to find

{noformat}
[2020-04-16T20:54:09.742Z] bash: 
/home/jenkins/jenkins-slave/workspace/HBase_Nightly_master/yetus-git/precommit/test-patch.sh:
 No such file or directory
{noformat}

Looks like we don't adjust the path to test-patch.sh based on working from a 
source tarball vs. a release tarball.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-24201) Fix PR builds on branch-2.2

2020-04-16 Thread Nick Dimiduk (Jira) 
Nick Dimiduk created HBASE-24201:


 Summary: Fix PR builds on branch-2.2
 Key: HBASE-24201
 URL: https://issues.apache.org/jira/browse/HBASE-24201
 Project: HBase
  Issue Type: Task
  Components: build
Affects Versions: 2.2.5
Reporter: Nick Dimiduk


>From a recent [PR 
>build|https://builds.apache.org/blue/organizations/jenkins/HBase-PreCommit-GitHub-PR/detail/PR-1532/1/pipeline/]

{noformat}
[2020-04-16T18:43:21.548Z] Setting up ruby2.3 (2.3.3-1+deb9u7) ...
[2020-04-16T18:43:21.548Z] Setting up ruby2.3-dev:amd64 (2.3.3-1+deb9u7) ...
[2020-04-16T18:43:21.548Z] Setting up ruby-dev:amd64 (1:2.3.3) ...
[2020-04-16T18:43:21.548Z] Setting up ruby (1:2.3.3) ...
[2020-04-16T18:43:22.261Z] Processing triggers for libc-bin (2.24-11+deb9u3) ...
[2020-04-16T18:43:22.975Z] Successfully installed rake-13.0.1
[2020-04-16T18:43:22.975Z] Building native extensions.  This could take a 
while...
[2020-04-16T18:43:25.277Z] ERROR:  Error installing rubocop:
[2020-04-16T18:43:25.277Z]  rubocop requires Ruby version >= 2.4.0.
{noformat}

Looks like the Dockerfile on branch-2.2 has bit-rot. I suspect package versions 
are partially pinned or not pinned at all: the rubocop version has incremented 
by ruby version has not.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Want to join slack channel for HBase

2020-04-16 Thread Nick Dimiduk 
Invitation sent.

On Thu, Apr 16, 2020 at 10:12 AM shishir Rai 
wrote:

> Hi,
> I am a principal software developer working on HBase.I want to join slack
> channel
> to get quick resolutions regarding HBase queries .
> Please send me an invite to join the channel.
>
> --
> SHISHIR RAI
>

[jira] [Resolved] (HBASE-23697) Document new RegionProcedureStore operation and migration

2020-04-16 Thread Michael Stack (Jira) 


 [ 
https://issues.apache.org/jira/browse/HBASE-23697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Stack resolved HBASE-23697.
---
Hadoop Flags: Reviewed
Assignee: Michael Stack
  Resolution: Fixed

Thanks for reviews [~janh], [~huaxiang], and [~zhangduo]. Pushed to master.

> Document new RegionProcedureStore operation and migration
> -
>
> Key: HBASE-23697
> URL: https://issues.apache.org/jira/browse/HBASE-23697
> Project: HBase
>  Issue Type: Sub-task
>  Components: documentation
>Affects Versions: 2.3.0
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
>
> Add a few notes to the refguide on the new RegionProcedureStore, how it works 
> (A 'Region' but buried in the Master with dedicated flushing/compacting 
> threads and archivers for WAL and hfile), how it differs from WALPS, and note 
> it auto-migrates and there should be new issue moving on to the new store.
> Mention the configuration. Mention it is on WALFS even though it is a 
> 'Region', etc.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Want to join slack channel for HBase

2020-04-16 Thread shishir Rai 
Hi,
I am a principal software developer working on HBase.I want to join slack
channel
to get quick resolutions regarding HBase queries .
Please send me an invite to join the channel.

-- 
SHISHIR RAI

[jira] [Created] (HBASE-24200) Upgrade to Yetus 0.12.0

2020-04-16 Thread Nick Dimiduk (Jira) 
Nick Dimiduk created HBASE-24200:


 Summary: Upgrade to Yetus 0.12.0
 Key: HBASE-24200
 URL: https://issues.apache.org/jira/browse/HBASE-24200
 Project: HBase
  Issue Type: Task
  Components: build
Reporter: Nick Dimiduk


A new Yetus release is imminent.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] Arrange Events for 10-year Anniversary

2020-04-16 Thread Andrew Purtell 
We graduated to TLP on April 21, 2010. Confirmed. 

https://whimsy.apache.org/board/minutes/HBase.html#2010-04-21

> On Apr 15, 2020, at 12:58 PM, Sally Khudairi  wrote:
> 
> Thank you, Yu. Hello, everyone --my apologies for seemingly ignoring you. 
> 
> Misty Linville contacted me earlier about this, but I've been snagged with 
> some urgent business, thus my delayed response.
> 
> I'll be happy to support your anniversary, and can publish a Foundation 
> announcement, similar to https://s.apache.org/ApacheSVN20 but likely with 
> fewer/shorter testimonials. We can incorporate some from the suggested bullet 
> point below into this document.
> 
> If the community would like to pursue this, we'll need to coordinate with the 
> PMC. If you can please confirm the date of the anniversary, I'd appreciate it.
> 
> Warmly,
> Sally
> 
> + copying press@ to keep everyone in the loop
> 
> - - -
> Vice President Marketing & Publicity
> Vice President Sponsor Relations
> The Apache Software Foundation
> 
> Tel +1 617 921 8656 | s...@apache.org
> 
> 
>> On Wed, Apr 15, 2020, at 08:49, Yu Li wrote:
>> Dear all,
>> 
>> Since our project has reached its 10th birthday, and 10 years is definitely 
>> a great milestone, I propose to arrange some special (virtual) events for 
>> celebration. What comes into my mind include:
>> 
>> * Open threads to collect voices from our dev/user mailing list, like "what 
>> do you want to say to HBase for its 10th birthday" (as well as our twitter 
>> accounts maybe, if any)
>> 
>> * Arrange some online interviews to both PMC members and our customers. Some 
>> of us have been in this project all the way and there must be some good 
>> stories to tell, as well as expectations for the future.
>> 
>> * Join the Apache Feathercast as suggested in another thread.
>> 
>> * Form a blogpost to include all above events as an official celebration.
>> 
>> What do you think? Any other good ideas? Looking forward to more voices 
>> (smile).
>> 
>> Best Regards,
>> Yu

Re: [DISCUSS] New User Experience and Data Durability Guarantees on LocalFileSystem (HBASE-24086)

2020-04-16 Thread Andrew Purtell 
The data can not be said to be durable because there is one set of files that 
can be irreversibly corrupted or lost. 

> On Apr 15, 2020, at 3:52 PM, Vladimir Rodionov  wrote:
> 
> FileOutputStream.getFileChannel().force(true) will get all durability we
> need. Just a simple code change?
> 
> 
>> On Wed, Apr 15, 2020 at 12:32 PM Andrew Purtell 
>> wrote:
>> 
>> This thread talks of “durability” via filesystem characteristics but also
>> for single system quick Start type deployments. For durability we need
>> multi server deployments. No amount of hacking a single system deployment
>> is going to give us durability as users will expect (“don’t lose my data”).
>> I believe my comments are on topic.
>> 
>> 
 On Apr 15, 2020, at 11:03 AM, Nick Dimiduk  wrote:
>>> 
>>> On Wed, Apr 15, 2020 at 10:28 AM Andrew Purtell 
>> wrote:
>>> 
 Nick's mail doesn't make a distinction between avoiding data loss via
 typical tmp cleaner configurations, unfortunately adjacent to mention of
 "durability", and real data durability, which implies more than what a
 single system configuration can offer, no matter how many tweaks we
>> make to
 LocalFileSystem. Maybe I'm being pedantic but this is something to be
 really clear about IMHO.
 
>>> 
>>> I prefer to focus the attention of this thread to the question of data
>>> durability via `FileSystem` characteristics. I agree that there are
>>> concerns of durability (and others) around the use of the path under
>> /tmp.
>>> Let's keep that discussion in the other thread.
>>> 
 On Wed, Apr 15, 2020 at 10:05 AM Sean Busbey  wrote:
 
> I think the first assumption no longer holds. Especially with the move
> to flexible compute environments I regularly get asked by folks what
> the smallest HBase they can start with for production. I can keep
> saying 3/5/7 nodes or whatever but I guarantee there are folks who
> want to and will run HBase with a single node. Probably those
> deployments won't want to have the distributed flag set. None of them
> really have a good option for where the WALs go, and failing loud when
> they try to go to LocalFileSystem is the best option I've seen so far
> to make sure folks realize they are getting into muddy waters.
> 
> I agree with the second assumption. Our quickstart in general is too
> complicated. Maybe if we include big warnings in the guide itself, we
> could make a quickstart specific artifact to download that has the
> unsafe disabling config in place?
> 
> Last fall I toyed with the idea of adding an "hbase-local" module to
> the hbase-filesystem repo that could start us out with some
> optimizations for single node set ups. We could start with a fork of
> RawLocalFileSystem (which will call OutputStream flush operations in
> response to hflush/hsync) that properly advertises its
> StreamCapabilities to say that it supports the operations we need.
> Alternatively we could make our own implementation of FileSystem that
> uses NIO stuff. Either of these approaches would solve both problems.
> 
> On Wed, Apr 15, 2020 at 11:40 AM Nick Dimiduk 
 wrote:
>> 
>> Hi folks,
>> 
>> I'd like to bring up the topic of the experience of new users as it
>> pertains to use of the `LocalFileSystem` and its associated (lack of)
> data
>> durability guarantees. By default, an unconfigured HBase runs with its
> root
>> directory on a `file:///` path. This patch is picked up as an instance
 of
>> `LocalFileSystem`. Hadoop has long offered this class, but it has
>> never
>> supported `hsync` or `hflush` stream characteristics. Thus, when HBase
> runs
>> on this configuration, it is unable to ensure that WAL writes are
> durable,
>> and thus will ACK a write without this assurance. This is the case,
 even
>> when running in a fully durable WAL mode.
>> 
>> This impacts a new user, someone kicking the tires on HBase following
 our
>> Getting Started docs. On Hadoop 2.8 and before, an unconfigured HBase
> will
>> WARN and cary on. Hadoop 2.10+, HBase will refuse to start. The book
>> describes a process of disabling enforcement of stream capability
>> enforcement as a first step. This is a mandatory configuration for
> running
>> HBase directly out of our binary distribution.
>> 
>> HBASE-24086 restores the behavior on Hadoop 2.10+ to that of running
>> on
>> 2.8: log a warning and cary on. The critique of this approach is that
> it's
>> far too subtle, too quiet for a system operating in a state known to
 not
>> provide data durability.
>> 
>> I have two assumptions/concerns around the state of things, which
> prompted
>> my solution on HBASE-24086 and the associated doc update on
 HBASE-24106.
>> 
>> 1. No one should be running a production system on

[jira] [Resolved] (HBASE-24158) [Flakey Tests] TestAsyncTableGetMultiThreaded

2020-04-16 Thread Michael Stack (Jira) 


 [ 
https://issues.apache.org/jira/browse/HBASE-24158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Stack resolved HBASE-24158.
---
Resolution: Fixed

Re-resolving after pushing addendum (see attached). Pushed to branch-2.2+ -- 
its a but in the AsyncRegionLocatorHelper.

> [Flakey Tests] TestAsyncTableGetMultiThreaded
> -
>
> Key: HBASE-24158
> URL: https://issues.apache.org/jira/browse/HBASE-24158
> Project: HBase
>  Issue Type: Bug
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.2.5
>
> Attachments: 
> 0001-HBASE-24158-Flakey-Tests-TestAsyncTableGetMultiThrea.addendum.patch, 
> 0001-HBASE-24158-Flakey-Tests-TestAsyncTableGetMultiThrea.patch
>
>
> I've already cut down the number of threads used by this test but it failed 
> in nightly last night unable to close out its xml and locally it failed too 
> in a run overnight. I ran it under harness and it seems well-behaved. It 
> doesn't use much memory -- 700MB -- and thread counts are usual (~450).  It 
> does use near 100% CPU which is a little unusual. Otherwise, looks fine.
> Let me keep an eye on it. Could down the thread count more and use less 
> processes... this makes it use less CPU. There does seems a bunch of overlap 
> with tests done elsewhere.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Reopened] (HBASE-24158) [Flakey Tests] TestAsyncTableGetMultiThreaded

2020-04-16 Thread Michael Stack (Jira) 


 [ 
https://issues.apache.org/jira/browse/HBASE-24158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Stack reopened HBASE-24158:
---

Reopening to apply addendum to address NPE above.

> [Flakey Tests] TestAsyncTableGetMultiThreaded
> -
>
> Key: HBASE-24158
> URL: https://issues.apache.org/jira/browse/HBASE-24158
> Project: HBase
>  Issue Type: Bug
>Reporter: Michael Stack
>Assignee: Michael Stack
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: 
> 0001-HBASE-24158-Flakey-Tests-TestAsyncTableGetMultiThrea.patch
>
>
> I've already cut down the number of threads used by this test but it failed 
> in nightly last night unable to close out its xml and locally it failed too 
> in a run overnight. I ran it under harness and it seems well-behaved. It 
> doesn't use much memory -- 700MB -- and thread counts are usual (~450).  It 
> does use near 100% CPU which is a little unusual. Otherwise, looks fine.
> Let me keep an eye on it. Could down the thread count more and use less 
> processes... this makes it use less CPU. There does seems a bunch of overlap 
> with tests done elsewhere.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-24199) Procedure related metrics is not displayed in the JMX metric dump

2020-04-16 Thread ramkrishna.s.vasudevan (Jira) 
ramkrishna.s.vasudevan created HBASE-24199:
--

 Summary: Procedure related metrics is not displayed in the JMX 
metric dump
 Key: HBASE-24199
 URL: https://issues.apache.org/jira/browse/HBASE-24199
 Project: HBase
  Issue Type: Improvement
  Components: metrics
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
 Fix For: 3.0.0, 2.3.0, 2.4.0, 2.1.10, 2.2.5


We have ProcedureMetrics and that is being tracked for every procedure that we 
create for all the ops in the system. 
But when we check the UI, the UI does not display those information at all. It 
may be useful to know atleast in the case of ServerCrashProcedure exactly to 
know how much it has taken for the procedure to complete. Similarly other 
procedures also can be added to the UI. 
Currently checked in master code - but think it will apply to other branches 
also. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HBASE-24198) Make nightly job run hadoop3 check on JDK8

2020-04-16 Thread Duo Zhang (Jira) 
Duo Zhang created HBASE-24198:
-

 Summary: Make nightly job run hadoop3 check on JDK8
 Key: HBASE-24198
 URL: https://issues.apache.org/jira/browse/HBASE-24198
 Project: HBase
  Issue Type: Sub-task
  Components: hadoop2, hadoop3, pom, scripts
Reporter: Duo Zhang


Now we will run hadoop2 check on jdk8 and hadoop3 check on jdk11, since the 
support of hadoop2 has been dropped, let's make jdk8 check also on hadoop3.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HBASE-24170) Remove hadoop-2.0 profile

2020-04-16 Thread Duo Zhang (Jira) 


 [ 
https://issues.apache.org/jira/browse/HBASE-24170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Duo Zhang resolved HBASE-24170.
---
Hadoop Flags: Reviewed
  Resolution: Fixed

Merged to master.

Thanks [~stack] and [~busbey] for reviewing.

> Remove hadoop-2.0 profile
> -
>
> Key: HBASE-24170
> URL: https://issues.apache.org/jira/browse/HBASE-24170
> Project: HBase
>  Issue Type: Sub-task
>  Components: hadoop2, hadoop3, pom
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HBASE-24151) [rsgroup] The master server aborted for IllegalThreadStateException

2020-04-16 Thread Viraj Jasani (Jira) 


 [ 
https://issues.apache.org/jira/browse/HBASE-24151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Viraj Jasani resolved HBASE-24151.
--
Fix Version/s: 2.2.5
   2.1.10
   2.3.0
 Hadoop Flags: Reviewed
   Resolution: Fixed

Pushed to all branch-2s. Thanks for contribution [~gk_coder]

> [rsgroup] The master server  aborted for IllegalThreadStateException
> 
>
> Key: HBASE-24151
> URL: https://issues.apache.org/jira/browse/HBASE-24151
> Project: HBase
>  Issue Type: Bug
>  Components: rsgroup
>Affects Versions: 2.3.0, 2.1.7, 2.2.3
>Reporter: kangkang.guo
>Assignee: kangkang.guo
>Priority: Minor
> Fix For: 2.3.0, 2.1.10, 2.2.5
>
>
> if 
> 'hbase.master.loadbalancer.class=org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer'
>  and 
> 'hbase.rsgroup.grouploadbalancer.class=org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer',
> The master server will aborted for IllegalThreadStateException.
> {code:java}
> 2020-04-09 11:24:26,142 ERROR [master/xx.xx.xx.xx:16000:becomeActiveMaster] 
> master.HMaster: Failed to become active master
> java.lang.IllegalThreadStateException
> at java.lang.Thread.start(Thread.java:705)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl.start(RSGroupInfoManagerImpl.java:172)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.initialize(RSGroupBasedLoadBalancer.java:417)
> at 
> org.apache.hadoop.hbase.rsgroup.RSGroupBasedLoadBalancer.initialize(RSGroupBasedLoadBalancer.java:430)
> at 
> org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:968)
> at 
> org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2275)
> at 
> org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:584)
> at java.lang.Thread.run(Thread.java:745)
> 2020-04-09 11:24:26,144 ERROR [master/xx.xx.xx.xx:16000:becomeActiveMaster] 
> master.HMaster: Master server abort: loaded coprocessors are: 
> [org.apache.hadoop.hbase.security.access.AccessController, 
> org.apache.hadoop.hbase.quotas.MasterQuotasObserver, 
> org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint]{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HBASE-24197) TestHttpServer.testBindAddress failure with latest jetty

2020-04-16 Thread Peter Somogyi (Jira) 


 [ 
https://issues.apache.org/jira/browse/HBASE-24197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Somogyi resolved HBASE-24197.
---
Fix Version/s: 2.2.5
   2.4.0
   2.3.0
   3.0.0
   Resolution: Fixed

Pushed to branch-2.2+. Thanks for the fix [~stoty]!

> TestHttpServer.testBindAddress failure with latest jetty
> 
>
> Key: HBASE-24197
> URL: https://issues.apache.org/jira/browse/HBASE-24197
> Project: HBase
>  Issue Type: Bug
>Affects Versions: master
>Reporter: Istvan Toth
>Assignee: Istvan Toth
>Priority: Major
> Fix For: 3.0.0, 2.3.0, 2.4.0, 2.2.5
>
>
> The latest jetty version (tested with 9.4.28.v20200408 which is not included 
> in HBase yet) wraps BindException into an IOException when it fails to bind 
> to a port.
> This breaks HttpServer's findPort functionality, which manifests in 
> TestHttpServer.testBindAddress failing.
> The proposed patch handles both the old and the new jetty behaviour correctly.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HBASE-24191) HRegion#processRowsWithLocks count memstore size wrong when sync failed

2020-04-16 Thread Lijin Bin (Jira) 


 [ 
https://issues.apache.org/jira/browse/HBASE-24191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lijin Bin resolved HBASE-24191.
---
Resolution: Fixed

Push to branch-1

> HRegion#processRowsWithLocks count memstore size wrong when sync failed
> ---
>
> Key: HBASE-24191
> URL: https://issues.apache.org/jira/browse/HBASE-24191
> Project: HBase
>  Issue Type: Bug
>  Components: regionserver
>Affects Versions: 1.7.0
>Reporter: Lijin Bin
>Assignee: Lijin Bin
>Priority: Major
> Fix For: 1.7.0
>
>
> HRegion#processRowsWithLocks when wal sync failed, it will roll back the 
> memstore in Store and reduce the size in DefaultMemStore, but still add 
> memStoreSize to HRegion#memstoreSize.
> And this only affect branch-1 verions, not apply to master/branch-2.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Build failed in Jenkins: HBase-Nightly-ARM #80

2020-04-16 Thread Apache Jenkins Server 
See 


Changes:

[github] HBASE-24183 [flakey test] replication.TestAddToSerialReplicationPeer

[stack] HBASE-23697 Document new RegionProcedureStore operation and migration

[stack] HBASE-24175 [Flakey Tests] TestSecureExportSnapshot


--
[...truncated 934.91 KB...]
[INFO] ---
[INFO]  T E S T S
[INFO] ---
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] 
[INFO] --- maven-surefire-plugin:3.0.0-M4:test 
(secondPartTestsExecution) @ hbase-shell ---
[INFO] 
[INFO] ---
[INFO]  T E S T S
[INFO] ---
[INFO] Running 
org.apache.hadoop.hbase.client.TestShellNoCluster
[INFO] Tests run: 1, Failures: 0, Errors: 0, 
Skipped: 0, Time elapsed: 6.093 s - in 
org.apache.hadoop.hbase.client.TestShellNoCluster
[INFO] Running org.apache.hadoop.hbase.client.TestShell
[INFO] Running org.apache.hadoop.hbase.client.TestTableShell
[INFO] Tests run: 1, Failures: 0, Errors: 0, 
Skipped: 0, Time elapsed: 24.172 s - in 
org.apache.hadoop.hbase.client.TestTableShell
[INFO] Tests run: 1, Failures: 0, Errors: 0, 
Skipped: 0, Time elapsed: 36.388 s - in 
org.apache.hadoop.hbase.client.TestShell
[INFO] Running 
org.apache.hadoop.hbase.client.TestReplicationShell
[INFO] Running org.apache.hadoop.hbase.client.TestRSGroupShell
[INFO] Tests run: 1, Failures: 0, Errors: 0, 
Skipped: 0, Time elapsed: 17.487 s - in 
org.apache.hadoop.hbase.client.TestRSGroupShell
[INFO] Running org.apache.hadoop.hbase.client.TestQuotasShell
[INFO] Tests run: 1, Failures: 0, Errors: 0, 
Skipped: 0, Time elapsed: 119.706 s - in 
org.apache.hadoop.hbase.client.TestReplicationShell
[INFO] Running org.apache.hadoop.hbase.client.TestAdminShell2
[INFO] Tests run: 1, Failures: 0, Errors: 0, 
Skipped: 0, Time elapsed: 141.875 s - in 
org.apache.hadoop.hbase.client.TestAdminShell2
[INFO] Running org.apache.hadoop.hbase.client.TestAdminShell
[INFO] Tests run: 1, Failures: 0, Errors: 0, 
Skipped: 0, Time elapsed: 363.277 s - in 
org.apache.hadoop.hbase.client.TestQuotasShell
[INFO] Tests run: 1, Failures: 0, Errors: 0, 
Skipped: 0, Time elapsed: 158.949 s - in 
org.apache.hadoop.hbase.client.TestAdminShell
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] 
[INFO] --< 
org.apache.hbase:hbase-endpoint >---
[INFO] Building Apache HBase - Coprocessor Endpoint 
3.0.0-SNAPSHOT  [22/41]
[INFO] [ jar 
]-
[INFO] 
[INFO] --- maven-clean-plugin:3.1.0:clean 
(default-clean) @ hbase-endpoint ---
[INFO] Deleting 

[INFO] 
[INFO] --- build-helper-maven-plugin:3.0.0:bsh-property 
(negate-license-bundles-property) @ hbase-endpoint ---
[INFO] 
[INFO] --- 
build-helper-maven-plugin:3.0.0:regex-property 
(create-license-file-path-property) @ hbase-endpoint ---
[INFO] No match to regex '\\' found in 
'
 The initial value 
'
 is left as-is...
[INFO] 
[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce 
(enforce-maven-version) @ hbase-endpoint ---
[INFO] 
[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce 
(hadoop-profile-min-maven-min-java-banned-xerces) @ 
hbase-endpoint ---
[INFO] 
[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce 
(banned-jsr305) @ hbase-endpoint ---
[INFO]

Re: [DISCUSS] New User Experience and Data Durability Guarantees on LocalFileSystem (HBASE-24086)

2020-04-16 Thread Duo Zhang 
Maybe still keep the enforcement on stream capability but provide a docker
image for quick start? In the docker image we could set the flag to false.

Vladimir Rodionov  于2020年4月16日周四 上午6:55写道：

> This should work for locally attached storage for sure.
>
> On Wed, Apr 15, 2020 at 3:52 PM Vladimir Rodionov 
> wrote:
>
> > FileOutputStream.getFileChannel().force(true) will get all durability we
> > need. Just a simple code change?
> >
> >
> > On Wed, Apr 15, 2020 at 12:32 PM Andrew Purtell <
> andrew.purt...@gmail.com>
> > wrote:
> >
> >> This thread talks of “durability” via filesystem characteristics but
> also
> >> for single system quick Start type deployments. For durability we need
> >> multi server deployments. No amount of hacking a single system
> deployment
> >> is going to give us durability as users will expect (“don’t lose my
> data”).
> >> I believe my comments are on topic.
> >>
> >>
> >> > On Apr 15, 2020, at 11:03 AM, Nick Dimiduk 
> wrote:
> >> >
> >> > On Wed, Apr 15, 2020 at 10:28 AM Andrew Purtell  >
> >> wrote:
> >> >
> >> >> Nick's mail doesn't make a distinction between avoiding data loss via
> >> >> typical tmp cleaner configurations, unfortunately adjacent to mention
> >> of
> >> >> "durability", and real data durability, which implies more than what
> a
> >> >> single system configuration can offer, no matter how many tweaks we
> >> make to
> >> >> LocalFileSystem. Maybe I'm being pedantic but this is something to be
> >> >> really clear about IMHO.
> >> >>
> >> >
> >> > I prefer to focus the attention of this thread to the question of data
> >> > durability via `FileSystem` characteristics. I agree that there are
> >> > concerns of durability (and others) around the use of the path under
> >> /tmp.
> >> > Let's keep that discussion in the other thread.
> >> >
> >> >> On Wed, Apr 15, 2020 at 10:05 AM Sean Busbey 
> >> wrote:
> >> >>
> >> >>> I think the first assumption no longer holds. Especially with the
> move
> >> >>> to flexible compute environments I regularly get asked by folks what
> >> >>> the smallest HBase they can start with for production. I can keep
> >> >>> saying 3/5/7 nodes or whatever but I guarantee there are folks who
> >> >>> want to and will run HBase with a single node. Probably those
> >> >>> deployments won't want to have the distributed flag set. None of
> them
> >> >>> really have a good option for where the WALs go, and failing loud
> when
> >> >>> they try to go to LocalFileSystem is the best option I've seen so
> far
> >> >>> to make sure folks realize they are getting into muddy waters.
> >> >>>
> >> >>> I agree with the second assumption. Our quickstart in general is too
> >> >>> complicated. Maybe if we include big warnings in the guide itself,
> we
> >> >>> could make a quickstart specific artifact to download that has the
> >> >>> unsafe disabling config in place?
> >> >>>
> >> >>> Last fall I toyed with the idea of adding an "hbase-local" module to
> >> >>> the hbase-filesystem repo that could start us out with some
> >> >>> optimizations for single node set ups. We could start with a fork of
> >> >>> RawLocalFileSystem (which will call OutputStream flush operations in
> >> >>> response to hflush/hsync) that properly advertises its
> >> >>> StreamCapabilities to say that it supports the operations we need.
> >> >>> Alternatively we could make our own implementation of FileSystem
> that
> >> >>> uses NIO stuff. Either of these approaches would solve both
> problems.
> >> >>>
> >> >>> On Wed, Apr 15, 2020 at 11:40 AM Nick Dimiduk 
> >> >> wrote:
> >> 
> >>  Hi folks,
> >> 
> >>  I'd like to bring up the topic of the experience of new users as it
> >>  pertains to use of the `LocalFileSystem` and its associated (lack
> of)
> >> >>> data
> >>  durability guarantees. By default, an unconfigured HBase runs with
> >> its
> >> >>> root
> >>  directory on a `file:///` path. This patch is picked up as an
> >> instance
> >> >> of
> >>  `LocalFileSystem`. Hadoop has long offered this class, but it has
> >> never
> >>  supported `hsync` or `hflush` stream characteristics. Thus, when
> >> HBase
> >> >>> runs
> >>  on this configuration, it is unable to ensure that WAL writes are
> >> >>> durable,
> >>  and thus will ACK a write without this assurance. This is the case,
> >> >> even
> >>  when running in a fully durable WAL mode.
> >> 
> >>  This impacts a new user, someone kicking the tires on HBase
> following
> >> >> our
> >>  Getting Started docs. On Hadoop 2.8 and before, an unconfigured
> HBase
> >> >>> will
> >>  WARN and cary on. Hadoop 2.10+, HBase will refuse to start. The
> book
> >>  describes a process of disabling enforcement of stream capability
> >>  enforcement as a first step. This is a mandatory configuration for
> >> >>> running
> >>  HBase directly out of our binary distribution.
> >> 
> >>  HBASE-24086 restores the behavior on Hadoop 2.10+ to that of
>