[RESULT] Re: [VOTE] First release candidate for HBase 1.2.6 is available

[Sean Busbey]

With five binding +1s and one non-binding +1, this vote passes.

Thanks folks!



On Mon, May 29, 2017 at 11:31 AM, Sean Busbey  wrote:
> The first release candidate for HBase 1.2.6 is available for download at:
>
> https://dist.apache.org/repos/dist/dev/hbase/hbase-1.2.6RC0/
>
> Maven artifacts are also available in a staging repository at:
>
> https://repository.apache.org/content/repositories/orgapachehbase-1168/
>
> Artifacts are signed with my key (0D80DB7C) published in our KEYS
> file at http://www.apache.org/dist/hbase/KEYS
>
> The RC corresponds to the signed tag 1.2.6RC0, which currently points
> to commit ref
>
> 2f9b9e17d0522e36063bf52ecc58576243d20b3f
>
> The detailed source and binary compatibility report vs 1.2.5 has been
> published for your review, at:
>
> https://s.apache.org/hbase-1.2.6-rc0-compat-report
>
> HBase 1.2.6 is the sixth maintenance release in the HBase 1.2 line,
> continuing on the theme of bringing a stable, reliable database to
> the Hadoop and NoSQL communities. This release includes over 25
> bug fixes since 1.2.5.
>
> Critical fixes include:
>
> * HBASE-17287 - Master becomes a zombie if filesystem object closes
> * HBASE-16630 - Fragmentation in long running Bucket Cache
> * HBASE-17717 - Incorrect ZK ACL set for HBase superuser
> * HBASE-15941 - HBCK repair should not unsplit healthy splitted region
>
> The full list of fixes included in this release is available at:
>
> https://s.apache.org/hbase-1.2.6-jira-release-notes
>
> and in the CHANGES.txt file included in the distribution.
>
> Please try out this candidate and vote +1/-1 on whether we should
> release these artifacts as HBase 1.2.6.
>
> The VOTE will remain open for at least 72 hours. Given sufficient votes
> I would like to close it on June 5th, 2017.
>
> thanks!

Re: [VOTE] First release candidate for HBase 1.2.6 is available

[Sean Busbey]

no worries. thanks for the follow up!

On Fri, Jun 2, 2017 at 9:27 PM, Alexander Leblang
 wrote:
> Unfortunately it took me longer than I expected to understand where to go
> to change the java version so I won't be able to test this rc by the end of
> this week. So don't wait on my account. Hopefully I can make this into a
> regular thing and I can test the next one.
>
> On Thu, Jun 1, 2017 at 11:25 AM, Sean Busbey  wrote:
>
>> On Thu, Jun 1, 2017 at 1:13 PM, Alexander Leblang
>>  wrote:
>> > If you wouldn't mind, I'd like some more time. I'm working on using
>> > clusterdock to run some testing on the candidate.
>> >
>> > - Alex
>> >
>>
>> Sounds great Alex, I'll hold off. Feel free to ping the list if you
>> get stuck on clusterdock. I think it's been a while since any of us
>> have dug in, but hopefully we can still help.
>>
>> --
>> Sean
>>

Re: [VOTE] First release candidate for HBase 1.2.6 is available

[Alexander Leblang]

Unfortunately it took me longer than I expected to understand where to go
to change the java version so I won't be able to test this rc by the end of
this week. So don't wait on my account. Hopefully I can make this into a
regular thing and I can test the next one.

On Thu, Jun 1, 2017 at 11:25 AM, Sean Busbey  wrote:

> On Thu, Jun 1, 2017 at 1:13 PM, Alexander Leblang
>  wrote:
> > If you wouldn't mind, I'd like some more time. I'm working on using
> > clusterdock to run some testing on the candidate.
> >
> > - Alex
> >
>
> Sounds great Alex, I'll hold off. Feel free to ping the list if you
> get stuck on clusterdock. I think it's been a while since any of us
> have dug in, but hopefully we can still help.
>
> --
> Sean
>

[jira] [Resolved] (HBASE-18153) [C++] Add valgrind to the docker image

[Enis Soztutar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-18153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Enis Soztutar resolved HBASE-18153.
---
   Resolution: Fixed
Fix Version/s: HBASE-14850

Pushed this, trivial patch. 

> [C++] Add valgrind to the docker image
> --
>
> Key: HBASE-18153
> URL: https://issues.apache.org/jira/browse/HBASE-18153
> Project: HBase
>  Issue Type: Sub-task
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: HBASE-14850
>
> Attachments: hbase-18153_v1.patch
>
>
> We need valgrind. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HBASE-18153) Add valgrind to the docker image

[Enis Soztutar (JIRA)]

Enis Soztutar created HBASE-18153:
-

 Summary: Add valgrind to the docker image
 Key: HBASE-18153
 URL: https://issues.apache.org/jira/browse/HBASE-18153
 Project: HBase
  Issue Type: Sub-task
Reporter: Enis Soztutar
Assignee: Enis Soztutar


We need valgrind. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Resolved] (HBASE-17530) Readd TestMergeTableRegionsProcedure mistakenly removed by HBASE-16786

[stack (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

stack resolved HBASE-17530.
---
Resolution: Fixed

The test was added back as part of the HBASE-14614 commit. Resolving.

> Readd TestMergeTableRegionsProcedure mistakenly removed by HBASE-16786
> --
>
> Key: HBASE-17530
> URL: https://issues.apache.org/jira/browse/HBASE-17530
> Project: HBase
>  Issue Type: Sub-task
>  Components: test
>Reporter: stack
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: 
> 0001-HBASE-17530-Readd-TestMergeTableRegionsProcedure-mis.patch, 
> HBASE-17530.master.001.patch
>
>
> HBASE-16786 mistakenly removed the test TestMergeTableRegionsProcedure. Need 
> to restore. Filing issue here as critical. Test does basic exercise of merge 
> functionality. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Hadoop QA failed to build in branch-1 & branch-1.3

[Ted Yu]

Here is the download page for JDK 7 from Oracle:

http://www.oracle.com/technetwork/java/javase/downloads/java-archive-
downloads-javase7-521261.html

Even if we accept license on the above page, clicking download will
navigate to login page. This is why it's getting 404.

If we continue to use Oracle JDK 7, whose login should we use ?

On Thu, Jun 1, 2017 at 7:06 AM, Sean Busbey  wrote:

> Are we willing to switch to OpenJDK for our Java 7 tests?
>
> Has anyone checked to see if there's an updated means for install JDK
> 7 from Oracle that works?
>
> On Wed, May 31, 2017 at 11:25 PM, Ted Yu  wrote:
> > Here is the action from hadoop to get around this problem: HADOOP-14474
> >
> > On Tue, May 30, 2017 at 8:26 PM, OpenInx  wrote:
> >
> >> The current problem is:  we got 404 not found returned when we install
> jdk7
> >> ( Oracle Corp changed the jdk7 download link and need accept it's
> license &
> >> login with oracle account ) .
> >> Could we download  JDK7 from our own server, which is independent of
> >> oracle's ?
> >>
> >> On Tue, May 30, 2017 at 10:38 PM, Mike Drob  wrote:
> >>
> >> > The anecdotal assumption is that Oracle JDK is more stable than
> OpenJDK,
> >> > right? So shouldn't we be running tests with the OpenJDK. Kind of
> like a
> >> > runner practicing with weights on their legs before the actual
> marathon.
> >> >
> >> > Mike
> >> >
> >> > On Sun, May 28, 2017 at 9:18 PM, Sean Busbey 
> >> > wrote:
> >> >
> >> > > I first heard about the issue from Duo on JIRA, I do'nt know if
> anyone
> >> > > has made a jira yet.
> >> > >
> >> > > The primary reason I can think of to test Oracle JDK instead of
> >> > > OpenJDK is the presumption that the Oracle JDK will be more common
> in
> >> > > production settings.
> >> > >
> >> > > We don't have, like, a survey of users@ to back that up of course.
> >> > > ¯\_(ツ)_/¯
> >> > >
> >> > > On Sat, May 27, 2017 at 8:26 PM, Dima Spivak  >
> >> > > wrote:
> >> > > > Oops, gotcha. Should have scrolled further below the checking out
> of
> >> > the
> >> > > > master branch; I see further below that it was the 1.3 job.
> >> > > >
> >> > > > Is there a JIRA to address the JDK package not being available,
> Sean?
> >> > Any
> >> > > > reason we insist upon using Oracle for these runs instead of the
> >> > OpenJDK
> >> > > > equivalent?
> >> > > >
> >> > > > On Sat, May 27, 2017 at 12:14 PM Sean Busbey 
> >> > wrote:
> >> > > >
> >> > > >> Also, this is specifically when the per-branch docker image is
> >> > > >> attempting to install the jdk(s) that are appropriate for the
> given
> >> > > >> branch.
> >> > > >>
> >> > > >> like branch-1 that is mentioned above is here:
> >> > > >>
> >> > > >> https://github.com/apache/hbase/blob/branch-1/dev-
> >> > > support/docker/Dockerfile
> >> > > >>
> >> > > >> On Sat, May 27, 2017 at 2:10 PM, Sean Busbey 
> >> > wrote:
> >> > > >> > Dima, the issue pointed out above is specific to the branch-1*
> >> > > >> > branches that do support java 7. the master branch already
> doesn't
> >> > do
> >> > > >> > any java 7, which is why the failures don't happen on it.
> >> > > >> >
> >> > > >> > On Sat, May 27, 2017 at 11:58 AM, Dima Spivak <
> >> > dimaspi...@apache.org>
> >> > > >> wrote:
> >> > > >> >> Hey there,
> >> > > >> >>
> >> > > >> >> The master branch actually doesn't support Java 7 at all, so
> we
> >> > > should
> >> > > >> >> remove that; good catch. Would you mind opening a JIRA for the
> >> > issue
> >> > > and
> >> > > >> >> one of the members of the team with Jenkins access can address
> >> it?
> >> > > >> >>
> >> > > >> >> On Sat, May 27, 2017 at 3:18 AM OpenInx 
> >> wrote:
> >> > > >> >>
> >> > > >> >>> Hi Guys
> >> > > >> >>>
> >> > > >> >>> Seems like we always fail to install jdk7 for Hadoop QA
> >> > currently. (
> >> > > >> RUN
> >> > > >> >>> apt-get -q update && apt-get -q install -y
> >> oracle-java7-installer
> >> > > >> failed)
> >> > > >> >>>
> >> > > >> >>> I'm not familiar with Hadoop QA environment,   Could anyone
> help
> >> > to
> >> > > (or
> >> > > >> >>> tell how to)  fix it ?
> >> > > >> >>>
> >> > > >> >>> 1.  https://builds.apache.org/job/
> PreCommit-HBASE-Build/6921/
> >> > > console
> >> > > >> >>> 2.  https://builds.apache.org/job/
> PreCommit-HBASE-Build/6960/
> >> > > console
> >> > > >> >>> 3.  https://issues.apache.org/jira/browse/HBASE-18066?
> >> > > >> >>> focusedCommentId=16025718=com.atlassian.jira.
> >> > > >> >>> plugin.system.issuetabpanels:comment-tabpanel#comment-160257
> 18
> >> > > >> >>>
> >> > > >> >>>
> >> > > >> >>> Thanks.
> >> > > >> >>>
> >> > > >> >> --
> >> > > >> >> -Dima
> >> > > >>
> >> > > > --
> >> > > > -Dima
> >> > >
> >> > >
> >> > >
> >> > > --
> >> > > Sean
> >> > >
> >> >
> >>
> >>
> >>
> >> --
> >> ==
> >> Openinx  blog : 

[jira] [Reopened] (HBASE-15995) Separate replication WAL reading from shipping

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu reopened HBASE-15995:


> Separate replication WAL reading from shipping
> --
>
> Key: HBASE-15995
> URL: https://issues.apache.org/jira/browse/HBASE-15995
> Project: HBase
>  Issue Type: Sub-task
>  Components: Replication
>Affects Versions: 2.0.0
>Reporter: Vincent Poon
>Assignee: Vincent Poon
> Fix For: 2.0.0
>
> Attachments: HBASE-15995.branch-1.v7.patch, 
> HBASE-15995.master.v1.patch, HBASE-15995.master.v2.patch, 
> HBASE-15995.master.v3.patch, HBASE-15995.master.v4.patch, 
> HBASE-15995.master.v6.patch, HBASE-15995.master.v7.patch, 
> replicationV1_100ms_delay.png, replicationV2_100ms_delay.png
>
>
> Currently ReplicationSource reads edits from the WAL and ships them in the 
> same thread.
> By breaking out the reading from the shipping, we can introduce greater 
> parallelism and lay the foundation for further refactoring to a pipelined, 
> streaming model.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HBASE-18152) [AMv2] Corrupt Procedure WAL file; procedure data stored out of order

[stack (JIRA)]

stack created HBASE-18152:
-

 Summary: [AMv2] Corrupt Procedure WAL file; procedure data stored 
out of order
 Key: HBASE-18152
 URL: https://issues.apache.org/jira/browse/HBASE-18152
 Project: HBase
  Issue Type: Bug
  Components: Region Assignment
Affects Versions: 2.0.0
Reporter: stack
Assignee: stack
Priority: Critical
 Fix For: 2.0.0


I've seen corruption from time-to-time testing.  Its rare enough. Often we can 
get over it but sometimes we can't. It took me a while to capture an instance 
of corruption. Turns out we are write to the WAL out-of-order which undoes a 
basic tenet; that WAL content is ordered in line w/ execution.

Below I'll post a corrupt WAL.

Looking at the write-side, there is a lot going on. I'm not clear on how we 
could write out of order. Will try and get more insight. Meantime parking this 
issue here to fill data into.





--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HBASE-18151) Improvement on Histogram

[Phil Yang (JIRA)]

Phil Yang created HBASE-18151:
-

 Summary: Improvement on Histogram
 Key: HBASE-18151
 URL: https://issues.apache.org/jira/browse/HBASE-18151
 Project: HBase
  Issue Type: Improvement
Reporter: Phil Yang


There are some bugs and some places we can improvement in the new native 
Histogram.

The default maxExpected value in HistogramImpl is Integer.MAX_VALUE << 2 
(https://github.com/apache/hbase/blob/master/hbase-metrics/src/main/java/org/apache/hadoop/hbase/metrics/impl/HistogramImpl.java#L41
 ) which is overflow... and will be regarded as -4... And binsMax pf 
FastLongHistogram will be 255(by the reason of this bug) which is too low (In 
FastLongHistogram#L75 it says the expected binsMax is 1000? I think it may be 
also too low. Even if there is no overflow, Integer.MAX_VALUE*4 is too large?

We may need pass the right expected min/max value when we use it because the 
value may be different in scenarios. And the default value should not be 1000 
or Integer.MAX_VALUE*4.

There is an assumption that the value is in uniform distribution. But if the 
value is latency which is the most use case, the distribution is not uniform. 
The lower the value, the higher the probability.

Because of these reasons, the 75/90/99 or other percentages may have big error 
than we expected. We need find a way to reduce the error.




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HBASE-18150) hbase.version file is created under both hbase.rootdir and hbase.wal.dir

[Xiang Li (JIRA)]

Xiang Li created HBASE-18150:


 Summary: hbase.version file is created under both hbase.rootdir 
and hbase.wal.dir
 Key: HBASE-18150
 URL: https://issues.apache.org/jira/browse/HBASE-18150
 Project: HBase
  Issue Type: Bug
Reporter: Xiang Li
Assignee: Xiang Li
Priority: Critical






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)