Re: [VOTE] First release candidate for HBase 1.2.9 is available

[Ashish Singhi]

+1
- Built tar ball from src
- Exercised some shell commands
- Checked process and audit logs for operations
- mvn test passed

Regards,
Ashish

On Sun, Nov 18, 2018 at 11:39 AM Sean Busbey  wrote:

> The first release candidate for HBase 1.2.9 is available for download:
>
> https://dist.apache.org/repos/dist/dev/hbase/hbase-1.2.9RC0/
>
> Maven artifacts are also available in a staging repository at:
>
> https://repository.apache.org/content/repositories/orgapachehbase-1236/
>
> Artifacts are signed with my key (0D80DB7C) published in our KEYS
> file at http://www.apache.org/dist/hbase/KEYS
>
> The RC corresponds to the signed tag 1.2.9RC0, which currently points
>  to commit ref
>
>  fd0d55b1e5ef54eb9bf60cce1f0a8e4c1da073ef
>
>  HBase 1.2.9 is the ninth maintenance release in the HBase 1.2 line,
>  continuing on the theme of bringing a stable, reliable database to
>  the Hadoop and NoSQL communities. This release includes about a half
>  dozen bug fixes done in the month since 1.2.8.
>
>  The detailed source and binary compatibility report vs 1.2.8 has been
>  published for your review, at:
>
>  https://s.apache.org/hbase-1.2.9-rc0-compat-report
>
>  The report shows no incompatibilities.
>
>  Critical fixes include:
>
>  * HBASE-21347 Backport HBASE-21200 "Memstore flush doesn't finish
>because of seekToPreviousRow() in memstore scanner." to
>branch-1
>  * HBASE-21357 RS should abort if OOM in Reader thread
>  * HBASE-20604 ProtobufLogReader#readNext can incorrectly loop to the
>same position in the stream until the the WAL is rolled
>
>  The full list of fixes included in this release is available at:
>
>  https://s.apache.org/hbase-1.2.9-jira-release-notes
>
>  and in the CHANGES.txt file included in the distribution.
>
>  Please try out this candidate and vote +1/-1 on whether we should
>  release these artifacts as HBase 1.2.9.
>
>  The VOTE will remain open for at least 72 hours. Given sufficient votes
>  I would like to close it on November 26th, 2018.
>
>  thanks!
>
>  -busbey
>
>  as of this email the posted artifacts have the following SHA512:
>
>  hbase-1.2.9-src.tar.gz:
>  49498136 FA865B7C E70C0A81 18F6B221 CA3D0266 C26DDDE2
>  513C206C 4D28E228 048171C2 DD608DB2 0DA164BD ADBED4BF
>  80DC40E4 6D93CC11 6C25C617 CA25E5A7
>
>  hbase-1.2.9-bin.tar.gz:
>  4D9E1278 8634E250 841E55CB 8879EAC1 5C9BCF7F 09DE018A
>  AD0CCB0E 118C2D02 BCA5F4F3 EACB9857 D8236E07 6E3CD0A4
>  43B90567 3AB5C83C AD642547 037588BA
>

Re: [ANNOUNCE] New HBase committer Jingyun Tian

[Ashish Singhi]

Congratulations & Welcome!

Regards,
Ashish

On Tue, Nov 13, 2018 at 1:24 PM 张铎(Duo Zhang)  wrote:

> On behalf of the Apache HBase PMC, I am pleased to announce that Jingyun
> Tian has accepted the PMC's invitation to become a committer on the
> project. We appreciate all of Jingyun's generous contributions thus far and
> look forward to his continued involvement.
>
> Congratulations and welcome, Jingyun!
>

Re: [VOTE] First release candidate for HBase 1.2.7 is available

[Ashish Singhi]

+1
- Built tar ball from src
- Exercised some shell commands
- Checked process and audit logs for operations
- mvn test passed

Regards,
Ashish


On Sat, Sep 8, 2018 at 9:24 AM, Sean Busbey  wrote:

> The first release candidate for HBase 1.2.7 is available for download:
>
> https://dist.apache.org/repos/dist/dev/hbase/hbase-1.2.7RC0/
>
> Maven artifacts are also available in a staging repository at:
>
> https://repository.apache.org/content/repositories/orgapachehbase-1232/
>
> Artifacts are signed with my key (0D80DB7C) published in our KEYS
> file at http://www.apache.org/dist/hbase/KEYS
>
> The RC corresponds to the signed tag 1.2.7RC0, which currently points
>  to commit ref
>
> ac57c51f7ad25e312b4275665d62b34a5945422f
>
> HBase 1.2.7 is the seventh maintenance release in the HBase 1.2 line,
> continuing on the theme of bringing a stable, reliable database to
> the Hadoop and NoSQL communities. This release includes over 250
> bug fixes done in the 15 months since 1.2.6.
>
> This release candidate contains the following incompatible changes,
> details in the release notes for the specific issue:
>
> * HBASE-20884 Replace usage of our Base64 implementation with java's
> * HBASE-18577 shaded client should not include non-relocated third party
>   dependencies
> * HBASE-18142 delete in HBase shell should not delete previous versions
>   of a cell
> * HBASE-18731 some protected methods of QuotaSettings have been marked
>   IA.Private and deprecated
> * HBASE-16459 HBase shell no longer recognizes the --format option
>
> The detailed source and binary compatibility report vs 1.2.6 has been
> published for your review, at:
>
> https://s.apache.org/hbase-1.2.7-rc0-compat-report
>
> The report shows some expected incompatibilities and one false positive.
> Details on HBASE-18276.
>
> Critical fixes include:
>
> * HBASE-18036 Data locality is not maintained after cluster restart
> * HBASE-18233 We shouldn't wait for readlock in doMiniBatchMutation in
>   case of deadlock
> * HBASE-9393  Region Server fails to properly close socket resulting in
>   many CLOSE_WAIT to Data Nodes
> * HBASE-19924 RPC throttling does not work for multi() with request
>   count rater.
> * HBASE-18192 Replication drops recovered queues on Region Server
>   shutdown
> * HBASE-18282 ReplicationLogCleaner can delete WALs not yet replicated
>   in case of a KeeperException
> * HBASE-19796 ReplicationSynUp tool is not replicating data if a WAL is
>   moved to splitting directory
> * HBASE-17648 HBase table-level synchronization fails between two
>   secured(kerberized) clusters
> * HBASE-18137 Replication gets stuck for empty WALs
> * HBASE-18577 shaded client should not include non-relocated third party
>   dependencies
> * HBASE-19900 Region level exceptions destroy the result of batch client
>   operations
> * HBASE-21007 Memory leak in HBase REST server
>
> The full list of fixes included in this release is available at:
>
> https://s.apache.org/hbase-1.2.7-jira-release-notes
>
> and in the CHANGES.txt file included in the distribution.
>
> Please try out this candidate and vote +1/-1 on whether we should
> release these artifacts as HBase 1.2.7.
>
> The VOTE will remain open for at least 72 hours. Given sufficient votes
> I would like to close it on September 12th, 2018.
>
> thanks!
>
> -busbey
>
> as of this email the posted artifacts have the following SHA512:
>
> hbase-1.2.7-bin.tar.gz:
> 00FC806A 335DFBDD 30720411 9FC0DD19 01AAC2E4 C8220B90
> D54263F4 4F49D49A 111C30D0 6E4CC6D3 249C4F5A 7A66064B
> 9EF97A92 B8E559F9 11705137 C3B652F2
>
> hbase-1.2.7-src.tar.gz:
> F142B8E2 4F615D32 2B3816B1 71A61D9A 618FDD99 1EE69772
> E3D52226 D30E34B8 0F9A469E 5AA00F6F 12AB77D6 C7FC6ADE
> 8CAB7254 8B2AF8E6 6D00E9EC D0E93AB4
>
>
>

Re: [VOTE] The first HBase 1.4.7 release candidate (RC0) is available

[Ashish Singhi]

+1
- Built tar ball from src
- Exercised some shell commands
- Checked process and audit logs for operations
- Verified replication

Regards,
Ashish

On Wed, Aug 29, 2018 at 3:40 AM, Andrew Purtell  wrote:

> The first HBase 1.4.7 release candidate (RC0) is available for download at
> https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.7RC0/ and Maven
> artifacts are available in the temporary repository
> https://repository.apache.org/content/repositories/orgapachehbase-1230/ .
>
> The git tag corresponding to the candidate is '1.4.7RC0' (763f27f583).
>
> A detailed source and binary compatibility report for this release is
> available for your review at
> https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.7RC0/
> compat-check-report.html
> . There are no reported compatibility issues.
>
> A list of the 18 issues resolved in this release can be found at
> https://s.apache.org/PvLv .
>
> Please try out the candidate and vote +1/0/-1.
>
> The vote will be open for at least 72 hours. Unless objection I will try to
> close it Monday September 3, 2018 if we have sufficient votes.
>
> Prior to making this announcement I made the following preflight checks:
>
> RAT check passes (7u80)
> Unit test suite passes (7u80)
> LTT load 1M rows with 100% verification and 20% updates (8u181)
> ITBLL 500M rows with slowDeterministic and serverKilling monkies
> (8u181)
>
> --
> Andrew
>

Re: [VOTE] Second release candidate for hbase-2.0.2 (RC1) is available for download

[Ashish Singhi]

+1
- Built tar ball from src
- Exercised some shell commands
- Checked process and audit logs for operations
- Verified replication

Regards,
Ashish

On Wed, Aug 29, 2018 at 10:08 AM, Stack  wrote:

> The second release candidate for Apache HBase 2.0.2 (RC1) is available
> for download and testing. This is a bug fix release with 100+ commits [1].
> Release notes are available here [2]. The compatibility report is at [3].
>
> Artifacts are available here:
>
>  https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.2RC1/
>
> Corresponding convenience artifacts for maven use are in the staging
> repository:
>
>  https://repository.apache.org/content/repositories/orgapachehbase-1231
>
> All artifacts are signed with my code signing key, 8ACC93D2, which is
> also in the project KEYS file:
>
> http://www.apache.org/dist/hbase/KEYS
>
> These artifacts correspond to commit ref
>
>  1cfab033e779df840d5612a85277f42a6a4e8172
>
> which has been tagged as 2.0.2RC1.
>
> Please take a few minutes to verify the release and vote on releasing it:
>
> [ ] +1 Release these artifacts as Apache HBase 2.0.2
> [ ] -1 Do not release this package because ...
>
> This VOTE thread will remain open for at least 72 hours. It will close for
> sure
> Saturday morning.
>
> Thanks,
> S
>
> 1. https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.2RC1/CHANGES.md
> 2.
> https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.2RC1/
> RELEASENOTES.md
> 3.
> https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.2RC1/
> compatibility_report_2.0.1_2.0.2.html
>

Re: Live Broadcast link for HBaseConAsia2018

[Ashish Singhi]

This is so great to see! Thanks for sharing the link.

On Fri, Aug 17, 2018 at 8:59 AM, Yu Li  wrote:

> Hi All,
>
> HBaseConAsia2018 is ongoing and here comes the live broadcast link[1].
> Please follow the link to watch the speaks if failed to come to the scene.
> Enjoy the day.
>
> 大家好，
>
> HBaseConAsia2018正在北京歌华开元大酒店进行中，对于无法到现场的用户我们提供了直播链接[1]，大家可以在网上观看直播。
>
> [1]
> https://yq.aliyun.com/promotion/631?id=129203=
> groupmessage=0
>
> Yu - On behalf of the HBaseConAsia2018 PC
>

Re: [ANNOUNCE] New Committer: Toshihiro Suzuki

[Ashish Singhi]

Congratulations and Welcome!

On Wed, Aug 1, 2018 at 8:17 PM, Josh Elser  wrote:

> On behalf of the HBase PMC, I'm pleased to announce that Toshihiro Suzuki
> (aka Toshi, brfn169) has accepted our invitation to become an HBase
> committer. This was extended to Toshi as a result of his consistent,
> high-quality contributions to HBase. Thanks for all of your hard work, and
> we look forward to working with you even more!
>
> Please join me in extending a hearty "congrats" to Toshi!
>
> - Josh
>

Re: [VOTE] Second release candidate for HBase 2.1.0 (RC1) is available

[Ashish Singhi]

+1
- Built tar ball from src
- Exercised some shell commands
- Checked process and audit logs for operations
- Verified replication

Regards,
Ashish

On Wed, Jul 11, 2018 at 6:39 AM, 张铎(Duo Zhang) 
wrote:

> The second release candidate for Apache HBase 2.1.0 is available for
> downloading and testing.
>
> Artifacts and compatibility report are available here:
>
> https://dist.apache.org/repos/dist/dev/hbase/2.1.0RC1/
>
> Notice that due to HBASE-19735, now we have a
> hbase-2.1.0-client-bin.tar.gz, which contains a reduced set of dependencies
> for client.
>
> Maven artifacts are available in the staging repository at:
>
> https://repository.apache.org/content/repositories/orgapachehbase-1225
>
> All artifacts are signed with my signing key 9AD2AE49, which is also in the
> project KEYS file at
>
> http://www.apache.org/dist/hbase/KEYS
>
> These artifacts were tagged 2.1.0RC1 at hash
> e1673bb0bbfea21d6e5dba73e013b09b8b49b89b.
>
> There are 245 issues resolved since 2.0.0. Some new features are:
>
> 1. Procedure v2 based replication peer modification. The advantage here is
> that all the replication peer modification will be synchronous, which means
> after the you come back from the call, you can make sure that all the
> region servers have loaded the changes. Please see HBASE-19397 for more
> details.
> 2. Serial Replication. Now in replication we can make sure the order of
> pushing logs is same as the order of requests from client. Please see
> HBASE-20046 and HBASE-9465 for more details.
>
> For other important changes:
> 1. The minimum hadoop version has been changed to 2.7.1.
> 2. We have successfully done a rolling upgrade from 1.4.3 to 2.1.0 which
> shows that rolling upgrade from 1.x to 2.x is possible. Notice that this is
> only a experimental feature, as there are likely uncovered corner cases in
> our limited test. See
> http://hbase.apache.org/book.html#upgrade2.0.rolling.upgrades for more
> details.
>
> And HBASE-20862 has fixed the compatibility issues. You can read the issues
> for more details about the compatibility between 2.0.0 and 2.1.0.
>
> For all the changes, please see the CHANGES.md and RELEASENOTES.md.
>
> Please take a few minutes to verify the release and vote on releasing it:
>
> [ ] +1 Release this package as Apache HBase 2.1.0
> [ ] +0 No opinion
> [ ] -1 Do not release this package because...
>
> The vote will be closed at next Tuesday, 7.17.
>
> Thanks to the myriad who have helped out with this release,
> Your 2.1.0 Release Manager
>

Re: [VOTE] Merge branch HBASE-19064 back to master

[Ashish Singhi]

SGTM. +1

Regards,
Ashish

On Tue, Jun 26, 2018 at 12:30 PM, 张铎(Duo Zhang) 
wrote:

> Here is my +1.
>
> As the performance test shows that there is no performance downgrading if
> we do not enable sync replication, and the feature basically works, I think
> it is OK to merge it back to master branch. We will continue working on
> polishing and optimizing it for the 3.0 release.
>
> 2018-06-26 14:52 GMT+08:00 OpenInx :
>
> > Hi guys
> >
> > I've tested the write performance on both master branch and  HBASE-19064
> > branch.  Here's the JIRA [1].
> >
> > The conclusion(Pasted from the comment):
> > The QPS & AVG latency are similar when comparing the master branch with
> > async replication and HBASE-19064 with async replication.
> > BTW the QPS of HBASE-19064 with sync replication dropped about 13%, we
> will
> > continue to optimize the sync replication in phrase#2 [2].
> >
> > 1. https://issues.apache.org/jira/browse/HBASE-20751
> > 2. https://issues.apache.org/jira/browse/HBASE-20422
> >
> >
> > On Thu, Jun 21, 2018 at 9:57 PM, Guanghao Zhang 
> > wrote:
> >
> > > +1
> > >
> > > 2018-06-21 18:26 GMT+08:00 mengli721014 on 163 dot com <
> > > mengli721...@163.com
> > > >:
> > >
> > > > 0
> > > >
> > > > 使用AquaMail for Android发送
> > > > http://www.aqua-mail.com
> > > >
> > > >
> > > >
> > > > 在 2018年6月21日 下午4:21:26 "张铎(Duo Zhang)"  写道:
> > > >
> > > > In HBASE-19064 we aim to implement sync replication feature for
> HBase.
> > > You
> > > >> can see the design doc for more details on how it works
> > > >>
> > > >> https://docs.google.com/document/d/193D3aOxD-muPIZuQfI4Zo3_
> > > >> qg6-Nepeu_kraYJVQkiE/edit#
> > > >>
> > > >> And now the feature basically works, we have tested it on real
> > clusters,
> > > >> and also provide the operational documentation in the ref guide. You
> > can
> > > >> see the 'Synchronous Replication' section in the ref guide of branch
> > > >> HBASE-19064.
> > > >>
> > > >> There are known limitations for this feature, and we have
> > > >> created HBASE-20422 to track them.
> > > >>
> > > >> Please vote:
> > > >> [+1] Agree
> > > >> [-1] Disagree
> > > >> [0] Neutral
> > > >>
> > > >> Thanks.
> > > >>
> > > >>
> > > >
> > > >
> > > >
> > >
> >
>

Re: [ANNOUNCE] New HBase committer Reid Chan

[Ashish Singhi]

Congrats and Welcome.

Regards,
Ashish

On Tue, Jun 26, 2018 at 7:29 AM, Chia-Ping Tsai  wrote:

> On behalf of the Apache HBase PMC, I am pleased to announce that Reid Chan
> has accepted the PMC's invitation to become a committer on the project. We
> appreciate all of Reid’s generous contributions thus far and look forward
> to his continued involvement.
>
> Congratulations and welcome, Reid!
>
> --
> Chia-Ping
>

RE: [ANNOUNCE] New HBase committer Guangxu Cheng

[ashish singhi]

Congrats and Welcome!

Regards,
Ashish 
-Original Message-
From: 张铎(Duo Zhang) [mailto:palomino...@gmail.com] 
Sent: Monday, June 04, 2018 12:30 PM
To: HBase Dev List ; hbase-user 
Subject: [ANNOUNCE] New HBase committer Guangxu Cheng

On behalf of the Apache HBase PMC, I am pleased to announce that Guangxu Cheng 
has accepted the PMC's invitation to become a committer on the project. We 
appreciate all of Guangxu's generous contributions thus far and look forward to 
his continued involvement.

Congratulations and welcome, Guangxu!

[jira] [Created] (HBASE-20590) REST Java client is not able to negotiate with the server in the secure mode

[Ashish Singhi (JIRA)]

Ashish Singhi created HBASE-20590:
-

 Summary: REST Java client is not able to negotiate with the server 
in the secure mode
 Key: HBASE-20590
 URL: https://issues.apache.org/jira/browse/HBASE-20590
 Project: HBase
  Issue Type: Bug
  Components: REST, security
Affects Versions: 1.3.1
Reporter: Ashish Singhi
Assignee: Ashish Singhi






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (HBASE-5742) LoadIncrementalHFiles throws too generic of an exception

[Ashish Singhi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-5742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi resolved HBASE-5742.
--
Resolution: Not A Problem

The constructor in the context no more exists.

> LoadIncrementalHFiles throws too generic of an exception
> 
>
> Key: HBASE-5742
> URL: https://issues.apache.org/jira/browse/HBASE-5742
> Project: HBase
>  Issue Type: Improvement
>  Components: mapreduce
>Reporter: David Capwell
>    Assignee: Ashish Singhi
>Priority: Major
>
> In 0.90 the LoadIncrementalHFiles constructor did not throw an exception, now 
> it throws Exception.  The constructor should ether not throw an exception or 
> throw ZooKeeperConnectionException, and MasterNotRunningException since those 
> come from the HBaseAdmin call.
> https://github.com/apache/hbase/blob/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java#L105



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [VOTE] The first HBase 1.4.4 release candidate (RC0) is available

[Ashish Singhi]

+1

Downloaded the binary and checked the following,

- Checked LICENSE and NOTICE files, looks ok
- Apache RAT check, looks good

Verified the following in a non-secure setup,

- Exercised basic shell commands
- Ran LTT with 1M row, checked read and write operations
- Poked around webUI
- Started Rest server and executed some commands
- Checked logs all looks like fine

Regards,
Ashish

On Wed, Apr 25, 2018 at 8:48 AM, Andrew Purtell  wrote:

> The first HBase 1.4.4 release candidate (RC0) is available for download at
> https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.4RC0/ and Maven
> artifacts are available in the temporary repository
> https://repository.apache.org/content/repositories/orgapachehbase-1214/ .
>
> The git tag corresponding to the candidate is '1.4.4RC0' (fe146eb48c).
>
> A detailed source and binary compatibility report for this release is
> available for your review at
> https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.4RC0/
> compat-check-report.html
> .
>
> A list of the 21 issues resolved in this release can be found at
> https://s.apache.org/E6GG .
>
> Please try out the candidate and vote +1/0/-1.
>
> This vote will be open for at least 72 hours. Unless objection I will try
> to close it Monday April 30, 2018 if we have sufficient votes.
>
> Prior to making this announcement I made the following preflight checks:
>
> RAT check passes (7u80)
> Unit test suite passes (7u80)
> LTT load 1M rows with 100% verification and 20% updates (8u152)
> ITBLL Loop 1 1B rows with serverKilling monkey (8u152)
>
> --
> Best regards,
> Andrew
>

RE: [VOTE] First release candidate for HBase 2.0.0 (RC0) is available

[ashish singhi]

bq. signatures & sums- NOT OK
(md5 checksums missing)

This is intentional I think, check HBASE-20385.

Regards,
Ashish 

-Original Message-
From: Umesh Agashe [mailto:uaga...@cloudera.com] 
Sent: Tuesday, April 17, 2018 4:01 AM
To: dev@hbase.apache.org
Subject: Re: [VOTE] First release candidate for HBase 2.0.0 (RC0) is available

-1 non-binding (hbck with write operations disabled not included)

download src & bin tar ball   - OK
signatures & sums- NOT OK
(md5 checksums missing)
build from source (openjdk version "1.8.0_151")  - OK
rat check   - OK
start local instance from bin & CRUD from shell  - OK
LTT write, read1 million rows, 2 cols/row  - OK
check logs - OK


On Fri, Apr 13, 2018 at 10:55 AM, Stack  wrote:

> On Fri, Apr 13, 2018 at 10:53 AM, Josh Elser  wrote:
>
> > Was poking around with PE on a few nodes (I forget the exact 
> > circumstances, need to look back at this), and ran into a case where 
> > ~35 regions were left as RIT
> >
> > 2018-04-12 22:05:24,431 ERROR 
> > [master/ctr-e138-1518143905142-221855-01-
> 02:16000]
> > procedure2.ProcedureExecutor: Corrupt pid=3580, ppid=3534, 
> > state=RUNNABLE:REGION_TRANSITION_QUEUE; AssignProcedure table=TestTable,
> > region=71fef  ffe6b5b3cf1cb6d3328a5a58690
> >
> > Saw entries like this (I think) for each region which was stuck. A 
> > simple `assign` in the shell brought them back, but I need to dig in 
> > some more
> to
> > understand what went wrong.
> >
> >
> Log?
>
> HBASE-18152?
>
> Thanks Josh,
> S
>
>
>
>
> >
> > On 4/10/18 4:47 PM, Stack wrote:
> >
> >> The first release candidate for Apache HBase 2.0.0 is available for 
> >> downloading and testing.
> >>
> >> Artifacts are available here:
> >>
> >>   https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.0RC0/
> >>
> >> Maven artifacts are available in the staging repository at:
> >>
> >>   https://repository.apache.org/content/repositories/
> orgapachehbase-1209
> >>
> >> All artifacts are signed with my signing key 8ACC93D2, which is 
> >> also in the project KEYS file at
> >>
> >>   http://www.apache.org/dist/hbase/KEYS
> >>
> >> These artifacts were tagged 2.0.0RC0 at hash 
> >> 011dd2dae33456b3a2bcc2513e9fdd29de23be46
> >>
> >> Please review 'Upgrading from 1.x to 2.x' in the bundled HBase 
> >> 2.0.0 Reference Guide before installing or upgrading for a list of 
> >> incompatibilities, major changes, and notable new features. Be 
> >> aware
> that
> >> according to our adopted Semantic Versioning guidelines[1], we've 
> >> allow ourselves to make breaking changes in this major version 
> >> release. For example, Coprocessors will need to be recast to fit 
> >> more constrained CP APIs and a rolling upgrade of an hbase-1.x 
> >> install to hbase-2.x without downtime is (currently) not possible. 
> >> That said, a bunch of effort has been expended mitigating 
> >> differences; a hbase-1.x client can perform DML against an hbase-2 
> >> cluster.
> >>
> >> For the full list of ~6k issues addressed, see [2]. There are also 
> >> CHANGES.md and RELEASENOTES.md in the root directory of the source 
> >> tarball.
> >>
> >> Please take a few minutes to verify the release and vote on 
> >> releasing
> it:
> >>
> >> [ ] +1 Release this package as Apache HBase 2.0.0 [ ] +0 no opinion 
> >> [ ] -1 Do not release this package because...
> >>
> >> This VOTE will run for one week and close Tuesday, April 17, 2018 @
> 13:00
> >> PST.
> >>
> >> Thanks to the myriad who have helped out with this release, Your 
> >> 2.0.0 Release Manager
> >>
> >> 1. http://hbase.apache.org/2.0/book.html#hbase.versioning.post10
> >> 2.  https://s.apache.org/zwS9
> >>
> >>
>

[jira] [Resolved] (HBASE-16499) slow replication for small HBase clusters

[Ashish Singhi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi resolved HBASE-16499.
---
Resolution: Fixed

> slow replication for small HBase clusters
> -
>
> Key: HBASE-16499
> URL: https://issues.apache.org/jira/browse/HBASE-16499
> Project: HBase
>  Issue Type: Bug
>  Components: Replication
>Reporter: Vikas Vishwakarma
>    Assignee: Ashish Singhi
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-16499-addendum.patch, HBASE-16499.patch, 
> HBASE-16499.patch
>
>
> For small clusters 10-20 nodes we recently observed that replication is 
> progressing very slowly when we do bulk writes and there is lot of lag 
> accumulation on AgeOfLastShipped / SizeOfLogQueue. From the logs we observed 
> that the number of threads used for shipping wal edits in parallel comes from 
> the following equation in HBaseInterClusterReplicationEndpoint
> int n = Math.min(Math.min(this.maxThreads, entries.size()/100+1),
>   replicationSinkMgr.getSinks().size());
> ... 
>   for (int i=0; i<n; i++) {
> entryLists.add(new ArrayList(entries.size()/n+1)); <-- 
> batch size
>   }
> ...
> for (int i=0; i<entryLists.size(); i++) {
>  .
> // RuntimeExceptions encountered here bubble up and are handled 
> in ReplicationSource
> pool.submit(createReplicator(entryLists.get(i), i));  <-- 
> concurrency 
> futures++;
>   }
> }
> maxThreads is fixed & configurable and since we are taking min of the three 
> values n gets decided based replicationSinkMgr.getSinks().size() when we have 
> enough edits to replicate
> replicationSinkMgr.getSinks().size() is decided based on 
> int numSinks = (int) Math.ceil(slaveAddresses.size() * ratio);
> where ratio is this.ratio = conf.getFloat("replication.source.ratio", 
> DEFAULT_REPLICATION_SOURCE_RATIO);
> Currently DEFAULT_REPLICATION_SOURCE_RATIO is set to 10% so for small 
> clusters of size 10-20 RegionServers  the value we get for numSinks and hence 
> n is very small like 1 or 2. This substantially reduces the pool concurrency 
> used for shipping wal edits in parallel effectively slowing down replication 
> for small clusters and causing lot of lag accumulation in AgeOfLastShipped. 
> Sometimes it takes tens of hours to clear off the entire replication queue 
> even after the client has finished writing on the source side. 
> We are running tests by varying replication.source.ratio and have seen 
> multi-fold improvement in total replication time (will update the results 
> here). I wanted to propose here that we should increase the default value for 
> replication.source.ratio also so that we have sufficient concurrency even for 
> small clusters. We figured it out after lot of iterations and debugging so 
> probably slightly higher default will save the trouble. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HBASE-20146) Regions are stuck to get opened when WAL is disabled

[Ashish Singhi (JIRA)]

Ashish Singhi created HBASE-20146:
-

 Summary: Regions are stuck to get opened when WAL is disabled
 Key: HBASE-20146
 URL: https://issues.apache.org/jira/browse/HBASE-20146
 Project: HBase
  Issue Type: Bug
  Components: wal
Affects Versions: 1.3.1
Reporter: Ashish Singhi


On a running cluster we had set {{hbase.regionserver.hlog.enabled}} to false, 
to disable the WAL for complete cluster, after restarting HBase service, 
regions are not getting opened leading to HMaster abort as Namespace table 
regions are not getting assigned. 

jstack for region open:
{noformat}
"RS_OPEN_PRIORITY_REGION-BLR106595:16045-1" #159 prio=5 os_prio=0 
tid=0x7fdfa4341000 nid=0x419d waiting on condition [0x7fdfa0467000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x87554448> (a 
java.util.concurrent.CountDownLatch$Sync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
at org.apache.hadoop.hbase.wal.WALKey.getWriteEntry(WALKey.java:98)
at 
org.apache.hadoop.hbase.regionserver.wal.WALUtil.writeMarker(WALUtil.java:131)
at 
org.apache.hadoop.hbase.regionserver.wal.WALUtil.writeRegionEventMarker(WALUtil.java:88)
at 
org.apache.hadoop.hbase.regionserver.HRegion.writeRegionOpenMarker(HRegion.java:1026)
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6849)
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6803)
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6774)
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6730)
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6681)
at 
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:363)
at 
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:129)
at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:129)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{noformat}
This used to work with HBase 1.0.2 version.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

RE: [VOTE] The second HBase 1.4.2 release candidate (RC1) is available

[ashish singhi]

+1

Downloaded the binary and checked the followings,

- Checked LICENSE and NOTICE files, looks ok
- Apache RAT check, looks good

Verified the following in a non-secure setup,

- Exercised basic shell commands
- Ran LTT with 1M row, checked read and write operations
- Poked around webUI
- Started Rest server and executed some commands
- Checked logs

Regards,
Ashish

-Original Message-
From: Andrew Purtell [mailto:apurt...@apache.org] 
Sent: Friday, February 23, 2018 5:26 AM
To: dev 
Subject: [VOTE] The second HBase 1.4.2 release candidate (RC1) is available

The second HBase 1.4.2 release candidate (RC1) is available for download at 
https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.2RC1/ and Maven 
artifacts are available in the temporary repository 
https://repository.apache.org/content/repositories/orgapachehbase-1196/ .

The git tag corresponding to the candidate is '1.4.2RC1' (b4ec89059c). Note 
this is different from branch-1.4 by one commit because we had an intervening 
commit between tag and push (08b9939974, HBASE-20016) while I was running 
tests. The only difference is the CHANGES.txt update for
1.4.2RC1 is placed ahead of the commit for HBASE-20016, but does not mention 
it, which is fine, because 1.4.2RC1 does not contain HBASE-20016.

A detailed source and binary compatibility report for this release is available 
for your review at 
https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.2RC1/compat-check-report.html
.

A list of the 22 issues resolved in this release can be found at 
https://s.apache.org/aGcb .

Please try out the candidate and vote +1/0/-1.

This vote will be open for at least 72 hours. Unless objection I will try to 
close it Thursday February 28, 2018 if we have sufficient votes.

Prior to making this announcement I made the following preflight checks:

   - RAT check passes (7u80)
   - Unit test suite passes (8u131)
   - LTT load 1M rows with 100% verification and 20% updates (8u131)
   - PE sequentialWrite, sequentialRead, randomWrite, randomRead,
   scanRange100 (8u131)
   - ITBLL Loop 1 100M rows (8u131)



--
Best regards,
Andrew

[jira] [Created] (HBASE-20004) Client is not able to execute REST queries through browser in a secure cluster

[Ashish Singhi (JIRA)]

Ashish Singhi created HBASE-20004:
-

 Summary: Client is not able to execute REST queries through 
browser in a secure cluster
 Key: HBASE-20004
 URL: https://issues.apache.org/jira/browse/HBASE-20004
 Project: HBase
  Issue Type: Bug
  Components: REST
Affects Versions: 1.3.1
 Environment: Firefox browser is not able to negotiate REST queries 
with server in secure mode.
Reporter: Ashish Singhi
Assignee: Ashish Singhi






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

RE: Considering branching for 1.5 and other branch-1 release planning

[ashish singhi]

We at Huawei have been testing this for more than 8 months now, did not find 
any critical issues thus far and launched a service on Huawei public cloud also 
which is based HBase 1.3.1 version.
With that, I am +1 on moving the stable pointer.

Regards,
Ashish

-Original Message-
From: Zach York [mailto:zyork.contribut...@gmail.com] 
Sent: Tuesday, February 06, 2018 3:28 AM
To: dev@hbase.apache.org
Subject: Re: Considering branching for 1.5 and other branch-1 release planning

> If someone else is using 1.3 your feedback would be very
valuable.

EMR has shipped the 1.3 line since EMR 5.4.0 (March 08, 2017). We have not ran 
into any unresolved critical issues and it has been fairly stable overall.

I would be a +1 on moving the stable pointer.

Thanks,
Zach

On Mon, Feb 5, 2018 at 1:30 PM, Andrew Purtell 
wrote:

> Thanks so much for the feedback Francis.
>
> I think we are just about there to move the stable pointer.
>
>
> On Feb 5, 2018, at 9:48 AM, Francis Liu  wrote:
>
> >> If someone else is using 1.3 your feedback would be very
> > valuable.
> >
> > We are running 1.3 in production, full rollout ongoing. Ran into 
> > some issues but it's generally been stable. We'll prolly gonna be on 
> > 1.3 for a while.
> >
> > Cheers,
> > Francis
> >
> >
> >> On Sun, Feb 4, 2018 at 10:59 AM Andrew Purtell 
> >> 
> wrote:
> >>
> >> Hi Ted,
> >>
> >> If Hadoop 3 support is in place for an (eventual) 1.5.0 release, I 
> >> think that would be great.
> >>
> >>
> >>> On Sun, Feb 4, 2018 at 10:55 AM, Ted Yu  wrote:
> >>>
> >>> Andrew:
> >>> Do you think making 1.5 release support hadoop 3 is among the goals ?
> >>>
> >>> Cheers
> >>>
> >>> On Fri, Feb 2, 2018 at 3:28 PM, Andrew Purtell 
> >>> 
> >>> wrote:
> >>>
>  The backport of RSGroups to branch-1 triggered the opening of the 
>  1.4
> >>> code
>  line as branch-1.4 and releases 1.4.0 and 1.4.1.
> 
>  After the commit of HBASE-19858 (Backport HBASE-14061 (Support
> CF-level
>  Storage Policy) to branch-1), storage policy aware file placement
> might
> >>> be
>  useful enough to trigger a new minor release from branch-1. This 
>  would
> >> be
>  branch-1.5, and at least release 1.5.0. I am not sure about this yet.
> >> It
>  needs testing. I'd like to mock up a couple of use cases and 
>  determine
> >> if
>  what we have is sufficient on its own or more changes will be needed.
> I
>  want to get the idea of a 1.5 on your radar. though.
> 
>  Also, I would like to make one more release of branch-1.3 before 
>  we
> >>> retire
>  it. Mikhail passed the reins. We might have a volunteer to RM 1.3.2.
> If
>  not, I will do it. I'm expecting 1.4 will supersede 1.3 but this 
>  will
> >> be
>  decided organically depending on uptake.
> 
>  --
>  Best regards,
>  Andrew
> 
>  Words like orphans lost among the crosstalk, meaning torn from 
>  truth's decrepit hands
>    - A23, Crosstalk
> 
> >>>
> >>
> >>
> >>
> >> --
> >> Best regards,
> >> Andrew
> >>
> >> Words like orphans lost among the crosstalk, meaning torn from 
> >> truth's decrepit hands
> >>   - A23, Crosstalk
> >>
>

Re: [VOTE] The first HBase 1.4.1 release candidate (RC0) is available

[Ashish Singhi]

+1 (non-binding)

Downloaded the binary and checked the following,

- Checked LICENSE and NOTICE files, looks ok
- Apache RAT check, looks good

Verified the following in a non-secure setup,

- Exercised basic shell commands
- Ran LTT with 1M row, checked read and write operations
- Poked around webUI
- Started Rest server and executed some commands
- Checked logs
- Browsed book

Regards,
Ashish

On Thu, Jan 25, 2018 at 8:15 AM, Andrew Purtell  wrote:

> The first HBase 1.4.1 release candidate (RC0) is available for download at
> https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.1RC0/ and Maven
> artifacts are available in the temporary repository
> https://repository.apache.org/content/repositories/orgapachehbase-1194 .
>
> The git tag corresponding to the candidate is '1.4.1RC0' (4b25debc83).
>
> A detailed source and binary compatibility report for this release is
> available for your review at
> https://dist.apache.org/repos/dist/dev/hbase/hbase-1.4.1RC0/
> compat-check-report.html
> .
>
> A list of the 38 issues resolved in this release can be found at
> https://s.apache.org/tx1w .
>
> Please try out the candidate and vote +1/0/-1.
>
> This vote will be open for at least 72 hours. Unless objection I will try
> to close it Friday February 2, 2018 if we have sufficient votes.
>
> Prior to making this announcement I made the following preflight checks:
>
>- RAT check passes (7u80)
>- Unit test suite passes 10 of 10 iterations (8u131)
>- LTT load 1M rows with 100% verification and 20% updates (8u131)
>- PE sequentialWrite, sequentialRead, randomWrite, randomRead,
>scanRange100 (8u131)
>
>
> --
> Best regards,
> Andrew
>
> Words like orphans lost among the crosstalk, meaning torn from truth's
> decrepit hands
>- A23, Crosstalk
>

[jira] [Created] (HBASE-19905) ReplicationSyncUp tool will not exit if a peer replication is disabled

[Ashish Singhi (JIRA)]

Ashish Singhi created HBASE-19905:
-

 Summary: ReplicationSyncUp tool will not exit if a peer 
replication is disabled
 Key: HBASE-19905
 URL: https://issues.apache.org/jira/browse/HBASE-19905
 Project: HBase
  Issue Type: Bug
  Components: Replication
Affects Versions: 1.3.1
Reporter: Ashish Singhi
Assignee: Ashish Singhi






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HBASE-19796) ReplicationSynUp tool is not replicating the data if the WAL is moved to splititing directory

[Ashish Singhi (JIRA)]

Ashish Singhi created HBASE-19796:
-

 Summary: ReplicationSynUp tool is not replicating the data if the 
WAL is moved to splititing directory
 Key: HBASE-19796
 URL: https://issues.apache.org/jira/browse/HBASE-19796
 Project: HBase
  Issue Type: Bug
  Components: Replication
Affects Versions: 1.3.1
Reporter: Ashish Singhi
Assignee: Ashish Singhi


In our test cluster we found that ReplictionSyncUp tool is not replicating the 
data from the source cluster RS WAL which is moved to WAL splitting directory 
to the peer cluster.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [VOTE] The fourth hbase-2.0.0-beta-1 Release Candidate is available for download

[Ashish Singhi]

I ran RAT check on head of the branch-2 code.
I did first mvn clean install -DskipTests and then ran mvn apache-rat:check

Anything I am missing or I need to do ?

Regards,
Ashish

On Sat, Jan 13, 2018 at 10:08 PM, Ted Yu <yuzhih...@gmail.com> wrote:

> hbase-spark and hbase-prefix-tree modules should no longer be part of the
> hbase 2.0 build process.
>
> Can you check the files to see if they were left over from previous build ?
>
> Cheers
>
> On Sat, Jan 13, 2018 at 8:29 AM, Ashish Singhi <ashishsin...@apache.org>
> wrote:
>
> > RAT check fails for me, below error I see in the log file,
> >
> > 4 Unknown Licenses
> >
> > *
> >
> > Files with unapproved licenses:
> >
> >   hbase-spark/target/test-classes.-284428056.timestamp
> >   hbase-spark/target/classes.786562742.timestamp
> >   hbase-spark/target/maven-archiver/pom.properties
> >   hbase-prefix-tree/target/maven-archiver/pom.properties
> >
> > *
> >
> > CHANGES.txt file doesn't look correct.
> >
> >
> >
> > Downloaded the binary and checked the following,
> >
> > - Checked LICENSE and NOTICE files, looks ok
> >
> > Verified the following in a non-kerberos setup,
> > - On a one node setup did upgrade from 1.3.2-SNAPSHOT version to
> > 2.0.0-beta-1
> > - Performed get and scan operation on data which were loaded in
> > 1.3.2-SNAPSHOT version
> > - Exercised basic shell commands
> > - Ran LTT with 1M row, checked read and write operations
> > - Poked around webUI
> > - Started Rest server and executed few commands
> > - Checked logs
> > - Browsed book
> >
> > Regards,
> > Ashish
> >
> > On Sat, Jan 13, 2018 at 10:18 AM, Stack <st...@duboce.net> wrote:
> >
> > > The fourth release candidate for HBase 2.0.0-beta-1 is up at:
> > >
> > >   https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.0-beta-1-RC3/
> > >
> > > Maven artifacts are available from a staging directory here:
> > >
> > >   https://repository.apache.org/content/repositories/
> orgapachehbase-1193
> > >
> > > All was signed with my key at 8ACC93D2 [1].
> > >
> > > I tagged the RC as 2.0.0-beta-1-RC1.7 at hash 026f535a7747b89003252ded9
> > > 585e827686aa79f
> > >
> > > This RC has some nice bug fixes over RC0-2 including fixing MOST of the
> > > failing and flakey tests (We are still working through them).
> > >
> > > hbase-2.0.0-beta-1 is our first beta release. It includes all that was
> in
> > > previous alphas (new assignment manager, offheap read/write path,
> > in-memory
> > > compactions, etc.). The APIs and feature-set are sealed.
> > >
> > > hbase-2.0.0-beta-1 is a not-for-production preview of hbase-2.0.0. It
> is
> > > meant for devs and downstreamers to test drive and flag us if we messed
> > up
> > > on anything ahead of our rolling GAs. We are particular interested in
> > > hearing from Coprocessor developers.
> > >
> > > The list of features addressed in 2.0.0 so far can be found here [3].
> > There
> > > are thousands. The list of ~2k+ fixes in 2.0.0 exclusively can be found
> > > here [4] (My JIRA JQL foo is a bit dodgy -- forgive me if mistakes).
> > >
> > > I've updated our overview doc. on the state of 2.0.0 [6]. We'll do one
> > more
> > > beta before we put up our first 2.0.0 Release Candidate by the end of
> > > January, 2.0.0-beta-2. Its focus will be making it so users can do a
> > > rolling upgrade on to hbase-2.x from hbase-1.x (and any bug fixes found
> > > running beta-1). Here is the list of what we have targeted so far for
> > > beta-2 [5]. Check it out.
> > >
> > > One known issue is that the User API has not been properly filtered so
> it
> > > shows more than just InterfaceAudience Public content (HBASE-19663, to
> be
> > > fixed by beta-2).
> > >
> > > Please take this beta for a spin. Please vote on whether it ok to put
> out
> > > this RC as our first beta (Note CHANGES has not yet been updated). Let
> > the
> > > VOTE be open for 72 hours (Lets say Tuesday morning!)
> > >
> > > Thanks,
> > > Your 2.0.0 Release Manager
> > >
> > > 1. http://pgp.mit.edu/pks/lookup?op=get=0x9816C7FC8ACC93D2
> > > 3. https://goo.gl/scYjJr
> > > 4. https://goo.gl/dFFT8b
> > > 5. https://issues.apache.org/jira/projects/HBASE/versions/12340862
> > > 6. https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4
> > > z9iEu_ktczrlKHK8N4SZzs/
> > >
> >
>

Re: [VOTE] The fourth hbase-2.0.0-beta-1 Release Candidate is available for download

[Ashish Singhi]

RAT check fails for me, below error I see in the log file,

4 Unknown Licenses

*

Files with unapproved licenses:

  hbase-spark/target/test-classes.-284428056.timestamp
  hbase-spark/target/classes.786562742.timestamp
  hbase-spark/target/maven-archiver/pom.properties
  hbase-prefix-tree/target/maven-archiver/pom.properties

*

CHANGES.txt file doesn't look correct.



Downloaded the binary and checked the following,

- Checked LICENSE and NOTICE files, looks ok

Verified the following in a non-kerberos setup,
- On a one node setup did upgrade from 1.3.2-SNAPSHOT version to
2.0.0-beta-1
- Performed get and scan operation on data which were loaded in
1.3.2-SNAPSHOT version
- Exercised basic shell commands
- Ran LTT with 1M row, checked read and write operations
- Poked around webUI
- Started Rest server and executed few commands
- Checked logs
- Browsed book

Regards,
Ashish

On Sat, Jan 13, 2018 at 10:18 AM, Stack  wrote:

> The fourth release candidate for HBase 2.0.0-beta-1 is up at:
>
>   https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.0-beta-1-RC3/
>
> Maven artifacts are available from a staging directory here:
>
>   https://repository.apache.org/content/repositories/orgapachehbase-1193
>
> All was signed with my key at 8ACC93D2 [1].
>
> I tagged the RC as 2.0.0-beta-1-RC1.7 at hash 026f535a7747b89003252ded9
> 585e827686aa79f
>
> This RC has some nice bug fixes over RC0-2 including fixing MOST of the
> failing and flakey tests (We are still working through them).
>
> hbase-2.0.0-beta-1 is our first beta release. It includes all that was in
> previous alphas (new assignment manager, offheap read/write path, in-memory
> compactions, etc.). The APIs and feature-set are sealed.
>
> hbase-2.0.0-beta-1 is a not-for-production preview of hbase-2.0.0. It is
> meant for devs and downstreamers to test drive and flag us if we messed up
> on anything ahead of our rolling GAs. We are particular interested in
> hearing from Coprocessor developers.
>
> The list of features addressed in 2.0.0 so far can be found here [3]. There
> are thousands. The list of ~2k+ fixes in 2.0.0 exclusively can be found
> here [4] (My JIRA JQL foo is a bit dodgy -- forgive me if mistakes).
>
> I've updated our overview doc. on the state of 2.0.0 [6]. We'll do one more
> beta before we put up our first 2.0.0 Release Candidate by the end of
> January, 2.0.0-beta-2. Its focus will be making it so users can do a
> rolling upgrade on to hbase-2.x from hbase-1.x (and any bug fixes found
> running beta-1). Here is the list of what we have targeted so far for
> beta-2 [5]. Check it out.
>
> One known issue is that the User API has not been properly filtered so it
> shows more than just InterfaceAudience Public content (HBASE-19663, to be
> fixed by beta-2).
>
> Please take this beta for a spin. Please vote on whether it ok to put out
> this RC as our first beta (Note CHANGES has not yet been updated). Let the
> VOTE be open for 72 hours (Lets say Tuesday morning!)
>
> Thanks,
> Your 2.0.0 Release Manager
>
> 1. http://pgp.mit.edu/pks/lookup?op=get=0x9816C7FC8ACC93D2
> 3. https://goo.gl/scYjJr
> 4. https://goo.gl/dFFT8b
> 5. https://issues.apache.org/jira/projects/HBASE/versions/12340862
> 6. https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4
> z9iEu_ktczrlKHK8N4SZzs/
>

Re: HBase 1.3.2-SNAPSHOT (20180113T013910Z)

[Ashish Singhi]

+1 (non-binding)

Downloaded the binary and checked the following,

CHANGES.txt file doesn't contain the changes done in this release. I have
no idea whether this is ok or not.

- Checked LICENSE and NOTICE files, looks ok
- Apache RAT check, looks good

Verified the following in a non-kerberos setup,

- Exercised basic shell commands
- Ran LTT with 1M row, checked read and write operations
- Poked around webUI
- Started Rest server and executed few commands
- Checked logs
- Browsed book

Regards,
Ashish


On Sat, Jan 13, 2018 at 7:44 AM, Andrew Purtell  wrote:

> A snapshot of HBase 1.3.2 is available for testing in Apache's Maven or at
> ​​
> https://dist.apache.org/repos/dist/dev/hbase/hbase-1.3.2-SNAPSHOT/ . This
> snapshot was generated from the git SHA b6f4f511a6 .
>
> ​The compatibility report with respect to the previous release can be found
> at ​
> https://dist.apache.org/repos/dist/dev/hbase/hbase-1.3.2-
> SNAPSHOT/compat-check-report.html
> .
>
> This is a snapshot, not an official release. These artifacts are provided
> for development and testing convenience. Please use with caution. Do not
> deploy to production.
>
> We should be releasing 1.3.2 within the next month or so. On behalf of the
> HBase project I apologize for the length of time it has been since the
> 1.3.1 release, and for the number of changes ​included in the upcoming
> patch release as a result.
>
> The changes in this snapshot since the last 1.3 release or snapshot are:
>
>- HBASE-8758 Error in RegionCoprocessorHost class preScanner method
>documentation
>- HBASE-9393 Hbase does not closing a closed socket resulting in many
>CLOSE_WAIT
>- HBASE-9899 for idempotent operation dups, return the result instead of
>throwing conflict exception (addendum for branch-1)
>- HBASE-14220 nightly check that we can build a source tarball.
>- HBASE-15497 Incorrect javadoc for atomicity guarantee of Increment and
>Append
>- HBASE-15548 SyncTable: sourceHashDir is supposed to be optional but
>won't work without (Dave Latham)
>- HBASE-15607 Deprecating SnapShotInfo (Ram)
>- HBASE-15691 ConcurrentModificationException in BucketAllocator
>- HBASE-15947 Classes used only for tests included in main code base
>- HBASE-16011 TableSnapshotScanner and TableSnapshotInputFormat can
>produce duplicate rows if split table.
>- HBASE-16090 ResultScanner is not closed in
>SyncTable#finishRemainingHashRanges()
>- HBASE-16116 Remove redundant pattern *.iml
>- HBASE-16351 Improve error reporting for license failures
>- HBASE-16459 Remove unused hbase shell --format option
>- HBASE-16615 Fix flaky TestScannerHeartbeatMessages (Duo Zhang)
>- HBASE-16939 ExportSnapshot: set owner and permission on right
> directory
>- HBASE-17131 Avoid livelock caused by HRegion#processRowsWithLocks
>- HBASE-17352 Fix hbase-assembly build with bash 4
>- HBASE-17441 Fix invalid quoting around hadoop-3 build in yetus
>personality
>- HBASE-17514 emit a warning if thrift1 proxy user is configured but
>hbase.regionserver.thrift.http is not
>- HBASE-17534 Avoid re-wrapping IOExceptions as IOExceptions
>- HBASE-17617 Backport HBASE-16731 (Inconsistent results from the
>Get/Scan if we use the empty FilterList) to branch-1
>- HBASE-17648 HBase Table-level synchronization fails between two
>secured(kerberized) cluster
>- HBASE-17658 Fix bookkeeping error with max regions for a table
>- HBASE-17803 PE always re-creates table when we specify the split
> policy
>- HBASE-17817 add table name to output (if available) when removing
>coprocessors
>- HBASE-17862 Fix a condition that always returns true
>- HBASE-17877 Improve HBase's byte[] comparator.
>- HBASE-17887 Row-level consistency is broken for read
>- HBASE-17902 Backport HBASE-16367 "Race between master and region
>server initialization may lead to premature server abort" to 1.3
>- HBASE-17916 Error message not clear when the permission of staging dir
>is not as expected
>- HBASE-17925 mvn assembly:single fails against hadoop3-alpha2
>- HBASE-17934 Backport HBASE-17779 "disable_table_replication returns
>misleading message and does not turn off replication"
>- HBASE-17934 Backport HBASE-17779 "disable_table_replication returns
>misleading message and does not turn off replication" (Janos Gub)
>- HBASE-17937 Memstore size becomes negative in case of expensive
>postPut/Delete Coprocessor call
>- HBASE-17944 Removed unused JDK version parsing from ClassSize.
>- HBASE-17954 Switched findbugs implementation to spotbugs
>- HBASE-17968 Fix NOTICE.txt for src-release
>- HBASE-17985 Inline apt-get update calls in Yetus Dockerfile
>- HBASE-18000 Make sure we always return the scanner id with
> ScanResponse
>- HBASE-18014 A case of Region remain unassigned when table enabled
>(Allan Yang)
>- 

RE: [VOTE] Merge branch HBASE-19397 back to master and branch-2.

[ashish singhi]

+1 to merge on master and 2.1.
Great work.

Thanks,
Ashish

-Original Message-
From: 张铎(Duo Zhang) [mailto:palomino...@gmail.com] 
Sent: Tuesday, January 09, 2018 6:53 AM
To: dev@hbase.apache.org
Subject: Re: [VOTE] Merge branch HBASE-19397 back to master and branch-2.

Anyway, if no objections on merging this into master, let's do it? So that we 
can start working on the follow-on features, such as table based replication 
storage, and synchronous replication, etc.

Thanks.

2018-01-09 7:19 GMT+08:00 Stack :

> On Mon, Jan 8, 2018 at 2:50 PM, 张铎(Duo Zhang) 
> wrote:
>
> > This 'new' feature only changes DDL part, not the core part of
> replication,
> > i.e, how to read wal entries and how to replicate it to the remote
> cluster,
> > etc. And also there is no pb message/storage layout change, you can 
> > think of this as a big refactoring. Theoretically we even do not 
> > need to add
> new
> > UTs for this feature, i.e, no extra stability works. The only 
> > visible change to users is that it may require more time on 
> > modifying peers in shell. So in general I think it is less hurt to 
> > include it in the coming release?
> >
> > And why I think it SHOULD be included in our 2.0 release is that, 
> > the synchronous guarantee is really a good thing for our replication 
> > related UTs. The correctness of the current Test***Replication 
> > usually depends
> on a
> > flakey condition - we will not do a log rolling between the 
> > modification
> on
> > zk and the actual loading of the modification on RS. And we have 
> > also
> done
> > a hard work to cleanup the lockings in ReplicationSourceManager and 
> > add a fat comment to say why it should be synchronized in this way. 
> > In general, the new code is much easier to read, test and debug, and 
> > also reduce the possibility of flakeyness, which could help us a lot 
> > when we start to stabilize our build.
> >
> > Thanks.
> >
> >
> You see it as a big bug fix Duo?
>
Kind of. Just like the AM v1, we can do lots of fix to make it more stable, but 
we know that we can not fix all the problems since we store state in several 
places and it is a 'mission impossible' to make all the states stay in sync 
under every situation... So we introduce AM v2.
For the replication peer tracking, it is the same problem. It is hard to do 
fencing with zk watcher since it is asynchronous, so the UTs are always kind of 
flakey in theoretical. And we depend on replication heavily at Xiaomi, it is 
always a pain for us.

>
> I'm late to review. Will have a look after beta-1 goes out. This stuff 
> looks great from outside, especially distributed procedure framework 
> which we need all over the place.
>
> In general I have no problem w/ this in master and an hbase 2.1 (2.1 
> could be soon after 2.0). Its late for big stuff in 2.0 but let me 
> take a looksee sir.
>
Thanks sir. All the concerns here about whether we should merge this into
2.0 are reasonable, I know. Although I really want this in 2.0 because I 
believe it will help a lot for stabilizing,  I'm OK with merge it to 2.1 only 
if you guys all think so.

>
> St.Ack
>
>
>
>
>
> > 2018-01-09 4:53 GMT+08:00 Apekshit Sharma :
> >
> > > Same questions as Josh's.
> > > 1) We have RCs for beta1 now, which means only commits that can go 
> > > in
> are
> > > bug fixes only. This change - although important, needed from long 
> > > time
> > and
> > > well done (testing, summary, etc) - seems rather very large to get 
> > > into
> > 2.0
> > > now. Needs good justification why it has to be 2.1 instead of 2.0.
> > >
> > > -- Appy
> > >
> > >
> > > On Mon, Jan 8, 2018 at 8:34 AM, Josh Elser  wrote:
> > >
> > > > -0 From a general project planning point-of-view (not based on 
> > > > the technical merit of the code) I am uncomfortable about 
> > > > pulling in a
> > brand
> > > > new feature after we've already made one beta RC.
> > > >
> > > > Duo -- can you expand on why this feature is so important that 
> > > > we
> > should
> > > > break our release plan? Are there problems that would make 
> > > > including
> > this
> > > > in a 2.1/3.0 unnecessarily difficult? Any kind of color you can
> provide
> > > on
> > > > "why does this need to go into 2.0?" would be helpful.
> > > >
> > > >
> > > > On 1/6/18 1:54 AM, Duo Zhang wrote:
> > > >
> > > >> https://issues.apache.org/jira/browse/HBASE-19397
> > > >>
> > > >> We aim to move the peer modification framework from zk watcher 
> > > >> to procedure
> > > >> v2 in this issue and the work is done now.
> > > >>
> > > >> Copy the release note here:
> > > >>
> > > >> Introduce 5 procedures to do peer modifications:
> > > >>
> > > >>> AddPeerProcedure
> > > >>> RemovePeerProcedure
> > > >>> UpdatePeerConfigProcedure
> > > >>> EnablePeerProcedure
> > > >>> DisablePeerProcedure
> > > >>>
> > > >>> The procedures are all executed with the following stage:
> > > >>> 1. Call pre CP hook, if 

Re: [ANNOUNCE] New HBase committer Zheng Hu

[Ashish Singhi]

Congratulations.

On Mon, Oct 23, 2017 at 11:48 AM, Duo Zhang  wrote:

> On behalf of the Apache HBase PMC, I am pleased to announce that Zheng Hu
> has accepted the PMC's invitation to become a committer on the project. We
> appreciate all of Zheng's generous contributions thus far and look forward
> to his continued involvement.
>
> Congratulations and welcome, Zheng!
>

[jira] [Created] (HBASE-18939) Backport HBASE-16538 to branch-1.3

[Ashish Singhi (JIRA)]

Ashish Singhi created HBASE-18939:
-

 Summary: Backport HBASE-16538 to branch-1.3
 Key: HBASE-18939
 URL: https://issues.apache.org/jira/browse/HBASE-18939
 Project: HBase
  Issue Type: Sub-task
Affects Versions: 1.3.1
Reporter: Ashish Singhi
 Fix For: 1.3.2






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-18938) Backport HBASE-16985 to branch-1.3

[Ashish Singhi (JIRA)]

Ashish Singhi created HBASE-18938:
-

 Summary: Backport HBASE-16985 to branch-1.3
 Key: HBASE-18938
 URL: https://issues.apache.org/jira/browse/HBASE-18938
 Project: HBase
  Issue Type: Sub-task
Reporter: Ashish Singhi






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-18937) Backport HBASE-16815 to branch-1.3

[Ashish Singhi (JIRA)]

Ashish Singhi created HBASE-18937:
-

 Summary: Backport HBASE-16815 to branch-1.3
 Key: HBASE-18937
 URL: https://issues.apache.org/jira/browse/HBASE-18937
 Project: HBase
  Issue Type: Sub-task
Reporter: Ashish Singhi






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-18936) Backport HBASE-16870 to branch-1.3

[Ashish Singhi (JIRA)]

Ashish Singhi created HBASE-18936:
-

 Summary: Backport HBASE-16870 to branch-1.3
 Key: HBASE-18936
 URL: https://issues.apache.org/jira/browse/HBASE-18936
 Project: HBase
  Issue Type: Sub-task
Affects Versions: 1.3.1
Reporter: Ashish Singhi
 Fix For: 1.3.2






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (HBASE-18935) Backport bug fixes to 1.3.x which are fixed in 1.2.x and 1.4.x

[Ashish Singhi (JIRA)]

Ashish Singhi created HBASE-18935:
-

 Summary: Backport bug fixes to 1.3.x which are fixed in 1.2.x and 
1.4.x
 Key: HBASE-18935
 URL: https://issues.apache.org/jira/browse/HBASE-18935
 Project: HBase
  Issue Type: Umbrella
Affects Versions: 1.3.1
Reporter: Ashish Singhi
 Fix For: 1.3.2


Backport the bug fixes which were fixed in 1.2.x and 1.4.x branch but missed in 
1.3.x branch. 
I have used the below query to those jiras.
*project = HBASE AND status in (Resolved, Closed) AND fixVersion in (1.4.0, 
1.4.1) AND fixVersion in (1.2.7, 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 
1.2.6) AND fixVersion not in (1.3.0, 1.3.1, 1.3.2)*



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Welcome Chia-Ping Tsai to the HBase PMC

[Ashish Singhi]

Congratulations, Chia-Ping.

On Sat, Sep 30, 2017 at 3:49 AM, Misty Stanley-Jones 
wrote:

> The HBase PMC is delighted to announce that Chia-Ping Tsai has agreed to
> join
> the HBase PMC, and help to make the project run smoothly. Chia-Ping became
> an
> HBase committer over 6 months ago, based on long-running participate in the
> HBase project, a consistent record of resolving HBase issues, and
> contributions
> to testing and performance.
>
> Thank you for stepping up to serve, Chia-Ping!
>
> As a reminder, if anyone would like to nominate another person as a
> committer or PMC member, even if you are not currently a committer or PMC
> member, you can always drop a note to priv...@hbase.apache.org to let us
> know!
>
> Thanks,
> Misty (on behalf of the HBase PMC)
>

RE: Please congratulate our new PMC Chair Misty Stanley-Jones

[ashish singhi]

Many Congratulations for the new role, Misty.

-Original Message-
From: Andrew Purtell [mailto:apurt...@apache.org] 
Sent: 22 September 2017 00:38
To: dev@hbase.apache.org; u...@hbase.apache.org
Subject: Please congratulate our new PMC Chair Misty Stanley-Jones

At today's meeting of the Board, Special Resolution B changing the HBase 
project Chair to Misty Stanley-Jones was passed unanimously.

Please join me in congratulating Misty on her new role!

​(If you need any help or advice please don't hesitate to ping me, Misty, but I 
suspect you'll do just fine and won't need it.)​


--
Best regards,
Andrew

RE: [VOTE] First hbase-2.0.0-alpha-3 Release Candidate is available

[ashish singhi]

+1(non-binding)

Built tar from source code
Verified few commands on shell and other few from Java API, all looks ok
Verified audit logs
Verified HBase replication on a single node cluster, checked metrics
Checked zk_dump and web UI, looks ok

Regards,
Ashish

-Original Message-
From: saint@gmail.com [mailto:saint@gmail.com] On Behalf Of Stack
Sent: 15 September 2017 23:33
To: HBase Dev List 
Cc: d...@phoenix.apache.org
Subject: [VOTE] First hbase-2.0.0-alpha-3 Release Candidate is available

The first release candidate for HBase 2.0.0-alpha-3 is up at:

  https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.0-alpha-3RC0/

Maven artifacts are available from a staging directory here:

  https://repository.apache.org/content/repositories/orgapachehbase-1175

All was signed with my key at y 8ACC93D2 


I tagged the RC as
2.0.0-alpha-3RC0.2 (a5c8461ca87d6324f16ffd126b765146fdd5315a)

hbase-2.0.0-alpha-3 is our third alpha release on our march toward an 
hbase-2.0.0. It includes all that was in previous alphas (new assignment 
manager, offheap read/write path, in-memory compactions, etc.), but had a focus 
on polishing our public API: old API that had been deprecated since
hbase-1.0.0 or before was purged and new API was added with sympathetic 
deprecation of the previous. Along the Admin plane, incompatible changes were 
unavoidable; you will not be able to administer a hbase2 cluster using an 
hbase1 client (see adjacent "[DISCUSS] hbase-2.0.0 compatibility expectations" 
thread for discussion and see [1] for current list of Incompatibles).

What is here will be our public API for 2.0.0 unless we get pushback from our 
gracious downstreamers. Please try it out. Shout now if you find a problem so 
we can fix it before we get to beta.

Alpha-3 does not have the final version of our Coprocessor API. Finishing the 
Coprocessor API for hbase-2.0.0 is the topic of our last planned alpha, 
2.0.0-alpha-4. The Coprocessor API changes pretty radically in hbase-2.0.0 
(though Coprocessor Endpoints will continue to work across an upgrade). See [2] 
for why and why it was unavoidable. Input now from Coprocessor API users before 
alpha-4 would be especially effective (I've cc'd our Phoenix brothers and 
sisters toward this end).

hbase-2.0.0-alpha-3RC0 is a rough cut ('alpha'), not-for-production preview of 
what hbase-2.0.0 will look like. It is meant for devs and downstreamers to test 
drive and flag us early if we messed up anything ahead of our rolling GAs.

The list of features addressed in 2.0.0 so far can be found here [3]. There are 
about 2700+. The list of ~500 fixes in 2.0.0 exclusively can be found here [3].

I've updated our overview doc. on the state of 2.0.0 [6] but JIRA 2.0.0 label 
[5] has undergone extensive weeding and presents a fairly good picture on what 
is yet to do before we 2.0.0. Check it out.

Please take it for a spin and vote on whether it ok to put out as our first 
alpha (bar is low for an 'alpha'). Let the VOTE be open for 72 hours
(Sunday)

Thanks,
St.Ack

1. Current list of Incompatibles:
https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9iEu_ktczrlKHK8N4SZzs/edit#heading=h.723jjn18p2jr
2. Why CPs are Incompatible:
https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9iEu_ktczrlKHK8N4SZzs/edit#heading=h.9k7mjbauv0wj
3. goo.gl/Gcrp4f
4. goo.gl/6dPqzG
5. https://issues.apache.org/jira/projects/HBASE/versions/12327188
6.
https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9iEu_ktczrlKHK8N4SZzs/

Re: [VOTE] First release candidate for HBase 1.1.12 (RC0) is available

[Ashish Singhi]

+1(non-binding)

Built tar from source code
Verified few commands on shell and other few from Java API, all looks ok
Added AccessController coprocessor to master, rs and region coprocessor
classes config and verified audit logs
Verified HBase replication on a single node cluster, checked metrics
Checked zk_dump and web UI, looks ok

Regards,
Ashish

On Sun, Aug 13, 2017 at 3:39 AM, Nick Dimiduk  wrote:

> I'm happy to announce the first release candidate of HBase 1.1.12
> (HBase-1.1.12RC0) is available for download at https://dist.apache.org/
> repos/dist/dev/hbase/hbase-1.1.12RC0/
>
> Maven artifacts are also available in the staging repository
> https://repository.apache.org/content/repositories/orgapachehbase-1172
>
> Artifacts are signed with my code signing subkey 0xAD9039071C3489BD,
> available in the Apache keys directory https://people.apache.org/
> keys/committer/ndimiduk.asc and in our KEYS file
> http://www-us.apache.org/dist/hbase/KEYS.
>
> There's also a signed tag for this release at
> https://git-wip-us.apache.org/
> repos/asf?p=hbase.git;a=tag;h=29a930cdcbef2a64b601a110bad72955b3b04afc
>
> The detailed source and binary compatibility report vs 1.1.9 has been
> published for your review, at https://home.apache.org/~
> ndimiduk/1.1.11_1.1.12RC0_compat_report.html
>
> HBase 1.1.12 is the twelfth patch release in the HBase 1.1 line, continuing
> on the theme of bringing a stable, reliable database to the Hadoop and
> NoSQL communities. This release includes nearly 10 bug fixes since the
> 1.1.11 release. Notable correctness fixes include HBASE-9393,
> HBASE-15691, HBASE-18390,
> HBASE-18330.
>
> The full list of fixes included in this release is available at
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12310753=12340864 and and in the CHANGES.txt file
> included in the distribution.
>
> Please try out this candidate and vote +/-1 by 23:59 Pacific time on
> Saturday, 2017-08-19 as to whether we should release these artifacts as
> HBase 1.1.12.
>
> Thanks,
> Nick
>

Re: [VOTE] First hbase-2.0.0-alpha-2 Release Candidate is available

[Ashish Singhi]

+1(non-binding)

Built tar from source code
Verified few commands on shell and other few from Java API, all looks ok
Verified audit logs
Verified HBase replication on a single node cluster, checked metrics
Checked zk_dump and web UI, looks ok

Regards,
Ashish

On Thu, Aug 17, 2017 at 2:13 AM, Stack  wrote:

> The first release candidate for HBase 2.0.0-alpha-2 is up at:
>
>   https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.0-alpha-2RC0/
>
> Maven artifacts are available from a staging directory here:
>
>   https://repository.apache.org/content/repositories/orgapachehbase-1173
>
> All was signed with my key at y 8ACC93D2
> 
>
> I tagged the RC as 2.0.0-alpha-2RC0
> (7149f999786b6fd5a3fc1f7aec1214afb738925e)
>
> hbase-2.0.0-alpha-2 includes all that was in alpha-1 (new assignment
> manager, offheap read/write path, in-memory compactions, etc.) plus a
> return to branch-1 deploy mode where master carries no regions (branch-2
> and master up to this had master carrying system tables), a purge of all
> checked-in generated content, updates to all dependencies including a move
> to rely on new hbase-thirdparty lib for relocated versions of guava, netty,
> and protobuf, etc.
>
> This is a rough cut ('alpha'), not-for-production preview of what
> hbase-2.0.0 will look like meant for devs and downstreamers to test drive
> and flag us early if we messed up anything ahead of our rolling GAs.
>
> The list of features addressed in 2.0.0 so far can be found here [4]. There
> are about 2700+. The list of ~500 fixes in 2.0.0 exclusively can be found
> here [3].
>
> I've updated our overview doc. on the state of 2.0.0 [1] but JIRA 2.0.0
> label [2] has undergone extensive weeding and presents a fairly good
> picture on what is yet to do before we 2.0.0. Check it out.
>
> Please take it for a spin and vote on whether it ok to put out as our first
> alpha (bar is low for an 'alpha'). Let the VOTE be open for 72 hours
> (Saturday)
>
> Thanks,
> St.Ack
>
> 1.
> https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9iEu_
> ktczrlKHK8N4SZzs/edit#
> 2. https://issues.apache.org/jira/projects/HBASE/versions/12327188
> 3.
> https://issues.apache.org/jira/browse/HBASE-18611?jql=
> project%20%3D%20HBASE%20%20and%20(fixVersion%20%3D%202.
> 0.0%20%7C%7C%20fixVersion%20%3D%202.0.0.alpha1%20%7C%7C%
> 20fixVersion%20%3D%202.0.0-alpha-2)%20and%20fixVersion%
> 20not%20in%20(1.0.0%2C%201.0.1%2C%201.0.2%2C%201.0.3%2C%
> 201.0.4%2C%201.0.5%2C%201.0.6%2C%201.1.0%2C%201.1.1%2C%201.
> 1.2%2C%201.1.3%2C%201.1.4%2C%201.1.5%2C%201.1.6%2C%201.1.7%
> 2C%201.1.8%2C%201.1.9%2C%201.1.10%2C%201.2.0%2C%201.2.1%2C%
> 201.2.2%2C%201.2.3%2C%201.2.4%2C%201.2.5%2C%201.2.6%2C%201.
> 3.0%2C%201.3.1%2C%201.4.0)%20and%20%20(status%20%3D%
> 20Open%20or%20status%20%3D%20%22Patch%20Available%22)
> 4.
> https://issues.apache.org/jira/browse/HBASE-18599?jql=
> project%20%3D%20HBASE%20%20and%20(%20fixVersion%20%3D%
> 202.0.0%20%7C%7C%20fixVersion%20%3D%202.0.0-alpha-1%20%7C%
> 7C%20fixVersion%20%3D%202.0.0-alpha-2)%20and%20(status%20%3D%20Resolved)
>

RE: Notes from dev meetup in Shenzhen, August 5th, 2017

[ashish singhi]

Great write up, Stack. Covering everything what we all discussed.
It was very nice meeting you all and hope we can continue this HBaseCon Asia.

Regards,
Ashish

From: saint@gmail.com [mailto:saint@gmail.com] On Behalf Of Stack
Sent: 08 August 2017 00:07
To: HBase Dev List 
Subject: Notes from dev meetup in Shenzhen, August 5th, 2017

At fancy Huawei headquarters, 10:00-12:00AM or so (with nice coffee and fancy 
little cake squares provided about half way through the session).

For list of attendees, see picture at end of this email.

Discussion was mostly in Chinese with about 25% in English plus some gracious 
sideline translation so the below is patchy. Hopefully you get the gist.

For client-side scanner going against hfiles directly; is there a means of 
being able to pass the permissions from hbase to hdfs?

Issues w/ the hbase 99th percentile were brought up. "DynamoDB can do 10ms". 
How to do better?

SSD is not enough.

GC messes us up.

Will the Distributed Log Replay come back to help improve MTTR? We could redo 
on new ProcedureV2 basis. ZK timeout is the biggest issue. Do as we used to and 
just rely on the regionserver heartbeating...

Read replica helps w/ MTTR.

Ratis incubator project to do a quorum based hbase?

Digression on licensing issues around fb wangle and folly.

Redo of hbase but quorum based would be another project altogether.

Decided to go around the table to talk about concerns and what people are 
working on.

Jieshan wondered what could be done to improve OLAP over hbase.

Client side scanner was brought up again as means of skipping RS overhead and 
doing better OLAP.

Have HBase compact to parquet files. Query parquet and hbase.

At Huawei, they are using 1.0 hbase. Most problems are assignment. They have 
.5M regions. RIT is a killer. Double assignment issues. And RIT. They run their 
own services. Suggested they upgrade to get fixes at least. Then 2.0.

Will HBase federate like HDFS? Can Master handle load at large scale? It needs 
to do federation too?

Anyone using Bulk loaded replication? (Yes, it just works so no one talks about 
it...)

Request that fixes be backported to all active branches, not just most current.

Andrew was good at backporting... not all RMs are.

Too many branches. What should we do?

Proliferation of branches makes for too much work.

Need to cleanup bugs in 1.3. Make it stable release now.

Lets do more active EOL'ing of branches. 1.1?.

Hubert asked if we can have clusters where RS are differently capable? i.e. 
several generations of HW all running in the same cluster.

What if fat server goes down.

Balancer could take of it all. RS Capacity. Balancer can take it into account.
Regionserver labels like YARN labels. Characteristics.

Or run it all in docker when heterogeneous cluster. The K8 talk from day before 
was mentioned; we should all look at being able to deploy in k8 and docker.

Lets put out kubernetes blog...(Doing).

Alibaba looking at HBase as native YARN app.

i/o is hard even when containers.

Use autoscaler of K8 when heavy user.

Limit i/o use w/ CP. Throttle.

Spark and client-side scanner came up again.

Snapshot input format in spark.

HBase federation came up again. jd.com talking of 3k to 4k nodes 
in a cluster. Millions of regions. Region assignment is messing them up.

Maybe federation is good idea? Argument that it is too much operational 
conplexity. Can we fix master load w/ splittable meta, etc?

Was brought up that even w/ 100s of RS there is scale issue, nvm thousands.

Alibaba talked about disaster recovery. Described issue where HDFS has fencing 
problem during an upgrade. There was no active NN. All RS went down.
ZK is another POF. If ZK is not available. Operators were being asked how much 
longer the cluster was going to be down but they could not answer the question. 
No indicators from HBase on how much longer it will be down or how many WALs 
its processed and how many more to go. Operator unable to tell his org how long 
it would be before it all came back on line. Should say how many regions are 
online and how many more to do.

Alibaba use SQL to lower cost. HBase API is low-level. Row-key construction is 
tricky. New users make common mistakes. If you don't do schema right, 
high-performance is difficult.

Alibaba are using a subset of Phoenix... simple sql only; throws exceptions if 
user tries to do joins, etc.., anything but basic ops.

HareQL is using hive for meta store.  Don't have data typing in hbase.

HareQL could perhaps contribute some piece... or a module in hbase to sql... 
From phoenix?

Secondary index.

Client is complicated in phoenix. Was suggested thin client just does parse... 
and then offload to server for optimization and execution.

Then secondary index. Need transaction engine. Consistency of secondary index.

We adjourned.

Your dodgy secretary,
St.Ack
P.S. Please add to this base set of notes if I missed anything.

RE: Moving 2.0 forward

[ashish singhi]

One question regarding upgrade: 
Do we expect the user to move to latest branch-1 release and then upgrade to 
2.0 version ?

I think that should not be imposed on the user. 

Regards,
Ashish

-Original Message-
From: 张铎(Duo Zhang) [mailto:palomino...@gmail.com] 
Sent: 26 July 2017 12:10
To: dev@hbase.apache.org
Subject: Re: Moving 2.0 forward

I set HBASE-18169 as a Blocker because I found some critial problems on our 
current CPs. The semantics are broken. Although we are allowed to break CP in a 
major release, I think we need to provide the same ability(in another way).

2017-07-25 1:25 GMT+08:00 Josh Elser :

>
>
> On 7/21/17 12:03 PM, Stack wrote:
>
>> Status update girls and boys!
>>
>> hbase-2.0.0-alpha1 went out June 22nd.
>>
>> alpha2 has been a bit slow to follow (holidays) though there has been 
>> steady progress closing out blockers and criticals by a bunch of you all.
>> The plan is for a release in the first week or so of August. It 
>> should be fully up on hbase-thirdparty using updated (and relocated) 
>> versions of netty, guava, and protobuf as well as a default deploy 
>> that has master-carrying-no-regions.
>>
>> alpha3 will follow soon after and will focus on making sure our 
>> user-facing APIs are clean (branch-1 compatible, no illicit 
>> removals/mods, and so on) and that basic upgrade 'works'.
>>
>> betas start in September?
>>
>> I've been keeping a rough general state here [1] (please update any 
>> section that is lagging actuality) but for details on what blockers 
>> and criticals remain, see the JIRA 2.0 view [2]. Recent 
>> issue-gardening has brought 2.0 into better focus. Feel free to 
>> review and punt items you think can wait till 3.0 or 2.1. If you want 
>> to pull in more stuff, please ask first.
>>
>
> Chia-Ping (I think? -- JIRA is being a pain) had asked on the 
> space-quota
> phase2 work (include size of hbase snapshots in a table's "quota 
> usage") if we should try to also include that work in 2.0.
>
> I like the idea of this also hitting 2.0 as it would make the feature 
> a bit more "real", but am obviously a little nervous (I have no reason 
> to be nervous though). I am pretty happy with the feature in terms of 
> how much it is covered via testing.
>
> https://issues.apache.org/jira/browse/HBASE-17748
>
>
> Thanks,
>> St.Ack
>>
>> 1.
>> https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9i
>> Eu_ktczrlKHK8N4SZzs/edit#
>> 2. https://issues.apache.org/jira/projects/HBASE/versions/12327188
>>
>
> - Josh
>

[jira] [Created] (HBASE-18437) Revoke access permissions of a user from a table does not work as expected

[Ashish Singhi (JIRA)]

Ashish Singhi created HBASE-18437:
-

 Summary: Revoke access permissions of a user from a table does not 
work as expected
 Key: HBASE-18437
 URL: https://issues.apache.org/jira/browse/HBASE-18437
 Project: HBase
  Issue Type: Bug
  Components: security
Affects Versions: 1.1.12
Reporter: Ashish Singhi
Assignee: Ashish Singhi


A table for which a user was granted 'RW' permission. Now when we want to 
revoke its 'W' permission only, code removes the user itself from that table 
permissions.
Below is the test code which reproduces the issue.

{noformat}
@Test(timeout = 18)
  public void testRevokeOnlySomePerms() throws Throwable {
TableName name = TableName.valueOf("testAgain");
HTableDescriptor htd = new HTableDescriptor(name);
HColumnDescriptor hcd = new HColumnDescriptor("cf");
htd.addFamily(hcd);
createTable(TEST_UTIL, htd);
TEST_UTIL.waitUntilAllRegionsAssigned(name);

try (Connection conn = ConnectionFactory.createConnection(conf)) {
  AccessControlClient.grant(conn, name, USER_RO.getShortName(), null, null, 
Action.READ, Action.WRITE);
  ListMultimap<String, TablePermission> tablePermissions = 
AccessControlLists.getTablePermissions(conf, name);
  // hbase user and USER_RO has permis
  assertEquals(2, tablePermissions.size());

  AccessControlClient.revoke(conn, name, USER_RO.getShortName(), null, 
null, Action.READ, Action.WRITE);
  tablePermissions = AccessControlLists.getTablePermissions(conf, name);
  List userPerm = 
tablePermissions.get(USER_RO.getShortName());
  assertEquals(1, userPerm.size());

} finally {
  deleteTable(TEST_UTIL, name);
}
  }
{noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [ANNOUNCE] Chunhui Shen joins the Apache HBase PMC

[Ashish Singhi]

Congratulations!

Sent from my iPhone

> On 04-Jul-2017, at 10:54 AM, Yu Li  wrote:
> 
> On behalf of the Apache HBase PMC I am pleased to announce that Chunhui Shen
> has accepted our invitation to become a PMC member on the Apache
> HBase project. He has been an active contributor to HBase for past many
> years. Looking forward for many more contributions from him.
> 
> Please join me in welcoming Chunhui to the HBase PMC!
> 
> Best Regards,
> Yu

Re: [ANNOUNCE] Devaraj Das joins the Apache HBase PMC

[Ashish Singhi]

Congratulations!

Sent from my iPhone

> On 05-Jul-2017, at 9:57 PM, Josh Elser  wrote:
> 
> I'm pleased to announce yet another PMC addition in the form of Devaraj Das. 
> One of the "old guard" in the broader Hadoop umbrella, he's also a 
> long-standing member in our community. We all look forward to the continued 
> contributions and project leadership.
> 
> Please join me in welcoming Devaraj!
> 
> - Josh (on behalf of the PMC)

Re: [VOTE] First hbase-2.0.0-alpha-1 Release Candidate is available

[Ashish Singhi]

Hi Umesh,
I am still not able to reproduce it in branch-2. I
ran TestEncryptionKeyRotation#testCFKeyRotation which
exercise FSDataInputStreamWrapper#unbuffer code flow also.
Is your server side Hadoop also 2.7.1 version ?

Regards,
Ashish

On Mon, Jun 12, 2017 at 9:04 PM, Umesh Agashe  wrote:

> Hi Stack,
>
> I am using hadoop-2.7.1. It was on single node.
>
> Thanks,
> Umesh
>
>

Re: [VOTE] First hbase-2.0.0-alpha-1 Release Candidate is available

[Ashish Singhi]

I installed the version on a single node cluster with hadoop 2.7.3 version
on server side and default hadoop version at client side in HBase i.e.,
2.7.1 and didn't see any such log warning message.
@Umesh: what hadoop version are you using ?


Belated +1 on the RC.
Build the tar ball from the source code,
Exercised some shell commands.
Ran bulk load, import, export job.
Checked audit logs for the operations executed.

Regards,
Ashish

On Sun, Jun 11, 2017 at 11:48 AM, Ashish Singhi <ashishsin...@apache.org>
wrote:

> It should not.
> I will try to find some time today and install the alpha release and check
> this part.
>
> Thanks,
> Ashish
>
> On Sun, Jun 11, 2017 at 12:02 AM, Stack <st...@duboce.net> wrote:
>
>> On Sat, Jun 10, 2017 at 1:35 AM, Ashish Singhi <ashishsin...@apache.org>
>> wrote:
>>
>> > Its not an issue. The log warnings are coming by HBASE-9393 patch. If
>> user
>> > Hadoop version is not 2.6.4+ or 2.7.0+ then this warn message will be
>> > logged, notifying user about a CLOSE_WAIT connection left behind. As
>> this
>> > log message is in common code flow of read, should we up the log level
>> to
>> > avoid too many same warnings ?
>> >
>> >
>> The alpha has hadoop-2.7.1. Should we be getting the above logs in this
>> case? (Correct me Umesh if you came up on a different hadoop -- I don't
>> think you did).
>>
>> S
>>
>>
>>
>> > Regards,
>> > Ashish
>> >
>> > On Fri, Jun 9, 2017 at 10:45 AM, Stack <st...@duboce.net> wrote:
>> >
>> > > Oh, maybe file issues for what you found boss?
>> > > St.Ack
>> > >
>> > > On Thu, Jun 8, 2017 at 10:15 PM, Stack <st...@duboce.net> wrote:
>> > >
>> > > > Thanks Umesh!
>> > > >
>> > > > On Thu, Jun 8, 2017 at 2:36 PM, Umesh Agashe <uaga...@cloudera.com>
>> > > wrote:
>> > > >
>> > > >> +1
>> > > >>
>> > > >> - Downloaded the src tarball.
>> > > >>   - Verified the checksums and signature. Looks good.
>> > > >>   - Built from source. openjdk version "1.8.0_102"
>> > > >>   - mvn apache-rat:check   Passed
>> > > >>
>> > > >> - Download the bin tarball.
>> > > >>   - Verified the checksums and signature. Looks good.
>> > > >>   - Extracted it. Layout looks good.
>> > > >>   - Started a local instance. Ran the 'hbase shell' with some basic
>> > > table
>> > > >> commands. Looks good.
>> > > >>   - Ran ltt with 10 million rows and 2 columns/ row
>> > > >>
>> > > >> Note:
>> > > >>   - Did not run unit tests. There are a problems with the build.
>> See:
>> > > >> https://builds.apache.org/job/HBase-2.0/jdk=JDK%201.8%20(lat
>> > > >> est),label=Hadoop/10/console
>> > > >>   - Noticed a few errors in the log file like:
>> > > >> WARN  [MemStoreFlusher.1] io.FSDataInputStreamWrapper: Failed to
>> > invoke
>> > > >> 'unbuffer' method in class class org.apache.hadoop.fs.
>> > FSDataInputStream
>> > > .
>> > > >> So there may be a TCP socket connection left open in CLOSE_WAIT
>> state.
>> > > >>
>> > > >>
>>
>

Re: [VOTE] First hbase-2.0.0-alpha-1 Release Candidate is available

[Ashish Singhi]

It should not.
I will try to find some time today and install the alpha release and check
this part.

Thanks,
Ashish

On Sun, Jun 11, 2017 at 12:02 AM, Stack <st...@duboce.net> wrote:

> On Sat, Jun 10, 2017 at 1:35 AM, Ashish Singhi <ashishsin...@apache.org>
> wrote:
>
> > Its not an issue. The log warnings are coming by HBASE-9393 patch. If
> user
> > Hadoop version is not 2.6.4+ or 2.7.0+ then this warn message will be
> > logged, notifying user about a CLOSE_WAIT connection left behind. As this
> > log message is in common code flow of read, should we up the log level to
> > avoid too many same warnings ?
> >
> >
> The alpha has hadoop-2.7.1. Should we be getting the above logs in this
> case? (Correct me Umesh if you came up on a different hadoop -- I don't
> think you did).
>
> S
>
>
>
> > Regards,
> > Ashish
> >
> > On Fri, Jun 9, 2017 at 10:45 AM, Stack <st...@duboce.net> wrote:
> >
> > > Oh, maybe file issues for what you found boss?
> > > St.Ack
> > >
> > > On Thu, Jun 8, 2017 at 10:15 PM, Stack <st...@duboce.net> wrote:
> > >
> > > > Thanks Umesh!
> > > >
> > > > On Thu, Jun 8, 2017 at 2:36 PM, Umesh Agashe <uaga...@cloudera.com>
> > > wrote:
> > > >
> > > >> +1
> > > >>
> > > >> - Downloaded the src tarball.
> > > >>   - Verified the checksums and signature. Looks good.
> > > >>   - Built from source. openjdk version "1.8.0_102"
> > > >>   - mvn apache-rat:check   Passed
> > > >>
> > > >> - Download the bin tarball.
> > > >>   - Verified the checksums and signature. Looks good.
> > > >>   - Extracted it. Layout looks good.
> > > >>   - Started a local instance. Ran the 'hbase shell' with some basic
> > > table
> > > >> commands. Looks good.
> > > >>   - Ran ltt with 10 million rows and 2 columns/ row
> > > >>
> > > >> Note:
> > > >>   - Did not run unit tests. There are a problems with the build.
> See:
> > > >> https://builds.apache.org/job/HBase-2.0/jdk=JDK%201.8%20(lat
> > > >> est),label=Hadoop/10/console
> > > >>   - Noticed a few errors in the log file like:
> > > >> WARN  [MemStoreFlusher.1] io.FSDataInputStreamWrapper: Failed to
> > invoke
> > > >> 'unbuffer' method in class class org.apache.hadoop.fs.
> > FSDataInputStream
> > > .
> > > >> So there may be a TCP socket connection left open in CLOSE_WAIT
> state.
> > > >>
> > > >>
> > > >> On Thu, Jun 8, 2017 at 8:33 AM, Stack <st...@duboce.net> wrote:
> > > >>
> > > >> > On Thu, Jun 8, 2017 at 7:52 AM, Sean Busbey <bus...@apache.org>
> > > wrote:
> > > >> >
> > > >> > > The minimum voting period is 72 hours for a PMC to bless a
> > release.
> > > >> > > Can we aim for that?
> > > >> > >
> > > >> > >
> > > >> > Sure (Stickler!)
> > > >> > St.Ack
> > > >> >
> > > >> >
> > > >> >
> > > >> > > On Thu, Jun 8, 2017 at 2:33 AM, Stack <st...@duboce.net> wrote:
> > > >> > > > The first release candidate for HBase 2.0.0-alpha-1 is up at:
> > > >> > > >
> > > >> > > >  https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.0-alp
> > > >> ha-1RC0/
> > > >> > > >
> > > >> > > > Maven artifacts are available from a staging directory here:
> > > >> > > >
> > > >> > > >  https://repository.apache.org/content/repositories/
> > > >> > orgapachehbase-1169
> > > >> > > >
> > > >> > > > All was signed w/ my key 8ACC93D2
> > > >> > > > <http://pgp.mit.edu/pks/lookup?op=get=
> 0x9816C7FC8ACC93D2
> > >
> > > >> > > >
> > > >> > > > I tagged the RC as 2.0.0-alpha-1RC0
> > > >> > > > (c830a0f47f58d4892dd3300032c8244d6278aecc).
> > > >> > > >
> > > >> > > > hbase-2.0.0-alpha-1 will be our first 2.0.0 release. It is a
> > rough
> > > >> cut
> > > >> > > > ('alpha') not-for-product

Re: [VOTE] First hbase-2.0.0-alpha-1 Release Candidate is available

[Ashish Singhi]

Its not an issue. The log warnings are coming by HBASE-9393 patch. If user
Hadoop version is not 2.6.4+ or 2.7.0+ then this warn message will be
logged, notifying user about a CLOSE_WAIT connection left behind. As this
log message is in common code flow of read, should we up the log level to
avoid too many same warnings ?

Regards,
Ashish

On Fri, Jun 9, 2017 at 10:45 AM, Stack  wrote:

> Oh, maybe file issues for what you found boss?
> St.Ack
>
> On Thu, Jun 8, 2017 at 10:15 PM, Stack  wrote:
>
> > Thanks Umesh!
> >
> > On Thu, Jun 8, 2017 at 2:36 PM, Umesh Agashe 
> wrote:
> >
> >> +1
> >>
> >> - Downloaded the src tarball.
> >>   - Verified the checksums and signature. Looks good.
> >>   - Built from source. openjdk version "1.8.0_102"
> >>   - mvn apache-rat:check   Passed
> >>
> >> - Download the bin tarball.
> >>   - Verified the checksums and signature. Looks good.
> >>   - Extracted it. Layout looks good.
> >>   - Started a local instance. Ran the 'hbase shell' with some basic
> table
> >> commands. Looks good.
> >>   - Ran ltt with 10 million rows and 2 columns/ row
> >>
> >> Note:
> >>   - Did not run unit tests. There are a problems with the build. See:
> >> https://builds.apache.org/job/HBase-2.0/jdk=JDK%201.8%20(lat
> >> est),label=Hadoop/10/console
> >>   - Noticed a few errors in the log file like:
> >> WARN  [MemStoreFlusher.1] io.FSDataInputStreamWrapper: Failed to invoke
> >> 'unbuffer' method in class class org.apache.hadoop.fs.FSDataInputStream
> .
> >> So there may be a TCP socket connection left open in CLOSE_WAIT state.
> >>
> >>
> >> On Thu, Jun 8, 2017 at 8:33 AM, Stack  wrote:
> >>
> >> > On Thu, Jun 8, 2017 at 7:52 AM, Sean Busbey 
> wrote:
> >> >
> >> > > The minimum voting period is 72 hours for a PMC to bless a release.
> >> > > Can we aim for that?
> >> > >
> >> > >
> >> > Sure (Stickler!)
> >> > St.Ack
> >> >
> >> >
> >> >
> >> > > On Thu, Jun 8, 2017 at 2:33 AM, Stack  wrote:
> >> > > > The first release candidate for HBase 2.0.0-alpha-1 is up at:
> >> > > >
> >> > > >  https://dist.apache.org/repos/dist/dev/hbase/hbase-2.0.0-alp
> >> ha-1RC0/
> >> > > >
> >> > > > Maven artifacts are available from a staging directory here:
> >> > > >
> >> > > >  https://repository.apache.org/content/repositories/
> >> > orgapachehbase-1169
> >> > > >
> >> > > > All was signed w/ my key 8ACC93D2
> >> > > > 
> >> > > >
> >> > > > I tagged the RC as 2.0.0-alpha-1RC0
> >> > > > (c830a0f47f58d4892dd3300032c8244d6278aecc).
> >> > > >
> >> > > > hbase-2.0.0-alpha-1 will be our first 2.0.0 release. It is a rough
> >> cut
> >> > > > ('alpha') not-for-production preview of what hbase-2.0.0 will look
> >> > like.
> >> > > It
> >> > > > is what we used to call a 'Developer' release[1] meant mostly for
> >> devs
> >> > > and
> >> > > > downstreamers to test drive and flag us early if we there are
> issues
> >> > > we’ve
> >> > > > missed ahead of our rolling a production-worthy release.
> >> > > >
> >> > > > hbase-2.0.0 includes a fleet of new features that include a new
> >> > > assignment
> >> > > > manager, means for keeping read and write path off-heap, in-memory
> >> > > > compactions, and more. I have been keeping a running doc on the
> >> state
> >> > of
> >> > > > 2.0.0 here [2]. There is much to do still (see aforementioned
> doc).
> >> > > >
> >> > > > The list of features addressed in 2.0.0 so far can be found here
> >> [4].
> >> > > There
> >> > > > are about 2500. The list of ~500 fixes in 2.0.0 exclusively can be
> >> > found
> >> > > > here [3].
> >> > > >
> >> > > > Please take it for a spin and vote on whether it ok to put out as
> >> our
> >> > > first
> >> > > > alpha (bar is low for an 'alpha'). Let the VOTE be open for 24
> >> hours.
> >> > > >
> >> > > > Thanks,
> >> > > > St.Ack
> >> > > >
> >> > > > 1. http://hbase.apache.org/book.html#hbase.versioning.pre10
> >> > > > 2.
> >> > > > https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9iEu_
> >> > > ktczrlKHK8N4SZzs/edit#heading=h.v21r9nz8g01j
> >> > > > 3.
> >> > > > https://issues.apache.org/jira/browse/HBASE-17852?jql=
> >> > > project%20%3D%20HBASE%20%20and%20fixVersion%20%3D%202.
> >> > > 0.0%20and%20fixVersion%20not%20in%20(1.0.0%2C%201.0.1%2C%
> >> > > 201.0.2%2C%201.0.3%2C%201.0.4%2C%201.0.5%2C%201.0.6%2C%201.
> >> > > 1.0%2C%201.1.1%2C%201.1.2%2C%201.1.3%2C%201.1.4%2C%201.1.5%
> >> > > 2C%201.1.6%2C%201.1.7%2C%201.1.8%2C%201.1.9%2C%201.1.10%2C%
> >> > > 201.2.0%2C%201.2.1%2C%201.2.2%2C%201.2.3%2C%201.2.4%2C%201.
> >> > > 2.5%2C%201.2.6%2C%201.3.0%2C%201.3.1%2C%201.4.0)%20and%20%
> >> > > 20(status%20%3D%20Open%20or%20status%20%3D%20%22Patch%
> 20Available%22)
> >> > > > 4.
> >> > > > https://issues.apache.org/jira/browse/HBASE-18191?jql=
> >> > > project%20%3D%20HBASE%20%20and%20(%20fixVersion%20%3D%
> >> > > 

[jira] [Resolved] (HBASE-17468) unread messages in TCP connections - possible connection leak

[Ashish Singhi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi resolved HBASE-17468.
---
Resolution: Duplicate

Duplicate of HBASE-9393

> unread messages in TCP connections - possible connection leak
> -
>
> Key: HBASE-17468
> URL: https://issues.apache.org/jira/browse/HBASE-17468
> Project: HBase
>  Issue Type: Bug
>Reporter: Shridhar Sahukar
>Priority: Critical
>
> We are running HBase 1.2.0-cdh5.7.1 (Cloudera distribution).
> On our Hadoop cluster, we are seeing that each HBase region server has large 
> number of TCP connections to all the HDFS data nodes and all these 
> connections have unread data in socket buffers. Some of these connections are 
> also in CLOSE_WAIT or FIN_WAIT1 state while the rest are in ESTABLISHED state.
> Looks like HBase is creating some connections requesting data from HDFS, but 
> its forgetting about those connections before it could read the data. Thus 
> the connections are left lingering around with large data stuck in their 
> receive buffers. Also, it seems HDFS closes these connections after a while, 
> but since there is data in receive buffer the connection is left in 
> CLOSE_WAIT/FIN_WAIT1 states.
> Below is a snapshot from one of the region servers:
> ## Total number of connections to HDFS  (pid of region server is 143722)
> [bda@md-bdadev-42 hbase]$ sudo netstat -anp|grep 143722 | wc -l
> 827
> ## Connections that are not in ESTABLISHED state
> [bda@md-bdadev-42 hbase]$ sudo netstat -anp|grep 143722 | grep -v ESTABLISHED 
> | wc -l
> 344
> ##Snapshot of some of these connections:
> tcp   133887  0 146.1.180.43:48533  146.1.180.40:50010  
> ESTABLISHED 143722/java
> tcp82934  0 146.1.180.43:59647  146.1.180.42:50010  
> ESTABLISHED 143722/java
> tcp0  0 146.1.180.43:50761  146.1.180.27:2181   
> ESTABLISHED 143722/java
> tcp   234084  0 146.1.180.43:58335  146.1.180.42:50010  
> ESTABLISHED 143722/java
> tcp   967667  0 146.1.180.43:56136  146.1.180.68:50010  
> ESTABLISHED 143722/java
> tcp   156037  0 146.1.180.43:59659  146.1.180.42:50010  
> ESTABLISHED 143722/java
> tcp   212488  0 146.1.180.43:56810  146.1.180.48:50010  
> ESTABLISHED 143722/java
> tcp61871  0 146.1.180.43:53593  146.1.180.35:50010  
> ESTABLISHED 143722/java
> tcp   121216  0 146.1.180.43:35324  146.1.180.38:50010  
> ESTABLISHED 143722/java
> tcp1  0 146.1.180.43:32982  146.1.180.42:50010  
> CLOSE_WAIT  143722/java
> tcp82934  0 146.1.180.43:42359  146.1.180.54:50010  
> ESTABLISHED 143722/java
> tcp   159422  0 146.1.180.43:59731  146.1.180.42:50010  
> ESTABLISHED 143722/java
> tcp   134573  0 146.1.180.43:60210  146.1.180.76:50010  
> ESTABLISHED 143722/java
> tcp82934  0 146.1.180.43:59713  146.1.180.42:50010  
> ESTABLISHED 143722/java
> tcp   135765  0 146.1.180.43:44412  146.1.180.29:50010  
> ESTABLISHED 143722/java
> tcp   161655  0 146.1.180.43:43117  146.1.180.42:50010  
> ESTABLISHED 143722/java
> tcp75990  0 146.1.180.43:59729  146.1.180.42:50010  
> ESTABLISHED 143722/java
> tcp78583  0 146.1.180.43:59971  146.1.180.42:50010  
> ESTABLISHED 143722/java
> tcp1  0 146.1.180.43:39893  146.1.180.67:50010  
> CLOSE_WAIT  143722/java
> tcp1  0 146.1.180.43:38834  146.1.180.47:50010  
> CLOSE_WAIT  143722/java
> tcp1  0 146.1.180.43:40707  146.1.180.50:50010  
> CLOSE_WAIT  143722/java
> tcp   106102  0 146.1.180.43:48208  146.1.180.75:50010  
> ESTABLISHED 143722/java
> tcp   332013  0 146.1.180.43:34795  146.1.180.37:50010  
> ESTABLISHED 143722/java
> tcp1  0 146.1.180.43:57644  146.1.180.67:50010  
> CLOSE_WAIT  143722/java
> tcp79119  0 146.1.180.43:54438  146.1.180.70:50010  
> ESTABLISHED 143722/java
> tcp77438  0 146.1.180.43:35259  146.1.180.38:50010  
> ESTABLISHED 143722/java
> tcp1  0 146.1.180.43:57579  146.1.180.41:50010  
> CLOSE_WAIT  143722/java
> tcp   318091  0 146.1.180.43:60124  146.1.180.42:50010  
> ESTABLISHED 1437

Re: HBASE and MOB

[Ashish Singhi]

OK. Agree for not back porting it to branch-1 considering the above fair
comments.
Thanks for the comments.

Regards,
Ashish

On Tue, May 16, 2017 at 2:20 AM, Huaxiang Sun <h...@cloudera.com> wrote:

> +1, not a trivial effort.
>
> > On May 15, 2017, at 1:31 PM, Mikhail Antonov <olorinb...@gmail.com>
> wrote:
> >
> > +1 for not trying to backport massive new features to branch-1.
> >
> > IMO Branch-1 release lines should be continuing the path towards
> > maintenance mode only.
> >
> > -Mikhail
> >
> > On Mon, May 15, 2017 at 1:21 PM, Ted Yu <yuzhih...@gmail.com> wrote:
> >
> >> bq. concentrate on getting MOB out the door in 2.0.
> >>
> >> +1
> >>
> >>
> >> On Mon, May 15, 2017 at 1:11 PM, Sean Busbey <bus...@apache.org> wrote:
> >>
> >>> -user@hbase to bcc (I encourage folks interest to subscribe to
> dev@hbase
> >> )
> >>>
> >>> Personally, I'd rather see us concentrate on getting MOB out the door
> in
> >>> 2.0.
> >>>
> >>> branch-1 is now over 2 years old; I'd want some rigorous proof that
> we're
> >>> not setting our downstream users up for surprises by making major
> changes
> >>> to the critical path.
> >>>
> >>> On May 15, 2017 06:45, "ashish singhi" <ashish.sin...@huawei.com>
> wrote:
> >>>
> >>>> Hi.
> >>>>
> >>>> Can we revive HBASE-15370: Backport Moderate Object Storage (MOB) to
> >>>> branch-1 ?
> >>>>
> >>>> Even we have customers using this feature and it requires lots of
> >> effort
> >>>> to backport MOB patches from master branch to the released versions as
> >>> the
> >>>> code base has lots of differences.
> >>>>
> >>>> Regards,
> >>>> Ashish
> >>>>
> >>>> -Original Message-
> >>>> From: Ted Yu [mailto:yuzhih...@gmail.com]
> >>>> Sent: 12 May 2017 22:25
> >>>> To: u...@hbase.apache.org
> >>>> Subject: Re: HBASE and MOB
> >>>>
> >>>> MOB is also backported in HDP 2.5.x
> >>>>
> >>>> FYI
> >>>>
> >>>> On Fri, May 12, 2017 at 9:51 AM, anil gupta <anilgupt...@gmail.com>
> >>> wrote:
> >>>>
> >>>>> Backporting MOB wont be a trivial task.
> >>>>> AFAIK, Cloudera backported MOB to HBase1.x  branch for CDH(its not in
> >>>>> apache HBase1.x branch yet). It might be easier to just use CDH for
> >>> MOB.
> >>>>>
> >>>>> On Fri, May 12, 2017 at 8:51 AM, Jean-Marc Spaggiari <
> >>>>> jean-m...@spaggiari.org> wrote:
> >>>>>
> >>>>>> Thanks for those details.
> >>>>>>
> >>>>>> How big are you PDF? Are they all small size? If they are not above
> >>>>>> 1MB, MOBs will not really be 100% mandatory. Even if few of them
> >> are
> >>>> above.
> >>>>>>
> >>>>>> If you want to apply the patch on another branch,this is what is
> >>>>>> called a back port (like Ted said before) and will require a pretty
> >>>>>> good amount of work. You can jump on that, but if you are not used
> >>>>>> to the HBase code, it might be a pretty big challenge...
> >>>>>>
> >>>>>> Another way is to look for an HBase distribution that already
> >>>>>> includes
> >>>>> the
> >>>>>> MOB code already.
> >>>>>>
> >>>>>> JMS
> >>>>>>
> >>>>>> 2017-05-12 11:21 GMT-04:00 F. T. <bibo...@hotmail.fr>:
> >>>>>>
> >>>>>>> Hi Jean Marc
> >>>>>>>
> >>>>>>> I'm using a 1.2.3 version. I downloaded a "bin" version from
> >>>>>>> Apache official web site. Maybe I've to install it from the "src"
> >>>>>>> option with
> >>>>>> mvn ?
> >>>>>>>
> >>>>>>> I would like index PDF into Hbase and use it in a Solr
> >> collection.
> >>>>>>>
> >>>>>>> In fact I would like reproduce this process :
> >&g

RE: HBASE and MOB

[ashish singhi]

Hi.

Can we revive HBASE-15370: Backport Moderate Object Storage (MOB) to branch-1 ?

Even we have customers using this feature and it requires lots of effort to 
backport MOB patches from master branch to the released versions as the code 
base has lots of differences.

Regards,
Ashish

-Original Message-
From: Ted Yu [mailto:yuzhih...@gmail.com] 
Sent: 12 May 2017 22:25
To: u...@hbase.apache.org
Subject: Re: HBASE and MOB

MOB is also backported in HDP 2.5.x

FYI

On Fri, May 12, 2017 at 9:51 AM, anil gupta  wrote:

> Backporting MOB wont be a trivial task.
> AFAIK, Cloudera backported MOB to HBase1.x  branch for CDH(its not in 
> apache HBase1.x branch yet). It might be easier to just use CDH for MOB.
>
> On Fri, May 12, 2017 at 8:51 AM, Jean-Marc Spaggiari < 
> jean-m...@spaggiari.org> wrote:
>
> > Thanks for those details.
> >
> > How big are you PDF? Are they all small size? If they are not above 
> > 1MB, MOBs will not really be 100% mandatory. Even if few of them are above.
> >
> > If you want to apply the patch on another branch,this is what is 
> > called a back port (like Ted said before) and will require a pretty 
> > good amount of work. You can jump on that, but if you are not used 
> > to the HBase code, it might be a pretty big challenge...
> >
> > Another way is to look for an HBase distribution that already 
> > includes
> the
> > MOB code already.
> >
> > JMS
> >
> > 2017-05-12 11:21 GMT-04:00 F. T. :
> >
> > > Hi Jean Marc
> > >
> > > I'm using a 1.2.3 version. I downloaded a "bin" version from 
> > > Apache official web site. Maybe I've to install it from the "src" 
> > > option with
> > mvn ?
> > >
> > > I would like index PDF into Hbase and use it in a Solr collection.
> > >
> > > In fact I would like reproduce this process :
> > > http://blog.cloudera.com/blog/2015/10/how-to-index-scanned-
> > > pdfs-at-scale-using-fewer-than-50-lines-of-code/
> > >
> > >
> > > But maybe is there another solution to reproduce it .
> > >
> > > Fred
> > >
> > >
> > > 
> > > De : Jean-Marc Spaggiari  Envoyé : 
> > > vendredi 12 mai 2017 17:06 À : user Objet : Re: HBASE and MOB
> > >
> > > Hi Fred,
> > >
> > > Can you please confirm the following information?
> > >
> > > 1) What exact version of HBase are you using? From a distribution,
> build
> > by
> > > yourself, from the JARs, etc.
> > > 2) Why do you think you need the MOB feature
> > > 3) Is an upgrade an option for you or not really.
> > >
> > > Thanks,
> > >
> > > JMS
> > >
> > >
> > > 2017-05-12 11:02 GMT-04:00 Ted Yu :
> > >
> > > > It is defined here in
> > > > hbase-client/src/main/java/org/apache/hadoop/hbase/
> > > HColumnDescriptor.java:
> > > >   public static final String IS_MOB = "IS_MOB";
> > > >
> > > > MOB feature hasn't been backported to branch-1 (or earlier releases).
> > > >
> > > > Looks like you're using a vendor's release.
> > > >
> > > > Consider contacting the corresponding mailing list if you are stuck.
> > > >
> > > > On Fri, May 12, 2017 at 7:59 AM, F. T.  wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > I'd like to use MOB in HBase to store PDF files. I'm using 
> > > > > Hbase
> > 1.2.3
> > > > but
> > > > > I'get this error creating a table with MOB column : NameError:
> > > > > uninitialized constant IS_MOB.
> > > > >
> > > > > A lot of web sites (including Apache official web site) talk 
> > > > > about
> > the
> > > > > patch 11339 or HBase 2.0.0, but, I don't find any explanation 
> > > > > about
> > the
> > > > way
> > > > > to install this patch and
> > > > >
> > > > > I can't find the 2.0.0 version anywhere. So I'm completly lost.
> Could
> > > you
> > > > > help me please ?
> > > > >
> > > > >
> > > >
> > >
> >
>
>
>
> --
> Thanks & Regards,
> Anil Gupta
>

RE: [ANNOUNCE] - Welcome our new HBase committer Anastasia Braginsky

[ashish singhi]

Congrats and Welcome!

-Original Message-
From: ramkrishna vasudevan [mailto:ramkrishna.s.vasude...@gmail.com] 
Sent: 27 March 2017 18:08
To: dev@hbase.apache.org; u...@hbase.apache.org
Subject: [ANNOUNCE] - Welcome our new HBase committer Anastasia Braginsky

Hi All

Welcome Anastasia Braginsky, one more female committer to HBase. She has been 
active now for a while with her Compacting memstore feature and she along with 
Eshcar have done lot of talks in various meetups and HBaseCon on their feature.

Welcome onboard and looking forward to work with you Anastasia !!!

Regards
Ram

Re: [ANNOUNCE] New HBase committer Chia-Ping Tsai

[Ashish Singhi]

Many Congratulations !

On Sat, Mar 18, 2017 at 4:00 AM, Anoop John  wrote:

> On behalf of the Apache HBase PMC, I am pleased to announce that Chia-Ping
> Tsai
> has accepted the PMC's invitation to become a committer on the project. We
> appreciate all of his contributions thus far and look forward
> to his continued involvement.
>
> Congratulation and welcome Chia-Ping Tsai!
>
> -Anoop-
>

RE: ANNOUNCE: Good news!!!! Two new PMC and a new Committer!

[ashish singhi]

Many congratulations, Josh, Jing Chen He and Eshcar !!!

-Original Message-
From: saint@gmail.com [mailto:saint@gmail.com] On Behalf Of Stack
Sent: 17 March 2017 11:07
To: HBase Dev List 
Subject: ANNOUNCE: Good news Two new PMC and a new Committer!

Josh Elser and Jing Chen (Jerry) have both been added to the HBase PMC.
These lads have been stellar helping the project along; Jerry for a good few 
years now and Josh, though less time served, has made up for the lack with 
style. It makes sense that they be made PMCers!

Let me take this opportunity while I have your attention to also welcome Eshcar 
Hillel as a Committer on Apache HBase. We are very glad to have her on-board. 
She's a brainbox who has been working on the inmemory compaction project among 
other things and is without fear when it comes to taking a deep dive into crud!

Welcome aboard Josh, Jing Chen He, and Eshcar!

St.Ack

[jira] [Resolved] (HBASE-17687) hive on hbase table and phoenix table can't be selected

[Ashish Singhi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi resolved HBASE-17687.
---
  Resolution: Invalid
Hadoop Flags:   (was: Incompatible change)

> hive on hbase table and phoenix table can't  be selected
> 
>
> Key: HBASE-17687
> URL: https://issues.apache.org/jira/browse/HBASE-17687
> Project: HBase
>  Issue Type: Improvement
>  Components: hbase
>Affects Versions: 1.0.2
> Environment: hadoop 2.7.2
> hbase 1.0.2
> phoenix 4.4
> hive 1.3
> all above are based on huawei FusionInsight HD(FusionInsight 
> V100R002C60U10SPC001)
>Reporter: yunliangchen
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> First , I created a table on phoenix, as this:
> ---
> DROP TABLE IF EXISTS bidwd_test01 CASCADE;
> CREATE TABLE IF NOT EXISTS bidwd_test01(
>rk VARCHAR,
>c1 integer,
>c2 VARCHAR,
>c3 VARCHAR,
>c4 VARCHAR
>constraint bidwd_test01_pk primary key(rk)
> )
> COMPRESSION='SNAPPY'
> ;
> ---
> And then , I upserted two rows into the table:
> ---
> upsert into bidwd_test01 values('001',1,'zhangsan','20170217','2017-02-17 
> 12:34:22');
> upsert into bidwd_test01 values('002',2,'lisi','20170216','2017-02-16 
> 12:34:22');
> ---
> At last , I scaned the table like this:
> ---
> select * from bidwd_test01;
> ---
> It's OK by now, but, I want to create a hive on hbase table ,that mapping to 
> the phoenix table , the script likes this:
> ---
> USE BIDWD;
> DROP TABLE test01;
> CREATE EXTERNAL TABLE test01
> (
>  rk string,
>  id int,
>  name string,
>  datekey string,
>  time_stamp string
> )
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'  
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,0:C1,0:C2,0:C3,0:C4")  
> TBLPROPERTIES ("hbase.table.name" = "BIDWD_TEST01");
> ---
> So,I also try to insert some data into the table,and scan this table:
> ---
> set hive.execution.engine=mr;
> insert into test01 values('003',3,'lisi2','20170215','2017-02-15 12:34:22');
> select * from test01;
> ---
> But,there are some problems like this:
> +++--+-+--+--+
> | test01.rk  | test01.id  | test01.name  | test01.datekey  |  
> test01.time_stamp   |
> +++--+-+--+--+
> | 001| NULL   | zhangsan | 20170217| 2017-02-17 
> 12:34:22  |
> | 002| NULL   | lisi | 20170216| 2017-02-16 
> 12:34:22  |
> | 003| 3  | lisi2| 20170215| 2017-02-15 
> 12:34:22  |
> +++--+-+--+--+
> the column "id" 's value was null,only the last row is ok.
> but,when I scan data in the phoenix ,there are some errors like this:
> Error: ERROR 201 (22000): Illegal data. Expected length of at least 115 
> bytes, but had 31 (state=22000,code=201)
> java.sql.SQLException: ERROR 201 (22000): Illegal data. Expected length of at 
> least 115 bytes, but had 31
>   at 
> org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:389)
>   at 
> org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145)
>   at 
> org.apache.phoenix.schema.KeyValueSchema.next(KeyValueSchema.java:211)
>   at 
> org.apache.phoenix.expression.ProjectedColumnExpression.evaluate(ProjectedColumnExpression.java:113)
>   at 
> org.apache.phoenix.compile.ExpressionProjector.getValue(ExpressionProjector.java:69)
>   at 
> org.apache.phoenix.jdbc.PhoenixResultSet.getString(PhoenixResultSet.java:591)
>   at sqlline.Rows$Row.(Rows.java:183)
>   at sqlline.B

Re: [VOTE] The 1st HBase 1.3.0 release candidate (RC0) is available

[Ashish Singhi]

+1

Built tar from the source code.
Verified some shell commands,
Ran export-import snapshot, copyTable, importtsv commands.
Verified replication, bulk loaded data replication, audit logs.
Verified proc-v2 enable table flow by killing master when table was in
ENABLING state in ZK during enable table procedure execution and once
master was restarted the table was in ENABLED state.

Regards,
Ashish

On Sat, Jan 7, 2017 at 2:12 AM, Mikhail Antonov 
wrote:

> Hello everyone,
>
> I'm pleased to announce the first release candidate for HBase 1.3.0 is
> available to download and testing.
>
> Artifacts are available here:
>
> https://dist.apache.org/repos/dist/dev/hbase/1.3.0RC0/
>
> Maven artifacts are available in the staging repository:
>
> https://repository.apache.org/content/repositories/orgapachehbase-1162/
>
> All artifacts are signed with my code signing key 35A4ABE2, which is also
> in the project KEYS file at
>
> http://www.apache.org/dist/hbase/KEYS
>
> these artifacts correspond to commit hash
>
> e359c76e8d9fd0d67396456f92bcbad9ecd7a710 tagged as 1.3.0RC0.
>
> HBase 1.3.0 is the third minor release in the HBase 1.x line and includes
> approximately 1700 resolved issues.
>
> Notable new features include:
>
>  - Date-based tiered compactions (HBASE-15181, HBASE-15339)
>  - Maven archetypes for HBase client applications (HBASE-14877)
>  - Throughput controller for flushes (HBASE-14969)
>  - Controlled delay (CoDel) based RPC scheduler (HBASE-15136)
>  - Bulk loaded HFile replication (HBASE-13153)
>  - More improvements to Procedure V2
>  - Improvements to Multi WAL (HBASE-14457)
>  - Many improvements and optimizations in metrics subsystem
>  - Reduced memory allocation in RPC layer (HBASE-15177)
>  - Region location lookups optimizations in HBase client
>
> and numerous bugfixes and performance improvements.
>
> The full list of issues addressed is available at
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12310753=12332794
>
> and also in the CHANGES.txt file in the root directory of the source
> tarball.
>
> Please take a few minutes to verify the release and vote on
> releasing it:
>
> [ ] +1 Release this package as Apache HBase 1.3.0
> [ ] +0 no opinion
> [ ] -1 Do not release this package because...
>
> This Vote will run for one week and close Fri Jan 13, 2017 22:00 PST.
>
> Thank you!
> Mikhail
>

[jira] [Resolved] (HBASE-17203) unable to see the namespaces and tables when grant read-only privleges

[Ashish Singhi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-17203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi resolved HBASE-17203.
---
Resolution: Invalid

This is correct.
To see to namespaces a user needs to have Admin privilege.
Please check https://hbase.apache.org/book.html#appendix_acl_matrix

> unable to see the namespaces and tables when grant read-only privleges
> --
>
> Key: HBASE-17203
> URL: https://issues.apache.org/jira/browse/HBASE-17203
> Project: HBase
>  Issue Type: Task
>  Components: hbase, shell
>Affects Versions: 1.0.0
>Reporter: sandeep vura
>
> Hi,
> i have created namespaces and given grant  read-only privileges. When i run 
> "list_namespace" on hbase shell i am unable to see the namespaces and also 
> the tables.
> But namespaces and tables are able to see in HUE --> HBase browser.
> Executed commands:
> create_namespace 'q1'
> grant '@hadooptest','R','@q1'
> Please advise.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-16493) MapReduce counters not updated with ScanMetrics

[Ashish Singhi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-16493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi resolved HBASE-16493.
---
Resolution: Duplicate

Duplicate of HBASE-16678

> MapReduce counters not updated with ScanMetrics 
> 
>
> Key: HBASE-16493
> URL: https://issues.apache.org/jira/browse/HBASE-16493
> Project: HBase
>  Issue Type: Bug
>  Components: mapreduce
>Affects Versions: 1.0.0, 2.0.0, 1.1.0
>Reporter: Jacobo Coll
>
> ScanMetrics were introduced in the [HBASE-4145]. These metrics were able to 
> work even in a parallel environment such as MapReduce.
> The TableRecordReader creates a Scanner with a copy of the given "scan", 
> called "currentScan". The ScanMetrics are captured by the Scanner and 
> modifies the given Scan instance, "currentScan". The TableRecordReader, after 
> reading the last value, updates the job counters to aggregate them. 
> But since [HBASE-13030], the TableRecordReader reads the scanMetrics from the 
> object "scan" instead of using the "currentScan"



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

RE: [ANNOUNCE] Duo Zhang (张铎) joins the Apache HBase PMC

[ashish singhi]

Congratulations!

-Original Message-
From: saint@gmail.com [mailto:saint@gmail.com] On Behalf Of Stack
Sent: 07 September 2016 09:56
To: HBase Dev List; Hbase-User
Subject: [ANNOUNCE] Duo Zhang (张铎) joins the Apache HBase PMC

On behalf of the Apache HBase PMC I am pleased to announce that 张铎
has accepted our invitation to become a PMC member on the Apache HBase project. 
Duo has healthy notions on where the project should be headed and over the last 
year and more has been working furiously to take us there.

Please join me in welcoming Duo to the HBase PMC!

One of us!
St.Ack

RE: [ANNOUNCE] Dima Spivak joins the Apache HBase PMC

[ashish singhi]

Congratulations Dima!

-Original Message-
From: Andrew Purtell [mailto:apurt...@apache.org] 
Sent: 01 September 2016 01:08
To: dev@hbase.apache.org; u...@hbase.apache.org
Subject: [ANNOUNCE] Dima Spivak joins the Apache HBase PMC

On behalf of the Apache HBase PMC I am pleased to announce that Dima Spivak has 
accepted our invitation to become a committer and PMC member on the Apache 
HBase project. Dima has been an active contributor for some time, particularly 
in development and contribution of release tooling that all of our RMs now use, 
such as the API compatibility checker. Dima has also been active in testing and 
voting on release candidates. Release voting is important to project health and 
momentum and demonstrates interest and capability above and beyond just 
committing. We wish to recognize this and make those release votes binding. 
Please join me in thanking Dima for his contributions to date and anticipation 
of many more contributions.

Welcome to the HBase project, Dima!

--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein (via 
Tom White)

RE: Hbase replucation

[ashish singhi]

Hi.

From HBase side, only table data is replicated once the table is enabled for 
replication. If any modification done later to table properties it will not be 
replicated automatically, client has to do it manually.

Regards,
Ashish

-Original Message-
From: darshit kandpal [mailto:dkandp...@gmail.com] 
Sent: 06 July 2016 01:29
To: James Taylor; dev@hbase.apache.org
Subject: Re: Hbase replucation

Hello James,
Thanks for the update. There was another issue:
Once the table has been created and replication has been enabled, if I do alter 
table in master cluster it should be replicated at the slave since the column 
family and table structure remains same at base level. However the phoenix 
table structure remains the same in the slave cluster. Is there a way to enable 
this automatically?

Thanks,
Darshit

On Sun, Jun 26, 2016 at 3:19 AM, James Taylor 
wrote:

> Hi Darshit,
> The way I've seen this dealt with in the past is really more through
> process: have a upgrade schema client that's responsible for running 
> DDL both on the primary and secondary cluster (and disable replication 
> while these commands are running to prevent any race conditions). More 
> details here [1].
> Thanks,
> James
>
> [1]
> http://mail-archives.apache.org/mod_mbox/phoenix-user/201606.mbox/%3CC
> AAF1JdggbF5YSCjHAnrjqTZU04dxLdNkCLvh-tpdeok0j853Gg%40mail.gmail.com%3E
>
>
> On Sun, Jun 26, 2016 at 12:14 AM, darshit kandpal 
> 
> wrote:
>
>> Hello folks,
>>
>> I was struggling with the problem of automatically syncing up the 
>> table schema of the phoenix tables as soon as any change is made or a 
>> new table is created because hbase replication works only if the two 
>> clusters have same tables.
>>
>>
>> Is there a trap based method for this which triggers the same table 
>> to be created in the other cluster as soon as it is created in the master 
>> cluster.
>>
>> Also when the waledits are shipped only the puts and deletes are 
>> replayed. Shouldnt replaying the creates and alters solve this 
>> problem permanently? If not i wanted to understand why not?
>>
>> Thanks for your time
>>
>> Darshit
>>
>>
>

[jira] [Created] (HBASE-15952) Bulk load data replication is not working when RS user does not have permission on hfile-refs node

[Ashish Singhi (JIRA)]

Ashish Singhi created HBASE-15952:
-

 Summary: Bulk load data replication is not working when RS user 
does not have permission on hfile-refs node
 Key: HBASE-15952
 URL: https://issues.apache.org/jira/browse/HBASE-15952
 Project: HBase
  Issue Type: Bug
  Components: Replication
Affects Versions: 1.3.0
Reporter: Ashish Singhi
Assignee: Ashish Singhi


In our recent testing in secure cluster we found that when a RS user does not 
have permission on hfile-refs znode then RS was failing to replicate the bulk 
loaded data as the hfile-refs znode is created by hbase client and RS user may 
not have permission to it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

RE: [ANNOUNCE] Mikhail Antonov joins the Apache HBase PMC

[ashish singhi]

Congratulations!

Regards,
Ashish

-Original Message-
From: Andrew Purtell [mailto:apurt...@apache.org] 
Sent: 27 May 2016 00:00
To: dev@hbase.apache.org
Cc: u...@hbase.apache.org
Subject: [ANNOUNCE] Mikhail Antonov joins the Apache HBase PMC

On behalf of the Apache HBase PMC I am pleased to announce that Mikhail Antonov 
has accepted our invitation to become a PMC member on the Apache HBase project. 
Mikhail has been an active contributor in many areas, including recently taking 
on the Release Manager role for the upcoming 1.3.x code line. Please join me in 
thanking Mikhail for his contributions to date and anticipation of many more 
contributions.

Welcome to the PMC, Mikhail!

--
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein (via 
Tom White)

[jira] [Created] (HBASE-15888) Extend HBASE-12769 for bulk load data replication

[Ashish Singhi (JIRA)]

Ashish Singhi created HBASE-15888:
-

 Summary: Extend HBASE-12769 for bulk load data replication
 Key: HBASE-15888
 URL: https://issues.apache.org/jira/browse/HBASE-15888
 Project: HBase
  Issue Type: Task
  Components: Replication
Affects Versions: 1.3.0
Reporter: Ashish Singhi
Assignee: Ashish Singhi


Include hfile-refs queue check and fix also in hbck -fixReplication command



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-7065) alter table with multiple actions should do just one schema update (shell)

[Ashish Singhi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-7065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi resolved HBASE-7065.
--
Resolution: Duplicate

Handled as part of HBASE-15641

> alter table with multiple actions should do just one schema update (shell)
> --
>
> Key: HBASE-7065
> URL: https://issues.apache.org/jira/browse/HBASE-7065
> Project: HBase
>  Issue Type: Improvement
>  Components: shell
>Affects Versions: 0.95.2
>Reporter: Sergey Shelukhin
>Priority: Minor
>
> In the shell, when one runs alter command with multiple things to do (e.g. 
> "alter 't1', 'f1', { NAME => 'f2', VERSIONS => 5 }, { 'delete' => 'f3' }, 
> MAX_FILESIZE => 1232353") the shell does the schema update after each action.
> I'd assume it takes a long time on large clusters, and can also be unexpected 
> from the logical perspective, especially if it fails in the middle for 
> example.
> Should it do one schema update instead?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-8039) Make HDFS replication number configurable for a column family

[Ashish Singhi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-8039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi resolved HBASE-8039.
--
   Resolution: Duplicate
Fix Version/s: (was: 2.0.0)

Implemented as part of HBASE-14154

> Make HDFS replication number configurable for a column family
> -
>
> Key: HBASE-8039
> URL: https://issues.apache.org/jira/browse/HBASE-8039
> Project: HBase
>  Issue Type: Improvement
>  Components: HFile
>Reporter: Maryann Xue
>Priority: Minor
>
> To allow users to decide which column family's data is more important and 
> which is less important by specifying a replica number instead of using the 
> default replica number.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

RE: Please welcome new HBase committer Ashish Singhi

[ashish singhi]

Thanks everyone for your kind words. 
Hopefully I can repay you all with more patch contributions and reviews.

Regards,
Ashish

-Original Message-
From: saint@gmail.com [mailto:saint@gmail.com] On Behalf Of Stack
Sent: 08 April 2016 10:53
To: HBase Dev List
Subject: Please welcome new HBase committer Ashish Singhi

Ashish has contributed loads of stuff starting out basic doing rakes of simple 
fixes but then ramping up to take on the hard stuff going out of his way to 
land the 'best' fix. He's been doing great work. Keep it up Ashish!

St.Ack

[jira] [Created] (HBASE-15578) Handle HBASE-15234 for ReplicationHFileCleaner

[Ashish Singhi (JIRA)]

Ashish Singhi created HBASE-15578:
-

 Summary: Handle HBASE-15234 for ReplicationHFileCleaner
 Key: HBASE-15578
 URL: https://issues.apache.org/jira/browse/HBASE-15578
 Project: HBase
  Issue Type: Bug
  Components: Replication
Affects Versions: 1.3.0
Reporter: Ashish Singhi
Assignee: Ashish Singhi


HBASE-15234 is handled for ReplicationLogCleaner need to handle similarly for 
ReplicationHFileCleaner.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-11684) HBase replicationSource should support multithread to ship the log entry

[Ashish Singhi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-11684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi resolved HBASE-11684.
---
Resolution: Duplicate
  Assignee: (was: Tianying Chang)

Duplicate of HBASE-12988

> HBase replicationSource should support multithread to ship the log entry
> 
>
> Key: HBASE-11684
> URL: https://issues.apache.org/jira/browse/HBASE-11684
> Project: HBase
>  Issue Type: Improvement
>  Components: Performance, Replication
>Reporter: Tianying Chang
>
> We found the replication rate cannot keep up with the write rate when the 
> master cluster is write heavy. We got huge log queue build up due to that. 
> But when we do a rolling restart of master cluster, we found that the 
> appliedOpsRate doubled due to the extra thread created to help recover the 
> log of the restarted RS. ReplicateLogEntries is a synchronous blocking call, 
> it becomes the bottleneck when is only runs with one thread. I think we 
> should support multi-thread for the replication source to ship the data. I 
> don't see any consistency problem. Any other concern here? 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [ANNOUNCE] New HBase committer Yu Li

[Ashish Singhi]

Many Congratulations!

On Thu, Mar 17, 2016 at 7:19 AM, Nick Dimiduk  wrote:

> On behalf of the Apache HBase PMC, I am pleased to announce that Yu Li
> has accepted the PMC's invitation to become a committer on the
> project. We appreciate all of Yu's generous contributions thus far and
> look forward to his continued involvement.
>
> Congratulations and welcome, Yu!
>
> -n
>

[jira] [Created] (HBASE-15446) Do not abort RS when WAL append for bulk load event marker is failed

[Ashish Singhi (JIRA)]

Ashish Singhi created HBASE-15446:
-

 Summary: Do not abort RS when WAL append for bulk load event 
marker is failed
 Key: HBASE-15446
 URL: https://issues.apache.org/jira/browse/HBASE-15446
 Project: HBase
  Issue Type: Bug
Affects Versions: 1.3.0
Reporter: Ashish Singhi
Assignee: Ashish Singhi


During bulk load process when the RS fails to append the bulk load event marker 
in WAL we abort that RS which is actually not really required. Instead just 
throw back the exception to the client and let the client retry. This will be 
helpful in case of replication of bulk loaded data.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-15425) Failing to write bulk load event marker in WAL is ignored

[Ashish Singhi (JIRA)]

Ashish Singhi created HBASE-15425:
-

 Summary: Failing to write bulk load event marker in WAL is ignored
 Key: HBASE-15425
 URL: https://issues.apache.org/jira/browse/HBASE-15425
 Project: HBase
  Issue Type: Bug
Affects Versions: 1.3.0
Reporter: Ashish Singhi
Assignee: Ashish Singhi


During LoadIncrementalHFiles process if we fail to write the bulk load event 
marker in the WAL it is ignored. So this will lead to data mismatch issue in 
source and peer cluster in case of bulk loaded data replication scenario.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-15424) Add bulk load hfile-refs for replication in ZK after the event is appended in the WAL

[Ashish Singhi (JIRA)]

Ashish Singhi created HBASE-15424:
-

 Summary: Add bulk load hfile-refs for replication in ZK after the 
event is appended in the WAL
 Key: HBASE-15424
 URL: https://issues.apache.org/jira/browse/HBASE-15424
 Project: HBase
  Issue Type: Bug
Affects Versions: 2.0.0, 1.3.0, 1.4.0
Reporter: Ashish Singhi
Assignee: Ashish Singhi
Priority: Minor


Currenlty hfile-refs znode used for tracking the bulk loaded data replication 
is added first and then the bulk load event in appended in the WAL. So this may 
lead to a issue where the znode is added in ZK but append to WAL is failed(due 
to some probelm in DN), so this znode will be left in ZK as it is and will not 
allow hfile to get deleted from archive directory. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-15397) Create bulk load replication znode(hfile-refs) in ZK replication queue by default

[Ashish Singhi (JIRA)]

Ashish Singhi created HBASE-15397:
-

 Summary: Create bulk load replication znode(hfile-refs) in ZK 
replication queue by default
 Key: HBASE-15397
 URL: https://issues.apache.org/jira/browse/HBASE-15397
 Project: HBase
  Issue Type: Bug
Affects Versions: 1.3.0
Reporter: Ashish Singhi
Assignee: Ashish Singhi


Create bulk load replication znode(hfile-refs) in ZK replication queue by 
default same as hbase replication znode. 
Otherwise the problem what happens is currently replication admin directly 
operates on ZK without routing through HM/RS. So suppose if a user enables the 
replication for bulk loaded data in server but fails to do the same in the 
client configurations then add peer will not add hfile-refs znode, resulting in 
replication failure for bulk loaded data.
So after fixing this the behavior will be same as mutation replication.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-15393) Enable table replication command will fail when parent znode is not /hbase(default) in peer cluster

[Ashish Singhi (JIRA)]

Ashish Singhi created HBASE-15393:
-

 Summary: Enable table replication command will fail when parent 
znode is not /hbase(default) in peer cluster
 Key: HBASE-15393
 URL: https://issues.apache.org/jira/browse/HBASE-15393
 Project: HBase
  Issue Type: Bug
  Components: Replication
Affects Versions: 0.98.17, 1.0.3
Reporter: Ashish Singhi
Assignee: Ashish Singhi


Enable table replication command will fail when parent znode is not 
/hbase(default) in peer cluster and there is only one peer cluster added in the 
source cluster.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-15313) Error compiling hbase with Java8 and Hadoop2

[Ashish Singhi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi resolved HBASE-15313.
---
Resolution: Invalid

> Error compiling hbase with Java8 and Hadoop2
> 
>
> Key: HBASE-15313
> URL: https://issues.apache.org/jira/browse/HBASE-15313
> Project: HBase
>  Issue Type: Bug
>  Components: API
>Affects Versions: 0.98.8
> Environment: Apache Maven 3.3.9
> Maven home: /home/c1/apache-maven-3.3.9
> Java version: jdk1.8.0_25, vendor: Oracle Corporation
> Java home: /usr/lib/jvm/jdk1.8.0_25/jre
> Default locale: en_IN, platform encoding: UTF-8
> OS name: "linux/centos", version: "2.6.32-573.12.1.el6.x86_64", arch: 
> "amd64", family: "unix"
> Hbase : hbase-0.98.8-hadoop2
> Hadoop: 2.5.2
>Reporter: Kshitij Shukla
>Priority: Minor
>  Labels: maven
>
> Getting this error while compiling hbase-0.98.8 using this command *mvn 
> package -DskipTests"* with JAVA8.
> {quote} PoolMap.java error: name clash: remove(K,V) in PoolMap and 
> remove(Object,Object) in Map have the same erasure, yet neither overrides the 
> other {quote}
> However its working when tried Java7. Sorry If I sound less informative about 
> error. I am a newbie and not very familiar with the issue posting practice :) 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-15254) Enhance Kerberos name support for cross realm HBase replication

[Ashish Singhi (JIRA)]

Ashish Singhi created HBASE-15254:
-

 Summary: Enhance Kerberos name support for cross realm HBase 
replication
 Key: HBASE-15254
 URL: https://issues.apache.org/jira/browse/HBASE-15254
 Project: HBase
  Issue Type: Improvement
Reporter: Ashish Singhi
Assignee: Ashish Singhi


HBase replication will not work with Kerberos cross realm trust when domain 
name in the principal is not hostname. 
A mail was also sent related to this in user mailing list, [mail | 
https://groups.google.com/forum/#!topic/nosql-databases/AYhQnU9Fc7E]

The problem here is when ever a user adds a new host to cluster he/she also 
needs to add a principal name for that host in KDC, generate a new keytab file 
and update it across other hosts accordingly if required. 
To save all this efforts users may prefer to have a fixed domain name in the 
principal for all the hosts and in that case HBase replication will fail.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-15230) ReplicationSource is replicating existing WAL data in a newly added peer

[Ashish Singhi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-15230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi resolved HBASE-15230.
---
Resolution: Duplicate
  Assignee: (was: Ashish Singhi)

Duplicate of HBASE-9888.

> ReplicationSource is replicating existing WAL data in a newly added peer
> 
>
> Key: HBASE-15230
> URL: https://issues.apache.org/jira/browse/HBASE-15230
> Project: HBase
>  Issue Type: Bug
>    Reporter: Ashish Singhi
>
> Scenario:
> 1. Add a peer 'p1' and enable table replication for a table 't1'
> 2. Put some data in the table 't1' and its gets replicated to peer 'p1' as 
> expected.
> 3. Remove peer 'p1' and truncate the table 't1' in both source and peer 
> cluster.
> 4. Now add peer 'p2' , there is no data in source cluster in table 't1' but 
> in peer cluster 'p2' where table 't1' already exists, existing WAL data of 
> table 't1' is getting replicated in 'p2'.
> Expectation: Table 't1' in peer cluster 'p2' should only have data which is 
> inserted in source cluster table 't1' after adding peer 'p2'



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-15230) ReplicationSource is replicating existing WAL data in a newly added peer

[Ashish Singhi (JIRA)]

Ashish Singhi created HBASE-15230:
-

 Summary: ReplicationSource is replicating existing WAL data in a 
newly added peer
 Key: HBASE-15230
 URL: https://issues.apache.org/jira/browse/HBASE-15230
 Project: HBase
  Issue Type: Bug
Reporter: Ashish Singhi
Assignee: Ashish Singhi


Scenario:
1. Add a peer 'p1' and enable table replication for a table 't1'
2. Put some data in the table 't1' and its gets replicated to peer 'p1' as 
expected.
3. Remove peer 'p1' and truncate the table 't1' in both source and peer cluster.
4. Now add peer 'p2' , there is no data in source cluster in table 't1' but in 
peer cluster 'p2' where table 't1' already exists, existing WAL data of table 
't1' is getting replicated in 'p2'.

Expectation: Table 't1' in peer cluster 'p2' should only have data which is 
inserted in source cluster table 't1' after adding peer 'p2'



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (HBASE-14979) Update to the newest Zookeeper release

[Ashish Singhi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi reopened HBASE-14979:
---

Reopening the issue as the code change was reverted.

> Update to the newest Zookeeper release
> --
>
> Key: HBASE-14979
> URL: https://issues.apache.org/jira/browse/HBASE-14979
> Project: HBase
>  Issue Type: Improvement
>  Components: Zookeeper
>Affects Versions: 1.2.0
>Reporter: Elliott Clark
>Assignee: Elliott Clark
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-14979.patch
>
>
> ZOOKEEPER-706 is nice to have for anyone running replication that sometimes 
> gets stalled. We should update to the latest patch version.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (HBASE-9393) Hbase does not closing a closed socket resulting in many CLOSE_WAIT

[Ashish Singhi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-9393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi reopened HBASE-9393:
--

This is a issue we need to fix it, so reopening it.

> Hbase does not closing a closed socket resulting in many CLOSE_WAIT 
> 
>
> Key: HBASE-9393
> URL: https://issues.apache.org/jira/browse/HBASE-9393
> Project: HBase
>  Issue Type: Bug
>Affects Versions: 0.94.2, 0.98.0
> Environment: Centos 6.4 - 7 regionservers/datanodes, 8 TB per node, 
> 7279 regions
>Reporter: Avi Zrachya
>Assignee: Ashish Singhi
>
> HBase dose not close a dead connection with the datanode.
> This resulting in over 60K CLOSE_WAIT and at some point HBase can not connect 
> to the datanode because too many mapped sockets from one host to another on 
> the same port.
> The example below is with low CLOSE_WAIT count because we had to restart 
> hbase to solve the porblem, later in time it will incease to 60-100K sockets 
> on CLOSE_WAIT
> [root@hd2-region3 ~]# netstat -nap |grep CLOSE_WAIT |grep 21592 |wc -l
> 13156
> [root@hd2-region3 ~]# ps -ef |grep 21592
> root 17255 17219  0 12:26 pts/000:00:00 grep 21592
> hbase21592 1 17 Aug29 ?03:29:06 
> /usr/java/jdk1.6.0_26/bin/java -XX:OnOutOfMemoryError=kill -9 %p -Xmx8000m 
> -ea -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode 
> -Dhbase.log.dir=/var/log/hbase 
> -Dhbase.log.file=hbase-hbase-regionserver-hd2-region3.swnet.corp.log ...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-15018) Inconsistent way of handling TimeoutException in the rpc client implemenations

[Ashish Singhi (JIRA)]

Ashish Singhi created HBASE-15018:
-

 Summary: Inconsistent way of handling TimeoutException in the rpc 
client implemenations
 Key: HBASE-15018
 URL: https://issues.apache.org/jira/browse/HBASE-15018
 Project: HBase
  Issue Type: Bug
Affects Versions: 1.1.0, 2.0.0, 1.2.0
Reporter: Ashish Singhi
Assignee: Ashish Singhi


If there is any rpc timeout when using RpcClientImpl then we wrap the exception 
in IOE and throw it,
{noformat}
2015-11-16 04:05:24,935 WARN [main-EventThread.replicationSource,peer2] 
regionserver.HBaseInterClusterReplicationEndpoint: Can't replicate because of a 
local or network error:
java.io.IOException: Call to host-XX:16040 failed on local exception: 
org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=510, waitTime=180001, 
operationTimeout=18 expired.
at 
org.apache.hadoop.hbase.ipc.RpcClientImpl.wrapException(RpcClientImpl.java:1271)
at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1239)
at 
org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213)
at 
org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287)
at 
org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$BlockingStub.replicateWALEntry(AdminProtos.java:25690)
at 
org.apache.hadoop.hbase.protobuf.ReplicationProtbufUtil.replicateWALEntry(ReplicationProtbufUtil.java:77)
at 
org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint$Replicator.call(HBaseInterClusterReplicationEndpoint.java:322)
at 
org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint$Replicator.call(HBaseInterClusterReplicationEndpoint.java:308)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=510, 
waitTime=180001, operationTimeout=18 expired.
at org.apache.hadoop.hbase.ipc.Call.checkAndSetTimeout(Call.java:70)
at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1213)
... 10 more
{noformat}
But that isn't case with AsyncRpcClient, we don't wrap and throw 
CallTimeoutException as it is.
{noformat}
2015-12-21 14:27:33,093 WARN  
[RS_OPEN_REGION-host-XX:16201-0.replicationSource.host-XX%2C16201%2C1450687255593,1]
 regionserver.HBaseInterClusterReplicationEndpoint: Can't replicate because of 
a local or network error: 
org.apache.hadoop.hbase.ipc.CallTimeoutException: callId=2, 
method=ReplicateWALEntry, rpcTimeout=60, param {TODO: class 
org.apache.hadoop.hbase.protobuf.generated.AdminProtos$ReplicateWALEntryRequest}
at 
org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:257)
at 
org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217)
at 
org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:295)
at 
org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$BlockingStub.replicateWALEntry(AdminProtos.java:23707)
at 
org.apache.hadoop.hbase.protobuf.ReplicationProtbufUtil.replicateWALEntry(ReplicationProtbufUtil.java:73)
at 
org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint$Replicator.call(HBaseInterClusterReplicationEndpoint.java:387)
at 
org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint$Replicator.call(HBaseInterClusterReplicationEndpoint.java:370)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{noformat}

I think we should have same behavior across both the implementations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-14957) issue in starting start-hbase.sh

[Ashish Singhi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi resolved HBASE-14957.
---
   Resolution: Invalid
Fix Version/s: (was: 0.94.9)

> issue in starting start-hbase.sh
> 
>
> Key: HBASE-14957
> URL: https://issues.apache.org/jira/browse/HBASE-14957
> Project: HBase
>  Issue Type: Bug
>  Components: build
>Affects Versions: 0.94.9
>Reporter: gaurav kandpal
>Priority: Blocker
>   Original Estimate: 12h
>  Remaining Estimate: 12h
>
> for initializing nutch, after configuring nutch whent I am trying to kickoff 
> start-hbase.sh I am getting the below error.
> /cygdrive/c/users/gaurav.kandpal/desktop/test/hbase-0.94.9.tar/hbase-0.94.9/hbase-0.94.9/bin/../conf/hbase-env.sh:
>  line 101: $'\r': command not found
> /cygdrive/c/users/gaurav.kandpal/desktop/test/hbase-0.94.9.tar/hbase-0.94.9/hbase-0.94.9/bin/../conf/hbase-env.sh:
>  line 104: $'\r': command not found
> /cygdrive/c/users/gaurav.kandpal/desktop/test/hbase-0.94.9.tar/hbase-0.94.9/hbase-0.94.9/bin/../conf/hbase-env.sh:
>  line 107: $'\r': command not found
> /cygdrive/c/users/gaurav.kandpal/desktop/test/hbase-0.94.9.tar/hbase-0.94.9/hbase-0.94.9/bin/../conf/hbase-env.sh:
>  line 110: $'\r': command not found
> /cygdrive/c/users/gaurav.kandpal/desktop/test/hbase-0.94.9.tar/hbase-0.94.9/hbase-0.94.9/bin/../conf/hbase-env.sh:
>  line 115: $'\r': command not found
> Error: Could not create the Java Virtual Machine.
> Error: A fatal exception has occurred. Program will exit.
> 'nrecognized VM option 'UseConcMarkSweepGC
> Error: Could not create the Java Virtual Machine.
> Error: A fatal exception has occurred. Program will exit.
> 'nrecognized VM option 'UseConcMarkSweepGC
> starting master, logging to 
> /cygdrive/c/users/gaurav.kandpal/desktop/test/hbase-0.94.9.tar/hbase-0.94.9/hbase-0.94.9/bin/../logs/hbase-Gaurav.Kandpal-master-gauravk.out
> localhost: 
> /cygdrive/c/users/gaurav.kandpal/desktop/test/hbase-0.94.9.tar/hbase-0.94.9/hbase-0.94.9/bin/regionservers.sh:
>  line 64: ssh: command not found
> reference URL is 
> https://gist.github.com/xrstf/b48a970098a8e76943b9
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-14937) Make rpc call timeout for replication adaptive

[Ashish Singhi (JIRA)]

Ashish Singhi created HBASE-14937:
-

 Summary: Make rpc call timeout for replication adaptive
 Key: HBASE-14937
 URL: https://issues.apache.org/jira/browse/HBASE-14937
 Project: HBase
  Issue Type: Improvement
Reporter: Ashish Singhi
Assignee: Ashish Singhi


When peer cluster replication is disabled and lot of writes are happening in 
active cluster and later on peer cluster replication is enabled then there are 
chances that replication requests to peer cluster may time out.
This is possible after HBASE-13153 and it can also happen with many and many 
WAL data replication still pending to replicate.

Approach to this problem will be discussed in the comments.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-14938) Limit to and fro requests size from ZK in bulk loaded hfile replication

[Ashish Singhi (JIRA)]

Ashish Singhi created HBASE-14938:
-

 Summary: Limit to and fro requests size from ZK in bulk loaded 
hfile replication
 Key: HBASE-14938
 URL: https://issues.apache.org/jira/browse/HBASE-14938
 Project: HBase
  Issue Type: Improvement
Reporter: Ashish Singhi
Assignee: Ashish Singhi


In ZK the maximum allowable size of the data array is 1 MB. Until we have fixed 
HBASE-10295 we need to handle this.
Approach to this problem will be discussed in the comments section.

Note: We have done internal testing with more than 3k nodes in ZK yet to be 
replicated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-14939) Document bulk loaded hfile replication

[Ashish Singhi (JIRA)]

Ashish Singhi created HBASE-14939:
-

 Summary: Document bulk loaded hfile replication
 Key: HBASE-14939
 URL: https://issues.apache.org/jira/browse/HBASE-14939
 Project: HBase
  Issue Type: Task
  Components: documentation
Reporter: Ashish Singhi
Assignee: Ashish Singhi


After HBASE-13153 is committed we need to add that information under the 
Cluster Replication section in HBase book.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

RE: Consesus on removing a public method from IA.Private class whose instance is returned from IA.public class.

[ashish singhi]

Sorry for the late response. I was on vacation.

St.Ack: 
Class 'A' is HBaseAdmin and class 'B' is HBaseTestingUtility.
I was thinking this from a general perspective, irrespective of the classes and 
can note it down in our book and avoid this kind of discussions in dev mailing 
list in future. But seems like we have agreed to take up the call based on the 
particulars.

Regards,
Ashish Singhi

-Original Message-
From: saint@gmail.com [mailto:saint@gmail.com] On Behalf Of Stack
Sent: 03 December 2015 06:07
To: HBase Dev List
Subject: Re: Consesus on removing a public method from IA.Private class whose 
instance is returned from IA.public class.

Appy makes a pretty good argument.

What in particular are we discussing Ashish?

Thanks,
St.Ack

On Wed, Dec 2, 2015 at 3:49 PM, Apekshit Sharma <a...@cloudera.com> wrote:

> I think it should be okay to remove them. When I imagine myself on 
> other side, a dev using client api of Foo project, and I see a class 
> returning reference to an internal class, I would not trust that 
> function. If I really need something from returned ref, I'd probably 
> use it, and if it goes away tomorrow, i'd probably curse it too, but I 
> can't complain since it's not anything I trusted to being with.
>
> Now thinking as dev:
> Given that documentation explicitly states that functions can 
> disappear from private class, and nothing about transitiveness, I 
> believe we can remove functions from private class A.
>
>
> On Thu, Nov 26, 2015 at 7:10 AM, Ted Yu <yuzhih...@gmail.com> wrote:
>
> > bq. we should not remove it directly
> >
> > My understanding is the same.
> >
> > Cheers
> >
> > On Wed, Nov 25, 2015 at 10:22 PM, ashish singhi <
> ashish.sin...@huawei.com>
> > wrote:
> >
> > > Hello to all.
> > >
> > > I know that we can safely remove any APIs from a
> > InterfaceAudience.Private
> > > class and the same is described in the HBase book.
> > >
> > > Now consider a case (I did not find this mentioned in our semvar), 
> > > There is a class 'A' with InterfaceAudience.Private and class 'B' 
> > > with InterfaceAudience.Public and then there is a public API in class 'B'
> > which
> > > returns an instance of class 'A'. (We missed to mark the public 
> > > method
> in
> > > class 'B' as deprecated when we marked the class 'A' as 
> > > InterfaceAudience.Private).
> > >
> > > Now the question is, can we remove the public APIs from class 'A'
> without
> > > marking them deprecated ?
> > >
> > > I think we should not remove it directly, it will break the users 
> > > using
> > > these(removed) public methods from class 'A'.
> > > P.S: This point came up  in HBASE-14769.
> > >
> > > Regards,
> > > Ashish Singhi
> > >
> >
>
>
>
> --
>
> Regards
>
> Apekshit Sharma | Software Engineer, Cloudera | Palo Alto, California 
> |
> 650-963-6311
>

Consesus on removing a public method from IA.Private class whose instance is returned from IA.public class.

[ashish singhi]

Hello to all.

I know that we can safely remove any APIs from a InterfaceAudience.Private 
class and the same is described in the HBase book.

Now consider a case (I did not find this mentioned in our semvar),
There is a class 'A' with InterfaceAudience.Private and class 'B' with 
InterfaceAudience.Public and then there is a public API in class 'B' which 
returns an instance of class 'A'. (We missed to mark the public method in class 
'B' as deprecated when we marked the class 'A' as InterfaceAudience.Private).

Now the question is, can we remove the public APIs from class 'A' without 
marking them deprecated ?

I think we should not remove it directly, it will break the users using 
these(removed) public methods from class 'A'.
P.S: This point came up  in HBASE-14769.

Regards,
Ashish Singhi

[jira] [Created] (HBASE-14685) Procedure Id is not set for MasterRpcServices#modifyTable

[Ashish Singhi (JIRA)]

Ashish Singhi created HBASE-14685:
-

 Summary: Procedure Id is not set for  MasterRpcServices#modifyTable
 Key: HBASE-14685
 URL: https://issues.apache.org/jira/browse/HBASE-14685
 Project: HBase
  Issue Type: Sub-task
Affects Versions: 2.0.0
Reporter: Ashish Singhi
 Fix For: 2.0.0


Procedure Id is not set for  MasterRpcServices#modifyTable



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-14491) ReplicationSource#countDistinctRowKeys code logic is not correct

[Ashish Singhi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi resolved HBASE-14491.
---
Resolution: Fixed
  Assignee: (was: Ashish Singhi)

Fixed by Enis as part of HBASE-14501.

> ReplicationSource#countDistinctRowKeys code logic is not correct
> 
>
> Key: HBASE-14491
> URL: https://issues.apache.org/jira/browse/HBASE-14491
> Project: HBase
>  Issue Type: Bug
>    Reporter: Ashish Singhi
>Priority: Minor
>
> {code}
>   Cell lastCell = cells.get(0);
>   for (int i = 0; i < edit.size(); i++) {
> if (!CellUtil.matchingRow(cells.get(i), lastCell)) {
>   distinctRowKeys++;
> }
>   }
> {code}
> The above logic for finding the distinct row keys in the list needs to be 
> corrected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-14491) ReplicationSource#countDistinctRowKeys code logic is not correct

[Ashish Singhi (JIRA)]

Ashish Singhi created HBASE-14491:
-

 Summary: ReplicationSource#countDistinctRowKeys code logic is not 
correct
 Key: HBASE-14491
 URL: https://issues.apache.org/jira/browse/HBASE-14491
 Project: HBase
  Issue Type: Bug
Reporter: Ashish Singhi
Assignee: Ashish Singhi
Priority: Minor


{code}
  Cell lastCell = cells.get(0);
  for (int i = 0; i < edit.size(); i++) {
if (!CellUtil.matchingRow(cells.get(i), lastCell)) {
  distinctRowKeys++;
}
  }
{code}
The above logic for finding the distinct row keys in the list needs to be 
corrected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-14419) Error: Could not find or load main class org.apache.hadoop.hbase.util.HBaseConfTool

[Ashish Singhi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi resolved HBASE-14419.
---
Resolution: Invalid

Please write in to u...@hbase.apache.org to help troubleshooting issues.
Jira is for the project development tracker.

> Error: Could not find or load main class 
> org.apache.hadoop.hbase.util.HBaseConfTool
> ---
>
> Key: HBASE-14419
> URL: https://issues.apache.org/jira/browse/HBASE-14419
> Project: HBase
>  Issue Type: Bug
> Environment: centos7
>Reporter: chenlong
>Priority: Critical
>
> [root@localhost local]# ll
> lrwxrwxrwx.  1 root root   11 Sep 12 21:34 hbase -> hbase-1.1.2
> drwxr-xr-x. 30 root root 4096 Sep 12 21:34 hbase-1.1.2
> [root@localhost local]# ./hbase/bin/start-hbase.sh
> Error: Could not find or load main class 
> org.apache.hadoop.hbase.util.HBaseConfTool
> Error: Could not find or load main class 
> org.apache.hadoop.hbase.zookeeper.ZKServerTool
> starting master, logging to 
> /usr/local/hbase/logs/hbase-root-master-localhost.out
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; 
> support was removed in 8.0
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; 
> support was removed in 8.0
> Error: Could not find or load main class 
> org.apache.hadoop.hbase.master.HMaster
> starting regionserver, logging to 
> /usr/local/hbase/logs/hbase-root-1-regionserver-localhost.out
> Error: Could not find or load main class 
> org.apache.hadoop.hbase.regionserver.HRegionServer
> why show this error ?
> It exist 。
> [root@localhost local]# find ./ -name HBaseConfTool.class
> ./hbase-1.1.2/hbase-server/target/classes/org/apache/hadoop/hbase/util/HBaseConfTool.class
> /etc/profle:
> export JAVA_HOME=/usr/local/jdk1.8.0_20
> export HBASE_HOME=/usr/local/hbase
> export PATH=$JAVA_HOME/bin:$HBASE_HOME/bin:$PATH
> export 
> CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$HBASE_HOME/hbase-server/target/classes
> I add  $HBASE_HOME/hbase-server/target/classes, but it still not find class 
> file。version 0.98xx has no this problem，why the version 1.1.2  so eggache？ I 
> am just a newer，geting start follow official docs, but  can not run。I am so 
> sad。。。sos。。。



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (HBASE-14396) audit log should record the operation object

[Ashish Singhi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi reopened HBASE-14396:
---

> audit log should  record the operation object
> -
>
> Key: HBASE-14396
> URL: https://issues.apache.org/jira/browse/HBASE-14396
> Project: HBase
>  Issue Type: Bug
>Reporter: sunhaitao
>
> Currently the hbase audit log only records the user and scope,we can't know 
> which table the user is operating on unless the scope is table.
> It would be better to know what's going on if we record the exact table and 
> column family we are operating on besides the scope.
>   String logMessage =
> "Access " + (result.isAllowed() ? "allowed" : "denied") + " for user "
> + (result.getUser() != null ? result.getUser().getShortName() : 
> "UNKNOWN")
> + "; reason: " + result.getReason() + "; remote address: "
> + (remoteAddr != null ? remoteAddr : "") + "; request: " + 
> result.getRequest()
> + "; context: " + result.toContextString();



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-14396) audit log should record the operation object

[Ashish Singhi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Singhi resolved HBASE-14396.
---
Resolution: Not A Problem

> audit log should  record the operation object
> -
>
> Key: HBASE-14396
> URL: https://issues.apache.org/jira/browse/HBASE-14396
> Project: HBase
>  Issue Type: Bug
>Reporter: sunhaitao
>
> Currently the hbase audit log only records the user and scope,we can't know 
> which table the user is operating on unless the scope is table.
> It would be better to know what's going on if we record the exact table and 
> column family we are operating on besides the scope.
>   String logMessage =
> "Access " + (result.isAllowed() ? "allowed" : "denied") + " for user "
> + (result.getUser() != null ? result.getUser().getShortName() : 
> "UNKNOWN")
> + "; reason: " + result.getReason() + "; remote address: "
> + (remoteAddr != null ? remoteAddr : "") + "; request: " + 
> result.getRequest()
> + "; context: " + result.toContextString();



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Please welcome new HBase committer Stephen Yuan Jiang

[Ashish Singhi]

Congratulations Stephen!

On Fri, Aug 21, 2015 at 7:39 AM, Andrew Purtell apurt...@apache.org wrote:

 On behalf of the Apache HBase PMC, I am pleased to announce that Stephen
 Jiang has accepted the PMC's invitation to become a committer on the
 project. We appreciate all of Stephen's hard work and generous
 contributions thus far, and look forward to his continued involvement.

 Congratulations and welcome, Stephen!

 --
 Best regards,

- Andy

 Problems worthy of attack prove their worth by hitting back. - Piet Hein
 (via Tom White)

RE: Comments in HBase shell

[ashish singhi]

What is the problem you see when the prompt ends with '*' ?

Regards,
Ashish Singhi

-Original Message-
From: dinusw...@gmail.com [mailto:dinusw...@gmail.com] 
Sent: 18 August 2015 12:57
To: dev@hbase.apache.org
Subject: Re: Comments in HBase shell

Hi Ashish,


I tried; But it results with incorrect prompt; 


Instead of ‘’ I got ‘*’ in prompt; Is this the behaviour or issue in HBase 
shell?


hbase(main):016:0 # comment
hbase(main):017:0* list
TABLE




FYI: I am using Windows OS;





Sent from Windows Mail





From: ashish singhi
Sent: ‎Tuesday‎, ‎18‎ ‎August‎ ‎2015 ‎11‎:‎57‎ ‎AM
To: dev@hbase.apache.org





Hi,

Did you try '#' ?

Regards,
Ashish Singhi

-Original Message-
From: dinusw...@gmail.com [mailto:dinusw...@gmail.com] 
Sent: 18 August 2015 11:43
To: dev@hbase.apache.org
Subject: Comments in HBase shell

Hi,


I would like execute a comment line in HBase shell; But I couldn't find the 
syntax for the same; Can any one please tell how can I execute a ‘comment’ in 
HBase shell?


For reference:

Hive CLI comment: 
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Cli#LanguageManualCli-HiveInteractiveShellCommands



hive--some comment;





Thanks,

Dinesh Kumar P




Sent from Windows Mail

RE: Comments in HBase shell

[ashish singhi]

So to conclude this, ending with '*' is ok. It is not a issue.

Regards,
Ashish Singhi

-Original Message-
From: dinusw...@gmail.com [mailto:dinusw...@gmail.com] 
Sent: 18 August 2015 14:12
To: dev@hbase.apache.org
Subject: Re: Comments in HBase shell

I don't see any problem when the prompt ends with ‘*’ 


The next command ‘list’ worked fine and it ended with correct prompt 
‘hbase(main):016:0’


Sent from Windows Mail





From: ashish singhi
Sent: ‎Tuesday‎, ‎18‎ ‎August‎ ‎2015 ‎01‎:‎26‎ ‎PM
To: dev@hbase.apache.org





What is the problem you see when the prompt ends with '*' ?

Regards,
Ashish Singhi

-Original Message-
From: dinusw...@gmail.com [mailto:dinusw...@gmail.com] 
Sent: 18 August 2015 12:57
To: dev@hbase.apache.org
Subject: Re: Comments in HBase shell

Hi Ashish,


I tried; But it results with incorrect prompt; 


Instead of ‘’ I got ‘*’ in prompt; Is this the behaviour or issue in HBase 
shell?


hbase(main):016:0 # comment
hbase(main):017:0* list
TABLE




FYI: I am using Windows OS;





Sent from Windows Mail





From: ashish singhi
Sent: ‎Tuesday‎, ‎18‎ ‎August‎ ‎2015 ‎11‎:‎57‎ ‎AM
To: dev@hbase.apache.org





Hi,

Did you try '#' ?

Regards,
Ashish Singhi

-Original Message-
From: dinusw...@gmail.com [mailto:dinusw...@gmail.com] 
Sent: 18 August 2015 11:43
To: dev@hbase.apache.org
Subject: Comments in HBase shell

Hi,


I would like execute a comment line in HBase shell; But I couldn't find the 
syntax for the same; Can any one please tell how can I execute a ‘comment’ in 
HBase shell?


For reference:

Hive CLI comment: 
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Cli#LanguageManualCli-HiveInteractiveShellCommands



hive--some comment;





Thanks,

Dinesh Kumar P




Sent from Windows Mail

RE: Comments in HBase shell

[ashish singhi]

Hi,

Did you try '#' ?

Regards,
Ashish Singhi

-Original Message-
From: dinusw...@gmail.com [mailto:dinusw...@gmail.com] 
Sent: 18 August 2015 11:43
To: dev@hbase.apache.org
Subject: Comments in HBase shell

Hi,


I would like execute a comment line in HBase shell; But I couldn't find the 
syntax for the same; Can any one please tell how can I execute a ‘comment’ in 
HBase shell?


For reference:

Hive CLI comment: 
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Cli#LanguageManualCli-HiveInteractiveShellCommands



hive--some comment;





Thanks,

Dinesh Kumar P




Sent from Windows Mail

Re: [Replication] ScopeWALEntryFilter Vs TableCfWALEntryFilter

[Ashish Singhi]

Thanks Ram for the response. Got it.
I did not think of a scenario having two peers for the cluster.

Regards,
Ashish

On Fri, Aug 7, 2015 at 12:00 AM, ramkrishna vasudevan 
ramkrishna.s.vasude...@gmail.com wrote:

 I checked the code. I think the ScopeWalEditFilter is something where for
 the source cluster you set the scope of that Edit. There you can mention
 that is is for Global Replication.

 Now if you have two peers for the cluster and one of the peers you have
 specifically asked not to accept edits of a column family then such Edit
 can be ignored.  Did you see addPeers() API that takes a param that
 represnts a table with Cf list which says what column family's edit can be
 accepted?

 Makes sense?  Am I missing something here?

 Regards
 Ram

 On Thu, Aug 6, 2015 at 10:41 PM, Ashish Singhi 
 ashish.singhi.apa...@gmail.com wrote:

  Hi Ted.
  Thanks for the response.
 
  I was talking about WAL edit replication. I guess that is different from
  region replica replication, no ?
 
  Regards,
  Ashish
 
  On Thu, Aug 6, 2015 at 7:54 PM, Ted Yu yuzhih...@gmail.com wrote:
 
   Please take a look at getScopeWALEntryFilter()
   in RegionReplicaReplicationEndpoint.java
  
   You can examine git log of the above file to see related JIRAs.
  
   Cheers
  
   On Thu, Aug 6, 2015 at 5:49 AM, ashish singhi 
 ashish.sin...@huawei.com
   wrote:
  
Hello All.
   
By default we are adding both ScopeWALEntryFilter and
TableCfWALEntryFilter filter to the list of filters (as part of
BaseReplicationEndpoint#getWALEntryfilter) to filter out the WAL
  entries
that needs to be replicated.
Here I am not able to find out the importance of having
TableCfWALEntryFilter over ScopeWALEntryFilter.
According to me all the entries which needs to be replicated are
  already
filtered out from ScopeWALEntryFilter. So I do not see any advantage
 of
having TableCfWALEntryFilter.
Please let me know If I am missing something.
   
Regards,
Ashish Singhi
   
   
  
 

[Replication] ScopeWALEntryFilter Vs TableCfWALEntryFilter

[ashish singhi]

Hello All.

By default we are adding both ScopeWALEntryFilter and TableCfWALEntryFilter 
filter to the list of filters (as part of 
BaseReplicationEndpoint#getWALEntryfilter) to filter out the WAL entries that 
needs to be replicated.
Here I am not able to find out the importance of having TableCfWALEntryFilter 
over ScopeWALEntryFilter.
According to me all the entries which needs to be replicated are already 
filtered out from ScopeWALEntryFilter. So I do not see any advantage of having 
TableCfWALEntryFilter.
Please let me know If I am missing something.

Regards,
Ashish Singhi

Re: [Replication] ScopeWALEntryFilter Vs TableCfWALEntryFilter

[Ashish Singhi]

Hi Ted.
Thanks for the response.

I was talking about WAL edit replication. I guess that is different from
region replica replication, no ?

Regards,
Ashish

On Thu, Aug 6, 2015 at 7:54 PM, Ted Yu yuzhih...@gmail.com wrote:

 Please take a look at getScopeWALEntryFilter()
 in RegionReplicaReplicationEndpoint.java

 You can examine git log of the above file to see related JIRAs.

 Cheers

 On Thu, Aug 6, 2015 at 5:49 AM, ashish singhi ashish.sin...@huawei.com
 wrote:

  Hello All.
 
  By default we are adding both ScopeWALEntryFilter and
  TableCfWALEntryFilter filter to the list of filters (as part of
  BaseReplicationEndpoint#getWALEntryfilter) to filter out the WAL entries
  that needs to be replicated.
  Here I am not able to find out the importance of having
  TableCfWALEntryFilter over ScopeWALEntryFilter.
  According to me all the entries which needs to be replicated are already
  filtered out from ScopeWALEntryFilter. So I do not see any advantage of
  having TableCfWALEntryFilter.
  Please let me know If I am missing something.
 
  Regards,
  Ashish Singhi