[jira] [Created] (HBASE-13476) Procedure v2 - Add Replay Order logic for child procedures
Matteo Bertozzi created HBASE-13476: --- Summary: Procedure v2 - Add Replay Order logic for child procedures Key: HBASE-13476 URL: https://issues.apache.org/jira/browse/HBASE-13476 Project: HBase Issue Type: Sub-task Components: proc-v2 Affects Versions: 2.0.0, 1.1.0 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi The current replay order logic is only for single-level procedures (which is what we are using today for master operations). To complete the implementation for the notification-bus we need to be able to replay in correct order child procs too. this will not impact the the current procs implementation (create/delete/modify/...) it is just a change at the framework level. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-13202) Procedure v2 - core framework
[ https://issues.apache.org/jira/browse/HBASE-13202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi resolved HBASE-13202. - Resolution: Fixed Fix Version/s: 1.1.0 > Procedure v2 - core framework > - > > Key: HBASE-13202 > URL: https://issues.apache.org/jira/browse/HBASE-13202 > Project: HBase > Issue Type: Sub-task > Components: master >Affects Versions: 2.0.0, 1.1.0 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi > Fix For: 2.0.0, 1.1.0 > > Attachments: HBASE-13202-v0-hbase-12439.patch, > HBASE-13202-v1-hbase-12439.patch, HBASE-13202-v2.patch, > HBASE-13203-v2-branch_1.patch, ProcedureV2-overview.pdf > > > core package, part of HBASE-12439 > this is just the proc-v2 submodule. it depends only on hbase-common. > https://reviews.apache.org/r/27703/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-13203) Procedure v2 - master create/delete table
[ https://issues.apache.org/jira/browse/HBASE-13203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi resolved HBASE-13203. - Resolution: Fixed Fix Version/s: 1.1.0 > Procedure v2 - master create/delete table > - > > Key: HBASE-13203 > URL: https://issues.apache.org/jira/browse/HBASE-13203 > Project: HBase > Issue Type: Sub-task > Components: master >Affects Versions: 2.0.0, 1.1.0 >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Fix For: 2.0.0, 1.1.0 > > Attachments: HBASE-13203-v0.patch, HBASE-13203-v1-branch-1.patch, > HBASE-13203-v1.patch > > > master side, part of HBASE-12439 > starts up the procedure executor on the master > and replaces the create/delete table handlers with the procedure version. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-13204) Procedure v2 - client create/delete table sync
[ https://issues.apache.org/jira/browse/HBASE-13204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi resolved HBASE-13204. - Resolution: Fixed Fix Version/s: 1.1.0 > Procedure v2 - client create/delete table sync > -- > > Key: HBASE-13204 > URL: https://issues.apache.org/jira/browse/HBASE-13204 > Project: HBase > Issue Type: Sub-task > Components: master >Reporter: Matteo Bertozzi >Assignee: Matteo Bertozzi >Priority: Minor > Fix For: 2.0.0, 1.1.0 > > Attachments: HBASE-13204-v0-branch-1.patch, HBASE-13204-v0.patch > > > client side, part of HBASE-12439/HBASE-13203 > it uses the new procedure code to be know when the procedure is completed, > and have a proper sync behavior on create/delete table. > Review: https://reviews.apache.org/r/32391/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-13209) Procedure V2 - master Add/Modify/Delete Column Family
[ https://issues.apache.org/jira/browse/HBASE-13209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi resolved HBASE-13209. - Resolution: Fixed Fix Version/s: 1.1.0 > Procedure V2 - master Add/Modify/Delete Column Family > - > > Key: HBASE-13209 > URL: https://issues.apache.org/jira/browse/HBASE-13209 > Project: HBase > Issue Type: Sub-task > Components: master >Affects Versions: 2.0.0, 1.1.0 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang > Labels: reliability > Fix For: 2.0.0, 1.1.0 > > Attachments: AlterColumnFamily-no-gen-file.v1-master.patch, > HBASE-13209-v2.patch, HBASE-13209-v3-branch-1.patch, HBASE-13209-v3.patch > > Original Estimate: 168h > Time Spent: 384h > Remaining Estimate: 0h > > master side, part of HBASE-12439 > starts up the procedure executor on the master > and replaces the add/modify/delete handlers with the procedure version. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-13210) Procedure V2 - master Modify table
[ https://issues.apache.org/jira/browse/HBASE-13210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi resolved HBASE-13210. - Resolution: Fixed Fix Version/s: 1.1.0 > Procedure V2 - master Modify table > -- > > Key: HBASE-13210 > URL: https://issues.apache.org/jira/browse/HBASE-13210 > Project: HBase > Issue Type: Sub-task > Components: master >Affects Versions: 2.0.0, 1.1.0 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang > Labels: reliablity > Fix For: 2.0.0, 1.1.0 > > Attachments: HBASE-13210-v2.patch, HBASE-13210-v3.patch, > ModifyTableProcedure-no-gen-file.v1-master.patch > > Original Estimate: 72h > Time Spent: 168h > Remaining Estimate: 0h > > master side, part of HBASE-12439 > starts up the procedure executor on the master > and replaces the modify table handlers with the procedure version. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-13211) Procedure V2 - master Enable/Disable table
[ https://issues.apache.org/jira/browse/HBASE-13211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi resolved HBASE-13211. - Resolution: Fixed Fix Version/s: 1.1.0 > Procedure V2 - master Enable/Disable table > -- > > Key: HBASE-13211 > URL: https://issues.apache.org/jira/browse/HBASE-13211 > Project: HBase > Issue Type: Sub-task > Components: master >Affects Versions: 2.0.0, 1.1.0 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang > Labels: reliability > Fix For: 2.0.0, 1.1.0 > > Attachments: EnableDisableTableServer-no-gen-files.v1-master.patch, > HBASE-13211-v2-branch-1.patch, HBASE-13211-v2.patch > > Original Estimate: 120h > Time Spent: 216h > Remaining Estimate: 0h > > master side, part of HBASE-12439 > starts up the procedure executor on the master > and replaces the enable/disable table handlers with the procedure version. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-13290) Procedure v2 - client enable/disable table sync
[ https://issues.apache.org/jira/browse/HBASE-13290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi resolved HBASE-13290. - Resolution: Fixed Fix Version/s: 1.1.0 > Procedure v2 - client enable/disable table sync > --- > > Key: HBASE-13290 > URL: https://issues.apache.org/jira/browse/HBASE-13290 > Project: HBase > Issue Type: Sub-task > Components: Client >Affects Versions: 2.0.0, 1.1.0 >Reporter: Stephen Yuan Jiang >Assignee: Stephen Yuan Jiang > Fix For: 2.0.0, 1.1.0 > > Attachments: EnableDisableTableClientPV2-draft.v0-master.patch, > HBASE-13290-v2-branch-1.patch, HBASE-13290-v2.patch > > Original Estimate: 120h > Time Spent: 48h > Remaining Estimate: 72h > > client side, part of HBASE-12439/HBASE-13211 > it uses the new procedure code to be know when the procedure is completed, > and have a proper sync/async behavior on enable/disable table. > For 1.1, It has to be binary compatible (1.0 client can talk to 1.1 server && > 1.1 client can talk to 1.0 server). Binary compatible is TBD for 2.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: [VOTE] The 4th HBase 0.98.12 release candidate (RC3) is available, vote closing 4/17/2015
Gentle reminder that there are two days left to vote on this RC. We should be all set here, let's get it out. If we need more time, I can extend the vote. On Thu, Apr 9, 2015 at 10:14 PM, Andrew Purtell wrote: > [Just a resend without an annoying typo in the subject. Responding to > either thread will work, I'll tally them both.] > > > The 4 > th > HBase 0.98.12 release candidate (RC3) is available for download at > http://people.apache.org/~apurtell/0.98.12RC3/ and Maven artifacts are > also available in the temporary repository > https://repository.apache.org/content/repositories/orgapachehbase-1073/ > > The API compatibility report with respect to the previous release can be > found at > http://people.apache.org/~apurtell/0.98.11_0.98.12RC3_compat_report.html . > The difference from the last RC is HBASE-13397 changes the return type of > getRemoteAddress in the LimitedPrivate/Evolving interface RpcServer from > String to the more appropriate (and consistent with later branches) > InetAddress. > > Signed with my code signing key D5365CCD. > > The issues resolved in this release can be found at http://s.apache.org/zH > > Please try out the candidate and vote +1/-1 by midnight Pacific Time > (00:00 -0800 GMT) on April 17 on whether or not we should release this as > 0.98.12. Three +1 votes from PMC will be required to release. > > -- > Best regards, > >- Andy > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > (via Tom White) > > -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
[jira] [Created] (HBASE-13477) Create metrics on failed requests
Elliott Clark created HBASE-13477: - Summary: Create metrics on failed requests Key: HBASE-13477 URL: https://issues.apache.org/jira/browse/HBASE-13477 Project: HBase Issue Type: Bug Reporter: Elliott Clark Assignee: Elliott Clark Add a metric on how many requests failed/errored out. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: [VOTE] The 4th HBase 0.98.12 release candidate (RC3) is available, vote closing 4/17/2015
+1 - build from 0.98.12RC3 tag - inspected generated tarball - inserted some data, scanned, flushed, major_compacted, scanned again. All the usual. - nothing undue in the logs Super, duper minor nit: Seeing logs like this "wal.FSHLog: RS:0;localhost:35670-WAL.AsyncSyncer3 interrupted while waiting for notification from AsyncWriter thread" in the logs upon shutdown. No harm, but still unnecessary to show these. -- Lars - Original Message - From: Andrew Purtell To: "dev@hbase.apache.org" Cc: Sent: Wednesday, April 15, 2015 10:04 AM Subject: Re: [VOTE] The 4th HBase 0.98.12 release candidate (RC3) is available, vote closing 4/17/2015 Gentle reminder that there are two days left to vote on this RC. We should be all set here, let's get it out. If we need more time, I can extend the vote. On Thu, Apr 9, 2015 at 10:14 PM, Andrew Purtell wrote: > [Just a resend without an annoying typo in the subject. Responding to > either thread will work, I'll tally them both.] > > > The 4 > th > HBase 0.98.12 release candidate (RC3) is available for download at > http://people.apache.org/~apurtell/0.98.12RC3/ and Maven artifacts are > also available in the temporary repository > https://repository.apache.org/content/repositories/orgapachehbase-1073/ > > The API compatibility report with respect to the previous release can be > found at > http://people.apache.org/~apurtell/0.98.11_0.98.12RC3_compat_report.html . > The difference from the last RC is HBASE-13397 changes the return type of > getRemoteAddress in the LimitedPrivate/Evolving interface RpcServer from > String to the more appropriate (and consistent with later branches) > InetAddress. > > Signed with my code signing key D5365CCD. > > The issues resolved in this release can be found at http://s.apache.org/zH > > Please try out the candidate and vote +1/-1 by midnight Pacific Time > (00:00 -0800 GMT) on April 17 on whether or not we should release this as > 0.98.12. Three +1 votes from PMC will be required to release. > > -- > Best regards, > >- Andy > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > (via Tom White) > > -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
[jira] [Created] (HBASE-13478) Document the change of default master ports being used .
Srikanth Srungarapu created HBASE-13478: --- Summary: Document the change of default master ports being used . Key: HBASE-13478 URL: https://issues.apache.org/jira/browse/HBASE-13478 Project: HBase Issue Type: Sub-task Components: documentation Reporter: Srikanth Srungarapu Priority: Minor In 1.0.x, master by default binds to the region server ports. But in 1.1 and 2.0 branches, we have undone this changes and brought back the usage of old master ports to make the migration from 0.98 -> 1.1 hassle free. Please see the parent jira for more background. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: [VOTE] The 4th HBase 0.98.12 release candidate (RC3) is available, vote closing 4/17/2015
+1 Downloaded src. Checked sig and md5 Built it, poked it, and ran it. Loaded a bit of data and verified it there in standalone mode. Nothing untoward in logs. St.Ack On Thu, Apr 9, 2015 at 10:14 PM, Andrew Purtell wrote: > [Just a resend without an annoying typo in the subject. Responding to > either thread will work, I'll tally them both.] > > > The 4 > th > HBase 0.98.12 release candidate (RC3) is available for download at > http://people.apache.org/~apurtell/0.98.12RC3/ and Maven artifacts are > also > available in the temporary repository > https://repository.apache.org/content/repositories/orgapachehbase-1073/ > > The API compatibility report with respect to the previous release can be > found at > http://people.apache.org/~apurtell/0.98.11_0.98.12RC3_compat_report.html . > The difference from the last RC is HBASE-13397 changes the return type of > getRemoteAddress in the LimitedPrivate/Evolving interface RpcServer from > String to the more appropriate (and consistent with later branches) > InetAddress. > > Signed with my code signing key D5365CCD. > > The issues resolved in this release can be found at http://s.apache.org/zH > > Please try out the candidate and vote +1/-1 by midnight Pacific Time (00:00 > -0800 GMT) on April 17 on whether or not we should release this as > 0.98.12. Three +1 votes from PMC will be required to release. > > -- > Best regards, > >- Andy > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > (via Tom White) >
[jira] [Created] (HBASE-13479) [branch-1.0] Master should not bind to region server ports
Enis Soztutar created HBASE-13479:
-
Summary: [branch-1.0] Master should not bind to region server ports
Key: HBASE-13479
URL: https://issues.apache.org/jira/browse/HBASE-13479
Project: HBase
Issue Type: Sub-task
Reporter: Enis Soztutar
Priority: Blocker
Fix For: 1.0.1
In parent jira, we have brought back old config value {{hbase.master.port}} and
made it so that if it is configured (which is by default), master does not bind
to the region servers's RPC and info ports now. We have realized that master
binding to RS ports turns out to be a pain point and unnecessary frustration
for users (although we were expecting the opposite).
Let's use this jira to see what we can do in 1.0.x series. In 1.0.0, master
binds to ports
{{hbase.regionserver.port}}
{{hbase.master.info.port}}
{{hbase.regionserver.info.port}}
In 1.0.x, we have a couple of options I think:
(a) don't do anything.
(b) Commit patch from master as it is. All upgrades from 1.0.0 will observe
changed ports.
(c) Commit the patch, disabled by default. If user has {{hbase.master.port}}
set, bind to the port. Only users who set the conf value will observe changed
ports.
(d) Commit the patch with an extra config option {{hbase.master.compat.mode}}.
The users have to explicitly enable this in 1.0.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-13480) ShortCircuitConnection doesn't short-circuit all calls as expected
Josh Elser created HBASE-13480:
--
Summary: ShortCircuitConnection doesn't short-circuit all calls as
expected
Key: HBASE-13480
URL: https://issues.apache.org/jira/browse/HBASE-13480
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 1.0.0, 2.0.0, 1.1.0
Reporter: Josh Elser
Assignee: Josh Elser
Fix For: 2.0.0, 1.1.0, 1.0.2
Noticed the following situation in debugging unexpected unit tests failures in
HBASE-13351.
{{ConnectionUtils#createShortCircuitHConnection(Connection, ServerName,
AdminService.BlockingInterface, ClientService.BlockingInterface)}} is intended
to avoid the extra RPC by calling the server's instantiation of the protobuf
rpc stub directly for the AdminService and ClientService.
The problem is that this is insufficient to actually avoid extra "remote" RPCs
as all other calls to the Connection are routed to a "real" Connection
instance. As such, any object created by the "real" Connection (such as an
HTable) will use the real Connection, not the SSC.
The end result is that
{{MasterRpcService#reportRegionStateTransition(RpcController,
ReportRegionStateTransitionRequest)}} will make additional "remote" RPCs over
what it thinks is an SSC through a {{Get}} on {{HTable}} which was constructed
using the SSC, but the {{Get}} itself will use the underlying real Connection
instead of the SSC. With insufficiently sized thread pools, this has been
observed to result in RPC deadlock in the HMaster where an RPC attempts to make
another RPC but there are no more threads available to service the second RPC
so the first RPC blocks indefinitely.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: [VOTE] The 4th HBase 0.98.12 release candidate (RC3) is available, vote closing 4/17/2015
+1 (non-binding) * built internally from git tag: 0.98.12RC3 * loaded data in dev cluster, read/write OK * custom co-processors work fine thanks -adrian On 4/15/15, 20:04, "Andrew Purtell" wrote: >Gentle reminder that there are two days left to vote on this RC. We should >be all set here, let's get it out. If we need more time, I can extend the >vote. > > >On Thu, Apr 9, 2015 at 10:14 PM, Andrew Purtell >wrote: > >> [Just a resend without an annoying typo in the subject. Responding to >> either thread will work, I'll tally them both.] >> >> >> The 4 >> th >> HBase 0.98.12 release candidate (RC3) is available for download at >> http://people.apache.org/~apurtell/0.98.12RC3/ and Maven artifacts are >> also available in the temporary repository >> https://repository.apache.org/content/repositories/orgapachehbase-1073/ >> >> The API compatibility report with respect to the previous release can be >> found at >> >>http://people.apache.org/~apurtell/0.98.11_0.98.12RC3_compat_report.html >>. >> The difference from the last RC is HBASE-13397 changes the return type >>of >> getRemoteAddress in the LimitedPrivate/Evolving interface RpcServer from >> String to the more appropriate (and consistent with later branches) >> InetAddress. >> >> Signed with my code signing key D5365CCD. >> >> The issues resolved in this release can be found at >>http://s.apache.org/zH >> >> Please try out the candidate and vote +1/-1 by midnight Pacific Time >> (00:00 -0800 GMT) on April 17 on whether or not we should release this >>as >> 0.98.12. Three +1 votes from PMC will be required to release. >> >> -- >> Best regards, >> >>- Andy >> >> Problems worthy of attack prove their worth by hitting back. - Piet Hein >> (via Tom White) >> >> > > >-- >Best regards, > > - Andy > >Problems worthy of attack prove their worth by hitting back. - Piet Hein >(via Tom White)
[jira] [Created] (HBASE-13481) Master should respect master (old) DNS/bind related configurations
Enis Soztutar created HBASE-13481:
-
Summary: Master should respect master (old) DNS/bind related
configurations
Key: HBASE-13481
URL: https://issues.apache.org/jira/browse/HBASE-13481
Project: HBase
Issue Type: Sub-task
Reporter: Enis Soztutar
Fix For: 2.0.0, 1.1.0
This is a continuation of parent HBASE-13453. We should continue respecting the
following parameters that 1.0.0 does not:
{code}
hbase.master.dns.interface
hbase.master.dns.nameserver
hbase.master.ipc.address
{code}
Credit goes to [~jerryhe] for pointing that out.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: [VOTE] Second release candidate for HBase 1.0.1 (RC1) is available. Please vote by April 16 2015
I will sink this RC in favor of getting https://issues.apache.org/jira/browse/HBASE-13479 in. For those who are not following the parent jira, and the changes recently: 1.0.0 is released with master being a region server. We had removed configuration options hbase.master.port, hbase.master.info.port and others. Master by default binds to RS ports. The idea was that we want to go to a model where master is a RS and over the long term we get rid of backup masters and masters and have only 1 daemon type. This is not happening soon enough, and we do not want to cause further inconvenience to users. So, we should bring the 0.98 master config options back so that users do not take a hit unnecessarily. See https://issues.apache.org/jira/browse/HBASE-13453 for more context. If you have any concerns, now would be a good time to raise. Enis On Sat, Apr 11, 2015 at 6:42 AM, Ted Yu wrote: > +1 > > Ran unit test suite > Exercised basic shell commands > Checked signatures > > > On Fri, Apr 10, 2015 at 3:51 PM, Enis Söztutar wrote: > > > I am pleased to announce that the second release candidate for the > release > > 1.0.1 > > (HBase-1.0.1RC1), is available for download at > > https://dist.apache.org/repos/dist/dev/hbase/hbase-1.0.1RC1/ > > > > Maven artifacts are also available in the temporary repository > > https://repository.apache.org/content/repositories/orgapachehbase-1074 > > > > Signed with my code signing key E964B5FF. Can be found here: > > https://people.apache.org/keys/committer/enis.asc > > > > Signed tag in the repository can be found here: > > > > > https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=tag;h=117177a4531829cad19529b8387bc688eabe38af > > > > > > HBase 1.0.1 is the next “patch” release in the 1.0.x release line and > > supersedes 1.0.0. > > According to the HBase’s semantic version guide (See [1]), the release > > candidate is > > source and binary compatible with 1.0.0 for client applications and > server > > side libraries > > (coprocessors, filters, etc). > > > > Binary / source compatibility report with 1.0.0 can be reached here: > > https://people.apache.org/~enis/1.0.0_1.0.1RC1_compat_report.html > > > > > > 1.0.1 release contains 109 fixes on top of 1.0.0 release. Most of > > the changes are > > bug fixes except for the following: > > > > [HBASE-13002] - Make encryption cipher configurable > > [HBASE-13044] - Configuration option for disabling coprocessor loading > > [HBASE-13054] - Provide more tracing information for locking/latching > > events. > > [HBASE-13059] - Set executable bit for scripts in dev-support > > [HBASE-13086] - Show ZK root node on Master WebUI > > [HBASE-13109] - Make better SEEK vs SKIP decisions during scanning > > [HBASE-13120] - Allow disabling hadoop classpath and native library > lookup > > [HBASE-13132] - Improve RemoveColumn action debug message > > [HBASE-13162] - Add capability for cleaning hbase acls to hbase cleanup > > script. > > [HBASE-13168] - Backport HBASE-12590 "A solution for data skew in > > HBase-Mapreduce Job" > > [HBASE-13183] - Make ZK tickTime configurable in standalone HBase > > [HBASE-13342] - Fix incorrect interface annotations > > [HBASE-12869] - Add a REST API implementation of the ClusterManager > > interface > > [HBASE-13380] - Cherry pick the HBASE-12808 compatibility checker tool > back > > to 0.98+ > > > > Full list of the issues can be found at: > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12329042&projectId=12310753 > > < > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12329042&projectId=12310753 > > > > > > > Compatibility > > - > > This release (1.0.1) is source, wire and binary compatible with 1.0.0 > > release. Client > > applications does not have to be recompiled with the new version (unless > > new API is used) > > if upgrading from 1.0.0. It is a drop-in replacement. > > > > See release notes for 1.0.0 [2] for compatibility with earlier > > versions (0.94, 0.96, 0.98). > > Compatibility of 1.0.1 with earlier versions is the same as in 1.0.0. > > > > Source Compatibility: > > Client side code in HBase-1.0.x is (mostly) source compatible with 0.98.x > > versions. Some minor API changes might be needed from the client side. > > > > Wire Compatibility: > > HBase-1.0.x release is wire compatible with 0.98.x releases. Clients and > > servers running in different versions as long as new features are not > used > > should be possible. > > A rolling upgrade from 0.98.x clusters to 1.0.x is supported as well. > > Rolling upgrade from 0.96 directly to 1.0.x is not supported. > > 1.0.x is NOT wire compatible with earlier releases (0.94, etc). > > > > Binary Compatibility: > > Binary compatibility at the Java API layer with earlier versions (0.98.x, > > 0.96.x and 0.94.x) is not supported. You may have to recompile your > client > > code and any server side code (coprocessors, filters etc) referring to > > hbase jars. > > > > Other Compatibility issues
[jira] [Created] (HBASE-13482) Phoenix is failing to scan tables on secure environments.
Alicia Ying Shu created HBASE-13482: --- Summary: Phoenix is failing to scan tables on secure environments. Key: HBASE-13482 URL: https://issues.apache.org/jira/browse/HBASE-13482 Project: HBase Issue Type: Bug Reporter: Alicia Ying Shu When executed on secure environments, phoenix query is getting the following exception message: -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13483) [docs] onheap is not a valid bucket cache IO engine.
Esteban Gutierrez created HBASE-13483:
-
Summary: [docs] onheap is not a valid bucket cache IO engine.
Key: HBASE-13483
URL: https://issues.apache.org/jira/browse/HBASE-13483
Project: HBase
Issue Type: Bug
Components: documentation
Affects Versions: 2.0.0
Reporter: Esteban Gutierrez
>From the HBase book:
>http://hbase.apache.org/book.html#hbase_default_configurations
:
{code}
hbase.bucketcache.ioengine
Description
Where to store the contents of the bucketcache. One of: *onheap*, offheap, or
file. If a file, set it to file:PATH_TO_FILE. See
https://hbase.apache.org/apidocs/org/apache/hadoop/hbase/io/hfile/CacheConfig.html
for more information.
{code}
Instead of onheap it should be heap.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-13484) [docs] docs need to be specific about hbase.bucketcache.size range
Esteban Gutierrez created HBASE-13484:
-
Summary: [docs] docs need to be specific about
hbase.bucketcache.size range
Key: HBASE-13484
URL: https://issues.apache.org/jira/browse/HBASE-13484
Project: HBase
Issue Type: Bug
Reporter: Esteban Gutierrez
This is not 100% clear for users:
if hbase.bucketcache.size is between 0.0 and 1.0 then is a percentage. But if
the value is above 1 then the value is expressed in KBs:
>From CacheConfig.getBucketCache():
{noformat}
long bucketCacheSize = (long) (bucketCachePercentage < 1? mu.getMax() *
bucketCachePercentage:
bucketCachePercentage * 1024 * 1024);
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-13485) HBase on Ceph
Andrew Purtell created HBASE-13485: -- Summary: HBase on Ceph Key: HBASE-13485 URL: https://issues.apache.org/jira/browse/HBASE-13485 Project: HBase Issue Type: Task Reporter: Andrew Purtell Ceph (http://ceph.com/ceph-storage/ is an open source, scalable, distributed, clustered block storage system with interesting dynamic data placement properties and support for tiered storage and multiple strategies for resilience. Ceph is a layered system: a block storage layer, known as RADOS, and a filesystem layer implemented on top of the block store, of varying degrees of stability. It might be interesting to try out HBase on Ceph using the Hadoop filesystem binding for Ceph (https://github.com/ceph/cephfs-hadoop). Alternatively, implement "HBase filesystem" support for RADOS via the Java bindings (https://github.com/ceph/rados-java, via JNA). Considerations here will be similar to what we might want to do when HDFS decouples block management from namespace. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: [VOTE] The 4th HBase 0.98.12 release candidate (RC3) is available, vote closing 4/17/2015
+1 Un-tar'd hadoop-2 and src tars. Checked their contents Ran a local instance everything looked good. Checked headers. On Wed, Apr 15, 2015 at 1:33 PM, Adrian Muraru wrote: > +1 (non-binding) > > * built internally from git tag: 0.98.12RC3 > * loaded data in dev cluster, read/write OK > * custom co-processors work fine > > thanks > -adrian > > On 4/15/15, 20:04, "Andrew Purtell" wrote: > > >Gentle reminder that there are two days left to vote on this RC. We should > >be all set here, let's get it out. If we need more time, I can extend the > >vote. > > > > > >On Thu, Apr 9, 2015 at 10:14 PM, Andrew Purtell > >wrote: > > > >> [Just a resend without an annoying typo in the subject. Responding to > >> either thread will work, I'll tally them both.] > >> > >> > >> The 4 > >> th > >> HBase 0.98.12 release candidate (RC3) is available for download at > >> http://people.apache.org/~apurtell/0.98.12RC3/ and Maven artifacts are > >> also available in the temporary repository > >> https://repository.apache.org/content/repositories/orgapachehbase-1073/ > >> > >> The API compatibility report with respect to the previous release can be > >> found at > >> > >>http://people.apache.org/~apurtell/0.98.11_0.98.12RC3_compat_report.html > >>. > >> The difference from the last RC is HBASE-13397 changes the return type > >>of > >> getRemoteAddress in the LimitedPrivate/Evolving interface RpcServer from > >> String to the more appropriate (and consistent with later branches) > >> InetAddress. > >> > >> Signed with my code signing key D5365CCD. > >> > >> The issues resolved in this release can be found at > >>http://s.apache.org/zH > >> > >> Please try out the candidate and vote +1/-1 by midnight Pacific Time > >> (00:00 -0800 GMT) on April 17 on whether or not we should release this > >>as > >> 0.98.12. Three +1 votes from PMC will be required to release. > >> > >> -- > >> Best regards, > >> > >>- Andy > >> > >> Problems worthy of attack prove their worth by hitting back. - Piet Hein > >> (via Tom White) > >> > >> > > > > > >-- > >Best regards, > > > > - Andy > > > >Problems worthy of attack prove their worth by hitting back. - Piet Hein > >(via Tom White) > >
Re: Question on Reading Sequencefile data using mapreduce
Of course. Thanks Gary.
On Tue, Apr 14, 2015 at 11:18 AM, Gary Helmling wrote:
> I believe the recommended approach would be to use CellUtil. It is marked
> as Public/Evolving and exposes a number of static createCell() methods that
> will generate Cell instances for different combinations of parameters.
>
> On Tue, Apr 14, 2015 at 9:04 AM Nick Dimiduk wrote:
>
> > Heya devs,
> >
> > Looking for an appropriate answer for this question, it seems we don't
> have
> > a public Cell implementation that can be used for generating HFiles. How
> > are folks expected to generate Cells instances if KeyValue is
> > @InterfaceAudience.Private ?
> >
> > -n
> >
> > On Tue, Apr 14, 2015 at 9:00 AM, Nick Dimiduk
> wrote:
> >
> > > What version of HBase are you using?
> > >
> > > In recent versions, HFileOutputFormat is a deprecated class (replaced
> by
> > > HFileOutputFormat2), and KeyValue is an internal API (see the class
> > > annotation @InterfaceAudience.Private, basically, use at your own
> peril).
> > > The javadoc on the KeyValue constructor you're using says "Creates a
> > > KeyValue from the start of the specified byte array. *Presumes bytes
> > > content is formatted as a KeyValue blob*." (emphasis my own) It looks
> > > like the value you're using for bytes is not in the KeyValue blob
> format.
> > >
> > > Try instead one of the other KeyValue constructors, such as
> > > KeyValue(byte[] row, byte[] family, byte[] qualifier, byte[] value).
> > >
> > > Thanks,
> > > Nick
> > >
> > >
> > > On Thu, Apr 9, 2015 at 12:23 PM, yuantao peng
> > > wrote:
> > >
> > >> I am learning how to upload binary data to HBASE using mapreduce.
> Here
> > >> are the steps I am following assuming my binary file is testlist
> > >> (1) wrote a sequencefilewrite.java to read the local testlist file and
> > >> save
> > >> a sequence file to HDFS
> > >> (2) wrote a MapReduce program to read the generated sequence file, and
> > >> generate a HFile
> > >> (3) bulk import this HFile to HBASE
> > >>
> > >> I am stuck at step (2) as I keep getting exception. I am absolutely
> new
> > >> to
> > >> hadoop/hbase, code is posted below, any comments or suggestions are
> > >> appreciated!!!
> > >>
> > >> Sequencewrite.java is like this:
> > >>
> > >> public class SequenceFileWrite
> > >> {
> > >> public static void main(String[] args) throws IOException {
> > >> String uri = args[1];
> > >> Configuration conf = new Configuration();
> > >> FileSystem fs = FileSystem.get(conf);
> > >> Path path = new Path(uri);
> > >> File infile = new File(args[0]);
> > >> SequenceFile.Writer writer = null;
> > >> try
> > >> {
> > >>
> > >> BytesWritable key, value;
> > >> writer = SequenceFile.createWriter(fs, conf, path,
> > >> BytesWritable.class, BytesWritable.class);
> > >> FileInputStream fin = new FileInputStream(infile);
> > >> for(int i=0; i<10; ++i) {
> > >> key = new BytesWritable();
> > >> value = new BytesWritable();
> > >> byte[] keybuf = new byte[2];
> > >> byte[] valbuf = new byte[2];
> > >> fin.read(keybuf);
> > >> fin.read(valbuf);
> > >> key.set(keybuf,0,2);
> > >> value.set(valbuf,0,2);
> > >> writer.append(key,value);
> > >> }
> > >> } finally {
> > >>IOUtils.closeStream(writer);
> > >> }
> > >> }
> > >> }
> > >>
> > >> And my mapper is like this:
> > >>
> > >> public class HBaseTkrHdrMapper extends Mapper > >> BytesWritable,
> > >> ImmutableBytesWritable, KeyValue> {
> > >>
> > >> int tipOffSeconds = 0;
> > >> String tableName = "";
> > >>
> > >> ImmutableBytesWritable hKey = new ImmutableBytesWritable();
> > >> KeyValue kv;
> > >>
> > >> @Override
> > >> protected void setup(Context context) throws IOException,
> > >> InterruptedException {
> > >> Configuration c = context.getConfiguration();
> > >> tipOffSeconds = c.getInt("epoch.seconds.tipoff", 0);
> > >> tableName = c.get("hbase.table.mrtest");
> > >> }
> > >>
> > >> @Override
> > >> protected void map(BytesWritable key, BytesWritable value, Context
> > >> context) throws IOException, InterruptedException {
> > >> ImmutableBytesWritable hkey = new
> > >> ImmutableBytesWritable(key.getBytes());
> > >> KeyValue hval = new KeyValue(value.getBytes());
> > >> context.write(hkey, hval);
> > >> }
> > >> }
> > >>
> > >> Driver code is as follows:
> > >>
> > >> public class Driver {
> > >> public static void main(String[] args) throws Exception {
> > >> Configuration conf = new Configuration();
> > >> args = new GenericOptionsParser(conf, args).getRemainingArgs();
> > >>
> > >> @SuppressWarnings("deprecation")
> > >> Job job = new Job(conf, "Bulk Import");
> > >> job.setJarByClass(HBaseTkrHdrMapper.class);
> > >>
> > >> job.setMapperClass(HBaseTkrHdrMapper.class);
> > >> job.setMapOutputKeyClass(ImmutableBytesWritable.class)
