[jira] [Created] (HBASE-17530) Readd TestMergeTableRegionsProcedure mistakenly removed by HBASE-16786
stack created HBASE-17530: - Summary: Readd TestMergeTableRegionsProcedure mistakenly removed by HBASE-16786 Key: HBASE-17530 URL: https://issues.apache.org/jira/browse/HBASE-17530 Project: HBase Issue Type: Sub-task Components: test Reporter: stack Assignee: stack Priority: Critical HBASE-16786 mistakenly removed the test TestMergeTableRegionsProcedure. Need to restore. Filing issue here as critical. Test does basic exercise of merge functionality. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-17529) MergeTableRegionsProcedure failed due to ArrayIndexOutOfBoundsException
Ted Yu created HBASE-17529:
--
Summary: MergeTableRegionsProcedure failed due to
ArrayIndexOutOfBoundsException
Key: HBASE-17529
URL: https://issues.apache.org/jira/browse/HBASE-17529
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
I built tar ball using master branch based on commit
616f4801b06a8427a03ceca9fb8345700ce1ad71.
Was running the following command:
hbase org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList
-DinMemoryCompaction=BASIC Loop 4 6 100 /tmp/hbase-biglinkedlist-verify 6
--monkey slowDeterministic
Here was related snippet:
{code}
2017-01-24 21:29:00,107 DEBUG
[RpcServer.deafult.FPBQ.Fifo.handler=0,queue=0,port=16000]
procedure2.ProcedureExecutor: Stored MergeTableRegionsProcedure
(table=IntegrationTestBigLinkedList
regions=[IntegrationTestBigLinkedList,,1485292220242.4c5ea240e86ef22ec7264b1153dd557d.,
IntegrationTestBigLinkedList,\x0E8\xE3\x8E8\xE3\x8E8,1485292220242.6cdb98dfed41ea689b3cd66478c2c580.
] forcible=false), procId=12, owner=hbase,
state=RUNNABLE:MERGE_TABLE_REGIONS_PREPARE
2017-01-24 21:29:00,108 DEBUG [ProcedureExecutorWorker-14]
wal.WALProcedureStore: Set running procedure count=1, slots=24
2017-01-24 21:29:00,127 ERROR [ProcedureExecutorWorker-14]
procedure2.ProcedureExecutor: CODE-BUG: Uncatched runtime exception for
procedure: MergeTableRegionsProcedure (table=IntegrationTestBigLinkedList
regions=[IntegrationTestBigLinkedList,,1485292220242.4c5ea240e86ef22ec7264b1153dd557d.,
IntegrationTestBigLinkedList,\x0E8\xE3\x8E8\xE3\x8E8,1485292220242.6cdb98dfed41ea689b3cd66478c2c580.
] forcible=false), procId=12, owner=hbase,
state=RUNNABLE:MERGE_TABLE_REGIONS_MOVE_REGION_TO_SAME_RS
java.lang.ArrayIndexOutOfBoundsException
at
org.apache.hadoop.hbase.util.ByteBufferUtils.copyFromBufferToArray(ByteBufferUtils.java:1024)
at org.apache.hadoop.hbase.nio.MultiByteBuff.get(MultiByteBuff.java:628)
at
org.apache.hadoop.hbase.ipc.RpcServer$ByteBuffByteInput.read(RpcServer.java:1483)
at
org.apache.hadoop.hbase.shaded.com.google.protobuf.ByteInputByteString.copyToInternal(ByteInputByteString.java:105)
at
org.apache.hadoop.hbase.shaded.com.google.protobuf.ByteString.toByteArray(ByteString.java:651)
at org.apache.hadoop.hbase.RegionLoad.getName(RegionLoad.java:50)
at
org.apache.hadoop.hbase.ServerLoad.getRegionsLoad(ServerLoad.java:236)
at
org.apache.hadoop.hbase.master.procedure.MergeTableRegionsProcedure.getRegionLoad(MergeTableRegionsProcedure.java:774)
at
org.apache.hadoop.hbase.master.procedure.MergeTableRegionsProcedure.MoveRegionsToSameRS(MergeTableRegionsProcedure.java:461)
at
org.apache.hadoop.hbase.master.procedure.MergeTableRegionsProcedure.executeFromState(MergeTableRegionsProcedure.java:142)
at
org.apache.hadoop.hbase.master.procedure.MergeTableRegionsProcedure.executeFromState(MergeTableRegionsProcedure.java:72)
at
org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:154)
at
org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:708)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1332)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1133)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:76)
at
org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1588)
{code}
Master log to be attached.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-17528) Remove the Scan.ReadType flag introduced in HBASE-17045
Duo Zhang created HBASE-17528: - Summary: Remove the Scan.ReadType flag introduced in HBASE-17045 Key: HBASE-17528 URL: https://issues.apache.org/jira/browse/HBASE-17528 Project: HBase Issue Type: Task Components: scan Affects Versions: 2.0.0 Reporter: Duo Zhang Fix For: 2.0.0 It is used to keep the old behavior of small scan. We should have a mechanism which selects pread or stream automatically at server side. And in general, we have discussed many times whether it is OK to always use pread for a user scan. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-17527) Allow IntegrationTestBigLinkedList table to be in memory compacted
Ted Yu created HBASE-17527: -- Summary: Allow IntegrationTestBigLinkedList table to be in memory compacted Key: HBASE-17527 URL: https://issues.apache.org/jira/browse/HBASE-17527 Project: HBase Issue Type: Improvement Reporter: Ted Yu Currently IntegrationTestBigLinkedList table doesn't support specification of in memory compaction scheme. This JIRA is to add support for specifying different in memory compaction policies. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-17503) [C++] Configuration should be settable and used w/o XML files
[ https://issues.apache.org/jira/browse/HBASE-17503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar resolved HBASE-17503. --- Resolution: Fixed Pushed this to the branch. Thanks for the reviews. > [C++] Configuration should be settable and used w/o XML files > - > > Key: HBASE-17503 > URL: https://issues.apache.org/jira/browse/HBASE-17503 > Project: HBase > Issue Type: Sub-task >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: HBASE-14850 > > Attachments: hbase-17503_v1.patch, hbase-17503_v2.patch > > > Configuration right now is read-only, and there is only XML based > configuration loader. > However, in testing, we need the Config object w/o the XML files, and we need > to be able to set specific values in the conf. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-17526) Procedure v2 - cleanup isSuspended from MasterProcedureScheduler#Queue
Appy created HBASE-17526: Summary: Procedure v2 - cleanup isSuspended from MasterProcedureScheduler#Queue Key: HBASE-17526 URL: https://issues.apache.org/jira/browse/HBASE-17526 Project: HBase Issue Type: Task Reporter: Appy Assignee: Appy Priority: Minor Queue#setSuspended() is not used anywhere, probably because when queue wait/wakes on an event, it gets removed or added back to the fairq. Removing state, functions, and uses of isSuspended() -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-17525) SecureBulkLoadEndpoint not implemented
Josh Elser created HBASE-17525: -- Summary: SecureBulkLoadEndpoint not implemented Key: HBASE-17525 URL: https://issues.apache.org/jira/browse/HBASE-17525 Project: HBase Issue Type: Sub-task Reporter: Josh Elser Assignee: Josh Elser Looks like I only implemented the RSRpcServices method for loading hfiles, not the SecureBulkLoadEndpoint. Need to do that one too and add a test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-17524) HBase 1.3.1 release
Mikhail Antonov created HBASE-17524: --- Summary: HBase 1.3.1 release Key: HBASE-17524 URL: https://issues.apache.org/jira/browse/HBASE-17524 Project: HBase Issue Type: Umbrella Reporter: Mikhail Antonov Let's have this umbrella jira to track backports and bufgixes that we want to go to 1.3.1 release. Please add tasks and comment if you want something to be backported. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-17523) Add Timeline Consistency support to hbase-thrift
Ted Yu created HBASE-17523: -- Summary: Add Timeline Consistency support to hbase-thrift Key: HBASE-17523 URL: https://issues.apache.org/jira/browse/HBASE-17523 Project: HBase Issue Type: Improvement Reporter: Ted Yu Currently struct TScan (defined in Hbase.thrift) doesn't support passing timeline Consistency. This issue is to add related code for timeline Consistency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
HBaseContext with Spark
Hello HBase Folks, Currently I am using HBase 1.2.4 and Hive 1.2.1, I am looking for Bulk Load from Hbase to Hive. I have seen couple of good example at HBase Github Repo: https://github.com/apache/hbase/tree/master/hbase-spark If I would like to use HBaseContext with HBase 1.2.4, how it can be done ? Or which version of HBase has more stability with HBaseContext ? Thanks. Chetan
Successful: HBase Generate Website
Build status: Successful
If successful, the website and docs have been generated. To update the live
site, follow the instructions below. If failed, skip to the bottom of this
email.
Use the following commands to download the patch and apply it to a clean branch
based on origin/asf-site. If you prefer to keep the hbase-site repo around
permanently, you can skip the clone step.
git clone https://git-wip-us.apache.org/repos/asf/hbase-site.git
cd hbase-site
wget -O-
https://builds.apache.org/job/hbase_generate_website/469/artifact/website.patch.zip
| funzip > 616f4801b06a8427a03ceca9fb8345700ce1ad71.patch
git fetch
git checkout -b asf-site-616f4801b06a8427a03ceca9fb8345700ce1ad71
origin/asf-site
git am --whitespace=fix 616f4801b06a8427a03ceca9fb8345700ce1ad71.patch
At this point, you can preview the changes by opening index.html or any of the
other HTML pages in your local
asf-site-616f4801b06a8427a03ceca9fb8345700ce1ad71 branch.
There are lots of spurious changes, such as timestamps and CSS styles in
tables, so a generic git diff is not very useful. To see a list of files that
have been added, deleted, renamed, changed type, or are otherwise interesting,
use the following command:
git diff --name-status --diff-filter=ADCRTXUB origin/asf-site
To see only files that had 100 or more lines changed:
git diff --stat origin/asf-site | grep -E '[1-9][0-9]{2,}'
When you are satisfied, publish your changes to origin/asf-site using these
commands:
git commit --allow-empty -m "Empty commit" # to work around a current ASF
INFRA bug
git push origin asf-site-616f4801b06a8427a03ceca9fb8345700ce1ad71:asf-site
git checkout asf-site
git branch -D asf-site-616f4801b06a8427a03ceca9fb8345700ce1ad71
Changes take a couple of minutes to be propagated. You can verify whether they
have been propagated by looking at the Last Published date at the bottom of
http://hbase.apache.org/. It should match the date in the index.html on the
asf-site branch in Git.
As a courtesy- reply-all to this email to let other committers know you pushed
the site.
If failed, see https://builds.apache.org/job/hbase_generate_website/469/console
Re: HBase Service Level Authorization
I can see how this all started:
public class HBasePolicyProvider extends PolicyProvider {
protected static Service[] services = {
new Service("security.client.protocol.acl", HRegionInterface.class),
new Service("security.admin.protocol.acl", HMasterInterface.class),
new Service("security.masterregion.protocol.acl",
HMasterRegionInterface.class)
};
There naming made sense. But over time this got changed 3-4 times and
added the newer interface or, at last, the protobuf services. But the
current ACL name to service name is a mystery to me. Maybe we can
start the discussion there?
Lars
On Tue, Jan 24, 2017 at 8:36 AM, Lars George wrote:
> Hi,
>
> Looking at our `PolicyProvider` implementation I see this:
>
> /**
> * Implementation of secure Hadoop policy provider for mapping
> * protocol interfaces to hbase-policy.xml entries.
> */
> @InterfaceAudience.Private
> public class HBasePolicyProvider extends PolicyProvider {
> protected final static Service[] services = {
> new Service("security.client.protocol.acl",
> ClientService.BlockingInterface.class),
> new Service("security.client.protocol.acl",
> AdminService.BlockingInterface.class),
> new Service("security.admin.protocol.acl",
> MasterService.BlockingInterface.class),
> new Service("security.masterregion.protocol.acl",
> RegionServerStatusService.BlockingInterface.class)
> };
>
> How does that all make sense? Both the `ClientService` and
> `AdminService` are controlled by the `security.client.protocol.acl`
> setting, allowing (or disallowing) access to the two client side APIs.
> But then there is the `security.admin.protocol.acl` for the
> `MasterService` that controls another half of the administrative
> calls?
>
> And then the last of them, which I presume by looking at the provided
> methods that it covers the Master-RegionServer communication. It is
> controlled by `security.masterregion.protocol.acl`... did we run out
> of characters to not name that at least
> `security.masterregionserver.protocol.acl`?
>
> Can anyone explain what we are doing here? And why?
>
> Best,
> Lars
