[jira] [Created] (HBASE-14605) Split fails due to 'No valid credentials' error when SecureBulkLoadEndpoint#start tries to access hdfs

[Ted Yu (JIRA)]

Ted Yu created HBASE-14605:
--

 Summary: Split fails due to 'No valid credentials' error when 
SecureBulkLoadEndpoint#start tries to access hdfs
 Key: HBASE-14605
 URL: https://issues.apache.org/jira/browse/HBASE-14605
 Project: HBase
  Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
 Attachments: 14605-v1.txt

During recent testing in secure cluster (with HBASE-14475), we found the 
following when user X (non-super user) split a table with region replica:
{code}
2015-10-12 10:58:18,955 ERROR [FifoRpcScheduler.handler1-thread-9] 
master.HMaster: Region server hbase-4-4.novalocal,60020,1444645588137 reported 
a fatal error:
ABORTING region server hbase-4-4.novalocal,60020,1444645588137: The coprocessor 
org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw an 
unexpected   exception
Cause:
java.lang.IllegalStateException: Failed to get FileSystem instance
  at 
org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.start(SecureBulkLoadEndpoint.java:148)
  at 
org.apache.hadoop.hbase.coprocessor.CoprocessorHost$Environment.startup(CoprocessorHost.java:415)
  at 
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadInstance(CoprocessorHost.java:257)
  at 
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.loadSystemCoprocessors(CoprocessorHost.java:160)
  at 
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.(RegionCoprocessorHost.java:192)
  at org.apache.hadoop.hbase.regionserver.HRegion.(HRegion.java:701)
  at org.apache.hadoop.hbase.regionserver.HRegion.(HRegion.java:608)
...
Caused by: java.io.IOException: Failed on local exception: java.io.IOException: 
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: 
No valid  credentials provided (Mechanism level: Failed to find any 
Kerberos tgt)]; Host Details : local host is: "hbase-4-4/172.22.66.186"; 
destination host is: "os-r6-  okarus-hbase-4-2.novalocal":8020;
  at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
  at org.apache.hadoop.ipc.Client.call(Client.java:1473)
  at org.apache.hadoop.ipc.Client.call(Client.java:1400)
  at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
  at com.sun.proxy.$Proxy18.mkdirs(Unknown Source)
  at 
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.mkdirs(ClientNamenodeProtocolTranslatorPB.java:555)
  at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
  at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:606)
  at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
  at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
  at com.sun.proxy.$Proxy19.mkdirs(Unknown Source)
  at org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:2775)
  at org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:2746)
  at 
org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:967)
  at 
org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:963)
  at 
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
{code}
The cause was that SecureBulkLoadEndpoint#start tried to create staging dir in 
hdfs as user X but didn't pass authentication.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-14606) TestSecureLoadIncrementalHFiles tests timed out in trunk build on apache

[stack (JIRA)]

stack created HBASE-14606:
-

 Summary: TestSecureLoadIncrementalHFiles tests timed out in trunk 
build on apache
 Key: HBASE-14606
 URL: https://issues.apache.org/jira/browse/HBASE-14606
 Project: HBase
  Issue Type: Bug
Reporter: stack


Opening an issue to keep an eye on these tests. Looking at history, they've 
been failing irregularly over time.

https://builds.apache.org/job/HBase-TRUNK/6907/testReport/

... has a few tests in this suite timing out:

{code}
org.junit.runners.model.TestTimedOutException: test timed out after 6 
milliseconds
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:461)
at 
io.netty.util.concurrent.DefaultPromise.await0(DefaultPromise.java:355)
at 
io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:266)
at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:42)
at 
org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:248)
at 
org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217)
at 
org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:295)
at 
org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.disableTable(MasterProtos.java:60544)
at 
org.apache.hadoop.hbase.client.ConnectionImplementation$2.disableTable(ConnectionImplementation.java:1533)
at 
org.apache.hadoop.hbase.client.HBaseAdmin$10.call(HBaseAdmin.java:1236)
at 
org.apache.hadoop.hbase.client.HBaseAdmin$10.call(HBaseAdmin.java:1230)
at 
org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:118)
at 
org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4115)
at 
org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4108)
at 
org.apache.hadoop.hbase.client.HBaseAdmin.disableTableAsync(HBaseAdmin.java:1229)
at 
org.apache.hadoop.hbase.client.HBaseAdmin.disableTable(HBaseAdmin.java:1188)
at 
org.apache.hadoop.hbase.HBaseTestingUtility.deleteTable(HBaseTestingUtility.java:1936)
at 
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles.runTest(TestLoadIncrementalHFiles.java:275)
at 
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles.runTest(TestLoadIncrementalHFiles.java:230)
at 
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles.runTest(TestLoadIncrementalHFiles.java:217)
at 
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles.testRegionCrossingHFileSplit(TestLoadIncrementalHFiles.java:193)
at 
org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles.testRegionCrossingHFileSplit(TestLoadIncrementalHFiles.java:171)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at 
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
at 
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.lang.Thread.run(Thread.java:744)
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-14603) Disable timestamping of generated Javadocs so they are not all modified by each build

[Misty Stanley-Jones (JIRA)]

Misty Stanley-Jones created HBASE-14603:
---

 Summary: Disable timestamping of generated Javadocs so they are 
not all modified by each build
 Key: HBASE-14603
 URL: https://issues.apache.org/jira/browse/HBASE-14603
 Project: HBase
  Issue Type: Bug
  Components: documentation
Affects Versions: 2.0.0
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Fix For: 2.0.0


If we disable the timestamps (which are in HTML comments), then every Javadoc 
file won't show up as different every time we rebuild the site.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

PoweredByHBase moved, and needs updates

[Misty Stanley-Jones]

Hi all,

We converted the PoweredByHBase page from the old Wiki to
http://hbase.apache.org/poweredbyhbase.html.

In the process of converting it, we noticed that a lot of the info is very
old and probably out of date.

*If your company or project is listed there*, or if you know that an entry
on the page no longer applies, can you please let us know?

*If you are not listed there and would like to*, instructions are on the
page for getting listed.

Posted to both dev@ and user@ for maximum visibility.

Thanks,
Misty

Re: 7885499 "Typo in FamilyFilter" without associated JIRA

[Misty Stanley-Jones]

Done. Sorry for not noticing the commit message.

On Thu, Oct 8, 2015 at 5:27 AM, Nick Dimiduk  wrote:

> Heya folks,
>
> I noticed the above commit is lacking a subject of the proper format.
> What's the JIRA for this change? Mind reverting and re-applying with title
> as JIRA and Subject? Should be something like
>
> $ git revert 7885499
> $ git cherry-pick 7885499
> $ git commit --amend ...
>
> confirm commit, and then
>
> $ git push
>
> I'm happy to do gittery if it's giving you troubles, just let me know the
> JIRA.
>
> Thanks,
> Nick
>

[jira] [Created] (HBASE-14604) Improve MoveCostFunction in StochasticLoadBalancer

[Guanghao Zhang (JIRA)]

Guanghao Zhang created HBASE-14604:
--

 Summary: Improve MoveCostFunction in StochasticLoadBalancer
 Key: HBASE-14604
 URL: https://issues.apache.org/jira/browse/HBASE-14604
 Project: HBase
  Issue Type: Bug
  Components: Balancer
Reporter: Guanghao Zhang
Assignee: Guanghao Zhang


The code in MoveCoseFunction:
{code}
return scale(0, cluster.numRegions + META_MOVE_COST_MULT, moveCost);
{code}
It uses cluster.numRegions + META_MOVE_COST_MULT as the max value when scale 
moveCost to [0,1]. But this should use maxMoves as the max value when cluster 
have a lot of regions.

Assume a cluster have 1 regions, maxMoves is 2500, it only scale moveCost 
to [0, 0.25].

Improve moveCost by use maxMoves.
{code}
return scale(0, Math.min(cluster.numRegions, maxMoves) + META_MOVE_COST_MULT, 
moveCost);
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (HBASE-14603) Disable timestamping of generated Javadocs so they are not all modified by each build

[Misty Stanley-Jones (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Misty Stanley-Jones reopened HBASE-14603:
-

Reverted the commit because it does not actually work. More research is needed.

> Disable timestamping of generated Javadocs so they are not all modified by 
> each build
> -
>
> Key: HBASE-14603
> URL: https://issues.apache.org/jira/browse/HBASE-14603
> Project: HBase
>  Issue Type: Bug
>  Components: documentation
>Affects Versions: 2.0.0
>Reporter: Misty Stanley-Jones
>Assignee: Misty Stanley-Jones
> Fix For: 2.0.0
>
> Attachments: HBASE-14603.patch
>
>
> If we disable the timestamps (which are in HTML comments), then every Javadoc 
> file won't show up as different every time we rebuild the site.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-14609) Can't config all day as OffPeakHours

[Guanghao Zhang (JIRA)]

Guanghao Zhang created HBASE-14609:
--

 Summary: Can't config all day as OffPeakHours
 Key: HBASE-14609
 URL: https://issues.apache.org/jira/browse/HBASE-14609
 Project: HBase
  Issue Type: Bug
Reporter: Guanghao Zhang
Assignee: Guanghao Zhang
Priority: Minor


The offpeak hours is [startHour, endHour) and endhour is exclusive. But endHour 
is not valid when config as 24, so we can't config all day as OffPeakHours.
{code}
  private static boolean isValidHour(int hour) {
return 0 <= hour && hour <= 23; 
  }
{code}
Let endHour=24 is valid or  enable startHour==endHour can fix this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (HBASE-14609) Can't config all day as OffPeakHours

[Vladimir Rodionov (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-14609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladimir Rodionov reopened HBASE-14609:
---

> Can't config all day as OffPeakHours
> 
>
> Key: HBASE-14609
> URL: https://issues.apache.org/jira/browse/HBASE-14609
> Project: HBase
>  Issue Type: Bug
>Reporter: Guanghao Zhang
>Assignee: Guanghao Zhang
>Priority: Minor
>
> The offpeak hours is [startHour, endHour) and endhour is exclusive. But 
> endHour is not valid when config as 24, so we can't config all day as 
> OffPeakHours.
> {code}
>   private static boolean isValidHour(int hour) {
> return 0 <= hour && hour <= 23; 
>   }
> {code}
> Let endHour=24 is valid or  enable startHour==endHour can fix this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-14611) Backport HBASE-14473 (Compute region locality in parallel) to 0.98

[Andrew Purtell (JIRA)]

Andrew Purtell created HBASE-14611:
--

 Summary: Backport HBASE-14473 (Compute region locality in 
parallel) to 0.98
 Key: HBASE-14611
 URL: https://issues.apache.org/jira/browse/HBASE-14611
 Project: HBase
  Issue Type: Task
Reporter: Andrew Purtell
Assignee: Andrew Purtell
 Fix For: 0.98.16


[~eclark] contributed a nice change on HBASE-14473 that scales calculation of 
locality balance cost to larger clusters. I'd like to bring this back to 0.98 
for folks running it on larger clusters. The changes require a partial rewrite 
of RegionLocationFinder hence a backport issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-14610) IntegrationTestRpcClient from HBASE-14535 is failing with Async RPC client

[Enis Soztutar (JIRA)]

Enis Soztutar created HBASE-14610:
-

 Summary: IntegrationTestRpcClient from HBASE-14535 is failing with 
Async RPC client
 Key: HBASE-14610
 URL: https://issues.apache.org/jira/browse/HBASE-14610
 Project: HBase
  Issue Type: Sub-task
Reporter: Enis Soztutar


HBASE-14535 introduces an IT to simulate a running cluster with RPC servers and 
RPC clients doing requests against the servers. 

It passes with the sync client, but fails with async client. Probably we need 
to take a look. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-14607) Convert SupportingProjects wiki page to a site page

[Misty Stanley-Jones (JIRA)]

Misty Stanley-Jones created HBASE-14607:
---

 Summary: Convert SupportingProjects wiki page to a site page
 Key: HBASE-14607
 URL: https://issues.apache.org/jira/browse/HBASE-14607
 Project: HBase
  Issue Type: Sub-task
  Components: documentation
Affects Versions: 2.0.0
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
 Fix For: 2.0.0


https://wiki.apache.org/hadoop/SupportingProjects



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-14612) Export Snapshot command example should mention property instead of "/hbase" in example in HBase Doc

[Anil Gupta (JIRA)]

Anil Gupta created HBASE-14612:
--

 Summary: Export Snapshot command example should mention 
 property instead of "/hbase" in example in HBase Doc
 Key: HBASE-14612
 URL: https://issues.apache.org/jira/browse/HBASE-14612
 Project: HBase
  Issue Type: Bug
  Components: documentation, hbase
Reporter: Anil Gupta
Priority: Minor


In HBase Docs(http://hbase.apache.org/book.html#ops.snapshots), Export Snapshot 
example specified remote dir as: "hdfs://remotehdfs:8020/hbase"
It doesnt really specifies what is the logical meaning of that directory. 
After struggling with trying to clone snapshot on remote cluster: 
http://search-hadoop.com/m/YGbbuzfA42JAwbh1
I analyzed that export dir should be  . I think, we should 
specify this requirement in HBase Snapshot docs: 
http://hbase.apache.org/book.html#ops.snapshots

PS: CDH uses /hbase while HDP uses /apps/hbase/data. Also, user can configure 
any other directory as hbase.rootdir. 
Hence, the documentation should not say "/hbase".




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-14613) Remove MemStoreChunkPool?

[Lars Hofhansl (JIRA)]

Lars Hofhansl created HBASE-14613:
-

 Summary: Remove MemStoreChunkPool?
 Key: HBASE-14613
 URL: https://issues.apache.org/jira/browse/HBASE-14613
 Project: HBase
  Issue Type: Brainstorming
Reporter: Lars Hofhansl
Priority: Minor


I just stumbled across MemStoreChunkPool. The idea behind is to reuse chunks of 
allocations rather than letting the GC handle this.

Now, it's off by default, and it seems to me to be of dubious value. I'd 
recommend just removing it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)