[
https://issues.apache.org/jira/browse/KYLIN-957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14726881#comment-14726881
]
Shaofeng SHI commented on KYLIN-957:
------------------------------------
BuildCubeWithEngineTest is failed at bulk load step, got a HDFS File permission
error:
{code}
L4J [2015-09-01
23:36:01,440][ERROR][org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles] -
Encountered unrecoverable error from region server, additional details: row ''
on table 'KYLIN_ZV010J7E12' at
region=KYLIN_ZV010J7E12,,1441173220820.91f46e3da6fa2032b61c008840b05d73.,
hostname=sandbox,60020,1432800398732, seqNum=2
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
attempts=35, exceptions:
Tue Sep 01 22:55:38 GMT-07:00 2015,
org.apache.hadoop.hbase.client.RpcRetryingCaller@1493b72d, java.io.IOException:
java.io.IOException: Exception in rename
at
org.apache.hadoop.hbase.regionserver.HRegionFileSystem.rename(HRegionFileSystem.java:977)
at
org.apache.hadoop.hbase.regionserver.HRegionFileSystem.commitStoreFile(HRegionFileSystem.java:377)
at
org.apache.hadoop.hbase.regionserver.HRegionFileSystem.bulkLoadStoreFile(HRegionFileSystem.java:451)
at
org.apache.hadoop.hbase.regionserver.HStore.bulkLoadHFile(HStore.java:750)
at
org.apache.hadoop.hbase.regionserver.HRegion.bulkLoadHFiles(HRegion.java:4894)
at
org.apache.hadoop.hbase.regionserver.HRegion.bulkLoadHFiles(HRegion.java:4799)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.bulkLoadHFile(HRegionServer.java:3377)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29996)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2078)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108)
at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hadoop.security.AccessControlException: Permission
denied: user=hbase, access=WRITE,
inode="/kylin/kylin_metadata/kylin-ae068aac-a858-4f2e-8030-c3fb3b514d5a/test_kylin_cube_with_slr_empty/hfile/F1":jenkins:hdfs:drwxr-xr-x
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkFsPermission(FSPermissionChecker.java:271)
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:257)
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:238)
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:182)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:6795)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.renameToInternal(FSNamesystem.java:4052)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.renameToInt(FSNamesystem.java:4022)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.renameTo(FSNamesystem.java:3986)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.rename(NameNodeRpcServer.java:755)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.rename(ClientNamenodeProtocolServerSideTranslatorPB.java:567)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2039)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2035)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2033)
at sun.reflect.GeneratedConstructorAccessor581.newInstance(Unknown
Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
at
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)
at org.apache.hadoop.hdfs.DFSClient.rename(DFSClient.java:1866)
at
org.apache.hadoop.hdfs.DistributedFileSystem.rename(DistributedFileSystem.java:575)
at
org.apache.hadoop.fs.FilterFileSystem.rename(FilterFileSystem.java:226)
at
org.apache.hadoop.hbase.regionserver.HRegionFileSystem.rename(HRegionFileSystem.java:969)
{code}
Any idea?
> Support HBase in a separate cluster
> -----------------------------------
>
> Key: KYLIN-957
> URL: https://issues.apache.org/jira/browse/KYLIN-957
> Project: Kylin
> Issue Type: Improvement
> Components: General
> Affects Versions: v0.7.2
> Reporter: Dayue Gao
> Assignee: Shaofeng SHI
> Labels: features
> Fix For: v1.1
>
> Attachments: KYLIN-957-0.7-staging-v2.patch,
> KYLIN-957-0.7-staging.patch
>
>
> Currently Kylin assumes that HBase is deployed in the same cluster where Hive
> tables resides, which is not necessarily the case.
> We should support Kylin to write cubes to HBase in another cluster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
