date:20210818

Re: [DISCUSS] About using masters as ConnectionRegistry endpoint

2021-08-18 Thread Duo Zhang

We already have a connection to zookeeper at region server side, and
we already watched a lot of things, so watching one more znode does
not add too much pressure to zk.
And I've been implementing a framework to replicate meta location
without zookeeper, this could reduce the pressure on zookeeper a lot.

On the data freshness, this is a problem. But anyway, in the old
master registry implementation, if you go to backup masters, you still
can't always get the newest data. This is a problem we should try to
do better, like using the above framework to replicate the
modification quickly, without involving active master in the normal
read/write path.

Nick Dimiduk  于2021年8月19日周四 上午2:38写道：
>
> I recall that there was a concern about data freshness, that the masters
> were all tracking the necessary state and all of them would always be
> up-to-date. Region servers were not already in this business, and doing so
> would put undue burden on ZooKeeper.
>
> Perhaps I miss-remember, or the discussion evolved since I last checked?
>
> On Sun, Aug 15, 2021 at 5:20 PM Bharath Vissapragada 
> wrote:
>
> > Thanks, Duo. I commented on the PR but want to respond here too to kick
> > start the discussion and in case anyone else has different viewpoints.
> >
> > I agree that the original decision of inlining active masters needs to be
> > corrected going forward. I vote for the proposal to deprecate the master
> > based registry in 2.5.0 in favor of a "RegionServer" based registry and
> > remove it completely in 4.0.0.  IMO we should *not *expose any opt-in
> > configuration to allow masters as that violates the design principle that
> > we all agreed upon and instead only use region servers as the registry
> > hosting services.
> >
> >
> > On Fri, Aug 13, 2021 at 7:59 PM 张铎(Duo Zhang) 
> > wrote:
> >
> > > In HBASE-18095, the community provided a new way to get the registry
> > > information of a cluster, without touching ZooKeeper. The decision at
> > > that time was to use masters(including active and backup masters) as
> > > the connection registry endpoint.
> > >
> > > Later, when discussing how to implement splittable meta, we planned to
> > > make use of this framework to hide the actual ROOT table
> > > implementation. But then we found out that the approach of using
> > > masters as connection registry, violates one of our tendencies that we
> > > do not want to inline masters, especially the active master in the
> > > normal read/write path.
> > >
> > > The several sub tasks of HBASE-26149 aims to solve this problem. We
> > > all agree that by default, we should not inline masters, but there are
> > > some conflicts on whether to still allow end users to configure that
> > > they want to use masters as registry endpoints, as it is a feature
> > > which has already been published in our releases.
> > >
> > > There are some discussions in the PR for HBASE-26172
> > > https://github.com/apache/hbase/pull/3566#discussion_r684494130
> > >
> > > Feel free to post your opinion here.
> > >
> > > Thanks.
> > >
> >

Re: [DISCUSS] About using masters as ConnectionRegistry endpoint

2021-08-18 Thread Bharath Vissapragada

Are you referring to meta region location(s) cache? If so, yes we will have
more network chatter accessing those znode(s) whenever meta (replicas) move
around (what happens in split-meta is still under discussion as you
probably are already following, but it is likely that we will still use ZK
as the notification layer, so your question still holds). This design
offsets the reduction in client-ZK access load as clients don't hit ZK
anymore, so theoretically it is not any worse than what we have today. On
the plus side, as I mentioned we remove masters from the picture.

One simple improvement is to restrict the no. of regionservers maintaining
that cache this information (HBASE-26173
), we don't need every
RS to do that and we can maintain a minimum # (or a %) that can choose to
do that.

On Wed, Aug 18, 2021 at 11:39 AM Nick Dimiduk  wrote:

> I recall that there was a concern about data freshness, that the masters
> were all tracking the necessary state and all of them would always be
> up-to-date. Region servers were not already in this business, and doing so
> would put undue burden on ZooKeeper.
>
> Perhaps I miss-remember, or the discussion evolved since I last checked?
>
> On Sun, Aug 15, 2021 at 5:20 PM Bharath Vissapragada 
> wrote:
>
> > Thanks, Duo. I commented on the PR but want to respond here too to kick
> > start the discussion and in case anyone else has different viewpoints.
> >
> > I agree that the original decision of inlining active masters needs to be
> > corrected going forward. I vote for the proposal to deprecate the master
> > based registry in 2.5.0 in favor of a "RegionServer" based registry and
> > remove it completely in 4.0.0.  IMO we should *not *expose any opt-in
> > configuration to allow masters as that violates the design principle that
> > we all agreed upon and instead only use region servers as the registry
> > hosting services.
> >
> >
> > On Fri, Aug 13, 2021 at 7:59 PM 张铎(Duo Zhang) 
> > wrote:
> >
> > > In HBASE-18095, the community provided a new way to get the registry
> > > information of a cluster, without touching ZooKeeper. The decision at
> > > that time was to use masters(including active and backup masters) as
> > > the connection registry endpoint.
> > >
> > > Later, when discussing how to implement splittable meta, we planned to
> > > make use of this framework to hide the actual ROOT table
> > > implementation. But then we found out that the approach of using
> > > masters as connection registry, violates one of our tendencies that we
> > > do not want to inline masters, especially the active master in the
> > > normal read/write path.
> > >
> > > The several sub tasks of HBASE-26149 aims to solve this problem. We
> > > all agree that by default, we should not inline masters, but there are
> > > some conflicts on whether to still allow end users to configure that
> > > they want to use masters as registry endpoints, as it is a feature
> > > which has already been published in our releases.
> > >
> > > There are some discussions in the PR for HBASE-26172
> > > https://github.com/apache/hbase/pull/3566#discussion_r684494130
> > >
> > > Feel free to post your opinion here.
> > >
> > > Thanks.
> > >
> >
>

Re: [DISCUSS] About using masters as ConnectionRegistry endpoint

2021-08-18 Thread Nick Dimiduk

I recall that there was a concern about data freshness, that the masters
were all tracking the necessary state and all of them would always be
up-to-date. Region servers were not already in this business, and doing so
would put undue burden on ZooKeeper.

Perhaps I miss-remember, or the discussion evolved since I last checked?

On Sun, Aug 15, 2021 at 5:20 PM Bharath Vissapragada 
wrote:

> Thanks, Duo. I commented on the PR but want to respond here too to kick
> start the discussion and in case anyone else has different viewpoints.
>
> I agree that the original decision of inlining active masters needs to be
> corrected going forward. I vote for the proposal to deprecate the master
> based registry in 2.5.0 in favor of a "RegionServer" based registry and
> remove it completely in 4.0.0.  IMO we should *not *expose any opt-in
> configuration to allow masters as that violates the design principle that
> we all agreed upon and instead only use region servers as the registry
> hosting services.
>
>
> On Fri, Aug 13, 2021 at 7:59 PM 张铎(Duo Zhang) 
> wrote:
>
> > In HBASE-18095, the community provided a new way to get the registry
> > information of a cluster, without touching ZooKeeper. The decision at
> > that time was to use masters(including active and backup masters) as
> > the connection registry endpoint.
> >
> > Later, when discussing how to implement splittable meta, we planned to
> > make use of this framework to hide the actual ROOT table
> > implementation. But then we found out that the approach of using
> > masters as connection registry, violates one of our tendencies that we
> > do not want to inline masters, especially the active master in the
> > normal read/write path.
> >
> > The several sub tasks of HBASE-26149 aims to solve this problem. We
> > all agree that by default, we should not inline masters, but there are
> > some conflicts on whether to still allow end users to configure that
> > they want to use masters as registry endpoints, as it is a feature
> > which has already been published in our releases.
> >
> > There are some discussions in the PR for HBASE-26172
> > https://github.com/apache/hbase/pull/3566#discussion_r684494130
> >
> > Feel free to post your opinion here.
> >
> > Thanks.
> >
>

Re: Hbase export is very slow - help needed

2021-08-18 Thread Nguyen, Tai Van (EXT - VN)

Hi HBase Team

Image can see here :

  *   Export with single regionserver: https://imgur.com/86wSUMV
  *   Export with two regionservers: https://imgur.com/a/XMovlZx

Log show about time was:

root@solaltiplano-track4-master:~/hbase-exporting/latest# cat 
hbase_export_compress_default.log | grep export
Starting hbase export at Fri Jun 11 12:22:46 UTC 2021
tsdb table exported in  6279 seconds
tsdb-meta table exported in  6 seconds
tsdb-tree table exported in  7 seconds
tsdb-uid table exported in  90 seconds
Ending hbase export at Fri Jun 11 14:09:08 UTC 2021


  *

Thanks,
Tai



From: Mathews, Jacob 1. (Nokia - IN/Bangalore) 
Sent: Monday, August 16, 2021 6:47 PM
To: Nguyen, Tai Van (EXT - VN) 
Subject: FW: Hbase export is very slow - help needed






From: Mathews, Jacob 1. (Nokia - IN/Bangalore)
Sent: Friday, August 6, 2021 12:38 PM
To: user@hbase.apache.org
Subject: Hbase export is very slow - help needed



Hi HBase team,



We are trying to use Hbase export mentioned here: 
http://hbase.apache.org/book.html#export

But it is happening sequentially row by row as seen from the logs.

we tried many options of the Hbase export, but all were taking long time.



Backup folder contents size:



bash-4.2$ du -kh

16K ./tsdb-tree

16K ./tsdb-meta

60M   ./tsdb-uid

5.9G   ./tsdb

6.0G   .



took around 104 minutes for 6gb compressed data.



Is there a way we can parallelise this and improve the export time.



Below are the charts from Hbase .



Export with single regionserver:



[cid:image001.jpg@01D78ABF.E9E6C280]



Export with two regionservers:



[cid:image002.jpg@01D78ABF.E9E6C280]



Scaling the HBase Region server also did not help, the export still happens 
sequentially.



Thanks

Jacob Mathews

How can I connect hbase with kerberos

2021-08-18 Thread igyu

String krb5Path = "D:\\file\\krb5.conf";
String principal = "jztwk/had...@join.com";
String keytabPath = "D:\\file\\jztwk.keytab";
System.setProperty("java.security.krb5.conf", krb5Path);
Configuration conf = new Configuration();
conf.set("hadoop.security.authentication", "Kerberos");
UserGroupInformation.setConfiguration(conf);
UserGroupInformation.loginUserFromKeytab(principal, keytabPath);

Configuration Hconf = HBaseConfiguration.create();


//Hconf.set(HConstants.ZOOKEEPER_QUORUM, 
"192.168.0.10:2181,192.168.0.238:2181");
Hconf.set(HConstants.ZOOKEEPER_QUORUM, 
"bigdser5:2181,bigdser2:2181,bigdser3:2181");

HBaseAdmin.available(Hconf);
UserGroupInformation.loginUserFromKeytab(principal, keytabPath); is OK
but HBaseAdmin.available(Hconf); throw exception

org.apache.hadoop.hbase.MasterNotRunningException: 
org.apache.hadoop.hbase.MasterNotRunningException: java.io.IOException: Call to 
bigdser2/10.3.87.24:16000 failed on local exception: java.io.IOException: 
你的主机中的软件中止了一个已建立的连接。
at 
org.apache.hadoop.hbase.client.ConnectionImplementation.isMasterRunning(ConnectionImplementation.java:585)
at org.apache.hadoop.hbase.client.HBaseAdmin.available(HBaseAdmin.java:2366)
at App.main(App.java:31)
Caused by: org.apache.hadoop.hbase.MasterNotRunningException: 
java.io.IOException: Call to bigdser2/10.3.87.24:16000 failed on local 
exception: java.io.IOException: 你的主机中的软件中止了一个已建立的连接。
at 
org.apache.hadoop.hbase.client.ConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionImplementation.java:1175)
at 
org.apache.hadoop.hbase.client.ConnectionImplementation.getKeepAliveMasterService(ConnectionImplementation.java:1234)
at 
org.apache.hadoop.hbase.client.ConnectionImplementation.isMasterRunning(ConnectionImplementation.java:583)
... 2 more
Caused by: java.io.IOException: Call to bigdser2/10.3.87.24:16000 failed on 
local exception: java.io.IOException: 你的主机中的软件中止了一个已建立的连接。
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:221)
at 
org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:390)
at 
org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:95)
at 
org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:410)
at 
org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:406)
at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:103)
at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:118)
at 
org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.cleanupCalls(NettyRpcDuplexHandler.java:203)
at 
org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.exceptionCaught(NettyRpcDuplexHandler.java:220)
at 
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:282)
at 
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:261)
at 
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:253)
at 
org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:131)
at 
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:282)
at 
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:261)
at 
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:253)
at 
org.apache.hbase.thirdparty.io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:131)
at 
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:282)
at 
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:261)
at 
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:253)
at 
org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1375)
at 
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:282)
at

Re: Re: I can't connect hbase

2021-08-18 Thread igyu

Hconf.set("hbase.regionserver.kerberos.principal","hbase/_h...@join.com");
Hconf.set("hbase.master.kerberos.principal","hbase/_h...@join.com");
Hconf.set("dfs.namenode.kerberos.principal", "hdfs/_h...@join.com");
I add this,but aslo get error

igyu

From: igyu
Date: 2021-08-18 15:19
To: user
Subject: Re: I can't connect hbase
UserGroupInformation.loginUserFromKeytab(principal, keytabPath) is OK

HBaseAdmin.available(Hconf) throw exception

igyu
From: igyu
Date: 2021-08-18 14:55
To: user
Subject: I can't connect hbase
try {
String krb5Path = "D:\\file\\krb5.conf";
String principal = "jztwk/had...@join.com";
String keytabPath = "D:\\file\\jztwk.keytab";
System.setProperty("java.security.krb5.conf", krb5Path);
Configuration conf = new Configuration();
conf.set("hadoop.security.authentication", "Kerberos");
UserGroupInformation.setConfiguration(conf);
Configuration Hconf = HBaseConfiguration.create();
Hconf.set(HConstants.ZOOKEEPER_QUORUM, 
"bigdser5:2181,bigdser2:2181,bigdser3:2181");
//conf.set(HConstants.ZOOKEEPER_CLIENT_PORT, port);
Hconf.set("hadoop.security.authentication", "Kerberos");
Hconf.set("hbase.security.authentication", "Kerberos");
//Hconf.set("dfs.namenode.kerberos.principal", 
"hdfs/_h...@join.com");
//Hconf.set(HConstants.ZK_CLIENT_KEYTAB_FILE, keytabPath);
//Hconf.set(HConstants.ZK_CLIENT_KERBEROS_PRINCIPAL, principal);
//Hconf.set(HConstants.ZK_SERVER_KEYTAB_FILE, keytabPath);
//Hconf.set(HConstants.ZK_SERVER_KERBEROS_PRINCIPAL, principal);
UserGroupInformation.loginUserFromKeytab(principal, keytabPath);
HBaseAdmin.available(Hconf);

}catch (Exception err){
err.printStackTrace();
}
I get a error
org.apache.hadoop.hbase.MasterNotRunningException: 
org.apache.hadoop.hbase.MasterNotRunningException: java.io.IOException: Call to 
bigdser2/10.3.87.24:16000 failed on local exception: java.io.IOException: 
Failed to specify server's Kerberos principal name
at 
org.apache.hadoop.hbase.client.ConnectionImplementation.isMasterRunning(ConnectionImplementation.java:585)
at org.apache.hadoop.hbase.client.HBaseAdmin.available(HBaseAdmin.java:2366)
at App$1.run(App.java:40)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:360)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1855)
at App.main(App.java:36)
Caused by: org.apache.hadoop.hbase.MasterNotRunningException: 
java.io.IOException: Call to bigdser2/10.3.87.24:16000 failed on local 
exception: java.io.IOException: Failed to specify server's Kerberos principal 
name
at 
org.apache.hadoop.hbase.client.ConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionImplementation.java:1175)
at 
org.apache.hadoop.hbase.client.ConnectionImplementation.getKeepAliveMasterService(ConnectionImplementation.java:1234)
at 
org.apache.hadoop.hbase.client.ConnectionImplementation.isMasterRunning(ConnectionImplementation.java:583)
... 6 more
Caused by: java.io.IOException: Call to bigdser2/10.3.87.24:16000 failed on 
local exception: java.io.IOException: Failed to specify server's Kerberos 
principal name
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:221)
at 
org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:390)
at 
org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:95)
at 
org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:410)
at 
org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:406)
at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:103)
at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:118)
at 
org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:92)
at 
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326)
at 
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312)
at 
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304)
at 
org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1426)
at

Re: I can't connect hbase

2021-08-18 Thread igyu

UserGroupInformation.loginUserFromKeytab(principal, keytabPath) is OK

 HBaseAdmin.available(Hconf) throw exception

igyu

From: igyu
Date: 2021-08-18 14:55
To: user
Subject: I can't connect hbase
try {
String krb5Path = "D:\\file\\krb5.conf";
String principal = "jztwk/had...@join.com";
String keytabPath = "D:\\file\\jztwk.keytab";

System.setProperty("java.security.krb5.conf", krb5Path);
Configuration conf = new Configuration();
conf.set("hadoop.security.authentication", "Kerberos");
UserGroupInformation.setConfiguration(conf);

Configuration Hconf = HBaseConfiguration.create();

Hconf.set(HConstants.ZOOKEEPER_QUORUM, 
"bigdser5:2181,bigdser2:2181,bigdser3:2181");
//conf.set(HConstants.ZOOKEEPER_CLIENT_PORT, port);
Hconf.set("hadoop.security.authentication", "Kerberos");
Hconf.set("hbase.security.authentication", "Kerberos");
//Hconf.set("dfs.namenode.kerberos.principal", 
"hdfs/_h...@join.com");
//Hconf.set(HConstants.ZK_CLIENT_KEYTAB_FILE, keytabPath);
//Hconf.set(HConstants.ZK_CLIENT_KERBEROS_PRINCIPAL, principal);
//Hconf.set(HConstants.ZK_SERVER_KEYTAB_FILE, keytabPath);
//Hconf.set(HConstants.ZK_SERVER_KERBEROS_PRINCIPAL, principal);

UserGroupInformation.loginUserFromKeytab(principal, keytabPath);

HBaseAdmin.available(Hconf);

}catch (Exception err){
err.printStackTrace();
}
I get a error

org.apache.hadoop.hbase.MasterNotRunningException: 
org.apache.hadoop.hbase.MasterNotRunningException: java.io.IOException: Call to 
bigdser2/10.3.87.24:16000 failed on local exception: java.io.IOException: 
Failed to specify server's Kerberos principal name
at 
org.apache.hadoop.hbase.client.ConnectionImplementation.isMasterRunning(ConnectionImplementation.java:585)
at org.apache.hadoop.hbase.client.HBaseAdmin.available(HBaseAdmin.java:2366)
at App$1.run(App.java:40)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:360)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1855)
at App.main(App.java:36)
Caused by: org.apache.hadoop.hbase.MasterNotRunningException: 
java.io.IOException: Call to bigdser2/10.3.87.24:16000 failed on local 
exception: java.io.IOException: Failed to specify server's Kerberos principal 
name
at 
org.apache.hadoop.hbase.client.ConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionImplementation.java:1175)
at 
org.apache.hadoop.hbase.client.ConnectionImplementation.getKeepAliveMasterService(ConnectionImplementation.java:1234)
at 
org.apache.hadoop.hbase.client.ConnectionImplementation.isMasterRunning(ConnectionImplementation.java:583)
... 6 more
Caused by: java.io.IOException: Call to bigdser2/10.3.87.24:16000 failed on 
local exception: java.io.IOException: Failed to specify server's Kerberos 
principal name
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:221)
at 
org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:390)
at 
org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:95)
at 
org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:410)
at 
org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:406)
at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:103)
at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:118)
at 
org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:92)
at 
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326)
at 
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312)
at 
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304)
at 
org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1426)
at 
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326)
at 
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312)
at

I can't connect hbase

2021-08-18 Thread igyu

try {
String krb5Path = "D:\\file\\krb5.conf";
String principal = "jztwk/had...@join.com";
String keytabPath = "D:\\file\\jztwk.keytab";

System.setProperty("java.security.krb5.conf", krb5Path);
Configuration conf = new Configuration();
conf.set("hadoop.security.authentication", "Kerberos");
UserGroupInformation.setConfiguration(conf);

Configuration Hconf = HBaseConfiguration.create();

Hconf.set(HConstants.ZOOKEEPER_QUORUM, 
"bigdser5:2181,bigdser2:2181,bigdser3:2181");
//conf.set(HConstants.ZOOKEEPER_CLIENT_PORT, port);
Hconf.set("hadoop.security.authentication", "Kerberos");
Hconf.set("hbase.security.authentication", "Kerberos");
//Hconf.set("dfs.namenode.kerberos.principal", 
"hdfs/_h...@join.com");
//Hconf.set(HConstants.ZK_CLIENT_KEYTAB_FILE, keytabPath);
//Hconf.set(HConstants.ZK_CLIENT_KERBEROS_PRINCIPAL, principal);
//Hconf.set(HConstants.ZK_SERVER_KEYTAB_FILE, keytabPath);
//Hconf.set(HConstants.ZK_SERVER_KERBEROS_PRINCIPAL, principal);

UserGroupInformation.loginUserFromKeytab(principal, keytabPath);

HBaseAdmin.available(Hconf);

}catch (Exception err){
err.printStackTrace();
}
I get a error

org.apache.hadoop.hbase.MasterNotRunningException: 
org.apache.hadoop.hbase.MasterNotRunningException: java.io.IOException: Call to 
bigdser2/10.3.87.24:16000 failed on local exception: java.io.IOException: 
Failed to specify server's Kerberos principal name
at 
org.apache.hadoop.hbase.client.ConnectionImplementation.isMasterRunning(ConnectionImplementation.java:585)
at org.apache.hadoop.hbase.client.HBaseAdmin.available(HBaseAdmin.java:2366)
at App$1.run(App.java:40)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:360)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1855)
at App.main(App.java:36)
Caused by: org.apache.hadoop.hbase.MasterNotRunningException: 
java.io.IOException: Call to bigdser2/10.3.87.24:16000 failed on local 
exception: java.io.IOException: Failed to specify server's Kerberos principal 
name
at 
org.apache.hadoop.hbase.client.ConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionImplementation.java:1175)
at 
org.apache.hadoop.hbase.client.ConnectionImplementation.getKeepAliveMasterService(ConnectionImplementation.java:1234)
at 
org.apache.hadoop.hbase.client.ConnectionImplementation.isMasterRunning(ConnectionImplementation.java:583)
... 6 more
Caused by: java.io.IOException: Call to bigdser2/10.3.87.24:16000 failed on 
local exception: java.io.IOException: Failed to specify server's Kerberos 
principal name
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:221)
at 
org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:390)
at 
org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:95)
at 
org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:410)
at 
org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:406)
at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:103)
at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:118)
at 
org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:92)
at 
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326)
at 
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312)
at 
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304)
at 
org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1426)
at 
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326)
at 
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312)
at 
org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:924)
at 
org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:179)
at

Re: [DISCUSS] About using masters as ConnectionRegistry endpoint

Re: [DISCUSS] About using masters as ConnectionRegistry endpoint

Re: [DISCUSS] About using masters as ConnectionRegistry endpoint

Re: Hbase export is very slow - help needed

How can I connect hbase with kerberos

Re: Re: I can't connect hbase

Re: I can't connect hbase

I can't connect hbase

8 matches

Site Navigation

Mail list logo

Footer information