Please find my reply Inline. On Fri, Dec 19, 2014 at 12:08 PM, Imran Mohammed <[email protected]> wrote: > > Thanks Anil, > > Appreciate your effort in providing direction on this problem. > > Please find the error along with command used . > > I have few more questions , *10.248.16.79 is my Hbase client and one of > the zookeeper quroum cluster , there are 2 more of them. Do I need to have > access to all these ZK cluster machines ? and is it required to consider > them all during sqlline invoke ?* > Anil: Yes, you need to have access to all ZK nodes. Yes, it is required to provide hostnames of all the ZK nodes.
> > > *I looked at the links shared and probably they are pointing in direction > of firewall turnoff . Is it must ? If yes I will check with sys admin to > look into it.* > Anil: The client needs to have access to all the machines on the cluster. Selective access to machines might create problems like this. Can you make sure that you can access all the machines on all the required ports. Please try again after following all the above suggestions. Also what's your HBase version? > > *[osv9005@htnalpapphdp01 bin]$ ./sqlline.py > 10.248.16.79:2181:/hbase:[email protected]:/home/osv9005/osv9005.keytab* > Setting property: [isolation, TRANSACTION_READ_COMMITTED] > issuing: !connect jdbc:phoenix:10.248.16.79:2181 > :/hbase:[email protected]:/home/osv9005/osv9005.keytab > none none org.apache.phoenix.jdbc.PhoenixDriver > Connecting to jdbc:phoenix:10.248.16.79:2181 > :/hbase:[email protected]:/home/osv9005/osv9005.keytab > 14/12/19 13:58:11 WARN util.NativeCodeLoader: Unable to load native-hadoop > library for your platform... using builtin-java classes where applicable > 14/12/19 13:58:28 ERROR zookeeper.RecoverableZooKeeper: ZooKeeper exists > failed after 4 attempts > 14/12/19 13:58:28 ERROR zookeeper.ZooKeeperWatcher: > hconnection-0x32ee5169, quorum=10.248.16.79:2181, baseZNode=/hbase > Received unexpected KeeperException, re-throwing exception > org.apache.zookeeper.KeeperException$ConnectionLossException: > KeeperErrorCode = ConnectionLoss for /hbase/hbaseid > at > org.apache.zookeeper.KeeperException.create(KeeperException.java:99) > at > org.apache.zookeeper.KeeperException.create(KeeperException.java:51) > at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045) > at > org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:199) > at > org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:479) > at > org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:65) > at > org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:83) > at > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.retrieveClusterId(HConnectionManager.java:858) > at > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnectionManager.java:663) > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:526) > at > org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:415) > at > org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:310) > at > org.apache.phoenix.query.HConnectionFactory$HConnectionFactoryImpl.createConnection(HConnectionFactory.java:47) > at > org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection(ConnectionQueryServicesImpl.java:254) > at > org.apache.phoenix.query.ConnectionQueryServicesImpl.access$300(ConnectionQueryServicesImpl.java:153) > at > org.apache.phoenix.query.ConnectionQueryServicesImpl$9.call(ConnectionQueryServicesImpl.java:1512) > at > org.apache.phoenix.query.ConnectionQueryServicesImpl$9.call(ConnectionQueryServicesImpl.java:1491) > at > org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:77) > at > org.apache.phoenix.query.ConnectionQueryServicesImpl.init(ConnectionQueryServicesImpl.java:1491) > at > org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:162) > at > org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.connect(PhoenixEmbeddedDriver.java:126) > at > org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:133) > at sqlline.SqlLine$DatabaseConnection.connect(SqlLine.java:4650) > at > sqlline.SqlLine$DatabaseConnection.getConnection(SqlLine.java:4701) > at sqlline.SqlLine$Commands.connect(SqlLine.java:3942) > at sqlline.SqlLine$Commands.connect(SqlLine.java:3851) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > sqlline.SqlLine$ReflectiveCommandHandler.execute(SqlLine.java:2810) > at sqlline.SqlLine.dispatch(SqlLine.java:817) > at sqlline.SqlLine.initArgs(SqlLine.java:633) > at sqlline.SqlLine.begin(SqlLine.java:680) > at sqlline.SqlLine.mainWithInputRedirection(SqlLine.java:441) > at sqlline.SqlLine.main(SqlLine.java:424) > *14/12/19 13:58:28 WARN client.ZooKeeperRegistry: Can't retrieve clusterId > from Zookeeper* > > > On Fri, Dec 19, 2014 at 12:31 PM, anil gupta <[email protected]> > wrote: >> >> What is the command you are using to invoke sqlline? Also, can you share >> Phoenix client logs? Do you see any logs saying "connecting to a secure >> cluster"? >> Also, regarding " ConnectionLoss for /hbase/hbaseid", have a look at >> >> http://stackoverflow.com/questions/6153560/hbase-client-connectionloss-for-hbase-error >> >> http://search-hadoop.com/m/Rx0v22KOVBY/ConnectionLoss+for+%252Fhbase%252Fhbaseid&subj=Zookeeper+issue+ConnectionLoss+for+hbase+hbaseid+ >> >> >> On Fri, Dec 19, 2014 at 7:21 AM, Imran Mohammed <[email protected]> >> wrote: >>> >>> On sqlline.py I'm getting the error below , with respect to connecting >>> hbase directly yes i'm able to . >>> >>> 14/12/19 09:18:54 ERROR zookeeper.RecoverableZooKeeper: ZooKeeper exists >>> failed after 4 attempts >>> 14/12/19 09:18:54 ERROR zookeeper.ZooKeeperWatcher: >>> hconnection-0x32ee5169, quorum=10.248.16.79:2181, baseZNode=/hbase >>> Received unexpected KeeperException, re-throwing exception >>> org.apache.zookeeper.KeeperException$ConnectionLossException: >>> KeeperErrorCode = ConnectionLoss for /hbase/hbaseid >>> at >>> org.apache.zookeeper.KeeperException.create(KeeperException.java:99) >>> at >>> org.apache.zookeeper.KeeperException.create(KeeperException.java:51) >>> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045) >>> at >>> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:199) >>> at >>> org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:479) >>> at >>> org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:65) >>> at >>> org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:83) >>> at >>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.retrieveClusterId(HConnectionManager.java:858) >>> at >>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnectionManager.java:663) >>> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native >>> Method) >>> at >>> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) >>> at >>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) >>> at >>> java.lang.reflect.Constructor.newInstance(Constructor.java:526) >>> at >>> org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:415) >>> at >>> org.apache.hadoop.hbase.client.HConnectionManager.createConnection(HConnectionManager.java:310) >>> at >>> org.apache.phoenix.query.HConnectionFactory$HConnectionFactoryImpl.createConnection(HConnectionFactory.java:47) >>> at >>> org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection(ConnectionQueryServicesImpl.java:254) >>> at >>> org.apache.phoenix.query.ConnectionQueryServicesImpl.access$300(ConnectionQueryServicesImpl.java:153) >>> at >>> org.apache.phoenix.query.ConnectionQueryServicesImpl$9.call(ConnectionQueryServicesImpl.java:1512) >>> at >>> org.apache.phoenix.query.ConnectionQueryServicesImpl$9.call(ConnectionQueryServicesImpl.java:1491) >>> at >>> org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:77) >>> at >>> org.apache.phoenix.query.ConnectionQueryServicesImpl.init(ConnectionQueryServicesImpl.java:1491) >>> at >>> org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:162) >>> at >>> org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.connect(PhoenixEmbeddedDriver.java:126) >>> at >>> org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:133) >>> at sqlline.SqlLine$DatabaseConnection.connect(SqlLine.java:4650) >>> at >>> sqlline.SqlLine$DatabaseConnection.getConnection(SqlLine.java:4701) >>> at sqlline.SqlLine$Commands.connect(SqlLine.java:3942) >>> at sqlline.SqlLine$Commands.connect(SqlLine.java:3851) >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>> at java.lang.reflect.Method.invoke(Method.java:606) >>> at >>> sqlline.SqlLine$ReflectiveCommandHandler.execute(SqlLine.java:2810) >>> at sqlline.SqlLine.dispatch(SqlLine.java:817) >>> at sqlline.SqlLine.initArgs(SqlLine.java:633) >>> at sqlline.SqlLine.begin(SqlLine.java:680) >>> at sqlline.SqlLine.mainWithInputRedirection(SqlLine.java:441) >>> at sqlline.SqlLine.main(SqlLine.java:424) >>> >>> >>> On Thu, Dec 18, 2014 at 5:43 PM, anil gupta <[email protected]> >>> wrote: >>>> >>>> Hi Imran, >>>> >>>> Since 4.1 release, connecting to a secure HBase cluster is supported by >>>> Phoenix. Can you try using sqlline and invoke it by command: sqlline.py >>>> jdbc:phoenix:<quorom>:<port>:<rootNode>:<principal>:<keytab> >>>> Please report back the outcome. Also, are you able to connect to hbase >>>> from the node without using Phoenix? Maybe, there is a firewall or kerberos >>>> client is not setup. >>>> >>>> Thanks, >>>> Anil Gupta >>>> >>>> On Thu, Dec 18, 2014 at 2:45 PM, Alex Kamil <[email protected]> >>>> wrote: >>>>> >>>>> Imran, Are you able to connect with sqlline following these >>>>> instructions >>>>> http://bigdatanoob.blogspot.com/2013/09/connect-phoenix-to-secure-hbase-cluster.html >>>>> ? >>>>> also see previous threads >>>>> <https://www.google.com/webhp?sourceid=chrome-instant&rlz=1CAASUA_enUS609US609&ion=1&espv=2&ie=UTF-8&safe=active&ssui=on#q=apache%2Bhbase%2Bphoenix%2Bsecure%2Bkerberos+&safe=active&ssui=on> >>>>> for debugging tips >>>>> >>>>> >>>>> On Thu, Dec 18, 2014 at 11:37 AM, Imran Mohammed < >>>>> [email protected]> wrote: >>>>>> >>>>>> Hey, >>>>>> >>>>>> >>>>>> >>>>>> I'm using Phoenix 4.XX on CDH5 Hbase 0.98 cluster. >>>>>> >>>>>> >>>>>> >>>>>> On our sandbox environment which is a non-secure cluster , I'm able >>>>>> to connect through squirrel without any problem. >>>>>> >>>>>> >>>>>> >>>>>> But on our production environment which is a secure cluster it’s not >>>>>> working. I'm providing keytab and principal as part of my JDBC url but no >>>>>> luck. >>>>>> >>>>>> >>>>>> >>>>>> Can someone help with some guideance on this. >>>>>> >>>>>> >>>>>> >>>>>> Please find the stack trace below >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> java.util.concurrent.TimeoutException >>>>>> >>>>>> at >>>>>> java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:258) >>>>>> >>>>>> at >>>>>> java.util.concurrent.FutureTask.get(FutureTask.java:119) >>>>>> >>>>>> at >>>>>> net.sourceforge.squirrel_sql.client.mainframe.action.OpenConnectionCommand.awaitConnection(OpenConnectionCommand.java:132) >>>>>> >>>>>> at >>>>>> net.sourceforge.squirrel_sql.client.mainframe.action.OpenConnectionCommand.access$100(OpenConnectionCommand.java:45) >>>>>> >>>>>> at >>>>>> net.sourceforge.squirrel_sql.client.mainframe.action.OpenConnectionCommand$2.run(OpenConnectionCommand.java:115) >>>>>> >>>>>> at >>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) >>>>>> >>>>>> at >>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) >>>>>> >>>>>> at >>>>>> java.util.concurrent.FutureTask.run(FutureTask.java:166) >>>>>> >>>>>> at >>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) >>>>>> >>>>>> at >>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) >>>>>> >>>>>> at java.lang.Thread.run(Thread.java:722) >>>>>> >>>>>> >>>>>> >>>>>> Regards, >>>>>> >>>>>> Imran A Mohammed. >>>>>> >>>>>> >>>>>> >>>>> >>>> >>>> -- >>>> Thanks & Regards, >>>> Anil Gupta >>>> >>> >> >> -- >> Thanks & Regards, >> Anil Gupta >> > -- Thanks & Regards, Anil Gupta
