Thanks for the quick response.
I don’t think it’s a DNS issue. I am using /etc/hosts. I’ll double check the
entries. I did, however, use the hostnames when setting up the passwordless
ssh.
The stack trace I put below.
16:37:33.842 [main] DEBUG
org.apache.accumulo.core.rpc.SaslConnectionParams(117) - Kerberos krb5
configuration not found, setting default realm to empty
16:37:33.843 [main] TRACE org.apache.accumulo.core.rpc.ThriftUtil(365) -
Opening normal transport
16:37:38.878 [main] DEBUG org.apache.accumulo.core.clientImpl.MasterClient(78)
- Failed to connect to master=haz11:9999, will retry...
org.apache.thrift.transport.TTransportException: java.net.ConnectException:
Connection refused
at org.apache.thrift.transport.TSocket.open(TSocket.java:226)
~[libthrift-0.12.0.jar:0.12.0]
at
org.apache.accumulo.core.rpc.ThriftUtil.createClientTransport(ThriftUtil.java:368)
~[accumulo-core-2.0.0.jar:2.0.0]
at
org.apache.accumulo.core.clientImpl.ThriftTransportPool.createNewTransport(ThriftTransportPool.java:496)
~[accumulo-core-2.0.0.jar:2.0.0]
at
org.apache.accumulo.core.clientImpl.ThriftTransportPool.getTransport(ThriftTransportPool.java:432)
~[accumulo-core-2.0.0.jar:2.0.0]
at
org.apache.accumulo.core.clientImpl.ThriftTransportPool.getTransport(ThriftTransportPool.java:410)
~[accumulo-core-2.0.0.jar:2.0.0]
at
org.apache.accumulo.core.rpc.ThriftUtil.getClient(ThriftUtil.java:150)
~[accumulo-core-2.0.0.jar:2.0.0]
at
org.apache.accumulo.core.rpc.ThriftUtil.getClientNoTimeout(ThriftUtil.java:113)
~[accumulo-core-2.0.0.jar:2.0.0]
at
org.apache.accumulo.core.clientImpl.MasterClient.getConnection(MasterClient.java:70)
[accumulo-core-2.0.0.jar:2.0.0]
at
org.apache.accumulo.core.clientImpl.MasterClient.getConnectionWithRetry(MasterClient.java:47)
[accumulo-core-2.0.0.jar:2.0.0]
at
org.apache.accumulo.core.clientImpl.TableOperationsImpl.beginFateOperation(TableOperationsImpl.java:246)
[accumulo-core-2.0.0.jar:2.0.0]
at
org.apache.accumulo.core.clientImpl.TableOperationsImpl.doFateOperation(TableOperationsImpl.java:352)
[accumulo-core-2.0.0.jar:2.0.0]
at
org.apache.accumulo.core.clientImpl.TableOperationsImpl.doFateOperation(TableOperationsImpl.java:342)
[accumulo-core-2.0.0.jar:2.0.0]
at
org.apache.accumulo.core.clientImpl.TableOperationsImpl.doTableFateOperation(TableOperationsImpl.java:1599)
[accumulo-core-2.0.0.jar:2.0.0]
at
org.apache.accumulo.core.clientImpl.TableOperationsImpl.create(TableOperationsImpl.java:234)
[accumulo-core-2.0.0.jar:2.0.0]
at
org.apache.accumulo.core.clientImpl.TableOperationsImpl.create(TableOperationsImpl.java:206)
[accumulo-core-2.0.0.jar:2.0.0]
at
gov.cdc.dhcp.hehr.dcp.cda2d4m.AccumuloComponent.initAccumulo(AccumuloComponent.java:84)
[classes/:?]
at gov.cdc.dhcp.hehr.dcp.cda2d4m.CDA2D4M.run(CDA2D4M.java:79)
[classes/:?]
at gov.cdc.dhcp.hehr.dcp.cda2d4m.CDA2D4M.main(CDA2D4M.java:133)
[classes/:?]
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect0(Native Method) ~[?:?]
at sun.nio.ch.Net.connect(Net.java:493) ~[?:?]
at sun.nio.ch.Net.connect(Net.java:482) ~[?:?]
at sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:588) ~[?:?]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:339) ~[?:?]
at java.net.Socket.connect(Socket.java:585) ~[?:?]
at org.apache.thrift.transport.TSocket.open(TSocket.java:221)
~[libthrift-0.12.0.jar:0.12.0]
... 17 more
16:37:39.134 [main] TRACE
org.apache.accumulo.core.clientImpl.ClientContext(366) - tid=1 Looking up
master location in zookeeper.
16:37:39.134 [main] TRACE
org.apache.accumulo.core.clientImpl.ClientContext(374) - tid=1 Found master at
haz11:9999 in 0.000 secs
16:37:39.134 [main] TRACE org.apache.accumulo.core.rpc.ThriftUtil(365) -
Opening normal transport
16:37:39.160 [main] DEBUG org.apache.accumulo.core.clientImpl.MasterClient(78)
- Failed to connect to master=haz11:9999, will retry...
org.apache.thrift.transport.TTransportException: java.net.ConnectException:
Connection refused
at org.apache.thrift.transport.TSocket.open(TSocket.java:226)
~[libthrift-0.12.0.jar:0.12.0]
at
org.apache.accumulo.core.rpc.ThriftUtil.createClientTransport(ThriftUtil.java:368)
~[accumulo-core-2.0.0.jar:2.0.0]
at
org.apache.accumulo.core.clientImpl.ThriftTransportPool.createNewTransport(ThriftTransportPool.java:496)
~[accumulo-core-2.0.0.jar:2.0.0]
at
org.apache.accumulo.core.clientImpl.ThriftTransportPool.getTransport(ThriftTransportPool.java:432)
~[accumulo-core-2.0.0.jar:2.0.0]
at
org.apache.accumulo.core.clientImpl.ThriftTransportPool.getTransport(ThriftTransportPool.java:410)
~[accumulo-core-2.0.0.jar:2.0.0]
at
org.apache.accumulo.core.rpc.ThriftUtil.getClient(ThriftUtil.java:150)
~[accumulo-core-2.0.0.jar:2.0.0]
at
org.apache.accumulo.core.rpc.ThriftUtil.getClientNoTimeout(ThriftUtil.java:113)
~[accumulo-core-2.0.0.jar:2.0.0]
at
org.apache.accumulo.core.clientImpl.MasterClient.getConnection(MasterClient.java:70)
[accumulo-core-2.0.0.jar:2.0.0]
at
org.apache.accumulo.core.clientImpl.MasterClient.getConnectionWithRetry(MasterClient.java:47)
[accumulo-core-2.0.0.jar:2.0.0]
at
org.apache.accumulo.core.clientImpl.TableOperationsImpl.beginFateOperation(TableOperationsImpl.java:246)
[accumulo-core-2.0.0.jar:2.0.0]
at
org.apache.accumulo.core.clientImpl.TableOperationsImpl.doFateOperation(TableOperationsImpl.java:352)
[accumulo-core-2.0.0.jar:2.0.0]
at
org.apache.accumulo.core.clientImpl.TableOperationsImpl.doFateOperation(TableOperationsImpl.java:342)
[accumulo-core-2.0.0.jar:2.0.0]
at
org.apache.accumulo.core.clientImpl.TableOperationsImpl.doTableFateOperation(TableOperationsImpl.java:1599)
[accumulo-core-2.0.0.jar:2.0.0]
at
org.apache.accumulo.core.clientImpl.TableOperationsImpl.create(TableOperationsImpl.java:234)
[accumulo-core-2.0.0.jar:2.0.0]
at
org.apache.accumulo.core.clientImpl.TableOperationsImpl.create(TableOperationsImpl.java:206)
[accumulo-core-2.0.0.jar:2.0.0]
at
gov.cdc.dhcp.hehr.dcp.cda2d4m.AccumuloComponent.initAccumulo(AccumuloComponent.java:84)
[classes/:?]
at gov.cdc.dhcp.hehr.dcp.cda2d4m.CDA2D4M.run(CDA2D4M.java:79)
[classes/:?]
at gov.cdc.dhcp.hehr.dcp.cda2d4m.CDA2D4M.main(CDA2D4M.java:133)
[classes/:?]
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.Net.connect0(Native Method) ~[?:?]
at sun.nio.ch.Net.connect(Net.java:493) ~[?:?]
at sun.nio.ch.Net.connect(Net.java:482) ~[?:?]
at sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:588) ~[?:?]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:339) ~[?:?]
at java.net.Socket.connect(Socket.java:585) ~[?:?]
at org.apache.thrift.transport.TSocket.open(TSocket.java:221)
~[libthrift-0.12.0.jar:0.12.0]
... 17 more
Geoffry Roberts
Lead Technologist
702.290.9098
[email protected]
Booz | Allen | Hamilton
BoozAllen.com
From: Christopher <[email protected]>
Reply-To: "[email protected]" <[email protected]>
Date: Tuesday, December 29, 2020 at 8:17 PM
To: accumulo-user <[email protected]>
Subject: [External] Re: Accumulo access question
Is there more to the debug message? A full stack trace perhaps? It might show
the specific IOException causing the problem connecting.
The two most likely causes are DNS/name service causing the server host name to
not be resolved (very common), and the port (9999 in this case) being
unreachable from the client (in EC2, this can happen with an inadequate
security group configuration, for example).
On Tue, Dec 29, 2020, 17:04 Roberts, Geoffry [USA]
<[email protected]<mailto:[email protected]>> wrote:
All,
I have am using Accumulo in a different setup than ever before and I am not
getting a connect.
I am having to ssh tunnel in. I have 2181 open (forwarded) and I can connect
to Zookeeper using the Zk client from my laptop. When I try to connect to
Accumulo using an Accumulo client from my laptop I get:
DEBUG org.apache.accumulo.core.clientImpl.MasterClient(78) - Failed to connect
to master=haz11:9999, will retry...
(haz11 is one of my nodes.) I do have passwordless ssh working between all
nodes. Netstat shows that
0.0.0.0:9999<https://urldefense.com/v3/__http:/0.0.0.0:9999__;!!May37g!ZUmDBXH4QTS4EGb6irABAqtH8rZ5iY12zGUjHqIwMJOT3j5aMh0mcq_hZ_tAr_EPABqp$>
… LISTEN
I have a small (three node installation). I have all three hosts listed in all
files (gc, masters, tservers, workers, monitors, et. al.). It all starts up
well enough. I see the proper processes running on all nodes. I can start a
shell—locally—from one of the nodes and run shell commands, all very normal.
Geoffry Roberts
Lead Technologist
702.290.9098
[email protected]<mailto:[email protected]>
Booz | Allen | Hamilton
BoozAllen.com
