On Tue, Apr 2, 2013 at 1:12 PM, Chip Childers <chip.child...@sungard.com>wrote:
> On Tue, Apr 02, 2013 at 10:10:28PM +0200, Wido den Hollander wrote: > > Hi, > > > > On 04/02/2013 08:50 PM, Sheng Yang wrote: > > >On Tue, Apr 2, 2013 at 11:49 AM, Sheng Yang <sh...@yasker.org> wrote: > > >>On Tue, Apr 2, 2013 at 11:32 AM, Wido den Hollander <w...@widodh.nl> > wrote: > > >>>Hi, > > >>> > > >>>Since I've upgraded my management server to 4.1 my agents refuse to > connect, > > >>>their log says: > > >>> > > >>>2013-04-02 20:26:11,207 INFO [utils.nio.NioClient] > (Agent-Selector:null) > > >>>Connecting to 31.25.X.X:8250 > > >>>2013-04-02 20:26:11,209 ERROR [utils.nio.NioConnection] > > >>>(Agent-Selector:null) Unable to initialize the threads. > > >>>java.io.IOException: SSL: Fail to init SSL! java.io.IOException: > Connection > > >>>closed with -1 on reading size. > > >>> at com.cloud.utils.nio.NioClient.init(NioClient.java:83) > > >>> at > com.cloud.utils.nio.NioConnection.run(NioConnection.java:108) > > >>> at java.lang.Thread.run(Thread.java:679) > > >>> > > >>>So I tried a telnet connection: > > >>> > > >>>Connection closed by foreign host. > > >>>root@stack01:~# telnet 31.25.X.X 8250 > > >>>Trying 31.25.X.X... > > >>>Connected to 31.X.X.X. > > >>>Escape character is '^]'. > > >>>Connection closed by foreign host. > > >>>root@stack01:~# > > >>> > > >>>So I didn't close the connection, but it was the management server. > > >>> > > >>>I cranked up the logging to DEBUG, but nothing shows in any of the > logs, so > > >>>I have no clue why this isn't working. > > >>> > > >>>On the mgmt server I see Java in LISTEN state on port 8250 > > >>> > > >>>There is no firewall on the management server (it's my lab!). > > >>> > > >>>Any clues to what this could be before I start filing a in Jira? > Since I'm > > >>>not sure if this is a bug. > > >> > > >>Have you upgrade your agent? > > > > > > > Yes, the Agent is also 4.1 > > > > >Also you could try to enable TRACE for com.cloud.utils.nio to see more > > >log of NIO. > > > > > > > I think you mean at the agent? > > > > But what I think is weird is that the mgmt server directly closes > > the telnet connection as well and nothing shows up in the logs. > > > > In the mgmt server log I found this: > > > > 2013-04-02 22:08:41,156 INFO [utils.nio.NioServer] > > (AgentManager-Selector:null) NioConnection started and listening on > > 0.0.0.0/0.0.0.0:8250 > > > > What I did notice is that when I shutdown the mgmt server the Agent > > connects for a second: > > > > 2013-04-02 22:07:53,689 INFO [utils.nio.NioClient] > > (Agent-Selector:null) Connecting to 31.25.X.X:8250 > > 2013-04-02 22:07:53,934 INFO [utils.nio.NioClient] > > (Agent-Selector:null) SSL: Handshake done > > > > And then the mgmt server exists and the connection gets lost. > > > > ANY clues about this weird behavior? > > > > Wido > > > > >--Sheng > > >> > > >>--Sheng > > >>> > > >>>Wido > > > > Wido - Can you please open a blocker bug to track this? It may end up > being a non-issue or something... but I want to know that we have it > outstanding when considering when to cut the RC. > It remind me of one weird bug. The corrupted ssl keystore(due to some encryption/decryption of db which I don't understand well, I remember Kelven knows more about it) on mgmt server make it impossible to connect to mgmt server. Could you clear "ssl.keystore" entry in configuration table of db, as well as cloud.keystore file in the configuration path and restart the mgmt server and try again? --Sheng
