That's very odd, especially since the upgrade doesn't touch the topology tables. Are you using MySQL by any chance? If so, can you check to make sure that your database engine is Innodb and not MyISAM. You have an integrity violation here which doesn't seem possible unless you're using a database which doesn't support foreign key constraints.
There's probably some SQL which you can run to insert an entry into the topology_logical_request table, but it's probably best to understand why this happened first. On Mar 8, 2016, at 5:55 AM, cs user <acldstk...@gmail.com<mailto:acldstk...@gmail.com>> wrote: Hi All, I've upgraded Ambari from version 2.1.2-377 to version 2.2.1.0-161. After performing the upgrade on the server, agents, upgrading the database and starting everything up, I keep seeing the following error in the logs on the server: 08 Mar 2016 10:07:05,087 INFO [qtp-ambari-agent-55] HostRequest:125 - HostRequest: Successfully recovered host request for host: Host Assignment Pending 08 Mar 2016 10:07:05,088 INFO [qtp-ambari-agent-55] LogicalRequest:420 - LogicalRequest.createHostRequests: created new outstanding host request ID = 3 08 Mar 2016 10:07:05,120 INFO [qtp-ambari-agent-55] HostRequest:125 - HostRequest: Successfully recovered host request for host: Host Assignment Pending 08 Mar 2016 10:07:05,120 INFO [qtp-ambari-agent-55] LogicalRequest:420 - LogicalRequest.createHostRequests: created new outstanding host request ID = 5 08 Mar 2016 10:07:05,134 INFO [qtp-ambari-agent-55] HostRequest:125 - HostRequest: Successfully recovered host request for host: Host Assignment Pending 08 Mar 2016 10:07:05,134 INFO [qtp-ambari-agent-55] LogicalRequest:420 - LogicalRequest.createHostRequests: created new outstanding host request ID = 8 08 Mar 2016 10:07:05,147 INFO [qtp-ambari-agent-55] HostRequest:125 - HostRequest: Successfully recovered host request for host: Host Assignment Pending 08 Mar 2016 10:07:05,148 INFO [qtp-ambari-agent-55] LogicalRequest:420 - LogicalRequest.createHostRequests: created new outstanding host request ID = 7 08 Mar 2016 10:07:05,158 INFO [qtp-ambari-agent-55] HostRequest:125 - HostRequest: Successfully recovered host request for host: Host Assignment Pending 08 Mar 2016 10:07:05,158 INFO [qtp-ambari-agent-55] LogicalRequest:420 - LogicalRequest.createHostRequests: created new outstanding host request ID = 6 08 Mar 2016 10:07:05,170 INFO [qtp-ambari-agent-55] HostRequest:125 - HostRequest: Successfully recovered host request for host: Host Assignment Pending 08 Mar 2016 10:07:05,170 INFO [qtp-ambari-agent-55] LogicalRequest:420 - LogicalRequest.createHostRequests: created new outstanding host request ID = 2 08 Mar 2016 10:07:05,184 INFO [qtp-ambari-agent-55] HostRequest:125 - HostRequest: Successfully recovered host request for host: Host Assignment Pending 08 Mar 2016 10:07:05,185 INFO [qtp-ambari-agent-55] LogicalRequest:420 - LogicalRequest.createHostRequests: created new outstanding host request ID = 1 08 Mar 2016 10:07:05,194 INFO [qtp-ambari-agent-55] HostRequest:125 - HostRequest: Successfully recovered host request for host: Host Assignment Pending 08 Mar 2016 10:07:05,194 INFO [qtp-ambari-agent-55] LogicalRequest:420 - LogicalRequest.createHostRequests: created new outstanding host request ID = 4 08 Mar 2016 10:07:05,290 INFO [qtp-ambari-agent-55] HostRequest:125 - HostRequest: Successfully recovered host request for host: ambdevtestdc2host-group-21.node.example 08 Mar 2016 10:07:05,328 INFO [qtp-ambari-agent-55] HostRequest:125 - HostRequest: Successfully recovered host request for host: ambdevtestdc2host-group-51.node.example 08 Mar 2016 10:07:05,384 INFO [qtp-ambari-agent-55] HostRequest:125 - HostRequest: Successfully recovered host request for host: ambdevtestdc2host-group-11.node.example 08 Mar 2016 10:07:05,428 INFO [qtp-ambari-agent-55] HostRequest:125 - HostRequest: Successfully recovered host request for host: ambdevtestdc2host-group-41.node.example 08 Mar 2016 10:07:05,507 INFO [qtp-ambari-agent-55] HostRequest:125 - HostRequest: Successfully recovered host request for host: ambdevtestdc2host-group-31.node.example 08 Mar 2016 10:07:05,575 INFO [qtp-ambari-agent-55] HostRequest:125 - HostRequest: Successfully recovered host request for host: ambdevtestdc2host-group-53.node.example 08 Mar 2016 10:07:05,627 INFO [qtp-ambari-agent-55] HostRequest:125 - HostRequest: Successfully recovered host request for host: ambdevtestdc2host-group-52.node.example 08 Mar 2016 10:07:05,644 WARN [qtp-ambari-agent-55] ServletHandler:563 - /agent/v1/register/ambdevtestdc2host-group-51.node.example java.lang.NullPointerException at org.apache.ambari.server.topology.PersistedStateImpl.getAllRequests(PersistedStateImpl.java:157) at org.apache.ambari.server.topology.TopologyManager.ensureInitialized(TopologyManager.java:131) at org.apache.ambari.server.topology.TopologyManager.onHostRegistered(TopologyManager.java:315) at org.apache.ambari.server.state.host.HostImpl$HostRegistrationReceived.transition(HostImpl.java:301) at org.apache.ambari.server.state.host.HostImpl$HostRegistrationReceived.transition(HostImpl.java:266) at org.apache.ambari.server.state.fsm.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:354) at org.apache.ambari.server.state.fsm.StateMachineFactory.doTransition(StateMachineFactory.java:294) at org.apache.ambari.server.state.fsm.StateMachineFactory.access$300(StateMachineFactory.java:39) at org.apache.ambari.server.state.fsm.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:440) at org.apache.ambari.server.state.host.HostImpl.handleEvent(HostImpl.java:570) at org.apache.ambari.server.agent.HeartBeatHandler.handleRegistration(HeartBeatHandler.java:966) at org.apache.ambari.server.agent.rest.AgentResource.register(AgentResource.java:95) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185) at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:540) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:715) at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1496) at org.apache.ambari.server.security.SecurityFilter.doFilter(SecurityFilter.java:67) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1467) at org.apache.ambari.server.api.AmbariPersistFilter.doFilter(AmbariPersistFilter.java:47) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1467) at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:82) at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:294) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1467) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:501) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:370) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494) at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240) at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) at org.eclipse.jetty.io.nio.SslConnection.handle(SslConnection.java:196) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696) at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Thread.java:745) This is not specific to host group ambdevtestdc2host-group-51.node.example, it is happening for all host groups. On the agents I see the following: <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 500 Server Error</title> </head> <body> <h2>HTTP ERROR: 500</h2> <p>Problem accessing /agent/v1/register/ambdevtestdc2host-group-51.node.example Reason: <pre> Server Error</pre></p> <hr /><i><small>Powered by Jetty://</small></i> Is there a work around for this? It's just a test cluster, but it would be good to know how to work around this, as I've seen it a number of times now. Is there anything that can be modified in the database to resolve it? Thanks!
