zhaoliangzl opened a new issue, #2141: URL: https://github.com/apache/incubator-hugegraph/issues/2141
### Bug Type (问题类型) server status (启动/运行异常) ### Before submit - [X] 我已经确认现有的 [Issues](https://github.com/apache/hugegraph/issues) 与 [FAQ](https://hugegraph.apache.org/docs/guides/faq/) 中没有相同 / 重复问题 (I have confirmed and searched that there are no similar problems in the historical issue and documents) ### Environment (环境信息) - Server Version: 1.0.0 (Apache Release Version) - Backend: RocksDB x nodes, HDD or SSD - OS: xx CPUs, xx G RAM, Ubuntu 2x.x / CentOS 7.x - Data Size: xx vertices, xx edges <!-- (like 1000W 点, 9000W 边) --> <img width="1423" alt="截屏2023-03-03 下午6 10 53" src="https://user-images.githubusercontent.com/25294531/222693233-34174661-2d46-414b-b9d4-916fc863853e.png"> ### Expected & Actual behavior (期望与实际表现) 2023-03-03 17:49:43 [JRaft-ElectionTimer-<lemma/10.216.59.27:2028>0] [WARN] c.a.s.j.c.NodeImpl - Node <lemma/10.216.59.27:2028> channel init failed, address=10.216.59.23:2015. 2023-03-03 17:49:46 [Rpc-netty-server-worker-1-thread-2] [WARN] c.a.s.j.r.i.BoltRaftRpcFactory - JRaft SET bolt.rpc.dispatch-msg-list-in-default-executor to be false for replicator pipeline optimistic. 2023-03-03 17:49:46 [lemma/PeerPair[10.216.59.27:2028 -> 10.216.59.23:2011]-AppendEntriesThread0] [WARN] c.a.s.j.c.NodeImpl - Node <lemma/10.216.59.27:2028> reject term_unmatched AppendEntriesRequest from 10.216.59.23:2011, term=8, prevLogIndex=651448, prevLogTerm=7, localPrevLogTerm=0, lastLogIndex=649216, entriesSize=0. 2023-03-03 17:49:46 [JRaft-FSMCaller-Disruptor-0] [INFO] o.a.h.b.s.r.StoreStateMachine - The node 10.216.59.27:2028 become to follower 2023-03-03 17:49:50 [main] [INFO] o.a.h.b.s.r.RaftNode - Waited for raft group 'lemma' leader elected successfully 2023-03-03 17:49:50 [main] [INFO] o.a.h.b.s.r.RaftNode - Waiting for raft group 'lemma' log synced 2023-03-03 17:49:55 [JRaft-FSMCaller-Disruptor-0] [INFO] o.a.h.b.s.r.StoreStateMachine - The node 10.216.59.27:2028 abdicated from follower 2023-03-03 17:49:55 [JRaft-FSMCaller-Disruptor-0] [INFO] o.a.h.b.s.r.StoreStateMachine - The node 10.216.59.27:2028 become to follower 2023-03-03 17:49:55 [main] [INFO] o.a.h.b.s.r.RaftNode - Waited for raft group 'lemma' log synced successfully 2023-03-03 17:49:55 [main] [INFO] o.a.h.b.s.r.RaftBackendStoreProvider - The raft store is started 2023-03-03 17:49:55 [main] [INFO] o.a.h.StandardHugeGraph - Close graph standardhugegraph[lemma] 2023-03-03 17:49:55 [main] [INFO] o.a.h.b.s.r.RaftContext - Stop raft server: 10.216.59.27:2028 2023-03-03 17:49:55 [main] [INFO] o.a.h.b.s.r.RaftNode - Shutdown raft node: [lemma-10.216.59.27:2028] 2023-03-03 17:49:55 [JRaft-FSMCaller-Disruptor-0] [INFO] o.a.h.b.s.r.StoreStateMachine - The node 10.216.59.27:2028 abdicated from follower 2023-03-03 17:49:55 [main] [INFO] o.a.h.StandardHugeGraph - Close graph standardhugegraph[genealogy] 2023-03-03 17:49:55 [main] [INFO] o.a.h.b.s.r.RaftContext - Stop raft server: null 2023-03-03 17:49:55 [main] [WARN] o.a.h.c.GraphManager - Failed to close graph 'standardhugegraph[genealogy]' java.lang.NullPointerException: null at org.apache.hugegraph.backend.store.raft.RaftContext.shutdownRpcServer(RaftContext.java:406) ~[hugegraph-core-1.0.0.jar:1.0.0] at org.apache.hugegraph.backend.store.raft.RaftContext.close(RaftContext.java:175) ~[hugegraph-core-1.0.0.jar:1.0.0] at org.apache.hugegraph.backend.store.raft.RaftBackendStoreProvider.close(RaftBackendStoreProvider.java:163) ~[hugegraph-core-1.0.0.jar:1.0.0] at org.apache.hugegraph.StandardHugeGraph.close(StandardHugeGraph.java:940) ~[hugegraph-core-1.0.0.jar:1.0.0] at org.apache.hugegraph.core.GraphManager.close(GraphManager.java:261) ~[hugegraph-api-1.0.0.jar:0.69.0.0] at org.apache.hugegraph.server.ApplicationConfig$GraphManagerFactory$1.onEvent(ApplicationConfig.java:132) ~[hugegraph-api-1.0.0.jar:0.69.0.0] at org.glassfish.jersey.server.internal.monitoring.CompositeApplicationEventListener.onEvent(CompositeApplicationEventListener.java:49) ~[jersey-server-3.0.3.jar:?] at org.glassfish.jersey.server.internal.monitoring.MonitoringContainerListener.onStartup(MonitoringContainerListener.java:56) ~[jersey-server-3.0.3.jar:?] at org.glassfish.jersey.server.ApplicationHandler.onStartup(ApplicationHandler.java:711) ~[jersey-server-3.0.3.jar:?] at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.start(GrizzlyHttpContainer.java:330) ~[jersey-container-grizzly2-http-3.0.3.jar:?] at org.glassfish.grizzly.http.server.HttpHandlerChain.start(HttpHandlerChain.java:376) ~[grizzly-http-server-3.0.1.jar:3.0.1] at org.glassfish.grizzly.http.server.HttpServer.setupHttpHandler(HttpServer.java:268) ~[grizzly-http-server-3.0.1.jar:3.0.1] at org.glassfish.grizzly.http.server.HttpServer.start(HttpServer.java:245) ~[grizzly-http-server-3.0.1.jar:3.0.1] at org.apache.hugegraph.server.RestServer.start(RestServer.java:71) ~[hugegraph-api-1.0.0.jar:0.69.0.0] at org.apache.hugegraph.server.RestServer.start(RestServer.java:178) ~[hugegraph-api-1.0.0.jar:0.69.0.0] at org.apache.hugegraph.dist.HugeRestServer.start(HugeRestServer.java:32) ~[hugegraph-dist-1.0.0.jar:1.0.0] at org.apache.hugegraph.dist.HugeGraphServer.<init>(HugeGraphServer.java:60) ~[hugegraph-dist-1.0.0.jar:1.0.0] at org.apache.hugegraph.dist.HugeGraphServer.main(HugeGraphServer.java:120) ~[hugegraph-dist-1.0.0.jar:1.0.0] 2023-03-03 17:49:55 [main] [INFO] o.a.h.r.RpcServer - RpcServer stop on port 2028 2023-03-03 17:49:55 [main] [ERROR] c.a.s.r.s.ServerFactory - RPC-020100018: Error when destroy server with key [2,028]. com.alipay.remoting.LifeCycleException: this component has closed at com.alipay.remoting.AbstractLifeCycle.shutdown(AbstractLifeCycle.java:41) ~[bolt-1.6.4.jar:?] at com.alipay.remoting.AbstractRemotingServer.shutdown(AbstractRemotingServer.java:110) ~[bolt-1.6.4.jar:?] at com.alipay.remoting.AbstractRemotingServer.stop(AbstractRemotingServer.java:84) ~[bolt-1.6.4.jar:?] at com.alipay.sofa.rpc.server.bolt.BoltServer.stop(BoltServer.java:166) ~[sofa-rpc-all-5.7.6.jar:5.7.6] at com.alipay.sofa.rpc.server.bolt.BoltServer.destroy(BoltServer.java:228) ~[sofa-rpc-all-5.7.6.jar:5.7.6] at com.alipay.sofa.rpc.server.ServerFactory.destroyServer(ServerFactory.java:159) [sofa-rpc-all-5.7.6.jar:5.7.6] at com.alipay.sofa.rpc.config.ServerConfig.destroy(ServerConfig.java:245) [sofa-rpc-all-5.7.6.jar:5.7.6] at org.apache.hugegraph.rpc.RpcServer.destroy(RpcServer.java:138) [hugegraph-rpc-1.0.0.jar:1.0.0] at org.apache.hugegraph.core.GraphManager.destroyRpcServer(GraphManager.java:320) [hugegraph-api-1.0.0.jar:0.69.0.0] at org.apache.hugegraph.core.GraphManager.close(GraphManager.java:266) [hugegraph-api-1.0.0.jar:0.69.0.0] at org.apache.hugegraph.server.ApplicationConfig$GraphManagerFactory$1.onEvent(ApplicationConfig.java:132) [hugegraph-api-1.0.0.jar:0.69.0.0] at org.glassfish.jersey.server.internal.monitoring.CompositeApplicationEventListener.onEvent(CompositeApplicationEventListener.java:49) [jersey-server-3.0.3.jar:?] at org.glassfish.jersey.server.internal.monitoring.MonitoringContainerListener.onStartup(MonitoringContainerListener.java:56) [jersey-server-3.0.3.jar:?] at org.glassfish.jersey.server.ApplicationHandler.onStartup(ApplicationHandler.java:711) [jersey-server-3.0.3.jar:?] at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.start(GrizzlyHttpContainer.java:330) [jersey-container-grizzly2-http-3.0.3.jar:?] at org.glassfish.grizzly.http.server.HttpHandlerChain.start(HttpHandlerChain.java:376) [grizzly-http-server-3.0.1.jar:3.0.1] at org.glassfish.grizzly.http.server.HttpServer.setupHttpHandler(HttpServer.java:268) [grizzly-http-server-3.0.1.jar:3.0.1] at org.glassfish.grizzly.http.server.HttpServer.start(HttpServer.java:245) [grizzly-http-server-3.0.1.jar:3.0.1] at org.apache.hugegraph.server.RestServer.start(RestServer.java:71) [hugegraph-api-1.0.0.jar:0.69.0.0] at org.apache.hugegraph.server.RestServer.start(RestServer.java:178) [hugegraph-api-1.0.0.jar:0.69.0.0] at org.apache.hugegraph.dist.HugeRestServer.start(HugeRestServer.java:32) [hugegraph-dist-1.0.0.jar:1.0.0] at org.apache.hugegraph.dist.HugeGraphServer.<init>(HugeGraphServer.java:60) [hugegraph-dist-1.0.0.jar:1.0.0] at org.apache.hugegraph.dist.HugeGraphServer.main(HugeGraphServer.java:120) [hugegraph-dist-1.0.0.jar:1.0.0] 2023-03-03 17:49:55 [main] [ERROR] o.a.h.d.HugeGraphServer - HugeRestServer start error: java.lang.RuntimeException: Processor with interest key [com.alipay.sofa.jraft.rpc.RpcRequests$AppendEntriesRequest] has already been registered to rpc server, can not register again! at com.alipay.remoting.rpc.protocol.UserProcessorRegisterHelper.registerUserProcessor(UserProcessorRegisterHelper.java:51) ~[bolt-1.6.4.jar:?] at com.alipay.remoting.rpc.RpcServer.registerUserProcessor(RpcServer.java:503) ~[bolt-1.6.4.jar:?] at com.alipay.sofa.jraft.rpc.impl.BoltRpcServer.registerProcessor(BoltRpcServer.java:97) ~[jraft-core-1.3.11.jar:?] at com.alipay.sofa.jraft.rpc.RaftRpcServerFactory.addRaftRequestProcessors(RaftRpcServerFactory.java:101) ~[jraft-core-1.3.11.jar:?] at com.alipay.sofa.jraft.rpc.RaftRpcServerFactory.addRaftRequestProcessors(RaftRpcServerFactory.java:85) ~[jraft-core-1.3.11.jar:?] at org.apache.hugegraph.backend.store.raft.RaftContext.wrapRpcServer(RaftContext.java:400) ~[hugegraph-core-1.0.0.jar:1.0.0] at org.apache.hugegraph.backend.store.raft.RaftContext.initRaftNode(RaftContext.java:150) ~[hugegraph-core-1.0.0.jar:1.0.0] at org.apache.hugegraph.backend.store.raft.RaftBackendStoreProvider.waitReady(RaftBackendStoreProvider.java:152) ~[hugegraph-core-1.0.0.jar:1.0.0] at org.apache.hugegraph.StandardHugeGraph.waitReady(StandardHugeGraph.java:318) ~[hugegraph-core-1.0.0.jar:1.0.0] at org.apache.hugegraph.core.GraphManager.waitGraphsReady(GraphManager.java:387) ~[hugegraph-api-1.0.0.jar:0.69.0.0] at org.apache.hugegraph.core.GraphManager.init(GraphManager.java:111) ~[hugegraph-api-1.0.0.jar:0.69.0.0] at org.apache.hugegraph.server.ApplicationConfig$GraphManagerFactory$1.onEvent(ApplicationConfig.java:130) ~[hugegraph-api-1.0.0.jar:0.69.0.0] at org.glassfish.jersey.server.internal.monitoring.CompositeApplicationEventListener.onEvent(CompositeApplicationEventListener.java:49) ~[jersey-server-3.0.3.jar:?] at org.glassfish.jersey.server.internal.monitoring.MonitoringContainerListener.onStartup(MonitoringContainerListener.java:56) ~[jersey-server-3.0.3.jar:?] at org.glassfish.jersey.server.ApplicationHandler.onStartup(ApplicationHandler.java:711) ~[jersey-server-3.0.3.jar:?] at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.start(GrizzlyHttpContainer.java:330) ~[jersey-container-grizzly2-http-3.0.3.jar:?] at org.glassfish.grizzly.http.server.HttpHandlerChain.start(HttpHandlerChain.java:376) ~[grizzly-http-server-3.0.1.jar:3.0.1] at org.glassfish.grizzly.http.server.HttpServer.setupHttpHandler(HttpServer.java:268) ~[grizzly-http-server-3.0.1.jar:3.0.1] at org.glassfish.grizzly.http.server.HttpServer.start(HttpServer.java:245) ~[grizzly-http-server-3.0.1.jar:3.0.1] at org.apache.hugegraph.server.RestServer.start(RestServer.java:71) ~[hugegraph-api-1.0.0.jar:0.69.0.0] at org.apache.hugegraph.server.RestServer.start(RestServer.java:178) ~[hugegraph-api-1.0.0.jar:0.69.0.0] at org.apache.hugegraph.dist.HugeRestServer.start(HugeRestServer.java:32) ~[hugegraph-dist-1.0.0.jar:1.0.0] at org.apache.hugegraph.dist.HugeGraphServer.<init>(HugeGraphServer.java:60) ~[hugegraph-dist-1.0.0.jar:1.0.0] at org.apache.hugegraph.dist.HugeGraphServer.main(HugeGraphServer.java:120) ~[hugegraph-dist-1.0.0.jar:1.0.0] 2023-03-03 17:49:55 [main] [INFO] o.a.h.HugeFactory - HugeFactory shutdown Exception in thread "main" java.lang.RuntimeException: Processor with interest key [com.alipay.sofa.jraft.rpc.RpcRequests$AppendEntriesRequest] has already been registered to rpc server, can not register again! at com.alipay.remoting.rpc.protocol.UserProcessorRegisterHelper.registerUserProcessor(UserProcessorRegisterHelper.java:51) at com.alipay.remoting.rpc.RpcServer.registerUserProcessor(RpcServer.java:503) at com.alipay.sofa.jraft.rpc.impl.BoltRpcServer.registerProcessor(BoltRpcServer.java:97) at com.alipay.sofa.jraft.rpc.RaftRpcServerFactory.addRaftRequestProcessors(RaftRpcServerFactory.java:101) at com.alipay.sofa.jraft.rpc.RaftRpcServerFactory.addRaftRequestProcessors(RaftRpcServerFactory.java:85) at org.apache.hugegraph.backend.store.raft.RaftContext.wrapRpcServer(RaftContext.java:400) at org.apache.hugegraph.backend.store.raft.RaftContext.initRaftNode(RaftContext.java:150) at org.apache.hugegraph.backend.store.raft.RaftBackendStoreProvider.waitReady(RaftBackendStoreProvider.java:152) at org.apache.hugegraph.StandardHugeGraph.waitReady(StandardHugeGraph.java:318) at org.apache.hugegraph.core.GraphManager.waitGraphsReady(GraphManager.java:387) at org.apache.hugegraph.core.GraphManager.init(GraphManager.java:111) at org.apache.hugegraph.server.ApplicationConfig$GraphManagerFactory$1.onEvent(ApplicationConfig.java:130) at org.glassfish.jersey.server.internal.monitoring.CompositeApplicationEventListener.onEvent(CompositeApplicationEventListener.java:49) at org.glassfish.jersey.server.internal.monitoring.MonitoringContainerListener.onStartup(MonitoringContainerListener.java:56) at org.glassfish.jersey.server.ApplicationHandler.onStartup(ApplicationHandler.java:711) at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.start(GrizzlyHttpContainer.java:330) at org.glassfish.grizzly.http.server.HttpHandlerChain.start(HttpHandlerChain.java:376) at org.glassfish.grizzly.http.server.HttpServer.setupHttpHandler(HttpServer.java:268) at org.glassfish.grizzly.http.server.HttpServer.start(HttpServer.java:245) at org.apache.hugegraph.server.RestServer.start(RestServer.java:71) at org.apache.hugegraph.server.RestServer.start(RestServer.java:178) at org.apache.hugegraph.dist.HugeRestServer.start(HugeRestServer.java:32) at org.apache.hugegraph.dist.HugeGraphServer.<init>(HugeGraphServer.java:60) at org.apache.hugegraph.dist.HugeGraphServer.main(HugeGraphServer.java:120) ### Vertex/Edge example (问题点 / 边数据举例) _No response_ ### Schema [VertexLabel, EdgeLabel, IndexLabel] (元数据结构) _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
