Hi Sanju, Thanks for the response.
I have resolved the issue, actually I have updated from 3.7.6 to 5.0, in new version RPC is coming from libtirpb , but I forgot to enable "--with-libtirpc" in configuration. After enabling able to start glusterd. Regards, Abhishek On Wed, Mar 6, 2019 at 12:58 PM Sanju Rakonde <srako...@redhat.com> wrote: > Abhishek, > > We need below information on investigate this issue. > 1. gluster --version > 2. Please run glusterd in gdb, so that we can capture the backtrace. I see > some rpc errors in log, but backtrace will be more helpful. > To run glusterd in gdb, you need start glusterd in gdb (i.e. gdb > glusterd, and then give the command "run -N"). when you see a segmentation > fault, please capture the backtrace and paste it here. > > On Wed, Mar 6, 2019 at 10:07 AM ABHISHEK PALIWAL <abhishpali...@gmail.com> > wrote: > >> Hi Team, >> >> I am facing the issue where at the time of starting the glusterd >> segmentation fault is reported. >> >> Below are the logs >> >> root@128:/usr/sbin# ./glusterd --debug >> [1970-01-01 15:19:43.940386] I [MSGID: 100030] [glusterfsd.c:2691:main] >> 0-./glusterd: Started running ./glusterd version 5.0 (args: ./glusterd >> --debug) >> [1970-01-01 15:19:43.940855] D >> [logging.c:1833:__gf_log_inject_timer_event] 0-logging-infra: Starting >> timer now. Timeout = 120, current buf size = 5 >> [1970-01-01 15:19:43.941736] D [MSGID: 0] [glusterfsd.c:747:get_volfp] >> 0-glusterfsd: loading volume file /etc/glusterfs/glusterd.vol >> [1970-01-01 15:19:43.945796] D [MSGID: 101097] >> [xlator.c:341:xlator_dynload_newway] 0-xlator: dlsym(xlator_api) on >> /usr/lib64/glusterfs/5.0/xlator/mgmt/glusterd.so: undefined symbol: >> xlator_api. Fall back to old symbols >> [1970-01-01 15:19:43.946279] I [MSGID: 106478] [glusterd.c:1435:init] >> 0-management: Maximum allowed open file descriptors set to 65536 >> [1970-01-01 15:19:43.946419] I [MSGID: 106479] [glusterd.c:1491:init] >> 0-management: Using /var/lib/glusterd as working directory >> [1970-01-01 15:19:43.946515] I [MSGID: 106479] [glusterd.c:1497:init] >> 0-management: Using /var/run/gluster as pid file working directory >> [1970-01-01 15:19:43.946968] D [MSGID: 0] >> [glusterd.c:458:glusterd_rpcsvc_options_build] 0-glusterd: listen-backlog >> value: 10 >> [1970-01-01 15:19:43.947139] D [rpcsvc.c:2607:rpcsvc_init] 0-rpc-service: >> RPC service inited. >> [1970-01-01 15:19:43.947241] D [rpcsvc.c:2146:rpcsvc_program_register] >> 0-rpc-service: New program registered: GF-DUMP, Num: 123451501, Ver: 1, >> Port: 0 >> [1970-01-01 15:19:43.947379] D [rpc-transport.c:269:rpc_transport_load] >> 0-rpc-transport: attempt to load file >> /usr/lib64/glusterfs/5.0/rpc-transport/socket.so >> [1970-01-01 15:19:43.955198] D [socket.c:4464:socket_init] >> 0-socket.management: Configued transport.tcp-user-timeout=0 >> [1970-01-01 15:19:43.955316] D [socket.c:4482:socket_init] >> 0-socket.management: Reconfigued transport.keepalivecnt=9 >> [1970-01-01 15:19:43.955415] D >> [socket.c:4167:ssl_setup_connection_params] 0-socket.management: SSL >> support on the I/O path is NOT enabled >> [1970-01-01 15:19:43.955504] D >> [socket.c:4170:ssl_setup_connection_params] 0-socket.management: SSL >> support for glusterd is NOT enabled >> [1970-01-01 15:19:43.955612] D [name.c:572:server_fill_address_family] >> 0-socket.management: option address-family not specified, defaulting to >> inet6 >> [1970-01-01 15:19:43.955928] D [rpc-transport.c:269:rpc_transport_load] >> 0-rpc-transport: attempt to load file >> /usr/lib64/glusterfs/5.0/rpc-transport/rdma.so >> [1970-01-01 15:19:43.956079] E [rpc-transport.c:273:rpc_transport_load] >> 0-rpc-transport: /usr/lib64/glusterfs/5.0/rpc-transport/rdma.so: cannot >> open shared object file: No such file or directory >> [1970-01-01 15:19:43.956177] W [rpc-transport.c:277:rpc_transport_load] >> 0-rpc-transport: volume 'rdma.management': transport-type 'rdma' is not >> valid or not found on this machine >> [1970-01-01 15:19:43.956270] W [rpcsvc.c:1789:rpcsvc_create_listener] >> 0-rpc-service: cannot create listener, initing the transport failed >> [1970-01-01 15:19:43.956362] E [MSGID: 106244] [glusterd.c:1798:init] >> 0-management: creation of 1 listeners failed, continuing with succeeded >> transport >> [1970-01-01 15:19:43.956459] D [rpcsvc.c:2146:rpcsvc_program_register] >> 0-rpc-service: New program registered: GlusterD svc peer, Num: 1238437, >> Ver: 2, Port: 0 >> [1970-01-01 15:19:43.956561] D [rpcsvc.c:2146:rpcsvc_program_register] >> 0-rpc-service: New program registered: GlusterD svc cli read-only, Num: >> 1238463, Ver: 2, Port: 0 >> [1970-01-01 15:19:43.956666] D [rpcsvc.c:2146:rpcsvc_program_register] >> 0-rpc-service: New program registered: GlusterD svc mgmt, Num: 1238433, >> Ver: 2, Port: 0 >> [1970-01-01 15:19:43.956758] D [rpcsvc.c:2146:rpcsvc_program_register] >> 0-rpc-service: New program registered: GlusterD svc mgmt v3, Num: 1238433, >> Ver: 3, Port: 0 >> [1970-01-01 15:19:43.956853] D [rpcsvc.c:2146:rpcsvc_program_register] >> 0-rpc-service: New program registered: Gluster Portmap, Num: 34123456, Ver: >> 1, Port: 0 >> [1970-01-01 15:19:43.956946] D [rpcsvc.c:2146:rpcsvc_program_register] >> 0-rpc-service: New program registered: Gluster Handshake, Num: 14398633, >> Ver: 2, Port: 0 >> [1970-01-01 15:19:43.957062] D [rpcsvc.c:2146:rpcsvc_program_register] >> 0-rpc-service: New program registered: Gluster MGMT Handshake, Num: >> 1239873, Ver: 1, Port: 0 >> [1970-01-01 15:19:43.957205] D [rpcsvc.c:2607:rpcsvc_init] 0-rpc-service: >> RPC service inited. >> [1970-01-01 15:19:43.957303] D [rpcsvc.c:2146:rpcsvc_program_register] >> 0-rpc-service: New program registered: GF-DUMP, Num: 123451501, Ver: 1, >> Port: 0 >> [1970-01-01 15:19:43.957408] D [rpc-transport.c:269:rpc_transport_load] >> 0-rpc-transport: attempt to load file >> /usr/lib64/glusterfs/5.0/rpc-transport/socket.so >> [1970-01-01 15:19:43.957563] D [socket.c:4424:socket_init] >> 0-socket.management: disabling nodelay >> [1970-01-01 15:19:43.957650] D [socket.c:4464:socket_init] >> 0-socket.management: Configued transport.tcp-user-timeout=0 >> [1970-01-01 15:19:43.957738] D [socket.c:4482:socket_init] >> 0-socket.management: Reconfigued transport.keepalivecnt=9 >> [1970-01-01 15:19:43.957830] D >> [socket.c:4167:ssl_setup_connection_params] 0-socket.management: SSL >> support on the I/O path is NOT enabled >> [1970-01-01 15:19:43.957922] D >> [socket.c:4170:ssl_setup_connection_params] 0-socket.management: SSL >> support for glusterd is NOT enabled >> [1970-01-01 15:19:43.958186] D [rpcsvc.c:2146:rpcsvc_program_register] >> 0-rpc-service: New program registered: GlusterD svc cli, Num: 1238463, Ver: >> 2, Port: 0 >> [1970-01-01 15:19:43.958280] D [rpcsvc.c:2146:rpcsvc_program_register] >> 0-rpc-service: New program registered: Gluster Handshake (CLI Getspec), >> Num: 14398633, Ver: 2, Port: 0 >> [1970-01-01 15:19:43.958461] D [MSGID: 0] >> [glusterd-utils.c:7878:glusterd_sm_tr_log_init] 0-glusterd: returning 0 >> [1970-01-01 15:19:43.958557] D [MSGID: 0] [glusterd.c:1875:init] >> 0-management: cannot get run-with-valgrind value >> [1970-01-01 15:19:43.960895] E [MSGID: 101032] >> [store.c:447:gf_store_handle_retrieve] 0-: Path corresponding to >> /var/lib/glusterd/glusterd.info. [No such file or directory] >> [1970-01-01 15:19:43.961016] D [MSGID: 0] >> [store.c:452:gf_store_handle_retrieve] 0-: Returning -1 >> [1970-01-01 15:19:43.961108] D [MSGID: 0] >> [glusterd-store.c:2169:glusterd_retrieve_op_version] 0-management: Unable >> to get store handle! >> [1970-01-01 15:19:43.961216] E [MSGID: 101032] >> [store.c:447:gf_store_handle_retrieve] 0-: Path corresponding to >> /var/lib/glusterd/glusterd.info. [No such file or directory] >> [1970-01-01 15:19:43.961325] D [MSGID: 0] >> [store.c:452:gf_store_handle_retrieve] 0-: Returning -1 >> [1970-01-01 15:19:43.961428] D [MSGID: 0] >> [glusterd-store.c:2345:glusterd_retrieve_uuid] 0-management: Unable to get >> storehandle! >> [1970-01-01 15:19:43.961523] D [MSGID: 0] >> [glusterd-store.c:2366:glusterd_retrieve_uuid] 0-management: Returning -1 >> [1970-01-01 15:19:43.961617] I [MSGID: 106514] >> [glusterd-store.c:2304:glusterd_restore_op_version] 0-management: Detected >> new install. Setting op-version to maximum : 50000 >> [1970-01-01 15:19:43.962658] D [MSGID: 0] >> [store.c:432:gf_store_handle_new] 0-: Returning 0 >> [1970-01-01 15:19:43.962750] D [MSGID: 0] >> [store.c:452:gf_store_handle_retrieve] 0-: Returning 0 >> [1970-01-01 15:19:43.963047] D [MSGID: 0] [store.c:515:gf_store_iter_new] >> 0-: Returning with 0 >> [1970-01-01 15:19:43.963194] D [MSGID: 0] >> [store.c:632:gf_store_iter_get_next] 0-: Returning with 0 >> [1970-01-01 15:19:43.963318] D [MSGID: 0] >> [store.c:632:gf_store_iter_get_next] 0-: Returning with -1 >> [1970-01-01 15:19:43.963455] D [MSGID: 0] >> [store.c:473:gf_store_handle_destroy] 0-: Returning 0 >> [1970-01-01 15:19:43.963757] D [MSGID: 0] >> [glusterd-store.c:3546:glusterd_store_retrieve_volumes] 0-management: >> Returning with 0 >> [1970-01-01 15:19:43.964159] D [MSGID: 0] >> [glusterd-store.c:4662:glusterd_store_retrieve_peers] 0-management: >> Returning with 0 >> [1970-01-01 15:19:43.964471] I [MSGID: 106194] >> [glusterd-store.c:3983:glusterd_store_retrieve_missed_snaps_list] >> 0-management: No missed snaps list. >> [1970-01-01 15:19:43.964580] D [MSGID: 0] >> [glusterd-store.c:4104:glusterd_store_retrieve_snaps] 0-management: >> Returning with 0 >> [1970-01-01 15:19:43.964680] D [MSGID: 0] >> [glusterd-store.c:4894:glusterd_restore] 0-management: Returning 0 >> [1970-01-01 15:19:43.965060] D [MSGID: 0] >> [options.c:1225:xlator_option_init_int32] 0-management: option >> event-threads using set value 1 >> Final graph: >> >> +------------------------------------------------------------------------------+ >> 1: volume management >> 2: type mgmt/glusterd >> 3: option rpc-auth.auth-glusterfs on >> 4: option rpc-auth.auth-unix on >> 5: option rpc-auth.auth-null on >> 6: option rpc-auth-allow-insecure on >> 7: option transport.listen-backlog 10 >> 8: option event-threads 1 >> 9: option ping-timeout 0 >> 10: option transport.socket.read-fail-log off >> 11: option transport.socket.keepalive-interval 2 >> 12: option transport.socket.keepalive-time 10 >> 13: option transport-type rdma >> 14: option working-directory /var/lib/glusterd >> 15: end-volume >> 16: >> >> +------------------------------------------------------------------------------+ >> [1970-01-01 15:19:43.966808] I [MSGID: 101190] >> [event-epoll.c:622:event_dispatch_epoll_worker] 0-epoll: Started thread >> with index 1 >> [1970-01-01 15:19:44.840454] E [rpcsvc.c:513:rpcsvc_request_create] >> 0-rpc-service: RPC version not supported (XID: 0x0, Ver: 0, Program: 0, >> ProgVers: 0, Proc: 2) from trans (socket.management) >> [1970-01-01 15:19:44.840884] D [rpcsvc.c:1416:rpcsvc_error_reply] (--> >> /usr/lib64/libglusterfs.so.0(_gf_log_callingfn-0xffbac)[0x3fffa12acbe4] >> (--> /usr/lib64/libgfrpc.so.0(+0xc5f4)[0x3fffa12525f4] (--> >> /usr/lib64/libgfrpc.so.0(+0xcf00)[0x3fffa1252f00] (--> >> /usr/lib64/libgfrpc.so.0(+0xd224)[0x3fffa1253224] (--> >> /usr/lib64/libgfrpc.so.0(+0xd84c)[0x3fffa125384c] ))))) 0-: sending a RPC >> error reply >> [1970-01-01 15:19:44.841055] D [logging.c:1805:gf_log_flush_extra_msgs] >> 0-logging-infra: Log buffer size reduced. About to flush 5 extra log >> messages >> [1970-01-01 15:19:44.841156] D [logging.c:1808:gf_log_flush_extra_msgs] >> 0-logging-infra: Just flushed 5 extra log messages >> pending frames: >> patchset: git://git.gluster.org/glusterfs.git >> signal received: 11 >> time of crash: >> 1970-01-01 15:19:44 >> configuration details: >> argp 1 >> backtrace 1 >> dlfcn 1 >> libpthread 1 >> llistxattr 1 >> setfsid 1 >> spinlock 1 >> epoll.h 1 >> xattr.h 1 >> st_atim.tv_nsec 1 >> package-string: glusterfs 5.0 >> /usr/lib64/libglusterfs.so.0(+0x422a4)[0x3fffa12ab2a4] >> /usr/lib64/libglusterfs.so.0(gf_print_trace-0xf5080)[0x3fffa12b82e0] >> ./glusterd(glusterfsd_print_trace-0x22fa4)[0x100067ec] >> linux-vdso64.so.1(__kernel_sigtramp_rt64+0x0)[0x3fffa13f0478] >> /lib64/libc.so.6(xdr_accepted_reply-0x72d3c)[0x3fffa11375cc] >> /lib64/libc.so.6(xdr_accepted_reply-0x72d9c)[0x3fffa113756c] >> /lib64/libc.so.6(xdr_union-0x63a94)[0x3fffa1147dd4] >> /lib64/libc.so.6(xdr_replymsg-0x72c58)[0x3fffa11376e0] >> /lib64/libc.so.6(xdr_sizeof-0x62a78)[0x3fffa1149120] >> /usr/lib64/libgfrpc.so.0(+0x9b0c)[0x3fffa124fb0c] >> /usr/lib64/libgfrpc.so.0(rpcsvc_submit_generic-0x149f4)[0x3fffa125228c] >> /usr/lib64/libgfrpc.so.0(+0xc614)[0x3fffa1252614] >> /usr/lib64/libgfrpc.so.0(+0xcf00)[0x3fffa1252f00] >> /usr/lib64/libgfrpc.so.0(+0xd224)[0x3fffa1253224] >> /usr/lib64/libgfrpc.so.0(+0xd84c)[0x3fffa125384c] >> /usr/lib64/libgfrpc.so.0(rpc_transport_notify-0x10eec)[0x3fffa125610c] >> /usr/lib64/glusterfs/5.0/rpc-transport/socket.so(+0xc09c)[0x3fff9d51709c] >> /usr/lib64/libglusterfs.so.0(+0xb84bc)[0x3fffa13214bc] >> /lib64/libpthread.so.0(+0xbb30)[0x3fffa11bdb30] >> /lib64/libc.so.6(clone-0x9e964)[0x3fffa110817c] >> --------- >> Segmentation fault (core dumped) >> >> Could you please help me, what actually the problem? >> >> >> -- >> >> >> >> >> Regards >> Abhishek Paliwal >> _______________________________________________ >> Gluster-users mailing list >> Gluster-users@gluster.org >> https://lists.gluster.org/mailman/listinfo/gluster-users > > > > -- > Thanks, > Sanju > -- Regards Abhishek Paliwal
_______________________________________________ Gluster-users mailing list Gluster-users@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-users