On 16/06/15 15:03, LEIBOVICI Thomas wrote: > I understood your servers run Lustre 2.7.
Correct. > Does robinhood run on a Lustre 2.7 client or an older version ? Lustre 2.7 client. > Did you build robinhood on a host where Lustre 2.7 is installed or an > older version? yes, robinhood has been compiled on the same client when it was already running lustre 2.7. > Could it be possible that robinhood is currently processing changelog > records that were generated while you was still running older Lustre (2.5?) I'm not entirely sure because I didn't check how far it got in processing the files but I don't think so. We have upgraded the servers about two weeks ago but left the robinhood running on one of the clients we didn't upgrade. Yesterday we finally also upgraded the robinhood server machine (and robinhood). So robinhood would have been happily processing changelogs while running on a 2.5 client (using robinhood 2.5.4.), but started crashing after upgrading the client to lustre 2.7 (and robinhood to 2.5.5). > Also, it would be helpful to run "gdb" on a binary that include debug > information, basically from the source tree: > ./configure > gdb --args ./src/robinhood/robinhood --readlog > > run [bnh65367@cs04r-sc-serv-92 robinhood-2.5.5]$ sudo !! sudo gdb --args ./src/robinhood/robinhood --readlog GNU gdb (GDB) Red Hat Enterprise Linux (7.2-75.el6) Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /tmp/robinhood-2.5.5/src/robinhood/robinhood...done. (gdb) run Starting program: /tmp/robinhood-2.5.5/src/robinhood/robinhood --readlog [Thread debugging using libthread_db enabled] Using config file '/etc/robinhood.d/tmpfs/lustre03.conf'. [New Thread 0x7ffff7fe3700 (LWP 77963)] [New Thread 0x7ffff75e2700 (LWP 77964)] [Thread 0x7ffff75e2700 (LWP 77964) exited] [New Thread 0x7ffff75e2700 (LWP 77965)] [New Thread 0x7ffff67ce700 (LWP 77966)] [New Thread 0x7ffff5dcd700 (LWP 77967)] [New Thread 0x7fffeffff700 (LWP 77968)] [New Thread 0x7ffff53cc700 (LWP 77969)] [New Thread 0x7fffef5fe700 (LWP 77970)] [New Thread 0x7fffeebfd700 (LWP 77972)] [New Thread 0x7fffee1fc700 (LWP 77974)] [New Thread 0x7fffed7fb700 (LWP 77979)] Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7fffed7fb700 (LWP 77979)] 0x0000000000429c0e in create_fake_unlink_record (p_info=<value optimized out>, rec_in=<value optimized out>, insert_flags=0x7fffed7fadbc) at chglog_reader.c:657 657 rh_get_cl_cr_name(rec)[name_len] = 0; /* terminate string */ Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.149.el6_6.7.x86_64 keyutils-libs-1.4-5.el6.x86_64 krb5-libs-1.10.3-37.el6_6.x86_64 libcom_err-1.41.12-21.el6.x86_64 libgcc-4.4.7-11.el6.x86_64 libselinux-2.0.94-5.8.el6.x86_64 lustre-client-2.7.0-2.6.32_504.16.2.el6.x86_64.x86_64 mysql-libs-5.1.73-3.el6_5.x86_64 nss-softokn-freebl-3.14.3-22.el6_6.x86_64 openssl-1.0.1e-30.el6_6.8.x86_64 zlib-1.2.3-29.el6.x86_64 (gdb) where #0 0x0000000000429c0e in create_fake_unlink_record (p_info=<value optimized out>, rec_in=<value optimized out>, insert_flags=0x7fffed7fadbc) at chglog_reader.c:657 #1 0x000000000042b179 in process_log_rec (arg=0x6b5b80) at chglog_reader.c:823 #2 chglog_reader_thr (arg=0x6b5b80) at chglog_reader.c:1082 #3 0x0000003c260079d1 in start_thread () from /lib64/libpthread.so.0 #4 0x0000003c258e88fd in clone () from /lib64/libc.so.6 (gdb) Thanks, Frederik -- Frederik Ferner Senior Computer Systems Administrator (storage) phone: +44 1235 77 8624 Diamond Light Source Ltd. mob: +44 7917 08 5110 Duty Sys Admin can be reached at x8596 (Apologies in advance for the lines below. Some bits are a legal requirement and I have no control over them.) -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom ------------------------------------------------------------------------------ _______________________________________________ robinhood-support mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/robinhood-support
