Re: [lustre-discuss] Static lfs?

[Ben Evans]

I also seem to recall that make install and RPM use different install 
directories for things, and you could be getting old libraries from the RPMs 
but the new lfs.

I know tests go into either /usr/lib/lustre or /usr/lib64/lustre depending, and 
the .ko's go into different places as well.

-Ben

From: lustre-discuss 
>
 on behalf of "Dilger, Andreas" 
>
Date: Saturday, March 24, 2018 at 6:02 PM
To: Patrick Farrell >
Cc: "lustre-discuss@lists.lustre.org" 
>
Subject: Re: [lustre-discuss] Static lfs?

The "lfs" found in the build tree is a script wrapper generated by libtool to 
set up LD_LIBRARY_PATH and call the actual binary, which is in 
lustre/utils/.libs/lt-lfs or something similar.  I'm not much for libtool, but 
I figured out this much a few weeks ago when I wanted to run strace on lfs.

This is needed because the lfs binary is dynamically linked to libraries in the 
build tree, which are needed for it to run directly from the build tree, but 
would not be found by "ld" in their current locations otherwise.

If you run "ldd" on the real binary, it will tell you which libraries are 
needed. You can copy the requisite paths to the new system library directories 
to be able to run the actual binary there, and just ignore the lfs libtool 
wrapper script.

Cheers, Andreas

On Mar 23, 2018, at 15:59, Patrick Farrell 
> wrote:


Another off list note pointing out that lfs is likely a script now.  So here's 
the bitter end:


Ah, it looks like you're correct.  There's still an lfs.c but it no longer 
generates the "lfs" executable as it previously - Instead there's a lengthy and 
complex script named "lfs" which is not invoked by "make", but only during the 
install process.  That generates the lfs binary that is actually installed...

Uck.  Well, I found where it squirrels away the real binary when executed.

Run the script lustre/utils/lfs in your build dir, and it will start lfs.  Quit 
it, and you will find the actual lfs binary in lustre/utils/.libs/lt-lfs

Maybe this particular bit of build tooling would be clearer if it didn't try to 
pretend it didn't exist by apeing the binary without actually being it?

Thanks to John Bauer for help with this.



From: lustre-discuss 
>
 on behalf of Patrick Farrell >
Sent: Friday, March 23, 2018 3:17:14 PM
To: lustre-discuss@lists.lustre.org
Subject: Re: [lustre-discuss] Static lfs?


Ah, interesting – I got a question off list about this, but I thought I’d reply 
here.



‘ldd’ on the lfs binary says “not a dynamic executable”.



So it seems I’m confused (never was much for compilers and linkers).  Here are 
the errors I get trying to run it on another node:
./lfs: line 202: cd: /home/build/paf/[……..]/lustre/utils: No such file or 
directory

gcc: error: lfs.o: No such file or directory

gcc: error: lfs_project.o: No such file or directory

gcc: error: ./.libs/liblustreapi.so: No such file or directory

gcc: error: ../../lnet/utils/lnetconfig/.libs/liblnetconfig.so



From: lustre-discuss 
>
 on behalf of Patrick Farrell >
Date: Friday, March 23, 2018 at 3:03 PM
To: "lustre-discuss@lists.lustre.org" 
>
Subject: [lustre-discuss] Static lfs?



Good afternoon,



I’ve got a developer question that perhaps someone has some insight on.  After 
some recent (a few months ago now) changes to make the Lustre libraries and 
utilities build dynamically linked rather than statically linked, I’ve got a 
problem.  If I build an lfs binary just by doing “make”, the resultant binary 
looks for various libraries in the build directories and cannot be run on any 
system other than the one it was built on (well, I guess without replicating 
the build directory structure).  When doing make rpms and installing the RPMs, 
it works fine.  The problem is “make rpms” takes ~5 minutes, as opposed to ~1 
second for “make” in /utils.  (I assume “make install” works too, but I 
explicitly need to test on nodes other than the one where I’m doing the build, 
so that’s not an option.)



Does anyone have any insight on a way around this for a developer?  Either some 
tweak I can make locally to get static builds again, or some fix to make that 
would let the dynamically linked binary from “make” have correct library paths? 
 (To be completely 

Re: [lustre-discuss] MDT LBUG after every restart

[Thomas Roth]

For the record:

This was triggered by one job script using a working directory in a directory tree under MDT0 and 
redirecting stderr of gzip commands to a directory in a treen under MDT1.

Once the user put all of it in either one or the other tree, the problem 
disappeared.

Thomas

On 03/12/2018 09:54 AM, Thomas Roth wrote:

Hi all,

our production system running Lustre 2.5.3 has broken down, and I'm quite 
clueless.

The second (of two) MDTs crashed and after reboot + recovery LBUGs again with:


Mar 11 20:02:37 lxmds15 kernel: Lustre: nyx-MDT0001: Recovery over after 1:36, of 720 clients 720 
recovered and 0 were evicted.


Mar 11 20:02:37 lxmds15 kernel: LustreError: 
6705:0:(osp_precreate.c:719:osp_precreate_cleanup_orphans()) nyx-OST0001-osc-MDT0001: cannot cleanup

orphans: rc = -108

Mar 11 20:02:37 lxmds15 kernel: LustreError: 
6705:0:(osp_precreate.c:719:osp_precreate_cleanup_orphans()) Skipped 74 previous similar messages


Mar 11 20:02:37 lxmds15 kernel: LustreError: 6574:0:(mdt_handler.c:2706:mdt_object_lock0()) ASSERTION( 
!(ibits & (MDS_INODELOCK_UPDATE |

MDS_INODELOCK_PERM)) ) failed: nyx-MDT0001: wrong bit 0x2 for remote obj 
[0x5100027c70:0x17484:0x0]

Mar 11 20:02:37 lxmds15 kernel: LustreError: 
6574:0:(mdt_handler.c:2706:mdt_object_lock0()) LBUG



This seems to be LU-6071, but I am wondering what actually causes it - there should be no ongoing 
attempts from a client to create a directory on the second MDT.



After doing an e2fsck on the MDT, it mounts and then crashes with a different FID each time. (If 
mounted without fsck, the crashing FID remains the same.)



Is there any way we can find out more about the cause?

If it is a finite number of troubling inodes, is there a trick to 
manipulate/clear these?


Regards,
Thomas



___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org