Re: [lustre-discuss] Static lfs?
I also seem to recall that make install and RPM use different install directories for things, and you could be getting old libraries from the RPMs but the new lfs. I know tests go into either /usr/lib/lustre or /usr/lib64/lustre depending, and the .ko's go into different places as well. -Ben From: lustre-discuss <lustre-discuss-boun...@lists.lustre.org<mailto:lustre-discuss-boun...@lists.lustre.org>> on behalf of "Dilger, Andreas" <andreas.dil...@intel.com<mailto:andreas.dil...@intel.com>> Date: Saturday, March 24, 2018 at 6:02 PM To: Patrick Farrell <p...@cray.com<mailto:p...@cray.com>> Cc: "lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org>" <lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org>> Subject: Re: [lustre-discuss] Static lfs? The "lfs" found in the build tree is a script wrapper generated by libtool to set up LD_LIBRARY_PATH and call the actual binary, which is in lustre/utils/.libs/lt-lfs or something similar. I'm not much for libtool, but I figured out this much a few weeks ago when I wanted to run strace on lfs. This is needed because the lfs binary is dynamically linked to libraries in the build tree, which are needed for it to run directly from the build tree, but would not be found by "ld" in their current locations otherwise. If you run "ldd" on the real binary, it will tell you which libraries are needed. You can copy the requisite paths to the new system library directories to be able to run the actual binary there, and just ignore the lfs libtool wrapper script. Cheers, Andreas On Mar 23, 2018, at 15:59, Patrick Farrell <p...@cray.com<mailto:p...@cray.com>> wrote: Another off list note pointing out that lfs is likely a script now. So here's the bitter end: Ah, it looks like you're correct. There's still an lfs.c but it no longer generates the "lfs" executable as it previously - Instead there's a lengthy and complex script named "lfs" which is not invoked by "make", but only during the install process. That generates the lfs binary that is actually installed... Uck. Well, I found where it squirrels away the real binary when executed. Run the script lustre/utils/lfs in your build dir, and it will start lfs. Quit it, and you will find the actual lfs binary in lustre/utils/.libs/lt-lfs Maybe this particular bit of build tooling would be clearer if it didn't try to pretend it didn't exist by apeing the binary without actually being it? Thanks to John Bauer for help with this. From: lustre-discuss <lustre-discuss-boun...@lists.lustre.org<mailto:lustre-discuss-boun...@lists.lustre.org>> on behalf of Patrick Farrell <p...@cray.com<mailto:p...@cray.com>> Sent: Friday, March 23, 2018 3:17:14 PM To: lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> Subject: Re: [lustre-discuss] Static lfs? Ah, interesting – I got a question off list about this, but I thought I’d reply here. ‘ldd’ on the lfs binary says “not a dynamic executable”. So it seems I’m confused (never was much for compilers and linkers). Here are the errors I get trying to run it on another node: ./lfs: line 202: cd: /home/build/paf/[……..]/lustre/utils: No such file or directory gcc: error: lfs.o: No such file or directory gcc: error: lfs_project.o: No such file or directory gcc: error: ./.libs/liblustreapi.so: No such file or directory gcc: error: ../../lnet/utils/lnetconfig/.libs/liblnetconfig.so From: lustre-discuss <lustre-discuss-boun...@lists.lustre.org<mailto:lustre-discuss-boun...@lists.lustre.org>> on behalf of Patrick Farrell <p...@cray.com<mailto:p...@cray.com>> Date: Friday, March 23, 2018 at 3:03 PM To: "lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org>" <lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org>> Subject: [lustre-discuss] Static lfs? Good afternoon, I’ve got a developer question that perhaps someone has some insight on. After some recent (a few months ago now) changes to make the Lustre libraries and utilities build dynamically linked rather than statically linked, I’ve got a problem. If I build an lfs binary just by doing “make”, the resultant binary looks for various libraries in the build directories and cannot be run on any system other than the one it was built on (well, I guess without replicating the build directory structure). When doing make rpms and installing the RPMs, it works fine. The problem is “make rpms” takes ~5 minutes, as opposed to ~1 second for “make” in /utils. (I assume “make install” works too, but I explicitly need to test on nodes other than the one where I’m doing the build, so that’s not an option.) Does anyone have any insight on
Re: [lustre-discuss] Static lfs?
The "lfs" found in the build tree is a script wrapper generated by libtool to set up LD_LIBRARY_PATH and call the actual binary, which is in lustre/utils/.libs/lt-lfs or something similar. I'm not much for libtool, but I figured out this much a few weeks ago when I wanted to run strace on lfs. This is needed because the lfs binary is dynamically linked to libraries in the build tree, which are needed for it to run directly from the build tree, but would not be found by "ld" in their current locations otherwise. If you run "ldd" on the real binary, it will tell you which libraries are needed. You can copy the requisite paths to the new system library directories to be able to run the actual binary there, and just ignore the lfs libtool wrapper script. Cheers, Andreas On Mar 23, 2018, at 15:59, Patrick Farrell <p...@cray.com<mailto:p...@cray.com>> wrote: Another off list note pointing out that lfs is likely a script now. So here's the bitter end: Ah, it looks like you're correct. There's still an lfs.c but it no longer generates the "lfs" executable as it previously - Instead there's a lengthy and complex script named "lfs" which is not invoked by "make", but only during the install process. That generates the lfs binary that is actually installed... Uck. Well, I found where it squirrels away the real binary when executed. Run the script lustre/utils/lfs in your build dir, and it will start lfs. Quit it, and you will find the actual lfs binary in lustre/utils/.libs/lt-lfs Maybe this particular bit of build tooling would be clearer if it didn't try to pretend it didn't exist by apeing the binary without actually being it? Thanks to John Bauer for help with this. From: lustre-discuss <lustre-discuss-boun...@lists.lustre.org<mailto:lustre-discuss-boun...@lists.lustre.org>> on behalf of Patrick Farrell <p...@cray.com<mailto:p...@cray.com>> Sent: Friday, March 23, 2018 3:17:14 PM To: lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> Subject: Re: [lustre-discuss] Static lfs? Ah, interesting – I got a question off list about this, but I thought I’d reply here. ‘ldd’ on the lfs binary says “not a dynamic executable”. So it seems I’m confused (never was much for compilers and linkers). Here are the errors I get trying to run it on another node: ./lfs: line 202: cd: /home/build/paf/[……..]/lustre/utils: No such file or directory gcc: error: lfs.o: No such file or directory gcc: error: lfs_project.o: No such file or directory gcc: error: ./.libs/liblustreapi.so: No such file or directory gcc: error: ../../lnet/utils/lnetconfig/.libs/liblnetconfig.so From: lustre-discuss <lustre-discuss-boun...@lists.lustre.org<mailto:lustre-discuss-boun...@lists.lustre.org>> on behalf of Patrick Farrell <p...@cray.com<mailto:p...@cray.com>> Date: Friday, March 23, 2018 at 3:03 PM To: "lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org>" <lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org>> Subject: [lustre-discuss] Static lfs? Good afternoon, I’ve got a developer question that perhaps someone has some insight on. After some recent (a few months ago now) changes to make the Lustre libraries and utilities build dynamically linked rather than statically linked, I’ve got a problem. If I build an lfs binary just by doing “make”, the resultant binary looks for various libraries in the build directories and cannot be run on any system other than the one it was built on (well, I guess without replicating the build directory structure). When doing make rpms and installing the RPMs, it works fine. The problem is “make rpms” takes ~5 minutes, as opposed to ~1 second for “make” in /utils. (I assume “make install” works too, but I explicitly need to test on nodes other than the one where I’m doing the build, so that’s not an option.) Does anyone have any insight on a way around this for a developer? Either some tweak I can make locally to get static builds again, or some fix to make that would let the dynamically linked binary from “make” have correct library paths? (To be completely clear: The dynamically linked binary from “make” looks for libraries in the locations where they are built, regardless of whether or not they’re already installed in the normal system library locations.) Regards, Patrick Farrell ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org<mailto:lustre-discuss@lists.lustre.org> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
Re: [lustre-discuss] Static lfs?
Another off list note pointing out that lfs is likely a script now. So here's the bitter end: Ah, it looks like you're correct. There's still an lfs.c but it no longer generates the "lfs" executable as it previously - Instead there's a lengthy and complex script named "lfs" which is not invoked by "make", but only during the install process. That generates the lfs binary that is actually installed... Uck. Well, I found where it squirrels away the real binary when executed. Run the script lustre/utils/lfs in your build dir, and it will start lfs. Quit it, and you will find the actual lfs binary in lustre/utils/.libs/lt-lfs Maybe this particular bit of build tooling would be clearer if it didn't try to pretend it didn't exist by apeing the binary without actually being it? Thanks to John Bauer for help with this. From: lustre-discuss <lustre-discuss-boun...@lists.lustre.org> on behalf of Patrick Farrell <p...@cray.com> Sent: Friday, March 23, 2018 3:17:14 PM To: lustre-discuss@lists.lustre.org Subject: Re: [lustre-discuss] Static lfs? Ah, interesting – I got a question off list about this, but I thought I’d reply here. ‘ldd’ on the lfs binary says “not a dynamic executable”. So it seems I’m confused (never was much for compilers and linkers). Here are the errors I get trying to run it on another node: ./lfs: line 202: cd: /home/build/paf/[……..]/lustre/utils: No such file or directory gcc: error: lfs.o: No such file or directory gcc: error: lfs_project.o: No such file or directory gcc: error: ./.libs/liblustreapi.so: No such file or directory gcc: error: ../../lnet/utils/lnetconfig/.libs/liblnetconfig.so From: lustre-discuss <lustre-discuss-boun...@lists.lustre.org> on behalf of Patrick Farrell <p...@cray.com> Date: Friday, March 23, 2018 at 3:03 PM To: "lustre-discuss@lists.lustre.org" <lustre-discuss@lists.lustre.org> Subject: [lustre-discuss] Static lfs? Good afternoon, I’ve got a developer question that perhaps someone has some insight on. After some recent (a few months ago now) changes to make the Lustre libraries and utilities build dynamically linked rather than statically linked, I’ve got a problem. If I build an lfs binary just by doing “make”, the resultant binary looks for various libraries in the build directories and cannot be run on any system other than the one it was built on (well, I guess without replicating the build directory structure). When doing make rpms and installing the RPMs, it works fine. The problem is “make rpms” takes ~5 minutes, as opposed to ~1 second for “make” in /utils. (I assume “make install” works too, but I explicitly need to test on nodes other than the one where I’m doing the build, so that’s not an option.) Does anyone have any insight on a way around this for a developer? Either some tweak I can make locally to get static builds again, or some fix to make that would let the dynamically linked binary from “make” have correct library paths? (To be completely clear: The dynamically linked binary from “make” looks for libraries in the locations where they are built, regardless of whether or not they’re already installed in the normal system library locations.) Regards, Patrick Farrell ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
Re: [lustre-discuss] Static lfs?
Ah, interesting – I got a question off list about this, but I thought I’d reply here. ‘ldd’ on the lfs binary says “not a dynamic executable”. So it seems I’m confused (never was much for compilers and linkers). Here are the errors I get trying to run it on another node: ./lfs: line 202: cd: /home/build/paf/[……..]/lustre/utils: No such file or directory gcc: error: lfs.o: No such file or directory gcc: error: lfs_project.o: No such file or directory gcc: error: ./.libs/liblustreapi.so: No such file or directory gcc: error: ../../lnet/utils/lnetconfig/.libs/liblnetconfig.so From: lustre-discuss <lustre-discuss-boun...@lists.lustre.org> on behalf of Patrick Farrell <p...@cray.com> Date: Friday, March 23, 2018 at 3:03 PM To: "lustre-discuss@lists.lustre.org" <lustre-discuss@lists.lustre.org> Subject: [lustre-discuss] Static lfs? Good afternoon, I’ve got a developer question that perhaps someone has some insight on. After some recent (a few months ago now) changes to make the Lustre libraries and utilities build dynamically linked rather than statically linked, I’ve got a problem. If I build an lfs binary just by doing “make”, the resultant binary looks for various libraries in the build directories and cannot be run on any system other than the one it was built on (well, I guess without replicating the build directory structure). When doing make rpms and installing the RPMs, it works fine. The problem is “make rpms” takes ~5 minutes, as opposed to ~1 second for “make” in /utils. (I assume “make install” works too, but I explicitly need to test on nodes other than the one where I’m doing the build, so that’s not an option.) Does anyone have any insight on a way around this for a developer? Either some tweak I can make locally to get static builds again, or some fix to make that would let the dynamically linked binary from “make” have correct library paths? (To be completely clear: The dynamically linked binary from “make” looks for libraries in the locations where they are built, regardless of whether or not they’re already installed in the normal system library locations.) Regards, Patrick Farrell ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
[lustre-discuss] Static lfs?
Good afternoon, I’ve got a developer question that perhaps someone has some insight on. After some recent (a few months ago now) changes to make the Lustre libraries and utilities build dynamically linked rather than statically linked, I’ve got a problem. If I build an lfs binary just by doing “make”, the resultant binary looks for various libraries in the build directories and cannot be run on any system other than the one it was built on (well, I guess without replicating the build directory structure). When doing make rpms and installing the RPMs, it works fine. The problem is “make rpms” takes ~5 minutes, as opposed to ~1 second for “make” in /utils. (I assume “make install” works too, but I explicitly need to test on nodes other than the one where I’m doing the build, so that’s not an option.) Does anyone have any insight on a way around this for a developer? Either some tweak I can make locally to get static builds again, or some fix to make that would let the dynamically linked binary from “make” have correct library paths? (To be completely clear: The dynamically linked binary from “make” looks for libraries in the locations where they are built, regardless of whether or not they’re already installed in the normal system library locations.) Regards, Patrick Farrell ___ lustre-discuss mailing list lustre-discuss@lists.lustre.org http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
