The LTO driver requires libelf and currently grovels around in the system directories looking for it, which may not always be the right place to find it. (This bit me when building LTO on our new Linux machines, which do not have libelf installed.) The Right Thing would be to add a --with-libelf flag to configure so we wouldn't have to assume it's always installed in one particular place. A small LTO project would be for someone to add a --with-libelf to configure.
I'm not an expert at configury; would somebody else like to take a whack at this? -Nathan