bug#21076: dynamic-link often fails to load libraries

[Ludovic Courtès]

Hi Isaac,

Isaac Jurado  skribis:

> From b337d412b3c10cabe355df07d2295c4d0a560b10 Mon Sep 17 00:00:00 2001
> From: Isaac Jurado 
> Date: Sat, 8 Jun 2019 14:00:29 +0200
> Subject: [PATCH] Interpret dynamic library name as literal path first.
>
> Fixes .
>
> * libguile/dynl.c (sysdep_dyn_link): Try plain lt_dlopen first, to
>   interpret fname as a literal path.
> * doc/ref/api-foreign.texi: Update explanation to describe the new
>   behavior.

It’s taken a while (!), but thanks to Matt Wette’s ping, it’s now
applied on ‘master’ and will make it into 3.0.2.

I slightly modified the documentation wording as shown in the patch
below (the GNU convention is to use “file name” for file names and
“path” for search paths).

Like you wrote, it’s hard to test, but I think it’s OK to leave it as is.

Thanks!

Ludo’.

diff --git a/doc/ref/api-foreign.texi b/doc/ref/api-foreign.texi
index 94fabf23c..b0d6c249b 100644
--- a/doc/ref/api-foreign.texi
+++ b/doc/ref/api-foreign.texi
@@ -75,14 +75,14 @@ Scheme object suitable for representing the linked object file.
 Otherwise an error is thrown.  How object files are searched is system
 dependent.
 
-Guile first tries to load @var{library} as a full path to a shared
-library file.  If that fails, then it falls back to interpret
+Guile first tries to load @var{library} as the absolute file name of a shared
+library.  If that fails, it then falls back to interpret
 @var{library} as just the name of some shared library that will be
 searched for in the places where shared libraries usually reside, such
 as @file{/usr/lib} and @file{/usr/local/lib}.
 
 @var{library} should not contain an extension such as @code{.so}, unless
-@var{library} represents the full path to the shared library file.  The
+@var{library} represents the absolute file name to the shared library.  The
 correct file name extension for the host operating system is provided
 automatically, according to libltdl's rules (@pxref{Libltdl interface,
 lt_dlopenext, @code{lt_dlopenext}, libtool, Shared Library Support for

bug#21076: dynamic-link often fails to load libraries

[Isaac Jurado]

I've experienced this issue as well, but the fix is very simple
(attaching Git patch).

The fix does NOT break foreign.test but I have no idea, yet, on how to
test the new behaviour in a platform independent or continuous
integration friendly way.

The commit has been applied to the stable-2.2 branch, but it would
also apply cleanly to master.

Any feedback welcomed.

Best regards.

-- 
Isaac Jurado

"The noblest pleasure is the joy of understanding"
Leonardo da Vinci
From b337d412b3c10cabe355df07d2295c4d0a560b10 Mon Sep 17 00:00:00 2001
From: Isaac Jurado 
Date: Sat, 8 Jun 2019 14:00:29 +0200
Subject: [PATCH] Interpret dynamic library name as literal path first.

Fixes .

* libguile/dynl.c (sysdep_dyn_link): Try plain lt_dlopen first, to
  interpret fname as a literal path.
* doc/ref/api-foreign.texi: Update explanation to describe the new
  behavior.
---
 doc/ref/api-foreign.texi | 13 -
 libguile/dynl.c  |  7 +++
 2 files changed, 11 insertions(+), 9 deletions(-)

diff --git a/doc/ref/api-foreign.texi b/doc/ref/api-foreign.texi
index d99a33300..94fabf23c 100644
--- a/doc/ref/api-foreign.texi
+++ b/doc/ref/api-foreign.texi
@@ -75,11 +75,14 @@ Scheme object suitable for representing the linked object file.
 Otherwise an error is thrown.  How object files are searched is system
 dependent.
 
-Normally, @var{library} is just the name of some shared library file
-that will be searched for in the places where shared libraries usually
-reside, such as in @file{/usr/lib} and @file{/usr/local/lib}.
-
-@var{library} should not contain an extension such as @code{.so}.  The
+Guile first tries to load @var{library} as a full path to a shared
+library file.  If that fails, then it falls back to interpret
+@var{library} as just the name of some shared library that will be
+searched for in the places where shared libraries usually reside, such
+as @file{/usr/lib} and @file{/usr/local/lib}.
+
+@var{library} should not contain an extension such as @code{.so}, unless
+@var{library} represents the full path to the shared library file.  The
 correct file name extension for the host operating system is provided
 automatically, according to libltdl's rules (@pxref{Libltdl interface,
 lt_dlopenext, @code{lt_dlopenext}, libtool, Shared Library Support for
diff --git a/libguile/dynl.c b/libguile/dynl.c
index 2a25e5d2e..0096e8cf5 100644
--- a/libguile/dynl.c
+++ b/libguile/dynl.c
@@ -79,10 +79,9 @@ sysdep_dynl_link (const char *fname, const char *subr)
 {
   lt_dlhandle handle;
 
-  if (fname == NULL)
-/* Return a handle for the program as a whole.  */
-handle = lt_dlopen (NULL);
-  else
+  /* Try the literal filename first or, if NULL, the program itself */
+  handle = lt_dlopen (fname);
+  if (handle == NULL)
 {
   handle = lt_dlopenext (fname);
 
-- 
2.17.1

bug#21076: dynamic-link often fails to load libraries

[Isaac Jurado]

I've experienced this issue as well, but the fix is very simple
(attaching Git patch).

The fix does NOT break foreign.test but I have no idea, yet, on how to
test the new behaviour in a platform independent or continuous
integration friendly way.

The commit has been applied to the stable-2.2 branch, but it would
also apply cleanly to master.

Any feedback welcomed.

Best regards.

-- 
Isaac Jurado

"The noblest pleasure is the joy of understanding"
Leonardo da Vinci
From b337d412b3c10cabe355df07d2295c4d0a560b10 Mon Sep 17 00:00:00 2001
From: Isaac Jurado 
Date: Sat, 8 Jun 2019 14:00:29 +0200
Subject: [PATCH] Interpret dynamic library name as literal path first.

Fixes .

* libguile/dynl.c (sysdep_dyn_link): Try plain lt_dlopen first, to
  interpret fname as a literal path.
* doc/ref/api-foreign.texi: Update explanation to describe the new
  behavior.
---
 doc/ref/api-foreign.texi | 13 -
 libguile/dynl.c  |  7 +++
 2 files changed, 11 insertions(+), 9 deletions(-)

diff --git a/doc/ref/api-foreign.texi b/doc/ref/api-foreign.texi
index d99a33300..94fabf23c 100644
--- a/doc/ref/api-foreign.texi
+++ b/doc/ref/api-foreign.texi
@@ -75,11 +75,14 @@ Scheme object suitable for representing the linked object file.
 Otherwise an error is thrown.  How object files are searched is system
 dependent.
 
-Normally, @var{library} is just the name of some shared library file
-that will be searched for in the places where shared libraries usually
-reside, such as in @file{/usr/lib} and @file{/usr/local/lib}.
-
-@var{library} should not contain an extension such as @code{.so}.  The
+Guile first tries to load @var{library} as a full path to a shared
+library file.  If that fails, then it falls back to interpret
+@var{library} as just the name of some shared library that will be
+searched for in the places where shared libraries usually reside, such
+as @file{/usr/lib} and @file{/usr/local/lib}.
+
+@var{library} should not contain an extension such as @code{.so}, unless
+@var{library} represents the full path to the shared library file.  The
 correct file name extension for the host operating system is provided
 automatically, according to libltdl's rules (@pxref{Libltdl interface,
 lt_dlopenext, @code{lt_dlopenext}, libtool, Shared Library Support for
diff --git a/libguile/dynl.c b/libguile/dynl.c
index 2a25e5d2e..0096e8cf5 100644
--- a/libguile/dynl.c
+++ b/libguile/dynl.c
@@ -79,10 +79,9 @@ sysdep_dynl_link (const char *fname, const char *subr)
 {
   lt_dlhandle handle;
 
-  if (fname == NULL)
-/* Return a handle for the program as a whole.  */
-handle = lt_dlopen (NULL);
-  else
+  /* Try the literal filename first or, if NULL, the program itself */
+  handle = lt_dlopen (fname);
+  if (handle == NULL)
 {
   handle = lt_dlopenext (fname);
 
-- 
2.17.1

bug#21076: dynamic-link often fails to load libraries

[Andy Wingo]

On Fri 17 Jul 2015 01:00, Andreas Rottmann  writes:

>> Alternatively, the (system foreign) module in guile could provide a
>> simple low-level wrapper around lt_dlopen, and possibly implement
>> the higher level functionality of `dynamic-link' (additional search
>> paths, guessing extensions etc.) in scheme.
>>
> +1

Either one of yall want to give a shot at this or something like it?  I
think in an ideal world we could lose the ltdl dependency.  I gave a
first crack at it but got mired in C; I never considered implementing it
in Scheme, which seems like a nice idea.

Andy

bug#21076: dynamic-link often fails to load libraries

[Frank Webster]

On my Debian system, `dynamic-link' often fails to load shared
libraries:

Loading using a versioned soname never works:

scheme@(guile-user) (dynamic-link libGL.so.1)
ERROR: In procedure dynamic-link:
ERROR: In procedure dynamic-link: file: libGL.so.1, message: file not found

Loading without a library extension doesn't work unless the
corresponding *-dev package is installed:

scheme@(guile-user) (dynamic-link libGL)
ERROR: In procedure dynamic-link:
ERROR: In procedure dynamic-link: file: libGL, message: file not found

It is supposed to work with or without the extension, however because
of limitations in the underlying lt_dlopenext function in libtool,
neither works.

The first doesn't work because contrary to its
documentation, lt_dlopenext doesn't always try loading
the bare filename first, without appending an extention.

The second doesn't work because the versioned soname extension (.so.1)
isn't among the ones that lt_dlopenext tries to append. It does try the
unversioned extension (.so), but in Debian the unversioned symlink
is only available when the corresponding *-dev package is installed.

Here is a related libtool bug report: https://debbugs.gnu.org/8976

Note that there are arguably two distinct issues:
(1) lt_dlopenext not trying the bare filename in all cases
(2) lt_dlopenext not trying versioned soname extensions.

Obviously one way to address this would be to fix these two issues in
lt_dlopenext in libtool.

Alternatively, the (system foreign) module in guile could provide a
simple low-level wrapper around lt_dlopen, and possibly implement
the higher level functionality of `dynamic-link' (additional search
paths, guessing extensions etc.) in scheme.