Re: [fpc-pascal] linux: should we hard-code versioned or unversioned shared libraries in our apps?

2012-08-15 Thread Jorge Aldo G. de F. Junior
But creating correct symlinks IS one of the tasks of a system
administrator... His up to know the distro he is using and the
software he wants to install.

I bet a in the field sysadmin will be much more flexible than a
hardcoded implementation that would need to take zillions of obscure
distros in consideration from the limited perspective of compile
time...

2012/8/15 Graeme Geldenhuys graemeg.li...@gmail.com:
 Hi,

 This issue has come up before in a fcl-db discussion regarding the
 Firebird Database, where the libfbclient.so was missing from many
 Linux distros (eg: Ubuntu 10.04), but a libfbclient.so.2.0 was
 available instead. FCL-DB only checks for the unversioned shared
 libraries, thus your application will probably not run out of the box.

 I've stumbled across this problem again today with OpenSUSE 12.1 and
 Synapse where I send emails from inside my applications via a secure
 SMTP connection. The Synapse library is looking for libssl.so, but by
 default OpenSUSE only has libssl.so.1.0.0.

 In both these cases, I manually created unversioned symlinks to those
 libraries, and that got my applications working again. This is not
 ideal, but I don't know how else to handle this.

 Does any body know what is the most correct way of handling this? Am
 I supposed to modify my copies of fcl-db and synapse to look for
 specific versions of these shared libraries, or should I somehow add
 a function in my application installation that checks in unversioned
 shared libraries exist, and if not, try and create those (which would
 require root access - and might cause problems an client installs).

 Anybody know of any Linux documentation URL that explain when
 unversioned shared libraries are used and when not, and how
 applications are supposed to handle this?

 --
 Regards,
   - Graeme -


 ___
 fpGUI - a cross-platform Free Pascal GUI toolkit
 http://fpgui.sourceforge.net
 ___
 fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
 http://lists.freepascal.org/mailman/listinfo/fpc-pascal
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] linux: should we hard-code versioned or unversioned shared libraries in our apps?

2012-08-15 Thread Jonas Maebe

On 15 Aug 2012, at 12:43, Graeme Geldenhuys wrote:

 This issue has come up before in a fcl-db discussion regarding the
 Firebird Database, where the libfbclient.so was missing from many
 Linux distros (eg: Ubuntu 10.04), but a libfbclient.so.2.0 was
 available instead. FCL-DB only checks for the unversioned shared
 libraries, thus your application will probably not run out of the box.
 
 In both these cases, I manually created unversioned symlinks to those
 libraries, and that got my applications working again. This is not
 ideal, but I don't know how else to handle this.

The official way to get the unversioned symbolic links is to install the -dev 
or -devel package for that library. Of course, you're not supposed to require 
end-users to do that.

 Does any body know what is the most correct way of handling this?

Determine which major version of the library you require (e.g. 1.x, or 2.x, or 
even something like 5.1.x if the library in question only supports API 
compatibility across minor releases), and explicitly load that one. That is 
also how it works when linking against a library at compile time: the 
unversioned name is simply a symbolic link to a versioned file name, and it's 
that versioned file name that the linker will hardcode in your application. The 
reasoning is that your application was written and tested against that version 
of the library, so letting it use arbitrary other versions is not a good idea.

If you can work with multiple major versions, you can try to dynamically load 
them all until you have found one that exists. At least most (if not all) FCL 
units that dynamically load libraries have a function that you can call to 
explicitly specify the name of the dynamic library to use. Since the units 
themselves have presumably been written to work with certain versions of those 
libraries, it would however probably be best if that code were directly added 
to those units (and to remove any *default* dynamic loading of unversioned file 
names, since such libraries usually only exist on a developer's machine -- 
except maybe on non-Unix platforms).


Jonas___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] linux: should we hard-code versioned or unversioned shared libraries in our apps?

2012-08-15 Thread Mark Morgan Lloyd

Graeme Geldenhuys wrote:

Hi,

This issue has come up before in a fcl-db discussion regarding the
Firebird Database, where the libfbclient.so was missing from many
Linux distros (eg: Ubuntu 10.04), but a libfbclient.so.2.0 was
available instead. FCL-DB only checks for the unversioned shared
libraries, thus your application will probably not run out of the box.

I've stumbled across this problem again today with OpenSUSE 12.1 and
Synapse where I send emails from inside my applications via a secure
SMTP connection. The Synapse library is looking for libssl.so, but by
default OpenSUSE only has libssl.so.1.0.0.

In both these cases, I manually created unversioned symlinks to those
libraries, and that got my applications working again. This is not
ideal, but I don't know how else to handle this.


Quite a long way from ideal, since it implies that an administrator has 
to be involved even if the program is only sitting in an unprivileged 
user's home directory (or is a symlink in ~ good enough?).


I'm tempted to say that this is a distro issue, and that if an upstream 
project (FireBird, OpenSSL, PostgreSQL) normally publishes an 
unversioned library that a distro is at fault if it decorates it with 
an appended version number without providing a symlink chain.


I'd suggest that the best solution would be for FPC policy to mandate 
that all shared library (.so etc.) usage in the RTL/FCL/LCL should be 
capable of taking a library name parameter, and for application code to 
allow this name to be specified in a .ini file. I think this would be 
safer than having the app try to deduce the location of the appropriate 
file, since sooner or later there will be a case where it sees multiple 
versions and selects the wrong one, or assumes that e.g. /usr/local/opt 
is permanently available when in fact it's a short-term NFS mount.


And doing it that way means that the user would be aware of the problem, 
and if enough users are aware of the problem it might trickle through to 
the awareness of the distreaux maintainers.


--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] linux: should we hard-code versioned or unversioned shared libraries in our apps?

2012-08-15 Thread Graeme Geldenhuys
Hi,

On 15 August 2012 12:10, Jonas Maebe jonas.ma...@elis.ugent.be wrote:

 The official way to get the unversioned symbolic links is to install the -dev 
 or
 -devel package for that library. Of course, you're not supposed to require
 end-users to do that.

Yes, I know that bit, but even as a developer, if I don't do actual
Firebird or OpenSSL development (I don't work on those project, I
simply use there libraries), I don't need to install those libraries.
That is why my Ubuntu and OpenSUSE system didn't have the -devel
packages for them installed. FCL-db and Synapse dynamically load those
libraries on my development machines, so no -devel package
requirement.


 ... that the linker will hardcode in your application. The reasoning is that 
 your
  application was written and tested against that version of the library, so 
 letting
  it use arbitrary other versions is not a good idea.

In a way I understand the reasoning for the unversioned symlink, I
just don't know how best to handle it in my apps. I also don't know
the rules as to which version the unversioned symlib will normally
point to.I know my apps work with Firebird 2.1 and Firebird 2.5, so
should I then change fcl-db to look for libfbclient.so.2 or
libfbclient.so.2.0 - instead of the unversioned default. Because the
unversioned default could very easily point to a v1.x library of
Firebird, and that would be untested with my app (it will probably not
work).


 If you can work with multiple major versions, you can try to dynamically
 load them all until you have found one that exists.

OK, I'll take a look at this idea. That seems better, because then I
know I am trying to load library versions my app is known to work
with.

I'm not so much concerned about my development system (I can fix that)
- I am thinking more in line of when I deploy my apps. I want them to
obviously work as easily as possible.


-- 
Regards,
  - Graeme -


___
fpGUI - a cross-platform Free Pascal GUI toolkit
http://fpgui.sourceforge.net
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] linux: should we hard-code versioned or unversioned shared libraries in our apps?

2012-08-15 Thread Mark Morgan Lloyd

Jonas Maebe wrote:


In both these cases, I manually created unversioned symlinks to those
libraries, and that got my applications working again. This is not
ideal, but I don't know how else to handle this.


The official way to get the unversioned symbolic links is to install the -dev 
or -devel package for that library. Of course, you're not supposed to require 
end-users to do that.


Does not always work, e.g. in the case of Debian. -dev will install the 
.h and .a files, but won't necessarily set up unversioned symlinks.


Anybody: what's the position with Ubuntu, as the leading Debian derivative?

--
Mark Morgan Lloyd
markMLl .AT. telemetry.co .DOT. uk

[Opinions above are the author's, not those of his employers or colleagues]
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] linux: should we hard-code versioned or unversioned shared libraries in our apps?

2012-08-15 Thread Graeme Geldenhuys
Hi,

On 15 August 2012 12:27, Mark Morgan Lloyd
markmll.fpc-pas...@telemetry.co.uk wrote:
 Quite a long way from ideal, since it implies that an administrator has to
 be involved even if the program is only sitting in an unprivileged user's
 home directory (or is a symlink in ~ good enough?).

Correct, definitely not ideal. I was just thinking about the $HOME
directory. I'll try and play with that now, and see what happens to my
apps. Maybe my apps should all have startup scripts that define local
LD_LIBRARY_PATH options?


 I'm tempted to say that this is a distro issue, and that if an upstream
 project (FireBird, OpenSSL, PostgreSQL) normally publishes an unversioned
 library that a distro is at fault if it decorates it with an appended
 version number without providing a symlink chain.

They explicitly mention that general runtime libraries should not
include the unversioned symlink. Only the -devel packages should
include those. That makes me think, why do we then need the
unversioned symlink in the first place!

Fedora:
   http://fedoraproject.org/wiki/Packaging:Guidelines#Devel_Packages

OpenSUSE:
   
http://en.opensuse.org/openSUSE:Shared_library_packaging_policy#Unversioned_packages

Ubuntu/Debian:
   ?? I can't remember the URL now.


-- 
Regards,
  - Graeme -


___
fpGUI - a cross-platform Free Pascal GUI toolkit
http://fpgui.sourceforge.net
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] linux: should we hard-code versioned or unversioned shared libraries in our apps?

2012-08-15 Thread Henry Vermaak
On 15/08/12 12:37, Mark Morgan Lloyd wrote:
 Jonas Maebe wrote:
 
 In both these cases, I manually created unversioned symlinks to those
 libraries, and that got my applications working again. This is not
 ideal, but I don't know how else to handle this.

 The official way to get the unversioned symbolic links is to install
 the -dev or -devel package for that library. Of course, you're not
 supposed to require end-users to do that.
 
 Does not always work, e.g. in the case of Debian. -dev will install the
 .h and .a files, but won't necessarily set up unversioned symlinks.

I've never found a -dev package that doesn't set up symlinks in many
years of using Debian for development.  (Except if only static versions
of a specific library exists, obviously).  Could you give an example?

Henry
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] linux: should we hard-code versioned or unversioned shared libraries in our apps?

2012-08-15 Thread Jonas Maebe

On 15 Aug 2012, at 13:32, Graeme Geldenhuys wrote:

 On 15 August 2012 12:10, Jonas Maebe jonas.ma...@elis.ugent.be wrote:
 
 The official way to get the unversioned symbolic links is to install the 
 -dev or
 -devel package for that library. Of course, you're not supposed to require
 end-users to do that.
 
 Yes, I know that bit, but even as a developer, if I don't do actual
 Firebird or OpenSSL development (I don't work on those project, I
 simply use there libraries), I don't need to install those libraries.

The dev/devel packages are not for people working on those libraries, they are 
for building programs that use those libraries. It's developers that do work on 
those packages themselves that don't necessarily need those packages, since 
they can just use their own source code.

 That is why my Ubuntu and OpenSUSE system didn't have the -devel
 packages for them installed. FCL-db and Synapse dynamically load those
 libraries on my development machines, so no -devel package
 requirement.

You said that you manually created the symbolic link. I simply explained that 
you should never do that, and instead install the development packages because 
they will do that (correctly) for you. Whether or not it is desirable for the 
FCL/Synapse units to require an unversioned symlink to be present is a separate 
issue.

 ... that the linker will hardcode in your application. The reasoning is that 
 your
 application was written and tested against that version of the library, so 
 letting
 it use arbitrary other versions is not a good idea.
 
 In a way I understand the reasoning for the unversioned symlink, I
 just don't know how best to handle it in my apps. I also don't know
 the rules as to which version the unversioned symlib will normally
 point to.

It will point to the library of the version corresponding to the dev/devel 
package that has been installed.

 I know my apps work with Firebird 2.1 and Firebird 2.5, so
 should I then change fcl-db to look for libfbclient.so.2 or
 libfbclient.so.2.0 - instead of the unversioned default.

libfbclient.so.2.0 means Firebird 2.0.x. libfbclient.so.2 means Firebird 2.x. 
If you only support 2.1 and 2.5, you should link against libfbclient.so.2 and 
then call a function to get the actual version. If the library does not contain 
such a function, then there is simply no way to dynamically load that library 
and use it in a safe way in your scenario.


Jonas___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] linux: should we hard-code versioned or unversioned shared libraries in our apps?

2012-08-15 Thread Graeme Geldenhuys
Hi,

On 15 August 2012 12:49, Jonas Maebe jonas.ma...@elis.ugent.be wrote:
 You said that you manually created the symbolic link. I simply explained that 
 you should
 never do that, and instead install the development packages because they will 
 do that
 (correctly) for you.

So my applications that I deploy to clients will now require -devel
packages? This just doesn't sound sane, sorry. Maybe there is just a
fundamental flaw in all distros and how they package software.
Initially a great idea (in theory), but in reality it is seriously
flawed.

 Whether or not it is desirable for the FCL/Synapse units to require an 
 unversioned
 symlink to be present is a separate issue.

FCL-DB uses dynamic linking by default, and looks for the unversioned
shared library. So what specific Firebird version is the FCL-DB coded
too? Because remember, libfbclient.so can point to a v1.x or a v2.x -
depending on what -devel package was installed.

This uncertain (version mismatch) is what I would like to avoid, plus
the fact that my clients now need to install -devel packages too.

Maybe I should simply install my apps into /home/user/appname. My
installer would then ship the required/expected *.so files, and setup
the unversioned symlink. My app would then be launched via a script
which sets the LD_LIBRARY_PATH to the application directory. My app
would then be totally self-contained. This seems like the only way to
get around these flawed package management rules and dependencies.



 It will point to the library of the version corresponding to the dev/devel 
 package that has been installed.

So one minute it could point to a v1.x library, and the next it could
point to a v2.x library. This will surely break many apps that only
look at the unversioned symlink.


 libfbclient.so.2.0 means Firebird 2.0.x. libfbclient.so.2 means Firebird 2.x.

OK, thanks for that.

My apps are tiOPF based, and tiOPF uses the bare minimum functionality
from each backend database server. Basically just the CRUD (create,
read, update and delete) functions. So even though I say our apps use
Firebird DB v2.x, it will most probably run on a v1.x system too, but
like I said, that would be untested. We only test with v2.x Firebird
servers.

-- 
Regards,
  - Graeme -


___
fpGUI - a cross-platform Free Pascal GUI toolkit
http://fpgui.sourceforge.net
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] linux: should we hard-code versioned or unversioned shared libraries in our apps?

2012-08-15 Thread Henry Vermaak
On 15/08/12 13:43, Mark Morgan Lloyd wrote:
 
 I do agree with Graeme's position though: a -dev is described as
 containing files for developers, and it should not be necessary for a
 non-developer user to start encumbering his system with .h files etc.
 What's more, part of the reason that I've had to create symlinks
 manually in the past is because things like (if I recall correctly)
 OpenOffice's interface to PostgreSQL require libpq.so without declaring
 -dev as a prerequisite... which rather reinforces my position that this
 is a distro issue even if they prefer to disclaim it.

I'd say that it's the OpenOffice interface that's broken.  They
shouldn't link to a library that doesn't specify a version.  They need
to link to a specific version by using the naming conventions as Jonas
explained.  This is to protect yourself from api/feature breakage.
Linking directly to, e.g. libpq.so, is only o.k. if you can check the
version by means of some api, which the library needs to supply in all
the versions that exist - past, present and future.  Which is probably
what they're doing.  This isn't very good practice, in my opinion, but
can work.

If they really support multiple versions of libpq, then they will have
to try to dynamically link against all the versions they support.  This
won't be much more effort than checking the version in the first place.

Even on Windows some libraries are using names that reflect the version,
e.g. I link to some opencv libs, which are named
libopencv_{module}231.dll.  I suspect this is much less of a problem in
the Windows world, since library reuse isn't that common.

Henry
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] linux: should we hard-code versioned or unversioned shared libraries in our apps?

2012-08-15 Thread Jonas Maebe

On 15 Aug 2012, at 15:14, Graeme Geldenhuys wrote:

 On 15 August 2012 12:49, Jonas Maebe jonas.ma...@elis.ugent.be wrote:
 You said that you manually created the symbolic link. I simply explained 
 that you should
 never do that, and instead install the development packages because they 
 will do that
 (correctly) for you.
 
 So my applications that I deploy to clients will now require -devel
 packages?

They always have required them until now (as have any other applications that 
use fcl-db).

 This just doesn't sound sane, sorry. Maybe there is just a
 fundamental flaw in all distros and how they package software.
 Initially a great idea (in theory), but in reality it is seriously
 flawed.

It's the fcl-db/Synapse units that are flawed. As explained in my first reply, 
the unversioned symlink is only for use by the linker (ld), and they work 
perfectly well for that purpose.

 Whether or not it is desirable for the FCL/Synapse units to require an 
 unversioned
 symlink to be present is a separate issue.
 
 FCL-DB uses dynamic linking by default, and looks for the unversioned
 shared library. So what specific Firebird version is the FCL-DB coded
 too?

I have no idea, which is why I also said in my first reply:

***
At least most (if not all) fcl units that dynamically load libraries have a 
function that you can call to explicitly specify the name of the dynamic 
library to use. Since the units themselves have presumably been written to work 
with certain versions of those libraries, it would however probably be best if 
that code were directly added to those units (and to remove any *default* 
dynamic loading of unversioned file names, since such libraries usually only 
exist on a developer's machine -- except maybe on non-Unix platforms).
***

 Maybe I should simply install my apps into /home/user/appname. My
 installer would then ship the required/expected *.so files, and setup
 the unversioned symlink. My app would then be launched via a script
 which sets the LD_LIBRARY_PATH to the application directory. My app
 would then be totally self-contained. This seems like the only way to
 get around these flawed package management rules and dependencies.

While shipping all libraries yourself is an option, the package management 
rules and dependencies are not flawed (or at least the scenario you are 
describing currently does not point to a flaw in them). If you start shipping 
libraries yourself, you'd better make sure that you also ship all libraries 
they in turn depend on as well, because otherwise you're more likely to make 
the problem worse than solve it.

 It will point to the library of the version corresponding to the dev/devel 
 package that has been installed.
 
 So one minute it could point to a v1.x library, and the next it could
 point to a v2.x library. This will surely break many apps that only
 look at the unversioned symlink.

As mentioned in my previous replies, applications should never look at the 
unversioned link in the first place when dynamically loading a library.


Jonas___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] linux: should we hard-code versioned or unversioned shared libraries in our apps?

2012-08-15 Thread Marco van de Voort
In our previous episode, Graeme Geldenhuys said:
 In both these cases, I manually created unversioned symlinks to those
 libraries, and that got my applications working again. This is not
 ideal, but I don't know how else to handle this.

 Does any body know what is the most correct way of handling this?

(1) avoid _dyn versions. 
(2) override the built-in naming by calling
ibase60dyn.initializeibase60('whatever.so');
 
before connection or other db parts initializes.
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] linux: should we hard-code versioned or unversioned shared libraries in our apps?

2012-08-15 Thread Marco van de Voort
In our previous episode, Graeme Geldenhuys said:
  You said that you manually created the symbolic link. I simply explained 
  that you should
  never do that, and instead install the development packages because they 
  will do that
  (correctly) for you.
 
 So my applications that I deploy to clients will now require -devel
 packages? This just doesn't sound sane, sorry. Maybe there is just a
 fundamental flaw in all distros and how they package software.

No. It all works fine WITHIN a distribution (in a particular version). And
basically that is the only supported way on Linux.

Anything crossdistribution and cross-distroversion is assumed to be on
source level, not binary level (IOW rebuild your app). According to common
linux philosophy distributing crossdistro binaries is asking for pain.

Basically if you link to a .so the normal way, to simplify build systems, a
symlink from a generic name to a versioned name is used.  But the generic
binary contains a reference to the versioned name, and thus will work
without symlink, which is why end-users systems don't have the link.

But this way of linking is more dependent on the exact version of the
library, so in the past some people changed everything to dynamically
loaded via dlload/dlsym.

This alleviates the versioning problem somewhat, but discards the
only distribution provided way to fix this.

I still think that changing everything to _dyn created more problems than
it solved. (except for special disaster case mysql maybe)

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal


Re: [fpc-pascal] linux: should we hard-code versioned or unversioned shared libraries in our apps?

2012-08-15 Thread Graeme Geldenhuys
Hi,

On 15 August 2012 14:39, Jonas Maebe jonas.ma...@elis.ugent.be wrote:
 FCL-DB uses dynamic linking by default, and looks for the unversioned
 shared library. So what specific Firebird version is the FCL-DB coded
 too?

 I have no idea, which is why I also said in my first reply:

Umm, so ideally the ibconnection.pp unit should really be split into
various units with version numbers in their names. That way we will
know to which Firebird version they are referring too.

eg:   ibconnection.pp   // old interbase only (eg: Delphi 7)
firebird-1.x.pp
firebird-2.1.pp
firebird-2.5.pp

Include files could probably be used to reduce some duplication of code.


 As mentioned in my previous replies, applications should never look at the
 unversioned link in the first place when dynamically loading a library.

OK, for now I'll modify my fcl-db and synapse code to look for the
versioned shared libraries that I know I have tested with. eg:
libfbclient.so.2


Thanks for all the help guys.


-- 
Regards,
  - Graeme -


___
fpGUI - a cross-platform Free Pascal GUI toolkit
http://fpgui.sourceforge.net
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-pascal