Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
On Mon, Dec 12, 2011 at 10:32:26AM +0100, Lionel Elie Mamane wrote: On Mon, Dec 12, 2011 at 09:46:37AM +0100, Fridrich Strba wrote: On 12/12/11 08:14, Lionel Elie Mamane wrote: This also means that by our we have to build everything we ship rule, we have to internalise, according to my first quick survey: - MIT Kerberos: - Kerberos for Windows: that's not part of the OS - internalise (http://web.mit.edu/kerberos/dist/index.html#kfw-3.2) Do we really really need this on Windows? It depends what value we give to really. So, I asked on the PostgreSQL devs mailing list. Briefly the answer is that MIT Kerberos on Windows is bitrotten, unmaintained and a pain in the backside: open security bugs will likely not be fixed soon, etc. Other software in the PostgreSQL ecosystem have stopped linking against it. As to what feature is lost: - On Windows XP, any kind of decent crypto in the context of Kerberos authentication. - On Windows 7, the system Kerberos is actually quite OK. So, all in all, we better *not* link against MIT Kerberos on Microsoft Windows. That is our current situation, no change needed, just for your information ( vindication). -- Lionel ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
On 12/12/11 08:14, Lionel Elie Mamane wrote: I did that because it seemed the easiest thing to do at first sight. However, we are losing the advantage of that, and thus I propose we switch to dynamic linking, and bundling the lib on platforms / situations that need it. Where are we actually losing the advantage? The way I quickly hacked the internal postgresql for windows was a proof of concept and surely we can extend it. - no SSL/TLS support, for encrypted communication with the DB server and authentication by X.509 certificate. We bundle OpenSSL with LibreOffice, so should be possible to build it against the internal or system OpenSSL according how the build is configured. - no Kerberos support: authentication; apparently can be used for windows domain single signon from a non-windows machine. So, this should not be a problem for windows then. Should it? - no GSSAPI support: yet another authentication mechanism - no LDAP support: yes, in the client library: this allows to store the connection info in LDAP and thus manage it centrally. Can be really convenient in enterprise environment. I started hacking our internal postgresql to activate all this, but this introduces dependencies on yet other libraries: OpenSSL, MIT Kerberos, com_err, ldap, ... This means that linking statically against libpq is not the easiest anymore, since static linking does not automatically recurse, in the way that dynamic linking does :) Yes, but this is feasible, since the stuff will be linked only in connectivity/source/drivers/postgresql, so having the dll linking in postgresql with all those libraries, or having the uno component linking against static libpq and all those libraries in connectivity is the same work. So, to me it looks like we'll switch to dynamic linking against libpq and shipping the DLL/SO/DYLIB on the platforms/cases that need it. Any platform where this will be harder than what I presume? It will not solve anything unless you want to use a shared library from somewhere else. You will still have to build the shared library with all the support and link it with the external libraries. Moreover, it might actually introduce a problem when your internal shared library of a certain name could be a bit different then the system one of the same name. Been there, done that. Not really a good solution. As an added bonus, this makes it easier for our users to upgrade the libraries we link against (for bugfixes or new features in new versions or ...): just overwrite the DLL/SO/DYLIB file with an upstream-provided one (downloaded from http://www.postgresql.org). No, no, no! Then you will have a mix of environment that you don't know compiled with options that you cannot be sure about and you will have an unmaintainable code. This also means that by our we have to build everything we ship rule, we have to internalise, according to my first quick survey: - OpenSSL: I expect all platforms. It is a system lib on most GNU/Linux distributions, but the soname is not stable. We ship it already, so no problem there. - MIT Kerberos: * MacOS X bundles Kerberos since 10.2, which AFAIK is behind our baseline of 10.4; is it binary-compatible across releases? Then we can consider it a system library on MacOS X and not internalise / ship it. * GNU/Linux: What's our baseline there? Has the soname changed since our baseline? Is the soname consistent across distributions? If noyes, then system lib and don't ship it. Debian seems to have a stable soname since woody (released 19 July 2002), so I expect this will be OK. If we have to internalise it, it depends on: ° libcom_err: built from e2fsprogs sources on my Debian, soname libcom_err.so.2. Google tells me (some?) RPM distributions have a libcom_err.so.3 that is shipped by the MIT Kerberos RPM itself. It should be possible to assume as a system library for Linux and MacOSX - Kerberos for Windows: that's not part of the OS - internalise (http://web.mit.edu/kerberos/dist/index.html#kfw-3.2) Do we really really need this on Windows? - LDAP: We already have OpenLDAP-related stuff in configure.in... So can postgresql-sdbc just use that, or is the existing stuff too optional? I mean it seems to be replacing some Mozilla parts, not sure we enable it on our builds or if it just a service to distributions. If cannot just use: * MacOS X: is it bundled? Some googling finds LDAP-related commands in MacOS X 10.4, our baseline. So we are good to consider it a system lib? * GNU/Linux: Debian has stable soname since 2008. Assuming other distros are similar, is that sufficient to consider it a system library? So, in LibreOffice, we have two ways of use LDAP: Either we use the ldap
Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
On Mon, Dec 12, 2011 at 09:46:37AM +0100, Fridrich Strba wrote: On 12/12/11 08:14, Lionel Elie Mamane wrote: I did that because it seemed the easiest thing to do at first sight. However, we are losing the advantage of that, and thus I propose we switch to dynamic linking, and bundling the lib on platforms / situations that need it. Where are we actually losing the advantage? Well, the advantage of linking statically is that we don't have to bundle a shared library and muck with rpathfriends to have it found at runtime. Is there another one I'm not aware of? If we have to link dynamically against other libraries that we have to bundle in the extension, then we have to do the rpath-thing anyway for these other libs, and for N0, doing it for N+1 libs is not more difficult than doing it for N libs. But indeed if we end up not shipping any extra shared library in the extension (that is N=0), then indeed static linking is easier for us. - no SSL/TLS support, for encrypted communication with the DB server and authentication by X.509 certificate. We bundle OpenSSL with LibreOffice, We do? Oh, great, I had missed that. As an added bonus, this makes it easier for our users to upgrade the libraries we link against (for bugfixes or new features in new versions or ...): just overwrite the DLL/SO/DYLIB file with an upstream-provided one (downloaded from http://www.postgresql.org). No, no, no! Then you will have a mix of environment that you don't know compiled with options that you cannot be sure about and you will have an unmaintainable code. We will have (and ship) the code we compiled, with the options we are sure about; if users overwrite our binaries with others, well, they have what they put there. We leave them freedom to tinker more easily (no need to recompile, just overwrite files), and yes, sometimes by tinkering one breaks stuff. That's the nature of it. This also means that by our we have to build everything we ship rule, we have to internalise, according to my first quick survey: - MIT Kerberos: It should be possible to assume as a system library for Linux and MacOSX OK, good. - Kerberos for Windows: that's not part of the OS - internalise (http://web.mit.edu/kerberos/dist/index.html#kfw-3.2) Do we really really need this on Windows? It depends what value we give to really. My guess is that it is desirable. Without it our internal libpq has one less feature. I don't have a good idea of how significant that feature is, since Windows has some Kerberos implementation, too; libpq (if compiled with support for both) allows to choose which one to use at runtime. What are the compared compatibility issues and/or features of both, I don't know. For example, can the Windows Kerberos client authenticate against all Unix Kerberos setups that MIT-Kerberos/GSSAPI can authenticate against? I guess that if the PostgreSQL project went the extra mile of allowing simultaneous support for both, selectable at run-time and all that, this means at least some users benefit from it. - LDAP: We already have OpenLDAP-related stuff in configure.in... So can postgresql-sdbc just use that, or is the existing stuff too optional? I mean it seems to be replacing some Mozilla parts, not sure we enable it on our builds or if it just a service to distributions. If cannot just use: * MacOS X: is it bundled? Some googling finds LDAP-related commands in MacOS X 10.4, our baseline. So we are good to consider it a system lib? * GNU/Linux: Debian has stable soname since 2008. Assuming other distros are similar, is that sufficient to consider it a system library? So, in LibreOffice, we have two ways of use LDAP: Either we use the ldap functionality from the bundled mozilla, or we use OpenLDAP. Yes, but libpq as is AFAIK cannot use Mozilla-LDAP. Maybe it can be hacked to do that. Are OpenLDAP and Mozilla-LDAP API-compatible, so that we can just use Mozilla-LDAP headers and libraries where libpq assumes OpenLDAP, and not change anything the in the libpq sources? Then I guess it would be easy enough indeed. -- Lionel ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
Hello, On 12/12/11 10:32, Lionel Elie Mamane wrote: We bundle OpenSSL with LibreOffice, We do? Oh, great, I had missed that. OK, so what I did now is following: 1) We build the static library with ssl and ldap support on all platforms currently. 2) I patched the postgresql configury so that it has an option --with-mozldap. This one is used when we use internally in LO mozilla for ldap connections. Without that option, the stuff falls on openldap. Not sure it works, but it builds. It should be possible to assume as a system library for Linux and MacOSX OK, good. I did completely leave the kerberos outside. Anybody that wants to make it work on MacOSX will not have too difficult work I guess. I don't dare to do anything about it on Linux, since I don't know how it will affect our baseline. Nevertheless, let us assume that typical LO user will be ok with ldap and ssl. It depends what value we give to really. My guess is that it is desirable. Without it our internal libpq has one less feature. I don't have a good idea of how significant that feature is, since Windows has some Kerberos implementation, too; libpq (if compiled with support for both) allows to choose which one to use at runtime. What are the compared compatibility issues and/or features of both, I don't know. For example, can the Windows Kerberos client authenticate against all Unix Kerberos setups that MIT-Kerberos/GSSAPI can authenticate against? I guess that if the PostgreSQL project went the extra mile of allowing simultaneous support for both, selectable at run-time and all that, this means at least some users benefit from it. OK, in my humble opinion, let Windows users select this kind of thing on runtime is a sure path to hell. But as it stands, the libpq that we build uses some of the win32 security apis for authentication, so I would assume that the users should be able to do what they need from their windows posts. Those productive things like mailmerge with pgsql database of all employees and so on :) Yes, but libpq as is AFAIK cannot use Mozilla-LDAP. Maybe it can be hacked to do that. Are OpenLDAP and Mozilla-LDAP API-compatible, so that we can just use Mozilla-LDAP headers and libraries where libpq assumes OpenLDAP, and not change anything the in the libpq sources? Then I guess it would be easy enough indeed. Yup, it was hacked to do that now :) Cheers F. ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
Le 12/12/2011 09:57, Lionel Elie Mamane a écrit : Hi all, Which suggests --without-system-postgresql is broken, need to check that out. Works, at least on Debian GNU/Linux. Does it mean that the --without-system-postgresql is working now? On my machine (Debian GNU/Linux amd64), it build and runs (can connect to database and retrieve data). I checked manually (with ldd) that the postgresql-sdbc-impl.uno.so has no dependency on libpq (i.e. was statically linked). Fails miserably on Mac OSX though, looking for libpq-fe.h, and other includes. Alex ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
Le 12/12/2011 13:34, Fridrich Strba a écrit : Hi Fridrich, OK, so what I did now is following: 1) We build the static library with ssl and ldap support on all platforms currently. 2) I patched the postgresql configury so that it has an option --with-mozldap. This one is used when we use internally in LO mozilla for ldap connections. Without that option, the stuff falls on openldap. Not sure it works, but it builds. This fails on Mac OSX (10.6.8 SnowLeopard server, which has builtin LDAP implementation, not sure which though) trying to link -lldap (which it can't find). If moz is needed then that's a problem on Mac, because I have my switches set to --disable-mozilla and --disable-mozilla-build otherwise the whole moz stuff doesn't build on Mac OSX (at least not on my machine and hasn't for a while). So, with the LDAP commit, I can no longer build using --with-system-postgresql either. Alex ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
Any chance you can come to irc so that we sort it out? Cheers Fridrich On 12/12/11 13:56, Alex Thurgood wrote: Le 12/12/2011 13:34, Fridrich Strba a écrit : Hi Fridrich, OK, so what I did now is following: 1) We build the static library with ssl and ldap support on all platforms currently. 2) I patched the postgresql configury so that it has an option --with-mozldap. This one is used when we use internally in LO mozilla for ldap connections. Without that option, the stuff falls on openldap. Not sure it works, but it builds. This fails on Mac OSX (10.6.8 SnowLeopard server, which has builtin LDAP implementation, not sure which though) trying to link -lldap (which it can't find). If moz is needed then that's a problem on Mac, because I have my switches set to --disable-mozilla and --disable-mozilla-build otherwise the whole moz stuff doesn't build on Mac OSX (at least not on my machine and hasn't for a while). So, with the LDAP commit, I can no longer build using --with-system-postgresql either. Alex ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
On Mon, Dec 12, 2011 at 01:34:11PM +0100, Fridrich Strba wrote: On 12/12/11 10:32, Lionel Elie Mamane wrote: We bundle OpenSSL with LibreOffice, We do? Oh, great, I had missed that. OK, so what I did now is following: 1) We build the static library with ssl and ldap support on all platforms currently. It fails on GNU/Linux if system openssl-dev is not installed. Probably just a missing -I$(SOLARINC)/external somewhere. -- Lionel ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
On Mon, Dec 12, 2011 at 01:49:21PM +0100, Alex Thurgood wrote: Le 12/12/2011 09:57, Lionel Elie Mamane a écrit : Does it mean that the --without-system-postgresql is working now? On my machine (Debian GNU/Linux amd64), it build and runs (can connect to database and retrieve data). I checked manually (with ldd) that the postgresql-sdbc-impl.uno.so has no dependency on libpq (i.e. was statically linked). Fails miserably on Mac OSX though, looking for libpq-fe.h, and other includes. Forgot to add SOLARINC to CFLAGS. Testing and committing soon. -- Lionel ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
On Mon, Dec 12, 2011 at 03:56:01PM +0100, Lionel Elie Mamane wrote: On Mon, Dec 12, 2011 at 01:49:21PM +0100, Alex Thurgood wrote: Le 12/12/2011 09:57, Lionel Elie Mamane a écrit : Does it mean that the --without-system-postgresql is working now? On my machine (Debian GNU/Linux amd64), it build and runs (can connect to database and retrieve data). I checked manually (with ldd) that the postgresql-sdbc-impl.uno.so has no dependency on libpq (i.e. was statically linked). Fails miserably on Mac OSX though, looking for libpq-fe.h, and other includes. Forgot to add SOLARINC to CFLAGS. Testing and committing soon. No, that was not the problem. So let's check other things: Did you build and deliver the postgresql module? cd postgresql build --all -P2 -- -P2 deliver Replace 2 by another number to adjust parallelism. If you did do that, please look in solver/unxmac*/inc/postgresql; the include files should be there. The library itself in solver/unxmac*/lib/ If still not, show us build log. -- Lionel ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
I have impression I fixed it with the CPPFLAGS in the configure_command. But maybe not. The best would be to check the latest latest latest checkout, since I was moving that territory quite a lot. Cheers F. On 12/12/11 15:56, Lionel Elie Mamane wrote: On Mon, Dec 12, 2011 at 01:49:21PM +0100, Alex Thurgood wrote: Le 12/12/2011 09:57, Lionel Elie Mamane a écrit : Does it mean that the --without-system-postgresql is working now? On my machine (Debian GNU/Linux amd64), it build and runs (can connect to database and retrieve data). I checked manually (with ldd) that the postgresql-sdbc-impl.uno.so has no dependency on libpq (i.e. was statically linked). Fails miserably on Mac OSX though, looking for libpq-fe.h, and other includes. Forgot to add SOLARINC to CFLAGS. Testing and committing soon. ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
On Mon, Dec 12, 2011 at 04:22:44PM +0100, Fridrich Strba wrote: I have impression I fixed it with the CPPFLAGS in the configure_command. Yes, I wrote after too quick a glance, indeed the right -I for libpq-fe.h was there. However, the right -I and -L for internal OpenSSL were missing, I added them in the same style as you put the one for libpq-fe.h. Is there any specific reason we are not simply using $(SOLARINC) and $(SOLARLIB) instead of reconstructing them partially from $(SOLARVER) and $(INPATH)? On 12/12/11 15:56, Lionel Elie Mamane wrote: On Mon, Dec 12, 2011 at 01:49:21PM +0100, Alex Thurgood wrote: Le 12/12/2011 09:57, Lionel Elie Mamane a écrit : Does it mean that the --without-system-postgresql is working now? On my machine (Debian GNU/Linux amd64), it build and runs (can connect to database and retrieve data). I checked manually (with ldd) that the postgresql-sdbc-impl.uno.so has no dependency on libpq (i.e. was statically linked). Fails miserably on Mac OSX though, looking for libpq-fe.h, and other includes. Forgot to add SOLARINC to CFLAGS. Testing and committing soon. ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
On 12/09/2011 11:19 PM, Lionel Elie Mamane wrote:
Well, as a system lib I understand it can be installed nearly anywhere:
- The one click installer puts it in /Library/PostgreSQL/${VERSION}/lib
- fink and MacPorts put it *by default* in
/sw/opt/postgresql/${VERSION}/bin
/opt/local/lib/postgresql${VERSION}/bin
But from documentation on the web, I gather that the user can
change the /sw/ and /opt/local prefix to anything.
So I wonder what we would put as absolute path. Are we supposed to
bundle it with LibreOffice, à la Microsoft Windows?
So it is a library that some LO library links against, but which is not
part of the Mac OS X system (i.e., is not guaranteed to be available on
every machine, in a fixed, well-known location)?
In that case, do not link any LO library directly against it at all.
Have some logic to programmatically find where the library is on a given
machine (if it is available at all), then access it via dlopen/dlysm.
Or, include a copy of the library in the LO installation set and link
against that one.
Stephan
___
LibreOffice mailing list
LibreOffice@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
On Sat, Dec 10, 2011 at 11:05:02AM +0100, Alexander Thurgood wrote:
Le 09/12/11 23:10, Thorsten Behrens a écrit :
And with attached patch, also a plain --enable-ext-postgresql-sdbc
builds - Norbert, want to double-check commit?
So I built yesterday using :
--with-ext-postgresql-sdbc
--with-system-postgresql
If I install this connector.oxt on another machine with MacOSX and a
running pg instance, when I try to connect I get the following error :
A driver is not registered for the URL
sdbc:posgresql:dbhost=localhost dbname=demo
I got a similar error on my Debian GNU/Linux, solved by:
cd /libo/build/tree
rm -r solver/INPATH/installation/opt/share/extensions/postgresql-sdbc/
solver/INPATH/inc/postgresql/ solver/INPATH/lib/libpq.*
connectivity/INPATH/lib/postgresql-sdbc*.{so,dylib,dll}
#build deliver in connectivity
#forcefully install newly generated .oxt file
cd ~/.config/libreoffice/3/user/extensions/*/extensions.db
Replace INPATH by its value on your platform (something like
unxmaci); ~/.config/libreoffice/ is LibreOffice's user profile
directory on Unix, I'm not sure what it is on MacOS X.
Basically, the --with-system-postgresql build was still linking
statically against the libpq.a in the solver, so essentially was
behaving as a --without-system-postgresql :-| Which suggests
--without-system-postgresql is broken, need to check that out.
Does your patch fix this problem of portability, i.e. from build
machine to user machine ?
I don't think so, we need to work on that. In the current situation,
*maybe* if the user machine also install libpq (postgresql C client
library) at the _same_ location than on the build machine, it might
work, I'm not even sure of that. Given the multiplicity of ways to
install libpq on MacOS X, we don't want to require that, but I'm not
sure how to fix that cleanly. I'm waiting for a LibO/MacOS X dev to
chip in on that.
I'm just trying to suss out the implications for me as the provider
of the db connector extensions.
For MacOS X, we are still in the getting it to work stage.. We need
your testing for that, thanks for sticking around.
--
Lionel
___
LibreOffice mailing list
LibreOffice@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
On Sun, Dec 11, 2011 at 03:09:41PM +0100, Lionel Elie Mamane wrote: Which suggests --without-system-postgresql is broken, need to check that out. Works, at least on Debian GNU/Linux. -- Lionel ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
On Sat, Dec 10, 2011 at 01:53:20AM +0200, Tor Lillqvist wrote: Stephan Bergmann sberg...@redhat.com wrote: libpq.5.dylib (compatibility version 5.0.0, current version 5.3.0) That way it won't be found at runtime. If it is a lib from the system, it should have an absolute path (like the ones below), if it is part of LO it should be referenced relatively via @loader_path (like the ones above). Well, as a system lib I understand it can be installed nearly anywhere Er... huh? Isn't the very definition of a system library one that is installed in a fixed known location, as part of the operating system? With that definition of a system library, libpq is neither a lib from the system, nor part of LO. -- Lionel ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
On 09/12/11 23:10, Thorsten Behrens wrote: And with attached patch, also a plain --enable-ext-postgresql-sdbc builds - Norbert, want to double-check commit? Sorry, I added that all-static-lib to the target unconditionally, since that is not harming anything. I did not read this e-mail. I would otherwise give you the credits for it. Cheers F. -- Please avoid sending me Word, Excel or PowerPoint attachments. See http://www.gnu.org/philosophy/no-word-attachments.html ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
On 11/12/11 15:17, Lionel Elie Mamane wrote: Which suggests --without-system-postgresql is broken, need to check that out. Works, at least on Debian GNU/Linux. Does it mean that the --without-system-postgresql is working now? If not, first thing to check (on other platforms then x86 would be whether the static library is built with PIC. Since it needs to end in a shared object, that might be necessary. Cheers F. -- Please avoid sending me Word, Excel or PowerPoint attachments. See http://www.gnu.org/philosophy/no-word-attachments.html ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
On Thu, Dec 08, 2011 at 06:45:30PM +0100, Lionel Elie Mamane wrote: On Thu, Dec 08, 2011 at 04:19:04PM +, Michael Meeks wrote: * postgresql fun. non-internal + Windows - run-time DLL dependency a pain (Lionel) + ship DLL ourselves ? This is not a question mark, we have to do that, or link statically, be it only because of Fridrich-mentioned problems. Note that currently postgresql-sdbc always links statically when using internal PostgreSQL. I did that because it seemed the easiest thing to do at first sight. However, we are losing the advantage of that, and thus I propose we switch to dynamic linking, and bundling the lib on platforms / situations that need it. I started actually testing the internal PostgreSQL C client library build in use, and I realised it is rather crippled: - no SSL/TLS support, for encrypted communication with the DB server and authentication by X.509 certificate. - no Kerberos support: authentication; apparently can be used for windows domain single signon from a non-windows machine. - no GSSAPI support: yet another authentication mechanism - no LDAP support: yes, in the client library: this allows to store the connection info in LDAP and thus manage it centrally. Can be really convenient in enterprise environment. Leaving only hardcoded-in-LibreOffice's .odb file connection string with user/password authentication. Of these features I'd expect SSL (encrypted connection) is the most widely used one. I started hacking our internal postgresql to activate all this, but this introduces dependencies on yet other libraries: OpenSSL, MIT Kerberos, com_err, ldap, ... This means that linking statically against libpq is not the easiest anymore, since static linking does not automatically recurse, in the way that dynamic linking does :) More deeply, we don't gain anything/much by linking statically against libpq, but dynamically against these other libs. But linking statically against some of these other libs is significantly harder than dynamic link: - At least on my Debian, libssl is not compiled with -fPIC, so a dynamic library (postgresql-sdbc-impl.so) cannot link statically against it. This could be solved by recompiling OpenSSL ourselves. - MIT kerberos does not support static linking; vaguely discussed how to allow it in 2006, but no progress since then: http://krbdev.mit.edu/rt/Ticket/Display.html?id=3838 That's not something we'll solve simply by recompiling MIT krb5... So, to me it looks like we'll switch to dynamic linking against libpq and shipping the DLL/SO/DYLIB on the platforms/cases that need it. Any platform where this will be harder than what I presume? As an added bonus, this makes it easier for our users to upgrade the libraries we link against (for bugfixes or new features in new versions or ...): just overwrite the DLL/SO/DYLIB file with an upstream-provided one (downloaded from http://www.postgresql.org). What I'm not totally clear about is how we will generate the list of libraries to bundle. I could imagine recursing through the DT_NEEDED fields of the .so files (and the equivalent of that on other platforms), and stopping when hitting a known system library (such as libc, libresolv, etc); I imagine this exact same problem must have been solved already elsewhere in LibreOffice and we have a dmake/make recipe we can apply / reuse? Or is it not so? This also means that by our we have to build everything we ship rule, we have to internalise, according to my first quick survey: - OpenSSL: I expect all platforms. It is a system lib on most GNU/Linux distributions, but the soname is not stable. - MIT Kerberos: * MacOS X bundles Kerberos since 10.2, which AFAIK is behind our baseline of 10.4; is it binary-compatible across releases? Then we can consider it a system library on MacOS X and not internalise / ship it. * GNU/Linux: What's our baseline there? Has the soname changed since our baseline? Is the soname consistent across distributions? If noyes, then system lib and don't ship it. Debian seems to have a stable soname since woody (released 19 July 2002), so I expect this will be OK. If we have to internalise it, it depends on: ° libcom_err: built from e2fsprogs sources on my Debian, soname libcom_err.so.2. Google tells me (some?) RPM distributions have a libcom_err.so.3 that is shipped by the MIT Kerberos RPM itself. - Kerberos for Windows: that's not part of the OS - internalise (http://web.mit.edu/kerberos/dist/index.html#kfw-3.2) - LDAP: We already have OpenLDAP-related stuff in configure.in... So can postgresql-sdbc just use that, or is the existing stuff too optional? I mean it seems to be replacing some Mozilla parts, not sure we enable it on our builds or if it just a service to distributions. If cannot just use:
Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
Le 09/12/11 23:10, Thorsten Behrens a écrit : Hi Thorsten, And with attached patch, also a plain --enable-ext-postgresql-sdbc builds - Norbert, want to double-check commit? So I built yesterday using : --with-ext-postgresql-sdbc --with-system-postgresql and I get a connector OXT that contains the following : ns1:~ alex$ ls -an /Users/Shared/postgresql-sdbc total 2184 drwx-- 9 501 20 306 10 déc 10:53 . drwxrwxrwt+ 17 00 578 10 déc 10:53 .. drwxr-xr-x 3 501 20 102 10 déc 10:53 META-INF drwxr-xr-x 3 501 20 102 10 déc 10:53 description -rw-r--r-- 1 501 20 791 9 déc 12:31 description.xml -rwxr-xr-x 1 501 20 1037244 9 déc 15:26 postgresql-sdbc-impl.uno.dylib -rwxr-xr-x 1 501 2059948 9 déc 12:31 postgresql-sdbc.uno.dylib -rw-r--r-- 1 501 20 29 9 déc 12:31 postgresql-sdbc.unorc -rw-r--r-- 1 501 20 4866 9 déc 12:31 postgresql.xcu At least we have the postgresql-sdbc-impl.uno.dylib there. If I install this connector.oxt on another machine with MacOSX and a running pg instance, when I try to connect I get the following error : A driver is not registered for the URL sdbc:posgresql:dbhost=localhost dbname=demo Which means that my connector can not be put on the Extensions site for all to use. Does your patch fix this problem of portability, i.e. from build machine to user machine ? I'm sorry for being dim, but I don't understand the implications of static/dynamic linking at build time, so I'm just trying to suss out the implications for me as the provider of the db connector extensions. Alex ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
On 10/12/11 11:05, Alexander Thurgood wrote: So I built yesterday using : --with-ext-postgresql-sdbc --with-system-postgresql Would you mind to try --without-system-postgresql (and fix eventual breakage). That might actually help making a nice extension that would run on a system that does not have pgsql installed. Cheers F. -- Please avoid sending me Word, Excel or PowerPoint attachments. See http://www.gnu.org/philosophy/no-word-attachments.html ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
Hi, Le 09/12/11 01:52, Lionel Elie Mamane a écrit : Now that I think of it, since libpq is in a path like /Library/PostgreSQL/9.1/lib/libpq.5.dylib, how is it supposed to be found by the dynamic linker at runtime? That's probably the problem, unless that directory is added to the equivalent of /etc/ld.so.conf on GNU/Linux... Try: otool -L /path/to/your/postgresql-sdbc-impl.uno.dylib If that's indeed the problem, you can solve it with the DYLD_LIBRARY_PATH envvar, but then, how do we fix it for *all* our users (that have an ABI-compatible libpq installed) automatically, since their libq.5.dylib may be at another path than on our build machine? Or maybe MacOS X work à la Microsoft Windows for libraries in that every application is supposed to bundle a copy of every dynamic library it uses? I wouldn't have expected so. If I compare with the postgres OXT file on Linux, the connector includes the impl.uno file, which weighs in at 6.2Mb uncompressed. However, the Mac OXT in comparison does not include the impl.uno file in its bundle, so maybe it is just a packaging issue ? Alex ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
Hi Lionel, Le 09/12/11 01:52, Lionel Elie Mamane a écrit : Try: otool -L /path/to/your/postgresql-sdbc-impl.uno.dylib If that's indeed the problem, you can solve it with the DYLD_LIBRARY_PATH envvar, but then, how do we fix it for *all* our users (that have an ABI-compatible libpq installed) automatically, since their libq.5.dylib may be at another path than on our build machine? Or maybe MacOS X work à la Microsoft Windows for libraries in that every application is supposed to bundle a copy of every dynamic library it uses? I wouldn't have expected so. Here's my otool output : otool -L /Users/Shared/LO/master/connectivity/unxmacxi.pro/lib/postgresql-sdbc-impl.uno.dylib /Users/Shared/LO/master/connectivity/unxmacxi.pro/lib/postgresql-sdbc-impl.uno.dylib: @__OOO/postgresql-sdbc-impl.uno.dylib (compatibility version 0.0.0, current version 0.0.0) /usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 227.0.0) @loader_path/../ure-link/lib/libuno_cppu.dylib.3 (compatibility version 0.0.0, current version 0.0.0) @loader_path/../ure-link/lib/libuno_cppuhelpergcc3.dylib.3 (compatibility version 0.0.0, current version 0.0.0) @loader_path/../ure-link/lib/libuno_sal.dylib.3 (compatibility version 0.0.0, current version 0.0.0) @loader_path/../ure-link/lib/libuno_salhelpergcc3.dylib.3 (compatibility version 0.0.0, current version 0.0.0) libpq.5.dylib (compatibility version 5.0.0, current version 5.3.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 88.3.11) /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.4.0) /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0) Note that I actually have 2 postgresql-sdbc-impl.uno.dylib files in my tree, which have different sizes ! Not quite sure why, as my understanding was that the one in solver was simply a copy of the one in connectivity ? Alex ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
On Fri, Dec 09, 2011 at 10:35:51AM +0100, Alexander Thurgood wrote: Le 09/12/11 01:52, Lionel Elie Mamane a écrit : Now that I think of it, since libpq is in a path like /Library/PostgreSQL/9.1/lib/libpq.5.dylib, how is it supposed to be found by the dynamic linker at runtime? If I compare with the postgres OXT file on Linux, the connector includes the impl.uno file, which weighs in at 6.2Mb uncompressed. However, the Mac OXT in comparison does not include the impl.uno file in its bundle, so maybe it is just a packaging issue ? That would indeed make it not work, as this contains the actual driver! But I'm clueless as to why that is so. The Mac OS .oxt file has postgresql-sdbc.uno.dylib, but not postgresql-sdbc-impl.uno.dylib if I understand right? Maybe I can find a clue in verbose build log of postgresql-sdbc? Here's how to get one: source Env.Host.sh cd connectivity find unxmac* -name '*postgresql*' -or -name 'pq_*' | xargs rm build verbose=T 21 | tee build.log And then the build log is in file build.log -- Lionel ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
On 12/09/2011 10:44 AM, Alexander Thurgood wrote: otool -L /Users/Shared/LO/master/connectivity/unxmacxi.pro/lib/postgresql-sdbc-impl.uno.dylib /Users/Shared/LO/master/connectivity/unxmacxi.pro/lib/postgresql-sdbc-impl.uno.dylib: @__OOO/postgresql-sdbc-impl.uno.dylib (compatibility version 0.0.0, current version 0.0.0) /usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 227.0.0) @loader_path/../ure-link/lib/libuno_cppu.dylib.3 (compatibility version 0.0.0, current version 0.0.0) @loader_path/../ure-link/lib/libuno_cppuhelpergcc3.dylib.3 (compatibility version 0.0.0, current version 0.0.0) @loader_path/../ure-link/lib/libuno_sal.dylib.3 (compatibility version 0.0.0, current version 0.0.0) @loader_path/../ure-link/lib/libuno_salhelpergcc3.dylib.3 (compatibility version 0.0.0, current version 0.0.0) libpq.5.dylib (compatibility version 5.0.0, current version 5.3.0) That way it won't be found at runtime. If it is a lib from the system, it should have an absolute path (like the ones below), if it is part of LO it should be referenced relatively via @loader_path (like the ones above). /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 88.3.11) /usr/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.4.0) /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0) Note that I actually have 2 postgresql-sdbc-impl.uno.dylib files in my tree, which have different sizes ! Not quite sure why, as my understanding was that the one in solver was simply a copy of the one in connectivity ? For still-dmake-based modules, libraries are stripped when copied from the local output tree to solver. Stephan ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
Norbert Thiebaud wrote: the connector can now be built on MacOSX using system postgresql. (fixed a couple of WaE.. nothing serious) And with attached patch, also a plain --enable-ext-postgresql-sdbc builds - Norbert, want to double-check commit? Cheers, -- Thorsten diff --git a/postgresql/makefile.mk b/postgresql/makefile.mk index 2aae5db..e4c1a7d 100644 --- a/postgresql/makefile.mk +++ b/postgresql/makefile.mk @@ -64,6 +64,11 @@ BUILD_DIR=src/interfaces/libpq # but that's quite some guessing work. CONFIGURE_ACTION = ./configure --without-readline --disable-shared BUILD_ACTION = make -j$(GMAKE_MODULE_PARALLELISM) + +.IF $(OS)==MACOSX +BUILD_ACTION += all-static-lib +.ENDIF + .ENDIF # --- Targets -- pgpKJNqweneUi.pgp Description: PGP signature ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
On Fri, Dec 09, 2011 at 02:24:15PM +0100, Stephan Bergmann wrote:
On 12/09/2011 10:44 AM, Alexander Thurgood wrote:
otool -L
/Users/Shared/LO/master/connectivity/unxmacxi.pro/lib/postgresql-sdbc-impl.uno.dylib
/Users/Shared/LO/master/connectivity/unxmacxi.pro/lib/postgresql-sdbc-impl.uno.dylib:
@__OOO/postgresql-sdbc-impl.uno.dylib
(compatibility version 0.0.0, current version 0.0.0)
(compatibility version 0.0.0, current version 0.0.0)
libpq.5.dylib (compatibility version 5.0.0, current version 5.3.0)
That way it won't be found at runtime. If it is a lib from the
system, it should have an absolute path (like the ones below), if it
is part of LO it should be referenced relatively via @loader_path
(like the ones above).
Well, as a system lib I understand it can be installed nearly anywhere:
- The one click installer puts it in /Library/PostgreSQL/${VERSION}/lib
- fink and MacPorts put it *by default* in
/sw/opt/postgresql/${VERSION}/bin
/opt/local/lib/postgresql${VERSION}/bin
But from documentation on the web, I gather that the user can
change the /sw/ and /opt/local prefix to anything.
So I wonder what we would put as absolute path. Are we supposed to
bundle it with LibreOffice, à la Microsoft Windows?
--
Lionel
___
LibreOffice mailing list
LibreOffice@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
Well, as a system lib I understand it can be installed nearly anywhere Er... huh? Isn't the very definition of a system library one that is installed in a fixed known location, as part of the operating system? --tml ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
On Fri, Dec 9, 2011 at 5:53 PM, Tor Lillqvist t...@iki.fi wrote: Well, as a system lib I understand it can be installed nearly anywhere Er... huh? Isn't the very definition of a system library one that is installed in a fixed known location, as part of the operating system? Or at least there be a mechanism to find where it is a run-time Norbert ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
On Thu, Dec 08, 2011 at 02:04:16PM -0600, Norbert Thiebaud wrote: the connector can now be built on MacOSX using system postgresql. (fixed a couple of WaE.. nothing serious) Great, thanks. -- Lionel ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
Le 08/12/2011 21:04, Norbert Thiebaud a écrit : Hi Norbert, As a follow-up: the connector can now be built on MacOSX using system postgresql. (fixed a couple of WaE.. nothing serious) What was the size of your Mac OSX pg connector ? Did you actually test it ? Mine turned out to be 300kb, whereas on Linux it is 2.2Mb. Rather unsurprisingly, although the extension can be registered, it did not work (whereas on Linux it does). When I tried to connect to a running pg instance, I got the error message : The driver could not be loaded (in French). For some reason, the OXT seems to be missing a chunk of what it needs. Will investigate more tomorrow. Alex ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
On 08/12/11 18:45, Lionel Elie Mamane wrote: - knock up our own Makefile for the subpart we need: more fragile wrt to new versions of PostgreSQL (may need to update that makefile); but possibly still the easiest solution. Ok, I checked the released sources. The good news is that there is a src/win32.mak nmake file that produces a standalone libpq. Not sure whether static or dynamic but that should be easy to change. Not sure though whether it build with MSVC 2008 that we use, but should be adaptable I guess. The source tarball is 14MB (on the higher side, but still smaller then our sources). Cheers F. -- Please avoid sending me Word, Excel or PowerPoint attachments. See http://www.gnu.org/philosophy/no-word-attachments.html ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
On Thu, Dec 8, 2011 at 4:03 PM, Alex Thurgood alex.thurg...@gmail.com wrote: Le 08/12/2011 21:04, Norbert Thiebaud a écrit : Hi Norbert, As a follow-up: the connector can now be built on MacOSX using system postgresql. (fixed a couple of WaE.. nothing serious) What was the size of your Mac OSX pg connector ? -rwxr-xr-x 1 n_th staff 1.0M Dec 8 16:35 postgresql-sdbc-impl.uno.dylib -rw-r--r-- 1 n_th staff 305K Dec 8 16:35 postgresql-sdbc.oxt -rwxr-xr-x 1 n_th staff59K Dec 8 16:35 postgresql-sdbc.uno.dylib -rw-r--r-- 1 n_th staff29B Dec 8 16:35 postgresql-sdbc.unorc -rw-r--r-- 1 n_th staff 4.8K Dec 8 16:35 postgresql.xcu Did you actually test it ? no Norbert ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] postresql-sdbc bits out of minutes of tech. steering call
On Thu, Dec 08, 2011 at 11:03:08PM +0100, Alex Thurgood wrote: Le 08/12/2011 21:04, Norbert Thiebaud a écrit : the connector can now be built on MacOSX using system postgresql. (fixed a couple of WaE.. nothing serious) What was the size of your Mac OSX pg connector ? Did you actually test it ? Mine turned out to be 300kb, whereas on Linux it is 2.2Mb. Rather unsurprisingly, although the extension can be registered, it did not work (whereas on Linux it does). Now that I think of it, since libpq is in a path like /Library/PostgreSQL/9.1/lib/libpq.5.dylib, how is it supposed to be found by the dynamic linker at runtime? That's probably the problem, unless that directory is added to the equivalent of /etc/ld.so.conf on GNU/Linux... Try: otool -L /path/to/your/postgresql-sdbc-impl.uno.dylib If that's indeed the problem, you can solve it with the DYLD_LIBRARY_PATH envvar, but then, how do we fix it for *all* our users (that have an ABI-compatible libpq installed) automatically, since their libq.5.dylib may be at another path than on our build machine? Or maybe MacOS X work à la Microsoft Windows for libraries in that every application is supposed to bundle a copy of every dynamic library it uses? I wouldn't have expected so. -- Lionel ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
