Re: New release
On Fri, Nov 10, 2023 at 9:16 AM Daniel Sahlberg wrote: > > Den fre 10 nov. 2023 kl 15:09 skrev Nathan Hartman : >> >> On Fri, Nov 10, 2023 at 8:02 AM Johan Corveleyn wrote: >>> >>> Hi Nathan, >>> >>> Sorry I haven't helped so far ... I should be able to say something >>> useful here, since I'm a java dev in my dayjob. But I have been >>> drowning a bit in that same dayjob lately, so I can barely find the >>> time to scan these mails, and then forget about them because my >>> attention is needed elsewhere. >>> >>> Anyway, I'll try to find some more time to take a closer look at the >>> errors you get. >>> >>> Just as a first shot: if there is no RunTests.class in the location >>> where it should be (one of the directories in the classpath), then it >>> probably has not been compiled (or it has not been put in the right >>> location). My first guess would also be, like Daniel mentioned, that >>> you'll need the --with-junit option (referring to a junit-X.Y.jar >>> somewhere on your system) and then the build scripts should compile >>> the test classes. >>> >>> But I guess you already tried that, after Daniel's reply, and you're >>> still running into problems. I'll take a closer look when you send >>> your detailed mail then. >>> >>> -- >>> Johan >> >> >> >> Thanks Johan. Apologies for not sending the detailed mail yet. I wanted to >> test a few specific theories but didn't get that far last night because I >> hit a problem unrelated to SVN. >> >> One thing I'm struggling with is to understand which part of the build >> system is supposed to compile RunTests, i.e., how/where the Java compiler is >> being invoked on it. I'm not seeing it in the build logs, though I do see >> the other Java files being compiled, so that's certainly interesting. > > > I'm fairly sure that RunTests is compiled only when --with-junit is present > (don't know if "present" is enough or if it has to point to a valid JAR > file). Next question is if check-javahl should be available at all if > --with-junit is missing or if check-javahl should make sure RunTests is > built. Either way, I think it is a bug in the build system. This question > should probably be carved out to its own e-mail thread. I'll try to figure > something out and start a new thread. > > Kind regards, > Daniel Hi all, Success! I have finally seen the JavaHL tests build and run! I'd like to say a great big Thank You to everyone for helping me get this far! I was writing on IRC as I went through a number of steps [1]. What changed? (1) Switched from my customized Makefile.svn to SVN's Makefile.svn, as of the latest rev on trunk now (r1913724). (2) Worked around a problem building gettext. (3) Worked around a problem with serf-1.3.10 that DSahlberg saw also. Regarding #1: Note: Makefile.svn refers to tools/dev/unix-build/Makefile.svn. My customized Makefile.svn was based on SVN's Makefile.svn@1871775. As to my customizations, I can provide them later, but the tl;dr is: (a) Build newer versions of some dependencies. (b) Remove support for things I didn't need (like serf-old). (c) Tweak some details to workaround build problems. In all cases, any changes I made were done to workaround various build problems I had encountered. Where appropriate, I upstreamed certain changes: r1875794, r1872380. I didn't upstream changes that I considered to be tweaks that only pertained to my machine. This made it possible for me to build, develop, and test [local,svn,serf] x [bdb,fsfs], plus all the bindings, except JavaHL (until now). In the meantime (since 2019 when r1871775 happened), SVN's Makefile.svn has received many updates, including much improved handling of Python-2/3 issues, so I wanted to adopt this and eliminate my customized one. Regarding #2: Starting a few days ago, I ran into a weird problem with gettext. This manifests with both versions of Makefile.svn (mine and SVN's). I am not sure why this started happening, but when gettext's configure runs, there is a nested gettext-tools's configure, which runs until it prints "checking for Java virtual machine... yes" and then it gets stuck there indefinitely. It happens with multiple versions of gettext and after a few nights of investigation, I still don't know why. I could find nothing about this online. (This is the "brick wall" I ran into that I mentioned on IRC [2], for those who saw that and wondered.) It is clearly some idiosyncracy of my machine. Tonight I found a workaround: I ran Makefile.svn's gettext-configure target, let it reach that point, and then killed the process group. Then I looked in the logs to see exactly how gettext's configure was invoked. I then invoked it "manually" by copying and pasting that incantation into the terminal. To my surprise, this ran to completion. After this, I ran 'touch objdir/gettext/.configured' (this empty dotfile serves as a flag for Makefile.svn not to run configure again). Then I ran Makefile.svn's gettext-compile target, and it
Re: Building with tools/dev/unix-build/Makefile.svn - problems linking with OpenSSL
Den lör 11 nov. 2023 kl 17:14 skrev Stefan Sperling : > On Sat, Nov 11, 2023 at 05:08:28PM +0100, Daniel Sahlberg wrote: > > ne_openssl.o isn't built, the build fails already when running configure > on > > neon. > > Can you show the cofigure log? E.g. via paste.apache.org? > I am also in #svn-dev on Libera IRC if you want to chat there. > The discussion between Stefan and myself continued over on Libera IRC but to tie up the loose end. It was an error on my system, I was missing pkg-config. After installing this, the neon configure worked well and I'm back on track again (or at least, stuck on other thinkgs that i need to look at). Thanks Stefan! Kind regards, Daniel
Re: Building with tools/dev/unix-build/Makefile.svn - problems linking with OpenSSL
On Sat, Nov 11, 2023 at 05:08:28PM +0100, Daniel Sahlberg wrote: > ne_openssl.o isn't built, the build fails already when running configure on > neon. Can you show the cofigure log? E.g. via paste.apache.org? I am also in #svn-dev on Libera IRC if you want to chat there.
Re: Building with tools/dev/unix-build/Makefile.svn - problems linking with OpenSSL
Den lör 11 nov. 2023 kl 16:43 skrev Stefan Sperling : > On Sat, Nov 11, 2023 at 04:34:07PM +0100, Stefan Sperling wrote: > > On Sat, Nov 11, 2023 at 04:23:52PM +0100, Daniel Sahlberg wrote: > > > As for the original problem, updating to the latest version of the Neon > > > library doesn't help, I still get the same linking error. I''m sure > there > > > is something fishy with my Debian install... > > > > Which libssl is your build trying to use? > The libssl-dev package, which is OpenSSL version 1.1.1w-0+deb11u1. > > > > There is nothing in Makefile.svn that would try to pick a particular > > version of OpenSSL. It just expects that a compatible version is > > already installed. > > > > Maybe you have multiple versions installed and that confuses things? > Don't think so. There is only one libssl.a (and libssl.so) on my system. Doing nm -g | grep OPENSSL_init_ssl (which is not the function in the error message but as I've written before SSL_library_init is just a #define for OPENSSL_init_ssl, with the appropriate define for OPENSSL_API_COMPAT) finds: 00270 T OPENSSL_init_ssl > What is the output of this command on your system (in the ~/svn > build dir): > > nm objdir/neon-0.32.5/src/ne_openssl.o | grep -i SSL_library_init > > This doesn't print anything for me. If this shows the symbol > on your system that would indicate a wrong OPENSSL_VERSION macro > being used during the build. > ne_openssl.o isn't built, the build fails already when running configure on neon. Kind regards, Daniel
Re: Building with tools/dev/unix-build/Makefile.svn - problems linking with OpenSSL
On Sat, Nov 11, 2023 at 04:34:07PM +0100, Stefan Sperling wrote: > On Sat, Nov 11, 2023 at 04:23:52PM +0100, Daniel Sahlberg wrote: > > As for the original problem, updating to the latest version of the Neon > > library doesn't help, I still get the same linking error. I''m sure there > > is something fishy with my Debian install... > > Which libssl is your build trying to use? > > There is nothing in Makefile.svn that would try to pick a particular > version of OpenSSL. It just expects that a compatible version is > already installed. > > Maybe you have multiple versions installed and that confuses things? What is the output of this command on your system (in the ~/svn build dir): nm objdir/neon-0.32.5/src/ne_openssl.o | grep -i SSL_library_init This doesn't print anything for me. If this shows the symbol on your system that would indicate a wrong OPENSSL_VERSION macro being used during the build.
Re: Building with tools/dev/unix-build/Makefile.svn - problems linking with OpenSSL
On Sat, Nov 11, 2023 at 04:23:52PM +0100, Daniel Sahlberg wrote: > As for the original problem, updating to the latest version of the Neon > library doesn't help, I still get the same linking error. I''m sure there > is something fishy with my Debian install... Which libssl is your build trying to use? There is nothing in Makefile.svn that would try to pick a particular version of OpenSSL. It just expects that a compatible version is already installed. Maybe you have multiple versions installed and that confuses things?
Re: Building with tools/dev/unix-build/Makefile.svn - problems linking with OpenSSL
On Sat, Nov 11, 2023 at 04:07:52PM +0100, Stefan Sperling wrote: > On Sat, Nov 11, 2023 at 04:00:58PM +0100, Stefan Sperling wrote: > > On Sat, Nov 11, 2023 at 03:53:09PM +0100, Stefan Sperling wrote: > > > If neon requests OPENSSL_API_COMPAT 0x1000L and also calls > > > OPENSSL_init_ssl() then that is a bug in neon. > > > > The neon version used by Makefile.svn is quite old. > > Newer releases are available here: https://notroj.github.io/neon/ > > You could try using the latest 0.32.5 release instead of 0.30.2. > > That should deal just fine with newer versions of OpenSSL. > > This should be enough to get started: Building with this I got an error during 'make install' in neon. I could fix that by installing xmlto and calling the 'make docs' target explicitly. I have committed these changes. The update was long overdue in any case.
Re: Building with tools/dev/unix-build/Makefile.svn - problems linking with OpenSSL
Den lör 11 nov. 2023 kl 15:54 skrev Stefan Sperling :
> On Sat, Nov 11, 2023 at 02:22:55PM +0100, Daniel Sahlberg wrote:
> > Hi,
> >
> > I'm separating this out to a new thread since it doesn't relate to the
> > upcoming release, but I'm doing the work trying to reproduce Nathan's
> > problem.
> >
> > I've installed Debian Bullseye (on Hyper-V, but that shouldn't matter I
> > think). When I try to do the build, I end up having trouble building
> neon.
> > The error message is "could not find library containing
> SSL_library_init".
> > I have checked that the package libssl-dev is installed (with version
> > 1.1.1w-0+deb11u1).
> >
> > I've tried a very simple file:
> > #include
> > int main(void)
> > {
> > SSL_library_init();
> > return 0;
> > }
> >
> > $ gcc -lssl -lcrypto -DOPENSSL_API_COMPAT=0x1000L ssl.c
> > [..] undefined reference to OPENSSL_init_ssl [...]
> >
> > This makes sense, since ssl.h defines SSL_library_init() as
> > OPENSSL_init_ssl(0, NULL) (protected by an #if checking
> OPENSSL_API_COMPAT
> > < 0x1010L).
> >
>
> This function first appeared in OpenSSL 1.1.0.
> Which means trying to call it on earlier version of OpenSSL is an
> expected failure.
>
I've got OpenSSL 1.1.1w. The code is calling SSL_library_init which is a
define pointing to OPENSSL_init_ssl().
The code compiles just fine, but I don't understand why the code isn't
linking.
As for the original problem, updating to the latest version of the Neon
library doesn't help, I still get the same linking error. I''m sure there
is something fishy with my Debian install...
Cheers,
Daniel
Re: Building with tools/dev/unix-build/Makefile.svn - problems linking with OpenSSL
On Sat, Nov 11, 2023 at 04:00:58PM +0100, Stefan Sperling wrote:
> On Sat, Nov 11, 2023 at 03:53:09PM +0100, Stefan Sperling wrote:
> > If neon requests OPENSSL_API_COMPAT 0x1000L and also calls
> > OPENSSL_init_ssl() then that is a bug in neon.
>
> The neon version used by Makefile.svn is quite old.
> Newer releases are available here: https://notroj.github.io/neon/
> You could try using the latest 0.32.5 release instead of 0.30.2.
> That should deal just fine with newer versions of OpenSSL.
This should be enough to get started:
Index: Makefile.svn
===
--- Makefile.svn(revision 1913716)
+++ Makefile.svn(working copy)
@@ -118,7 +118,7 @@
APR_UTIL_VER = 1.6.1
PCRE_VER = 8.41
HTTPD_VER = 2.4.37
-NEON_VER = 0.30.2
+NEON_VER = 0.32.5
SERF_VER = 1.3.10
CYRUS_SASL_VER = 2.1.28
SQLITE_VER = 3390400
@@ -157,7 +157,7 @@
SHA256_${GNU_ICONV_DIST} =
ccf536620a45458d26ba83887a983b96827001e92a13847b45e4925cc8913178
SHA256_${PCRE_DIST} =
244838e1f1d14f7e2fa7681b857b3a8566b74215f28133f14a8f5e59241b682c
SHA256_${HTTPD_DIST} =
aa97a834a32d51974be8d8a013b561e28d327387cb1da2c3c2762acd0146aabd
-SHA256_${NEON_DIST} =
db0bd8cdec329b48f53a6f00199c92d5ba40b0f015b153718d1b15d3d967fbca
+SHA256_${NEON_DIST} =
4872e12f802572dedd4b02f870065814b2d5141f7dbdaf708eedab826b51a58a
SHA256_${CYRUS_SASL_DIST} =
7ccfc6abd01ed67c1a0924b353e526f1b766b21f42d4562ee635a8ebfc5bb38c
SHA256_${SQLITE_DIST} =
f31d445b48e67e284cf206717cc170ab63cbe4fd7f79a82793b772285e78fdbb
SHA256_${LIBMAGIC_DIST} =
694c2432e5240187524c9e7cf1ec6acc77b47a0e19554d34c14773e43dbbf214
@@ -212,8 +212,7 @@
APR_UTIL_URL = https://svn.apache.org/repos/asf/apr/apr-util
PCRE_URL =
https://downloads.sourceforge.net/project/pcre/pcre/$(PCRE_VER)/$(PCRE_DIST)
HTTPD_URL = https://archive.apache.org/dist/httpd/$(HTTPD_DIST)
-#NEON_URL = http://webdav.org/neon/$(NEON_DIST)
-NEON_URL = http://ftp.openbsd.org/pub/OpenBSD/distfiles/$(NEON_DIST)
+NEON_URL = https://notroj.github.io/neon/$(NEON_DIST)
SERF_URL = https://svn.apache.org/repos/asf/serf/tags/$(SERF_VER)
SQLITE_URL = https://www.sqlite.org/2022/$(SQLITE_DIST)
CYRUS_SASL_URL =
https://github.com/cyrusimap/cyrus-sasl/releases/download/cyrus-sasl-${CYRUS_SASL_VER}/$(CYRUS_SASL_DIST)
Re: Building with tools/dev/unix-build/Makefile.svn - problems linking with OpenSSL
On Sat, Nov 11, 2023 at 03:53:09PM +0100, Stefan Sperling wrote: > If neon requests OPENSSL_API_COMPAT 0x1000L and also calls > OPENSSL_init_ssl() then that is a bug in neon. The neon version used by Makefile.svn is quite old. Newer releases are available here: https://notroj.github.io/neon/ You could try using the latest 0.32.5 release instead of 0.30.2. That should deal just fine with newer versions of OpenSSL.
Re: Building with tools/dev/unix-build/Makefile.svn - problems linking with OpenSSL
On Sat, Nov 11, 2023 at 02:22:55PM +0100, Daniel Sahlberg wrote:
> Hi,
>
> I'm separating this out to a new thread since it doesn't relate to the
> upcoming release, but I'm doing the work trying to reproduce Nathan's
> problem.
>
> I've installed Debian Bullseye (on Hyper-V, but that shouldn't matter I
> think). When I try to do the build, I end up having trouble building neon.
> The error message is "could not find library containing SSL_library_init".
> I have checked that the package libssl-dev is installed (with version
> 1.1.1w-0+deb11u1).
>
> I've tried a very simple file:
> #include
> int main(void)
> {
> SSL_library_init();
> return 0;
> }
>
> $ gcc -lssl -lcrypto -DOPENSSL_API_COMPAT=0x1000L ssl.c
> [..] undefined reference to OPENSSL_init_ssl [...]
>
> This makes sense, since ssl.h defines SSL_library_init() as
> OPENSSL_init_ssl(0, NULL) (protected by an #if checking OPENSSL_API_COMPAT
> < 0x1010L).
>
This function first appeared in OpenSSL 1.1.0.
Which means trying to call it on earlier version of OpenSSL is an
expected failure.
> I've tried changing the order of the libraries (-lcrypto -lssl) with the
> same result.
>
> I'm sure I'm missing something simple but I've spent the better half of a
> day figuring out what.
If neon requests OPENSSL_API_COMPAT 0x1000L and also calls
OPENSSL_init_ssl() then that is a bug in neon.
Re: Building with tools/dev/unix-build/Makefile.svn - problems linking with OpenSSL
Hello, On 2023/11/11 22:22, Daniel Sahlberg wrote: Hi, I'm separating this out to a new thread since it doesn't relate to the upcoming release, but I'm doing the work trying to reproduce Nathan's problem. I've installed Debian Bullseye (on Hyper-V, but that shouldn't matter I think). When I try to do the build, I end up having trouble building neon. The error message is "could not find library containing SSL_library_init". I have checked that the package libssl-dev is installed (with version 1.1.1w-0+deb11u1). https://www.openssl.org/docs/manmaster/man3/SSL_library_init.html says: "The SSL_library_init() and OpenSSL_add_ssl_algorithms() functions were deprecated in OpenSSL 1.1.0 by OPENSSL_init_ssl(). " Cheers, -- Yasuhito FUTATSUKI
Building with tools/dev/unix-build/Makefile.svn - problems linking with OpenSSL
Hi,
I'm separating this out to a new thread since it doesn't relate to the
upcoming release, but I'm doing the work trying to reproduce Nathan's
problem.
I've installed Debian Bullseye (on Hyper-V, but that shouldn't matter I
think). When I try to do the build, I end up having trouble building neon.
The error message is "could not find library containing SSL_library_init".
I have checked that the package libssl-dev is installed (with version
1.1.1w-0+deb11u1).
I've tried a very simple file:
#include
int main(void)
{
SSL_library_init();
return 0;
}
$ gcc -lssl -lcrypto -DOPENSSL_API_COMPAT=0x1000L ssl.c
[..] undefined reference to OPENSSL_init_ssl [...]
This makes sense, since ssl.h defines SSL_library_init() as
OPENSSL_init_ssl(0, NULL) (protected by an #if checking OPENSSL_API_COMPAT
< 0x1010L).
I've tried changing the order of the libraries (-lcrypto -lssl) with the
same result.
I'm sure I'm missing something simple but I've spent the better half of a
day figuring out what.
Kind regards,
Daniel
