Re: Integration of a Perl XS module with an Automake build system
On 30 June 2015 at 23:06, Bob Friesenhahn wrote: > On Tue, 30 Jun 2015, Gavin Smith wrote: >> >> >> What you say about using a different compiler is worrying, though, and >> I hadn't thought about it before. So basically when compiling an XS >> module, you need to use whatever compiler was used to compile perl in >> the first place? I think we could retrieve this information and use it >> only when compiling the XS module, allowing the rest of the program to >> be compiled with the compiler that the configure script or user >> chooses. > > > If you use a system which supports several completely different compilers > targeting the same ABI, you immediately find dismay unless you compile > Python using the same compiler you plan to use. > > This is quite evident on Solaris systems which use the Sun/Oracle compiler > to build the bundled Python so it produces Sun compiler options which don't > work with GCC. Assuming the same problem exists with Perl, then the only solution to this I can think of is to generate the compiler options we need automatically. Hopefully libtool can do this, and the diversity among the ways that Perl can be compiled isn't so great that we are left in the position of trying to convert from the compiler flags that Perl was compiled with to the compiler flags of another compiler. If the ABI is the same, it should be possible for Perl to load in a properly compiled shared library object. Through a process of elimination, I've found that the flag that made the difference in my case was -D_FILE_OFFSET_BITS=64. Without this I get: Not a CODE reference at Texinfo/Convert/XSParagraph/lib/XSParagraph.pm line 83. END failed--call queue aborted at Texinfo/Convert/XSParagraph/lib/XSParagraph.pm line 83. Compilation failed in require at Texinfo/Convert/Plaintext.pm line 34. BEGIN failed--compilation aborted at Texinfo/Convert/Plaintext.pm line 34. Compilation failed in require at Texinfo/Convert/Info.pm line 25. BEGIN failed--compilation aborted at Texinfo/Convert/Info.pm line 25. Compilation failed in require at ./texi2any.pl line 115. BEGIN failed--compilation aborted at ./texi2any.pl line 115. It still worked when I removed other flags like "-fstack-protector", so hopefully as long as _FILE_OFFSET_BITS and maybe a few other symbols are defined to the same values as when Perl was compiled, everything will work. > Not specifically when cross-compiling. Perl has a particular way to name the > directories in which it puts the module in the build tree and installation > tree. Part of this includes the basic architecture of the machine (as known > to Perl). As long as we use the directory layout Perl is expecting, everything should be fine, right?
Re: Integration of a Perl XS module with an Automake build system
On Tue, 30 Jun 2015, Gavin Smith wrote: What you say about using a different compiler is worrying, though, and I hadn't thought about it before. So basically when compiling an XS module, you need to use whatever compiler was used to compile perl in the first place? I think we could retrieve this information and use it only when compiling the XS module, allowing the rest of the program to be compiled with the compiler that the configure script or user chooses. If you use a system which supports several completely different compilers targeting the same ABI, you immediately find dismay unless you compile Python using the same compiler you plan to use. This is quite evident on Solaris systems which use the Sun/Oracle compiler to build the bundled Python so it produces Sun compiler options which don't work with GCC. Something else which changes is the identity of the target, which determines where build products go in the build tree and when installed. What, you mean if cross-compiling? It seems impossible: how are we supposed to know what compiler flags to use? Some information might have to be provided manually. The same problem must have occurred with libtool as well, for cross-compiling shared libraries. Not specifically when cross-compiling. Perl has a particular way to name the directories in which it puts the module in the build tree and installation tree. Part of this includes the basic architecture of the machine (as known to Perl). Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/
Re: Integration of a Perl XS module with an Automake build system
On 30 June 2015 at 22:41, Bob Friesenhahn wrote: >> as well as a heap of other variable definitions copied from the >> MakeMaker Makefile. Now I have to see how ExtUtils:MakeMaker knew what >> flags to use to see if I can replicate it. > > > These definitely come from how Perl was originally compiled and built. If a > different compiler sas used, the provided options may be completely > different and not work with the compiler you are currently using (even if > the generated code would interoperate). > > It would be good to find a way to tease these parameters out of the Perl > installation. I discovered the parameters were from a file "/usr/lib/perl5/Config_heavy.pl" on my system, and they should be retrievable using the Config perl module. I have the idea to retrieve this information in configure.ac and substitute it into the Makefile. What you say about using a different compiler is worrying, though, and I hadn't thought about it before. So basically when compiling an XS module, you need to use whatever compiler was used to compile perl in the first place? I think we could retrieve this information and use it only when compiling the XS module, allowing the rest of the program to be compiled with the compiler that the configure script or user chooses. > Something else which changes is the identity of the target, which determines > where build products go in the build tree and when installed. What, you mean if cross-compiling? It seems impossible: how are we supposed to know what compiler flags to use? Some information might have to be provided manually. The same problem must have occurred with libtool as well, for cross-compiling shared libraries.
Re: Fwd: installation of automake, autoconf, m4 error
On Tue, 30 Jun 2015, Krishnan Rajiyah wrote: However, the problem I had with m4 make was not with perl, it said that automake-1.14 was missing from the system. This does not make sense since that would mean that there would be a cycle in the dependency tree. Most automake projects know how to automatically regenerate the autoconf/automake parts if a file has changed. If the Makefile is trying to regenerate autotools parts, then a file has been edited or a file timestamp is incorrect. If a network filesystem like NFS is involved, then make sure that the client and server agree on the time (such as by using NTP). If you checked the files into a version control system and are using checked-out files, then it may be that the version control system did not preserve the time-stamps. Can you please tell me what I am doing wrong? Is your software even compatible with a NetBSD-4.0.1-x68k system? I doubt that NetBSD is excluded. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/
Re: Integration of a Perl XS module with an Automake build system
On Tue, 30 Jun 2015, Gavin Smith wrote: as well as a heap of other variable definitions copied from the MakeMaker Makefile. Now I have to see how ExtUtils:MakeMaker knew what flags to use to see if I can replicate it. These definitely come from how Perl was originally compiled and built. If a different compiler sas used, the provided options may be completely different and not work with the compiler you are currently using (even if the generated code would interoperate). It would be good to find a way to tease these parameters out of the Perl installation. Something else which changes is the identity of the target, which determines where build products go in the build tree and when installed. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/
Fwd: installation of automake, autoconf, m4 error
-- Forwarded message - From: Krishnan Rajiyah Date: Mon, Jun 29, 2015 at 10:40 AM Subject: installation of automake, autoconf, m4 error To: , Hello GNU, I have been trying to install automake, autoconf, and m4 on my NetBSD-4.0.1-x68k system. As I understand from your website, automake requires autoconf which requires m4 which requires perl. I havent installed perl yet, because I am getting errors saying that it cant make built-in (still trying to find a way around this). However, the problem I had with m4 make was not with perl, it said that automake-1.14 was missing from the system. This does not make sense since that would mean that there would be a cycle in the dependency tree. I am trying to install the latest version of autoconf (2012 release), the latest version of m4 (2013 release), and the second latest verison of automake (2013 release). Can you please tell me what I am doing wrong? Is your software even compatible with a NetBSD-4.0.1-x68k system? Thanks, Krishnan Rajiyah
Re: Integration of a Perl XS module with an Automake build system
On 30 June 2015 at 14:25, Bob Friesenhahn wrote:
> In addition to behavior issues, a major problem is that the re-link
> capabilities offered by libtool are not available. If one links from the
> Perl module to a shared library in the build tree, then there is the risk of
> continued reference to the build tree after installation (a
> security/reliability issue) and the module might not even run. The Perl test
> suite is likely to use the wrong shared libraries while testing due to
> GNU/Linux's strong preference to using already-installed libraries cached by
> 'ldconfig'. Due to this, I changed GraphicsMagick so that it only links its
> Perl module at 'make install' and time it uses the already-installed shared
> library. This still does not help with DESTDIR installs.
>
> After I made this change, some GNU/Linux distributions stopped building the
> GraphicsMagick Perl module at all due to claiming that the build is
> "broken". They expect that everything builds in tree in one step and that
> 'make check' works as one would expect. I don't think that this is possible
> to accomplish in a portable way using ExtUtils::MakeMaker.
>
> If there is a viable solution which avoids ExtUtils::MakeMaker and allows
> Automake to drive behavior, with linkage managed by libtool, then I am all
> ears.
I'm not linking to another dynamic library from the XS module, just a
static library, but I've been trying to build the library with libtool
using a Makefile.am of my own writing, using the Makefile output by
ExtUtils::MakeMaker as a reference. I've had a little bit success so
far, so hopefully I'm not on a wild-goose chase. I load the *.so file
libtool produces with the Perl DynaLoader module. The main problem
seems to be getting the right compiler flags. The first time I tried I
got a bizarre error message like "Not a CODE reference" when I tried
to run the bootstrap function in the module. (Although the object
wasn't null, I couldn't find out what it was.) I got it to work when I
copied a bunch of compiler flags over from the MakeMaker Makefile. My
Makefile.am currently has the lines:
-
pkglib_LTLIBRARIES = XSParagraph.la
XSParagraph_la_SOURCES = XSParagraph.c mylib/xspara.c mylib/xspara.h
BUILT_SOURCES = XSParagraph.c
XSParagraph_la_CPPFLAGS = -I. -I$(PERL_INC) -D_REENTRANT -D_GNU_SOURCE
-I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
XSParagraph_la_CFLAGS = -fno-strict-aliasing -pipe -fstack-protector
-O2 -march=i486 -mtune=i686 -DVERSION=\"6.0\" -DXS_VERSION=\"6.0\"
$(CCCDLFLAGS) "-I/usr/lib/perl5/CORE"
XSParagraph_la_LDFLAGS=-module
as well as a heap of other variable definitions copied from the
MakeMaker Makefile. Now I have to see how ExtUtils:MakeMaker knew what
flags to use to see if I can replicate it.
Using the DynaLoader module instead of XSLoader gives us the
flexibility to go looking for a dynamically-loaded library file
ourselves. The code to load the module looked like this:
---
our $VERSION = '6.0';
#bootstrap XSParagraph $VERSION;
# TODO: Get this from XSParagraph.la, the dlname field
my $dlname = "Texinfo/Convert/XSParagraph/.libs/XSParagraph.so.0";
my $module = "XSParagraph";
# Following steps under "bootstrap" in "man DynaLoader".
# Doesn't work
#push @dl_require_symbols, "boot_$module";
# TODO: Execute blib/arch/auto/XSParagraph/XSParagraph.bs ?
# That file is empty.
#my $flags = dl_load_flags $module; # This is 0 in DynaLoader
my $flags = 0;
my $libref = DynaLoader::dl_load_file($dlname, $flags);
if (!$libref) {
die "Couldn't find *.so file\n.";
}
my @undefined_symbols = DynaLoader::dl_undef_symbols();
if ($#undefined_symbols+1 != 0) {
warn "XSParagraph: still have undefined symbols after dl_load_file\n";
}
my $symref = DynaLoader::dl_find_symbol($libref, "boot_$module");
if (!$symref) {
die "XSParagraph: Couldn't find boot_$module symbol\n";
}
my $xs = DynaLoader::dl_install_xsub("${module}::bootstrap",
$symref, $dlname);
if (!$xs) {
die "XSParagraph: Couldn't bootstrap\n";
}
# Doesn't work
#push DynaLoader::@dl_shared_objects, $file; # record files loaded
&$xs($module, $VERSION);
Re: Integration of a Perl XS module with an Automake build system
On Tue, 30 Jun 2015, Gavin Smith wrote: Hello all, I wonder if anyone could give pointers to advice on how to integrate a Perl XS module with an Automake build system? Over at the Texinfo project we're starting to replace parts of makeinfo with loadable native-code modules, for speed. The problem is that the Makefiles produced by ExtUtils::MakeMaker (the module used by the XS tutorial, "man perlxstut") doesn't follow the standards required by Automake. (Info node "(automake)Third-Party Makefiles", and the GNU Coding Standards). For example, "make clean" actually removes the Makefile. In addition to behavior issues, a major problem is that the re-link capabilities offered by libtool are not available. If one links from the Perl module to a shared library in the build tree, then there is the risk of continued reference to the build tree after installation (a security/reliability issue) and the module might not even run. The Perl test suite is likely to use the wrong shared libraries while testing due to GNU/Linux's strong preference to using already-installed libraries cached by 'ldconfig'. Due to this, I changed GraphicsMagick so that it only links its Perl module at 'make install' and time it uses the already-installed shared library. This still does not help with DESTDIR installs. After I made this change, some GNU/Linux distributions stopped building the GraphicsMagick Perl module at all due to claiming that the build is "broken". They expect that everything builds in tree in one step and that 'make check' works as one would expect. I don't think that this is possible to accomplish in a portable way using ExtUtils::MakeMaker. If there is a viable solution which avoids ExtUtils::MakeMaker and allows Automake to drive behavior, with linkage managed by libtool, then I am all ears. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/
Integration of a Perl XS module with an Automake build system
Hello all, I wonder if anyone could give pointers to advice on how to integrate a Perl XS module with an Automake build system? Over at the Texinfo project we're starting to replace parts of makeinfo with loadable native-code modules, for speed. The problem is that the Makefiles produced by ExtUtils::MakeMaker (the module used by the XS tutorial, "man perlxstut") doesn't follow the standards required by Automake. (Info node "(automake)Third-Party Makefiles", and the GNU Coding Standards). For example, "make clean" actually removes the Makefile. I found a couple of web pages that mentioned this, so basically I was looking for any updates since then, or other information. http://www.eyrie.org/~eagle/journal/2013-01/025.html suggested not to use ExtUtils::MakeMaker at all. http://lists.gnu.org/archive/html/automake/2009-10/msg00034.html gave a link to how t was done in GraphicsMagick. Today's version of that appears to be at http://hg.code.sf.net/p/graphicsmagick/code/file/f11c2c4e798c/PerlMagick/Makefile.am. Many thanks, Gavin
Re: 回复: Re: why forbidding "include" a sub-makefile.am with absolute path
On Tue, 30 Jun 2015, 远猷 wrote: thanks for your reply! but why forbidding “include” a sub-makefiles.am with absolute path? Automake projects are expected to be self-contained and not refer to anything outside of the package. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/
Re: Turn off C compiler warnings in automake
於 二，2015-06-30 於 09:37 +0200，Thomas Jahns 提到： > On 06/29/15 17:31, Alex Vong wrote: > > Thanks for telling me there is no portable flag for doing so. > > I am now using AC_SUBST() to set the value of STREAM and append ` > > $(STREAM)>/dev/null' to every make command. If the user configure with > > --enable-verbose-compiler, then STREAM will be set to 0, otherwise > > STREAM will be set to 2. This keeps the flexibility. Does it sound > > reasonable? > > I'm not sure if an output redirection on stdin is portable. But I think given > the way you are going about this you could easily set a Makefile variable to > the > full redirection (or none), i.e. lose the >/dev/null and instead make STREAM > be > either '2>/dev/null' or the empty string ''? > > Regards, Thomas Hi Thomas, I tried substituting `2>/dev/null' before but without success. However, I try again just now and find out that if I add a pair of single quotes, then it will work just fine. In short, the following code will work AS_IF([test "x$enable_verbose_compiler" != "xyes"], [AC_SUBST([REDIRECTION], ['2>/dev/null'])] ) Thanks for the suggestion! Cheers, Alex
Re: Turn off C compiler warnings in automake
On 06/29/15 17:31, Alex Vong wrote: Thanks for telling me there is no portable flag for doing so. I am now using AC_SUBST() to set the value of STREAM and append ` $(STREAM)>/dev/null' to every make command. If the user configure with --enable-verbose-compiler, then STREAM will be set to 0, otherwise STREAM will be set to 2. This keeps the flexibility. Does it sound reasonable? I'm not sure if an output redirection on stdin is portable. But I think given the way you are going about this you could easily set a Makefile variable to the full redirection (or none), i.e. lose the >/dev/null and instead make STREAM be either '2>/dev/null' or the empty string ''? Regards, Thomas -- Thomas Jahns HD(CP)^2 Abteilung Anwendungssoftware Deutsches Klimarechenzentrum GmbH Bundesstraße 45a • D-20146 Hamburg • Germany Phone: +49 40 460094-151 Fax:+49 40 460094-270 Email: Thomas Jahns URL:www.dkrz.de Geschäftsführer: Prof. Dr. Thomas Ludwig Sitz der Gesellschaft: Hamburg Amtsgericht Hamburg HRB 39784 smime.p7s Description: S/MIME Cryptographic Signature
