Re: [R-pkg-devel] Package Load fails to find 3rd Party DLL
Not sure if I reported the success here or not. Copying the 3rd party DLL, but not the .lib file, to the src directory does work around the bug in the linker. The complete working solution can be seen at https://github.com/ralmond/RNetica. Thanks for the help. --Russell On 7/17/23 5:30 AM, Ivan Krylov wrote: В Fri, 14 Jul 2023 22:25:51 +0300 Ivan Krylov пишет: Maybe it's a red herring. Maybe the message from nm about missing file has always been harmless, and what we're seeing here is a bug in the toolchain; perhaps ld.exe doesn't like something about Netica.lib so much that it crashes. I think that's less likely. If you run the commands manually but without mentioning NeticaDLL, do you get a DLL in the end? Judging by your build logs, this could be a toolchain bug. If you set the *.lib file aside and only give the *.dll to the linker (using -l:Netica.dll if necessary), does it still fail? I know that GCC can link directly to *.dll files, without relying on import libraries. __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Package Load fails to find 3rd Party DLL
Okay, I've changed my Makevars.win so that it has:
PKG_LIBS = -L. -L${NETICA_LIB} -lNetica
and
all: NeticaDLL
where ${NETICA_LIB} is set to the appropriate lib subdirectory of the
unpacked sources. I'm no longer getting the nm.exe error, so it may be
a bug in ld.exe.
| gcc -shared -s -static-libgcc -o RNetica.dll tmp.def Cases.o
Continuous.o Edges.o Experience.o Inference.o Networks.o Node.o Random.o
Registration.o Session.o -L.
-L/c/Users/ralmond/Projects/RNetica/src/Netica/Netica_API_510/lib64
-lNetica -LC:/rtools43/x86_64-w64-mingw32.static.posix/lib/x64
-LC:/rtools43/x86_64-w64-mingw32.static.posix/lib
-LC:/PROGRA~1/R/R-43~1.1/bin/x64 -lR
| C:\rtools43\x86_64-w64-mingw32.static.posix\bin/ld.exe: internal
error: aborting at ../../binutils-2.40/ld/ldlang.c:527 in compare_section
| C:\rtools43\x86_64-w64-mingw32.static.posix\bin/ld.exe: please report
this bug
| collect2.exe: error: ld returned 1 exit status
However, when I manually moved Netica.dll to the source directory, and
removed NETICA_LIB from the path, it compiled correctly.
So, I'm guessing this is a bug in ld.exe, and in fact it is here:
https://github.com/msys2/MINGW-packages/issues/15469
So I'll see if I can try to fix my script to only copy the .dll and not
the .lib.
--Russell
On 7/17/23 5:30 AM, Ivan Krylov wrote:
В Fri, 14 Jul 2023 22:25:51 +0300
Ivan Krylov пишет:
Maybe it's a red herring. Maybe the message from nm about missing file
has always been harmless, and what we're seeing here is a bug in the
toolchain; perhaps ld.exe doesn't like something about Netica.lib so
much that it crashes. I think that's less likely. If you run the
commands manually but without mentioning NeticaDLL, do you get a DLL
in the end?
Judging by your build logs, this could be a toolchain bug. If you set
the *.lib file aside and only give the *.dll to the linker (using
-l:Netica.dll if necessary), does it still fail? I know that GCC can
link directly to *.dll files, without relying on import libraries.
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Package Load fails to find 3rd Party DLL
Thanks for the suggestion. In previous versions of the build tools, I know I needed the .lib files. I'm also not sure I'm copying the DLL files into the right directory, so that maybe the linker isn't seeing it. This always confuses me as the location used to build and compile is (potentially) different from the test location and the final build location, and I'm not sure of the best way to refer to these directories in my script. I'm thinking maybe I need to copy the DLL into the src directory. Unfortunately, I don't have a windows box on which to easily test. I'm trying to get a virtualBox setup working so I can test more quickly. --Russell On 7/17/23 5:30 AM, Ivan Krylov wrote: В Fri, 14 Jul 2023 22:25:51 +0300 Ivan Krylov пишет: Maybe it's a red herring. Maybe the message from nm about missing file has always been harmless, and what we're seeing here is a bug in the toolchain; perhaps ld.exe doesn't like something about Netica.lib so much that it crashes. I think that's less likely. If you run the commands manually but without mentioning NeticaDLL, do you get a DLL in the end? Judging by your build logs, this could be a toolchain bug. If you set the *.lib file aside and only give the *.dll to the linker (using -l:Netica.dll if necessary), does it still fail? I know that GCC can link directly to *.dll files, without relying on import libraries. __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Package Load fails to find 3rd Party DLL
Thanks. I know know the problem is in the Makevars.win; however, I'm
still confused.
My `Makevars.win` had
| .PHONY: all NeticaDLL clean
| all: $(SHLIB)
| $(SHLIB): NeticaDLL
|
| NeticaDLL:
| mkdir -p "$(INSTALL_LIB)"
| cp "${NETICA_LIB}/Netica.dll" "${INSTALL_LIB}"
| cp "${NETICA_LIB}/Netica.lib" "${INSTALL_LIB}"
[BTW, I tried change this to
| all: NeticaDLL $(SHLIB)
and got the same problem.]
This looks very much like the `mylibs` example in
https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Using-Makevars
So I'm confused. Why is the Makevars -> Makefile conversion assuming
that all targets of $(SHLIB) (or all) are executable files, ignoring the
PHONY declaration?
Also, does the example in the Writing R Extensions manual still work?
--Russell
On 7/14/23 11:14 AM, Ivan Krylov wrote:
> В Wed, 12 Jul 2023 09:41:11 -0400
> Russell Almond пишет:
>
>> C:\rtools43\x86_64-w64-mingw32.static.posix\bin\nm.exe:
>> 'NeticaDLL': No such file
> This is where the problem starts. You can retrace the steps that R
> takes when building and installing the package by running sh
> configure.win manually and then running something like
> R_PACKAGE_DIR="$(pwd)" R CMD SHLIB -n *.c in the src subdirectory of
> the package. That will in turn tell you the exact command lines to be
> run while building your package, including the following:
>
> nm Cases.o Continuous.o Edges.o Experience.o Inference.o Networks.o \
>Node.o Random.o Registration.o Session.o NeticaDLL \
> | sed -n 's/^.* [BCDRT] / /p' \
> | sed -e '/[.]refptr[.]/d' -e '/[.]weak[.]/d' \
> | sed 's/[^ ][^ ]*/"&"/g' \
> >> tmp.def;
>
> That "NeticaDLL" at the end of the list of object files doesn't belong
> there. I think it gets added because it's among the dependencies of the
> $(SHLIB) Make target. It would be best to make that target a real
> object file that nm.exe can process. Otherwise, you could also write
> your own .def file and skip its automatic generation.
>
> After nm fails, you get a crash in the linker (while parsing the
> resulting incomplete .def file?), which leaves your package without a
> shared library to use:
>
>> C:\rtools43\x86_64-w64-mingw32.static.posix\bin/ld.exe: internal
>> error: aborting at ../../binutils-2.40/ld/ldlang.c:527 in
>> compare_section
>> C:\rtools43\x86_64-w64-mingw32.static.posix\bin/ld.exe: please report
>> this bug
> There must be a way to streamline this process. Maybe put all the
> library-downloading and extraction code into a portable
> tools/configure.R (to be launched manually from the configure shell
> script), leaving src/Makevars only to compile your own code, link with
> Netica using PKG_LIBS, then copy the additional Netica DLL from a
> custom install.libs.R file?
>
--
Russell Almond
https://ralmond.net/
[[alternative HTML version deleted]]
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
[R-pkg-devel] Package Load fails to find 3rd Party DLL
I have an R package (RNetica available at https://ralmond.r-universe.dev/RNetica and https://github.com/ralmond/RNetica) which links to a 3rd party library Netica.dll, so RNetica.dll (built from my C code) calls the 3rd party code. The config.win script downloads Netica.dll and moves it into the libs/x64 directory, where it should get loaded when RNetica.dll is loaded. However this is not happening: Here is the relevant portion of the build log (build is on R-universe, but I think it is the same script as CRAN): ``` cp "/d/a/ralmond/ralmond/RNETIC~1.RCH/00_PKG~1/RNetica/src/Netica/Netica_API_5 10/lib64/Netica.dll" "D:/a/ralmond/ralmond/RNetica.Rcheck/00LOCK-RNetica/00new/R Netica/libs/x64" cp "/d/a/ralmond/ralmond/RNETIC~1.RCH/00_PKG~1/RNetica/src/Netica/Netica_API_5 10/lib64/Netica.lib" "D:/a/ralmond/ralmond/RNetica.Rcheck/00LOCK-RNetica/00new/R Netica/libs/x64" C:\rtools43\x86_64-w64-mingw32.static.posix\bin\nm.exe: 'NeticaDLL': No such f ile gcc -shared -s -static-libgcc -o RNetica.dll tmp.def Cases.o Continuous.o Edge s.o Experience.o Inference.o Networks.o Node.o Random.o Registration.o Session.o -L. -LD:/a/ralmond/ralmond/RNetica.Rcheck/00LOCK-RNetica/00new/RNetica/libs/x64 -lNetica -LC:/rtools43/x86_64-w64-mingw32.static.posix/lib/x64 -LC:/rtools43/x8 6_64-w64-mingw32.static.posix/lib -LC:/R/bin/x64 -lR C:\rtools43\x86_64-w64-mingw32.static.posix\bin/ld.exe: internal error: aborti ng at ../../binutils-2.40/ld/ldlang.c:527 in compare_section C:\rtools43\x86_64-w64-mingw32.static.posix\bin/ld.exe: please report this bug collect2.exe: error: ld returned 1 exit status ``` A little bit of searching on the internet, indicates that Windows sometimes reports Dll A not found when Dll A needs Dll B and it can't find B. This used to work under older versions of R and the tool chain and I don't think I've changed anything related to the C side of the code. 1) Have the paths changed, so I no longer should be moving the (64 bit version of the) 3rd party DLL to `libs/x64`? 2) Is there something that has changed with the mingw tools (nm.exe and ld.exe) which are changing things? 3) Is there a change on how win32 and win64 variants are handled (I have both 32 and 64 bit copies of the 3rd party DLL, I just need to move them to the right places). Thanks for any enlightenment you can offer, --Russell Almond -- Russell Almond https://ralmond.net/ [[alternative HTML version deleted]] __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
[R-pkg-devel] Adding package with 3rd party library to CRAN: legal question
This is related to my previous technical question, but really is a question about CRAN policy. I'm currently the maintainer of the RNetica package (https://github.com/ralmond/RNetica) which provides an R binding for the C API of the Netica Bayes net library (https://norsys.com). The Netica library is available in both a "student" and "professional" version, the difference being a license code available from Norsys. The student version is sufficiently functional to run the test suite and to be used for educational purposes. Installing the package from github installs the student version, which can be converted to a professional version by purchasing a license code. In the past, Norsys has been fine with me distributing binaries which included the Netica code (but not the license keys), and I'm sure that they would be okay with CRAN distributing student-mode binaries as well. Currently, the source version will download the Netica API kit from the Norsys web site and install Netica (still working out one or two technical glitches). What are the policy issues associated with distributing this package through CRAN? My own code is licensed under the Artistic license (which is CRAN compatible), but the 3rd party shared library has a different license from Netica. Can I distribute RNetica through CRAN or am I stuck with a github-only distribution. Thanks for your assistance. --Russell Almond __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
[R-pkg-devel] Creating a package that links to a 3rd party library: Technical questions
I'm working on a package called RNetica which links to a 3rd party
library: Netica. Netica is available in both static (libnetica.a) and
shared (libnetica.so) version (also Netica.dll for windows, but that is
a separate topic).
I have a version of my package which works very well
(github_install("ralmond/RNetica")), but it requires manually moving the
`libnetica.so` file into someplace like `/usr/local/lib` where R will
find it.
My preferences for how to deal with this are:
1) Figure out a set of flags to send to the linker so that R CMD INSTALL
links against the static rather than the dynamic library.
2) Figure out how to add a line to the makefile so that `libnetica.so`
will be copied to a directory where R will find it. This is a bit
tricky as a user without admin privileges may not have write permission
on the proper directory.
I would be grateful for any suggestions.
--Russell Almond
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] R CMD build with configure-args
The behavior changed somewhere along the line. I think it is related to the version bump from 3.1 to 3.2, but it could have been earlier. It used to be when I wanted to release a version of my package I would run: INSTALL check build INSTALL --build (on each of a Windows and Mac box). to create source and binary tarballs. I was never quite sure if I needed the check syntax or the INSTALL syntax for build. Somewhere along the line it stopped working. I'm not sure exactly when, because I was working around by creating the source tarball on the Windows box where I was using an environmental variable instead of the command line arg. I suspect that what must have happened is that previously build did not try to recompile the package (or used my source directory where the binaries where already in place). Now it does, so it would probably be useful to add the --install-args switch. --Russell On 07/30/2015 04:08 PM, Duncan Murdoch wrote: On 30/07/2015 2:53 PM, Russell Almond wrote: I've got a package that uses C code to link R to a third-party library. I've set up my configure script to take an argument --with-netica which passes the location of the library, header files, c. So I can install the package using: $ R CMD INSTALL RNetica --configure-args='--with-netica=/home/ralmond/software/Netica_API_504/' and check the packge (build the manual) using: $ R CMD check RNetica --install-args=--configure-args='--with-netica=/home/ralmond/software/Netica_API_504/' My problem comes when I want to build a source tarball for distribution. I used to use: $ R CMD build RNetica --install-args=--configure-args='--with-netica=/home/ralmond/software/Netica_API_504/' When you say used to use, when was that? There was no --install-args option in R 3.0. I think there should be, and I can add it, but I'd like to know if it was removed for a reason. Duncan Murdoch but that generates an error: gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -I. -I/src -fpic -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c Cases.c -o Cases.o In file included from Cases.c:18:0: ./RNetica.h:4:20: fatal error: Netica.h: No such file or directory #include Netica.h ^ basically, it the Includes directory is not getting set correctly. If I use the same syntax as the INSTALL (i.e., just configure-args and not the install-args wrapper) I get the same error message. R CMD INSTALL --build creates a binary tarball and not a source. I think used to work correctly under R 3.1. Did I miss something in the release notes? I'm having this problem under Ubuntu 14.4 (I think I'm also having the problem on Mac OS X; I'm not having the problem under Windows, but mostly because I'm using an environmental variable instead of a command line argument to pass the location of the 3rd party library.) Thanks in advance for any help. -- Russell Almond Associate Professor, Measurement Statistics Educational Psychology and Learning Systems 1114 W. Call St. Florida State University Tallahassee, FL 32306 (850) 644-5203 ralm...@fsu.edu http://ralmond.net/ __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] R CMD build with configure-args
Thanks for the suggestion. I do have a configure script (that is what is reading the --with-netica arg) but I didn't think about searching the standard install locations. I can probably use that to work around this problem. However, that doesn't completely solve the problem. If I want to force the configure script to use a non-standard location (say to use an experimental version of the library), I still need to be able to pass configure args to INSTALL and check. I admit the need for doing this with build is a little bit less clear, although it is also unclear why build (which is making a source tarball) needs to compile the C code. --Russell On 07/30/2015 05:11 PM, Dirk Eddelbuettel wrote: Besides what Duncan said, relying on user to supply arguments is pretty bad as it more or less guarantees _any_ automated test will not succeed (for lack of involvement of the sage user). Writing configure scripts feels like yet another painful step, but it really is not that hard if you know a little shell scripting already. You can even do simple things and just loop over a fixed (but hopefully long) list of know locations -- which is what RPostgreSQL does: https://github.com/cran/RPostgreSQL/blob/master/configure.in#L38-L69 Not my proudest moment, but it worked for a few (ten-)thousands of installations and builds ... Dirk -- Russell Almond Associate Professor, Measurement Statistics Educational Psychology and Learning Systems 1114 W. Call St. Florida State University Tallahassee, FL 32306 (850) 644-5203 ralm...@fsu.edu http://ralmond.net/ __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
[R-pkg-devel] R CMD build with configure-args
I've got a package that uses C code to link R to a third-party library. I've set up my configure script to take an argument --with-netica which passes the location of the library, header files, c. So I can install the package using: $ R CMD INSTALL RNetica --configure-args='--with-netica=/home/ralmond/software/Netica_API_504/' and check the packge (build the manual) using: $ R CMD check RNetica --install-args=--configure-args='--with-netica=/home/ralmond/software/Netica_API_504/' My problem comes when I want to build a source tarball for distribution. I used to use: $ R CMD build RNetica --install-args=--configure-args='--with-netica=/home/ralmond/software/Netica_API_504/' but that generates an error: gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -I. -I/src -fpic -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c Cases.c -o Cases.o In file included from Cases.c:18:0: ./RNetica.h:4:20: fatal error: Netica.h: No such file or directory #include Netica.h ^ basically, it the Includes directory is not getting set correctly. If I use the same syntax as the INSTALL (i.e., just configure-args and not the install-args wrapper) I get the same error message. R CMD INSTALL --build creates a binary tarball and not a source. I think used to work correctly under R 3.1. Did I miss something in the release notes? I'm having this problem under Ubuntu 14.4 (I think I'm also having the problem on Mac OS X; I'm not having the problem under Windows, but mostly because I'm using an environmental variable instead of a command line argument to pass the location of the 3rd party library.) Thanks in advance for any help. -- Russell Almond Associate Professor, Measurement Statistics Educational Psychology and Learning Systems 1114 W. Call St. Florida State University Tallahassee, FL 32306 (850) 644-5203 ralm...@fsu.edu http://ralmond.net/ __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
