Hi Viral, Thanks. I have already informed the sysad to check the time sync of the servers to avoid future issues.
I'll try to compile in local disks and get back to you. Yes, I understand it may take some work to create the bin image but highly appreciate the fact that Julia will run now in power architecture. I think it was several months ago when a fellow IBMer from US tried to port it in power machine and had discussion here also but had some issues. It will be cool if Julia will support power architecture in time for 1.0 release... On Fri, Jul 22, 2016 at 7:51 PM, Viral Shah <[email protected]> wrote: > It appears to me that you are building on some kind of a network > filesystem and timestamps are getting messed up. Can you build in /tmp or > somewhere, which may be guaranteed to have local storage and hence avoid > these clock issues? > > If still doesn’t work, I will create some binaries and upload and put up > all the instructions. Since openblas still has some issues to be fixed on > power, I am using an ATLAS build (of which we are using the latest RC), and > packaging it all up is going to be some work. > > > -viral > > > > > On Jul 22, 2016, at 1:50 PM, Paulito Palmes <[email protected]> wrote: > > > > This problem might be specific to Redhat? Unfortunately, I don't have > debian-based Power system running to test. > > > > On Fri, Jul 22, 2016 at 6:40 PM, Paulito Palmes <[email protected]> > wrote: > > Here's the error using Power8: same error which happens during linking. > > > > Can you send me the tarballs of successfully compiled image in Power8? > > > > ----------------------------------------- > > > > LINK usr/lib/libjulia.so.0.5.0 > > make[1]: warning: Clock skew detected. Your build may be incomplete. > > make[1]: Warning: File `../src/julia_version.h' has modification time > 228 s in the future > > CC ui/repl.o > > LINK usr/bin/julia > > make[1]: warning: Clock skew detected. Your build may be incomplete. > > PERL base/pcre_h.jl > > PERL base/errno_h.jl > > PERL base/build_h.jl.phony > > PERL base/fenv_constants.jl > > PERL base/file_constants.jl > > PERL base/uv_constants.jl > > PERL base/version_git.jl.phony > > JULIA usr/lib/julia/inference.ji > > LLVM ERROR: unable to evaluate offset to undefined symbol '' > > make[1]: *** [/home/paulpalm/power8/julia/usr/lib/julia/inference.ji] > Error 1 > > make: *** [julia-inference] Error 2 > > > > > > On Fri, Jul 22, 2016 at 5:01 PM, Paulito Palmes <[email protected]> > wrote: > > It caused a core-dump during linking ;( in Power7 > > > > ... > > CC src/cgmemmgr.o > > LINK usr/lib/libjulia.so.0.5.0 > > CC ui/repl.o > > LINK usr/bin/julia > > PERL base/pcre_h.jl > > PERL base/errno_h.jl > > PERL base/build_h.jl.phony > > PERL base/fenv_constants.jl > > PERL base/file_constants.jl > > PERL base/uv_constants.jl > > PERL base/version_git.jl.phony > > JULIA usr/lib/julia/inference.ji > > /bin/sh: line 1: 28699 Segmentation fault (core dumped) > /home/paulpalm/julia/usr/bin/julia -C native --output-ji > /home/paulpalm/julia/usr/lib/julia/inference.ji --startup-file=no coreimg.jl > > make[1]: *** [/home/paulpalm/julia/usr/lib/julia/inference.ji] Error 139 > > make: *** [julia-inference] Error 2 > > > > > > On Fri, Jul 22, 2016 at 4:48 PM, Paulito Palmes <[email protected]> > wrote: > > I'm compiling in both Power7 and Power8 and not yet finished. I started > with make -j 20 but after some time, it had errors. Then I switch to make > -j 1 but takes time to finish. I hope it will compile in both power7 and > power8. > > > > I'll wait and try to check if I can fix some obvious errors. Otherwise, > I'll ask the tarball so that I can check the difference from what I have > specially which objects were not compiled. > > > > thanks. > > > > On Fri, Jul 22, 2016 at 4:40 PM, Viral Shah <[email protected]> wrote: > > Did it go through? I usually build with make -j 20 on the machines that > have enough cores. Let me know if you still want me to create the tarball. > Using ATLAS makes the whole thing a bit nonstandard for creating > distributions, but I’ll see what I can do if needed. > > > > -viral > > > > > On Jul 22, 2016, at 10:50 AM, Paulito Palmes <[email protected]> > wrote: > > > > > > Ah, forgot to update Make.user.... > > > > > > > > > On Fri, Jul 22, 2016 at 3:50 PM, Paulito Palmes <[email protected]> > wrote: > > > Hi, > > > > > > Got this error using Power7: Linux abc.com 3.10.0-327.4.5.el7.ppc64 > #1 SMP Thu Jan 21 04:12:40 EST 2016 ppc64 ppc64 ppc64 GNU/Linux > > > > > > Red Hat Enterprise Linux Server release 7.2 (Maipo) > > > -------------- > > > make -C build/openblas-12ab1804b6ebcd38b26960d65d254314d8bc33d6/ > CC=gcc FC=gfortran RANLIB=ranlib FFLAGS= -O2 -fPIC TARGET= BINARY= > USE_THREAD=1 GEMM_MULTITHREADING_THRESHOLD=50 NUM_THREADS=8 NO_AFFINITY=1 > DYNAMIC_ARCH=1 MAKE_NB_JOBS=0 > > > dynamic.c: In function ‘support_avx’: > > > dynamic.c:107:3: warning: implicit declaration of function ‘cpuid’ > [-Wimplicit-function-declaration] > > > cpuid(1, &eax, &ebx, &ecx, &edx); > > > ^ > > > dynamic.c:97:3: error: impossible register constraint in ‘asm’ > > > __asm__ __volatile__ > > > ^ > > > make[3]: *** [dynamic.o] Error 1 > > > make[2]: *** [libs] Error 1 > > > \033[33;1m*** Clean the OpenBLAS build with 'make -C deps > clean-openblas'. Rebuild with 'make OPENBLAS_USE_THREAD=0' if OpenBLAS had > trouble linking libpthread.so, and with 'make OPENBLAS_TARGET_ARCH=NEHALEM' > if there were errors building SandyBridge support. Both these options can > also be used simultaneously. ***\033[0m > > > make[1]: *** > [build/openblas-12ab1804b6ebcd38b26960d65d254314d8bc33d6/libopenblas.so] > Error 1 > > > make: *** [julia-deps] Error 2 > > > ---------------------- > > > > > > On Fri, Jul 22, 2016 at 3:10 PM, Paulito Palmes <[email protected]> > wrote: > > > Hi Viral, > > > > > > uname -a is: > > > > Linux abc.com 3.10.0-327.18.2.el7.ppc64le #1 SMP Fri Apr 8 05:10:45 > EDT 2016 ppc64le ppc64le ppc64le GNU/Linux > > > > > > and using Power8: Red Hat Enterprise Linux Server release 7.2 (Maipo) > > > > > > > > > Can you send me the tarball image of Julia compiled in Power8? I would > like to test it if the problem is only during compilation. Since I have no > git access on this machine, I used make -C deps getall to pull the deps > before copying the source files to Power8. It may be possible the deps were > not properly resolved? > > > > > > On Fri, Jul 22, 2016 at 1:58 PM, Viral Shah <[email protected]> wrote: > > > Not sure what to do here. Is the filesystem NFS or something? Perhaps > just try building again with make -j 1. > > > > > > What is uname -a? > > > > > > -viral > > > > > > > > > On Jul 22, 2016 8:52 AM, "Paulito Palmes" <[email protected]> wrote: > > > Hi, > > > > > > I'm compiling using Red Hat Enterprise Linux Server release 7.2 in > Power8 and I got this error: > > > > > > ./julia/usr/lib64/libunwind*.a: No such file or directory > > > > > > > > > the make process didn't create usr/lib64 and the libunwind*.a are in > usr/lib > > > > > > I created the links of libunwind*.a in usr/lib64 > > > > > > but got these errors during compile: > > > > > > make[2]: Warning: File > `/home/paulpalm/julia/deps/srccache/patchelf-0.9/aclocal.m4' has > modification time 293 s in the future > > > > > > cd /home/paulpalm/julia/deps/srccache/patchelf-0.9 && /bin/sh > /home/paulpalm/julia/deps/srccache/patchelf-0.9/build-aux/missing > automake-1.15 --foreign > > > > > > /home/paulpalm/julia/deps/srccache/patchelf-0.9/build-aux/missing: > line 81: automake-1.15: command not found > > > > > > WARNING: 'automake-1.15' is missing on your system. > > > > > > You should only need it if you modified 'Makefile.am' or > > > > > > 'configure.ac' or m4 files included by 'configure.ac'. > > > > > > The 'automake' program is part of the GNU Automake package: > > > > > > <http://www.gnu.org/software/automake> > > > > > > It also requires GNU Autoconf, GNU m4 and Perl in order to > run: > > > > > > <http://www.gnu.org/software/autoconf> > > > > > > <http://www.gnu.org/software/m4/> > > > > > > <http://www.perl.org/> > > > > > > make[2]: *** [julia/deps/srccache/patchelf-0.9/Makefile.in] Error 1 > > > > > > make[1]: *** [build/patchelf-0.9/src/patchelf] Error 2 > > > > > > make: *** [julia-deps] Error 2 > > > > > > > > > I have automake-1.13 while it looks for automake-1.15. > > > > > > On Wednesday, 20 July 2016 20:57:36 UTC+1, Viral Shah wrote: > > > Just a heads up, there is a bug in atlas that causes the linalg/lq > test to fail (thanks Andreas for finding!). The bugfix is going to be in > the next atlas release - 3.10.3 > > > > > > https://sourceforge.net/p/math-atlas/bugs/254/#64f2 > > > > > > Turns out at this moment, both atlas and openblas are not passing > julia’s tests on power. > > > > > > -viral > > > > > > > > > > > > > On Jul 20, 2016, at 3:11 PM, James Fairbanks <[email protected]> > wrote: > > > > > > > > Updating to master and using that Make.user worked for me. > > > > > > > > Success! > > > > [jpf@power8 julia-dev]$ ./julia > > > > _ > > > > _ _ _(_)_ | A fresh approach to technical computing > > > > (_) | (_) (_) | Documentation: http://docs.julialang.org > > > > _ _ _| |_ __ _ | Type "?help" for help. > > > > | | | | | | |/ _` | | > > > > | | |_| | | | (_| | | Version 0.5.0-dev+5549 (2016-07-20 15:47 > UTC) > > > > _/ |\__'_|_|_|\__'_| | Commit 80fcb57 (0 days old master) > > > > |__/ | ppc64le-redhat-linux > > > > > > > > Thanks for going through this with me. Hopefully it will be easier > for the next person. > > > > > > > > On Wednesday, July 20, 2016 at 10:27:27 AM UTC-4, Viral Shah wrote: > > > > > > > > Julia now builds on master. No longer need to disable threading. > Make.user for me is now: > > > > > > > > override USE_SYSTEM_BLAS = 1 > > > > override LIBBLAS = -L/usr/lib64/atlas -lsatlas > > > > override LIBBLASNAME = libsatlas > > > > override USE_BLAS64 = 0 > > > > > > > > -viral > > > > > > > > > > > > > On Jul 19, 2016, at 7:00 PM, Viral Shah <[email protected]> > wrote: > > > > > > > > > > I am testing on CentOS Linux release 7.2.1511 (AltArch) > > > > > > > > > > This is the line I am using: > > > > > export LD_LIBRARY_PATH=/usr/lib64/atlas:$LD_LIBRARY_PATH > > > > > > > > > > Our README says LAPACK >= 3.5, so I suspect the system lapack > won’t work out. Assuming that symbol is present in the libsatlas file, > perhaps try also copying it in julia’s usr/lib. > > > > > > > > > > The other thing is to run make again to make sure that all the > rpath and linking stuff has gone through fine. Or perhaps make cleanall and > make. > > > > > > > > > > -viral > > > > > > > > > > > > > > > > > > > >> On Jul 19, 2016, at 6:55 PM, James Fairbanks <[email protected]> > wrote: > > > > >> > > > > >> For completeness I should say that the julia executable appears > not to have the libjulia.so correctly linked. > > > > >> > > > > >> $ ./julia > > > > >> ./julia: symbol lookup error: > /home/jpf/julia/usr/bin/../lib/liblapack.so: undefined symbol: lsame_ > > > > >> $ ldd ./julia > > > > >> linux-vdso64.so.1 => (0x00003fff9d140000) > > > > >> libjulia.so.0.5 => not found > > > > >> libdl.so.2 => /lib64/libdl.so.2 (0x00003fff9d100000) > > > > >> librt.so.1 => /lib64/power8/librt.so.1 (0x00003fff9d0d0000) > > > > >> libpthread.so.0 => /lib64/power8/libpthread.so.0 > (0x00003fff9d090000) > > > > >> libc.so.6 => /lib64/power8/libc.so.6 (0x00003fff9ceb0000) > > > > >> /lib64/ld64.so.2 (0x00000000251e0000) > > > > >> > > > > >> $ ldd ./usr/lib/libjulia.so > > > > >> linux-vdso64.so.1 => (0x00003fff99680000) > > > > >> libLLVM-3.8.so => /home/jpf/julia/./usr/lib/libLLVM-3.8.so > (0x00003fff97d80000) > > > > >> libdl.so.2 => /lib64/libdl.so.2 (0x00003fff97d40000) > > > > >> librt.so.1 => /lib64/power8/librt.so.1 (0x00003fff97d10000) > > > > >> libpthread.so.0 => /lib64/power8/libpthread.so.0 > (0x00003fff97cd0000) > > > > >> libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00003fff97b50000) > > > > >> libm.so.6 => /lib64/power8/libm.so.6 (0x00003fff97a60000) > > > > >> libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00003fff97a20000) > > > > >> libc.so.6 => /lib64/power8/libc.so.6 (0x00003fff97840000) > > > > >> libz.so.1 => /lib64/libz.so.1 (0x00003fff97800000) > > > > >> /lib64/ld64.so.2 (0x0000000035a00000) > > > > >> > > > > >> On Tuesday, July 19, 2016 at 6:30:34 PM UTC-4, James Fairbanks > wrote: > > > > >> Adding /usr/lib64/atlas was insufficient so I created a symlink > in ln -s /usr/lib/atlas/libsatlas.so.3 /usr/lib/atlas/libsatlas.so > > > > >> This allowed the build process to complete but leads to an error > when running the julia executable. > > > > >> > > > > >> The error is: > > > > >> /home/jpf/julia/usr/bin/julia: symbol lookup error: > /home/jpf/julia/usr/bin/../lib/liblapack.so: undefined symbol: lsame_ > > > > >> > > > > >> Is it a good idea to go with the RHEL ppc64le system lapack > instead of having julia build lapack against system blas? > > > > >> I have this version of lapack available from the system. > > > > >> > > > > >> Available Packages > > > > >> Name : lapack > > > > >> Arch : ppc64le > > > > >> Version : 3.4.2 > > > > >> Release : 5.el7 > > > > >> Size : 4.9 M > > > > >> Repo : rhel-7-for-power-le-rpms/7Server/ppc64le > > > > >> > > > > >> > > > > >> Thanks, > > > > >> James > > > > >> On Tuesday, July 19, 2016 at 4:46:22 PM UTC-4, Viral Shah wrote: > > > > >> > > > > >> Yes - I forgot to mention that. On the machine I am using, I had > to add /usr/lib64/atlas or something like that to LD_LIBRARY_PATH. I can’t > login at the moment for some reason, or else I could have retrieved the > exact line. > > > > >> > > > > >> > > > > >> -viral > > > > >> > > > > >> > > > > >> > > > > >>> On Jul 19, 2016, at 4:12 PM, James Fairbanks <[email protected]> > wrote: > > > > >>> > > > > >>> I set it up like you said and got the following error > > > > >>> > > > > >>> $ make > > > > >>> ... a bunch of stuff successfully building ... > > > > >>> /usr/bin/ld: cannot find -lsatlas > > > > >>> collect2: error: ld returned 1 exit status > > > > >>> make[1]: *** [build/lapack-3.5.0/liblapack.so] Error 1 > > > > >>> make: *** [julia-deps] Error 2 > > > > >>> > > > > >>> Do I need to add something to LD_LIBRARY_PATH in the makefile or > bash environment? > > > > >>> > > > > >>> On Tuesday, July 19, 2016 at 3:59:55 PM UTC-4, Viral Shah wrote: > > > > >>> There is some old ATLAS stuff in there to build atlas that > hasn’t been used for a very long time. We are going to delete it. If we > find this atlas stuff to be generally useful, we can bring it into the > Makefile - but I definitely don’t want to support a source build. > > > > >>> > > > > >>> override USE_SYSTEM_BLAS = 1 > > > > >>> override LIBBLAS = -L/usr/lib64/atlas -lsatlas > > > > >>> override LIBBLASNAME = libsatlas > > > > >>> override USE_BLAS64 = 0 > > > > >>> override JULIA_THREADS := 0 > > > > >>> > > > > >>> Use the vs/ccall-ppc branch that I just pushed, with the above > Make.user. Master still doesn’t work. > > > > >>> > > > > >>> -viral > > > > >>> > > > > >>> > > > > >>>> On Jul 19, 2016, at 3:49 PM, James Fairbanks < > [email protected]> wrote: > > > > >>>> > > > > >>>> You are overriding the SYSTEM_BLAS with ATLAS. There is a > USE_ATLAS option. Sould that work? > > > > >>>> > > > > >>>> the relevant part from Make.inc is > > > > >>>> > > > > >>>> USE_ATLAS := 0 > > > > >>>> ATLAS_LIBDIR := $(build_libdir) > > > > >>>> # or ATLAS_LIBDIR := /path/to/atlas > > > > >>>> > > > > >>>> > > > > >>>> > > > > >>>> On Saturday, July 9, 2016 at 2:07:38 AM UTC-4, Viral Shah wrote: > > > > >>>> The current master now seems to be in good shape for Power, for > those interested in trying it out. OpenBLAS is still working out a few > bugs, but in the meanwhile, I was able to successfully link against Atlas > using the following Make.user: > > > > >>>> > > > > >>>> override USE_SYSTEM_BLAS = 1 > > > > >>>> override LIBBLAS = -L/usr/lib64/atlas -lsatlas > > > > >>>> override LIBBLASNAME = libsatlas > > > > >>>> override USE_BLAS64 = 0 > > > > >>>> > > > > >>>> Apart from multi-threading, all the other tests passed. > > > > >>>> > > > > >>>> -viral > > > > >>>> > > > > >>> > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
