Re: WIP: Update of math/py-numpy to 1.16.5

2020-04-01 Thread Stuart Henderson
On 2020/04/01 21:30, Paul Irofti wrote:
> Here is a final diff that incorporates jca's patch and sthen's. OK?

OK with me.

> 
> Index: Makefile
> ===
> RCS file: /cvs/ports/math/py-numpy/Makefile,v
> retrieving revision 1.57
> diff -u -p -u -p -r1.57 Makefile
> --- Makefile  10 Mar 2020 03:31:45 -  1.57
> +++ Makefile  1 Apr 2020 18:28:16 -
> @@ -2,11 +2,10 @@
>  
>  COMMENT= fast array and numeric programming library for Python
>  
> -MODPY_EGG_VERSION=   1.14.6
> +MODPY_EGG_VERSION=   1.16.5
>  DISTNAME=numpy-${MODPY_EGG_VERSION}
>  PKGNAME= py-${DISTNAME}
>  EXTRACT_SUFX=.zip
> -REVISION=2
>  
>  CATEGORIES=  math devel
>  
> @@ -15,7 +14,8 @@ HOMEPAGE=   https://www.numpy.org/
>  # BSD
>  PERMIT_PACKAGE = Yes
>  
> -WANTLIB= blas lapack m pthread ${MODFORTRAN_WANTLIB} 
> ${MODPY_WANTLIB}
> +WANTLIB += ${MODFORTRAN_WANTLIB} ${MODPY_WANTLIB}
> +WANTLIB += blas cblas lapack m pthread
>  
>  MODULES= lang/python \
>   fortran
> @@ -27,13 +27,14 @@ MODPY_PI =Yes
>  MODPY_SETUPTOOLS=Yes
>  MODPY_DISTUTILS_BUILDARGS = --fcompiler=gnu95
>  
> -TEST_DEPENDS=devel/py-nose${MODPY_FLAVOR} \
> - devel/py-tz${MODPY_FLAVOR}
> -
> +TEST_DEPENDS=${FULLPKGNAME}:${FULLPKGPATH} \
> + devel/py-test${MODPY_FLAVOR} \
> + devel/py-tz${MODPY_FLAVOR}
>  MODFORTRAN_COMPILER =gfortran
>  
>  BUILD_DEPENDS = ${MODFORTRAN_BUILD_DEPENDS}
> -LIB_DEPENDS=math/lapack \
> +LIB_DEPENDS= math/cblas \
> + math/lapack \
>   ${MODFORTRAN_LIB_DEPENDS}
>  
>  # Cython 0.28.* uses __attribute__((optimize("Os"))) which
> @@ -54,7 +55,7 @@ pre-build:
>   cp -f ${WRKSRC}/numpy/distutils/site.cfg ${WRKSRC}/site.cfg
>  
>  do-test:
> - cd ${WRKDIR} && ${MAKE_ENV} PYTHONPATH=${WRKINST}${MODPY_SITEPKG} 
> ${MODPY_BIN} -c \
> + cd ${WRKDIR} && ${MAKE_ENV} ${MODPY_BIN} -c \
>   'import numpy ; numpy.test()'
>  
>  .include 
> Index: distinfo
> ===
> RCS file: /cvs/ports/math/py-numpy/distinfo,v
> retrieving revision 1.13
> diff -u -p -u -p -r1.13 distinfo
> --- distinfo  26 Oct 2018 21:09:31 -  1.13
> +++ distinfo  1 Apr 2020 18:28:16 -
> @@ -1,2 +1,2 @@
> -SHA256 (numpy-1.14.6.zip) = ElDt9vbEPh14I/CWdBa8GCWLsnHcU2KY6w6gCp5FuAo=
> -SIZE (numpy-1.14.6.zip) = 4906096
> +SHA256 (numpy-1.16.5.zip) = i7RS2U6WSzEiBbDeEjjdcgnaRSNDZTqyFLXWgXgOegw=
> +SIZE (numpy-1.16.5.zip) = 5138208
> Index: patches/patch-numpy_core_include_numpy_npy_common_h
> ===
> RCS file: 
> /cvs/ports/math/py-numpy/patches/patch-numpy_core_include_numpy_npy_common_h,v
> retrieving revision 1.6
> diff -u -p -u -p -r1.6 patch-numpy_core_include_numpy_npy_common_h
> --- patches/patch-numpy_core_include_numpy_npy_common_h   30 Jun 2018 
> 21:49:33 -  1.6
> +++ patches/patch-numpy_core_include_numpy_npy_common_h   1 Apr 2020 
> 18:28:16 -
> @@ -1,13 +1,8 @@
>  $OpenBSD: patch-numpy_core_include_numpy_npy_common_h,v 1.6 2018/06/30 
> 21:49:33 daniel Exp $
>  
> -(1) optimize function attributes were added in gcc 4.4. Apply a fix for the 
> warnings. But
> -the next release of numpy should fix this in a better way. See:
> -https://github.com/numpy/numpy/issues/5451
> +XXX recheck powerpc, is this still needed?
>  
> -Gets rid of these warnings from the build:
> -numpy/core/src/multiarray/lowlevel_strided_loops.c.src:120: warning: 
> 'optimize' attribute directive ignored
> -
> -(2) py-numpy only checks for expl to determine whether extended-precision
> +py-numpy only checks for expl to determine whether extended-precision
>  support is present.  since we don't have it yet;  it implements
>  it's own.  however, on alpha, powerpc, it declared functions with
>  types that conflict with C99 (double for *l), therefore failed.
> @@ -15,36 +10,7 @@ types that conflict with C99 (double for
>  Index: numpy/core/include/numpy/npy_common.h
>  --- numpy/core/include/numpy/npy_common.h.orig
>  +++ numpy/core/include/numpy/npy_common.h
> -@@ -20,6 +20,7 @@
> - #endif
> - #endif
> - 
> -+#if defined(__GNUC__) && (__GNUC__ > 4 || (__GNUC__ == 4 && (__GNUC_MINOR__ 
> > 4)))
> - /*
> -  * gcc does not unroll even with -O3
> -  * use with care, unrolling on modern cpus rarely speeds things up
> -@@ -30,13 +31,20 @@
> - #else
> - #define NPY_GCC_UNROLL_LOOPS
> - #endif
> -+#else
> -+#define NPY_GCC_UNROLL_LOOPS
> -+#endif
> - 
> -+#if defined(__GNUC__) && (__GNUC__ > 4 || (__GNUC__ == 4 && (__GNUC_MINOR__ 
> > 4)))
> - /* highest gcc optimization level, enabled autovectorizer */
> - #ifdef HAVE_ATTRIBUTE_OPTIMIZE_OPT_3
> - #define NPY_GCC_OPT_3 

Re: WIP: Update of math/py-numpy to 1.16.5

2020-04-01 Thread Paul Irofti
On Sun, Mar 29, 2020 at 03:38:33PM +0100, Stuart Henderson wrote:
> On 2020/03/29 11:49, Christian Weisgerber wrote:
> > On 2020-03-28, Stuart Henderson  wrote:
> > 
> > >> > Can somenoe commit the numpy update as well then?
> > >> 
> > >> I am about to start a test bulk build on i386 with this.
> > >
> > > No build problems relating to the numpy update.
> > 
> > Meanwhile, the existing port (1.14.6p2) errored out in my latest bulk
> > build:
> > 
> > numpy/core/src/multiarray/temp_elide.c:87:10: fatal error: execinfo.h: No 
> > such file or directory
> >  #include 
> >   ^~~~
> > compilation terminated.
> > 
> > Please check that the updated port doesn't haphazardly pick up
> > execinfo too.
> > 
> > -- 
> > Christian "naddy" Weisgerber  na...@mips.inka.de
> > 
> 
> It will need this or an alternative. (Unless we can either get the
> backtrace functions into libc, or remove the libexecinfo port ;-)

Here is a final diff that incorporates jca's patch and sthen's. OK?


Index: Makefile
===
RCS file: /cvs/ports/math/py-numpy/Makefile,v
retrieving revision 1.57
diff -u -p -u -p -r1.57 Makefile
--- Makefile10 Mar 2020 03:31:45 -  1.57
+++ Makefile1 Apr 2020 18:28:16 -
@@ -2,11 +2,10 @@
 
 COMMENT=   fast array and numeric programming library for Python
 
-MODPY_EGG_VERSION= 1.14.6
+MODPY_EGG_VERSION= 1.16.5
 DISTNAME=  numpy-${MODPY_EGG_VERSION}
 PKGNAME=   py-${DISTNAME}
 EXTRACT_SUFX=  .zip
-REVISION=  2
 
 CATEGORIES=math devel
 
@@ -15,7 +14,8 @@ HOMEPAGE= https://www.numpy.org/
 # BSD
 PERMIT_PACKAGE =   Yes
 
-WANTLIB=   blas lapack m pthread ${MODFORTRAN_WANTLIB} 
${MODPY_WANTLIB}
+WANTLIB += ${MODFORTRAN_WANTLIB} ${MODPY_WANTLIB}
+WANTLIB += blas cblas lapack m pthread
 
 MODULES=   lang/python \
fortran
@@ -27,13 +27,14 @@ MODPY_PI =  Yes
 MODPY_SETUPTOOLS=  Yes
 MODPY_DISTUTILS_BUILDARGS = --fcompiler=gnu95
 
-TEST_DEPENDS=  devel/py-nose${MODPY_FLAVOR} \
-   devel/py-tz${MODPY_FLAVOR}
-
+TEST_DEPENDS=  ${FULLPKGNAME}:${FULLPKGPATH} \
+   devel/py-test${MODPY_FLAVOR} \
+   devel/py-tz${MODPY_FLAVOR}
 MODFORTRAN_COMPILER =  gfortran
 
 BUILD_DEPENDS = ${MODFORTRAN_BUILD_DEPENDS}
-LIB_DEPENDS=math/lapack \
+LIB_DEPENDS=   math/cblas \
+   math/lapack \
${MODFORTRAN_LIB_DEPENDS}
 
 # Cython 0.28.* uses __attribute__((optimize("Os"))) which
@@ -54,7 +55,7 @@ pre-build:
cp -f ${WRKSRC}/numpy/distutils/site.cfg ${WRKSRC}/site.cfg
 
 do-test:
-   cd ${WRKDIR} && ${MAKE_ENV} PYTHONPATH=${WRKINST}${MODPY_SITEPKG} 
${MODPY_BIN} -c \
+   cd ${WRKDIR} && ${MAKE_ENV} ${MODPY_BIN} -c \
'import numpy ; numpy.test()'
 
 .include 
Index: distinfo
===
RCS file: /cvs/ports/math/py-numpy/distinfo,v
retrieving revision 1.13
diff -u -p -u -p -r1.13 distinfo
--- distinfo26 Oct 2018 21:09:31 -  1.13
+++ distinfo1 Apr 2020 18:28:16 -
@@ -1,2 +1,2 @@
-SHA256 (numpy-1.14.6.zip) = ElDt9vbEPh14I/CWdBa8GCWLsnHcU2KY6w6gCp5FuAo=
-SIZE (numpy-1.14.6.zip) = 4906096
+SHA256 (numpy-1.16.5.zip) = i7RS2U6WSzEiBbDeEjjdcgnaRSNDZTqyFLXWgXgOegw=
+SIZE (numpy-1.16.5.zip) = 5138208
Index: patches/patch-numpy_core_include_numpy_npy_common_h
===
RCS file: 
/cvs/ports/math/py-numpy/patches/patch-numpy_core_include_numpy_npy_common_h,v
retrieving revision 1.6
diff -u -p -u -p -r1.6 patch-numpy_core_include_numpy_npy_common_h
--- patches/patch-numpy_core_include_numpy_npy_common_h 30 Jun 2018 21:49:33 
-  1.6
+++ patches/patch-numpy_core_include_numpy_npy_common_h 1 Apr 2020 18:28:16 
-
@@ -1,13 +1,8 @@
 $OpenBSD: patch-numpy_core_include_numpy_npy_common_h,v 1.6 2018/06/30 
21:49:33 daniel Exp $
 
-(1) optimize function attributes were added in gcc 4.4. Apply a fix for the 
warnings. But
-the next release of numpy should fix this in a better way. See:
-https://github.com/numpy/numpy/issues/5451
+XXX recheck powerpc, is this still needed?
 
-Gets rid of these warnings from the build:
-numpy/core/src/multiarray/lowlevel_strided_loops.c.src:120: warning: 
'optimize' attribute directive ignored
-
-(2) py-numpy only checks for expl to determine whether extended-precision
+py-numpy only checks for expl to determine whether extended-precision
 support is present.  since we don't have it yet;  it implements
 it's own.  however, on alpha, powerpc, it declared functions with
 types that conflict with C99 (double for *l), therefore failed.
@@ -15,36 +10,7 @@ types that conflict with C99 (double for
 Index: numpy/core/include/numpy/npy_common.h
 --- 

Re: WIP: Update of math/py-numpy to 1.16.5

2020-03-29 Thread Antoine Jacoutot
On Sun, Mar 29, 2020 at 03:38:33PM +0100, Stuart Henderson wrote:
> On 2020/03/29 11:49, Christian Weisgerber wrote:
> > On 2020-03-28, Stuart Henderson  wrote:
> > 
> > >> > Can somenoe commit the numpy update as well then?
> > >> 
> > >> I am about to start a test bulk build on i386 with this.
> > >
> > > No build problems relating to the numpy update.
> > 
> > Meanwhile, the existing port (1.14.6p2) errored out in my latest bulk
> > build:
> > 
> > numpy/core/src/multiarray/temp_elide.c:87:10: fatal error: execinfo.h: No 
> > such file or directory
> >  #include 
> >   ^~~~
> > compilation terminated.
> > 
> > Please check that the updated port doesn't haphazardly pick up
> > execinfo too.
> > 
> > -- 
> > Christian "naddy" Weisgerber  na...@mips.inka.de
> > 
> 
> It will need this or an alternative. (Unless we can either get the
> backtrace functions into libc, or remove the libexecinfo port ;-)

I would encourage any solution.
What we have now is a total mess :-(

> Index: patches/patch-numpy_core_setup_common_py
> ===
> RCS file: patches/patch-numpy_core_setup_common_py
> diff -N patches/patch-numpy_core_setup_common_py
> --- /dev/null 1 Jan 1970 00:00:00 -
> +++ patches/patch-numpy_core_setup_common_py  29 Mar 2020 14:36:51 -
> @@ -0,0 +1,14 @@
> +$OpenBSD$
> +
> +Index: numpy/core/setup_common.py
> +--- numpy/core/setup_common.py.orig
>  numpy/core/setup_common.py
> +@@ -111,7 +111,7 @@ OPTIONAL_STDFUNCS = ["expm1", "log1p", "acosh", "asinh
> + "rint", "trunc", "exp2", "log2", "hypot", "atan2", "pow",
> + "copysign", "nextafter", "ftello", "fseeko",
> + "strtoll", "strtoull", "cbrt", "strtold_l", "fallocate",
> +-"backtrace", "madvise"]
> ++"madvise"]
> + 
> + 
> + OPTIONAL_HEADERS = [
> 

-- 
Antoine



Re: WIP: Update of math/py-numpy to 1.16.5

2020-03-29 Thread Stuart Henderson
On 2020/03/29 11:49, Christian Weisgerber wrote:
> On 2020-03-28, Stuart Henderson  wrote:
> 
> >> > Can somenoe commit the numpy update as well then?
> >> 
> >> I am about to start a test bulk build on i386 with this.
> >
> > No build problems relating to the numpy update.
> 
> Meanwhile, the existing port (1.14.6p2) errored out in my latest bulk
> build:
> 
> numpy/core/src/multiarray/temp_elide.c:87:10: fatal error: execinfo.h: No 
> such file or directory
>  #include 
>   ^~~~
> compilation terminated.
> 
> Please check that the updated port doesn't haphazardly pick up
> execinfo too.
> 
> -- 
> Christian "naddy" Weisgerber  na...@mips.inka.de
> 

It will need this or an alternative. (Unless we can either get the
backtrace functions into libc, or remove the libexecinfo port ;-)

Index: patches/patch-numpy_core_setup_common_py
===
RCS file: patches/patch-numpy_core_setup_common_py
diff -N patches/patch-numpy_core_setup_common_py
--- /dev/null   1 Jan 1970 00:00:00 -
+++ patches/patch-numpy_core_setup_common_py29 Mar 2020 14:36:51 -
@@ -0,0 +1,14 @@
+$OpenBSD$
+
+Index: numpy/core/setup_common.py
+--- numpy/core/setup_common.py.orig
 numpy/core/setup_common.py
+@@ -111,7 +111,7 @@ OPTIONAL_STDFUNCS = ["expm1", "log1p", "acosh", "asinh
+ "rint", "trunc", "exp2", "log2", "hypot", "atan2", "pow",
+ "copysign", "nextafter", "ftello", "fseeko",
+ "strtoll", "strtoull", "cbrt", "strtold_l", "fallocate",
+-"backtrace", "madvise"]
++"madvise"]
+ 
+ 
+ OPTIONAL_HEADERS = [



Re: WIP: Update of math/py-numpy to 1.16.5

2020-03-29 Thread Christian Weisgerber
On 2020-03-28, Stuart Henderson  wrote:

>> > Can somenoe commit the numpy update as well then?
>> 
>> I am about to start a test bulk build on i386 with this.
>
> No build problems relating to the numpy update.

Meanwhile, the existing port (1.14.6p2) errored out in my latest bulk
build:

numpy/core/src/multiarray/temp_elide.c:87:10: fatal error: execinfo.h: No such 
file or directory
 #include 
  ^~~~
compilation terminated.

Please check that the updated port doesn't haphazardly pick up
execinfo too.

-- 
Christian "naddy" Weisgerber  na...@mips.inka.de



Re: WIP: Update of math/py-numpy to 1.16.5

2020-03-28 Thread Stuart Henderson
On 2020/03/27 12:16, Stuart Henderson wrote:
> On 2020/03/24 19:30, Paul Irofti wrote:
> > Can somenoe commit the numpy update as well then?
> 
> I am about to start a test bulk build on i386 with this.

No build problems relating to the numpy update.



Re: WIP: Update of math/py-numpy to 1.16.5

2020-03-27 Thread Stuart Henderson
On 2020/03/24 19:30, Paul Irofti wrote:
> Can somenoe commit the numpy update as well then?

I am about to start a test bulk build on i386 with this.



Re: WIP: Update of math/py-numpy to 1.16.5

2020-03-24 Thread Paul Irofti
On Fri, Mar 13, 2020 at 02:40:05PM +0100, Jeremie Courreges-Anglas wrote:
> On Fri, Mar 13 2020, Paul Irofti  wrote:
> > On Wed, Mar 11, 2020 at 08:58:27PM +0100, Martin Reindl wrote:
> >> Am 11.03.20 um 18:53 schrieb Theo Buehler:
> >> > On Wed, Mar 11, 2020 at 04:12:56AM +0100, Jeremie Courreges-Anglas wrote:
> >> >> On Tue, Mar 10 2020, Theo Buehler  wrote:
> >> >>> On Tue, Mar 10, 2020 at 06:35:04PM +0100, Jeremie Courreges-Anglas 
> >> >>> wrote:
> >>  On Mon, Mar 09 2020, Stuart Henderson  wrote:
> >> > On 2020/03/09 10:42, Theo Buehler wrote:
> >> >> On Mon, Jan 13, 2020 at 12:50:32PM +, Stuart Henderson wrote:
> >> >>> 2/3 through a bulk build and I see that this breaks scipy (missing 
> >> >>> symbols,
> >> >>> blas/cblas-related) so needs a bit more work, but I think it's 
> >> >>> generally
> >> >>> along the right lines.
> >> >>
> >> >> Not sure if this provides any useful clue, but py-numpy doesn't 
> >> >> build at
> >> >> all on sparc64 with this diff, also due to missing blas/cblas 
> >> >> symbols:
> >> >
> >> > You'll probably see the same on amd64 with USE_LLD=no.
> >> 
> >>  I managed to build scipy with no changes on amd64, so I'm not sure 
> >>  what
> >>  the problem is on this arch (did not try with USE_LLD=No).
> >> 
> >>  However I took a look at the issue reported by tb on sparc64.
> >> 
> >>  --8<--
> >>  creating /tmp/tmpKcZ0cd/tmp
> >>  creating /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd
> >>  compile options: '-I/usr/local/include -I/usr/include -c'
> >>  cc: /tmp/tmpKcZ0cd/source.c
> >>  cc /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd/source.o -L/usr/local/lib -lcblas -o 
> >>  /tmp/tmpKcZ0cd/a.out
> >>  /usr/local/lib/libcblas.so.1.0: undefined reference to `ztbsv_'
> >>  /usr/local/lib/libcblas.so.1.0: undefined reference to `dasum_'
> >> 
> >>  [...]
> >> 
> >>  /usr/local/lib/libcblas.so.1.0: undefined reference to `zsymm_'
> >>  /usr/local/lib/libcblas.so.1.0: undefined reference to `ztrsm_'
> >>  /usr/local/lib/libcblas.so.1.0: undefined reference to `sswap_'
> >>  collect2: error: ld returned 1 exit status
> >>  cc /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd/source.o -L/usr/local/lib -lblas -o 
> >>  /tmp/tmpKcZ0cd/a.out
> >>  /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd/source.o: In function `main':
> >>  source.c:(.text.startup+0xdc): undefined reference to `cblas_ddot'
> >>  collect2: error: ld returned 1 exit status
> >>  -->8--
> >> 
> >>  libcblas.so doesn't depend on libblas.so so missing symbols are to be
> >>  expected if one links with -lcblas instead of -lcblas -lblas.  The
> >>  second linking test fails because libblas.so doesn't provide cblas
> >>  symbols.
> >> >>>
> >> >>> Thanks, this makes sense. But why does this work with ld.lld?
> >> >>
> >> >> ld.lld doesn't bother checking that all symbols in libcblas.so can be
> >> >> resolved, ld.bfd does.  This means that if you link against a library
> >> >> that references a bogus symbol or lacks some library interdependency
> >> >> (DT_NEEDED) you only get a crash at run time.
> >> >>
> >> >> On amd64, using the testcase from numpy:
> >> >>
> >> >> --8<--
> >> >> russell /tmp$ cat r.c
> >> >> #include 
> >> >> int main(int argc, const char *argv[])
> >> >> {
> >> >> double a[4] = {1,2,3,4};
> >> >> double b[4] = {5,6,7,8};
> >> >> return cblas_ddot(4, a, 1, b, 1) > 10;
> >> >> }
> >> >> russell /tmp$ cc -I/usr/local/include r.c -L/usr/local/lib -lcblas
> >> >> russell /tmp$ ./a.out
> >> >> a.out:/usr/local/lib/libcblas.so.1.0: undefined symbol 'ddot_'
> >> >> ld.so: a.out: lazy binding failed!
> >> >> Killed
> >> >> -->8--
> >> >>
> >> >> I suspect Stuart hit a similar problem with this numpy update and scipy.
> >> >>
> >> >> Using -fuse-ld=bfd in the testcase above would result in the same errors
> >> >> as in your log.
> >> > 
> >> > I see. Thank you very much for your explanations.
> >> > 
> >> > FWIW your cblas diff is ok tb (also tested on macppc).
> >> > 
> >> 
> >> Also tested on arm64, with numpy-1.16.6:
> >> 42 failed, 7268 passed, 93 skipped, 168 deselected, 12 xfailed, 1
> >> xpassed, 1 warnings
> >> 
> >> I think this is ready to go into the tree with the cblas diff on top?
> >> The update to 1.16.6 is straightforward if you want to stick to 1.16.5 now.
> >
> > Can you please commit this? Thank you!
> 
> By popular demand I have committed the cblas diff.  I haven't looked
> closely at the numpy update, you folks probably know better.

Great, thank you!

Can somenoe commit the numpy update as well then?

Paul



Re: WIP: Update of math/py-numpy to 1.16.5

2020-03-13 Thread Jeremie Courreges-Anglas
On Fri, Mar 13 2020, Paul Irofti  wrote:
> On Wed, Mar 11, 2020 at 08:58:27PM +0100, Martin Reindl wrote:
>> Am 11.03.20 um 18:53 schrieb Theo Buehler:
>> > On Wed, Mar 11, 2020 at 04:12:56AM +0100, Jeremie Courreges-Anglas wrote:
>> >> On Tue, Mar 10 2020, Theo Buehler  wrote:
>> >>> On Tue, Mar 10, 2020 at 06:35:04PM +0100, Jeremie Courreges-Anglas wrote:
>>  On Mon, Mar 09 2020, Stuart Henderson  wrote:
>> > On 2020/03/09 10:42, Theo Buehler wrote:
>> >> On Mon, Jan 13, 2020 at 12:50:32PM +, Stuart Henderson wrote:
>> >>> 2/3 through a bulk build and I see that this breaks scipy (missing 
>> >>> symbols,
>> >>> blas/cblas-related) so needs a bit more work, but I think it's 
>> >>> generally
>> >>> along the right lines.
>> >>
>> >> Not sure if this provides any useful clue, but py-numpy doesn't build 
>> >> at
>> >> all on sparc64 with this diff, also due to missing blas/cblas symbols:
>> >
>> > You'll probably see the same on amd64 with USE_LLD=no.
>> 
>>  I managed to build scipy with no changes on amd64, so I'm not sure what
>>  the problem is on this arch (did not try with USE_LLD=No).
>> 
>>  However I took a look at the issue reported by tb on sparc64.
>> 
>>  --8<--
>>  creating /tmp/tmpKcZ0cd/tmp
>>  creating /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd
>>  compile options: '-I/usr/local/include -I/usr/include -c'
>>  cc: /tmp/tmpKcZ0cd/source.c
>>  cc /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd/source.o -L/usr/local/lib -lcblas -o 
>>  /tmp/tmpKcZ0cd/a.out
>>  /usr/local/lib/libcblas.so.1.0: undefined reference to `ztbsv_'
>>  /usr/local/lib/libcblas.so.1.0: undefined reference to `dasum_'
>> 
>>  [...]
>> 
>>  /usr/local/lib/libcblas.so.1.0: undefined reference to `zsymm_'
>>  /usr/local/lib/libcblas.so.1.0: undefined reference to `ztrsm_'
>>  /usr/local/lib/libcblas.so.1.0: undefined reference to `sswap_'
>>  collect2: error: ld returned 1 exit status
>>  cc /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd/source.o -L/usr/local/lib -lblas -o 
>>  /tmp/tmpKcZ0cd/a.out
>>  /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd/source.o: In function `main':
>>  source.c:(.text.startup+0xdc): undefined reference to `cblas_ddot'
>>  collect2: error: ld returned 1 exit status
>>  -->8--
>> 
>>  libcblas.so doesn't depend on libblas.so so missing symbols are to be
>>  expected if one links with -lcblas instead of -lcblas -lblas.  The
>>  second linking test fails because libblas.so doesn't provide cblas
>>  symbols.
>> >>>
>> >>> Thanks, this makes sense. But why does this work with ld.lld?
>> >>
>> >> ld.lld doesn't bother checking that all symbols in libcblas.so can be
>> >> resolved, ld.bfd does.  This means that if you link against a library
>> >> that references a bogus symbol or lacks some library interdependency
>> >> (DT_NEEDED) you only get a crash at run time.
>> >>
>> >> On amd64, using the testcase from numpy:
>> >>
>> >> --8<--
>> >> russell /tmp$ cat r.c
>> >> #include 
>> >> int main(int argc, const char *argv[])
>> >> {
>> >> double a[4] = {1,2,3,4};
>> >> double b[4] = {5,6,7,8};
>> >> return cblas_ddot(4, a, 1, b, 1) > 10;
>> >> }
>> >> russell /tmp$ cc -I/usr/local/include r.c -L/usr/local/lib -lcblas
>> >> russell /tmp$ ./a.out
>> >> a.out:/usr/local/lib/libcblas.so.1.0: undefined symbol 'ddot_'
>> >> ld.so: a.out: lazy binding failed!
>> >> Killed
>> >> -->8--
>> >>
>> >> I suspect Stuart hit a similar problem with this numpy update and scipy.
>> >>
>> >> Using -fuse-ld=bfd in the testcase above would result in the same errors
>> >> as in your log.
>> > 
>> > I see. Thank you very much for your explanations.
>> > 
>> > FWIW your cblas diff is ok tb (also tested on macppc).
>> > 
>> 
>> Also tested on arm64, with numpy-1.16.6:
>> 42 failed, 7268 passed, 93 skipped, 168 deselected, 12 xfailed, 1
>> xpassed, 1 warnings
>> 
>> I think this is ready to go into the tree with the cblas diff on top?
>> The update to 1.16.6 is straightforward if you want to stick to 1.16.5 now.
>
> Can you please commit this? Thank you!

By popular demand I have committed the cblas diff.  I haven't looked
closely at the numpy update, you folks probably know better.

-- 
jca | PGP : 0x1524E7EE / 5135 92C1 AD36 5293 2BDF  DDCC 0DFA 74AE 1524 E7EE



Re: WIP: Update of math/py-numpy to 1.16.5

2020-03-13 Thread Paul Irofti
On Wed, Mar 11, 2020 at 08:58:27PM +0100, Martin Reindl wrote:
> Am 11.03.20 um 18:53 schrieb Theo Buehler:
> > On Wed, Mar 11, 2020 at 04:12:56AM +0100, Jeremie Courreges-Anglas wrote:
> >> On Tue, Mar 10 2020, Theo Buehler  wrote:
> >>> On Tue, Mar 10, 2020 at 06:35:04PM +0100, Jeremie Courreges-Anglas wrote:
>  On Mon, Mar 09 2020, Stuart Henderson  wrote:
> > On 2020/03/09 10:42, Theo Buehler wrote:
> >> On Mon, Jan 13, 2020 at 12:50:32PM +, Stuart Henderson wrote:
> >>> 2/3 through a bulk build and I see that this breaks scipy (missing 
> >>> symbols,
> >>> blas/cblas-related) so needs a bit more work, but I think it's 
> >>> generally
> >>> along the right lines.
> >>
> >> Not sure if this provides any useful clue, but py-numpy doesn't build 
> >> at
> >> all on sparc64 with this diff, also due to missing blas/cblas symbols:
> >
> > You'll probably see the same on amd64 with USE_LLD=no.
> 
>  I managed to build scipy with no changes on amd64, so I'm not sure what
>  the problem is on this arch (did not try with USE_LLD=No).
> 
>  However I took a look at the issue reported by tb on sparc64.
> 
>  --8<--
>  creating /tmp/tmpKcZ0cd/tmp
>  creating /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd
>  compile options: '-I/usr/local/include -I/usr/include -c'
>  cc: /tmp/tmpKcZ0cd/source.c
>  cc /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd/source.o -L/usr/local/lib -lcblas -o 
>  /tmp/tmpKcZ0cd/a.out
>  /usr/local/lib/libcblas.so.1.0: undefined reference to `ztbsv_'
>  /usr/local/lib/libcblas.so.1.0: undefined reference to `dasum_'
> 
>  [...]
> 
>  /usr/local/lib/libcblas.so.1.0: undefined reference to `zsymm_'
>  /usr/local/lib/libcblas.so.1.0: undefined reference to `ztrsm_'
>  /usr/local/lib/libcblas.so.1.0: undefined reference to `sswap_'
>  collect2: error: ld returned 1 exit status
>  cc /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd/source.o -L/usr/local/lib -lblas -o 
>  /tmp/tmpKcZ0cd/a.out
>  /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd/source.o: In function `main':
>  source.c:(.text.startup+0xdc): undefined reference to `cblas_ddot'
>  collect2: error: ld returned 1 exit status
>  -->8--
> 
>  libcblas.so doesn't depend on libblas.so so missing symbols are to be
>  expected if one links with -lcblas instead of -lcblas -lblas.  The
>  second linking test fails because libblas.so doesn't provide cblas
>  symbols.
> >>>
> >>> Thanks, this makes sense. But why does this work with ld.lld?
> >>
> >> ld.lld doesn't bother checking that all symbols in libcblas.so can be
> >> resolved, ld.bfd does.  This means that if you link against a library
> >> that references a bogus symbol or lacks some library interdependency
> >> (DT_NEEDED) you only get a crash at run time.
> >>
> >> On amd64, using the testcase from numpy:
> >>
> >> --8<--
> >> russell /tmp$ cat r.c
> >> #include 
> >> int main(int argc, const char *argv[])
> >> {
> >> double a[4] = {1,2,3,4};
> >> double b[4] = {5,6,7,8};
> >> return cblas_ddot(4, a, 1, b, 1) > 10;
> >> }
> >> russell /tmp$ cc -I/usr/local/include r.c -L/usr/local/lib -lcblas
> >> russell /tmp$ ./a.out
> >> a.out:/usr/local/lib/libcblas.so.1.0: undefined symbol 'ddot_'
> >> ld.so: a.out: lazy binding failed!
> >> Killed
> >> -->8--
> >>
> >> I suspect Stuart hit a similar problem with this numpy update and scipy.
> >>
> >> Using -fuse-ld=bfd in the testcase above would result in the same errors
> >> as in your log.
> > 
> > I see. Thank you very much for your explanations.
> > 
> > FWIW your cblas diff is ok tb (also tested on macppc).
> > 
> 
> Also tested on arm64, with numpy-1.16.6:
> 42 failed, 7268 passed, 93 skipped, 168 deselected, 12 xfailed, 1
> xpassed, 1 warnings
> 
> I think this is ready to go into the tree with the cblas diff on top?
> The update to 1.16.6 is straightforward if you want to stick to 1.16.5 now.

Can you please commit this? Thank you!



Re: WIP: Update of math/py-numpy to 1.16.5

2020-03-11 Thread Theo Buehler
On Fri, Jan 10, 2020 at 04:00:10PM +, Stuart Henderson wrote:
[...]
> I also removed part of patch-numpy_core_include_numpy_npy_common_h
> that was dealing with gcc-<4.4 which we don't have to worry about
> (the gfortran module uses gcc for C as well as Fortran, so it will
> always be built with 4.4+ for us). I left the second part in but
> we could do with testing powerpc with that file removed completely
> (I added an XXX).

I believe this second part is still needed. Without it, there are lots
of warnings of this type:

/usr/include/math.h:425:32: note: expected 'long double *' but argument is of 
type 'npy_longdouble *' {aka 'double *'}


On Tue, Mar 10, 2020 at 06:41:22PM +0100, Jeremie Courreges-Anglas wrote:
[...]
> Here's an updated diff for numpy-1.16.5, for convenience I decided to
> drop the hard requirements I had on cblas>=1.1 (WANTLIB) /
> math/cblas>=1.0p7 (LIB_DEPENDS).

Somebody with a better knowledge of powerpc should probably take a look:
Both flavors build, but the regress tests abort due to a bus error in
the multiarray code between 18% and 19% in (trace below).  This is new.
The 14.0.6 tests only had a handful (9?) unexpected failures (also with
jca@'s cblas diff).


It's probably unrelated, but there are new warnings from dragon4.c that
are of the same kind as the ones fixed by the patch sthen mentioned, e.g.:

numpy/core/src/multiarray/dragon4.c:3160:1: note: in expansion of macro 
'make_dragon4_typefuncs'
 make_dragon4_typefuncs(LongDouble, npy_longdouble, NPY_LONGDOUBLE_BINFMT_NAME)
 ^~
numpy/core/src/multiarray/dragon4.c:2373:48: note: expected 'npy_float64 *' 
{aka 'double *'} but argument is of type 'npy_longdouble *' {aka 'long double 
*'}
 Dragon4_Scratch *scratch, npy_float64 *value, Dragon4_Options *opt)
   ~^


And here's the start of the trace of the bus error during tests. The
python3 version is almost the same.

#0  0xd79f731c in _contig_cast_cfloat_to_cdouble ()
   from 
/usr/obj/pobj/py-numpy-1.16.5/fake-powerpc/usr/local/lib/python2.7/site-packages/numpy/core/_multiarray_umath.so
#1  0xd797be04 in raw_array_assign_array ()
   from 
/usr/obj/pobj/py-numpy-1.16.5/fake-powerpc/usr/local/lib/python2.7/site-packages/numpy/core/_multiarray_umath.so
#2  0xd797c55c in PyArray_AssignArray ()
   from 
/usr/obj/pobj/py-numpy-1.16.5/fake-powerpc/usr/local/lib/python2.7/site-packages/numpy/core/_multiarray_umath.so
#3  0xd798b2bc in PyArray_CastToType ()
   from 
/usr/obj/pobj/py-numpy-1.16.5/fake-powerpc/usr/local/lib/python2.7/site-packages/numpy/core/_multiarray_umath.so
#4  0xd7aa8908 in PyUFunc_GenericFunction ()
   from 
/usr/obj/pobj/py-numpy-1.16.5/fake-powerpc/usr/local/lib/python2.7/site-packages/numpy/core/_multiarray_umath.so
#5  0xd7aa9648 in ufunc_generic_call ()
   from 
/usr/obj/pobj/py-numpy-1.16.5/fake-powerpc/usr/local/lib/python2.7/site-packages/numpy/core/_multiarray_umath.so
#6  0xa9513200 in PyObject_Call () from /usr/local/lib/libpython2.7.so.0.0
#7  0xa9513ca4 in PyObject_CallFunctionObjArgs () from 
/usr/local/lib/libpython2.7.so.0.0
#8  0xd7a48a00 in PyArray_GenericBinaryFunction ()
   from 
/usr/obj/pobj/py-numpy-1.16.5/fake-powerpc/usr/local/lib/python2.7/site-packages/numpy/core/_multiarray_umath.so
#9  0xd7a49640 in array_add ()
   from 
/usr/obj/pobj/py-numpy-1.16.5/fake-powerpc/usr/local/lib/python2.7/site-packages/numpy/core/_multiarray_umath.so
#10 0xa950cfe4 in binary_op1 () from /usr/local/lib/libpython2.7.so.0.0
...



Re: WIP: Update of math/py-numpy to 1.16.5

2020-03-11 Thread Martin Reindl
Am 11.03.20 um 18:53 schrieb Theo Buehler:
> On Wed, Mar 11, 2020 at 04:12:56AM +0100, Jeremie Courreges-Anglas wrote:
>> On Tue, Mar 10 2020, Theo Buehler  wrote:
>>> On Tue, Mar 10, 2020 at 06:35:04PM +0100, Jeremie Courreges-Anglas wrote:
 On Mon, Mar 09 2020, Stuart Henderson  wrote:
> On 2020/03/09 10:42, Theo Buehler wrote:
>> On Mon, Jan 13, 2020 at 12:50:32PM +, Stuart Henderson wrote:
>>> 2/3 through a bulk build and I see that this breaks scipy (missing 
>>> symbols,
>>> blas/cblas-related) so needs a bit more work, but I think it's generally
>>> along the right lines.
>>
>> Not sure if this provides any useful clue, but py-numpy doesn't build at
>> all on sparc64 with this diff, also due to missing blas/cblas symbols:
>
> You'll probably see the same on amd64 with USE_LLD=no.

 I managed to build scipy with no changes on amd64, so I'm not sure what
 the problem is on this arch (did not try with USE_LLD=No).

 However I took a look at the issue reported by tb on sparc64.

 --8<--
 creating /tmp/tmpKcZ0cd/tmp
 creating /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd
 compile options: '-I/usr/local/include -I/usr/include -c'
 cc: /tmp/tmpKcZ0cd/source.c
 cc /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd/source.o -L/usr/local/lib -lcblas -o 
 /tmp/tmpKcZ0cd/a.out
 /usr/local/lib/libcblas.so.1.0: undefined reference to `ztbsv_'
 /usr/local/lib/libcblas.so.1.0: undefined reference to `dasum_'

 [...]

 /usr/local/lib/libcblas.so.1.0: undefined reference to `zsymm_'
 /usr/local/lib/libcblas.so.1.0: undefined reference to `ztrsm_'
 /usr/local/lib/libcblas.so.1.0: undefined reference to `sswap_'
 collect2: error: ld returned 1 exit status
 cc /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd/source.o -L/usr/local/lib -lblas -o 
 /tmp/tmpKcZ0cd/a.out
 /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd/source.o: In function `main':
 source.c:(.text.startup+0xdc): undefined reference to `cblas_ddot'
 collect2: error: ld returned 1 exit status
 -->8--

 libcblas.so doesn't depend on libblas.so so missing symbols are to be
 expected if one links with -lcblas instead of -lcblas -lblas.  The
 second linking test fails because libblas.so doesn't provide cblas
 symbols.
>>>
>>> Thanks, this makes sense. But why does this work with ld.lld?
>>
>> ld.lld doesn't bother checking that all symbols in libcblas.so can be
>> resolved, ld.bfd does.  This means that if you link against a library
>> that references a bogus symbol or lacks some library interdependency
>> (DT_NEEDED) you only get a crash at run time.
>>
>> On amd64, using the testcase from numpy:
>>
>> --8<--
>> russell /tmp$ cat r.c
>> #include 
>> int main(int argc, const char *argv[])
>> {
>> double a[4] = {1,2,3,4};
>> double b[4] = {5,6,7,8};
>> return cblas_ddot(4, a, 1, b, 1) > 10;
>> }
>> russell /tmp$ cc -I/usr/local/include r.c -L/usr/local/lib -lcblas
>> russell /tmp$ ./a.out
>> a.out:/usr/local/lib/libcblas.so.1.0: undefined symbol 'ddot_'
>> ld.so: a.out: lazy binding failed!
>> Killed
>> -->8--
>>
>> I suspect Stuart hit a similar problem with this numpy update and scipy.
>>
>> Using -fuse-ld=bfd in the testcase above would result in the same errors
>> as in your log.
> 
> I see. Thank you very much for your explanations.
> 
> FWIW your cblas diff is ok tb (also tested on macppc).
> 

Also tested on arm64, with numpy-1.16.6:
42 failed, 7268 passed, 93 skipped, 168 deselected, 12 xfailed, 1
xpassed, 1 warnings

I think this is ready to go into the tree with the cblas diff on top?
The update to 1.16.6 is straightforward if you want to stick to 1.16.5 now.

-m



Re: WIP: Update of math/py-numpy to 1.16.5

2020-03-11 Thread Theo Buehler
On Wed, Mar 11, 2020 at 04:12:56AM +0100, Jeremie Courreges-Anglas wrote:
> On Tue, Mar 10 2020, Theo Buehler  wrote:
> > On Tue, Mar 10, 2020 at 06:35:04PM +0100, Jeremie Courreges-Anglas wrote:
> >> On Mon, Mar 09 2020, Stuart Henderson  wrote:
> >> > On 2020/03/09 10:42, Theo Buehler wrote:
> >> >> On Mon, Jan 13, 2020 at 12:50:32PM +, Stuart Henderson wrote:
> >> >> > 2/3 through a bulk build and I see that this breaks scipy (missing 
> >> >> > symbols,
> >> >> > blas/cblas-related) so needs a bit more work, but I think it's 
> >> >> > generally
> >> >> > along the right lines.
> >> >> 
> >> >> Not sure if this provides any useful clue, but py-numpy doesn't build at
> >> >> all on sparc64 with this diff, also due to missing blas/cblas symbols:
> >> >
> >> > You'll probably see the same on amd64 with USE_LLD=no.
> >> 
> >> I managed to build scipy with no changes on amd64, so I'm not sure what
> >> the problem is on this arch (did not try with USE_LLD=No).
> >> 
> >> However I took a look at the issue reported by tb on sparc64.
> >> 
> >> --8<--
> >> creating /tmp/tmpKcZ0cd/tmp
> >> creating /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd
> >> compile options: '-I/usr/local/include -I/usr/include -c'
> >> cc: /tmp/tmpKcZ0cd/source.c
> >> cc /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd/source.o -L/usr/local/lib -lcblas -o 
> >> /tmp/tmpKcZ0cd/a.out
> >> /usr/local/lib/libcblas.so.1.0: undefined reference to `ztbsv_'
> >> /usr/local/lib/libcblas.so.1.0: undefined reference to `dasum_'
> >> 
> >> [...]
> >> 
> >> /usr/local/lib/libcblas.so.1.0: undefined reference to `zsymm_'
> >> /usr/local/lib/libcblas.so.1.0: undefined reference to `ztrsm_'
> >> /usr/local/lib/libcblas.so.1.0: undefined reference to `sswap_'
> >> collect2: error: ld returned 1 exit status
> >> cc /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd/source.o -L/usr/local/lib -lblas -o 
> >> /tmp/tmpKcZ0cd/a.out
> >> /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd/source.o: In function `main':
> >> source.c:(.text.startup+0xdc): undefined reference to `cblas_ddot'
> >> collect2: error: ld returned 1 exit status
> >> -->8--
> >> 
> >> libcblas.so doesn't depend on libblas.so so missing symbols are to be
> >> expected if one links with -lcblas instead of -lcblas -lblas.  The
> >> second linking test fails because libblas.so doesn't provide cblas
> >> symbols.
> >
> > Thanks, this makes sense. But why does this work with ld.lld?
> 
> ld.lld doesn't bother checking that all symbols in libcblas.so can be
> resolved, ld.bfd does.  This means that if you link against a library
> that references a bogus symbol or lacks some library interdependency
> (DT_NEEDED) you only get a crash at run time.
> 
> On amd64, using the testcase from numpy:
> 
> --8<--
> russell /tmp$ cat r.c
> #include 
> int main(int argc, const char *argv[])
> {
> double a[4] = {1,2,3,4};
> double b[4] = {5,6,7,8};
> return cblas_ddot(4, a, 1, b, 1) > 10;
> }
> russell /tmp$ cc -I/usr/local/include r.c -L/usr/local/lib -lcblas
> russell /tmp$ ./a.out
> a.out:/usr/local/lib/libcblas.so.1.0: undefined symbol 'ddot_'
> ld.so: a.out: lazy binding failed!
> Killed
> -->8--
> 
> I suspect Stuart hit a similar problem with this numpy update and scipy.
> 
> Using -fuse-ld=bfd in the testcase above would result in the same errors
> as in your log.

I see. Thank you very much for your explanations.

FWIW your cblas diff is ok tb (also tested on macppc).



Re: WIP: Update of math/py-numpy to 1.16.5

2020-03-10 Thread Jeremie Courreges-Anglas
On Tue, Mar 10 2020, Theo Buehler  wrote:
> On Tue, Mar 10, 2020 at 06:35:04PM +0100, Jeremie Courreges-Anglas wrote:
>> On Mon, Mar 09 2020, Stuart Henderson  wrote:
>> > On 2020/03/09 10:42, Theo Buehler wrote:
>> >> On Mon, Jan 13, 2020 at 12:50:32PM +, Stuart Henderson wrote:
>> >> > 2/3 through a bulk build and I see that this breaks scipy (missing 
>> >> > symbols,
>> >> > blas/cblas-related) so needs a bit more work, but I think it's generally
>> >> > along the right lines.
>> >> 
>> >> Not sure if this provides any useful clue, but py-numpy doesn't build at
>> >> all on sparc64 with this diff, also due to missing blas/cblas symbols:
>> >
>> > You'll probably see the same on amd64 with USE_LLD=no.
>> 
>> I managed to build scipy with no changes on amd64, so I'm not sure what
>> the problem is on this arch (did not try with USE_LLD=No).
>> 
>> However I took a look at the issue reported by tb on sparc64.
>> 
>> --8<--
>> creating /tmp/tmpKcZ0cd/tmp
>> creating /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd
>> compile options: '-I/usr/local/include -I/usr/include -c'
>> cc: /tmp/tmpKcZ0cd/source.c
>> cc /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd/source.o -L/usr/local/lib -lcblas -o 
>> /tmp/tmpKcZ0cd/a.out
>> /usr/local/lib/libcblas.so.1.0: undefined reference to `ztbsv_'
>> /usr/local/lib/libcblas.so.1.0: undefined reference to `dasum_'
>> 
>> [...]
>> 
>> /usr/local/lib/libcblas.so.1.0: undefined reference to `zsymm_'
>> /usr/local/lib/libcblas.so.1.0: undefined reference to `ztrsm_'
>> /usr/local/lib/libcblas.so.1.0: undefined reference to `sswap_'
>> collect2: error: ld returned 1 exit status
>> cc /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd/source.o -L/usr/local/lib -lblas -o 
>> /tmp/tmpKcZ0cd/a.out
>> /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd/source.o: In function `main':
>> source.c:(.text.startup+0xdc): undefined reference to `cblas_ddot'
>> collect2: error: ld returned 1 exit status
>> -->8--
>> 
>> libcblas.so doesn't depend on libblas.so so missing symbols are to be
>> expected if one links with -lcblas instead of -lcblas -lblas.  The
>> second linking test fails because libblas.so doesn't provide cblas
>> symbols.
>
> Thanks, this makes sense. But why does this work with ld.lld?

ld.lld doesn't bother checking that all symbols in libcblas.so can be
resolved, ld.bfd does.  This means that if you link against a library
that references a bogus symbol or lacks some library interdependency
(DT_NEEDED) you only get a crash at run time.

On amd64, using the testcase from numpy:

--8<--
russell /tmp$ cat r.c
#include 
int main(int argc, const char *argv[])
{
double a[4] = {1,2,3,4};
double b[4] = {5,6,7,8};
return cblas_ddot(4, a, 1, b, 1) > 10;
}
russell /tmp$ cc -I/usr/local/include r.c -L/usr/local/lib -lcblas
russell /tmp$ ./a.out
a.out:/usr/local/lib/libcblas.so.1.0: undefined symbol 'ddot_'
ld.so: a.out: lazy binding failed!
Killed
-->8--

I suspect Stuart hit a similar problem with this numpy update and scipy.

Using -fuse-ld=bfd in the testcase above would result in the same errors
as in your log.

>> 
>> I think the way forward is to make libcblas.so depend on libblas.so
>> (this is what you get eg on Debian).  It would probably make sense to do
>> the same with lapack.
>> 
>> With the following cblas diff I can build py-numpy-1.16.5 on amd64 and
>> sparc64.  cc'ing Steven.
>> 
>> ok?
>> 
>> 
>> Index: Makefile
>> ===
>> RCS file: /cvs/ports/math/cblas/Makefile,v
>> retrieving revision 1.20
>> diff -u -p -r1.20 Makefile
>> --- Makefile 12 Jul 2019 20:47:40 -  1.20
>> +++ Makefile 10 Mar 2020 16:53:46 -
>> @@ -5,8 +5,8 @@ COMMENT= C interface to the BLAS library
>>  VERSION=1.0
>>  DISTNAME=   cblas
>>  PKGNAME=${DISTNAME}-${VERSION}
>> -REVISION=   6
>> -SHARED_LIBS=cblas   1.0
>> +REVISION=   7
>> +SHARED_LIBS=cblas   1.1
>>  
>>  CATEGORIES= math
>>  
>> Index: files/Makefile
>> ===
>> RCS file: /cvs/ports/math/cblas/files/Makefile,v
>> retrieving revision 1.1.1.1
>> diff -u -p -r1.1.1.1 Makefile
>> --- files/Makefile   2 Oct 2006 21:58:25 -   1.1.1.1
>> +++ files/Makefile   10 Mar 2020 16:53:46 -
>> @@ -42,5 +42,6 @@ cblas_dgemm.c   cblas_sdsdot.c  
>>  cblas_dgemv.c   cblas_sgbmv.c   cblas_zhemv.c \
>>  cblas_dger.ccblas_sgemm.c   cblas_zher.c \
>>  cblas_dnrm2.c   cblas_sgemv.c   cblas_zher2.c
>> +LDADD=-lblas -lgfortran
>>  
>>  .include 
>> Index: pkg/PLIST
>> ===
>> RCS file: /cvs/ports/math/cblas/pkg/PLIST,v
>> retrieving revision 1.4
>> diff -u -p -r1.4 PLIST
>> --- pkg/PLIST16 Mar 2015 18:07:49 -  1.4
>> +++ pkg/PLIST10 Mar 2020 16:53:46 -
>> @@ -1,6 +1,6 @@
>>  @comment $OpenBSD: PLIST,v 1.4 2015/03/16 18:07:49 naddy Exp $
>>  include/cblas.h
>>  include/cblas_f77.h
>> -lib/libcblas.a
>> +@static-lib lib/libcblas.a

Re: WIP: Update of math/py-numpy to 1.16.5

2020-03-10 Thread Theo Buehler
On Tue, Mar 10, 2020 at 06:35:04PM +0100, Jeremie Courreges-Anglas wrote:
> On Mon, Mar 09 2020, Stuart Henderson  wrote:
> > On 2020/03/09 10:42, Theo Buehler wrote:
> >> On Mon, Jan 13, 2020 at 12:50:32PM +, Stuart Henderson wrote:
> >> > 2/3 through a bulk build and I see that this breaks scipy (missing 
> >> > symbols,
> >> > blas/cblas-related) so needs a bit more work, but I think it's generally
> >> > along the right lines.
> >> 
> >> Not sure if this provides any useful clue, but py-numpy doesn't build at
> >> all on sparc64 with this diff, also due to missing blas/cblas symbols:
> >
> > You'll probably see the same on amd64 with USE_LLD=no.
> 
> I managed to build scipy with no changes on amd64, so I'm not sure what
> the problem is on this arch (did not try with USE_LLD=No).
> 
> However I took a look at the issue reported by tb on sparc64.
> 
> --8<--
> creating /tmp/tmpKcZ0cd/tmp
> creating /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd
> compile options: '-I/usr/local/include -I/usr/include -c'
> cc: /tmp/tmpKcZ0cd/source.c
> cc /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd/source.o -L/usr/local/lib -lcblas -o 
> /tmp/tmpKcZ0cd/a.out
> /usr/local/lib/libcblas.so.1.0: undefined reference to `ztbsv_'
> /usr/local/lib/libcblas.so.1.0: undefined reference to `dasum_'
> 
> [...]
> 
> /usr/local/lib/libcblas.so.1.0: undefined reference to `zsymm_'
> /usr/local/lib/libcblas.so.1.0: undefined reference to `ztrsm_'
> /usr/local/lib/libcblas.so.1.0: undefined reference to `sswap_'
> collect2: error: ld returned 1 exit status
> cc /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd/source.o -L/usr/local/lib -lblas -o 
> /tmp/tmpKcZ0cd/a.out
> /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd/source.o: In function `main':
> source.c:(.text.startup+0xdc): undefined reference to `cblas_ddot'
> collect2: error: ld returned 1 exit status
> -->8--
> 
> libcblas.so doesn't depend on libblas.so so missing symbols are to be
> expected if one links with -lcblas instead of -lcblas -lblas.  The
> second linking test fails because libblas.so doesn't provide cblas
> symbols.

Thanks, this makes sense. But why does this work with ld.lld?

> 
> I think the way forward is to make libcblas.so depend on libblas.so
> (this is what you get eg on Debian).  It would probably make sense to do
> the same with lapack.
> 
> With the following cblas diff I can build py-numpy-1.16.5 on amd64 and
> sparc64.  cc'ing Steven.
> 
> ok?
> 
> 
> Index: Makefile
> ===
> RCS file: /cvs/ports/math/cblas/Makefile,v
> retrieving revision 1.20
> diff -u -p -r1.20 Makefile
> --- Makefile  12 Jul 2019 20:47:40 -  1.20
> +++ Makefile  10 Mar 2020 16:53:46 -
> @@ -5,8 +5,8 @@ COMMENT=  C interface to the BLAS library
>  VERSION= 1.0
>  DISTNAME=cblas
>  PKGNAME= ${DISTNAME}-${VERSION}
> -REVISION=6
> -SHARED_LIBS= cblas   1.0
> +REVISION=7
> +SHARED_LIBS= cblas   1.1
>  
>  CATEGORIES=  math
>  
> Index: files/Makefile
> ===
> RCS file: /cvs/ports/math/cblas/files/Makefile,v
> retrieving revision 1.1.1.1
> diff -u -p -r1.1.1.1 Makefile
> --- files/Makefile2 Oct 2006 21:58:25 -   1.1.1.1
> +++ files/Makefile10 Mar 2020 16:53:46 -
> @@ -42,5 +42,6 @@ cblas_dgemm.c   cblas_sdsdot.c  
>  cblas_dgemv.c   cblas_sgbmv.c   cblas_zhemv.c \
>  cblas_dger.ccblas_sgemm.c   cblas_zher.c \
>  cblas_dnrm2.c   cblas_sgemv.c   cblas_zher2.c
> +LDADD=-lblas -lgfortran
>  
>  .include 
> Index: pkg/PLIST
> ===
> RCS file: /cvs/ports/math/cblas/pkg/PLIST,v
> retrieving revision 1.4
> diff -u -p -r1.4 PLIST
> --- pkg/PLIST 16 Mar 2015 18:07:49 -  1.4
> +++ pkg/PLIST 10 Mar 2020 16:53:46 -
> @@ -1,6 +1,6 @@
>  @comment $OpenBSD: PLIST,v 1.4 2015/03/16 18:07:49 naddy Exp $
>  include/cblas.h
>  include/cblas_f77.h
> -lib/libcblas.a
> +@static-lib lib/libcblas.a
>  @lib lib/libcblas.so.${LIBcblas_VERSION}
> -lib/libcblas_p.a
> +@static-lib lib/libcblas_p.a
> 
> 
> -- 
> jca | PGP : 0x1524E7EE / 5135 92C1 AD36 5293 2BDF  DDCC 0DFA 74AE 1524 E7EE
> 



Re: WIP: Update of math/py-numpy to 1.16.5

2020-03-10 Thread Jeremie Courreges-Anglas
On Tue, Mar 10 2020, Jeremie Courreges-Anglas  wrote:
> On Mon, Mar 09 2020, Stuart Henderson  wrote:
>> On 2020/03/09 10:42, Theo Buehler wrote:
>>> On Mon, Jan 13, 2020 at 12:50:32PM +, Stuart Henderson wrote:
>>> > 2/3 through a bulk build and I see that this breaks scipy (missing 
>>> > symbols,
>>> > blas/cblas-related) so needs a bit more work, but I think it's generally
>>> > along the right lines.
>>> 
>>> Not sure if this provides any useful clue, but py-numpy doesn't build at
>>> all on sparc64 with this diff, also due to missing blas/cblas symbols:
>>
>> You'll probably see the same on amd64 with USE_LLD=no.
>
> I managed to build scipy with no changes on amd64, so I'm not sure what
> the problem is on this arch (did not try with USE_LLD=No).
>
> However I took a look at the issue reported by tb on sparc64.
>
> --8<--
> creating /tmp/tmpKcZ0cd/tmp
> creating /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd
> compile options: '-I/usr/local/include -I/usr/include -c'
> cc: /tmp/tmpKcZ0cd/source.c
> cc /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd/source.o -L/usr/local/lib -lcblas -o 
> /tmp/tmpKcZ0cd/a.out
> /usr/local/lib/libcblas.so.1.0: undefined reference to `ztbsv_'
> /usr/local/lib/libcblas.so.1.0: undefined reference to `dasum_'
>
> [...]
>
> /usr/local/lib/libcblas.so.1.0: undefined reference to `zsymm_'
> /usr/local/lib/libcblas.so.1.0: undefined reference to `ztrsm_'
> /usr/local/lib/libcblas.so.1.0: undefined reference to `sswap_'
> collect2: error: ld returned 1 exit status
> cc /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd/source.o -L/usr/local/lib -lblas -o 
> /tmp/tmpKcZ0cd/a.out
> /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd/source.o: In function `main':
> source.c:(.text.startup+0xdc): undefined reference to `cblas_ddot'
> collect2: error: ld returned 1 exit status
> -->8--
>
> libcblas.so doesn't depend on libblas.so so missing symbols are to be
> expected if one links with -lcblas instead of -lcblas -lblas.  The
> second linking test fails because libblas.so doesn't provide cblas
> symbols.
>
> I think the way forward is to make libcblas.so depend on libblas.so
> (this is what you get eg on Debian).  It would probably make sense to do
> the same with lapack.
>
> With the following cblas diff I can build py-numpy-1.16.5 on amd64 and
> sparc64.  cc'ing Steven.
>
> ok?

Here's an updated diff for numpy-1.16.5, for convenience I decided to
drop the hard requirements I had on cblas>=1.1 (WANTLIB) /
math/cblas>=1.0p7 (LIB_DEPENDS).


Index: Makefile
===
RCS file: /cvs/ports/math/py-numpy/Makefile,v
retrieving revision 1.57
diff -u -p -r1.57 Makefile
--- Makefile10 Mar 2020 03:31:45 -  1.57
+++ Makefile10 Mar 2020 17:06:13 -
@@ -2,11 +2,10 @@
 
 COMMENT=   fast array and numeric programming library for Python
 
-MODPY_EGG_VERSION= 1.14.6
+MODPY_EGG_VERSION= 1.16.5
 DISTNAME=  numpy-${MODPY_EGG_VERSION}
 PKGNAME=   py-${DISTNAME}
 EXTRACT_SUFX=  .zip
-REVISION=  2
 
 CATEGORIES=math devel
 
@@ -15,7 +14,8 @@ HOMEPAGE= https://www.numpy.org/
 # BSD
 PERMIT_PACKAGE =   Yes
 
-WANTLIB=   blas lapack m pthread ${MODFORTRAN_WANTLIB} 
${MODPY_WANTLIB}
+WANTLIB += ${MODFORTRAN_WANTLIB} ${MODPY_WANTLIB}
+WANTLIB += blas cblas lapack m pthread
 
 MODULES=   lang/python \
fortran
@@ -27,13 +27,14 @@ MODPY_PI =  Yes
 MODPY_SETUPTOOLS=  Yes
 MODPY_DISTUTILS_BUILDARGS = --fcompiler=gnu95
 
-TEST_DEPENDS=  devel/py-nose${MODPY_FLAVOR} \
-   devel/py-tz${MODPY_FLAVOR}
-
+TEST_DEPENDS=  ${FULLPKGNAME}:${FULLPKGPATH} \
+   devel/py-test${MODPY_FLAVOR} \
+   devel/py-tz${MODPY_FLAVOR}
 MODFORTRAN_COMPILER =  gfortran
 
 BUILD_DEPENDS = ${MODFORTRAN_BUILD_DEPENDS}
-LIB_DEPENDS=math/lapack \
+LIB_DEPENDS=   math/cblas \
+   math/lapack \
${MODFORTRAN_LIB_DEPENDS}
 
 # Cython 0.28.* uses __attribute__((optimize("Os"))) which
Index: distinfo
===
RCS file: /cvs/ports/math/py-numpy/distinfo,v
retrieving revision 1.13
diff -u -p -r1.13 distinfo
--- distinfo26 Oct 2018 21:09:31 -  1.13
+++ distinfo10 Mar 2020 17:06:13 -
@@ -1,2 +1,2 @@
-SHA256 (numpy-1.14.6.zip) = ElDt9vbEPh14I/CWdBa8GCWLsnHcU2KY6w6gCp5FuAo=
-SIZE (numpy-1.14.6.zip) = 4906096
+SHA256 (numpy-1.16.5.zip) = i7RS2U6WSzEiBbDeEjjdcgnaRSNDZTqyFLXWgXgOegw=
+SIZE (numpy-1.16.5.zip) = 5138208
Index: patches/patch-numpy_core_include_numpy_npy_common_h
===
RCS file: 
/cvs/ports/math/py-numpy/patches/patch-numpy_core_include_numpy_npy_common_h,v
retrieving revision 1.6
diff -u -p -r1.6 patch-numpy_core_include_numpy_npy_common_h
--- patches/patch-numpy_core_include_numpy_npy_common_h 30 Jun 2018 21:49:33 
-  1.6
+++ 

Re: WIP: Update of math/py-numpy to 1.16.5

2020-03-10 Thread Jeremie Courreges-Anglas
On Mon, Mar 09 2020, Stuart Henderson  wrote:
> On 2020/03/09 10:42, Theo Buehler wrote:
>> On Mon, Jan 13, 2020 at 12:50:32PM +, Stuart Henderson wrote:
>> > 2/3 through a bulk build and I see that this breaks scipy (missing symbols,
>> > blas/cblas-related) so needs a bit more work, but I think it's generally
>> > along the right lines.
>> 
>> Not sure if this provides any useful clue, but py-numpy doesn't build at
>> all on sparc64 with this diff, also due to missing blas/cblas symbols:
>
> You'll probably see the same on amd64 with USE_LLD=no.

I managed to build scipy with no changes on amd64, so I'm not sure what
the problem is on this arch (did not try with USE_LLD=No).

However I took a look at the issue reported by tb on sparc64.

--8<--
creating /tmp/tmpKcZ0cd/tmp
creating /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd
compile options: '-I/usr/local/include -I/usr/include -c'
cc: /tmp/tmpKcZ0cd/source.c
cc /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd/source.o -L/usr/local/lib -lcblas -o 
/tmp/tmpKcZ0cd/a.out
/usr/local/lib/libcblas.so.1.0: undefined reference to `ztbsv_'
/usr/local/lib/libcblas.so.1.0: undefined reference to `dasum_'

[...]

/usr/local/lib/libcblas.so.1.0: undefined reference to `zsymm_'
/usr/local/lib/libcblas.so.1.0: undefined reference to `ztrsm_'
/usr/local/lib/libcblas.so.1.0: undefined reference to `sswap_'
collect2: error: ld returned 1 exit status
cc /tmp/tmpKcZ0cd/tmp/tmpKcZ0cd/source.o -L/usr/local/lib -lblas -o 
/tmp/tmpKcZ0cd/a.out
/tmp/tmpKcZ0cd/tmp/tmpKcZ0cd/source.o: In function `main':
source.c:(.text.startup+0xdc): undefined reference to `cblas_ddot'
collect2: error: ld returned 1 exit status
-->8--

libcblas.so doesn't depend on libblas.so so missing symbols are to be
expected if one links with -lcblas instead of -lcblas -lblas.  The
second linking test fails because libblas.so doesn't provide cblas
symbols.

I think the way forward is to make libcblas.so depend on libblas.so
(this is what you get eg on Debian).  It would probably make sense to do
the same with lapack.

With the following cblas diff I can build py-numpy-1.16.5 on amd64 and
sparc64.  cc'ing Steven.

ok?


Index: Makefile
===
RCS file: /cvs/ports/math/cblas/Makefile,v
retrieving revision 1.20
diff -u -p -r1.20 Makefile
--- Makefile12 Jul 2019 20:47:40 -  1.20
+++ Makefile10 Mar 2020 16:53:46 -
@@ -5,8 +5,8 @@ COMMENT=C interface to the BLAS library
 VERSION=   1.0
 DISTNAME=  cblas
 PKGNAME=   ${DISTNAME}-${VERSION}
-REVISION=  6
-SHARED_LIBS=   cblas   1.0
+REVISION=  7
+SHARED_LIBS=   cblas   1.1
 
 CATEGORIES=math
 
Index: files/Makefile
===
RCS file: /cvs/ports/math/cblas/files/Makefile,v
retrieving revision 1.1.1.1
diff -u -p -r1.1.1.1 Makefile
--- files/Makefile  2 Oct 2006 21:58:25 -   1.1.1.1
+++ files/Makefile  10 Mar 2020 16:53:46 -
@@ -42,5 +42,6 @@ cblas_dgemm.c   cblas_sdsdot.c  
 cblas_dgemv.c   cblas_sgbmv.c   cblas_zhemv.c \
 cblas_dger.ccblas_sgemm.c   cblas_zher.c \
 cblas_dnrm2.c   cblas_sgemv.c   cblas_zher2.c
+LDADD=-lblas -lgfortran
 
 .include 
Index: pkg/PLIST
===
RCS file: /cvs/ports/math/cblas/pkg/PLIST,v
retrieving revision 1.4
diff -u -p -r1.4 PLIST
--- pkg/PLIST   16 Mar 2015 18:07:49 -  1.4
+++ pkg/PLIST   10 Mar 2020 16:53:46 -
@@ -1,6 +1,6 @@
 @comment $OpenBSD: PLIST,v 1.4 2015/03/16 18:07:49 naddy Exp $
 include/cblas.h
 include/cblas_f77.h
-lib/libcblas.a
+@static-lib lib/libcblas.a
 @lib lib/libcblas.so.${LIBcblas_VERSION}
-lib/libcblas_p.a
+@static-lib lib/libcblas_p.a


-- 
jca | PGP : 0x1524E7EE / 5135 92C1 AD36 5293 2BDF  DDCC 0DFA 74AE 1524 E7EE



Re: WIP: Update of math/py-numpy to 1.16.5

2020-03-09 Thread Stuart Henderson
On 2020/03/09 10:42, Theo Buehler wrote:
> On Mon, Jan 13, 2020 at 12:50:32PM +, Stuart Henderson wrote:
> > 2/3 through a bulk build and I see that this breaks scipy (missing symbols,
> > blas/cblas-related) so needs a bit more work, but I think it's generally
> > along the right lines.
> 
> Not sure if this provides any useful clue, but py-numpy doesn't build at
> all on sparc64 with this diff, also due to missing blas/cblas symbols:

You'll probably see the same on amd64 with USE_LLD=no.



Re: WIP: Update of math/py-numpy to 1.16.5

2020-03-09 Thread Theo Buehler
On Mon, Jan 13, 2020 at 12:50:32PM +, Stuart Henderson wrote:
> 2/3 through a bulk build and I see that this breaks scipy (missing symbols,
> blas/cblas-related) so needs a bit more work, but I think it's generally
> along the right lines.

Not sure if this provides any useful clue, but py-numpy doesn't build at
all on sparc64 with this diff, also due to missing blas/cblas symbols:

===> py-numpy-1.16.5 depends on: g95->=8,<9 -> g95-8.3.0p4
===> py-numpy-1.16.5 depends on: python->=2.7,<2.8 -> python-2.7.17p1
===> py-numpy-1.16.5 depends on: py-setuptools->=39.0.1v0 -> 
py-setuptools-41.6.0v0
===> py-numpy-1.16.5 depends on: gcc->=8,<9 -> gcc-8.3.0p4
===> py-numpy-1.16.5 depends on: unzip-* -> unzip-6.0p12
===> py-numpy-1.16.5 depends on: cblas-* -> cblas-1.0p6
===> py-numpy-1.16.5 depends on: lapack-* -> lapack-3.8.0p1
===> py-numpy-1.16.5 depends on: gcc-libs->=8,<9 -> gcc-libs-8.3.0p4
===>  Verifying specs:  gfortran>=8 python2.7 blas cblas lapack m pthread 
gfortran>=8
===>  found gfortran.8.0 python2.7.0.0 blas.2.1 cblas.1.0 lapack.7.1 m.10.1 
pthread.26.1
===>  Checking files for py-numpy-1.16.5
`/usr/ports/distfiles/numpy-1.16.5.zip' is up to date.
>> (SHA256) numpy-1.16.5.zip: OK
===>  Extracting for py-numpy-1.16.5
===>  Patching for py-numpy-1.16.5
===>   Applying OpenBSD patch patch-numpy_core_include_numpy_npy_common_h
Hmm...  Looks like a unified diff to me...
The text leading up to this was:
--
|$OpenBSD: patch-numpy_core_include_numpy_npy_common_h,v 1.6 2018/06/30 
21:49:33 daniel Exp $
|
|XXX recheck powerpc, is this still needed?
|
|py-numpy only checks for expl to determine whether extended-precision
|support is present.  since we don't have it yet;  it implements
|it's own.  however, on alpha, powerpc, it declared functions with
|types that conflict with C99 (double for *l), therefore failed.
|
|Index: numpy/core/include/numpy/npy_common.h
|--- numpy/core/include/numpy/npy_common.h.orig
|+++ numpy/core/include/numpy/npy_common.h
--
Patching file numpy/core/include/numpy/npy_common.h using Plan A...
Hunk #1 succeeded at 320.
done
===>   Ignoring patchfile patch-numpy_core_include_numpy_npy_common_h.orig
===>   Applying OpenBSD patch patch-numpy_distutils_command_build_src_py
Hmm...  Looks like a unified diff to me...
The text leading up to this was:
--
|$OpenBSD: patch-numpy_distutils_command_build_src_py,v 1.3 2018/06/30 21:49:33 
daniel Exp $
|
|fix build of other packages (e.g. py-scipy) in some cases (e.g. when
|WRKOBJDIR has a trailing slash)
|
|Index: numpy/distutils/command/build_src.py
|--- numpy/distutils/command/build_src.py.orig
|+++ numpy/distutils/command/build_src.py
--
Patching file numpy/distutils/command/build_src.py using Plan A...
Hunk #1 succeeded at 370.
done
===>   Ignoring patchfile patch-numpy_distutils_command_build_src_py.orig
===>   Applying OpenBSD patch patch-numpy_distutils_fcompiler_gnu_py
Hmm...  Looks like a unified diff to me...
The text leading up to this was:
--
|$OpenBSD: patch-numpy_distutils_fcompiler_gnu_py,v 1.2 2018/06/30 21:49:33 
daniel Exp $
|
|Causes segmentation fault on powerpc when building py-scipy.
|
|See discussion at:
|https://github.com/numpy/numpy/issues/5451
|
|Index: numpy/distutils/fcompiler/gnu.py
|--- numpy/distutils/fcompiler/gnu.py.orig
|+++ numpy/distutils/fcompiler/gnu.py
--
Patching file numpy/distutils/fcompiler/gnu.py using Plan A...
Hunk #1 succeeded at 245.
done
===>   Ignoring patchfile patch-numpy_distutils_fcompiler_gnu_py.orig
===>   Applying OpenBSD patch patch-numpy_distutils_site_cfg
Hmm...  Looks like a unified diff to me...
The text leading up to this was:
--
|$OpenBSD: patch-numpy_distutils_site_cfg,v 1.2 2009/02/16 10:10:09 eric Exp $
|--- numpy/distutils/site.cfg.orig  Fri Feb 13 15:41:03 2009
|+++ numpy/distutils/site.cfg   Fri Feb 13 15:41:47 2009
--
(Creating file numpy/distutils/site.cfg...)
Patching file numpy/distutils/site.cfg using Plan A...
Empty context always matches.
Hunk #1 succeeded at 1.
done
===>  Compiler link: gcc -> /usr/local/bin/egcc
===>  Compiler link: cc -> /usr/local/bin/egcc
===>  Compiler link: gfortran -> /usr/local/bin/egfortran
===>  Compiler link: c++ -> /usr/bin/c++
===>  Generating configure for py-numpy-1.16.5
===>  Configuring for py-numpy-1.16.5
===>  Building for py-numpy-1.16.5
cp -f /usr/ports/pobj/py-numpy-1.16.5/numpy-1.16.5/numpy/distutils/site.cfg 
/usr/ports/pobj/py-numpy-1.16.5/numpy-1.16.5/site.cfg
Running from numpy source directory.
running egg_info
creating numpy.egg-info
writing numpy.egg-info/PKG-INFO
writing top-level names to numpy.egg-info/top_level.txt
writing dependency_links to numpy.egg-info/dependency_links.txt
writing entry points to numpy.egg-info/entry_points.txt
writing manifest file 'numpy.egg-info/SOURCES.txt'
reading manifest file 

Re: WIP: Update of math/py-numpy to 1.16.5

2020-01-13 Thread Stuart Henderson
2/3 through a bulk build and I see that this breaks scipy (missing symbols, 
blas/cblas-related) so needs a bit more work, but I think it's generally 
along the right lines.

--
Sent from a phone, apologies for poor formatting.

On 10 January 2020 16:00:43 Stuart Henderson  wrote:


On 2019/11/02 02:55, Kurt Mosiejczuk wrote:

I got this as far as compiling with flang, running its tests mostly
sucessfully, and packaging. I'm about to start traveling tomorrow
so I won't get to testing all the consumers right away. I figured I'd
send this out in case someone wants to throw it through a bulk or pick
up where I left off.


Here's a diff based on Kurt's but reinstating gfortran as compiler.
I'm not sure what the compiler change was originally done for, perhaps
it was just to avoid the problem with gcc flags that jca already fixed
in python/2.7?

I also removed part of patch-numpy_core_include_numpy_npy_common_h
that was dealing with gcc-<4.4 which we don't have to worry about
(the gfortran module uses gcc for C as well as Fortran, so it will
always be built with 4.4+ for us). I left the second part in but
we could do with testing powerpc with that file removed completely
(I added an XXX).

Builds ok (py2+3) on amd64, tests are in good shape:

py2: 10 failed, 7173 passed, 159 skipped, 166 deselected, 12 xfailed, 1 
xpassed, 3 warnings in 265.80 seconds
py3: 10 failed, 7266 passed, 83 skipped, 167 deselected, 12 xfailed, 1 
xpassed, 4 warnings in 234.12 seconds


I think it would be fair to ask for at least a bulk build on one fast
arch, plus builds of NumPy itself and at least some kind of runtime
testing on a couple others, but we are still at a time in the release
cycle where we can check build logs on other arches and get things
fixed, and expect to get some feedback from users if there are runtime
problems before things get baked into a release, so we don't need to
go overboard on this.

Index: Makefile
===
RCS file: /cvs/ports/math/py-numpy/Makefile,v
retrieving revision 1.56
diff -u -p -r1.56 Makefile
--- Makefile18 Nov 2019 04:10:31 -  1.56
+++ Makefile10 Jan 2020 15:46:06 -
@@ -2,11 +2,10 @@

COMMENT=fast array and numeric programming library for Python

-MODPY_EGG_VERSION= 1.14.6
+MODPY_EGG_VERSION= 1.16.5
DISTNAME=   numpy-${MODPY_EGG_VERSION}
PKGNAME=py-${DISTNAME}
EXTRACT_SUFX=   .zip
-REVISION=  1

CATEGORIES= math devel

@@ -17,7 +16,8 @@ MAINTAINER=   Daniel Dickman RCS file: 
/cvs/ports/math/py-numpy/patches/patch-numpy_core_include_numpy_npy_common_h,v

retrieving revision 1.6
diff -u -p -r1.6 patch-numpy_core_include_numpy_npy_common_h
--- patches/patch-numpy_core_include_numpy_npy_common_h	30 Jun 2018 
21:49:33 -	1.6
+++ patches/patch-numpy_core_include_numpy_npy_common_h	10 Jan 2020 
15:46:06 -

@@ -1,13 +1,8 @@
$OpenBSD: patch-numpy_core_include_numpy_npy_common_h,v 1.6 2018/06/30 
21:49:33 daniel Exp $


-(1) optimize function attributes were added in gcc 4.4. Apply a fix for 
the warnings. But

-the next release of numpy should fix this in a better way. See:
-https://github.com/numpy/numpy/issues/5451
+XXX recheck powerpc, is this still needed?

-Gets rid of these warnings from the build:
-numpy/core/src/multiarray/lowlevel_strided_loops.c.src:120: warning: 
'optimize' attribute directive ignored

-
-(2) py-numpy only checks for expl to determine whether extended-precision
+py-numpy only checks for expl to determine whether extended-precision
support is present.  since we don't have it yet;  it implements
it's own.  however, on alpha, powerpc, it declared functions with
types that conflict with C99 (double for *l), therefore failed.
@@ -15,36 +10,7 @@ types that conflict with C99 (double for
Index: numpy/core/include/numpy/npy_common.h
--- numpy/core/include/numpy/npy_common.h.orig
+++ numpy/core/include/numpy/npy_common.h
-@@ -20,6 +20,7 @@
- #endif
- #endif
-
-+#if defined(__GNUC__) && (__GNUC__ > 4 || (__GNUC__ == 4 && 
(__GNUC_MINOR__ > 4)))

- /*
-  * gcc does not unroll even with -O3
-  * use with care, unrolling on modern cpus rarely speeds things up
-@@ -30,13 +31,20 @@
- #else
- #define NPY_GCC_UNROLL_LOOPS
- #endif
-+#else
-+#define NPY_GCC_UNROLL_LOOPS
-+#endif
-
-+#if defined(__GNUC__) && (__GNUC__ > 4 || (__GNUC__ == 4 && 
(__GNUC_MINOR__ > 4)))

- /* highest gcc optimization level, enabled autovectorizer */
- #ifdef HAVE_ATTRIBUTE_OPTIMIZE_OPT_3
- #define NPY_GCC_OPT_3 __attribute__((optimize("O3")))
- #else
- #define NPY_GCC_OPT_3
- #endif
-+#else
-+#define NPY_GCC_OPT_3
-+#endif
-
- /* compile target attributes */
- #if defined HAVE_ATTRIBUTE_TARGET_AVX && defined HAVE_LINK_AVX
-@@ -336,14 +344,8 @@ typedef unsigned char npy_bool;
+@@ -320,14 +320,8 @@ typedef unsigned char npy_bool;
 #define NPY_FALSE 0
 #define NPY_TRUE 1

Index: patches/patch-numpy_distutils_command_build_src_py

Re: WIP: Update of math/py-numpy to 1.16.5

2020-01-10 Thread Stuart Henderson
On 2019/11/02 02:55, Kurt Mosiejczuk wrote:
> I got this as far as compiling with flang, running its tests mostly
> sucessfully, and packaging. I'm about to start traveling tomorrow 
> so I won't get to testing all the consumers right away. I figured I'd
> send this out in case someone wants to throw it through a bulk or pick
> up where I left off.

Here's a diff based on Kurt's but reinstating gfortran as compiler.
I'm not sure what the compiler change was originally done for, perhaps
it was just to avoid the problem with gcc flags that jca already fixed
in python/2.7?

I also removed part of patch-numpy_core_include_numpy_npy_common_h
that was dealing with gcc-<4.4 which we don't have to worry about
(the gfortran module uses gcc for C as well as Fortran, so it will
always be built with 4.4+ for us). I left the second part in but
we could do with testing powerpc with that file removed completely
(I added an XXX).

Builds ok (py2+3) on amd64, tests are in good shape:

py2: 10 failed, 7173 passed, 159 skipped, 166 deselected, 12 xfailed, 1 
xpassed, 3 warnings in 265.80 seconds
py3: 10 failed, 7266 passed, 83 skipped, 167 deselected, 12 xfailed, 1 xpassed, 
4 warnings in 234.12 seconds

I think it would be fair to ask for at least a bulk build on one fast
arch, plus builds of NumPy itself and at least some kind of runtime
testing on a couple others, but we are still at a time in the release
cycle where we can check build logs on other arches and get things
fixed, and expect to get some feedback from users if there are runtime
problems before things get baked into a release, so we don't need to
go overboard on this.

Index: Makefile
===
RCS file: /cvs/ports/math/py-numpy/Makefile,v
retrieving revision 1.56
diff -u -p -r1.56 Makefile
--- Makefile18 Nov 2019 04:10:31 -  1.56
+++ Makefile10 Jan 2020 15:46:06 -
@@ -2,11 +2,10 @@
 
 COMMENT=   fast array and numeric programming library for Python
 
-MODPY_EGG_VERSION= 1.14.6
+MODPY_EGG_VERSION= 1.16.5
 DISTNAME=  numpy-${MODPY_EGG_VERSION}
 PKGNAME=   py-${DISTNAME}
 EXTRACT_SUFX=  .zip
-REVISION=  1
 
 CATEGORIES=math devel
 
@@ -17,7 +16,8 @@ MAINTAINER=   Daniel Dickman https://github.com/numpy/numpy/issues/5451
+XXX recheck powerpc, is this still needed?
 
-Gets rid of these warnings from the build:
-numpy/core/src/multiarray/lowlevel_strided_loops.c.src:120: warning: 
'optimize' attribute directive ignored
-
-(2) py-numpy only checks for expl to determine whether extended-precision
+py-numpy only checks for expl to determine whether extended-precision
 support is present.  since we don't have it yet;  it implements
 it's own.  however, on alpha, powerpc, it declared functions with
 types that conflict with C99 (double for *l), therefore failed.
@@ -15,36 +10,7 @@ types that conflict with C99 (double for
 Index: numpy/core/include/numpy/npy_common.h
 --- numpy/core/include/numpy/npy_common.h.orig
 +++ numpy/core/include/numpy/npy_common.h
-@@ -20,6 +20,7 @@
- #endif
- #endif
- 
-+#if defined(__GNUC__) && (__GNUC__ > 4 || (__GNUC__ == 4 && (__GNUC_MINOR__ > 
4)))
- /*
-  * gcc does not unroll even with -O3
-  * use with care, unrolling on modern cpus rarely speeds things up
-@@ -30,13 +31,20 @@
- #else
- #define NPY_GCC_UNROLL_LOOPS
- #endif
-+#else
-+#define NPY_GCC_UNROLL_LOOPS
-+#endif
- 
-+#if defined(__GNUC__) && (__GNUC__ > 4 || (__GNUC__ == 4 && (__GNUC_MINOR__ > 
4)))
- /* highest gcc optimization level, enabled autovectorizer */
- #ifdef HAVE_ATTRIBUTE_OPTIMIZE_OPT_3
- #define NPY_GCC_OPT_3 __attribute__((optimize("O3")))
- #else
- #define NPY_GCC_OPT_3
- #endif
-+#else
-+#define NPY_GCC_OPT_3
-+#endif
- 
- /* compile target attributes */
- #if defined HAVE_ATTRIBUTE_TARGET_AVX && defined HAVE_LINK_AVX
-@@ -336,14 +344,8 @@ typedef unsigned char npy_bool;
+@@ -320,14 +320,8 @@ typedef unsigned char npy_bool;
  #define NPY_FALSE 0
  #define NPY_TRUE 1
  
Index: patches/patch-numpy_distutils_command_build_src_py
===
RCS file: 
/cvs/ports/math/py-numpy/patches/patch-numpy_distutils_command_build_src_py,v
retrieving revision 1.3
diff -u -p -r1.3 patch-numpy_distutils_command_build_src_py
--- patches/patch-numpy_distutils_command_build_src_py  30 Jun 2018 21:49:33 
-  1.3
+++ patches/patch-numpy_distutils_command_build_src_py  10 Jan 2020 15:46:06 
-
@@ -6,7 +6,7 @@ WRKOBJDIR has a trailing slash)
 Index: numpy/distutils/command/build_src.py
 --- numpy/distutils/command/build_src.py.orig
 +++ numpy/distutils/command/build_src.py
-@@ -372,6 +372,7 @@ class build_src(build_ext.build_ext):
+@@ -370,6 +370,7 @@ class build_src(build_ext.build_ext):
  #incl_dirs.append(self.build_src)
  build_dir = os.path.join(*([self.build_src]\
 

Re: WIP: Update of math/py-numpy to 1.16.5

2019-11-03 Thread Daniel Dickman



> On Nov 4, 2019, at 6:18 AM, Kurt Mosiejczuk  wrote:
> 
> On Sun, Nov 03, 2019 at 10:06:52PM +0100, Marc Espie wrote:
>> On Mon, Nov 04, 2019 at 02:32:03AM +0900, Daniel Dickman wrote:
 clang doesn't error out, gcc does.
 Why does numpy use gcc?
> 
>>> Because that's what upstream recommends in their docs.
> 
>>> 'Note that NumPy is developed mainly using GNU compilers. Compilers
>>> from other vendors such as Intel, Absoft, Sun, NAG, Compaq, Vast,
>>> Portland, Lahey, HP, IBM, Microsoft are only supported in the form
>>> of community feedback, and may not work out of the box. GCC 4.x (and
>>> later) compilers are recommended.'
> 
>>> See: https://numpy.org/devdocs/user/building.html
> 
>> That's probably not a good enough reason, especially considering how
>> many security improvements are in base clang as opposed to base gcc...
> 
> That page may be out of date, too. It says Python 2.7 is supported,
> whereas the 1.17.x release no longer support Python 2.7.

Kurt, please feel to take it up with upstream not me.

> 
> --Kurt
> 



Re: WIP: Update of math/py-numpy to 1.16.5

2019-11-03 Thread Daniel Dickman



> On Nov 4, 2019, at 6:06 AM, Marc Espie  wrote:
> 
> On Mon, Nov 04, 2019 at 02:32:03AM +0900, Daniel Dickman wrote:
>>> clang doesn't error out, gcc does.
>>> Why does numpy use gcc?
>> 
>> Because that’s what upstream recommends in their docs.
>> 
>> “Note that NumPy is developed mainly using GNU compilers. Compilers from 
>> other vendors such as Intel, Absoft, Sun, NAG, Compaq, Vast, Portland, 
>> Lahey, HP, IBM, Microsoft are only supported in the form of community 
>> feedback, and may not work out of the box. GCC 4.x (and later) compilers are 
>> recommended.”
>> 
>> See: https://numpy.org/devdocs/user/building.html
> 
> That's probably not a good enough reason, especially considering how many
> security improvements are in base clang as opposed to base gcc...

Marc please take it up with upstream not me.




Re: WIP: Update of math/py-numpy to 1.16.5

2019-11-03 Thread Jeremie Courreges-Anglas
On Sun, Nov 03 2019, Marc Espie  wrote:
> On Mon, Nov 04, 2019 at 02:32:03AM +0900, Daniel Dickman wrote:
>> > clang doesn't error out, gcc does.
>> > Why does numpy use gcc?
>> 
>> Because that’s what upstream recommends in their docs.
>> 
>> “Note that NumPy is developed mainly using GNU compilers. Compilers
>> from other vendors such as Intel, Absoft, Sun, NAG, Compaq, Vast,
>> Portland, Lahey, HP, IBM, Microsoft are only supported in the form of
>> community feedback, and may not work out of the box. GCC 4.x (and
>> later) compilers are recommended.”
>> 
>> See: https://numpy.org/devdocs/user/building.html
>
> That's probably not a good enough reason, especially considering how many
> security improvements are in base clang as opposed to base gcc...

hmmpf.

I initially thought that maybe the failure was due to /usr/bin/gcc being
used instead of /usr/bin/cc.  I was wrong.

numpy uses gfortran from ports/lang/gcc, so it makes perfect sense for
some gcc-ish option parser to bail out when passed a bogus -Oprout
option.  And indeed math/py-numpy fails the same way with the previous
python2 package on an amd64 system without /usr/bin/gcc.

I'm not willing to spend more time on this but I'm not keen on seeing
others waste time on this either.  Sorry for bringing this up without
actually checking in the first place.

-- 
jca | PGP : 0x1524E7EE / 5135 92C1 AD36 5293 2BDF  DDCC 0DFA 74AE 1524 E7EE



Re: WIP: Update of math/py-numpy to 1.16.5

2019-11-03 Thread Marc Espie
On Mon, Nov 04, 2019 at 02:32:03AM +0900, Daniel Dickman wrote:
> > clang doesn't error out, gcc does.
> > Why does numpy use gcc?
> 
> Because that’s what upstream recommends in their docs.
> 
> “Note that NumPy is developed mainly using GNU compilers. Compilers from 
> other vendors such as Intel, Absoft, Sun, NAG, Compaq, Vast, Portland, Lahey, 
> HP, IBM, Microsoft are only supported in the form of community feedback, and 
> may not work out of the box. GCC 4.x (and later) compilers are recommended.”
> 
> See: https://numpy.org/devdocs/user/building.html

That's probably not a good enough reason, especially considering how many
security improvements are in base clang as opposed to base gcc...



Re: WIP: Update of math/py-numpy to 1.16.5

2019-11-03 Thread Daniel Dickman
Now that the python issues seem to be solved, just one comment on the proposed 
diff:

> -MODFORTRAN_COMPILER =gfortran
> +MODFORTRAN_COMPILER =flang

numpy developers basically say to use gnu compilers, otherwise you’re on your 
own.

And numpy’s fortran compiler can’t be switched in isolation. We’d have to do a 
switch across all the forward and reverse dependencies in the tree at the same 
time.



Re: WIP: Update of math/py-numpy to 1.16.5

2019-11-03 Thread Daniel Dickman
> clang doesn't error out, gcc does.
> Why does numpy use gcc?

Because that’s what upstream recommends in their docs.

“Note that NumPy is developed mainly using GNU compilers. Compilers from other 
vendors such as Intel, Absoft, Sun, NAG, Compaq, Vast, Portland, Lahey, HP, 
IBM, Microsoft are only supported in the form of community feedback, and may 
not work out of the box. GCC 4.x (and later) compilers are recommended.”

See: https://numpy.org/devdocs/user/building.html

Re: WIP: Update of math/py-numpy to 1.16.5

2019-11-03 Thread Stuart Henderson

Ok with the REVISION bump.

--
Sent from a phone, apologies for poor formatting.

On 2 November 2019 22:49:21 Kurt Mosiejczuk  wrote:


On Sat, Nov 02, 2019 at 10:16:03PM +0100, Jeremie Courreges-Anglas wrote:


To stay on the safe side, here's a simple diff for lang/python/2.7 which
unbreaks numpy and maybe others.  clang doesn't error out, gcc does.
Why does numpy use gcc?



Index: Makefile
===
RCS file: /cvs/ports/lang/python/2.7/Makefile,v
retrieving revision 1.63
diff -u -p -r1.63 Makefile
--- Makefile1 Nov 2019 17:40:08 -   1.63
+++ Makefile2 Nov 2019 21:13:16 -
@@ -11,6 +11,7 @@ SHARED_LIBS = python2.7 0.0
 VERSION_SPEC = >=2.7,<2.8

 CONFIGURE_ARGS += --with-ensurepip=no
+CONFIGURE_ENV += ac_cv_opt_olimit_ok=no

 PORTROACH = limit:^2\.7


This works for me. Shouldn't we bump REVISION though?

--Kurt






Re: WIP: Update of math/py-numpy to 1.16.5

2019-11-03 Thread Stuart Henderson
On 2019/11/02 10:40, Daniel Dickman wrote:
> 
> 
> > On Nov 2, 2019, at 10:33 AM, Kurt Mosiejczuk  wrote:
> > 
> > On Sat, Nov 02, 2019 at 07:03:57AM -0400, Daniel Dickman wrote:
> > 
>  On Nov 2, 2019, at 2:56 AM, Kurt Mosiejczuk  wrote:
> >> 
>  'So the existing py-numpy is unhappy with the 2.7 update and it
> >>> seems to be clashes between clang and gcc.
> > 
> >> Should the 2.7 python update be backed out? Or is there some smaller
> >> fix for the existing numpy version?
> > 
> > Perhaps it should be backed out. That will also be rather complicated
> > though.
> 
> Rolling back to previous and cranking EPOCH doesn’t do the trick?

With EPOCH bumped, the version string used in dependencies no longer
matches. It's not as much work to fix as I first feared (thanks to
PKGSPEC), but it still would mean that pkg_add will have to merge a
huge update set, which is usually best avoided if possible.



Re: WIP: Update of math/py-numpy to 1.16.5

2019-11-02 Thread Jeremie Courreges-Anglas
On Sat, Nov 02 2019, Kurt Mosiejczuk  wrote:
> So the existing py-numpy is unhappy with the 2.7 update and it seems to 
> be clashes between clang and gcc.
>
> This is the last version of py-numpy to work with Python 2.7. 1.17.x will
> be Python 3.5+ only.



To stay on the safe side, here's a simple diff for lang/python/2.7 which
unbreaks numpy and maybe others.  clang doesn't error out, gcc does.
Why does numpy use gcc?


Index: Makefile
===
RCS file: /cvs/ports/lang/python/2.7/Makefile,v
retrieving revision 1.63
diff -u -p -r1.63 Makefile
--- Makefile1 Nov 2019 17:40:08 -   1.63
+++ Makefile2 Nov 2019 21:13:16 -
@@ -11,6 +11,7 @@ SHARED_LIBS = python2.7 0.0
 VERSION_SPEC = >=2.7,<2.8
 
 CONFIGURE_ARGS += --with-ensurepip=no
+CONFIGURE_ENV += ac_cv_opt_olimit_ok=no
 
 PORTROACH = limit:^2\.7
 


-- 
jca | PGP : 0x1524E7EE / 5135 92C1 AD36 5293 2BDF  DDCC 0DFA 74AE 1524 E7EE



Re: WIP: Update of math/py-numpy to 1.16.5

2019-11-02 Thread Daniel Dickman



> On Nov 2, 2019, at 10:33 AM, Kurt Mosiejczuk  wrote:
> 
> On Sat, Nov 02, 2019 at 07:03:57AM -0400, Daniel Dickman wrote:
> 
 On Nov 2, 2019, at 2:56 AM, Kurt Mosiejczuk  wrote:
>> 
 'So the existing py-numpy is unhappy with the 2.7 update and it
>>> seems to be clashes between clang and gcc.
> 
>> Should the 2.7 python update be backed out? Or is there some smaller
>> fix for the existing numpy version?
> 
> Perhaps it should be backed out. That will also be rather complicated
> though.

Rolling back to previous and cranking EPOCH doesn’t do the trick?




Re: WIP: Update of math/py-numpy to 1.16.5

2019-11-02 Thread Daniel Dickman



> On Nov 2, 2019, at 2:56 AM, Kurt Mosiejczuk  wrote:
> 
> So the existing py-numpy is unhappy with the 2.7 update and it seems to 
> be clashes between clang and gcc.

Should the 2.7 python update be backed out? Or is there some smaller fix for 
the existing numpy version?

Anyway can you share more details of the problem? I am traveling and probably 
can’t get to a box for at least another week.

> 
> This is the last version of py-numpy to work with Python 2.7. 1.17.x will
> be Python 3.5+ only.

Even if 1.17 supported python2, it can’t go in right now due to avx issues that 
I thought I’d solved but apparently haven’t and am I’m still trying to debug.

> 
> I got this as far as compiling with flang, running its tests mostly
> sucessfully, and packaging. I'm about to start traveling tomorrow

I am also traveling at the moment and not back for a week which is why I think 
we should back out the python update.


> so I won't get to testing all the consumers right away. I figured I'd
> send this out in case someone wants to throw it through a bulk or pick
> up where I left off.

The challenge with numpy is we really need to test that the direct consumers 
package *and* we want to do runtime tests as well (ie test the consumers of 
those consumers). That’s a lot of stuff to build which is why I don’t update 
numpy very often and try to do it carefully.

Going to 1.16.5 as a fix for python, without testing numpy in a bulk, 
preferably on multiple archs is asking for trouble it may just work but it 
also might cause more problems.

I wouldn’t go that route in a rush.


> 
> (cc maintainer)
> 
> --Kurt
> 
> Index: Makefile
> ===
> RCS file: /cvs/ports/math/py-numpy/Makefile,v
> retrieving revision 1.55
> diff -u -p -r1.55 Makefile
> --- Makefile7 Jul 2019 14:58:04 -1.55
> +++ Makefile2 Nov 2019 06:50:17 -
> @@ -2,11 +2,10 @@
> 
> COMMENT=fast array and numeric programming library for Python
> 
> -MODPY_EGG_VERSION=1.14.6
> +MODPY_EGG_VERSION=1.16.5
> DISTNAME=numpy-${MODPY_EGG_VERSION}
> PKGNAME=py-${DISTNAME}
> EXTRACT_SUFX=.zip
> -REVISION=1
> 
> CATEGORIES=math devel
> 
> @@ -17,7 +16,10 @@ MAINTAINER=Daniel Dickman  # BSD
> PERMIT_PACKAGE =Yes
> 
> -WANTLIB=blas lapack m pthread ${MODFORTRAN_WANTLIB} ${MODPY_WANTLIB}
> +COMPILER=base-clang ports-gcc base-gcc
> +
> +WANTLIB=blas lapack m pthread ${MODFORTRAN_WANTLIB} \
> +${MODPY_WANTLIB}
> 
> MODULES=lang/python \
>   fortran
> @@ -27,12 +29,12 @@ FLAVOR?=
> 
> MODPY_PI =Yes
> MODPY_SETUPTOOLS=Yes
> -MODPY_DISTUTILS_BUILDARGS = --fcompiler=gnu95
> 
> -TEST_DEPENDS=devel/py-nose${MODPY_FLAVOR} \
> -devel/py-tz${MODPY_FLAVOR}
> +TEST_DEPENDS=${FULLPKGNAME}:${FULLPKGPATH} \
> +devel/py-test${MODPY_FLAVOR} \
> +devel/py-tz${MODPY_FLAVOR}
> 
> -MODFORTRAN_COMPILER =gfortran
> +MODFORTRAN_COMPILER =flang
> 
> BUILD_DEPENDS = ${MODFORTRAN_BUILD_DEPENDS}
> LIB_DEPENDS=math/lapack \
> @@ -55,8 +57,8 @@ CFLAGS +=-D CYTHON_SMALL_CODE=
> pre-build:
>   cp -f ${WRKSRC}/numpy/distutils/site.cfg ${WRKSRC}/site.cfg
> 
> -do-test: fake
> -cd ${WRKDIR} && ${MAKE_ENV} PYTHONPATH=${WRKINST}${MODPY_SITEPKG} 
> ${MODPY_BIN} -c \
> +do-test:
> +cd ${WRKDIR} && ${MAKE_ENV} ${MODPY_BIN} -c \
>   'import numpy ; numpy.test()'
> 
> .include 
> Index: distinfo
> ===
> RCS file: /cvs/ports/math/py-numpy/distinfo,v
> retrieving revision 1.13
> diff -u -p -r1.13 distinfo
> --- distinfo26 Oct 2018 21:09:31 -1.13
> +++ distinfo2 Nov 2019 06:50:17 -
> @@ -1,2 +1,2 @@
> -SHA256 (numpy-1.14.6.zip) = ElDt9vbEPh14I/CWdBa8GCWLsnHcU2KY6w6gCp5FuAo=
> -SIZE (numpy-1.14.6.zip) = 4906096
> +SHA256 (numpy-1.16.5.zip) = i7RS2U6WSzEiBbDeEjjdcgnaRSNDZTqyFLXWgXgOegw=
> +SIZE (numpy-1.16.5.zip) = 5138208
> Index: pkg/PLIST
> ===
> RCS file: /cvs/ports/math/py-numpy/pkg/PLIST,v
> retrieving revision 1.12
> diff -u -p -r1.12 PLIST
> --- pkg/PLIST30 Jun 2018 21:49:33 -1.12
> +++ pkg/PLIST2 Nov 2019 06:50:17 -
> @@ -5,6 +5,8 @@ lib/python${MODPY_VERSION}/site-packages
> lib/python${MODPY_VERSION}/site-packages/numpy-${MODPY_EGG_VERSION}-py${MODPY_VERSION}.egg-info/PKG-INFO
> lib/python${MODPY_VERSION}/site-packages/numpy-${MODPY_EGG_VERSION}-py${MODPY_VERSION}.egg-info/SOURCES.txt
> lib/python${MODPY_VERSION}/site-packages/numpy-${MODPY_EGG_VERSION}-py${MODPY_VERSION}.egg-info/dependency_links.txt
> +lib/python${MODPY_VERSION}/site-packages/numpy-${MODPY_EGG_VERSION}-py${MODPY_VERSION}.egg-info/entry_points.txt
> +lib/python${MODPY_VERSION}/site-packages/numpy-${MODPY_EGG_VERSION}-py${MODPY_VERSION}.egg-info/not-zip-safe
>