[Numpy-discussion] recent changes in np.maximum.accumulate ?
I'm trying to track down test failures of statsmodels against recent master dev versions of numpy and scipy. The core computation is the following in one set of tests that fail pvals_corrected_raw = pvals * np.arange(ntests, 0, -1) pvals_corrected = np.maximum.accumulate(pvals_corrected_raw) this numpy version numpy-1.19.0.dev0%2B20200214184618_1f9ab28-cp38-cp38-manylinux2010_x86_64.whl is in the test run with failures (the first time statsmodel master failed) the previous version in the test runs didn't have these failures numpy-1.19.0.dev0%2B20200212232857_af0dfce-cp38-cp38-manylinux1_x86_64.whl I'm right now just fishing for candidates for the failures. And I'm not running any dev versions on my computer. Were there any recent changes that affect np.maximum.accumulate? Josef ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] recent changes in np.maximum.accumulate ?
On Tue, 2020-02-18 at 10:14 -0500, josef.p...@gmail.com wrote: > I'm trying to track down test failures of statsmodels against recent > master dev versions of numpy and scipy. > > The core computation is the following in one set of tests that fail > > pvals_corrected_raw = pvals * np.arange(ntests, 0, -1) > pvals_corrected = np.maximum.accumulate(pvals_corrected_raw) > Hmmm, the two git hashes indicate few changes between the two versions (mainly unicode related). However, recently there was also the addition of AVX-512F loops to maximum, so that seems like the most reasonable candidate (although I am unsure it changed exactly between those versions, it is also more complex maybe due to needing a machine that supports the instructions). Some details about the input could be nice. But if this is all that is as input, it sounds like it should be a contiguous array? I guess it might include subnormal numbers or NaN? Can you open an issue with some of those details if you have them? - Sebastian > this numpy version > numpy-1.19.0.dev0%2B20200214184618_1f9ab28-cp38-cp38- > manylinux2010_x86_64.whl > is in the test run with failures (the first time statsmodel master > failed) > > the previous version in the test runs didn't have these failures > numpy-1.19.0.dev0%2B20200212232857_af0dfce-cp38-cp38- > manylinux1_x86_64.whl > > > I'm right now just fishing for candidates for the failures. And I'm > not running any dev versions on my computer. > > Were there any recent changes that affect np.maximum.accumulate? > > Josef > > > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion signature.asc Description: This is a digitally signed message part ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] recent changes in np.maximum.accumulate ?
I have tried to locally reproduce this but cannot. I do not have AVX-512 on my machine. Fortunately we dump CPU info on travis and lo and behold it has AVX-512. *-cpu:0 description: CPU product: Intel(R) Xeon(R) CPU vendor: Intel Corp. physical id: 1001 bus info: cpu@0 slot: CPU 1 size: 2GHz capacity: 2GHz width: 64 bits capabilities: fpu fpu_exception wp vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp x86-64 constant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves arat avx512_vnni md_clear arch_capabilities Sounds like the AVX-512 path is bugged. Kevin From: Sebastian BergSent: Tuesday, February 18, 2020 3:42 PMTo: numpy-discussion@python.orgSubject: Re: [Numpy-discussion] recent changes in np.maximum.accumulate ? On Tue, 2020-02-18 at 10:14 -0500, josef.p...@gmail.com wrote:> I'm trying to track down test failures of statsmodels against recent> master dev versions of numpy and scipy.> > The core computation is the following in one set of tests that fail> > pvals_corrected_raw = pvals * np.arange(ntests, 0, -1)> pvals_corrected = np.maximum.accumulate(pvals_corrected_raw)> Hmmm, the two git hashes indicate few changes between the two versions(mainly unicode related). However, recently there was also the addition of AVX-512F loops tomaximum, so that seems like the most reasonable candidate (although Iam unsure it changed exactly between those versions, it is also morecomplex maybe due to needing a machine that supports the instructions). Some details about the input could be nice. But if this is all that isas input, it sounds like it should be a contiguous array? I guess itmight include subnormal numbers or NaN? Can you open an issue with some of those details if you have them? - Sebastian > this numpy version > numpy-1.19.0.dev0%2B20200214184618_1f9ab28-cp38-cp38-> manylinux2010_x86_64.whl> is in the test run with failures (the first time statsmodel master> failed)> > the previous version in the test runs didn't have these failures> numpy-1.19.0.dev0%2B20200212232857_af0dfce-cp38-cp38-> manylinux1_x86_64.whl> > > I'm right now just fishing for candidates for the failures. And I'm> not running any dev versions on my computer.> > Were there any recent changes that affect np.maximum.accumulate?> > Josef> > > > ___> NumPy-Discussion mailing list> NumPy-Discussion@python.org> https://mail.python.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] recent changes in np.maximum.accumulate ?
On Tue, Feb 18, 2020 at 10:43 AM Sebastian Berg wrote: > On Tue, 2020-02-18 at 10:14 -0500, josef.p...@gmail.com wrote: > > I'm trying to track down test failures of statsmodels against recent > > master dev versions of numpy and scipy. > > > > The core computation is the following in one set of tests that fail > > > > pvals_corrected_raw = pvals * np.arange(ntests, 0, -1) > > pvals_corrected = np.maximum.accumulate(pvals_corrected_raw) > > > > Hmmm, the two git hashes indicate few changes between the two versions > (mainly unicode related). > > However, recently there was also the addition of AVX-512F loops to > maximum, so that seems like the most reasonable candidate (although I > am unsure it changed exactly between those versions, it is also more > complex maybe due to needing a machine that supports the instructions). > > Some details about the input could be nice. But if this is all that is > as input, it sounds like it should be a contiguous array? I guess it > might include subnormal numbers or NaN? > The test failures are on a Travis machine https://travis-ci.org/statsmodels/statsmodels/jobs/650430129 I can extract the numbers and examples from the unit test on my Windows computer. But, if it's machine dependent, then that might not be enough. The main reason why maximum.accumulate might be the problem is that in some tests we don't get monotonically increasing values, e.g. Ex: array([0.012, 0.02 , 0.024, 0.024, 0.02 , 0.012]) Ey: array([0.012, 0.02 , 0.024, 0.024, 0.024, 0.024]) first row is computed, second row is expected Josef > > Can you open an issue with some of those details if you have them? > > - Sebastian > > > > > this numpy version > > numpy-1.19.0.dev0%2B20200214184618_1f9ab28-cp38-cp38- > > manylinux2010_x86_64.whl > > is in the test run with failures (the first time statsmodel master > > failed) > > > > the previous version in the test runs didn't have these failures > > numpy-1.19.0.dev0%2B20200212232857_af0dfce-cp38-cp38- > > manylinux1_x86_64.whl > > > > > > I'm right now just fishing for candidates for the failures. And I'm > > not running any dev versions on my computer. > > > > Were there any recent changes that affect np.maximum.accumulate? > > > > Josef > > > > > > > > ___ > > NumPy-Discussion mailing list > > NumPy-Discussion@python.org > > https://mail.python.org/mailman/listinfo/numpy-discussion > ___ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] NumPy Community Meeting Wednesday, Feb. 19
Hi all, There will be a NumPy Community meeting Wednesday February 19 at 11 am Pacific Time. Everyone is invited to join in and edit the work-in- progress meeting topics and notes: https://hackmd.io/76o-IxCjQX2mOXO_wwkcpg?both Best wishes Sebastian signature.asc Description: This is a digitally signed message part ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] numpy_financial functions
I teach finance at IU Bloomington and use the numpy_financial module pretty heavily. I've written a couple of functions for bond math for my own use (duration, convexity, forward rates, etc.). Is there any appetite for expanding numpy_financial beyond the core Excel functions? Is there a point-person for numpy_financial with whom I could correspond about contributing? Thanks, Jordan___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] numpy_financial functions
On Tue, Feb 18, 2020 at 2:33 PM Jordan wrote: > I teach finance at IU Bloomington and use the numpy_financial module > pretty heavily. I've written a couple of functions for bond math for my own > use (duration, convexity, forward rates, etc.). Is there any appetite for > expanding numpy_financial beyond the core Excel functions? Is there a > point-person for numpy_financial with whom I could correspond about > contributing? > Thanks, > Jordan > The financial package is separate from NumPy at this point. I don't see a problem with it being extended as long as it is maintained. Your best bet might be to make a PR at https://github.com/numpy/numpy-financial and initiate a conversation. I suspect the current maintainer(s) would be happy for help. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Tensor Developer Summit
Hi all, This has been mentioned on the community calls, but not on the mailing list, so a reminder about the Tensor Developer Summit happening at March in Berkeley: https://xd-con.org/tensor-2020/ We would love to have developers and advanced users of NumPy (or other array libraries with Python interfaces) attend. Registration closes 20 February. Best regards, Stéfan ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
