[Numpy-discussion] Re: Proposal to Extend NumPy Broadcasting for (D₁, D₂, ..., N, M) → (D₁, D₂, ..., K, M) When K is a Multiple of N (K % N == 0)

2025-03-25 Thread Kevin Sheppard via NumPy-Discussion
I think one aspect that would be hard to think about is how the tiling would happen when broadcasting from K -> N where N/K is an integer. There are at least 2 different ways to tile that would produce different results. Suppose you have the array [[1, 2, 3]] which is (1,3). If you wanted to bro

[Numpy-discussion] Expected behavior of np.array(..., copy=True)

2024-10-08 Thread Kevin Sheppard via NumPy-Discussion
Can anyone shed some light on the expected behavior of code using array(..., copy=True) with pandas objects? We ran into this in statsmodels and I think there are probably plenty of places where we explicitly call array(..., copy=True) and think we should have a totally independent copy of the data

[Numpy-discussion] Re: Arrays of variable itemsize

2024-03-13 Thread Kevin Sheppard
Does the new DType system in NumPy 2 make something like this more possible? I would suspect that the user would have to write a lot of code to have reasonable performance if it was. Kevin On Wed, Mar 13, 2024 at 3:55 PM Nathan wrote: > Yes, an array of references still has a fixed size width

[Numpy-discussion] Re: Switching default order to column-major

2023-11-12 Thread Kevin Sheppard
I think you can always using order="F" in your own code. If you patched NumPy and then the downstream libraries had to use your customized NumPy I think you would see some breaks. Probably not a lot, since many use the python numpy API which handles C or F well. Some code does do things like cal

[Numpy-discussion] Re: NEP 55 - Add a UTF-8 Variable-Width String DType to NumPy

2023-09-20 Thread Kevin Sheppard
On Wed, Sep 20, 2023 at 11:23 AM Ralf Gommers wrote: > > > On Wed, Sep 20, 2023 at 8:26 AM Warren Weckesser < > warren.weckes...@gmail.com> wrote: > >> >> >> On Fri, Sep 15, 2023 at 3:18 PM Warren Weckesser < >> warren.weckes...@gmail.com> wrote: >> > >> > >> > >> > On Mon, Sep 11, 2023 at 12:25 

[Numpy-discussion] Re: reminder: put an upper bound on setuptools if you use numpy.distutils!

2023-08-31 Thread Kevin Sheppard
On Sun, Aug 21, 2022 at 6:36 PM Ralf Gommers wrote: > Hi all, > > After setuptools 65.0 was released a few days ago, all users of > numpy.distutils had their builds broken. This is already fixed in > setuptools 65.0.2 because the breakage was particularly bad. However, the > next breakage may not

[Numpy-discussion] Re: Arbitrarily large random integers

2023-08-19 Thread Kevin Sheppard
The easiest way to do this would to to write a pure python implementation using Python ints of a masked integer sampler. This way you could draw unsigned integers and then treat this as a bit pool. You would than take the number of bits needed for your integer, transform these to be a Python int,

[Numpy-discussion] Re: How to avoid this memory copy overhead in d=a*b+c?

2022-09-16 Thread Kevin Sheppard
of simple calculation is negligible > compared to memory access, so I guess I still need a better way. > > So much thanks again for your reply! > > Kevin Sheppard 于2022年9月16日周五 15:38写道: > >> You can use inplace operators where appropriate to avoid memory >> allocat

[Numpy-discussion] Re: How to avoid this memory copy overhead in d=a*b+c?

2022-09-16 Thread Kevin Sheppard
You can use inplace operators where appropriate to avoid memory allocation. a *= bc += a Kevin  From: 腾刘Sent: Friday, September 16, 2022 8:35 AMTo: Discussion of Numerical PythonSubject: [Numpy-discussion] How to avoid this memory copy overhead in d=a*b+c? Hello everyone, I 'm here again to ask a n

[Numpy-discussion] Re: Representation of NumPy scalars

2022-09-09 Thread Kevin Sheppard
+1 from me. They are a frequent source of confusion when starting, and there appear to be far fewer now then in earlier releases. It also might make it easier to spot any inadvertent scalars coming out if these could be Python floats. Kevin On Fri, Sep 9, 2022, 07:23 Stefan van der Walt wrote:

[Numpy-discussion] Re: Setting C and linker flags for Windows build

2022-06-29 Thread Kevin Sheppard
> > > Hi, > > I am very sorry - I feel I should know this, or be able to work it > out, but is there a way of setting the flags to the C compiler and the > linker, for the Numpy build, on Windows? > > I'm trying to set the flags for a build with Windows mingw-w64 - but I > believe Numpy is ignoring

[Numpy-discussion] Re: seed for new random number generator

2022-06-19 Thread Kevin Sheppard
On Sun, Jun 19, 2022 at 4:36 PM Robert Kern wrote: > On Sun, Jun 19, 2022 at 9:37 AM Pieter Eendebak > wrote: > >> Hi everyone, >> >> The new numpy random interface (e.g. r=numpy.random.default_rng; >> r.random) is much faster than the old one (e.g. np.random.random). When >> converting code fr

[Numpy-discussion] Re: MacPython travis funds have run out (again)

2022-01-14 Thread Kevin Sheppard
> > > ...> Matti> > > [0]> https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=434709&view=logs&jobId=e9afaa34-1a0f-534e-78fc-fae528ccd915&j=e9afaa34-1a0f-534e-78fc-fae528ccd915&t=48395e60-bc5f-5e45-7455-d404f0c95f3a> >

[Numpy-discussion] Re: MacPython travis funds have run out (again)

2022-01-14 Thread Kevin Sheppard
As a data point against relying on QEMU: statsmodels times out (5+ hours) when trying to build with QEMU rising cibuildwheel and GitHub actions. Kevin On Fri, Jan 14, 2022, 08:53 Matti Picus wrote: > > On 14/1/22 3:04 am, Charles R Harris wrote: > > Hi All, > > > > Travis wheel build problems

[Numpy-discussion] Re: Proposal: Automatic estimation of number of histogram bins for weighted data

2021-12-13 Thread Kevin Sheppard
To me, this feels like it might be a better fit for SciPy or possibly statsmodels (but maybe not since neither have histogram functions anymore).The challenge with weighted estimators is how the weights should be interpreted. Stata covers the most important cases of weights https://www.reed.edu/psy

[Numpy-discussion] Re: Consistency of random number functions APIs.

2021-11-01 Thread Kevin Sheppard
> Some NumPy random number generation functions take a dtype parameter > whereas others don't. Some of them take an out parameter whereas others > don't. Just glancing at it, there seems to be no rhyme or reason why this > would be the case but is there some hidden consistency underneath the hood

[Numpy-discussion] Re: What happened to the numpy.random documentation?

2021-10-15 Thread Kevin Sheppard
On Thu, Oct 14, 2021 at 7:22 PM Ralf Gommers wrote: > > > On Thu, Oct 14, 2021 at 7:19 PM Kevin Sheppard > wrote: > >> I think the issue in random specifically is that a raw list of >> available functions does not provide suitable guidance for someone looking >&

[Numpy-discussion] Re: What happened to the numpy.random documentation?

2021-10-14 Thread Kevin Sheppard
I think the issue in random specifically is that a raw list of available functions does not provide suitable guidance for someone looking for random variate generating function. This is because the module-level API is mostly dominated by methods of the singleton RandomState instance. Best practice

[Numpy-discussion] Re: Running numpy.test() after pip install

2021-09-28 Thread Kevin Sheppard
I think NumPy has always tried to have effectively no install dependencies at least when installed from source (minimal install from git only requires Cython).  This seems like a good goal to me and there are scenarios where extra packages are a burden (e.g., Amazon Lambda).  AFAICT pretty much eve

Re: [Numpy-discussion] reducing effort spent on wheel builds?

2021-07-15 Thread Kevin Sheppard
When thinking about supporting a platform, it seems reasonable to consider other sources for pre-compiled binaries, e.g., conda and especially conda-forge. Conda-forge has linux-ppc64le  v1.7.0linux-64  v1.7.0linux-aarch64  v1.7.0osx-arm64  v1.7.0osx-64  v1.7.0win-32  v1.2.1win-64  v1.7.0 If this l

Re: [Numpy-discussion] Interface for wrapping random distribution functions (__array_function__, __ufunc__, ?)

2021-06-30 Thread Kevin Sheppard
There is an issue that suggests reimplementing many of the Generator methods as ufuncs so they they would inherit all of the free features of ufuncs. It seems like a doable but large project for some useful but nonessential features. Kevin On Wed, Jun 30, 2021, 20:49 Robert Kern wrote: > On We

Re: [Numpy-discussion] bad CRC errors when using np.savez, only sometimes though!

2021-05-14 Thread Kevin Sheppard
You could use pandas if your arrays are 2d to serialize using parquet or HDF.  If your arrays are 3d, you could use xarray and NetCDF. I think in both cases if your arrays are standard dtypes then you should be able to avoid copies. Both support compression. Kevin  From: Isaac GergSent: Friday, May

Re: [Numpy-discussion] two questions about `choose`

2021-04-18 Thread Kevin Sheppard
Oh. I answered thinking about choice and not choose. Please ignore both parts. On Sun, Apr 18, 2021, 17:56 Robert Kern wrote: > On Sat, Apr 17, 2021 at 4:28 PM Kevin Sheppard > wrote: > >> 1. I suppose it only uses the (Native int or int64) dtype since each one >> would ne

Re: [Numpy-discussion] two questions about `choose`

2021-04-17 Thread Kevin Sheppard
1. I suppose it only uses the (Native int or int64) dtype since each one would need a code path to run quickly. 2. I would describe this a a bug. I think sequences are converted to arrays and in this case the conversion is not returning a 2 element object array but expanding and then concatenation

Re: [Numpy-discussion] Dot + add operation

2021-03-31 Thread Kevin Sheppard
Or just use SciPy's get_blas_funcs to access *gemm, which directly exposes this function: https://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.blas.dgemm.html

Re: [Numpy-discussion] Using logfactorial instead of loggamma in random_poisson sampler

2021-03-08 Thread Kevin Sheppard
I did a quick test and using random_loggam was about 6% faster than using logfactorial (on Windows). Kevin On Sun, Mar 7, 2021 at 2:40 AM Robert Kern wrote: > On Sat, Mar 6, 2021 at 1:45 PM Warren Weckesser < > warren.weckes...@gmail.com> wrote: > >> At the time, making that change was not a h

Re: [Numpy-discussion] String accessor methods

2021-03-07 Thread Kevin Sheppard
I think that and string functions that are exposed from an ndarray would have to be guaranteed to work in-place. Requiring casting to objects to use the methods feels more like syntactic sugar than an essential case. I think most of the ones mentioned are low performance and can't take advantage of

Re: [Numpy-discussion] Guidelines for floating point comparison

2021-02-24 Thread Kevin Sheppard
In my experience it is most common to use reasonable but not exceedingly tight bounds in complex applications where there isn’t a proof that the maximum error must be smaller than some number.  I would also caution against using a single system to find the tightest tolerance a test passes at.  For

Re: [Numpy-discussion] ENH: Proposal to add KML_BLAS support

2021-02-22 Thread Kevin Sheppard
It is probably best to not send around xlsx files.  Perhaps reformat as a table in an email? Kevin  From: ChunLin FangSent: Monday, February 22, 2021 12:16 PMTo: numpy-discussion@python.orgSubject: [Numpy-discussion] ENH: Proposal to add KML_BLAS support Part of the performance benchmark results. 

Re: [Numpy-discussion] Question about optimizing random_standard_normal

2021-02-08 Thread Kevin Sheppard
this is the first time a substantially new algo has replaced an existing one. Kevin  From: Robert KernSent: Monday, February 8, 2021 4:06 PMTo: Discussion of Numerical PythonSubject: Re: [Numpy-discussion] Question about optimizing random_standard_normal On Mon, Feb 8, 2021 at 10:53 AM Kevin Sheppard

Re: [Numpy-discussion] Question about optimizing random_standard_normal

2021-02-08 Thread Kevin Sheppard
PythonSubject: Re: [Numpy-discussion] Question about optimizing random_standard_normal On Mon, Feb 8, 2021 at 3:05 AM Kevin Sheppard <kevin.k.shepp...@gmail.com> wrote:If I understand correctly, there is no gain when applying this patch to Generator.  I'm not that surprised that this

Re: [Numpy-discussion] Question about optimizing random_standard_normal

2021-02-08 Thread Kevin Sheppard
If I understand correctly, there is no gain when applying this patch to Generator. I'm not that surprised that this is the case since the compiler is much more limited in when it can do in Generator than what it can when filling a large array directly with functions available for inlining and unro

Re: [Numpy-discussion] Question about optimizing random_standard_normal

2021-02-06 Thread Kevin Sheppard
Have you benchmarked it using the generator interface? The structure of this as a no monolithic generator makes it a good deal slower than generating in straight C (with everything inline). While I'm not sure a factor of 2 is enough to justify a change (for me 10x, 1.2x is not but I don't know whe

Re: [Numpy-discussion] numpy.linalg interference with matplot.lib

2021-01-11 Thread Kevin Sheppard
Are you using NumPy <= 1.19.3 and WIndows 2004 or 20H2? If so, you should upgrade to NumPy 1.19.5. Kevin On Mon, Jan 11, 2021 at 10:51 AM Frederix96 wrote: > Dear numpy community, > > I am having an issue with numpy.linalg where a matrix inversion of a > complex matrix using numpy.linalg.inv

Re: [Numpy-discussion] How to speed up array generating

2021-01-09 Thread Kevin Sheppard
Actually I would try a broadcast multiply followed by ravel first. Kevin On Sat, Jan 9, 2021, 21:12 Kevin Sheppard wrote: > What about arange and then an integer divide or mod? > > Kevin > > > On Sat, Jan 9, 2021, 20:54 wrote: > >> np.meshgrid, indexing, resh

Re: [Numpy-discussion] How to speed up array generating

2021-01-09 Thread Kevin Sheppard
What about arange and then an integer divide or mod? Kevin On Sat, Jan 9, 2021, 20:54 wrote: > np.meshgrid, indexing, reshape > > 09.01.2021, 22:30, "Joseph Fox-Rabinovitz" : > > What other ways have you tried? > > On Sat, Jan 9, 2021 at 2:15 PM wrote: > > Hello. There is a random 1D array m_

Re: [Numpy-discussion] Deprecate np.testing.dec

2020-12-23 Thread Kevin Sheppard
Have you performed a search on GitHub to look for use of the decorators?  I think external use is more of a concern than internal. Kevin  From: rpolleySent: Wednesday, December 23, 2020 6:05 PMTo: numpy-discussion@python.orgSubject: [Numpy-discussion] Deprecate np.testing.dec Saw some discussion in

Re: [Numpy-discussion] locking np.random.Generator in a cython nogil context?

2020-12-14 Thread Kevin Sheppard
np.random.Generator in a cython nogil context? On Mon, Dec 14, 2020 at 4:46 PM Kevin Sheppard wrote:> > You need to reacquire the gil, then you can get the lock and rerelease the gil.> > I think this works (on phone, so untested)> > with gil:> with nogil, lock:> .. 

Re: [Numpy-discussion] Installing numpy

2020-12-14 Thread Kevin Sheppard
NumPy does not support Python 2.7. If you must use 2.7 you need to use NumPy 1.16.6.  Otherwise, you should probably be using Python 3.8 for new work. Kevin  From: Lianyuan ZhengSent: Monday, December 14, 2020 2:15 PMTo: Discussion of Numerical PythonSubject: Re: [Numpy-discussion] Installing numpy

Re: [Numpy-discussion] locking np.random.Generator in a cython nogil context?

2020-12-14 Thread Kevin Sheppard
You need to reacquire the gil, then you can get the lock and rerelease the gil. I think this works (on phone, so untested) with gil: with nogil, lock: ... Kevin On Mon, Dec 14, 2020, 13:37 Evgeni Burovski wrote: > Hi, > > What would be the correct way of locking the bit generator

Re: [Numpy-discussion] problem with numpy 1.19.4 install via pip on Win 10

2020-12-03 Thread Kevin Sheppard
I believe the next releases (1.19.x or 1.20.0, whichever if first) of NumPy should have 0.3.12 built using a smaller buffersize which will make this import error stop appearing on Windows, irrespective of the MS-provided fix.  From: Oscar BenjaminSent: Friday, December 4, 2020 12:05 AMTo: Discussio

Re: [Numpy-discussion] How did Numpy get its latest version of the documentation to appear at the top of Google search results?

2020-11-13 Thread Kevin Sheppard
I’m not sure if NumPy got these in place, but you could add a canonical link to each page that has  If your pages.  Would require some sphinx mods to your theme.  This would tell all search engines that they should refer to the latest as the authoritative page for competing pages. See https://yoast

Re: [Numpy-discussion] NumPy 1.20.x branch in two weeks

2020-11-01 Thread Kevin Sheppard
I also feel the NEP is very useful since it helps downstream to drop older versions relatively early.  I’ve found this very useful in projects that have less maintenance bandwidth. Kevin  From: Jarrod MillmanSent: Monday, November 2, 2020 2:56 AMTo: Discussion of Numerical PythonSubject: Re: [Numpy

Re: [Numpy-discussion] Efficient way to draw multinomial distribution random samples

2020-11-01 Thread Kevin Sheppard
This is in the pending PR. Hopefully out in 1.20. Kevin On Sun, Nov 1, 2020, 23:55 Currurant wrote: > I realized that neither numpy.random.multinomial nor rng.multinomial has > the > ability to draw from different multinomial distributions at the same time > like what MATLAB mnrnd() does here

Re: [Numpy-discussion] Should we transfer MacPython/numpy-wheels to the NumPy org?

2020-10-29 Thread Kevin Sheppard
Hi Matti, Could you migrate the statsmodels-wheels repo? ThanksKevin  From: Matti PicusSent: Thursday, October 29, 2020 4:19 PMTo: numpy-discussion@python.orgSubject: Re: [Numpy-discussion] Should we transfer MacPython/numpy-wheels to the NumPy org?  On 10/29/20 4:42 PM, Charles R Harris wrote: My

Re: [Numpy-discussion] Proposal: split numpy.distutils into it's own package

2020-10-25 Thread Kevin Sheppard
NumPy could take an explicit runtime dependency on numpy-distutils so that the code would be technically in a different repo bit would always be available through NumPy. Eventually this could be removed as a runtime dependency. Kevin On Sun, Oct 25, 2020, 09:23 Matti Picus wrote: > > On 10/25/

Re: [Numpy-discussion] Proposal: split numpy.distutils into it's own package

2020-10-24 Thread Kevin Sheppard
Separating it is definitely a good idea. The only thing that I think would be better would be if they key features that are not in setuptools could be added there so that NumPy distutils could eventually be retired. Kevin On Sat, Oct 24, 2020, 08:12 Dustin Spicuzza wrote: > Hey all, > > [copie

Re: [Numpy-discussion] new function: broadcast_shapes

2020-10-16 Thread Kevin Sheppard
There is at least one custom implementation in `Generator` IIRC, so a formal API addition sounds like a good idea. Kevin On Thu, Oct 15, 2020 at 8:52 PM Stephan Hoyer wrote: > On Thu, Oct 15, 2020 at 11:46 AM Warren Weckesser < > warren.weckes...@gmail.com> wrote: > >> On 10/15/20, Madhulika J

[Numpy-discussion] Replacement for Rackspace

2020-08-07 Thread Kevin Sheppard
The Rackspace hosted wheel endpoints at  https://7933911d6844c6c53a7d-47bd50c35cd79bd838daf386af554a83.ssl.cf2.rackcdn.com/ and https://3f23b170c54c2533c070-1c8a9b3114517dc5fe17b7c3f8c63a43.ssl.cf2.rackcdn.com/ seem to not be working.  I know NumPy, SciPy, pandas and scikit-learn are all using a co

Re: [Numpy-discussion] NumPy dtype API improvement suggestion

2020-07-26 Thread Kevin Sheppard
Better would be to have an object like NamedTuple in typing that would allow class Point(DType): x: np.int16 y: np.int16 On Sun, Jul 26, 2020 at 3:22 PM Eyal Kutz wrote: > I am interested in suggesting an API improvement for NumPy. > I wish to make it so that the following code: > @np

Re: [Numpy-discussion] NumPy Documentation Gallery structure

2020-07-08 Thread Kevin Sheppard
statsmodels uses a homegrown solution with some templating to make a gallery of jupyter notebooks. https://github.com/statsmodels/statsmodels/blob/master/docs/source/examples/index.rst   From: Saber ToothSent: Wednesday, July 8, 2020 8:06 AMTo: Matti PicusCc: numpy-discussion@python.org; numpy-scip

Re: [Numpy-discussion] reseed random generator (1.19)

2020-06-29 Thread Kevin Sheppard
nstead?On Mon, Jun 29, 2020 at 5:37 PM Kevin Sheppard wrote:> > The best practice is to use a SeedSequence to spawn child SeedSequences, and then to use these children to initialize your generators or bit generators.> > > > > > from numpy.random import SeedSe

Re: [Numpy-discussion] reseed random generator (1.19)

2020-06-29 Thread Kevin Sheppard
` framework?A human-friendly way would be preferable if possible. Thanks, Evgeni  On Mon, Jun 29, 2020 at 3:20 PM Kevin Sheppard wrote:> > If you want to use the same entropy-initialized generator for temporarily-reproducible experiments, then you can use> > > > gen = np.random.defa

Re: [Numpy-discussion] reseed random generator (1.19)

2020-06-29 Thread Kevin Sheppard
If you want to use the same entropy-initialized generator for temporarily-reproducible experiments, then you can use gen = np.random.default_rng()state = gen.bit_generator.stategen.standard_normal()# 0.5644742559549797, will vary across runsgen.bit_generator.state = stategen.standard_normal()# Alwa

Re: [Numpy-discussion] reseed random generator (1.19)

2020-06-24 Thread Kevin Sheppard
Just call rs = default_rng() Again. On Wed, Jun 24, 2020, 20:31 Neal Becker wrote: > Consider the following: > > from numpy.random import default_rng > rs = default_rng() > > Now how do I re-seed the generator? > I thought perhaps rs.bit_generator.seed(), but there is no such attribute. > >

[Numpy-discussion] ANN: randomgen 1.19.0 release

2020-06-24 Thread Kevin Sheppard
randomgen 1.19.0 has been released with bug fixes and new features. *New Features* - A helper class that lets users define custom bit generators in Python (slow) or numba (fast). This simplifies experimenting with alternative configurations. The UserBitGenerator can be used with numpy

Re: [Numpy-discussion] Armv8 server donation

2020-06-11 Thread Kevin Sheppard
Someone could setup https://drone.io/ which has both aarch64 and armv8l servers. I have used this service without issue in randomgen which requires building NumPy (but not testing it). On Thu, Jun 11, 2020 at 2:21 PM Stefan van der Walt wrote: > On Thu, Jun 11, 2020, at 13:47, ChunLin Fang wr

Re: [Numpy-discussion] Feelings about type aliases in NumPy

2020-04-24 Thread Kevin Sheppard
Typing is for library developers more than end users. I would also worry that putting it into the top level might discourage other typing classes since it is more difficult to add to the top level than to a lower level module. np.typing seems very clear to me. On Sat, Apr 25, 2020, 07:41 Stephan H

Re: [Numpy-discussion] recent changes in np.maximum.accumulate ?

2020-02-18 Thread Kevin Sheppard
I have tried to locally reproduce this but cannot.  I do not have AVX-512 on my machine.  Fortunately we dump CPU info on travis and lo and behold it has AVX-512. *-cpu:0   description: CPU   product: Intel(R) Xeon(R) CPU   vendor: Intel Corp.   physical id: 1001   bus info: cpu

Re: [Numpy-discussion] what to clean up, what to leave as is

2020-01-24 Thread Kevin Sheppard
I think some types of clean-ups, for example, imports, are pretty low cost, low risk and don't have much bearing on and it might be best to do them all at once. f-strings are also pretty simple but can be abused to the detriment of code around (for example, moving a string defined in a variable ou

Re: [Numpy-discussion] Is the build-system section of pyproject.toml a maintained list of the build dependencies?

2020-01-08 Thread Kevin Sheppard
With recent versions of pip it will read the pyproject.toml file to get the dependencies, and then install these in an isolated environment to build the wheel, and then install the wheel. The requires=[...] in the pyproject is not installed in the original environment, so that when run on NumPy, y

Re: [Numpy-discussion] Low-level API for Random

2019-09-25 Thread Kevin Sheppard
> > I'd like to clarify what exactly we mean by exposing a C API. Do we have > in mind that our random number generators can be used from standalone C > code, or via Cython `cimport` like with the current numpy.pxd? > > It sounds like we want to expose the highest level generators; do we also > wa

Re: [Numpy-discussion] Low-level API for Random

2019-09-19 Thread Kevin Sheppard
On Thu, Sep 19, 2019 at 10:23 AM Ralf Gommers wrote: > > > On Thu, Sep 19, 2019 at 10:28 AM Kevin Sheppard < > kevin.k.shepp...@gmail.com> wrote: > >> There are some users of the NumPy C code in randomkit. This was never >> officially supported. There has been

[Numpy-discussion] Low-level API for Random

2019-09-19 Thread Kevin Sheppard
There are some users of the NumPy C code in randomkit. This was never officially supported. There has been a long open issue to provide this officially. When I wrote randomgen I supplied .pdx files that make it simpler to write Cython code that uses the components. The lower-level API has not h

[Numpy-discussion] Creating a subclass that never propagates

2019-07-16 Thread Kevin Sheppard
I am trying to make a subclass that never propagates so that when interacted with another ndarray, or even itself so that the return type is always ndarray. Is this possible? I got pretty far with def __array_wrap__(self, out_arr, context=None): if out_arr.shape == (): return out_arr

Re: [Numpy-discussion] overhauling numpy.random and randomgen Message-ID:

2019-04-19 Thread Kevin Sheppard
> Rather than "base RNG", what about calling these classes a "random source" or "random stream"? In particular, I would suggest defining two Python classes: > - np.random.Generator as a less redundant name for what is currently called RandomGenerator > - np.random.Source or np.random.Stream as an

[Numpy-discussion] Complex Normal Generator in NEP-19 extension

2019-03-29 Thread Kevin Sheppard
One part of moving randomgen closer to fulfilling NEP-19 is rationalizing the API, especially new features not in RandomState. Matti Picus has made a lot of progress in getting it integrated, especially the part of replacing RandomState shimed version of the new generator. There is only one new me

[Numpy-discussion] NEP 21: Simplified and explicit advanced indexing

2018-06-26 Thread Kevin Sheppard
This seems like progress and a clear method to outer indexing will help many users. As for names, I prefer .ox and .vx for shorthand of .oindex and .vindex. I don’t like the .ox_ or .o_ syntax. Before any deprecation warnings or any other warnings are added it would be helpful to have some wa

Re: [Numpy-discussion] NEP: Random Number Generator Policy (Robert Kern)

2018-06-11 Thread Kevin Sheppard
Maybe a good place for a stable, testing focused generator would be in numpy.random.testing. This could host a default implementation of StableGenerator, although a better name might be TestingGenerator. It would also help users decide that this is not the generator they are looking for (I th

Re: [Numpy-discussion] NEP: Random Number Generator Policy

2018-06-04 Thread Kevin Sheppard
I’m not sure if this is within the scope of the NEP or an implementation detail, but I think a new PRNG should use platform independent integer types rather than depending on the platform’s choice of 64-bit data model. This should be enough to ensure that any integer distribution that only uses

Re: [Numpy-discussion] NEP: Random Number Generator Policy

2018-06-03 Thread Kevin Sheppard
The seed() discussion seems unnecessary. StableRandom will need to have a method to set/get state which can be used by any project that needs to get reproducible numbers from the module-level generator. While this is an implementation detail, many generators have much smaller states than MT199

Re: [Numpy-discussion] Prototype CorePRNG implementation feedback (Neal Becker)

2018-02-22 Thread Kevin Sheppard
> > > What is the syntax to construct an initialized generator? > RandomGenerator(Xoroshiro128(my_seed))? > > Not 100% certain on this. There was talk in the earlier thread that seed should be killed, although I wasn't clear what mathod would be preferrable to get easily reproducible random number

[Numpy-discussion] Prototype CorePRNG implementation feedback

2018-02-22 Thread Kevin Sheppard
I have implemented a working prototype of a pluggable RandomState. The repo is located at https://github.com/bashtage/core-prng. The main design has a small PRNG (currently only SplitMix64 and Xoroshiro128 are implemented) that does not have any public functions available to generate random numbe

Re: [Numpy-discussion] Moving NumPy's PRNG Forward

2018-01-29 Thread Kevin Sheppard
om: Robert Kern > To: Discussion of Numerical Python > Subject: Re: [Numpy-discussion] Moving NumPy's PRNG Forward > Message-ID: > w...@mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > On Sat, Jan 27, 2018 at 1:14 AM, Kevin Sheppard <

Re: [Numpy-discussion] Moving NumPy's PRNG Forward

2018-01-26 Thread Kevin Sheppard
I am a firm believer that the current situation is not sustainable. There are a lot of improvements that can practically be incorporated. While many of these are performance related, there are also improvements in accuracy over some ranges of parameters that cannot be incorporated. I also think t

[Numpy-discussion] ENH: Add complex random normal generator (PR #9561)

2017-09-08 Thread Kevin Sheppard
I would like to add a complex random normal generator to mtrand/RandomState. A scalar complex normal is a (double) bivariate normal. The main motivation is to simplify the construction of complex normals where are generally parameterized in terms of three values: location, covariance and relation