Re: [Numpy-discussion] Google Season of Docs

y is also encouraged to participate with suggestions and > comments. You can check out the timeline and more details about GSoD here: > https://developers.google.com/season-of-docs/docs/timeline > > Cheers, > > Melissa > -- next part -- > An HTML att

Re: [Numpy-discussion] bad CRC errors when using np.savez, only sometimes though!

Is it zlib or zipfile? On Fri, May 14, 2021 at 11:38 AM Benjamin Root wrote: > Isaac, > > What I mean is that your bug might be similar to the savemat() bug that > was fixed in scipy in 2019. Completely different functions, but both > functions need to properly interact with zlib in order to wor

Re: [Numpy-discussion] Google Season of Docs

On Fri, May 14, 2021 at 8:47 PM Stefan van der Walt wrote: > On Fri, May 14, 2021, at 11:41, Melissa Mendonça wrote: > > I am happy to announce we found our technical writer for the Google Season > of Docs program. After looking at all submitted proposals we have decided > to hire Mukulika. You c

Re: [Numpy-discussion] Google Season of Docs

On Fri, May 14, 2021, at 11:41, Melissa Mendonça wrote: > I am happy to announce we found our technical writer for the Google Season of > Docs program. After looking at all submitted proposals we have decided to > hire Mukulika. You can see her Statement of Interest here: > https://mail.python.o

[Numpy-discussion] Google Season of Docs

Hello, all! I am happy to announce we found our technical writer for the Google Season of Docs program. After looking at all submitted proposals we have decided to hire Mukulika. You can see her Statement of Interest here: https://mail.python.org/pipermail/numpy-discussion/2021-May/081746.html We

Re: [Numpy-discussion] bad CRC errors when using np.savez, only sometimes though!

If changing the on-disk format is an option, I would suggest h5py which allows to save numpy arrays in HDF5 format. Stefano On 14 May 2021, at 16:22, numpy-discussion-requ...@python.org wrote: Aside from writing m

Re: [Numpy-discussion] bad CRC errors when using np.savez, only sometimes though!

Isaac, What I mean is that your bug might be similar to the savemat() bug that was fixed in scipy in 2019. Completely different functions, but both functions need to properly interact with zlib in order to work properly. On Fri, May 14, 2021 at 10:22 AM Isaac Gerg wrote: > Hi Ben, I am not sur

Re: [Numpy-discussion] bad CRC errors when using np.savez, only sometimes though!

You could use pandas if your arrays are 2d to serialize using parquet or HDF.  If your arrays are 3d, you could use xarray and NetCDF. I think in both cases if your arrays are standard dtypes then you should be able to avoid copies. Both support compression. Kevin  From: Isaac GergSent: Friday, May

Re: [Numpy-discussion] bad CRC errors when using np.savez, only sometimes though!

Hi Ben, I am not sure. However, in looking at the dates, it looks like that was fixed in scipy as of 2019. Would you recommend using the scipy save interface as opposed to the numpy one? On Fri, May 14, 2021 at 10:16 AM Benjamin Root wrote: > Perhaps it is a similar bug as this one? > https:/

Re: [Numpy-discussion] bad CRC errors when using np.savez, only sometimes though!

Perhaps it is a similar bug as this one? https://github.com/scipy/scipy/issues/6999 Basically, it turned out that the CRC was getting computed on an unflushed buffer, or something like that. On Fri, May 14, 2021 at 10:05 AM Isaac Gerg wrote: > I am using 1.19.5 on Windows 10 using Python 3.8.6

[Numpy-discussion] bad CRC errors when using np.savez, only sometimes though!

I am using 1.19.5 on Windows 10 using Python 3.8.6 (tags/v3.8.6:db45529, Sep 23 2020, 15:52:53) [MSC v.1927 64 bit (AMD64)]. I have two python processes running (i.e. no threads) which do independent processing jobs and NOT writing to the same directories. Each process runs for 5-10 hours and the