[Numpy-discussion] ANN: python-blosc 1.0.4 released
= Announcing python-blosc 1.0.4 = What is it? === A Python wrapper for the Blosc compression library. Blosc (http://blosc.pytables.org) is a high performance compressor optimized for binary data. It has been designed to transmit data to the processor cache faster than the traditional, non-compressed, direct memory fetch approach via a memcpy() OS call. Blosc works well for compressing numerical arrays that contains data with relatively low entropy, like sparse data, time series, grids with regular-spaced values, etc. python-blosc is a Python package that wraps it. What is new? Optimized the amount of data copied during compression (using _PyBytes_Resize() now instead of previous PyBytes_FromStringAndSize()). This leads to improvements in compression speed ranging from 1.2x for highly compressible chunks up to 7x for mostly uncompressible data. Thanks to Valentin Haenel for this nice contribution. For more info, you can see the release notes in: https://github.com/FrancescAlted/python-blosc/wiki/Release-notes More docs and examples are available in the Quick User's Guide wiki page: https://github.com/FrancescAlted/python-blosc/wiki/Quick-User's-Guide Download sources Go to: http://github.com/FrancescAlted/python-blosc and download the most recent release from there. Blosc is distributed using the MIT license, see LICENSES/BLOSC.txt for details. Mailing list There is an official mailing list for Blosc at: bl...@googlegroups.com http://groups.google.es/group/blosc -- Francesc Alted ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Change in behavior of np.concatenate for upcoming release
Hi,
On Thu, Sep 13, 2012 at 7:00 PM, Matthew Brett matthew.br...@gmail.com wrote:
Hi,
On Thu, Sep 13, 2012 at 11:31 AM, Matthew Brett matthew.br...@gmail.com
wrote:
On Wed, Sep 12, 2012 at 4:19 PM, Nathaniel Smith n...@pobox.com wrote:
On Wed, Sep 12, 2012 at 2:46 PM, Matthew Brett matthew.br...@gmail.com
wrote:
Hi,
I just noticed that this works for numpy 1.6.1:
In [36]: np.concatenate(([2, 3], [1]), 1)
Out[36]: array([2, 3, 1])
but the beta release branch:
In [3]: np.concatenate(([2, 3], [1]), 1)
---
IndexErrorTraceback (most recent call last)
/Users/mb312/ipython-input-3-0fa244c8aaa8 in module()
1 np.concatenate(([2, 3], [1]), 1)
IndexError: axis 1 out of bounds [0, 1)
In the interests of backward compatibility maybe it would be better to
raise a warning for this release, rather than an error?
Yep, that'd be a good idea. Want to write a patch? :-)
https://github.com/numpy/numpy/pull/440
Thinking about the other thread, and the 'number of elements' check, I
noticed this:
In [51]: np.__version__
Out[51]: '1.6.1'
In [52]: r4 = range(4)
In [53]: r3 = range(3)
In [54]: np.concatenate((r4, r3), None)
Out[54]: array([0, 1, 2, 3, 0, 1, 2])
but:
In [46]: np.__version__
Out[46]: '1.7.0rc1.dev-ea23de8'
In [47]: np.concatenate((r4, r3), None)
---
ValueErrorTraceback (most recent call last)
/Users/mb312/tmp/ipython-input-47-e354b8880702 in module()
1 np.concatenate((r4, r3), None)
ValueError: all the input arrays must have same number of elements
The change requiring the same number of elements appears to have been
added explicitly by Mark in commit 9194b3af . Mark - what was the
reason for that check?
Appealing for anyone who might understand that part of the code :
there's a check in multiarraymodule.c at around line 477:
/*
* Figure out the final concatenated shape starting from the first
* array's shape.
*/
for (iarrays = 1; iarrays narrays; ++iarrays) {
if (PyArray_SIZE(arrays[iarrays]) != shape[1]) {
PyErr_SetString(PyExc_ValueError,
all the input arrays must have same
number of elements);
return NULL;
}
}
I don't understand the following code so I don't know what this check is for...
Cheers,
Matthew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] John Hunter has been awarded the first Distinguished Service Award by the PSF
Hi folks, you may have already seen this, but in case you haven't, I'm thrilled to share that the Python Software Foundation has just created its newest and highest distinction, the Distinguished Service Award, and has chosen John as its first recipient: http://pyfound.blogspot.com/2012/09/announcing-2012-distinctive-service.html This is a fitting tribute to his many contributions. Cheers, f ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Change in behavior of np.concatenate for upcoming release
I think there is something wrong with the implementation.. I would expect each incoming array in PyArray_ConcatenateFlattenedArrays to be flattened and the sizes of all of them added into a one-dimensional shape. Now the shape is two-dimensional, which does not make sense to me. Also the requirement that all sizes must be equal between the incoming arrays only makes sense when you want to stack them into a two-dimensional array, which makes it unnecessarily complicated. The difficulty here is to use PyArray_CopyAsFlat without having to transform/copy each incoming array to the priority dtype, because they can have different item sizes between them, but other than that it should be pretty straightforward, imo. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
