[Python-announce] [ANN} Numba 0.58.0 and llvmlite 0.41.0
Dear *, I am happy to announce the release of Numba 0.58.0 and llvmlite 0.41.0. Please point your browsers at the following link for more information: https://numba.discourse.group/t/ann-numba-0-58-0-and-llvmlite-0-41-0/2148 V- ___ Python-announce-list mailing list -- python-announce-list@python.org To unsubscribe send an email to python-announce-list-le...@python.org https://mail.python.org/mailman3/lists/python-announce-list.python.org/ Member address: arch...@mail-archive.com
[Python-announce] [ANN] Numba 0.57.1 / llvmlite 0.40.1
Dear *, I am happy to announce the following patch releases: * Numba 0.57.1 * llvmlite 0.40.1 Please point your bowsers at our discourse for more information: https://numba.discourse.group/t/ann-numba-0-57-1/1996 https://numba.discourse.group/t/ann-llvmlite-0-40-1/1995 Best, V- ___ Python-announce-list mailing list -- python-announce-list@python.org To unsubscribe send an email to python-announce-list-le...@python.org https://mail.python.org/mailman3/lists/python-announce-list.python.org/ Member address: arch...@mail-archive.com
[Python-announce] [ANN] Numba 0.56.4
Dear all, on behalf of the Numba development community, I am happy to announce the release of Numba 0.56.4. This is a bugfix release to fix a regression in the CUDA target in relation to the .view() method on CUDA device arrays that is present when using NumPy version 1.23.0 or later. For more information, please point your browser at: https://numba.discourse.group/t/ann-numba-0-56-4/1632 EOM ___ Python-announce-list mailing list -- python-announce-list@python.org To unsubscribe send an email to python-announce-list-le...@python.org https://mail.python.org/mailman3/lists/python-announce-list.python.org/ Member address: arch...@mail-archive.com
[Python-announce] [ANN] Numba 0.56.3
Dear all, on behalf of the Numba development community, I am happy to announce the release of Numba 0.56.3. This is a bugfix release to remove the version restriction applied to the setuptools package and to fix a bug in the CUDA target in relation to copying zero length device arrays to zero length host arrays. For more information, please point your browser at: https://numba.discourse.group/t/ann-numba-0-56-3/1600 EOM ___ Python-announce-list mailing list -- python-announce-list@python.org To unsubscribe send an email to python-announce-list-le...@python.org https://mail.python.org/mailman3/lists/python-announce-list.python.org/ Member address: arch...@mail-archive.com
[Python-announce] [ANN] Numba 0.56.1 and llvmlite 0.39.1
Dear all, in behalf of the Numba team I am happy to announce the patch releases: Numba 0.56.1 and llvmlite 0.39.1. Besides the usual bug fixes, this Numba release adds support for NumPy 1.23. For more information about Numba, please point your browsers at: https://numba.pydata.org/ For more information about the releases, please point your browsers at: https://numba.discourse.group/t/ann-numba-0-56-2-llvmlite-0-39-1/1542 Best wishes and have a great day! V- ___ Python-announce-list mailing list -- python-announce-list@python.org To unsubscribe send an email to python-announce-list-le...@python.org https://mail.python.org/mailman3/lists/python-announce-list.python.org/ Member address: arch...@mail-archive.com
[Python-announce] [ANN] Numba 0.56.0 and llvmlite 0.39.0
Dear all, Numba 0.56.0 and llvmlite 0.39.0 have been released. Please point your browsers at our Discourse for more information: https://numba.discourse.group/t/ann-numba-0-56-0/1461 https://numba.discourse.group/t/ann-llvmlite-0-39-0/1460 Best, V- ___ Python-announce-list mailing list -- python-announce-list@python.org To unsubscribe send an email to python-announce-list-le...@python.org https://mail.python.org/mailman3/lists/python-announce-list.python.org/ Member address: arch...@mail-archive.com
[Python-announce] [ANN] Numba 0.55.2 and llvmlite 0.38.1
Dear all, On behalf of the Numba team, I would like to announce that Numba 0.55.2 and llvmlite 0.38.1 have been released. Please point your browsers at: https://numba.discourse.group/t/ann-numba-0-55-2/1353/3 .. and ... https://numba.discourse.group/t/llvmlite-0-38-1/1340 .. for more information. V- ___ Python-announce-list mailing list -- python-announce-list@python.org To unsubscribe send an email to python-announce-list-le...@python.org https://mail.python.org/mailman3/lists/python-announce-list.python.org/ Member address: arch...@mail-archive.com
[Python-announce] [ANN] Numba 0.55.1
Dear *, on behalf of the Numba team, I am happy to announce that Numba 0.55.1 has become available! This is a bugfix release that closes all the remaining issues from the accelerated release of 0.55.0 and also any release critical regressions discovered since then. For more information, please point your browsers at: https://numba.discourse.group/t/ann-numba-0-55-1/1161 Best, V- ___ Python-announce-list mailing list -- python-announce-list@python.org To unsubscribe send an email to python-announce-list-le...@python.org https://mail.python.org/mailman3/lists/python-announce-list.python.org/ Member address: arch...@mail-archive.com
[Python-announce] Numba 0.55.0 and llvmlite 0.38.0 are out!
Dear all, 0.55.0 and llvmlite 0.38.0 have been released. Please point your browsers at our discourse for more information: https://numba.discourse.group/t/numba-0-55-0-and-llvmlite-0-38-0-final-release/1136 Best, V- ___ Python-announce-list mailing list -- python-announce-list@python.org To unsubscribe send an email to python-announce-list-le...@python.org https://mail.python.org/mailman3/lists/python-announce-list.python.org/ Member address: arch...@mail-archive.com
[ANN] python-blosc v1.8.0
= Announcing python-blosc 1.8.0 = What is new? This is a maintenance and fetaure release. A regression affecting windows users has been fixed by Robert McLeod and support for older C compilers has been contributed by Nicholas Devenish. Also, c-blosc v1.16.2 has been included and support for the new `cbuffer_validate` is included. Lastly there have been several minor improvements and cleanups as usual. For more info, you can have a look at the release notes in: https://github.com/Blosc/python-blosc/blob/master/RELEASE_NOTES.rst More docs and examples are available in the documentation site: http://python-blosc.blosc.org What is it? === Blosc (http://www.blosc.org) is a high performance compressor optimized for binary data. It has been designed to transmit data to the processor cache faster than the traditional, non-compressed, direct memory fetch approach via a memcpy() OS call. Blosc works well for compressing numerical arrays that contains data with relatively low entropy, like sparse data, time series, grids with regular-spaced values, etc. python-blosc (http://python-blosc.blosc.org/) is the Python wrapper for the Blosc compression library, with added functions (`compress_ptr()` and `pack_array()`) for efficiently compressing NumPy arrays, minimizing the number of memory copies during the process. python-blosc can be used to compress in-memory data buffers for transmission to other machines, persistence or just as a compressed cache. There is also a handy tool built on top of python-blosc called Bloscpack (https://github.com/Blosc/bloscpack). It features a commmand line interface that allows you to compress large binary datafiles on-disk. It also comes with a Python API that has built-in support for serializing and deserializing Numpy arrays both on-disk and in-memory at speeds that are competitive with regular Pickle/cPickle machinery. Sources repository == The sources and documentation are managed through github services at: http://github.com/Blosc/python-blosc **Enjoy data!** -- https://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
[ANN] python-blosc 1.7.0
= Announcing python-blosc 1.7.0 = What is new? This is a maintenance release which takes care of several housekeepting tasks. Support for older versions of Python (2.6 and 3.3) has been removed from the codebase. A new version of C-Blosc (1.5.1) that now passes all unit and integration tests across all supported platforms has been included. Finally, a the vendored cpuinfo.py has been upgraded and the automatic tests on Windows via Appveyor have been upgraded to include a larger variety of Windows/Python combinations. A big thank you goes out to Daniel Stender from the Debian project for his continued efforts to package the Blosc stack -- including python-blosc -- for Debian. This also means it is likely that a recent version of python-blosc will be included in Buster. For more info, you can have a look at the release notes in: https://github.com/Blosc/python-blosc/blob/master/RELEASE_NOTES.rst More docs and examples are available in the documentation site: http://python-blosc.blosc.org What is it? === Blosc (http://www.blosc.org) is a high performance compressor optimized for binary data. It has been designed to transmit data to the processor cache faster than the traditional, non-compressed, direct memory fetch approach via a memcpy() OS call. Blosc works well for compressing numerical arrays that contains data with relatively low entropy, like sparse data, time series, grids with regular-spaced values, etc. python-blosc (http://python-blosc.blosc.org/) is the Python wrapper for the Blosc compression library, with added functions (`compress_ptr()` and `pack_array()`) for efficiently compressing NumPy arrays, minimizing the number of memory copies during the process. python-blosc can be used to compress in-memory data buffers for transmission to other machines, persistence or just as a compressed cache. There is also a handy tool built on top of python-blosc called Bloscpack (https://github.com/Blosc/bloscpack). It features a commmand line interface that allows you to compress large binary datafiles on-disk. It also comes with a Python API that has built-in support for serializing and deserializing Numpy arrays both on-disk and in-memory at speeds that are competitive with regular Pickle/cPickle machinery. Sources repository == The sources and documentation are managed through github services at: http://github.com/Blosc/python-blosc **Enjoy data!** -- https://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
[ANN] python-blosc 1.6.2
= Announcing python-blosc 1.6.2 = What is new? The `import numpy` statement in `toplevel.py` has been moved to a later point. This makes python-blosc usable without needing numpy once again. This behaviour changed in 1.6.1 and has now been reversed to restore the old behaviour. For more info, you can have a look at the release notes in: https://github.com/Blosc/python-blosc/blob/master/RELEASE_NOTES.rst More docs and examples are available in the documentation site: http://python-blosc.blosc.org What is it? === Blosc (http://www.blosc.org) is a high performance compressor optimized for binary data. It has been designed to transmit data to the processor cache faster than the traditional, non-compressed, direct memory fetch approach via a memcpy() OS call. Blosc works well for compressing numerical arrays that contains data with relatively low entropy, like sparse data, time series, grids with regular-spaced values, etc. python-blosc (http://python-blosc.blosc.org/) is the Python wrapper for the Blosc compression library, with added functions (`compress_ptr()` and `pack_array()`) for efficiently compressing NumPy arrays, minimizing the number of memory copies during the process. python-blosc can be used to compress in-memory data buffers for transmission to other machines, persistence or just as a compressed cache. There is also a handy tool built on top of python-blosc called Bloscpack (https://github.com/Blosc/bloscpack). It features a commmand line interface that allows you to compress large binary datafiles on-disk. It also comes with a Python API that has built-in support for serializing and deserializing Numpy arrays both on-disk and in-memory at speeds that are competitive with regular Pickle/cPickle machinery. Sources repository == The sources and documentation are managed through github services at: http://github.com/Blosc/python-blosc **Enjoy data!** -- https://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
[ANN] Bloscpack v0.15.0
Announcing Bloscpack v0.15.0 What is new? Two new high-level API functions have been added: * 'pack_bytes_to_bytes' * 'unpack_bytes_from_bytes' As you might expect from the naming, these allow you to perform fully in-memory based compression and decompression via the bytes datatype. Additionally there are a few bugfixes, support for python-blosc 1.6.1 and support for Python 3.7. For more info, have a look at the changelog: https://github.com/Blosc/bloscpack#changelog Documentation and examples are available at: https://github.com/Blosc/bloscpack What is it? === Bloscpack is a command-line interface and serialization format for Blosc. Blosc (http://www.blosc.org) is an extremely fast meta-codec designed for high compression speeds. Bloscpack allows you to use Blosc from the command-line to compress and decompress files. Additionally, Bloscpack has a Python-API that allows you to compress and serialize data to a file system. Additionally, Bloscpack supports efficient serialization and de-serialization of Numpy arrays and might in fact be one of the fastest ways to save arrays to disk. Bloscpack uses the Python bindings for Blosc (http://python-blosc.blosc.org/) under the hood. **Enjoy data!** -- https://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
[ANN] python-blosc 1.6.1
= Announcing python-blosc 1.6.1 = What is new? C-Blosc has been updated to 1.14.3. Additionally there have been a number of improvments such as not compiling snappy by default and the implementation of the `get_blocksize()` function. For more info, you can have a look at the release notes in: https://github.com/Blosc/python-blosc/blob/master/RELEASE_NOTES.rst More docs and examples are available in the documentation site: http://python-blosc.blosc.org What is it? === Blosc (http://www.blosc.org) is a high performance compressor optimized for binary data. It has been designed to transmit data to the processor cache faster than the traditional, non-compressed, direct memory fetch approach via a memcpy() OS call. Blosc works well for compressing numerical arrays that contains data with relatively low entropy, like sparse data, time series, grids with regular-spaced values, etc. python-blosc (http://python-blosc.blosc.org/) is the Python wrapper for the Blosc compression library, with added functions (`compress_ptr()` and `pack_array()`) for efficiently compressing NumPy arrays, minimizing the number of memory copies during the process. python-blosc can be used to compress in-memory data buffers for transmission to other machines, persistence or just as a compressed cache. There is also a handy tool built on top of python-blosc called Bloscpack (https://github.com/Blosc/bloscpack). It features a commmand line interface that allows you to compress large binary datafiles on-disk. It also comes with a Python API that has built-in support for serializing and deserializing Numpy arrays both on-disk and in-memory at speeds that are competitive with regular Pickle/cPickle machinery. Sources repository == The sources and documentation are managed through github services at: http://github.com/Blosc/python-blosc **Enjoy data!** -- https://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
[ANN] bcolz v0.10.0
===
Announcing bcolz 0.10.0
===
What's new
==
This is a cleanup-and-refactor release with many internal optimizations
and a few bug fixes. For users, the most important improvement is the
new-and-shiny context manager for bcolz objects. For example for the
ctable constructor::
>>> with bcolz.ctable(np.empty(0, dtype="i4,f8"),
...: rootdir='mydir', mode="w") as ct:
...: for i in xrange(N):
...:ct.append((i, i**2))
...:
>>> bcolz.ctable(rootdir='mydir')
ctable((10,), [('f0', 'http://bcolz.blosc.org/reference.html#top-level-functions.
For a complete list of changes check the release notes at:
https://github.com/Blosc/bcolz/blob/master/RELEASE_NOTES.rst
What it is
==
*bcolz* provides columnar and compressed data containers that can live
either on-disk or in-memory. Column storage allows for efficiently
querying tables with a large number of columns. It also allows for
cheap addition and removal of column. In addition, bcolz objects are
compressed by default for reducing memory/disk I/O needs. The
compression process is carried out internally by Blosc, an
extremely fast meta-compressor that is optimized for binary data. Lastly,
high-performance iterators (like ``iter()``, ``where()``) for querying
the objects are provided.
bcolz can use numexpr internally so as to accelerate many vector and
query operations (although it can use pure NumPy for doing so too).
numexpr optimizes the memory usage and use several cores for doing the
computations, so it is blazing fast. Moreover, since the carray/ctable
containers can be disk-based, and it is possible to use them for
seamlessly performing out-of-memory computations.
bcolz has minimal dependencies (NumPy), comes with an exhaustive test
suite and fully supports both 32-bit and 64-bit platforms. Also, it is
typically tested on both UNIX and Windows operating systems.
Together, bcolz and the Blosc compressor, are finally fulfilling the
promise of accelerating memory I/O, at least for some real scenarios:
http://nbviewer.ipython.org/github/Blosc/movielens-bench/blob/master/querying-ep14.ipynb#Plots
Other users of bcolz are Visualfabriq (http://www.visualfabriq.com/) the
Blaze project (http://blaze.pydata.org/), Quantopian
(https://www.quantopian.com/) and Scikit-Allel
(https://github.com/cggh/scikit-allel) which you can read more about by
pointing your browser at the links below.
* Visualfabriq:
* *bquery*, A query and aggregation framework for Bcolz:
* https://github.com/visualfabriq/bquery
* Blaze:
* Notebooks showing Blaze + Pandas + BColz interaction:
* http://nbviewer.ipython.org/url/blaze.pydata.org/notebooks/timings-csv.ipynb
*
http://nbviewer.ipython.org/url/blaze.pydata.org/notebooks/timings-bcolz.ipynb
* Quantopian:
* Using compressed data containers for faster backtesting at scale:
* https://quantopian.github.io/talks/NeedForSpeed/slides.html
* Scikit-Allel
* Provides an alternative backend to work with compressed arrays
* https://scikit-allel.readthedocs.org/en/latest/bcolz.html
Installing
==
bcolz is in the PyPI repository, so installing it is easy::
$ pip install -U bcolz
Resources
=
Visit the main bcolz site repository at:
http://github.com/Blosc/bcolz
Manual:
http://bcolz.blosc.org
Home of Blosc compressor:
http://blosc.org
User's mail list:
bc...@googlegroups.com
http://groups.google.com/group/bcolz
License is the new BSD:
https://github.com/Blosc/bcolz/blob/master/LICENSES/BCOLZ.txt
Release notes can be found in the Git repository:
https://github.com/Blosc/bcolz/blob/master/RELEASE_NOTES.rst
**Enjoy data!**
--
https://mail.python.org/mailman/listinfo/python-announce-list
Support the Python Software Foundation:
http://www.python.org/psf/donations/
[ANN] bcolz v0.9.0
== Announcing bcolz 0.9.0 == What's new == This is mostly a smallish feature and bugfix release. One large topic was implementing 'addcol' and 'delcol' to properly handle on-disk tables. 'addcol' now has a new keyword argument 'move' that allows you to specify if you want to move or copy the data. 'delcol' has a new keyword argument 'keep' which allows you preserve the data on disk when removing a column. Additionally, ctable now supports an 'auto_flush' keyword that makes it flush to disk automatically after any methods that may write data. Another important aspect is handling the GIL. In this release, we do keep the GIL while calling Blosc compress and decompress in order to support lock-free operation of newer Blosc versions (1.5.x and beyond) that no longer have a global state. Furthermore we now distribute the 'carray_ext.pxd' as part of the package via PyPi to ease building applications on bcolz, for example *bquery*. Finally, the Sphinx based API documentation is now autogenerated from the docstrings in the Python sources. For the full list, please check the release notes at: https://github.com/Blosc/bcolz/blob/v0.9.0/RELEASE_NOTES.rst What it is == *bcolz* provides columnar and compressed data containers that can live either on-disk or in-memory. Column storage allows for efficiently querying tables with a large number of columns. It also allows for cheap addition and removal of column. In addition, bcolz objects are compressed by default for reducing memory/disk I/O needs. The compression process is carried out internally by Blosc, an extremely fast meta-compressor that is optimized for binary data. Lastly, high-performance iterators (like ``iter()``, ``where()``) for querying the objects are provided. bcolz can use numexpr internally so as to accelerate many vector and query operations (although it can use pure NumPy for doing so too). numexpr optimizes the memory usage and use several cores for doing the computations, so it is blazing fast. Moreover, since the carray/ctable containers can be disk-based, and it is possible to use them for seamlessly performing out-of-memory computations. bcolz has minimal dependencies (NumPy), comes with an exhaustive test suite and fully supports both 32-bit and 64-bit platforms. Also, it is typically tested on both UNIX and Windows operating systems. Together, bcolz and the Blosc compressor, are finally fulfilling the promise of accelerating memory I/O, at least for some real scenarios: http://nbviewer.ipython.org/github/Blosc/movielens-bench/blob/master/querying-ep14.ipynb#Plots Other users of bcolz are Visualfabriq (http://www.visualfabriq.com/) the Blaze project (http://blaze.pydata.org/), Quantopian (https://www.quantopian.com/) and Scikit-Allel (https://github.com/cggh/scikit-allel) which you can read more about by pointing your browser at the links below. * Visualfabriq: * *bquery*, A query and aggregation framework for Bcolz: * https://github.com/visualfabriq/bquery * Blaze: * Notebooks showing Blaze + Pandas + BColz interaction: * http://nbviewer.ipython.org/url/blaze.pydata.org/notebooks/timings-csv.ipynb * http://nbviewer.ipython.org/url/blaze.pydata.org/notebooks/timings-bcolz.ipynb * Quantopian: * Using compressed data containers for faster backtesting at scale: * https://quantopian.github.io/talks/NeedForSpeed/slides.html * Scikit-Allel * Provides an alternative backend to work with compressed arrays * https://scikit-allel.readthedocs.org/en/latest/bcolz.html Installing == bcolz is in the PyPI repository, so installing it is easy:: $ pip install -U bcolz Resources = Visit the main bcolz site repository at: http://github.com/Blosc/bcolz Manual: http://bcolz.blosc.org Home of Blosc compressor: http://blosc.org User's mail list: bc...@googlegroups.com http://groups.google.com/group/bcolz License is the new BSD: https://github.com/Blosc/bcolz/blob/master/LICENSES/BCOLZ.txt Release notes can be found in the Git repository: https://github.com/Blosc/bcolz/blob/master/RELEASE_NOTES.rst **Enjoy data!** -- https://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
[ANN] python-blosc v1.2.5
= Announcing python-blosc 1.2.5 = What is new? This release contains support for Blosc v1.5.4 including changes to how the GIL is kept. This was required because Blosc was refactored in the v1.5.x line to remove global variables and to use context objects instead. As such, it became necessary to keep the GIL while calling Blosc from Python code that uses the multiprocessing module. In addition, is now possible to change the blocksize used by Blosc using ``set_blocksize``. When using this however, bear in mind that the blocksize has been finely tuned to be a good default value and that randomly messing with this value may have unforeseen and unpredictable consequences on the performance of Blosc. Additionally, we can now compile on Posix architectures, thanks again to Andreas Schwab for that one. For more info, you can have a look at the release notes in: https://github.com/Blosc/python-blosc/wiki/Release-notes More docs and examples are available in the documentation site: http://python-blosc.blosc.org What is it? === Blosc (http://www.blosc.org) is a high performance compressor optimized for binary data. It has been designed to transmit data to the processor cache faster than the traditional, non-compressed, direct memory fetch approach via a memcpy() OS call. Blosc is the first compressor that is meant not only to reduce the size of large datasets on-disk or in-memory, but also to accelerate object manipulations that are memory-bound (http://www.blosc.org/docs/StarvingCPUs.pdf). See http://www.blosc.org/synthetic-benchmarks.html for some benchmarks on how much speed it can achieve in some datasets. Blosc works well for compressing numerical arrays that contains data with relatively low entropy, like sparse data, time series, grids with regular-spaced values, etc. python-blosc (http://python-blosc.blosc.org/) is the Python wrapper for the Blosc compression library. There is also a handy tool built on Blosc called Bloscpack (https://github.com/Blosc/bloscpack). It features a commmand line interface that allows you to compress large binary datafiles on-disk. It also comes with a Python API that has built-in support for serializing and deserializing Numpy arrays both on-disk and in-memory at speeds that are competitive with regular Pickle/cPickle machinery. Installing == python-blosc is in PyPI repository, so installing it is easy: $ pip install -U blosc # yes, you should omit the python- prefix Download sources The sources are managed through github services at: http://github.com/Blosc/python-blosc Documentation = There is Sphinx-based documentation site at: http://python-blosc.blosc.org/ Mailing list There is an official mailing list for Blosc at: bl...@googlegroups.com http://groups.google.es/group/blosc Licenses Both Blosc and its Python wrapper are distributed using the MIT license. See: https://github.com/Blosc/python-blosc/blob/master/LICENSES for more details. **Enjoy data!** -- https://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
[ANN] bcolz v0.8.0
== Announcing bcolz 0.8.0 == What's new == This version adds a public API in the form of a Cython definitions file (``carray_ext.pxd``) for the ``carray`` class! This means, other libraries can use the Cython definitions to build more complex programs using the objects provided by bcolz. In fact, this feature was specifically requested and there already exists a nascent application called *bquery* (https://github.com/visualfabriq/bquery) which provides an efficient out-of-core groupby implementation for the ``ctable`` object Because this is a fairly sweeping change, the minor version number was incremented and no additional major features or bugfixes were added to this release. We kindly ask any users of bcolz to try this version carefully and report back any issues, bugs, or even slow-downs you experience. I.e. please, please be careful when deploying this version into production. Many, many kudos to Francesc Elies and Carst Vaartjes of Visualfabriq for their hard work, continued effort to push this feature and their work on bquery which makes use of it! What it is == *bcolz* provides columnar and compressed data containers that can live either on-disk or in-memory. Column storage allows for efficiently querying tables with a large number of columns. It also allows for cheap addition and removal of column. In addition, bcolz objects are compressed by default for reducing memory/disk I/O needs. The compression process is carried out internally by Blosc, an extremely fast meta-compressor that is optimized for binary data. Lastly, high-performance iterators (like ``iter()``, ``where()``) for querying the objects are provided. bcolz can use numexpr internally so as to accelerate many vector and query operations (although it can use pure NumPy for doing so too). numexpr optimizes the memory usage and use several cores for doing the computations, so it is blazing fast. Moreover, since the carray/ctable containers can be disk-based, and it is possible to use them for seamlessly performing out-of-memory computations. bcolz has minimal dependencies (NumPy), comes with an exhaustive test suite and fully supports both 32-bit and 64-bit platforms. Also, it is typically tested on both UNIX and Windows operating systems. Together, bcolz and the Blosc compressor, are finally fulfilling the promise of accelerating memory I/O, at least for some real scenarios: http://nbviewer.ipython.org/github/Blosc/movielens-bench/blob/master/querying-ep14.ipynb#Plots Other users of bcolz are Visualfabriq (http://www.visualfabriq.com/) the Blaze project (http://blaze.pydata.org/) and Quantopian (https://www.quantopian.com/) which you can read more about by pointing your browser at the links below. * Visualfabriq: * *bquery*, A query and aggregation framework for Bcolz: * https://github.com/visualfabriq/bquery * Blaze: * Notebooks showing Blaze + Pandas + BColz interaction: * http://nbviewer.ipython.org/url/blaze.pydata.org/notebooks/timings-csv.ipynb * http://nbviewer.ipython.org/url/blaze.pydata.org/notebooks/timings-bcolz.ipynb * Quantopian: * Using compressed data containers for faster backtesting at scale: * https://quantopian.github.io/talks/NeedForSpeed/slides.html Installing == bcolz is in the PyPI repository, so installing it is easy:: $ pip install -U bcolz Resources = Visit the main bcolz site repository at: http://github.com/Blosc/bcolz Manual: http://bcolz.blosc.org Home of Blosc compressor: http://blosc.org User's mail list: bc...@googlegroups.com http://groups.google.com/group/bcolz License is the new BSD: https://github.com/Blosc/bcolz/blob/master/LICENSES/BCOLZ.txt Release notes can be found in the Git repository: https://github.com/Blosc/bcolz/blob/master/RELEASE_NOTES.rst **Enjoy data!** -- https://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
[ANN] bcolz 0.7.3
== Announcing bcolz 0.7.3 == What's new == This release includes the support for pickling persistent carray/ctable objects contributed by Matthew Rocklin. Also, the included version of Blosc is updated to ``v1.5.2``. Lastly, several minor issues and typos have been fixed, please see the release notes for details. ``bcolz`` is a renaming of the ``carray`` project. The new goals for the project are to create simple, yet flexible compressed containers, that can live either on-disk or in-memory, and with some high-performance iterators (like `iter()`, `where()`) for querying them. Together, bcolz and the Blosc compressor, are finally fulfilling the promise of accelerating memory I/O, at least for some real scenarios: http://nbviewer.ipython.org/github/Blosc/movielens-bench/blob/master/querying-ep14.ipynb#Plots For more detailed info, see the release notes in: https://github.com/Blosc/bcolz/wiki/Release-Notes What it is == bcolz provides columnar and compressed data containers. Column storage allows for efficiently querying tables with a large number of columns. It also allows for cheap addition and removal of column. In addition, bcolz objects are compressed by default for reducing memory/disk I/O needs. The compression process is carried out internally by Blosc, a high-performance compressor that is optimized for binary data. bcolz can use numexpr internally so as to accelerate many vector and query operations (although it can use pure NumPy for doing so too). numexpr optimizes the memory usage and use several cores for doing the computations, so it is blazing fast. Moreover, the carray/ctable containers can be disk-based, and it is possible to use them for seamlessly performing out-of-memory computations. bcolz has minimal dependencies (NumPy), comes with an exhaustive test suite and fully supports both 32-bit and 64-bit platforms. Also, it is typically tested on both UNIX and Windows operating systems. Installing == bcolz is in the PyPI repository, so installing it is easy:: $ pip install -U bcolz Resources = Visit the main bcolz site repository at: http://github.com/Blosc/bcolz Manual: http://bcolz.blosc.org Home of Blosc compressor: http://blosc.org User's mail list: bc...@googlegroups.com http://groups.google.com/group/bcolz License is the new BSD: https://github.com/Blosc/bcolz/blob/master/LICENSES/BCOLZ.txt **Enjoy data!** -- https://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
[ANN] bcolz 0.7.2
== Announcing bcolz 0.7.2 == What's new == This is a maintenance release that fixes various bits and pieces. Importantly, compatibility with Numpy 1.9 and Cython 0.21 has been fixed and the test suit no longer segfaults on 32 bit UNIX. Feature-wise a new ``carray.view()`` method has been introduced which allows carrays to share the same raw data. ``bcolz`` is a renaming of the ``carray`` project. The new goals for the project are to create simple, yet flexible compressed containers, that can live either on-disk or in-memory, and with some high-performance iterators (like `iter()`, `where()`) for querying them. Together, bcolz and the Blosc compressor, are finally fulfilling the promise of accelerating memory I/O, at least for some real scenarios: http://nbviewer.ipython.org/github/Blosc/movielens-bench/blob/master/querying-ep14.ipynb#Plots For more detailed info, see the release notes in: https://github.com/Blosc/bcolz/wiki/Release-Notes What it is == bcolz provides columnar and compressed data containers. Column storage allows for efficiently querying tables with a large number of columns. It also allows for cheap addition and removal of column. In addition, bcolz objects are compressed by default for reducing memory/disk I/O needs. The compression process is carried out internally by Blosc, a high-performance compressor that is optimized for binary data. bcolz can use numexpr internally so as to accelerate many vector and query operations (although it can use pure NumPy for doing so too). numexpr optimizes the memory usage and use several cores for doing the computations, so it is blazing fast. Moreover, the carray/ctable containers can be disk-based, and it is possible to use them for seamlessly performing out-of-memory computations. bcolz has minimal dependencies (NumPy), comes with an exhaustive test suite and fully supports both 32-bit and 64-bit platforms. Also, it is typically tested on both UNIX and Windows operating systems. Installing == bcolz is in the PyPI repository, so installing it is easy:: $ pip install -U bcolz Resources = Visit the main bcolz site repository at: http://github.com/Blosc/bcolz Manual: http://bcolz.blosc.org Home of Blosc compressor: http://blosc.org User's mail list: bc...@googlegroups.com http://groups.google.com/group/bcolz License is the new BSD: https://github.com/Blosc/bcolz/blob/master/LICENSES/BCOLZ.txt **Enjoy data!** -- https://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
