Hello community, here is the log from the commit of package python3-numexpr for openSUSE:Factory checked in at 2016-06-19 11:08:41 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/python3-numexpr (Old) and /work/SRC/openSUSE:Factory/.python3-numexpr.new (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "python3-numexpr" Changes: -------- --- /work/SRC/openSUSE:Factory/python3-numexpr/python3-numexpr.changes 2016-05-25 21:24:59.000000000 +0200 +++ /work/SRC/openSUSE:Factory/.python3-numexpr.new/python3-numexpr.changes 2016-06-19 11:08:58.000000000 +0200 @@ -1,0 +2,17 @@ +Sat Jun 18 18:05:31 UTC 2016 - a...@gmx.de + +- update to version 2.6.0: + * Introduced a new re_evaluate() function for re-evaluating the + previous executed array expression without any check. This is + meant for accelerating loops that are re-evaluating the same + expression repeatedly without changing anything else than the + operands. If unsure, use evaluate() which is safer. + * The BLOCK_SIZE1 and BLOCK_SIZE2 constants have been re-checked in + order to find a value maximizing most of the benchmarks in bench/ + directory. The new values (8192 and 16 respectively) give somewhat + better results (~5%) overall. The CPU used for fine tuning is a + relatively new Haswell processor (E3-1240 v3). + * The '--name' flag for setup.py returning the name of the package + is honored now (issue #215). + +------------------------------------------------------------------- @@ -7 +23,0 @@ - Old: ---- numexpr-2.5.2.tar.gz New: ---- numexpr-2.6.0.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ python3-numexpr.spec ++++++ --- /var/tmp/diff_new_pack.1hSqpj/_old 2016-06-19 11:08:59.000000000 +0200 +++ /var/tmp/diff_new_pack.1hSqpj/_new 2016-06-19 11:08:59.000000000 +0200 @@ -17,7 +17,7 @@ Name: python3-numexpr -Version: 2.5.2 +Version: 2.6.0 Release: 0 Url: https://github.com/pydata/numexpr Summary: Fast numerical expression evaluator for NumPy ++++++ numexpr-2.5.2.tar.gz -> numexpr-2.6.0.tar.gz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.5.2/ANNOUNCE.rst new/numexpr-2.6.0/ANNOUNCE.rst --- old/numexpr-2.5.2/ANNOUNCE.rst 2016-04-07 11:35:39.000000000 +0200 +++ new/numexpr-2.6.0/ANNOUNCE.rst 2016-06-01 13:53:04.000000000 +0200 @@ -1,5 +1,5 @@ ========================= - Announcing Numexpr 2.5.2 + Announcing Numexpr 2.6.0 ========================= Numexpr is a fast numerical expression evaluator for NumPy. With it, @@ -21,11 +21,20 @@ What's new ========== -This is a maintenance release shaking some remaining problems with VML -(it is nice to see how Anaconda VML's support helps raising hidden -issues). Now conj() and abs() are actually added as VML-powered -functions, preventing the same problems than log10() before (PR #212); -thanks to Tom Kooij. Upgrading to this release is highly recommended. +This is a minor version bump because it introduces a new function. +Also some minor fine tuning for recent CPUs has been done: + +- Introduced a new re_evaluate() function for re-evaluating the + previous executed array expression without any check. This is meant + for accelerating loops that are re-evaluating the same expression + repeatedly without changing anything else than the operands. If + unsure, use evaluate() which is safer. + +- The BLOCK_SIZE1 and BLOCK_SIZE2 constants have been re-checked in + order to find a value maximizing most of the benchmarks in bench/ + directory. The new values (8192 and 16 respectively) give somewhat + better results (~5%) overall. The CPU used for fine tuning is a + relatively new Haswell processor (E3-1240 v3). In case you want to know more in detail what has changed in this version, see: diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.5.2/PKG-INFO new/numexpr-2.6.0/PKG-INFO --- old/numexpr-2.5.2/PKG-INFO 2016-04-07 11:41:55.000000000 +0200 +++ new/numexpr-2.6.0/PKG-INFO 2016-06-01 13:58:34.000000000 +0200 @@ -1,6 +1,6 @@ Metadata-Version: 1.0 Name: numexpr -Version: 2.5.2 +Version: 2.6.0 Summary: Fast numerical expression evaluator for NumPy Home-page: https://github.com/pydata/numexpr Author: David M. Cooke, Francesc Alted and others diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.5.2/README.rst new/numexpr-2.6.0/README.rst --- old/numexpr-2.5.2/README.rst 2016-04-07 11:40:55.000000000 +0200 +++ new/numexpr-2.6.0/README.rst 2016-05-30 17:29:04.000000000 +0200 @@ -35,30 +35,31 @@ This allows further acceleration of transcendent expressions. -How Numexpr achieves high performance -================================================ +How Numexpr achieves high performance +===================================== -The main reason why Numexpr achieves better performance than NumPy -is that it avoids allocating memory for intermediate results. This +The main reason why Numexpr achieves better performance than NumPy is +that it avoids allocating memory for intermediate results. This results in better cache utilization and reduces memory access in -general. Due to this, Numexpr works best with large arrays. +general. Due to this, Numexpr works best with large arrays. Numexpr parses expressions into its own op-codes that are then used by an integrated computing virtual machine. The array operands are split -into small chunks that easily fit in the cache of the CPU and passed to -the virtual machine. The virtual machine then applies the operations -on each chunk. It's worth noting that all temporaries and constants -in the expression are also chunked. +into small chunks that easily fit in the cache of the CPU and passed +to the virtual machine. The virtual machine then applies the +operations on each chunk. It's worth noting that all temporaries and +constants in the expression are also chunked. The result is that Numexpr can get the most of your machine computing -capabilities for array-wise computations. Common speed-ups with regard -to NumPy are usually between 0.95x (for very simple expressions -like ’a + 1’) and 4x (for relatively complex ones like 'a*b-4.1*a > 2.5*b'), -although much higher speed-ups can be achieved (up to 15x in some cases). - -Numexpr performs best on matrices that do not fit in CPU cache. -In order to get a better idea on the different speed-ups -that can be achieved on your platform, run the provided benchmarks. +capabilities for array-wise computations. Common speed-ups with regard +to NumPy are usually between 0.95x (for very simple expressions like +’a + 1’) and 4x (for relatively complex ones like 'a*b-4.1*a > +2.5*b'), although much higher speed-ups can be achieved (up to 15x in +some cases). + +Numexpr performs best on matrices that do not fit in CPU cache. In +order to get a better idea on the different speed-ups that can be +achieved on your platform, run the provided benchmarks. See more info about how Numexpr works in the `wiki <https://github.com/pydata/numexpr/wiki>`_. @@ -109,9 +110,10 @@ estimations about the memory consumption during the computation of your expressions. -Also, the types in Numexpr conditions are somewhat more restrictive -than those of Python. For instance, the only valid constants for booleans -are `True` and `False`, and they are never automatically cast to integers. +Also, the types in Numexpr conditions are somewhat more restrictive +than those of Python. For instance, the only valid constants for +booleans are `True` and `False`, and they are never automatically cast +to integers. Casting rules @@ -232,6 +234,13 @@ Evaluate a simple array expression element-wise. See docstrings for more info on parameters. Also, see examples above. + * re_evaluate(local_dict=None): + Re-evaluate the previous executed array expression without any + check. This is meant for accelerating loops that are + re-evaluating the same expression repeatedly without changing + anything else than the operands. If unsure, use evaluate() which + is safer. + * test(): Run all the tests in the test suite. * print_versions(): Print the versions of software that numexpr @@ -271,7 +280,7 @@ This call is equivalent to the `vmlSetMode()` in the VML library. -:: +:: * set_vml_num_threads(nthreads): Suggests a maximum number of threads to be used in VML operations. diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.5.2/RELEASE_NOTES.rst new/numexpr-2.6.0/RELEASE_NOTES.rst --- old/numexpr-2.5.2/RELEASE_NOTES.rst 2016-04-03 18:35:07.000000000 +0200 +++ new/numexpr-2.6.0/RELEASE_NOTES.rst 2016-06-01 13:49:26.000000000 +0200 @@ -1,7 +1,26 @@ ====================================== - Release notes for Numexpr 2.5 series + Release notes for Numexpr 2.6 series ====================================== +Changes from 2.5.2 to 2.6.0 +=========================== + +- Introduced a new re_evaluate() function for re-evaluating the + previous executed array expression without any check. This is meant + for accelerating loops that are re-evaluating the same expression + repeatedly without changing anything else than the operands. If + unsure, use evaluate() which is safer. + +- The BLOCK_SIZE1 and BLOCK_SIZE2 constants have been re-checked in + order to find a value maximizing most of the benchmarks in bench/ + directory. The new values (8192 and 16 respectively) give somewhat + better results (~5%) overall. The CPU used for fine tuning is a + relatively new Haswell processor (E3-1240 v3). + +- The '--name' flag for `setup.py` returning the name of the package + is honored now (issue #215). + + Changes from 2.5.1 to 2.5.2 =========================== diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.5.2/numexpr/__init__.py new/numexpr-2.6.0/numexpr/__init__.py --- old/numexpr-2.5.2/numexpr/__init__.py 2016-03-22 22:17:01.000000000 +0100 +++ new/numexpr-2.6.0/numexpr/__init__.py 2016-05-30 13:31:56.000000000 +0200 @@ -38,7 +38,7 @@ import os, os.path import platform from numexpr.expressions import E -from numexpr.necompiler import NumExpr, disassemble, evaluate +from numexpr.necompiler import NumExpr, disassemble, evaluate, re_evaluate from numexpr.tests import test, print_versions from numexpr.utils import ( get_vml_version, set_vml_accuracy_mode, set_vml_num_threads, diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.5.2/numexpr/interpreter.hpp new/numexpr-2.6.0/numexpr/interpreter.hpp --- old/numexpr-2.5.2/numexpr/interpreter.hpp 2016-03-22 22:17:01.000000000 +0100 +++ new/numexpr-2.6.0/numexpr/interpreter.hpp 2016-05-30 11:13:38.000000000 +0200 @@ -99,4 +99,4 @@ int vm_engine_iter_task(NpyIter *iter, npy_intp *memsteps, const vm_params& params, int *pc_error, char **errmsg); -#endif // NUMEXPR_INTERPRETER_HPP \ No newline at end of file +#endif // NUMEXPR_INTERPRETER_HPP diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.5.2/numexpr/necompiler.py new/numexpr-2.6.0/numexpr/necompiler.py --- old/numexpr-2.5.2/numexpr/necompiler.py 2016-04-03 18:31:48.000000000 +0200 +++ new/numexpr-2.6.0/numexpr/necompiler.py 2016-05-30 17:28:35.000000000 +0200 @@ -704,9 +704,28 @@ return [a.value for a in input_order], ex_uses_vml +def getArguments(names, local_dict=None, global_dict=None): + """Get the arguments based on the names.""" + call_frame = sys._getframe(2) + if local_dict is None: + local_dict = call_frame.f_locals + if global_dict is None: + global_dict = call_frame.f_globals + + arguments = [] + for name in names: + try: + a = local_dict[name] + except KeyError: + a = global_dict[name] + arguments.append(numpy.asarray(a)) + return arguments + + # Dictionaries for caching variable names and compiled expressions _names_cache = CacheDict(256) _numexpr_cache = CacheDict(256) +_numexpr_last = {} evaluate_lock = threading.Lock() @@ -754,40 +773,55 @@ like float64 to float32, are allowed. * 'unsafe' means any data conversions may be done. """ + global _numexpr_last + if not isinstance(ex, (str, unicode)): + raise ValueError("must specify expression as a string") + # Get the names for this expression + context = getContext(kwargs, frame_depth=1) + expr_key = (ex, tuple(sorted(context.items()))) + if expr_key not in _names_cache: + _names_cache[expr_key] = getExprNames(ex, context) + names, ex_uses_vml = _names_cache[expr_key] + arguments = getArguments(names, local_dict, global_dict) + + # Create a signature + signature = [(name, getType(arg)) for (name, arg) in + zip(names, arguments)] + + # Look up numexpr if possible. + numexpr_key = expr_key + (tuple(signature),) + try: + compiled_ex = _numexpr_cache[numexpr_key] + except KeyError: + compiled_ex = _numexpr_cache[numexpr_key] = \ + NumExpr(ex, signature, **context) + kwargs = {'out': out, 'order': order, 'casting': casting, + 'ex_uses_vml': ex_uses_vml} + _numexpr_last = dict(ex=compiled_ex, argnames=names, kwargs=kwargs) with evaluate_lock: - if not isinstance(ex, (str, unicode)): - raise ValueError("must specify expression as a string") - # Get the names for this expression - context = getContext(kwargs, frame_depth=1) - expr_key = (ex, tuple(sorted(context.items()))) - if expr_key not in _names_cache: - _names_cache[expr_key] = getExprNames(ex, context) - names, ex_uses_vml = _names_cache[expr_key] - # Get the arguments based on the names. - call_frame = sys._getframe(1) - if local_dict is None: - local_dict = call_frame.f_locals - if global_dict is None: - global_dict = call_frame.f_globals - - arguments = [] - for name in names: - try: - a = local_dict[name] - except KeyError: - a = global_dict[name] - arguments.append(numpy.asarray(a)) + return compiled_ex(*arguments, **kwargs) - # Create a signature - signature = [(name, getType(arg)) for (name, arg) in zip(names, arguments)] - # Look up numexpr if possible. - numexpr_key = expr_key + (tuple(signature),) - try: - compiled_ex = _numexpr_cache[numexpr_key] - except KeyError: - compiled_ex = _numexpr_cache[numexpr_key] = \ - NumExpr(ex, signature, **context) - kwargs = {'out': out, 'order': order, 'casting': casting, - 'ex_uses_vml': ex_uses_vml} - return compiled_ex(*arguments, **kwargs) +def re_evaluate(local_dict=None): + """Re-evaluate the previous executed array expression without any check. + + This is meant for accelerating loops that are re-evaluating the same + expression repeatedly without changing anything else than the operands. + If unsure, use evaluate() which is safer. + + Parameters + ---------- + + local_dict : dictionary, optional + A dictionary that replaces the local operands in current frame. + + """ + try: + compiled_ex = _numexpr_last['ex'] + except KeyError: + raise RuntimeError("not a previous evaluate() execution found") + argnames = _numexpr_last['argnames'] + args = getArguments(argnames, local_dict) + kwargs = _numexpr_last['kwargs'] + with evaluate_lock: + return compiled_ex(*args, **kwargs) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.5.2/numexpr/numexpr_config.hpp new/numexpr-2.6.0/numexpr/numexpr_config.hpp --- old/numexpr-2.5.2/numexpr/numexpr_config.hpp 2016-03-22 22:17:01.000000000 +0100 +++ new/numexpr-2.6.0/numexpr/numexpr_config.hpp 2016-06-01 13:41:46.000000000 +0200 @@ -8,19 +8,14 @@ #endif #ifdef USE_VML -/* The values below have been tuned for a nowadays Core2 processor */ -/* Note: with VML functions a larger block size (e.g. 4096) allows to make use +/* The values below have been tuned for a Haswell processor (E3-1240 v3) */ +/* Note: with VML functions a larger block size (e.g. 8192) allows to make use * of the automatic multithreading capabilities of the VML library */ -#define BLOCK_SIZE1 4096 -#define BLOCK_SIZE2 32 +#define BLOCK_SIZE1 8192 +#define BLOCK_SIZE2 16 #else -/* The values below have been tuned for a nowadays Core2 processor */ -/* Note: without VML available a smaller block size is best, specially - * for the strided and unaligned cases. Recent implementation of - * multithreading make it clear that larger block sizes benefit - * performance (although it seems like we don't need very large sizes - * like VML yet). */ -#define BLOCK_SIZE1 1024 +/* The values below have been tuned for a Haswell processor (E3-1240 v3) */ +#define BLOCK_SIZE1 8192 #define BLOCK_SIZE2 16 #endif @@ -54,4 +49,4 @@ #include "msvc_function_stubs.hpp" #endif -#endif // NUMEXPR_CONFIG_HPP \ No newline at end of file +#endif // NUMEXPR_CONFIG_HPP diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.5.2/numexpr/tests/test_numexpr.py new/numexpr-2.6.0/numexpr/tests/test_numexpr.py --- old/numexpr-2.5.2/numexpr/tests/test_numexpr.py 2016-04-03 18:55:33.000000000 +0200 +++ new/numexpr-2.6.0/numexpr/tests/test_numexpr.py 2016-05-30 17:18:14.000000000 +0200 @@ -28,7 +28,7 @@ from numpy import shape, allclose, array_equal, ravel, isnan, isinf import numexpr -from numexpr import E, NumExpr, evaluate, disassemble, use_vml +from numexpr import E, NumExpr, evaluate, re_evaluate, disassemble, use_vml import unittest @@ -308,6 +308,22 @@ y = evaluate("x") assert_array_equal(x, y) + def test_re_evaluate(self): + a = array([1., 2., 3.]) + b = array([4., 5., 6.]) + c = array([7., 8., 9.]) + x = evaluate("2*a + 3*b*c") + x = re_evaluate() + assert_array_equal(x, array([86., 124., 168.])) + + def test_re_evaluate_dict(self): + a = array([1., 2., 3.]) + b = array([4., 5., 6.]) + c = array([7., 8., 9.]) + x = evaluate("2*a + 3*b*c", local_dict={'a': a, 'b': b, 'c': c}) + x = re_evaluate() + assert_array_equal(x, array([86., 124., 168.])) + # Test for issue #37 if sys.version_info[0] < 3: # In python 3 '/' perforns true division, not integer division. diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.5.2/numexpr/version.py new/numexpr-2.6.0/numexpr/version.py --- old/numexpr-2.5.2/numexpr/version.py 2016-04-07 11:36:02.000000000 +0200 +++ new/numexpr-2.6.0/numexpr/version.py 2016-06-01 13:53:46.000000000 +0200 @@ -8,4 +8,4 @@ # rights to use. #################################################################### -version = '2.5.2' +version = '2.6.0' diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.5.2/numexpr.egg-info/PKG-INFO new/numexpr-2.6.0/numexpr.egg-info/PKG-INFO --- old/numexpr-2.5.2/numexpr.egg-info/PKG-INFO 2016-04-07 11:41:55.000000000 +0200 +++ new/numexpr-2.6.0/numexpr.egg-info/PKG-INFO 2016-06-01 13:58:34.000000000 +0200 @@ -1,6 +1,6 @@ Metadata-Version: 1.0 Name: numexpr -Version: 2.5.2 +Version: 2.6.0 Summary: Fast numerical expression evaluator for NumPy Home-page: https://github.com/pydata/numexpr Author: David M. Cooke, Francesc Alted and others diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.5.2/setup.py new/numexpr-2.6.0/setup.py --- old/numexpr-2.5.2/setup.py 2016-03-22 22:17:01.000000000 +0100 +++ new/numexpr-2.6.0/setup.py 2016-05-30 11:33:24.000000000 +0200 @@ -42,8 +42,10 @@ install_requires=requirements, setup_requires=requirements ) - if (len(sys.argv) >= 2 and ('--help' in sys.argv[1:] or sys.argv[1] - in ('--help-commands', 'egg_info', '--version', 'clean'))): + if (len(sys.argv) >= 2 and + ('--help' in sys.argv[1:] or + (sys.argv[1] in ( + '--help-commands', 'egg_info', '--version', 'clean', '--name')))): # For these actions, NumPy is not required. # @@ -77,7 +79,8 @@ old_build_py.run(self) def find_package_modules(self, package, package_dir): - modules = old_build_py.find_package_modules(self, package, package_dir) + modules = old_build_py.find_package_modules( + self, package, package_dir) # Find build_src generated *.py files. build_src = self.get_finalized_command('build_src') @@ -116,8 +119,8 @@ #try to find configuration for MKL, either from environment or site.cfg if op.exists('site.cfg'): mkl_config_data = config.get_info('mkl') - # some version of MKL need to be linked with libgfortran, for this, use - # entries of DEFAULT section in site.cfg + # Some version of MKL needs to be linked with libgfortran. + # For this, use entries of DEFAULT section in site.cfg. default_config = system_info() dict_append(mkl_config_data, libraries=default_config.get_libraries(), @@ -125,7 +128,7 @@ else: mkl_config_data = {} - #setup information for C extension + # setup information for C extension if os.name == 'nt': pthread_win = ['numexpr/win32/pthread.c'] else:
