Script 'mail_helper' called by obssrc Hello community, here is the log from the commit of package python-numexpr for openSUSE:Factory checked in at 2023-09-06 18:56:58 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/python-numexpr (Old) and /work/SRC/openSUSE:Factory/.python-numexpr.new.1766 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "python-numexpr" Wed Sep 6 18:56:58 2023 rev:20 rq:1109001 version:2.8.4 Changes: -------- --- /work/SRC/openSUSE:Factory/python-numexpr/python-numexpr.changes 2023-08-28 17:17:31.385623302 +0200 +++ /work/SRC/openSUSE:Factory/.python-numexpr.new.1766/python-numexpr.changes 2023-09-06 18:58:46.488802539 +0200 @@ -1,0 +2,7 @@ +Mon Sep 4 19:07:09 UTC 2023 - Ben Greiner <c...@bnavigator.de> + +- Revert to version 2.8.4: Patch release breaks its API and thus + Pandas -- gh#pydata/numexpr#442, gh#pydata/numexpr#444 +- Move to PEP517 build + +------------------------------------------------------------------- Old: ---- numexpr-2.8.5.tar.gz New: ---- numexpr-2.8.4.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ python-numexpr.spec ++++++ --- /var/tmp/diff_new_pack.L5OPgr/_old 2023-09-06 18:58:47.576841324 +0200 +++ /var/tmp/diff_new_pack.L5OPgr/_new 2023-09-06 18:58:47.576841324 +0200 @@ -17,7 +17,7 @@ Name: python-numexpr -Version: 2.8.5 +Version: 2.8.4 Release: 0 Summary: Numerical expression evaluator for NumPy License: MIT @@ -26,7 +26,9 @@ Source: https://files.pythonhosted.org/packages/source/n/numexpr/numexpr-%{version}.tar.gz BuildRequires: %{python_module devel >= 3.7} BuildRequires: %{python_module numpy-devel >= 1.13.3} +BuildRequires: %{python_module pip} BuildRequires: %{python_module setuptools} +BuildRequires: %{python_module wheel} BuildRequires: fdupes BuildRequires: gcc-c++ BuildRequires: python-rpm-macros @@ -48,10 +50,10 @@ %build export CFLAGS="%{optflags} -fno-strict-aliasing" -%python_build +%pyproject_wheel %install -%python_install +%pyproject_install %python_expand %fdupes %{buildroot}%{$python_sitearch} %check @@ -65,7 +67,7 @@ %files %{python_files} %doc ANNOUNCE.rst AUTHORS.txt README.rst RELEASE_NOTES.rst site.cfg.example %license LICENSE.txt -%{python_sitearch}/numexpr/ -%{python_sitearch}/numexpr-%{version}-py*.egg-info +%{python_sitearch}/numexpr +%{python_sitearch}/numexpr-%{version}.dist-info %changelog ++++++ numexpr-2.8.5.tar.gz -> numexpr-2.8.4.tar.gz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.8.5/ANNOUNCE.rst new/numexpr-2.8.4/ANNOUNCE.rst --- old/numexpr-2.8.5/ANNOUNCE.rst 2023-08-06 22:16:44.000000000 +0200 +++ new/numexpr-2.8.4/ANNOUNCE.rst 2022-10-26 04:43:06.000000000 +0200 @@ -1,38 +1,28 @@ ======================== -Announcing NumExpr 2.8.5 +Announcing NumExpr 2.8.4 ======================== Hi everyone, -In 2.8.5 we have added a new function, `validate` which checks an expression `ex` -for validity, for usage where the program is parsing a user input. There are also -consequences for this sort of usage, since `eval(ex)` is called, and as such we -do some string sanitization as described below. +This is a maintenance and bug-fix release for NumExpr. In particular, now we have +added Python 3.11 support. Project documentation is available at: http://numexpr.readthedocs.io/ -Changes from 2.8.4 to 2.8.5 + +Changes from 2.8.3 to 2.8.4 --------------------------- -* A `validate` function has been added. This function checks the inputs, returning - `None` on success or raising an exception on invalid inputs. This function was - added as numerous projects seem to be using NumExpr for parsing user inputs. - `re_evaluate` may be called directly following `validate`. -* As an addendum to the use of NumExpr for parsing user inputs, is that NumExpr - calls `eval` on the inputs. A regular expression is now applied to help sanitize - the input expression string, forbidding '__', ':', and ';'. Attribute access - is also banned except for '.r' for real and '.i' for imag. -* Thanks to timbrist for a fix to behavior of NumExpr with integers to negative - powers. NumExpr was pre-checking integer powers for negative values, which - was both inefficient and causing parsing errors in some situations. Now NumExpr - will simply return 0 as a result for such cases. While NumExpr generally tries - to follow NumPy behavior, performance is also critical. -* Thanks to peadar for some fixes to how NumExpr launches threads for embedded - applications. -* Thanks to de11n for making parsing of the `site.cfg` for MKL consistent among - all shared platforms. +* Support for Python 3.11 has been added. +* Thanks to Tobias Hangleiter for an improved accuracy complex `expm1` function. + While it is 25 % slower, it is significantly more accurate for the real component + over a range of values and matches NumPy outputs much more closely. +* Thanks to Kirill Kouzoubov for a range of fixes to constants parsing that was + resulting in duplicated constants of the same value. +* Thanks to Mark Harfouche for noticing that we no longer need `numpy` version + checks. `packaging` is no longer a requirement as a result. What's Numexpr? diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.8.5/PKG-INFO new/numexpr-2.8.4/PKG-INFO --- old/numexpr-2.8.5/PKG-INFO 2023-08-06 22:30:12.274681800 +0200 +++ new/numexpr-2.8.4/PKG-INFO 2022-10-26 04:59:27.071168200 +0200 @@ -1,6 +1,6 @@ Metadata-Version: 2.1 Name: numexpr -Version: 2.8.5 +Version: 2.8.4 Summary: Fast numerical expression evaluator for NumPy Home-page: https://github.com/pydata/numexpr Author: David M. Cooke, Francesc Alted, and others diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.8.5/RELEASE_NOTES.rst new/numexpr-2.8.4/RELEASE_NOTES.rst --- old/numexpr-2.8.5/RELEASE_NOTES.rst 2023-08-06 22:16:44.000000000 +0200 +++ new/numexpr-2.8.4/RELEASE_NOTES.rst 2022-10-26 04:43:06.000000000 +0200 @@ -2,28 +2,6 @@ Release notes for NumExpr 2.8 series ==================================== -Changes from 2.8.4 to 2.8.5 ---------------------------- - -* A `validate` function has been added. This function checks the inputs, returning - `None` on success or raising an exception on invalid inputs. This function was - added as numerous projects seem to be using NumExpr for parsing user inputs. - `re_evaluate` may be called directly following `validate`. -* As an addendum to the use of NumExpr for parsing user inputs, is that NumExpr - calls `eval` on the inputs. A regular expression is now applied to help sanitize - the input expression string, forbidding '__', ':', and ';'. Attribute access - is also banned except for '.r' for real and '.i' for imag. -* Thanks to timbrist for a fix to behavior of NumExpr with integers to negative - powers. NumExpr was pre-checking integer powers for negative values, which - was both inefficient and causing parsing errors in some situations. Now NumExpr - will simply return 0 as a result for such cases. While NumExpr generally tries - to follow NumPy behavior, performance is also critical. -* Thanks to peadar for some fixes to how NumExpr launches threads for embedded - applications. -* Thanks to de11n for making parsing of the `site.cfg` for MKL consistent among - all shared platforms. - - Changes from 2.8.3 to 2.8.4 --------------------------- @@ -47,8 +25,8 @@ variable: ``` -x = np.empty(5, dtype=np.uint8)[1:].view(np.int32) -ne.evaluate('3', out=x) + x = np.empty(5, dtype=np.uint8)[1:].view(np.int32) + ne.evaluate('3', out=x) ``` We think the risk of issues is very low, but if you are using NumExpr as a @@ -66,7 +44,7 @@ * Fixed dependency list. * Added ``pyproject.toml`` and modernize the ``setup.py`` script. Thanks to - Antonio Valentino for the PR. +Antonio Valentino for the PR. Changes from 2.7.3 to 2.8.0 --------------------------- diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.8.5/numexpr/__init__.py new/numexpr-2.8.4/numexpr/__init__.py --- old/numexpr-2.8.5/numexpr/__init__.py 2023-08-06 22:16:44.000000000 +0200 +++ new/numexpr-2.8.4/numexpr/__init__.py 2022-10-26 04:43:06.000000000 +0200 @@ -31,8 +31,7 @@ import os, os.path import platform from numexpr.expressions import E -from numexpr.necompiler import (NumExpr, disassemble, evaluate, re_evaluate, - validate) +from numexpr.necompiler import NumExpr, disassemble, evaluate, re_evaluate from numexpr.utils import (_init_num_threads, get_vml_version, set_vml_accuracy_mode, set_vml_num_threads, diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.8.5/numexpr/expressions.py new/numexpr-2.8.4/numexpr/expressions.py --- old/numexpr-2.8.5/numexpr/expressions.py 2023-08-06 22:16:44.000000000 +0200 +++ new/numexpr-2.8.4/numexpr/expressions.py 2022-10-26 04:43:06.000000000 +0200 @@ -35,7 +35,9 @@ from numexpr import interpreter -class Expression(): +class Expression(object): + def __init__(self): + object.__init__(self) def __getattr__(self, name): if name.startswith('_'): @@ -269,13 +271,17 @@ @ophelper def pow_op(a, b): - + if (b.astKind in ('int', 'long') and + a.astKind in ('int', 'long') and + numpy.any(b.value < 0)): + + raise ValueError( + 'Integers to negative integer powers are not allowed.') + + if allConstantNodes([a, b]): + return ConstantNode(a.value ** b.value) if isinstance(b, ConstantNode): x = b.value - if ( a.astKind in ('int', 'long') and - b.astKind in ('int', 'long') and x < 0) : - raise ValueError( - 'Integers to negative integer powers are not allowed.') if get_optimization() == 'aggressive': RANGE = 50 # Approximate break even point with pow(x,y) # Optimize all integral and half integral powers in [-RANGE, RANGE] @@ -306,8 +312,7 @@ if r is None: r = OpNode('ones_like', [a]) if x < 0: - # Issue #428 - r = truediv_op(ConstantNode(1), r) + r = OpNode('div', [ConstantNode(1), r]) return r if get_optimization() in ('moderate', 'aggressive'): if x == -1: @@ -373,7 +378,7 @@ } -class ExpressionNode(): +class ExpressionNode(object): """ An object that represents a generic number object. @@ -383,6 +388,7 @@ astType = 'generic' def __init__(self, value=None, kind=None, children=None): + object.__init__(self) self.value = value if kind is None: kind = 'none' @@ -470,7 +476,7 @@ LeafNode.__init__(self, value=value, kind=kind) -class RawNode(): +class RawNode(object): """ Used to pass raw integers to interpreter. For instance, for selecting what function to use in func1. diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.8.5/numexpr/interp_body.cpp new/numexpr-2.8.4/numexpr/interp_body.cpp --- old/numexpr-2.8.5/numexpr/interp_body.cpp 2023-08-06 22:16:44.000000000 +0200 +++ new/numexpr-2.8.4/numexpr/interp_body.cpp 2022-10-26 04:43:06.000000000 +0200 @@ -7,14 +7,7 @@ See LICENSE.txt for details about copyright and rights to use. **********************************************************************/ -// WARNING: This file is included multiple times in `interpreter.cpp`. It is -// essentially a very macro-heavy jump table. Interpretation is best done by -// the developer by expanding all macros (e.g. adding `'-E'` to the `extra_cflags` -// argument in `setup.py` and looking at the resulting `interpreter.cpp`. -// -// Changes made to this file will not be recognized by the compile, so the developer -// must make a trivial change is made to `interpreter.cpp` or delete the `build/` -// directory in-between each build. + { #define VEC_LOOP(expr) for(j = 0; j < BLOCK_SIZE; j++) { \ expr; \ @@ -263,7 +256,7 @@ case OP_MUL_III: VEC_ARG2(i_dest = i1 * i2); case OP_DIV_III: VEC_ARG2(i_dest = i2 ? (i1 / i2) : 0); case OP_POW_III: VEC_ARG2(i_dest = (i2 < 0) ? (1 / i1) : (int)pow((double)i1, i2)); - case OP_MOD_III: VEC_ARG2(i_dest = i2 == 0 ? 0 :((i1 % i2) + i2) % i2); + case OP_MOD_III: VEC_ARG2(i_dest = i2 ? (i1 % i2) : 0); case OP_LSHIFT_III: VEC_ARG2(i_dest = i1 << i2); case OP_RSHIFT_III: VEC_ARG2(i_dest = i1 >> i2); @@ -283,7 +276,7 @@ #else case OP_POW_LLL: VEC_ARG2(l_dest = (l2 < 0) ? (1 / l1) : (long long)llround(pow((long double)l1, (long double)l2))); #endif - case OP_MOD_LLL: VEC_ARG2(l_dest = l2 == 0 ? 0 :((l1 % l2) + l2) % l2); + case OP_MOD_LLL: VEC_ARG2(l_dest = l2 ? (l1 % l2) : 0); case OP_LSHIFT_LLL: VEC_ARG2(l_dest = l1 << l2); case OP_RSHIFT_LLL: VEC_ARG2(l_dest = l1 >> l2); diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.8.5/numexpr/interpreter.cpp new/numexpr-2.8.4/numexpr/interpreter.cpp --- old/numexpr-2.8.5/numexpr/interpreter.cpp 2023-08-06 22:16:44.000000000 +0200 +++ new/numexpr-2.8.4/numexpr/interpreter.cpp 2022-10-26 04:43:06.000000000 +0200 @@ -990,10 +990,7 @@ int r, pc_error = 0; int reduction_axis = -1; npy_intp reduction_size = -1; // For #277 change this 1 -> -1 to be in-line with NumPy 1.8, -#ifdef USE_VML - int ex_uses_vml = 0 -#endif - int is_reduction = 0; + int ex_uses_vml = 0, is_reduction = 0; bool reduction_outer_loop = false, need_output_buffering = false, full_reduction = false; // To specify axes when doing a reduction @@ -1043,11 +1040,9 @@ return PyErr_Format(PyExc_ValueError, "ex_uses_vml parameter is required"); } -#ifdef USE_VML if (tmp == Py_True) { ex_uses_vml = 1; } -#endif // borrowed ref operands[0] = (PyArrayObject *)PyDict_GetItemString(kwds, "out"); if (operands[0] != NULL) { diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.8.5/numexpr/module.cpp new/numexpr-2.8.4/numexpr/module.cpp --- old/numexpr-2.8.5/numexpr/module.cpp 2023-08-06 22:16:44.000000000 +0200 +++ new/numexpr-2.8.4/numexpr/module.cpp 2022-10-26 04:43:06.000000000 +0200 @@ -13,8 +13,6 @@ #include <structmember.h> #include <vector> -#include <signal.h> - #include "interpreter.hpp" #include "numexpr_object.hpp" @@ -211,26 +209,7 @@ gs.count_threads = 0; /* Reset threads counter */ gs.barrier_passed = 0; - /* - * Our worker threads should not deal with signals from the rest of the - * application - mask everything temporarily in this thread, so our workers - * can inherit that mask - */ - sigset_t sigset_block_all, sigset_restore; - rc = sigfillset(&sigset_block_all); - if (rc != 0) { - fprintf(stderr, "ERROR; failed to block signals: sigfillset: %s", - strerror(rc)); - exit(-1); - } - rc = pthread_sigmask( SIG_BLOCK, &sigset_block_all, &sigset_restore); - if (rc != 0) { - fprintf(stderr, "ERROR; failed to block signals: pthread_sigmask: %s", - strerror(rc)); - exit(-1); - } - - /* Now create the threads */ + /* Finally, create the threads */ for (tid = 0; tid < gs.nthreads; tid++) { gs.tids[tid] = tid; rc = pthread_create(&gs.threads[tid], NULL, th_worker, @@ -243,18 +222,6 @@ } } - /* - * Restore the signal mask so the main thread can process signals as - * expected - */ - rc = pthread_sigmask( SIG_SETMASK, &sigset_restore, NULL); - if (rc != 0) { - fprintf(stderr, - "ERROR: failed to restore signal mask: pthread_sigmask: %s", - strerror(rc)); - exit(-1); - } - gs.init_threads_done = 1; /* Initialization done! */ gs.pid = (int)getpid(); /* save the PID for this process */ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.8.5/numexpr/necompiler.py new/numexpr-2.8.4/numexpr/necompiler.py --- old/numexpr-2.8.5/numexpr/necompiler.py 2023-08-06 22:16:44.000000000 +0200 +++ new/numexpr-2.8.4/numexpr/necompiler.py 2022-10-26 04:43:06.000000000 +0200 @@ -8,12 +8,10 @@ # rights to use. #################################################################### -from typing import Optional, Dict import __future__ import sys import numpy import threading -import re is_cpu_amd_intel = False # DEPRECATION WARNING: WILL BE REMOVED IN FUTURE RELEASE from numexpr import interpreter, expressions, use_vml @@ -70,7 +68,7 @@ ] -class ASTNode(): +class ASTNode(object): """Abstract Syntax Tree node. Members: @@ -85,6 +83,7 @@ cmpnames = ['astType', 'astKind', 'value', 'children'] def __init__(self, astType='generic', astKind='unknown', value=None, children=()): + object.__init__(self) self.astType = astType self.astKind = astKind self.value = value @@ -219,7 +218,7 @@ [typeCompileAst(c) for c in children]) -class Register(): +class Register(object): """Abstraction for a register in the VM. Members: @@ -261,18 +260,9 @@ return 'Immediate(%d)' % (self.node.value,) -_forbidden_re = re.compile('[\;[\:]|__|\.[abcdefghjklmnopqstuvwxyzA-Z_]') def stringToExpression(s, types, context): """Given a string, convert it to a tree of ExpressionNode's. """ - # sanitize the string for obvious attack vectors that NumExpr cannot - # parse into its homebrew AST. This is to protect the call to `eval` below. - # We forbid `;`, `:`. `[` and `__`, and attribute access via '.'. - # We cannot ban `.real` or `.imag` however... - no_whitespace = re.sub(r'\s+', '', s) - if _forbidden_re.search(no_whitespace) is not None: - raise ValueError(f'Expression {s} has forbidden control characters.') - old_ctx = expressions._context.get_current_context() try: expressions._context.set_new_context(context) @@ -295,10 +285,8 @@ t = types.get(name, default_type) names[name] = expressions.VariableNode(name, type_to_kind[t]) names.update(expressions.functions) - # now build the expression ex = eval(c, names) - if expressions.isConstant(ex): ex = expressions.ConstantNode(ex, expressions.getKind(ex)) elif not isinstance(ex, expressions.ExpressionNode): @@ -539,7 +527,7 @@ ] -def getContext(kwargs, _frame_depth=1): +def getContext(kwargs, frame_depth=1): d = kwargs.copy() context = {} for name, allowed, default in context_info: @@ -548,11 +536,11 @@ context[name] = value else: raise ValueError("'%s' must be one of %s" % (name, allowed)) - + if d: raise ValueError("Unknown keyword argument '%s'" % d.popitem()[0]) if context['truediv'] == 'auto': - caller_globals = sys._getframe(_frame_depth + 1).f_globals + caller_globals = sys._getframe(frame_depth + 1).f_globals context['truediv'] = caller_globals.get('division', None) == __future__.division return context @@ -623,12 +611,14 @@ Returns a `NumExpr` object containing the compiled function. """ - - # In that case _frame_depth is wrong (it should be 2) but it doesn't matter + # NumExpr can be called either directly by the end-user, in which case + # kwargs need to be sanitized by getContext, or by evaluate, + # in which case kwargs are in already sanitized. + # In that case frame_depth is wrong (it should be 2) but it doesn't matter # since it will not be used (because truediv='auto' has already been # translated to either True or False). - _frame_depth = 1 - context = getContext(kwargs, _frame_depth=_frame_depth) + + context = getContext(kwargs, frame_depth=1) threeAddrProgram, inputsig, tempsig, constants, input_names = precompile(ex, signature, context) program = compileThreeAddrForm(threeAddrProgram) return interpreter.NumExpr(inputsig.encode('ascii'), @@ -728,11 +718,11 @@ return [a.value for a in input_order], ex_uses_vml -def getArguments(names, local_dict=None, global_dict=None, _frame_depth: int=2): +def getArguments(names, local_dict=None, global_dict=None): """ Get the arguments based on the names. """ - call_frame = sys._getframe(_frame_depth) + call_frame = sys._getframe(2) clear_local_dict = False if local_dict is None: @@ -768,45 +758,35 @@ _names_cache = CacheDict(256) _numexpr_cache = CacheDict(256) _numexpr_last = {} + evaluate_lock = threading.Lock() -# MAYBE: decorate this function to add attributes instead of having the -# _numexpr_last dictionary? -def validate(ex: str, - local_dict: Optional[Dict] = None, - global_dict: Optional[Dict] = None, - out: numpy.ndarray = None, - order: str = 'K', - casting: str = 'safe', - _frame_depth: int = 2, - **kwargs) -> Optional[Exception]: - """ - Validate a NumExpr expression with the given `local_dict` or `locals()`. - Returns `None` on success and the Exception object if one occurs. Note that - you can proceed directly to call `re_evaluate()` if you use `validate()` - to sanitize your expressions and variables in advance. +def evaluate(ex, local_dict=None, global_dict=None, + out=None, order='K', casting='safe', **kwargs): + """ + Evaluate a simple array expression element-wise, using the new iterator. + + ex is a string forming an expression, like "2*a+3*b". The values for "a" + and "b" will by default be taken from the calling function's frame + (through use of sys._getframe()). Alternatively, they can be specifed + using the 'local_dict' or 'global_dict' arguments. Parameters ---------- - ex: str - a string forming an expression, like "2*a+3*b". The values for "a" - and "b" will by default be taken from the calling function's frame - (through use of sys._getframe()). Alternatively, they can be specified - using the 'local_dict' or 'global_dict' arguments. - local_dict: dictionary, optional + local_dict : dictionary, optional A dictionary that replaces the local operands in current frame. - global_dict: dictionary, optional + global_dict : dictionary, optional A dictionary that replaces the global operands in current frame. - out: NumPy array, optional + out : NumPy array, optional An existing array where the outcome is going to be stored. Care is required so that this array has the same shape and type than the actual outcome of the computation. Useful for avoiding unnecessary new array allocations. - order: {'C', 'F', 'A', or 'K'}, optional + order : {'C', 'F', 'A', or 'K'}, optional Controls the iteration order for operands. 'C' means C order, 'F' means Fortran order, 'A' means 'F' order if all the arrays are Fortran contiguous, 'C' order otherwise, and 'K' means as close to @@ -814,7 +794,7 @@ efficient computations, typically 'K'eep order (the default) is desired. - casting: {'no', 'equiv', 'safe', 'same_kind', 'unsafe'}, optional + casting : {'no', 'equiv', 'safe', 'same_kind', 'unsafe'}, optional Controls what kind of data casting may occur when making a copy or buffering. Setting this to 'unsafe' is not recommended, as it can adversely affect accumulations. @@ -825,125 +805,37 @@ * 'same_kind' means only safe casts or casts within a kind, like float64 to float32, are allowed. * 'unsafe' means any data conversions may be done. - - _frame_depth: int - The calling frame depth. Unless you are a NumExpr developer you should - not set this value. - - Note - ---- - Both `validate` and by extension `evaluate` call `eval(ex)`, which is - potentially dangerous on unsanitized inputs. As such, NumExpr does some - sanitization, banning the character ':;[', the dunder '__', and attribute - access to all but '.r' for real and '.i' for imag access to complex numbers. """ global _numexpr_last + if not isinstance(ex, str): + raise ValueError("must specify expression as a string") + + # Get the names for this expression + context = getContext(kwargs, frame_depth=1) + expr_key = (ex, tuple(sorted(context.items()))) + if expr_key not in _names_cache: + _names_cache[expr_key] = getExprNames(ex, context) + names, ex_uses_vml = _names_cache[expr_key] + arguments = getArguments(names, local_dict, global_dict) + + # Create a signature + signature = [(name, getType(arg)) for (name, arg) in + zip(names, arguments)] + # Look up numexpr if possible. + numexpr_key = expr_key + (tuple(signature),) try: - - if not isinstance(ex, str): - raise ValueError("must specify expression as a string") - - # Get the names for this expression - context = getContext(kwargs) - expr_key = (ex, tuple(sorted(context.items()))) - if expr_key not in _names_cache: - _names_cache[expr_key] = getExprNames(ex, context) - names, ex_uses_vml = _names_cache[expr_key] - arguments = getArguments(names, local_dict, global_dict, _frame_depth=_frame_depth) - - # Create a signature - signature = [(name, getType(arg)) for (name, arg) in - zip(names, arguments)] - - # Look up numexpr if possible. - numexpr_key = expr_key + (tuple(signature),) - try: - compiled_ex = _numexpr_cache[numexpr_key] - except KeyError: - compiled_ex = _numexpr_cache[numexpr_key] = NumExpr(ex, signature, **context) - kwargs = {'out': out, 'order': order, 'casting': casting, - 'ex_uses_vml': ex_uses_vml} - _numexpr_last = dict(ex=compiled_ex, argnames=names, kwargs=kwargs) - except Exception as e: - return e - return None - -def evaluate(ex: str, - local_dict: Optional[Dict] = None, - global_dict: Optional[Dict] = None, - out: numpy.ndarray = None, - order: str = 'K', - casting: str = 'safe', - _frame_depth: int = 3, - **kwargs) -> numpy.ndarray: - """ - Evaluate a simple array expression element-wise using the virtual machine. - - Parameters - ---------- - ex: str - a string forming an expression, like "2*a+3*b". The values for "a" - and "b" will by default be taken from the calling function's frame - (through use of sys._getframe()). Alternatively, they can be specified - using the 'local_dict' or 'global_dict' arguments. - - local_dict: dictionary, optional - A dictionary that replaces the local operands in current frame. - - global_dict: dictionary, optional - A dictionary that replaces the global operands in current frame. - - out: NumPy array, optional - An existing array where the outcome is going to be stored. Care is - required so that this array has the same shape and type than the - actual outcome of the computation. Useful for avoiding unnecessary - new array allocations. - - order: {'C', 'F', 'A', or 'K'}, optional - Controls the iteration order for operands. 'C' means C order, 'F' - means Fortran order, 'A' means 'F' order if all the arrays are - Fortran contiguous, 'C' order otherwise, and 'K' means as close to - the order the array elements appear in memory as possible. For - efficient computations, typically 'K'eep order (the default) is - desired. + compiled_ex = _numexpr_cache[numexpr_key] + except KeyError: + compiled_ex = _numexpr_cache[numexpr_key] = NumExpr(ex, signature, **context) + kwargs = {'out': out, 'order': order, 'casting': casting, + 'ex_uses_vml': ex_uses_vml} + _numexpr_last = dict(ex=compiled_ex, argnames=names, kwargs=kwargs) + with evaluate_lock: + return compiled_ex(*arguments, **kwargs) - casting: {'no', 'equiv', 'safe', 'same_kind', 'unsafe'}, optional - Controls what kind of data casting may occur when making a copy or - buffering. Setting this to 'unsafe' is not recommended, as it can - adversely affect accumulations. - * 'no' means the data types should not be cast at all. - * 'equiv' means only byte-order changes are allowed. - * 'safe' means only casts which can preserve values are allowed. - * 'same_kind' means only safe casts or casts within a kind, - like float64 to float32, are allowed. - * 'unsafe' means any data conversions may be done. - - _frame_depth: int - The calling frame depth. Unless you are a NumExpr developer you should - not set this value. - - Note - ---- - Both `validate` and by extension `evaluate` call `eval(ex)`, which is - potentially dangerous on unsanitized inputs. As such, NumExpr does some - sanitization, banning the character ':;[', the dunder '__', and attribute - access to all but '.r' for real and '.i' for imag access to complex numbers. - """ - # We could avoid code duplication if we called validate and then re_evaluate - # here, but they we have difficulties with the `sys.getframe(2)` call in - # `getArguments` - e = validate(ex, local_dict=local_dict, global_dict=global_dict, - out=out, order=order, casting=casting, - _frame_depth=_frame_depth, **kwargs) - if e is None: - return re_evaluate(local_dict=local_dict, _frame_depth=_frame_depth) - else: - raise e - -def re_evaluate(local_dict: Optional[Dict] = None, - _frame_depth: int=2) -> numpy.ndarray: +def re_evaluate(local_dict=None): """ Re-evaluate the previous executed array expression without any check. @@ -953,20 +845,17 @@ Parameters ---------- - local_dict: dictionary, optional + + local_dict : dictionary, optional A dictionary that replaces the local operands in current frame. - _frame_depth: int - The calling frame depth. Unless you are a NumExpr developer you should - not set this value. - """ - global _numexpr_last + """ try: compiled_ex = _numexpr_last['ex'] except KeyError: - raise RuntimeError("A previous evaluate() execution was not found, please call `validate` or `evaluate` once before `re_evaluate`") + raise RuntimeError("not a previous evaluate() execution found") argnames = _numexpr_last['argnames'] - args = getArguments(argnames, local_dict, _frame_depth=_frame_depth) + args = getArguments(argnames, local_dict) kwargs = _numexpr_last['kwargs'] with evaluate_lock: return compiled_ex(*args, **kwargs) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.8.5/numexpr/tests/test_numexpr.py new/numexpr-2.8.4/numexpr/tests/test_numexpr.py --- old/numexpr-2.8.5/numexpr/tests/test_numexpr.py 2023-08-06 22:16:44.000000000 +0200 +++ new/numexpr-2.8.4/numexpr/tests/test_numexpr.py 2022-10-26 04:43:06.000000000 +0200 @@ -31,7 +31,7 @@ from numpy import shape, allclose, array_equal, ravel, isnan, isinf import numexpr -from numexpr import E, NumExpr, evaluate, re_evaluate, validate, disassemble, use_vml +from numexpr import E, NumExpr, evaluate, re_evaluate, disassemble, use_vml from numexpr.expressions import ConstantNode import unittest @@ -370,43 +370,13 @@ assert_array_equal(x, array([86., 124., 168.])) def test_re_evaluate_dict(self): - a1 = array([1., 2., 3.]) - b1 = array([4., 5., 6.]) - c1 = array([7., 8., 9.]) - local_dict={'a': a1, 'b': b1, 'c': c1} - x = evaluate("2*a + 3*b*c", local_dict=local_dict) - x = re_evaluate(local_dict=local_dict) - assert_array_equal(x, array([86., 124., 168.])) - - def test_validate(self): a = array([1., 2., 3.]) b = array([4., 5., 6.]) c = array([7., 8., 9.]) - retval = validate("2*a + 3*b*c") - assert(retval is None) + x = evaluate("2*a + 3*b*c", local_dict={'a': a, 'b': b, 'c': c}) x = re_evaluate() assert_array_equal(x, array([86., 124., 168.])) - def test_validate_missing_var(self): - a = array([1., 2., 3.]) - b = array([4., 5., 6.]) - retval = validate("2*a + 3*b*c") - assert(isinstance(retval, KeyError)) - - def test_validate_syntax(self): - retval = validate("2+") - assert(isinstance(retval, SyntaxError)) - - def test_validate_dict(self): - a1 = array([1., 2., 3.]) - b1 = array([4., 5., 6.]) - c1 = array([7., 8., 9.]) - local_dict={'a': a1, 'b': b1, 'c': c1} - retval = validate("2*a + 3*b*c", local_dict=local_dict) - assert(retval is None) - x = re_evaluate(local_dict=local_dict) - assert_array_equal(x, array([86., 124., 168.])) - # Test for issue #22 def test_true_div(self): x = arange(10, dtype='i4') @@ -504,63 +474,11 @@ a = arange(3) try: evaluate("a < [0, 0, 0]") - except (ValueError, TypeError): - pass - else: - self.fail() - - def test_forbidden_tokens(self): - # Forbid dunder - try: - evaluate('__builtins__') - except ValueError: - pass - else: - self.fail() - - # Forbid colon for lambda funcs - try: - evaluate('lambda x: x') - except ValueError: - pass - else: - self.fail() - - # Forbid indexing - try: - evaluate('locals()[]') - except ValueError: - pass - else: - self.fail() - - # Forbid semicolon - try: - evaluate('import os;') - except ValueError: - pass - else: - self.fail() - - # Attribute access - try: - evaluate('os.cpucount()') - except ValueError: + except TypeError: pass else: self.fail() - # But decimal point must pass - a = 3.0 - evaluate('a*2.') - evaluate('2.+a') - - - - - - - def test_disassemble(self): assert_equal(disassemble(NumExpr( "where(m, a, -1)", [('m', bool), ('a', float)])), @@ -593,30 +511,6 @@ evaluate('3', out=a) assert_equal(a, 3) - def test_negative_mod(self): - # Test for issue #413, modulus of negative integers. C modulus is - # actually remainder op, and hence different from Python modulus. - a = np.array([-500, -135, 0, 0, 135, 500], dtype=np.int32) - n = np.array([-360, -360, -360, 360, 360, 360], dtype=np.int32) - out_i = evaluate('a % n') - assert_equal(out_i, np.mod(a, n)) - - b = a.astype(np.int64) - m = n.astype(np.int64) - out_l = evaluate('b % m') - assert_equal(out_l, np.mod(b, m)) - - def test_negative_power_scalar(self): - # Test for issue #428, where the power is negative and the base is an - # integer. This was running afoul in the precomputation in `expressions.py:pow_op()` - base = np.array([-2, -1, 0, 1, 2, 3], dtype=np.int32) - out_i = evaluate('base ** -1.0') - assert_equal(out_i, np.power(base, -1.0)) - - base = np.array([-2, -1, 0, 1, 2, 3], dtype=np.int64) - out_l = evaluate('base ** -1.0') - assert_equal(out_l, np.power(base, -1.0)) - def test_ex_uses_vml(self): vml_funcs = [ "sin", "cos", "tan", "arcsin", "arccos", "arctan", @@ -730,7 +624,7 @@ return (shape(a) == shape(b)) and alltrue(ravel(a) == ravel(b), axis=0) else: if hasattr(a, 'dtype') and a.dtype == 'f4': - atol = 1e-5 # Relax precision for special opcodes, like fmod + atol = 1e-5 # Relax precission for special opcodes, like fmod else: atol = 1e-8 return (shape(a) == shape(b) and diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.8.5/numexpr/version.py new/numexpr-2.8.4/numexpr/version.py --- old/numexpr-2.8.5/numexpr/version.py 2023-08-06 22:30:12.000000000 +0200 +++ new/numexpr-2.8.4/numexpr/version.py 2022-10-26 04:59:26.000000000 +0200 @@ -1,4 +1,4 @@ # THIS FILE IS GENERATED BY `SETUP.PY` -version = '2.8.5' -numpy_build_version = '1.23.2' +version = '2.8.4' +numpy_build_version = '1.21.6' platform_machine = 'AMD64' diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.8.5/numexpr/win32/pthread.h new/numexpr-2.8.4/numexpr/win32/pthread.h --- old/numexpr-2.8.5/numexpr/win32/pthread.h 2023-08-06 22:16:44.000000000 +0200 +++ new/numexpr-2.8.4/numexpr/win32/pthread.h 2022-10-26 04:43:06.000000000 +0200 @@ -93,25 +93,6 @@ extern int win32_pthread_join(pthread_t *thread, void **value_ptr); -/* - * The POSIX signal system has a more developed interface than what's in - * Windows. We create a no-op shim layer to proivde enough of the API to - * pretend to support what's used when creating threads on POSIX systems. - */ -typedef int sigset_t; -enum sigop { - SIG_BLOCK, - SIG_UNBLOCK, - SIG_SETMASK -}; - -static inline int sigemptyset(sigset_t *sigs) { return 0; } -static inline int sigfillset(sigset_t *sigs) { return 0; } -static inline int sigaddset(sigset_t *sigs, int sig) { return 0; } -static inline int sigdelset(sigset_t *sigs, int sig) { return 0; } -static inline int pthread_sigmask(int how, sigset_t *newmask, - sigset_t *oldmask) { return 0; } - #ifdef __cplusplus } // extern "C" #endif diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.8.5/numexpr.egg-info/PKG-INFO new/numexpr-2.8.4/numexpr.egg-info/PKG-INFO --- old/numexpr-2.8.5/numexpr.egg-info/PKG-INFO 2023-08-06 22:30:12.000000000 +0200 +++ new/numexpr-2.8.4/numexpr.egg-info/PKG-INFO 2022-10-26 04:59:26.000000000 +0200 @@ -1,6 +1,6 @@ Metadata-Version: 2.1 Name: numexpr -Version: 2.8.5 +Version: 2.8.4 Summary: Fast numerical expression evaluator for NumPy Home-page: https://github.com/pydata/numexpr Author: David M. Cooke, Francesc Alted, and others diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.8.5/setup.cfg new/numexpr-2.8.4/setup.cfg --- old/numexpr-2.8.5/setup.cfg 2023-08-06 22:30:12.274681800 +0200 +++ new/numexpr-2.8.4/setup.cfg 2022-10-26 04:59:27.071168200 +0200 @@ -1,6 +1,6 @@ [metadata] name = numexpr -version = 2.8.5 +version = 2.8.4 description = Fast numerical expression evaluator for NumPy author = David M. Cooke, Francesc Alted, and others maintainer = Robert A. McLeod diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/numexpr-2.8.5/setup.py new/numexpr-2.8.4/setup.py --- old/numexpr-2.8.5/setup.py 2023-08-06 22:16:44.000000000 +0200 +++ new/numexpr-2.8.4/setup.py 2022-10-26 04:43:06.000000000 +0200 @@ -29,7 +29,7 @@ fh.write("platform_machine = '%s'\n" % platform.machine()) lib_dirs = [] -inc_dirs = [np.get_include()] +inc_dirs = [np.get_include(), op.join('framestream')] libs = [] # Pre-built libraries ONLY, like python36.so clibs = [] def_macros = [] @@ -71,10 +71,8 @@ site['mkl']['include_dirs'].split(os.pathsep)) lib_dirs.extend( site['mkl']['library_dirs'].split(os.pathsep)) - # numpy's site.cfg splits libraries by comma, but numexpr historically split by os.pathsep. - # For compatibility, we split by both. libs.extend( - site['mkl']['libraries'].replace(os.pathsep, ',').split(',')) + site['mkl']['libraries'].split(os.pathsep)) def_macros.append(('USE_VML', None)) print(f'FOUND MKL IMPORT')
