Re: [Numpy-discussion] Add a function to broadcast arrays to a given shape to numpy's stride_tricks?
On Sun, Dec 7, 2014 at 11:31 PM, Pierre Haessig pierre.haes...@crans.org wrote: Instead of putting this function in stride_tricks (which is quite hidden), could it be added instead as a boolean flag to the existing `reshape` method ? Something like: x.reshape(y.shape, broadcast=True) What other people think ? I agree that it would be nice to expose this more directly, but I see two (small) downsides to putting this in reshape: 1. This would one of those flags that changes a method to an entirely different mode -- there's not much the way of shared logic with reshape. 2. reshape is written in C (like all ndarray methods, I believe), so implementing this there will be a little trickier than adding a new function. Cheers, Stephan ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Should ndarray be a context manager?
On Mi, 2014-12-10 at 07:25 +, Sturla Molden wrote: Nathaniel Smith n...@pobox.com wrote: This should be pretty trivial to implement. AFAICT you don't need any complicated cython I have a bad habit of thinking in terms of too complicated C instead of just using NumPy. @contextmanager def tmp_zeros(*args, **kwargs): arr = np.zeros(*args, **kwargs) try: yield arr finally: arr.resize((0,), check_refs=False) Given how intrinsically dangerous this is, and how easily it can be implemented using numpy's existing public API, I think maybe we should leave this for third-party daredevils instead of implementing it in numpy proper. It seems so :-) Completly agree, we may tell the user where the gun is, but we shouldn't put it in their hand and then point it at their feet as well ;). - Sebastian Sturla ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion signature.asc Description: This is a digitally signed message part ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Fwd: [Python-Dev] Python 2.x and 3.x use survey, 2014 edition
-- Forwarded message -- From: Bruno Cauet brunoca...@gmail.com Date: 10 Dec 2014 17:07 Subject: [Python-Dev] Python 2.x and 3.x use survey, 2014 edition To: python-...@python.org, python-l...@python.org Cc: Dan Stromberg strom...@gmail.com Hi all, Last year a survey was conducted on python 2 and 3 usage. Here is the 2014 edition, slightly updated (from 9 to 11 questions). It should not take you more than 1 minute to fill. I would be pleased if you took that time. Here's the url: http://goo.gl/forms/tDTcm8UzB3 I'll publish the results around the end of the year. Last year results: https://wiki.python.org/moin/2.x-vs-3.x-survey Thank you Bruno ___ Python-Dev mailing list python-...@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/njs%40pobox.com ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Should ndarray be a context manager?
On Tue, Dec 9, 2014 at 11:03 PM, Sturla Molden sturla.mol...@gmail.com wrote: Nathaniel Smith n...@pobox.com wrote: @contextmanager def tmp_zeros(*args, **kwargs): arr = np.zeros(*args, **kwargs) try: yield arr finally: arr.resize((0,), check_refs=False) That one is interesting. I have actually never used ndarray.resize(). It did not even occur to me that such an abomination existed :-) and I thought that it would only work if there were no other references to the array, in which case it gets garbage collected anyway, but I see the nifty check_refs keyword. However: In [32]: arr = np.ones((100,100)) In [33]: arr.resize((0,), check_refs=False) --- TypeError Traceback (most recent call last) ipython-input-33-f0e634534904 in module() 1 arr.resize((0,), check_refs=False) TypeError: 'check_refs' is an invalid keyword argument for this function In [34]: np.__version__ Out[34]: '1.9.1' Was that just added (or removed?) -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Should ndarray be a context manager?
The second argument is named `refcheck` rather than check_refs. Eric On Wed, Dec 10, 2014 at 2:36 PM, Chris Barker chris.bar...@noaa.gov wrote: On Tue, Dec 9, 2014 at 11:03 PM, Sturla Molden sturla.mol...@gmail.com wrote: Nathaniel Smith n...@pobox.com wrote: @contextmanager def tmp_zeros(*args, **kwargs): arr = np.zeros(*args, **kwargs) try: yield arr finally: arr.resize((0,), check_refs=False) That one is interesting. I have actually never used ndarray.resize(). It did not even occur to me that such an abomination existed :-) and I thought that it would only work if there were no other references to the array, in which case it gets garbage collected anyway, but I see the nifty check_refs keyword. However: In [32]: arr = np.ones((100,100)) In [33]: arr.resize((0,), check_refs=False) --- TypeError Traceback (most recent call last) ipython-input-33-f0e634534904 in module() 1 arr.resize((0,), check_refs=False) TypeError: 'check_refs' is an invalid keyword argument for this function In [34]: np.__version__ Out[34]: '1.9.1' Was that just added (or removed?) -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Should ndarray be a context manager?
On 10 December 2014 at 20:36, Chris Barker chris.bar...@noaa.gov wrote: On Tue, Dec 9, 2014 at 11:03 PM, Sturla Molden sturla.mol...@gmail.com wrote: Nathaniel Smith n...@pobox.com wrote: @contextmanager def tmp_zeros(*args, **kwargs): arr = np.zeros(*args, **kwargs) try: yield arr finally: arr.resize((0,), check_refs=False) That one is interesting. I have actually never used ndarray.resize(). It did not even occur to me that such an abomination existed :-) and I thought that it would only work if there were no other references to the array, in which case it gets garbage collected anyway, but I see the nifty check_refs keyword. However: In [32]: arr = np.ones((100,100)) In [33]: arr.resize((0,), check_refs=False) --- TypeError Traceback (most recent call last) ipython-input-33-f0e634534904 in module() 1 arr.resize((0,), check_refs=False) TypeError: 'check_refs' is an invalid keyword argument for this function In [34]: np.__version__ Out[34]: '1.9.1' Was that just added (or removed?) http://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.resize.html The argument is not check_refs, but refcheck. Andrea. Imagination Is The Only Weapon In The War Against Reality. http://www.infinity77.net # - # def ask_mailing_list_support(email): if mention_platform_and_version() and include_sample_app(): send_message(email) else: install_malware() erase_hard_drives() # - # ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Question about dtype
Hi,
I am using numpy version 1.9.0 and Python 2.7.9 and have a question
about the dtype:
In [14]: np.dtype(f8)
Out[14]: dtype('float64')
In [15]: np.dtype(uf8)
Out[15]: dtype('float64')
In [16]: np.dtype([(f8, f8)])
Out[16]: dtype([('f8', 'f8')])
So far so good. Now what happens if I use unicode?
In [17]: np.dtype([(uf8, f8)])
---
TypeError Traceback (most recent call
last)
ipython-input-17-ce004acab7f5 in module()
1 np.dtype([(uf8, f8)])
TypeError: data type not understood
Also, it really does need to be a tuple?
In [18]: np.dtype([[f8, f8]])
---
TypeError Traceback (most recent call
last)
ipython-input-18-c82761d7306d in module()
1 np.dtype([[f8, f8]])
TypeError: data type not understood
Is this expected behaviour or should I file a bug-report/submit a
pull-request?
best,
V-
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] help using np.correlate to produce correlograms.
Dear Pierre, thank you very much for your time to correct my notebook and to point me in the direction of my wrong lag estimation. It has been very useful! Best Jose On 09/12/14 17:23, Pierre Haessig wrote: Hi, Le 08/12/2014 22:02, Jose Guzman a écrit : I'm trying to compute the cross correlation and cross correlograms from some signals. For that, I'm testing first np.correlate with some idealized traces (sine waves) that are exactly 1 ms separated from each other. You can have a look here: http://nbviewer.ipython.org/github/JoseGuzman/myIPythonNotebooks/blob/master/Signal_Processing/Cross%20correlation.ipynb Unfortunately I am not able to retrieve the correct lag of 1 ms for the option 'full'. Strange enough, if I perform an autocorrelation of any of the signals,I obtain the correct value for a lags =0 ms. I' think I'm doing something wrong to obtain the lags. I looked at your Notebook and I believe that you had an error in the definition of the delay. In you first cell, you were creating of delay of 20ms instead of 1ms (and because the sine is periodic, this was not obvious). In addition, to get a good estimation of the delay with cross correlation, you need many perdiods. Here is a modification of your notebook : http://nbviewer.ipython.org/gist/pierre-haessig/e2dda384ae0e08943f9a I've updated the delay definition and the number of periods. Finally, you may be able to automate a bit your plot by using matplotlib's xcorr (which uses np.correlate) http://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.xcorr best, Pierre ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion -- Jose Guzman http://www.ist.ac.at/~jguzman/ ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Should ndarray be a context manager?
On Wed, Dec 10, 2014 at 11:44 AM, Andrea Gavana andrea.gav...@gmail.com wrote: The argument is not check_refs, but refcheck. thanks -- yup, that works. Useful -- but dangerous! I haven't managed to trigger a segfault yet but it sure looks like I could... -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Should ndarray be a context manager?
On Wed, Dec 10, 2014 at 9:03 PM, Chris Barker chris.bar...@noaa.gov wrote: On Wed, Dec 10, 2014 at 11:44 AM, Andrea Gavana andrea.gav...@gmail.com wrote: The argument is not check_refs, but refcheck. thanks -- yup, that works. Useful -- but dangerous! I haven't managed to trigger a segfault yet but it sure looks like I could... On Linux at least this should work reliably: In [1]: a = np.zeros(2 ** 20) In [2]: b = a[...] In [3]: a.resize((0,), refcheck=False) In [4]: b[1000] = 1 zsh: segmentation fault ipython -n -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Question about dtype
On Wed, Dec 10, 2014 at 8:26 PM, Valentin Haenel valen...@haenel.co wrote:
Hi,
I am using numpy version 1.9.0 and Python 2.7.9 and have a question
about the dtype:
In [14]: np.dtype(f8)
Out[14]: dtype('float64')
In [15]: np.dtype(uf8)
Out[15]: dtype('float64')
In [16]: np.dtype([(f8, f8)])
Out[16]: dtype([('f8', 'f8')])
So far so good. Now what happens if I use unicode?
In [17]: np.dtype([(uf8, f8)])
---
TypeError Traceback (most recent call
last)
ipython-input-17-ce004acab7f5 in module()
1 np.dtype([(uf8, f8)])
TypeError: data type not understood
Yep, looks like a bug to me. (I guess this is particularly relevant
when __future__.unicode_literals is in effect.)
Also, it really does need to be a tuple?
In [18]: np.dtype([[f8, f8]])
---
TypeError Traceback (most recent call
last)
ipython-input-18-c82761d7306d in module()
1 np.dtype([[f8, f8]])
TypeError: data type not understood
Lists and tuples are both valid inputs to np.dtype, but they're
interpreted differently -- the problem here isn't that you used a
list, it's that if you use a list then numpy expects different
contents. See:
http://docs.scipy.org/doc/numpy/user/basics.rec.html
-n
--
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Add a function to broadcast arrays to a given shape to numpy's stride_tricks?
On Sun, Dec 7, 2014 at 7:10 AM, Stephan Hoyer sho...@gmail.com wrote: I recently wrote function to manually broadcast an ndarray to a given shape according to numpy's broadcasting rules (using strides): https://github.com/xray/xray/commit/7aee4a3ed2dfd3b9aff7f3c5c6c68d51df2e3ff3 The same functionality can be done pretty straightforwardly with np.broadcast_arrays, but that function does both too much (I don't actually have a second array that needs to be broadcast) and not enough (I need to create a dummy array to broadcast against it). This approach is simpler, and also, according to my benchmarks, about 3x faster than np.broadcast_arrays: In [1]: import xray In [2]: import numpy as np In [3]: x = np.random.randn(4) In [4]: y = np.empty((2, 3, 4)) In [5]: %timeit xray.core.utils.as_shape(x, y.shape) 10 loops, best of 3: 17 µs per loop In [6]: %timeit np.broadcast_arrays(x, y)[0] 1 loops, best of 3: 47.4 µs per loop Would this be a welcome addition to numpy's lib.stride_tricks? If so, I will put together a PR. In my search, I turned up a Stack Overflow post looking for similar functionality: https://stackoverflow.com/questions/11622692/is-there-a-better-way-to-broadcast-arrays Seems like a useful addition to me -- I've definitely wanted this in the past. I agree with Stephan that reshape() might not be the best place, though; I wouldn't think to look for it there. Two API ideas, which are not mutually exclusive: 1) Give broadcast_arrays an extra_shapes=[shape1, shape2, ...] argument. Each entry in that list is a tuple of integers; broadcast_arrays chooses the final output shape as if additional arrays with the given shapes had been passed in, in addition to the ones in *args. 2) Add a broadcast_to(arr, shape) function, which broadcasts the array to exactly the shape given, or else errors out if this is not possible. Given (1), (2) could just be: def broadcast_to(arr, shape): output = broadcast_arrays(arr, extra_shapes=[shape]) if output.shape != shape: raise ... return output -n -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Should ndarray be a context manager?
Chris Barker chris.bar...@noaa.gov wrote: I haven't managed to trigger a segfault yet but it sure looks like I could... You can also trigger random errors. If the array is small, Python's memory mamager might keep the memory in the heap for reuse by PyMem_Malloc. And then you can actually modify some random Python object with bogus bits. It can screw things up even if you do not see a segfault. Sturla ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Add a function to broadcast arrays to a given shape to numpy's stride_tricks?
On Wed, Dec 10, 2014 at 4:00 PM, Nathaniel Smith n...@pobox.com wrote: 2) Add a broadcast_to(arr, shape) function, which broadcasts the array to exactly the shape given, or else errors out if this is not possible. I like np.broadcast_to as a new function. We can document it alongside broadcast and broadcast_arrays under array manipulation routines, which would make it at least as discoverable as the standard broadcasting functions. I'm not opposed to adding extra_shapes as a keyword argument to broadcast_arrays, but it seems unnecessarily complex, Implementation wise, I think it would actual make more sense to make broadcast_arrays depend on broadcast_to (e.g., by composing a function to calculate the broadcast shape with broadcast_to). Stephan ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Symbol table not found compiling numpy from git repository on Windows
Hey Guys, I'm having the same problem with building a python wrapper for a C library using windows. Tried applying the patch mentioned above, but still receiving the following error message. Any thoughts? Thanks Mel Looking for python34.dll Building import library (arch=AMD64): c:\Python34\libs\libpython34.a (from C:\ Windows\system32\python34.dll) objdump.exe: 'C:\Windows\system32\python34.dll': No such file Traceback (most recent call last): File setup.py, line 24, in module ext_modules = [module1]) File c:\Python34\lib\distutils\core.py, line 148, in setup dist.run_commands() File c:\Python34\lib\distutils\dist.py, line 955, in run_commands self.run_command(cmd) File c:\Python34\lib\distutils\dist.py, line 974, in run_command cmd_obj.run() File c:\Python34\lib\distutils\command\build.py, line 126, in run self.run_command(cmd_name) File c:\Python34\lib\distutils\cmd.py, line 313, in run_command self.distribution.run_command(command) File c:\Python34\lib\distutils\dist.py, line 974, in run_command cmd_obj.run() File c:\Python34\lib\distutils\command\build_ext.py, line 317, in run force=self.force) File c:\Python34\lib\site-packages\numpy\distutils\ccompiler.py, line 562, i n new_compiler compiler = klass(None, dry_run, force) File c:\Python34\lib\site-packages\numpy\distutils\mingw32ccompiler.py, line 91, in __init__ build_import_library() File c:\Python34\lib\site-packages\numpy\distutils\mingw32ccompiler.py, line 379, in build_import_library return _build_import_library_amd64() File c:\Python34\lib\site-packages\numpy\distutils\mingw32ccompiler.py, line 400, in _build_import_library_amd64 generate_def(dll_file, def_file) File c:\Python34\lib\site-packages\numpy\distutils\mingw32ccompiler.py, line 279, in generate_def raise ValueError(Symbol table not found) ValueError: Symbol table not found make: *** [all] Error 1 -- View this message in context: http://numpy-discussion.10968.n7.nabble.com/Symbol-table-not-found-compiling-numpy-from-git-repository-on-Windows-tp31481p39284.html Sent from the Numpy-discussion mailing list archive at Nabble.com. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
