Re: [Numpy-discussion] Should ndarray be a context manager?

2014-12-11 Thread Chris Barker
On Wed, Dec 10, 2014 at 8:40 PM, Sturla Molden sturla.mol...@gmail.com wrote: I haven't managed to trigger a segfault yet but it sure looks like I could... You can also trigger random errors. If the array is small, Python's memory mamager might keep the memory in the heap for reuse by

Re: [Numpy-discussion] Should ndarray be a context manager?

2014-12-10 Thread Sebastian Berg
On Mi, 2014-12-10 at 07:25 +, Sturla Molden wrote: Nathaniel Smith n...@pobox.com wrote: This should be pretty trivial to implement. AFAICT you don't need any complicated cython I have a bad habit of thinking in terms of too complicated C instead of just using NumPy.

Re: [Numpy-discussion] Should ndarray be a context manager?

2014-12-10 Thread Chris Barker
On Tue, Dec 9, 2014 at 11:03 PM, Sturla Molden sturla.mol...@gmail.com wrote: Nathaniel Smith n...@pobox.com wrote: @contextmanager def tmp_zeros(*args, **kwargs): arr = np.zeros(*args, **kwargs) try: yield arr finally: arr.resize((0,),

Re: [Numpy-discussion] Should ndarray be a context manager?

2014-12-10 Thread Eric Moore
The second argument is named `refcheck` rather than check_refs. Eric On Wed, Dec 10, 2014 at 2:36 PM, Chris Barker chris.bar...@noaa.gov wrote: On Tue, Dec 9, 2014 at 11:03 PM, Sturla Molden sturla.mol...@gmail.com wrote: Nathaniel Smith n...@pobox.com wrote: @contextmanager def

Re: [Numpy-discussion] Should ndarray be a context manager?

2014-12-10 Thread Andrea Gavana
On 10 December 2014 at 20:36, Chris Barker chris.bar...@noaa.gov wrote: On Tue, Dec 9, 2014 at 11:03 PM, Sturla Molden sturla.mol...@gmail.com wrote: Nathaniel Smith n...@pobox.com wrote: @contextmanager def tmp_zeros(*args, **kwargs): arr = np.zeros(*args, **kwargs) try:

Re: [Numpy-discussion] Should ndarray be a context manager?

2014-12-10 Thread Chris Barker
On Wed, Dec 10, 2014 at 11:44 AM, Andrea Gavana andrea.gav...@gmail.com wrote: The argument is not check_refs, but refcheck. thanks -- yup, that works. Useful -- but dangerous! I haven't managed to trigger a segfault yet but it sure looks like I could... -CHB -- Christopher Barker,

Re: [Numpy-discussion] Should ndarray be a context manager?

2014-12-10 Thread Nathaniel Smith
On Wed, Dec 10, 2014 at 9:03 PM, Chris Barker chris.bar...@noaa.gov wrote: On Wed, Dec 10, 2014 at 11:44 AM, Andrea Gavana andrea.gav...@gmail.com wrote: The argument is not check_refs, but refcheck. thanks -- yup, that works. Useful -- but dangerous! I haven't managed to trigger a

Re: [Numpy-discussion] Should ndarray be a context manager?

2014-12-10 Thread Sturla Molden
Chris Barker chris.bar...@noaa.gov wrote: I haven't managed to trigger a segfault yet but it sure looks like I could... You can also trigger random errors. If the array is small, Python's memory mamager might keep the memory in the heap for reuse by PyMem_Malloc. And then you can actually

[Numpy-discussion] Should ndarray be a context manager?

2014-12-09 Thread Sturla Molden
I wonder if ndarray should be a context manager so we can write something like this: with np.zeros(n) as x: [...] The difference should be that __exit__ should free the memory in x (if owned by x) and make x a zero size array. Unlike the current ndarray, which does not have an

Re: [Numpy-discussion] Should ndarray be a context manager?

2014-12-09 Thread Eelco Hoogendoorn
My impression is that this level of optimization does and should not fall within the scope of numpy.. -Original Message- From: Sturla Molden sturla.mol...@gmail.com Sent: ‎9-‎12-‎2014 16:02 To: numpy-discussion@scipy.org numpy-discussion@scipy.org Subject: [Numpy-discussion] Should

Re: [Numpy-discussion] Should ndarray be a context manager?

2014-12-09 Thread Julian Taylor
I don't think that makes much sense, context managers are useful for managing the lifetime of objects owning resources not already managed by the garbage collector. E.g. file descriptors, a gc has no clue that a piece of memory contains a descriptor and thus never has a reason to release it in

Re: [Numpy-discussion] Should ndarray be a context manager?

2014-12-09 Thread Sturla Molden
On 09/12/14 18:39, Julian Taylor wrote: A context manager will also not help you with reference cycles. If will because __exit__ is always executed. Even if the PyArrayObject struct lingers, the data buffer will be released. Sturla ___

Re: [Numpy-discussion] Should ndarray be a context manager?

2014-12-09 Thread Robert Kern
On Tue, Dec 9, 2014 at 5:57 PM, Julian Taylor jtaylor.deb...@googlemail.com wrote: On 09.12.2014 18:55, Sturla Molden wrote: On 09/12/14 18:39, Julian Taylor wrote: A context manager will also not help you with reference cycles. If will because __exit__ is always executed. Even if the

Re: [Numpy-discussion] Should ndarray be a context manager?

2014-12-09 Thread Chris Barker
On Tue, Dec 9, 2014 at 7:01 AM, Sturla Molden sturla.mol...@gmail.com wrote: I wonder if ndarray should be a context manager so we can write something like this: with np.zeros(n) as x: [...] The difference should be that __exit__ should free the memory in x (if owned by x)

Re: [Numpy-discussion] Should ndarray be a context manager?

2014-12-09 Thread Robert Kern
On Tue, Dec 9, 2014 at 8:15 PM, Chris Barker chris.bar...@noaa.gov wrote: (although I'm still confused as to why it's so important (in cPython) to have a file context manager..) Because you want the file to close when the exception is raised and not at some indeterminate point thereafter when

Re: [Numpy-discussion] Should ndarray be a context manager?

2014-12-09 Thread Sturla Molden
Chris Barker chris.bar...@noaa.gov wrote: my first thought iust that you can just do: x = np.zeros(n) [... your code here ] del x x's ref count will go down, and it will be deleted if there are no other references to it. 1. This depends on reference counting. PyPy supports numpy too

Re: [Numpy-discussion] Should ndarray be a context manager?

2014-12-09 Thread Nathaniel Smith
On 9 Dec 2014 15:03, Sturla Molden sturla.mol...@gmail.com wrote: I wonder if ndarray should be a context manager so we can write something like this: with np.zeros(n) as x: [...] The difference should be that __exit__ should free the memory in x (if owned by x) and make x a

Re: [Numpy-discussion] Should ndarray be a context manager?

2014-12-09 Thread Sturla Molden
Nathaniel Smith n...@pobox.com wrote: @contextmanager def tmp_zeros(*args, **kwargs): arr = np.zeros(*args, **kwargs) try: yield arr finally: arr.resize((0,), check_refs=False) That one is interesting. I have actually never used ndarray.resize(). It did not

Re: [Numpy-discussion] Should ndarray be a context manager?

2014-12-09 Thread Sturla Molden
Nathaniel Smith n...@pobox.com wrote: This should be pretty trivial to implement. AFAICT you don't need any complicated cython I have a bad habit of thinking in terms of too complicated C instead of just using NumPy. @contextmanager def tmp_zeros(*args, **kwargs): arr =